Commit Graph

552 Commits

Author SHA1 Message Date
Ilya Goncharov 08e3cb300a [JS] Migrate tests onto IR compiler with outputDir API instead of outputFile
^KT-61117 fixed
2023-11-29 11:21:56 +00:00
Aleksei.Cherepanov c002af6365 Make plugin classpath serialization path agnostic
This is needed for correct usage of portable caches

^KT-63799 Fixed
2023-11-25 10:46:49 +00:00
Hung Nguyen 5562c95155 [IC] Refactor IC maps to reuse code and ensure consistency - Part 2
In commit 4e89dcf, we have prepared the API for IC maps in top
interfaces and provide the implementation in abstract classes.

In this commit, we refactor IC maps so that they directly inherit/reuse
the implementation from the superclasses without having to reimplement
the APIs for a map.

Test: Existing tests (refactoring change)
^KT-63456: In progress

Authored-by: Hung Nguyen <hungnv@google.com>

Merge-request: KOTLIN-MR-801
Merged-by: Evgenii Mazhukin <evgenii.mazhukin@jetbrains.com>
2023-11-15 12:22:40 +00:00
Alexander.Likhachev b460c06907 [BT] Add support for in-process incremental compilation
#KT-61865 Fixed
2023-10-23 13:04:17 +00:00
Dmitriy Novozhilov 704e2ef5c5 Suppress K2 specific warnings in the codebase
^KT-62472
2023-10-18 07:59:27 +00:00
Hung Nguyen ecbf69504a [IC] Add back flush() API to PersistentStorage
In commit 4e89dcf, we removed `flush()` from `PersistentStorage`
(previously called `LazyStorage`) because it is not used in Gradle
builds.

However, `flush()` is still used in JPS builds, so we need to add that
API back in this commit.

#KT-62486 Fixed

Co-authored-by: Hung Nguyen <hungnv@google.com>


Merge-request: KOTLIN-MR-796
Merged-by: Aleksei Cherepanov <aleksei.cherepanov@jetbrains.com>
2023-10-16 18:30:47 +00:00
Hung Nguyen 4e89dcf031 [IC] Refactor IC maps to reuse code and ensure consistency
The key changes include:
  - Top interface: `PersistentStorage`
  - Implementation:
      + `LazyStorage`
      + `InMemoryStorage`
  - Extended functionality:
      + `BasicMap`
      + `PersistentStorageAdapter`

 The remaining changes are small cleanups to adapt to the above key
 changes.

 Test: Existing tests (refactoring change)
 Bug: N/A (code cleanup)
2023-10-05 11:12:25 +00:00
Aleksei.Cherepanov a32c65e4e1 Convert absolute to relative pluginsClasspath..
..to avoid recompilation on build with portable caches

#KT-62256 Fixed


Merge-request: KT-MR-12377
Merged-by: Aleksei Cherepanov <aleksei.cherepanov@jetbrains.com>
2023-10-05 10:25:53 +00:00
Mikhail Glukhikh 156097fe17 Restore KotlinCompilerVersion.IS_PRE_RELEASE
This commit is a slightly modified revert of 4f29c113.
IS_PRE_RELEASE allows to make LATEST_STABLE version behave as
experimental when this flag is set to true.
The general goal is to prepare fix of KT-62058; after this commit
one can do it by changing IS_PRE_RELEASE flag to true.
The fix of KT-62058 is planned to be done during bootstrapping.
This preparation and the future fix are parts of umbrella KT-61951.
2023-09-21 20:09:25 +00:00
Hung Nguyen 0d04b8783c [IC] Handle custom source set located outside project directory
To support build cache relocatability, we need to convert absolute paths
into relative paths before storing them in IC caches.

Before this commit, we computed relative paths based on the (root)
project directory and FAIL if some source files are located outside the
project directory.

With this commit, we will NOT FAIL in that case. This means relative
paths may start with "../". It's not "clean", but it can work.

Test: New test in BuildCacheRelocationIT
^KT-61852 Fixed
2023-09-21 14:39:52 +00:00
Dmitriy Novozhilov 9e5ee3afa0 [FE] Add isLocal name to ClassId constructor calls where it needed 2023-09-21 12:40:44 +00:00
Alexander.Likhachev 6eaccc997f [Build] Fix the typo junit jupyter -> jupiter 2023-09-06 22:47:34 +00:00
Alexander.Likhachev 6f96be0b76 [Build] Get rid of the testApiJUnit5 method
#KTI-1349 In Progress
2023-09-06 22:47:34 +00:00
Alexander.Likhachev 53fde520d5 [Build] Add jUnit dependencies in testApiJUnit5 to the implementation configuration
Adding these dependencies to the `api` configuration pollutes classpath for each dependant modules even if it doesn't need them. Instead, the dependencies should be declared more granularly if they're required
#KTI-1349 In Progress
2023-09-06 22:47:34 +00:00
Alexander.Likhachev 0ecbb64c64 [Build] Remove testApiJUnit5 parameters
Instead of declaring additional dependencies inside the method, declare them directly
#KTI-1349 In Progress
2023-09-06 22:47:34 +00:00
Alexander.Likhachev ef0fb557a5 [Build] Split junit-jupiter into api and engine
#KTI-1349 In Progress
2023-09-06 22:47:33 +00:00
Alexander.Likhachev 357d12fc8e [Build] Move JUnit dependencies into the version catalog
The `kotlin-test` dependencies are left untouched as changing them affects publications, thus these versions are independent from the used inside our build
#KTI-1349 In Progress
2023-09-06 22:47:33 +00:00
Aleksei.Cherepanov 68af705664 Add import tracker
This tracker reports import directives to IJ's JPS to track changes and detectwehen it is needed to adjust compilation scope.

#KT-44835 Fixed
2023-08-30 12:42:39 +00:00
Alexander.Likhachev 51a43a02f7 [IC] Restore old IncrementalCompilationContext constructor for KSP compatibility 2023-08-18 14:20:57 +00:00
Hung Nguyen 894ba9ab80 [IC] Relocatable IC caches for projects with custom buildDir
IC caches often contain file paths. To make them relocatable, we need
to convert these file paths into relative paths, relative to a base
directory.
  - If the file paths are source files, we can use the root project
    directory as base.
  - If the file paths are class files, we should use the classes
    directory as base (before this commit, we used the root project
    directory in both cases, that's why we hit KT-58547).

The key changes in this commit include:
  - RelocatableFileToPathConverter: converts paths to relative paths
  - IncrementalCompilationContext: contains 2 different path converters,
    one for source files and one for class files
  - SourceToOutputFilesMap: maps source files to class files using the
    above path converters
  - IncrementalCompilerRunner: creates the path converters based on file
    locations

Test: RelocatableFileToPathConverterTest unit test
      SourceToOutputFilesMapTest unit test
      BuildCacheRelocationIT.testCustomBuildDirectory integration test
^KT-58547 Fixed
2023-08-16 13:40:19 +00:00
Artem Kobzar a29fa428b3 [K/JS] Add warning for ES-modules on the klibgen stage on the uniqueness of the exported names from the module 2023-08-08 15:45:02 +00:00
Alexander.Likhachev b3677fa0ca [IC] Allow to not specify the modules info 2023-08-01 16:24:37 +00:00
Nataliya.Valtman ed2dd4b2ae Split Gradle and JPS metrics
#KT-58026 In progress
2023-07-27 11:18:54 +00:00
Troels Bjerre Lund 111bb461a9 CLI: Change kotlin reflection to java reflection
The command line argument parser is using between 0.25s and 0.5s
(depending on platform) on finding annotated properties. This fix
replaces the slow kotlin reflection with java reflection, which is an
order of magnitude faster.

 #KT-58183 Fixed
2023-06-27 08:34:40 +00:00
Hung Nguyen 078356164b IC: Fix regression in detecting constants in KotlinClassInfo.kt
A constant is a static final field with non-null value. In a previous
commit (0b09be7), we accidentally removed the *non-null value*
filter when looking for constants in the bytecode.

This commit re-adds that filter to make sure the detection is correct.

Test: Added KotlinOnlyClasspathChangesComputerTest.testDelegatedProperties

^KT-58986: Fixed
2023-06-16 18:40:01 +00:00
Hung Nguyen ff612f15cf IC: Small cleanup in KotlinClassInfo.kt
This is to address the not-yet-resolved comments in
https://github.com/JetBrains/kotlin/pull/5127, plus a few other small
(non-functional) changes.

^KT-58986: In progress
2023-06-16 18:40:00 +00:00
Aleksei.Cherepanov e11f7daae4 Disable JvmTargetDefaultSubstitutor
Disable JvmTargetDefaultSubstitutor as far as minimal supported jvmTarget version is 1.8

#KTIJ-25445 Fixed
2023-05-17 21:47:21 +00:00
Hung Nguyen 0b09be73c6 New IC: Detect changes to class annotations
Both the new and old incremental compilation (IC) analysis rely on
Kotlin class metadata to detect a change.

However, Kotlin metadata currently doesn't contain info about
annotations (KT-57919), so the IC will not be able to detect a change
to them.

With this commit, we'll fix the new IC such that it can detect a change
to class annotations by not relying only on metadata.

We currently scope this fix to the new IC (cross-module analysis) first.
We'll fix this issue for within-module analysis later.

Performance: There seems to be no performance impact from this change.
Snapshotting the 400MB ideaIC-2022.1.4/app.jar takes 4.1s before and
after this change.

Test: Added ClasspathChangesComputerTest.testChangedAnnotations
^KT-58289: Fixed
2023-05-11 15:24:05 +00:00
Nataliya.Valtman 2a391f7330 Move kotlin-build-statistic project to :compiler 2023-04-25 11:29:09 +00:00
nataliya.valtman e34dd043da Add minimal statistic report for JPS build
Fix build stat for gradle 8

#KT-56438 Fixed
2023-04-25 11:29:09 +00:00
Aleksei.Cherepanov 2e453051f9 Add cache comparator for incremental gradle tests
Add logic that rebuilds all sources after the last incremental round of each Gradle IC test and compares caches. The same was already implemented for JPS, but not for Gradle. After all rounds of incremental compilation are completed, another clean build from scratch is produced. All caches after the rebuild are compared with the caches of the last round of incremental compilation. This check is necessary because incremental compilation artifacts should depend on the state of the project, source files, and configuration, and not the chain of changes and incremental builds that led to this state. After the launch, there were several tests that did not satisfy the above conditions, and were muted  (KT-56681, KT-55195, KT-56242, KT-56698)

#KT-54991 In Progress
2023-04-24 14:54:45 +00:00
Yahor Berdnikau eb4e96a113 Add kotlin-build-tools-enum-compat workaround
It acts as a workaround for the case when build tools or dependencies
are compiled with latest 'kotlin-stdlib' version, but at a runtime older
 'kotlin-stdlib' is provided, which does not know about new
 `EnumEntries`.

 ^KT-57317 Fixed
2023-04-21 14:57:17 +00:00
Roman Golyshev d4cffb8a5a [213] Switch to 213 platform
KTI-1114
2023-04-21 13:19:04 +00:00
Sebastian Sellmair b6ca62f330 [CLI][Minor] toArgumentStrings: Use List Builder (like) style
KTIJ-24976
2023-04-06 16:03:04 +00:00
Sebastian Sellmair 082a38216d [CLI] argumentsToStrings: Add 'compactArgumentValues' option
Passing 'false' will pass Array or List based arguments multiple times
instead of using the Delimiter.

eg:
compactArgumentValues = true:
"-cp", "library1;library2;library3"

compactArgumentValues = false:
"-cp", "library1", "-cp", -"library2", "-cp", "library3"

Using compactArgumentValues = false can be beneficial
when the many compiler arguments are held in memory.
In this case the raw arguments can intern individual string values, which
is highly effective when the arguments refer to files on disk.

KTIJ-24976
2023-04-06 16:03:04 +00:00
Sebastian Sellmair 7f91e94e7a [CLI] Arguments: Implement raw vs resolved delimiters
KTIJ-24976
2023-04-06 16:03:03 +00:00
Sebastian Sellmair d07b1b6502 [CLI] Implement CommonToolArguments.toStringList (to replace convertArgumentsToStringList`)
KTIJ-24976
2023-04-06 16:03:02 +00:00
Ivan Kylchik 5d5582d201 Move ignoreConstOptimizationErrors compiler key from jvm to common
#KT-56023
2023-03-24 15:55:03 +00:00
hungvietnguyen 4f3244fb78 IC: Compute symbols impacted by classpath changes (#5111)
IC: Compute symbols impacted by classpath changes

Incremental compilation has 4 key steps:
  1. Compile changed/impacted files
  2. Detect symbols that have changed after compiling
  3. Detect symbols that are impacted by the changed symbols
  4. Based on the changed-or-impacted symbols, identify files that need
     to be recompiled. Go back to step 1.

Normally, step 2 and 3 are done together when the changed symbols
and impacted symbols are in the same module.

However, if the changed symbols and impacted symbols are in different
modules (e.g., a `Subclass` in lib1 extends a `Superclass` in lib2),
we currently do not compute symbols in the current module that are
impacted by changes in another module (step 3 above).

This is the case for both the new IC and the old IC.

In this commit, we will compute impacted symbols for the new IC. We can
fix the old IC later if necessary (they can't be fixed together easily).

Test: Added BaseIncrementalCompilationMultiProjectIT.testChangeInterfaceInLib
^KT-56197 Fixed
2023-03-21 13:12:15 +01:00
nataliya.valtman 79fff92dc1 Don't use aggregated GC metric for build reports 2023-03-14 20:58:45 +00:00
nataliya.valtman a34657195b Add GC metric to build statistic 2023-03-14 20:58:45 +00:00
nataliya.valtman c16dabba1f Add Kotlin daemon memory usage into build report
#KT-56211: Fixed
2023-03-14 20:58:45 +00:00
Alexander.Likhachev 8bb0c5135b [IC] Add unit tests for InMemoryStorageWrapper
#KT-56052 Fixed
2023-03-07 16:19:25 +00:00
Alexander.Likhachev 4aedf77431 [IC] Avoid many collection allocations in InMemoryStorageWrapper
#KT-56052 In Progress
2023-03-07 16:19:25 +00:00
Alexander.Likhachev cc16ca2bf8 [IC] Add synchronization to all the DefaultInMemoryStorageWrapper public methods
#KT-56052 In Progress
2023-03-07 16:19:25 +00:00
Alexander.Likhachev 99830d7f28 [IC] Segregate LazyStorage into LazyStorage and AppendableLazyStorage
#KT-56052 In Progress
2023-03-07 16:19:24 +00:00
Alexander.Likhachev ca8d0bd100 [IC] Add unit tests for resetting in-memory caches wrappers in a transaction
#KT-56052 In Progress
2023-03-07 16:19:23 +00:00
Alexander.Likhachev 04a5c7761c [IC] Add unit tests for CachesManager closing in transaction
#KT-56052 In Progress
2023-03-07 16:19:23 +00:00
Alexander.Likhachev 2bff38dd27 [IC] Fix transaction wasn't marked as unsuccessful on execution exception
#KT-56052 In Progress
2023-03-07 16:19:22 +00:00
Alexander.Likhachev 8825cf7bf6 [IC] Fix RecoverableCompilationTransaction wasn't throwing an exception properly
#KT-56052 In Progress
2023-03-07 16:19:22 +00:00