1. CLASS_LEVEL: allows tracking whether a .class file has changed
without tracking what specific parts of the .class file (e.g.,
fields or methods) have changed.
2. CLASS_MEMBER_LEVEL: allows tracking not only whether a .class file
has changed but also what specific parts of the .class file (e.g.,
fields or methods) have changed.
The idea is that for better performance we will use CLASS_LEVEL for
classpath entries that are usually unchanged, and CLASS_MEMBER_LEVEL
for classpath entries that are frequently changed. We'll work out the
specifics in a following commit after some measurements.
Support running kotlinc on Windows in ClasspathSnapshotTestCommon
Also add tests for different Kotlin class kinds.
Add unit tests for CLASS_LEVEL snapshotting and diffing
Test: Updated ClasspathSnapshotterTest + ClasspathChangesComputerTest
Add ClasspathChangesComputerTest.testMixedClassSnapshotGranularities
Make sure that:
- Compilation with -Pkotlin.incremental.useClasspathSnapshot=true or
-Dkotlin.incremental.classpath.snapshot.enabled=true is incremental
after cache hit.
- Default compilation (using build history files) is non-incremental
after cache hit.
Also unify the related tests into
BuildCacheRelocationIT.testKotlinIncrementalCompilation*. We don't need
corresponding tests in BuildCacheIT because BuildCacheRelocationIT
already covers it.
- Clean up BuildCacheRelocationIT.testKotlinIncrementalCompilation
- Delete BuildCacheIT.testKotlinCompileIncrementalBuildWithoutRelocation
as it is already covered by the test in BuildCacheRelocationIT
- Add BuildCacheRelocationIT.testKotlinIncrementalCompilation_withClasspathSnapshot
- Updated BuildCacheRelocationIT.testKotlinIncrementalCompilation_withClasspathSnapshot
If LeverVersion is set to greater than 1 of the current compiler version, metadata became null here org.jetbrains.kotlin.load.kotlin.header.ReadKotlinClassHeaderAnnotationVisitor.createHeader, which leads to NPE during incremental analysis. Using the same incompatible data as fallback prevent it
#KTIJ-20954 Fixed
Merge-request: KT-MR-5680
Merged-by: Aleksei Cherepanov <aleksei.cherepanov@jetbrains.com>
to ensure precision (otherwise, rounding errors to milliseconds may
add up and cause unexplainable gaps in the running time).
We can still use milliseconds in the final report after all the precise
sub-build-times have been aggregated.
Previously IC state was stored in System properties. As result parallel
compilation might cause incorrect state of IC, what led to corruption
of kotlin_module files. Now IC state is stored via CompilerArguments
and CompilerConfiguration
#KT-46038 Fixed
to make it easier to add more tests in the next commits.
- Add unit tests for constants and inline functions
Also add tests for different kinds of Kotlin classes: CLASS,
FILE_FACADE, MULTIFILE_CLASS.
-Add unit test for nested classes
Also remove the existing integration test for nested classes to keep the
integration tests focused on the key scenarios while unit tests will
cover the corner cases.
Ignore inline functions that are private
Currently, we shrink classpath snapshots at 2 steps:
- Classpath diffing: Shrink the current classpath snapshot against
the previous lookup symbols
- Classpath snapshot saving: Shrink the current classpath snapshot
against the current lookup symbols
With this commit, the shrinking at the second step is now incremental.
The shrinking at the first step is still non-incremental.
for performance reasons: (1) the snapshots are too big, and (2) they are
usually updated only at the end of the task execution--in a failed task
run, they are usually unchanged and therefore don't need to be restored.
Also visit a class file with ASM once to extract all information we need
in advance, instead of visiting the class file each time some piece of
info is needed.
as we need access to the lookup tracker to compute classpath changes
more efficiently and reduce the size of the saved classpath snapshot.
The previous commit only changed the files' paths, this
commit actually updates the files' contents.
Note that classpath snapshotting still happens in Gradle artifact
transforms. (However, the previous commit also moved the code for
classpath snapshotting together with the code for classpath diffing as
they are closely related.)
This is faster than the current approach which creates
`JavaClassDescriptor`s, converts them to protos, and then snapshots
these protos.
- Refactor unit tests to faciliate further changes
- moves test data to a directory that matches the tests' package name
- moves expected snapshots to a separate directory
- adds public and private fields/properties to sample class
- Compute changes between ASM-based Java class snapshots
- Don't collect members of an added Java class as changes
as it's enough to report the name of the added Java class as changed (we
also do that for added Kotlin classes and Kotlin/Java removed classes).
- Add unit tests for impact analysis in advance
- Compute impacted symbols of changed symbols
Also do not collect added classes/class members as they don't impact
recompilation.
-Use ClassId when computing Java class changes
It is more precise than JvmClassName, which can be ambiguous around the
`$` character (e.g., ClassId "com/example/A$B.C" and "com/example/A.B$C"
both have the same JvmClassName "com/example/A$B$C").
- Compute impacted set of changed symbols across Kotlin and Java
- Add unit tests for impact analysis across Kotlin and Java
- Compute supertypes of Kotlin classes during snapshotting
- Handle inner classes when computing list of changed symbols.
For the reported symbols, always check all options:
class member, inner class, top level class, top level member.
Test: IncrementalJavaChangeClasspathSnapshotIT.testAddingInnerClass
to avoid unnecessarily loading unchanged ones.
Duplicate classes will make this a bit tricky. This commit outlines the
algorithm to handle them, the full implementation will follow later.
Also handle removed classes when computing classpath changes.
Test: New tests in ClasspathChangesComputerTest
Workers usually execute later than task action and does not raise
exception to the task. Such behaviour skips task outputs restoration to
the pre-execution state.
Kotlin compiler on incremental compilation error leaves 'dirty*' files
in the build output, forcing Gradle to run non-incrementally on the
next build. Fixed it by moving task outputs reset logic into worker as
well.
^KT-46406 In Progress
There are certain classes that we are not yet able to take snapshots of,
either because the class is faulty, or there is a bug in our code.
For these classes, we will use a special snapshot and fall back to the
existing approach to compute classpath changes.
In the short run, we will update this list of cases as they arise. In
the long run, we will:
- Fix all cases that are caused by bugs in our code.
- Decide what to with the faulty jars in general and remove the list.
Test: The following should pass
- Step 1: ./gradlew publish
- Step 2: ./gradlew publish -Pbootstrap.local=true
-Pbootstrap.local.path=/path/to/kotlin/build/repo
-Pkotlin.incremental.useClasspathSnapshot=true
Simplify API of InlineConstTracker
In ConstLowering: Move transformer logic to another class to avoid mutable state. Avoid marking all files in module as module. Support of inner classes.
#KT-46506 Fixed
Create `JavaClassDescriptor`s for Java classes
Ignore anonymous and synthetic classes
as they can't impact recompilation.
Clean up handling of local and anonymous classes
Bug: KT-45777
Test: New JavaClassDescriptorCreatorTest
Now, milestones of major Kotlin releases (e.g. 1.6.0-M1) will NOT
generate prerelease binaries anymore.
The reason for that is it's proven to be quite a complicated process to
turn on IR_PRE_RELEASE after the release is branched, perform double
bootstrap and fix tests, and then do it again in reverse just before
release. With the new release cadence, we don't have that much time to
do it and verify that everything works as intended.
Note that this only removes the "global" prerelease flag. Compiler will
still generate prerelease binaries if a non-stable language version is
used. For example, Kotlin 1.6.0-M1 with `-language-version 1.7` will
generate prerelease binaries.
Reuse the existing IncrementalJvmCache to take snapshots and compute
changes for Kotlin classes.
Java classes will be handled next.
Bug: KT-45777
Test: New ClasspathSnapshotterTest, ClasspathChangesComputerTest
Extract minimal required information into the new `KotlinClassInfo`
class. This info will be part of the classpath snapshot of the
`KotlinCompile` task in an upcoming change.
Bug: KT-45777
Test: Existing tests should pass (this is a refactoring-only change)
This commit wires necessary components only. The actual classpath
changes will be provided later.
Bug: KT-45777
Test: Existing IncrementalCompilationClasspathSnapshotJvmMultiProjectIT
and IncrementalJavaChangeClasspathSnapshotIT
Reduce size of lookup map after rebuild, reduce waiting time by replacing operations of read+write with append, also split remove garbage process into smaller operations in get
#KT-46804 Fixed
Hung Nguyen