When computing impacted symbols of changed symbols, previously we
considered only the supertypes-inheritors type of impact, which is the
most common type. This commit adds the constants-in-companion-objects
type of impact to address KT-53266.
We've also cleaned up impact computation to make it easier to add new
types of impact in the future.
^KT-53266 In progress
When incremental compilation fails, we currently log it at the `debug`
level (and fall back to non-incremental compilation). This commit will
change it to `warning` so that we can get more user reports, which will
allow us to fix the root cause.
Also make sure the warning includes a stack trace.
Additionally, let ReportSeverity.fromCode() return a non-null value
or throw an exception otherwise as that case is not expected.
^KT-52839 In Progress
- Remove soft references that are too old.
- Compute memory usage based on Runtime.totalMemory() instead of
Runtime.maxMemory() as the latter is not reliable.
- Remove debug logs that haven't proved to be useful.
- Collect metrics on the number of cache hits (in addition to cache
misses).
#KT-52329 In Progress
[New IC] Reduce memory usage of classpath snapshot cache
Use object interning for commonly shared objects. These include:
- supertypes of classes
- package names of classes
One experiment showed that with the above optimization, memory usage was
reduced from 660 MB down to 280 MB (+ 4 MB for the interning pool).
More aggressive object interning didn't reduce memory usage much
further, but would increase interning overhead and code complexity, so
we didn't do this to more objects.
Note that this commit optimizes the size of classpath snapshots in
memory, not their serialized data on disk. (I attempted the latter,
but the size was only reduced from 160 MB down to 130 MB, while the code
complexity became much higher as multiple interning pools would need to
be stored to disk and later loaded from disk, each per classpath entry
snapshot or shrunk classpath snapshot.)
#KT-52329 Fixed
To snapshot a Java class (+ its fields and methods), previously we used
Gson to serialize a class field/method to a string via reflection, and
hash that string.
We now use an ASM ClassWriter to write a placeholder class containing
the field/method of interest and hash the bytecode of that class.
One experiment showed that this new approach is ~10 times faster than
the previous approach (140s down to 16s when snapshotting 600 jars).
Test: Updated expectation files for JavaClassSnapshotterTest unit tests
+ Existing integration tests to prevent regression
^KT-52141 In Progress
Implement an in-memory cache that uses a combination of strong
references and `SoftReference`s so that it adapts to memory
availability.
Cache eviction is currently performed after loading a classpath snapshot
(this can be changed later if necessary).
Evicted cache entries' values will be converted from strong references
into `SoftReference`s so that they can still be used for as long as the
JVM allows them.
There are 2 types of cache eviction:
- Least recently used: Oldest entries will be evicted
- Memory usage limit: If memory is limited, all entries will be
evicted
Test: Added InMemoryCacheWithEvictionTest unit test
^KT-51978 In Progress
1. CLASS_LEVEL: allows tracking whether a .class file has changed
without tracking what specific parts of the .class file (e.g.,
fields or methods) have changed.
2. CLASS_MEMBER_LEVEL: allows tracking not only whether a .class file
has changed but also what specific parts of the .class file (e.g.,
fields or methods) have changed.
The idea is that for better performance we will use CLASS_LEVEL for
classpath entries that are usually unchanged, and CLASS_MEMBER_LEVEL
for classpath entries that are frequently changed. We'll work out the
specifics in a following commit after some measurements.
Support running kotlinc on Windows in ClasspathSnapshotTestCommon
Also add tests for different Kotlin class kinds.
Add unit tests for CLASS_LEVEL snapshotting and diffing
Test: Updated ClasspathSnapshotterTest + ClasspathChangesComputerTest
Add ClasspathChangesComputerTest.testMixedClassSnapshotGranularities
to make it easier to add more tests in the next commits.
- Add unit tests for constants and inline functions
Also add tests for different kinds of Kotlin classes: CLASS,
FILE_FACADE, MULTIFILE_CLASS.
-Add unit test for nested classes
Also remove the existing integration test for nested classes to keep the
integration tests focused on the key scenarios while unit tests will
cover the corner cases.
Ignore inline functions that are private
Currently, we shrink classpath snapshots at 2 steps:
- Classpath diffing: Shrink the current classpath snapshot against
the previous lookup symbols
- Classpath snapshot saving: Shrink the current classpath snapshot
against the current lookup symbols
With this commit, the shrinking at the second step is now incremental.
The shrinking at the first step is still non-incremental.
If fail.txt is present in root directory of test then exception from
test will be muted. If there were no exceptions and fail.txt exists
then test fail with suggestion to remove fail.txt
Content of fail.txt does not matter, so it can be used to store
information about why this test doesn't pass
Also visit a class file with ASM once to extract all information we need
in advance, instead of visiting the class file each time some piece of
info is needed.
as we need access to the lookup tracker to compute classpath changes
more efficiently and reduce the size of the saved classpath snapshot.
The previous commit only changed the files' paths, this
commit actually updates the files' contents.
Note that classpath snapshotting still happens in Gradle artifact
transforms. (However, the previous commit also moved the code for
classpath snapshotting together with the code for classpath diffing as
they are closely related.)
Some of IDEA services (like in `com.intellij/execution`) was copied,
because they are used in tests but jars with them compiled with
jdk 11 and we run our tests on jdk 8, so their bytecode can not
be read
Add tests for KT-45191 [JPS] Marking method as "default" in Java SAM interface doesn't affect dependencies, which was fixed in intlellij(212): cbad0d91 support kotlin-generated lambda usage
#KT-45191 Fixed
Support use-cases when the build dir is outside of root project dir.
Many projects set up output dir as <root_project>/../out/<project_name>/build
to be able to clean build dir more easily. This commit adds support
to build history file detection and it tries to find build history
files for artifacts that are:
- under root project dir (a default structure)
- under root project's build dir parent (to support typical custom setup)
Fixes https://youtrack.jetbrains.com/issue/KT-40875
Test: BaseIncrementalCompilationMultiProjectIT
See the test added: there's a non-denotable T!! type inside flexible type
that wasn't handled before.
ConeKotlinType::contains handles flexible types content and some other cases
Also, it has better asymptotics
Set TargetBackend.JVM_IR for these tests by default; remove the
generated IR-based test and add a new old-backend-based test. This fixes
the issue where some (3) of these tests were not properly ignored
because of incorrect target backend used in the test generator.
Also update test data for some tests which use local functions, which
are not generated to separate anonymous classes in JVM IR.
Hung Nguyen