Review: https://jetbrains.team/p/kt/reviews/6753
Meaningful semantic change was splitted into 5 commits to simplify the
change review. Sinle commit would be too big.
Why replace source to binary: to get rid of kotlin-reflect in Kotlin
plugin artifact KTIJ-22276
Note: Kotlin Maven artifacts (./gradlew publish) changed their
dependency on kotlin-reflect
When computing impacted symbols of changed symbols, previously we
considered only the supertypes-inheritors type of impact, which is the
most common type. This commit adds the constants-in-companion-objects
type of impact to address KT-53266.
We've also cleaned up impact computation to make it easier to add new
types of impact in the future.
^KT-53266 In progress
If incremental compilation fails, we currently fall back to
non-incremental compilation. When that happens, it would be incorrect to
shrink classpath snapshot incrementally, so this commit makes sure we'll
shrink classpath snapshot non-incrementally in that case.
^KT-53231 In progress
With new syntax each plugin should be registered in separate argument with syntax
`-Xcompiler-plugin=classpath1,classpath2[=argument1=value1,argument2=value2]`
When incremental compilation fails, we currently log it at the `debug`
level (and fall back to non-incremental compilation). This commit will
change it to `warning` so that we can get more user reports, which will
allow us to fix the root cause.
Also make sure the warning includes a stack trace.
Additionally, let ReportSeverity.fromCode() return a non-null value
or throw an exception otherwise as that case is not expected.
^KT-52839 In Progress
The current logic works as follows:
- Try either incremental compilation or non-incremental compilation
- If the above (or any of its surrounding work) fails, fall back to
non-incremental compilation
This means we may perform non-incremental compilation twice.
This commit will fix that logic so that we fall back to non-incremental
compilation only if *incremental compilation* fails.
A nice consequence of this change is that it also resolves the critical
bugs described at KT-52669 (which occur because the current logic is
flawed).
#KT-52669 Fixed
- Remove soft references that are too old.
- Compute memory usage based on Runtime.totalMemory() instead of
Runtime.maxMemory() as the latter is not reliable.
- Remove debug logs that haven't proved to be useful.
- Collect metrics on the number of cache hits (in addition to cache
misses).
#KT-52329 In Progress
[New IC] Reduce memory usage of classpath snapshot cache
Use object interning for commonly shared objects. These include:
- supertypes of classes
- package names of classes
One experiment showed that with the above optimization, memory usage was
reduced from 660 MB down to 280 MB (+ 4 MB for the interning pool).
More aggressive object interning didn't reduce memory usage much
further, but would increase interning overhead and code complexity, so
we didn't do this to more objects.
Note that this commit optimizes the size of classpath snapshots in
memory, not their serialized data on disk. (I attempted the latter,
but the size was only reduced from 160 MB down to 130 MB, while the code
complexity became much higher as multiple interning pools would need to
be stored to disk and later loaded from disk, each per classpath entry
snapshot or shrunk classpath snapshot.)
#KT-52329 Fixed
To snapshot a Java class (+ its fields and methods), previously we used
Gson to serialize a class field/method to a string via reflection, and
hash that string.
We now use an ASM ClassWriter to write a placeholder class containing
the field/method of interest and hash the bytecode of that class.
One experiment showed that this new approach is ~10 times faster than
the previous approach (140s down to 16s when snapshotting 600 jars).
Test: Updated expectation files for JavaClassSnapshotterTest unit tests
+ Existing integration tests to prevent regression
^KT-52141 In Progress
Implement an in-memory cache that uses a combination of strong
references and `SoftReference`s so that it adapts to memory
availability.
Cache eviction is currently performed after loading a classpath snapshot
(this can be changed later if necessary).
Evicted cache entries' values will be converted from strong references
into `SoftReference`s so that they can still be used for as long as the
JVM allows them.
There are 2 types of cache eviction:
- Least recently used: Oldest entries will be evicted
- Memory usage limit: If memory is limited, all entries will be
evicted
Test: Added InMemoryCacheWithEvictionTest unit test
^KT-51978 In Progress
along with source lines mapping, allows to "emulate" usage of the
PSI files which allows to extract source file and line mapping info
on every stage from source element.
It makes sense to use this mapping for the error reporting too.
to reduce the size of the snapshots.
- Track metrics for Gradle classpath snapshot artifact transform
- Format size metrics so it's more readable
^KT-45777 In Progress
Alexander Udalov