klibs provide information about the container source file
in the KlibMetadataProtoBuf extensions for functions,
properties and classes.
This information is deserialized and attached to the
`klibSourceFile` extension (stored in FirDeclarationDataRegistry)
^KT-66271 Fixed
FirConstExpression is usually confused with "constant" calculations,
while in fact, it just denotes a simple literal expression
and `1 + 1` isn't represented by a FirConstExpression.
^KT-64314 Fixed
K1 implementation of kotlinx.serialization used SerializationPluginMetadataExtensions
for two purposes:
- to extract the source-order of properties
- to identify which properties are actually serializable
In K2, the first point is covered by the compiler itself (as it correctly orders
properties during deserialization), and to cover the second point we
need to save in the propery information if it was mentioned in this extension
or not
KT-64312
- Previously, only callable package name sets were implemented, because
the compiler cannot economically compute classifier package sets for
libraries. This has not changed. However, the K2 IntelliJ plugin and
standalone Analysis API can very easily compute classifier package
sets. Hence, this commit adds support to `FirSymbolNamesProvider` for
such sets.
- Similar to callable package sets, classifier package sets (1) improve
the memory usage of symbol names providers and (2) improve the
performance of `mayHaveTopLevelClassifier`, which is a significant
bottleneck in the IDE.
- In many cases, the package sets for callables and classifiers are the
same. For example, the IDE Kotlin declaration provider computes the
set of packages that contain any classifier and/or callable, for the
following reasons: (1) indexing package names without filtering for
declarations is much faster, (2) computing separate sets is not free
both in time and memory, and (3) the performance impact of having a
more narrow set for callables is expected to be negligible. For this
reason, `FirSymbolNamesProvider.getPackageNames` exists to provide a
shared package set.
- The `hasSpecific*PackageNamesComputation` properties are required to
avoid caching the same package set in cached symbol names providers
twice. Because these properties are constant, they can be checked very
quickly, and no time has to be wasted trying a specific package set
computation to find out whether it's supported.
### IDE Performance Results
Package set construction performance improved in the IDE in multiple
benchmarks. This improves the performance of symbol providers overall,
which has a direct impact on completion, code analysis, and Find Usages.
In a local manual run of the `intellij_commit/setUp` Find Usages
performance test, the total time spent in `getClassLikeSymbolByClassId`
improved from ~18.7s to ~11.2s. Due to parallel resolve, this does not
translate to a wall clock improvement of 7 seconds, but rather of a few
seconds.
Some performance tests improved markedly in warmup, with for example
`toolbox_enterprise/genUuid` Find Usages having an improvement in
`StubBasedFirDeserializedSymbolProvider.getClassLikeSymbolByClassId`
from 2.4s to 0.2s. This has a direct impact on the first-run performance
of the tested Find Usages command.
So far, classifier package sets in the IDE are only implemented for
libraries, and library sessions are cached after the first warmup.
Because the biggest impact of classifier package sets is avoiding
computation of "class names in package" sets, the impact of this
optimization is not accurately reflected in the timings reported by our
performance tests. `toolbox_enterprise/genUuid` above is a good example,
as the warmup timings are great, but after warmup,
`StubBasedFirDeserializedSymbolProvider.getClassLikeSymbolByClassId`
improved only from 150ms to 70ms.
`toolbox_enterprise/genUuid` is another example, as we can confidently
say that the first-run Find Usages performance has improved (which makes
a difference for the user), but it is unclear by how much, as warmup is
not measured in performance tests.
The same optimization for source sessions will be easier to measure, as
source sessions are invalidated after each performance test run. This
commit lays the groundwork for that as well, because source session
support only requires the requisite package set computation in the
IDE declaration provider to be implemented.
^KT-62553 fixed
- This is more consistent with "top-level callable names in package" and
simplifies the set construction quite a bit, as we can avoid a lot of
`asString` conversions.
- Rename DeserializedClassConfigurator -> FirDeserializationExtension to
abstract the existing behavior (about the Serializable supertype), as
well as any future JVM-specific deserialization behavior.
- Rename JvmDeserializedClassConfigurator ->
FirJvmDeserializationExtension and move it to fir:java.
- In LL FIR, we have increasingly formalized symbol name caches as
palpable objects. The main reasons for this formalization were the
need to share implementations of caching between different (LL FIR)
symbol providers, the need to build composite name caches from
individual name caches, and the introduction of resolve extensions
which may provide additional declarations and thus complicate the name
set construction for Kotlin symbol providers in LL FIR.
- `LLFirSymbolProviderNameCache` also shared a lot of similarities with
cache handling in FIR providers like
`FirCachingCompositeSymbolProvider` and
`AbstractFirDeserializedSymbolProvider`.
- This commit introduces a `FirSymbolNamesProvider` as a component of
`FirSymbolProvider`. This symbol names provider's task is to provide
the sets of names which `FirSymbolProvider` previously provided. It
also allows sharing implementations of `mayHaveTopLevel*` once and for
all, which is an improvement over the previously scattered
implementations (the same ideas replicated many times throughout
different symbol providers).
- `FirSymbolNamesProvider` by design doesn't cache, as many symbol
providers may not need such a cache. `FirCachedSymbolNamesProvider`
can be used to cache symbol names if needed. The symbol name provider
architecture also makes it easier to switch between caching and
non-caching, without the need to reimplement caches every time.
- Synthetic function types complicate the picture, but this complication
is now exposed with the rest of the API, instead of being hidden in a
few implementations here and there. This allows symbol providers to
more explicitly state whether they can provide generated function
types, which is an advantage for the correctness of composite symbol
providers.
Some specific notes:
- In `FirSyntheticFunctionInterfaceProviderBase`, the class ID check has
been replaced with a full `mayHaveTopLevelClassifier` check so that
the cache doesn't get filled with `null` entries.
- `LLFirKotlinSymbolProviderNameCache` is turned into a non-caching
`LLFirKotlinSymbolNamesProvider` so that this symbol names provider
and those of resolve extensions can be composed into one caching
symbol provider in `LLFirProviderHelper` without creating layers of
caches. If the Kotlin symbol names provider was caching out of the
box, `LLFirProviderHelper.symbolNameCache` would cache the
names (1) in the combined symbol names cache and (2) in the Kotlin
symbol names cache.
- A caching Kotlin symbol names cache can still be created easily with
the `LLFirKotlinSymbolNamesProvider.cached` constructor function.
This commit adds missing pieces for the puzzle:
Annotation instantiation feature uses IrProperty's initializer to instantiate
properties from other modules that have default values which weren't
specified on call site.
To support this feature properly, Fir2IrVisitor should fill LazyIrProperty's
backing field initializer with information from Fir.
To get this information into Fir, FirMemberDeserializer should be able to read
it from KotlinJvmBinaryClass with AnnotationLoaderVisitorImpl. (klibs are unsupported for now)
There's a catch with enum entries references: we can't access session.SymbolProvider to resolve it
because we're still at the deserialization stage, and it can cause StackOverflow if enum is nested in the
same class (see RequiresOptIn.Level). To mitigate this, a new FirEnumEntryDeserializedAccessExpression is produced
instead; it is later replaced with the correct reference in the Fir2IrVisitor.
^KT-58137 Fixed
Also add test to loadJava folder with annotations default values that
verifies metadata loading
^KTIJ-24638
Notice on `DebugSymbolRenderer`:
stub based deserializer sets source directly,
but it's available in IDE mode only.
Thus, standalone and IDE tests have different results.
In order to avoid this, sources for compiled code are explicitly ignored
Notice on distinct callables:
for a file which belong to multiple libraries, decompiled code would be build per library.
In order to avoid ambiguity errors for members in that file,
we need to distinct provided elements by origins
failed test from IJ repo:
FirReferenceResolveWithCrossLibTestGenerated#testSetWithTypeParameters
Some of the incoming paths "absoluteness" may not match the
one of the corresponding library path, and that leaded to incorrect
filtering out some items in the deserialized symbol providers.
Fix the filtering to account for the mismatch.
#KT-57535 fixed
During extracting type attributes from annotations we should expand
typealiases of annotation type to handle cases when user makes a typealias
on some special annotation, like `kotlin.internal.Exact`. And to exapnd
typealias we should resolve annotation class id to symbol.
This leads to a cycle during class deserialization, if some nested annotation
is used as type annotation in some declaration in the same class
```
interface SomeInterface {
interface NestedInterface : @Ann Some
interface Some
@Target(AnnotationTarget.TYPE)
annotation class Ann
}
```
Attempt to find symbol for SomeInterface.Ann during deserialization of
SomeInterface.NestedInterface wil lead to second attempt to deserialize
class SomeInterface, which eventually leads to StackOverFlow. And at the
same time expanding typealiases for annotations from binaries has not
much sense, because types in binaries are already expanded
So to fix this issue it's enough to just not expand typealiases on type
annotations for types of deserialized declarations
^KT-57876 Fixed
Move metadata extension with property order from kotlinx.serialization to core
After fix of KT-54792 properties will be deserialized in declaration order
if corresponding class was compiled with modern compiler. But this order
is needed for kotlinx.serialization for binaries compiled with any
kotlin compiler >= 1.4. Since we don't plan to add any extension points
into (de)serialization into FIR, we need to take into account existing
metadata extension from kotlinx.serialization in compiler itself
^KT-57769 Fixed
Sebastian Sellmair