[FIR] FirSymbolNamesProvider: Implement classifier package name sets

- Previously, only callable package name sets were implemented, because the compiler cannot economically compute classifier package sets for libraries. This has not changed. However, the K2 IntelliJ plugin and standalone Analysis API can very easily compute classifier package sets. Hence, this commit adds support to `FirSymbolNamesProvider` for such sets. - Similar to callable package sets, classifier package sets (1) improve the memory usage of symbol names providers and (2) improve the performance of `mayHaveTopLevelClassifier`, which is a significant bottleneck in the IDE. - In many cases, the package sets for callables and classifiers are the same. For example, the IDE Kotlin declaration provider computes the set of packages that contain any classifier and/or callable, for the following reasons: (1) indexing package names without filtering for declarations is much faster, (2) computing separate sets is not free both in time and memory, and (3) the performance impact of having a more narrow set for callables is expected to be negligible. For this reason, `FirSymbolNamesProvider.getPackageNames` exists to provide a shared package set. - The `hasSpecific*PackageNamesComputation` properties are required to avoid caching the same package set in cached symbol names providers twice. Because these properties are constant, they can be checked very quickly, and no time has to be wasted trying a specific package set computation to find out whether it's supported. ### IDE Performance Results Package set construction performance improved in the IDE in multiple benchmarks. This improves the performance of symbol providers overall, which has a direct impact on completion, code analysis, and Find Usages. In a local manual run of the `intellij_commit/setUp` Find Usages performance test, the total time spent in `getClassLikeSymbolByClassId` improved from ~18.7s to ~11.2s. Due to parallel resolve, this does not translate to a wall clock improvement of 7 seconds, but rather of a few seconds. Some performance tests improved markedly in warmup, with for example `toolbox_enterprise/genUuid` Find Usages having an improvement in `StubBasedFirDeserializedSymbolProvider.getClassLikeSymbolByClassId` from 2.4s to 0.2s. This has a direct impact on the first-run performance of the tested Find Usages command. So far, classifier package sets in the IDE are only implemented for libraries, and library sessions are cached after the first warmup. Because the biggest impact of classifier package sets is avoiding computation of "class names in package" sets, the impact of this optimization is not accurately reflected in the timings reported by our performance tests. `toolbox_enterprise/genUuid` above is a good example, as the warmup timings are great, but after warmup, `StubBasedFirDeserializedSymbolProvider.getClassLikeSymbolByClassId` improved only from 150ms to 70ms. `toolbox_enterprise/genUuid` is another example, as we can confidently say that the first-run Find Usages performance has improved (which makes a difference for the user), but it is unclear by how much, as warmup is not measured in performance tests. The same optimization for source sessions will be easier to measure, as source sessions are invalidated after each performance test run. This commit lays the groundwork for that as well, because source session support only requires the requisite package set computation in the IDE declaration provider to be implemented. ^KT-62553 fixed
2023-10-23 17:41:56 +02:00
parent 6d1ca3d379
commit 6474ff88fa
21 changed files with 333 additions and 44 deletions
@@ -46,11 +46,41 @@ public abstract class KotlinDeclarationProvider : KotlinComposableProvider {
    public abstract fun findFilesForScript(scriptFqName: FqName): Collection<KtScript>

    /**
-     * Calculates the set of package names which can be provided by this declaration provider and contain callables.
+     * Calculates the set of package names which can be provided by this declaration provider.
+     *
+     * The set may contain false positives. `null` may be returned if the package set is too expensive or impossible to compute.
+     *
+     * [computePackageNames] is used as the default implementation for [computePackageNamesWithTopLevelClassifiers] and
+     * [computePackageNamesWithTopLevelCallables] if either returns `null`. It depends on the declaration provider whether it's worth
+     * computing separate package sets for classifiers and callables, or just one set containing all package names.
+     */
+    public open fun computePackageNames(): Set<String>? = null
+
+    /**
+     * Whether the declaration provider has a specific implementation of [computePackageNamesWithTopLevelClassifiers]. This allows the
+     * Analysis API backend to determine whether classifier package sets are computed and cached separately or with [computePackageNames].
+     */
+    public abstract val hasSpecificClassifierPackageNamesComputation: Boolean
+
+    /**
+     * Calculates the set of package names which contain classifiers and can be provided by this declaration provider.
     *
     * The set may contain false positives. `null` may be returned if the package set is too expensive or impossible to compute.
     */
-    public abstract fun computePackageSetWithTopLevelCallableDeclarations(): Set<String>?
+    public open fun computePackageNamesWithTopLevelClassifiers(): Set<String>? = computePackageNames()
+
+    /**
+     * Whether the declaration provider has a specific implementation of [computePackageNamesWithTopLevelCallables]. This allows the
+     * Analysis API backend to determine whether callable package sets are computed and cached separately or with [computePackageNames].
+     */
+    public abstract val hasSpecificCallablePackageNamesComputation: Boolean
+
+    /**
+     * Calculates the set of package names which contain callables and can be provided by this declaration provider.
+     *
+     * The set may contain false positives. `null` may be returned if the package set is too expensive or impossible to compute.
+     */
+    public open fun computePackageNamesWithTopLevelCallables(): Set<String>? = computePackageNames()
 }

 public abstract class KotlinDeclarationProviderFactory {
@@ -40,6 +40,7 @@ import org.jetbrains.kotlin.psi.stubs.KotlinClassOrObjectStub
 import org.jetbrains.kotlin.psi.stubs.elements.KtStubElementTypes
 import org.jetbrains.kotlin.psi.stubs.impl.*
 import org.jetbrains.kotlin.serialization.deserialization.builtins.BuiltInSerializerProtocol
+import org.jetbrains.kotlin.utils.ifEmpty
 import java.util.concurrent.ConcurrentHashMap

 public class KotlinStaticDeclarationProvider internal constructor(
@@ -100,10 +101,23 @@ public class KotlinStaticDeclarationProvider internal constructor(
        return index.scriptMap[scriptFqName].orEmpty().filter { it.containingKtFile.virtualFile in scope }
    }

-    override fun computePackageSetWithTopLevelCallableDeclarations(): Set<String> {
-        val packageNames = index.topLevelPropertyMap.keys + index.topLevelFunctionMap.keys
-        return packageNames.mapTo(mutableSetOf()) { it.asString() }
-    }
+    override val hasSpecificClassifierPackageNamesComputation: Boolean get() = true
+
+    override fun computePackageNamesWithTopLevelClassifiers(): Set<String> =
+        buildPackageNamesSetFrom(index.classMap.keys, index.typeAliasMap.keys)
+
+    override val hasSpecificCallablePackageNamesComputation: Boolean get() = true
+
+    override fun computePackageNamesWithTopLevelCallables(): Set<String> =
+        buildPackageNamesSetFrom(index.topLevelPropertyMap.keys, index.topLevelFunctionMap.keys)
+
+    @Suppress("NOTHING_TO_INLINE")
+    private inline fun buildPackageNamesSetFrom(vararg fqNameSets: Set<FqName>): Set<String> =
+        buildSet {
+            for (fqNameSet in fqNameSets) {
+                fqNameSet.mapTo(this, FqName::asString)
+            }
+        }

    override fun getTopLevelProperties(callableId: CallableId): Collection<KtProperty> =
        index.topLevelPropertyMap[callableId.packageName]
@@ -66,8 +66,22 @@ public class CompositeKotlinDeclarationProvider private constructor(
        return providers.flatMapTo(mutableListOf()) { it.findFilesForScript(scriptFqName) }
    }

-    override fun computePackageSetWithTopLevelCallableDeclarations(): Set<String>? {
-        return providers.flatMapToNullableSet { it.computePackageSetWithTopLevelCallableDeclarations() }
+    override fun computePackageNames(): Set<String>? {
+        return providers.flatMapToNullableSet { it.computePackageNames() }
+    }
+
+    override val hasSpecificClassifierPackageNamesComputation: Boolean
+        get() = providers.any { it.hasSpecificClassifierPackageNamesComputation }
+
+    override fun computePackageNamesWithTopLevelClassifiers(): Set<String>? {
+        return providers.flatMapToNullableSet { it.computePackageNamesWithTopLevelClassifiers() }
+    }
+
+    override val hasSpecificCallablePackageNamesComputation: Boolean
+        get() = providers.any { it.hasSpecificCallablePackageNamesComputation }
+
+    override fun computePackageNamesWithTopLevelCallables(): Set<String>? {
+        return providers.flatMapToNullableSet { it.computePackageNamesWithTopLevelCallables() }
    }

    public companion object {
@@ -25,5 +25,8 @@ public object EmptyKotlinDeclarationProvider : KotlinDeclarationProvider() {
    override fun findFilesForFacade(facadeFqName: FqName): List<KtFile> = emptyList()
    override fun findInternalFilesForFacade(facadeFqName: FqName): List<KtFile> = emptyList()
    override fun findFilesForScript(scriptFqName: FqName): List<KtScript> = emptyList()
-    override fun computePackageSetWithTopLevelCallableDeclarations(): Set<String> = emptySet()
-}
+
+    override fun computePackageNames(): Set<String> = emptySet()
+    override val hasSpecificClassifierPackageNamesComputation: Boolean get() = false
+    override val hasSpecificCallablePackageNamesComputation: Boolean get() = false
+}
@@ -129,9 +129,11 @@ public class FileBasedKotlinDeclarationProvider(public val kotlinFile: KtFile) :
    }

    override fun findInternalFilesForFacade(facadeFqName: FqName): Collection<KtFile> = emptyList()
-    override fun computePackageSetWithTopLevelCallableDeclarations(): Set<String> {
-        return setOf(kotlinFile.packageFqName.asString())
-    }
+
+    override fun computePackageNames(): Set<String> = setOf(kotlinFile.packageFqName.asString())
+
+    override val hasSpecificClassifierPackageNamesComputation: Boolean get() = false
+    override val hasSpecificCallablePackageNamesComputation: Boolean get() = false

    override fun findFilesForScript(scriptFqName: FqName): Collection<KtScript> =
        listOfNotNull(kotlinFile.script?.takeIf { it.fqName == scriptFqName })