JPS: Refactor cache compatibility checking and build targets loading/dependency analysis.

CacheVersion class refactoring:

Responsibilities of class CacheVersion are splitted into:
	- interface CacheAttributesManager<Attrs>, that should:
	    - load actual cache attribute values from FS
	    - provide expected attribute values (that is required for current build)
	    - checks when the existed cache (with actual attributes) values is suitable for current build (expected atribute values)
	    - write new values to FS for next build
	- CacheAttributesDiff is created by calling CacheAttributesManager.loadDiff extension method. This is just pair of actual and expected cache attributes values, with reference to manager. Result of loadDiff can be saved.

CacheAttributesDiff are designed to be used as facade of attributes operations: CacheAttributesDiff.status are calculated based on actual and expected attribute values. Based on that status system may perform required actions (i.e. rebuild something, clearing caches, etc...).

Methods of CacheAttributesManager other then loadDiff should be used only through CacheAttributesDiff.
Build system should work in this order:
    - get implementation of CacheAttributesManager for particular compiler and cache
    - call loadDiff __once__ and save it result
    - perform actions based on `diff.status`
    - save new cache attribute values by calling `diff.saveExpectedIfNeeded()`

There are 2 implementation of CacheAttributesManager:
    - CacheVersionManager that simple checks cache version number.
    - CompositeLookupsCacheAttributesManager - manager for global lookups cache that may contain lookups for several compilers (jvm, js).

Gradle:

Usages of CacheVersion in gradle are kept as is. For compatibility this methods are added: CacheAttributesManager.saveIfNeeded, CacheAttributesManager.clean. This methods should not be used in new code.

JPS:

All JPS logic that was responsible for cache version checking completely rewritten.

To write proper implementation for version checking, this things also changed:
	- KotlinCompileContext introduced. This context lives between first calling build of kotlin target until build finish. As of now all kotlin targets are loaded on KotlinCompileContext initialization. This is required to collect kotlin target types used in this build (jvm/js). Also all build-wide logic are moved from KotlinBuilder to KotlinCompileContext. Chunk dependency calculation also moved to build start which improves performance for big projects #KT-26113
	- Kotlin bindings to JPS build targets also stored in KotlinCompileContext, and binding is fixed. Previously it is stored in local Context and reacreated for each chunk, now they stored in KotlinCompileContext which is binded by GlobalContextKey with this exception: source roots are calculated for each round, since temporary source roots with groovy stubs are created at build time and visible only in local compile context.
	- KotlinChunk introduced. All chunk-wide logic are moved from KotlinModuleBuildTarget (i.e compiler, language, cache version checking and dependent cache loading)
	- Fix legacy MPP common dependent modules

Cache version checking logic now works as following:
- At first chunk building all targets are loaded and used platforms are collected. Lookups cache manger is created based on this set. Actual cache attributes are loaded from FS. Based on CacheAttributesDiff.status this actions are performed: if cache is invalid all kotlin will be rebuilt. If cache is not required anymore it will be cleaned.
- Before build of each chunk local chunk cache attributes will be checked. If cache is invalid, chunk will be rebuilt. If cache is not required anymore it will be cleaned.

#KT-26113 Fixed
#KT-26072 Fixed

Original commit: 437fc9d749
This commit is contained in:
Sergey Rostov
2018-08-16 13:35:32 +03:00
parent dc022aef69
commit aa1eb9795d
55 changed files with 1509 additions and 781 deletions
@@ -16,14 +16,13 @@
package org.jetbrains.kotlin.jps.build
import org.jetbrains.jps.incremental.ModuleBuildTarget
import org.jetbrains.kotlin.incremental.testingUtils.BuildLogFinder
import org.jetbrains.kotlin.jps.incremental.CacheVersionProvider
import org.jetbrains.kotlin.jps.targets.KotlinModuleBuildTarget
abstract class AbstractDataContainerVersionChangedTest : AbstractIncrementalCacheVersionChangedTest() {
override val buildLogFinder: BuildLogFinder
get() = BuildLogFinder(isDataContainerBuildLogEnabled = true)
override fun getVersions(cacheVersionProvider: CacheVersionProvider, targets: Iterable<ModuleBuildTarget>) =
listOf(cacheVersionProvider.dataContainerVersion())
override fun getVersionManagersToTest(target: KotlinModuleBuildTarget<*>) =
listOf(kotlinCompileContext.lookupsCacheAttributesManager.versionManagerForTesting)
}
@@ -16,15 +16,14 @@
package org.jetbrains.kotlin.jps.build
import org.jetbrains.jps.incremental.ModuleBuildTarget
import org.jetbrains.kotlin.incremental.storage.version.CacheVersionManager
import org.jetbrains.kotlin.incremental.testingUtils.Modification
import org.jetbrains.kotlin.incremental.testingUtils.ModifyContent
import org.jetbrains.kotlin.jps.incremental.CacheVersionProvider
import org.jetbrains.kotlin.jps.targets.KotlinModuleBuildTarget
abstract class AbstractIncrementalCacheVersionChangedTest : AbstractIncrementalJpsTest(allowNoFilesWithSuffixInTestData = true) {
override fun performAdditionalModifications(modifications: List<Modification>) {
val modifiedFiles = modifications.filterIsInstance<ModifyContent>().map { it.path }
val paths = projectDescriptor.dataManager.dataPaths
val targets = projectDescriptor.allModuleTargets
val hasKotlin = HasKotlinMarker(projectDescriptor.dataManager)
@@ -33,13 +32,18 @@ abstract class AbstractIncrementalCacheVersionChangedTest : AbstractIncrementalJ
}
if (modifiedFiles.none { it.endsWith("do-not-change-cache-versions") }) {
val cacheVersionProvider = CacheVersionProvider(paths, isIncrementalCompilationEnabled = true)
val versions = getVersions(cacheVersionProvider, targets)
val versionFiles = versions.map { it.formatVersionFile }.filter { it.exists() }
versionFiles.forEach { it.writeText("777") }
val versions = targets.flatMap {
getVersionManagersToTest(kotlinCompileContext.targetsBinding[it]!!)
}
versions.forEach {
if (it.versionFileForTesting.exists()) {
it.versionFileForTesting.writeText("777")
}
}
}
}
protected open fun getVersions(cacheVersionProvider: CacheVersionProvider, targets: Iterable<ModuleBuildTarget>) =
targets.map { cacheVersionProvider.normalVersion(it) }
protected open fun getVersionManagersToTest(target: KotlinModuleBuildTarget<*>): List<CacheVersionManager> =
listOf(target.localCacheVersionManager)
}
@@ -44,13 +44,15 @@ import org.jetbrains.jps.model.java.JpsJavaDependencyScope
import org.jetbrains.jps.model.java.JpsJavaExtensionService
import org.jetbrains.jps.util.JpsPathUtil
import org.jetbrains.kotlin.config.IncrementalCompilation
import org.jetbrains.kotlin.incremental.CacheVersion
import org.jetbrains.kotlin.incremental.LookupSymbol
import org.jetbrains.kotlin.incremental.isJavaFile
import org.jetbrains.kotlin.incremental.storage.version.CacheAttributesDiff
import org.jetbrains.kotlin.incremental.storage.version.CacheVersionManager
import org.jetbrains.kotlin.incremental.testingUtils.*
import org.jetbrains.kotlin.jps.incremental.CompositeLookupsCacheAttributesManager
import org.jetbrains.kotlin.jps.incremental.getKotlinCache
import org.jetbrains.kotlin.jps.incremental.withLookupStorage
import org.jetbrains.kotlin.jps.platforms.kotlinBuildTargets
import org.jetbrains.kotlin.jps.targets.KotlinModuleBuildTarget
import org.jetbrains.kotlin.test.KotlinTestUtils
import org.jetbrains.kotlin.utils.Printer
import org.jetbrains.kotlin.utils.keysToMap
@@ -82,6 +84,8 @@ abstract class AbstractIncrementalJpsTest(
protected var mapWorkingToOriginalFile: MutableMap<File, File> = hashMapOf()
lateinit var kotlinCompileContext: KotlinCompileContext
protected open val buildLogFinder: BuildLogFinder
get() = BuildLogFinder()
@@ -145,19 +149,21 @@ abstract class AbstractIncrementalJpsTest(
projectDescriptor = createProjectDescriptor(BuildLoggingManager(logger))
val lookupTracker = TestLookupTracker()
projectDescriptor.project.setTestingContext(TestingContext(lookupTracker, logger))
val testingContext = TestingContext(lookupTracker, logger)
projectDescriptor.project.setTestingContext(testingContext)
try {
val builder = IncProjectBuilder(projectDescriptor, BuilderRegistry.getInstance(), myBuildParams, CanceledStatus.NULL, mockConstantSearch, true)
val buildResult = BuildResult()
builder.addMessageHandler(buildResult)
val finalScope = scope.build()
builder.build(finalScope, false)
lookupTracker.lookups.mapTo(lookupsDuringTest) { LookupSymbol(it.name, it.scopeFqName) }
// testingContext.kotlinCompileContext is initialized in KotlinBuilder.initializeKotlinContext
kotlinCompileContext = testingContext.kotlinCompileContext!!
// for getting kotlin platform only
val dummyCompileContext = CompileContextImpl.createContextForTests(finalScope, projectDescriptor)
lookupTracker.lookups.mapTo(lookupsDuringTest) { LookupSymbol(it.name, it.scopeFqName) }
if (!buildResult.isSuccessful) {
val errorMessages =
@@ -169,7 +175,7 @@ abstract class AbstractIncrementalJpsTest(
return MakeResult(logger.log + "$COMPILATION_FAILED\n" + errorMessages + "\n", true, null)
}
else {
return MakeResult(logger.log, false, createMappingsDump(projectDescriptor, dummyCompileContext))
return MakeResult(logger.log, false, createMappingsDump(projectDescriptor))
}
}
finally {
@@ -287,20 +293,18 @@ abstract class AbstractIncrementalJpsTest(
}
private fun createMappingsDump(
project: ProjectDescriptor,
dummyCompileContext: CompileContext
) = createKotlinIncrementalCacheDump(project, dummyCompileContext) + "\n\n\n" +
project: ProjectDescriptor
) = createKotlinIncrementalCacheDump(project) + "\n\n\n" +
createLookupCacheDump(project) + "\n\n\n" +
createCommonMappingsDump(project) + "\n\n\n" +
createJavaMappingsDump(project)
private fun createKotlinIncrementalCacheDump(
project: ProjectDescriptor,
dummyCompileContext: CompileContext
project: ProjectDescriptor
): String {
return buildString {
for (target in project.allModuleTargets.sortedBy { it.presentableName }) {
val kotlinCache = project.dataManager.getKotlinCache(dummyCompileContext.kotlinBuildTargets[target])
val kotlinCache = project.dataManager.getKotlinCache(kotlinCompileContext.targetsBinding[target])
if (kotlinCache != null) {
append("<target $target>\n")
append(kotlinCache.dump())
@@ -443,14 +447,23 @@ abstract class AbstractIncrementalJpsTest(
override fun doGetProjectDir(): File? = workDir
private class MyLogger(val rootPath: String) : ProjectBuilderLoggerBase(), BuildLogger {
private class MyLogger(val rootPath: String) : ProjectBuilderLoggerBase(), TestingBuildLogger {
private val markedDirtyBeforeRound = ArrayList<File>()
private val markedDirtyAfterRound = ArrayList<File>()
override fun actionsOnCacheVersionChanged(actions: List<CacheVersion.Action>) {
if (actions.size > 1 && actions.any { it != CacheVersion.Action.DO_NOTHING }) {
logLine("Actions after cache changed: $actions")
override fun invalidOrUnusedCache(
chunk: KotlinChunk?,
target: KotlinModuleBuildTarget<*>?,
attributesDiff: CacheAttributesDiff<*>
) {
val cacheManager = attributesDiff.manager
val cacheTitle = when (cacheManager) {
is CacheVersionManager -> "Local cache for ${chunk ?: target}"
is CompositeLookupsCacheAttributesManager -> "Lookups cache"
else -> error("Unknown cache manager $cacheManager")
}
logLine("$cacheTitle are $attributesDiff")
}
override fun markedAsDirtyBeforeRound(files: Iterable<File>) {
@@ -461,13 +474,15 @@ abstract class AbstractIncrementalJpsTest(
markedDirtyAfterRound.addAll(files)
}
override fun buildStarted(context: CompileContext, chunk: ModuleChunk) {
override fun chunkBuildStarted(context: CompileContext, chunk: ModuleChunk) {
logDirtyFiles(markedDirtyBeforeRound) // files can be marked as dirty during build start (KotlinCompileContext initialization)
if (!chunk.isDummy(context) && context.projectDescriptor.project.modules.size > 1) {
logLine("Building ${chunk.modules.sortedBy { it.name }.joinToString { it.name }}")
}
}
override fun afterBuildStarted(context: CompileContext, chunk: ModuleChunk) {
override fun afterChunkBuildStarted(context: CompileContext, chunk: ModuleChunk) {
logDirtyFiles(markedDirtyBeforeRound)
}
@@ -44,13 +44,14 @@ import org.jetbrains.jps.model.java.JpsJavaDependencyScope
import org.jetbrains.jps.model.java.JpsJavaExtensionService
import org.jetbrains.jps.util.JpsPathUtil
import org.jetbrains.kotlin.config.IncrementalCompilation
import org.jetbrains.kotlin.incremental.CacheVersion
import org.jetbrains.kotlin.incremental.LookupSymbol
import org.jetbrains.kotlin.incremental.isJavaFile
import org.jetbrains.kotlin.incremental.storage.version.CacheAttributesDiff
import org.jetbrains.kotlin.incremental.storage.version.CacheVersionManager
import org.jetbrains.kotlin.incremental.testingUtils.*
import org.jetbrains.kotlin.jps.incremental.CompositeLookupsCacheAttributesManager
import org.jetbrains.kotlin.jps.incremental.getKotlinCache
import org.jetbrains.kotlin.jps.incremental.withLookupStorage
import org.jetbrains.kotlin.jps.platforms.kotlinBuildTargets
import org.jetbrains.kotlin.jps.targets.KotlinModuleBuildTarget
import org.jetbrains.kotlin.test.KotlinTestUtils
import org.jetbrains.kotlin.utils.Printer
import org.jetbrains.kotlin.utils.keysToMap
@@ -82,6 +83,8 @@ abstract class AbstractIncrementalJpsTest(
protected var mapWorkingToOriginalFile: MutableMap<File, File> = hashMapOf()
lateinit var kotlinCompileContext: KotlinCompileContext
protected open val buildLogFinder: BuildLogFinder
get() = BuildLogFinder()
@@ -144,19 +147,20 @@ abstract class AbstractIncrementalJpsTest(
projectDescriptor = createProjectDescriptor(BuildLoggingManager(logger))
val lookupTracker = TestLookupTracker()
projectDescriptor.project.setTestingContext(TestingContext(lookupTracker, logger))
val testingContext = TestingContext(lookupTracker, logger)
projectDescriptor.project.setTestingContext(testingContext)
try {
val builder = IncProjectBuilder(projectDescriptor, BuilderRegistry.getInstance(), myBuildParams, CanceledStatus.NULL, mockConstantSearch, true)
val buildResult = BuildResult()
builder.addMessageHandler(buildResult)
val finalScope = scope.build()
builder.build(finalScope, false)
lookupTracker.lookups.mapTo(lookupsDuringTest) { LookupSymbol(it.name, it.scopeFqName) }
// testingContext.kotlinCompileContext is initialized in KotlinBuilder.initializeKotlinContext
kotlinCompileContext = testingContext.kotlinCompileContext!!
// for getting kotlin platform only
val dummyCompileContext = CompileContextImpl.createContextForTests(finalScope, projectDescriptor)
if (!buildResult.isSuccessful) {
val errorMessages =
@@ -168,7 +172,7 @@ abstract class AbstractIncrementalJpsTest(
return MakeResult(logger.log + "$COMPILATION_FAILED\n" + errorMessages + "\n", true, null)
}
else {
return MakeResult(logger.log, false, createMappingsDump(projectDescriptor, dummyCompileContext))
return MakeResult(logger.log, false, createMappingsDump(projectDescriptor))
}
}
finally {
@@ -286,20 +290,18 @@ abstract class AbstractIncrementalJpsTest(
}
private fun createMappingsDump(
project: ProjectDescriptor,
dummyCompileContext: CompileContext
) = createKotlinIncrementalCacheDump(project, dummyCompileContext) + "\n\n\n" +
project: ProjectDescriptor
) = createKotlinIncrementalCacheDump(project) + "\n\n\n" +
createLookupCacheDump(project) + "\n\n\n" +
createCommonMappingsDump(project) + "\n\n\n" +
createJavaMappingsDump(project)
private fun createKotlinIncrementalCacheDump(
project: ProjectDescriptor,
dummyCompileContext: CompileContext
project: ProjectDescriptor
): String {
return buildString {
for (target in project.allModuleTargets.sortedBy { it.presentableName }) {
val kotlinCache = project.dataManager.getKotlinCache(dummyCompileContext.kotlinBuildTargets[target])
val kotlinCache = project.dataManager.getKotlinCache(kotlinCompileContext.targetsBinding[target])
if (kotlinCache != null) {
append("<target $target>\n")
append(kotlinCache.dump())
@@ -442,14 +444,23 @@ abstract class AbstractIncrementalJpsTest(
override fun doGetProjectDir(): File? = workDir
private class MyLogger(val rootPath: String) : ProjectBuilderLoggerBase(), BuildLogger {
private class MyLogger(val rootPath: String) : ProjectBuilderLoggerBase(), TestingBuildLogger {
private val markedDirtyBeforeRound = ArrayList<File>()
private val markedDirtyAfterRound = ArrayList<File>()
override fun actionsOnCacheVersionChanged(actions: List<CacheVersion.Action>) {
if (actions.size > 1 && actions.any { it != CacheVersion.Action.DO_NOTHING }) {
logLine("Actions after cache changed: $actions")
override fun invalidOrUnusedCache(
chunk: KotlinChunk?,
target: KotlinModuleBuildTarget<*>?,
attributesDiff: CacheAttributesDiff<*>
) {
val cacheManager = attributesDiff.manager
val cacheTitle = when (cacheManager) {
is CacheVersionManager -> "Local cache for ${chunk ?: target}"
is CompositeLookupsCacheAttributesManager -> "Lookups cache"
else -> error("Unknown cache manager $cacheManager")
}
logLine("$cacheTitle are $attributesDiff")
}
override fun markedAsDirtyBeforeRound(files: Iterable<File>) {
@@ -460,13 +471,15 @@ abstract class AbstractIncrementalJpsTest(
markedDirtyAfterRound.addAll(files)
}
override fun buildStarted(context: CompileContext, chunk: ModuleChunk) {
override fun chunkBuildStarted(context: CompileContext, chunk: ModuleChunk) {
logDirtyFiles(markedDirtyBeforeRound) // files can be marked as dirty during build start (KotlinCompileContext initialization)
if (!chunk.isDummy(context) && context.projectDescriptor.project.modules.size > 1) {
logLine("Building ${chunk.modules.sortedBy { it.name }.joinToString { it.name }}")
}
}
override fun afterBuildStarted(context: CompileContext, chunk: ModuleChunk) {
override fun afterChunkBuildStarted(context: CompileContext, chunk: ModuleChunk) {
logDirtyFiles(markedDirtyBeforeRound)
}
@@ -18,17 +18,14 @@ package org.jetbrains.kotlin.jps.build
import com.intellij.testFramework.UsefulTestCase
import org.jetbrains.jps.builders.BuildTarget
import org.jetbrains.jps.builders.CompileScopeTestBuilder
import org.jetbrains.jps.builders.storage.BuildDataPaths
import org.jetbrains.jps.incremental.CompileContextImpl
import org.jetbrains.kotlin.config.IncrementalCompilation
import org.jetbrains.kotlin.incremental.KOTLIN_CACHE_DIRECTORY_NAME
import org.jetbrains.kotlin.incremental.storage.BasicMapsOwner
import org.jetbrains.kotlin.incremental.testingUtils.Modification
import org.jetbrains.kotlin.incremental.testingUtils.ModifyContent
import org.jetbrains.kotlin.jps.incremental.CacheVersionProvider
import org.jetbrains.kotlin.jps.incremental.KotlinDataContainerTarget
import org.jetbrains.kotlin.jps.platforms.kotlinBuildTargets
import org.jetbrains.kotlin.jps.targets.KotlinModuleBuildTarget
import org.jetbrains.kotlin.utils.Printer
import java.io.File
@@ -85,44 +82,55 @@ abstract class AbstractIncrementalLazyCachesTest : AbstractIncrementalJpsTest()
private fun dumpKotlinCachesFileNames(): String {
val sb = StringBuilder()
val p = Printer(sb)
val targets = projectDescriptor.allModuleTargets
val printer = Printer(sb)
val chunks = kotlinCompileContext.targetsIndex.chunks
val dataManager = projectDescriptor.dataManager
val paths = dataManager.dataPaths
val versions = CacheVersionProvider(paths, isIncrementalCompilationEnabled = true)
dumpCachesForTarget(p, paths, KotlinDataContainerTarget, versions.dataContainerVersion().formatVersionFile)
dumpCachesForTarget(
printer,
paths,
KotlinDataContainerTarget,
kotlinCompileContext.lookupsCacheAttributesManager.versionManagerForTesting.versionFileForTesting
)
// for getting kotlin platform only
val dummyCompileContext = CompileContextImpl.createContextForTests(CompileScopeTestBuilder.make().build(), projectDescriptor)
data class TargetInChunk(val chunk: KotlinChunk, val target: KotlinModuleBuildTarget<*>)
for (target in targets.sortedBy { it.presentableName }) {
val kotlinModuleBuildTarget = dummyCompileContext.kotlinBuildTargets[target]!!
val metaBuildInfo = kotlinModuleBuildTarget.buildMetaInfoFile(target, dataManager)
dumpCachesForTarget(p, paths, target,
versions.normalVersion(target).formatVersionFile,
metaBuildInfo,
subdirectory = KOTLIN_CACHE_DIRECTORY_NAME)
val allTargets = chunks.flatMap { chunk ->
chunk.targets.map { target ->
TargetInChunk(chunk, target)
}
}.sortedBy { it.target.jpsModuleBuildTarget.presentableName }
allTargets.forEach { (chunk, target) ->
val metaBuildInfo = chunk.buildMetaInfoFile(target.jpsModuleBuildTarget)
dumpCachesForTarget(
printer, paths, target.jpsModuleBuildTarget,
target.localCacheVersionManager.versionFileForTesting,
metaBuildInfo,
subdirectory = KOTLIN_CACHE_DIRECTORY_NAME
)
}
return sb.toString()
}
private fun dumpCachesForTarget(
p: Printer,
paths: BuildDataPaths,
target: BuildTarget<*>,
vararg cacheVersionsFiles: File,
subdirectory: String? = null
p: Printer,
paths: BuildDataPaths,
target: BuildTarget<*>,
vararg cacheVersionsFiles: File,
subdirectory: String? = null
) {
p.println(target)
p.pushIndent()
val dataRoot = paths.getTargetDataRoot(target).let { if (subdirectory != null) File(it, subdirectory) else it }
cacheVersionsFiles
.filter(File::exists)
.sortedBy { it.name }
.forEach { p.println(it.name) }
.filter(File::exists)
.sortedBy { it.name }
.forEach { p.println(it.name) }
kotlinCacheNames(dataRoot).sorted().forEach { p.println(it) }
@@ -58,13 +58,13 @@ import org.jetbrains.kotlin.codegen.AsmUtil
import org.jetbrains.kotlin.codegen.JvmCodegenUtil
import org.jetbrains.kotlin.config.IncrementalCompilation
import org.jetbrains.kotlin.config.KotlinCompilerVersion.TEST_IS_PRE_RELEASE_SYSTEM_PROPERTY
import org.jetbrains.kotlin.incremental.CacheVersion
import org.jetbrains.kotlin.incremental.components.LookupTracker
import org.jetbrains.kotlin.incremental.storage.version.CacheAttributesDiff
import org.jetbrains.kotlin.incremental.withIC
import org.jetbrains.kotlin.jps.build.KotlinJpsBuildTest.LibraryDependency.*
import org.jetbrains.kotlin.jps.model.kotlinCommonCompilerArguments
import org.jetbrains.kotlin.jps.model.kotlinCompilerArguments
import org.jetbrains.kotlin.jps.platforms.productionBuildTarget
import org.jetbrains.kotlin.jps.targets.KotlinModuleBuildTarget
import org.jetbrains.kotlin.load.kotlin.PackagePartClassUtils
import org.jetbrains.kotlin.name.FqName
import org.jetbrains.kotlin.test.KotlinTestUtils
@@ -970,6 +970,9 @@ open class KotlinJpsBuildTest : AbstractKotlinJpsBuildTestCase() {
}
}
// c -> b -exported-> a
// c2 -> b2 ------------^
val a = addModuleWithSourceAndTestRoot("a")
val b = addModuleWithSourceAndTestRoot("b")
val c = addModuleWithSourceAndTestRoot("c")
@@ -983,15 +986,21 @@ open class KotlinJpsBuildTest : AbstractKotlinJpsBuildTestCase() {
val actual = StringBuilder()
buildCustom(CanceledStatus.NULL, TestProjectBuilderLogger(), BuildResult()) {
project.setTestingContext(TestingContext(LookupTracker.DO_NOTHING, object : BuildLogger {
override fun buildStarted(context: CompileContext, chunk: ModuleChunk) {
actual.append("Targets dependent on ${chunk.targets.joinToString() }:\n")
actual.append(getDependentTargets(chunk.targets, context).map { it.toString() }.sorted().joinToString("\n"))
project.setTestingContext(TestingContext(LookupTracker.DO_NOTHING, object : TestingBuildLogger {
override fun chunkBuildStarted(context: CompileContext, chunk: ModuleChunk) {
actual.append("Targets dependent on ${chunk.targets.joinToString()}:\n")
val dependentRecursively = mutableSetOf<KotlinChunk>()
context.kotlin.getChunk(chunk)!!.collectDependentChunksRecursivelyExportedOnly(dependentRecursively)
dependentRecursively.asSequence().map { it.targets.joinToString() }.sorted().joinTo(actual, "\n")
actual.append("\n---------\n")
}
override fun afterBuildStarted(context: CompileContext, chunk: ModuleChunk) {}
override fun actionsOnCacheVersionChanged(actions: List<CacheVersion.Action>) {}
override fun afterChunkBuildStarted(context: CompileContext, chunk: ModuleChunk) {}
override fun invalidOrUnusedCache(
chunk: KotlinChunk?,
target: KotlinModuleBuildTarget<*>?,
attributesDiff: CacheAttributesDiff<*>
) {}
override fun buildFinished(exitCode: ModuleLevelBuilder.ExitCode) {}
override fun markedAsDirtyBeforeRound(files: Iterable<File>) {}
override fun markedAsDirtyAfterRound(files: Iterable<File>) {}
@@ -0,0 +1,82 @@
/*
* Copyright 2010-2018 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license
* that can be found in the license/LICENSE.txt file.
*/
package org.jetbrains.kotlin.jps.incremental
import org.jetbrains.kotlin.incremental.storage.version.CacheStatus
import org.jetbrains.kotlin.incremental.storage.version.loadDiff
import org.junit.Test
import java.io.File
import kotlin.test.assertEquals
class CompositeLookupsCacheAttributesManagerTest {
val manager = CompositeLookupsCacheAttributesManager(File("not-used"), setOf())
@Test
fun testNothingToJava() {
assertEquals(
CacheStatus.INVALID,
manager.loadDiff(
actual = null,
expected = CompositeLookupsCacheAttributes(1, setOf("jvm"))
).status
)
}
@Test
fun testNothingToJavaAndJs() {
assertEquals(
CacheStatus.INVALID,
manager.loadDiff(
actual = null,
expected = CompositeLookupsCacheAttributes(1, setOf("jvm", "js"))
).status
)
}
@Test
fun testJsToJava() {
assertEquals(
CacheStatus.INVALID,
manager.loadDiff(
actual = CompositeLookupsCacheAttributes(1, setOf("jvm")),
expected = CompositeLookupsCacheAttributes(1, setOf("js"))
).status
)
}
@Test
fun testJsAndJavaToJava() {
assertEquals(
CacheStatus.VALID,
manager.loadDiff(
actual = CompositeLookupsCacheAttributes(1, setOf("jvm", "js")),
expected = CompositeLookupsCacheAttributes(1, setOf("jvm"))
).status
)
}
@Test
fun testJsAndJavaToJavaWithOtherVersion() {
assertEquals(
CacheStatus.INVALID,
manager.loadDiff(
actual = CompositeLookupsCacheAttributes(1, setOf("jvm", "js")),
expected = CompositeLookupsCacheAttributes(2, setOf("jvm"))
).status
)
}
@Test
fun testJavaToJsAndJava() {
assertEquals(
CacheStatus.INVALID,
manager.loadDiff(
actual = CompositeLookupsCacheAttributes(1, setOf("jvm")),
expected = CompositeLookupsCacheAttributes(1, setOf("jvm", "js"))
).status
)
}
}
@@ -89,7 +89,9 @@ class FSOperationsHelper(
}
fun markInChunkOrDependents(files: Iterable<File>, excludeFiles: Set<File>) {
markFilesImpl(files, currentRound = false) { it !in excludeFiles && it.exists() && moduleBasedFilter.accept(it) }
markFilesImpl(files, currentRound = false) {
it !in excludeFiles && it.exists() && moduleBasedFilter.accept(it)
}
}
private inline fun markFilesImpl(
@@ -17,9 +17,7 @@
package org.jetbrains.kotlin.jps.build
import com.intellij.openapi.diagnostic.Logger
import com.intellij.util.containers.ContainerUtil
import org.jetbrains.jps.ModuleChunk
import org.jetbrains.jps.builders.BuildTarget
import org.jetbrains.jps.builders.DirtyFilesHolder
import org.jetbrains.jps.builders.FileProcessor
import org.jetbrains.jps.builders.impl.DirtyFilesHolderBase
@@ -30,9 +28,6 @@ import org.jetbrains.jps.incremental.ModuleLevelBuilder.ExitCode.*
import org.jetbrains.jps.incremental.java.JavaBuilder
import org.jetbrains.jps.incremental.storage.BuildDataManager
import org.jetbrains.jps.model.JpsProject
import org.jetbrains.jps.model.java.JpsJavaClasspathKind
import org.jetbrains.jps.model.java.JpsJavaExtensionService
import org.jetbrains.jps.model.module.JpsModule
import org.jetbrains.kotlin.build.GeneratedFile
import org.jetbrains.kotlin.cli.common.arguments.CommonCompilerArguments
import org.jetbrains.kotlin.cli.common.messages.CompilerMessageSeverity
@@ -45,20 +40,20 @@ import org.jetbrains.kotlin.daemon.common.isDaemonEnabled
import org.jetbrains.kotlin.incremental.*
import org.jetbrains.kotlin.incremental.components.ExpectActualTracker
import org.jetbrains.kotlin.incremental.components.LookupTracker
import org.jetbrains.kotlin.jps.incremental.*
import org.jetbrains.kotlin.jps.incremental.JpsIncrementalCache
import org.jetbrains.kotlin.jps.incremental.withLookupStorage
import org.jetbrains.kotlin.jps.model.kotlinKind
import org.jetbrains.kotlin.jps.platforms.KotlinJvmModuleBuildTarget
import org.jetbrains.kotlin.jps.platforms.KotlinModuleBuildTarget
import org.jetbrains.kotlin.jps.platforms.kotlinBuildTargets
import org.jetbrains.kotlin.jps.targets.KotlinJvmModuleBuildTarget
import org.jetbrains.kotlin.jps.targets.KotlinModuleBuildTarget
import org.jetbrains.kotlin.name.FqName
import org.jetbrains.kotlin.preloading.ClassCondition
import org.jetbrains.kotlin.utils.KotlinPaths
import org.jetbrains.kotlin.utils.KotlinPathsFromHomeDir
import org.jetbrains.kotlin.utils.PathUtil
import org.jetbrains.kotlin.utils.keysToMapExceptNulls
import java.io.File
import java.util.*
import kotlin.collections.HashSet
import kotlin.system.measureTimeMillis
class KotlinBuilder : ModuleLevelBuilder(BuilderCategory.SOURCE_PROCESSOR) {
companion object {
@@ -89,12 +84,16 @@ class KotlinBuilder : ModuleLevelBuilder(BuilderCategory.SOURCE_PROCESSOR) {
override fun getCompilableFileExtensions() = arrayListOf("kt", "kts")
override fun buildStarted(context: CompileContext) {
logSettings(context)
}
private fun logSettings(context: CompileContext) {
LOG.debug("==========================================")
LOG.info("is Kotlin incremental compilation enabled for JVM: ${IncrementalCompilation.isEnabledForJvm()}")
LOG.info("is Kotlin incremental compilation enabled for JS: ${IncrementalCompilation.isEnabledForJs()}")
if (IncrementalCompilation.isEnabledForJs()) {
val messageCollector = MessageCollectorAdapter(context, null)
messageCollector.report(CompilerMessageSeverity.INFO, "Using experimental incremental compilation for Kotlin/JS")
messageCollector.report(INFO, "Using experimental incremental compilation for Kotlin/JS")
}
LOG.info("is Kotlin compiler daemon enabled: ${isDaemonEnabled()}")
@@ -105,8 +104,66 @@ class KotlinBuilder : ModuleLevelBuilder(BuilderCategory.SOURCE_PROCESSOR) {
}
}
override fun buildFinished(context: CompileContext?) {
statisticsLogger.reportTotal()
/**
* Ensure Kotlin Context initialized.
* Kotlin Context should be initialized only when required (before first kotlin chunk build).
*/
private fun ensureKotlinContextInitialized(context: CompileContext): KotlinCompileContext {
val kotlinCompileContext = context.getUserData(kotlinCompileContextKey)
if (kotlinCompileContext != null) return kotlinCompileContext
// don't synchronize on context, since it is chunk local only
synchronized(kotlinCompileContextKey) {
val actualKotlinCompileContext = context.getUserData(kotlinCompileContextKey)
if (actualKotlinCompileContext != null) return actualKotlinCompileContext
try {
return initializeKotlinContext(context)
} catch (t: Throwable) {
jpsReportInternalBuilderError(context, Error("Cannot initialize Kotlin context: ${t.message}", t))
throw t
}
}
}
private fun initializeKotlinContext(context: CompileContext): KotlinCompileContext {
lateinit var kotlinContext: KotlinCompileContext
val time = measureTimeMillis {
kotlinContext = KotlinCompileContext(context)
context.putUserData(kotlinCompileContextKey, kotlinContext)
context.testingContext?.kotlinCompileContext = kotlinContext
if (kotlinContext.shouldCheckCacheVersions && kotlinContext.hasKotlin()) {
kotlinContext.checkCacheVersions()
}
kotlinContext.cleanupCaches()
}
LOG.info("Total Kotlin global compile context initialization time: $time ms")
return kotlinContext
}
override fun buildFinished(context: CompileContext) {
ensureKotlinContextDisposed(context)
}
private fun ensureKotlinContextDisposed(context: CompileContext) {
if (context.getUserData(kotlinCompileContextKey) != null) {
// don't synchronize on context, since it chunk local only
synchronized(kotlinCompileContextKey) {
val kotlinCompileContext = context.getUserData(kotlinCompileContextKey)
if (kotlinCompileContext != null) {
kotlinCompileContext.dispose()
context.putUserData(kotlinCompileContextKey, null)
statisticsLogger.reportTotal()
}
}
}
}
override fun chunkBuildStarted(context: CompileContext, chunk: ModuleChunk) {
@@ -114,18 +171,40 @@ class KotlinBuilder : ModuleLevelBuilder(BuilderCategory.SOURCE_PROCESSOR) {
if (chunk.isDummy(context)) return
val kotlinContext = ensureKotlinContextInitialized(context)
val buildLogger = context.testingContext?.buildLogger
buildLogger?.buildStarted(context, chunk)
buildLogger?.chunkBuildStarted(context, chunk)
if (JavaBuilderUtil.isForcedRecompilationAllJavaModules(context)) return
val targets = chunk.targets
val dataManager = context.projectDescriptor.dataManager
val hasKotlin = HasKotlinMarker(dataManager)
if (targets.none { kotlinContext.hasKotlinMarker[it] == true }) return
if (targets.none { hasKotlin[it] == true }) return
val kotlinChunk = kotlinContext.getChunk(chunk) ?: return
kotlinContext.checkChunkCacheVersion(kotlinChunk)
val roundDirtyFiles = KotlinDirtySourceFilesHolder(
if (!kotlinContext.rebuildingAllKotlin) {
markAdditionalFilesForInitialRound(kotlinChunk, chunk, kotlinContext)
}
buildLogger?.afterChunkBuildStarted(context, chunk)
}
/**
* Invalidate usages of removed classes.
* See KT-13677 for more details.
*
* todo(1.2.80): move to KotlinChunk
* todo(1.2.80): got rid of jpsGlobalContext usages (replace with KotlinCompileContext)
*/
private fun markAdditionalFilesForInitialRound(
kotlinChunk: KotlinChunk,
chunk: ModuleChunk,
kotlinContext: KotlinCompileContext
) {
val context = kotlinContext.jpsContext
val dirtyFilesHolder = KotlinDirtySourceFilesHolder(
chunk,
context,
object : DirtyFilesHolderBase<JavaSourceRootDescriptor, ModuleBuildTarget>(context) {
@@ -134,50 +213,16 @@ class KotlinBuilder : ModuleLevelBuilder(BuilderCategory.SOURCE_PROCESSOR) {
}
}
)
val fsOperations = FSOperationsHelper(context, chunk, roundDirtyFiles, LOG)
val fsOperations = FSOperationsHelper(context, chunk, dirtyFilesHolder, LOG)
val jpsRepresentativeTarget = chunk.representativeTarget()
val representativeTarget = context.kotlinBuildTargets[jpsRepresentativeTarget]
if (representativeTarget == null) {
LOG.warn("Unable to find Kotlin build target for JPS target ${jpsRepresentativeTarget.presentableName}")
}
if (System.getProperty(SKIP_CACHE_VERSION_CHECK_PROPERTY) == null && representativeTarget != null) {
val cacheVersionsProvider = CacheVersionProvider(dataManager.dataPaths, representativeTarget.isIncrementalCompilationEnabled)
val actions = checkCachesVersions(context, cacheVersionsProvider, chunk)
applyActionsOnCacheVersionChange(actions, cacheVersionsProvider, context, dataManager, targets, fsOperations)
if (CacheVersion.Action.REBUILD_ALL_KOTLIN in actions) {
return
}
}
val representativeTarget = kotlinContext.targetsBinding[chunk.representativeTarget()] ?: return
// try to perform a lookup
// request rebuild if storage is corrupted
try {
dataManager.withLookupStorage {
it.get(LookupSymbol("<#NAME#>", "<#SCOPE#>"))
}
} catch (e: Exception) {
// todo: report to Intellij when IDEA-187115 is implemented
LOG.info(e)
markAllKotlinForRebuild(context, fsOperations, "Lookup storage is corrupted")
return
}
// dependent caches are not required, since we are not going to update caches
val incrementalCaches = kotlinChunk.loadCaches(loadDependent = false)
markAdditionalFilesForInitialRound(chunk, context, fsOperations, roundDirtyFiles)
buildLogger?.afterBuildStarted(context, chunk)
}
private fun markAdditionalFilesForInitialRound(
chunk: ModuleChunk,
context: CompileContext,
fsOperations: FSOperationsHelper,
dirtyFilesHolder: KotlinDirtySourceFilesHolder
) {
val representativeTarget = context.kotlinBuildTargets[chunk.representativeTarget()] ?: return
val incrementalCaches = getIncrementalCaches(chunk, context)
val messageCollector = MessageCollectorAdapter(context, representativeTarget)
val environment = createCompileEnvironment(
kotlinContext.jpsContext,
representativeTarget,
incrementalCaches,
LookupTracker.DO_NOTHING,
@@ -187,10 +232,10 @@ class KotlinBuilder : ModuleLevelBuilder(BuilderCategory.SOURCE_PROCESSOR) {
) ?: return
val removedClasses = HashSet<String>()
for (target in chunk.targets) {
for (target in kotlinChunk.targets) {
val cache = incrementalCaches[target] ?: continue
val dirtyFiles = dirtyFilesHolder.getDirtyFiles(target)
val removedFiles = dirtyFilesHolder.getRemovedFiles(target)
val dirtyFiles = dirtyFilesHolder.getDirtyFiles(target.jpsModuleBuildTarget)
val removedFiles = dirtyFilesHolder.getRemovedFiles(target.jpsModuleBuildTarget)
val existingClasses = JpsKotlinCompilerRunner().classesFqNamesByFiles(environment, dirtyFiles)
val previousClasses = cache.classesFqNamesBySources(dirtyFiles + removedFiles)
@@ -209,23 +254,6 @@ class KotlinBuilder : ModuleLevelBuilder(BuilderCategory.SOURCE_PROCESSOR) {
fsOperations.markFilesForCurrentRound(affectedByRemovedClasses)
}
private fun checkCachesVersions(
context: CompileContext,
cacheVersionsProvider: CacheVersionProvider,
chunk: ModuleChunk
): Set<CacheVersion.Action> {
val targets = chunk.targets
val dataManager = context.projectDescriptor.dataManager
val allVersions = cacheVersionsProvider.allVersions(targets)
val actions = allVersions.map { it.checkVersion() }.toMutableSet()
val kotlinModuleBuilderTarget = context.kotlinBuildTargets[chunk.representativeTarget()]!!
kotlinModuleBuilderTarget.checkCachesVersions(chunk, dataManager, actions)
return actions
}
override fun chunkBuildFinished(context: CompileContext, chunk: ModuleChunk) {
super.chunkBuildFinished(context, chunk)
@@ -252,7 +280,7 @@ class KotlinBuilder : ModuleLevelBuilder(BuilderCategory.SOURCE_PROCESSOR) {
if (chunk.modules.size > 1) {
messageCollector.report(
CompilerMessageSeverity.ERROR,
"Cyclically dependent modules is not supported in multiplatform projects"
"Cyclically dependent modules are not supported in multiplatform projects"
)
return ABORT
}
@@ -299,39 +327,47 @@ class KotlinBuilder : ModuleLevelBuilder(BuilderCategory.SOURCE_PROCESSOR) {
return NOTHING_DONE
}
val kotlinContext = context.kotlin
val projectDescriptor = context.projectDescriptor
val dataManager = projectDescriptor.dataManager
val targets = chunk.targets
val hasKotlin = HasKotlinMarker(dataManager)
val rebuildAfterCacheVersionChanged = RebuildAfterCacheVersionChangeMarker(dataManager)
val isChunkRebuilding = JavaBuilderUtil.isForcedRecompilationAllJavaModules(context)
|| targets.any { rebuildAfterCacheVersionChanged[it] == true }
|| targets.any { kotlinContext.rebuildAfterCacheVersionChanged[it] == true }
if (kotlinDirtyFilesHolder.hasDirtyOrRemovedFiles) {
if (!isChunkRebuilding && !representativeTarget.isIncrementalCompilationEnabled) {
targets.forEach { rebuildAfterCacheVersionChanged[it] = true }
return CHUNK_REBUILD_REQUIRED
}
} else {
if (!kotlinDirtyFilesHolder.hasDirtyOrRemovedFiles) {
if (isChunkRebuilding) {
targets.forEach { hasKotlin[it] = false }
targets.forEach {
kotlinContext.hasKotlinMarker[it] = false
}
}
targets.forEach { rebuildAfterCacheVersionChanged.clean(it) }
targets.forEach { kotlinContext.rebuildAfterCacheVersionChanged.clean(it) }
return NOTHING_DONE
}
// Request CHUNK_REBUILD when IC is off and there are dirty Kotlin files
// Otherwise unexpected compile error might happen, when there are Groovy files,
// but they are not dirty, so Groovy builder does not generate source stubs,
// and Kotlin builder is filtering out output directory from classpath
// (because it may contain outdated Java classes).
if (!isChunkRebuilding && !representativeTarget.isIncrementalCompilationEnabled) {
targets.forEach { kotlinContext.rebuildAfterCacheVersionChanged[it] = true }
return CHUNK_REBUILD_REQUIRED
}
val targetsWithoutOutputDir = targets.filter { it.outputDir == null }
if (targetsWithoutOutputDir.isNotEmpty()) {
messageCollector.report(ERROR, "Output directory not specified for " + targetsWithoutOutputDir.joinToString())
return ABORT
}
val kotlinChunk = chunk.toKotlinChunk(context)!!
val project = projectDescriptor.project
val lookupTracker = getLookupTracker(project, representativeTarget)
val exceptActualTracer = ExpectActualTrackerImpl()
val incrementalCaches = getIncrementalCaches(chunk, context)
val incrementalCaches = kotlinChunk.loadCaches()
val environment = createCompileEnvironment(
context,
representativeTarget,
incrementalCaches,
lookupTracker,
@@ -340,7 +376,7 @@ class KotlinBuilder : ModuleLevelBuilder(BuilderCategory.SOURCE_PROCESSOR) {
messageCollector
) ?: return ABORT
val commonArguments = representativeTarget.compilerArgumentsForChunk(chunk).apply {
val commonArguments = kotlinChunk.compilerArguments.apply {
reportOutputFiles = true
version = true // Always report the version to help diagnosing user issues if they submit the compiler output
if (languageVersion == null) languageVersion = VersionView.RELEASED_VERSION.versionString
@@ -352,8 +388,15 @@ class KotlinBuilder : ModuleLevelBuilder(BuilderCategory.SOURCE_PROCESSOR) {
val start = System.nanoTime()
val outputItemCollector = doCompileModuleChunk(
chunk, representativeTarget, commonArguments, context, kotlinDirtyFilesHolder, fsOperations,
environment, incrementalCaches
kotlinChunk,
chunk,
representativeTarget,
commonArguments,
context,
kotlinDirtyFilesHolder,
fsOperations,
environment,
incrementalCaches
)
statisticsLogger.registerStatistic(chunk, System.nanoTime() - start)
@@ -373,22 +416,23 @@ class KotlinBuilder : ModuleLevelBuilder(BuilderCategory.SOURCE_PROCESSOR) {
val generatedFiles = getGeneratedFiles(context, chunk, environment.outputItemsCollector)
registerOutputItems(outputConsumer, generatedFiles)
representativeTarget.saveVersions(context, chunk, commonArguments)
kotlinChunk.saveVersions()
if (targets.any { hasKotlin[it] == null }) {
if (targets.any { kotlinContext.hasKotlinMarker[it] == null }) {
fsOperations.markChunk(recursively = false, kotlinOnly = true, excludeFiles = kotlinDirtyFilesHolder.allDirtyFiles)
}
for (target in targets) {
hasKotlin[target] = true
rebuildAfterCacheVersionChanged.clean(target)
kotlinContext.hasKotlinMarker[target] = true
kotlinContext.rebuildAfterCacheVersionChanged.clean(target)
}
chunk.targets.forEach {
context.kotlinBuildTargets[it]?.doAfterBuild()
kotlinChunk.targets.forEach {
it.doAfterBuild()
}
representativeTarget.updateChunkMappings(
context,
chunk,
kotlinDirtyFilesHolder,
generatedFiles,
@@ -407,8 +451,8 @@ class KotlinBuilder : ModuleLevelBuilder(BuilderCategory.SOURCE_PROCESSOR) {
val changesCollector = ChangesCollector()
for ((target, files) in generatedFiles) {
val kotlinModuleBuilderTarget = context.kotlinBuildTargets[target]!!
kotlinModuleBuilderTarget.updateCaches(incrementalCaches[target]!!, files, changesCollector, environment)
val kotlinModuleBuilderTarget = kotlinContext.targetsBinding[target]!!
kotlinModuleBuilderTarget.updateCaches(incrementalCaches[kotlinModuleBuilderTarget]!!, files, changesCollector, environment)
}
updateLookupStorage(lookupTracker, dataManager, kotlinDirtyFilesHolder)
@@ -426,85 +470,10 @@ class KotlinBuilder : ModuleLevelBuilder(BuilderCategory.SOURCE_PROCESSOR) {
return OK
}
private fun applyActionsOnCacheVersionChange(
actions: Set<CacheVersion.Action>,
cacheVersionsProvider: CacheVersionProvider,
context: CompileContext,
dataManager: BuildDataManager,
targets: MutableSet<ModuleBuildTarget>,
fsOperations: FSOperationsHelper
) {
val hasKotlin = HasKotlinMarker(dataManager)
val sortedActions = actions.sorted()
context.testingContext?.buildLogger?.actionsOnCacheVersionChanged(sortedActions)
for (status in sortedActions) {
when (status) {
CacheVersion.Action.REBUILD_ALL_KOTLIN -> {
markAllKotlinForRebuild(context, fsOperations, "Kotlin global lookup map format changed")
return
}
CacheVersion.Action.REBUILD_CHUNK -> {
LOG.info("Clearing caches for " + targets.joinToString { it.presentableName })
val rebuildAfterCacheVersionChanged = RebuildAfterCacheVersionChangeMarker(dataManager)
val kotlinBuildTargets = context.kotlinBuildTargets
for (target in targets) {
dataManager.getKotlinCache(kotlinBuildTargets[target])?.clean()
hasKotlin.clean(target)
rebuildAfterCacheVersionChanged[target] = true
}
fsOperations.markChunk(recursively = false, kotlinOnly = true)
return
}
CacheVersion.Action.CLEAN_NORMAL_CACHES -> {
LOG.info("Clearing caches for all targets")
val kotlinBuildTargets = context.kotlinBuildTargets
for (target in context.allTargets()) {
dataManager.getKotlinCache(kotlinBuildTargets[target])?.clean()
}
}
CacheVersion.Action.CLEAN_DATA_CONTAINER -> {
LOG.info("Clearing lookup cache")
dataManager.cleanLookupStorage(LOG)
cacheVersionsProvider.dataContainerVersion().clean()
}
else -> {
assert(status == CacheVersion.Action.DO_NOTHING) { "Unknown version status $status" }
}
}
}
}
private fun CompileContext.allTargets() =
projectDescriptor.buildTargetIndex.allTargets.filterIsInstanceTo<ModuleBuildTarget, MutableSet<ModuleBuildTarget>>(HashSet())
private fun markAllKotlinForRebuild(context: CompileContext, fsOperations: FSOperationsHelper, reason: String) {
LOG.info("Rebuilding all Kotlin: $reason")
val project = context.projectDescriptor.project
val sourceRoots = project.modules.flatMap { it.sourceRoots }
val dataManager = context.projectDescriptor.dataManager
val rebuildAfterCacheVersionChanged = RebuildAfterCacheVersionChangeMarker(dataManager)
for (sourceRoot in sourceRoots) {
val ktFiles = sourceRoot.file.walk().filter { it.isKotlinSourceFile }
fsOperations.markFiles(ktFiles.toList())
}
val kotlinBuildTargets = context.kotlinBuildTargets
for (target in context.allTargets()) {
dataManager.getKotlinCache(kotlinBuildTargets[target])?.clean()
rebuildAfterCacheVersionChanged[target] = true
}
dataManager.cleanLookupStorage(LOG)
}
// todo(1.2.80): got rid of ModuleChunk (replace with KotlinChunk)
// todo(1.2.80): introduce KotlinRoundCompileContext, move dirtyFilesHolder, fsOperations, environment to it
private fun doCompileModuleChunk(
kotlinChunk: KotlinChunk,
chunk: ModuleChunk,
representativeTarget: KotlinModuleBuildTarget<*>,
commonArguments: CommonCompilerArguments,
@@ -512,9 +481,42 @@ class KotlinBuilder : ModuleLevelBuilder(BuilderCategory.SOURCE_PROCESSOR) {
dirtyFilesHolder: KotlinDirtySourceFilesHolder,
fsOperations: FSOperationsHelper,
environment: JpsCompilerEnvironment,
incrementalCaches: Map<ModuleBuildTarget, JpsIncrementalCache>
incrementalCaches: Map<KotlinModuleBuildTarget<*>, JpsIncrementalCache>
): OutputItemsCollector? {
loadPlugins(representativeTarget, commonArguments, context)
kotlinChunk.targets.forEach {
it.nextRound(context)
}
if (representativeTarget.isIncrementalCompilationEnabled) {
for (target in kotlinChunk.targets) {
val cache = incrementalCaches[target]
val jpsTarget = target.jpsModuleBuildTarget
val targetDirtyFiles = dirtyFilesHolder.byTarget[jpsTarget]
if (cache != null && targetDirtyFiles != null) {
val complementaryFiles = cache.clearComplementaryFilesMapping(
targetDirtyFiles.dirty + targetDirtyFiles.removed
)
fsOperations.markFilesForCurrentRound(jpsTarget, complementaryFiles)
cache.markDirty(targetDirtyFiles.dirty + targetDirtyFiles.removed)
}
}
}
val isDoneSomething = representativeTarget.compileModuleChunk(chunk, commonArguments, dirtyFilesHolder, environment)
return if (isDoneSomething) environment.outputItemsCollector else null
}
private fun loadPlugins(
representativeTarget: KotlinModuleBuildTarget<*>,
commonArguments: CommonCompilerArguments,
context: CompileContext
) {
fun concatenate(strings: Array<String>?, cp: List<String>) = arrayOf(*strings.orEmpty(), *cp.toTypedArray())
for (argumentProvider in ServiceLoader.load(KotlinJpsCompilerArgumentsProvider::class.java)) {
@@ -532,29 +534,12 @@ class KotlinBuilder : ModuleLevelBuilder(BuilderCategory.SOURCE_PROCESSOR) {
LOG.debug("Plugin loaded: ${argumentProvider::class.java.simpleName}")
}
if (representativeTarget.isIncrementalCompilationEnabled) {
for (target in chunk.targets) {
val cache = incrementalCaches[target]
val targetDirtyFiles = dirtyFilesHolder.byTarget[target]
if (cache != null && targetDirtyFiles != null) {
val complementaryFiles = cache.clearComplementaryFilesMapping(targetDirtyFiles.dirty + targetDirtyFiles.removed)
fsOperations.markFilesForCurrentRound(target, complementaryFiles)
cache.markDirty(targetDirtyFiles.dirty + targetDirtyFiles.removed)
}
}
}
val isDoneSomething = representativeTarget.compileModuleChunk(chunk, commonArguments, dirtyFilesHolder, environment)
return if (isDoneSomething) environment.outputItemsCollector else null
}
private fun createCompileEnvironment(
context: CompileContext,
kotlinModuleBuilderTarget: KotlinModuleBuildTarget<*>,
incrementalCaches: Map<ModuleBuildTarget, JpsIncrementalCache>,
incrementalCaches: Map<KotlinModuleBuildTarget<*>, JpsIncrementalCache>,
lookupTracker: LookupTracker,
exceptActualTracer: ExpectActualTracker,
chunk: ModuleChunk,
@@ -580,7 +565,7 @@ class KotlinBuilder : ModuleLevelBuilder(BuilderCategory.SOURCE_PROCESSOR) {
classesToLoadByParent,
messageCollector,
OutputItemsCollectorImpl(),
ProgressReporterImpl(kotlinModuleBuilderTarget.context, chunk)
ProgressReporterImpl(context, chunk)
)
}
@@ -622,11 +607,13 @@ class KotlinBuilder : ModuleLevelBuilder(BuilderCategory.SOURCE_PROCESSOR) {
val representativeTarget = chunk.representativeTarget()
fun SimpleOutputItem.target() =
sourceFiles.firstOrNull()?.let { sourceToTarget[it] } ?: chunk.targets.singleOrNull {
it.outputDir?.let {
outputFile.startsWith(it)
} ?: false
} ?: representativeTarget
sourceFiles.firstOrNull()?.let { sourceToTarget[it] }
?: chunk.targets.singleOrNull { target ->
target.outputDir?.let { outputDir ->
outputFile.startsWith(outputDir)
} ?: false
}
?: representativeTarget
return outputItemCollector.outputs.groupBy(SimpleOutputItem::target, SimpleOutputItem::toGeneratedFile)
}
@@ -697,68 +684,4 @@ private fun getLookupTracker(project: JpsProject, representativeTarget: KotlinMo
if (representativeTarget.isIncrementalCompilationEnabled) return LookupTrackerImpl(testLookupTracker)
return testLookupTracker
}
private fun getIncrementalCaches(
chunk: ModuleChunk,
context: CompileContext
): Map<ModuleBuildTarget, JpsIncrementalCache> {
val dataManager = context.projectDescriptor.dataManager
val kotlinBuildTargets = context.kotlinBuildTargets
val chunkCaches = chunk.targets.keysToMapExceptNulls {
dataManager.getKotlinCache(kotlinBuildTargets[it])
}
val dependentTargets = getDependentTargets(chunkCaches.keys, context)
val dependentCaches = dependentTargets.mapNotNull {
dataManager.getKotlinCache(kotlinBuildTargets[it])
}
for (chunkCache in chunkCaches.values) {
for (dependentCache in dependentCaches) {
chunkCache.addJpsDependentCache(dependentCache)
}
}
return chunkCaches
}
fun getDependentTargets(
compilingChunkTargets: Set<ModuleBuildTarget>,
context: CompileContext
): Set<ModuleBuildTarget> {
if (compilingChunkTargets.isEmpty()) return setOf()
val compilingChunkModules: Set<JpsModule> = compilingChunkTargets.mapTo(mutableSetOf()) { it.module }
val compilingChunkIsTests = compilingChunkTargets.any { it.isTests }
val classpathKind = JpsJavaClasspathKind.compile(compilingChunkIsTests)
fun dependsOnCompilingChunk(target: BuildTarget<*>): Boolean {
if (target !is ModuleBuildTarget || compilingChunkIsTests && !target.isTests) return false
val dependencies = getDependenciesRecursively(target.module, classpathKind)
return ContainerUtil.intersects(dependencies, compilingChunkModules)
}
val dependentTargets = HashSet<ModuleBuildTarget>()
val sortedChunks = context.projectDescriptor.buildTargetIndex.getSortedTargetChunks(context).iterator()
// skip chunks that are compiled before compilingChunk
while (sortedChunks.hasNext()) {
if (sortedChunks.next().targets == compilingChunkTargets) break
}
// process chunks that compiled after compilingChunk
for (followingChunk in sortedChunks) {
if (followingChunk.targets.none(::dependsOnCompilingChunk)) continue
dependentTargets.addAll(followingChunk.targets.filterIsInstance<ModuleBuildTarget>())
}
return dependentTargets
}
private fun getDependenciesRecursively(module: JpsModule, kind: JpsJavaClasspathKind): Set<JpsModule> =
JpsJavaExtensionService.dependencies(module).includedIn(kind).recursivelyExportedOnly().modules
}
@@ -0,0 +1,159 @@
/*
* Copyright 2010-2018 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license
* that can be found in the license/LICENSE.txt file.
*/
package org.jetbrains.kotlin.jps.build
import org.jetbrains.jps.incremental.ModuleBuildTarget
import org.jetbrains.kotlin.config.ApiVersion
import org.jetbrains.kotlin.config.LanguageVersion
import org.jetbrains.kotlin.config.VersionView
import org.jetbrains.kotlin.incremental.storage.version.CacheStatus
import org.jetbrains.kotlin.jps.incremental.JpsIncrementalCache
import org.jetbrains.kotlin.jps.incremental.getKotlinCache
import org.jetbrains.kotlin.jps.model.kotlinCompilerArguments
import org.jetbrains.kotlin.jps.targets.KotlinModuleBuildTarget
import org.jetbrains.kotlin.utils.keysToMapExceptNulls
import java.io.File
/**
* Chunk of cyclically dependent [KotlinModuleBuildTarget]s
*/
class KotlinChunk internal constructor(val context: KotlinCompileContext, val targets: List<KotlinModuleBuildTarget<*>>) {
val containsTests = targets.any { it.isTests }
lateinit var dependencies: List<KotlinModuleBuildTarget.Dependency>
// Should be initialized only in KotlinChunk.calculateChunkDependencies
internal set
lateinit var dependent: List<KotlinModuleBuildTarget.Dependency>
// Should be initialized only in KotlinChunk.calculateChunkDependencies
internal set
// used only during dependency calculation
internal var _dependent: MutableSet<KotlinModuleBuildTarget.Dependency>? = mutableSetOf()
val representativeTarget
get() = targets.first()
private val presentableModulesToCompilersList: String
get() = targets.joinToString { "${it.module.name} (${it.globalLookupCacheId})" }
init {
check(targets.all { it.javaClass == representativeTarget.javaClass }) {
"Cyclically dependent modules $presentableModulesToCompilersList should have same compiler."
}
}
val compilerArguments = representativeTarget.jpsModuleBuildTarget.module.kotlinCompilerArguments
val langVersion =
compilerArguments.languageVersion?.let { LanguageVersion.fromVersionString(it) } ?: VersionView.RELEASED_VERSION
val apiVersion =
compilerArguments.apiVersion?.let { ApiVersion.parse(it) } ?: ApiVersion.createByLanguageVersion(
langVersion
)
fun shouldRebuild(): Boolean {
val buildMetaInfo = representativeTarget.buildMetaInfoFactory.create(compilerArguments)
targets.forEach { target ->
if (target.isVersionChanged(this, buildMetaInfo)) {
KotlinBuilder.LOG.info("$target version changed, rebuilding $this")
return true
}
if (target.initialLocalCacheAttributesDiff.status == CacheStatus.INVALID) {
context.testingLogger?.invalidOrUnusedCache(this, null, target.initialLocalCacheAttributesDiff)
KotlinBuilder.LOG.info("$target cache is invalid ${target.initialLocalCacheAttributesDiff}, rebuilding $this")
return true
}
}
return false
}
fun buildMetaInfoFile(target: ModuleBuildTarget): File =
File(
context.dataPaths.getTargetDataRoot(target),
representativeTarget.buildMetaInfoFileName
)
fun saveVersions() {
context.ensureLookupsCacheAttributesSaved()
targets.forEach {
it.initialLocalCacheAttributesDiff.saveExpectedIfNeeded()
}
val serializedMetaInfo = representativeTarget.buildMetaInfoFactory.serializeToString(compilerArguments)
targets.forEach {
buildMetaInfoFile(it.jpsModuleBuildTarget).writeText(serializedMetaInfo)
}
}
fun collectDependentChunksRecursivelyExportedOnly(result: MutableSet<KotlinChunk> = mutableSetOf()) {
dependent.forEach {
if (result.add(it.src.chunk)) {
if (it.exported) {
it.src.chunk.collectDependentChunksRecursivelyExportedOnly(result)
}
}
}
}
fun loadCaches(loadDependent: Boolean = true): Map<KotlinModuleBuildTarget<*>, JpsIncrementalCache> {
val dataManager = context.dataManager
val cacheByChunkTarget = targets.keysToMapExceptNulls {
dataManager.getKotlinCache(it)
}
if (loadDependent) {
addDependentCaches(cacheByChunkTarget.values)
}
return cacheByChunkTarget
}
private fun addDependentCaches(targetsCaches: Collection<JpsIncrementalCache>) {
val dependentChunks = mutableSetOf<KotlinChunk>()
collectDependentChunksRecursivelyExportedOnly(dependentChunks)
val dataManager = context.dataManager
dependentChunks.forEach { decedentChunk ->
decedentChunk.targets.forEach {
val dependentCache = dataManager.getKotlinCache(it)
if (dependentCache != null) {
for (chunkCache in targetsCaches) {
chunkCache.addJpsDependentCache(dependentCache)
}
}
}
}
}
/**
* The same as [org.jetbrains.jps.ModuleChunk.getPresentableShortName]
*/
val presentableShortName: String
get() = buildString {
if (containsTests) append("tests of ")
append(targets.first().module.name)
if (targets.size > 1) {
val andXMore = "and ${targets.size - 1} more"
val other = ", " + targets.asSequence().drop(1).joinToString()
append(if (other.length < andXMore.length) other else andXMore)
}
}
override fun toString(): String {
return "KotlinChunk<${representativeTarget.javaClass.simpleName}>" +
"(${targets.joinToString { it.jpsModuleBuildTarget.presentableName }})"
}
}
@@ -0,0 +1,240 @@
/*
* Copyright 2010-2018 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license
* that can be found in the license/LICENSE.txt file.
*/
package org.jetbrains.kotlin.jps.build
import org.jetbrains.jps.ModuleChunk
import org.jetbrains.jps.incremental.CompileContext
import org.jetbrains.jps.incremental.FSOperations
import org.jetbrains.jps.incremental.GlobalContextKey
import org.jetbrains.jps.incremental.fs.CompilationRound
import org.jetbrains.kotlin.incremental.LookupSymbol
import org.jetbrains.kotlin.incremental.storage.version.CacheAttributesDiff
import org.jetbrains.kotlin.incremental.storage.version.CacheStatus
import org.jetbrains.kotlin.incremental.storage.version.loadDiff
import org.jetbrains.kotlin.jps.incremental.*
import org.jetbrains.kotlin.jps.targets.KotlinTargetsIndex
import org.jetbrains.kotlin.jps.targets.KotlinTargetsIndexBuilder
import java.io.File
import java.util.concurrent.atomic.AtomicBoolean
/**
* KotlinCompileContext is shared between all threads (i.e. it is [GlobalContextKey]).
*
* It is initialized lazily, and only before building of first chunk with kotlin code,
* and will be disposed on build finish.
*/
internal val CompileContext.kotlin: KotlinCompileContext
get() {
val userData = getUserData(kotlinCompileContextKey)
if (userData != null) return userData
// here is error (KotlinCompilation available only at build phase)
// let's also check for concurrent initialization
val errorMessage = "KotlinCompileContext available only at build phase " +
"(between first KotlinBuilder.chunkBuildStarted and KotlinBuilder.buildFinished)"
synchronized(kotlinCompileContextKey) {
val newUsedData = getUserData(kotlinCompileContextKey)
if (newUsedData != null) {
error("Concurrent CompileContext.kotlin getter call and KotlinCompileContext initialization detected: $errorMessage")
}
}
error(errorMessage)
}
internal val kotlinCompileContextKey = GlobalContextKey<KotlinCompileContext>("kotlin")
class KotlinCompileContext(val jpsContext: CompileContext) {
val dataManager = jpsContext.projectDescriptor.dataManager
val dataPaths = dataManager.dataPaths
val testingLogger: TestingBuildLogger?
get() = jpsContext.testingContext?.buildLogger
val targetsIndex: KotlinTargetsIndex = KotlinTargetsIndexBuilder(this).build()
val targetsBinding
get() = targetsIndex.byJpsTarget
val lookupsCacheAttributesManager: CompositeLookupsCacheAttributesManager = makeLookupsCacheAttributesManager()
val initialLookupsCacheStateDiff: CacheAttributesDiff<*> = loadLookupsCacheStateDiff()
val shouldCheckCacheVersions = System.getProperty(KotlinBuilder.SKIP_CACHE_VERSION_CHECK_PROPERTY) == null
val hasKotlinMarker = HasKotlinMarker(dataManager)
/**
* Flag to prevent rebuilding twice.
*
* TODO: looks like it is not required since cache version checking are refactored
*/
val rebuildAfterCacheVersionChanged = RebuildAfterCacheVersionChangeMarker(dataManager)
var rebuildingAllKotlin = false
private fun makeLookupsCacheAttributesManager(): CompositeLookupsCacheAttributesManager {
val expectedLookupsCacheComponents = mutableSetOf<String>()
targetsIndex.chunks.forEach { chunk ->
chunk.targets.forEach { target ->
if (target.isIncrementalCompilationEnabled) {
expectedLookupsCacheComponents.add(target.globalLookupCacheId)
}
}
}
val lookupsCacheRootPath = dataPaths.getTargetDataRoot(KotlinDataContainerTarget)
return CompositeLookupsCacheAttributesManager(lookupsCacheRootPath, expectedLookupsCacheComponents)
}
private fun loadLookupsCacheStateDiff(): CacheAttributesDiff<CompositeLookupsCacheAttributes> {
val diff = lookupsCacheAttributesManager.loadDiff()
if (diff.status == CacheStatus.VALID) {
// try to perform a lookup
// request rebuild if storage is corrupted
try {
dataManager.withLookupStorage {
it.get(LookupSymbol("<#NAME#>", "<#SCOPE#>"))
}
} catch (e: Exception) {
jpsReportInternalBuilderError(jpsContext, Error("Lookup storage is corrupted, probe failed: ${e.message}", e))
markAllKotlinForRebuild("Lookup storage is corrupted")
return diff.copy(actual = null)
}
}
return diff
}
fun hasKotlin() = targetsIndex.chunks.any { chunk ->
chunk.targets.any { target ->
hasKotlinMarker[target] == true
}
}
fun checkCacheVersions() {
when (initialLookupsCacheStateDiff.status) {
CacheStatus.INVALID -> {
// global cache needs to be rebuilt
testingLogger?.invalidOrUnusedCache(null, null, initialLookupsCacheStateDiff)
if (initialLookupsCacheStateDiff.actual != null) {
markAllKotlinForRebuild("Kotlin incremental cache settings or format was changed")
clearLookupCache()
} else {
markAllKotlinForRebuild("Kotlin incremental cache is missed or corrupted")
}
}
CacheStatus.VALID -> Unit
CacheStatus.SHOULD_BE_CLEARED -> {
jpsContext.testingContext?.buildLogger?.invalidOrUnusedCache(null, null, initialLookupsCacheStateDiff)
KotlinBuilder.LOG.info("Removing global cache as it is not required anymore: $initialLookupsCacheStateDiff")
clearAllCaches()
}
CacheStatus.CLEARED -> Unit
}
}
private val lookupAttributesSaved = AtomicBoolean(false)
/**
* Called on every successful compilation
*/
fun ensureLookupsCacheAttributesSaved() {
if (lookupAttributesSaved.compareAndSet(false, true)) {
initialLookupsCacheStateDiff.saveExpectedIfNeeded()
}
}
fun checkChunkCacheVersion(chunk: KotlinChunk) {
if (shouldCheckCacheVersions && !rebuildingAllKotlin) {
if (chunk.shouldRebuild()) markChunkForRebuildBeforeBuild(chunk)
}
}
private fun logMarkDirtyForTestingBeforeRound(file: File, shouldProcess: Boolean): Boolean {
if (shouldProcess) {
testingLogger?.markedAsDirtyBeforeRound(listOf(file))
}
return shouldProcess
}
private fun markAllKotlinForRebuild(reason: String) {
if (rebuildingAllKotlin) return
rebuildingAllKotlin = true
KotlinBuilder.LOG.info("Rebuilding all Kotlin: $reason")
val dataManager = jpsContext.projectDescriptor.dataManager
targetsIndex.chunks.forEach {
markChunkForRebuildBeforeBuild(it)
}
dataManager.cleanLookupStorage(KotlinBuilder.LOG)
}
private fun markChunkForRebuildBeforeBuild(chunk: KotlinChunk) {
chunk.targets.forEach {
FSOperations.markDirty(jpsContext, CompilationRound.NEXT, it.jpsModuleBuildTarget) { file ->
logMarkDirtyForTestingBeforeRound(file, file.isKotlinSourceFile)
}
dataManager.getKotlinCache(it)?.clean()
hasKotlinMarker.clean(it)
rebuildAfterCacheVersionChanged[it] = true
}
}
private fun clearAllCaches() {
clearLookupCache()
KotlinBuilder.LOG.info("Clearing caches for all targets")
targetsIndex.chunks.forEach { chunk ->
chunk.targets.forEach {
dataManager.getKotlinCache(it)?.clean()
}
}
}
private fun clearLookupCache() {
KotlinBuilder.LOG.info("Clearing lookup cache")
dataManager.cleanLookupStorage(KotlinBuilder.LOG)
initialLookupsCacheStateDiff.saveExpectedIfNeeded()
}
fun cleanupCaches() {
// todo: remove lookups for targets with disabled IC (or split global lookups cache into several caches for each compiler)
targetsIndex.chunks.forEach { chunk ->
chunk.targets.forEach { target ->
if (target.initialLocalCacheAttributesDiff.status == CacheStatus.SHOULD_BE_CLEARED) {
KotlinBuilder.LOG.info(
"$target caches is cleared as not required anymore: ${target.initialLocalCacheAttributesDiff}"
)
testingLogger?.invalidOrUnusedCache(null, target, target.initialLocalCacheAttributesDiff)
dataManager.getKotlinCache(target)?.clean()
}
}
}
}
fun dispose() {
}
fun getChunk(rawChunk: ModuleChunk): KotlinChunk? {
val rawRepresentativeTarget = rawChunk.representativeTarget()
if (rawRepresentativeTarget !in targetsBinding) return null
return targetsIndex.chunksByJpsRepresentativeTarget[rawRepresentativeTarget]
?: error("Kotlin binding for chunk $this is not loaded at build start")
}
}
@@ -48,7 +48,7 @@ class KotlinDirtySourceFilesHolder(
* and during KotlinDirtySourceFilesHolder initialization.
*/
internal fun _markDirty(file: File) {
_dirty.add(file)
_dirty.add(file.canonicalFile)
}
}
@@ -55,17 +55,12 @@ class KotlinSourceRootProvider : AdditionalRootsProviderService<JavaSourceRootDe
// new multiplatform model support:
module.sourceSetModules.forEach { sourceSetModule ->
addModuleSourceRoots(result, sourceSetModule, target, false)
addModuleSourceRoots(result, sourceSetModule, target)
}
// legacy multiplatform model support:
module.expectedByModules.forEach { commonModule ->
// At the moment, incremental compilation is not supported by K2Metadata compiler.
// To avoid long running compilation of common modules, we do not run K2Metadata at all:
// instead all the common source roots are transitively added to the final platform modules.
val isRecursive = true
addModuleSourceRoots(result, commonModule, target, isRecursive)
addModuleSourceRoots(result, commonModule, target)
}
return result
@@ -74,8 +69,7 @@ class KotlinSourceRootProvider : AdditionalRootsProviderService<JavaSourceRootDe
private fun addModuleSourceRoots(
result: MutableList<JavaSourceRootDescriptor>,
module: JpsModule,
target: ModuleBuildTarget,
recursive: Boolean = false
target: ModuleBuildTarget
) {
for (commonSourceRoot in module.sourceRoots) {
val isCommonTestsRootType = commonSourceRoot.rootType.isTestsRootType
@@ -94,21 +88,6 @@ class KotlinSourceRootProvider : AdditionalRootsProviderService<JavaSourceRootDe
)
}
}
if (recursive) {
JpsJavaExtensionService.dependencies(module)
.also {
if (!target.isTests) it.productionOnly()
}
.processModules {
addModuleSourceRoots(
result,
it,
ModuleBuildTarget(it, target.targetType as JavaModuleBuildTargetType),
true
)
}
}
}
}
@@ -20,12 +20,16 @@ import org.jetbrains.jps.builders.storage.BuildDataPaths
import org.jetbrains.jps.incremental.ModuleBuildTarget
import org.jetbrains.jps.incremental.storage.BuildDataManager
import org.jetbrains.kotlin.incremental.KOTLIN_CACHE_DIRECTORY_NAME
import org.jetbrains.kotlin.jps.targets.KotlinModuleBuildTarget
import java.io.File
private val HAS_KOTLIN_MARKER_FILE_NAME = "has-kotlin-marker.txt"
private val REBUILD_AFTER_CACHE_VERSION_CHANGE_MARKER = "rebuild-after-cache-version-change-marker.txt"
abstract class MarkerFile(private val fileName: String, private val paths: BuildDataPaths) {
operator fun get(target: KotlinModuleBuildTarget<*>): Boolean? =
get(target.jpsModuleBuildTarget)
operator fun get(target: ModuleBuildTarget): Boolean? {
val file = target.markerFile
@@ -34,6 +38,9 @@ abstract class MarkerFile(private val fileName: String, private val paths: Build
return file.readText().toBoolean()
}
operator fun set(target: KotlinModuleBuildTarget<*>, value: Boolean) =
set(target.jpsModuleBuildTarget, value)
operator fun set(target: ModuleBuildTarget, value: Boolean) {
val file = target.markerFile
@@ -45,6 +52,9 @@ abstract class MarkerFile(private val fileName: String, private val paths: Build
file.writeText(value.toString())
}
fun clean(target: KotlinModuleBuildTarget<*>) =
clean(target.jpsModuleBuildTarget)
fun clean(target: ModuleBuildTarget) {
target.markerFile.delete()
}
@@ -12,7 +12,7 @@ import org.jetbrains.kotlin.cli.common.messages.CompilerMessageLocation
import org.jetbrains.kotlin.cli.common.messages.CompilerMessageSeverity
import org.jetbrains.kotlin.cli.common.messages.MessageCollector
import org.jetbrains.kotlin.config.CompilerRunnerConstants
import org.jetbrains.kotlin.jps.platforms.KotlinModuleBuildTarget
import org.jetbrains.kotlin.jps.targets.KotlinModuleBuildTarget
import java.io.File
class MessageCollectorAdapter(
@@ -16,16 +16,19 @@
package org.jetbrains.kotlin.jps.build
import org.jetbrains.jps.ModuleChunk
import org.jetbrains.jps.incremental.CompileContext
import org.jetbrains.jps.incremental.ModuleLevelBuilder
import org.jetbrains.kotlin.incremental.CacheVersion
import org.jetbrains.kotlin.incremental.storage.version.CacheAttributesDiff
import org.jetbrains.kotlin.jps.targets.KotlinModuleBuildTarget
import java.io.File
interface BuildLogger {
fun actionsOnCacheVersionChanged(actions: List<CacheVersion.Action>)
fun buildStarted(context: CompileContext, chunk: ModuleChunk)
fun afterBuildStarted(context: CompileContext, chunk: ModuleChunk)
/**
* Used for assertions in tests.
*/
interface TestingBuildLogger {
fun invalidOrUnusedCache(chunk: KotlinChunk?, target: KotlinModuleBuildTarget<*>?, attributesDiff: CacheAttributesDiff<*>)
fun chunkBuildStarted(context: CompileContext, chunk: org.jetbrains.jps.ModuleChunk)
fun afterChunkBuildStarted(context: CompileContext, chunk: org.jetbrains.jps.ModuleChunk)
fun buildFinished(exitCode: ModuleLevelBuilder.ExitCode)
fun markedAsDirtyBeforeRound(files: Iterable<File>)
fun markedAsDirtyAfterRound(files: Iterable<File>)
@@ -24,7 +24,7 @@ import org.jetbrains.jps.model.JpsSimpleElement
import org.jetbrains.jps.model.ex.JpsElementChildRoleBase
import org.jetbrains.kotlin.incremental.components.LookupTracker
private val TESTING_CONTEXT = JpsElementChildRoleBase.create<JpsSimpleElement<out TestingContext>>("Testing context")
private val TESTING_CONTEXT = JpsElementChildRoleBase.create<JpsSimpleElement<out TestingContext>>("Testing kcontext")
@TestOnly
fun JpsProject.setTestingContext(context: TestingContext) {
@@ -40,5 +40,7 @@ val CompileContext.testingContext: TestingContext?
class TestingContext(
val lookupTracker: LookupTracker,
val buildLogger: BuildLogger
)
val buildLogger: TestingBuildLogger
) {
var kotlinCompileContext: KotlinCompileContext? = null
}
@@ -0,0 +1,14 @@
/*
* Copyright 2010-2018 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license
* that can be found in the license/LICENSE.txt file.
*/
package org.jetbrains.kotlin.jps.build
import org.jetbrains.jps.incremental.CompileContext
import org.jetbrains.jps.incremental.messages.CompilerMessage
fun jpsReportInternalBuilderError(context: CompileContext, error: Throwable) {
val builderError = CompilerMessage.createInternalBuilderError("Kotlin", error)
context.processMessage(builderError)
}
@@ -0,0 +1,13 @@
/*
* Copyright 2010-2018 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license
* that can be found in the license/LICENSE.txt file.
*/
package org.jetbrains.kotlin.jps.build
import org.jetbrains.jps.incremental.CompileContext
import org.jetbrains.jps.incremental.messages.CompilerMessage
fun jpsReportInternalBuilderError(context: CompileContext, error: Throwable) {
KotlinBuilder.LOG.info(error)
}
@@ -0,0 +1,13 @@
/*
* Copyright 2010-2018 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license
* that can be found in the license/LICENSE.txt file.
*/
package org.jetbrains.kotlin.jps.build
import org.jetbrains.jps.incremental.CompileContext
import org.jetbrains.jps.incremental.messages.CompilerMessage
fun jpsReportInternalBuilderError(context: CompileContext, error: Throwable) {
KotlinBuilder.LOG.info(error)
}
@@ -0,0 +1,13 @@
/*
* Copyright 2010-2018 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license
* that can be found in the license/LICENSE.txt file.
*/
package org.jetbrains.kotlin.jps.build
import org.jetbrains.jps.incremental.CompileContext
import org.jetbrains.jps.incremental.messages.CompilerMessage
fun jpsReportInternalBuilderError(context: CompileContext, error: Throwable) {
KotlinBuilder.LOG.info(error)
}
@@ -0,0 +1,13 @@
/*
* Copyright 2010-2018 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license
* that can be found in the license/LICENSE.txt file.
*/
package org.jetbrains.kotlin.jps.build
import org.jetbrains.jps.incremental.CompileContext
import org.jetbrains.jps.incremental.messages.CompilerMessage
fun jpsReportInternalBuilderError(context: CompileContext, error: Throwable) {
KotlinBuilder.LOG.info(error)
}
@@ -0,0 +1,13 @@
/*
* Copyright 2010-2018 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license
* that can be found in the license/LICENSE.txt file.
*/
package org.jetbrains.kotlin.jps.build
import org.jetbrains.jps.incremental.CompileContext
import org.jetbrains.jps.incremental.messages.CompilerMessage
fun jpsReportInternalBuilderError(context: CompileContext, error: Throwable) {
KotlinBuilder.LOG.info(error)
}
@@ -17,9 +17,31 @@
package org.jetbrains.kotlin.jps.build
import org.jetbrains.jps.ModuleChunk
import org.jetbrains.jps.builders.java.JavaModuleBuildTargetType
import org.jetbrains.jps.incremental.CompileContext
import org.jetbrains.jps.incremental.ModuleBuildTarget
import org.jetbrains.jps.model.module.JpsModule
fun ModuleChunk.isDummy(context: CompileContext): Boolean {
val targetIndex = context.projectDescriptor.buildTargetIndex
return targets.all { targetIndex.isDummy(it) }
}
}
@Deprecated("Use `kotlin.targetBinding` instead", ReplaceWith("kotlin.targetsBinding"))
val CompileContext.kotlinBuildTargets
get() = kotlin.targetsBinding
fun ModuleChunk.toKotlinChunk(context: CompileContext): KotlinChunk? =
context.kotlin.getChunk(this)
fun ModuleBuildTarget(module: JpsModule, isTests: Boolean) =
ModuleBuildTarget(
module,
if (isTests) JavaModuleBuildTargetType.TEST else JavaModuleBuildTargetType.PRODUCTION
)
val JpsModule.productionBuildTarget
get() = ModuleBuildTarget(this, false)
val JpsModule.testBuildTarget
get() = ModuleBuildTarget(this, true)
@@ -1,49 +0,0 @@
/*
* Copyright 2010-2015 JetBrains s.r.o.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.jetbrains.kotlin.jps.incremental
import org.jetbrains.jps.builders.BuildTarget
import org.jetbrains.jps.builders.storage.BuildDataPaths
import org.jetbrains.jps.incremental.ModuleBuildTarget
import org.jetbrains.kotlin.incremental.CacheVersion
import org.jetbrains.kotlin.incremental.dataContainerCacheVersion
import org.jetbrains.kotlin.incremental.normalCacheVersion
import java.io.File
class CacheVersionProvider(
private val paths: BuildDataPaths,
private val isIncrementalCompilationEnabled: Boolean
) {
private val BuildTarget<*>.dataRoot: File
get() = paths.getTargetDataRoot(this)
fun normalVersion(target: ModuleBuildTarget): CacheVersion = normalCacheVersion(target.dataRoot, isIncrementalCompilationEnabled)
fun dataContainerVersion(): CacheVersion = dataContainerCacheVersion(KotlinDataContainerTarget.dataRoot, isIncrementalCompilationEnabled)
fun allVersions(targets: Iterable<ModuleBuildTarget>): Iterable<CacheVersion> {
val versions = arrayListOf<CacheVersion>()
versions.add(dataContainerCacheVersion(KotlinDataContainerTarget.dataRoot, isIncrementalCompilationEnabled))
for (dataRoot in targets.map { it.dataRoot }) {
versions.add(normalCacheVersion(dataRoot, isIncrementalCompilationEnabled))
}
return versions
}
}
@@ -0,0 +1,78 @@
/*
* Copyright 2010-2018 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license
* that can be found in the license/LICENSE.txt file.
*/
package org.jetbrains.kotlin.jps.incremental
import org.jetbrains.annotations.TestOnly
import org.jetbrains.kotlin.incremental.storage.version.CacheAttributesManager
import org.jetbrains.kotlin.incremental.storage.version.CacheVersion
import org.jetbrains.kotlin.incremental.storage.version.lookupsCacheVersionManager
import java.io.File
import java.io.IOException
/**
* Attributes manager for global lookups cache that may contain lookups for several compilers (jvm, js).
* Works by delegating to [lookupsCacheVersionManager] and managing additional file with list of executed compilers (cache components).
*
* TODO(1.2.80): got rid of shared lookup cache, replace with individual lookup cache for each compiler
*/
class CompositeLookupsCacheAttributesManager(
rootPath: File,
expectedComponents: Set<String>
) : CacheAttributesManager<CompositeLookupsCacheAttributes> {
private val versionManager = lookupsCacheVersionManager(
rootPath,
expectedComponents.isNotEmpty()
)
private val actualComponentsFile = File(rootPath, "components.txt")
override val expected: CompositeLookupsCacheAttributes? =
if (expectedComponents.isEmpty()) null
else CompositeLookupsCacheAttributes(versionManager.expected!!.version, expectedComponents)
override fun loadActual(): CompositeLookupsCacheAttributes? {
val version = versionManager.loadActual() ?: return null
if (!actualComponentsFile.exists()) return null
val components = try {
actualComponentsFile.readLines().toSet()
} catch (e: IOException) {
return null
}
return CompositeLookupsCacheAttributes(version.version, components)
}
override fun writeActualVersion(values: CompositeLookupsCacheAttributes?) {
if (values == null) {
versionManager.writeActualVersion(null)
actualComponentsFile.delete()
} else {
versionManager.writeActualVersion(CacheVersion(values.version))
actualComponentsFile.parentFile.mkdirs()
actualComponentsFile.writeText(values.components.joinToString("\n"))
}
}
override fun isCompatible(actual: CompositeLookupsCacheAttributes, expected: CompositeLookupsCacheAttributes): Boolean {
// cache can be reused when all required (expected) components are present
// (components that are not required anymore are not not interfere)
return actual.version == expected.version && actual.components.containsAll(expected.components)
}
@get:TestOnly
val versionManagerForTesting
get() = versionManager
}
data class CompositeLookupsCacheAttributes(
val version: Int,
val components: Set<String>
) {
override fun toString() = "($version, $components)"
}
@@ -25,7 +25,7 @@ import org.jetbrains.kotlin.incremental.IncrementalCacheCommon
import org.jetbrains.kotlin.incremental.IncrementalJsCache
import org.jetbrains.kotlin.incremental.IncrementalJvmCache
import org.jetbrains.kotlin.jps.build.KotlinBuilder
import org.jetbrains.kotlin.jps.platforms.KotlinModuleBuildTarget
import org.jetbrains.kotlin.jps.targets.KotlinModuleBuildTarget
import java.io.File
interface JpsIncrementalCache : IncrementalCacheCommon, StorageOwner {
@@ -1,99 +0,0 @@
/*
* Copyright 2010-2018 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license
* that can be found in the license/LICENSE.txt file.
*/
package org.jetbrains.kotlin.jps.platforms
import com.intellij.openapi.util.Key
import org.jetbrains.jps.builders.ModuleBasedBuildTargetType
import org.jetbrains.jps.builders.java.JavaModuleBuildTargetType
import org.jetbrains.jps.incremental.CompileContext
import org.jetbrains.jps.incremental.ModuleBuildTarget
import org.jetbrains.jps.model.library.JpsOrderRootType
import org.jetbrains.jps.model.module.JpsModule
import org.jetbrains.jps.util.JpsPathUtil
import org.jetbrains.kotlin.jps.model.platform
import org.jetbrains.kotlin.platform.DefaultIdeTargetPlatformKindProvider
import org.jetbrains.kotlin.platform.IdePlatformKind
import org.jetbrains.kotlin.platform.impl.*
import org.jetbrains.kotlin.utils.LibraryUtils
import java.util.concurrent.ConcurrentHashMap
fun ModuleBuildTarget(module: JpsModule, isTests: Boolean) =
ModuleBuildTarget(module, if (isTests) JavaModuleBuildTargetType.TEST else JavaModuleBuildTargetType.PRODUCTION)
val JpsModule.productionBuildTarget
get() = ModuleBuildTarget(this, false)
private val kotlinBuildTargetsCompileContextKey = Key<KotlinBuildTargets>("kotlinBuildTargets")
val CompileContext.kotlinBuildTargets: KotlinBuildTargets
get() {
val value = getUserData(kotlinBuildTargetsCompileContextKey)
if (value != null) return value
synchronized(this) {
val actualValue = getUserData(kotlinBuildTargetsCompileContextKey)
if (actualValue != null) return actualValue
val newValue = KotlinBuildTargets(this)
putUserData(kotlinBuildTargetsCompileContextKey, newValue)
return newValue
}
}
class KotlinBuildTargets internal constructor(val compileContext: CompileContext) {
private val byJpsModuleBuildTarget = ConcurrentHashMap<ModuleBuildTarget, KotlinModuleBuildTarget<*>>()
private val isKotlinJsStdlibJar = ConcurrentHashMap<String, Boolean>()
@JvmName("getNullable")
operator fun get(target: ModuleBuildTarget?): KotlinModuleBuildTarget<*>? {
if (target == null) return null
return get(target)
}
operator fun get(target: ModuleBuildTarget): KotlinModuleBuildTarget<*>? {
if (target.targetType !is ModuleBasedBuildTargetType) return null
return byJpsModuleBuildTarget.computeIfAbsent(target) {
val platform = target.module.platform?.kind ?: detectTargetPlatform(target)
when {
platform.isCommon -> KotlinCommonModuleBuildTarget(compileContext, target)
platform.isJavaScript -> KotlinJsModuleBuildTarget(compileContext, target)
platform.isJvm -> KotlinJvmModuleBuildTarget(compileContext, target)
else -> error("Invalid platform $platform")
}
}
}
/**
* Compatibility for KT-14082
* todo: remove when all projects migrated to facets
*/
private fun detectTargetPlatform(target: ModuleBuildTarget): IdePlatformKind<*> {
if (hasJsStdLib(target)) {
return JsIdePlatformKind
}
return JvmIdePlatformKind
return DefaultIdeTargetPlatformKindProvider.defaultPlatform.kind
}
private fun hasJsStdLib(target: ModuleBuildTarget): Boolean {
KotlinJvmModuleBuildTarget(compileContext, target).allDependencies.libraries.forEach { library ->
for (root in library.getRoots(JpsOrderRootType.COMPILED)) {
val url = root.url
val isKotlinJsLib = isKotlinJsStdlibJar.computeIfAbsent(url) {
LibraryUtils.isKotlinJavascriptStdLibrary(JpsPathUtil.urlToFile(url))
}
if (isKotlinJsLib) return true
}
}
return false
}
}
@@ -3,11 +3,10 @@
* that can be found in the license/LICENSE.txt file.
*/
package org.jetbrains.kotlin.jps.platforms
package org.jetbrains.kotlin.jps.targets
import org.jetbrains.jps.ModuleChunk
import org.jetbrains.jps.builders.storage.BuildDataPaths
import org.jetbrains.jps.incremental.CompileContext
import org.jetbrains.jps.incremental.ModuleBuildTarget
import org.jetbrains.jps.model.library.JpsOrderRootType
import org.jetbrains.jps.model.module.JpsModule
@@ -17,15 +16,16 @@ import org.jetbrains.kotlin.cli.common.arguments.CommonCompilerArguments
import org.jetbrains.kotlin.cli.common.arguments.K2MetadataCompilerArguments
import org.jetbrains.kotlin.compilerRunner.JpsCompilerEnvironment
import org.jetbrains.kotlin.compilerRunner.JpsKotlinCompilerRunner
import org.jetbrains.kotlin.config.IncrementalCompilation
import org.jetbrains.kotlin.jps.build.KotlinCompileContext
import org.jetbrains.kotlin.jps.build.KotlinDirtySourceFilesHolder
import org.jetbrains.kotlin.jps.build.ModuleBuildTarget
import org.jetbrains.kotlin.jps.model.k2MetadataCompilerArguments
import org.jetbrains.kotlin.jps.model.kotlinCompilerSettings
private const val COMMON_BUILD_META_INFO_FILE_NAME = "common-build-meta-info.txt"
class KotlinCommonModuleBuildTarget(context: CompileContext, jpsModuleBuildTarget: ModuleBuildTarget) :
KotlinModuleBuildTarget<CommonBuildMetaInfo>(context, jpsModuleBuildTarget) {
class KotlinCommonModuleBuildTarget(kotlinContext: KotlinCompileContext, jpsModuleBuildTarget: ModuleBuildTarget) :
KotlinModuleBuildTarget<CommonBuildMetaInfo>(kotlinContext, jpsModuleBuildTarget) {
override val isIncrementalCompilationEnabled: Boolean
get() = false
@@ -36,6 +36,9 @@ class KotlinCommonModuleBuildTarget(context: CompileContext, jpsModuleBuildTarge
override val buildMetaInfoFileName
get() = COMMON_BUILD_META_INFO_FILE_NAME
override val globalLookupCacheId: String
get() = "metadata-compiler"
override fun compileModuleChunk(
chunk: ModuleChunk,
commonArguments: CommonCompilerArguments,
@@ -87,7 +90,7 @@ class KotlinCommonModuleBuildTarget(context: CompileContext, jpsModuleBuildTarge
result: MutableList<String>,
isTests: Boolean
) {
val dependencyBuildTarget = context.kotlinBuildTargets[ModuleBuildTarget(module, isTests)]
val dependencyBuildTarget = kotlinContext.targetsBinding[ModuleBuildTarget(module, isTests)]
if (dependencyBuildTarget != this@KotlinCommonModuleBuildTarget &&
dependencyBuildTarget is KotlinCommonModuleBuildTarget &&
@@ -3,11 +3,10 @@
* that can be found in the license/LICENSE.txt file.
*/
package org.jetbrains.kotlin.jps.platforms
package org.jetbrains.kotlin.jps.targets
import org.jetbrains.jps.ModuleChunk
import org.jetbrains.jps.builders.storage.BuildDataPaths
import org.jetbrains.jps.incremental.CompileContext
import org.jetbrains.jps.incremental.ModuleBuildTarget
import org.jetbrains.jps.model.library.JpsOrderRootType
import org.jetbrains.jps.model.module.JpsModule
@@ -27,7 +26,9 @@ import org.jetbrains.kotlin.incremental.js.IncrementalDataProvider
import org.jetbrains.kotlin.incremental.js.IncrementalDataProviderFromCache
import org.jetbrains.kotlin.incremental.js.IncrementalResultsConsumer
import org.jetbrains.kotlin.incremental.js.IncrementalResultsConsumerImpl
import org.jetbrains.kotlin.jps.build.KotlinCompileContext
import org.jetbrains.kotlin.jps.build.KotlinDirtySourceFilesHolder
import org.jetbrains.kotlin.jps.build.ModuleBuildTarget
import org.jetbrains.kotlin.jps.incremental.JpsIncrementalCache
import org.jetbrains.kotlin.jps.incremental.JpsIncrementalJsCache
import org.jetbrains.kotlin.jps.model.k2JsCompilerArguments
@@ -42,8 +43,10 @@ import java.net.URI
private const val JS_BUILD_META_INFO_FILE_NAME = "js-build-meta-info.txt"
class KotlinJsModuleBuildTarget(compileContext: CompileContext, jpsModuleBuildTarget: ModuleBuildTarget) :
KotlinModuleBuildTarget<JsBuildMetaInfo>(compileContext, jpsModuleBuildTarget) {
class KotlinJsModuleBuildTarget(kotlinContext: KotlinCompileContext, jpsModuleBuildTarget: ModuleBuildTarget) :
KotlinModuleBuildTarget<JsBuildMetaInfo>(kotlinContext, jpsModuleBuildTarget) {
override val globalLookupCacheId: String
get() = "js"
override val isIncrementalCompilationEnabled: Boolean
get() = IncrementalCompilation.isEnabledForJs()
@@ -56,13 +59,13 @@ class KotlinJsModuleBuildTarget(compileContext: CompileContext, jpsModuleBuildTa
val isFirstBuild: Boolean
get() {
val targetDataRoot = context.projectDescriptor.dataManager.dataPaths.getTargetDataRoot(jpsModuleBuildTarget)
val targetDataRoot = jpsGlobalContext.projectDescriptor.dataManager.dataPaths.getTargetDataRoot(jpsModuleBuildTarget)
return !IncrementalJsCache.hasHeaderFile(targetDataRoot)
}
override fun makeServices(
builder: Services.Builder,
incrementalCaches: Map<ModuleBuildTarget, JpsIncrementalCache>,
incrementalCaches: Map<KotlinModuleBuildTarget<*>, JpsIncrementalCache>,
lookupTracker: LookupTracker,
exceptActualTracer: ExpectActualTracker
) {
@@ -72,7 +75,7 @@ class KotlinJsModuleBuildTarget(compileContext: CompileContext, jpsModuleBuildTa
register(IncrementalResultsConsumer::class.java, IncrementalResultsConsumerImpl())
if (isIncrementalCompilationEnabled && !isFirstBuild) {
val cache = incrementalCaches[jpsModuleBuildTarget] as IncrementalJsCache
val cache = incrementalCaches[this@KotlinJsModuleBuildTarget] as IncrementalJsCache
register(
IncrementalDataProvider::class.java,
@@ -188,7 +191,7 @@ class KotlinJsModuleBuildTarget(compileContext: CompileContext, jpsModuleBuildTa
result: MutableList<String>,
isTests: Boolean
) {
val dependencyBuildTarget = context.kotlinBuildTargets[ModuleBuildTarget(module, isTests)]
val dependencyBuildTarget = kotlinContext.targetsBinding[ModuleBuildTarget(module, isTests)]
if (dependencyBuildTarget != this@KotlinJsModuleBuildTarget &&
dependencyBuildTarget is KotlinJsModuleBuildTarget &&
@@ -3,7 +3,7 @@
* that can be found in the license/LICENSE.txt file.
*/
package org.jetbrains.kotlin.jps.platforms
package org.jetbrains.kotlin.jps.targets
import com.intellij.openapi.util.io.FileUtil
import com.intellij.openapi.util.text.StringUtil
@@ -32,6 +32,7 @@ import org.jetbrains.kotlin.incremental.*
import org.jetbrains.kotlin.incremental.components.ExpectActualTracker
import org.jetbrains.kotlin.incremental.components.LookupTracker
import org.jetbrains.kotlin.jps.build.KotlinBuilder
import org.jetbrains.kotlin.jps.build.KotlinCompileContext
import org.jetbrains.kotlin.jps.build.KotlinDirtySourceFilesHolder
import org.jetbrains.kotlin.jps.incremental.JpsIncrementalCache
import org.jetbrains.kotlin.jps.incremental.JpsIncrementalJvmCache
@@ -48,8 +49,8 @@ import java.io.IOException
private const val JVM_BUILD_META_INFO_FILE_NAME = "jvm-build-meta-info.txt"
class KotlinJvmModuleBuildTarget(compileContext: CompileContext, jpsModuleBuildTarget: ModuleBuildTarget) :
KotlinModuleBuildTarget<JvmBuildMetaInfo>(compileContext, jpsModuleBuildTarget) {
class KotlinJvmModuleBuildTarget(kotlinContext: KotlinCompileContext, jpsModuleBuildTarget: ModuleBuildTarget) :
KotlinModuleBuildTarget<JvmBuildMetaInfo>(kotlinContext, jpsModuleBuildTarget) {
override val isIncrementalCompilationEnabled: Boolean
get() = IncrementalCompilation.isEnabledForJvm()
@@ -64,7 +65,7 @@ class KotlinJvmModuleBuildTarget(compileContext: CompileContext, jpsModuleBuildT
override fun makeServices(
builder: Services.Builder,
incrementalCaches: Map<ModuleBuildTarget, JpsIncrementalCache>,
incrementalCaches: Map<KotlinModuleBuildTarget<*>, JpsIncrementalCache>,
lookupTracker: LookupTracker,
exceptActualTracer: ExpectActualTracker
) {
@@ -74,7 +75,7 @@ class KotlinJvmModuleBuildTarget(compileContext: CompileContext, jpsModuleBuildT
register(
IncrementalCompilationComponents::class.java,
IncrementalCompilationComponentsImpl(
incrementalCaches.mapKeys { context.kotlinBuildTargets[it.key]!!.targetId } as Map<TargetId, IncrementalCache>
incrementalCaches.mapKeys { it.key.targetId } as Map<TargetId, IncrementalCache>
)
)
}
@@ -141,7 +142,7 @@ class KotlinJvmModuleBuildTarget(compileContext: CompileContext, jpsModuleBuildT
var hasDirtySources = false
val targets = chunk.targets.mapNotNull { this.context.kotlinBuildTargets[it] as? KotlinJvmModuleBuildTarget }
val targets = chunk.targets.mapNotNull { kotlinContext.targetsBinding[it] as? KotlinJvmModuleBuildTarget }
val outputDirs = targets.map { it.outputDir }.toSet()
@@ -159,7 +160,7 @@ class KotlinJvmModuleBuildTarget(compileContext: CompileContext, jpsModuleBuildT
kotlinModuleId.name,
outputDir.absolutePath,
moduleSources,
target.findSourceRoots(context),
target.findSourceRoots(dirtyFilesHolder.context),
target.findClassPathRoots(),
commonSources,
target.findModularJdkRoot(),
@@ -256,14 +257,18 @@ class KotlinJvmModuleBuildTarget(compileContext: CompileContext, jpsModuleBuildT
updateIncrementalCache(files, jpsIncrementalCache as IncrementalJvmCache, changesCollector, null)
}
override val globalLookupCacheId: String
get() = "jvm"
override fun updateChunkMappings(
localContext: CompileContext,
chunk: ModuleChunk,
dirtyFilesHolder: KotlinDirtySourceFilesHolder,
outputItems: Map<ModuleBuildTarget, Iterable<GeneratedFile>>,
incrementalCaches: Map<ModuleBuildTarget, JpsIncrementalCache>
incrementalCaches: Map<KotlinModuleBuildTarget<*>, JpsIncrementalCache>
) {
val previousMappings = context.projectDescriptor.dataManager.mappings
val callback = JavaBuilderUtil.getDependenciesRegistrar(context)
val previousMappings = localContext.projectDescriptor.dataManager.mappings
val callback = JavaBuilderUtil.getDependenciesRegistrar(localContext)
val targetDirtyFiles: Map<ModuleBuildTarget, Set<File>> = chunk.targets.keysToMap {
val files = HashSet<File>()
@@ -273,7 +278,7 @@ class KotlinJvmModuleBuildTarget(compileContext: CompileContext, jpsModuleBuildT
}
fun getOldSourceFiles(target: ModuleBuildTarget, generatedClass: GeneratedJvmClass): Set<File> {
val cache = incrementalCaches[target] ?: return emptySet()
val cache = incrementalCaches[kotlinContext.targetsBinding[target]] ?: return emptySet()
cache as JpsIncrementalJvmCache
val className = generatedClass.outputClass.className
@@ -301,7 +306,7 @@ class KotlinJvmModuleBuildTarget(compileContext: CompileContext, jpsModuleBuildT
}
val allCompiled = dirtyFilesHolder.allDirtyFiles
JavaBuilderUtil.registerFilesToCompile(context, allCompiled)
JavaBuilderUtil.registerSuccessfullyCompiled(context, allCompiled)
JavaBuilderUtil.registerFilesToCompile(localContext, allCompiled)
JavaBuilderUtil.registerSuccessfullyCompiled(localContext, allCompiled)
}
}
@@ -3,14 +3,13 @@
* that can be found in the license/LICENSE.txt file.
*/
package org.jetbrains.kotlin.jps.platforms
package org.jetbrains.kotlin.jps.targets
import org.jetbrains.jps.ModuleChunk
import org.jetbrains.jps.builders.storage.BuildDataPaths
import org.jetbrains.jps.incremental.CompileContext
import org.jetbrains.jps.incremental.ModuleBuildTarget
import org.jetbrains.jps.incremental.ProjectBuildException
import org.jetbrains.jps.incremental.storage.BuildDataManager
import org.jetbrains.jps.model.java.JpsJavaClasspathKind
import org.jetbrains.jps.model.java.JpsJavaExtensionService
import org.jetbrains.jps.model.module.JpsModule
@@ -21,20 +20,18 @@ import org.jetbrains.kotlin.build.GeneratedFile
import org.jetbrains.kotlin.cli.common.arguments.CommonCompilerArguments
import org.jetbrains.kotlin.cli.common.messages.CompilerMessageSeverity
import org.jetbrains.kotlin.compilerRunner.JpsCompilerEnvironment
import org.jetbrains.kotlin.config.*
import org.jetbrains.kotlin.incremental.CacheVersion
import org.jetbrains.kotlin.config.ApiVersion
import org.jetbrains.kotlin.config.LanguageVersion
import org.jetbrains.kotlin.config.Services
import org.jetbrains.kotlin.incremental.ChangesCollector
import org.jetbrains.kotlin.incremental.ExpectActualTrackerImpl
import org.jetbrains.kotlin.incremental.LookupTrackerImpl
import org.jetbrains.kotlin.incremental.components.ExpectActualTracker
import org.jetbrains.kotlin.incremental.components.LookupTracker
import org.jetbrains.kotlin.jps.build.KotlinBuilder
import org.jetbrains.kotlin.jps.build.KotlinIncludedModuleSourceRoot
import org.jetbrains.kotlin.jps.build.KotlinDirtySourceFilesHolder
import org.jetbrains.kotlin.jps.build.isKotlinSourceFile
import org.jetbrains.kotlin.jps.incremental.CacheVersionProvider
import org.jetbrains.kotlin.incremental.storage.version.CacheAttributesDiff
import org.jetbrains.kotlin.incremental.storage.version.loadDiff
import org.jetbrains.kotlin.incremental.storage.version.localCacheVersionManager
import org.jetbrains.kotlin.jps.build.*
import org.jetbrains.kotlin.jps.incremental.JpsIncrementalCache
import org.jetbrains.kotlin.jps.model.kotlinCompilerArguments
import org.jetbrains.kotlin.jps.model.productionOutputFilePath
import org.jetbrains.kotlin.jps.model.testOutputFilePath
import org.jetbrains.kotlin.modules.TargetId
@@ -46,12 +43,35 @@ import java.io.File
/**
* Properties and actions for Kotlin test / production module build target.
*/
abstract class KotlinModuleBuildTarget<BuildMetaInfoType : BuildMetaInfo>(
val context: CompileContext,
abstract class KotlinModuleBuildTarget<BuildMetaInfoType : BuildMetaInfo> internal constructor(
val kotlinContext: KotlinCompileContext,
val jpsModuleBuildTarget: ModuleBuildTarget
) {
/**
* Note: beware of using this context for getting compilation round dependent data:
* for example groovy can provide temp source roots with stubs, and it will be visible
* only in round local compile context.
*
* TODO(1.2.80): got rid of jpsGlobalContext and replace it with kotlinContext
*/
val jpsGlobalContext: CompileContext
get() = kotlinContext.jpsContext
// Initialized in KotlinCompileContext.loadTargets
lateinit var chunk: KotlinChunk
abstract val globalLookupCacheId: String
abstract val isIncrementalCompilationEnabled: Boolean
@Suppress("LeakingThis")
val localCacheVersionManager = localCacheVersionManager(
kotlinContext.dataPaths.getTargetDataRoot(jpsModuleBuildTarget),
isIncrementalCompilationEnabled
)
val initialLocalCacheAttributesDiff: CacheAttributesDiff<*> = localCacheVersionManager.loadDiff()
val module: JpsModule
get() = jpsModuleBuildTarget.module
@@ -76,8 +96,8 @@ abstract class KotlinModuleBuildTarget<BuildMetaInfoType : BuildMetaInfo>(
val explicitOutputPath = if (isTests) module.testOutputFilePath else module.productionOutputFilePath
val explicitOutputDir = explicitOutputPath?.let { File(it).absoluteFile.parentFile }
return@lazy explicitOutputDir
?: jpsModuleBuildTarget.outputDir
?: throw ProjectBuildException("No output directory found for " + this)
?: jpsModuleBuildTarget.outputDir
?: throw ProjectBuildException("No output directory found for " + this)
}
val friendBuildTargets: List<KotlinModuleBuildTarget<*>>
@@ -85,8 +105,8 @@ abstract class KotlinModuleBuildTarget<BuildMetaInfoType : BuildMetaInfo>(
val result = mutableListOf<KotlinModuleBuildTarget<*>>()
if (isTests) {
result.addIfNotNull(context.kotlinBuildTargets[module.productionBuildTarget])
result.addIfNotNull(context.kotlinBuildTargets[relatedProductionModule?.productionBuildTarget])
result.addIfNotNull(kotlinContext.targetsBinding[module.productionBuildTarget])
result.addIfNotNull(kotlinContext.targetsBinding[relatedProductionModule?.productionBuildTarget])
}
return result.filter { it.sources.isNotEmpty() }
@@ -100,26 +120,48 @@ abstract class KotlinModuleBuildTarget<BuildMetaInfoType : BuildMetaInfo>(
private val relatedProductionModule: JpsModule?
get() = JpsJavaExtensionService.getInstance().getTestModuleProperties(module)?.productionModule
data class Dependency(
val src: KotlinModuleBuildTarget<*>,
val target: KotlinModuleBuildTarget<*>,
val exported: Boolean
)
// TODO(1.2.80): try replace allDependencies with KotlinChunk.collectDependentChunksRecursivelyExportedOnly
@Deprecated("Consider using precalculated KotlinChunk.collectDependentChunksRecursivelyExportedOnly")
val allDependencies by lazy {
JpsJavaExtensionService.dependencies(module).recursively().exportedOnly()
.includedIn(JpsJavaClasspathKind.compile(isTests))
}
val sources: Map<File, Source> by lazy {
mutableMapOf<File, Source>().also { result ->
collectSources(result)
/**
* All sources of this target (including non dirty).
* Initialized lazily based on global context and will be updated on each round based on round local context.
*
* Update required since source roots can be changed, for example groovy can provide new temporary source roots with stubs.
* Lazy initialization is required for friend build targets, when friends are not compiled in this build run.
*/
val sources: Map<File, Source>
get() = _sources ?: synchronized(this) {
_sources ?: updateSourcesList(jpsGlobalContext)
}
@Volatile
private var _sources: Map<File, Source>? = null
fun nextRound(localContext: CompileContext) {
updateSourcesList(localContext)
}
private fun collectSources(receiver: MutableMap<File, Source>) {
private fun updateSourcesList(localContext: CompileContext): Map<File, Source> {
val result = mutableMapOf<File, Source>()
val moduleExcludes = module.excludeRootsList.urls.mapTo(java.util.HashSet(), JpsPathUtil::urlToFile)
val compilerExcludes = JpsJavaExtensionService.getInstance()
.getOrCreateCompilerConfiguration(module.project)
.compilerExcludes
val buildRootIndex = context.projectDescriptor.buildRootIndex
val roots = buildRootIndex.getTargetRoots(jpsModuleBuildTarget, context)
val buildRootIndex = localContext.projectDescriptor.buildRootIndex
val roots = buildRootIndex.getTargetRoots(jpsModuleBuildTarget, localContext)
roots.forEach { rootDescriptor ->
val isIncludedSourceRoot = rootDescriptor is KotlinIncludedModuleSourceRoot
@@ -127,11 +169,14 @@ abstract class KotlinModuleBuildTarget<BuildMetaInfoType : BuildMetaInfo>(
.onEnter { file -> file !in moduleExcludes }
.forEach { file ->
if (!compilerExcludes.isExcluded(file) && file.isFile && file.isKotlinSourceFile) {
receiver[file] = Source(file, isIncludedSourceRoot)
result[file] = Source(file, isIncludedSourceRoot)
}
}
}
this._sources = result
return result
}
/**
@@ -179,9 +224,6 @@ abstract class KotlinModuleBuildTarget<BuildMetaInfoType : BuildMetaInfo>(
return false
}
fun compilerArgumentsForChunk(chunk: ModuleChunk): CommonCompilerArguments =
chunk.representativeTarget().module.kotlinCompilerArguments
open fun doAfterBuild() {
}
@@ -193,10 +235,11 @@ abstract class KotlinModuleBuildTarget<BuildMetaInfoType : BuildMetaInfo>(
* Called for `ModuleChunk.representativeTarget`
*/
open fun updateChunkMappings(
localContext: CompileContext,
chunk: ModuleChunk,
dirtyFilesHolder: KotlinDirtySourceFilesHolder,
outputItems: Map<ModuleBuildTarget, Iterable<GeneratedFile>>,
incrementalCaches: Map<ModuleBuildTarget, JpsIncrementalCache>
incrementalCaches: Map<KotlinModuleBuildTarget<*>, JpsIncrementalCache>
) {
// by default do nothing
}
@@ -213,7 +256,7 @@ abstract class KotlinModuleBuildTarget<BuildMetaInfoType : BuildMetaInfo>(
open fun makeServices(
builder: Services.Builder,
incrementalCaches: Map<ModuleBuildTarget, JpsIncrementalCache>,
incrementalCaches: Map<KotlinModuleBuildTarget<*>, JpsIncrementalCache>,
lookupTracker: LookupTracker,
exceptActualTracer: ExpectActualTracker
) {
@@ -222,7 +265,7 @@ abstract class KotlinModuleBuildTarget<BuildMetaInfoType : BuildMetaInfo>(
register(ExpectActualTracker::class.java, exceptActualTracer)
register(CompilationCanceledStatus::class.java, object : CompilationCanceledStatus {
override fun checkCanceled() {
if (context.cancelStatus.isCanceled) throw CompilationCanceledException()
if (jpsGlobalContext.cancelStatus.isCanceled) throw CompilationCanceledException()
}
})
}
@@ -261,7 +304,7 @@ abstract class KotlinModuleBuildTarget<BuildMetaInfoType : BuildMetaInfo>(
val hasRemovedSources = dirtyFilesHolder.getRemovedFiles(target.jpsModuleBuildTarget).isNotEmpty()
val hasDirtyOrRemovedSources = moduleSources.isNotEmpty() || hasRemovedSources
if (hasDirtyOrRemovedSources) {
val logger = context.loggingManager.projectBuilderLogger
val logger = jpsGlobalContext.loggingManager.projectBuilderLogger
if (logger.isEnabled) {
logger.logCompiledFiles(moduleSources, KotlinBuilder.KOTLIN_BUILDER_NAME, "Compiling files:")
}
@@ -274,66 +317,48 @@ abstract class KotlinModuleBuildTarget<BuildMetaInfoType : BuildMetaInfo>(
abstract val buildMetaInfoFileName: String
fun buildMetaInfoFile(target: ModuleBuildTarget, dataManager: BuildDataManager): File =
File(dataManager.dataPaths.getTargetDataRoot(target), buildMetaInfoFileName)
fun isVersionChanged(chunk: KotlinChunk, buildMetaInfo: BuildMetaInfo): Boolean {
val file = chunk.buildMetaInfoFile(jpsModuleBuildTarget)
if (!file.exists()) return false
fun saveVersions(context: CompileContext, chunk: ModuleChunk, commonArguments: CommonCompilerArguments) {
val dataManager = context.projectDescriptor.dataManager
val targets = chunk.targets
val cacheVersionsProvider = CacheVersionProvider(dataManager.dataPaths, isIncrementalCompilationEnabled)
cacheVersionsProvider.allVersions(targets).forEach { it.saveIfNeeded() }
val buildMetaInfo = buildMetaInfoFactory.create(commonArguments)
val serializedMetaInfo = buildMetaInfoFactory.serializeToString(buildMetaInfo)
for (target in chunk.targets) {
buildMetaInfoFile(target, dataManager).writeText(serializedMetaInfo)
}
}
fun checkCachesVersions(chunk: ModuleChunk, dataManager: BuildDataManager, actions: MutableSet<CacheVersion.Action>) {
val args = compilerArgumentsForChunk(chunk)
val currentBuildMetaInfo = buildMetaInfoFactory.create(args)
for (target in chunk.targets) {
val file = buildMetaInfoFile(target, dataManager)
if (!file.exists()) continue
val lastBuildMetaInfo =
try {
buildMetaInfoFactory.deserializeFromString(file.readText()) ?: continue
} catch (e: Exception) {
KotlinBuilder.LOG.error("Could not deserialize build meta info", e)
continue
}
val lastBuildLangVersion = LanguageVersion.fromVersionString(lastBuildMetaInfo.languageVersionString)
val lastBuildApiVersion = ApiVersion.parse(lastBuildMetaInfo.apiVersionString)
val currentLangVersion =
args.languageVersion?.let { LanguageVersion.fromVersionString(it) } ?: VersionView.RELEASED_VERSION
val currentApiVersion =
args.apiVersion?.let { ApiVersion.parse(it) } ?: ApiVersion.createByLanguageVersion(currentLangVersion)
val reasonToRebuild = when {
currentLangVersion != lastBuildLangVersion -> {
"Language version was changed ($lastBuildLangVersion -> $currentLangVersion)"
}
currentApiVersion != lastBuildApiVersion -> {
"Api version was changed ($lastBuildApiVersion -> $currentApiVersion)"
}
lastBuildLangVersion != LanguageVersion.KOTLIN_1_0 && lastBuildMetaInfo.isEAP && !currentBuildMetaInfo.isEAP -> {
// If EAP->Non-EAP build with IC, then rebuild all kotlin
"Last build was compiled with EAP-plugin"
}
else -> null
val prevBuildMetaInfo =
try {
buildMetaInfoFactory.deserializeFromString(file.readText()) ?: return false
} catch (e: Exception) {
KotlinBuilder.LOG.error("Could not deserialize build meta info", e)
return false
}
if (reasonToRebuild != null) {
KotlinBuilder.LOG.info("$reasonToRebuild. Performing non-incremental rebuild (kotlin only)")
actions.add(CacheVersion.Action.REBUILD_ALL_KOTLIN)
val prevLangVersion = LanguageVersion.fromVersionString(prevBuildMetaInfo.languageVersionString)
val prevApiVersion = ApiVersion.parse(prevBuildMetaInfo.apiVersionString)
val reasonToRebuild = when {
chunk.langVersion != prevLangVersion -> "Language version was changed ($prevLangVersion -> ${chunk.langVersion})"
chunk.apiVersion != prevApiVersion -> "Api version was changed ($prevApiVersion -> ${chunk.apiVersion})"
prevLangVersion != LanguageVersion.KOTLIN_1_0 && prevBuildMetaInfo.isEAP && !buildMetaInfo.isEAP -> {
// If EAP->Non-EAP build with IC, then rebuild all kotlin
"Last build was compiled with EAP-plugin"
}
else -> null
}
if (reasonToRebuild != null) {
KotlinBuilder.LOG.info("$reasonToRebuild. Performing non-incremental rebuild (kotlin only)")
return true
}
return false
}
}
private fun checkRepresentativeTarget(chunk: KotlinChunk) {
check(chunk.representativeTarget == this)
}
private fun checkRepresentativeTarget(chunk: ModuleChunk) {
check(chunk.representativeTarget() == jpsModuleBuildTarget)
}
private fun checkRepresentativeTarget(chunk: List<KotlinModuleBuildTarget<*>>) {
check(chunk.first() == this)
}
}
@@ -0,0 +1,178 @@
/*
* Copyright 2010-2018 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license
* that can be found in the license/LICENSE.txt file.
*/
package org.jetbrains.kotlin.jps.targets
import org.jetbrains.jps.incremental.ModuleBuildTarget
import org.jetbrains.jps.model.java.JpsJavaClasspathKind
import org.jetbrains.jps.model.java.JpsJavaExtensionService
import org.jetbrains.jps.model.library.JpsOrderRootType
import org.jetbrains.jps.util.JpsPathUtil
import org.jetbrains.kotlin.jps.build.KotlinBuilder
import org.jetbrains.kotlin.jps.build.KotlinChunk
import org.jetbrains.kotlin.jps.build.KotlinCompileContext
import org.jetbrains.kotlin.jps.build.ModuleBuildTarget
import org.jetbrains.kotlin.jps.model.platform
import org.jetbrains.kotlin.platform.DefaultIdeTargetPlatformKindProvider
import org.jetbrains.kotlin.platform.IdePlatformKind
import org.jetbrains.kotlin.platform.impl.*
import org.jetbrains.kotlin.utils.LibraryUtils
import kotlin.system.measureTimeMillis
class KotlinTargetsIndex(
val byJpsTarget: Map<ModuleBuildTarget, KotlinModuleBuildTarget<*>>,
val chunks: List<KotlinChunk>,
val chunksByJpsRepresentativeTarget: Map<ModuleBuildTarget, KotlinChunk>
)
internal class KotlinTargetsIndexBuilder internal constructor(
private val uninitializedContext: KotlinCompileContext
) {
private val byJpsModuleBuildTarget = mutableMapOf<ModuleBuildTarget, KotlinModuleBuildTarget<*>>()
private val isKotlinJsStdlibJar = mutableMapOf<String, Boolean>()
private val chunks = mutableListOf<KotlinChunk>()
fun build(): KotlinTargetsIndex {
val time = measureTimeMillis {
val jpsContext = uninitializedContext.jpsContext
// visit all kotlin build targets
jpsContext.projectDescriptor.buildTargetIndex.getSortedTargetChunks(jpsContext).forEach { chunk ->
val moduleBuildTargets = chunk.targets.mapNotNull {
if (it is ModuleBuildTarget) ensureLoaded(it)!!
else null
}
if (moduleBuildTargets.isNotEmpty()) {
val kotlinChunk = KotlinChunk(uninitializedContext, moduleBuildTargets)
moduleBuildTargets.forEach {
it.chunk = kotlinChunk
}
chunks.add(kotlinChunk)
}
}
calculateChunkDependencies()
}
KotlinBuilder.LOG.info("KotlinTargetsIndex created in $time ms")
return KotlinTargetsIndex(
byJpsModuleBuildTarget,
chunks,
chunks.associateBy { it.representativeTarget.jpsModuleBuildTarget }
)
}
private fun calculateChunkDependencies() {
chunks.forEach { chunk ->
val dependencies = mutableSetOf<KotlinModuleBuildTarget.Dependency>()
chunk.targets.forEach {
dependencies.addAll(calculateTargetDependencies(it))
}
chunk.dependencies = dependencies.toList()
chunk.dependencies.forEach { dependency ->
dependency.target.chunk._dependent!!.add(dependency)
}
}
chunks.forEach {
it.dependent = it._dependent!!.toList()
it._dependent = null
}
}
private fun calculateTargetDependencies(srcTarget: KotlinModuleBuildTarget<*>): List<KotlinModuleBuildTarget.Dependency> {
val dependencies = mutableListOf<KotlinModuleBuildTarget.Dependency>()
val classpathKind = JpsJavaClasspathKind.compile(srcTarget.isTests)
// TODO(1.2.80): Ask for JPS API
// Unfortunately JPS has no API for accessing "exported" flag while enumerating module dependencies,
// but has API for getting all and exported only dependent modules.
// So, lets first get set of all dependent targets, then remove exported only.
val dependentTargets = mutableSetOf<KotlinModuleBuildTarget<*>>()
JpsJavaExtensionService.dependencies(srcTarget.module)
.includedIn(classpathKind)
.processModules { destModule ->
val destKotlinTarget = byJpsModuleBuildTarget[ModuleBuildTarget(destModule, srcTarget.isTests)]
if (destKotlinTarget != null) {
dependentTargets.add(destKotlinTarget)
}
}
JpsJavaExtensionService.dependencies(srcTarget.module)
.includedIn(classpathKind)
.exportedOnly()
.processModules { module ->
val destKotlinTarget = byJpsModuleBuildTarget[ModuleBuildTarget(module, srcTarget.isTests)]
if (destKotlinTarget != null) {
dependentTargets.remove(destKotlinTarget)
dependencies.add(KotlinModuleBuildTarget.Dependency(srcTarget, destKotlinTarget, true))
}
}
dependentTargets.forEach { destTarget ->
dependencies.add(KotlinModuleBuildTarget.Dependency(srcTarget, destTarget, false))
}
if (srcTarget.isTests) {
val srcProductionTarget = byJpsModuleBuildTarget[ModuleBuildTarget(srcTarget.module, false)]
if (srcProductionTarget != null) {
dependencies.add(KotlinModuleBuildTarget.Dependency(srcTarget, srcProductionTarget, true))
}
}
return dependencies
}
private fun ensureLoaded(target: ModuleBuildTarget): KotlinModuleBuildTarget<*>? {
return byJpsModuleBuildTarget.computeIfAbsent(target) {
val platform = target.module.platform?.kind ?: detectTargetPlatform(target)
when {
platform.isCommon -> KotlinCommonModuleBuildTarget(uninitializedContext, target)
platform.isJavaScript -> KotlinJsModuleBuildTarget(uninitializedContext, target)
platform.isJvm -> KotlinJvmModuleBuildTarget(uninitializedContext, target)
else -> error("Unsupported platform $platform")
}
}
}
/**
* Compatibility for KT-14082
* todo: remove when all projects migrated to facets
*/
private fun detectTargetPlatform(target: ModuleBuildTarget): IdePlatformKind<*> {
if (hasJsStdLib(target)) return JsIdePlatformKind
return DefaultIdeTargetPlatformKindProvider.defaultPlatform.kind
}
private fun hasJsStdLib(target: ModuleBuildTarget): Boolean {
JpsJavaExtensionService.dependencies(target.module)
.recursively()
.exportedOnly()
.includedIn(JpsJavaClasspathKind.compile(target.isTests))
.libraries
.forEach { library ->
for (root in library.getRoots(JpsOrderRootType.COMPILED)) {
val url = root.url
val isKotlinJsLib = isKotlinJsStdlibJar.computeIfAbsent(url) {
LibraryUtils.isKotlinJavascriptStdLibrary(JpsPathUtil.urlToFile(url))
}
if (isKotlinJsLib) return true
}
}
return false
}
}
@@ -1,7 +1,9 @@
================ Step #1 =================
Building module1
Actions after cache changed: [REBUILD_CHUNK, DO_NOTHING]
Local cache for KotlinChunk<KotlinJvmModuleBuildTarget>(Module 'module1' production) are INVALID: actual=CacheVersion(version=777) -> expected=CacheVersion(version=9011001)
Marked as dirty by Kotlin:
module1/src/A.kt
Cleaning output files:
out/production/module1/META-INF/module1.kotlin_module
out/production/module1/a/A.class
@@ -12,7 +14,9 @@ End of files
Exit code: OK
------------------------------------------
Building module2
Actions after cache changed: [REBUILD_CHUNK, DO_NOTHING]
Local cache for KotlinChunk<KotlinJvmModuleBuildTarget>(Module 'module2' production) are INVALID: actual=CacheVersion(version=777) -> expected=CacheVersion(version=9011001)
Marked as dirty by Kotlin:
module2/src/B.kt
Cleaning output files:
out/production/module2/META-INF/module2.kotlin_module
out/production/module2/b/B.class
@@ -23,7 +27,9 @@ End of files
Exit code: OK
------------------------------------------
Building module3
Actions after cache changed: [REBUILD_CHUNK, DO_NOTHING]
Local cache for KotlinChunk<KotlinJvmModuleBuildTarget>(Module 'module3' production) are INVALID: actual=CacheVersion(version=777) -> expected=CacheVersion(version=9011001)
Marked as dirty by Kotlin:
module3/src/C.kt
Cleaning output files:
out/production/module3/META-INF/module3.kotlin_module
out/production/module3/c/C.class
@@ -34,7 +40,9 @@ End of files
Exit code: OK
------------------------------------------
Building module4
Actions after cache changed: [REBUILD_CHUNK, DO_NOTHING]
Local cache for KotlinChunk<KotlinJvmModuleBuildTarget>(Module 'module4' production) are INVALID: actual=CacheVersion(version=777) -> expected=CacheVersion(version=9011001)
Marked as dirty by Kotlin:
module4/src/D.kt
Cleaning output files:
out/production/module4/D/D.class
out/production/module4/META-INF/module4.kotlin_module
@@ -43,4 +51,4 @@ Compiling files:
module4/src/D.kt
End of files
Exit code: OK
------------------------------------------
------------------------------------------
@@ -1,7 +1,12 @@
================ Step #1 =================
Lookups cache are INVALID: actual=(777, [jvm]) -> expected=(3011001, [jvm])
Marked as dirty by Kotlin:
module1/src/A.kt
module2/src/B.kt
module3/src/C.kt
module4/src/D.kt
Building module1
Actions after cache changed: [REBUILD_ALL_KOTLIN, DO_NOTHING]
Cleaning output files:
out/production/module1/META-INF/module1.kotlin_module
out/production/module1/a/A.class
@@ -9,15 +14,9 @@ End of files
Compiling files:
module1/src/A.kt
End of files
Marked as dirty by Kotlin:
module1/src/A.kt
module2/src/B.kt
module3/src/C.kt
module4/src/D.kt
Exit code: OK
------------------------------------------
Building module2
Actions after cache changed: [REBUILD_CHUNK, DO_NOTHING]
Cleaning output files:
out/production/module2/META-INF/module2.kotlin_module
out/production/module2/b/B.class
@@ -28,7 +27,6 @@ End of files
Exit code: OK
------------------------------------------
Building module3
Actions after cache changed: [REBUILD_CHUNK, DO_NOTHING]
Cleaning output files:
out/production/module3/META-INF/module3.kotlin_module
out/production/module3/c/C.class
@@ -39,7 +37,6 @@ End of files
Exit code: OK
------------------------------------------
Building module4
Actions after cache changed: [REBUILD_CHUNK, DO_NOTHING]
Cleaning output files:
out/production/module4/D/D.class
out/production/module4/META-INF/module4.kotlin_module
@@ -48,4 +45,4 @@ Compiling files:
module4/src/D.kt
End of files
Exit code: OK
------------------------------------------
------------------------------------------
@@ -1,7 +1,9 @@
================ Step #1 =================
Building module1
Actions after cache changed: [REBUILD_CHUNK, DO_NOTHING]
Local cache for KotlinChunk<KotlinJvmModuleBuildTarget>(Module 'module1' production) are INVALID: actual=CacheVersion(version=777) -> expected=CacheVersion(version=9011001)
Marked as dirty by Kotlin:
module1/src/a.kt
Cleaning output files:
out/production/module1/META-INF/module1.kotlin_module
out/production/module1/module1/A.class
@@ -13,7 +15,9 @@ End of files
Exit code: OK
------------------------------------------
Building module2
Actions after cache changed: [REBUILD_CHUNK, DO_NOTHING]
Local cache for KotlinChunk<KotlinJvmModuleBuildTarget>(Module 'module2' production) are INVALID: actual=CacheVersion(version=777) -> expected=CacheVersion(version=9011001)
Marked as dirty by Kotlin:
module2/src/b.kt
Cleaning output files:
out/production/module2/META-INF/module2.kotlin_module
out/production/module2/module2/BKt.class
@@ -24,7 +28,9 @@ End of files
Exit code: OK
------------------------------------------
Building module3
Actions after cache changed: [REBUILD_CHUNK, DO_NOTHING]
Local cache for KotlinChunk<KotlinJvmModuleBuildTarget>(Module 'module3' production) are INVALID: actual=CacheVersion(version=777) -> expected=CacheVersion(version=9011001)
Marked as dirty by Kotlin:
module3/src/c.kt
Cleaning output files:
out/production/module3/META-INF/module3.kotlin_module
out/production/module3/module3/CKt.class
@@ -35,7 +41,9 @@ End of files
Exit code: OK
------------------------------------------
Building module4
Actions after cache changed: [REBUILD_CHUNK, DO_NOTHING]
Local cache for KotlinChunk<KotlinJvmModuleBuildTarget>(Module 'module4' production) are INVALID: actual=CacheVersion(version=777) -> expected=CacheVersion(version=9011001)
Marked as dirty by Kotlin:
module4/src/d.kt
Cleaning output files:
out/production/module4/META-INF/module4.kotlin_module
out/production/module4/module4/D.class
@@ -47,4 +55,4 @@ Exit code: OK
------------------------------------------
Building module5
Exit code: NOTHING_DONE
------------------------------------------
------------------------------------------
@@ -1,7 +1,12 @@
================ Step #1 =================
Lookups cache are INVALID: actual=(777, [jvm]) -> expected=(3011001, [jvm])
Marked as dirty by Kotlin:
module1/src/a.kt
module2/src/b.kt
module3/src/c.kt
module4/src/d.kt
Building module1
Actions after cache changed: [REBUILD_ALL_KOTLIN, DO_NOTHING]
Cleaning output files:
out/production/module1/META-INF/module1.kotlin_module
out/production/module1/module1/A.class
@@ -10,15 +15,9 @@ End of files
Compiling files:
module1/src/a.kt
End of files
Marked as dirty by Kotlin:
module1/src/a.kt
module2/src/b.kt
module3/src/c.kt
module4/src/d.kt
Exit code: OK
------------------------------------------
Building module2
Actions after cache changed: [REBUILD_CHUNK, DO_NOTHING]
Cleaning output files:
out/production/module2/META-INF/module2.kotlin_module
out/production/module2/module2/BKt.class
@@ -29,7 +28,6 @@ End of files
Exit code: OK
------------------------------------------
Building module3
Actions after cache changed: [REBUILD_CHUNK, DO_NOTHING]
Cleaning output files:
out/production/module3/META-INF/module3.kotlin_module
out/production/module3/module3/CKt.class
@@ -40,7 +38,6 @@ End of files
Exit code: OK
------------------------------------------
Building module4
Actions after cache changed: [REBUILD_CHUNK, DO_NOTHING]
Cleaning output files:
out/production/module4/META-INF/module4.kotlin_module
out/production/module4/module4/D.class
@@ -52,4 +49,4 @@ Exit code: OK
------------------------------------------
Building module5
Exit code: NOTHING_DONE
------------------------------------------
------------------------------------------
@@ -1,7 +1,9 @@
================ Step #1 =================
Building module1
Actions after cache changed: [REBUILD_CHUNK, DO_NOTHING]
Local cache for KotlinChunk<KotlinJvmModuleBuildTarget>(Module 'module1' production) are INVALID: actual=CacheVersion(version=777) -> expected=CacheVersion(version=9011001)
Marked as dirty by Kotlin:
module1/src/a.kt
Cleaning output files:
out/production/module1/META-INF/module1.kotlin_module
out/production/module1/module1/A.class
@@ -13,7 +15,9 @@ End of files
Exit code: OK
------------------------------------------
Building module2
Actions after cache changed: [REBUILD_CHUNK, DO_NOTHING]
Local cache for KotlinChunk<KotlinJvmModuleBuildTarget>(Module 'module2' production) are INVALID: actual=CacheVersion(version=777) -> expected=CacheVersion(version=9011001)
Marked as dirty by Kotlin:
module2/src/b.kt
Cleaning output files:
out/production/module2/META-INF/module2.kotlin_module
out/production/module2/module2/BKt.class
@@ -22,4 +26,4 @@ Compiling files:
module2/src/b.kt
End of files
Exit code: OK
------------------------------------------
------------------------------------------
@@ -1,7 +1,10 @@
================ Step #1 =================
Lookups cache are INVALID: actual=(777, [jvm]) -> expected=(3011001, [jvm])
Marked as dirty by Kotlin:
module1/src/a.kt
module2/src/b.kt
Building module1
Actions after cache changed: [REBUILD_ALL_KOTLIN, DO_NOTHING]
Cleaning output files:
out/production/module1/META-INF/module1.kotlin_module
out/production/module1/module1/A.class
@@ -10,13 +13,9 @@ End of files
Compiling files:
module1/src/a.kt
End of files
Marked as dirty by Kotlin:
module1/src/a.kt
module2/src/b.kt
Exit code: OK
------------------------------------------
Building module2
Actions after cache changed: [REBUILD_CHUNK, DO_NOTHING]
Cleaning output files:
out/production/module2/META-INF/module2.kotlin_module
out/production/module2/module2/BKt.class
@@ -25,4 +24,4 @@ Compiling files:
module2/src/b.kt
End of files
Exit code: OK
------------------------------------------
------------------------------------------
@@ -1,7 +1,9 @@
================ Step #1 =================
Building module1
Actions after cache changed: [REBUILD_CHUNK, DO_NOTHING]
Local cache for KotlinChunk<KotlinJvmModuleBuildTarget>(Module 'module1' production) are INVALID: actual=CacheVersion(version=777) -> expected=CacheVersion(version=9011001)
Marked as dirty by Kotlin:
module1/src/A.kt
Cleaning output files:
out/production/module1/META-INF/module1.kotlin_module
out/production/module1/a/AKt.class
@@ -12,7 +14,9 @@ End of files
Exit code: OK
------------------------------------------
Building module2
Actions after cache changed: [REBUILD_CHUNK, DO_NOTHING]
Local cache for KotlinChunk<KotlinJvmModuleBuildTarget>(Module 'module2' production) are INVALID: actual=CacheVersion(version=777) -> expected=CacheVersion(version=9011001)
Marked as dirty by Kotlin:
module2/src/B.kt
Cleaning output files:
out/production/module2/META-INF/module2.kotlin_module
out/production/module2/b/B.class
@@ -21,4 +25,4 @@ Compiling files:
module2/src/B.kt
End of files
Exit code: OK
------------------------------------------
------------------------------------------
@@ -1,7 +1,10 @@
================ Step #1 =================
Lookups cache are INVALID: actual=(777, [jvm]) -> expected=(3011001, [jvm])
Marked as dirty by Kotlin:
module1/src/A.kt
module2/src/B.kt
Building module1
Actions after cache changed: [REBUILD_ALL_KOTLIN, DO_NOTHING]
Cleaning output files:
out/production/module1/META-INF/module1.kotlin_module
out/production/module1/a/AKt.class
@@ -9,13 +12,9 @@ End of files
Compiling files:
module1/src/A.kt
End of files
Marked as dirty by Kotlin:
module1/src/A.kt
module2/src/B.kt
Exit code: OK
------------------------------------------
Building module2
Actions after cache changed: [REBUILD_CHUNK, DO_NOTHING]
Cleaning output files:
out/production/module2/META-INF/module2.kotlin_module
out/production/module2/b/B.class
@@ -24,4 +23,4 @@ Compiling files:
module2/src/B.kt
End of files
Exit code: OK
------------------------------------------
------------------------------------------
@@ -1,7 +1,9 @@
================ Step #1 =================
Building module1
Actions after cache changed: [REBUILD_CHUNK, DO_NOTHING]
Local cache for KotlinChunk<KotlinJvmModuleBuildTarget>(Module 'module1' production) are INVALID: actual=CacheVersion(version=777) -> expected=CacheVersion(version=9011001)
Marked as dirty by Kotlin:
module1/src/A.kt
Cleaning output files:
out/production/module1/META-INF/module1.kotlin_module
out/production/module1/a/A.class
@@ -13,7 +15,9 @@ End of files
Exit code: OK
------------------------------------------
Building module2
Actions after cache changed: [REBUILD_CHUNK, DO_NOTHING]
Local cache for KotlinChunk<KotlinJvmModuleBuildTarget>(Module 'module2' production) are INVALID: actual=CacheVersion(version=777) -> expected=CacheVersion(version=9011001)
Marked as dirty by Kotlin:
module2/src/B.kt
Cleaning output files:
out/production/module2/META-INF/module2.kotlin_module
out/production/module2/b/B.class
@@ -22,4 +26,4 @@ Compiling files:
module2/src/B.kt
End of files
Exit code: OK
------------------------------------------
------------------------------------------
@@ -1,7 +1,10 @@
================ Step #1 =================
Lookups cache are INVALID: actual=(777, [jvm]) -> expected=(3011001, [jvm])
Marked as dirty by Kotlin:
module1/src/A.kt
module2/src/B.kt
Building module1
Actions after cache changed: [REBUILD_ALL_KOTLIN, DO_NOTHING]
Cleaning output files:
out/production/module1/META-INF/module1.kotlin_module
out/production/module1/a/A.class
@@ -10,13 +13,9 @@ End of files
Compiling files:
module1/src/A.kt
End of files
Marked as dirty by Kotlin:
module1/src/A.kt
module2/src/B.kt
Exit code: OK
------------------------------------------
Building module2
Actions after cache changed: [REBUILD_CHUNK, DO_NOTHING]
Cleaning output files:
out/production/module2/META-INF/module2.kotlin_module
out/production/module2/b/B.class
@@ -25,4 +24,4 @@ Compiling files:
module2/src/B.kt
End of files
Exit code: OK
------------------------------------------
------------------------------------------
@@ -1,6 +1,10 @@
================ Step #1 =================
Actions after cache changed: [REBUILD_CHUNK, DO_NOTHING]
Local cache for KotlinChunk<KotlinJvmModuleBuildTarget>(Module 'module' production) are INVALID: actual=CacheVersion(version=777) -> expected=CacheVersion(version=9011001)
Marked as dirty by Kotlin:
src/a.kt
src/b.kt
src/other.kt
Cleaning output files:
out/production/module/META-INF/module.kotlin_module
out/production/module/other/OtherKt.class
@@ -1,6 +1,10 @@
================ Step #1 =================
Actions after cache changed: [REBUILD_ALL_KOTLIN, DO_NOTHING]
Lookups cache are INVALID: actual=(777, [jvm]) -> expected=(3011001, [jvm])
Marked as dirty by Kotlin:
src/a.kt
src/b.kt
src/other.kt
Cleaning output files:
out/production/module/META-INF/module.kotlin_module
out/production/module/other/OtherKt.class
@@ -12,9 +16,5 @@ Compiling files:
src/b.kt
src/other.kt
End of files
Marked as dirty by Kotlin:
src/a.kt
src/b.kt
src/other.kt
Exit code: OK
------------------------------------------
------------------------------------------
@@ -1,6 +1,10 @@
================ Step #1 =================
Actions after cache changed: [REBUILD_CHUNK, DO_NOTHING]
Local cache for KotlinChunk<KotlinJvmModuleBuildTarget>(Module 'module' production) are INVALID: actual=CacheVersion(version=777) -> expected=CacheVersion(version=9011001)
Marked as dirty by Kotlin:
src/a.kt
src/b.kt
src/other.kt
Cleaning output files:
out/production/module/A.class
out/production/module/META-INF/module.kotlin_module
@@ -17,4 +21,4 @@ Exit code: OK
------------------------------------------
Compiling files:
src/A.java
End of files
End of files
@@ -1,6 +1,10 @@
================ Step #1 =================
Actions after cache changed: [REBUILD_ALL_KOTLIN, DO_NOTHING]
Lookups cache are INVALID: actual=(777, [jvm]) -> expected=(3011001, [jvm])
Marked as dirty by Kotlin:
src/a.kt
src/b.kt
src/other.kt
Cleaning output files:
out/production/module/A.class
out/production/module/META-INF/module.kotlin_module
@@ -13,12 +17,8 @@ Compiling files:
src/b.kt
src/other.kt
End of files
Marked as dirty by Kotlin:
src/a.kt
src/b.kt
src/other.kt
Exit code: OK
------------------------------------------
Compiling files:
src/A.java
End of files
End of files
@@ -1,6 +1,9 @@
================ Step #1 =================
Actions after cache changed: [REBUILD_CHUNK, DO_NOTHING]
Local cache for KotlinChunk<KotlinJvmModuleBuildTarget>(Module 'module' production) are INVALID: actual=CacheVersion(version=777) -> expected=CacheVersion(version=9011001)
Marked as dirty by Kotlin:
src/a.kt
src/b.kt
Cleaning output files:
out/production/module/META-INF/module.kotlin_module
out/production/module/test/AKt.class
@@ -11,4 +14,4 @@ Compiling files:
src/b.kt
End of files
Exit code: OK
------------------------------------------
------------------------------------------
@@ -1,6 +1,9 @@
================ Step #1 =================
Actions after cache changed: [REBUILD_ALL_KOTLIN, DO_NOTHING]
Lookups cache are INVALID: actual=(777, [jvm]) -> expected=(3011001, [jvm])
Marked as dirty by Kotlin:
src/a.kt
src/b.kt
Cleaning output files:
out/production/module/META-INF/module.kotlin_module
out/production/module/test/AKt.class
@@ -10,8 +13,5 @@ Compiling files:
src/a.kt
src/b.kt
End of files
Marked as dirty by Kotlin:
src/a.kt
src/b.kt
Exit code: OK
------------------------------------------
------------------------------------------
@@ -1,7 +1,10 @@
================ Step #1 =================
Building module1
Actions after cache changed: [REBUILD_CHUNK, DO_NOTHING]
Local cache for KotlinChunk<KotlinJvmModuleBuildTarget>(Module 'module1' production) are INVALID: actual=CacheVersion(version=777) -> expected=CacheVersion(version=9011001)
Marked as dirty by Kotlin:
module1/src/a.kt
module1/src/f.kt
Cleaning output files:
out/production/module1/META-INF/module1.kotlin_module
out/production/module1/module1/A.class
@@ -27,7 +30,9 @@ End of files
Exit code: OK
------------------------------------------
Building module2
Actions after cache changed: [REBUILD_CHUNK, DO_NOTHING]
Local cache for KotlinChunk<KotlinJvmModuleBuildTarget>(Module 'module2' production) are INVALID: actual=CacheVersion(version=777) -> expected=CacheVersion(version=9011001)
Marked as dirty by Kotlin:
module2/src/b.kt
Cleaning output files:
out/production/module2/META-INF/module2.kotlin_module
out/production/module2/module2/BKt.class
@@ -38,7 +43,9 @@ End of files
Exit code: OK
------------------------------------------
Building module3
Actions after cache changed: [REBUILD_CHUNK, DO_NOTHING]
Local cache for KotlinChunk<KotlinJvmModuleBuildTarget>(Module 'module3' production) are INVALID: actual=CacheVersion(version=777) -> expected=CacheVersion(version=9011001)
Marked as dirty by Kotlin:
module3/src/c.kt
Cleaning output files:
out/production/module3/META-INF/module3.kotlin_module
out/production/module3/module3/CKt.class
@@ -49,7 +56,9 @@ End of files
Exit code: OK
------------------------------------------
Building module4
Actions after cache changed: [REBUILD_CHUNK, DO_NOTHING]
Local cache for KotlinChunk<KotlinJvmModuleBuildTarget>(Module 'module4' production) are INVALID: actual=CacheVersion(version=777) -> expected=CacheVersion(version=9011001)
Marked as dirty by Kotlin:
module4/src/d.kt
Cleaning output files:
out/production/module4/META-INF/module4.kotlin_module
out/production/module4/module4/D.class
@@ -61,4 +70,4 @@ Exit code: OK
------------------------------------------
Building module5
Exit code: NOTHING_DONE
------------------------------------------
------------------------------------------
@@ -1,7 +1,13 @@
================ Step #1 =================
Lookups cache are INVALID: actual=(777, [jvm]) -> expected=(3011001, [jvm])
Marked as dirty by Kotlin:
module1/src/a.kt
module1/src/f.kt
module2/src/b.kt
module3/src/c.kt
module4/src/d.kt
Building module1
Actions after cache changed: [REBUILD_ALL_KOTLIN, DO_NOTHING]
Cleaning output files:
out/production/module1/META-INF/module1.kotlin_module
out/production/module1/module1/A.class
@@ -12,12 +18,6 @@ Compiling files:
module1/src/a.kt
module1/src/f.kt
End of files
Marked as dirty by Kotlin:
module1/src/a.kt
module1/src/f.kt
module2/src/b.kt
module3/src/c.kt
module4/src/d.kt
Exit code: ABORT
------------------------------------------
COMPILATION FAILED
@@ -26,21 +26,13 @@ Name expected
================ Step #2 =================
Building module1
Actions after cache changed: [REBUILD_ALL_KOTLIN, REBUILD_CHUNK]
Compiling files:
module1/src/a.kt
module1/src/f.kt
End of files
Marked as dirty by Kotlin:
module1/src/a.kt
module1/src/f.kt
module2/src/b.kt
module3/src/c.kt
module4/src/d.kt
Exit code: OK
------------------------------------------
Building module2
Actions after cache changed: [REBUILD_CHUNK, DO_NOTHING]
Cleaning output files:
out/production/module2/META-INF/module2.kotlin_module
out/production/module2/module2/BKt.class
@@ -51,7 +43,6 @@ End of files
Exit code: OK
------------------------------------------
Building module3
Actions after cache changed: [REBUILD_CHUNK, DO_NOTHING]
Cleaning output files:
out/production/module3/META-INF/module3.kotlin_module
out/production/module3/module3/CKt.class
@@ -62,7 +53,6 @@ End of files
Exit code: OK
------------------------------------------
Building module4
Actions after cache changed: [REBUILD_CHUNK, DO_NOTHING]
Cleaning output files:
out/production/module4/META-INF/module4.kotlin_module
out/production/module4/module4/D.class
@@ -74,4 +64,4 @@ Exit code: OK
------------------------------------------
Building module5
Exit code: NOTHING_DONE
------------------------------------------
------------------------------------------
@@ -1,7 +1,11 @@
================ Step #1 =================
Lookups cache are SHOULD_BE_CLEARED: actual=(3011001, [jvm]) -> expected=null
Local cache for Module 'module1' production are SHOULD_BE_CLEARED: actual=CacheVersion(version=9011001) -> expected=null
Local cache for Module 'module2' production are SHOULD_BE_CLEARED: actual=CacheVersion(version=9011001) -> expected=null
Local cache for Module 'module3' production are SHOULD_BE_CLEARED: actual=CacheVersion(version=9011001) -> expected=null
Local cache for Module 'module4' production are SHOULD_BE_CLEARED: actual=CacheVersion(version=9011001) -> expected=null
Building module1
Actions after cache changed: [CLEAN_NORMAL_CACHES, CLEAN_DATA_CONTAINER]
Exit code: NOTHING_DONE
------------------------------------------
Building module2
@@ -1,7 +1,11 @@
================ Step #1 =================
Lookups cache are SHOULD_BE_CLEARED: actual=(3011001, [jvm]) -> expected=null
Local cache for Module 'module1' production are SHOULD_BE_CLEARED: actual=CacheVersion(version=9011001) -> expected=null
Local cache for Module 'module2' production are SHOULD_BE_CLEARED: actual=CacheVersion(version=9011001) -> expected=null
Local cache for Module 'module3' production are SHOULD_BE_CLEARED: actual=CacheVersion(version=9011001) -> expected=null
Local cache for Module 'module4' production are SHOULD_BE_CLEARED: actual=CacheVersion(version=9011001) -> expected=null
Building module1
Actions after cache changed: [CLEAN_NORMAL_CACHES, CLEAN_DATA_CONTAINER]
Exit code: NOTHING_DONE
------------------------------------------
Building module2
@@ -31,15 +35,7 @@ Exit code: NOTHING_DONE
================ Step #2 =================
Building module1
Actions after cache changed: [REBUILD_ALL_KOTLIN, REBUILD_CHUNK]
Cleaning output files:
out/production/module1/META-INF/module1.kotlin_module
out/production/module1/foo/Z.class
End of files
Compiling files:
module1/src/z.kt
End of files
Lookups cache are INVALID: actual=null -> expected=(3011001, [jvm])
Marked as dirty by Kotlin:
module1/src/z.kt
module2/src/a.kt
@@ -47,10 +43,17 @@ Marked as dirty by Kotlin:
module2/src/c.kt
module3/src/d.kt
module4/src/e.kt
Building module1
Cleaning output files:
out/production/module1/META-INF/module1.kotlin_module
out/production/module1/foo/Z.class
End of files
Compiling files:
module1/src/z.kt
End of files
Exit code: OK
------------------------------------------
Building module2
Actions after cache changed: [REBUILD_CHUNK, DO_NOTHING]
Cleaning output files:
out/production/module2/META-INF/module2.kotlin_module
out/production/module2/foo/AKt.class
@@ -65,7 +68,6 @@ End of files
Exit code: OK
------------------------------------------
Building module3
Actions after cache changed: [REBUILD_CHUNK, DO_NOTHING]
Cleaning output files:
out/production/module3/META-INF/module3.kotlin_module
out/production/module3/foo/D.class
@@ -76,7 +78,6 @@ End of files
Exit code: OK
------------------------------------------
Building module4
Actions after cache changed: [REBUILD_CHUNK, DO_NOTHING]
Cleaning output files:
out/production/module4/META-INF/module4.kotlin_module
out/production/module4/foo/E.class
@@ -1,7 +1,10 @@
================ Step #1 =================
Lookups cache are SHOULD_BE_CLEARED: actual=(3011001, [jvm]) -> expected=null
Local cache for Module 'module1' production are SHOULD_BE_CLEARED: actual=CacheVersion(version=9011001) -> expected=null
Local cache for Module 'module2' production are SHOULD_BE_CLEARED: actual=CacheVersion(version=9011001) -> expected=null
Local cache for Module 'module3' production are SHOULD_BE_CLEARED: actual=CacheVersion(version=9011001) -> expected=null
Building module1
Actions after cache changed: [CLEAN_NORMAL_CACHES, CLEAN_DATA_CONTAINER]
Exit code: NOTHING_DONE
------------------------------------------
Building module2
@@ -13,8 +16,12 @@ Exit code: NOTHING_DONE
================ Step #2 =================
Lookups cache are INVALID: actual=null -> expected=(3011001, [jvm])
Marked as dirty by Kotlin:
module1/src/a.kt
module2/src/c.kt
module3/src/d.kt
Building module1
Actions after cache changed: [REBUILD_ALL_KOTLIN, REBUILD_CHUNK]
Cleaning output files:
out/production/module1/AKt.class
out/production/module1/META-INF/module1.kotlin_module
@@ -22,14 +29,9 @@ End of files
Compiling files:
module1/src/a.kt
End of files
Marked as dirty by Kotlin:
module1/src/a.kt
module2/src/c.kt
module3/src/d.kt
Exit code: OK
------------------------------------------
Building module2
Actions after cache changed: [REBUILD_CHUNK, DO_NOTHING]
Cleaning output files:
out/production/module2/B.class
out/production/module2/CKt.class
@@ -44,7 +46,6 @@ Compiling files:
module2/src/B.java
End of files
Building module3
Actions after cache changed: [REBUILD_CHUNK, DO_NOTHING]
Cleaning output files:
out/production/module3/DKt.class
out/production/module3/META-INF/module3.kotlin_module
@@ -7,4 +7,4 @@ Exit code: NOTHING_DONE
------------------------------------------
Compiling files:
src/A.java
End of files
End of files