azalea/kotlin-fork
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

JPS: Refactor cache compatibility checking and build targets loading/dependency analysis.

Browse Source 
CacheVersion class refactoring:

Responsibilities of class CacheVersion are splitted into:
	- interface CacheAttributesManager<Attrs>, that should:
	    - load actual cache attribute values from FS
	    - provide expected attribute values (that is required for current build)
	    - checks when the existed cache (with actual attributes) values is suitable for current build (expected atribute values)
	    - write new values to FS for next build
	- CacheAttributesDiff is created by calling CacheAttributesManager.loadDiff extension method. This is just pair of actual and expected cache attributes values, with reference to manager. Result of loadDiff can be saved.

CacheAttributesDiff are designed to be used as facade of attributes operations: CacheAttributesDiff.status are calculated based on actual and expected attribute values. Based on that status system may perform required actions (i.e. rebuild something, clearing caches, etc...).

Methods of CacheAttributesManager other then loadDiff should be used only through CacheAttributesDiff.
Build system should work in this order:
    - get implementation of CacheAttributesManager for particular compiler and cache
    - call loadDiff __once__ and save it result
    - perform actions based on `diff.status`
    - save new cache attribute values by calling `diff.saveExpectedIfNeeded()`

There are 2 implementation of CacheAttributesManager:
    - CacheVersionManager that simple checks cache version number.
    - CompositeLookupsCacheAttributesManager - manager for global lookups cache that may contain lookups for several compilers (jvm, js).

Gradle:

Usages of CacheVersion in gradle are kept as is. For compatibility this methods are added: CacheAttributesManager.saveIfNeeded, CacheAttributesManager.clean. This methods should not be used in new code.

JPS:

All JPS logic that was responsible for cache version checking completely rewritten.

To write proper implementation for version checking, this things also changed:
	- KotlinCompileContext introduced. This context lives between first calling build of kotlin target until build finish. As of now all kotlin targets are loaded on KotlinCompileContext initialization. This is required to collect kotlin target types used in this build (jvm/js). Also all build-wide logic are moved from KotlinBuilder to KotlinCompileContext. Chunk dependency calculation also moved to build start which improves performance for big projects #KT-26113
	- Kotlin bindings to JPS build targets also stored in KotlinCompileContext, and binding is fixed. Previously it is stored in local Context and reacreated for each chunk, now they stored in KotlinCompileContext which is binded by GlobalContextKey with this exception: source roots are calculated for each round, since temporary source roots with groovy stubs are created at build time and visible only in local compile context.
	- KotlinChunk introduced. All chunk-wide logic are moved from KotlinModuleBuildTarget (i.e compiler, language, cache version checking and dependent cache loading)
	- Fix legacy MPP common dependent modules

Cache version checking logic now works as following:
- At first chunk building all targets are loaded and used platforms are collected. Lookups cache manger is created based on this set. Actual cache attributes are loaded from FS. Based on CacheAttributesDiff.status this actions are performed: if cache is invalid all kotlin will be rebuilt. If cache is not required anymore it will be cleaned.
- Before build of each chunk local chunk cache attributes will be checked. If cache is invalid, chunk will be rebuilt. If cache is not required anymore it will be cleaned.

#KT-26113 Fixed
#KT-26072 Fixed
This commit is contained in:
Sergey Rostov
2018-08-16 13:35:32 +03:00
parent 3bdfd811e8
commit 437fc9d749
71 changed files with 2233 additions and 1239 deletions
Download Patch File Download Diff File Expand all files Collapse all files
build-common/src/org/jetbrains/kotlin/build/BuildMetaInfo.kt
+3
View File
@@ -64,6 +64,9 @@ abstract class BuildMetaInfoFactory<T : BuildMetaInfo>(private val metaInfoClass
)
}
fun serializeToString(args: CommonCompilerArguments): String =
serializeToString(create(args))
fun serializeToString(info: T): String =
serializeToPlainText(info, metaInfoClass)
build-common/src/org/jetbrains/kotlin/incremental/CacheVersion.kt
-111
View File
@@ -1,111 +0,0 @@
/*
* Copyright 2010-2015 JetBrains s.r.o.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.jetbrains.kotlin.incremental
import org.jetbrains.annotations.TestOnly
import org.jetbrains.kotlin.config.IncrementalCompilation
import org.jetbrains.kotlin.metadata.jvm.deserialization.JvmBytecodeBinaryVersion
import org.jetbrains.kotlin.metadata.jvm.deserialization.JvmMetadataVersion
import java.io.File
import java.io.IOException
private val NORMAL_VERSION = 10
private val DATA_CONTAINER_VERSION = 3
private val NORMAL_VERSION_FILE_NAME = "format-version.txt"
private val DATA_CONTAINER_VERSION_FILE_NAME = "data-container-format-version.txt"
class CacheVersion(
private val ownVersion: Int,
private val versionFile: File,
private val whenVersionChanged: CacheVersion.Action,
private val whenTurnedOn: CacheVersion.Action,
private val whenTurnedOff: CacheVersion.Action,
private val isEnabled: Boolean
) {
private val actualVersion: Int?
get() = try {
versionFile.readText().toInt()
}
catch (e: NumberFormatException) {
null
}
catch (e: IOException) {
null
}
private val expectedVersion: Int
get() {
val metadata = JvmMetadataVersion.INSTANCE
val bytecode = JvmBytecodeBinaryVersion.INSTANCE
return ownVersion * 1000000 +
bytecode.major * 10000 + bytecode.minor * 100 +
metadata.major * 1000 + metadata.minor
}
fun checkVersion(): Action =
when (versionFile.exists() to isEnabled) {
true to true -> if (actualVersion != expectedVersion) whenVersionChanged else Action.DO_NOTHING
false to true -> whenTurnedOn
true to false -> whenTurnedOff
else -> Action.DO_NOTHING
}
fun saveIfNeeded() {
if (!isEnabled) return
if (!versionFile.parentFile.exists()) {
versionFile.parentFile.mkdirs()
}
versionFile.writeText(expectedVersion.toString())
}
fun clean() {
versionFile.delete()
}
@get:TestOnly
val formatVersionFile: File
get() = versionFile
// Order of entries is important, because actions are sorted in KotlinBuilder::checkVersions
enum class Action {
REBUILD_ALL_KOTLIN,
REBUILD_CHUNK,
CLEAN_NORMAL_CACHES,
CLEAN_DATA_CONTAINER,
DO_NOTHING
}
}
fun normalCacheVersion(dataRoot: File, enabled: Boolean): CacheVersion =
CacheVersion(ownVersion = NORMAL_VERSION,
versionFile = File(dataRoot, NORMAL_VERSION_FILE_NAME),
whenVersionChanged = CacheVersion.Action.REBUILD_CHUNK,
whenTurnedOn = CacheVersion.Action.REBUILD_CHUNK,
whenTurnedOff = CacheVersion.Action.CLEAN_NORMAL_CACHES,
isEnabled = enabled)
fun dataContainerCacheVersion(dataRoot: File, enabled: Boolean): CacheVersion =
CacheVersion(ownVersion = DATA_CONTAINER_VERSION,
versionFile = File(dataRoot, DATA_CONTAINER_VERSION_FILE_NAME),
whenVersionChanged = CacheVersion.Action.REBUILD_ALL_KOTLIN,
whenTurnedOn = CacheVersion.Action.REBUILD_ALL_KOTLIN,
whenTurnedOff = CacheVersion.Action.CLEAN_DATA_CONTAINER,
isEnabled = enabled)
build-common/src/org/jetbrains/kotlin/incremental/IncrementalJvmCache.kt
+84 -65
View File
@@ -25,6 +25,8 @@ import org.jetbrains.annotations.TestOnly
import org.jetbrains.kotlin.build.GeneratedJvmClass
import org.jetbrains.kotlin.config.IncrementalCompilation
import org.jetbrains.kotlin.incremental.storage.*
import org.jetbrains.kotlin.incremental.storage.version.clean
import org.jetbrains.kotlin.incremental.storage.version.localCacheVersionManager
import org.jetbrains.kotlin.inline.inlineFunctionsJvmNames
import org.jetbrains.kotlin.load.kotlin.header.KotlinClassHeader
import org.jetbrains.kotlin.load.kotlin.incremental.components.IncrementalCache
@@ -43,8 +45,8 @@ import java.util.*
val KOTLIN_CACHE_DIRECTORY_NAME = "kotlin"
open class IncrementalJvmCache(
private val targetDataRoot: File,
targetOutputDir: File?
private val targetDataRoot: File,
targetOutputDir: File?
) : AbstractIncrementalCache<JvmClassName>(File(targetDataRoot, KOTLIN_CACHE_DIRECTORY_NAME)), IncrementalCache {
companion object {
private val PROTO_MAP = "proto"
@@ -81,16 +83,16 @@ open class IncrementalJvmCache(
// used in gradle
@Suppress("unused")
fun classesBySources(sources: Iterable<File>): Iterable<JvmClassName> =
sources.flatMap { sourceToClassesMap[it] }
sources.flatMap { sourceToClassesMap[it] }
fun sourceInCache(file: File): Boolean =
sourceToClassesMap.contains(file)
fun sourcesByInternalName(internalName: String): Collection<File> =
internalNameToSource[internalName]
internalNameToSource[internalName]
fun isMultifileFacade(className: JvmClassName): Boolean =
className in multifileFacadeToParts
className in multifileFacadeToParts
override fun getClassFilePath(internalClassName: String): String {
return toSystemIndependentName(File(outputDir, "$internalClassName.class").canonicalPath)
@@ -129,7 +131,7 @@ open class IncrementalJvmCache(
}
KotlinClassHeader.Kind.MULTIFILE_CLASS -> {
val partNames = kotlinClass.classHeader.data?.toList()
?: throw AssertionError("Multifile class has no parts: ${kotlinClass.className}")
?: throw AssertionError("Multifile class has no parts: ${kotlinClass.className}")
multifileFacadeToParts[className] = partNames
// When a class is replaced with a facade with the same name,
// the class' proto wouldn't ever be deleted,
@@ -177,15 +179,15 @@ open class IncrementalJvmCache(
}
fun getObsoleteJavaClasses(): Collection<ClassId> =
dirtyOutputClassesMap.getDirtyOutputClasses()
.mapNotNull {
javaSourcesProtoMap[it]?.classId
}
dirtyOutputClassesMap.getDirtyOutputClasses()
.mapNotNull {
javaSourcesProtoMap[it]?.classId
}
fun isJavaClassToTrack(classId: ClassId): Boolean {
val jvmClassName = JvmClassName.byClassId(classId)
return dirtyOutputClassesMap.isDirty(jvmClassName) ||
jvmClassName !in javaSourcesProtoMap
jvmClassName !in javaSourcesProtoMap
}
fun isJavaClassAlreadyInCache(classId: ClassId): Boolean {
@@ -210,8 +212,7 @@ open class IncrementalJvmCache(
if (notRemovedParts.isEmpty()) {
multifileFacadeToParts.remove(facade)
}
else {
} else {
multifileFacadeToParts[facade] = notRemovedParts
}
}
@@ -265,7 +266,7 @@ open class IncrementalJvmCache(
override fun clean() {
super.clean()
normalCacheVersion(targetDataRoot, IncrementalCompilation.isEnabledForJvm()).clean()
localCacheVersionManager(targetDataRoot, IncrementalCompilation.isEnabledForJvm()).clean()
}
private inner class ProtoMap(storageFile: File) : BasicStringMap<ProtoMapValue>(storageFile, ProtoMapValueExternalizer) {
@@ -290,9 +291,11 @@ open class IncrementalJvmCache(
val key = kotlinClass.className.internalName
val oldData = storage[key]
val newData = ProtoMapValue(header.kind != KotlinClassHeader.Kind.CLASS,
BitEncoding.decodeBytes(header.data!!),
header.strings!!)
val newData = ProtoMapValue(
header.kind != KotlinClassHeader.Kind.CLASS,
BitEncoding.decodeBytes(header.data!!),
header.strings!!
)
storage[key] = newData
val packageFqName = kotlinClass.className.packageFqName
@@ -300,10 +303,10 @@ open class IncrementalJvmCache(
}
operator fun contains(className: JvmClassName): Boolean =
className.internalName in storage
className.internalName in storage
operator fun get(className: JvmClassName): ProtoMapValue? =
storage[className.internalName]
storage[className.internalName]
fun remove(className: JvmClassName, changesCollector: ChangesCollector) {
val key = className.internalName
@@ -319,15 +322,16 @@ open class IncrementalJvmCache(
}
}
private inner class JavaSourcesProtoMap(storageFile: File) : BasicStringMap<SerializedJavaClass>(storageFile, JavaClassProtoMapValueExternalizer) {
private inner class JavaSourcesProtoMap(storageFile: File) :
BasicStringMap<SerializedJavaClass>(storageFile, JavaClassProtoMapValueExternalizer) {
fun process(jvmClassName: JvmClassName, newData: SerializedJavaClass, changesCollector: ChangesCollector) {
val key = jvmClassName.internalName
val oldData = storage[key]
storage[key] = newData
changesCollector.collectProtoChanges(
oldData?.toProtoData(), newData.toProtoData(),
collectAllMembersForNewClass = true
oldData?.toProtoData(), newData.toProtoData(),
collectAllMembersForNewClass = true
)
}
@@ -340,13 +344,13 @@ open class IncrementalJvmCache(
}
operator fun get(className: JvmClassName): SerializedJavaClass? =
storage[className.internalName]
storage[className.internalName]
operator fun contains(className: JvmClassName): Boolean =
className.internalName in storage
className.internalName in storage
override fun dumpValue(value: SerializedJavaClass): String =
java.lang.Long.toHexString(value.proto.toByteArray().md5())
java.lang.Long.toHexString(value.proto.toByteArray().md5())
}
// todo: reuse code with InlineFunctionsMap?
@@ -368,7 +372,7 @@ open class IncrementalJvmCache(
}
operator fun contains(className: JvmClassName): Boolean =
className.internalName in storage
className.internalName in storage
fun process(kotlinClass: LocalFileKotlinClass, changesCollector: ChangesCollector) {
val key = kotlinClass.className.internalName
@@ -377,8 +381,7 @@ open class IncrementalJvmCache(
val newMap = getConstantsMap(kotlinClass.fileContents)
if (newMap.isNotEmpty()) {
storage[key] = newMap
}
else {
} else {
storage.remove(key)
}
@@ -392,7 +395,7 @@ open class IncrementalJvmCache(
}
override fun dumpValue(value: Map<String, Any>): String =
value.dumpMap(Any::toString)
value.dumpMap(Any::toString)
}
private inner class PackagePartMap(storageFile: File) : BasicStringMap<Boolean>(storageFile, BooleanDataDescriptor.INSTANCE) {
@@ -405,21 +408,22 @@ open class IncrementalJvmCache(
}
fun isPackagePart(className: JvmClassName): Boolean =
className.internalName in storage
className.internalName in storage
override fun dumpValue(value: Boolean) = ""
}
private inner class MultifileClassFacadeMap(storageFile: File) : BasicStringMap<Collection<String>>(storageFile, StringCollectionExternalizer) {
private inner class MultifileClassFacadeMap(storageFile: File) :
BasicStringMap<Collection<String>>(storageFile, StringCollectionExternalizer) {
operator fun set(className: JvmClassName, partNames: Collection<String>) {
storage[className.internalName] = partNames
}
operator fun get(className: JvmClassName): Collection<String>? =
storage[className.internalName]
storage[className.internalName]
operator fun contains(className: JvmClassName): Boolean =
className.internalName in storage
className.internalName in storage
fun remove(className: JvmClassName) {
storage.remove(className.internalName)
@@ -428,13 +432,14 @@ open class IncrementalJvmCache(
override fun dumpValue(value: Collection<String>): String = value.dumpCollection()
}
private inner class MultifileClassPartMap(storageFile: File) : BasicStringMap<String>(storageFile, EnumeratorStringDescriptor.INSTANCE) {
private inner class MultifileClassPartMap(storageFile: File) :
BasicStringMap<String>(storageFile, EnumeratorStringDescriptor.INSTANCE) {
fun set(partName: String, facadeName: String) {
storage[partName] = facadeName
}
fun get(partName: JvmClassName): String? =
storage[partName.internalName]
storage[partName.internalName]
fun remove(className: JvmClassName) {
storage.remove(className.internalName)
@@ -443,20 +448,21 @@ open class IncrementalJvmCache(
override fun dumpValue(value: String): String = value
}
inner class InternalNameToSourcesMap(storageFile: File) : BasicStringMap<Collection<String>>(storageFile, EnumeratorStringDescriptor(), PathCollectionExternalizer) {
inner class InternalNameToSourcesMap(storageFile: File) :
BasicStringMap<Collection<String>>(storageFile, EnumeratorStringDescriptor(), PathCollectionExternalizer) {
operator fun set(internalName: String, sourceFiles: Iterable<File>) {
storage[internalName] = sourceFiles.map { it.canonicalPath }
}
operator fun get(internalName: String): Collection<File> =
(storage[internalName] ?: emptyList()).map(::File)
(storage[internalName] ?: emptyList()).map(::File)
fun remove(internalName: String) {
storage.remove(internalName)
}
override fun dumpValue(value: Collection<String>): String =
value.dumpCollection()
value.dumpCollection()
}
private fun addToClassStorage(kotlinClass: LocalFileKotlinClass, srcFile: File) {
@@ -464,7 +470,8 @@ open class IncrementalJvmCache(
addToClassStorage(proto, nameResolver, srcFile)
}
private inner class InlineFunctionsMap(storageFile: File) : BasicStringMap<Map<String, Long>>(storageFile, StringToLongMapExternalizer) {
private inner class InlineFunctionsMap(storageFile: File) :
BasicStringMap<Map<String, Long>>(storageFile, StringToLongMapExternalizer) {
private fun getInlineFunctionsMap(header: KotlinClassHeader, bytes: ByteArray): Map<String, Long> {
val inlineFunctions = inlineFunctionsJvmNames(header)
if (inlineFunctions.isEmpty()) return emptyMap()
@@ -472,7 +479,13 @@ open class IncrementalJvmCache(
val result = HashMap<String, Long>()
ClassReader(bytes).accept(object : ClassVisitor(Opcodes.ASM5) {
override fun visitMethod(access: Int, name: String, desc: String, signature: String?, exceptions: Array<out String>?): MethodVisitor? {
override fun visitMethod(
access: Int,
name: String,
desc: String,
signature: String?,
exceptions: Array<out String>?
): MethodVisitor? {
val dummyClassWriter = ClassWriter(Opcodes.ASM5)
return object : MethodVisitor(Opcodes.ASM5, dummyClassWriter.visitMethod(0, name, desc, null, exceptions)) {
@@ -499,30 +512,35 @@ open class IncrementalJvmCache(
val newMap = getInlineFunctionsMap(kotlinClass.classHeader, kotlinClass.fileContents)
if (newMap.isNotEmpty()) {
storage[key] = newMap
}
else {
} else {
storage.remove(key)
}
for (fn in oldMap.keys + newMap.keys) {
changesCollector.collectMemberIfValueWasChanged(kotlinClass.scopeFqName(), functionNameBySignature(fn), oldMap[fn], newMap[fn])
changesCollector.collectMemberIfValueWasChanged(
kotlinClass.scopeFqName(),
functionNameBySignature(fn),
oldMap[fn],
newMap[fn]
)
}
}
// TODO get name in better way instead of using substringBefore
private fun functionNameBySignature(signature: String): String =
signature.substringBefore("(")
signature.substringBefore("(")
fun remove(className: JvmClassName) {
storage.remove(className.internalName)
}
override fun dumpValue(value: Map<String, Long>): String =
value.dumpMap { java.lang.Long.toHexString(it) }
value.dumpMap { java.lang.Long.toHexString(it) }
}
}
private object PathCollectionExternalizer : CollectionExternalizer<String>(PathStringDescriptor, { THashSet(FileUtil.PATH_HASHING_STRATEGY) })
private object PathCollectionExternalizer :
CollectionExternalizer<String>(PathStringDescriptor, { THashSet(FileUtil.PATH_HASHING_STRATEGY) })
sealed class ChangeInfo(val fqName: FqName) {
open class MembersChanged(fqName: FqName, val names: Collection<String>) : ChangeInfo(fqName) {
@@ -542,10 +560,10 @@ sealed class ChangeInfo(val fqName: FqName) {
}
private fun LocalFileKotlinClass.scopeFqName() =
when (classHeader.kind) {
KotlinClassHeader.Kind.CLASS -> className.fqNameForClassNameWithoutDollars
else -> className.packageFqName
}
when (classHeader.kind) {
KotlinClassHeader.Kind.CLASS -> className.fqNameForClassNameWithoutDollars
else -> className.packageFqName
}
fun ByteArray.md5(): Long {
val d = MessageDigest.getInstance("MD5").digest(this)!!
@@ -557,23 +575,24 @@ fun ByteArray.md5(): Long {
or ((d[5].toLong() and 0xFFL) shl 40)
or ((d[6].toLong() and 0xFFL) shl 48)
or ((d[7].toLong() and 0xFFL) shl 56)
)
)
}
@TestOnly
fun <K : Comparable<K>, V> Map<K, V>.dumpMap(dumpValue: (V)->String): String =
buildString {
append("{")
for (key in keys.sorted()) {
if (length != 1) {
append(", ")
}
val value = get(key)?.let(dumpValue) ?: "null"
append("$key -> $value")
fun <K : Comparable<K>, V> Map<K, V>.dumpMap(dumpValue: (V) -> String): String =
buildString {
append("{")
for (key in keys.sorted()) {
if (length != 1) {
append(", ")
}
append("}")
}
@TestOnly fun <T : Comparable<T>> Collection<T>.dumpCollection(): String =
"[${sorted().joinToString(", ", transform = Any::toString)}]"
val value = get(key)?.let(dumpValue) ?: "null"
append("$key -> $value")
}
append("}")
}
@TestOnly
fun <T : Comparable<T>> Collection<T>.dumpCollection(): String =
"[${sorted().joinToString(", ", transform = Any::toString)}]"
build-common/src/org/jetbrains/kotlin/incremental/storage/version/CacheAttributesDiff.kt
+37
View File
@@ -0,0 +1,37 @@
/*
* Copyright 2010-2018 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license
* that can be found in the license/LICENSE.txt file.
*/
package org.jetbrains.kotlin.incremental.storage.version
/**
* Diff between actual and expected cache attributes.
* [status] are calculated based on this diff (see [CacheStatus]).
* Based on that [status] system may perform required actions (i.e. rebuild something, clearing caches, etc...).
*
* [CacheAttributesDiff] can be used to cache current attribute values and as facade for version operations.
*/
data class CacheAttributesDiff<Attrs: Any>(
val manager: CacheAttributesManager<Attrs>,
val actual: Attrs?,
val expected: Attrs?
) {
val status: CacheStatus
get() =
if (expected != null) {
if (actual != null && manager.isCompatible(actual, expected)) CacheStatus.VALID
else CacheStatus.INVALID
} else {
if (actual != null) CacheStatus.SHOULD_BE_CLEARED
else CacheStatus.CLEARED
}
fun saveExpectedIfNeeded() {
if (expected != actual) manager.writeActualVersion(expected)
}
override fun toString(): String {
return "$status: actual=$actual -> expected=$expected"
}
}
build-common/src/org/jetbrains/kotlin/incremental/storage/version/CacheAttributesManager.kt
+79
View File
@@ -0,0 +1,79 @@
/*
* Copyright 2010-2018 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license
* that can be found in the license/LICENSE.txt file.
*/
package org.jetbrains.kotlin.incremental.storage.version
/**
* Manages cache attributes values.
*
* Attribute values can be loaded by calling [loadActual].
* Based on loaded actual and fixed [expected] values [CacheAttributesDiff] can be constructed which can calculate [CacheStatus].
* Build system may perform required actions based on that (i.e. rebuild something, clearing caches, etc...).
*
* [CacheAttributesDiff] can be used to cache current attribute values and then can be used as facade for cache version operations.
*/
interface CacheAttributesManager<Attrs : Any> {
/**
* Cache attribute values expected by the current version of build system and compiler.
* `null` means that cache is not required (incremental compilation is disabled).
*/
val expected: Attrs?
/**
* Load actual cache attribute values.
* `null` means that cache is not yet created.
*
* This is internal operation that should be implemented by particular implementation of CacheAttributesManager.
* Consider using `loadDiff().actual` for getting actual values.
*/
fun loadActual(): Attrs?
/**
* Write [values] as cache attributes for next build execution.
*
* This is internal operation that should be implemented by particular implementation of CacheAttributesManager.
* Consider using `loadDiff().saveExpectedIfNeeded()` for saving attributes values for next build.
*/
fun writeActualVersion(values: Attrs?)
/**
* Check if cache with [actual] attributes values can be used when [expected] attributes are required.
*/
fun isCompatible(actual: Attrs, expected: Attrs): Boolean = actual == expected
}
fun <Attrs : Any> CacheAttributesManager<Attrs>.loadDiff(
actual: Attrs? = this.loadActual(),
expected: Attrs? = this.expected
) = CacheAttributesDiff(this, actual, expected)
fun <Attrs : Any> CacheAttributesManager<Attrs>.loadAndCheckStatus() =
loadDiff().status
/**
* This method is kept only for compatibility.
* Save [expected] cache attributes values if it is enabled and not equals to [actual].
*/
@Deprecated(
message = "Consider using `this.loadDiff().saveExpectedIfNeeded()` and cache `loadDiff()` result.",
replaceWith = ReplaceWith("loadDiff().saveExpectedIfNeeded()")
)
fun <Attrs : Any> CacheAttributesManager<Attrs>.saveIfNeeded(
actual: Attrs? = this.loadActual(),
expected: Attrs = this.expected
?: error("To save disabled cache status [delete] should be called (this behavior is kept for compatibility)")
) = loadDiff(actual, expected).saveExpectedIfNeeded()
/**
* This method is kept only for compatibility.
* Delete actual cache attributes values if it existed.
*/
@Deprecated(
message = "Consider using `this.loadDiff().saveExpectedIfNeeded()` and cache `loadDiff()` result.",
replaceWith = ReplaceWith("writeActualVersion(null)")
)
fun CacheAttributesManager<*>.clean() {
writeActualVersion(null)
}
build-common/src/org/jetbrains/kotlin/incremental/storage/version/CacheStatus.kt
+31
View File
@@ -0,0 +1,31 @@
/*
* Copyright 2010-2018 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license
* that can be found in the license/LICENSE.txt file.
*/
package org.jetbrains.kotlin.incremental.storage.version
/**
* Status that is used by system to perform required actions (i.e. rebuild something, clearing caches, etc...).
*/
enum class CacheStatus {
/**
* Cache is valid and ready to use.
*/
VALID,
/**
* Cache is not exists or have outdated versions and/or other attributes.
*/
INVALID,
/**
* Cache is exists, but not required anymore.
*/
SHOULD_BE_CLEARED,
/**
* Cache is not exists and not required.
*/
CLEARED
}
build-common/src/org/jetbrains/kotlin/incremental/storage/version/CacheVersionManager.kt
+60
View File
@@ -0,0 +1,60 @@
/*
* Copyright 2010-2018 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license
* that can be found in the license/LICENSE.txt file.
*/
package org.jetbrains.kotlin.incremental.storage.version
import org.jetbrains.annotations.TestOnly
import org.jetbrains.kotlin.metadata.jvm.deserialization.JvmBytecodeBinaryVersion
import org.jetbrains.kotlin.metadata.jvm.deserialization.JvmMetadataVersion
import java.io.File
import java.io.IOException
/**
* Manages files with actual version [loadActual] and provides expected version [expected].
* Based on that actual and expected versions [CacheStatus] can be calculated.
* This can be done by constructing [CacheAttributesDiff] and calling [CacheAttributesDiff.status].
* Based on that status system may perform required actions (i.e. rebuild something, clearing caches, etc...).
*/
class CacheVersionManager(
private val versionFile: File,
expectedOwnVersion: Int?
) : CacheAttributesManager<CacheVersion> {
override val expected: CacheVersion? =
if (expectedOwnVersion == null) null
else {
val metadata = JvmMetadataVersion.INSTANCE
val bytecode = JvmBytecodeBinaryVersion.INSTANCE
CacheVersion(
expectedOwnVersion * 1000000 +
bytecode.major * 10000 + bytecode.minor * 100 +
metadata.major * 1000 + metadata.minor
)
}
override fun loadActual(): CacheVersion? =
if (!versionFile.exists()) null
else try {
CacheVersion(versionFile.readText().toInt())
} catch (e: NumberFormatException) {
null
} catch (e: IOException) {
null
}
override fun writeActualVersion(values: CacheVersion?) {
if (values == null) versionFile.delete()
else {
versionFile.parentFile.mkdirs()
versionFile.writeText(values.version.toString())
}
}
@get:TestOnly
val versionFileForTesting: File
get() = versionFile
}
data class CacheVersion(val version: Int)
build-common/src/org/jetbrains/kotlin/incremental/storage/version/local.kt
+17
View File
@@ -0,0 +1,17 @@
/*
* Copyright 2010-2018 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license
* that can be found in the license/LICENSE.txt file.
*/
package org.jetbrains.kotlin.incremental.storage.version
import java.io.File
private val NORMAL_VERSION = 9
private val NORMAL_VERSION_FILE_NAME = "format-version.txt"
fun localCacheVersionManager(dataRoot: File, isCachesEnabled: Boolean) =
CacheVersionManager(
File(dataRoot, NORMAL_VERSION_FILE_NAME),
if (isCachesEnabled) NORMAL_VERSION else null
)
build-common/src/org/jetbrains/kotlin/incremental/storage/version/lookups.kt
+20
View File
@@ -0,0 +1,20 @@
/*
* Copyright 2010-2018 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license
* that can be found in the license/LICENSE.txt file.
*/
package org.jetbrains.kotlin.incremental.storage.version
import java.io.File
private val DATA_CONTAINER_VERSION_FILE_NAME = "data-container-format-version.txt"
private val DATA_CONTAINER_VERSION = 3
fun lookupsCacheVersionManager(dataRoot: File, isEnabled: Boolean) =
CacheVersionManager(
File(dataRoot, DATA_CONTAINER_VERSION_FILE_NAME),
if (isEnabled) DATA_CONTAINER_VERSION else null
)
fun readLookupsCacheStatus(dataRoot: File, isEnabled: Boolean): CacheStatus =
lookupsCacheVersionManager(dataRoot, isEnabled).loadAndCheckStatus()