diff --git a/libraries/kotlinx-metadata/jvm/ReadMe.md b/libraries/kotlinx-metadata/jvm/ReadMe.md index badbc1a063f..bedb82c499a 100644 --- a/libraries/kotlinx-metadata/jvm/ReadMe.md +++ b/libraries/kotlinx-metadata/jvm/ReadMe.md @@ -33,7 +33,7 @@ dependencies { } ``` -## Overview +## Reading and introspecting The entry point for reading the Kotlin metadata of a `.class` file is [`KotlinClassMetadata.readStrict`](src/kotlinx/metadata/jvm/KotlinClassMetadata.kt). The data it takes is the [`kotlin.Metadata`](../../stdlib/jvm/runtime/kotlin/Metadata.kt) annotation on the class file generated by the Kotlin compiler. @@ -90,11 +90,38 @@ if (function.isSuspend) { } ``` -## Writing metadata +## Transforming metadata -To create metadata of a Kotlin class file from scratch, construct an instance of `KmClass`/`KmPackage`/`KmLambda`, fill it with the data, and call corresponding `KotlinClassMetadata.write` function. -Resulting `kotlin.Metadata` annotation can be written to a class file. +If you transform some classes produced by Kotlin compiler (e.g., change visibilities or strip methods), then you will likely need to transform Kotlin metadata as well. +Process of doing that is relatively simple due to the fact that every property inside `KotlinClassMetadata` and Km nodes is mutable; you can rewrite/edit desired parts in-place. +After doing desired changes, the `KotlinClassMetadata.write` member method can give you the new `kotlin.Metadata` instance to put into transformed classfiles: +```kotlin +val classMetadata: KotlinClassMetadata.Class = KotlinClassMetadata.readStrict(oldAnnotation) as KotlinClassMetadata.Class +classMetadata.kmClass.functions.removeIf { it.visibility == Visibility.PRIVATE } +val newAnnotation: Metadata = classMetadata.write() +// store newAnnotation with ASM, etc. +``` + +It is recommended to retain the original instance of `KotlinClassMetadata.*` because it preserves `version` and `flags` from the original metadata. + +To simplify the transformation process even more, there is an utility method `KotlinClassMetadata.transform` that performs reading and writing for you: + +```kotlin +val oldAnnotation: Metadata = ... +val newAnnotation: Metadata = KotlinClassMetadata.transform(oldAnnotation) { metadata -> + when(metadata) { + is KotlinClassMetadata.Class -> metadata.kmClass.functions.removeIf { it.visibility == Visibility.PRIVATE } + is KotlinClassMetadata.FileFacade -> metadata.kmPackage.functions.removeIf { it.visibility == Visibility.PRIVATE } + else -> { /* no-op */ } + } + // at the end of the lambda, metadata is written automatically. +} +``` + +## Creating metadata from scratch + +To create metadata of a Kotlin class file from scratch, first, construct an instance of `KmClass`/`KmPackage`/`KmLambda`, and fill it with the data. When using metadata writers from Kotlin source code, it is very convenient to use Kotlin scoping functions such as `apply` to reduce boilerplate: ```kotlin @@ -116,16 +143,56 @@ val klass = KmClass().apply { } ``` -Then, you can encode a resulting KmClass to an annotation. - -val annotation = KotlinClassMetadata.writeClass(klass) +Then, you can put a resulting `KmClass`/`KmPackage`/`KmLambda` into an appropriate container and write it. +Pay attention to the metadata version. Usually, `JvmMetadataVersion.CURRENT` is a good choice, but you may want to write a specific version you obtained from existing Kotlin classfiles in your project. +You also have to set up the `flags`. For description of all the available flags, see [Metadata.extraInt](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin/-metadata/extra-int.html). +`0` (no flags) may be a good choice, but in case you already have some ground truth files in your project, you may want to copy flags from them. +```kotlin +val classMetadata = KotlinClassMetadata.Class(klass, JvmMetadataVersion.CURRENT, 0) +val newAnnotation: Metadata = classMetadata.write() // Write annotation directly or use annotation.kind, annotation.data1, annotation.data2, etc. ``` Please refer to [`MetadataSmokeTest.produceKotlinClassFile`](test/kotlinx/metadata/test/MetadataSmokeTest.kt) for an example where metadata of a simple Kotlin class is created, and then the class file is produced with ASM and loaded by Kotlin reflection. +## Working with different versions + +### Short guide + +There are two methods to read metadata: + +`readStrict()`: This method allows you to read the metadata strictly, meaning it will throw an exception if the metadata version is greater than what kotlinx-metadata-jvm understands. +It's suitable when your tooling can't tolerate reading potentially incomplete or incorrect information due to version differences. +It's also the only method that allows metadata transformation and `KotlinClassMetadata.write` subsequent calls. + +`readLenient()`: This method allows you to read the metadata leniently. +If the metadata version is higher than what kotlinx-metadata-jvm can interpret, it may ignore parts of the metadata it doesn't understand. +It’s more suitable when your tooling needs to read metadata of possibly newer Kotlin versions and can handle incomplete data, because it is interested only in part of it (e.g. visibility of declarations) +Keep in mind that this method will still throw an exception if metadata is changed in an unpredictable way. +**Metadata read in lenient mode can not be written back.** + +### Detailed explanation + +Kotlin compiler and its features evolve over time, and so does its metadata format. Metadata format version is equal to the Kotlin compiler version. +Naturally, evolving metadata format usually involves adding new fields for new Kotlin language features. Therefore, +some problems may occur when you're reading new metadata with an older version of Kotlin compiler or kotlinx-metadata-jvm library. + +By default, the Kotlin/JVM compiler (and similar, kotlinx-metadata-jvm library) has [forward compatibility](https://kotlinlang.org/docs/kotlin-evolution.html#evolving-the-binary-format) for versions not higher than current + 1. +It means that Kotlin compiler 2.1 can read metadata from Kotlin compiler 2.2, but not 2.3. The same is true for `KotlinClassMetadata.readStrict()` +method: it will throw an exception if you try to read metadata with version higher than `COMPATIBLE_METADATA_VERSION` + 1. +Such restriction comes from the fact that higher metadata versions (e.g. 2.3) might have some unknown fields that we skip during reading; therefore, if we write +transformed metadata back, missing some fields may result in corrupted metadata that is no longer valid for version 2.3. + +However, there are a lot of use cases for metadata introspection alone, without further transformations — for example, [binary-compatibility-validator](https://github.com/Kotlin/binary-compatibility-validator) which is interested only in visibility and modality of declarations. +For such use cases it seems overly restrictive to prohibit reading newer metadata versions (and therefore, requiring authors to do frequent updates of kotlinx-metadata-jvm dependency), +so there is a relaxed version of the reading method: `KotlinClassMetadata.readLenient()`. It is a best-effort reading method that will potentially skip unknown data, +but still provide some access to metadata. Keep in mind that this method has limitations: + +1. Metadata returned by this method can not be written back, because we are not sure if it is still valid format for newer versions. It is intended for introspection alone. +2. We cannot guarantee that metadata is not changed in the other unpredictable ways in the future. `readLenient()` tries its best, but still can throw a decoding exception or even return incorrect result. + ## Module metadata Similarly to how `KotlinClassMetadata` is used to read/write metadata of Kotlin `.class` files, [`KotlinModuleMetadata`](src/kotlinx/metadata/jvm/KotlinModuleMetadata.kt) @@ -143,39 +210,3 @@ val module = metadata.kmModule val bytes = KotlinModuleMetadata.write(module) File("META-INF/main.kotlin_module").writeBytes(bytes) ``` - -## Working with different versions - -### Short guide - -There are two methods to read metadata: - -`readStrict()`: This method allows you to read the metadata strictly, meaning it will throw an exception if the metadata version is greater than what kotlinx-metadata-jvm understands. -It's suitable when your tooling can't tolerate reading potentially incomplete or incorrect information due to version differences. -It's also the only method that allows metadata transformation and `KotlinClassMetadata.write` subsequent calls. - -`readLenient()`: This method allows you to read the metadata leniently. -If the metadata version is higher than what kotlinx-metadata-jvm can interpret, it may ignore parts of the metadata it doesn't understand but it won't throw an exception. -It’s more suitable when your tooling needs to read metadata of possibly newer Kotlin versions and can handle incomplete data, because it is interested only in part of it (e.g. visibility of declarations) -**Metadata read in lenient mode can not be written back.** - -### Detailed explanation - -Kotlin compiler and its features evolve over time, and so its metadata format. Metadata format version is equal to the Kotlin compiler version. -As you might guess, evolving metadata format usually involves adding new fields for new Kotlin language features. Therefore, -some problems may occur when you're reading new metadata with an older version of Kotlin compiler or kotlinx-metadata-jvm library. - -By default, the Kotlin compiler (and similar, kotlinx-metadata-jvm library) have forward compatibility for versions not higher than current + 1. -It means that Kotlin compiler 2.1 can read metadata from Kotlin compiler 2.2, but not 2.3. The same is true for `KotlinClassMetadata.readStrict()` -method: it will throw an exception if you try to read metadata with version higher than `COMPATIBLE_METADATA_VERSION` + 1. -Such restriction comes from the fact that higher metadata versions (e.g. 2.3) might have some unknown fields that we skip during reading; therefore, if we write -transformed metadata back, missing some fields may result in corrupted metadata that is no longer valid for version 2.3. - -However, there are a lot of use-cases for metadata introspection alone, without further transformations — for example, binary-compatibility-validator which is interested only in visibility and modality of declarations. -For such use-cases it seems over restrictive to prohibit reading newer metadata versions (and therefore, requiring authors to do frequent updates of kotlinx-metadata-jvm dependency), -so there is a relaxed version of the reading method: `KotlinClassMetadata.readLenient()`. It is a best-effort reading method that will potentially skip all unknown fields, -but still provide some access to metadata. Keep in mind that this method has limitations: - -1. Metadata returned by this method can not be written back, because we are not sure if it is still valid format for newer versions. It is intended for introspection alone. -2. While some unknown fields are skipped, we cannot guarantee that metadata is not changed in the other unpredictable ways in the future. `readLenient()` tries its best, but still may throw a decoding exception if metadata cannot be read at all. - diff --git a/libraries/kotlinx-metadata/jvm/src/kotlinx/metadata/jvm/KotlinClassMetadata.kt b/libraries/kotlinx-metadata/jvm/src/kotlinx/metadata/jvm/KotlinClassMetadata.kt index 32d9a258637..ee876b13394 100644 --- a/libraries/kotlinx-metadata/jvm/src/kotlinx/metadata/jvm/KotlinClassMetadata.kt +++ b/libraries/kotlinx-metadata/jvm/src/kotlinx/metadata/jvm/KotlinClassMetadata.kt @@ -800,7 +800,8 @@ public sealed class KotlinClassMetadata { * or equivalent [KotlinClassHeader] can be used. * * This method makes best effort to read unsupported metadata versions. - * If metadata version is greater than [JvmMetadataVersion.CURRENT] + 1, this method may ignore parts of the metadata it does not understand but it will not throw an exception. + * If metadata version is greater than [JvmMetadataVersion.CURRENT] + 1, this method still attempts to read it and may ignore parts of the metadata it does not understand. + * Keep in mind that this method will still throw an exception if metadata is changed in an unpredictable way. * Because obtained metadata can be incomplete, its [KotlinClassMetadata.write] method will throw an exception. * This method still cannot read metadata produced by pre-1.0 compilers. *