811 Commits

Author SHA1 Message Date
Nataliya.Valtman 5885514c3d Add JSON output type for build reports
#KT-65792 Fixed
2024-02-27 08:32:36 +00:00
Evgenii Mazhukin 347eedb0a0 [JPS][IC] Fix the use of Jps-dependency-graph API for lookups
All dependencies should be registered before the Backend::associate().

^KT-65429


Merge-request: KT-MR-14616
Merged-by: Evgenii Mazhukin <evgenii.mazhukin@jetbrains.com>
2024-02-27 07:42:15 +00:00
Evgenii Mazhukin cc2dcf0078 [JPS][IC] Register lookups as usages in jps dependency graph
^KT-65429 In Progress


Merge-request: KT-MR-14485
Merged-by: Evgenii Mazhukin <evgenii.mazhukin@jetbrains.com>
2024-02-23 16:08:29 +00:00
Xuan-Son Trinh f4f009cb9a [JPS] Fix KotlinJvmModuleBuildTarget.findJavaSourceRoots to only return Java source roots
This change is needed because otherwise, source files that are not Java will be added as `JavaSourceRoot`, this is incorrect and causes assertion error in `com.intellij.core.JavaCoreProjectEnvironment.addSourcesToClasspath`. Refer to KT-65325 for more context.


Merge-request: KT-MR-14088
Merged-by: Xuan Son Trinh <xuanson.trinh@jetbrains.com>
2024-01-31 16:52:42 +00:00
Aleksei.Cherepanov 1e7e42baf2 [JPS] Improve JPS dumb mode
For the correct work of the compiler, some maps should be fulfilled by the build system. In the first implementation of dumb mode for JPS all maps were disabled. Now flag was introduced to determine IC behavior: to collect data needed for the compiler's code generation and disable maps and IC analysis

^KT-65043 Fixed
2024-01-17 08:22:27 +00:00
Aleksei.Cherepanov 3e41faf91f Add KotlinBuilder 'dumb mode' flag
A new system property was added to enable a 'dumb mode' for KotlinBuilder.
In this mode it is possible to disable IC caches, but don't request rebuild
And all IC trackers will still work

^KT-64305 Fixed


Merge-request: KT-MR-13524
Merged-by: Aleksei Cherepanov <aleksei.cherepanov@jetbrains.com>
2023-12-15 00:20:54 +00:00
Nataliya.Valtman 4294c7bab7 Add limitation for changed files list for JPS build report
#KT-63549
2023-12-04 18:19:15 +00:00
Ilya Goncharov 08e3cb300a [JS] Migrate tests onto IR compiler with outputDir API instead of outputFile
^KT-61117 fixed
2023-11-29 11:21:56 +00:00
Aleksei.Cherepanov c002af6365 Make plugin classpath serialization path agnostic
This is needed for correct usage of portable caches

^KT-63799 Fixed
2023-11-25 10:46:49 +00:00
Nataliya.Valtman 6bbf5b83c8 Add compiler metrics to JPS build report
#KT-63549: Fixed
2023-11-22 11:30:08 +00:00
Nataliya.Valtman fff089eba8 Fix ClassCastException in JPS statistics ad update log messages
#KT-63594 Fixed
2023-11-21 15:28:27 +00:00
Aleksei.Cherepanov 6f3204da0a Stop using IJ's maps in graph implementation of JPS 2023-11-16 08:53:13 +00:00
Aleksei.Cherepanov a32c65e4e1 Convert absolute to relative pluginsClasspath..
..to avoid recompilation on build with portable caches

#KT-62256 Fixed


Merge-request: KT-MR-12377
Merged-by: Aleksei Cherepanov <aleksei.cherepanov@jetbrains.com>
2023-10-05 10:25:53 +00:00
Aleksei.Cherepanov 68af705664 Add import tracker
This tracker reports import directives to IJ's JPS to track changes and detectwehen it is needed to adjust compilation scope.

#KT-44835 Fixed
2023-08-30 12:42:39 +00:00
Hung Nguyen 894ba9ab80 [IC] Relocatable IC caches for projects with custom buildDir
IC caches often contain file paths. To make them relocatable, we need
to convert these file paths into relative paths, relative to a base
directory.
  - If the file paths are source files, we can use the root project
    directory as base.
  - If the file paths are class files, we should use the classes
    directory as base (before this commit, we used the root project
    directory in both cases, that's why we hit KT-58547).

The key changes in this commit include:
  - RelocatableFileToPathConverter: converts paths to relative paths
  - IncrementalCompilationContext: contains 2 different path converters,
    one for source files and one for class files
  - SourceToOutputFilesMap: maps source files to class files using the
    above path converters
  - IncrementalCompilerRunner: creates the path converters based on file
    locations

Test: RelocatableFileToPathConverterTest unit test
      SourceToOutputFilesMapTest unit test
      BuildCacheRelocationIT.testCustomBuildDirectory integration test
^KT-58547 Fixed
2023-08-16 13:40:19 +00:00
Nataliya.Valtman fb04356829 Add JPS iteration metric
#KT-58026 In progress


Co-authored-by: Nataliya Valtman <Nataliya.Valtman@jetbrains.com>


Merge-request: KT-MR-11365
Merged-by: Aleksei Cherepanov <aleksei.cherepanov@jetbrains.com>
2023-07-31 09:15:28 +00:00
Nataliya.Valtman ed2dd4b2ae Split Gradle and JPS metrics
#KT-58026 In progress
2023-07-27 11:18:54 +00:00
Alexander.Likhachev 1005d0244a [BT] Move KotlinLogger to the bt-api package
#KT-57398 In Progress
2023-06-23 12:34:23 +00:00
Nataliya.Valtman 2a391f7330 Move kotlin-build-statistic project to :compiler 2023-04-25 11:29:09 +00:00
nataliya.valtman e34dd043da Add minimal statistic report for JPS build
Fix build stat for gradle 8

#KT-56438 Fixed
2023-04-25 11:29:09 +00:00
Roman Golyshev d4cffb8a5a [213] Switch to 213 platform
KTI-1114
2023-04-21 13:19:04 +00:00
Leonid Shalupov 7480befe32 Replace reflection-based compiler arguments copying with generated code
Using of Kotlin reflection for simple operations like bean management is very slow

First time initialization time: 261 ms for `copyBean(K2JVMCompilerArguments())`
Subsequent calls of `copyBean(K2JVMCompilerArguments())` take 1.7 ms per call

Unfortunately compiler argument handling is also used in Kotlin IntelliJ plugin
to parse facet settings. Big projects may have thousands of Kotlin facets

The same `ArgumentUtilsKt.copyProperties` frame is seen across various freezes:
IDEA-252440 2-3 minutes freeze on Kotlin project reimporting in last 203 eap
IDEA-253107 A lot of short freezes (1-3 sec) during Kotlin project development
KTIJ-23501 Make main run configuration detection lighter
KTIJ-22435 Unresponsive UI with 100% cpu

Reflection issue:
KT-56358 KClasses.getMemberProperties takes too much time

This commit replaces all reflection stuff with a simple code generation
Now `K2JVMCompilerArguments().clone()` goes to hard-to-measure time
2023-04-03 15:47:35 +02:00
hungvietnguyen 4f3244fb78 IC: Compute symbols impacted by classpath changes (#5111)
IC: Compute symbols impacted by classpath changes

Incremental compilation has 4 key steps:
  1. Compile changed/impacted files
  2. Detect symbols that have changed after compiling
  3. Detect symbols that are impacted by the changed symbols
  4. Based on the changed-or-impacted symbols, identify files that need
     to be recompiled. Go back to step 1.

Normally, step 2 and 3 are done together when the changed symbols
and impacted symbols are in the same module.

However, if the changed symbols and impacted symbols are in different
modules (e.g., a `Subclass` in lib1 extends a `Superclass` in lib2),
we currently do not compute symbols in the current module that are
impacted by changes in another module (step 3 above).

This is the case for both the new IC and the old IC.

In this commit, we will compute impacted symbols for the new IC. We can
fix the old IC later if necessary (they can't be fixed together easily).

Test: Added BaseIncrementalCompilationMultiProjectIT.testChangeInterfaceInLib
^KT-56197 Fixed
2023-03-21 13:12:15 +01:00
Alexander.Likhachev 493da2a8b4 [IC] Rename DummyCompilationTransaction -> NonRecoverableCompilationTransaction 2023-03-07 16:19:21 +00:00
Aleksei.Cherepanov 57da858810 [JPS] Don't use non-optimal Files.exist() in JPS context check on dry start-up
After moving JPS from the IJ repo back to the Kotlin repo, some file operations were accidentally changed to non-optimal ones.

#KTIJ-21161 Fixed
2023-02-17 09:58:00 +00:00
Aleksei.Cherepanov 772745b0ad [JPS] Don't compute compiler args on start-up of the build process
Initialization of compiler arguments takes a long time because of its copying, but for initial context check, there is no need to do it at all for all targets except Common.

#KTIJ-21161 In Progress
2023-02-17 09:58:00 +00:00
Alexander.Likhachev 581bc89849 [IC] Introduce ICContext to simplify configuration propagation to caches
#KT-49785 In Progress
2023-01-17 22:15:56 +00:00
Alexander Likhachev 274c9b6294 [IC] Register JS IC header file and lookup counter in transaction
#KT-49785 In Progress
2023-01-17 22:15:53 +00:00
Mikhail Glukhikh 0c4a0360ac Deserialization/class reading: pass chosen JvmMetadataVersion whenever possible 2023-01-16 18:16:07 +01:00
Andrey Uskov 41ff283856 Don't call File.getCanonicalPath in KGP
Relates to KT-54579.
2022-12-04 18:49:30 +00:00
Aleksei.Cherepanov 26e7c29a91 [JPS] Rebuild module on facet change
The logic of detecting changes in Kotlin facets was changed from "Include selected fields" to "Include all compiler arguments and exclude selected". This will help to avoid multiple IC issues when new change-sensitive compiler arguments will be added

(#KTIJ-17137, #KT-51536, #KTIJ-17170, #KTIJ-17300, #KT-47983) Fixed

Merge-request: KT-MR-7455
Merged-by: Aleksei Cherepanov <aleksei.cherepanov@jetbrains.com>
2022-10-26 09:45:27 +00:00
Aleksei.Cherepanov 377082ee43 Update cache version due to change of logic in PathStringDescriptor
Update cache version after critical change: be71d884 Use case-sensitive logic in PathStringDescriptor for all platforms

#KT-53932 Fixed
2022-10-04 09:13:19 +00:00
Hung Nguyen 5b6afef67d Rename ICReporter.report/reportVerbose to info/debug
to make it clear what severity levels they are mapped to.

^KT-52839 Fixed
2022-07-15 10:48:34 +00:00
Hung Nguyen e01c2bc651 Warn in Gradle log when incremental compilation fails
When incremental compilation fails, we currently log it at the `debug`
level (and fall back to non-incremental compilation). This commit will
change it to `warning` so that we can get more user reports, which will
allow us to fix the root cause.

Also make sure the warning includes a stack trace.

Additionally, let ReportSeverity.fromCode() return a non-null value
or throw an exception otherwise as that case is not expected.

^KT-52839 In Progress
2022-07-15 10:48:34 +00:00
Andrey Uskov deb2ef9a51 Add details about compilation rounds in IC tests 2022-07-12 05:53:09 +00:00
Nikita Bobko 2c5bd4a065 Drop unused DefaultIdeTargetPlatformKindProvider class
KMA-455

This class was used in intellij but usages was dropped in
a55a62106e40f8545d654d5519576fa4e732d0fd (intellij repo).

DefaultIdeTargetPlatformKindProvider is useless because
`DefaultIdeTargetPlatformKindProvider.defaultPlatform` implementation
always returns `JvmPlatforms.defaultJvmPlatform`
2022-06-29 15:35:57 +02:00
Nikita Bobko c1f2d66ed8 Kotlin JPS plugin: drop intellij-core dependency
Drop dependency because
https://youtrack.jetbrains.com/issue/IDEA-292483/UnsupportedClassVersionError-when-trying-to-run-JUnit5-unit-test#focus=Comments-27-6034750.0-0

IDEA plans to drop `PathUtil` from JPS classpath, we should prepare to
that

`KotlinFacetSettingsProvider` isn't used in jps so it was moved into
intellij repo. It was moved to
`community/plugins/kotlin/idea/src/org/jetbrains/kotlin/config/KotlinFacetSettingsProvider.kt`
path (so you can find it git history)

Review: KT-MR-6195
2022-04-29 14:51:03 +02:00
Aleksei.Cherepanov 3d8f140d6b [JPS] Fix incremental build after changing Java enum used in Kotlin when
EnumWhenTracker implemented for tracking changed java enum class items, that used in kotlin when expression.

#KT-47824 Fixed
2022-04-19 18:39:41 +00:00
Aleksei.Cherepanov e0029b14ee Rebuild module on plugin classpaths change
Add another one reason for rebuild: if pluginClasspaths was changed or even its jar content, module will be marked for rebuild

Merge-request: KT-MR-5839
Merged-by: Aleksei Cherepanov <aleksei.cherepanov@jetbrains.com>
2022-04-18 11:06:15 +00:00
Dmitry Gridin 57b9648041 make jps-plugin testable from IJ infrastructure
^KTIJ-20456
2022-04-14 11:17:47 +07:00
Nikita Bobko a471ddf059 Kotlin JPS: Inject KOTLIN_BUNDLED macro into JPS
KTIJ-21292
2022-03-14 13:53:15 +00:00
Aleksei.Cherepanov ee756638a0 Fix creation of temp file in case of bad parent dir for Gradle
newTmpFile can fail, if you pass non-existent path as temp parent directory or java.io.tmpdir is empty or invalid. Now we try to create non-existent dirs and provide more info if it also fails. Also fix error throwing in JPS part(46c0c4f9)

#KT-51374 Fixed
2022-02-28 08:50:02 +00:00
Aleksei.Cherepanov 46c0c4f9d9 Fix creation of temp file in case of bad parent dir
createTempFile can fail, if you pass non-existent path to kotlin.jps.dir.for.module.files or java.io.tmpdir. Now we try to create non-existent dirs and provide more info if it also fails

#KTIJ-17280 Fixed
2022-02-21 08:34:11 +00:00
Aleksei.Cherepanov 44d796e850 Fix JPS tests after JPS module moved to Kotlin repo
Fix testdata of new compiler tests for JPS, as it is build-log-sensitive:
ClassHierarchyAffected.testWithIntermediateBodiesChanged
PureKotlin.testSerializedSubClassAndChangedInterfaces
WithJava.JavaUsedInKotlin.testRawErrorTypeDuringSerialization

Mute JavaUsedInKotlin.SamConversions tests, as it was fixed in 212 idea

Add missed part of commit 5f4be072 [JPS] Fix JS incremental compilation
Part of commit, located in JPS module, was missed during JPS relocation

Fix test MixedInheritance for non-JPS builds
Depends on commit [CHERRY PICKED FROM IJ] [testdata] Replace test IncrementalJvmJpsTest...
2022-01-26 10:03:05 +00:00
Aleksei.Cherepanov b3f6b78fbe [CHERRY PICKED FROM IJ] Fix debug of incremental JPS tests in intellij repo
Create extension of "load classes by parent classloader" logic to bypass clash of the same classes with different versions, loaded by different classloaders in case of running in-process compilation in debug configuration.

#KT-49786 Fixed

GitOrigin-RevId: 173749ef1a36fdb1df44e98f3d373941e00f29ad
Original commit: https://github.com/JetBrains/intellij-community/commit/16e76333d4163c3772fa6e6ef3c1c7667a4c12f9
2022-01-12 15:25:23 +01:00
Aleksei.Cherepanov ea9422fedd [CHERRY PICKED FROM IJ] Update Kotlin caches version due to adding LookupSymbolKeyDescriptor versioning
Since versioning of LookupSymbolKeyDescriptor was added in 1.5.30: f6b428f2 "KT-34862 use abi snapshot instead of build history files", it is necessary to update version of caches for JPS, as new caches are not compatible with previous.

#KT-48399 Fixed

GitOrigin-RevId: 59b1071c1a1090f78c51824d45e6c000f4ef2f3f
Original commit: https://github.com/JetBrains/intellij-community/commit/658f4aeb603bb224a14a92933ebe519948db7745
2022-01-12 15:25:22 +01:00
Aleksei.Cherepanov 7181244129 [CHERRY PICKED FROM IJ] Fix test IncrementalJvmJpsTestGenerated$WithJava$Other.testMultifileClassFileAdded
Fix test data and also remove useless marking of multifiles in KotlinBuilder after 2ee7f2b "incremental annotation processing environment" as it makes the same thing and all results are ignored

GitOrigin-RevId: 379eb3f350539cb1ee27098eecb28931df73b626
Original commit: https://github.com/JetBrains/intellij-community/commit/ee2865c35c2dadc3651518b2760b5176835b251c
2022-01-12 15:25:22 +01:00
Aleksei.Cherepanov a4bf7c9772 [CHERRY PICKED FROM IJ] Fix JPS compilation
GitOrigin-RevId: a799879de2add4ca719307ddf9a143605468e731
Original commit: https://github.com/JetBrains/intellij-community/commit/9746206871853f44205b31dd8757ed2a4ad586ca
2022-01-12 15:25:21 +01:00
Aleksei.Cherepanov 6c7a6bf3b1 [CHERRY PICKED FROM IJ] Simplify API for InlineConstTracker and add testdata
GitOrigin-RevId: b2bcde9e3102d575ca459f475c00022515b6ccba
Original commit: https://github.com/JetBrains/intellij-community/commit/f78e98d1ba5b6c142076b67f88c54282c3964c7f
2022-01-12 15:25:20 +01:00
Aleksei.Cherepanov 40dc548fec [CHERRY PICKED FROM IJ] [JPS] Fix incremental build after changing Java constant
InlineConstantTracker implemented for tracking changed java static final constants, that used in kotlin.

#KT-46506 Fixed

GitOrigin-RevId: 63830580acb09d31fe0deede9f82bc1a2847443d
Original commit: https://github.com/JetBrains/intellij-community/commit/3736c00c2c8757241e6fd217faef733421eb9e80
2022-01-12 15:25:19 +01:00