Commit Graph

410 Commits

Author SHA1 Message Date
Alexander.Likhachev 209dbae893 [Daemon] Make daemon port validator more precise
Previously, the validation was that the port is within the [1, 65535) range. Considering the defined range [17001, 18000) for the random port selector, it makes more sense to check that range instead.
2023-12-05 13:51:14 +00:00
Alexander.Likhachev 2c8501eb10 [Daemon] Add unit tests for LastDaemonCliOutputsTest 2023-12-05 13:51:14 +00:00
Alexander.Likhachev 3ca1249926 [Daemon] Add unit tests for LimitedLinkedList 2023-12-05 13:51:14 +00:00
Alexander.Likhachev f7b139466d [Build] Move the daemon client sources into the default directory 2023-12-05 13:51:14 +00:00
Alexander.Likhachev b000b66ef7 [Daemon] Improve some daemon log messages
A few messages were improved:
  * If the daemon process died at the startup, add its last 10 lines of the output to the log and report to the EXCEPTION (ERROR in the case of Gradle) level
  * If some exception occurs during connection attempt, add its stacktrace and report to the EXCEPTION (ERROR in the case of Gradle) level
  * Added more DEBUG level messages
  * Some important messages are moved to the INFO level
  * Added a suggestion to report issue to kotl.in/issue
^KT-55322 Fixed
2023-12-05 13:51:14 +00:00
Alexander.Likhachev 8ceee3e8b9 [Daemon] Improve daemon connection algorithm
Before this change, the logic was to find the most suitable daemon and try connecting to it in a loop, ignoring the fact that it could be dying. Now, instead of trying to connect to the daemon dying daemon in a loop, we will make only 1 attemp and then skip if it's already dying.
^KT-55322 In Progress
2023-12-05 13:51:14 +00:00
Alexander.Likhachev 44d6807aba [Daemon] Fix deprecation warning in KotlinCompilerClient.kt 2023-12-05 13:51:14 +00:00
Alexander.Likhachev 3172e74a90 [Daemon] Apply autoformatting to ClientUtils.kt 2023-12-05 13:51:14 +00:00
Alexander.Likhachev 02b1f9854f [Daemon] Apply autoformatting to KotlinCompilerClient.kt 2023-12-05 13:51:14 +00:00
Alexander.Likhachev 628f6e981d [Daemon] Add test for KT-55322 2023-12-05 13:51:14 +00:00
Alexander.Likhachev b8d1398bdb [Daemon] Use JUnit Vintage engine to run JUnit4 tests 2023-12-05 13:51:14 +00:00
Nataliya.Valtman 6bbf5b83c8 Add compiler metrics to JPS build report
#KT-63549: Fixed
2023-11-22 11:30:08 +00:00
Alexander.Likhachev b460c06907 [BT] Add support for in-process incremental compilation
#KT-61865 Fixed
2023-10-23 13:04:17 +00:00
Alexander.Likhachev 357d12fc8e [Build] Move JUnit dependencies into the version catalog
The `kotlin-test` dependencies are left untouched as changing them affects publications, thus these versions are independent from the used inside our build
#KTI-1349 In Progress
2023-09-06 22:47:33 +00:00
Aleksei.Cherepanov 68af705664 Add import tracker
This tracker reports import directives to IJ's JPS to track changes and detectwehen it is needed to adjust compilation scope.

#KT-44835 Fixed
2023-08-30 12:42:39 +00:00
Yahor Berdnikau e80988d98c [Gradle] Don't collect module-info on using new IC/JVM approach
Collecting 'moduleInfo' is required for the old history-file-based
incremental compilation approach. And the way how it is being collected
violates Gradle project-isolation feature as it accesses all Kotlin
compile tasks from other Gradle subprojects.

When the new IC based on classpath snapshot is enabled, the plugin will
not collect 'moduleInfo' making it compatible with project isolation.

^KT-59826 Fixed
2023-08-29 15:21:57 +00:00
Mikhail Glukhikh f9c7a88175 Add -Xuse-fir-lt=false to some scripting tests
make them compatible with K2
2023-08-25 08:57:53 +00:00
Hung Nguyen 894ba9ab80 [IC] Relocatable IC caches for projects with custom buildDir
IC caches often contain file paths. To make them relocatable, we need
to convert these file paths into relative paths, relative to a base
directory.
  - If the file paths are source files, we can use the root project
    directory as base.
  - If the file paths are class files, we should use the classes
    directory as base (before this commit, we used the root project
    directory in both cases, that's why we hit KT-58547).

The key changes in this commit include:
  - RelocatableFileToPathConverter: converts paths to relative paths
  - IncrementalCompilationContext: contains 2 different path converters,
    one for source files and one for class files
  - SourceToOutputFilesMap: maps source files to class files using the
    above path converters
  - IncrementalCompilerRunner: creates the path converters based on file
    locations

Test: RelocatableFileToPathConverterTest unit test
      SourceToOutputFilesMapTest unit test
      BuildCacheRelocationIT.testCustomBuildDirectory integration test
^KT-58547 Fixed
2023-08-16 13:40:19 +00:00
Artem Kobzar a29fa428b3 [K/JS] Add warning for ES-modules on the klibgen stage on the uniqueness of the exported names from the module 2023-08-08 15:45:02 +00:00
Alexander.Likhachev 61f404e493 [BT] Implement incremental compilation within the daemon
#KT-57398 In Progress
2023-08-01 16:24:37 +00:00
Alexander.Likhachev 91dc33f398 [IC] Allow to not specify the root project dir
When it's specified, it's used to calculate relative paths
2023-08-01 16:24:37 +00:00
Alexander.Likhachev b3677fa0ca [IC] Allow to not specify the modules info 2023-08-01 16:24:37 +00:00
Alexander.Likhachev e33611a88a [IC] Allow to not specify the build history file 2023-08-01 16:24:37 +00:00
Nataliya.Valtman ed2dd4b2ae Split Gradle and JPS metrics
#KT-58026 In progress
2023-07-27 11:18:54 +00:00
Aleksei.Cherepanov ae8428e8d0 Refactor Infrastructure of IC tests
1) Move root util functions to separate CompileRunnerUtils file from IncrementalJvmCompileRunner to leave the only class instance there and to separate util functions
2) Move TestLookupTracker from the abstract parent class location to a separate file as an implementation class
3) Small change naming of building functions just to clarify what they do
4) Some small code-style refactorings
2023-04-24 14:54:45 +00:00
Yahor Berdnikau eb4e96a113 Add kotlin-build-tools-enum-compat workaround
It acts as a workaround for the case when build tools or dependencies
are compiled with latest 'kotlin-stdlib' version, but at a runtime older
 'kotlin-stdlib' is provided, which does not know about new
 `EnumEntries`.

 ^KT-57317 Fixed
2023-04-21 14:57:17 +00:00
Nikita Bobko 9ca7ee3b97 Don't forget to configure JdkHome for CompilerConfiguration
Review: https://jetbrains.team/p/kt/reviews/9562

This commit fixes IdeaJsr223Test in kotlin plugin. The test failure was
caused by b50a803b6f commit where we
dropped `put(JVMConfigurationKeys.JDK_HOME, ...)` from
`configureJdkClasspathRoots`. Ilya Chernikov says that it's incorrect to
set JDK_HOME in `configureJdkClasspathRoots`.

To reproduce the test failure:
1. Update `.idea/libraries/kotlinc_kotlin_dist.xml` and
   `.idea/libraries/kotlinc_kotlin_jps_plugin_classpath.xml` with new
   Kotlin (in other words: bump bundled JPS)
2. Run the test

Test failure stacktrace:
```
javax.script.ScriptException: error: cannot access 'java.io.Serializable' which is a supertype of 'kotlin.Int'. Check your module classpath for missing or conflicting dependencies
y.first() + 2
          ^
  at org.jetbrains.kotlin.cli.common.repl.KotlinJsr223JvmScriptEngineBase.asJsr223EvalResult(KotlinJsr223JvmScriptEngineBase.kt:104)
  at org.jetbrains.kotlin.cli.common.repl.KotlinJsr223JvmScriptEngineBase.compileAndEval(KotlinJsr223JvmScriptEngineBase.kt:63)
  at org.jetbrains.kotlin.cli.common.repl.KotlinJsr223JvmScriptEngineBase.eval(KotlinJsr223JvmScriptEngineBase.kt:31)
  at java.scripting/javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:262)
  at org.jetbrains.kotlin.idea.repl.IdeaJsr223Test.testJsr223Engine(IdeaJsr223Test.kt:31)
  at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
  at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.base/java.lang.reflect.Method.invoke(Method.java:568)
  at junit.framework.TestCase.runTest(TestCase.java:177)
  at com.intellij.testFramework.UsefulTestCase.lambda$runBare$11(UsefulTestCase.java:479)
  at com.intellij.testFramework.UsefulTestCase.lambda$wrapTestRunnable$13(UsefulTestCase.java:500)
```
2023-04-12 13:16:10 +00:00
nataliya.valtman 730502638a Dont call GC for every task 2023-03-20 11:33:03 +00:00
nataliya.valtman a34657195b Add GC metric to build statistic 2023-03-14 20:58:45 +00:00
nataliya.valtman c16dabba1f Add Kotlin daemon memory usage into build report
#KT-56211: Fixed
2023-03-14 20:58:45 +00:00
Aleksei.Cherepanov 6471624d0c [Gradle] Disable usePreciseJavaTracking flag for K2
Historically flag was enabled for Gradle projects, but we don't have fir-based JavaTracker for the K2 compiler for now, so we need to use a fallback strategy.
Note: it was disabled for tests with K2
Note2: after adding a fir-based java classes tracker this value should be set to true (KT-57147)

#KT-56886 Fixed

Merge-request: KT-MR-9053
Merged-by: Aleksei Cherepanov <aleksei.cherepanov@jetbrains.com>
2023-03-09 13:41:25 +00:00
Alexander.Likhachev 3ed651a7a6 [Gradle] Add a property to control if the IC caches in-memory wrapper is enabled
#KT-56052 In Progress
2023-03-07 16:19:20 +00:00
Yahor Berdnikau 8f07a9cc61 Cleanup unused Kotlin daemon dependencies 2023-03-02 16:24:54 +01:00
Yahor Berdnikau 0cad069522 [Build] Pin api and language level to 1.8 in Kotlin compiler modules used by KGP
This is required to be able to compile KGP and it's dependencies which
set LV to 1.4 when repo will use LV 1.9. This caused by the change how
enums are compiled (KT-48872).
2023-02-18 13:26:54 +00:00
Alexander.Likhachev 06fb658887 [Build] Remove remaining obsolete files of the new daemon
KT-50846 Related
2023-02-07 14:45:35 +00:00
Dmitriy Novozhilov 88efa6bfb6 Update tests after switching to LV 1.9 2023-01-30 09:29:57 +00:00
Alexander Likhachev 20ed029ba5 [Gradle, IC] Add gradle property to control precise compilation outputs backup
#KT-49785 In Progress
2023-01-17 22:15:52 +00:00
Denis.Zharkov d3549c2c5b Optimize CoreJrtFileSystem
Do not recompute children on each file request
At the same time, we should clear the roots after each compilation
just the same way as we do for common jars
2022-12-29 09:59:16 +00:00
Andrey Uskov 1550a37b17 Cleanup: remove deprecated methods from Kotlin daemon 2022-12-05 16:02:04 +00:00
Andrey Uskov c9e0100214 Remove experimental implementation of Kotlin Compiler Daemon
#KT-50846 Fixed
2022-12-05 16:02:03 +00:00
Andrey Uskov 41ff283856 Don't call File.getCanonicalPath in KGP
Relates to KT-54579.
2022-12-04 18:49:30 +00:00
nataliya.valtman 70fb18e160 KT-54941: Use Debug level for process launch fallback reason 2022-12-02 17:42:23 +01:00
Ilya Chernikov 23144b92d8 Fix FastJarFS caches cleaning
- add FastJarFS cache cleaning to the daemon implementation, so
the behavior is now the same as for CoreJarFS
- clear not only mappings but also handlers cache: this may result some
slowdown on comparison with the previous cleaning implementation,
but should avoid unreliable behavior on heavy parallel build usages.
2022-10-31 16:02:50 +00:00
nataliya.valtman 2379185398 Log daemon exception for In process compilation into error.log 2022-10-14 15:41:48 +00:00
Hung Nguyen def886cd31 Clean up fall-back logic in IncrementalCompilerRunner
Make it clear that there 3 distinct cases:
   1. Incremental compilation completed with an ExitCode.
   2. Incremental compilation was not possible for some valid reason
      (e.g., for a clean build), and we will perform non-incremental
      compilation.
   3. Incremental compilation failed with an exception.
      In this case, we will:
        - Print a warning with a stack trace
        - Ask the user to file a bug
        - Collect rebuild reason enum for analytics
           + TODO: Collect the stack trace too
        - Fall back to non-incremental compilation

Test: Existing BaseIncrementalCompilationMultiProjectIT.testFailureHandling_UserError,
      Updated BaseIncrementalCompilationMultiProjectIT.testFailureHandling_ToolError

^KT-53015: In progress
2022-09-19 07:33:16 +03:00
Abduqodiri Qurbonzoda dce62275c6 Fix daemon-common-new compile 2022-09-17 01:06:56 +00:00
Nikita Bobko 2a4f3f41aa 2/5 Replace source dependency on kotlin-reflect with binary dependency
Review: https://jetbrains.team/p/kt/reviews/6753

Meaningful semantic change was splitted into 5 commits to simplify the
change review. Sinle commit would be too big.

Why replace source to binary: to get rid of kotlin-reflect in Kotlin
plugin artifact KTIJ-22276

Note: Kotlin Maven artifacts (./gradlew publish) changed their
dependency on kotlin-reflect
2022-08-22 15:43:49 +02:00
Nikita Bobko 8f79e833a8 Drop all redundant kotlin-reflect, kotlin-reflect-api dependencies
Review: https://jetbrains.team/p/kt/reviews/6753

All redundant I managed to find, of course.

Why: I'm going to process all reflect dependencies in the next commits.
Cleanup reflect dependency before processing.

They are redundant because:
1. if `compileOnly` then compilation didn't break after dropping the
   dependency
2. if `test*` then tests didn't break after dropping the dependency.
3. `analysis/analysis-api-fir/analysis-api-fir-generator/build.gradle.kts`
   `compiler/fir/checkers/checkers-component-generator/build.gradle.kts`
   Drop `implementation(project(":kotlin-reflect-api"))` because the
   module already depends on
   `implementation(project(":kotlin-reflect"))`
4. `compiler/daemon/daemon-client/build.gradle.kts`. Drop `runtimeOnly`
   because after dropping `compileOnly` compilation didn't break (so
   `runtimeOnly` looks suspicious). Less safe than 1-3
2022-08-22 15:42:57 +02:00
Aleksei.Cherepanov df017ea187 Clean JavaClassesTracker and LookupTracker on call RetryWithAdditionalRoots
#KT-38576 Fixed
2022-08-16 14:38:01 +00:00
Hung Nguyen 5b6afef67d Rename ICReporter.report/reportVerbose to info/debug
to make it clear what severity levels they are mapped to.

^KT-52839 Fixed
2022-07-15 10:48:34 +00:00