Commit Graph

228 Commits

Author SHA1 Message Date
Alexey Tsvetkov ba5795519b Minor: move compile iteration result to daemon common
The class is used on both server and client, but it was defined in
server module.
Gradle plugin worked, because all classes are present in kotlin compiler
embeddable, but the code was red in IDE (which is correct because
Gradle plugin does not depend on daemon server module).
2018-11-14 11:41:43 +03:00
Sergey Rostov d91f6f8c43 IC: move CacheVersionManager from build_common to jps plugin 2018-10-16 12:29:58 +03:00
Sergey Rostov 940861c245 IC: remove CacheVersionManager usages from build_common.
They are kept up to date but was never used.
2018-10-16 12:29:58 +03:00
Alexander Udalov 009f18f1f4 Split AnalysisFlag values to AnalysisFlags and JvmAnalysisFlags
Declare AnalysisFlags in module 'frontend', and JvmAnalysisFlags in
module 'frontend.java', to avoid leaking Java-related logic to common
compiler code
2018-10-01 13:31:00 +02:00
Ilya Chernikov 0303fc036c Fix gradle support for non-kts scripts when daemon is used 2018-09-27 16:27:17 +02:00
Vyacheslav Gerasimov c768130f89 Remove gradle/ext-releases-local repo, native platform is now available on jcenter 2018-09-12 15:52:15 +03:00
Ilya Gorbunov a18770fbbb Remove projectDist dependency helper usages
Use default configuration dependencies instead of projectDist ones.
2018-09-12 06:05:05 +03:00
Sergey Rostov 437fc9d749 JPS: Refactor cache compatibility checking and build targets loading/dependency analysis.
CacheVersion class refactoring:

Responsibilities of class CacheVersion are splitted into:
	- interface CacheAttributesManager<Attrs>, that should:
	    - load actual cache attribute values from FS
	    - provide expected attribute values (that is required for current build)
	    - checks when the existed cache (with actual attributes) values is suitable for current build (expected atribute values)
	    - write new values to FS for next build
	- CacheAttributesDiff is created by calling CacheAttributesManager.loadDiff extension method. This is just pair of actual and expected cache attributes values, with reference to manager. Result of loadDiff can be saved.

CacheAttributesDiff are designed to be used as facade of attributes operations: CacheAttributesDiff.status are calculated based on actual and expected attribute values. Based on that status system may perform required actions (i.e. rebuild something, clearing caches, etc...).

Methods of CacheAttributesManager other then loadDiff should be used only through CacheAttributesDiff.
Build system should work in this order:
    - get implementation of CacheAttributesManager for particular compiler and cache
    - call loadDiff __once__ and save it result
    - perform actions based on `diff.status`
    - save new cache attribute values by calling `diff.saveExpectedIfNeeded()`

There are 2 implementation of CacheAttributesManager:
    - CacheVersionManager that simple checks cache version number.
    - CompositeLookupsCacheAttributesManager - manager for global lookups cache that may contain lookups for several compilers (jvm, js).

Gradle:

Usages of CacheVersion in gradle are kept as is. For compatibility this methods are added: CacheAttributesManager.saveIfNeeded, CacheAttributesManager.clean. This methods should not be used in new code.

JPS:

All JPS logic that was responsible for cache version checking completely rewritten.

To write proper implementation for version checking, this things also changed:
	- KotlinCompileContext introduced. This context lives between first calling build of kotlin target until build finish. As of now all kotlin targets are loaded on KotlinCompileContext initialization. This is required to collect kotlin target types used in this build (jvm/js). Also all build-wide logic are moved from KotlinBuilder to KotlinCompileContext. Chunk dependency calculation also moved to build start which improves performance for big projects #KT-26113
	- Kotlin bindings to JPS build targets also stored in KotlinCompileContext, and binding is fixed. Previously it is stored in local Context and reacreated for each chunk, now they stored in KotlinCompileContext which is binded by GlobalContextKey with this exception: source roots are calculated for each round, since temporary source roots with groovy stubs are created at build time and visible only in local compile context.
	- KotlinChunk introduced. All chunk-wide logic are moved from KotlinModuleBuildTarget (i.e compiler, language, cache version checking and dependent cache loading)
	- Fix legacy MPP common dependent modules

Cache version checking logic now works as following:
- At first chunk building all targets are loaded and used platforms are collected. Lookups cache manger is created based on this set. Actual cache attributes are loaded from FS. Based on CacheAttributesDiff.status this actions are performed: if cache is invalid all kotlin will be rebuilt. If cache is not required anymore it will be cleaned.
- Before build of each chunk local chunk cache attributes will be checked. If cache is invalid, chunk will be rebuilt. If cache is not required anymore it will be cleaned.

#KT-26113 Fixed
#KT-26072 Fixed
2018-08-31 11:13:18 +03:00
Vyacheslav Gerasimov bc544d764a Refactor build repositories to reduce number of unnecessary request
Setup for all projects only local, jcenter and bootstrap repositories
2018-08-27 18:09:26 +03:00
Denis Zharkov 16a27d593c Optimize memory-footprint for RemoteLookupTrackerClient
Observations:
- In case of !requiresPosition we don't need to track position
- Also, in the latter case we don't need scopeKind (implicit contract)
- There were a lot of LookupInfo instances having two references
(fileName/scopeFqName) pointing to the same strings.
Looks like in common case (reasonable amount of files/fqnames) it's more
sensible to group lookups in the two-level-tree (implemented with nested maps)

It's expected to decrease memory consumption up to 15M in case of compiling
`idea` module in Kotlin project
2018-08-21 18:18:43 +03:00
Alexander Udalov 0f003802fe Introduce -Xcommon-sources and pass it correctly from build tool plugins
#KT-25196 In Progress
2018-08-21 12:49:10 +02:00
Alexey Tsvetkov 30f3622b27 Implement inter-project JS IC with Gradle
#KT-25025 fixed
2018-08-21 04:32:31 +03:00
Denis Zharkov 8063db5f80 Fix RemoteException that happens in JPS with compiler daemon
Recently, the container for "lookups" has changed to THashSet
And it lead to exception:
java.io.InvalidClassException: gnu.trove.THashSet;
local class incompatible: stream classdesc
serialVersionUID = -8659895033752433145,
local class serialVersionUID = -1699000958968314003

The reasons for different versions of THashSet are unknown
(though likely related to a slightly different classpathes),
but the fix is rather straight-forward

 #KT-26011 Fixed
2018-08-13 10:15:09 +07:00
Denis Zharkov 8050869e45 Use THashSet for sets that are assumed to be big
Otherwise, a lot of memory is wasted on nodes instances

(cherry picked from commit 4922f87)
2018-08-08 15:17:25 +07:00
Alexey Tsvetkov 0b18380770 Use separate system properties to control IC for JVM and JS
Also this commit effectively disables JS IC by default

     #KT-25563 fixed
2018-07-23 15:43:16 +03:00
Alexander Udalov 4122021090 Add BinaryVersion to DeserializationContext
This will be useful to implement version-dependent deserialization,
which is needed for gradual fixes of issues in metadata

 #KT-25120 In Progress
2018-07-18 17:58:46 +02:00
Vyacheslav Gerasimov 01db78d776 Build: Introduce Project extension properties for source sets 2018-06-22 21:42:30 +03:00
Vyacheslav Gerasimov 992a31af88 Build: introduce javaPluginConvention extension on project
`ExtensionAware.the<T>()` introduced in gradle 4.7 made existing calls `the<JavaPluginConvention>()` invalid (on wrong receiver)
2018-06-13 19:43:03 +03:00
Sergey Rostov 80ed963f97 jps, daemon: (minor) formatting 2018-06-08 09:46:53 +03:00
Sergey Rostov ce96c1b9a8 jps, daemon: fix javascript inline functions tracking
inline function hash should computed after compilation
2018-06-08 09:46:50 +03:00
Sergey Rostov c2d87140ec jps, daemon: fix JsFunction.toString() serialization 2018-06-08 09:46:43 +03:00
Sergey Rostov 1cbef873ec jps: implement mpp and js daemon compiler services
#KT-24635 fixed
2018-06-08 09:46:41 +03:00
Yan Zhulanow 662b79780d Kapt: Remove kapt1 support from Gradle and incremental compilation 2018-05-24 01:01:51 +03:00
Alexey Tsvetkov 53266300d4 Implement module detection for Gradle IC with Android
#KT-22431 fixed
2018-05-17 20:55:26 +03:00
Alexey Tsvetkov e5cdc64f4f Track changes for jar files for non-Android Gradle projects 2018-05-17 20:55:26 +03:00
Alexey Tsvetkov 6a45310830 Remove global artifact history cache in Gradle
Each Kotlin task now writes build history to separate file.
A map of output directories to history files is used to get changes for
modified files.

    #KT-22623 fixed
2018-05-17 20:46:52 +03:00
Ilya Chernikov e727e20973 [minor] Improve daemon exceptions logging 2018-05-08 15:59:45 +02:00
Vyacheslav Gerasimov 8c88ac69df Cleanup .181 bunch files 2018-04-27 18:25:21 +03:00
Vyacheslav Gerasimov bc403ce744 Switch to 181 platform 2018-04-27 18:25:17 +03:00
Denis Zharkov 003f381fcd Replace default-type dependencies to :kotlin-stdlib with distJar
Use distJar configuration instead.
It's necessary because currently when using default-type, subproject
starts having a transitive dependency to :kotlin-stdlib-common
and that leads to exception from KT-20897 when building light classes

This change might be reverted once KT-23942 is fixed

 #KT-23942 Submitted
2018-04-26 16:11:27 +03:00
Alexey Tsvetkov 13ac64b84a Move parseFileUtils to incremental-compilation-impl 2018-04-11 19:56:15 +03:00
Vyacheslav Gerasimov a0e20eefc6 181: Fix dependencies for 181 2018-04-11 16:28:29 +03:00
Alexey Tsvetkov 6f59b6da8e Use daemon if available for parsing files in JPS 2018-04-03 21:11:12 +03:00
Alexey Tsvetkov b420671a1d Introduce jps-compatible-base plugin to create necessary configurations
To avoid generating static accessors (via kotlinDslAccessorsSnapshot),
we need to create all necessary configurations during 'plugins'
block evaluation.
2018-03-26 16:09:29 +03:00
Alexey Tsvetkov 8a82c1618c Use Plugins DSL in Kotlin Gradle scripts
When plugins DSL is used, there is no need to
manually generate typesafe accessors for extensions and
conventions (by running `./gradlew kotlinDslAccessorsSnapshot`).
2018-03-26 16:09:29 +03:00
Alexey Tsvetkov 30d0cc3a34 Ensure all output directories are cleared on IC rebuild
In some cases IC needs to perform a rebuild.
Before this change IC was not clearing output directories
besides destination dir for classes, so for example
kapt stubs were not cleared.

Stalled stubs might lead to compile errors.
For example:
1. foo/XGen.java is generated from annotated class foo/X (XGen also
references X).
2. foo/X is moved to bar/X and some other change forces IC to rebuild.
3. kapt generates bar/X stub, but foo/X stub
was not removed because stubs dir is not cleared.
4. kapt runs annotation processors, foo/XGen.java is generated from
foo/X stub, bar/XGen.java is generated from bar/X stub.
5. kotlinc rebuilds. Since destination dir is cleared properly,
only bar/X.class exists.
6. javac tries to compile foo/XGen and fails, because it
compiles against actual Kotlin classes, not stubs.

This commit fixes the issue by passing all output directories
of a task from Gradle to Kotlin IC.

   #KT-21735 fixed
2018-03-19 17:39:47 +03:00
Alexey Tsvetkov c736f229ce Always depend on runtimeJar of kotlin-daemon-client 2018-03-14 13:29:32 +03:00
Yan Zhulanow 6e65a4810e Make a project-wide embeddedComponents configuration for embedding external binaries to project artifacts 2018-03-02 03:15:32 +03:00
Yan Zhulanow 07ede20dc5 Pill: Import KotlinPlugin artifact 2018-03-02 03:15:28 +03:00
Yan Zhulanow 34d1611ed4 Add Pill support to compiler and IDE modules 2018-03-02 03:15:17 +03:00
Ilya Chernikov d70b9c1d76 Detect new activity after delayed shutdown correctly
fixes #KT-22549, test

(cherry picked from commit 50f34db)
2018-01-31 10:25:29 +01:00
Ilya Chernikov 0a95e7b20f Fix dependencies and artifacts contents after review 2018-01-30 17:06:16 +03:00
Ilya Chernikov 4eb557724c Convert compiler projects to the new intellij deps 2018-01-30 17:06:13 +03:00
Ilya Chernikov a4f28cd94f Make all dependencies to idea sdk intransitive 2018-01-30 17:06:11 +03:00
Ilya Chernikov 78d2c655d5 Switch native-platform usages to regular dependency 2018-01-30 17:06:09 +03:00
Ilya Chernikov 74411d9b9c Convert all compiler modules to intellij plugin 2018-01-30 17:06:06 +03:00
Alexey Tsvetkov d1d786dffa Fix lookup tracking in JPS with enabled daemon
#KT-21962 fixed
2018-01-16 21:09:57 +03:00
Alexey Tsvetkov 220fab0d3f Test JPS with Daemon and IC 2018-01-16 21:09:57 +03:00
Ilya Chernikov 6f135e89d7 Make daemon starting more tolerant to the failures - retry
(cherry picked from commit 386cfec)
2018-01-10 13:38:20 +01:00
Alexey Tsvetkov dfe176efca Avoid computing source files changes in IC for Gradle 2017-12-22 16:12:19 +03:00