ClassCodegen bails before generating code if the filter supplied to
the GenerationState rejects it based on the containing file or class
declaration itself.
All tests in this commit _pass_ to document existing behavior, but
this will change in a separate commit as we reflect desired behavior
in the test expectations.
This commit introduces support for calling and referencing local functions and
objects in evaluate expression on the IR backend.
The primary incision is a lowering inserted after Local Declaration Lowering,
that uses the intermediate data structures recorded by LDL to rewrite calls to
local functions to the appropriate function in the binary, instead of predicting
the compilation strategy. The required changes to the rest of the pipeline
facilitate piping the required data around.
The key to this transformation is that _captures by the local function_ must be
introduced as _captures by the fragment function_, such that the evaluator
infrastructure can find the appropriate values at run-time. This is necessary
due to the strategy of compiling local functions to static functions instead of
closures.
Additional test coverage of stepping behavior support the corresponding changes
in the Evaluator, part of the Kotlin Debugger plug-in.
Inline functions called by the fragment is detected by the fragment
frontend and the PSI for them is supplied to the compiler backend.
For the purpose of compilation, the inline function is considered as
part of the same source module as the fragment, and then discarded
upon loading the compiled fragment for evaluation.
This commit changes the slice implementation used for
`USED_AS_EXPRESSION`.
Here is the problem as identified and mitigated by this commit:
Slices are key-value maps. They are not total over the domain of keys,
so "looking up" a key not in the slice is a valid operation, yielding
`null`.
A binding context is a collection of slices.
Binding contexts can be stacked as composite binding contexts to
e.g. facilitate scoped analyses. Looking up a key proceeds down the
stack, yielding the first non-null value, or `null` if no slice in the
stack of binding contexts contain that key.
A slice made by `createSimpleSetSlice` models a "set" of keys that can
be enlarged by inclusion: adding `(k, true)` to the slice indicates
`k` belongs to the set. `(k, false)` indicates non-membership.
However, looking up a key _not in_ the slice yields _false_, rather
than null.
Hence, simple set slices do not compose in composite binding traces.
This was encountered porting the expression evaluator to the IR
backend. PSI2IR uses `USED_AS_EXPRESSION` to generate expression body
functions properly. The frontend analysis of the fragment is layered
ontop of the binding context from the editor's analysis of the
underlying project being debugged, which contains speficially the
analysis results of inline functions called from the fragment.
evaluator fragments
- Refactor JvmIrCodegenFactory to allow for caller's choice of
lowerings.
- Refactor of SyntheticAccessorLowering to expose the accessibility
check used to determine the need for accessors.
- Replacement of field, method and constructor uses with corresponding
java.lang.reflection API via additional lowering running _before_
the existing IR pipeline.
- Emulating super calls in the context of class methods via new
compiler intrinsic.
- psi2ir: Add support for `_field` suffix on properties in the
evaluator for accessing underlying fields when applicable.
- Add `JvmGeneratorExtensions` point for determining whether a
property accessor has a user-supplied body in order to predict how
they are ultimately compiled for the JVM.
This commit introduces a new entrypoint for the IR backend, and starts
the work of accomodating the evaluator in psi2ir.
The evaluator expects a certain class and method structure of the
compiled fragment, but PSI is only supplied for the actual fragment.
So, to that end, this commit introduces a new "front end" of psi2ir
that "synthesizes" the module, class and method structure around the
fragment before calling into the existing psi2ir pipeline to obtain
the IR for the fragment.
The primary complication so far is handling the captured variables of
the fragment: they are mapped to parameters of the method surrounding
the fragment, and passed as arguments on evaluation. Hence, the IR
translation of the fragment needs to remap captured variables to the
appropriate parameter, in essentially all places they can be referred
to in the fragment (and hence in psi2ir). This commit introduces a
decorated symbol table that intercepts symbol look-ups and remaps as
appropriate.
Other cases that dispatches based on descriptor (see
`CallGenerator.kt`) needs other metadata to generate correct code.
It also introduces a shim in DeserializedContainerSource in the psi2ir
pipeline to facilitate facade class generation for the code that is
being debugged (which are generated as "external ir declarations").
Finally, in passing we resolve a small leftover from previous
refactoring that left an asssertion re. allowing IR to _assign_ to
parameters of methods.
This solves no immediate deficiency, but is a to-do that arose as part
of an ongoing effort to port the old, slightly too restrictive
checkLocalVariableTable tests to the more "functional" debugger
infrastructure.
This just shaves a little bit of overhead off the test expectations
and couples them less tightly to the specifics of the test
infrastructure.
This change improves the debugging experience around local functions
on the IR backend. The changes include moving old
checkLocalVariablesTable (cLVT) tests to the new stepping/local variable
infrastructure in order to refine the tests and further define the
behavior of the two JVM backends, and their differences.
The primary ported test case is cLVT/localFun.kt that documents the
discrepancy in implementation strategy for local functions on the two
backends. The old backend implements local functions as lambdas
assigned to a local variable while the IR backend lifts them out as
static funtions on the surrounding class. The discrepancies and their
consequences are documented in bytecodeListing, idea-stepping,
localVariableTable and debugStepping tests.
The only _code change_ is disabling the captured variable name
mangling for captured variables on the IR backend. Captured variables
are passed as arguments to the static function, so in the debugger,
they really just are local variables. For them to show properly in the
debugger and be detectable by evaluate expression, they simply need no
mangling.
Finally, this change cleans 3 redundant cLVT tests, copyFunction.kt
and destructuringInlineLambda.kt and destructuringInFor.kt, that are
all covered in the new suite. The stepping behavior needs to be made
precise around for loops, but that is an entirely seperate issue.
Reveals discrepancy in LVT presence on lambda implementations on the
old and new backend.
The generated code in the constructors of Suspend Lambda objects is
identical, but the IR backend generates an LVT with the constructor
parameters.
The user has to be very insistent to see this ("for step into" +
disabling "Show only kotlin variables"), but it is an observable
difference.
The existing backend restores LVs and parameters from the suspend lambda
fields used for spilling between suspension points, hence they are
visible in the debugger as local variables, plain and simple.
This PR introduces the same pattern to the IR backend, to bring the
debugging experience in line with the existing backend.
Both backends are still at the mercy of the liveness analysis
performed in the coroutine transformer where a liveness analysis
minimizes live ranges of entries in the LVT. E.g. an unused parameter
will be dropped entirely.
Adjusted existing test expectations accounting for the differences in
LV behavior.
The debug experiece of destructuring patterns in lambdas is different
across the two backends due to the IR backend moving local variables
to fields.
However, since the destructuring variable is never actually visible in
the debugger (no linenumbers in the live range of the variable), and
the variable is never used for anything other than hiding it from the
debugger, we propose that it is not actually necessary to include it
in the LVT (and in fact, could be left out of the LVT on the old
backend).
This commit unifies the expectation format between the stepping and
LVT tests in the new debugging test harness.
Furthermore, it introduces a compression of runs of locations without
linenumbers. These default to showing as bytecode offsets from
previous line number, which overspecifies the tests: the bytecodes
chosen should not be constrained by a debugging step test.
This is the first step in a push to improve the test coverage of the
IR debugging experience. This commit improves on the sketch of local
variable table tests built on the new debugger stepping
infrastructure. As improvements on the existing
checkLocalVariableTableTest they:
- don't overspecify codegen strategy: no hard requirements on slots.
- test the observed lifespans of locals by stepping through code.
Ultimately this should enable us to bring over all existing tests for
improved coverage of both old and new JVM backends.
WIP list:
- Gracefully handle absent LVT: treat as empty Add type of local and
- type of value stored there to expectations Print values in local
- slots if primitive or java.lang.String Enable specifying
- expectations per backend Gracefully handle absent box methods
- Gracefully handle null values in slots of reference type
- Port a first LVT test
- unmute tests
- add test to ensure JVM target is respected
- add test to cover smart-casted cases
- implement function matching and replacement
- Switching on uint constants
- introduce lowering for standard library replacements
... for an _even leaner_ codegen!
- move discard from extention method to abstract method. This will at
length avoid some calls to materialize.
- use newly abstract method to specialize discard in cases where it
would introduce _and_ coerce values only to be popped by discard.
- move coerce to member method, introduce materializeAt with a view to
eliminate calls to coerce outside of PromisedValue. This will allow
us to specilaize materialization at particular types in some
instances, at length avoiding casts etc.
- no calls to coerce outside of PromisedValue.kt! Progress.
- inlined coerce into materializeAt, anticipating specializing
materializeAt across implementations of PromisedValue
- made materializeAt abstract, copied implementation everywhere!
- made materialize an extention function! Ready to specialize materializeAt
- coerce is no more!
- simplified and specialized all inlinings of materializeAt.
- adjust docs to match refactoring
This commit ports the (parameterless) main integration tests in
`CompilerSmokeTest` to the IR backend. It also includes a simple
suspend main test.
The advanced ones (like `helloAppSuspendParameterlessMain`) are
currently blocked by pending changes to capturing suspend lambdas,
which are underway.
This adds supports for (parameterless) suspend main entry points for
the JVM IR backend.
In case main is a suspend function, it gains a continuation during
lowering, so we simply generate a plain old `public static void
main(String[] args)`. This entry point invokes `suspend main` via
`kotlin.coroutines.jvm.internal.RunSuspendKt#runSuspend`.
This PR introduces `runSuspend` as a built-in, and generates the
following `main`, passing `args` as appropriate:
```
fun main(args: Array<String>) {
runSuspend { main(args) }
}
```
The phase ordering has been reshuffled countrary to previous
discussion on #2780, as the MainMethodGeneration pass now introduces lambdas in
the IR. Hence, it has to run before InventNamesForLocalClasses, yet
still after JvmOverloadsAnnotations.
Some dead code was discovered in AddContinuationLowering
- Uncomment tests
- Add proper visibility to companion field
+ Make exception for interfaces -> they need to respect language versions 1.8
before they can put private members there.
- Adjust synthetic accessor lowering to look for enclosing classes with access
via companion object.
This ensures correct generation of generic signatures in the resulting
byte code, but it _is_ a work in progress: the actual type *arguments*
passed for these parameters during compilation are dummy `Any?` types.
Sites that need more work are indicated with TODO's.
- copy type parameters of interfaces to methods moved to DefaultImpls
- implement type parameter renaming scheme from JVM, with proper
renaming and substitution.
- adjust call sites in bridges in classes->DefaultImpls
- adjust call sites in bridges from DefaultImpls->Interface
- adjust call sites in bridges from DefaultImpls->DefaultImpls
- adjust super calls ->DefaultImpls
- adjust calls in code of Interfaces->DefaultImpls
When a property backed by a final field is the delegate for interface
implementation by delegation, the backend should reuse the backing
field rather than introduce a new, extraneous `$$delegate_n` field.
Rename and refactor interface delegation phase. It didn't actually do
what was said on the box. It should be more in line with the
intentions of the phase now.
Like the old backend, always leave private @JvmDefault annotated
interface members (properties, methods) on the interface, just like
the old backend. Fix naming, and introduce test to document the naming
scheme.
- introduce a scoped counter instead of a global one for name
generation for accessors. Naive solution not working.
- Introduced hardcoded "jd" suffix for accessors on interfaces, under
the assumption that the only such accessors are due to JvmDefault
and their bridges from `$DefaultImpls`. Removed all associated
templated tests, so the old and IR backend correspond on this matter
again.
- Respecialized writeFlags from regexps to string-equality: we are
going for exact matches now!
- Fixed package calculation in `IrUtils.kt`.
- Accessors for static members must be due to accessing super
classes. Actual super-qualified calls are naturally also accessing
super classes. Hence the `$s+{hashcode(superClassName)}`
suffix. Added test to affirm this naming scheme.
- Field getters/setters for static fields must be companion accessors,
otherwise just labelled as accessors. They are also tagged with `s`
suffix when accessing static fields.
- For naming of accessors to coincide with the old backend, field
renaming to avoid JVM signature clashes must be done _after_
generation of accessors for those fields.
Change the treatment of default implementations on interfaces in JVM
compatibility mode. Previously, the IR backend moved the actual
default implementation to the DefaultImpls class, and then bridged to
it from the interface default. The old backend did the reverse, at the
cost of an additional accessor, in order to gain better binary
compatibility properties. See #2612 for discussion.
The accessor needs to call a specific implementation, so must be
performed through an `invokespecial`. We trick the
SyntheticAccessorLowering into doing this for us, by marking the
bridging call as a super call. We do this in want of an explicit
`invokespecial` Ir Node.
InterfaceDefaultCallsPhase previously assumed the old behaviour of the
IR backend (that calls to default implementations, e.g. `foo$default`
should target `DefaultImpls.foo$default`). But now the bridge to
foo$default resides on `DefaultImpls` already, causing that pass to
create a recursive loop. We cut that loop with a simple check.
The JVM IR was too agressive in delegating statically to $DefaultImpls
class.
Consider the following library, compiled separately, and then client,
compiled against that library:
```
//library
interface A { @JvmDefault fun foo() = "A" }
interface Left : A { }
interface Right : A { @JvmDefault override fun foo() = "Right" }
//client
interface C : Left, Right {}
fun main() {
val x = object : C {}
println(x.foo())
}
```
Previously, the IR backend generates an overriding bridge in C, which
calls C$DefaultImpls which calls statically into Right$DefaultImpls.
When then library is recompilerd with an override of foo in Left, the
existing binary of the client continues to run against the recompiled
library erroneously running Right.foo().
The old backend throws an IncompatibleClassChange exception due to
ambiguity of multiple inherited default implementations. kotlinc indeed
rejects the client when recompiled against the new library.
We have no tests constraining this behaviour, but we could conceivably
strap something together in CustomBinaries tests.
Resolves the interaction of @JvmOverloads annotations and
parameterless main methods.
In the following code, both mechanisms generate methods that
ultimately produce the signature `public static void main(String[] args)`
of which there can be only one (true in general of any signature).
```
fun main() { }
@JvmOverloads
fun main(Array<String> args, x: Int = 42) { }
```
This PR simply shuffles the lowerings around, letting parameterless
main methods detect the presence of the default overload produced by
the annotation.
Additionally, this PR improves the testing of parameterless main
methods by actual bytecode patterns, and not simple check for
successful compilation (as @sfs and I discovered, there are issues in
flagging an error on duplicate signatures on the IR backend).
- Added tests to demonstrate broken behaviour: the interaction of inline
functions and callable references with varargs and defaults in various
combinations.
- Refactored InlineCallableReferencesToLambdaPhase to look like and use
some of the infrastructure from CallableReferenceLowering.
- Lifted some of this infrastructure out to be broadly reusable.
- Extract all backend codegen tests that specifically target behaviour
in to-be-deprecated functionality from language versions < 1.3"
- Remove those tests from the JVM IR test suite.
This commit:
- introduces tests explicating what is and isn't considered a
proper main method on the JVM backends.
- implements support for parameterless main methods on the JVM IR
backend
- See KT-34338 for more tests.
Kristoffer Andersen