This change avoids looping through the source stack, looking for the
last element, in the case where the target stack is empty. This could
matter in two places:
* when merging ready_ and used_ into unswept_ in PrepareForGC. The first
of these will be faster with this change.
* when merging finalizer queues (in CR-592, not merged yet), where we
expect the target queue to be empty.
Since we can expect used_ to be larger than ready_, since we are about
to do a GC, the order of these two has also been reversed.
Co-authored-by: Troels Lund <troels@google.com>
Merge-request: KOTLIN-MR-599
Merged-by: Alexander Shabalin <alexander.shabalin@jetbrains.com>
The default behavior on a sweep is to push empty pages into a separate
stack, which will be freed if not used before the next GC starts. This
serves two purposes: it reduces the number of system allocations, and it
avoids a race condition inside AtomicStack::Pop.
Neither of these are relevant for LargePages, since LargePages are never
reused and it is only the GC thread that calls
AtomicStack<LargePage>::Pop. The change is to free LargePages
immediately instead of waiting for the next GC cycle.
Co-authored-by: Troels Lund <troels@google.com>
Merge-request: KOTLIN-MR-598
Merged-by: Alexander Shabalin <alexander.shabalin@jetbrains.com>
In before thread waiting for finalizers done could be unsuspended
before statistics about finalization is written.
Probably it doesn't affect real code, but it can lead to test failures.
Enable custom allocator with -Xallocator=custom.
If the gc is cms (default):
* a patched version of the memory manager is used that does not build a
list of allocated objects.
* a patched version of the cms is used that defers to the allocator for
sweeping.
Otherwise, a warning is printed, and the allocator is used using the
standard api.
Design doc:
https://docs.google.com/document/d/15xMp-nE-DWL8OrtOc8DoXB80AHUphFICEGjj5K0aNFc
Co-authored-by: Troels Lund <troels@google.com>
Merge-request: KOTLIN-MR-546
Merged-by: Alexander Shabalin <alexander.shabalin@jetbrains.com>
Calls checker was trying to read thread state while unlocking mutex in
unregister thread function. At this point thread is already
unregistered and reference to current thread node is dangling.
To avoid this, we nullify this reference in advance, as its anyway
explicitly passed to unregister function, not read from global.
If the count argument is above 2**29, then memberSize does not fit in
uint32_t.
Co-authored-by: Troels Lund <troels@google.com>
Merge-request: KOTLIN-MR-533
Merged-by: Alexander Shabalin <alexander.shabalin@jetbrains.com>
* Add fallible push to intrusive_forward_list.
* Add fallible pop to intrusive_forward_list.
* Get rid of size_ member in intrusive_forward_list.
* Force all intrusive_forward_list items to have non-null next().
* Get rid of explicit colors in STMS and CMS.
* Do not queue weak reference for later processing in Mark.
Merge-request: KT-MR-7263
Merged-by: Alexander Shabalin <Alexander.Shabalin@jetbrains.com>
- Introduce ComparableTimeMark interface extending TimeMark,
and TimeSource.WithComparableMarks - a time source that returns such time marks.
- Implement ComparableTimeMark in ValueTimeMark and AbstractLong/DoubleTimeMark classes.
Previous implementation of startCoroutineUninterceptedOrReturn expected
that the receiver (i.e. the suspend function object to start) of type
SuspendFunctionN (i.e. suspend function type with arity = N) is also an
instance of FunctionN+1 (i.e. regular function type with arity = N + 1)
with proper convention.
While compiler tries to achieve that (for better compatibility with
Kotlin/JVM, by generating additional supertypes and methods for classes
implementing suspend function types), this doesn't work e.g. for
implementations defined in Swift.
Fix this by adding fallbacks for cases when the receiver is not
a FunctionN+1, while keeping the fastpath for default FunctionN+1
cases.
^KT-51043 Fixed
Make `createCoroutineUnintercepted` invoke
`startCoroutineUninterceptedOrReturn` when suspend function object is
not a `BaseContinuationImpl` (e.g. for a custom suspend function type
subclass), instead of using the copy of its implementation.
This change parallelizes marking by making each non-native Kotlin thread
mark its own view of the heap. To make this safe, we make flipping the
marking bit atomic which ensures that threads do not try to trace the
same objects.
Co-authored-by: Johan Bay <jobay@google.com>
Merge-request: KOTLIN-MR-423
Merged-by: Alexander Shabalin <alexander.shabalin@jetbrains.com>
Troels Bjerre Lund