There are several performance optimizations:
* ByteBuffer/CharBuffer/StringBuilder objects pre-allocated and are
reused on each call to readLine.
* The state for readLine is lazily allocated via JVM classloading
(using a singleton object).
* There is an auto-detection heuristic for "directEOL" encodings which
represent LF ('\n') directly as the corresponding byte
(UTF-8 and many single-byte encodings are like that).
When "directEOL" encoding is used, then bytes are batched into
ByteBuffer for a single call to CharsetDecoder.decode which
results in higher throughput. Otherwise (UTF-16, etc), slower
byte-by-byte approach is used.
* Bytes and chars are directly moved in/out of byte/char arrays and
ByteBuffer/CharBuffer wrappers are used only to interface with
JVM CharsetDecoder class (which is the slowest piece).
* StringBuilder is not used at all for short lines (<=32 chars).
There are also some function improvements to readLine functionality:
* Restriction on "max chars per byte" is lifted, so readLine works with
all encodings that JVM supports.
* It support on-the-fly changes to system default charset, because
it rechecks current charset on each call and updates it decoder
when needed.
All the other features of readLine function are retained:
* It does not read more bytes from System.in than needed, so it
is compatible with other ways to read System.in. On-the-fly
changes to System.in are supported.
* It is thread-safe. Its internal mutable state is protected by
synchronization.
* There is an internal method for tests that supports explicit
charset specification, but the name of this method has changed.
There are additional tests:
* Check all supported encodings on JVM to make sure that readLine
works correctly with them all.
* Check unicode code points of different bits length with all standard
unicode encodings (UTF-8, UTF-16, and UTF-32 in LE/HE byte orders).
Benchmarks that compare different implementations of readLine,
including this one (readLine6NoLV in the set) can be found here:
https://github.com/elizarov/ReadLineBenchmark
Taking BufferedReader as 100% baseline we see that:
* Current readLine is 7.5 times slower than BufferedReader baseline.
* New implementation in this commit is 2.5 timer slower than baseline.
It is ~3 times faster than existing implementation of readLine.
Altogether these optimizations are enough to enable reading of
~500K lines in sports programming setting under 2s time-limit with
plenty of headroom in time. Example that is using this version of
readLine can be found here:
https://codeforces.com/contest/1322/submission/73005366
#KT-37416 Fixed
*given that it runs on JDK7 and higher. The addSuppressed member
is called with reflection when it's available.
kotlin-stdlib-jdk7 extension still overrides that and calls
addSuppressed member statically as before.
#KT-30560 Fixed
CharBuffer had capacity to hold 2 chars, which was not enough to append
a surrogate pair when the buffer was not empty.
The buffer was extended and the decoding algorithm rewritten to deal with
a buffer of any length.
#KT-28572 Fixed
Instead of mutating the matcher create a new one when `next()` is called.
This allows getting named groups from that matcher later.
Add lookbehind in matchSequence test to ensure this change doesn't alter
the existing behavior.
Also fixes #KT-20865
Throw an exception immediately before an overflow becomes observable.
Place check to prevent negative index from indexOf, indexOfFirst.
Do not insert overflow checks for arrays, lists, maps and char sequences.
#KT-16097
Ilya Gorbunov