Fixed and extended performance tests

This commit is contained in:
Vladimir Dolzhenko
2019-06-11 10:08:30 +02:00
parent 5010824167
commit 47a16cb252
11 changed files with 573 additions and 56 deletions
+42
View File
@@ -0,0 +1,42 @@
# Goal
The main goal of these performance tests is to collect statistics over
routines (open project, highlighting, inspection, autocompletion etc) for
further analysis like anomalies, degradations and reference for optimizations.
## Resources
- Clone another copy of kotlin project to `${KOTLIN_PROJECT}/../perfTestProject`
- `${KOTLIN_PROJECT}/idea/testData/perfTest/helloKotlin` as _hello world_ project in Kotlin
## Performance test types
### AbstractKotlinProjectsPerformanceTest
Subclasses of `AbstractKotlinProjectsPerformanceTest` provide statistics
(execution time in **ms**) over some **particular** (semi-synthetic) actions.
The output is provided to console in TeamCity format like
```
##teamcity[testSuiteStarted name='Kotlin']
##teamcity[testStarted name='Project opening perfTestProject' captureStandardOutput='true']
##teamcity[testMetadata key='Project opening perfTestProject' type='number' value='63']
##teamcity[testFinished name='Project opening perfTestProject' duration='63']
##teamcity[testSuiteFinished name='Kotlin']
```
as well in `build/stats.csv`
File | ProcessID | Time
--- | --- | ---
HelloMain.kt | Highlighting file warm-up HelloMain.kt | 114
KtFile.kt | Highlighting file KtFile.kt | 1255
### WholeProjectPerformanceTest
**TBD**: drop `@Ignore` from some test cases
Subclasses of `WholeProjectPerformanceTest` collect performance metrics to
figure out some significant anomalies (e.g. inspection takes ages or
even **fails**) over huge number of files (like `kotlin` project itself)
on various **real** use cases.