Add build regression benchmark script template

This template provides all infrastructure to write Kotlin scripts for
build regression benchmarks. These benchmarks will use gradle-profiler
to run provided scenarios on user projects. All benchmark results
in the script then will be aggregated into single one showing
difference between them.

Generally such kind of benchmarks should be used to track early Gradle
 build regressions between releases.

^KT-49921 In Progress
This commit is contained in:
Yahor Berdnikau
2021-12-20 14:36:58 +01:00
committed by Space
parent 8a58e98530
commit 7418d3c898
10 changed files with 655 additions and 0 deletions
@@ -0,0 +1,18 @@
## Description
Provides Kotlin script template to define and run [gradle-profiler](https://github.com/gradle/gradle-profiler) benchmarks.
### Writing benchmark script
This template is automatically applied to all script files with `.benchmark.kts` file extension.
The script itself should have following:
- A `@file:BenchmarkProject` annotation providing name of the project-under-benchmark,
git url to the project repo and git commit sha which should be used to run benchmarks.
- definition of gradle-profiler scenarios is written in `suite { .. }` DSL.
- provide optional git patch files that will be applied to the project before running benchmarks.
To run the actual benchmarks script either could call specific steps or just call `runAllBenchmarks()` function.
This function calls required steps in order, runs provided benchmarks and then shows aggregated result.
Aggregated results are available in form of CSV or HTML files.