From b8e1c14b8e8dfeb37947ebf80f3d45b208e261af Mon Sep 17 00:00:00 2001
From: Vladimir Sukharev <vladimir.sukharev@gmail.com>
Date: Tue, 9 Aug 2022 10:30:38 +0000
Subject: [PATCH] Improve benchmarking docs

Merge-request: KT-MR-6808
Merged-by: Vladimir Sukharev <Vladimir.Sukharev@jetbrains.com>
---
 kotlin-native/HACKING.md | 73 +++++++++++++---------------------------
 1 file changed, 24 insertions(+), 49 deletions(-)

diff --git a/kotlin-native/HACKING.md b/kotlin-native/HACKING.md
index 0e546626064..620a8642861 100644
--- a/kotlin-native/HACKING.md
+++ b/kotlin-native/HACKING.md
@@ -104,7 +104,8 @@ In common, they are called "target-specific tests".
 
 To run Kotlin/Native target-specific tests use (takes time):
 
-    ./gradlew :kotlin-native:backend.native:tests:run
+    ./gradlew :kotlin-native:backend.native:tests:sanity  2>&1 | tee log                             # quick one
+    ./gradlew :kotlin-native:backend.native:tests:run  2>&1 | tee log                                # sanity tests + platform tests
 
 * **-Pprefix** allows one to choose tests by prefix to run.
 
@@ -245,24 +246,19 @@ so it's need to be additionally enabled while compiling application with `-Xbina
 compiler flag or corresponding setting in gradle build script. After doing this, Kotlin/Native runtime in application
 is debuggable in CLion, with Attach to process tool.
 
+
  ## Performance measurement
-  
- Firstly, it's necessary to build analyzer tool to have opportunity to compare different performance results:
- 
-    cd kotlin-native/tools/benchmarksAnalyzer
-    ../../../gradlew build
-    
+ ### Pre-requisite
+  **konanRun** task needs built compiler and platform POSIX libs. To test against working tree make sure to run
+
+    ./gradlew :kotlin-native:dist :kotlin-native:distPlatformLibs
+
+ ### Run tests
  To measure performance of Kotlin/Native compiler on existing benchmarks:
  
     cd kotlin-native/performance
     ../../gradlew :konanRun
-
- **NOTE**: **konanRun** task needs built compiler and libs. To test against working tree make sure to run
-
-    ./gradlew :kotlin-native:dist :kotlin-native:distPlatformLibs
-
- before **konanRun**
-    
+ 
  **konanRun** task can be run separately for one/several benchmark applications:
  
     cd kotlin-native/performance
@@ -272,6 +268,7 @@ is debuggable in CLion, with Attach to process tool.
  
     cd kotlin-native/performance
     ../../gradlew :cinterop:konanRun --filter=struct,macros
+    ../../gradlew :ring:konanRun --filter=Euler.problem9,ForLoops.charArrayIndicesLoop
     
  Or you can use `filterRegex` if you want to specify the filter as regexes:
  
@@ -292,22 +289,25 @@ is debuggable in CLion, with Attach to process tool.
  
     cd kotlin-native/performance
     ../../gradlew :jvmRun
-    
- Files with results of benchmarks run are saved in `performance/build/nativeReport.json` for konanRun and `jvmReport.json` for jvmRun.
- You can change the output filename by setting the `nativeJson` property for konanRun and `jvmJson` for jvmRun:
-
-    cd kotlin-native/performance
-    ../../gradlew :ring:konanRun --filter=String.*,Loop.* -PnativeJson=stringsAndLoops.json
 
  You can use the `compilerArgs` property to pass flags to the compiler used to compile the benchmarks:
 
     cd kotlin-native/performance
     ../../gradlew :konanRun -PcompilerArgs="--time -g"
 
- To compare different results run benchmarksAnalyzer tool:
- 
-    cd kotlin-native/tools/benchmarksAnalyzer/build/bin/<target>/benchmarksAnalyzerReleaseExecutable/
-    ./benchmarksAnalyzer.kexe <file1> <file2>
+ ### Analyze the results
+ Files with results of benchmarks run are saved in `kotlin-native/performance/build` folder: `nativeReport.json` for konanRun and `jvmReport.json` for jvmRun.
+ You can change the output filename by setting the `nativeJson` property for konanRun and `jvmJson` for jvmRun:
+
+    cd kotlin-native/performance
+    ../../gradlew :ring:konanRun --filter=String.*,Loop.* -PnativeJson=stringsAndLoops.json
+
+ To compare different results use benchmarksAnalyzer tool:
+
+    ./gradlew macos_arm64PlatformLibs  # use target of your laptop here instead
+    cd kotlin-native/tools/benchmarksAnalyzer
+    ../../../gradlew build
+    ./build/bin/<target>/benchmarksAnalyzerReleaseExecutable/benchmarksAnalyzer.kexe <file1> <file2>
     
  Tool has several renders which allow produce output report in different forms (text, html, etc.). To set up render use flag `--render/-r`.
  Output can be redirected to file with flag `--output/-o`.
@@ -337,31 +337,6 @@ is debuggable in CLion, with Attach to process tool.
 
     ./benchmarksAnalyzer.kexe -f <file1> <file2>
 
-### Testing native
-
-For a quick check use:
-```
-$ ./gradlew :kotlin-native:backend.native:tests:sanity 2>&1 | tee log
-```
-
-For a longer, more thorough testing build the complete build. Make sure you are running it on a macOS. 
-
-
-Have a complete build:
-
-```
-$ ./gradlew :kotlin-native:bundle # includes dist as its part
-```
-
-then run two test sets:
-
-```
-$ ./gradlew :kotlin-native:backend.native:tests:run 2>&1 | tee log
-
-$ ./gradlew :kotlin-native:backend.native:tests:runExternal -Ptest_two_stage=true 2>&1 | tee log
-
-```
-
 ## LLVM
 
 See [BUILDING_LLVM.md](BUILDING_LLVM.md) if you want to build and use your own LLVM distribution