[K/N] Update documentation on building and using LLVM

kotlin-native/BUILDING_LLVM.md

@@ -1,56 +1,21 @@
-# Building Apple LLVM for Kotlin/Native
+# Building LLVM for Kotlin/Native
 
-This document describes how to compile LLVM distribution and use it to build Kotlin/Native on macOS.  
+This document describes how to compile LLVM distribution and use it to build Kotlin/Native.  
 Usually, you don't need to compile LLVM by yourself: it is downloaded
 automatically when you run or build Kotlin/Native compiler.
 
 But if you don't want to download prebuilt LLVM or want to experiment with your own distribution, 
-you came to the right place.
+you come to the right place.
 
-## Part 1. Building the right LLVM version for macOS.
+## Part 1. Building the right LLVM version.
 
-For macOS host we use LLVM from [Apple downstream](https://github.com/apple/llvm-project). 
+Use [package.py](tools/llvm_builder/README.md) script to build LLVM distribution the same way the Kotlin team does.
-Branch is [**apple/stable/20190104**](https://github.com/apple/llvm-project/tree/apple/stable/20190104) 
-because it is similar (or even the same) to what Apple ships with Xcode 11.*. 
-After cloning the repo and changing the branch we perform the following steps to build LLVM toolchain:
-
-```bash
-mkdir build
-
-cd build
-
-cmake -DLLVM_ENABLE_PROJECTS="clang;lld;libcxx;libcxxabi" \
- -DCMAKE_BUILD_TYPE=Release \
- -DLLVM_ENABLE_ASSERTIONS=Off \
- -G Ninja \
- -DCMAKE_INSTALL_PREFIX=clang-llvm-apple-8.0.0-darwin-macos \
- ../llvm
-
-ninja install
-```
-
-After these steps `clang-llvm-apple-8.0.0-darwin-macos` directory will contain LLVM distribution that is suitable for building Kotlin/Native.
 
 ## Part 2. Building Kotlin/Native against given LLVM distribution.
 
-By default, Kotlin/Native will try to download LLVM distribution from CDN if it is not present in `$HOME/.konan/dependencies` folder. 
+To do so, we need to edit [konan.properties file](konan/konan.properties):
-There are two ways to bypass this behaviour.
+1. `llvmHome.<HOST_NAME>` should point to the freshly built LLVM distribution.
-
+2. `llvmVersion.<HOST_NAME>` should specify its version (for example, `11.1.0`).
-#### Option A. Substitute prebuilt distribution.
-This option doesn't require you to edit compiler sources, but a bit harder.
-
-The compiler checks dependency presence by reading contents of `$HOME/.konan/dependencies/.extracted` file. 
-So to avoid LLVM downloading, we should manually add a record to the `.extracted` file:
-1. Create `$HOME/.konan/dependencies/.extracted` file if it is not created.
-2. Add `clang-llvm-apple-8.0.0-darwin-macos` line.
-
-and put `clang-llvm-apple-8.0.0-darwin-macos` directory from the Part 1 to `$HOME/.konan/dependencies/`.
-
-#### Option B. Provide an absolute path to the distribution.
-This option requires user to edit [konan.properties file](konan/konan.properties).
-Set `llvmHome.<HOST_NAME>` to an absolute path to your LLVM distribution and 
-set `llvmVersion.<HOST_NAME>` to its version.
-For example, provide a path to `clang-llvm-apple-8.0.0-darwin-macos` from the Part 1 and set version to 8.0.0.
 
 Now we are ready to build Kotlin/Native itself. The process is described in [README.md](README.md).
 
@@ -59,5 +24,5 @@ Now we are ready to build Kotlin/Native itself. The process is described in [REA
 — Can I override `.konan` location?  
 — Yes, by setting `$KONAN_DATA_DIR` environment variable. See [HACKING.md](HACKING.md#compiler-environment-variables).
 
-- Can I use another LLVM distribution without rebuilding Kotlin/Native?
+— Can I use another LLVM distribution without rebuilding Kotlin/Native?  
-- Yes, see [HACKING.md](HACKING.md#using-different-llvm-distributions-as-part-of-kotlinnative-compilation-pipeline).
+— Yes, see [HACKING.md](HACKING.md#using-different-llvm-distributions-as-part-of-kotlinnative-compilation-pipeline).

kotlin-native/HACKING.md

@@ -367,22 +367,15 @@ instead of provided one.
 
 ### Using different LLVM distributions as part of Kotlin/Native compilation pipeline.
 
-`llvmHome.<HOST_NAME>` variable in `<distribution_location>/konan/konan.properties` controls 
+`-Xllvm-variant` compiler option allows to choose which LLVM distribution should be used during compilation.
-which LLVM distribution Kotlin/Native will use in its compilation pipeline. 
+The following values are supported:
-You can replace its value with either `$llvm.<HOST_NAME>.{dev, user}` to use one of predefined distributions
+* `user` — The compiler downloads (if necessary) and uses small LLVM distribution that contains only necessary tools. This is what compiler does by default.
-or pass an absolute to your own distribution.
+* `dev` — The compiler downloads (if necessary) and uses large LLVM distribution that contains additional development tools like `llvm-nm`, `opt`, etc.
-Don't forget to set `llvmVersion.<HOST_NAME>` to the version of your LLVM distribution.
+* `<absolute path>` — Use local distribution of LLVM.
-
-#### Example. Using LLVM from an absolute path.
-Assuming LLVM distribution is installed at `/usr` path, one can specify a path to it 
-with the `-Xoverride-konan-properties` option:
-```
-konanc main.kt -Xoverride-konan-properties=llvmHome.linux_x64=/usr
-```
 
 ### Playing with compilation pipeline.
 
-Following compiler phases control different parts of LLVM pipeline:
+The following compiler phases control different parts of LLVM pipeline:
 1. `LinkBitcodeDependencies`. Linkage of produced bitcode with runtime and some other dependencies.
 2. `BitcodeOptimization`. Running LLVM optimization pipeline.
 3. `ObjectFiles`. Compilation of bitcode with Clang.
@@ -400,20 +393,16 @@ Please note:
 3. Use `clang -cc1 -help` to see a list of available options.
  
 Another useful compiler option is `-Xtemporary-files-dir=<PATH>` which allows
-to specify a directory for intermediate compiler artifacts like bitcode and object files.
+ specifying a directory for intermediate compiler artifacts like bitcode and object files.
+For example, it allows to store LLVM IR after a particular compiler phase.
 
-#### Example 1. Bitcode right after IR to Bitcode translation.
 ```shell script
-konanc main.kt -produce bitcode -o bitcode.bc
+konanc main.kt -Xsave-llvm-ir-after=BitcodeOptimization -Xtemporary-files-dir=<PATH>
 ```
 
-#### Example 2. Bitcode after LLVM optimizations.
+`<PATH>/out.BitcodeOptimization.ll` will contain LLVM IR after LLVM optimization pipeline.
-```shell script
-konanc main.kt -Xtemporary-files-dir=<PATH> -o <OUTPUT_NAME>
-```
-`<PATH>/<OUTPUT_NAME>.kt.bc` will contain bitcode after LLVM optimization pipeline.
 
-#### Example 3. Replace predefined LLVM pipeline with Clang options.
+#### Example: replace predefined LLVM pipeline with Clang options.
 ```shell script
 CLANG_FLAGS="clangFlags.macos_x64=-cc1 -emit-obj;clangNooptFlags.macos_x64=-O2"
 konanc main.kt -Xdisable-phases=BitcodeOptimization -Xoverride-konan-properties="$CLANG_FLAGS"

kotlin-native/backend.native/cli.bc/src/org/jetbrains/kotlin/cli/bc/K2NativeCompilerArguments.kt

@@ -343,7 +343,7 @@ class K2NativeCompilerArguments : CommonCompilerArguments() {
 
     @Argument(
             value = "-Xllvm-variant",
-            valueDescription = "{dev|user}",
+            valueDescription = "{dev|user|absolute path to llvm}",
             description = "Choose LLVM distribution which will be used during compilation."
     )
     var llvmVariant: String? = null

kotlin-native/tools/llvm_builder/README.md

@@ -30,11 +30,11 @@ python3 package.py --build-targets install-distribution --distribution-component
 ```
 Set of required `$DISTRIBUTION_COMPONENTS` depends on OS:
 
-| OS | Distribution components |
+| OS      | Distribution components                                                                |
-| --- | --- |
+|---------|----------------------------------------------------------------------------------------|
-|Windows | `clang libclang lld llvm-cov llvm-profdata llvm-ar clang-resource-headers` |
+| Windows | `clang libclang lld llvm-cov llvm-profdata llvm-ar clang-resource-headers`             |
-|macOS | `clang libclang lld llvm-cov llvm-profdata llvm-ar clang-resource-headers` |
+| macOS   | `clang libclang lld llvm-cov llvm-profdata llvm-ar clang-resource-headers`             |
-|Linux | `clang libclang lld llvm-cov llvm-profdata llvm-ar clang-resource-headers compiler_rt` |
+| Linux   | `clang libclang lld llvm-cov llvm-profdata llvm-ar clang-resource-headers compiler_rt` |
 
 ### Tuning
 Run `python3 package.py -h` to check how one can tune script's behavior.