From 21399b4eddaa6227313a95559541aa53d92f196c Mon Sep 17 00:00:00 2001 From: Sergey Bogolepov Date: Mon, 18 Apr 2022 18:15:41 +0300 Subject: [PATCH] [K/N] Update documentation on building and using LLVM --- kotlin-native/BUILDING_LLVM.md | 55 ++++--------------- kotlin-native/HACKING.md | 33 ++++------- .../cli/bc/K2NativeCompilerArguments.kt | 2 +- kotlin-native/tools/llvm_builder/README.md | 10 ++-- 4 files changed, 27 insertions(+), 73 deletions(-) diff --git a/kotlin-native/BUILDING_LLVM.md b/kotlin-native/BUILDING_LLVM.md index 65dd212571e..05d1508f838 100644 --- a/kotlin-native/BUILDING_LLVM.md +++ b/kotlin-native/BUILDING_LLVM.md @@ -1,56 +1,21 @@ -# Building Apple LLVM for Kotlin/Native +# Building LLVM for Kotlin/Native -This document describes how to compile LLVM distribution and use it to build Kotlin/Native on macOS. +This document describes how to compile LLVM distribution and use it to build Kotlin/Native. Usually, you don't need to compile LLVM by yourself: it is downloaded automatically when you run or build Kotlin/Native compiler. But if you don't want to download prebuilt LLVM or want to experiment with your own distribution, -you came to the right place. +you come to the right place. -## Part 1. Building the right LLVM version for macOS. +## Part 1. Building the right LLVM version. -For macOS host we use LLVM from [Apple downstream](https://github.com/apple/llvm-project). -Branch is [**apple/stable/20190104**](https://github.com/apple/llvm-project/tree/apple/stable/20190104) -because it is similar (or even the same) to what Apple ships with Xcode 11.*. -After cloning the repo and changing the branch we perform the following steps to build LLVM toolchain: - -```bash -mkdir build - -cd build - -cmake -DLLVM_ENABLE_PROJECTS="clang;lld;libcxx;libcxxabi" \ - -DCMAKE_BUILD_TYPE=Release \ - -DLLVM_ENABLE_ASSERTIONS=Off \ - -G Ninja \ - -DCMAKE_INSTALL_PREFIX=clang-llvm-apple-8.0.0-darwin-macos \ - ../llvm - -ninja install -``` - -After these steps `clang-llvm-apple-8.0.0-darwin-macos` directory will contain LLVM distribution that is suitable for building Kotlin/Native. +Use [package.py](tools/llvm_builder/README.md) script to build LLVM distribution the same way the Kotlin team does. ## Part 2. Building Kotlin/Native against given LLVM distribution. -By default, Kotlin/Native will try to download LLVM distribution from CDN if it is not present in `$HOME/.konan/dependencies` folder. -There are two ways to bypass this behaviour. - -#### Option A. Substitute prebuilt distribution. -This option doesn't require you to edit compiler sources, but a bit harder. - -The compiler checks dependency presence by reading contents of `$HOME/.konan/dependencies/.extracted` file. -So to avoid LLVM downloading, we should manually add a record to the `.extracted` file: -1. Create `$HOME/.konan/dependencies/.extracted` file if it is not created. -2. Add `clang-llvm-apple-8.0.0-darwin-macos` line. - -and put `clang-llvm-apple-8.0.0-darwin-macos` directory from the Part 1 to `$HOME/.konan/dependencies/`. - -#### Option B. Provide an absolute path to the distribution. -This option requires user to edit [konan.properties file](konan/konan.properties). -Set `llvmHome.` to an absolute path to your LLVM distribution and -set `llvmVersion.` to its version. -For example, provide a path to `clang-llvm-apple-8.0.0-darwin-macos` from the Part 1 and set version to 8.0.0. +To do so, we need to edit [konan.properties file](konan/konan.properties): +1. `llvmHome.` should point to the freshly built LLVM distribution. +2. `llvmVersion.` should specify its version (for example, `11.1.0`). Now we are ready to build Kotlin/Native itself. The process is described in [README.md](README.md). @@ -59,5 +24,5 @@ Now we are ready to build Kotlin/Native itself. The process is described in [REA — Can I override `.konan` location? — Yes, by setting `$KONAN_DATA_DIR` environment variable. See [HACKING.md](HACKING.md#compiler-environment-variables). -- Can I use another LLVM distribution without rebuilding Kotlin/Native? -- Yes, see [HACKING.md](HACKING.md#using-different-llvm-distributions-as-part-of-kotlinnative-compilation-pipeline). \ No newline at end of file +— Can I use another LLVM distribution without rebuilding Kotlin/Native? +— Yes, see [HACKING.md](HACKING.md#using-different-llvm-distributions-as-part-of-kotlinnative-compilation-pipeline). \ No newline at end of file diff --git a/kotlin-native/HACKING.md b/kotlin-native/HACKING.md index 5cc35146e7f..67ae982d89b 100644 --- a/kotlin-native/HACKING.md +++ b/kotlin-native/HACKING.md @@ -367,22 +367,15 @@ instead of provided one. ### Using different LLVM distributions as part of Kotlin/Native compilation pipeline. -`llvmHome.` variable in `/konan/konan.properties` controls -which LLVM distribution Kotlin/Native will use in its compilation pipeline. -You can replace its value with either `$llvm..{dev, user}` to use one of predefined distributions -or pass an absolute to your own distribution. -Don't forget to set `llvmVersion.` to the version of your LLVM distribution. - -#### Example. Using LLVM from an absolute path. -Assuming LLVM distribution is installed at `/usr` path, one can specify a path to it -with the `-Xoverride-konan-properties` option: -``` -konanc main.kt -Xoverride-konan-properties=llvmHome.linux_x64=/usr -``` +`-Xllvm-variant` compiler option allows to choose which LLVM distribution should be used during compilation. +The following values are supported: +* `user` — The compiler downloads (if necessary) and uses small LLVM distribution that contains only necessary tools. This is what compiler does by default. +* `dev` — The compiler downloads (if necessary) and uses large LLVM distribution that contains additional development tools like `llvm-nm`, `opt`, etc. +* `` — Use local distribution of LLVM. ### Playing with compilation pipeline. -Following compiler phases control different parts of LLVM pipeline: +The following compiler phases control different parts of LLVM pipeline: 1. `LinkBitcodeDependencies`. Linkage of produced bitcode with runtime and some other dependencies. 2. `BitcodeOptimization`. Running LLVM optimization pipeline. 3. `ObjectFiles`. Compilation of bitcode with Clang. @@ -400,20 +393,16 @@ Please note: 3. Use `clang -cc1 -help` to see a list of available options. Another useful compiler option is `-Xtemporary-files-dir=` which allows -to specify a directory for intermediate compiler artifacts like bitcode and object files. + specifying a directory for intermediate compiler artifacts like bitcode and object files. +For example, it allows to store LLVM IR after a particular compiler phase. -#### Example 1. Bitcode right after IR to Bitcode translation. ```shell script -konanc main.kt -produce bitcode -o bitcode.bc +konanc main.kt -Xsave-llvm-ir-after=BitcodeOptimization -Xtemporary-files-dir= ``` -#### Example 2. Bitcode after LLVM optimizations. -```shell script -konanc main.kt -Xtemporary-files-dir= -o -``` -`/.kt.bc` will contain bitcode after LLVM optimization pipeline. +`/out.BitcodeOptimization.ll` will contain LLVM IR after LLVM optimization pipeline. -#### Example 3. Replace predefined LLVM pipeline with Clang options. +#### Example: replace predefined LLVM pipeline with Clang options. ```shell script CLANG_FLAGS="clangFlags.macos_x64=-cc1 -emit-obj;clangNooptFlags.macos_x64=-O2" konanc main.kt -Xdisable-phases=BitcodeOptimization -Xoverride-konan-properties="$CLANG_FLAGS" diff --git a/kotlin-native/backend.native/cli.bc/src/org/jetbrains/kotlin/cli/bc/K2NativeCompilerArguments.kt b/kotlin-native/backend.native/cli.bc/src/org/jetbrains/kotlin/cli/bc/K2NativeCompilerArguments.kt index d8b9faefd17..e9516fafbf8 100644 --- a/kotlin-native/backend.native/cli.bc/src/org/jetbrains/kotlin/cli/bc/K2NativeCompilerArguments.kt +++ b/kotlin-native/backend.native/cli.bc/src/org/jetbrains/kotlin/cli/bc/K2NativeCompilerArguments.kt @@ -343,7 +343,7 @@ class K2NativeCompilerArguments : CommonCompilerArguments() { @Argument( value = "-Xllvm-variant", - valueDescription = "{dev|user}", + valueDescription = "{dev|user|absolute path to llvm}", description = "Choose LLVM distribution which will be used during compilation." ) var llvmVariant: String? = null diff --git a/kotlin-native/tools/llvm_builder/README.md b/kotlin-native/tools/llvm_builder/README.md index 60f35a49273..29fc1d31b61 100644 --- a/kotlin-native/tools/llvm_builder/README.md +++ b/kotlin-native/tools/llvm_builder/README.md @@ -30,11 +30,11 @@ python3 package.py --build-targets install-distribution --distribution-component ``` Set of required `$DISTRIBUTION_COMPONENTS` depends on OS: -| OS | Distribution components | -| --- | --- | -|Windows | `clang libclang lld llvm-cov llvm-profdata llvm-ar clang-resource-headers` | -|macOS | `clang libclang lld llvm-cov llvm-profdata llvm-ar clang-resource-headers` | -|Linux | `clang libclang lld llvm-cov llvm-profdata llvm-ar clang-resource-headers compiler_rt` | +| OS | Distribution components | +|---------|----------------------------------------------------------------------------------------| +| Windows | `clang libclang lld llvm-cov llvm-profdata llvm-ar clang-resource-headers` | +| macOS | `clang libclang lld llvm-cov llvm-profdata llvm-ar clang-resource-headers` | +| Linux | `clang libclang lld llvm-cov llvm-profdata llvm-ar clang-resource-headers compiler_rt` | ### Tuning Run `python3 package.py -h` to check how one can tune script's behavior.