|
| 1 | +# Writing GN Templates |
| 2 | +GN and Ninja are documented here: |
| 3 | +* GN: https://gn.googlesource.com/gn/+/master/docs/ |
| 4 | +* Ninja: https://ninja-build.org/manual.html |
| 5 | + |
| 6 | +[TOC] |
| 7 | + |
| 8 | +## Things to Consider When Writing Templates |
| 9 | +### Inputs and Depfiles |
| 10 | +* List all files read (or executed) by an action as `inputs`. |
| 11 | + * It is [not enough](https://chromium-review.googlesource.com/c/chromium/src/+/1090231) |
| 12 | + to have inputs listed by dependent targets. They must be listed directly by targets that use them. |
| 13 | + * Non-system Python imports are inputs! For scripts that import such modules, |
| 14 | + use [`action_with_pydeps`](https://cs.chromium.org/chromium/src/build/config/python.gni?rcl=320ee4295eb7fabaa112f08d1aacc88efd1444e5&l=75) |
| 15 | + to ensure all dependent Python files are captured as inputs. |
| 16 | +* For action inputs that are not computable during "gn gen", actions can write |
| 17 | + depfiles (.d files) to add additional input files as dependencies for |
| 18 | + subsequent builds. They are relevant only for incremental builds. |
| 19 | + * Depfiles should not list files that GN already lists as `inputs`. |
| 20 | + * Besides being redundant, listing them also makes it harder to remove |
| 21 | + inputs, since removing them from GN does not immediately remove them from |
| 22 | + depfiles. |
| 23 | + * Stale paths in depfiles can cause ninja to complain of circular |
| 24 | + dependencies [in some cases](https://bugs.chromium.org/p/chromium/issues/detail?id=639042). |
| 25 | + |
| 26 | +### Ensuring "gn analyze" Knows About your Inputs |
| 27 | +"gn analyze" is used by bots to run only affected tests and build only affected |
| 28 | +targets. Try it out locally via: |
| 29 | +```bash |
| 30 | +echo "compute_inputs_for_analyze = true" >> out/Debug/args.gn |
| 31 | +gn analyze //out/Debug <(echo '{ |
| 32 | + "files": ["//BUILD.gn"], |
| 33 | + "test_targets": ["//base"], |
| 34 | + "additional_compile_targets":[]}') result.txt; cat result.txt |
| 35 | +``` |
| 36 | +* For analyze to work properly, GN must know about all inputs. |
| 37 | +* Inputs added by depfiles are *not available* to "gn analyze". |
| 38 | + * When paths listed in a target's depfile are listed as `inputs` to a |
| 39 | + dependent target, analyze will be correct. |
| 40 | + * Example: An `AndroidManifest.xml` file is an input to an |
| 41 | + `android_library()` and is included in an `android_apk()`'s depfile. |
| 42 | + `gn analyze` will know that a change to the file will require the APK |
| 43 | + to be rebuilt, because the file is marked as an input to the library, and |
| 44 | + the library is a dep of the APK. |
| 45 | + * When paths listed in a target's depfile are *not* listed as `inputs` to a |
| 46 | + dependent target, a few options exist: |
| 47 | + * Rather than putting the inputs in a depfile, force users of your template |
| 48 | + to list them, and then have your action re-compute them and assert that |
| 49 | + they were correct. |
| 50 | + * `jinja_template()` does this. |
| 51 | + * Rather than putting the inputs in a depfile, compute them beforehand and |
| 52 | + save them to a text file. Have your template Use `read_file()` to read |
| 53 | + them in. |
| 54 | + * `action_with_pydeps()` does this. |
| 55 | + * Continue using a depfile, but use an `exec_script()` to compute them when |
| 56 | + [`compute_inputs_for_analyze`](https://cs.chromium.org/chromium/src/build/config/compute_inputs_for_analyze.gni) |
| 57 | + is set. |
| 58 | + * `grit()` does this. |
| 59 | + |
| 60 | +### Outputs |
| 61 | +Do not list files as `outputs` unless they are important. Outputs are important |
| 62 | +if they are: |
| 63 | + * used as an input by another target, or |
| 64 | + * are leaves in the dependency graph (e.g. binaries, apks, etc). |
| 65 | + |
| 66 | +Example: |
| 67 | +* An action runs a binary that creates an output as well as a log file. Do not |
| 68 | + list the log file as an output. |
| 69 | + |
| 70 | +## Best Practices for Python Actions |
| 71 | +Outputs should be atomic and take advantage of `restat=1`. |
| 72 | +* Make outputs atomic by writing to temporary files and then moving them to |
| 73 | + their final location. |
| 74 | + * Rationale: An interrupted write can leave a file with an updated timestamp |
| 75 | + and corrupt contents. Ninja looks only at timestamps. |
| 76 | +* Do not overwrite an existing output with identical contents. |
| 77 | + * Rationale: `restat=1` is a ninja feature enabled for all actions that |
| 78 | + short-circuits a build when output timestamps do not change. This feature is |
| 79 | + the reason that the total number of build steps sometimes decreases when |
| 80 | + building.. |
| 81 | +* Use [`build_utils.AtomicOutput()`](https://cs.chromium.org/chromium/src/build/android/gyp/util/build_utils.py?rcl=7d6ba28e92bec865a7b7876c35b4621d56fb37d8&l=128) |
| 82 | + to perform both of these techniques. |
| 83 | + |
| 84 | +Actions should be deterministic in order to avoid hard-to-reproduce bugs. |
| 85 | +Given identical inputs, they should produce byte-for-byte identical outputs. |
| 86 | +* Some common mistakes: |
| 87 | + * Depending on filesystem iteration order. |
| 88 | + * Writing timestamps in files (or in zip entries). |
| 89 | + * Writing absolute paths in outputs. |
| 90 | + |
| 91 | +## Style Guide |
| 92 | +Chromium GN files follow |
| 93 | +[GN's Style Guide](https://gn.googlesource.com/gn/+/master/docs/style_guide.md) |
| 94 | +with a few additions. |
| 95 | + |
| 96 | +### Action Granularity |
| 97 | + * Prefer writing new Python scripts that do what you want over |
| 98 | + composing multiple separate actions within a template. |
| 99 | + * Fewer targets makes for a simpler build graph. |
| 100 | + * GN logic and build logic winds up much simpler. |
| 101 | + |
| 102 | +Bad: |
| 103 | +```python |
| 104 | +template("generate_zipped_sources") { |
| 105 | + generate_files("${target_name}__gen") { |
| 106 | + ... |
| 107 | + outputs = [ "$target_gen_dir/$target_name.temp" ] |
| 108 | + } |
| 109 | + zip(target_name) { |
| 110 | + deps = [ ":${target_name}__gen" ] |
| 111 | + inputs = [ "$target_gen_dir/$target_name.temp" ] |
| 112 | + outputs = [ invoker.output_zip ] |
| 113 | + } |
| 114 | +} |
| 115 | +``` |
| 116 | + |
| 117 | +Good: |
| 118 | +```python |
| 119 | +template("generate_zipped_sources") { |
| 120 | + action(target_name) { |
| 121 | + script = "generate_and_zip.py" |
| 122 | + ... |
| 123 | + outputs = [ invoker.output_zip ] |
| 124 | + } |
| 125 | +} |
| 126 | +``` |
| 127 | + |
| 128 | +### Naming for Intermediate Targets |
| 129 | +Targets that are not relevant to users of your template should be named as: |
| 130 | +`${target_name}__$something`. |
| 131 | + |
| 132 | +Example: |
| 133 | +```python |
| 134 | +template("my_template") { |
| 135 | + action("${target_name}__helper") { |
| 136 | + ... |
| 137 | + } |
| 138 | + action(target_name) { |
| 139 | + deps = [ ":${target_name}__helper" ] |
| 140 | + ... |
| 141 | + } |
| 142 | +} |
| 143 | +``` |
| 144 | + |
| 145 | +### Variables |
| 146 | +Prefix variables within templates and targets with an underscore. For example: |
| 147 | + |
| 148 | +```python |
| 149 | +template("example") { |
| 150 | + _outer_sources = invoker.extra_sources |
| 151 | + |
| 152 | + source_set(target_name) { |
| 153 | + _inner_sources = invoker.sources |
| 154 | + sources = _outer_sources + _inner_sources |
| 155 | + } |
| 156 | +} |
| 157 | +``` |
| 158 | + |
| 159 | +This convention conveys that `sources` is relevant to `source_set`, while |
| 160 | +`_outer_sources` and `_inner_sources` are not. |
| 161 | + |
| 162 | +### Passing Arguments to Targets |
| 163 | +Pass arguments to targets by assigning them directly within target definitions. |
| 164 | + |
| 165 | +When a GN template goes to resolve `invoker.FOO`, GN will look in all enclosing |
| 166 | +scopes of the target's definition. It is hard to figure out where `invoker.FOO` |
| 167 | +is coming from when it is not assigned directly within the target definition. |
| 168 | + |
| 169 | +Bad: |
| 170 | +```python |
| 171 | +template("hello") { |
| 172 | + script = "..." |
| 173 | + action(target_name) { |
| 174 | + # This action will see "script" from the enclosing scope. |
| 175 | + } |
| 176 | +} |
| 177 | +``` |
| 178 | + |
| 179 | +Good: |
| 180 | +```python |
| 181 | +template("hello") { |
| 182 | + action(target_name) { |
| 183 | + script = "..." # This is equivalent, but much more clear. |
| 184 | + } |
| 185 | +} |
| 186 | +``` |
| 187 | + |
| 188 | +**Exception:** `testonly` and `visibility` can be set in the outer scope so that |
| 189 | +they are implicitly passed to all targets within a template. |
| 190 | + |
| 191 | +This is okay: |
| 192 | +```python |
| 193 | +template("hello") { |
| 194 | + testonly = true # Applies to all nested targets. |
| 195 | + action(target_name) { |
| 196 | + script = "..." |
| 197 | + } |
| 198 | +} |
| 199 | +``` |
| 200 | + |
| 201 | +### Using forward_variables_from() |
| 202 | +Using `forward_variables_from()` is encouraged, but `testonly` and `visibility` |
| 203 | +should always be listed explicitly in case they are assigned in an enclosing |
| 204 | +scope (applies to the `"*"` variant of `forward_variables_from()`). |
| 205 | +See [this bug](https://bugs.chromium.org/p/chromium/issues/detail?id=862232) |
| 206 | +for more context. |
| 207 | + |
| 208 | +```python |
| 209 | +template("action_wrapper") { |
| 210 | + action(target_name) { |
| 211 | + forward_variables_from(invoker, "*", [ "testonly", "visibility" ]) |
| 212 | + forward_variables_from(invoker, [ "testonly", "visibility" ]) |
| 213 | + ... |
| 214 | + } |
| 215 | +} |
| 216 | +``` |
| 217 | + |
| 218 | +## Useful Ninja Flags |
| 219 | +Useful ninja flags when developing build rules: |
| 220 | +* `ninja -v` - log the full command-line of every target. |
| 221 | +* `ninja -v -n` - log the full command-line of every target without having |
| 222 | + to wait for a build. |
| 223 | +* `ninja -w dupbuild=err` - fail if multiple targets have the same output. |
| 224 | +* `ninja -d keeprsp` - prevent ninja from deleting response files. |
| 225 | +* `ninja -n -d explain` - print why ninja thinks a target is dirty. |
| 226 | +* `ninja -j1` - execute only one command at a time. |
0 commit comments