alex/esp-idf
alex/esp-idf
1
0
Fork 0
mirror of https://github.com/espressif/esp-idf.git synced 2024-10-05 20:47:46 -04:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity

docs(ci): add docs about rules.yml and the auto-generate feature

Browse Source
This commit is contained in:
Fu Hanxi 2021-02-09 12:50:24 +08:00
parent 86287504ba
commit fea3939980
2 changed files with 268 additions and 52 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

220
.gitlab/ci/README.md
View File

@ -1,126 +1,241 @@
# Rules for `rules.yml`
# IDF CI
- [Rules for `rules.yml`](#rules-for-rulesyml)
- [IDF CI](#idf-ci)
- [General Workflow](#general-workflow)
- [What if Expected Jobs ARE NOT Created?](#what-if-expected-jobs-are-not-created)
- [MR labels for additional jobs](#mr-labels-for-additional-jobs)
- [Supported MR Labels](#supported-mr-labels)
- [Usages](#usages)
- [How to trigger a `detached` pipeline without pushing new commits?](#how-to-trigger-a-detached-pipeline-without-pushing-new-commits)
- [How to Develop With `rules.yml`?](#how-to-develop-with-rulesyml)
- [General Concepts](#general-concepts)
- [How to Add a New `Job`?](#how-to-add-a-new-job)
- [How to Add a New `Rules` Template?](#how-to-add-a-new-rules-template)
- [How to Add a New `if` Anchor?](#how-to-add-a-new-if-anchor)
- [Naming Rules](#naming-rules)
- [Common Naming Rules](#common-naming-rules)
- [`if` Anchors Naming Rules](#if-anchors-naming-rules)
- [Common Phrases/Abbreviations](#common-phrasesabbreviations)
- [`rules` Template Naming Rules](#rules-template-naming-rules)
- [Naming Rules](#naming-rules)
- [Common Naming Rules](#common-naming-rules)
- [`if` Anchors Naming Rules](#if-anchors-naming-rules)
- [`rules` Template Naming Rules](#rules-template-naming-rules)
- [Reusable Shell Script `tools/ci/utils.sh`](#reusable-shell-script-toolsciutilssh)
- [Functions](#functions)
- [CI Job Related](#ci-job-related)
- [Shell Script Related](#shell-script-related)
## General Workflow
1. Push to a remote branch
2. Create an MR, choose related labels (not required)
3. A `detached` pipeline will be created.
4. if you push a new commit, a new pipeline will be created automatically.
**Details:**
1. If an MR title starts with `WIP:` or `Draft:`, push commit will NOT trigger a merge-request pipeline
2. If a commit message starts with `test ci:`, pushing a commit will trigger a merge-request pipeline even when the MR title starts with `WIP:` or `Draft:`.
3. If a commit message starts with `WIP:` or `Draft:`, push commit will NOT trigger a pipeline
## What if Expected Jobs ARE NOT Created?
1. check the file patterns
If you found a job that is not running as expected with some file changes, a git commit to improve the `pattern` will be appreciated.
2. please add MR labels to run additional tests
## MR labels for additional jobs
### Supported MR Labels
- `build`
- `build_docs`
- `component_ut[_esp32/esp32s2/...]`
- `custom_test[_esp32/esp32s2/...]`
- `docker`
- `docs`
- `example_test[_esp32/esp32s2/...]`
- `fuzzer_test`
- `host_test`
- `integration_test`
- `iperf_stress_test`
- `macos`
- `macos_test`
- `nvs_coverage`
- `unit_test[_esp32/esp32s2/...]`
- `weekend_test`
- `windows`
There are two general labels (not recommended since these two labels will trigger a lot of jobs)
- `target_test`: includes all target for `example_test`, `custom_test`, `component_ut`, `unit_test`, `integration_test`
- `all_test`: includes all test labels
### Usages
We have two ways to run additional jobs
- Add these labels in the MR `labels`
- Add these labels in the commit message (not the first line). For example:
```
ci: detect file changes to assign jobs
test labels: example_test_esp32, custom_test_esp32
```
The additional test labels line should start with `test label(s):` and the labels should be separated by space or comma.
### How to trigger a `detached` pipeline without pushing new commits?
Go to MR web page -> `Pipelines` tab -> click `Run pipeline` button
## How to Develop With `rules.yml`?
### General Concepts
- `pattern`: Defined in an array. A GitLab job will be created if the changed files in this MR matched one of the patterns. For example:
```yaml
.patterns-python-files: &patterns-python-files
- "**/*.py"
```
- `label`: (deprecated). Defined in an if clause, similar as the previous bot command. A GitLab job will be created if the pipeline variables contains variables in `BOT_LABEL_xxx` format. For example:
```yaml
.if-label-build_docs: &if-label-build_docs
if: '$BOT_LABEL_BUILD_DOCS'
```
- `title`: Defined in an if clause. A GitLab job will be created if this title included in the MR labels or in the commit message title. For example:
```yaml
.if-title-docs: &if-title-docs
if: '$CI_MERGE_REQUEST_LABELS =~ /^(?:\w+,)*docs(?:,\w+)*$/i || $CI_COMMIT_TITLE =~ /\((?:\w+\s+)*docs(?:\s+\w+)*\)$/i'
```
- `rule`: A combination of various patterns, labels, and titles. It will be used by GitLab YAML `extends` keyword to tell GitLab in what conditions will this job be created. For example:
```yaml
.rules:build:docs:
rules:
- <<: *if-protected
- <<: *if-label-build
- <<: *if-title-build
- <<: *if-label-build_docs
- <<: *if-title-build_docs
- <<: *if-label-docs
- <<: *if-title-docs
- <<: *if-dev-push
changes: *patterns-docs
```
An example for GitLab job on how to use extends:
```yaml
check_docs_lang_sync:
extends:
- .pre_check_job_template
- .rules:build:docs
script:
- cd docs
- ./check_lang_folder_sync.sh
```
### How to Add a New `Job`?
check if there's a suitable `.rules:<rules-you-need>` template
1. if there is, put this in the job `extends`. All done, now you can close this window. (`extends` could be array or string)
2. if there isn't
1. check [How to Add a New `Rules` Template?](#how-to-add-a-new-rules-template), create a suitable one
2. follow step 1
1. check [How to Add a New `Rules` Template?](#how-to-add-a-new-rules-template), create a suitable one
2. follow step 1
### How to Add a New `Rules` Template?
check if this rule is related to `labels`, `patterns`
1. if it is, please refer to [dependencies/README.md](./dependencies/README.md) and add new rules by auto-generating
2. if it isn't, please continue reading
check if there's a suitable `.if-<if-anchor-you-need>` anchor
1. if there is, create a rule following [`rules` Template Naming Rules](#rules-template-naming-rules).For detail information, please refer to [GitLab Documentation `rules-if`](https://docs.gitlab.com/ee/ci/yaml/README.html#rulesif). Here's an example.
1. if there is, create a rule following [`rules` Template Naming Rules](#rules-template-naming-rules).For detail information, please refer to [GitLab Documentation `rules-if`](https://docs.gitlab.com/ee/ci/yaml/README.html#rulesif). Here's an example.
```yaml
.rules:dev:
rules:
- <<: *if-trigger
- <<: *if-dev-push
```
```yaml
.rules:dev:
rules:
- <<: *if-trigger
- <<: *if-dev-push
```
2. if there isn't
1. check [How to Add a New `if` Anchor?](#how-to-add-a-new-if-anchor), create a suitable one
2. follow step 1
1. check [How to Add a New `if` Anchor?](#how-to-add-a-new-if-anchor), create a suitable one
2. follow step 1
### How to Add a New `if` Anchor?
Create an `if` anchor following [`if` Anchors Naming Rules](#if-anchors-naming-rules). For detail information about how to write the condition clause, please refer to [GitLab Documentation `only/except (advanced)](https://docs.gitlab.com/ee/ci/yaml/README.html#onlyexcept-advanced). Here's an example.
Create an `if` anchor following [`if` Anchors Naming Rules](#if-anchors-naming-rules). For detailed information about how to write the condition clause, please refer to [GitLab Documentation `only/except (advanced)](https://docs.gitlab.com/ee/ci/yaml/README.html#onlyexcept-advanced). Here's an example.
```yaml
.if-schedule: &if-schedule:
if: '$CI_PIPELINE_SOURCE == "schedule"'
```
## Naming Rules
### Naming Rules
### Common Naming Rules
#### Common Naming Rules
if a phrase has multi words, use `_` to concat them.
if a phrase has multi words, use `_` to concatenate them.
> e.g. `regular_test`
if a name have multi phrase, use `-` to concat them.
if a name has multi phrases, use `-` to concatenate them.
> e.g. `regular_test-example_test`
### `if` Anchors Naming Rules
#### `if` Anchors Naming Rules
- if it's a label: `.if-label-<label_name>`
- if it's a ref: `.if-ref-<ref_name>`
- if it's a branch: `.if-branch-<branch_name>`
- if it's a tag: `.if-tag-<tag_name>`
- if it's a operating system: `.if-os-mac`
- if it's multi-type combination: `.if-ref-<release_name>-branch-<branch_name>`
#### Common Phrases/Abbreviations
**Common Phrases/Abbreviations**
- `no_label`
- `no_label`
`$BOT_TRIGGER_WITH_LABEL == null`
`$BOT_TRIGGER_WITH_LABEL == null`
- `protected`
- `protected`
`($CI_COMMIT_REF_NAME == "master" || $CI_COMMIT_BRANCH =~ /^release\/v/ || $CI_COMMIT_TAG =~ /^v\d+\.\d+(\.\d+)?($|-)/)`
`($CI_COMMIT_REF_NAME == "master" || $CI_COMMIT_BRANCH =~ /^release\/v/ || $CI_COMMIT_TAG =~ /^v\d+\.\d+(\.\d+)?($|-)/)`
- `target_test`
- `target_test`
`example_test` or `custom_test` or `unit_test-all_targets`
a combination of `example_test`, `custom_test`, `unit_test`, `component_ut`, `integration_test` and all targets
### `rules` Template Naming Rules
#### `rules` Template Naming Rules
- if it's os related: `.rules:os:<os_name>`
- if it's tag related: `.rules:tag:<tag_1>-<tag_2>`
- if it's label related: `.rules:labels:<label_1>-<label_2>`
By default, all `.rules:labels` should include both `if-label-regular_test` and `if-protected-no-label` implicitly, unless they have special postfixes:
- slim: `regular_test` not included
- only: only have the phrases listed
- if it's target test related: `.rules:tests:<test_type_1>-<test_type_2>`
By default, all `.rules:tests` should include `if-protected-no_label` implicitly, unless they have special postfixes (same as above)
- if it needs to build at first, then do some target test: `.rules:build_tests:<test_type_1>-<test_type_2>`
By default, all `.rules:build_tests` should include `if-protected-no-label`, `if-label-build`, `if-label-regular-test` implictly, unless they have special postfixes (same as above)
- if a job supports all targets, use `-all_targets` as postfix
- if it's test related: `.rules:test:<test_type>`
- if it's build related: `.rules:build:<build_type>`
- if it's pattern related: `.rules:patterns:<patterns>`
## Reusable Shell Script `tools/ci/utils.sh`
It is used to put all the reusable shell script as small functions. If you want to set `before_script: []` for you job, now you can set `extends: .before_script_slim` instead. it will only run `source tools/ci/utils.sh`
It is used to put all the reusable shell scripts as small functions. If you want to set `before_script: []` for you job, now you can set `extends: .before_script_slim` instead. it will only run `source tools/ci/utils.sh`
If you're developing CI shell scripts, you can use these functions without source. They're already included in all `before_script`
If you're developing CI shell scripts, you can use these functions without `source` them. They're already included in all `before_script`
To run these commands in shell script locally, place `source tools/ci/utils.sh` at the very beginning.
### Functions
#### CI Job Related
- `apply_bot_filter`
- `add_gitlab_ssh_keys`
- `add_github_ssh_keys`
- `add_doc_server_ssh_keys`
@ -128,6 +243,7 @@ To run these commands in shell script locally, place `source tools/ci/utils.sh`
- `get_all_submodules`
#### Shell Script Related
- `error`: log in red color
- `warning`: log in orange color
- `info`: log in green color

100
.gitlab/ci/dependencies/README.md Normal file
View File

@ -0,0 +1,100 @@
# How the `generate_rules.py` works
## Functionalities
This script can do only two things:
1. Auto-generate some labels/titles/rules we need and update them in `rules.yml`
2. Generate a dependency tree graph
## Schema
This file only used basic YAML grammar and has nothing to do with the GitLab version YAML file.
It has five custom keywords:
- `matrix`: An array of sub-arrays, used to replicate rules by formatting strings. You can use the format string everywhere, it will be formatted recursively
- `labels`: An array of `labels`. Also indicates `titles` with the same names
- `patterns`: An array of `patterns`. Patterns that not included
- `included_in`: An array of other `rule` names. It indicates the `labels` and `patterns` will be included in all specified `rules` as well
- `deploy`: An array of strings, used to replicate rules by adding postfix `-<item in deploy array>`. It indicates the extra `label` used in `rules`, which will explain later.
## How to use this file to generate `rules.yml`
Let's take a complicated example to help understand the process
```yaml
"test-{0}-{1}":
matrix:
- [a, b]
- [c, d]
labels:
- "{0}-{1}"
patterns:
- "{0}"
- pattern-not-exist
included_in:
- build-{0}
```
1. expand the mapping dicts defined by `matrix`
After this step, it will turn into 4 dicts:
| key | labels | patterns | included_in |
| -------- | ------ | -------- | ----------- |
| test-a-c | a-c | a | build-a |
| test-a-d | a-d | a | build-a |
| test-b-c | b-c | b | build-b |
| test-b-d | b-d | b | build-b |
**Advanced Usage: You can overwrite a mapping by declaring it again later**, For example:
If we concatenate this part to the previous example,
```yaml
# ... The same as the previous example
test-a-c:
labels:
- overwrite
```
`rule` `test-a-c` will be turned into:
| key | labels |
| -------- | --------- |
| test-a-c | overwrite |
**Mappings with the keyword `deploy` will also replicate by adding a postfix `-<item in deploy array>` to the mapping key**
2. create rules by `included_in`
After this step, it will turn into 6 mapping dicts:
| key | labels | patterns |
| -------- | -------- | -------- |
| test-a-c | a-c | a |
| test-a-d | a-d | a |
| test-b-c | b-c | b |
| test-b-d | b-d | b |
| build-a | a-c, a-d | a |
| build-b | b-c, b-d | b |
3. replace the auto-generated region in `rules.yml` with `labels`, `titles`, and `rules`. Each mapping will generate a `rule` and all the required labels/titles. `patterns` are pre-defined in `rules.yml` and could not be generated automatically. If a mapping is using a `pattern` undefined, the `pattern` will be ignored.
- If a mapping key has postfix '-preview', no `if-protected-xxx` clause will be added
- else if a mapping key has postfix '-production', `if-protected-no_label` clause will be added
- else: a mapping key `if-protected` clause will be added
## Graph
All `label` nodes are in green, `pattern` nodes are in cyan, `rule` nodes are in blue
### Requirements
There are a few extra dependencies while generating the dependency tree graph, please refer to [pygraphviz](https://github.com/pygraphviz/pygraphviz/blob/master/INSTALL.txt) documentation to install both `graphviz` and `pygraphviz`
### CLI usage
`python generate_rules.py --graph OUTPUT_PATH`