esp-idf/.gitlab/ci/dependencies
2023-12-07 11:15:55 +01:00
..
dependencies.yml ci: run pytest_build_system_win only with labels and schedule pipeline 2023-12-07 11:15:55 +01:00
README.md ci: remove unused rules 2022-10-10 16:58:11 +08:00

How the generate_rules.py works

Functionalities

This script can do only two things:

  1. Auto-generate some labels/rules we need and update them in rules.yml
  2. Generate a dependency tree graph

Schema

This file only used basic YAML grammar and has nothing to do with the GitLab version YAML file.

It has five custom keywords:

  • matrix: An array of sub-arrays, used to replicate rules by formatting strings. You can use the format string everywhere, it will be formatted recursively
  • labels: An array of labels.
  • patterns: An array of patterns. Patterns that not included
  • included_in: An array of other rule names. It indicates the labels and patterns will be included in all specified rules as well
  • deploy: An array of strings, used to replicate rules by adding postfix -<item in deploy array>. It indicates the extra label used in rules, which will explain later.

How to use this file to generate rules.yml

Let's take a complicated example to help understand the process

"test-{0}-{1}":
    matrix:
        - [a, b]
        - [c, d]
    labels:
        - "{0}-{1}"
    patterns:
        - "{0}"
        - pattern-not-exist
    included_in:
        - build-{0}
  1. expand the mapping dicts defined by matrix

    After this step, it will turn into 4 dicts:

    key labels patterns included_in
    test-a-c a-c a build-a
    test-a-d a-d a build-a
    test-b-c b-c b build-b
    test-b-d b-d b build-b

    Advanced Usage: You can overwrite a mapping by declaring it again later, For example:

    If we concatenate this part to the previous example,

    # ... The same as the previous example
    
    test-a-c:
        labels:
            - overwrite
    

    rule test-a-c will be turned into:

    key labels
    test-a-c overwrite

    Mappings with the keyword deploy will also replicate by adding a postfix -<item in deploy array> to the mapping key

  2. create rules by included_in

    After this step, it will turn into 6 mapping dicts:

    key labels patterns
    test-a-c a-c a
    test-a-d a-d a
    test-b-c b-c b
    test-b-d b-d b
    build-a a-c, a-d a
    build-b b-c, b-d b
  3. replace the auto-generated region in rules.yml with labels, and rules. Each mapping will generate a rule and all the required labels. patterns are pre-defined in rules.yml and could not be generated automatically. If a mapping is using a pattern undefined, the pattern will be ignored.

    • If a mapping key has postfix -preview, no if-protected-xxx clause will be added
    • else if a mapping key has postfix -production, an if-protected-no_label clause will be added
    • else, an if-protected clause will be added

Graph

All label nodes are in green, pattern nodes are in cyan, rule nodes are in blue

Requirements

There are a few extra dependencies while generating the dependency tree graph, please refer to pygraphviz documentation to install both graphviz and pygraphviz

CLI usage

python $IDF_PATH/tools/ci/generate_rules.py --graph OUTPUT_PATH