Commit Graph

893 Commits

Author SHA1 Message Date
Fu Hanxi
fba96d58c2
ci: dynamic pipeline
build:
- upgrade idf-build-apps to 2.x
- unify get_pytest_apps and get_cmake_apps to get_all_apps
	- returns (test_apps, non_test_apps) tuple
- add tests for the new get_all_apps

assign:
- generate build report
- generate target test pipeline based on the build report

target test:
- download artifacts from minio server
- users can use `pytest --pipeline-id xxxxx` to download and flash
the binaries from the artifacts

.post:
- generate target test reports
2024-01-10 15:37:34 +01:00
Fu Hanxi
114af706bb
ci(wifi): migrate from .gitlab/ci/rules.yml to .build-test-rules.yml 2024-01-10 15:31:21 +01:00
Fu Hanxi
bb375ee79a
ci(ethernet): migrate from .gitlab/ci/rules.yml to .build-test-rules.yml 2024-01-10 15:31:11 +01:00
Fu Hanxi
83d4e8c6ab
ci(flash_encryption_wifi_high_traffic): migrate from .gitlab/ci/rules.yml to .build-test-rules.yml 2024-01-10 15:30:53 +01:00
Fu Hanxi
6f6092ed8d
ci(sdio): migrate from .gitlab/ci/rules.yml to .build-test-rules.yml 2024-01-10 15:30:52 +01:00
Fu Hanxi
d44f236f7b
ci(usb): migrate from .gitlab/ci/rules.yml to .build-test-rules.yml 2024-01-10 15:29:44 +01:00
Fu Hanxi
ed1c892a8a
ci(adc): migrate from .gitlab/ci/rules.yml to .build-test-rules.yml 2024-01-10 15:29:44 +01:00
Fu Hanxi
9a9b1cd11f
ci(ecdsa): migrate from .gitlab/ci/rules.yml to .build-test-rules.yml 2024-01-10 15:29:43 +01:00
Fu Hanxi
7138b977f8
ci(nvs_encr_hmac): migrate from .gitlab/ci/rules.yml to .build-test-rules.yml 2024-01-10 15:29:43 +01:00
Fu Hanxi
3645c041e2
ci(flash_multi): migrate from .gitlab/ci/rules.yml to .build-test-rules.yml 2024-01-10 15:29:43 +01:00
Fu Hanxi
b6a93bf81e
ci(i154): migrate from .gitlab/ci/rules.yml to .build-test-rules.yml 2024-01-10 15:29:42 +01:00
Fu Hanxi
b709c880dd
ci: add linter for gitlab yaml files
- remove duplicated artifacts default values
- migrate check_artifacts_expire_time.py
- migrate check_rules_yml.py
2024-01-10 15:29:42 +01:00
Fu Hanxi
7962746c65
ci: remove ip101 jobs and twai_std jobs 2024-01-10 10:10:53 +01:00
Kevin (Lao Kaiyao)
1e6a76419b Merge branch 'feature/support_hello_world_on_esp32c5' into 'master'
feat(esp32c5): support hello world on esp32c5 beta3 (stage 8/8)

See merge request espressif/esp-idf!27870
2024-01-09 18:41:43 +08:00
laokaiyao
d0a8f3e5c4 feat(esp32c5): support esptool on esp32c5 beta3 2024-01-09 13:11:11 +08:00
Fu Hanxi
c633ba167d
ci: fix code_quality_check 2024-01-08 10:13:08 +01:00
wanlei
8536ffb490 fix(twai): enable twai interactive test for all targets on new std runner 2024-01-05 15:22:09 +08:00
Armando
1ab742b3c3 ci(p4): enable esp32p4 target test 2024-01-04 09:34:55 +08:00
Xu Si Yu
13fa6c7af2 feat(ieee802154): Add ieee802154 CLI example 2024-01-03 19:05:16 +08:00
Frantisek Hrbata
0f898539a0 ci: add sbom validation using pre-commit's post-commit stage
With 09ba070b7e ("fix: allow to use older pre-commit hook version")
and b93cc581a5 ("fix: set default pre-commit hook stages to pre-commit..")
we set the default pre-commit hook stage to commit/pre-commit, which avoids
running the pre-commit plugins/hooks multiple times for different stages.
Unfortunately this also affects "pre-commit run" used in CI and
now it runs only plugins which are by default using the
commit/pre-commit stage. Meaning the sbom validation, which uses
post-commit, is not run at all. This adds an explicit pre-commit command
for validate-sbom-manifest plugin, which needs to be started in
post-commit stage.

Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
2024-01-02 10:47:11 +01:00
Ivan Grokhotkov
763ba14885
feat(docs): use a faster fork of breathe-doc/breathe
See https://github.com/breathe-doc/breathe/discussions/962
2023-12-21 16:33:20 +01:00
Chen Yudong
77597f7ac8 ci: do not upload caches for dev branches by default 2023-12-19 20:36:55 +08:00
Chen Yudong
ff015d1872 ci: differnet cache key for different idf ver
update caches on release tags and nightly schedules
2023-12-19 16:56:53 +08:00
Marius Vikhammer
852fbce397 ci(qemu): disable WDT when running qemu tests
WDTs are flakey in QEMU, disable while testing for now to
stop spurious test failures.
2023-12-13 18:47:26 +08:00
Anton Maklakov
9f0da8b6ed Merge branch 'ci/upload_pipeline_env_fil_into_minio' into 'master'
ci: Upload a report about modified files and components to s3 bucket

See merge request espressif/esp-idf!27758
2023-12-08 12:35:00 +08:00
Aleksei Apaseev
e97826e877 ci: Upload report about modified files and components to s3 bucket 2023-12-08 10:49:17 +08:00
Aleksei Apaseev
0219e8e85d fix(idf_tools): Opt for the recommended tool in tools.json rather than the supported one 2023-12-08 10:14:19 +08:00
Fu Hanxi
d3858b493d
ci: run pytest_build_system_win only with labels and schedule pipeline 2023-12-07 11:15:55 +01:00
Frantisek Hrbata
5df4decf07 feat: use esp-idf-sbom pre-commit plugin
Currently sbom manifest is checked only in .gitmodules and
this check is done in pre-commit and also in CI. Meaning it's running
three times(pre-commit before push if user has it enabled, in CI
as there is the pre-commit run again and again with test in CI). Since
esp-idf-sbom contains a full manifest validation support and pre-commit
plugin for it, let's use it. This removes all the current sbom testing
and replaces it with a signle pre-commit plugin which validates all
manifests files(sbom.yml, idf_component.yml, .gitmodules and also
referenced manifests) in repository. Note that this checks all
manifests, not only ones which were modified. The check is reasonably
fast though, so it should not cause any problem. The reason for
validating all manifest files is that we want to make sure that the sbom
information in .gitmodules is updated too and that the hash
recorded in .gitmodules is up-to-date. Meaning submodule update
would not trigger this plugin, because no manifest was changed.

Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
2023-12-06 10:08:54 +01:00
Ivan Grokhotkov
178b27ed80
ci(qemu): use qemu from tools.json, don't require a special image 2023-12-03 07:45:19 +01:00
Ivan Grokhotkov
71153c7dbb
feat(tools): add 'idf.py qemu' extension
This extension allows running programs in QEMU similar to running
them on a real chip:

- 'idf.py qemu' — builds and runs the program in QEMU. User gets
  a QEMU instance launched, and can work with it as a normal QEMU
  instance.
- 'idf.py qemu monitor' — same, but QEMU is launched in the
  background, and idf_monitor runs in the foreground, showing QEMU
  output. Compared to only running 'idf.py qemu' this enables, for
  example, automatic backtrace decoding.
- 'idf.py qemu gdb' — launches QEMU in the background and opens an
  interactive GDB prompt, connecting it to QEMU.
- 'idf.py qemu --gdb monitor' and 'idf.py gdb' in another shell:
  launches QEMU in the background, keeps it suspended until GDB is
  connected, and opens idf_monitor. GDB can be used in another shell
  to debug the application.
2023-12-03 07:45:19 +01:00
Roland Dobai
9c4b657ac3 Merge branch 'feature/add_windows_pytest' into 'master'
Tools: Add pytest build system on Windows runner

Closes IDF-7691, IDF-8214, and IDF-1193

See merge request espressif/esp-idf!26603
2023-12-01 16:40:13 +08:00
C.S.M
2d4a9240dc Merge branch 'feat/add_example_usj' into 'master'
change(usb_serial_jtag): Add example for usb serial jtag echo

Closes IDFGH-11503, IDFGH-11493, and IDFGH-11478

See merge request espressif/esp-idf!27366
2023-12-01 14:39:14 +08:00
Liu Linyan
9879615053 Merge branch 'update/ble_mesh_put_lib_in_submodule' into 'master'
fix(ble_mesh): Put Mesh 1.1 lib files in submodule

Closes IDFCI-1860

See merge request espressif/esp-idf!26987
2023-12-01 09:56:19 +08:00
Roland Dobai
bc1d74bb88 fix(ci): Remove unittest-xml-reporting dependency from host tests 2023-11-30 14:03:41 +01:00
Liu Linyan
94d3f5037b fix(ble_mesh): Use submodule for mesh 1.1 lib files 2023-11-30 16:56:58 +08:00
Cao Sen Miao
2a0debde3c change(usb_serial_jtag): Add a usb_serial_jtag echo example for how to use usb_serial_jtag APIs,
Closes https://github.com/espressif/esp-idf/issues/12620,
Closes https://github.com/espressif/esp-idf/issues/12605
2023-11-30 14:38:06 +08:00
Armando (Dou Yiwen)
d36fe73960 Merge branch 'test/improve_sdmmc_tests' into 'master'
sd: improve tests and added on CI

See merge request espressif/esp-idf!27384
2023-11-30 10:21:11 +08:00
Marek Fiala
0a3b57e48a feat(tools): Add pytest build system on Windows runner 2023-11-29 23:10:37 +01:00
Tomas Sebestik
0b3ec55183 Merge branch 'remove/migrate-danger-gitlab' into 'master'
Remove/migrate danger gitlab

Closes RDT-523

See merge request espressif/esp-idf!27379
2023-11-29 15:14:23 +08:00
Armando
9b0d75f2df refactor(sdmmc): added component pytest cases and enabled them on CI 2023-11-29 12:13:03 +08:00
Fu Hanxi
09eec2ba47 Merge branch 'ci/fix_upload_failed_job_log_script' into 'master'
ci: fix upload failed job log script

See merge request espressif/esp-idf!27357
2023-11-27 18:32:45 +08:00
Tomas Sebestik
742ec21371
ci(danger-gitlab): migrate code; Danger Gitlab from Shared-CI-Danger
- set only brew runners to run danger, tag "dangerjs"
2023-11-27 09:36:34 +01:00
Roland Dobai
cad04baba1 feat(ci): Merge tool test jobs and produce jUnit reports 2023-11-27 09:11:35 +01:00
Chen Yu Dong
ab03c2ea13 Merge branch 'ci/fix_fetch_git_diff_on_schedules' into 'master'
ci: fix fetch git diff from pipeline schedules

See merge request espressif/esp-idf!27263
2023-11-24 21:15:41 +08:00
Fu Hanxi
4ef1a8cc3a
ci: fix upload failed job log script 2023-11-23 08:03:42 +01:00
Armando (Dou Yiwen)
b8121227b6 Merge branch 'refactor/esp_driver_sdio' into 'master'
refactor(sdio): place sdio slave driver into a new component

Closes IDF-8373

See merge request espressif/esp-idf!27125
2023-11-22 21:13:00 +08:00
Armando
b860930545 ci(sdio): updated sdio rules.yml 2023-11-22 14:23:14 +08:00
Chen Yu Dong
d7345a89df Merge branch 'ci/fix_fetch_from_mirror_url_if_exists' into 'master'
ci: fix fetch_from_mirror_url_if_exists

See merge request espressif/esp-idf!27184
2023-11-21 12:03:52 +08:00
Chen Yudong
294222eabf ci: fix fetch git diff from pipeline schedules 2023-11-20 22:08:57 +08:00
Chen Yudong
d738185ff8 ci: fix fetch_from_mirror_url_if_exists 2023-11-20 16:00:01 +08:00
Marius Vikhammer
42ba49390f ci(docs): revert git strategy change for deploy docs 2023-11-16 10:23:25 +08:00
Fu Hanxi
ac0a1b4e8e Merge branch 'ci/git_strategy_optimization' into 'master'
ci: optimize git strategy

Closes IDFCI-1854

See merge request espressif/esp-idf!26797
2023-11-15 14:42:41 +08:00
Fu Hanxi
a6ccc2e18e
ci: optimize git strategy 2023-11-14 15:31:30 +01:00
Armando
faa6e51086 feat(sdio): sdio cross chip test app 2023-11-13 11:21:47 +08:00
Fu Hanxi
6ba8c75f1d Merge branch 'ci/build_log_artifacts' into 'master'
ci: added build log artifacts for linux

Closes IDFCI-1848

See merge request espressif/esp-idf!26948
2023-11-08 21:54:59 +08:00
Jiang Jiang Jian
d63f92eb3e Merge branch 'ci/add_label_trigger_for_integration_test' into 'master'
ci: add label rules for integration test

See merge request espressif/esp-idf!26922
2023-11-07 15:32:14 +08:00
Aleksei Apaseev
999f5acf6b Merge branch 'ci/add_artifacts_upload_to_after_script' into 'master'
ci: Move upload artifacts execution to the after script

See merge request espressif/esp-idf!26919
2023-11-07 13:31:19 +08:00
Aleksei Apaseev
b64da91823 ci: Move upload artifacts execution of build and target_test stages to the after script section 2023-11-06 18:24:10 +08:00
Marius Vikhammer
501dbcfacf ci(docs): stop build docs jobs from downloading artifacts 2023-11-06 16:59:31 +08:00
igor udot (horw)
2dcb6d1fbb ci: added build log artifacts for linux 2023-11-06 14:34:33 +08:00
Chen Yudong
9b8d50a8f5 ci: add label rules for integration test
fix typo
2023-11-03 19:18:52 +08:00
morris
b2497d1ce6 Merge branch 'ci/update_constraint_file' into 'master'
ci: update constraint file to 5.3

See merge request espressif/esp-idf!26751
2023-11-02 23:07:12 +08:00
Fu Hanxi
a9079626bb Merge branch 'ci/upload_artifacts_of_target_stage' into 'master'
ci: Upload artifacts of target stage jobs to s3 bucket

Closes RDT-571

See merge request espressif/esp-idf!26722
2023-10-31 18:34:20 +08:00
Fu Hanxi
5e1826678d Merge branch 'ci/improve_check_pylint' into 'master'
ci: improve check pylint get files command

See merge request espressif/esp-idf!26578
2023-10-30 19:29:44 +08:00
Fu Hanxi
14d8fc82fe
ci: update constraint file to 5.3 2023-10-30 12:00:29 +01:00
Aleksei Apaseev
3a998bd0ce ci: Upload artifacts of target stage jobs to s3 bucket 2023-10-29 12:44:11 +08:00
Ivan Grokhotkov
8171b22c40
change(version): Update version to 5.3.0 2023-10-27 03:06:55 +02:00
Chen Yu Dong
c6b6a7829d Merge branch 'ci/remove_unused_integration_test_files' into 'master'
ci: remove unused integration test files

See merge request espressif/esp-idf!26655
2023-10-27 05:12:47 +08:00
Marius Vikhammer
eceefa878f Merge branch 'ci/build_test_add_known_failure' into 'master'
ci(build_test): add known-failure check

See merge request espressif/esp-idf!26618
2023-10-26 15:30:19 +08:00
Marius Vikhammer
4ea0538a88 Merge branch 'ci/system_example_test_reduction' into 'master'
ci: reduce the number of system examples built/tested in regular pipelines

See merge request espressif/esp-idf!26519
2023-10-26 10:36:18 +08:00
Marius Vikhammer
7559a1eccf Merge branch 'ci/delay_build_docs_master' into 'master'
ci: run build_docs jobs after target tests on protected branches

See merge request espressif/esp-idf!26508
2023-10-26 10:28:56 +08:00
Chen Yudong
f1c6bc9e68 ci: remove unused integration test rules 2023-10-25 14:16:45 +08:00
Fu Hanxi
b4268c874a Merge branch 'ci/tune_build_jobs_parallel_count_master' into 'master'
ci: tune build test jobs parallel count

See merge request espressif/esp-idf!26627
2023-10-24 21:19:51 +08:00
Marius Vikhammer
d98e77a4a7 ci: run build_docs jobs after target tests on protected branches
On master running costly build docs jobs for every pipeline uses a lot of resources.
This refactor moves the build doc stage after target tests for protected branches.

For regular MRs we still get the desired behavior of build docs being independent of targets tests,
as we dont want developers to have to pass all target tests to test their doc changes.
2023-10-24 09:33:39 +08:00
Fu Hanxi
1726ce888d
ci: tune build test jobs parallel count 2023-10-23 10:43:51 +02:00
Fu Hanxi
eb37150780
ci: add all nightly run env vars while checking 2023-10-23 09:19:31 +02:00
Fu Hanxi
ad9b07f589
ci: add idf_pytest package into patterns to trigger target test jobs 2023-10-23 08:54:02 +02:00
Anton Maklakov
57e03f05e3 Merge branch 'ci/optimize_deploy_job_dependencies' into 'master'
ci: optimize deploy jobs dependencies

See merge request espressif/esp-idf!26610
2023-10-23 12:08:52 +08:00
Marius Vikhammer
9b8aa6a792 ci(build_test): add known-failure check 2023-10-22 09:12:45 +08:00
Marius Vikhammer
6a3cf1faa2 ci(build): add idf.py and idf.py actions as trigger for build-system test 2023-10-21 12:40:02 +08:00
Chen Yudong
a91f692358 ci: optimize deploy jobs dependencies 2023-10-20 18:51:30 +08:00
Fu Hanxi
ba3d23342d
ci: improve check pylint get files command 2023-10-19 11:53:18 +02:00
Chen Yudong
f1d8927ba0 ci: add qa-test-tag rules 2023-10-18 13:01:07 +08:00
Chen Yudong
1a2953ae13 ci: dynamic integration test child pipeline 2023-10-18 12:59:29 +08:00
Fu Hanxi
4dbf8c8948
ci: remove ttfw related files 2023-10-16 10:23:09 +02:00
Fu Hanxi
786a641851 Merge branch 'ci/build_jobs_use_cache' into 'master'
ci: Upload Artifacts to Minio Server Instead of Gitlab Server

Closes IDFCI-1653

See merge request espressif/esp-idf!25664
2023-10-16 00:49:47 +08:00
Chen Yudong
02f91d4f25 ci: fix pytest components ethernet rules 2023-10-13 21:33:25 +08:00
Fu Hanxi
79bc66b323
ci: upload to minio server instead of gitlab artifacts for pytest jobs 2023-10-13 13:01:23 +02:00
Fu Hanxi
9bef0db402
ci: remove SETUP_TOOLS since now we run install.sh for all jobs 2023-10-13 13:01:23 +02:00
Fu Hanxi
6be05d415b
ci: remove unit-test-app ci jobs 2023-10-13 13:01:16 +02:00
Fu Hanxi
a3a8c2acb9
ci: move default variables, before(after)_script into a single file
could help reuse the default settings on main pipeline and child
pipeline

renamed:
- .before_script_minimal -> .before_script:minimal
- .before_script_macos -> .before_script:build:macos
- .before_script_build_jobs -> .before_script:build
2023-10-13 12:55:51 +02:00
Ondrej Kosta
4f6c86ebb8 Merge branch 'feature/extend_eth_test_targets' into 'master'
CI tests for all supported Ethernet tests

Closes IDF-5704

See merge request espressif/esp-idf!24468
2023-10-13 18:44:10 +08:00
Peter Dragun
95b764970e Merge branch 'feat/uf2-to-esptool' into 'master'
feat(tools): move uf2 file generation to esptool

Closes IDF-7434

See merge request espressif/esp-idf!25655
2023-10-12 16:03:16 +08:00
Ondrej
3a3656ba38 ci(esp_eth): added tests of all supported Etherent chips 2023-10-12 07:23:51 +00:00
Anton Maklakov
2472c62fff Merge branch 'feature/add_qemu_to_tools-json' into 'master'
Add QEMU to tools.json

Closes QEMU-164

See merge request espressif/esp-idf!25973
2023-10-11 12:54:46 +08:00
Roland Dobai
81804be35f Merge branch 'fix/kconfigs_check_move' into 'master'
fix(tools/kconfig): Moved check_konfigs.py to esp-idf-kconfig package

Closes IDFGH-9516

See merge request espressif/esp-idf!26033
2023-10-10 17:58:43 +08:00
C.S.M
b2d7f4d0cd Merge branch 'bugfix/compile_usj_fail_c6' into 'master'
ci(usb_serial_jtag): Add a pytest for usb_serial_jtag print.

Closes IDF-3201

See merge request espressif/esp-idf!26231
2023-10-10 13:47:19 +08:00
Anton Maklakov
02802ea20a feat(tools): Add QEMU 8.0.0_20230522 to tools.json
Process wildcards in the install and download lists of idf_tools
    Fix the install and download handlers to get common behaviour
2023-10-10 09:59:38 +07:00
Cao Sen Miao
403f237825 feat(usb_serial_jtag): Add pytest for usb_serial_jtag 2023-10-10 10:05:46 +08:00