Aleksei Apaseev
63bd3a18ad
ci: add failed jobs report generator. Improve Target Test Report
...
Introduced changes:
- refactor the cli script used for report generation
- introduce failed jobs report generator
- cover job report generation with tests
- add job failure rate
- add test cases failure rate
- add current branch / other branches statistic for failed jobs / testcases
2024-06-20 14:40:54 +08:00
Aleksei Apaseev
40c8a16ad4
ci: add known failure cases to the target test report
...
target test report:
- refactor the methods to ensure reusability
- cover report generation with tests
- introduce a known failure tests block in report
- add grafana link to testcases dashboard
2024-05-28 15:36:22 +08:00
Fu Hanxi
152db1a4ba
ci: move log dir from pytest_embedded_log to pytest-embedded
2024-05-09 12:23:04 +02:00
Fu Hanxi
42d6316e91
ci: improve error message when no required tags, and no mandatory env marker
2024-04-29 10:47:33 +02:00
Fu Hanxi
31f5c1ffc4
ci: upload sdkconfig file
2024-04-24 21:01:34 +08:00
Fu Hanxi
ed2d492bde
ci: check missing runners only when needed
2024-04-22 15:38:36 +02:00
igor.udot
99a0ce0db0
ci: revert pytest.ini -s -vv
2024-04-19 22:26:29 +08:00
Xu Si Yu
0d01a3ed72
ci: re-enable openthread runner
2024-04-09 20:52:05 +08:00
Fu Hanxi
8f2aebdc20
ci: ignore openthread runner temporarily
2024-04-08 14:44:17 +02:00
Chen Yudong
7d13f8210f
ci: fix pytest generic env markers
2024-04-03 18:10:43 +08:00
Fu Hanxi
85f63f7ad2
ci: decrease test case amount per job
...
for now some jobs are reaching 1 hour timeout
2024-03-26 10:59:16 +01:00
Fu Hanxi
6acd57df02
ci: support trigger full pipeline by components
2024-03-20 10:03:48 +01:00
Fu Hanxi
d2b95201c3
ci: fix dynamic pipeline for tag pipelines
2024-03-01 08:17:20 +01:00
Chen Yu Dong
6af93c154b
Revert "Merge branch 'ci/ignore_c6_wifi_two_dut_runners' into 'master'"
...
This reverts merge request !28944
2024-02-27 14:24:37 +08:00
Fu Hanxi
0df25d5229
ci: ignore c3 flash_multi cases
2024-02-23 14:15:35 +01:00
Fu Hanxi
b5e6b883a5
ci: fix test related apps are force been built
2024-02-20 08:30:22 +01:00
Fu Hanxi
80a32713bb
ci: ignore esp32c6_2,wifi_two_dut test cases
2024-02-06 13:38:25 +01:00
Chen Yudong
a72d5f4b41
ci: update protocol/ethernet pytest markers
2024-02-01 22:47:18 +08:00
Chen Yudong
b27cf4d444
ci: update pre_commit jobs
2024-01-30 22:45:21 +08:00
Fu Hanxi
ecae464c52
ci: ignore p4,jtag runner
2024-01-30 09:08:35 +01:00
Fu Hanxi
ceb2ef7208
ci: add init known warnings while generating the test child pipeline
2024-01-29 13:52:40 +01:00
Fu Hanxi
08312412ec
ci: check warnings while generating the target test jobs
2024-01-29 13:52:40 +01:00
igor.udot
f5972ab592
ci: change known_failure_cases workflow
2024-01-21 15:36:25 +08:00
Fu Hanxi
b0af857d50
Merge branch 'ci/fix_search_multi_dut_with_markers' into 'master'
...
ci: fix collect multi-dut test case with markers issue
Closes IDFCI-1945
See merge request espressif/esp-idf!28448
2024-01-16 22:48:09 +08:00
Ondrej Kosta
9a3133f22a
Merge branch 'bugfix/ci_extra_flags' into 'master'
...
ci: added bridge test credentials to PYTEST_EXTRA_FLAGS variable
See merge request espressif/esp-idf!28470
2024-01-16 17:03:55 +08:00
Fu Hanxi
c8dffc9378
ci: fix collect multi-dut test case with markers issue
...
While collecting, we stop registering the new plugin.
Otherwise the new created plugin will override the one we passed
also run the tests inside idf. the behavior is different.
2024-01-15 22:36:09 +01:00
Fu Hanxi
9d3abb8686
Merge branch 'ci/fix_no_junit_report_for_failed_target_test_jobs' into 'master'
...
ci: always upload junit report
See merge request espressif/esp-idf!28435
2024-01-15 16:40:06 +08:00
Ondrej Kosta
e339da981d
ci: added bridge test credentials to PYTEST_EXTRA_FLAGS variable
2024-01-15 09:38:11 +01:00
Fu Hanxi
b581d81fc1
ci: support build only label
2024-01-14 21:18:20 +01:00
Fu Hanxi
055f943028
ci: always upload junit report
2024-01-12 09:15:48 +01:00
Fu Hanxi
19efb30536
ci: print report url
...
master/release pipelines does not have a mr comment, print it here
2024-01-12 08:57:26 +01:00
Fu Hanxi
fba96d58c2
ci: dynamic pipeline
...
build:
- upgrade idf-build-apps to 2.x
- unify get_pytest_apps and get_cmake_apps to get_all_apps
- returns (test_apps, non_test_apps) tuple
- add tests for the new get_all_apps
assign:
- generate build report
- generate target test pipeline based on the build report
target test:
- download artifacts from minio server
- users can use `pytest --pipeline-id xxxxx` to download and flash
the binaries from the artifacts
.post:
- generate target test reports
2024-01-10 15:37:34 +01:00
