Commit Graph

187 Commits

Author SHA1 Message Date
Fu Hanxi
636b311ec0 ci: support keyword nightly_run in decorator
The test case with keyword `nightly_run` would be skipped in normal CI
pipelines. only would be triggered when `NIGHTLY_RUN` in environment
variables.
2021-11-03 11:11:38 +08:00
Krzysztof
3247f66783 tools: update ttfw to support ESP32-H2 and ESP32-C6 chips 2021-10-22 16:29:46 +08:00
Ivan Grokhotkov
70d1f94739 ci: retry Gitlab operations on error 500 2021-10-14 17:42:48 +02:00
SalimTerryLi
4c7b31f6ff
ci: test_apps: add option to parse original unity output 2021-09-08 18:54:20 +08:00
laokaiyao
04970fe487 iperfUtility: add udp rx bandwidth scan 2021-09-01 19:11:28 +08:00
Sachin Parekh
cfcb893289 ttfw_idf: Added ESP32-S3 FPGA support
Provision to burn and reset efuses of ESP32S3 on FPGA

test_app: Enabled automated test cases for ESP32S3
2021-08-27 10:41:31 +05:30
Sachin Parekh
65db2d3d80 ttfw_idf: Added IDFFPGADUT class
Provision to burn and reset efuses of ESP32C3 chip on FPGA
2021-08-27 09:58:26 +05:30
He Yin Ling
2849f3bf44 Merge branch 'test/add_ethernet_iperf_example_test_case' into 'master'
Test: add ethernet iperf example test case

Closes TCI-463

See merge request espressif/esp-idf!13696
2021-07-28 01:39:12 +00:00
Lu Ai Jun
d29335710d test: add ethernet iperf example test case 2021-07-28 09:39:02 +08:00
He Yin Ling
35f8caf1cb Merge branch 'bugfix/ttfw_fix_flush_index_error' into 'master'
test: TTFW fix flush index error

See merge request espressif/esp-idf!14217
2021-07-22 12:55:27 +00:00
He Yin Ling
1708febe48 ttfw: fix incorrect length when flush data cache after expect:
data cache is unicode. while we use bytes in RegEx expect. The index of
matched pattern is calculated with bytes, could be different from
unicode. Now we fix this issue by using unicode in expect.
2021-07-13 13:07:08 +08:00
He Yin Ling
b79778d273 ttfw: full_stdout should not return data after pattern in expect 2021-07-13 13:07:08 +08:00
Angus Gratton
bfb602d0fc ci: Fix error message if insufficient jobs for test case tag 2021-07-05 12:08:36 +08:00
He Yin Ling
31773b8ed2 ttfw: fix DUT check functions are called after case finished:
DUT check functions are executed in receive thread. If we put data to
data cache before execute the check functions, the main thread could
handle the data cache first and mark test case finished. We will execute
check functions first to make sure it's executed before case finished.
2021-06-18 09:58:19 +08:00
Angus Gratton
059353b0c2 Merge branch 'feature/refactor_common_secure_boot_code' into 'master'
secure_boot/flash_encryption: Refactoring

Closes IDF-2582 and IDF-2035

See merge request espressif/esp-idf!12963
2021-06-16 23:21:45 +00:00
Konstantin Kondrashov
f339b3fc96 efuse(esp32): Deprecate esp_efuse_burn_new_values() & esp_efuse_write_random_key()
These functions were used only for esp32 in secure_boot and flash encryption.
Use idf efuse APIs instead of efuse regs.
2021-06-17 07:21:36 +08:00
Marius Vikhammer
d9f5126d24 CI: enable running unit tests in CI for S3 2021-06-15 11:13:50 +08:00
Fu Hanxi
a624d3bf32 ci: retry download artifacts 2021-06-08 15:04:06 +08:00
Shivani Tipnis
2d22374460 ble-wifi-example-tests: Add fixes and cleanups to ble and wifi tests 2021-05-28 19:50:28 +05:30
He Yin Ling
201018da43 Merge branch 'ci/shorter_performance_test_log' into 'master'
ci: optimize performance test log

See merge request espressif/esp-idf!13607
2021-05-21 06:48:16 +00:00
He Yin Ling
79cd45fdee Merge branch 'test/support_multiple_targets_for_iperf_example_test' into 'master'
test: support multiple targets for iperf example test

Closes TCI-656

See merge request espressif/esp-idf!13527
2021-05-21 06:43:21 +00:00
He Yin Ling
016a37c8ad ci: save built binaries could be tested locally:
we have some test cases not executed in CI. we need to save those
binaries as artifacts so we can test locally.
2021-05-20 16:58:39 +08:00
He Yin Ling
8434c0c731 ttfw: fix DUT exception not added to junit report 2021-05-20 16:28:20 +08:00
Fu Hanxi
bccaab7e6c fix(ci): fix missing [Performance] prefix in junit report 2021-05-20 14:13:54 +08:00
Fu Hanxi
4304866780 ci: shorter performance test log 2021-05-20 14:13:54 +08:00
KonstantinKondrashov
2e4b625f59 simple_ota_example: Adds sha256 check for app images 2021-05-05 22:18:40 +08:00
Cao Sen Miao
2059990c94 ci: Add more information for flash performance test 2021-04-25 17:59:10 +08:00
Fu Hanxi
9b8d1e7826 fix(ci): return as failed when no test case result generated 2021-04-20 11:18:12 +08:00
Fu Hanxi
15b6389ca7 fix(ci): ci_fetch_submodule return full match name first 2021-04-19 19:34:11 +08:00
Fu Hanxi
7651e031cb fix: running ttfw tests locally hang forever while detecting ports 2021-04-14 11:43:13 +08:00
Fu Hanxi
92a9ad0b31 ci: allow wildcard in known failure cases 2021-04-01 19:29:04 +08:00
Fu Hanxi
4ffd458ddd ci: bugfix: known failure cases cannot read files 2021-02-09 15:33:49 +08:00
Ivan Grokhotkov
9a20283485 Merge branch 'bugfix/scan_test_missing_build_apps_without_tests' into 'master'
ci: bugfix: scan_test missing build apps without tests

See merge request espressif/esp-idf!12138
2021-01-28 17:48:24 +08:00
Fu Hanxi
85d4bca81a feat(tiny_test_fw): unify all junit report test case name
new format: <target>.<config>.<case_name>, the default value of
"config" is "default"
2021-01-28 12:25:14 +08:00
Fu Hanxi
d63069068f feat(tiny_test_fw): ignore known failure cases result
py
2021-01-28 12:25:14 +08:00
Fu Hanxi
be3d74efb2 fix(ci): scan_test missing build apps without tests 2021-01-27 12:35:49 +08:00
Fu Hanxi
0146f258d7 style: format python files with isort and double-quote-string-fixer 2021-01-26 10:49:01 +08:00
Aditya Patwardhan
b77c85c1e1 ttfw/IDFDUT: i) Added option to erase_flash in the IDFDUT class
ii) Renamed the method dump_flush to dump_flash
2021-01-20 11:12:14 +00:00
He Yin Ling
5befaac4be ttfw: fix get dut failed when dut port is set as env variable 2021-01-14 17:22:11 +08:00
Angus Gratton
6d3e06a9a3 ttfw: Move TestCaseFailed exception and handle differently to other exceptions
But also ensure the string form of this exception is never empty, if it ends up
somewhere else.
2021-01-12 11:42:03 +11:00
Angus Gratton
441d1306be ci ttfw: Consistently handle unexpected exceptions in test cases 2021-01-12 11:42:03 +11:00
He Yin Ling
97220af848 test: use pyecharts to replace matplotlib in iperf test 2021-01-07 17:10:41 +08:00
He Yin Ling
4e82540730 ttfw: save console log to file 2021-01-07 17:10:40 +08:00
He Yin Ling
16a488c405 test: fix iperf example errors:
1. fix TypeError when running with python3
2. fix throughput chart x/y axis label error
3. make test case compatibile with iperf bin on earlier release branches
2021-01-07 17:10:40 +08:00
Angus Gratton
ec75f923ef Merge branch 'ci/raise_esptool_errors' into 'master'
ci: Don't swallow exception details from flashing process

See merge request espressif/esp-idf!11803
2020-12-30 11:22:57 +08:00
Marius Vikhammer
7fe16bae25 tools: merge C3 changes into master 2020-12-30 07:04:12 +08:00
Angus Gratton
88d0d1fa02 ci: Don't swallow exception details from flashing process
If flashing fails, throw an exception that includes some information about the failure.
2020-12-29 15:41:22 +11:00
Omar Chebib
fcbc706a66 ci: fix flasher_args.json parser (iterate over dictionary)
Closes IDFCI-347
2020-12-29 12:27:51 +08:00
Ivan Grokhotkov
0b31f6a5fc Merge branch 'feature/ci_flasher_args_json_encrypted_flag' into 'master'
ci: use "encrypted" information in flasher_args.json

Closes IDF-2231

See merge request espressif/esp-idf!11551
2020-12-28 06:37:05 +08:00
Omar Chebib
05f97542b8 ci: use "encrypted" information in flasher_args.json
Take into account the new field "encrypted" that is part of the
partition entries in flasher_args.json file
Closes IDF-2231
2020-12-23 18:16:17 +08:00
Fu Hanxi
eda42035d2 scan test: bugfix: now can build multi target for one app 2020-12-15 17:56:53 +08:00
Fu Hanxi
3a88626ec3 search case: print target as well 2020-12-15 17:56:53 +08:00
Fu Hanxi
a5bcaea9cc use "additional_duts" to overwrite dut class for different targets 2020-12-15 17:55:54 +08:00
Fu Hanxi
7281a5a0cc Split component_ut tests based on targets
- add more label in CIScanTest.py
- fix the bug that will cause labels not iterate correctly
2020-11-30 14:08:54 +08:00
Angus Gratton
190aed41ee Merge branch 'bugfix/ttfw_example_test_fails_port_remains_open' into 'master'
ci: Fix CI issue if one example test fails, and if serial port returns some noise

Closes IDFCI-236

See merge request espressif/esp-idf!11273
2020-11-26 08:04:00 +08:00
Angus Gratton
26fb74bc97 ci: ttfw: Encode serial port data to whatever the console encoding is
This is a bit of a hack, but gives us a way to always log it
2020-11-24 18:32:47 +11:00
Angus Gratton
37838803db ci: Log failure to close any DUT 2020-11-24 18:32:47 +11:00
He Yin Ling
e568daf213 ttfw: fix idf performance check doesn't work 2020-11-12 09:49:44 +00:00
Mahavir Jain
0fb65d0bae Merge branch 'bugfix/ttfw_idf_find_artifacts_inaccurate_with_same_prefix_path' into 'master'
CI: fix the bug returns wrong artifact path when app_name is a subset of a longer one

Closes IDFCI-217

See merge request espressif/esp-idf!11187
2020-11-12 12:12:51 +08:00
Angus Gratton
66fb5a29bb Whitespace: Automated whitespace fixes (large commit)
Apply the pre-commit hook whitespace fixes to all files in the repo.

(Line endings, blank lines at end of file, trailing whitespace)
2020-11-11 07:36:35 +00:00
Fu Hanxi
5fa935e5ea CI: fix the bug returns wrong artifact path when app_name is a subset of a longer one 2020-11-11 12:31:39 +08:00
Fu Hanxi
75ab9223f9 scan test when weekend test. 2020-10-26 10:25:21 +08:00
Ivan Grokhotkov
2218ac91c2 tools/ci: fix import error when running ttfw tests locally 2020-10-13 18:09:22 +02:00
morris
9fa06719fa global: enable build uinit test for esp32-s3 2020-09-22 15:15:03 +08:00
Ivan Grokhotkov
d17a3c76d0 ci: fetch submodules from mirror, if enabled 2020-09-16 21:22:26 +02:00
Ivan Grokhotkov
f16d231197 Merge branch 'feature/improve_load_env_log' into 'master'
CI: Improve load env config log output

See merge request espressif/esp-idf!9587
2020-09-16 01:59:05 +08:00
Fu Hanxi
edc7cc9c00 add component_ut in assign-test and target-test stage.
Add one template_test python file to get test case
2020-09-07 12:12:11 +08:00
Fu Hanxi
e35328afd9 build component unit test apps
use idf_py_actions supported targets
2020-09-07 12:11:55 +08:00
Fu Hanxi
ef2813a068 CI: Improve load env config log output 2020-08-05 07:49:02 +00:00
Fu Hanxi
0478edc6ba CI: download only required bin for unit-tests. Refactor AssignTest related code 2020-08-04 13:27:16 +08:00
He Yin Ling
4c32c71db9 Merge branch 'feature/support_python3_autotest' into 'master'
test: run IT jobs with python3

See merge request espressif/esp-idf!9673
2020-07-23 16:07:43 +08:00
Chen Yudong
4088702077 test: auto test script supports python3 2020-07-22 16:14:21 +08:00
Alexey Gerenkov
0e3c7918e4 examples: Fixes 'apptrace_to_host' to work with updated OpenOCD tracing command 2020-07-22 06:49:27 +00:00
Fu Hanxi
6885421976 CI: add size info for binaries 2020-07-21 16:00:05 +08:00
Fu Hanxi
34f459c1e1 remove this line. it will block pipelines without unit test 2020-07-06 19:05:44 +08:00
Fu Hanxi
4087152305 remove build_or_not attr. 2020-07-06 19:05:44 +08:00
Fu Hanxi
270e7dec38 update after MR reviews 2020-07-06 19:05:44 +08:00
Fu Hanxi
010d7e9023 apply to make build system 2020-07-06 19:05:44 +08:00
Fu Hanxi
d7639d5cf8 add build-all flag and BUILD_ALL_APPS ci environment variable 2020-07-06 19:05:44 +08:00
Fu Hanxi
a238b266c2 fix after rebase 2020-07-06 19:05:44 +08:00
Fu Hanxi
53aacef29c code clean up 2020-07-06 19:05:44 +08:00
Fu Hanxi
b26d42afe3 Now scan_tests will generate 'scan_<target>.json`
also updated CI yaml and shell scripts
2020-07-06 19:05:44 +08:00
Angus Gratton
002a13a71a ci provisioning: Add some experiments to check the runner's STA state
Not clear why sometimes it associates without negotiating a new DHCP lease.
2020-06-28 22:52:45 +00:00
Angus Gratton
7934df3a6f ci wifi_tools: Log the wpa_supplicant interface state when trying to connect
Trigger reconnection if wpa_supplicant seems to have dropped the connection.
2020-06-28 22:52:45 +00:00
Angus Gratton
40d7ea68af Merge branch 'bugfix/ci_softap_example_test' into 'master'
Set sleep time to allow wlan0 to connect to softap

See merge request espressif/esp-idf!9258
2020-06-25 09:06:55 +08:00
Shivani Tipnis
76232fd263 ci: Set sleep time to allow wlan0 to connect to softap 2020-06-24 23:07:46 +05:30
Roland Dobai
50e817fc77 CI: Ignore error if py_debug_backend not installed 2020-06-23 13:45:12 +02:00
Ivan Grokhotkov
b3a76a9d83 Merge branch 'feature/idf_monitor_debug_ws' into 'master'
tools/idf_monitor: add WebSocket client for IDE integration

Closes IDF-1719

See merge request espressif/esp-idf!9032
2020-06-22 15:50:11 +08:00
Ivan Grokhotkov
cf155161c4 Merge branch 'feature/higher_level_com_gdb' into 'master'
CI: Use higher-level interaction with GDB in example tests and test apps

Closes IDF-1622

See merge request espressif/esp-idf!8840
2020-06-19 16:13:01 +08:00
Fu Hanxi
23822e2e3f ci: retry download if catched IOError/EOFError 2020-06-17 16:03:49 +08:00
Roland Dobai
e67314f646 tools/idf_monitor: add WebSocket client for IDE integration 2020-06-10 10:56:20 +00:00
Roland Dobai
493c852b73 CI: Use higher-level interaction with GDB in example tests and test apps 2020-06-05 14:58:34 +00:00
Fu Hanxi
b76df30915 don't overwrite special cases dut 2020-06-01 13:39:09 +08:00
Fu Hanxi
3b17b9e393 panic_test contains some reassign dut cases 2020-06-01 12:08:57 +08:00
Fu Hanxi
377d3eaaa5 Move local target detection to ttfw_idf
rename upper_list to upper_list_or_str
minor fix for `unit_test.py` `is 'name'` -> `== 'name`
2020-06-01 12:08:57 +08:00
Fu Hanxi
e99172fbac make upper_list do not make str into list 2020-05-29 12:42:57 +08:00
Fu Hanxi
d06d2a2b9f remove typing as a dependency.
return None if the input is None
2020-05-29 12:42:56 +08:00
Fu Hanxi
5c92d36078 Add list support for ttfw_idf test decorators. Only replicate supported keys 2020-05-29 12:42:56 +08:00
Fu Hanxi
e553092d62 move ci_target_check to ttfw_idf, move ci_target_filter to AssignTest
mark `supported_in_ci` for AssignTest filter.
2020-05-29 12:42:56 +08:00
Fu Hanxi
38c288bc0e Add BOT_TARGET_FILTER to _apply_bot_filter
fix the bug that will calculate twice if ci_target is a list.
2020-05-29 12:42:56 +08:00