Commit Graph

50 Commits

Author SHA1 Message Date
Fu Hanxi
851d832eb6 ci: enable test_apps s3 c3 tests
Refactor the tags a bit to make the tags could reuse the env markers.
They suppose to be the same thing.
2022-03-10 11:56:15 +08:00
Fu Hanxi
1f14dc5b74 ci: abort with all ImportErrors when detecting ImportError while search tests cases 2021-11-25 18:56:19 +08:00
He Yin Ling
2849f3bf44 Merge branch 'test/add_ethernet_iperf_example_test_case' into 'master'
Test: add ethernet iperf example test case

Closes TCI-463

See merge request espressif/esp-idf!13696
2021-07-28 01:39:12 +00:00
Lu Ai Jun
d29335710d test: add ethernet iperf example test case 2021-07-28 09:39:02 +08:00
He Yin Ling
35f8caf1cb Merge branch 'bugfix/ttfw_fix_flush_index_error' into 'master'
test: TTFW fix flush index error

See merge request espressif/esp-idf!14217
2021-07-22 12:55:27 +00:00
He Yin Ling
1708febe48 ttfw: fix incorrect length when flush data cache after expect:
data cache is unicode. while we use bytes in RegEx expect. The index of
matched pattern is calculated with bytes, could be different from
unicode. Now we fix this issue by using unicode in expect.
2021-07-13 13:07:08 +08:00
He Yin Ling
b79778d273 ttfw: full_stdout should not return data after pattern in expect 2021-07-13 13:07:08 +08:00
Angus Gratton
bfb602d0fc ci: Fix error message if insufficient jobs for test case tag 2021-07-05 12:08:36 +08:00
He Yin Ling
31773b8ed2 ttfw: fix DUT check functions are called after case finished:
DUT check functions are executed in receive thread. If we put data to
data cache before execute the check functions, the main thread could
handle the data cache first and mark test case finished. We will execute
check functions first to make sure it's executed before case finished.
2021-06-18 09:58:19 +08:00
He Yin Ling
201018da43 Merge branch 'ci/shorter_performance_test_log' into 'master'
ci: optimize performance test log

See merge request espressif/esp-idf!13607
2021-05-21 06:48:16 +00:00
He Yin Ling
8434c0c731 ttfw: fix DUT exception not added to junit report 2021-05-20 16:28:20 +08:00
Fu Hanxi
bccaab7e6c fix(ci): fix missing [Performance] prefix in junit report 2021-05-20 14:13:54 +08:00
Fu Hanxi
9b8d1e7826 fix(ci): return as failed when no test case result generated 2021-04-20 11:18:12 +08:00
Fu Hanxi
92a9ad0b31 ci: allow wildcard in known failure cases 2021-04-01 19:29:04 +08:00
Fu Hanxi
4ffd458ddd ci: bugfix: known failure cases cannot read files 2021-02-09 15:33:49 +08:00
Fu Hanxi
85d4bca81a feat(tiny_test_fw): unify all junit report test case name
new format: <target>.<config>.<case_name>, the default value of
"config" is "default"
2021-01-28 12:25:14 +08:00
Fu Hanxi
d63069068f feat(tiny_test_fw): ignore known failure cases result
py
2021-01-28 12:25:14 +08:00
Fu Hanxi
0146f258d7 style: format python files with isort and double-quote-string-fixer 2021-01-26 10:49:01 +08:00
He Yin Ling
5befaac4be ttfw: fix get dut failed when dut port is set as env variable 2021-01-14 17:22:11 +08:00
Angus Gratton
6d3e06a9a3 ttfw: Move TestCaseFailed exception and handle differently to other exceptions
But also ensure the string form of this exception is never empty, if it ends up
somewhere else.
2021-01-12 11:42:03 +11:00
Angus Gratton
441d1306be ci ttfw: Consistently handle unexpected exceptions in test cases 2021-01-12 11:42:03 +11:00
He Yin Ling
4e82540730 ttfw: save console log to file 2021-01-07 17:10:40 +08:00
Fu Hanxi
3a88626ec3 search case: print target as well 2020-12-15 17:56:53 +08:00
Angus Gratton
26fb74bc97 ci: ttfw: Encode serial port data to whatever the console encoding is
This is a bit of a hack, but gives us a way to always log it
2020-11-24 18:32:47 +11:00
Angus Gratton
37838803db ci: Log failure to close any DUT 2020-11-24 18:32:47 +11:00
Angus Gratton
66fb5a29bb Whitespace: Automated whitespace fixes (large commit)
Apply the pre-commit hook whitespace fixes to all files in the repo.

(Line endings, blank lines at end of file, trailing whitespace)
2020-11-11 07:36:35 +00:00
Ivan Grokhotkov
f16d231197 Merge branch 'feature/improve_load_env_log' into 'master'
CI: Improve load env config log output

See merge request espressif/esp-idf!9587
2020-09-16 01:59:05 +08:00
Fu Hanxi
edc7cc9c00 add component_ut in assign-test and target-test stage.
Add one template_test python file to get test case
2020-09-07 12:12:11 +08:00
Fu Hanxi
ef2813a068 CI: Improve load env config log output 2020-08-05 07:49:02 +00:00
Fu Hanxi
53aacef29c code clean up 2020-07-06 19:05:44 +08:00
Fu Hanxi
b26d42afe3 Now scan_tests will generate 'scan_<target>.json`
also updated CI yaml and shell scripts
2020-07-06 19:05:44 +08:00
Fu Hanxi
b76df30915 don't overwrite special cases dut 2020-06-01 13:39:09 +08:00
Fu Hanxi
377d3eaaa5 Move local target detection to ttfw_idf
rename upper_list to upper_list_or_str
minor fix for `unit_test.py` `is 'name'` -> `== 'name`
2020-06-01 12:08:57 +08:00
Fu Hanxi
5c92d36078 Add list support for ttfw_idf test decorators. Only replicate supported keys 2020-05-29 12:42:56 +08:00
Fu Hanxi
e553092d62 move ci_target_check to ttfw_idf, move ci_target_filter to AssignTest
mark `supported_in_ci` for AssignTest filter.
2020-05-29 12:42:56 +08:00
Fu Hanxi
38c288bc0e Add BOT_TARGET_FILTER to _apply_bot_filter
fix the bug that will calculate twice if ci_target is a list.
2020-05-29 12:42:56 +08:00
Fu Hanxi
bc026133c5 add ci_target filter for target and local target check 2020-05-29 12:42:56 +08:00
Fu Hanxi
9f8b63da38 Pass current target to test_method **overwrite. 2020-05-29 12:42:56 +08:00
Fu Hanxi
6c98d7e4bd Add multi target support for search examples
drop keyword `dut`, use `target` instead to assign`dut_class` to `Env`
2020-05-29 12:42:56 +08:00
Ivan Grokhotkov
3fe4adc98d ttfw: allow relative imports in test scripts 2020-05-12 22:27:06 +02:00
Ivan Grokhotkov
26f6136a7d ttfw: only send relevant DUT kwargs to serial.serial_for_url 2020-05-12 22:27:06 +02:00
Fu Hanxi
04342a73f2 Fix some python bugs left before 2020-04-30 06:47:49 +00:00
Fu Hanxi
76d118577e test_config: fix the bug that will generate !!python/unicode
As an example, [7964999 example_test_002_](https://gitlab.espressif.cn:6688/espressif/esp-idf/-/jobs/7964999/artifacts/file/examples/test_configs/example_test_002_.yml)
2020-04-08 11:19:13 +08:00
David Cermak
a463e80d43 ci: fix ttfw ignoring test failure if exception in print_debug_info 2020-04-05 20:07:27 +02:00
Ivan Grokhotkov
e63764b468 CI: test-apps introduction
Introducing feature of adding arbitrary projects which could be build or
executed in the CI for the only purpose as testing

Closes IDF-641
2020-02-06 16:46:41 +00:00
He Yin Ling
8df08de486 ttfw: fix incorrect relative import for python3 2019-12-30 20:20:14 +08:00
David Cermak
cec408c760 ci: ttfw update to support qemu dut_class
Updated missing example tests per latest ttfw refactoring,
updated pythons requirements.txt for ttfw_idf
2019-12-17 14:06:40 +01:00
Ivan Grokhotkov
85c97967d5 tiny-test-fw: add QEMU DUT 2019-12-17 14:06:40 +01:00
He Yin Ling
c906e2afee test: update example and unit tests with new import roles:
tiny_test_fw is a python package now. import it using normal way.
2019-12-07 10:34:54 +08:00
He Yin Ling
d621d0e88e tiny-test-fw: move to tools/esp_python_packages:
make `tiny_test_fw` as a package and move to root path of idf python
packages
2019-12-06 17:29:47 +08:00