He Yin Ling
3c99de352c
tiny-test-fw: only load module from the same file one time:
...
we should only load one module once.
if we load one module twice, python will regard the same object loaded in the first time and second time as different objects.
it will lead to strange errors like `isinstance(object, type_of_this_object)` return False
2019-11-23 20:40:23 +08:00
He Yin Ling
cb4c25bc67
ci: use more flexible assign for unit test cases
2019-11-23 20:39:19 +08:00
He Yin Ling
757e2d021e
tiny-test-fw: print useful debug info when assign test:
...
* total requirements (number of required jobs vs tags)
* unused CI jobs
* missing jobs (number of missing jobs vs tags)
2019-11-23 20:39:19 +08:00
He Yin Ling
b12a8bce5f
ci: optimize unit test assignment:
...
Currently we use config and test function as filter when assign cases to one CI job. It's not necessary as the runner can run test with different configs / test functions. Now we will try to assign as many cases to a job as possible, to reduce the amount of jobs required.
2019-11-23 20:39:19 +08:00
He Yin Ling
74ca3fc571
tiny-test-fw: revise unit test job config file:
...
`overwrite` should be included by each case config
2019-11-23 20:39:19 +08:00
Ivan Grokhotkov
0e6e7f49be
tiny-test-fw: add build config and target options
2019-11-21 15:36:06 +01:00
Ivan Grokhotkov
dd7dd08bfb
Merge branch 'bugfix/tinyfw_local_test' into 'master'
...
tiny-test-fw: restore the ability to run tests locally
See merge request espressif/esp-idf!6620
2019-11-07 21:27:58 +08:00
Roland Dobai
86fd13230e
CI: Don't download esp32s2beta builds into example tests
2019-11-07 06:56:03 +00:00
Ivan Grokhotkov
6306ea5a92
tiny-test-fw: fix running tests which use IDFDUT class
2019-11-06 15:31:16 +01:00
Angus Gratton
d3157910e0
ci: Fix "local variable referenced before assignment" if connecting to DUT fails
2019-10-17 14:38:49 +11:00
Ivan Grokhotkov
5830f529d8
Merge branch 'master' into feature/esp32s2beta_merge
2019-10-02 19:01:39 +02:00
Roland Dobai
5a916ce126
Support ELF files loadable with gdb
2019-09-24 07:19:50 +00:00
Angus Gratton
adfc06a530
Merge branch 'master' into feature/esp32s2beta_merge
2019-09-20 10:28:37 +10:00
Angus Gratton
438d513a95
Merge branch 'master' into feature/esp32s2beta_merge
2019-09-16 16:18:48 +10:00
Sergei Silnov
c57dfbc0b8
python: Fix yaml.load warnings
...
Since pyyaml 5.1 yaml.load without specifing loader is deprecated
Details: https://msg.pyyaml.org/load
To keep code compatible with older versions of pyyaml
and keep best perfomance CLoader with fallback to Loader is used.
2019-09-13 10:29:38 +00:00
Michael (XIAO Xufeng)
63329b169b
ci: limit example test to ESP32s
2019-09-04 10:53:25 +10:00
Michael (XIAO Xufeng)
3a9ae4acc6
ci: pass target from build stage to unit test
2019-09-04 10:53:25 +10:00
Michael (XIAO Xufeng)
4558824150
IDFDUT: seperate into different classes
...
The DUT should be created as the correct sub classes. This can be done
in the config file (UT_xxx_x.yml) Filter
2019-09-04 10:53:25 +10:00
Ivan Grokhotkov
12c9d9a564
spi_flash: remove duplicate definition of spi_flash_unlock
...
The other (static) definition is in flash_ops.c, all references are
also in flash_ops.c.
2019-08-23 12:37:55 +08:00
He Yin Ling
81dd9d4e27
CI: use parallel attribute in CI config file
2019-07-03 16:53:34 +08:00
He Yin Ling
af4daf6a22
test: include chip in test case ID
2019-04-19 14:39:44 +08:00
He Yin Ling
b75400294a
test: collect unit-test performance items and save to report
2019-04-19 14:39:44 +08:00
He Yin Ling
0462620a23
tiny-test-fw: support save performance to junit report:
...
1. support get performance from DUT
2. update performance to `testcase.stdout` in JunitReport
2019-04-19 12:17:13 +08:00
David Cermak
f05228be36
tiny-test-fw: update sdkconfig python dictionary to contain values without trailing newlines
2019-03-22 11:37:26 +01:00
He Yin Ling
b85e9e5cda
tiny-test-fw: support translate backtrace in IDFDUT
2019-03-19 11:24:08 +08:00
He Yin Ling
f11eba7802
tiny-test-fw: support detect exception in IDFDUT
2019-03-19 11:24:08 +08:00
Ivan Grokhotkov
b65a088791
tinyfw: update to support new esptool argument
2019-01-22 18:40:25 +08:00
David Cermak
b13a536041
mqtt tests: connect to local broker when running in CI to make the tests more reliable
2019-01-11 15:30:15 +01:00
He Yin Ling
1c54630eaa
test: fix failed to download in example test:
...
1. example test uses auto detect flash size. we need to call
`detect_flash_size` before write flash
2. fix incorrect baudrate used: when using WROVER-Kits, it's likely that download with baudrate 921600
will fail. If we don't reset serial setting in decorator, 921600 will
become the default baudrate. This causes all the subsequent
communication fails
3. do hw reset after used esptool function
2019-01-02 20:49:03 +08:00
Roland Dobai
4ca2b149e5
examples, tools: Fix Python3 deprecation warning for the imp module
2018-12-20 12:44:33 +01:00
Roland Dobai
bfa9610f58
tools: Fix the Python coding style
2018-12-19 11:56:24 +01:00
Angus Gratton
b3249d56f3
unit tests: Fix retrying with a new baud rate in IDFDUT.start_app()
2018-12-18 14:09:29 +11:00
Angus Gratton
f6e857c2b9
unit tests: Keep serial port open when running esptool
...
* Call esptool directly not via subprocess
* Use the same serial port instance for listener thread and esptool
* Includes some refactoring for encapsulation of App vs DUT members
2018-12-18 14:09:29 +11:00
He Yin Ling
5ae3a11c8d
tiny-test-fw: fix exception when expect RegEx:
...
match.groups() could return None. We can't do decode on None object.
2018-11-29 23:06:16 +08:00
He Yin Ling
ef9eaddd88
ci: assign unit test cases in sequence of config and name
2018-11-26 10:07:41 +08:00
He Yin Ling
644571f78b
test: generate junit test report according to executed cases
2018-11-26 10:07:41 +08:00
He Yin Ling
0ad226b375
tiny-test-fw: support capture raw data in DUT:
...
test cases might want to use `expect` and get raw data from DUT at the same time. New added capture method provides a way to do that.
2018-11-26 10:07:41 +08:00
He Yin Ling
8e70582cad
tiny-test-fw: enhanced junit test report support:
...
1. replace xunitgen with junit_xml, which can log more info
2. allow test cases to handle junit test report by them own
3. allow test cases to log some info into report via `sysout` tag
2018-11-22 20:49:15 +08:00
David Cermak
8ce1f55b59
mqtt: example tests refactored to provide descriptive failures, addapted to python23, removed workarround for launching without mqtt-paho package
2018-11-01 09:04:09 +01:00
Angus Gratton
f55509bd3f
Merge branch 'bugfix/py3_ci' into 'master'
...
Fix Python3 compatibility issues
See merge request idf/esp-idf!3487
2018-10-26 13:45:19 +08:00
Roland Dobai
9daf51e6be
Fix Python 3 compatibility issues
2018-10-24 11:20:54 +02:00
He Yin Ling
722a7e3b53
CI: only create required jobs when use bot to trigger test:
...
Gitlab CI now allow us to define if a job need to be created by
varialbes. This commit add label to CI jobs, so we can make some jobs
not created in certain scenarios when we trigger pipeline with @bot.
2018-10-24 09:19:39 +00:00
Roland Dobai
374f92c0a0
Tiny-test-fw: Fix string comparison which cases ignore of test cases
...
Example tests from examples/protocols/http_server/ were silently ignored
because of incompatible string comparisons.
2018-10-17 11:07:04 +02:00
Roland Dobai
9ea9d111c8
tools: Make Utility.console_log accept Unicode and byte strings as well
2018-09-26 12:56:00 +02:00
Roland Dobai
975688b97f
examples: Use flushed print to see logs on the CI server
2018-09-25 08:05:56 +02:00
Roland Dobai
17b7959de9
Examples: Add Python 2&3 support
2018-09-25 08:05:56 +02:00
Roland Dobai
130a8ea324
tiny-test-fw: Detect either 'download.config' or 'flash_project_args'
2018-09-20 06:39:04 +00:00
Roland Dobai
c3d99dd2e4
Encourage to install python packages without administrative rights
...
Using 'sudo pip install' can cause a lot of problems because usually
there are a couple of Python interpreters installed. This fix encourages
developers to use `pip install --user` instead which is safer.
2018-09-11 08:54:37 +02:00
Roland Dobai
bc307c920c
tiny-test-fw: Print more useful info when download.config doesn't exist
2018-08-27 12:47:08 +02:00
Ivan Grokhotkov
fd61c1440b
tiny-test-fw: invoke expect callbacks in order of appearance
...
Previously the callbacks were invoked in the same order as the list of
patterns to be matched.
2018-08-20 16:49:20 +08:00