mirror of
https://github.com/espressif/esp-idf.git
synced 2024-09-20 00:36:01 -04:00
docs: add multi dut chapter
This commit is contained in:
parent
52b5a8348e
commit
3697cd0440
@ -19,6 +19,11 @@ In ESP-IDF, we use the following plugins by default:
|
|||||||
|
|
||||||
All the introduced concepts and usages are based on the default behavior in ESP-IDF. Not all of them are available in vanilla pytest.
|
All the introduced concepts and usages are based on the default behavior in ESP-IDF. Not all of them are available in vanilla pytest.
|
||||||
|
|
||||||
|
Installation
|
||||||
|
------------
|
||||||
|
|
||||||
|
``$ pip install -U pytest-embedded-serial-esp~=0.7.0 pytest-embedded-idf~=0.7.0``
|
||||||
|
|
||||||
Basic Concepts
|
Basic Concepts
|
||||||
--------------
|
--------------
|
||||||
|
|
||||||
@ -95,12 +100,12 @@ Pytest Execution Process
|
|||||||
|
|
||||||
1. Get all the python files with the prefix ``pytest_``
|
1. Get all the python files with the prefix ``pytest_``
|
||||||
2. Get all the test functions with the prefix ``test_``
|
2. Get all the test functions with the prefix ``test_``
|
||||||
3. Apply the `params <https://docs.pytest.org/en/latest/how-to/parametrize.html>`__, duplicate the test functions.
|
3. Apply the `params <https://docs.pytest.org/en/latest/how-to/parametrize.html>`__, and duplicate the test functions.
|
||||||
4. Filter the test cases with CLI options. Introduced detail usages `here <#filter-the-test-cases>`__
|
4. Filter the test cases with CLI options. Introduced detailed usages `here <#filter-the-test-cases>`__
|
||||||
|
|
||||||
3. Test Running Phase
|
3. Test Running Phase
|
||||||
|
|
||||||
1. Construct the `fixtures <https://docs.pytest.org/en/latest/how-to/fixtures.html>`__. In ESP-IDF, the common fixtures are initialized with this order:
|
1. Construct the `fixtures <https://docs.pytest.org/en/latest/how-to/fixtures.html>`__. In ESP-IDF, the common fixtures are initialized in this order:
|
||||||
|
|
||||||
1. ``pexpect_proc``: `pexpect <https://github.com/pexpect/pexpect>`__ instance
|
1. ``pexpect_proc``: `pexpect <https://github.com/pexpect/pexpect>`__ instance
|
||||||
|
|
||||||
@ -116,7 +121,7 @@ Pytest Execution Process
|
|||||||
|
|
||||||
2. Run the real test function
|
2. Run the real test function
|
||||||
|
|
||||||
3. Deconstruct the fixtures with this order:
|
3. Deconstruct the fixtures in this order:
|
||||||
|
|
||||||
1. ``dut``
|
1. ``dut``
|
||||||
|
|
||||||
@ -198,7 +203,7 @@ You can use ``pytest.mark.parametrize`` with “config” to apply the same test
|
|||||||
'nohistory', # <-- run with app built by sdkconfig.ci.nohistory
|
'nohistory', # <-- run with app built by sdkconfig.ci.nohistory
|
||||||
], indirect=True) # <-- `indirect=True` is required
|
], indirect=True) # <-- `indirect=True` is required
|
||||||
|
|
||||||
Overall, this test case would be duplicated to 4 test functions:
|
Overall, this test function would be replicated to 4 test cases:
|
||||||
|
|
||||||
- esp32.history.test_console_advanced
|
- esp32.history.test_console_advanced
|
||||||
- esp32.nohistory.test_console_advanced
|
- esp32.nohistory.test_console_advanced
|
||||||
@ -208,6 +213,80 @@ Overall, this test case would be duplicated to 4 test functions:
|
|||||||
Advanced Examples
|
Advanced Examples
|
||||||
~~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
Multi Dut Tests with the Same App
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
This code example is taken from :idf_file:`pytest_usb_host.py <tools/test_apps/peripherals/usb/pytest_usb_host.py>`.
|
||||||
|
|
||||||
|
.. code:: python
|
||||||
|
|
||||||
|
@pytest.mark.esp32s2
|
||||||
|
@pytest.mark.esp32s3
|
||||||
|
@pytest.mark.usb_host
|
||||||
|
@pytest.mark.parametrize('count', [
|
||||||
|
2,
|
||||||
|
], indirect=True)
|
||||||
|
def test_usb_host(dut: Tuple[IdfDut, IdfDut]) -> None:
|
||||||
|
device = dut[0] # <-- assume the first dut is the device
|
||||||
|
host = dut[1] # <-- and the second dut is the host
|
||||||
|
...
|
||||||
|
|
||||||
|
After setting the param ``count`` to 2, all these fixtures are changed into tuples.
|
||||||
|
|
||||||
|
Multi Dut Tests with Different Apps
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
This code example is taken from :idf_file:`pytest_wifi_getting_started.py <examples/wifi/getting_started/pytest_wifi_getting_started.py>`.
|
||||||
|
|
||||||
|
.. code:: python
|
||||||
|
|
||||||
|
@pytest.mark.esp32
|
||||||
|
@pytest.mark.multi_dut_generic
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
'count, app_path', [
|
||||||
|
(2,
|
||||||
|
f'{os.path.join(os.path.dirname(__file__), "softAP")}|{os.path.join(os.path.dirname(__file__), "station")}'),
|
||||||
|
], indirect=True
|
||||||
|
)
|
||||||
|
def test_wifi_getting_started(dut: Tuple[IdfDut, IdfDut]) -> None:
|
||||||
|
softap = dut[0]
|
||||||
|
station = dut[1]
|
||||||
|
...
|
||||||
|
|
||||||
|
Here the first dut was flashed with the app :idf_file:`softap <examples/wifi/getting_started/softAP/main/softap_example_main.c>`, and the second dut was flashed with the app :idf_file:`station <examples/wifi/getting_started/station/main/station_example_main.c>`.
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
|
||||||
|
Here the ``app_path`` should be set with absolute path. the ``__file__`` macro in python would return the absolute path of the test script itself.
|
||||||
|
|
||||||
|
Multi Dut Tests with Different Apps, and Targets
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
This code example is taken from :idf_file:`pytest_wifi_getting_started.py <examples/wifi/getting_started/pytest_wifi_getting_started.py>`. As the comment says, for now it's not running in the ESP-IDF CI.
|
||||||
|
|
||||||
|
.. code:: python
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
'count, app_path, target', [
|
||||||
|
(2,
|
||||||
|
f'{os.path.join(os.path.dirname(__file__), "softAP")}|{os.path.join(os.path.dirname(__file__), "station")}',
|
||||||
|
'esp32|esp32s2'),
|
||||||
|
(2,
|
||||||
|
f'{os.path.join(os.path.dirname(__file__), "softAP")}|{os.path.join(os.path.dirname(__file__), "station")}',
|
||||||
|
'esp32s2|esp32'),
|
||||||
|
],
|
||||||
|
indirect=True,
|
||||||
|
)
|
||||||
|
def test_wifi_getting_started(dut: Tuple[IdfDut, IdfDut]) -> None:
|
||||||
|
softap = dut[0]
|
||||||
|
station = dut[1]
|
||||||
|
...
|
||||||
|
|
||||||
|
Overall, this test function would be replicated to 2 test cases:
|
||||||
|
|
||||||
|
- softap with esp32 target, and station with esp32s2 target
|
||||||
|
- softap with esp32s2 target, and station with esp32 target
|
||||||
|
|
||||||
Support different targets with different sdkconfig files
|
Support different targets with different sdkconfig files
|
||||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
@ -278,7 +357,7 @@ Mark Known Failure Cases
|
|||||||
Sometimes a test couldn't pass for the following reasons:
|
Sometimes a test couldn't pass for the following reasons:
|
||||||
|
|
||||||
- Has a bug
|
- Has a bug
|
||||||
- Success ratio too low because of environment issue, such as network issue. Retry couldn't help
|
- The success ratio is too low because of environment issue, such as network issue. Retry couldn't help
|
||||||
|
|
||||||
Now you may mark this test case with marker `xfail <https://docs.pytest.org/en/latest/how-to/skipping.html#xfail-mark-test-functions-as-expected-to-fail>`__ with a user-friendly readable reason.
|
Now you may mark this test case with marker `xfail <https://docs.pytest.org/en/latest/how-to/skipping.html#xfail-mark-test-functions-as-expected-to-fail>`__ with a user-friendly readable reason.
|
||||||
|
|
||||||
|
Loading…
Reference in New Issue
Block a user