Merge branch 'docs/translate_esp_idf_tests_with_pytest' into 'master'

docs: update CN trans for esp-idf-tests-with-pytest.rst

Closes DOC-7863

See merge request espressif/esp-idf!30957
This commit is contained in:
Zhang Shu Xian 2024-05-29 18:59:48 +08:00
commit 200885d46b
2 changed files with 727 additions and 692 deletions

View File

@ -33,18 +33,18 @@ On the host side, ESP-IDF employs the pytest framework (alongside certain pytest
Installation
============
All dependencies could be installed by running the install script with the ``--enable-pytest`` argument:
All dependencies could be installed by running the ESP-IDF install script with the ``--enable-pytest`` argument:
.. code-block:: bash
$ install.sh --enable-pytest
We have implemented several mechanisms to ensure the successful execution of all installation processes. If you encounter any issues during the installation, please submit an issue report to our `GitHub issue tracker <https://github.com/espressif/esp-idf/issues>`__.
Several mechanisms have been implemented to ensure the successful execution of the installation processes. If you encounter any issues during installation, please submit an issue report to our `GitHub issue tracker <https://github.com/espressif/esp-idf/issues>`__.
Common Concepts
===============
A **test app** is a set of binaries which is being built from an IDF project that is used to test a particular feature of your project. Test apps are usually located under ``${IDF_PATH}/examples``, ``${IDF_PATH}/tools/test_apps``, and ``${IDF_PATH}/components/<COMPONENT_NAME>/test_apps``.
A **test app** is a set of binaries which are built from an IDF project that is used to test a particular feature of your project. Test apps are usually located under ``${IDF_PATH}/examples``, ``${IDF_PATH}/tools/test_apps``, and ``${IDF_PATH}/components/<COMPONENT_NAME>/test_apps``.
A **Device under test (DUT)** is a set of ESP chip(s) which connect to a host (e.g., a PC). The host is responsible for flashing the apps to the DUT, triggering the test cases, and inspecting the test results.
@ -91,7 +91,7 @@ Getting Started
def test_hello_world(dut) -> None:
dut.expect('Hello world!')
This is a simple test script that could run with our getting-started example :example:`get-started/hello_world`.
This is a simple test script that could run with the ESP-IDF getting-started example :example:`get-started/hello_world`.
First two lines are the target markers:
@ -114,7 +114,7 @@ Next, we have the environment marker:
Finally, we have the test function. With a ``dut`` fixture. In single-dut test cases, the ``dut`` fixture is an instance of ``IdfDut`` class, for multi-dut test cases, it is a tuple of ``IdfDut`` instances. For more details regarding the ``IdfDut`` class, please refer to `pytest-embedded IdfDut API reference <https://docs.espressif.com/projects/pytest-embedded/en/latest/api.html#pytest_embedded_idf.dut.IdfDut>`__.
Same App with Different sdkconfig Files
Same App With Different sdkconfig Files
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
For some test cases, you may need to run the same app with different sdkconfig files. For detailed documentation regarding sdkconfig related concepts, please refer to `idf-build-apps Documentation <https://docs.espressif.com/projects/idf-build-apps/en/latest/find_build.html>`__.
@ -150,10 +150,10 @@ If the test case needs to run all supported targets with these two sdkconfig fil
All markers will impact the test case simultaneously. Overall, this test function would be replicated to 4 test cases:
- ``test_foo_bar``, with esp32 target, and sdkconfig.ci.foo as the sdkconfig file
- ``test_foo_bar``, with esp32 target, and sdkconfig.ci.bar as the sdkconfig file
- ``test_foo_bar``, with esp32s2 target, and sdkconfig.ci.foo as the sdkconfig file
- ``test_foo_bar``, with esp32s2 target, and sdkconfig.ci.bar as the sdkconfig file
- ``test_foo_bar``, with esp32 target, and ``sdkconfig.ci.foo`` as the sdkconfig file
- ``test_foo_bar``, with esp32 target, and ``sdkconfig.ci.bar`` as the sdkconfig file
- ``test_foo_bar``, with esp32s2 target, and ``sdkconfig.ci.foo`` as the sdkconfig file
- ``test_foo_bar``, with esp32s2 target, and ``sdkconfig.ci.bar`` as the sdkconfig file
Sometimes in the test script or the log file, you may see the following format:
@ -185,7 +185,7 @@ The test case ID is used to identify the test case in the JUnit report.
'esp32s2',
], indirect=True)
Same App with Different sdkconfig Files, Different Targets
Same App With Different sdkconfig Files, Different Targets
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
For some test cases, you may need to run the same app with different sdkconfig files. These sdkconfig files supports different targets. We may use ``pytest.param`` to achieve this. Let's use the same folder structure as above.
@ -225,7 +225,7 @@ Multi-DUT Test Cases
Multi-Target Tests with the Same App
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
In some cases a test may involve multiple targets running the same test app. Parametrize ``count`` to the number of DUTs you want to test with.
In some cases a test may involve multiple targets running the same test app. Parameterize ``count`` to the number of DUTs you want to test with.
.. code-block:: python
@ -240,7 +240,7 @@ In some cases a test may involve multiple targets running the same test app. Par
dut[0].expect('Hello world!')
dut[1].expect('Hello world!')
The ``|`` symbol in all parametrized items is used for separating the settings for each DUT. In this example, the test case would be tested with:
The ``|`` symbol in all parameterized items is used for separating the settings for each DUT. In this example, the test case would be tested with:
* esp32, esp32s2
* esp32s3, esp32s3
@ -307,7 +307,7 @@ In some cases, a test may involve multiple targets running different test apps (
│ └── CMakeLists.txt
└── pytest_master_slave.py
In this case, we can parametrize the ``app_path`` to the path of the test apps you want to test with.
In this case, we can parameterize the ``app_path`` to the path of the test apps you want to test with.
.. code-block:: python
@ -338,7 +338,7 @@ The test case here will be replicated to 2 test cases:
Test Cases with Unity Test Framework
------------------------------------
We use `Unity test framework <https://github.com/ThrowTheSwitch/Unity>`__ in our unit tests. Overall, we have three types of test cases (`Unity test framework <https://github.com/ThrowTheSwitch/Unity>`__):
We use the `Unity test framework <https://github.com/ThrowTheSwitch/Unity>`__ in our unit tests. Overall, we have three types of test cases (`Unity test framework <https://github.com/ThrowTheSwitch/Unity>`__):
* Normal test cases (single DUT)
* Multi-stage test cases (single DUT)
@ -484,12 +484,12 @@ By default, each test case looks for the required binary files in the following
- ``build_<sdkconfig>``
- ``build``
As long as one of the above directories exists, the test case uses that directory to flash the binaries. If non of the above directories exists, the test case fails with an error.
As long as one of the above directories exists, the test case uses that directory to flash the binaries. If none of the above directories exists, the test case fails with an error.
Test Your Test Script
---------------------
Single-DUT Test Cases with ``sdkconfig.defaults``
Single-DUT Test Cases With ``sdkconfig.defaults``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
This is the simplest use case. Let's take :project:`examples/get-started/hello_world` as an example. Assume we're testing with a ESP32 board.
@ -500,7 +500,7 @@ This is the simplest use case. Let's take :project:`examples/get-started/hello_w
$ idf.py set-target esp32 build
$ pytest --target esp32
Single-DUT Test Cases with ``sdkconfig.ci.xxx``
Single-DUT Test Cases With ``sdkconfig.ci.xxx``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Some test cases may need to run with different sdkconfig files. Let's take :project:`examples/system/console/basic` as an example. Assume we're testing with a ESP32 board, and test with ``sdkconfig.ci.history``.
@ -566,7 +566,7 @@ Of course we can build the required binaries manually, but we can also use our C
Debug CI Test Cases
-------------------
Sometimes you can't reprocude the CI test case failure locally. In this case, you may need to debug the test case with the binaries built in CI.
Sometimes you can't reproduce the CI test case failure locally. In this case, you may need to debug the test case with the binaries built in CI.
Run pytest with ``--pipeline-id <pipeline_id>`` to force pytest to download the binaries from CI. For example:
@ -581,8 +581,8 @@ Even if you have ``build_esp32_default``, or ``build`` directory locally, pytest
<pipeline_id> should be the parent pipeline id. You can copy it in your MR page.
Pytest Tips and Tricks
======================
Pytest Tips & Tricks
====================
Custom Classes
--------------
@ -725,7 +725,7 @@ The above example would log the performance item with pre-defined format: ``[per
``check_performance``
^^^^^^^^^^^^^^^^^^^^^
We provide C macros ``TEST_PERFORMANCE_LESS_THAN`` and ``TEST_PERFORMANCE_GREATER_THAN`` to log the performance item and check if the value is in the valid range. Sometimes the performance item value could not be measured in C code, so we also provide a Python function for the same purpose. Please note that using C macros is the preferred approach, since the Python function could not recognize the threshold values of the same performance item under different ifdef blocks well.
We provide C macros ``TEST_PERFORMANCE_LESS_THAN`` and ``TEST_PERFORMANCE_GREATER_THAN`` to log the performance item and check if the value is in the valid range. Sometimes the performance item value could not be measured in C code, so we also provide a Python function for the same purpose. Please note that using C macros is the preferred approach, since the Python function could not recognize the threshold values of the same performance item under different ``#ifdef`` blocks well.
.. code-block:: python
@ -743,5 +743,5 @@ Let us assume the value of ``IDF_PERFORMANCE_MAX_RSA_2048KEY_PUBLIC_OP`` is 1900
Further Readings
================
- pytest documentation: https://docs.pytest.org/en/latest/contents.html
- pytest-embedded documentation: https://docs.espressif.com/projects/pytest-embedded/en/latest/
- `pytest documentation <https://docs.pytest.org/en/latest/contents.html/>`_
- `pytest-embedded documentation <https://docs.espressif.com/projects/pytest-embedded/en/latest/>`_

File diff suppressed because it is too large Load Diff