mirror of https://github.com/espressif/esp-idf.git synced 2024-10-05 20:47:46 -04:00

History

Ivan Grokhotkov 47659be5b8 build system: remove lwip from common requirements lwip was added to common requirements list to provide "sys/socket.h" header to all components without additional requirements specified. However, lwip pulls in a lot of dependencies on other components. This commit removes lwip from common requirements to reduce the number of components in G1-only apps. To compensate for this removal, the following changes are made: - newlib (which is a common requirement) has a public dependency on lwip if lwip is present in the build. This ensures that sys/socket.h is available as long as lwip component is included into the build. - lwip is now a public requirement of esp-tls since esp_tls.h includes sys/socket.h header. - lwip is now a public requirement o esp_http_client because sys/socket.h is included from esp_http_client.h - lwip is now a private requirement of esp_wifi for "smartconfig_ack" - lwip is now a private requirement of mqtt for socket functions - lwip is now a public requirement of tcp_transport because esp_transport_tcp.h includes sys/socket.h header. - mbedtls checks if lwip component is present in the build. If yes, net_sockets.c is added to the build, along with the dependency on lwip. Previously lwip was a public requirement of mbedtls unconditionally. system/g1_components test app is updated to reflect the changes Default public dependencies of a component before and after this change, except common requirements: - esp_timer (public dependency of freertos) - bootloader_support (public dependency of esp_hw_support) - vfs (public dependency of lwip) - esp_wifi (public dependency of lwip) - esp_event (public dependency of esp_wifi) - esp_netif (public dependency of esp_event) - esp_eth (public dependency of esp_netif) - esp_phy (public dependency of esp_wifi) After: - esp_timer (public dependency of freertos) - bootloader_support (public dependency of esp_hw_support) Altogether, the following components have been always added as public requirements to all other components, and are not added now ([breaking-change]): - lwip - vfs - esp_wifi - esp_event - esp_netif - esp_eth - esp_phy Application components now need to explicitly declare dependencies on these components.		2022-05-02 20:47:17 +02:00
..
build_system	esp8684: rename target to esp32c2	2022-01-19 11:08:57 +08:00
peripherals	usb_host: Pin test task to core 0	2022-03-21 18:34:25 +01:00
phy/phy_multi_init_data_test	ci: refactor a few test scripts to pytest	2022-03-10 11:51:45 +08:00
protocols	examples: Fix implicit includes after legacy code removal	2022-03-10 10:36:38 +01:00
security/secure_boot	esp_hw_support/esp_system: Re-evaluate header inclusions and include directories	2022-03-07 11:18:08 +05:30
system	build system: remove lwip from common requirements	2022-05-02 20:47:17 +02:00
README.md	docs: add README about pytest in IDF	2022-03-03 14:36:50 +08:00

README.md

Test Apps

This directory contains a set of ESP-IDF projects to be used as tests only, which aim to exercise various configuration of components to check completely arbitrary functionality should it be building only, executing under various conditions or combination with other components, including custom test frameworks.

The test apps are not intended to demonstrate the ESP-IDF functionality in any way.

Test Apps projects

Test applications are treated the same way as ESP-IDF examples, so each project directory shall contain

Build recipe in cmake and the main component with app sources
Configuration files, sdkconfig.ci and similar (see below)
Test executor in ttfw_idf format if the project is intended to also run tests (otherwise the example is build only)
- test file in the project dir must end with _test.py, by should be named app_test.py
- test cases shall be decorated with @ttfw_idf.idf_custom_test(env_tag="...")

Test Apps layout

The test apps should be grouped into subdirectories by category. Categories are:

protocols contains test of protocol interactions.
network contains system network tests
system contains tests on the internal chip features, debugging and development tools.
security contains tests on the chip security features.

CI Behavior

Configuration Files

For each project in test_apps (and also examples):

If a file sdkconfig.ci exists then it's built as the default CI config.
If any additional files sdkconfig.ci.<CONFIG> exist then these are built as alternative configs, with the specified <CONFIG> name.

The CI system expects to see at least a "default" config, so add sdkconfig.ci before adding any sdkconfig.ci.CONFIG files.

By default, every CI configurations is built for every target SoC (an m * n configuration matrix). However if any sdkconfig.ci.* file contains a line of the form CONFIG_IDF_TARGET="targetname" then that CI config is only built for that one target. This only works in sdkconfig.ci.CONFIG, not in the default sdkconfig.ci.
Each configuration is also built with the contents of any sdkconfig.defaults file or a file named sdkconfig.defaults.<TARGET> appended. (Same as a normal ESP-IDF project build.)

Test Execution

If an example test or test app test supports more targets than just ESP32, then the app_test.py file needs to specify the list of supported targets in the test decorator. For example:

@ttfw_idf.idf_example_test(env_tag='Example_GENERIC', target=['esp32', 'esp32s2'])
def test_app_xyz(env, extra_data):

If the app test supports multiple targets but you only want some of these targets to be run automatically in CI, the list can be further filtered down by adding the ci_target list:

@ttfw_idf.idf_example_test(env_tag='Example_GENERIC', target=['esp32', 'esp32s2'], ci_target=['esp32'])
def test_app_xyz(env, extra_data):

(If no ci_target list is specified, all supported targets will be tested in CI.)

Test Apps local execution (ttfw)

All the following instructions are general. Part of them may be complemented by more particular instructions in the corresponding app's README.

Requirements

The following requirements need to be satisfied in the IDF python virtual environment.

ttfw needs to be in the PYTHONPATH. Add it like this: export PYTHONPATH=$PYTHONPATH:$IDF_PATH/tools/ci/python_packages
Install all requirements from tools/ci/python_packages/ttfw_idf/requirements.txt: pip install -r $IDF_PATH/tools/ci/python_packages/ttfw_idf/requirements.txt

You should also set the port via the environment variable ESPPORT to prevent the tools from looking and iterating over all serial ports. The latter causes much trouble, currently:

export ESPPORT=/dev/ttyUSB<X>

Test Apps local execution (pytest)

Some of the examples have pytest_....py scripts that are using the pytest as the test framework. For detailed information, please refer to the "Run the Tests Locally" Section under ESP-IDF tests in Pytest documentation

Using pytest is the recommended way to write new tests. We will migrate all the test apps scripts to this new framework soon.

Execution

Create an sdkconfig file from the relevant sdkconfig.ci.<CONFIG> and sdkconfig.defaults: cat sdkconfig.defaults sdkconfig.ci.<CONFIG> > sdkconfig
Run idf.py menuconfig to configure local project attributes
Run idf.py build to build the test app
Run python app_test.py to run the test locally