Commit Graph

136 Commits

Author SHA1 Message Date
Aleksei Apaseev
63bd3a18ad ci: add failed jobs report generator. Improve Target Test Report
Introduced changes:
- refactor the cli script used for report generation
- introduce failed jobs report generator
- cover job report generation with tests
- add job failure rate
- add test cases failure rate
- add current branch / other branches statistic for failed jobs / testcases
2024-06-20 14:40:54 +08:00
Aleksei Apaseev
40c8a16ad4 ci: add known failure cases to the target test report
target test report:
- refactor the methods to ensure reusability
- cover report generation with tests
- introduce a known failure tests block in report
- add grafana link to testcases dashboard
2024-05-28 15:36:22 +08:00
Fu Hanxi
61559dc3e4 ci: ignore collect import error on CI with env var 2024-05-24 16:29:39 +02:00
Sudeep Mohanty
50460d3483 feat(ci): Added CI host test for the linux target on MacOS runners
This commit adds a CI host test for the linux target to build and run
MacOS runners. Additionally, it adds a MacOS test for the hellow_world
app.
2024-05-24 16:29:22 +02:00
Fu Hanxi
d3abfa3d73 Merge branch 'ci/fix_pytest_embedded_log_dir' into 'master'
ci: move log dir from pytest_embedded_log to pytest-embedded

See merge request espressif/esp-idf!30747
2024-05-10 16:28:49 +08:00
Fu Hanxi
152db1a4ba
ci: move log dir from pytest_embedded_log to pytest-embedded 2024-05-09 12:23:04 +02:00
Frantisek Hrbata
9df59a2193 ci: add simple test for idf_size.py python compatibility
This adds a simple test that tries to run idf_size.py help and check
if the process does not exit with error. This is just to make sure
that idf_size.py can be used with minimum required python version.

Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
2024-05-08 13:38:19 +02:00
Roland Dobai
6f6706cb8e ci(tools): Fix IDF_MIRROR_PREFIX_MAP for including all tools from local 2024-04-26 15:17:39 +02:00
Anton Maklakov
e1ef214203 feat(tools): stop installation if tool is invalid
install/check commands - stop on error
export/list commands - print a warning
2024-04-12 14:03:49 +08:00
wanlei
535afdd7f4 feat(esp32c61): ci enable header check, fix c61 build 2024-03-27 19:39:59 +08:00
Fu Hanxi
b5318eee59
ci: move check_public_headers from pre_check stage to host_test
this job is the bottleneck of the pre_check stage
2024-03-26 13:18:41 +01:00
Fu Hanxi
b70d65444a
ci: always set expire_in and when with artifacts
set it under `default` does not work due to gitlab bugs.
will use `default` back once the bugs are resolved.
2024-02-16 13:43:53 +01:00
igor.udot
f5972ab592 ci: change known_failure_cases workflow 2024-01-21 15:36:25 +08:00
Fu Hanxi
fba96d58c2
ci: dynamic pipeline
build:
- upgrade idf-build-apps to 2.x
- unify get_pytest_apps and get_cmake_apps to get_all_apps
	- returns (test_apps, non_test_apps) tuple
- add tests for the new get_all_apps

assign:
- generate build report
- generate target test pipeline based on the build report

target test:
- download artifacts from minio server
- users can use `pytest --pipeline-id xxxxx` to download and flash
the binaries from the artifacts

.post:
- generate target test reports
2024-01-10 15:37:34 +01:00
Fu Hanxi
b709c880dd
ci: add linter for gitlab yaml files
- remove duplicated artifacts default values
- migrate check_artifacts_expire_time.py
- migrate check_rules_yml.py
2024-01-10 15:29:42 +01:00
Marius Vikhammer
852fbce397 ci(qemu): disable WDT when running qemu tests
WDTs are flakey in QEMU, disable while testing for now to
stop spurious test failures.
2023-12-13 18:47:26 +08:00
Frantisek Hrbata
5df4decf07 feat: use esp-idf-sbom pre-commit plugin
Currently sbom manifest is checked only in .gitmodules and
this check is done in pre-commit and also in CI. Meaning it's running
three times(pre-commit before push if user has it enabled, in CI
as there is the pre-commit run again and again with test in CI). Since
esp-idf-sbom contains a full manifest validation support and pre-commit
plugin for it, let's use it. This removes all the current sbom testing
and replaces it with a signle pre-commit plugin which validates all
manifests files(sbom.yml, idf_component.yml, .gitmodules and also
referenced manifests) in repository. Note that this checks all
manifests, not only ones which were modified. The check is reasonably
fast though, so it should not cause any problem. The reason for
validating all manifest files is that we want to make sure that the sbom
information in .gitmodules is updated too and that the hash
recorded in .gitmodules is up-to-date. Meaning submodule update
would not trigger this plugin, because no manifest was changed.

Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
2023-12-06 10:08:54 +01:00
Ivan Grokhotkov
178b27ed80
ci(qemu): use qemu from tools.json, don't require a special image 2023-12-03 07:45:19 +01:00
Ivan Grokhotkov
71153c7dbb
feat(tools): add 'idf.py qemu' extension
This extension allows running programs in QEMU similar to running
them on a real chip:

- 'idf.py qemu' — builds and runs the program in QEMU. User gets
  a QEMU instance launched, and can work with it as a normal QEMU
  instance.
- 'idf.py qemu monitor' — same, but QEMU is launched in the
  background, and idf_monitor runs in the foreground, showing QEMU
  output. Compared to only running 'idf.py qemu' this enables, for
  example, automatic backtrace decoding.
- 'idf.py qemu gdb' — launches QEMU in the background and opens an
  interactive GDB prompt, connecting it to QEMU.
- 'idf.py qemu --gdb monitor' and 'idf.py gdb' in another shell:
  launches QEMU in the background, keeps it suspended until GDB is
  connected, and opens idf_monitor. GDB can be used in another shell
  to debug the application.
2023-12-03 07:45:19 +01:00
Roland Dobai
bc1d74bb88 fix(ci): Remove unittest-xml-reporting dependency from host tests 2023-11-30 14:03:41 +01:00
Roland Dobai
cad04baba1 feat(ci): Merge tool test jobs and produce jUnit reports 2023-11-27 09:11:35 +01:00
Fu Hanxi
a6ccc2e18e
ci: optimize git strategy 2023-11-14 15:31:30 +01:00
igor udot (horw)
2dcb6d1fbb ci: added build log artifacts for linux 2023-11-06 14:34:33 +08:00
Fu Hanxi
a3a8c2acb9
ci: move default variables, before(after)_script into a single file
could help reuse the default settings on main pipeline and child
pipeline

renamed:
- .before_script_minimal -> .before_script:minimal
- .before_script_macos -> .before_script:build:macos
- .before_script_build_jobs -> .before_script:build
2023-10-13 12:55:51 +02:00
Peter Dragun
95b764970e Merge branch 'feat/uf2-to-esptool' into 'master'
feat(tools): move uf2 file generation to esptool

Closes IDF-7434

See merge request espressif/esp-idf!25655
2023-10-12 16:03:16 +08:00
Anton Maklakov
02802ea20a feat(tools): Add QEMU 8.0.0_20230522 to tools.json
Process wildcards in the install and download lists of idf_tools
    Fix the install and download handlers to get common behaviour
2023-10-10 09:59:38 +07:00
Peter Dragun
9acf412a07 feat(tools): move uf2 generation to esptool 2023-10-09 15:38:30 +02:00
Fu Hanxi
5e4427ab13
ci(pytest): add plugin pytest-ignore-test-results
pass with warning if test job got ignored.
2023-08-18 10:50:18 +08:00
Fu Hanxi
f5984d482a
ci: fix skipping qemu test always 2023-07-19 12:02:59 +08:00
Marius Vikhammer
91aaf8b1a0 ci(qemu): temporarily allow qemu tests to fail 2023-07-13 11:16:38 +08:00
Fu Hanxi
537bd3efec
ci: enable check app dependency for qemu and linux tests 2023-07-11 15:32:57 +08:00
Zim Kalinowski
8fe961b3ff console: enable qemu tests 2023-06-20 09:37:39 +02:00
Frantisek Hrbata
925bc17cd0 tools: add sbom information for submodules
This adds SBOM information for submodules, which are not managed
by Espressif. Meaning there is no fork for them in the espressif
namespace. Other submodules should add sbom.yml manifest file to
the root of their git repository.

The SBOM information for submodules is stored in the .gitmodules file.
Each SBOM related variable has the "sbom-" prefix and the following
variables may be used:

sbom-version:
   submodule version

sbom-cpe:
   CPE record if available in NVD. This will be used by the SBOM
   tool to check for possible submodule vulnerabilities. The
   version in the CPE can be replaced with the "{}" placeholder,
   which will be replaced by the "sbom-version" value from above.

sbom-supplier:
   Person or organization who is providing the submodule.
   It has to start with "Person:" or "Organization:" prefix
   as required by the SPDX-2.2 standard.

sbom-url:
   URL to the project if exists, e.g. github.

sbom-description:
   Project description.

sbom-hash:
   Submodule SHA as recorded in the git-tree. This field is used by
   CI to check that the submodule checkout hash and info in .gitmodules
   are in sync. IOW if submodule is updated and it has SBOM info in
   .gitmodules, the .gitmodules has to be updated too. The test is
   part of this commit. The checkout has of the submodule can be found
   by using "git submodule status".

Example for micro-ecc submodule
---8<---
[submodule "components/bootloader/subproject/components/micro-ecc/micro-ecc"]
	path = components/bootloader/subproject/components/micro-ecc/micro-ecc
	url = ../../kmackay/micro-ecc.git
	sbom-version = 1.0
	sbom-cpe = cpe:2.3🅰️micro-ecc_project:micro-ecc:{}:*:*:*:*:*:*:*
	sbom-supplier = Person: Ken MacKay
	sbom-url = https://github.com/kmackay/micro-ecc
	sbom-description = A small and fast ECDH and ECDSA implementation for 8-bit, 32-bit, and 64-bit processors
	sbom-hash = d037ec89546fad14b5c4d5456c2e23a71e554966
---8<---

Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
2023-06-06 15:01:08 +02:00
Alexey Lapshin
6a54c25823 esp_gdbstub: add tests for riscv jump instructions decoding 2023-05-19 20:15:58 +08:00
Alexey Lapshin
0b078f5cd3 coredump: remove tests (moved to esp-coredump repo) 2023-05-11 12:13:52 +08:00
David Cermak
b2af4d9689 lwip/linux: Add lwip support for networking component under linux
linux/lwip: Wrap some IO posix functions
* to workaourd the FreeRTOS EINTR issue (when building without lwip)
* to correctly choose the sub-system based on fd (when building with
lwip) -- passing control to either linux/system or to lwip
This commit also addapts tapio-if to provide DHCP client by default and
configurable settings for static IP
2023-05-05 05:03:39 +00:00
Radek Tandler
d8f2c0e715 Merge branch 'feature/storage_host_test_ffs' into 'master'
fatfs: host test migrated to CMake and esp_partition emulation for linux

See merge request espressif/esp-idf!23265
2023-04-25 15:30:09 +08:00
radek.tandler
13a305656d host_test: cleanup of make based obsolete code in wl, fatfs and spi_flash 2023-04-21 10:45:17 +02:00
laokaiyao
1f84f6c6ed esp32h4: remove esp32h4 target from ci 2023-04-20 15:19:47 +08:00
radek.tandler
0402874d3c host_test: wl migrated to Cmake and linux emulation of esp_partition
- build system changed to CMake
 - host tests changed to use partition api on linux instead of mocked code
 - extended wl flash host tests to cover power off recovery code
2023-04-11 16:16:53 +02:00
Rocha Euripedes
98b75727ba Merge branch 'feature/socks_transport' into 'master'
tcp_transport: Adds SOCKS4 proxy transport

See merge request espressif/esp-idf!20479
2023-03-28 20:33:28 +08:00
Djordje Nedic
3e0f612a7e tools: Move out idf_size.py in favour of using the esp-idf-size package
This commit removes all idf_size.py files and references to them and adds esp-idf-size as a dependency and adequate wrappers to avoid breaking changes.
2023-03-27 19:40:33 +02:00
Euripedes Rocha
d6db90a3cd [tcp_transport] - Adds Socks4 proxy transport
- Basic implementation of Socks4 protocol.
- Introduce basic host testing.
2023-03-27 14:46:40 +02:00
David Cermak
924bac9956 lwip: Remove AFL based fuzzer tests
Also removed compilation pre-checks and jobs from the CI, and test data
2023-03-20 17:47:23 +00:00
KonstantinKondrashov
a0408f7324 efuse: Updates efuse table for esp32c6 2023-02-28 17:13:21 +08:00
KonstantinKondrashov
f3394c488a efuse: Add support for esp32h2 2023-02-21 08:18:14 +00:00
Fu Hanxi
6cda2550ea ci: ignore known failures in linux host test 2023-02-13 16:04:44 +08:00
Peter Dragun
3e3533f918 feat(idf_monitor): move idf_monitor to separate repo 2023-02-03 11:20:15 +01:00
David Cermak
fa97004faf lwip: Support for linux target
Implement linux port layer and reuse the original FreeRTOS layer
that's compiled and used on linux target as well, by means of FreeRTOS
simulator.
2023-01-31 08:43:45 +01:00
Fu Hanxi
649389cbb3 Merge branch 'fix/host_test_with_brew_runners' into 'master'
ci: use brew runners by default for host test jobs

See merge request espressif/esp-idf!22059
2023-01-29 16:12:26 +08:00