Commit Graph

60 Commits

Author SHA1 Message Date
Marek Fiala
364f777f96 feat(tools): Allow test_spaces_bundle4 to fail on Win 2024-08-21 14:37:06 +02:00
Marek Fiala
e95aff7ca9 feat(tools): sanitize all fixtures using test names
The parametrization include `[` and `]` in names. This is not
possible to use within path. Expand the sanitization into all
fixtures using request.node.name.
2024-08-21 14:37:06 +02:00
Marek Fiala
ccbb415be8 feat(tools): add pytest mark idf_copy_with_space 2024-08-21 14:37:05 +02:00
Marek Fiala
f0e8966ca5 feat(tools): Use custom directory in failing pytest 2024-08-21 14:37:05 +02:00
Ivan Grokhotkov
5ef75d5073 Merge branch 'bugfix/reproducible_builds_improvements' into 'master'
build system: reproducible build improvements

Closes IDFGH-12690

See merge request espressif/esp-idf!32734
2024-08-16 19:02:40 +08:00
Ivan Grokhotkov
cbc52e7d30
test(build_system): check reproducible builds with spaces in paths 2024-08-15 16:41:32 +02:00
Ivan Grokhotkov
bef4cb05ff
fix(build_system): fix toolchain sysroot directory affecting builds
Closes https://github.com/espressif/esp-idf/issues/13680
2024-08-15 16:41:31 +02:00
Ivan Grokhotkov
c521fcb6fd
fix(cmake): report correct error on unknown component name
Previously the error message was

CMake Error at /home/user/esp-idf/tools/cmake/build.cmake:296 (__component_get_property):
  __component_get_property Function invoked with incorrect arguments for
  function named: __component_get_property
Call Stack (most recent call first):
  /home/user/esp-idf/tools/cmake/build.cmake:341 (__build_resolve_and_add_req)
  /home/user/esp-idf/tools/cmake/build.cmake:638 (__build_expand_requirements)
  /home/user/esp-idf/tools/cmake/project.cmake:710 (idf_build_process)
  CMakeLists.txt:6 (project)

Now it will be:

CMake Error at /home/user/esp-idf/tools/cmake/build.cmake:298 (message):
  Failed to resolve component 'whatever' required by component 'main'.
Call Stack (most recent call first):
  /home/user/esp-idf/tools/cmake/build.cmake:345 (__build_resolve_and_add_req)
  /home/user/esp-idf/tools/cmake/build.cmake:642 (__build_expand_requirements)
  /home/user/esp-idf/tools/cmake/project.cmake:710 (idf_build_process)
  CMakeLists.txt:6 (project)

Also improved the hint to show the component name in quotes.
2024-08-12 17:54:44 +02:00
Ivan Grokhotkov
fc4c445643
test(build_system): move reproducible build test to pytest 2024-08-11 20:24:03 +02:00
Ivan Grokhotkov
d6edcba3b6
test(build_system): extract file helpers, add 'bin_files_differ'
- Move file-related functions bin_file_contains and file_contains
  from idf_utils.py and existing functions from editing.py into a new
  file file_utils.py
- Add a function 'bin_files_differ' to compare binary files
2024-08-11 20:23:27 +02:00
gojimmypi
f6a852e1cf fix(cmake): check for missing component_target
Closes https://github.com/espressif/esp-idf/pull/14036

[Frantisek Hrbata: fixed spelling not related to this change so spellcheck succeeds]
[Frantisek Hrbata: modified the error message to be recognized by the existing hint]
[Frantisek Hrbata: added also check in idf_component_set_property and simple tests]
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
2024-07-11 18:33:41 +02:00
Jan Beran
950428c2f4 feat(tools): Add often used espefuse subcommands to idf.py 2024-06-07 16:23:17 +02:00
Frantisek Hrbata
fbe8bf89ee fix: properly resolve component dependencies
There is currently a bug in the __build_resolve_and_add_req function in
tools/cmake/build.cmake where the check for registered component
requirements is incorrectly applied to the component itself rather than
its dependencies. This issue likely originated from a typo, using
component_target instead of _component_target. To prevent further
confusion, _component_target has been renamed to _req_target.

Fixing this revealed multiple incorrect dependencies for the Linux
target, which have now been resolved by explicitly specifying the
dependencies for the Linux target.

Closes https://github.com/espressif/esp-idf/issues/13447

Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
2024-05-30 22:01:55 +08:00
Marek Fiala
e351a4407a fix(tools): fix failing create readonly project build system test
Changed the way of removing copied esp-idf folder.
2024-04-30 16:42:05 +02:00
Roland Dobai
af302c0bee Merge branch 'feat/add_merged_bin_cmd' into 'master'
feat(tools): Add idf.py merge-bin command and cmake target

See merge request espressif/esp-idf!29996
2024-04-22 17:55:36 +08:00
Fu Hanxi
2cde75b531
ci: improve pytest build system tests
- remove temp dirs
- remove idf-component-manager unit test
2024-04-16 15:33:43 +02:00
Fu Hanxi
a4f691b88a
ci: always cleanup idf copy in ci 2024-04-16 15:32:10 +02:00
Jan Beran
0dec6fe65d feat(tools): Add idf.py merge-bin command and cmake target 2024-04-16 12:49:18 +02:00
Marek Fiala
f2b75d81b3 fix(system): fix idf.py build ulp examples containing spaces in path on Win 2024-04-09 14:00:34 +02:00
Fu Hanxi
74843a442b fix: component manager load all component dirs even set(COMPONENTS ...) 2024-03-19 19:52:20 +08:00
Fu Hanxi
d2542dd5c5 test: improve test_dependency_lock test case 2024-03-19 19:52:20 +08:00
Marek Fiala
52da5b765f feat(tools): Run skipped build system pytests on Win 2024-02-08 11:23:51 +01:00
Harshit Malpani
bbbede07d0
fix(partition_table): Check partition size for type APP
The size of partition of type APP should be multiple of 4 KB. Partition
generation tool now make this as a mandatory requirement. This is
minimum flash erase size. If the size of the APP type partition is not
aligned to 4 KB then the last erase operation could go beyond the allocated
partition and hence may fail. This issue would only be observed when the
firmware size grows very close to the allocated partition size, and hence
causing the OTA update to fail.

For already deployed devices on-field with the size of APP partition not
aligned to flash sector boundary, it is best to ensure that firmware
size always remains within the lower 4 KB boundary of the total
allocated space. While migrating to ESP-IDF 5.3 release, partition table
for an existing project can be adjusted accordingly for the build to
succeed.

Found during discussion in https://github.com/espressif/esp-idf/pull/12460
2023-12-11 10:14:57 +05:30
kohait00
9beda4ce48 feat(tools/cmake): Added VERSION argument to the project() macro in cmake
This commit enables the standad VERSION argument for the project() macro
in ESP-IDF. The VERSION argument is compilant with the requirements of
cmake 3.16. This commit also adds new test cases for verifying the
validity of the version argument.

Merges https://github.com/espressif/esp-idf/pull/12461

Co-authored-by: Sudeep Mohanty <sudeep.mohanty@espressif.com>
2023-12-05 11:29:43 +01:00
Marek Fiala
04ade501aa feat(tools): Disable failing build system tests on Windows runner 2023-11-29 23:10:38 +01:00
Marek Fiala
b535ec9a99 feat(tools): Fix some failing tests on Windows runner 2023-11-29 23:10:38 +01:00
Frantisek Hrbata
ea84bd51c9 feat(ci): add test for custom cmake CMAKE_EXECUTABLE_SUFFIX
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
2023-11-16 08:14:19 +01:00
Peter Dragun
9acf412a07 feat(tools): move uf2 generation to esptool 2023-10-09 15:38:30 +02:00
Marek Fiala
b39e541652 feat(tools): Do not run test_create_project_with_idf_readonly on macOS
It corrupts environment on macOS runner because it uses shell, not docker-executor,
and makes all tree read-only for a non-root user.

+ Replace examples with real files in example README.md
2023-08-31 12:54:19 +02:00
Marek Fiala
7d2d8b7839 refactor(tools): Remove test_build_system_cmake.sh & test_build_system_spaces.py 2023-08-31 12:54:19 +02:00
Marek Fiala
7943e01850 refactor(tools): Rewrite build system unit tests to python
Rewritten tests: test_build_system_spaces.py
2023-08-08 16:41:30 +02:00
Omar Chebib
7b5c0fb96b ci(build_system): add a test case for bootloader override 2023-07-28 15:04:21 +08:00
Roland Dobai
79b15bd410 Merge branch 'feature/rewrite_build_sys_tests_v6' into 'master'
Tools: Rewrite build system unit tests to python - git

Closes IDF-7386

See merge request espressif/esp-idf!23860
2023-07-19 01:20:06 +08:00
Marek Fiala
f4e3c9c03a refactor(tools): Rewrite build system unit tests to python
Rewritten parts:
 - git
 - version
 - build
2023-07-18 15:51:04 +02:00
Marek Fiala
e8ee59f68b fix(tools): create-project with proper file permission - pre-commit fix
Closes https://github.com/espressif/esp-idf/pull/11836
2023-07-18 14:08:37 +02:00
Ryan Yin
354e2c4673 feat: fix: create-project & create_component with proper file permission 2023-07-17 19:32:30 +08:00
Peter Dragun
d4428ca1db feat(tools/kconfig): add initial and current versions of IDF to kconfig 2023-07-10 17:01:51 +02:00
Marek Fiala
5033637899 Tools: Rewrite build system unit tests to python - partitions, components, docs 2023-06-14 12:17:53 +00:00
Frantisek Hrbata
7819965039 tools: do not disable pytest-embedded
As pointed out by Fu Hanxi, the pytest_build_system job is currently
using --parallel-index and --parallel-count, which are provided by
pytest-embedded, so we should not disable it. Moreover to properly
disable pytest-embedded we should use "no:pytest_embedded". Meaning
this probably was not working as indented anyway.

Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
2023-06-06 15:01:08 +02:00
Marek Fiala
fb0dc46183 Tools: Rewrite build system unit tests to python - sdkconfig, bootloader, components 2023-05-23 09:47:39 +00:00
laokaiyao
1f84f6c6ed esp32h4: remove esp32h4 target from ci 2023-04-20 15:19:47 +08:00
Fu Hanxi
0f2a0058a3 ci: fix pytest build system 2023-04-07 10:42:12 +08:00
Roland Dobai
7d4835e817 Merge branch 'feature/rewrite_build_sys_tests_v3' into 'master'
Rewrite build system unit tests to python v3

Closes IDF-6842

See merge request espressif/esp-idf!22280
2023-04-06 15:21:32 +08:00
Roland Dobai
b3a54cf643 Merge branch 'bugfix/sdkconfig_abspath' into 'master'
ci: add build test for sdkconfig specified with abspath

Closes IDF-764

See merge request espressif/esp-idf!23020
2023-04-03 16:33:47 +08:00
Frantisek Hrbata
f6af455f57 ci: add build test for sdkconfig specified with abspath
On Windows, when path is specified as absolute for cmdl argument, cmake can
interpret parts of the path as invalid escape chars. For example "C:\Users\..."
will result in "Invalid character escape '\U'." Externally specified
paths should be converted into cmake's representation, which uses '/'.
This can be done e.g. by using 'get_filename_component()'. Currently
there doesn't seem to be any problem with this, but let's add a test for
this.

Suggested-by: Ivan Grokhotkov <ivan@espressif.com>
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
2023-03-31 09:55:25 +02:00
Marek Fiala
b4446e1748 Tools: Rewrite build system unit tests to python - cmake libraries and Kconfig 2023-03-28 11:20:42 +02:00
Frantisek Hrbata
9b731e5e35 tools: fix set-target if IDF_TARGET env is set and stale sdkconfig exists
Currently if the IDF_TARGET env is set, and old sdkconfig exists with
different target value in CONFIG_IDF_TARGET, the set-target action fails
complaining about the IDF_TARGET env and value in sdkconfig being different.
We should ignore IDF_TARGET value from sdkconfig, because we are
actually setting new target and the old sdkconfig is renamed in cmake.

This can be easily reproduced with

---8<---
$ IDF_TARGET=esp32 idf.py set-target esp32
$ IDF_TARGET=esp32s3 idf.py set-target esp32s3

Project sdkconfig '/home/fhrbata/work/hello_world/sdkconfig' was generated
for target 'esp32s3', but environment variable IDF_TARGET is set to 'esp32'.
Run 'idf.py set-target esp32' to generate new sdkconfig file for target esp32.
---8<---

This also adds test for this use case to test_non_default_target.py.

Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
2023-03-28 11:18:04 +02:00
Frantisek Hrbata
1ca9e63e79 tools: add target consistency checks to cmake
Extend target checks in cmake, in case it's run directly and not via
idf.py or if idf.py misses something. This may happen
for example if cmake variables are set in project's CMakeLists.txt.

Some clean-ups are included along with the new checks and tests.

1. __target_check() function is removed. IIUC it should never fail,
   because the selected target is explicitly passed as environmental
   variable to kconfgen. Meaning the IDF_TARGET from environment variable may
   not be actually used in kconfgen if IDF_TARGET is already set it cmake cache.
   Note that the IDF_TARGET environment variable used for kconfgen is not
   based on the actual IDF_TARGET environment variable set for idf.py, but
   rather on the value set in __target_init() with

	set(IDF_TARGET ${env_idf_target} CACHE STRING "IDF Build Target")

   My understanding is that the original check was introduced to handle
   situation, where IDF_TARGET was already set in cmake's cache and
   the IDF_TARGET from environment variable was different. Since
   the kconfgen would use the original environment variable(not
   explicitly passed as it is now) the IDF_TARGET in cmake and in
   sdkconfig could differ. IOW I think the original check was introduced
   to cope with the following cmake behaviour

	set(VARIABLE "value1" CACHE STRING "test variable")
	set(VARIABLE "value2" CACHE STRING "test variable")
	message("Variable value: ${VARIABLE}")
	output: Variable value: value1

2. I scratched by head how it is possible that the following code
   in __target_check()

   	if(NOT ${IDF_TARGET} STREQUAL ${env_idf_target})

   could fail if IDF_TARGET is not set. For example in clean project

	IDF_TARGET=esp32 idf.py reconfigure

   Here env_idf_target==esp32 and IDF_TARGET is not set, so I would
   expect that cmake will fail with error message that the cache
   and env target do not match. The thing is that the variable
   evaluation is done before the if command, so it actually
   sees this

   	if(NOT  STREQUAL esp32)

   which is false and the error is not printed. It can be seen
   with 'cmake --trace-expand' command. I don't know if this
   was used on purpose or it worked just as a coincidence, but
   I find it very confusing, so I added explicit check if the
   IDF_TARGET is defined before the actual check. Same for
   CMAKE_TOOLCHAIN_FILE.

3. Error messages are not formated(line-wrapped) by cmake's markup
   so it's easier to check the output in tests.

Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
2023-03-08 16:45:47 +01:00
Frantisek Hrbata
8e912faad1 tools: add target consistency checks for target specified with -D option
Extend existing target consistency checks for the two following cases.
1. Target does not match currently used toolchain
   $ IDF_TARGET=esp32s2 idf.py reconfigure
   $ idf.py -DIDF_TARGET=esp32c3 build

2. Target is ambiguous, because it's specified also as env. var.
   IDF_TARGET=esp32s3 idf.py set-target esp32c2

Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
2023-03-08 16:43:42 +01:00
Frantisek Hrbata
0d859f2786 tools: move target guessing into cmake
The _guess_or_check_idf_target() function has sdkconfig and sdkconfig.defaults
file names hardcoded. Since config file names may be specified with SDKCONFIG
or SDKCONFIG_DEFAULTS cmake vars, directly in CMakeLists.txt or passed in with
the -D cmake option, they are not respected.

Problem is when SDKCONFIG or SDKCONFIG_DEFAULTS is set in
CMakeLists.txt. While idf can detect cmake vars passed through it
to cmake via the -D option, detecting SDKCONFIG and SDKCONFIG_DEFAULTS
vars settings in CMakeLists.txt would require to parse it. This seems
like error prone approach. Also if the vars defined by the -D option
are passed directly to cmake, not via idf, they will not be visible to idf.

It seems reasonable to move the logic into cmake, where we know the correct
SDKCONFIG and SDKCONFIG_DEFAULTS values. So the IDF_TARGET detection/guessing
is moved into targets.cmake, where the IDF_TARGET is actually set. The target
is guessed based on the following precendence.

1) $ENV{IDF_TARGET}
2) IDF_TARGET
3) SDKCONFIG
4) sdkconfig
5) SDKCONFIG_DEFAULTS if non-empty or
   $ENV{SDKCONFIG_DEFAULTS} if non-empty or
   sdkconfig.defaults
6) esp32

All config files referred in $ENV{SDKCONFIG_DEFAULTS} and SDKCONFIG_DEFAULTS
are searched, compared to the current behaviour. First target found in the
above chain is used.

The original _guess_or_check_idf_target() is renamed to _check_idf_target() and
used for the target consistency checks only.

The get_sdkconfig_filename() helper is now used to get the sdkconfig file
for consistency checks. It looks in SDKCONFIG specified with the -D
option and project_description.json.

With this change config full paths are reported in messages, so it's clear
e.g. from which config the target was guessed from or which config has
consistency problem. test_non_default_target.py was adjusted to this
change and also new test for testing the IDF_TARGET guessing was added.

Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
2023-03-08 16:37:58 +01:00