Extend target checks in cmake, in case it's run directly and not via
idf.py or if idf.py misses something. This may happen
for example if cmake variables are set in project's CMakeLists.txt.
Some clean-ups are included along with the new checks and tests.
1. __target_check() function is removed. IIUC it should never fail,
because the selected target is explicitly passed as environmental
variable to kconfgen. Meaning the IDF_TARGET from environment variable may
not be actually used in kconfgen if IDF_TARGET is already set it cmake cache.
Note that the IDF_TARGET environment variable used for kconfgen is not
based on the actual IDF_TARGET environment variable set for idf.py, but
rather on the value set in __target_init() with
set(IDF_TARGET ${env_idf_target} CACHE STRING "IDF Build Target")
My understanding is that the original check was introduced to handle
situation, where IDF_TARGET was already set in cmake's cache and
the IDF_TARGET from environment variable was different. Since
the kconfgen would use the original environment variable(not
explicitly passed as it is now) the IDF_TARGET in cmake and in
sdkconfig could differ. IOW I think the original check was introduced
to cope with the following cmake behaviour
set(VARIABLE "value1" CACHE STRING "test variable")
set(VARIABLE "value2" CACHE STRING "test variable")
message("Variable value: ${VARIABLE}")
output: Variable value: value1
2. I scratched by head how it is possible that the following code
in __target_check()
if(NOT ${IDF_TARGET} STREQUAL ${env_idf_target})
could fail if IDF_TARGET is not set. For example in clean project
IDF_TARGET=esp32 idf.py reconfigure
Here env_idf_target==esp32 and IDF_TARGET is not set, so I would
expect that cmake will fail with error message that the cache
and env target do not match. The thing is that the variable
evaluation is done before the if command, so it actually
sees this
if(NOT STREQUAL esp32)
which is false and the error is not printed. It can be seen
with 'cmake --trace-expand' command. I don't know if this
was used on purpose or it worked just as a coincidence, but
I find it very confusing, so I added explicit check if the
IDF_TARGET is defined before the actual check. Same for
CMAKE_TOOLCHAIN_FILE.
3. Error messages are not formated(line-wrapped) by cmake's markup
so it's easier to check the output in tests.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The _guess_or_check_idf_target() function has sdkconfig and sdkconfig.defaults
file names hardcoded. Since config file names may be specified with SDKCONFIG
or SDKCONFIG_DEFAULTS cmake vars, directly in CMakeLists.txt or passed in with
the -D cmake option, they are not respected.
Problem is when SDKCONFIG or SDKCONFIG_DEFAULTS is set in
CMakeLists.txt. While idf can detect cmake vars passed through it
to cmake via the -D option, detecting SDKCONFIG and SDKCONFIG_DEFAULTS
vars settings in CMakeLists.txt would require to parse it. This seems
like error prone approach. Also if the vars defined by the -D option
are passed directly to cmake, not via idf, they will not be visible to idf.
It seems reasonable to move the logic into cmake, where we know the correct
SDKCONFIG and SDKCONFIG_DEFAULTS values. So the IDF_TARGET detection/guessing
is moved into targets.cmake, where the IDF_TARGET is actually set. The target
is guessed based on the following precendence.
1) $ENV{IDF_TARGET}
2) IDF_TARGET
3) SDKCONFIG
4) sdkconfig
5) SDKCONFIG_DEFAULTS if non-empty or
$ENV{SDKCONFIG_DEFAULTS} if non-empty or
sdkconfig.defaults
6) esp32
All config files referred in $ENV{SDKCONFIG_DEFAULTS} and SDKCONFIG_DEFAULTS
are searched, compared to the current behaviour. First target found in the
above chain is used.
The original _guess_or_check_idf_target() is renamed to _check_idf_target() and
used for the target consistency checks only.
The get_sdkconfig_filename() helper is now used to get the sdkconfig file
for consistency checks. It looks in SDKCONFIG specified with the -D
option and project_description.json.
With this change config full paths are reported in messages, so it's clear
e.g. from which config the target was guessed from or which config has
consistency problem. test_non_default_target.py was adjusted to this
change and also new test for testing the IDF_TARGET guessing was added.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Checking that the host is macOS is not sufficient here, since the
linker is still a GNU linker when cross-compiling for a chip.
Instead, use the linker_type variable introduced in the previous
commit.
When compiling for a chip target with Clang,
CMAKE_C_COMPILER_ID="Clang" but the linker is still a GNU linker.
Therefore we can still generate the map file.
Host tests of nvs_flash eligible to run in Linux implementation of nvs flash were migrated. Remaining test cases
were left in original folder. Migrated test cases use CMake instead of make.
This commit removes all kconfig_new files and references to them and adds esp-idf-kconfig as a dependency and adequate wrappers to avoid breaking changes.
This commit ads the corresponding --output-file option from idf_size.py to idf.py.
This is needed because piping the idf.py size output to a file would result in CMake output being present in the file as well.
Unlike COMPILE_OPTIONS, COMPILE_DEFINITIONS CMake property assumes
values without the -D prefix, such as NAME or NAME=VAL.
Previously, IDF build system was passing COMPILE_DEFINITIONS build
property to CMake COMPILE_OPTIONS property, so -D prefix was not
a problem.
Now that COMPILE_DEFINITIONS CMake property is used, -D prefix has
to be removed.
(Note that this doesn't affect 'target_compile_definitions' function,
which strips -D prefix before adding the definition to the property.)
Now that the supported CMake version is >=3.16, this code can be
simplified.
The code to deduplicate the directories can be removed since this is
handled by target_link_directories.
Related to https://github.com/espressif/esp-idf/issues/4162
Setting this option informs CMake that it should pass
-fcolor-diagnostics flag to the compiler.
(Colorized build system output, like from GNU Make, is produced even
without this flag.)
Note that if the build is done using Ninja and the build output is
redirected (not a TTY), Ninja will still strip the escape codes from
the output. For the case of idf.py, this is handled in the next
commit.
Related to https://github.com/espressif/esp-idf/issues/4162
Setting this option informs CMake that it should pass
-fcolor-diagnostics flag to the compiler.
(Colorized build system output, like from GNU Make, is produced even
without this flag.)
Note that if the build is done using Ninja and the build output is
redirected (not a TTY), Ninja will still strip the escape codes from
the output. For the case of idf.py, this is handled in the next
commit.
This adds CSV support to idf_size.py and idf.py size actions and using the --format argument which accepts 'text', 'json' or 'csv' as input.
idf_size.py --json argument is deprecated but left to avoid a breaking change.
For idf.py size actions OUTPUT_JSON environment variable set at configuration time is overriden at target build time if --format is used.
Additionally, this commit refactors big parts of code, unified usage of json_dict and manually generated dictionaries for textual output and improves code quality in many parts.
'linux' component provides some of the common header files, such as
'sys/queue.h' and 'sys/lock.h'. For chip targets, it is possible to
include these files without having to add any extra requirements.
With this change, the same behavior will apply for the linux target.
`__COMPONENT_TARGETS` is evaluated very early when components and
component directories are added to the build, which means that all
components (including the ones which are in EXCLUDE_COMPONENTS) have
a build system target defined. The component manager was given the
list of all known components (derived from the list of targets), not
the list of components after EXCLUDE_COMPONENTS were processed.
Because of that, EXCLUDE_COMPONENTS didn't effectively exclude the
component from the consideration of the component manager.
This updates the minimal supported version of CMake to 3.16, which in turn enables us to use more CMake features and have a cleaner build system.
This is the version that provides most new features and also the one we use in our latest docker image for CI.
After refactoring the target components (e.g. esp32) no longer contained any real functionality.
What remained in these components have been moved elsewhere and the component itself deleted from the
build system.
lwip was added to common requirements list to provide "sys/socket.h"
header to all components without additional requirements specified.
However, lwip pulls in a lot of dependencies on other components.
This commit removes lwip from common requirements to reduce the number
of components in G1-only apps.
To compensate for this removal, the following changes are made:
- newlib (which is a common requirement) has a public dependency on
lwip if lwip is present in the build. This ensures that sys/socket.h
is available as long as lwip component is included into the build.
- lwip is now a public requirement of esp-tls since esp_tls.h includes
sys/socket.h header.
- lwip is now a public requirement o esp_http_client because
sys/socket.h is included from esp_http_client.h
- lwip is now a private requirement of esp_wifi for "smartconfig_ack"
- lwip is now a private requirement of mqtt for socket functions
- lwip is now a public requirement of tcp_transport because
esp_transport_tcp.h includes sys/socket.h header.
- mbedtls checks if lwip component is present in the build. If yes,
net_sockets.c is added to the build, along with the dependency on
lwip. Previously lwip was a public requirement of mbedtls
unconditionally.
system/g1_components test app is updated to reflect the changes
Default public dependencies of a component before and after this
change, except common requirements:
- esp_timer (public dependency of freertos)
- bootloader_support (public dependency of esp_hw_support)
- vfs (public dependency of lwip)
- esp_wifi (public dependency of lwip)
- esp_event (public dependency of esp_wifi)
- esp_netif (public dependency of esp_event)
- esp_eth (public dependency of esp_netif)
- esp_phy (public dependency of esp_wifi)
After:
- esp_timer (public dependency of freertos)
- bootloader_support (public dependency of esp_hw_support)
Altogether, the following components have been always added as
public requirements to all other components, and are not added now
([breaking-change]):
- lwip
- vfs
- esp_wifi
- esp_event
- esp_netif
- esp_eth
- esp_phy
Application components now need to explicitly declare dependencies on
these components.
This component property allows including all component object files
into the executable. It is equivalent to wrapping the component
library with -Wl,--whole-archive and -Wl,--no-whole-archive flags.
Closes https://github.com/espressif/esp-idf/issues/8667
When building for "linux" (~POSIX) target on macOS, the system linker
is normally used. MacOS linker doesn't recognise --gc-sections, but
has a -dead_strip flag which is equivalent.
Tools: Add "idf.py save-defconfig" command to generate sdkconfig.defaults based on current sdkconfig
Closes IDF-2970
See merge request espressif/esp-idf!16409
COMPONENT_DIRS and EXTRA_COMPONENT_DIRS should be defined as CMake
lists, using 'set' or 'list' commands. Some applications written
for earlier versions of ESP-IDF used to define these variables as
space separated strings.
For example, the following is correct:
set(EXTRA_COMPONENT_DIRS path/to/components path/to/more/components)
The following is not correct:
set(EXTRA_COMPONENT_DIRS "${EXTRA_COMPONENT_DIRS} component1")
set(EXTRA_COMPONENT_DIRS "${EXTRA_COMPONENT_DIRS} component2")
The string "component1 component2" may indicate a single directory
name with a space, or two directory names separated by space.
However due to the fact that such way of defining EXTRA_COMPONENT_DIRS
was supported in IDF 4.3 and earlier, we need to provide backward
compatibility for it.
This commit introduces a new script, split_paths_by_spaces.py, which
is invoked if EXTRA_COMPONENT_DIRS or COMPONENT_DIRS variable contains
spaces. The script tries to determine if each space should be
interpreted as a separator or as part of the directory name.
When this cannot be done unambiguously, the script reports an error.
In all cases when space separators are detected, the script reports
a warning, and prints instructions for fixing the CMakeLists.txt.
Breaking change in this commit: specifying non-existent directories
in COMPONENT_DIRS or EXTRA_COMPONENT_DIRS is no longer allowed.
New —-list-separator argument of confgen.py and
prepare_kconfig_files.py is used to select which character is used
as list separator. For compatibility with esp-docs, we still keep
support for space separator. Otherwise esp-docs would have to choose
the separator depending on the IDF version.
It can be used in component CMakeLists.txt files when adding "weak"
dependencies between component. A "weak" dependency from A to B is
similar to target_link_libraries(A PRIVATE B), but it gets added only
if B is included in the build.
Use it instead of specifying B as part of PRIV_REQUIRES when the
component can be built (even if with reduced functionality) without
component B being available.
