Other changes:
* Command line argument name and descriptions updated and formatted
* Some exception messages updated for clarity
* READMEs updated for tool and all provisioning examples
* Minor update in example test scripts due to change in esp_prov.get_transport() API
* Transport_HTTP now forces connect on initialization
use `subprocess.Popen` when catch TypeError:
```
Traceback (most recent call last):
File "tools/idf_tools.py", line 1249, in <module>
main(sys.argv[1:])
File "tools/idf_tools.py", line 1245, in main
action_func(args)
File "tools/idf_tools.py", line 1038, in action_install
tool_obj.find_installed_versions()
File "tools/idf_tools.py", line 468, in find_installed_versions
ver_str = self.check_version()
File "tools/idf_tools.py", line 426, in check_version
version_cmd_result = run_cmd_check_output(cmd, None, extra_paths)
File "tools/idf_tools.py", line 176, in run_cmd_check_output
result = subprocess.run(cmd, capture_output=True, check=True, input=input_text)
File "/opt/pyenv/pyenv-1.2.6/versions/3.5.5/lib/python3.5/subprocess.py", line 383, in run
with Popen(*popenargs, **kwargs) as process:
TypeError: __init__() got an unexpected keyword argument 'capture_output'
```
List of changes:
* New component esp_local_ctrl added
* Example added under examples/protocols/esp_local_ctrl
* Documentation added under protocols/esp_local_ctrl
* Demo client side app esp_local_ctrl.py added under examples/protocols/esp_local_ctrl/scripts
* protocomm_ble : protocomm_ble_config_t given struct name for allowing forward declaration
* esp_prov/transport_softap renamed to transport_http
* transport_http module supports verification of server certificate
* transport_http module performs name resolution before connection
1. revision defined in bot message
2. branch name (or tag name) of current IDF
3. CI_MERGE_REQUEST_TARGET_BRANCH_NAME
4. branch name parsed from `git describe`
5. default branch
- Makefile didn't quite work with out-of-tree build unless there was already an in-tree build done.
- CMake needs to delete some of the in-tree build artifacts or they're used instead of the correct
files.
Previously, wordexp() was used. However for providing Windows builds
of mconf-idf we can't use wordexp() so we use this simplified
environment variable expansion code instead.
The reasoning here is to make the behaviour consistent across Windows
(CMake vs GNU Make), Linux & macOS mconf.
List of changes in components/wifi_provisioning:
* Manager version is now v1.1
* .proto files and protocomm handler added for sending Wi-Fi scan command and receiving scan results
* Implemented handlers for wifi_scan protocomm endpoint
* Update manager context data structure to hold scan state and results
* scheme_softap now runs Wi-Fi in APSTA mode
* Wi-Fi is started in AP mode when provisioning is started. This is necessary for scan list to work
* Docs updates with information about new wifi_scan endpoint
List of changes in tools/esp_prov:
* Added functions for sending and receiving protobuf messages compatible with wifi_scan protocomm endpoint
* Added feature to display/refresh scan results and accept user selection at runtime
* New functions:
* get_version() : only returns the protocol version string
* has_capability() : check is a capability is present according to proto-ver response
* wifi_scan feature is provided only if the `wifi_scan` capability is present
Other changes:
* Replace recursive mutex with plain mutex
* assert on return value of mutex give / take calls
* replace all calls with macros ACQUIRE_LOCK and RELEASE_LOCK
* some checks added in scanning related private APIs
* free and nullify scanning context and state if service is stopped while ongoing scan
This MR improves existing flash encryption document to provide simplified steps
Adds two new modes for user: Development & Release
Adds a simple example
Supports encrypted write through make command
How idf_build_component and the COMPONENTS argument to idf_build_process
interact is not clear/misleading. Clarify their interaction in the docs.
Closes: https://github.com/espressif/esp-idf/issues/3630
This commit refactors backtracing within the panic handler so that a common
function esp_backtrace_get_next_frame() is used iteratively to traverse a
callstack.
A esp_backtrace_print() function has also be added that allows the printing
of a backtrace at runtime. The esp_backtrace_print() function allows unity to
print the backtrace of failed test cases and jump back to the main test menu
without the need reset the chip. esp_backtrace_print() can also be used as a
debugging function by users.
- esp_stack_ptr_is_sane() moved to soc_memory_layout.h
- removed uncessary includes of "esp_debug_helpers.h"
we use `-` command to check if DUT reset pass. If we input `-` command
during DUT bootup, DUT could only receive `\n` and print test cases.
Print test cases could take long time and lead to reset check timeout.
Now we will add delay after reset, and enlarge reset check timeout to
solve this problem.
Clang tidy 9.0.0 is to perform static analysis of IDF sources. All component sources are analysed with default sdkconfig configuration, based on examples/get-started/hello_world project (compilation commands are extracted from default build commands for this project). Configuration of static analysis is defined in tools/ci/static-analysis-rules.yml
Closes https://github.com/espressif/esp-idf/issues/145
!4452 used setting LINK_LIBRARIES and INTERFACE_LINK_LIBRARIES to link
components built under ESP-IDF build system. However, LINK_LIBRARIES does
not produce behavior same as linking PRIVATE. This MR uses the new
signature for target_link_libraries directly instead. This also moves
setting dependencies during component registration rather than after all
components have been processed.
The consequence is that internally, components have to use the new
signature form as well. This does not affect linking the components to
external targets, such as with idf_as_lib example. This only affects
linking additional libraries to ESP-IDF libraries outside component processing (after
idf_build_process), which is not even possible for CMake<v3.13 as
target_link_libraries is not valid for targets not created in current
directory. See https://cmake.org/cmake/help/v3.13/policy/CMP0079.html#policy:CMP0079
Previously ldgen determines the output file name on its own. This commit
makes it so that user can dictate what the output file name will be
for the processed template, if the user needs it for something else.
!4452 simplified early expansion by using an early expansion script that
only does one thing: get the public and private requirements for each
component, albeit one by one. This was also dependent on parsing
the command output of the expansion script. This commit makes it so that a list of all
components to be processed to passed to the expansion script, generating a cmake
file that sets each component requirements in one go.
This also makes sure that only components that registered themselves get
included in the final build list.
!4452 had config generation first before building the component list
to be used in the build. This proved to be detrimental when a new target
is added as config generation would consider configs from both targets.
For example, if a renamed option CONFIG_NEW is a bool with value "n", kconfiglib will not generate a define for it in the Kconfig file. The define (#define CONFIG_NEW 1) will only be generated if the option is "y" or "m".
However the compatibility definition was always generated: #define CONFIG_OLD CONFIG_NEW. This broke the #ifdef checks which depended on the old option names.
For example, if a renamed option CONFIG_NEW is a bool with value “n”,
kconfiglib will not generate a define for it in the Kconfig file. The
define (#define CONFIG_NEW 1) will only be generated if the option is
“y” or “m”. However the compatibility definition was always
generated: #define CONFIG_OLD CONFIG_NEW. This broke the #ifdef
checks which depended on the old option names.
This commit wraps each compatibility definition:
#ifdef CONFIG_NEW
#define CONFIG_OLD CONFIG_NEW
#endif
so that the CONFIG_OLD definition is only generated if CONFIG_NEW is
defined.
Changes argument parsing mechanism from argparse to a new one, that provides better support for extensions and options that are only applicable to specific subcommands,
Breaking changes:
1. All global options should go before subcommands, i.e. `idf.py build -C ~/some/project` will not work anymore, only `idf.py -C ~/some/project build` is acceptable
2. To provide multiple values to an option like `--define-cache-entry` it's necessary to repeat option many times, i.e. `idf.py -D entry1 entry2 entry3` will not work, right way is: `idf.py -D entry1 -D entry2 -D entry3`
At the moment there are 3 options like this: `--define-cache-entry` in base list and `--test-components` and `--test-exclude-components` in the unit test extensions
3. Drops `defconfig` and `bootloader-clean` subcommands
Closes https://github.com/espressif/esp-idf/issues/3570
Closes https://github.com/espressif/esp-idf/issues/3571
Using xxx_periph.h in whole IDF instead of xxx_reg.h, xxx_struct.h, xxx_channel.h ... .
Cleaned up header files from unnecessary headers (releated to soc/... headers).
Removes the need to know/guess the paths to these libraries. Once we are gcc 8 only, we
can remove -nostdlib and no additional arguments are needed for system libraries.
The catch is: any time IDF overrides a symbol in the toolchain sysroot, we need
an undefined linker marker to make sure this symbol is seen by linker.
Adds --tags and --dirty flags to cmake git_describe() calls, and not
pass the HEAD commit hash (incompatible with --dirty)
Makes IDF_VER output the same as in Make build system
Thanks to @william-ferguson-au for reporting this:
https://github.com/espressif/esp-idf/issues/3378#event-2355460974
List of changes:
* Retrieve UUID property from Bluez device object before connecting to retrieve UUID contained in advertisement
* Read Characteristic User Descriptions attribute of each UUID for mapping endpoint names
* To support older implementations with hardcoded Name-UUID map, revert to fallback mode in order if advertisement data has no UUID field
This MR removes the common dependency from every IDF components to the SOC component.
Currently, in the ``idf_functions.cmake`` script, we include the header path of SOC component by default for all components.
But for better code organization (or maybe also benifits to the compiling speed), we may remove the dependency to SOC components for most components except the driver and kernel related components.
In CMAKE, we have two kinds of header visibilities (set by include path visibility):
(Assume component A --(depends on)--> B, B is the current component)
1. public (``COMPONENT_ADD_INCLUDEDIRS``): means this path is visible to other depending components (A) (visible to A and B)
2. private (``COMPONENT_PRIV_INCLUDEDIRS``): means this path is only visible to source files inside the component (visible to B only)
and we have two kinds of depending ways:
(Assume component A --(depends on)--> B --(depends on)--> C, B is the current component)
1. public (```COMPONENT_REQUIRES```): means B can access to public include path of C. All other components rely on you (A) will also be available for the public headers. (visible to A, B)
2. private (``COMPONENT_PRIV_REQUIRES``): means B can access to public include path of C, but don't propagate this relation to other components (A). (visible to B)
1. remove the common requirement in ``idf_functions.cmake``, this makes the SOC components invisible to all other components by default.
2. if a component (for example, DRIVER) really needs the dependency to SOC, add a private dependency to SOC for it.
3. some other components that don't really depends on the SOC may still meet some errors saying "can't find header soc/...", this is because it's depended component (DRIVER) incorrectly include the header of SOC in its public headers. Moving all this kind of #include into source files, or private headers
4. Fix the include requirements for some file which miss sufficient #include directives. (Previously they include some headers by the long long long header include link)
This is a breaking change. Previous code may depends on the long include chain.
You may need to include the following headers for some files after this commit:
- soc/soc.h
- soc/soc_memory_layout.h
- driver/gpio.h
- esp_sleep.h
The major broken include chain includes:
1. esp_system.h no longer includes esp_sleep.h. The latter includes driver/gpio.h and driver/touch_pad.h.
2. ets_sys.h no longer includes soc/soc.h
3. freertos/portmacro.h no longer includes soc/soc_memory_layout.h
some peripheral headers no longer includes their hw related headers, e.g. rom/gpio.h no longer includes soc/gpio_pins.h and soc/gpio_reg.h
BREAKING CHANGE
Reported in https://github.com/espressif/esp-idf/issues/3299
If a config item contains a reserved filename in Windows like _CON_ then an invalid
directory is created.
We don't rely on this feature all, so disable it.
1. separate rom include files and linkscript to esp_rom
2. modefiy "include rom/xxx.h" to "include esp32/rom/xxx.h"
3. Forward compatible
4. update mqtt
Works around bug where components/soc/CMakeLists.txt was testing "EXISTS
${COMPONENT_PATH}/${SOC_NAME}" and this test could pass during early
expansion if COMPONENT_PATH was empty and a directory /esp32 exists
on Windows.
Closes https://github.com/espressif/esp-idf/issues/3195
V2 adds:
* Independent result for visibility (showing/hiding menus)
* Includes adding IDs for all items (menus & symbols) in kconfig_menus.json
Still backwards compatible with V1, with some small changes (menu items now listed in results).
Also added some protocol docs, changed the "listening on stdin" message to come after any kconfiglib warnings
Linker script generator produces build/esp32/esp32.common.ld from
components/esp32/ld/esp32.common.ld.in
This works fine until IDF is downgraded to V3.1 which uses components/esp32/ld/esp32.common.ld and
doesn't track build/esp32/esp32.common.ld at all.
At this point, the linker runs in the build/esp32 directory and "-T esp32.common.ld" picks up the
linker script generated .ld file, which causes mis-builds.
As reported on forums: https://esp32.com/viewtopic.php?f=13&t=9684&p=40105
A problem if the Python interpreter used for idf.py (or set via PYTHON
variable) didn't match
"/usr/bin/env python" (or the associated executable for .py files, on
Windows).
Closes https://github.com/espressif/esp-idf/issues/3160
Possibly also fix for https://github.com/espressif/esp-idf/issues/2936
Adds build system test to catch any future direct execution of Python in
the standard build process.
When generating JSON metadata for ranges where there are conditional ranges (ie different allowed range
depending on another config setting), the JSON metadata would always have the last named range as
the expression was not evaluated properly.
Thanks to ulfalizer on GitHub for pointing this out.
Closes https://github.com/espressif/esp-idf/issues/2195
Previously, this compiler flag was not being applied
regardless of CONFIG_SPIRAM_CACHE_WORKAROUND setting.
Explanation: add_compile_options() only applies to
source files added after the function is run, or in
subdirectories added after the function is run. In
this case, no new source files were being added after
this function was run.
1. ref_clock used in unit tests occasionally produces time off by ~100
microseconds shortly after being started. Add a delay to let
ref_clock stabilise, until the cause is found.
2. Reduce roundoff error accumulation which would occasionally cause
the test to fail, by choosing an overflow value which can be divided
by APB frequency.
3. Move time sampling part of the test into an IRAM function to
reduce variations due to cache behavior.
4. Remove calculation of "standard deviation" in the test, as what was
calculated was not actually standard deviation, and it did not add any
useful information.
List of changes:
* Version endpoint now sends the set version string instead of verifying the incoming version string. This simplifies fetching version info from the provisioning application.
* esp_prov script updated to expect version string as response.
1. When one of the COMPONENT_DIRS points to a component directory
(i.e. a directory containing component.mk, not a directory of multiple
components), and there is a subdirectory in it which also contains
a component, the subdirectory was mistakenly added to the list of
components and compiled.
For example:
main/
component.mk
main.c
test/
component.mk
test_main.c
Would compile test_main.c and link libtest.a.
2. When one of the COMPONENT_DIRS points to a component directory, and
the parent directory contained a directory with the same name as
another component, that directory would be mistakenly added to the
COMPONENT_PATHS.
For example:
esp/
esp-idf/
esp32/
(random stuff)
mycomponent/
component.mk
mycomponent.c
myproject/
main/
Makefile
and Makefile sets EXTRA_COMPONENT_DIRS=$(realpath ../mycomponent),
then "esp32" directory which is at the same level as mycomponent
was added to COMPONENT_PATHS.
3. If EXTRA_COMPONENT_DIRS pointed to a directory with a list of
components, and one of the subdirectories was not a component, but
had the same name as another component, than that directory would be
mistakenly added to COMPONENT_PATHS instead of the real esp32
component directory.
For example:
my_components/
my_component/
component.mk
my_component.c
esp32/
(some random stuff)
and EXTRA_COMPONENT_DIRS would point to my_components/, then "esp32"
directory would be added to COMPONENT_PATHS instead of the real esp32
component directory.
When IDF_PATH is not set by the user, cmake infers the same through
other means and sets it as an environment variable. However, some
sub-invocation of cmake may not see this variable as it is not set
in the parent. This change adds a custom command for exporting IDF_PATH
just before invoking ldgen so that IDF_PATH can be used in Kconfig
without any errors.
1. Gitlab will only regard failure (not error) as failed case. Use log
failure instead of log error, to display all error msg in test result
2. fix failed case not recognized by test script when failed during
reset DUT
1. example test uses auto detect flash size. we need to call
`detect_flash_size` before write flash
2. fix incorrect baudrate used: when using WROVER-Kits, it's likely that download with baudrate 921600
will fail. If we don't reset serial setting in decorator, 921600 will
become the default baudrate. This causes all the subsequent
communication fails
3. do hw reset after used esptool function
* Prevents section type conflict errors if (say) const & non-const data
is put into the same section (ie with DRAM_ATTR)
* Allows linker --gc-sections to remove unused custom sections
* Call esptool directly not via subprocess
* Use the same serial port instance for listener thread and esptool
* Includes some refactoring for encapsulation of App vs DUT members
Added a new structure esp_app_desc_t. It has info about firmware:
version, secure_version, project_name, time/date build and IDF version.
Added the ability to add a custom structure with a description of the firmware.
The esp_app_desc_t is located in fixed place in start of ROM secotor. It is located after structures esp_image_header_t and esp_image_segment_header_t.
app_version is filed from PROJECT_VER variable (if set in custom make file) or PROJECT_PATH/version.txt or git repo (git describe).
Add API to get app_desc from partition.
1. replace xunitgen with junit_xml, which can log more info
2. allow test cases to handle junit test report by them own
3. allow test cases to log some info into report via `sysout` tag
New unity component can be used for testing other applications.
Upstream version of Unity is included as a submodule.
Utilities specific to ESP-IDF unit tests (partitions, leak checking
setup/teardown functions, etc) are kept only in unit-test-app.
Kconfig options are added to allow disabling certain Unity features.
There's still a small chance that can't process command issue still
happen on some runners. Now we change the way to detect if the runner
reset finished: input `-` to check test history for a few times until it
succeeded. If DUT can process check history command, then it should be
able to process other test commands.
Gitlab CI now allow us to define if a job need to be created by
varialbes. This commit add label to CI jobs, so we can make some jobs
not created in certain scenarios when we trigger pipeline with @bot.
Previous version assumed having winpty on the Path meant that it would be needed,
but some tools (like Atlassian Sourcetree) permanently add winpty to the path.
Symptom is arrow keys not working in menuconfig dialog.
Uninstaller is still unsigned, as currently building and signing in 'wine'
and running Linux osslsigncode inside Inno Setup inside wine is awkward.
Closes https://github.com/espressif/esp-idf/issues/1909
TW20810
* BLE and softAP provisioning examples added along with tests.
* An application specific example added for demonstrating custom provisioning scheme.
* tools/esp_prov/esp_prov.py can be used as a provisioning client for SoftAP, BLE and CLI based transports.
Co-Authored-By: Amey Inamdar <amey@espressif.com>
Co-Authored-By: Anurag Kar <anurag.kar@espressif.com>
set shorter time to save some space on server. For those artifacts need
to be stored for long time, we can manually press "keep" button in job page.
Also add a CI job to check if all artifacts have set expire time.
gen_esp_err_to_name.py includes files like
components/lwip/include/apps/esp_ping.h as "apps/esp_ping.h" because all
subdirectories of "include" are used. If "components/lwip/include/apps"
is in the path of the compiler instead of "components/lwip/include" then
include "apps/esp_ping.h" will fail. This fix allows to process lines
like:
'// gen_esp_err_to_name.py: some text "esp_ping.h" some text'
in header files for overwriting the include.
Changes:
KConfig: The option CONFIG_ESP32_RTCDATA_IN_FAST_MEM is added in Kconfig file for esp32 component.
esp32.common.ld: added support of RTC_DATA_ATTR, RTC_RODATA_ATTR data placement into appropriate segment according to Kconfig option.
esp32.ld: linker script is modified to set alias for memory segment selected by Kconfig option to place data. The segments for force placement are added for RTC_FAST_ATTR, RTC_SLOW_ATTR attributes.
esp_attr.h: added new attributes RTC_FAST_ATTR, RTC_SLOW_ATTR for force placement into fest/slow memory.
test_rtc_fast.c: Added unit test cases to check data placement into appropriate memory segment.
Updated documentation for RTC_DATA_ATTR, RTC_FAST_ATTR, RTC_SLOW_ATTR in deep_sleep_stub.rst file.
TW#18001
Closes https://github.com/espressif/esp-idf/issues/1553
added docs and tests for mqtt library, small fixes (removed warnings, option for custom outbox, websocket bug fixed for longer transports). refactored to use common tcp_transport component, support for CMake build system.
Closes#2108
Using 'sudo pip install' can cause a lot of problems because usually
there are a couple of Python interpreters installed. This fix encourages
developers to use `pip install --user` instead which is safer.
Guarantees that a component's project_include.cmake will be called after its dependent components'
project_include.cmake. Because of cycles in the dependency graph, this is less useful than you'd
think but it gives a strong guarantee for any component which is not part of a cycle.
Also applies deterministic ordering (ordering is initialised as COMPONENT_REQUIRES_COMMON then all
COMPONENTS in alphabetical order, but then the sorting by dependencies is applied.)
1. Add the 100 times test when the private key is generated by the random number;
2. Add the bt components to the unit-test-app/config directory.
3. Added the bt unit test case to CI.
Add a command `make erase_ota` - erases otadata if it there is in the
partition table otherwise error 'Partition table does not have ota_data partition'.
Closes https://github.com/espressif/esp-idf/issues/1821
If project.mk is included twice in recursive invocation of Make, some
variables defined on the first pass will not be redefined on the
second pass. Rather than cleaning up these variables before calling
Make recursively, don’t include IDF project.mk at all, if one of the
ut- targets is requested.
We found some SD card on Raspberry Pi could have very bad performance.
It could take seconds to save small amount of data.
If the DUT receives data and save it as log, then it stops receiving data until log is saved.
This could lead to expect timeout. As an workaround to this issue,
``BaseDUT`` class will create a thread to save logs.
Then data will be passed to ``expect`` as soon as received.
51e42d8e92
introduced filtering options which handles the last (unterminated) line
with a delay. This introduced poor performance for console applications
when the user interacts with the ESP32 device in the same line because
there is an artificial delay for each key-press.
bb152030a0
decreased the delay but there are still delays for each key-presses.
The current fix makes only one delay per line instead of each
key-presses. When an unterminated line is detected then no more
"delayed" print is used for the given line.
Fixes https://github.com/espressif/esp-idf/issues/2054
This change allows readonly FATFS to be mounted without wear levelling
support. This will provide the customers a simple way to mount FATFS images
generated on host and flashed onto the chip during factory provisioning.
Since NVS encryption is not supported yet and NVS entry size is limited,
the change will provide an easy alternative for securing the provisioning data
by just marking FATFS parition as encrypted.
The IDF monitor waits for one second before printing the last
unterminated line. This is necessary for the correct filtering.
However, this causes poor performance, for example, in the case of the
system/console example.
This fix reduces the waiting time which improves the response time of
the console example and doesn't seem to influence the monitor filter.
Closes https://github.com/espressif/esp-idf/issues/2054
For some test configurations, not all tests need to be run. This
change allows adding a comment in the configuration file of the
following form:
unit-test-app makefile will use it to determine which components to
build. If the comment is not present, all components will be built
(TESTS_ALL=1).
Also add test configuration for libsodium. It is a separate test
configuration due to the large binary size generated when building
libsodium tests.
Allows you to move the partition table, it gives more space for the bootloader.
Added a new utility - parttool.py. This utility can search for the offset and/or size of the partitions by name and type/subtype. Use for getting APP_OFFSET and PHY_DATA_OFFSET.
The linker(esp32.bootloader.ld) made changes that allow you to write a custom bootloader code more.
TW14125
Before entering the deep sleep, the RTC and FRC counters are synchronized. Updating the boot_time.
Added a unit test for this case.
Fixed warnings for MULTIPLE_STAGES
Closes https://github.com/espressif/esp-idf/issues/1840
These versions of conf/mconf are forked from the upstream, and the CMake system allows mconf-idf to
be installed externally on the path. So the best option is to rename.
Includes changes to Windows installer for v1.1
Don't show the "Python 2 interpreter" option in menuconfig when using CMake.
This is a stop-gap until we support Python 2 & 3 together in ESP-IDF (soon).
Closes https://github.com/espressif/esp-idf/issues/1924
Will give hint when any job failed to assign:
```
Please add the following jobs to .gitlab-ci.yml with specific tags:
* Add job with: psram, ESP32_IDF, UT_T1_1
* Add job with: psram, ESP32_IDF, UT_T1_1
```
We have built unit-test-app with different configs.
Currently we use the config name as tags to match runners.
It's not easy to add new configs (need to update tags to existed runners).
Now we'll parse required test runner tags from `sdkconfig` file.
For example, if config enables `CONFIG_SPIRAM_SUPPORT`, then it requires `psram` tag.
This will make adding new configs easier.
In this commit we change the one behavior of assign test: match keys of
cases should be exactly the same with job tags. This fixes cases select
jobs include their tags, and jobs requires those tags can't be assigned.
1. configs are not functional
* decorator will be executed when search case, need to set default config before search case.
2. fix DUT encode/decode errors
* python3 serial don't support write string/unicode, need to convert to bytes first.
* python2 string could failed to encode/decode non-acsii chars
3. fix bug that log folder not created
4. fix bug that test suite name is not correctly passed:
* the keyward arg should be `test_suite_name` not `test_name`
5. fix bug that test stopped on failed case
6. fix DUT `read` don't return all data cache
* `_DataCache.get_data` should first copy all data from queue to data cache and then return to user.
7. fix bug that `expect_all` failed even all expected item passed
8. optimize error info for expect
* print pattern for regular expression when match failed
9. fix bug that set default config doesn't work
1. Attenuator: control programmable attenuator
2. PowerControl: control APC PDU to power on/off devices
3. LineChart: use matplotlib to draw line chart
Refactor IDF "project" functionality under a wrapping of the default
"project" command, so we can tweak it a bit...
Will need more testing in other environments.
* Philosophical: "explicit is better than implicit".
* Practical: Allows useful errors if invalid directories given in components as the defaults aren't
always used. Also trims the -I path from a number of components that have no actual include
directory.
* Simplifies knowing which components will be header-only and which won't
Unit test runner expects to see the test name echoed after test
selection is made. If the unit test immediately goes into sleep mode,
UART output will not be complete, and test runner will not see the
test name. This flushes the stream buffer and waits for UART FIFO to
be empty before starting the test.
Additionally some parts of code used unity_printf and some used
printf. Since unity_printf was only useful during very early tests
when newlib was not available, replace all its usages with printf.
By default, any job will run unless a filter is given, in that case
the filter will determine if the job should run or not. Some jobs do
not need to be run by default, and should only be triggered using the
bot. For such jobs, BOT_NEEDS_TRIGGER_BY_NAME can added to
environment variables.
With V=0, build process would print “including .../Makefile.projbuild" lines, causing problems for print_flash_cmd target.
The issue was due to the way macro expansion works in make. To delay evaluation of info function until the execution of expanded block, two dollar signs are required.
Test for print_flash_cmd target added.
Since !2190, source files located outside of the component root
produce object files inside build directory. This change fixes
handling of COMPONENT_OBJEXCLUDE variable for such files. Tests are
added.
We had some problems:
simultaneous compiling of kconfig in the same tree;
attempt to use menuconfig in some examples and ut in batch mode (w/o interactive console)
Also increase debug abilities in CI:
force non-interactive building;
add variable DEBUG_SHELL to toggle verbosity of scripts
tools/gen_esp_err_to_name.py generates
components/esp32/esp_err_to_name.c for lookup of the error codes from
all of the IDF project and easily identify all codes in one place
current unit-test-app don't support test components need to communicate
with each other (like GPIO, SPI ...). Now we add multiple devices mode
to unit test app, support writing and running test with multiple DUTs.
please refer to `docs/api-guides/unit-tests.rst` for detail.
This commit backports vTaskDelete() behavior from FreeRTOS v9.0.0 which
allows for the immediate freeing of task memory if the task being deleted
is not currently running and not pinned to the other core. This commit also
fixes a bug in prvCheckTasksWaitingTermination which prevented the
Idle Task from cleaning up all tasks awaiting deletion. Each iteration of the Idle
Task should traverse the xTasksWaitingTermination list and clean up all tasks
not pinned to the other core. The previous implementation would cause
prvCheckTasksWaitingTermination to return when encountering a task
pinned to the other core whilst traversing the xTasksWaitingTermination list.
The test case for vTaskDelete() has been updated to test for the bugfix and
backported deletion behavior.
This commit backports the following features from FreeRTOS v9.0.0
- uxSemaphoreGetCount()
- vTimerSetTimerId(), xTimerGetPeriod(), xTimerGetExpiryTime()
- xTimerCreateStatic()
- xEventGroupCreateStatic()
- uxSemaphoreGetCount()
Functions backported previously
- xTaskCreateStatic()
- xQueueCreateStatic()
- xSemaphoreCreateBinaryStatic(), xSemaphoreCreateCountingStatic()
- xSemaphoreCreateMutexStatic(), xSemaphoreCreateRecursiveMutexStatic()
- pcQueueGetName()
- vTaskSetThreadLocalStoragePointer()
- pvTaskGetThreadLocalStoragePointer()
Unit tests were also written for the functions above (except for pcQueueGetName
which is tested in a separate Queue Registry MR). The original tlsp and del cb test case
was deleted and integrated into the test cases of this MR.
This commit makes the configQUEUE_REGISTRY_SIZE and
configGENERATE_RUN_TIME_STATS configurable in menuconfig.
- configQUEUE_REGISTRY_SIZE can now be set in menuconfig.
- The functions vQueueAddToRegistry() and vQueueUnregisterQueue() were made
SMP compatbile
- pcQueueGetName() was backported from FreeRTOS v9.0.0
- Added test case for Queue Registry functions
- configGENERATE_RUN_TIME_STATS can now be enabled in menuconfig. CCOUNT or
esp_timer can be selected as the FreeRTOS run time clock in menuconfig as
well, although CCOUNT will overflow quickly.
- Run time stats collection (in vTaskSwitchContext) and generation (in
uxTaskGetSystemState) have been made SMP compatible. Therefore
vTaskGetRunTimeStats() now displays the run time usage of each task as a
percentage of total runtime of both CPUs
Squash
This commit makes configUSE_TRACE_FACILITY and
configUSE_STATS_FORMATTING_FUNCTIONS configurable in kconfig. Test cases fro the
functions enabled by the two configurations above have also been added.
Test cases for the following functions have been added...
- uxTaskGetSystemState()
- uxTaskGetTaskNumber()
- vTaskSetTaskNumber()
- xEventGroupClearBitsFromISR()
- xEventGroupSetBitsFromISR()
- uxEventGroupGetNumber()
- uxQueueGetQueueNumber()
- vQueueSetQueueNumber()
- ucQueueGetQueueType()
Test cases for the following functions were not required...
- prvListTaskWithinSingleList()
- prvWriteNameToBuffer()
- vTaskList()
* Only direct invocation of cypto_hash_sha256 & crypto_hash_sha512 produced correct output.
* Some remaining header file mixups, so changed approach to wrapping the state structures.
* Fixes https://github.com/espressif/esp-idf/issues/1044 (crypto_sign problems)
* Add option to disable mbedTLS implementation for SHA256 & SHA512 in libsodium
Fix warnings where undefined vars are used.
Make Kconfig emit "FOO=" for unset bool options
To ensure make variables are always defined, even if empty.
When writing auto.conf, include symbols disabled by dependency to make sure all make variables are always defined.
Fixesespressif/esp-idf#137
Cherry-picked from https://github.com/espressif/esp-idf/pull/138
Too hard to stage the dependencies so that all clean steps complete before any build steps begin. Also, using and then
deleting and then regenerating SDKCONFIG_MAKEFILE in one pass is really hard to manage successfully.
Other terminal emulators (screen, minicom) default to sending CR when
ENTER is pressed on the keyboard. Make behavior of idf_monitor
consistent with them.
Between 3.1.0 and 3.3.0, pyserial had thread cancellation implemented
using a select, which blocked on the stdin and an auxiliary pipe. When
thread had to be cancelled, a byte would be sent into the pipe,
unblocking stdin.
Unfortunately, this method suffers from a problem with using select on
a StreamReader (which represents the decoder wrapped around stdin).
In some cases, when the TTY sends an escape sequence in response to
an escape sequence received from serial, this escape sequence will not
be read from stdin until some key is pressed.
In https://github.com/pyserial/pyserial/commit/cab3dab, this method
was replaced with an TIOCSTI ioctl. This change makes sure we use the
new cancellation method even if the script is running with older
pyserial.
1. Hello World application shows no footprint difference before and
after this change
2. examples/ethernet/ethernet application compiles properly (can't
test with my board)
Makes app image booting more reliable (256-bit rather than 8-bit verification.)
Some measurements, time to boot a 655KB app.bin file and run to app_main() execution.
(All for rev 1 silicon, ie no 340ms spurious WDT delay.)
80MHz QIO mode:
before = 300ms
after = 140ms
40MHz DIO mode:
before = 712ms
after = 577ms
40MHz DIO mode, secure boot enabled
before = 1380ms
after = 934ms
(Secure boot involves two ECC signature verifications (partition table, app) that take approx 300ms each with 80MHz CPU.)
For config-only components, component.mk should now contain "COMPONENT_CONFIG_ONLY := 1"
Also refactored some of the generation of linker paths, library list. This required cleaning up the way the bootloader
project works, it's now mostly independent from the parent.
Now the tests do not pass
Before we had incorrect code of the error code checking
in build_examples.sh for that case and did nothing in test_build_system.sh.
The CI uses the mirror-submodule-update.sh for non-permanent reassignment
of 3rdparty sub-modules to local mirrors (only for 'non-master' branches).
List of the mirrors in mirror-list.txt
You can use the script locally to speed up data retrieval on network problems
check_submodule_sync job uses 'GIT_STRATEGY: clone' to check the availability of public sources
Implements support for system level traces compatible with SEGGER
SystemView tool on top of ESP32 application tracing module.
That kind of traces can help to analyse program's behaviour.
SystemView can show timeline of tasks/ISRs execution, context switches,
statistics related to the CPUs' load distribution etc.
Also this commit adds useful feature to ESP32 application tracing module:
- Trace data buffering is implemented to handle temporary peaks of events load
Add new "make size_components" and "make size_files" targets for analysing makeup of the ELF file.
Plain "make size" output now looks like this:
Total sizes:
DRAM .data size: 11156 bytes
DRAM .bss size: 22304 bytes
Used static DRAM: 33460 bytes ( 294220 available, 10.2% used)
Used static IRAM: 80480 bytes ( 50592 available, 61.4% used)
Flash code: 421463 bytes
Flash rodata: 103824 bytes
Total image size:~ 616923 bytes (.bin may be padded larger)
Run some unit test cases based (poorly) around sodium's test infrastructure.
Increase in unity test task stack is due to signature tests, load a lot of data onto the stack.
CI: auto generate configs for test jobs
Currently CI test jobs are static configured by several pre-generated config files.
This approach have several disadvantages:
1. not flexible to select test cases to run, which is important feature of @bot
2. difficult to update test as we need to pre-generate quite a lot files
3. need to maintain extra config files in IDF
4. not flexible to support new test apps or chips, can't use some new features of test bench
Therefore we'll add assign_test stage between build and test, to generate configs for test jobs.
See merge request !738
CI: save download config in CI build jobs:
IDF built app download config may change with some modification.
save the download configs for built APP in build stage.
then we can use the correct download config in test stage.
See merge request !676
FATFS fixes and tests
This MR includes a set of fixes related to FATFS, SDMMC, and wear levelling:
- `esp_vfs_fat_spiflash_mount` uses `FM_SFD` flag when creating the partition. The volume layout (given in `VolToPart` variable) was not compatible with SFD mode, so mkfs was failing. This fixes the volume layout to use "autodetect" for both volumes. Merges https://github.com/espressif/esp-idf/pull/559.
- fix `prepend_drive_to_path`function, which didn't prepend drive to path (while consuming 2k of stack space)
- fix stack overflow in vfs_fat_link function which allocated two 4kbyte `FIL` structures on the stack
- fix support for having two FATFS instances (in flash and SD) mounted at the same time
- unit tests written for FATFS on SDMMC are made common between SDMMC and WL implementations; FATFS unit tests on WL will run during CI
- fix inconsistent definition of PATH_MAX and ARG_MAX (TW12207, TW12104, https://github.com/espressif/esp-idf/issues/289)
See merge request !732
IDF built app download config may change with some modification.
save the download configs for built APP in build stage.
then we can use the correct download config in test stage.
wear levelling
This MR adds wear levelling component. It presents an interface similar to the interface of spi_flash and esp_partition. Inside, it stores data inside a partition to in a way that reduces worst case number of erase cycles for any given sector.
Also included are APIs similar to the ones provided for SDMMC to mount FAT filesystem on top of the wear levelling partition. A simple example shows how this API can be used.
Ref TW10338.
See merge request !567
- Implements application tracing module which allows to send arbitrary
data to host over JTAG. This feature is useful for analyzing
program modules behavior, dumping run-time application data etc.
- Implements printf-like logging functions on top of apptrace module.
This feature is a kind of semihosted printf functionality with lower
overhead and impact on system behaviour as compared to standard printf.
Windows setup improvements
* Tweak Makefile to significantly reduce startup time on Windows (also reduced on other platforms, but less obvious.)
* Revise windows setup installation script to fix various issues, use latest toolchain.
* Tweak Windows config docs
See merge request !597
SPI flash operations lock for shorter periods
Based on bug report here:
https://esp32.com/viewtopic.php?f=13&t=1489&p=6816#p6773
Long SPI flash erase (and possibly write/read) cycles block all tasks on both CPUs for an extended period, and can lead to WiFi dissassociation (and general priority inversion.)
This MR inserts preemption breaks into large operations (all erases, writes every 8KB, reads every 16KB).
Overhead of a single spi_flash_guart_start()/spi_flash_guard_end() cycle measured at approx 67us (assuming no preemption.)
See merge request !600
* "make monitor" not passed the configured baud rate
Closes#436https://github.com/espressif/esp-idf/issues/436
* Pass toolchain prefix from sdkconfig into monitor tool
* Allow setting EOL in idf_monitor.py, use CRLF by default
* Detect if /dev/tty.X is used on macOS, warn and replace with /dev/cu.X
* If a build fails or gdb exits, ignore Ctrl-T (allowing Ctrl-T Ctrl-A/F to be same key sequence everywhere)
* Add a note about winpty on Windows
Ref 02fdf8271d (commitcomment-21369196)
Expand 'make monitor' support
New 'make monitor' idf_monitor tool for better monitor output. Running 'make monitor' will now:
* Automatically look up code addresses via addr2line and print function, source file, line number in terminal.
* Can reset the ESP32 by typing Ctrl-T Ctrl-R.
* Can run "make flash" by typing Ctrl-T Ctrl-F.
* Can run "make app-flash" by typing Ctrl-T Ctrl-A.
* If gdb stub starts, monitor will automatically run gdb and connect. When gdb exits, ESP32 resets and monitor resumes.
* Exit is still Ctrl-[
Have some more features I'd like to add (log output to file, crash dump support) but I think this is at the point of being useful.
See merge request !565
unit tests: Make UnitTestParser ignore non-directories inside 'components' dirs
Tiny fix to UnitTestParser script behaviour, if you happen to have any stray files in your "components" directories.
See merge request !561
Partition/SPI/OTA docs & OTA new functionality
* Update partition, SPI flash & OTA docs to reflect functionality changes
* Refactor OTA implementation to perform checks mentioned in API doc
* Add new functions to OTA API: esp_ota_get_running_partition() & esp_ota_get_next_update_partition() functions
* Add spi_flash_cache2phys() & spi_flash_phys2cache() functions to support esp_ota_get_running_partition()
See merge request !513
Add python script that parses list of unit test cases for CI from component test folder
Modify .gitlab-ci.yml to run this script as part of build unit tests stage
Create temp config in the same dir as target
Fixes case where SDK and app are on different mounts
(In our usage of kconfig, dirname is the empty string & basename is the full path to sdkconfig)
Github pull request https://github.com/espressif/esp-idf/pull/151
See merge request !309
Add dynamic interrupt allocation mechanism
This adds:
- Dynamic allocation of interrupts. Pass it the features of the interrupt you want, it'll set you up with an int.
- Shared interrupts. Enables multiple peripheral drivers to use the same interrupt.
- Marking what interrupts are fully executable from IRAM; if an int isn't marked like that it will get disabled once flash cache gets disabled.
Also:
- Modifies driver to be in line with these changes
See merge request !254
unit-test-app: Don't auto-display menu after test run finishes
Previous code would sometimes display menu multiple times, depending on UART activity.
See merge request !253
rename nvs host test folder, modify .gitlab-ci.yml
remove unit-test-app build
re-format unit test files
remove extra newlines in project.mk
some refactoring for unit test part in project.mk
add build files of unit-test-app in gitignore
add README.md for unit test app
correct headings in README.md
remove files and make minor tweaks in unit test app
update .gitlab-ci.yml to use unit test app
delete unused lines in component_wrapper.mk
delete periph_i2s.h and lcd test
add text floating point in components/esp32/test/Kconfig
correct idf test build paths in .gitlab-ci.yml
Allow OS X to build without libintl
https://github.com/espressif/esp-idf/pull/42
I have tested this with the following configurations:
- OS X with Xcode command line tools, no macports, no homebrew
- OS X with Xcode command line tools and homebrew
- OS X with Xcode IDE and macports
All three work with this change, without it the first two had issues.
See merge request !172
Extra space is needed in echo -n "-lncurses ", otherwise if pkg-config outputs a directory after that, it will result in "-lncurses-L/opt/local/lib" (without space).
-Wno-format-security flag is needed on macOS to silence warnings about printf(gettext("message")) constructs.
Generated with Eclipse Neon, should work in earlier versions as well.
This file can be imported
- into the workspace via Preferences > C++ > Code Style > Formatter > Import
- or into the project via Project Preferences > C++ General > Formatter > Import
Configuration options in Eclipse are not exactly the same as in astyle.
There may be some discrepancy between the format astyle and Eclipse will produce.
If anyone notices that, please let me know.
This adds initial code style guide. Only section on code formatting is written, other sections to be added later.
Also adds scripts to format code using astyle.
Used by mbedTLS to set MBEDTLS_CONFIG_FILE in all components.
This change sets CFLAGS/etc at the project level and then exports those
variables for components, rather than setting them independently each time
a component Makefile is invoked.
