Currently hints are processed only once the process is finished and
exits with non-error exit code. In interactive mode, e.g. for monitor,
we want to process ouput lines for hints right away.
This adds a simple buffer, which keeps the last line and once EOL is
reached, it is processed for hints.
Since the original hints processing was file based, a new helper
function was added to allow processing hints in string.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This is partial backport of following commit, which changes
print_hints to generate_hints.
commit 92ef2a4c83
Author: simon.chupin <simon.chupin@espressif.com>
Date: Tue Aug 9 15:39:23 2022 +0200
Tools: Add unit tests for idf.py hints
Only hunks for core_ext.py and tools.py are picked. It would be possible
to use the original print_hints approach, where the hints are directly
printed out and not returned, but it seems to make sense to keep it
closer to most recent version. It should make further backports easier
and it allows to cherry pick the iterative hints approach.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently the forced progress in RunTool is trying to fit the output
line into a terminal width, but it doesn't take into an account a
situation when the terminal width is reported as zero. This manifests
when running in docker image with redirected output and can be seen
in the github workflow output for esp-idf-ci-action.
docker run --rm -t my_ubuntu_esp python3 -c 'import os,sys; print(os.get_terminal_size(), sys.stdout.isatty())'
os.terminal_size(columns=238, lines=59) True
vs
docker run --rm -t my_ubuntu_esp python3 -c 'import os,sys; print(os.get_terminal_size(), sys.stdout.isatty())' | tee
os.terminal_size(columns=0, lines=0) True
Since the output is reported as tty and the terminal width as 0, the
fit_text_in_terminal() function returns empty string. I also verified this
by running idf.py build inside a testing docker image.
This fix adjusts the fit_text_in_terminal() function to return original
line if the terminal width is zero.
Also simplify the progress print and use same approach as ninja
https://github.com/ninja-build/ninja/blob/master/src/line_printer.cc#L66
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This commit fixes an issue where paths on Windows are case insensitive, for instance when setting the build folder its name would be converted to lowercase.
The culprit is our realpath() function, that was calling os.path.normcase() internally, since we are removing that call it makes sense to just remove the function entirely and call os.path.realpath() wherever necessary.
Closes https://github.com/espressif/esp-idf/issues/10282
The actual output from the build tool (CMake/Ninja) may or may not
contain color escape codes, depending on various factors. The output
written to the log file should never include color escape codes,
though. This is because color escape codes in files are usually not
rendered as "color" in editors, and complicate reading. Also escape
codes would break the regular expressions used to display hints for
compilation errors.
If stdout is a TTY (meaning that the output is not redirected), tell
the build tool (GNU Make or Ninja) to enable colorized output.
GNU Make and Ninja also check if their stdout is redirected and
strip color escape sequences in that case. CLICOLOR_FORCE environment
variable overrides this behavior.
With this change, if the compiler was launched with the
-fcolor-diagnostics flag and idf.py output is not redirected, the
final output in the terminal will be colorized.
(-fcolor-diagnostics is handled at CMake level by the previous commit)
This fixes the issue with build output not being colorized on Windows,
while the hints messages are colorized.
The issue occurred because sys.stdout and sys.stderr get overridden
by colorama.init() at runtime, but the default argument
output_stream=sys.stdout holds the reference to the"original"
sys.stdout.
colorama.init() (which, by the way, gets called via a curious chain
of imports, via idf_component_tools.manifest and tqdm package)
overrides standard streams, on Windows only. The overridden streams
contain logic to convert ANSI color codes into Windows Console API
calls to colorize the text.
Since read_and_write_stream function used the default value of
output_stream evaluated at module loading time, it was using the
original sys.stdout, not the one overridden by colorama.
One extra note is that while this does fix the coloring issue, the
solution is a bit fragile, as it relies on one of the following
(on Windows):
- colorama.init() is called (this can change if idf-component-manager
stops importing tqdm)
- Sufficiently new version of Windows 10 is used, and ANSI color codes
support is enabled in the Registry.
Add docs subcommand with options:
--no-browser - Prints url for current documentation instead of opens browser with documentation
--language - Select documentation language
--starting-page - Choice section of documentation to open
--version - Choice version of esp-idf
--target - Choice model of your target
Can still be enabled by passing --cmake-warn-uninitialized on the command line
Prevents CMake warnings printed by default if IDF_PATH is underneath the CMake
project directory.
The reason for this is that CMake --warn-uninitialized only enables checks
inside the project directory (ie top-level CMakeLists.txt directory and
subdirectories), it doesn't enable for files included from other directories.
(The only way to enable warnings in other directories is to pass
--check-system-dirs and this looks like it's only useful for CMake's own
developers as it prints a lot of warnings from inside CMake otherwise - see
https://gitlab.kitware.com/cmake/cmake/-/issues/19645 )
Plan to follow up with a later commit to clean up most of the warnings (which
aren't problems for CMake execution), but we'll also disable this option by
default to avoid this unexpected triggering of IDF warnings.
This solves the issue that target is changed to the default one after
idf.py fullclean.
Also allow setting the default target using sdkconfig.defaults, e.g.
CONFIG_IDF_TARGET="esp32s2"
Closes IDF-1040
Python 2 expect the environ variables are all of type 'str', but
sometimes wrong 'unicode' type is given.
Here we force all variables that are not str to become str.
