alex/esp-idf
alex/esp-idf
1
0
Fork 0
mirror of https://github.com/espressif/esp-idf.git synced 2024-10-05 20:47:46 -04:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity

Merge branch 'feat/tools_python_constraints_env' into 'master'

Browse Source 
Tools: Disable Python constraint files with environment variable

Closes IDFGH-7722

See merge request espressif/esp-idf!18830
This commit is contained in:
Roland Dobai 2022-07-14 23:05:22 +08:00
commit 958819323d
3 changed files with 16 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
docs/en/api-guides/tools/idf-tools.rst
View File

@ -109,9 +109,9 @@ Any mirror server can be used provided the URL matches the ``github.com`` downlo
* ``check``: For each tool, checks whether the tool is available in the system path and in ``IDF_TOOLS_PATH``.
* ``install-python-env``: Create a Python virtual environment in the ``${IDF_TOOLS_PATH}/python_env`` directory and install there the required Python packages. An optional ``--features`` argument allows one to specify a comma-separated list of features to be added or removed. Feature that begins with ``-`` will be removed and features with ``+`` or without any sign will be added. Example syntax for removing feature ``XY`` is ``--features=-XY`` and for adding ``--features=+XY`` or ``--features=XY``. If both removing and adding options are provided with the same feature, no operation is performed. For each feature a requirements file must exist. For example, feature ``XY`` is a valid feature if ``${IDF_PATH}/tools/requirements/requirements.XY.txt`` is an existing file with a list of Python packages to be installed. There is one mandatory ``core`` feature ensuring core functionality of ESP-IDF (build, flash, monitor, debug in console). There can be an arbitrary number of optional features. The selected list of features is stored in ``idf-env.json``. The requirement files contain a list of the desired Python packages to be installed and ``espidf.constraints.*.txt`` downloaded from https://dl.espressif.com and stored in ``${IDF_TOOLS_PATH}`` the package version requirements for a given ESP-IDF version.
* ``install-python-env``: Create a Python virtual environment in the ``${IDF_TOOLS_PATH}/python_env`` directory and install there the required Python packages. An optional ``--features`` argument allows one to specify a comma-separated list of features to be added or removed. Feature that begins with ``-`` will be removed and features with ``+`` or without any sign will be added. Example syntax for removing feature ``XY`` is ``--features=-XY`` and for adding ``--features=+XY`` or ``--features=XY``. If both removing and adding options are provided with the same feature, no operation is performed. For each feature a requirements file must exist. For example, feature ``XY`` is a valid feature if ``${IDF_PATH}/tools/requirements/requirements.XY.txt`` is an existing file with a list of Python packages to be installed. There is one mandatory ``core`` feature ensuring core functionality of ESP-IDF (build, flash, monitor, debug in console). There can be an arbitrary number of optional features. The selected list of features is stored in ``idf-env.json``. The requirement files contain a list of the desired Python packages to be installed and ``espidf.constraints.*.txt`` downloaded from https://dl.espressif.com and stored in ``${IDF_TOOLS_PATH}`` the package version requirements for a given ESP-IDF version. Althought it is not recommended, the download and use of constraint files can be disabled with the ``--no-constraints`` argument or setting the ``IDF_PYTHON_CHECK_CONSTRAINTS`` environment variable to ``no``.
* ``check-python-dependencies``: Checks if all required Python packages are installed. Packages from ``${IDF_PATH}/tools/requirements/requirements.*.txt`` files selected by the feature list of ``idf-env.json`` are checked with the package versions specified in the ``espidf.constraints.*.txt`` file. The constraint file will be downloaded from https://dl.espressif.com if this step hasn't been done already in the last day.
* ``check-python-dependencies``: Checks if all required Python packages are installed. Packages from ``${IDF_PATH}/tools/requirements/requirements.*.txt`` files selected by the feature list of ``idf-env.json`` are checked with the package versions specified in the ``espidf.constraints.*.txt`` file. The constraint file will be downloaded from https://dl.espressif.com if this step hasn't been done already in the last day. The use of constraints files can be disabled similarly to the ``install-python-env`` command.
* ``uninstall``: Print and remove tools, that are currently not used by active ESP-IDF version.

4
tools/docker/Dockerfile
View File

@ -75,6 +75,10 @@ RUN : \
&& rm -rf $IDF_TOOLS_PATH/dist \
&& :
# The constraint file has been downloaded and the right Python package versions installed. No need to check and
# download this at every invocation of the container.
ENV IDF_PYTHON_CHECK_CONSTRAINTS=no
# Ccache is installed, enable it by default
ENV IDF_CCACHE_ENABLE=1
COPY entrypoint.sh /opt/esp/entrypoint.sh

16
tools/idf_tools.py
View File

@ -1879,8 +1879,7 @@ def get_constraints(idf_version): # type: (str) -> str
try:
age = datetime.date.today() - datetime.date.fromtimestamp(os.path.getmtime(constraint_path))
if age < datetime.timedelta(days=1):
info(f'Skipping the download of {constraint_path} because it was downloaded recently. If you believe '
f'that this is causing you trouble then remove it manually and re-run your install script.')
info(f'Skipping the download of {constraint_path} because it was downloaded recently.')
return constraint_path
except OSError:
# doesn't exist or inaccessible
@ -1902,6 +1901,7 @@ def get_constraints(idf_version): # type: (str) -> str
return constraint_path
else:
fatal('Failed to download, and retry count has expired')
info('See the help on how to disable constraints in order to work around this issue.')
raise DownloadError()
@ -2402,6 +2402,8 @@ def main(argv): # type: (list[str]) -> None
uninstall.add_argument('--dry-run', help='Print unused tools.', action='store_true')
uninstall.add_argument('--remove-archives', help='Remove old archive versions and archives from unused tools.', action='store_true')
no_constraints_default = os.environ.get('IDF_PYTHON_CHECK_CONSTRAINTS', '').lower() in ['0', 'n', 'no']
if IDF_MAINTAINER:
for subparser in [download, install]:
subparser.add_argument('--mirror-prefix-map', nargs='*',
@ -2419,9 +2421,10 @@ def main(argv): # type: (list[str]) -> None
install_python_env.add_argument('--no-index', help='Work offline without retrieving wheels index')
install_python_env.add_argument('--features', default='core', help='A comma separated list of desired features for installing.'
' It defaults to installing just the core funtionality.')
install_python_env.add_argument('--no-constraints', action='store_true', default=False,
install_python_env.add_argument('--no-constraints', action='store_true', default=no_constraints_default,
help='Disable constraint settings. Use with care and only when you want to manage '
'package versions by yourself.')
'package versions by yourself. It can be set with the IDF_PYTHON_CHECK_CONSTRAINTS '
'environment variable.')
if IDF_MAINTAINER:
add_version = subparsers.add_parser('add-version', help='Add or update download info for a version')
@ -2446,9 +2449,10 @@ def main(argv): # type: (list[str]) -> None
check_python_dependencies = subparsers.add_parser('check-python-dependencies',
help='Check that all required Python packages are installed.')
check_python_dependencies.add_argument('--no-constraints', action='store_true', default=False,
check_python_dependencies.add_argument('--no-constraints', action='store_true', default=no_constraints_default,
help='Disable constraint settings. Use with care and only when you want '
'to manage package versions by yourself.')
'to manage package versions by yourself. It can be set with the IDF_PYTHON_CHECK_CONSTRAINTS '
'environment variable.')
args = parser.parse_args(argv)