2020-04-22 02:49:05 -04:00
|
|
|
import logging
|
tools: add {find,build}_apps.py, scripts to build multiple apps
This commit adds a pair of scripts, find_apps.py and build_apps.py.
These scripts are intended to be used in various CI jobs, building
multiple applications with different configurations and targets.
The first script, find_apps.py, is used to prepare the list of builds:
1. It finds apps for the given build system.
2. For each app, it finds configurations (sdkconfig files) which need
to be built.
3. It filters out the apps and configurations which are not compatible
with the given target.
4. It outputs the list of builds into stdout or a file. Currently the
format is a list of lines, each line a JSON string. In the future,
the tool can be updated to output YAML files.
The lists of builds can be concatenated and processed with standard
command line tools, like sed.
The second script, build_apps.py, executes the builds from the list.
It can execute a subset of builds based on --parallel-count and
--parallel-index arguments.
These two scripts are intended to replace build_examples_make,
build_examples_cmake, and the custom unit-test-app logic (in the
Makefile and idf_ext.py).
Closes IDF-641
2019-09-11 05:23:10 -04:00
|
|
|
import os
|
2020-06-19 06:58:03 -04:00
|
|
|
import shlex
|
2020-04-22 02:49:05 -04:00
|
|
|
import subprocess
|
|
|
|
import sys
|
|
|
|
|
2021-01-25 21:49:01 -05:00
|
|
|
from .common import BuildError, BuildSystem
|
tools: add {find,build}_apps.py, scripts to build multiple apps
This commit adds a pair of scripts, find_apps.py and build_apps.py.
These scripts are intended to be used in various CI jobs, building
multiple applications with different configurations and targets.
The first script, find_apps.py, is used to prepare the list of builds:
1. It finds apps for the given build system.
2. For each app, it finds configurations (sdkconfig files) which need
to be built.
3. It filters out the apps and configurations which are not compatible
with the given target.
4. It outputs the list of builds into stdout or a file. Currently the
format is a list of lines, each line a JSON string. In the future,
the tool can be updated to output YAML files.
The lists of builds can be concatenated and processed with standard
command line tools, like sed.
The second script, build_apps.py, executes the builds from the list.
It can execute a subset of builds based on --parallel-count and
--parallel-index arguments.
These two scripts are intended to replace build_examples_make,
build_examples_cmake, and the custom unit-test-app logic (in the
Makefile and idf_ext.py).
Closes IDF-641
2019-09-11 05:23:10 -04:00
|
|
|
|
|
|
|
# Same for the Makefile projects:
|
2021-01-25 21:49:01 -05:00
|
|
|
MAKE_PROJECT_LINE = r'include $(IDF_PATH)/make/project.mk'
|
tools: add {find,build}_apps.py, scripts to build multiple apps
This commit adds a pair of scripts, find_apps.py and build_apps.py.
These scripts are intended to be used in various CI jobs, building
multiple applications with different configurations and targets.
The first script, find_apps.py, is used to prepare the list of builds:
1. It finds apps for the given build system.
2. For each app, it finds configurations (sdkconfig files) which need
to be built.
3. It filters out the apps and configurations which are not compatible
with the given target.
4. It outputs the list of builds into stdout or a file. Currently the
format is a list of lines, each line a JSON string. In the future,
the tool can be updated to output YAML files.
The lists of builds can be concatenated and processed with standard
command line tools, like sed.
The second script, build_apps.py, executes the builds from the list.
It can execute a subset of builds based on --parallel-count and
--parallel-index arguments.
These two scripts are intended to replace build_examples_make,
build_examples_cmake, and the custom unit-test-app logic (in the
Makefile and idf_ext.py).
Closes IDF-641
2019-09-11 05:23:10 -04:00
|
|
|
|
2021-01-25 21:49:01 -05:00
|
|
|
BUILD_SYSTEM_MAKE = 'make'
|
tools: add {find,build}_apps.py, scripts to build multiple apps
This commit adds a pair of scripts, find_apps.py and build_apps.py.
These scripts are intended to be used in various CI jobs, building
multiple applications with different configurations and targets.
The first script, find_apps.py, is used to prepare the list of builds:
1. It finds apps for the given build system.
2. For each app, it finds configurations (sdkconfig files) which need
to be built.
3. It filters out the apps and configurations which are not compatible
with the given target.
4. It outputs the list of builds into stdout or a file. Currently the
format is a list of lines, each line a JSON string. In the future,
the tool can be updated to output YAML files.
The lists of builds can be concatenated and processed with standard
command line tools, like sed.
The second script, build_apps.py, executes the builds from the list.
It can execute a subset of builds based on --parallel-count and
--parallel-index arguments.
These two scripts are intended to replace build_examples_make,
build_examples_cmake, and the custom unit-test-app logic (in the
Makefile and idf_ext.py).
Closes IDF-641
2019-09-11 05:23:10 -04:00
|
|
|
|
2020-04-22 05:05:59 -04:00
|
|
|
try:
|
2021-06-16 19:21:36 -04:00
|
|
|
string_type = basestring # type: ignore
|
2020-04-22 05:05:59 -04:00
|
|
|
except NameError:
|
|
|
|
string_type = str
|
|
|
|
|
tools: add {find,build}_apps.py, scripts to build multiple apps
This commit adds a pair of scripts, find_apps.py and build_apps.py.
These scripts are intended to be used in various CI jobs, building
multiple applications with different configurations and targets.
The first script, find_apps.py, is used to prepare the list of builds:
1. It finds apps for the given build system.
2. For each app, it finds configurations (sdkconfig files) which need
to be built.
3. It filters out the apps and configurations which are not compatible
with the given target.
4. It outputs the list of builds into stdout or a file. Currently the
format is a list of lines, each line a JSON string. In the future,
the tool can be updated to output YAML files.
The lists of builds can be concatenated and processed with standard
command line tools, like sed.
The second script, build_apps.py, executes the builds from the list.
It can execute a subset of builds based on --parallel-count and
--parallel-index arguments.
These two scripts are intended to replace build_examples_make,
build_examples_cmake, and the custom unit-test-app logic (in the
Makefile and idf_ext.py).
Closes IDF-641
2019-09-11 05:23:10 -04:00
|
|
|
|
|
|
|
class MakeBuildSystem(BuildSystem):
|
|
|
|
NAME = BUILD_SYSTEM_MAKE
|
|
|
|
|
2020-05-05 22:40:23 -04:00
|
|
|
@classmethod
|
|
|
|
def build(cls, build_item):
|
|
|
|
build_path, work_path = cls.build_prepare(build_item)
|
2020-04-22 02:49:05 -04:00
|
|
|
commands = [
|
|
|
|
'make clean',
|
|
|
|
'make defconfig',
|
|
|
|
'make all',
|
2021-06-16 19:21:36 -04:00
|
|
|
# In case if secure_boot is enabled then for bootloader build need to add `bootloader` cmd
|
|
|
|
'make bootloader',
|
2020-04-22 02:49:05 -04:00
|
|
|
'make print_flash_cmd',
|
|
|
|
]
|
|
|
|
|
|
|
|
log_file = None
|
|
|
|
build_stdout = sys.stdout
|
|
|
|
build_stderr = sys.stderr
|
|
|
|
if build_item.build_log_path:
|
2021-01-25 21:49:01 -05:00
|
|
|
logging.info('Writing build log to {}'.format(build_item.build_log_path))
|
|
|
|
log_file = open(build_item.build_log_path, 'w')
|
2020-04-22 02:49:05 -04:00
|
|
|
build_stdout = log_file
|
|
|
|
build_stderr = log_file
|
|
|
|
|
|
|
|
for cmd in commands:
|
|
|
|
cmd = shlex.split(cmd) if isinstance(cmd, string_type) else cmd
|
|
|
|
try:
|
|
|
|
subprocess.check_call(cmd, stdout=build_stdout, stderr=build_stderr, cwd=work_path)
|
|
|
|
except subprocess.CalledProcessError as e:
|
|
|
|
if log_file:
|
|
|
|
log_file.close()
|
2021-01-25 21:49:01 -05:00
|
|
|
raise BuildError('Build failed with exit code {}'.format(e.returncode))
|
tools: add {find,build}_apps.py, scripts to build multiple apps
This commit adds a pair of scripts, find_apps.py and build_apps.py.
These scripts are intended to be used in various CI jobs, building
multiple applications with different configurations and targets.
The first script, find_apps.py, is used to prepare the list of builds:
1. It finds apps for the given build system.
2. For each app, it finds configurations (sdkconfig files) which need
to be built.
3. It filters out the apps and configurations which are not compatible
with the given target.
4. It outputs the list of builds into stdout or a file. Currently the
format is a list of lines, each line a JSON string. In the future,
the tool can be updated to output YAML files.
The lists of builds can be concatenated and processed with standard
command line tools, like sed.
The second script, build_apps.py, executes the builds from the list.
It can execute a subset of builds based on --parallel-count and
--parallel-index arguments.
These two scripts are intended to replace build_examples_make,
build_examples_cmake, and the custom unit-test-app logic (in the
Makefile and idf_ext.py).
Closes IDF-641
2019-09-11 05:23:10 -04:00
|
|
|
|
2020-07-21 04:00:05 -04:00
|
|
|
build_item.size_json_fp = build_item.get_size_json_fp()
|
|
|
|
|
tools: add {find,build}_apps.py, scripts to build multiple apps
This commit adds a pair of scripts, find_apps.py and build_apps.py.
These scripts are intended to be used in various CI jobs, building
multiple applications with different configurations and targets.
The first script, find_apps.py, is used to prepare the list of builds:
1. It finds apps for the given build system.
2. For each app, it finds configurations (sdkconfig files) which need
to be built.
3. It filters out the apps and configurations which are not compatible
with the given target.
4. It outputs the list of builds into stdout or a file. Currently the
format is a list of lines, each line a JSON string. In the future,
the tool can be updated to output YAML files.
The lists of builds can be concatenated and processed with standard
command line tools, like sed.
The second script, build_apps.py, executes the builds from the list.
It can execute a subset of builds based on --parallel-count and
--parallel-index arguments.
These two scripts are intended to replace build_examples_make,
build_examples_cmake, and the custom unit-test-app logic (in the
Makefile and idf_ext.py).
Closes IDF-641
2019-09-11 05:23:10 -04:00
|
|
|
@staticmethod
|
|
|
|
def is_app(path):
|
2021-01-25 21:49:01 -05:00
|
|
|
makefile_path = os.path.join(path, 'Makefile')
|
tools: add {find,build}_apps.py, scripts to build multiple apps
This commit adds a pair of scripts, find_apps.py and build_apps.py.
These scripts are intended to be used in various CI jobs, building
multiple applications with different configurations and targets.
The first script, find_apps.py, is used to prepare the list of builds:
1. It finds apps for the given build system.
2. For each app, it finds configurations (sdkconfig files) which need
to be built.
3. It filters out the apps and configurations which are not compatible
with the given target.
4. It outputs the list of builds into stdout or a file. Currently the
format is a list of lines, each line a JSON string. In the future,
the tool can be updated to output YAML files.
The lists of builds can be concatenated and processed with standard
command line tools, like sed.
The second script, build_apps.py, executes the builds from the list.
It can execute a subset of builds based on --parallel-count and
--parallel-index arguments.
These two scripts are intended to replace build_examples_make,
build_examples_cmake, and the custom unit-test-app logic (in the
Makefile and idf_ext.py).
Closes IDF-641
2019-09-11 05:23:10 -04:00
|
|
|
if not os.path.exists(makefile_path):
|
|
|
|
return False
|
2021-01-25 21:49:01 -05:00
|
|
|
with open(makefile_path, 'r') as makefile:
|
tools: add {find,build}_apps.py, scripts to build multiple apps
This commit adds a pair of scripts, find_apps.py and build_apps.py.
These scripts are intended to be used in various CI jobs, building
multiple applications with different configurations and targets.
The first script, find_apps.py, is used to prepare the list of builds:
1. It finds apps for the given build system.
2. For each app, it finds configurations (sdkconfig files) which need
to be built.
3. It filters out the apps and configurations which are not compatible
with the given target.
4. It outputs the list of builds into stdout or a file. Currently the
format is a list of lines, each line a JSON string. In the future,
the tool can be updated to output YAML files.
The lists of builds can be concatenated and processed with standard
command line tools, like sed.
The second script, build_apps.py, executes the builds from the list.
It can execute a subset of builds based on --parallel-count and
--parallel-index arguments.
These two scripts are intended to replace build_examples_make,
build_examples_cmake, and the custom unit-test-app logic (in the
Makefile and idf_ext.py).
Closes IDF-641
2019-09-11 05:23:10 -04:00
|
|
|
makefile_content = makefile.read()
|
|
|
|
if MAKE_PROJECT_LINE not in makefile_content:
|
|
|
|
return False
|
|
|
|
return True
|
2020-06-19 06:58:03 -04:00
|
|
|
|
2020-12-07 22:54:17 -05:00
|
|
|
@classmethod
|
|
|
|
def supported_targets(cls, app_path):
|
|
|
|
readme_supported_targets = cls._supported_targets(app_path)
|
|
|
|
if readme_supported_targets and 'esp32' in readme_supported_targets:
|
|
|
|
return ['esp32']
|
|
|
|
else:
|
|
|
|
return []
|