feat: use esp-idf-sbom pre-commit plugin

Currently sbom manifest is checked only in .gitmodules and
this check is done in pre-commit and also in CI. Meaning it's running
three times(pre-commit before push if user has it enabled, in CI
as there is the pre-commit run again and again with test in CI). Since
esp-idf-sbom contains a full manifest validation support and pre-commit
plugin for it, let's use it. This removes all the current sbom testing
and replaces it with a signle pre-commit plugin which validates all
manifests files(sbom.yml, idf_component.yml, .gitmodules and also
referenced manifests) in repository. Note that this checks all
manifests, not only ones which were modified. The check is reasonably
fast though, so it should not cause any problem. The reason for
validating all manifest files is that we want to make sure that the sbom
information in .gitmodules is updated too and that the hash
recorded in .gitmodules is up-to-date. Meaning submodule update
would not trigger this plugin, because no manifest was changed.

Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This commit is contained in:
Frantisek Hrbata 2023-12-05 10:06:05 +01:00
parent ce2102eed4
commit 22637ff824
4 changed files with 4 additions and 92 deletions

View File

@ -281,13 +281,6 @@ test_mkuf2:
- cd ${IDF_PATH}/tools/test_mkuf2
- ./test_mkuf2.py
test_sbom:
extends:
- .host_test_template
- .rules:patterns:sbom
script:
- python ${IDF_PATH}/tools/test_sbom/test_submodules.py
test_autocomplete:
extends: .host_test_template
image: $CI_DOCKER_REGISTRY/linux-shells:1

View File

@ -48,9 +48,6 @@
- "tools/tools.json"
- "tools/ci/test_build_system*.sh"
.patterns-sbom: &patterns-sbom
- "tools/test_sbom/*"
.patterns-custom_test: &patterns-custom_test
- "tools/ci/python_packages/gitlab_api.py"
- "tools/ci/python_packages/tiny_test_fw/**/*"
@ -257,14 +254,6 @@
- <<: *if-dev-push
changes: *patterns-sonarqube-files
.rules:patterns:sbom:
rules:
- <<: *if-protected
- <<: *if-dev-push
changes: *patterns-sbom
- <<: *if-dev-push
changes: *patterns-submodule
# !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
# DO NOT place comments or maintain any code from this line
#

View File

@ -123,3 +123,7 @@ repos:
hooks:
- id: file-contents-sorter
files: 'tools\/ci\/(executable-list\.txt|mypy_ignore_list\.txt)'
- repo: https://github.com/espressif/esp-idf-sbom.git
rev: v0.11.0
hooks:
- id: validate-sbom-manifest

View File

@ -1,74 +0,0 @@
# SPDX-FileCopyrightText: 2023 Espressif Systems (Shanghai) CO LTD
# SPDX-License-Identifier: Apache-2.0
import os
from subprocess import PIPE, run
from typing import Dict, List
def run_cmd(cmd: List[str]) -> str:
"""Simple helper to run command and return it's stdout."""
proc = run(cmd, stdout=PIPE, stderr=PIPE, check=True, universal_newlines=True)
return proc.stdout.strip()
def get_gitwdir() -> str:
"""Return absolute path to the current git working tree."""
return run_cmd(['git', 'rev-parse', '--show-toplevel'])
def get_submodules_config() -> Dict[str,Dict[str,str]]:
"""Return dictionary, where key is submodule name and value
is a dictionary with variable:value pairs."""
gitmodules_fn = os.path.join(get_gitwdir(), '.gitmodules')
gitmodules_data = run_cmd(['git', 'config', '--list', '--file', gitmodules_fn])
prefix = 'submodule.'
config: Dict[str, Dict[str,str]] = {}
for line in gitmodules_data.splitlines():
if not line.startswith(prefix):
continue
splitted = line.split('=', maxsplit=1)
if len(splitted) != 2:
continue
section, val = splitted
# remove "submodule." prefix
section = section[len(prefix):]
# split section into module name and variable
splitted = section.rsplit('.', maxsplit=1)
if len(splitted) != 2:
continue
module_name, var = splitted
if module_name not in config:
config[module_name] = {}
config[module_name][var] = val
return config
def test_sha() -> None:
""" Check that submodule SHA in git-tree and .gitmodules match
if sbom-hash variable is available in the .gitmodules file.
"""
submodules = get_submodules_config()
for name, variables in submodules.items():
sbom_hash = variables.get('sbom-hash')
if not sbom_hash:
continue
module_path = variables.get('path')
if not module_path:
continue
output = run_cmd(['git', 'ls-tree', 'HEAD', module_path])
if not output:
continue
module_hash = output.split()[2]
msg = (f'Submodule \"{name}\" SHA \"{module_hash}\" in git '
f'tree does not match SHA \"{sbom_hash}\" recorded in .gitmodules. '
f'Please update \"sbom-hash\" in .gitmodules for \"{name}\" '
f'and also please do not forget to update version and other submodule '
f'information if necessary. It is important to keep this information '
f'up-to-date for SBOM generation.')
assert module_hash == sbom_hash, msg
if __name__ == '__main__':
test_sha()