mirror of
https://github.com/espressif/esp-idf.git
synced 2024-10-05 20:47:46 -04:00
Merge branch 'ag_make_fixes' into 'master'
This commit is contained in:
Angus Gratton
commit
1643bab9c5
134
README.md
134
README.md
@ -8,27 +8,30 @@
|
||||
|
||||
# Compiling your project
|
||||
|
||||
`make app`
|
||||
`make all`
|
||||
|
||||
# Flashing the Bootloader
|
||||
... will compile app, bootloader and generate a partition table based on the config.
|
||||
|
||||
ESP32 has a bootloader in ROM which runs after reset, but ESP-IDF also uses a second stage software bootloader. The ROM bootloader loads the software bootloader, which then loads the firmware app of the ESP32. The software bootloader must be flashed to offset 0x5000 in the flash.
|
||||
# Flashing your project
|
||||
|
||||
To build the software bootloader, navigate to your project's top-level directory and run:
|
||||
When `make all` finishes, it will print a command line to use esptool.py to flash the chip. However you can also do this from make by running:
|
||||
|
||||
``` shell
|
||||
make bootloader
|
||||
```
|
||||
`make flash`
|
||||
|
||||
If you've configured the serial port details in `make menuconfig`, then
|
||||
This will flash the entire project (app, bootloader and partition table) to a new chip. The settings for serial port flashing can be configured with `make menuconfig`.
|
||||
|
||||
``` shell
|
||||
make bootloader-flash
|
||||
```
|
||||
You don't need to run `make all` before running `make flash`, `make flash` will automatically rebuild anything which needs it.
|
||||
|
||||
... will automatically run esptool.py to flash the image. Otherwise, you can customise the `esptool.py` command that is printed out as part of `make bootloader`.
|
||||
# Compiling & Flashing Just the App
|
||||
|
||||
You only need to flash the ESP32 bootloader once.
|
||||
After the initial flash, you may just want to build and flash just your app, not the bootloader and partition table:
|
||||
|
||||
* `make app` - build just the app.
|
||||
* `make app-flash` - flash just the app.
|
||||
|
||||
`make app-flash` will automatically rebuild the app if it needs it.
|
||||
|
||||
(There's no downside to reflashing the bootloader and partition table each time, if they haven't changed.)
|
||||
|
||||
# The Partition Table
|
||||
|
||||
@ -45,107 +48,4 @@ The simplest way to use the partition table is to `make menuconfig` and choose o
|
||||
|
||||
In both cases the factory app is flashed at offset 0x10000. If you `make partition_table` then it will print a summary of the partition table.
|
||||
|
||||
Here is the summary printed for the "Single factory app, no OTA" configuration:
|
||||
|
||||
```
|
||||
# Espressif ESP32 Partition Table
|
||||
# Name, Type, SubType, Offset, Size
|
||||
factory, app, factory, 0x10000, 1M
|
||||
rfdata, data, rf, 0x110000, 256K
|
||||
wifidata,data, wifi, 0x150000, 256K
|
||||
```
|
||||
|
||||
* At a 0x10000 (64KB) offset in the flash is the app labelled "factory". The bootloader will run this app by default.
|
||||
* There are also two data regions defined in the partition table for storing RF & Wifi calibration data.
|
||||
|
||||
Here is the summary printed for the "Factory app, two OTA definitions" configuration:
|
||||
|
||||
```
|
||||
# Espressif ESP32 Partition Table
|
||||
# Name, Type, SubType, Offset, Size
|
||||
factory, app, factory, 0x10000, 1M
|
||||
ota_0, app, ota_0, 0x110000, 1M
|
||||
ota_1, app, ota_1, 0x210000, 1M
|
||||
rfdata, data, rf, 0x310000, 256K
|
||||
wifidata,data, wifi, 0x350000, 256K
|
||||
otadata, data, ota, 0x390000, 256K
|
||||
```
|
||||
|
||||
* There are now three app partition definitions.
|
||||
* The type of all three are set as "app", but the subtype varies between the factory app at 0x10000 and the next two "OTA" apps.
|
||||
* There is also a new "ota data" slot, which holds the data for OTA updates. The bootloader consults this data in order to know which app to execute. If "ota data" is empty, it will execute the factory app.
|
||||
|
||||
|
||||
## Creating Custom Tables
|
||||
|
||||
If you choose "Custom partition table CSV" in menuconfig then you can also enter the name of a CSV file (in the project directory) to use for your partition table. The CSV file can describe any number of definitions for the table you need.
|
||||
|
||||
The CSV format is the same format as printed in the summaries shown above. However, not all fields are required in the CSV. For example, here is the "input" CSV for the OTA partition table:
|
||||
|
||||
```
|
||||
# Name, Type, SubType, Offset, Size
|
||||
factory, app, factory, 0x10000, 1M
|
||||
ota_0, app, ota_0, , 1M
|
||||
ota_1, app, ota_1, , 1M
|
||||
rfdata, data, rf, , 256K
|
||||
wifidata, data, wifi, , 256K
|
||||
otadata, data, ota, , 256K
|
||||
```
|
||||
|
||||
* Whitespace between fields is ignored, and so is any line starting with # (comments).
|
||||
* Each non-comment line in the CSV file is a partition definition.
|
||||
* Only the offset for the first partition is supplied. The gen_esp32part.py tool fills in each remaining offset to start after the preceding partition.
|
||||
|
||||
### Name field
|
||||
|
||||
Name field can be any meaningful name. It is not significant to the ESP32. Names longer than 16 characters will be truncated.
|
||||
|
||||
### Type field
|
||||
|
||||
Type field can be specified as app (0) or data (1). Or it can be a number 0-254 (or as hex 0x00-0xFE). Types 0x00-0x3F are reserved for Espressif. If your application needs to store data, please add a custom partition type in the range 0x40-0xFE.
|
||||
|
||||
The bootloader ignores any types other than 0 & 1.
|
||||
|
||||
### Subtype
|
||||
|
||||
When type is "app", the subtype field can be specified as factory (0), ota_0 (0x10) ... ota_15 (0x1F) and test (0x20). Or it can be any number 0-255 (0x00-0xFF). The bootloader will execute the factory app unless there it sees a partition of type data/ota, in which case it reads this partition to determine which OTA image to boot
|
||||
|
||||
When type is "data", the subtype field can be specified as ota (0), rf (1), wifi (2). Or it can be a number 0x00-0xFF. The bootloader ignores all data subtypes except for ota. Other "data" subtypes are reserved for Espressif use. To create custom data partition subtypes then use a custom type value, and choose any subtype 0x00-0xFF.
|
||||
|
||||
### Offset & Size
|
||||
|
||||
Only the first offset field is required (we recommend using 0x10000). Partitions with blank offsets will start after the previous partition.
|
||||
|
||||
App partitions have to be at offsets aligned to 0x10000 (64K). If you leave the offset field blank, the tool will automatically align the partition. If you specify an unaligned offset for an app partition, the tool will return an error.
|
||||
|
||||
Sizes and offsets can be specified as decimal numbers, hex numbers with the prefix 0x, or size multipliers M or K (1024 and 1024*1024 bytes).
|
||||
|
||||
## Generating Binary Partition Table
|
||||
|
||||
The partition table which is flashed to the ESP32 is in a binary format, not CSV. The tool components/partition_table/gen_esp32part.py is used to convert between CSV and binary formats.
|
||||
|
||||
If you configure the partition table CSV name in `make menuconfig` and then `make partition_table`, this conversion is done for you.
|
||||
|
||||
To convert CSV to Binary manually:
|
||||
|
||||
``` shell
|
||||
python components/partition_table/gen_esp32part.py --verify input_partitions.csv binary_partitions.bin
|
||||
```
|
||||
|
||||
To convert binary format back to CSV:
|
||||
|
||||
``` shell
|
||||
python components/partition_table/gen_esp32part.py --verify binary_partitions.bin input_partitions.csv
|
||||
```
|
||||
|
||||
To display the contents of a binary partition table on stdout (this is how the summaries displayed when running `make partition_table` are generated:
|
||||
|
||||
``` shell
|
||||
python components/partition_table/gen_esp32part.py binary_partitions.bin
|
||||
```
|
||||
|
||||
`gen_esp32part.py` takes one optional argument, `--verify`, which will also verify the partition table during conversion (checking for overlapping partitions, unaligned partitions, etc.)
|
||||
|
||||
# Flashing the partition table
|
||||
|
||||
The command `make partition_table-flash` will flash the partition table with esptool.py. However a manual flashing command is printed as part of `make partition_table`.
|
||||
For more details about partition tables and how to create custom variations, view the `docs/partition_tables.rst` file.
|
||||
|
||||
@ -10,30 +10,34 @@
|
||||
#
|
||||
|
||||
BOOTLOADER_COMPONENT_PATH := $(COMPONENT_PATH)
|
||||
EXTRA_CLEAN_TARGETS+=bootloader-clean
|
||||
BOOTLOADER_BIN=$(BUILD_DIR_BASE)/bootloader.bin
|
||||
BOOTLOADER_BUILD_DIR=$(BUILD_DIR_BASE)/bootloader
|
||||
BOOTLOADER_BIN=$(BOOTLOADER_BUILD_DIR)/bootloader.bin
|
||||
|
||||
.PHONY: bootloader-clean bootloader-flash bootloader
|
||||
|
||||
$(BOOTLOADER_BIN): $(COMPONENT_PATH)/src/sdkconfig
|
||||
$(Q) PROJECT_PATH= \
|
||||
COMPONENT_LDFLAGS= \
|
||||
COMPONENT_INCLUDES= \
|
||||
LDFLAGS= \
|
||||
CFLAGS= \
|
||||
BUILD_DIR_BASE=$(BUILD_DIR_BASE)/bootloader \
|
||||
make -C $(BOOTLOADER_COMPONENT_PATH)/src MAKEFLAGS= V=$(V) TARGET_BIN_LAYOUT="$(BOOTLOADER_TARGET_BIN_LAYOUT)"
|
||||
BUILD_DIR_BASE=$(BOOTLOADER_BUILD_DIR) \
|
||||
$(MAKE) -C $(BOOTLOADER_COMPONENT_PATH)/src MAKEFLAGS= V=$(V) TARGET_BIN_LAYOUT="$(BOOTLOADER_TARGET_BIN_LAYOUT)" $(BOOTLOADER_BIN)
|
||||
|
||||
bootloader-clean:
|
||||
$(Q) PROJECT_PATH= \
|
||||
COMPONENT_LDFLAGS= \
|
||||
COMPONENT_INCLUDES= \
|
||||
LDFLAGS= \
|
||||
CFLAGS= \
|
||||
BUILD_DIR_BASE=$(BUILD_DIR_BASE)/bootloader \
|
||||
make -C $(BOOTLOADER_COMPONENT_PATH)/src clean MAKEFLAGS= V=$(V)
|
||||
BUILD_DIR_BASE=$(BOOTLOADER_BUILD_DIR) \
|
||||
$(MAKE) -C $(BOOTLOADER_COMPONENT_PATH)/src app-clean MAKEFLAGS= V=$(V)
|
||||
|
||||
bootloader: $(BUILD_DIR_BASE)/bootloader.bin
|
||||
clean: bootloader-clean
|
||||
|
||||
bootloader: $(BOOTLOADER_BIN)
|
||||
@echo "Bootloader built. Default flash command is:"
|
||||
@echo "$(ESPTOOLPY_SERIAL) write_flash 0x1000 $(BOOTLOADER_BIN)"
|
||||
|
||||
all_binaries: $(BOOTLOADER_BIN)
|
||||
|
||||
ESPTOOL_ALL_FLASH_ARGS += 0x1000 $(BOOTLOADER_BIN)
|
||||
|
||||
# synchronise the project level config to the component's
|
||||
# config
|
||||
@ -42,4 +46,4 @@ $(COMPONENT_PATH)/src/sdkconfig: $(PROJECT_PATH)/sdkconfig
|
||||
|
||||
# bootloader-flash calls flash in the bootloader dummy project
|
||||
bootloader-flash: $(BOOTLOADER_BIN)
|
||||
make -C $(BOOTLOADER_COMPONENT_PATH)/src flash MAKEFLAGS= V=$(V) CONFIG_APP_OFFSET=0x1000
|
||||
$(MAKE) -C $(BOOTLOADER_COMPONENT_PATH)/src flash MAKEFLAGS= V=$(V)
|
||||
|
||||
@ -12,14 +12,15 @@ PYTHON ?= $(call dequote,$(CONFIG_PYTHON))
|
||||
ESPTOOLPY := $(PYTHON) $(SDK_PATH)/bin/esptool.py --chip esp32
|
||||
ESPTOOLPY_SERIAL := $(ESPTOOLPY) --port $(ESPPORT) --baud $(ESPBAUD)
|
||||
|
||||
PROJECT_FLASH_COMMAND=$(ESPTOOLPY_SERIAL) write_flash $(CONFIG_APP_OFFSET) $(PROJECT_BIN)
|
||||
APP_FLASH_COMMAND=$(ESPTOOLPY_SERIAL) write_flash $(CONFIG_APP_OFFSET) $(APP_BIN)
|
||||
ESPTOOL_ALL_FLASH_ARGS += $(CONFIG_APP_OFFSET) $(APP_BIN)
|
||||
|
||||
$(PROJECT_BIN): $(PROJECT_ELF)
|
||||
$(APP_BIN): $(APP_ELF)
|
||||
$(Q) $(ESPTOOLPY) elf2image -o $@ $<
|
||||
|
||||
flash: $(PROJECT_BIN)
|
||||
flash: all_binaries
|
||||
@echo "Flashing project app to $(CONFIG_APP_OFFSET)..."
|
||||
$(Q) $(PROJECT_FLASH_COMMAND)
|
||||
$(Q) $(ESPTOOLPY_SERIAL) write_flash $(ESPTOOL_ALL_FLASH_ARGS)
|
||||
|
||||
# convenience target to flash bootloader, partitions, app all at once
|
||||
flash_all: bootloader-flash partition_table-flash flash
|
||||
app-flash: $(APP_BIN)
|
||||
$(Q) $(APP_FLASH_COMMAND)
|
||||
|
||||
@ -8,8 +8,6 @@
|
||||
#
|
||||
.PHONY: partition_table partition_table-flash partition_table-clean
|
||||
|
||||
EXTRA_CLEAN_TARGETS+=partition_table-clean
|
||||
|
||||
# NB: gen_esp32part.py lives in the sdk/bin/ dir not component dir
|
||||
GEN_ESP32PART := $(PYTHON) $(SDK_PATH)/bin/gen_esp32part.py -q
|
||||
|
||||
@ -24,7 +22,10 @@ $(PARTITION_TABLE_BIN): $(PARTITION_TABLE_CSV_PATH)
|
||||
@echo "Building partitions from $(PARTITION_TABLE_CSV_PATH)..."
|
||||
$(Q) $(GEN_ESP32PART) $< $@
|
||||
|
||||
all_binaries: $(PARTITION_TABLE_BIN)
|
||||
|
||||
PARTITION_TABLE_FLASH_CMD = $(ESPTOOLPY_SERIAL) write_flash 0x4000 $(PARTITION_TABLE_BIN)
|
||||
ESPTOOL_ALL_FLASH_ARGS += 0x4000 $(PARTITION_TABLE_BIN)
|
||||
|
||||
partition_table: $(PARTITION_TABLE_BIN)
|
||||
@echo "Partition table binary generated. Contents:"
|
||||
@ -40,3 +41,5 @@ partition_table-flash: $(PARTITION_TABLE_BIN)
|
||||
|
||||
partition_table-clean:
|
||||
$(Q) rm -f $(PARTITION_TABLE_BIN)
|
||||
|
||||
clean: partition_table-clean
|
||||
|
||||
@ -21,7 +21,7 @@ Import New Project
|
||||
|
||||
* Once Eclipse is running, choose File -> Import...
|
||||
|
||||
* In the dialog that pops up, choose "C/C++" -> "EXisting Code as Makefile Project" and click Next.
|
||||
* In the dialog that pops up, choose "C/C++" -> "Existing Code as Makefile Project" and click Next.
|
||||
|
||||
* On the next page, enter "Existing Code Location" to be the directory of your SDK project. Don't specify the path to the SDK itself.
|
||||
|
||||
@ -39,7 +39,7 @@ Project Properties
|
||||
|
||||
*Windows users only, follow these two additional steps:*
|
||||
|
||||
* On the same Environment property page, edit the PATH environment variable and append ";C:\msys32\usr\bin;C:\msys32\mingw32\bin;C:\msys32\opt\xtensa-esp32-elf\bin" to the end of the default value. (If you installed msys32 to a different directory then you'll need to change these paths to match.)
|
||||
* On the same Environment property page, edit the PATH environment variable and append ``;C:\msys32\usr\bin;C:\msys32\mingw32\bin;C:\msys32\opt\xtensa-esp32-elf\bin`` to the end of the default value. (If you installed msys32 to a different directory then you'll need to change these paths to match.)
|
||||
|
||||
* Click on the "C/C++ Build" top-level properties page then uncheck "Use default build command" and enter this for the custom build command: ``bash ${SDK_PATH}/bin/eclipse_windows_make.sh``.
|
||||
|
||||
@ -60,7 +60,7 @@ Flash from Eclipse
|
||||
|
||||
You can integrate the "make flash" target into your Eclipse project to flash using esptool.py from the Eclipse UI:
|
||||
|
||||
* Right-click the "esp-idf-tests" project in Project Explorer (important to make sure you don't select a subdirectory of the project or Eclipse may find the wrong Makefile.)
|
||||
* Right-click your project in Project Explorer (important to make sure you don't select a subdirectory of the project or Eclipse may find the wrong Makefile.)
|
||||
|
||||
* Select Make Targets -> Create from the context menu.
|
||||
|
||||
|
||||
119
docs/partition-tables.rst
Normal file
119
docs/partition-tables.rst
Normal file
@ -0,0 +1,119 @@
|
||||
Partition Tables
|
||||
----------------
|
||||
|
||||
A single ESP32's flash can contain multiple apps, as well as many different kinds of data (calibration data, filesystems, parameter storage, etc). For this reason a partition table is flashed to offset 0x4000 in the flash.
|
||||
|
||||
Each entry in the partition table has a name (label), type (app, data, or something else), subtype and the offset in flash where the partition is loaded.
|
||||
|
||||
The simplest way to use the partition table is to `make menuconfig` and choose one of the simple predefined partition tables:
|
||||
|
||||
* "Single factory app, no OTA"
|
||||
* "Factory app, two OTA definitions"
|
||||
|
||||
In both cases the factory app is flashed at offset 0x10000. If you `make partition_table` then it will print a summary of the partition table.
|
||||
|
||||
Built-in Partition Tables
|
||||
-------------------------
|
||||
|
||||
Here is the summary printed for the "Single factory app, no OTA" configuration::
|
||||
|
||||
# Espressif ESP32 Partition Table
|
||||
# Name, Type, SubType, Offset, Size
|
||||
factory, app, factory, 0x10000, 1M
|
||||
rfdata, data, rf, 0x110000, 256K
|
||||
wifidata,data, wifi, 0x150000, 256K
|
||||
|
||||
* At a 0x10000 (64KB) offset in the flash is the app labelled "factory". The bootloader will run this app by default.
|
||||
* There are also two data regions defined in the partition table for storing RF & Wifi calibration data.
|
||||
|
||||
Here is the summary printed for the "Factory app, two OTA definitions" configuration::
|
||||
|
||||
# Espressif ESP32 Partition Table
|
||||
# Name, Type, SubType, Offset, Size
|
||||
factory, app, factory, 0x10000, 1M
|
||||
ota_0, app, ota_0, 0x110000, 1M
|
||||
ota_1, app, ota_1, 0x210000, 1M
|
||||
rfdata, data, rf, 0x310000, 256K
|
||||
wifidata,data, wifi, 0x350000, 256K
|
||||
otadata, data, ota, 0x390000, 256K
|
||||
|
||||
* There are now three app partition definitions.
|
||||
* The type of all three are set as "app", but the subtype varies between the factory app at 0x10000 and the next two "OTA" apps.
|
||||
* There is also a new "ota data" slot, which holds the data for OTA updates. The bootloader consults this data in order to know which app to execute. If "ota data" is empty, it will execute the factory app.
|
||||
|
||||
|
||||
Creating Custom Tables
|
||||
----------------------
|
||||
|
||||
If you choose "Custom partition table CSV" in menuconfig then you can also enter the name of a CSV file (in the project directory) to use for your partition table. The CSV file can describe any number of definitions for the table you need.
|
||||
|
||||
The CSV format is the same format as printed in the summaries shown above. However, not all fields are required in the CSV. For example, here is the "input" CSV for the OTA partition table::
|
||||
|
||||
# Name, Type, SubType, Offset, Size
|
||||
factory, app, factory, 0x10000, 1M
|
||||
ota_0, app, ota_0, , 1M
|
||||
ota_1, app, ota_1, , 1M
|
||||
rfdata, data, rf, , 256K
|
||||
wifidata, data, wifi, , 256K
|
||||
otadata, data, ota, , 256K
|
||||
|
||||
* Whitespace between fields is ignored, and so is any line starting with # (comments).
|
||||
* Each non-comment line in the CSV file is a partition definition.
|
||||
* Only the offset for the first partition is supplied. The gen_esp32part.py tool fills in each remaining offset to start after the preceding partition.
|
||||
|
||||
Name field
|
||||
~~~~~~~~~~
|
||||
|
||||
Name field can be any meaningful name. It is not significant to the ESP32. Names longer than 16 characters will be truncated.
|
||||
|
||||
Type field
|
||||
~~~~~~~~~~
|
||||
|
||||
Type field can be specified as app (0) or data (1). Or it can be a number 0-254 (or as hex 0x00-0xFE). Types 0x00-0x3F are reserved for Espressif. If your application needs to store data, please add a custom partition type in the range 0x40-0xFE.
|
||||
|
||||
The bootloader ignores any types other than 0 & 1.
|
||||
|
||||
Subtype
|
||||
~~~~~~~
|
||||
|
||||
When type is "app", the subtype field can be specified as factory (0), ota_0 (0x10) ... ota_15 (0x1F) and test (0x20). Or it can be any number 0-255 (0x00-0xFF). The bootloader will execute the factory app unless there it sees a partition of type data/ota, in which case it reads this partition to determine which OTA image to boot
|
||||
|
||||
When type is "data", the subtype field can be specified as ota (0), rf (1), wifi (2). Or it can be a number 0x00-0xFF. The bootloader ignores all data subtypes except for ota. Other "data" subtypes are reserved for Espressif use. To create custom data partition subtypes then use a custom type value, and choose any subtype 0x00-0xFF.
|
||||
|
||||
Offset & Size
|
||||
~~~~~~~~~~~~~
|
||||
|
||||
Only the first offset field is required (we recommend using 0x10000). Partitions with blank offsets will start after the previous partition.
|
||||
|
||||
App partitions have to be at offsets aligned to 0x10000 (64K). If you leave the offset field blank, the tool will automatically align the partition. If you specify an unaligned offset for an app partition, the tool will return an error.
|
||||
|
||||
Sizes and offsets can be specified as decimal numbers, hex numbers with the prefix 0x, or size multipliers M or K (1024 and 1024*1024 bytes).
|
||||
|
||||
Generating Binary Partition Table
|
||||
---------------------------------
|
||||
|
||||
The partition table which is flashed to the ESP32 is in a binary format, not CSV. The tool bin/gen_esp32part.py is used to convert between CSV and binary formats.
|
||||
|
||||
If you configure the partition table CSV name in ``make menuconfig`` and then ``make partition_table``, this conversion is done for you.
|
||||
|
||||
To convert CSV to Binary manually::
|
||||
|
||||
python bin/gen_esp32part.py --verify input_partitions.csv binary_partitions.bin
|
||||
|
||||
To convert binary format back to CSV::
|
||||
|
||||
python bin/gen_esp32part.py --verify binary_partitions.bin input_partitions.csv
|
||||
|
||||
To display the contents of a binary partition table on stdout (this is how the summaries displayed when running `make partition_table` are generated::
|
||||
|
||||
python bin/gen_esp32part.py binary_partitions.bin
|
||||
|
||||
``gen_esp32part.py`` takes one optional argument, ``--verify``, which will also verify the partition table during conversion (checking for overlapping partitions, unaligned partitions, etc.)
|
||||
|
||||
Flashing the partition table
|
||||
----------------------------
|
||||
|
||||
* ``make partition_table-flash``: will flash the partition table with esptool.py.
|
||||
* ``make flash``: Will flash everything including the partition table.
|
||||
|
||||
A manual flashing command is also printed as part of ``make partition_table``.
|
||||
@ -83,5 +83,6 @@ Use ``cd`` to change to the project directory (not the SDK directory.) Type ``ma
|
||||
If you'd like to use the Eclipse IDE instead of running ``make``, check out the Eclipse setup guide in this directory.
|
||||
|
||||
|
||||
.. _Eclipse: eclipse-setup.rst
|
||||
.. _MSYS2: https://msys2.github.io/
|
||||
.. _github: https://github.com/espressif/esp-idf-template
|
||||
|
||||
@ -10,18 +10,41 @@
|
||||
# Makefile is located.
|
||||
#
|
||||
|
||||
.PHONY: build-components menuconfig all build clean
|
||||
all: project
|
||||
.PHONY: build-components menuconfig all build clean all_binaries
|
||||
all: all_binaries # other components will add dependencies to 'all_binaries'
|
||||
@echo "To flash all build output, run 'make flash' or:"
|
||||
@echo $(ESPTOOLPY_SERIAL) write_flash $(ESPTOOL_ALL_FLASH_ARGS)
|
||||
|
||||
# (the reason all_binaries is used instead of 'all' is so that the flash target
|
||||
# can build everything without triggering the per-component "to flash..."
|
||||
# output targets.)
|
||||
|
||||
help:
|
||||
@echo "Welcome to Espressif IDF build system. Some useful make targets:"
|
||||
@echo ""
|
||||
@echo "make menuconfig - Configure IDF project"
|
||||
@echo ""
|
||||
@echo "make all - Build app, bootloader, partition table"
|
||||
@echo "make flash - Flash all components to a fresh chip"
|
||||
@echo "make clean - Remove all build output"
|
||||
@echo ""
|
||||
@echo "make app - Build just the app"
|
||||
@echo "make app-flash - Flash just the app"
|
||||
@echo "make app-clean - Clean just the app"
|
||||
@echo ""
|
||||
@echo "See also 'make bootloader', 'make bootloader-flash', 'make bootloader-clean', "
|
||||
@echo "'make partition_table', etc, etc."
|
||||
|
||||
# disable built-in make rules, makes debugging saner
|
||||
MAKEFLAGS +=-rR
|
||||
|
||||
# Figure out PROJECT_PATH if not set, check for unacceptable Makefile entry points
|
||||
# Figure out PROJECT_PATH if not set
|
||||
ifeq ("$(PROJECT_PATH)","")
|
||||
#The path to the project: we assume the Makefile including this file resides
|
||||
#in the root of that directory.
|
||||
PROJECT_PATH := $(abspath $(dir $(firstword $(MAKEFILE_LIST))))
|
||||
export PROJECT_PATH
|
||||
endif
|
||||
|
||||
#The directory where we put all objects/libraries/binaries. The project Makefile can
|
||||
#configure this if needed.
|
||||
@ -59,26 +82,26 @@ COMPONENT_PATHS_BUILDABLE := $(foreach cp,$(COMPONENT_PATHS),$(if $(wildcard $(c
|
||||
|
||||
# Assemble global list of include dirs (COMPONENT_INCLUDES), and
|
||||
# LDFLAGS args (COMPONENT_LDFLAGS) supplied by each component.
|
||||
COMPONENT_INCLUDES :=
|
||||
COMPONENT_LDFLAGS :=
|
||||
#
|
||||
# Also add any inter-component dependencies for each component.
|
||||
|
||||
# Extract a variable from a child make process
|
||||
#
|
||||
# $(1) - path to directory to invoke make in
|
||||
# $(2) - name of variable to print via the getvariable target (passed in GET_VARIABLE)
|
||||
# $(2) - name of variable to print via the get_variable target (passed in GET_VARIABLE)
|
||||
#
|
||||
# needs 'sed' processing of stdout because make sometimes echoes other stuff on stdout,
|
||||
# even if asked not to.
|
||||
#
|
||||
# Debugging this? Replace $(shell with $(error and you'll see the full command as-run.
|
||||
define GetVariable
|
||||
$(shell "$(MAKE)" -s --no-print-directory -C $(1) get_variable COMPONENT_INCLUDES=dummy COMPONENT_LDFLAGS=dummy PROJECT_PATH=$(PROJECT_PATH) GET_VARIABLE=$(2) | sed -En "s/^$(2)=(.+)/\1/p" )
|
||||
$(shell "$(MAKE)" -s --no-print-directory -C $(1) get_variable PROJECT_PATH=$(PROJECT_PATH) GET_VARIABLE=$(2) | sed -En "s/^$(2)=(.+)/\1/p" )
|
||||
endef
|
||||
|
||||
ifeq ("$(COMPONENT_INCLUDES)","")
|
||||
COMPONENT_INCLUDES := $(abspath $(foreach comp,$(COMPONENT_PATHS_BUILDABLE),$(addprefix $(comp)/, \
|
||||
$(call GetVariable,$(comp),COMPONENT_ADD_INCLUDEDIRS))))
|
||||
endif
|
||||
|
||||
#Also add project include path, for sdk includes
|
||||
COMPONENT_INCLUDES += $(PROJECT_PATH)/build/include/
|
||||
@ -86,11 +109,9 @@ export COMPONENT_INCLUDES
|
||||
|
||||
#COMPONENT_LDFLAGS has a list of all flags that are needed to link the components together. It's collected
|
||||
#in the same way as COMPONENT_INCLUDES is.
|
||||
ifeq ("$(COMPONENT_LDFLAGS)","")
|
||||
COMPONENT_LDFLAGS := $(foreach comp,$(COMPONENT_PATHS_BUILDABLE), \
|
||||
$(call GetVariable,$(comp),COMPONENT_ADD_LDFLAGS))
|
||||
export COMPONENT_LDFLAGS
|
||||
endif
|
||||
|
||||
# Generate component dependency targets from dependencies lists
|
||||
# each component gains a target of its own <name>-build with dependencies
|
||||
@ -128,8 +149,10 @@ export CC CXX LD AR OBJCOPY
|
||||
|
||||
PYTHON=$(call dequote,$(CONFIG_PYTHON))
|
||||
|
||||
PROJECT_ELF:=$(BUILD_DIR_BASE)/$(PROJECT_NAME).elf
|
||||
PROJECT_BIN:=$(PROJECT_ELF:.elf=.bin)
|
||||
# the app is the main executable built by the project
|
||||
APP_ELF:=$(BUILD_DIR_BASE)/$(PROJECT_NAME).elf
|
||||
APP_MAP:=$(APP_ELF:.elf=.map)
|
||||
APP_BIN:=$(APP_ELF:.elf=.bin)
|
||||
|
||||
# Include any Makefile.projbuild file letting components add
|
||||
# configuration at the project level
|
||||
@ -139,17 +162,21 @@ COMPONENT_PATH := $(1)
|
||||
endef
|
||||
$(foreach componentpath,$(COMPONENT_PATHS),$(eval $(call includeProjBuildMakefile,$(componentpath))))
|
||||
|
||||
# once we know component paths, we can include the config
|
||||
include $(SDK_PATH)/make/project_config.mk
|
||||
|
||||
# ELF depends on the -build target of every component
|
||||
$(PROJECT_ELF): $(addsuffix -build,$(notdir $(COMPONENT_PATHS_BUILDABLE)))
|
||||
$(APP_ELF): $(addsuffix -build,$(notdir $(COMPONENT_PATHS_BUILDABLE)))
|
||||
$(vecho) LD $(notdir $@)
|
||||
$(Q) $(CC) $(LDFLAGS) -o $@ -Wl,-Map=$(patsubst %.elf,%.map,$@)
|
||||
$(Q) $(CC) $(LDFLAGS) -o $@ -Wl,-Map=$(APP_MAP)
|
||||
|
||||
# Generation of $(PROJECT_BIN) from $(PROJECT_ELF) is added by the esptool
|
||||
# Generation of $(APP_BIN) from $(APP_ELF) is added by the esptool
|
||||
# component's Makefile.projbuild
|
||||
|
||||
project: $(PROJECT_BIN)
|
||||
app: $(APP_BIN)
|
||||
@echo "App built. Default flash app command is:"
|
||||
@echo $(PROJECT_FLASH_COMMAND) # PROJECT_FLASH_COMMAND is set in esptool_py's Makefile.projbuild
|
||||
@echo $(APP_FLASH_COMMAND) # APP_FLASH_COMMAND is set in esptool_py's Makefile.projbuild
|
||||
|
||||
all_binaries: $(APP_BIN)
|
||||
|
||||
$(BUILD_DIR_BASE):
|
||||
mkdir -p $(BUILD_DIR_BASE)
|
||||
@ -174,12 +201,10 @@ $(foreach component,$(COMPONENT_PATHS_BUILDABLE),$(eval $(call GenerateComponent
|
||||
$(foreach component,$(COMPONENT_PATHS_BUILDABLE),$(eval $(call GenerateComponentTarget,$(component),build,$(PROJECT_PATH)/build/include/sdkconfig.h)))
|
||||
$(foreach component,$(COMPONENT_PATHS_BUILDABLE),$(eval $(call GenerateComponentTarget,$(component),clean)))
|
||||
|
||||
include $(SDK_PATH)/make/project_config.mk
|
||||
app-clean: $(addsuffix -clean,$(notdir $(COMPONENT_PATHS_BUILDABLE)))
|
||||
$(vecho) RM $(APP_ELF)
|
||||
$(Q) rm -f $(APP_ELF) $(APP_BIN) $(APP_MAP)
|
||||
|
||||
clean: $(addsuffix -clean,$(notdir $(COMPONENT_PATHS_BUILDABLE))) $(EXTRA_CLEAN_TARGETS)
|
||||
$(vecho) RM $(PROJECT_ELF)
|
||||
$(Q) rm -f $(PROJECT_ELF)
|
||||
$(Q) rm -rf $(PROJECT_PATH)/build/include/config $(PROJECT_PATH)/build/include/sdkconfig.h
|
||||
clean: app-clean
|
||||
|
||||
endif
|
||||
|
||||
|
||||
@ -12,6 +12,7 @@ $(KCONFIG_TOOL_DIR)/mconf $(KCONFIG_TOOL_DIR)/conf:
|
||||
MAKEFLAGS="" \
|
||||
CFLAGS="" \
|
||||
LDFLAGS="" \
|
||||
CC=$(HOSTCC) LD=$(HOSTLD) \
|
||||
$(MAKE) -C $(KCONFIG_TOOL_DIR)
|
||||
|
||||
menuconfig: $(KCONFIG_TOOL_DIR)/mconf $(SDK_PATH)/Kconfig $(BUILD_DIR_BASE)
|
||||
@ -30,7 +31,7 @@ endif
|
||||
|
||||
# Work out of whether we have to build the Kconfig makefile
|
||||
# (auto.conf), or if we're in a situation where we don't need it
|
||||
NON_CONFIG_TARGETS := clean %-clean get_variable
|
||||
NON_CONFIG_TARGETS := clean %-clean get_variable help
|
||||
AUTO_CONF_REGEN_TARGET := $(PROJECT_PATH)/build/include/config/auto.conf
|
||||
|
||||
# disable AUTO_CONF_REGEN_TARGET if all targets are non-config targets
|
||||
@ -58,6 +59,7 @@ $(AUTO_CONF_REGEN_TARGET) $(PROJECT_PATH)/build/include/sdkconfig.h: $(PROJECT_P
|
||||
|
||||
clean: config-clean
|
||||
.PHONY: config-clean
|
||||
EXTRA_CLEAN_TARGETS += config-clean
|
||||
config-clean:
|
||||
$(vecho RM CONFIG)
|
||||
$(MAKE) -C $(KCONFIG_TOOL_DIR) clean
|
||||
$(Q) rm -rf $(PROJECT_PATH)/build/include/config $(PROJECT_PATH)/build/include/sdkconfig.h
|
||||
|
||||
@ -1,343 +0,0 @@
|
||||
#!/usr/bin/env python
|
||||
#
|
||||
# ESP32 partition table generation tool
|
||||
#
|
||||
# Takes partition input from a CSV file of this format:
|
||||
#
|
||||
# Name,Type,SubType,Offset,Size
|
||||
#
|
||||
# Any row which starts with # is ignored as a comment. Whitespace at beginning/end of field is ignored.
|
||||
#
|
||||
# EXAMPLE CSV FILE CONTENTS
|
||||
# -------------------------
|
||||
"""
|
||||
# Name, Type, SubType, Offset, Size
|
||||
factory, 0, 0, , 1M
|
||||
ota_0, 0, 0x11, , 1M
|
||||
ota_1, 0, 0x12, , 1M
|
||||
ota_data, 1, 0, 5M, 128K
|
||||
"""
|
||||
#
|
||||
# SAMPLE USAGE
|
||||
# ------------
|
||||
#
|
||||
# Convert CSV layout to binary:
|
||||
# gen_esp32part.py table.csv table.bin
|
||||
#
|
||||
# Convert binary to CSV, print on stdout:
|
||||
# gen_esp32part.py table.bin
|
||||
#
|
||||
# FIELD DETAILS
|
||||
# -------------
|
||||
#
|
||||
# Name: Human-readable name of partition. Values longer than 16 bytes will be truncated.
|
||||
#
|
||||
# Type: Numeric value for type field, or special keywords 'app' (for 0x00) or 'data' (for 0x01).
|
||||
#
|
||||
# SubType: Numeric value for subtype field, or special keywords depending on known types:
|
||||
# app (0x00) : factory, ota_0 through ota_15, test.
|
||||
# data (0x01) : ota, rf, wifi
|
||||
#
|
||||
# Default is zero if left blank.
|
||||
#
|
||||
#
|
||||
# Offset: Start of partition. Examples: 0x800, 0x10000, 512, 2K, 1M
|
||||
# - Offset can be left blank, and gen_esp32part will find the next valid offset (aligned to 64KB for app partitions.)
|
||||
#
|
||||
# Size: Size of partition. Examples: 0x800, 0x10000, 512, 2K, 1M
|
||||
#
|
||||
# SubType: 0
|
||||
#
|
||||
# Offset: End of previous partition, aligned to 64kB if Type = 0 (app) & 4 bytes otherwise.
|
||||
#
|
||||
#
|
||||
#
|
||||
import struct
|
||||
import argparse
|
||||
import sys
|
||||
|
||||
__version__ = '1.0'
|
||||
|
||||
def debug(msg):
|
||||
""" Print to stderr """
|
||||
sys.stderr.write(msg)
|
||||
sys.stderr.write('\n')
|
||||
|
||||
class PartitionTable(list):
|
||||
def __init__(self):
|
||||
super(PartitionTable, self).__init__(self)
|
||||
|
||||
@classmethod
|
||||
def from_csv(cls, csv_contents):
|
||||
res = PartitionTable()
|
||||
lines = csv_contents.split("\n")
|
||||
for line_no in range(len(lines)):
|
||||
line = lines[line_no].strip()
|
||||
if line.startswith("#") or len(line) == 0:
|
||||
continue
|
||||
try:
|
||||
res.append(PartitionDefinition.from_csv(line))
|
||||
except InputError as e:
|
||||
raise InputError("Error at line %d: %s" % (line_no+1, e))
|
||||
except Exception:
|
||||
debug("Unexpected error parsing line %d: %s" % (line_no+1, line))
|
||||
raise
|
||||
|
||||
# fix up missing offsets & negative sizes
|
||||
last_end = 0x5000 # first offset after partition table
|
||||
for e in res:
|
||||
if e.offset is None:
|
||||
pad_to = 0x10000 if e.type == PartitionDefinition.APP_TYPE else 4
|
||||
if last_end % pad_to != 0:
|
||||
last_end += pad_to - (last_end % pad_to)
|
||||
e.offset = last_end
|
||||
if e.size < 0:
|
||||
e.size = -e.size - e.offset
|
||||
last_end = e.offset + e.size
|
||||
|
||||
return res
|
||||
|
||||
def __getitem__(self, item):
|
||||
""" Allow partition table access via name as well as by
|
||||
numeric index. """
|
||||
if isinstance(item, str):
|
||||
for x in self:
|
||||
if x.name == item:
|
||||
return x
|
||||
raise ValueError("No partition entry named '%s'" % item)
|
||||
else:
|
||||
return super(PartitionTable, self).__getitem__(item)
|
||||
|
||||
def verify(self):
|
||||
# verify each partition individually
|
||||
for p in self:
|
||||
p.verify()
|
||||
# check for overlaps
|
||||
last = None
|
||||
for p in sorted(self):
|
||||
if p.offset < 0x5000:
|
||||
raise InputError("Partition offset 0x%x is below 0x5000" % p.offset)
|
||||
if last is not None and p.offset < last.offset + last.size:
|
||||
raise InputError("Partition at 0x%x overlaps 0x%x-0x%x" % (p.offset, last.offset, last.offset+last.size-1))
|
||||
last = p
|
||||
|
||||
@classmethod
|
||||
def from_binary(cls, b):
|
||||
if len(b) % 32 != 0:
|
||||
raise InputError("Partition table length must be a multiple of 32 bytes. Got %d bytes." % len(b))
|
||||
result = cls()
|
||||
for o in range(0,len(b),32):
|
||||
result.append(PartitionDefinition.from_binary(b[o:o+32]))
|
||||
return result
|
||||
|
||||
def to_binary(self):
|
||||
return "".join(e.to_binary() for e in self)
|
||||
|
||||
def to_csv(self, simple_formatting=False):
|
||||
rows = [ "# Espressif ESP32 Partition Table",
|
||||
"# Name, Type, SubType, Offset, Size" ]
|
||||
rows += [ x.to_csv(simple_formatting) for x in self ]
|
||||
return "\n".join(rows)
|
||||
|
||||
class PartitionDefinition(object):
|
||||
APP_TYPE = 0x00
|
||||
DATA_TYPE = 0x01
|
||||
TYPES = {
|
||||
"app" : APP_TYPE,
|
||||
"data" : DATA_TYPE,
|
||||
}
|
||||
|
||||
SUBTYPES = {
|
||||
APP_TYPE : {
|
||||
"factory" : 0x00,
|
||||
"test" : 0x20,
|
||||
},
|
||||
DATA_TYPE : {
|
||||
"ota" : 0x00,
|
||||
"rf" : 0x01,
|
||||
"wifi" : 0x02,
|
||||
},
|
||||
}
|
||||
|
||||
MAGIC_BYTES = "\xAA\x50"
|
||||
|
||||
ALIGNMENT = {
|
||||
APP_TYPE : 0x1000,
|
||||
DATA_TYPE : 0x04,
|
||||
}
|
||||
|
||||
# add subtypes for the 16 OTA slot values ("ota_XXX, etc.")
|
||||
for ota_slot in range(16):
|
||||
SUBTYPES[TYPES["app"]]["ota_%d" % ota_slot] = 0x10 + ota_slot
|
||||
|
||||
def __init__(self):
|
||||
self.name = ""
|
||||
self.type = None
|
||||
self.subtype = None
|
||||
self.offset = None
|
||||
self.size = None
|
||||
|
||||
@classmethod
|
||||
def from_csv(cls, line):
|
||||
""" Parse a line from the CSV """
|
||||
line_w_defaults = line + ",,," # lazy way to support default fields
|
||||
fields = [ f.strip() for f in line_w_defaults.split(",") ]
|
||||
|
||||
res = PartitionDefinition()
|
||||
res.name = fields[0]
|
||||
res.type = res.parse_type(fields[1])
|
||||
res.subtype = res.parse_subtype(fields[2])
|
||||
res.offset = res.parse_address(fields[3])
|
||||
res.size = res.parse_address(fields[4])
|
||||
if res.size is None:
|
||||
raise InputError("Size field can't be empty")
|
||||
return res
|
||||
|
||||
def __eq__(self, other):
|
||||
return self.name == other.name and self.type == other.type \
|
||||
and self.subtype == other.subtype and self.offset == other.offset \
|
||||
and self.size == other.size
|
||||
|
||||
def __repr__(self):
|
||||
def maybe_hex(x):
|
||||
return "0x%x" % x if x is not None else "None"
|
||||
return "PartitionDefinition('%s', 0x%x, 0x%x, %s, %s)" % (self.name, self.type, self.subtype or 0,
|
||||
maybe_hex(self.offset), maybe_hex(self.size))
|
||||
|
||||
def __str__(self):
|
||||
return "Part '%s' %d/%d @ 0x%x size 0x%x" % (self.name, self.type, self.subtype, self.offset or -1, self.size or -1)
|
||||
|
||||
def __cmp__(self, other):
|
||||
return self.offset - other.offset
|
||||
|
||||
def parse_type(self, strval):
|
||||
if strval == "":
|
||||
raise InputError("Field 'type' can't be left empty.")
|
||||
return parse_int(strval, self.TYPES)
|
||||
|
||||
def parse_subtype(self, strval):
|
||||
if strval == "":
|
||||
return 0 # default
|
||||
return parse_int(strval, self.SUBTYPES.get(self.type, {}))
|
||||
|
||||
def parse_address(self, strval):
|
||||
if strval == "":
|
||||
return None # PartitionTable will fill in default
|
||||
return parse_int(strval)
|
||||
|
||||
def verify(self):
|
||||
if self.type is None:
|
||||
raise ValidationError("Type field is not set")
|
||||
if self.subtype is None:
|
||||
raise ValidationError("Subtype field is not set")
|
||||
if self.offset is None:
|
||||
raise ValidationError("Offset field is not set")
|
||||
align = self.ALIGNMENT.get(self.type, 4)
|
||||
if self.offset % align:
|
||||
raise ValidationError("%s offset 0x%x is not aligned to 0x%x" % (self.name, self.offset, align))
|
||||
if self.size is None:
|
||||
raise ValidationError("Size field is not set")
|
||||
|
||||
STRUCT_FORMAT = "<2sBBLL16sL"
|
||||
|
||||
@classmethod
|
||||
def from_binary(cls, b):
|
||||
if len(b) != 32:
|
||||
raise InputError("Partition definition length must be exactly 32 bytes. Got %d bytes." % len(b))
|
||||
res = cls()
|
||||
(magic, res.type, res.subtype, res.offset,
|
||||
res.size, res.name, reserved) = struct.unpack(cls.STRUCT_FORMAT, b)
|
||||
if "\x00" in res.name: # strip null byte padding from name string
|
||||
res.name = res.name[:res.name.index("\x00")]
|
||||
if magic != cls.MAGIC_BYTES:
|
||||
raise InputError("Invalid magic bytes (%r) for partition definition" % magic)
|
||||
if reserved != 0:
|
||||
debug("WARNING: Partition definition had unexpected reserved value 0x%08x. Newer binary format?" % reserved)
|
||||
return res
|
||||
|
||||
def to_binary(self):
|
||||
return struct.pack(self.STRUCT_FORMAT,
|
||||
self.MAGIC_BYTES,
|
||||
self.type, self.subtype,
|
||||
self.offset, self.size,
|
||||
self.name,
|
||||
0) # reserved
|
||||
|
||||
def to_csv(self, simple_formatting=False):
|
||||
def addr_format(a):
|
||||
if not simple_formatting:
|
||||
for (val, suffix) in [ (0x100000, "M"), (0x400, "K") ]:
|
||||
if a % val == 0:
|
||||
return "%d%s" % (a / val, suffix)
|
||||
return "0x%x" % a
|
||||
|
||||
def lookup_keyword(t, keywords):
|
||||
for k,v in keywords.items():
|
||||
if simple_formatting == False and t == v:
|
||||
return k
|
||||
return "%d" % t
|
||||
|
||||
return ",".join([ self.name,
|
||||
lookup_keyword(self.type, self.TYPES),
|
||||
lookup_keyword(self.subtype, self.SUBTYPES),
|
||||
addr_format(self.offset),
|
||||
addr_format(self.size) ])
|
||||
|
||||
class InputError(RuntimeError):
|
||||
def __init__(self, e):
|
||||
super(InputError, self).__init__(e)
|
||||
|
||||
def parse_int(v, keywords={}):
|
||||
"""Generic parser for integer fields - int(x,0) with provision for
|
||||
k/m/K/M suffixes and 'keyword' value lookup.
|
||||
"""
|
||||
try:
|
||||
for letter, multiplier in [ ("k",1024), ("m",1024*1024) ]:
|
||||
if v.lower().endswith(letter):
|
||||
return parse_int(v[:-1], keywords) * multiplier
|
||||
return int(v, 0)
|
||||
except ValueError:
|
||||
if len(keywords) == 0:
|
||||
raise InputError("Invalid field value %s" % v)
|
||||
try:
|
||||
return keywords[v.lower()]
|
||||
except KeyError:
|
||||
raise InputError("Value '%s' is not valid. Known keywords: %s" % (v, ", ".join(keywords)))
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description='ESP32 partition table utility')
|
||||
|
||||
parser.add_argument('--verify', '-v', help='Verify partition table fields', default=True, action='store_false')
|
||||
|
||||
parser.add_argument('input', help='Path to CSV or binary file to parse. Will use stdin if omitted.', type=argparse.FileType('r'), default=sys.stdin)
|
||||
parser.add_argument('output', help='Path to output converted binary or CSV file. Will use stdout if omitted, unless the --display argument is also passed (in which case only the summary is printed.)',
|
||||
type=argparse.FileType('w'), nargs='?',
|
||||
default=sys.stdout)
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
input = args.input.read()
|
||||
input_is_binary = input[0:2] == PartitionDefinition.MAGIC_BYTES
|
||||
if input_is_binary:
|
||||
debug("Parsing binary partition input...")
|
||||
table = PartitionTable.from_binary(input)
|
||||
else:
|
||||
debug("Parsing CSV input...")
|
||||
table = PartitionTable.from_csv(input)
|
||||
|
||||
if args.verify:
|
||||
debug("Verifying table...")
|
||||
table.verify()
|
||||
|
||||
if input_is_binary:
|
||||
output = table.to_csv()
|
||||
else:
|
||||
output = table.to_binary()
|
||||
args.output.write(output)
|
||||
|
||||
if __name__ == '__main__':
|
||||
try:
|
||||
main()
|
||||
except InputError as e:
|
||||
print(e)
|
||||
sys.exit(2)
|
||||
@ -1,313 +0,0 @@
|
||||
#!/usr/bin/env python
|
||||
import unittest
|
||||
import struct
|
||||
import csv
|
||||
import sys
|
||||
import subprocess
|
||||
import tempfile
|
||||
import os
|
||||
sys.path.append("..")
|
||||
from gen_esp32part import *
|
||||
|
||||
SIMPLE_CSV = """
|
||||
# Name,Type,SubType,Offset,Size
|
||||
factory,0,2,65536,1048576
|
||||
"""
|
||||
|
||||
LONGER_BINARY_TABLE = ""
|
||||
# type 0x00, subtype 0x00,
|
||||
# offset 64KB, size 1MB
|
||||
LONGER_BINARY_TABLE += "\xAA\x50\x00\x00" + \
|
||||
"\x00\x00\x01\x00" + \
|
||||
"\x00\x00\x10\x00" + \
|
||||
"factory\0" + ("\0"*8) + \
|
||||
"\x00\x00\x00\x00"
|
||||
# type 0x01, subtype 0x20,
|
||||
# offset 0x110000, size 128KB
|
||||
LONGER_BINARY_TABLE += "\xAA\x50\x01\x20" + \
|
||||
"\x00\x00\x11\x00" + \
|
||||
"\x00\x02\x00\x00" + \
|
||||
"data" + ("\0"*12) + \
|
||||
"\x00\x00\x00\x00"
|
||||
# type 0x10, subtype 0x00,
|
||||
# offset 0x150000, size 1MB
|
||||
LONGER_BINARY_TABLE += "\xAA\x50\x10\x00" + \
|
||||
"\x00\x00\x15\x00" + \
|
||||
"\x00\x10\x00\x00" + \
|
||||
"second" + ("\0"*10) + \
|
||||
"\x00\x00\x00\x00"
|
||||
|
||||
|
||||
class CSVParserTests(unittest.TestCase):
|
||||
|
||||
def test_simple_partition(self):
|
||||
table = PartitionTable.from_csv(SIMPLE_CSV)
|
||||
self.assertEqual(len(table), 1)
|
||||
self.assertEqual(table[0].name, "factory")
|
||||
self.assertEqual(table[0].type, 0)
|
||||
self.assertEqual(table[0].subtype, 2)
|
||||
self.assertEqual(table[0].offset, 65536)
|
||||
self.assertEqual(table[0].size, 1048576)
|
||||
|
||||
|
||||
def test_require_type(self):
|
||||
csv = """
|
||||
# Name,Type, SubType,Offset,Size
|
||||
ihavenotype,
|
||||
"""
|
||||
with self.assertRaisesRegexp(InputError, "type"):
|
||||
PartitionTable.from_csv(csv)
|
||||
|
||||
|
||||
def test_type_subtype_names(self):
|
||||
csv_magicnumbers = """
|
||||
# Name, Type, SubType, Offset, Size
|
||||
myapp, 0, 0,, 0x100000
|
||||
myota_0, 0, 0x10,, 0x100000
|
||||
myota_1, 0, 0x11,, 0x100000
|
||||
myota_15, 0, 0x1f,, 0x100000
|
||||
mytest, 0, 0x20,, 0x100000
|
||||
myota_status, 1, 0,, 0x100000
|
||||
"""
|
||||
csv_nomagicnumbers = """
|
||||
# Name, Type, SubType, Offset, Size
|
||||
myapp, app, factory,, 0x100000
|
||||
myota_0, app, ota_0,, 0x100000
|
||||
myota_1, app, ota_1,, 0x100000
|
||||
myota_15, app, ota_15,, 0x100000
|
||||
mytest, app, test,, 0x100000
|
||||
myota_status, data, ota,, 0x100000
|
||||
"""
|
||||
# make two equivalent partition tables, one using
|
||||
# magic numbers and one using shortcuts. Ensure they match
|
||||
magic = PartitionTable.from_csv(csv_magicnumbers)
|
||||
magic.verify()
|
||||
nomagic = PartitionTable.from_csv(csv_nomagicnumbers)
|
||||
nomagic.verify()
|
||||
|
||||
self.assertEqual(nomagic["myapp"].type, 0)
|
||||
self.assertEqual(nomagic["myapp"].subtype, 0)
|
||||
self.assertEqual(nomagic["myapp"], magic["myapp"])
|
||||
self.assertEqual(nomagic["myota_0"].type, 0)
|
||||
self.assertEqual(nomagic["myota_0"].subtype, 0x10)
|
||||
self.assertEqual(nomagic["myota_0"], magic["myota_0"])
|
||||
self.assertEqual(nomagic["myota_15"], magic["myota_15"])
|
||||
self.assertEqual(nomagic["mytest"], magic["mytest"])
|
||||
self.assertEqual(nomagic["myota_status"], magic["myota_status"])
|
||||
|
||||
#self.assertEqual(nomagic.to_binary(), magic.to_binary())
|
||||
|
||||
def test_unit_suffixes(self):
|
||||
csv = """
|
||||
# Name, Type, Subtype, Offset, Size
|
||||
one_megabyte, app, factory, 32k, 1M
|
||||
"""
|
||||
t = PartitionTable.from_csv(csv)
|
||||
t.verify()
|
||||
self.assertEqual(t[0].offset, 32*1024)
|
||||
self.assertEqual(t[0].size, 1*1024*1024)
|
||||
|
||||
def test_default_offsets(self):
|
||||
csv = """
|
||||
# Name, Type, Subtype, Offset, Size
|
||||
first, app, factory,, 1M
|
||||
second, data, 0x15,, 1M
|
||||
minidata, data, 0x40,, 32K
|
||||
otherapp, app, factory,, 1M
|
||||
"""
|
||||
t = PartitionTable.from_csv(csv)
|
||||
# 'first'
|
||||
self.assertEqual(t[0].offset, 0x010000) # 64KB boundary as it's an app image
|
||||
self.assertEqual(t[0].size, 0x100000) # Size specified in CSV
|
||||
# 'second'
|
||||
self.assertEqual(t[1].offset, 0x110000) # prev offset+size
|
||||
self.assertEqual(t[1].size, 0x100000) # Size specified in CSV
|
||||
# 'minidata'
|
||||
self.assertEqual(t[2].offset, 0x210000)
|
||||
# 'otherapp'
|
||||
self.assertEqual(t[3].offset, 0x220000) # 64KB boundary as it's an app image
|
||||
|
||||
def test_negative_size_to_offset(self):
|
||||
csv = """
|
||||
# Name, Type, Subtype, Offset, Size
|
||||
first, app, factory, 0x10000, -2M
|
||||
second, data, 0x15, , 1M
|
||||
"""
|
||||
t = PartitionTable.from_csv(csv)
|
||||
t.verify()
|
||||
# 'first'
|
||||
self.assertEqual(t[0].offset, 0x10000) # in CSV
|
||||
self.assertEqual(t[0].size, 0x200000 - t[0].offset) # Up to 2M
|
||||
# 'second'
|
||||
self.assertEqual(t[1].offset, 0x200000) # prev offset+size
|
||||
|
||||
def test_overlapping_offsets_fail(self):
|
||||
csv = """
|
||||
first, app, factory, 0x100000, 2M
|
||||
second, app, ota_0, 0x200000, 1M
|
||||
"""
|
||||
t = PartitionTable.from_csv(csv)
|
||||
with self.assertRaisesRegexp(InputError, "overlap"):
|
||||
t.verify()
|
||||
|
||||
class BinaryOutputTests(unittest.TestCase):
|
||||
def test_binary_entry(self):
|
||||
csv = """
|
||||
first, 0x30, 0xEE, 0x100400, 0x300000
|
||||
"""
|
||||
t = PartitionTable.from_csv(csv)
|
||||
tb = t.to_binary()
|
||||
self.assertEqual(len(tb), 32)
|
||||
self.assertEqual('\xAA\x50', tb[0:2]) # magic
|
||||
self.assertEqual('\x30\xee', tb[2:4]) # type, subtype
|
||||
eo, es = struct.unpack("<LL", tb[4:12])
|
||||
self.assertEqual(eo, 0x100400) # offset
|
||||
self.assertEqual(es, 0x300000) # size
|
||||
|
||||
def test_multiple_entries(self):
|
||||
csv = """
|
||||
first, 0x30, 0xEE, 0x100400, 0x300000
|
||||
second,0x31, 0xEF, , 0x100000
|
||||
"""
|
||||
t = PartitionTable.from_csv(csv)
|
||||
tb = t.to_binary()
|
||||
self.assertEqual(len(tb), 64)
|
||||
self.assertEqual('\xAA\x50', tb[0:2])
|
||||
self.assertEqual('\xAA\x50', tb[32:34])
|
||||
|
||||
|
||||
class BinaryParserTests(unittest.TestCase):
|
||||
def test_parse_one_entry(self):
|
||||
# type 0x30, subtype 0xee,
|
||||
# offset 1MB, size 2MB
|
||||
entry = "\xAA\x50\x30\xee" + \
|
||||
"\x00\x00\x10\x00" + \
|
||||
"\x00\x00\x20\x00" + \
|
||||
"0123456789abc\0\0\0" + \
|
||||
"\x00\x00\x00\x00"
|
||||
# verify that parsing 32 bytes as a table
|
||||
# or as a single Definition are the same thing
|
||||
t = PartitionTable.from_binary(entry)
|
||||
self.assertEqual(len(t), 1)
|
||||
t[0].verify()
|
||||
|
||||
e = PartitionDefinition.from_binary(entry)
|
||||
self.assertEqual(t[0], e)
|
||||
e.verify()
|
||||
|
||||
self.assertEqual(e.type, 0x30)
|
||||
self.assertEqual(e.subtype, 0xEE)
|
||||
self.assertEqual(e.offset, 0x100000)
|
||||
self.assertEqual(e.size, 0x200000)
|
||||
self.assertEqual(e.name, "0123456789abc")
|
||||
|
||||
def test_multiple_entries(self):
|
||||
t = PartitionTable.from_binary(LONGER_BINARY_TABLE)
|
||||
t.verify()
|
||||
|
||||
self.assertEqual(3, len(t))
|
||||
self.assertEqual(t[0].type, PartitionDefinition.APP_TYPE)
|
||||
self.assertEqual(t[0].name, "factory")
|
||||
|
||||
self.assertEqual(t[1].type, PartitionDefinition.DATA_TYPE)
|
||||
self.assertEqual(t[1].name, "data")
|
||||
|
||||
self.assertEqual(t[2].type, 0x10)
|
||||
self.assertEqual(t[2].name, "second")
|
||||
|
||||
round_trip = t.to_binary()
|
||||
self.assertEqual(round_trip, LONGER_BINARY_TABLE)
|
||||
|
||||
def test_bad_magic(self):
|
||||
bad_magic = "OHAI" + \
|
||||
"\x00\x00\x10\x00" + \
|
||||
"\x00\x00\x20\x00" + \
|
||||
"0123456789abc\0\0\0" + \
|
||||
"\x00\x00\x00\x00"
|
||||
with self.assertRaisesRegexp(InputError, "Invalid magic bytes"):
|
||||
PartitionTable.from_binary(bad_magic)
|
||||
|
||||
def test_bad_length(self):
|
||||
bad_length = "OHAI" + \
|
||||
"\x00\x00\x10\x00" + \
|
||||
"\x00\x00\x20\x00" + \
|
||||
"0123456789"
|
||||
with self.assertRaisesRegexp(InputError, "32 bytes"):
|
||||
PartitionTable.from_binary(bad_length)
|
||||
|
||||
|
||||
class CSVOutputTests(unittest.TestCase):
|
||||
|
||||
def test_output_simple_formatting(self):
|
||||
table = PartitionTable.from_csv(SIMPLE_CSV)
|
||||
as_csv = table.to_csv(True)
|
||||
c = csv.reader(as_csv.split("\n"))
|
||||
# first two lines should start with comments
|
||||
self.assertEqual(c.next()[0][0], "#")
|
||||
self.assertEqual(c.next()[0][0], "#")
|
||||
row = c.next()
|
||||
self.assertEqual(row[0], "factory")
|
||||
self.assertEqual(row[1], "0")
|
||||
self.assertEqual(row[2], "2")
|
||||
self.assertEqual(row[3], "0x10000") # reformatted as hex
|
||||
self.assertEqual(row[4], "0x100000") # also hex
|
||||
|
||||
# round trip back to a PartitionTable and check is identical
|
||||
roundtrip = PartitionTable.from_csv(as_csv)
|
||||
self.assertEqual(roundtrip, table)
|
||||
|
||||
def test_output_smart_formatting(self):
|
||||
table = PartitionTable.from_csv(SIMPLE_CSV)
|
||||
as_csv = table.to_csv(False)
|
||||
c = csv.reader(as_csv.split("\n"))
|
||||
# first two lines should start with comments
|
||||
self.assertEqual(c.next()[0][0], "#")
|
||||
self.assertEqual(c.next()[0][0], "#")
|
||||
row = c.next()
|
||||
self.assertEqual(row[0], "factory")
|
||||
self.assertEqual(row[1], "app")
|
||||
self.assertEqual(row[2], "2")
|
||||
self.assertEqual(row[3], "64K")
|
||||
self.assertEqual(row[4], "1M")
|
||||
|
||||
# round trip back to a PartitionTable and check is identical
|
||||
roundtrip = PartitionTable.from_csv(as_csv)
|
||||
self.assertEqual(roundtrip, table)
|
||||
|
||||
class CommandLineTests(unittest.TestCase):
|
||||
|
||||
def test_basic_cmdline(self):
|
||||
try:
|
||||
binpath = tempfile.mktemp()
|
||||
csvpath = tempfile.mktemp()
|
||||
|
||||
# copy binary contents to temp file
|
||||
with open(binpath, 'w') as f:
|
||||
f.write(LONGER_BINARY_TABLE)
|
||||
|
||||
# run gen_esp32part.py to convert binary file to CSV
|
||||
subprocess.check_call([sys.executable, "../gen_esp32part.py",
|
||||
binpath, csvpath])
|
||||
# reopen the CSV and check the generated binary is identical
|
||||
with open(csvpath, 'r') as f:
|
||||
from_csv = PartitionTable.from_csv(f.read())
|
||||
self.assertEqual(from_csv.to_binary(), LONGER_BINARY_TABLE)
|
||||
|
||||
# run gen_esp32part.py to conver the CSV to binary again
|
||||
subprocess.check_call([sys.executable, "../gen_esp32part.py",
|
||||
csvpath, binpath])
|
||||
# assert that file reads back as identical
|
||||
with open(binpath, 'rb') as f:
|
||||
binary_readback = f.read()
|
||||
self.assertEqual(binary_readback, LONGER_BINARY_TABLE)
|
||||
|
||||
finally:
|
||||
for path in binpath, csvpath:
|
||||
try:
|
||||
os.remove(path)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
if __name__ =="__main__":
|
||||
unittest.main()
|
||||
Loading…
x
Reference in New Issue
Block a user