alex/esp-idf
alex/esp-idf
1
0
Fork 0
mirror of https://github.com/espressif/esp-idf.git synced 2024-09-19 14:26:01 -04:00
Code Issues Actions 3 Packages Projects Releases Wiki Activity

Merge branch 'fix/usb_uvc_example' into 'master'

Browse Source 
fix(usb/uvc): Add negotiation retry for some cameras

Closes IDF-8085

See merge request espressif/esp-idf!27309
This commit is contained in:
Tomas Rezucha 2023-11-29 15:38:53 +08:00
commit d315b97948
8 changed files with 148 additions and 144 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
examples/peripherals/usb/device/tusb_composite_msc_serialdevice/sdkconfig.defaults
View File

@ -1,7 +1,6 @@
# This file was generated using idf.py save-defconfig. It can be edited manually.
# Espressif IoT Development Framework (ESP-IDF) Project Minimal Configuration
#
CONFIG_TINYUSB=y
CONFIG_TINYUSB_MSC_ENABLED=y
CONFIG_PARTITION_TABLE_CUSTOM=y

221
examples/peripherals/usb/host/uvc/README.md
View File

@ -10,14 +10,11 @@ This example demonstrates how to:
- Capture video from a USB camera using the `libuvc` library.
- Stream the video over WiFi by hosting a TCP server.
The example enumerates connected camera, negotiates selected resolution together with `FPS` and starts capturing video.
`frame_callback` function is then invoked after receiving each frame. User can process received frame according to his needs.
The example enumerates a connected USB camera, negotiates a selected resolution along with an associated `FPS`, then starts capturing video. The `frame_callback` function is then invoked after receiving each frame. Users can process the received frames according to their needs.
Optionally, captured video can be visualized on computer with help of `player.py` script located in this example.
After setting `Enable streaming` menuconfig option, example will create TCP server upon start, and wait until `player.py` connects to server.
Once connection is established, example streams each received frame to computer for visualization.
Optionally, the captured video can be visualized on a PC with help of the `player.py` script provided in this example. After setting the `Example Configuration->Enable streaming` option in menuconfig, the example will create a TCP server upon startup, and waits until `player.py` connects to the server. Once a connection is established, the example streams each received frame to the PC for visualization. The network connection can be configured in menuconfig via `Example Connection Configuration`.
**Notice** that `libuvc` selects highest possible `dwMaxPayloadTransferSize` by default, so example has to manually overwrite this value to 512 bytes (maximum transfer size supported by ESP32-S2/S3).
**Notice** that `libuvc` selects highest possible `dwMaxPayloadTransferSize` by default. As a result, this example will manually overwrite this value to 512 bytes (the maximum packet size supported by ESP32-S2/S3).
(See the README.md file in the upper level 'examples' directory for more information about examples.)
@ -25,8 +22,11 @@ Once connection is established, example streams each received frame to computer
### Hardware Required
This example requires any ESP32-S2 or ESP32-S3 with external PSRAM and exposed USB connector attached to USB camera.
*ESP module without external PSRAM will fail to initialize*
* ESP with USB peripheral and external PSRAM
* Exposed USB host connector
* USB camera
Running this example on an **ESP module without external PSRAM will fail on initialization**. Please select your PSRAM configuration in menuconfig `Component config->ESP PSRAM`. If you manually disable PSRAM, the required framebuffers might not fit into DRAM (especially on ESP32-S2).
### Configure the project
@ -51,36 +51,31 @@ In the `Example Configuration` menu:
Optional: If you need, change the other options according to your requirements.
Additionally, `player.py` python script makes use of `opencv-python` and `numpy` packages,
not included in `idf-env` environment. Run following commands to install:
* `pip install opencv-python`
* `pip install numpy`
Additionally, the `player.py` python script makes use of the `opencv-python` and `numpy` packages which are not included in the `idf-env` environment by default. Run following command to install those packages:
```bash
pip install opencv-python numpy
```
#### UVC Protocol Mode: Auto
When protocol mode set to Auto, the example tries to make three attempts to negotiatiate
the protocol with following parameters:
When the protocol mode is set to Auto, the example will make three attempts to negotiate the protocol with following parameters:
1 Attempt: 640x480, 15 FPS, MJPEG
2 Attempt: 320x240, 30 FPS, MJPEG
3 Attempt: 320x240, first available FPS, MJPEG
1. Attempt: 640x480, 15 FPS, MJPEG
2. Attempt: 320x240, 30 FPS, MJPEG
3. Attempt: 320x240, first available FPS, MJPEG
If all three attempts result in an error, the example displays the error message and
suggests to try another USB UVC Device.
If all three attempts result in an error, the example displays the error message and suggests to try another USB UVC Device.
#### UVC Protocol Mode: Custom
When protocol mode set to Custom, the example tries to negotiate protocol with
configured parameters: Attempts, Width, Heighs, FPS, Frame Coding format.
After all attemts result in an error, the example displays the error message and
suggests to try another USB UVC Device.
When the protocol mode set to Custom, the example tries to negotiate the protocol using user provided values for the following parameters: Attempts, Width, Height, FPS, and Frame Coding format. After all attempts result in an error, the example displays the error message and suggests to try another USB UVC device.
### Build and Flash
Build the project and flash it to the board, then run the monitor tool to view the serial output:
Run `idf.py set-target esp32s2` to set target chip.
Run `idf.py set-target esp32xx` to set target chip.
Run `idf.py -p PORT flash monitor` to build, flash and monitor the project.
@ -90,14 +85,14 @@ See the Getting Started Guide for all the steps to configure and use the ESP-IDF
## Known limitations
Having only Full Speed USB peripheral and hardware limited MPS (maximum packet size) to 512 bytes, ESP32-S2/S3 is capable of reading about 0.5 MB of data per second. When connected to Full Speed USB host, cameras normally provide resolution no larger than 640x480 pixels.
Following two supported formats are the most common (both encoded in MJPEG):
* 320x240 30 FPS
* 640x480 15 FPS
Having only a Full Speed USB peripheral and hardware limited MPS (maximum packet size) to 512 bytes, the ESP32-S2/S3 is capable of reading data at approximately 0.5 MB/s. When connected to Full Speed USB host, cameras normally provide resolution no larger than 640x480 pixels. The following two formats are the most commonly supported at Full Speed (both encoded in MJPEG):
* 320x240 30 FPS
* 640x480 15 FPS
## Tested cameras
* Logitech C980
* CANYON CNE-CWC2
* Logitech C980
* Logitech C270
* CANYON CNE-CWC2
## Example Output
@ -108,89 +103,89 @@ I (1606) example: Device found
DEVICE CONFIGURATION (0c45:6340/ S) ---
Status: idle
VideoControl:
bcdUVC: 0x0100
bcdUVC: 0x0100
VideoStreaming(1):
bEndpointAddress: 129
Formats:
MJPEGFormat(1)
bits per pixel: 0
GUID: 4d4a5047000000000000000000000000 (MJPG)
default frame: 1
aspect ratio: 0x0
interlace flags: 00
copy protect: 00
FrameDescriptor(1)
capabilities: 00
size: 640x480
bit rate: 24576000-147456000
max frame size: 614400
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
interval[2]: 1/20
interval[3]: 1/15
interval[4]: 1/10
interval[5]: 1/5
FrameDescriptor(2)
capabilities: 00
size: 352x288
bit rate: 8110080-48660480
max frame size: 202752
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
interval[2]: 1/20
interval[3]: 1/15
interval[4]: 1/10
interval[5]: 1/5
FrameDescriptor(3)
capabilities: 00
size: 320x240
bit rate: 6144000-36864000
max frame size: 153600
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
interval[2]: 1/20
interval[3]: 1/15
interval[4]: 1/10
interval[5]: 1/5
FrameDescriptor(4)
capabilities: 00
size: 176x144
bit rate: 2027520-12165120
max frame size: 50688
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
interval[2]: 1/20
interval[3]: 1/15
interval[4]: 1/10
interval[5]: 1/5
FrameDescriptor(5)
capabilities: 00
size: 160x120
bit rate: 1536000-9216000
max frame size: 38400
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
interval[2]: 1/20
interval[3]: 1/15
interval[4]: 1/10
interval[5]: 1/5
StillFrameDescriptor
bEndPointAddress: 00
wWidth(1) = 640
wHeight(1) = 480
wWidth(2) = 352
wHeight(2) = 288
wWidth(3) = 320
wHeight(3) = 240
wWidth(4) = 176
wHeight(4) = 144
wWidth(5) = 160
wHeight(5) = 120
bEndpointAddress: 129
Formats:
MJPEGFormat(1)
bits per pixel: 0
GUID: 4d4a5047000000000000000000000000 (MJPG)
default frame: 1
aspect ratio: 0x0
interlace flags: 00
copy protect: 00
FrameDescriptor(1)
capabilities: 00
size: 640x480
bit rate: 24576000-147456000
max frame size: 614400
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
interval[2]: 1/20
interval[3]: 1/15
interval[4]: 1/10
interval[5]: 1/5
FrameDescriptor(2)
capabilities: 00
size: 352x288
bit rate: 8110080-48660480
max frame size: 202752
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
interval[2]: 1/20
interval[3]: 1/15
interval[4]: 1/10
interval[5]: 1/5
FrameDescriptor(3)
capabilities: 00
size: 320x240
bit rate: 6144000-36864000
max frame size: 153600
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
interval[2]: 1/20
interval[3]: 1/15
interval[4]: 1/10
interval[5]: 1/5
FrameDescriptor(4)
capabilities: 00
size: 176x144
bit rate: 2027520-12165120
max frame size: 50688
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
interval[2]: 1/20
interval[3]: 1/15
interval[4]: 1/10
interval[5]: 1/5
FrameDescriptor(5)
capabilities: 00
size: 160x120
bit rate: 1536000-9216000
max frame size: 38400
default interval: 1/30
interval[0]: 1/30
interval[1]: 1/25
interval[2]: 1/20
interval[3]: 1/15
interval[4]: 1/10
interval[5]: 1/5
StillFrameDescriptor
bEndPointAddress: 00
wWidth(1) = 640
wHeight(1) = 480
wWidth(2) = 352
wHeight(2) = 288
wWidth(3) = 320
wHeight(3) = 240
wWidth(4) = 176
wHeight(4) = 144
wWidth(5) = 160
wHeight(5) = 120
END DEVICE CONFIGURATION
I (1796) example: Negotiate streaming profile 640x480, fps 15 ...
I (1816) example: Negotiation complete.

10
examples/peripherals/usb/host/uvc/main/CMakeLists.txt
View File

@ -1,3 +1,11 @@
idf_component_register(SRCS "main.c" "tcp_server.c"
INCLUDE_DIRS ""
PRIV_REQUIRES protocol_examples_common nvs_flash usb mdns esp_ringbuf esp_timer esp_wifi driver)
PRIV_REQUIRES
nvs_flash
usb
esp_ringbuf
esp_psram # Required for CONFIG_SPIRAM
esp_timer
esp_wifi
esp_driver_gpio
)

10
examples/peripherals/usb/host/uvc/main/Kconfig.projbuild
View File

@ -30,15 +30,15 @@ menu "Example Configuration"
endchoice
menu "UVC Protocol parameters"
depends on EXAMPLE_UVC_PROTOCOL_MODE_CUSTOM
config EXAMPLE_NEGOTIATION_ATTEMPTS
int "Attempts"
config EXAMPLE_NEGOTIATION_ATTEMPTS
int "Negotiation attempts"
default 3
help
Number of attempts to negotiate custom protocol parameters.
menu "UVC Protocol parameters"
depends on EXAMPLE_UVC_PROTOCOL_MODE_CUSTOM
config EXAMPLE_WIDTH_PARAM
int "Width resolution in pixels"
default 320

8
examples/peripherals/usb/host/uvc/main/idf_component.yml
View File

@ -1,9 +1,7 @@
## IDF Component Manager Manifest File
dependencies:
idf: ">=4.4"
usb_host_uvc: "1.0.0"
mdns:
rules:
- if: "idf_version >= 5.0"
idf: ">=5.0"
usb_host_uvc: "^1.0.0"
mdns: "^1.2.0"
protocol_examples_common:
path: ${IDF_PATH}/examples/common_components/protocol_examples_common

30
examples/peripherals/usb/host/uvc/main/main.c
View File

@ -170,34 +170,36 @@ static uvc_error_t uvc_negotiate_stream_profile(uvc_device_handle_t *devh,
uvc_stream_ctrl_t *ctrl)
{
uvc_error_t res;
int attempt = CONFIG_EXAMPLE_NEGOTIATION_ATTEMPTS;
#if (CONFIG_EXAMPLE_UVC_PROTOCOL_MODE_AUTO)
for (int idx = 0; idx < EXAMPLE_UVC_PROTOCOL_AUTO_COUNT; idx++) {
ESP_LOGI(TAG, "Negotiate streaming profile %s ...", uvc_stream_profiles[idx].name);
res = uvc_get_stream_ctrl_format_size(devh,
ctrl,
uvc_stream_profiles[idx].format,
uvc_stream_profiles[idx].width,
uvc_stream_profiles[idx].height,
uvc_stream_profiles[idx].fps);
do {
/*
The uvc_get_stream_ctrl_format_size() function will attempt to set the desired format size.
On first attempt, some cameras would reject the format, even if they support it.
So we ask 3x by default. The second attempt is usually successful.
*/
ESP_LOGI(TAG, "Negotiate streaming profile %s ...", uvc_stream_profiles[idx].name);
res = uvc_get_stream_ctrl_format_size(devh,
ctrl,
uvc_stream_profiles[idx].format,
uvc_stream_profiles[idx].width,
uvc_stream_profiles[idx].height,
uvc_stream_profiles[idx].fps);
} while (--attempt && !(UVC_SUCCESS == res));
if (UVC_SUCCESS == res) {
break; // stream profile negotiated
break;
}
sleep(1);
ESP_LOGE(TAG, "Negotiation failed with error %d.", res);
}
#endif // CONFIG_EXAMPLE_UVC_PROTOCOL_MODE_AUTO
#if (CONFIG_EXAMPLE_UVC_PROTOCOL_MODE_CUSTOM)
int attempt = CONFIG_EXAMPLE_NEGOTIATION_ATTEMPTS;
while (attempt--) {
ESP_LOGI(TAG, "Negotiate streaming profile %dx%d, %d fps ...", WIDTH, HEIGHT, FPS);
res = uvc_get_stream_ctrl_format_size(devh, ctrl, FORMAT, WIDTH, HEIGHT, FPS);
if (UVC_SUCCESS == res) {
break;
}
sleep(1);
ESP_LOGE(TAG, "Negotiation failed. Try again (%d) ...", attempt);
}
#endif // CONFIG_EXAMPLE_UVC_PROTOCOL_MODE_CUSTOM

8
examples/peripherals/usb/host/uvc/sdkconfig.defaults
View File

@ -4,11 +4,17 @@
CONFIG_SPIRAM=y
CONFIG_SPIRAM_MALLOC_RESERVE_INTERNAL=150000
CONFIG_SPIRAM_TRY_ALLOCATE_WIFI_LWIP=y
CONFIG_SPIRAM_SPEED_80M=y
#
# SYSTEM
#
CONFIG_ESP_DEFAULT_CPU_FREQ_MHZ_240=y
#
# USB
#
CONFIG_USB_HOST_CONTROL_TRANSFER_MAX_SIZE=1024
CONFIG_USB_HOST_CONTROL_TRANSFER_MAX_SIZE=3000
CONFIG_USB_HOST_HW_BUFFER_BIAS_IN=y
#

4
tools/ci/ignore_build_warnings.txt
View File

@ -56,20 +56,16 @@ warning: unknown kconfig symbol 'SPIRAM_MALLOC_ALWAYSINTERNAL' assigned to '0' i
warning: unknown kconfig symbol 'SPIRAM_MALLOC_ALWAYSINTERNAL' assigned to '0' in /builds/espressif/esp-idf/components/fatfs/test_apps/sdcard/sdkconfig.ci.psram
warning: unknown kconfig symbol 'SPIRAM_MALLOC_ALWAYSINTERNAL' assigned to '0' in /builds/espressif/esp-idf/components/spiffs/test_apps/sdkconfig.ci.psram
warning: unknown kconfig symbol 'SPIRAM_MALLOC_ALWAYSINTERNAL' assigned to '0' in /builds/espressif/esp-idf/components/vfs/test_apps/*
warning: unknown kconfig symbol 'SPIRAM_MALLOC_RESERVE_INTERNAL' assigned to '150000' in /builds/espressif/esp-idf/examples/peripherals/usb/host/uvc/sdkconfig.defaults
warning: unknown kconfig symbol 'SPIRAM_RODATA' assigned to 'y' in /builds/espressif/esp-idf/components/spi_flash/test_apps/mspi_test/sdkconfig.ci.psram
warning: unknown kconfig symbol 'SPIRAM_TRY_ALLOCATE_WIFI_LWIP' assigned to 'y' in /builds/espressif/esp-idf/examples/peripherals/usb/host/uvc/sdkconfig.defaults
warning: unknown kconfig symbol 'SPIRAM' assigned to 'y' in /builds/espressif/esp-idf/components/fatfs/test_apps/flash_wl/sdkconfig.ci.psram
warning: unknown kconfig symbol 'SPIRAM' assigned to 'y' in /builds/espressif/esp-idf/components/fatfs/test_apps/sdcard/sdkconfig.ci.psram
warning: unknown kconfig symbol 'SPIRAM' assigned to 'y' in /builds/espressif/esp-idf/components/spiffs/test_apps/sdkconfig.ci.psram
warning: unknown kconfig symbol 'SPIRAM' assigned to 'y' in /builds/espressif/esp-idf/components/vfs/test_apps/*
warning: unknown kconfig symbol 'SPIRAM' assigned to 'y' in /builds/espressif/esp-idf/examples/bluetooth/esp_ble_mesh/fast_provisioning/fast_prov_server/sdkconfig.ci.psram
warning: unknown kconfig symbol 'SPIRAM' assigned to 'y' in /builds/espressif/esp-idf/examples/network/simple_sniffer/sdkconfig.ci.mem
warning: unknown kconfig symbol 'SPIRAM' assigned to 'y' in /builds/espressif/esp-idf/examples/peripherals/usb/host/uvc/sdkconfig.defaults
warning: unknown kconfig symbol 'SPIRAM' assigned to 'y' in /builds/espressif/esp-idf/examples/protocols/http2_request/sdkconfig.ci
warning: unknown kconfig symbol 'SPIRAM' assigned to 'y' in /builds/espressif/esp-idf/examples/protocols/https_mbedtls/sdkconfig.ci
warning: unknown kconfig symbol 'SPIRAM' assigned to 'y' in /builds/espressif/esp-idf/examples/protocols/https_request/sdkconfig.ci
warning: unknown kconfig symbol 'SPIRAM' assigned to 'y' in /builds/espressif/esp-idf/examples/protocols/https_request/sdkconfig.ci.ssldyn
warning: unknown kconfig symbol 'TINYUSB' assigned to 'y' in /builds/espressif/esp-idf/examples/peripherals/usb/device/tusb_composite_msc_serialdevice/sdkconfig.defaults
warning: unknown kconfig symbol 'UNITY_FREERTOS_STACK_SIZE' assigned to '12288' in /builds/espressif/esp-idf/components/bt/test_apps/sdkconfig.defaults
warning: unknown kconfig symbol 'WPA3_SAE' assigned to 'y' in /builds/espressif/esp-idf/components/wpa_supplicant/test_apps/sdkconfig.defaults