esp-idf/docs/en/api-reference/peripherals/jpeg.rst

JPEG Encoder and Decoder
========================

Introduction
------------

JPEG is a commonly used method of lossy compression for digital images, particularly for those images produced by digital photography. The degree of compression can be adjusted, allowing a selectable tradeoff between storage size and image quality. JPEG typically achieves 10:1 compression with little perceptible loss in image quality.

JPEG codec on {IDF_TARGET_NAME} is an image codec, which is based on the JPEG baseline standard, for compressing and decompressing images to reduce the bandwidth required to transmit images or the space required to store images, making it possible to process large-resolution images. But please note, at one time, the codec engine can only work as either encoder or decoder.

Functional Overview
-------------------

The JPEG driver offers following services:

-  `Resource Allocation <#resource-allocation>`__ - covers how to allocate JPEG resources with properly set of configurations. It also covers how to recycle the resources when they finished working.
-  `JPEG Decoder Engine <#jpeg_decoder_engine>`__ - covers behavior of JPEG decoder engine. Introduce how to use decoder engine functions to decode an image (from jpg format to raw format).
-  `JPEG Encoder Engine <#jpeg_encoder_engine>`__ - covers behavior of JPEG encoder engine. Introduce how to use encoder engine functions to encode an image (from raw format to jpg format).
-  `Thread Safety <#thread-safety>`__ - lists which APIs are guaranteed to be thread safe by the driver.
-  `Kconfig Options <#kconfig-options>`__ - lists the supported Kconfig options that can bring different effects to the driver.

Resource Allocation
^^^^^^^^^^^^^^^^^^^

Install JPEG decoder engine
~~~~~~~~~~~~~~~~~~~~~~~~~~~

JPEG decoder engine requires the configuration that specified by :cpp:type:`jpeg_decode_engine_cfg_t`:

- :cpp:member:`jpeg_decode_engine_cfg_t::intr_priority` Set the priority of the interrupt. If set to ``0`` , then the driver will use a interrupt with low or medium priority (priority level may be one of 1,2 or 3), otherwise use the priority indicated by :cpp:member:`jpeg_decode_engine_cfg_t::intr_priority` Please use the number form (1,2,3) , not the bitmask form ((1<<1),(1<<2),(1<<3)).

If the configurations in :cpp:type:`jpeg_decode_engine_cfg_t` is specified, users can call :cpp:func:`jpeg_new_decoder_engine` to allocate and initialize a JPEG decoder engine. This function will return an JPEG decoder handle if it runs correctly. You can take following code as reference.

.. code:: c

    jpeg_decoder_handle_t jpgd_handle;

    jpeg_decode_engine_cfg_t decode_eng_cfg = {
        .intr_priority = 0,
        .timeout_ms = 40,
    };

    ESP_ERROR_CHECK(jpeg_new_decoder_engine(&decode_eng_cfg, &jpgd_handle));


Uninstall JPEG decoder engine
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If a previously installed JPEG engine is no longer needed, it's recommended to recycle the resource by calling :cpp:func:`jpeg_del_decoder_engine`, so that to release the underlying hardware.

Install JPEG encoder engine
~~~~~~~~~~~~~~~~~~~~~~~~~~~

The JPEG encoder engine requires the configuration specified by :cpp:type:`jpeg_encode_engine_cfg_t`.

If the configurations in :cpp:type:`jpeg_encode_engine_cfg_t` is specified, users can call :cpp:func:`jpeg_new_encoder_engine` to allocate and initialize a JPEG decoder engine. This function will return an JPEG decoder handle if it runs correctly. You can take following code as reference.

.. code:: c

    jpeg_encoder_handle_t encoder_engine;

    jpeg_encode_engine_cfg_t encode_eng_cfg = {
        .intr_priority = 0,
        .timeout_ms = 40,
    };

    ESP_ERROR_CHECK(jpeg_new_encoder_engine(&encode_eng_cfg, &encoder_engine));

Uninstall JPEG encoder engine
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If a previously installed JPEG engine is no longer needed, it's recommended to recycle the resource by calling :cpp:func:`jpeg_del_encoder_engine`, so that the underlying hardware is released.

JPEG Decoder Engine
^^^^^^^^^^^^^^^^^^^

After installing the JPEG decoder driver by :cpp:func:`jpeg_new_decoder_engine`, {IDF_TARGET_NAME} is ready to decode JPEG pictures by :cpp:func:`jpeg_decoder_process`. :cpp:func:`jpeg_decoder_process` is flexible for decoding different types of pictures by a configurable parameter called :cpp:type:`jpeg_decode_cfg_t`:

- :cpp:member:`jpeg_decode_cfg_t::output_format` Set the output raw image format.
- :cpp:member:`jpeg_decode_cfg_t::rgb_order` Set the output pixel order. (RGB or BGR).
- :cpp:member:`jpeg_decode_cfg_t::jpeg_yuv_rgb_conv_std_t` Set the output YUV and RGB conversion standard (BT601 or BT709)

Moreover, our jpeg decoder api provides a helper function which helps you get the basic information of your given image. Calling :cpp:func:`jpeg_decoder_get_info` would return the picture information structure called :cpp:func:`jpeg_decoder_get_info`. If you already know the picture basic information, this functions is unnecessary to be called.

Overall, You can take following code as reference, the code is going to decode a 1080*1920 picture.

.. code:: c

    jpeg_decode_cfg_t decode_cfg_rgb = {
        .output_format = JPEG_DECODE_OUT_FORMAT_RGB888,
        .rgb_order = JPEG_DEC_RGB_ELEMENT_ORDER_BGR,
    };

    size_t tx_buffer_size;
    size_t rx_buffer_size;

    jpeg_decode_memory_alloc_cfg_t rx_mem_cfg = {
        .buffer_direction = JPEG_DEC_ALLOC_OUTPUT_BUFFER,
    };

    jpeg_decode_memory_alloc_cfg_t tx_mem_cfg = {
        .buffer_direction = JPEG_DEC_ALLOC_INPUT_BUFFER,
    };

    uint8_t *bit_stream = (uint8_t*)jpeg_alloc_decoder_mem(jpeg_size, &tx_mem_cfg, &tx_buffer_size);
    uint8_t *out_buf = (uint8_t*)jpeg_alloc_decoder_mem(1920 * 1088 * 3, &rx_mem_cfg, &rx_buffer_size);

    jpeg_decode_picture_info_t header_info;
    ESP_ERROR_CHECK(jpeg_decoder_get_info(bit_stream, bit_stream_size, &header_info));
    uint32_t out_size = 0;
    ESP_ERROR_CHECK(jpeg_decoder_process(jpgd_handle, &decode_cfg_rgb, bit_stream, bit_stream_size, out_buf, &out_size));


There are some Tips that can help you use this driver more accurately:

Firstly, in above code, you should make sure the `bit_stream` and `out_buf` should be aligned by certain rules. We provide a helper function :cpp:func:`jpeg_alloc_decoder_mem` to help you malloc a buffer which is aligned in both size and address.

Secondly, the content of `bit_stream` buffer should not be changed until :cpp:func:`jpeg_decoder_process` returns.

Thirdly, the width and height of output picture would be 16 bytes aligned if original picture is formatted by YUV420 or YUV422. For example, if the input picture is 1080*1920, the output picture will be 1088*1920. That is the restriction of jpeg protocol. Please provide sufficient output buffer memory.

JPEG Encoder Engine
^^^^^^^^^^^^^^^^^^^

After installing the JPEG encoder driver by :cpp:func:`jpeg_new_encoder_engine`, {IDF_TARGET_NAME} is ready to decode JPEG pictures by :cpp:func:`jpeg_encoder_process`. :cpp:func:`jpeg_encoder_process` is flexible for decoding different types of pictures by a configurable parameter called :cpp:type:`jpeg_encode_cfg_t`:

Below is the example of code that encodes a 1080*1920 picture:

.. code:: c

    jpeg_encode_cfg_t enc_config = {
        .src_type = JPEG_ENCODE_IN_FORMAT_RGB888,
        .sub_sample = JPEG_DOWN_SAMPLING_YUV422,
        .image_quality = 80,
        .width = 1920,
        .height = 1080,
    };

    uint8_t *raw_buf_1080p = (uint8_t*)jpeg_alloc_encoder_mem(raw_size_1080p);
    if (raw_buf_1080p == NULL) {
        ESP_LOGE(TAG, "alloc 1080p tx buffer error");
        return;
    }
    uint8_t *jpg_buf_1080p = (uint8_t*)jpeg_alloc_encoder_mem(raw_size_1080p / 10); // Assume that compression ratio of 10 to 1
    if (jpg_buf_1080p == NULL) {
        ESP_LOGE(TAG, "alloc jpg_buf_1080p error");
        return;
    }

    ESP_ERROR_CHECK(jpeg_encoder_process(jpeg_handle, &enc_config, raw_buf_1080p, raw_size_1080p, jpg_buf_1080p, &jpg_size_1080p););

There are some Tips that can help you use this driver more accurately:

Firstly, in above code, you should make sure the `raw_buf_1080p` and `jpg_buf_1080p` should aligned by calling :cpp:func:`jpeg_alloc_encoder_mem`.

Secondly, the content of `raw_buf_1080p` buffer should not be changed until :cpp:func:`jpeg_encoder_process` returns.

Thread Safety
^^^^^^^^^^^^^

The factory function :cpp:func:`jpeg_new_decoder_engine`, :cpp:func:`jpeg_decoder_get_info`, :cpp:func:`jpeg_decoder_process`, and :cpp:func:`jpeg_del_decoder_engine` are guaranteed to be thread safe by the driver, which means, user can call them from different RTOS tasks without protection by extra locks.

Kconfig Options
^^^^^^^^^^^^^^^

- :ref:`CONFIG_JPEG_ENABLE_DEBUG_LOG` is used to enable the debug log at the cost of increased firmware binary size.

API Reference
-------------

.. only:: SOC_JPEG_DECODE_SUPPORTED

    .. include-build-file:: inc/jpeg_decode.inc

.. only:: SOC_JPEG_ENCODE_SUPPORTED

    .. include-build-file:: inc/jpeg_encode.inc

.. include-build-file:: inc/components/esp_driver_jpeg/include/driver/jpeg_types.inc
.. include-build-file:: inc/components/hal/include/hal/jpeg_types.inc