Merge branch 'doc/multi_target_inc_dir' into 'master'

docs: multi target include dir support

Closes IDF-1371

See merge request espressif/esp-idf!7763

docs/en/api-guides/jtag-debugging/tips-and-quirks.rst

@@ -69,9 +69,10 @@ FreeRTOS support
 OpenOCD has explicit support for the ESP-IDF FreeRTOS. GDB can see FreeRTOS tasks as threads. Viewing them all can be done using the GDB ``i threads`` command, changing to a certain task is done with ``thread n``, with ``n`` being the number of the thread. FreeRTOS detection can be disabled in target's configuration. For more details see :ref:`jtag-debugging-tip-openocd-configure-target`.
 
 
-.. _jtag-debugging-tip-code-flash-voltage:
 
-.. only:: esp33
+.. only:: esp32
+
+    .. _jtag-debugging-tip-code-flash-voltage:
 
     Why to set SPI flash voltage in OpenOCD configuration?
     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -83,7 +84,11 @@ OpenOCD has explicit support for the ESP-IDF FreeRTOS. GDB can see FreeRTOS task
     Check specification of ESP32 module connected to JTAG, what is the power supply voltage of SPI flash chip. Then set ``ESP32_FLASH_VOLTAGE`` accordingly. Most WROOM modules use 3.3 V flash, while WROVER modules use 1.8 V flash.
 
 
-.. _jtag-debugging-tip-optimize-jtag-speed:
+    .. _jtag-debugging-tip-optimize-jtag-speed:
+
+.. only:: esp32s2
+
+    .. _jtag-debugging-tip-optimize-jtag-speed:
 
 Optimize JTAG speed
 ^^^^^^^^^^^^^^^^^^^
@@ -156,19 +161,18 @@ Disable RTOS support
 
 Comment out this line to have RTOS support.
 
-
-Power supply voltage of ESP32's SPI flash chip
-""""""""""""""""""""""""""""""""""""""""""""""
-
-::
-
-    set ESP32_FLASH_VOLTAGE 1.8
-
-Comment out this line to set 3.3 V, ref: :ref:`jtag-debugging-tip-code-flash-voltage`
-
-
 .. only:: esp32
 
+    Power supply voltage of ESP32's SPI flash chip
+    """"""""""""""""""""""""""""""""""""""""""""""
+
+    ::
+
+        set ESP32_FLASH_VOLTAGE 1.8
+
+    Comment out this line to set 3.3 V, ref: :ref:`jtag-debugging-tip-code-flash-voltage`
+
+
     Configuration file for ESP32 targets
     """"""""""""""""""""""""""""""""""""
 

docs/en/api-reference/system/intr_alloc.rst

@@ -10,9 +10,9 @@ Overview
   to the interrupt mux. Because there are more interrupt sources than interrupts, sometimes it makes sense to share an interrupt in
   multiple drivers. The esp_intr_alloc abstraction exists to hide all these implementation details.
 
-.. only.. esp32s2
+.. only:: esp32s2
 
-  The {IDF_TARGET_NAME} has one cores, with 32 interrupts each. Each interrupt has a certain priority level, most (but not all) interrupts are connected
+  The {IDF_TARGET_NAME} has one core, with 32 interrupts. Each interrupt has a certain priority level, most (but not all) interrupts are connected
   to the interrupt mux. Because there are more interrupt sources than interrupts, sometimes it makes sense to share an interrupt in
   multiple drivers. The esp_intr_alloc abstraction exists to hide all these implementation details.
 

docs/en/api-reference/system/system_time.rst

@@ -19,16 +19,7 @@ The settings for the system time source are as follows:
 - High-resolution timer
 - None
 
-..
-    (Implementation note: Using only blocks in this file not IDF_CONFIG_CFG_PREFIX, until the zh_CN translation for this document is done.)
-
-.. only:: esp32
-
-    It is recommended to stick to the default setting which provides maximum accuracy. If you want to choose a different timer, configure :ref:`CONFIG_ESP32_TIME_SYSCALL` in project configuration.
-
-.. only:: esp32s2
-
-    It is recommended to stick to the default setting which provides maximum accuracy. If you want to choose a different timer, configure :ref:`CONFIG_ESP32S2_TIME_SYSCALL` in project configuration.
+It is recommended to stick to the default setting which provides maximum accuracy. If you want to choose a different timer, configure :ref:`CONFIG_{IDF_TARGET_CFG_PREFIX}_TIME_SYSCALL` in project configuration.
 
 
 RTC Clock Source
@@ -44,13 +35,7 @@ The RTC timer has the following clock sources:
 
 - ``Internal 8.5MHz oscillator, divided by 256 (~33kHz)``. Provides better frequency stability than the ``internal 150kHz RC oscillator`` at the expense of higher (by 5 uA) deep sleep current consumption. It also does not require external components.
 
-.. only:: esp32
-
-    The choice depends on your requirements for system time accuracy and power consumption in sleep modes. To modify the RTC clock source, set :ref:`CONFIG_ESP32_RTC_CLK_SRC` in project configuration.
-
-.. only:: esp32s2
-
-    The choice depends on your requirements for system time accuracy and power consumption in sleep modes. To modify the RTC clock source, set :ref:`CONFIG_ESP32S2_RTC_CLK_SRC` in project configuration.
+The choice depends on your requirements for system time accuracy and power consumption in sleep modes. To modify the RTC clock source, set :ref:`CONFIG_{IDF_TARGET_CFG_PREFIX}_RTC_CLK_SRC` in project configuration.
 
 More details on wiring requirements for the ``External 32kHz crystal`` and ``External 32kHz oscillator at 32K_XP pin`` sources can be found in Section 2.1.4 *Crystal Oscillator* of `Hardware Design Guidelines <https://www.espressif.com/sites/default/files/documentation/esp32_hardware_design_guidelines_en.pdf#page=10>`_.
 

docs/en/contribute/add-ons-reference.rst

@@ -157,9 +157,10 @@ Other Extensions
 
     This will define a replacement of the tag {\IDF_TARGET_TX_PIN} in the current rst-file.
 
+    The extension also overrides the default ``.. include::`` directive in order to format any included content using the same rules.
+
     These replacements cannot be used inside markup that rely on alignment of characters, e.g. tables.
 
-    These replacement can't be used in a file which is `::include`-ed from another file. *This includes any English document where the ``zh_CN`` translation includes then ``en`` translation*.
 
 Related Documents
 -----------------

docs/en/contribute/documenting-code.rst

@@ -255,7 +255,7 @@ Exclusion of content based on chip-target
 Occasionally there will be content that is only relevant for one of targets. When this is the case, you can exclude that content by using the ''.. only:: TARGET'' directive, where you replace 'TARGET' with one of the chip names. As of now the following targets are available:
 
 * esp32
-* esp32s2 
+* esp32s2
 
 Example:
 
@@ -267,7 +267,30 @@ Example:
 
 This functionality is provided by the `Sphinx selective exclude <https://github.com/pfalcon/sphinx_selective_exclude>`_ extension.
 
-The '':TARGET:'' role is used for excluding content from a table of content tree. For example:
+A weakness in this extension is that it does not correctly handle the case were you exclude a section, and that is directly followed by a labeled new section. In these cases everything will render correctly, but the label will not correctly link to the section that follows. A temporary work-around for the cases were this can't be avoided is the following:
+
+.. code-block:: none
+
+    .. only:: esp32
+
+        .. _section_1_label:
+
+        Section 1
+        ^^^^^^^^^
+
+        Section one content
+
+        .. _section_2_label:
+
+    .. only:: esp32s2
+
+        _section_2_label:
+
+    Section 2
+    ^^^^^^^^^
+    Section 2 content
+
+The :TARGET: role is used for excluding content from a table of content tree. For example:
 
 .. code-block:: none
 
@@ -303,8 +326,6 @@ This extension also supports markup for defining a local (for a single .rst-file
 
 {\IDF_TARGET_TX_PIN:default="IO3",esp32="IO4",esp32s2beta="IO5"} will define a substitution for the tag {\IDF_TARGET_TX_PIN}, which would be replaced by the text IO5 if sphinx was called with the tag esp32s2beta.
 
-.. note:: Due to limitations in Sphinx processing, these substitutions are not applied to any document that is included via the ``.. include::` directive. In these cases it's necessary to use the ``only`` blocks and write per-target sections instead. Unfortunately this includes any document which is not yet translated, as the ``zh_CN`` version will include the ``en`` version.
-
 Put it all together
 -------------------
 

docs/idf_extensions/format_idf_target.py

@@ -1,14 +1,24 @@
 import re
+import os
+import os.path
+from docutils import io, nodes, statemachine, utils
+from docutils.utils.error_reporting import SafeString, ErrorString
+from docutils.parsers.rst import directives
+from sphinx.directives.other import Include as BaseInclude
 
-TARGET_NAMES = {'esp32': 'ESP32', 'esp32s2': 'ESP32-S2'}
-TOOLCHAIN_NAMES = {'esp32': 'esp', 'esp32s2': 'esp32s2'}
-CONFIG_PREFIX = {'esp32': 'ESP32', 'esp32s2': 'ESP32S2'}
 
-TRM_EN_URL = {'esp32': 'https://www.espressif.com/sites/default/files/documentation/esp32_technical_reference_manual_en.pdf',
-              'esp32s2': 'https://www.espressif.com/sites/default/files/documentation/esp32-s2_technical_reference_manual_en.pdf'}
+def setup(app):
+    sub = StringSubstituter()
 
-TRM_CN_URL = {'esp32': 'https://www.espressif.com/sites/default/files/documentation/esp32_technical_reference_manual_cn.pdf',
-              'esp32s2': 'https://www.espressif.com/sites/default/files/documentation/esp32-s2_technical_reference_manual_cn.pdf'}
+    # Config values not available when setup is called
+    app.connect('config-inited', lambda _, config: sub.init_sub_strings(config))
+    app.connect('source-read', sub.substitute_source_read_cb)
+
+    # Override the default include directive to include formatting with idf_target
+    # This is needed since there are no source-read events for includes
+    app.add_directive('include', FormatedInclude, override=True)
+
+    return {'parallel_read_safe': True, 'parallel_write_safe': True, 'version': '0.2'}
 
 
 class StringSubstituter:
@@ -29,6 +39,15 @@ class StringSubstituter:
         This will define a replacement of the tag {IDF_TARGET_TX_PIN} in the current rst-file, see e.g. uart.rst for example
 
     """
+    TARGET_NAMES = {'esp32': 'ESP32', 'esp32s2': 'ESP32-S2'}
+    TOOLCHAIN_NAMES = {'esp32': 'esp', 'esp32s2': 'esp32s2'}
+    CONFIG_PREFIX = {'esp32': 'ESP32', 'esp32s2': 'ESP32S2'}
+
+    TRM_EN_URL = {'esp32': 'https://www.espressif.com/sites/default/files/documentation/esp32_technical_reference_manual_en.pdf',
+                  'esp32s2': 'https://www.espressif.com/sites/default/files/documentation/esp32-s2_technical_reference_manual_en.pdf'}
+
+    TRM_CN_URL = {'esp32': 'https://www.espressif.com/sites/default/files/documentation/esp32_technical_reference_manual_cn.pdf',
+                  'esp32s2': 'https://www.espressif.com/sites/default/files/documentation/esp32-s2_technical_reference_manual_cn.pdf'}
     RE_PATTERN = re.compile(r'^\s*{IDF_TARGET_(\w+?):(.+?)}', re.MULTILINE)
 
     def __init__(self):
@@ -38,15 +57,15 @@ class StringSubstituter:
     def add_pair(self, tag, replace_value):
         self.substitute_strings[tag] = replace_value
 
-    def init_sub_strings(self, app, config):
+    def init_sub_strings(self, config):
         self.target_name = config.idf_target
 
-        self.add_pair("{IDF_TARGET_NAME}", TARGET_NAMES[config.idf_target])
+        self.add_pair("{IDF_TARGET_NAME}", self.TARGET_NAMES[config.idf_target])
         self.add_pair("{IDF_TARGET_PATH_NAME}", config.idf_target)
-        self.add_pair("{IDF_TARGET_TOOLCHAIN_NAME}", TOOLCHAIN_NAMES[config.idf_target])
-        self.add_pair("{IDF_TARGET_CFG_PREFIX}", CONFIG_PREFIX[config.idf_target])
-        self.add_pair("{IDF_TARGET_TRM_EN_URL}", TRM_EN_URL[config.idf_target])
-        self.add_pair("{IDF_TARGET_TRM_CN_URL}", TRM_CN_URL[config.idf_target])
+        self.add_pair("{IDF_TARGET_TOOLCHAIN_NAME}", self.TOOLCHAIN_NAMES[config.idf_target])
+        self.add_pair("{IDF_TARGET_CFG_PREFIX}", self.CONFIG_PREFIX[config.idf_target])
+        self.add_pair("{IDF_TARGET_TRM_EN_URL}", self.TRM_EN_URL[config.idf_target])
+        self.add_pair("{IDF_TARGET_TRM_CN_URL}", self.TRM_CN_URL[config.idf_target])
 
     def add_local_subs(self, matches):
 
@@ -71,30 +90,122 @@ class StringSubstituter:
 
             self.local_sub_strings[tag] = sub_value
 
-    def substitute(self, app, docname, source):
+    def substitute(self, content):
         # Add any new local tags that matches the reg.ex.
-        sub_defs = re.findall(self.RE_PATTERN, source[0])
+        sub_defs = re.findall(self.RE_PATTERN, content)
 
         if len(sub_defs) != 0:
             self.add_local_subs(sub_defs)
 
         # Remove the tag defines
-        source[0] = re.sub(self.RE_PATTERN,'', source[0])
+        content = re.sub(self.RE_PATTERN,'', content)
 
         for key in self.local_sub_strings:
-            source[0] = source[0].replace(key, self.local_sub_strings[key])
+            content = content.replace(key, self.local_sub_strings[key])
 
         self.local_sub_strings = {}
 
         for key in self.substitute_strings:
-            source[0] = source[0].replace(key, self.substitute_strings[key])
+            content = content.replace(key, self.substitute_strings[key])
+
+        return content
+
+    def substitute_source_read_cb(self, app, docname, source):
+        source[0] = self.substitute(source[0])
 
 
-def setup(app):
-    sub = StringSubstituter()
+class FormatedInclude(BaseInclude):
 
-    # Config values not available when setup is called
-    app.connect('config-inited', sub.init_sub_strings)
-    app.connect('source-read', sub.substitute)
+    """
+    Include and format content read from a separate source file.
 
-    return {'parallel_read_safe': True, 'parallel_write_safe': True, 'version': '0.1'}
+    Code is based on the default include directive from docutils
+    but extended to also format the content according to IDF target.
+
+    """
+    def run(self):
+
+        # For code or literal include blocks we run the normal include
+        if 'literal' in self.options or 'code' in self.options:
+            return super(FormatedInclude, self).run()
+
+        """Include a file as part of the content of this reST file."""
+        if not self.state.document.settings.file_insertion_enabled:
+            raise self.warning('"%s" directive disabled.' % self.name)
+        source = self.state_machine.input_lines.source(
+            self.lineno - self.state_machine.input_offset - 1)
+
+        source_dir = os.path.dirname(os.path.abspath(source))
+
+        rel_filename, filename = self.env.relfn2path(self.arguments[0])
+        self.arguments[0] = filename
+        self.env.note_included(filename)
+        path = directives.path(self.arguments[0])
+
+        if path.startswith('<') and path.endswith('>'):
+            path = os.path.join(self.standard_include_path, path[1:-1])
+        path = os.path.normpath(os.path.join(source_dir, path))
+
+        path = utils.relative_path(None, path)
+        path = nodes.reprunicode(path)
+
+        encoding = self.options.get(
+            'encoding', self.state.document.settings.input_encoding)
+        e_handler = self.state.document.settings.input_encoding_error_handler
+        tab_width = self.options.get(
+            'tab-width', self.state.document.settings.tab_width)
+        try:
+            self.state.document.settings.record_dependencies.add(path)
+            include_file = io.FileInput(source_path=path,
+                                        encoding=encoding,
+                                        error_handler=e_handler)
+        except UnicodeEncodeError:
+            raise self.severe(u'Problems with "%s" directive path:\n'
+                              'Cannot encode input file path "%s" '
+                              '(wrong locale?).' %
+                              (self.name, SafeString(path)))
+        except IOError as error:
+            raise self.severe(u'Problems with "%s" directive path:\n%s.' %
+                              (self.name, ErrorString(error)))
+        startline = self.options.get('start-line', None)
+        endline = self.options.get('end-line', None)
+        try:
+            if startline or (endline is not None):
+                lines = include_file.readlines()
+                rawtext = ''.join(lines[startline:endline])
+            else:
+                rawtext = include_file.read()
+        except UnicodeError as error:
+            raise self.severe(u'Problem with "%s" directive:\n%s' %
+                              (self.name, ErrorString(error)))
+
+        # Format input
+        sub = StringSubstituter()
+        config = self.state.document.settings.env.config
+        sub.init_sub_strings(config)
+        rawtext = sub.substitute(rawtext)
+
+        # start-after/end-before: no restrictions on newlines in match-text,
+        # and no restrictions on matching inside lines vs. line boundaries
+        after_text = self.options.get('start-after', None)
+        if after_text:
+            # skip content in rawtext before *and incl.* a matching text
+            after_index = rawtext.find(after_text)
+            if after_index < 0:
+                raise self.severe('Problem with "start-after" option of "%s" '
+                                  'directive:\nText not found.' % self.name)
+            rawtext = rawtext[after_index + len(after_text):]
+        before_text = self.options.get('end-before', None)
+        if before_text:
+            # skip content in rawtext after *and incl.* a matching text
+            before_index = rawtext.find(before_text)
+            if before_index < 0:
+                raise self.severe('Problem with "end-before" option of "%s" '
+                                  'directive:\nText not found.' % self.name)
+            rawtext = rawtext[:before_index]
+
+        include_lines = statemachine.string2lines(rawtext, tab_width,
+                                                  convert_whitespace=True)
+
+        self.state_machine.insert_input(include_lines, path)
+        return []

docs/zh_CN/api-guides/jtag-debugging/index.rst

@@ -276,19 +276,10 @@ Windows 用户::
 
 本节列出了本指南中提到的所有注意事项和补充内容的链接。
 
-* :ref:`jtag-debugging-tip-breakpoints`
-* :ref:`jtag-debugging-tip-where-breakpoints`
-* :ref:`jtag-debugging-tip-flash-mappings`
-* :ref:`jtag-debugging-tip-why-next-works-as-step`
-* :ref:`jtag-debugging-tip-code-options`
-* :ref:`jtag-debugging-tip-freertos-support`
-* :ref:`jtag-debugging-tip-code-flash-voltage`
-* :ref:`jtag-debugging-tip-optimize-jtag-speed`
-* :ref:`jtag-debugging-tip-debugger-startup-commands`
-* :ref:`jtag-debugging-tip-openocd-configure-target`
-* :ref:`jtag-debugging-tip-reset-by-debugger`
-* :ref:`jtag-debugging-tip-jtag-pins-reconfigured`
-* :ref:`jtag-debugging-tip-reporting-issues`
+.. toctree::
+    :maxdepth: 2
+
+    tips-and-quirks
 
 
 相关文档

docs/zh_CN/api-guides/jtag-debugging/tips-and-quirks.rst

@@ -67,10 +67,9 @@ ESP-IDF 有一些针对 OpenOCD 调试功能的选项可以在编译时进行设
 
 OpenOCD 完全支持 ESP-IDF 自带的 FreeRTOS 操作系统，GDB 会将 FreeRTOS 中的任务当做线程。使用 GDB 命令 ``i threads`` 可以查看所有的线程，使用命令 ``thread n`` 可以切换到某个具体任务的堆栈，其中 ``n`` 是线程的编号。检测 FreeRTOS 的功能可以在配置目标时被禁用。更多详细信息，请参阅 :ref:`jtag-debugging-tip-openocd-configure-target`.
 
+.. only:: esp32
 
-.. _jtag-debugging-tip-code-flash-voltage:
-
-.. only:: esp33
+    .. _jtag-debugging-tip-code-flash-voltage:
 
     在 OpenOCD 的配置文件中设置 SPI 闪存的工作电压
     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -81,8 +80,11 @@ OpenOCD 完全支持 ESP-IDF 自带的 FreeRTOS 操作系统，GDB 会将 FreeRT
 
     查看 JTAG 连接的 ESP32 模组的规格书，检查其 SPI 闪存芯片的供电电压值，然后再相应的设置 ``ESP32_FLASH_VOLTAGE``。大多数的 WROOM 模组使用 3.3 V 的闪存芯片，但是 WROVER 模组使用 1.8 V 的闪存芯片。
 
+    .. _jtag-debugging-tip-optimize-jtag-speed:
 
-.. _jtag-debugging-tip-optimize-jtag-speed:
+.. only:: esp32s2
+
+    .. _jtag-debugging-tip-optimize-jtag-speed:
 
 优化 JTAG 的速度
 ^^^^^^^^^^^^^^^^
@@ -156,24 +158,23 @@ OpenOCD 需要知道当前使用的 JTAG 适配器的类型，以及其连接的
 
 如果要支持 RTOS， 请注释掉这一行。
 
-
-ESP32 的 SPI 闪存芯片的电源电压
-"""""""""""""""""""""""""""""""
-
-::
-
-    set ESP32_FLASH_VOLTAGE 1.8
-
-如果 SPI 闪存芯片的电源电压为 3.3 V， 请注释掉这一行，更多信息请参阅： :ref:`jtag-debugging-tip-code-flash-voltage`。
-
-
-
-
-ESP32 的目标配置文件
-""""""""""""""""""""
-
 .. only:: esp32
 
+    ESP32 的 SPI 闪存芯片的电源电压
+    """"""""""""""""""""""""""""""""""
+
+    ::
+
+        set ESP32_FLASH_VOLTAGE 1.8
+
+    如果 SPI 闪存芯片的电源电压为 3.3 V， 请注释掉这一行，更多信息请参阅： :ref:`jtag-debugging-tip-code-flash-voltage`。
+
+
+
+
+    ESP32 的目标配置文件
+    """"""""""""""""""""
+
     ::
 
         source [find target/esp32.cfg]

docs/zh_CN/api-reference/system/power_management.rst

@@ -33,13 +33,7 @@ ESP-IDF 中集成的电源管理算法可以根据应用程序组件的需求，
 
 - ``light_sleep_enable``：没有获取任何管理锁时，决定系统是否需要自动进入 Light-sleep 状态 (``true``/``false``)。
 
-.. only:: esp32
-
-    或者，如果在 menuconfig 中启用了 :ref:`CONFIG_PM_DFS_INIT_AUTO` 选项，最大 CPU 频率将由 :ref:`CONFIG_ESP32_DEFAULT_CPU_FREQ_MHZ` 设置决定，最小 CPU 频率将锁定为 XTAL 频率。
-
-.. only:: esp32s2
-
-    或者，如果在 menuconfig 中启用了 :ref:`CONFIG_PM_DFS_INIT_AUTO` 选项，最大 CPU 频率将由 :ref:`CONFIG_ESP32S2_DEFAULT_CPU_FREQ_MHZ` 设置决定，最小 CPU 频率将锁定为 XTAL 频率。
+或者，如果在 menuconfig 中启用了 :ref:`CONFIG_PM_DFS_INIT_AUTO` 选项，最大 CPU 频率将由 :ref:`CONFIG_{IDF_TARGET_CFG_PREFIX}_DEFAULT_CPU_FREQ_MHZ` 设置决定，最小 CPU 频率将锁定为 XTAL 频率。
 
 .. note::