myst_parser.mdit_to_docutils.base

Convert Markdown-it tokens to docutils nodes.

1.  Module Contents

1.1.  Classes

DocutilsRenderer

A markdown-it-py renderer to populate (in-place) a docutils.document AST.

1.2.  Functions

make_document	Create a new docutils document, with the parser classes’ default settings.
token_line	Retrieve the initial line of a token.
html_meta_to_nodes	Replicate the meta directive, by converting a dictionary to a list of pending meta nodes
clean_astext	Like node.astext(), but ignore images. Copied from sphinx.
default_slugify	Default slugify function.
compute_unique_slug	Compute the slug for a token.

1.3.  Data

REGEX_SCHEME	RFC 7595: A non-empty scheme component followed by a colon (:), consisting of a sequence of characters beginning with a letter and followed by any combination of letters, digits, plus (+), period (.), or hyphen (-). Although schemes are case-insensitive, the canonical form is lowercase and documents that specify schemes must do so with lowercase letters.
REGEX_URI_TEMPLATE
REGEX_DIRECTIVE_START

1.4.  API

myst_parser.mdit_to_docutils.base.make_document(source_path='notset', parser_cls=RSTParser) → docutils.nodes.document

Create a new docutils document, with the parser classes’ default settings.

myst_parser.mdit_to_docutils.base.REGEX_SCHEME = 'compile(...)'

RFC 7595: A non-empty scheme component followed by a colon (:), consisting of a sequence of characters beginning with a letter and followed by any combination of letters, digits, plus (+), period (.), or hyphen (-). Although schemes are case-insensitive, the canonical form is lowercase and documents that specify schemes must do so with lowercase letters.

myst_parser.mdit_to_docutils.base.REGEX_URI_TEMPLATE = 'compile(...)'

myst_parser.mdit_to_docutils.base.REGEX_DIRECTIVE_START = 'compile(...)'

myst_parser.mdit_to_docutils.base.token_line(token: markdown_it.tree.SyntaxTreeNode, default: int | None = None) → int

Retrieve the initial line of a token.

class myst_parser.mdit_to_docutils.base.DocutilsRenderer(parser: markdown_it.main.MarkdownIt)

Bases: markdown_it.renderer.RendererProtocol

A markdown-it-py renderer to populate (in-place) a docutils.document AST.

Note, this render is not dependent on Sphinx.

Initialization

Load the renderer (called by MarkdownIt)

setup_render(options: dict[str, Any], env: collections.abc.MutableMapping[str, Any]) → None

Setup the renderer with per render variables.

property sphinx_env: sphinx.environment.BuildEnvironment | None

Return the sphinx env, if using Sphinx.

create_warning(message: str, subtype: myst_parser.warnings_.MystWarnings | str, *, wtype: str | None = None, line: int | None = None, append_to: docutils.nodes.Element | None = None) → docutils.nodes.system_message | None

Generate a warning, logging if it is necessary.

If the warning type is listed in the suppress_warnings configuration, then None will be returned and no warning logged.

render(tokens: collections.abc.Sequence[markdown_it.token.Token], options, md_env: collections.abc.MutableMapping[str, Any]) → docutils.nodes.document

Run the render on a token stream.

Parameters:

• tokens – list on block tokens to render

• options – params of parser instance

• md_env – the markdown-it environment sandbox associated with the tokens, containing additional metadata like reference info

nested_render_text(text: str, lineno: int, inline: bool = False, temp_root_node: None | docutils.nodes.Element = None, heading_offset: int = 0) → None

Render unparsed text (appending to the current node).

Parameters:

• text – the text to render

• lineno – the starting line number of the text, within the full source

• inline – whether the text is inline or block

• temp_root_node – If set, allow sections to be created as children of this node

• heading_offset – offset heading levels by this amount

current_node_context(node: docutils.nodes.Element, append: bool = False) → collections.abc.Iterator[None]

Context manager for temporarily setting the current node.

render_children(token: markdown_it.tree.SyntaxTreeNode) → None

Render the children of a token.

add_line_and_source_path(node, token: markdown_it.tree.SyntaxTreeNode) → None

Copy the line number and document source path to the docutils node.

add_line_and_source_path_r(nodes_: list[docutils.nodes.Element], token: markdown_it.tree.SyntaxTreeNode) → None

Copy the line number and document source path to the docutils nodes, and recursively to all descendants.

copy_attributes(token: markdown_it.tree.SyntaxTreeNode, node: docutils.nodes.Element, keys: collections.abc.Sequence[str] = ('class',), *, converters: dict[str, collections.abc.Callable[[str], Any]] | None = None, aliases: dict[str, str] | None = None) → None

Copy attributes on the token to the docutils node.

Parameters:

• token – the token to copy attributes from

• node – the node to copy attributes to

• keys – the keys to copy from the token (after aliasing)

• converters – a dictionary of converters for the attributes

• aliases – a dictionary mapping the token key name to the node key name

update_section_level_state(section: docutils.nodes.section, level: int) → None

Update the section level state, with the new current section and level.

renderInlineAsText(tokens: list[markdown_it.tree.SyntaxTreeNode]) → str

Special kludge for image alt attributes to conform CommonMark spec.

Don’t try to use it! Spec requires to show alt content with stripped markup, instead of simple escaping.

render_paragraph(token: markdown_it.tree.SyntaxTreeNode) → None

render_inline(token: markdown_it.tree.SyntaxTreeNode) → None

render_text(token: markdown_it.tree.SyntaxTreeNode) → None

render_bullet_list(token: markdown_it.tree.SyntaxTreeNode) → None

render_ordered_list(token: markdown_it.tree.SyntaxTreeNode) → None

render_list_item(token: markdown_it.tree.SyntaxTreeNode) → None

render_em(token: markdown_it.tree.SyntaxTreeNode) → None

render_softbreak(token: markdown_it.tree.SyntaxTreeNode) → None

render_hardbreak(token: markdown_it.tree.SyntaxTreeNode) → None

render_strong(token: markdown_it.tree.SyntaxTreeNode) → None

render_blockquote(token: markdown_it.tree.SyntaxTreeNode) → None

render_hr(token: markdown_it.tree.SyntaxTreeNode) → None

render_code_inline(token: markdown_it.tree.SyntaxTreeNode) → None

create_highlighted_code_block(text: str, lexer_name: str | None, number_lines: bool = False, lineno_start: int = 1, source: str | None = None, line: int | None = None, node_cls: type[docutils.nodes.Element] = nodes.literal_block, emphasize_lines: list[int] | str | None = None) → docutils.nodes.Element

Create a literal block with syntax highlighting.

This mimics the behaviour of the code-block directive.

In docutils, this directive directly parses the text with the pygments lexer, whereas in sphinx, the lexer name is only recorded as the language attribute, and the text is lexed later by pygments within the visit_literal_block method of the output format SphinxTranslator.

Note, this function does not add the literal block to the document.

render_code_block(token: markdown_it.tree.SyntaxTreeNode) → None

render_fence(token: markdown_it.tree.SyntaxTreeNode) → None

Render a fenced code block.

property blocks_mathjax_processing: bool

Only add mathjax ignore classes if using sphinx, and using the dollarmath extension, and myst_update_mathjax=True.

generate_heading_target(token: markdown_it.tree.SyntaxTreeNode, level: int, node: docutils.nodes.Element, title_node: docutils.nodes.Element) → None

Generate a heading target, and add it to the document.

render_heading(token: markdown_it.tree.SyntaxTreeNode) → None

Render a heading, e.g. # Heading.

render_link(token: markdown_it.tree.SyntaxTreeNode) → None

Parse <http://link.com> or [text](link “title”) syntax to docutils AST:

• If myst_all_links_external is True, forward to render_link_url

• If the link token has a class attribute containing external,

  forward to render_link_url

• If the link is an id link (e.g. #id), forward to render_link_anchor

• If the link has a schema, and the schema is in url_schemes (e.g. http:), forward to render_link_url

• If the link has an inv: schema, forward to render_link_inventory

• If the link is an autolink/linkify type link, forward to render_link_url

• Otherwise, forward to render_link_internal

render_link_url(token: markdown_it.tree.SyntaxTreeNode, conversion: None | myst_parser.config.main.UrlSchemeType = None) → None

Render link token (including autolink and linkify), where the link has been identified as an external URL.

render_link_path(token: markdown_it.tree.SyntaxTreeNode) → None

Render a link token like <path:…>.

render_link_project(token: markdown_it.tree.SyntaxTreeNode) → None

Render a link token like <project:…>.

render_link_anchor(token: markdown_it.tree.SyntaxTreeNode, target: str) → None

Render link token like [text](#target), to a local target.

Target:

the target id, e.g. #target

render_link_unknown(token: markdown_it.tree.SyntaxTreeNode) → None

Render link token [text](link “title”), where the link has not been identified as an external URL:

<reference refname="link" title="title">
    text

text can contain nested syntax, e.g. [**bold**](link “title”).

Note, this is overridden by SphinxRenderer, to use pending_xref nodes.

render_link_inventory(token: markdown_it.tree.SyntaxTreeNode) → None

Create a link to an inventory object.

This assumes the href is of the form <scheme>:<path>#<target>. The path is of the form <invs>:<domains>:<otypes>, where each of the parts is optional, hence <scheme>:#<target> is also valid. Each of the path parts can contain the * wildcard, for example: <scheme>:key:*:obj#targe*. * is treated as a plain *.

get_inventory_matches(*, invs: str | None, domains: str | None, otypes: str | None, target: str | None) → list[myst_parser.inventory.InvMatch]

Return inventory matches.

This will be overridden for sphinx, to use intersphinx config.

render_html_inline(token: markdown_it.tree.SyntaxTreeNode) → None

render_html_block(token: markdown_it.tree.SyntaxTreeNode) → None

render_image(token: markdown_it.tree.SyntaxTreeNode) → None

render_span(token: markdown_it.tree.SyntaxTreeNode) → None

Render an inline span token.

render_front_matter(token: markdown_it.tree.SyntaxTreeNode) → None

Pass document front matter data.

dict_to_fm_field_list(data: dict[str, Any], language_code: str, line: int = 0) → docutils.nodes.field_list

Render each key/val pair as a docutils field_node.

Bibliographic keys below will be parsed as Markdown, all others will be left as literal text.

The field list should be at the start of the document, and will then be converted to a docinfo node during the docutils.docutils.transforms.frontmatter.DocInfo transform (priority 340), and bibliographic keys (or their translation) will be converted to nodes:

{'author': docutils.nodes.author,
'authors': docutils.nodes.authors,
'organization': docutils.nodes.organization,
'address': docutils.nodes.address,
'contact': docutils.nodes.contact,
'version': docutils.nodes.version,
'revision': docutils.nodes.revision,
'status': docutils.nodes.status,
'date': docutils.nodes.date,
'copyright': docutils.nodes.copyright,
'dedication': docutils.nodes.topic,
'abstract': docutils.nodes.topic}

Also, the ‘dedication’ and ‘abstract’ will be placed outside the docinfo, and so will always be shown in the document.

If using sphinx, this docinfo node will later be extracted from the AST, by the DoctreeReadEvent transform (priority 880), calling MetadataCollector.process_doc. In this case keys and values will be converted to strings and stored in app.env.metadata[app.env.docname]

See https://www.sphinx-doc.org/en/master/usage/restructuredtext/field-lists.html for docinfo fields used by sphinx.

render_table(token: markdown_it.tree.SyntaxTreeNode) → None

render_table_row(token: markdown_it.tree.SyntaxTreeNode) → None

render_s(token: markdown_it.tree.SyntaxTreeNode) → None

Render a strikethrough token.

render_math_inline(token: markdown_it.tree.SyntaxTreeNode) → None

render_math_inline_double(token: markdown_it.tree.SyntaxTreeNode) → None

render_math_single(token: markdown_it.tree.SyntaxTreeNode) → None

render_math_block(token: markdown_it.tree.SyntaxTreeNode) → None

render_math_block_label(token: markdown_it.tree.SyntaxTreeNode) → None

render_amsmath(token: markdown_it.tree.SyntaxTreeNode) → None

render_footnote_ref(token: markdown_it.tree.SyntaxTreeNode) → None

Footnote references are added as auto-numbered, .i.e. [^a] is read as rST [#a]_

render_footnote_reference(token: markdown_it.tree.SyntaxTreeNode) → None

Despite the name, this is actually a footnote definition, e.g. [^a]: …

render_myst_block_break(token: markdown_it.tree.SyntaxTreeNode) → None

render_myst_target(token: markdown_it.tree.SyntaxTreeNode) → None

render_myst_line_comment(token: markdown_it.tree.SyntaxTreeNode) → None

render_myst_role(token: markdown_it.tree.SyntaxTreeNode) → None

render_colon_fence(token: markdown_it.tree.SyntaxTreeNode) → None

Render a div block, with : colon delimiters.

render_dl(token: markdown_it.tree.SyntaxTreeNode) → None

Render a definition list.

render_field_list(token: markdown_it.tree.SyntaxTreeNode) → None

Render a field list.

render_restructuredtext(token: markdown_it.tree.SyntaxTreeNode) → None

Render the content of the token as restructuredtext.

render_directive(token: markdown_it.tree.SyntaxTreeNode, name: str, arguments: str, *, additional_options: dict[str, str] | None = None) → None

Render special fenced code blocks as directives.

Parameters:

• token – the token to render

• name – the name of the directive

• arguments – The remaining text on the same line as the directive name.

run_directive(name: str, first_line: str, content: str, position: int, additional_options: dict[str, str] | None = None) → list[docutils.nodes.Element]

Run a directive and return the generated nodes.

Parameters:

• name – the name of the directive

• first_line – The text on the same line as the directive name. May be an argument or body text, dependent on the directive

• content – All text after the first line. Can include options.

• position – The line number of the first line

• additional_options – Additional options to add to the directive, above those parsed from the content.

render_substitution_inline(token: markdown_it.tree.SyntaxTreeNode) → None

Render inline substitution {{key}}.

render_substitution_block(token: markdown_it.tree.SyntaxTreeNode) → None

Render block substitution {{key}}.

render_substitution(token: markdown_it.tree.SyntaxTreeNode, inline: bool) → None

Substitutions are rendered by:

1. Combining global substitutions with front-matter substitutions to create a variable context (front-matter takes priority)

2. Add the sphinx env to the variable context (if available)

3. Create the string content with Jinja2 (passing it the variable context)

4. If the substitution is inline and not a directive, parse to nodes ignoring block syntaxes (like lists or block-quotes), otherwise parse to nodes with all syntax rules.

myst_parser.mdit_to_docutils.base.html_meta_to_nodes(data: dict[str, Any], document: docutils.nodes.document, line: int, reporter: docutils.utils.Reporter) → list[docutils.nodes.pending | docutils.nodes.system_message]

Replicate the meta directive, by converting a dictionary to a list of pending meta nodes

See: https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#html-metadata

myst_parser.mdit_to_docutils.base.clean_astext(node: docutils.nodes.Element) → str

Like node.astext(), but ignore images. Copied from sphinx.

myst_parser.mdit_to_docutils.base.default_slugify(title: str) → str

Default slugify function.

This aims to mimic the GitHub Markdown format, see:

• jch/html-pipeline

• https://gist.github.com/asabaylus/3071099

myst_parser.mdit_to_docutils.base.compute_unique_slug(token_tree: markdown_it.tree.SyntaxTreeNode, slugs: collections.abc.Iterable[str], slug_func: None | collections.abc.Callable[[str], str] = None) → str

Compute the slug for a token.

This directly mirrors the logic in mdit_py_plugins.anchors_plugin