myst_parser.mdit_to_docutils.base#

Convert Markdown-it tokens to docutils nodes.

1.  Module Contents#

1.1.  Classes#

DocutilsRenderer

A markdown-it-py renderer to populate (in-place) a docutils.document AST.

1.2.  Functions#

make_document

Create a new docutils document, with the parser classes’ default settings.

token_line

Retrieve the initial line of a token.

html_meta_to_nodes

Replicate the meta directive, by converting a dictionary to a list of pending meta nodes

clean_astext

Like node.astext(), but ignore images. Copied from sphinx.

default_slugify

Default slugify function.

compute_unique_slug

Compute the slug for a token.

1.3.  Data#

REGEX_SCHEME

RFC 7595: A non-empty scheme component followed by a colon (:), consisting of a sequence of characters beginning with a letter and followed by any combination of letters, digits, plus (+), period (.), or hyphen (-). Although schemes are case-insensitive, the canonical form is lowercase and documents that specify schemes must do so with lowercase letters.

REGEX_URI_TEMPLATE

REGEX_DIRECTIVE_START

1.4.  API#

myst_parser.mdit_to_docutils.base.make_document(source_path='notset', parser_cls=RSTParser) docutils.nodes.document[source]#

Create a new docutils document, with the parser classes’ default settings.

myst_parser.mdit_to_docutils.base.REGEX_SCHEME = 'compile(...)'#

RFC 7595: A non-empty scheme component followed by a colon (:), consisting of a sequence of characters beginning with a letter and followed by any combination of letters, digits, plus (+), period (.), or hyphen (-). Although schemes are case-insensitive, the canonical form is lowercase and documents that specify schemes must do so with lowercase letters.

myst_parser.mdit_to_docutils.base.REGEX_URI_TEMPLATE = 'compile(...)'#
myst_parser.mdit_to_docutils.base.REGEX_DIRECTIVE_START = 'compile(...)'#
myst_parser.mdit_to_docutils.base.token_line(token: markdown_it.tree.SyntaxTreeNode, default: int | None = None) int[source]#

Retrieve the initial line of a token.

class myst_parser.mdit_to_docutils.base.DocutilsRenderer(parser: markdown_it.main.MarkdownIt)[source]#

Bases: markdown_it.renderer.RendererProtocol

A markdown-it-py renderer to populate (in-place) a docutils.document AST.

Note, this render is not dependent on Sphinx.

Initialization

Load the renderer (called by MarkdownIt)

setup_render(options: dict[str, Any], env: MutableMapping[str, Any]) None[source]#

Setup the renderer with per render variables.

property sphinx_env: sphinx.environment.BuildEnvironment | None#

Return the sphinx env, if using Sphinx.

create_warning(message: str, subtype: myst_parser.warnings_.MystWarnings, *, line: int | None = None, append_to: docutils.nodes.Element | None = None) docutils.nodes.system_message | None[source]#

Generate a warning, logging if it is necessary.

If the warning type is listed in the suppress_warnings configuration, then None will be returned and no warning logged.

render(tokens: Sequence[markdown_it.token.Token], options, md_env: MutableMapping[str, Any]) docutils.nodes.document[source]#

Run the render on a token stream.

Parameters:
  • tokens – list on block tokens to render

  • options – params of parser instance

  • md_env – the markdown-it environment sandbox associated with the tokens, containing additional metadata like reference info

nested_render_text(text: str, lineno: int, inline: bool = False, temp_root_node: None | docutils.nodes.Element = None, heading_offset: int = 0) None[source]#

Render unparsed text (appending to the current node).

Parameters:
  • text – the text to render

  • lineno – the starting line number of the text, within the full source

  • inline – whether the text is inline or block

  • temp_root_node – If set, allow sections to be created as children of this node

  • heading_offset – offset heading levels by this amount

current_node_context(node: docutils.nodes.Element, append: bool = False) Iterator[None][source]#

Context manager for temporarily setting the current node.

render_children(token: markdown_it.tree.SyntaxTreeNode) None[source]#

Render the children of a token.

add_line_and_source_path(node, token: markdown_it.tree.SyntaxTreeNode) None[source]#

Copy the line number and document source path to the docutils node.

add_line_and_source_path_r(nodes_: list[docutils.nodes.Element], token: markdown_it.tree.SyntaxTreeNode) None[source]#

Copy the line number and document source path to the docutils nodes, and recursively to all descendants.

copy_attributes(token: markdown_it.tree.SyntaxTreeNode, node: docutils.nodes.Element, keys: Sequence[str] = ('class',), *, converters: dict[str, Callable[[str], Any]] | None = None, aliases: dict[str, str] | None = None) None[source]#

Copy attributes on the token to the docutils node.

Parameters:
  • token – the token to copy attributes from

  • node – the node to copy attributes to

  • keys – the keys to copy from the token (after aliasing)

  • converters – a dictionary of converters for the attributes

  • aliases – a dictionary mapping the token key name to the node key name

update_section_level_state(section: docutils.nodes.section, level: int) None[source]#

Update the section level state, with the new current section and level.

renderInlineAsText(tokens: list[markdown_it.tree.SyntaxTreeNode]) str[source]#

Special kludge for image alt attributes to conform CommonMark spec.

Don’t try to use it! Spec requires to show alt content with stripped markup, instead of simple escaping.

render_paragraph(token: markdown_it.tree.SyntaxTreeNode) None[source]#
render_inline(token: markdown_it.tree.SyntaxTreeNode) None[source]#
render_text(token: markdown_it.tree.SyntaxTreeNode) None[source]#
render_bullet_list(token: markdown_it.tree.SyntaxTreeNode) None[source]#
render_ordered_list(token: markdown_it.tree.SyntaxTreeNode) None[source]#
render_list_item(token: markdown_it.tree.SyntaxTreeNode) None[source]#
render_em(token: markdown_it.tree.SyntaxTreeNode) None[source]#
render_softbreak(token: markdown_it.tree.SyntaxTreeNode) None[source]#
render_hardbreak(token: markdown_it.tree.SyntaxTreeNode) None[source]#
render_strong(token: markdown_it.tree.SyntaxTreeNode) None[source]#
render_blockquote(token: markdown_it.tree.SyntaxTreeNode) None[source]#
render_hr(token: markdown_it.tree.SyntaxTreeNode) None[source]#
render_code_inline(token: markdown_it.tree.SyntaxTreeNode) None[source]#
create_highlighted_code_block(text: str, lexer_name: str | None, number_lines: bool = False, lineno_start: int = 1, source: str | None = None, line: int | None = None, node_cls: type[docutils.nodes.Element] = nodes.literal_block, emphasize_lines: list[int] | str | None = None) docutils.nodes.Element[source]#

Create a literal block with syntax highlighting.

This mimics the behaviour of the code-block directive.

In docutils, this directive directly parses the text with the pygments lexer, whereas in sphinx, the lexer name is only recorded as the language attribute, and the text is lexed later by pygments within the visit_literal_block method of the output format SphinxTranslator.

Note, this function does not add the literal block to the document.

render_code_block(token: markdown_it.tree.SyntaxTreeNode) None[source]#
render_fence(token: markdown_it.tree.SyntaxTreeNode) None[source]#

Render a fenced code block.

property blocks_mathjax_processing: bool#

Only add mathjax ignore classes if using sphinx, and using the dollarmath extension, and myst_update_mathjax=True.

generate_heading_target(token: markdown_it.tree.SyntaxTreeNode, level: int, node: docutils.nodes.Element, title_node: docutils.nodes.Element) None[source]#

Generate a heading target, and add it to the document.

render_heading(token: markdown_it.tree.SyntaxTreeNode) None[source]#

Render a heading, e.g. # Heading.

Parse <http://link.com> or [text](link β€œtitle”) syntax to docutils AST:

  • If myst_all_links_external is True, forward to render_link_url

  • If the link token has a class attribute containing external,

    forward to render_link_url

  • If the link is an id link (e.g. #id), forward to render_link_anchor

  • If the link has a schema, and the schema is in url_schemes (e.g. http:), forward to render_link_url

  • If the link has an inv: schema, forward to render_link_inventory

  • If the link is an autolink/linkify type link, forward to render_link_url

  • Otherwise, forward to render_link_internal

Render link token (including autolink and linkify), where the link has been identified as an external URL.

Render a link token like <path:…>.

Render a link token like <project:…>.

Render link token like [text](#target), to a local target.

Target:

the target id, e.g. #target

Render link token [text](link β€œtitle”), where the link has not been identified as an external URL:

<reference refname="link" title="title">
    text

text can contain nested syntax, e.g. [**bold**](link β€œtitle”).

Note, this is overridden by SphinxRenderer, to use pending_xref nodes.

Create a link to an inventory object.

This assumes the href is of the form <scheme>:<path>#<target>. The path is of the form <invs>:<domains>:<otypes>, where each of the parts is optional, hence <scheme>:#<target> is also valid. Each of the path parts can contain the * wildcard, for example: <scheme>:key:*:obj#targe*. * is treated as a plain *.

get_inventory_matches(*, invs: str | None, domains: str | None, otypes: str | None, target: str | None) list[myst_parser.inventory.InvMatch][source]#

Return inventory matches.

This will be overridden for sphinx, to use intersphinx config.

render_html_inline(token: markdown_it.tree.SyntaxTreeNode) None[source]#
render_html_block(token: markdown_it.tree.SyntaxTreeNode) None[source]#
render_image(token: markdown_it.tree.SyntaxTreeNode) None[source]#
render_span(token: markdown_it.tree.SyntaxTreeNode) None[source]#

Render an inline span token.

render_front_matter(token: markdown_it.tree.SyntaxTreeNode) None[source]#

Pass document front matter data.

dict_to_fm_field_list(data: dict[str, Any], language_code: str, line: int = 0) docutils.nodes.field_list[source]#

Render each key/val pair as a docutils field_node.

Bibliographic keys below will be parsed as Markdown, all others will be left as literal text.

The field list should be at the start of the document, and will then be converted to a docinfo node during the docutils.docutils.transforms.frontmatter.DocInfo transform (priority 340), and bibliographic keys (or their translation) will be converted to nodes:

{'author': docutils.nodes.author,
'authors': docutils.nodes.authors,
'organization': docutils.nodes.organization,
'address': docutils.nodes.address,
'contact': docutils.nodes.contact,
'version': docutils.nodes.version,
'revision': docutils.nodes.revision,
'status': docutils.nodes.status,
'date': docutils.nodes.date,
'copyright': docutils.nodes.copyright,
'dedication': docutils.nodes.topic,
'abstract': docutils.nodes.topic}

Also, the β€˜dedication’ and β€˜abstract’ will be placed outside the docinfo, and so will always be shown in the document.

If using sphinx, this docinfo node will later be extracted from the AST, by the DoctreeReadEvent transform (priority 880), calling MetadataCollector.process_doc. In this case keys and values will be converted to strings and stored in app.env.metadata[app.env.docname]

See https://www.sphinx-doc.org/en/master/usage/restructuredtext/field-lists.html for docinfo fields used by sphinx.

render_table(token: markdown_it.tree.SyntaxTreeNode) None[source]#
render_table_row(token: markdown_it.tree.SyntaxTreeNode) None[source]#
render_s(token: markdown_it.tree.SyntaxTreeNode) None[source]#

Render a strikethrough token.

render_math_inline(token: markdown_it.tree.SyntaxTreeNode) None[source]#
render_math_inline_double(token: markdown_it.tree.SyntaxTreeNode) None[source]#
render_math_single(token: markdown_it.tree.SyntaxTreeNode) None[source]#
render_math_block(token: markdown_it.tree.SyntaxTreeNode) None[source]#
render_math_block_label(token: markdown_it.tree.SyntaxTreeNode) None[source]#
render_amsmath(token: markdown_it.tree.SyntaxTreeNode) None[source]#
render_footnote_ref(token: markdown_it.tree.SyntaxTreeNode) None[source]#

Footnote references are added as auto-numbered, .i.e. [^a] is read as rST [#a]_

render_footnote_reference(token: markdown_it.tree.SyntaxTreeNode) None[source]#
render_myst_block_break(token: markdown_it.tree.SyntaxTreeNode) None[source]#
render_myst_target(token: markdown_it.tree.SyntaxTreeNode) None[source]#
render_myst_line_comment(token: markdown_it.tree.SyntaxTreeNode) None[source]#
render_myst_role(token: markdown_it.tree.SyntaxTreeNode) None[source]#
render_colon_fence(token: markdown_it.tree.SyntaxTreeNode) None[source]#

Render a div block, with : colon delimiters.

render_dl(token: markdown_it.tree.SyntaxTreeNode) None[source]#

Render a definition list.

render_field_list(token: markdown_it.tree.SyntaxTreeNode) None[source]#

Render a field list.

render_restructuredtext(token: markdown_it.tree.SyntaxTreeNode) None[source]#

Render the content of the token as restructuredtext.

render_directive(token: markdown_it.tree.SyntaxTreeNode, name: str, arguments: str, *, additional_options: dict[str, str] | None = None) None[source]#

Render special fenced code blocks as directives.

Parameters:
  • token – the token to render

  • name – the name of the directive

  • arguments – The remaining text on the same line as the directive name.

run_directive(name: str, first_line: str, content: str, position: int, additional_options: dict[str, str] | None = None) list[docutils.nodes.Element][source]#

Run a directive and return the generated nodes.

Parameters:
  • name – the name of the directive

  • first_line – The text on the same line as the directive name. May be an argument or body text, dependent on the directive

  • content – All text after the first line. Can include options.

  • position – The line number of the first line

  • additional_options – Additional options to add to the directive, above those parsed from the content.

render_substitution_inline(token: markdown_it.tree.SyntaxTreeNode) None[source]#

Render inline substitution {{key}}.

render_substitution_block(token: markdown_it.tree.SyntaxTreeNode) None[source]#

Render block substitution {{key}}.

render_substitution(token: markdown_it.tree.SyntaxTreeNode, inline: bool) None[source]#

Substitutions are rendered by:

  1. Combining global substitutions with front-matter substitutions to create a variable context (front-matter takes priority)

  2. Add the sphinx env to the variable context (if available)

  3. Create the string content with Jinja2 (passing it the variable context)

  4. If the substitution is inline and not a directive, parse to nodes ignoring block syntaxes (like lists or block-quotes), otherwise parse to nodes with all syntax rules.

myst_parser.mdit_to_docutils.base.html_meta_to_nodes(data: dict[str, Any], document: docutils.nodes.document, line: int, reporter: docutils.utils.Reporter) list[docutils.nodes.pending | docutils.nodes.system_message][source]#

Replicate the meta directive, by converting a dictionary to a list of pending meta nodes

See: https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#html-metadata

myst_parser.mdit_to_docutils.base.clean_astext(node: docutils.nodes.Element) str[source]#

Like node.astext(), but ignore images. Copied from sphinx.

myst_parser.mdit_to_docutils.base.default_slugify(title: str) str[source]#

Default slugify function.

This aims to mimic the GitHub Markdown format, see:

myst_parser.mdit_to_docutils.base.compute_unique_slug(token_tree: markdown_it.tree.SyntaxTreeNode, slugs: Iterable[str], slug_func: None | Callable[[str], str] = None) str[source]#

Compute the slug for a token.

This directly mirrors the logic in mdit_py_plugins.anchors_plugin