Commit Graph

35 Commits

Author SHA1 Message Date
dmsnell 8683a502a0 HTML API: Call `$this->is_tag_closer()` in HTML Processor.
The HTML Processor had been calling the parent class `is_tag_closer()`
method, but since visiting virtual nodes was introduced, it's important
that all of the methods are called on the subclass.

This patch fixes one issue identified where the parent method was called
instead, and it fixes another case where the change from calling the
parent method to the `$this` method was done improperly.

Developed in https://github.com/WordPress/wordpress-develop/pull/6726
Discussed in https://core.trac.wordpress.org/ticket/61348

Props jonsurrell.
See #61348.
Follow-up to [58304].

Built from https://develop.svn.wordpress.org/trunk@58365


git-svn-id: http://core.svn.wordpress.org/trunk@57814 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2024-06-08 12:12:08 +00:00
dmsnell cf2a88a397 HTML API: Fix three small type-related bugs.
This applies three fixes that appeared as part of
adding type annotations to the HTML API.

Developed in https://github.com/WordPress/wordpress-develop/pull/6754
Discussed in https://core.trac.wordpress.org/ticket/61399

Props jonsurrell.
See #61399.
Follow-up to [55203], [56274].

Built from https://develop.svn.wordpress.org/trunk@58364


git-svn-id: http://core.svn.wordpress.org/trunk@57813 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2024-06-08 11:12:11 +00:00
dmsnell 4b8fa1bc8b HTML API: Return subclass from `::create_fragment`
When the `WP_HTML_Processor` was introduced with its `::create_fragment()`
static creator method, that method has been returning a `new self(...)`.
Unfortunately, this means that subclasses cannot use that method since it
will return the `WP_HTML_Processor` instead of the subclass.

With this patch, the static creator method returns `new static(...)` to preserve
the intended behavior. A new test asserts this behavior for future changes.

Developed in https://github.com/WordPress/wordpress-develop/pull/6729
Discussed in https://core.trac.wordpress.org/ticket/61374

Props dmsnell, jonsurrell.
Follow-up to [56274].
Fixes #61374.

Built from https://develop.svn.wordpress.org/trunk@58363


git-svn-id: http://core.svn.wordpress.org/trunk@57812 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2024-06-08 10:57:14 +00:00
dmsnell 2107b186fe HTML API: Report real and virtual nodes in the HTML Processor.
HTML is a kind of short-hand for a DOM structure. This means that there are
many cases in HTML where an element's opening tag or closing tag is missing (or
both). This is because many of the parsing rules imply creating elements in the
DOM which may not exist in the text of the HTML.

The HTML Processor, being the higher-level counterpart to the Tag Processor, is
already aware of these nodes, but since it's inception has not paused on them
when scanning through a document. Instead, these are visible when pausing on a
child of such an element, but otherwise not seen.

In this patch the HTML Processor starts exposing those implicitly-created nodes,
including opening tags, and closing tags, that aren't foudn in the text content
of the HTML input document.

Previously, the sequence of matched tokens when scanning with 
`WP_HTML_Processor::next_token()` would depend on how the HTML document was written,
but with this patch, all semantically equal HTML documents will parse and scan in
the same exact manner, presenting an idealized or "perfect" view of the document
the same way as would occur when traversing a DOM in a browser.

Developed in https://github.com/WordPress/wordpress-develop/pull/6348
Discussed in https://core.trac.wordpress.org/ticket/61348

Props audrasjb, dmsnell, gziolo, jonsurrell.
Fixes #61348.

Built from https://develop.svn.wordpress.org/trunk@58304


git-svn-id: http://core.svn.wordpress.org/trunk@57761 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2024-06-03 19:47:15 +00:00
dmsnell ec7c61e320 HTML API: Add `expects_closer()` method to HTML Processor
This patch adds a new method, `WP_HTML_Processor->expects_closer()` to indicate
if the currently-matched node expects to find a closing token. For example, a
`DIV` element expects a closing `</div>` tag, but an `<img>` expects none, because
it's a void element. Similarly, `#text` nodes and HTML comments only appear as
unitary nodes on the stack of open elements. Once proceeding further in the
document they are immediately removed without any closing tag.

This new method serves as a helper to indicate whether or not to expect the
closer, as this can be more complicated than it seems, and calling code
shouldn't have to build custom interpretations and implementations. Instead,
the HTML Processor ought to export its internal knowledge to make it easy for
consuming code and projects.

Developed in https://github.com/WordPress/wordpress-develop/pull/6600
Discussed in https://core.trac.wordpress.org/ticket/61257

Fixes #61257.
Props dmsnell, jonsurrell.


Built from https://develop.svn.wordpress.org/trunk@58192


git-svn-id: http://core.svn.wordpress.org/trunk@57655 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2024-05-24 01:21:10 +00:00
dmsnell 07a35d5d41 HTML API: Add method to report depth of currently-matched node.
The HTML Processor maintains a stack of open elements, where every element,
every `#text` node, every HTML comment, and other node is pushed and popped while
traversing the document. The "depth" of each of these nodes represents how deep
that stack is where the node appears. Unfortunately this information isn't
exposed to calling code, which has led different projects to attempt to
calculate this value externally. This isn't always trivial, but the HTML
Processor could make it so by exposing the internal knowledge in a new method.

In this patch the `get_current_depth()` method returns just that. Since the
processor always exists within a context, the depth includes nesting from the
always-present html element and also the body, since currently the HTML
Processor only supports parsing in the IN BODY context.

This means that the depth reported for the `DIV` in `<div>` is 3, not 1, because
its breadcrumbs path is `HTML > BODY > DIV`.

Developed in https://github.com/WordPress/wordpress-develop/pull/6589
Discussed in https://core.trac.wordpress.org/ticket/61255

Fixes #61255.
Props dmsnell, jonsurrell.

Built from https://develop.svn.wordpress.org/trunk@58191


git-svn-id: http://core.svn.wordpress.org/trunk@57654 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2024-05-23 23:37:12 +00:00
dmsnell c3b76697cd HTML API: Respect `class_name` query arg in `HTML_Processor::next_tag()`
Previously the HTML Process was ignoring the `class_name` argument in
the `next_tag()` query if it existed. This was wrong and would lead to
calling code finding the wrong tag if provided.

This patch adds the class detection code back into `next_tag()` to fix
the bug.

Developed in https://github.com/WordPress/wordpress-develop/pull/6618
Discussed in https://core.trac.wordpress.org/ticket/58517

See #58517.
Follow-up to [56274].
Props: dmsnell, jonsurrell.

Built from https://develop.svn.wordpress.org/trunk@58190


git-svn-id: http://core.svn.wordpress.org/trunk@57653 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2024-05-23 22:45:13 +00:00
dmsnell 6badd91e7b HTML API: Trigger active format reconstruction when reaching text nodes.
When encountering text nodes in an HTML document, the HTML parser needs
to run the active format reconstruction algorithm, even if it doesn't
stop to visit those text nodes. This is because the formats, which might
need reconstructing, will impact the breadcrumbs of all downstream nodes
from the text node.

In this patch, this process is triggered, which properly triggers the
active format reconstruction. It also enables the visiting of other token
types as is possible in the Tag Processor.

Developed in https://github.com/WordPress/wordpress-develop/pull/6054
Discussed in https://core.trac.wordpress.org/ticket/60170

Props: dmsnell, jonsurrell, westonruter.
Fixes: #60455.
Follow-up to: [57348].


Built from https://develop.svn.wordpress.org/trunk@57806


git-svn-id: http://core.svn.wordpress.org/trunk@57307 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2024-03-12 00:24:09 +00:00
dmsnell 9e0a116dbf HTML API: Ensure that breadcrumbs are properly retained after seeking.
In some cases, it's possible to seek back into a location found inside
an element which has been closed before the point in the document where
the `seek()` was made. In these cases the breadcrumb stack is lost, and
calling `get_breadcrumbs()` after the seek will return the wrong information.

In this patch, the HTML Processor takes a conservative approach and
moves to the front of the document, then reparses the document until
it reaches the sought-after location. This ensures consistency on
the stack of open elements and active formats, and preserves
breadcrumbs.

Developed in https://github.com/WordPress/wordpress-develop/pull/6185
Discussed in https://core.trac.wordpress.org/ticket/60687

Props jonsurrell.
Follow-up to [60687].
See #58517.
Fixes #60687.


Built from https://develop.svn.wordpress.org/trunk@57768


git-svn-id: http://core.svn.wordpress.org/trunk@57269 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2024-03-05 05:34:11 +00:00
dmsnell fd1cbd8980 HTML API: Add subclassed has_bookmark() and fix seek()
The WP_HTML_Processor::has_bookmark() method has not correctly reported bookmarks
which have been set, because it wraps the given bookmark names when setting them.
Additionally, WP_HTML_Processor::seek() does not seek to correct location if HTML
has been updated because it wasn't flushing enqueued updates to the document.

In this patch both problems are resolved and added tests guard these behaviors
against future regressions.

Developed in https://github.com/WordPress/wordpress-develop/pull/6039
Discussed in https://core.trac.wordpress.org/ticket/60474

Follow-up to [56274].
Props dmsnell, jonsurrell.
Fixes #60474.


Built from https://develop.svn.wordpress.org/trunk@57582


git-svn-id: http://core.svn.wordpress.org/trunk@57083 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2024-02-10 22:54:13 +00:00
dmsnell 9fb7267542 HTML API: Fix typo setting the wrong self-closing flag.
The HTML Processor tracks whether a token was found with the self-closing flag.
Depending on the context, this flag may or may not indicate that the element is
self closing. Unfortunately it's been tracking the wrong flag: it's been tracking
the end-tag flag, which indicates that a token is an end tag.

In this patch the right flag is set in the HTML Processor. This hasn't been an
issue because the HTML Processor doesn't yet read that stored flag, but it's an
important fix to make before adding support for foreign content (SVG and MathML)
since that behavior depends on reading the correct flag.

Follow-up to [56274].

Props dmsnell.


Built from https://develop.svn.wordpress.org/trunk@57528


git-svn-id: http://core.svn.wordpress.org/trunk@57029 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2024-02-02 23:27:14 +00:00
dmsnell 0dc136bcb9 HTML API: Fix void tag nesting with next_token
When `next_token()` was introduced, it introduced a regression in the HTML
Processor whereby void tags remain on the stack of open elements when they
shouldn't. This led to invalid values returned from `get_breadcrumbs()`.

The reason was that calling `next_token()` works through a different code path
than the HTML Processor runs everything else. To solve this, its sub-classed
`next_token()` called `step( self::REPROCESS_CURRENT_TOKEN )` so that the proper
HTML accounting takes place.

Unfortunately that same reprocessing code path skipped the step whereby void
and self-closing elements are popped from the stack of open elements.

In this patch, that step is run with a third mode for `step()`, which is the
new `self::PROCESS_CURRENT_TOKEN`. This mode acts as if `self::PROCESS_NEXT_NODE`
were called, except it doesn't advance the parser.

Developed in https://github.com/WordPress/wordpress-develop/pull/5975
Discussed in https://core.trac.wordpress.org/ticket/60382

Follow-up to [57348]

Props dmsnell, jonsurrell
Fixes #60382


Built from https://develop.svn.wordpress.org/trunk@57507


git-svn-id: http://core.svn.wordpress.org/trunk@57008 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2024-02-01 00:43:15 +00:00
dmsnell 62e0ef411b HTML API: Scan all syntax tokens in a document, read modifiable text.
Since its introduction in WordPress 6.2 the HTML Tag Processor has
provided a way to scan through all of the HTML tags in a document and
then read and modify their attributes. In order to reliably do this, it
also needed to be aware of other kinds of HTML syntax, but it didn't
expose those syntax tokens to consumers of the API.

In this patch the Tag Processor introduces a new scanning method and a
few helper methods to read information about or from each token. Most
significantly, this introduces the ability to read `#text` nodes in the
document.

What's new in the Tag Processor?
================================

 - `next_token()` visits every distinct syntax token in a document.
 - `get_token_type()` indicates what kind of token it is.
 - `get_token_name()` returns something akin to `DOMNode.nodeName`.
 - `get_modifiable_text()` returns the text associated with a token.
 - `get_comment_type()` indicates why a token represents an HTML comment.

Example usage.
==============

{{{
<?php
function strip_all_tags( $html ) {
        $text_content = '';
        $processor    = new WP_HTML_Tag_Processor( $html );

        while ( $processor->next_token() ) {
                if ( '#text' !== $processor->get_token_type() ) {
                        continue;
                }

                $text_content .= $processor->get_modifiable_text();
        }

        return $text_content;
}
}}}

What changes in the Tag Processor?
==================================

Previously, the Tag Processor would scan the opening and closing tag of
every HTML element separately. Now, however, there are special tags
which it only visits once, as if those elements were void tags without
a closer.

These are special tags because their content contains no other HTML or
markup, only non-HTML content.

 - SCRIPT elements contain raw text which is isolated from the rest of
   the HTML document and fed separately into a JavaScript engine. There
   are complicated rules to avoid escaping the script context in the HTML.
   The contents are left verbatim, and character references are not decoded.

 - TEXTARA and TITLE elements contain plain text which is decoded
   before display, e.g. transforming `&amp;` into `&`. Any markup which
   resembles tags is treated as verbatim text and not a tag.

 - IFRAME, NOEMBED, NOFRAMES, STYLE, and XMP elements are similar to the
   textarea and title elements, but no character references are decoded.
   For example, `&amp;` inside a STYLE element is passed to the CSS engine
   as the literal string `&amp;` and _not_ as `&`.

Because it's important not treat this inner content separately from the
elements containing it, the Tag Processor combines them when scanning
into a single match and makes their content available as modifiable
text (see below).

This means that the Tag Processor will no longer visit a closing tag for
any of these elements unless that tag is unexpected.

{{{
    <title>There is only a single token in this line</title>
    <title>There are two tokens in this line></title></title>
    </title><title>There are still two tokens in this line></title>
}}}

What are tokens?
================

The term "token" here is a parsing term, which means a primitive unit in
HTML. There are only a few kinds of tokens in HTML:

 - a tag has a name, attributes, and a closing or self-closing flag.
 - a text node, or `#text` node contains plain text which is displayed
   in a browser and which is decoded before display.
 - a DOCTYPE declaration indicates how to parse the document.
 - a comment is hidden from the display on a page but present in the HTML.

There are a few more kinds of tokens that the HTML Tag Processor will
recognize, some of which don't exist as concepts in HTML. These mostly
comprise XML syntax elements that aren't part of HTML (such as CDATA and
processing instructions) and invalid HTML syntax that transforms into
comments.

What is a funky comment?
========================

This patch treats a specific kind of invalid comment in a special way.
A closing tag with an invalid name is considered a "funky comment." In
the browser these become HTML comments just like any other, but their
syntax is convenient for representing a variety of bits of information
in a well-defined way and which cannot be nested or recursive, given
the parsing rules handling this invalid syntax.

 - `</1>`
 - `</%avatar_url>`
 - `</{"wp_bit": {"type": "post-author"}}>`
 - `</[post-author]>`
 - `</__( 'Save Post' );>`

All of these examples become HTML comments in the browser. The content
inside the funky content is easily parsable, whereby the only rule is
that it starts at the `<` and continues until the nearest `>`. There
can be no funky comment inside another, because that would imply having
a `>` inside of one, which would actually terminate the first one.

What is modifiable text?
========================

Modifiable text is similar to the `innerText` property of a DOM node.
It represents the span of text for a given token which may be modified
without changing the structure of the HTML document or the token.

There is currently no mechanism to change the modifiable text, but this
is planned to arrive in a later patch.

Tags
====

Most tags have no modifiable text because they have child nodes where
text nodes are found. Only the special tags mentioned above have
modifiable text.

{{{
    <div class="post">Another day in HTML</div>
    └─ tag ──────────┘└─ text node ─────┘└────┴─ tag
}}}

{{{
    <title>Is <img> &gt; <image>?</title>
    │      └ modifiable text ───┘       │ "Is <img> > <image>?"
    └─ tag ─────────────────────────────┘
}}}

Text nodes
==========

Text nodes are entirely modifiable text.

{{{
    This HTML document has no tags.
    └─ modifiable text ───────────┘
}}}

Comments
========

The modifiable text inside a comment is the portion of the comment that
doesn't form its syntax. This applies for a number of invalid comments.

{{{
    <!-- this is inside a comment -->
    │   └─ modifiable text ──────┘  │
    └─ comment token ───────────────┘
}}}

{{{
    <!-->
    This invalid comment has no modifiable text.
}}}

{{{
    <? this is an invalid comment -->
    │ └─ modifiable text ────────┘  │
    └─ comment token ───────────────┘
}}}

{{{
    <[CDATA[this is an invalid comment]]>
    │       └─ modifiable text ───────┘ │
    └─ comment token ───────────────────┘
}}}

Other token types also have modifiable text. Consult the code or tests
for further information.

Developed in https://github.com/WordPress/wordpress-develop/pull/5683
Discussed in https://core.trac.wordpress.org/ticket/60170

Follows [57575]

Props bernhard-reiter, dlh, dmsnell, jonsurrell, zieladam
Fixes #60170


Built from https://develop.svn.wordpress.org/trunk@57348


git-svn-id: http://core.svn.wordpress.org/trunk@56854 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2024-01-24 23:37:16 +00:00
dmsnell 7f22c4d4ea HTML API: Support INPUT tags.
Adds support for the following HTML elements to the HTML Processor:

 - INPUT

Previously this element was not supported and the HTML Processor would bail when encountering one. Now, with this patch applied, it will proceed to parse the HTML document.

Developed in https://github.com/WordPress/wordpress-develop/pull/5907
Discussed in https://core.trac.wordpress.org/ticket/60283

Props jonsurrell
See #60283


Built from https://develop.svn.wordpress.org/trunk@57343


git-svn-id: http://core.svn.wordpress.org/trunk@56849 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2024-01-24 03:30:14 +00:00
dmsnell 3efed38163 HTML API: Support PARAM, SOURCE, and TRACK tags.
Adds support for the following HTML elements to the HTML Processor:

 - PARAM, SOURCE, TRACK

Previously these elements were not supported and the HTML Processor would bail when encountering them. Now, with this patch applied, it will proceed to parse an HTML document when encountering those tags.

Props jonsurrell, dmsnell
Fixes #60283


Built from https://develop.svn.wordpress.org/trunk@57326


git-svn-id: http://core.svn.wordpress.org/trunk@56832 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2024-01-23 01:51:16 +00:00
dmsnell 1d4b95336a HTML API: Cleanup tests and list of void elements.
This patch adds newly supported elements to tests that should have been updated
in recent PRs, but which were merged without that. Those PRs removed failing
tests showing that the elements were unsupported, but did not add the elements
to the list of supported ones.

It also removes some elements from the special-exclusion list of unsupported IN
BODY elements. These did not present in failing tests because earlier
conditions in the switch structure caught the tags before hitting the default
block.

Finally it adds some missing elements to the list of void elements. These
elements are not listed as void in the HTML specification because they are
deprecated. However, they are treated as void for the sake of HTML
serialization and the parsing rules indicate that they behave as void elements,
so it's safe to list them within the HTML API as void.

Developed in WordPress/wordpress-develop#5913

Fixes #60307


Built from https://develop.svn.wordpress.org/trunk@57319


git-svn-id: http://core.svn.wordpress.org/trunk@56825 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2024-01-20 00:15:13 +00:00
dmsnell cf88a3098c HTML API: Add support for PRE and LISTING elements.
Adds support for the following HTML elements to the HTML Processor:

 - PRE, LISTING

Previously, these elements were not supported and the HTML Processor would bail when encountering them. Now, with this patch applied, it will proceed to parse an HTML document when encountering those tags.

Developed in WordPress/wordpress-develop#5903

Props jonsurrell, dmsnell
Fixes #60283


Built from https://develop.svn.wordpress.org/trunk@57317


git-svn-id: http://core.svn.wordpress.org/trunk@56823 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2024-01-19 22:07:09 +00:00
dmsnell 050c342a46 HTML API: Add support for BR, EMBED, & other tags.
Adds support for the following HTML elements to the HTML Processor:

 - AREA, BR, EMBED, KEYGEN, WBR
 - Only the opening BR tag is supported, as the invalid closer `</br>`
   involves more complicated rules, to be implemented later.

Previously, these elements were not supported and the HTML Processor
would bail when encountering them. With this patch it will proceed to
parse an HTML document when encountering those tags as long as other
normal conditions don't cause it to bail (such as complicated format
reconstruction rules).

Props jonsurrell, dmsnell
Fixes #60283


Built from https://develop.svn.wordpress.org/trunk@57316


git-svn-id: http://core.svn.wordpress.org/trunk@56822 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2024-01-19 21:41:12 +00:00
dmsnell 141ba4ff59 HTML API: Add support for HR element.
Adds support for the following HTML elements to the HTML Processor:

 - HR

Previously, this element was not supported and the HTML Processor would bail when encountering
it. Now, with this patch, it will proceed to parse an HTML document when encountering one.

Developed in WordPress/wordpress-develop#5897

Props jonsurrell, dmsnell
Fixes #60283


Built from https://develop.svn.wordpress.org/trunk@57314


git-svn-id: http://core.svn.wordpress.org/trunk@56820 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2024-01-19 19:03:09 +00:00
dmsnell 29dd837333 HTML API: Add support for list elements.
Adds support for the following HTML elements to the HTML Processor:

 - LI, OL, UL.
 - DD, DL, DT.

Previously, these elements were not supported and the HTML Processor would bail when encountering them.
With this patch it will proceed to parse an HTML document when encountering those tags as long as other normal conditions don't cause it to bail (such as complicated format reconstruction).

Props audrasjb, jonsurrell, bernhard-reiter.
Fixes #60215.


Built from https://develop.svn.wordpress.org/trunk@57264


git-svn-id: http://core.svn.wordpress.org/trunk@56770 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2024-01-10 14:05:17 +00:00
Bernhard Reiter 372e3c4b97 HTML API: Add explicit handling or failure for all tags.
The HTML API HTML processor does not yet support all tags. Many tags (e.g. list elements) have some complicated rules in the [https://html.spec.whatwg.org/#parsing-main-inbody "in body" insertion mode].

Implementing these special rules is blocking the implementation for a catch-all rule for "any other tag" because we need to prevent special rules from being handled by the catch-all.

  Any other start tag
  Reconstruct the active formatting elements, if any.

  Insert an HTML element for the token.

  …

This change ensures the HTML Processor fails when handling special tags. This is the same as existing behavior, but will allow us to implement the catch-all "any other tag" handling without unintentionally handling special elements.

Additionally, we add tests that assert the special elements are unhandled. As these tags are implemented, this should help to ensure they're removed from the unsupported tag list.

Props jonsurrell, dmsnell.
Fixes #60092.
Built from https://develop.svn.wordpress.org/trunk@57248


git-svn-id: http://core.svn.wordpress.org/trunk@56754 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2024-01-08 14:05:24 +00:00
Bernhard Reiter a586b85d8e HTML API: Apply linting changes to `@TODO` comments.
Lowercase `@TODO` comments introduced by [57186], and remove spurious colons after them.

Props dmsnell, TobiasBg, mukesh27, sergeybiryukov, jonsurrell.
Fixes #60060.
Built from https://develop.svn.wordpress.org/trunk@57209


git-svn-id: http://core.svn.wordpress.org/trunk@56715 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2023-12-20 12:36:31 +00:00
Bernhard Reiter 511dcb7520 HTML API: Add support for H1-H6 elements in the HTML Processor.
Previously these have been unsupported, but in this patch, support is added for the tags so that the HTML Processor can process documents containing them.

There was a design discussion about introducing a constant to communicate "any of the H1 - H6 elements" but this posed a number of challenges that don't need to be answered in this patch. For the time being, because the HTML specification treats H1 - H6 specially as a single kind of element, the HTML Processor uses an internal hard-coded string to indicate this. By using a hard-coded string it's possible to avoid introducing a class constant which cannot be made private due to PHP's class design. In the future, this will probably appear as a special constant in a new constant-containing class.

Props dmsnell, jonsurrell.
Fixes #60060.
Built from https://develop.svn.wordpress.org/trunk@57186


git-svn-id: http://core.svn.wordpress.org/trunk@56697 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2023-12-13 17:53:19 +00:00
Bernhard Reiter b86334b447 HTML API: Add support for containers elements, including ARTICLE.
There are a handful of elements which behave similarly and are generically container elements. These are the following elements:

    ADDRESS, ARTICLE, ASIDE, BLOCKQUOTE, CENTER, DETAILS, DIALOG, DIR,
    DL, DIV, FIELDSET, FIGCAPTION, FIGURE, FOOTER, HEADER, HGROUP, MAIN,
    MENU, NAV, SEARCH, SECTION, SUMMARY

This patch adds support to the HTML Processor for handling these elements. They do not require any additional logic in the rest of the class, and carry no specific semantic rules for parsing beyond what is listed in their group in the IN BODY section of the HTML5 specification.

Props dmsnell.
Fixes #59914.
Built from https://develop.svn.wordpress.org/trunk@57115


git-svn-id: http://core.svn.wordpress.org/trunk@56626 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2023-11-17 06:13:25 +00:00
Sergey Biryukov 7da5644617 Docs: Use proper case for `@todo` tags.
The correct tag is `@todo`, not `@TODO` or `@todo:` (note the trailing colon).

Reference: [https://developer.wordpress.org/coding-standards/inline-documentation-standards/php/#phpdoc-tags PHP Documentation Standards: PHPDoc tags].

Follow-up to [55203], [56274], [56565], [56698].

Props jrf.
See #59651.
Built from https://develop.svn.wordpress.org/trunk@57077


git-svn-id: http://core.svn.wordpress.org/trunk@56588 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2023-11-07 00:14:23 +00:00
Sergey Biryukov 4fd39c6620 HTML API: Rename `WP_HTML_Processor::createFragment()` to follow WPCS.
`WP_HTML_Processor::create_fragment()` is the correct method name as per the WordPress PHP coding standards.

Follow-up to [56274].

Props dmsnell, jrf, hellofromTonya, SergeyBiryukov.
Fixes #59547.
Built from https://develop.svn.wordpress.org/trunk@56790


git-svn-id: http://core.svn.wordpress.org/trunk@56302 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2023-10-05 22:42:26 +00:00
Bernhard Reiter 374bcd9b3d HTML API: Add `matches_breadcrumbs()` method for better querying.
Inside a `next_tag()` loop it can be challenging to use breadcrumbs because they are only exposed inside the call to `next_tag()` via the `$query` arg.

In this patch a new method, `matches_breadcrumbs()`, is exposed which allows for querying within the `next_tag()` loop for more complicated queries.

This method exposes a wildcard `*` operator to allow matching ''any HTML tag'' that the currently-matched tag is a child or descendant of.

Props dmsnell, westonruter, mukesh27.
Fixes #59400.
Built from https://develop.svn.wordpress.org/trunk@56702


git-svn-id: http://core.svn.wordpress.org/trunk@56214 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2023-09-26 08:20:17 +00:00
Bernhard Reiter 15c3f036fd HTML API: Update documentation and rename internal variable on HTML Processor
This patch updates documentation and an internal variable name within the
HTML Processor class so that they are more helpful and complete to a reader.

There should be no functional or visual changes in this patch.

Props dmsnell, mukesh27.
Fixes #59267.
Built from https://develop.svn.wordpress.org/trunk@56565


git-svn-id: http://core.svn.wordpress.org/trunk@56077 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2023-09-13 13:02:17 +00:00
Bernhard Reiter 251a9c7653 HTML API: Store current token reference in HTML Processor state.
The `$current_token` reference has been stored in the HTML Processor itself, but I suggested to move it into the externalized state so that it can be stored and replaced.

In this patch the reference is moved to that state variable and it should become more possible to save and load state, to resume execution after pausing.

Props dmsnell.
Fixes #59268.
Built from https://develop.svn.wordpress.org/trunk@56558


git-svn-id: http://core.svn.wordpress.org/trunk@56070 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2023-09-12 15:12:17 +00:00
Bernhard Reiter c67fe4b6e5 HTML API: Stop processing HTML when encountering unsupported markup.
It was a design goal of the HTML Processor to abort processing its input document when encountering unsupported markup. Unfortunately there was no test for this and so-far, the HTML Processor has paused, but continued processing in these situations.

In this patch a new test ensures that the HTML Processor stops and refuses to move forward after encountering any unsupported markup. It also ensures that it doesn't report any current tag names since unsupported markup could imply that the read tag name is different than the parsed tag name.

Props dmsnell.
Fixes #59167.
Built from https://develop.svn.wordpress.org/trunk@56493


git-svn-id: http://core.svn.wordpress.org/trunk@56005 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2023-08-30 15:39:16 +00:00
Bernhard Reiter 12884f0361 HTML API: Add support for BUTTON element.
This patch adds support to process the BUTTON element. This requires adding some additional semantic rules to handle situations where a BUTTON element is already in scope.

Also included is a fixup to enforce that `WP_HTML_Processor::next_tag()` never returns for a tag closer. This is useful with the Tag Processor, but not for the HTML Processor. There were tests relying on this behavior to assert that internal processes were working as they should, but those tests have been updated to use the semi-private `step()` function, which does stop on tag closers.

This patch is one in a series of changes to expand support within the HTML API, moving gradually to allow for more focused changes that are easier to review and test. The HTML Processor is a work in progress with a certain set of features slated to be ready and tested by 6.4.0, but it will only contain partial support of the HTML5 specification even after that. Whenever it cannot positively recognize and process its input it bails, and certain function stubs and logical stubs exist to structure future expansions of support.

Props dmsnell.
Fixes #58961.
Built from https://develop.svn.wordpress.org/trunk@56380


git-svn-id: http://core.svn.wordpress.org/trunk@55892 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2023-08-10 08:37:20 +00:00
Aaron Jorbin 5341f9b212 HTML API: Fix missing * for docblock.
Follow up to [56363].

Props dmsnell.
See #58918. Fixes #59010.

Built from https://develop.svn.wordpress.org/trunk@56376


git-svn-id: http://core.svn.wordpress.org/trunk@55888 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2023-08-08 23:01:26 +00:00
Bernhard Reiter fa83c8e1cd HTML API: Adjust code styling to Gutenberg's linter's preferences.
Adjust the code style according to the rules that the linting process in Gutenberg requires.

There are only a couple code changes that should have no effect on the runtime:
 - A missing check to verify that only `UTF-8` is supported has been added (brought up because it was identified as an undefined variable).
 - A few `return false;` statements have been added to avoid having the linter complain that functions don't return a value despite indicating they return `bool`. The functions are stubs for coming support and currently `throw`, so the `return` statements are unreachable.

Props dmsnell, costdev, davidbaumwald, peterwilsoncc, SergeyBiryukov.
Fixes #58918.
Built from https://develop.svn.wordpress.org/trunk@56363


git-svn-id: http://core.svn.wordpress.org/trunk@55875 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2023-08-07 13:50:27 +00:00
Bernhard Reiter 8fa9aad5e6 HTML API: Add support for SPAN element.
In this patch we're introducing support for the SPAN element, which is the first
in the class of "any other tag" in the "in body" insertion mode.

This patch introduces the mechanisms required to handle that class of tags but
only introduces SPAN to keep the change focused. With the tests and mechanisms
in place it will be possible to follow-up and add another limited set of tags.

It's important that this not use the default catch-all in the switch handling
`step_in_body` because that would catch tags that have specific rules in previous
case statements that aren't yet added. For example, we don't want to treat the
`TABLE` element as "any other tag".

Props dmsnell.
Fixes #58907.
Built from https://develop.svn.wordpress.org/trunk@56331


git-svn-id: http://core.svn.wordpress.org/trunk@55843 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2023-08-01 07:56:23 +00:00
Bernhard Reiter 44629e6286 HTML-API: Introduce minimal HTML Processor.
This patch introduces the //first// of //many// iterations on the evolution of the HTML API, the HTML Processor, which is built in order to understand HTML structure including nesting, misnesting, and complicated semantic rules.

In the first iteration, the HTML Processor is arbitrarily limited to a minimal subset of functionality so that we can review it, ship it, test it, and collect feedback before moving forward. This means that this patch is more or less an extension to the Tag Processor query language, providing the ability not only to scan for a tag of a given name, but also to find an HTML element in a specific nesting path.

The HTML Processor also aborts any time it encounters:
 - a tag that isn't a `P`, `DIV`, `FIGURE`, `FIGCAPTION`, `IMG`, `STRONG`, `B`, `EM`, `I`, `A`, `BIG`, `CODE`, `FONT`, `SMALL`, `STRIKE`, `TT`, or `U` tag. this limit exists because many HTML elements require specific rules and we are trying to limit the number of rules introduced at once. this work is targeted at existing work in places like the image block.
 - certain misnesting constructs that evoke complicated resolution inside the HTML spec. where possible and where simple to do reliably, certain parse errors are handled. in most cases the HTML Processor aborts.

The structure of the HTML Processor is established in this patch. Further spec-compliance comes through filling out //more of the same// kind and nature of code as is found in this patch. Certain critical HTML algorithms are partially supported, and where support requires more than is present, the HTML Processor acknowledges this and refuses to operate.

In this patch are explorations for how to verify that new HTML support is fully added (instead of allowing for partial updates that leave some code paths non-compliant). Performance is hard to measure since support is so limited at the current time, but it should generally follow the performance of the Tag Processor somewhat close as the overhead is minimized as much as practical.

Props dmsnell, zieladam, costdev.
Fixes #58517.
Built from https://develop.svn.wordpress.org/trunk@56274


git-svn-id: http://core.svn.wordpress.org/trunk@55786 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2023-07-20 13:43:25 +00:00