This patch introduces the //first// of //many// iterations on the evolution of the HTML API, the HTML Processor, which is built in order to understand HTML structure including nesting, misnesting, and complicated semantic rules.
In the first iteration, the HTML Processor is arbitrarily limited to a minimal subset of functionality so that we can review it, ship it, test it, and collect feedback before moving forward. This means that this patch is more or less an extension to the Tag Processor query language, providing the ability not only to scan for a tag of a given name, but also to find an HTML element in a specific nesting path.
The HTML Processor also aborts any time it encounters:
- a tag that isn't a `P`, `DIV`, `FIGURE`, `FIGCAPTION`, `IMG`, `STRONG`, `B`, `EM`, `I`, `A`, `BIG`, `CODE`, `FONT`, `SMALL`, `STRIKE`, `TT`, or `U` tag. this limit exists because many HTML elements require specific rules and we are trying to limit the number of rules introduced at once. this work is targeted at existing work in places like the image block.
- certain misnesting constructs that evoke complicated resolution inside the HTML spec. where possible and where simple to do reliably, certain parse errors are handled. in most cases the HTML Processor aborts.
The structure of the HTML Processor is established in this patch. Further spec-compliance comes through filling out //more of the same// kind and nature of code as is found in this patch. Certain critical HTML algorithms are partially supported, and where support requires more than is present, the HTML Processor acknowledges this and refuses to operate.
In this patch are explorations for how to verify that new HTML support is fully added (instead of allowing for partial updates that leave some code paths non-compliant). Performance is hard to measure since support is so limited at the current time, but it should generally follow the performance of the Tag Processor somewhat close as the overhead is minimized as much as practical.
Props dmsnell, zieladam, costdev.
Fixes#58517.
Built from https://develop.svn.wordpress.org/trunk@56274
git-svn-id: http://core.svn.wordpress.org/trunk@55786 1a063a9b-81f0-0310-95a4-ce76da25c4cd
This moves a reference link in `::get_attribute_names_with_prefix()` below the code example, so that it is correctly displayed in the Developer Resources.
Includes updating some other `@see` tags for consistency as per the documentation standards.
Additionally, the example code for `WP_HTML_Tag_Processor::get_tag()` is updated to show lowercase tag names in the input HTML, so that it does not convey the wrong impression that the uppercase output from `::get_tag()` depends on the case of the input HTML.
Follow-up to [55203].
Props dmsnell, johnbillion, audrasjb, SergeyBiryukov.
Fixes#58254.
Built from https://develop.svn.wordpress.org/trunk@55724
git-svn-id: http://core.svn.wordpress.org/trunk@55236 1a063a9b-81f0-0310-95a4-ce76da25c4cd
In [55718] the Unicode replacement character was mistakenly removed. The purpose of including the character was to communicate what it looks like and why the Tag Processor won't insert it into the document.
This changeset brings the character back and adds a small clue to fix the confusion that may lead to its removal.
Follow-up to [55718].
Props dmsnell.
Fixes#58256
See #57840.
Built from https://develop.svn.wordpress.org/trunk@55723
git-svn-id: http://core.svn.wordpress.org/trunk@55235 1a063a9b-81f0-0310-95a4-ce76da25c4cd
A bug was discovered where where the parser wasn't returning to the
start of the affected tag after making some updates.
In few words, the Tag Processor has not been treating its own internal
pointer `bytes_already_parsed` the same way it treats its bookmarks.
That is, when updates are applied to the input document and then
`get_updated_html()` is called, the internal pointer transfers to
the newly-updated content as if no updates had been applied since
the previous call to `get_updated_html()`.
In this patch we're creating a new "shift accumulator" to account for
all of the updates that accrue before calling `get_updated_html()`.
This accumulated shift will be applied when swapping the input document
with the output buffer, which should result in the pointer pointing to
the same logical spot in the document it did before the udpate.
In effect this patch adds a single workaround for treating the
internal pointer like a bookmark, plus a temporary pointer which points
to the beginning of the current tag when calling `get_updated_html()`.
This will preserve the assumption that updating a document doesn't
move that pointer, or shift which tag is currently matched.
Props dmsnell, zieladam.
Fixes#58179.
Built from https://develop.svn.wordpress.org/trunk@55706
git-svn-id: http://core.svn.wordpress.org/trunk@55218 1a063a9b-81f0-0310-95a4-ce76da25c4cd
- Comments created by means of a tag closer with an invalid tag name, e.g. `</3>`.
- Comments closed with the invalid `--!>` closer. (Comments should be closed by `-->` but if the `!` appears it will also close it, in error.)
- Empty tag name elements, which are technically skipped over and aren't comments, e.g. `</>`.
Props dmsnell, costdev.
Fixes#58007.
Built from https://develop.svn.wordpress.org/trunk@55667
git-svn-id: http://core.svn.wordpress.org/trunk@55179 1a063a9b-81f0-0310-95a4-ce76da25c4cd
When setting a new value for an attribute multiple times and providing
multiple case variations of the attribute name the Tag Processor has
been appending multiple copies of the attribute into the updated HTML.
This means that only the first attribute set determines the value in
the final output, plus the output will //appear// wrong.
In this patch we're adding a test to catch the situation and resolving it
by using the appropriate comparable attribute name as a key for storing
the updates as we go. Previously we stored updates to the attribute by
its given `$name`, but when a new update of the same name with a
case variant was queued, it would not override the previously-enqueued
value as it out to have.
Props dmsnell, zieladam.
Fixes#58146.
Built from https://develop.svn.wordpress.org/trunk@55659
git-svn-id: http://core.svn.wordpress.org/trunk@55171 1a063a9b-81f0-0310-95a4-ce76da25c4cd
In this patch we're adding `has_self_closing_flag()` to the HTML Tag Processor.
This exposes whether a currently-matched tag contains the self-closing flag `/`.
This information is critical for the evolution of the HTML API in order
to track and parse HTML structure, specifically, knowing whether an
HTML foreign element is self-closing or not.
Props dmsnell, zieladam.
Fixes#58009.
Built from https://develop.svn.wordpress.org/trunk@55619
git-svn-id: http://core.svn.wordpress.org/trunk@55131 1a063a9b-81f0-0310-95a4-ce76da25c4cd
While `WP_HTML_Tag_Processor` currently only supports changing a given tag's attributes, the plan is to provide methods to make broader changes (possibly through a subclass of `WP_HTML_Tag_Processor`). The API will have the potential of replacing a tag that a bookmark points to. To prepare, this changeset makes sure that all bookmarks affected by a HTML replacement are invalidated (i.e. released).
Changes:
* Extends the existing loop in `WP_HTML_Tag_Processor::apply_attributes_updates()` that adjusts bookmarks' start and end positions upon HTML changes to check if the entire bookmark is within a portion of the HTML that has been replaced.
* Adds `WP_HTML_Tag_Processor::has_bookmark() to check whether the given bookmark name exists.
References:
* [https://github.com/WordPress/gutenberg/pull/47559 Gutenberg PR 47559]
* [https://github.com/WordPress/gutenberg/releases/tag/v15.3.0 Released in Gutenberg 15.3.0]
Follow-up to [55203].
Props bernhard-reiter, dmsnell, zieladam.
Fixes#57788.
Built from https://develop.svn.wordpress.org/trunk@55555
git-svn-id: http://core.svn.wordpress.org/trunk@55067 1a063a9b-81f0-0310-95a4-ce76da25c4cd
Documents the shorthand usage, i.e. `$this->next_tag( 'img' )`, of `WP_HTML_Tag_Processor::next_tag()`.
Also includes table alignments and formatting adjustments in the class docs.
Follow-up to [55203], [55206].
Props zieladam, poena, dmsnell, costdev, hellofromTonya.
Fixes#57863.
See #57575.
Built from https://develop.svn.wordpress.org/trunk@55477
git-svn-id: http://core.svn.wordpress.org/trunk@55010 1a063a9b-81f0-0310-95a4-ce76da25c4cd
Setting a bookmark on a tag should set its "start" position before the opening "<", e.g.:
{{{
<div> Testing a <b>Bookmark</b>
----------------^
}}}
The previous calculation assumed this is always one byte to the left from `$tag_name_starts_at`.
However, in a closing tag that index points to a solidus symbol "/":
{{{
<div> Testing a <b>Bookmark</b>
----------------------------^
}}}
The bookmark should therefore start two bytes before the tag name:
{{{
<div> Testing a <b>Bookmark</b>
---------------------------^
}}}
This changeset achieves this by:
* Using the correct starting index for closing tag bookmarks.
* Adding `array( 'tag_closers' => 'visit' )` in `WP_HTML_Tag_Processor::seek()`.
Follow-up to [55203].
Props zieladam, dmsnell, flixos90.
Fixes#57787.
See #57575.
Built from https://develop.svn.wordpress.org/trunk@55407
git-svn-id: http://core.svn.wordpress.org/trunk@54940 1a063a9b-81f0-0310-95a4-ce76da25c4cd
When the HTML API was introduced a number of fields were switched from private visibility to protected so that Gutenberg and other systems could more easily enhance the behaviors through subclassing. The $this->html property was overlooked but important for systems using the Tag Processor to stich HTML, specifically performing operations on innerHTML and innerText.
Follow-up [55203].
Props dmsnell.
See #57575.
Built from https://develop.svn.wordpress.org/trunk@55402
git-svn-id: http://core.svn.wordpress.org/trunk@54935 1a063a9b-81f0-0310-95a4-ce76da25c4cd
This commit pulls in the HTML Tag Processor from the Gutenbeg repository.
The Tag Processor attempts to be an HTML5-spec-compliant parser that provides the ability in PHP to find specific HTML tags and then add, remove, or update attributes on that tag. It provides a safe and reliable way to modify the attribute on HTML tags.
More information: https://github.com/WordPress/wordpress-develop/pull/3920.
Props: antonvlasenko, bernhard-reiter, costdev, dmsnell, felixarntz, gziolo, hellofromtonya, zieladam, flixos90, ntsekouras, peterwilsoncc, swissspidy, andrewserong, onemaggie, get_dave, aristath, scruffian, justlevine, andraganescu, noisysocks, dlh, soean, cbirdsong, revgeorge, azaozz.
Fixes#57575.
Built from https://develop.svn.wordpress.org/trunk@55203
git-svn-id: http://core.svn.wordpress.org/trunk@54736 1a063a9b-81f0-0310-95a4-ce76da25c4cd
Bernhard Reiter
Andrew Ozz
Sergey Biryukov
audrasjb
John Blackbourn
zieladam
hellofromTonya
gziolo