Commit Graph

23 Commits

Author SHA1 Message Date
Bernhard Reiter 44629e6286 HTML-API: Introduce minimal HTML Processor.
This patch introduces the //first// of //many// iterations on the evolution of the HTML API, the HTML Processor, which is built in order to understand HTML structure including nesting, misnesting, and complicated semantic rules.

In the first iteration, the HTML Processor is arbitrarily limited to a minimal subset of functionality so that we can review it, ship it, test it, and collect feedback before moving forward. This means that this patch is more or less an extension to the Tag Processor query language, providing the ability not only to scan for a tag of a given name, but also to find an HTML element in a specific nesting path.

The HTML Processor also aborts any time it encounters:
 - a tag that isn't a `P`, `DIV`, `FIGURE`, `FIGCAPTION`, `IMG`, `STRONG`, `B`, `EM`, `I`, `A`, `BIG`, `CODE`, `FONT`, `SMALL`, `STRIKE`, `TT`, or `U` tag. this limit exists because many HTML elements require specific rules and we are trying to limit the number of rules introduced at once. this work is targeted at existing work in places like the image block.
 - certain misnesting constructs that evoke complicated resolution inside the HTML spec. where possible and where simple to do reliably, certain parse errors are handled. in most cases the HTML Processor aborts.

The structure of the HTML Processor is established in this patch. Further spec-compliance comes through filling out //more of the same// kind and nature of code as is found in this patch. Certain critical HTML algorithms are partially supported, and where support requires more than is present, the HTML Processor acknowledges this and refuses to operate.

In this patch are explorations for how to verify that new HTML support is fully added (instead of allowing for partial updates that leave some code paths non-compliant). Performance is hard to measure since support is so limited at the current time, but it should generally follow the performance of the Tag Processor somewhat close as the overhead is minimized as much as practical.

Props dmsnell, zieladam, costdev.
Fixes #58517.
Built from https://develop.svn.wordpress.org/trunk@56274


git-svn-id: http://core.svn.wordpress.org/trunk@55786 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2023-07-20 13:43:25 +00:00
Andrew Ozz 817c701f29 HTML API: Fix a fatal error when processing malformed document with unclosed attribute.
Props: dlh, costdev, dmsnell.
Fixes: #58637.
Built from https://develop.svn.wordpress.org/trunk@56133


git-svn-id: http://core.svn.wordpress.org/trunk@55645 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2023-07-04 20:45:23 +00:00
Sergey Biryukov 5e0592d8f4 Docs: Improve HTML API file and class headers per the documentation standards.
Follow-up to [55203], [55304], [55718], [55724], [55727].

See #57840.
Built from https://develop.svn.wordpress.org/trunk@55734


git-svn-id: http://core.svn.wordpress.org/trunk@55246 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2023-05-09 11:19:21 +00:00
Sergey Biryukov 2ffb7ef083 Docs: Update code examples formatting in `WP_HTML_Tag_Processor` documentation.
Per the [https://developer.wordpress.org/coding-standards/inline-documentation-standards/php/#description documentation standards], code samples should be created by indenting every line of the code by 4 spaces, with a blank line before and after. This matches the format used by the rest of core.

Follow-up to [55203], [55304], [55718], [55724].

Props juanmaguitar, coffee2code, azaozz, costdev, dmsnell, johnbillion, SergeyBiryukov.
Fixes #58028.
Built from https://develop.svn.wordpress.org/trunk@55727


git-svn-id: http://core.svn.wordpress.org/trunk@55239 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2023-05-08 17:32:24 +00:00
Sergey Biryukov 8b53b81183 Docs: Remove `@return void` from various DocBlocks.
Per the documentation standards, it should not be used outside of the default bundled themes.

Follow-up to [52049], [52051], [53331], [54156], [54214], [55203], [55719].

See #57840.
Built from https://develop.svn.wordpress.org/trunk@55725


git-svn-id: http://core.svn.wordpress.org/trunk@55237 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2023-05-06 11:40:24 +00:00
Sergey Biryukov 24eb807cbd Docs: Correct the placement of `@see` tags in `WP_HTML_Tag_Processor` class.
This moves a reference link in `::get_attribute_names_with_prefix()` below the code example, so that it is correctly displayed in the Developer Resources.

Includes updating some other `@see` tags for consistency as per the documentation standards.

Additionally, the example code for `WP_HTML_Tag_Processor::get_tag()` is updated to show lowercase tag names in the input HTML, so that it does not convey the wrong impression that the uppercase output from `::get_tag()` depends on the case of the input HTML.

Follow-up to [55203].

Props dmsnell, johnbillion, audrasjb, SergeyBiryukov.
Fixes #58254.
Built from https://develop.svn.wordpress.org/trunk@55724


git-svn-id: http://core.svn.wordpress.org/trunk@55236 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2023-05-05 12:43:21 +00:00
audrasjb 3911398ec5 HTML API: Restore mistakenly-removed content in documentation.
In [55718] the Unicode replacement character was mistakenly removed. The purpose of including the character was to communicate what it looks like and why the Tag Processor won't insert it into the document.

This changeset brings the character back and adds a small clue to fix the confusion that may lead to its removal.

Follow-up to [55718].

Props dmsnell.
Fixes #58256
See #57840.

Built from https://develop.svn.wordpress.org/trunk@55723


git-svn-id: http://core.svn.wordpress.org/trunk@55235 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2023-05-05 10:58:23 +00:00
Bernhard Reiter f2d315036b HTML API: Adjust coding style to pass Gutenberg linter.
This patch adjusts some minor neutral whitespace that the Gutenberg linting rejects.
There are no code changes otherwise.

Props dmsnell.
Fixes #58250.
Built from https://develop.svn.wordpress.org/trunk@55721


git-svn-id: http://core.svn.wordpress.org/trunk@55233 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2023-05-04 08:42:22 +00:00
John Blackbourn 89cbdce5c3 Docs: Improve formatting of markup in the docs for `WP_HTML_Tag_Processor`.
Code blocks wrapped inside backtacks don't need to be indented.

See #57840

Built from https://develop.svn.wordpress.org/trunk@55718


git-svn-id: http://core.svn.wordpress.org/trunk@55230 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2023-05-03 23:55:16 +00:00
Bernhard Reiter 5a32396d8f HTML API: Accumulate shift for internal parsing pointer.
A bug was discovered where where the parser wasn't returning to the
start of the affected tag after making some updates.

In few words, the Tag Processor has not been treating its own internal
pointer `bytes_already_parsed` the same way it treats its bookmarks.
That is, when updates are applied to the input document and then
`get_updated_html()` is called, the internal pointer transfers to
the newly-updated content as if no updates had been applied since
the previous call to `get_updated_html()`.

In this patch we're creating a new "shift accumulator" to account for
all of the updates that accrue before calling `get_updated_html()`.
This accumulated shift will be applied when swapping the input document
with the output buffer, which should result in the pointer pointing to
the same logical spot in the document it did before the udpate.

In effect this patch adds a single workaround for treating the
internal pointer like a bookmark, plus a temporary pointer which points
to the beginning of the current tag when calling `get_updated_html()`.
This will preserve the assumption that updating a document doesn't
move that pointer, or shift which tag is currently matched.

Props dmsnell, zieladam.
Fixes #58179.
Built from https://develop.svn.wordpress.org/trunk@55706


git-svn-id: http://core.svn.wordpress.org/trunk@55218 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2023-05-03 11:31:18 +00:00
zieladam 8659101491 HTML API: Fix a case where updates are overlooked when seeking to earlier locations.
This retains the WP_HTML_Tag_Processor attribute updates applied before calling seek() – they were erroneously erased in some cases.

Props dmsnell.
Fixes #58160.




Built from https://develop.svn.wordpress.org/trunk@55675


git-svn-id: http://core.svn.wordpress.org/trunk@55187 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2023-04-21 13:32:25 +00:00
zieladam 1919350606 HTML API: Update code style so it passes when backported into Gutenberg.
This changes the indentation of a variable in class-wp-html-tag-processor.php 
to satisfy both WordPress and Gutenberg linters.

Props dmsnell, ntsekouras.
Fixes #58170.


Built from https://develop.svn.wordpress.org/trunk@55674


git-svn-id: http://core.svn.wordpress.org/trunk@55186 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2023-04-21 12:33:23 +00:00
Bernhard Reiter b116fcdb27 HTML API: Add support for a few invalid HTML comment forms.
- Comments created by means of a tag closer with an invalid tag name, e.g. `</3>`.
 - Comments closed with the invalid `--!>` closer. (Comments should be closed by `-->` but if the `!` appears it will also close it, in error.)
 - Empty tag name elements, which are technically skipped over and aren't comments, e.g. `</>`.

Props dmsnell, costdev.
Fixes #58007.
Built from https://develop.svn.wordpress.org/trunk@55667


git-svn-id: http://core.svn.wordpress.org/trunk@55179 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2023-04-20 17:10:20 +00:00
Bernhard Reiter 3b58785908 HTML API: Ensure attribute updates happen only once for case variants.
When setting a new value for an attribute multiple times and providing
multiple case variations of the attribute name the Tag Processor has
been appending multiple copies of the attribute into the updated HTML.

This means that only the first attribute set determines the value in
the final output, plus the output will //appear// wrong.

In this patch we're adding a test to catch the situation and resolving it
by using the appropriate comparable attribute name as a key for storing
the updates as we go. Previously we stored updates to the attribute by
its given `$name`, but when a new update of the same name with a
case variant was queued, it would not override the previously-enqueued
value as it out to have.

Props dmsnell, zieladam.
Fixes #58146.
Built from https://develop.svn.wordpress.org/trunk@55659


git-svn-id: http://core.svn.wordpress.org/trunk@55171 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2023-04-19 09:27:22 +00:00
Bernhard Reiter fbc006e2b2 HTML API: Add `has_self_closing_flag()` to Tag Processor.
In this patch we're adding `has_self_closing_flag()` to the HTML Tag Processor.
This exposes whether a currently-matched tag contains the self-closing flag `/`.

This information is critical for the evolution of the HTML API in order
to track and parse HTML structure, specifically, knowing whether an
HTML foreign element is self-closing or not.

Props dmsnell, zieladam.
Fixes #58009.
Built from https://develop.svn.wordpress.org/trunk@55619


git-svn-id: http://core.svn.wordpress.org/trunk@55131 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2023-04-04 10:06:27 +00:00
hellofromTonya 95f3aceea7 HTML API: Add bookmark invalidation logic.
While `WP_HTML_Tag_Processor` currently only supports changing a given tag's attributes, the plan is to provide methods to make broader changes (possibly through a subclass of `WP_HTML_Tag_Processor`). The API will have the potential of replacing a tag that a bookmark points to. To prepare, this changeset makes sure that all bookmarks affected by a HTML replacement are invalidated (i.e. released).

Changes:
* Extends the existing loop in `WP_HTML_Tag_Processor::apply_attributes_updates()` that adjusts bookmarks' start and end positions upon HTML changes to check if the entire bookmark is within a portion of the HTML that has been replaced.
* Adds `WP_HTML_Tag_Processor::has_bookmark() to check whether the given bookmark name exists.

References:
* [https://github.com/WordPress/gutenberg/pull/47559 Gutenberg PR 47559]
* [https://github.com/WordPress/gutenberg/releases/tag/v15.3.0 Released in Gutenberg 15.3.0]

Follow-up to [55203].

Props bernhard-reiter, dmsnell, zieladam.
Fixes #57788.
Built from https://develop.svn.wordpress.org/trunk@55555


git-svn-id: http://core.svn.wordpress.org/trunk@55067 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2023-03-16 13:11:24 +00:00
hellofromTonya 2257e9b451 HTML API: Document shorthand usage of the next_tag().
Documents the shorthand usage, i.e. `$this->next_tag( 'img' )`, of `WP_HTML_Tag_Processor::next_tag()`.

Also includes table alignments and formatting adjustments in the class docs.

Follow-up to [55203], [55206].

Props zieladam, poena, dmsnell, costdev, hellofromTonya.
Fixes #57863.
See #57575.
Built from https://develop.svn.wordpress.org/trunk@55477


git-svn-id: http://core.svn.wordpress.org/trunk@55010 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2023-03-07 16:48:21 +00:00
hellofromTonya 39308664bf HTML API: Fix finding RCData and Script tag closers.
Fixes finding the following tag closers `</script>`, `</textarea>`, and `</title>` in `WP_HTML_Tag_Processor`.

Follow-up to [55407], [55203].

Props zieladam, dmsnell, hellofromTonya.
Fixes #57852.
See #57575.
Built from https://develop.svn.wordpress.org/trunk@55469


git-svn-id: http://core.svn.wordpress.org/trunk@55002 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2023-03-06 18:55:21 +00:00
hellofromTonya 1579f32cc6 HTML API: Fix finding bookmarks set on closing tag WP_HTML_Tag_Processor.
Setting a bookmark on a tag should set its "start" position before the opening "<", e.g.:
{{{
<div> Testing a <b>Bookmark</b>
----------------^
}}}

The previous calculation assumed this is always one byte to the left from `$tag_name_starts_at`.

However, in a closing tag that index points to a solidus symbol "/":
{{{
<div> Testing a <b>Bookmark</b>
----------------------------^
}}}

The bookmark should therefore start two bytes before the tag name:
{{{
<div> Testing a <b>Bookmark</b>
---------------------------^
}}}

This changeset achieves this by:
* Using the correct starting index for closing tag bookmarks.
* Adding `array( 'tag_closers' => 'visit' )` in `WP_HTML_Tag_Processor::seek()`.

Follow-up to [55203].

Props zieladam, dmsnell, flixos90.
Fixes #57787.
See #57575.
Built from https://develop.svn.wordpress.org/trunk@55407


git-svn-id: http://core.svn.wordpress.org/trunk@54940 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2023-02-22 20:55:23 +00:00
gziolo 582feb3ffb HTML API: Set $this->html to protected to support subclassing
When the HTML API was introduced a number of fields were switched from private visibility to protected so that Gutenberg and other systems could more easily enhance the behaviors through subclassing. The $this->html property was overlooked but important for systems using the Tag Processor to stich HTML, specifically performing operations on innerHTML and innerText.

Follow-up [55203].
Props dmsnell.
See #57575.


Built from https://develop.svn.wordpress.org/trunk@55402


git-svn-id: http://core.svn.wordpress.org/trunk@54935 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2023-02-22 06:24:22 +00:00
Sergey Biryukov 09cc873d3a Docs: Replace short array syntax in `WP_HTML_Tag_Processor` documentation.
Per [https://developer.wordpress.org/coding-standards/wordpress-coding-standards/php/#declaring-arrays WordPress PHP Coding Standards]:
> Using long array syntax ( `array( 1, 2, 3 )` ) for declaring arrays is generally more readable than short array syntax ( `[ 1, 2, 3 ]` ), particularly for those with vision difficulties. Additionally, it’s much more descriptive for beginners.
> 
> Arrays must be declared using long array syntax.

Original PR from Gutenberg repository:
* [https://github.com/WordPress/gutenberg/pull/47958 #47958 Docs: Don't recommend using short array syntax in WP_HTML_Tag_Processor]

Follow-up to [55203], [55206].

Props aristath, poena.
Fixes #57691.
Built from https://develop.svn.wordpress.org/trunk@55304


git-svn-id: http://core.svn.wordpress.org/trunk@54837 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2023-02-10 10:59:25 +00:00
Andrew Ozz 7bc792bb73 Fix couple of typos in inline docs.
Props: ironprogrammer.
See #57575.
Built from https://develop.svn.wordpress.org/trunk@55206


git-svn-id: http://core.svn.wordpress.org/trunk@54739 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2023-02-03 02:14:19 +00:00
Andrew Ozz be73904dc7 Introduce HTML API with HTML Tag Processor
This commit pulls in the HTML Tag Processor from the Gutenbeg repository.

The Tag Processor attempts to be an HTML5-spec-compliant parser that provides the ability in PHP to find specific HTML tags and then add, remove, or update attributes on that tag. It provides a safe and reliable way to modify the attribute on HTML tags.

More information: https://github.com/WordPress/wordpress-develop/pull/3920.

Props: antonvlasenko, bernhard-reiter, costdev, dmsnell, felixarntz, gziolo, hellofromtonya, zieladam, flixos90, ntsekouras, peterwilsoncc, swissspidy, andrewserong, onemaggie, get_dave, aristath, scruffian, justlevine, andraganescu, noisysocks, dlh, soean, cbirdsong, revgeorge, azaozz.
Fixes #57575.
Built from https://develop.svn.wordpress.org/trunk@55203


git-svn-id: http://core.svn.wordpress.org/trunk@54736 1a063a9b-81f0-0310-95a4-ce76da25c4cd
2023-02-03 01:05:17 +00:00