Break out logic from the `next_token()` method into a private method which may call itself recursively. This allows for subclasses to override the `next_token()` method and be assured that each call to `next_token()` corresponds with the consumption of one single token. This also parallels how `WP_HTML_Tag_Processor::next_token()` wraps a private `base_class_next_token()` method.
Props westonruter, jonsurrell.
Fixes#62269.
Built from https://develop.svn.wordpress.org/trunk@59285
git-svn-id: http://core.svn.wordpress.org/trunk@58677 1a063a9b-81f0-0310-95a4-ce76da25c4cd
204d60314b