These token interfaces will make it easier to reason about tokens in the
parser and in specs.
Previously, it was never clear what items could appear in the `parts`
array of a token given a particular `TokenType`. Now, each token interface
declares a labelled tuple for the parts, which helps to document the token
better.
PR Close#42062
Previously, the way templates were tokenized meant that we lost information
about the location of interpolations if the template contained encoded HTML
entities. This meant that the mapping back to the source interpolated strings
could be offset incorrectly.
Also, the source-span assigned to an i18n message did not include leading
whitespace. This confused the output source-mappings so that the first text
nodes of the message stopped at the first non-whitespace character.
This commit makes use of the previous refactorings, where more fine grain
information was provided in text tokens, to enable the parser to identify
the location of the interpolations in the original source more accurately.
Fixes#41034
PR Close#42062
The tests were checking that the source-span of parsed HTML nodes were
accurate, but they were not checking the span when it includes the
"leading trivia", which are given by the `fullStart` rather than `start`
location.
PR Close#42062
When it was tokenized, text content is split into parts that can include
interpolations and encoded entities tokens.
To make this information available to downstream processing, this commit
adds these tokens to the `Text` AST nodes, with suitable processing.
PR Close#42062
The lexer now splits encoded entity tokens out from text and attribute value tokens.
Previously encoded entities would be decoded and the decoded value would be
included as part of the text token of the surrounding text. Now the entities
have their own tokens. There are two scenarios: text and attribute values.
Previously the contents of `<div>Hello & goodbye</div>` would be a single
TEXT token. Now it will be three tokens:
```
TEXT: "Hello "
ENCODED_ENTITY: "&", "&"
TEXT: " goodbye"
```
Previously the attribute value in `<div title="Hello & goodbye">` would be
a single text token. Now it will be three tokens:
```
ATTR_VALUE_TEXT: "Hello "
ENCODED_ENTITY: "&", "&"
ATTR_VALUE_TEXT: " goodbye"
```
- ENCODED_ENTITY tokens have two parts: "decoded" and "encoded".
- ENCODED_ENTITY tokens are always preceded and followed by either TEXT tokens
or ATTR_VALUE_TEXT tokens, depending upon the context, even if they represent
an empty string.
The HTML parser has been modified to recombine these tokens to allow this
refactoring to have limited effect in this commit. Further refactorings
to use these new tokens will follow in subsequent commits.
PR Close#42062
The lexer now splits interpolation tokens out from attribute value tokens.
Previously the attribute value of `<div attr="Hello, {{ name}}">` would be a single
token. Now it will be three tokens:
```
ATTR_VALUE_TEXT: "Hello, "
ATTR_VALUE_INTERPOLATION: "{{", " name", "}}"
ATTR_VALUE_TEXT: ""
```
- ATTR_VALUE_INTERPOLATION tokens have three parts, "start marker",
"expression" and "end marker".
- ATTR_VALUE_INTERPOLATION tokens are always preceded and followed
by TEXT tokens, even if they represent an empty string.
The HTML parser has been modified to recombine these tokens to allow this
refactoring to have limited effect in this commit. Further refactorings
to use these new tokens will follow in subsequent commits.
PR Close#42062
The lexer now splits interpolation tokens out from text tokens.
Previously the contents of `<div>Hello, {{ name}}<div>` would be a single
text token. Now it will be three tokens:
```
TEXT: "Hello, "
INTERPOLATION: "{{", " name", "}}"
TEXT: ""
```
- INTERPOLATION tokens have three parts, "start marker", "expression"
and "end marker".
- INTERPOLATION tokens are always preceded and followed by TEXT tokens,
even if they represent an empty string.
The HTML parser has been modified to recombine these tokens to allow this
refactoring to have limited effect in this commit. Further refactorings
to use these new tokens will follow in subsequent commits.
PR Close#42062
In combination with the TS `noImplicitOverride` compatibility changes,
we also want to follow the best-practice of adding `override` to
members which are implemented as part of abstract classes. This
commit fixes all instances which will be flagged as part of the
custom `no-implicit-override-abstract` TSLint rule.
PR Close#42512
As of ES2021, JavaScript allows using underscores as separators inside numbers, in order to make them more readable (e.g. `1_000_000` vs `1000000`). TypeScript has had support for separators for a while so these changes expand the template parser to handle them as well.
PR Close#42672
Previously the lexer would break out of consuming a text token if it contains
a `<` character. Then if the next characters did not indicate an HTML syntax
item, such as a tag or comment, then it would start a new text token. These
consecutive text tokens are then merged into each other in a post tokenization
step.
In the commit before this, interpolation no longer leaks across text tokens.
The approach given above to handling `<` characters that appear in text is
no longer adequate. This change ensures that the lexer only breaks out of
a text token if the next characters indicate a valid HTML tag, comment,
CDATA etc.
PR Close#42605
When consuming a text token, the lexer tracks whether it is reading characters
from inside an interpolation so that it can identify invalid ICU expressions.
Inside an interpolation there will be no ICU expression so it is safe to
have unmatched `{` characters, but outside an interpolation this is an error.
Previously, if an interpolation was started, by an opening marker (e.g. `{{`)
in a text token but the text came to an end before the closing marker (e.g. `}}`)
then the lexer was not clearing its internal state that tracked that it was
inside an interpolation. When the next text token was being consumed,
the lexer, incorrectly thought it was already within an interpolation.
This resulted in invalid ICU expression errors not being reported.
For example, in the following snippet, the first text block has a prematurely
ended interpolation, and the second text block contains an invalid `{` character.
```
<div>{{</div>
<div>{</div>
```
Previously, the lexer would not have identified this as an error. Now there
will be an EOF error that looks like:
```
TS-995002: Unexpected character "EOF"
(Do you have an unescaped "{" in your template? Use "{{ '{' }}") to escape it.)
```
PR Close#42605
Adds support for shorthand property declarations inside Angular templates. E.g. doing `{foo, bar}` instead of `{foo: foo, bar: bar}`.
Fixes#10277.
PR Close#42421
This commit updates the parser logic to continue to try to match an end
tag to an unclosed open tag on the stack. Previously, it would only
push an error to the list and stop looking at unclosed elements.
For example, the invalid HTML of `<li><div></li>`, has an unclosed
element stack of [`li`, `div`] when it encounters the close `li` tag.
We compare against the previously unclosed tag `div` and see that this is
unexpected. Instead of simply giving up here, we continue to move up the
unclosed tags until we find a match (if there is one).
PR Close#42554
Currently we support safe property (`a?.b`) and method (`a?.b()`) accesses, but we don't handle safe keyed reads (`a?.[0]`) which is inconsistent. These changes expand the compiler in order to support safe key read expressions as well.
PR Close#41911
We have some internal proxies for all of the Jasmine functions, as well as some other helpers. This code hasn't been touched in more than 5 years, it can lead to confusion and it isn't really necessary since the same can be achieved using Jasmine.
These changes remove most of the code and clean up our existing unit tests.
PR Close#42177
TypeScript supports ECMAScript private identifiers. It can happen that
developers intend to access such members from within an expression.
This currently results in an unclear error from the lexer. e.g.
```
'Parser Error: Unexpected token # at column 1 in [{{#myField}}] in C:/test.ts@5:2
```
We could improve such errors by tokenizing private identifiers similar to
how the TypeScript scanner processes them. Later we can report better
errors in the expression parser or in the typecheck block. This commit
causes all private identifier tokens to be disallowed, so it never
reaches the type checker. This is done intentionally as private
identifiers should not be considered valid Angular syntax, especially
because private fields are not guaranteed to be accessible from within
a component/directive definition (e.g. there cases where a template
function is generated outside of the class; which results in private
members not being accessible; and this results in mixed/confusing
behavior).
Fixes#36003.
PR Close#42027
Currently the compiler treats `@page` rules in the same way as `@media`, however that is incorrect and it results in invalid CSS, because `@page` allows style declarations at the root (e.g. `@page (margin: 50%) {}`) and it only allows a limited set of at-rules to be nested into it. Given these restrictions, we can't really encapsulate the styles since they apply at the document level when the user tries to print.
These changes make it so that `@page` rules are preserved so that we don't break the user's CSS.
More information: https://www.w3.org/TR/css-page-3Fixes#26269.
PR Close#41915
This commit adds support for `$` in when selecting attributes.
Resolves#41244.
test(language-service): Add test to expose bug caused by source file change (#41500)
This commit adds a test to expose the bug caused by source file change in
between typecheck programs.
PR Close#41500
PR Close#41567
`@font-face` rules cannot contain nested selectors. Nor can they be
nested under a selector. Normally this would be a syntax error by the
author of the styles. But in some rare cases, such as importing styles
from a library, and applying `:host ::ng-deep` to the imported styles,
we can end up with broken css if the imported styles happen to contain
`@font-face` rules.
This commit works around this problem by sanitizing such cases (erasing
any scoping selectors) during emulated ShadowDOM encapsulation style
processing.
Fixes#41751
PR Close#41815
`EmptyExpr` is somewhat unique, in that it's constructed in a circumstance
where the parser has been looking for a particular token or string of tokens
and has failed to find any. This means the parser state when constructing
`EmptyExpr` is fairly unique.
This gives rise to a bug where the parser constructs `EmptyExpr` with a
backwards span - a `start` value that's beyond the `end` value. This likely
happens because of the strange state the parser is in when recovering with
`EmptyExpr`.
This commit adds a backstop/workaround to avoid constructing such broken
`EmptyExpr` spans (or any other kind of span). Eventually, the parser state
should be fixed such that this does not occur, but that requires a
significant change to the parser's functionality, so a simple fix in th
interim is in order.
PR Close#41581
This commit adds a separate span to `MethodCall` and `SafeMethodCall` which
tracks the text span between the `(` and `)` tokens of the call. Tools like
the Language Service can use this span to more accurately understand a
cursor position within a method call expression.
PR Close#41581
This commit changes the partial compilation so that it outputs declarations
rather than definitions for injectables.
The JIT compiler and the linker are updated to be able to handle these
new declarations.
PR Close#41316
In #41104 the list of used directives was split into two arrays of used
directives and components, but the JIT side was not updated. This commit
fixes the JIT integration by including the list of used components.
Fixes#41318
PR Close#41353
This enumeration will now start to appear in publicly facing code,
as part of declarations, so we remove the R3 to make it less specific
to the internal name for the Ivy renderer/compiler.
PR Close#41231
Now that other values were removed from `R3ResolvedDependencyType`,
its meaning can now be inferred from the other properties in the
`R3DeclareDependencyMetadata` type. This commit removes this enum
and updates the code to work without it.
PR Close#41231
This instruction was created to work around a problem with injecting a
`ChangeDetectorRef` into a pipe. See #31438. This fix required special
metadata for when the thing being injected was a `ChangeDetectorRef`.
Now this is handled by adding a flag `InjectorFlags.ForPipe` to the
`ɵɵdirectiveInject()` call, which avoids the need to special test_cases
`ChangeDetectorRef` in the generated code.
PR Close#41231
This commit changes the partial compilation so that it outputs declaration
calls rather than compiled factory functions.
The JIT compiler and the linker are updated to be able to handle these
new declarations.
PR Close#41231
Adds a `collectCommentNodes` option on `ParseTemplateOptions` which will cause the returned `ParsedTemplate` to include an array of all html comments found in the template.
PR Close#41251
When there was more than one rule in a single style string, only the first
rule was having its `:host` selector processed correctly. Now subsequent
rules will also be processed accurately.
Fixes#41237
PR Close#41261
The current logic in the compiler is to bail when there are errors when
parsing a template into an HTML AST or when there are errors in the i18n
metadata. As a result, a template with these types of parse errors
_will not have any information for the language service_. This is because we
never attempt to conver the HTML AST to a template AST in these
scenarios, so there are no template AST nodes for the language service
to look at for information. In addition, this also means that the errors
are never displayed in the template to the user because there are no
nodes to map the error to.
This commit adds an option to the template parser to temporarily ignore
the html parse and i18n meta errors and always perform the template AST
conversion. At the end, the i18n and HTML parse errors are appended to
the returned errors list. While this seems risky, it at least provides
us with more information than we had before (which was 0) and it's only
done in the context of the language service, when the compiler is
configured to use poisoned data (HTML parse and i18n meta errors can be
interpreted as a "poisoned" template).
fixes angular/vscode-ng-language-service#1140
PR Close#41068
The compiler's parsing code has logic to recover from incomplete open
tags (i.e. `<div`) but the recovery logic does not handle when the
incomplete tag is terminated by an EOF. This commit updates the logic to
allow for the EOF character to be interpreted as the end of the tag open
so that the parser can continue processing. It will then fail to find
the end tag and recover by marking the open tag as incomplete.
Part of https://github.com/angular/vscode-ng-language-service/issues/1140
PR Close#41054
The previous commits refactored the `ShadowCss` emulator to support
desirable use-cases of `:host-context()`, but it dropped support
for passing a comma separated list of selectors to the `:host-context()` .
This commit rectifies that omission, despite the use-case not being
valid according to the ShadowDOM spec, to ensure backward compatibility
with the previous implementation.
PR Close#40494
In `ViewEncapsulation.Emulated` mode the compiler converts `:host` and
`:host-context` pseudo classes into new CSS selectors.
Previously, when there was both `:host-context` and `:host` classes in a
selector, the compiler was generating incorrect selectors. There are two
scenarios:
* Both classes are on the same element (i.e. not separated). E.g.
`:host-context(.foo):host(.bar)`. This setup should only match the
host element if it has both `foo` and `bar` classes. So the generated
CSS selector should be: `.foo.bar<hostmarker>`.
* The `:host` class is on a descendant of the `:host-context`. E.g.
`:host-context(.foo) :host(.bar)`. This setup should only match the
`.foo` selector if it is a proper ancestor of the host (and not on the
host itself). So the generated CSS selector should be:
`.foo .bar<hostmarker>`.
This commit fixes the generation to handle these scenarios.
Fixes#14349
PR Close#40494
In `ViewEncapsulation.Emulated` mode, the compiler must generate additional
combinations of selectors to handle the `:host-context()` pseudo-class function.
Previously, when there is was more than one `:host-context()` selector in a
rule, the compiler was generating invalid selectors.
This commit generates all possible combinations of selectors needed to
match the same elements as the native `:host-context()` selector.
Fixes#19199
PR Close#40494
The parser does not include parenthesis in the AST, so if a LHS
expression would be parenthesized then its start span would start
after the opening parenthesis. Previously, some parent AST nodes would
be created with the start span of its LHS as its own start, so this
resulted in the parent AST node not encompassing the opening parenthesis
in its source span. This commit fixes the issue by capturing the start
index prior to parsing a child AST tree, which is then used as the
start of the source span of the the parent AST node that is parsed.
Fixes#40721
PR Close#40740
