ngtsc has a robust suite of testing utilities, designed for in-memory
testing of a TypeScript compiler. Previously these utilities lived in the
`test` directory for the compiler-cli package.
This commit moves those utilities to an `ngtsc/testing` package, enabling
them to be depended on separately and opening the door for using them from
the upcoming language server testing infrastructure.
As part of this refactoring, the `fake_core` package (a lightweight API
replacement for @angular/core) is expanded to include functionality needed
for Language Service test use cases.
PR Close#39594
The result of utf-8 encoding a string was represented in a string, where
each individual character represented a single byte according to its
character code. All usages of this data were interested in the byte
itself, so this required conversion from a character back to its code.
This commit simply stores the individual bytes in array to avoid the
conversion. This yields a ~10% performance improvement for i18n message
ID computation.
PR Close#39694
Message ID computation makes extensive use of big integer
multiplications in order to translate the message's fingerprint into
a numerical representation. In large compilations with heavy use of i18n
this was showing up high in profiler sessions.
There are two factors contributing to the bottleneck:
1. a suboptimal big integer representation using strings, which requires
repeated allocation and conversion from a character to numeric digits
and back.
2. repeated computation of the necessary base-256 exponents and their
multiplication factors.
The first bottleneck is addressed using a representation that uses an
array of individual digits. This avoids repeated conversion and
allocation overhead is also greatly reduced, as adding two big integers
can now be done in-place with virtually no memory allocations.
The second point is addressed by a memoized exponentiation pool to
optimize the multiplication of a base-256 exponent.
As an additional optimization are the two 32-bit words now converted to
decimal per word, instead of going through an intermediate byte buffer
and doing the decimal conversion per byte.
The results of these optimizations depend a lot on the number of i18n
messages for which a message should be computed. Benchmarks have shown
that computing message IDs is now ~6x faster for 1,000 messages, ~14x
faster for 10,000 messages, and ~24x faster for 100,000 messages.
PR Close#39694
JIT needs to identify which type is `ChangeDetectorRef`. It was doing so
by importing `ChangeDetectorRef` and than comparing the types. This creates
circular dependency as well as prevents tree shaking. The new solution is
to brand the class with `__ChangeDetectorRef__` so that it can be identified
without creating circular dependency.
PR Close#39621
Similar to #39613, #39609, and #38898, we should store the `keySpan` for
Reference nodes so that we can accurately map from a template node to a
span in the original file. This is most notably an issue at the moment
for directive references `#ref="exportAs"`. The current behavior for the
language service when requesting information for the reference
is that it will return a text span that results in
highlighting the entire source when it should only highlight "ref" (test
added for this case as well).
PR Close#39616
Though we currently have the knowledge of where the `key` for an
event binding appears during parsing, we do not propagate this
information to the output AST. This means that once we produce the
template AST, we have no way of mapping a template position to the key
span alone. The best we can currently do is map back to the
`sourceSpan`. This presents problems downstream, specifically for the
language service, where we cannot provide correct information about a
position in a template because the AST is not granular enough.
This is essentially identical to the change from #38898, but for event
bindings rather than input bindings.
PR Close#39609
Similar to #39609 and #38898, though we currently have the knowledge of where the key for an
attribute appears during parsing, we do not propagate this
information to the output AST. This means that once we produce the
template AST, we have no way of mapping a template position to the key
span alone. The best we can currently do is map back to the
sourceSpan. This presents problems downstream, specifically for the
language service, where we cannot provide correct information about a
position in a template because the AST is not granular enough.
PR Close#39613
Tokenized text node may have leading whitespace skipped from their
source-span. But the source-span is used to compute where there are
interpolated blocks, resulting in placeholder nodes whose source-spans
are offset by the amount of skipped characters.
This fix uses the `fullStart` location of text source-spans for computing
the source-span of placeholders, so that they are accurate.
Fixes#39195
PR Close#39486
This commit ensures that when leading whitespace is skipped by
the tokenizer, the original start location (before skipping) is captured
in the `fullStart` property of the token's source-span.
PR Close#39486
The lexer is able to skip leading trivia in the `start` location of tokens.
This makes the source-span more friendly since things like elements
appear to begin at the start of the opening tag, rather than at the
start of any leading whitespace, which could include newlines.
But some tooling requires the full source-span to be available, such
as when tokenizing a text span into an Angular expression.
This commit simply adds the `fullStart` location to the `ParseSourceSpan`
class, and ensures that places where such spans are cloned, this
property flows through too.
PR Close#39486
In an i18n message, two placeholders next to each other must have
an "empty" message-part to separate them. Previously, the source-span
for this message-part was pointing to the wrong original location.
This caused problems in the generated source-maps and lead to extracted
i18n messages from being rendered incorrectly.
PR Close#39486
For consistency with other generated code, the partial declaration
functions are renamed to use the `ɵɵ` prefix which indicates that it is
generated API.
This commit also removes the declaration from the public API golden
file, as it's not yet considered stable at this point. Once the linker
is finalized will these declaration function be included into the golden
file.
PR Close#39518
This commit implements partial code generation for directives, which
will be transformed by the linker plugin to fully AOT compiled code in
follow-up work.
PR Close#39518
Currently expressions `$event.foo()` and `this.$event.foo()`, as well as `$any(foo)` and
`this.$any(foo)`, are treated as the same expression by the compiler, because `this` is considered
the same implicit receiver as when the receiver is omitted. This introduces the following issues:
1. Any time something called `$any` is used, it'll be stripped away, leaving only the first parameter.
2. If something called `$event` is used anywhere in a template, it'll be preserved as `$event`,
rather than being rewritten to `ctx.$event`, causing the value to undefined at runtime. This
applies to listener, property and text bindings.
These changes resolve the first issue and part of the second one by preserving anything that
is accessed through `this`, even if it's one of the "special" ones like `$any` or `$event`.
Furthermore, these changes only expose the `$event` global variable inside event listeners,
whereas previously it was available everywhere.
Fixes#30278.
PR Close#39323
To support recovery of malformed binding property names like `([a)`,
`[a`, or `()`, the binding parser needs to be more permissive w.r.t. the
kinds of bindings it can detect. This is difficult to do maintainably
with a regex, but is trivial with a "hand-rolled" string parser. This
commit refactors render3's binding attribute parsing to use this method
for multi-delimited bindings (namely via the `()`, `[]`, and `[()]`)
syntax, making the way recovery of malformed bindings in a future patch.
Note that we can keep using a regex for prefix-only binding syntax
(e.g. `bind-`, `ref-`) because validation of the binding is complete
once we have matched the prefix, and the only thing left to do is check
that the binding identifier is non-empty, which is trivial.
Part of #38596
PR Close#39375
This is follow-up from [an earlier discussion](https://github.com/angular/angular/pull/39408#discussion_r511908358).
After some testing, it looks like the type of `Element.attributes` was correct in specifying that it
only has `TextAttribute` instances. This means that the extra checks that filter out `BoundAttribute`
instances from the array isn't necessary. There is another loop a bit further down that actually
extracts the bound i18n attributes.
PR Close#39498
This commit handles the following cases:
- incomplete pipes in a pipe chain
- incomplete arguments in a pipe chain
- incomplete arguments provided to a pipe
- nested pipes
The idea is to unconditionally recover on the presence of a pipe, which
should be okay because expression parsing can be independently between
pipes.
PR Close#39437
Angular-internal type definitions for Trusted Types were added in #39211.
When compiled using the Closure compiler with certain optimization
flags, identifiers from these type definitions (such as createPolicy)
are currently uglified and renamed to shorter strings. This causes
Angular applications compiled in this way to fail to create a Trusted
Types policy, and fall bock to using strings.
To fix this, mark the internal Trusted Types definitions as declarations
using the "declare" keyword. Also convert types to interfaces, for
the reasons explained in https://ncjamieson.com/prefer-interfaces/
PR Close#39471
Currently render3's `parseTemplate` throws away the parsed AST and
returns an empty list of HTML nodes if HTML->R3 translation failed. This
is not preferrable in some contexts like that of a language service,
where we would like a well-formed AST even if it is has errors.
PR Close#39413
Currently `i18n` attributes are treated the same no matter if they have data bindings or not. This
both generates more code since they have to go through the `ɵɵi18nAttributes` instruction and
prevents the translated attributes from being injected using the `@Attribute` decorator.
These changes makes it so that static translated attributes are treated in the same way as regular
static attributes and all other `i18n` attributes go through the old code path.
Fixes#38231.
PR Close#39408
Runtime i18n logic doesn't distinguish `<ng-content>` tag placeholders and regular element tag
placeholders in i18n messages, so there is no need to have a special marker for projection-based
placeholders and element markers can be used instead.
PR Close#39172
There is no actionable change in this commit other than to pretty-print
EOF tokens. Actual parsing of unterminated pipes is already supported,
this just adds a test for it.
Part of #38596
PR Close#39113
The JIT compiler uses the Function constructor to compile arbitrary
strings into executable code at runtime, which causes Trusted Types
violations. To address this, JitEvaluator is instead made to use the
Trusted Types compatible Function constructor introduced by Angular's
Trusted Types policy for JIT.
PR Close#39210
Introduce a Trusted Types policy for use by Angular's JIT compiler named
angular#unsafe-jit. As the compiler turns arbitrary untrusted strings
into executable code at runtime, using Angular's main Trusted Types
policy does not seem appropriate, unless it can be ensured that the
provided strings are indeed trusted. Until then, this JIT policy can be
allowed by applications that rely on the JIT compiler but want to
enforce Trusted Types, knowing that a compromise of the JIT compiler can
lead to arbitrary script execution. In particular, this is required for
enabling Trusted Types in Angular unit tests, since they make use of the
JIT compiler.
Also export the internal Trusted Types definitions from the core package
so that they can be used in the compiler package.
PR Close#39210
Angular treats constant values of attributes and properties in templates
as secure. This means that these values are not sanitized, and are
instead passed directly to the corresponding setAttribute or setProperty
function. In cases where the given attribute or property is
security-sensitive, this causes a Trusted Types violation.
To address this, functions for promoting constant strings to each of the
three Trusted Types are introduced to Angular's private codegen API. The
compiler is updated to wrap constant strings with calls to these
functions as appropriate when constructing the `consts` array. This is
only done for security-sensitive attributes and properties, as
classified by Angular's dom_security_schema.
PR Close#39211
At a high level, the current shadow DOM shim logic works by escaping the content of a CSS rule
(e.g. `div {color: red;}` becomes `div {%BLOCK%}`), using a regex to parse out things like the
selector and the rule body, and then re-adding the content after the selector has been modified.
The problem is that the regex has to be very broad in order capture all of the different use cases,
which can cause it to match strings suffixed with a semi-colon in some places where it shouldn't,
like this URL from Google Fonts `https://fonts.googleapis.com/css2?family=Roboto:wght@400;500&display=swap`.
Most of the time this is fine, because the logic that escapes the rule content to `%BLOCK%` will
have converted it to something that won't be matched by the regex. However, it breaks down for rules
like `@import` which don't have a body, but can still have quoted content with characters that can
match the regex.
These changes resolve the issue by making a second pass over the escaped string and replacing all
of the remaining quoted content with `%QUOTED%` before parsing it with the regex. Once everything
has been processed, we make a final pass where we restore the quoted content.
In a previous iteration of this PR, I went with a shorter approach which narrowed down the
regex so that it doesn't capture rules without a body. It fixed the issue, but it also ended
up breaking some of the more contrived unit test cases. I decided not to pursue it further, because
we would've ended up with a very long and brittle regex that likely would've broken in even weirder
ways.
Fixes#38587.
PR Close#38716
Removes `ViewEncapsulation.Native` which has been deprecated for several major versions.
BREAKING CHANGES:
* `ViewEncapsulation.Native` has been removed. Use `ViewEncapsulation.ShadowDom` instead. Existing
usages will be updated automatically by `ng update`.
PR Close#38882
Expressions within ICU expressions in templates were not previously
type-checked, as they were skipped while traversing the elements
within a template. This commit enables type checking of these
expressions by actually visiting the expressions.
BREAKING CHANGE:
Expressions within ICUs are now type-checked again, fixing a regression
in Ivy. This may cause compilation failures if errors are found in
expressions that appear within an ICU. Please correct these expressions
to resolve the type-check errors.
Fixes#39064
PR Close#39072
Prior to this change, expressions within ICUs would have a source span
corresponding with the whole ICU. This commit narrows down the source
spans of these expressions to the exact location in the source file, as
a prerequisite for reporting type check errors within these expressions.
PR Close#39072
The template binding API in @angular/compiler exposes information about a
template that is synthesized from the template structure and its scope
(associated directives and pipes).
This commit introduces a new API, `getEntitiesInTemplateScope`, which
accepts a `Template` object (or `null` to indicate the root template) and
returns all `Reference` and `Variable` nodes that are visible at that level
of the template, including those declared in parent templates.
This API is needed by the template type-checker to support autocompletion
APIs for the Language Service.
PR Close#39048
The expression parser already has support for recovering on malformed
property reads, but did not have tests describing the recovered ast in
such cases. This commit adds tests to demonstrate such cases; in
particular, the recovered ast is a full PropertyRead but with an empty
property name. This is likely the most preferred option, as it does not
constrain consumers of the AST to what the property name should look
like. Furthermore, we cannot mark the property name as empty in any
other way (e.g. an EmptyExpr) because the property name, as of present,
is a string field rather than an AST itself.
Note that tokens past a malformed property read are not preserved in the
AST (for example in `foo.1234`, `1234` is not preserved in the AST).
This is because the extra tokens do not belong to the singular
expression created by the property read, and there is not a meaningful
way to interpret a secondary expression in a single parsed expression.
Part of #38596
PR Close#38998
This patch refactors the interpolation parser to do so iteratively
rather than using a regex. Doing so prepares us for supporting granular
recovery on poorly-formed interpolations, for example when an
interpolation does not terminate (`{{ 1 + 2`) or is not terminated
properly (`{{ 1 + 2 {{ 2 + 3 }}`).
Part of #38596
PR Close#38977
This patch adds support for recovering well-formed (and near-complete)
ASTs for semantically malformed keyed reads and keyed writes. See the
added tests for details on the types of semantics we can now recover;
in particular, notice that some assumptions are made about the form of
a keyed read/write intended by a user. For example, in the malformed
expression `a[1 + = 2`, we assume that the user meant to write a binary
expression for the key of `a`, and assign that key the value `2`. In
particular, we now parse this as `a[1 + <empty expression>] = 2`. There
are some different interpretations that can be made here, but I think
this is reasonable.
The actual changes in the parser code are fairly minimal (a nice
surprise!); the biggest addition is a `writeContext` that marks whether
the `=` operator can serve as a recovery point after error detection.
Part of #38596
PR Close#39004
Currently it is impossible to determine the source of a binding that
generates `BoundAttribute` because all bound attributes generated from a
microsyntax expression share the same source span.
For example, in
```html
<div *ngFor="let item of items; trackBy: trackByFn"></div>
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
source span for all `BoundAttribute`s generated from microsyntax
```
the `BoundAttribute` for both `ngForOf` and `ngForTrackBy`
share the same source span.
A lot of hacks were necessary in View Engine language service to work
around this limitation. It was done by inspecting the whole source span
then figuring out the relative position of the cursor.
With this change, we introduce a flag to set the binding span as the
source span of the `ParsedProperty` in Ivy AST.
This flag is needed so that we don't have to change VE ASTs.
Note that in the binding parser, we already set `bindingSpan` as the
source span for a `ParsedVariable`, and `keySpan` as the source span for
a literal attribute. This change makes the Ivy AST more consistent by
propagating the binding span to `ParsedProperty` as well.
PR Close#39036
This commit updates the symbols in the TemplateTypeCheck API and methods
for retrieving them:
* Include `isComponent` and `selector` for directives so callers can determine which
attributes on an element map to the matched directives.
* Add a new `TextAttributeSymbol` and return this when requesting a symbol for a `TextAttribute`.
* When requesting a symbol for `PropertyWrite` and `MethodCall`, use the
`nameSpan` to retrieve symbols.
* Add fix to retrieve generic directives attached to elements/templates.
PR Close#38844
Now that we have `keySpan` for `BoundAttribute` (implemented in
https://github.com/angular/angular/pull/38898) we could do the same
for `Variable`.
This would allow us to distinguish the LHS and RHS from the whole source
span.
PR Close#38965
Some compiler tests take a long time to run, even using multiple
executors. A profiling session revealed that most time is spent in
parsing source files, especially the default libraries are expensive to
parse.
The default library files are constant across all tests, so this commit
introduces a shared cache of parsed source files of the default
libraries. This achieves a significant improvement for several targets
on my machine:
//packages/compiler-cli/test/compliance: from 23s to 5s.
//packages/compiler-cli/test/ngtsc: from 115s to 11s.
Note that the number of shards for the compliance tests has been halved,
as the extra shards no longer provide any speedup.
PR Close#38909
Though we currently have the knowledge of where the `key` for an
attribute binding appears during parsing, we do not propagate this
information to the output AST. This means that once we produce the
template AST, we have no way of mapping a template position to the key
span alone. The best we can currently do is map back to the
`sourceSpan`. This presents problems downstream, specifically for the
language service, where we cannot provide correct information about a
position in a template because the AST is not granular enough.
PR Close#38898
Using an interface makes the code cleaner and more readable.
This change also adds the `range` property to the type to be used
for source-mapping.
PR Close#38775
Let's say we have a code like
```html
<div<span>123</span>
```
Currently this gets parsed into a tree with the element tag `div<span`.
This has at least two downsides:
- An incorrect diagnostic that `</span>` doesn't close an element is
emitted.
- A consumer of the parse tree using it for editor services is unable to
provide correct completions for the opening `<span>` tag.
This patch attempts to fix both issues by instead parsing the code into
the same tree that would be parsed for `<div></div><span>123</span>`.
In particular, we do this by optimistically scanning an open tag as
usual, but if we do not notice a terminating '>', we mark the tag as
"incomplete". A parser then emits an error for the incomplete tag and
adds a synthetic (recovered) element node to the tree with the
incomplete open tag's name.
What's the downside of this? For one, a breaking change.
<ol>
<li>
The first breaking change is that `<` symbols that are ambiguously text
or opening tags will be parsed as opening tags instead of text in
element bodies. Take the code
```html
<p>a<b</p>
```
Clearly we cannot have the best of both worlds, and this patch chooses
to swap the parsing strategy to support the new feature. Of course, `<`
can still be inserted as text via the `<` entity.
</li>
</ol>
Part of #38596
PR Close#38681
The current tests print out the span numbers, which are really difficult to verify
since it requires manually going to the template string and looking at what
characters appear within those indexes. The better humanization would be
to use the toString method of the spans, which prints the span text itself
PR Close#38902
Common AST formats such as TS and Babel do not use a separate
node for comments, but instead attach comments to other AST nodes.
Previously this was worked around in TS by creating a `NotEmittedStatement`
AST node to attach the comment to. But Babel does not have this facility,
so it will not be a viable approach for the linker.
This commit refactors the output AST, to remove the `CommentStmt` and
`JSDocCommentStmt` nodes. Instead statements have a collection of
`leadingComments` that are rendered/attached to the final AST nodes
when being translated or printed.
PR Close#38811
Recent work on compiler internals in #38539 led to an unexpected failure,
where a pipe used exclusively inside of an ICU would no longer be
emitted into the compilation output. This caused runtime errors due to
missing pipes.
The issue occurred because the change in #38539 would determine the set
of used pipes up-front, independent from the template compilation using
the `R3TargetBinder`. However, `R3TargetBinder` did not consider
expressions within ICUs, so any pipe usages within those expressions
would not be detected. This fix unblocks #38539 and also concerns
upcoming linker work, given that prelink compilations would not go
through full template compilation but only `R3TargetBinder`.
PR Close#38810
In a microsyntax expressions, some attributes are not bound after
desugaring. For example,
```html
<div *ngFor="let item of items">
</div>
```
gets desugared to
```html
<ng-template ngFor let-items [ngForOf]="items">
</ngtemplate>
```
In this case, `ngFor` should be a literal attribute with no RHS value.
Therefore, its source span should be just the `keySpan` and not the
source span of the original template node.
This allows language service to precisely pinpoint different spans in a
microsyntax to provide accurate information.
PR Close#38766
Previously, localized strings had very limited or incorrect source-mapping
information available.
Now the i18n AST nodes and related output AST nodes include source-span
information about message-parts and placeholders - including closing tag
placeholders.
This information is then used when generating the final localized string
ASTs to ensure that the correct source-mapping is rendered.
See #38588 (comment)
PR Close#38645
The `MessagePiece` and derived classes, `LiteralPiece` and `PlaceholderPiece`
need to be referenced in the `LocalizedString` output AST class, so that we
can render the source-spans of each piece.
PR Close#38645
Alex Rickabaugh
JoostK
Misko Hevery
Andrew Scott
Pete Bacon Darwin
Kristiyan Kostadinov
ayazhafiz
Bjarki
Alan Agius
pmartijena
Andrew Kushnir
Keen Yee Liau
Keen Yee Liau