In Ivy, template type-checking has 3 modes: basic, full, and strict. The
primary difference between basic and full modes is that basic mode only
checks the top-level template, whereas full mode descends into nested
templates (embedded views like ngIfs and ngFors). Ivy applies this approach
to all of its template type-checking, including the DOM schema checks which
validate whether an element is a valid component/directive or not.
View Engine has both the basic and the full mode, with the same distinction.
However in View Engine, DOM schema checks happen for the full template even
in the basic mode.
Ivy's behavior here is technically a "fix" as it does not make sense for
some checks to apply to the full template and others only to the top-level
view. However, since g3 relies exclusively on the basic mode of checking and
developers there are used to DOM checks applying throughout their template,
this commit re-enables the nested schema checks even in basic mode only in
g3. This is done by enabling the checks only when Closure Compiler
annotations are requested.
Outside of g3, it's recommended that applications use at least the full mode
of checking (controlled by the `fullTemplateTypeCheck` flag), and ideally
the strict mode (`strictTemplates`).
PR Close#38943
The statements generated in the TCB are optimized for performance and producing diagnostics.
These optimizations can result in generating a TCB that does not have all the information
needed by the `TemplateTypeChecker` for retrieving `Symbol`s. For example, as an optimization,
the TCB will not generate variable declaration statements for directives that have no
references, inputs, or outputs. However, the `TemplateTypeChecker` always needs these
statements to be present in order to provide `ts.Symbol`s and `ts.Type`s for the directives.
This commit adds logic to the TCB generation to ensure the required
information is available in a form that the `TemplateTypeChecker` can
consume. It also adds an option to the `NgCompiler` that makes this
generation configurable.
PR Close#38618
The template type-checking engine includes utilities for creating
`ts.Diagnostic`s for component templates. Previously only the template type-
checker itself created such diagnostics. However, the template parser also
produces errors which should be represented as template diagnostics.
This commit prepares for that conversion by extracting the machinery for
producing template diagnostics into its own sub-package, so that other parts
of the compiler can depend on it without depending on the entire template
type-checker.
PR Close#38576
The compiler does not currently report errors when there's an `@Input()`
for a `private`, `protected`, or `readonly` directive/component class member.
This change adds an option to enable reporting errors when a template
attempts to bind to one of these restricted input fields.
PR Close#38249
Previously, the `TemplateTypeChecker` abstraction allowed fetching
diagnostics for a single file, but under the hood would generate type
checking code for the entire program to satisfy the request.
With this commit, an `OptimizeFor` hint is passed to `getDiagnosticsForFile`
which indicates whether the user intends to request diagnostics for the
whole program or is truly interested in just the single file. If the latter,
the `TemplateTypeChecker` can perform only the work needed to produce
diagnostics for just that file, thus returning answers more efficiently.
PR Close#38105
This commit significantly refactors the 'typecheck' package to introduce a
new abstraction, the `TemplateTypeChecker`. To achieve this:
* a 'typecheck:api' package is introduced, containing common interfaces that
consumers of the template type-checking infrastructure can depend on
without incurring a dependency on the template type-checking machinery as
a whole.
* interfaces for `TemplateTypeChecker` and `TypeCheckContext` are introduced
which contain the abstract operations supported by the implementation
classes `TemplateTypeCheckerImpl` and `TypeCheckContextImpl` respectively.
* the `TemplateTypeChecker` interface supports diagnostics on a whole
program basis to start with, but the implementation is purposefully
designed to support incremental diagnostics at a per-file or per-component
level.
* `TemplateTypeChecker` supports direct access to the type check block of a
component.
* the testing utility is refactored to be a lot more useful, and new tests
are added for the new abstraction.
PR Close#38105
Incremental compilation allows for the output state of one compilation to be
reused as input to the next compilation. This involves retaining references
to instances from prior compilations, which must be done carefully to avoid
memory leaks.
This commit fixes such a leak with a complicated retention chain:
* `TrackedIncrementalBuildStrategy` unnecessarily hangs on to the previous
`IncrementalDriver` (state of the previous compilation) once the current
compilation completes.
In general this is unnecessary, but should be safe as long as the chain
only goes back one level - if the `IncrementalDriver` doesn't retain any
previous `TrackedIncrementalBuildStrategy` instances. However, this does
happen:
* `NgCompiler` indirectly causes retention of previous `NgCompiler`
instances (and thus previous `TrackedIncrementalBuildStrategy` instances)
through accidental capture of the `this` context in a closure created in
its constructor. This closure is wrapped in a `ts.ModuleResolutionCache`
used to create a `ModuleResolver` class, which is passed to the program's
`TraitCompiler` on construction.
* The `IncrementalDriver` retains a reference to the `TraitCompiler` of the
previous compilation, completing the reference chain.
The final retention chain thus looks like:
* `TrackedIncrementalBuildStrategy` of current program
* `.previous`: `IncrementalDriver` of previous program
* `.lastGood.traitCompiler`: `TraitCompiler`
* `.handlers[..].moduleResolver.moduleResolutionCache`: cache
* (via `getCanonicalFileName` closure): `NgCompiler`
* `.incrementalStrategy`: `TrackedIncrementalBuildStrategy` of previous
program.
The closure link is the "real" leak here. `NgCompiler` is creating a closure
for `getCanonicalFileName`, delegating to its
`this.adapter.getCanonicalFileName`, for the purposes of creating a
`ts.ModuleResolutionCache`. The fact that the closure references
`NgCompiler` thus eventually causes previous `NgCompiler` iterations to be
retained. This is also potentially problematic due to the shared nature of
`ts.ModuleResolutionCache`, which is potentially retained across multiple
compilations intentionally.
This commit fixes the first two links in the retention chain: the build
strategy is patched to not retain a `previous` pointer, and the `NgCompiler`
is patched to not create a closure in the first place, but instead pass a
bound function. This ensures that the `NgCompiler` does not retain previous
instances of itself in the first place, even if the build strategy does
end up retaining the previous incremental state unnecessarily.
The third link (`IncrementalDriver` unnecessarily retaining the whole
`TraitCompiler`) is not addressed in this commit as it's a more
architectural problem that will require some refactoring. However, the leak
potential of this retention is eliminated thanks to fixing the first two
issues.
PR Close#37835
Commit 24b2f1da2b introduced an `NgCompiler` which operates on a
`ts.Program` independently of the `NgtscProgram`. The NgCompiler got its
`IncrementalDriver` (for incremental reuse of Angular compilation results)
by looking at a monkey-patched property on the `ts.Program`.
This monkey-patching operation causes problems with the Angular indexer
(specifically, it seems to cause the indexer to retain too much of prior
programs, resulting in OOM issues). To work around this, `IncrementalDriver`
reuse is now handled by a dedicated `IncrementalBuildStrategy`. One
implementation of this interface is used by the `NgtscProgram` to perform
the old-style reuse, relying on the previous instance of `NgtscProgram`
instead of monkey-patching. Only for `NgTscPlugin` is the monkey-patching
strategy used, as the plugin sits behind an interface which only provides
access to the `ts.Program`, not a prior instance of the plugin.
PR Close#37339
In v7 of Angular we removed `tsickle` from the default `ngc` pipeline.
This had the negative potential of breaking ES2015 output and SSR due
to a limitation in TypeScript.
TypeScript by default preserves type information for decorated constructor
parameters when `emitDecoratorMetadata` is enabled. For example,
consider this snippet below:
```
@Directive()
export class MyDirective {
constructor(button: MyButton) {}
}
export class MyButton {}
```
TypeScript would generate metadata for the `MyDirective` class it has
a decorator applied. This metadata would be needed in JIT mode, or
for libraries that provide `MyDirective` through NPM. The metadata would
look as followed:
```
let MyDirective = class MyDir {}
MyDirective = __decorate([
Directive(),
__metadata("design:paramtypes", [MyButton]),
], MyDirective);
let MyButton = class MyButton {}
```
Notice that TypeScript generated calls to `__decorate` and
`__metadata`. These calls are needed so that the Angular compiler
is able to determine whether `MyDirective` is actually an directive,
and what types are needed for dependency injection.
The limitation surfaces in this concrete example because `MyButton`
is declared after the `__metadata(..)` call, while `__metadata`
actually directly references `MyButton`. This is illegal though because
`MyButton` has not been declared at this point. This is due to the
so-called temporal dead zone in JavaScript. Errors like followed will
be reported at runtime when such file/code evaluates:
```
Uncaught ReferenceError: Cannot access 'MyButton' before initialization
```
As noted, this is a TypeScript limitation because ideally TypeScript
shouldn't evaluate `__metadata`/reference `MyButton` immediately.
Instead, it should defer the reference until `MyButton` is actually
declared. This limitation will not be fixed by the TypeScript team
though because it's a limitation as per current design and they will
only revisit this once the tc39 decorator proposal is finalized
(currently stage-2 at time of writing).
Given this wontfix on the TypeScript side, and our heavy reliance on
this metadata in libraries (and for JIT mode), we intend to fix this
from within the Angular compiler by downleveling decorators to static
properties that don't need to evaluate directly. For example:
```
MyDirective.ctorParameters = () => [MyButton];
```
With this snippet above, `MyButton` is not referenced directly. Only
lazily when the Angular runtime needs it. This mitigates the temporal
dead zone issue caused by a limitation in TypeScript's decorator
metadata output. See: https://github.com/microsoft/TypeScript/issues/27519.
In the past (as noted; before version 7), the Angular compiler by
default used tsickle that already performed this transformation. We
moved the transformation to the CLI for JIT and `ng-packager`, but now
we realize that we can move this all to a single place in the compiler
so that standalone ngc consumers can benefit too, and that we can
disable tsickle in our Bazel `ngc-wrapped` pipeline (that currently
still relies on tsickle to perform this decorator processing).
This transformation also has another positive side-effect of making
Angular application/library code more compatible with server-side
rendering. In principle, TypeScript would also preserve type information
for decorated class members (similar to how it did that for constructor
parameters) at runtime. This becomes an issue when your application
relies on native DOM globals for decorated class member types. e.g.
```
@Input() panelElement: HTMLElement;
```
Your application code would then reference `HTMLElement` directly
whenever the source file is loaded in NodeJS for SSR. `HTMLElement`
does not exist on the server though, so that will become an invalid
reference. One could work around this by providing global mocks for
these DOM symbols, but that doesn't match up with other places where
dependency injection is used for mocking DOM/browser specific symbols.
More context in this issue: #30586. The TL;DR here is that the Angular
compiler does not care about types for these class members, so it won't
ever reference `HTMLElement` at runtime.
Fixes#30106. Fixes#30586. Fixes#30141.
Resolves FW-2196. Resolves FW-2199.
PR Close#37382
`NgCompiler` is the heart of ngtsc and can be used to analyze and compile
Angular programs in a variety of environments. Most of these integrations
rely on `NgProgram` and the creation of an `NgCompilerHost` in order to
create a `ts.Program` with the right shape for `NgCompiler`.
However, certain environments (such as the Angular Language Service) have
their own mechanisms for creating `ts.Program`s that don't make use of a
`ts.CompilerHost`. In such environments, an `NgCompilerHost` does not make
sense.
This commit breaks the dependency of `NgCompiler` on `NgCompilerHost` and
extracts the specific interface of the host on which `NgCompiler` depends
into a new interface, `NgCompilerAdapter`. This interface includes methods
from `ts.CompilerHost`, the `ExtendedTsCompilerHost`, as well as APIs from
`NgCompilerHost`.
A consumer such as the language service can implement this API without
needing to jump through hoops to create an `NgCompilerHost` implementation
that somehow wraps its specific environment.
PR Close#37118
The type checking infrastrure uses file-paths that may come from the
TS compiler. Such paths will have been canonicalized, and so the type
checking classes must also canonicalize paths when matching.
PR Close#36859
Previously in v9, we deprecated the pattern of undecorated base classes
that rely on Angular features. We ran a migration for this in version 9
and will run the same on in version 10 again.
To ensure that projects do not regress and start using the unsupported
pattern again, we report an error in ngtsc if such undecorated classes
are discovered.
We keep the compatibility code enabled in ngcc so that libraries
can be still be consumed, even if they have not been migrated yet.
Resolves FW-2130.
PR Close#36921
This optimization builds on a lot of prior work to finally make type-
checking of templates incremental.
Incrementality requires two main components:
- the ability to reuse work from a prior compilation.
- the ability to know when changes in the current program invalidate that
prior work.
Prior to this commit, on every type-checking pass the compiler would
generate new .ngtypecheck files for each original input file in the program.
1. (Build #1 main program): empty .ngtypecheck files generated for each
original input file.
2. (Build #1 type-check program): .ngtypecheck contents overridden for those
which have corresponding components that need type-checked.
3. (Build #2 main program): throw away old .ngtypecheck files and generate
new empty ones.
4. (Build #2 type-check program): same as step 2.
With this commit, the `IncrementalDriver` now tracks template type-checking
_metadata_ for each input file. The metadata contains information about
source mappings for generated type-checking code, as well as some
diagnostics which were discovered at type-check analysis time. The actual
type-checking code is stored in the TypeScript AST for type-checking files,
which is now re-used between programs as follows:
1. (Build #1 main program): empty .ngtypecheck files generated for each
original input file.
2. (Build #1 type-check program): .ngtypecheck contents overridden for those
which have corresponding components that need type-checked, and the
metadata registered in the `IncrementalDriver`.
3. (Build #2 main program): The `TypeCheckShimGenerator` now reuses _all_
.ngtypecheck `ts.SourceFile` shims from build #1's type-check program in
the construction of build #2's main program. Some of the contents of
these files might be stale (if a component's template changed, for
example), but wholesale reuse here prevents unnecessary changes in the
contents of the program at this point and makes TypeScript's job a lot
easier.
4. (Build #2 type-check program): For those input files which have not
"logically changed" (meaning components within are semantically the same
as they were before), the compiler will re-use the type-check file
metadata from build #1, and _not_ generate a new .ngtypecheck shim.
For components which have logically changed or where the previous
.ngtypecheck contents cannot otherwise be reused, code generation happens
as before.
PR Close#36211
As a performance optimization, this commit splits the single
__ngtypecheck__.ts file which was previously added to the user's program as
a container for all template type-checking code into multiple .ngtypecheck
shim files, one for each original file in the user's program.
In larger applications, the generation, parsing, and checking of this single
type-checking file was a huge performance bottleneck, with the file often
exceeding 1 MB in text content. Particularly in incremental builds,
regenerating this single file for the entire application proved especially
expensive.
This commit introduces a new strategy for template type-checking code which
makes use of a new interface, the `TypeCheckingProgramStrategy`. This
interface abstracts the process of creating a new `ts.Program` to type-check
a particular compilation, and allows the mechanism there to be kept separate
from the more complex logic around dealing with multiple .ngtypecheck files.
A new `TemplateTypeChecker` hosts that logic and interacts with the
`TypeCheckingProgramStrategy` to actually generate and return diagnostics.
The `TypeCheckContext` class, previously the workhorse of template type-
checking, is now solely focused on collecting and generating type-checking
file contents.
A side effect of implementing the new `TypeCheckingProgramStrategy` in this
way is that the API is designed to be suitable for use by the Angular
Language Service as well. The LS also needs to type-check components, but
has its own method for constructing a `ts.Program` with type-checking code.
Note that this commit does not make the actual checking of templates at all
_incremental_ just yet. That will happen in a future commit.
PR Close#36211
Shim generation was built on a lie.
Shims are files added to the program which aren't original files authored by
the user, but files authored effectively by the compiler. These fall into
two categories: files which will be generated (like the .ngfactory shims we
generate for View Engine compatibility) as well as files used internally in
compilation (like the __ng_typecheck__.ts file).
Previously, shim generation was driven by the `rootFiles` passed to the
compiler as input. These are effectively the `files` listed in the
`tsconfig.json`. Each shim generator (e.g. the `FactoryGenerator`) would
examine the `rootFiles` and produce a list of shim file names which it would
be responsible for generating. These names would then be added to the
`rootFiles` when the program was created.
The fatal flaw here is that `rootFiles` does not always account for all of
the files in the program. In fact, it's quite rare that it does. Users don't
typically specify every file directly in `files`. Instead, they rely on
TypeScript, during program creation, starting with a few root files and
transitively discovering all of the files in the program.
This happens, however, during `ts.createProgram`, which is too late to add
new files to the `rootFiles` list.
As a result, shim generation was only including shims for files actually
listed in the `tsconfig.json` file, and not for the transitive set of files
in the user's program as it should.
This commit completely rewrites shim generation to use a different technique
for adding files to the program, inspired by View Engine's shim generator.
In this new technique, as the program is being created and `ts.SourceFile`s
are being requested from the `NgCompilerHost`, shims for those files are
generated and a reference to them is patched onto the original file's
`ts.SourceFile.referencedFiles`. This causes TS to think that the original
file references the shim, and causes the shim to be included in the program.
The original `referencedFiles` array is saved and restored after program
creation, hiding this little hack from the rest of the system.
The new shim generation engine differentiates between two kinds of shims:
top-level shims (such as the flat module entrypoint file and
__ng_typecheck__.ts) and per-file shims such as ngfactory or ngsummary
files. The former are included via `rootFiles` as before, the latter are
included via the `referencedFiles` of their corresponding original files.
As a result of this change, shims are now correctly generated for all files
in the program, not just the ones named in `tsconfig.json`.
A few mitigating factors prevented this bug from being realized until now:
* in g3, `files` does include the transitive closure of files in the program
* in CLI apps, shims are not really used
This change also makes use of a novel technique for associating information
with source files: the use of an `NgExtension` `Symbol` to patch the
information directly onto the AST object. This is used in several
circumstances:
* For shims, metadata about a `ts.SourceFile`'s status as a shim and its
origins are held in the extension data.
* For original files, the original `referencedFiles` are stashed in the
extension data for later restoration.
The main benefit of this technique is a lot less bookkeeping around `Map`s
of `ts.SourceFile`s to various kinds of data, which need to be tracked/
invalidated as part of incremental builds.
This technique is based on designs used internally in the TypeScript
compiler and is serving as a prototype of this design in ngtsc. If it works
well, it could have benefits across the rest of the compiler.
PR Close#36211
The compiler needs to track the dependencies of a component, including any
NgModules which happen to be present in a component's scope. If an upstream
NgModule changes, any downstream components need to have their templates
re-compiled and re-typechecked.
Previously, the compiler handled this well for the A -> B -> C case where
module A imports module B which re-exports module C. However, it fell apart
in the A -> B -> C -> D case, because previously tracking focused on changes
to components/directives in the scope, and not NgModules specifically.
This commit introduces logic to track which NgModules contributed to a given
scope, and treat them as dependencies of any components within.
This logic also contains a bug, which is intentional for now. It
purposefully does not track transitive dependencies of the NgModules which
contribute to a scope. If it did, using the current dependency system, this
would treat all components and directives (even those not exported into the
scope) as dependencies, causing a major performance bottleneck. Only those
dependencies which contributed to the module's export scope should be
considered, but the current system is incapable of making this distinction.
This will be fixed at a later date.
PR Close#36211
The html parser already normalizes line endings (converting `\r\n` to `\n`)
for most text in templates but it was missing the expressions of ICU expansions.
In ViewEngine backticked literal strings, used to define inline templates,
were already normalized by the TypeScript parser.
In Ivy we are parsing the raw text of the source file directly so the line
endings need to be manually normalized.
This change ensures that inline templates have the line endings of ICU
expression normalized correctly, which matches the ViewEngine.
In ViewEngine external templates, defined in HTML files, the behavior was
different, since TypeScript was not normalizing the line endings.
Specifically, ICU expansion "expressions" are not being normalized.
This is a problem because it means that i18n message ids can be different on
different machines that are setup with different line ending handling,
or if the developer moves a template from inline to external or vice versa.
The goal is always to normalize line endings, whether inline or external.
But this would be a breaking change since it would change i18n message ids
that have been previously computed. Therefore this commit aligns the ivy
template parsing to have the same "buggy" behavior for external templates.
There is now a compiler option `i18nNormalizeLineEndingsInICUs`, which
if set to `true` will ensure the correct non-buggy behavior. For the time
being this option defaults to `false` to ensure backward compatibility while
allowing opt-in to the desired behavior. This option's default will be
flipped in a future breaking change release.
Further, when this option is set to `false`, any ICU expression tokens,
which have not been normalized, are added to the `ParseResult` from the
`HtmlParser.parse()` method. In the future, this collection of tokens could
be used to diagnose and encourage developers to migrate their i18n message
ids. See FW-2106.
Closes#36725
PR Close#36741
Under View Engine's default (non-fullTemplateTypeCheck) checking, object and
array literals which appear in templates are treated as having type `any`.
This allows a number of patterns which would not otherwise compile, such as
indexing an object literal by a string:
```html
{{ {'a': 1, 'b': 2}[value] }}
```
(where `value` is `string`)
Ivy, meanwhile, has always inferred strong types for object literals, even
in its compatibility mode. This commit fixes the bug, and adds the
`strictLiteralTypes` flag to specifically control this inference. When the
flag is `false` (in compatibility mode), object and array literals receive
the `any` type.
PR Close#35462
This commit moves the calculation of `ignoreFiles` - the set of files to be
ignored by a consumer of the `NgCompiler` API - from its `prepareEmit`
operation to its initialization. It's now available as a field on
`NgCompiler`.
This will allow a consumer to skip gathering diagnostics for `ignoreFiles`
as well as skip emit.
PR Close#34792
Previously, NgtscProgram lived in the main @angular/compiler-cli package
alongside the legacy View Engine compiler. As a result, the main package
depended on all of the ngtsc internal packages, and a significant portion of
ngtsc logic lived in NgtscProgram.
This commit refactors NgtscProgram and moves the main logic of compilation
into a new 'core' package. The new package defines a new API which enables
implementers of TypeScript compilers (compilers built using the TS API) to
support Angular transpilation as well. It involves a new NgCompiler type
which takes a ts.Program and performs Angular analysis and transformations,
as well as an NgCompilerHost which wraps an input ts.CompilerHost and adds
any extra Angular files.
Together, these two classes are used to implement a new NgtscProgram which
adapts the legacy api.Program interface used by the View Engine compiler
onto operations on the new types. The new NgtscProgram implementation is
significantly smaller and easier to reason about.
The new NgCompilerHost replaces the previous GeneratedShimsHostWrapper which
lived in the 'shims' package.
A new 'resource' package is added to support the HostResourceLoader which
previously lived in the outer compiler package.
As a result of the refactoring, the dependencies of the outer
@angular/compiler-cli package on ngtsc internal packages are significantly
trimmed.
This refactoring was driven by the desire to build a plugin interface to the
compiler so that tsc_wrapped (another consumer of the TS compiler APIs) can
perform Angular transpilation on user request.
PR Close#34887
