Updates to rules_nodejs 2.2.0. This is the first major release in 7 months and includes a number of features as well
as breaking changes.
Release notes: https://github.com/bazelbuild/rules_nodejs/releases/tag/2.0.0
Features of note for angular/angular:
* stdout/stderr/exit code capture; this could be potentially be useful
* TypeScript (ts_project); a simpler tsc rule that ts_library that can be used in the repo where ts_library is too
heavy weight
Breaking changes of note for angular/angular:
* loading custom rules from npm packages: `ts_library` is no longer loaded from `@npm_bazel_typescript//:index.bzl`
(which no longer exists) but is now loaded from `@npm//@bazel/typescript:index.bzl`
* with the loading changes above, `load("@npm//:install_bazel_dependencies.bzl", "install_bazel_dependencies")` is
no longer needed in the WORKSPACE which also means that yarn_install does not need to run unless building/testing
a target that depends on @npm. In angular/angular this is a minor improvement as almost everything depends on @npm.
* @angular/bazel package is also updated in this PR to support the new load location; Angular + Bazel users that
require it for ng_package (ng_module is no longer needed in OSS with Angular 10) will need to load from
`@npm//@angular/bazel:index.bzl`. I investigated if it was possible to maintain backward compatability for the old
load location `@npm_angular_bazel` but it is not since the package itself needs to be updated to load from
`@npm//@bazel/typescript:index.bzl` instead of `@npm_bazel_typescript//:index.bzl` as it depends on ts_library
internals for ng_module.
* runfiles.resolve will now throw instead of returning undefined to match behavior of node require
Other changes in angular/angular:
* integration/bazel has been updated to use both ng_module and ts_libary with use_angular_plugin=true.
The latter is the recommended way for rules_nodejs users to compile Angular 10 with Ivy. Bazel + Angular ViewEngine is
supported with @angular/bazel <= 9.0.5 and Angular <= 8. There is still Angular ViewEngine example on rules_nodejs
https://github.com/bazelbuild/rules_nodejs/tree/stable/examples/angular_view_engine on these older versions but users
that want to update to Angular 10 and are on Bazel must switch to Ivy and at that point ts_library with
use_angular_plugin=true is more performant that ng_module. Angular example in rules_nodejs is configured this way
as well: https://github.com/bazelbuild/rules_nodejs/tree/stable/examples/angular. As an aside, we also have an
example of building Angular 10 with architect() rule directly instead of using ts_library with angular plugin:
https://github.com/bazelbuild/rules_nodejs/tree/stable/examples/angular_bazel_architect.
NB: ng_module is still required for angular/angular repository as it still builds ViewEngine & @angular/bazel
also provides the ng_package rule. ng_module can be removed in the future if ViewEngine is no longer needed in
angular repo.
* JSModuleInfo provider added to ng_module. this is for forward compat for future rules_nodejs versions.
PR Close#39182
Temporarily disable the //packages/compiler-cli/integrationtest:integrationtest
target while continuing to investigate its unknown failures
PR Close#39168
The right needs to be wrapped in parens or we cannot accurately match its
span to just the RHS. For example, the span in `e = $event /*0,10*/` is ambiguous.
It could refer to either the whole binary expression or just the RHS.
We should instead generate `e = ($event /*0,10*/)` so we know the span 0,10 matches RHS.
This is specifically needed for the TemplateTypeChecker/Language Service
when mapping template positions to items in the TCB.
PR Close#39143
This commit introduces a new API for the `TemplateTypeChecker` which allows
for autocompletion in a global expression context (for example, in a new
interpolation expression such as `{{|}}`). This API returns instances of the
type `GlobalCompletion`, which can represent either a completion result from
the template's component context or a declaration such as a local reference
or template variable. The Language Service will use this API to implement
autocompletion within templates.
PR Close#39048
Previously the value passed to `AstFactory.attachComments()` could be
`undefined` which is counterintuitive, since why attach something that
doesn't exist? Now it expects there to be a defined array. Further it no
longer returns a statement. Both these aspects of the interface were designed
to make the usage simpler but has the result of complicating the implemenation.
The `ExpressionTranslatorVisitor` now has a helper function (`attachComments()`)
to handle `leadingComments` being undefined and also returning the statement.
This keeps the usage in the translator simple, while ensuring that the `AstFactory`
API is not influenced by how it is used.
PR Close#39076
This is needed so that the Language Service can provide the module name
in the quick info for a directive/component.
To accomplish this, the compiler's `LocalModuleScope` is provided to the
`TemplateTypeCheckerImpl`. This will also allow the `TemplateTypeChecker` to
provide more completions in the future, giving it a way to determine all the
directives/pipes/etc. available to a template.
PR Close#39099
The compiler maintains an internal dependency graph of all resource
dependencies for application source files. This information can be useful
for tools that integrate the compiler and need to support file watching.
This change adds a `getResourceDependencies` method to the
`NgCompiler` class that allows compiler integrations to access resource
dependencies of files within the compilation.
PR Close#38048
Updates to rules_nodejs 2.2.0. This is the first major release in 7 months and includes a number of features as well
as breaking changes.
Release notes: https://github.com/bazelbuild/rules_nodejs/releases/tag/2.0.0
Features of note for angular/angular:
* stdout/stderr/exit code capture; this could be potentially be useful
* TypeScript (ts_project); a simpler tsc rule that ts_library that can be used in the repo where ts_library is too
heavy weight
Breaking changes of note for angular/angular:
* loading custom rules from npm packages: `ts_library` is no longer loaded from `@npm_bazel_typescript//:index.bzl`
(which no longer exists) but is now loaded from `@npm//@bazel/typescript:index.bzl`
* with the loading changes above, `load("@npm//:install_bazel_dependencies.bzl", "install_bazel_dependencies")` is
no longer needed in the WORKSPACE which also means that yarn_install does not need to run unless building/testing
a target that depends on @npm. In angular/angular this is a minor improvement as almost everything depends on @npm.
* @angular/bazel package is also updated in this PR to support the new load location; Angular + Bazel users that
require it for ng_package (ng_module is no longer needed in OSS with Angular 10) will need to load from
`@npm//@angular/bazel:index.bzl`. I investigated if it was possible to maintain backward compatability for the old
load location `@npm_angular_bazel` but it is not since the package itself needs to be updated to load from
`@npm//@bazel/typescript:index.bzl` instead of `@npm_bazel_typescript//:index.bzl` as it depends on ts_library
internals for ng_module.
* runfiles.resolve will now throw instead of returning undefined to match behavior of node require
Other changes in angular/angular:
* integration/bazel has been updated to use both ng_module and ts_libary with use_angular_plugin=true.
The latter is the recommended way for rules_nodejs users to compile Angular 10 with Ivy. Bazel + Angular ViewEngine is
supported with @angular/bazel <= 9.0.5 and Angular <= 8. There is still Angular ViewEngine example on rules_nodejs
https://github.com/bazelbuild/rules_nodejs/tree/stable/examples/angular_view_engine on these older versions but users
that want to update to Angular 10 and are on Bazel must switch to Ivy and at that point ts_library with
use_angular_plugin=true is more performant that ng_module. Angular example in rules_nodejs is configured this way
as well: https://github.com/bazelbuild/rules_nodejs/tree/stable/examples/angular. As an aside, we also have an
example of building Angular 10 with architect() rule directly instead of using ts_library with angular plugin:
https://github.com/bazelbuild/rules_nodejs/tree/stable/examples/angular_bazel_architect.
NB: ng_module is still required for angular/angular repository as it still builds ViewEngine & @angular/bazel
also provides the ng_package rule. ng_module can be removed in the future if ViewEngine is no longer needed in
angular repo.
* JSModuleInfo provider added to ng_module. this is for forward compat for future rules_nodejs versions.
@josephperrott, this touches `packages/bazel/src/external.bzl` which will make the sync to g3 non-trivial.
PR Close#37727
This commit adds the `AstHost` interface, along with implementations for
both Babel and TS.
It also implements the Babel vesion of the `AstFactory` interface, along
with a linker specific implementation of the `ImportGenerator` interface.
These classes will be used by the new "ng-linker" to transform prelinked
library code using a Babel plugin.
PR Close#38866
The `AstFactory.createFunctionDeclaration()` was allowing `null` to be
passed as the function `name` value. This is not actually possible, since
function declarations must always have a name.
PR Close#38866
The tests were assuming that newlines were `\n` characters but this is not
the case on Windows. This was fixed in #38925, but a better solution is to
configure the TS printer to always use `\n` characters for newlines.
PR Close#38866
These free standing functions rely upon the "current" `FileSystem`,
but it is safer to explicitly pass the `FileSystem` into functions or
classes that need it.
PR Close#39006
To verify the correctness of the linker output, we leverage the existing
compliance tests. The plan is to test the linker by running all compliance
tests using a full round trip of pre-linking and subsequently post-linking,
where the generated code should be identical to a full AOT compile.
This commit adds an additional Bazel target that runs the compliance
tests in partial mode. Follow-up work is required to implement the logic
for running the linker round trip.
PR Close#38938
This is a precursor to introducing the Angular linker. As an initial
step, a compiler option to configure the compilation mode is introduced.
This option is initially internal until the linker is considered ready.
PR Close#38938
* Add `templateNode` to `ElementSymbol` and `TemplateSymbol` so callers
can use the information about the attributes on the
`TmplAstElement`/`TmplAstTemplate` for directive matching
* Remove helper function `getSymbolOfVariableDeclaration` and favor
more specific handling for scenarios. The generic function did not
easily handle different scenarios for all types of variable declarations
in the TCB
PR Close#39047
This commit adds an API to `NgCompiler`, a method called
`getComponentsWithTemplateFile`. Given a filesystem path to an external
template file, it retrieves a `Set` (actually a `ReadonlySet`) of component
declarations which are using this template. In most cases, this will only be
a single component.
This information is easily determined by the compiler during analysis, but
is hard for a lot of Angular tooling (e.g. the language service) to infer
independently. Therefore, it makes sense to expose this as a compiler API.
PR Close#39002
With the introduction of incremental type checking in #36211, an
intermediate `ts.Program` for type checking is only created if there are
any templates to check. This rendered some tests ineffective at avoiding
regressions, as the intermediate `ts.Program` was required for the tests
to fail if the scenario under test would not be accounted for. This
commit adds a single component to these tests, to ensure the
intermediate `ts.Program` is in fact created.
PR Close#39011
Prior to this fix, incremental rebuilds could fail to type check due to
missing ambient types from auto-discovered declaration files in @types
directories, or type roots in general. This was caused by the
intermediary `ts.Program` that is created for template type checking,
for which a `ts.CompilerHost` was used which did not implement the
optional `directoryExists` methods. As a result, auto-discovery of types
would not be working correctly, and this would retain into the
`ts.Program` that would be created for an incremental rebuild.
This commit fixes the issue by forcing the custom `ts.CompilerHost` used
for type checking to properly delegate into the original
`ts.CompilerHost`, even for optional methods. This is accomplished using
a base class `DelegatingCompilerHost` which is typed in such a way that
newly introduced `ts.CompilerHost` methods must be accounted for.
Fixes#38979
PR Close#39011
We weren't resolving a path correctly which resulted in an error on Windows.
For reference, here's the error. Note the extra slash before `C:`:
```
Error: ENOENT: no such file or directory, scandir '/C:/bazel_output_root/yxvwd24o/external/npm/node_modules/typescript'
at Object.readdirSync (fs.js:854:3)
```
PR Close#39005
This commit updates the symbols in the TemplateTypeCheck API and methods
for retrieving them:
* Include `isComponent` and `selector` for directives so callers can determine which
attributes on an element map to the matched directives.
* Add a new `TextAttributeSymbol` and return this when requesting a symbol for a `TextAttribute`.
* When requesting a symbol for `PropertyWrite` and `MethodCall`, use the
`nameSpan` to retrieve symbols.
* Add fix to retrieve generic directives attached to elements/templates.
PR Close#38844
Prior to this change, each invocation of `loadStandardTestFiles` would
load the necessary files from disk. This function is typically called
at the top-level of a test module in order to share the result across
tests. The `//packages/compiler-cli/test/ngtsc` target has 8 modules
where this call occurs, each loading their own copy of
`node_modules/typescript` which is ~60MB in size, so the memory overhead
used to be significant. This commit loads the individual packages into
a standalone `Folder` and mounts this folder into the filesystem of
standard test files, such that all file contents are no longer
duplicated in memory.
PR Close#38909
Some compiler tests take a long time to run, even using multiple
executors. A profiling session revealed that most time is spent in
parsing source files, especially the default libraries are expensive to
parse.
The default library files are constant across all tests, so this commit
introduces a shared cache of parsed source files of the default
libraries. This achieves a significant improvement for several targets
on my machine:
//packages/compiler-cli/test/compliance: from 23s to 5s.
//packages/compiler-cli/test/ngtsc: from 115s to 11s.
Note that the number of shards for the compliance tests has been halved,
as the extra shards no longer provide any speedup.
PR Close#38909
In Ivy, template type-checking has 3 modes: basic, full, and strict. The
primary difference between basic and full modes is that basic mode only
checks the top-level template, whereas full mode descends into nested
templates (embedded views like ngIfs and ngFors). Ivy applies this approach
to all of its template type-checking, including the DOM schema checks which
validate whether an element is a valid component/directive or not.
View Engine has both the basic and the full mode, with the same distinction.
However in View Engine, DOM schema checks happen for the full template even
in the basic mode.
Ivy's behavior here is technically a "fix" as it does not make sense for
some checks to apply to the full template and others only to the top-level
view. However, since g3 relies exclusively on the basic mode of checking and
developers there are used to DOM checks applying throughout their template,
this commit re-enables the nested schema checks even in basic mode only in
g3. This is done by enabling the checks only when Closure Compiler
annotations are requested.
Outside of g3, it's recommended that applications use at least the full mode
of checking (controlled by the `fullTemplateTypeCheck` flag), and ideally
the strict mode (`strictTemplates`).
PR Close#38943
This commit refactors the `ExpressionTranslatorVisitor` so that it
is not tied directly to the TypeScript AST. Instead it uses generic
`TExpression` and `TStatement` types that are then converted
to concrete types by the `TypeScriptAstFactory`.
This paves the way for a `BabelAstFactory` that can be used to
generate Babel AST nodes instead of TypeScript, which will be
part of the new linker tool.
PR Close#38775
Previously each identifier was being imported individually, which made for a
very long import statement, but also obscurred, in the code, which identifiers
came from the compiler.
PR Close#38775
This file contains a number of classes making it long and hard to work with.
This commit splits the `ImportManager`, `Context` and `TypeTranslatorVisitor`
classes, along with associated functions and types into their own files.
PR Close#38775
When the target of the compiler is ES2015 or newer then we should
be generating `let` and `const` variable declarations rather than `var`.
PR Close#38775
The cast to `ts.Identifier` was a hack that "just happened to work".
The new approach is more robust and doesn't have to undermine
the type checker.
PR Close#38775
This commit re-enables some tests that were temporarily disabled on Windows,
as they failed on native Windows CI. The Windows filesystem emulation has
been corrected in an earlier commit, such that the original failure would
now also occur during emulation on Linux CI.
PR Close#37782
In native windows, the drive letter is a capital letter, while our Windows
filesystem emulation would use lowercase drive letters. This difference may
introduce tests to behave differently in native Windows versus emulated
Windows, potentially causing unexpected CI failures on Windows CI after a PR
has been merged.
Resolves FW-2267
PR Close#37782
The logic for computing identifiers, specifically for bound attributes
can be simplified by using the value span of the binding rather than the
source span.
PR Close#38899
In #38666 we changed how ngcc deals with type expressions, where it
would now always emit the original type expression into the generated
code as a "local" type value reference instead of synthesizing new
imports using an "imported" type value reference. This was done as a fix
to properly deal with renamed symbols, however it turns out that the
compiler has special handling for certain imported symbols, e.g.
`ChangeDetectorRef` from `@angular/core`. The "local" type value
reference prevented this special logic from being hit, resulting in
incorrect compilation of pipe factories.
This commit fixes the issue by manually inspecting the import of the
type expression, in order to return an "imported" type value reference.
By manually inspecting the import we continue to handle renamed symbols.
Fixes#38883
PR Close#38892
Common AST formats such as TS and Babel do not use a separate
node for comments, but instead attach comments to other AST nodes.
Previously this was worked around in TS by creating a `NotEmittedStatement`
AST node to attach the comment to. But Babel does not have this facility,
so it will not be a viable approach for the linker.
This commit refactors the output AST, to remove the `CommentStmt` and
`JSDocCommentStmt` nodes. Instead statements have a collection of
`leadingComments` that are rendered/attached to the final AST nodes
when being translated or printed.
PR Close#38811
This change prevents comments from a resolved node from appearing at
each location the resolved expression is used and also prevents callers
of `Scope#resolve` from accidentally modifying / adding comments to the
declaration site.
PR Close#38857
In the integration test suite of ngcc, we load a set of files from
`node_modules` into memory. This includes the `typescript` package and
`@angular` scoped packages, which account for a large number of large
files that needs to be loaded from disk. This commit moves this work
to the top-level, such that it doesn't have to be repeated in all tests.
PR Close#38840
Recent optimizations to ngcc have significantly reduced the total time
it takes to process `node_modules`, to such extend that sharding across
multiple processes has become less effective. Previously, running
ngcc asynchronously would allow for up to 8 workers to be allocated,
however these workers have to repeat work that could otherwise be shared.
Because ngcc is now able to reuse more shared computations, the overhead
of multiple workers is increased and therefore becomes less effective.
As an additional benefit, having fewer workers requires less memory and
less startup time.
To give an idea, using the following test setup:
```bash
npx @angular/cli new perf-test
cd perf-test
yarn ng add @angular/material
./node_modules/.bin/ngcc --properties es2015 module main \
--first-only --create-ivy-entry-points
```
We observe the following figures on CI:
| | 10.1.1 | PR #38840 |
| ----------------- | --------- | --------- |
| Sync | 85s | 25s |
| Async (8 workers) | 22s | 16s |
| Async (4 workers) | - | 11s |
In addition to changing the default number of workers, ngcc will now
use the environment variable `NGCC_MAX_WORKERS` that may be configured
to either reduce or increase the number of workers.
PR Close#38840
ngcc creates typically two `ts.Program` instances for each entry-point,
one for processing sources and another one for processing the typings.
The creation of these programs is somewhat expensive, as it concerns
module resolution and parsing of source files.
This commit implements several layers of caching to optimize the
creation of programs:
1. A shared module resolution cache across all entry-points within a
single invocation of ngcc. Both the sources and typings program
benefit from this cache.
2. Sharing the parsed `ts.SourceFile` for a single entry-point between
the sources and typings program.
3. Sharing parsed `ts.SourceFile`s of TypeScript's default libraries
across all entry-points within a single invocation. Some of these
default library typings are large and therefore expensive to parse,
so sharing the parsed source files across all entry-points offers
a significant performance improvement.
Using a bare CLI app created using `ng new` + `ng add @angular/material`,
the above changes offer a 3-4x improvement in ngcc's processing time
when running synchronously and ~2x improvement for asynchronous runs.
PR Close#38840
When type-checking a component, the declaring NgModule scope is used
to create a directive matcher that contains flattened directive metadata,
i.e. the metadata of a directive and its base classes. This computation
is done for all components, whereas the type-check scope is constant per
NgModule. Additionally, the flattening of metadata is constant per
directive instance so doesn't necessarily have to be recomputed for
each component.
This commit introduces a `TypeCheckScopes` class that is responsible
for flattening directives and computing the scope per NgModule. It
caches the computed results as appropriate to avoid repeated computation.
PR Close#38539
For the compilation of a component, the compiler has to prepare some
information about the directives and pipes that are used in the template.
This information includes an expression for directives/pipes, for usage
within the compilation output. For large NgModule compilation scopes
this has shown to introduce a performance hotspot, as the generation of
expressions is quite expensive. This commit reduces the performance
overhead by only generating expressions for the directives/pipes that
are actually used within the template, significantly cutting down on
the compiler's resolve phase.
PR Close#38539
Adds `TemplateTypeChecker` operation to retrieve the `Symbol` of a
`TmplAstVariable` or `TmplAstReference` in a template.
Sometimes we need to traverse an intermediate variable declaration to arrive at
the correct `ts.Symbol`. For example, loop variables are declared using an intermediate:
```
<div *ngFor="let user of users">
{{user.name}}
</div>
```
Getting the symbol of user here (from the expression) is tricky, because the TCB looks like:
```
var _t0 = ...; // type of NgForOf
var _t1: any; // context of embedded view for NgForOf structural directive
if (NgForOf.ngTemplateContextGuard(_t0, _t1)) {
// _t1 is now NgForOfContext<...>
var _t2 = _t1.$implicit; // let user = '$implicit'
_t2.name; // user.name expression
}
```
Just getting the `ts.Expression` for the `AST` node `PropRead(ImplicitReceiver, 'user')`
via the sourcemaps will yield the `_t2` expression. This function recognizes that `_t2`
is a variable declared locally in the TCB, and actually fetch the `ts.Symbol` of its initializer.
These special handlings show the versatility of the `Symbol`
interface defined in the API. With this, when we encounter a template variable,
we can provide the declaration node, as well as specific information
about the variable instance, such as the `ts.Type` and `ts.Symbol`.
PR Close#38618
Adds support to the `TemplateTypeChecker` to get a `Symbol` of an AST
expression in a component template.
Not all expressions will have `ts.Symbol`s (e.g. there is no `ts.Symbol`
associated with the expression `a + b`, but there are for both the a and b
nodes individually).
PR Close#38618
Adds support to the `TemplateTypeChecker` for retrieving a `Symbol` for
`TmplAstTemplate` and `TmplAstElement` nodes in a component template.
PR Close#38618
Specifically, this commit adds support for retrieving a `Symbol` from a
`TmplAstBoundEvent` or `TmplAstBoundAttribute`. Other template nodes
will be supported in following commits.
PR Close#38618
The statements generated in the TCB are optimized for performance and producing diagnostics.
These optimizations can result in generating a TCB that does not have all the information
needed by the `TemplateTypeChecker` for retrieving `Symbol`s. For example, as an optimization,
the TCB will not generate variable declaration statements for directives that have no
references, inputs, or outputs. However, the `TemplateTypeChecker` always needs these
statements to be present in order to provide `ts.Symbol`s and `ts.Type`s for the directives.
This commit adds logic to the TCB generation to ensure the required
information is available in a form that the `TemplateTypeChecker` can
consume. It also adds an option to the `NgCompiler` that makes this
generation configurable.
PR Close#38618
This commit defines the interfaces which outline the information the
`TemplateTypeChecker` can return when requesting a Symbol for an item in the
`TemplateAst`.
Rather than providing the `ts.Symbol`, `ts.Type`, etc.
information in several separate functions, the `TemplateTypeChecker` can
instead provide all the useful information it knows about a particular
node in the `TemplateAst` and allow the callers to determine what to do
with it.
PR Close#38618
Greg Magolan
Joey Perrott
Andrew Scott
Alex Rickabaugh
Pete Bacon Darwin
Charles Lyding
JoostK
Kristiyan Kostadinov