In #34288, ngtsc was refactored to separate the result of the analysis
and resolve phase for more granular incremental rebuilds. In this model,
any errors in one phase transition the trait into an error state, which
prevents it from being ran through subsequent phases. The ngcc compiler
on the other hand did not adopt this strict error model, which would
cause incomplete metadata—due to errors in earlier phases—to be offered
for compilation that could result in a hard crash.
This commit updates ngcc to take advantage of ngtsc's `TraitCompiler`,
that internally manages all Ivy classes that are part of the
compilation. This effectively replaces ngcc's own `AnalyzedFile` and
`AnalyzedClass` types, together with all of the logic to drive the
`DecoratorHandler`s. All of this is now handled in the `TraitCompiler`,
benefiting from its explicit state transitions of `Trait`s so that the
ngcc crash is a thing of the past.
Fixes#34500
Resolves FW-1788
PR Close#34889
This syntax is invalid in these source files and does result in
compilation errors as the constructor parameters could not be resolved.
This hasn't been an issue until now as those errors were ignored in the
tests, but future work to introduce the Trait system of ngtsc into
ngcc will cause these errors to prevent compilation, resulting in broken
tests.
PR Close#34889
Previously, while trying to build an `NgccReflectionHost`'s
`privateDtsDeclarationMap`, `computePrivateDtsDeclarationMap()` would
try to collect exported declarations from all source files of the
program (i.e. without checking whether they were within the target
package, as happens for declarations in `.d.ts` files).
Most of the time, that would not be a problem, because external packages
would be represented as `.d.ts` files in the program. But when an
external package had no typings, the JS files would be used instead. As
a result, the `ReflectionHost` would try to (unnecessarilly) parse the
file in order to extract exported declarations, which in turn would be
harmless in most cases.
There are certain cases, though, where the `ReflectionHost` would throw
an error, because it cannot parse the external package's JS file. This
could happen, for example, in `UmdReflectionHost`, which expects the
file to contain exactly one statement. See #34544 for more details on a
real-world failure.
This commit fixes the issue by ensuring that
`computePrivateDtsDeclarationMap()` will only collect exported
declarations from files within the target package.
Jira issue: [FW-1794](https://angular-team.atlassian.net/browse/FW-1794)
Fixes#34544
PR Close#34811
Consider a library that uses a shared constant for host bindings. e.g.
```ts
export const BASE_BINDINGS= {
'[class.mat-themed]': '_isThemed',
}
----
@Directive({
host: {...BASE_BINDINGS, '(click)': '...'}
})
export class Dir1 {}
@Directive({
host: {...BASE_BINDINGS, '(click)': '...'}
})
export class Dir2 {}
```
Previously when these components were shipped as part of the
library to NPM, consumers were able to consume `Dir1` and `Dir2`.
No errors showed up.
Now with Ivy, when ngcc tries to process the library, an error
will be thrown. The error is stating that the host bindings should
be an object (which they obviously are). This happens because
TypeScript transforms the object spread to individual
`Object.assign` calls (for compatibility).
The partial evaluator used by the `@Directive` annotation handler
is unable to process this expression because there is no
integrated support for `Object.assign`. In View Engine, this was
not a problem because the `metadata.json` files from the library
were used to compute the host bindings.
Fixes#34659
PR Close#34661
Ngcc adds properties to the `package.json` files of the entry-points it
processes to mark them as processed for a format and point to the
created Ivy entry-points (in case of `--create-ivy-entry-points`). When
running ngcc in parallel mode (which is the default for the standalone
ngcc command), multiple formats can be processed simultaneously for the
same entry-point and the order of completion is not deterministic.
Previously, ngcc would append new properties at the end of the target
object in `package.json` as soon as the format processing was completed.
As a result, the order of properties in the resulting `package.json`
(when processing multiple formats for an entry-point in parallel) was
not deterministic. For tools that use file hashes for caching purposes
(such as Bazel), this lead to a high probability of cache misses.
This commit fixes the problem by ensuring that the position of
properties added to `package.json` files is deterministic and
independent of the order in which each format is processed.
Jira issue: [FW-1801](https://angular-team.atlassian.net/browse/FW-1801)
Fixes#34635
PR Close#34870
Previously, it was possible for multiple instance of ngcc to be running
at the same time, but this is not supported and can cause confusing and
flakey errors at build time.
Now, only one instance of ngcc can run at a time. If a second instance
tries to execute it fails with an appropriate error message.
See https://github.com/angular/angular/issues/32431#issuecomment-571825781
PR Close#34722
Since I was learning the codebase and had a hard time understanding what was going on I've done a
bunch of changes in one commit that under normal circumstances should have been split into several
commits. Because this code is likely going to be overwritten with Misko's changes I'm not going to
spend the time with trying to split this up.
Overall I've done the following:
- I processed review feedback from #34307
- I did a bunch of renaming to make the code easier to understand
- I refactored some internal functions that were either inefficient or hard to read
- I also updated lots of type signatures to correct them and to remove many casts in the code
PR Close#34307
The major one that affects the angular repo is the removal of the bootstrap attribute in nodejs_binary, nodejs_test and jasmine_node_test in favor of using templated_args --node_options=--require=/path/to/script. The side-effect of this is that the bootstrap script does not get the require.resolve patches with explicitly loading the targets _loader.js file.
PR Close#34736
When searching the typings program for a package for imports a
distinction is drawn between missing entry-points and deep imports.
Previously in the `DtsDependencyHost` these deep imports may be
marked as missing if there was no typings file at the deep import path.
Instead there may be a javascript file instead. In practice this means
the import is "deep" and not "missing".
Now the `DtsDependencyHost` will also consider `.js` files when checking
for deep-imports, and it will also look inside `@types/...` for a suitable
deep-imported typings file.
Fixes#34720
PR Close#34695
Previously, `CommonJsDependencyHost.collectDependencies()` would only
find dependencies via imports of the form `var foo = require('...');` or
`var foo = require('...'), bar = require('...');` However, CommonJS
files can have imports in many different forms. By failing to recognize
other forms of imports, the associated dependencies were missed, which
in turn resulted in entry-points being compiled out-of-order and failing
due to that.
While we cannot easily capture all different types of imports, this
commit enhances `CommonJsDependencyHost` to recognize the following
common forms of imports:
- Imports in property assignments. E.g.:
`exports.foo = require('...');` or
`module.exports = {foo: require('...')};`
- Imports for side-effects only. E.g.:
`require('...');`
- Star re-exports (with both emitted and imported heleprs). E.g.:
`__export(require('...'));` or
`tslib_1.__exportStar(require('...'), exports);`
PR Close#34528
Currently the decorator handlers are run against all `SourceFile`s in the compilation, but we shouldn't be doing it against declaration files. This initially came up as a CI issue in #33264 where it was worked around only for the `DirectiveDecoratorHandler`. These changes move the logic into the `TraitCompiler` and `DecorationAnalyzer` so that it applies to all of the handlers.
PR Close#34557
The major one that affects the angular repo is the removal of the bootstrap attribute in nodejs_binary, nodejs_test and jasmine_node_test in favor of using templated_args --node_options=--require=/path/to/script. The side-effect of this is that the bootstrap script does not get the require.resolve patches with explicitly loading the targets _loader.js file.
PR Close#34589
Previously, the `CommonJsReflectionHost` and `UmdReflectionHost` would
only recognize re-exports of the form `__export(...)`. This is what
re-exports look like, when the TypeScript helpers are emitted inline
(i.e. when compiling with the default [TypeScript compiler options][1]
that include `noEmitHelpers: false` and `importHelpers: false`).
However, when compiling with `importHelpers: true` and [tslib][2] (which
is the recommended way for optimized bundles), the re-exports will look
like: `tslib_1.__exportStar(..., exports)`
These types of re-exports were previously not recognized by the
CommonJS/UMD `ReflectionHost`s and thus ignored.
This commit fixes this by ensuring both re-export formats are
recognized.
[1]: https://www.typescriptlang.org/docs/handbook/compiler-options.html
[2]: https://www.npmjs.com/package/tslib
PR Close#34527
If a class was defined as a class expression
in a variable declaration, the definitions
were being inserted before the statment's
final semi-colon.
Now the insertion point will be after the
full statement.
Fixes#34648
PR Close#34677
In some cases, where a module imports a dependency
but does not actually use it, UMD bundlers may remove
the dependency parameter from the UMD factory function
definition.
For example:
```
import * as x from 'x';
import * as z from 'z';
export const y = x;
```
may result in a UMD bundle including:
```
(function (global, factory) {
typeof exports === 'object' && typeof module !== 'undefined' ?
factory(exports, require('x'), require('z')) :
typeof define === 'function' && define.amd ?
define(['exports', 'x', 'z'], factory) :
(global = global || self, factory(global.myBundle = {}, global.x));
}(this, (function (exports, x) { 'use strict';
...
})));
```
Note that while the `z` dependency is provide in the call,
the factory itself only accepts `exports` and `x` as parameters.
Previously ngcc appended new dependencies to the end of the factory
function, but this breaks in the above scenario. Now the new
dependencies are prefixed at the front of parameters/arguments
already in place.
Fixes#34653
PR Close#34660
Previously, in cases were values were expensive to compute and would be
used multiple times, a combination of a regular `Map` and a helper
function (`getOrDefault()`) was used to ensure values were only computed
once.
This commit uses a special `Map`-like structure to compute and memoize
such expensive values without the need to a helper function.
PR Close#34512
While different, CommonJS and UMD have a lot in common regarding the
their exports are constructed. Therefore, there was some code
duplication between `CommonJsReflectionHost` and `UmdReflectionHost`.
This commit extracts some of the common bits into a separate file as
helpers to allow reusing the code in both `ReflectionHost`s.
PR Close#34512
Previously, `UmdReflectionHost` would only recognize re-exports of the
form `__export(someIdentifier)` and not `__export(require('...'))`.
However, it is possible in some UMD variations to have the latter format
as well. See discussion in https://github.com/angular/angular/pull/34254/files#r359515373
This commit adds support for re-export of the form
`__export(require('...'))` in UMD.
PR Close#34512
This fix was part of a broader `ngtsc`/`ngcc` fix in 02bab8cf9 (see
there for details). In 02bab8cf9, the fix was only applied to
`CommonJsReflectionHost`, but it is equally applicable to
`UmdReflectionHost`. Later in #34254, the fix was partially ported to
`UmdReflectionHost` by fixing the `extractUmdReexports()` method.
This commit fully fixes `ngcc`'s handling of inline exports for code in
UMD format.
PR Close#34512
Previously, if `UmdRenderingFormatter#addImports()` was called with an
empty list of imports to add (i.e. no new imports were needed), it would
add trailing commas in several locations (arrays, function arguments,
function parameters), thus making the code imcompatible with legacy
browsers such as IE11.
This commit fixes it by ensuring that no trailing commas are added if
`addImports()` is called with an empty list of imports.
This is a follow-up to #34353.
Fixes#34525
PR Close#34545
ngcc computes a dependency graph of entry-points to ensure that
entry-points are processed in the correct order. Previously only the imports
in source files were analysed to determine the dependencies for each
entry-point.
This is not sufficient when an entry-point has a "type-only" dependency
- for example only importing an interface from another entry-point.
In this case the "type-only" import does not appear in the
source code. It only appears in the typings files. This can cause a
dependency to be missed on the entry-point.
This commit fixes this by additionally processing the imports in the
typings program, as well as the source program.
Note that these missing dependencies could cause unexpected flakes when
running ngcc in async mode on multiple processes due to the way that
ngcc caches files when they are first read from disk.
Fixes#34411
// FW-1781
PR Close#34494
The `DependencyHost` implementations were duplicating the "postfix" strings
which are used to find matching paths when resolving module specifiers.
Now the hosts reuse the postfixes given to the `ModuleResolver` that is
passed to the host.
PR Close#34494
Rather than return a new object of dependency info from calls to
`collectDependencies()` we now pass in an object that will be updated
with the dependency info. This is in preparation of a change where
we will collect dependency information from more than one
`DependencyHost`.
Also to better fit with this approach the name is changed from
`findDependencies()` to `collectDependencies()`.
PR Close#34494
Angular View Engine uses global knowledge to compile the following code:
```typescript
export class Base {
constructor(private vcr: ViewContainerRef) {}
}
@Directive({...})
export class Dir extends Base {
// constructor inherited from base
}
```
Here, `Dir` extends `Base` and inherits its constructor. To create a `Dir`
the arguments to this inherited constructor must be obtained via dependency
injection. View Engine is able to generate a correct factory for `Dir` to do
this because via metadata it knows the arguments of `Base`'s constructor,
even if `Base` is declared in a different library.
In Ivy, DI is entirely a runtime concept. Currently `Dir` is compiled with
an ngDirectiveDef field that delegates its factory to `getInheritedFactory`.
This looks for some kind of factory function on `Base`, which comes up
empty. This case looks identical to an inheritance chain with no
constructors, which works today in Ivy.
Both of these cases will now become an error in this commit. If a decorated
class inherits from an undecorated base class, a diagnostic is produced
informing the user of the need to either explicitly declare a constructor or
to decorate the base class.
PR Close#34460
Now that the source to typings matching is able to handle
aliasing of exports, there is no need to handle aliases in private
declarations analysis.
These were originally added to cope when the typings files had
to use the name that the original source files used when exporting.
PR Close#34254
Previously the identifiers used in the typings files were the same as
those used in the source files.
When the typings files and the source files do not match exactly, e.g.
when one of them is flattened, while the other is a deep tree, it is
possible for identifiers to be renamed.
This commit ensures that the correct identifier is used in typings files
when the typings file does not export the same name as the source file.
Fixes https://github.com/angular/ngcc-validation/pull/608
PR Close#34254
The naïve matching algorithm we previously used to match declarations in
source files to declarations in typings files was based only on the name
of the thing being declared. This did not handle cases where the declared
item had been exported via an alias - a common scenario when one of the two
file sets (source or typings) has been flattened, while the other has not.
The new algorithm tries to overcome this by creating two maps of export
name to declaration (i.e. `Map<string, ts.Declaration>`).
One for the source files and one for the typings files.
It then joins these two together by matching export names, resulting in a
new map that maps source declarations to typings declarations directly
(i.e. `Map<ts.Declaration, ts.Declaration>`).
This new map can handle the declaration names being different between the
source and typings as long as they are ultimately both exported with the
same alias name.
Further more, there is one map for "public exports", i.e. exported via the
root of the source tree (the entry-point), and another map for "private
exports", which are exported from individual files in the source tree but
not necessarily from the root. This second map can be used to "guess"
the mapping between exports in a deep (non-flat) file tree, which can be
used by ngcc to add required private exports to the entry-point.
Fixes#33593
PR Close#34254
In TS we can re-export imports using statements of the form:
```
export * from 'some-import';
```
This is downleveled in UMD to:
```
function factory(exports, someImport) {
function __export(m) {
for (var p in m) if (!exports.hasOwnProperty(p)) exports[p] = m[p];
}
__export(someImport);
}
```
This commit adds support for this.
PR Close#34254
In TS we can re-export imports using statements of the form:
```
export * from 'some-import';
```
This can be downleveled in CommonJS to either:
```
__export(require('some-import'));
```
or
```
var someImport = require('some-import');
__export(someImport);
```
Previously we only supported the first downleveled version.
This commit adds support for the second version.
PR Close#34254
Previously individual properties of the src bundle program were
passed to the reflection host constructors. But going forward,
more properties will be required. To prevent the signature getting
continually larger and more unwieldy, this change just passes the
whole src bundle to the constructor, allowing it to extract what it
needs.
PR Close#34254
Previously, the compiler performed an incremental build by analyzing and
resolving all classes in the program (even unchanged ones) and then using
the dependency graph information to determine which .js files were stale and
needed to be re-emitted. This algorithm produced "correct" rebuilds, but the
cost of re-analyzing the entire program turned out to be higher than
anticipated, especially for component-heavy compilations.
To achieve performant rebuilds, it is necessary to reuse previous analysis
results if possible. Doing this safely requires knowing when prior work is
viable and when it is stale and needs to be re-done.
The new algorithm implemented by this commit is such:
1) Each incremental build starts with knowledge of the last known good
dependency graph and analysis results from the last successful build,
plus of course information about the set of files changed.
2) The previous dependency graph's information is used to determine the
set of source files which have "logically" changed. A source file is
considered logically changed if it or any of its dependencies have
physically changed (on disk) since the last successful compilation. Any
logically unchanged dependencies have their dependency information copied
over to the new dependency graph.
3) During the `TraitCompiler`'s loop to consider all source files in the
program, if a source file is logically unchanged then its previous
analyses are "adopted" (and their 'register' steps are run). If the file
is logically changed, then it is re-analyzed as usual.
4) Then, incremental build proceeds as before, with the new dependency graph
being used to determine the set of files which require re-emitting.
This analysis reuse avoids template parsing operations in many circumstances
and significantly reduces the time it takes ngtsc to rebuild a large
application.
Future work will increase performance even more, by tackling a variety of
other opportunities to reuse or avoid work.
PR Close#34288
Prior to this commit, the `IvyCompilation` tracked the state of each matched
`DecoratorHandler` on each class in the `ts.Program`, and how they
progressed through the compilation process. This tracking was originally
simple, but had grown more complicated as the compiler evolved. The state of
each specific "target" of compilation was determined by the nullability of
a number of fields on the object which tracked it.
This commit formalizes the process of compilation of each matched handler
into a new "trait" concept. A trait is some aspect of a class which gets
created when a `DecoratorHandler` matches the class. It represents an Ivy
aspect that needs to go through the compilation process.
Traits begin in a "pending" state and undergo transitions as various steps
of compilation take place. The `IvyCompilation` class is renamed to the
`TraitCompiler`, which manages the state of all of the traits in the active
program.
Making the trait concept explicit will support future work to incrementalize
the expensive analysis process of compilation.
PR Close#34288
Previously the UMD rendering formatter assumed that
there would already be import (and an export) arguments
to the UMD factory function.
This commit adds support for this corner case.
Fixes#34138
PR Close#34353
When statically evalulating UMD code it is possible to find
that we are looking for the declaration of an identifier that
actually came from a typings file (rather than a UMD file).
Previously, the UMD reflection host would always try to use
a UMD specific algorithm for finding identifier declarations,
but when the id is actually in a typings file this resulted in the
returned declaration being the containing file of the declaration
rather than the declaration itself.
Now the UMD reflection host will check to see if the file containing
the identifier is a typings file and use the appropriate stategy.
PR Close#34356
The undecorated child migration creates a synthetic decorator, which
contained `"exportAs": ["exportName"]` as obtained from the metadata of
the parent class. This is a problem, as `exportAs` needs to specified
as a comma-separated string instead of an array. This commit fixes the
bug by transforming the array of export names back to a comma-separated
string.
PR Close#34014
When ngcc is analyzing synthetically inserted decorators from a
migration, it is typically not expected that any diagnostics are
produced. In the situation where a diagnostic is produced, however, the
diagnostic would not be reported at all. This commit ensures that
diagnostics in migrations are reported.
PR Close#34014
JoostK
George Kalpakas
Paul Gschwendtner
Pete Bacon Darwin
Igor Minar
Greg Magolan
crisbeto
atscott
Alex Rickabaugh