Commit Graph

410 Commits

Author SHA1 Message Date
Misko Hevery 70962465b5 fix(core): Switch `emitDistinctChangesOnlyDefaultValue` to true (#41121)
BREAKING CHANGE:

Switching default of `emitDistinctChangesOnlyDefaultValue`
which changes the default behavior and may cause some applications which
rely on the incorrect behavior to fail.

`emitDistinctChangesOnly` flag has also been deprecated and will be
removed in a future major release.

The previous implementation would fire changes `QueryList.changes.subscribe`
whenever the `QueryList` was recomputed. This resulted in an artificially
high number of change notifications, as it is possible that recomputing
`QueryList` results in the same list. When the `QueryList` gets recomputed
is an implementation detail, and it should not be the thing that determines
how often change event should fire.

Unfortunately, fixing the behavior outright caused too many existing
applications to fail. For this reason, Angular considers this fix a
breaking fix and has introduced a flag in `@ContentChildren` and
`@ViewChildren`, that controls the behavior.

```
export class QueryCompWithStrictChangeEmitParent {
  @ContentChildren('foo', {
    // This option is the new default with this change.
    emitDistinctChangesOnly: true,
  })
  foos!: QueryList<any>;
}
```
For backward compatibility before v12
`emitDistinctChangesOnlyDefaultValue` was set to `false. This change
changes the default to `true`.

PR Close #41121
2021-03-12 10:47:56 -08:00
Igor Minar 9c210281d4 feat(compiler): emit @__PURE__ or @pureOrBreakMyCode annotations in the generated code (#41096)
This change marks all relevant define* callsites as pure, causing the compiler to
emmit either @__PURE__ or @pureOrBreakMyCode annotation based on whether we are
compiling code annotated for closure or terser.

This change is needed in g3 where we don't run build optimizer but we
need the code to be annotated for the closure compiler.

Additionally this change allows for simplification of CLI and build optimizer as they
will no longer need to rewrite the generated code (there are still other places where
a build optimizer rewrite will be necessary so we can't remove it, we can only simplify it).

PR Close #41096
2021-03-08 10:30:08 -08:00
JoostK fed6a7ce7d perf(compiler-cli): detect semantic changes and their effect on an incremental rebuild (#40947)
In Angular programs, changing a file may require other files to be
emitted as well due to implicit NgModule dependencies. For example, if
the selector of a directive is changed then all components that have
that directive in their compilation scope need to be recompiled, as the
change of selector may affect the directive matching results.

Until now, the compiler solved this problem using a single dependency
graph. The implicit NgModule dependencies were represented in this
graph, such that a changed file would correctly also cause other files
to be re-emitted. This approach is limited in a few ways:

1. The file dependency graph is used to determine whether it is safe to
   reuse the analysis data of an Angular decorated class. This analysis
   data is invariant to unrelated changes to the NgModule scope, but
   because the single dependency graph also tracked the implicit
   NgModule dependencies the compiler had to consider analysis data as
   stale far more often than necessary.
2. It is typical for a change to e.g. a directive to not affect its
   public API—its selector, inputs, outputs, or exportAs clause—in which
   case there is no need to re-emit all declarations in scope, as their
   compilation output wouldn't have changed.

This commit implements a mechanism by which the compiler is able to
determine the impact of a change by comparing it to the prior
compilation. To achieve this, a new graph is maintained that tracks all
public API information of all Angular decorated symbols. During an
incremental compilation this information is compared to the information
that was captured in the most recently succeeded compilation. This
determines the exact impact of the changes to the public API, which
is then used to determine which files need to be re-emitted.

Note that the file dependency graph remains, as it is still used to
track the dependencies of analysis data. This graph does no longer track
the implicit NgModule dependencies, which allows for better reuse of
analysis data.

These changes also fix a bug where template type-checking would fail to
incorporate changes made to a transitive base class of a
directive/component. This used to be a problem because transitive base
classes were not recorded as a transitive dependency in the file
dependency graph, such that prior type-check blocks would erroneously
be reused.

This commit also fixes an incorrectness where a change to a declaration
in NgModule `A` would not cause the declarations in NgModules that
import from NgModule `A` to be re-emitted. This was intentionally
incorrect as otherwise the performance of incremental rebuilds would
have been far worse. This is no longer a concern, as the compiler is now
able to only re-emit when actually necessary.

Fixes #34867
Fixes #40635
Closes #40728

PR Close #40947
2021-03-08 08:41:19 -08:00
Pete Bacon Darwin 0b69fabcf5 fix(compiler-cli): ensure ngcc can handle wildcard base-paths (#41033)
Ngcc uses the `paths` property to compute the potential base-paths
for packages that are being processed. If the `paths` contain a wildcard
`*` within a path segment, ngcc was not finding the base-path correctly.

Now when a wildcard is found, there is an additional search to look for
paths that might match the wildcard.

Fixes #41014

PR Close #41033
2021-03-01 15:25:44 -08:00
George Kalpakas 284af7308b fix(ngcc): do not fail hard when a format-path points to a non-existing or empty file (#40985)
Previously, when `ngcc` encountered an entry-point with a format-path
that pointed to a non-existing or empty file it would throw an error and
stop processing the remaining tasks.

In the past, we used to ignore such format-paths and continue processing
the rest of the tasks ([see code][1]). This was changed to a hard
failure in 2954d1b5ca. Looking at the code
history, the reason for changing the behavior was an (incorrect)
assumption that the condition could not fail. This assumption failed to
take into account the case where a 3rd-party library has an invalid
format-path in its `package.json`. This is an issue with the library,
but it should not prevent `ngcc` from processing other
packages/entry-points/formats.

This commit fixes this by reporting the task as failed but not throwing
an error, thus allowing `ngcc` to continue processing other tasks.

[1]: https://github.com/angular/angular/blob/3077c9a1f89c5bd75fb96c16e/packages/compiler-cli/ngcc/src/main.ts#L124

Fixes #40965

PR Close #40985
2021-02-26 08:26:33 -08:00
Pete Bacon Darwin 8d13f631d9 refactor(ngcc): support processing only the typings files of packages (#40976)
Some tools (such as Language Server and ng-packagr) only care about
the processed typings generated by ngcc. Forcing these tools to process
the JavaScript files as well has two disadvantages:

First, unnecessary work is being done, which is time consuming.

But more importantly, it is not always possible to know how the final bundling
tools will want the processed JavaScript to be configured. For example,
the CLI would prefer the `--create-ivy-entry-points` option but this would
break non-CLI build tooling.

This commit adds a new option (`--typings-only` on the command line, and
`typingsOnly` via programmatic API) that instructs ngcc to only render changes
to the typings files for the entry-points that it finds, and not to write any
JavaScript files.

In order to process the typings, a JavaScript format will need to be analysed, but
it will not be rendered to disk. When using this option, it is best to offer ngcc a
wide range of possible JavaScript formats to choose from, and it will use the
first format that it finds. Ideally you would configure it to try the `ES2015` FESM
format first, since this will be the most performant.

Fixes #40969

PR Close #40976
2021-02-24 14:23:14 -08:00
Alex Rickabaugh 80f4ff3338 fix(compiler-cli): set TS original node on imported namespace identifiers (#40711)
This commit causes imports added by ngtsc's `ImportManager` to have their
TypeScript "original node" set to the generated `ts.ImportDeclaration`
statement.

In g3, the tsickle transformer runs after the Angular transformer and post-
processes Angular's compilation output. One of its post-processing tasks is
to transform generated imports and references to imported symbols from the
commonjs module system to the g3 module system. Part of this transformation
involves recognizing modules with specific metadata and altering references
to symbols from those modules accordingly.

Normally, tsickle can rely on TypeScript's binding for an imported symbol to
find its origin module and thus the correct metadata for the symbol. However
the Angular transform generates new synthetic imports which don't have such
binding information. Angular's imports are always namespace imports of the
form:

```
import * as qualifier 'module/specifier';
```

References to such an import are then of the form `qualifier.SymbolName`.

To process such imports properly, tsickle needs to be able to associate the
reference to `qualifier` in the expression `qualifer.SymbolName` with the
`ts.ImportDeclaration` statement that defines it. It expects to do this by
looking at the `ts.getOriginalNode()` for the `qualifier` reference, which
should be the `ts.ImportDeclaration`. This commit changes ngtsc's import
generation mechanism to set the original node on `qualifier` identifiers
according to this expectation.

This commit is not tested in the direct compiler tests, since:

1) there is no observable behavior externally from setting the original node
2) we don't have tests that intercept transformer operations (which could be
   used to directly assert against the AST nodes)
3) tsickle's published version does not (yet) contain the g3-specific
   transformations which rely on the original node and would thus allow the
   behavior to be observed.

Instead, we rely on the g3 testing suite to validate the correctness of this
fix. Breaking this functionality would cause g3 compilation errors for
targets, since tsickle would be unable to transform imports correctly.

PR Close #40711
2021-02-11 15:58:25 -08:00
Joey Perrott d7f5755f80 fix(compiler-cli): update ngcc integration tests for latest changes in rules_nodejs (#40710)
Update the ngcc integration tests to handle the changes in the latest version of rules_nodejs.

PR Close #40710
2021-02-09 10:48:43 -08:00
Joey Perrott b75d7cb11f fix(compiler-cli): update type castings for JSON.parse usage (#40710)
Update usages of JSON.parse to be cast as specific types.

PR Close #40710
2021-02-09 10:48:43 -08:00
Misko Hevery d516113803 refactor(core): Remove the need for explicit static query instruction (#40091)
Because the query now has `flags` which specify the mode, the static query
instruction can now be remove. It is simply normal query with `static` flag.

PR Close #40091
2021-01-14 13:55:02 -08:00
Pete Bacon Darwin ad0fb9c6bb fix(ngcc): copy (and update) source-maps for unmodified source files (#40429)
When using the `NewEntryPointWriter` we copy unmodified files over to the new
entry-point in addition to writing out the source files that are processed by ngcc.
But we were not copying over associated source-map files for these unmodified
source files, leading to warnings in downstream tooling.

Now we will also copy over source-maps that reside as siblings of unmodified
source files. We have to make sure that the sources of the source-map point
to the correct files, so we also update the `sourceRoot` property of the copied
source-map.

Fixes #40358

PR Close #40429
2021-01-14 11:30:39 -08:00
Pete Bacon Darwin afd11662a3 fix(ngcc): compute the correct package paths for target entry-points (#40376)
Previously, if there were path-mapped entry-points, where one contaied the
string of another - for example `worker-client` and `worker` - then the
base paths were incorrectly computed resulting in the wrong package path
for the longer entry-point. This was because, when searching for a matching
base path, the strings were tested using `startsWith()`, whereas we should
only match if the path was contained in a directory from a file-system
point of view.

Now we not only check whether the target path "starts with" the base path
but then also whether the target path is actually contained in the base path
using `fs.relative()`.

Fixes #40352
Fixes #40357

PR Close #40376
2021-01-11 10:38:16 -08:00
Pete Bacon Darwin d100a15998 refactor(compiler-cli): update to use new file-system interfaces (#40281)
Now that `ReadonlyFileSystem` and `PathManipulation` interfaces are
available, this commit updates the compiler-cli to use these more
focussed interfaces.

PR Close #40281
2021-01-08 09:34:44 -08:00
Pete Bacon Darwin 7c167629b8 fix(compiler-cli): ngcc - remove outdated link (#40285)
The link to the "speeding-up-ngcc-compilation" URL does not exist,
it was removed shortly after it was added, but the link in the ngcc
error message was not updated.

Fixes #39837

PR Close #40285
2021-01-06 07:10:39 -08:00
JoostK 1f73af77a7 refactor(compiler-cli): use `ngDevMode` guard for `setClassMetadata` call (#39987)
Prior to this change, the `setClassMetadata` call would be invoked
inside of an IIFE that was marked as pure. This allows the call to be
tree-shaken away in production builds, as the `setClassMetadata` call
is only present to make the original class metadata available to the
testing infrastructure. The pure marker is problematic, though, as the
`setClassMetadata` call does in fact have the side-effect of assigning
the metadata into class properties. This has worked under the assumption
that only build optimization tools perform tree-shaking, however modern
bundlers are also able to elide calls that have been marked pure so this
assumption does no longer hold. Instead, an `ngDevMode` guard is used
which still allows the call to be elided but only by tooling that is
configured to consider `ngDevMode` as constant `false` value.

PR Close #39987
2020-12-10 13:23:13 -08:00
Bjarki ef892743ec feat(compiler): support tagged template literals in code generator (#39122)
Add a TaggedTemplateExpr to represent tagged template literals in
Angular's syntax tree (more specifically Expression in output_ast.ts).
Also update classes that implement ExpressionVisitor to add support for
tagged template literals in different contexts, such as JIT compilation
and conversion to JS.

Partial support for tagged template literals had already been
implemented to support the $localize tag used by Angular's i18n
framework. Where applicable, this code was refactored to support
arbitrary tags, although completely replacing the i18n-specific support
for the $localize tag with the new generic support for tagged template
literals may not be completely trivial, and is left as future work.

PR Close #39122
2020-12-07 16:20:04 -08:00
Alex Rickabaugh 6d42954327 fix(compiler-cli): remove the concept of an errored trait (#39923)
Previously, if a trait's analysis step resulted in diagnostics, the trait
would be considered "errored" and no further operations, including register,
would be performed. Effectively, this meant that the compiler would pretend
the class in question was actually undecorated.

However, this behavior is problematic for several reasons:

1. It leads to inaccurate diagnostics being reported downstream.

For example, if a component is put into the error state, for example due to
a template error, the NgModule which declares the component would produce a
diagnostic claiming that the declaration is neither a directive nor a pipe.
This happened because the compiler wouldn't register() the component trait,
so the component would not be recorded as actually being a directive.

2. It can cause incorrect behavior on incremental builds.

This bug is more complex, but the general issue is that if the compiler
fails to associate a component and its module, then incremental builds will
not correctly re-analyze the module when the component's template changes.
Failing to register the component as such is one link in the larger chain of
issues that result in these kinds of issues.

3. It lumps together diagnostics produced during analysis and resolve steps.

This is not causing issues currently as the dependency graph ensures the
right classes are re-analyzed when needed, instead of showing stale
diagnostics. However, the dependency graph was not intended to serve this
role, and could potentially be optimized in ways that would break this
functionality.

This commit removes the concept of an "errored" trait entirely from the
trait system. Instead, analyzed and resolved traits have corresponding (and
separate) diagnostics, in addition to potentially `null` analysis results.
Analysis (but not resolution) diagnostics are carried forward during
incremental build operations. Compilation (emit) is only performed when
a trait reaches the resolved state with no diagnostics.

This change is functionally different than before as the `register` step is
now performed even in the presence of analysis errors, as long as analysis
results are also produced. This fixes problem 1 above, and is part of the
larger solution to problem 2.

PR Close #39923
2020-12-03 13:42:13 -08:00
Alex Rickabaugh 3613e7c4e5 test(compiler-cli): move testing utils to separate package (#39594)
ngtsc has a robust suite of testing utilities, designed for in-memory
testing of a TypeScript compiler. Previously these utilities lived in the
`test` directory for the compiler-cli package.

This commit moves those utilities to an `ngtsc/testing` package, enabling
them to be depended on separately and opening the door for using them from
the upcoming language server testing infrastructure.

As part of this refactoring, the `fake_core` package (a lightweight API
replacement for @angular/core) is expanded to include functionality needed
for Language Service test use cases.

PR Close #39594
2020-11-17 11:59:56 -08:00
Pete Bacon Darwin 0a63eeaff1 Revert "perf(ngcc): allow immediately reporting a stale lock file (#37250)" (#39435)
This reverts commit 561c0f81a0.

The original commit provided a quick escape from an already terminal
situation by killing the process if the PID in the lockfile was not
found in the list of processes running on the current machine.

But this broke use-cases where the node_modules was being shared between
multiple machines (or more commonly Docker containers on the same actual
machine).

Fixes #38875

PR Close #39435
2020-10-27 13:36:27 -07:00
Pete Bacon Darwin 413b55273b fix(ngcc): capture UMD/CommonJS inner class implementation node correctly (#39346)
Previously, UMD/CommonJS class inline declarations of the form:

```ts
exports.Foo = (function() { function Foo(); return Foo; })();
```

were capturing the whole IIFE as the implementation, rather than
the inner class (i.e. `function Foo() {}` in this case). This caused
the interpreter to break when it was trying to access such an export,
since it would try to evaluate the IIFE rather than treating it as a class
declaration.

PR Close #39346
2020-10-23 15:17:11 -07:00
Pete Bacon Darwin 822b838fbc fix(ngcc): ensure that "inline exports" can be interpreted correctly (#39267)
Previously, inline exports of the form `exports.foo = <implementation>;` were
being interpreted (by the ngtsc `PartialInterpeter`) as `Reference` objects.
This is not what is desired since it prevents the value of the export
from being unpacked, such as when analyzing `NgModule` declarations:

```
exports.directives = [Directive1, Directive2];

@NgImport({declarations: [exports.directives]})
class AppModule {}
```

In this example the interpreter would think that `exports.directives`
was a reference rather than an array that needs to be unpacked.

This bug was picked up by the ngcc-validation repository. See
https://github.com/angular/ngcc-validation/pull/1990 and
https://circleci.com/gh/angular/ngcc-validation/17130

PR Close #39267
2020-10-14 14:11:45 -07:00
JoostK 898be92f70 perf(ngcc): do not rescan program source files when referenced from multiple root files (#39254)
When ngcc is configured to run with the `--use-program-dependencies`
flag, as is the case in the CLI's asynchronous processing, it will scan
all source files in the program, starting from the program's root files
as configured in the tsconfig. Each individual root file could
potentially rescan files that had already been scanned for an earlier
root file, causing a severe performance penalty if the number of root
files is large. This would be the case if glob patterns are used in the
"include" specification of a tsconfig file.

This commit avoids the performance penalty by keeping track of the files
that have been scanned across all root files, such that no source file
is scanned multiple times.

Fixes #39240

PR Close #39254
2020-10-14 09:34:11 -07:00
Pete Bacon Darwin f4fee86f77 fix(ngcc): support inline export declarations in UMD files (#38959)
Previously, any declarations that were defined "inline" were not
recognised by the `UmdReflectionHost`.

For example, the following syntax was completely unrecognized:

```ts
var Foo_1;
exports.Foo = Foo_1 = (function() {
  function Foo() {}
  return Foo;
})();
exports.Foo = Foo_1 = __decorate(SomeDecorator, Foo);
```

Such inline classes were ignored and not processed by ngcc.

This lack of processing led to failures in Ivy applications that relied
on UMD formats of libraries such as `syncfusion/ej2-angular-ui-components`.

Now all known inline UMD exports are recognized and processed accordingly.

Fixes #38947

PR Close #38959
2020-10-12 08:32:46 -07:00
Pete Bacon Darwin 47eab61cad test(ngcc): use `isNamedDeclaration()` helper to simplify tests (#38959)
Previously these tests were checking multiple specific expression
types. The new helper function is more general and will also support
`PropertyAccessExpression` nodes for `InlineDeclaration` types.

PR Close #38959
2020-10-12 08:32:46 -07:00
Pete Bacon Darwin 0accd1e68d refactor(compiler-cli): implement `DeclarationNode` node type (#38959)
Previously the `ConcreteDeclaration` and `InlineDeclaration` had
different properties for the underlying node type. And the `InlineDeclaration`
did not store a value that represented its declaration.

It turns out that a natural declaration node for an inline type is the
expression. For example in UMD/CommonJS this would be the `exports.<name>`
property access node.

So this expression is now used for the `node` of `InlineDeclaration` types
and the `expression` property is dropped.

To support this the codebase has been refactored to use a new `DeclarationNode`
type which is a union of `ts.Declaration|ts.Expression` instead of `ts.Declaration`
throughout.

PR Close #38959
2020-10-12 08:32:46 -07:00
Pete Bacon Darwin 2c0282f4c2 test(ngcc): use `isNamedFunctionDeclaration()` in UMD tests (#38959)
This makes these tests more resilient to changes in the test code
structure. For example switching from

```
var SomeClass = <implementation>;
exports.SomeClass = SomeClass;
```

to

```
exports.SomeClass = <implementation>;
```

PR Close #38959
2020-10-12 08:32:46 -07:00
Pete Bacon Darwin 6650d71fe2 test(compiler-cli): make the `getDeclaration()` utility more resilient to code format (#38959)
Previously `getDeclaration()` would only return the first node that matched
the name passed in and then assert the predicate on this single node.
It also only considered a subset of possible declaration types that we might
care about.

Now the function will parse the whole tree collecting an array of all the
nodes that match the name. It then filters this array based on the predicate
and only errors if the filtered array is empty.

This makes this function much more resilient to more esoteric code formats
such as UMD.

PR Close #38959
2020-10-12 08:32:46 -07:00
Pete Bacon Darwin 5038e5741b fix(ngcc): handle aliases in UMD export declarations (#38959)
Sometimes UMD exports appear in the following form:

```
exports.MyClass = alias1 = alias2 = <<declaration>>
```

Previously the declaration of the export would have been captured
as `alias1 = alias2 = <<declaration>>`, which the `PartialInterpreter`
would have failed on, since it cannot handle assignments.

Now we skip over these aliases capturing only the `<<declaration>>`
expression.

Fixes #38947

PR Close #38959
2020-10-12 08:32:46 -07:00
Pete Bacon Darwin 11485d96fb fix(ngcc): map `exports` to the current module in UMD files (#38959)
UMD files export values by assigning them to an `exports` variable.
When evaluating expressions ngcc was failing to cope with expressions
like `exports.MyComponent`.

This commit fixes the `UmdReflectionHost.getDeclarationOfIdentifier()`
method to map the `exports` variable to the current source file.

PR Close #38959
2020-10-12 08:32:46 -07:00
Pete Bacon Darwin acfce0ba1b test(ngcc): fix incorrect test setup (#38959)
The `SIMPLE_CLASS_FILE` contained a `ChildClass` that had an
internal aliases implementation and extended a `SuperClass` base
class. The call to `__extends` was using the wrong argument for
the child class.

PR Close #38959
2020-10-12 08:32:45 -07:00
JoostK 9d04b95166 refactor(compiler-cli): setup compilation mode to enable generating linker code (#38938)
This is a precursor to introducing the Angular linker. As an initial
step, a compiler option to configure the compilation mode is introduced.
This option is initially internal until the linker is considered ready.

PR Close #38938
2020-09-30 12:49:16 -07:00
Pete Bacon Darwin 297b123151 refactor(compiler-cli): make the output AST translator generic (#38775)
This commit refactors the `ExpressionTranslatorVisitor` so that it
is not tied directly to the TypeScript AST. Instead it uses generic
`TExpression` and `TStatement` types that are then converted
to concrete types by the `TypeScriptAstFactory`.

This paves the way for a `BabelAstFactory` that can be used to
generate Babel AST nodes instead of TypeScript, which will be
part of the new linker tool.

PR Close #38775
2020-09-21 12:27:27 -07:00
Pete Bacon Darwin 123bff7cb6 fix(compiler-cli): generate `let` statements in ES2015+ mode (#38775)
When the target of the compiler is ES2015 or newer then we should
be generating `let` and `const` variable declarations rather than `var`.

PR Close #38775
2020-09-21 12:27:27 -07:00
JoostK e4424863c2 fix(ngcc): fix compilation of `ChangeDetectorRef` in pipe constructors (#38892)
In #38666 we changed how ngcc deals with type expressions, where it
would now always emit the original type expression into the generated
code as a "local" type value reference instead of synthesizing new
imports using an "imported" type value reference. This was done as a fix
to properly deal with renamed symbols, however it turns out that the
compiler has special handling for certain imported symbols, e.g.
`ChangeDetectorRef` from `@angular/core`. The "local" type value
reference prevented this special logic from being hit, resulting in
incorrect compilation of pipe factories.

This commit fixes the issue by manually inspecting the import of the
type expression, in order to return an "imported" type value reference.
By manually inspecting the import we continue to handle renamed symbols.

Fixes #38883

PR Close #38892
2020-09-18 08:02:46 -07:00
Pete Bacon Darwin d795a00137 refactor(compiler): replace Comment nodes with leadingComments property (#38811)
Common AST formats such as TS and Babel do not use a separate
node for comments, but instead attach comments to other AST nodes.
Previously this was worked around in TS by creating a `NotEmittedStatement`
AST node to attach the comment to. But Babel does not have this facility,
so it will not be a viable approach for the linker.

This commit refactors the output AST, to remove the `CommentStmt` and
`JSDocCommentStmt` nodes. Instead statements have a collection of
`leadingComments` that are rendered/attached to the final AST nodes
when being translated or printed.

PR Close #38811
2020-09-18 08:01:25 -07:00
JoostK a1c1c450dc test(ngcc): load standard files only once (#38840)
In the integration test suite of ngcc, we load a set of files from
`node_modules` into memory. This includes the `typescript` package and
`@angular` scoped packages, which account for a large number of large
files that needs to be loaded from disk. This commit moves this work
to the top-level, such that it doesn't have to be repeated in all tests.

PR Close #38840
2020-09-15 11:23:13 -07:00
JoostK fd44d84a33 perf(ngcc): reduce maximum worker count (#38840)
Recent optimizations to ngcc have significantly reduced the total time
it takes to process `node_modules`, to such extend that sharding across
multiple processes has become less effective. Previously, running
ngcc asynchronously would allow for up to 8 workers to be allocated,
however these workers have to repeat work that could otherwise be shared.
Because ngcc is now able to reuse more shared computations, the overhead
of multiple workers is increased and therefore becomes less effective.
As an additional benefit, having fewer workers requires less memory and
less startup time.

To give an idea, using the following test setup:

```bash
npx @angular/cli new perf-test
cd perf-test
yarn ng add @angular/material
./node_modules/.bin/ngcc --properties es2015 module main \
  --first-only --create-ivy-entry-points
```

We observe the following figures on CI:

|                   | 10.1.1    | PR #38840 |
| ----------------- | --------- | --------- |
| Sync              | 85s       | 25s       |
| Async (8 workers) | 22s       | 16s       |
| Async (4 workers) | -         | 11s       |

In addition to changing the default number of workers, ngcc will now
use the environment variable `NGCC_MAX_WORKERS` that may be configured
to either reduce or increase the number of workers.

PR Close #38840
2020-09-15 11:23:09 -07:00
JoostK f0688b4d18 perf(ngcc): introduce cache for sharing data across entry-points (#38840)
ngcc creates typically two `ts.Program` instances for each entry-point,
one for processing sources and another one for processing the typings.
The creation of these programs is somewhat expensive, as it concerns
module resolution and parsing of source files.

This commit implements several layers of caching to optimize the
creation of programs:

1. A shared module resolution cache across all entry-points within a
   single invocation of ngcc. Both the sources and typings program
   benefit from this cache.
2. Sharing the parsed `ts.SourceFile` for a single entry-point between
   the sources and typings program.
3. Sharing parsed `ts.SourceFile`s of TypeScript's default libraries
   across all entry-points within a single invocation. Some of these
   default library typings are large and therefore expensive to parse,
   so sharing the parsed source files across all entry-points offers
   a significant performance improvement.

Using a bare CLI app created using `ng new` + `ng add @angular/material`,
the above changes offer a 3-4x improvement in ngcc's processing time
when running synchronously and ~2x improvement for asynchronous runs.

PR Close #38840
2020-09-15 11:23:04 -07:00
Pete Bacon Darwin 7869de6136 fix(ngcc): use aliased exported types correctly (#38666)
If a type has been renamed when it was exported, we need to
reference the external public alias name rather than the internal
original name for the type. Otherwise we will try to import the
type by its internal name, which is not publicly accessible.

Fixes #38238

PR Close #38666
2020-09-08 11:41:21 -07:00
Paul Gschwendtner 3b9c802dee fix(ngcc): detect synthesized delegate constructors for downleveled ES2015 classes (#38463)
Similarly to the change we landed in the `@angular/core` reflection
capabilities, we need to make sure that ngcc can detect pass-through
delegate constructors for classes using downleveled ES2015 output.

More details can be found in the preceding commit, and in the issue
outlining the problem: #38453.

Fixes #38453.

PR Close #38463
2020-08-17 10:55:40 -07:00
JoostK 18098d38b8 fix(compiler-cli): avoid creating value expressions for symbols from type-only imports (#37912)
In TypeScript 3.8 support was added for type-only imports, which only brings in
the symbol as a type, not their value. The Angular compiler did not yet take
the type-only keyword into account when representing symbols in type positions
as value expressions. The class metadata that the compiler emits would include
the value expression for its parameter types, generating actual imports as
necessary. For type-only imports this should not be done, as it introduces an
actual import of the module that was originally just a type-only import.

This commit lets the compiler deal with type-only imports specially, preventing
a value expression from being created.

Fixes #37900

PR Close #37912
2020-08-11 09:53:25 -07:00
Pete Bacon Darwin b358495a6c fix(ngcc): report a warning if ngcc tries to use a solution-style tsconfig (#38003)
In CLI v10 there was a move to use the new solution-style tsconfig
which became available in TS 3.9.

The result of this is that the standard tsconfig.json no longer contains
important information such as "paths" mappings, which ngcc might need to
correctly compute dependencies.

ngcc (and ngc and tsc) infer the path to tsconfig.json if not given an
explicit tsconfig file-path. But now that means it infers the solution
tsconfig rather than one that contains the useful information it used to
get.

This commit logs a warning in this case to inform the developer
that they might not have meant to load this tsconfig and offer
alternative options.

Fixes #36386

PR Close #38003
2020-07-14 13:21:31 -07:00
JoostK a87951a28f fix(ngcc): prevent including JavaScript sources outside of the package (#37596)
When ngcc creates an entry-point program, the `allowJs` option is enabled
in order to operate on the JavaScript source files of the entry-point.
A side-effect of this approach is that external modules that don't ship
declaration files will also have their JavaScript source files loaded
into the program, as the `allowJs` flag allows for them to be imported.
This may pose an issue in certain edge cases, where ngcc would inadvertently
operate on these external modules. This can introduce all sorts of undesirable
behavior and incompatibilities, e.g. the reflection host that is selected for
the entry-point's format could be incompatible with that of the external
module's JavaScript bundles.

To avoid these kinds of issues, module resolution that would resolve to
a JavaScript file located outside of the package will instead be rejected,
as if the file would not exist. This would have been the behavior when
`allowJs` is set to false, which is the case in typical Angular compilations.

Fixes #37508

PR Close #37596
2020-06-29 12:21:22 -07:00
JoostK 6b565ba8f2 refactor(ngcc): let `isWithinPackage` operate on paths instead of source files (#37596)
Changes `isWithinPackage` to take an `AbsoluteFsPath` instead of `ts.SourceFile`,
to allow for an upcoming change to use it when no `ts.SourceFile` is available,
but just a path.

PR Close #37596
2020-06-29 12:21:22 -07:00
Pete Bacon Darwin 78a44768e0 fix(ngcc): ensure lockfile is removed when analyzeFn fails (#37739)
Previously an error thrown in the `analyzeFn` would cause
the ngcc process to exit immediately without removing the
lockfile, and potentially before the unlocker process had been
successfully spawned resulting in the lockfile being orphaned
and left behind.

Now we catch these errors and remove the lockfile as needed.

PR Close #37739
2020-06-29 10:29:11 -07:00
Pete Bacon Darwin 9318e23e64 perf(ngcc): use `EntryPointManifest` to speed up noop `ProgramBaseEntryPointFinder` (#37665)
Previously the `ProgramBasedEntryPointFinder` was parsing all the
entry-points referenced by the program for dependencies even if all the
entry-points had been processed already.

Now this entry-point finder will re-use the `EntryPointManifest` to load
the entry-point dependencies when possible which avoids having to parse
them all again, on every invocation of ngcc.

Previously the `EntryPointManifest` was only used in the
`DirectoryWalkerEntryPointFinder`, which also contained the logic for
computing the contents of the manifest. This logic has been factored out
into an `EntryPointCollector` class. Both the `ProgramBasedEntryPointFinder`
and `DirectoryWalkerEntryPointFinder` now use the `EntryPointManifest` and
the `EntryPointCollector`.

The result of this change is that there is a small cost on the first run of
ngcc to compute and store the manifest - the processing takes 102% of the
processing time before this PR. But on subsequent runs there is a
significant benefit on subsequent runs - the processing takes around 50%
of the processing time before this PR.

PR Close #37665
2020-06-25 14:11:03 -07:00
Pete Bacon Darwin bedc0451a0 docs(ngcc): add additional next steps to an error (#37672)
The file-writing error in the this commit can also be the result
of the ngcc process dying in the middle of writing files.

This commit improves the error message to offer a resolution
in case this is the reason for the error.

Fixes #36393

PR Close #37672
2020-06-25 11:37:43 -07:00
Pete Bacon Darwin 2b53b07c70 refactor(ngcc): move `sourcemaps` into `ngtsc` (#37114)
The `SourceFile` and associated code is general and reusable in
other projects (such as `@angular/localize`). Moving it to `ngtsc`
makes it more easily shared.

PR Close #37114
2020-06-22 13:38:47 -07:00
Pete Bacon Darwin 6de5a12a9d refactor(ngcc): move `logging` code into `ngtsc` (#37114)
The `Logger` interface and its related classes are general purpose
and could be used by other tooling. Moving it into ngtsc is a more
suitable place from which to share it - similar to the FileSystem stuff.

PR Close #37114
2020-06-22 13:38:47 -07:00
George Kalpakas bf682d73d4 fix(ngcc): correctly get config for packages in nested `node_modules/` (#37040)
Previously, ngcc would only be able to match an ngcc configuration to
packages that were located inside the project's top-level
`node_modules/`. However, if there are multiple versions of a package in
a project (e.g. as a transitive dependency of other packages), multiple
copies of a package (at different versions) may exist in nested
`node_modules/` directories. For example, one at
`<project-root>/node_modules/some-package/` and one at
`<project-root>/node_modules/other-package/node_modules/some-package/`.
In such cases, ngcc was only able to detect the config for the first
copy but not for the second.

This commit fixes this by returning a new instance of
`ProcessedNgccPackageConfig` for each different package path (even if
they refer to the same package name). In these
`ProcessedNgccPackageConfig`, the `entryPoints` paths have been
processed to take the package path into account.

PR Close #37040
2020-06-11 18:58:36 -07:00