Some declaration files may not be referenced from an entry-point's
main typings file, as it may declare types that are only used internally.
ngcc has logic to include declaration files based on all source files,
to ensure internal declaration files are available.
For packages following APF layout, however, this logic was insufficient.
Consider an entry-point with base path of `/esm2015/testing` and typings
residing in `/testing`, the file
`/esm2015/testing/src/nested/internal.js` has its typings file at
`/testing/src/nested/internal.d.ts`. Previously, the declaration was
assumed to be located at `/esm2015/testing/testing/internal.d.ts` (by
means of `/esm2015/testing/src/nested/../../testing/internal.d.ts`)
which is not where the declaration file can be found. This commit
resolves the issue by looking in the correct directory.
PR Close#33875
In flat bundle formats, multiple classes that have the same name can be
suffixed to become unique. In ES5-like bundles this results in the outer
declaration from having a different name from the "implementation"
declaration within the class' IIFE, as the implementation declaration
may not have been suffixed.
As an example, the following code would fail to have a `Directive`
decorator as ngcc would search for `__decorate` calls that refer to
`AliasedDirective$1` by name, whereas the `__decorate` call actually
uses the `AliasedDirective` name.
```javascript
var AliasedDirective$1 = /** @class */ (function () {
function AliasedDirective() {}
AliasedDirective = tslib_1.__decorate([
Directive({ selector: '[someDirective]' }),
], AliasedDirective);
return AliasedDirective;
}());
```
This commit fixes the problem by not relying on comparing names, but
instead finding the declaration and matching it with both the outer
and inner declaration.
PR Close#33878
Previously, ngcc's `Renderer` would add some constants in the processed
files which were emitted as ES2015 code (e.g. `const` declarations).
This would result in invalid ES5 generated code that would break when
run on browsers that do not support the emitted format.
This commit fixes it by adding a `printStatement()` method to
`RenderingFormatter`, which can convert statements to JavaScript code in
a suitable format for the corresponding `RenderingFormatter`.
Additionally, the `translateExpression()` and `translateStatement()`
ngtsc helper methods are augmented to accept an extra hint to know
whether the code needs to be translated to ES5 format or not.
Fixes#32665
PR Close#33514
While processing class metadata, ngtsc generates a `setClassMetadata()`
call which (among other things) contains info about property decorators.
Previously, processing getter/setter pairs with some of ngcc's
`ReflectionHost`s resulted in multiple metadata entries for the same
property, which resulted in duplicate object keys, which in turn causes
an error in ES5 strict mode.
This commit fixes it by ensuring that there are no duplicate property
names in the `setClassMetadata()` calls.
In addition, `generateSetClassMetadataCall()` is updated to treat
`ClassMember#decorators: []` the same as `ClassMember.decorators: null`
(i.e. omitting the `ClassMember` from the generated `setClassMetadata()`
call). Alternatively, ngcc's `ReflectionHost`s could be updated to do
this transformation (`decorators: []` --> `decorators: null`) when
reflecting on class members, but this would require changes in many
places and be less future-proof.
For example, given a class such as:
```ts
class Foo {
@Input() get bar() { return 'bar'; }
set bar(value: any) {}
}
```
...previously the generated `setClassMetadata()` call would look like:
```ts
ɵsetClassMetadata(..., {
bar: [{type: Input}],
bar: [],
});
```
The same class will now result in a call like:
```ts
ɵsetClassMetadata(..., {
bar: [{type: Input}],
});
```
Fixes#30569
PR Close#33514
The reflection hosts have been updated to support the following
code forms, which were found in some minified library code:
* The class IIFE not being wrapped in parentheses.
* Calls to `__decorate()` being combined with the IIFE return statement.
PR Close#33777
Previously we only removed `__decorate()` calls that looked like:
```
SomeClass = __decorate(...);
```
But in some minified scenarios this call gets wrapped up with the
return statement of the IIFE.
```
return SomeClass = __decorate(...);
```
This is now removed also, leaving just the return statement:
```
return SomeClass;
```
PR Close#33777
The `dist/` directory has a duplicate `package.json` pointing to the
same files, which (under certain configurations) can causes ngcc to try
to process the files twice and fail.
This commit adds a default ngcc config for `ng2-dragula` to ignore the
`dist/` entry-point.
Fixes#33718
PR Close#33797
Previously the renderers were fixed so that they inserted extra
"adjacent" statements after the last static property of classes.
In order to help the build-optimizer (in Angular CLI) to be able to
tree-shake classes effectively, these statements should also appear
after any helper calls, such as `__decorate()`.
This commit moves the computation of this positioning into the
`NgccReflectionHost` via the `getEndOfClass()` method, which
returns the last statement that is related to the class.
FW-1668
PR Close#33689
When ngcc is configured to generate reexports for a package using the
`generateDeepReexports` configuration option, it could incorrectly
render the reexports as often as the number of compiled classes in the
declaration file. This would cause compilation errors due to duplicated
declarations.
PR Close#33658
We already have special cases for the `__spread` helper function and with this change we handle the new tslib helper introduced in version 1.10 `__spreadArrays`.
For more context see: https://github.com/microsoft/tslib/releases/tag/1.10.0Fixes: #33614
PR Close#33617
When decorating classes with ivy definitions (e.g. `ɵfac` or `ɵdir`)
the inner name of the class declaration must be used.
This is because in ES5 the definitions are inside the class's IIFE
where the outer declaration has not yet been initialized.
PR Close#33533
In ES5 the class consists of an outer variable declaration that is
initialised by an IIFE. Inside the IIFE the class is implemented by
an inner function declaration that is returned from the IIFE.
This inner declaration may have a different name to the outer
declaration.
This commit overrides `getInternalNameOfClass()` and
`getAdjacentNameOfClass()` in `Esm5ReflectionHost` with methods that
can find the correct inner declaration name identifier.
PR Close#33533
Previously declarations that were imported via a namespace import
were given the same `bestGuessOwningModule` as the context
where they were imported to. This causes problems with resolving
`ModuleWithProviders` that have a type that has been imported in
this way, causing errors like:
```
ERROR in Symbol UIRouterModule declared in
.../@uirouter/angular/uiRouterNgModule.d.ts
is not exported from
.../@uirouter/angular/uirouter-angular.d.ts
(import into .../src/app/child.module.ts)
```
This commit modifies the `TypescriptReflectionHost.getDirectImportOfIdentifier()`
method so that it also understands how to attach the correct `viaModule` to
the identifier of the namespace import.
Resolves#32166
PR Close#33495
Now that we've replaced `ngBaseDef` with an abstract directive definition, there are a lot more cases where we generate a directive definition without a selector. These changes make it so that we don't generate the `selectors` array if it's going to be empty.
PR Close#33431
Removes `ngBaseDef` from the compiler and any runtime code that was still referring to it. In the cases where we'd previously generate a base def we now generate a definition for an abstract directive.
PR Close#33264
In Angular View Engine, there are two kinds of decorator inheritance:
1) both the parent and child classes have decorators
This case is supported by InheritDefinitionFeature, which merges some fields
of the definitions (such as the inputs or queries).
2) only the parent class has a decorator
If the child class is missing a decorator, the compiler effectively behaves
as if the parent class' decorator is applied to the child class as well.
This is the "undecorated child" scenario, and this commit adds a migration
to ngcc to support this pattern in Ivy.
This migration has 2 phases. First, the NgModules of the application are
scanned for classes in 'declarations' which are missing decorators, but
whose base classes do have decorators. These classes are the undecorated
children. This scan is performed recursively, so even if a declared class
has a base class that itself inherits a decorator, this case is handled.
Next, a synthetic decorator (either @Component or @Directive) is created
on the child class. This decorator copies some critical information such
as 'selector' and 'exportAs', as well as supports any decorated fields
(@Input, etc). A flag is passed to the decorator compiler which causes a
special feature `CopyDefinitionFeature` to be included on the compiled
definition. This feature copies at runtime the remaining aspects of the
parent definition which `InheritDefinitionFeature` does not handle,
completing the "full" inheritance of the child class' decorator from its
parent class.
PR Close#33362
When upgrading an Angular application to a new version using the Angular
CLI, built-in schematics are being run to update user code from
deprecated patterns to the new way of working. For libraries that have
been built for older versions of Angular however, such schematics have
not been executed which means that deprecated code patterns may still be
present, potentially resulting in incorrect behavior.
Some of the logic of schematics has been ported over to ngcc migrations,
which are automatically run on libraries. These migrations achieve the
same goal of the regular schematics, but operating on published library
sources instead of used code.
PR Close#33362
Previously, the (currently disabled) undecorated parent migration in
ngcc would produce errors when a base class could not be determined
statically or when a class extends from a class in another package. This
is not ideal, as it would cause the library to fail compilation without
a workaround, whereas those problems are not guaranteed to cause issues.
Additionally, inheritance chains were not handled. This commit reworks
the migration to address these limitations.
PR Close#33362
In ngcc's migration system, synthetic decorators can be injected into a
compilation to ensure that certain classes are compiled with Angular
logic, where the original library code did not include the necessary
decorators. Prior to this change, synthesized decorators would have a
fake AST structure as associated node and a made-up identifier. In
theory, this may introduce issues downstream:
1) a decorator's node is used for diagnostics, so it must have position
information. Having fake AST nodes without a position is therefore a
problem. Note that this is currently not a problem in practice, as
injected synthesized decorators would not produce any diagnostics.
2) the decorator's identifier should refer to an imported symbol.
Therefore, it is required that the symbol is actually imported.
Moreover, bundle formats such as UMD and CommonJS use namespaces for
imports, so a bare `ts.Identifier` would not be suitable to use as
identifier. This was also not a problem in practice, as the identifier
is only used in the `setClassMetadata` generated code, which is omitted
for synthetically injected decorators.
To remedy these potential issues, this commit makes a decorator's
identifier optional and switches its node over from a fake AST structure
to the class' name.
PR Close#33362
A class that is provided as Angular service is required to have an
`@Injectable()` decorator so that the compiler generates its injectable
definition for the runtime. Applications are automatically migrated
using the "missing-injectable" schematic, however libraries built for
older version of Angular may not yet satisfy this requirement.
This commit ports the "missing-injectable" schematic to a migration that
is ran when ngcc is processing a library. This ensures that any service
that is provided from an NgModule or Directive/Component will have an
`@Injectable()` decorator.
PR Close#33362
ngcc has an internal cache of computed decorator information for
reflected classes, which could previously be mutated by consumers of the
reflection host. With the ability to inject synthesized decorators, such
decorators would inadvertently be added into the array of decorators
that was owned by the internal cache of the reflection host, incorrectly
resulting in synthesized decorators to be considered real decorators on
a class. This commit fixes the issue by cloning the cached array before
returning it.
PR Close#33362
This commit adapts the private NgModule re-export system (using aliasing) to
ngcc. Not all ngcc compilations are compatible with these re-exports, as
they assume a 1:1 correspondence between .js and .d.ts files. The primary
concern here is supporting them for commonjs-only packages.
PR Close#33177
In ES5 modules, the class declarations consist of an IIFE with inner
and outer declarations that represent the class. The `EsmReflectionHost`
has logic to ensure that `getDeclarationOfIdentifier()` always returns the
outer declaration.
Before this commit, if an identifier referred to an alias of the inner
declaration, then `getDeclarationOfIdentifier()` was failing to find
the outer declaration - instead returning the inner declaration.
Now the identifier is correctly resolved up to the outer declaration
as expected.
This should fix some of the failing 3rd party packages discussed in
https://github.com/angular/ngcc-validation/issues/57.
PR Close#33252
This commit fixes ngtsc's import generator to use the ReflectionHost when
looking through the exports of an ES module to find the export of a
particular declaration that's being imported. This is necessary because
some module formats like CommonJS have unusual export mechanics, and the
normal TypeScript ts.TypeChecker does not understand them.
This fixes an issue with ngcc + CommonJS where exports were not being
enumerated correctly.
FW-1630 #resolve
PR Close#33192
Normally, when ngcc encounters a package with missing dependencies while
attempting to determine a compilation ordering, it will ignore that package.
This commit adds a configuration for a flag to tell ngcc to compile the
package anyway, regardless of any missing dependencies.
FW-1931 #resolve
PR Close#33192
In the ReflectionHost API, a 'viaModule' indicates that a particular value
originated in another absolute module. It should always be 'null' for values
originating in relatively-imported modules.
This commit fixes a bug in the CommonJsReflectionHost where viaModule would
be reported even for relatively-imported values, which causes invalid import
statements to be generated during compilation.
A test is added to verify the correct behavior.
FW-1628 #resolve
PR Close#33192
This allows disabling parallelism in ngcc if desired, which is mainly useful
for debugging. The implementation creates the flag and passes its value to
mainNgcc.
No tests are added since the feature mainly exists already - ngcc supports
both parallel and serial execution. This commit only allows switching the
flag via the commandline.
PR Close#33192
Previously, when `ngcc` was reflecting on class members it did not
account for the fact that a member could be of the kind
`IndexSignature`. This can happen, for example, on abstract classes (as
is the case for [JsonCallbackContext][1]).
Trying to reflect on such members (and failing to recognize their kind),
resulted in warnings, such as:
```
Warning: Unknown member type: "[key: string]: (data: any) => void;
```
While these warnings are harmless, they can be confusing and worrisome
for users.
This commit avoids such warnings by detecting class members of the
`IndexSignature` kind and ignoring them.
[1]: https://github.com/angular/angular/blob/4659cc26e/packages/common/http/src/jsonp.ts#L39
PR Close#33198
Injector defs are not considered public API, so the property
that contains them should be prefixed with Angular's marker
for "private" ('ɵ') to discourage apps from relying on def
APIs directly.
This commit adds the prefix and shortens the name from
ngInjectorDef to inj. This is because property names
cannot be minified by Uglify without turning on property
mangling (which most apps have turned off) and are thus
size-sensitive.
PR Close#33151
Module defs are not considered public API, so the property
that contains them should be prefixed with Angular's marker
for "private" ('ɵ') to discourage apps from relying on def
APIs directly.
This commit adds the prefix and shortens the name from
ngModuleDef to mod. This is because property names
cannot be minified by Uglify without turning on property
mangling (which most apps have turned off) and are thus
size-sensitive.
PR Close#33142
Factory defs are not considered public API, so the property
that contains them should be prefixed with Angular's marker
for "private" ('ɵ') to discourage apps from relying on def
APIs directly.
This commit adds the prefix and shortens the name from
ngFactoryDef to fac. This is because property names
cannot be minified by Uglify without turning on property
mangling (which most apps have turned off) and are thus
size-sensitive.
Note that the other "defs" (ngPipeDef, etc) will be
prefixed and shortened in follow-up PRs, in an attempt to
limit how large and conflict-y this change is.
PR Close#33116
Previously, the list of missing dependencies was not explicitly joined,
which resulted in the default `,` joiner being used during
stringification.
This commit explicitly joins the missing dependency lines to avoid
unnecessary commas.
Before:
```
The target entry-point "some-entry-point" has missing dependencies:
- dependency 1
, - dependency 2
, - dependency 3
```
After:
```
The target entry-point "some-entry-point" has missing dependencies:
- dependency 1
- dependency 2
- dependency 3
```
PR Close#33139
Previously, the executable for the Angular Compatibility Compiler
(`ngcc`) was called `ivy-ngcc`. This would be confusing for users not
familiar with our internal terminology, especially given that we call it
`ngcc` in all our docs and presentations.
This commit renames the executable to `ngcc` and replaces `ivy-ngcc`
with a script that errors with an informative message (prompting the
user to use `ngcc` instead).
Jira issue: [FW-1624](https://angular-team.atlassian.net/browse/FW-1624)
PR Close#33140
Directive defs are not considered public API, so the property
that contains them should be prefixed with Angular's marker
for "private" ('ɵ') to discourage apps from relying on def
APIs directly.
This commit adds the prefix and shortens the name from
ngDirectiveDef to dir. This is because property names
cannot be minified by Uglify without turning on property
mangling (which most apps have turned off) and are thus
size-sensitive.
Note that the other "defs" (ngFactoryDef, etc) will be
prefixed and shortened in follow-up PRs, in an attempt to
limit how large and conflict-y this change is.
PR Close#33110
Component defs are not considered public API, so the property
that contains them should be prefixed with Angular's marker
for "private" ('ɵ') to discourage apps from relying on def
APIs directly.
This commit adds the prefix and shortens the name from
`ngComponentDef` to `cmp`. This is because property names
cannot be minified by Uglify without turning on property
mangling (which most apps have turned off) and are thus
size-sensitive.
Note that the other "defs" (ngDirectiveDef, etc) will be
prefixed and shortened in follow-up PRs, in an attempt to
limit how large and conflict-y this change is.
PR Close#33088
It is now possible to include a set of default ngcc configurations
that ship with ngcc out of the box. This allows ngcc to handle a
set of common packages, which are unlikely to be fixed, without
requiring the application developer to write their own configuration
for them.
Any packages that are configured at the package or project level
will override these default configurations. This allows a reasonable
level of control at the package and user level.
PR Close#33008
Currently Ivy stores the element attributes into an array above the component def and passes it into the relevant instructions, however the problem is that upon minification the array will get a unique name which won't compress very well. These changes move the attributes array into the component def and pass in the index into the instructions instead.
Before:
```
const _c0 = ['foo', 'bar'];
SomeComp.ngComponentDef = defineComponent({
template: function() {
element(0, 'div', _c0);
}
});
```
After:
```
SomeComp.ngComponentDef = defineComponent({
consts: [['foo', 'bar']],
template: function() {
element(0, 'div', 0);
}
});
```
A couple of cases that this PR doesn't handle:
* Template references are still in a separate array.
* i18n attributes are still in a separate array.
PR Close#32798
The `$localize` library uses a new message digest function for
computing message ids. This means that translations in legacy
translation files will no longer match the message ids in the code
and so will not be translated.
This commit adds the ability to specify the format of your legacy
translation files, so that the appropriate message id can be rendered
in the `$localize` tagged strings. This results in larger code size
and requires that all translations are in the legacy format.
Going forward the developer should migrate their translation files
to use the new message id format.
PR Close#32937
This PR updates Angular to compile with TypeScript 3.6 while retaining
compatibility with TS3.5. We achieve this by inserting several `as any`
casts for compatiblity around `ts.CompilerHost` APIs.
PR Close#32908
With #31953 we moved the factories for components, directives and pipes into a new field called `ngFactoryDef`, however I decided not to do it for injectables, because they needed some extra logic. These changes set up the `ngFactoryDef` for injectables as well.
For reference, the extra logic mentioned above is that for injectables we have two code paths:
1. For injectables that don't configure how they should be instantiated, we create a `factory` that proxies to `ngFactoryDef`:
```
// Source
@Injectable()
class Service {}
// Output
class Service {
static ngInjectableDef = defineInjectable({
factory: () => Service.ngFactoryFn(),
});
static ngFactoryFn: (t) => new (t || Service)();
}
```
2. For injectables that do configure how they're created, we keep the `ngFactoryDef` and generate the factory based on the metadata:
```
// Source
@Injectable({
useValue: DEFAULT_IMPL,
})
class Service {}
// Output
export class Service {
static ngInjectableDef = defineInjectable({
factory: () => DEFAULT_IMPL,
});
static ngFactoryFn: (t) => new (t || Service)();
}
```
PR Close#32433
Recently ng-packagr was updated to include a transform that used to be
done in tsickle (https://github.com/ng-packagr/ng-packagr/pull/1401),
where only constructor parameter decorators are emitted in tsickle's
format, not any of the other decorators.
ngcc used to extract decorators from only a single format, so once it
saw the `ctorParameters` static property it assumed the library is using
the tsickle format. Therefore, none of the `__decorate` calls were
considered. This resulted in missing decorator information, preventing
proper processing of a package.
This commit changes how decorators are extracted by always looking at
both the static properties and the `__decorate` calls, merging these
sources appropriately.
Resolves FW-1573
PR Close#32901
ngcc may need to insert public exports into the bundle's source as well
as to the entry-point's declaration file, as the Ivy compiler may need
to create import statements to internal library types. The way ngcc
knows which exports to add is through the references registry, to which
references to things that require a public export are added by the
various analysis steps that are executed.
One of these analysis steps is the augmentation of declaration files
where functions that return `ModuleWithProviders` are updated so that a
generic type argument is added that corresponds with the `NgModule` that
is actually imported. This type has to be publicly exported, so the
analyzer step has to add the module type to the references registry.
A problem occurs when `ModuleWithProviders` already has a generic type
argument, in which case no update of the declaration file is necessary.
This may happen when 1) ngcc is processing additional bundle formats, so
that the declaration file has already been updated while processing the
first bundle format, or 2) when a package is processed which already
contains the generic type in its source. In both scenarios it may occur
that the referenced `NgModule` type does not yet have a public export,
so it is crucial that a reference to the type is added to the
references registry, which ngcc failed to do.
This commit fixes the issue by always adding the referenced `NgModule`
type to the references registry, so that a public export will always be
created if necessary.
Resolves FW-1575
PR Close#32902
Previously we were looking for a global factory call that looks like:
```ts
(factory((global.ng = global.ng || {}, global.ng.common = {}), global.ng.core))"
```
but in some cases it looks like:
```ts
(global = global || self, factory((global.ng = global.ng || {}, global.ng.common = {}), global.ng.core))"
```
Note the `global = global || self` at the start of the statement.
This commit makes the test when finding the global factory
function call resilient to being in a comma list.
PR Close#32709
In ngcc's reflection host for UMD and CommonJS bundles, custom logic is
present to resolve import details of an identifier. However, this custom
logic is unable to resolve an import for an identifier inside of
declaration files, as such files use the regular ESM import syntax.
As a consequence of this limitation, ngtsc is unable to resolve
`ModuleWithProviders` imports that are declared in an external library.
In that situation, ngtsc determines the type of the actual `NgModule`
that is imported, by looking in the library's declaration files for the
generic type argument on `ModuleWithProviders`. In this process, ngtsc
resolves the import for the `ModuleWithProviders` identifier to verify
that it is indeed the `ModuleWithProviders` type from `@angular/core`.
So, when the UMD reflection host was in use this resolution would fail,
therefore no `NgModule` type could be detected.
This commit fixes the bug by using the regular import resolution logic
in addition to the custom resolution logic that is required for UMD
and CommonJS bundles.
Fixes#31791
PR Close#32619
In ESM2015 bundles, a class with decorators may be emitted as follows:
```javascript
var MyClass_1;
let MyClass = MyClass_1 = class MyClass {};
MyClass.decorators = [/* here be decorators */];
```
Such a class has two declarations: the publicly visible `let MyClass`
and the implementation `class MyClass {}` node. In #32539 a refactoring
took place to handle such classes more consistently, however the logic
to find static properties was mistakenly kept identical to its broken
state before the refactor, by looking for static properties on the
implementation symbol (the one for `class MyClass {}`) whereas the
static properties need to be obtained from the symbol corresponding with
the `let MyClass` declaration, as that is where the `decorators`
property is assigned to in the example above.
This commit fixes the behavior by looking for static properties on the
public declaration symbol. This fixes an issue where decorators were not
found for classes that do in fact have decorators, therefore preventing
the classes from being compiled for Ivy.
Fixes#31791
PR Close#32619
In ngcc's reflection hosts for compiled JS bundles, such as ESM2015,
special care needs to be taken for classes as there may be an outer
declaration (referred to as "declaration") and an inner declaration
(referred to as "implementation") for a given class. Therefore, there
will also be two `ts.Symbol`s bound per class, and ngcc needs to switch
between those declarations and symbols depending on where certain
information can be found.
Prior to this commit, the `NgccReflectionHost` interface had methods
`getClassSymbol` and `findClassSymbols` that would return a `ts.Symbol`.
These class symbols would be used to kick off compilation of components
using ngtsc, so it is important for these symbols to correspond with the
publicly visible outer declaration of the class. However, the ESM2015
reflection host used to return the `ts.Symbol` for the inner
declaration, if the class was declared as follows:
```javascript
var MyClass = class MyClass {};
```
For the above code, `Esm2015ReflectionHost.getClassSymbol` would return
the `ts.Symbol` corresponding with the `class MyClass {}` declaration,
whereas it should have corresponded with the `var MyClass` declaration.
As a consequence, no `NgModule` could be resolved for the component, so
no components/directives would be in scope for the component. This
resulted in errors during runtime.
This commit resolves the issue by introducing a `NgccClassSymbol` that
contains references to both the outer and inner `ts.Symbol`, instead of
just a single `ts.Symbol`. This avoids the unclarity of whether a
`ts.Symbol` corresponds with the outer or inner declaration.
More details can be found here: https://hackmd.io/7nkgWOFWQlSRAuIW_8KPPwFixes#32078
Closes FW-1507
PR Close#32539
This gives an overview of how much time is spent in each operation/phase
and makes it easy to do rough comparisons of how different
configurations or changes affect performance.
PR Close#32427
`ngcc` supports both synchronous and asynchronous execution. The default
mode when using `ngcc` programmatically (which is how `@angular/cli` is
using it) is synchronous. When running `ngcc` from the command line
(i.e. via the `ivy-ngcc` script), it runs in async mode.
Previously, the work would be executed in the same way in both modes.
This commit improves the performance of `ngcc` in async mode by
processing tasks in parallel on multiple processes. It uses the Node.js
built-in [`cluster` module](https://nodejs.org/api/cluster.html) to
launch a cluster of Node.js processes and take advantage of multi-core
systems.
Preliminary comparisons indicate a 1.8x to 2.6x speed improvement when
processing the angular.io app (apparently depending on the OS, number of
available cores, system load, etc.). Further investigation is needed to
better understand these numbers and identify potential areas of
improvement.
Inspired by/Based on @alxhub's prototype: alxhub/angular@cb631bdb1
Original design doc: https://hackmd.io/uYG9CJrFQZ-6FtKqpnYJAA?view
Jira issue: [FW-1460](https://angular-team.atlassian.net/browse/FW-1460)
PR Close#32427
This commit adds a new `TaskQueue` implementation that supports
executing multiple tasks in parallel (while respecting interdependencies
between them).
This new implementation is currently not used, thus the behavior of
`ngcc` is not affected by this change. The parallel `TaskQueue` will be
used in a subsequent commit that will introduce parallel task execution.
PR Close#32427
This change does not alter the current behavior, but makes it easier to
introduce `TaskQueue`s implementing different task selection algorithms,
for example to support executing multiple tasks in parallel (while
respecting interdependencies between them).
Inspired by/Based on @alxhub's prototype: alxhub/angular@cb631bdb1
PR Close#32427
Previously, `ngcc` needed to store some metadata related to the
processing of each entry-point. This metadata was stored in a `Map`, in
the form of `EntryPointProcessingMetadata` and passed around as needed.
After some recent refactorings, it turns out that this metadata (with
its only remaining property, `hasProcessedTypings`) was no longer used,
because the relevant information was extracted from other sources (such
as the `processDts` property on `Task`s).
This commit cleans up the code by removing the unused code and types.
PR Close#32427
In the past, a task's processability didn't use to be known in advance.
It was possible that a task would be created and added to the queue
during the analysis phase and then later (during the compilation phase)
it would be found out that the task (i.e. the associated format
property) was not processable.
As a result, certain checks had to be delayed, until a task's processing
had started or even until all tasks had been processed. Examples of
checks that had to be delayed are:
- Whether a task can be skipped due to `compileAllFormats: false`.
- Whether there were entry-points for which no format at all was
successfully processed.
It turns out that (as made clear by the refactoring in 9537b2ff8), once
a task starts being processed it is expected to either complete
successfully (with the associated format being processed) or throw an
error (in which case the process will exit). In other words, a task's
processability is known in advance.
This commit takes advantage of this fact by moving certain checks
earlier in the process (e.g. in the analysis phase instead of the
compilation phase), which in turn allows avoiding some unnecessary work.
More specifically:
- When `compileAllFormats` is `false`, tasks are created _only_ for the
first suitable format property for each entry-point, since the rest of
the tasks would have been skipped during the compilation phase anyway.
This has the following advantages:
1. It avoids the slight overhead of generating extraneous tasks and
then starting to process them (before realizing they should be
skipped).
2. In a potential future parallel execution mode, unnecessary tasks
might start being processed at the same time as the first (useful)
task, even if their output would be later discarded, wasting
resources. Alternatively, extra logic would have to be added to
prevent this from happening. The change in this commit avoids these
issues.
- When an entry-point is not processable, an error will be thrown
upfront without having to wait for other tasks to be processed before
failing.
PR Close#32427
Previously, `ngcc`'s programmatic API would run and complete
synchronously. This was necessary for specific usecases (such as how the
`@angular/cli` invokes `ngcc` as part of the TypeScript module
resolution process), but not for others (e.g. running `ivy-ngcc` as a
`postinstall` script).
This commit adds a new option (`async`) that enables turning on
asynchronous execution. I.e. it signals that the caller is OK with the
function call to complete asynchronously, which allows `ngcc` to
potentially run in a more efficient mode.
Currently, there is no difference in the way tasks are executed in sync
vs async mode, but this change sets the ground for adding new execution
options (that require asynchronous operation), such as processing tasks
in parallel on multiple processes.
NOTE:
When using the programmatic API, the default value for `async` is
`false`, thus retaining backwards compatibility.
When running `ngcc` from the command line (i.e. via the `ivy-ngcc`
script), it runs in async mode (to be able to take advantage of future
optimizations), but that is transparent to the caller.
PR Close#32427
This change does not alter the current behavior, but makes it easier to
introduce new types of `Executors` , for example to do the required work
in parallel (on multiple processes).
Inspired by/Based on @alxhub's prototype: alxhub/angular@cb631bdb1
PR Close#32427
To persist some of its state, `ngcc` needs to update `package.json`
files (both in memory and on disk).
This refactoring abstracts these operations behind the
`PackageJsonUpdater` interface, making it easier to orchestrate them
from different contexts (e.g. when running tasks in parallel on multiple
processes).
Inspired by/Based on @alxhub's prototype: alxhub/angular@cb631bdb1
PR Close#32427
In order to prevent `ngcc`'d packages (e.g. libraries) from getting
accidentally published, `ngcc` overwrites the `prepublishOnly` npm
script to log a warning and exit with an error. In case we want to
restore the original script (e.g. "undo" `ngcc` processing), we keep a
backup of the original `prepublishOnly` script.
Previously, running `ngcc` a second time (e.g. for a different format)
would create a backup of the overwritten `prepublishOnly` script (if
there was originally no `prepublishOnly` script). As a result, if we
ever tried to "undo" `ngcc` processing and restore the original
`prepublishOnly` script, the error-throwing script would be restored
instead.
This commit fixes it by ensuring that we only back up a `prepublishOnly`
script, iff it is not the one we created ourselves (i.e. the
error-throwing one).
PR Close#32427
Previously, any diagnostics reported during the compilation of an
entry-point would not be shown to the user, but either be ignored or
cause a hard crash in case of a `FatalDiagnosticError`. This is
unfortunate, as such error instances contain information on which code
was responsible for producing the error, whereas only its error message
would not. Therefore, it was quite hard to determine where the error
originates from.
This commit introduces behavior to deal with error diagnostics in a more
graceful way. Such diagnostics will still cause the compilation to fail,
however the error message now contains formatted diagnostics.
Closes#31977
Resolves FW-1374
PR Close#31996
If a project has nested projects that contain node_modules folders
that get processed by ngcc, it can be confusing when the ngcc
version changes since the error message is very generic:
```
The ngcc compiler has changed since the last ngcc build.
Please completely remove `node_modules` and try again.
```
This commit augments the error message with the path of
the entry-point that failed so that it is more obvious which
node_modules folder to remove.
BREAKING CHANGE:
This commit removes the public export of `hasBeenProcessed()`.
This was exported to be availble to the CLI integration but was never
used. The change to the function signature is a breaking change in itself
so we remove the function altogether to simplify and lower the public
API surface going forward.
PR Close#32396
Reworks the compiler to output the factories for directives, components and pipes under a new static field called `ngFactoryFn`, instead of the usual `factory` property in their respective defs. This should eventually allow us to inject any kind of decorated class (e.g. a pipe).
**Note:** these changes are the first part of the refactor and they don't include injectables. I decided to leave injectables for a follow-up PR, because there's some more cases we need to handle when it comes to their factories. Furthermore, directives, components and pipes make up most of the compiler output tests that need to be refactored and it'll make follow-up PRs easier to review if the tests are cleaned up now.
This is part of the larger refactor for FW-1468.
PR Close#31953
When ngcc is called for a specific entry-point, it has to determine
which dependencies to transitively process. To accomplish this, ngcc
traverses the full import graph of the entry-points it encounters, for
which it uses a dependency host to find all module imports. Since
imports look different in the various bundle formats ngcc supports, a
specific dependency host is used depending on the information provided
in an entry-points `package.json` file. If there's not enough
information in the `package.json` file for ngcc to be able to determine
which dependency host to use, ngcc would fail with an error.
If, however, the entry-point is not compiled by Angular, it is not
necessary to process any of its dependencies. None of them can have
been compiled by Angular so ngcc does not need to know about them.
Therefore, this commit changes the behavior to avoid recursing into
dependencies of entry-points that are not compiled by Angular.
In particular, this fixes an issue for packages that have dependencies
on the `date-fns` package. This package has various secondary
entry-points that have a `package.json` file only containing a `typings`
field, without providing additional fields for ngcc to know which
dependency host to use. By not needing a dependency host at all, the
error is avoided.
Fixes#32302
PR Close#32303
Previously, `ngcc` assumed that if a format property was defined in
`package.json` it would point to a valid format-path (i.e. a file that
is an entry-point for a specific format). This is generally the case,
except if a format property is set to a non-string value (such as
`package.json`) - either directly in the `package.json` (which is unusual)
or in ngcc.config.js (which is a valid usecase, when one wants a
format property to be ignored by `ngcc`).
For example, the following config file would cause `ngcc` to throw:
```
module.exports = {
packages: {
'test-package': {
entryPoints: {
'.': {
override: {
fesm2015: undefined,
},
},
},
},
},
};
```
This commit fixes it by ensuring that only format properties whose value
is a string are considered by `ngcc`.
For reference, this regression was introduced in #32052.
Fixes#32188
PR Close#32205
During the dependency analysis phase of ngcc, imports are resolved to
files on disk according to certain module resolution rules. Since module
specifiers are typically missing extensions, or can refer to index.js
barrel files within a directory, the module resolver attempts several
postfixes when searching for a module import on disk. Module specifiers
that already include an extension, however, would fail to be resolved as
ngcc's module resolver failed to check the location on disk without
adding any postfixes.
Closes#32097
PR Close#32181
During the recursive processing of dependencies, ngcc resolves the
requested file to an actual location on disk, by testing various
extensions. For recursive calls however, the path is known to have been
resolved in the module resolver. Therefore, it is safe to move the path
resolution to the initial caller into the recursive process.
Note that this is not expected to improve the performance of ngcc, as
the call to `resolveFileWithPostfixes` is known to succeed immediately,
as the provided path is known to exist without needing to add any
postfixes. Furthermore, the FileSystem caches whether files exist, so
the additional check that we used to do was cheap.
PR Close#32181
ngcc needs to solve a unique problem when compiling typings for an
entrypoint: it must resolve a declaration within a .js file to its
representation in a .d.ts file. Since such .d.ts files can be used in deep
imports without ever being referenced from the "root" .d.ts, it's not enough
to simply match exported types to the root .d.ts. ngcc must build an index
of all .d.ts files.
Previously, this operation had a bug: it scanned all .d.ts files in the
.d.ts program, not only those within the package. Thus, if a class in the
program happened to share a name with a class exported from a dependency's
.d.ts, ngcc might accidentally modify the wrong .d.ts file, causing a
variety of issues downstream.
To fix this issue, ngcc's .d.ts scanner now limits the .d.ts files it
indexes to only those declared in the current package.
PR Close#32129
One of the compiler's tasks is to enumerate the exports of a given ES
module. This can happen for example to resolve `foo.bar` where `foo` is a
namespace import:
```typescript
import * as foo from './foo';
@NgModule({
directives: [foo.DIRECTIVES],
})
```
In this case, the compiler must enumerate the exports of `foo.ts` in order
to evaluate the expression `foo.DIRECTIVES`.
When this operation occurs under ngcc, it must deal with the different
module formats and types of exports that occur. In commonjs code, a problem
arises when certain exports are downleveled.
```typescript
export const DIRECTIVES = [
FooDir,
BarDir,
];
```
can be downleveled to:
```javascript
exports.DIRECTIVES = [
FooDir,
BarDir,
```
Previously, ngtsc and ngcc expected that any export would have an associated
`ts.Declaration` node. `export class`, `export function`, etc. all retain
`ts.Declaration`s even when downleveled. But the `export const` construct
above does not. Therefore, ngcc would not detect `DIRECTIVES` as an export
of `foo.ts`, and the evaluation of `foo.DIRECTIVES` would therefore fail.
To solve this problem, the core concept of an exported `Declaration`
according to the `ReflectionHost` API is split into a `ConcreteDeclaration`
which has a `ts.Declaration`, and an `InlineDeclaration` which instead has
a `ts.Expression`. Differentiating between these allows ngcc to return an
`InlineDeclaration` for `DIRECTIVES` and correctly keep track of this
export.
PR Close#32129
Previously if only a component template changed then we would know to
rebuild its component source file. But the compilation was incorrect if the
component was part of an NgModule, since we were not capturing the
compilation scope information that had a been acquired from the NgModule
and was not being regenerated since we were not needing to recompile
the NgModule.
Now we register compilation scope information for each component, via the
`ComponentScopeRegistry` interface, so that it is available for incremental
compilation.
The `ComponentDecoratorHandler` now reads the compilation scope from a
`ComponentScopeReader` interface which is implemented as a compound
reader composed of the original `LocalModuleScopeRegistry` and the
`IncrementalState`.
Fixes#31654
PR Close#31932
Publishing of NGCC packages should not be allowed. It is easy for a user to publish an NGCC'd version of a library they have workspace libraries which are being used in a workspace application.
If a users builds a library and afterwards the application, the library will be transformed with NGCC and since NGCC taints the distributed files that should be published.
With this change we use the npm/yarn `prepublishOnly` hook to display and error and abort the process with a non zero error code when a user tries to publish an NGCC version of the package.
More info: https://docs.npmjs.com/misc/scripts
PR Close#32031
Previously, when run with `createNewEntryPointFormats: true`, `ngcc`
would only update `package.json` with the new entry-point for the first
format property that mapped to a format-path. Subsequent properties
mapping to the same format-path would be detected as processed and not
have their new entry-point format recorded in `package.json`.
This commit fixes this by ensuring `package.json` is updated for all
matching format properties, when writing an `EntryPointBundle`.
PR Close#32052
Remove the `formatProperty` property from the `EntryPointBundle`
interface, because the property is not directly related to that type.
It was only used in one place, when calling `fileWriter.writeBundle()`,
but we can pass `formatProperty` directrly to `writeBundle()`.
PR Close#32052
This refactoring more clearly separates the different phases of the work
performed by `ngcc`, setting the ground for being able to run each phase
independently in the future and improve performance via parallelization.
Inspired by/Based on @alxhub's prototype: alxhub/angular@cb631bdb1
PR Close#32052
This change basically moves some checks to happen up front and ensures
we don't try to process any more properties than we absolutely need.
(The properties would not be processed before either, but we would
consider them, before finding out that they have already been processed
or that they do not exist in the entry-point's `package.json`.)
This change should make no difference in the work done by `ngcc`, but it
transforms the code in a way that makes the actual work known earlier,
thus making it easier to parallelize the processing of each property in
the future.
PR Close#32052
In commit 7b55ba58b (part of PR #29092), the implementation of
`makeEntryPointBundle()` was changed such that it now always return
`EntryPointBundle` (and not `null`).
However, the return type was not updated and as result we continued to
unnecessarily handle `null` as a potential return value in some places.
This commit fixes the return type to reflect the implementation and
removes the redundant code that was dealing with `null`.
PR Close#32052
ngcc analyzes the dependency structure of the entrypoints it needs to
process, as the compilation of entrypoints is ordering sensitive: any
dependent upon entrypoint must be compiled before its dependees. As part
of the analysis of the dependency graph, it is detected when a
dependency of entrypoint is not installed, in which case that entrypoint
will be marked as ignored.
For libraries that work with Angular Universal to run in NodeJS, imports
into builtin NodeJS modules can be present. ngcc's dependency analyzer
can only resolve imports within the TypeScript compilation, which
builtin modules are not part of. Therefore, such imports would
erroneously cause the entrypoint to become ignored.
This commit fixes the problem by taking the NodeJS builtins into account
when dealing with missing imports.
Fixes#31522
PR Close#31872
ngcc analyzes the dependency structure of the entrypoints it needs to
process, as the compilation of entrypoints is ordering sensitive: any
dependent upon entrypoint must be compiled before its dependees. As part
of the analysis of the dependency graph, it is detected when a
dependency of entrypoint is not installed, in which case that entrypoint
will be marked as ignored.
When a target entrypoint to compile is provided, it could occur that
given target is considered ignored because one of its dependencies might
be missing. This situation was not dealt with currently, instead
resulting in a crash of ngcc.
This commit prevents the crash by taking the above scenario into account.
PR Close#31872
Previously, `ngcc` would avoid processing a `formatPath` that a property
in `package.json` mapped to, if either the _property_ was marked as
processed or the `formatPath` (i.e. the file(s)) was processed in the
same `ngcc` run (since the `compiledFormats` set was not persisted
across runs).
This could lead in a situation where a `formatPath` would be compiled
twice (if for example properties `a` and `b` both mapped to the same
`formatPath` and one would run `ngcc` for property `a` and then `b`).
This commit fixes it by ensuring that as soon as a `formatPath` has been
processed all corresponding properties are marked as processed (which
persists across `ngcc` runs).
PR Close#32003
Previously, when `ngcc` was called with `compileAllFormats === false`
(i.e. how `@angular/cli` calls it), it would not attempt to process
more properties, once the first was successfully processed. However, it
_would_ continue looping over them and perform some unnecessary
operations, such as:
- Determining the format each property maps to (which can be an
expensive operation for some properties mapping to either UMD or
CommonJS).
- Checking whether each property has been processed (which involves
checking whether any property has been processed with a different
version of `ngcc` each time).
- Potentially marking properties as processed (which involves a
file-write operation).
This commit avoids the unnecessary operations by entirely skipping
subsequent properties, once the first one has been successfully
processed. While this theoretically improves performance, it is not
expected to have any noticeable impact in practice, since the list of
`propertiesToConsider` is typically small and the most expensive
operation (marking a property as processed) has low likelihood of
happening (plus these operations are a tiny fraction of `ngcc`'s work).
PR Close#32003
Previously, when `ngcc` needed to mark multiple properties as processed
(e.g. a processed format property and `typings` or all supported
properties for a non-Angular entry-point), it would update each one
separately and write the file to disk multiple times.
This commit changes this, so that multiple properties can be updated at
once with one file-write operation. While this theoretically improves
performance (reducing the I/O operations), it is not expected to have
any noticeable impact in practice, since these operations are a tiny
fraction of `ngcc`'s work.
This change will be useful for a subsequent change to mark all
properties that map to the same `formatPath` as processed, once it is
processed the first time.
PR Close#32003
This commit changes the emit order of ngcc when a class has multiple static
fields being assigned. Previously, ngcc would emit each static field
followed immediately by any extra statements specified for that field. This
causes issues with downstream tooling such as build optimizer, which expects
all of the static fields for a class to be grouped together. ngtsc already
groups static fields and additional statements. This commit changes ngcc's
ordering to match.
PR Close#31933
In #31426 a fix was implemented to render namespaced decorator imports
correctly, however it turns out that the fix only worked when decorator
information was extracted from static properties, not when using
`__decorate` calls.
This commit fixes the issue by creating the decorator metadata with the
full decorator expression, instead of only its name.
Closes#31394
PR Close#31614
An identifier may become repeated when bundling multiple source files
into a single bundle, so bundlers have a strategy of suffixing non-unique
identifiers with a suffix like $2. Since ngcc operates on such bundles,
it needs to process potentially suffixed identifiers in their canonical
form without the suffix. The "ngx-pagination" package was previously not
compiled fully, as most decorators were not recognized.
This commit ensures that identifiers are first canonicalized by removing
the suffix, such that they are properly recognized and processed by ngcc.
Fixes#31540
PR Close#31614
Any decorator information present in TypeScript is emitted into the
generated JavaScript sources by means of `__decorate` call. This call
contains both the decorators as they existed in the original source
code, together with calls to `tslib` helpers that convey additional
information on e.g. type information and parameter decorators. These
different kinds of decorator calls were not previously distinguished on
their own, but instead all treated as `Decorator` by themselves. The
"decorators" that were actually `tslib` helper calls were conveniently
filtered out because they were not imported from `@angular/core`, a
characteristic that ngcc uses to drop certain decorators.
Note that this posed an inconsistency in ngcc when it processes
`@angular/core`'s UMD bundle, as the `tslib` helper functions have been
inlined in said bundle. Because of the inlining, the `tslib` helpers
appear to be from `@angular/core`, so ngcc would fail to drop those
apparent "decorators". This inconsistency does not currently cause any
issues, as ngtsc is specifically looking for decorators based on their
name and any remaining decorators are simply ignored.
This commit rewrites the decorator analysis of a class to occur all in a
single phase, instead of all throughout the `ReflectionHost`. This
allows to categorize the various decorate calls in a single sweep,
instead of constantly needing to filter out undesired decorate calls on
the go. As an added benefit, the computed decorator information is now
cached per class, such that subsequent reflection queries that need
decorator information can reuse the cached info.
PR Close#31614
Currently we always generate the `read` parameter for the view and content query instructions, however since most of the time the `read` parameter won't be set, we'll end up generating `null` which adds 5 bytes for each query when minified. These changes make it so that the `read` parameter only gets generated if it has a value.
PR Close#31667
The support for decorators that were imported via a namespace,
e.g. `import * as core from `@angular/core` was implemented
piecemeal. This meant that it was easy to miss situations where
a decorator identifier needed to be handled as a namepsaced
import rather than a direct import.
One such issue was that UMD processing of decorators was not
correct: the namespace was being omitted from references to
decorators.
Now the types have been modified to make it clear that a
`Decorator.identifier` could hold a namespaced identifier,
and the corresponding code that uses these types has been
fixed.
Fixes#31394
PR Close#31426
There are two places in the ngcc processing where it needs to load the
content of a file given by a general path:
* when determining the format of an entry-point.
To do this ngcc uses the value of the relevant property in package.json.
But in the case of `main` it must parse the contents of the entry-point
file to decide whether the format is UMD or CommonJS.
* when parsing the source files for dependencies to determine the order in
which compilation must occur. The relative imports in each file are parsed
and followed recursively, looking for external imports.
Previously, we naively assumed that the path would match the file name exactly.
But actually we must consider the standard module resolution conventions.
E.g. the extension (.js) may be missing, or the path may refer to a directory
containing an index.js file.
This commit fixes both places.
This commit now requires the `DependencyHost` instances to check
the existence of more files than before (at worst all the different possible
post-fixes). This should not create a significant performance reduction for
ngcc. Since the results of the checks will be cached, and similar work is
done inside the TS compiler, so what we lose in doing it here, is saved later
in the processing. The main performance loss would be where there are lots
of files that need to be parsed for dependencies that do not end up being
processed by TS. But compared to the main ngcc processing this dependency
parsing is a small proportion of the work done and so should not impact
much on the overall performance of ngcc.
// FW-1444
PR Close#31509
When determining if a `main` path points to a UMD or CommonJS
format, the contents of the file need to be loaded and parsed.
Previously, it was assumed that the path referred to the exact filename,
but did not account for normal module resolution semantics, where the
path may be missing an extension or refer to a directory containing an
`index.js` file.
// FW-1444
PR Close#31509
Paths can be mapped directly to files, which was not being taken
into account when computing `basePaths` for the `EntryPointFinder`s.
Now if a `pathMapping` pattern does not exist or is a file, then we try
the containing folder instead.
Fixes#31424
PR Close#30525
Previously, ngcc had to walk the entire `node_modules` tree looking for
entry-points, even if it only needed to process a single target entry-point
and its dependencies.
This added up to a few seconds to each execution of ngcc, which is noticeable
when being run via the CLI integration.
Now, if an entry-point target is provided, only that target and its entry-points
are considered rather than the whole folder tree.
PR Close#30525
When profiling ngcc it is notable that a large amount of time
is spent dealing with an exception that is thrown (and handled
internally by fs) when checking the existence of a file.
We check file existence a lot in both finding entry-points
and when TS is compiling code. This commit adds a simple
cached `FileSystem`, which wraps a real `FileSystem` delegate.
This will reduce the number of calls through to `fs.exists()` and
`fs.readFile()` on the delegate.
Initial benchmarks indicate that the cache is miss to hit ratio
for `exists()` is about 2:1, which means that we save about 1/3
of the calls to `fs.existsSync()`.
Note that this implements a "non-expiring" cache, so it is not suitable
for a long lived `FileSystem`, where files may be modified externally.
The cache will be updated if a file is changed or moved via
calls to `FileSystem` methods but it will not be aware of changes
to the files system from outside the `FileSystem` service.
For ngcc we must create a new `FileSystem` service
for each run of `mainNgcc` and ensure that all file operations
(including TS compilation) use the `FileSystem` service.
This ensures that it is very unlikely that a file will change
externally during `mainNgcc` processing.
PR Close#30525
This message gets called if a format has already been
compiled and we only want the first. So the message itself
is wrong but it is also not very useful anyway.
PR Close#30525
The ngcc tool adds namespaced imports to files when compiling. The ngtsc
tooling was not processing types correctly when they were imported via
such namespaces. For example:
```
export declare class SomeModule {
static withOptions(...): ModuleWithProviders<ɵngcc1.BaseModule>;
```
In this case the `BaseModule` was being incorrectly attributed to coming
from the current module rather than the imported module, represented by
`ɵngcc1`.
Fixes#31342
PR Close#31367
If a package delcares a class internally on an NgModule, ngcc
needs to be able to add a public export to this class's type.
Previously, if the typing file for the declared is not imported
from the typings entry-point file, then ngcc cannot find it.
Now we try to guess the .d.ts files from the equivalent .js
files.
PR Close#31411
Non-wild-card path-mappings were not being matched correctly.
Further path-mapped secondary entry-points that
were imported from the associated primary entry-point were not
being martched correctly.
Fixes#31274
PR Close#31450
Some formats of CommonJS put the decorator helper calls
outside the class IIFE as statements on the top level of the
source file.
This commit adds support to the `CommonJSReflectionHost`
for this format.
PR Close#31335
If an entry-point has missing dependencies then it cannot be
processed and is marked as invalid. Similarly, if an entry-point
has dependencies that have been marked as invalid then that
entry-point too is invalid. In all these cases, ngcc should quietly
ignore these entry-points and continue processing what it can.
Previously, if an entry-point had more than one entry-point that
was transitively invalid then ngcc was crashing rather than
ignoring the entry-point.
PR Close#31276
Our module resolution prefers `.js` files over `.d.ts` files because
occasionally libraries publish their typings in the same directory
structure as the compiled JS files, i.e. adjacent to each other.
The standard TS module resolution would pick up the typings
file and add that to the `ts.Program` and so they would be
ignored by our analyzers. But we need those JS files, if they
are part of the current package.
But this meant that we also bring in JS files from external
imports from outside the package, which is not desired.
This was happening for the `@fire/storage` enty-point
that was importing the `firebase/storage` path.
In this commit we solve this problem, for the case of imports
coming from a completely different package, by saying that any
file that is outside the package root directory must be an external
import and so we do not analyze those files.
This does not solve the potential problem of imports between
secondary entry-points within a package but so far that does
not appear to be a problem.
PR Close#30591
Rather than passing a number of individual arguments, we can
just pass an `EntryPointBundle`, which already contains them.
This is also a precursor to using more of the properties in the bundle.
PR Close#30591
This will allow users of the `EntryPointBundle` to use some of the `EntryPoint`
properties without us having to pass them around one by one.
PR Close#30591
Previously we expected the constructor parameter `decorators`
property to be an array wrapped in a function. Now we also support
an array not wrapped in a function.
PR Close#30591
Some packages do not actually provide a `typings` field in their
package.json. But TypeScript naturally infers the typings file from
the location of the JavaScript source file.
This commit modifies ngcc to do a similar inference when finding
entry-points to process.
Fixes#28603 (FW-1299)
PR Close#30591
There are scenarios where it is not possible for ngcc to guess the format
or configuration of an entry-point just from the files on disk.
Such scenarios include:
1) Unwanted entry-points: A spurious package.json makes ngcc think
there is an entry-point when there should not be one.
2) Deep-import entry-points: some packages allow deep-imports but do not
provide package.json files to indicate to ngcc that the imported path is
actually an entry-point to be processed.
3) Invalid/missing package.json properties: For example, an entry-point
that does not provide a valid property to a required format.
The configuration is provided by one or more `ngcc.config.js` files:
* If placed at the root of the project, this file can provide configuration
for named packages (and their entry-points) that have been npm installed
into the project.
* If published as part of a package, the file can provide configuration
for entry-points of the package.
The configured of a package at the project level will override any
configuration provided by the package itself.
PR Close#30591
Previously each test relied on large shared mock file-systems, which
makes it difficult to reason about what is actually being tested.
This commit breaks up these big mock file-systems into smaller more
focused chunks.
PR Close#30591
To improve cross platform support, all file access (and path manipulation)
is now done through a well known interface (`FileSystem`).
For testing a number of `MockFileSystem` implementations are provided.
These provide an in-memory file-system which emulates operating systems
like OS/X, Unix and Windows.
The current file system is always available via the static method,
`FileSystem.getFileSystem()`. This is also used by a number of static
methods on `AbsoluteFsPath` and `PathSegment`, to avoid having to pass
`FileSystem` objects around all the time. The result of this is that one
must be careful to ensure that the file-system has been initialized before
using any of these static methods. To prevent this happening accidentally
the current file system always starts out as an instance of `InvalidFileSystem`,
which will throw an error if any of its methods are called.
You can set the current file-system by calling `FileSystem.setFileSystem()`.
During testing you can call the helper function `initMockFileSystem(os)`
which takes a string name of the OS to emulate, and will also monkey-patch
aspects of the TypeScript library to ensure that TS is also using the
current file-system.
Finally there is the `NgtscCompilerHost` to be used for any TypeScript
compilation, which uses a given file-system.
All tests that interact with the file-system should be tested against each
of the mock file-systems. A series of helpers have been provided to support
such tests:
* `runInEachFileSystem()` - wrap your tests in this helper to run all the
wrapped tests in each of the mock file-systems.
* `addTestFilesToFileSystem()` - use this to add files and their contents
to the mock file system for testing.
* `loadTestFilesFromDisk()` - use this to load a mirror image of files on
disk into the in-memory mock file-system.
* `loadFakeCore()` - use this to load a fake version of `@angular/core`
into the mock file-system.
All ngcc and ngtsc source and tests now use this virtual file-system setup.
PR Close#30921
Packages that have been compiled using an older version of TypeScript
can have their decorators at the top-level of the ES5 bundles, instead
of inside the IIFE that is emitted for the class. Before this change,
ngcc only took static property assignments inside the IIFE into account,
therefore missing the decorators that were assigned at the top-level.
This commit extends the ES5 host to look for static properties in two
places. Testcases for all bundle formats that contain ES5 have been added
to ensure that this works in the various flavours.
A patch is included to support UMD bundles. The UMD factory affects how
TypeScripts binds the static properties to symbols, see the docblock of
the patch function for more details.
PR Close#30795
The `EntryPointFinder` computes the base paths to consider
when searching for entry-points. When there are `pathMappings`
provided it works out the best top level base-paths that cover all
the potential mappings.
If this computed basePath happens to coincide with an entry-point
path itself then we were missing it.
Now we check for an entry-point even at the base-path itself.
Related to https://github.com/angular/angular-cli/pull/14755
PR Close#31027
9d9c9e43e5 has been created a few days ago
and wasn't rebased on top of recent changes that introduces a commonjs host.
This means that tests for the commonjs host haven't been updated to work with
the changes from from #30492 and now fail in `master`.
PR Close#30967
Currently undecorated classes are intentionally not processed
with ngcc. This is causing unexpected behavior because decorator
handlers such as `base_def.ts` are specifically interested in class
definitions without top-level decorators, so that the base definition
can be generated if there are Angular-specific class members.
In order to ensure that undecorated base-classes work as expected
with Ivy, we need to run the decorator handlers for all top-level
class declarations (not only for those with decorators). This is similar
to when `ngtsc` runs decorator handlers when analyzing source-files.
Resolves FW-1355. Fixes https://github.com/angular/components/issues/16178
PR Close#30821
When determining the module type of a bundle pointed to by the "main"
property, ngcc needs to read the bundle to figure out if it is CommonJS
or UMD format. However, when the "main" property does not exist ngcc
would crash while determining the path to the main bundle file.
This commit fixes the crash by checking if the "main" property is present
at all, before attempting to derive a full path to the bundle file.
Fixes#30916
Fixes FW-1369
PR Close#30950
The usage of array spread syntax in source code may be downleveled to a
call to TypeScript's `__spread` helper function from `tslib`, depending
on the options `downlevelIteration` and `emitHelpers`. This proves
problematic for ngcc when it is processing ES5 formats, as the static
evaluator won't be able to interpret those calls.
A custom foreign function resolver is not sufficient in this case, as
`tslib` may be emitted into the library code itself. In that case, a
helper function can be resolved to an actual function with body, such
that it won't be considered as foreign function. Instead, a reflection
host can now indicate that the definition of a function corresponds with
a certain TypeScript helper, such that it becomes statically evaluable
in ngtsc.
Resolves#30299
PR Close#30492
- Refactors compiler to stop generating `ɵɵselect(0)` instructions
- Alters template execution to always call the equivalent of `ɵɵselect(0)` before running a template in update mode
- Updates tests to not check for or call `ɵɵselect(0)`.
The goal here is to reduce the size of generated templates
PR Close#30830
When ngcc processes an entrypoint, it updates `package.json` with
metadata about the processed format. Previously, it overwrote the
`package.json` with the stringified JSON object without spaces. This
made the file difficult to read (for example when looking at the file
while debugging an ngcc failure).
This commit fixes it by using spaces in the new `package.json` content.
PR Close#30831
The AbsoluteModuleStrategy in ngtsc assumed that the source code is
formatted as TypeScript with regards to module exports.
In ngcc this is not always the case, so this commit changes
`AbsoluteModuleStrategy` so that it relies upon a `ReflectionHost` to
compute the exports of a module.
PR Close#30200
There is an encoding issue with using delta `Δ`, where the browser will attempt to detect the file encoding if the character set is not explicitly declared on a `<script/>` tag, and Chrome will find the `Δ` character and decide it is window-1252 encoding, which misinterprets the `Δ` character to be some other character that is not a valid JS identifier character
So back to the frog eyes we go.
```
__
/ɵɵ\
( -- ) - I am ineffable. I am forever.
_/ \_
/ \ / \
== == ==
```
PR Close#30546
Previously we were relying upon the `.get()` method to return `undefined`
but it is clearer and safer to always check with `.has()` first.
PR Close#25445
Previously the same `Renderer` was used to render typings (.d.ts)
files. But the new `UmdRenderer` is not able to render typings files
correctly.
This commit splits out the typings rendering from the src rendering.
To achieve this the previous renderers have been refactored from
sub-classes of the abstract `Renderer` class to classes that implement
the `RenderingFormatter` interface, which are then passed to the
`Renderer` and `DtsRenderer` to modify its rendering behaviour.
Along the way a few utility interfaces and classes have been moved
around and renamed for clarity.
PR Close#25445
Previously these fake files were full TypeScript source
files (`.ts`) but this is not necessary as we only need the
typings not the implementation.
PR Close#25445
Previously we were using an anonymous type `{specifier: string; qualifier: string;}`
throughout the code base. This commit gives this type a name and ensures it
is only defined in one place.
PR Close#25445
Preserve compatibility with rollup_bundle rule.
Add missing npm dependencies, which are now enforced by the strict_deps plugin in tsc_wrapped
PR Close#30370
At the moment the module resolver will end up in an infinite loop in Windows because we are assuming that the root directory is always `/` however in windows this can be any drive letter example `c:/` or `d:/` etc...
With this change we also resolve the drive letter in windows, when using `AbsoluteFsPath.from` for consistence so under `/foo` will be converted to `c:/foo` this is also needed because of relative paths with different drive letters.
PR Close#30297
If an entry-point has a missing dependency then all the entry-points
that would have pointed to that dependency are also removed from
the dependency graph.
Previously we were still processing the dependencies of an entry-point
even if it had already been removed from the graph because it depended
upon a missing dependency that had previously been removed due to another
entry-point depending upon it.
This caused the dependency processing to crash rather than gracefully
logging and handling the missing invalid entry-point.
Fixes#29624
PR Close#30270
Packages that do not follow APF may have the declaration files in the same
directory as one source format, typically ES5. This is problematic for ngcc,
as it needs to create a TypeScript program with all JavaScript sources of
an entry-point, whereas TypeScript's module resolution mechanism would have
resolved an internal module import to the external facing .d.ts declaration
file, instead of the JavaScript source file. This behavior results in the
program to be analysed being incomplete.
This commit introduces a custom compiler host that recognizes the above
scenario and rewires the resolution of a .d.ts declaration file to its
JavaScript counterpart, if applicable.
Fixes#29939
PR Close#30017
This commit introduces a new interface, which abstracts access
to the underlying `FileSystem`. There is initially one concrete
implementation, `NodeJsFileSystem`, which is simply wrapping the
`fs` library of NodeJs.
Going forward, we can provide a `MockFileSystem` for test, which
should allow us to stop using `mock-fs` for most of the unit tests.
We could also implement a `CachedFileSystem` that may improve the
performance of ngcc.
PR Close#29643
By passing a `pathMappings` configuration (a subset of the
`ts.CompilerOptions` interface), we can instuct ngcc to process
additional paths outside the `node_modules` folder.
PR Close#29643
When working out the dependencies between entry-points
ngcc must parse the import statements and then resolve the
import path to the actual file. This is complicated because module
resolution is not trivial.
Previously ngcc used the node.js `require.resolve`, with some
hacking to resolve modules. This change refactors the `DependencyHost`
to use a new custom `ModuleResolver`, which is optimized for this use
case.
Moreover, because we are in full control of the resolution,
we can support TS `paths` aliases, where not all imports come from
`node_modules`. This is the case in some CLI projects where there are
compiled libraries that are stored locally in a `dist` folder.
See //FW-1210.
PR Close#29643
Previously we completely ignored entry-points that had not been
compiled with Angular, since we do not need to compile them
with ngcc. But this makes it difficult to reason about dependencies
between entry-points that were compiled with Angular and those that
were not.
Now we do track these non-Angular compiled entry-points but they
are marked as `compiledByAngular: false`.
PR Close#29643
The test now attempts to compile an entry-point (@angular/common/http/testing)
that has a transient "private" dependency. A private dependency is one that is
only visible by looking at the compiled JS code, rather than the generated TS
typings files.
This proves that we can't rely on typings files alone for computing the
dependencies between entry-points.
PR Close#29643
The `Transformer` and `Renderer` classes do not
actually need a `sourcePath` value as by the time
they are doing their work we are only working directly
with full absolute paths.
PR Close#29643
Only the JS files that are actually part of the entry-point
should be copied to the new entry-point folder in the
`NewEntryPointFileWriter`.
Previously some typings and external JS files were
being copied which was messing up the node_modules
structure.
Fixes https://github.com/angular/angular-cli/issues/14193
PR Close#30085
Previously, ngcc would insert new imports at the beginning of the file, for
convenience. This is problematic for imports that have side-effects, as the
side-effects imposed by such imports may affect the behavior of subsequent
imports.
This commit teaches ngcc to insert imports after any existing imports. Special
care has been taken to ensure inserted constants will still follow after the
inserted imports.
Resolves FW-1271
PR Close#30029
Previously, all of a program's files would be copied into the __ivy_ngcc__
folder where ngcc then writes its modifications into. The set of source files
in a program however is much larger than the source files contained within
the entry-point of interest, so many more files were copied than necessary.
Even worse, it may occur that an unrelated file in the program would collide
with an already existing source file, resulting in incorrectly overwriting
a file with unrelated content. This behavior has actually been observed
with @angular/animations and @angular/platform-browser/animations, where
the former package would overwrite declaration files of the latter package.
This commit fixes the issue by only copying relevant source files when cloning
a bundle's content into __ivy_ngcc__.
Fixes#29960
PR Close#30020
Previously, Template.templateAttrs was introduced to capture attribute
bindings which originated from microsyntax (e.g. bindings in *ngFor="...").
This means that a Template node can have two different structures, depending
on whether it originated from microsyntax or from a literal <ng-template>.
In the literal case, the node behaves much like an Element node, it has
attributes, inputs, and outputs which determine which directives apply.
In the microsyntax case, though, only the templateAttrs should be used
to determine which directives apply.
Previously, both the t2_binder and the TemplateDefinitionBuilder were using
the wrong set of attributes to match directives - combining the attributes,
inputs, outputs, and templateAttrs of the Template node regardless of its
origin. In the TDB's case this wasn't a problem, since the TDB collects a
global Set of directives used in the template, so it didn't matter whether
the directive was also recognized on the <ng-template>. t2_binder's API
distinguishes between directives on specific nodes, though, so it's more
sensitive to mismatching.
In particular, this showed up as an assertion failure in template type-
checking in certain cases, when a directive was accidentally matched on
a microsyntax template element and also had a binding which referenced a
variable declared in the microsyntax. This resulted in the type-checker
attempting to generate a reference to a variable that didn't exist in that
scope.
The fix is to distinguish between the two cases and select the appropriate
set of attributes to match on accordingly.
Testing strategy: tested in the t2_binder tests.
PR Close#29698
Previously the template type-checking code only considered the metadata of
directive classes actually referenced in the template. If those directives
had base classes, any inputs/outputs/etc of the base classes were not
tracked when generating the TCB. This resulted in bindings to those inputs
being incorrectly attributed to the host component or element.
This commit uses the new metadata package to follow directive inheritance
chains and use the full metadata for a directive for TCB generation.
Testing strategy: Template type-checking tests included.
PR Close#29698
Previously, metadata registration (the recording of collected metadata
during analysis of directives, pipes, and NgModules) was only used to
produce the `LocalModuleScope`, and thus was handled by the
`LocalModuleScopeRegistry`.
However, the template type-checker also needs information about registered
directives, outside of the NgModule scope determinations. Rather than
reuse the scope registry for an unintended purpose, this commit introduces
new abstractions for metadata registration and lookups in a separate
'metadata' package, which the scope registry implements.
This paves the way for a future commit to make use of this metadata for the
template type-checking system.
Testing strategy: this commit is a refactoring which introduces no new
functionality, so existing tests are sufficient.
PR Close#29698
The `Δ` caused issue with other infrastructure, and we are temporarily
changing it to `ɵɵ`.
This commit also patches ts_api_guardian_test and AIO to understand `ɵɵ`.
PR Close#29850
Previously we always walked the whole folder tree looking for
entry-points before we tested whether a target package had been
processed already. This could take >10secs!
This commit does a quick check of the target package before doing
the full walk which brings down the execution time for ngcc in this
case dramatically.
```
$ time ./node_modules/.bin/ivy-ngcc -t @angular/common/http/testing
Compiling @angular/core : fesm2015 as esm2015
Compiling @angular/core : fesm5 as esm5
Compiling @angular/core : esm2015 as esm2015
Compiling @angular/core : esm5 as esm5
Compiling @angular/common/http : fesm2015 as esm2015
Compiling @angular/common/http : fesm5 as esm5
Compiling @angular/common/http : esm2015 as esm2015
Compiling @angular/common/http : esm5 as esm5
Compiling @angular/common/http/testing : fesm2015 as esm2015
Compiling @angular/common/http/testing : fesm5 as esm5
Compiling @angular/common/http/testing : esm2015 as esm2015
Compiling @angular/common/http/testing : esm5 as esm5
real 0m19.766s
user 0m28.533s
sys 0m2.262s
```
```
$ time ./node_modules/.bin/ivy-ngcc -t @angular/common/http/testing
The target entry-point has already been processed
real 0m0.666s
user 0m0.605s
sys 0m0.113s
```
PR Close#29740
Prior to this change, all module metadata would be included in the
`defineNgModule` call that is set as the `ngModuleDef` field of module
types. Part of the metadata is scope information like declarations,
imports and exports that is used for computing the transitive module
scope in JIT environments, preventing those references from being
tree-shaken for production builds.
This change moves the metadata for scope computations to a pure function
call that patches the scope references onto the module type. Because the
function is marked pure, it may be tree-shaken out during production builds
such that references to declarations and exports are dropped, which in turn
allows for tree-shaken any declaration that is not otherwise referenced.
Fixes#28077, FW-1035
PR Close#29598
With the new API, where you can choose to only process the first
matching format, it is possible to process an entry-point multiple
times, if you pass in a different format each time.
Previously, ngcc would always try to process the typings files for
the entry-point along with processing the first format of the current
execution of ngcc. But this meant that it would be trying to process
the typings a second time.
Now we only process the typings if they have not already been
processed as part of processing another format in another
even if it was in a different execution of ngcc.
PR Close#29657
In ES2015, classes could have been emitted as a variable declaration
initialized with a class expression. In certain situations, an intermediary
variable suffixed with `_1` is present such that the variable
declaration's initializer becomes a binary expression with its rhs being
the class expression, and its lhs being the identifier of the intermediate
variable. This structure was not recognized, resulting in such classes not
being considered as a class in `Esm2015ReflectionHost`.
As a consequence, the analysis of functions/methods that return a
`ModuleWithProviders` object did not take the methods of such classes into
account.
Another edge-case with such intermediate variable was that static
properties would not be considered as class members. A testcase was added
to prevent regressions.
Fixes#29078
PR Close#29119
This commit adds a `tracePerformance` option for tsconfig.json. When
specified, it causes a JSON file with timing information from the ngtsc
compiler to be emitted at the specified path.
This tracing system is used to instrument the analysis/emit phases of
compilation, and will be useful in debugging future integration work with
@angular/cli.
See ngtsc/perf/README.md for more details.
PR Close#29380
JoostK
George Kalpakas
Pete Bacon Darwin
Alan Agius
crisbeto
Alex Rickabaugh
Igor Minar
Kara Erickson
Gabriel Medeiros Coelho
Martin Probst
atscott
Elvis Begovic
Matias Niemelä
Paul Gschwendtner
Ben Lesh
Alex Eagle
Filipe Silva