In ES5 modules, the class declarations consist of an IIFE with inner
and outer declarations that represent the class. The `EsmReflectionHost`
has logic to ensure that `getDeclarationOfIdentifier()` always returns the
outer declaration.
Before this commit, if an identifier referred to an alias of the inner
declaration, then `getDeclarationOfIdentifier()` was failing to find
the outer declaration - instead returning the inner declaration.
Now the identifier is correctly resolved up to the outer declaration
as expected.
This should fix some of the failing 3rd party packages discussed in
https://github.com/angular/ngcc-validation/issues/57.
PR Close#33252
This commit fixes ngtsc's import generator to use the ReflectionHost when
looking through the exports of an ES module to find the export of a
particular declaration that's being imported. This is necessary because
some module formats like CommonJS have unusual export mechanics, and the
normal TypeScript ts.TypeChecker does not understand them.
This fixes an issue with ngcc + CommonJS where exports were not being
enumerated correctly.
FW-1630 #resolve
PR Close#33192
Normally, when ngcc encounters a package with missing dependencies while
attempting to determine a compilation ordering, it will ignore that package.
This commit adds a configuration for a flag to tell ngcc to compile the
package anyway, regardless of any missing dependencies.
FW-1931 #resolve
PR Close#33192
In the ReflectionHost API, a 'viaModule' indicates that a particular value
originated in another absolute module. It should always be 'null' for values
originating in relatively-imported modules.
This commit fixes a bug in the CommonJsReflectionHost where viaModule would
be reported even for relatively-imported values, which causes invalid import
statements to be generated during compilation.
A test is added to verify the correct behavior.
FW-1628 #resolve
PR Close#33192
This allows disabling parallelism in ngcc if desired, which is mainly useful
for debugging. The implementation creates the flag and passes its value to
mainNgcc.
No tests are added since the feature mainly exists already - ngcc supports
both parallel and serial execution. This commit only allows switching the
flag via the commandline.
PR Close#33192
Previously, when `ngcc` was reflecting on class members it did not
account for the fact that a member could be of the kind
`IndexSignature`. This can happen, for example, on abstract classes (as
is the case for [JsonCallbackContext][1]).
Trying to reflect on such members (and failing to recognize their kind),
resulted in warnings, such as:
```
Warning: Unknown member type: "[key: string]: (data: any) => void;
```
While these warnings are harmless, they can be confusing and worrisome
for users.
This commit avoids such warnings by detecting class members of the
`IndexSignature` kind and ignoring them.
[1]: https://github.com/angular/angular/blob/4659cc26e/packages/common/http/src/jsonp.ts#L39
PR Close#33198
Injector defs are not considered public API, so the property
that contains them should be prefixed with Angular's marker
for "private" ('ɵ') to discourage apps from relying on def
APIs directly.
This commit adds the prefix and shortens the name from
ngInjectorDef to inj. This is because property names
cannot be minified by Uglify without turning on property
mangling (which most apps have turned off) and are thus
size-sensitive.
PR Close#33151
Module defs are not considered public API, so the property
that contains them should be prefixed with Angular's marker
for "private" ('ɵ') to discourage apps from relying on def
APIs directly.
This commit adds the prefix and shortens the name from
ngModuleDef to mod. This is because property names
cannot be minified by Uglify without turning on property
mangling (which most apps have turned off) and are thus
size-sensitive.
PR Close#33142
Factory defs are not considered public API, so the property
that contains them should be prefixed with Angular's marker
for "private" ('ɵ') to discourage apps from relying on def
APIs directly.
This commit adds the prefix and shortens the name from
ngFactoryDef to fac. This is because property names
cannot be minified by Uglify without turning on property
mangling (which most apps have turned off) and are thus
size-sensitive.
Note that the other "defs" (ngPipeDef, etc) will be
prefixed and shortened in follow-up PRs, in an attempt to
limit how large and conflict-y this change is.
PR Close#33116
Previously, the list of missing dependencies was not explicitly joined,
which resulted in the default `,` joiner being used during
stringification.
This commit explicitly joins the missing dependency lines to avoid
unnecessary commas.
Before:
```
The target entry-point "some-entry-point" has missing dependencies:
- dependency 1
, - dependency 2
, - dependency 3
```
After:
```
The target entry-point "some-entry-point" has missing dependencies:
- dependency 1
- dependency 2
- dependency 3
```
PR Close#33139
Previously, the executable for the Angular Compatibility Compiler
(`ngcc`) was called `ivy-ngcc`. This would be confusing for users not
familiar with our internal terminology, especially given that we call it
`ngcc` in all our docs and presentations.
This commit renames the executable to `ngcc` and replaces `ivy-ngcc`
with a script that errors with an informative message (prompting the
user to use `ngcc` instead).
Jira issue: [FW-1624](https://angular-team.atlassian.net/browse/FW-1624)
PR Close#33140
Directive defs are not considered public API, so the property
that contains them should be prefixed with Angular's marker
for "private" ('ɵ') to discourage apps from relying on def
APIs directly.
This commit adds the prefix and shortens the name from
ngDirectiveDef to dir. This is because property names
cannot be minified by Uglify without turning on property
mangling (which most apps have turned off) and are thus
size-sensitive.
Note that the other "defs" (ngFactoryDef, etc) will be
prefixed and shortened in follow-up PRs, in an attempt to
limit how large and conflict-y this change is.
PR Close#33110
Component defs are not considered public API, so the property
that contains them should be prefixed with Angular's marker
for "private" ('ɵ') to discourage apps from relying on def
APIs directly.
This commit adds the prefix and shortens the name from
`ngComponentDef` to `cmp`. This is because property names
cannot be minified by Uglify without turning on property
mangling (which most apps have turned off) and are thus
size-sensitive.
Note that the other "defs" (ngDirectiveDef, etc) will be
prefixed and shortened in follow-up PRs, in an attempt to
limit how large and conflict-y this change is.
PR Close#33088
It is now possible to include a set of default ngcc configurations
that ship with ngcc out of the box. This allows ngcc to handle a
set of common packages, which are unlikely to be fixed, without
requiring the application developer to write their own configuration
for them.
Any packages that are configured at the package or project level
will override these default configurations. This allows a reasonable
level of control at the package and user level.
PR Close#33008
Currently Ivy stores the element attributes into an array above the component def and passes it into the relevant instructions, however the problem is that upon minification the array will get a unique name which won't compress very well. These changes move the attributes array into the component def and pass in the index into the instructions instead.
Before:
```
const _c0 = ['foo', 'bar'];
SomeComp.ngComponentDef = defineComponent({
template: function() {
element(0, 'div', _c0);
}
});
```
After:
```
SomeComp.ngComponentDef = defineComponent({
consts: [['foo', 'bar']],
template: function() {
element(0, 'div', 0);
}
});
```
A couple of cases that this PR doesn't handle:
* Template references are still in a separate array.
* i18n attributes are still in a separate array.
PR Close#32798
The `$localize` library uses a new message digest function for
computing message ids. This means that translations in legacy
translation files will no longer match the message ids in the code
and so will not be translated.
This commit adds the ability to specify the format of your legacy
translation files, so that the appropriate message id can be rendered
in the `$localize` tagged strings. This results in larger code size
and requires that all translations are in the legacy format.
Going forward the developer should migrate their translation files
to use the new message id format.
PR Close#32937
This PR updates Angular to compile with TypeScript 3.6 while retaining
compatibility with TS3.5. We achieve this by inserting several `as any`
casts for compatiblity around `ts.CompilerHost` APIs.
PR Close#32908
With #31953 we moved the factories for components, directives and pipes into a new field called `ngFactoryDef`, however I decided not to do it for injectables, because they needed some extra logic. These changes set up the `ngFactoryDef` for injectables as well.
For reference, the extra logic mentioned above is that for injectables we have two code paths:
1. For injectables that don't configure how they should be instantiated, we create a `factory` that proxies to `ngFactoryDef`:
```
// Source
@Injectable()
class Service {}
// Output
class Service {
static ngInjectableDef = defineInjectable({
factory: () => Service.ngFactoryFn(),
});
static ngFactoryFn: (t) => new (t || Service)();
}
```
2. For injectables that do configure how they're created, we keep the `ngFactoryDef` and generate the factory based on the metadata:
```
// Source
@Injectable({
useValue: DEFAULT_IMPL,
})
class Service {}
// Output
export class Service {
static ngInjectableDef = defineInjectable({
factory: () => DEFAULT_IMPL,
});
static ngFactoryFn: (t) => new (t || Service)();
}
```
PR Close#32433
Recently ng-packagr was updated to include a transform that used to be
done in tsickle (https://github.com/ng-packagr/ng-packagr/pull/1401),
where only constructor parameter decorators are emitted in tsickle's
format, not any of the other decorators.
ngcc used to extract decorators from only a single format, so once it
saw the `ctorParameters` static property it assumed the library is using
the tsickle format. Therefore, none of the `__decorate` calls were
considered. This resulted in missing decorator information, preventing
proper processing of a package.
This commit changes how decorators are extracted by always looking at
both the static properties and the `__decorate` calls, merging these
sources appropriately.
Resolves FW-1573
PR Close#32901
ngcc may need to insert public exports into the bundle's source as well
as to the entry-point's declaration file, as the Ivy compiler may need
to create import statements to internal library types. The way ngcc
knows which exports to add is through the references registry, to which
references to things that require a public export are added by the
various analysis steps that are executed.
One of these analysis steps is the augmentation of declaration files
where functions that return `ModuleWithProviders` are updated so that a
generic type argument is added that corresponds with the `NgModule` that
is actually imported. This type has to be publicly exported, so the
analyzer step has to add the module type to the references registry.
A problem occurs when `ModuleWithProviders` already has a generic type
argument, in which case no update of the declaration file is necessary.
This may happen when 1) ngcc is processing additional bundle formats, so
that the declaration file has already been updated while processing the
first bundle format, or 2) when a package is processed which already
contains the generic type in its source. In both scenarios it may occur
that the referenced `NgModule` type does not yet have a public export,
so it is crucial that a reference to the type is added to the
references registry, which ngcc failed to do.
This commit fixes the issue by always adding the referenced `NgModule`
type to the references registry, so that a public export will always be
created if necessary.
Resolves FW-1575
PR Close#32902
Previously we were looking for a global factory call that looks like:
```ts
(factory((global.ng = global.ng || {}, global.ng.common = {}), global.ng.core))"
```
but in some cases it looks like:
```ts
(global = global || self, factory((global.ng = global.ng || {}, global.ng.common = {}), global.ng.core))"
```
Note the `global = global || self` at the start of the statement.
This commit makes the test when finding the global factory
function call resilient to being in a comma list.
PR Close#32709
In ngcc's reflection host for UMD and CommonJS bundles, custom logic is
present to resolve import details of an identifier. However, this custom
logic is unable to resolve an import for an identifier inside of
declaration files, as such files use the regular ESM import syntax.
As a consequence of this limitation, ngtsc is unable to resolve
`ModuleWithProviders` imports that are declared in an external library.
In that situation, ngtsc determines the type of the actual `NgModule`
that is imported, by looking in the library's declaration files for the
generic type argument on `ModuleWithProviders`. In this process, ngtsc
resolves the import for the `ModuleWithProviders` identifier to verify
that it is indeed the `ModuleWithProviders` type from `@angular/core`.
So, when the UMD reflection host was in use this resolution would fail,
therefore no `NgModule` type could be detected.
This commit fixes the bug by using the regular import resolution logic
in addition to the custom resolution logic that is required for UMD
and CommonJS bundles.
Fixes#31791
PR Close#32619
In ESM2015 bundles, a class with decorators may be emitted as follows:
```javascript
var MyClass_1;
let MyClass = MyClass_1 = class MyClass {};
MyClass.decorators = [/* here be decorators */];
```
Such a class has two declarations: the publicly visible `let MyClass`
and the implementation `class MyClass {}` node. In #32539 a refactoring
took place to handle such classes more consistently, however the logic
to find static properties was mistakenly kept identical to its broken
state before the refactor, by looking for static properties on the
implementation symbol (the one for `class MyClass {}`) whereas the
static properties need to be obtained from the symbol corresponding with
the `let MyClass` declaration, as that is where the `decorators`
property is assigned to in the example above.
This commit fixes the behavior by looking for static properties on the
public declaration symbol. This fixes an issue where decorators were not
found for classes that do in fact have decorators, therefore preventing
the classes from being compiled for Ivy.
Fixes#31791
PR Close#32619
In ngcc's reflection hosts for compiled JS bundles, such as ESM2015,
special care needs to be taken for classes as there may be an outer
declaration (referred to as "declaration") and an inner declaration
(referred to as "implementation") for a given class. Therefore, there
will also be two `ts.Symbol`s bound per class, and ngcc needs to switch
between those declarations and symbols depending on where certain
information can be found.
Prior to this commit, the `NgccReflectionHost` interface had methods
`getClassSymbol` and `findClassSymbols` that would return a `ts.Symbol`.
These class symbols would be used to kick off compilation of components
using ngtsc, so it is important for these symbols to correspond with the
publicly visible outer declaration of the class. However, the ESM2015
reflection host used to return the `ts.Symbol` for the inner
declaration, if the class was declared as follows:
```javascript
var MyClass = class MyClass {};
```
For the above code, `Esm2015ReflectionHost.getClassSymbol` would return
the `ts.Symbol` corresponding with the `class MyClass {}` declaration,
whereas it should have corresponded with the `var MyClass` declaration.
As a consequence, no `NgModule` could be resolved for the component, so
no components/directives would be in scope for the component. This
resulted in errors during runtime.
This commit resolves the issue by introducing a `NgccClassSymbol` that
contains references to both the outer and inner `ts.Symbol`, instead of
just a single `ts.Symbol`. This avoids the unclarity of whether a
`ts.Symbol` corresponds with the outer or inner declaration.
More details can be found here: https://hackmd.io/7nkgWOFWQlSRAuIW_8KPPwFixes#32078
Closes FW-1507
PR Close#32539
This gives an overview of how much time is spent in each operation/phase
and makes it easy to do rough comparisons of how different
configurations or changes affect performance.
PR Close#32427
`ngcc` supports both synchronous and asynchronous execution. The default
mode when using `ngcc` programmatically (which is how `@angular/cli` is
using it) is synchronous. When running `ngcc` from the command line
(i.e. via the `ivy-ngcc` script), it runs in async mode.
Previously, the work would be executed in the same way in both modes.
This commit improves the performance of `ngcc` in async mode by
processing tasks in parallel on multiple processes. It uses the Node.js
built-in [`cluster` module](https://nodejs.org/api/cluster.html) to
launch a cluster of Node.js processes and take advantage of multi-core
systems.
Preliminary comparisons indicate a 1.8x to 2.6x speed improvement when
processing the angular.io app (apparently depending on the OS, number of
available cores, system load, etc.). Further investigation is needed to
better understand these numbers and identify potential areas of
improvement.
Inspired by/Based on @alxhub's prototype: alxhub/angular@cb631bdb1
Original design doc: https://hackmd.io/uYG9CJrFQZ-6FtKqpnYJAA?view
Jira issue: [FW-1460](https://angular-team.atlassian.net/browse/FW-1460)
PR Close#32427
This commit adds a new `TaskQueue` implementation that supports
executing multiple tasks in parallel (while respecting interdependencies
between them).
This new implementation is currently not used, thus the behavior of
`ngcc` is not affected by this change. The parallel `TaskQueue` will be
used in a subsequent commit that will introduce parallel task execution.
PR Close#32427
This change does not alter the current behavior, but makes it easier to
introduce `TaskQueue`s implementing different task selection algorithms,
for example to support executing multiple tasks in parallel (while
respecting interdependencies between them).
Inspired by/Based on @alxhub's prototype: alxhub/angular@cb631bdb1
PR Close#32427
Previously, `ngcc` needed to store some metadata related to the
processing of each entry-point. This metadata was stored in a `Map`, in
the form of `EntryPointProcessingMetadata` and passed around as needed.
After some recent refactorings, it turns out that this metadata (with
its only remaining property, `hasProcessedTypings`) was no longer used,
because the relevant information was extracted from other sources (such
as the `processDts` property on `Task`s).
This commit cleans up the code by removing the unused code and types.
PR Close#32427
In the past, a task's processability didn't use to be known in advance.
It was possible that a task would be created and added to the queue
during the analysis phase and then later (during the compilation phase)
it would be found out that the task (i.e. the associated format
property) was not processable.
As a result, certain checks had to be delayed, until a task's processing
had started or even until all tasks had been processed. Examples of
checks that had to be delayed are:
- Whether a task can be skipped due to `compileAllFormats: false`.
- Whether there were entry-points for which no format at all was
successfully processed.
It turns out that (as made clear by the refactoring in 9537b2ff8), once
a task starts being processed it is expected to either complete
successfully (with the associated format being processed) or throw an
error (in which case the process will exit). In other words, a task's
processability is known in advance.
This commit takes advantage of this fact by moving certain checks
earlier in the process (e.g. in the analysis phase instead of the
compilation phase), which in turn allows avoiding some unnecessary work.
More specifically:
- When `compileAllFormats` is `false`, tasks are created _only_ for the
first suitable format property for each entry-point, since the rest of
the tasks would have been skipped during the compilation phase anyway.
This has the following advantages:
1. It avoids the slight overhead of generating extraneous tasks and
then starting to process them (before realizing they should be
skipped).
2. In a potential future parallel execution mode, unnecessary tasks
might start being processed at the same time as the first (useful)
task, even if their output would be later discarded, wasting
resources. Alternatively, extra logic would have to be added to
prevent this from happening. The change in this commit avoids these
issues.
- When an entry-point is not processable, an error will be thrown
upfront without having to wait for other tasks to be processed before
failing.
PR Close#32427
Previously, `ngcc`'s programmatic API would run and complete
synchronously. This was necessary for specific usecases (such as how the
`@angular/cli` invokes `ngcc` as part of the TypeScript module
resolution process), but not for others (e.g. running `ivy-ngcc` as a
`postinstall` script).
This commit adds a new option (`async`) that enables turning on
asynchronous execution. I.e. it signals that the caller is OK with the
function call to complete asynchronously, which allows `ngcc` to
potentially run in a more efficient mode.
Currently, there is no difference in the way tasks are executed in sync
vs async mode, but this change sets the ground for adding new execution
options (that require asynchronous operation), such as processing tasks
in parallel on multiple processes.
NOTE:
When using the programmatic API, the default value for `async` is
`false`, thus retaining backwards compatibility.
When running `ngcc` from the command line (i.e. via the `ivy-ngcc`
script), it runs in async mode (to be able to take advantage of future
optimizations), but that is transparent to the caller.
PR Close#32427
This change does not alter the current behavior, but makes it easier to
introduce new types of `Executors` , for example to do the required work
in parallel (on multiple processes).
Inspired by/Based on @alxhub's prototype: alxhub/angular@cb631bdb1
PR Close#32427
To persist some of its state, `ngcc` needs to update `package.json`
files (both in memory and on disk).
This refactoring abstracts these operations behind the
`PackageJsonUpdater` interface, making it easier to orchestrate them
from different contexts (e.g. when running tasks in parallel on multiple
processes).
Inspired by/Based on @alxhub's prototype: alxhub/angular@cb631bdb1
PR Close#32427
In order to prevent `ngcc`'d packages (e.g. libraries) from getting
accidentally published, `ngcc` overwrites the `prepublishOnly` npm
script to log a warning and exit with an error. In case we want to
restore the original script (e.g. "undo" `ngcc` processing), we keep a
backup of the original `prepublishOnly` script.
Previously, running `ngcc` a second time (e.g. for a different format)
would create a backup of the overwritten `prepublishOnly` script (if
there was originally no `prepublishOnly` script). As a result, if we
ever tried to "undo" `ngcc` processing and restore the original
`prepublishOnly` script, the error-throwing script would be restored
instead.
This commit fixes it by ensuring that we only back up a `prepublishOnly`
script, iff it is not the one we created ourselves (i.e. the
error-throwing one).
PR Close#32427
Previously, any diagnostics reported during the compilation of an
entry-point would not be shown to the user, but either be ignored or
cause a hard crash in case of a `FatalDiagnosticError`. This is
unfortunate, as such error instances contain information on which code
was responsible for producing the error, whereas only its error message
would not. Therefore, it was quite hard to determine where the error
originates from.
This commit introduces behavior to deal with error diagnostics in a more
graceful way. Such diagnostics will still cause the compilation to fail,
however the error message now contains formatted diagnostics.
Closes#31977
Resolves FW-1374
PR Close#31996
If a project has nested projects that contain node_modules folders
that get processed by ngcc, it can be confusing when the ngcc
version changes since the error message is very generic:
```
The ngcc compiler has changed since the last ngcc build.
Please completely remove `node_modules` and try again.
```
This commit augments the error message with the path of
the entry-point that failed so that it is more obvious which
node_modules folder to remove.
BREAKING CHANGE:
This commit removes the public export of `hasBeenProcessed()`.
This was exported to be availble to the CLI integration but was never
used. The change to the function signature is a breaking change in itself
so we remove the function altogether to simplify and lower the public
API surface going forward.
PR Close#32396
Reworks the compiler to output the factories for directives, components and pipes under a new static field called `ngFactoryFn`, instead of the usual `factory` property in their respective defs. This should eventually allow us to inject any kind of decorated class (e.g. a pipe).
**Note:** these changes are the first part of the refactor and they don't include injectables. I decided to leave injectables for a follow-up PR, because there's some more cases we need to handle when it comes to their factories. Furthermore, directives, components and pipes make up most of the compiler output tests that need to be refactored and it'll make follow-up PRs easier to review if the tests are cleaned up now.
This is part of the larger refactor for FW-1468.
PR Close#31953
When ngcc is called for a specific entry-point, it has to determine
which dependencies to transitively process. To accomplish this, ngcc
traverses the full import graph of the entry-points it encounters, for
which it uses a dependency host to find all module imports. Since
imports look different in the various bundle formats ngcc supports, a
specific dependency host is used depending on the information provided
in an entry-points `package.json` file. If there's not enough
information in the `package.json` file for ngcc to be able to determine
which dependency host to use, ngcc would fail with an error.
If, however, the entry-point is not compiled by Angular, it is not
necessary to process any of its dependencies. None of them can have
been compiled by Angular so ngcc does not need to know about them.
Therefore, this commit changes the behavior to avoid recursing into
dependencies of entry-points that are not compiled by Angular.
In particular, this fixes an issue for packages that have dependencies
on the `date-fns` package. This package has various secondary
entry-points that have a `package.json` file only containing a `typings`
field, without providing additional fields for ngcc to know which
dependency host to use. By not needing a dependency host at all, the
error is avoided.
Fixes#32302
PR Close#32303
Previously, `ngcc` assumed that if a format property was defined in
`package.json` it would point to a valid format-path (i.e. a file that
is an entry-point for a specific format). This is generally the case,
except if a format property is set to a non-string value (such as
`package.json`) - either directly in the `package.json` (which is unusual)
or in ngcc.config.js (which is a valid usecase, when one wants a
format property to be ignored by `ngcc`).
For example, the following config file would cause `ngcc` to throw:
```
module.exports = {
packages: {
'test-package': {
entryPoints: {
'.': {
override: {
fesm2015: undefined,
},
},
},
},
},
};
```
This commit fixes it by ensuring that only format properties whose value
is a string are considered by `ngcc`.
For reference, this regression was introduced in #32052.
Fixes#32188
PR Close#32205
During the dependency analysis phase of ngcc, imports are resolved to
files on disk according to certain module resolution rules. Since module
specifiers are typically missing extensions, or can refer to index.js
barrel files within a directory, the module resolver attempts several
postfixes when searching for a module import on disk. Module specifiers
that already include an extension, however, would fail to be resolved as
ngcc's module resolver failed to check the location on disk without
adding any postfixes.
Closes#32097
PR Close#32181
During the recursive processing of dependencies, ngcc resolves the
requested file to an actual location on disk, by testing various
extensions. For recursive calls however, the path is known to have been
resolved in the module resolver. Therefore, it is safe to move the path
resolution to the initial caller into the recursive process.
Note that this is not expected to improve the performance of ngcc, as
the call to `resolveFileWithPostfixes` is known to succeed immediately,
as the provided path is known to exist without needing to add any
postfixes. Furthermore, the FileSystem caches whether files exist, so
the additional check that we used to do was cheap.
PR Close#32181
Pete Bacon Darwin
Igor Minar
Alex Rickabaugh
George Kalpakas
Kara Erickson
Gabriel Medeiros Coelho
crisbeto
Martin Probst
JoostK
atscott
Elvis Begovic