This commit adds a `packageName` property to the `EntryPoint` interface.
In a subsequent commit this will be used to retrieve the correct ngcc
configuration for each package, regardless of its path.
PR Close#37040
In order to retrieve the ngcc configuration (if any) for an entry-point,
ngcc has to detect the containing package's version.
Previously, ngcc would try to read the version from the entry-point's
`package.json` file, which was different than the package's top-level
`package.json` for secondary entry-points. For example, it would try to
read it from `node_modules/@angular/common/http/package.json` for
entry-point `@angular/common/http`. However, the `package.json` files
for secondary entry-points are not guaranteed to include a `version`
property.
This commit fixes this by first trying to read the version from the
_package's_ `package.json` (falling back to the entry-point's
`package.json`). For example, it will first try to read it from
`@angular/common/package.json` for entry-point `@angular/common/http`.
PR Close#37040
Rename the `package` property to `packagePath` on the `EntryPoint`
interface. This makes it more clear that the `packagePath` property
holds the absolute path to the containing package (similar to how `path`
holds the path to the entry-point). This will also align with the
`packageName` property that will be added in a subsequent commit.
This commit also re-orders the `EntryPoint` properties to group related
properties together and to match the order of properties on instances
with that on the interface.
PR Close#37040
Previously, when an entry-point was ignored via an ngcc config, ngcc
would scan sub-directories for sub-entry-points, but would not use the
correct `packagePath`. For example, if `@angular/common` was ignored, it
would look at `@angular/common/http` but incorrectly use
`.../@angular/common/http` as the `packagePath` (instead of
`.../@angular/common`). As a result, it would not retrieve the correct
ngcc config for the actual package.
This commit fixes it by ensuring the correct `packagePath` is used, even
if the primary entry-point corresponding to that path is ignored. In
order to do this, a new return value for `getEntryPointInfo()` is added:
`IGNORED_ENTRY_POINT`. This is used to differentiate between directories
that correspond to no or an incompatible entry-point and those that
correspond to an entry-point that could otherwise be valid but is
explicitly ignored. Consumers of `getEntryPointInfo()` can then use this
info to discard ignored entry-points, but still use the correct
`packagePath` when scanning their sub-directories for secondary
entry-points.
PR Close#37040
Adds @nocollapse to static properties added by ngcc
iff annotateForClosureCompiler is true.
The Closure Compiler will collapse static properties
into the global namespace. Adding this annotation keeps
the properties attached to their respective object, which
allows them to be referenced via a class's constructor.
The annotation is already added by ngtsc and ngc under the
same option, this commit extends the functionality to ngcc.
Closes#36618.
PR Close#36652
It is quite common for the TS compiler to have to add synthetic
types to function signatures, where the developer has not
explicitly provided them. This results in `import(...)` expressions
appearing in typings files. For example in `@ngrx/data` there is a
class with a getter that has an implicit type:
```ts
export declare class EntityCollectionServiceBase<...> {
...
get store() {
return this.dispatcher.store;
}
...
}
```
In the d.ts file for this we get:
```ts
get store(): Store<import("@ngrx/data").EntityCache>;
```
Given that this file is within the `@ngrx/data` package already,
this caused ngcc to believe that there was a circular dependency,
causing it to fail to process the package - and in fact crash!
This commit resolves this problem by ignoring `import()` expressions
when scanning typings programs for dependencies. This ability was
only introduced very recently in a 10.0.0 RC release, and so it has
limited benefit given that up till now ngcc has been able to process
libraries effectively without it. Moreover, in the rare case that a
package does have such a dependency, it should get picked up
by the sync ngcc+CLI integration point.
PR Close#37503
Now in TS 3.9, classes in ES2015 can be wrapped in an IIFE.
This commit ensures that we still find the static properties that contain
decorator information, even if they are attached to the adjacent node
of the class, rather than the implementation or declaration.
Fixes#37330
PR Close#37436
This finder is designed to only process entry-points that are reachable
by the program defined by a tsconfig.json file.
It is triggered by calling `mainNgcc()` with the `findEntryPointsFromTsConfigProgram`
option set to true. It is ignored if a `targetEntryPointPath` has been
provided as well.
It is triggered from the command line by adding the `--use-program-dependencies`
option, which is also ignored if the `--target` option has been provided.
Using this option can speed up processing in cases where there is a large
number of dependencies installed but only a small proportion of the
entry-points are actually imported into the application.
PR Close#37075
Previously we only checked for static import declaration statements.
This commit also finds import paths from dynamic import expressions.
Also this commit should speed up processing: Previously we were parsing
the source code contents into a `ts.SourceFile` and then walking the parsed
AST to find import paths.
Generating an AST is unnecessary work and it is faster and creates less
memory pressure to just scan the source code contents with the TypeScript
scanner, identifying import paths from the tokens.
PR Close#37075
Previously this host was skipping files if they had imports that spanned
multiple lines, or if the import was a dynamic import expression.
PR Close#37075
This commit will store a cached copy of the parsed tsconfig
that can be reused if the tsconfig path is the same.
This will improve the ngcc "noop" case, where there is no processing
to do, when the entry-points have already been processed.
Previously we were parsing this config every time we checked for
entry-points to process, which can take up to seconds in some
cases.
Resolves#36882
PR Close#37417
Currently, if an ngcc process is killed in a manner that it doesn't clean
up its lock file (or is killed too quickly) the compiler reports that it
is waiting on the PID of a process that doesn't exist, and that it will
wait up to a maximum of N seconds. This PR updates the locking code to
additionally check if the process exists, and if it does not it will
immediately bail out, and print the location of the lock file so a user
may clean it up.
PR Close#37250
Inline source-maps in typings files can impact IDE performance
so ngcc should only add such maps if the original typings file
contains inline source-maps.
Fixes#37324
PR Close#37363
In #37221 we disabled tsickle passes from transforming the tsc output that is used to publish all
Angular framework and components packages (@angular/*).
This change however revealed a bug in the ngc that caused __decorate and __metadata calls to still
be emitted in the JS code even though we don't depend on them.
Additionally it was these calls that caused code in @angular/material packages to fail at runtime
due to circular dependency in the emitted decorator code documeted as
https://github.com/microsoft/TypeScript/issues/27519.
This change partially rolls back #37221 by reenabling the decorator to static fields (static
properties) downleveling.
This is just a temporary workaround while we are also fixing root cause in `ngc` - tracked as
FW-2199.
Resolves FW-2198.
Related to FW-2196
PR Close#37317
As of TypeScript 3.9, the tsc emit is not compatible with Closure
Compiler due to
https://github.com/microsoft/TypeScript/pull/32011.
There is some hope that this will be fixed by a solution like the one
proposed in
https://github.com/microsoft/TypeScript/issues/38374 but currently it's
unclear if / when that will
happen.
Since the Closure support has been somewhat already broken, and the
tsickle pass has been a source
of headaches for some time for Angular packages, we are removing it for
now while we rethink our
strategy to make Angular Closure compatible outside of Google.
This change has no effect on our Closure compatibility within Google
which work well because all the
code is compiled from sources and passed through tsickle.
This change only disables the tsickle pass but doesn't remove it.
A follow up PR should either remove all the traces of tscikle or
re-enable the fixed version.
BREAKING CHANGE: Angular npm packages no longer contain jsdoc comments
to support Closure Compiler's advanced optimizations
The support for Closure compiler in Angular packages has been
experimental and broken for quite some
time.
As of TS3.9 Closure is unusable with the JavaScript emit. Please follow
https://github.com/microsoft/TypeScript/issues/38374 for more
information and updates.
If you used Closure compiler with Angular in the past, you will likely
be better off consuming
Angular packages built from sources directly rather than consuming the
version we publish on npm
which is primarily optimized for Webpack/Rollup + Terser build pipeline.
As a temporary workaround you might consider using your current build
pipeline with Closure flag
`--compilation_level=SIMPLE`. This flag will ensure that your build
pipeline produces buildable and
runnable artifacts, at the cost of increased payload size due to
advanced optimizations being disabled.
If you were affected by this change, please help us understand your
needs by leaving a comment on https://github.com/angular/angular/issues/37234.
PR Close#37221
In ES2015 IIFE wrapped classes, the identifier that would reference the class
of the NgModule may be an alias variable. Previously the `Esm2015ReflectionHost`
was not able to match this alias to the original class declaration. This resulted
in failing to identify some `ModuleWithProviders` functions in such case.
These IIFE wrapped classes were introduced in TypeScript 3.9, which is why
this issue is only recently appearing. Since 9.1.x does not support TS 3.9
there is no reason to backport this commit to that branch.
Fixes#37189
PR Close#37206
To better check that the code is working, this commit gives a
distinct name (`DecoratedWrappedClass_1`) to the "adjacent"
class declaration in the tests.
PR Close#37206
In TypeScript 3.9 some re-export syntaxes have changed to be getter
functions (created by calls to `Object.defineProperty()`) rather than
simple property accessors.
This commit adds support into the CommonJS and UMD reflection hosts
for this style of re-export syntax.
PR Close#36989
In the CommonJS and UMD reflection hosts, the logic for computing the
`viaModule` property of `Declaration` objects was not correct for some
cases when getting the exports of modules.
In these cases it was setting `viaModule` to the path of the local module
rather than `null`.
PR Close#36989
Using backtick multiline strings leads to confusing layout
that does not fit with the surrounding indentation. Also it
can lead to test fragility due to automated code formatting.
This commit changes just one set of subject code to use
a more resilient string concatenation approach.
PR Close#36989
After the refactoring of the reflection hosts to accommodate
ES2015 classes wrapped in IIFEs. The same treatment needs to
be applied to the rendering formatters.
PR Close#36989
In TS 3.9, ES2015 output can contain ES classes that are wrapped in an
IIFE. So now ES2015 class declarations can look like one of:
```
class OuterClass1 {}
```
```
let OuterClass = class InnerClass {};
```
```
var AliasClass;
let OuterClass = AliasClass = class InnerClass {};
```
```
let OuterClass = (() => class InnerClass {}};
```
```
var AliasClass;
let OuterClass = AliasClass = (() => class InnerClass {})();
```
```
let OuterClass = (() => {
let AdjacentClass = class InnerClass {};
// ... static properties or decorators attached to `AdjacentClass`
return AdjacentClass;
})();
```
```
var AliasClass;
let OuterClass = AliasClass = (() => {
let AdjacentClass = class InnerClass {};
// ... static properties or decorators attached to `AdjacentClass`
return AdjacentClass;
})();
```
The `Esm5ReflectionHost` already handles slightly different IIFE wrappers
around function-based classes. This can be substantially reused when
fixing `Esm2015ReflectionHost`, since there is a lot of commonality
between the two.
This commit moves code from the `Esm5ReflectionHost` into the `Esm2015ReflectionHost`
and looks to share as much as possible between the two hosts.
PR Close#36989
As of version 10, libraries following the APF will no longer contain
ESM5 output. Hence, tests in ngcc need to be updated as they currently
rely on the release output of `@angular/core`.
Additionally, we'd need to support in ngcc that the `module`
property of entry-points no longer necessarily refers to
`esm5` output, but instead can also target `esm2015`.
We currently achieve this by checking the path the `module`
property points to. We can do this because as per APF, the
folder name is known for the esm2015 output. Long-term for
more coverage, we want to sniff the format by looking for
known ES2015 constructs in the file `module` refers to.
PR Close#36944
In #36892 the `ModuleWithProviders` type parameter becomes required.
This exposes a bug in ngcc, where it can only handle functions that have a
specific form:
```
function forRoot() {
return { ... };
}
```
In other words, it only accepts functions that return an object literal.
In some libraries, the function instead returns a call to another function.
For example in `angular-in-memory-web-api`:
```
InMemoryWebApiModule.forFeature = function (dbCreator, options) {
return InMemoryWebApiModule_1.forRoot(dbCreator, options);
};
```
This commit changes the parsing of such functions to use the
`PartialEvaluator`, which can evaluate these more complex function
bodies.
PR Close#36948
Previously this method was implemented on the `NgccReflectionHost`,
but really it is asking too much of the host, since it actually needs to do
some static evaluation of the code to be able to support a wider range
of function shapes. Also there was only one implementation of the method
in the `Esm2015ReflectionHost` since it has no format specific code in
in.
This commit moves the whole function (and supporting helpers) into the
`ModuleWithProvidersAnalyzer`, which is the only place it was being used.
This class will be able to do further static evaluation of the function bodies
in order to support more function shapes than the host can do on its own.
The commit removes a whole set of reflection host tests but these are
already covered by the tests of the analyzer.
PR Close#36948
This function needs to deduplicate the paths that are found from the
paths mappings. Previously this deduplication was not linear and also
called the expensive `relative()` function many times.
This commit, suggested by @JoostK, reduces the complexity of the deduplication
by using a tree structure built from the segments of each path.
PR Close#36881
In TS 3.9 the compiler will start to wrap ES2015 classes in an IIFE to help with
tree-shaking when the class has "associated" statements.
E.g.
```ts
let PlatformLocation = /** @class */ (() => {
...
class PlatformLocation {
}
...
return PlatformLocation;
})();
```
This commit updates `Esm2015ReflectionHost` to support this format.
PR Close#36884
The commit adds support to the ngcc.config.js file for setting the
`retryAttempts` and `retryDelay` options for the `AsyncLocker`.
An integration test adds a new check for a timeout and actually uses the
ngcc.config.js to reduce the timeout time to prevent the test from taking
too long to complete.
PR Close#36838
Strictly this method only returns config for packages. So this commit
renames it to `getPackageConfig()`, which frees us up to add other
"getXxxxConfig()` methods later.
PR Close#36838
This test is basically duplicated (and slightly enhanced) in the
following test. So it is superfluous. (I suspect it was the result
of a broken rebase.)
PR Close#36838
When ngcc is having to pause and wait for another process
it provides a message to the user. This commit adds the extra
information about how to remove the lockfile if desired, since
this message is not shown if you Ctrl-C out of the process before
the timeout period ends.
PR Close#36838
Now that `ngcc/src/ngcc_options` imports `FileWriter` type, there is a
circular dependency detected by the `ts-circular-deps:check` lint check:
```
ngcc/src/ngcc_options.ts
→ ngcc/src/writing/file_writer.ts
→ ngcc/src/packages/entry_point_bundle.ts
→ ngcc/src/ngcc_options.ts
```
This commit moves the `PathMappings` type (and related helpers) to a
separate file to avoid the circular dependency.
NOTE:
The circular dependency was only with taking types into account. There
was no circular dependency for the actual (JS) code.
PR Close#36626
When running in parallel mode, worker processes forward errors thrown
during task processing to the master process, which in turn exits with
an error.
However, there are cases where the error is not directly related to
processing the entry-point. One such case is when there is not enough
memory (for example, due to all the other tasks being processed
simultaneously).
Previously, an `ENOMEM` error thrown on a worker process would propagate
to the master process, eventually causing ngcc to exit with an error.
Example failure: https://circleci.com/gh/angular/angular/682198
This commit improves handling of these low-memory situations by
detecting `ENOMEM` errors and killing the worker process, thus allowing
the master process to decide how to handle that. The master process will
put the task back into the tasks queue and continue processing tasks
with the rest of the worker processes (and thus with lower memory
pressure).
PR Close#36626
Previously, when running in parallel mode and a worker process crashed
while processing a task, it was not possible for ngcc to continue
without risking ending up with a corrupted entry-point and therefore it
exited with an error. This, for example, could happen when a worker
process received a `SIGKILL` signal, which was frequently observed in CI
environments. This was probably the result of Docker killing processes
due to increased memory pressure.
One factor that amplifies the problem under Docker (which is often used
in CI) is that it is not possible to distinguish between the available
CPU cores on the host machine and the ones made available to Docker
containers, thus resulting in ngcc spawning too many worker processes.
This commit addresses these issues in the following ways:
1. We take advantage of the fact that files are written to disk only
after an entry-point has been fully analyzed/compiled. The master
process can now determine whether a worker process has not yet
started writing files to disk (even if it was in the middle of
processing a task) and just put the task back into the tasks queue if
the worker process crashes.
2. The master process keeps track of the transformed files that a worker
process will attempt to write to disk. If the worker process crashes
while writing files, the master process can revert any changes and
put the task back into the tasks queue (without risking corruption).
3. When a worker process crashes while processing a task (which can be a
result of increased memory pressure or too many worker processes),
the master process will not try to re-spawn it. This way the number
or worker processes is gradually adjusted to a level that can be
accomodated by the system's resources.
Examples of ngcc being able to recover after a worker process crashed:
- While idling: https://circleci.com/gh/angular/angular/682197
- While compiling: https://circleci.com/gh/angular/angular/682209
- While writing files: https://circleci.com/gh/angular/angular/682267
Jira issue: [FW-2008](https://angular-team.atlassian.net/browse/FW-2008)
Fixes#36278
PR Close#36626
This commit adds a `revertFile()` method to `FileWriter`, which can
revert a transformed file (and its backup - if any) written by the
`FileWriter`.
In a subsequent commit, this will be used to allow ngcc to recover
when a worker process crashes in the middle of processing a task.
PR Close#36626
With this commit, worker processes will notify the master process about
the transformed files they are about to write to disk before starting
writing them.
In a subsequent commit, this will be used to allow ngcc to recover when
a worker process crashes in the middle of processing a task.
PR Close#36626
This commit enhances the `CompileFn`, which is used to process each
entry-point, to support running a passed-in callback (and wait for it to
complete) before proceeding with writing the transformed files to disk.
This functionality is currently not used. In a subsequent commit, it
will be used for passing info from worker processes to the master
process that will allow ngcc to recover when a worker process crashes in
the middle of processing a task.
PR Close#36626
Rename the `markTaskCompleted()` method to be consistent with the other
similar methods of `TaskQueue` (`markAsFailed()` and
`markAsUnprocessed()`).
PR Close#36626
This commit adds support for stopping processing an in-progress task
and moving it back to the list of pending tasks.
In a subsequent commit, this will be used to allow ngcc to recover when
a worker process crashes in the middle of processing a task.
PR Close#36626
An enum declaration in TypeScript code will be emitted into JavaScript
as a regular variable declaration, with the enum members being declared
inside an IIFE. For ngcc to support interpreting such variable
declarations as enum declarations with its members, ngcc needs to
recognize the enum declaration emit structure and extract all member
from the statements in the IIFE.
This commit extends the `ConcreteDeclaration` structure in the
`ReflectionHost` abstraction to be able to capture the enum members
on a variable declaration, as a substitute for the original
`ts.EnumDeclaration` as it existed in TypeScript code. The static
interpreter has been extended to handle the extracted enum members
as it would have done for `ts.EnumDeclaration`.
Fixes#35584
Resolves FW-2069
PR Close#36550
The cached file-system was implemented to speed up ngcc
processing, but in reality most files are not accessed many times
and there is no noticeable degradation in speed by removing it.
Benchmarking `ngcc -l debug` for AIO on a local machine
gave a range of 196-236 seconds with the cache and 197-224
seconds without the cache.
Moreover, when running in parallel mode, ngcc has a separate
file cache for each process. This results in excess memory usage.
Notably the master process, which only does analysis of entry-points
holds on to up to 500Mb for AIO when using the cache compared to
only around 30Mb when not using the cache.
Finally, the file-system cache being incorrectly primed with file
contents before being processed has been the cause of a number
of bugs. For example https://github.com/angular/angular-cli/issues/16860#issuecomment-614694269.
PR Close#36687
The current ngcc lock-file strategy spawns a new process in order to
capture a potential `SIGINT` and remove the lock-file. For more
information see #35861.
Previously, this unlocker process was spawned as soon as the `LockFile`
was instantiated in order to have it available as soon as possible
(given that spawning a process is an asynchronous operation). Since the
`LockFile` was instantiated and passed to the `Executor`, this meant
that an unlocker process was spawned for each cluster worker, when
running ngcc in parallel mode. These processes were not needed, since
the `LockFile` was not used in cluster workers, but we still had to pay
the overhead of each process' own memory and V8 instance.
(NOTE: This overhead was small compared to the memory consumed by ngcc's
normal operations, but still unnecessary.)
This commit avoids the extra processes by only spawning an unlocker
process when running on the cluster master process and not on worker
processes.
PR Close#36569
Previously, when we needed to detect whether a file is external to a
package, we only checked whether the relative path to the file from the
package's root started with `..`. This would detect external imports
when the packages were siblings (e.g. peer dependencies or hoisted to
the top of `node_modules/` by the package manager), but would fail to
detect imports from packages located in nested `node_modules/` as
external. For example, importing `node_modules/foo/node_modules/bar`
from a file in `node_modules/foo/` would be considered internal to the
`foo` package.
This could result in processing/analyzing more files than necessary.
More importantly it could lead to errors due to trying to analyze
non-Angular packages that were direct dependencies of Angular packages.
This commit fixes it by also verifying that the relative path to a file
does not start with `node_modules/`.
Jira issue: [FW-2068](https://angular-team.atlassian.net/browse/FW-2068)
Fixes#36526
PR Close#36559
The base path for package and entry-points is known so there is
no need to store these in the file. Also this commit avoids storing
empty arrays unnecessarily.
PR Close#36486
Previously, even if an entry-point did not need to be processed,
ngcc would always parse the files of the entry-point to compute
its dependencies. This can take a lot of time for large node_modules.
Now these dependencies are cached in the entry-point manifest,
and read from there rather than computing them every time.
See https://github.com/angular/angular/issues/36414\#issuecomment-608401834
FW-2047
PR Close#36486
When the compiler needs to convert a type reference to a value
expression, it may encounter a type that refers to a namespaced symbol.
Such namespaces need to be handled specially as there's various forms
available. Consider a namespace named "ns":
1. One can refer to a namespace by itself: `ns`. A namespace is only
allowed to be used in a type position if it has been merged with a
class, but even if this is the case it may not be possible to convert
that type into a value expression depending on the import form. More
on this later (case a below)
2. One can refer to a type within the namespace: `ns.Foo`. An import
needs to be generated to `ns`, from which the `Foo` property can then
be read.
3. One can refer to a type in a nested namespace within `ns`:
`ns.Foo.Bar` and possibly even deeper nested. The value
representation is similar to case 2, but includes additional property
accesses.
The exact strategy of how to deal with these cases depends on the type
of import used. There's two flavors available:
a. A namespaced import like `import * as ns from 'ns';` that creates
a local namespace that is irrelevant to the import that needs to be
generated (as said import would be used instead of the original
import).
If the local namespace "ns" itself is referred to in a type position,
it is invalid to convert it into a value expression. Some JavaScript
libraries publish a value as default export using `export = MyClass;`
syntax, however it is illegal to refer to that value using "ns".
Consequently, such usage in a type position *must* be accompanied by
an `@Inject` decorator to provide an explicit token.
b. An explicit namespace declaration within a module, that can be
imported using a named import like `import {ns} from 'ns';` where the
"ns" module declares a namespace using `declare namespace ns {}`.
In this case, it's the namespace itself that needs to be imported,
after which any qualified references into the namespace are converted
into property accesses.
Before this change, support for namespaces in the type-to-value
conversion was limited and only worked correctly for a single qualified
name using a namespace import (case 2a). All other cases were either
producing incorrect code or would crash the compiler (case 1a).
Crashing the compiler is not desirable as it does not indicate where
the issue is. Moreover, the result of a type-to-value conversion is
irrelevant when an explicit injection token is provided using `@Inject`,
so referring to a namespace in a type position (case 1) could still be
valid.
This commit introduces logic to the type-to-value conversion to be able
to properly deal with all type references to namespaced symbols.
Fixes#36006
Resolves FW-1995
PR Close#36106
In cc4b813e75 the `getBasePaths()`
function was changed to log a warning if a `basePath()` computed from
the `paths` mappings did not exist. It turns out this is a common and
accepted scenario, so we should not log warnings in this case.
Fixes#36518
PR Close#36525
1. update jasmine to 3.5
2. update @types/jasmine to 3.5
3. update @types/jasminewd2 to 2.0.8
Also fix several cases, the new jasmine 3 will help to create test cases correctly,
such as in the `jasmine 2.x` version, the following case will pass
```
expect(1 == 2);
```
But in jsamine 3, the case will need to be
```
expect(1 == 2).toBeTrue();
```
PR Close#34625
Previously, `isRelativePath()` assumed paths are *nix-style. This caused
Windows-style paths (such as `C:\foo\some-package\some-file.js`) to not
be recognized as "relative" imports.
This commit fixes this by using the OS-agnostic `isRooted()` helper and
also accounting for both styles of path delimiters: `/` and `\`
PR Close#36372
When TypeScript downlevels ES2015+ code to ES5, it uses some helper
functions to emulate some ES2015+ features, such as spread syntax. The
TypeScript compiler can be configured to emit these helpers into the
transpiled code (which is controlled by the `noEmitHelpers` option -
false by default). It can also be configured to import these helpers
from the `tslib` module (which is controlled by the `importHelpers`
option - false by default).
While most of the time the helpers will be either emitted or imported,
it is possible that one configures their app to neither emit nor import
them. In that case, the helpers could, for example, be made available on
the global object. This is what `@nativescript/angular`
v9.0.0-next-2019-11-12-155500-01 does. See, for example, [common.js][1].
Ngcc must be able to detect and statically evaluate these helpers.
Previously, it was only able to detect emitted or imported helpers.
This commit adds support for detecting these helpers if they are neither
emitted nor imported. It does this by checking identifiers for which no
declaration (either concrete or inline) can be found against a list of
known TypeScript helper function names.
[1]: https://unpkg.com/browse/@nativescript/angular@9.0.0-next-2019-11-12-155500-01/common.js
PR Close#36418
The source-map flattening was throwing an error when there
is a cyclic dependency between source files and source-maps.
The error was either a custom one describing the cycle, or a
"Maximum call stack size exceeded" one.
Now this is handled more leniently, resulting in a partially loaded
source file (or source-map) and a warning logged.
Fixes#35727Fixes#35757
Fixes https://github.com/angular/angular-cli/issues/17106
Fixes https://github.com/angular/angular-cli/issues/17115
PR Close#36452
Recently we added support for ignoring specified deep-import
warnings by providing sets of regular expressions within the
`ngcc.config.js` file. But this was only working for the project
level configuration.
This commit fixes ngcc so that it will also read these regular
expressions from package level configuration too.
Fixes#35750
PR Close#36423
The `browser` package.json property is now supported to the same
level as `main` - i.e. it is sniffed for UMD, ESM5 and CommonJS.
The `browser` property can also contain an object with file overrides
but this is not supported by ngcc.
Fixes#36062
PR Close#36396
Previously, `main` was only checked for `umd` or `commonjs`
formats. Now if there are `import` or `export` statements in the
source file it will be deemed to be in `esm5` format.
Fixes#35788
PR Close#36396
In #36089, `DelegatingReflectionHost` was introduced. Under the hood, it
delegates another `NgccReflectionHost` in order to reflect over the
program's source files, while using a different TypeScript
`ReflectionHost` to reflect over `.d.ts` files (which is how external
dependencies are represented in the program).
Previously, the `NgccReflectionHost`s were used directly in tests. This
does not exercise them in the way they are exercised in the actual
program, because (when used directly) they will also reflect on `.d.ts`
files too (instead of delegating to the TypeScript `ReflectionHost`).
This could hide bugs that would happen on the actual program.
This commit fixes this by using the `DelegatingReflectionHost` in the
various `NgccReflectionHost` tests.
NOTE:
This change will cause some of the existing tests to start failing.
These failures demonstrate pre-existing bugs in ngcc, that were hidden
due to the tests' being inconsistent with how the `ReflectionHost`s are
used in the actual program. They will be fixed in the next commit.
PR Close#36284
Ngcc supports providing a project-level configuration to affect how
certain dependencies are processed and also has a built-in fallback
configuration for some unmaintained packages. Each entry in these
configurations could be scoped to specific versions of a package by
providing a version range. If no version range is provided for a
package, it defaults to `*` (with the intention of matching any
version).
Previously, the installed version of a package was tested against the
version range using the [semver][1] package's `satisfies()` function
with the default options. By default, `satisfies()` does not match
pre-releases (see [here][2] for more details on reasoning). While this
makes sense when determining what version of a dependency to install
(trying to avoid unexpected breaking changes), it is not desired in the
case of ngcc.
This commit fixes it by explicitly specifying that pre-release versions
should be matched normally.
[1]: https://www.npmjs.com/package/semver
[2]: https://github.com/npm/node-semver#prerelease-tags
PR Close#36370
Previously, a bad baseUrl or path mapping passed to an `EntryPointFinder`
could cause the original `sourceDirectory` to be superceded by a higher
directory. This could result in none of the sourceDirectory entry-points being
processed.
Now missing basePaths computed from path-mappings are discarded with
a warning. Further, if the `baseUrl` is the root directory then a warning is
given as this is most likely an error in the tsconfig.json.
Resolves#36313Resolves#36283
PR Close#36331
The previous optimizations in #35756 to the
`DirectoryWalkerEntryPointFinder` were over zealous
with regard to packages that have entry-points stored
in "container" directories in the package, where the
container directory was not an entry-point itself.
Now we will also walk such "container" folders as long
as they do not contain `.js` files, which we regard as an
indicator that the directory will not contain entry-points.
Fixes#36216
PR Close#36305
This commit simplifies the `DirectoryWalkerEntryPointFinder` inter-method
calling to make it easier to follow, and also to support controlling
walking of a directory based on its children.
PR Close#36305
Previously we only searched for package paths below the set of `basePaths`
that were computed from the `basePath` provided to ngcc and the set of
`pathMappings`.
In some scenarios, such as hoisted packages, the entry-point is not within
any of the `basePaths` identified above. For example:
```
project
packages
app
node_modules
app-lib (depends on lib1)
node_modules
lib1 (depends on lib2)
node_modules
lib2 (depends on lib3/entry-point)
lib3
entry-point
```
When CLI is compiling `app-lib` ngcc will be given
`project/packages/app/node_modules` as the `basePath.
If ngcc is asked to target `lib2`, the `targetPath` will be
`project/node_modules/lib1/node_modules/lib2`.
Since `lib2` depends upon `lib3/entry-point`, ngcc will need to compute
the package path for `project/node_modules/lib3/entry-point`.
Since `project/node_modules/lib3/entry-point` is not contained in the `basePath`
`project/packages/app/node_modules`, ngcc failed to compute the `packagePath`
correctly, instead assuming that it was the same as the entry-point path.
Now we also consider the nearest `node_modules` folder to the entry-point
path as an additional `basePath`. If one is found then we use the first
directory directly below that `node_modules` directory as the package path.
In the case of our example this extra `basePath` would be `project/node_modules`
which allows us to compute the `packagePath` of `project/node_modules/lib3`.
Fixes#35747
PR Close#36249
Previously ngcc never preserved whitespaces but this is at odds
with how the ViewEngine compiler works. In ViewEngine, library
templates are recompiled with the current application's tsconfig
settings, which meant that whitespace preservation could be set
in the application tsconfig file.
This commit allows ngcc to use the `preserveWhitespaces` setting
from tsconfig when compiling library templates. One should be aware
that this disallows different projects with different tsconfig settings
to share the same node_modules folder, with regard to whitespace
preservation. But this is already the case in the current ngcc since
this configuration is hard coded right now.
Fixes#35871
PR Close#36189
This commit augments the `FactoryDef` declaration of Angular decorated
classes to contain information about the parameter decorators used in
the constructor. If no constructor is present, or none of the parameters
have any Angular decorators, then this will be represented using the
`null` type. Otherwise, a tuple type is used where the entry at index `i`
corresponds with parameter `i`. Each tuple entry can be one of two types:
1. If the associated parameter does not have any Angular decorators,
the tuple entry will be the `null` type.
2. Otherwise, a type literal is used that may declare at least one of
the following properties:
- "attribute": if `@Attribute` is present. The injected attribute's
name is used as string literal type, or the `unknown` type if the
attribute name is not a string literal.
- "self": if `@Self` is present, always of type `true`.
- "skipSelf": if `@SkipSelf` is present, always of type `true`.
- "host": if `@Host` is present, always of type `true`.
- "optional": if `@Optional` is present, always of type `true`.
A property is only present if the corresponding decorator is used.
Note that the `@Inject` decorator is currently not included, as it's
non-trivial to properly convert the token's value expression to a
type that is valid in a declaration file.
Additionally, the `ComponentDefWithMeta` declaration that is created for
Angular components has been extended to include all selectors on
`ng-content` elements within the component's template.
This additional metadata is useful for tooling such as the Angular
Language Service, as it provides the ability to offer suggestions for
directives/components defined in libraries. At the moment, such
tooling extracts the necessary information from the _metadata.json_
manifest file as generated by ngc, however this metadata representation
is being replaced by the information emitted into the declaration files.
Resolves FW-1870
PR Close#35695
When computing the dependencies between packages which are not in
node_modules, we may need to rely upon path-mappings to find the path
to the imported entry-point.
This commit allows ngcc to use the path-mappings from a tsconfig
file to find dependencies. By default any tsconfig.json file in the directory
above the `basePath` is loaded but it is possible to use a path to a
specific file by providing the `tsConfigPath` property to mainNgcc,
or to turn off loading any tsconfig file by setting `tsConfigPath` to `null`.
At the command line this is controlled via the `--tsconfig` option.
Fixes#36119
PR Close#36180
When two entry-points overlap, ngcc may attempt to process some
files twice. Previously, when this occured ngcc would just exit with an
error preventing any other entry-points from being processed.
This commit changes ngcc so that if `errorOnFailedEntryPoint` is false, it will
simply log an error and continue to process entry-points. This is useful when
ngcc is processing the entire node_modules folder and there are some invalid
entry-points that the project doesn't actually use.
PR Close#36083
Previously, when an entry-point contained code that caused its compilation
to fail, ngcc would exit in the middle of processing, possibly leaving other
entry-points in a corrupt state.
This change adds a new `errorOnFailedEntryPoint` option to `mainNgcc` that
specifies whether ngcc should exit immediately or log an error and continue
processing other entry-points.
The default is `false` so that ngcc will not error but continue processing
as much as possible. This is useful in post-install hooks, and async CLI
integration, where we do not have as much control over which entry-points
should be processed.
The option is forced to true if the `targetEntryPointPath` is provided,
such as the sync integration with the CLI, since in that case it is targeting
an entry-point that will actually be used in the current project so we do want
ngcc to exit with an error at that point.
PR Close#36083
Later when we implement the ability to continue processing when tasks have
failed to compile, we will also need to avoid processing tasks that depend
upon the failed task.
This refactoring exposes this list of dependent tasks in a way that can be
used to skip processing of tasks that depend upon a failed task.
It also changes the blocking model of the parallel mode of operation so
that non-typings tasks are now blocked on their corresponding typings task.
Previously the non-typings tasks could be triggered to run in parallel to
the typings task, since they do not have a hard dependency on each other,
but this made it difficult to skip task correctly if the typings task failed,
since it was possible that a non-typings task was already in flight when
the typings task failed. The result of this is a small potential degradation
of performance in async parallel processing mode, in the rare cases that
there were not enough unblocked tasks to make use of all the available
workers.
PR Close#36083
Moving the definition of the `onTaskCompleted` callback into `mainNgcc()`
allows it to be configured based on options passed in there more easily.
This will be the case when we want to configure whether to log or throw
an error for tasks that failed to be processed successfully.
This commit also creates two new folders and moves the code around a bit
to make it easier to navigate the code§:
* `execution/tasks`: specific helpers such as task completion handlers
* `execution/tasks/queues`: the `TaskQueue` implementations and helpers
PR Close#36083
When ngcc is compiling an entry-point, it uses a `ReflectionHost` that
is specific to its format, e.g. ES2015, ES5, UMD or CommonJS. During the
compilation of that entry-point however, the reflector may be used to
reflect into external libraries using their declaration files.
Up until now this was achieved by letting all `ReflectionHost` classes
consider their parent class for reflector queries, thereby ending up in
the `TypeScriptReflectionHost` that is a common base class for all
reflector hosts. This approach has proven to be prone to bugs, as
failing to call into the base class would cause incompatibilities with
reading from declaration files.
The observation can be made that there's only two distinct kinds of
reflection host queries:
1. the reflector query is about code that is part of the entry-point
that is being compiled, or
2. the reflector query is for an external library that the entry-point
depends on, in which case the information is reflected
from the declaration files.
The `ReflectionHost` that was chosen for the entry-point should serve
only reflector queries for the first case, whereas a regular
`TypeScriptReflectionHost` should be used for the second case. This
avoids the problem where a format-specific `ReflectionHost` fails to
handle the second case correctly, as it isn't even considered for such
reflector queries.
This commit introduces a `ReflectionHost` that delegates to the
`TypeScriptReflectionHost` for AST nodes within declaration files,
otherwise delegating to the format-specific `ReflectionHost`.
Fixes#35078
Resolves FW-1859
PR Close#36089
The format property for ES5 bundles should be "module" or "es5"/"esm5",
but was "main" instead. The "main" property is appropriate for CommonJS
and UMD bundles, not for ES5 bundles.
PR Close#36089
Previously, calculations related to the position of and difference between
SegmentMarkers required extensive computation based around the line,
line start positions and columns of each segment.
PR Close#36027
The merging algorithm needs to find, for a given segment, what the next
segment in the source file is. This change modifies the `generatedSegment`
properties in the mappings so that they have a link directly to the following
segment.
PR Close#36027
By computing and caching the start of each line, rather than the length
of each line, we can save a lot of duplicated computation in the `segmentDiff()`
and `offsetSegment()` functions.
PR Close#36027
Previously the list of original segments that was searched for incoming
mappings did not differentiate between different original source files.
Now there is a separate array of segments to search for each of the
original source files.
PR Close#36027
The `@angular/core` package has a large number of source files
and mappings which exposed performance issues in the new source-map
flattening algorithm.
This change uses a binary search (rather than linear) when finding
matching mappings to merge. Initial measurements indicate that this
reduces processing time for `@angular/core` by about 50%.
PR Close#36027
In some scenarios it is useful for the developer to indicate
to ngcc that it should not use the entry-point manifest
file, and instead write a new one.
In the ngcc command line tool, this option is set by specfying
```
--invalidate-entry-point-manifest
```
PR Close#35931
The `DirectoryWalkerEntryPointFinder` has to traverse the
entire node_modules library everytime it executes in order to
identify the entry-points that need to be processed. This is
very time consuming (several seconds for big projects on
Windows).
This commit changes the `DirectoryWalkerEntryPointFinder` to
use the `EntryPointManifest` to store the paths to entry-points
that were found when doing this initial node_modules traversal
in a file to be reused for subsequent calls.
This dramatically speeds up ngcc processing when it has been run once
already.
PR Close#35931
The new `EntryPointManifest` class can read and write a
manifest file that contains all the paths to the entry-points
that have been found in a node_modules folder.
This can be used to speed up finding entry-points in
subsequent runs.
The manifest file stores the ngcc version and hashes of
the package lock-file and project config, since if these
change the manifest will need to be recomputed.
PR Close#35931
This commit adds support in the Angular monorepo and in the Angular
compiler(s) for TypeScript 3.8. All packages can now compile with
TS 3.8.
For most of the repo, only a handful few typings adjustments were needed:
* TS 3.8 has a new `CustomElementConstructor` DOM type, which enforces a
zero-argument constructor. The `NgElementConstructor` type previously
declared a required `injector` argument despite the fact that its
implementation allowed `injector` to be optional. The interface type was
updated to reflect the optionality of the argument.
* Certain error messages were changed, and expectations in tests were
updated as a result.
* tsserver (part of language server) now returns performance information in
responses, so test expectations were changed to only assert on the actual
body content of responses.
For compiler-cli and schematics (which use the TypeScript AST) a major
breaking change was the introduction of the export form:
```typescript
export * as foo from 'bar';
```
This is a `ts.NamespaceExport`, and the `exportClause` of a
`ts.ExportDeclaration` can now take this type as well as `ts.NamedExports`.
This broke a lot of places where `exportClause` was assumed to be
`ts.NamedExports`.
For the most part these breakages were in cases where it is not necessary
to handle the new `ts.NamedExports` anyway. ngtsc's design uses the
`ts.TypeChecker` APIs to understand syntax and so automatically supports the
new form of exports.
The View Engine compiler on the other hand extracts TS structures into
metadata.json files, and that format was not designed for namespaced
exports. As a result it will take a nontrivial amount of work if we want to
support such exports in View Engine. For now, these new exports are not
accounted for in metadata.json, and so using them in "folded" Angular
expressions will result in errors (probably claiming that the referenced
exported namespace doesn't exist).
Care was taken to only use TS APIs which are present in 3.7/3.6, as Angular
needs to remain compatible with these for the time being.
This commit does not update angular.io.
PR Close#35864
This version of `LockFile` creates an "unlocker" child-process that monitors
the main ngcc process and deletes the lock file if it exits unexpectedly.
This resolves the issue where the main process could not be killed by pressing
Ctrl-C at the terminal.
Fixes#35761
PR Close#35861
The previous implementation mixed up the management
of locking a piece of code (both sync and async) with the
management of writing and removing the lockFile that is
used as the flag for which process has locked the code.
This change splits these two concepts up. Apart from
avoiding the awkward base class it allows the `LockFile`
implementation to be replaced cleanly.
PR Close#35861
This reduces the time that `findEntryPoints` takes from 9701.143ms to 4177.278ms, by reducing the file operations done.
Reference: #35717
PR Close#35756
George Kalpakas
David Neil
Pete Bacon Darwin
Terence D. Honles
Igor Minar
Joey Perrott
Paul Gschwendtner
JoostK
JiaLiPassion
Alex Rickabaugh
Alan Agius