In Ivy, Angular decorators are compiled into static fields that are
inserted into a class declaration in a TypeScript transform. When
targeting Closure compiler such fields need to be annotated with
`@nocollapse` to prevent them from being lifted from a static field into
a variable, as that would prevent the Ivy runtime from being able to
find the compiled definitions.
Previously, there was a bug in TypeScript where synthetic comments added
in a transform would not be emitted at all, so as a workaround a global
regex-replace was done in the emit's `writeFile` callback that would add
the `@nocollapse` annotation to all static Ivy definition fields. This
approach is no longer possible when ngtsc is running as TypeScript
plugin, as a plugin cannot control emit behavior.
The workaround is no longer necessary, as synthetic comments are now
properly emitted, likely as of
https://github.com/microsoft/TypeScript/pull/22141 which has been
released with TypeScript 2.8.
This change is required for running ngtsc as TypeScript plugin in
Bazel's `ts_library` rule, to move away from the custom `ngc_wrapped`
approach.
Resolves FW-1952
PR Close#35932
Ngcc supports providing a project-level configuration to affect how
certain dependencies are processed and also has a built-in fallback
configuration for some unmaintained packages. Each entry in these
configurations could be scoped to specific versions of a package by
providing a version range. If no version range is provided for a
package, it defaults to `*` (with the intention of matching any
version).
Previously, the installed version of a package was tested against the
version range using the [semver][1] package's `satisfies()` function
with the default options. By default, `satisfies()` does not match
pre-releases (see [here][2] for more details on reasoning). While this
makes sense when determining what version of a dependency to install
(trying to avoid unexpected breaking changes), it is not desired in the
case of ngcc.
This commit fixes it by explicitly specifying that pre-release versions
should be matched normally.
[1]: https://www.npmjs.com/package/semver
[2]: https://github.com/npm/node-semver#prerelease-tags
PR Close#36370
Previously, a bad baseUrl or path mapping passed to an `EntryPointFinder`
could cause the original `sourceDirectory` to be superceded by a higher
directory. This could result in none of the sourceDirectory entry-points being
processed.
Now missing basePaths computed from path-mappings are discarded with
a warning. Further, if the `baseUrl` is the root directory then a warning is
given as this is most likely an error in the tsconfig.json.
Resolves#36313Resolves#36283
PR Close#36331
The previous optimizations in #35756 to the
`DirectoryWalkerEntryPointFinder` were over zealous
with regard to packages that have entry-points stored
in "container" directories in the package, where the
container directory was not an entry-point itself.
Now we will also walk such "container" folders as long
as they do not contain `.js` files, which we regard as an
indicator that the directory will not contain entry-points.
Fixes#36216
PR Close#36305
This commit simplifies the `DirectoryWalkerEntryPointFinder` inter-method
calling to make it easier to follow, and also to support controlling
walking of a directory based on its children.
PR Close#36305
Previously we only searched for package paths below the set of `basePaths`
that were computed from the `basePath` provided to ngcc and the set of
`pathMappings`.
In some scenarios, such as hoisted packages, the entry-point is not within
any of the `basePaths` identified above. For example:
```
project
packages
app
node_modules
app-lib (depends on lib1)
node_modules
lib1 (depends on lib2)
node_modules
lib2 (depends on lib3/entry-point)
lib3
entry-point
```
When CLI is compiling `app-lib` ngcc will be given
`project/packages/app/node_modules` as the `basePath.
If ngcc is asked to target `lib2`, the `targetPath` will be
`project/node_modules/lib1/node_modules/lib2`.
Since `lib2` depends upon `lib3/entry-point`, ngcc will need to compute
the package path for `project/node_modules/lib3/entry-point`.
Since `project/node_modules/lib3/entry-point` is not contained in the `basePath`
`project/packages/app/node_modules`, ngcc failed to compute the `packagePath`
correctly, instead assuming that it was the same as the entry-point path.
Now we also consider the nearest `node_modules` folder to the entry-point
path as an additional `basePath`. If one is found then we use the first
directory directly below that `node_modules` directory as the package path.
In the case of our example this extra `basePath` would be `project/node_modules`
which allows us to compute the `packagePath` of `project/node_modules/lib3`.
Fixes#35747
PR Close#36249
Previously ngcc never preserved whitespaces but this is at odds
with how the ViewEngine compiler works. In ViewEngine, library
templates are recompiled with the current application's tsconfig
settings, which meant that whitespace preservation could be set
in the application tsconfig file.
This commit allows ngcc to use the `preserveWhitespaces` setting
from tsconfig when compiling library templates. One should be aware
that this disallows different projects with different tsconfig settings
to share the same node_modules folder, with regard to whitespace
preservation. But this is already the case in the current ngcc since
this configuration is hard coded right now.
Fixes#35871
PR Close#36189
This commit augments the `FactoryDef` declaration of Angular decorated
classes to contain information about the parameter decorators used in
the constructor. If no constructor is present, or none of the parameters
have any Angular decorators, then this will be represented using the
`null` type. Otherwise, a tuple type is used where the entry at index `i`
corresponds with parameter `i`. Each tuple entry can be one of two types:
1. If the associated parameter does not have any Angular decorators,
the tuple entry will be the `null` type.
2. Otherwise, a type literal is used that may declare at least one of
the following properties:
- "attribute": if `@Attribute` is present. The injected attribute's
name is used as string literal type, or the `unknown` type if the
attribute name is not a string literal.
- "self": if `@Self` is present, always of type `true`.
- "skipSelf": if `@SkipSelf` is present, always of type `true`.
- "host": if `@Host` is present, always of type `true`.
- "optional": if `@Optional` is present, always of type `true`.
A property is only present if the corresponding decorator is used.
Note that the `@Inject` decorator is currently not included, as it's
non-trivial to properly convert the token's value expression to a
type that is valid in a declaration file.
Additionally, the `ComponentDefWithMeta` declaration that is created for
Angular components has been extended to include all selectors on
`ng-content` elements within the component's template.
This additional metadata is useful for tooling such as the Angular
Language Service, as it provides the ability to offer suggestions for
directives/components defined in libraries. At the moment, such
tooling extracts the necessary information from the _metadata.json_
manifest file as generated by ngc, however this metadata representation
is being replaced by the information emitted into the declaration files.
Resolves FW-1870
PR Close#35695
When computing the dependencies between packages which are not in
node_modules, we may need to rely upon path-mappings to find the path
to the imported entry-point.
This commit allows ngcc to use the path-mappings from a tsconfig
file to find dependencies. By default any tsconfig.json file in the directory
above the `basePath` is loaded but it is possible to use a path to a
specific file by providing the `tsConfigPath` property to mainNgcc,
or to turn off loading any tsconfig file by setting `tsConfigPath` to `null`.
At the command line this is controlled via the `--tsconfig` option.
Fixes#36119
PR Close#36180
When two entry-points overlap, ngcc may attempt to process some
files twice. Previously, when this occured ngcc would just exit with an
error preventing any other entry-points from being processed.
This commit changes ngcc so that if `errorOnFailedEntryPoint` is false, it will
simply log an error and continue to process entry-points. This is useful when
ngcc is processing the entire node_modules folder and there are some invalid
entry-points that the project doesn't actually use.
PR Close#36083
Previously, when an entry-point contained code that caused its compilation
to fail, ngcc would exit in the middle of processing, possibly leaving other
entry-points in a corrupt state.
This change adds a new `errorOnFailedEntryPoint` option to `mainNgcc` that
specifies whether ngcc should exit immediately or log an error and continue
processing other entry-points.
The default is `false` so that ngcc will not error but continue processing
as much as possible. This is useful in post-install hooks, and async CLI
integration, where we do not have as much control over which entry-points
should be processed.
The option is forced to true if the `targetEntryPointPath` is provided,
such as the sync integration with the CLI, since in that case it is targeting
an entry-point that will actually be used in the current project so we do want
ngcc to exit with an error at that point.
PR Close#36083
Later when we implement the ability to continue processing when tasks have
failed to compile, we will also need to avoid processing tasks that depend
upon the failed task.
This refactoring exposes this list of dependent tasks in a way that can be
used to skip processing of tasks that depend upon a failed task.
It also changes the blocking model of the parallel mode of operation so
that non-typings tasks are now blocked on their corresponding typings task.
Previously the non-typings tasks could be triggered to run in parallel to
the typings task, since they do not have a hard dependency on each other,
but this made it difficult to skip task correctly if the typings task failed,
since it was possible that a non-typings task was already in flight when
the typings task failed. The result of this is a small potential degradation
of performance in async parallel processing mode, in the rare cases that
there were not enough unblocked tasks to make use of all the available
workers.
PR Close#36083
Moving the definition of the `onTaskCompleted` callback into `mainNgcc()`
allows it to be configured based on options passed in there more easily.
This will be the case when we want to configure whether to log or throw
an error for tasks that failed to be processed successfully.
This commit also creates two new folders and moves the code around a bit
to make it easier to navigate the code§:
* `execution/tasks`: specific helpers such as task completion handlers
* `execution/tasks/queues`: the `TaskQueue` implementations and helpers
PR Close#36083
When ngcc is compiling an entry-point, it uses a `ReflectionHost` that
is specific to its format, e.g. ES2015, ES5, UMD or CommonJS. During the
compilation of that entry-point however, the reflector may be used to
reflect into external libraries using their declaration files.
Up until now this was achieved by letting all `ReflectionHost` classes
consider their parent class for reflector queries, thereby ending up in
the `TypeScriptReflectionHost` that is a common base class for all
reflector hosts. This approach has proven to be prone to bugs, as
failing to call into the base class would cause incompatibilities with
reading from declaration files.
The observation can be made that there's only two distinct kinds of
reflection host queries:
1. the reflector query is about code that is part of the entry-point
that is being compiled, or
2. the reflector query is for an external library that the entry-point
depends on, in which case the information is reflected
from the declaration files.
The `ReflectionHost` that was chosen for the entry-point should serve
only reflector queries for the first case, whereas a regular
`TypeScriptReflectionHost` should be used for the second case. This
avoids the problem where a format-specific `ReflectionHost` fails to
handle the second case correctly, as it isn't even considered for such
reflector queries.
This commit introduces a `ReflectionHost` that delegates to the
`TypeScriptReflectionHost` for AST nodes within declaration files,
otherwise delegating to the format-specific `ReflectionHost`.
Fixes#35078
Resolves FW-1859
PR Close#36089
The format property for ES5 bundles should be "module" or "es5"/"esm5",
but was "main" instead. The "main" property is appropriate for CommonJS
and UMD bundles, not for ES5 bundles.
PR Close#36089
Currently, when Angular code is built with Bazel and with Ivy, generated
factory shims (.ngfactory files) are not processed via the majority of
tsickle's transforms. This is a subtle effect of the build infrastructure,
but it boils down to a TsickleHost method `shouldSkipTsickleProcessing`.
For ngc_wrapped builds (Bazel + Angular), this method is defined in the
`@bazel/typescript` (aka bazel rules_typescript) implementation of
`CompilerHost`. The default behavior is to skip tsickle processing for files
which are not present in the original `srcs[]` of the build rule. In
Angular's case, this includes all generated shim files.
For View Engine factories this is probably desirable as they're quite
complex and they've never been tested with tsickle. Ivy factories however
are smaller and very straightforward, and it makes sense to treat them like
any other output.
This commit adjusts two independent implementations of
`shouldSkipTsickleProcessing` to enable transformation of Ivy shims:
* in `@angular/bazel` aka ngc_wrapped, the upstream `@bazel/typescript`
`CompilerHost` is patched to treat .ngfactory files the same as their
original source file, with respect to tsickle processing.
It is currently not possible to test this change as we don't have any test
that inspects tsickle output with bazel. It will be extensively tested in
g3.
* in `ngc`, Angular's own implementation is adjusted to allow for the
processing of shims when compiling with Ivy. This enables a unit test to
be written to validate the correct behavior of tsickle when given a host
that's appropriately configured to process factory shims.
For ngtsc-as-a-plugin, a similar fix will need to be submitted upstream in
tsc_wrapped.
PR Close#35848
PR Close#35975
This commit propagates the `sourceSpan` and `valueSpan` of a `VariableBinding`
in a microsyntax expression to `ParsedVariable`, and subsequently to
View Engine Variable AST and Ivy Variable AST.
Note that this commit does not propagate the `keySpan`, because it involves
significant changes to the template AST.
PR Close#36047
Prior to this commit, Ivy compiler didn't handle directive inputs with interpolations located on `<ng-template>` elements (e.g. `<ng-template dir="{{ field }}">`). That was the case for regular inputs as well as inputs that should be processed via i18n subsystem (e.g. `<ng-template i18n-dir dir="Hello {{ name }}">`). This commit adds support for such expressions for explicit `<ng-template>`s as well as a number of tests to confirm the behavior.
Fixes#35752.
PR Close#35984
Previously, calculations related to the position of and difference between
SegmentMarkers required extensive computation based around the line,
line start positions and columns of each segment.
PR Close#36027
The merging algorithm needs to find, for a given segment, what the next
segment in the source file is. This change modifies the `generatedSegment`
properties in the mappings so that they have a link directly to the following
segment.
PR Close#36027
By computing and caching the start of each line, rather than the length
of each line, we can save a lot of duplicated computation in the `segmentDiff()`
and `offsetSegment()` functions.
PR Close#36027
Previously the list of original segments that was searched for incoming
mappings did not differentiate between different original source files.
Now there is a separate array of segments to search for each of the
original source files.
PR Close#36027
The `@angular/core` package has a large number of source files
and mappings which exposed performance issues in the new source-map
flattening algorithm.
This change uses a binary search (rather than linear) when finding
matching mappings to merge. Initial measurements indicate that this
reduces processing time for `@angular/core` by about 50%.
PR Close#36027
ts-api-guardian uses `require.resolve` to resolve the actual and golden files under bazel. In Windows for these files to be resolved correct the full path including the workspace name as per the MANIFEST entries is required.
This used to be the case until the recent changes done to use npm_integration tests
83c74ceacf/tools/public_api_guard/public_api_guard.bzl (L19)83c74ceacf/tools/public_api_guard/public_api_guard.bzl (L28)
```
bazel test //packages/... --test_tag_filters=api_guard
//packages/animations:animations_api (cached) PASSED in 18.4s
//packages/common:common_api (cached) PASSED in 25.5s
//packages/compiler-cli:compiler_options_api (cached) PASSED in 12.4s
//packages/compiler-cli:error_code_api (cached) PASSED in 11.6s
//packages/core:core_api (cached) PASSED in 20.6s
//packages/core:ng_global_utils_api (cached) PASSED in 13.5s
//packages/elements:elements_api (cached) PASSED in 11.9s
//packages/forms:forms_api (cached) PASSED in 13.9s
//packages/http:http_api (cached) PASSED in 14.8s
//packages/localize:localize_api (cached) PASSED in 6.3s
//packages/platform-browser:platform-browser_api (cached) PASSED in 18.1s
//packages/platform-browser-dynamic:platform-browser-dynamic_api (cached) PASSED in 14.0s
//packages/platform-server:platform-server_api (cached) PASSED in 13.9s
//packages/platform-webworker:platform-webworker_api (cached) PASSED in 13.7s
//packages/platform-webworker-dynamic:platform-webworker-dynamic_api (cached) PASSED in 11.7s
//packages/router:router_api (cached) PASSED in 19.9s
//packages/service-worker:service-worker_api (cached) PASSED in 18.1s
//packages/upgrade:upgrade_api (cached) PASSED in 13.5s
```
Reference: DEV-71
PR Close#36034
In some scenarios it is useful for the developer to indicate
to ngcc that it should not use the entry-point manifest
file, and instead write a new one.
In the ngcc command line tool, this option is set by specfying
```
--invalidate-entry-point-manifest
```
PR Close#35931
The `DirectoryWalkerEntryPointFinder` has to traverse the
entire node_modules library everytime it executes in order to
identify the entry-points that need to be processed. This is
very time consuming (several seconds for big projects on
Windows).
This commit changes the `DirectoryWalkerEntryPointFinder` to
use the `EntryPointManifest` to store the paths to entry-points
that were found when doing this initial node_modules traversal
in a file to be reused for subsequent calls.
This dramatically speeds up ngcc processing when it has been run once
already.
PR Close#35931
The new `EntryPointManifest` class can read and write a
manifest file that contains all the paths to the entry-points
that have been found in a node_modules folder.
This can be used to speed up finding entry-points in
subsequent runs.
The manifest file stores the ngcc version and hashes of
the package lock-file and project config, since if these
change the manifest will need to be recomputed.
PR Close#35931
Currently, when running the ngcc binary directly and provide an invalid option ngcc will not error out and the user might have a hard time telling why ngcc is behaving not as expected.
With this change we now output an actionable error:
```
yarn ngcc --unknown-option
Options:
--version Show version number [boolean]
-s, --source A path (relative to the working directory)
of the `node_modules` folder to process.
[default: "./node_modules"]
-p, --properties An array of names of properties in
package.json to compile (e.g. `module` or
`es2015`)
Each of these properties should hold the
path to a bundle-format.
If provided, only the specified properties
are considered for processing.
If not provided, all the supported format
properties (e.g. fesm2015, fesm5, es2015,
esm2015, esm5, main, module) in the
package.json are considered. [array]
-t, --target A relative path (from the `source` path) to
a single entry-point to process (plus its
dependencies).
--first-only If specified then only the first matching
package.json property will be compiled.
[boolean]
--create-ivy-entry-points If specified then new `*_ivy_ngcc`
entry-points will be added to package.json
rather than modifying the ones in-place.
For this to work you need to have custom
resolution set up (e.g. in webpack) to look
for these new entry-points.
The Angular CLI does this already, so it is
safe to use this option if the project is
being built via the CLI. [boolean]
--legacy-message-ids Render `$localize` messages with legacy
format ids.
The default value is `true`. Only set this
to `false` if you do not want legacy
message ids to
be rendered. For example, if you are not
using legacy message ids in your
translation files
AND are not doing compile-time inlining of
translations, in which case the extra
message ids
would add unwanted size to the final source
bundle.
It is safe to leave this set to true if you
are doing compile-time inlining because the
extra
legacy message ids will all be stripped
during translation.
[boolean] [default: true]
--async Whether to compile asynchronously. This is
enabled by default as it allows
compilations to be parallelized.
Disabling asynchronous compilation may be
useful for debugging.
[boolean] [default: true]
-l, --loglevel The lowest severity logging message that
should be output.
[choices: "debug", "info", "warn", "error"]
--invalidate-entry-point-manifest If this is set then ngcc will not read an
entry-point manifest file from disk.
Instead it will walking the directory tree
as normal looking for entry-points, and
then write a new manifest file.
[boolean] [default: false]
--help Show help [boolean]
Unknown arguments: unknown-option, unknownOption
```
PR Close#36010
Moves the public api .d.ts files from tools/public_api_guard to
goldens/public-api.
Additionally, provides a README in the goldens directory and a script
assist in testing the current state of the repo against the goldens as
well as a command for accepting all changes to the goldens in a single
command.
PR Close#35768
This commit adds support in the Angular monorepo and in the Angular
compiler(s) for TypeScript 3.8. All packages can now compile with
TS 3.8.
For most of the repo, only a handful few typings adjustments were needed:
* TS 3.8 has a new `CustomElementConstructor` DOM type, which enforces a
zero-argument constructor. The `NgElementConstructor` type previously
declared a required `injector` argument despite the fact that its
implementation allowed `injector` to be optional. The interface type was
updated to reflect the optionality of the argument.
* Certain error messages were changed, and expectations in tests were
updated as a result.
* tsserver (part of language server) now returns performance information in
responses, so test expectations were changed to only assert on the actual
body content of responses.
For compiler-cli and schematics (which use the TypeScript AST) a major
breaking change was the introduction of the export form:
```typescript
export * as foo from 'bar';
```
This is a `ts.NamespaceExport`, and the `exportClause` of a
`ts.ExportDeclaration` can now take this type as well as `ts.NamedExports`.
This broke a lot of places where `exportClause` was assumed to be
`ts.NamedExports`.
For the most part these breakages were in cases where it is not necessary
to handle the new `ts.NamedExports` anyway. ngtsc's design uses the
`ts.TypeChecker` APIs to understand syntax and so automatically supports the
new form of exports.
The View Engine compiler on the other hand extracts TS structures into
metadata.json files, and that format was not designed for namespaced
exports. As a result it will take a nontrivial amount of work if we want to
support such exports in View Engine. For now, these new exports are not
accounted for in metadata.json, and so using them in "folded" Angular
expressions will result in errors (probably claiming that the referenced
exported namespace doesn't exist).
Care was taken to only use TS APIs which are present in 3.7/3.6, as Angular
needs to remain compatible with these for the time being.
This commit does not update angular.io.
PR Close#35864
Prior to this commit, while calculating the scope for a module, Ivy compiler processed `declarations` field first and `imports` after that. That results in a couple issues:
* for Pipes with the same `name` and present in `declarations` and in an imported module, Pipe from imported module was selected. In View Engine the logic is opposite: Pipes from `declarations` field receive higher priority.
* for Directives with the same selector and present in `declarations` and in an imported module, we first invoked the logic of a Directive from `declarations` field and after that - imported Directive logic. In View Engine, it was the opposite and the logic of a Directive from the `declarations` field was invoked last.
In order to align Ivy and View Engine behavior, this commit updates the logic in which we populate module scope: we first process all imports and after that handle `declarations` field. As a result, in Ivy both use-cases listed above work similar to View Engine.
Resolves#35502.
PR Close#35850
This commit splits the ngtsc `core` package's api entrypoint, which
previously was a single `api.ts` file, into an api/ directory with multiple
files. This is done to isolate the parts of the API definitions pertaining
to the public-facing `angularCompilerOptions` field in tsconfig.json into a
single file, which will enable a public API guard test to be added in a
future commit.
PR Close#35885
Currently, when Angular code is built with Bazel and with Ivy, generated
factory shims (.ngfactory files) are not processed via the majority of
tsickle's transforms. This is a subtle effect of the build infrastructure,
but it boils down to a TsickleHost method `shouldSkipTsickleProcessing`.
For ngc_wrapped builds (Bazel + Angular), this method is defined in the
`@bazel/typescript` (aka bazel rules_typescript) implementation of
`CompilerHost`. The default behavior is to skip tsickle processing for files
which are not present in the original `srcs[]` of the build rule. In
Angular's case, this includes all generated shim files.
For View Engine factories this is probably desirable as they're quite
complex and they've never been tested with tsickle. Ivy factories however
are smaller and very straightforward, and it makes sense to treat them like
any other output.
This commit adjusts two independent implementations of
`shouldSkipTsickleProcessing` to enable transformation of Ivy shims:
* in `@angular/bazel` aka ngc_wrapped, the upstream `@bazel/typescript`
`CompilerHost` is patched to treat .ngfactory files the same as their
original source file, with respect to tsickle processing.
It is currently not possible to test this change as we don't have any test
that inspects tsickle output with bazel. It will be extensively tested in
g3.
* in `ngc`, Angular's own implementation is adjusted to allow for the
processing of shims when compiling with Ivy. This enables a unit test to
be written to validate the correct behavior of tsickle when given a host
that's appropriately configured to process factory shims.
For ngtsc-as-a-plugin, a similar fix will need to be submitted upstream in
tsc_wrapped.
PR Close#35848
This version of `LockFile` creates an "unlocker" child-process that monitors
the main ngcc process and deletes the lock file if it exits unexpectedly.
This resolves the issue where the main process could not be killed by pressing
Ctrl-C at the terminal.
Fixes#35761
PR Close#35861
The previous implementation mixed up the management
of locking a piece of code (both sync and async) with the
management of writing and removing the lockFile that is
used as the flag for which process has locked the code.
This change splits these two concepts up. Apart from
avoiding the awkward base class it allows the `LockFile`
implementation to be replaced cleanly.
PR Close#35861
JoostK
George Kalpakas
Pete Bacon Darwin
ayazhafiz
Alex Rickabaugh
Keen Yee Liau
Andrew Kushnir
Alan Agius
Joey Perrott
Matias Niemelä