The cached file-system was implemented to speed up ngcc
processing, but in reality most files are not accessed many times
and there is no noticeable degradation in speed by removing it.
Benchmarking `ngcc -l debug` for AIO on a local machine
gave a range of 196-236 seconds with the cache and 197-224
seconds without the cache.
Moreover, when running in parallel mode, ngcc has a separate
file cache for each process. This results in excess memory usage.
Notably the master process, which only does analysis of entry-points
holds on to up to 500Mb for AIO when using the cache compared to
only around 30Mb when not using the cache.
Finally, the file-system cache being incorrectly primed with file
contents before being processed has been the cause of a number
of bugs. For example https://github.com/angular/angular-cli/issues/16860#issuecomment-614694269.
PR Close#36687
Currently, when running the ngcc binary directly and provide an invalid option ngcc will not error out and the user might have a hard time telling why ngcc is behaving not as expected.
With this change we now output an actionable error:
```
yarn ngcc --unknown-option
Options:
--version Show version number [boolean]
-s, --source A path (relative to the working directory)
of the `node_modules` folder to process.
[default: "./node_modules"]
-p, --properties An array of names of properties in
package.json to compile (e.g. `module` or
`es2015`)
Each of these properties should hold the
path to a bundle-format.
If provided, only the specified properties
are considered for processing.
If not provided, all the supported format
properties (e.g. fesm2015, fesm5, es2015,
esm2015, esm5, main, module) in the
package.json are considered. [array]
-t, --target A relative path (from the `source` path) to
a single entry-point to process (plus its
dependencies).
--first-only If specified then only the first matching
package.json property will be compiled.
[boolean]
--create-ivy-entry-points If specified then new `*_ivy_ngcc`
entry-points will be added to package.json
rather than modifying the ones in-place.
For this to work you need to have custom
resolution set up (e.g. in webpack) to look
for these new entry-points.
The Angular CLI does this already, so it is
safe to use this option if the project is
being built via the CLI. [boolean]
--legacy-message-ids Render `$localize` messages with legacy
format ids.
The default value is `true`. Only set this
to `false` if you do not want legacy
message ids to
be rendered. For example, if you are not
using legacy message ids in your
translation files
AND are not doing compile-time inlining of
translations, in which case the extra
message ids
would add unwanted size to the final source
bundle.
It is safe to leave this set to true if you
are doing compile-time inlining because the
extra
legacy message ids will all be stripped
during translation.
[boolean] [default: true]
--async Whether to compile asynchronously. This is
enabled by default as it allows
compilations to be parallelized.
Disabling asynchronous compilation may be
useful for debugging.
[boolean] [default: true]
-l, --loglevel The lowest severity logging message that
should be output.
[choices: "debug", "info", "warn", "error"]
--invalidate-entry-point-manifest If this is set then ngcc will not read an
entry-point manifest file from disk.
Instead it will walking the directory tree
as normal looking for entry-points, and
then write a new manifest file.
[boolean] [default: false]
--help Show help [boolean]
Unknown arguments: unknown-option, unknownOption
```
PR Close#36010
This commit moves the build-related scripts
(`build-ivy-npm-packages.js`, `build-packages-dist.js` and
`package-builder.js`) to a dedicated directory to keep the `scripts/`
directory cleaner.
It also moves the logic for building the `zone.js` package to a separate
script, `zone-js-builder.js`, to make it re-usable. A subsequent commit
will use it to build the `zone.js` package when building the Ivy Angular
packages as well.
PR Close#35780
* it's tricky to get out of the runfiles tree with `bazel test` as `BUILD_WORKSPACE_DIRECTORY` is not set but I employed a trick to read the `DO_NOT_BUILD_HERE` file that is one level up from `execroot` and that contains the workspace directory. This is experimental and if `bazel test //:test.debug` fails than `bazel run` is still guaranteed to work as `BUILD_WORKSPACE_DIRECTORY` will be set in that context
* test //integration:bazel_test and //integration:bazel-schematics_test exclusively
* run "exclusive" and "manual" bazel-in-bazel integration tests in their own CI job as they take 8m+ to execute
```
//integration:bazel-schematics_test PASSED in 317.2s
//integration:bazel_test PASSED in 167.8s
```
* Skip all integration tests that are now handled by angular_integration_test except the tests that are tracked for payload size; these are:
- cli-hello-world*
- hello_world__closure
* add & pin @babel deps as newer versions of babel break //packages/localize/src/tools/test:test
@babel/core dep had to be pinned to 7.6.4 or else //packages/localize/src/tools/test:test failed. Also //packages/localize uses @babel/generator, @babel/template, @babel/traverse & @babel/types so these deps were added to package.json as they were not being hoisted anymore from @babel/core transitive.
NB: integration/hello_world__systemjs_umd test must run with systemjs 0.20.0
NB: systemjs must be at 0.18.10 for legacy saucelabs job to pass
NB: With Bazel 2.0, the glob for the files to test `"integration/bazel/**"` is empty if integation/bazel is in .bazelignore. This glob worked under these conditions with 1.1.0. I did not bother testing with 1.2.x as not having integration/bazel in .bazelignore is correct.
PR Close#33927
Previously, while trying to build an `NgccReflectionHost`'s
`privateDtsDeclarationMap`, `computePrivateDtsDeclarationMap()` would
try to collect exported declarations from all source files of the
program (i.e. without checking whether they were within the target
package, as happens for declarations in `.d.ts` files).
Most of the time, that would not be a problem, because external packages
would be represented as `.d.ts` files in the program. But when an
external package had no typings, the JS files would be used instead. As
a result, the `ReflectionHost` would try to (unnecessarilly) parse the
file in order to extract exported declarations, which in turn would be
harmless in most cases.
There are certain cases, though, where the `ReflectionHost` would throw
an error, because it cannot parse the external package's JS file. This
could happen, for example, in `UmdReflectionHost`, which expects the
file to contain exactly one statement. See #34544 for more details on a
real-world failure.
This commit fixes the issue by ensuring that
`computePrivateDtsDeclarationMap()` will only collect exported
declarations from files within the target package.
Jira issue: [FW-1794](https://angular-team.atlassian.net/browse/FW-1794)
Fixes#34544
PR Close#34811
Previously, if `UmdRenderingFormatter#addImports()` was called with an
empty list of imports to add (i.e. no new imports were needed), it would
add trailing commas in several locations (arrays, function arguments,
function parameters), thus making the code imcompatible with legacy
browsers such as IE11.
This commit fixes it by ensuring that no trailing commas are added if
`addImports()` is called with an empty list of imports.
This is a follow-up to #34353.
Fixes#34525
PR Close#34545
This commit updates `@angular/material` and `@angular/cdk` to the
latest release candidate. Doing so exposed a bug in ngcc, which is fixed
by the preceding commit. Also the layout of these libraries changed, so
the checks in the integration test need a bit of tweaking.
PR Close#34212
Since we cannot run `yarn install` with the `--frozen-lockfile` option
(because we want to be able to install the locally built Angular
packages), integration project lockfiles are susceptible to getting
out-of-sync with the corresponding `package.json`. When this happens,
yarn will install the latest available version that satisfies the
version range specified in `package.json`.
This commit adds another line of defense, by specifying exact versions
for the dependencies in `package.json` files (i.e. `1.33.7` instead of
`^1.33.0`). While transitive dependencies will be unpinned, this still
ensures that the same version of direct dependencies will be installed
in case of an out-of-sync lockfile, thus reducing the probability of
random failures.
PR Close#33968
In the `integration_test` CircleCI job, we run `yarn install` on all
projects in the `integration/` directory. If a project has no lockfile
or if the lockfile is out-of-sync with the corresponding `package.json`
file, then the installed dependency versions are no longer pinned, which
can result in different versions being installed between different runs
of the same job (if, for example, a new version is released for a
package) and breaks hermeticity.
This could be prevented by using the `--frozen-lockfile` option with
`yarn install`, but this is not possible with the current setup, because
yarn needs to be able to install the locally built Angular packages,
whose checksums will be different from the ones in the lockfile.
Therefore, we have to manually ensure that the lockfiles remain in-sync
with the corresponding `package.json` files for the rest of the
dependencies.
For example, previously, [cli-hello-world-lazy/yarn.lock][1] had an
entry for `@angular-devkit/build-angular@0.900.0-next.9` (pinned to
`0.900.0-next.9`), but [cli-hello-world-lazy/package.json][2] specified
the `@angular-devkit/build-angular` version as `^0.900.0-rc.0` (note the
leading caret). As a result, since the version in the lock file does not
much the one in `package.json`, the lockfile is ignored and the latest
available version that matches `^0.900.0-rc.0` is installed.
This, for example, started causing unrelated CI failures ([example][3]),
when `@angular-devkit/build-angular@9.0.0-rc.3` was released with a size
improvement.
This commit ensures that all integration projects have a lockfile and
that lockfiles are up-to-date (with the current `package.json` files).
[1]: https://github.com/angular/angular/blob/fc2f6b845/integration/cli-hello-world-lazy/yarn.lock#L13
[2]: https://github.com/angular/angular/blob/fc2f6b845/integration/cli-hello-world-lazy/package.json#L26
[3]: https://circleci.com/gh/angular/angular/535959#tests/containers/2
PR Close#33968
In #33823, `scripts/package-builds.sh` (which is used by both
`build-packages-dist.sh` and `build-ivy-npm-packages.sh`) was updated to
use `realpath`. It turns out that `realpath` does not exist on macOS, so
the build scripts do not work there.
In order to fix this (and also reduce the likelihood of introducing
similar issues in the future), this commit changes these bash scripts to
Node.js scripts (using [ShellJS](https://github.com/shelljs/shelljs) for
a cross-platform implementation of Unix shell commands where necessary).
PR Close#33854
Previously, ngcc's `Renderer` would add some constants in the processed
files which were emitted as ES2015 code (e.g. `const` declarations).
This would result in invalid ES5 generated code that would break when
run on browsers that do not support the emitted format.
This commit fixes it by adding a `printStatement()` method to
`RenderingFormatter`, which can convert statements to JavaScript code in
a suitable format for the corresponding `RenderingFormatter`.
Additionally, the `translateExpression()` and `translateStatement()`
ngtsc helper methods are augmented to accept an extra hint to know
whether the code needs to be translated to ES5 format or not.
Fixes#32665
PR Close#33514
In #33046, internal uses of `zone.js` were switched to reference it
directly from source (built with Bazel) instead of npm. As a result, the
necessary scripts were updated to build `zone.js` as necessary. However,
some `integration/**/debug-test.sh` scripts were missed (apparently
because they are not used on CI, but only locally as helpers for
debugging the integration projects).
This commit updates the `scripts/build-packages-dist.sh` script to also
build `zone.js`, so that other scripts (such as the various
`debug-test.sh` scripts) can use it.
PR Close#33733
Removes `ngBaseDef` from the compiler and any runtime code that was still referring to it. In the cases where we'd previously generate a base def we now generate a definition for an abstract directive.
PR Close#33264
Injector defs are not considered public API, so the property
that contains them should be prefixed with Angular's marker
for "private" ('ɵ') to discourage apps from relying on def
APIs directly.
This commit adds the prefix and shortens the name from
ngInjectorDef to inj. This is because property names
cannot be minified by Uglify without turning on property
mangling (which most apps have turned off) and are thus
size-sensitive.
PR Close#33151
Module defs are not considered public API, so the property
that contains them should be prefixed with Angular's marker
for "private" ('ɵ') to discourage apps from relying on def
APIs directly.
This commit adds the prefix and shortens the name from
ngModuleDef to mod. This is because property names
cannot be minified by Uglify without turning on property
mangling (which most apps have turned off) and are thus
size-sensitive.
PR Close#33142
Previously, the executable for the Angular Compatibility Compiler
(`ngcc`) was called `ivy-ngcc`. This would be confusing for users not
familiar with our internal terminology, especially given that we call it
`ngcc` in all our docs and presentations.
This commit renames the executable to `ngcc` and replaces `ivy-ngcc`
with a script that errors with an informative message (prompting the
user to use `ngcc` instead).
Jira issue: [FW-1624](https://angular-team.atlassian.net/browse/FW-1624)
PR Close#33140
`ngcc` supports both synchronous and asynchronous execution. The default
mode when using `ngcc` programmatically (which is how `@angular/cli` is
using it) is synchronous. When running `ngcc` from the command line
(i.e. via the `ivy-ngcc` script), it runs in async mode.
Previously, the work would be executed in the same way in both modes.
This commit improves the performance of `ngcc` in async mode by
processing tasks in parallel on multiple processes. It uses the Node.js
built-in [`cluster` module](https://nodejs.org/api/cluster.html) to
launch a cluster of Node.js processes and take advantage of multi-core
systems.
Preliminary comparisons indicate a 1.8x to 2.6x speed improvement when
processing the angular.io app (apparently depending on the OS, number of
available cores, system load, etc.). Further investigation is needed to
better understand these numbers and identify potential areas of
improvement.
Inspired by/Based on @alxhub's prototype: alxhub/angular@cb631bdb1
Original design doc: https://hackmd.io/uYG9CJrFQZ-6FtKqpnYJAA?view
Jira issue: [FW-1460](https://angular-team.atlassian.net/browse/FW-1460)
PR Close#32427
Bundle size changed in both zone.js(legacy) and zone-evergreen.js
- zone.js(legacy) package increased a little because the following feature and fixes.
1. #31699, handle MSPointer events PR
2. https://github.com/angular/zone.js/pull/1219 to add __zone_symbol__ customization support
- zone-evergreen.js package decreased because
1. the MSPointer PR only for legacy
2. the Object.defineProperty patch is moved to legacy #31660
PR Close#31975
In #31426 a fix was implemented to render namespaced decorator imports
correctly, however it turns out that the fix only worked when decorator
information was extracted from static properties, not when using
`__decorate` calls.
This commit fixes the issue by creating the decorator metadata with the
full decorator expression, instead of only its name.
Closes#31394
PR Close#31614
The support for decorators that were imported via a namespace,
e.g. `import * as core from `@angular/core` was implemented
piecemeal. This meant that it was easy to miss situations where
a decorator identifier needed to be handled as a namepsaced
import rather than a direct import.
One such issue was that UMD processing of decorators was not
correct: the namespace was being omitted from references to
decorators.
Now the types have been modified to make it clear that a
`Decorator.identifier` could hold a namespaced identifier,
and the corresponding code that uses these types has been
fixed.
Fixes#31394
PR Close#31426
Currently undecorated classes are intentionally not processed
with ngcc. This is causing unexpected behavior because decorator
handlers such as `base_def.ts` are specifically interested in class
definitions without top-level decorators, so that the base definition
can be generated if there are Angular-specific class members.
In order to ensure that undecorated base-classes work as expected
with Ivy, we need to run the decorator handlers for all top-level
class declarations (not only for those with decorators). This is similar
to when `ngtsc` runs decorator handlers when analyzing source-files.
Resolves FW-1355. Fixes https://github.com/angular/components/issues/16178
PR Close#30821
When ngcc processes an entrypoint, it updates `package.json` with
metadata about the processed format. Previously, it overwrote the
`package.json` with the stringified JSON object without spaces. This
made the file difficult to read (for example when looking at the file
while debugging an ngcc failure).
This commit fixes it by using spaces in the new `package.json` content.
PR Close#30831
There is an encoding issue with using delta `Δ`, where the browser will attempt to detect the file encoding if the character set is not explicitly declared on a `<script/>` tag, and Chrome will find the `Δ` character and decide it is window-1252 encoding, which misinterprets the `Δ` character to be some other character that is not a valid JS identifier character
So back to the frog eyes we go.
```
__
/ɵɵ\
( -- ) - I am ineffable. I am forever.
_/ \_
/ \ / \
== == ==
```
PR Close#30546
The `Δ` caused issue with other infrastructure, and we are temporarily
changing it to `ɵɵ`.
This commit also patches ts_api_guardian_test and AIO to understand `ɵɵ`.
PR Close#29850
Currently our plan is to skip the publish, docgen, and update steps for this package.
During RC, we'll determine if the breaking change is too difficult for users, in which case we might restore the package for another major.
PR Close#29550
If `targetEntryPointPath` is provided to `mainNgcc` then we will now mark all
the `propertiesToConsider` for that entry-point if we determine that
it does not contain code that was compiled by Angular (for instance it has
no `...metadata.json` file).
The commit also renames `__modified_by_ngcc__` to `__processed_by_ivy_ngcc__`, since
there may be entry-points that are marked despite ngcc not actually compiling anything.
PR Close#29092
You can now specify a list of properties in the package.json that
should be considered (in order) to find the path to the format to compile.
The build marker system has been updated to store the markers in
the package.json rather than an additional external file.
Also instead of tracking the underlying bundle format that was compiled,
it now tracks the package.json property.
BREAKING CHANGE:
The `proertiesToConsider` option replaces the previous `formats` option,
which specified the final bundle format, rather than the property in the
package.json.
If you were using this option to compile only specific bundle formats,
you must now modify your usage to pass in the properties in the package.json
that map to the format that you wish to compile.
In the CLI, the `--formats` is no longer available. Instead use the
`--properties` option.
FW-1120
PR Close#29092
In ESM5 decorated classes can be indicated by calls to `__decorate()`.
Previously the `ReflectionHost.findDecoratedClasses()` call would identify
helper calls of the form:
```
SomeClass = tslib_1.__decorate(...);
```
But it was missing calls of the form:
```
SomeClass = SomeClass_1 = tslib_1.__decorate(...);
```
This form is common in `@NgModule()` decorations, where the class
being decorated is referenced inside the decorator or another
member.
This commit now ensures that a chain of assignments, of any length,
is now identified as a class decoration if it results in a call to
`__decorate()`.
Fixes#27841
PR Close#27848
A constructor function may have been "synthesized" by TypeScript during
JavaScript emit, in the case no user-defined constructor exists and e.g.
property initializers are used. Those initializers need to be emitted
into a constructor in JavaScript, so the TypeScript compiler generates a
synthetic constructor.
This commit adds identification of such constructors as ngcc needs to be
able to tell if a class did originally have a constructor in the
TypeScript source. When a class has a superclass, a synthesized
constructor must not be considered as a user-defined constructor as that
prevents a base factory call from being created by ngtsc, resulting in a
factory function that does not inject the dependencies of the superclass.
Hence, we identify a default synthesized super call in the constructor
body, according to the structure that TypeScript emits.
PR Close#27897
When ngtsc compiles @angular/core, it rewrites core imports to the
r3_symbols.ts file that exposes all internal symbols under their
external name. When creating the FESM bundle, the r3_symbols.ts file
causes the external symbol names to be rewritten to their internal name.
Under ngcc compilations of FESM bundles, the indirection of
r3_symbols.ts is no longer in place such that the external names are
retained in the bundle. Previously, the external name `ɵdefineNgModule`
was explicitly declared internally to resolve this issue, but the
recently added `setClassMetadata` was not declared as such, causing
runtime errors.
Instead of relying on the r3_symbols.ts file to perform the rewrite of
the external modules to their internal variants, the translation is
moved into the `ImportManager` during the compilation itself. This
avoids the need for providing the external name manually.
PR Close#27055
This integration test was created from a vanilla CLI generated
project with the following modifications:
* remove `PercentPipe` usage from `app.component.html`
- these are not yet supported by ivy
* changed `ng test` in `package.json` to only to `ng build`
- right now we can only confirm that the app will build
* hard-code `ngDevMode` in `index.html`
- the CLI does not yet set this correctly
PR Close#26403
* rename test helper script
* add material to the ngcc integration test
* add MatButton to ngcc integration test checks
* remove platform-server from ngcc integration test
This package does not yet compile as it contains a package-private
(internal) decorated class, which the ngcc compiler does not yet
handle.
PR Close#26403
Originally, the ivy_switch mechanism used Bazel genrules to conditionally
compile one TS file or another depending on whether ngc or ngtsc was the
selected compiler. This was done because we wanted to avoid importing
certain modules (and thus pulling them into the build) if Ivy was on or
off. This mechanism had a major drawback: ivy_switch became a bottleneck
in the import graph, as it both imports from many places in the codebase
and is imported by many modules in the codebase. This frequently resulted
in cyclic imports which caused issues both with TS and Closure compilation.
It turns out ngcc needs both code paths in the bundle to perform the switch
during its operation anyway, so import switching was later abandoned. This
means that there's no real reason why the ivy_switch mechanism needed to
operate at the Bazel level, and for the ivy_switch file to be a bottleneck.
This commit removes the Bazel-level ivy_switch mechanism, and introduces
an additional TypeScript transform in ngtsc (and the pass-through tsc
compiler used for testing JIT) to perform the same operation that ngcc
does, and flip the switch during ngtsc compilation. This allows the
ivy_switch file to be removed, and the individual switches to be located
directly next to their consumers in the codebase, greatly mitigating the
circular import issues and making the mechanism much easier to use.
As part of this commit, the tag for marking switched variables was changed
from __PRE_NGCC__ to __PRE_R3__, since it's no longer just ngcc which
flips these tags. Most variables were renamed from R3_* to SWITCH_* as well,
since they're referenced mostly in render2 code.
Test strategy: existing test coverage is more than sufficient - if this
didn't work correctly it would break the hello world and todo apps.
PR Close#26550
Pete Bacon Darwin
JiaLiPassion
Alan Agius
George Kalpakas
Greg Magolan
crisbeto
Filipe Silva
Igor Minar
Kara Erickson
Misko Hevery
JoostK
Paul Gschwendtner
Ben Lesh
Alex Rickabaugh
Alex Eagle