As a performance optimization, this commit splits the single
__ngtypecheck__.ts file which was previously added to the user's program as
a container for all template type-checking code into multiple .ngtypecheck
shim files, one for each original file in the user's program.
In larger applications, the generation, parsing, and checking of this single
type-checking file was a huge performance bottleneck, with the file often
exceeding 1 MB in text content. Particularly in incremental builds,
regenerating this single file for the entire application proved especially
expensive.
This commit introduces a new strategy for template type-checking code which
makes use of a new interface, the `TypeCheckingProgramStrategy`. This
interface abstracts the process of creating a new `ts.Program` to type-check
a particular compilation, and allows the mechanism there to be kept separate
from the more complex logic around dealing with multiple .ngtypecheck files.
A new `TemplateTypeChecker` hosts that logic and interacts with the
`TypeCheckingProgramStrategy` to actually generate and return diagnostics.
The `TypeCheckContext` class, previously the workhorse of template type-
checking, is now solely focused on collecting and generating type-checking
file contents.
A side effect of implementing the new `TypeCheckingProgramStrategy` in this
way is that the API is designed to be suitable for use by the Angular
Language Service as well. The LS also needs to type-check components, but
has its own method for constructing a `ts.Program` with type-checking code.
Note that this commit does not make the actual checking of templates at all
_incremental_ just yet. That will happen in a future commit.
PR Close#36211
Shim generation was built on a lie.
Shims are files added to the program which aren't original files authored by
the user, but files authored effectively by the compiler. These fall into
two categories: files which will be generated (like the .ngfactory shims we
generate for View Engine compatibility) as well as files used internally in
compilation (like the __ng_typecheck__.ts file).
Previously, shim generation was driven by the `rootFiles` passed to the
compiler as input. These are effectively the `files` listed in the
`tsconfig.json`. Each shim generator (e.g. the `FactoryGenerator`) would
examine the `rootFiles` and produce a list of shim file names which it would
be responsible for generating. These names would then be added to the
`rootFiles` when the program was created.
The fatal flaw here is that `rootFiles` does not always account for all of
the files in the program. In fact, it's quite rare that it does. Users don't
typically specify every file directly in `files`. Instead, they rely on
TypeScript, during program creation, starting with a few root files and
transitively discovering all of the files in the program.
This happens, however, during `ts.createProgram`, which is too late to add
new files to the `rootFiles` list.
As a result, shim generation was only including shims for files actually
listed in the `tsconfig.json` file, and not for the transitive set of files
in the user's program as it should.
This commit completely rewrites shim generation to use a different technique
for adding files to the program, inspired by View Engine's shim generator.
In this new technique, as the program is being created and `ts.SourceFile`s
are being requested from the `NgCompilerHost`, shims for those files are
generated and a reference to them is patched onto the original file's
`ts.SourceFile.referencedFiles`. This causes TS to think that the original
file references the shim, and causes the shim to be included in the program.
The original `referencedFiles` array is saved and restored after program
creation, hiding this little hack from the rest of the system.
The new shim generation engine differentiates between two kinds of shims:
top-level shims (such as the flat module entrypoint file and
__ng_typecheck__.ts) and per-file shims such as ngfactory or ngsummary
files. The former are included via `rootFiles` as before, the latter are
included via the `referencedFiles` of their corresponding original files.
As a result of this change, shims are now correctly generated for all files
in the program, not just the ones named in `tsconfig.json`.
A few mitigating factors prevented this bug from being realized until now:
* in g3, `files` does include the transitive closure of files in the program
* in CLI apps, shims are not really used
This change also makes use of a novel technique for associating information
with source files: the use of an `NgExtension` `Symbol` to patch the
information directly onto the AST object. This is used in several
circumstances:
* For shims, metadata about a `ts.SourceFile`'s status as a shim and its
origins are held in the extension data.
* For original files, the original `referencedFiles` are stashed in the
extension data for later restoration.
The main benefit of this technique is a lot less bookkeeping around `Map`s
of `ts.SourceFile`s to various kinds of data, which need to be tracked/
invalidated as part of incremental builds.
This technique is based on designs used internally in the TypeScript
compiler and is serving as a prototype of this design in ngtsc. If it works
well, it could have benefits across the rest of the compiler.
PR Close#36211
The compiler needs to track the dependencies of a component, including any
NgModules which happen to be present in a component's scope. If an upstream
NgModule changes, any downstream components need to have their templates
re-compiled and re-typechecked.
Previously, the compiler handled this well for the A -> B -> C case where
module A imports module B which re-exports module C. However, it fell apart
in the A -> B -> C -> D case, because previously tracking focused on changes
to components/directives in the scope, and not NgModules specifically.
This commit introduces logic to track which NgModules contributed to a given
scope, and treat them as dependencies of any components within.
This logic also contains a bug, which is intentional for now. It
purposefully does not track transitive dependencies of the NgModules which
contribute to a scope. If it did, using the current dependency system, this
would treat all components and directives (even those not exported into the
scope) as dependencies, causing a major performance bottleneck. Only those
dependencies which contributed to the module's export scope should be
considered, but the current system is incapable of making this distinction.
This will be fixed at a later date.
PR Close#36211
Add a mechanism to replace file contents for a specific file. This
allows us to write custom test scenarios in code without modifying the
test project.
Since we are no longer mocking the language service host, the file
overwrite needs to happen via the project service.
Project service manages a set of script infos, and overwriting the files
is a matter of updating the relevant script infos.
Note that the actual project service is wrapped inside a Mock Service.
Tests should not have direct access to the project service. All
manipulations should take place via the Mock Service.
The MockService provides a `reset()` method to undo temporary overwrites
after each test.
PR Close#36923
Previously, some preview server tests were only running for public
builds. In the past, these tests were run for both public and non-public
builds. The non-public builds tests were disabled in #23576, probably
during debugging some failure.
This commit fixes it by ensuring the tests are run for both public and
non-public builds.
PR Close#36837
The test was introduced in #23576, but the behavior the test was
verifying does not match the actual behavior of
`BuildCleaner#getOpenPrNumbers()`.
The reason that the test did not fail is that the verification happened
asynchronously, but the test completed synchronously (by accident).
PR Close#36837
Previously, when the preview server `build-cleanup` script failed, the
error was logged but not reflected to the commands exit code. This seems
to have been accidentally broken in #23576.
This commit fixes it by ensuring the error is re-thrown from the
`BuildCleaner#cleanUp()` method to allow the process to exit with an
error exit code.
PR Close#36837
Previously, the `dev` npm script in `aio/aio-builds-setup/scripts-js/`
(the PR preview server implementation) would run both linting and unit
tests. This was slow and delayed the feedback loop on each change.
Since the `dev` script should be run during development and give
feedback as fast as possible, this commit removes the linting from the
`dev` script and only keeps the unit tests. Linting is still run in the
`test` npm script (which is more comprehensive). Also, in most cases the
developer's IDE will show linting errors in real time in the editor.
PR Close#36837
Update the order in which the `update-preview-server.sh` script expects
its arguments (and the associated docs) to be consistent with the order
of arguments in other commands/docs (such as
`vm-setup--start-docker-container.md`).
PR Close#36837
I recently went through the process of setting up a preview server VM
again and updated the instructions and references based on the latest
docs for Debian, Docker, Google Compute Engine, etc.
PR Close#36837
Previously, in order to remain as deterministic as possible, the
Dockerfile for the preview server Docker image had all dependencies
pinned to specific versions. It turns out that some packages (such as
`nginx`, `nodejs`, and `openssl` - potentially others too) make older
versions unavailable on the repositories once a newer version is
available.
See for example:
- https://github.com/nodesource/distributions/issues/33
- https://askubuntu.com/questions/715104/how-can-i-downgrade-openssl-via-apt-get
This commit, therefore, removes the exact versions for these packages.
The latest versions will be installed everytime the Docker image is
built (subject to Docker caching).
PR Close#36837
In order to ease local development, self-signed SSL/TLS certificates are
created when building the preview server Docker image. These
certificates are valid for 365 days. Thus, it is possible for an old
certificate to be re-used past its expiration date due to Docker's
caching intermediate layers.
Previously, this would lead to hard-to-debug failures in the
`aio-health-check` and `aio-verify-setup` checks. Even after finding out
that the failures were caused by an expired certificate, it was not
obvious why that would be the case.
This commit adds an additional check to the `aio-health-check` command
that checks the certificates' expiration dates. This helps surface such
errors. It also prints a more helpful message, prompting the user to
build the Docker image with the `--no-cache` option to fix the problem
with self-signed certificates.
PR Close#36837
Previously, the preview server Docker image was based on Debian 9
(stretch).
This commit upgrades the preview server Docker image to Debian 10
(buster) and also upgrades all dependencies to latest versions
(including upgrading Node.js from v10 to v12).
(The GCE VM running the preview server Docker container was also
upgraded from Debian 9 to 10 on 2020-04-27.)
---
Other changes:
- Pinned the installed version of `curl` to make the `aio-health-check`
and `aio-verify-setup` checks (which use `curl`) more deterministic.
- Dropped the `*-backports` Debian repositories, since they are no
longer needed. The `*-backports` repositories were introduced in
593fe5ed25 to install `nginx` from, but
became obsolete in 2f1a862b83, which
switched to installing `nginx` from the regular repositories again.
- Added `vim` to the list of installed dependencies (for convenience
during debugging).
PR Close#36837
This commit upgrades all dependencies in `scripts-js/` to latest
versions and also includes all necessary code changes to ensure the
tests are passing with the new dependency versions.
PR Close#36837
The legacy HTTP package was deprecated in v5 with the launch of
@angular/common/http. The legacy package hasn't been published
since v7, and will therefore not include a migration.
PR Close#27038
Remove TypeScript 3.6 and 3.7 support from Angular along with tests that
ensure those TS versions work.
BREAKING CHANGE: typescript 3.6 and 3.7 are no longer supported, please
update to typescript 3.8
PR Close#36329
The current benchmark for transplanted views only exercises the path
when the declaration location is dirty and the insertion is not. This
test adds a benchmark for when both insertion and declaration are dirty.
PR Close#36722
This commit fixes 2 separate issues related to root nodes retrieval from
embedded views with `<ng-content>`:
1) we did not account for the case where there were no projectable nodes
for a given `<ng-content>`;
2) we did not account for the case where projectable nodes for a given
`<ng-content>` were represented as an array of native nodes (happens in
the case of dynamically created components with projectable nodes);
Fixes#35967
PR Close#36051
A caching mechanism was put in place to prevent repeated calls to
the Github API. As the CI setup no longer relies on calls to the
Github API, this caching is no longer necessary.
It was discovered that this caching was causing a contention issue
for saucelabs testing as the same tunnel was being reused for
multiple jobs simultaneously. With this caching mechanism removed
the jobs will once again run via separate tunnels.
PR Close#36936
The `elements` tests were disabled on Saucelabs, because they were failing on IE10. The problem was that we were loading an es2015 file from npm directly, causing a syntax error. These changes transpile the file to es5.
PR Close#36929
Parse Angular compiler options in Angular language service.
In View Engine, only TypeScript compiler options are read, Angular
compiler options are not. With Ivy, there could be different modes of
compilation, most notably how strict the templates should be checked.
This commit makes the behavior of language service consistent with the
Ivy compiler.
PR Close#36922
Disables Bazel runfile tree creation. Only if a given
execution strategy relies on runfile tree creation, the
runfile tree is created lazily. This helps as currently
Bazel spends unnecessary time on CI building runfile trees
for tests which are cached and aren't re-run.
The goal is to spend less time on CI for cached test/build
targets. We can't measure impact yet as there are targets
for the integration tests that hide the potential benefits.
on the components repo a 80% time reduction could be observed.
PR Close#36914
Previously we were passing a string form of the value to pluralize
to the `getLocalePluralCase()` function that is extracted from the
locale data. But some locales have functions that rely upon this
value being a number not a string.
Now we convert the value to a number before passing it to the
locale data function.
Fixes#36888
PR Close#36901
Previously, using undecorated base classes and using
ModuleWithProviders without a generic were listed
as deprecated features.
In v10, these features will be removed and an error
will be thrown instead. This commit updates the
deprecation guide to reflect this change.
PR Close#36891
The index of the deprecation guide contains a list
of deprecated APIs and when they can be removed.
This commit updates the likely removal version for
APIs that were previously listed as v10, as we are
not removing them in this version.
PR Close#36891
Add documentation in the deprecations markdown file about the deprecation of IE 9 and 10.
Additionally, add note in browser support document about deprecation.
PR Close#36887
The `getLocation()` method was not working as there were typos in the
properties it was reading. This was not picked up because there were
neither typings for these properties nor unit tests to check it worked.
PR Close#36853
This commit also updates the projects to more closely match what a newly
generated app would look like with the exception of `tslint.json` files,
which would create too many linting failures. These will be updated in a
follow-up PR.
PR Close#36145
Previously, in the `test_aio` CI job, we ran ngcc before building the
app with `yarn build`. This was supposed to have the benefit of taking
advantage of the parallel capabilities of standalone ngcc (vs implicitly
running it via `ng build`).
It turns out that the work done by the standalone ngcc was thrown away
before the `ng build`, resulting in `ng build` having to run ngcc all
over again. This happened because the `yarn build` script (run after the
standalone ngcc step) also runs `yarn install`, which essentially cleans
up `node_modules/`, thus discarding all the work already done by ngcc.
Here is an [example CI job][1], where this can be seen in action:
One can see the "Compiling <some-package> : es2015 as esm2015" logs in
the `yarn --cwd aio ngcc --properties es2015` step (as the standalone
ngcc processes the various entry-points) and then see the same logs in
the `yarn --cwd aio build --progress=false` step (as ngcc has to process
the entry-points all over again).
This commit removes the redundant standalone ngcc run and lets the CLI
handle ngcc via `ng build`. It is possible to instrument the build
process in a way that we can run the standalone ngcc after
`yarn install` and thus take advantage of the performance gains in
parallel mode, but the latest version of the CLI can already run ngcc in
parallel mode as a pre-build step, so this is unnecessary.
[1]: https://circleci.com/gh/angular/angular/658691
PR Close#36145
Update the Angular CLI and Angular framework packages to latest `@next`
versions. Also, update the app to look more closely to how a newly
generated app with the latest CLI would look like.
PR Close#36145
This commit updates all payload sizes for angular.io to make it easier
to compare payload size changes as a result of upgrading Angular
packages and other dependencies in subsequent commits.
PR Close#36145
Previously, the behavior of the `minLength` and `maxLength` validators
caused confusion, as they appeared to work with numeric values but
did not in fact produce consistent results. This commit fixes the issue
by skipping validation altogether when a numeric value is used.
BREAKING CHANGES:
* The `minLength` and `maxLength` validators now verify that a value has
numeric `length` property and invoke validation only if that's the case.
Previously, falsey values without the length property (such as `0` or
`false` values) were triggering validation errors. If your code relies on
the old behavior, you can include other validators such as [min][1] or
[requiredTrue][2] to the list of validators for a particular field.
[1]: https://angular.io/api/forms/Validators#min
[2]: https://angular.io/api/forms/Validators#requiredTrueCloses#35591
PR Close#36157
This commit adds a method `overrideInlineTemplate` to the
`MockTypescriptHost`. This allows us to override an inline template
in a Component without changing the TypeScript parts. This methods works
in a similar way as `MockTypescriptHost.override()`, which is used for
overriding external template.
PR Close#36890
This function needs to deduplicate the paths that are found from the
paths mappings. Previously this deduplication was not linear and also
called the expensive `relative()` function many times.
This commit, suggested by @JoostK, reduces the complexity of the deduplication
by using a tree structure built from the segments of each path.
PR Close#36881
Previously the `basePaths` were computed when the finder was instantiated.
This was a waste of effort in the case that the targeted entry-point is already
processed.
This change makes the computation of `basePaths` lazy, so that the work is
only done if they are actually needed.
Fixes#36874
PR Close#36881
In TS 3.9 the compiler will start to wrap ES2015 classes in an IIFE to help with
tree-shaking when the class has "associated" statements.
E.g.
```ts
let PlatformLocation = /** @class */ (() => {
...
class PlatformLocation {
}
...
return PlatformLocation;
})();
```
This commit updates `Esm2015ReflectionHost` to support this format.
PR Close#36884
This commit adds a new mock host for testing the ivy language service.
Unlike the existing mock_host which mocks the LanguageServiceHost, the
Ivy mock host mocks just the filesystem interface, aka ts.ServerHost.
This is because Ivy language service requires an actual Project to
perform operations like adding synthetic typecheck files to the project,
and by extension, to the ts.Program. These requirements make the existing
mock host unsuitable to be reused.
This new testing structure also improves test performance, because the
old mock host copies (it actually creates symlinks, but still that's
relatively expensive due to the sheer number of files involved) all
@angular/* packages along with the typescript package to a temporary
node_modules directory. This is done every time setup() is called.
Instead, this new mock host just loads them from a pre-determined path
in Bazel runfiles.
PR Close#36879
Previously the setup-rbe.sh script did not allow accounts that did
have domains of angular.io or google.com. Since we add emails from
other domains into everyone@angular.io, we are unable to be certain
in the script that the account is not actually a member of the
required group. This change adds the option to choose to continue
with an email account logged in which we cannot verify by domain.
PR Close#36846
Migrates away from inline searching for files and running buildifier
directly, instead using ng-dev for formatting. Additionally, provides
a deprecation message for any usages of the previous commands.
PR Close#36842
Alex Rickabaugh
Keen Yee Liau
George Kalpakas
Adam Plumer
Andrew Scott
Pawel Kozlowski
Joey Perrott
crisbeto
Paul Gschwendtner
Pete Bacon Darwin
Walter Werner SCHNEIDER
Kara Erickson
David Shevitz 🔵
Sonu Kapoor
Issei Horie