When multiple occurrences of the same package exist within a single
TypeScript compilation unit, TypeScript deduplicates the source files
by introducing redirected source file proxies. Such proxies are
recreated during an incremental compilation even if the original
declaration file did not change, which caused the compiler not to reuse
any work from the prior compilation.
This commit changes the incremental driver to recognize a redirected
source file and treat them as their unredirected source file.
PR Close#41448
The Angular compiler has to actively keep default import statements
alive if they were only used in type-only positions, but have been
emitted as value expressions for DI purposes. A problem occurred in
incremental recompilations, where the relationship between an identifier
usage and its corresponding default import would not be considered. This
could result in the removal of the default import statement and caused
a `ReferenceError` at runtime.
This commit fixes the issue by storing the association from an
identifier to its default import declaration on the source file itself,
instead of within the `DefaultImportTracker` instance. The
`DefaultImportTracker` instance is only valid for a single compilation,
whereas the association from an identifier to a default import
declaration is valid as long as the `ts.SourceFile` is the same
instance.
A subsequent commit refactor the `DefaultImportTracker` to no longer
be responsible for registering the association, as its lifetime is
conceptually too short to do so.
Fixes#41377
PR Close#41557
In Angular programs, changing a file may require other files to be
emitted as well due to implicit NgModule dependencies. For example, if
the selector of a directive is changed then all components that have
that directive in their compilation scope need to be recompiled, as the
change of selector may affect the directive matching results.
Until now, the compiler solved this problem using a single dependency
graph. The implicit NgModule dependencies were represented in this
graph, such that a changed file would correctly also cause other files
to be re-emitted. This approach is limited in a few ways:
1. The file dependency graph is used to determine whether it is safe to
reuse the analysis data of an Angular decorated class. This analysis
data is invariant to unrelated changes to the NgModule scope, but
because the single dependency graph also tracked the implicit
NgModule dependencies the compiler had to consider analysis data as
stale far more often than necessary.
2. It is typical for a change to e.g. a directive to not affect its
public API—its selector, inputs, outputs, or exportAs clause—in which
case there is no need to re-emit all declarations in scope, as their
compilation output wouldn't have changed.
This commit implements a mechanism by which the compiler is able to
determine the impact of a change by comparing it to the prior
compilation. To achieve this, a new graph is maintained that tracks all
public API information of all Angular decorated symbols. During an
incremental compilation this information is compared to the information
that was captured in the most recently succeeded compilation. This
determines the exact impact of the changes to the public API, which
is then used to determine which files need to be re-emitted.
Note that the file dependency graph remains, as it is still used to
track the dependencies of analysis data. This graph does no longer track
the implicit NgModule dependencies, which allows for better reuse of
analysis data.
These changes also fix a bug where template type-checking would fail to
incorporate changes made to a transitive base class of a
directive/component. This used to be a problem because transitive base
classes were not recorded as a transitive dependency in the file
dependency graph, such that prior type-check blocks would erroneously
be reused.
This commit also fixes an incorrectness where a change to a declaration
in NgModule `A` would not cause the declarations in NgModules that
import from NgModule `A` to be re-emitted. This was intentionally
incorrect as otherwise the performance of incremental rebuilds would
have been far worse. This is no longer a concern, as the compiler is now
able to only re-emit when actually necessary.
Fixes#34867Fixes#40635Closes#40728
PR Close#40947
ngtsc has a robust suite of testing utilities, designed for in-memory
testing of a TypeScript compiler. Previously these utilities lived in the
`test` directory for the compiler-cli package.
This commit moves those utilities to an `ngtsc/testing` package, enabling
them to be depended on separately and opening the door for using them from
the upcoming language server testing infrastructure.
As part of this refactoring, the `fake_core` package (a lightweight API
replacement for @angular/core) is expanded to include functionality needed
for Language Service test use cases.
PR Close#39594
With the introduction of incremental type checking in #36211, an
intermediate `ts.Program` for type checking is only created if there are
any templates to check. This rendered some tests ineffective at avoiding
regressions, as the intermediate `ts.Program` was required for the tests
to fail if the scenario under test would not be accounted for. This
commit adds a single component to these tests, to ensure the
intermediate `ts.Program` is in fact created.
PR Close#39011
Prior to this fix, incremental rebuilds could fail to type check due to
missing ambient types from auto-discovered declaration files in @types
directories, or type roots in general. This was caused by the
intermediary `ts.Program` that is created for template type checking,
for which a `ts.CompilerHost` was used which did not implement the
optional `directoryExists` methods. As a result, auto-discovery of types
would not be working correctly, and this would retain into the
`ts.Program` that would be created for an incremental rebuild.
This commit fixes the issue by forcing the custom `ts.CompilerHost` used
for type checking to properly delegate into the original
`ts.CompilerHost`, even for optional methods. This is accomplished using
a base class `DelegatingCompilerHost` which is typed in such a way that
newly introduced `ts.CompilerHost` methods must be accounted for.
Fixes#38979
PR Close#39011
This optimization builds on a lot of prior work to finally make type-
checking of templates incremental.
Incrementality requires two main components:
- the ability to reuse work from a prior compilation.
- the ability to know when changes in the current program invalidate that
prior work.
Prior to this commit, on every type-checking pass the compiler would
generate new .ngtypecheck files for each original input file in the program.
1. (Build #1 main program): empty .ngtypecheck files generated for each
original input file.
2. (Build #1 type-check program): .ngtypecheck contents overridden for those
which have corresponding components that need type-checked.
3. (Build #2 main program): throw away old .ngtypecheck files and generate
new empty ones.
4. (Build #2 type-check program): same as step 2.
With this commit, the `IncrementalDriver` now tracks template type-checking
_metadata_ for each input file. The metadata contains information about
source mappings for generated type-checking code, as well as some
diagnostics which were discovered at type-check analysis time. The actual
type-checking code is stored in the TypeScript AST for type-checking files,
which is now re-used between programs as follows:
1. (Build #1 main program): empty .ngtypecheck files generated for each
original input file.
2. (Build #1 type-check program): .ngtypecheck contents overridden for those
which have corresponding components that need type-checked, and the
metadata registered in the `IncrementalDriver`.
3. (Build #2 main program): The `TypeCheckShimGenerator` now reuses _all_
.ngtypecheck `ts.SourceFile` shims from build #1's type-check program in
the construction of build #2's main program. Some of the contents of
these files might be stale (if a component's template changed, for
example), but wholesale reuse here prevents unnecessary changes in the
contents of the program at this point and makes TypeScript's job a lot
easier.
4. (Build #2 type-check program): For those input files which have not
"logically changed" (meaning components within are semantically the same
as they were before), the compiler will re-use the type-check file
metadata from build #1, and _not_ generate a new .ngtypecheck shim.
For components which have logically changed or where the previous
.ngtypecheck contents cannot otherwise be reused, code generation happens
as before.
PR Close#36211
The compiler needs to track the dependencies of a component, including any
NgModules which happen to be present in a component's scope. If an upstream
NgModule changes, any downstream components need to have their templates
re-compiled and re-typechecked.
Previously, the compiler handled this well for the A -> B -> C case where
module A imports module B which re-exports module C. However, it fell apart
in the A -> B -> C -> D case, because previously tracking focused on changes
to components/directives in the scope, and not NgModules specifically.
This commit introduces logic to track which NgModules contributed to a given
scope, and treat them as dependencies of any components within.
This logic also contains a bug, which is intentional for now. It
purposefully does not track transitive dependencies of the NgModules which
contribute to a scope. If it did, using the current dependency system, this
would treat all components and directives (even those not exported into the
scope) as dependencies, causing a major performance bottleneck. Only those
dependencies which contributed to the module's export scope should be
considered, but the current system is incapable of making this distinction.
This will be fixed at a later date.
PR Close#36211
This commit fixes a bug in the incremental rebuild engine of ngtsc, where if
a component was removed from its NgModule, it would not be properly
re-emitted.
The bug stemmed from the fact that whether to emit a file was a decision
based purely on the updated dependency graph, which captures the dependency
structure of the rebuild program. This graph has no edge from the component
to its former module (as it was removed, of course), so the compiler
erroneously decides not to emit the component.
The bug here is that the compiler does know, from the previous dependency
graph, that the component file has logically changed, since its previous
dependency (the module file) has changed. This information was not carried
forward into the set of files which need to be emitted, because it was
assumed that the updated dependency graph was a more accurate source of that
information.
With this commit, the set of files which need emit is pre-populated with the
set of logically changed files, to cover edge cases like this.
Fixes#34813
PR Close#34912
Previously, the compiler performed an incremental build by analyzing and
resolving all classes in the program (even unchanged ones) and then using
the dependency graph information to determine which .js files were stale and
needed to be re-emitted. This algorithm produced "correct" rebuilds, but the
cost of re-analyzing the entire program turned out to be higher than
anticipated, especially for component-heavy compilations.
To achieve performant rebuilds, it is necessary to reuse previous analysis
results if possible. Doing this safely requires knowing when prior work is
viable and when it is stale and needs to be re-done.
The new algorithm implemented by this commit is such:
1) Each incremental build starts with knowledge of the last known good
dependency graph and analysis results from the last successful build,
plus of course information about the set of files changed.
2) The previous dependency graph's information is used to determine the
set of source files which have "logically" changed. A source file is
considered logically changed if it or any of its dependencies have
physically changed (on disk) since the last successful compilation. Any
logically unchanged dependencies have their dependency information copied
over to the new dependency graph.
3) During the `TraitCompiler`'s loop to consider all source files in the
program, if a source file is logically unchanged then its previous
analyses are "adopted" (and their 'register' steps are run). If the file
is logically changed, then it is re-analyzed as usual.
4) Then, incremental build proceeds as before, with the new dependency graph
being used to determine the set of files which require re-emitting.
This analysis reuse avoids template parsing operations in many circumstances
and significantly reduces the time it takes ngtsc to rebuild a large
application.
Future work will increase performance even more, by tackling a variety of
other opportunities to reuse or avoid work.
PR Close#34288
During incremental compilations, ngtsc needs to know which metadata
from a previous compilation can be reused, versus which metadata has to
be recomputed as some dependency was updated. Changes to
directives/components should cause the NgModule in which they are
declared to be recompiled, as the NgModule's compilation is dependent
on its directives/components.
When a dependent source file of a directive/component is updated,
however, a more subtle dependency should also cause to NgModule's source
file to be invalidated. During the reconciliation of state from a
previous compilation into the new program, the component's source file
is invalidated because one of its dependency has changed, ergo the
NgModule needs to be invalidated as well. Up until now, this implicit
dependency was not imposed on the NgModule. Additionally, any change to
a dependent file may influence the module scope to change, so all
components within the module must be invalidated as well.
This commit fixes the bug by introducing additional file dependencies,
as to ensure a proper rebuild of the module scope and its components.
Fixes#32416
PR Close#33522
Previously if only a component template changed then we would know to
rebuild its component source file. But the compilation was incorrect if the
component was part of an NgModule, since we were not capturing the
compilation scope information that had a been acquired from the NgModule
and was not being regenerated since we were not needing to recompile
the NgModule.
Now we register compilation scope information for each component, via the
`ComponentScopeRegistry` interface, so that it is available for incremental
compilation.
The `ComponentDecoratorHandler` now reads the compilation scope from a
`ComponentScopeReader` interface which is implemented as a compound
reader composed of the original `LocalModuleScopeRegistry` and the
`IncrementalState`.
Fixes#31654
PR Close#31932
Versions of CLI prior to angular/angular-cli@0e339ee did not expose the host.getModifiedResourceFiles() method.
This meant that null was being passed through to the IncrementalState.reconcile() method
to indicate that there were either no changes or the host didn't support that method.
This commit fixes a bug where we were checking for undefined rather than null when
deciding whether any resource files had changed, causing a null reference error to be thrown.
This bug was not caught by the unit testing because the tests set up the changed files
via a slightly different process, not having access to the CompilerHost, and these test
were making the erroneous assumption that undefined indicated that there were no
changed files.
PR Close#31322
To improve cross platform support, all file access (and path manipulation)
is now done through a well known interface (`FileSystem`).
For testing a number of `MockFileSystem` implementations are provided.
These provide an in-memory file-system which emulates operating systems
like OS/X, Unix and Windows.
The current file system is always available via the static method,
`FileSystem.getFileSystem()`. This is also used by a number of static
methods on `AbsoluteFsPath` and `PathSegment`, to avoid having to pass
`FileSystem` objects around all the time. The result of this is that one
must be careful to ensure that the file-system has been initialized before
using any of these static methods. To prevent this happening accidentally
the current file system always starts out as an instance of `InvalidFileSystem`,
which will throw an error if any of its methods are called.
You can set the current file-system by calling `FileSystem.setFileSystem()`.
During testing you can call the helper function `initMockFileSystem(os)`
which takes a string name of the OS to emulate, and will also monkey-patch
aspects of the TypeScript library to ensure that TS is also using the
current file-system.
Finally there is the `NgtscCompilerHost` to be used for any TypeScript
compilation, which uses a given file-system.
All tests that interact with the file-system should be tested against each
of the mock file-systems. A series of helpers have been provided to support
such tests:
* `runInEachFileSystem()` - wrap your tests in this helper to run all the
wrapped tests in each of the mock file-systems.
* `addTestFilesToFileSystem()` - use this to add files and their contents
to the mock file system for testing.
* `loadTestFilesFromDisk()` - use this to load a mirror image of files on
disk into the in-memory mock file-system.
* `loadFakeCore()` - use this to load a fake version of `@angular/core`
into the mock file-system.
All ngcc and ngtsc source and tests now use this virtual file-system setup.
PR Close#30921
Optimizations to skip compiling source files that had not changed
did not account for the case where only a resource file changes,
such as an external template or style file.
Now we track such dependencies and trigger a recompilation
if any of the previously tracked resources have changed.
This will require a change on the CLI side to provide the list of
resource files that changed to trigger the current compilation by
implementing `CompilerHost.getModifiedResourceFiles()`.
Closes#30947
PR Close#30954
Now that the dependent files and compilation scopes are being tracked in
the incremental state, we can skip analysing and emitting source files if
none of their dependent files have changed since the last compile.
The computation of what files (and their dependencies) are unchanged is
computed during reconciliation.
This commit also removes the previous emission skipping logic, since this
approach covers those cases already.
PR Close#30238
ngtsc previously could attempt to reuse the main ts.Program twice. This
occurred when template type-checking was enabled and then an incremental
build was performed. This breaks a TypeScript invariant - ts.Programs can
only be reused once.
The creation of the template type-checking program reuses the main program,
rendering it moot. Then, on the next incremental build the main program
would be subject to reuse again, which would crash inside TypeScript.
This commit fixes the issue by reusing the template type-checking program
from the previous run on the next incremental build. Since under normal
circumstances the files in the type-checking program aren't changed, this
should be just as fast.
Testing strategy: a test is added in the incremental_spec which validates
that program reuse with type-checking turned on does not crash the compiler.
Fixes#30079
PR Close#30090
This commit introduces a mechanism for incremental compilation to the ngtsc
compiler.
Previously, incremental information was used in the construction of the
ts.Program for subsequent compilations, but was not used in ngtsc itself.
This commit adds an IncrementalState class, which tracks state between ngtsc
compilations. Currently, this supports skipping the TypeScript emit step
when the compiler can prove the contents of emit have not changed.
This is implemented for @Injectables as well as for files which don't
contain any Angular decorated types. These are the only files which can be
proven to be safe today.
See ngtsc/incremental/README.md for more details.
PR Close#29380
JoostK
Kristiyan Kostadinov
Alex Rickabaugh
Joey Perrott
Pete Bacon Darwin
Matt Lewis
Jon Wallsten