In Angular programs, changing a file may require other files to be
emitted as well due to implicit NgModule dependencies. For example, if
the selector of a directive is changed then all components that have
that directive in their compilation scope need to be recompiled, as the
change of selector may affect the directive matching results.
Until now, the compiler solved this problem using a single dependency
graph. The implicit NgModule dependencies were represented in this
graph, such that a changed file would correctly also cause other files
to be re-emitted. This approach is limited in a few ways:
1. The file dependency graph is used to determine whether it is safe to
reuse the analysis data of an Angular decorated class. This analysis
data is invariant to unrelated changes to the NgModule scope, but
because the single dependency graph also tracked the implicit
NgModule dependencies the compiler had to consider analysis data as
stale far more often than necessary.
2. It is typical for a change to e.g. a directive to not affect its
public API—its selector, inputs, outputs, or exportAs clause—in which
case there is no need to re-emit all declarations in scope, as their
compilation output wouldn't have changed.
This commit implements a mechanism by which the compiler is able to
determine the impact of a change by comparing it to the prior
compilation. To achieve this, a new graph is maintained that tracks all
public API information of all Angular decorated symbols. During an
incremental compilation this information is compared to the information
that was captured in the most recently succeeded compilation. This
determines the exact impact of the changes to the public API, which
is then used to determine which files need to be re-emitted.
Note that the file dependency graph remains, as it is still used to
track the dependencies of analysis data. This graph does no longer track
the implicit NgModule dependencies, which allows for better reuse of
analysis data.
These changes also fix a bug where template type-checking would fail to
incorporate changes made to a transitive base class of a
directive/component. This used to be a problem because transitive base
classes were not recorded as a transitive dependency in the file
dependency graph, such that prior type-check blocks would erroneously
be reused.
This commit also fixes an incorrectness where a change to a declaration
in NgModule `A` would not cause the declarations in NgModules that
import from NgModule `A` to be re-emitted. This was intentionally
incorrect as otherwise the performance of incremental rebuilds would
have been far worse. This is no longer a concern, as the compiler is now
able to only re-emit when actually necessary.
Fixes#34867Fixes#40635Closes#40728
PR Close#40947
A couple reasons to justify removing the flag:
* It adds code to the compiler that is only meant to support test cases
and not any production. We should avoid code in that's only
meant to support tests.
* The flag enables writing tests that do not mimic real-world behavior
because they allow invalid applications
PR Close#40013
Previously, if a component had an external template with a hard error, the
compiler would "forget" the link between that component and its NgModule.
Additionally, the NgModule would be marked as being in error, because the
template issue would prevent the compiler from registering the component
class as a component, so from the NgModule it would look like a declaration
of a non-directive/pipe class. As a combined result, the next incremental
step could fix the template error, but would not refresh diagnostics for the
NgModule, leading to an incrementality issue.
The various facets of this problem were fixed in prior commits. This commit
adds a test verifying the above case works now as expected.
PR Close#39923
ngtsc has a robust suite of testing utilities, designed for in-memory
testing of a TypeScript compiler. Previously these utilities lived in the
`test` directory for the compiler-cli package.
This commit moves those utilities to an `ngtsc/testing` package, enabling
them to be depended on separately and opening the door for using them from
the upcoming language server testing infrastructure.
As part of this refactoring, the `fake_core` package (a lightweight API
replacement for @angular/core) is expanded to include functionality needed
for Language Service test use cases.
PR Close#39594
Previously, the compiler performed an incremental build by analyzing and
resolving all classes in the program (even unchanged ones) and then using
the dependency graph information to determine which .js files were stale and
needed to be re-emitted. This algorithm produced "correct" rebuilds, but the
cost of re-analyzing the entire program turned out to be higher than
anticipated, especially for component-heavy compilations.
To achieve performant rebuilds, it is necessary to reuse previous analysis
results if possible. Doing this safely requires knowing when prior work is
viable and when it is stale and needs to be re-done.
The new algorithm implemented by this commit is such:
1) Each incremental build starts with knowledge of the last known good
dependency graph and analysis results from the last successful build,
plus of course information about the set of files changed.
2) The previous dependency graph's information is used to determine the
set of source files which have "logically" changed. A source file is
considered logically changed if it or any of its dependencies have
physically changed (on disk) since the last successful compilation. Any
logically unchanged dependencies have their dependency information copied
over to the new dependency graph.
3) During the `TraitCompiler`'s loop to consider all source files in the
program, if a source file is logically unchanged then its previous
analyses are "adopted" (and their 'register' steps are run). If the file
is logically changed, then it is re-analyzed as usual.
4) Then, incremental build proceeds as before, with the new dependency graph
being used to determine the set of files which require re-emitting.
This analysis reuse avoids template parsing operations in many circumstances
and significantly reduces the time it takes ngtsc to rebuild a large
application.
Future work will increase performance even more, by tackling a variety of
other opportunities to reuse or avoid work.
PR Close#34288
Previously, our incremental build system kept track of the changes between
the current compilation and the previous one, and used its knowledge of
inter-file dependencies to evaluate the impact of each change and emit the
right set of output files.
However, a problem arose if the compiler was not able to extract a
dependency graph successfully. This typically happens if the input program
contains errors. In this case the Angular analysis part of compilation is
never executed.
If a file changed in one of these failed builds, in the next build it
appears unchanged. This means that the compiler "forgets" to emit it!
To fix this problem, the compiler needs to know the set of changes made
_since the last successful build_, not simply since the last invocation.
This commit changes the incremental state system to much more explicitly
pass information from the previous to the next compilation, and in the
process to keep track of changes across multiple failed builds, until the
program can be analyzed successfully and the results of those changes
incorporated into the emit plan.
Fixes#32214
PR Close#33971
Previously, the ngtsc compiler attempted to reuse analysis work from the
previous program during an incremental build. To do this, it had to prove
that the work was safe to reuse - that no changes made to the new program
would invalidate the previous analysis.
The implementation of this had a significant design flaw: if the previous
program had errors, the previous analysis would be missing significant
information, and the dependency graph extracted from it would not be
sufficient to determine which files should be re-analyzed to fill in the
gaps. This often meant that the build output after an error was resolved
would be wholly incorrect.
This commit switches ngtsc to take a simpler approach to incremental
rebuilds. Instead of attempting to reuse prior analysis work, the entire
program is re-analyzed with each compilation. This is actually not as
expensive as one might imagine - analysis is a fairly small part of overall
compilation time.
Based on the dependency graph extracted during this analysis, the compiler
then can make accurate decisions on whether to emit specific files. A new
suite of tests is added to validate behavior in the presence of source code
level errors.
This new approach is dramatically simpler than the previous algorithm, and
should always produce correct results for a semantically correct program.s
Fixes#32388Fixes#32214
PR Close#33862
JoostK
Andrew Scott
Alex Rickabaugh
Joey Perrott