37 Commits

Author SHA1 Message Date
Paul Gschwendtner
b5ab7aff43 refactor: add override keyword to members implementing abstract declarations (#42512)
In combination with the TS `noImplicitOverride` compatibility changes,
we also want to follow the best-practice of adding `override` to
members which are implemented as part of abstract classes. This
commit fixes all instances which will be flagged as part of the
custom `no-implicit-override-abstract` TSLint rule.

PR Close #42512
2021-07-12 13:11:17 -07:00
JoostK
30c82cd177 fix(compiler-cli): inline type checking instructions no longer prevent incremental reuse (#42759)
Source files that contain directives or components that need an inline
type constructor or inline template type-check block would always be
considered as affected in incremental rebuilds. The inline operations
cause the source file to be updated in the TypeScript program that is
created for template type-checking, which becomes the reuse program
in a subsequent incremental rebuild.

In an incremental rebuild, the source files from the new user program
are compared to those from the reuse program. The updated source files
are not the same as the original source file from the user program, so
the incremental engine would mark the file which needed inline
operations as affected. This prevents incremental reuse for these files,
causing sub-optimal rebuild performance.

This commit attaches the original source file for source files that have
been updated with inline operations, such that the incremental engine
is able to compare source files using the original source file.

Fixes #42543

PR Close #42759
2021-07-07 15:17:25 -07:00
JoostK
22bda2226b fix(compiler-cli): prevent prior compilations from being retained in watch builds (#42537)
In watch builds, the compiler attempts to reuse as much information from
a prior compilation as possible. To accomplish this, it keeps a
reference to the most recently succeeded `TraitCompiler`, which contains
all analysis data for the program. However, `TraitCompiler` has an
internal reference to an `IncrementalBuild`, which is itself built on
top of its prior state. Consequently, all prior compilations continued
to be referenced, preventing garbage collection from cleaning up these
instances.

This commit changes the `AnalyzedIncrementalState` to no longer retain
a `TraitCompiler` instance, but only the analysis data it contains. This
breaks the retainer path to the prior incremental state, allowing it to
be garbage collected.

PR Close #42537
2021-06-09 16:10:04 -07:00
Paul Gschwendtner
2d0ff0a5d3 ci: add lint error for files with missing trailing new-line (#42478)
For quite a while it is an unspoken convention to add a trailing
new-line files within the Angular repository. This was never enforced
automatically, but has been frequently raised in pull requests through
manual review. This commit sets up a lint rule so that this is
"officially" enforced and doesn't require manual review.

PR Close #42478
2021-06-04 13:31:03 -07:00
Alex Rickabaugh
dee95994b8 refactor(compiler-cli): support ts.SourceFile versioning (#41475)
Generally, the compiler assumes that `ts.SourceFile`s are immutable objects.
If a new `ts.Program` is compared to an old one, and a `ts.SourceFile`
within that program has not changed its object identity, the compiler will
assume that its prior analysis and understanding of that source file is
still valid.

However, not all TypeScript workflows uphold this assumption. For
`ts.Program`s that originate from the `ts.LanguageService`, some source
files may be re-parsed or otherwise undergo mutations without changing their
object identity. This breaks the compiler's incremental workflow.

Within such environments, it's necessary to track source file changes
differently. In addition to object identity, it's necessary to compare a
"version" string associated with each source file, between when that file is
analyzed originally and when a new program is presented that still contains
it. It's possible for the object identity of the source file to be the same,
but the version string to have changed, indicating that the source file
should be treated as changed.

This commit adds an optional method `getSourceFileVersion` to the
`ProgramDriver`, to provide access to version information if available. When
this method is present, the compiler will build a map of source file version
strings, and use this map to augment identity comparison during incremental
compilation.

PR Close #41475
2021-04-13 13:05:36 -07:00
Alex Rickabaugh
94ec0af582 refactor(compiler-cli): replace the IncrementalDriver with a new design (#41475)
This commit replaces the `IncrementalDriver` abstraction which powered
incremental compilation in the compiler with a new `IncrementalCompilation`
design. Principally, it separates two concerns which were tied together in
the previous implementation:

1. Tracking the reusable state of a compilation at any given point that
   could be reused in a subsequent future compilation.

2. Making use of a prior compilation's state to accelerate the current one.

The new abstraction adds explicit tracking and types to deal with both of
these concerns separately, which greatly reduces the complexity of the state
tracking that `IncrementalDriver` used to perform.

PR Close #41475
2021-04-13 13:05:35 -07:00
JoostK
ffea31f433 perf(compiler-cli): allow incremental compilation in the presence of redirected source files (#41448)
When multiple occurrences of the same package exist within a single
TypeScript compilation unit, TypeScript deduplicates the source files
by introducing redirected source file proxies. Such proxies are
recreated during an incremental compilation even if the original
declaration file did not change, which caused the compiler not to reuse
any work from the prior compilation.

This commit changes the incremental driver to recognize a redirected
source file and treat them as their unredirected source file.

PR Close #41448
2021-04-12 21:03:26 -07:00
Alex Rickabaugh
48fec08c95 perf(compiler-cli): refactor the performance tracing infrastructure (#41125)
ngtsc has an internal performance tracing package, which previously has not
really seen much use. It used to track performance statistics on a very
granular basis (microseconds per actual class analysis, for example). This
had two problems:

* it produced voluminous amounts of data, complicating the analysis of such
  results and providing dubious value.
* it added nontrivial overhead to compilation when used (which also affected
  the very performance of the operations being measured).

This commit replaces the old system with a streamlined performance tracing
setup which is lightweight and designed to be always-on. The new system
tracks 3 metrics:

* time taken by various phases and operations within the compiler
* events (counters) which measure the shape and size of the compilation
* memory usage measured at various points of the compilation process

If the compiler option `tracePerformance` is set, the compiler will
serialize these metrics to a JSON file at that location after compilation is
complete.

PR Close #41125
2021-03-24 13:42:24 -07:00
JoostK
fed6a7ce7d perf(compiler-cli): detect semantic changes and their effect on an incremental rebuild (#40947)
In Angular programs, changing a file may require other files to be
emitted as well due to implicit NgModule dependencies. For example, if
the selector of a directive is changed then all components that have
that directive in their compilation scope need to be recompiled, as the
change of selector may affect the directive matching results.

Until now, the compiler solved this problem using a single dependency
graph. The implicit NgModule dependencies were represented in this
graph, such that a changed file would correctly also cause other files
to be re-emitted. This approach is limited in a few ways:

1. The file dependency graph is used to determine whether it is safe to
   reuse the analysis data of an Angular decorated class. This analysis
   data is invariant to unrelated changes to the NgModule scope, but
   because the single dependency graph also tracked the implicit
   NgModule dependencies the compiler had to consider analysis data as
   stale far more often than necessary.
2. It is typical for a change to e.g. a directive to not affect its
   public API—its selector, inputs, outputs, or exportAs clause—in which
   case there is no need to re-emit all declarations in scope, as their
   compilation output wouldn't have changed.

This commit implements a mechanism by which the compiler is able to
determine the impact of a change by comparing it to the prior
compilation. To achieve this, a new graph is maintained that tracks all
public API information of all Angular decorated symbols. During an
incremental compilation this information is compared to the information
that was captured in the most recently succeeded compilation. This
determines the exact impact of the changes to the public API, which
is then used to determine which files need to be re-emitted.

Note that the file dependency graph remains, as it is still used to
track the dependencies of analysis data. This graph does no longer track
the implicit NgModule dependencies, which allows for better reuse of
analysis data.

These changes also fix a bug where template type-checking would fail to
incorporate changes made to a transitive base class of a
directive/component. This used to be a problem because transitive base
classes were not recorded as a transitive dependency in the file
dependency graph, such that prior type-check blocks would erroneously
be reused.

This commit also fixes an incorrectness where a change to a declaration
in NgModule `A` would not cause the declarations in NgModules that
import from NgModule `A` to be re-emitted. This was intentionally
incorrect as otherwise the performance of incremental rebuilds would
have been far worse. This is no longer a concern, as the compiler is now
able to only re-emit when actually necessary.

Fixes #34867
Fixes #40635
Closes #40728

PR Close #40947
2021-03-08 08:41:19 -08:00
Alex Rickabaugh
21e24d1474 refactor(compiler-cli): introduce CompilationTicket system for NgCompiler (#40561)
Previously, the incremental flow for NgCompiler was simple: when creating a
new NgCompiler instance, the consumer could pass state from a previous
compilation, which would cause the new compilation to be performed
incrementally. "Local" information about TypeScript files which had not
changed would be passed from the old compilation to the new and reused,
while "global" information would always be recalculated.

However, this flow could be made more efficient in certain cases, such as
when no TypeScript files are changed in a new compilation. In this case,
_all_ information extracted during the first compilation is reusable. Doing
this involves reusing the previous `NgCompiler` instance (the container for
such global information) and updating it, instead of creating a new one for
the next compilation. This approach works cleanly, but complicates the
lifecycle of `NgCompiler`.

To prevent consumers from having to deal with the mechanics of reuse vs
incremental steps of `NgCompiler`, a new `CompilationTicket` mechanism is
added in this commit. Consumers obtain a `CompilationTicket` via one of
several code paths depending on the nature of the incoming compilation, and
use the `CompilationTicket` to obtain an `NgCompiler` instance. This
instance may be a fresh compilation, a new `NgCompiler` for an incremental
compilation, or an existing `NgCompiler` that's been updated to optimally
process a resource-only change. Consumers can use the new `NgCompiler`
without knowledge of its provenance.

PR Close #40561
2021-01-27 10:45:57 -08:00
Alex Rickabaugh
c7c5b2fc1e fix(compiler-cli): correct incremental behavior even with broken imports (#39923)
When the compiler is invoked via ngc or the Angular CLI, its APIs are used
under the assumption that Angular analysis/diagnostics are only requested if
the program has no TypeScript-level errors. A result of this assumption is
that the incremental engine has not needed to resolve changes via its
dependency graph when the program contained broken imports, since broken
imports are a TypeScript error.

The Angular Language Service for Ivy is using the compiler as a backend, and
exercising its incremental compilation APIs without enforcing this
assumption. As a result, the Language Service has run into issues where
broken imports cause incremental compilation to fail and produce incorrect
results.

This commit introduces a mechanism within the compiler to keep track of
files for which dependency analysis has failed, and to always treat such
files as potentially affected by future incremental steps. This is tested
via the Language Service infrastructure to ensure that the compiler is doing
the right thing in the case of invalid imports.

PR Close #39923
2020-12-03 13:42:13 -08:00
Charles Lyding
5dbf357224 feat(compiler-cli): support getting resource dependencies for a source file (#38048)
The compiler maintains an internal dependency graph of all resource
dependencies for application source files. This information can be useful
for tools that integrate the compiler and need to support file watching.
This change adds a `getResourceDependencies` method to the
`NgCompiler` class that allows compiler integrations to access resource
dependencies of files within the compilation.

PR Close #38048
2020-10-02 14:19:39 -07:00
Alex Rickabaugh
16c7441c2f refactor(compiler-cli): introduce the TemplateTypeChecker abstraction (#38105)
This commit significantly refactors the 'typecheck' package to introduce a
new abstraction, the `TemplateTypeChecker`. To achieve this:

* a 'typecheck:api' package is introduced, containing common interfaces that
  consumers of the template type-checking infrastructure can depend on
  without incurring a dependency on the template type-checking machinery as
  a whole.
* interfaces for `TemplateTypeChecker` and `TypeCheckContext` are introduced
  which contain the abstract operations supported by the implementation
  classes `TemplateTypeCheckerImpl` and `TypeCheckContextImpl` respectively.
* the `TemplateTypeChecker` interface supports diagnostics on a whole
  program basis to start with, but the implementation is purposefully
  designed to support incremental diagnostics at a per-file or per-component
  level.
* `TemplateTypeChecker` supports direct access to the type check block of a
  component.
* the testing utility is refactored to be a lot more useful, and new tests
  are added for the new abstraction.

PR Close #38105
2020-07-29 10:31:20 -07:00
Alex Rickabaugh
736f6337b2 refactor(compiler-cli): make file/shim split 1:n instead of 1:1 (#38105)
Previously in the template type-checking engine, it was assumed that every
input file would have an associated type-checking shim. The type check block
code for all components in the input file would be generated into this shim.

This is fine for whole-program type checking operations, but to support the
language service's requirements for low latency, it would be ideal to be
able to check a single component in isolation, especially if the component
is declared along with many others in a single file.

This commit removes the assumption that the file/shim mapping is 1:1, and
introduces the concept of component-to-shim mapping. Any
`TypeCheckingProgramStrategy` must provide such a mapping.

To achieve this:

 * type checking record information is now split into file-level data as
   well as per-shim data.
 * components are now assigned a stable `TemplateId` which is unique to the
   file in which they're declared.

PR Close #38105
2020-07-29 10:31:20 -07:00
Alex Rickabaugh
71956250dd perf(compiler-cli): fix memory leak in retained incremental state (#37835)
Incremental compilation allows for the output state of one compilation to be
reused as input to the next compilation. This involves retaining references
to instances from prior compilations, which must be done carefully to avoid
memory leaks.

This commit fixes such a leak with a complicated retention chain:

* `TrackedIncrementalBuildStrategy` unnecessarily hangs on to the previous
  `IncrementalDriver` (state of the previous compilation) once the current
  compilation completes.

  In general this is unnecessary, but should be safe as long as the chain
  only goes back one level - if the `IncrementalDriver` doesn't retain any
  previous `TrackedIncrementalBuildStrategy` instances. However, this does
  happen:

* `NgCompiler` indirectly causes retention of previous `NgCompiler`
  instances (and thus previous `TrackedIncrementalBuildStrategy` instances)
  through accidental capture of the `this` context in a closure created in
  its constructor. This closure is wrapped in a `ts.ModuleResolutionCache`
  used to create a `ModuleResolver` class, which is passed to the program's
  `TraitCompiler` on construction.

* The `IncrementalDriver` retains a reference to the `TraitCompiler` of the
  previous compilation, completing the reference chain.

The final retention chain thus looks like:

* `TrackedIncrementalBuildStrategy` of current program
* `.previous`: `IncrementalDriver` of previous program
* `.lastGood.traitCompiler`: `TraitCompiler`
* `.handlers[..].moduleResolver.moduleResolutionCache`: cache
* (via `getCanonicalFileName` closure): `NgCompiler`
* `.incrementalStrategy`: `TrackedIncrementalBuildStrategy` of previous
  program.

The closure link is the "real" leak here. `NgCompiler` is creating a closure
for `getCanonicalFileName`, delegating to its
`this.adapter.getCanonicalFileName`, for the purposes of creating a
`ts.ModuleResolutionCache`. The fact that the closure references
`NgCompiler` thus eventually causes previous `NgCompiler` iterations to be
retained. This is also potentially problematic due to the shared nature of
`ts.ModuleResolutionCache`, which is potentially retained across multiple
compilations intentionally.

This commit fixes the first two links in the retention chain: the build
strategy is patched to not retain a `previous` pointer, and the `NgCompiler`
is patched to not create a closure in the first place, but instead pass a
bound function. This ensures that the `NgCompiler` does not retain previous
instances of itself in the first place, even if the build strategy does
end up retaining the previous incremental state unnecessarily.

The third link (`IncrementalDriver` unnecessarily retaining the whole
`TraitCompiler`) is not addressed in this commit as it's a more
architectural problem that will require some refactoring. However, the leak
potential of this retention is eliminated thanks to fixing the first two
issues.

PR Close #37835
2020-06-29 16:34:51 -07:00
Alex Rickabaugh
300c2fec9c refactor(compiler-cli): make IncrementalBuild strategy configurable (#37339)
Commit 24b2f1da2b introduced an `NgCompiler` which operates on a
`ts.Program` independently of the `NgtscProgram`. The NgCompiler got its
`IncrementalDriver` (for incremental reuse of Angular compilation results)
by looking at a monkey-patched property on the `ts.Program`.

This monkey-patching operation causes problems with the Angular indexer
(specifically, it seems to cause the indexer to retain too much of prior
programs, resulting in OOM issues). To work around this, `IncrementalDriver`
reuse is now handled by a dedicated `IncrementalBuildStrategy`. One
implementation of this interface is used by the `NgtscProgram` to perform
the old-style reuse, relying on the previous instance of `NgtscProgram`
instead of monkey-patching. Only for `NgTscPlugin` is the monkey-patching
strategy used, as the plugin sits behind an interface which only provides
access to the `ts.Program`, not a prior instance of the plugin.

PR Close #37339
2020-06-15 09:50:08 -07:00
Joey Perrott
d1ea1f4c7f build: update license headers to reference Google LLC (#37205)
Update the license headers throughout the repository to reference Google LLC
rather than Google Inc, for the required license headers.

PR Close #37205
2020-05-26 14:26:58 -04:00
Alex Rickabaugh
ecffc3557f perf(compiler-cli): perform template type-checking incrementally (#36211)
This optimization builds on a lot of prior work to finally make type-
checking of templates incremental.

Incrementality requires two main components:
- the ability to reuse work from a prior compilation.
- the ability to know when changes in the current program invalidate that
  prior work.

Prior to this commit, on every type-checking pass the compiler would
generate new .ngtypecheck files for each original input file in the program.

1. (Build #1 main program): empty .ngtypecheck files generated for each
   original input file.

2. (Build #1 type-check program): .ngtypecheck contents overridden for those
   which have corresponding components that need type-checked.

3. (Build #2 main program): throw away old .ngtypecheck files and generate
   new empty ones.

4. (Build #2 type-check program): same as step 2.

With this commit, the `IncrementalDriver` now tracks template type-checking
_metadata_ for each input file. The metadata contains information about
source mappings for generated type-checking code, as well as some
diagnostics which were discovered at type-check analysis time. The actual
type-checking code is stored in the TypeScript AST for type-checking files,
which is now re-used between programs as follows:

1. (Build #1 main program): empty .ngtypecheck files generated for each
   original input file.

2. (Build #1 type-check program): .ngtypecheck contents overridden for those
   which have corresponding components that need type-checked, and the
   metadata registered in the `IncrementalDriver`.

3. (Build #2 main program): The `TypeCheckShimGenerator` now reuses _all_
   .ngtypecheck `ts.SourceFile` shims from build #1's type-check program in
   the construction of build #2's main program. Some of the contents of
   these files might be stale (if a component's template changed, for
   example), but wholesale reuse here prevents unnecessary changes in the
   contents of the program at this point and makes TypeScript's job a lot
   easier.

4. (Build #2 type-check program): For those input files which have not
   "logically changed" (meaning components within are semantically the same
   as they were before), the compiler will re-use the type-check file
   metadata from build #1, and _not_ generate a new .ngtypecheck shim.
   For components which have logically changed or where the previous
   .ngtypecheck contents cannot otherwise be reused, code generation happens
   as before.

PR Close #36211
2020-05-05 18:40:42 -07:00
Alex Rickabaugh
b861e9c0ac perf(compiler-cli): split Ivy template type-checking into multiple files (#36211)
As a performance optimization, this commit splits the single
__ngtypecheck__.ts file which was previously added to the user's program as
a container for all template type-checking code into multiple .ngtypecheck
shim files, one for each original file in the user's program.

In larger applications, the generation, parsing, and checking of this single
type-checking file was a huge performance bottleneck, with the file often
exceeding 1 MB in text content. Particularly in incremental builds,
regenerating this single file for the entire application proved especially
expensive.

This commit introduces a new strategy for template type-checking code which
makes use of a new interface, the `TypeCheckingProgramStrategy`. This
interface abstracts the process of creating a new `ts.Program` to type-check
a particular compilation, and allows the mechanism there to be kept separate
from the more complex logic around dealing with multiple .ngtypecheck files.

A new `TemplateTypeChecker` hosts that logic and interacts with the
`TypeCheckingProgramStrategy` to actually generate and return diagnostics.
The `TypeCheckContext` class, previously the workhorse of template type-
checking, is now solely focused on collecting and generating type-checking
file contents.

A side effect of implementing the new `TypeCheckingProgramStrategy` in this
way is that the API is designed to be suitable for use by the Angular
Language Service as well. The LS also needs to type-check components, but
has its own method for constructing a `ts.Program` with type-checking code.

Note that this commit does not make the actual checking of templates at all
_incremental_ just yet. That will happen in a future commit.

PR Close #36211
2020-05-05 18:40:42 -07:00
Alex Rickabaugh
0a69a2832b style(compiler-cli): reformat of codebase with new clang-format version (#36520)
This commit reformats the packages/compiler-cli tree using the new version
of clang-format.

PR Close #36520
2020-04-08 14:51:08 -07:00
JoostK
5cada5cce1 fix(ivy): recompile on template change in ngc watch mode on Windows (#34015)
In #33551, a bug in `ngc --watch` mode was fixed so that a component is
recompiled when its template file is changed. Due to insufficient
normalization of files paths, this fix did not have the desired effect
on Windows.

Fixes #32869

PR Close #34015
2020-02-04 10:40:22 -08:00
Alex Rickabaugh
5aa0507f6a docs(ivy): move incremental package README file to the correct location (#34912)
It was erroneously committed to src/.

PR Close #34912
2020-01-23 13:30:10 -08:00
Alex Rickabaugh
5b2fa3cfd3 fix(ivy): correctly emit component when it's removed from its module (#34912)
This commit fixes a bug in the incremental rebuild engine of ngtsc, where if
a component was removed from its NgModule, it would not be properly
re-emitted.

The bug stemmed from the fact that whether to emit a file was a decision
based purely on the updated dependency graph, which captures the dependency
structure of the rebuild program. This graph has no edge from the component
to its former module (as it was removed, of course), so the compiler
erroneously decides not to emit the component.

The bug here is that the compiler does know, from the previous dependency
graph, that the component file has logically changed, since its previous
dependency (the module file) has changed. This information was not carried
forward into the set of files which need to be emitted, because it was
assumed that the updated dependency graph was a more accurate source of that
information.

With this commit, the set of files which need emit is pre-populated with the
set of logically changed files, to cover edge cases like this.

Fixes #34813

PR Close #34912
2020-01-23 13:30:10 -08:00
crisbeto
6d534f10e6 fix(ivy): don't run decorator handlers against declaration files (#34557)
Currently the decorator handlers are run against all `SourceFile`s in the compilation, but we shouldn't be doing it against declaration files. This initially came up as a CI issue in #33264 where it was worked around only for the `DirectiveDecoratorHandler`. These changes move the logic into the `TraitCompiler` and `DecorationAnalyzer` so that it applies to all of the handlers.

PR Close #34557
2020-01-10 15:54:51 -08:00
Alex Rickabaugh
74edde0a94 perf(ivy): reuse prior analysis work during incremental builds (#34288)
Previously, the compiler performed an incremental build by analyzing and
resolving all classes in the program (even unchanged ones) and then using
the dependency graph information to determine which .js files were stale and
needed to be re-emitted. This algorithm produced "correct" rebuilds, but the
cost of re-analyzing the entire program turned out to be higher than
anticipated, especially for component-heavy compilations.

To achieve performant rebuilds, it is necessary to reuse previous analysis
results if possible. Doing this safely requires knowing when prior work is
viable and when it is stale and needs to be re-done.

The new algorithm implemented by this commit is such:

1) Each incremental build starts with knowledge of the last known good
   dependency graph and analysis results from the last successful build,
   plus of course information about the set of files changed.

2) The previous dependency graph's information is used to determine the
   set of source files which have "logically" changed. A source file is
   considered logically changed if it or any of its dependencies have
   physically changed (on disk) since the last successful compilation. Any
   logically unchanged dependencies have their dependency information copied
   over to the new dependency graph.

3) During the `TraitCompiler`'s loop to consider all source files in the
   program, if a source file is logically unchanged then its previous
   analyses are "adopted" (and their 'register' steps are run). If the file
   is logically changed, then it is re-analyzed as usual.

4) Then, incremental build proceeds as before, with the new dependency graph
   being used to determine the set of files which require re-emitting.

This analysis reuse avoids template parsing operations in many circumstances
and significantly reduces the time it takes ngtsc to rebuild a large
application.

Future work will increase performance even more, by tackling a variety of
other opportunities to reuse or avoid work.

PR Close #34288
2019-12-12 13:11:45 -08:00
Alex Rickabaugh
4cf197998a fix(ivy): track changes across failed builds (#33971)
Previously, our incremental build system kept track of the changes between
the current compilation and the previous one, and used its knowledge of
inter-file dependencies to evaluate the impact of each change and emit the
right set of output files.

However, a problem arose if the compiler was not able to extract a
dependency graph successfully. This typically happens if the input program
contains errors. In this case the Angular analysis part of compilation is
never executed.

If a file changed in one of these failed builds, in the next build it
appears unchanged. This means that the compiler "forgets" to emit it!

To fix this problem, the compiler needs to know the set of changes made
_since the last successful build_, not simply since the last invocation.

This commit changes the incremental state system to much more explicitly
pass information from the previous to the next compilation, and in the
process to keep track of changes across multiple failed builds, until the
program can be analyzed successfully and the results of those changes
incorporated into the emit plan.

Fixes #32214

PR Close #33971
2019-11-22 17:39:35 -05:00
Alex Rickabaugh
4be8929844 fix(ivy): always re-analyze the program during incremental rebuilds (#33862)
Previously, the ngtsc compiler attempted to reuse analysis work from the
previous program during an incremental build. To do this, it had to prove
that the work was safe to reuse - that no changes made to the new program
would invalidate the previous analysis.

The implementation of this had a significant design flaw: if the previous
program had errors, the previous analysis would be missing significant
information, and the dependency graph extracted from it would not be
sufficient to determine which files should be re-analyzed to fill in the
gaps. This often meant that the build output after an error was resolved
would be wholly incorrect.

This commit switches ngtsc to take a simpler approach to incremental
rebuilds. Instead of attempting to reuse prior analysis work, the entire
program is re-analyzed with each compilation. This is actually not as
expensive as one might imagine - analysis is a fairly small part of overall
compilation time.

Based on the dependency graph extracted during this analysis, the compiler
then can make accurate decisions on whether to emit specific files. A new
suite of tests is added to validate behavior in the presence of source code
level errors.

This new approach is dramatically simpler than the previous algorithm, and
should always produce correct results for a semantically correct program.s

Fixes #32388
Fixes #32214

PR Close #33862
2019-11-20 11:46:02 -08:00
JoostK
15f8638b1c fix(ivy): ensure module scope is rebuild on dependent change (#33522)
During incremental compilations, ngtsc needs to know which metadata
from a previous compilation can be reused, versus which metadata has to
be recomputed as some dependency was updated. Changes to
directives/components should cause the NgModule in which they are
declared to be recompiled, as the NgModule's compilation is dependent
on its directives/components.

When a dependent source file of a directive/component is updated,
however, a more subtle dependency should also cause to NgModule's source
file to be invalidated. During the reconciliation of state from a
previous compilation into the new program, the component's source file
is invalidated because one of its dependency has changed, ergo the
NgModule needs to be invalidated as well. Up until now, this implicit
dependency was not imposed on the NgModule. Additionally, any change to
a dependent file may influence the module scope to change, so all
components within the module must be invalidated as well.

This commit fixes the bug by introducing additional file dependencies,
as to ensure a proper rebuild of the module scope and its components.

Fixes #32416

PR Close #33522
2019-11-12 13:56:30 -08:00
JoostK
a42057d0f8 fix(ivy): support abstract directives in template type checking (#33131)
Recently it was made possible to have a directive without selector,
which are referred to as abstract directives. Such directives should not
be registered in an NgModule, but can still contain decorators for
inputs, outputs, queries, etc. The information from these decorators and
the `@Directive()` decorator itself needs to be registered with the
central `MetadataRegistry` so that other areas of the compiler can
request information about a given directive, an example of which is the
template type checker that needs to know about the inputs and outputs of
directives.

Prior to this change, however, abstract directives would only register
themselves with the `MetadataRegistry` as being an abstract directive,
without all of its other metadata like inputs and outputs. This meant
that the template type checker was unable to resolve the inputs and
outputs of these abstract directives, therefore failing to check them
correctly. The typical error would be that some property does not exist
on a DOM element, whereas said property should have been bound to the
abstract directive's input.

This commit fixes the problem by always registering the metadata of a
directive or component with the `MetadataRegistry`. Tests have been
added to ensure abstract directives are handled correctly in the
template type checker, together with tests to verify the form of
abstract directives in declaration files.

Fixes #30080

PR Close #33131
2019-10-24 12:44:30 -07:00
atscott
cfed0c0cf1 fix(ivy): Support selector-less directive as base classes (#32125)
Following #31379, this adds support for directives without a selector to
Ivy.

PR Close #32125
2019-08-20 09:56:54 -07:00
Pete Bacon Darwin
eb5412d76f fix(ivy): reuse compilation scope for incremental template changes. (#31932)
Previously if only a component template changed then we would know to
rebuild its component source file. But the compilation was incorrect if the
component was part of an NgModule, since we were not capturing the
compilation scope information that had a been acquired from the NgModule
and was not being regenerated since we were not needing to recompile
the NgModule.

Now we register compilation scope information for each component, via the
`ComponentScopeRegistry` interface, so that it is available for incremental
compilation.

The `ComponentDecoratorHandler` now reads the compilation scope from a
`ComponentScopeReader` interface which is implemented as a compound
reader composed of the original `LocalModuleScopeRegistry` and the
`IncrementalState`.

Fixes #31654

PR Close #31932
2019-08-09 10:50:40 -07:00
Matt Lewis
4aecf9253b fix(ivy): support older CLI versions that do not pass a list of changed files (#31322)
Versions of CLI prior to angular/angular-cli@0e339ee did not expose the host.getModifiedResourceFiles() method.

This meant that null was being passed through to the IncrementalState.reconcile() method
to indicate that there were either no changes or the host didn't support that method.

This commit fixes a bug where we were checking for undefined rather than null when
deciding whether any resource files had changed, causing a null reference error to be thrown.

This bug was not caught by the unit testing because the tests set up the changed files
via a slightly different process, not having access to the CompilerHost, and these test
were making the erroneous assumption that undefined indicated that there were no
changed files.

PR Close #31322
2019-07-18 14:22:07 -07:00
Pete Bacon Darwin
48def92cad fix(ivy): ensure that changes to component resources trigger incremental builds (#30954)
Optimizations to skip compiling source files that had not changed
did not account for the case where only a resource file changes,
such as an external template or style file.

Now we track such dependencies and trigger a recompilation
if any of the previously tracked resources have changed.

This will require a change on the CLI side to provide the list of
resource files that changed to trigger the current compilation by
implementing `CompilerHost.getModifiedResourceFiles()`.

Closes #30947

PR Close #30954
2019-06-21 10:13:46 -07:00
Pete Bacon Darwin
fbff03b476 feat(ivy): skip analysis of unchanged components (#30238)
Now that the dependent files and compilation scopes are being tracked in
the incremental state, we can skip analysing and emitting source files if
none of their dependent files have changed since the last compile.

The computation of what files (and their dependencies) are unchanged is
computed during reconciliation.

This commit also removes the previous emission skipping logic, since this
approach covers those cases already.

PR Close #30238
2019-05-10 12:10:40 -07:00
Pete Bacon Darwin
411524d341 feat(ivy): track compilation scope dependencies for components (#30238)
To support skipping analysis of a file containing a component
we need to know that none of the declarations that might affect
its ngtsc compilation have not changed. The files that we need to
check are those that contain classes from the `CompilationScope`
of the component. These classes are already tracked in the
`LocalModuleScopeRegistry`.

This commit modifies the `IvyCompilation` class to record the
files that are in each declared class's `CompilationScope` via
a new method, `recordNgModuleScopeDependencies()`, that is called
after all the handlers have been "resolved".

Further, if analysis is skipped for a declared class, then we need
to recover the analysis from the previous compilation run. To
support this, the `IncrementalState` class has been updated to
expose the `MetadataReader` and `MetadataRegistry` interfaces.
This is included in the `metaRegistry` object to capture these analyses,
and also in the `localMetaReader` as a fallback to use if the
current compilation analysis was skipped.

PR Close #30238
2019-05-10 12:10:40 -07:00
Pete Bacon Darwin
0a0b4c1d8f feat(ivy): track file dependencies due to partial evaluation (#30238)
As part of incremental compilation performance improvements, we need
to track the dependencies of files due to expressions being evaluated by
the `PartialEvaluator`.

The `PartialEvaluator` now accepts a `DependencyTracker` object, which is
used to track which files are visited when evaluating an expression.
The interpreter computes this `originatingFile` and stores it in the evaluation
`Context` so it can pass this to the `DependencyTracker.

The `IncrementalState` object implements this interface, which allows it to be
passed to the `PartialEvaluator` and so capture the file dependencies.

PR Close #30238
2019-05-10 12:10:40 -07:00
Alex Rickabaugh
7041e61562 perf(ivy): basic incremental compilation for ngtsc (#29380)
This commit introduces a mechanism for incremental compilation to the ngtsc
compiler.

Previously, incremental information was used in the construction of the
ts.Program for subsequent compilations, but was not used in ngtsc itself.

This commit adds an IncrementalState class, which tracks state between ngtsc
compilations. Currently, this supports skipping the TypeScript emit step
when the compiler can prove the contents of emit have not changed.

This is implemented for @Injectables as well as for files which don't
contain any Angular decorated types. These are the only files which can be
proven to be safe today.

See ngtsc/incremental/README.md for more details.

PR Close #29380
2019-04-01 15:13:56 -07:00