2019-01-29 11:33:37 -08:00
|
|
|
/**
|
|
|
|
|
* @license
|
2020-05-19 12:08:49 -07:00
|
|
|
* Copyright Google LLC All Rights Reserved.
|
2019-01-29 11:33:37 -08:00
|
|
|
*
|
|
|
|
|
* Use of this source code is governed by an MIT-style license that can be
|
|
|
|
|
* found in the LICENSE file at https://angular.io/license
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
import * as ts from 'typescript';
|
|
|
|
|
|
2020-01-17 16:00:44 -08:00
|
|
|
import {NgCompiler, NgCompilerHost} from './core';
|
|
|
|
|
import {NgCompilerOptions, UnifiedModulesHost} from './core/api';
|
|
|
|
|
import {NodeJSFileSystem, setFileSystem} from './file_system';
|
2020-05-28 16:08:52 -07:00
|
|
|
import {PatchedProgramIncrementalBuildStrategy} from './incremental';
|
2020-01-17 16:00:44 -08:00
|
|
|
import {NOOP_PERF_RECORDER} from './perf';
|
perf(compiler-cli): fix regressions in incremental program reuse (#37641)
Commit 4213e8d5 introduced shim reference tagging into the compiler, and
changed how the `TypeCheckProgramHost` worked under the hood during the
creation of a template type-checking program. This work enabled a more
incremental flow for template type-checking, but unintentionally introduced
several regressions in performance, caused by poor incrementality during
`ts.Program` creation.
1. The `TypeCheckProgramHost` was made to rely on the `ts.CompilerHost` to
retrieve instances of `ts.SourceFile`s from the original program. If the
host does not return the original instance of such files, but instead
creates new instances, this has two negative effects: it incurs
additional parsing time, and it interferes with TypeScript's ability to
reuse information about such files.
2. During the incremental creation of a `ts.Program`, TypeScript compares
the `referencedFiles` of `ts.SourceFile` instances from the old program
with those in the new program. If these arrays differ, TypeScript cannot
fully reuse the old program. The implementation of reference tagging
introduced in 4213e8d5 restores the original `referencedFiles` array
after a `ts.Program` is created, which means that future incremental
operations involving that program will always fail this comparison,
effectively limiting the incrementality TypeScript can achieve.
Problem 1 exacerbates problem 2: if a new `ts.SourceFile` is created by the
host after shim generation has been disabled, it will have an untagged
`referencedFiles` array even if the original file's `referencedFiles` was
not restored, triggering problem 2 when creating the template type-checking
program.
To fix these issues, `referencedFiles` arrays are now restored on the old
`ts.Program` prior to the creation of a new incremental program. This allows
TypeScript to get the most out of reusing the old program's data.
Additionally, the `TypeCheckProgramHost` now uses the original `ts.Program`
to retrieve original instances of `ts.SourceFile`s where possible,
preventing issues when a host would otherwise return fresh instances.
Together, these fixes ensure that program reuse is as incremental as
possible, and tests have been added to verify this for certain scenarios.
An optimization was further added to prevent the creation of a type-checking
`ts.Program` in the first place if no type-checking is necessary.
PR Close #37641
2020-06-19 12:55:13 -07:00
|
|
|
import {untagAllTsFiles} from './shims';
|
perf(compiler-cli): split Ivy template type-checking into multiple files (#36211)
As a performance optimization, this commit splits the single
__ngtypecheck__.ts file which was previously added to the user's program as
a container for all template type-checking code into multiple .ngtypecheck
shim files, one for each original file in the user's program.
In larger applications, the generation, parsing, and checking of this single
type-checking file was a huge performance bottleneck, with the file often
exceeding 1 MB in text content. Particularly in incremental builds,
regenerating this single file for the entire application proved especially
expensive.
This commit introduces a new strategy for template type-checking code which
makes use of a new interface, the `TypeCheckingProgramStrategy`. This
interface abstracts the process of creating a new `ts.Program` to type-check
a particular compilation, and allows the mechanism there to be kept separate
from the more complex logic around dealing with multiple .ngtypecheck files.
A new `TemplateTypeChecker` hosts that logic and interacts with the
`TypeCheckingProgramStrategy` to actually generate and return diagnostics.
The `TypeCheckContext` class, previously the workhorse of template type-
checking, is now solely focused on collecting and generating type-checking
file contents.
A side effect of implementing the new `TypeCheckingProgramStrategy` in this
way is that the API is designed to be suitable for use by the Angular
Language Service as well. The LS also needs to type-check components, but
has its own method for constructing a `ts.Program` with type-checking code.
Note that this commit does not make the actual checking of templates at all
_incremental_ just yet. That will happen in a future commit.
PR Close #36211
2020-03-04 15:50:12 -08:00
|
|
|
import {ReusedProgramStrategy} from './typecheck/src/augmented_program';
|
2019-01-29 11:33:37 -08:00
|
|
|
|
2020-01-17 16:00:44 -08:00
|
|
|
// The following is needed to fix a the chicken-and-egg issue where the sync (into g3) script will
|
|
|
|
|
// refuse to accept this file unless the following string appears:
|
|
|
|
|
// import * as plugin from '@bazel/typescript/internal/tsc_wrapped/plugin_api';
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* A `ts.CompilerHost` which also returns a list of input files, out of which the `ts.Program`
|
|
|
|
|
* should be created.
|
|
|
|
|
*
|
|
|
|
|
* Currently mirrored from @bazel/typescript/internal/tsc_wrapped/plugin_api (with the naming of
|
|
|
|
|
* `fileNameToModuleName` corrected).
|
|
|
|
|
*/
|
|
|
|
|
interface PluginCompilerHost extends ts.CompilerHost, Partial<UnifiedModulesHost> {
|
|
|
|
|
readonly inputFiles: ReadonlyArray<string>;
|
2019-01-29 11:33:37 -08:00
|
|
|
}
|
|
|
|
|
|
2020-01-17 16:00:44 -08:00
|
|
|
/**
|
|
|
|
|
* Mirrors the plugin interface from tsc_wrapped which is currently under active development. To
|
|
|
|
|
* enable progress to be made in parallel, the upstream interface isn't implemented directly.
|
|
|
|
|
* Instead, `TscPlugin` here is structurally assignable to what tsc_wrapped expects.
|
|
|
|
|
*/
|
|
|
|
|
interface TscPlugin {
|
|
|
|
|
readonly name: string;
|
|
|
|
|
|
|
|
|
|
wrapHost(
|
|
|
|
|
host: ts.CompilerHost&Partial<UnifiedModulesHost>, inputFiles: ReadonlyArray<string>,
|
|
|
|
|
options: ts.CompilerOptions): PluginCompilerHost;
|
|
|
|
|
|
|
|
|
|
setupCompilation(program: ts.Program, oldProgram?: ts.Program): {
|
|
|
|
|
ignoreForDiagnostics: Set<ts.SourceFile>,
|
|
|
|
|
ignoreForEmit: Set<ts.SourceFile>,
|
|
|
|
|
};
|
|
|
|
|
|
|
|
|
|
getDiagnostics(file?: ts.SourceFile): ts.Diagnostic[];
|
|
|
|
|
|
|
|
|
|
getOptionDiagnostics(): ts.Diagnostic[];
|
|
|
|
|
|
|
|
|
|
getNextProgram(): ts.Program;
|
|
|
|
|
|
2020-02-14 11:21:50 -08:00
|
|
|
createTransformers(): ts.CustomTransformers;
|
2020-01-17 16:00:44 -08:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/**
|
|
|
|
|
* A plugin for `tsc_wrapped` which allows Angular compilation from a plain `ts_library`.
|
|
|
|
|
*/
|
2019-01-29 11:33:37 -08:00
|
|
|
export class NgTscPlugin implements TscPlugin {
|
2020-01-17 16:00:44 -08:00
|
|
|
name = 'ngtsc';
|
|
|
|
|
|
|
|
|
|
private options: NgCompilerOptions|null = null;
|
|
|
|
|
private host: NgCompilerHost|null = null;
|
|
|
|
|
private _compiler: NgCompiler|null = null;
|
|
|
|
|
|
|
|
|
|
get compiler(): NgCompiler {
|
|
|
|
|
if (this._compiler === null) {
|
|
|
|
|
throw new Error('Lifecycle error: setupCompilation() must be called first.');
|
|
|
|
|
}
|
|
|
|
|
return this._compiler;
|
2019-01-30 08:30:53 -08:00
|
|
|
}
|
|
|
|
|
|
2020-04-07 12:43:43 -07:00
|
|
|
constructor(private ngOptions: {}) {
|
|
|
|
|
setFileSystem(new NodeJSFileSystem());
|
|
|
|
|
}
|
2020-01-17 16:00:44 -08:00
|
|
|
|
|
|
|
|
wrapHost(
|
2020-07-10 12:41:35 -04:00
|
|
|
host: ts.CompilerHost&Partial<UnifiedModulesHost>, inputFiles: readonly string[],
|
2020-01-17 16:00:44 -08:00
|
|
|
options: ts.CompilerOptions): PluginCompilerHost {
|
perf(compiler-cli): fix regressions in incremental program reuse (#37641)
Commit 4213e8d5 introduced shim reference tagging into the compiler, and
changed how the `TypeCheckProgramHost` worked under the hood during the
creation of a template type-checking program. This work enabled a more
incremental flow for template type-checking, but unintentionally introduced
several regressions in performance, caused by poor incrementality during
`ts.Program` creation.
1. The `TypeCheckProgramHost` was made to rely on the `ts.CompilerHost` to
retrieve instances of `ts.SourceFile`s from the original program. If the
host does not return the original instance of such files, but instead
creates new instances, this has two negative effects: it incurs
additional parsing time, and it interferes with TypeScript's ability to
reuse information about such files.
2. During the incremental creation of a `ts.Program`, TypeScript compares
the `referencedFiles` of `ts.SourceFile` instances from the old program
with those in the new program. If these arrays differ, TypeScript cannot
fully reuse the old program. The implementation of reference tagging
introduced in 4213e8d5 restores the original `referencedFiles` array
after a `ts.Program` is created, which means that future incremental
operations involving that program will always fail this comparison,
effectively limiting the incrementality TypeScript can achieve.
Problem 1 exacerbates problem 2: if a new `ts.SourceFile` is created by the
host after shim generation has been disabled, it will have an untagged
`referencedFiles` array even if the original file's `referencedFiles` was
not restored, triggering problem 2 when creating the template type-checking
program.
To fix these issues, `referencedFiles` arrays are now restored on the old
`ts.Program` prior to the creation of a new incremental program. This allows
TypeScript to get the most out of reusing the old program's data.
Additionally, the `TypeCheckProgramHost` now uses the original `ts.Program`
to retrieve original instances of `ts.SourceFile`s where possible,
preventing issues when a host would otherwise return fresh instances.
Together, these fixes ensure that program reuse is as incremental as
possible, and tests have been added to verify this for certain scenarios.
An optimization was further added to prevent the creation of a type-checking
`ts.Program` in the first place if no type-checking is necessary.
PR Close #37641
2020-06-19 12:55:13 -07:00
|
|
|
// TODO(alxhub): Eventually the `wrapHost()` API will accept the old `ts.Program` (if one is
|
|
|
|
|
// available). When it does, its `ts.SourceFile`s need to be re-tagged to enable proper
|
|
|
|
|
// incremental compilation.
|
2020-04-07 12:43:43 -07:00
|
|
|
this.options = {...this.ngOptions, ...options} as NgCompilerOptions;
|
fix(compiler): switch to 'referencedFiles' for shim generation (#36211)
Shim generation was built on a lie.
Shims are files added to the program which aren't original files authored by
the user, but files authored effectively by the compiler. These fall into
two categories: files which will be generated (like the .ngfactory shims we
generate for View Engine compatibility) as well as files used internally in
compilation (like the __ng_typecheck__.ts file).
Previously, shim generation was driven by the `rootFiles` passed to the
compiler as input. These are effectively the `files` listed in the
`tsconfig.json`. Each shim generator (e.g. the `FactoryGenerator`) would
examine the `rootFiles` and produce a list of shim file names which it would
be responsible for generating. These names would then be added to the
`rootFiles` when the program was created.
The fatal flaw here is that `rootFiles` does not always account for all of
the files in the program. In fact, it's quite rare that it does. Users don't
typically specify every file directly in `files`. Instead, they rely on
TypeScript, during program creation, starting with a few root files and
transitively discovering all of the files in the program.
This happens, however, during `ts.createProgram`, which is too late to add
new files to the `rootFiles` list.
As a result, shim generation was only including shims for files actually
listed in the `tsconfig.json` file, and not for the transitive set of files
in the user's program as it should.
This commit completely rewrites shim generation to use a different technique
for adding files to the program, inspired by View Engine's shim generator.
In this new technique, as the program is being created and `ts.SourceFile`s
are being requested from the `NgCompilerHost`, shims for those files are
generated and a reference to them is patched onto the original file's
`ts.SourceFile.referencedFiles`. This causes TS to think that the original
file references the shim, and causes the shim to be included in the program.
The original `referencedFiles` array is saved and restored after program
creation, hiding this little hack from the rest of the system.
The new shim generation engine differentiates between two kinds of shims:
top-level shims (such as the flat module entrypoint file and
__ng_typecheck__.ts) and per-file shims such as ngfactory or ngsummary
files. The former are included via `rootFiles` as before, the latter are
included via the `referencedFiles` of their corresponding original files.
As a result of this change, shims are now correctly generated for all files
in the program, not just the ones named in `tsconfig.json`.
A few mitigating factors prevented this bug from being realized until now:
* in g3, `files` does include the transitive closure of files in the program
* in CLI apps, shims are not really used
This change also makes use of a novel technique for associating information
with source files: the use of an `NgExtension` `Symbol` to patch the
information directly onto the AST object. This is used in several
circumstances:
* For shims, metadata about a `ts.SourceFile`'s status as a shim and its
origins are held in the extension data.
* For original files, the original `referencedFiles` are stashed in the
extension data for later restoration.
The main benefit of this technique is a lot less bookkeeping around `Map`s
of `ts.SourceFile`s to various kinds of data, which need to be tracked/
invalidated as part of incremental builds.
This technique is based on designs used internally in the TypeScript
compiler and is serving as a prototype of this design in ngtsc. If it works
well, it could have benefits across the rest of the compiler.
PR Close #36211
2020-02-26 16:12:39 -08:00
|
|
|
this.host = NgCompilerHost.wrap(host, inputFiles, this.options, /* oldProgram */ null);
|
2020-01-17 16:00:44 -08:00
|
|
|
return this.host;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
setupCompilation(program: ts.Program, oldProgram?: ts.Program): {
|
|
|
|
|
ignoreForDiagnostics: Set<ts.SourceFile>,
|
|
|
|
|
ignoreForEmit: Set<ts.SourceFile>,
|
|
|
|
|
} {
|
|
|
|
|
if (this.host === null || this.options === null) {
|
|
|
|
|
throw new Error('Lifecycle error: setupCompilation() before wrapHost().');
|
|
|
|
|
}
|
perf(compiler-cli): fix regressions in incremental program reuse (#37641)
Commit 4213e8d5 introduced shim reference tagging into the compiler, and
changed how the `TypeCheckProgramHost` worked under the hood during the
creation of a template type-checking program. This work enabled a more
incremental flow for template type-checking, but unintentionally introduced
several regressions in performance, caused by poor incrementality during
`ts.Program` creation.
1. The `TypeCheckProgramHost` was made to rely on the `ts.CompilerHost` to
retrieve instances of `ts.SourceFile`s from the original program. If the
host does not return the original instance of such files, but instead
creates new instances, this has two negative effects: it incurs
additional parsing time, and it interferes with TypeScript's ability to
reuse information about such files.
2. During the incremental creation of a `ts.Program`, TypeScript compares
the `referencedFiles` of `ts.SourceFile` instances from the old program
with those in the new program. If these arrays differ, TypeScript cannot
fully reuse the old program. The implementation of reference tagging
introduced in 4213e8d5 restores the original `referencedFiles` array
after a `ts.Program` is created, which means that future incremental
operations involving that program will always fail this comparison,
effectively limiting the incrementality TypeScript can achieve.
Problem 1 exacerbates problem 2: if a new `ts.SourceFile` is created by the
host after shim generation has been disabled, it will have an untagged
`referencedFiles` array even if the original file's `referencedFiles` was
not restored, triggering problem 2 when creating the template type-checking
program.
To fix these issues, `referencedFiles` arrays are now restored on the old
`ts.Program` prior to the creation of a new incremental program. This allows
TypeScript to get the most out of reusing the old program's data.
Additionally, the `TypeCheckProgramHost` now uses the original `ts.Program`
to retrieve original instances of `ts.SourceFile`s where possible,
preventing issues when a host would otherwise return fresh instances.
Together, these fixes ensure that program reuse is as incremental as
possible, and tests have been added to verify this for certain scenarios.
An optimization was further added to prevent the creation of a type-checking
`ts.Program` in the first place if no type-checking is necessary.
PR Close #37641
2020-06-19 12:55:13 -07:00
|
|
|
this.host.postProgramCreationCleanup();
|
|
|
|
|
untagAllTsFiles(program);
|
perf(compiler-cli): split Ivy template type-checking into multiple files (#36211)
As a performance optimization, this commit splits the single
__ngtypecheck__.ts file which was previously added to the user's program as
a container for all template type-checking code into multiple .ngtypecheck
shim files, one for each original file in the user's program.
In larger applications, the generation, parsing, and checking of this single
type-checking file was a huge performance bottleneck, with the file often
exceeding 1 MB in text content. Particularly in incremental builds,
regenerating this single file for the entire application proved especially
expensive.
This commit introduces a new strategy for template type-checking code which
makes use of a new interface, the `TypeCheckingProgramStrategy`. This
interface abstracts the process of creating a new `ts.Program` to type-check
a particular compilation, and allows the mechanism there to be kept separate
from the more complex logic around dealing with multiple .ngtypecheck files.
A new `TemplateTypeChecker` hosts that logic and interacts with the
`TypeCheckingProgramStrategy` to actually generate and return diagnostics.
The `TypeCheckContext` class, previously the workhorse of template type-
checking, is now solely focused on collecting and generating type-checking
file contents.
A side effect of implementing the new `TypeCheckingProgramStrategy` in this
way is that the API is designed to be suitable for use by the Angular
Language Service as well. The LS also needs to type-check components, but
has its own method for constructing a `ts.Program` with type-checking code.
Note that this commit does not make the actual checking of templates at all
_incremental_ just yet. That will happen in a future commit.
PR Close #36211
2020-03-04 15:50:12 -08:00
|
|
|
const typeCheckStrategy = new ReusedProgramStrategy(
|
|
|
|
|
program, this.host, this.options, this.host.shimExtensionPrefixes);
|
|
|
|
|
this._compiler = new NgCompiler(
|
2020-05-28 16:08:52 -07:00
|
|
|
this.host, this.options, program, typeCheckStrategy,
|
2020-08-27 11:32:33 -07:00
|
|
|
new PatchedProgramIncrementalBuildStrategy(), /** enableTemplateTypeChecker */ false,
|
fix(compiler-cli): track poisoned scopes with a flag (#39923)
To avoid overwhelming a user with secondary diagnostics that derive from a
"root cause" error, the compiler has the notion of a "poisoned" NgModule.
An NgModule becomes poisoned when its declaration contains semantic errors:
declarations which are not components or pipes, imports which are not other
NgModules, etc. An NgModule also becomes poisoned if it imports or exports
another poisoned NgModule.
Previously, the compiler tracked this poisoned status as an alternate state
for each scope. Either a correct scope could be produced, or the entire
scope would be set to a sentinel error value. This meant that the compiler
would not track any information about a scope that was determined to be in
error.
This method presents several issues:
1. The compiler is unable to support the language service and return results
when a component or its module scope is poisoned.
This is fine for compilation, since diagnostics will be produced showing the
error(s), but the language service needs to still work for incorrect code.
2. `getComponentScopes()` does not return components with a poisoned scope,
which interferes with resource tracking of incremental builds.
If the component isn't included in that list, then the NgModule for it will
not have its dependencies properly tracked, and this can cause future
incremental build steps to produce incorrect results.
This commit changes the tracking of poisoned module scopes to use a flag on
the scope itself, rather than a sentinel value that replaces the scope. This
means that the scope itself will still be tracked, even if it contains
semantic errors. A test is added to the language service which verifies that
poisoned scopes can still be used in template type-checking.
PR Close #39923
2020-11-25 15:02:23 -08:00
|
|
|
/* usePoisonedData */ false, oldProgram, NOOP_PERF_RECORDER);
|
2020-01-17 16:00:44 -08:00
|
|
|
return {
|
|
|
|
|
ignoreForDiagnostics: this._compiler.ignoreForDiagnostics,
|
|
|
|
|
ignoreForEmit: this._compiler.ignoreForEmit,
|
2019-01-29 11:33:37 -08:00
|
|
|
};
|
|
|
|
|
}
|
|
|
|
|
|
2020-01-17 16:00:44 -08:00
|
|
|
getDiagnostics(file?: ts.SourceFile): ts.Diagnostic[] {
|
|
|
|
|
return this.compiler.getDiagnostics(file);
|
2019-01-29 11:33:37 -08:00
|
|
|
}
|
2020-01-17 16:00:44 -08:00
|
|
|
|
2020-04-07 12:43:43 -07:00
|
|
|
getOptionDiagnostics(): ts.Diagnostic[] {
|
|
|
|
|
return this.compiler.getOptionDiagnostics();
|
|
|
|
|
}
|
2020-01-17 16:00:44 -08:00
|
|
|
|
2020-04-07 12:43:43 -07:00
|
|
|
getNextProgram(): ts.Program {
|
|
|
|
|
return this.compiler.getNextProgram();
|
|
|
|
|
}
|
2020-01-17 16:00:44 -08:00
|
|
|
|
2020-04-07 12:43:43 -07:00
|
|
|
createTransformers(): ts.CustomTransformers {
|
|
|
|
|
return this.compiler.prepareEmit().transformers;
|
|
|
|
|
}
|
2019-01-29 11:33:37 -08:00
|
|
|
}
|