honeymoose
/
angular-cn
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

fix(ivy): track changes across failed builds (#33971) 

Previously, our incremental build system kept track of the changes between
the current compilation and the previous one, and used its knowledge of
inter-file dependencies to evaluate the impact of each change and emit the
right set of output files.

However, a problem arose if the compiler was not able to extract a
dependency graph successfully. This typically happens if the input program
contains errors. In this case the Angular analysis part of compilation is
never executed.

If a file changed in one of these failed builds, in the next build it
appears unchanged. This means that the compiler "forgets" to emit it!

To fix this problem, the compiler needs to know the set of changes made
_since the last successful build_, not simply since the last invocation.

This commit changes the incremental state system to much more explicitly
pass information from the previous to the next compilation, and in the
process to keep track of changes across multiple failed builds, until the
program can be analyzed successfully and the results of those changes
incorporated into the emit plan.

Fixes #32214

PR Close #33971
This commit is contained in:
Alex Rickabaugh 2019-11-21 14:37:53 -08:00 committed by Matias Niemelä
parent a4c3ceeddb
commit 4cf197998a
6 changed files with 565 additions and 76 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
packages/compiler-cli/src/ngtsc/incremental/index.ts
View File

@ -6,4 +6,4 @@
* found in the LICENSE file at https://angular.io/license
*/
export {IncrementalState} from './src/state';
export {IncrementalDriver} from './src/state';

75
packages/compiler-cli/src/ngtsc/incremental/src/README.md
View File

@ -2,21 +2,7 @@
This package contains logic related to incremental compilation in ngtsc.
In particular, it tracks dependencies between `ts.SourceFile`s, so the compiler can make intelligent decisions about when it's safe to skip certain operations.
# How does incremental compilation work?
The initial compilation is no different from a standalone compilation; the compiler is unaware that incremental compilation will be utilized.
When an `NgtscProgram` is created for a _subsequent_ compilation, it is initialized with the `NgtscProgram` from the previous compilation. It is therefore able to take advantage of any information present in the previous compilation to optimize the next one.
This information is leveraged in two major ways:
1) The previous `ts.Program` itself is used to create the next `ts.Program`, allowing TypeScript internally to leverage information from the previous compile in much the same way.
2) An `IncrementalState` instance is constructed from the old and new `ts.Program`s.
The compiler then proceeds normally, analyzing all of the Angular code within the program. As a part of this process, the compiler maps out all of the dependencies between files in the `IncrementalState`.
In particular, it tracks dependencies between `ts.SourceFile`s, so the compiler can make intelligent decisions about when it's safe to skip certain operations. The main class performing this task is the `IncrementalDriver`.
# What optimizations are made?
@ -30,6 +16,65 @@ The second condition is challenging to prove, as Angular allows statically evalu
The emit of a file is the most expensive part of TypeScript/Angular compilation, so skipping emits when they are not necessary is one of the most valuable things the compiler can do to improve incremental build performance.
# How does incremental compilation work?
The initial compilation is no different from a standalone compilation; the compiler is unaware that incremental compilation will be utilized.
When an `NgtscProgram` is created for a _subsequent_ compilation, it is initialized with the `NgtscProgram` from the previous compilation. It is therefore able to take advantage of any information present in the previous compilation to optimize the next one.
This information is leveraged in two major ways:
1) The previous `ts.Program` itself is used to create the next `ts.Program`, allowing TypeScript internally to leverage information from the previous compile in much the same way.
2) An `IncrementalDriver` instance is constructed from the old and new `ts.Program`s, and the previous program's `IncrementalDriver`.
The compiler then proceeds normally, analyzing all of the Angular code within the program. As a part of this process, the compiler maps out all of the dependencies between files in the `IncrementalDriver`.
## Determination of files to emit
The principle question the incremental build system must answer is "which TS files need to be emitted for a given compilation?"
To determine whether an individual TS file needs to be emitted, the compiler must determine 3 things about the file:
1. Have its contents changed since the last time it was emitted?
2. Has any resource file that the TS file depends on (like an HTML template) changed since the last time it was emitted?
3. Have any of the dependencies of the TS file changed since the last time it was emitted?
If the answer to any of these questions is yes, then the TS file needs to be re-emitted.
## Tracking of changes
On every invocation, the compiler receives (or can easily determine) several pieces of information:
* The set of `ts.SourceFile`s that have changed since the last invocation.
* The set of resources (`.html` files) that have changed since the last invocation.
With this information, the compiler can perform rebuild optimizations:
1. The compiler analyzes the full program and generates a dependency graph, which describes the relationships between files in the program.
2. Based on this graph, the compiler can make a determination for each TS file whether it needs to be re-emitted or can safely be skipped. This produces a set called `pendingEmit` of every file which requires a re-emit.
3. The compiler cycles through the files and emits those which are necessary, removing them from `pendingEmit`.
Theoretically, after this process `pendingEmit` should be empty. As a precaution against errors which might happen in the future, `pendingEmit` is also passed into future compilations, so any files which previously were determined to need an emit (but have not been successfully produced yet) will be retried on subsequent compilations. This is mostly relevant if a client of `ngtsc` attempts to implement emit-on-error functionality.
However, normally the execution of these steps requires a correct input program. In the presence of TypeScript errors, the compiler cannot perform this process. It might take many invocations for the user to fix all their TypeScript errors and reach a compilation that can be analyzed.
As a result, the compiler must accumulate the set of these changes (to source files and resource files) from build to build until analysis can succeed.
This accumulation happens via a type called `BuildState`. This type is a union of two possible states.
### `PendingBuildState`
This is the initial state of any build, and the final state of any unsuccessful build. This state tracks both `pendingEmit` files from the previous program as well as any source or resource files which have changed since the last successful analysis.
If a new build starts and inherits from a failed build, it will merge the failed build's `PendingBuildState` into its own, including the sets of changed files.
### `AnalyzedBuildState`
After analysis is successfully performed, the compiler uses its dependency graph to evaluate the impact of any accumulated changes from the `PendingBuildState`, and updates `pendingEmit` with all of the pending files. At this point, the compiler transitions from a `PendingBuildState` to an `AnalyzedBuildState`, which only tracks `pendingEmit`. In `AnalyzedBuildState` this set is complete, and the raw changes can be forgotten.
If a new build is started after a successful build, only `pendingEmit` from the `AnalyzedBuildState` needs to be merged into the new build's `PendingBuildState`.
# What optimizations are possible in the future?
There is plenty of room for improvement here, with diminishing returns for the work involved.

250
packages/compiler-cli/src/ngtsc/incremental/src/state.ts
View File

@ -12,49 +12,156 @@ import {DependencyTracker} from '../../partial_evaluator';
import {ResourceDependencyRecorder} from '../../util/src/resource_recorder';
/**
* Accumulates state between compilations.
* Drives an incremental build, by tracking changes and determining which files need to be emitted.
*/
export class IncrementalState implements DependencyTracker, ResourceDependencyRecorder {
private constructor(
private unchangedFiles: Set<ts.SourceFile>,
private metadata: Map<ts.SourceFile, FileMetadata>,
private modifiedResourceFiles: Set<string>|null) {}
export class IncrementalDriver implements DependencyTracker, ResourceDependencyRecorder {
/**
* State of the current build.
*
* This transitions as the compilation progresses.
*/
private state: BuildState;
/**
* Tracks metadata related to each `ts.SourceFile` in the program.
*/
private metadata = new Map<ts.SourceFile, FileMetadata>();
private constructor(state: PendingBuildState, private allTsFiles: Set<ts.SourceFile>) {
this.state = state;
}
/**
* Construct an `IncrementalDriver` with a starting state that incorporates the results of a
* previous build.
*
* The previous build's `BuildState` is reconciled with the new program's changes, and the results
* are merged into the new build's `PendingBuildState`.
*/
static reconcile(
oldProgram: ts.Program, newProgram: ts.Program,
modifiedResourceFiles: Set<string>|null): IncrementalState {
const unchangedFiles = new Set<ts.SourceFile>();
const metadata = new Map<ts.SourceFile, FileMetadata>();
const oldFiles = new Set<ts.SourceFile>(oldProgram.getSourceFiles());
oldProgram: ts.Program, oldDriver: IncrementalDriver, newProgram: ts.Program,
modifiedResourceFiles: Set<string>|null): IncrementalDriver {
// Initialize the state of the current build based on the previous one.
let state: PendingBuildState;
if (oldDriver.state.kind === BuildStateKind.Pending) {
// The previous build never made it past the pending state. Reuse it as the starting state for
// this build.
state = oldDriver.state;
} else {
// The previous build was successfully analyzed. `pendingEmit` is the only state carried
// forward into this build.
state = {
kind: BuildStateKind.Pending,
pendingEmit: oldDriver.state.pendingEmit,
changedResourcePaths: new Set<string>(),
changedTsPaths: new Set<string>(),
};
}
// Compute the set of files that are unchanged (both in themselves and their dependencies).
for (const newFile of newProgram.getSourceFiles()) {
if (newFile.isDeclarationFile && !oldFiles.has(newFile)) {
// Bail out and re-emit everything if a .d.ts file has changed - currently the compiler does
// not track dependencies into .d.ts files very well.
return IncrementalState.fresh();
} else if (oldFiles.has(newFile)) {
unchangedFiles.add(newFile);
// Merge the freshly modified resource files with any prior ones.
if (modifiedResourceFiles !== null) {
for (const resFile of modifiedResourceFiles) {
state.changedResourcePaths.add(resFile);
}
}
return new IncrementalState(unchangedFiles, metadata, modifiedResourceFiles);
// Next, process the files in the new program, with a couple of goals:
// 1) Determine which TS files have changed, if any, and merge them into `changedTsFiles`.
// 2) Produce a list of TS files which no longer exist in the program (they've been deleted
// since the previous compilation). These need to be removed from the state tracking to avoid
// leaking memory.
// All files in the old program, for easy detection of changes.
const oldFiles = new Set<ts.SourceFile>(oldProgram.getSourceFiles());
// Assume all the old files were deleted to begin with. Only TS files are tracked.
const deletedTsPaths = new Set<string>(tsOnlyFiles(oldProgram).map(sf => sf.fileName));
for (const newFile of newProgram.getSourceFiles()) {
if (!newFile.isDeclarationFile) {
// This file exists in the new program, so remove it from `deletedTsPaths`.
deletedTsPaths.delete(newFile.fileName);
}
if (oldFiles.has(newFile)) {
// This file hasn't changed; no need to look at it further.
continue;
}
// The file has changed since the last successful build. The appropriate reaction depends on
// what kind of file it is.
if (!newFile.isDeclarationFile) {
// It's a .ts file, so track it as a change.
state.changedTsPaths.add(newFile.fileName);
} else {
// It's a .d.ts file. Currently the compiler does not do a great job of tracking
// dependencies on .d.ts files, so bail out of incremental builds here and do a full build.
// This usually only happens if something in node_modules changes.
return IncrementalDriver.fresh(newProgram);
}
}
// The last step is to remove any deleted files from the state.
for (const filePath of deletedTsPaths) {
state.pendingEmit.delete(filePath);
// Even if the file doesn't exist in the current compilation, it still might have been changed
// in a previous one, so delete it from the set of changed TS files, just in case.
state.changedTsPaths.delete(filePath);
}
// `state` now reflects the initial compilation state of the current
return new IncrementalDriver(state, new Set<ts.SourceFile>(tsOnlyFiles(newProgram)));
}
static fresh(): IncrementalState {
return new IncrementalState(
new Set<ts.SourceFile>(), new Map<ts.SourceFile, FileMetadata>(), null);
static fresh(program: ts.Program): IncrementalDriver {
// Initialize the set of files which need to be emitted to the set of all TS files in the
// program.
const tsFiles = tsOnlyFiles(program);
const state: PendingBuildState = {
kind: BuildStateKind.Pending,
pendingEmit: new Set<string>(tsFiles.map(sf => sf.fileName)),
changedResourcePaths: new Set<string>(),
changedTsPaths: new Set<string>(),
};
return new IncrementalDriver(state, new Set(tsFiles));
}
safeToSkip(sf: ts.SourceFile): boolean {
// It's safe to skip emitting a file if:
// 1) it hasn't changed
// 2) none if its resource dependencies have changed
// 3) none of its source dependencies have changed
return this.unchangedFiles.has(sf) && !this.hasChangedResourceDependencies(sf) &&
this.getFileDependencies(sf).every(dep => this.unchangedFiles.has(dep));
recordSuccessfulAnalysis(): void {
if (this.state.kind !== BuildStateKind.Pending) {
// Changes have already been incorporated.
return;
}
const pendingEmit = this.state.pendingEmit;
const state: PendingBuildState = this.state;
for (const sf of this.allTsFiles) {
// It's safe to skip emitting a file if:
// 1) it hasn't changed
// 2) none if its resource dependencies have changed
// 3) none of its source dependencies have changed
if (state.changedTsPaths.has(sf.fileName) || this.hasChangedResourceDependencies(sf) ||
this.getFileDependencies(sf).some(dep => state.changedTsPaths.has(dep.fileName))) {
// Something has changed which requires this file be re-emitted.
pendingEmit.add(sf.fileName);
}
}
// Update the state to an `AnalyzedBuildState`.
this.state = {
kind: BuildStateKind.Analyzed,
pendingEmit,
};
}
recordSuccessfulEmit(sf: ts.SourceFile): void { this.state.pendingEmit.delete(sf.fileName); }
safeToSkipEmit(sf: ts.SourceFile): boolean { return !this.state.pendingEmit.has(sf.fileName); }
trackFileDependency(dep: ts.SourceFile, src: ts.SourceFile) {
const metadata = this.ensureMetadata(src);
metadata.fileDependencies.add(dep);
@ -87,12 +194,14 @@ export class IncrementalState implements DependencyTracker, ResourceDependencyRe
}
private hasChangedResourceDependencies(sf: ts.SourceFile): boolean {
if (this.modifiedResourceFiles === null || !this.metadata.has(sf)) {
if (!this.metadata.has(sf)) {
return false;
}
const resourceDeps = this.metadata.get(sf) !.resourcePaths;
return Array.from(resourceDeps.keys())
.some(resourcePath => this.modifiedResourceFiles !.has(resourcePath));
.some(
resourcePath => this.state.kind === BuildStateKind.Pending &&
this.state.changedResourcePaths.has(resourcePath));
}
}
@ -104,3 +213,80 @@ class FileMetadata {
fileDependencies = new Set<ts.SourceFile>();
resourcePaths = new Set<string>();
}
type BuildState = PendingBuildState | AnalyzedBuildState;
enum BuildStateKind {
Pending,
Analyzed,
}
interface BaseBuildState {
kind: BuildStateKind;
/**
* The heart of incremental builds. This `Set` tracks the set of files which need to be emitted
* during the current compilation.
*
* This starts out as the set of files which are still pending from the previous program (or the
* full set of .ts files on a fresh build).
*
* After analysis, it's updated to include any files which might have changed and need a re-emit
* as a result of incremental changes.
*
* If an emit happens, any written files are removed from the `Set`, as they're no longer pending.
*
* Thus, after compilation `pendingEmit` should be empty (on a successful build) or contain the
* files which still need to be emitted but have not yet been (due to errors).
*
* `pendingEmit` is tracked as as `Set<string>` instead of a `Set<ts.SourceFile>`, because the
* contents of the file are not important here, only whether or not the current version of it
* needs to be emitted. The `string`s here are TS file paths.
*
* See the README.md for more information on this algorithm.
*/
pendingEmit: Set<string>;
}
/**
* State of a build before the Angular analysis phase completes.
*/
interface PendingBuildState extends BaseBuildState {
kind: BuildStateKind.Pending;
/**
* Set of files which are known to need an emit.
*
* Before the compiler's analysis phase completes, `pendingEmit` only contains files that were
* still pending after the previous build.
*/
pendingEmit: Set<string>;
/**
* Set of TypeScript file paths which have changed since the last successfully analyzed build.
*/
changedTsPaths: Set<string>;
/**
* Set of resource file paths which have changed since the last successfully analyzed build.
*/
changedResourcePaths: Set<string>;
}
interface AnalyzedBuildState extends BaseBuildState {
kind: BuildStateKind.Analyzed;
/**
* Set of files which are known to need an emit.
*
* After analysis completes (that is, the state transitions to `AnalyzedBuildState`), the
* `pendingEmit` set takes into account any on-disk changes made since the last successfully
* analyzed build.
*/
pendingEmit: Set<string>;
}
function tsOnlyFiles(program: ts.Program): ReadonlyArray<ts.SourceFile> {
return program.getSourceFiles().filter(sf => !sf.isDeclarationFile);
}

38
packages/compiler-cli/src/ngtsc/program.ts
View File

@ -19,7 +19,7 @@ import {ErrorCode, ngErrorCode} from './diagnostics';
import {FlatIndexGenerator, ReferenceGraph, checkForPrivateExports, findFlatIndexEntryPoint} from './entry_point';
import {AbsoluteFsPath, LogicalFileSystem, absoluteFrom} from './file_system';
import {AbsoluteModuleStrategy, AliasStrategy, AliasingHost, DefaultImportTracker, FileToModuleAliasingHost, FileToModuleHost, FileToModuleStrategy, ImportRewriter, LocalIdentifierStrategy, LogicalProjectStrategy, ModuleResolver, NoopImportRewriter, PrivateExportAliasingHost, R3SymbolsImportRewriter, Reference, ReferenceEmitStrategy, ReferenceEmitter, RelativePathStrategy} from './imports';
import {IncrementalState} from './incremental';
import {IncrementalDriver} from './incremental';
import {IndexedComponent, IndexingContext} from './indexer';
import {generateAnalysis} from './indexer/src/transform';
import {CompoundMetadataReader, CompoundMetadataRegistry, DtsMetadataReader, LocalMetadataRegistry, MetadataReader} from './metadata';
@ -66,7 +66,7 @@ export class NgtscProgram implements api.Program {
private defaultImportTracker: DefaultImportTracker;
private perfRecorder: PerfRecorder = NOOP_PERF_RECORDER;
private perfTracker: PerfTracker|null = null;
private incrementalState: IncrementalState;
private incrementalDriver: IncrementalDriver;
private typeCheckFilePath: AbsoluteFsPath;
private modifiedResourceFiles: Set<string>|null;
@ -183,10 +183,11 @@ export class NgtscProgram implements api.Program {
this.cycleAnalyzer = new CycleAnalyzer(new ImportGraph(this.moduleResolver));
this.defaultImportTracker = new DefaultImportTracker();
if (oldProgram === undefined) {
this.incrementalState = IncrementalState.fresh();
this.incrementalDriver = IncrementalDriver.fresh(this.tsProgram);
} else {
this.incrementalState = IncrementalState.reconcile(
oldProgram.reuseTsProgram, this.tsProgram, this.modifiedResourceFiles);
this.incrementalDriver = IncrementalDriver.reconcile(
oldProgram.reuseTsProgram, oldProgram.incrementalDriver, this.tsProgram,
this.modifiedResourceFiles);
}
}
@ -253,6 +254,10 @@ export class NgtscProgram implements api.Program {
.filter((result): result is Promise<void> => result !== undefined));
this.perfRecorder.stop(analyzeSpan);
this.compilation.resolve();
// At this point, analysis is complete and the compiler can now calculate which files need to be
// emitted, so do that.
this.incrementalDriver.recordSuccessfulAnalysis();
}
listLazyRoutes(entryRoute?: string|undefined): api.LazyRoute[] {
@ -315,6 +320,10 @@ export class NgtscProgram implements api.Program {
});
this.perfRecorder.stop(analyzeSpan);
this.compilation.resolve();
// At this point, analysis is complete and the compiler can now calculate which files need to
// be emitted, so do that.
this.incrementalDriver.recordSuccessfulAnalysis();
}
return this.compilation;
}
@ -334,6 +343,17 @@ export class NgtscProgram implements api.Program {
(fileName: string, data: string, writeByteOrderMark: boolean,
onError: ((message: string) => void) | undefined,
sourceFiles: ReadonlyArray<ts.SourceFile>| undefined) => {
if (sourceFiles !== undefined) {
// Record successful writes for any `ts.SourceFile` (that's not a declaration file)
// that's an input to this write.
for (const writtenSf of sourceFiles) {
if (writtenSf.isDeclarationFile) {
continue;
}
this.incrementalDriver.recordSuccessfulEmit(writtenSf);
}
}
if (this.closureCompilerEnabled && fileName.endsWith('.js')) {
data = nocollapseHack(data);
}
@ -377,7 +397,7 @@ export class NgtscProgram implements api.Program {
continue;
}
if (this.incrementalState.safeToSkip(targetSourceFile)) {
if (this.incrementalDriver.safeToSkipEmit(targetSourceFile)) {
continue;
}
@ -571,7 +591,7 @@ export class NgtscProgram implements api.Program {
this.aliasingHost = new FileToModuleAliasingHost(this.fileToModuleHost);
}
const evaluator = new PartialEvaluator(this.reflector, checker, this.incrementalState);
const evaluator = new PartialEvaluator(this.reflector, checker, this.incrementalDriver);
const dtsReader = new DtsMetadataReader(checker, this.reflector);
const localMetaRegistry = new LocalMetadataRegistry();
const localMetaReader: MetadataReader = localMetaRegistry;
@ -605,7 +625,7 @@ export class NgtscProgram implements api.Program {
this.options.preserveWhitespaces || false, this.options.i18nUseExternalIds !== false,
this.getI18nLegacyMessageFormat(), this.moduleResolver, this.cycleAnalyzer,
this.refEmitter, this.defaultImportTracker, this.closureCompilerEnabled,
this.incrementalState),
this.incrementalDriver),
new DirectiveDecoratorHandler(
this.reflector, evaluator, metaRegistry, this.defaultImportTracker, this.isCore,
this.closureCompilerEnabled),
@ -621,7 +641,7 @@ export class NgtscProgram implements api.Program {
];
return new IvyCompilation(
handlers, this.reflector, this.importRewriter, this.incrementalState, this.perfRecorder,
handlers, this.reflector, this.importRewriter, this.incrementalDriver, this.perfRecorder,
this.sourceToFactorySymbols, scopeRegistry);
}

16
packages/compiler-cli/src/ngtsc/transform/src/compilation.ts
View File

@ -11,7 +11,7 @@ import * as ts from 'typescript';
import {ErrorCode, FatalDiagnosticError} from '../../diagnostics';
import {ImportRewriter} from '../../imports';
import {IncrementalState} from '../../incremental';
import {IncrementalDriver} from '../../incremental';
import {IndexingContext} from '../../indexer';
import {PerfRecorder} from '../../perf';
import {ClassDeclaration, ReflectionHost, isNamedClassDeclaration, reflectNameOfDeclaration} from '../../reflection';
@ -78,7 +78,7 @@ export class IvyCompilation {
*/
constructor(
private handlers: DecoratorHandler<any, any>[], private reflector: ReflectionHost,
private importRewriter: ImportRewriter, private incrementalState: IncrementalState,
private importRewriter: ImportRewriter, private incrementalDriver: IncrementalDriver,
private perf: PerfRecorder, private sourceToFactorySymbols: Map<string, Set<string>>|null,
private scopeRegistry: LocalModuleScopeRegistry) {}
@ -308,11 +308,11 @@ export class IvyCompilation {
// A change to any dependency of the declaration causes the declaration to be invalidated,
// which requires the NgModule to be invalidated as well.
const deps = this.incrementalState.getFileDependencies(file);
this.incrementalState.trackFileDependencies(deps, ngModuleFile);
const deps = this.incrementalDriver.getFileDependencies(file);
this.incrementalDriver.trackFileDependencies(deps, ngModuleFile);
// A change to the NgModule file should cause the declaration itself to be invalidated.
this.incrementalState.trackFileDependency(ngModuleFile, file);
this.incrementalDriver.trackFileDependency(ngModuleFile, file);
// A change to any directive/pipe in the compilation scope should cause the declaration to be
// invalidated.
@ -321,19 +321,19 @@ export class IvyCompilation {
// When a directive in scope is updated, the declaration needs to be recompiled as e.g.
// a selector may have changed.
this.incrementalState.trackFileDependency(dirSf, file);
this.incrementalDriver.trackFileDependency(dirSf, file);
// When any of the dependencies of the declaration changes, the NgModule scope may be
// affected so a component within scope must be recompiled. Only components need to be
// recompiled, as directives are not dependent upon the compilation scope.
if (directive.isComponent) {
this.incrementalState.trackFileDependencies(deps, dirSf);
this.incrementalDriver.trackFileDependencies(deps, dirSf);
}
});
scope.pipes.forEach(pipe => {
// When a pipe in scope is updated, the declaration needs to be recompiled as e.g.
// the pipe's name may have changed.
this.incrementalState.trackFileDependency(pipe.ref.node.getSourceFile(), file);
this.incrementalDriver.trackFileDependency(pipe.ref.node.getSourceFile(), file);
});
});
this.perf.stop(recordSpan);

260
packages/compiler-cli/test/ngtsc/incremental_error_spec.ts
View File

@ -39,6 +39,9 @@ runInEachFileSystem(() => {
// Validate that 2x the size of `files` have been written (one .d.ts, one .js) and no more.
expect(set.size).toBe(2 * files.length);
// Reset for the next compilation.
env.flushWrittenFileTracking();
}
it('should handle an error in an unrelated file', () => {
@ -54,12 +57,12 @@ runInEachFileSystem(() => {
// Start with a clean compilation.
env.driveMain();
env.flushWrittenFileTracking();
// Introduce the error.
env.write('other.ts', `
export class Other // missing braces
`);
env.flushWrittenFileTracking();
const diags = env.driveDiagnostics();
expect(diags.length).toBe(1);
expect(diags[0].file !.fileName).toBe(_('/other.ts'));
@ -69,12 +72,120 @@ runInEachFileSystem(() => {
env.write('other.ts', `
export class Other {}
`);
env.flushWrittenFileTracking();
env.driveMain();
expectToHaveWritten(['/other.js']);
});
it('should emit all files after an error on the initial build', () => {
// Intentionally start with a broken compilation.
env.write('cmp.ts', `
import {Component} from '@angular/core';
@Component({selector: 'test-cmp', template: '...'})
export class TestCmp {}
`);
env.write('other.ts', `
export class Other // missing braces
`);
const diags = env.driveDiagnostics();
expect(diags.length).toBe(1);
expect(diags[0].file !.fileName).toBe(_('/other.ts'));
expectToHaveWritten([]);
// Remove the error. All files should be emitted.
env.write('other.ts', `
export class Other {}
`);
env.driveMain();
expectToHaveWritten(['/cmp.js', '/other.js', '/unrelated.js']);
});
it('should emit files introduced at the same time as an unrelated error', () => {
env.write('other.ts', `
// Needed so that the initial program contains @angular/core's .d.ts file.
import '@angular/core';
export class Other {}
`);
// Clean compile.
env.driveMain();
env.flushWrittenFileTracking();
env.write('cmp.ts', `
import {Component} from '@angular/core';
@Component({selector: 'test-cmp', template: '...'})
export class TestCmp {}
`);
env.write('other.ts', `
export class Other // missing braces
`);
const diags = env.driveDiagnostics();
expect(diags.length).toBe(1);
expect(diags[0].file !.fileName).toBe(_('/other.ts'));
expectToHaveWritten([]);
// Remove the error. All files should be emitted.
env.write('other.ts', `
export class Other {}
`);
env.driveMain();
expectToHaveWritten(['/cmp.js', '/other.js']);
});
it('should emit dependent files even in the face of an error', () => {
env.write('cmp.ts', `
import {Component} from '@angular/core';
import {SELECTOR} from './selector';
@Component({selector: SELECTOR, template: '...'})
export class TestCmp {}
`);
env.write('selector.ts', `
export const SELECTOR = 'test-cmp';
`);
env.write('other.ts', `
// Needed so that the initial program contains @angular/core's .d.ts file.
import '@angular/core';
export class Other {}
`);
// Clean compile.
env.driveMain();
env.flushWrittenFileTracking();
env.write('cmp.ts', `
import {Component} from '@angular/core';
@Component({selector: 'test-cmp', template: '...'})
export class TestCmp {}
`);
env.write('other.ts', `
export class Other // missing braces
`);
const diags = env.driveDiagnostics();
expect(diags.length).toBe(1);
expect(diags[0].file !.fileName).toBe(_('/other.ts'));
expectToHaveWritten([]);
// Remove the error. All files should be emitted.
env.write('other.ts', `
export class Other {}
`);
env.driveMain();
expectToHaveWritten(['/cmp.js', '/other.js']);
});
it('should recover from an error in a component\'s metadata', () => {
env.write('test.ts', `
import {Component} from '@angular/core';
@ -85,18 +196,18 @@ runInEachFileSystem(() => {
// Start with a clean compilation.
env.driveMain();
env.flushWrittenFileTracking();
// Introduce the error.
env.write('test.ts', `
import {Component} from '@angular/core';
@Component({selector: 'test-cmp', template: ...})
@Component({selector: 'test-cmp', template: ...}) // invalid template
export class TestCmp {}
`);
env.flushWrittenFileTracking();
const diags = env.driveDiagnostics();
expect(diags.length).toBeGreaterThan(0);
expect(env.getFilesWrittenSinceLastFlush()).not.toContain(_('/test.js'));
expectToHaveWritten([]);
// Clear the error and verify that the compiler now emits test.js again.
env.write('test.ts', `
@ -106,9 +217,7 @@ runInEachFileSystem(() => {
export class TestCmp {}
`);
env.flushWrittenFileTracking();
env.driveMain();
expectToHaveWritten(['/test.js']);
});
@ -141,6 +250,7 @@ runInEachFileSystem(() => {
// Start with a clean compilation.
env.driveMain();
env.flushWrittenFileTracking();
// Introduce the syntactic error.
env.write('test.ts', `
@ -149,7 +259,6 @@ runInEachFileSystem(() => {
@Component({selector: ..., template: '...'}) // ... is not valid syntax
export class TestCmp {}
`);
env.flushWrittenFileTracking();
const diags = env.driveDiagnostics();
expect(diags.length).toBeGreaterThan(0);
expectToHaveWritten([]);
@ -162,7 +271,6 @@ runInEachFileSystem(() => {
export class TestCmp {}
`);
env.flushWrittenFileTracking();
env.driveMain();
expectToHaveWritten([
@ -233,6 +341,7 @@ runInEachFileSystem(() => {
// Start with a clean compilation.
env.driveMain();
env.flushWrittenFileTracking();
// Introduce the error in LibModule
env.write('lib.ts', `
@ -250,7 +359,6 @@ runInEachFileSystem(() => {
})
export class LibModule // missing braces
`);
env.flushWrittenFileTracking();
// env.driveMain();
const diags = env.driveDiagnostics();
expect(diags.length).toBeGreaterThan(0);
@ -273,7 +381,6 @@ runInEachFileSystem(() => {
export class LibModule {}
`);
env.flushWrittenFileTracking();
env.driveMain();
expectToHaveWritten([
@ -288,5 +395,136 @@ runInEachFileSystem(() => {
'/lib.js',
]);
});
describe('chained errors', () => {
it('should remember a change to a TS file across broken builds', () => {
// Two components, an NgModule, and a random file.
writeTwoComponentSystem(env);
writeRandomFile(env, 'other.ts');
// Start with a clean build.
env.driveMain();
env.flushWrittenFileTracking();
// Update ACmp to have a different selector, isn't matched in BCmp's template.
env.write('a.ts', `
import {Component} from '@angular/core';
@Component({selector: 'not-a-cmp', template: '...'})
export class ACmp {}
`);
// Update the file to have an error, simultaneously.
writeRandomFile(env, 'other.ts', {error: true});
// This build should fail.
const diags = env.driveDiagnostics();
expect(diags.length).not.toBe(0);
expectToHaveWritten([]);
// Fix the error.
writeRandomFile(env, 'other.ts');
// Rebuild.
env.driveMain();
// If the compiler behaves correctly, it should remember that 'a.ts' was updated before, and
// should regenerate b.ts.
expectToHaveWritten([
// Because they directly changed
'/other.js',
'/a.js',
// Bcause they depend on a.ts
'/b.js',
'/module.js',
]);
});
it('should remember a change to a template file across broken builds', () => {
// This is basically the same test as above, except a.html is changed instead of a.ts.
// Two components, an NgModule, and a random file.
writeTwoComponentSystem(env);
writeRandomFile(env, 'other.ts');
// Start with a clean build.
env.driveMain();
env.flushWrittenFileTracking();
// Invalidate ACmp's template.
env.write('a.html', 'Changed template');
// Update the file to have an error, simultaneously.
writeRandomFile(env, 'other.ts', {error: true});
// This build should fail.
const diags = env.driveDiagnostics();
expect(diags.length).not.toBe(0);
expectToHaveWritten([]);
// Fix the error.
writeRandomFile(env, 'other.ts');
// Rebuild.
env.flushWrittenFileTracking();
env.driveMain();
// If the compiler behaves correctly, it should remember that 'a.html' was updated before,
// and should regenerate a.js. Because the compiler knows a.html is a _resource_ dependency
// of a.ts, it should only regenerate a.js and not its module and dependent components (as
// it would if a.ts were itself changed like in the test above).
expectToHaveWritten([
// Because it directly changed.
'/other.js',
// Because a.html changed
'/a.js',
// b.js and module.js should not be re-emitted, because specifically when tracking
// resource dependencies, the compiler knows that a change to a resource file only affects
// the direct emit of dependent file.
]);
});
});
});
});
/**
* Two components, ACmp and BCmp, where BCmp depends on ACmp.
*
* ACmp has its template in a separate file.
*/
export function writeTwoComponentSystem(env: NgtscTestEnvironment): void {
env.write('a.html', 'This is the template for CmpA');
env.write('a.ts', `
import {Component} from '@angular/core';
@Component({selector: 'a-cmp', templateUrl: './a.html'})
export class ACmp {}
`);
env.write('b.ts', `
import {Component} from '@angular/core';
@Component({selector: 'b-cmp', template: '<a-cmp></a-cmp>'})
export class BCmp {}
`);
env.write('module.ts', `
import {NgModule} from '@angular/core';
import {ACmp} from './a';
import {BCmp} from './b';
@NgModule({
declarations: [ACmp, BCmp],
})
export class Module {}
`);
}
export function writeRandomFile(
env: NgtscTestEnvironment, name: string, options: {error?: true} = {}): void {
env.write(name, `
// If options.error is set, this class has missing braces.
export class Other ${options.error !== true ? '{}' : ''}
`);
}