209 lines
8.2 KiB
TypeScript
Raw Normal View History

/**
* @license
* Copyright Google Inc. All Rights Reserved.
*
* Use of this source code is governed by an MIT-style license that can be
* found in the LICENSE file at https://angular.io/license
*/
import * as ts from 'typescript';
refactor(ivy): implement a virtual file-system layer in ngtsc + ngcc (#30921) To improve cross platform support, all file access (and path manipulation) is now done through a well known interface (`FileSystem`). For testing a number of `MockFileSystem` implementations are provided. These provide an in-memory file-system which emulates operating systems like OS/X, Unix and Windows. The current file system is always available via the static method, `FileSystem.getFileSystem()`. This is also used by a number of static methods on `AbsoluteFsPath` and `PathSegment`, to avoid having to pass `FileSystem` objects around all the time. The result of this is that one must be careful to ensure that the file-system has been initialized before using any of these static methods. To prevent this happening accidentally the current file system always starts out as an instance of `InvalidFileSystem`, which will throw an error if any of its methods are called. You can set the current file-system by calling `FileSystem.setFileSystem()`. During testing you can call the helper function `initMockFileSystem(os)` which takes a string name of the OS to emulate, and will also monkey-patch aspects of the TypeScript library to ensure that TS is also using the current file-system. Finally there is the `NgtscCompilerHost` to be used for any TypeScript compilation, which uses a given file-system. All tests that interact with the file-system should be tested against each of the mock file-systems. A series of helpers have been provided to support such tests: * `runInEachFileSystem()` - wrap your tests in this helper to run all the wrapped tests in each of the mock file-systems. * `addTestFilesToFileSystem()` - use this to add files and their contents to the mock file system for testing. * `loadTestFilesFromDisk()` - use this to load a mirror image of files on disk into the in-memory mock file-system. * `loadFakeCore()` - use this to load a fake version of `@angular/core` into the mock file-system. All ngcc and ngtsc source and tests now use this virtual file-system setup. PR Close #30921
2019-06-06 20:22:32 +01:00
import {absoluteFrom} from '../../../src/ngtsc/file_system';
import {Declaration, Import} from '../../../src/ngtsc/reflection';
import {Logger} from '../logging/logger';
import {BundleProgram} from '../packages/bundle_program';
fix(ngcc): correctly detect emitted TS helpers in ES5 (#35191) In ES5 code, TypeScript requires certain helpers (such as `__spreadArrays()`) to be able to support ES2015+ features. These helpers can be either imported from `tslib` (by setting the `importHelpers` TS compiler option to `true`) or emitted inline (by setting the `importHelpers` and `noEmitHelpers` TS compiler options to `false`, which is the default value for both). Ngtsc's `StaticInterpreter` (which is also used during ngcc processing) is able to statically evaluate some of these helpers (currently `__assign()`, `__spread()` and `__spreadArrays()`), as long as `ReflectionHost#getDefinitionOfFunction()` correctly detects the declaration of the helper. For this to happen, the left-hand side of the corresponding call expression (i.e. `__spread(...)` or `tslib.__spread(...)`) must be evaluated as a function declaration for `getDefinitionOfFunction()` to be called with. In the case of imported helpers, the `tslib.__someHelper` expression was resolved to a function declaration of the form `export declare function __someHelper(...args: any[][]): any[];`, which allows `getDefinitionOfFunction()` to correctly map it to a TS helper. In contrast, in the case of emitted helpers (and regardless of the module format: `CommonJS`, `ESNext`, `UMD`, etc.)), the `__someHelper` identifier was resolved to a variable declaration of the form `var __someHelper = (this && this.__someHelper) || function () { ... }`, which upon further evaluation was categorized as a `DynamicValue` (prohibiting further evaluation by the `getDefinitionOfFunction()`). As a result of the above, emitted TypeScript helpers were not evaluated in ES5 code. --- This commit changes the detection of TS helpers to leverage the existing `KnownFn` feature (previously only used for built-in functions). `Esm5ReflectionHost` is changed to always return `KnownDeclaration`s for TS helpers, both imported (`getExportsOfModule()`) as well as emitted (`getDeclarationOfIdentifier()`). Similar changes are made to `CommonJsReflectionHost` and `UmdReflectionHost`. The `KnownDeclaration`s are then mapped to `KnownFn`s in `StaticInterpreter`, allowing it to statically evaluate call expressions involving any kind of TS helpers. Jira issue: https://angular-team.atlassian.net/browse/FW-1689 PR Close #35191
2020-02-06 18:44:49 +02:00
import {FactoryMap, getTsHelperFnFromIdentifier, isDefined, stripExtension} from '../utils';
import {ExportDeclaration, ExportStatement, findNamespaceOfIdentifier, findRequireCallReference, isExportStatement, isRequireCall, isWildcardReexportStatement, RequireCall, WildcardReexportStatement} from './commonjs_umd_utils';
import {Esm5ReflectionHost} from './esm5_host';
import {NgccClassSymbol} from './ngcc_host';
export class CommonJsReflectionHost extends Esm5ReflectionHost {
protected commonJsExports = new FactoryMap<ts.SourceFile, Map<string, Declaration>|null>(
sf => this.computeExportsOfCommonJsModule(sf));
protected topLevelHelperCalls =
new FactoryMap<string, FactoryMap<ts.SourceFile, ts.CallExpression[]>>(
helperName => new FactoryMap<ts.SourceFile, ts.CallExpression[]>(
sf => sf.statements.map(stmt => this.getHelperCall(stmt, [helperName]))
.filter(isDefined)));
protected program: ts.Program;
protected compilerHost: ts.CompilerHost;
constructor(logger: Logger, isCore: boolean, src: BundleProgram, dts: BundleProgram|null = null) {
super(logger, isCore, src, dts);
this.program = src.program;
this.compilerHost = src.host;
}
getImportOfIdentifier(id: ts.Identifier): Import|null {
const requireCall = this.findCommonJsImport(id);
if (requireCall === null) {
return null;
}
return {from: requireCall.arguments[0].text, name: id.text};
}
getDeclarationOfIdentifier(id: ts.Identifier): Declaration|null {
fix(ngcc): consistently delegate to TypeScript host for typing files (#36089) When ngcc is compiling an entry-point, it uses a `ReflectionHost` that is specific to its format, e.g. ES2015, ES5, UMD or CommonJS. During the compilation of that entry-point however, the reflector may be used to reflect into external libraries using their declaration files. Up until now this was achieved by letting all `ReflectionHost` classes consider their parent class for reflector queries, thereby ending up in the `TypeScriptReflectionHost` that is a common base class for all reflector hosts. This approach has proven to be prone to bugs, as failing to call into the base class would cause incompatibilities with reading from declaration files. The observation can be made that there's only two distinct kinds of reflection host queries: 1. the reflector query is about code that is part of the entry-point that is being compiled, or 2. the reflector query is for an external library that the entry-point depends on, in which case the information is reflected from the declaration files. The `ReflectionHost` that was chosen for the entry-point should serve only reflector queries for the first case, whereas a regular `TypeScriptReflectionHost` should be used for the second case. This avoids the problem where a format-specific `ReflectionHost` fails to handle the second case correctly, as it isn't even considered for such reflector queries. This commit introduces a `ReflectionHost` that delegates to the `TypeScriptReflectionHost` for AST nodes within declaration files, otherwise delegating to the format-specific `ReflectionHost`. Fixes #35078 Resolves FW-1859 PR Close #36089
2020-02-04 22:15:06 +01:00
return this.getCommonJsImportedDeclaration(id) || super.getDeclarationOfIdentifier(id);
}
getExportsOfModule(module: ts.Node): Map<string, Declaration>|null {
return super.getExportsOfModule(module) || this.commonJsExports.get(module.getSourceFile());
}
/**
* Search statements related to the given class for calls to the specified helper.
*
* In CommonJS these helper calls can be outside the class's IIFE at the top level of the
* source file. Searching the top level statements for helpers can be expensive, so we
* try to get helpers from the IIFE first and only fall back on searching the top level if
* no helpers are found.
*
* @param classSymbol the class whose helper calls we are interested in.
* @param helperNames the names of the helpers (e.g. `__decorate`) whose calls we are interested
* in.
* @returns an array of nodes of calls to the helper with the given name.
*/
protected getHelperCallsForClass(classSymbol: NgccClassSymbol, helperNames: string[]):
ts.CallExpression[] {
const esm5HelperCalls = super.getHelperCallsForClass(classSymbol, helperNames);
if (esm5HelperCalls.length > 0) {
return esm5HelperCalls;
} else {
fix(ngcc): consistently use outer declaration for classes (#32539) In ngcc's reflection hosts for compiled JS bundles, such as ESM2015, special care needs to be taken for classes as there may be an outer declaration (referred to as "declaration") and an inner declaration (referred to as "implementation") for a given class. Therefore, there will also be two `ts.Symbol`s bound per class, and ngcc needs to switch between those declarations and symbols depending on where certain information can be found. Prior to this commit, the `NgccReflectionHost` interface had methods `getClassSymbol` and `findClassSymbols` that would return a `ts.Symbol`. These class symbols would be used to kick off compilation of components using ngtsc, so it is important for these symbols to correspond with the publicly visible outer declaration of the class. However, the ESM2015 reflection host used to return the `ts.Symbol` for the inner declaration, if the class was declared as follows: ```javascript var MyClass = class MyClass {}; ``` For the above code, `Esm2015ReflectionHost.getClassSymbol` would return the `ts.Symbol` corresponding with the `class MyClass {}` declaration, whereas it should have corresponded with the `var MyClass` declaration. As a consequence, no `NgModule` could be resolved for the component, so no components/directives would be in scope for the component. This resulted in errors during runtime. This commit resolves the issue by introducing a `NgccClassSymbol` that contains references to both the outer and inner `ts.Symbol`, instead of just a single `ts.Symbol`. This avoids the unclarity of whether a `ts.Symbol` corresponds with the outer or inner declaration. More details can be found here: https://hackmd.io/7nkgWOFWQlSRAuIW_8KPPw Fixes #32078 Closes FW-1507 PR Close #32539
2019-09-03 21:26:58 +02:00
const sourceFile = classSymbol.declaration.valueDeclaration.getSourceFile();
return this.getTopLevelHelperCalls(sourceFile, helperNames);
}
}
/**
* Find all the helper calls at the top level of a source file.
*
* We cache the helper calls per source file so that we don't have to keep parsing the code for
* each class in a file.
*
* @param sourceFile the source who may contain helper calls.
* @param helperNames the names of the helpers (e.g. `__decorate`) whose calls we are interested
* in.
* @returns an array of nodes of calls to the helper with the given name.
*/
private getTopLevelHelperCalls(sourceFile: ts.SourceFile, helperNames: string[]):
ts.CallExpression[] {
const calls: ts.CallExpression[] = [];
helperNames.forEach(helperName => {
const helperCallsMap = this.topLevelHelperCalls.get(helperName);
calls.push(...helperCallsMap.get(sourceFile));
});
return calls;
}
private computeExportsOfCommonJsModule(sourceFile: ts.SourceFile): Map<string, Declaration> {
const moduleMap = new Map<string, Declaration>();
for (const statement of this.getModuleStatements(sourceFile)) {
if (isExportStatement(statement)) {
const exportDeclaration = this.extractCommonJsExportDeclaration(statement);
fix(ivy): in ngcc, handle inline exports in commonjs code (#32129) One of the compiler's tasks is to enumerate the exports of a given ES module. This can happen for example to resolve `foo.bar` where `foo` is a namespace import: ```typescript import * as foo from './foo'; @NgModule({ directives: [foo.DIRECTIVES], }) ``` In this case, the compiler must enumerate the exports of `foo.ts` in order to evaluate the expression `foo.DIRECTIVES`. When this operation occurs under ngcc, it must deal with the different module formats and types of exports that occur. In commonjs code, a problem arises when certain exports are downleveled. ```typescript export const DIRECTIVES = [ FooDir, BarDir, ]; ``` can be downleveled to: ```javascript exports.DIRECTIVES = [ FooDir, BarDir, ``` Previously, ngtsc and ngcc expected that any export would have an associated `ts.Declaration` node. `export class`, `export function`, etc. all retain `ts.Declaration`s even when downleveled. But the `export const` construct above does not. Therefore, ngcc would not detect `DIRECTIVES` as an export of `foo.ts`, and the evaluation of `foo.DIRECTIVES` would therefore fail. To solve this problem, the core concept of an exported `Declaration` according to the `ReflectionHost` API is split into a `ConcreteDeclaration` which has a `ts.Declaration`, and an `InlineDeclaration` which instead has a `ts.Expression`. Differentiating between these allows ngcc to return an `InlineDeclaration` for `DIRECTIVES` and correctly keep track of this export. PR Close #32129
2019-08-13 16:08:53 -07:00
moduleMap.set(exportDeclaration.name, exportDeclaration.declaration);
} else if (isWildcardReexportStatement(statement)) {
const reexports = this.extractCommonJsWildcardReexports(statement, sourceFile);
for (const reexport of reexports) {
moduleMap.set(reexport.name, reexport.declaration);
}
}
}
return moduleMap;
}
private extractCommonJsExportDeclaration(statement: ExportStatement): ExportDeclaration {
const exportExpression = statement.expression.right;
const declaration = this.getDeclarationOfExpression(exportExpression);
const name = statement.expression.left.name.text;
fix(ivy): in ngcc, handle inline exports in commonjs code (#32129) One of the compiler's tasks is to enumerate the exports of a given ES module. This can happen for example to resolve `foo.bar` where `foo` is a namespace import: ```typescript import * as foo from './foo'; @NgModule({ directives: [foo.DIRECTIVES], }) ``` In this case, the compiler must enumerate the exports of `foo.ts` in order to evaluate the expression `foo.DIRECTIVES`. When this operation occurs under ngcc, it must deal with the different module formats and types of exports that occur. In commonjs code, a problem arises when certain exports are downleveled. ```typescript export const DIRECTIVES = [ FooDir, BarDir, ]; ``` can be downleveled to: ```javascript exports.DIRECTIVES = [ FooDir, BarDir, ``` Previously, ngtsc and ngcc expected that any export would have an associated `ts.Declaration` node. `export class`, `export function`, etc. all retain `ts.Declaration`s even when downleveled. But the `export const` construct above does not. Therefore, ngcc would not detect `DIRECTIVES` as an export of `foo.ts`, and the evaluation of `foo.DIRECTIVES` would therefore fail. To solve this problem, the core concept of an exported `Declaration` according to the `ReflectionHost` API is split into a `ConcreteDeclaration` which has a `ts.Declaration`, and an `InlineDeclaration` which instead has a `ts.Expression`. Differentiating between these allows ngcc to return an `InlineDeclaration` for `DIRECTIVES` and correctly keep track of this export. PR Close #32129
2019-08-13 16:08:53 -07:00
if (declaration !== null) {
return {name, declaration};
} else {
return {
name,
declaration: {
node: null,
known: null,
fix(ivy): in ngcc, handle inline exports in commonjs code (#32129) One of the compiler's tasks is to enumerate the exports of a given ES module. This can happen for example to resolve `foo.bar` where `foo` is a namespace import: ```typescript import * as foo from './foo'; @NgModule({ directives: [foo.DIRECTIVES], }) ``` In this case, the compiler must enumerate the exports of `foo.ts` in order to evaluate the expression `foo.DIRECTIVES`. When this operation occurs under ngcc, it must deal with the different module formats and types of exports that occur. In commonjs code, a problem arises when certain exports are downleveled. ```typescript export const DIRECTIVES = [ FooDir, BarDir, ]; ``` can be downleveled to: ```javascript exports.DIRECTIVES = [ FooDir, BarDir, ``` Previously, ngtsc and ngcc expected that any export would have an associated `ts.Declaration` node. `export class`, `export function`, etc. all retain `ts.Declaration`s even when downleveled. But the `export const` construct above does not. Therefore, ngcc would not detect `DIRECTIVES` as an export of `foo.ts`, and the evaluation of `foo.DIRECTIVES` would therefore fail. To solve this problem, the core concept of an exported `Declaration` according to the `ReflectionHost` API is split into a `ConcreteDeclaration` which has a `ts.Declaration`, and an `InlineDeclaration` which instead has a `ts.Expression`. Differentiating between these allows ngcc to return an `InlineDeclaration` for `DIRECTIVES` and correctly keep track of this export. PR Close #32129
2019-08-13 16:08:53 -07:00
expression: exportExpression,
viaModule: null,
},
};
}
}
private extractCommonJsWildcardReexports(
statement: WildcardReexportStatement, containingFile: ts.SourceFile): ExportDeclaration[] {
const reexportArg = statement.expression.arguments[0];
const requireCall = isRequireCall(reexportArg) ?
reexportArg :
ts.isIdentifier(reexportArg) ? findRequireCallReference(reexportArg, this.checker) : null;
if (requireCall === null) {
return [];
}
const importPath = requireCall.arguments[0].text;
const importedFile = this.resolveModuleName(importPath, containingFile);
if (importedFile === undefined) {
return [];
}
const importedExports = this.getExportsOfModule(importedFile);
if (importedExports === null) {
return [];
}
const viaModule = stripExtension(importedFile.fileName);
const reexports: ExportDeclaration[] = [];
importedExports.forEach((decl, name) => {
if (decl.node !== null) {
reexports.push({
name,
declaration: {node: decl.node, known: null, viaModule, identity: decl.identity}
});
} else {
reexports.push(
{name, declaration: {node: null, known: null, expression: decl.expression, viaModule}});
}
});
return reexports;
}
private findCommonJsImport(id: ts.Identifier): RequireCall|null {
// Is `id` a namespaced property access, e.g. `Directive` in `core.Directive`?
// If so capture the symbol of the namespace, e.g. `core`.
const nsIdentifier = findNamespaceOfIdentifier(id);
return nsIdentifier && findRequireCallReference(nsIdentifier, this.checker);
}
private getCommonJsImportedDeclaration(id: ts.Identifier): Declaration|null {
const importInfo = this.getImportOfIdentifier(id);
if (importInfo === null) {
return null;
}
const importedFile = this.resolveModuleName(importInfo.from, id.getSourceFile());
if (importedFile === undefined) {
return null;
}
const viaModule = !importInfo.from.startsWith('.') ? importInfo.from : null;
return {node: importedFile, known: getTsHelperFnFromIdentifier(id), viaModule, identity: null};
}
private resolveModuleName(moduleName: string, containingFile: ts.SourceFile): ts.SourceFile
|undefined {
if (this.compilerHost.resolveModuleNames) {
const moduleInfo = this.compilerHost.resolveModuleNames(
[moduleName], containingFile.fileName, undefined, undefined,
this.program.getCompilerOptions())[0];
refactor(ivy): implement a virtual file-system layer in ngtsc + ngcc (#30921) To improve cross platform support, all file access (and path manipulation) is now done through a well known interface (`FileSystem`). For testing a number of `MockFileSystem` implementations are provided. These provide an in-memory file-system which emulates operating systems like OS/X, Unix and Windows. The current file system is always available via the static method, `FileSystem.getFileSystem()`. This is also used by a number of static methods on `AbsoluteFsPath` and `PathSegment`, to avoid having to pass `FileSystem` objects around all the time. The result of this is that one must be careful to ensure that the file-system has been initialized before using any of these static methods. To prevent this happening accidentally the current file system always starts out as an instance of `InvalidFileSystem`, which will throw an error if any of its methods are called. You can set the current file-system by calling `FileSystem.setFileSystem()`. During testing you can call the helper function `initMockFileSystem(os)` which takes a string name of the OS to emulate, and will also monkey-patch aspects of the TypeScript library to ensure that TS is also using the current file-system. Finally there is the `NgtscCompilerHost` to be used for any TypeScript compilation, which uses a given file-system. All tests that interact with the file-system should be tested against each of the mock file-systems. A series of helpers have been provided to support such tests: * `runInEachFileSystem()` - wrap your tests in this helper to run all the wrapped tests in each of the mock file-systems. * `addTestFilesToFileSystem()` - use this to add files and their contents to the mock file system for testing. * `loadTestFilesFromDisk()` - use this to load a mirror image of files on disk into the in-memory mock file-system. * `loadFakeCore()` - use this to load a fake version of `@angular/core` into the mock file-system. All ngcc and ngtsc source and tests now use this virtual file-system setup. PR Close #30921
2019-06-06 20:22:32 +01:00
return moduleInfo && this.program.getSourceFile(absoluteFrom(moduleInfo.resolvedFileName));
} else {
const moduleInfo = ts.resolveModuleName(
moduleName, containingFile.fileName, this.program.getCompilerOptions(),
this.compilerHost);
return moduleInfo.resolvedModule &&
refactor(ivy): implement a virtual file-system layer in ngtsc + ngcc (#30921) To improve cross platform support, all file access (and path manipulation) is now done through a well known interface (`FileSystem`). For testing a number of `MockFileSystem` implementations are provided. These provide an in-memory file-system which emulates operating systems like OS/X, Unix and Windows. The current file system is always available via the static method, `FileSystem.getFileSystem()`. This is also used by a number of static methods on `AbsoluteFsPath` and `PathSegment`, to avoid having to pass `FileSystem` objects around all the time. The result of this is that one must be careful to ensure that the file-system has been initialized before using any of these static methods. To prevent this happening accidentally the current file system always starts out as an instance of `InvalidFileSystem`, which will throw an error if any of its methods are called. You can set the current file-system by calling `FileSystem.setFileSystem()`. During testing you can call the helper function `initMockFileSystem(os)` which takes a string name of the OS to emulate, and will also monkey-patch aspects of the TypeScript library to ensure that TS is also using the current file-system. Finally there is the `NgtscCompilerHost` to be used for any TypeScript compilation, which uses a given file-system. All tests that interact with the file-system should be tested against each of the mock file-systems. A series of helpers have been provided to support such tests: * `runInEachFileSystem()` - wrap your tests in this helper to run all the wrapped tests in each of the mock file-systems. * `addTestFilesToFileSystem()` - use this to add files and their contents to the mock file system for testing. * `loadTestFilesFromDisk()` - use this to load a mirror image of files on disk into the in-memory mock file-system. * `loadFakeCore()` - use this to load a fake version of `@angular/core` into the mock file-system. All ngcc and ngtsc source and tests now use this virtual file-system setup. PR Close #30921
2019-06-06 20:22:32 +01:00
this.program.getSourceFile(absoluteFrom(moduleInfo.resolvedModule.resolvedFileName));
}
}
}