diff --git a/packages/compiler/design/separate_compilation.md b/packages/compiler/design/separate_compilation.md new file mode 100644 index 0000000000..29e81694a4 --- /dev/null +++ b/packages/compiler/design/separate_compilation.md @@ -0,0 +1,835 @@ +# DESIGN DOC (Ivy): Separate Compilation + +AUTHOR: chuckj@ + +## Background + +### Angular 5 (Renderer2) + +In 5.0 and prior versions of Angular the compiler performs whole program +analysis and generates template and injector definitions that are using +this global knowledge to flatten injector scope definitions, inline +directives into the component, pre-calculate queries, pre-calculate +content projection, etc. This global knowledge requires that module and +component factories are generated as a final global step when compiling +a module. If any of the transitive information changed then all factories +need to be regenerated. + +Separate component and module compilation is supported only at the module +definition level and only from source. That is, npm packages must contain +the metadata necessary to generate the factories, they cannot contain, +themselves, the generated factories. This is because, if any of there +dependencies change, their factories would be invalid preventing them from +using version ranges in their dependencies. To support producing factories +from compiled source (already translated by TypeScript into JavaScript) +libraries include metadata that describe the content of the the Angular +decorators. + +This document refers to this style of code generation as Renderer2 after the +name of the renderer class it uses at runtime. + +### Angular Ivy + +In Ivy, the runtime is crafted in a way that allows for separate compilation +by performing at runtime much of what was previously pre-calculated by +compiler. This allows the definition of components to change without +requiring modules and components that depend on them being recompiled. + +The mental model of Ivy is that the decorator is the compiler. That is +the decorator can be thought of as parameters to a class transformer that +transforms the class by generating definitions based on the decorator +parameters. An `@Component` decorator transforms the class by adding +an `ngComponentDef` static property, `@Directive` adds `ngDirectiveDef`, +`@Pipe` adds `ngPipeDef`, etc. In most cases values supplied to the +decorator is sufficient to generate the definition. However, in the case of +interpreting the template, the compiler needs to know the selector defined for +each component, directive and pipe that are in scope of the template. The +purpose of this document is to define the information is needed by the +compiler and how that information is is serialized to be discovered and +used by subsequent calls to `ngc`. + +This document refers to this style of code generation as ivy after the code +name of the project to create it. It would be more consistent to refer to it +as Renderer3 that looks too similar to Renderer2. + +## Information needed + +The information available across compilations in Angular 5 is represented in +the compiler by a summary description. For example, components and directives +are represented by the [`CompileDirectiveSummary`](https://github.com/angular/angular/blob/d3827a0017fd5ff5ac0f6de8a19692ce47bf91b4/packages/compiler/src/compile_metadata.ts#L257). +The following table shows where this information ends up in an ivy compiled +class: + +### `CompileDirectiveSummary` + +| field | destination | +|---------------------|-----------------------| +| `type` | implicit | +| `isComponent` | `ngComponentDef` | +| `selector` | `ngModuleScope` | +| `exportAs` | `ngDirectiveDef` | +| `inputs` | `ngDirectiveDef` | +| `outputs` | `ngDirectiveDef` | +| `hostListeners` | `ngDirectiveDef` | +| `hostProperties` | `ngDirectiveDef` | +| `hostAttributes` | `ngDirectiveDef` | +| `providers` | `ngInjectorDef` | +| `viewProviders` | `ngComponentDef` | +| `queries` | `ngDirectiveDef` | +| `guards` | not used | +| `viewQueries` | `ngComponentDef` | +| `entryComponents` | not used | +| `changeDetection` | `ngComponentDef` | +| `template` | `ngComponentDef` | +| `componentViewType` | not used | +| `renderType` | not used | +| `componentFactory` | not used | + +Only one definition is generated per class. All components are directives so a +`ngComponentDef` contains all the `ngDirectiveDef` information. All directives +are injectable so `ngComponentDef` and `ngDirectiveDef` contain `ngInjectableDef` +information. + +For `CompilePipeSummary` the table looks like: + +#### `CompilePipeSummary` + +| field | destination | +|---------------------|-----------------------| +| `type` | implicit | +| `name` | `ngModuleScope` | +| `pure` | `ngPipeDef` | + +The only pieces of information that are not generated into the definition are +the directive selector and the pipe name as they go into the module scope. + +The information needed to build an `ngModuleScope` needs to be communicated +from the directive and pipe to the module that declares them. + +## Metadata + +### Angular 5 + +Angular 5 uses `.metadata.json` files to store information that is directly +inferred from the `.ts` files and include value information that is not +included in the `.d.ts` file produced by TypeScript. Because only exports for +types are included in `.d.ts` files and might not include the exports necessary +for values, the metadata includes export clauses from the `.ts` file. + +When a module is flattened into a FESM (Flat ECMAScript Module), a flat +metadata file is also produced which is the metadata for all symbols exported +from the module index. The metadata represents what the `.metadata.json` file +would look like if all the symbols were declared in the index instead of +reexported from the index. + +### Angular Ivy + +The metadata for a class in ivy is transformed to be what the metadata of the +transformed .js file produced by the ivy compiler would be. For example, a +component's `@Component` is removed by the compiler and replaced by a `ngComponentDef`. +The `.metadata.json` file is similarly transformed but the content of the +value assigned is elided (e.g. `"ngComponentDef": {}`). The compiler doesn't +record the selector declared for a component but it is needed to produce the +`ngModuleScope` so the information is recorded as if a static field +`ngSelector` was declared on class with the value of the `selector` field +from the `@Component` or `@Directive` decorator. + +The following transformations are performed: + +#### `@Component` + +The metadata for a component is transformed by: + +1. Removing the `@Component` directive. +2. Add `"ngComponentDef": {}` static field. +3. Add `"ngSelector": ` static field. + +##### Example + +*my.component.ts* +```ts +@Component({ + selector: 'my-comp', + template: `

Hello, {{name}}!

` +}) +export class MyComponent { + @Input() name: string; +} +``` + +*my.component.js* +```js +export class MyComponent { + name: string; + static ngComponentDef = defineComponent({...}); +} +``` + +*my.component.metadata.json* +```json +{ + "__symbolic": "module", + "version": 4, + "metadata": { + "MyComponent": { + "__symbolic": "class", + "statics": { + "ngComponentDef": {}, + "ngSelector": "my-comp" + } + } + } +} +``` + +Note that this is exactly what is produced if the transform had been done +manually or by some other compiler before `ngc` compiler is invoked. That is +this model has the advantage that there is no magic introduced by the compiler +as it treats classes annotated by `@Component` identically to those produced +manually. + +#### `@Directive` + +The metadata for a directive is transformed by: + +1. Removing the `@Directive` directive. +2. Add `"ngDirectiveDef": {}` static field. +3. Add `"ngSelector": ` static field. + +##### example + +*my.directive.ts* + +```ts +@Directive({selector: '[my-dir]'}) +export class MyDirective { + @HostBinding('id') dirId = 'some id'; +} +``` + +*my.directive.js* +```js +export class MyDirective { + constructor() { + this.dirId = 'some id'; + } + static ngDirectiveDef = defineDirective({...}); +} +``` + +*my.directive.metadata.json* +```json +{ + "__symbolic": "module", + "version": 4, + "metadata": { + "MyDirective": { + "__symbolic": "class", + "statics": { + "ngDirectiveDef": {}, + "ngSelector": "[my-dir]" + } + } + } +} +``` + +#### `@Pipe` + +The metadata for a pipe is transformed by: + +1. Removing the `@Pipe` directive. +2. Add `"ngPipeDef": {}` static field. +3. Add `"ngSelector": ` static field. + +##### example + +*my.pipe.ts* +```ts +@Pipe({name: 'myPipe'}) +export class MyPipe implements PipeTransform { + transform(...) ... +} +``` + +*my.pipe.js* +```js +export class MyPipe { + transform(...) ... + static ngPipeDef = definePipe({...}); +} +``` + +*my.pipe.metadata.json* +```json +{ + "__symbolic": "module", + "version": 4, + "metadata": { + "MyPipe": { + "__symbolic": "class", + "statics": { + "ngPipeDef": {}, + "ngSelector": "myPipe" + } + } + } +} +``` + +#### `@NgModule` + +The metadata for a module is transformed by: + +1. Remove the `@NgModule` directive. +2. Add `"ngInjectorDef": {}` static field. +3. Add `"ngModuleScope": ` static field. + +The scope value is an array the following type: + +```ts +export type ModuleScope = ModuleScopeEntry[]; + +export interface ModuleDirectiveEntry { + type: Type; + selector: string; +} + +export interface ModulePipeEntry { + type: Type; + name: string; + isPipe: true; +} + +export interface ModuleExportEntry { + type: Type; + isModule: true; +} + +type ModuleScopeEntry = ModuleDirectiveEntry | ModulePipeEntry | ModuleExportEntry; +``` + +where the `type` values are generated as references. + +##### example + +*my.module.ts* +```ts +@NgModule({ + imports: [CommonModule, UtilityModule], + declarations: [MyComponent, MyDirective, MyComponent], + exports: [MyComponent, MyDirective, MyPipe, UtilityModule], + providers: [{ + provide: Service, useClass: ServiceImpl + }] +}) +export class MyModule {} +``` + +*my.module.js* +```js +export class MyModule { + static ngInjectorDef = defineInjector(...); +} +``` + +*my.module.metadata.json* +```json +{ + "__symbolic": "module", + "version": 4, + "metadata": { + "MyModule": { + "__symbolic": "class", + "statics": { + "ngInjectorDef": {}, + "ngModuleScope": [ + { + "type": { + "__symbolic": "reference", + "module": "./my.component", + "name": "MyComponent" + }, + "selector": "my-comp" + }, + { + "type": { + "__symbolic": "reference", + "module": "./my.directive", + "name": "MyDirective" + }, + "selector": "[my-dir]" + }, + { + "type": { + "__symbolic": "reference", + "module": "./my.pipe", + "name": "MyPipe" + }, + "name": "myPipe", + "isPipe": true + }, + { + "type": { + "__symbolic": "reference", + "module": "./utility.module", + "name": "UtilityModule" + }, + "isModule": true + } + ] + } + } + } +} +``` + +Note that this is identical to what would have been generated if the this was +manually written as: + +```ts +export class MyModule { + static ngInjectorDef = defineInjector({ + providers: [{ + provide: Service, useClass: ServiceImpl + }], + imports: [CommonModule, UtilityModule] + }); + static ngModuleScope = [{ + type: MyComponent, + selector: 'my-comp' + }, { + type: MyDirective, + selector: '[my-dir]' + }, { + type: MyPipe, + name: 'myPipe' + }, { + type: UtilityModule, + isModule: true + }]; +} +``` + +except for the call to `defineInjector` would generate a `{ __symbolic: 'error' }` +value which is ignored by the ivy compiler. This allows the system to ignore +the difference between manually and mechanically created module definitions. + + +## Manual Considerations + +With this proposal, the compiler treats manually and mechanically generated +Angular definitions identically. This allows flexibility not only in the future +for how the declarations are mechanically produced it also allows alternative +mechanism to generate declarations be easily explored without altering the +compiler or dependent tool chain. It also allows third-party code generators +with possibly different component syntaxes to generate a component fully +understood by the compiler. + +Unfortunately, however, manually generated modules contain references to +classes that might not be necessary at runtime. Manually or third-party +components can get the same payload properties of an Angular generated +component by annotating the `ngSelector` and `ngModuleScope` properties with +`// @__BUILD_OPTIMIZER_REMOVE_` comment which will cause the build optimizer +to remove the declaration. + +##### example + +For example the above manually created module would have better payload +properties by including a `// @__BUILD_OPTIMIZER_REMOVE_` comment: + +```ts +export class MyModule { + static ngInjectorDef = defineInjector({ + providers: [{ + provide: Service, useClass: ServiceImpl + }], + imports: [CommonModule, UtilityModule] + }); + + // @__BUILD_OPTIMIZER_REMOVE_ + static ngModuleScope = [{ + type: MyComponent, + selector: 'my-comp' + }, { + type: MyDirective, + selector: '[my-dir]' + }, { + type: MyPipe, + name: 'myPipe' + }, { + type: UtilityModule, + isModule: true + }]; +} +``` + +## `ngc` output (non-Bazel) + +The cases that `ngc` handle are producing an application and producing a +reusable library used in an application. + +### Application output + +The output of the ivy compiler only optionally generates the factories +generated by the Renderer2 style output of Angular 5.0. In ivy, the information +that was generated in factories is now generated in Angular as a definition +that is generated as a static field on the Angular decorated class. + +Renderer2 requires that, when building the final application, all factories for +all libraries also be generated. In ivy, the definitions are generated when +the library is compiled. + +The ivy compile can adapt Renderer2 target libraries by generating the factories +for them and back-patching, at runtime, the static property into the class. + +#### Back-patching module (`"renderer2BackPatching"`) + +When an application contains Renderer2 target libraries the ivy definitions +need to be back-patch onto the component, directive, module, pipe, and +injectable classes. + +If the Angular compiler option `"renderer2BackPatching"` is enabled, the +compiler will generate an `angular.back-patch` module in the to root output +directory of the project. If `"generateRenderer2Factories"` is set to `true` +then the default value for `"renderer2BackPatching"` is `true` and it is and +error for it to be `false`. `"renderer2BackPatching"` is ignored if `"enableIvy"` +is `false`. + +`angular.back-patch` exports a function per `@NgModule` for the entire +application, including previously compiled libraries. The name of the function +is determined by name of the imported module with all non alphanumeric +character, including '`/`' and '`.`', replaced by '`_`'. + +The back-patch functions will call the back-patch function of any module they +import. This means that only the application's module and lazy loaded modules +back-patching functions needs to be called. If using the Renderer2 module factory +instances, this is performed automatically when the first application module +instance is created. + +#### Renderer2 Factories (`"generateRenderer2Factories"`) + +`ngc` can generate an implementation of `NgModuleFactory` in the same location +that Angular 5.0 would generate it. This implementation of `NgModuleFactory` +will back-patch the Renderer2 style classes when the first module instance is +created by calling the correct back-patching function generated in the`angular.back-patch` +module. + +Renderer2 style factories are created when the `"generateRenderer2Factories"` +Angular compiler option is `true`. Setting `"generateRenderer2Factories"` implies +`"renderer2BackPatching"` is also `true` and it is an error to explicitly set it +to `false`. `"generateRenderer2Factories"` is ignored if `"enableIvy"` is +`false`. + +When this option is `true` a factory module is created with the same public API +at the same location as Angular 5.0 whenever Angular 5.0 would have generated a +factory. + +### Recommended options + +The recommended options for producing a ivy application are + +| option | value | | +|--------------------------------|----------|-------------| +| `"enableIvy"` | `true` | required | +| `"generateRenderer2Factories"` | `true` | implied | +| `"renderer2BackPatching"` | `true` | implied | +| `"generateCodeForLibraries"` | `true` | default | +| `"annotationsAs"` | `remove` | implied | +| `"enableLegacyTemplate"` | `false` | default | +| `"preserveWhitespaces"` | `false` | default | +| `"skipMetadataEmit"` | `true` | default | +| `"strictMetadataEmit"` | `false` | implied | +| `"skipTemplateCodegen"` | | ignored | + +The options marked "implied" are implied by other options having the +recommended value and do not need to be explicitly set. Options marked +"default" also do not need to be set explicitly. + +## Library output + +Building an ivy library with `ngc` differs from Renderer2 in that the +declarations are included in the generated output and should be included in the +package published to `npm`. The `.metadata.json` files still need to be +included but they are transformed as described below. + +### Transforming metadata + +As described above, when the compiler adds the declaration to the class it will +also transform the `.metadata.json` file to reflect the new static fields added +to the class. + +Once the static fields are added to the metadata, the ivy compiler no longer +needs the the information in the decorator. When `"enableIvy"` is `true` this +information is removed from the `.metadata.json` file. + +### Recommended options + +The recommended options for producing a ivy library are: + +| option | value | | +|--------------------------------|----------|-------------| +| `"enableIvy"` | `true` | required | +| `"generateRenderer2Factories"` | `false` | | +| `"renderer2BackPatching"` | `false` | default | +| `"generateCodeForLibraries"` | `false` | | +| `"annotationsAs"` | `remove` | implied | +| `"enableLegacyTemplate"` | `false` | default | +| `"preserveWhitespaces"` | `false` | default | +| `"skipMetadataEmit"` | `false` | | +| `"strictMetadataEmit"` | `true ` | | +| `"skipTemplateCodegen"` | | ignored | + +The options marked "implied" are implied by other options having the +recommended value and do not need to be explicitly set. Options marked +"default" also do not need to be set explicitly. + +## Simplified options + +The default Angular Compiler options default to, mostly, the recommended set of +options but the options necessary to set for specific targets are not clear and +mixing them can produce nonsensical results. The `"target"` option can be used +to simplify the setting of the compiler options to the recommended values +depending on the target: + +| target | option | value | | +|-------------------|--------------------------------|--------------|-------------| +| `"application"` | `"generateRenderer2Factories"` | `true` | enforced | +| | `"renderer2BackPatching"` | `true` | enforced | +| | `"generateCodeForLibraries"` | `true` | | +| | `"annotationsAs"` | `remove` | | +| | `"enableLegacyTemplate"` | `false` | | +| | `"preserveWhitespaces"` | `false` | | +| | `"skipMetadataEmit"` | `false` | | +| | `"strictMetadataEmit"` | `true` | | +| | `"skipTemplateCodegen"` | `false` | | +| | `"fullTemplateTypeCheck"` | `true` | | +| | `"enableLegacyTemplate"` | `false` | | +| | | | | +| `"library"` | `"generateRenderer2Factories"` | `false` | enforced | +| | `"renderer2BackPatching"` | `false` | enforced | +| | `"generateCodeForLibraries"` | `false` | enforced | +| | `"annotationsAs"` | `decorators` | | +| | `"enableLegacyTemplate"` | `false` | | +| | `"preserveWhitespaces"` | `false` | | +| | `"skipMetadataEmit"` | `false` | enforced | +| | `"strictMetadataEmit"` | `true` | | +| | `"skipTemplateCodegen"` | `false` | enforced | +| | `"fullTemplateTypeCheck"` | `true` | | +| | `"enableLegacyTemplate"` | `false` | | +| | | | | +| `"package"` | `"flatModuleOutFile"` | | required | +| | `"flatModuleId"` | | required | +| | `"enableIvy"` | `false` | enforced | +| | `"generateRenderer2Factories"` | `false` | enforced | +| | `"renderer2BackPatching"` | `false` | enforced | +| | `"generateCodeForLibraries"` | `false` | enforced | +| | `"annotationsAs"` | `remove` | | +| | `"enableLegacyTemplate"` | `false` | | +| | `"preserveWhitespaces"` | `false` | | +| | `"skipMetadataEmit"` | `false` | enforced | +| | `"strictMetadataEmit"` | `true` | | +| | `"skipTemplateCodegen"` | `false` | enforced | +| | `"fullTemplateTypeCheck"` | `true` | | +| | `"enableLegacyTemplate"` | `false` | | + +Options that are marked "enforced" are reported as an error if they are +explicitly set to a value different from what is specified here. The options +marked "required" are required to be set and an error message is displayed if +no value is supplied but no default is provided. + +The purpose of the "application" target is for the options used when the `ngc` +invocation contains the root application module. Lazy loaded modules should +also be considered "application" targets. + +The purpose of the "library" target is for are all `ngc` invocations that do +not contain the root application module or a lazy loaded module. + +The purpose of the "package" target is to produce a library package that will +be an entry point for an npm package. Each entry point should be separately +compiled using a "package" target. + +##### example - application + +To produce a Renderer2 application the options would look like, + +```json +{ + "compileOptions": { + ... + }, + "angularCompilerOptions": { + "target": "application" + } +} +``` + +alternately, since the recommended `"application"` options are the default +values, the `"angularCompilerOptions"` can be out. + +##### example - library + +To produce a Renderer2 library the options would look like, + +```json +{ + "compileOptions": { + ... + }, + "angularCompilerOptions": { + "target": "library" + } +} +``` + +##### example - package + +To produce a Renderer2 package the options would look like, + +```json +{ + "compileOptions": { + ... + }, + "angularCompilerOptions": { + "target": "package" + } +} +``` + +##### example - ivy application + +To produce an ivy application the options would look like, + +```json +{ + "compileOptions": { + ... + }, + "angularCompilerOptions": { + "target": "application", + "enableIvy": true + } +} +``` + +##### example - ivy library + +To produce an ivy application the options would look like, + +```json +{ + "compileOptions": { + ... + }, + "angularCompilerOptions": { + "target": "library", + "enableIvy": true + } +} +``` + +##### example - ivy package + +Ivy packages are not supported in Angular 6.0 as they are not recommended in +npm packages as they would only be usable if in ivy application where an ivy +application. Ivy application support Renderer2 libraries so npm packages +should all be Renderer2 libraries. + +## `ng_module` output (Bazel) + +The `ng_module` rule describes the source necessary to produce a Angular +library that is reusable and composable into an application. + +### Angular 5.0 + +The `ng_module` rule invokes `ngc`[1](#ngc_wrapped) to produce +the Angular output. However, `ng_module` uses a feature, the `.ngsummary.json` +file, not normally used and is often difficult to configure correctly. + +The `.ngsummary.json` describes all the information that is necessary for +the compiler to use a generated factory. It is produced by actions defined +in the `ng_module` rule and is consumed by actions defined by `ng_module` +rules that depend on other `ng_module` rules. + +### Angular Ivy + +The `ng_module` rule will still use `ngc` to produce the Angular output but, +when producing ivy output, it no longer will need the `.ngsummary.json` file. + +#### `ng_experimental_ivy_srcs` + +The `ng_experimental_ivy_srcs` can be used as use to cause the ivy versions of +files to be generated. It is intended the sole dependency of a `ts_dev_server` +rule and the `ts_dev_server` sources move to `ng_experimental_iv_srcs`. + +#### `ng_module` ivy output + +The `ng_module` is able to provide the ivy version of the `.js` files which +will be generated with as `.ivy.js` for the development sources and `.ivy.closure.js` +for the production sources. + +The `ng_module` rule will also generate a `angular.back_patch.js` and `.closure.js` +files and a `module_scope.json` file. The type of the `module_scope.json` file will +be: + +```ts +interface ModuleScopeSummary { + [moduleName: string]: ModuleScopeEntry[]; +} +``` + +where `moduleName` is the name of the as it would appear in an import statement +in a `.ts` file at the same relative location in the source tree. All the +references in this file are also relative this location. + +##### example + +The following is a typical Angular application build in bazel: + +*src/BUILD.bazel* +```py +ng_module( + name = "src", + srcs = glob(["*.ts"]), + deps= ["//common/component"], +) + +ts_dev_server( + name = "server", + srcs = ":src", +) +``` + +To use produce an ivy version you would add: + +```py +ng_experimental_ivy_srcs( + name = "ivy_srcs", + srcs = ":src", +) + +ts_dev_server( + name = "server_ivy", + srcs = [":ivy_srcs"] +) +``` + +To serve the Renderer2 version, you would run: + +```sh +bazel run :server +``` + +to serve the ivy version you would run + +```sh +bazel run :server_ivy +``` + +The `ng_experimental_ivy_srcs` rule is only needed when ivy is experimental. Once ivy +is released the `ng_experimental_ivy_srcs`, dependent rules, can be removed. + +--- +1 More correctly, it calls `performCompilation` +from the `@angular/compiler-cli` which is what `ngc` does too.