build(aio): add `doc-watch` yarn task for docs authors

This task is suitable for day to day docs authoring.

This task cuts corners, which makes it much faster than a full `yarn docs`
run but it does not produce completely valid output.
In general this isgood enough for authors to see their changes as they make them.

The task is triggered by a call to

```
yarn docs-watch
```

This sets up watchers on the `aio/contents` and `packages` folders.
Any changes to files below these folders new doc generation run to start.
The input to the generation is confined to a collection of files related
to the changed file. For example:

* a change to a file in `aio/content/marketing` will generate all the
marketing files.
* a change to a file in `aio/content/tutorial` or `aio/examples/toh-*`
will generate all the tutorial files (and their embedded examples).
* a change to a file in `aio/guide` or `aio/examples` (but not a `toh-`
example) will generate the appropriate guide and its embedded examples
* a change to a file in `packages` or `packages/examples` will generate
the appropriate API doc and its embedded examples.

Be aware that the mapping between docs and its examples are based on doc file
and example folder structure being equivalent. Sometimes a doc will reference
an example in a different folder, in which case the generated doc will be
inaccurate. Mostly this is not a big problem.

aio/README.md

@@ -1,31 +1,88 @@
-# Site
+# Angular documentation project (https://angular.io)
 
-This project was generated with [angular-cli](https://github.com/angular/angular-cli) version 1.0.0-beta.26.
+Everything in this folder is part of the documentation project. This includes
 
-## Development server
+* the web site for displaying the documentation
-Run `ng serve` for a dev server. Navigate to `http://localhost:4200/`. The app will automatically reload if you change any of the source files.
+* the dgeni configuration for converting source files to rendered files that can be viewed in the web site.
+* the tooling for setting up examples for development; and generating plunkers and zip files from the examples.
 
-## Code scaffolding
+## Developer tasks
 
-Run `ng generate component component-name` to generate a new component. You can also use `ng generate directive/pipe/service/class/module`.
+We use `yarn` to manage the dependencies and to run build tasks.
+You should run all these tasks from the `angular/aio` folder.
+Here are the most important tasks you might need to use:
 
-## Build
+* `yarn` - install all the dependencies.
 
-Run `ng build` to build the project. The build artifacts will be stored in the `dist/` directory. Use the `--prod` flag for a production build.
+* `yarn start` - run a development web server that watches the files; then builds the doc-viewer and reloads the page, as necessary.
+* `yarn lint` - check that the doc-viewer code follows our style rules.
+* `yarn test` - watch all the source files, for the doc-viewer, and run all the unit tests when any change.
+* `yarn e2e` - run all the e2e tests for the doc-viewer.
 
-## Running unit tests
+* `yarn docs` - generate all the docs from the source files.
+* `yarn docs-watch` - watch the Angular source and the docs files and run a short-circuited doc-gen for the docs that changed.
+* `yarn docs-lint` - check that the doc gen code follows our style rules.
+* `yarn docs-test` - run the unit tests for the doc generation code.
 
-Run `ng test` to execute the unit tests via [Karma](https://karma-runner.github.io).
+* `yarn boilerplate:add` - generate all the boilerplate code for the examples, so that they can be run locally.
+* `yarn boilerplate:remove` - remove all the boilerplate code that was added via `yarn boilerplate:add`.
+* `yarn generate-plunkers` - generate the plunker files that are used by the `live-example` tags in the docs.
 
-## Running end-to-end tests
 
-Run `ng e2e` to execute the end-to-end tests via [Protractor](http://www.protractortest.org/).
+## Guide to authoring
-Before running the tests make sure you are serving the app via `ng serve`.
 
-## Deploying to GitHub Pages
+There are two types of content in the documentatation:
 
-Run `ng github-pages:deploy` to deploy to GitHub Pages.
+* **API docs**: descriptions of the modules, classes, interfaces, decorators, etc that make up the Angular platform.
+API docs are generated directly from the source code.
+The source code is contained in TypeScript files, located in the `angular/packages` folder.
+Each API item may have a preceding comment, which contains JSDoc style tags and content.
+The content is written in markdown.
 
-## Further help
+* **Other content**: guides, tutorials, and other marketing material.
+All other content is written using markdown in text files, located in the `angular/aio/content` folder.
+More specifically, there are sub-folders that contain particular types of content: guides, tutorial and marketing.
 
-To get more help on the `angular-cli` use `ng help` or go check out the [Angular-CLI README](https://github.com/angular/angular-cli/blob/master/README.md).
+We use the [dgeni](https://github.com/angular/dgeni) tool to convert these files into docs that can be viewed in the doc-viewer.
+
+### Generating the complete docs
+
+The main task for generating the docs is `yarn docs`. This will process all the source files (API and other),
+extracting the documentation and generating JSON files that can be consumed by the doc-viewer.
+
+### Partial doc generation for editors
+
+Full doc generation can take up to one minute. That's too slow for efficient document creation and editing.
+
+While you can make small changes in a smart editor that displays formatted markdown (e.g,. VS Code), you
+also want to see those changes displayed properly in the doc viewer. You'll want a quicker edit/view cycle
+time.
+
+For this purpose, use the `yarn docs-watch` task, which watches for changes to source files and only
+re-processes the the files necessary to generate the docs that are related to the file that has changed.
+Since this task takes shortcuts, it is much faster (often less than 1 second) but it won't produce full
+fidelity content. For example, links to other docs and code examples may not render correctly. This is
+most particularly noticed in links to other docs and in the embedded examples, which may not always render
+correctly.
+
+The general setup is as follows:
+
+* Open a terminal, ensure the dependencies are installed; run an initial doc generation; then start the doc-viewer:
+
+```bash
+yarn
+yarn docs
+yarn start
+```
+
+* Open a second terminal and start watching the docs
+
+```bash
+yarn docs-watch
+```
+
+* Open a browser at https://localhost:4200/ and navigate to the document on which you want to work.
+You can automatically open the browser by using `yarn start -- -o` in the first terminal.
+
+* Make changes to the page's associated doc or example files. Every time a file is saved, the doc will
+be regenerated, the app will rebuild and the page will reload.

aio/package.json

@@ -21,6 +21,7 @@
     "check-env": "node ../tools/check-environment.js",
     "predocs": "rimraf src/content",
     "docs": "dgeni ./transforms/angular.io-package",
+    "docs-watch": "node transforms/authors-package/watchr.js",
     "docs-lint": "eslint --ignore-path=\"transforms/.eslintignore\" transforms",
     "docs-test": "node ../dist/tools/cjs-jasmine/index-tools ../../transforms/**/*.spec.js",
     "~~update-webdriver": "webdriver-manager update --standalone false --gecko false",
@@ -84,6 +85,7 @@
     "ts-node": "~2.0.0",
     "tslint": "~4.5.0",
     "typescript": "2.2.0",
-    "vrsource-tslint-rules": "^4.0.1"
+    "vrsource-tslint-rules": "^4.0.1",
+    "watchr": "^3.0.1"
   }
 }

aio/transforms/README.md

@@ -8,20 +8,20 @@ such as `docs/cheatsheet-package` and `docs/content-package`, etc. And some are
 
 ## Generating the docs
 
-To generate the documentation simply run `gulp docs` from the command line.
+To generate the documentation simply run `yarn docs` from the command line.
 
 ## Testing the dgeni packages
 
-The local packages have unit tests that you can execute by running `gulp docs-test` from the command line.
+The local packages have unit tests that you can execute by running `yarn docs-test` from the command line.
 
 ## What does it generate?
 
-The output from dgeni is written to files in the `dist/docs` folder.
+The output from dgeni is written to files in the `src/content` folder.
 
-Notably this includes a partial HTML file for each "page" of the documentation, such as API pages and guides.
+Notably this includes a JSON file containing the partial HTML for each "page" of the documentation, such as API pages and guides.
-It also includes JavaScript files that contain metadata about the documentation such as navigation data and
+It also includes JSON files that contain metadata about the documentation such as navigation data and
 keywords for building a search index.
 
 ## Viewing the docs
 
-You can view the dummy demo app using a simple HTTP server hosting `dist/docs/index.html`
+You can view the pages by running `yarn start` and navigating to https://localhost:4200.

aio/transforms/authors-package/api-package.js

@@ -0,0 +1,92 @@
+/**
+ * @license
+ * Copyright Google Inc. All Rights Reserved.
+ *
+ * Use of this source code is governed by an MIT-style license that can be
+ * found in the LICENSE file at https://angular.io/license
+ */
+const Package = require('dgeni').Package;
+const { basePackage, API_SOURCE_PATH } = require('./base-package');
+const packageMap = {
+  common: ['common/index.ts', 'common/testing/index.ts'],
+  core: ['core/index.ts', 'core/testing/index.ts'],
+  forms: ['forms/index.ts'],
+  http: ['http/index.ts', 'http/testing/index.ts'],
+  'platform-browser': ['platform-browser/index.ts', 'platform-browser/testing/index.ts'],
+  'platform-browser-dynamic': ['platform-browser-dynamic/index.ts', 'platform-browser-dynamic/testing/index.ts'],
+  'platform-server': ['platform-server/index.ts', 'platform-server/testing/index.ts'],
+  'platform-webworker': ['platform-webworker/index.ts'],
+  'platform-webworker-dynamic': 'platform-webworker-dynamic/index.ts',
+  router: ['router/index.ts', 'router/testing/index.ts'],
+  upgrade: ['upgrade/index.ts', 'upgrade/static.ts']
+};
+function createPackage(packageName) {
+
+  return new Package('author-api', [require('dgeni-packages/typescript'), basePackage])
+    .processor(require('../angular.io-package/processors/convertPrivateClassesToInterfaces'))
+    .processor(require('../angular.io-package/processors/mergeDecoratorDocs'))
+    .processor(require('../angular.io-package/processors/extractDecoratedClasses'))
+    .processor(require('../angular.io-package/processors/matchUpDirectiveDecorators'))
+    .processor(require('../angular.io-package/processors/filterMemberDocs'))
+    .processor(require('../angular.io-package/processors/markBarredODocsAsPrivate'))
+    .processor(require('../angular.io-package/processors/filterPrivateDocs'))
+    .processor(require('../angular.io-package/processors/filterIgnoredDocs'))
+    .config(function(readTypeScriptModules) {
+      // API files are typescript
+      readTypeScriptModules.basePath = API_SOURCE_PATH;
+      readTypeScriptModules.ignoreExportsMatching = [/^_/];
+      readTypeScriptModules.hidePrivateMembers = true;
+      readTypeScriptModules.sourceFiles = packageMap[packageName];
+    })
+    .config(function(readFilesProcessor) {
+      readFilesProcessor.sourceFiles = [
+        {
+          basePath: API_SOURCE_PATH,
+          include: `${API_SOURCE_PATH}/examples/${packageName}/**/*`,
+          fileReader: 'exampleFileReader'
+        }
+      ];
+    })
+    .config(function(
+        computeIdsProcessor, computePathsProcessor, EXPORT_DOC_TYPES) {
+
+      const API_SEGMENT = 'api';
+
+      // Replace any path templates inherited from other packages
+      // (we want full and transparent control)
+      computePathsProcessor.pathTemplates = [
+        {
+          docTypes: ['module'],
+          getPath: function computeModulePath(doc) {
+            doc.moduleFolder = `${API_SEGMENT}/${doc.id.replace(/\/index$/, '')}`;
+            return doc.moduleFolder;
+          },
+          outputPathTemplate: '${moduleFolder}.json'
+        },
+        {
+          docTypes: EXPORT_DOC_TYPES.concat(['decorator', 'directive', 'pipe']),
+          pathTemplate: '${moduleDoc.moduleFolder}/${name}',
+          outputPathTemplate: '${moduleDoc.moduleFolder}/${name}.json',
+        },
+        {docTypes: ['example-region'], getOutputPath: function() {}},
+        {
+          docTypes: ['content'],
+          getPath: (doc) => `${doc.id.replace(/\/index$/, '')}`,
+          outputPathTemplate: '${path}.json'
+        },
+        {docTypes: ['navigation-json'], pathTemplate: '${id}', outputPathTemplate: '../${id}.json'},
+        {docTypes: ['contributors-json'], pathTemplate: '${id}', outputPathTemplate: '../${id}.json'}
+      ];
+    })
+
+    .config(function(convertToJsonProcessor, EXPORT_DOC_TYPES) {
+      convertToJsonProcessor.docTypes = EXPORT_DOC_TYPES.concat([
+        'content', 'decorator', 'directive', 'pipe', 'module'
+      ]);
+    });
+}
+
+
+module.exports = {
+  createPackage
+};

aio/transforms/authors-package/base-package.js

@@ -0,0 +1,194 @@
+/**
+ * @license
+ * Copyright Google Inc. All Rights Reserved.
+ *
+ * Use of this source code is governed by an MIT-style license that can be
+ * found in the LICENSE file at https://angular.io/license
+ */
+/* eslint no-console: "off" */
+const path = require('path');
+const fs = require('fs');
+const Package = require('dgeni').Package;
+
+const jsdocPackage = require('dgeni-packages/jsdoc');
+const nunjucksPackage = require('dgeni-packages/nunjucks');
+const linksPackage = require('../links-package');
+const examplesPackage = require('../examples-package');
+const targetPackage = require('../target-package');
+const contentPackage = require('../content-package');
+const remarkPackage = require('../remark-package');
+
+const PROJECT_ROOT = path.resolve(__dirname, '../../..');
+const API_SOURCE_PATH = path.resolve(PROJECT_ROOT, 'packages');
+const AIO_PATH = path.resolve(PROJECT_ROOT, 'aio');
+const CONTENTS_PATH = path.resolve(AIO_PATH, 'content');
+const TEMPLATES_PATH = path.resolve(AIO_PATH, 'transforms/templates');
+const OUTPUT_PATH = path.resolve(AIO_PATH, 'src/content');
+const DOCS_OUTPUT_PATH = path.resolve(OUTPUT_PATH, 'docs');
+
+const basePackage = new Package('authors-base', [
+  jsdocPackage, nunjucksPackage, linksPackage, examplesPackage,
+  targetPackage, contentPackage, remarkPackage
+])
+
+  // Register the processors
+  .processor(require('../angular.io-package/processors/checkUnbalancedBackTicks'))
+  .processor(require('../angular.io-package/processors/convertToJson'))
+  .processor(require('../angular.io-package/processors/fixInternalDocumentLinks'))
+  .processor(require('../angular.io-package/processors/copyContentAssets'))
+
+  // overrides base packageInfo and returns the one for the 'angular/angular' repo.
+  .factory('packageInfo', function() { return require(path.resolve(PROJECT_ROOT, 'package.json')); })
+  .factory(require('../angular.io-package/readers/json'))
+  .factory(require('../angular.io-package/services/copyFolder'))
+
+  .config(function(checkAnchorLinksProcessor) {
+    // TODO: re-enable
+    checkAnchorLinksProcessor.$enabled = false;
+  })
+
+  // Where do we get the source files?
+  .config(function(readFilesProcessor, collectExamples, jsonFileReader) {
+
+    readFilesProcessor.basePath = PROJECT_ROOT;
+    readFilesProcessor.fileReaders.push(jsonFileReader);
+    collectExamples.exampleFolders = ['examples'];
+  })
+
+  // Where do we write the output files?
+  .config(function(writeFilesProcessor) { writeFilesProcessor.outputFolder = DOCS_OUTPUT_PATH; })
+
+
+  // Target environments
+  // TODO: remove this stuff when we have no more target inline tags
+  .config(function(targetEnvironments) {
+    const ALLOWED_LANGUAGES = ['ts', 'js', 'dart'];
+    const TARGET_LANGUAGE = 'ts';
+
+    ALLOWED_LANGUAGES.forEach(target => targetEnvironments.addAllowed(target));
+    targetEnvironments.activate(TARGET_LANGUAGE);
+  })
+
+
+  // Configure jsdoc-style tag parsing
+  .config(function(parseTagsProcessor, getInjectables, inlineTagProcessor) {
+    // Load up all the tag definitions in the tag-defs folder
+    parseTagsProcessor.tagDefinitions =
+        parseTagsProcessor.tagDefinitions.concat(getInjectables(requireFolder('../angular.io-package/tag-defs')));
+
+    // We actually don't want to parse param docs in this package as we are getting the data
+    // out using TS
+    // TODO: rewire the param docs to the params extracted from TS
+    parseTagsProcessor.tagDefinitions.forEach(function(tagDef) {
+      if (tagDef.name === 'param') {
+        tagDef.docProperty = 'paramData';
+        tagDef.transforms = [];
+      }
+    });
+
+    inlineTagProcessor.inlineTagDefinitions.push(require('../angular.io-package/inline-tag-defs/anchor'));
+  })
+
+  // Configure nunjucks rendering of docs via templates
+  .config(function(
+      renderDocsProcessor, templateFinder, templateEngine, getInjectables) {
+
+    // Where to find the templates for the doc rendering
+    templateFinder.templateFolders = [TEMPLATES_PATH];
+
+    // Standard patterns for matching docs to templates
+    templateFinder.templatePatterns = [
+      '${ doc.template }', '${ doc.id }.${ doc.docType }.template.html',
+      '${ doc.id }.template.html', '${ doc.docType }.template.html',
+      '${ doc.id }.${ doc.docType }.template.js', '${ doc.id }.template.js',
+      '${ doc.docType }.template.js', '${ doc.id }.${ doc.docType }.template.json',
+      '${ doc.id }.template.json', '${ doc.docType }.template.json', 'common.template.html'
+    ];
+
+    // Nunjucks and Angular conflict in their template bindings so change Nunjucks
+    templateEngine.config.tags = {variableStart: '{$', variableEnd: '$}'};
+
+    templateEngine.filters =
+        templateEngine.filters.concat(getInjectables(requireFolder('../angular.io-package/rendering')));
+
+    // helpers are made available to the nunjucks templates
+    renderDocsProcessor.helpers.relativePath = function(from, to) {
+      return path.relative(from, to);
+    };
+  })
+
+  // We are not going to be relaxed about ambiguous links
+  .config(function(getLinkInfo) {
+    getLinkInfo.useFirstAmbiguousLink = false;
+  })
+
+  .config(function(computeIdsProcessor, computePathsProcessor) {
+
+    // Replace any path templates inherited from other packages
+    // (we want full and transparent control)
+    computePathsProcessor.pathTemplates = [
+      {docTypes: ['example-region'], getOutputPath: function() {}},
+      {
+        docTypes: ['content'],
+        getPath: (doc) => `${doc.id.replace(/\/index$/, '')}`,
+        outputPathTemplate: '${path}.json'
+      },
+      {docTypes: ['navigation-json'], pathTemplate: '${id}', outputPathTemplate: '../${id}.json'},
+      {docTypes: ['contributors-json'], pathTemplate: '${id}', outputPathTemplate: '../${id}.json'},
+      {docTypes: ['resources-json'], pathTemplate: '${id}', outputPathTemplate: '../${id}.json'}
+    ];
+  })
+
+  .config(function(convertToJsonProcessor) {
+    convertToJsonProcessor.docTypes = ['content'];
+  })
+
+  .config(function(copyContentAssetsProcessor) {
+    copyContentAssetsProcessor.assetMappings.push(
+      { from: path.resolve(CONTENTS_PATH, 'images'), to: path.resolve(OUTPUT_PATH, 'images') }
+    );
+  });
+
+function requireFolder(folderPath) {
+  const absolutePath = path.resolve(__dirname, folderPath);
+  return fs.readdirSync(absolutePath)
+      .filter(p => !/[._]spec\.js$/.test(p))  // ignore spec files
+      .map(p => require(path.resolve(absolutePath, p)));
+}
+
+function getBoilerPlateExcludes() {
+  return [
+    '**/*plnkr.no-link.html',
+    '**/node_modules/**',
+    // _boilerplate files
+    '**/_boilerplate/**',
+    '**/*/src/styles.css',
+    '**/*/src/systemjs-angular-loader.js',
+    '**/*/src/systemjs.config.js',
+    '**/*/src/tsconfig.json',
+    '**/*/bs-config.e2e.json',
+    '**/*/bs-config.json',
+    '**/*/package.json',
+    '**/*/tslint.json',
+    // example files
+    '**/_test-output',
+    '**/protractor-helpers.js',
+    '**/e2e-spec.js',
+    '**/ts/**/*.js',
+    '**/js-es6*/**/*.js',
+    '**/ts-snippets/**/*.js',
+  ];
+}
+
+module.exports = {
+  basePackage,
+  PROJECT_ROOT,
+  API_SOURCE_PATH,
+  AIO_PATH,
+  CONTENTS_PATH,
+  TEMPLATES_PATH,
+  OUTPUT_PATH,
+  DOCS_OUTPUT_PATH,
+  requireFolder,
+  getBoilerPlateExcludes
+};

aio/transforms/authors-package/guide-package.js

@@ -0,0 +1,31 @@
+/**
+ * @license
+ * Copyright Google Inc. All Rights Reserved.
+ *
+ * Use of this source code is governed by an MIT-style license that can be
+ * found in the LICENSE file at https://angular.io/license
+ */
+const Package = require('dgeni').Package;
+const { basePackage, CONTENTS_PATH, getBoilerPlateExcludes } = require('./base-package');
+
+function createPackage(exampleName) {
+  return new Package('author-guide', [basePackage])
+    .config(function(readFilesProcessor) {
+      readFilesProcessor.sourceFiles = [
+        {
+          basePath: CONTENTS_PATH,
+          include: `${CONTENTS_PATH}/guide/${exampleName}.md`,
+          fileReader: 'contentFileReader'
+        },
+        {
+          basePath: CONTENTS_PATH,
+          include: `${CONTENTS_PATH}/examples/*${exampleName}/**/*`,
+          exclude: getBoilerPlateExcludes(),
+          fileReader: 'exampleFileReader'
+        }
+      ];
+    });
+}
+
+
+module.exports = { createPackage };

aio/transforms/authors-package/index.js

@@ -0,0 +1,50 @@
+/**
+ * @license
+ * Copyright Google Inc. All Rights Reserved.
+ *
+ * Use of this source code is governed by an MIT-style license that can be
+ * found in the LICENSE file at https://angular.io/license
+ */
+/* eslint no-console: "off" */
+
+function createPackage(changedFile) {
+  const marketingMatch = /^aio\/content\/marketing\/(.*)/.exec(changedFile);
+  if (marketingMatch) {
+    console.log('Building marketing docs');
+    return require('./marketing-package').createPackage();
+  }
+
+  const tutorialMatch = /^aio\/content\/tutorial\/|^aio\/content\/examples\/toh-\d/.exec(changedFile);
+  if (tutorialMatch) {
+    console.log('Building tutorial docs');
+    return require('./tutorial-package').createPackage();
+  }
+
+  const guideMatch = /^aio\/content\/guide\/([^\.]+)\.md/.exec(changedFile);
+  const exampleMatch = /^aio\/content\/examples\/(?:cb-)?([^\/]+)\//.exec(changedFile);
+  if (guideMatch || exampleMatch) {
+    const exampleName = guideMatch && guideMatch[1] || exampleMatch[1];
+    console.log(`Building guide doc: ${exampleName}.md`);
+    return require('./guide-package').createPackage(exampleName);
+  }
+
+  const apiExamplesMatch = /^packages\/examples\/([^\/]+)\//.exec(changedFile);
+  const apiMatch = /^packages\/([^\/]+)\//.exec(changedFile);
+  if (apiExamplesMatch || apiMatch) {
+    const packageName = apiExamplesMatch && apiExamplesMatch[1] || apiMatch[1];
+    console.log('Building API docs for', packageName);
+    return require('./api-package').createPackage(packageName);
+  }
+}
+
+module.exports = {
+  generateDocs: function(changedFile) {
+    const {Dgeni} = require('dgeni');
+    var dgeni = new Dgeni([createPackage(changedFile)]);
+    const start = Date.now();
+    return dgeni.generate()
+      .then(
+        () => console.log('Generated docs in ' + (Date.now() - start)/1000 + ' secs'),
+        err => console.log('Error generating docs', err));
+  }
+};

aio/transforms/authors-package/index.spec.js

@@ -0,0 +1,102 @@
+const fs = require('fs');
+const {resolve} = require('canonical-path');
+const {generateDocs} = require('./index.js');
+const rootPath = resolve(__dirname, '../../..');
+const outputPath = resolve(rootPath, 'aio/src/content/docs');
+
+describe('authors-package', () => {
+  let files;
+
+  beforeEach(() => {
+    files = [];
+    spyOn(fs, 'writeFile').and.callFake((file, content, callback) => {
+      files.push(file);
+      callback();
+    });
+  });
+
+  it('should generate marketing docs if the "fileChanged" is a marketing doc', (done) => {
+    generateDocs('aio/content/marketing/about.html').then(() => {
+      expect(fs.writeFile).toHaveBeenCalled();
+      expect(files).toContain(resolve(outputPath, 'about.json'));
+      expect(files).toContain(resolve(outputPath, '../navigation.json'));
+      expect(files).toContain(resolve(outputPath, '../contributors.json'));
+      expect(files).toContain(resolve(outputPath, '../resources.json'));
+      done();
+    });
+  }, 2000);
+
+  it('should generate tutorial docs if the "fileChanged" is a tutorial doc', (done) => {
+    generateDocs('aio/content/tutorial/toh-pt5.md').then(() => {
+      expect(fs.writeFile).toHaveBeenCalled();
+      expect(files).toContain(resolve(outputPath, 'tutorial.json'));
+      expect(files).toContain(resolve(outputPath, 'tutorial/toh-pt1.json'));
+      expect(files).toContain(resolve(outputPath, 'tutorial/toh-pt2.json'));
+      expect(files).toContain(resolve(outputPath, 'tutorial/toh-pt3.json'));
+      expect(files).toContain(resolve(outputPath, 'tutorial/toh-pt4.json'));
+      expect(files).toContain(resolve(outputPath, 'tutorial/toh-pt5.json'));
+      expect(files).toContain(resolve(outputPath, 'tutorial/toh-pt6.json'));
+      done()
+    });
+  }, 2000);
+
+  it('should generate tutorial docs if the "fileChanged" is the tutorial index', (done) => {
+    generateDocs('aio/content/tutorial/index.md').then(() => {
+      expect(fs.writeFile).toHaveBeenCalled();
+      expect(files).toContain(resolve(outputPath, 'tutorial.json'));
+      expect(files).toContain(resolve(outputPath, 'tutorial/toh-pt1.json'));
+      expect(files).toContain(resolve(outputPath, 'tutorial/toh-pt2.json'));
+      expect(files).toContain(resolve(outputPath, 'tutorial/toh-pt3.json'));
+      expect(files).toContain(resolve(outputPath, 'tutorial/toh-pt4.json'));
+      expect(files).toContain(resolve(outputPath, 'tutorial/toh-pt5.json'));
+      expect(files).toContain(resolve(outputPath, 'tutorial/toh-pt6.json'));
+      done();
+    });
+  }, 2000);
+
+  it('should generate tutorial docs if the "fileChanged" is a tutorial example', (done) => {
+    generateDocs('aio/content/examples/toh-3/app/app.component.1.html').then(() => {
+      expect(fs.writeFile).toHaveBeenCalled();
+      expect(files).toContain(resolve(outputPath, 'tutorial.json'));
+      expect(files).toContain(resolve(outputPath, 'tutorial/toh-pt1.json'));
+      expect(files).toContain(resolve(outputPath, 'tutorial/toh-pt2.json'));
+      expect(files).toContain(resolve(outputPath, 'tutorial/toh-pt3.json'));
+      expect(files).toContain(resolve(outputPath, 'tutorial/toh-pt4.json'));
+      expect(files).toContain(resolve(outputPath, 'tutorial/toh-pt5.json'));
+      expect(files).toContain(resolve(outputPath, 'tutorial/toh-pt6.json'));
+      done();
+    });
+  }, 2000);
+
+  it('should generate guide doc if the "fileChanged" is a guide doc', (done) => {
+    generateDocs('aio/content/guide/architecture.md').then(() => {
+      expect(fs.writeFile).toHaveBeenCalled();
+      expect(files).toContain(resolve(outputPath, 'guide/architecture.json'));
+      done();
+    });
+  }, 2000);
+
+  it('should generate guide doc if the "fileChanged" is a guide example', (done) => {
+    generateDocs('aio/content/examples/architecture/src/app/app.module.ts').then(() => {
+      expect(fs.writeFile).toHaveBeenCalled();
+      expect(files).toContain(resolve(outputPath, 'guide/architecture.json'));
+      done();
+    });
+  }, 2000);
+
+  it('should generate API doc if the "fileChanged" is an API doc', (done) => {
+    generateDocs('packages/forms/src/form_builder.ts').then(() => {
+      expect(fs.writeFile).toHaveBeenCalled();
+      expect(files).toContain(resolve(outputPath, 'api/forms/FormBuilder.json'));
+      done();
+    });
+  }, 4000);
+
+  it('should generate API doc if the "fileChanged" is an API example', (done) => {
+    generateDocs('packages/examples/forms/ts/formBuilder/form_builder_example.ts').then(() => {
+      expect(fs.writeFile).toHaveBeenCalled();
+      expect(files).toContain(resolve(outputPath, 'api/forms/FormBuilder.json'));
+      done();
+    });
+  }, 4000);
+});

aio/transforms/authors-package/marketing-package.js

@@ -0,0 +1,41 @@
+/**
+ * @license
+ * Copyright Google Inc. All Rights Reserved.
+ *
+ * Use of this source code is governed by an MIT-style license that can be
+ * found in the LICENSE file at https://angular.io/license
+ */
+const Package = require('dgeni').Package;
+const { basePackage, CONTENTS_PATH } = require('./base-package');
+
+function createPackage() {
+  return new Package('author-marketing', [basePackage])
+    .config(function(readFilesProcessor) {
+      readFilesProcessor.sourceFiles = [
+        {
+          basePath: CONTENTS_PATH + '/marketing',
+          include: CONTENTS_PATH + '/marketing/**/*.{html,md}',
+          fileReader: 'contentFileReader'
+        },
+        {
+          basePath: CONTENTS_PATH,
+          include: CONTENTS_PATH + '/*.md',
+          exclude: [CONTENTS_PATH + '/index.md'],
+          fileReader: 'contentFileReader'
+        },
+        {
+          basePath: CONTENTS_PATH,
+          include: CONTENTS_PATH + '/marketing/*.json',
+          fileReader: 'jsonFileReader'
+        },
+        {
+          basePath: CONTENTS_PATH,
+          include: CONTENTS_PATH + '/navigation.json',
+          fileReader: 'jsonFileReader'
+        },
+      ];
+    });
+}
+
+
+module.exports = { createPackage };

aio/transforms/authors-package/tutorial-package.js

@@ -0,0 +1,31 @@
+/**
+ * @license
+ * Copyright Google Inc. All Rights Reserved.
+ *
+ * Use of this source code is governed by an MIT-style license that can be
+ * found in the LICENSE file at https://angular.io/license
+ */
+const Package = require('dgeni').Package;
+const { basePackage, CONTENTS_PATH, getBoilerPlateExcludes } = require('./base-package');
+
+function createPackage() {
+  return new Package('author-tutorial', [basePackage])
+    .config(function(readFilesProcessor) {
+      readFilesProcessor.sourceFiles = [
+        {
+          basePath: CONTENTS_PATH,
+          include: CONTENTS_PATH + '/tutorial/**/*.md',
+          fileReader: 'contentFileReader'
+        },
+        {
+          basePath: CONTENTS_PATH,
+          include: CONTENTS_PATH + '/examples/toh-*/**/*',
+          exclude: getBoilerPlateExcludes(),
+          fileReader: 'exampleFileReader'
+        }
+      ];
+    });
+}
+
+
+module.exports = { createPackage };

aio/transforms/authors-package/watchr.js

@@ -0,0 +1,25 @@
+/* eslint no-console: "off" */
+const watchr = require('watchr');
+const {resolve, relative} = require('canonical-path');
+const {generateDocs} = require('./index.js');
+const rootPath = resolve(__dirname, '../../..');
+const contentsPath = resolve(rootPath, 'aio/content');
+const apiPath = resolve(rootPath, 'packages');
+
+function listener(changeType, fullPath) {
+  try {
+    const relativePath = relative(rootPath, fullPath);
+    console.log('The file', relativePath, `was ${changeType}d at`, new Date().toUTCString());
+    generateDocs(relativePath);
+  } catch(err) {
+    console.log('Error generating docs', err);
+  }
+}
+
+console.log('Started watching files in:');
+console.log(' - ', contentsPath);
+console.log(' - ', apiPath);
+console.log('Doc gen will run when you change a file in either of these folders.');
+
+watchr.open(contentsPath, listener);
+watchr.open(apiPath, listener);

aio/yarn.lock

@@ -257,6 +257,13 @@ alphanum-sort@^1.0.1, alphanum-sort@^1.0.2:
   version "1.0.2"
   resolved "https://registry.yarnpkg.com/alphanum-sort/-/alphanum-sort-1.0.2.tgz#97a1119649b211ad33691d9f9f486a8ec9fbe0a3"
 
+ambi@^2.2.0, ambi@^2.5.0:
+  version "2.5.0"
+  resolved "https://registry.yarnpkg.com/ambi/-/ambi-2.5.0.tgz#7c8e372be48891157e7cea01cb6f9143d1f74220"
+  dependencies:
+    editions "^1.1.1"
+    typechecker "^4.3.0"
+
 amdefine@>=0.0.4:
   version "1.0.1"
   resolved "https://registry.yarnpkg.com/amdefine/-/amdefine-1.0.1.tgz#4a5282ac164729e93619bcfd3ad151f817ce91f5"
@@ -1478,6 +1485,10 @@ crypto-random-string@^1.0.0:
   version "1.0.0"
   resolved "https://registry.yarnpkg.com/crypto-random-string/-/crypto-random-string-1.0.0.tgz#a230f64f568310e1498009940790ec99545bca7e"
 
+csextends@^1.0.3:
+  version "1.0.3"
+  resolved "https://registry.yarnpkg.com/csextends/-/csextends-1.0.3.tgz#df41407bfddb1837ecc2dd28587725d6af9550f8"
+
 css-color-names@0.0.4:
   version "0.0.4"
   resolved "https://registry.yarnpkg.com/css-color-names/-/css-color-names-0.0.4.tgz#808adc2e79cf84738069b646cb20ec27beb629e0"
@@ -1908,6 +1919,13 @@ duplexify@^3.2.0:
     readable-stream "^2.0.0"
     stream-shift "^1.0.0"
 
+eachr@^3.2.0:
+  version "3.2.0"
+  resolved "https://registry.yarnpkg.com/eachr/-/eachr-3.2.0.tgz#2c35e43ea086516f7997cf80b7aa64d55a4a4484"
+  dependencies:
+    editions "^1.1.1"
+    typechecker "^4.3.0"
+
 ecc-jsbn@~0.1.1:
   version "0.1.1"
   resolved "https://registry.yarnpkg.com/ecc-jsbn/-/ecc-jsbn-0.1.1.tgz#0fc73a9ed5f0d53c38193398523ef7e543777505"
@@ -1930,6 +1948,10 @@ ecstatic@^1.4.0:
     minimist "^1.1.0"
     url-join "^1.0.0"
 
+editions@^1.1.1, editions@^1.1.2, editions@^1.3.1, editions@^1.3.3:
+  version "1.3.3"
+  resolved "https://registry.yarnpkg.com/editions/-/editions-1.3.3.tgz#0907101bdda20fac3cbe334c27cbd0688dc99a5b"
+
 ee-first@1.1.1:
   version "1.1.1"
   resolved "https://registry.yarnpkg.com/ee-first/-/ee-first-1.1.1.tgz#590c61156b0ae2f4f0255732a158b266bc56b21d"
@@ -2340,6 +2362,13 @@ extend@3, extend@^3.0.0, extend@~3.0.0:
   version "3.0.0"
   resolved "https://registry.yarnpkg.com/extend/-/extend-3.0.0.tgz#5a474353b9f3353ddd8176dfd37b91c83a46f1d4"
 
+extendr@^3.2.0, extendr@^3.2.2:
+  version "3.2.2"
+  resolved "https://registry.yarnpkg.com/extendr/-/extendr-3.2.2.tgz#c6e46fe6d90b2e3e8812a6654bd6182cbf91cd06"
+  dependencies:
+    editions "^1.1.2"
+    typechecker "^4.3.0"
+
 external-editor@^2.0.1:
   version "2.0.1"
   resolved "https://registry.yarnpkg.com/external-editor/-/external-editor-2.0.1.tgz#4c597c6c88fa6410e41dbbaa7b1be2336aa31095"
@@ -2837,7 +2866,7 @@ got@^6.7.1:
     unzip-response "^2.0.1"
     url-parse-lax "^1.0.0"
 
-graceful-fs@^4.1.11, graceful-fs@^4.1.2, graceful-fs@^4.1.6, graceful-fs@^4.1.9:
+graceful-fs@*, graceful-fs@^4.1.11, graceful-fs@^4.1.2, graceful-fs@^4.1.4, graceful-fs@^4.1.6, graceful-fs@^4.1.9:
   version "4.1.11"
   resolved "https://registry.yarnpkg.com/graceful-fs/-/graceful-fs-4.1.11.tgz#0e8bdfe4d1ddb8854d64e04ea7c00e2a026e5658"
 
@@ -2849,7 +2878,7 @@ handle-thing@^1.2.4:
   version "1.2.5"
   resolved "https://registry.yarnpkg.com/handle-thing/-/handle-thing-1.2.5.tgz#fd7aad726bf1a5fd16dfc29b2f7a6601d27139c4"
 
-handlebars@4.0.5:
+handlebars@4.0.5, handlebars@^4.0.3:
   version "4.0.5"
   resolved "https://registry.yarnpkg.com/handlebars/-/handlebars-4.0.5.tgz#92c6ed6bb164110c50d4d8d0fbddc70806c6f8e7"
   dependencies:
@@ -2867,16 +2896,6 @@ handlebars@^1.3.0:
   optionalDependencies:
     uglify-js "~2.3"
 
-handlebars@^4.0.3:
-  version "4.0.6"
-  resolved "https://registry.yarnpkg.com/handlebars/-/handlebars-4.0.6.tgz#2ce4484850537f9c97a8026d5399b935c4ed4ed7"
-  dependencies:
-    async "^1.4.0"
-    optimist "^0.6.1"
-    source-map "^0.4.4"
-  optionalDependencies:
-    uglify-js "^2.6"
-
 har-validator@~2.0.6:
   version "2.0.6"
   resolved "https://registry.yarnpkg.com/har-validator/-/har-validator-2.0.6.tgz#cdcbc08188265ad119b6a5a7c8ab70eecfb5d27d"
@@ -3158,6 +3177,17 @@ ignore@^3.2.0:
   version "3.2.6"
   resolved "https://registry.yarnpkg.com/ignore/-/ignore-3.2.6.tgz#26e8da0644be0bb4cb39516f6c79f0e0f4ffe48c"
 
+ignorefs@^1.0.0, ignorefs@^1.1.1:
+  version "1.2.0"
+  resolved "https://registry.yarnpkg.com/ignorefs/-/ignorefs-1.2.0.tgz#da59fb858976e4a5e43702ccd1f282fdbc9e5756"
+  dependencies:
+    editions "^1.3.3"
+    ignorepatterns "^1.1.0"
+
+ignorepatterns@^1.1.0:
+  version "1.1.0"
+  resolved "https://registry.yarnpkg.com/ignorepatterns/-/ignorepatterns-1.1.0.tgz#ac8f436f2239b5dfb66d5f0d3a904a87ac67cc5e"
+
 image-size@~0.5.0:
   version "0.5.1"
   resolved "https://registry.yarnpkg.com/image-size/-/image-size-0.5.1.tgz#28eea8548a4b1443480ddddc1e083ae54652439f"
@@ -5772,6 +5802,19 @@ safe-buffer@^5.0.1:
   version "5.0.1"
   resolved "https://registry.yarnpkg.com/safe-buffer/-/safe-buffer-5.0.1.tgz#d263ca54696cd8a306b5ca6551e92de57918fbe7"
 
+safefs@^3.1.2:
+  version "3.2.2"
+  resolved "https://registry.yarnpkg.com/safefs/-/safefs-3.2.2.tgz#8170c1444d7038e08caea05a374fae2fa349e15c"
+  dependencies:
+    graceful-fs "*"
+
+safefs@^4.1.0:
+  version "4.1.0"
+  resolved "https://registry.yarnpkg.com/safefs/-/safefs-4.1.0.tgz#f82aeb4bdd7ae51f653eb20f6728b3058c8d6445"
+  dependencies:
+    editions "^1.1.1"
+    graceful-fs "^4.1.4"
+
 sass-graph@^2.1.1:
   version "2.1.2"
   resolved "https://registry.yarnpkg.com/sass-graph/-/sass-graph-2.1.2.tgz#965104be23e8103cb7e5f710df65935b317da57b"
@@ -5806,6 +5849,14 @@ sax@>=0.6.0, sax@^1.2.1, sax@~1.2.1:
   version "1.2.2"
   resolved "https://registry.yarnpkg.com/sax/-/sax-1.2.2.tgz#fd8631a23bc7826bef5d871bdb87378c95647828"
 
+scandirectory@^2.5.0:
+  version "2.5.0"
+  resolved "https://registry.yarnpkg.com/scandirectory/-/scandirectory-2.5.0.tgz#6ce03f54a090b668e3cbedbf20edf9e310593e72"
+  dependencies:
+    ignorefs "^1.0.0"
+    safefs "^3.1.2"
+    taskgroup "^4.0.5"
+
 script-loader@^0.7.0:
   version "0.7.0"
   resolved "https://registry.yarnpkg.com/script-loader/-/script-loader-0.7.0.tgz#685dc7e7069e0dee7a92674f0ebc5b0f55baa5ec"
@@ -6421,6 +6472,22 @@ tar@^2.0.0, tar@^2.2.0, tar@~2.2.1:
     fstream "^1.0.2"
     inherits "2"
 
+taskgroup@^4.0.5:
+  version "4.3.1"
+  resolved "https://registry.yarnpkg.com/taskgroup/-/taskgroup-4.3.1.tgz#7de193febd768273c457730497024d512c27915a"
+  dependencies:
+    ambi "^2.2.0"
+    csextends "^1.0.3"
+
+taskgroup@^5.0.1:
+  version "5.0.1"
+  resolved "https://registry.yarnpkg.com/taskgroup/-/taskgroup-5.0.1.tgz#08736c9b24683b1434774231eb4b73aa7c3f79b5"
+  dependencies:
+    ambi "^2.5.0"
+    eachr "^3.2.0"
+    editions "^1.1.1"
+    extendr "^3.2.0"
+
 temp@0.8.3:
   version "0.8.3"
   resolved "https://registry.yarnpkg.com/temp/-/temp-0.8.3.tgz#e0c6bc4d26b903124410e4fed81103014dfc1f59"
@@ -6648,6 +6715,12 @@ type-is@~1.6.14:
     media-typer "0.3.0"
     mime-types "~2.1.13"
 
+typechecker@^4.3.0:
+  version "4.4.1"
+  resolved "https://registry.yarnpkg.com/typechecker/-/typechecker-4.4.1.tgz#f97b95f51b038417212d677d45a373ee7bced7e6"
+  dependencies:
+    editions "^1.3.3"
+
 typedarray@^0.0.6:
   version "0.0.6"
   resolved "https://registry.yarnpkg.com/typedarray/-/typedarray-0.0.6.tgz#867ac74e3864187b1d3d47d996a78ec5c8830777"
@@ -7028,6 +7101,18 @@ watchpack@^1.2.0:
     chokidar "^1.4.3"
     graceful-fs "^4.1.2"
 
+watchr@^3.0.1:
+  version "3.0.1"
+  resolved "https://registry.yarnpkg.com/watchr/-/watchr-3.0.1.tgz#3b7e078160d12484481d81a8640d5fd880408540"
+  dependencies:
+    eachr "^3.2.0"
+    editions "^1.3.1"
+    extendr "^3.2.2"
+    ignorefs "^1.1.1"
+    safefs "^4.1.0"
+    scandirectory "^2.5.0"
+    taskgroup "^5.0.1"
+
 wbuf@^1.1.0, wbuf@^1.4.0:
   version "1.7.2"
   resolved "https://registry.yarnpkg.com/wbuf/-/wbuf-1.7.2.tgz#d697b99f1f59512df2751be42769c1580b5801fe"