fccffc647b
The AIO search index is built in a WebWorker on the browser from a set of page information that is downloaded as a JSON file (`search-data.json`). We want to keep this file as small as possible while providing enough data to generate a useful index to query against. Previously, we only included one copy of each (non-ignored) term from each doc but this prevents more subtle ranking of query results, since the number of occurences of a term in a doc is lost. This commit changes the generated file in the following ways: - All non-ignored terms are now included in the order in which they appear in the doc. - The terms are indexed into a dictonary to avoid the text of the term being repeated in every doc that contains the term. - Each term is pre-"stemmed" using the same Porter Stemming algorith that the Lunr search engine uses. The web-worker has been updated to decode the new format of the file. Now that all terms are included, it may enable some level of phrase based matching in the future. The size of the generated file is considerably larger than previously, but on production HTTP servers the data is sent compressed, which reduces the size dramatically. PR Close #41368
Overview
All the content that is rendered by the AIO application, and some of its configuration files, are generated from source files by Dgeni. Dgeni is a general purpose documentation generation tool.
Markdown files in /aio/content, code comments in the core Angular source files and example files are processed and transformed into files that are consumed by the AIO application.
Dgeni is configured by "packages", which contain services and processors.
Some of these packages are installed as node_modules from the dgeni-packages and some are specific to the AIO project.
The project specific packages are stored in this folder (aio/tools/transforms).
If you are an author and want to know how to generate the documentation, the steps are outlined in the top level README.md.
Root packages
To run Dgeni, you must specify a root package, which acts as the entry point to the documentation generation.
This root package, in turn requires a number of other packages, some are defined locally in the tools/transforms folder, such as tools/transforms/cheatsheet-package and tools/transforms/content-package, etc.
And some are brought in from the dgeni-packages node modules, such as jsdoc and nunjucks.
- The primary root package is defined in
tools/transforms/angular.io-package/index.js. This package is used to run a full generation of all the documentation. - There are also root packages defined in
tools/transforms/authors-package/*-package.js. These packages are used by the documentation authors when writing docs, since it allows them to run partial doc generation, which is not complete but is faster for quickly seeing changes to the document that you are working on.
Other packages
- angular-base-package
- angular-api-package
- angular-content-package
- content-package
- examples-package
- links-package
- post-process-package
- remark-package
- target-package
Templates
All the templates for the angular.io dgeni transformations are stoted in the tools/transforms/templates folder.
See the README.