Commit Graph

15 Commits

Author SHA1 Message Date
George Kalpakas 99f2ffc740 build: update stemmer to version 2.0.0 (#41724)
NOTE:
`stemmer` v2.0.0 switched to ES modules (see
words/stemmer@03519229c8), which means
that the only way to consume it in our CommonJS setup (for example, in
[generateKeywords][1]) is via an async `import()`.

This commit makes the `generateKeywords` processor asynchronous in order
to be able to dynamically import and use `stemmer`.

[1]: 251bec159a/aio/tools/transforms/angular-base-package/processors/generateKeywords.js

PR Close #41724
2021-04-26 11:00:59 -07:00
Pete Bacon Darwin 14a64acb72 build(docs-infra): store search dictionary as a string (#41447)
Previously, the search index info file contained an array of strings that is
the dictionary of terms in the corpus.
Storing this as a space separated string reduces the size of the file.

PR Close #41447
2021-04-07 13:51:08 -07:00
Pete Bacon Darwin 65cd44e731 build(docs-infra): tidy up the generateKeywords processor (#41447)
The recent PR #41368 contained some changes that could be improved.

PR Close #41447
2021-04-07 13:51:08 -07:00
George Kalpakas d42019d412 build(docs-infra): update `eslint` and `eslint-plugin-jasmine` to latest versions (#41429)
This commit updates the `eslint` and `eslint-plugin-jasmine` packages to
latest versions to take advantage of latest fixes and improvements.

PR Close #41429
2021-04-06 10:22:53 -07:00
Pete Bacon Darwin fccffc647b refactor(docs-infra): include more info in search index data (#41368)
The AIO search index is built in a WebWorker on the browser from a set
of page information that is downloaded as a JSON file (`search-data.json`).
We want to keep this file as small as possible while providing enough
data to generate a useful index to query against.

Previously, we only included one copy of each (non-ignored) term from each
doc but this prevents more subtle ranking of query results, since the number
of occurences of a term in a doc is lost.

This commit changes the generated file in the following ways:

- All non-ignored terms are now included in the order in which they appear
  in the doc.
- The terms are indexed into a dictonary to avoid the text of the term being
  repeated in every doc that contains the term.
- Each term is pre-"stemmed" using the same Porter Stemming algorith that the
  Lunr search engine uses.

The web-worker has been updated to decode the new format of the file.
Now that all terms are included, it may enable some level of phrase based
matching in the future.

The size of the generated file is considerably larger than previously, but
on production HTTP servers the data is sent compressed, which reduces the
size dramatically.

PR Close #41368
2021-04-01 12:02:37 -07:00
Sonu Kapoor 7ca2a2a863 feat(docs-infra): add `searchKeywords` preprocessor (#35539)
This commit adds a new preprocessor to use `${@searchKeywords}`, allowing
the docs to use a list of custom search phrases that will be
prioritized over the keywords found in the content.

Closes #35449

PR Close #35539
2020-02-27 10:47:52 -08:00
Pete Bacon Darwin 7c730fe5b3 build(docs-infra): include inherited members in search index (#26676)
Closes #23800

PR Close #26676
2018-10-26 13:16:33 -04:00
Pete Bacon Darwin f233131974 build(docs-infra): refactor generateKeywords processor (#26676)
PR Close #26676
2018-10-26 13:16:32 -04:00
Pete Bacon Darwin 026b60cd70 build(docs-infra): expose deprecated status on items more clearly (#25750)
PR Close #25750
2018-09-21 10:26:48 -07:00
Pete Bacon Darwin 96f9f03d25 build(docs-infra): improve search quality (#25750)
PR Close #25750
2018-09-21 10:26:48 -07:00
Pete Bacon Darwin 9a0700f5bd build(aio): add API static members to search index (#21988)
Previously searching for `compose` did not include `Validators`
in the search results because we were not including all the
`static` members of API docs in the index.

PR Close #21988
2018-02-05 13:00:47 -08:00
Peter Bacon Darwin e8bbf86e66 build(aio): add terms from heading to the search index 2017-07-11 15:04:37 +01:00
Peter Bacon Darwin dfbbbb5e3e build(aio): the the captured h1 as the title for the search index
If there is no title already provided, use the one captured from the renderedContent.
2017-05-31 08:45:31 +03:00
Peter Bacon Darwin 978376a46e build(aio): `doc.searchTitle` can override name in search results 2017-05-04 13:45:04 +01:00
Peter Bacon Darwin 3cad5da5a4 build(aio): refactor dgeni packages
This is to tidy up the `author-packagse`, which currently duplicates a
lot of the configuration in the main packages. We need to
DRY this up so that we don't fall foul of a change in one being missed in
the other.
2017-04-23 22:50:33 +01:00