Commit Graph

38 Commits

Author SHA1 Message Date
Pete Bacon Darwin b0592c1be6 build(docs-infra): use case-insensitive encoding for content files (#42414)
To avoid having content files that have the same file path on case-insensitive
file-systems, we now encode the paths to remove uppercase characters.

PR Close #42414
2021-06-17 18:14:53 +00:00
George Kalpakas 2076bde100 build(docs-infra): take disambiguated doc paths into account when generating the sitemap (#41842)
In ##41788, the `disambiguateDocsPathsProcessor` was introduced to fix
an issue with case-insensitively equal paths. This processor may alter
the paths of some docs and thus their final URL in the app.

Previously, both the `disambiguateDocPathsProcessor` and the
`createSitemap` processor (which relies on the docs' computed paths to
generate the sitemap file) were configured to run before the
"rendering-docs" phase. However, this resulted in the
`disambiguateDocPathsProcessor`'s running after `createSitemap`, which
meant that the sitemap did not include the updated doc paths.

This commit fixes it by ensuring that the
`disambiguateDocPathsProcessor` is explicitly run before the
`createSitemap` processor, so that the latter will be able to take into
account any changes made by the former.

PR Close #41842
2021-04-27 14:05:23 -07:00
Pete Bacon Darwin 538286df16 build(docs-infra): add path disambiguation (#41788)
When two documents have the same `outputPath`, only differing by
letter casing, there can be problems on case-insensitive file-systems:
Only one of each of the docs would end up being written.

Moreover, the Webpack 5 bundler will error if it comes across files
that have this kind of ambiguous paths.

This commit adds a new docType: `disambiguator`, which will display
a list of the docs that match an ambiguous path. Each of the ambiguous
docs is then given a unique path and outputPath to ensure there are no
collisions.

PR Close #41788
2021-04-26 12:12:00 -07:00
George Kalpakas 99f2ffc740 build: update stemmer to version 2.0.0 (#41724)
NOTE:
`stemmer` v2.0.0 switched to ES modules (see
words/stemmer@03519229c8), which means
that the only way to consume it in our CommonJS setup (for example, in
[generateKeywords][1]) is via an async `import()`.

This commit makes the `generateKeywords` processor asynchronous in order
to be able to dynamically import and use `stemmer`.

[1]: 251bec159a/aio/tools/transforms/angular-base-package/processors/generateKeywords.js

PR Close #41724
2021-04-26 11:00:59 -07:00
Pete Bacon Darwin 14a64acb72 build(docs-infra): store search dictionary as a string (#41447)
Previously, the search index info file contained an array of strings that is
the dictionary of terms in the corpus.
Storing this as a space separated string reduces the size of the file.

PR Close #41447
2021-04-07 13:51:08 -07:00
Pete Bacon Darwin 65cd44e731 build(docs-infra): tidy up the generateKeywords processor (#41447)
The recent PR #41368 contained some changes that could be improved.

PR Close #41447
2021-04-07 13:51:08 -07:00
George Kalpakas d42019d412 build(docs-infra): update `eslint` and `eslint-plugin-jasmine` to latest versions (#41429)
This commit updates the `eslint` and `eslint-plugin-jasmine` packages to
latest versions to take advantage of latest fixes and improvements.

PR Close #41429
2021-04-06 10:22:53 -07:00
Pete Bacon Darwin fccffc647b refactor(docs-infra): include more info in search index data (#41368)
The AIO search index is built in a WebWorker on the browser from a set
of page information that is downloaded as a JSON file (`search-data.json`).
We want to keep this file as small as possible while providing enough
data to generate a useful index to query against.

Previously, we only included one copy of each (non-ignored) term from each
doc but this prevents more subtle ranking of query results, since the number
of occurences of a term in a doc is lost.

This commit changes the generated file in the following ways:

- All non-ignored terms are now included in the order in which they appear
  in the doc.
- The terms are indexed into a dictonary to avoid the text of the term being
  repeated in every doc that contains the term.
- Each term is pre-"stemmed" using the same Porter Stemming algorith that the
  Lunr search engine uses.

The web-worker has been updated to decode the new format of the file.
Now that all terms are included, it may enable some level of phrase based
matching in the future.

The size of the generated file is considerably larger than previously, but
on production HTTP servers the data is sent compressed, which reduces the
size dramatically.

PR Close #41368
2021-04-01 12:02:37 -07:00
twerske fbfd4889a9 build(docs-infra): error guides in docs (#40060)
add /errors to docs
create formatting for error guides infra

PR Close #40060
2021-01-14 11:28:52 -08:00
Marcono1234 3e1e5a15ba docs: update links to use HTTPS as protocol (#39718)
PR Close #39718
2020-11-20 12:52:16 -08:00
Joey Perrott ffe89fb07d test(docs-infra): remove usage of blacklist in test naming (#38927)
Remove usage of blacklist in test naming of processor tranform tests.

PR Close #38927
2020-09-23 15:46:02 -04:00
ajitsinghkaler 5b33798796 feat(docs-infra): created new widget for events page (#36517)
Data in events page was hardcoded and it is manually moved in the table.

Created a new events widget which will automatically move past and upcoming
events from events.json (`aio/content/marketing/events.json`) file to the
relevant table in the events tab

PR Close #36517
2020-09-16 15:14:18 -07:00
Sonu Kapoor 03afa36334 refactor(docs-infra): refactors `checkUnbalancedBackTicks` (#37065)
This commit removes the dependency on the `lodash` module and refactors
the `checkUnbalancedBackTicks` method.

PR Close #37065
2020-05-15 10:13:19 -07:00
Sonu Kapoor 7ca2a2a863 feat(docs-infra): add `searchKeywords` preprocessor (#35539)
This commit adds a new preprocessor to use `${@searchKeywords}`, allowing
the docs to use a list of custom search phrases that will be
prioritized over the keywords found in the content.

Closes #35449

PR Close #35539
2020-02-27 10:47:52 -08:00
George Kalpakas d7ca263cc4 test(docs-infra): run tests in random order (and make them pass) (#31527)
This commit updates the necessary config files to run the angular.io and
docs tooling unit tests in random order (and fixes the tests that were
failing due to their dependence on the previous ordered execution).

Besides being a good idea anyway, running tests in random order is the
new [default behavior in jasmine@3.0.0][1], so this commit is in
preparation of upgrading jasmine to the latest version.

[1]: https://github.com/jasmine/jasmine/blob/v3.0.0/release_notes/3.0.md#breaking-changes

PR Close #31527
2019-07-18 10:17:13 -07:00
George Kalpakas 8bbf481d60 refactor(docs-infra): rename properties (blacklisted --> ignored) (#29754)
PR Close #29754
2019-04-08 09:46:34 -07:00
George Kalpakas c5bba8d9f2 build(docs-infra): do not include `announcements.json` in sitemap (#29754)
The `announcements.json` file should not be included in the sitemap and
including it causes an error in Google Search Console (because the
generated URL does not exist).

(This is a follow-up to fbef94a8e.)

PR Close #29754
2019-04-08 09:46:33 -07:00
Pete Bacon Darwin a4f3f3f81d build(docs-infra): support doc aliases via `@alias` dgeni tag (#29673)
Now, one can add an `@alias` tag to API docs, which tells dgeni that this
API element (usually a `const`) is really just an alias for some API element
defined elsewhere.

Dgeni will then look up this API element and copy over the properties from
the alias to the current doc.

For example, we would like to privately export an Enum from `@angular/core`
but then publicly export this from `@angular/common`:

**packages/core/private_exports.ts**

```ts
/**
 * Description of this document.
 */
export enum ɵSomeEnum { ... }
```

**packages/common/public_api.ts**

```ts
import {ɵSomeEnum} from '@angular/core';

 /**
 * @alias core/ɵSomeEnum
 */
export const SomeEnum = ɵSomeEnum;
```

In the generated docs there will be a page for `common/SomeEnum`, which
will be rendered as an enum, rather than a const, showing the description
extracted from the `core/ɵSomeEnum`.

---

The implementation of this feature required some refactoring of the other
processing:

1. Previously `ɵ` prefixed exports were not even considered.
2. Due to 1. some processors needed to have guards added to ignore such
   private exports (`addMetadataAliases` and `checkContentRules`).
3. The processing of package pages had to be reworked (and split) so that
   it picked up the aliased export docs after their alias proeprties had
   been copied.

See FW-1207, FW-632, #29249

PR Close #29673
2019-04-04 10:52:36 -07:00
Pete Bacon Darwin 7c730fe5b3 build(docs-infra): include inherited members in search index (#26676)
Closes #23800

PR Close #26676
2018-10-26 13:16:33 -04:00
Pete Bacon Darwin f233131974 build(docs-infra): refactor generateKeywords processor (#26676)
PR Close #26676
2018-10-26 13:16:32 -04:00
Pete Bacon Darwin 026b60cd70 build(docs-infra): expose deprecated status on items more clearly (#25750)
PR Close #25750
2018-09-21 10:26:48 -07:00
Pete Bacon Darwin 96f9f03d25 build(docs-infra): improve search quality (#25750)
PR Close #25750
2018-09-21 10:26:48 -07:00
Pete Bacon Darwin 293ec78069 build(aio): don't constrain `checkContentRules` to run before another processor (#24000)
We don't really care when this processor runs as long as it happens
after the tags have been extracted.
By not constraining its `runBefore` property we can ensure that other
processors can be run before it.

PR Close #24000
2018-06-13 16:47:40 -07:00
Pete Bacon Darwin 700e55ce14 build(docs-infra): log warning rather than error if content errors are not fatal (#24320)
PR Close #24320
2018-06-06 10:25:04 -07:00
Pete Bacon Darwin e0ae74d40e build(aio): add checkContentRules processor (#22759)
This processor will enable us to write rules about
how the content should appear, such as:

* no headings in markdown content
* only one sentence per line
* no single character parameter names
* etc.

PR Close #22759
2018-04-12 00:06:49 -07:00
Pete Bacon Darwin 6f0dad1710 build(aio): render doc-gen issues in overview dump (#22675)
Related to #22138

PR Close #22675
2018-03-14 10:20:30 -07:00
Pete Bacon Darwin 71ea931df5 build(aio): blacklist unwanted URLs from the generated sitemap.xml (#22061)
Closes #22017

PR Close #22061
2018-02-07 12:02:01 -08:00
Pete Bacon Darwin 9a0700f5bd build(aio): add API static members to search index (#21988)
Previously searching for `compose` did not include `Validators`
in the search results because we were not including all the
`static` members of API docs in the index.

PR Close #21988
2018-02-05 13:00:47 -08:00
Pete Bacon Darwin a64af40c0b build(aio): generate sitemap from the generated pages (#21689)
Closes #21684

PR Close #21689
2018-01-22 12:55:15 -08:00
Peter Bacon Darwin 9ab1f4a9c9 style(aio): fix docs linting issues (#19702)
These issues appeared after upgrade of eslint jasmine plugin

PR Close #19702
2017-11-01 15:24:46 -07:00
Peter Bacon Darwin 8b571309ed build(aio): append information about links in and out of docs (#19583)
Closes #19560

PR Close #19583
2017-10-10 17:06:51 -07:00
Peter Bacon Darwin e8bbf86e66 build(aio): add terms from heading to the search index 2017-07-11 15:04:37 +01:00
Peter Bacon Darwin dfbbbb5e3e build(aio): the the captured h1 as the title for the search index
If there is no title already provided, use the one captured from the renderedContent.
2017-05-31 08:45:31 +03:00
Peter Bacon Darwin 89f317915d build(aio): use the captured h1 as the title if necessary
If the doc does not already have a `title` tag
then use the one captured from the renderedContent in
the final JSON output, instead.
2017-05-31 08:45:31 +03:00
Peter Bacon Darwin 978376a46e build(aio): `doc.searchTitle` can override name in search results 2017-05-04 13:45:04 +01:00
Peter Bacon Darwin ca17d4f639 build(aio): auto-fill width/height to all image tags
Parse all `<img>` tags, during doc-gen, and insert the width and height of
the sourceed image, if neither are already specified.

Warnings are reported if the `<img>` tag has no `src` attribute or the image
cannot be loaded.

The work is done in the `addImageDimensions` post-processor, which must be
configured with a `basePath` so that it knows where to find the images.

Closes #15888
2017-05-01 15:52:51 -07:00
Peter Bacon Darwin 9945ce2259 build(aio): move copyContentAssets processor to the base package
This allows other processors who need to know about the copyContentAssets
processors to ensure that the runs after the content has been copied.
2017-05-01 15:52:51 -07:00
Peter Bacon Darwin 3cad5da5a4 build(aio): refactor dgeni packages
This is to tidy up the `author-packagse`, which currently duplicates a
lot of the configuration in the main packages. We need to
DRY this up so that we don't fall foul of a change in one being missed in
the other.
2017-04-23 22:50:33 +01:00