For API golden tests not running against a NPM package, we extract
all transitive declarations of the specified `data` targets. This is
necessary because API extractor needs to resolve other targets that have
been linked by the Bazel NodeJS rules. The linker by default only
provides access to JavaScript sources, but the API extractor is
specifically concerned with type definitions that we need to manually
extract.
PR Close#42828
The API golden test tool should not include all types
from the `node_modules/`. This results in unnecessary
type resolution when the API golden tool is run outside
of sandbox (i.e. on windows or with `bazel run` for accept).
PR Close#42828
Adds better caching for browser archives and their extraction.
This is done because the archives are currently extracted as a build
action and these are actions are invalidated frequently, causing
flakiness on the CI and slow-down in local development.
Here is an example flaky error on the CI (that surfaces often
with RBE execution):
```
ERROR:
/home/circleci/.cache/bazel/_bazel_circleci/9ce5c2144ecf75d11717c0aa41e45a8d/external/npm/@angular/dev-infra-private/bazel/browsers/chromium/BUILD.bazel:22:17:
Extracting ../org_chromium_chromium_amd64/file/chrome-linux.zip failed:
(Exit 34): extract.sh failed: error executing command
external/io_bazel_rules_webtesting/web/internal/extract.sh
external/org_chromium_chromium_amd64/file/chrome-linux.zip ...
(remaining 2 argument(s) skipped). Note: Remote connection/protocol
failed with: execution failed
```
We fix this by introducing a new rule that downloads a browser
archive and unpacks it directly into a Bazel repository. Before
this change, the archive would just be downloaded but extracted
later as part of a build action. This is unnecessary and results
in less efficient caching as build actions are invalidated more
often, especially if developers run `bazel clean` in between.
The root cause on why the extraction often fails in RBE containers
is unclear. It's unclear why the extacted archive is not cached
properly as part of a build action (most likely some hermeticity
issue within `rules_webtesting`, but it seems more Bazel-idiomatic
to unpack the archives as part of the repository anyway, and this solves
the flakiness issue.
PR Close#42814
In combination with the TS `noImplicitOverride` compatibility changes,
we also want to follow the best-practice of adding `override` to
members which are implemented as part of abstract classes. This
commit fixes all instances which will be flagged as part of the
custom `no-implicit-override-abstract` TSLint rule.
PR Close#42512
TypeScript introduced a new flag called `noImplicitOverride` as part
of TypeScript v4.3. This flag introduces a new keyword called `override`
that can be applied to members which override declarations from a base
class. This helps with code health as TS will report an error if e.g.
the base class changes the method name but the override would still
have the old method name. Similarly, if the base class removes the method
completely, TS would complain that the memeber with `override` no longer
overrides any method.
A similar concept applies to abstract methods, with the exception that
TypeScript's builtin `noImplicitOverride` option does not flag members
which are implemented as part of an abstract class. We want to enforce
this as a best-practice in the repository as adding `override` to such
implemented members will cause TS to complain if an abstract member is
removed, but still implemented by derived classes.
More details: https://github.com/microsoft/TypeScript/issues/44457.
PR Close#42512
Introduces a rule that collects declared JavaScript module output files
from a list of dependencies based on a configurable JavaScript module
provider. The extracted outputs are exposed within the `DefaultInfo` provider.
Targets defined using this rule can be used as input for rules that
require JavaScript sources, or if there are multiple JavaScript output
variants defined for a target while for example only the `JSModuleInfo`
outputs are of interest.
As an example: This rule is helpful in combination with `ts_library` and
`ng_module` as those rule expose multiple output flavors (which are
distinguishable by the JavaScript module providers as imported from `providers.bzl`).
i.e. these rules expose flavors for named AMD modules and ECMAScript module output.
If we want to ship a NPM package only using ECMAScript modules for
example, we could extract all `JSEcmaScriptModuleInfo`-denoted output
and feed that into the `pkg_npm` rule, compared to bringing in all
output flavors.
For reference:
https://github.com/bazelbuild/rules_nodejs/blob/stable/packages/typescript/internal/build_defs.bzl#L334-L337
PR Close#42809
Updates the Bazel NodeJS rules to v4.0.0-beta.0. This is necessary
so that the Angular components repo can update, and it's generally
good to stay as up-to-date as possible with the Bazel rules as it's
easy to fall behind, and updating early allows us to discover issues
affecting our tooling earlier (where they are easier to address due to
e.g. potential breaking change policy).
PR Close#42760
Rollup just prints a warning if an import cannot be resolved and ends up
being treated as an external dependency. This in combination with the
`silent = True` attribute for `rollup_bundle` means that bundles might
end up being extremely small without people noticing that it misses
actual imports.
To improve this situation, the warning is replaced by an error if
an import cannot be resolved.
This unveiles an issue with the `ng_rollup_bundle` macro from
dev-infra where imports in View Engine were not resolved but ended
up being treated as external. This did not prevent benchmarks using
this macro from working because the ConcatJS devserver had builtin
resolution for workspace manifest paths. Though given the new check
for no unresolved imports, this will now cause errors within Rollup, and
we need to fix the resolution. We can fix the issue by temporarily
enabling workspace linking. This does not have any performance
downsides.
To enable workspace linking (which we might need more often in the
future given the linker taking over patched module resolution), we
had to rename the `angular` dependency to a more specific one so
that the Angular linker could link into `node_modules/angular`.
PR Close#42760
Currently if a top-level target or file within `//dev-infra` is
being referenced. like `//dev-infra:tsconfig`, then the label
is not subsituted properly and breaks consumption of the built
NPM package.
PR Close#42754
Tools that are shipped as a Bazel rule with the shared dev-infra
tool require a specific `tsconfig` as otherwise `ts_library` will
accidentally look for a tsconfig in `@npm//:tsconfig` and the build
will fail. We bring in our dev-infra tsconfig and reference it
explicitly.
PR Close#42737
Previously, the commit SHAs and PR numbers referenced in the generated
`CHANGELOG.md` were not automatically converted to links in the GitHub
UI (as happens for release notes and issue/PR comments). This made it
less straight-forward for someone reading the changelog to get to the
commit/PR corresponding to a change.
This commit updates the tooling that generates the changelog to convert
the commit SHA and the corresponding PR number (referenced at the end of
the commit message header) to links.
PR Close#42732
For the last years the Angular repositories relied on `ts-api-guardian`
for testing the public API signature. This project worked well in
general but its another inconvenience to maintain if we could rely on
Microsoft's `api-extractor` tool.
Especially since with TypeScript 4.3 issues with export aliases appeared
that would require us to extend TS API guardian to support such exports.
This is not as straightforward as it sounds, given it requires rewriting
of declarations to show-case the proper name in the API golden. Microsoft's
API extractor has integrated support for this.
As of TypeScript 4.3, we want to start using the new `override` keyword.
We are not able to use that keyword currently because an old version of
API extractor is used in the `ng_module` rule to flatten the types into
a single file. To fix this, we need to update `api-extractor`, but this
unveils the issue with TS API guardian because the most recent version
of api-extractor uses alias exports to avoid potential conflicts
with globals available through the TypeScript default libraries (e.g.
`dom.d.ts`).
PR Close#42688
We rely on a Github API `/branches` request to determine the active
release trains. Currently this logic is broken if more than 100
protected branches exist within a repository. This issue surfaced
recently where the `items_per_page` setting was set to `30`, causing
the merge tooling and release tooling to not detect the proper "latest"
release train.
This commit uses Github pagination for retrieving branches to determine
the active release trains, and makes the logic more long-term proof.
PR Close#42666
We previously held off with updating Octokit to v18 due to
their more noticable issues with typings. This commit updates
us to the latest version in order to take advantage of the new
pagination API (which is also strongly-typed), and to not fall
behind too much over time (Octokit seems to change quite often..)
We work around the problem with the types for `getContent` by just
using a type cast with a TODO (and link to the issue). Similarly we
work around a problem where the Octokit types have an incorrect type
for the name of the labels array in an API response.
PR Close#42666
Currently the active release trains are printed when a developer
runs `ng-dev publish release`. This is not ideal because it requires
the developer to provide an OAuth token, to be on the next branch, and
to have no uncommitted changes, while the actual release train
information is not dependent on these checks.
This commit introduces a new command called `ng-dev release info` that
can be used to retrieve relase information without the aforementioned
requirements. Note that this command provides more detailed information
about release branches than the `ng-dev caretaker check` command (which
also requires on authentication as a side note). The `release info`
command also prints active LTS branches for example.
PR Close#42644
This commit fixes an issue with the ng-dev tool, where Github's API returns
paginated branch data. Only 30 branches are returned by default, and Angular
now has more than 30 branches in its repo. This commit increases the number
of branches returned to the API limit of 100, which should buy us some time
until we can implement proper pagination.
PR Close#42658
Update the pullapprove verification tooling to ensure a reviewer is defined for
each group. This is being done in preparation for the upcoming change to how
pullapprove billing works. The new billing will work on a seats based approach
rather than flat usage.
PR Close#42614
With the recent refactorings to `GitClient`, where singletons
are created and can be retrieved through a static method, the
test has been updated to also install spies for the static methods
of `GitClient`. This commit updates the spy installation so that
the same mock git client is used that is also passed manually to
the release actions. Having two separate instances of the mock
git client could result in false-positive test results.
PR Close#42468
Currently the `GitClient` accepts a generic parameter for determining
whether the `githubToken` should be set or not. This worked fine so far
in terms of distinguishing between an authenticated and
non-authenticated git client instance, but if we intend to conditionally
show methods only for authenticated instances, the generic parameter
is not suitable.
This commit splits up the `GitClient` into two classes. One for
the base logic without any authorization, and a second class that
extends the base logic with authentication logic. i.e. the
`AuthenticatedGitClient`. This allows us to have specific methods only
for the authenticated instance. e.g.
* `hasOauthScopes` has been moved to only exist for authenticated
instances.
* the GraphQL functionality within `gitClient.github` is not
accessible for non-authenticated instances. GraphQL API requires
authentication as per Github.
The initial motiviation for this was that we want to throw if
`hasOAuthScopes` is called without the Octokit instance having
a token configured. This should help avoiding issues as within
3b434ed94d
that prevented the caretaker process momentarily.
Additionally, the Git client has moved from `index.ts` to
`git-client.ts` for better discoverability in the codebase.
PR Close#42468
In the dev-infra merge tool, a test fails locally if it runs on
a system with a different system locale. We can either make the
test more flexible, or update the merge tool to always print the date
in `en-US` format. The latter is more reasonable given our console
messages being in english anyway.
PR Close#42470
This change moves the `dev-infra/browsers` folder into `dev-infra/bazel`.
The browser folder is providing custom configuration for Bazel, so it
should live within the `bazel` folder for a more well-structured
`dev-infra` folder.
PR Close#42268
Removes the duplicate `hasLocalChanges` method from the Git client. We
already have `hasUncommittedChanges`. Also removes a TODO for adding
`assertNoLocalChanges` as it seems more flexible to manually check
(i.e. better messaging with context on the current tool; e.g. "cannot
perform rebase")
PR Close#42409
As per SemVer, deprecations should only be released if the
minor version is incremented. This means that we should error
if commits w/ deprecations target an upcoming patch, LTS release.
Additionally, we also error if deprecations are made as part of
an upcoming release-candidate.
PR Close#42454
Adds the JSDoc `@throws` annotation to functions that would throw
within in the ng-dev tool. We want to add `@throws` so clearly
communicate if the invocation of a function results in errors or not.
This helps if IDEs show the `@throws` annotation on invocation, or
if TS ever lands a feature like Java `throws`, with more fine-grained
and typed error handling.
PR Close#42454
If a commit message currently mentions the breaking change or
deprecation note keywords, the commit message parse logic
accidentally picks up the note. This could then accidentally
prevent the commit from being merged (e.g. if the commit targets
the patch branch but mentioned the `BREAKING CHANGE: ` marker).
This commit switches the commit message notes pattern to only
capture notes at the beginning of a line (also allowing accidental
whitespace). This matches with the format we describe in our
contribution guide, as well as with our commit message validation
logic that also assumes notes at the beginning of a line.
PR Close#42436
Currently the commit message validation tool from `ng-dev` validates
the `BREAKING CHANGE:` commit message notes. This commit adds a similar
check for `DEPRECATED:` commit message notes.
Additionally, the check for breaking changes is reworked slightly to
be more tolerant (i.e. if there is only a single line break after the
summary; this is acceptable as per the parser and commonly done in the
COMP repo). The checks have been updated to capture wrong keywords that
are commonly used instead of the correct one. e.g. if a commit message
uses `DEPRECATIONS:` instead of `DEPRECATED:`, the validation will fail.
This prevents changelog generation issues where breaking change notes,
or deprecations are missing. This happened in the COMP repo where
the `DEPRECATED:` keyword was used incorrectly. See:
99391e7939
PR Close#42436
Updates the outdated build-worker.js file. This file has
been accidentally not updated when we fixed the Octokit
authentication to unblock the caretaker.
We recently updated `@octokit/rest` to a more recent version. For this
the Github client had to be refactored to account for new types of
Octokit. With the refactorings the authentication of the Github
client broke as we used an incorrect option for setting the OAuth
token. The TS transpilation did not fail because the Octokit types
support arbitrary options in the constructor. This is not solvable
on our side, so we cannot prevent similar issues in the future
unfortunately. Adding tests for authentication is not a reasonable
option either.
Add a command to build the release output without stamping for release
and link via `yarn link` the generated builds to a project provided.
PR Close#42319
Move the release notes tooling under its own directory under release rather than
within publish, in preparation to have a release note generation command for ad-hoc
release note generation.
PR Close#42225
When `ng-dev format --check` is run, the ng-dev tool prints out
all files that are out-of-date. We recently updated the format
tool to also capture `stderr` for failed files. This broke the
console message as we did not unwrap the `FormatFailure` to
their file path when printing the "ng-dev format" fix command.
PR Close#42252
Currently if formatting for a file fails due a formatter error,
the `ng-dev` tool reports that formatting failed, but no actual
error (or involved file) is printed out. This commit prints out
the failed files with their error message.
PR Close#42202
Previously, the dev-infra release tool would publish major versions
directly to the NPM `@latest` dist tag. This is correct in theory, but
rather unpractical given that we want to publish packages first as
`@next` so that other dependent Angular packages can update too,
allowing us to publish all main Angular packages (from FW, COMP
and TOOL) at the same time to `@latest` on NPM.
This involves creating a new release action for re-tagging the
previously released major as `@latest` on NPM.
PR Close#42133
Instead of passing `string` in the release tool for NPM dist tags, we
should use a union string type that limits the tags to `latest`, `next`
and anything matching `v{number}-lts`. This avoids mistakes at
compilation-level if an invalid/unknown tag would be set by a release
action.
PR Close#42133
Currently the ng-dev release tool always run `bazel clean` before
calling the configured build release function. The clean is necessary
to ensure the release output is actually built; and not restored
from previous builds which could have different bazel workspace
status variables (which provide the NPM package version).
Instead of doing this as part of the release tool, the
actual script running to build the release output should
run the `bazel clean`. The release tool does not intend to
know about details on how the release output is built. This
is necessary because the build setup could vary between version
branches (especially for older ones; such as LTS version branches).
PR Close#42101
Currently if a PR modifies any file that configures a Github action
(e.g. a workflow file), the caretaker might face an error when merging
such PR:
```
! [remote rejected] merge_pr_target_11.2.x -> 11.2.x (refusing to allow a Personal Access Token to create or update workflow
```
This happens because Github requires the token being used for the
push operation to have the `workflow` scope set. This is a special
scope added by Github to ensure that no changes can be made on
upstream branches that might expose the `GITHUB_TOKEN` environment
variable, which is available for push builds and could cause the
token being leaked.
With this commit we enforce that the caretaker adds the workflow
scope to their github token. Since PRs can only be merged if reviewed
thoroughly, it's acceptable to allow workflow file changes being
merged through the merge tool by the caretaker (especially since we
also allow CircleCI config files being merged with the default
`repo`/`public_repo` scope).
PR Close#41989
Andrew Kushnir
Joey Perrott
Paul Gschwendtner
George Kalpakas
Alan Agius
Alex Rickabaugh