We rely on a Github API `/branches` request to determine the active
release trains. Currently this logic is broken if more than 100
protected branches exist within a repository. This issue surfaced
recently where the `items_per_page` setting was set to `30`, causing
the merge tooling and release tooling to not detect the proper "latest"
release train.
This commit uses Github pagination for retrieving branches to determine
the active release trains, and makes the logic more long-term proof.
PR Close#42666
We previously held off with updating Octokit to v18 due to
their more noticable issues with typings. This commit updates
us to the latest version in order to take advantage of the new
pagination API (which is also strongly-typed), and to not fall
behind too much over time (Octokit seems to change quite often..)
We work around the problem with the types for `getContent` by just
using a type cast with a TODO (and link to the issue). Similarly we
work around a problem where the Octokit types have an incorrect type
for the name of the labels array in an API response.
PR Close#42666
Currently the `GitClient` accepts a generic parameter for determining
whether the `githubToken` should be set or not. This worked fine so far
in terms of distinguishing between an authenticated and
non-authenticated git client instance, but if we intend to conditionally
show methods only for authenticated instances, the generic parameter
is not suitable.
This commit splits up the `GitClient` into two classes. One for
the base logic without any authorization, and a second class that
extends the base logic with authentication logic. i.e. the
`AuthenticatedGitClient`. This allows us to have specific methods only
for the authenticated instance. e.g.
* `hasOauthScopes` has been moved to only exist for authenticated
instances.
* the GraphQL functionality within `gitClient.github` is not
accessible for non-authenticated instances. GraphQL API requires
authentication as per Github.
The initial motiviation for this was that we want to throw if
`hasOAuthScopes` is called without the Octokit instance having
a token configured. This should help avoiding issues as within
3b434ed94d
that prevented the caretaker process momentarily.
Additionally, the Git client has moved from `index.ts` to
`git-client.ts` for better discoverability in the codebase.
PR Close#42468
In the dev-infra merge tool, a test fails locally if it runs on
a system with a different system locale. We can either make the
test more flexible, or update the merge tool to always print the date
in `en-US` format. The latter is more reasonable given our console
messages being in english anyway.
PR Close#42470
Removes the duplicate `hasLocalChanges` method from the Git client. We
already have `hasUncommittedChanges`. Also removes a TODO for adding
`assertNoLocalChanges` as it seems more flexible to manually check
(i.e. better messaging with context on the current tool; e.g. "cannot
perform rebase")
PR Close#42409
As per SemVer, deprecations should only be released if the
minor version is incremented. This means that we should error
if commits w/ deprecations target an upcoming patch, LTS release.
Additionally, we also error if deprecations are made as part of
an upcoming release-candidate.
PR Close#42454
Adds the JSDoc `@throws` annotation to functions that would throw
within in the ng-dev tool. We want to add `@throws` so clearly
communicate if the invocation of a function results in errors or not.
This helps if IDEs show the `@throws` annotation on invocation, or
if TS ever lands a feature like Java `throws`, with more fine-grained
and typed error handling.
PR Close#42454
Currently if a PR modifies any file that configures a Github action
(e.g. a workflow file), the caretaker might face an error when merging
such PR:
```
! [remote rejected] merge_pr_target_11.2.x -> 11.2.x (refusing to allow a Personal Access Token to create or update workflow
```
This happens because Github requires the token being used for the
push operation to have the `workflow` scope set. This is a special
scope added by Github to ensure that no changes can be made on
upstream branches that might expose the `GITHUB_TOKEN` environment
variable, which is available for push builds and could cause the
token being leaked.
With this commit we enforce that the caretaker adds the workflow
scope to their github token. Since PRs can only be merged if reviewed
thoroughly, it's acceptable to allow workflow file changes being
merged through the merge tool by the caretaker (especially since we
also allow CircleCI config files being merged with the default
`repo`/`public_repo` scope).
PR Close#41989
As `getRepoBaseDir()` relies on git, it should be a method on `GitClient` for retrieval
rather than its own utility outside of the common GitClient used for all git ineractions.
PR Close#41527
With this change we update several dependencies to avoid Renovate creating a lot of PRs during onboarding. We also remove yarn workspaces as after further analysis these are not needed.
Certain dependencies such as `@octokit/rest`, `remark` and `@babel/*` have not been updated as they require a decent amount of work to update, and it's best to leave them for a seperate PR.
PR Close#41434
Creates a singleton class for GitClient rather than relying on creating an instance to
require being passed around throughout its usages.
PR Close#41515
During merging with `ng-dev pr merge` tooling will ensure that pull requests are
properly labeled for breaking changes. Pull requests with commits noting breaking
changes must also be labeled as such, additionally pull requests with breaking
change labels must contain commits noting breaking changes.
Fixes#38776
PR Close#41546
After updating to use the v4 graphql api in the merge tooling, the githubTargetBranch
was set to be the sha of the latest commit rather than the branch name of the target
branch. This caused our tooling to mismatch which branches were actually being targeted
with the effect that if a PR targeted only the patch branch (i.e. labeled `target: patch`
and targeting `11.2.x` in github)it would still expect to merge into both `11.2.x` and
`master`. This is now corrected to once again use the branch name, restoring to the
previous functionality.
PR Close#41523
Add a property, `targetLabelExemptScopes`, to the merge configuration allowing certain
scopes to be exempted from requirements for features and breaking changes only included
in PRs targetting certain labels.
PR Close#41459
Check commits in pull requests to ensure the pr can be merged into the target branch. Confirms
that prs targeting minor do not contain breaking changes, and prs targeting patch or lts do not
contain breaking changes or `feat` commits.
PR Close#41459
Check a range of commits by retrieving the log files to be parsed with the expected
format for the parser.
This change is in part of a larger set of changes making the process for obtaining
and parsing commits for release note creation and message validation consistent.
This consistency will make it easier to debug as well as ease the design of tooling
which is built on top of these processes.
PR Close#41341
Use conventional-commits-parser for parsing commits for validation, this is being done
in anticipation of relying on this parser for release note creation. Unifying how commits
are parsed will provide the most consistency in our tooling.
PR Close#41286
Add support for a `--no-branch-prompt` flag for the `ng-dev pr merge` tool. This
flag enables suppression of the confirmation prompt for which branches the specified
PR will merge into.
PR Close#40528
When multiple target labels are applied to a PR, it should be considered
invalid as our tooling does not support a single PR targetting multiple
trains/versions.
PR Close#40156
With the migration to husky@5, the environment variable used to skip husky changed from
`HUSKY_SKIP_HOOKS` to `HUSKY`. To continue skipping husky during the merge process as
expected, the environment variable used is updated.
PR Close#40025
Previously, when a PR which does not target the master branch in the Github UI was
merged it would not close automatically. This change detects when this case occurs
and closes the PR via the Github API.
For example:
A PR which targets the 11.0.x branch in the Github UI has the `target: patch` label
This PR is only pushed into the 11.0.x branch, which does not trigger Github's
reference based actions to close the PR.
PR Close#39979
When fetch is run in normal mode, the `git-fetch-pack` plumbing command
outputs progress to stderr. Since this is unnecessary progress
information for ng-dev usages, it should be suppressed instead.
PR Close#39503
Rather than running ng-dev via ts-node, going forward ng-dev is generated and run
locally via node. Additionally, the generated file is tested on each commit to
ensure that the local generated version stays up to date.
PR Close#39089
Perviously, it was not immediately clear what branches a PR would merge
into during the merge process. This prompt allows for caretakers to
understand and acknowledge where the PR will merge to.
PR Close#39333
Updates to rules_nodejs 2.2.0. This is the first major release in 7 months and includes a number of features as well
as breaking changes.
Release notes: https://github.com/bazelbuild/rules_nodejs/releases/tag/2.0.0
Features of note for angular/angular:
* stdout/stderr/exit code capture; this could be potentially be useful
* TypeScript (ts_project); a simpler tsc rule that ts_library that can be used in the repo where ts_library is too
heavy weight
Breaking changes of note for angular/angular:
* loading custom rules from npm packages: `ts_library` is no longer loaded from `@npm_bazel_typescript//:index.bzl`
(which no longer exists) but is now loaded from `@npm//@bazel/typescript:index.bzl`
* with the loading changes above, `load("@npm//:install_bazel_dependencies.bzl", "install_bazel_dependencies")` is
no longer needed in the WORKSPACE which also means that yarn_install does not need to run unless building/testing
a target that depends on @npm. In angular/angular this is a minor improvement as almost everything depends on @npm.
* @angular/bazel package is also updated in this PR to support the new load location; Angular + Bazel users that
require it for ng_package (ng_module is no longer needed in OSS with Angular 10) will need to load from
`@npm//@angular/bazel:index.bzl`. I investigated if it was possible to maintain backward compatability for the old
load location `@npm_angular_bazel` but it is not since the package itself needs to be updated to load from
`@npm//@bazel/typescript:index.bzl` instead of `@npm_bazel_typescript//:index.bzl` as it depends on ts_library
internals for ng_module.
* runfiles.resolve will now throw instead of returning undefined to match behavior of node require
Other changes in angular/angular:
* integration/bazel has been updated to use both ng_module and ts_libary with use_angular_plugin=true.
The latter is the recommended way for rules_nodejs users to compile Angular 10 with Ivy. Bazel + Angular ViewEngine is
supported with @angular/bazel <= 9.0.5 and Angular <= 8. There is still Angular ViewEngine example on rules_nodejs
https://github.com/bazelbuild/rules_nodejs/tree/stable/examples/angular_view_engine on these older versions but users
that want to update to Angular 10 and are on Bazel must switch to Ivy and at that point ts_library with
use_angular_plugin=true is more performant that ng_module. Angular example in rules_nodejs is configured this way
as well: https://github.com/bazelbuild/rules_nodejs/tree/stable/examples/angular. As an aside, we also have an
example of building Angular 10 with architect() rule directly instead of using ts_library with angular plugin:
https://github.com/bazelbuild/rules_nodejs/tree/stable/examples/angular_bazel_architect.
NB: ng_module is still required for angular/angular repository as it still builds ViewEngine & @angular/bazel
also provides the ng_package rule. ng_module can be removed in the future if ViewEngine is no longer needed in
angular repo.
* JSModuleInfo provider added to ng_module. this is for forward compat for future rules_nodejs versions.
PR Close#39182
Updates to rules_nodejs 2.2.0. This is the first major release in 7 months and includes a number of features as well
as breaking changes.
Release notes: https://github.com/bazelbuild/rules_nodejs/releases/tag/2.0.0
Features of note for angular/angular:
* stdout/stderr/exit code capture; this could be potentially be useful
* TypeScript (ts_project); a simpler tsc rule that ts_library that can be used in the repo where ts_library is too
heavy weight
Breaking changes of note for angular/angular:
* loading custom rules from npm packages: `ts_library` is no longer loaded from `@npm_bazel_typescript//:index.bzl`
(which no longer exists) but is now loaded from `@npm//@bazel/typescript:index.bzl`
* with the loading changes above, `load("@npm//:install_bazel_dependencies.bzl", "install_bazel_dependencies")` is
no longer needed in the WORKSPACE which also means that yarn_install does not need to run unless building/testing
a target that depends on @npm. In angular/angular this is a minor improvement as almost everything depends on @npm.
* @angular/bazel package is also updated in this PR to support the new load location; Angular + Bazel users that
require it for ng_package (ng_module is no longer needed in OSS with Angular 10) will need to load from
`@npm//@angular/bazel:index.bzl`. I investigated if it was possible to maintain backward compatability for the old
load location `@npm_angular_bazel` but it is not since the package itself needs to be updated to load from
`@npm//@bazel/typescript:index.bzl` instead of `@npm_bazel_typescript//:index.bzl` as it depends on ts_library
internals for ng_module.
* runfiles.resolve will now throw instead of returning undefined to match behavior of node require
Other changes in angular/angular:
* integration/bazel has been updated to use both ng_module and ts_libary with use_angular_plugin=true.
The latter is the recommended way for rules_nodejs users to compile Angular 10 with Ivy. Bazel + Angular ViewEngine is
supported with @angular/bazel <= 9.0.5 and Angular <= 8. There is still Angular ViewEngine example on rules_nodejs
https://github.com/bazelbuild/rules_nodejs/tree/stable/examples/angular_view_engine on these older versions but users
that want to update to Angular 10 and are on Bazel must switch to Ivy and at that point ts_library with
use_angular_plugin=true is more performant that ng_module. Angular example in rules_nodejs is configured this way
as well: https://github.com/bazelbuild/rules_nodejs/tree/stable/examples/angular. As an aside, we also have an
example of building Angular 10 with architect() rule directly instead of using ts_library with angular plugin:
https://github.com/bazelbuild/rules_nodejs/tree/stable/examples/angular_bazel_architect.
NB: ng_module is still required for angular/angular repository as it still builds ViewEngine & @angular/bazel
also provides the ng_package rule. ng_module can be removed in the future if ViewEngine is no longer needed in
angular repo.
* JSModuleInfo provider added to ng_module. this is for forward compat for future rules_nodejs versions.
@josephperrott, this touches `packages/bazel/src/external.bzl` which will make the sync to g3 non-trivial.
PR Close#37727
We initially added logic for determining active release trains into
the merge script. Given we now build more tools that rely on this
information, we move the logic into a more general "versioning" folder
that can contain common logic following the versioning document for the
Angular organization.
PR Close#38656
Exposes logic for dealing with LTS branches, so that the release
tool can re-use it for cutting LTS patch releases.
Eventually, we can move all of this logic to a more dedicated
folder instead of having it inside the merge folder.
PR Close#38656
Instead of maintaining multiple interface for grouping
owner name and repo name, we expose a shared interface
describing a Github repository.
One unfortunate downside is that the GraphQL Github
and Rest API diverge slightly with the key for the
repository name. i.e. rest uses `repo` for the name
of a repository, while GraphQL uses `name` for the name.
If that would be consistent, we could use the rest operator
to pass a repository to the Octokit REST or GraphQL API. This
does not work, so we have a small manual overhead as seen
in the `branches.ts` file.
PR Close#38656
Previously, the logic for determing the active release trains did not
return the resolved version of a release train. With the publish script
being created, we need this information and can just pass it through,
so that we do not need to fetch and parse the package.json of given
branches multiple times.
PR Close#38656
Instead of repeating the logic for adding the github token to
a repository git url, we add a shared function for automatically
computing the URls with token.
Additionally, URLs for updating/generating tokens have been moved
to a dedicated file in the `utils` folder. Also while being at it,
the yargs github token helper is also moved into the dedicated
Git/Github related util folder.
PR Close#38656
During the merge process, all validations have already been completed so git commit
hooks can be safely skipped. This additionally, prevents errors from occuring which
would be caused the commit hooks executing, such as when yarn updates and then yarn
commands are unable to run within the same process.
PR Close#38888
Creates a mixin for requiring a github token to be provided to a command. This mixin
allows for a centralized management of the requirement and handling of the github-token.
PR Close#38630
Currently the merge script default branch configuration throws an error
if an unexpected version branch is discovered. The error right now
assumes to much knowledge of the logic and the document outlining
the release trains conceptually.
We change it to something more easy to understand that doesn't require
full understanding of the versioning/labeling/branching document that
has been created for the Angular organization.
PR Close#38622
Creates a tool within ng-dev to checkout a pending PR from the upstream repository. This automates
an action that many developers on the Angular team need to do periodically in the process of testing
and reviewing incoming PRs.
Example usage:
ng-dev pr checkout <pr-number>
PR Close#38474
Paul Gschwendtner
Joey Perrott
George Kalpakas
Alan Agius
Greg Magolan