Write to the unique log file, to prevent being overwritten, for `FATAL_ERROR`
failures in the release tooling. This will help to assist in determining where
something goes wrong in the process as well as being able to resume the action.
PR Close#40524
When fetch is run in normal mode, the `git-fetch-pack` plumbing command
outputs progress to stderr. Since this is unnecessary progress
information for ng-dev usages, it should be suppressed instead.
PR Close#39503
Some usages of the `GitClient` are better served by suppressing the
logging of lines that express what commands are being run. Many usages
of `GitClient` are contained within tools which are best served by
keeping the output clean as mostly read actions are occurring.
PR Close#39474
Rather than running ng-dev via ts-node, going forward ng-dev is generated and run
locally via node. Additionally, the generated file is tested on each commit to
ensure that the local generated version stays up to date.
PR Close#39089
Updates to rules_nodejs 2.2.0. This is the first major release in 7 months and includes a number of features as well
as breaking changes.
Release notes: https://github.com/bazelbuild/rules_nodejs/releases/tag/2.0.0
Features of note for angular/angular:
* stdout/stderr/exit code capture; this could be potentially be useful
* TypeScript (ts_project); a simpler tsc rule that ts_library that can be used in the repo where ts_library is too
heavy weight
Breaking changes of note for angular/angular:
* loading custom rules from npm packages: `ts_library` is no longer loaded from `@npm_bazel_typescript//:index.bzl`
(which no longer exists) but is now loaded from `@npm//@bazel/typescript:index.bzl`
* with the loading changes above, `load("@npm//:install_bazel_dependencies.bzl", "install_bazel_dependencies")` is
no longer needed in the WORKSPACE which also means that yarn_install does not need to run unless building/testing
a target that depends on @npm. In angular/angular this is a minor improvement as almost everything depends on @npm.
* @angular/bazel package is also updated in this PR to support the new load location; Angular + Bazel users that
require it for ng_package (ng_module is no longer needed in OSS with Angular 10) will need to load from
`@npm//@angular/bazel:index.bzl`. I investigated if it was possible to maintain backward compatability for the old
load location `@npm_angular_bazel` but it is not since the package itself needs to be updated to load from
`@npm//@bazel/typescript:index.bzl` instead of `@npm_bazel_typescript//:index.bzl` as it depends on ts_library
internals for ng_module.
* runfiles.resolve will now throw instead of returning undefined to match behavior of node require
Other changes in angular/angular:
* integration/bazel has been updated to use both ng_module and ts_libary with use_angular_plugin=true.
The latter is the recommended way for rules_nodejs users to compile Angular 10 with Ivy. Bazel + Angular ViewEngine is
supported with @angular/bazel <= 9.0.5 and Angular <= 8. There is still Angular ViewEngine example on rules_nodejs
https://github.com/bazelbuild/rules_nodejs/tree/stable/examples/angular_view_engine on these older versions but users
that want to update to Angular 10 and are on Bazel must switch to Ivy and at that point ts_library with
use_angular_plugin=true is more performant that ng_module. Angular example in rules_nodejs is configured this way
as well: https://github.com/bazelbuild/rules_nodejs/tree/stable/examples/angular. As an aside, we also have an
example of building Angular 10 with architect() rule directly instead of using ts_library with angular plugin:
https://github.com/bazelbuild/rules_nodejs/tree/stable/examples/angular_bazel_architect.
NB: ng_module is still required for angular/angular repository as it still builds ViewEngine & @angular/bazel
also provides the ng_package rule. ng_module can be removed in the future if ViewEngine is no longer needed in
angular repo.
* JSModuleInfo provider added to ng_module. this is for forward compat for future rules_nodejs versions.
PR Close#39182
Updates to rules_nodejs 2.2.0. This is the first major release in 7 months and includes a number of features as well
as breaking changes.
Release notes: https://github.com/bazelbuild/rules_nodejs/releases/tag/2.0.0
Features of note for angular/angular:
* stdout/stderr/exit code capture; this could be potentially be useful
* TypeScript (ts_project); a simpler tsc rule that ts_library that can be used in the repo where ts_library is too
heavy weight
Breaking changes of note for angular/angular:
* loading custom rules from npm packages: `ts_library` is no longer loaded from `@npm_bazel_typescript//:index.bzl`
(which no longer exists) but is now loaded from `@npm//@bazel/typescript:index.bzl`
* with the loading changes above, `load("@npm//:install_bazel_dependencies.bzl", "install_bazel_dependencies")` is
no longer needed in the WORKSPACE which also means that yarn_install does not need to run unless building/testing
a target that depends on @npm. In angular/angular this is a minor improvement as almost everything depends on @npm.
* @angular/bazel package is also updated in this PR to support the new load location; Angular + Bazel users that
require it for ng_package (ng_module is no longer needed in OSS with Angular 10) will need to load from
`@npm//@angular/bazel:index.bzl`. I investigated if it was possible to maintain backward compatability for the old
load location `@npm_angular_bazel` but it is not since the package itself needs to be updated to load from
`@npm//@bazel/typescript:index.bzl` instead of `@npm_bazel_typescript//:index.bzl` as it depends on ts_library
internals for ng_module.
* runfiles.resolve will now throw instead of returning undefined to match behavior of node require
Other changes in angular/angular:
* integration/bazel has been updated to use both ng_module and ts_libary with use_angular_plugin=true.
The latter is the recommended way for rules_nodejs users to compile Angular 10 with Ivy. Bazel + Angular ViewEngine is
supported with @angular/bazel <= 9.0.5 and Angular <= 8. There is still Angular ViewEngine example on rules_nodejs
https://github.com/bazelbuild/rules_nodejs/tree/stable/examples/angular_view_engine on these older versions but users
that want to update to Angular 10 and are on Bazel must switch to Ivy and at that point ts_library with
use_angular_plugin=true is more performant that ng_module. Angular example in rules_nodejs is configured this way
as well: https://github.com/bazelbuild/rules_nodejs/tree/stable/examples/angular. As an aside, we also have an
example of building Angular 10 with architect() rule directly instead of using ts_library with angular plugin:
https://github.com/bazelbuild/rules_nodejs/tree/stable/examples/angular_bazel_architect.
NB: ng_module is still required for angular/angular repository as it still builds ViewEngine & @angular/bazel
also provides the ng_package rule. ng_module can be removed in the future if ViewEngine is no longer needed in
angular repo.
* JSModuleInfo provider added to ng_module. this is for forward compat for future rules_nodejs versions.
@josephperrott, this touches `packages/bazel/src/external.bzl` which will make the sync to g3 non-trivial.
PR Close#37727
Creates a tool for staging and publishing releases as per the
new branching and versioning that has been outlined in the following
document. The tool is intended to be used across the organization to
ensure consistent branching/versioning and labeling:
https://docs.google.com/document/d/197kVillDwx-RZtSVOBtPb4BBIAw0E9RT3q3v6DZkykU/edit#heading=h.s3qlps8f4zq7dd
The tool implements the actions as outlined in the following
initial plan: https://hackmd.io/2Le8leq0S6G_R5VEVTNK9A.
The implementation slightly diverged in so far that it performs
staging and publishing together so that releasing is a single
convenient command. In case of errors for which re-running the
full command is not sufficient, we want to consider adding
recover functionality. e.g. when the staging completed, but the
actual NPM publishing aborted unexpectedly due to build errors.
PR Close#38656
Introduces a new command for `ng-dev release`, so that the NPM
dist tag can be set for all configured NPM packages. This command
can be useful in case a manual tag needs to be set, but it is
primarily used by the release tooling when a new stable version
is cut, and when the previous patch branch needs to be set as LTS
version through a `v{major}-lts` dist tag.
It is necessary to have this as a command so that the release tool
can execute it for old branches where other packages might have been
configured. This is similar to the separate `ng-dev build` command
that we created.
Note that we also added logic for spawning a process conveniently
with different "console output" modes. This will be useful for
other command invocations in the release tool and it's generally
better than directly using native `child_process` as that one doesn't
log to the dev-infra debug log file.
PR Close#38656
Adds a new folder to dev-infra where shared testing utilities
could be placed in. This commit already adds initial testing
utilities for dealing with the `GitClient` and SemVer versions.
The `GitClient` in the testing utilities simulates actual Git
behavior in a virtual manner. It's not complete at all, but can
be extended based on our needs. The currently implemented commands
are the most basic ones that we'd need for our release tooling.
PR Close#38656
Adds a method for printing active release trains for a configured
project. This is helpful for the release tool that will print
the active release trains. Also this can be useful for the
caretaker status command, where we could print the active
version branches (i.e. "is there currently a feature-freeze branch").
PR Close#38656
Cleans up outdated comments in the shared dev-infra Git
utilities. We also export the Graphql client for consistency
as we expose the `GithubClient` and `GitClient` too.
PR Close#38656
Instead of maintaining multiple interface for grouping
owner name and repo name, we expose a shared interface
describing a Github repository.
One unfortunate downside is that the GraphQL Github
and Rest API diverge slightly with the key for the
repository name. i.e. rest uses `repo` for the name
of a repository, while GraphQL uses `name` for the name.
If that would be consistent, we could use the rest operator
to pass a repository to the Octokit REST or GraphQL API. This
does not work, so we have a small manual overhead as seen
in the `branches.ts` file.
PR Close#38656
The git client respects the `SpawnSyncOptions` when a command
is executed. Currently it does not hide the command info
messages when commands are run in silent mode.
We fix this as part of this commit, so that the command info
is only printed to `debug` if `stdio` is set to `ignore`.
Additonally, the github token is made public so that it can be
used by commands if other repositories like forks are targeted.
PR Close#38656
Instead of repeating the logic for adding the github token to
a repository git url, we add a shared function for automatically
computing the URls with token.
Additionally, URLs for updating/generating tokens have been moved
to a dedicated file in the `utils` folder. Also while being at it,
the yargs github token helper is also moved into the dedicated
Git/Github related util folder.
PR Close#38656
Currently we validate the configuration file on each `getConfig`
invocation. We can only validate once since the configuration
is cached.
Also while being at it, renames the cache variables to lower-case as those
do not represent constants (which are convention-wise upper case).
PR Close#38808
As not all users, particularly contributors consistently contributing with a deep
understanding of our commit message guidelines, will not want to rely on the
commit message wizard, we allow a user to opt out of using this wizard during
commit message creation.
PR Close#38701
Create a utility for loading a local user configuration object to describe
local configuration values, such as skipping the commit message wizard.
PR Close#38701
The angular team relies on a number of services for hosting code, running CI, etc. This
tool allows for checking the operational status of all services at once as well as the current
state of the repository with respect to merge and triage ready issues and prs.
PR Close#38601
Creates a mixin for requiring a github token to be provided to a command. This mixin
allows for a centralized management of the requirement and handling of the github-token.
PR Close#38630
Creates infrastructure to write outputs of command runs to ng-dev log file.
Additionally, on commands which fail with an exit code greater than 1, an
error log file is created with the posix timestamp of the commands run time
as an identifier.
PR Close#38599
Creates a tool within ng-dev to checkout a pending PR from the upstream repository. This automates
an action that many developers on the Angular team need to do periodically in the process of testing
and reviewing incoming PRs.
Example usage:
ng-dev pr checkout <pr-number>
PR Close#38474
Adds an ng-dev formatter option to format all of the staged files. This will can
be used to format only the staged files during the pre-commit hook.
PR Close#38402
The merge script currently accepts a configuration function that will
be invoked _only_ when the `ng-dev merge` command is executed. This
has been done that way because the merge tooling usually relies on
external requests to Git or NPM for constructing the branch configurations.
We do not want to perform these slow external queries on any `ng-dev` command
though, so this became a lazily invoked function.
This commit adds support for these configuration functions to run
asynchronously (by returning a Promise that will be awaited), so that
requests could also be made to the Github API. This is benefical as it
could avoid dependence on the local Git state and the HTTP requests
are more powerful/faster.
Additionally, in order to be able to perform Github API requests
with an authenticated instance, the merge tool will pass through
a `GithubClient` instance that uses the specified `--github-token`
(or from the environment). This ensures that all API requests use
the same `GithubClient` instance and can be authenticated (mitigating
potential rate limits).
PR Close#38223
Conditions can refer to the groups array that is a list of the preceding groups.
This commit adds support to the verification for those conditions.
This commit also adds some tests to the parsing and condition matching
to ensure everything works as expected.
PR Close#38164
We recently added OAuth scope checking to the dev-infra Git client
and started leveraging it for the merge script. We set the `repo` scope
as required for running the merge script. We can loosen this requirement
as in the Angular org where the script is consumed, only pull requests on
public repositories are merged through the script.
This should help with reducing the risk with compromised tokens as no
access had to be granted on `repo:invite`, `repo_deployment` etc.
PR Close#37718
Scripts provided in the `ng-dev` command might use local `git`
commands. For such scripts, we keep track of the branch that
has been checked out before the command has been invoked.
We do this so that we can later (upon command completion)
restore back to the original branch. We do not want to
leave the Git repository in a dirty state.
It looks like this logic currently only deals with branches
but does not work properly when a command is invoked from
a detached head. We can make it work by just checking out
the previous revision (if no branch is checked out).
PR Close#37737
GitClient now uses GithubClient for github API interactions. GithubClient is
a class which extends Octokit and provides a member which allows for GraphQL
requests against the Github GraphQL api, as well as providing convenience methods
for common/repeated Github API requests.
PR Close#37593
Looks like we broke the `hasLocalChanges` check in the git client
when we moved it over from the merge script. The problem is that
we are using `git` in the first argument of `git.run`. That
means that we under-the-hood run `git git <..>`.
This commit fixes that, but also switches to a better variant
for ensuring no local changes because it exits with non-zero
when there are local changes.
PR Close#37489
We recently moved over the git client from the merge script to the
common dev-infra utils. This made specifying a token optional, but
it looks like the logic for sanitizing messages doesn't account
for that, and we currently add `<TOKEN>` between every message
character. e.g.
```
Executing: git <TOKEN>g<TOKEN>i<TOKEN>t<TOKEN>
<TOKEN>s<TOKEN>t<TOKEN>a<TOKEN>t<TOKEN>u<TOKEN>s<TOKEN>
```
PR Close#37489
There is an `exec()` helper provided by `utils/shelljs.ts`, which is a
wrapper around ShellJS' `exec()` with some default options (currently
`silent: true`). The intention is to avoid having to pass these options
to every invocation of the `exec()` function.
This commit updates all code inside `dev-infra/` to use this helper
whenever possible).
NOTE: For simplicity, the `utils/shelljs` helper does not support some
of the less common call signatures of the original `exec()`
helper, so in some cases we still need to use the original.
PR Close#37444
Migrate to using .ng-dev directory for ng-dev configuration to better
allow management of the configuration using multiple files. The
intention is to prevent the config file from becoming unruly.
PR Close#37142
Adds .group and .groupEnd functions to each of the logging functions
to allow creating groups in the logged output. Additionally removes
the color parameter from logging functions, in favor of the color
being applied to the string at the call site.
PR Close#37232
We recently added support for automatic registration of `ts-node`
when the dev-infra configuration is loaded.
In addition to registering ts-node, we should also ensure that the
`commonjs` module is set up. By default, `ts-node` would use ES module
imports that are not supported by default in NodeJS.
PR Close#37217
Creates common logging functions at different levels. Allows for providing
logging statements which are actually printed to the console based on the
LOG_LEVEL environment variable.
PR Close#37192
`ts-node` is now an optional peer dependency of the shared dev-infra
package. Whenever a `ng-dev` command runs, and a TypeScript-based
configuration file exists, `ts-node` is set up if available.
That allows consumers of the package (as the components repo) to more
conveniently use a TypeScript-based configuration for dev-infra.
Currently, commands would need to be proxied through `ts-node`
which rather complicates the setup:
```
NG_DEV_COMMAND="ts-node ./node_modules/@angular/dev-infra-private/cli.js"
```
I'm thinking that it should be best-practice to use TypeScript for
writing the configuration files. Given that the tool is used primarily
in Angular projects (for which most sources are TypeScript), this should
be acceptable.
PR Close#37196
Creates a tool in ng-dev which rebases a PR automatically and pushes
the rebase commits back to the PR. This is meant to be a replacement
to the local merge script currently in the repo and currently has
feature parity.
PR Close#37055
Integrates the merge script into the `ng-dev` CLI. The goal is that
caretakers can run the same command across repositories to merge a pull
request. The command is as followed: `yarn ng-dev pr merge <number>`.
PR Close#37138
Joey Perrott
Alexey Elin
George Kalpakas
Greg Magolan
Paul Gschwendtner
Ikko Ashimine
Andrew Scott