Commit Graph

51 Commits

Author SHA1 Message Date
Greg Magolan db56cf18ba build: upgrade angular build, integration/bazel and @angular/bazel package to rule_nodejs 2.2.0 (#37727)
Updates to rules_nodejs 2.2.0. This is the first major release in 7 months and includes a number of features as well
as breaking changes.

Release notes: https://github.com/bazelbuild/rules_nodejs/releases/tag/2.0.0

Features of note for angular/angular:

* stdout/stderr/exit code capture; this could be potentially be useful

* TypeScript (ts_project); a simpler tsc rule that ts_library that can be used in the repo where ts_library is too
  heavy weight

Breaking changes of note for angular/angular:

* loading custom rules from npm packages: `ts_library` is no longer loaded from `@npm_bazel_typescript//:index.bzl`
  (which no longer exists) but is now loaded from `@npm//@bazel/typescript:index.bzl`

* with the loading changes above, `load("@npm//:install_bazel_dependencies.bzl", "install_bazel_dependencies")` is
  no longer needed in the WORKSPACE which also means that yarn_install does not need to run unless building/testing
  a target that depends on @npm. In angular/angular this is a minor improvement as almost everything depends on @npm.

* @angular/bazel package is also updated in this PR to support the new load location; Angular + Bazel users that
  require it for ng_package (ng_module is no longer needed in OSS with Angular 10) will need to load from
  `@npm//@angular/bazel:index.bzl`. I investigated if it was possible to maintain backward compatability for the old
  load location `@npm_angular_bazel` but it is not since the package itself needs to be updated to load from
  `@npm//@bazel/typescript:index.bzl` instead of `@npm_bazel_typescript//:index.bzl` as it depends on ts_library
  internals for ng_module.

* runfiles.resolve will now throw instead of returning undefined to match behavior of node require

Other changes in angular/angular:

* integration/bazel has been updated to use both ng_module and ts_libary with use_angular_plugin=true.
  The latter is the recommended way for rules_nodejs users to compile Angular 10 with Ivy. Bazel + Angular ViewEngine is
  supported with @angular/bazel <= 9.0.5 and Angular <= 8. There is still Angular ViewEngine example on rules_nodejs
  https://github.com/bazelbuild/rules_nodejs/tree/stable/examples/angular_view_engine on these older versions but users
  that want to update to Angular 10 and are on Bazel must switch to Ivy and at that point ts_library with
  use_angular_plugin=true is more performant that ng_module. Angular example in rules_nodejs is configured this way
  as well: https://github.com/bazelbuild/rules_nodejs/tree/stable/examples/angular. As an aside, we also have an
  example of building Angular 10 with architect() rule directly instead of using ts_library with angular plugin:
  https://github.com/bazelbuild/rules_nodejs/tree/stable/examples/angular_bazel_architect.

NB: ng_module is still required for angular/angular repository as it still builds ViewEngine & @angular/bazel
also provides the ng_package rule. ng_module can be removed in the future if ViewEngine is no longer needed in
angular repo.

* JSModuleInfo provider added to ng_module. this is for forward compat for future rules_nodejs versions.
  @josephperrott, this touches `packages/bazel/src/external.bzl` which will make the sync to g3 non-trivial.

PR Close #37727
2020-10-01 15:34:36 -07:00
Paul Gschwendtner f96dcc5ce0 feat(dev-infra): tool for staging and publishing releases (#38656)
Creates a tool for staging and publishing releases as per the
new branching and versioning that has been outlined in the following
document. The tool is intended to be used across the organization to
ensure consistent branching/versioning and labeling:

https://docs.google.com/document/d/197kVillDwx-RZtSVOBtPb4BBIAw0E9RT3q3v6DZkykU/edit#heading=h.s3qlps8f4zq7dd

The tool implements the actions as outlined in the following
initial plan: https://hackmd.io/2Le8leq0S6G_R5VEVTNK9A.

The implementation slightly diverged in so far that it performs
staging and publishing together so that releasing is a single
convenient command. In case of errors for which re-running the
full command is not sufficient, we want to consider adding
recover functionality. e.g. when the staging completed, but the
actual NPM publishing aborted unexpectedly due to build errors.

PR Close #38656
2020-09-28 16:11:42 -04:00
Paul Gschwendtner 372a6cf8d7 feat(dev-infra): add release command for setting NPM dist tag (#38656)
Introduces a new command for `ng-dev release`, so that the NPM
dist tag can be set for all configured NPM packages. This command
can be useful in case a manual tag needs to be set, but it is
primarily used by the release tooling when a new stable version
is cut, and when the previous patch branch needs to be set as LTS
version through a `v{major}-lts` dist tag.

It is necessary to have this as a command so that the release tool
can execute it for old branches where other packages might have been
configured. This is similar to the separate `ng-dev build` command
that we created.

Note that we also added logic for spawning a process conveniently
with different "console output" modes. This will be useful for
other command invocations in the release tool and it's generally
better than directly using native `child_process` as that one doesn't
log to the dev-infra debug log file.

PR Close #38656
2020-09-28 16:11:42 -04:00
Paul Gschwendtner 8b01d42e5d feat(dev-infra): add shared testing utilities folder with git mock (#38656)
Adds a new folder to dev-infra where shared testing utilities
could be placed in. This commit already adds initial testing
utilities for dealing with the `GitClient` and SemVer versions.

The `GitClient` in the testing utilities simulates actual Git
behavior in a virtual manner. It's not complete at all, but can
be extended based on our needs. The currently implemented commands
are the most basic ones that we'd need for our release tooling.

PR Close #38656
2020-09-28 16:11:41 -04:00
Paul Gschwendtner 474ecf6d00 feat(dev-infra): add logic for printing active release trains (#38656)
Adds a method for printing active release trains for a configured
project. This is helpful for the release tool that will print
the active release trains. Also this can be useful for the
caretaker status command, where we could print the active
version branches (i.e. "is there currently a feature-freeze branch").

PR Close #38656
2020-09-28 16:11:41 -04:00
Paul Gschwendtner 69bb49e530 refactor(dev-infra): cleanup comments in git utilities (#38656)
Cleans up outdated comments in the shared dev-infra Git
utilities. We also export the Graphql client for consistency
as we expose the `GithubClient` and `GitClient` too.

PR Close #38656
2020-09-28 16:11:41 -04:00
Paul Gschwendtner d7ff8d765c refactor(dev-infra): use shared github repo interface (#38656)
Instead of maintaining multiple interface for grouping
owner name and repo name, we expose a shared interface
describing a Github repository.

One unfortunate downside is that the GraphQL Github
and Rest API diverge slightly with the key for the
repository name. i.e. rest uses `repo` for the name
of a repository, while GraphQL uses `name` for the name.

If that would be consistent, we could use the rest operator
to pass a repository to the Octokit REST or GraphQL API. This
does not work, so we have a small manual overhead as seen
in the `branches.ts` file.

PR Close #38656
2020-09-28 16:11:40 -04:00
Paul Gschwendtner 9e0e763156 refactor(dev-infra): do not print git commands in silent mode (#38656)
The git client respects the `SpawnSyncOptions` when a command
is executed. Currently it does not hide the command info
messages when commands are run in silent mode.

We fix this as part of this commit, so that the command info
is only printed to `debug` if `stdio` is set to `ignore`.

Additonally, the github token is made public so that it can be
used by commands if other repositories like forks are targeted.

PR Close #38656
2020-09-28 16:11:39 -04:00
Paul Gschwendtner 758d0e2045 refactor(dev-infra): share more github code between commands (#38656)
Instead of repeating the logic for adding the github token to
a repository git url, we add a shared function for automatically
computing the URls with token.

Additionally, URLs for updating/generating tokens have been moved
to a dedicated file in the `utils` folder. Also while being at it,
the yargs github token helper is also moved into the dedicated
Git/Github related util folder.

PR Close #38656
2020-09-28 16:11:38 -04:00
Joey Perrott e00535a2a4 fix(dev-infra): remove ANSI escape codes from log file outputs (#38792)
Remove the ANSI codes from the log file outputs to make the ng-dev log files
more readable.

PR Close #38792
2020-09-15 08:44:00 -07:00
Paul Gschwendtner e162da0753 refactor(dev-infra): do not validate config file multiple times (#38808)
Currently we validate the configuration file on each `getConfig`
invocation. We can only validate once since the configuration
is cached.

Also while being at it, renames the cache variables to lower-case as those
do not represent constants (which are convention-wise upper case).

PR Close #38808
2020-09-14 08:34:15 -07:00
Joey Perrott 6768fe9927 fix(dev-infra): correct build setup for dev-infra (#38815)
Correct's the missing dependencies and incorrect build dependencies for the
dev-infra bazel targets.

PR Close #38815
2020-09-11 13:59:30 -07:00
Joey Perrott 1373a98e25 feat(dev-infra): Allow local user ng-dev configuration to disable commit message wizard (#38701)
As not all users, particularly contributors consistently contributing with a deep
understanding of our commit message guidelines, will not want to rely on the
commit message wizard, we allow a user to opt out of using this wizard during
commit message creation.

PR Close #38701
2020-09-09 16:31:17 -07:00
Joey Perrott 1ed6913b8b feat(dev-infra): Add support for local user ng-dev configuration (#38701)
Create a utility for loading a local user configuration object to describe
local configuration values, such as skipping the commit message wizard.

PR Close #38701
2020-09-09 16:31:16 -07:00
Ikko Ashimine 2cb3d58b42 docs(dev-infra): fix typo in comment (arguements --> arguments) (#38653)
PR Close #38653
2020-09-03 09:45:02 -07:00
Joey Perrott a6f3cd93a9 feat(dev-infra): check services/status information of the repository for caretaker (#38601)
The angular team relies on a number of services for hosting code, running CI, etc. This
tool allows for checking the operational status of all services at once as well as the current
state of the repository with respect to merge and triage ready issues and prs.

PR Close #38601
2020-09-01 13:05:30 -07:00
Joey Perrott 4f28192d62 refactor(dev-infra): use a mixin to require a `github-token` for an ng-dev command (#38630)
Creates a mixin for requiring a github token to be provided to a command.  This mixin
allows for a centralized management of the requirement and handling of the github-token.

PR Close #38630
2020-08-31 12:32:27 -07:00
Joey Perrott 3487b549fd feat(dev-infra): write outputs of command runs to ng-dev log file (#38599)
Creates infrastructure to write outputs of command runs to ng-dev log file.
Additionally, on commands which fail with an exit code greater than 1, an
error log file is created with the posix timestamp of the commands run time
as an identifier.

PR Close #38599
2020-08-31 08:47:15 -07:00
Joey Perrott f77fd5e02a feat(dev-infra): create a wizard for building commit messages (#38457)
Creates a wizard to walk through creating a commit message in the correct
template for commit messages in Angular repositories.

PR Close #38457
2020-08-18 17:01:14 -07:00
Joey Perrott 63ba74fe4e feat(dev-infra): tooling to check out pending PR (#38474)
Creates a tool within ng-dev to checkout a pending PR from the upstream repository.  This automates
an action that many developers on the Angular team need to do periodically in the process of testing
and reviewing incoming PRs.

Example usage:
  ng-dev pr checkout <pr-number>

PR Close #38474
2020-08-18 16:22:47 -07:00
Joey Perrott 28534d83ee feat(dev-infra): Add support for formatting all staged files (#38402)
Adds an ng-dev formatter option to format all of the staged files. This will can
be used to format only the staged files during the pre-commit hook.

PR Close #38402
2020-08-11 16:32:54 -07:00
Paul Gschwendtner 576e329f33 feat(dev-infra): provide github API instance to lazy merge configuration (#38223)
The merge script currently accepts a configuration function that will
be invoked _only_ when the `ng-dev merge` command is executed. This
has been done that way because the merge tooling usually relies on
external requests to Git or NPM for constructing the branch configurations.

We do not want to perform these slow external queries on any `ng-dev` command
though, so this became a lazily invoked function.

This commit adds support for these configuration functions to run
asynchronously (by returning a Promise that will be awaited), so that
requests could also be made to the Github API. This is benefical as it
could avoid dependence on the local Git state and the HTTP requests
are more powerful/faster.

Additionally, in order to be able to perform Github API requests
with an authenticated instance, the merge tool will pass through
a `GithubClient` instance that uses the specified `--github-token`
(or from the environment). This ensures that all API requests use
the same `GithubClient` instance and can be authenticated (mitigating
potential rate limits).

PR Close #38223
2020-08-05 10:53:17 -07:00
Andrew Scott d0060dcfdc refactor(dev-infra): Add support for groups in the conditions evaluator (#38164)
Conditions can refer to the groups array that is a list of the preceding groups.
This commit adds support to the verification for those conditions.

This commit also adds some tests to the parsing and condition matching
to ensure everything works as expected.

PR Close #38164
2020-07-23 11:05:42 -07:00
Paul Gschwendtner 3ee666580a fix(dev-infra): merge script should not always require full repo permissions (#37718)
We recently added OAuth scope checking to the dev-infra Git client
and started leveraging it for the merge script. We set the `repo` scope
as required for running the merge script. We can loosen this requirement
as in the Angular org where the script is consumed, only pull requests on
public repositories are merged through the script.

This should help with reducing the risk with compromised tokens as no
access had to be granted on `repo:invite`, `repo_deployment` etc.

PR Close #37718
2020-06-26 09:52:33 -07:00
Paul Gschwendtner dbc2364d16 fix(dev-infra): support running scripts from within a detached head (#37737)
Scripts provided in the `ng-dev` command might use local `git`
commands. For such scripts, we keep track of the branch that
has been checked out before the command has been invoked.

We do this so that we can later (upon command completion)
restore back to the original branch. We do not want to
leave the Git repository in a dirty state.

It looks like this logic currently only deals with branches
but does not work properly when a command is invoked from
a detached head. We can make it work by just checking out
the previous revision (if no branch is checked out).

PR Close #37737
2020-06-26 09:51:09 -07:00
Joey Perrott a8af8551ec refactor(dev-infra): migrate github api in GitClient to rely on GithubClient (#37593)
GitClient now uses GithubClient for github API interactions.  GithubClient is
a class which extends Octokit and provides a member which allows for GraphQL
requests against the Github GraphQL api, as well as providing convenience methods
for common/repeated Github API requests.

PR Close #37593
2020-06-22 10:54:43 -07:00
Paul Gschwendtner a3313709ea fix(dev-infra): local changes check not working (#37489)
Looks like we broke the `hasLocalChanges` check in the git client
when we moved it over from the merge script. The problem is that
we are using `git` in the first argument of `git.run`. That
means that we under-the-hood run `git git <..>`.

This commit fixes that, but also switches to a better variant
for ensuring no local changes because it exits with non-zero
when there are local changes.

PR Close #37489
2020-06-09 08:27:32 -07:00
Paul Gschwendtner bb924b63e6 fix(dev-infra): incorrect token sanitization when no token is specified (#37489)
We recently moved over the git client from the merge script to the
common dev-infra utils. This made specifying a token optional, but
it looks like the logic for sanitizing messages doesn't account
for that, and we currently add `<TOKEN>` between every message
character. e.g.

```
Executing: git <TOKEN>g<TOKEN>i<TOKEN>t<TOKEN>
<TOKEN>s<TOKEN>t<TOKEN>a<TOKEN>t<TOKEN>u<TOKEN>s<TOKEN>
```

PR Close #37489
2020-06-09 08:27:31 -07:00
Paul Gschwendtner 17996bf719 fix(dev-infra): properly determine oauth scopes for git client token (#37462)
Resubmit of b2bd38699b since
85b6c94cc6 accidentally reverted
the fix due to rebasing most likely.

PR Close #37462
2020-06-05 11:04:36 -07:00
Joey Perrott 85b6c94cc6 refactor(dev-infra): move GitClient to common util (#37318)
Moves GitClient from merge script into common utils for unified
method of performing git actions throughout the ng-dev toolset.

PR Close #37318
2020-06-05 09:46:40 -07:00
George Kalpakas d4c0962c7b refactor(dev-infra): use the `exec()` helper from `utils/shelljs` whenever possible (#37444)
There is an `exec()` helper provided by `utils/shelljs.ts`, which is a
wrapper around ShellJS' `exec()` with some default options (currently
`silent: true`). The intention is to avoid having to pass these options
to every invocation of the `exec()` function.

This commit updates all code inside `dev-infra/` to use this helper
whenever possible).

NOTE: For simplicity, the `utils/shelljs` helper does not support some
      of the less common call signatures of the original `exec()`
      helper, so in some cases we still need to use the original.

PR Close #37444
2020-06-05 09:21:18 -07:00
Joey Perrott 4e7a4543b8 style(dev-infra): correct tslint failures in dev-infra directory (#37233)
Fixes tslint failures in dev-infra directory as the directory is now
part of the tslint enforced files.

PR Close #37233
2020-06-04 12:44:46 -07:00
Joey Perrott 3d154ab795 fix(dev-infra): migrate dev-infra config loading to .ng-dev directory (#37142)
Migrate to using .ng-dev directory for ng-dev configuration to better
allow management of the configuration using multiple files.  The
intention is to prevent the config file from becoming unruly.

PR Close #37142
2020-05-26 16:38:22 -04:00
Joey Perrott d1ea1f4c7f build: update license headers to reference Google LLC (#37205)
Update the license headers throughout the repository to reference Google LLC
rather than Google Inc, for the required license headers.

PR Close #37205
2020-05-26 14:26:58 -04:00
Joey Perrott e0002ef9ba feat(dev-infra): migrate ng-dev utils to use new logging system (#37232)
Migrate the ng-dev utils to use new logging system rather
than directly calling console.* to create a better experience
for users.

PR Close #37232
2020-05-21 17:40:35 -04:00
Joey Perrott 52c7aae4df feat(dev-infra): add group functions to logging system and remove color param (#37232)
Adds .group and .groupEnd functions to each of the logging functions
to allow creating groups in the logged output.  Additionally removes
the color parameter from logging functions, in favor of the color
being applied to the string at the call site.

PR Close #37232
2020-05-21 17:40:34 -04:00
Paul Gschwendtner 3e5fa56956 fix(dev-infra): ensure ts-node is registered with commonjs as module (#37217)
We recently added support for automatic registration of `ts-node`
when the dev-infra configuration is loaded.

In addition to registering ts-node, we should also ensure that the
`commonjs` module is set up. By default, `ts-node` would use ES module
imports that are not supported by default in NodeJS.

PR Close #37217
2020-05-21 10:35:23 -07:00
Joey Perrott 7db177d181 feat(dev-infra): create logging utils with support for defined logging levels (#37192)
Creates common logging functions at different levels. Allows for providing
logging statements which are actually printed to the console based on the
LOG_LEVEL environment variable.

PR Close #37192
2020-05-20 13:32:04 -07:00
Paul Gschwendtner 383f04b96d feat(dev-infra): register ts-node when reading configuration (#37196)
`ts-node` is now an optional peer dependency of the shared dev-infra
package. Whenever a `ng-dev` command runs, and a TypeScript-based
configuration file exists, `ts-node` is set up if available.

That allows consumers of the package (as the components repo) to more
conveniently use a TypeScript-based configuration for dev-infra.

Currently, commands would need to be proxied through `ts-node`
which rather complicates the setup:

```
NG_DEV_COMMAND="ts-node ./node_modules/@angular/dev-infra-private/cli.js"
```

I'm thinking that it should be best-practice to use TypeScript for
writing the configuration files. Given that the tool is used primarily
in Angular projects (for which most sources are TypeScript), this should
be acceptable.

PR Close #37196
2020-05-19 15:21:57 -07:00
Joey Perrott 009f394237 feat(dev-infra): create rebase-pr tool in ng-dev (#37055)
Creates a tool in ng-dev which rebases a PR automatically and pushes
the rebase commits back to the PR.  This is meant to be a replacement
to the local merge script currently in the repo and currently has
feature parity.

PR Close #37055
2020-05-18 16:18:32 -07:00
Paul Gschwendtner 8a3493af47 feat(dev-infra): integrate merge script into ng-dev cli (#37138)
Integrates the merge script into the `ng-dev` CLI. The goal is that
caretakers can run the same command across repositories to merge a pull
request. The command is as followed: `yarn ng-dev pr merge <number>`.

PR Close #37138
2020-05-18 11:50:08 -07:00
Joey Perrott 3d7c85b2aa feat(dev-infra): create tool to determine conflicts created by a PR (#37051)
Creates a tool in ng-dev to determine the PRs which become conflicted
by merging a specified PR.  Often the question is brought up of how
many PRs require a rebase as a result of a change.  This script allows
to determine this impact.

PR Close #37051
2020-05-15 11:29:35 -07:00
Joey Perrott 564ca9e8ec build: add github config for ng-dev configuration (#37097)
Adds the gitub configuration to the ng-dev configuration. This github
configuration provides information needed for making API requests to
github.  Upcoming tooling related PRs will require these API requests
being possible.

PR Close #37097
2020-05-15 10:02:41 -07:00
Joey Perrott 20d377c11b feat(dev-infra): add github to common config for ng-dev configuration typings (#37097)
Adds a gitub object to the common configuration for ng-dev. This github
configuration provides information needed for making API requests to
github.

PR Close #37097
2020-05-15 10:02:40 -07:00
Joey Perrott 14c0ec97d8 feat(dev-infra): introduce validators for ng-dev config loading (#37049)
Introduces infrastructure to validate configuration of the ng-dev
command at run time.  Allowing for errors to be returned to the
user running the command.

PR Close #37049
2020-05-13 15:58:46 -07:00
Joey Perrott 8b65946eb8 feat(dev-infra): support typescript config file (#37017)
Previously ng-dev loaded the config through a javascript file, this
change allows for the loaded file to be either javascript or
typescript.  This enables configurations to be written with type
safety.

PR Close #37017
2020-05-08 14:40:11 -07:00
Joey Perrott 4e628a887a fix(dev-infra): update the config file loading for ng-dev to expect js (#36918)
Migrating to a js file for providing a configuration allows for more
extensive configuration at run time.  This allows for configs to include
logic and move beyond static values found in JSON files.

PR Close #36918
2020-05-08 09:38:25 -07:00
Joey Perrott 7b5a0ba8c3 feat(dev-infra): create format tool in @angular/dev-infra-private (#36726)
Previously we used gulp to run our formatter, currently clang-format,
across our repository.  This new tool within ng-dev allows us to
migrate away from our gulp based solution as our gulp solution had
issue with memory pressure and would cause OOM errors with too large
of change sets.

PR Close #36726
2020-04-24 12:32:18 -07:00
Joey Perrott 698b0288be build: reformat repo to new clang@1.4.0 (#36613)
PR Close #36613
2020-04-14 12:08:36 -07:00
Joey Perrott 6fc85073d2 feat(dev-infra): create commit-message validation script/tooling (#36117)
PR Close #36117
2020-03-20 10:20:12 -07:00