Commit Graph

269 Commits

Author SHA1 Message Date
Joey Perrott 4ac55ca676 fix(dev-infra): use template strings for release note templates (#42224)
Use template strings for release note templates rather than loading `ejs` files at runtime.

PR Close #42224
2021-05-24 14:48:02 +00:00
Paul Gschwendtner 8c158babdc fix(dev-infra): formatting errors not reported with failure message (#42202)
Currently if formatting for a file fails due a formatter error,
the `ng-dev` tool reports that formatting failed, but no actual
error (or involved file) is printed out. This commit prints out
the failed files with their error message.

PR Close #42202
2021-05-21 18:06:47 +00:00
Joey Perrott ec78e68a36 fix(dev-infra): correct FormatConfig interface (#42154)
Correct the FormatConfig interface to use `[key: string]` instead of the incorrect
`[keyof: string]`.

PR Close #42154
2021-05-18 15:40:23 -07:00
Paul Gschwendtner 9d75687f62 feat(dev-infra): publish major versions to "next" NPM dist tag (#42133)
Previously, the dev-infra release tool would publish major versions
directly to the NPM `@latest` dist tag. This is correct in theory, but
rather unpractical given that we want to publish packages first as
`@next` so that other dependent Angular packages can update too,
allowing us to publish all main Angular packages (from FW, COMP
and TOOL) at the same time to `@latest` on NPM.

This involves creating a new release action for re-tagging the
previously released major as `@latest` on NPM.

PR Close #42133
2021-05-18 14:04:14 -07:00
Paul Gschwendtner 48f49bacb4 refactor(dev-infra): improve type safety of NPM dist tags in release tool (#42133)
Instead of passing `string` in the release tool for NPM dist tags, we
should use a union string type that limits the tags to `latest`, `next`
and anything matching `v{number}-lts`. This avoids mistakes at
compilation-level if an invalid/unknown tag would be set by a release
action.

PR Close #42133
2021-05-18 14:04:14 -07:00
Joey Perrott 6d87895ffd test(dev-infra): add testing for integration of ReleaseNotes in publishing tooling (#41967)
Add testing of the ReleaseNotes integration with the release publishing tooling.

PR Close #41967
2021-05-14 21:07:27 +00:00
Paul Gschwendtner 7fb8527e1e refactor(dev-infra): remove invoke bazel clean command from release tool (#42101)
Currently the ng-dev release tool always run `bazel clean` before
calling the configured build release function. The clean is necessary
to ensure the release output is actually built; and not restored
from previous builds which could have different bazel workspace
status variables (which provide the NPM package version).

Instead of doing this as part of the release tool, the
actual script running to build the release output should
run the `bazel clean`. The release tool does not intend to
know about details on how the release output is built. This
is necessary because the build setup could vary between version
branches (especially for older ones; such as LTS version branches).

PR Close #42101
2021-05-14 21:06:09 +00:00
Paul Gschwendtner 2843f15e8c fix(dev-infra): merge tool should ensure that token has `workflow` oauth scope (#41989)
Currently if a PR modifies any file that configures a Github action
(e.g. a workflow file), the caretaker might face an error when merging
such PR:

```
! [remote rejected]       merge_pr_target_11.2.x -> 11.2.x (refusing to allow a Personal Access Token to create or update workflow
```

This happens because Github requires the token being used for the
push operation to have the `workflow` scope set. This is a special
scope added by Github to ensure that no changes can be made on
upstream branches that might expose the `GITHUB_TOKEN` environment
variable, which is available for push builds and could cause the
token being leaked.

With this commit we enforce that the caretaker adds the workflow
scope to their github token. Since PRs can only be merged if reviewed
thoroughly, it's acceptable to allow workflow file changes being
merged through the merge tool by the caretaker (especially since we
also allow CircleCI config files being merged with the default
`repo`/`public_repo` scope).

PR Close #41989
2021-05-07 14:10:39 -04:00
Joey Perrott e1c5cea2e7 fix(dev-infra): set the default LogLevel of GitClient logging to DEBUG (#41899)
Previously by default GitClient would log the commands it was executing at the
INFO level. This change moves the default level of this logging to DEBUG, while
still allowing callers of the methods to set the log level back to INFO.

PR Close #41899
2021-05-07 10:15:20 -04:00
Joey Perrott 4d7294d9f7 feat(dev-infra): include release notes in Github release entries when publishing (#41964)
Include the release notes entry for the release in the release created on Github.

PR Close #41964
2021-05-06 15:45:48 -04:00
Joey Perrott 393ce94718 feat(dev-infra): Set up new common release notes generation tooling (#41905)
Enables the new common release notes generation within the ng-dev release publishing
tooling.

PR Close #41905
2021-05-06 09:40:29 -04:00
Paul Gschwendtner 530e45f093 fix(dev-infra): publish script python check not working on windows (#41943)
We recently added a sanity check that ensures that `python` is
available for Bazel. The check relies on `/usr/bin/python` to check
if python is available w/ the correct version. The problem is
that on Windows there is no guarantee that the `/usr/bin` folder
exists, so the `ng-dev release publish` command always fails.

We fix this by just accessing the `env` binary that is also consulted
by scripts executed within Bazel actions. The `env` binary can be
assumed exist in the shell's `$PATH` and can point us to Python
as if it would be executed within Bazel.

PR Close #41943
2021-05-05 13:46:03 -07:00
Paul Gschwendtner 21c2a06811 fix(dev-infra): spawned child processes messing with tty output (#41948)
Currently we have a common utility method for running commands
in a child process. This method pipes all stdout and stderr, but sets
the `stdin` to `inherited`. This seemed to work as expected in terms of
allowing interactive commands being executed, but it messes with the
TTY in Windows (and potentially other platforms) so that colors and
prompts no longer work properly. See attached screenshot.

We fix this by not inheriting the stdin by default; but exposing
a dedicated method for interactive commands. This results in more
readable and obvious code too, so it's worth making this change
regardless of the TTY issues.

PR Close #41948
2021-05-05 13:45:28 -07:00
Paul Gschwendtner 19b12d7286 refactor(dev-infra): switch away from deprecated platform execution properties (#41941)
Bazel no longer recommends the use of `remote_execution_properties`
within `platform` definitions. Bazel intends to replace this attribute
with an object literal based attribute called `exec_properties`.

This simplifies the platform configuration and makes it more
readable. Additionally this make inheritance and overriding easier.

PR Close #41941
2021-05-04 21:09:30 -07:00
Paul Gschwendtner 5947239f89 fix(dev-infra): do not set lts dist tag on packages from release-candidate train (#41946)
Currently if a major release-train in the `release-candidate`/`feature-freeze`
phase becomes `latest`, we intend to set the NPM LTS dist tag for all packages
of the previous major (as the old release train in `latest` moves into LTS phase).

The logic for this exists but the release tool sets the NPM dist tag for
all packages of the new major. This means that the script might error if
a new package is part of the new major; or it could cause a deleted
package to not receive the LTS tag properly.

PR Close #41946
2021-05-04 21:07:57 -07:00
Zach Arend fdf2e02bff build: fix crash when validating commit messages (#41949)
Fixes a crash when running the `ValidateFileModule`.

PR Close #41949
2021-05-04 21:05:18 -07:00
Joey Perrott c0694bf538 fix(dev-infra): remove getBaseDir method from GitClient (#41898)
Remove the getBaseDir method from GitClient as the baseDir is determined
once during construction and does not need to be rediscovered.  The property
access should always be used.

PR Close #41898
2021-04-30 14:33:59 -07:00
George Kalpakas fff2ca540f fix(dev-infra): correctly detect prettier config (#41860)
This commit uses the correct property name (`prettier` vs `pretter`) to
check whether prettier is enabled via the config.

(It also fixes some typos in `dev-infra`.)

PR Close #41860
2021-04-28 09:21:09 -07:00
Joey Perrott 0140d04ced feat(dev-infra): create the prettier formatter for ng-dev format tooling (#41824)
Add support to use prettier for formatting using `ng-dev format`.

PR Close #41824
2021-04-27 12:23:05 -07:00
Joey Perrott b1ae31c712 fix(dev-infra): add github token requirement for discover-new-conflicts (#41825)
Add the github token option/requirement for the pr discover-new-conflicts
command.

PR Close #41825
2021-04-27 10:23:16 -07:00
Paul Gschwendtner 1b00533f8b test(dev-infra): publish test should not rely on external http request (#41767)
A test for the dev-infra publish tool currently relies on an external
http request by accident. This will break if we disallow internet
access within RBE containers.

PR Close #41767
2021-04-23 11:01:40 -07:00
Paul Gschwendtner 59bb24148e feat(dev-infra): create shared platform for remote execution (#41767)
Introduces a shared Bazel platform for remote execution builds
using Google cloud. Previously we used `bazel_toolchains` for
providing the platform w/ additional CPP and Java toolchains

`bazel_toolchains` no longer provides default toolchains with the
latest version, but provides a tool (linux and windows only) for
generating toolchain/platforms, which then need to be checked
into the repository. This is quite inconvenient and cumbersome
(especially with no macOS support), so we just provide our own
platform and CPP toolchain within `//dev-infra`. This is more
simple than all the effort we'd need to make the toolchain
generation tool work (while it would also increase the amount
of checked-in sources significantly; with more unused toolchains
for CPP or Java)

PR Close #41767
2021-04-23 11:01:40 -07:00
Joey Perrott a4a55f0687 fix(dev-infra): verify python3 is available before release (#41753)
Verify that the /usr/bin/python points to the python3 interpreter binary.

PR Close #41753
2021-04-21 17:31:22 -07:00
Alan Agius 8bfd9e886b build: update typed-graphqlify to version 3.1.1 (#41738)
Use latest version of typed-graphqlify

PR Close #41738
2021-04-21 17:30:28 -07:00
Alan Agius e00a9c1b99 build: update `chalk` to version 4.0.1 (#41738)
Update `chalk` to `4.0.1` so that it can be managed by Renovate.

PR Close #41738
2021-04-21 17:30:28 -07:00
George Kalpakas 7c353d452e refactor(dev-infra): remove unused argument from `publish` script (#41688)
Since recently, the `githubToken` parameter passed to the `ReleaseTool`
constructor is no longer used. This commit removes the unused parameter
and also the corresponding argument from the `publish` script.

PR Close #41688
2021-04-19 08:28:34 -07:00
George Kalpakas 8058396c52 docs(dev-infra): update jsdoc comments to reflect recent code changes (#41688)
PR Close #41688
2021-04-19 08:28:34 -07:00
George Kalpakas 5ba77f4f6a docs(dev-infra): fix typos in `dev-infra` docs/comments (#41688)
PR Close #41688
2021-04-19 08:28:34 -07:00
Joey Perrott 83d6a96163 feat(dev-infra): prevent pr merges of features or breaking changes on rc (#41660)
Prevent pull requests targeting `target: rc` from merging PRs with breaking
changes or features from being merged.

PR Close #41660
2021-04-16 13:09:47 -07:00
Joey Perrott dd4c3dba3f refactor(dev-infra): move `getRepoBaseDir()` to `GitClient` (#41527)
As `getRepoBaseDir()` relies on git, it should be a method on `GitClient` for retrieval
rather than its own utility outside of the common GitClient used for all git ineractions.

PR Close #41527
2021-04-15 11:28:12 -07:00
Joey Perrott a63cf7221b feat(dev-infra): prevent attempting to merge draft, closed or merged pull requests (#41604)
During pull request validation, assert that the pull request is not in
draft mode, already merged or already closed.

PR Close #41604
2021-04-14 15:44:55 -07:00
Joey Perrott 15c307b200 fix(dev-infra): only create authenticated github instance once in yargs (#41603)
Fix github token option for yargs to only create an authenticated token one time.

PR Close #41603
2021-04-13 11:50:48 -07:00
Alan Agius ed7d288364 build: update several dependencies (#41434)
With this change we update several dependencies to avoid Renovate creating a lot of PRs during onboarding. We also remove yarn workspaces as after further analysis these are not needed.

Certain dependencies such as `@octokit/rest`, `remark` and `@babel/*` have not been updated as they require a decent amount of work to update, and it's best to leave them for a seperate PR.

PR Close #41434
2021-04-12 16:46:29 -07:00
Joey Perrott 9bf8e5164d refactor(dev-infra): use a singleton for GitClient (#41515)
Creates a singleton class for GitClient rather than relying on creating an instance to
require being passed around throughout its usages.

PR Close #41515
2021-04-12 16:43:54 -07:00
Joey Perrott 4a69a7b952 feat(dev-infra): verify breaking changes are properly labeled before merging (#41546)
During merging with `ng-dev pr merge` tooling will ensure that pull requests are
properly labeled for breaking changes.  Pull requests with commits noting breaking
changes must also be labeled as such, additionally pull requests with breaking
change labels must contain commits noting breaking changes.

Fixes #38776

PR Close #41546
2021-04-12 10:11:17 -07:00
Joey Perrott 5e0d5a9ec2 feat(dev-infra): create ReleaseNotes class for generating release notes during publishing (#41476)
Generate release notes to be used for entries in both CHANGELOG.md files as well as
Github releases.

PR Close #41476
2021-04-09 07:49:48 -07:00
Joey Perrott 7e6989ee4b test(dev-infra): extract commit message build function into testing util function (#41476)
Creates a testing utility function to build commit message strings.

PR Close #41476
2021-04-09 07:49:48 -07:00
Joey Perrott 7a40d8cefd fix(dev-infra): use base ref name rather than sha for the githubTargetBranch (#41523)
After updating to use the v4 graphql api in the merge tooling, the githubTargetBranch
was set to be the sha of the latest commit rather than the branch name of the target
branch.  This caused our tooling to mismatch which branches were actually being targeted
with the effect that if a PR targeted only the patch branch (i.e. labeled `target: patch`
and targeting `11.2.x` in github)it would still expect to merge into both `11.2.x` and
`master`.  This is now corrected to once again use the branch name, restoring to the
previous functionality.

PR Close #41523
2021-04-08 15:18:02 -07:00
Joey Perrott c7d86bca21 feat(dev-infra): add support for `targetLabelExemptScopes` for merging (#41459)
Add a property, `targetLabelExemptScopes`, to the merge configuration allowing certain
scopes to be exempted from requirements for features and breaking changes only included
in PRs targetting certain labels.

PR Close #41459
2021-04-07 11:52:35 -07:00
Joey Perrott 7dba0711c2 feat(dev-infra): prevent merges for PRs with invalid breaking changes or commit types (#41459)
Check commits in pull requests to ensure the pr can be merged into the target branch. Confirms
that prs targeting minor do not contain breaking changes, and prs targeting patch or lts do not
contain breaking changes or `feat` commits.

PR Close #41459
2021-04-07 11:52:35 -07:00
Joey Perrott 2d3539f4df refactor(dev-infra): use graphql to query PRs in merge tooling (#41459)
Migrate to use graqhql to query for PR information during the validation and
preperation portions of the pr merge tooling.

PR Close #41459
2021-04-07 11:52:35 -07:00
Joey Perrott f98f379dd8 fix(dev-infra): make scopes for `test` and `refactor` commit types optional (#41486)
As the `test` and `refactor` commit types are not used in release notes and there
are solid use cases for having multiple scopes/scopeless uses of these types, they
are made to be optional instead of required on commits.

PR Close #41486
2021-04-07 09:40:31 -07:00
Joey Perrott c63d00e5b0 feat(dev-infra): update commit-message functions to properly type commits from git log (#41458)
For commits from git log entries additional fields are available such as the reference
hash and author name, update the utility functions in commit-message to include the
parsed fields.  Additionally define, per commit message type, whether to include the
commit in a release notes entry.

PR Close #41458
2021-04-06 10:04:53 -07:00
Joey Perrott ba3344ddbe feat(dev-infra): add utility method to GitClient to get latest SemVer tag (#41455)
Create a utility method for the latest git tag, sorted by committerdate, which
matches SemVer, representing the latest version released on the branch.

PR Close #41455
2021-04-06 10:04:31 -07:00
Joey Perrott 2cef385e43 feat(dev-infra): prevent `git push` from being called in dryRun mode (#41387)
Update GitClient to prevent the `push` command from being run in dryMode.

PR Close #41387
2021-04-05 08:57:13 -07:00
Joey Perrott f23406462b feat(dev-infra): create dry-run environment variable flag utils (#41387)
Create utility functions for adding a dry-run flag and checking the environment
for a dry run mode.

PR Close #41387
2021-04-05 08:57:13 -07:00
Alan Agius 64b979e12b fix(dev-infra): exclude node_modules from circular dependency checks (#41407)
With this change we exclude node_modules from circular dependency checks.

PR Close #41407
2021-04-02 15:47:33 -07:00
Joey Perrott a43f36babd fix(dev-infra): require npm login for all publishes via wombat proxy (#41422)
Due to an issue with wombat proxy returning the login state of the generated tokens,
we will need to require a login for all `ng-dev release publish` runs to ensure npm
login has occured.

PR Close #41422
2021-04-02 10:30:18 -07:00
Alan Agius c5e9944c42 fix(dev-infra): allow npm like scopes as commit message scopes (#41430)
In the CLI and Universal, we use the package name as commit message scope. The recent changes that introduced `conventional-commits-parser` in https://github.com/angular/angular/pull/41286 breaks the parsing of such commit scopes and caused commit validations to fail.

Example: https://app.circleci.com/pipelines/github/angular/angular-cli/14420/workflows/85feb5c9-184f-4088-b924-6b9e6c91f062/jobs/238446/parallel-runs/0/steps/0-102

PR Close #41430
2021-04-02 10:28:08 -07:00
Joey Perrott 381ea9d7d4 refactor(dev-infra): set up new method for checking range of commits (#41341)
Check a range of commits by retrieving the log files to be parsed with the expected
format for the parser.

This change is in part of a larger set of changes making the process for obtaining
and parsing commits for release note creation and message validation consistent.
This consistency will make it easier to debug as well as ease the design of tooling
which is built on top of these processes.

PR Close #41341
2021-04-01 11:30:26 -07:00