Commit Graph

105 Commits

Author SHA1 Message Date
Joey Perrott d3b0d1e3fb refactor(dev-infra): add spawnSync to child process utils, normalize naming of child-process utils (#42911)
Create a `spawnSync` command for common usage, additionally update naming to use `spawn` instead of
`spawnWithDebugOutput`

PR Close #42911
2021-07-21 22:33:37 +00:00
Joey Perrott 282e86ad71 fix(dev-infra): use the version from package.json rather than tags (#42872)
Use the version value from the primary package.json file rather than
checking the branch for the latest semver tag.  This allows for us
to explictly create changelogs from the previous version to the new
version.

PR Close #42872
2021-07-20 11:30:33 -07:00
Joey Perrott 445fee4174 feat(dev-infra): update package version verification to allow for experimental version (#42898)
The angular-cli repo publishes experimental versioned packages in addition to standard
versioned packages.  Both experimental and standard verions, based on the expected new
version provided are treated as valid and correct.

PR Close #42898
2021-07-20 11:30:00 -07:00
Joey Perrott 96d31d8c9f fix(dev-infra): prevent verbose messaging of git checkout and commit for release (#42871)
Prevent logging verbose messages to the output for commit and checkout during
the release process.

PR Close #42871
2021-07-19 17:15:53 -07:00
Joey Perrott fd92a3e87f fix(dev-infra): perform actual npm login verification for wombot proxy (#42871)
Previously the wombot proxy was not able to be queried for login state, now
that it is able to be checked via `npm whoami` a specific alternate flow
is no longer needed.

PR Close #42871
2021-07-19 17:15:53 -07:00
Joey Perrott 89509dffd2 refactor(dev-infra): provide entire pull request object to waitForPullRequestToBeMerged (#42871)
`waitForPullRequestToBeMerged` should get the entire `PullRequest` object rather than
just the id to allow it to have more information/context for logging and messaging.

PR Close #42871
2021-07-19 17:15:53 -07:00
Joey Perrott 53279262be fix(dev-infra): don't pollute standard out during commit message hooks (#42871)
When a commit message is not restored, there is not need to log this to
stdout.  It is a better experience to silently succeed.

PR Close #42871
2021-07-19 17:15:53 -07:00
Joey Perrott bc1da1fe14 feat(dev-infra): create caretaker handoff assistant to update github team membership (#42859)
Create a handoff assistant, currently only assists in updating github team membership
for caretaking team.

PR Close #42859
2021-07-15 13:34:58 -07:00
Andrew Kushnir c1c1cda866 Revert "refactor(dev-infra): add spawnSync to child process utils, normalize naming of child-process utils (#42394)" (#42829)
This reverts commit 08444c6679.

PR Close #42829
2021-07-12 15:24:55 -07:00
Andrew Kushnir 3d668162d9 Revert "refactor(dev-infra): remove usages and dependency on shelljs (#42394)" (#42829)
This reverts commit f0d857eff8.

PR Close #42829
2021-07-12 15:24:54 -07:00
Joey Perrott f0d857eff8 refactor(dev-infra): remove usages and dependency on shelljs (#42394)
Remove usages of shelljs and instead use spawn/spawnSync.

PR Close #42394
2021-07-12 14:39:08 -07:00
Joey Perrott 08444c6679 refactor(dev-infra): add spawnSync to child process utils, normalize naming of child-process utils (#42394)
Create a `spawnSync` command for common usage, additionally update naming to use `spawn` instead of
`spawnWithDebugOutput`

PR Close #42394
2021-07-12 14:39:07 -07:00
George Kalpakas c3620f9a5f fix(dev-infra): convert commit SHAs and PR numbers to links when generating changelog (#42732)
Previously, the commit SHAs and PR numbers referenced in the generated
`CHANGELOG.md` were not automatically converted to links in the GitHub
UI (as happens for release notes and issue/PR comments). This made it
less straight-forward for someone reading the changelog to get to the
commit/PR corresponding to a change.

This commit updates the tooling that generates the changelog to convert
the commit SHA and the corresponding PR number (referenced at the end of
the commit message header) to links.

PR Close #42732
2021-07-02 09:23:45 -07:00
Alan Agius 8528073f6c fix(dev-infra): remove bots from special thanks section (#42697)
With this change we remove known used bots from special thanks section in the changelog.

PR Close #42697
2021-06-29 11:13:50 -07:00
Paul Gschwendtner f29fe5ced0 fix(dev-infra): use API pagination for retrieving project branches (#42666)
We rely on a Github API `/branches` request to determine the active
release trains. Currently this logic is broken if more than 100
protected branches exist within a repository. This issue surfaced
recently where the `items_per_page` setting was set to `30`, causing
the merge tooling and release tooling to not detect the proper "latest"
release train.

This commit uses Github pagination for retrieving branches to determine
the active release trains, and makes the logic more long-term proof.

PR Close #42666
2021-06-29 10:33:54 -07:00
Paul Gschwendtner 279e63f65f refactor(dev-infra): update octokit to latest version v18 (#42666)
We previously held off with updating Octokit to v18 due to
their more noticable issues with typings. This commit updates
us to the latest version in order to take advantage of the new
pagination API (which is also strongly-typed), and to not fall
behind too much over time (Octokit seems to change quite often..)

We work around the problem with the types for `getContent` by just
using a type cast with a TODO (and link to the issue). Similarly we
work around a problem where the Octokit types have an incorrect type
for the name of the labels array in an API response.

PR Close #42666
2021-06-29 10:33:54 -07:00
Paul Gschwendtner 41823ff277 feat(dev-infra): add command for printing release train information (#42644)
Currently the active release trains are printed when a developer
runs `ng-dev publish release`. This is not ideal because it requires
the developer to provide an OAuth token, to be on the next branch, and
to have no uncommitted changes, while the actual release train
information is not dependent on these checks.

This commit introduces a new command called `ng-dev release info` that
can be used to retrieve relase information without the aforementioned
requirements. Note that this command provides more detailed information
about release branches than the `ng-dev caretaker check` command (which
also requires on authentication as a side note). The `release info`
command also prints active LTS branches for example.

PR Close #42644
2021-06-28 11:50:57 -07:00
Alex Rickabaugh 261b060fa1 fix(dev-infra): fetch 100 branches from Github instead of 30 (#42658)
This commit fixes an issue with the ng-dev tool, where Github's API returns
paginated branch data. Only 30 branches are returned by default, and Angular
now has more than 30 branches in its repo. This commit increases the number
of branches returned to the API limit of 100, which should buy us some time
until we can implement proper pagination.

PR Close #42658
2021-06-24 17:26:24 -07:00
Joey Perrott 81a19e4e65 feat(dev-infra): update pullapprove verification to ensure all groups have reviewers (#42614)
Update the pullapprove verification tooling to ensure a reviewer is defined for
each group. This is being done in preparation for the upcoming change to how
pullapprove billing works. The new billing will work on a seats based approach
rather than flat usage.

PR Close #42614
2021-06-22 16:27:36 +00:00
Paul Gschwendtner 67f65a9d25 refactor(dev-infra): improve type-safety of git client utility (#42468)
Currently the `GitClient` accepts a generic parameter for determining
whether the `githubToken` should be set or not. This worked fine so far
in terms of distinguishing between an authenticated and
non-authenticated git client instance, but if we intend to conditionally
show methods only for authenticated instances, the generic parameter
is not suitable.

This commit splits up the `GitClient` into two classes. One for
the base logic without any authorization, and a second class that
extends the base logic with authentication logic. i.e. the
`AuthenticatedGitClient`. This allows us to have specific methods only
for the authenticated instance. e.g.

  * `hasOauthScopes` has been moved to only exist for authenticated
    instances.
  * the GraphQL functionality within `gitClient.github` is not
    accessible for non-authenticated instances. GraphQL API requires
    authentication as per Github.

The initial motiviation for this was that we want to throw if
`hasOAuthScopes` is called without the Octokit instance having
a token configured. This should help avoiding issues as within
3b434ed94d
that prevented the caretaker process momentarily.

Additionally, the Git client has moved from `index.ts` to
`git-client.ts` for better discoverability in the codebase.

PR Close #42468
2021-06-03 14:34:33 -07:00
Paul Gschwendtner 49c3ce58bf refactor(dev-infra): remove duplicate method that checks for uncommitted changes (#42409)
Removes the duplicate `hasLocalChanges` method from the Git client. We
already have `hasUncommittedChanges`. Also removes a TODO for adding
`assertNoLocalChanges` as it seems more flexible to manually check
(i.e. better messaging with context on the current tool; e.g. "cannot
perform rebase")

PR Close #42409
2021-06-03 10:36:40 -07:00
Paul Gschwendtner 97c5a3ae6f feat(dev-infra): prevent deprecations from being merged into patch branch (#42454)
As per SemVer, deprecations should only be released if the
minor version is incremented. This means that we should error
if commits w/ deprecations target an upcoming patch, LTS release.

Additionally, we also error if deprecations are made as part of
an upcoming release-candidate.

PR Close #42454
2021-06-02 13:23:54 -07:00
Paul Gschwendtner d3531a7d41 fix(dev-infra): breaking change and deprecation notes incorrectly picked up (#42436)
If a commit message currently mentions the breaking change or
deprecation note keywords, the commit message parse logic
accidentally picks up the note. This could then accidentally
prevent the commit from being merged (e.g. if the commit targets
the patch branch but mentioned the `BREAKING CHANGE: ` marker).

This commit switches the commit message notes pattern to only
capture notes at the beginning of a line (also allowing accidental
whitespace). This matches with the format we describe in our
contribution guide, as well as with our commit message validation
logic that also assumes notes at the beginning of a line.

PR Close #42436
2021-06-02 13:22:55 -07:00
Paul Gschwendtner bc5a8f4d37 feat(dev-infra): validate deprecation notes in commit messages (#42436)
Currently the commit message validation tool from `ng-dev` validates
the `BREAKING CHANGE:` commit message notes. This commit adds a similar
check for `DEPRECATED:` commit message notes.

Additionally, the check for breaking changes is reworked slightly to
be more tolerant (i.e. if there is only a single line break after the
summary; this is acceptable as per the parser and commonly done in the
COMP repo). The checks have been updated to capture wrong keywords that
are commonly used instead of the correct one. e.g. if a commit message
uses `DEPRECATIONS:` instead of `DEPRECATED:`, the validation will fail.

This prevents changelog generation issues where breaking change notes,
or deprecations are missing. This happened in the COMP repo where
the `DEPRECATED:` keyword was used incorrectly. See:

99391e7939

PR Close #42436
2021-06-02 13:22:55 -07:00
Paul Gschwendtner 3b434ed94d
fix(dev-infra): github oauth token not picked up by Octokit (#42456)
We recently updated `@octokit/rest` to a more recent version. For this
the Github client had to be refactored to account for new types of
Octokit. With the refactorings the authentication of the Github
client broke as we used an incorrect option for setting the OAuth
token. The TS transpilation did not fail because the Octokit types
support arbitrary options in the constructor. This is not solvable
on our side, so we cannot prevent similar issues in the future
unfortunately. Adding tests for authentication is not a reasonable
option either.
2021-06-02 12:48:51 -07:00
Joey Perrott aeb407296b refactor(dev-infra): update to later version of @octokit/rest and remove class extenstion of Octokit (#42395)
Update @octokit/rest and remove the usage of a class extension of Octokit as the
class does not have a class define constructor.

PR Close #42395
2021-06-01 10:14:48 -07:00
Joey Perrott acfa0647b7 fix(dev-infra): build with stamping for release builds (#42354)
`ng-dev release build` should always build with version stamping rather than
snapshot stamping.

PR Close #42354
2021-05-26 17:51:56 +00:00
Joey Perrott 4afad03312 feat(dev-infra): add `ng-dev build-and-link` command (#42319)
Add a command to build the release output without stamping for release
and link via `yarn link` the generated builds to a project provided.

PR Close #42319
2021-05-25 22:11:03 +00:00
Joey Perrott 594e63315e feat(dev-infra): add release notes generation to ng-dev (#42225)
Adds tooling to create ad-hoc release note entries via `ng-dev release notes`.

PR Close #42225
2021-05-25 18:00:04 +00:00
Joey Perrott a9584c929b refactor(dev-infra): move the release notes into its own directory under release (#42225)
Move the release notes tooling under its own directory under release rather than
within publish, in preparation to have a release note generation command for ad-hoc
release note generation.

PR Close #42225
2021-05-25 18:00:04 +00:00
Paul Gschwendtner c3cbfc22e0 fix(dev-infra): format command incorrectly prints outdated files (#42252)
When `ng-dev format --check` is run, the ng-dev tool prints out
all files that are out-of-date. We recently updated the format
tool to also capture `stderr` for failed files. This broke the
console message as we did not unwrap the `FormatFailure` to
their file path when printing the "ng-dev format" fix command.

PR Close #42252
2021-05-25 17:57:01 +00:00
Joey Perrott 4ac55ca676 fix(dev-infra): use template strings for release note templates (#42224)
Use template strings for release note templates rather than loading `ejs` files at runtime.

PR Close #42224
2021-05-24 14:48:02 +00:00
Paul Gschwendtner 8c158babdc fix(dev-infra): formatting errors not reported with failure message (#42202)
Currently if formatting for a file fails due a formatter error,
the `ng-dev` tool reports that formatting failed, but no actual
error (or involved file) is printed out. This commit prints out
the failed files with their error message.

PR Close #42202
2021-05-21 18:06:47 +00:00
Paul Gschwendtner 9d75687f62 feat(dev-infra): publish major versions to "next" NPM dist tag (#42133)
Previously, the dev-infra release tool would publish major versions
directly to the NPM `@latest` dist tag. This is correct in theory, but
rather unpractical given that we want to publish packages first as
`@next` so that other dependent Angular packages can update too,
allowing us to publish all main Angular packages (from FW, COMP
and TOOL) at the same time to `@latest` on NPM.

This involves creating a new release action for re-tagging the
previously released major as `@latest` on NPM.

PR Close #42133
2021-05-18 14:04:14 -07:00
Paul Gschwendtner 48f49bacb4 refactor(dev-infra): improve type safety of NPM dist tags in release tool (#42133)
Instead of passing `string` in the release tool for NPM dist tags, we
should use a union string type that limits the tags to `latest`, `next`
and anything matching `v{number}-lts`. This avoids mistakes at
compilation-level if an invalid/unknown tag would be set by a release
action.

PR Close #42133
2021-05-18 14:04:14 -07:00
Joey Perrott 6d87895ffd test(dev-infra): add testing for integration of ReleaseNotes in publishing tooling (#41967)
Add testing of the ReleaseNotes integration with the release publishing tooling.

PR Close #41967
2021-05-14 21:07:27 +00:00
Paul Gschwendtner 7fb8527e1e refactor(dev-infra): remove invoke bazel clean command from release tool (#42101)
Currently the ng-dev release tool always run `bazel clean` before
calling the configured build release function. The clean is necessary
to ensure the release output is actually built; and not restored
from previous builds which could have different bazel workspace
status variables (which provide the NPM package version).

Instead of doing this as part of the release tool, the
actual script running to build the release output should
run the `bazel clean`. The release tool does not intend to
know about details on how the release output is built. This
is necessary because the build setup could vary between version
branches (especially for older ones; such as LTS version branches).

PR Close #42101
2021-05-14 21:06:09 +00:00
Paul Gschwendtner 2843f15e8c fix(dev-infra): merge tool should ensure that token has `workflow` oauth scope (#41989)
Currently if a PR modifies any file that configures a Github action
(e.g. a workflow file), the caretaker might face an error when merging
such PR:

```
! [remote rejected]       merge_pr_target_11.2.x -> 11.2.x (refusing to allow a Personal Access Token to create or update workflow
```

This happens because Github requires the token being used for the
push operation to have the `workflow` scope set. This is a special
scope added by Github to ensure that no changes can be made on
upstream branches that might expose the `GITHUB_TOKEN` environment
variable, which is available for push builds and could cause the
token being leaked.

With this commit we enforce that the caretaker adds the workflow
scope to their github token. Since PRs can only be merged if reviewed
thoroughly, it's acceptable to allow workflow file changes being
merged through the merge tool by the caretaker (especially since we
also allow CircleCI config files being merged with the default
`repo`/`public_repo` scope).

PR Close #41989
2021-05-07 14:10:39 -04:00
Joey Perrott e1c5cea2e7 fix(dev-infra): set the default LogLevel of GitClient logging to DEBUG (#41899)
Previously by default GitClient would log the commands it was executing at the
INFO level. This change moves the default level of this logging to DEBUG, while
still allowing callers of the methods to set the log level back to INFO.

PR Close #41899
2021-05-07 10:15:20 -04:00
Joey Perrott 4d7294d9f7 feat(dev-infra): include release notes in Github release entries when publishing (#41964)
Include the release notes entry for the release in the release created on Github.

PR Close #41964
2021-05-06 15:45:48 -04:00
Joey Perrott 393ce94718 feat(dev-infra): Set up new common release notes generation tooling (#41905)
Enables the new common release notes generation within the ng-dev release publishing
tooling.

PR Close #41905
2021-05-06 09:40:29 -04:00
Paul Gschwendtner 530e45f093 fix(dev-infra): publish script python check not working on windows (#41943)
We recently added a sanity check that ensures that `python` is
available for Bazel. The check relies on `/usr/bin/python` to check
if python is available w/ the correct version. The problem is
that on Windows there is no guarantee that the `/usr/bin` folder
exists, so the `ng-dev release publish` command always fails.

We fix this by just accessing the `env` binary that is also consulted
by scripts executed within Bazel actions. The `env` binary can be
assumed exist in the shell's `$PATH` and can point us to Python
as if it would be executed within Bazel.

PR Close #41943
2021-05-05 13:46:03 -07:00
Paul Gschwendtner 21c2a06811 fix(dev-infra): spawned child processes messing with tty output (#41948)
Currently we have a common utility method for running commands
in a child process. This method pipes all stdout and stderr, but sets
the `stdin` to `inherited`. This seemed to work as expected in terms of
allowing interactive commands being executed, but it messes with the
TTY in Windows (and potentially other platforms) so that colors and
prompts no longer work properly. See attached screenshot.

We fix this by not inheriting the stdin by default; but exposing
a dedicated method for interactive commands. This results in more
readable and obvious code too, so it's worth making this change
regardless of the TTY issues.

PR Close #41948
2021-05-05 13:45:28 -07:00
Paul Gschwendtner 5947239f89 fix(dev-infra): do not set lts dist tag on packages from release-candidate train (#41946)
Currently if a major release-train in the `release-candidate`/`feature-freeze`
phase becomes `latest`, we intend to set the NPM LTS dist tag for all packages
of the previous major (as the old release train in `latest` moves into LTS phase).

The logic for this exists but the release tool sets the NPM dist tag for
all packages of the new major. This means that the script might error if
a new package is part of the new major; or it could cause a deleted
package to not receive the LTS tag properly.

PR Close #41946
2021-05-04 21:07:57 -07:00
Zach Arend fdf2e02bff build: fix crash when validating commit messages (#41949)
Fixes a crash when running the `ValidateFileModule`.

PR Close #41949
2021-05-04 21:05:18 -07:00
Joey Perrott c0694bf538 fix(dev-infra): remove getBaseDir method from GitClient (#41898)
Remove the getBaseDir method from GitClient as the baseDir is determined
once during construction and does not need to be rediscovered.  The property
access should always be used.

PR Close #41898
2021-04-30 14:33:59 -07:00
George Kalpakas fff2ca540f fix(dev-infra): correctly detect prettier config (#41860)
This commit uses the correct property name (`prettier` vs `pretter`) to
check whether prettier is enabled via the config.

(It also fixes some typos in `dev-infra`.)

PR Close #41860
2021-04-28 09:21:09 -07:00
Joey Perrott 0140d04ced feat(dev-infra): create the prettier formatter for ng-dev format tooling (#41824)
Add support to use prettier for formatting using `ng-dev format`.

PR Close #41824
2021-04-27 12:23:05 -07:00
Joey Perrott b1ae31c712 fix(dev-infra): add github token requirement for discover-new-conflicts (#41825)
Add the github token option/requirement for the pr discover-new-conflicts
command.

PR Close #41825
2021-04-27 10:23:16 -07:00
Joey Perrott a4a55f0687 fix(dev-infra): verify python3 is available before release (#41753)
Verify that the /usr/bin/python points to the python3 interpreter binary.

PR Close #41753
2021-04-21 17:31:22 -07:00