Commit Graph

196 Commits

Author SHA1 Message Date
Paul Gschwendtner c1b47bcfaa refactor(dev-infra): expose logic for dealing with LTS branches (#38656)
Exposes logic for dealing with LTS branches, so that the release
tool can re-use it for cutting LTS patch releases.

Eventually, we can move all of this logic to a more dedicated
folder instead of having it inside the merge folder.

PR Close #38656
2020-09-28 16:11:40 -04:00
Paul Gschwendtner d7ff8d765c refactor(dev-infra): use shared github repo interface (#38656)
Instead of maintaining multiple interface for grouping
owner name and repo name, we expose a shared interface
describing a Github repository.

One unfortunate downside is that the GraphQL Github
and Rest API diverge slightly with the key for the
repository name. i.e. rest uses `repo` for the name
of a repository, while GraphQL uses `name` for the name.

If that would be consistent, we could use the rest operator
to pass a repository to the Octokit REST or GraphQL API. This
does not work, so we have a small manual overhead as seen
in the `branches.ts` file.

PR Close #38656
2020-09-28 16:11:40 -04:00
Paul Gschwendtner 3a598cf5ed build: add tsconfig with strict flag to dev-infra package (#38656)
The dev-infra package is currently built with Bazel and ts-node.
In Bazel, the shared tsconfig from the `packages/` folder is used.

This means that the code is built in strict mode, but IDEs and
ts-node do not know about the strictness. This is because the tsconfig
is part of the `packages` folder and not accessible from the
dev-infra package. We fix this by adding an IDE and ts-node specific
tsconfig to the dev-infra package.

This helps with spotting compilation failures before building
with Bazel / waiting for CI to check build state.

PR Close #38656
2020-09-28 16:11:39 -04:00
Paul Gschwendtner b041c118e3 refactor(dev-infra): expose version for determined release trains (#38656)
Previously, the logic for determing the active release trains did not
return the resolved version of a release train. With the publish script
being created, we need this information and can just pass it through,
so that we do not need to fetch and parse the package.json of given
branches multiple times.

PR Close #38656
2020-09-28 16:11:39 -04:00
Paul Gschwendtner 6af638d58e build: set up ora for progress spinners in dev-infra package (#38656)
Sets up the NPM `ora` package in the project and in dev-infra,
so that we can show progress spinners when needed. This is useful
in the publish release script when we wait for a pull request to
be merged.

PR Close #38656
2020-09-28 16:11:39 -04:00
Paul Gschwendtner 9e0e763156 refactor(dev-infra): do not print git commands in silent mode (#38656)
The git client respects the `SpawnSyncOptions` when a command
is executed. Currently it does not hide the command info
messages when commands are run in silent mode.

We fix this as part of this commit, so that the command info
is only printed to `debug` if `stdio` is set to `ignore`.

Additonally, the github token is made public so that it can be
used by commands if other repositories like forks are targeted.

PR Close #38656
2020-09-28 16:11:39 -04:00
Paul Gschwendtner 758d0e2045 refactor(dev-infra): share more github code between commands (#38656)
Instead of repeating the logic for adding the github token to
a repository git url, we add a shared function for automatically
computing the URls with token.

Additionally, URLs for updating/generating tokens have been moved
to a dedicated file in the `utils` folder. Also while being at it,
the yargs github token helper is also moved into the dedicated
Git/Github related util folder.

PR Close #38656
2020-09-28 16:11:38 -04:00
Wagner Maciel b1682526dd refactor(dev-infra): simplify runBenchmark (#38941)
* Make url and params optional in runBenchmark
* Make url optional in openBrowser
* Remove unused code from runBenchmark

PR Close #38941
2020-09-22 15:05:34 -07:00
Andrew Kushnir 2d6105a784 feat(dev-infra): output the number of new and fixed cycles (#38805)
This commit adds a logic to ouput the number of new and fixed cycles after running circular
dependency checker. This information is useful to better understand an impact of changes in case
the number of new/fixed cycles is relatively big.

PR Close #38805
2020-09-18 11:20:08 -07:00
Joey Perrott 55485713a3 fix(dev-infra): skip husky git commit hooks while merging pull requests (#38888)
During the merge process, all validations have already been completed so git commit
hooks can be safely skipped.  This additionally, prevents errors from occuring which
would be caused the commit hooks executing, such as when yarn updates and then yarn
commands are unable to run within the same process.

PR Close #38888
2020-09-18 11:14:55 -07:00
Joey Perrott 722699fb0c fix(dev-infra): handle no caretaker config being defined (#38862)
Properly handle cases where no caretaker config is provided to ng-dev commands

PR Close #38862
2020-09-16 15:22:10 -07:00
Joey Perrott d192c87f6a refactor(dev-infra): refactor commit-message files (#38845)
Refactor the commit-message files to be consistent with how other ng-dev tooling
is structured.

PR Close #38845
2020-09-15 16:05:42 -07:00
Joey Perrott b9ca6d20cc feat(dev-infra): include CI status check in the caretaker check (#38779)
Add a CI status check in the ng-dev caretaker check command.

PR Close #38779
2020-09-15 08:45:02 -07:00
Joey Perrott e00535a2a4 fix(dev-infra): remove ANSI escape codes from log file outputs (#38792)
Remove the ANSI codes from the log file outputs to make the ng-dev log files
more readable.

PR Close #38792
2020-09-15 08:44:00 -07:00
Joey Perrott 78e1ecb161 feat(dev-infra): allow local ng-dev configuration to error on invalid commit messages (#38784)
As part of the commit message conformance check, local commit message checks are
made to be warnings rather than failures. An additional local option is also in
place to allow for the commit message validation failures to be considered errors
instead.

PR Close #38784
2020-09-14 14:34:50 -07:00
Paul Gschwendtner e162da0753 refactor(dev-infra): do not validate config file multiple times (#38808)
Currently we validate the configuration file on each `getConfig`
invocation. We can only validate once since the configuration
is cached.

Also while being at it, renames the cache variables to lower-case as those
do not represent constants (which are convention-wise upper case).

PR Close #38808
2020-09-14 08:34:15 -07:00
Joey Perrott 6768fe9927 fix(dev-infra): correct build setup for dev-infra (#38815)
Correct's the missing dependencies and incorrect build dependencies for the
dev-infra bazel targets.

PR Close #38815
2020-09-11 13:59:30 -07:00
Wagner Maciel da14b72550 refactor(dev-infra): add default params to runBenchmark (#38748)
* Use '' as the default for 'url'
* Use [] as the default for 'params'
* Use true as the default for 'ignoreBrowserSynchronization'

PR Close #38748
2020-09-11 08:44:35 -07:00
Joey Perrott 1373a98e25 feat(dev-infra): Allow local user ng-dev configuration to disable commit message wizard (#38701)
As not all users, particularly contributors consistently contributing with a deep
understanding of our commit message guidelines, will not want to rely on the
commit message wizard, we allow a user to opt out of using this wizard during
commit message creation.

PR Close #38701
2020-09-09 16:31:17 -07:00
Joey Perrott 1ed6913b8b feat(dev-infra): Add support for local user ng-dev configuration (#38701)
Create a utility for loading a local user configuration object to describe
local configuration values, such as skipping the commit message wizard.

PR Close #38701
2020-09-09 16:31:16 -07:00
Joey Perrott f9421184ef refactor(dev-infra): update commit message validation to return validation result (#38703)
Previously, the validateCommitMessage function ran validation and logged the results.
The validateCommitMessage function now returns an object containing the validation
results and the cli action functions are instead responsible for logging the results.

This is being done as a prefactor for a change which allows for commit message
hook validation to be either a blocking error or a warning.

PR Close #38703
2020-09-09 16:22:36 -07:00
Joey Perrott d896c33b0e fix(dev-infra): set build commit message type to allow an optional scope (#38745)
Allow, optionally, a scope to be used with the build type commit message.

PR Close #38745
2020-09-09 09:06:33 -07:00
Joey Perrott a29f9a9fb3 fix(dev-infra): change logging of commit message restoration to debug (#38704)
Use debug level of logging for messages in commit message restoration.

PR Close #38704
2020-09-08 10:07:03 -07:00
Ikko Ashimine 2cb3d58b42 docs(dev-infra): fix typo in comment (arguements --> arguments) (#38653)
PR Close #38653
2020-09-03 09:45:02 -07:00
Joey Perrott a6f3cd93a9 feat(dev-infra): check services/status information of the repository for caretaker (#38601)
The angular team relies on a number of services for hosting code, running CI, etc. This
tool allows for checking the operational status of all services at once as well as the current
state of the repository with respect to merge and triage ready issues and prs.

PR Close #38601
2020-09-01 13:05:30 -07:00
Joey Perrott 4f28192d62 refactor(dev-infra): use a mixin to require a `github-token` for an ng-dev command (#38630)
Creates a mixin for requiring a github token to be provided to a command.  This mixin
allows for a centralized management of the requirement and handling of the github-token.

PR Close #38630
2020-08-31 12:32:27 -07:00
Paul Gschwendtner ebc0e46501 refactor(dev-infra): improve error message for unexpected version branches (#38622)
Currently the merge script default branch configuration throws an error
if an unexpected version branch is discovered. The error right now
assumes to much knowledge of the logic and the document outlining
the release trains conceptually.

We change it to something more easy to understand that doesn't require
full understanding of the versioning/labeling/branching document that
has been created for the Angular organization.

PR Close #38622
2020-08-31 09:29:58 -07:00
Joey Perrott 3487b549fd feat(dev-infra): write outputs of command runs to ng-dev log file (#38599)
Creates infrastructure to write outputs of command runs to ng-dev log file.
Additionally, on commands which fail with an exit code greater than 1, an
error log file is created with the posix timestamp of the commands run time
as an identifier.

PR Close #38599
2020-08-31 08:47:15 -07:00
George Kalpakas 364284b0dc fix(dev-infra): ignore comments when validating commit messages (#38438)
When creating a commit with the git cli, git pre-populates the editor
used to enter the commit message with some comments (i.e. lines starting
with `#`). These comments contain helpful instructions or information
regarding the changes that are part of the commit. As happens with all
commit message comments, they are removed by git and do not end up in
the final commit message.

However, the file that is passed to the `commit-msg` to be validated
still contains these comments. This may affect the outcome of the commit
message validation. In such cases, the author will not realize that the
commit message is not in the desired format until the linting checks
fail on CI (which validates the final commit messages and is not
affected by this issue), usually several minutes later.

Possible ways in which the commit message validation outcome can be
affected:
- The minimum body length check may pass incorrectly, even if there is
  no actual body, because the comments are counted as part of the body.
- The maximum line length check may fail incorrectly due to a very long
  line in the comments.

This commit fixes the problem by removing comment lines before
validating a commit message.

Fixes #37865

PR Close #38438
2020-08-21 12:17:14 -07:00
Joey Perrott f77fd5e02a feat(dev-infra): create a wizard for building commit messages (#38457)
Creates a wizard to walk through creating a commit message in the correct
template for commit messages in Angular repositories.

PR Close #38457
2020-08-18 17:01:14 -07:00
Joey Perrott 63ba74fe4e feat(dev-infra): tooling to check out pending PR (#38474)
Creates a tool within ng-dev to checkout a pending PR from the upstream repository.  This automates
an action that many developers on the Angular team need to do periodically in the process of testing
and reviewing incoming PRs.

Example usage:
  ng-dev pr checkout <pr-number>

PR Close #38474
2020-08-18 16:22:47 -07:00
Joey Perrott 301513311e refactor(dev-infra): update yargs and typings for yargs (#38470)
Updating yargs and typings for the updated yargs module.

PR Close #38470
2020-08-17 15:30:32 -07:00
Joey Perrott 9f7a37b4e9 feat(dev-infra): migrate to unified commit message types in commit message linting (#38430)
Previously commit message types were provided as part of the ng-dev config in the repository
using the ng-dev toolset.  This change removes this configuration expectation and instead
predefines the valid types for commit messages.

Additionally, with this new unified set of types requirements around providing a scope have
been put in place.  Scopes are either required, optional or forbidden for a given commit
type.

PR Close #38430
2020-08-13 09:11:17 -07:00
Paul Gschwendtner 773f7908c0 feat(dev-infra): update to latest benchpress version (#38440)
We recently updated the benchpress package to have a more loose
Angular core peer dependency, and less other unused dependencies.

We should make sure to use that in the dev-infra package so that
peer dependencies can be satisified in consumer projects, and so
that less unused dependencies are brought into projects.

PR Close #38440
2020-08-13 09:09:31 -07:00
Joey Perrott f4ced74e3a feat(dev-infra): save invalid commit message attempts to be restored on next commit attempt (#38304)
When a commit message fails validation, rather than throwing out the commit message entirely
the commit message is saved into a draft file and restored on the next commit attempt.

PR Close #38304
2020-08-13 08:45:25 -07:00
Joey Perrott 8366effeec refactor(dev-infra): extract the commit message parsing function into its own file (#38429)
Extracts the commit message parsing function into its own file.

PR Close #38429
2020-08-12 16:10:07 -07:00
Joey Perrott aa3520eb7d refactor(dev-infra): use promptConfirm util in ng-dev's formatter (#38419)
Use the promptConfirm util instead of manually creating a confirm prompt with
inquirer.

PR Close #38419
2020-08-12 11:25:10 -07:00
Joey Perrott 28534d83ee feat(dev-infra): Add support for formatting all staged files (#38402)
Adds an ng-dev formatter option to format all of the staged files. This will can
be used to format only the staged files during the pre-commit hook.

PR Close #38402
2020-08-11 16:32:54 -07:00
Paul Gschwendtner f0766a4474 feat(dev-infra): provide organization-wide merge-tool label configuration (#38223)
Previously, each Angular repository had its own strategy/configuration
for merging pull requests and cherry-picking. We worked out a new
strategy for labeling/branching/versioning that should be the canonical
strategy for all actively maintained projects in the Angular organization.

This PR provides a `ng-dev` merge configuration that implements the
labeling/branching/merging as per the approved proposal.

See the following document for the proposal this commit is based on
for the merge script labeling/branching: https://docs.google.com/document/d/197kVillDwx-RZtSVOBtPb4BBIAw0E9RT3q3v6DZkykU

The merge tool label configuration can be conveniently accesed
within each `.ng-dev` configuration, and can also be extended
if there are special labels on individual projects. This is one
of the reasons why the labels are not directly built into the
merge script. The script should remain unopinionated and flexible.

The configuration is conceptually powerful enough to achieve the
procedures as outlined in the versioning/branching/labeling proposal.

PR Close #38223
2020-08-05 10:53:17 -07:00
Paul Gschwendtner 6f0f0d3ea2 feat(dev-infra): support user-failures when computing branches for target label (#38223)
The merge tool provides a way for configurations to determine the branches
for a label lazily. This is supported because it allows labels to respect
the currently selected base branch through the Github UI. e.g. if `target: label`
is applied on a PR and the PR is based on the patch branch, then the change
could only go into the selected target branch, while if it would be based on
`master`, the change would be cherry-picked to `master` too. This allows for
convenient back-porting of changes if they did not apply cleanly to the primary
development branch (`master`).

We want to expand this function so that it is possible to report failures if an
invalid target label is appplied (e.g. `target: major` not allowed in
some situations), or if the Github base branch is not valid for the given target
label (e.g. if `target: lts` is used, but it's not based on a LTS branch).

PR Close #38223
2020-08-05 10:53:17 -07:00
Paul Gschwendtner 576e329f33 feat(dev-infra): provide github API instance to lazy merge configuration (#38223)
The merge script currently accepts a configuration function that will
be invoked _only_ when the `ng-dev merge` command is executed. This
has been done that way because the merge tooling usually relies on
external requests to Git or NPM for constructing the branch configurations.

We do not want to perform these slow external queries on any `ng-dev` command
though, so this became a lazily invoked function.

This commit adds support for these configuration functions to run
asynchronously (by returning a Promise that will be awaited), so that
requests could also be made to the Github API. This is benefical as it
could avoid dependence on the local Git state and the HTTP requests
are more powerful/faster.

Additionally, in order to be able to perform Github API requests
with an authenticated instance, the merge tool will pass through
a `GithubClient` instance that uses the specified `--github-token`
(or from the environment). This ensures that all API requests use
the same `GithubClient` instance and can be authenticated (mitigating
potential rate limits).

PR Close #38223
2020-08-05 10:53:17 -07:00
Andrew Scott 67e3ecc7e3 fix(dev-infra): Ensure conditions with groups do not fail verification (#37798)
There are a few changes in this PR to ensure conditions that are based
on groups (i.e. `- groups.pending.length == 0`) do not fail the verify
task:

* Remove the warning when a condition is encountered that depends on the
`groups` state. The warning will otherwise be printed once for every
file that triggers the execution of the condition (400,000+ times)
* Add an `unverifiable` flag to `GroupCondition` interface and set it to
true when an error is encountered due to attempting to get the state of
`groups` in a condition
* Ignore any unverifiable conditions when gathering unmatched
conditions. These should not be considered `unmatched` for verification
purposes.
* Print the unverifiable conditions by group in the results

Sample output:
```

┌──────────────────────────────────────────────────────────────────────────────┐
│                         PullApprove results by group                         │
└──────────────────────────────────────────────────────────────────────────────┘
Groups skipped (4 groups)
Matched conditions by Group (37 groups)
Unmatched conditions by Group (0 groups)
Unverifiable conditions by Group (3 groups)
  [public-api]
    len(groups.pending.exclude("required-minimum-review")...
    len(groups.rejected.exclude("required-minimum-review")...
  [size-tracking]
    len(groups.pending.exclude("required-minimum-review")...
    len(groups.rejected.exclude("required-minimum-review")...
  [circular-dependencies]
    len(groups.pending.exclude("required-minimum-review")...
    len(groups.rejected.exclude("required-minimum-review")...

```

PR Close #37798
2020-07-24 17:59:39 -07:00
Andrew Scott d0060dcfdc refactor(dev-infra): Add support for groups in the conditions evaluator (#38164)
Conditions can refer to the groups array that is a list of the preceding groups.
This commit adds support to the verification for those conditions.

This commit also adds some tests to the parsing and condition matching
to ensure everything works as expected.

PR Close #38164
2020-07-23 11:05:42 -07:00
Wagner Maciel 09eee5b312 fix(dev-infra): add missing BUILD file to dev-infra/bazel:files (#38026)
* Without this BUILD file we were seeing errors about the reference to
  expand_template.bzl in ng_rollup_bundle.bzl because dev-infra/bazel
  was not considered a package.

PR Close #38026
2020-07-15 12:34:46 -07:00
Wagner Maciel 904ebfebf3 fix(dev-infra): fix broken zone.js substitution for dev-infra:npm_package (#38026)
* fix substitution that was broken by PR #36540 to match
  the new import path

PR Close #38026
2020-07-15 12:34:46 -07:00
Paul Gschwendtner 545f1dbac9 feat(dev-infra): add bazel firefox browser with RBE compatibility (#38029)
Adds Firefox as browser to `dev-infra/browsers` with RBE
compatibility. The default Firefox browser is not compatible similar to
the default Chromium version exposed by `rules_webtesting`.

The Angular Components repository will use this browser target as
it enables RBE support. Also it gives us more flexibility about
the Firefox version we test against. The version provided by
`rules_webtesting` is very old and most likely not frequently
updated (based on past experience).

PR Close #38029
2020-07-15 12:34:05 -07:00
Paul Gschwendtner 69472a1ed0 refactor(dev-infra): allow for consumption with rules_nodejs v2.0.0 (#37968)
With `rules_nodejs` v2.0.0 being in RC phase currently, we should
make sure that the package is compatible so that we can use it
in the components repo in combination with rules_nodejs v2.0.0.

In v2.0.0 of the NodeJS rules, Bazel workspaces are recommended
to no longer be symlinked under a separate repository. Instead,
bazel rules and targets should be accessed directly from the
user-selected NPM repository. Usually `@npm`, so that the import
changes to `@npm//@angular/dev-infra-private/<..>`.

PR Close #37968
2020-07-13 14:18:22 -07:00
Paul Gschwendtner 9bd4b74b06 feat(dev-infra): commit message validation should skip lines consisting of URLs (#37890)
The dev-infra commit message validation optionally can check for lines
to not exceed a given amount of characters. This is desired for most
commit messages, but sometimes not actionable if a long URL is inserted
into the commit message. With this commit, we skip the max line length
check for lines that start with an URL.

PR Close #37890
2020-07-10 10:59:28 -07:00
Paul Gschwendtner 4b4b74548d fix(dev-infra): include `bazel` utility files in npm package (#37891)
We recently added a new folder for common bazel utilities
to `dev-infra`. The `ng_rollup_bundle` rule relies on an
utility that is provided by this `bazel/` folder.

Unfortunately though it looks like this folder is currently
not included in the NPM package, so that the `ng_rollup_bundle`
rule does not work as expected. This commit fixes that by
including the bazel utilities in the NPM package.

PR Close #37891
2020-07-10 10:06:11 -07:00
Paul Gschwendtner cb2ab26296 feat(dev-infra): merge script should link to original commit when cherry-picking with API strategy (#37889)
The merge script uses `git cherry-pick` for both the API merge strategy
and the autosquash strategy. It uses cherry-pick to push commits to
different target branches (e.g. into the `10.0.x` branch).

Those commits never point to the commits that landed in the primary
Github branch though. For the autosquash strategy the pull request number
is always included, so there is a way to go back to the source. On the other
hand though, for commits cherry-picked in the API merge strategy, the
pull request number might not always be included (due to Github's
implementation of the rebase merge method).

e.g.
27f52711c0

For those cases we'd want to link the cherry-picked commits to the
original commits so that the corresponding PR is easier to track
down. This is not needed for the autosquash strategy (as outlined
before), but it would have been good for consistency. Unfortunately
though this would rather complicate the strategy as the autosquash
strategy cherry-picks directly from the PR head, so the SHAs that
are used in the primary branch are not known.

PR Close #37889
2020-07-07 12:16:21 -07:00
Igor Minar c5b125b7db feat(dev-infra): add support for minBodyLengthTypeExcludes to commit-message validation (#37764)
This feature will allow us to exclude certain commits from the 100 chars minBodyLength requirement for commit
messages which is hard to satisfy for commits that make trivial changes (e.g. fixing typos in docs or comments).

PR Close #37764
2020-06-26 11:13:09 -07:00
Paul Gschwendtner 3ee666580a fix(dev-infra): merge script should not always require full repo permissions (#37718)
We recently added OAuth scope checking to the dev-infra Git client
and started leveraging it for the merge script. We set the `repo` scope
as required for running the merge script. We can loosen this requirement
as in the Angular org where the script is consumed, only pull requests on
public repositories are merged through the script.

This should help with reducing the risk with compromised tokens as no
access had to be granted on `repo:invite`, `repo_deployment` etc.

PR Close #37718
2020-06-26 09:52:33 -07:00
Paul Gschwendtner dbc2364d16 fix(dev-infra): support running scripts from within a detached head (#37737)
Scripts provided in the `ng-dev` command might use local `git`
commands. For such scripts, we keep track of the branch that
has been checked out before the command has been invoked.

We do this so that we can later (upon command completion)
restore back to the original branch. We do not want to
leave the Git repository in a dirty state.

It looks like this logic currently only deals with branches
but does not work properly when a command is invoked from
a detached head. We can make it work by just checking out
the previous revision (if no branch is checked out).

PR Close #37737
2020-06-26 09:51:09 -07:00
Wagner Maciel 5218916a7e feat(dev-infra): add a way to pass assets down to a benchmark application (#37695)
* add a param called ng_assets to the component_benchmark macro to allow static assets to be provided to the base angular app, not just through the ts_devserver

PR Close #37695
2020-06-25 11:51:51 -07:00
Paul Gschwendtner 1601ee6f6a refactor(dev-infra): ng_rollup_bundle rule should leverage `@bazel/rollup` (#37623)
Refactors the `ng_rollup_bundle` rule to a macro that relies on
the `@bazel/rollup` package. This means that the rule no longer
deals with custom ESM5 flavour output, but rather only builds
prodmode ES2015 output. This matches the common build output
in Angular projects, and optimizations done in CLI where
ES2015 is the default optimization input.

The motiviation for this change is:

* Not duplicating rollup Bazel rules. Instead leveraging the official
rollup rule.
* Not dealing with a third TS output flavor in Bazel.The ESM5 flavour has the
potential of slowing down local development (as it requires compilation replaying)
* Updating the rule to be aligned with current CLI optimizations.

This also _fixes_ a bug that surfaced in the old rollup bundle rule.
Code that is unused, is not removed properly. The new rule fixes this by
setting the `toplevel` flag. This instructs terser to remove unused
definitions at top-level. This matches the optimization applied in CLI
projects. Notably the CLI doesn't need this flag, as code is always
wrapped by Webpack. Hence, the unused code eliding runs by default.

PR Close #37623
2020-06-22 10:55:28 -07:00
Joey Perrott a8af8551ec refactor(dev-infra): migrate github api in GitClient to rely on GithubClient (#37593)
GitClient now uses GithubClient for github API interactions.  GithubClient is
a class which extends Octokit and provides a member which allows for GraphQL
requests against the Github GraphQL api, as well as providing convenience methods
for common/repeated Github API requests.

PR Close #37593
2020-06-22 10:54:43 -07:00
Paul Gschwendtner 53e95233c3 refactor(dev-infra): cleanup shared package dependencies (#37594)
Cleans up the dependencies used in the shared dev-infra package
configuration. With the recent benchmarking utilities that have
been added, a lot of peer dependencies have been added.

We decided that we don't want to list every used dependencies as
peer dependency as that could result in unnecessary churn/noise
for consumers of the dev-infra package. Additionally, not all parts
of the dev-infra package are necessarily used.

Due to this, we want to apply the following rules for the package
dependencies:

1. If a dependency is only used in a shipped Bazel macro/rule that can be
optionally consumed, omit it from `package.json`. Bazel reports the
missing dependency on its own, so we want to avoid adding it to the
package json file.

2. Otherwise, if the dependency is large and commonly used (like
buildifier), add it to the `peerDependencies`. If not, add it
to the dependencies that are always brought in. We consider it
as acceptable to bring in a few small dependencies that might not
be used or not. Making all of those option would complicate the
use of the dev-infra package.

ds

PR Close #37594
2020-06-18 16:03:14 -07:00
Paul Gschwendtner 879a674420 refactor(dev-infra): use shelljs instead of fs-extra for benchmark utils (#37594)
We added a new dependency on `fs-extra` to the dev-infra package. We can
remove this dependency and replace it with `shelljs` that is extensively
used in other places already.

The motiviation is that we can reduce dependencies needed for
for consumption of the shared dev-infra package.

PR Close #37594
2020-06-18 16:03:14 -07:00
Paul Gschwendtner 8154bbd538 feat(dev-infra): support for caretaker note label in merge script (#37595)
Adds support for a caretaker note label to the merge script.
Whenever a configured label is applied, the merge script will
not merge automatically, but instead prompt first in order
to ensure that the caretaker paid attention to the manual
caretaker note on the PR. This helps if a PR needs special
attention.

PR Close #37595
2020-06-16 11:58:55 -07:00
Joey Perrott cc37af2314 fix(dev-infra): allow for deep merging of pullapprove config aliases (#36915)
Set the yaml parser to support deep merges of yaml aliases,
to support having a default value for all rules to build upon.

PR Close #36915
2020-06-15 14:29:32 -07:00
Joey Perrott b45acb8d32 feat(dev-infra): add `#` to outputed list of PRs for discover-new-conflicts (#37556)
Adding in a `#` prepended to each PR number in the list of conflicting PRs
found by the discover-new-conflicts script will allow for users to copy
paste the output from the script into a github comment and have the PRs
automatically link.

PR Close #37556
2020-06-12 15:08:53 -07:00
JiaLiPassion 583a9d38a1 feat(zone.js): upgrade zone.js to angular package format(APF) (#36540)
Close #35157

In the current version of zone.js, zone.js uses it's own package format, and it is not following the rule
of Angualr package format(APF), so it is not easily to be consumed by Angular CLI or other bundle tools.
For example, zone.js npm package has two bundles,

1. zone.js/dist/zone.js, this is a `es5` bundle.
2. zone.js/dist/zone-evergreen.js, this is a `es2015` bundle.

And Angular CLI has to add some hard-coding code to handle this case, o5376a8b139/packages/schematics/angular/application/files/src/polyfills.ts.template (L55-L58)

This PR upgrade zone.js npm package format to follow APF rule, https://docs.google.com/document/d/1CZC2rcpxffTDfRDs6p1cfbmKNLA6x5O-NtkJglDaBVs/edit#heading=h.k0mh3o8u5hx

The updated points are:

1. in package.json, update all bundle related properties

```
  "main": "./bundles/zone.umd.js",
  "module": "./fesm2015/zone.js",
  "es2015": "./fesm2015/zone.js",
  "fesm2015": "./fesm2015/zone.js",
```

2. re-organize dist folder, for example for `zone.js` bundle, now we have

```
  dist/
      bundles/
             zone.js            // this is the es5 bundle
      fesm2015/
             zone.js            // this is the es2015 bundle (in the old version is `zone-evergreen.js`)
```

3. have several sub-packages.

1. `zone-testing`, provide zone-testing bundles include zone.js and testing libraries
2. `zone-node`, provide zone.js implemention for NodeJS
3. `zone-mix`, provide zone.js patches for both Browser and NodeJS

All those sub-packages will have their own `package.json` and the bundle will reference `bundles(es5)` and `fesm2015(es2015)`.

4. keep backward compatibility, still keep the `zone.js/dist` folder, and all bundles will be redirected to `zone.js/bundles` or `zone.js/fesm2015` folders.

PR Close #36540
2020-06-11 11:08:48 -07:00
Paul Gschwendtner a3313709ea fix(dev-infra): local changes check not working (#37489)
Looks like we broke the `hasLocalChanges` check in the git client
when we moved it over from the merge script. The problem is that
we are using `git` in the first argument of `git.run`. That
means that we under-the-hood run `git git <..>`.

This commit fixes that, but also switches to a better variant
for ensuring no local changes because it exits with non-zero
when there are local changes.

PR Close #37489
2020-06-09 08:27:32 -07:00
Paul Gschwendtner 0eacba5829 fix(dev-infra): rebase pr script not working (#37489)
The dev-infra rebase PR script currently does not work due to
the following issues:

1. The push refspec is incorrect. It refers to the `base` of the PR, and
not to the `head` of the PR.
2. The push `--force-with-lease` option does not work in a detached head
as no remote-tracking branch is set up.

PR Close #37489
2020-06-09 08:27:32 -07:00
Paul Gschwendtner bb924b63e6 fix(dev-infra): incorrect token sanitization when no token is specified (#37489)
We recently moved over the git client from the merge script to the
common dev-infra utils. This made specifying a token optional, but
it looks like the logic for sanitizing messages doesn't account
for that, and we currently add `<TOKEN>` between every message
character. e.g.

```
Executing: git <TOKEN>g<TOKEN>i<TOKEN>t<TOKEN>
<TOKEN>s<TOKEN>t<TOKEN>a<TOKEN>t<TOKEN>u<TOKEN>s<TOKEN>
```

PR Close #37489
2020-06-09 08:27:31 -07:00
Joey Perrott ed3c549063 ci: remove IgorMinar from reviewers list for pullapprove fallback group (#36456)
Historically we have had a pullapprove group `fallback` which acted as
a catch all for files which did not match any other groups.  This
group assigned reviews to IgorMinar, however it was not apparent that
this group was assigned.  This change removes this assignment.  This
group as active should always coincide with failures of the pullapprove
verification script. We continue to have this group as a secondary test
ensuring all files in the repo are captured by the pullapprove config.

PR Close #36456
2020-06-08 10:07:45 -07:00
Wagner Maciel 77efddcac3 fix(dev-infra): await setup in runBenchmark (#37428)
* Fix for issue #36986.
* Changes runBenchmark into an async function.
* Awaits config.setup in runBenchmark.

PR Close #37428
2020-06-08 09:17:34 -07:00
Greg Magolan d887ba85ad build: update to latest stable Chromium 83.0.4103 in both rules_webtesting and puppeteer (#37427)
Also added in detailed instructions of the process to determine the URLs corresponding to Chromium version desired

PR Close #37427
2020-06-08 09:16:40 -07:00
Paul Gschwendtner 17996bf719 fix(dev-infra): properly determine oauth scopes for git client token (#37462)
Resubmit of b2bd38699b since
85b6c94cc6 accidentally reverted
the fix due to rebasing most likely.

PR Close #37462
2020-06-05 11:04:36 -07:00
Joey Perrott 85b6c94cc6 refactor(dev-infra): move GitClient to common util (#37318)
Moves GitClient from merge script into common utils for unified
method of performing git actions throughout the ng-dev toolset.

PR Close #37318
2020-06-05 09:46:40 -07:00
George Kalpakas d4c0962c7b refactor(dev-infra): use the `exec()` helper from `utils/shelljs` whenever possible (#37444)
There is an `exec()` helper provided by `utils/shelljs.ts`, which is a
wrapper around ShellJS' `exec()` with some default options (currently
`silent: true`). The intention is to avoid having to pass these options
to every invocation of the `exec()` function.

This commit updates all code inside `dev-infra/` to use this helper
whenever possible).

NOTE: For simplicity, the `utils/shelljs` helper does not support some
      of the less common call signatures of the original `exec()`
      helper, so in some cases we still need to use the original.

PR Close #37444
2020-06-05 09:21:18 -07:00
Joey Perrott 4e7a4543b8 style(dev-infra): correct tslint failures in dev-infra directory (#37233)
Fixes tslint failures in dev-infra directory as the directory is now
part of the tslint enforced files.

PR Close #37233
2020-06-04 12:44:46 -07:00
Joey Perrott dffcca73e4 fix(dev-infra): clean up usages within pullapprove tooling (#37338)
Clean up pullapprove tooling to use newly created common utils.
Additionally, use newly created logging levels rather than
verbose flagging.

PR Close #37338
2020-06-04 12:43:44 -07:00
Paul Gschwendtner b2bd38699b fix(dev-infra): properly determine oauth scopes for git client token (#37439)
We recently added a better reporting mechanism for oauth tokens
in the dev-infra git util. Unfortunately the logic broke as part
of addressing PR review feedback. Right now, always the empty
promise from `oauthScopes` will be used as `getAuthScopes` considers
it as the already-requested API value. This is not the case as
the default promise is also truthy. We should just fix this by making
the property nullable.

PR Close #37439
2020-06-04 10:42:53 -07:00
Joey Perrott c025357fb8 feat(dev-infra): Add oauth scope check to ensure necessary permissions for merge tooling (#37421)
Adds an assertion that the provided TOKEN has OAuth scope permissions for `repo`
as this is required for all merge attempts.

On failure, provides detailed error message with remediation steps for the user.

PR Close #37421
2020-06-04 09:35:59 -07:00
Wagner Maciel 40f3bb5638 refactor(dev-infra): small changes and fixes (#36800)
Rename bazel workspace from npm_dev_infra to npm_angular_dev_infra_private to make it clear that this package is private to angular.
Change driver-utilities module_name to match the new bazel workspace name.
Correct a comment by rewording it from "deployed version" to "published version".
Fix merge conflicts in tmpl-package.json
Make "//packages/bazel/src:esm5.bzl" replacement more generalized so that importing from "//packages/bazel" works.
Deleted "dev_infra/*" path from modules/benchmarks tsconfig.
Moved //dev-infra/benchmark/browsers to //dev-infra/browsers.

PR Close #36800
2020-06-03 13:12:30 -07:00
Wagner Maciel 64d6f35f94 revert: "revert: "build(dev-infra): update package.json and :npm_package (#36434)" (#36798)" (#36800)
This reverts commit f5ff2068a4.

PR Close #36800
2020-06-03 13:12:30 -07:00
Wagner Maciel caa4ab3235 revert: "revert: "feat(dev-infra): exposed new rule 'component_benchmark' via dev_infra (#36434)" (#36798)" (#36800)
This reverts commit ad8c4cdd75.

PR Close #36800
2020-06-03 13:12:30 -07:00
Joey Perrott 3d154ab795 fix(dev-infra): migrate dev-infra config loading to .ng-dev directory (#37142)
Migrate to using .ng-dev directory for ng-dev configuration to better
allow management of the configuration using multiple files.  The
intention is to prevent the config file from becoming unruly.

PR Close #37142
2020-05-26 16:38:22 -04:00
Joey Perrott d1ea1f4c7f build: update license headers to reference Google LLC (#37205)
Update the license headers throughout the repository to reference Google LLC
rather than Google Inc, for the required license headers.

PR Close #37205
2020-05-26 14:26:58 -04:00
Joey Perrott 540c29cd6b feat(dev-infra): migrate release tool to use new logging system (#37232)
Migrate the release tool in ng-dev to use new logging system rather
than directly calling console.* to create a better experience
for users.

PR Close #37232
2020-05-21 17:40:35 -04:00
Joey Perrott 7827501d2a feat(dev-infra): migrate ts-circular-dependencies tool to use new logging system (#37232)
Migrate the ts-circular-dependencies tool in ng-dev to use new logging system rather
than directly calling console.* to create a better experience
for users.

PR Close #37232
2020-05-21 17:40:35 -04:00
Joey Perrott e28f13a102 feat(dev-infra): migrate merge tool to use new logging system (#37232)
Migrate the merge tool in ng-dev to use new logging system rather
than directly calling console.* to create a better experience
for users.

PR Close #37232
2020-05-21 17:40:35 -04:00
Joey Perrott e0002ef9ba feat(dev-infra): migrate ng-dev utils to use new logging system (#37232)
Migrate the ng-dev utils to use new logging system rather
than directly calling console.* to create a better experience
for users.

PR Close #37232
2020-05-21 17:40:35 -04:00
Joey Perrott 367f6fe12d feat(dev-infra): migrate pullapprove tool to use new logging system (#37232)
Migrate the pullapprove tool in ng-dev to use new logging system rather
than directly calling console.* to create a better experience
for users.

PR Close #37232
2020-05-21 17:40:35 -04:00
Joey Perrott 9b3ca19675 feat(dev-infra): migrate rebase tool to use new logging system (#37232)
Migrate the rebase tool in ng-dev to use new logging system rather
than directly calling console.*  to create a better experience
for users.

PR Close #37232
2020-05-21 17:40:35 -04:00
Joey Perrott 9388e47e1e feat(dev-infra): migrate discover-new-conflicts tool to use new logging system (#37232)
Migrate the discover-new-conflicts tool in ng-dev to use new logging system
rather than directly calling console.* to create a better experience
for users.

PR Close #37232
2020-05-21 17:40:35 -04:00
Joey Perrott 171d967993 feat(dev-infra): migrate commit-message tool to use new logging system (#37232)
Migrate the commit-message tool in ng-dev to use new logging system rather
than directly calling console.* to create a better experience
for users.

PR Close #37232
2020-05-21 17:40:35 -04:00
Joey Perrott 40e357411e feat(dev-infra): migrate format tool to use new logging system (#37232)
Migrate the formatting tool in ng-dev to use new logging system rather
than directly calling console.* to create a better experience
for users.

PR Close #37232
2020-05-21 17:40:35 -04:00
Joey Perrott 52c7aae4df feat(dev-infra): add group functions to logging system and remove color param (#37232)
Adds .group and .groupEnd functions to each of the logging functions
to allow creating groups in the logged output.  Additionally removes
the color parameter from logging functions, in favor of the color
being applied to the string at the call site.

PR Close #37232
2020-05-21 17:40:34 -04:00
Paul Gschwendtner cdb5d07606 feat(dev-infra): expose script for determining merge branches (#37217)
The components repo and framework repository follow the same patch
branch concept. We should be able to share a script for determining
these merge branches.

Additonally the logic has been improved compared to the old merge script because
we no longer consult `git ls-remote` unless really needed. Currently,
`git ls-remote` is always consulted, even though not necessarily needed.

This can slow down the merge script and the caretaker process when a
couple of PRs are merged (personally saw around ~4 seconds per merge).

Additionally, the new logic is more strict and will ensure (in most
cases) that no wrong patch/minor branch is determined. Previously,
the script just used the lexicographically greatest patch branch.
This _could_ be wrong when a new patch branch has been created too
early, or by accident.

PR Close #37217
2020-05-21 10:35:23 -07:00
Paul Gschwendtner 3e5fa56956 fix(dev-infra): ensure ts-node is registered with commonjs as module (#37217)
We recently added support for automatic registration of `ts-node`
when the dev-infra configuration is loaded.

In addition to registering ts-node, we should also ensure that the
`commonjs` module is set up. By default, `ts-node` would use ES module
imports that are not supported by default in NodeJS.

PR Close #37217
2020-05-21 10:35:23 -07:00
Joey Perrott 59e7203a40 build: complete removal of bazel format yarn commands (#37148)
Remove bazel yarn format deprecation message to complete the removal of
formatting bazel related files via yarn command command.

PR Close #37148
2020-05-20 13:32:40 -07:00
Joey Perrott 7db177d181 feat(dev-infra): create logging utils with support for defined logging levels (#37192)
Creates common logging functions at different levels. Allows for providing
logging statements which are actually printed to the console based on the
LOG_LEVEL environment variable.

PR Close #37192
2020-05-20 13:32:04 -07:00
Joey Perrott 362fba4bde fix(dev-infra): make commit-message-filter script executable for merge (#37209)
Marks the commit-mesage-filter.js file as executable, allowing it
to be run by 'git filter-branch' during the merge script process.

PR Close #37209
2020-05-19 15:35:27 -07:00
Paul Gschwendtner 383f04b96d feat(dev-infra): register ts-node when reading configuration (#37196)
`ts-node` is now an optional peer dependency of the shared dev-infra
package. Whenever a `ng-dev` command runs, and a TypeScript-based
configuration file exists, `ts-node` is set up if available.

That allows consumers of the package (as the components repo) to more
conveniently use a TypeScript-based configuration for dev-infra.

Currently, commands would need to be proxied through `ts-node`
which rather complicates the setup:

```
NG_DEV_COMMAND="ts-node ./node_modules/@angular/dev-infra-private/cli.js"
```

I'm thinking that it should be best-practice to use TypeScript for
writing the configuration files. Given that the tool is used primarily
in Angular projects (for which most sources are TypeScript), this should
be acceptable.

PR Close #37196
2020-05-19 15:21:57 -07:00
Joey Perrott 009f394237 feat(dev-infra): create rebase-pr tool in ng-dev (#37055)
Creates a tool in ng-dev which rebases a PR automatically and pushes
the rebase commits back to the PR.  This is meant to be a replacement
to the local merge script currently in the repo and currently has
feature parity.

PR Close #37055
2020-05-18 16:18:32 -07:00
Paul Gschwendtner c2c14c2a1f fix(dev-infra): incorrect base revision for merge script autosquash (#37138)
As mentioned in the previous commit, the autosquash strategy has
not been used in the components repo, so we could easily regress.

After thorough manual testing of the autosquash strategy again,
now that the merge script will be moved to framework, it came
to mind that there is a bug with the base revision in the
autosquash merge strategy. The problem is that the base revision
of a given PR is relying on the amount of commits in a PR.

This is prone to error because the amount of commits could easily
change in the autosquash merge strategy, because fixup or squash
commits will be collapsed. Basically invalidating the base revision.

To fix this, we fixate the base revision by determining the actual
SHA. This one is guaranteed to not change after the autosquash rebase.

The current merge script in framework fixates the revision by creating
a separate branch, but there is no benefit in that, compared to just
using an explicit SHA that doesn't need to be cleaned up..

PR Close #37138
2020-05-18 11:50:08 -07:00
Paul Gschwendtner e19930e70d fix(dev-infra): merge script autosquash should be interactive (#37138)
The components repo does not use the autosquash merge strategy, so
recent changes to that seem to broke the autosquash strategy.

Since we don't run the rebase in interactive mode, the `--autosquash`
flag has no effect. This is by design in Git. We can make it work by
setting the git sequence editor to `true` so that the rebase seems
like an interactive one to Git, while it isn't one for the user.

This matches conceptually with the merge script currently used in
framework. The only difference is that we allow a real interactive
rebase if the `commit message fixup` label is applied. This allows
commit message modifications (and others) if needed.

PR Close #37138
2020-05-18 11:50:08 -07:00
Paul Gschwendtner 8a3493af47 feat(dev-infra): integrate merge script into ng-dev cli (#37138)
Integrates the merge script into the `ng-dev` CLI. The goal is that
caretakers can run the same command across repositories to merge a pull
request. The command is as followed: `yarn ng-dev pr merge <number>`.

PR Close #37138
2020-05-18 11:50:08 -07:00