Commit Graph

367 Commits

Author SHA1 Message Date
Joey Perrott d37dad82f1 build: ensure that refs and shas for PRs only need to be requested once (#36207)
This is done by requesting the refs and shas for the PR when the
env.sh script is run.  Additionally, the env.sh script is now setup
to write all of the environment variables created to a cache file
and subsequent loads of the environment load the values from there.

The get-refs-and-shas-for-target.js script now also first attempts
to load the refs and shas from an environment variable before
falling back to requesting from github via the API.

PR Close #36207
2020-03-25 11:49:42 -07:00
Joey Perrott 58f4254fba fix(dev-infra): use @angular/dev-infra-private package for pullapprove verification (#35996)
Adds devDependency on @angular/dev-infra-private and removes the verify script
from tools, relying instead on the script from ng-dev.

PR Close #35996
2020-03-24 10:14:05 -07:00
Paul Gschwendtner d2c60cc216 ci: cache downloaded bazel version in circleci (#36098)
Similarly to what is done in `angular/components`, we should
cache the downloaded Bazel version (from `bazelisk`).

This reduces the overhead of downloading Bazel, and also avoids
the dependency on the external download server. We should avoid
external server dependencies as much as possible (see how the yarn
registry was flaky in the past).

PR Close #36098
2020-03-18 10:01:50 -07:00
George Kalpakas 3e08347d8a ci(docs-infra): use the regular (non-Ivy) Angular packages in `test_docs_examples_ivy` CI job (#36015)
The docs examples tests are run both with Ivy turned off and on. When
Ivy is turned on, ngcc is used to convert all dependencies (including
the Angular framework packages to Ivy).

Previously, in order to speed things up, the `test_docs_examples_ivy` CI
job would use Angular packages built with Ivy (from
`dist/packages-dist-ivy-aot`). This however was a deviation from what
happens in real-world applications.

This commit changes the `test_docs_examples_ivy` CI job to always use
the regular Angular packages (as published on npm) and use ngcc to
convert them to Ivy.

Relevant discussion:
https://github.com/angular/angular/pull/35091#discussion_r373775396

PR Close #36015
2020-03-18 10:00:00 -07:00
Greg Magolan a1cae28283 build: fix saucelabs jobs that were broken by bazelisk change (#36109)
PR Close #36109
2020-03-17 12:28:02 -07:00
Paul Gschwendtner 22be967104 ci: run ts-circular-deps check in lint job on ci (#35647)
PR Close #35647
2020-03-17 12:27:07 -07:00
Alex Eagle 46fe8fb8b4 build: depend on bazelisk rather than directly on Bazel (#36078)
This has a couple benefits:
- we now use a .bazelversion file rather than package.json to pin the version of bazel we want. This means even if you install bazel on your computer rather than via yarn, you'll still get a warning if your bazel version is wrong.
- you no longer end up downloading three copies of bazel due to bugs in both npm and yarn where they download all tarballs before checking the metadata to see which are usable on the local platform.
- bazelisk correctly handles the tools/bazel trick for wrapping functionality, which we want to use to instrument developer build latencies

PR Close #36078
2020-03-16 10:58:06 -07:00
Greg Magolan 2b6028b643 build: remove legacy integration test runner (#35985)
* integration tests target definitions in integration/BUILD.bazel updated to use a single dict
* payload tracking for integration tests updated to work under Bazel
* legacy integration_test CI job removed
* integration/run_tests.sh script no longer used in CI so it has been updated for running integration tests locally in the legacy way

PR Close #35985
2020-03-11 15:12:33 -07:00
Paul Gschwendtner 287bfefade ci: update components-repo-unit-tests job commit (#35961)
Updates the commit that the `components-repo-unit-tests` job runs
against to the latest available commit at time of writing.

The motivation for updating is that a lot of changes have been made, and
that a upcoming framework PR that fixes check no changes for OnPush
components exposed a test failure in `angular/components`.

See: eae5cf886d

PR Close #35961
2020-03-10 13:29:02 -04:00
George Kalpakas 82a6fc5ef9 build(docs-infra): use local version of Zone.js when testing against local packages (#35858)
In some cases, we want to test the AIO app or docs examples against the
locally built Angular packages (for example to ensure that the changes
in a commit do not introduce a breaking change). In order to achieve
this, we have the `ng-packages-installer` script that handles updating
a project's `package.json` file to use the locally built Angular
packages (and appropriate versions for their (dev-/peer-)dependencies).

Previously, `ng-packages-installer` would only consider the locally
built Angular packages (from `dist/packages-dist/`). However, given that
Zone.js is now part of the `angular/angular` repo, it makes sense to
also use the locally built Zone.js package (from `dist/zone.js-dist/`).
Otherwise, the tests might fail for commits that update both the Angular
packages (and related docs examples) and the Zone.js package. An example
of such a simultaneous change (that would have broken tests) is #33838.

This commit updates the script to install the locally built Zone.js
package (in addition to the Angular ones). The commit ensures that the
Zone.js package will always be available alongside the Angular packages
(i.e. that the Zone.js package will be built by the same script that
builds the Angular packages and that the `dist/zone.js-dist/` directory
will be cached on CI).

Note: This problem was discovered while enabling docs examples unit
tests in #34374.

PR Close #35858
2020-03-06 17:30:20 -05:00
Andrew Scott 1b8a66893e Revert "build(docs-infra): use local version of Zone.js when testing against local packages (#35780)" (#35857)
This reverts commit 7d832ae1001b6264bb7124086089e9e69c10c9b6; breaks CI
with error `Concurrent upstream jobs persisted the same file(s) into the workspace:`

PR Close #35857
2020-03-04 10:57:36 -08:00
George Kalpakas 7d832ae100 build(docs-infra): use local version of Zone.js when testing against local packages (#35780)
In some cases, we want to test the AIO app or docs examples against the
locally built Angular packages (for example to ensure that the changes
in a commit do not introduce a breaking change). In order to achieve
this, we have the `ng-packages-installer` script that handles updating
a project's `package.json` file to use the locally built Angular
packages (and appropriate versions for their (dev-/peer-)dependencies).

Previously, `ng-packages-installer` would only consider the locally
built Angular packages (from `dist/packages-dist/`). However, given that
Zone.js is now part of the `angular/angular` repo, it makes sense to
also use the locally built Zone.js package (from `dist/zone.js-dist/`).
Otherwise, the tests might fail for commits that update both the Angular
packages (and related docs examples) and the Zone.js package. An example
of such a simultaneous change (that would have broken tests) is #33838.

This commit updates the script to install the locally built Zone.js
package (in addition to the Angular ones). The commit ensures that the
Zone.js package will always be available alongside the Angular packages
(i.e. that the Zone.js package will be built by the same script that
builds the Angular packages and that the `dist/zone.js-dist/` directory
will be cached on CI).

Note: This problem was discovered while enabling docs examples unit
tests in #34374.

PR Close #35780
2020-03-04 08:35:26 -08:00
George Kalpakas 3f88de9407 build: move build scripts to dedicated directory (#35780)
This commit moves the build-related scripts
(`build-ivy-npm-packages.js`, `build-packages-dist.js` and
`package-builder.js`) to a dedicated directory to keep the `scripts/`
directory cleaner.

It also moves the logic for building the `zone.js` package to a separate
script, `zone-js-builder.js`, to make it re-usable. A subsequent commit
will use it to build the `zone.js` package when building the Ivy Angular
packages as well.

PR Close #35780
2020-03-04 08:35:26 -08:00
Paul Gschwendtner 5615928df9 build: no longer run tslint from within gulp task (#35800)
Switches our tslint setup to the standard `tslint.json` linter excludes.
The set of files that need to be linted is specified through a Yarn script.

For IDEs, open files are linted with the closest tslint configuration, if the
tslint IDE extension is set up, and the source file is not excluded.

We cannot use the language service plugin for tslint as we have multiple nested
tsconfig files, and we don't want to add the plugin to each tsconfig. We
could reduce that bloat by just extending from a top-level tsconfig that
defines the language service plugin, but unfortunately the tslint plugin does
not allow the use of tslint configs which are not part of the tsconfig project.

This is problematic since the tslint configuration is at the project root, and we
don't want to copy tslint configurations next to each tsconfig file.

Additionally, linting of `d.ts` files has been re-enabled. This has been
disabled in the past and a TODO has been left. This commit fixes the
lint issues and re-enables linting.

PR Close #35800
2020-03-03 09:20:49 -08:00
Pete Bacon Darwin 5e38056459 ci: bump CircleCI cache version prefix (#35711)
The changes to yarn.lock for the `$localize` typings
support are not filtering through because the cache
contains nested packages that are causing compilation
errors.

This change will force the cache to invalidate and allows
us to have a clean node_modules folder.

PR Close #35711
2020-02-28 07:09:50 -08:00
Greg Magolan 184a8493c8 test: run /integration/ng_elements_schematics test with bazel (#35669)
PR Close #35669
2020-02-26 12:58:35 -08:00
Greg Magolan f1ffa7a367 ci: move saucelabs_ivy & saucelabs_view_engine to the daily monitoring CircleCI workflow (#35516)
PR Close #35516
2020-02-24 17:27:21 -08:00
Greg Magolan 3887be464b test: disable broken saucelabs tests with “fixme-saucelabs-ivy” & “fixme-saucelabs-ve” tags (#35516)
PR Close #35516
2020-02-24 17:27:21 -08:00
Greg Magolan f89d9f384e test: saucelab targets for all karma tests (#35516)
PR Close #35516
2020-02-24 17:27:21 -08:00
Greg Magolan 40ae89e3bf build: add new integration tests & bazel-in-bazel tests to … glob (#33927)
Move bazel-in-bazel them to test job & increase it is 2xlarge+. test_integration_bazel is removed. Overall CI credit usage is reduced.

test: include ng_elements_schematics in legacy integration tests temporarily

This test was recently added and use a new pattern that doesn't work with npm_integration_test out of the box. It needs some refactoring to work. Left a TODO for this

PR Close #33927
2020-02-24 08:59:18 -08:00
Greg Magolan 6375454fe6 ci: bump CircleCi test job to xlarge2 CI job (#33927)
Now that large integration tests are running locally in parallel (they can't run on RBE yet as they require network access for yarn install), this test is running out of memory consistently with the xlarge machine

PR Close #33927
2020-02-24 08:59:18 -08:00
Greg Magolan 92b82605b9 ci: install java runtime in test job (#33927)
Install java runtime which is required by some integration tests such as `//integration:hello_world__closure_test`, `//integration:i18n_test` and `//integration:ng_elements_test` to run the closure compiler.

PR Close #33927
2020-02-24 08:59:18 -08:00
Greg Magolan dde68ff954 build: add npm_integration_test && angular_integration_test (#33927)
* it's tricky to get out of the runfiles tree with `bazel test` as `BUILD_WORKSPACE_DIRECTORY` is not set but I employed a trick to read the `DO_NOT_BUILD_HERE` file that is one level up from `execroot` and that contains the workspace directory. This is experimental and if `bazel test //:test.debug` fails than `bazel run` is still guaranteed to work as  `BUILD_WORKSPACE_DIRECTORY` will be set in that context

* test //integration:bazel_test and //integration:bazel-schematics_test exclusively

* run "exclusive" and "manual" bazel-in-bazel integration tests in their own CI job as they take 8m+ to execute

```
//integration:bazel-schematics_test                                      PASSED in 317.2s
//integration:bazel_test                                                 PASSED in 167.8s
```

* Skip all integration tests that are now handled by angular_integration_test except the tests that are tracked for payload size; these are:
- cli-hello-world*
- hello_world__closure

* add & pin @babel deps as newer versions of babel break //packages/localize/src/tools/test:test

@babel/core dep had to be pinned to 7.6.4 or else //packages/localize/src/tools/test:test failed. Also //packages/localize uses @babel/generator, @babel/template, @babel/traverse & @babel/types so these deps were added to package.json as they were not being hoisted anymore from @babel/core transitive.

NB: integration/hello_world__systemjs_umd test must run with systemjs 0.20.0
NB: systemjs must be at 0.18.10 for legacy saucelabs job to pass
NB: With Bazel 2.0, the glob for the files to test `"integration/bazel/**"` is empty if integation/bazel is in .bazelignore. This glob worked under these conditions with 1.1.0. I did not bother testing with 1.2.x as not having integration/bazel in .bazelignore is correct.

PR Close #33927
2020-02-24 08:59:18 -08:00
George Kalpakas 19396769e2 ci: get rid of the `CI_CHROMEDRIVER_VERSION_ARG` env var (#35381)
Previously, we needed to manually specify a ChromeDriver version to
download on CI that would be compatible with the browser version
provided by the docker image used to run the tests. This was kept in the
`CI_CHROMEDRIVER_VERSION_ARG` environment variable.

With recent commits, we use the browser provided by `puppeteer` and can
determine the correct ChromeDriver version programmatically. Therefore,
we no longer need the `CI_CHROMEDRIVER_VERSION_ARG` environment
variable.

NOTE:
There is still one place (the `bazel-schematics` integration project)
where a hard-coded ChromeDriver version is necessary. Since I am not
sure what is the best way to refactor the tests to not rely on a
hard-coded version, I left it as a TODO for a follow-up PR.

PR Close #35381
2020-02-18 12:42:48 -08:00
George Kalpakas bdfe71ae93 build(docs-infra): use `puppeteer` to get a browser for docs examples tests (#35381)
In #35049, integration and AIO tests were changed to use the browser
provided by `puppeteer` in tests. This commit switches the docs examples
tests to use the same setup.

IMPLEMENTATION NOTE:
The examples are used to create ZIP archives that docs users can
download to experiment with. Since we want the downloaded projects to
resemble an `@angular/cli` generated project, we do not want to affect
the project's Protractor configuration in order to use `puppeteer`.

To achieve this, a second Protractor configuration is created (which is
ignored when creating the ZIP archives) that extends the original one
and passes the approperiate arguments to use the browser provided by
`puppeteer`. This new configuration (`protractor-puppeteer.conf.js`) is
used when running the docs examples tests (on CI or locally during
development).

PR Close #35381
2020-02-18 12:42:47 -08:00
George Kalpakas ab8199f7c9 build: several minor fixes related to using `puppeteer` (#35381)
This is a follow-up to #35049 with a few minor fixes related to using
the browser provided by `puppeteer` to run tests. Included fixes:

- Make the `webdriver-manager-update.js` really portable. (Previously,
  it needed to be run from the directory that contained the
  `node_modules/` directory. Now, it can be executed from a subdirectory
  and will correctly resolve dependencies.)

- Use the `puppeteer`-based setup in AIO unit and e2e tests to ensure
  that the downloaded ChromeDriver version matches the browser version
  used in tests.

- Use the `puppeteer`-based setup in the `aio_monitoring_stable` CI job
  (as happens with `aio_monitoring_next`).

- Use the [recommended way][1] of getting the browser port when using
  `puppeteer` with `lighthouse` and avoid hard-coding the remote
  debugging port (to be able to handle multiple instances running
  concurrently).

[1]: https://github.com/GoogleChrome/lighthouse/blame/51df179a0/docs/puppeteer.md#L49

PR Close #35381
2020-02-18 12:42:47 -08:00
Joey Perrott a6aa35e598 ci: use pipeline values to define the CI_COMMIT_RANGE (#35348)
PR Close #35348
2020-02-12 16:40:47 -08:00
Joey Perrott 1530c28e05 ci: add verification of the pullapprove config (#35060)
Verify that all files in the repo are covered by the pullapprove config
and that all rules in the pullapprove config match at least one file
in the repo.

PR Close #35060
2020-02-12 16:39:13 -08:00
Greg Magolan afdd405995 test: use puppeteer in aio build instead to remove CI_CHROMEDRIVER_VERSION_ARG (#35049)
PR Close #35049
2020-02-11 13:16:53 -08:00
Greg Magolan acfd0edd38 test: use puppeteer in integration tests and to download correct chromedriver (#35049)
This means integration tests no longer need to depend on a $CI_CHROMEDRIVER_VERSION_ARG environment variable to specify which chromedriver version to download to match the locally installed chrome. This was bad DX and not having it specified was not reliable as webdriver-manager would not always download the chromedriver version to work with the locally installed chrome.

webdriver-manager update --gecko=false --standalone=false $CI_CHROMEDRIVER_VERSION_ARG is now replaced with node webdriver-manager-update.js in the root package.json, which checks which version of chrome puppeteer has come bundled with & downloads informs webdriver-manager to download the corresponding chrome driver version.

Integration tests now use "webdriver-manager": "file:../../node_modules/webdriver-manager" so they don't have to waste time calling webdriver-manager update in postinstall

"// resolutions": "Ensure a single version of webdriver-manager which comes from root node_modules that has already run webdriver-manager update",
"resolutions": {
"**/webdriver-manager": "file:../../node_modules/webdriver-manager"
}
This should speed up each integration postinstall by a few seconds.

Further, integration test package.json files link puppeteer via file:../../node_modules/puppeteer which is the ideal situation as the puppeteer post-install won't download chrome if it is already downloaded. In CI, since node_modules is cached it should not need to download Chrome either unless the node_modules cache is busted.

NB: each version of puppeteer comes bundles with a specific version of chrome. Root package.json & yarn.lock currently pull down puppeteer 2.1.0 which comes with chrome 80. See https://github.com/puppeteer/puppeteer#q-which-chromium-version-does-puppeteer-use for more info.

Only two references to CI_CHROMEDRIVER_VERSION_ARG left in integration tests at integration/bazel-schematics/test.sh which I'm not entirely sure how to get rid of it

Use a lightweight puppeteer=>chrome version mapping instead of launching chrome and calling browser.version()

Launching puppeteer headless chrome and calling browser.version() was a heavy-handed approach to determine the Chrome version. A small and easy to update mappings file is a better solution and it means that the `yarn install` step does not require chrome shared libs available on the system for its postinstall step

PR Close #35049
2020-02-11 13:16:52 -08:00
Paul Gschwendtner 13be065ed9 ci: re-enable disabled components-repo-unit-tests job (#35123)
We temporarily disabled the components-repo-unit-tests job as part of
a ngcc PR: #35079. This was necessary because the components repository
used postinstall patches for `@angular/compiler-cli/ngcc`. Due to
changes in ngcc, these patches no longer worked and caused the
`components-repo-unit-tests` job to fail.

The postinstall patch has been removed in the components repo, so the
job can be re-enabled.

PR Close #35123
2020-02-10 09:22:55 -08:00
Paul Gschwendtner 045a989a03 ci: update components-repo-unit-tests job commit (#35123)
Updates the commit the `components-repo-unit-tests` job runs against.
We need at least 2ec7254f88
which fixes a Ngcc postinstall patch conflict that required us to
temporarily disable the job in #35079.

PR Close #35123
2020-02-10 09:22:55 -08:00
JiaLiPassion daac33cdc8 feat: add basic jest support (#35080)
PR Close #35080
2020-02-07 11:43:21 -08:00
Pete Bacon Darwin def64efdfc build: remove the `exit 0` on components-unit-tests (#35090)
PR Close #35090
2020-02-07 11:34:20 -08:00
Paul Gschwendtner 363e1ab775 ci: ensure saucelabs browsers can load karma test page (#35171)
In the past we had connecitivity issues on Saucelabs. Browsers on
mobile devices were not able to properly resolve the `localhost`
hostname through the tunnel. This is because the device resolves
`localhost` or `127.0.0.1` to the actual Saucelabs device, while it
should resolve to the tunnel host machine (in our case the CircleCI VM).

In the past, we simply disabled the failing devices and re-enabled the
devices later. At this point, the Saucelabs team claimed that the
connecitivy/proxy issues were fixed.

Saucelabs seems to have a process for VMs which ensures that requests to
`localhost` / `127.0.0.1` are properly resolved through the tunnel. This
process is not very reliable and can cause tests to fail. Related issues have been
observed/mentioned in the Saucelabs support docs. e.g.

https://support.saucelabs.com/hc/en-us/articles/115002212447-Unable-to-Reach-Application-on-localhost-for-Tests-Run-on-Safari-8-and-9-and-Edge
https://support.saucelabs.com/hc/en-us/articles/225106887-Safari-and-Internet-Explorer-Won-t-Load-Website-When-Using-Sauce-Connect-on-Localhost

In order to ensure that requests are always resolved through the tunnel,
we add our own domain alias in the CircleCI's hosts file, and enforce that
it is always resolved through the tunnel (using the `--tunnel-domains` SC flag).
Saucelabs devices by default will never resolve this domain/hostname to the
actual local Saucelabs device.

PR Close #35171
2020-02-06 15:36:27 -08:00
Joey Perrott 27d16a7881 ci: Remove old vendoring solution in favor of relying on yarn-path (#35083)
Now that bazel respects the yarn-path value found in .yarnrc, we can
remove the last remaining reliances on our vendoring in
//third_party/github.com/yarnpkg/yarn/

PR Close #35083
2020-02-06 15:30:51 -08:00
Pete Bacon Darwin f0b8f0cc7c build: temporarily disable components-repo-unit-tests (#35079)
Disabling this job until the fix at
https://github.com/angular/components/pull/18355 or similar lands.

PR Close #35079
2020-01-31 17:02:44 -08:00
George Kalpakas d43187f7ef ci(docs-infra): ignore node version in `aio_monitoring_stable` CI job (for real) (#35033)
In #35004, we started ignoring yarn's engines check for `yarn install`
in AIO's `test-production.sh` script to fix a failure in the
`aio_monitoring_stable` CI job. (See #35004 for details.)

It turns out that the version of yarn used on the stable branch (1.17.3)
`--ignore-engines` is needed on all yarn commands (including `yarn
run`). Thus, #35004 is not enough to fix the failures.
New example failure: https://circleci.com/gh/angular/angular/604341

This commit turns of the engines check for the whole
`aio_monitoring_stable` CI job to fix the failure and make the job more
robust.

PR Close #35033
2020-01-29 09:25:46 -08:00
Greg Magolan dff4e1e19c refactor: simplify bazel saucelabs targets using karma pre-test wrapper and shared saucelabs connection between tests (#34769)
* Added a /tools/saucelabs/sauce-service.sh script that manages the sauce-connect as a service which is used by the karma-saucelabs.js wrapper to start the service.
* Added /tools/saucelabs/README.md that covers the details of SauceLabs karma testing with Bazel.

PR Close #34769
2020-01-28 13:47:00 -08:00
Joey Perrott 296dc0622f ci: Migrate from CODEOWNERS to PullApprove for PR review management (#34814)
We are migrating to PullApprove for our PR review management in an attempt
to allow for more granular and equitable code review assignments across the
team.  Currently this migration is equivalent in the review assignments
it will create. Once stable, our expectation is that we will be able to
take advantage of PullApproves additional features for things like staged
reviews.

PR Close #34814
2020-01-28 09:19:34 -08:00
Joey Perrott 8fbb48cc2e build: migrate to node@12.14.1 (#34955)
PR Close #34955
2020-01-27 09:31:22 -08:00
George Kalpakas ed24100976 ci(docs-infra): use the correct version of yarn in `aio_monitoring_stable` (#34954)
The `aio_monitoring_stable` CI job needs to check out (and execute
commands on) some files from the stable branch. Therefore, it needs to
use a version of `yarn` that is compatible with the `engines` version
specified in `aio/package.json`.

In #34902 (and subsequently #34952 for the 8.2.x branch), the way to
access our vendored version of yarn changed. This commit ensures that we
checkout the necessary files from the stable branch to use an
appropriate yarn version.

PR Close #34954
2020-01-24 11:32:58 -08:00
Paul Gschwendtner fa4ea34401 ci: rename `material-unit-tests` job to `components-repo-unit-tests` (#34898)
We rename the `material-unit-tests` job to `components-repo-unit-tests`
because the job runs all unit tests found in the Angular Components repository.
This includes the Angular CDK, Angular Material and more. Also the repository has
been renamed from `angular/material2` to `angular/components` in the past.

PR Close #34898
2020-01-23 13:29:02 -08:00
Paul Gschwendtner fd3cfbb678 build: create bazel marco to test for circular dependencies (#34774)
Creates a Bazel macro that can be used to test packages for
circular dependencies. We face one limitation with Bazel:

 * Built packages use module imports, and not relative source file
 paths. This means we need custom resolution.

Fortunately, tools like `madge` support custom resolution.

Also removes the outdated `check-cycles` gulp task that
didn't catch circular dependencies. It seems like the test
became broken when we switched the packages-dist output to Bazel. It
breaks because the Bazel output doesn't use relative paths, but uses
the module imports. This will be handled in the new Bazel macro/rule.

PR Close #34774
2020-01-23 11:36:40 -08:00
Joey Perrott cc956744d7 ci: move setup of bazel configurations into the env init scripts (#34834)
Since we always set up bazel on both our windows and linux CI
runs, we should consider this configuration part of the environment
setup.

PR Close #34834
2020-01-22 14:37:02 -05:00
Joey Perrott ef665889a1 ci: always set up RBE for bazel executions on CI (#34834)
Since we always execute our bazel commands using RBE on CI
we should not have separete tasks of configuring bazel
and setting up RBE for bazel.  Enabling RBE for bazel should
be part of our bazel configuration process on CI.

PR Close #34834
2020-01-22 14:37:02 -05:00
Paul Gschwendtner 7342d4b6f0 ci: remove typescript 3.7 patches in material-unit-tests job (#34863)
Previously, we added patches to the `angular/components` repository in
the `material-unit-tests` job in order to be able to run their tests
with TypeScript 3.7. This is no longer needed since the components
repository updated to Ts 3.7 already. Hence we can remove the patches.

PR Close #34863
2020-01-21 13:13:51 -05:00
Paul Gschwendtner 20419d3d49 ci: update material-unit-tests job commit to latest (#34863)
Updates the `material-unit-tests` job commit to the latest
available commit. We need to update since the `angular/components`
repository updated to TypeScript 3.7, which means that we can remove
the TS 3.7 workarounds we applied to get the job green when framework
updated to TS 3.7.

PR Close #34863
2020-01-21 13:13:51 -05:00
Andrius 9c211f9625 ci: update material-unit-tests commit (#33717)
Updates the SHA that will be tested against in the `material-unit-tests` job to the latest commit in the components repository. SHA 71955d2e194bfc5561f25daea16e68af266d6ff9 is needed in order to compile repository with typescript 3.7

PR Close #33717
2020-01-14 16:42:22 -08:00
Paul Gschwendtner c9f24e6a58 ci: overwrite angular/components yarn typescript resolution (#33717)
The components repository has a Yarn resolution to ensure that
dgeni-packages uses a specific TypeScript version. This resolution
causes the specified TS version to be considered as candidate for the
`@angular/bazel` peer dependency. Ultimately, Yarn decides to use
the TypeScript version from the resolution for `@angular/bazel`,
and builds will fail due to a version mismatch.

This is because `tsickle` will use the hoisted top-level TS
version (set to `3.7.4` ), while `@angular/bazel` uses the
version from the resolution (at the time of writing: v3.6.4)

PR Close #33717
2020-01-14 16:42:22 -08:00
Paul Gschwendtner 649d01e52a ci: update material-unit-tests commit (#33717)
Updates the commit SHA the `material-unit-tests` CircleCI
job runs against. We need to include a commit that makes
the node module installation more determinisitc (i.e. ensuring
that `tsickle` is always hoisted at the node module root).

31a50f7ad2

PR Close #33717
2020-01-14 16:42:22 -08:00
Filipe Silva 5d203c318f test: also install tsickle 0.38 on material (#33717)
PR Close #33717
2020-01-14 16:42:22 -08:00
Filipe Silva dd03dca321 test: save material ts override as a devdep (#33717)
PR Close #33717
2020-01-14 16:42:22 -08:00
Filipe Silva c7c38ac16e test: use ts 3.7 on material unit tests (#33717)
PR Close #33717
2020-01-14 16:42:22 -08:00
atscott 56ef500716 Revert "test: update to latest angular/components commit to pick up rules_nodejs 1.0 update (#34589)" (#34730)
This reverts commit d0400a1188.

PR Close #34730
2020-01-10 14:12:15 -08:00
atscott e2fe62d4cf Revert "test: bump material cache_key version (#34589)" (#34730)
This reverts commit 0e52d9297f.

PR Close #34730
2020-01-10 14:12:15 -08:00
Greg Magolan 0e52d9297f test: bump material cache_key version (#34589)
Also add comment that cache_key version should be bumped when switching forks or branches and added a comment from @devversion explaining how the fallback cache key works.

PR Close #34589
2020-01-10 08:31:59 -08:00
Greg Magolan d0400a1188 test: update to latest angular/components commit to pick up rules_nodejs 1.0 update (#34589)
This commit includes https://github.com/angular/components/pull/18064 which is a pre-req for updating the angular repo to rules_nodejs 1.0.

PR Close #34589
2020-01-10 08:31:59 -08:00
George Kalpakas b637b9322e ci(docs-infra): fix failure in `aio_monitoring_stable` due to yarn version mismatch (#34451)
The `aio_monitoring_stable` CI job is triggered as a cronjob on the
master branch and its purpose is to run some e2e tests against the
deployed stable version of the docs web-app at https://angular.io/. In
order for the tests to be compatible with the deployed version of the
web-app (which gets deployed from the stable branch), the stable branch
is checked out in git as part of the CI job.

Previously, we only checked out the `aio/` directory from the stable
branch, leaving the rest of the code at master. This doesn't matter as
long as the commands used to run the tests do not rely on code outside
of `aio/`. However, it turns out that there _is_ code outside of `aio/`
that affects the executed commands: It is our vendored version of yarn
(in `third_party/github.com/yarnpkg/`), which overwrites the global yarn
installed on the docker image on CI and must match the version range
specified in `aio/package.json > engines`.

Using the yarn version checked out from the master branch with the
`aio/` code checked out from the stable branch can lead to failures
such as [this one][1].

This commit fixes the problem by checking out both the `aio/` and
`third_party/github.com/yarnpkg/` directories from the stable branch and
re-running the steps to overwrite the global yarn executable with our
own version from `third_party/github.com/yarnpkg/`. This ensures that
the version of yarn used will be compatible with the version range
specified in `aio/package.json > engines`.

NOTE:
We cannot checkout everything from the stable branch, since the CI
config (`.circleci/config.yml` from the master branch) may try to run
certain scripts (such as `.circleci/get-vendored-yarn-path.js`) that are
not available on the stable branch. Therefore, we should only check out
the necessary bits from the stable branch.

[1]: https://circleci.com/gh/angular/angular/567315

PR Close #34451
2019-12-17 13:32:00 -08:00
Paul Gschwendtner 9ba8059e88 ci: update material-unit-tests job to include commit that reduced test flakiness (#34430)
Updates the material-unit-tests job commit SHA to the most recent
commit at the time of writing. The goal is to run the unit tests
with 6ae74a0eb2
that improved stability of a few menu tests that were flaky.

e.g. https://circleci.com/gh/angular/angular/564650

PR Close #34430
2019-12-17 11:41:17 -08:00
Paul Gschwendtner d0a04bf309 ci: fix saucelabs_view_engine master-only failing (#34429)
Currently the `saucelabs_view_engine` job fails because
the Saucelabs Bazel run script thinks that `--config=saucelabs`
is a flag targeting the actual script. This is not the case and
the flag should be actually part of the bazel command.

PR Close #34429
2019-12-16 08:32:51 -08:00
Paul Gschwendtner 6d3a25d897 ci: run acceptance tests on saucelabs with ivy (#34277)
Currently we only run Saucelabs on PRs using the legacy View Engine
build. Switching that build to Ivy is not trivial and there are various
options:

  1. Updating the R3 switches to use POST_R3 by default. At first glance,
  this doesn't look easy because the current ngtsc switch logic seems to
  be unidirectional (only PRE_R3 to POST_R3).

  2. Updating the legacy setup to run with Ivy. This sounds like the easiest
  solution at first.. but it turns out to be way more complicated. Packages
  would need to be built with ngtsc using legacy tools (i.e. first building
  the compiler-cli; and then building packages) and View Engine only tests
  would need to be determined and filtered out. Basically it will result in
  re-auditing all test targets. This is contradictory to the fact that we have
  this information in Bazel already.

  3. Creating a new job that runs tests on Saucelabs with Bazel. We specify
  fine-grained test targets that should run. This would be a good start
  (e.g. acceptance tests) and also would mean that we do not continue maintaining
  the legacy setup..

This commit implements the third option as it allows us to move forward
with the general Bazel migration. We don't want to spend too much time
on our legacy setup since it will be removed anyway in the future.

PR Close #34277
2019-12-16 07:43:41 -08:00
George Kalpakas c049cf2206 ci: use local, vendored yarn in Windows CI jobs (#34384)
We keep a version of yarn in the repo, at
`third_party/github.com/yarnpkg/`. All CI jobs should use that version
for consistency (and easier updates).

Previously, the Windows jobs did not use the local version. They used
the version that came pre-installed on the docker image that we used.
This made it more difficult to update the yarn version (something that
we might want to do independently of updating other dependencies, such
as Node.js).

This commit fixes this by setting up the Windows CI jobs to also use the
local, vendored version of yarn.

PR Close #34384
2019-12-16 07:39:58 -08:00
George Kalpakas 3ceb2b85da ci: avoid hard-coding path to local yarn executable (#34384)
We keep a version of yarn in the repo, at
`third_party/github.com/yarnpkg/`. All CI jobs (including Windows ones)
should use that version for consistency (and easier updates). The path
to the actual `yarn.js` script, however, changes depending on the
version (e.g. `third_party/github.com/yarnpkg/v1.21.1/...`).
(NOTE: The Windows jobs are currently not using this local version, but
that should be fixed in a subsequent commit.)

Previously, when updating the local version of yarn, we would
potentially have to update the path in several places.

This commit addresses the problem by adding a Node.js script that infers
the correct path. The script can be used in all places where we need to
use the local version of yarn (including both Linux and Windows CI
jobs), thus eliminating the need to update the path in several places.

PR Close #34384
2019-12-16 07:39:58 -08:00
George Kalpakas fac997c53b ci: remove unused variable from `.circleci/env.sh` (#34384)
Since #32537, the `.circleci/get-commit-range.js` script is no longer
used in `.circleci/env.sh`. This commit removes the now unused local
variable to the script's path.

PR Close #34384
2019-12-16 07:39:58 -08:00
Kara Erickson 17f7f06ca5 Revert "build: update to yarn@1.21.1" (#34402)
This reverts commit f029af50820765019413fa319330830306b80d6a while we investigate
some failures on master on Circle CI. Currently the Windows tests and the
"test-ivy-aot" jobs are red because of incompatible yarn versions.

PR Close #34402
2019-12-13 13:43:25 -08:00
Igor Minar b405942b0c build: update to yarn@1.21.1 (#34384)
This updates yarn throughout the monorepo for both build and CI.

PR Close #34384
2019-12-13 10:51:12 -08:00
Joey Perrott 89ef77f750 ci: update sha of components repo for components unit test integrations (#34280)
Before updating to remove the compile build variable, we must update
the components unit test integrations to a sha  in the components
repo which no longer relies on the compile build variable.

PR Close #34280
2019-12-06 11:04:20 -08:00
Joey Perrott 023c9bebe5 ci: update saucelabs to use angular-framework account (#34233)
Currently all saucelabs usage in our repos is done using the same
account angular-ci.  By migrating to use individual accounts
for each repo, we can better track the usage for each repo as well
as providing concurrency limiting on a per repo basis.

Additionally, we no longer use two separate accounts based on being
on master or a PR branch, so this logic can be removed.

PR Close #34233
2019-12-05 10:21:43 -08:00
George Kalpakas 485ce640fd ci: use more descriptive names for published build artifacts (#33957)
In efbbae5a4, the `publish_packages_as_artifacts` CircleCI job was
created to publish the build artifacts of PR builds on CI. In a8f4f14bd,
its scope was expanded to also publish build artifacts on non-PR builds.

The published artifacts names are constructed based on the PR number
(e.g. include `-pr12345-`), so on non-PR builds the names do not reflect
the source branch (instead, they include `-prfalse-`).

This commit fixes this by using the current branch name. For example,
artifact names for the `master` or `9.0.x` branch will include
`-master-` and `-9.0.x-` respectively (instead of `-prfalse-`).
(NOTE: For PRs, where branch name is `pull/12345`, the branch name is
transformed to `pr12345`.)

PR Close #33957
2019-11-22 14:05:52 -05:00
Joey Perrott e128d87030 ci: set up window CI to use remote http caching (#33970)
PR Close #33970
2019-11-21 16:43:42 -05:00
George Kalpakas 7eb3e3bce6 build: fix build scripts on macOS (#33854)
In #33823, `scripts/package-builds.sh` (which is used by both
`build-packages-dist.sh` and `build-ivy-npm-packages.sh`) was updated to
use `realpath`. It turns out that `realpath` does not exist on macOS, so
the build scripts do not work there.

In order to fix this (and also reduce the likelihood of introducing
similar issues in the future), this commit changes these bash scripts to
Node.js scripts (using [ShellJS](https://github.com/shelljs/shelljs) for
a cross-platform implementation of Unix shell commands where necessary).

PR Close #33854
2019-11-15 16:05:00 -08:00
George Kalpakas 2ce9cdab9d ci: publish tarballs for the zone.js package as CI build artifacts (#33733)
Since #33321, Angular packages have been persisted on each build as
CircleCI build artifacts (`.tgz` files), which can be used to install
dependencies on a project (for the purpose of testing or trying out a
change before a PR being merged and without having to build the packages
from source locally).

Previously, only packages published to npm under the `@angular` scope
were persisted as build artifacts.

This commit adds the `zone.js` package to the list of persisted
packages.

Fixes #33686

PR Close #33733
2019-11-12 09:55:17 -08:00
George Kalpakas 74b7d1ace1 test(ngcc): build `zone.js` from source in `scripts/build-packages-dist.sh` (#33733)
In #33046, internal uses of `zone.js` were switched to reference it
directly from source (built with Bazel) instead of npm. As a result, the
necessary scripts were updated to build `zone.js` as necessary. However,
some `integration/**/debug-test.sh` scripts were missed (apparently
because they are not used on CI, but only locally as helpers for
debugging the integration projects).

This commit updates the `scripts/build-packages-dist.sh` script to also
build `zone.js`, so that other scripts (such as the various
`debug-test.sh` scripts) can use it.

PR Close #33733
2019-11-12 09:55:16 -08:00
Paul Gschwendtner a1e9956290 ci: update material-unit-tests job commit (#33716)
Updates the commit we run the `material-unit-tests` job
against. The latest commit includes 1255139a38

This commit reduces the flakiness of a `MatMenu` test and therefore
improves the stability of the material-unit-tests job.

Example failing build: https://circleci.com/gh/angular/angular/521625

PR Close #33716
2019-11-11 09:39:24 -08:00
George Kalpakas 976bdf1087 build(docs-infra): update @angular/* to 9.0.0-rc.0 (#33547)
PR Close #33547
2019-11-11 09:38:04 -08:00
Greg Magolan 2ed04ffcdc build: update to latest material commit to pick up https://github.com/angular/components/pull/17620 (#33607)
PR Close #33607
2019-11-06 19:56:57 +00:00
JiaLiPassion 8c6fb17d29 build: reference zone.js from source directly instead of npm. (#33046)
Close #32482

PR Close #33046
2019-11-06 00:48:34 +00:00
Joey Perrott a8f4f14bdc ci: always run the publish_packages_as_artifacts jobs (#33519)
PR Close #33519
2019-11-01 17:49:58 +00:00
Paul Gschwendtner 31116f0ced ci: re-enable material unit tests job (#33530)
The Material units tests job has been disabled with the
typescript upgrade PR since the components repository was
still on an old TypeScript version (due to cyclic dependencies
between the framework and components repository).

Since the components repository has been updated to `9.0.0-next.15`
and now uses the compatible TypeScript version, we can re-enable
ds

Related to this change, we need to ignore the `package.json`
engines when installing the dependencies of the components repo. This
is because the components repo already updated to NodeJS v12 and Yarn
v1.19.1. This is not the case for the CI setup of framework. For now,
since we don't want to change the dev setup (as it slows down
development), we temporarily disable the engines.

Additionally, the material unit tests job now depends on the actual
release packages (not on the ngtsc compiled ones). This is because
the components repo setup relies on NGCC being run. This is also
helpful for validating ngcc against the framework packages.

PR Close #33530
2019-11-01 17:38:39 +00:00
George Kalpakas efbbae5a48 ci: publish tarballs for all Angular packages as build artifacts on PR builds (#33321)
Previously, when one wanted to try out the changes from a PR before it
was merged, they had to check out the PR locally and build the Angular
packages themselves (which is time-consuming and wasteful given that the
packages have already been built on CI).

This commit persists all Angular packages on each build as `.tgz` files,
which can be used to install dependencies on an project (supported by
both [npm][1] and [yarn][2]). In addition to individual `.tgz` files for
each package, a `.tgz` file including all packages is also stored, which
can be used to test the packages locally by overwriting the ones in the
`node_modules/` directory of a project.

CircleCI [build artifacts][3] an be used for longer-term storage of the
outputs of a build and are designed to be useful around the time of the
build, which suits our needs.

[1]: https://docs.npmjs.com/cli/install.html
[2]: https://yarnpkg.com/lang/en/docs/cli/add
[3]: https://circleci.com/docs/2.0/artifacts

PR Close #33321
2019-10-24 09:56:21 -07:00
George Kalpakas 22e483858e ci: use YAML aliases to define branch filters for CircleCI jobs (#33321)
PR Close #33321
2019-10-24 09:56:21 -07:00
Joey Perrott a86a179f45 build: use http caching on windows CI runs (#33298)
PR Close #33298
2019-10-22 13:09:28 -04:00
Joey Perrott fa0ab38546 build: update circleci config to handle GOOGLE_APPLICATION_CREDENTIALS for all linux bazel builds (#33109)
PR Close #33109
2019-10-21 15:56:51 -04:00
Igor Minar 86e1e6c082 feat: typescript 3.6 support (#32946)
BREAKING CHANGE: typescript 3.4 and 3.5 are no longer supported, please update to typescript 3.6

Fixes #32380

PR Close #32946
2019-10-18 13:15:16 -04:00
Matias Niemelä 117ca7cf39 revert: build: use http caching on windows CI runs (#33238) (#33254)
This reverts commit 7fb55f29b6.

PR Close #33254
2019-10-18 12:08:59 -04:00
Joey Perrott 7fb55f29b6 build: use http caching on windows CI runs (#33238)
PR Close #33238
2019-10-18 11:32:36 -04:00
Paul Gschwendtner b60541c92a ci: update commit for material-unit-tests job (#33221)
Updates the commit of the Angular components repository for
which the `material-unit-tests` job runs tests against. We need to
update to the latest commit at the time of writing, in order to be able
to remove the gesture tests from the material-ci/blocklist.

This is now possible because the Angular components repository
removes the dependency on `hammerjs` completely.

PR Close #33221
2019-10-17 14:14:53 -04:00
Greg Magolan dcbc3b197d ci: update material-unit-test commit (#33073)
Updates the `material-unit-test` job to run tests against
the latest commit of the Angular Components repository.

The components repository updated to rules_nodejs#0.38.2 before Angular Bazel did.
To do this, the `@angular/bazel` v0.38.2 compatibility changes were patched on postinstall.

This now conflicts because we install a `@angular/bazel` version in the `material-unit-tests` job
that already includes these compatibility changes. This would result in the patch being a noop
for which the `patch` command throws an error.

We can remove this once components can install a released version of `@angular/bazel` that is
compatible with `rules_nodejs#0.38.2`.

PR Close #33073
2019-10-14 20:25:57 +00:00
Joey Perrott 68f06c8dd6 build: using the xlarge resource class for zonejs tests on CI (#33148)
Fixes #33147

PR Close #33148
2019-10-14 20:13:26 +00:00
Paul Gschwendtner 3830f6fc14 ci: update material-unit-test commit (#32889)
Updates the `material-unit-test` job to run tests against
the latest commit of the Angular Components repository.

We need to update to a version where components already
updated to `rules_nodejs#0.38.0`.

PR Close #32889
2019-10-08 09:27:11 -07:00
George Kalpakas 442f323a32 ci: shorten the display name of `init_environment` steps that are shown in all jobs (#32998)
The `init_environment` custom command is used in almost all other jobs.
The this is implemented in CircleCI is that the steps of the command are
inlined into the other jobs.

Some of the `init_environment` commands are quite long and (since the
default display name for a step is its command) they clutter the
CircleCI UI. Additionally, multiple related commands are shown as
separate steps, which makes it more difficult to get to the actual steps
of a job.

This commit improves this by:
1. Defining explicit, short names for steps with long commands.
2. Grouping multiple related steps into one.

PR Close #32998
2019-10-04 08:39:20 -07:00
George Kalpakas eb72cecc42 build(docs-infra): turn on Ivy (#32923)
The angular.io project uses Angular and CLI v9, which by default turns
on Ivy mode. However, since ec4381dd4, we explicitly opt out of Ivy.

This commit removes the `enabledIvy: false` configuration, thus allowing
the default behavior of having Ivy on.

NOTE:
This commit only changes the angular.io projects. The docs examples need
to be updated separately (first to Angular and CLI v9 and then to Ivy).

PR Close #32923
2019-10-04 08:27:21 -07:00
Filipe Silva 0f3a48e4d4 ci: refactor notification commands into a single parametrized one (#32745)
notify_caretaker_on_fail and notify_dev_infra_on_fail are the same, except for the url they notify.

PR Close #32745
2019-10-02 09:34:12 -07:00
Filipe Silva 653eb0fef9 ci: use workspaces and cache for Windows jobs (#32745)
Making use of these functionalities lets our Windows jobs be more similar to the Linux ones, and draw common artifacts from them too.

PR Close #32745
2019-10-02 09:34:12 -07:00
Filipe Silva fc3260d87e ci: use CircleCI parameterized jobs (#32745)
Parameterized jobs lets us reduce duplication of very similar jobs.

See https://circleci.com/docs/2.0/reusing-config/#authoring-parameterized-jobs for more info.

PR Close #32745
2019-10-02 09:34:11 -07:00
Filipe Silva 296954041e ci: use CircleCI commands (#32745)
When we needed to run multiple commands in a reusable fashion, we needed to make a giant run block with multiple things inside. Using custom commands gives us a better way to do this.

See https://circleci.com/docs/2.0/reusing-config/#authoring-reusable-commands for more info.

PR Close #32745
2019-10-02 09:34:11 -07:00
Filipe Silva 1115961892 ci: keep cache key fallback in a var (#32745)
This way it's right next to the original key and it's easier to keep them in sync.

PR Close #32745
2019-10-02 09:34:11 -07:00
Filipe Silva 6db5c4a6f6 ci: use CircleCI executors (#32745)
Executors let you define custom execution contexts for jobs.

See https://circleci.com/docs/2.0/reusing-config/#authoring-reusable-executors for more information.

PR Close #32745
2019-10-02 09:34:11 -07:00
Joey Perrott 252966bcca ci: build bazel saucelabs tests remotely (#32719)
PR Close #32719
2019-09-17 15:14:17 -07:00