Commit Graph

158 Commits

Author SHA1 Message Date
George Kalpakas 3efdd39a18 ci(docs-infra): use the tests from the stable branch in `aio_monitoring_stable` CircleCI job (#30110)
Previously, the `aio_monitoring_stable` job (which runs tests against
https://angular.io/) was using the tests from the master branch. As a
result, if the master branch included changes in those tests that were
not yet backported to the stable branch (and thus deployed to
https://angular.io/), the tests would fail.

This commit fixes this by using the tests from the stable branch to test
against https://angular.io/.

Fixes #30101

PR Close #30110
2019-04-26 16:33:45 -07:00
George Kalpakas 582ef2e7b4 ci(docs-infra): split the `aio_monitoring` CircleCI job into two jobs (#30110)
Previously, the `aio_monitoring` job was testing both the stable
(https://angular.io/) and the @next (https://next.angular.io/) versions.

This commit splits the tests into two separate jobs (still run as part
of the same workflow). This speeds up the tests (since the two jobs can
now run in parallel) and makes it easier to isolate failures (e.g.
identify which branch is failing, disable one of the two, etc.).
(Credits to @petebacondarwin 😉)

PR Close #30110
2019-04-26 16:33:45 -07:00
George Kalpakas 214fef2ee4 ci(docs-infra): re-use `setup` CircleCI job in `aio_monitoring` (#30110)
PR Close #30110
2019-04-26 16:33:45 -07:00
George Kalpakas 531fcaa99a ci(docs-infra): re-enable `aio_monitoring` CircleCI job (#30110)
The job started failing for https://angular.io/, due to changes in tests
that only affected https://next.angular.io/, and was disabled in #30102.

This commit re-enables the job (since it does not block anything and it
will be fixed in a subsequent commit).

PR Close #30110
2019-04-26 16:33:45 -07:00
Andrew Kushnir a400429faa docs: temporary disable aio_monitoring job due to a version skew (#30102)
Redirects that were updated in 24c61cb63e break the aio_monitoring CircleCI job, since we run the tests against the production angular.io site (that doesn't have the latest redirects config yet).

This change temporary disables the aio_monitoring job to avoid failures for other PRs. The problem will be resolved and the job will be enabled in followup PRs.

PR Close #30102
2019-04-24 18:01:50 -07:00
Paul Gschwendtner 71498a407a ci: remove integration test "uname -a" command (#30006)
We recently added the "uname -a" command to the CircleCI
integration test in order to identify Linux kernels that
broke Chrome sandboxing.

Since this issue seems to be claimed as fixed by the CircleCI
support and we didn't see any sandboxing issues the last month,
we are removing the debugging command we added.

PR Close #30006
2019-04-22 08:46:51 -07:00
George Kalpakas f957c7c1cd ci(docs-infra): use pre-ivy packages in `test_aio_local_ivy` to test ngcc (#29989)
To better test ngcc (in addition to Ivy) on angular.io, change the
`test_aio_local_ivy` CircleCI job to use the pre-ivy Angular packages
(and have ngcc transform them to Ivy ones).

PR Close #29989
2019-04-19 09:46:48 -07:00
George Kalpakas 60e9d2da4f ci(docs-infra): increase wait for SW on localhost to avoid CI flakes (#29953)
The server used for testing on localhost has less optimizations (e.g.
serves uncompressed files), so we need to wait longer the ServiceWorker
to be loaded and registered to allow Lighthouse to reliably detect it,
especially on slower environments (e.g. CI).

Related: https://github.com/GoogleChrome/lighthouse/issues/5527#issuecomment-483710849

Fixes #29910

PR Close #29953
2019-04-17 12:14:39 -07:00
George Kalpakas 8027b3e19b ci(docs-infra): temporarily lower the min required PWA score for localhost tests (#29911)
The PWA score tests have been occasionally failing on CI recently
(possibly due to CI VM/network issues).

This commit temporarily disables them, until we investigate the
root-cause and/or put a work-around in place.

The PWA score tests are still run against the deployed versions (which
don't suffer as much) or PRs with public previews (as part of the
`test_aio_preview` job) and on upstream builds (as part of the
`deploy_aio` job).

Related to #29910.

[1]: https://github.com/angular/angular/blob/3a836c362/.circleci/config.yml#L390

PR Close #29911
2019-04-15 12:34:42 -07:00
Alex Eagle cb9ee3411f ci: simplify RBE execution (#29731)
Run all targets with RBE config. Previously we filtered out one target, //tools/ts-api-guardian:tests, and ran that with a different bazelrc

PR Close #29731
2019-04-08 12:03:54 -07:00
George Kalpakas fd122b0739 ci: cache Material `node_modules` based on the lockfile checksum (#29417)
This will increase the cache hit rate for the `material-unit-tests` job.
Related to https://github.com/angular/angular/pull/29416#discussion_r267321140.

PR Close #29417
2019-03-20 13:13:27 -04:00
Paul Gschwendtner a530ed11e8 ci: do not cache modified "node_modules" in "material-unit-tests" job (#29416)
Currently we cache the Material `node_modules` after
the `run_angular_material_unit_tests.sh` completed. This
means that the cache will incorrectly contain the Ivy NPM
package output which might be incompatible with the
other Material dependencies. e.g. the Material postinstall
command now uses a different NGC version that does not
work with the `typescript` version that has been specified in
the Material project.

PR Close #29416
2019-03-20 06:58:23 -07:00
Paul Gschwendtner 19ff32036e ci: enable parallelism for material-unit tests job (#29378)
PR Close #29378
2019-03-19 17:14:53 -04:00
Paul Gschwendtner 37cc514f0f ci: add yarn cache for material-unit tests job (#29378)
PR Close #29378
2019-03-19 17:14:53 -04:00
Paul Gschwendtner 80379697e2 ci: debug sandbox issue for circleci team (#29309)
The CircleCI team needs to know what causes the Kernel
inconsistency that most likely causes our no usable sandbox
errors. Therefore we add "uname -r"

PR Close #29309
2019-03-14 16:00:46 -04:00
Paul Gschwendtner b7c17ff207 ci: run legacy saucelabs for every build (#29255)
Recently we moved the Saucelabs job into a cronjob in order to avoid
heavy flakiness that we experienced due to a Saucelabs connect bug
that has been supposedly fixed by the Saucelabs team (no new version
is released yet though).

Our initial assumption was that we very rarely hit specific browser failures
and can therefore move the Saucelabs tests into a cronjob, but after some
days of having the cronjob, we realized that we actually hit browser-specific
failures quite often and that we should run the tests for every PR (like before)

PR Close #29255
2019-03-12 12:31:49 -07:00
Filipe Silva e6117a3a49 ci: rebase PRs on target branch (#29215)
PR Close #29215
2019-03-12 11:34:58 -07:00
Filipe Silva a8b432d55d ci: add descriptive message for merge conflicts (#29215)
Fix #29199

PR Close #29215
2019-03-12 11:34:58 -07:00
Paul Gschwendtner 4227126305 ci(docs-infra): run cli docs examples concurrently (#29103)
PR Close #29103
2019-03-12 10:46:03 -07:00
Paul Gschwendtner 69265b7b5d ci(docs-infra): "test_aio_local_ivy" job should attach ivy package output (#29117)
Currently the "test_docs_examples_ivy" job attaches
the legacy package output, while we can also attach
the Ivy NPM package output. We don't need Ngcc to downlevel
the Angular packages in order to build AIO with Ivy.

PR Close #29117
2019-03-05 11:41:44 -08:00
Paul Gschwendtner 01577b0bed ci: bazel saucelabs test job does not setup circleci bazelrc (#29106)
Currently the `test_saucelabs` job does not use our general
CircleCI bazel configuration. We should set this configuration
up, as it enables better logging, better use of the `xlarge`
resource class, and also sets up Bazel's integrated flakiness
retry functionality.

PR Close #29106
2019-03-05 09:48:38 -08:00
Paul Gschwendtner 58198075f2 ci: increase parallelism for "test_docs_examples" jobs (#28984)
Currently the docs example tests (`test_docs_examples_ivy`
and `test_docs_examples`) are the culprits for a slow-down
in our overall CI turnaround. We need to increase parallelism
in order to make our CI turnaround more _acceptable_. This is
temporary and the long-term goal is to move these tests to Bazel
with remote build execution.

References #28940

PR Close #28984
2019-02-28 10:46:12 -08:00
Paul Gschwendtner 2e43e15e12 ci: "test_docs_examples_ivy" should attach ivy package output (#28984)
Currently the "test_docs_examples_ivy" job attaches
the legacy package output, while we can also attach
the Ivy NPM package output. We don't need Ngcc to downlevel
the Angular packages in order to run the docs examples with Ivy.

PR Close #28984
2019-02-28 10:46:12 -08:00
George Kalpakas d207c4894a ci: speed up `publish_artifacts` CircleCI job for PRs (#29028)
PR Close #29028
2019-02-28 10:39:18 -08:00
Filipe Silva 3d48cde3b1 ci: add bazel_repository_cache to CircleCI workspace (#28935)
PR Close #28935
2019-02-25 08:14:49 -08:00
Filipe Silva 61495a138d ci: rename install job to setup (#28935)
Followup to https://github.com/angular/angular/pull/28928#pullrequestreview-207023800

PR Close #28935
2019-02-25 08:14:49 -08:00
Greg Magolan 10ba91b1d3 build: fix saucelabs_test cronjob in CI (#28939)
PR Close #28939
2019-02-22 16:21:43 -08:00
Filipe Silva f0989b786b ci: share data between jobs using workspaces (#28928)
PR Close #28928
2019-02-22 13:24:11 -08:00
Greg Magolan 3144bf4d73 build: fix CI config typo (#27721)
PR Close #27721
2019-02-22 13:07:08 -08:00
Greg Magolan 75357ecb32 build(bazel): run a number of web tests with karma_web_test in saucelabs in CircleCI (#27721)
PR Close #27721
2019-02-22 13:07:08 -08:00
Paul Gschwendtner 82820b0b2c ci: run saucelabs legacy tests every hour (#28903)
Initially when we introduced the cronjob for Saucelabs,
we specified that the tests should run every 4h. Since the
caretaker needs more recent results when merging PRs
or before creating a release, we should run the saucelabs
tests every hour. This still ensures that PRs are not
affected by Saucelabs flakiness or incidents, and
the caretaker can also react better to real browser
failures (and isn't blocked on a 4h time frame)

PR Close #28903
2019-02-22 09:52:06 -08:00
Filipe Silva 94223a09e5 ci: update circleci post checkout to v2 syntax (#28472)
The `post` key is part of the CircleCI 1.0 syntax is not available in the 2.0 syntax.

PR Close #28472
2019-02-21 22:17:38 -08:00
George Kalpakas 3f2b51b837 ci: make yarn globally available and independent of `$BASH_ENV` (#28889)
Previously, our yarn (which overwrote the pre-installed yarn on the
docker image) was only available through `$BASH_ENV` (as an exported
function). This caused it to be undefined for commands executed in other
shells (e.g. via Node.js' `child_process.spawn()` unless explicitly
configuring it to run with `bash`).

This commit fixes this by making our yarn version available globally via
a symlink (`/usr/local/bin/yarn`).

(This was accidentally broken in #28839.)

PR Close #28889
2019-02-21 12:37:53 -08:00
Paul Gschwendtner 13b96ac91d ci: run saucelabs unit tests as cronjob (#28787)
We no longer want to run Saucelabs for every PR/commit because
Saucelabs has been very flaky recently and it blocks most of the
PRs with a flaky failing state that we cannot fix most of the time due
to upstream Saucelabs failures/incidents. Since real browsers tests
rarely catch browser-specific failures (same as in Material), we should
only run Saucelabs in a cronjob on the upstream branches. This still
ensures/guarantees our browser compatibility, but makes our CI
more stable and the PR workflow more productive.

PR Close #28787
2019-02-20 16:45:59 -08:00
Paul Gschwendtner 623fd3fb5e build: remove legacy-unit-tests-local job (#28703)
Since all unit tests are now run with Bazel, we can remove
the local legacy unit tests job. We still need to keep the
Saucelabs legacy job until we can run all of these web
unit tests with Saucelabs and Bazel.

PR Close #28703
2019-02-20 16:31:48 -08:00
Alex Eagle 9cecb0b5d2 ci: Vendor yarn into our repo (#28839)
This avoids a dynamic dependency on fetching a package from the internet in our CI.
We have observed that this is not 100% reliable.

PR Close #28839
2019-02-20 09:19:03 -08:00
Paul Gschwendtner 00a8b07896 ci: combine "define_env_vars" and "download_yarn" anchor (#28788)
As discussed in https://github.com/angular/angular/pull/28546#discussion_r254068014, we want to combine the
`define_env_vars` and `download_yarn` anchor since downloading Yarn depends on setting up the
environment variables. In addition this simplifies our setup and reduces code-duplication.

PR Close #28788
2019-02-19 12:39:45 -08:00
Paul Gschwendtner dba2a406fd build: remove legacy e2e tests job (#28645)
Now that all e2e integration tests within `modules/` have been
migrated to Bazel, we can remove the legacy e2e tests job.

PR Close #28645
2019-02-13 12:15:02 -08:00
Brandon 99e3a04ea2 ci(docs-infra): test docs examples with Ivy (#28463)
PR Close #28463
2019-02-11 17:16:52 +00:00
Filipe Silva 4b7264f60f ci: use fallback caches (#28545)
This PR adds fallback cache hits for partial cache restoration.

PR Close #28545
2019-02-07 16:45:53 -08:00
George Kalpakas e3032a0d17 ci(docs-infra): ensure `aio_monitoring` failure notifications do not depend on earlier steps (#28555)
Previously, in order for the `aio_monitoring` failure notifications to
work, the steps up to `define_env_vars` should have succeeded. This
meant that any failures in earlier steps would not send notifications.

This commit fixes it by making the notification step independent of the
`define_env_vars` step.

PR Close #28555
2019-02-06 06:21:58 -08:00
George Kalpakas 01bb3c5820 ci: save the cache in `build-npm-packages` job instead of `test` (#28555)
In most cases, it doesn't make a difference, because the cache does
already exist and is not saved. In the few cases where the dependencies
change (and the cache needs to be updated), it makes more sense to save
the cache in the `build-npm-packages` job, because most jobs depend on
it and thus will be able to take advantage of the updated cache right
away.

This seems to be an oversight in b26ac1c22.

PR Close #28555
2019-02-06 06:21:58 -08:00
George Kalpakas 43467c95ab ci: run the `yarn_install` step when necessary (#28555)
PR Close #28555
2019-02-06 06:21:58 -08:00
George Kalpakas 69ecb96569 refactor: use steps consistently in CircleCI config (#28555)
PR Close #28555
2019-02-06 06:21:58 -08:00
George Kalpakas e991d825f5 ci(docs-infra): fix `download_yarn` in `aio_monitoring` (#28555)
The `download_yarn` step depends on the `CI_YARN_VERSION` environment
variable and thus has to be run after the `define_env_vars` step.
Accidentally broken in [#28546][1].

[1]: https://github.com/angular/angular/pull/28546/files#diff-1d37e48f9ceff6d8030570cd36286a61R447

PR Close #28555
2019-02-06 06:21:58 -08:00
Matias Niemelä 979582a2ed build: add some debug info for yarn installs (#28553)
PR Close #28553
2019-02-05 20:15:49 -05:00
Paul Gschwendtner 8d11627e6c build: update yarn to v1.13.0 (#28546)
Currently our version of Yarn is installed through
the "circleci/node" docker image. This is problematic
because in order to be able to update Yarn, we always
need to update the docker image to a version that
comes with the desired Yarn version. Sometimes there
is no docker image with the desired latest Yarn version,
and therefore we cannot easily update the Yarn version.

Additionally updating the docker image also means that
we need to update our version of NodeJS, as well as the
version of `openssl` might have changed (meaning that
our encrypted credential files may not be decodable with
the new version of `openssl`)

PR Close #28546
2019-02-05 16:48:56 -05:00
George Kalpakas 9ef8d2b823 ci: keep job alive when yarn request takes more than 10 minutes (#28458)
Occasionally, yarn's requests take more than 10 minutes to
complete/fail, by which time CircleCI jobs due to no output.

This commit works around the issue by periodically printing something to
stdout.

PR Close #28458
2019-02-05 13:07:41 -05:00
Paul Gschwendtner 07fb4b5677 build: fix bazel repositories not cached on circleci (#28515)
Previously all Bazel repositories were cached on CircleCI
because the `experimental_repository_cache` flag has been
specified and the given repository cache directory has been
included in the CircleCI cache storage.

The directory is currently still included in the CircleCI
cache storage, but the `--repository_cache` flag is no longer
specified, and the cache directory is basically empty all the
time. The flag seems to have been removed accidentally within
SHA c8b70ae8e4.

We should specifiy this flag on the CI again, so that Bazel
doesn't need to install the Bazel managed node modules
all the time. This would slow down analysis phase on CI; and also
makes us dependent on the Yarn/NPM registry which often times out
if we fetch a lot of dependencies.

Also in order to make sure that cached Bazel repositories are
also most of the time in sync with what's currently defined in
the workspace, we need to update the cache key.

PR Close #28515
2019-02-05 13:06:24 -05:00
Paul Gschwendtner f6805de8eb build: fix circleci not restoring cache for PRs (#28480)
Currently whenever someone creates a pull request, the NPM dependencies
are downloaded and installed. This is problematic because we have a lot
NPM dependencies with potentially large files that would need to be
downloaded (e.g. the Bazel binaries).

The caches are currently not being restored because we added the
`{Branch}` variable to the CircleCI cache key. Since every PR has a different
branch name (e.g. `devversion/refs/heads/my-banch`), the cache keys would
never match an existing cache key, and the PR would start fresh by
downloading everything.

We can safely remove the `{Branch}` variable from the cache key because
it does not provide any value since the cached files are based on the state
of the `yarn.lock` file and not based on the current branch name. This reduces
our dependency on the slow and sometimes **flaky** Yarn registry. We should
try to depend as few as possible on external services (e.g. see how Saucelabs
flakiness can cause trouble for the caretaker; same applies to flaky Yarn installs)

PR Close #28480
2019-02-04 02:48:48 -08:00