In the past we had connecitivity issues on Saucelabs. Browsers on
mobile devices were not able to properly resolve the `localhost`
hostname through the tunnel. This is because the device resolves
`localhost` or `127.0.0.1` to the actual Saucelabs device, while it
should resolve to the tunnel host machine (in our case the CircleCI VM).
In the past, we simply disabled the failing devices and re-enabled the
devices later. At this point, the Saucelabs team claimed that the
connecitivy/proxy issues were fixed.
Saucelabs seems to have a process for VMs which ensures that requests to
`localhost` / `127.0.0.1` are properly resolved through the tunnel. This
process is not very reliable and can cause tests to fail. Related issues have been
observed/mentioned in the Saucelabs support docs. e.g.
https://support.saucelabs.com/hc/en-us/articles/115002212447-Unable-to-Reach-Application-on-localhost-for-Tests-Run-on-Safari-8-and-9-and-Edgehttps://support.saucelabs.com/hc/en-us/articles/225106887-Safari-and-Internet-Explorer-Won-t-Load-Website-When-Using-Sauce-Connect-on-Localhost
In order to ensure that requests are always resolved through the tunnel,
we add our own domain alias in the CircleCI's hosts file, and enforce that
it is always resolved through the tunnel (using the `--tunnel-domains` SC flag).
Saucelabs devices by default will never resolve this domain/hostname to the
actual local Saucelabs device.
PR Close#35171
Now that bazel respects the yarn-path value found in .yarnrc, we can
remove the last remaining reliances on our vendoring in
//third_party/github.com/yarnpkg/yarn/
PR Close#35083
In #35004, we started ignoring yarn's engines check for `yarn install`
in AIO's `test-production.sh` script to fix a failure in the
`aio_monitoring_stable` CI job. (See #35004 for details.)
It turns out that the version of yarn used on the stable branch (1.17.3)
`--ignore-engines` is needed on all yarn commands (including `yarn
run`). Thus, #35004 is not enough to fix the failures.
New example failure: https://circleci.com/gh/angular/angular/604341
This commit turns of the engines check for the whole
`aio_monitoring_stable` CI job to fix the failure and make the job more
robust.
PR Close#35033
* Added a /tools/saucelabs/sauce-service.sh script that manages the sauce-connect as a service which is used by the karma-saucelabs.js wrapper to start the service.
* Added /tools/saucelabs/README.md that covers the details of SauceLabs karma testing with Bazel.
PR Close#34769
We are migrating to PullApprove for our PR review management in an attempt
to allow for more granular and equitable code review assignments across the
team. Currently this migration is equivalent in the review assignments
it will create. Once stable, our expectation is that we will be able to
take advantage of PullApproves additional features for things like staged
reviews.
PR Close#34814
The `aio_monitoring_stable` CI job needs to check out (and execute
commands on) some files from the stable branch. Therefore, it needs to
use a version of `yarn` that is compatible with the `engines` version
specified in `aio/package.json`.
In #34902 (and subsequently #34952 for the 8.2.x branch), the way to
access our vendored version of yarn changed. This commit ensures that we
checkout the necessary files from the stable branch to use an
appropriate yarn version.
PR Close#34954
We rename the `material-unit-tests` job to `components-repo-unit-tests`
because the job runs all unit tests found in the Angular Components repository.
This includes the Angular CDK, Angular Material and more. Also the repository has
been renamed from `angular/material2` to `angular/components` in the past.
PR Close#34898
Creates a Bazel macro that can be used to test packages for
circular dependencies. We face one limitation with Bazel:
* Built packages use module imports, and not relative source file
paths. This means we need custom resolution.
Fortunately, tools like `madge` support custom resolution.
Also removes the outdated `check-cycles` gulp task that
didn't catch circular dependencies. It seems like the test
became broken when we switched the packages-dist output to Bazel. It
breaks because the Bazel output doesn't use relative paths, but uses
the module imports. This will be handled in the new Bazel macro/rule.
PR Close#34774
Since we always set up bazel on both our windows and linux CI
runs, we should consider this configuration part of the environment
setup.
PR Close#34834
Since we always execute our bazel commands using RBE on CI
we should not have separete tasks of configuring bazel
and setting up RBE for bazel. Enabling RBE for bazel should
be part of our bazel configuration process on CI.
PR Close#34834
Previously, we added patches to the `angular/components` repository in
the `material-unit-tests` job in order to be able to run their tests
with TypeScript 3.7. This is no longer needed since the components
repository updated to Ts 3.7 already. Hence we can remove the patches.
PR Close#34863
Updates the `material-unit-tests` job commit to the latest
available commit. We need to update since the `angular/components`
repository updated to TypeScript 3.7, which means that we can remove
the TS 3.7 workarounds we applied to get the job green when framework
updated to TS 3.7.
PR Close#34863
Updates the SHA that will be tested against in the `material-unit-tests` job to the latest commit in the components repository. SHA 71955d2e194bfc5561f25daea16e68af266d6ff9 is needed in order to compile repository with typescript 3.7
PR Close#33717
The components repository has a Yarn resolution to ensure that
dgeni-packages uses a specific TypeScript version. This resolution
causes the specified TS version to be considered as candidate for the
`@angular/bazel` peer dependency. Ultimately, Yarn decides to use
the TypeScript version from the resolution for `@angular/bazel`,
and builds will fail due to a version mismatch.
This is because `tsickle` will use the hoisted top-level TS
version (set to `3.7.4` ), while `@angular/bazel` uses the
version from the resolution (at the time of writing: v3.6.4)
PR Close#33717
Updates the commit SHA the `material-unit-tests` CircleCI
job runs against. We need to include a commit that makes
the node module installation more determinisitc (i.e. ensuring
that `tsickle` is always hoisted at the node module root).
31a50f7ad2
PR Close#33717
Also add comment that cache_key version should be bumped when switching forks or branches and added a comment from @devversion explaining how the fallback cache key works.
PR Close#34589
The `aio_monitoring_stable` CI job is triggered as a cronjob on the
master branch and its purpose is to run some e2e tests against the
deployed stable version of the docs web-app at https://angular.io/. In
order for the tests to be compatible with the deployed version of the
web-app (which gets deployed from the stable branch), the stable branch
is checked out in git as part of the CI job.
Previously, we only checked out the `aio/` directory from the stable
branch, leaving the rest of the code at master. This doesn't matter as
long as the commands used to run the tests do not rely on code outside
of `aio/`. However, it turns out that there _is_ code outside of `aio/`
that affects the executed commands: It is our vendored version of yarn
(in `third_party/github.com/yarnpkg/`), which overwrites the global yarn
installed on the docker image on CI and must match the version range
specified in `aio/package.json > engines`.
Using the yarn version checked out from the master branch with the
`aio/` code checked out from the stable branch can lead to failures
such as [this one][1].
This commit fixes the problem by checking out both the `aio/` and
`third_party/github.com/yarnpkg/` directories from the stable branch and
re-running the steps to overwrite the global yarn executable with our
own version from `third_party/github.com/yarnpkg/`. This ensures that
the version of yarn used will be compatible with the version range
specified in `aio/package.json > engines`.
NOTE:
We cannot checkout everything from the stable branch, since the CI
config (`.circleci/config.yml` from the master branch) may try to run
certain scripts (such as `.circleci/get-vendored-yarn-path.js`) that are
not available on the stable branch. Therefore, we should only check out
the necessary bits from the stable branch.
[1]: https://circleci.com/gh/angular/angular/567315
PR Close#34451
Updates the material-unit-tests job commit SHA to the most recent
commit at the time of writing. The goal is to run the unit tests
with 6ae74a0eb2
that improved stability of a few menu tests that were flaky.
e.g. https://circleci.com/gh/angular/angular/564650
PR Close#34430
Currently the `saucelabs_view_engine` job fails because
the Saucelabs Bazel run script thinks that `--config=saucelabs`
is a flag targeting the actual script. This is not the case and
the flag should be actually part of the bazel command.
PR Close#34429
Currently we only run Saucelabs on PRs using the legacy View Engine
build. Switching that build to Ivy is not trivial and there are various
options:
1. Updating the R3 switches to use POST_R3 by default. At first glance,
this doesn't look easy because the current ngtsc switch logic seems to
be unidirectional (only PRE_R3 to POST_R3).
2. Updating the legacy setup to run with Ivy. This sounds like the easiest
solution at first.. but it turns out to be way more complicated. Packages
would need to be built with ngtsc using legacy tools (i.e. first building
the compiler-cli; and then building packages) and View Engine only tests
would need to be determined and filtered out. Basically it will result in
re-auditing all test targets. This is contradictory to the fact that we have
this information in Bazel already.
3. Creating a new job that runs tests on Saucelabs with Bazel. We specify
fine-grained test targets that should run. This would be a good start
(e.g. acceptance tests) and also would mean that we do not continue maintaining
the legacy setup..
This commit implements the third option as it allows us to move forward
with the general Bazel migration. We don't want to spend too much time
on our legacy setup since it will be removed anyway in the future.
PR Close#34277
We keep a version of yarn in the repo, at
`third_party/github.com/yarnpkg/`. All CI jobs should use that version
for consistency (and easier updates).
Previously, the Windows jobs did not use the local version. They used
the version that came pre-installed on the docker image that we used.
This made it more difficult to update the yarn version (something that
we might want to do independently of updating other dependencies, such
as Node.js).
This commit fixes this by setting up the Windows CI jobs to also use the
local, vendored version of yarn.
PR Close#34384
We keep a version of yarn in the repo, at
`third_party/github.com/yarnpkg/`. All CI jobs (including Windows ones)
should use that version for consistency (and easier updates). The path
to the actual `yarn.js` script, however, changes depending on the
version (e.g. `third_party/github.com/yarnpkg/v1.21.1/...`).
(NOTE: The Windows jobs are currently not using this local version, but
that should be fixed in a subsequent commit.)
Previously, when updating the local version of yarn, we would
potentially have to update the path in several places.
This commit addresses the problem by adding a Node.js script that infers
the correct path. The script can be used in all places where we need to
use the local version of yarn (including both Linux and Windows CI
jobs), thus eliminating the need to update the path in several places.
PR Close#34384
Since #32537, the `.circleci/get-commit-range.js` script is no longer
used in `.circleci/env.sh`. This commit removes the now unused local
variable to the script's path.
PR Close#34384
This reverts commit f029af50820765019413fa319330830306b80d6a while we investigate
some failures on master on Circle CI. Currently the Windows tests and the
"test-ivy-aot" jobs are red because of incompatible yarn versions.
PR Close#34402
Before updating to remove the compile build variable, we must update
the components unit test integrations to a sha in the components
repo which no longer relies on the compile build variable.
PR Close#34280
Currently all saucelabs usage in our repos is done using the same
account angular-ci. By migrating to use individual accounts
for each repo, we can better track the usage for each repo as well
as providing concurrency limiting on a per repo basis.
Additionally, we no longer use two separate accounts based on being
on master or a PR branch, so this logic can be removed.
PR Close#34233
In efbbae5a4, the `publish_packages_as_artifacts` CircleCI job was
created to publish the build artifacts of PR builds on CI. In a8f4f14bd,
its scope was expanded to also publish build artifacts on non-PR builds.
The published artifacts names are constructed based on the PR number
(e.g. include `-pr12345-`), so on non-PR builds the names do not reflect
the source branch (instead, they include `-prfalse-`).
This commit fixes this by using the current branch name. For example,
artifact names for the `master` or `9.0.x` branch will include
`-master-` and `-9.0.x-` respectively (instead of `-prfalse-`).
(NOTE: For PRs, where branch name is `pull/12345`, the branch name is
transformed to `pr12345`.)
PR Close#33957
In #33823, `scripts/package-builds.sh` (which is used by both
`build-packages-dist.sh` and `build-ivy-npm-packages.sh`) was updated to
use `realpath`. It turns out that `realpath` does not exist on macOS, so
the build scripts do not work there.
In order to fix this (and also reduce the likelihood of introducing
similar issues in the future), this commit changes these bash scripts to
Node.js scripts (using [ShellJS](https://github.com/shelljs/shelljs) for
a cross-platform implementation of Unix shell commands where necessary).
PR Close#33854
Since #33321, Angular packages have been persisted on each build as
CircleCI build artifacts (`.tgz` files), which can be used to install
dependencies on a project (for the purpose of testing or trying out a
change before a PR being merged and without having to build the packages
from source locally).
Previously, only packages published to npm under the `@angular` scope
were persisted as build artifacts.
This commit adds the `zone.js` package to the list of persisted
packages.
Fixes#33686
PR Close#33733
In #33046, internal uses of `zone.js` were switched to reference it
directly from source (built with Bazel) instead of npm. As a result, the
necessary scripts were updated to build `zone.js` as necessary. However,
some `integration/**/debug-test.sh` scripts were missed (apparently
because they are not used on CI, but only locally as helpers for
debugging the integration projects).
This commit updates the `scripts/build-packages-dist.sh` script to also
build `zone.js`, so that other scripts (such as the various
`debug-test.sh` scripts) can use it.
PR Close#33733
Updates the commit we run the `material-unit-tests` job
against. The latest commit includes 1255139a38
This commit reduces the flakiness of a `MatMenu` test and therefore
improves the stability of the material-unit-tests job.
Example failing build: https://circleci.com/gh/angular/angular/521625
PR Close#33716
The Material units tests job has been disabled with the
typescript upgrade PR since the components repository was
still on an old TypeScript version (due to cyclic dependencies
between the framework and components repository).
Since the components repository has been updated to `9.0.0-next.15`
and now uses the compatible TypeScript version, we can re-enable
ds
Related to this change, we need to ignore the `package.json`
engines when installing the dependencies of the components repo. This
is because the components repo already updated to NodeJS v12 and Yarn
v1.19.1. This is not the case for the CI setup of framework. For now,
since we don't want to change the dev setup (as it slows down
development), we temporarily disable the engines.
Additionally, the material unit tests job now depends on the actual
release packages (not on the ngtsc compiled ones). This is because
the components repo setup relies on NGCC being run. This is also
helpful for validating ngcc against the framework packages.
PR Close#33530
Previously, when one wanted to try out the changes from a PR before it
was merged, they had to check out the PR locally and build the Angular
packages themselves (which is time-consuming and wasteful given that the
packages have already been built on CI).
This commit persists all Angular packages on each build as `.tgz` files,
which can be used to install dependencies on an project (supported by
both [npm][1] and [yarn][2]). In addition to individual `.tgz` files for
each package, a `.tgz` file including all packages is also stored, which
can be used to test the packages locally by overwriting the ones in the
`node_modules/` directory of a project.
CircleCI [build artifacts][3] an be used for longer-term storage of the
outputs of a build and are designed to be useful around the time of the
build, which suits our needs.
[1]: https://docs.npmjs.com/cli/install.html
[2]: https://yarnpkg.com/lang/en/docs/cli/add
[3]: https://circleci.com/docs/2.0/artifacts
PR Close#33321
Updates the commit of the Angular components repository for
which the `material-unit-tests` job runs tests against. We need to
update to the latest commit at the time of writing, in order to be able
to remove the gesture tests from the material-ci/blocklist.
This is now possible because the Angular components repository
removes the dependency on `hammerjs` completely.
PR Close#33221
Updates the `material-unit-test` job to run tests against
the latest commit of the Angular Components repository.
The components repository updated to rules_nodejs#0.38.2 before Angular Bazel did.
To do this, the `@angular/bazel` v0.38.2 compatibility changes were patched on postinstall.
This now conflicts because we install a `@angular/bazel` version in the `material-unit-tests` job
that already includes these compatibility changes. This would result in the patch being a noop
for which the `patch` command throws an error.
We can remove this once components can install a released version of `@angular/bazel` that is
compatible with `rules_nodejs#0.38.2`.
PR Close#33073
Updates the `material-unit-test` job to run tests against
the latest commit of the Angular Components repository.
We need to update to a version where components already
updated to `rules_nodejs#0.38.0`.
PR Close#32889
The `init_environment` custom command is used in almost all other jobs.
The this is implemented in CircleCI is that the steps of the command are
inlined into the other jobs.
Some of the `init_environment` commands are quite long and (since the
default display name for a step is its command) they clutter the
CircleCI UI. Additionally, multiple related commands are shown as
separate steps, which makes it more difficult to get to the actual steps
of a job.
This commit improves this by:
1. Defining explicit, short names for steps with long commands.
2. Grouping multiple related steps into one.
PR Close#32998
The angular.io project uses Angular and CLI v9, which by default turns
on Ivy mode. However, since ec4381dd4, we explicitly opt out of Ivy.
This commit removes the `enabledIvy: false` configuration, thus allowing
the default behavior of having Ivy on.
NOTE:
This commit only changes the angular.io projects. The docs examples need
to be updated separately (first to Angular and CLI v9 and then to Ivy).
PR Close#32923
Removes the selection strategies for `AngularTemplateCompile` and `TypeScriptCompile` causing
them to use the automatic selection strategies instead.
PR Close#32678
Previously, the docker images used on CI where specified by a tag
(`10.16` and `10.16-browsers`). Since tags are not immutable, this only
pins specific characteristics of the environment (e.g. the OS type and
the Node.js version), but not others. Especially when using a tag that
does not specify the patch version (e.g. `10.16` instead of `10.16.0`),
it is inevitable that the image will change at some point, potentially
leading to unrelated failures due to changes in the environment.
One source of such failures can be the Chrome version used in tests.
Since we install a specific ChromeDriver version (that is only
compatible with specific Chrome version ranges), unexpectedly updating
to a newer Chrome version may break the tests if the new version falls
outside the range of supported version for our pinned ChromeDriver.
Using a tag that specifies the patch version (e.g. `10.16.0`) or even
the OS version (e.g. `10.16.0-buster`) is safer (i.e. has a lower
probability of introducing the kind of breakages described above), but
is still not fully hermetic.
This commit prevents such breakages by pinning the docker images by ID.
Image IDs are based on the image's digest (SHA256) and are thus
immutable, ensuring that all CI jobs will be running on the exact same
image.
See [here][1] for more info on pre-built CircleCI docker images and more
specifically [pinning images by ID][2].
[1]: https://circleci.com/docs/2.0/circleci-images
[2]: https://circleci.com/docs/2.0/circleci-images#using-a-docker-image-id-to-pin-an-image-to-a-fixed-version
PR Close#32602
This commit expands the `lint` CircleCI job to also run the
`tools/verify-codeownership.js` script. This script verifies that some
important files/directories in the codebase have code-owners assigned in
`.github/CODEOWNERS`.
The main purpose of this change is to prevent adding new directories
(e.g. packages or docs guides/examples) without assigning appropriate
code-owners. When no code-owner is explicitly assigned, corresponding
PRs will automatically request reviews from @igorminar, who is the
"fall-back" code-owner.
PR Close#32577
Updates the SHA that will be tested against in the `material-unit-tests` job
to the latest commit in the components repository. SHA 2817c9e2faa4140342336987a692d5dd30bf24c2
is needed in order to make the `material-unit-tests` job pass after the remove
of the `protractor_web_test_suite` bazel rule from @angular/bazel. `protractor_web_test_suite` is
now provided by the @bazel/protractor npm package.
PR Close#32485
The commit range that is associated with a CI build is used for a couple
of things (mostly related to payload-size tracking):
- Determine whether a size change was caused by application code or
dependencies (or both).
- Add the messages of the commits associated with the build (and thus
the payload-size change).
NOTE: The commit range is only used on push builds.
Previously, the commit range was computed based on the
`CIRCLE_COMPARE_URL` environment variable. With [CircleCI Pipelines][1]
enabled, `CIRCLE_COMPARE_URL` is no longer available and the commit
range cannot be reliably detected.
This commit switches `CI_COMMIT_RANGE` to only include the last commit.
This can be less accurate in some rare cases, but is true in the
majority of cases (on push builds). Additionally, it stores the CircleCI
build URL in the database along with the payload data, so the relevant
info can be retrieved when needed.
[1]: https://circleci.com/docs/2.0/build-processing
PR Close#32537
The docs examples e2e tests have been quite flaky recently. This causes
the `test_docs_examples` and `test_docs_examples_ivy` CircleCI jobs to
often fail (and block PRs) without a real reason.
This commit adds support for re-running failed docs examples e2e tests
and configures the `test_docs_examples` and `test_docs_examples_ivy`
jobs to try running each test that fails a second time, before giving up
and marking it as failed.
Closes#31841Closes#31842
PR Close#32497
These were initally added to allow for CircleCI to provide a better failure UI using junit data.
This information is not currently being created for bazel tests and once it is created it will
be available on the cloud status pages created by the BEP for each bazel execution.
PR Close#32292
Updates the SHA that will be tested against in the `material-unit-tests` job
to the latest commit in the components repository. SHA 18b9ef3f5529f0fa8f034944681486447af7b879
is needed in order to make the newly introduced material-ci test blocklist effective.
PR Close#32243
We don't need to build this branch as it's informative for the purposes of figuring out
the diff between the master and what's synced into google3.
PR Close#32202
Updates the `material-unit-tests` job to the latest commit
on the components repository. 097f4335a4e0b6e6b579829ae3a9cffce6292d2b.
This commit ensures that the postinstall script does not run NGC
on schematic code from `@angular/core`. Running NGC on the
generated schematic code can cause unexpected issues as some
migrations import types directly from `@angular/compiler-cli`
while the entry-point is not usable in all cases.
See: https://github.com/angular/angular/issues/29220.
PR Close#31650
The `setup-local` npm script uses `NgPackagesInstaller` to replace the
Angular packages with the locally built ones. Previously, it would (a)
assume that the packages were built and (b) it would do anything if the
currently installed versions already correspond to locally built
packages (even if not the latest version).
This could lead to all sorts of errors, such as:
- Confusing error messages, `dist/packages-dist/` was missing.
- Using outdated build artifacts from `dist/packages-dist/` without a
warning.
- Not installing the latest locally built packages, because the ones
installed already corresponded to locally built (but older) ones.
This commit fixes these issues by ensuring that:
- The local packages are always built before being used by
`NgPackagesInstaller`.
- The local packages are installed, even if the ones install already
correspond to local packages.
NOTE: Special `*-ci` scripts are introduced (for use on CI) that skip
building the local packages, since that step would have been taken
care of (in a more efficient way) in a previous CI step.
PR Close#31985
No longer locks the Material unit tests job to a specific branch, but rather allows
locking to a specific commit from a given branch. This allows us to use the
"master" branch from the `components` repository.
PR Close#31569
Previously, the ivy-2019 branch of the Material (aka components) repo was
used, which contains some changes that were necessary to work with Ivy.
These changes are not longer necessary, as Material's master branch is
fully working with Ivy today. To be up-to-date with recent Material
development and its support for more recent dependencies, e.g. TypeScript,
it is desirable for us to be on a newer version of Material.
This commit moves the Material tests away from the ivy-2019 branch, to a
recent commit on master. We are not targeting the master branch itself,
as that would introduce a moving target into Angular's CI checks, which
is undesirable.
Lastly, the usage of gulp to run Material's tests is changed into using
Bazel, as Material itself is now also built with Bazel.
PR Close#31569
Previously, the `test_saucelabs_bazel` job was run as part of the
`saucelabs_test` workflow every hour (for every "publish branch";
currently 8.0.x, 8.1.x and master).
Since this job runs a subset of the tests in
`legacy-unit-tests-saucelabs` (see [BUILD.bazel][1]) and is just a
proof-of-concept for running tests on SauceLabs via bazel, there is
little point in running is on all branches. It is also wasteful to run
it every hour, even if there were no changes pushed into the branch
since the last run.
This commit makes the job part of `default_workflow`, but limits it to
only be run on master builds (not on other branches or PRs). Based on
its recent history, the job is relatively stable and (since it will only
be run on master builds) it is not expected to affect our dev workflow.
[1]: https://github.com/angular/angular/blob/ef44f51d5/BUILD.bazel#L66-L92
PR Close#31636
Enables remote caching for CI jobs.
This configuration:
always reads from build cache on CI
only write to build cache for local builds for non-PR CI run
PR Close#31204
No longer uses docker in order to pull down the toolchain configs
for remote build execution. We don't need to make docker a prerequisite
for working on the Angular repository since we can leverage the checked-in
toolchain configurations from the `@bazel-toolchains` repository.
PR Close#31251
The publish_snapshots job is currently not able to decode the Github
token because the openssl version changed. This is because the default
digest for more recent openssl version has been updated and the github
token file has been encrypted with an old digest. We need to ensure
that the md5 digest is used for decryption as that matches the
digest used for encryption.
PR Close#31099
Updates the NodeJS version to the latest stable version at the time of
writing (v10.16.0). We need to update our image to use a minimum NodeJS
version of v10.15.0 because new CLI apps automatically install a non-locked
version of selenium-webdriver that now requires NodeJS >= 10.15.0 since the
latest release of 17th June 2019 (4.0.0-alpha.3).
See CI failures: https://circleci.com/gh/angular/angular/359077
PR Close#31088
Previously, we run the PWA score tests before unit/e2e tests, because
the latter would destroy the `dist/` directory required by the former.
Since cli@6, unit/e2e tests no longer detroy the `dist/` directory, so
it is now safe to run the unit/e2e tests first. This is preferrable,
since they are conceptually lower-level and any error messages (in case
of breakage) are more specific/actionable.
Related discussion about cli behavior:
- angular/angular-cli#4366
- angular/angular-cli#14701
PR Close#31047
After this PR is merged, maintainers no longer need to update .bazelrc
file, toolchain and platform related flags for RBE builds and tests
(unless there is a breaking change in Bazel related to those flags).
Maintainers just need to update the pin of @bazel-toolchains repo
regularly in the packages/bazel/package.bzl file according to
https://releases.bazel.build/bazel-toolchains.html to include the
latest checked-in toolchain configs. If rbe_autoconfig() cannot find
appropriate toolchain configs for the version of Bazel in the version of
@bazel_toolchains repo that is currently used by this project, it will pull
down the container and generate the configs on the fly as the beginning
of the build/test.
PR Close#29336
The `aio_monitoring_next` CircleCI job was disabled due to a failure in
[302254](https://circleci.com/gh/angular/angular/302254). It turned out
the failure was caused because the job happened to run after a change
had been merged into master and right before it was deployed to
https://next.angular.io/ causing the tests not to match the deployed
version.
This commit re-enables the job and moves it to a different time, when it
is less likely that PRs will be being merged (and thus reducing the risk
of a similar timming issue).
Fixes#30101
PR Close#30168
Previously, the `aio_monitoring_stable` job (which runs tests against
https://angular.io/) was using the tests from the master branch. As a
result, if the master branch included changes in those tests that were
not yet backported to the stable branch (and thus deployed to
https://angular.io/), the tests would fail.
This commit fixes this by using the tests from the stable branch to test
against https://angular.io/.
Fixes#30101
PR Close#30110
Previously, the `aio_monitoring` job was testing both the stable
(https://angular.io/) and the @next (https://next.angular.io/) versions.
This commit splits the tests into two separate jobs (still run as part
of the same workflow). This speeds up the tests (since the two jobs can
now run in parallel) and makes it easier to isolate failures (e.g.
identify which branch is failing, disable one of the two, etc.).
(Credits to @petebacondarwin 😉)
PR Close#30110
The job started failing for https://angular.io/, due to changes in tests
that only affected https://next.angular.io/, and was disabled in #30102.
This commit re-enables the job (since it does not block anything and it
will be fixed in a subsequent commit).
PR Close#30110
Redirects that were updated in 24c61cb63e break the aio_monitoring CircleCI job, since we run the tests against the production angular.io site (that doesn't have the latest redirects config yet).
This change temporary disables the aio_monitoring job to avoid failures for other PRs. The problem will be resolved and the job will be enabled in followup PRs.
PR Close#30102
We recently added the "uname -a" command to the CircleCI
integration test in order to identify Linux kernels that
broke Chrome sandboxing.
Since this issue seems to be claimed as fixed by the CircleCI
support and we didn't see any sandboxing issues the last month,
we are removing the debugging command we added.
PR Close#30006
To better test ngcc (in addition to Ivy) on angular.io, change the
`test_aio_local_ivy` CircleCI job to use the pre-ivy Angular packages
(and have ngcc transform them to Ivy ones).
PR Close#29989
The server used for testing on localhost has less optimizations (e.g.
serves uncompressed files), so we need to wait longer the ServiceWorker
to be loaded and registered to allow Lighthouse to reliably detect it,
especially on slower environments (e.g. CI).
Related: https://github.com/GoogleChrome/lighthouse/issues/5527#issuecomment-483710849Fixes#29910
PR Close#29953
The PWA score tests have been occasionally failing on CI recently
(possibly due to CI VM/network issues).
This commit temporarily disables them, until we investigate the
root-cause and/or put a work-around in place.
The PWA score tests are still run against the deployed versions (which
don't suffer as much) or PRs with public previews (as part of the
`test_aio_preview` job) and on upstream builds (as part of the
`deploy_aio` job).
Related to #29910.
[1]: https://github.com/angular/angular/blob/3a836c362/.circleci/config.yml#L390
PR Close#29911
Run all targets with RBE config. Previously we filtered out one target, //tools/ts-api-guardian:tests, and ran that with a different bazelrc
PR Close#29731
Currently we cache the Material `node_modules` after
the `run_angular_material_unit_tests.sh` completed. This
means that the cache will incorrectly contain the Ivy NPM
package output which might be incompatible with the
other Material dependencies. e.g. the Material postinstall
command now uses a different NGC version that does not
work with the `typescript` version that has been specified in
the Material project.
PR Close#29416
The CircleCI team needs to know what causes the Kernel
inconsistency that most likely causes our no usable sandbox
errors. Therefore we add "uname -r"
PR Close#29309
Recently we moved the Saucelabs job into a cronjob in order to avoid
heavy flakiness that we experienced due to a Saucelabs connect bug
that has been supposedly fixed by the Saucelabs team (no new version
is released yet though).
Our initial assumption was that we very rarely hit specific browser failures
and can therefore move the Saucelabs tests into a cronjob, but after some
days of having the cronjob, we realized that we actually hit browser-specific
failures quite often and that we should run the tests for every PR (like before)
PR Close#29255
Currently the "test_docs_examples_ivy" job attaches
the legacy package output, while we can also attach
the Ivy NPM package output. We don't need Ngcc to downlevel
the Angular packages in order to build AIO with Ivy.
PR Close#29117
Currently the `test_saucelabs` job does not use our general
CircleCI bazel configuration. We should set this configuration
up, as it enables better logging, better use of the `xlarge`
resource class, and also sets up Bazel's integrated flakiness
retry functionality.
PR Close#29106
Currently when `sauce-connect` times out after 2min, we just
print a message saying that the SauceLabs tunnel didn't establish
within 2min. In order to make debugging easier, we now print the
full log file output on failure.
PR Close#29084
Currently the docs example tests (`test_docs_examples_ivy`
and `test_docs_examples`) are the culprits for a slow-down
in our overall CI turnaround. We need to increase parallelism
in order to make our CI turnaround more _acceptable_. This is
temporary and the long-term goal is to move these tests to Bazel
with remote build execution.
References #28940
PR Close#28984
Currently the "test_docs_examples_ivy" job attaches
the legacy package output, while we can also attach
the Ivy NPM package output. We don't need Ngcc to downlevel
the Angular packages in order to run the docs examples with Ivy.
PR Close#28984
Initially when we introduced the cronjob for Saucelabs,
we specified that the tests should run every 4h. Since the
caretaker needs more recent results when merging PRs
or before creating a release, we should run the saucelabs
tests every hour. This still ensures that PRs are not
affected by Saucelabs flakiness or incidents, and
the caretaker can also react better to real browser
failures (and isn't blocked on a 4h time frame)
PR Close#28903
Previously, our yarn (which overwrote the pre-installed yarn on the
docker image) was only available through `$BASH_ENV` (as an exported
function). This caused it to be undefined for commands executed in other
shells (e.g. via Node.js' `child_process.spawn()` unless explicitly
configuring it to run with `bash`).
This commit fixes this by making our yarn version available globally via
a symlink (`/usr/local/bin/yarn`).
(This was accidentally broken in #28839.)
PR Close#28889
JiaLiPassion
Pete Bacon Darwin
Paul Gschwendtner
Joey Perrott
George Kalpakas
Greg Magolan
Andrius
Filipe Silva
atscott
Kara Erickson
Igor Minar
Joey Perrott
Matias Niemelä
JoostK
Alex Eagle
Xin Gao
Andrew Kushnir
Alex Eagle