Recently we moved the Saucelabs job into a cronjob in order to avoid
heavy flakiness that we experienced due to a Saucelabs connect bug
that has been supposedly fixed by the Saucelabs team (no new version
is released yet though).
Our initial assumption was that we very rarely hit specific browser failures
and can therefore move the Saucelabs tests into a cronjob, but after some
days of having the cronjob, we realized that we actually hit browser-specific
failures quite often and that we should run the tests for every PR (like before)
PR Close#29255
Currently the "test_docs_examples_ivy" job attaches
the legacy package output, while we can also attach
the Ivy NPM package output. We don't need Ngcc to downlevel
the Angular packages in order to build AIO with Ivy.
PR Close#29117
Currently the `test_saucelabs` job does not use our general
CircleCI bazel configuration. We should set this configuration
up, as it enables better logging, better use of the `xlarge`
resource class, and also sets up Bazel's integrated flakiness
retry functionality.
PR Close#29106
Currently the docs example tests (`test_docs_examples_ivy`
and `test_docs_examples`) are the culprits for a slow-down
in our overall CI turnaround. We need to increase parallelism
in order to make our CI turnaround more _acceptable_. This is
temporary and the long-term goal is to move these tests to Bazel
with remote build execution.
References #28940
PR Close#28984
Currently the "test_docs_examples_ivy" job attaches
the legacy package output, while we can also attach
the Ivy NPM package output. We don't need Ngcc to downlevel
the Angular packages in order to run the docs examples with Ivy.
PR Close#28984
Initially when we introduced the cronjob for Saucelabs,
we specified that the tests should run every 4h. Since the
caretaker needs more recent results when merging PRs
or before creating a release, we should run the saucelabs
tests every hour. This still ensures that PRs are not
affected by Saucelabs flakiness or incidents, and
the caretaker can also react better to real browser
failures (and isn't blocked on a 4h time frame)
PR Close#28903
Previously, our yarn (which overwrote the pre-installed yarn on the
docker image) was only available through `$BASH_ENV` (as an exported
function). This caused it to be undefined for commands executed in other
shells (e.g. via Node.js' `child_process.spawn()` unless explicitly
configuring it to run with `bash`).
This commit fixes this by making our yarn version available globally via
a symlink (`/usr/local/bin/yarn`).
(This was accidentally broken in #28839.)
PR Close#28889
We no longer want to run Saucelabs for every PR/commit because
Saucelabs has been very flaky recently and it blocks most of the
PRs with a flaky failing state that we cannot fix most of the time due
to upstream Saucelabs failures/incidents. Since real browsers tests
rarely catch browser-specific failures (same as in Material), we should
only run Saucelabs in a cronjob on the upstream branches. This still
ensures/guarantees our browser compatibility, but makes our CI
more stable and the PR workflow more productive.
PR Close#28787
Since all unit tests are now run with Bazel, we can remove
the local legacy unit tests job. We still need to keep the
Saucelabs legacy job until we can run all of these web
unit tests with Saucelabs and Bazel.
PR Close#28703
As discussed in https://github.com/angular/angular/pull/28546#discussion_r254068014, we want to combine the
`define_env_vars` and `download_yarn` anchor since downloading Yarn depends on setting up the
environment variables. In addition this simplifies our setup and reduces code-duplication.
PR Close#28788
Previously, in order for the `aio_monitoring` failure notifications to
work, the steps up to `define_env_vars` should have succeeded. This
meant that any failures in earlier steps would not send notifications.
This commit fixes it by making the notification step independent of the
`define_env_vars` step.
PR Close#28555
In most cases, it doesn't make a difference, because the cache does
already exist and is not saved. In the few cases where the dependencies
change (and the cache needs to be updated), it makes more sense to save
the cache in the `build-npm-packages` job, because most jobs depend on
it and thus will be able to take advantage of the updated cache right
away.
This seems to be an oversight in b26ac1c22.
PR Close#28555
Currently our version of Yarn is installed through
the "circleci/node" docker image. This is problematic
because in order to be able to update Yarn, we always
need to update the docker image to a version that
comes with the desired Yarn version. Sometimes there
is no docker image with the desired latest Yarn version,
and therefore we cannot easily update the Yarn version.
Additionally updating the docker image also means that
we need to update our version of NodeJS, as well as the
version of `openssl` might have changed (meaning that
our encrypted credential files may not be decodable with
the new version of `openssl`)
PR Close#28546
Occasionally, yarn's requests take more than 10 minutes to
complete/fail, by which time CircleCI jobs due to no output.
This commit works around the issue by periodically printing something to
stdout.
PR Close#28458
Previously all Bazel repositories were cached on CircleCI
because the `experimental_repository_cache` flag has been
specified and the given repository cache directory has been
included in the CircleCI cache storage.
The directory is currently still included in the CircleCI
cache storage, but the `--repository_cache` flag is no longer
specified, and the cache directory is basically empty all the
time. The flag seems to have been removed accidentally within
SHA c8b70ae8e4.
We should specifiy this flag on the CI again, so that Bazel
doesn't need to install the Bazel managed node modules
all the time. This would slow down analysis phase on CI; and also
makes us dependent on the Yarn/NPM registry which often times out
if we fetch a lot of dependencies.
Also in order to make sure that cached Bazel repositories are
also most of the time in sync with what's currently defined in
the workspace, we need to update the cache key.
PR Close#28515
Currently whenever someone creates a pull request, the NPM dependencies
are downloaded and installed. This is problematic because we have a lot
NPM dependencies with potentially large files that would need to be
downloaded (e.g. the Bazel binaries).
The caches are currently not being restored because we added the
`{Branch}` variable to the CircleCI cache key. Since every PR has a different
branch name (e.g. `devversion/refs/heads/my-banch`), the cache keys would
never match an existing cache key, and the PR would start fresh by
downloading everything.
We can safely remove the `{Branch}` variable from the cache key because
it does not provide any value since the cached files are based on the state
of the `yarn.lock` file and not based on the current branch name. This reduces
our dependency on the slow and sometimes **flaky** Yarn registry. We should
try to depend as few as possible on external services (e.g. see how Saucelabs
flakiness can cause trouble for the caretaker; same applies to flaky Yarn installs)
PR Close#28480
By default, `webdriver-manager update` will download the latest
ChromeDriver version, which might not be compatible with the Chrome
version included in the [docker image used on CI], causing CI failures.
Previously, we used to pin the ChromeDriver version on CI in
[ngcontainer's Dockerfile][2]. This was accidentally broken in #26691,
while moving from ngcontainer to default CircleCI docker images.
This commit fixes the issue by pinning ChromeDriver to a known
compatible version.
[1]: bfd48d156d/.circleci/config.yml (L16)
[2]: bfd48d156d/tools/ngcontainer/Dockerfile (L63)
PR Close#28494
* No longer builds the example e2e tests using "tsc". The examples are now built with Bazel and can therefore be built with Ivy by using the `--define=compile=aot` switch.
* No longer runs the example e2e tests using the protractor CLI. example e2e tests are executed with the Bazel protractor rule and can therefore run incrementally.
NOTE: Unit tests found within the examples are still running within the legacy jobs.
PR Close#28402
* Groups the two sharded `test_docs_examples` job using CircleCI's `parallelism` feature. This makes the amount of jobs that show up on a PR, more reduced and also reduces code duplication for maintaining the Circle job definition.
PR Close#27937
Moving the tests over to CircleCI in pretty much "as-is" state just so that we can drop the dependency on Travis.
In the followup changes we plan to migrate these tests to run on sauce under bazel. @gregmagolan is working on that.
I've previously verified that all the tests executed in legacy-unit-tests-local already under bazel.
Therefore the legacy-unit-tests-local job is strictly not necessary any more, but given how flaky legacy-unit-tests-saucelabs is,
it is good to have the -local job just so that we can quickly determine if any failure is a flake or legit issue
(the bazel version of these tests could theoretically run in a slightly different way and fail or not fail in a different way, so having -lcoal job is just an extra safety check).
This change was coauthored with @devversion
PR Close#27937
Switch from Skylint to buildifier --lint - this is required for the Bazel 0.20 upgrade since Bazel no longer lets us use the embedded JDK to build and run Java programs, and Skylint is a Java program
PR Close#27489
With this change, we no longer depend on CircleCI to trigger the webhook
(which it sometimes does with considerable delay or not at all).
This has the added benefit that other jobs will not unnecessarily
trigger webhooks and spam the preview server logs. It is only the
`aio_preview` job's webhook that we care about.
Related to #27352.
PR Close#27458
Currently the Github robot is not able to measure the artifacts of the `hello_world` and `todo` bundling tests because those will be only built with the `ivy-only` rule tag. This means that the current CircleCI job that is supposed to upload the size artifacts, does not even build those bundles.
PR Close#27042
Since 8fc4ae51f, the jobs that need Xvfb use `*-browser` CircleCI docker
image flavors (e.g. `circleci/node:10.12-browsers`), which automatically
start Xvfb.
PR Close#26869
* No longer depends on a custom CircleCI docker image that comes with Bazel pre-installed. Since Bazel is now available through NPM, we should be able to use the version from `@bazel/bazel` in order to enforce a consistent environment on CI and locally.
* This also reduces the amount of packages that need to be published (ngcontainer is removed)
PR Close#26691
By splitting the jobs, if something goes wrong with deploying (e.g. a
network issue), we can re-run just that part instead of having to wait
for all the tests to complete again.
In terms of total duration, the difference should be minimal, because
the two operations (testing and deploying) do not depend on shared
tasks. For example, we need to build again (for the specific target
environment; e.g. stable, next, etc.) before deploying anyway.
PR Close#26746
Some of the `aio`-/`docs`-related jobs rely on the locally built Angular
packages. When these jobs fail, it could mean that there is an issue
with the Angular packages (e.g. an unintentional breaking change).
This commit ensures that the `publish_artifacts` job is not run, unless
those `aio`-/`docs`-related jobs pass.
(The `test_aio_tools` job also uses the locally built Angular packages,
but it does not exercise them in a meaningful way to be worth making it
a prerequisite for `publish_artifacts`.)
PR Close#26722
We are close enough to blacklist a few test targets, rather than whitelist targets to run...
Because bazel rules can be composed of other rules that don't inherit tags automatically,
I had to explicitly mark all of our ts_library and ng_module targes with "ivy-local" and
"ivy-jit" tags so that we can create a query that excludes all fixme- tagged targets even
if those targets are composed of other targets that don't inherit this tag.
This is the updated overview of ivy related bazel tags:
- ivy-only: target that builds or runs only under ivy
- fixme-ivy-jit: target that doesn't yet build or run under ivy with --compile=jit
- fixme-ivy-local: target that doesn't yet build or run under ivy with --compile=local
- no-ivy-jit: target that is not intended to build or run under ivy with --compile=jit
- no-ivy-local: target that is not intended to build or run under ivy with --compile=local
PR Close#26471
The deployment of PR previews is triggered by the notification webhook
of the `aio_preview` CircleCI job (which creates and stores the build
artifacts).
This commit adds a new job (`test_aio_preview`), which waits for the
preview to be deployed (for PRs that do have a preview) and then runs
some tests against it (currently only PWA tests).
Fixes#23818
PR Close#25671
Enables Chrome users to search angular.io and its subdomains from the browsers navigation bar.
Not sure if compatible with Firefox yet.
The queried term in the URL is removed after closing the search-results.
PR Close#25479
Previously, Travis pushed the build artitfacts to the preview server.
This required us to use JWT to secure the POST request from Travis, to
ensure we couldn't receive malicious builds.
JWT has been deprecated and we are moving our builds to CircleCI.
This commit rewrites the TypeScript part of the preview server that
handles converting build artifact into hosted previews of the docs.
Now instead of pushing the AIO build artifacts to the preview server
from inside a Travis job, the artifacts are built and hosted on the
CircleCI infrastructure. The preview server will then pull these
down after being triggered by a CircleCI build webhook.
Two new CircleCI environments are created: test_ivy_jit and test_ivy_aot.
Both run a subset of the tests that have been marked with Bazel tags as
being appropriate for that environment.
Once all the tests pass, builds are published to the *-builds repo both
for the legacy View Engine compiled code as well as for ivy-jit and ivy-aot.
PR Close#24309
Bazel has a restriction that a single output (eg. a compiled version of
//packages/common) can only be produced by a single rule. This precludes
the Angular repo from having multiple rules that build the same code. And
the complexity of having a single rule produce multiple outputs (eg. an
ngc-compiled version of //packages/common and an Ivy-enabled version) is
too high.
Additionally, the Angular repo has lots of existing tests which could be
executed as-is under Ivy. Such testing is very valuable, and it would be
nice to share not only the code, but the dependency graph / build config
as well.
Thus, this change introduces a --define flag 'compile' with three potential
values. When --define=compile=X is set, the entire build system runs in a
particular mode - the behavior of all existing targets is controlled by
the flag. This allows us to reuse our entire build structure for testing
in a variety of different manners. The flag has three possible settings:
* legacy (the default): the traditional View Engine (ngc) build
* local: runs the prototype ngtsc compiler, which does not rely on global
analysis
* jit: runs ngtsc in a mode which executes tsickle, but excludes the
Angular related transforms, which approximates the behavior of plain
tsc. This allows the main packages such as common to be tested with
the JIT compiler.
Additionally, the ivy_ng_module() rule still exists and runs ngc in a mode
where Ivy-compiled output is produced from global analysis information, as
a stopgap while ngtsc is being developed.
PR Close#24056
This uses a new script and CircleCI job called "build-packages-dist"
which shims the new Bazel build to produce outputs matching the legacy
build. We'll use this to get AIO testing onto CircleCI as well.
We move the integration tests to a new circleCI job that depends on this
one, as well as the build publishing job.
Note that every PR will have a trivial green publishing status, because
we always create this job even for PRs. We'd rather not - see
https://discuss.circleci.com/t/workflows-pull-request-filter/14396/4
PR Close#23512
This commit configures a periodic job to be run on CircleCI, performing several
checks against the actual apps deployed to production (https://angular.io) and
staging (https://next.angular.io).
Fixes#21942
PR Close#22483
Also switch our CircleCI commands to just
bazel build //...
bazel test //...
as this is easier to understand.
Note, the reason this commit removes `firebase-tools` is:
1) firebase-tools has an optional dependency on
https://www.npmjs.com/package/@google-cloud/functions-emulator
2) yarn's `--ignore-optional` doesn't work for transitive deps, so
there's no way to yarn install without getting that functions-emulator
package
3) functions-emulator has a transitive dep on `grpc`
4) the version of `grpc` we get has `BUILD` files and no `WORKSPACE`
file so it always breaks `bazel build ...`
It could be solved by any of:
1) remove firebase-tools - this is what I did
2) fix yarn so you can omit optional deps of a transitive dep
3) make functions-emulator depend transitively on a more recent `grpc`
version
4) patch `grpc` after install by doing an `rm` command in our
postinstall or something
In its place we must install protobufjs. This is needed by the
ngc-wrapped test, which needs jasmine as well as bazel's worker mode
dependencies, and therefore cannot simply rely on
node_modules =
"@build_bazel_rules_typescript_tsc_wrapped_deps//:node_modules"
PR Close#22168
When I enabled bazel remote caching, I also switched to running
buildifier and skylint from the package.json script, which builds them
from head. With remote caching, we do get cache hits for these, but
looking up the action inputs actually takes quite a bit of time since we
have to first fetch the remote repository, then do loading and
analysis, then read the inputs to determine the cache key.
It's more important to keep the lint job fast, so I'm reverting that
part of the change for now. We can experiment with building them from
head in a less critical repo.
PR Close#22526
Note, the reason this commit removes `firebase-tools` is:
1) firebase-tools has an optional dependency on
https://www.npmjs.com/package/@google-cloud/functions-emulator
2) yarn's `--ignore-optional` doesn't work for transitive deps, so
there's no way to yarn install without getting that functions-emulator
package
3) functions-emulator has a transitive dep on `grpc`
4) the version of `grpc` we get has `BUILD` files and no `WORKSPACE`
file so it always breaks `bazel build ...`
It could be solved by any of:
1) remove firebase-tools - this is what I did
2) fix yarn so you can omit optional deps of a transitive dep
3) make functions-emulator depend transitively on a more recent `grpc`
version
4) patch `grpc` after install by doing an `rm` command in our
postinstall or something
In its place we must install protobufjs. This is needed by the
ngc-wrapped test, which needs jasmine as well as bazel's worker mode
dependencies, and therefore cannot simply rely on
node_modules =
"@build_bazel_rules_typescript_tsc_wrapped_deps//:node_modules"
PR Close#22168
This should cause Bazel builds to be incremental, only re-building parts of Angular affected by changes since the last build.
It also fixes a potential version skew, where CI was running the Bazel linter binaries in the ngcontainer docker image, but developers built them using the versions in WORKSPACE
PR Close#21784
This is a customization of the rollup_bundle rule from rules_nodejs
which adds the build-optimizer as a plugin.
Add a functional test with fast round-trip that asserts the minified app
still works.
Publish the min.js artifact on circleCI so we can track its size.
PR Close#22004
Add enough BUILD files to make it possible to
`bazel build packages/core/test`
Also re-format BUILD.bazel files with Buildifier.
Add a CI lint check that they stay formatted.
PR Close#20768
In previous version of tsickle abstract class methods were materialized.
The change resulted in 6Kb savings in angular.io bundle.
This change also required the removal of `@private` and `@return` type
annotation as it is explicitly dissalowed by tsickle.
NOTE: removed casts in front of `makeDecorator` due to:
https://github.com/angular/devkit/issues/45
```
14938 Jul 19 13:16 0.b19e913fbdd6507d346b.chunk.js
1535 Jul 19 13:16 inline.d8e019ea3cfdd86c2bd0.bundle.js
589178 Jul 19 13:16 main.54c97bcb6f254776b678.bundle.js
34333 Jul 19 13:16 polyfills.4a3c9ca9481d53803157.bundle.js
14938 Jul 18 16:55 0.b19e913fbdd6507d346b.chunk.js
1535 Jul 18 16:55 inline.0c83abb44fad9a2768a7.bundle.js
582786 Jul 18 16:55 main.ea290db71b051813e156.bundle.js
34333 Jul 18 16:55 polyfills.4a3c9ca9481d53803157.bundle.js
main savings: 589178 - 582786 = 6,392
```
PR Close#18236
We expect this behavior because it's what Travis does. Also it's better because we want
to test what happens if we merge the PR, not the status of the PR branch.
Paul Gschwendtner
Filipe Silva
George Kalpakas
Greg Magolan
Alex Eagle
Brandon
Matias Niemelä
Jeremy Elbourn
Igor Minar
Alex Eagle
Misko Hevery
Alex Eagle
Pete Bacon Darwin
Sergej
Wei Huang
Kara Erickson
Alex Rickabaugh
Fabian Wiles
Jason Aden
Tobias Bosch
Olivier Combe