Commit Graph

586 Commits

Author SHA1 Message Date
George Kalpakas 82a6fc5ef9 build(docs-infra): use local version of Zone.js when testing against local packages (#35858)
In some cases, we want to test the AIO app or docs examples against the
locally built Angular packages (for example to ensure that the changes
in a commit do not introduce a breaking change). In order to achieve
this, we have the `ng-packages-installer` script that handles updating
a project's `package.json` file to use the locally built Angular
packages (and appropriate versions for their (dev-/peer-)dependencies).

Previously, `ng-packages-installer` would only consider the locally
built Angular packages (from `dist/packages-dist/`). However, given that
Zone.js is now part of the `angular/angular` repo, it makes sense to
also use the locally built Zone.js package (from `dist/zone.js-dist/`).
Otherwise, the tests might fail for commits that update both the Angular
packages (and related docs examples) and the Zone.js package. An example
of such a simultaneous change (that would have broken tests) is #33838.

This commit updates the script to install the locally built Zone.js
package (in addition to the Angular ones). The commit ensures that the
Zone.js package will always be available alongside the Angular packages
(i.e. that the Zone.js package will be built by the same script that
builds the Angular packages and that the `dist/zone.js-dist/` directory
will be cached on CI).

Note: This problem was discovered while enabling docs examples unit
tests in #34374.

PR Close #35858
2020-03-06 17:30:20 -05:00
Joey Perrott b42e2e0dff build: create dev-infra-private npm package (#35862)
Creates the scaffolding for an @angular/dev-infra-private package
which will not be published to npm but will be pushed to
https://github.com/angular/dev-infra-private-builds repo for each
commit to master.

The contents of this npm package will then be depended on via
package.json dependency for angular/angular angular/angular-cli and
angular/components.

PR Close #35862
2020-03-05 18:55:40 -05:00
Andrew Scott 1b8a66893e Revert "build(docs-infra): use local version of Zone.js when testing against local packages (#35780)" (#35857)
This reverts commit 7d832ae1001b6264bb7124086089e9e69c10c9b6; breaks CI
with error `Concurrent upstream jobs persisted the same file(s) into the workspace:`

PR Close #35857
2020-03-04 10:57:36 -08:00
George Kalpakas 7d832ae100 build(docs-infra): use local version of Zone.js when testing against local packages (#35780)
In some cases, we want to test the AIO app or docs examples against the
locally built Angular packages (for example to ensure that the changes
in a commit do not introduce a breaking change). In order to achieve
this, we have the `ng-packages-installer` script that handles updating
a project's `package.json` file to use the locally built Angular
packages (and appropriate versions for their (dev-/peer-)dependencies).

Previously, `ng-packages-installer` would only consider the locally
built Angular packages (from `dist/packages-dist/`). However, given that
Zone.js is now part of the `angular/angular` repo, it makes sense to
also use the locally built Zone.js package (from `dist/zone.js-dist/`).
Otherwise, the tests might fail for commits that update both the Angular
packages (and related docs examples) and the Zone.js package. An example
of such a simultaneous change (that would have broken tests) is #33838.

This commit updates the script to install the locally built Zone.js
package (in addition to the Angular ones). The commit ensures that the
Zone.js package will always be available alongside the Angular packages
(i.e. that the Zone.js package will be built by the same script that
builds the Angular packages and that the `dist/zone.js-dist/` directory
will be cached on CI).

Note: This problem was discovered while enabling docs examples unit
tests in #34374.

PR Close #35780
2020-03-04 08:35:26 -08:00
George Kalpakas 3f88de9407 build: move build scripts to dedicated directory (#35780)
This commit moves the build-related scripts
(`build-ivy-npm-packages.js`, `build-packages-dist.js` and
`package-builder.js`) to a dedicated directory to keep the `scripts/`
directory cleaner.

It also moves the logic for building the `zone.js` package to a separate
script, `zone-js-builder.js`, to make it re-usable. A subsequent commit
will use it to build the `zone.js` package when building the Ivy Angular
packages as well.

PR Close #35780
2020-03-04 08:35:26 -08:00
Paul Gschwendtner 5615928df9 build: no longer run tslint from within gulp task (#35800)
Switches our tslint setup to the standard `tslint.json` linter excludes.
The set of files that need to be linted is specified through a Yarn script.

For IDEs, open files are linted with the closest tslint configuration, if the
tslint IDE extension is set up, and the source file is not excluded.

We cannot use the language service plugin for tslint as we have multiple nested
tsconfig files, and we don't want to add the plugin to each tsconfig. We
could reduce that bloat by just extending from a top-level tsconfig that
defines the language service plugin, but unfortunately the tslint plugin does
not allow the use of tslint configs which are not part of the tsconfig project.

This is problematic since the tslint configuration is at the project root, and we
don't want to copy tslint configurations next to each tsconfig file.

Additionally, linting of `d.ts` files has been re-enabled. This has been
disabled in the past and a TODO has been left. This commit fixes the
lint issues and re-enables linting.

PR Close #35800
2020-03-03 09:20:49 -08:00
George Kalpakas ab8199f7c9 build: several minor fixes related to using `puppeteer` (#35381)
This is a follow-up to #35049 with a few minor fixes related to using
the browser provided by `puppeteer` to run tests. Included fixes:

- Make the `webdriver-manager-update.js` really portable. (Previously,
  it needed to be run from the directory that contained the
  `node_modules/` directory. Now, it can be executed from a subdirectory
  and will correctly resolve dependencies.)

- Use the `puppeteer`-based setup in AIO unit and e2e tests to ensure
  that the downloaded ChromeDriver version matches the browser version
  used in tests.

- Use the `puppeteer`-based setup in the `aio_monitoring_stable` CI job
  (as happens with `aio_monitoring_next`).

- Use the [recommended way][1] of getting the browser port when using
  `puppeteer` with `lighthouse` and avoid hard-coding the remote
  debugging port (to be able to handle multiple instances running
  concurrently).

[1]: https://github.com/GoogleChrome/lighthouse/blame/51df179a0/docs/puppeteer.md#L49

PR Close #35381
2020-02-18 12:42:47 -08:00
Greg Magolan acfd0edd38 test: use puppeteer in integration tests and to download correct chromedriver (#35049)
This means integration tests no longer need to depend on a $CI_CHROMEDRIVER_VERSION_ARG environment variable to specify which chromedriver version to download to match the locally installed chrome. This was bad DX and not having it specified was not reliable as webdriver-manager would not always download the chromedriver version to work with the locally installed chrome.

webdriver-manager update --gecko=false --standalone=false $CI_CHROMEDRIVER_VERSION_ARG is now replaced with node webdriver-manager-update.js in the root package.json, which checks which version of chrome puppeteer has come bundled with & downloads informs webdriver-manager to download the corresponding chrome driver version.

Integration tests now use "webdriver-manager": "file:../../node_modules/webdriver-manager" so they don't have to waste time calling webdriver-manager update in postinstall

"// resolutions": "Ensure a single version of webdriver-manager which comes from root node_modules that has already run webdriver-manager update",
"resolutions": {
"**/webdriver-manager": "file:../../node_modules/webdriver-manager"
}
This should speed up each integration postinstall by a few seconds.

Further, integration test package.json files link puppeteer via file:../../node_modules/puppeteer which is the ideal situation as the puppeteer post-install won't download chrome if it is already downloaded. In CI, since node_modules is cached it should not need to download Chrome either unless the node_modules cache is busted.

NB: each version of puppeteer comes bundles with a specific version of chrome. Root package.json & yarn.lock currently pull down puppeteer 2.1.0 which comes with chrome 80. See https://github.com/puppeteer/puppeteer#q-which-chromium-version-does-puppeteer-use for more info.

Only two references to CI_CHROMEDRIVER_VERSION_ARG left in integration tests at integration/bazel-schematics/test.sh which I'm not entirely sure how to get rid of it

Use a lightweight puppeteer=>chrome version mapping instead of launching chrome and calling browser.version()

Launching puppeteer headless chrome and calling browser.version() was a heavy-handed approach to determine the Chrome version. A small and easy to update mappings file is a better solution and it means that the `yarn install` step does not require chrome shared libs available on the system for its postinstall step

PR Close #35049
2020-02-11 13:16:52 -08:00
Paul Gschwendtner c4ad12c1c3 ci: remove components-repo-ci blocklist (#35115)
Previously we needed the `components-repo-ci` blocklist to disable
tests that were failing during the development of Ivy. Since we fixed
all those failing tests, and we don't want to regress, we can remove the
blocklist logic.

Resolves FW-1807

PR Close #35115
2020-02-06 15:32:33 -08:00
Igor Minar 722251da40 build: update scripts/release/post-check with the latest versions (#35151)
these files are used as a post-release smoke tests and haven't been updated to reflect recent depedency changes
and package additions.

PR Close #35151
2020-02-06 09:37:26 -08:00
George Kalpakas 519e9e1ae8 ci: minor PullApprove fixes/improvements (#35015)
This is a follow-up to #34814 to fix some typos in patterns and make
them more similar to the old patterns from `.github/CODEOWNERS`.

PR Close #35015
2020-01-29 12:23:47 -08:00
Greg Magolan dff4e1e19c refactor: simplify bazel saucelabs targets using karma pre-test wrapper and shared saucelabs connection between tests (#34769)
* Added a /tools/saucelabs/sauce-service.sh script that manages the sauce-connect as a service which is used by the karma-saucelabs.js wrapper to start the service.
* Added /tools/saucelabs/README.md that covers the details of SauceLabs karma testing with Bazel.

PR Close #34769
2020-01-28 13:47:00 -08:00
Paul Gschwendtner fa4ea34401 ci: rename `material-unit-tests` job to `components-repo-unit-tests` (#34898)
We rename the `material-unit-tests` job to `components-repo-unit-tests`
because the job runs all unit tests found in the Angular Components repository.
This includes the Angular CDK, Angular Material and more. Also the repository has
been renamed from `angular/material2` to `angular/components` in the past.

PR Close #34898
2020-01-23 13:29:02 -08:00
Greg Magolan 45ae62787a build: package-build & release scripts should handle new pkg_npm naming (#34736)
bazel query now using: kind("ng_package|pkg_npm", //packages/...)

PR Close #34736
2020-01-15 14:58:07 -05:00
Paul Gschwendtner c9f24e6a58 ci: overwrite angular/components yarn typescript resolution (#33717)
The components repository has a Yarn resolution to ensure that
dgeni-packages uses a specific TypeScript version. This resolution
causes the specified TS version to be considered as candidate for the
`@angular/bazel` peer dependency. Ultimately, Yarn decides to use
the TypeScript version from the resolution for `@angular/bazel`,
and builds will fail due to a version mismatch.

This is because `tsickle` will use the hoisted top-level TS
version (set to `3.7.4` ), while `@angular/bazel` uses the
version from the resolution (at the time of writing: v3.6.4)

PR Close #33717
2020-01-14 16:42:22 -08:00
atscott 7e9ac7650b Revert "build: package-build & release scripts should handle new pkg_npm naming (#34589)" (#34730)
This reverts commit 053b23d902.

PR Close #34730
2020-01-10 14:12:15 -08:00
Greg Magolan 053b23d902 build: package-build & release scripts should handle new pkg_npm naming (#34589)
bazel query now using: kind("ng_package|pkg_npm", //packages/...)

PR Close #34589
2020-01-10 08:31:59 -08:00
Paul Gschwendtner 6d3a25d897 ci: run acceptance tests on saucelabs with ivy (#34277)
Currently we only run Saucelabs on PRs using the legacy View Engine
build. Switching that build to Ivy is not trivial and there are various
options:

  1. Updating the R3 switches to use POST_R3 by default. At first glance,
  this doesn't look easy because the current ngtsc switch logic seems to
  be unidirectional (only PRE_R3 to POST_R3).

  2. Updating the legacy setup to run with Ivy. This sounds like the easiest
  solution at first.. but it turns out to be way more complicated. Packages
  would need to be built with ngtsc using legacy tools (i.e. first building
  the compiler-cli; and then building packages) and View Engine only tests
  would need to be determined and filtered out. Basically it will result in
  re-auditing all test targets. This is contradictory to the fact that we have
  this information in Bazel already.

  3. Creating a new job that runs tests on Saucelabs with Bazel. We specify
  fine-grained test targets that should run. This would be a good start
  (e.g. acceptance tests) and also would mean that we do not continue maintaining
  the legacy setup..

This commit implements the third option as it allows us to move forward
with the general Bazel migration. We don't want to spend too much time
on our legacy setup since it will be removed anyway in the future.

PR Close #34277
2019-12-16 07:43:41 -08:00
Joey Perrott 89ef77f750 ci: update sha of components repo for components unit test integrations (#34280)
Before updating to remove the compile build variable, we must update
the components unit test integrations to a sha  in the components
repo which no longer relies on the compile build variable.

PR Close #34280
2019-12-06 11:04:20 -08:00
Joey Perrott 4022376b3f build: update scripting and tooling to rely on define=angular_ivy_enabled instream of define=compile (#33983)
We need to migrate to using angular_ivy_enabled value to determine whether to use
Ivy or ViewEngine for package building scripts and for size-tracking and
symbol-extract tooling.

PR Close #33983
2019-11-26 16:38:40 -05:00
George Kalpakas 9b5299131b ci: remove change type from uploaded payload size data (#33987)
The change type was only recorded for `aio/` and was not correct anyway.
For example:
- It considered `package.json` changes as `application` (even if only
  `package.json` and `yarn.lock` had changed).
- It failed to account for changes in `@angular/*` dependencies, when
  using the locally built Angular packages (instead reporting them as
  `other`).
- It only looked at the last commit, so it failed to provide accurate
  information for multi-commit builds (which are rare, but possible).

For the above reasons (and because there is no straight-forward way of
fixing it), this commit removes the change type from the uploaded data.
If necessary, it is still possible to find the type of changes from the
uploaded info (e.g. extract the associated commits and look at their
changes using git).

PR Close #33987
2019-11-25 16:36:07 -05:00
George Kalpakas 9bc8864d03 ci: always track and check payload size data for angular.io (#33987)
As part of the `payload-size` npm script in `aio/package.json` (which is
run on CI), the sizes of the angular.io app bundles are checked to
ensure they do not exceed certain limits and are also uploaded to
Firebase to be available for later analysis. The uploaded data include
the type of the changes (dependencies only, application only, or both).
The type of changes is inferred by looking at the files that have
changed inside the `aio/` directory.

When the `payload-size.sh` script was first introduced, the only files
that could affect bundle sizes were inside the `aio/` directory.
Therefore, the script would skip uploading the data and checking the
sizes if no changes were detected inside the `aio/` directory.

However, this assumption stopped being valid over time. For example:
- We started tracking/checking bundle sizes when building the angular.io
  app with the locally built Angular packages (which live outside the
  `aio/` directory.
- Due to CircleCI limitations, the `CI_COMMIT_RANGE` environment
  variable (which is used for determining what files have been affected)
  stopped reflecting the whole commit range of the build and only
  included the last commit instead.

Based on the above, there were many cases were size data would not be
uploaded to Firebase, even when they may have been affected (because the
affecting changes were outside `aio/` - e.g. in framework packages).
This makes it harder to analyze size regressions, because important
data-points are missing.

Even worse, in these cases, the sizes were not even checked against the
specified limits, thus making it possible for size regressions to go
unnoticed (unless caught by other similar tests).

This commit fixes the `scripts/ci/payload-size.sh` script to always
track and check payload sizes for angular.io bundles.

NOTE: This change will result in more data being recorded (i.e.
      recording data when it is not possible for the bundle sizes to
      have been affected by the changes). This is still preferable to
      failing to record and/or check when sizes could have been
      affected.

PR Close #33987
2019-11-25 16:36:07 -05:00
Kara Erickson d752e26eb2 ci: tighten size threshold to 1% or 500 bytes (#33969)
The size diff threshold of 1% has proven to be too lenient for us
to catch size regressions in AIO. Since the AIO main bundle is
between 400-500 KB, a size regression must be between 4-5 KB before
it will cause the tests to fail. As a result, we may merge many
changes with smaller regressions of a few KB before the size test
eventually lets us know that the number has increased. The hope is
that lowering the threshold will help us catch the smaller
regressions during code review and prevent the size tests failing at
a random later time when someone catches the size "hot potato".

PR Close #33969
2019-11-22 16:51:41 -05:00
George Kalpakas 485ce640fd ci: use more descriptive names for published build artifacts (#33957)
In efbbae5a4, the `publish_packages_as_artifacts` CircleCI job was
created to publish the build artifacts of PR builds on CI. In a8f4f14bd,
its scope was expanded to also publish build artifacts on non-PR builds.

The published artifacts names are constructed based on the PR number
(e.g. include `-pr12345-`), so on non-PR builds the names do not reflect
the source branch (instead, they include `-prfalse-`).

This commit fixes this by using the current branch name. For example,
artifact names for the `master` or `9.0.x` branch will include
`-master-` and `-9.0.x-` respectively (instead of `-prfalse-`).
(NOTE: For PRs, where branch name is `pull/12345`, the branch name is
transformed to `pr12345`.)

PR Close #33957
2019-11-22 14:05:52 -05:00
Joey Perrott 346e1c14d9 build: update setup-rbe.sh script and documentation to remove http-remote-cache config (#33907)
We have determined that the upload time cost of remote caching via http is too costly
to be effective for our distributed usage scenario.  However this cost is still worth
it for us for Windows CI runs on a full cache hit move from a ~60 minute task to ~10
minutes.

It is worth noting that this remote http-caching is entirely separate from RBE caching
mechanisms.

PR Close #33907
2019-11-21 10:45:57 -08:00
Pete Bacon Darwin be58159550 build: ensure package-builder finds bazel (#33904)
Since `scripts/package-builder.js` gets run from the
`aio` folder sometimes, it is important to set the cwd
for the command that finds the bazel bin directory.

PR Close #33904
2019-11-19 11:40:19 -08:00
George Kalpakas e2cba6ed7d build: replace `fs` methods with ShellJS' `test('-d')` (#33895)
This makes the JS script more similar to the original bash script it
replaced (see #33854) and will be easier to follow for people that are
less familiar with Node.js scripting.

Discussed in:
https://github.com/angular/angular/pull/33854#discussion_r346949765

PR Close #33895
2019-11-18 09:10:16 -08:00
George Kalpakas 7eb3e3bce6 build: fix build scripts on macOS (#33854)
In #33823, `scripts/package-builds.sh` (which is used by both
`build-packages-dist.sh` and `build-ivy-npm-packages.sh`) was updated to
use `realpath`. It turns out that `realpath` does not exist on macOS, so
the build scripts do not work there.

In order to fix this (and also reduce the likelihood of introducing
similar issues in the future), this commit changes these bash scripts to
Node.js scripts (using [ShellJS](https://github.com/shelljs/shelljs) for
a cross-platform implementation of Unix shell commands where necessary).

PR Close #33854
2019-11-15 16:05:00 -08:00
George Kalpakas 695bb6ca87 build: fix build scripts on macOS (#33843)
In #33823, `scripts/package-builds.sh` (which is used by both
`build-packages-dist.sh` and `build-ivy-npm-packages.sh`) was updated to
use `realpath`. It turns out that `realpath` does not exist on macOS, so
the build scripts do not work there.

This commit fixes it by switching from `realpath $(pwd)/..` to
`dirname $(pwd)`, which does practically the same thing (in this
situation) and works on macOS as well.

PR Close #33843
2019-11-15 10:36:54 -08:00
George Kalpakas e25b256aa4 build: minor improvements to build script output (#33823)
- Move "Building packages" log above actual work (computing targets).
- Print corrent script name (`build-ivy-npm-packages.sh` would claim to
  be `build-packages-dist.sh`).
- Make logs of different operations a little more consistent.

PR Close #33823
2019-11-14 09:29:49 -08:00
George Kalpakas 2ce9cdab9d ci: publish tarballs for the zone.js package as CI build artifacts (#33733)
Since #33321, Angular packages have been persisted on each build as
CircleCI build artifacts (`.tgz` files), which can be used to install
dependencies on a project (for the purpose of testing or trying out a
change before a PR being merged and without having to build the packages
from source locally).

Previously, only packages published to npm under the `@angular` scope
were persisted as build artifacts.

This commit adds the `zone.js` package to the list of persisted
packages.

Fixes #33686

PR Close #33733
2019-11-12 09:55:17 -08:00
George Kalpakas 74b7d1ace1 test(ngcc): build `zone.js` from source in `scripts/build-packages-dist.sh` (#33733)
In #33046, internal uses of `zone.js` were switched to reference it
directly from source (built with Bazel) instead of npm. As a result, the
necessary scripts were updated to build `zone.js` as necessary. However,
some `integration/**/debug-test.sh` scripts were missed (apparently
because they are not used on CI, but only locally as helpers for
debugging the integration projects).

This commit updates the `scripts/build-packages-dist.sh` script to also
build `zone.js`, so that other scripts (such as the various
`debug-test.sh` scripts) can use it.

PR Close #33733
2019-11-12 09:55:16 -08:00
Joey Perrott bf10e98891 build: delete jenkins scripts (#33598)
PR Close #33598
2019-11-05 20:55:18 +00:00
Paul Gschwendtner 31116f0ced ci: re-enable material unit tests job (#33530)
The Material units tests job has been disabled with the
typescript upgrade PR since the components repository was
still on an old TypeScript version (due to cyclic dependencies
between the framework and components repository).

Since the components repository has been updated to `9.0.0-next.15`
and now uses the compatible TypeScript version, we can re-enable
ds

Related to this change, we need to ignore the `package.json`
engines when installing the dependencies of the components repo. This
is because the components repo already updated to NodeJS v12 and Yarn
v1.19.1. This is not the case for the CI setup of framework. For now,
since we don't want to change the dev setup (as it slows down
development), we temporarily disable the engines.

Additionally, the material unit tests job now depends on the actual
release packages (not on the ngtsc compiled ones). This is because
the components repo setup relies on NGCC being run. This is also
helpful for validating ngcc against the framework packages.

PR Close #33530
2019-11-01 17:38:39 +00:00
Joey Perrott c4695bb387 build: ensure that version stamping is applied when the npm_package publish action is called (#33512)
PR Close #33512
2019-10-31 10:21:41 -07:00
George Kalpakas efbbae5a48 ci: publish tarballs for all Angular packages as build artifacts on PR builds (#33321)
Previously, when one wanted to try out the changes from a PR before it
was merged, they had to check out the PR locally and build the Angular
packages themselves (which is time-consuming and wasteful given that the
packages have already been built on CI).

This commit persists all Angular packages on each build as `.tgz` files,
which can be used to install dependencies on an project (supported by
both [npm][1] and [yarn][2]). In addition to individual `.tgz` files for
each package, a `.tgz` file including all packages is also stored, which
can be used to test the packages locally by overwriting the ones in the
`node_modules/` directory of a project.

CircleCI [build artifacts][3] an be used for longer-term storage of the
outputs of a build and are designed to be useful around the time of the
build, which suits our needs.

[1]: https://docs.npmjs.com/cli/install.html
[2]: https://yarnpkg.com/lang/en/docs/cli/add
[3]: https://circleci.com/docs/2.0/artifacts

PR Close #33321
2019-10-24 09:56:21 -07:00
Joey Perrott 418e9cf3c4 build: address comments, restructure setup-rbe.sh script (#33109)
PR Close #33109
2019-10-21 15:56:52 -04:00
Joey Perrott 37a87418b6 build: migrate to using google_default_credentials and update setup-rbe script (#33109)
PR Close #33109
2019-10-21 15:56:51 -04:00
George Kalpakas b6dfc8b08c ci: log calculated file sizes for each build (#33099)
The `payload-size.sh` script is mainly used on CI to calculate, check
and potentially save (on non-PR builds) the sizes of the bundles for
various apps (including angular.io). If everything goes well (i.e. the
checks pass, meaning that the sizes did not increase above the specified
threshold) nothing is shown in the CI logs.

In some cases, it is useful to be able to see what the sizes were in a
specific build; e.g. for debugging purposes or when investigating a
gradual increase that happened over time. (Some of this info is
available on https://size.angular.io/, but not all.)

Previously, the only way to find out what the sizes were for a specific
build was to checkout the corresponding commit locally and build the
target app, which in turn requires building all Angular packages and can
take some time. Given that the sizes are already calculated on CI, this
was a waste.

This commit makes it easy to find out the bundle sizes for a specific
build/commit by always printing out the calculated sizes (thus making
them show up in the CI logs).

PR Close #33099
2019-10-15 16:48:29 +00:00
Greg Magolan dcbc3b197d ci: update material-unit-test commit (#33073)
Updates the `material-unit-test` job to run tests against
the latest commit of the Angular Components repository.

The components repository updated to rules_nodejs#0.38.2 before Angular Bazel did.
To do this, the `@angular/bazel` v0.38.2 compatibility changes were patched on postinstall.

This now conflicts because we install a `@angular/bazel` version in the `material-unit-tests` job
that already includes these compatibility changes. This would result in the patch being a noop
for which the `patch` command throws an error.

We can remove this once components can install a released version of `@angular/bazel` that is
compatible with `rules_nodejs#0.38.2`.

PR Close #33073
2019-10-14 20:25:57 +00:00
Joey Perrott 5ede5b7807 ci: run bazel for package-builder via yarn rather than binary directly (#32983)
PR Close #32983
2019-10-10 13:57:54 -07:00
Paul Gschwendtner 78ada980a6 ci: avoid postinstall error when building material-unit-tests job (#32889)
The components repository updated to rules_nodejs#0.38.0 before Angular Bazel did.
To do this, the `@angular/bazel` v0.38.0 compatibility changes were patched on postinstall.

This now conflicts because we install a `@angular/bazel` version in the `material-unit-tests` job
that already includes these compatibility changes. This would result in the patch being a noop
for which the `patch` command throws an error.

We can remove this once components can install a released version of `@angular/bazel` that is
compatible with `rules_nodejs#0.38.0`.

PR Close #32889
2019-10-08 09:27:11 -07:00
Joey Perrott deaac32bbd build: create script for setting up RBE in local dev environment (#31200)
PR Close #31200
2019-10-03 12:17:56 -07:00
Paul Gschwendtner 7059f7af11 ci: exclude dev-app subpackage targets in material-unit-tests job (#32485)
Ensures that the `material-unit-tests` job does not accidentally build
subpackages of the `src/dev-app` in the components repo. This is now
an issue because the components repository updated their dev-app
to use Bazel with more individual subpackages. These subpackages
are not excluded by the `--deleted_packages` flag.

For best practice, we should use the exclusion target pattern
instead of the undocumented `--deleted_packages` flag anyway.

PR Close #32485
2019-09-10 15:19:31 -04:00
George Kalpakas b5eda603a2 ci: work around `CIRCLE_COMPARE_URL` not being available wih CircleCI Pipelines (#32537)
The commit range that is associated with a CI build is used for a couple
of things (mostly related to payload-size tracking):
- Determine whether a size change was caused by application code or
  dependencies (or both).
- Add the messages of the commits associated with the build (and thus
  the payload-size change).

NOTE: The commit range is only used on push builds.

Previously, the commit range was computed based on the
`CIRCLE_COMPARE_URL` environment variable. With [CircleCI Pipelines][1]
enabled, `CIRCLE_COMPARE_URL` is no longer available and the commit
range cannot be reliably detected.

This commit switches `CI_COMMIT_RANGE` to only include the last commit.
This can be less accurate in some rare cases, but is true in the
majority of cases (on push builds). Additionally, it stores the CircleCI
build URL in the database along with the payload data, so the relevant
info can be retrieved when needed.

[1]: https://circleci.com/docs/2.0/build-processing

PR Close #32537
2019-09-09 12:21:44 -04:00
Pete Bacon Darwin fa79f51645 refactor(ivy): update the compiler to emit `$localize` tags (#31609)
This commit changes the Angular compiler (ivy-only) to generate `$localize`
tagged strings for component templates that use `i18n` attributes.

BREAKING CHANGE

Since `$localize` is a global function, it must be included in any applications
that use i18n. This is achieved by importing the `@angular/localize` package
into an appropriate bundle, where it will be executed before the renderer
needs to call `$localize`. For CLI based projects, this is best done in
the `polyfills.ts` file.

```ts
import '@angular/localize';
```

For non-CLI applications this could be added as a script to the index.html
file or another suitable script file.

PR Close #31609
2019-08-30 12:53:26 -07:00
George Kalpakas ab30b2ad01 build: mention where to find the invalid commit message, when validation fails (#32420)
Whenever someone tries to commit (by running `git commit` directly or
indirectly), a `commit-msg` git hook is run to validate the commit
message. If the validation fails, an error message is printed and the
commit is aborted.

Occasionally, people may have written a non-trivial commit message which
could turn out to be invalid (due a small typo for example). In that
case, it is frustrating to "lose" the whole message and have to write it
all over again (from memory). This is frustrating and has happened to me
enough times to finally seek a solution.

Fortunately, it turns out that git stores the last commit message in
`.git/COMMIT_EDITMSG`, so it is easy to get it back (as long as you know
where to look for it). This commit mentions this info in the validation
error to help people that might not know about it.

(This issue is probably mostly relevant for people using git from the
command-line and not through a UI, but it won't hurt in either case.)

PR Close #32420
2019-08-30 13:52:54 -04:00
Keen Yee Liau 2b28b91fd9 ci: Add ANGULAR_PACKAGE to snapshot builds (#31900)
Ivy snapshots no longer contain metadata.json which breaks the detection logic for
Angular Package Format (APF) in npm/yarn install.
This PR adds ANGULAR_PACAKGE as a magic file to denote APF.
See https://github.com/bazelbuild/rules_nodejs/issues/927

PR Close #31900
2019-08-28 21:25:56 -07:00
Paul Gschwendtner 62f4140634 build: add blocklist for material unit tests (#32239)
Initially the blocklist has been removed because there were
no remaining disabled tests that failed. Also the blocklist
logic didn't work anymore because the `material-unit-tests` CI
job now runs against `angular/components#master` with Bazel.

388578fec9 tried to revert the removal
of the blocklist in favor of a new upcoming breaking change with
HammerJS, but the revert doesn't help since the blocklist still
doesn't work with Bazel.

In order to make the blocklist work with the unit tests running
with Bazel, a PR has been submitted on the components repository.
See: https://github.com/angular/components/pull/16833.

This commit updates the blocklist logic on the framework side to
work with the new logic on the components repo side.

PR Close #32239
2019-08-21 10:03:01 -07:00
Kara Erickson 388578fec9 Revert "ci: remove material-unit-tests failure blocklist (#32138)" (#32226)
This reverts commit 0660903784 so we
can add some tests to the blocklist.

PR Close #32226
2019-08-20 16:39:54 -07:00
Joey Perrott 43163523f6 ci: move bazel saucelabs execution to script to be used across all Angular repos (#32141)
PR Close #32141
2019-08-16 09:57:23 -07:00
Paul Gschwendtner 0660903784 ci: remove material-unit-tests failure blocklist (#32138)
Initially when the `material-unit-tests` job got wired up,
Ivy was not really backwards-compatible and a few bugs caused
test failures when running the Angular Material test suites w/ Ivy.

These bugs got fixed progressively and eventually the test
blocklist became empty. At this point we don't want to regress
in the future and the blocklist should never have new items.

Additionally since we switched the unit-tests job to run against
Angular Material `master` with Bazel, the blocklist is no
longer respected. Therefore we can safely remove the blocklist.

PR Close #32138
2019-08-14 12:02:12 -07:00
Paul Gschwendtner 0cd4c019cf build: lock material unit tests job to specific commit (#31569)
No longer locks the Material unit tests job to a specific branch, but rather allows
locking to a specific commit from a given branch. This allows us to use the
"master" branch from the `components` repository.

PR Close #31569
2019-07-25 13:08:33 -07:00
JoostK 7e46a6d99d test(ivy): update Material to recent commit from master branch (#31569)
Previously, the ivy-2019 branch of the Material (aka components) repo was
used, which contains some changes that were necessary to work with Ivy.
These changes are not longer necessary, as Material's master branch is
fully working with Ivy today. To be up-to-date with recent Material
development and its support for more recent dependencies, e.g. TypeScript,
it is desirable for us to be on a newer version of Material.

This commit moves the Material tests away from the ivy-2019 branch, to a
recent commit on master. We are not targeting the master branch itself,
as that would introduce a moving target into Angular's CI checks, which
is undesirable.

Lastly, the usage of gulp to run Material's tests is changed into using
Bazel, as Material itself is now also built with Bazel.

PR Close #31569
2019-07-25 13:08:33 -07:00
George Kalpakas 4c45aa39e4 ci: re-enable payload size tracking (#31138)
Payload size tracking was temporarily disabled in #31057, due to
`CIRCLE_COMPARE_URL` stopping being available. It turned out this was
related to turning on the new [Pipelines][1] feature, which was required
for testing Windows on CircleCI.

Since then, we have turned `Pipelines` off and got `CIRCLE_COMPARE_URL`
back (e.g. see [build 362971][2]). According to CircleCI, failing to
populate `CIRCLE_COMPARE_URL` with `Pipelines` on is a bug and they are
working on fixing it.

[1]: https://circleci.com/docs/2.0/build-processing/
[2]: https://circleci.com/gh/angular/angular/362971

Fixes #31121

PR Close #31138
2019-06-19 15:55:10 -07:00
George Kalpakas 48d11d5fa0 ci: temporarily disable payload size tracking (#31057)
The `CIRCLE_COMPARE_URL` is not available in builds any more since we
enabled `Pipelines` on CircleCI. We have contected CircleCI, but until
this is solved we cannot get the commit range and thus disabling
uploading of payload size data to avoid broken builds.

PR Close #31057
2019-06-14 09:19:17 -07:00
Paul Gschwendtner 8852b793df build: unable to run build-packages-dist script on windows (#30853)
Currently it's not possible to run the `./scripts/build-packages-dist.sh` script on Windows because
`bazel query` returns CRLF line-endings which result on array expansion in Bazel target names
that end with a carriage return (e.g. `//packages/core\r`). This then results in a build failure where
Bazel complains that target names should not end with a carriage return.

In order to fix this, we just strip off the carriage return line-endings from the bazel query stdout. Ideally
the script will be ported to a plain Node script eventually, but for now it prevents Windows users from
building the release packages and the simple workaround seems reasonable and sufficient.

PR Close #30853
2019-06-05 09:05:28 -07:00
Jason Aden 87a2366eeb build: update base SHA for rebase on merge-pr script 2019-05-21 17:07:19 -07:00
Ben Lesh a181e8e7d8 refactor: improve merge script formatting (#29991)
PR Close #29991
2019-04-19 14:33:30 -07:00
Pete Bacon Darwin 9f54d76ef5 refactor(upgrade): use Bazel packages to avoid symlinks in the source (#29466)
Previously we had to share code between upgrade/dynamic and upgrade/static
by symlinking the `src` folder, which allowed both packages to access
the upgrade/common files.

These symlinks are always problematic on Windows, where we had to run
a script to re-link them, and restore them.

This change uses Bazel packages to share the `upgrade/common` code,
which avoids the need for symlinking the `src` folder.

Also, the Windows specific scripts that fixup the symlinks have also
been removed as there is no more need for them.

PR Close #29466
2019-04-02 10:38:01 -07:00
Paul Gschwendtner 8badf9808a build: add pre-release check that validates the version name (#29551)
Currently with the release of "8.0.0-beta.10", the Bazel npm packag accidentally
was stamped with an incorrect version placeholder: `8.0.0-beta.10+1.sha-a28b3e3`.

This can happen because the placeholder is based on latest tag that matches the
Semver format. e.g. if `HEAD` equals to the commit that has the latest tag, the
version name will be correct and refer to the tag name (e.g. `8.0.0-beta.10`). Though
if the latest commit is not tagged with the most recent tag, the version
name will also include the SHA of the commit (e.g.  `8.0.0-beta.10+1.sha-a28b3e3`).

We can ensure that we don't accidentally release versions from a more recent commit
that shouldn't be part of the release by adding a pre-release check that ensures that
the `BUILD_SCM_VERSION` Bazel status variable matches the expected version format.

PR Close #29551
2019-03-27 12:26:13 -07:00
Filipe Silva e185d3a4ad ci: add codefresh (#29305)
PR Close #29305
2019-03-21 22:19:19 +00:00
Paul Gschwendtner 6f3052b799 ci: disable sauce-connect ssl bumping (#29447)
By default we disable SSL bumping for all requests. This is because SSL
bumping is not needed for our test setup and in order to perform the SSL
bumping, Saucelabs intercepts all HTTP requests in the tunnel VM and modifies
them. This can cause flakiness as it makes all requests dependent on the SSL bumping
middleware.

See: https://wiki.saucelabs.com/display/DOCS/Troubleshooting+Sauce+Connect#TroubleshootingSauceConnect-DisablingSSLBumping

PR Close #29447
2019-03-21 22:18:18 +00:00
Pete Bacon Darwin 7a67f8935d build: allow build-packages-dist.sh to be run from anywhere (#29092)
PR Close #29092
2019-03-20 14:45:54 -04:00
Paul Gschwendtner a530ed11e8 ci: do not cache modified "node_modules" in "material-unit-tests" job (#29416)
Currently we cache the Material `node_modules` after
the `run_angular_material_unit_tests.sh` completed. This
means that the cache will incorrectly contain the Ivy NPM
package output which might be incompatible with the
other Material dependencies. e.g. the Material postinstall
command now uses a different NGC version that does not
work with the `typescript` version that has been specified in
the Material project.

PR Close #29416
2019-03-20 06:58:23 -07:00
Paul Gschwendtner 37cc514f0f ci: add yarn cache for material-unit tests job (#29378)
PR Close #29378
2019-03-19 17:14:53 -04:00
Paul Gschwendtner 43ce6ec84a ci: print sauce-connect log output on timeout (#29084)
Currently when `sauce-connect` times out after 2min, we just
print a message saying that the SauceLabs tunnel didn't establish
within 2min. In order to make debugging easier, we now print the
full log file output on failure.

PR Close #29084
2019-03-04 10:10:28 -08:00
Greg Magolan 75357ecb32 build(bazel): run a number of web tests with karma_web_test in saucelabs in CircleCI (#27721)
PR Close #27721
2019-02-22 13:07:08 -08:00
Igor Minar 05c25ccca7 build: update deps in ./scripts/release/post-check
The versions were off and @angular/http dep was missing.
2019-02-20 13:51:31 -08:00
Paul Gschwendtner dba2a406fd build: remove legacy e2e tests job (#28645)
Now that all e2e integration tests within `modules/` have been
migrated to Bazel, we can remove the legacy e2e tests job.

PR Close #28645
2019-02-13 12:15:02 -08:00
Matias Niemelä 76a6eacb4e refactor(ivy): rename "blacklist" to "blocklist" (#28536)
PR Close #28536
2019-02-05 14:06:15 -05:00
Jeremy Elbourn 1b33142595 build: run angular/material2 unit tests on ci (#28197)
PR Close #28197
2019-02-01 12:15:35 -05:00
Paul Gschwendtner 40d64b6b58 build: run offline_compiler_test using bazel (#28191)
PR Close #28191
2019-01-28 20:07:22 -08:00
Paul Gschwendtner 98e5af1480 build: switch example e2e tests to bazel (#28402)
* No longer builds the example e2e tests using "tsc". The examples are now built with Bazel and can therefore be built with Ivy by using the `--define=compile=aot` switch.
* No longer runs the example e2e tests using the protractor CLI. example e2e tests are executed with the Bazel protractor rule and can therefore run incrementally.

NOTE: Unit tests found within the examples are still running within the legacy jobs.

PR Close #28402
2019-01-28 19:21:09 -08:00
Igor Minar 9dabeea807 ci: add a rebase check to the merge-pr script (#28250)
Adds a check to verify that each PR branch to be merged upstream contains SHAs of commits that significantly changed our CI infrastructure.

This check is used to enforce that we don't merge PRs that have not been rebased recently and could result in merging of non-approved or otherwise bad changes.

PR Close #28250
2019-01-22 09:26:53 -08:00
Paul Gschwendtner c7d1890aaa build: remove travisci leftovers (#27979)
PR Close #27979
2019-01-09 10:41:16 -08:00
Paul Gschwendtner 0199e26167 ci: remove travis ci setup (#27937)
we no longer need it... yay!!!

PR Close #27937
2019-01-07 15:35:09 -08:00
Paul Gschwendtner 7de7b7a16a ci: move e2e tests from travis to circleci (#27937)
PR Close #27937
2019-01-07 15:35:09 -08:00
Igor Minar bb5ddee710 ci: remove build steps that are no longer needed (#27937)
the metadata build seems to be needed only by the offline compiler tests which is currently disabled

PR Close #27937
2019-01-07 15:35:09 -08:00
Igor Minar 04ca3bcf10 ci: move local and saucelabs unit tests to circle (#27937)
Moving the tests over to CircleCI in pretty much "as-is" state just so that we can drop the dependency on Travis.

In the followup changes we plan to migrate these tests to run on sauce under bazel. @gregmagolan is working on that.

I've previously verified that all the tests executed in legacy-unit-tests-local already under bazel.
Therefore the legacy-unit-tests-local job is strictly not necessary any more, but given how flaky legacy-unit-tests-saucelabs is,
it is good to have the -local job just so that we can quickly determine if any failure is a flake or legit issue
(the bazel version of these tests could theoretically run in a slightly different way and fail or not fail in a different way, so having -lcoal job is just an extra safety check).

This change was coauthored with @devversion

PR Close #27937
2019-01-07 15:35:09 -08:00
Igor Minar 8a05199fb9 test: remove bower and polymer benchmarks (#27931)
the polymer benchmarks are super old and not relevant any more

and these benchmarks were the only reason why we needed bower at all

so long, bower. thanks for all the fish.

PR Close #27931
2019-01-04 12:02:22 -08:00
Paul Gschwendtner c6ae72987b build: re-enable saucelabs non-verbose logging (#27657)
Previously the logging to TravisCI has been disabled because the `print-logs.sh` file printed the Sauce-Connect logfile output that is `verbose` by default. See [for example](https://travis-ci.org/angular/angular/jobs/250578973).

Since the default stdout/stderr of sauce-connect is pretty much concise and can alert us if we run into any issues (e.g. rate limit, outdated tunnel version), we should stop piping these to `/dev/null`.

PR Close #27657
2018-12-14 10:19:54 -08:00
Paul Gschwendtner 5256a91fb2 build: update to latest karma-sauce-launcher version (#27634)
PR Close #27634
2018-12-13 10:58:18 -08:00
Igor Minar 8a63f6e245 ci: don't run node unit tests on travis (#27386)
we run all these tests on CircleCI with bazel.

PR Close #27386
2018-12-05 20:55:32 -08:00
Igor Minar 2ccf5c4ffe ci: remove validate-commit-message tests from travis (#27386)
we alredy run them via bazel.

I also removed a bunch of obsolete files that bazel doesn't need.

PR Close #27386
2018-12-05 20:55:32 -08:00
Igor Minar 55e24d2602 ci: don't run router tests on travis (#27386)
we already run them on circle under bazel

PR Close #27386
2018-12-05 20:55:31 -08:00
George Kalpakas 445f7896c1 build: error when files specified in payload size limits are missing (#27250)
PR Close #27250
2018-12-05 13:26:26 -08:00
Alex Eagle f5f323dae0 build: only stamp version info when releasing (#27362)
Also build releases into a dedicated output_base so you can't
accidentally publish with outdated version stamp.

Bump the version of rules_nodejs so we don't need to create the
symlink_prefixes for the .publish command to work.

PR Close #27362
2018-11-30 16:08:23 -08:00
Alex Eagle 026b7e34b3 build: update yarn version (#27193)
Some engineers were already on Yarn 0.10.x which was permitted by the range in our package.json#engines
However this introduced 'integrity sha512' lines into the yarn.lock files.
Then when engineers use yarn 0.9 (in particular, Bazel did this) then the lock files get tons of meaningless edits.
We could force everyone back to yarn 0.9 but this commit chooses to instead advance everyone past 0.10

PR Close #27193
2018-11-21 07:46:22 -08:00
Paul Gschwendtner 2a7210e382 build: release scripts should use bazel from yarn (#26899)
* With ec29fd3e7b, the prefix for the `bazel` command has been removed because it threw off the Bash variables. In order to fix this while still running Bazel from the `node_modules` (keeping the bazel versions consistent), we should run Bazel without Yarn (similar to how it's done for `build-packages-dist.sh`)

PR Close #26899
2018-11-20 10:44:34 -08:00
Fabio Picheli f9a4abb6dc fix: missing semi-colon and empty line (#26945)
PR Close #26945
2018-11-05 11:37:55 -08:00
Matias Niemelä ec29fd3e7b build: remove problematic `yarn` prefix for running bazel 2018-10-31 16:01:36 -07:00
Paul Gschwendtner 8fc4ae51fb build: use bazel version from node modules (#26691)
* No longer depends on a custom CircleCI docker image that comes with Bazel pre-installed. Since Bazel is now available through NPM, we should be able to use the version from `@bazel/bazel` in order to enforce a consistent environment on CI and locally.
* This also reduces the amount of packages that need to be published (ngcontainer is removed)

PR Close #26691
2018-10-30 16:19:13 -04:00
George Kalpakas 5e2ce9b2a6 build: clean up `*.gz` files created by `payload-size.sh` (#26746)
These files are not needed once the size has been calculated and there
is no point in keeping them around.

Deleting them prevents, for example, uploading unnecessary files from
`aio/dist/` to Firebase (because `deploy-to-firebase.sh` runs the
payload size checks right before deploying).

PR Close #26746
2018-10-25 21:17:52 -04:00
Igor Minar 07b89902d5 ci: don't publish ivy build artifacts on a non-master branch 2018-10-23 15:31:55 -07:00
George Kalpakas ce6948fc1b ci(docs-infra): remove jobs from Travis config (#26377)
PR Close #26377
2018-10-23 14:35:38 -07:00
George Kalpakas 38d626a3fa ci(docs-infra): move deployment to CircleCI (#26377)
PR Close #26377
2018-10-23 14:35:38 -07:00
George Kalpakas 9b8a244a15 ci: fix `payload-size.sh` (#26377)
PR Close #26377
2018-10-23 14:35:38 -07:00
George Kalpakas 3b24e0edb6 build: use CI-provider independent variable names (#26377)
PR Close #26377
2018-10-23 14:35:37 -07:00
Igor Minar ee0b857172 build: rename the ivy compile mode 'local' to 'aot' (#26686)
PR Close #26686
2018-10-23 14:14:49 -07:00