angular-cn/integration
George Kalpakas 3f68377c3d ci: check versions of non-local integration project dependencies (#33968)
In order to keep integration tests on CI as determinitstic as possible,
we need to ensure that the same dependencies (including transitive ones)
are installed each time. One way to ensure that is using a lockfile
(such as `yarn.lock`) to pin the dependencies to exact versions. This
works as long as the lockfile itself is in-sync with the corresponding
`package.json`, which specifies the dependencies.

Ideally, we would run `yarn install` with the `--frozen-lockfile` option
to verify that the lockfile is in-sync with `package.json`, but we
cannot do that for integration projects, because we want to be able to
install the locally built Angular packages). Therefore, we must manually
esnure that the integration project lockfiles remain in-sync, which is
error-prone.

This commit introduces a helper script that performs some checks on each
project's (non-local) dependencies:
- Ensure that exact versions (not version ranges) are specified in
  `package.json`. This reduces the probability of installing a breaking
  version of a direct or transitive dependency, in case of an
  out-of-sync lockfile.
- Ensure that the lockfile is in-sync with `package.json` wrt these
  dependencies.

While these checks are not full-proof, they provide yet another line of
defense against indeterminism.

PR Close #33968
2019-11-26 16:08:33 -08:00
..
bazel build: use exact versions for integration project dependencies (#33968) 2019-11-26 16:08:32 -08:00
bazel-schematics build: update lockfiles for integration projects (#33968) 2019-11-26 16:08:32 -08:00
cli-hello-world build: use exact versions for integration project dependencies (#33968) 2019-11-26 16:08:32 -08:00
cli-hello-world-ivy-compat build: use exact versions for integration project dependencies (#33968) 2019-11-26 16:08:32 -08:00
cli-hello-world-ivy-i18n build: move integration project dependencies to `devDependencies` (#33968) 2019-11-26 16:08:32 -08:00
cli-hello-world-ivy-minimal build: use exact versions for integration project dependencies (#33968) 2019-11-26 16:08:32 -08:00
cli-hello-world-lazy build: use exact versions for integration project dependencies (#33968) 2019-11-26 16:08:32 -08:00
cli-hello-world-lazy-rollup build: use exact versions for integration project dependencies (#33968) 2019-11-26 16:08:32 -08:00
dynamic-compiler build: update lockfiles for integration projects (#33968) 2019-11-26 16:08:32 -08:00
hello_world__closure test(ngcc): build `zone.js` from source in `scripts/build-packages-dist.sh` (#33733) 2019-11-12 09:55:16 -08:00
hello_world__systemjs_umd test(ngcc): build `zone.js` from source in `scripts/build-packages-dist.sh` (#33733) 2019-11-12 09:55:16 -08:00
i18n test(ngcc): build `zone.js` from source in `scripts/build-packages-dist.sh` (#33733) 2019-11-12 09:55:16 -08:00
injectable-def build: update lockfiles for integration projects (#33968) 2019-11-26 16:08:32 -08:00
ivy-i18n build: move integration project dependencies to `devDependencies` (#33968) 2019-11-26 16:08:32 -08:00
language_service_plugin build: update lockfiles for integration projects (#33968) 2019-11-26 16:08:32 -08:00
ng_elements test(ngcc): build `zone.js` from source in `scripts/build-packages-dist.sh` (#33733) 2019-11-12 09:55:16 -08:00
ng_update test(ngcc): build `zone.js` from source in `scripts/build-packages-dist.sh` (#33733) 2019-11-12 09:55:16 -08:00
ng_update_migrations build: use exact versions for integration project dependencies (#33968) 2019-11-26 16:08:32 -08:00
ngcc build: use exact versions for integration project dependencies (#33968) 2019-11-26 16:08:32 -08:00
platform-server build: use exact versions for integration project dependencies (#33968) 2019-11-26 16:08:32 -08:00
service-worker-schema build: update lockfiles for integration projects (#33968) 2019-11-26 16:08:32 -08:00
side-effects docs(ivy): improve the missing `$localize` error message (#33826) 2019-11-15 10:38:36 -08:00
terser build: update lockfiles for integration projects (#33968) 2019-11-26 16:08:32 -08:00
typings_test_ts36 build: update lockfiles for integration projects (#33968) 2019-11-26 16:08:32 -08:00
.gitignore test(ivy): i18n - add compile time translation to integration test (#32881) 2019-10-09 13:19:38 -07:00
README.md build: update lockfiles for integration projects (#33968) 2019-11-26 16:08:32 -08:00
_payload-limits.json build: update payload limits for `cli-hello-world-ivy-i18n` es2015 bundle (#34043) 2019-11-25 20:50:41 -05:00
check-dependencies.js ci: check versions of non-local integration project dependencies (#33968) 2019-11-26 16:08:33 -08:00
get-sharded-tests.js build: shard integration tests on circleci (#27937) 2019-01-07 15:35:09 -08:00
run_tests.sh ci: check versions of non-local integration project dependencies (#33968) 2019-11-26 16:08:33 -08:00

README.md

Integration tests for Angular

This directory contains end-to-end tests for Angular. Each directory is a self-contained application that exactly mimics how a user might expect Angular to work, so they allow high-fidelity reproductions of real-world issues.

For this to work, we first build the Angular distribution just like we would publish it to npm, then install the distribution into each app.

To test Angular CLI applications, we use the cli-hello-world-* integration tests. When a significant change is released in the CLI, the applications should be updated with ng update:

$ cd integration/cli-hello-world[-*]
$ yarn install
$ yarn ng update @angular/cli @angular-devkit/build-angular
# yarn build
# yarn test
# typescript version

Render3 tests

The directory cli-hello-world-ivy-compat contains a test for render3 used with the angular cli.

The cli-hello-world-ivy-minimal contains a minimal ivy app that is meant to mimic the bazel equivalent in packages/core/test/bundling/hello_world, and should be kept similar.

Writing an integration test

The API for each test is:

  • Each sub-directory here is an integration test
  • Each test should have a package.json file
  • The test runner will run yarn and yarn test on the package

This means that the test should be started by test script, like

"scripts": {"test": "runProgramA && assertResultIsGood"}

Note that the package.json file uses a special file:../../dist scheme to reference the Angular packages, so that the locally-built Angular is installed into the test app.

Also, beware of floating (non-locked) dependencies. If in doubt, you can install the package directly from file:../../node_modules.

WARNING

Always ensure that yarn.lock files are up-to-date with the corresponding package.json files (wrt the non-local dependencies - i.e. dependencies whose versions do not start with file:).

You can update a yarn.lock file by running yarn install in the project subdirectory.

Running integration tests

$ ./integration/run_tests.sh

The test runner will first re-build any stale npm packages, then cd into each subdirectory to execute the test.