angular-cn/integration
Alan Agius 1f89c6130e fix(ngcc): show helpful error when providing an invalid option (#36010)
Currently, when running the ngcc binary directly and provide an invalid option ngcc will not error out and the user might have a hard time telling why ngcc is behaving not as expected.

With this change we now output an actionable error:
```
 yarn ngcc --unknown-option
Options:
  --version                          Show version number               [boolean]
  -s, --source                       A path (relative to the working directory)
                                     of the `node_modules` folder to process.
                                                     [default: "./node_modules"]
  -p, --properties                   An array of names of properties in
                                     package.json to compile (e.g. `module` or
                                     `es2015`)
                                     Each of these properties should hold the
                                     path to a bundle-format.
                                     If provided, only the specified properties
                                     are considered for processing.
                                     If not provided, all the supported format
                                     properties (e.g. fesm2015, fesm5, es2015,
                                     esm2015, esm5, main, module) in the
                                     package.json are considered.        [array]
  -t, --target                       A relative path (from the `source` path) to
                                     a single entry-point to process (plus its
                                     dependencies).
  --first-only                       If specified then only the first matching
                                     package.json property will be compiled.
                                                                       [boolean]
  --create-ivy-entry-points          If specified then new `*_ivy_ngcc`
                                     entry-points will be added to package.json
                                     rather than modifying the ones in-place.
                                     For this to work you need to have custom
                                     resolution set up (e.g. in webpack) to look
                                     for these new entry-points.
                                     The Angular CLI does this already, so it is
                                     safe to use this option if the project is
                                     being built via the CLI.          [boolean]
  --legacy-message-ids               Render `$localize` messages with legacy
                                     format ids.
                                     The default value is `true`. Only set this
                                     to `false` if you do not want legacy
                                     message ids to
                                     be rendered. For example, if you are not
                                     using legacy message ids in your
                                     translation files
                                     AND are not doing compile-time inlining of
                                     translations, in which case the extra
                                     message ids
                                     would add unwanted size to the final source
                                     bundle.
                                     It is safe to leave this set to true if you
                                     are doing compile-time inlining because the
                                     extra
                                     legacy message ids will all be stripped
                                     during translation.
                                                       [boolean] [default: true]
  --async                            Whether to compile asynchronously. This is
                                     enabled by default as it allows
                                     compilations to be parallelized.
                                     Disabling asynchronous compilation may be
                                     useful for debugging.
                                                       [boolean] [default: true]
  -l, --loglevel                     The lowest severity logging message that
                                     should be output.
                                     [choices: "debug", "info", "warn", "error"]
  --invalidate-entry-point-manifest  If this is set then ngcc will not read an
                                     entry-point manifest file from disk.
                                     Instead it will walking the directory tree
                                     as normal looking for entry-points, and
                                     then write a new manifest file.
                                                      [boolean] [default: false]
  --help                             Show help                         [boolean]
Unknown arguments: unknown-option, unknownOption
```

PR Close #36010
2020-03-11 14:42:16 -04:00
..
bazel build: update to rules_nodejs 1.4.1 (#35999) 2020-03-10 20:57:40 -04:00
bazel-schematics build: fix integration tests flakes using local yarn cache for bazel-schematics & ng_elements_schematics demos (#35877) 2020-03-05 15:30:20 -05:00
cli-hello-world build: add npm_integration_test && angular_integration_test (#33927) 2020-02-24 08:59:18 -08:00
cli-hello-world-ivy-compat build: move build scripts to dedicated directory (#35780) 2020-03-04 08:35:26 -08:00
cli-hello-world-ivy-i18n build: add npm_integration_test && angular_integration_test (#33927) 2020-02-24 08:59:18 -08:00
cli-hello-world-ivy-minimal build: move build scripts to dedicated directory (#35780) 2020-03-04 08:35:26 -08:00
cli-hello-world-lazy build: add npm_integration_test && angular_integration_test (#33927) 2020-02-24 08:59:18 -08:00
cli-hello-world-lazy-rollup build: add npm_integration_test && angular_integration_test (#33927) 2020-02-24 08:59:18 -08:00
dynamic-compiler build: add npm_integration_test && angular_integration_test (#33927) 2020-02-24 08:59:18 -08:00
hello_world__closure build: add npm_integration_test && angular_integration_test (#33927) 2020-02-24 08:59:18 -08:00
hello_world__systemjs_umd build: add npm_integration_test && angular_integration_test (#33927) 2020-02-24 08:59:18 -08:00
i18n build: add npm_integration_test && angular_integration_test (#33927) 2020-02-24 08:59:18 -08:00
injectable-def build: add npm_integration_test && angular_integration_test (#33927) 2020-02-24 08:59:18 -08:00
ivy-i18n build: move build scripts to dedicated directory (#35780) 2020-03-04 08:35:26 -08:00
language_service_plugin build: typescript 3.8 support (#35864) 2020-03-10 17:51:20 -04:00
ng_elements test: run /integration/ng_elements_schematics test with bazel (#35669) 2020-02-26 12:58:35 -08:00
ng_elements_schematics build: fix integration tests flakes using local yarn cache for bazel-schematics & ng_elements_schematics demos (#35877) 2020-03-05 15:30:20 -05:00
ng_update test: use puppeteer in integration tests and to download correct chromedriver (#35049) 2020-02-11 13:16:52 -08:00
ng_update_migrations build: add npm_integration_test && angular_integration_test (#33927) 2020-02-24 08:59:18 -08:00
ngcc fix(ngcc): show helpful error when providing an invalid option (#36010) 2020-03-11 14:42:16 -04:00
platform-server build: remove dep on tsickle from platform-server (#33927) 2020-02-24 08:59:18 -08:00
service-worker-schema build: update lockfiles for integration projects (#33968) 2019-11-26 16:08:32 -08:00
side-effects Revert "refactor: use isObservable provided by rxjs 6.1+ (#27668)" 2019-11-27 13:00:59 -08:00
terser build: add npm_integration_test && angular_integration_test (#33927) 2020-02-24 08:59:18 -08:00
typings_test_ts36 build: add npm_integration_test && angular_integration_test (#33927) 2020-02-24 08:59:18 -08:00
typings_test_ts37 build: add new integration tests & bazel-in-bazel tests to … glob (#33927) 2020-02-24 08:59:18 -08:00
typings_test_ts38 build: typescript 3.8 support (#35864) 2020-03-10 17:51:20 -04:00
.gitignore build: add npm package manifest to npm_integration_test (#35669) 2020-02-26 12:58:35 -08:00
BUILD.bazel test: run /integration/ng_elements_schematics test with bazel (#35669) 2020-02-26 12:58:35 -08:00
README.md build: move build scripts to dedicated directory (#35780) 2020-03-04 08:35:26 -08:00
_payload-limits.json fix(core): remove side effects from `ɵɵgetInheritedFactory()` (#35769) 2020-03-03 08:50:03 -08:00
angular_integration_test.bzl build: no-remote-exec for integration tests (#33927) 2020-02-24 08:59:18 -08:00
check-dependencies.js ci: check versions of non-local integration project dependencies (#33968) 2019-11-26 16:08:33 -08:00
get-sharded-tests.js build: add npm_integration_test && angular_integration_test (#33927) 2020-02-24 08:59:18 -08:00
run_tests.sh build: move build scripts to dedicated directory (#35780) 2020-03-04 08:35:26 -08:00

README.md

Integration tests for Angular

This directory contains end-to-end tests for Angular. Each directory is a self-contained application that exactly mimics how a user might expect Angular to work, so they allow high-fidelity reproductions of real-world issues.

For this to work, we first build the Angular distribution via ./scripts/build/build-packages-dist.js, then install the distribution into each app.

To test Angular CLI applications, we use the cli-hello-world-* integration tests. When a significant change is released in the CLI, the applications should be updated with ng update:

$ cd integration/cli-hello-world[-*]
$ yarn install
$ yarn ng update @angular/cli @angular-devkit/build-angular
$ yarn build
$ yarn test

Afterwards the @angular/cli and @angular-devkit/build-angular should be reverted to the file:../ urls and the main package.json should be updated with the new versions.

Render3 tests

The directory cli-hello-world-ivy-compat contains a test for render3 used with the angular cli.

The cli-hello-world-ivy-minimal contains a minimal ivy app that is meant to mimic the bazel equivalent in packages/core/test/bundling/hello_world, and should be kept similar.

Writing an integration test

The API for each test is:

  • Each sub-directory here is an integration test
  • Each test should have a package.json file
  • The test runner will run yarn and yarn test on the package

This means that the test should be started by test script, like

"scripts": {"test": "runProgramA && assertResultIsGood"}

Note that the package.json file uses a special file:../../dist scheme to reference the Angular packages, so that the locally-built Angular is installed into the test app.

Also, beware of floating (non-locked) dependencies. If in doubt, you can install the package directly from file:../../node_modules.

WARNING

Always ensure that yarn.lock files are up-to-date with the corresponding package.json files (wrt the non-local dependencies - i.e. dependencies whose versions do not start with file:).

You can update a yarn.lock file by running yarn install in the project subdirectory.

Running integration tests

$ ./integration/run_tests.sh

The test runner will first re-build any stale npm packages, then cd into each subdirectory to execute the test.

Running integration tests under Bazel

The PR https://github.com/angular/angular/pull/33927 added the ability to run integration tests with Bazel. These tests can be resource intensive so it is recommended to limit the number of concurrent test jobs with the --local_test_jobs bazel flag.

Locally, if Bazel uses all of your cores to run the maximum number of integration tests in parallel then this can lead to test timeouts and flakes and freeze up your machine while these tests are running. You can limit the number of concurrent local integration tests that run with:

yarn bazel test --local_test_jobs=<N> //integration/...

Set a reasonable local_test_jobs limit for your local machine to prevent full cpu utilization during local development test runs.

To avoid having to specify this command line flag, you may want to include it in your .bazelrc.user file:

test --local_test_jobs=<N>

The downside of this is that this will apply to all tests and not just the resource intensive integration tests.

Bazel-in-bazel integration tests

Two of the integration tests that run Bazel-in-Bazel are particularly resource intensive and are tagged "manual" and "exclusive". To run these tests use,

yarn bazel test //integration:bazel_test
yarn bazel test //integration:bazel-schematics_test

Adding a new integration test

When adding a new integration test, follow the steps below to add a bazel test target for the new test.

  1. Add new test to INTEGRATION_TESTS object in /integration/BUILD.bazel (and tag as "no-ivy-aot" if not meant to be run against ivy bundles). NB: if the test requires any special attribute then make a new angular_integration_test target instead.
  2. If test requires ports and does not support ethereal ports then make sure the port is unique and add it to the "manually configured ports" comment to document which port it is using
  3. Add at least the following two entries .bazelignore (as they may contain BUILD files)
    1. integration/new_test/node_modules
    2. integration/new_test/.yarn_local_cache
  4. Add any other untracked folders to .bazelignore that may contain BUILD files
  5. If there are tracked BUILD files in the integration test folder (integration/bazel has these for example) add those folders to the build --deleted_packages and query --deleted_packages lines in .bazelrc

Browser tests

For integration tests we use the puppeteer provisioned version of Chrome. For both Karma and Protractor tests we set a number of browser testing flags. To avoid duplication, they will be listed and explained here and the code will reference this file for more information.

No Sandbox: --no-sandbox

The sandbox needs to be disabled with the --no-sandbox flag for both Karma and Protractor tests, because it causes Chrome to crash on some environments.

See: http://chromedriver.chromium.org/help/chrome-doesn-t-start See: https://github.com/puppeteer/puppeteer/blob/v1.0.0/docs/troubleshooting.md#chrome-headless-fails-due-to-sandbox-issues

Headless: --headless

So that browsers are not popping up and tearing down when running these tests we run Chrome in headless mode. The --headless flag puts Chrome in headless mode and a number of other flags are recommended in this mode as well:

  • --headless
  • --disable-gpu
  • --disable-dev-shm-usage
  • --hide-scrollbars
  • --mute-audio

These come from the flags that puppeteer passes to chrome when it launches it in headless mode: 18f2ecdffd/lib/Launcher.js (L91)

And from the flags that the Karma ChromeHeadless browser passes to Chrome: 5f70a76de8/index.js (L171)

Disable shared memory space: --disable-dev-shm-usage

The --disable-dev-shm-usage flag disables the usage of /dev/shm because it causes Chrome to crash on some environments.

On CircleCI, the puppeteer provisioned Chrome crashes with CI we get Root cause: org.openqa.selenium.WebDriverException: unknown error: DevToolsActivePort file doesn't exist which resolves without this flag.

See: https://github.com/puppeteer/puppeteer/blob/v1.0.0/docs/troubleshooting.md#tips See: https://stackoverflow.com/questions/50642308/webdriverexception-unknown-error-devtoolsactiveport-file-doesnt-exist-while-t