honeymoose/angular-cn
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
angular-cn/integration/README.md
Greg Magolan acfd0edd38 test: use puppeteer in integration tests and to download correct chromedriver (#35049) 
This means integration tests no longer need to depend on a $CI_CHROMEDRIVER_VERSION_ARG environment variable to specify which chromedriver version to download to match the locally installed chrome. This was bad DX and not having it specified was not reliable as webdriver-manager would not always download the chromedriver version to work with the locally installed chrome.

webdriver-manager update --gecko=false --standalone=false $CI_CHROMEDRIVER_VERSION_ARG is now replaced with node webdriver-manager-update.js in the root package.json, which checks which version of chrome puppeteer has come bundled with & downloads informs webdriver-manager to download the corresponding chrome driver version.

Integration tests now use "webdriver-manager": "file:../../node_modules/webdriver-manager" so they don't have to waste time calling webdriver-manager update in postinstall

"// resolutions": "Ensure a single version of webdriver-manager which comes from root node_modules that has already run webdriver-manager update",
"resolutions": {
"**/webdriver-manager": "file:../../node_modules/webdriver-manager"
}
This should speed up each integration postinstall by a few seconds.

Further, integration test package.json files link puppeteer via file:../../node_modules/puppeteer which is the ideal situation as the puppeteer post-install won't download chrome if it is already downloaded. In CI, since node_modules is cached it should not need to download Chrome either unless the node_modules cache is busted.

NB: each version of puppeteer comes bundles with a specific version of chrome. Root package.json & yarn.lock currently pull down puppeteer 2.1.0 which comes with chrome 80. See https://github.com/puppeteer/puppeteer#q-which-chromium-version-does-puppeteer-use for more info.

Only two references to CI_CHROMEDRIVER_VERSION_ARG left in integration tests at integration/bazel-schematics/test.sh which I'm not entirely sure how to get rid of it

Use a lightweight puppeteer=>chrome version mapping instead of launching chrome and calling browser.version()

Launching puppeteer headless chrome and calling browser.version() was a heavy-handed approach to determine the Chrome version. A small and easy to update mappings file is a better solution and it means that the `yarn install` step does not require chrome shared libs available on the system for its postinstall step

PR Close #35049
2020-02-11 13:16:52 -08:00

4.3 KiB
Raw Blame History

Integration tests for Angular

This directory contains end-to-end tests for Angular. Each directory is a self-contained application that exactly mimics how a user might expect Angular to work, so they allow high-fidelity reproductions of real-world issues.

For this to work, we first build the Angular distribution via ./scripts/build-packages-dist.js, then install the distribution into each app.

To test Angular CLI applications, we use the cli-hello-world-* integration tests. When a significant change is released in the CLI, the applications should be updated with ng update:

$ cd integration/cli-hello-world[-*]
$ yarn install
$ yarn ng update @angular/cli @angular-devkit/build-angular
$ yarn build
$ yarn test

Afterwards the @angular/cli and @angular-devkit/build-angular should be reverted to the file:../ urls and the main package.json should be updated with the new versions.

Render3 tests

The directory cli-hello-world-ivy-compat contains a test for render3 used with the angular cli.

The cli-hello-world-ivy-minimal contains a minimal ivy app that is meant to mimic the bazel equivalent in packages/core/test/bundling/hello_world, and should be kept similar.

Writing an integration test

The API for each test is:

  • Each sub-directory here is an integration test
  • Each test should have a package.json file
  • The test runner will run yarn and yarn test on the package

This means that the test should be started by test script, like

"scripts": {"test": "runProgramA && assertResultIsGood"}

Note that the package.json file uses a special file:../../dist scheme to reference the Angular packages, so that the locally-built Angular is installed into the test app.

Also, beware of floating (non-locked) dependencies. If in doubt, you can install the package directly from file:../../node_modules.

WARNING

Always ensure that yarn.lock files are up-to-date with the corresponding package.json files (wrt the non-local dependencies - i.e. dependencies whose versions do not start with file:).

You can update a yarn.lock file by running yarn install in the project subdirectory.

Running integration tests

$ ./integration/run_tests.sh

The test runner will first re-build any stale npm packages, then cd into each subdirectory to execute the test.

Browser tests

For integration tests we use the puppeteer provisioned version of Chrome. For both Karma and Protractor tests we set a number of browser testing flags. To avoid duplication, they will be listed and explained here and the code will reference this file for more information.

No Sandbox: --no-sandbox

The sandbox needs to be disabled with the --no-sandbox flag for both Karma and Protractor tests, because it causes Chrome to crash on some environments.

See: http://chromedriver.chromium.org/help/chrome-doesn-t-start See: https://github.com/puppeteer/puppeteer/blob/v1.0.0/docs/troubleshooting.md#chrome-headless-fails-due-to-sandbox-issues

Headless: --headless

So that browsers are not popping up and tearing down when running these tests we run Chrome in headless mode. The --headless flag puts Chrome in headless mode and a number of other flags are recommended in this mode as well:

  • --headless
  • --disable-gpu
  • --disable-dev-shm-usage
  • --hide-scrollbars
  • --mute-audio

These come from the flags that puppeteer passes to chrome when it launches it in headless mode: 18f2ecdffd/lib/Launcher.js (L91)

And from the flags that the Karma ChromeHeadless browser passes to Chrome: 5f70a76de8/index.js (L171)

Disable shared memory space: --disable-dev-shm-usage

The --disable-dev-shm-usage flag disables the usage of /dev/shm because it causes Chrome to crash on some environments.

On CircleCI, the puppeteer provisioned Chrome crashes with CI we get Root cause: org.openqa.selenium.WebDriverException: unknown error: DevToolsActivePort file doesn't exist which resolves without this flag.

See: https://github.com/puppeteer/puppeteer/blob/v1.0.0/docs/troubleshooting.md#tips See: https://stackoverflow.com/questions/50642308/webdriverexception-unknown-error-devtoolsactiveport-file-doesnt-exist-while-t