angular-cn/dev-infra/bazel/browsers/browser_configure.bzl

70 lines
2.7 KiB
Python
Raw Normal View History

feat(dev-infra): better caching for browser archive contents (#42814) Adds better caching for browser archives and their extraction. This is done because the archives are currently extracted as a build action and these are actions are invalidated frequently, causing flakiness on the CI and slow-down in local development. Here is an example flaky error on the CI (that surfaces often with RBE execution): ``` ERROR: /home/circleci/.cache/bazel/_bazel_circleci/9ce5c2144ecf75d11717c0aa41e45a8d/external/npm/@angular/dev-infra-private/bazel/browsers/chromium/BUILD.bazel:22:17: Extracting ../org_chromium_chromium_amd64/file/chrome-linux.zip failed: (Exit 34): extract.sh failed: error executing command external/io_bazel_rules_webtesting/web/internal/extract.sh external/org_chromium_chromium_amd64/file/chrome-linux.zip ... (remaining 2 argument(s) skipped). Note: Remote connection/protocol failed with: execution failed ``` We fix this by introducing a new rule that downloads a browser archive and unpacks it directly into a Bazel repository. Before this change, the archive would just be downloaded but extracted later as part of a build action. This is unnecessary and results in less efficient caching as build actions are invalidated more often, especially if developers run `bazel clean` in between. The root cause on why the extraction often fails in RBE containers is unclear. It's unclear why the extacted archive is not cached properly as part of a build action (most likely some hermeticity issue within `rules_webtesting`, but it seems more Bazel-idiomatic to unpack the archives as part of the repository anyway, and this solves the flakiness issue. PR Close #42814
2021-07-10 11:33:54 -04:00
load("@io_bazel_rules_webtesting//web/internal:metadata.bzl", "metadata")
load("@io_bazel_rules_webtesting//web/internal:provider.bzl", "WebTestInfo")
"""Converts the specified label to a manifest path"""
def _label_to_manifest_path(label):
if label.package != "":
return "%s/%s" % (label.workspace_name, label.package)
return label.workspace_name
"""Implementation of the `browser_configure` rule."""
def _browser_configure_impl(ctx):
named_files = {}
base_dir = _label_to_manifest_path(ctx.label)
# Update the named files to manifest paths that can be resolved
# with Bazel runfile resolution in web tests.
for n, p in ctx.attr.named_files.items():
named_files[n] = base_dir + "/" + p
# Create a web test metadata file that will be provided as part of
# the `WebTestInfo` provider.
metadata.create_file(
ctx = ctx,
output = ctx.outputs.web_test_metadata,
web_test_files = [
metadata.web_test_files(ctx = ctx, named_files = named_files),
],
)
return [
DefaultInfo(runfiles = ctx.runfiles(files = ctx.files.files)),
WebTestInfo(metadata = ctx.outputs.web_test_metadata),
]
"""
Rule that is used in combination with the `browser_archive` rule. It captures a set
of files which are needed for dealing with a browser. Additionally, specific files
for the browser can be denoted with an unique name so that web tests can access browser
files in a platform-agnostic way, regardless of which browser repository is selected.
The specified browser files are exposed as runfiles of the target defined through this
rule. The unique names with their associated files are captured within a metadata file
that is exposed through a `WebTestInfo` provider. Web tests will be able to deal with
this metadata file to resolve browser files in a platform-agnostic way.
More details on this can be found in the `browser_archive` rule.
"""
browser_configure = rule(
attrs = {
"files": attr.label_list(
mandatory = True,
allow_files = True,
doc = "List of files which are needed for the browser.",
),
"named_files": attr.string_dict(
doc = """
Dictionary that maps files to unique identifiers. This is useful
if browser archives are different on different platforms and the web
tests would not want to care about archive-specific paths. e.g. targets
expect a `CHROMIUM` key to point to the Chromium browser binary.
""",
mandatory = True,
),
},
outputs = {"web_test_metadata": "%{name}.gen.json"},
implementation = _browser_configure_impl,
)