9456eca7c5
Adds better caching for browser archives and their extraction. This is done because the archives are currently extracted as a build action and these are actions are invalidated frequently, causing flakiness on the CI and slow-down in local development. Here is an example flaky error on the CI (that surfaces often with RBE execution): ``` ERROR: /home/circleci/.cache/bazel/_bazel_circleci/9ce5c2144ecf75d11717c0aa41e45a8d/external/npm/@angular/dev-infra-private/bazel/browsers/chromium/BUILD.bazel:22:17: Extracting ../org_chromium_chromium_amd64/file/chrome-linux.zip failed: (Exit 34): extract.sh failed: error executing command external/io_bazel_rules_webtesting/web/internal/extract.sh external/org_chromium_chromium_amd64/file/chrome-linux.zip ... (remaining 2 argument(s) skipped). Note: Remote connection/protocol failed with: execution failed ``` We fix this by introducing a new rule that downloads a browser archive and unpacks it directly into a Bazel repository. Before this change, the archive would just be downloaded but extracted later as part of a build action. This is unnecessary and results in less efficient caching as build actions are invalidated more often, especially if developers run `bazel clean` in between. The root cause on why the extraction often fails in RBE containers is unclear. It's unclear why the extacted archive is not cached properly as part of a build action (most likely some hermeticity issue within `rules_webtesting`, but it seems more Bazel-idiomatic to unpack the archives as part of the repository anyway, and this solves the flakiness issue. PR Close #42814
70 lines
2.7 KiB
Python
70 lines
2.7 KiB
Python
load("@io_bazel_rules_webtesting//web/internal:metadata.bzl", "metadata")
|
|
load("@io_bazel_rules_webtesting//web/internal:provider.bzl", "WebTestInfo")
|
|
|
|
"""Converts the specified label to a manifest path"""
|
|
|
|
def _label_to_manifest_path(label):
|
|
if label.package != "":
|
|
return "%s/%s" % (label.workspace_name, label.package)
|
|
return label.workspace_name
|
|
|
|
"""Implementation of the `browser_configure` rule."""
|
|
|
|
def _browser_configure_impl(ctx):
|
|
named_files = {}
|
|
base_dir = _label_to_manifest_path(ctx.label)
|
|
|
|
# Update the named files to manifest paths that can be resolved
|
|
# with Bazel runfile resolution in web tests.
|
|
for n, p in ctx.attr.named_files.items():
|
|
named_files[n] = base_dir + "/" + p
|
|
|
|
# Create a web test metadata file that will be provided as part of
|
|
# the `WebTestInfo` provider.
|
|
metadata.create_file(
|
|
ctx = ctx,
|
|
output = ctx.outputs.web_test_metadata,
|
|
web_test_files = [
|
|
metadata.web_test_files(ctx = ctx, named_files = named_files),
|
|
],
|
|
)
|
|
|
|
return [
|
|
DefaultInfo(runfiles = ctx.runfiles(files = ctx.files.files)),
|
|
WebTestInfo(metadata = ctx.outputs.web_test_metadata),
|
|
]
|
|
|
|
"""
|
|
Rule that is used in combination with the `browser_archive` rule. It captures a set
|
|
of files which are needed for dealing with a browser. Additionally, specific files
|
|
for the browser can be denoted with an unique name so that web tests can access browser
|
|
files in a platform-agnostic way, regardless of which browser repository is selected.
|
|
|
|
The specified browser files are exposed as runfiles of the target defined through this
|
|
rule. The unique names with their associated files are captured within a metadata file
|
|
that is exposed through a `WebTestInfo` provider. Web tests will be able to deal with
|
|
this metadata file to resolve browser files in a platform-agnostic way.
|
|
|
|
More details on this can be found in the `browser_archive` rule.
|
|
"""
|
|
browser_configure = rule(
|
|
attrs = {
|
|
"files": attr.label_list(
|
|
mandatory = True,
|
|
allow_files = True,
|
|
doc = "List of files which are needed for the browser.",
|
|
),
|
|
"named_files": attr.string_dict(
|
|
doc = """
|
|
Dictionary that maps files to unique identifiers. This is useful
|
|
if browser archives are different on different platforms and the web
|
|
tests would not want to care about archive-specific paths. e.g. targets
|
|
expect a `CHROMIUM` key to point to the Chromium browser binary.
|
|
""",
|
|
mandatory = True,
|
|
),
|
|
},
|
|
outputs = {"web_test_metadata": "%{name}.gen.json"},
|
|
implementation = _browser_configure_impl,
|
|
)
|