Previously, Travis pushed the build artitfacts to the preview server.
This required us to use JWT to secure the POST request from Travis, to
ensure we couldn't receive malicious builds.
JWT has been deprecated and we are moving our builds to CircleCI.
This commit rewrites the TypeScript part of the preview server that
handles converting build artifact into hosted previews of the docs.
Now instead of pushing the AIO build artifacts to the preview server
from inside a Travis job, the artifacts are built and hosted on the
CircleCI infrastructure. The preview server will then pull these
down after being triggered by a CircleCI build webhook.
closes#19352
Previously, the upload server (for PR previews) was run as root and
"downleveled" to a non-privileged user from inside the node script.
Now, with the latest version of `pm2` (which is used to run the upload server
scripts), we can get rid of that workaround and set the desired UID directly
through `pm2`.
Since 808bd4af4, we are no longer pre-verifying PRs before uploading the build
artifacts to the preview server, thus we no longer need the
`travis-preverify-pr.sh` script or the `GITHUB_TEAM_MEMBERSHIP_CHECK_KEY`
variable.
This commit adds an API endpoint for notifying the preview server about PR
updates (`/pr-updated`). According to the update, the preview server can take
several actions. Currently, it will only check and (if necessary) update the
PR's preview visibility (but more actions could be supported in the future).
The API can be used with an automatic trigger (e.g. a GitHub webhook) to
instantly update a PR's preview visibility when it changes.
Fixes#16526
Previously, `BuildCreator#changePrVisibility()` would throw an error if the PR's
visibility was already up-to-date or if the PR directory did not exist (e.g. was
removed). This method was only used from inside `BuildCreator#create()`, which
had already checked for the existence of the directories.
This commit renames `changePrVisibility()` to `updatePrVisibility()` and makes
it more "forgiving" (i.e. it will only throw if both public and non-public
directories exist). This allows it to be used on events that may or may not have
caused the PR's visibility to change (e.g. a GitHub webhook triggered whenever a
PR's labels change).
Previously, there was a distinction between GET requests to invalid URLs and all
other requests. This was mainly because the upload-server only accepts GET
requests, but that is not a hard limitation and may change in the future.
Thus, it makes sense to return a 404 response for requests to invalid URLs
regardless of the method used.
Use the 7 first characters of the 40-chars long SHAs for shorter/cleaner URLs.
The collision probability is extremely low (since all SHAs are further
"namespaced" under the corresponding PR). In case of a collision, the second PR
will not be deployed, in order to avoid overwriting the original build.
(This is a design decision to keep the implementation simple. It can be changed
later if necessary.)
The previous clean-up code for PR directories on the preview server assumed that
all directories were named after the PR number. With the changes introduced
in #17640 it is possible to have PR directories that do not follow that naming
convention (e.g. "non-public" directories).
This PR ensures that both public and non-public directories are removed when
cleaning up.
When creating a new docker image for the preview server, the TypeScript source
code in `scripts-js/` is not copied over. Instead only the generated JavaScript
core in `scripts-js/dist/` are. Because of that, it is necessary to have run
`yarn build` before running `docker build`, so that the new docker image
contains the latest changes in `scripts-js/`.
This was previously part of the `create-image.sh` script, but was accidentally
removed in 21d213dfc.
This commit introduces the ability to show previews for PRs by any author. It works as follows:
- The build artifacts of all PRs are uploaded to the preview server.
- Automatically verified PRs (i.e. from trusted authors or having a specific label) are deployed and
publicly accessible as usual.
- PRs that could not be automatically verified are stored for later use (after re-verification).
- A PR can be marked as "trusted" and make its preview publicly accessible by adding the GitHub
label specified in the `AIO_TRUSTED_PR_LABEL` env var of the preview server.
At the moment, there is no automatic mechanism for notifying the preview server about changes to the
PR's verification status. The PR's "visibility" will be checked and updated every time a new build
is uploaded.
(Coincidentally), this wasn't an issue before fdfeaaf1f, because
pre-verification was run after `test.sh`, during which `aio-builds-setup` was
built.
Now that `deploy-staging.sh` is being run before `test.sh`, we need to build
the `aio-builds-setup` scripts first.
Previously, `aio/aio-builds-setup/scripts/travis-preverify-pr.sh` was supposed
to exit with 1 if a PR did not meet the preconditions and 2 if an error occurred
during pre-verification.
It relied on the exit codes of the node script that did the actual work, but
didn't account for errors that would be thrown in the `sh` script itself (e.g.
if the node script was not available). This caused such errors to appear as
non-verified PRs, instead of real errors that should fail the build.
This commit swaps the exit codes, so that now a 2 means non-verified PR and 1
designates an error.
Previously, only a few characters of the SHA would appear on the preview link
comment posted on the PR. This was usually enough for GitHub to create a link to
the corresponding commit, but it was possible to have collisions with other
commits with the same first characters (which prevented GitHub from identifying
the correct commit and create a link.)
This commit fixes this issue by including the full SHA on the commentso GitHub
can identify the correct commit and create the link. GitHub will automatically
truncate the link text (by default to 7 chars unless more are necessary to
uniquely identify the commit).
Previously, when trying to upload the build artifacts for a PR/SHA that was
already successfully deployed (e.g. when re-running a Travis job), the preview
server would return a 403 and the build would fail.
Since we have other mechanisms to verify that the PR author is trusted and the
artifacts do indeed come from the specified PR and since the new artifacts
should be the same with the already deployed ones (same SHA), there is no reason
to fail the build. The preview server will reject the request with a special
HTTP status code (409 - Conflict), which the `deploy-preview` script will
recognize and exit with 0.
This reverts commit d0bc83ca27.
Protractor-based prerendering is flakey on Travis and takes several minutes to
complete, slowing down the build. Prerendering has a lower impact now that we
use a ServiceWorker. We will revisit in the future (probably using a
`PlatformServer`-based approach).
PR Close#15346
Gaining access to another PR's JWT, would allow faking that PR's author wrt to
GitHub team membership verification for as long as the JWT is valid (currently
90 mins).
Pete Bacon Darwin
George Kalpakas
Veres Lajos
Victor Berchet
Ed Pelc