From f64c81c3f83ee00885a127fa4adc00c34583cc2c Mon Sep 17 00:00:00 2001 From: Chris Hostetter Date: Thu, 14 Oct 2021 10:50:16 -0700 Subject: [PATCH] LUCENE-10173: remove max-worker restriction added by LUCENE-9488 when 'useGpg' in effect Also update docs to remove the point of confusion that lead to thinking that restriction was useful --- help/publishing.txt | 52 ++++++++++----------- lucene/distribution/artifact-signing.gradle | 16 +++++-- 2 files changed, 37 insertions(+), 31 deletions(-) diff --git a/help/publishing.txt b/help/publishing.txt index 86852cf2172..a37cc657d44 100644 --- a/help/publishing.txt +++ b/help/publishing.txt @@ -54,8 +54,8 @@ Signing can be enabled by adding the "-Psign" option, for example: gradlew assembleRelease mavenToApacheReleases -Psign -If using signatures, make yourself familiar with how to pass the required signatory -credentials via ~/.gradle/gradle.properties or command-line options: +By default gradle uses a Java-based implementation of PGP for signing, which requieres +several "signing.*" properties via either ~/.gradle/gradle.properties or command-line options: https://docs.gradle.org/current/userguide/signing_plugin.html#sec:signatory_credentials @@ -67,23 +67,21 @@ The keyId is the last 8 digits of your key (gpg -k will print your keys). Gradle of secure passing of private key information and passwords. -Artifact signing using an external GPG (or GPG agent) +Artifact signing using an external GPG with GPG Agent ----------------------------------------------------- -You can use an external GPG command (or GPG agent) but this changes the options used and may require a -restriction on gradle concurrency: +You can use an external GPG command to deal with signing artifacts, with out needing to give gradle your passphrase, +by adding a "-PuseGpg" option, but this changes the properties you must specify: For gpg2: -gradlew [tasks] -Psign -PuseGpg --max-workers 1 -Psigning.gnupg.keyName=... +gradlew [tasks] -Psign -PuseGpg -Psigning.gnupg.keyName=... For gpg: -gradlew [tasks] -Psign -PuseGpg --max-workers 1 -Psigning.gnupg.keyName=... -Psigning.gnupg.useLegacyGpg=true +gradlew [tasks] -Psign -PuseGpg -Psigning.gnupg.keyName=... -Psigning.gnupg.useLegacyGpg=true -The keyName is the last 8 digits of your key (gpg -k will print your keys). +The keyName is the last 8 digits of your key (gpg -k will print your keys). -There are a few possible quirks when using an external GPG or GPG agent. -The following additional properties -- specified either on the command line via `-P...` -or in your `~/.gradle/gradle.properties` may be handy: +There are additional (optional) "signing.gnupg.*" properties which exist that may be useful/necessary in your system: signing.gnupg.useLegacyGpg=true # Changes the default executable from `gpg2` to `gpg` and explicitly sets `--use-agent` signing.gnupg.executable=gpg # Allows explicit control over what command executable used (ex: `gpg2`, `gpg`, `gpg.exe`, etc...) @@ -94,29 +92,38 @@ signing.gnupg.passphrase=... # Provide your passphrase to If in doubt, consult gradle's signing plugin documentation: https://docs.gradle.org/current/userguide/signing_plugin.html#sec:using_gpg_agent +"signing.gnupg.passphrase" is not recomended because there is no advantage to using an external GPG process if you use it. If you +are comfortable giving gradle your passphrase, then there is no reason to use an external GPG process via '-PuseGpg'. Just use the +"signing.*" options described previuosly to let gradle deal with your key directly. + +Because of how Gradle's signing plugin invokes GPG, using an external GPG process *only* works if your GPG configuration uses a +GPG agent (required by gpg2) and if the "pinentry" for your GPG agent does not require access to the tty to prompt you for a password. + +If you the following command fails with your GPG configuration, you can not use an external GPG process with gradle: + +echo foo | gpg --batch --no-tty --armor --detach-sign --use-agent --local-user YOUR_KEY_NAME + Notes About GPG Error Messages ------------------------------ -### `gpg: signing failed: Inappropriate ioctl for device` +### `gpg: signing failed: Inappropriate ioctl for device` or `Invalid IPC response` This typically happens if your `gpg-agent` is configured (either globally for your operating system, or personally in your `~/.gnupg/gpg-agent.conf`) to use a `pinentry` command which depends on using the same `tty` as the `gpg` command (ex: `pinentry-curses`, or `pinentry-tty`, etc...). -`tty` based `pinentry` implementations do not work when Gradle's `SigningPlugin` is attempting to invoke `gpg` -- among other problems: -Gradle is multi-threaded (hence --max-workers 1 above to force single-threaded execution), and we sign multiple artifacts by -default; so even if the `SigningPlugin` didn't automatically force `--no-tty` when running `gpg`, you could easily run into problems -where a second `pinentry` process wanted to read from the same `tty` in the middle of you typing in your passphrase to the first process. +`tty` based `pinentry` implementations do not work when Gradle's signing plugin is attempting to invoke `gpg` -- among other problems: +Gradle is multi-threaded and we sign multiple artifacts by default. Even if you use "--max-workers 1" to force single-threaded execution, +the signing plugin invokes gpg with `--batch --no-tty`, making it impossible for gpg (or a tty based pinentry) to prompt you for your passphrase +in the same terminal where you run Cradle. Developers are encouraged to configure a *non* `tty` based `pinentry` (ex: `pinentry-gnome`, `pinentry-x11`, `pinentry-qt`, `pinentry-mac`, `pinentry-wsl-ps1`, etc...) either globally in your operating system, or personally in your `~/.gnupg/gpg-agent.conf`, or in a new `gpg-agent.conf` file a new GnuPG configuration directory (containing a copy of your private keys) that you direct gradle to via `signing.gnupg.homeDir` -If none of these options are viable for you, then as a last resort you may wish to consider using the `signing.gnupg.passphrase=...` property. -This will expose your secret passphrase to the Gradle process, which will then pass it directly to each `gpg-agent` instance using -`--pinentry-mode=loopback`. +If this is not possible, then you should avoid using an external GPG process, and use the default (pure java) Artifact signing support ### `gpg: signing failed: No such file or directory` @@ -125,10 +132,3 @@ This may mean that there is a problem preventing `gpg` from communicating correc program) that is independent of gradle. Try running `pkill gpg-agent` and then retrying your `./gradlew` command -### `No value has been specified for property 'signatory.keyId'.` - -This typically means something went wrong when communicating with the external GPG. This is the -name of an internal property that the gradle's `SigningPlugin` expects in non-GPG mode. The error message is just confusing. - -If you see this error, it means you did not properly set `signing.gnupg.keyName` _AND_ you invoked a task which is attempting to use -the `SigningPlugin`. Please file a Jira issue and describe the problem, maybe there is a workaround for it. diff --git a/lucene/distribution/artifact-signing.gradle b/lucene/distribution/artifact-signing.gradle index d0bf6d3417a..205494aa739 100644 --- a/lucene/distribution/artifact-signing.gradle +++ b/lucene/distribution/artifact-signing.gradle @@ -30,14 +30,20 @@ task signReleaseArchives(type: Sign) { } -// Optionally, switch to using GPG command (or agent). This entails some additional -// oddities so add some extra sanity checks. +// Optionally, switch to using an external GPG command, using it's configured gpg-agent for key management if (propertyOrDefault("useGpg", null) != null) { + + // Do this check before 'useGpgCmd()', otherwise gradle will fail with a confusing error about 'signatory.keyId' + // + // 'signatory.keyId' is an implementation detail of the SigningPlugin that it populates from 'signing.gnupg.keyName' when useGpgCmd() + // is used -- but does not explain in the error produced if 'signing.gnupg.keyName' is not set. + def propName = 'signing.gnupg.keyName' + if (propertyOrDefault(propName, null) == null) { + throw new GradleException("'$propName' property must be set when using external GPG via 'useGpg', please see help/publishing.txt") + } + signing { useGpgCmd() } - if (gradle.startParameter.maxWorkerCount != 1) { - throw new GradleException("When using GPG for signing, specify --max-workers 1 to ensure sequential GPG calls.") - } }