hadoop/hadoop-common-project
Mehakmeet Singh f813554769
HADOOP-13887. Support S3 client side encryption (S3-CSE) using AWS-SDK (#2706)
This (big!) patch adds support for client side encryption in AWS S3,
with keys managed by AWS-KMS.

Read the documentation in encryption.md very, very carefully before
use and consider it unstable.

S3-CSE is enabled in the existing configuration option
"fs.s3a.server-side-encryption-algorithm":

fs.s3a.server-side-encryption-algorithm=CSE-KMS
fs.s3a.server-side-encryption.key=<KMS_KEY_ID>

You cannot enable CSE and SSE in the same client, although
you can still enable a default SSE option in the S3 console. 
  
* Filesystem list/get status operations subtract 16 bytes from the length
  of all files >= 16 bytes long to compensate for the padding which CSE
  adds.
* The SDK always warns about the specific algorithm chosen being
  deprecated. It is critical to use this algorithm for ranged
  GET requests to work (i.e. random IO). Ignore.
* Unencrypted files CANNOT BE READ.
  The entire bucket SHOULD be encrypted with S3-CSE.
* Uploading files may be a bit slower as blocks are now
  written sequentially.
* The Multipart Upload API is disabled when S3-CSE is active.

Contributed by Mehakmeet Singh
2021-07-27 11:08:51 +01:00
..
hadoop-annotations Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
hadoop-auth HADOOP-17644. Add back the exceptions removed by HADOOP-17432 for compatibility. Contributed by Quan Li. 2021-05-09 03:40:08 +05:30
hadoop-auth-examples Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
hadoop-common HADOOP-13887. Support S3 client side encryption (S3-CSE) using AWS-SDK (#2706) 2021-07-27 11:08:51 +01:00
hadoop-kms HADOOP-17794. Add a sample configuration to use ZKDelegationTokenSecretManager in Hadoop KMS (#3190) 2021-07-09 22:48:47 +09:00
hadoop-minikdc HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin (#2753) 2021-03-11 10:56:07 +09:00
hadoop-nfs HADOOP-11245. Update NFS gateway to use Netty4 (#2832) 2021-04-28 23:43:11 -07:00
hadoop-registry HADOOP-17317. [JDK 11] Upgrade dnsjava to remove illegal access warnings (#2442) 2021-07-24 14:48:25 +09:00
pom.xml Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30