Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
24,638 Commits 326 Branches 371 Tags 1.5 GiB
Java 92.7%
C++ 2.9%
C 1.9%
JavaScript 1.2%
Shell 0.5%
Other 0.5%
Go to file
Mehakmeet Singh aee975a136
HADOOP-13887. Support S3 client side encryption (S3-CSE) using AWS-SDK (#2706) 
This (big!) patch adds support for client side encryption in AWS S3,
with keys managed by AWS-KMS.

Read the documentation in encryption.md very, very carefully before
use and consider it unstable.

S3-CSE is enabled in the existing configuration option
"fs.s3a.server-side-encryption-algorithm":

fs.s3a.server-side-encryption-algorithm=CSE-KMS
fs.s3a.server-side-encryption.key=<KMS_KEY_ID>

You cannot enable CSE and SSE in the same client, although
you can still enable a default SSE option in the S3 console.

* Filesystem list/get status operations subtract 16 bytes from the length
  of all files >= 16 bytes long to compensate for the padding which CSE
  adds.
* The SDK always warns about the specific algorithm chosen being
  deprecated. It is critical to use this algorithm for ranged
  GET requests to work (i.e. random IO). Ignore.
* Unencrypted files CANNOT BE READ.
  The entire bucket SHOULD be encrypted with S3-CSE.
* Uploading files may be a bit slower as blocks are now
  written sequentially.
* The Multipart Upload API is disabled when S3-CSE is active.

Contributed by Mehakmeet Singh

Change-Id: Ie1a27a036a39db66a67e9c6d33bc78d54ea708a0
 		2021-10-05 11:37:41 +01:00
.github HADOOP-15184. Add GitHub pull request template. (#1419) 2019-09-11 11:10:11 +09:00
dev-support HADOOP-17891. Fix compilation error under skipShade (ADDENDUM) (#3441) 2021-09-16 10:12:28 -07:00
hadoop-assemblies Preparing for 3.3.2 development 2021-05-19 21:52:37 +08:00
hadoop-build-tools HADOOP-17897. Allow nested blocks in switch case in checkstyle settings. (#3394) 2021-09-08 04:59:05 +00:00
hadoop-client-modules HADOOP-17891. Fix compilation error under skipShade (ADDENDUM) (#3441) 2021-09-16 10:12:28 -07:00
hadoop-cloud-storage-project Preparing for 3.3.2 development 2021-05-19 21:52:37 +08:00
hadoop-common-project HADOOP-13887. Support S3 client side encryption (S3-CSE) using AWS-SDK (#2706) 2021-10-05 11:37:41 +01:00
hadoop-dist Preparing for 3.3.2 development 2021-05-19 21:52:37 +08:00
hadoop-hdfs-project HDFS-16235. Fix Deadlock in LeaseRenewer for static remove method (#3472) 2021-09-29 18:36:36 +08:00
hadoop-mapreduce-project Revert "MAPREDUCE-7303. Fix TestJobResourceUploader failures after HADOOP-16878. Contributed by Peter Bacsko." 2021-09-23 15:04:26 -07:00
hadoop-maven-plugins HADOOP-17777. Update clover-maven-plugin version from 3.3.0 to 4.4.1 (#3152) 2021-06-30 09:35:36 +08:00
hadoop-minicluster Preparing for 3.3.2 development 2021-05-19 21:52:37 +08:00
hadoop-project HADOOP-17941. Update xerces to 2.12.1 (#3496) 2021-09-29 18:49:52 +09:00
hadoop-project-dist Preparing for 3.3.2 development 2021-05-19 21:52:37 +08:00
hadoop-tools HADOOP-13887. Support S3 client side encryption (S3-CSE) using AWS-SDK (#2706) 2021-10-05 11:37:41 +01:00
hadoop-yarn-project YARN-10970. Standby RM should expose prom endpoint (#3480) 2021-09-29 15:48:02 +09:00
licenses HADOOP-17666. Update LICENSE for 3.3.1 (#3011) 2021-05-21 18:15:48 -07:00
licenses-binary HADOOP-17666. Update LICENSE for 3.3.1 (#3011) 2021-05-21 18:15:48 -07:00
.gitattributes HADOOP-13598. Add eol=lf for unix format files in .gitattributes. Contributed by Yiqun Lin. 2016-09-14 11:14:31 +09:00
.gitignore YARN-10407. Add phantomjsdriver.log to gitignore. (#2244) 2021-02-17 10:28:17 +09:00
BUILDING.txt HADOOP-17926. Maven-eclipse-plugin is no longer needed since Eclipse can import Maven projects by itself. (#3465) 2021-09-22 13:08:30 +00:00
LICENSE-binary HADOOP-17940. Upgrade Kafka to 2.8.1 (#3488) 2021-09-28 13:31:53 +09:00
LICENSE-binary-yarn-applications-catalog-webapp HADOOP-17666. Update LICENSE for 3.3.1 (#3011) 2021-05-21 18:15:48 -07:00
LICENSE-binary-yarn-ui HADOOP-17666. Update LICENSE for 3.3.1 (#3011) 2021-05-21 18:15:48 -07:00
LICENSE.txt HADOOP-17666. Update LICENSE for 3.3.1 (#3011) 2021-05-21 18:15:48 -07:00
NOTICE-binary HADOOP-17666. Update LICENSE for 3.3.1 (#3011) 2021-05-21 18:15:48 -07:00
NOTICE.txt HADOOP-15958. Revisiting LICENSE and NOTICE files. 2019-08-27 13:47:12 +09:00
README.txt HADOOP-15958. Revisiting LICENSE and NOTICE files. 2019-08-27 13:47:12 +09:00
pom.xml HADOOP-17777. Update clover-maven-plugin version from 3.3.0 to 4.4.1 (#3152) 2021-06-30 09:35:36 +08:00
start-build-env.sh HADOOP-17298. Backslash in username causes build failure in the environment started by start-build-env.sh. (#2367) 2020-10-20 10:11:18 +09:00

README.txt 

For the latest information about Hadoop, please visit our website at:

   http://hadoop.apache.org/

and our wiki, at:

   https://cwiki.apache.org/confluence/display/HADOOP/