Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
20,248 Commits 326 Branches 371 Tags
Commit Graph

20248 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Sunil G
fa48363e28 Revert "HDFS-13732. ECAdmin should print the policy name when an EC policy is set. Contributed by Zsolt Venczel." 
This reverts commit 7dc79a8b5b7af0bf37d25a221be8ed446b0edb74.
release-3.2.0-RC0 		2018-11-21 12:19:53 +05:30
Haibo Chen
c5203b0449 YARN-8990. Fix fair scheduler race condition in app submit and queue cleanup. (Contributed by Wilfred Spiegelenburg) 
(cherry picked from commit 524a7523c427b55273133078898ae3535897bada)
 2018-11-09 14:05:33 +05:30
Weiwei Yang
68e514dc66 YARN-8988. Reduce the verbose log on RM heartbeat path when distributed node-attributes is enabled. Contributed by Tao Yang. 
(cherry picked from commit e1bbf7dcdfc30a61a2b10bef09c59ff17d290488)
 2018-11-09 14:05:05 +05:30
Eric Yang
ded047b06f YARN-8687. Update YARN service file type in documentation. 
Contributed by Suma Shivaprasad

(cherry picked from commit ba7e81667ce12d5cf9d87ee18a8627323759cee0)
 2018-11-05 23:26:18 +05:30
Eric Yang
b990d5943a YARN-8938. Updated YARN service upgrade document. 
Contributed by Chandni Singh

(cherry picked from commit bbc6dcd3d0976932a49d8650804fb0a4018b3a02)
 2018-11-05 23:16:36 +05:30
Sunil G
83e5ebd940 HDFS-14033. [libhdfs++] Disable libhdfs++ build on systems that do not support thread_local. Contributed by Anatoli Shein. 
(cherry picked from commit 9c438abe52d4ee0b25345a4b7ec1697dd66f85e9)
 2018-10-31 12:34:20 +05:30
Rohith Sharma K S
3d853fc571 YARN-8950. Fix compilation issue due to dependency convergence error for hbase.profile=2.0. 
(cherry picked from commit 4ec4ec69711180d642c5b56cd3d3dbdf44d3c61f)
 2018-10-30 12:11:39 +05:30
Ted Yu
a73860c89d HADOOP-15850. CopyCommitter#concatFileChunks should check that the blocks per chunk is not 0. Contributed by Ted Yu. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit e2cecb681e2aab8b7c5465719cac53dce407a64c)
(cherry picked from commit a7dd244a4995cbf20feb054530fc5913ccf384ac)
 2018-10-27 12:18:32 -07:00
Sunil G
9977741497 HADOOP-15815. Upgrade Eclipse Jetty version to 9.3.24. Cotributed by Boris Vulikh. 
(cherry picked from commit e28c00c290a06813d156a845b93e806f71413dbf)
 2018-10-26 13:47:19 +05:30
Bharat Viswanadham
a28aad3c1c HADOOP-15882. Upgrade maven-shade-plugin from 2.4.3 to 3.2.0. Contributed by Takanobu Asanuma. 
(cherry picked from commit 34b2521f5e7388e6a4d76af537ef147206fd9b72)
 2018-10-26 09:04:02 +05:30
Akira Ajisaka
e213563c3c
HADOOP-15822. zstd compressor can fail with a small output buffer. Contributed by Jason Lowe. 
(cherry picked from commit 8f97d6f2cdfccefba5457ae3d561e9ce0109da3f)
(cherry picked from commit 63396beab4178f3855be27c869bfb78336c71166)
 2018-10-24 13:50:57 +09:00
Steve Loughran
7031fe17e4 HADOOP-15866. Renamed HADOOP_SECURITY_GROUP_SHELL_COMMAND_TIMEOUT keys break compatibility. 
Contributed by Wei-Chiu Chuang.

(cherry picked from commit 52cb766ad0bbea378a529fcff3cf372cf0a33cde)
 2018-10-22 15:53:19 +05:30
Sunil G
ece45559ca Preparing for 3.2.0 release 2018-10-19 14:32:23 +05:30
Giovanni Matteo Fumarola
aca6b715ff HDFS-14005. RBF: Web UI update to bootstrap-3.3.7. Contributed by Inigo Goiri. 2018-10-18 17:37:59 -07:00
Giovanni Matteo Fumarola
38895fcef1 HDFS-14007. Incompatible layout when generating FSImage. Contributed by Inigo Goiri. 2018-10-18 17:32:48 -07:00
Inigo Goiri
a7d813049c HDFS-14002. TestLayoutVersion#testNameNodeFeatureMinimumCompatibleLayoutVersions fails. Contributed by Takanobu Asanuma. 
(cherry picked from commit be1cffb0854cb28967beb062e9db7d61e6eeff1d)
 2018-10-18 17:15:28 -07:00
Inigo Goiri
30b65ea18f HDFS-14000. RBF: Documentation should reflect right scripts for v3.0 and above. Contributed by CR Hota. 
(cherry picked from commit 7347fa2df3cfd48857f4ded3539a60101d026daf)
 2018-10-17 16:41:43 -07:00
Jason Lowe
30fc5966a2 HADOOP-15859. ZStandardDecompressor.c mistakes a class for an instance. Contributed by Jason Lowe 
(cherry picked from commit 9abda83947a5babfe5a650b3409ad952f6782105)
 2018-10-17 14:40:57 -05:00
Sunil G
6380ee5512 YARN-8759. Copy of resource-types.xml is not deleted if test fails, causes other test failures. Contributed by Antal Bálint Steinbach. 
(cherry picked from commit 5085e5fa9e8c7e489a8518e8541c12b14f3651df)
 2018-10-17 16:06:07 +05:30
Steve Loughran
88ea877ac0
HADOOP-15861. Move DelegationTokenIssuer to the right path. 
Contributed by Wei-Chiu Chuang.

(cherry picked from commit 41b3603b5bcb74a7d78a314a4a5c177d941af27f)
 2018-10-17 11:02:47 +01:00
Xiao Chen
08b415d170 HDFS-13662. TestBlockReaderLocal#testStatisticsForErasureCodingRead is flaky 
(cherry picked from commit 533138718cc05b78e0afe583d7a9bd30e8a48fdc)
 2018-10-16 19:34:41 -07:00
Akira Ajisaka
83769b63d4
HADOOP-15802. start-build-env.sh creates an invalid /etc/sudoers.d/hadoop-build-${USER_ID} file entry. Contributed by Jon Boone. 
(cherry picked from commit e3342a1abaff71823ebd952baf24a6143e711b99)
 2018-10-17 10:14:30 +09:00
Eric Yang
fc25def7d7 YARN-8810. Fixed a YARN service bug in comparing ConfigFile object. 
Contributed by Chandni Singh
 2018-10-16 18:57:50 -04:00
Wangda Tan
80ee60c88c YARN-8892. YARN UI2 doc changes to update security status (verified under security environment). (Sunil G via wangda) 
Change-Id: I8bc8622936861b8d6de3e42a0b75af86ad8a3961
(cherry picked from commit 538250db26ce0b261bb74053348cddfc2d65cf52)
 2018-10-16 13:53:08 -07:00
Wangda Tan
aa23c8d739 YARN-8798. [Submarine] Job should not be submitted if --input_path option is missing. (Zhankun Tang via wangda) 
Change-Id: I7ae0e44eb5179b04a6ac861ec1c65f3b18c38f0f
(cherry picked from commit 143d74775b2b62884090fdd88874134b9eab2888)
 2018-10-16 13:53:03 -07:00
Wangda Tan
920401b296 YARN-8870. [Submarine] Add submarine installation scripts. (Xun Liu via wangda) 
Change-Id: I46e8d9fd32c7745c313030da62da41486a77b3ea
(cherry picked from commit 46d6e0016610ced51a76189daeb3ad0e3dbbf94c)
 2018-10-16 13:52:58 -07:00
Wangda Tan
8610c20724 YARN-8875. [Submarine] Add documentation for submarine installation script details. (Xun Liu via wangda) 
Change-Id: I1c8d39c394e5a30f967ea514919835b951f2c124
(cherry picked from commit ed08dd3b0c9cec20373e8ca4e34d6526bd759943)
 2018-10-16 13:52:53 -07:00
Steve Loughran
c19864d8c8
HADOOP-15848. ITestS3AContractMultipartUploader#testMultipartUploadEmptyPart test error. 
Contributed by Ewan Higgs.
 2018-10-16 19:58:58 +01:00
Xiao Chen
424c99681b MAPREDUCE-7132. JobSplitWriter prints unnecessary warnings if EC(RS10,4) is used. Contributed by Peter Bacsko. 
(cherry picked from commit 25f8fcb06476938826cdc92858a61124b18cd98d)
 2018-10-16 10:24:15 -07:00
Sunil G
a439aa79dc YARN-8879. Kerberos principal is needed when submitting a submarine job. Contributed by Zac Zhou. 
(cherry picked from commit 753f149fd3f5acf9a98cfc780d7899e307c19002)
 2018-10-16 22:18:30 +05:30
Márton Elek
7b06383163 HADOOP-15857. Addendum. Remove ozonefs class name definition from core-default.xml. Contributed by Elek, Marton. 2018-10-16 15:51:59 +02:00
Sunil G
c5bf07154f HADOOP-15857. Remove ozonefs class name definition from core-default.xml. Contributed by Elek, Marton. 2018-10-16 18:56:49 +05:30
Xiao Chen
ced2596094 HADOOP-14445. Addendum: Use DelegationTokenIssuer to create KMS delegation tokens that can authenticate to all KMS instances. 
(cherry picked from commit b6fc72a0250ac3f2341ebe8a14d19b073e6224c8)
 2018-10-15 11:00:21 -07:00
Sunil G
1ca5f974f1 YARN-8869. YARN Service Client might not work correctly with RM REST API for Kerberos authentication. Contributed by Eric Yang. 
(cherry picked from commit fa94d370b6e3cba9c7560c09b517583e6652f103)
 2018-10-15 21:22:45 +05:30
bibinchundatt
aff21d7271 YARN-8830. SLS tool fix node addition. Contributed by Bibin A Chundatt. 
(cherry picked from commit b4a38e7b3e530756fb79d23dd4e218beeb5e3190)
 2018-10-15 18:11:01 +05:30
Vinayakumar B
8a41edb089 Fix potential FSImage corruption. Contributed by Daryn Sharp. 
(cherry picked from commit b60ca37914b22550e3630fa02742d40697decb31)
 2018-10-15 15:49:56 +05:30
Xiao Chen
9cb0654fbb HADOOP-14445. Use DelegationTokenIssuer to create KMS delegation tokens that can authenticate to all KMS instances. 
Contributed by Daryn Sharp, Xiao Chen, Rushabh S Shah.

(cherry picked from commit 5ec86b445cc492f52c33639efb6a09a0d2f27475)
 2018-10-12 12:08:08 -07:00
Inigo Goiri
65c1469b1a HDFS-13802. RBF: Remove FSCK from Router Web UI. Contributed by Fei Hui. 
(cherry picked from commit f63ee083db7c261207b726f0f44be2e5a92124f3)
 2018-10-12 10:10:52 -07:00
Vinayakumar B
b185964b15 HDFS-13945. TestDataNodeVolumeFailure is Flaky. Contributed by Ayush Saxena. 
(cherry picked from commit 6e0e6daaf3215213c32cdffa79f3730d40e981ea)
 2018-10-12 17:34:28 +05:30
Vinayakumar B
d838d39a2c HDFS-13156. HDFS Block Placement Policy - Client Local Rack. Contributed by Ayush Saxena. 
(cherry picked from commit de44e1064f051248934ceffdd98a3cc13653d886)
 2018-10-12 17:28:48 +05:30
Vrushali C
62d329cac0 YARN-5742 Serve aggregated logs of historical apps from timeline service. Contributed by Rohith Sharma KS 
(cherry picked from commit 8d1981806feb8278966c02a9eff42d72541bb35e)
 2018-10-12 07:18:13 +05:30
Siyao Meng
551e911493 HDFS-13878. HttpFS: Implement GETSNAPSHOTTABLEDIRECTORYLIST. Contributed by Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 6dcfef79afe97eda6d09fe2567bb4d4074223141)
 2018-10-11 15:04:02 -07:00
Siyao Meng
15e0ba3fb4 HDFS-13877. HttpFS: Implement GETSNAPSHOTDIFF. Contributed by Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 396ce0d9f470a5e8af03987ad6396d0f08b3d225)
 2018-10-11 15:03:55 -07:00
Jason Lowe
cdbca8b133 YARN-8861. executorLock is misleading in ContainerLaunch. Contributed by Chandni Singh 
(cherry picked from commit e787d65a08f5d5245d2313fc34f2dde518bfaa5b)
 2018-10-11 10:58:48 -05:00
Steve Loughran
70d39c74d9
HADOOP-15837. DynamoDB table Update can fail S3A FS init. 
Contributed by Steve Loughran.

(cherry picked from commit ee816f1fd78b029b9efa567e8b1b62391563de14)
 2018-10-11 14:58:37 +01:00
Sunil G
f29a7fdb14 YARN-8753. [UI2] Lost nodes representation missing from Nodemanagers Chart. Contributed by Yesha Vora. 
(cherry picked from commit 8a379839909197690af170afc13c49ef7aa54286)
 2018-10-11 16:16:05 +05:30
Sunil G
0bf0fb5499 YARN-8666. [UI2] Remove application tab from YARN Queue Page. Contributed by Yesha Vora. 
(cherry picked from commit d3da741fa27c14f3d1391a76d6ade5d1a8365aa5)
 2018-10-11 16:07:08 +05:30
Hrishikesh Gadre
90a9837c9d HDFS-13956. iNotify should include information to identify a file as either replicated or erasure coded. Contributed by Hrishikesh Gadre. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit bf3d591f0cb0fedeab5d89cc8d2270d3b9a70313)
 2018-10-10 10:24:23 -07:00
Weiwei Yang
966d94ea2f HDFS-12459. Fix revert: Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 3ead525c71cba068e7abf1c76ad629bfeec10852)
 2018-10-10 10:14:57 -07:00
Steve Loughran
1ee09906ce
HADOOP-15839. Review + update cloud store sensitive keys in hadoop.security.sensitive-config-keys. 
Contributed by Steve Loughran.

(cherry picked from commit cdc4350718055189fef8c70e31314607001d4009)
 2018-10-10 17:02:00 +01:00