Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
20,457 Commits 326 Branches 371 Tags
Commit Graph

7555 Commits

Author SHA1 Message Date
Surendra Singh Lilhore
f4b9ba230b HDFS-14263. Remove unnecessary block file exists check from FsDatasetImpl#getBlockInputStream(). Contributed by Surendra Singh Lilhore 
(cherry picked from commit 29b411d5f065f177de05a69d2822836209f813c6)
 2019-02-13 23:51:36 +05:30
Surendra Singh Lilhore
e163bf2258 HDFS-14274. EC: NPE While Listing EC Policy For A Directory Following Replication Policy. Contributed by Ayush Saxena. 
(cherry picked from commit 35d4f32b32a3ba05332811caf3d245d9c3dcf1a5)
 2019-02-13 23:21:37 +05:30
Vinayakumar B
890736828b HDFS-14266. EC : Fsck -blockId shows null for EC Blocks if One Block Is Not Available. Contributed by Ayush Saxena. 2019-02-12 21:58:59 +05:30
Surendra Singh Lilhore
2501fcd26b HDFS-14140. JournalNodeSyncer authentication is failing in secure cluster. Contributed by Surendra Singh Lilhore. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 4be87353e35a30d95d8847b09a1890b014bfc6bb)
 2019-02-07 16:52:53 -08:00
Kitti Nanasi
38541b2375 HDFS-14231. DataXceiver#run() should not log exceptions caused by InvalidToken exception as an error. Contributed by Kitti Nanasi. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 49ddd8a6ed5b40d12defb0771b4c8b53d4ffde3f)
 2019-02-05 15:15:04 -08:00
Wangda Tan
649da5af04 Make upstream aware of 3.1.2 release 
Change-Id: I397bc6ef75498726df4763bd07a8bf8fe1c38365
(cherry picked from commit 308f3168fa221a336ef0dbb1a17358b1ad6f9356)
 2019-02-05 14:05:51 -08:00
Steve Loughran
bdd17be9ec
HDFS-13713. Add specification of Multipart Upload API to FS specification, with contract tests. 
Contributed by Ewan Higgs and Steve Loughran.

(cherry picked from commit c1d24f848345f6d34a2ac2d570d49e9787a0df6a)
 2019-02-04 17:10:19 +00:00
Masatake Iwasaki
a86893f875 HDFS-12729. Document special paths in HDFS. Contributed by Masatake Iwasaki. 
(cherry picked from commit dc5af4c6b7bdf437713a8c6ca3c535c4b6d6992e)
 2019-01-29 02:28:18 +09:00
Dinesh Chitlangia
eb77e60163 HDFS-14228. Incorrect getSnapshottableDirListing() javadoc. Contributed by Dinesh Chitlangia. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 45caeee6cfcf1ae3355cd880402159cf31e94a8a)
 2019-01-25 07:37:43 -08:00
Takanobu Asanuma
e88db02753 HDFS-14218. EC: Ls -e throw NPE when directory ec policy is disabled. Contributed by Ayush Saxena. 
(cherry picked from commit 7d6792e5d2001e1a83ae75a4777324a3b01c557a)
 2019-01-23 10:04:50 +09:00
Arpit Agarwal
8dd8121f7d HDFS-14221. Replace Guava Optional with Java Optional. Contributed by Arpit Agarwal. 
(cherry picked from commit 1ff658b2ef3fb933897712c728bc628f3f44bded)
 2019-01-21 20:46:50 -08:00
Arpit Agarwal
cd5e7f152f HDFS-14222. Make ThrottledAsyncChecker constructor public. Contributed by Arpit Agarwal. 
(cherry picked from commit 6f0756fc0e43b785d3dee72a669997d934b57e4c)
 2019-01-21 20:46:50 -08:00
Weiwei Yang
2253a86a3f HDFS-14207. ZKFC should catch exception when ha configuration missing. Contributed by Fei Hui. 
(cherry picked from commit de34fc148ca4785c2ae7204df15096b955e671c4)
 2019-01-22 12:08:03 +08:00
Surendra Singh Lilhore
5d2d0e74d5 HDFS-14198. Upload and Create button doesn't get enabled after getting reset. Contributed by Ayush Saxena. 
(cherry picked from commit 9aeaaa0479ea6b4c4135214722c8c7c39fb17d75)
 2019-01-11 14:55:18 +05:30
Akira Ajisaka
6bc2781b39
HDFS-14183. [SPS] Remove the -w parameter from the -satisfystoragepolicy usage. Contributed by Ayush Saxena. 
(cherry picked from commit f30473f2ef10482976fa49730dd9a139417150dc)
 2018-12-31 03:17:27 +09:00
Wei-Chiu Chuang
c48640ffbd HDFS-14171. Performance improvement in Tailing EditLog. Contributed by Kenneth Yang. 
(cherry picked from commit e9a005dfcf4d2249235aeff4da85b321dddc1ca9)
 2018-12-28 10:38:55 -08:00
Surendra Singh Lilhore
d2ec1ff020 HDFS-14046. In-Maintenance ICON is missing in datanode info page. Contributed by Ranith Sardar. 
(cherry picked from commit 686fcd4db34dfe8642ff4b25fffbc73e42217f30)
 2018-12-24 11:09:07 +05:30
Vinayakumar B
abe14d32d4 HDFS-14124. EC : Support EC Commands (set/get/unset EcPolicy) via WebHdfs. Contributed by Ayush Saxena. 
(cherry picked from commit 39dc7345b80e27ba8bd1ff4c19ca241aef5ac0fc)
 2018-12-11 18:10:51 +05:30
Tsz Wo Nicholas Sze
9d508f719b HDFS-14112. Avoid recursive call to external authorizer for getContentSummary. 2018-11-29 13:56:43 -08:00
Brahma Reddy Battula
e2fa9e8cdd HDFS-14095. EC: Track Erasure Coding commands in DFS statistics. Contributed by Ayush Saxena. 
(cherry picked from commit f534736867eed962899615ca1b7eb68bcf591d17)
 2018-11-30 00:28:04 +05:30
Vinayakumar B
538757ed49 HDFS-13816. dfs.getQuotaUsage() throws NPE on non-existent dir instead of FileNotFoundException. Contributed by Vinayakumar B. 
(cherry picked from commit b09828145432c8d986ac8f05ec33608d8e611328)
 2018-11-26 18:23:09 +05:30
Brahma Reddy Battula
7e10dd03b5 HDFS-14064. WEBHDFS: Support Enable/Disable EC Policy. Contributed by Ayush Saxena. 
(cherry picked from commit 15d8f592b51d08a658e88c6a7a7596e0edf5793b)
 2018-11-22 00:07:38 +05:30
Akira Ajisaka
f6227367fb
Add 2.9.2 release notes and changes documents. 
(cherry picked from commit 1a00b4e325146988375c9ce5b11016c45f059a4e)
 2018-11-20 14:25:48 +09:00
Vinayakumar B
7a2a806213 HDFS-14056. Fix error messages in HDFS-12716. Contributed by Ayush Saxena. 
(cherry picked from commit 886b935630bf2d46f4c6773de013c093d2e44c38)
 2018-11-16 18:01:43 +05:30
Vinayakumar B
71a6ec6058 HDFS-13963. NN UI is broken with IE11. Contributed by Ayush Saxena. 
(cherry picked from commit 8571507efa3af3e4a5213a7e24c65e26162eac4d)
 2018-11-16 17:29:25 +05:30
Xiao Chen
5afd7efe24 Revert "HDFS-13732. ECAdmin should print the policy name when an EC policy is set. Contributed by Zsolt Venczel." 
This reverts commit 7dc79a8b5b7af0bf37d25a221be8ed446b0edb74.
 2018-11-13 12:45:55 -08:00
Brahma Reddy Battula
5e86641265 HDFS-14070. Refactor NameNodeWebHdfsMethods to allow better extensibility. Contributed by CR Hota 
(cherry picked from commit e7b63baca1e10b28d8b4462fd80537b871951aa3)
 2018-11-13 13:09:03 +05:30
Arpit Agarwal
e6532f7eb2 HDFS-14065. Failed Storage Locations shows nothing in the Datanode Volume Failures. Contributed by Ayush Saxena. 
(cherry picked from commit b6d4e19f34f474ea8068ebb374f55e0db2f714da)
 2018-11-12 15:32:14 -08:00
Kitti Nanasi
57409874bb HDFS-14039. ec -listPolicies doesn't show correct state for the default policy when the default is not RS(6,3). Contributed by Kitti Nanasi. 
Signed-off-by: Xiao Chen <xiao@apache.org>
(cherry picked from commit 8d99648c203004045a9339ad27258092969145d6)
 2018-11-08 10:08:26 -08:00
Inigo Goiri
e7fba62e6a HDFS-14048. DFSOutputStream close() throws exception on subsequent call after DataNode restart. Contributed by Erik Krogen. 
(cherry picked from commit 887244de4adebe27693ed4ad3296a6f700cfa8c1)
 2018-11-06 11:20:06 -08:00
Inigo Goiri
378f189c4f HDFS-14043. Tolerate corrupted seen_txid file. Contributed by Lukas Majercak. 
(cherry picked from commit f3296501e09fa7f1e81548dfcefa56f20fe337ca)
 2018-11-05 16:49:12 -08:00
Giovanni Matteo Fumarola
6e1fad299a HDFS-14042. Fix NPE when PROVIDED storage is missing. Contributed by Virajith Jalaparti. 2018-11-05 11:33:32 -08:00
Xiao Chen
8788489df4 HDFS-14027. DFSStripedOutputStream should implement both hsync methods. 
(cherry picked from commit db7e636824a36b90ba1c8e9b2fba1162771700fe)
 2018-10-29 19:13:43 -07:00
Wei-Chiu Chuang
5f8dd40647 Revert "HADOOP-15864. Job submitter / executor fail when SBN domain name can not resolved. Contributed by He Xiaoqiao." 
This reverts commit fd02c501c0cea3ec55956e11b390111519cedc00.
 2018-10-28 17:05:45 -07:00
Anu Engineer
3cbb028dd2 HDFS-14028. HDFS OIV temporary dir deletes folder. 
Contributed by Adam Antal.

(cherry picked from commit 4f10d7e23fc0b757a9e94bc448187a6211b90f10)
 2018-10-26 16:47:53 -07:00
Arpit Agarwal
642b613a7d HDFS-14029. Sleep in TestLazyPersistFiles should be put into a loop. Contributed by Adam Antal. 
(cherry picked from commit 9b899f1ebd5126a756fceb43459164165488b203)
 2018-10-25 15:14:58 -07:00
Wei-Chiu Chuang
fd02c501c0 HADOOP-15864. Job submitter / executor fail when SBN domain name can not resolved. Contributed by He Xiaoqiao. 
(cherry picked from commit fb2b72e6fce019130e10964a644b94cddbab1c06)
 2018-10-25 09:36:28 -07:00
Inigo Goiri
5b995f85f5 HDFS-14003. Fix findbugs warning in trunk for FSImageFormatPBINode. Contributed by Yiqun Lin. 
(cherry picked from commit 0e56c883cd2310f3ff9d62afb306b1ab27419c36)
 2018-10-24 16:14:06 -07:00
Inigo Goiri
b6937e8e70 HDFS-14021. TestReconstructStripedBlocksWithRackAwareness#testReconstructForNotEnoughRacks fails intermittently. Contributed by Xiao Chen. 
(cherry picked from commit c1874046e2f9275ba330090fcf12c5611c6d6fc4)
 2018-10-24 09:38:32 -07:00
Arpit Agarwal
28a1cabb72 HDFS-13941. make storageId in BlockPoolTokenSecretManager.checkAccess optional. Contributed by Ajay Kumar. 
(cherry picked from commit c58811c77d5c0442c404a5b2876e09eaf6d16073)
 2018-10-22 14:45:11 -07:00
Inigo Goiri
c0f618f3d5 HDFS-14002. TestLayoutVersion#testNameNodeFeatureMinimumCompatibleLayoutVersions fails. Contributed by Takanobu Asanuma. 
(cherry picked from commit be1cffb0854cb28967beb062e9db7d61e6eeff1d)
 2018-10-18 17:14:56 -07:00
Sunil G
bde4fd5ed9 Preparing for 3.2.0 release 2018-10-18 17:07:45 +05:30
Xiao Chen
08b415d170 HDFS-13662. TestBlockReaderLocal#testStatisticsForErasureCodingRead is flaky 
(cherry picked from commit 533138718cc05b78e0afe583d7a9bd30e8a48fdc)
 2018-10-16 19:34:41 -07:00
Xiao Chen
ced2596094 HADOOP-14445. Addendum: Use DelegationTokenIssuer to create KMS delegation tokens that can authenticate to all KMS instances. 
(cherry picked from commit b6fc72a0250ac3f2341ebe8a14d19b073e6224c8)
 2018-10-15 11:00:21 -07:00
Vinayakumar B
8a41edb089 Fix potential FSImage corruption. Contributed by Daryn Sharp. 
(cherry picked from commit b60ca37914b22550e3630fa02742d40697decb31)
 2018-10-15 15:49:56 +05:30
Vinayakumar B
b185964b15 HDFS-13945. TestDataNodeVolumeFailure is Flaky. Contributed by Ayush Saxena. 
(cherry picked from commit 6e0e6daaf3215213c32cdffa79f3730d40e981ea)
 2018-10-12 17:34:28 +05:30
Vinayakumar B
d838d39a2c HDFS-13156. HDFS Block Placement Policy - Client Local Rack. Contributed by Ayush Saxena. 
(cherry picked from commit de44e1064f051248934ceffdd98a3cc13653d886)
 2018-10-12 17:28:48 +05:30
Hrishikesh Gadre
90a9837c9d HDFS-13956. iNotify should include information to identify a file as either replicated or erasure coded. Contributed by Hrishikesh Gadre. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit bf3d591f0cb0fedeab5d89cc8d2270d3b9a70313)
 2018-10-10 10:24:23 -07:00
Weiwei Yang
966d94ea2f HDFS-12459. Fix revert: Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 3ead525c71cba068e7abf1c76ad629bfeec10852)
 2018-10-10 10:14:57 -07:00
Inigo Goiri
bc400c8a33 HDFS-11396. TestNameNodeMetadataConsistency#testGenerationStampInFuture timed out. Contributed by Ayush Saxena. 2018-10-09 17:04:17 -07:00