Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
20,375 Commits 326 Branches 371 Tags
Commit Graph

7540 Commits

Author SHA1 Message Date
Wei-Chiu Chuang
c48640ffbd HDFS-14171. Performance improvement in Tailing EditLog. Contributed by Kenneth Yang. 
(cherry picked from commit e9a005dfcf4d2249235aeff4da85b321dddc1ca9)
 2018-12-28 10:38:55 -08:00
Surendra Singh Lilhore
d2ec1ff020 HDFS-14046. In-Maintenance ICON is missing in datanode info page. Contributed by Ranith Sardar. 
(cherry picked from commit 686fcd4db34dfe8642ff4b25fffbc73e42217f30)
 2018-12-24 11:09:07 +05:30
Vinayakumar B
abe14d32d4 HDFS-14124. EC : Support EC Commands (set/get/unset EcPolicy) via WebHdfs. Contributed by Ayush Saxena. 
(cherry picked from commit 39dc7345b80e27ba8bd1ff4c19ca241aef5ac0fc)
 2018-12-11 18:10:51 +05:30
Tsz Wo Nicholas Sze
9d508f719b HDFS-14112. Avoid recursive call to external authorizer for getContentSummary. 2018-11-29 13:56:43 -08:00
Brahma Reddy Battula
e2fa9e8cdd HDFS-14095. EC: Track Erasure Coding commands in DFS statistics. Contributed by Ayush Saxena. 
(cherry picked from commit f534736867eed962899615ca1b7eb68bcf591d17)
 2018-11-30 00:28:04 +05:30
Vinayakumar B
538757ed49 HDFS-13816. dfs.getQuotaUsage() throws NPE on non-existent dir instead of FileNotFoundException. Contributed by Vinayakumar B. 
(cherry picked from commit b09828145432c8d986ac8f05ec33608d8e611328)
 2018-11-26 18:23:09 +05:30
Brahma Reddy Battula
7e10dd03b5 HDFS-14064. WEBHDFS: Support Enable/Disable EC Policy. Contributed by Ayush Saxena. 
(cherry picked from commit 15d8f592b51d08a658e88c6a7a7596e0edf5793b)
 2018-11-22 00:07:38 +05:30
Akira Ajisaka
f6227367fb
Add 2.9.2 release notes and changes documents. 
(cherry picked from commit 1a00b4e325146988375c9ce5b11016c45f059a4e)
 2018-11-20 14:25:48 +09:00
Vinayakumar B
7a2a806213 HDFS-14056. Fix error messages in HDFS-12716. Contributed by Ayush Saxena. 
(cherry picked from commit 886b935630bf2d46f4c6773de013c093d2e44c38)
 2018-11-16 18:01:43 +05:30
Vinayakumar B
71a6ec6058 HDFS-13963. NN UI is broken with IE11. Contributed by Ayush Saxena. 
(cherry picked from commit 8571507efa3af3e4a5213a7e24c65e26162eac4d)
 2018-11-16 17:29:25 +05:30
Xiao Chen
5afd7efe24 Revert "HDFS-13732. ECAdmin should print the policy name when an EC policy is set. Contributed by Zsolt Venczel." 
This reverts commit 7dc79a8b5b7af0bf37d25a221be8ed446b0edb74.
 2018-11-13 12:45:55 -08:00
Brahma Reddy Battula
5e86641265 HDFS-14070. Refactor NameNodeWebHdfsMethods to allow better extensibility. Contributed by CR Hota 
(cherry picked from commit e7b63baca1e10b28d8b4462fd80537b871951aa3)
 2018-11-13 13:09:03 +05:30
Arpit Agarwal
e6532f7eb2 HDFS-14065. Failed Storage Locations shows nothing in the Datanode Volume Failures. Contributed by Ayush Saxena. 
(cherry picked from commit b6d4e19f34f474ea8068ebb374f55e0db2f714da)
 2018-11-12 15:32:14 -08:00
Kitti Nanasi
57409874bb HDFS-14039. ec -listPolicies doesn't show correct state for the default policy when the default is not RS(6,3). Contributed by Kitti Nanasi. 
Signed-off-by: Xiao Chen <xiao@apache.org>
(cherry picked from commit 8d99648c203004045a9339ad27258092969145d6)
 2018-11-08 10:08:26 -08:00
Inigo Goiri
e7fba62e6a HDFS-14048. DFSOutputStream close() throws exception on subsequent call after DataNode restart. Contributed by Erik Krogen. 
(cherry picked from commit 887244de4adebe27693ed4ad3296a6f700cfa8c1)
 2018-11-06 11:20:06 -08:00
Inigo Goiri
378f189c4f HDFS-14043. Tolerate corrupted seen_txid file. Contributed by Lukas Majercak. 
(cherry picked from commit f3296501e09fa7f1e81548dfcefa56f20fe337ca)
 2018-11-05 16:49:12 -08:00
Giovanni Matteo Fumarola
6e1fad299a HDFS-14042. Fix NPE when PROVIDED storage is missing. Contributed by Virajith Jalaparti. 2018-11-05 11:33:32 -08:00
Xiao Chen
8788489df4 HDFS-14027. DFSStripedOutputStream should implement both hsync methods. 
(cherry picked from commit db7e636824a36b90ba1c8e9b2fba1162771700fe)
 2018-10-29 19:13:43 -07:00
Wei-Chiu Chuang
5f8dd40647 Revert "HADOOP-15864. Job submitter / executor fail when SBN domain name can not resolved. Contributed by He Xiaoqiao." 
This reverts commit fd02c501c0cea3ec55956e11b390111519cedc00.
 2018-10-28 17:05:45 -07:00
Anu Engineer
3cbb028dd2 HDFS-14028. HDFS OIV temporary dir deletes folder. 
Contributed by Adam Antal.

(cherry picked from commit 4f10d7e23fc0b757a9e94bc448187a6211b90f10)
 2018-10-26 16:47:53 -07:00
Arpit Agarwal
642b613a7d HDFS-14029. Sleep in TestLazyPersistFiles should be put into a loop. Contributed by Adam Antal. 
(cherry picked from commit 9b899f1ebd5126a756fceb43459164165488b203)
 2018-10-25 15:14:58 -07:00
Wei-Chiu Chuang
fd02c501c0 HADOOP-15864. Job submitter / executor fail when SBN domain name can not resolved. Contributed by He Xiaoqiao. 
(cherry picked from commit fb2b72e6fce019130e10964a644b94cddbab1c06)
 2018-10-25 09:36:28 -07:00
Inigo Goiri
5b995f85f5 HDFS-14003. Fix findbugs warning in trunk for FSImageFormatPBINode. Contributed by Yiqun Lin. 
(cherry picked from commit 0e56c883cd2310f3ff9d62afb306b1ab27419c36)
 2018-10-24 16:14:06 -07:00
Inigo Goiri
b6937e8e70 HDFS-14021. TestReconstructStripedBlocksWithRackAwareness#testReconstructForNotEnoughRacks fails intermittently. Contributed by Xiao Chen. 
(cherry picked from commit c1874046e2f9275ba330090fcf12c5611c6d6fc4)
 2018-10-24 09:38:32 -07:00
Arpit Agarwal
28a1cabb72 HDFS-13941. make storageId in BlockPoolTokenSecretManager.checkAccess optional. Contributed by Ajay Kumar. 
(cherry picked from commit c58811c77d5c0442c404a5b2876e09eaf6d16073)
 2018-10-22 14:45:11 -07:00
Inigo Goiri
c0f618f3d5 HDFS-14002. TestLayoutVersion#testNameNodeFeatureMinimumCompatibleLayoutVersions fails. Contributed by Takanobu Asanuma. 
(cherry picked from commit be1cffb0854cb28967beb062e9db7d61e6eeff1d)
 2018-10-18 17:14:56 -07:00
Sunil G
bde4fd5ed9 Preparing for 3.2.0 release 2018-10-18 17:07:45 +05:30
Xiao Chen
08b415d170 HDFS-13662. TestBlockReaderLocal#testStatisticsForErasureCodingRead is flaky 
(cherry picked from commit 533138718cc05b78e0afe583d7a9bd30e8a48fdc)
 2018-10-16 19:34:41 -07:00
Xiao Chen
ced2596094 HADOOP-14445. Addendum: Use DelegationTokenIssuer to create KMS delegation tokens that can authenticate to all KMS instances. 
(cherry picked from commit b6fc72a0250ac3f2341ebe8a14d19b073e6224c8)
 2018-10-15 11:00:21 -07:00
Vinayakumar B
8a41edb089 Fix potential FSImage corruption. Contributed by Daryn Sharp. 
(cherry picked from commit b60ca37914b22550e3630fa02742d40697decb31)
 2018-10-15 15:49:56 +05:30
Vinayakumar B
b185964b15 HDFS-13945. TestDataNodeVolumeFailure is Flaky. Contributed by Ayush Saxena. 
(cherry picked from commit 6e0e6daaf3215213c32cdffa79f3730d40e981ea)
 2018-10-12 17:34:28 +05:30
Vinayakumar B
d838d39a2c HDFS-13156. HDFS Block Placement Policy - Client Local Rack. Contributed by Ayush Saxena. 
(cherry picked from commit de44e1064f051248934ceffdd98a3cc13653d886)
 2018-10-12 17:28:48 +05:30
Hrishikesh Gadre
90a9837c9d HDFS-13956. iNotify should include information to identify a file as either replicated or erasure coded. Contributed by Hrishikesh Gadre. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit bf3d591f0cb0fedeab5d89cc8d2270d3b9a70313)
 2018-10-10 10:24:23 -07:00
Weiwei Yang
966d94ea2f HDFS-12459. Fix revert: Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 3ead525c71cba068e7abf1c76ad629bfeec10852)
 2018-10-10 10:14:57 -07:00
Inigo Goiri
bc400c8a33 HDFS-11396. TestNameNodeMetadataConsistency#testGenerationStampInFuture timed out. Contributed by Ayush Saxena. 2018-10-09 17:04:17 -07:00
Hrishikesh Gadre
a99658cd85 HDFS-13926. ThreadLocal aggregations for FileSystem.Statistics are incorrect with striped reads. 
Contributed by Xiao Chen, Hrishikesh Gadre.

Signed-off-by: Xiao Chen <xiao@apache.org>
(cherry picked from commit 08bb6c49a5aec32b7d9f29238560f947420405d6)
 2018-10-08 20:39:48 -07:00
Yiqun Lin
b6698e2a82 HDFS-13962. Add null check for add-replica pool to avoid lock acquiring. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit 1043795f7fe44c98a34f8ea3cea708c801c3043b)
 2018-10-09 10:35:08 +08:00
Sunil G
2c392da8aa HADOOP-15791. Remove Ozone related sources from the 3.2 branch. Contributed by Elek, Marton. 2018-10-05 16:10:47 +05:30
Yiqun Lin
62d02eecd0 HDFS-13957. Fix incorrect option used in description of InMemoryAliasMap. 
(cherry picked from commit 619e490333fa89601fd476dedac6d16610e9a52a)
 2018-10-05 10:09:22 +08:00
Yiqun Lin
5689355783 HDFS-13768. Adding replicas to volume map makes DataNode start slowly. Contributed by Surendra Singh Lilhore. 2018-10-02 09:43:14 +08:00
Takanobu Asanuma
e2113500df HDFS-8196. Post enabled Erasure Coding Policies on NameNode UI. Contributed by Kitti Nanasi and Kai Sasaki. 2018-09-29 00:53:35 +09:00
Akira Ajisaka
f9c0221623
HDFS-13932. [JDK 11] Casts to BlockStoragePolicy[] in unit tests raise ClassCastExceptions. Contributed by Andrew Purtell. 2018-09-27 11:44:42 +09:00
Brahma Reddy Battula
e0ff8e2c10 HDFS-13840. RBW Blocks which are having less GS should be added to Corrupt. Contributed by Brahma Reddy Battula 2018-09-26 23:44:16 +05:30
Inigo Goiri
6275b42870 HDFS-13927. Addendum: Improve TestDataNodeMultipleRegistrations#testDNWithInvalidStorageWithHA wait. Contributed by Ayush Saxena. 2018-09-26 09:50:09 -07:00
Inigo Goiri
4758b4b6da HDFS-13927. Improve TestDataNodeMultipleRegistrations#testDNWithInvalidStorageWithHA wait. Contributed by Ayush Saxena. 2018-09-21 15:32:28 -07:00
Anu Engineer
6fc293fece HDFS-13892. Disk Balancer: Make execute command documentation better. 
Contributed by  Ranith Sardar.
 2018-09-19 20:48:41 -07:00
Arpit Agarwal
2eb597b151 HADOOP-15772. Remove the 'Path ... should be specified as a URI' warnings on startup. Contributed by Ayush Saxena. 2018-09-19 13:48:27 -07:00
Chen Liang
a30b4f9e71 HADOOP-15726. Create utility to limit frequency of log statements. Contributed by Erik Krogen. 2018-09-19 13:22:37 -07:00
Inigo Goiri
090272d7de HADOOP-15684. triggerActiveLogRoll stuck on dead name node, when ConnectTimeoutException happens. Contributed by Rong Tang. 2018-09-19 13:00:30 -07:00
Inigo Goiri
56e0d635e0 HDFS-13908. TestDataNodeMultipleRegistrations is flaky. Contributed by Ayush Saxena. 2018-09-19 09:30:25 -07:00