Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
19,188 Commits 326 Branches 371 Tags
Commit Graph

7322 Commits

Author SHA1 Message Date
Erik Krogen
9295e95174 HDFS-14500. NameNode StartupProgress should not allow new steps in an already-completed phase. Contributed by Erik Krogen. 
(cherry-picked from 55e0c134f002f74cb4a0360b6682a1b6796d1598)
(cherry-picked from c26cf22d6b3949a4798d8133669b451660a0f050)
 2019-05-24 13:19:39 -07:00
Konstantin V Shvachko
799ca572f7 HDFS-14502. keepResults option in NNThroughputBenchmark should call saveNamespace(). Contributed by Konstantin V Shvachko. 2019-05-24 12:35:32 -07:00
Siyao Meng
9196c84c9f HDFS-14507. Document -blockingDecommission option for hdfs dfsadmin -listOpenFiles 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
(cherry picked from commit 9dff6eff819a832a5918d786675f42aa243a13e8)
 2019-05-22 13:25:42 +09:00
Srinivasu Majeti
e599e3b546 HDFS-14323. Distcp fails in Hadoop 3.x when 2.x source webhdfs url has special characters in hdfs file path. Contributed by Srinivasu Majeti. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 3e5e5b028ad7e199d08e524fe7cddeee5db51a6d)
(cherry picked from commit 85f3664bb84f393f8006d3ab781d8ce5ebb1ab62)
 2019-05-17 19:22:03 +02:00
Siyao Meng
86fb5e1c8e HDFS-14463. Add Log Level link under NameNode and DataNode Web UI Utilities dropdown. Contributed by Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 7cb46f035a92056783bad23a9abc6a264d71285d)
(cherry picked from commit 0550aa32f1a4b1809c234f8ed785db08b2e33d13)
 2019-05-01 16:49:36 -07:00
Arpit Agarwal
7b51edbab9
HDFS-13677. Dynamic refresh Disk configuration results in overwriting VolumeMap. Contributed by xuzq. 
(cherry picked from commit 4b4200f1f87ad40d9c19ba160f706ffd0470a8d4)
 2019-04-29 14:54:11 -07:00
Puleya7
546ed4937c HDFS-14432. dfs.datanode.shared.file.descriptor.paths duplicated in hdfs-default.xml 
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit 13907d847986aa14a51cc2a3cc83151f22c43781)
 2019-04-17 21:57:06 +09:00
Akira Ajisaka
a70076b3a5
HDFS-14407. Fix misuse of SLF4j logging API in DatasetVolumeChecker#checkAllVolumes. Contributed by Wanqiang Ji. 
(cherry picked from commit 67020f09502a4f07342dee457e47bb52b03441ae)
 2019-04-05 12:31:14 +09:00
Stephen O'Donnell
d9899015eb HDFS-14389. getAclStatus returns incorrect permissions and owner when an iNodeAttributeProvider is configured. Contributed by Stephen O'Donnell. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit c528e427aa6745434672b1c1850738795ad1d6d2)
(cherry picked from commit 388f445dde577999b2d81f809adcfca8f0958499)
 2019-04-04 04:06:33 -07:00
Wei-Chiu Chuang
084fb9de22 HDFS-10477. Stop decommission a rack of DataNodes caused NameNode fail over to standby. Contributed by yunjiong zhao and Wei-Chiu Chuang. 
(cherry picked from commit be488b6070a124234c77f16193ee925d32ca9a20)
(cherry picked from commit c8703dda0727e17d759d7ad27f0caee88103a530)
 2019-04-03 12:41:07 -07:00
Gabor Bota
f01d592f3b
HADOOP-16220. Add findbugs ignores for unjustified issues during update to guava to 27.0-jre in hadoop-project 
This closes #665

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 53a86e2b8ecb83b666d4ed223fc270e1a46642c1)
 2019-04-01 13:52:44 +09:00
Takanobu Asanuma
1fc74178eb HDFS-14037. Fix SSLFactory truststore reloader thread leak in URLConnectionFactory. 
(cherry picked from commit 55fb3c32fb48ca26a629d4d5f3f07e2858d09594)
 2019-03-27 03:29:42 +09:00
Inigo Goiri
80732146c6 HDFS-14366. Improve HDFS append performance. Contributed by Chao Sun. 
(cherry picked from commit ff06ef0631cb8a0f67bbc39b5b5a1b0a81ca3b3c)
 2019-03-15 13:58:39 -07:00
Erik Krogen
cb76fc5618 HDFS-14346. Add better time precision to Configuration#getTimeDuration, allowing return unit and default unit to be specified independently. Contributed by Chao Sun. 
(cherry picked from commit 66357574ae1da09ced735da36bf7d80a40c3fa1b)
(cherry picked from commit fec7c5f3ebbaea7b290e0904570c3a485b541a22)
 2019-03-13 13:20:51 -07:00
Shweta Yakkali
a324db2cce HDFS-14081. hdfs dfsadmin -metasave metasave_test results NPE. Contributed by Shweta Yakkali. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 1bea785020a538115b3e08f41ff88167033d2775)
(cherry picked from commit 1ceefa726e1b531ec92c2ee2212b25c327644ef6)
 2019-03-12 16:06:16 -07:00
Stephen O'Donnell
327e57ae6e HDFS-14333. Datanode fails to start if any disk has errors during Namenode registration. Contributed by Stephen O'Donnell. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 34b14061b38dccab25058dff1b8743d8a3f82734)
(cherry picked from commit a21e2e4dbc1681175130242138ced0d36b6fafae)
 2019-03-12 10:19:37 -07:00
Erik Krogen
b0b8000240 HDFS-14317. Ensure checkpoints are created when in-progress edit log tailing is enabled with a period shorter than the log roll period. Contributed by Ekanth Sethuramalingam. 
(cherry-picked from commit 1bc282e0b3f74968c92751f0972746b012e72810)
(cherry-picked from commit 6d076dd5e8be8fee5062a04d4cd2aa0c5da0ea56)
 2019-03-07 08:43:39 -08:00
Wei-Chiu Chuang
d951497f57 HDFS-14314. fullBlockReportLeaseId should be reset after registering to NN. Contributed by star. 
(cherry picked from commit 387dbe587aa66ac99ec5f5b50827ec3e0a327613)
(cherry picked from commit e58ccca3ce131c955ceb115cd0b75e452eea828b)
 2019-03-04 10:46:00 -08:00
Erik Krogen
b7eb9116eb HDFS-14305. Fix serial number calculation in BlockTokenSecretManager to avoid token key ID overlap between NameNodes. Contributed by He Xiaoqiao. 2019-03-01 08:12:54 -08:00
Surendra Singh Lilhore
bd9e2826fe HDFS-14263. Remove unnecessary block file exists check from FsDatasetImpl#getBlockInputStream(). Contributed by Surendra Singh Lilhore 
(cherry picked from commit 29b411d5f065f177de05a69d2822836209f813c6)
 2019-02-13 23:56:34 +05:30
Surendra Singh Lilhore
3bcdff8fac HDFS-14274. EC: NPE While Listing EC Policy For A Directory Following Replication Policy. Contributed by Ayush Saxena. 
(cherry picked from commit 35d4f32b32a3ba05332811caf3d245d9c3dcf1a5)
 2019-02-13 23:26:14 +05:30
Vinayakumar B
1f3e737407 HDFS-14266. EC : Fsck -blockId shows null for EC Blocks if One Block Is Not Available. Contributed by Ayush Saxena. 2019-02-12 22:35:55 +05:30
Surendra Singh Lilhore
1f4be45ef6 HDFS-14140. JournalNodeSyncer authentication is failing in secure cluster. Contributed by Surendra Singh Lilhore. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 4be87353e35a30d95d8847b09a1890b014bfc6bb)
(cherry picked from commit 2501fcd26bd7bef2738a8f6660dc63862c755ce3)
 2019-02-07 16:53:12 -08:00
Kitti Nanasi
c25b641cec HDFS-14231. DataXceiver#run() should not log exceptions caused by InvalidToken exception as an error. Contributed by Kitti Nanasi. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 49ddd8a6ed5b40d12defb0771b4c8b53d4ffde3f)
(cherry picked from commit 38541b23753c5d0e2cb9baf1d2da536f846dc38c)
 2019-02-05 15:15:31 -08:00
Wangda Tan
a1e09b4c0c Make upstream aware of 3.1.2 release 
Change-Id: I397bc6ef75498726df4763bd07a8bf8fe1c38365
(cherry picked from commit 308f3168fa221a336ef0dbb1a17358b1ad6f9356)
(cherry picked from commit 649da5af04d57ee910dbe7fb0f4a7d9cb4c40eb0)
 2019-02-05 14:09:33 -08:00
Masatake Iwasaki
dd3e7fc0c2 HDFS-12729. Document special paths in HDFS. Contributed by Masatake Iwasaki. 
(cherry picked from commit dc5af4c6b7bdf437713a8c6ca3c535c4b6d6992e)
 2019-01-29 02:29:39 +09:00
Dinesh Chitlangia
b63941516f HDFS-14228. Incorrect getSnapshottableDirListing() javadoc. Contributed by Dinesh Chitlangia. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 45caeee6cfcf1ae3355cd880402159cf31e94a8a)
(cherry picked from commit eb77e60163e318ca42548e134fe79dba7289fdc6)
 2019-01-25 07:38:11 -08:00
Takanobu Asanuma
38e4bed224 HDFS-14218. EC: Ls -e throw NPE when directory ec policy is disabled. Contributed by Ayush Saxena. 
(cherry picked from commit 7d6792e5d2001e1a83ae75a4777324a3b01c557a)
 2019-01-23 10:05:39 +09:00
Arpit Agarwal
f5bfeac62e HDFS-14221. Replace Guava Optional with Java Optional. Contributed by Arpit Agarwal. 
(cherry picked from commit 1ff658b2ef3fb933897712c728bc628f3f44bded)
 2019-01-21 20:48:05 -08:00
Arpit Agarwal
78b45977b1 HDFS-14222. Make ThrottledAsyncChecker constructor public. Contributed by Arpit Agarwal. 
(cherry picked from commit 6f0756fc0e43b785d3dee72a669997d934b57e4c)
 2019-01-21 20:47:02 -08:00
Weiwei Yang
cf520b7d64 HDFS-14207. ZKFC should catch exception when ha configuration missing. Contributed by Fei Hui. 
(cherry picked from commit de34fc148ca4785c2ae7204df15096b955e671c4)
 2019-01-22 12:17:33 +08:00
Surendra Singh Lilhore
6c846cfbf2 HDFS-14198. Upload and Create button doesn't get enabled after getting reset. Contributed by Ayush Saxena. 
(cherry picked from commit 9aeaaa0479ea6b4c4135214722c8c7c39fb17d75)
 2019-01-11 14:59:17 +05:30
Wangda Tan
31ea2f7806 Preparing for 3.1.3 development 
Change-Id: I3c3d3ee47dc4fef239127b4452ff14676fa26e3d
 2019-01-07 10:04:58 -08:00
Wei-Chiu Chuang
f24684ca9c HDFS-14171. Performance improvement in Tailing EditLog. Contributed by Kenneth Yang. 
(cherry picked from commit e9a005dfcf4d2249235aeff4da85b321dddc1ca9)
(cherry picked from commit c48640ffbd3852b75f898325ae0b3ea0ba864578)
 2018-12-28 10:39:14 -08:00
Surendra Singh Lilhore
de2b04a74b HDFS-14046. In-Maintenance ICON is missing in datanode info page. Contributed by Ranith Sardar. 
(cherry picked from commit 686fcd4db34dfe8642ff4b25fffbc73e42217f30)
 2018-12-24 11:11:21 +05:30
Brahma Reddy Battula
0b83c95ff4 HDFS-14095. EC: Track Erasure Coding commands in DFS statistics. Contributed by Ayush Saxena. 
(cherry picked from commit f534736867eed962899615ca1b7eb68bcf591d17)
 2018-11-30 00:27:37 +05:30
Vinayakumar B
0254100074 HDFS-13816. dfs.getQuotaUsage() throws NPE on non-existent dir instead of FileNotFoundException. Contributed by Vinayakumar B. 
(cherry picked from commit b09828145432c8d986ac8f05ec33608d8e611328)
 2018-11-26 18:22:58 +05:30
Vinayakumar B
e5e4dc42a3 HDFS-14056. Fix error messages in HDFS-12716. Contributed by Ayush Saxena. 
(cherry picked from commit 886b935630bf2d46f4c6773de013c093d2e44c38)
 2018-11-16 18:01:54 +05:30
Vinayakumar B
53b6290949 HDFS-13963. NN UI is broken with IE11. Contributed by Ayush Saxena. 
(cherry picked from commit 8571507efa3af3e4a5213a7e24c65e26162eac4d)
 2018-11-16 17:31:25 +05:30
Xiao Chen
8ab6aa1b42 Revert "HDFS-13732. ECAdmin should print the policy name when an EC policy is set. Contributed by Zsolt Venczel." 
This reverts commit 0424715207cd07debeee5c624973e9db90d36fb6.
 2018-11-13 12:46:03 -08:00
Brahma Reddy Battula
d72c134889 HDFS-14070. Refactor NameNodeWebHdfsMethods to allow better extensibility. Contributed by CR Hota 
(cherry picked from commit e7b63baca1e10b28d8b4462fd80537b871951aa3)
 2018-11-13 13:11:39 +05:30
Arpit Agarwal
b6ef824e76 HDFS-14065. Failed Storage Locations shows nothing in the Datanode Volume Failures. Contributed by Ayush Saxena. 
(cherry picked from commit b6d4e19f34f474ea8068ebb374f55e0db2f714da)
 2018-11-12 15:32:05 -08:00
Kitti Nanasi
01b8197893 HDFS-14039. ec -listPolicies doesn't show correct state for the default policy when the default is not RS(6,3). Contributed by Kitti Nanasi. 
Signed-off-by: Xiao Chen <xiao@apache.org>
(cherry picked from commit 8d99648c203004045a9339ad27258092969145d6)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ErasureCodingPolicyManager.java
 2018-11-08 10:08:34 -08:00
Inigo Goiri
f65324aa47 HDFS-14048. DFSOutputStream close() throws exception on subsequent call after DataNode restart. Contributed by Erik Krogen. 
(cherry picked from commit 887244de4adebe27693ed4ad3296a6f700cfa8c1)
 2018-11-06 11:20:33 -08:00
Inigo Goiri
9bf4f3d614 HDFS-14043. Tolerate corrupted seen_txid file. Contributed by Lukas Majercak. 
(cherry picked from commit f3296501e09fa7f1e81548dfcefa56f20fe337ca)
 2018-11-05 16:52:43 -08:00
Giovanni Matteo Fumarola
a1321d020a HDFS-14042. Fix NPE when PROVIDED storage is missing. Contributed by Virajith Jalaparti. 2018-11-05 11:39:15 -08:00
Akira Ajisaka
dd70b1f538
Addendum: Fix potential FSImage corruption. Contributed by Xiao Chen. 2018-10-31 17:02:15 +09:00
Xiao Chen
399645ebc1 HDFS-14027. DFSStripedOutputStream should implement both hsync methods. 
(cherry picked from commit db7e636824a36b90ba1c8e9b2fba1162771700fe)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStream.java
 2018-10-29 19:13:51 -07:00
Wei-Chiu Chuang
35e09fe594 Revert "HADOOP-15864. Job submitter / executor fail when SBN domain name can not resolved. Contributed by He Xiaoqiao." 
This reverts commit ef9f8ca13d8dca8031c045bad801fceb3e00aa15.
 2018-10-28 17:06:40 -07:00
Anu Engineer
4a1be70099 HDFS-14028. HDFS OIV temporary dir deletes folder. 
Contributed by Adam Antal.

(cherry picked from commit 4f10d7e23fc0b757a9e94bc448187a6211b90f10)
 2018-10-26 16:51:52 -07:00