Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
19,068 Commits 326 Branches 371 Tags
Commit Graph

7007 Commits

Author SHA1 Message Date
Elek, Marton
293f8bbd84 HDFS-14078. Admin helper fails to prettify NullPointerExceptions. Contributed by Elek, Marton. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit cc229be9ebb1b3737c9f5fcb29788b51fa584e47)
(cherry picked from commit 1a99f9794519faee5a196ab3d9cb91d55bfd0002)
 2019-06-18 12:00:48 -07:00
Wei-Chiu Chuang
29c4aded34 HDFS-14010. Pass correct DF usage to ReservedSpaceCalculator builder. Contributed by Virajith Jalaparti. 
(cherry picked from commit 1cc1530b4e874391a17ec81e24cb55dfbbabd36a)
(cherry picked from commit 223a582eb839bb6e4298a2fe0243bc27d73fc977)
 2019-06-18 00:34:55 -07:00
Weiwei Yang
3bdb661f35 HDFS-12770. Add doc about how to disable client socket cache. Contributed by Weiwei Yang. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 6ec57fe6a45df0e9216bbbba85b5bf61da1bf2ca)
(cherry picked from commit f36cdc4d7f45d1a3f8edd2d804c38b0b39990410)
 2019-06-17 21:47:07 -07:00
Wei-Chiu Chuang
b3e0430c3b HDFS-14465. When the Block expected replications is larger than the number of DataNodes, entering maintenance will never exit. Contributed by Yicong Cai. 
(cherry picked from commit 7db922c6474cb29e0e9bad320fcda4769366340a)
(cherry picked from commit 0eb42e1e05b780dab718bb3adfdd55f527dd5545)
 2019-06-17 15:18:39 -07:00
Inigo Goiri
76a66998fe HDFS-14560. Allow block replication parameters to be refreshable. Contributed by Stephen O'Donnell. 2019-06-13 18:28:38 -07:00
Sean Mackrory
fee1e67453 HADOOP-16213. Update guava to 27.0-jre. Contributed by Gabor Bota. 2019-06-13 07:38:43 -06:00
Stephen O'Donnell
a9b8310584 HDFS-13231. Extend visualization for Decommissioning, Maintenance Mode under Datanode tab in the NameNode UI. Contributed by Stephen O'Donnell. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 50de0874d0f14526bfb7052f2a2e64543dc0fcbc)
(cherry picked from commit 71159f9540d5a2aac6fb904649639f75e457e94d)
 2019-06-12 10:31:21 -07:00
He Xiaoqiao
03ec4c4c7c HDFS-14553. Make queue size of BlockReportProcessingThread configurable. Contributed by He Xiaoqiao. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit bd46bdf9f9244f3f3474d316255ac98717ed5719)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml

(cherry picked from commit d7560c866ecbfcd533a43890522ccb8b732a5382)
 2019-06-10 17:27:48 -07:00
Hanisha Koneru
dd795edcc7 HDFS-10659. Namenode crashes after Journalnode re-installation in an HA cluster due to missing paxos directory. Contributed by star, Hanisha Koneru. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 7217494f40dd99068a3f3b155261b1dac6c67828)
(cherry picked from commit 210480a23cab052c5ea538fbfd9e50d888b8d3ff)
 2019-06-10 13:47:17 -07:00
Kihwal Lee
a935258e8c HDFS-14521. Suppress setReplication logging. Contributed by Kihwal Lee 
(cherry picked from commit 9122b9b6493d315c8910fbc06eaba96273ee335c)
 2019-05-30 17:31:30 -05:00
Ayush Saxena
90cc5ddd49 HDFS-14512. ONE_SSD policy will be violated while write data with DistributedFileSystem.create(....favoredNodes). Contributed by Ayush Saxena. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit c1caab40f27e3e4f58ff1b5ef3e93efc56bbecbe)
(cherry picked from commit 2cd7577a5a58db42a14a659b85fbd475091d0cb2)
 2019-05-29 20:58:34 -07:00
Takanobu Asanuma
8098ddaf40 HADOOP-16331. Fix ASF License check in pom.xml. Contributed by Akira Ajisaka. 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2019-05-29 17:39:49 +09:00
Akira Ajisaka
f8bd5deec1
HADOOP-16323. https everywhere in Maven settings. 2019-05-27 15:28:21 +09:00
Erik Krogen
9295e95174 HDFS-14500. NameNode StartupProgress should not allow new steps in an already-completed phase. Contributed by Erik Krogen. 
(cherry-picked from 55e0c134f002f74cb4a0360b6682a1b6796d1598)
(cherry-picked from c26cf22d6b3949a4798d8133669b451660a0f050)
 2019-05-24 13:19:39 -07:00
Konstantin V Shvachko
799ca572f7 HDFS-14502. keepResults option in NNThroughputBenchmark should call saveNamespace(). Contributed by Konstantin V Shvachko. 2019-05-24 12:35:32 -07:00
Siyao Meng
9196c84c9f HDFS-14507. Document -blockingDecommission option for hdfs dfsadmin -listOpenFiles 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
(cherry picked from commit 9dff6eff819a832a5918d786675f42aa243a13e8)
 2019-05-22 13:25:42 +09:00
Siyao Meng
86fb5e1c8e HDFS-14463. Add Log Level link under NameNode and DataNode Web UI Utilities dropdown. Contributed by Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 7cb46f035a92056783bad23a9abc6a264d71285d)
(cherry picked from commit 0550aa32f1a4b1809c234f8ed785db08b2e33d13)
 2019-05-01 16:49:36 -07:00
Arpit Agarwal
7b51edbab9
HDFS-13677. Dynamic refresh Disk configuration results in overwriting VolumeMap. Contributed by xuzq. 
(cherry picked from commit 4b4200f1f87ad40d9c19ba160f706ffd0470a8d4)
 2019-04-29 14:54:11 -07:00
Puleya7
546ed4937c HDFS-14432. dfs.datanode.shared.file.descriptor.paths duplicated in hdfs-default.xml 
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit 13907d847986aa14a51cc2a3cc83151f22c43781)
 2019-04-17 21:57:06 +09:00
Akira Ajisaka
a70076b3a5
HDFS-14407. Fix misuse of SLF4j logging API in DatasetVolumeChecker#checkAllVolumes. Contributed by Wanqiang Ji. 
(cherry picked from commit 67020f09502a4f07342dee457e47bb52b03441ae)
 2019-04-05 12:31:14 +09:00
Stephen O'Donnell
d9899015eb HDFS-14389. getAclStatus returns incorrect permissions and owner when an iNodeAttributeProvider is configured. Contributed by Stephen O'Donnell. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit c528e427aa6745434672b1c1850738795ad1d6d2)
(cherry picked from commit 388f445dde577999b2d81f809adcfca8f0958499)
 2019-04-04 04:06:33 -07:00
Wei-Chiu Chuang
084fb9de22 HDFS-10477. Stop decommission a rack of DataNodes caused NameNode fail over to standby. Contributed by yunjiong zhao and Wei-Chiu Chuang. 
(cherry picked from commit be488b6070a124234c77f16193ee925d32ca9a20)
(cherry picked from commit c8703dda0727e17d759d7ad27f0caee88103a530)
 2019-04-03 12:41:07 -07:00
Gabor Bota
f01d592f3b
HADOOP-16220. Add findbugs ignores for unjustified issues during update to guava to 27.0-jre in hadoop-project 
This closes #665

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 53a86e2b8ecb83b666d4ed223fc270e1a46642c1)
 2019-04-01 13:52:44 +09:00
Takanobu Asanuma
1fc74178eb HDFS-14037. Fix SSLFactory truststore reloader thread leak in URLConnectionFactory. 
(cherry picked from commit 55fb3c32fb48ca26a629d4d5f3f07e2858d09594)
 2019-03-27 03:29:42 +09:00
Inigo Goiri
80732146c6 HDFS-14366. Improve HDFS append performance. Contributed by Chao Sun. 
(cherry picked from commit ff06ef0631cb8a0f67bbc39b5b5a1b0a81ca3b3c)
 2019-03-15 13:58:39 -07:00
Erik Krogen
cb76fc5618 HDFS-14346. Add better time precision to Configuration#getTimeDuration, allowing return unit and default unit to be specified independently. Contributed by Chao Sun. 
(cherry picked from commit 66357574ae1da09ced735da36bf7d80a40c3fa1b)
(cherry picked from commit fec7c5f3ebbaea7b290e0904570c3a485b541a22)
 2019-03-13 13:20:51 -07:00
Shweta Yakkali
a324db2cce HDFS-14081. hdfs dfsadmin -metasave metasave_test results NPE. Contributed by Shweta Yakkali. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 1bea785020a538115b3e08f41ff88167033d2775)
(cherry picked from commit 1ceefa726e1b531ec92c2ee2212b25c327644ef6)
 2019-03-12 16:06:16 -07:00
Stephen O'Donnell
327e57ae6e HDFS-14333. Datanode fails to start if any disk has errors during Namenode registration. Contributed by Stephen O'Donnell. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 34b14061b38dccab25058dff1b8743d8a3f82734)
(cherry picked from commit a21e2e4dbc1681175130242138ced0d36b6fafae)
 2019-03-12 10:19:37 -07:00
Erik Krogen
b0b8000240 HDFS-14317. Ensure checkpoints are created when in-progress edit log tailing is enabled with a period shorter than the log roll period. Contributed by Ekanth Sethuramalingam. 
(cherry-picked from commit 1bc282e0b3f74968c92751f0972746b012e72810)
(cherry-picked from commit 6d076dd5e8be8fee5062a04d4cd2aa0c5da0ea56)
 2019-03-07 08:43:39 -08:00
Wei-Chiu Chuang
d951497f57 HDFS-14314. fullBlockReportLeaseId should be reset after registering to NN. Contributed by star. 
(cherry picked from commit 387dbe587aa66ac99ec5f5b50827ec3e0a327613)
(cherry picked from commit e58ccca3ce131c955ceb115cd0b75e452eea828b)
 2019-03-04 10:46:00 -08:00
Erik Krogen
b7eb9116eb HDFS-14305. Fix serial number calculation in BlockTokenSecretManager to avoid token key ID overlap between NameNodes. Contributed by He Xiaoqiao. 2019-03-01 08:12:54 -08:00
Surendra Singh Lilhore
bd9e2826fe HDFS-14263. Remove unnecessary block file exists check from FsDatasetImpl#getBlockInputStream(). Contributed by Surendra Singh Lilhore 
(cherry picked from commit 29b411d5f065f177de05a69d2822836209f813c6)
 2019-02-13 23:56:34 +05:30
Surendra Singh Lilhore
3bcdff8fac HDFS-14274. EC: NPE While Listing EC Policy For A Directory Following Replication Policy. Contributed by Ayush Saxena. 
(cherry picked from commit 35d4f32b32a3ba05332811caf3d245d9c3dcf1a5)
 2019-02-13 23:26:14 +05:30
Vinayakumar B
1f3e737407 HDFS-14266. EC : Fsck -blockId shows null for EC Blocks if One Block Is Not Available. Contributed by Ayush Saxena. 2019-02-12 22:35:55 +05:30
Surendra Singh Lilhore
1f4be45ef6 HDFS-14140. JournalNodeSyncer authentication is failing in secure cluster. Contributed by Surendra Singh Lilhore. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 4be87353e35a30d95d8847b09a1890b014bfc6bb)
(cherry picked from commit 2501fcd26bd7bef2738a8f6660dc63862c755ce3)
 2019-02-07 16:53:12 -08:00
Kitti Nanasi
c25b641cec HDFS-14231. DataXceiver#run() should not log exceptions caused by InvalidToken exception as an error. Contributed by Kitti Nanasi. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 49ddd8a6ed5b40d12defb0771b4c8b53d4ffde3f)
(cherry picked from commit 38541b23753c5d0e2cb9baf1d2da536f846dc38c)
 2019-02-05 15:15:31 -08:00
Wangda Tan
a1e09b4c0c Make upstream aware of 3.1.2 release 
Change-Id: I397bc6ef75498726df4763bd07a8bf8fe1c38365
(cherry picked from commit 308f3168fa221a336ef0dbb1a17358b1ad6f9356)
(cherry picked from commit 649da5af04d57ee910dbe7fb0f4a7d9cb4c40eb0)
 2019-02-05 14:09:33 -08:00
Masatake Iwasaki
dd3e7fc0c2 HDFS-12729. Document special paths in HDFS. Contributed by Masatake Iwasaki. 
(cherry picked from commit dc5af4c6b7bdf437713a8c6ca3c535c4b6d6992e)
 2019-01-29 02:29:39 +09:00
Dinesh Chitlangia
b63941516f HDFS-14228. Incorrect getSnapshottableDirListing() javadoc. Contributed by Dinesh Chitlangia. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 45caeee6cfcf1ae3355cd880402159cf31e94a8a)
(cherry picked from commit eb77e60163e318ca42548e134fe79dba7289fdc6)
 2019-01-25 07:38:11 -08:00
Takanobu Asanuma
38e4bed224 HDFS-14218. EC: Ls -e throw NPE when directory ec policy is disabled. Contributed by Ayush Saxena. 
(cherry picked from commit 7d6792e5d2001e1a83ae75a4777324a3b01c557a)
 2019-01-23 10:05:39 +09:00
Arpit Agarwal
f5bfeac62e HDFS-14221. Replace Guava Optional with Java Optional. Contributed by Arpit Agarwal. 
(cherry picked from commit 1ff658b2ef3fb933897712c728bc628f3f44bded)
 2019-01-21 20:48:05 -08:00
Arpit Agarwal
78b45977b1 HDFS-14222. Make ThrottledAsyncChecker constructor public. Contributed by Arpit Agarwal. 
(cherry picked from commit 6f0756fc0e43b785d3dee72a669997d934b57e4c)
 2019-01-21 20:47:02 -08:00
Weiwei Yang
cf520b7d64 HDFS-14207. ZKFC should catch exception when ha configuration missing. Contributed by Fei Hui. 
(cherry picked from commit de34fc148ca4785c2ae7204df15096b955e671c4)
 2019-01-22 12:17:33 +08:00
Surendra Singh Lilhore
6c846cfbf2 HDFS-14198. Upload and Create button doesn't get enabled after getting reset. Contributed by Ayush Saxena. 
(cherry picked from commit 9aeaaa0479ea6b4c4135214722c8c7c39fb17d75)
 2019-01-11 14:59:17 +05:30
Wangda Tan
31ea2f7806 Preparing for 3.1.3 development 
Change-Id: I3c3d3ee47dc4fef239127b4452ff14676fa26e3d
 2019-01-07 10:04:58 -08:00
Wei-Chiu Chuang
f24684ca9c HDFS-14171. Performance improvement in Tailing EditLog. Contributed by Kenneth Yang. 
(cherry picked from commit e9a005dfcf4d2249235aeff4da85b321dddc1ca9)
(cherry picked from commit c48640ffbd3852b75f898325ae0b3ea0ba864578)
 2018-12-28 10:39:14 -08:00
Surendra Singh Lilhore
de2b04a74b HDFS-14046. In-Maintenance ICON is missing in datanode info page. Contributed by Ranith Sardar. 
(cherry picked from commit 686fcd4db34dfe8642ff4b25fffbc73e42217f30)
 2018-12-24 11:11:21 +05:30
Brahma Reddy Battula
0b83c95ff4 HDFS-14095. EC: Track Erasure Coding commands in DFS statistics. Contributed by Ayush Saxena. 
(cherry picked from commit f534736867eed962899615ca1b7eb68bcf591d17)
 2018-11-30 00:27:37 +05:30
Vinayakumar B
0254100074 HDFS-13816. dfs.getQuotaUsage() throws NPE on non-existent dir instead of FileNotFoundException. Contributed by Vinayakumar B. 
(cherry picked from commit b09828145432c8d986ac8f05ec33608d8e611328)
 2018-11-26 18:22:58 +05:30
Vinayakumar B
e5e4dc42a3 HDFS-14056. Fix error messages in HDFS-12716. Contributed by Ayush Saxena. 
(cherry picked from commit 886b935630bf2d46f4c6773de013c093d2e44c38)
 2018-11-16 18:01:54 +05:30