Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
20,965 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

20965 Commits

Author SHA1 Message Date
Ayush Saxena ebd7a0f155 HDFS-14660. [SBN Read] ObserverNameNode should throw StandbyException for requests not from ObserverProxyProvider. Contributed by Chao Sun. 2019-10-03 16:00:38 -07:00
Shweta Yakkali 3186afa24c HDFS-14494. Move Server logging of StatedId inside receiveRequestState(). Contributed by Shweta Yakkali. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-10-03 15:58:00 -07:00
Ayush Saxena 9a9edfa87e HDFS-14849. Erasure Coding: the internal block is replicated many times when datanode is decommissioning. Contributed by HuangTao. 
(cherry picked from commit c4c8d5fd0e)
 2019-10-03 14:56:09 -07:00
HUAN-PING SU f2be3e92b1 HADOOP-16542. Update commons-beanutils version to 1.9.4. Contributed by kevin su. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 38c1a10024)
 2019-10-02 22:00:40 -07:00
Chen Liang 05a1eb788f HDFS-12979. [SBN read] StandbyNode should upload FsImage to ObserverNode after checkpointing. Contributed by Chen Liang. 2019-10-02 21:12:54 -07:00
LeonGao91 885dd17595 HDFS-14678. Allow triggerBlockReport to a specific namenode. (#1252). Contributed by Leon Gao. 
(cherry picked from commit 9a1d8cfaf5)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestTriggerBlockReport.java
 2019-10-02 17:35:33 -07:00
Chen Liang 8a866f01c9 HDFS-14858. [SBN read] Allow configurably enable/disable AlignmentContext on NameNode. Contributed by Chen Liang. 2019-10-02 16:39:26 -07:00
Tsz Wo Nicholas Sze 824f717882 HADOOP-16126. ipc.Client.stop() may sleep too long to wait for all connections. 
(cherry picked from commit 0edb0c51dc)
 2019-10-02 16:32:53 -07:00
Wei-Chiu Chuang 28c11ff154 HDFS-14523. Remove excess read lock for NetworkToplogy. Contributed by Wu Weiwei. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Chen Liang <cliang@apache.org>
(cherry picked from commit 971a4c8e83)
 2019-10-02 16:27:07 -07:00
paulward24 0559c1dba3 HDFS-14618. Incorrect synchronization of ArrayList field (ArrayList is thread-unsafe). Contributed by Paul Ward. 
(cherry picked from commit d203045c30)
 2019-10-02 15:07:40 -07:00
paulward24 76a9135947 HDFS-14610. HashMap is not thread safe. Field storageMap is typically synchronized by storageMap. However, in one place, field storageMap is not protected with synchronized. (#1015) 
(cherry picked from commit d8bac50e12)
 2019-10-02 14:31:13 -07:00
Inigo Goiri 6148acebc3 HDFS-14527. Stop all DataNodes may result in NN terminate. Contributed by He Xiaoqiao. 
(cherry picked from commit 944adc61b1)
 2019-10-02 13:55:47 -07:00
Ayush Saxena 2bb4ac18ec HDFS-14808. EC: Improper size values for corrupt ec block in LOG. Contributed by Ayush Saxena. 
(cherry picked from commit 66400c1cbb)
 2019-10-02 13:49:12 -07:00
Surendra Singh Lilhore 7d7330aff0 HDFS-14754. Erasure Coding : The number of Under-Replicated Blocks never reduced. Contributed by hemanthboyina. 
(cherry picked from commit 4852a90e4b)
 2019-10-02 13:39:40 -07:00
Inigo Goiri d2a7a128d9 HADOOP-16161. NetworkTopology#getWeightUsingNetworkLocation return unexpected result. Contributed by He Xiaoqiao. 
(cherry picked from commit 389e640f0c)
 2019-10-01 17:33:14 -07:00
Yiqun Lin c74027d9d3 HDFS-14313. Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du. Contributed by Lisheng Sun. 
(cherry picked from commit a5bb1e8ee8)

 Conflicts:
	hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
 2019-10-01 17:14:46 -07:00
Inigo Goiri 3510a9e5e5 HDFS-14192. Track missing DFS operations in Statistics and StorageStatistics. Contributed by Ayush Saxena. 
(cherry picked from commit f048512bb8)
 2019-10-01 16:57:20 -07:00
Inigo Goiri 917e53f066 HDFS-14460. DFSUtil#getNamenodeWebAddr should return HTTPS address based on policy configured. Contributed by CR Hota. 
(cherry picked from commit 865c328930)
 2019-10-01 09:53:36 -07:00
Surendra Singh Lilhore a131611b4b HDFS-14175. EC: Native XOR decoder should reset the output buffer before using it. Contributed by Ayush Saxena. 
(cherry picked from commit dacc1a759e)
 2019-10-01 09:47:18 -07:00
Inigo Goiri fc8a7a9e5b HDFS-14202. dfs.disk.balancer.max.disk.throughputInMBperSec property is not working as per set value. Contributed by Ranith Sardar. 
(cherry picked from commit 0e79a86582)
 2019-10-01 09:35:04 -07:00
Inigo Goiri 06b2eceb76 HDFS-14418. Remove redundant super user priveledge checks from namenode. Contributed by Ayush Saxena. 
(cherry picked from commit be6c8014e6)
 2019-10-01 09:22:15 -07:00
Wei-Chiu Chuang ab3fb28d47 HADOOP-15418. Hadoop KMSAuthenticationFilter needs to use getPropsByPrefix instead of iterator to avoid ConcurrentModificationException. Contributed by lqjack and Suma Shivaprasad 
(cherry picked from commit cd2158456d)
 2019-10-01 09:02:48 -07:00
Kihwal Lee ca18ae3bc1 HADOOP-15909. KeyProvider class should implement Closeable. Contributed 
by Kuhu Shukla.

(cherry picked from commit 33c009a45d)
 2019-10-01 08:53:55 -07:00
Sunil G 312cfa994a YARN-9801. SchedConfCli does not work wiwith https mode. Contributed by Prabhu Joseph 
(cherry picked from commit 99cd7572f1)
 2019-10-01 20:07:18 +05:30
bibinchundatt 5cd6eb2a18 YARN-9858. Optimize RMContext getExclusiveEnforcedPartitions. Contributed by Jonathan Hung. 2019-10-01 16:04:58 +05:30
Sunil G 52f815d39d YARN-9864. Format CS Configuration present in Configuration Store. Contributeed by Prabhu Joseph 
(cherry picked from commit 137546a78a)
 2019-10-01 09:09:22 +05:30
Erik Krogen f5fb7d8c18 HADOOP-16531. Log more timing information for slow RPCs. Contributed by Chen Zhang. 
(cherry picked from commit a23417533e)
 2019-09-30 17:35:45 -07:00
Wei-Chiu Chuang 46e72775f5 HADOOP-15865. ConcurrentModificationException in Configuration.overlay() method. Contributed by Oleksandr Shevchenko. 
(cherry picked from commit e872ceb810)
 2019-09-30 17:25:02 -07:00
Zsombor Gegesy f42c8a44ae HADOOP-15014. Addendum: KMS should log the IP address of the clients. Contributed by Zsombor Gegesy. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit b0131bc265)
 2019-09-30 17:16:10 -07:00
Zsombor Gegesy 85aff9facc HADOOP-15014. KMS should log the IP address of the clients. Contributed by Zsombor Gegesy. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 008766c119)
 2019-09-30 17:16:05 -07:00
Gopal V 79359e2091 HADOOP-16461. Regression: FileSystem cache lock parses XML within the lock. 
Contributed by Gopal V.

Change-Id: If6654f850e9c24ee0d9519a46fd6269b18e1a7a4
(cherry picked from commit aebac6d2d2)
 2019-09-30 17:05:29 -07:00
Konstantin V Shvachko 665360ec6f HDFS-14305. Fix serial number calculation in BlockTokenSecretManager to avoid token key ID overlap between NameNodes. Contributed by Konstantin V Shvachko. 2019-09-30 16:57:56 -07:00
Konstantin V Shvachko 3912a6fc7d Revert "HDFS-14305. Fix serial number calculation in BlockTokenSecretManager to avoid token key ID overlap between NameNodes. Contributed by He Xiaoqiao." 
This reverts commit 0feba4396f.
 2019-09-30 16:57:26 -07:00
Xiao Chen bd5b59feea HADOOP-15849. Upgrade netty version to 3.10.6. 
(cherry picked from commit 8853fc8a55)
 2019-10-01 06:54:20 +09:00
Ayush Saxena b7ee3938e9 HDFS-14853. NPE in DFSNetworkTopology#chooseRandomWithStorageType() when the excludedNode is not present. Contributed by Ranith Sardar. 2019-09-28 08:29:58 +05:30
Chen Liang 2d7b37ea2b HDFS-14785. [SBN read] Change client logging to be less aggressive. Contributed by Chen Liang. 2019-09-26 15:55:10 -07:00
Jonathan Hung f4f210d2e5 Addendum to YARN-9730. Support forcing configured partitions to be exclusive based on app node label 
(cherry picked from commit d86a1acc866cbda845fb3896dc824baf12217383)
 2019-09-25 17:49:37 -07:00
Jonathan Hung 806c7b7dfb YARN-9730. Support forcing configured partitions to be exclusive based on app node label 
(cherry picked from commit 73a044a63822303f792183244e25432528ecfb1e)
 2019-09-24 13:51:54 -07:00
Ayush Saxena a0db762206 HDFS-14655. [SBN Read] Namenode crashes if one of The JN is down. Contributed by Ayush Saxena. 2019-09-25 01:21:14 +05:30
Jonathan Hung a1fa9a8a7f YARN-9762. Add submission context label to audit logs. Contributed by Manoj Kumar 
(cherry picked from commit 3d78b1223d)
 2019-09-23 13:12:57 -07:00
Rohith Sharma K S f9f0338104 Make upstream aware of 3.2.1 release. 2019-09-23 12:03:53 +05:30
Yuval Degani e31594f20e HADOOP-16581. Revise ValueQueue to correctly replenish queues that go below the watermark (#1463) 
(cherry picked from dd0834696a)
 2019-09-20 09:58:39 -07:00
Takanobu Asanuma b2072444ab HADOOP-16069. Support configure ZK_DTSM_ZK_KERBEROS_PRINCIPAL in ZKDelegationTokenSecretManager using principal with Schema /_HOST. Contributed by luhuachao. 
(cherry picked from commit 298f07abce)
 2019-09-20 13:57:40 +09:00
Kihwal Lee 7477f8d2e9 HADOOP-16582. LocalFileSystem's mkdirs() does not work as expected under viewfs. Contributed by Kihwal Lee 
(cherry picked from commit d4205dce17)
 2019-09-19 08:24:39 -05:00
Chen Liang 4299bc0d3e HDFS-14822. [SBN read] Revisit GlobalStateIdContext locking when getting server state id. Contributed by Chen Liang. 2019-09-18 16:48:58 -07:00
Kihwal Lee e29f91bad3 HDFS-13959. TestUpgradeDomainBlockPlacementPolicy is flaky. Contributed by Ayush Saxena. 
(cherry picked from commit 1851d06eb3)
 2019-09-18 15:53:08 -05:00
Wei-Chiu Chuang 9c9920897d HDFS-14569. Result of crypto -listZones is not formatted properly. Contributed by hemanthboyina. 
(cherry picked from 0f2dad6679)
 2019-09-18 10:30:25 -07:00
Wei-Chiu Chuang b906f53b0d HDFS-14836. FileIoProvider should not increase FileIoErrors metric in datanode volume metric. Contributed by Aiphago. 
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
(cherry picked from commit 5dd859a8a0)
 2019-09-18 09:36:10 -07:00
Anu Engineer 2e9482bd20 HDFS-14759. HDFS cat logs an info message. 
Contributed by Eric Badger.

(cherry picked from commit 8aaf5e1a14)
 2019-09-18 15:23:28 +08:00
Sunil G 3e0025d877 YARN-9833. Race condition when DirectoryCollection.checkDirs() runs during container launch. Contributed by Peter Bacsko. 
(cherry picked from commit c474e24c0b)
 2019-09-18 09:22:48 +05:30