Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
19,381 Commits 326 Branches 371 Tags
Commit Graph

4923 Commits

Author SHA1 Message Date
Ayush Saxena
9434f36b43 HADOOP-16662. Remove unnecessary InnerNode check in NetworkTopology#add(). Contributed by Lisheng Sun. 2019-10-20 11:55:58 +05:30
Szilard Nemeth
f8c06f5a87 HADOOP-16580. Disable retry of FailoverOnNetworkExceptionRetry in case of AccessControlException. Contributed by Adam Antal 
(cherry picked from commit c79a5f2d9930f58ad95864c59cd0a6164cd53280)
(cherry picked from commit 0bbd48c7a8c0d1dd57ffe5d372d749e5e7e41d34)
 2019-10-16 14:24:17 +02:00
Jonathan Hung
0dc85bc182 YARN-9760. Support configuring application priorities on a workflow level. Contributed by Varun Saxena 2019-10-08 11:18:52 -07:00
Szilard Nemeth
2c8cfb4f64 YARN-6715. Fix documentation about NodeHealthScriptRunner. Contributed by Peter Bacsko 2019-10-08 17:34:44 +02:00
Erik Krogen
56caacac1f HDFS-14162. [SBN read] Allow Balancer to work with Observer node. Add a new ProxyCombiner allowing for multiple related protocols to be combined. Allow AlignmentContext to be passed in NameNodeProxyFactory. Contributed by Erik Krogen. 
(cherry picked from 64f28f9efa2ef3cd9dd54a6c5009029721e030ed)
(cherry picked from 69b0c513a9b11cc7f795747732173b36aacbe794)
 2019-10-04 10:48:17 -07:00
Tsz Wo Nicholas Sze
ce804be72e HADOOP-16127. In ipc.Client, put a new connection could happen after stop. 
(cherry picked from commit 9192f71e21847ad86bc9ff23847d8957dfe8ae58)
 2019-10-03 18:28:13 -07:00
Wei-Chiu Chuang
aa5dfb2f8a HADOOP-12282. Connection thread's name should be updated after address changing is detected. Contributed by Lisheng Sun. 
(cherry picked from commit b94eba9f11af66b10638dd255c224e946d842b8c)
 2019-10-03 17:30:00 -07:00
Giovanni Matteo Fumarola
76c30e4f3c HADOOP-16625: Backport HADOOP-14624 to branch-3.1. 
(cherry picked from commit 79c97f6a0bebc95ff81a8ef9b07d3619f05ed583)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSaveNamespace.java
 2019-10-03 15:45:23 -07:00
Tsz Wo Nicholas Sze
8d71b508ab HADOOP-16126. ipc.Client.stop() may sleep too long to wait for all connections. 
(cherry picked from commit 0edb0c51dc2c4ae2f353e260f01912e28033d70f)
 2019-10-02 16:32:40 -07:00
Wei-Chiu Chuang
41c0757a1f HDFS-14523. Remove excess read lock for NetworkToplogy. Contributed by Wu Weiwei. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Chen Liang <cliang@apache.org>
(cherry picked from commit 971a4c8e8328a4bdea65de4a0e84c82b5b2de24b)
 2019-10-02 16:27:00 -07:00
Inigo Goiri
f1fe3abac9 HADOOP-16161. NetworkTopology#getWeightUsingNetworkLocation return unexpected result. Contributed by He Xiaoqiao. 
(cherry picked from commit 389e640f0cc7d8528e9b4411457f04a528601c69)
 2019-10-01 17:33:08 -07:00
Yiqun Lin
3aea2bcf12 HDFS-14313. Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du. Contributed by Lisheng Sun. 
(cherry picked from commit a5bb1e8ee871df1111ff77d0f6921b13c8ffb50e)

 Conflicts:
	hadoop-common-project/hadoop-common/src/main/resources/core-default.xml

(cherry picked from commit c74027d9d34711c2c4baed7c98bc475d95097be0)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/BlockPoolSlice.java
 2019-10-01 17:16:55 -07:00
Stephen O'Donnell
9dfb26136b HADOOP-16140. hadoop fs expunge to add -immediate option to purge trash immediately. 
Contributed by Stephen O'Donnell.

Signed-off-by: Steve Loughran <stevel@apache.org>
(cherry picked from commit 686c0141eff0886c285b8e52fddade43c1ce4570)
 2019-10-01 13:13:59 -07:00
Surendra Singh Lilhore
5436df0c94 HDFS-14175. EC: Native XOR decoder should reset the output buffer before using it. Contributed by Ayush Saxena. 
(cherry picked from commit dacc1a759e3ba3eca000cbacc6145b231253b174)
 2019-10-01 09:47:11 -07:00
Wei-Chiu Chuang
57ed0ff225 HADOOP-15418. Hadoop KMSAuthenticationFilter needs to use getPropsByPrefix instead of iterator to avoid ConcurrentModificationException. Contributed by lqjack and Suma Shivaprasad 
(cherry picked from commit cd2158456db8c89eeea64b72654a736ea8607e23)
(cherry picked from commit ab3fb28d47e8fa260d2de7490aea59404d571c2e)
 2019-10-01 09:03:01 -07:00
Kihwal Lee
757c11e628 HADOOP-15909. KeyProvider class should implement Closeable. Contributed 
by Kuhu Shukla.

(cherry picked from commit 33c009a45d95c9b582db25acd205d087e01e2085)
 2019-10-01 08:53:47 -07:00
Erik Krogen
5e56914650 HADOOP-16531. Log more timing information for slow RPCs. Contributed by Chen Zhang. 
(cherry picked from commit a23417533e1ee052893baf207ec636c4993c5994)
 2019-09-30 17:35:38 -07:00
Wei-Chiu Chuang
72ef752876 HADOOP-15865. ConcurrentModificationException in Configuration.overlay() method. Contributed by Oleksandr Shevchenko. 
(cherry picked from commit e872ceb810a343da7fce7185dca78d3b9aad9b7b)
(cherry picked from commit 46e72775f5b2156f49b15beac8bffff9cd5aad7f)
 2019-09-30 17:32:23 -07:00
Zsombor Gegesy
db5b8358ca HADOOP-15014. Addendum: KMS should log the IP address of the clients. Contributed by Zsombor Gegesy. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit b0131bc265453051820e54908e70d39433c227ab)
 2019-09-30 17:15:51 -07:00
Zsombor Gegesy
b59911d841 HADOOP-15014. KMS should log the IP address of the clients. Contributed by Zsombor Gegesy. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 008766c119d9ed9d568f9458ed0c02136962da5b)
 2019-09-30 17:15:43 -07:00
Gopal V
b9dc2c17de HADOOP-16461. Regression: FileSystem cache lock parses XML within the lock. 
Contributed by Gopal V.

Change-Id: If6654f850e9c24ee0d9519a46fd6269b18e1a7a4
(cherry picked from commit aebac6d2d2e612e400a7d73be67dafb47e239211)
 2019-09-30 16:58:52 -07:00
Zhankun Tang
670d0657b4 Make upstream aware of 3.1.3 release. 2019-09-27 16:58:36 +08:00
Yuval Degani
b091abc471 HADOOP-16581. Revise ValueQueue to correctly replenish queues that go below the watermark (#1463) 
(cherry picked from dd0834696a694564af65a1355c9d13275f44df51)
(cherry picked from e31594f20ed43670a9ff1463a752d8b8e4b3b6f0)
 2019-09-20 09:58:57 -07:00
Takanobu Asanuma
6ef3204d54 HADOOP-16069. Support configure ZK_DTSM_ZK_KERBEROS_PRINCIPAL in ZKDelegationTokenSecretManager using principal with Schema /_HOST. Contributed by luhuachao. 
(cherry picked from commit 298f07abceb4bb854630c9058e1697d4ef3d2ae2)
 2019-09-20 13:58:19 +09:00
Kihwal Lee
56562b9117 HADOOP-16582. LocalFileSystem's mkdirs() does not work as expected under viewfs. Contributed by Kihwal Lee 
(cherry picked from commit d4205dce176287e863f567b333e0d408bf51ae6d)
 2019-09-19 08:26:36 -05:00
Wei-Chiu Chuang
6af1ecc8e4 HDFS-14569. Result of crypto -listZones is not formatted properly. Contributed by hemanthboyina. 
(cherry picked from 0f2dad6679b7fc35474a3d33dc40b0db89bb1d80)
(cherry picked from 9c9920897de182c3da3ad33ff53e506242bbd362)
 2019-09-18 10:30:52 -07:00
Weiwei Yang
1978317068 YARN-2255. YARN Audit logging not added to log4j.properties. Contributed by Aihua Xu. 
(cherry picked from commit f8c14326ee0026f2c57a4d061bff5fadec74b74d)
 2019-09-18 09:20:37 +08:00
Zhankun Tang
ef79d98788 Preparing for 3.1.4 development 2019-09-04 16:11:36 +08:00
Stephen O'Donnell
63b18d764a HDFS-14706. Checksums are not checked if block meta file is less than 7 bytes. Contributed by Stephen O'Donnell. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 915cbc91c0a12cc7b4d3ef4ea951941defbbcb33)
(cherry picked from commit 2c7f49743fba587e32398c81412db9eac426cf98)
 2019-09-02 09:58:48 -07:00
Chen Liang
0839cf1d4e HDFS-14611. Move handshake secret field from Token to BlockAccessToken. Contributed by Chen Liang. 2019-08-23 10:11:22 -07:00
Chen Liang
ef9b0a1d9a HDFS-13699. Add DFSClient sending handshake token to DataNode, and allow DataNode overwrite downstream QOP. Contributed by Chen Liang. 2019-08-23 10:11:22 -07:00
Chen Liang
3135cc35e3 HDFS-13617. Allow wrapping NN QOP into token in encrypted message. Contributed by Chen Liang 2019-08-23 10:11:22 -07:00
Chen Liang
3835a6c4e8 HDFS-13566. Add configurable additional RPC listener to NameNode. Contributed by Chen Liang. 2019-08-23 10:11:22 -07:00
Chen Liang
14265b8f03 HDFS-13547. Add ingress port based sasl resolver. Contributed by Chen Liang. 2019-08-23 10:11:22 -07:00
David Mollitor
0b7f9a09c2 HADOOP-15246. SpanReceiverInfo - Prefer ArrayList over LinkedList. Contributed by David Mollitor. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 901652e26321832ac42ce54daa18ebc1813772b5)
(cherry picked from commit 1b30b3fbe7b5e130778c79465ac7986772a83ab8)
 2019-08-18 18:44:02 -07:00
Christopher Gregorian
96ea7f0562 HDFS-14403. Cost-based extension to the RPC Fair Call Queue. Contributed by Christopher Gregorian. 
(cherry picked from 129576f628d370def74e56112aba3a93e97bbf70)
(cherry picked from e4b650f91e7353f98fee2a725490ca45aa81e1a4)
 2019-08-14 13:37:01 -07:00
Masatake Iwasaki
e66ad19395 HDFS-14423. Percent (%) and plus (+) characters no longer work in WebHDFS. 
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit da0006fe0473e353ee2d489156248a01aa982dfd)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java

(cherry picked from commit d7ca016d63d89e5c8377a035f93485a7c77c3430)
 2019-08-14 09:07:00 +09:00
Wei-Chiu Chuang
8501bed9cb HADOOP-15681. AuthenticationFilter should generate valid date format for Set-Cookie header regardless of default Locale. Contributed by Cao Manh Dat. 
(cherry picked from commit e20b19543ba148cf0242884a3e553ad2f39421b3)
(cherry picked from commit 490b18fd4f35a19dc2dd48249cb38a96dbf2f85c)
 2019-08-01 17:36:59 -07:00
He Xiaoqiao
5d35393912 HDFS-13529. Fix default trash policy emptier trigger time correctly. Contributed by He Xiaoqiao. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit f86de6f76a3079c2655df9b242fc968edfb17b9d)
(cherry picked from commit 555b56c0421be84efd09221db2b3f26c8b44ed45)
 2019-08-01 14:56:17 -07:00
Christopher Gregorian
7b8f08f59e HADOOP-16459. Backport of HADOOP-16266. Add more fine-grained processing time metrics to the RPC layer. Contributed by Christopher Gregorian. 
This commit also includes the follow-on commit 827a84778a4e3b8f165806dfd2966f0951a5e575.

(cherry-picked from f96a2df38d889f29314c57f4d94227b2e419a11f)
(cherry-picked from d4492bdd9edec60c236aff85de50b963097e5a7f)
 2019-07-30 14:45:00 -07:00
Akira Ajisaka
4662b13b26
HADOOP-16435. RpcMetrics should not retained forever. Contributed by Zoltan Haindrich. 
(cherry picked from commit cbfa3f3e988b8a3f142abadfe4f18201b6438ac9)
 2019-07-29 17:39:42 -07:00
Erik Krogen
4d21310865 HADOOP-16245. Restrict the effect of LdapGroupsMapping SSL configurations to avoid interfering with other SSL connections. Contributed by Erik Krogen. 
(cherry-picked from 62efb6300619670d3e0554c3ba14c264fa0c705b)
(cherry-picked from 5ec34bf43adb498797afd24bb02989d4fec25409)
 2019-07-29 14:00:24 -07:00
Steve Loughran
1b7c7dc860
HADOOP-9844. NPE when trying to create an error message response of SASL RPC 
This closes #55

Change-Id: I10a20380565fa89762f4aa564b2f1c83b9aeecdc
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 07530314c2130ecd1525682c59bf51f15b82c024)
 2019-07-26 17:55:00 +09:00
Gopal V
2c4d8c343e HADOOP-16341. ShutDownHookManager: Regressed performance on Hook removals after HADOOP-15679 
Contributed by Gopal V and Atilla Magyar.

Change-Id: I066d5eece332a1673594de0f9b484443f95530ec
(cherry picked from commit 76d2cd228336dde207e47c9a659ce48bcb9b4e8f)
 2019-07-17 17:13:03 +01:00
Konstantin V Shvachko
321e26ced1 HDFS-14170. [SBN read] Fix checkstyle warnings related to SBN reads. Contributed by Konstantin V Shvachko. 2019-07-12 12:24:43 -07:00
Chao Sun
8f36bdc3b9 HDFS-14149. [SBN read] Fix annotations on new interfaces/classes for SBN reads. Contributed by Chao Sun. 2019-07-12 12:24:41 -07:00
Erik Krogen
6c1bf61628 HDFS-14146. [SBN read] Handle exceptions from and prevent handler threads from blocking within internalQueueCall. Contributed by Chao Sun. 2019-07-12 12:24:40 -07:00
Chen Liang
c23755a792 HDFS-14138. [SBN read] Description errors in the comparison logic of transaction ID. Contributed by xiangheng. 2019-07-12 12:24:40 -07:00
Erik Krogen
67c4768825 HDFS-13873. [SBN read] ObserverNode should reject read requests when it is too far behind. Contributed by Konstantin Shvachko. 2019-07-12 12:24:36 -07:00
Chao Sun
c4d944aeb3 HDFS-14067. [SBN read] Allow manual failover between standby and observer. Contributed by Chao Sun. 2019-07-12 12:23:16 -07:00