Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
21,288 Commits 326 Branches 371 Tags
Commit Graph

5102 Commits

Author SHA1 Message Date
Wei-Chiu Chuang
bd2f79cfd8 HADOOP-15686. Supress bogus AbstractWadlGeneratorGrammarGenerator in KMS stderr. Contributed by Wei-Chiu Chuang. 
Reviewed-by: Xiaoyu Yao <xyao@apache.org>
(cherry picked from commit 9664b9c7a65d943e815c119e94234d3bf0b68dd4)
 2019-11-20 13:08:40 -08:00
Ayush Saxena
bcca7120e0 HADOOP-16712. Config ha.failover-controller.active-standby-elector.zk.op.retries is not in core-default.xml. Contributed by Xieming Li. 2019-11-18 19:52:36 +05:30
Lukas Majercak
b914f385a0 HADOOP-16705. MBeanInfoBuilder puts unnecessary memory pressure on the system with a debug log. 
MBeanInfoBuilder's get() method DEBUG logs all the MBeanAttributeInfo attributes that it gathered. This can have a high memory churn that can be easily avoided.

(cherry picked from commit c73334a924d2009d136a8ee62278aaac53418a24)
 2019-11-15 13:30:11 -08:00
Akira Ajisaka
ea9c74d9c1
HADOOP-15097. AbstractContractDeleteTest::testDeleteNonEmptyDirRecursive with misleading path. Contributed by Xieming Li. 
(cherry picked from commit 92c28c100ee1aa414948cd510321ad13cb8639bc)
 2019-11-15 13:51:09 +09:00
Xudong Cao
975f669141 HADOOP-16677. Recalculate the remaining timeout millis correctly while throwing an InterupptedException in SocketIOWithTimeout. (#1687) 
(cherry picked from commit df6b3162c11987ba5299c69cb251332228dacf36)
 2019-11-12 14:21:50 -08:00
Siyao Meng
e0cf1735e1 HADOOP-16676. Backport HADOOP-16152 to branch-3.2. Contributed by Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-11-12 11:38:42 -08:00
Chen Liang
431edfcbe4 HDFS-14941. Potential editlog race condition can cause corrupted file. Contributed by Chen Liang and Konstantin Shvachko. 2019-11-06 15:00:15 -08:00
Chen Liang
a58f466d07 HADOOP-16013. DecayRpcScheduler decay thread should run as a daemon. Contributed by Erik Krogen. 
(cherry picked from commit 01cb958af44b2376bcf579cc65d90566530f733d)
 2019-11-04 17:18:27 -08:00
Jonathan Hung
9ccfc2d6d3 Add 2.10.0 release notes for HDFS-12943 
(cherry picked from commit ef9d12df24c0db76fd37a95551db7920d27d740c)
 2019-10-31 20:34:36 -07:00
Jonathan Hung
5d2ffcc7aa Make upstream aware of 2.10.0 release 
(cherry picked from commit 7663db59c097c82eeed2df7a91168a4d7123c96b)
 2019-10-30 20:59:20 -07:00
Duo Zhang
a37a4bc62f HADOOP-16598. Backport "HADOOP-16558 [COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes" to all active branches 
Signed-off-by: stack <stack@apache.org>
 2019-10-30 20:26:54 -07:00
Steve Loughran
1932499eaf HADOOP-16669. TestRawLocalFileSystemContract.testPermission fails if no native library. 
Contributed by Steve Loughran.

Change-Id: I622e36f17a63c775c5e33e93b8b26e461d0bc451
 2019-10-29 17:10:46 +00:00
Rakesh Radhakrishnan
a6cdcf676e HDFS-14818. Check native pmdk lib by 'hadoop checknative' command. Contributed by Feilong He. 
(cherry picked from commit 659c88801d008bb352d10a1cb3bd0e401486cc9b)
 2019-10-28 10:41:40 +05:30
Sammi Chen
6eb5fb596f HDFS-14356. Implement HDFS cache on SCM with native PMDK libs. Contributed by Feilong He. 
(cherry picked from commit d1aad444907e1fc5314e8e64529e57c51ed7561c)
 2019-10-28 10:40:46 +05:30
Fei Hui
54d3e212bf HDFS-14923. Remove dead code from HealthMonitor. Contributed by Fei Hui. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 7be5508d9b35892f483ba6022b6aced7648b8fa3)
 2019-10-25 15:06:46 -07:00
Ayush Saxena
ae4b1325d1 HADOOP-16662. Remove unnecessary InnerNode check in NetworkTopology#add(). Contributed by Lisheng Sun. 2019-10-20 11:55:43 +05:30
Szilard Nemeth
0bbd48c7a8 HADOOP-16580. Disable retry of FailoverOnNetworkExceptionRetry in case of AccessControlException. Contributed by Adam Antal 
(cherry picked from commit c79a5f2d9930f58ad95864c59cd0a6164cd53280)
 2019-10-16 14:11:30 +02:00
Akira Ajisaka
eb4bd54938
YARN-7243. Moving logging APIs over to slf4j in hadoop-yarn-server-resourcemanager. (#1634) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit e40e2d6ad5cbe782c3a067229270738b501ed27e)

 Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/AbstractYarnScheduler.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/volume/csi/VolumeManagerImpl.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/volume/csi/lifecycle/VolumeImpl.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/ApplicationMasterServiceTestBase.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationMasterLauncher.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationMasterServiceInterceptor.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceManager.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairSchedulerConfiguration.java
 2019-10-10 15:27:54 +09:00
Jonathan Hung
ad3c98456d YARN-9760. Support configuring application priorities on a workflow level. Contributed by Varun Saxena 
(cherry picked from commit eebd313d76ed742fe82292bd8c0184970cdc5692)
 2019-10-08 11:17:05 -07:00
Szilard Nemeth
9b4aba49d1 YARN-6715. Fix documentation about NodeHealthScriptRunner. Contributed by Peter Bacsko 2019-10-08 17:41:46 +02:00
Erik Krogen
69b0c513a9 HDFS-14162. [SBN read] Allow Balancer to work with Observer node. Add a new ProxyCombiner allowing for multiple related protocols to be combined. Allow AlignmentContext to be passed in NameNodeProxyFactory. Contributed by Erik Krogen. 
(cherry picked from 64f28f9efa2ef3cd9dd54a6c5009029721e030ed)
 2019-10-04 10:48:33 -07:00
Tsz Wo Nicholas Sze
8acd9ab695 HADOOP-16127. In ipc.Client, put a new connection could happen after stop. 
(cherry picked from commit 9192f71e21847ad86bc9ff23847d8957dfe8ae58)
 2019-10-03 18:28:21 -07:00
Wei-Chiu Chuang
6facb3f7c6 HADOOP-12282. Connection thread's name should be updated after address changing is detected. Contributed by Lisheng Sun. 
(cherry picked from commit b94eba9f11af66b10638dd255c224e946d842b8c)
 2019-10-03 17:30:08 -07:00
Tsz Wo Nicholas Sze
824f717882 HADOOP-16126. ipc.Client.stop() may sleep too long to wait for all connections. 
(cherry picked from commit 0edb0c51dc2c4ae2f353e260f01912e28033d70f)
 2019-10-02 16:32:53 -07:00
Wei-Chiu Chuang
28c11ff154 HDFS-14523. Remove excess read lock for NetworkToplogy. Contributed by Wu Weiwei. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Chen Liang <cliang@apache.org>
(cherry picked from commit 971a4c8e8328a4bdea65de4a0e84c82b5b2de24b)
 2019-10-02 16:27:07 -07:00
Inigo Goiri
d2a7a128d9 HADOOP-16161. NetworkTopology#getWeightUsingNetworkLocation return unexpected result. Contributed by He Xiaoqiao. 
(cherry picked from commit 389e640f0cc7d8528e9b4411457f04a528601c69)
 2019-10-01 17:33:14 -07:00
Yiqun Lin
c74027d9d3 HDFS-14313. Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du. Contributed by Lisheng Sun. 
(cherry picked from commit a5bb1e8ee871df1111ff77d0f6921b13c8ffb50e)

 Conflicts:
	hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
 2019-10-01 17:14:46 -07:00
Surendra Singh Lilhore
a131611b4b HDFS-14175. EC: Native XOR decoder should reset the output buffer before using it. Contributed by Ayush Saxena. 
(cherry picked from commit dacc1a759e3ba3eca000cbacc6145b231253b174)
 2019-10-01 09:47:18 -07:00
Wei-Chiu Chuang
ab3fb28d47 HADOOP-15418. Hadoop KMSAuthenticationFilter needs to use getPropsByPrefix instead of iterator to avoid ConcurrentModificationException. Contributed by lqjack and Suma Shivaprasad 
(cherry picked from commit cd2158456db8c89eeea64b72654a736ea8607e23)
 2019-10-01 09:02:48 -07:00
Kihwal Lee
ca18ae3bc1 HADOOP-15909. KeyProvider class should implement Closeable. Contributed 
by Kuhu Shukla.

(cherry picked from commit 33c009a45d95c9b582db25acd205d087e01e2085)
 2019-10-01 08:53:55 -07:00
Erik Krogen
f5fb7d8c18 HADOOP-16531. Log more timing information for slow RPCs. Contributed by Chen Zhang. 
(cherry picked from commit a23417533e1ee052893baf207ec636c4993c5994)
 2019-09-30 17:35:45 -07:00
Wei-Chiu Chuang
46e72775f5 HADOOP-15865. ConcurrentModificationException in Configuration.overlay() method. Contributed by Oleksandr Shevchenko. 
(cherry picked from commit e872ceb810a343da7fce7185dca78d3b9aad9b7b)
 2019-09-30 17:25:02 -07:00
Zsombor Gegesy
f42c8a44ae HADOOP-15014. Addendum: KMS should log the IP address of the clients. Contributed by Zsombor Gegesy. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit b0131bc265453051820e54908e70d39433c227ab)
 2019-09-30 17:16:10 -07:00
Zsombor Gegesy
85aff9facc HADOOP-15014. KMS should log the IP address of the clients. Contributed by Zsombor Gegesy. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 008766c119d9ed9d568f9458ed0c02136962da5b)
 2019-09-30 17:16:05 -07:00
Gopal V
79359e2091 HADOOP-16461. Regression: FileSystem cache lock parses XML within the lock. 
Contributed by Gopal V.

Change-Id: If6654f850e9c24ee0d9519a46fd6269b18e1a7a4
(cherry picked from commit aebac6d2d2e612e400a7d73be67dafb47e239211)
 2019-09-30 17:05:29 -07:00
Rohith Sharma K S
f9f0338104 Make upstream aware of 3.2.1 release. 2019-09-23 12:03:53 +05:30
Yuval Degani
e31594f20e HADOOP-16581. Revise ValueQueue to correctly replenish queues that go below the watermark (#1463) 
(cherry picked from dd0834696a694564af65a1355c9d13275f44df51)
 2019-09-20 09:58:39 -07:00
Takanobu Asanuma
b2072444ab HADOOP-16069. Support configure ZK_DTSM_ZK_KERBEROS_PRINCIPAL in ZKDelegationTokenSecretManager using principal with Schema /_HOST. Contributed by luhuachao. 
(cherry picked from commit 298f07abceb4bb854630c9058e1697d4ef3d2ae2)
 2019-09-20 13:57:40 +09:00
Kihwal Lee
7477f8d2e9 HADOOP-16582. LocalFileSystem's mkdirs() does not work as expected under viewfs. Contributed by Kihwal Lee 
(cherry picked from commit d4205dce176287e863f567b333e0d408bf51ae6d)
 2019-09-19 08:24:39 -05:00
Wei-Chiu Chuang
9c9920897d HDFS-14569. Result of crypto -listZones is not formatted properly. Contributed by hemanthboyina. 
(cherry picked from 0f2dad6679b7fc35474a3d33dc40b0db89bb1d80)
 2019-09-18 10:30:25 -07:00
Weiwei Yang
7ec229244a YARN-2255. YARN Audit logging not added to log4j.properties. Contributed by Aihua Xu. 
(cherry picked from commit f8c14326ee0026f2c57a4d061bff5fadec74b74d)
 2019-09-18 09:19:38 +08:00
Rohith Sharma K S
7d5bb2ebb7 Preparing for 3.2.2-SNAPSHOT development. 2019-09-07 08:52:08 +05:30
Jungtaek Lim (HeartSaVioR)
8cda5a6598
HADOOP-16255. Add ChecksumFs.rename(path, path, boolean) 
Contributed by Jungtaek Lim

Change-Id: If00a4d7d30456c08eb2b0f7e2b242197bc4ee05d
 2019-09-06 21:54:09 +01:00
Stephen O'Donnell
61fd1a74a1 HDFS-14706. Checksums are not checked if block meta file is less than 7 bytes. Contributed by Stephen O'Donnell. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 915cbc91c0a12cc7b4d3ef4ea951941defbbcb33)
 2019-09-02 10:06:39 -07:00
Akira Ajisaka
2d8799f4bc HADOOP-15832. Upgrade BouncyCastle to 1.60. Contributed by Robert Kanter. 2019-08-27 19:08:39 +00:00
David Mollitor
1b30b3fbe7 HADOOP-15246. SpanReceiverInfo - Prefer ArrayList over LinkedList. Contributed by David Mollitor. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 901652e26321832ac42ce54daa18ebc1813772b5)
 2019-08-18 18:43:52 -07:00
Chen Liang
d8a6098a96 HDFS-14611. Move handshake secret field from Token to BlockAccessToken. Contributed by Chen Liang. 2019-08-16 10:50:00 -07:00
Chen Liang
f507bc059d HDFS-13699. Add DFSClient sending handshake token to DataNode, and allow DataNode overwrite downstream QOP. Contributed by Chen Liang. 2019-08-16 10:50:00 -07:00
Chen Liang
ca15968d09 HDFS-13617. Allow wrapping NN QOP into token in encrypted message. Contributed by Chen Liang 2019-08-16 10:50:00 -07:00
Chen Liang
22d81f7fd0 HDFS-13566. Add configurable additional RPC listener to NameNode. Contributed by Chen Liang. 2019-08-16 10:49:59 -07:00