Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,624 Commits 326 Branches 371 Tags
Commit Graph

17624 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Sammi Chen
a700d05f72 Add 2.9.1 release notes and changes documents 
(cherry picked from commit f3f544b00475583c4c9fe52be0d2004390979bd0)
 2018-05-14 15:18:19 +08:00
Vrushali C
67255cd000 YARN-8247 Incorrect HTTP status code returned by ATSv2 for non-whitelisted users. Contributed by Rohith Sharma K S 
(cherry picked from commit 3c95ca4f21dcfcaabdd0694e7d005a45baba953f)
 2018-05-14 10:32:33 +05:30
Rushabh Shah
9519c2ce1e HADOOP-15441. Log kms url and token service at debug level. Contributed by Gabor Bota 
(cherry picked from commit a5449d36312c3de953e05ba8b3b5acf818a0c7e7)
 2018-05-12 12:26:25 -05:00
Jason Lowe
e628206c26 YARN-8244. TestContainerSchedulerQueuing.testStartMultipleContainers failed. Contributed by Jim Brennan 
(cherry picked from commit dc912994a1bcb511dfda32a0649cef0c9bdc47d3)
 2018-05-11 14:17:19 -05:00
Inigo Goiri
3b57a236ed HDFS-13542. TestBlockManager#testNeededReplicationWhileAppending fails due to improper cluster shutdown in TestBlockManager#testBlockManagerMachinesArray on Windows. Contributed by Anbang Hu. 
(cherry picked from commit d50c4d71dc42576f96ae5c268856fd1a7795f936)
 2018-05-11 09:49:21 -07:00
Weiwei Yang
4e46dc764f YARN-7003. DRAINING state of queues is not recovered after RM restart. Contributed by Tao Yang. 2018-05-11 11:04:50 +08:00
Inigo Goiri
054a17387c HADOOP-15454. TestRollingFileSystemSinkWithLocal fails on Windows. Contributed by Xiao Liang. 
(cherry picked from commit 1da8d4190d6e574347ab9d3380513e9401569573)
 2018-05-10 09:42:01 -07:00
Inigo Goiri
d72a4b6303 Revert "HDFS-13388. RequestHedgingProxyProvider calls multiple configured NNs all the time. Contributed by Jinglun." 
This reverts commit 1049e180a7dbc85e17c17240d231ed9a4659ed09.
 2018-05-10 09:24:45 -07:00
bibinchundatt
c198550cc7 YARN-8201. Skip stacktrace of few exception from ClientRMService. Contributed by Bilwa S T. 
(cherry picked from commit cc0310a5266c8b8351f338f5fc8087a203c68cac)
 2018-05-10 09:32:45 +05:30
Inigo Goiri
e466b4e2a7 HDFS-13537. TestHdfsHelper does not generate jceks path properly for relative path in Windows. Contributed by Xiao Liang. 
(cherry picked from commit 914b98a713f70667d4380ac752f6c4de931520d9)
 2018-05-09 16:53:09 -07:00
Inigo Goiri
1049e180a7 HDFS-13388. RequestHedgingProxyProvider calls multiple configured NNs all the time. Contributed by Jinglun. 
(cherry picked from commit 63803e7051dcfcd8e3d28a8b3217b7735a00b7d8)
 2018-05-09 13:56:58 -07:00
Yongjun Zhang
21bf02023c Revert "HADOOP-13055. Implement linkMergeSlash and linkFallback for ViewFileSystem" 
This reverts commit 218a21a0f0013523a80eddf28c5f152ccf3b2212.
 2018-05-08 22:22:06 -07:00
Rohith Sharma K S
f13ec73c29 YARN-8253. HTTPS Ats v2 api call fails with 'bad HTTP parsed'. Contributed by Charan Hebri. 
(cherry picked from commit 7450583721757b8af2945ebd9be1a9efed11444c)
 2018-05-08 12:32:25 +05:30
Xiao Chen
fe66ed7da5 HDFS-12981. renameSnapshot a Non-Existent snapshot to itself should throw error. Contributed by Kitti Nanasi. 
(cherry picked from commit 696a4be0daac00dd3bb64801d9fbe659aef9e089)
 2018-05-07 15:39:33 -07:00
Xiao Chen
08cb6548c0 Revert "HADOOP-14445. Delegation tokens are not shared between KMS instances. Contributed by Xiao Chen and Rushabh S Shah." 
This reverts commit 583fa6ed48ad3df40bcaa9c591d5ccd07ce3ea81.

 Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/kms/KMSTokenRenewer.java
	hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMS.java

(cherry picked from commit f9692eecf0029a1682ffda48551ac45507144ff0)

 Conflicts:
	hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMS.java

(cherry picked from commit 752092a86082f77fcd4e8a28275078bbd123c801)
 2018-05-07 13:56:26 -07:00
Xiao Chen
22384580b2 Revert "HDFS-13430. Fix TestEncryptionZonesWithKMS failure due to HADOOP-14445." 
This reverts commit 9f19c3b03eac43e549a33e5c8e92c7a39432d736.
 2018-05-07 13:56:26 -07:00
Weiwei Yang
a0b7abf278 YARN-8025. UsersManangers#getComputedResourceLimitForActiveUsers throws NPE due to preComputedActiveUserLimit is empty. Contributed by Tao Yang. 
(Cherry picked from commit 67f239c42f676237290d18ddbbc9aec369267692)
 2018-05-07 13:59:50 +08:00
Rushabh Shah
d9fae993b4 HADOOP-14841 Kms client should disconnect if unable to get output stream from connection. Contributed by Rushabh S Shah 
(cherry picked from commit 4cdbdce752e192b45c2b9756c2d4bd24ceffdabd)
 2018-05-04 15:43:24 -05:00
Inigo Goiri
21442dcaa3 HDFS-13525. RBF: Add unit test TestStateStoreDisabledNameservice. Contributed by Yiqun Lin. 
(cherry picked from commit a3b416f69dc3965f247603f657df33bd74fd723e)
 2018-05-03 11:26:00 -07:00
Rohith Sharma K S
317deaafc4 YARN-8217. RmAuthenticationFilterInitializer and TimelineAuthenticationFilterInitializer should use Configuration.getPropsWithPrefix instead of iterator. Contributed by Suma Shivaprasad. 
(cherry picked from commit ee2ce923a922bfc3e89ad6f0f6a25e776fe91ffb)
 2018-05-03 18:21:11 +05:30
Daniel Templeton
184ecec29c HDFS-13481. TestRollingFileSystemSinkWithHdfs#testFlushThread: test failed intermittently 
(Contributed by Gabor Bota via Daniel Templeton)

Change-Id: I9921981dfa69669fe7912dd2a31ae8b638283204
(cherry picked from commit 87c23ef643393c39e8353ca9f495b0c8f97cdbd9)
(cherry picked from commit 57e41154365aa260ad2b2117779a77196c918eb0)
 2018-05-02 17:18:59 -07:00
Weiwei Yang
c2ed611885 YARN-8222. Fix potential NPE when gets RMApp from RM context. Contributed by Tao Yang. 
(Cherry picked from commit 251f528814c4a4647cac0af6effb9a73135db180)
 2018-05-02 18:07:34 +08:00
Yiqun Lin
01e924a2ce
YARN-6385. Fix checkstyle warnings in TestFileSystemApplicationHistoryStore 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 3265b55119d39ecbda6d75be04a9a1bf59c631f1)
 2018-05-02 18:16:09 +09:00
Jason Lowe
e5ef506dc1
HADOOP-15406. hadoop-nfs dependencies for mockito and junit are not test scope 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit e07156e8b07552b877a22565641465e211144f6f)
 2018-05-02 17:32:26 +09:00
Bibin A Chundatt
e6e2146ae0
MAPREDUCE-7073. Optimize TokenCache#obtainTokensForNamenodesInternal 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 1a95a4524a8c6c7be601ce8b92640a6a76164a2c)
 2018-05-02 16:35:18 +09:00
Yiqun Lin
221db515aa HDFS-13488. RBF: Reject requests when a Router is overloaded. Contributed by Inigo Goiri. 
(cherry picked from commit 37269261d1232bc71708f30c76193188258ef4bd)
(cherry picked from commit 5fef28d0d4b27c5df31d325650e46f5ab5f5630f)
 2018-05-02 15:06:53 +08:00
Inigo Goiri
d5aa4b9a9a HDFS-13503. Fix TestFsck test failures on Windows. Contributed by Xiao Liang. 
(cherry picked from commit 9e2cfb2d3f1a18984d07c81f9c46626dd842402a)
 2018-05-01 08:14:14 -07:00
Weiwei Yang
f12c78120e YARN-8212. Pending backlog for async allocation threads should be configurable. Contributed by Tao Yang. 
(Cherry picked from commit 2d319e37937c1e20c6a7dc4477ef88defd1f8464)
 2018-05-01 10:19:08 +08:00
Inigo Goiri
0e31e014f3 HDFS-13283. Percentage based Reserved Space Calculation for DataNode. Contributed by Lukas Majercak. 2018-04-30 15:22:30 -07:00
Inigo Goiri
af9bc982ce HDFS-13509. Bug fix for breakHardlinks() of ReplicaInfo/LocalReplica, and fix TestFileAppend failures on Windows. Contributed by Xiao Liang. 
(cherry picked from commit eb7fe1d588de903be2ff6e20384c25c184881532)
 2018-04-28 09:06:58 -07:00
Inigo Goiri
d83254312b HDFS-13508. RBF: Normalize paths (automatically) when adding, updating, removing or listing mount table entries. Contributed by Ekanth S. 
(cherry picked from commit 484440602c5b69fbd8106010603c61ae051056dd)
 2018-04-27 16:29:23 -07:00
Inigo Goiri
71e52c936d HDFS-13434. RBF: Fix dead links in RBF document. Contributed by Chetna Chaudhari. 
(cherry picked from commit f469628bba350ba79bc6a0d38f9dc1cb5eb65c77)
 2018-04-27 15:16:03 -07:00
Wangda Tan
c9f7209662 YARN-8210. AMRMClient logging on every heartbeat to track updation of AM RM token causes too many log lines to be generated in AM logs. (Suma Shivaprasad via wangda) 
Change-Id: I70edd6e301fd5e78d479e1882aedc9332a0827aa
(cherry picked from commit b1833d9ba2c078582161da45ac392dd5c361dcdf)
 2018-04-27 13:18:35 -07:00
Rohith Sharma K S
8d3e6f99c1 YARN-8221. RMWebServices also need to honor yarn.resourcemanager.display.per-user-apps. Contributed by Sunil G. 
(cherry picked from commit ef3ecc308dbea41c6a88bd4d16739c7bbc10cdda)
 2018-04-27 22:59:48 +05:30
Sunil G
f1f3cbb4df YARN-8004. Add unit tests for inter queue preemption for dominant resource calculator. Contributed by Zian Chen. 
(cherry picked from commit 71220d218db59cab0b03bbba427e5e9ef5b3003c)
 2018-04-27 10:43:24 +05:30
Sunil G
1d5bcb8ace YARN-8215. ATS v2 returns invalid YARN_CONTAINER_ALLOCATED_HOST_HTTP_ADDRESS from NM. Contributed by Rohith Sharma K S. 
(cherry picked from commit 7fc09c4f7255645025ddf902dff70a17092ea039)
 2018-04-27 10:34:52 +05:30
Sunil G
62921db4b9 YARN-8205. Application State is not updated to ATS if AM launching is delayed. Contributed by Rohith Sharma K S. 
(cherry picked from commit 1634de0fc1430d86b7688d16259a81462fba482f)
 2018-04-27 10:28:41 +05:30
Yiqun Lin
1c65529dc3 HDFS-13499. RBF: Show disabled name services in the UI. Contributed by Inigo Goiri. 
(cherry picked from commit 0a293de00d5f123c7c8248459aa8ae88a45f39b1)
 2018-04-27 10:40:13 +08:00
Inigo Goiri
a71d184f4f HDFS-13326. RBF: Improve the interfaces to modify and view mount tables. Contributed by Gang Li. 
(cherry picked from commit c394051a3d4d9d531f418503cb519606ae2b2e69)
 2018-04-26 13:13:02 -07:00
Sunil G
8e0e33b0fd MAPREDUCE-7042. Killed MR job data does not move to mapreduce.jobhistory.done-dir when ATS v2 is enabled. Contributed by Rohith Sharma K S. 
(cherry picked from commit 583a2f4edfdf71c32001cc2f2a03d16f9371d533)
 2018-04-26 19:08:20 +05:30
Wangda Tan
e6de818ef6 HADOOP-15411. AuthenticationFilter should use Configuration.getPropsWithPrefix instead of iterator. (Suma Shivaprasad via wangda) 
Change-Id: Ia19458a43aceb4ffdb1e8eccc519655235d92ba9
(cherry picked from commit 3559d8b1dacf5cf207424de37cb6ba8865d26ffe)
 2018-04-25 22:21:51 -07:00
Yiqun Lin
caffd538af HDFS-13484. RBF: Disable Nameservices from the federation. Contributed by Inigo Goiri. 
(cherry picked from commit 30fef0bf1e5c8c0ca073df99ad9b33cb0e4431a5)
 2018-04-25 15:39:16 +08:00
Wangda Tan
baa003035d YARN-8183. Fix ConcurrentModificationException inside RMAppAttemptMetrics#convertAtomicLongMaptoLongMap. (Suma Shivaprasad via wangda) 
Change-Id: I347871d672001653a3afe2e99adefd74e0d798cd
(cherry picked from commit bb3c504764f807fccba7f28298a12e2296f284cb)
(cherry picked from commit 3043a93d461fd8b9ccc2ff4b8d17e5430ed77615)
 2018-04-24 17:49:32 -07:00
Junping Du
51194439ba YARN-7598. Document how to use classpath isolation for aux-services in YARN. Contributed by Xuan Gong. 
(cherry picked from commit 56788d759f47b4b158617701f543a9dcb4df69cd)
 2018-04-24 18:30:26 +08:00
Yiqun Lin
14cfcf6b71 HDFS-13490. RBF: Fix setSafeMode in the Router. Contributed by Inigo Goiri. 
(cherry picked from commit b06601acce38ed60b726b99e2830f38a1ee3d2b5)
 2018-04-24 11:28:06 +08:00
Inigo Goiri
7bb2709c4a HDFS-13336. Test cases of TestWriteToReplica failed in windows. Contributed by Xiao Liang. 
(cherry picked from commit df92a17e02fe86279a6f4e413719d0a465b50837)
 2018-04-23 19:13:25 -07:00
Robert Kanter
101d9005dd HADOOP-15390. Yarn RM logs flooded by DelegationTokenRenewer trying to renew KMS tokens (xiaochen via rkanter) 
(cherry picked from commit 7ab08a9c37a76edbe02d556fcfb2e637f45afc21)
 2018-04-23 15:53:53 -07:00
Lei Xu
463fcfb50a Update CHANGES, RELEASENOTES, and jdiff for 3.0.2 release. 
(cherry picked from commit f6ecb76d0b919b9836600fe28ec9e637b223cd54)

Conflicts:
	hadoop-project-dist/pom.xml
 2018-04-23 14:17:29 -07:00
Chris Douglas
956ab12ede HDFS-13408. MiniDFSCluster to support being built on randomized base directory. Contributed by Xiao Liang 
(cherry picked from commit f411de6a79a0a87f03c09366cfe7a7d0726ed932)
(cherry picked from commit cf272c5179a9cb4b524016c0fca7c69c9eaa92f1)
 2018-04-23 11:15:15 -07:00
Arpit Agarwal
e8f62357c0 HDFS-13433. webhdfs requests can be routed incorrectly in federated cluster. Contributed by Arpit Agarwal. 2018-04-23 10:09:16 -07:00