Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
19,272 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

19272 Commits

Author SHA1 Message Date
Jonathan Hung d75693bd6e YARN-9824. Fall back to configured queue ordering policy class name 
(cherry picked from commit f8f8598ea5)
(cherry picked from commit 1dbf87c9ff)
 2019-09-10 15:31:58 -07:00
Jonathan Hung 80735a15a5 YARN-8541 (branch-3.1 addendum): RM startup failure on recovery after user deletion 2019-09-09 20:15:42 -07:00
bibinchundatt aee8fb567b YARN-8948. PlacementRule interface should be for all YarnSchedulers. Contributed by Bibin A Chundatt. 
(cherry picked from commit a68d766e87)
(cherry picked from commit e10050678d)
 2019-09-09 20:00:46 -07:00
Wangda Tan 9e8ff94d16 YARN-8361. Change App Name Placement Rule to use App Name instead of App Id for configuration. (Zian Chen via wangda) 
Change-Id: I17e5021f8f611a9c5e3bd4b38f25e08585afc6b1
(cherry picked from commit a2e49f41a8)
 2019-09-09 20:00:33 -07:00
Wangda Tan 81d63d5ea1 YARN-8016. Refine PlacementRule interface and add a app-name queue mapping rule as an example. (Zian Chen via wangda) 
Change-Id: I35caf1480e0f76f5f3a53528af09312e39414bbb
(cherry picked from commit a90471b3e6)
 2019-09-09 19:59:50 -07:00
Akira Ajisaka 244385d7eb
Revert "HADOOP-16061. Upgrade Yetus to 0.10.0" 
This reverts commit 565751e739.
 2019-09-09 18:58:38 +09:00
Jonathan Hung 0e88bcd8e6 YARN-9820. RM logs InvalidStateTransitionException when app is submitted. Contributed by Prabhu Joseph 2019-09-09 00:27:33 -07:00
Jonathan Hung 080fc6d943 YARN-9764. Print application submission context label in application summary. Contributed by Manoj Kumar 
(cherry picked from commit 43e389b980)
(cherry picked from commit 45220d1157)
 2019-09-08 19:15:12 -07:00
Wangda Tan 0ee7d09138 YARN-9813. RM does not start on JDK11 when UIv2 is enabled. (Adam Antal/Eric Yang via wangda) 
Change-Id: I18b8edc930b2efa0652f59c246931ad0d46827f3
(cherry picked from commit 34b82e6da0)
 2019-09-06 19:19:59 -07:00
Tao Yang 36c1caced5 YARN-8995. Log events info in AsyncDispatcher when event queue size cumulatively reaches a certain number every time(addendum). Contributed by Jonathan Hung. 2019-09-07 08:19:02 +08:00
Tao Yang 1f6f4a2457 YARN-9795. ClusterMetrics to include AM allocation delay. Contributed by Fengnan Li. 2019-09-07 07:54:30 +08:00
Tao Yang 3cc21350b5 YARN-9817. Fix failing testcases due to not initialized AsyncDispatcher - ArithmeticException: / by zero. Contributed by Prabhu Joseph. 2019-09-07 07:40:50 +08:00
Jonathan Hung 980a922481 YARN-9763. Print application tags in application summary. Contributed by Manoj Kumar 2019-09-06 10:52:37 -07:00
Jonathan Hung 11f6e3bc41 YARN-9761. Allow overriding application submissions based on server side configs. Contributed by Pralabh Kumar 2019-09-06 10:02:20 -07:00
Xiaoyu Yao ed253c4412
HDFS-14633. The StorageType quota and consume in QuotaFeature is not handled for rename. Contributed by Jinglun. 
(cherry picked from commit 62d71fbac3789c7d484bc76ced9ec7fa6ff94de1)
 2019-09-05 16:52:09 -07:00
Xiaoyu Yao fbdfb36fdf
Revert "HDFS-14633. The StorageType quota and consume in QuotaFeature is not handled for rename. Contributed by Jinglun." 
This reverts commit 29e6a97faa.
 2019-09-05 16:52:09 -07:00
Billie Rinaldi 619bd1e876 YARN-9718. Fixed yarn.service.am.java.opts shell injection. Contributed by Eric Yang 
(cherry picked from commit 2e2e5401f2)
 2019-09-05 14:19:05 -07:00
Jonathan Hung 37d1f8c81e YARN-9810. Add queue capacity/maxcapacity percentage metrics. Contributed by Shubham Gupta 
(cherry picked from commit 0ccf4b0fe1)
(cherry picked from commit cb806988d72bde1f9837c9e0fb82a3a6c032542c)
 2019-09-05 14:06:09 -07:00
Tao Yang d74c069427 YARN-8995. Log events info in AsyncDispatcher when event queue size cumulatively reaches a certain number every time. Contributed by zhuqi. 2019-09-05 16:54:55 +08:00
Ayush Saxena 8b702b2b4e HADOOP-16025. Update the year to 2019. 
Contributed by Ayush Saxena.

(cherry picked from commit cb26f15428)
 2019-09-04 17:08:16 +08:00
Zhankun Tang ef79d98788 Preparing for 3.1.4 development 2019-09-04 16:11:36 +08:00
Zhankun Tang fff4fbc957 YARN-9785. Fix DominantResourceCalculator when one resource is zero. Contributed by Bibin A Chundatt, Sunil Govindan, Bilwa S T. 2019-09-04 12:05:29 +08:00
Xiaoyu Yao 29e6a97faa
HDFS-14633. The StorageType quota and consume in QuotaFeature is not handled for rename. Contributed by Jinglun. 
(cherry picked from commit 62d71fbac3789c7d484bc76ced9ec7fa6ff94de1)
 2019-09-03 08:58:59 -07:00
bibinchundatt 3210d1e993 YARN-9797. LeafQueue#activateApplications should use resourceCalculator#fitsIn. Contributed by Bilwa S T. 
(cherry picked from commit 03489124ea)
 2019-09-03 11:56:19 +05:30
Stephen O'Donnell 63b18d764a HDFS-14706. Checksums are not checked if block meta file is less than 7 bytes. Contributed by Stephen O'Donnell. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 915cbc91c0)
(cherry picked from commit 2c7f49743fba587e32398c81412db9eac426cf98)
 2019-09-02 09:58:48 -07:00
Wei-Chiu Chuang 8ffaef78da Revert "HDFS-14706. Checksums are not checked if block meta file is less than 7 bytes. Contributed by Stephen O'Donnell." 
This reverts commit d42b48d4d0.
 2019-09-02 09:58:32 -07:00
Akira Ajisaka 3c9d2f5317 YARN-9162. Fix TestRMAdminCLI#testHelp. Contributed by Ayush Saxena. 
(cherry picked from commit 5db7c49062)
(cherry picked from commit a453f38015)
 2019-08-30 17:55:35 -07:00
Istvan Fajth e3c01e174c HDFS-8178. QJM doesn't move aside stale inprogress edits files. Contributed by Istvan Fajth. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-08-30 13:13:48 -07:00
Brian Grunkemeyer 7e9d324bcd HADOOP-15998. Ensure jar validation works on Windows. 
* use a different path separator for artifacts to test
* fail if and commands fail (including the jar listing)
* handle different line endings from jar listing
* make sure we have bash 3.1+

Co-authored-by: Sean Busbey <busbey@apache.org>
Signed-off-by: Abhishek Modi <abmodi@apache.org>
Signed-off-by: Rohith Sharma K S <rohithsharmaks@apache.org>
(cherry picked from commit b86582ce2b8b00c128add054bcb80951c38cb3d3)
 2019-08-29 23:08:39 -05:00
Ayush Saxena e44693953d HDFS-12212. Options.Rename.To_TRASH is considered even when Options.Rename.NONE is specified. Contributed by Vinayakumar B. 2019-08-30 08:59:52 +05:30
leosunli 0011136e66 Make DataNodePeerMetrics#minOutlierDetectionSamples configurable (#1314). Contributed by Lisheng Sun. 
Signed-off-by: sunlisheng <sunlisheng@xiaomi.com>
(cherry picked from commit 915271245b)
(cherry picked from commit b516d2ea7b)
 2019-08-29 18:46:25 -07:00
Stephen O'Donnell d42b48d4d0 HDFS-14706. Checksums are not checked if block meta file is less than 7 bytes. Contributed by Stephen O'Donnell. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 7bebad61d9)
(cherry picked from commit 9c0d6e1657)
 2019-08-29 18:26:14 -07:00
Eric E Payne 51896ff7e6 YARN-9756: Create metric that sums total memory/vcores preempted per round. Contributed by Manikandan R (manirajv06). 
(cherry picked from commit d562050cec)
 2019-08-28 21:05:23 +00:00
Jonathan Hung f73842780e YARN-9438. launchTime not written to state store for running applications 
(cherry picked from commit 9568656cd21d9c02168e18ce35c6726077bbf3a1)
(cherry picked from commit 0c498de6e87c6bdc959afa31deb03d0907e0e1a1)
 2019-08-27 15:45:42 -07:00
Jonathan Hung 79364a9c53 HDFS-14779. Fix logging error in TestEditLog#testMultiStreamsLoadEditWithConfMaxTxns 
(cherry picked from commit 8ab7020e64)
(cherry picked from commit d255efa5f8)
 2019-08-27 10:31:21 -07:00
Jonathan Hung 6baa0d1e4d YARN-9775. RMWebServices /scheduler-conf GET returns all hadoop configurations for ZKConfigurationStore. Contributed by Prabhu Joseph 
(cherry picked from commit 8660e48ca1)
(cherry picked from commit e4249c3202)
 2019-08-26 15:55:11 -07:00
bibinchundatt eb618e4f22 YARN-9642. Fix Memory Leak in AbstractYarnScheduler caused by timer. Contributed by Bibin A Chundatt. 
(cherry picked from commit d3ce53e507)
 2019-08-26 23:25:16 +05:30
Chen Liang d2bb764ffa HDFS-14674. [SBN read] Got an unexpected txid when tail editlog. Contributed by wangzhaohui. 2019-08-23 12:29:23 -07:00
Erik Krogen 9dc921f5e5 HDFS-13977. Override shouldForceSync in QuorumOutputStream to allow for proper auto-sync behavior. Contributed by Erik Krogen. 
(cherry picked from d699022fce)
(cherry picked from 2408c2491f)
 2019-08-23 11:48:51 -07:00
Chen Liang 0839cf1d4e HDFS-14611. Move handshake secret field from Token to BlockAccessToken. Contributed by Chen Liang. 2019-08-23 10:11:22 -07:00
Chen Liang ef9b0a1d9a HDFS-13699. Add DFSClient sending handshake token to DataNode, and allow DataNode overwrite downstream QOP. Contributed by Chen Liang. 2019-08-23 10:11:22 -07:00
Chen Liang 3135cc35e3 HDFS-13617. Allow wrapping NN QOP into token in encrypted message. Contributed by Chen Liang 2019-08-23 10:11:22 -07:00
Chen Liang 3835a6c4e8 HDFS-13566. Add configurable additional RPC listener to NameNode. Contributed by Chen Liang. 2019-08-23 10:11:22 -07:00
Chen Liang 14265b8f03 HDFS-13547. Add ingress port based sasl resolver. Contributed by Chen Liang. 2019-08-23 10:11:22 -07:00
Akira Ajisaka 83f67cca60
HDFS-14396. Failed to load image from FSImageFile when downgrade from 3.x to 2.x. Contributed by Fei Hui. 
(cherry picked from commit bd7baea5a5)
 2019-08-23 15:29:25 +09:00
Akira Ajisaka b40aa71cf0
HDFS-13596. NN restart fails after RollingUpgrade from 2.x to 3.x. Contributed by Fei Hui. 
(cherry picked from commit abc8fde4ca)
 2019-08-23 15:08:34 +09:00
Xiao Chen a5690b29a7
HDFS-13642. Creating a file with block size smaller than EC policy's cell size should fail. 
(cherry picked from commit cf4108313d)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStream.java
 2019-08-23 15:03:57 +09:00
Akira Ajisaka 42c6119e47
YARN-9774. Fix order of arguments for assertEquals in TestSLSUtils. Contributed by Nikhil Navadiya. 
(cherry picked from commit 84b1982060)
 2019-08-23 14:40:34 +09:00
Ayush Saxena 900384cf57 HDFS-14276. [SBN read] Reduce tailing overhead. Contributed by Wei-Chiu Chuang. 2019-08-22 22:54:50 +05:30
Akira Ajisaka 565751e739
HADOOP-16061. Upgrade Yetus to 0.10.0 
This closes #1298

(cherry picked from commit 5e156b9dde)

 Conflicts:
	dev-support/bin/yetus-wrapper
 2019-08-22 13:05:19 +09:00