Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
19,448 Commits 326 Branches 371 Tags
Commit Graph

4922 Commits

Author SHA1 Message Date
Sunil G
57c499ef19 YARN-9949. Add missing queue configs for root queue in RMWebService#CapacitySchedulerInfo. 
Contributed by Prabhu Joseph.
 2019-11-27 23:01:16 +05:30
Szilard Nemeth
1172ebd6e8 YARN-9937. Add missing queue configs in RMWebService#CapacitySchedulerQueueInfo. 
Contributed by Prabhu Joseph.
 2019-11-27 22:30:10 +05:30
Szilard Nemeth
80a97e6ac2 YARN-9011. Race condition during decommissioning. Contributed by Peter Bacsko 2019-11-26 14:27:55 +01:00
HUAN-PING SU
fb2df5234d
YARN-9966. Code duplication in UserGroupMappingPlacementRule (#1709) 
(cherry picked from commit f8e36e03b4e65f173bf2330715dceafdb6111190)
 2019-11-25 15:58:09 +09:00
Szilard Nemeth
df157211c9 YARN-9968. Public Localizer is exiting in NodeManager due to NullPointerException. Contributed by Tarun Parimi 2019-11-22 13:00:18 +01:00
Tao Yang
c95e772d5d YARN-9838. Fix resource inconsistency for queues when moving app with reserved container to another queue. Contributed by jiulongzhu. 2019-11-22 16:10:07 +08:00
Eric Yang
1669a5c1cb YARN-9983. Fixed typo in YARN Service overview. 
Contributed by Denes Gerencser
 2019-11-19 14:18:38 -05:00
Sunil G
9a51380208 YARN-9873. Version Number for each Scheduler Config Change. Contributed by Prabhu Joseph. 
(cherry picked from commit be901f49628a553cfe6a3942478cb528cce2b266)
 2019-11-19 17:17:45 +05:30
Abhishek Modi
96e07b7e97 YARN-9791. Queue Mutation API does not allow to remove a config. Contributed by Prabhu Joseph. 
(cherry picked from commit 751b5a1ac871734c46cb934d100a5367f1dbf791)
 2019-11-19 16:41:04 +05:30
Sunil G
dab4cc6320 YARN-9909. Offline Format of YarnConfigurationStore. Contributed by Prabhu Joseph. 2019-11-19 16:19:19 +05:30
Sunil G
9b17fa7091 YARN-9950. Unset Ordering Policy of Leaf/Parent queue converted from Parent/Leaf queue respectively. Contributed by Prabhu Joseph. 
(cherry picked from commit 51e7d1b37eb06165657befdae82c015055c58334)
 2019-11-19 15:43:19 +05:30
Sunil G
f24d1fac35 YARN-9984. FSPreemptionThread can cause NullPointerException while app is unregistered with containers running on a node. Contributed by Wilfred Spiegelenburg. 
(cherry picked from commit 215f2052fc3b7e366e8bd1bd332663966fa9206c)
 2019-11-19 14:05:51 +05:30
Prabhu Joseph
d271d174ca YARN-9900. Revert Invalid Config and Refresh Support in SchedulerConfig Format. 
Signed-off-by: prabhujoseph <pjoseph@cloudera.com>
 2019-11-18 15:02:59 +05:30
Jonathan Hung
4d274f60bc Make upstream aware of 2.10.0 release 
(cherry picked from commit 7663db59c097c82eeed2df7a91168a4d7123c96b)
(cherry picked from commit 5d2ffcc7aac608746946dc2ed5711549a01a2055)
 2019-10-30 21:00:55 -07:00
Eric Badger
f5a3a9e138 YARN-9914. Use separate configs for free disk space checking for full and not-full disks. Contributed by Jim Brennan 
(cherry picked from commit eef34f2d87a75e16b2cca870d99a5e1e28c31d9b)
 2019-10-25 17:17:14 +00:00
Zhankun Tang
7175fc91d0 YARN-9921. Issue in PlacementConstraint when YARN Service AM retries allocation on component failure. Contributed by Tarun Parimi 2019-10-24 10:07:40 +08:00
Eric E Payne
d8b4348036 YARN-9915: Fix FindBug issue in QueueMetrics. Contributed by Prabhu Joseph. 
(cherry picked from commit 83d148074f9299de02d5c896a3ed4e11292cba73)
 2019-10-21 21:14:23 +00:00
Eric E Payne
889255576b YARN-9773: Add QueueMetrics for Custom Resources. Contributed by Manikandan R. 
(cherry picked from commit a5034c7988b6bf54bb7dab208100a2d205e3929e)
 2019-10-16 21:23:38 +00:00
Haibo Chen
a70c6e9665 YARN-8842. Expose metrics for custom resource types in QueueMetrics. (Contributed by Szilard Nemeth) 
(cherry picked from commit 84e22a6af46db2859d7d2caf192861cae9b6a1a8)
 2019-10-15 22:20:56 +00:00
Haibo Chen
4ec409fe09 YARN-8750. Refactor TestQueueMetrics. (Contributed by Szilard Nemeth) 
(cherry picked from commit e60b797c88541f94cecc7fdbcaad010c4742cfdb)
 2019-10-15 15:41:20 +00:00
Szilard Nemeth
c44902212a YARN-8453. Additional Unit tests to verify queue limit and max-limit with multiple resource types. Contributed by Adam Antal 2019-10-15 13:25:57 +02:00
Szilard Nemeth
aabc18a777 Revert "YARN-9128. Use SerializationUtils from apache commons to serialize / deserialize ResourceMappings. Contributed by Zoltan Siegl" 
This reverts commit 73bc8ef9b86cb9b3ee9bbc1786b49600e0ee600d.
 2019-10-09 19:59:02 +02:00
Szilard Nemeth
57e88a63cf YARN-9552. FairScheduler: NODE_UPDATE can cause NoSuchElementException. Contributed by Peter Bacsko. 2019-10-09 14:19:56 +02:00
Szilard Nemeth
73bc8ef9b8 YARN-9128. Use SerializationUtils from apache commons to serialize / deserialize ResourceMappings. Contributed by Zoltan Siegl 
(cherry picked from commit 6f1ab9516869dbfdee034f4a17bccc379a39fc31)
(cherry picked from commit 42177e8b786efd7fa1365b68617382de022a5146)
 2019-10-09 13:28:44 +02:00
Szilard Nemeth
d1415b66b7 YARN-9356. Add more tests to ratio method in TestResourceCalculator. Contributed by Zoltan Siegl 
(cherry picked from commit 35f093f5b3f310af49a8bb476e62173862f5af51)
(cherry picked from commit 0ddb48a3038d9c5132efd9f48110f9d23c53ac67)
 2019-10-09 13:13:51 +02:00
Jonathan Hung
0dc85bc182 YARN-9760. Support configuring application priorities on a workflow level. Contributed by Varun Saxena 2019-10-08 11:18:52 -07:00
Szilard Nemeth
2c8cfb4f64 YARN-6715. Fix documentation about NodeHealthScriptRunner. Contributed by Peter Bacsko 2019-10-08 17:34:44 +02:00
Sunil G
933d81f0ce YARN-9801. SchedConfCli does not work wiwith https mode. Contributed by Prabhu Joseph 
(cherry picked from commit 99cd7572f113cbbb8f5ef89af60af3734fe8cfa7)
 2019-10-01 20:08:12 +05:30
Sunil G
71792f2122 YARN-9864. Format CS Configuration present in Configuration Store. Contributeed by Prabhu Joseph 2019-10-01 20:03:35 +05:30
bibinchundatt
6529a30d9e YARN-9858. Optimize RMContext getExclusiveEnforcedPartitions. Contributed by Jonathan Hung. 2019-10-01 16:06:38 +05:30
Jonathan Hung
95ec7050b5 Addendum to YARN-9730. Support forcing configured partitions to be exclusive based on app node label 
(cherry picked from commit d86a1acc866cbda845fb3896dc824baf12217383)
(cherry picked from commit f4f210d2e57e49999579efb7db039cc623798267)
 2019-09-25 17:49:52 -07:00
Jonathan Hung
783cbced1d YARN-9730. Support forcing configured partitions to be exclusive based on app node label 
(cherry picked from commit 73a044a63822303f792183244e25432528ecfb1e)
(cherry picked from commit dd094d79023f6598e47146166aa8c213e03d41b7)
 2019-09-24 13:52:14 -07:00
Jonathan Hung
6a1d2d56bd YARN-9762. Add submission context label to audit logs. Contributed by Manoj Kumar 
(cherry picked from commit 3d78b1223d3fdc29d500803cefd2931b54f44928)
(cherry picked from commit a1fa9a8a7f79a1a711cd881b526724b502e03456)
 2019-09-23 13:13:09 -07:00
Sunil G
45cf3de2e9 YARN-9833. Race condition when DirectoryCollection.checkDirs() runs during container launch. Contributed by Peter Bacsko. 
(cherry picked from commit c474e24c0b73c0f52a7d5af2495355f4a0799344)
 2019-09-18 09:23:46 +05:30
Weiwei Yang
1978317068 YARN-2255. YARN Audit logging not added to log4j.properties. Contributed by Aihua Xu. 
(cherry picked from commit f8c14326ee0026f2c57a4d061bff5fadec74b74d)
 2019-09-18 09:20:37 +08:00
Eric Yang
dae22c962d YARN-9837. Fixed reading YARN Service JSON spec file larger than 128k. 
Contributed by Tarun Parimi
 2019-09-17 13:17:13 -04:00
Jonathan Hung
d75693bd6e YARN-9824. Fall back to configured queue ordering policy class name 
(cherry picked from commit f8f8598ea5e86330fa5b553963f5632da286409d)
(cherry picked from commit 1dbf87c9ff33b5e2f6cbb0ce8efd348c4314a0e2)
 2019-09-10 15:31:58 -07:00
Jonathan Hung
80735a15a5 YARN-8541 (branch-3.1 addendum): RM startup failure on recovery after user deletion 2019-09-09 20:15:42 -07:00
bibinchundatt
aee8fb567b YARN-8948. PlacementRule interface should be for all YarnSchedulers. Contributed by Bibin A Chundatt. 
(cherry picked from commit a68d766e876631d7ee2e1a6504d4120ba628d178)
(cherry picked from commit e10050678d0bcd0e4384f2a684daf8c18e30ecb9)
 2019-09-09 20:00:46 -07:00
Wangda Tan
9e8ff94d16 YARN-8361. Change App Name Placement Rule to use App Name instead of App Id for configuration. (Zian Chen via wangda) 
Change-Id: I17e5021f8f611a9c5e3bd4b38f25e08585afc6b1
(cherry picked from commit a2e49f41a8bcc03ce0a85b294d0b86fee7e86f31)
 2019-09-09 20:00:33 -07:00
Wangda Tan
81d63d5ea1 YARN-8016. Refine PlacementRule interface and add a app-name queue mapping rule as an example. (Zian Chen via wangda) 
Change-Id: I35caf1480e0f76f5f3a53528af09312e39414bbb
(cherry picked from commit a90471b3e65326cc18ed31fe21aef654833b5883)
 2019-09-09 19:59:50 -07:00
Jonathan Hung
0e88bcd8e6 YARN-9820. RM logs InvalidStateTransitionException when app is submitted. Contributed by Prabhu Joseph 2019-09-09 00:27:33 -07:00
Jonathan Hung
080fc6d943 YARN-9764. Print application submission context label in application summary. Contributed by Manoj Kumar 
(cherry picked from commit 43e389b9801e09741fdf78fef067b8ac60f691c8)
(cherry picked from commit 45220d115797663b8749980b78a61bafcb2344f1)
 2019-09-08 19:15:12 -07:00
Wangda Tan
0ee7d09138 YARN-9813. RM does not start on JDK11 when UIv2 is enabled. (Adam Antal/Eric Yang via wangda) 
Change-Id: I18b8edc930b2efa0652f59c246931ad0d46827f3
(cherry picked from commit 34b82e6da0a471010cdae613ba39487889d79369)
 2019-09-06 19:19:59 -07:00
Tao Yang
36c1caced5 YARN-8995. Log events info in AsyncDispatcher when event queue size cumulatively reaches a certain number every time(addendum). Contributed by Jonathan Hung. 2019-09-07 08:19:02 +08:00
Tao Yang
1f6f4a2457 YARN-9795. ClusterMetrics to include AM allocation delay. Contributed by Fengnan Li. 2019-09-07 07:54:30 +08:00
Jonathan Hung
980a922481 YARN-9763. Print application tags in application summary. Contributed by Manoj Kumar 2019-09-06 10:52:37 -07:00
Jonathan Hung
11f6e3bc41 YARN-9761. Allow overriding application submissions based on server side configs. Contributed by Pralabh Kumar 2019-09-06 10:02:20 -07:00
Billie Rinaldi
619bd1e876 YARN-9718. Fixed yarn.service.am.java.opts shell injection. Contributed by Eric Yang 
(cherry picked from commit 2e2e5401f297545181323b126a69eaa2239afb02)
 2019-09-05 14:19:05 -07:00
Jonathan Hung
37d1f8c81e YARN-9810. Add queue capacity/maxcapacity percentage metrics. Contributed by Shubham Gupta 
(cherry picked from commit 0ccf4b0fe16a8c879a560f2a612a3185eb2df72b)
(cherry picked from commit cb806988d72bde1f9837c9e0fb82a3a6c032542c)
 2019-09-05 14:06:09 -07:00