Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,865 Commits 326 Branches 371 Tags
Commit Graph

1857 Commits

Author SHA1 Message Date
Wangda Tan
6a3929f2b2 YARN-6608. Backport all SLS improvements from trunk to branch-2. (Carlo Curino via wangda) 2017-10-20 11:22:26 -07:00
vrushali
3a5f1a6c26 Fixing checktyle warnings during rebase 2017-10-18 02:06:49 +05:30
Wangda Tan
c0c124c2bf YARN-5698. [YARN-3368] Launch new YARN UI under hadoop web app port. (Sunil G via wangda) 2017-10-18 02:06:47 +05:30
Wangda Tan
e5c0219228 YARN-4849. Addendum patch to fix javadocs. (Sunil G via wangda) 2017-10-18 02:06:47 +05:30
Wangda Tan
bb606e3d4b YARN-4515. [YARN-3368] Support hosting web UI framework inside YARN RM. (Sunil G via wangda) 
YARN-5000. [YARN-3368] App attempt page is not loading when timeline server is not started (Sunil G via wangda)
YARN-5038. [YARN-3368] Application and Container pages shows wrong values when RM is stopped. (Sunil G via wangda)
 2017-10-18 02:06:46 +05:30
Subru Krishnan
5e7491d532 YARN-7311. Fix TestRMWebServicesReservation parametrization for fair scheduler. (Yufei Gu via Subru). 
(cherry picked from commit 75323394fbc4211596a2c8fbb5e584f3183f742f)
 2017-10-17 12:43:36 -07:00
Subru Krishnan
adf5ea73d6 YARN-7095. Federation: routing getNode/getNodes/getMetrics REST invocations transparently to multiple RMs. (Giovanni Matteo Fumarola via Subru). 
(cherry picked from commit bac4e8cca8b54405f5e37b90e545b93bbadee0f4)
 2017-10-17 12:14:05 -07:00
Varun Saxena
56e0873e22 Fix javac, javadoc and checkstyle issues before branch-2 merge. 2017-10-17 23:36:41 +05:30
vrushali
8b98dba7ee YARN-7190 Ensure only NM classpath in 2.x gets TSv2 related hbase jars, not the user classpath (Contributed by Varun Saxena via Vrushali C) 2017-10-17 23:36:41 +05:30
bibinchundatt
f7637b55bb YARN-7141. Move logging APIs to slf4j in timelineservice after ATSv2 merge. Contributed by Varun Saxena 
(cherry picked from commit dcd0bedcc8113dd5e1d852c242ae9511d1b3d58e)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMHATimelineCollectors.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/reader/security/TimelineReaderWhitelistAuthorizationFilter.java
 2017-10-17 23:36:40 +05:30
Varun Saxena
1289b056ac YARN-6958. Addedndum for Moving logging APIs over to slf4j in hadoop-yarn-server-timelineservice 2017-10-17 23:36:40 +05:30
Rohith Sharma K S
5a7c272e0a YARN-6130. [ATSv2 Security] Generate a delegation token for AM when app collector is created and pass it to AM via NM and RM. Contributed by Varun Saxena. 2017-10-17 23:36:39 +05:30
Varun Saxena
8d9d40aa53 Addendum for YARN-6102. 2017-10-17 23:36:39 +05:30
Sunil G
4b65fe73ca YARN-6102. RMActiveService context to be updated with new RMContext on failover. Contributed by Rohith Sharma K S. 
(cherry picked from commit a657472b42c58f87fd3165e0a746d83b72182a24)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/AdminService.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMActiveServiceContext.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContextImpl.java
 2017-10-17 23:36:38 +05:30
Rohith Sharma K S
d01a3f1ba2 YARN-6801. NPE in RM while setting collectors map in NodeHeartbeatResponse. Contributed by Vrushali C. 
(cherry picked from commit 5791ced4bd864ff6c08cb2c3ee10ab62008f0168)
 2017-10-17 23:36:38 +05:30
Varun Saxena
c1f9f0529b Addendum for YARN-5865. Retrospect updateApplicationPriority api to handle state store exception in align with YARN-5611 2017-10-17 23:36:37 +05:30
Varun Saxena
ebcb575a12 Addendum for YARN-4218. Metric for resource*time that was preempted. 2017-10-17 23:36:37 +05:30
Rohith Sharma K S
1072232741 YARN-6424. TimelineCollector is not stopped when an app finishes in RM. Contributed by Varun Saxena. 
(cherry picked from commit 1a9439e299910032ce6a1919dece3107c1c9de5b)
 2017-10-17 23:36:37 +05:30
Sangjin Lee
a16a684fbb YARN-5928. Move ATSv2 HBase backend code into a new module that is only dependent at runtime by yarn servers. Contributed by Haibo Chen. 
(cherry picked from commit b92089c0e8ab1b87b8b5b55b1e3d4367ae5d847a)
 2017-10-17 23:36:36 +05:30
Sangjin Lee
b8cfb4fcb3 YARN-5792. Adopt the id prefix for YARN, MR, and DS entities. Contributed by Varun Saxena. 
(cherry picked from commit f734977b27a514ce0561638c0a6a17b1ef093026)
 2017-10-17 23:36:35 +05:30
Varun Saxena
484b1d2ec1 YARN-5599. Publish AM launch command to ATS (Rohith Sharma K S via Varun Saxena) 2017-10-17 23:36:34 +05:30
Varun Saxena
6562a354a9 YARN-3359. Recover collector list when RM fails over (Li Lu via Varun Saxena) 
(cherry picked from commit 1c8a57550bea2280d41a208baf6fea59f66c0d75)
 2017-10-17 23:36:34 +05:30
Varun Saxena
c3db98aa10 YARN-5743. [Atsv2] Publish queue name and RMAppMetrics to ATS (Rohith Sharma K S via Varun Saxena) 2017-10-17 23:36:34 +05:30
Sangjin Lee
853a5da041 YARN-5699. Retrospect yarn entity fields which are publishing in events info fields. Contributed by Rohith Sharma K S. 
(cherry picked from commit 787d5942003a5441ba455a79fa08da28f60d5777)
 2017-10-17 23:36:34 +05:30
Sangjin Lee
4675f22e6a YARN-5638. Introduce a collector timestamp to uniquely identify collectors creation order in collector discovery. Contributed by Li Lu. 
(cherry picked from commit 0c1863144649ea265da65ce25158707cc3a3fb4a)
 2017-10-17 23:36:34 +05:30
Sangjin Lee
e212e562ec YARN-5355. Backported YARN-2928 into our branch-2 feature branch. 2017-10-17 23:36:33 +05:30
Robert Kanter
759d7d58fa YARN-7309. TestClientRMService#testUpdateApplicationPriorityRequest and TestClientRMService#testUpdatePriorityAndKillAppWithZeroClusterResource test functionality not supported by FairScheduler (rkanter) 
(cherry picked from commit ec8bf9e48ac4dc675516a57c06a29890b5138548)
 2017-10-10 10:19:02 -07:00
Jonathan Hung
4bd3f0df42 YARN-7252. Removing queue then failing over results in exception 2017-10-09 11:12:08 -07:00
Jonathan Hung
06c8938ab0 YARN-7251. Misc changes to YARN-5734 2017-10-09 11:12:08 -07:00
Jonathan Hung
c8f9fe88d5 YARN-7046. Add closing logic to configuration store 2017-10-09 11:12:08 -07:00
Wangda Tan
887b366d5f YARN-6840. Implement zookeeper based store for scheduler configuration updates. (Jonathan Hung via wangda) 
Change-Id: I9debea674fe8c7e4109d4ca136965a1ea4c48bcc
 2017-10-09 11:12:07 -07:00
Xuan
36cda2ff80 YARN-7024: Fix issues on recovery in LevelDB store. Contributed by Jonathan Hung 2017-10-09 11:12:07 -07:00
Xuan
3a31c38885 YARN-6322: Disable queue refresh when configuration mutation is enabled. Contributed by Jonathan Hung 2017-10-09 11:12:07 -07:00
Xuan
cd58f5da5a YARN-5947: Create LeveldbConfigurationStore class using Leveldb as backing store. Contributed by Jonathan Hung 2017-10-09 11:12:07 -07:00
Xuan
16b27573ef YARN-5953 addendum: Move QueueConfigInfo and SchedConfUpdateInfo to package org.apache.hadoop.yarn.webapp.dao 2017-10-09 11:12:07 -07:00
Xuan
96f4cf7cde YARN-5953:Create CLI for changing YARN configurations. (Jonathan Hung via xgong) 2017-10-09 11:12:07 -07:00
Xuan
3c50084219 YARN-6575. Support global configuration mutation in MutableConfProvider. (Jonathan Hung via Xuan Gong) 2017-10-09 11:12:07 -07:00
Wangda Tan
a47823ca98 YARN-5949. Add pluggable configuration ACL policy interface and implementation. (Jonathan Hung via wangda) 
Change-Id: Ib98e82ff753bede21fcab2e6ca9ec1e7a5a2008f
 2017-10-09 11:12:07 -07:00
Wangda Tan
53a1268125 YARN-5952. Create REST API for changing YARN scheduler configurations. (Jonathan Hung via wangda) 2017-10-09 11:12:07 -07:00
Jonathan Hung
072527b31a YARN-5948. Implement MutableConfigurationManager for handling storage into configuration store 2017-10-09 11:12:07 -07:00
Xuan
38cf9ccd5b YARN-5946: Create YarnConfigurationStore interface and 
InMemoryConfigurationStore class. Contributed by Jonathan Hung
 2017-10-09 11:12:07 -07:00
Jonathan Hung
cd4bc54e6c YARN-5951. Changes to allow CapacityScheduler to use configuration store 2017-10-09 11:12:07 -07:00
Daniel Templeton
4d377c89bb YARN-2612 addendum: fixed javadoc error. 
(cherry picked from commit 6d6ca4c923142a893870fc8a6309c02f60dc2d24)
(cherry picked from commit bec03f7060d353e78a68bb64c88397cf7181727d)
 2017-10-09 14:16:50 +02:00
Sunil G
045526aca1 YARN-7245. Max AM Resource column in Active Users Info section of Capacity Scheduler UI page should be updated per-user. Contributed by Eric Payne. 
(cherry picked from commit 8137254a1f4367974bf1883f1982e31ab2865107)
 2017-10-09 14:44:33 +05:30
Yufei Gu
da0a96968d YARN-7207. Cache the RM proxy server address. (Yufei Gu) 
(cherry picked from commit 72d22b753abde4d07a727479d3f3d5d84d5dd6b2)

# Conflicts:
#	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContext.java
#	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContextImpl.java
 2017-10-06 09:56:24 -07:00
Yufei Gu
c949e8e83a YARN-2162. Add ability in Fair Scheduler to optionally configure maxResources in terms of percentage. (Yufei Gu) 2017-10-06 00:35:03 -07:00
Arun Suresh
cb91bb3c96 YARN-7258. Add Node and Rack Hints to Opportunistic Scheduler. (Kartheek Muthyala via asuresh). 
(cherry picked from commit b733348dde18a242e6c9074c512116a8baf1d281)
 2017-10-05 15:22:20 -07:00
Subru Krishnan
7fd4a997f4 YARN-5329. Placement Agent enhancements required to support recurring reservations in ReservationSystem. (Carlo Curino via Subru). 
(cherry picked from commit e6e614e380ed1d746973b50f666a9c40d272073e)
 2017-10-04 19:30:57 -07:00
Subru Krishnan
7836a6b59a YARN-2037. Add work preserving restart support for Unmanaged AMs. (Botong Huang via Subru) 2017-10-04 16:29:08 -07:00
Akira Ajisaka
5508b9b7c8
YARN-7044. TestContainerAllocation#testAMContainerAllocationWhenDNSUnavailable fails. 
(cherry picked from commit 2df1b2ac0509ba10fff606ada7e9b3562c12dd16)
 2017-10-05 06:08:46 +09:00