Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
13,740 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

3543 Commits

Author SHA1 Message Date
Subru Krishnan 7fd4a997f4 YARN-5329. Placement Agent enhancements required to support recurring reservations in ReservationSystem. (Carlo Curino via Subru). 
(cherry picked from commit e6e614e380)
 2017-10-04 19:30:57 -07:00
Subru Krishnan 7836a6b59a YARN-2037. Add work preserving restart support for Unmanaged AMs. (Botong Huang via Subru) 2017-10-04 16:29:08 -07:00
Akira Ajisaka 5508b9b7c8
YARN-7044. TestContainerAllocation#testAMContainerAllocationWhenDNSUnavailable fails. 
(cherry picked from commit 2df1b2ac05)
 2017-10-05 06:08:46 +09:00
Sidharta S 3ecbfff937 YARN-7226. Whitelisted variables do not support delayed variable expansion. Contributed by Jason Lowe 2017-10-04 14:05:01 -07:00
Wangda Tan 175ff0585d YARN-7269. Tracking URL in the app state does not get redirected to ApplicationMaster for Running applications. (wangda via Jian He) 
Change-Id: I582566e09d5ef67e9cc9e9805231ccbabe57c3d0
 2017-10-03 12:19:11 -07:00
Wangda Tan 42e873d4ea YARN-6625. yarn application -list returns a tracking URL for AM that doesn't work in secured and HA environment. (Yufei Gu) 2017-10-03 12:04:56 -07:00
Wangda Tan 598329e6f8 YARN-6550. Capture launch_container.sh logs to a separate log file. (Suma Shivaprasad via wangda) 
Change-Id: I66c9b50ae3955dda0b5bf0674a9bfb728db27acb
 2017-10-03 09:46:35 -07:00
Wangda Tan 8beae14a07 YARN-7259. Add size-based rolling policy to LogAggregationIndexedFileController. (xgong via wangda) 
Change-Id: Ifaf82c0aee6b73b9b6ebf103aa72e131e3942f31
(cherry picked from commit 280080fad0)
 2017-10-02 15:36:11 -07:00
Yufei Gu e84d508c1b YARN-6333. Improve doc for minSharePreemptionTimeout, fairSharePreemptionTimeout and fairSharePreemptionThreshold. (Contributed by Chetna Chaudhari via Yufei Gu) 
(cherry picked from commit 66c417167a)
 2017-09-29 18:08:55 -07:00
Wangda Tan 5fa5ba7f4d YARN-6509. Add a size threshold beyond which yarn logs will require a force option. (Xuan Gong via wangda) 
Change-Id: I755fe903337d4ff9ec35dae5b9cce638794e1d0f
(cherry picked from commit ec2ae3060a)
(cherry picked from commit d58d75ca50)
 2017-09-29 16:53:16 -07:00
Eric Payne b0ba31c518 YARN-7084. TestSchedulingMonitor#testRMStarts fails sporadically. Contributed by Jason Lowe. 
(cherry picked from commit 7f6118f918)
 2017-09-29 10:50:21 -05:00
Jason Lowe c0bfb385a8 YARN-7260. yarn.router.pipeline.cache-max-size is missing in yarn-default.xml. Contributed by Jason Lowe 2017-09-29 09:53:05 -05:00
Chris Trezzo cba1891b61 YARN-7250. Update Shared cache client api to use URLs. 
(cherry picked from commit c114da5e64)
 2017-09-28 15:32:18 -07:00
Subru Krishnan ffcf5ba1ce YARN-6962. Add support for updateContainers when allocating using FederationInterceptor. (Botong Huang via Subru). 
(cherry picked from commit ca669f9f8b)
 2017-09-28 13:11:32 -07:00
Subru Krishnan d6da014f67 YARN-6955. Handle concurrent register AM requests in FederationInterceptor. (Botong Huang via Subru). 
(cherry picked from commit c61f2c4198)
 2017-09-28 13:11:19 -07:00
Jason Lowe 913a64e4c9 YARN-7248. NM returns new SCHEDULED container status to older clients. Contributed by Arun Suresh 
(cherry picked from commit 85d81ae58e)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java
 2017-09-28 14:25:01 -05:00
Arun Suresh 53c1115908 YARN-6691. Update YARN daemon startup/shutdown scripts to include Router service. (Giovanni Matteo Fumarola via asuresh) 2017-09-28 11:41:43 -07:00
Xuan c143708acf YARN-7257. AggregatedLogsBlock reports a bad 'end' value as a bad 
'start' value. Contributed by Jason Lowe

(cherry picked from commit 28c4957fcc)
(cherry picked from commit 81e7872793)
 2017-09-27 16:04:06 -07:00
Rohith Sharma K S c570ddaa66 YARN-7157. Add admin configuration to filter per-user's apps in secure cluster. Contributed by Sunil G. 2017-09-27 11:49:59 +05:30
Jason Lowe c4de765fcb Revert "YARN-6570. No logs were found for running application, running" 
This reverts commit 6aaca3d08e.
 2017-09-26 10:44:16 -05:00
Rohith Sharma K S c27f682032 YARN-65. Reduce RM app memory footprint once app has completed. Contributed by Manikandan R. 
(cherry picked from commit 06e5a7b5cf)
 2017-09-26 14:50:34 +05:30
Wangda Tan 179d8dc83c YARN-7118. AHS REST API can return NullPointerException. (Billie Rinaldi via wangda) 
Change-Id: I16ca93385ab80e3680d4d173b8d1b79929ea7174
 2017-09-25 21:41:00 -07:00
Arun Suresh e7199cb494 YARN-7240. Add more states and transitions to stabilize the NM Container state machine. (Kartheek Muthyala via asuresh) 
(cherry picked from commit df800f6cf3)
 2017-09-25 16:36:58 -07:00
Akira Ajisaka c0dacda546
YARN-7153. Remove duplicated code in AMRMClientAsyncImpl.java. Contributed by Sen Zhao. 
(cherry picked from commit e928ee583c)
 2017-09-26 02:05:35 +09:00
Anu Engineer d0c0bd573b Revert "YARN-7118. AHS REST API can return NullPointerException. Contributed by Billie Rinaldi." 
This reverts commit de1d747644.
 2017-09-23 11:12:16 -07:00
Junping Du de1d747644 YARN-7118. AHS REST API can return NullPointerException. Contributed by Billie Rinaldi. 
(cherry picked from commit 4002bf0a9e)
 2017-09-22 13:59:03 -07:00
Carlo Curino 7cd9018b13 [YARN FEDERATION BACKPORT] Fixing more Java 1.7 compilation issues 2017-09-21 18:19:36 -07:00
Carlo Curino 88b32edb8f YARN-7010. Federation: routing REST invocations transparently to multiple RMs (part 2 - getApps). (Contributed by Giovanni Matteo Fumarola via curino) 
(cherry picked from commit cc8893edc0)
 2017-09-21 18:09:44 -07:00
Arun Suresh 2aacb9d3fb YARN-6923. Metrics for Federation Router. (Giovanni Matteo Fumarola via asuresh) 
(cherry picked from commit ae8fb13b31)
 2017-09-21 18:09:37 -07:00
Arun Suresh ac090b38ad YARN-5603. Metrics for Federation StateStore. (Ellen Hui via asuresh) 
(cherry picked from commit 75abc9a8e2)
 2017-09-21 18:09:30 -07:00
Subru Krishnan 261f769d79 YARN-6900. ZooKeeper based implementation of the FederationStateStore. (Íñigo Goiri via Subru). 
(cherry picked from commit de462da04e)
 2017-09-21 18:09:23 -07:00
Subru Krishnan 9ad067efe9 YARN-6996. Change javax.cache library implementation from JSR107 to Apache Geronimo. (Ray Chiang via Subru). 
(cherry picked from commit 18f3603bce)
 2017-09-21 18:09:16 -07:00
Carlo Curino 8220b19af7 YARN-6896. Federation: routing REST invocations transparently to multiple RMs (part 1 - basic execution). (Contributed by Giovanni Matteo Fumarola via curino) 
(cherry picked from commit cc59b5fb26)
 2017-09-21 18:09:07 -07:00
Subru Krishnan a1ee4ad77f YARN-6970. Add PoolInitializationException as retriable exception in FederationFacade. (Giovanni Matteo Fumarola via Subru). 
(cherry picked from commit ad2a350662)
 2017-09-21 18:08:30 -07:00
Carlo Curino d11be2dcac [YARN FEDERATION BACKPORT] Fix compilation issues due to: hadoop-router/pom.xml versions and Java 1.7 2017-09-21 17:56:46 -07:00
Allen Wittenauer baa2bec843 YARN-2280. Resource manager web service fields are not accessible (Krisztian Horvath via aw) 
(cherry picked from commit a5cf985bf5)
 2017-09-21 17:52:25 -07:00
Carlo Curino d3afff7603 YARN-6853. Add MySql Scripts for FederationStateStore. (Contributed by Giovanni Matteo Fumarola via curino) 
(cherry picked from commit 874ddbf0b5)
(cherry picked from commit 9625a030de)
 2017-09-21 17:15:36 -07:00
Carlo Curino bfd967d338 YARN-5412. Create a proxy chain for ResourceManager REST API in the Router. (Contributed by Giovanni Matteo Fumarola via curino) 
(cherry picked from commit b6240b92ab)
(cherry picked from commit acda6b96a4)
 2017-09-21 17:13:28 -07:00
Subru Krishnan 049f7c84a1 YARN-6866. Minor clean-up and fixes in anticipation of YARN-2915 merge with trunk. (Botong Huang via Subru). 
(cherry picked from commit 40453879ec)
(cherry picked from commit 86f05c3312)
 2017-09-21 16:55:43 -07:00
Subru Krishnan 66980c00f6 Bumping up hadoop-yarn-server-router module to 3.0.0-beta1 post rebase. 
(cherry picked from commit 69e3ed2680)
(cherry picked from commit 8737c2a8e9)
 2017-09-21 16:55:42 -07:00
Subru Krishnan 58e2458c2e Bumping up yarn-server-router (new) module pom to beta1 and fixing imports post rebase. 
(cherry picked from commit f1eff212c5)
(cherry picked from commit 712e97d4cf)
 2017-09-21 16:55:41 -07:00
Subru Krishnan fdbd214d4a YARN-6821. Move FederationStateStore SQL DDL files from test resource to sbin. 
(cherry picked from commit cd9db822f0)
(cherry picked from commit 3e1dc7ece0)
 2017-09-21 16:55:40 -07:00
Subru Krishnan 055138c5c4 YARN-6815. FederationStateStoreFacade return behavior should be consistent irrespective of whether caching is enabled or not. 
(cherry picked from commit 8820693cd7)
(cherry picked from commit 825108d08a)
 2017-09-21 16:55:40 -07:00
Subru Krishnan d3fb1711a4 YARN-6807. Adding required missing configs to Federation configuration guide based on e2e testing. (Tanuj Nayak via Subru). 
(cherry picked from commit b4ac9d1b63)
(cherry picked from commit f427e4201f)
 2017-09-21 16:55:39 -07:00
Subru Krishnan 169037cbb4 YARN-3659. Federation: routing client invocations transparently to multiple RMs. (Giovanni Matteo Fumarola via Subru). 
(cherry picked from commit 52daa6d971)
(cherry picked from commit 43a97174fe)
 2017-09-21 16:55:33 -07:00
Subru Krishnan 8ed4daddc4 YARN-6724. Add ability to blacklist sub-clusters when invoking Routing policies. (Giovanni Matteo Fumarola via Subru). 
(cherry picked from commit f8e5de5969)
(cherry picked from commit 4cfec943b1)
 2017-09-21 16:55:01 -07:00
Subru Krishnan 35a38330ef YARN-6511. Federation: transparently spanning application across multiple sub-clusters. (Botong Huang via Subru). 
(cherry picked from commit 8c988d235e)
(cherry picked from commit 70b1a757f1)
 2017-09-21 16:49:55 -07:00
Subru Krishnan 7ede8c1a53 YARN-3666. Federation Intercepting and propagating AM- home RM communications. (Botong Huang via Subru). 
(cherry picked from commit 2399eb8200)
(cherry picked from commit bed1832c93)
 2017-09-21 16:49:54 -07:00
Subru Krishnan 7dd6caff9e YARN-6484. Documenting the YARN Federation feature. (Carlo Curino via Subru). 
(cherry picked from commit 749ca890e6)
(cherry picked from commit d6c29b2f5b)
 2017-09-21 16:49:54 -07:00
Subru Krishnan b9fdc24e43 YARN-6666. Fix unit test failure in TestRouterClientRMService. (Botong Huang via Subru). 
(cherry picked from commit e750907d0a)
(cherry picked from commit 42949a6e01)
 2017-09-21 16:49:48 -07:00