Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
10,380 Commits 326 Branches 371 Tags
Commit Graph

1112 Commits

Author SHA1 Message Date
Jason Lowe
f3bca8bc1a YARN-4374. RM capacity scheduler UI rounds user limit factor. Contributed by Chang Li 
(cherry picked from commit 060cdcbe5db56383e74e282ac61eedff8248c11b)
 2015-11-20 23:13:27 +00:00
Arun Suresh
d8e60080e9 YARN-4184. Remove update reservation state api from state store as its not used by ReservationSystem (Sean Po via asuresh) 
(cherry picked from commit 6a61928fb790d553c942641761ea963b5e996ef6)
 2015-11-17 15:51:59 -08:00
Jian He
2cf084ac76 Revert "YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan" 
This reverts commit a4ff03b200cf02a49ab81fa3c71cb707f6d2fee1.

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-11-16 20:19:31 -08:00
Wangda Tan
13f8c7093b YARN-4347. Resource manager fails with Null pointer exception. (Jian He via wangda) 
(cherry picked from commit 7f55a18071ebe1c5591ff8a410b227c855c3f428)
 2015-11-12 11:24:04 -08:00
Wangda Tan
0c1fb15b29 YARN-4287. Capacity Scheduler: Rack Locality improvement (Nathan Roberts via wangda) 
(cherry picked from commit 796638d9bc86235b9f3e5d1a3a9a25bbf5c04d1c)
 2015-11-12 11:12:11 -08:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
bb5dd7f772 YARN-4183. Reverting the patch to fix behaviour change. 
Revert "YARN-4183. Enabling generic application history forces every job to get a timeline service delegation token (jeagles)"

This reverts commit 42bc525acb78088ed952e92aa49408413937de86.
 2015-11-11 10:42:52 -08:00
Jian He
a4ff03b200 YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan 
(cherry picked from commit 8fbea531d7f7b665f6f55af54c8ebf330118ff37)
 2015-11-09 11:05:12 -08:00
Jian He
2b197bbffd YARN-4127. RM fail with noAuth error if switched from failover to non-failover. Contributed by Varun Saxena 
(cherry picked from commit e5b1733e049dc0f1859b93618354e049a0efdc4a)
 2015-10-29 15:43:26 -07:00
Jonathan Eagles
42bc525acb YARN-4183. Enabling generic application history forces every job to get a timeline service delegation token (jeagles) 
(cherry picked from commit c293c58954cdab25c8c69418b0e839883b563fa4)
 2015-10-29 16:42:49 -05:00
Arun Suresh
8943abfb08 YARN-4310. FairScheduler: Log skipping reservation messages at DEBUG level (asuresh) 
(cherry picked from commit 58d1df585c2b8a4d2f78ae30726f07b4c6a94731)
 2015-10-29 13:43:43 -07:00
Rohith Sharma K S
e3fcb711d6 YARN-4130. Duplicate declaration of ApplicationId in RMAppManager#submitApplication method. (Kai Sasaki via rohithsharmaks) 
(cherry picked from commit 656c8f952738f469766ede729dc0f25a3ce4214e)
 2015-10-29 12:25:08 +05:30
Wangda Tan
1499af38eb YARN-3216. Max-AM-Resource-Percentage should respect node labels. (Sunil G via wangda) 
(cherry picked from commit 56e4f6237ae8b1852e82b186e08db3934f79a9db)
 2015-10-26 16:45:20 -07:00
Wangda Tan
9bf1e74171 YARN-4169. Fix racing condition of TestNodeStatusUpdaterForLabels. (Naganarasimha G R via wangda) 
(cherry picked from commit 6f606214e734d9600bc0f25a63142714f0fea633)
 2015-10-26 16:38:39 -07:00
Wangda Tan
baa2998e93 YARN-4285. Display resource usage as percentage of queue and cluster in the RM UI (Varun Vasudev via wangda) 
(cherry picked from commit 3cc73773eb26f7469c99b25a76814d6fad0be28e)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-10-26 13:12:56 -07:00
Jason Lowe
7d66e2e666 YARN-4284. condition for AM blacklisting is too narrow. Contributed by Sangjin Lee 
(cherry picked from commit 33a03af3c396097929b9cd9c790d7f52eddc13e0)
 2015-10-26 19:54:16 +00:00
Rohith Sharma K S
33455fa415 YARN-2729. Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup. (Naganarasimha G R via rohithsharmaks) 
(cherry picked from commit 5acdde4744c131e05db7b4b5f7d684fed7608b99)
 2015-10-26 15:45:22 +05:30
Arun Suresh
f0580dcac0 YARN-3738. Add support for recovery of reserved apps running under dynamic queues (subru via asuresh) 
(cherry picked from commit ab8eb8770c8b8bff41dacb1a399d75906abb1ac4)
 2015-10-24 22:54:45 -07:00
Akira Ajisaka
356e922632 YARN-4294. [JDK8] Fix javadoc errors caused by wrong reference and illegal tag. (aajisaka) 
(cherry picked from commit 7781fe1b9e89488b37217f81c246c1398407e47a)
 2015-10-24 11:56:37 +09:00
Jason Lowe
235e84bf3d YARN-4041. Slow delegation token renewal can severely prolong RM recovery. Contributed by Sunil G 
(cherry picked from commit d3a34a4f388155f6a7ef040e244ce7be788cd28b)
 2015-10-23 20:57:53 +00:00
Jonathan Eagles
6db7bfbcfd YARN-4009. CORS support for ResourceManager REST API. ( Varun Vasudev via jeagles) 
(cherry picked from commit f8adeb712dc834c27cec15c04a986f2f635aba83)
 2015-10-23 10:43:59 -05:00
Ming Ma
4bb7e68eb6 YARN-2913. Fair scheduler should have ability to set MaxResourceDefault for each queue. (Siqi Li via mingma) 
(cherry picked from commit 934d96a334598fcf0e5aba2043ff539469025f69)
 2015-10-23 08:37:46 -07:00
Junping Du
d5145b58f6 YARN-4243. Add retry on establishing Zookeeper conenction in EmbeddedElectorService#serviceInit. Contributed by Xuan Gong. 
(cherry picked from commit 0fce5f9a496925f0d53ea6c14318c9b513de9882)
 2015-10-22 13:42:12 -07:00
Zhihai Xu
9fe7ccc61c YARN-4256. YARN fair scheduler vcores with decimal values. Contributed by Jun Gong 
(cherry picked from commit 960201b79b9f2ca40f8eadb21a2f9fe37dde2b5d)
 2015-10-22 12:31:28 -07:00
Anubhav Dhoot
c44401f362 YARN-3739. Add reservation system recovery to RM recovery process. Contributed by Subru Krishnan. 
(cherry picked from commit 2798723a5443d04455b9d79c48d61f435ab52267)
 2015-10-22 06:52:41 -07:00
Arun Suresh
f2009dc89c YARN-3985. Make ReservationSystem persist state using RMStateStore reservation APIs. (adhoot via asuresh) 
(cherry picked from commit 506d1b1dbcb7ae5dad4a3dc4d415af241c72887c)
 2015-10-20 16:48:58 -07:00
Arun Suresh
acc0e718d6 YARN-4270. Limit application resource reservation on nodes for non-node/rack specific requests (asuresh) 
(cherry picked from commit 7e2837f830382835838c82398db6fc9823d612a7)
 2015-10-19 20:02:10 -07:00
Jian He
1ba73169c7 YARN-4170. AM need to be notified with priority in AllocateResponse. Contributed by Sunil G 
(cherry picked from commit f9da5cdb2b2dd071fd60fc01ea1edf0f79c0819b)
 2015-10-16 15:26:46 -07:00
Wangda Tan
204ee6a40f YARN-4162. CapacityScheduler: Add resource usage by partition and queue capacity by partition to REST API. (Naganarasimha G R via wangda) 
(cherry picked from commit 4337b263aa23464a01c316336db2319e2858a2b3)
 2015-10-16 15:10:08 -07:00
Jian He
2862057f11 YARN-4000. RM crashes with NPE if leaf queue becomes parent queue during restart. Contributed by Varun Saxena 
(cherry picked from commit cf23f2c2b5b4eb9e51de1a66b7aa57dee7ff30b5)
 2015-10-15 17:13:09 -07:00
rohithsharmaks
526be695c0 YARN-4250. NPE in AppSchedulingInfo#isRequestLabelChanged. (Brahma Reddy Battula via rohithsharmaks) 
(cherry picked from commit d6c8bad86964dbad3cc810914f786c7c4722227a)
 2015-10-14 16:13:26 +05:30
Jian He
ee1ce393f5 YARN-4230. RM crashes with NPE when increasing container resource if there is no headroom left. Contributed by Meng Ding 
(cherry picked from commit 9849c8b3865c7c9c9be81ae0ef8f29caa1d5f881)
 2015-10-12 11:52:04 -07:00
Zhihai Xu
9988b57e71 YARN-4201. AMBlacklist does not work for minicluster. Contributed by Jun Gong. 
(cherry picked from commit 049c6e8dc0f952b5ff7d394aa564ab7db13e4f26)
 2015-10-12 00:16:40 -07:00
Devaraj K
28edc7b129 YARN-3964. Support NodeLabelsProvider at Resource Manager side. 
Contributed by Dian Fu.

(cherry picked from commit db9304788187c700647c4d84caeb3b5ad6d868d8)
 2015-10-11 11:24:05 +05:30
Wangda Tan
37642abb83 YARN-4140. RM container allocation delayed incase of app submitted to Nodelabel partition. (Bibin A Chundatt via wangda) 
(cherry picked from commit def374e666ed0c1d665aeb1b7307e09769448138)
 2015-10-09 16:40:10 -07:00
Karthik Kambatla
2d8f9e3fd9 MAPREDUCE-6302. Incorrect headroom can lead to a deadlock between map and reduce allocations. (kasha) 
(cherry picked from commit 4aa9b3e75ca86917125e56e1b438668273a5d87f)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java
 2015-10-09 07:54:47 -07:00
Jason Lowe
0139517811 YARN-261. Ability to fail AM attempts. Contributed by Andrey Klochkov and Rohith Sharma K S 
(cherry picked from commit a0bca2b5ad2344fda5411d910a3730c85f12a0df)
 2015-10-09 14:19:41 +00:00
Rohith Sharma K S
21609e8e9a YARN-4235. FairScheduler PrimaryGroup does not handle empty groups returned for a user. (Anubhav Dhoot via rohithsharmaks) 
(cherry picked from commit 8f195387a4a4a5a278119bf4c2f15cad61f0e2c7)
 2015-10-09 10:11:53 +05:30
Rohith Sharma K S
99126714a3 YARN-4228. FileSystemRMStateStore use IOUtils#close instead of fs#close. (Bibin A Chundatt via rohithsharmaks) 
(cherry picked from commit 3793cbe4c3cce5d03c4a18d562cbcb7cacd8f743)
 2015-10-07 10:20:36 +05:30
Rohith Sharma K S
053fcf4c0d YARN-4209. RMStateStore FENCED state doesn’t work due to updateFencedState called by stateMachine.doTransition. (Zhihai Xu via rohithsharmaks) 
(cherry picked from commit 9156fc60c654e9305411686878acb443f3be1e67)
 2015-10-07 09:37:14 +05:30
Wangda Tan
5453a63612 YARN-4215. RMNodeLabels Manager Need to verify and replace node labels for the only modified Node Label Mappings in the request. (Naganarasimha G R via wangda) 
(cherry picked from commit 29a582ada0fe195989eca25e5a995895e178f4ea)
 2015-10-06 11:59:10 -07:00
Harsh J
21b4ba48ce HADOOP-12458. Retries is typoed to spell Retires in parts of hadoop-yarn and hadoop-common. Contributed by Neelesh Srinivas Salian. 
(cherry picked from commit c918f7be5e6aecaa3cb0c23a95819329802d32d4)
 2015-10-03 18:39:22 +05:30
Xuan
53bddc410f YARN-1897. CLI and core support for signal container functionality. Contributed by Ming Ma 
(cherry picked from commit 8f08532bde153811368e1b8336446fba4743f9d2)
 2015-10-02 18:53:04 -07:00
Karthik Kambatla
64a7617eb9 YARN-4066. Large number of queues choke fair scheduler. (Johan Gustavsson via kasha) 
(cherry picked from commit a0b5a0a419dfc07b7ac45c06b11b4c8dc7e79958)
 2015-09-29 08:13:25 -07:00
Anubhav Dhoot
22f2501476 YARN-4180. AMLauncher does not retry on failures when talking to NM. (adhoot) 
(cherry picked from commit 9735afe967a660f356e953348cb6c34417f41055)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationMasterLauncher.java
 2015-09-28 16:54:06 -07:00
Jason Lowe
600022ae69 YARN-4141. Runtime Application Priority change should not throw exception for applications at finishing states. Contributed by Sunil G 
(cherry picked from commit 9f53a95ff624f66a774fe3defeea4a3454f4c4af)
 2015-09-28 22:56:07 +00:00
Anubhav Dhoot
4d8b99423e YARN-4204. ConcurrentModificationException in FairSchedulerQueueInfo. (adhoot) 
(cherry picked from commit fb2e525c0775ccf218c8980676e9fb4005a406a6)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-09-28 09:39:52 -07:00
Jian He
ea2b6ee9e9 YARN-4171. Fix findbugs warnings in YARN-1197 branch. Contributed by Wangda Tan 
(cherry picked from commit b3f6b641dccb0d59df78855e2951d2cae7dff8ad)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-09-24 10:30:49 -07:00
Jian He
d458805690 YARN-1651. CapacityScheduler side changes to support container resize. Contributed by Wangda Tan 
(cherry picked from commit 89cab1ba5f0671f8ef30dbe7432079c18362b434)

Conflicts:
	hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/SLSCapacityScheduler.java
 2015-09-24 10:30:04 -07:00
Jian He
fa0a554ae0 YARN-3867. ContainerImpl changes to support container resizing. Contributed by Meng Ding 
(cherry picked from commit 5f5a968d65c44a831176764439e00db9203999ed)
 2015-09-24 10:23:46 -07:00
Jian He
c19e8ffdc7 YARN-1449. AM-NM protocol changes to support container resizing. Contributed by Meng Ding & Wangda Tan) 
(cherry picked from commit 83a18add10ee937a04e833a66e0a4642e776e510)
 2015-09-24 10:23:25 -07:00