Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,710 Commits 326 Branches 371 Tags
Commit Graph

2576 Commits

Author SHA1 Message Date
Naganarasimha
155f25061e YARN-3971. Skip RMNodeLabelsManager#checkRemoveFromClusterNodeLabelsOfQueue on nodelabel recovery. (addendum patch). Contributed by Bibin A chundatt 
(cherry picked from commit b4078bd17b41fbfff5a5c0bca5bf903a327826a7)
 2016-05-24 08:14:53 +08:00
Karthik Kambatla
f33f0b697b YARN-4979. FSAppAttempt demand calculation considers demands at multiple locality levels different. (Zhihai Xu via kasha) 
(cherry picked from commit 6d043aa4cf14bc88b1a19991e3bb0752499312b8)
 2016-05-23 14:30:05 -07:00
Jason Lowe
7602070457 YARN-5055. max apps per user can be larger than max per queue. Contributed by Eric Badger 
(cherry picked from commit ac954486c5102b8fbbc4229a0d3a512bcc7013c0)
 2016-05-23 15:56:00 +00:00
Jason Lowe
e8bd2a5be6 YARN-5103. With NM recovery enabled, restarting NM multiple times results in AM restart. Contributed by Junping Du 
(cherry picked from commit d1df0266cf4e9ff0ec70813c156556ca4e74f791)
 2016-05-23 15:16:52 +00:00
Junping Du
867cd2f5a2 YARN-5112. Excessive log warnings for directory permission issue on NM recovery. Contributed by Jian He. 2016-05-22 16:02:57 -07:00
Jian He
6e0522ee1b YARN-5020. Fix Documentation for Yarn Capacity Scheduler on Resource Calculator. Contributed by Takashi Ohnishi 
(cherry picked from commit d364ceac85622e99133b3eb3becef0c8188e6f89)
 2016-05-20 11:36:08 -07:00
Varun Vasudev
e7f396fd47 YARN-5016. Add support for a minimum retry interval for container retries. Contributed by Jun Gong. 
(cherry picked from commit 0287c491071c15e1c47dc640e6ea437be8cf32ae)
 2016-05-20 12:32:29 +05:30
Junping Du
2703ec6871 YARN-5076. YARN web interfaces lack XFS protection. Contributed by Jonathan Maron. 2016-05-19 14:10:15 -07:00
Jian He
2d846fdcbd YARN-4002. Make ResourceTrackerService#nodeHeartbeat more concurrent. Contributed by Rohith Sharma K S & Zhiguo Hong 
(cherry picked from commit feb90ffcca536e7deac50976b8a8774450fe089f)
 2016-05-19 13:02:05 -07:00
Junping Du
416274b53e YARN-5100. The YarnApplicationState is always running in ATS even application is finished. Contributed by Xuan Gong. 
(cherry picked from commit 141873ca7d68619e3458712b17aed57fec8fd7c8)
 2016-05-19 09:14:42 -07:00
Steve Loughran
151ecdf26d HADOOP-12767. Update apache httpclient version to 4.5.2; httpcore to 4.4.4. Artem Aliev via stevel. 2016-05-19 14:44:44 +01:00
Arun Suresh
15808717f3 YARN-5110. Fix OpportunisticContainerAllocator to insert complete HostAddress in issued ContainerTokenIds. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit 1597630681c784a3d59f5605b87e96197b8139d7)
 2016-05-18 23:20:54 -07:00
Arun Suresh
a555a320e8 YARN-5090. Add End-to-End test-cases for DistributedScheduling using MiniYarnCluster. (asuresh) 
(cherry picked from commit 8a9ecb7584b529be1de3a6e8850041e211c88ebc)
 2016-05-18 23:20:36 -07:00
Arun Suresh
f31602893e YARN-5075. Fix findbugs warnings in hadoop-yarn-common module. (asuresh) 
(cherry picked from commit ccc93e78127c14bfd84179395b055c4061ea436a)
 2016-05-18 23:20:08 -07:00
Arun Suresh
bbcea3abf3 YARN-4738. Notify the RM about the status of OPPORTUNISTIC containers (Konstantinos Karanasos via asuresh) 
(cherry picked from commit f45bc5a83ed2f2292ecf1f4c990b7153d8051fc9)
 2016-05-18 23:12:19 -07:00
Arun Suresh
69a9ce3494 YARN-2888. Corrective mechanisms for rebalancing NM container queues. (asuresh) 
(cherry picked from commit f0ac18d001d97914a9ee810b1fab56c5cebff830)
 2016-05-18 23:09:11 -07:00
Arun Suresh
005dae521c YARN-5073. Refactor startContainerInternal() in ContainerManager to remove unused parameter. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit d971bf252f36e902a0fe446cb11201835ad16bcb)
 2016-05-18 22:36:28 -07:00
Arun Suresh
1b4600abcf YARN-4412. Create ClusterMonitor to compute ordered list of preferred NMs for OPPORTUNITIC containers. (asuresh) 
(cherry picked from commit 341888a0aa23f24458b4e6e34868794b9735c06a)
 2016-05-18 22:35:55 -07:00
Arun Suresh
307cda70db YARN-5049. Extend NMStateStore to save queued container information. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit d464f4d1c4dec483852fc8c0496787cba0af8f57)
 2016-05-18 22:09:30 -07:00
Arun Suresh
d80d24aabf YARN-2885. Create AMRMProxy request interceptor and ContainerAllocator to distribute OPPORTUNISTIC containers to appropriate Nodes (asuresh) 
(cherry picked from commit 2bf025278a318b0452fdc9ece4427b4c42124e39)
(cherry picked from commit c282a08f3892e2e8ceb58e1e9a411062fbd1fb2b)
 2016-05-18 22:08:55 -07:00
Karthik Kambatla
b56fc51b70 YARN-2883. Queuing of container requests in the NM. (Konstantinos Karanasos and Arun Suresh via kasha) 
(cherry picked from commit c8172f5f143d2fefafa5a412899ab7cd081b406d)
 2016-05-18 22:08:20 -07:00
Arun Suresh
68964bbaa2 YARN-4335. Allow ResourceRequests to specify ExecutionType of a request ask (kkaranasos via asuresh) 
(cherry picked from commit 8ffabfdf4fec104cdaa68133385cfe9cf88b95a8)
(cherry picked from commit b2a654c5ee6524f81c971ea0b70e58ea0a455f1d)
 2016-05-18 22:02:43 -07:00
Karthik Kambatla
2c6c701aff YARN-2882. Add an OPPORTUNISTIC ExecutionType. (Konstantinos Karanasos and Inigo Goiri via kasha) 
(cherry picked from commit fb00794368e0aa7aafa9dfc8d453810f641b82b2)
 2016-05-18 22:02:28 -07:00
Naganarasimha
a166286b9e YARN-4925. ContainerRequest in AMRMClient, application should be able to specify nodes/racks together with nodeLabelExpression. Contributed by Bibin A Chundatt 
(cherry picked from commit f04c81c9ce93512bc714531a7731debbe6b794ce)
 2016-05-19 01:35:47 +08:00
Varun Vasudev
3c9392a31d YARN-4913. Yarn logs should take a -out option to write to a directory. Contributed by Xuan Gong. 
(cherry picked from commit ef1757790d89cc72f88f5330761b1c8901c59e94)
 2016-05-18 22:45:06 +05:30
Jian He
264c06a438 YARN-4832. NM side resource value should get updated if change applied in RM side. Contributed by Junping Du 2016-05-17 15:02:14 -07:00
Eric Payne
42b2a34ed8 YARN-5069. TestFifoScheduler.testResourceOverCommit race condition. Contributed by Eric Badger. 
(cherry picked from commit 1217c8f6b49e3afd8ca008ffddcf6615f0accfc5)
 2016-05-16 20:38:10 +00:00
Jason Lowe
575c056357 YARN-4325. Nodemanager log handlers fail to send finished/failed events in some cases. Contributed by Junping Du 
(cherry picked from commit 81effb7dcde2b31423438d6f1b8b8204d4ca05b3)
 2016-05-16 15:43:20 +00:00
Rohith Sharma K S
a37b3694ea YARN-5068. Expose scheduler queue to application master. (Harish Jaiprakash via rohithsharmaks) 2016-05-16 15:25:15 +05:30
Xuan
3fe4107909 YARN-5080. Cannot obtain logs using YARN CLI -am for either KILLED or 
RUNNING AM. Contributed by Xuan Gong

(cherry picked from commit 7be53b65f95dae89c760cde354b42f15f5ee3b5f)
 2016-05-14 12:22:06 -07:00
Jason Lowe
bb95a5e18f YARN-5053. More informative diagnostics when applications killed by a user. Contributed by Eric Badger 
(cherry picked from commit 013000fbc2081c2b7e78b790fc6d847de92b9384)
 2016-05-12 20:30:10 +00:00
Sangjin Lee
d6d13ec670 YARN-4577. Enable aux services to have their own custom classpath/jar file (Xuan Gong via sale) 
(cherry picked from commit 0bbe01f8d56191edfba3b50fb9f8859a0b3f826f)
 2016-05-12 10:14:24 -07:00
Karthik Kambatla
f96b2b6fac YARN-4995. FairScheduler: Display per-queue demand on the scheduler page. (xupeng via kasha) 
(cherry picked from commit 4b4e4c6ba83bc5c41d7bb69bb2483bcfe894a260)
 2016-05-11 17:36:42 -07:00
Junping Du
9dc3683d87 YARN-5029. RM needs to send update event with YarnApplicationState as Running to ATS/AHS. Contributed by Xuan Gong. 
(cherry picked from commit 39f2bac38b111f90d3402229201cdb4315f5d4f5)
 2016-05-11 09:33:16 -07:00
Vinod Kumar Vavilapalli
e0d6a9632d YARN-4842. Fixed "yarn logs" command to guess (and thus not require) the appOwner argument when viewing another user's logs. Contributed by Ram Venkatesh and Xuan Gong. 
(cherry picked from commit 87f5e351337a905af5215af76c72b9312616cd4f)
 2016-05-09 22:47:58 -07:00
Naganarasimha
3732a1e985 YARN-4926. Change nodelabel rest API invalid reponse status to 400. Contributed by Bibin A Chundatt 
(cherry picked from commit 2750fb900fc9c468570d051cb19e3a379a898c38)
 2016-05-08 23:02:07 +05:30
Yongjun Zhang
5172d0e7b1 YARN-5048. DelegationTokenRenewer#skipTokenRenewal may throw NPE (Jian He via Yongjun Zhang) 
(cherry picked from commit 47c41e7ac7e6b905a58550f8899f629c1cf8b138)
 2016-05-06 22:38:56 -07:00
Jason Lowe
3895058a67 YARN-4747. AHS error 500 due to NPE when container start event is missing. Contributed by Varun Saxena 
(cherry picked from commit b2ed6ae73197990a950ce71ece80c0f23221c384)
 2016-05-06 23:00:25 +00:00
Wangda Tan
b68e6b1d6d getApplicationReport call may raise NPE for removed queues. (Jian He via wangda) 
(cherry picked from commit 23248f63aab74a19dba38d348f2b231c8360770a)
 2016-05-06 15:32:15 -07:00
Jian He
a6b24c62ab YARN-4390. Do surgical preemption based on reserved container in CapacityScheduler. Contributed by Wangda Tan 
(cherry picked from commit bb62e0592566b2fcae7136b30972aad2d3ac55b0)
 2016-05-05 12:56:55 -07:00
Jason Lowe
ee86cef2fe YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla 
(cherry picked from commit d0da13229cf692579c8c9db47a93f6c6255392c8)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNode.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNodes.java
 2016-05-05 14:33:01 +00:00
Varun Vasudev
38a3b86141 YARN-4595. Add support for configurable read-only mounts when launching Docker containers. Contributed by Billie Rinaldi. 
(cherry picked from commit 72b047715c1ae89dff3dfe76c1af91dfe255ed70)
 2016-05-05 13:02:38 +05:30
Vinod Kumar Vavilapalli
8262ef8318 YARN-4905. Improved "yarn logs" command-line to optionally show log metadata also. Contributed by Xuan Gong. 
(cherry picked from commit 9e37fe3b7a3b5f0a193d228bb5e065f41acd2835)
 2016-05-04 14:17:09 -07:00
Wangda Tan
585299146a YARN-4984. LogAggregationService shouldn't swallow exception in handling createAppDir() which cause thread leak. (Junping Du via wangda) 
(cherry picked from commit 7bd418e48c71590fc8026d69f9b8f8ad42f2aade)
 2016-05-04 11:39:25 -07:00
Junping Du
1ffb0c43d6 YARN-4920. ATS/NM should support a link to dowload/get the logs in text format. Contributed by Xuan Gong. 
(cherry picked from commit 3cf223166d452a0f58f92676837a9edb8ddc1139)
 2016-05-04 10:36:31 -07:00
Rohith Sharma K S
5aad4070b2 YARN-4947. Test timeout is happening for TestRMWebServicesNodes. Contributed by Bibin A Chundatt 
(cherry picked from commit 75e0450593680fb6e57fbef4f4f3af1487023b5d)
 2016-05-04 10:26:25 +05:30
Jason Lowe
baac4e7db1 YARN-5003. Add container resource to RM audit log. Contributed by Nathan Roberts 
(cherry picked from commit ed54f5f1ff7862f8216f77d5ea8f9ccea674ccd1)
 2016-05-03 22:16:17 +00:00
Jason Lowe
d8fcf02069 YARN-4834. ProcfsBasedProcessTree doesn't track daemonized processes. Contributed by Nathan Roberts 
(cherry picked from commit c6b48391680c1b81a86aabc3ad4c725bfade6d2e)
 2016-05-03 22:14:50 +00:00
Junping Du
47f67ae447 YARN-4851. Metric improvements for ATS v1.5 storage components. Li Lu via junping_du. 
(cherry picked from commit 06413da72efed9a50e49efaf7110c220c88a7f4a)
 2016-05-03 04:18:01 -07:00
Robert Kanter
ac8fb579c6 Remove parent's env vars from child processes 2016-04-29 09:26:09 -07:00