Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,977 Commits 326 Branches 371 Tags
Commit Graph

2030 Commits

Author SHA1 Message Date
Jason Lowe
575c056357 YARN-4325. Nodemanager log handlers fail to send finished/failed events in some cases. Contributed by Junping Du 
(cherry picked from commit 81effb7dcde2b31423438d6f1b8b8204d4ca05b3)
 2016-05-16 15:43:20 +00:00
Rohith Sharma K S
a37b3694ea YARN-5068. Expose scheduler queue to application master. (Harish Jaiprakash via rohithsharmaks) 2016-05-16 15:25:15 +05:30
Jason Lowe
bb95a5e18f YARN-5053. More informative diagnostics when applications killed by a user. Contributed by Eric Badger 
(cherry picked from commit 013000fbc2081c2b7e78b790fc6d847de92b9384)
 2016-05-12 20:30:10 +00:00
Sangjin Lee
d6d13ec670 YARN-4577. Enable aux services to have their own custom classpath/jar file (Xuan Gong via sale) 
(cherry picked from commit 0bbe01f8d56191edfba3b50fb9f8859a0b3f826f)
 2016-05-12 10:14:24 -07:00
Karthik Kambatla
f96b2b6fac YARN-4995. FairScheduler: Display per-queue demand on the scheduler page. (xupeng via kasha) 
(cherry picked from commit 4b4e4c6ba83bc5c41d7bb69bb2483bcfe894a260)
 2016-05-11 17:36:42 -07:00
Junping Du
9dc3683d87 YARN-5029. RM needs to send update event with YarnApplicationState as Running to ATS/AHS. Contributed by Xuan Gong. 
(cherry picked from commit 39f2bac38b111f90d3402229201cdb4315f5d4f5)
 2016-05-11 09:33:16 -07:00
Naganarasimha
3732a1e985 YARN-4926. Change nodelabel rest API invalid reponse status to 400. Contributed by Bibin A Chundatt 
(cherry picked from commit 2750fb900fc9c468570d051cb19e3a379a898c38)
 2016-05-08 23:02:07 +05:30
Yongjun Zhang
5172d0e7b1 YARN-5048. DelegationTokenRenewer#skipTokenRenewal may throw NPE (Jian He via Yongjun Zhang) 
(cherry picked from commit 47c41e7ac7e6b905a58550f8899f629c1cf8b138)
 2016-05-06 22:38:56 -07:00
Jason Lowe
3895058a67 YARN-4747. AHS error 500 due to NPE when container start event is missing. Contributed by Varun Saxena 
(cherry picked from commit b2ed6ae73197990a950ce71ece80c0f23221c384)
 2016-05-06 23:00:25 +00:00
Wangda Tan
b68e6b1d6d getApplicationReport call may raise NPE for removed queues. (Jian He via wangda) 
(cherry picked from commit 23248f63aab74a19dba38d348f2b231c8360770a)
 2016-05-06 15:32:15 -07:00
Jian He
a6b24c62ab YARN-4390. Do surgical preemption based on reserved container in CapacityScheduler. Contributed by Wangda Tan 
(cherry picked from commit bb62e0592566b2fcae7136b30972aad2d3ac55b0)
 2016-05-05 12:56:55 -07:00
Jason Lowe
ee86cef2fe YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla 
(cherry picked from commit d0da13229cf692579c8c9db47a93f6c6255392c8)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNode.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNodes.java
 2016-05-05 14:33:01 +00:00
Varun Vasudev
38a3b86141 YARN-4595. Add support for configurable read-only mounts when launching Docker containers. Contributed by Billie Rinaldi. 
(cherry picked from commit 72b047715c1ae89dff3dfe76c1af91dfe255ed70)
 2016-05-05 13:02:38 +05:30
Wangda Tan
585299146a YARN-4984. LogAggregationService shouldn't swallow exception in handling createAppDir() which cause thread leak. (Junping Du via wangda) 
(cherry picked from commit 7bd418e48c71590fc8026d69f9b8f8ad42f2aade)
 2016-05-04 11:39:25 -07:00
Junping Du
1ffb0c43d6 YARN-4920. ATS/NM should support a link to dowload/get the logs in text format. Contributed by Xuan Gong. 
(cherry picked from commit 3cf223166d452a0f58f92676837a9edb8ddc1139)
 2016-05-04 10:36:31 -07:00
Rohith Sharma K S
5aad4070b2 YARN-4947. Test timeout is happening for TestRMWebServicesNodes. Contributed by Bibin A Chundatt 
(cherry picked from commit 75e0450593680fb6e57fbef4f4f3af1487023b5d)
 2016-05-04 10:26:25 +05:30
Jason Lowe
baac4e7db1 YARN-5003. Add container resource to RM audit log. Contributed by Nathan Roberts 
(cherry picked from commit ed54f5f1ff7862f8216f77d5ea8f9ccea674ccd1)
 2016-05-03 22:16:17 +00:00
Junping Du
47f67ae447 YARN-4851. Metric improvements for ATS v1.5 storage components. Li Lu via junping_du. 
(cherry picked from commit 06413da72efed9a50e49efaf7110c220c88a7f4a)
 2016-05-03 04:18:01 -07:00
Robert Kanter
ac8fb579c6 Remove parent's env vars from child processes 2016-04-29 09:26:09 -07:00
Varun Vasudev
6561e3b500 YARN-3998. Add support in the NodeManager to re-launch containers. Contributed by Jun Gong. 
(cherry picked from commit 0f25a1bb52bc56661fd020a6ba82df99f8c6ef1f)

 Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ProtoUtils.java
 2016-04-29 16:22:06 +05:30
Jian He
6ba39a1597 YARN-5009. NMLeveldbStateStoreService database can grow substantially leading to longer recovery times. Contributed by Jason Lowe 
(cherry picked from commit 4a8508501bc753858693dacdafba61d604702f71)
 2016-04-28 21:54:30 -07:00
Jian He
a9707dceaf YARN-5008. LeveldbRMStateStore database can grow substantially leading to long recovery times. Contributed by Jason Lowe 2016-04-28 21:28:03 -07:00
Li Lu
50b7a35d56 YARN-4956. findbug issue on LevelDBCacheTimelineStore. (Zhiyuan Yang via gtcarrera9) 
(cherry picked from commit f16722d2ef31338a57a13e2c8d18c1c62d58bbaf)
 2016-04-27 10:58:12 -07:00
Karthik Kambatla
864ecb4434 YARN-4807. MockAM#waitForState sleep duration is too long. (Yufei Gu via kasha) 
(cherry picked from commit 185c3d4de1ac4cf10cc1aa00aaaaf367b3880b80)
 2016-04-27 09:43:42 -07:00
Jian He
9d3ddb0b4d YARN-4983. JVM and UGI metrics disappear after RM transitioned to standby mode 
(cherry picked from commit 4beff013546dbc29c004a1bad1b019dc9d2b751f)
 2016-04-26 21:02:04 -07:00
Karthik Kambatla
52bfa90fed YARN-4795. ContainerMetrics drops records. (Daniel Templeton via kasha) 
(cherry picked from commit 1a3f1482e2738c7f9a983bc55189116930388d7b)
 2016-04-26 06:18:27 -07:00
Karthik Kambatla
a5edb45b18 YARN-1297. FairScheduler: Move some logs to debug and check if debug logging is enabled 
(cherry picked from commit 4b1dcbbe0c1d0036f65283be4b25d9b2211abed3)
 2016-04-26 05:10:29 -07:00
Wangda Tan
45ff579bfa YARN-4846. Fix random failures for TestCapacitySchedulerPreemption#testPreemptionPolicyShouldRespectAlreadyMarkedKillableContainers. (Bibin A Chundatt via wangda) 
(cherry picked from commit 7cb3a3da96e59fc9b6528644dae5fb0ac1e44eac)
 2016-04-22 11:41:08 -07:00
Eric Payne
a7f903b2ba YARN-4556. TestFifoScheduler.testResourceOverCommit fails. Contributed by Akihiro Suda 
(cherry picked from commit 3dce486d88895dcdf443f4d0064d1fb6e9116045)
 2016-04-21 21:27:10 +00:00
Li Lu
054fa104c5 YARN-4968. A couple of AM retry unit tests need to wait SchedulerApplicationAttempt stopped. (Wangda Tan via gtcarrera9) 
(cherry picked from commit 7c6339f66ac301406504be28841bc3f3bfebc8ae)
 2016-04-21 13:27:47 -07:00
Karthik Kambatla
75cf238354 YARN-4784. Fairscheduler: defaultQueueSchedulingPolicy should not accept FIFO. (Yufei Gu via kasha) 
(cherry picked from commit 170c4fd4cdbb8b4c1c4bb219e4c74fa0ba81b055)
 2016-04-20 23:58:30 -07:00
Wangda Tan
83a5cdc400 YARN-4890. Unit test intermittent failure: TestNodeLabelContainerAllocation#testQueueUsedCapacitiesUpdate. (Sunil G via wangda) 
(cherry picked from commit 33fd95a99c4df6c4de6b0a53dc6679f3cab629e0)
 2016-04-20 17:38:22 -07:00
Wangda Tan
41cafeb5a1 YARN-4934. Reserved Resource for QueueMetrics needs to be handled correctly in few cases. (Sunil G via wangda) 
(cherry picked from commit fdc46bfb37776d8c41b68f6c33a2379d0f329994)
 2016-04-16 22:50:00 -07:00
Jason Lowe
cd148cb347 YARN-4940. yarn node -list -all failed if RM start with decommissioned node. Contributed by sandflee 
(cherry picked from commit 69f3d428d5c3ab0c79cacffc22b1f59408622ae7)
 2016-04-15 20:38:04 +00:00
Jason Lowe
ece01478c5 YARN-4924. NM recovery race can lead to container not cleaned up. Contributed by sandflee 
(cherry picked from commit 3150ae8108a1fc40a67926be6254824c1e37cb38)
 2016-04-14 19:19:46 +00:00
Robert Kanter
e79a47670b YARN-4541. Change log message in LocalizedResource#handle() to DEBUG (rchiang via rkanter) 
(cherry picked from commit 0d9194df00fd68bfb7a8ba504b0cddd7d7c69b8a)
 2016-04-13 17:45:36 -07:00
Xuan
5bc64dafc3 YARN-4886. Add HDFS caller context for EntityGroupFSTimelineStore. Contributed by Li Lu 
(cherry picked from commit e0cb426758b3d716ff143f723fc16ef2f1e4971b)
 2016-04-13 10:39:31 -07:00
Naganarasimha
53c24e00e8 YARN-4810. NM applicationpage cause internal error 500. Contributed by Bibin A Chundatt. 
(cherry picked from commit 437e9d6475a91cafc4c993b206312912b5f13ad9)
 2016-04-12 18:25:11 +05:30
Vinod Kumar Vavilapalli
f1dcd40294 YARN-4168. Fixed a failing test TestLogAggregationService.testLocalFileDeletionOnDiskFull. Contributed by Takashi Ohnishi. 
(cherry picked from commit 44bbc50d919388e4ad08be2e9ba80ac7502d2579)
 2016-04-11 12:12:30 -07:00
Jason Lowe
f1a370ce8b Revert "YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla" 
This reverts commit 814ceeb4893ad9ed903ebcfd30fe3158ec1c71df.
 2016-04-11 15:56:29 +00:00
Junping Du
a3f8491410 YARN-4928. Some yarn.server.timeline.* tests fail on Windows attempting to use a test root path containing a colon. Contributed by Gergely Novák. 
(cherry picked from commit 08ddb3ac6da634c293c638d76b53af163ede3057)
 2016-04-11 08:51:22 -07:00
Akira Ajisaka
8cf6630fc6 YARN-4630. Remove useless boxing/unboxing code. Contributed by Kousuke Saruta. 
(cherry picked from commit 1ff27f9d12e8124c1b9a722708264c5b07fd0fde)
 2016-04-11 14:55:37 +09:00
Akira Ajisaka
88556294e2 YARN-4938. MiniYarnCluster should not request transitionToActive to RM on non-HA environment. Contributed by Eric Badger. 
(cherry picked from commit 1b78b2ba17cffdd19ae056555dd7f89b7a3eb06a)
 2016-04-11 01:32:49 +09:00
Karthik Kambatla
94a88ae87b YARN-4927. TestRMHA#testTransitionedToActiveRefreshFail fails with FairScheduler. (Bibin A Chundatt via kasha) 
(cherry picked from commit ff95fd547b3030529983fa4e701ac728c54ab63a)
 2016-04-09 10:31:29 -07:00
Wangda Tan
12ccdd6540 YARN-3215. Respect labels in CapacityScheduler when computing headroom. (Naganarasimha G R via wangda) 
(cherry picked from commit ec06957941367930c855b5e05e6a84ba676fd46a)
 2016-04-08 15:34:24 -07:00
Jian He
77a75de319 YARN-4740. AM may not receive the container complete msg when it restarts. Contributed by Jun Gong 2016-04-08 11:21:07 -07:00
Karthik Kambatla
2b97a50eec YARN-4756. Unnecessary wait in Node Status Updater during reboot. (Eric Badger via kasha) 
(cherry picked from commit e82f961a3925aadf9e53a009820a48ba9e4f78b6)
 2016-04-07 17:30:54 -07:00
Jian He
42bc565630 YARN-4769. Add support for CSRF header in the dump capacity scheduler logs and kill app buttons in RM web UI. Contributed by Varun Vasudev 2016-04-06 16:14:13 -07:00
Varun Vasudev
8f9b97ccce YARN-4906. Capture container start/finish time in container metrics. Contributed by Jian He. 
(cherry picked from commit b41e65e5bc9459b4d950a2c53860a223f1a0d2ec)
 2016-04-06 13:42:06 +05:30
Wangda Tan
11e796b5cd YARN-4699. Scheduler UI and REST o/p is not in sync when -replaceLabelsOnNode is used to change label of a node. (Sunil G via wangda) 
(cherry picked from commit 21eb4284487d6f8e4beedb8a0c3168e952f224fc)
 2016-04-05 16:25:55 -07:00