Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,377 Commits 326 Branches 371 Tags
Commit Graph

2641 Commits

Author SHA1 Message Date
Jason Lowe
575c056357 YARN-4325. Nodemanager log handlers fail to send finished/failed events in some cases. Contributed by Junping Du 
(cherry picked from commit 81effb7dcde2b31423438d6f1b8b8204d4ca05b3)
 2016-05-16 15:43:20 +00:00
Rohith Sharma K S
a37b3694ea YARN-5068. Expose scheduler queue to application master. (Harish Jaiprakash via rohithsharmaks) 2016-05-16 15:25:15 +05:30
Xuan
3fe4107909 YARN-5080. Cannot obtain logs using YARN CLI -am for either KILLED or 
RUNNING AM. Contributed by Xuan Gong

(cherry picked from commit 7be53b65f95dae89c760cde354b42f15f5ee3b5f)
 2016-05-14 12:22:06 -07:00
Jason Lowe
bb95a5e18f YARN-5053. More informative diagnostics when applications killed by a user. Contributed by Eric Badger 
(cherry picked from commit 013000fbc2081c2b7e78b790fc6d847de92b9384)
 2016-05-12 20:30:10 +00:00
Sangjin Lee
d6d13ec670 YARN-4577. Enable aux services to have their own custom classpath/jar file (Xuan Gong via sale) 
(cherry picked from commit 0bbe01f8d56191edfba3b50fb9f8859a0b3f826f)
 2016-05-12 10:14:24 -07:00
Karthik Kambatla
f96b2b6fac YARN-4995. FairScheduler: Display per-queue demand on the scheduler page. (xupeng via kasha) 
(cherry picked from commit 4b4e4c6ba83bc5c41d7bb69bb2483bcfe894a260)
 2016-05-11 17:36:42 -07:00
Junping Du
9dc3683d87 YARN-5029. RM needs to send update event with YarnApplicationState as Running to ATS/AHS. Contributed by Xuan Gong. 
(cherry picked from commit 39f2bac38b111f90d3402229201cdb4315f5d4f5)
 2016-05-11 09:33:16 -07:00
Vinod Kumar Vavilapalli
e0d6a9632d YARN-4842. Fixed "yarn logs" command to guess (and thus not require) the appOwner argument when viewing another user's logs. Contributed by Ram Venkatesh and Xuan Gong. 
(cherry picked from commit 87f5e351337a905af5215af76c72b9312616cd4f)
 2016-05-09 22:47:58 -07:00
Naganarasimha
3732a1e985 YARN-4926. Change nodelabel rest API invalid reponse status to 400. Contributed by Bibin A Chundatt 
(cherry picked from commit 2750fb900fc9c468570d051cb19e3a379a898c38)
 2016-05-08 23:02:07 +05:30
Yongjun Zhang
5172d0e7b1 YARN-5048. DelegationTokenRenewer#skipTokenRenewal may throw NPE (Jian He via Yongjun Zhang) 
(cherry picked from commit 47c41e7ac7e6b905a58550f8899f629c1cf8b138)
 2016-05-06 22:38:56 -07:00
Jason Lowe
3895058a67 YARN-4747. AHS error 500 due to NPE when container start event is missing. Contributed by Varun Saxena 
(cherry picked from commit b2ed6ae73197990a950ce71ece80c0f23221c384)
 2016-05-06 23:00:25 +00:00
Wangda Tan
b68e6b1d6d getApplicationReport call may raise NPE for removed queues. (Jian He via wangda) 
(cherry picked from commit 23248f63aab74a19dba38d348f2b231c8360770a)
 2016-05-06 15:32:15 -07:00
Jian He
a6b24c62ab YARN-4390. Do surgical preemption based on reserved container in CapacityScheduler. Contributed by Wangda Tan 
(cherry picked from commit bb62e0592566b2fcae7136b30972aad2d3ac55b0)
 2016-05-05 12:56:55 -07:00
Jason Lowe
ee86cef2fe YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla 
(cherry picked from commit d0da13229cf692579c8c9db47a93f6c6255392c8)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNode.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNodes.java
 2016-05-05 14:33:01 +00:00
Varun Vasudev
38a3b86141 YARN-4595. Add support for configurable read-only mounts when launching Docker containers. Contributed by Billie Rinaldi. 
(cherry picked from commit 72b047715c1ae89dff3dfe76c1af91dfe255ed70)
 2016-05-05 13:02:38 +05:30
Vinod Kumar Vavilapalli
8262ef8318 YARN-4905. Improved "yarn logs" command-line to optionally show log metadata also. Contributed by Xuan Gong. 
(cherry picked from commit 9e37fe3b7a3b5f0a193d228bb5e065f41acd2835)
 2016-05-04 14:17:09 -07:00
Wangda Tan
585299146a YARN-4984. LogAggregationService shouldn't swallow exception in handling createAppDir() which cause thread leak. (Junping Du via wangda) 
(cherry picked from commit 7bd418e48c71590fc8026d69f9b8f8ad42f2aade)
 2016-05-04 11:39:25 -07:00
Junping Du
1ffb0c43d6 YARN-4920. ATS/NM should support a link to dowload/get the logs in text format. Contributed by Xuan Gong. 
(cherry picked from commit 3cf223166d452a0f58f92676837a9edb8ddc1139)
 2016-05-04 10:36:31 -07:00
Rohith Sharma K S
5aad4070b2 YARN-4947. Test timeout is happening for TestRMWebServicesNodes. Contributed by Bibin A Chundatt 
(cherry picked from commit 75e0450593680fb6e57fbef4f4f3af1487023b5d)
 2016-05-04 10:26:25 +05:30
Jason Lowe
baac4e7db1 YARN-5003. Add container resource to RM audit log. Contributed by Nathan Roberts 
(cherry picked from commit ed54f5f1ff7862f8216f77d5ea8f9ccea674ccd1)
 2016-05-03 22:16:17 +00:00
Jason Lowe
d8fcf02069 YARN-4834. ProcfsBasedProcessTree doesn't track daemonized processes. Contributed by Nathan Roberts 
(cherry picked from commit c6b48391680c1b81a86aabc3ad4c725bfade6d2e)
 2016-05-03 22:14:50 +00:00
Junping Du
47f67ae447 YARN-4851. Metric improvements for ATS v1.5 storage components. Li Lu via junping_du. 
(cherry picked from commit 06413da72efed9a50e49efaf7110c220c88a7f4a)
 2016-05-03 04:18:01 -07:00
Robert Kanter
ac8fb579c6 Remove parent's env vars from child processes 2016-04-29 09:26:09 -07:00
Varun Vasudev
6561e3b500 YARN-3998. Add support in the NodeManager to re-launch containers. Contributed by Jun Gong. 
(cherry picked from commit 0f25a1bb52bc56661fd020a6ba82df99f8c6ef1f)

 Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ProtoUtils.java
 2016-04-29 16:22:06 +05:30
Jian He
6ba39a1597 YARN-5009. NMLeveldbStateStoreService database can grow substantially leading to longer recovery times. Contributed by Jason Lowe 
(cherry picked from commit 4a8508501bc753858693dacdafba61d604702f71)
 2016-04-28 21:54:30 -07:00
Jian He
a9707dceaf YARN-5008. LeveldbRMStateStore database can grow substantially leading to long recovery times. Contributed by Jason Lowe 2016-04-28 21:28:03 -07:00
Li Lu
50b7a35d56 YARN-4956. findbug issue on LevelDBCacheTimelineStore. (Zhiyuan Yang via gtcarrera9) 
(cherry picked from commit f16722d2ef31338a57a13e2c8d18c1c62d58bbaf)
 2016-04-27 10:58:12 -07:00
Karthik Kambatla
864ecb4434 YARN-4807. MockAM#waitForState sleep duration is too long. (Yufei Gu via kasha) 
(cherry picked from commit 185c3d4de1ac4cf10cc1aa00aaaaf367b3880b80)
 2016-04-27 09:43:42 -07:00
Varun Vasudev
cf3e93ee73 YARN-4966. Improve yarn logs to fetch container logs without specifying nodeId. Contributed by Xuan Gong. 
(cherry picked from commit 66b07d83740a2ec3e6bfb2bfd064863bae37a1b5)
 2016-04-27 15:57:17 +05:30
Jian He
9d3ddb0b4d YARN-4983. JVM and UGI metrics disappear after RM transitioned to standby mode 
(cherry picked from commit 4beff013546dbc29c004a1bad1b019dc9d2b751f)
 2016-04-26 21:02:04 -07:00
Karthik Kambatla
52bfa90fed YARN-4795. ContainerMetrics drops records. (Daniel Templeton via kasha) 
(cherry picked from commit 1a3f1482e2738c7f9a983bc55189116930388d7b)
 2016-04-26 06:18:27 -07:00
Karthik Kambatla
a5edb45b18 YARN-1297. FairScheduler: Move some logs to debug and check if debug logging is enabled 
(cherry picked from commit 4b1dcbbe0c1d0036f65283be4b25d9b2211abed3)
 2016-04-26 05:10:29 -07:00
Wangda Tan
45ff579bfa YARN-4846. Fix random failures for TestCapacitySchedulerPreemption#testPreemptionPolicyShouldRespectAlreadyMarkedKillableContainers. (Bibin A Chundatt via wangda) 
(cherry picked from commit 7cb3a3da96e59fc9b6528644dae5fb0ac1e44eac)
 2016-04-22 11:41:08 -07:00
Eric Payne
a7f903b2ba YARN-4556. TestFifoScheduler.testResourceOverCommit fails. Contributed by Akihiro Suda 
(cherry picked from commit 3dce486d88895dcdf443f4d0064d1fb6e9116045)
 2016-04-21 21:27:10 +00:00
Li Lu
054fa104c5 YARN-4968. A couple of AM retry unit tests need to wait SchedulerApplicationAttempt stopped. (Wangda Tan via gtcarrera9) 
(cherry picked from commit 7c6339f66ac301406504be28841bc3f3bfebc8ae)
 2016-04-21 13:27:47 -07:00
Chris Douglas
878924189d YARN-4976. Missing NullPointer check in ContainerLaunchContextPBImpl. Contributed by Giovanni Matteo Fumarola 
(cherry picked from commit 95a50466075c28110fa7c297e9c5246892076ca8)
 2016-04-21 10:46:35 -07:00
Karthik Kambatla
75cf238354 YARN-4784. Fairscheduler: defaultQueueSchedulingPolicy should not accept FIFO. (Yufei Gu via kasha) 
(cherry picked from commit 170c4fd4cdbb8b4c1c4bb219e4c74fa0ba81b055)
 2016-04-20 23:58:30 -07:00
Karthik Kambatla
cd74cf9525 YARN-4935. TestYarnClient#testSubmitIncorrectQueue fails when FairScheduler is the default. (Yufei Gu via kasha) 
(cherry picked from commit 6e297836d64498d90d0cd343789db2f2d490378a)
 2016-04-20 23:41:03 -07:00
Wangda Tan
83a5cdc400 YARN-4890. Unit test intermittent failure: TestNodeLabelContainerAllocation#testQueueUsedCapacitiesUpdate. (Sunil G via wangda) 
(cherry picked from commit 33fd95a99c4df6c4de6b0a53dc6679f3cab629e0)
 2016-04-20 17:38:22 -07:00
Vinod Kumar Vavilapalli
342bb438f2 Fixed TimelineClient to retry SocketTimeoutException too. Contributed by Xuan Gong. 
(cherry picked from commit 477003730e6a7c7eff11892f5cedf74073ca867b)
 2016-04-18 11:49:00 -07:00
Wangda Tan
41cafeb5a1 YARN-4934. Reserved Resource for QueueMetrics needs to be handled correctly in few cases. (Sunil G via wangda) 
(cherry picked from commit fdc46bfb37776d8c41b68f6c33a2379d0f329994)
 2016-04-16 22:50:00 -07:00
Xuan
94e4d349b0 YARN-4965. Distributed shell AM failed due to ClientHandlerException thrown by jersey. Contributed by Junping Du 
(cherry picked from commit e6c0742012ffeacad2bcaf712d86a7e5d1420b26)
 2016-04-16 19:40:11 -07:00
Arun Suresh
8460a323ce YARN-4468. Document the general ReservationSystem functionality, and the REST API. (subru and carlo via asuresh) 2016-04-15 17:20:05 -07:00
Jason Lowe
cd148cb347 YARN-4940. yarn node -list -all failed if RM start with decommissioned node. Contributed by sandflee 
(cherry picked from commit 69f3d428d5c3ab0c79cacffc22b1f59408622ae7)
 2016-04-15 20:38:04 +00:00
Naganarasimha
2213a40f30 YARN-4909. Fix intermittent failures of TestRMWebServices And TestRMWithCSRFFilter. Contributed by Bibin A Chundatt 
(cherry picked from commit fdbafbc9e59314d9f9f75e615de9d2dfdced017b)
 2016-04-15 23:40:07 +05:30
Jason Lowe
ece01478c5 YARN-4924. NM recovery race can lead to container not cleaned up. Contributed by sandflee 
(cherry picked from commit 3150ae8108a1fc40a67926be6254824c1e37cb38)
 2016-04-14 19:19:46 +00:00
Robert Kanter
e79a47670b YARN-4541. Change log message in LocalizedResource#handle() to DEBUG (rchiang via rkanter) 
(cherry picked from commit 0d9194df00fd68bfb7a8ba504b0cddd7d7c69b8a)
 2016-04-13 17:45:36 -07:00
Xuan
5bc64dafc3 YARN-4886. Add HDFS caller context for EntityGroupFSTimelineStore. Contributed by Li Lu 
(cherry picked from commit e0cb426758b3d716ff143f723fc16ef2f1e4971b)
 2016-04-13 10:39:31 -07:00
Naganarasimha
53c24e00e8 YARN-4810. NM applicationpage cause internal error 500. Contributed by Bibin A Chundatt. 
(cherry picked from commit 437e9d6475a91cafc4c993b206312912b5f13ad9)
 2016-04-12 18:25:11 +05:30
Rohith Sharma K S
d93cfefea5 YARN-4897. dataTables_wrapper change min height. Contributed by Bibin A Chundatt. 
(cherry picked from commit 4ee7f0a3e7814d66d6e11aa4b559da5ac93c4b99)
 2016-04-12 10:28:18 +05:30