Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,379 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

2629 Commits

Author SHA1 Message Date
Jason Lowe 8013529746 YARN-4834. ProcfsBasedProcessTree doesn't track daemonized processes. Contributed by Nathan Roberts 
(cherry picked from commit c6b4839168)
 2016-05-03 22:15:31 +00:00
Junping Du 4f1c198881 YARN-4851. Metric improvements for ATS v1.5 storage components. Li Lu via junping_du. 
(cherry picked from commit 06413da72e)
(cherry picked from commit 47f67ae447)
 2016-05-03 04:59:57 -07:00
rohithsharmaks 9ea567da42 YARN-4109. Exception on RM scheduler page loading with labels. (Mohammad Shahid Khan via rohithsharmaks) 
(cherry picked from commit 8c180a13c8)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2016-05-03 10:31:32 +05:30
Robert Kanter 3c3d003402 Remove parent's env vars from child processes 
(cherry picked from commit ac8fb579c6)
 2016-04-29 15:30:06 -07:00
Jian He 5ba79d77fb YARN-5009. NMLeveldbStateStoreService database can grow substantially leading to longer recovery times. Contributed by Jason Lowe 
(cherry picked from commit 4a8508501b)
 2016-04-28 21:54:53 -07:00
Jian He e62162db46 YARN-5008. LeveldbRMStateStore database can grow substantially leading to long recovery times. Contributed by Jason Lowe 
(cherry picked from commit dd80042c42)
 2016-04-28 21:36:23 -07:00
Li Lu 76b4ec32e6 YARN-4956. findbug issue on LevelDBCacheTimelineStore. (Zhiyuan Yang via gtcarrera9) 
(cherry picked from commit f16722d2ef)
 2016-04-27 10:58:50 -07:00
Wangda Tan 358b54d063 YARN-3215. Respect labels in CapacityScheduler when computing headroom. (Naganarasimha G R via wangda) 2016-04-22 15:59:49 -07:00
Eric Payne b726ae0876 YARN-4556. TestFifoScheduler.testResourceOverCommit fails. Contributed by Akihiro Suda 
(cherry picked from commit 3dce486d88)
 2016-04-21 22:01:07 +00:00
Vinod Kumar Vavilapalli 92e45b8321 Fixed TimelineClient to retry SocketTimeoutException too. Contributed by Xuan Gong. 
(cherry picked from commit 477003730e)
 2016-04-18 11:51:21 -07:00
Xuan 84ab382939 YARN-4965. Distributed shell AM failed due to ClientHandlerException thrown by jersey. Contributed by Junping Du 
(cherry picked from commit e6c0742012)
(cherry picked from commit 94e4d349b0)
 2016-04-16 19:40:46 -07:00
Arun Suresh aa457912ca YARN-4468. Document the general ReservationSystem functionality, and the REST API. (subru and carlo via asuresh) 
(cherry picked from commit 8460a323ce)
 2016-04-15 17:24:44 -07:00
Jason Lowe d389afb3da YARN-4940. yarn node -list -all failed if RM start with decommissioned node. Contributed by sandflee 
(cherry picked from commit 69f3d428d5)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestClientRMService.java
 2016-04-15 20:45:20 +00:00
Naganarasimha 3e223d5dc0 YARN-4909. Fix intermittent failures of TestRMWebServices And TestRMWithCSRFFilter. Contributed by Bibin A Chundatt 
(cherry picked from commit fdbafbc9e5)
 2016-04-16 00:00:03 +05:30
Jason Lowe 9b5c5bd42f YARN-4924. NM recovery race can lead to container not cleaned up. Contributed by sandflee 
(cherry picked from commit 3150ae8108)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/recovery/NMLeveldbStateStoreService.java
 2016-04-14 19:40:10 +00:00
Xuan 3127b86492 YARN-4886. Add HDFS caller context for EntityGroupFSTimelineStore. Contributed by Li Lu 
(cherry picked from commit e0cb426758)
(cherry picked from commit 5bc64dafc3)
 2016-04-13 10:40:04 -07:00
Junping Du c85a63aba1 YARN-4928. Some yarn.server.timeline.* tests fail on Windows attempting to use a test root path containing a colon. Contributed by Gergely Novák. 
(cherry picked from commit 08ddb3ac6d)
(cherry picked from commit a3f8491410)
 2016-04-13 09:05:09 -07:00
Rohith Sharma K S 9d4dedd65f YARN-4794. Deadlock in NMClientImpl. Contributed by Jian He. 
(cherry picked from commit ff722bbbdf)
 2016-04-12 10:05:35 +05:30
Vinod Kumar Vavilapalli 3589b9e10e YARN-4168. Fixed a failing test TestLogAggregationService.testLocalFileDeletionOnDiskFull. Contributed by Takashi Ohnishi. 
(cherry picked from commit 44bbc50d91)
 2016-04-11 12:13:37 -07:00
Jason Lowe 3cf7856878 Revert "YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla" 
This reverts commit 9d3c51eb5a.
 2016-04-11 15:57:56 +00:00
Akira Ajisaka 13be7a849d YARN-4630. Remove useless boxing/unboxing code. Contributed by Kousuke Saruta. 
(cherry picked from commit 1ff27f9d12)
(cherry picked from commit 8cf6630fc6)
 2016-04-11 14:55:54 +09:00
Akira Ajisaka 35fd4d4124 YARN-4938. MiniYarnCluster should not request transitionToActive to RM on non-HA environment. Contributed by Eric Badger. 
(cherry picked from commit 1b78b2ba17)
(cherry picked from commit 88556294e2)
 2016-04-11 01:33:02 +09:00
Wangda Tan 67ee302723 YARN-4826. Document configuration of ReservationSystem for CapacityScheduler. (Subru Krishnan via wangda) 
(cherry picked from commit fcb3fcd4c6)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/CapacityScheduler.md

(cherry picked from commit dac2021f87)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/CapacityScheduler.md
 2016-04-08 16:33:55 -07:00
Jian He 4cf0b1932f YARN-4740. AM may not receive the container complete msg when it restarts. Contributed by Jun Gong 2016-04-08 11:21:34 -07:00
Karthik Kambatla ddb1407980 YARN-4756. Unnecessary wait in Node Status Updater during reboot. (Eric Badger via kasha) 
(cherry picked from commit e82f961a39)
(cherry picked from commit 2b97a50eec)
 2016-04-07 17:35:06 -07:00
Wangda Tan b7c83d4883 YARN-4699. Scheduler UI and REST o/p is not in sync when -replaceLabelsOnNode is used to change label of a node. (Sunil G via wangda) 
(cherry picked from commit 21eb428448)
 2016-04-05 16:30:56 -07:00
Masatake Iwasaki deef54b261 YARN-4917. Fix typos in documentation of Capacity Scheduler. (Takashi Ohnishi via iwasakims) 
(cherry picked from commit 500e5a5952)
 2016-04-06 04:01:48 +09:00
Masatake Iwasaki 90adda5da5 YARN-4915. Fix typo in YARN Secure Containers documentation (Takashi Ohnishi via iwasakims) 
(cherry picked from commit 30206346cf)
 2016-04-06 03:47:53 +09:00
Junping Du 3df56a97df Small addendum to YARN-4893 to fix build failure. 2016-04-05 08:41:48 -07:00
Junping Du 9c2dc1a194 YARN-4893. Fix some intermittent test failures in TestRMAdminService. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 6be28bcc46)
(cherry picked from commit eeff2e35f8)
 2016-04-05 08:41:48 -07:00
Jason Lowe 9d3c51eb5a YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla 
(cherry picked from commit 1cbcd4a491)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
 2016-04-05 13:47:05 +00:00
Li Lu 3574b4559c YARN-4706. UI Hosting Configuration in TimelineServer doc is broken. (Akira AJISAKA via gtcarrera9) 
(cherry picked from commit f61de41736)
 2016-04-04 14:43:37 -07:00
naganarasimha 9bd089ac64 YARN-4746. yarn web services should convert parse failures of appId, appAttemptId and containerId to 400. Contributed by Bibin A Chundatt 
(cherry picked from commit 5092c94195)
 2016-04-04 18:16:23 +05:30
Allen Wittenauer 608223b51b YARN-4850. test-fair-scheduler.xml isn't valid xml (Yufei Gu via aw) 
(cherry picked from commit b1394d6307)
(cherry picked from commit 92a3dbe44f)
 2016-04-01 16:58:00 -07:00
Arun Suresh 82d1f858ee Missing file for YARN-4895. 
(cherry picked from commit 5686caa9fc)
 2016-04-01 15:58:54 -07:00
Arun Suresh 9def4d4d39 YARN-4895. Add subtractFrom method to ResourceUtilization class. Contributed by Konstantinos Karanasos. 
(cherry picked from commit 82621e38a0)
 2016-04-01 15:00:30 -07:00
Wangda Tan 6d454a5d6e YARN-4634. Scheduler UI/Metrics need to consider cases like non-queue label mappings. (Sunil G via wangda) 
(cherry picked from commit 12b11e2e68)
 2016-03-31 14:36:34 -07:00
Sangjin Lee 5e94e539e0 YARN-4183. Clarify the behavior of timeline service config properties (Naganarasimha G R via sjlee) 
(cherry picked from commit 6d67420dbc)
 2016-03-31 10:50:18 -07:00
Xuan d5ea2462c2 YARN-4863. AHS Security login should be in serviceInit() instead of serviceStart(). Contributed by Junping Du 
(cherry picked from commit 80182809ae)
(cherry picked from commit ffe01e05cd)
 2016-03-28 22:21:31 -07:00
Jason Lowe 35f9cfda61 YARN-4773. Log aggregation performs extraneous filesystem operations when rolling log aggregation is disabled. Contributed by Jun Gong 
(cherry picked from commit 948b758070)
 2016-03-28 23:03:16 +00:00
Jian He 7c81e374da YARN-4117. End to end unit test with mini YARN cluster for AMRMProxy Service. Contributed by Giovanni Matteo Fumarola 2016-03-28 13:23:53 -07:00
Jian He 0c84f9aee2 YARN-998. Keep NM resource updated through dynamic resource config for RM/NM restart. Contributed by Junping Du 
(cherry picked from commit c7d843af3b)
 2016-03-28 11:28:59 -07:00
Arun Suresh 1a194f8751 YARN-4823. Refactor the nested reservation id field in listReservation to simple string field. (subru via asuresh) 
(cherry picked from commit 00bebb7e58)
 2016-03-25 15:57:11 -07:00
Jason Lowe e06446be74 YARN-4814. ATS 1.5 timelineclient impl call flush after every event write. Contributed by Xuan Gong 
(cherry picked from commit e8fc81f9c8)
 2016-03-25 20:20:01 +00:00
Arun Suresh 945e3cd55c YARN-4687. Document Reservation ACLs (Contributed by Sean Po) 
(cherry picked from commit 2e1d0ff4e9)
 2016-03-24 10:42:07 -07:00
Arun Suresh 2399e1e46a YARN-4683. Document the List Reservations REST API. (Contributed by Sean Po) 
(cherry picked from commit d820975c1a)
 2016-03-24 10:32:13 -07:00
Arun Suresh 3c2b610b68 YARN-4825. Remove redundant code in ClientRMService::listReservations. (subru via asuresh) 
(cherry picked from commit d82e797b65)
 2016-03-24 10:02:17 -07:00
Junping Du 782db43d18 YARN-4820. ResourceManager web redirects in HA mode drops query parameters. Contributed by Varun Vasudev. 
(cherry picked from commit 19b645c938)
(cherry picked from commit c722262c75)
 2016-03-23 19:39:09 -07:00
Junping Du 762c7d4361 YARN-4814. ATS 1.5 timelineclient impl call flush after every event write. Contributed by Xuan Gong. 
(cherry picked from commit af1d125f9c)
(cherry picked from commit 76602161c0)
 2016-03-23 09:02:56 -07:00
Eric Payne 878e1cfc77 YARN-4686. MiniYARNCluster.start() returns before cluster is completely started. Contributed by Eric Badger. 
(cherry picked from commit 92b7e0d413)
 2016-03-18 17:19:06 +00:00
Junping Du 52ce763a54 YARN-4785. inconsistent value type of the type field for LeafQueueInfo in response of RM REST API. 
(cherry picked from commit ca8106d2dd)
(cherry picked from commit 66257613b4)
 2016-03-17 09:30:43 -07:00
Xuan d21bee170b Revert "YARN-4814. ATS 1.5 timelineclient impl call flush after every event write. (Xuan Gong via gtcarrera9)" 
This reverts commit 3a0b695441.

(cherry picked from commit a888b5bc53)
(cherry picked from commit 398dbde538)
 2016-03-15 22:12:05 -07:00
Li Lu 8640a9c45f YARN-4817. Change Log Level to DEBUG for putDomain call in ATS 1.5. (Xuan Gong via gtcarrera9) 
(cherry-picked from commit 5176a69868)
 2016-03-15 11:49:09 -07:00
Li Lu 0ebc947140 YARN-4814. ATS 1.5 timelineclient impl call flush after every event write. (Xuan Gong via gtcarrera9) 
(cherry picked from commit 3a0b695441)
 2016-03-15 11:43:39 -07:00
Junping Du 4f5b496b8a YARN-4545. Allow YARN distributed shell to use ATS v1.5 APIs. Li Lu via junping_du 
(cherry picked from commit f291d82cd4)
(cherry picked from commit 3d5ac829da)
 2016-03-14 08:34:08 -07:00
Li Lu 92e86ff0cb YARN-4696. Improving EntityGroupFSTimelineStore on exception handling, test setup, and concurrency. 
This commit amends commit d49cfb3504 with a missed test file.

(cherry picked from commit 017d2c127b)
 2016-03-10 13:06:02 -08:00
Li Lu 8e89bb9d38 YARN-4696. Improving EntityGroupFSTimelineStore on exception handling, test setup, and concurrency. (Steve Loughran via gtcarrera9) 
(cherry-picked from commit d49cfb3504)
 2016-03-10 10:58:51 -08:00
Allen Wittenauer 8b4a8ff1b2 Revert "HADOOP-12366. expose calculated paths (aw)" 
This reverts commit 81a7ad45c9.
 2016-03-09 09:59:33 -08:00
Wangda Tan 06088c4c67 YARN-4465. SchedulerUtils#validateRequest for Label check should happen only when nodelabel enabled. (Bibin A Chundatt via wangda) 
(cherry picked from commit 0233d4e0ee)
 2016-03-08 14:31:40 -08:00
Akira Ajisaka 2acd4c3bd2 HADOOP-12860. Expand section "Data Encryption on HTTP" in SecureMode documentation. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit f86850b544)
(cherry picked from commit 0586386472)
 2016-03-08 14:30:01 +09:00
Jason Lowe 0d734cea44 YARN-4760. proxy redirect to history server uses wrong URL. Contributed by Eric Badger 
(cherry picked from commit 4163e36c2b)
 2016-03-07 15:58:01 +00:00
Jason Lowe 53ec7c9243 YARN-4744. Too many signal to container failure in case of LCE. Contributed by Sidharta Seethana 
(cherry picked from commit 059caf9989)
 2016-03-07 15:48:06 +00:00
Zhihai Xu 1a99d5ab97 YARN-4761. NMs reconnecting with changed capabilities can lead to wrong cluster resource calculations on fair scheduler. Contributed by Sangjin Lee 
(cherry picked from commit e1ccc9622b)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestAbstractYarnScheduler.java
 2016-03-06 20:03:55 -08:00
Masatake Iwasaki 9d551a3adf HADOOP-12470. In-page TOC of documentation should be automatically generated by doxia macro (iwasakims) 
(cherry picked from commit cbd31328a6)

 Conflicts:
	hadoop-common-project/hadoop-common/src/site/markdown/CommandsManual.md
	hadoop-common-project/hadoop-common/src/site/markdown/UnixShellGuide.md

(cherry picked from commit 722182fdfb)
 2016-03-04 14:13:30 +09:00
Jian He c50f79d838 YARN-4671. There is no need to acquire CS lock when completing a container. Contributed by Meng Ding 2016-03-01 13:15:37 -08:00
Jian He 475a277e60 YARN-4748. ApplicationHistoryManagerOnTimelineStore should not swallow exceptions on generateApplicationReport. Contributed by Li Lu 
(cherry picked from commit d93c22ec27)
 2016-02-29 18:19:49 -08:00
Haohui Mai 69b195d619 HADOOP-12813. Migrate TestRPC and related codes to rebase on ProtobufRpcEngine. Contributed by Kai Zheng. 2016-02-29 14:15:25 -08:00
Jason Lowe da9f00ac8b YARN-4723. NodesListManager$UnknownNodeId ClassCastException. Contributed by Kuhu Shukla 
(cherry picked from commit 6b0f813e89)
 2016-02-26 20:26:31 +00:00
Ming Ma 6f5ca1b293 YARN-4720. Skip unnecessary NN operations in log aggregation. (Jun Gong via mingma) 
(cherry picked from commit 7f3139e54d)
 2016-02-26 08:49:51 -08:00
Li Lu b22f4db3a0 YARN-4680. TimerTasks leak in ATS V1.5 Writer. (Xuan Gong via 
gtcarrera9)

(cherry picked from commit 9e0f7b8b69)
 2016-02-24 14:08:28 -08:00
Sangjin Lee 6b59e59865 YARN-4722. AsyncDispatcher logs redundant event queue sizes (Jason Lowe via sjlee) 
(cherry picked from commit 553b591ba0)
 2016-02-24 09:32:54 -08:00
Jason Lowe 6f3f5a8c38 YARN-2046. Out of band heartbeats are sent only on container kill and possibly too early. Contributed by Ming Ma 
(cherry picked from commit d284e187b8)
 2016-02-23 20:51:41 +00:00
Junping Du fe43f57c2a YARN-3223. Resource update during NM graceful decommission. Contributed by Brook Zhou. 
(cherry picked from commit 9ed17f181d)
(cherry picked from commit e3ce0ffdc3)
 2016-02-23 03:41:27 -08:00
Varun Vasudev 5f68f640a5 YARN-4709. NMWebServices produces incorrect JSON for containers. Contributed by Varun Saxena. 
(cherry picked from commit 140cb5d745)
(cherry picked from commit 2c218ca8a8)
 2016-02-23 12:33:11 +05:30
Zhe Zhang 6d4a4e785b HADOOP-12668. Support excluding weak Ciphers in HttpServer2 through ssl-server.conf. Contributed by Vijay Singh. 
Change-Id: Ie46a5427d29188935427f67e55203c19fcd83335
 2016-02-22 14:15:11 -08:00
Junping Du 606672a5b1 YARN-4386. refreshNodesGracefully() should send recommission event to active RMNodes only. Contributed by Kuhu Shukla. 
(cherry picked from commit 3fab88540f)
(cherry picked from commit 7402448387)
 2016-02-22 07:39:52 -08:00
Akira Ajisaka 8ddf5822f3 YARN-4707. Remove the extra char (>) from SecureContainer.md. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 5e7d4d51f8)
(cherry picked from commit 34635f9e37)
 2016-02-22 17:18:17 +09:00
Rohith Sharma K S f18d092bda YARN-4654. Yarn node label CLI should parse "=" correctly when trying to remove all labels on a node. (Naganarasimha G R via rohithsharmaks) 
(cherry picked from commit c1afac3a98)
 2016-02-18 14:21:55 +05:30
Sangjin Lee 3a8c7ffeb4 YARN-4690. Skip object allocation in FSAppAttempt#getResourceUsage when possible (Ming Ma via sjlee) 
(cherry picked from commit 7de70680fe)
 2016-02-17 21:03:07 -08:00
Jian He e25d82bbc2 YARN-4653. Document YARN security model from the perspective of Application Developers. Contributed by Steve Loughran 2016-02-14 17:15:18 +08:00
Steve Loughran 706351afb8 YARN-4682. AMRM client to log when AMRM token updated. (Prabhu Joseph via stevel) 2016-02-12 10:58:09 +00:00
Arun Suresh 379c2f4c05 YARN-2575. Create separate ACLs for Reservation create/update/delete/list ops (Sean Po via asuresh) 2016-02-11 15:54:45 -08:00
Jian He fc06ae38ca YARN-4138. Roll back container resource allocation after resource increase token expires. Contributed by Meng Ding 2016-02-11 10:42:45 +08:00
Jian He 145add1aec YARN-4519. Potential deadlock of CapacityScheduler between decrease container and assign containers. Contributed Meng Ding 2016-02-11 10:30:02 +08:00
= 9220bb649b YARN-4420. Add REST API for List Reservations. (Sean Po via curino) 
(cherry picked from commit fc9eb4577c)
 2016-02-10 14:41:38 -08:00
Arun Suresh 685aeeb1a7 YARN-4360. Improve GreedyReservationAgent to support "early" allocations, and performance improvements (curino via asuresh) 
(cherry picked from commit 5cf5c41a89)
 2016-02-10 09:13:31 -08:00
Devaraj K 35651bab99 YARN-4667. RM Admin CLI for refreshNodesResources throws NPE when nothing 
is configured. Contributed by Naganarasimha G R.

(cherry picked from commit 565af873d5)
 2016-02-08 15:04:15 +05:30
Devaraj K aeea77ce14 YARN-4100. Add Documentation for Distributed and Delegated-Centralized 
Node Labels feature. Contributed by Naganarasimha G R.

(cherry picked from commit db144eb1c5)
 2016-02-02 12:08:56 +05:30
Wangda Tan c487453b91 YARN-4340. Add list API to reservation system. (Sean Po via wangda) 
(cherry picked from commit 9875325d5c)
 2016-02-02 10:23:25 +08:00
Jason Lowe 9862879e27 YARN-3102. Decommisioned Nodes not listed in Web UI. Contributed by Kuhu Shukla 
(cherry picked from commit ed55950164)
 2016-02-01 23:17:02 +00:00
Jason Lowe a9916057e3 YARN-4428. Redirect RM page to AHS page when AHS turned on and RM page is not available. Contributed by Chang Li 
(cherry picked from commit 772ea7b41b)
 2016-01-29 21:55:38 +00:00
Jian He e45799a61b YARN-4617. LeafQueue#pendingOrderingPolicy should always use fixed ordering policy instead of using same as active applications ordering policy. Contributed by Rohith Sharma K S 
(cherry picked from commit f4a57d4a53)
 2016-01-29 12:23:02 -08:00
Devaraj K eb2366eda1 YARN-4411. RMAppAttemptImpl#createApplicationAttemptReport throws 
IllegalArgumentException. Contributed by Bibin A Chundatt and yarntime.

(cherry picked from commit a277bdc9ed)
 2016-01-29 13:53:29 +05:30
Rohith Sharma K S d28b5b0c5c YARN-4219. addendum patch to fix javadoc errors 
(cherry picked from commit 09d831c95b)
 2016-01-29 11:53:51 +05:30
Xuan 353518f54c YARN-4219. New levelDB cache storage for timeline v1.5. Contributed by 
Li Lu

(cherry picked from commit 9fab22b366)
(cherry picked from commit 2f00d8d3a7)
 2016-01-28 14:37:27 -08:00
Jason Lowe 290b7db656 YARN-4643. Container recovery is broken with delegating container runtime. Contributed by Sidharta Seethana 
(cherry picked from commit 61382ff8fa)
 2016-01-28 19:01:28 +00:00
Jian He 389a7fb33f YARN-4520. Finished app info is unnecessarily persisted in NM state-store if container is acquired but not lunched on this node. Contributed by sandflee 2016-01-25 15:36:59 -08:00
Jason Lowe 435b6ae82b YARN-4452. NPE when submit Unmanaged application. Contributed by Naganarasimha G R. 
(cherry picked from commit 50bd067e1d)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2016-01-25 16:11:59 +00:00
Jason Lowe a025b7b7ea Revert "YARN-4452. NPE when submit Unmanaged application. Contributed by Naganarasimha G R." 
This reverts commit 92de57bdca.

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2016-01-25 16:03:27 +00:00
Akira Ajisaka a7e84dad41 YARN-4592. Remove unused GetContainerStatus proto. Contributed by Chang Li. 
(cherry picked from commit 56a0c17508)
(cherry picked from commit 02d4a1fce3)
 2016-01-25 15:20:57 +09:00
rohithsharmaks 80928af293 YARN-4614. Fix random failure in TestApplicationPriority#testApplicationPriorityAllocationWithChangeInPriority. (Sunil G via rohithsharmaks) 
(cherry picked from commit 99829eb221)
 2016-01-23 07:59:08 +05:30
Jason Lowe c1953e5e91 YARN-4598. Invalid event: RESOURCE_FAILED at CONTAINER_CLEANEDUP_AFTER_KILL. Contributed by tangshangwen 
(cherry picked from commit 46e5ea81e0)
 2016-01-22 21:56:37 +00:00
Jason Lowe e2762f035f YARN-4371. "yarn application -kill" should take multiple application ids. Contributed by Sunil G 
(cherry picked from commit e91e8b711c)
 2016-01-22 21:37:41 +00:00
Akira Ajisaka ecba5f2ed3 YARN-4605. Spelling mistake in the help message of "yarn applicationattempt" command. Contributed by Weiwei Yang. 
(cherry picked from commit 8f58f742ae)
(cherry picked from commit ddc22bbcb1)
 2016-01-22 19:47:57 +09:00
Jason Lowe d856f82b73 YARN-4610. Reservations continue looking for one app causes other apps to starve. Contributed by Jason Lowe 
(cherry picked from commit 468a53b22f)
 2016-01-21 18:34:03 +00:00
Akira Ajisaka a2d73b185f YARN-4608. Redundant code statement in WritingYarnApplications. Contributed by Kai Sasaki. 
(cherry picked from commit d3224e2d6274daf73a103c0dbe21ba69add7ca26)
(cherry picked from commit 01cd6dce63f2ac21a52f6f2539cb222e895044eb)
 2016-01-22 02:15:16 +09:00
Jason Lowe 15e84fd626 YARN-4492. Add documentation for preemption supported in Capacity scheduler. Contributed by Naganarasimha G R 
(cherry picked from commit f858ac2426)
 2016-01-21 15:40:30 +00:00
Colin Patrick Mccabe de5175d216 HADOOP-12712. Fix some cmake plugin and native build warnings (cmccabe) 
(cherry picked from commit b2c155f810)

 Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
	hadoop-maven-plugins/src/main/java/org/apache/hadoop/maven/plugin/cmakebuilder/CompileMojo.java
	hadoop-maven-plugins/src/main/java/org/apache/hadoop/maven/plugin/cmakebuilder/TestMojo.java
	hadoop-maven-plugins/src/main/java/org/apache/hadoop/maven/plugin/util/Exec.java
 2016-01-21 16:04:05 +05:30
Wangda Tan ada560bc4e YARN-4557. Fix improper Queues sorting in PartitionedQueueComparator when accessible-node-labels=*. (Naganarasimha G R via wangda) 
(cherry picked from commit c9fe0371a8ab40c0d3437fadd32788b320788a7a)
 2016-01-21 11:19:09 +08:00
Wangda Tan 5dc2e78c97 HADOOP-12356. Fix computing CPU usage statistics on Windows. (Inigo Goiri via wangda) 
(cherry picked from commit 89d1fd5dac)

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
 2016-01-19 21:31:35 +08:00
Jian He 914a8fff56 YARN-4565. Fix a bug that leads to AM resource limit not hornored when sizeBasedWeight enabled for FairOrderingPolicy. Contributed by Wangda Tan 2016-01-18 21:05:21 -08:00
Wangda Tan 2a2ea38a9b YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda) 
(cherry picked from commit a44ce3f14f)
 2016-01-19 09:33:52 +08:00
Wangda Tan bacb38aaf3 Revert "YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda)" 
This reverts commit 3fe5728563.

Conflicts:
	hadoop-yarn-project/CHANGES.txt

(cherry picked from commit 150f5ae034)
 2016-01-19 09:33:47 +08:00
Jian He cdef283cd0 YARN-4596. SystemMetricPublisher should not swallow error messages from TimelineClient#putEntities. Contributed by Li Lu 2016-01-18 16:59:24 -08:00
rohithsharmaks c70322d1cf YARN-4265 Addendum patch to fix compilation issue caused by project version number 2016-01-18 16:38:52 +05:30
Wangda Tan 1cb0ad00bb YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda) 
(cherry picked from commit 805a9ed85e)
(cherry picked from commit 6dab78c71a)
 2016-01-18 17:08:42 +08:00
Wangda Tan 2ab34fa89f YARN-4524. Cleanup AppSchedulingInfo. (Karthik Kambatla via wangda) 2016-01-18 17:00:15 +08:00
Wangda Tan ca7ac65453 YARN-4304. AM max resource configuration per partition to be displayed/updated correctly in UI and in various partition related metrics. (Sunil G via wangda) 
(cherry picked from commit b08ecf5c75)
 2016-01-18 11:34:26 +08:00
Wangda Tan 7bb65e3308 YARN-4538. QueueMetrics pending cores and memory metrics wrong. (Bibin A Chundatt via wangda) 
(cherry picked from commit 9523648d57)
 2016-01-18 11:18:07 +08:00
Junping Du a04f29ca90 YARN-4265. Provide new timeline plugin storage to support fine-grained entity caching. Contributed by Li Lu and Jason Lowe 
(cherry picked from commit 02f597c5db)
(cherry picked from commit 4a30a44b11)
 2016-01-17 17:44:17 -08:00
rohithsharmaks 9baeae9368 YARN-4389. Allow application to enable or disable am blacklisting. (Sunil G via rohithsharmaks) 
(cherry picked from commit d3c08cde68)
 2016-01-15 22:48:43 +05:30
Junping Du a03532e04e YARN-4581. AHS writer thread leak makes RM crash while RM is recovering. Contributed by sandflee. 
(cherry picked from commit fc6d3a3b23)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
(cherry picked from commit e09222306f)
 2016-01-15 08:53:48 -08:00
Akira Ajisaka 4417c3b6d2 YARN-4534. Remove the redundant symbol in yarn rmadmin help msg. Contributed by Lin Yiqun. 
(cherry picked from commit 62b0d61513)
(cherry picked from commit ffca0ebfff)
 2016-01-14 14:00:18 +09:00
Jason Lowe a8292161fc YARN-4414. Nodemanager connection errors are retried at multiple levels. Contributed by Chang Li 
(cherry picked from commit 13de8359a1)
 2016-01-12 15:57:44 +00:00
Wangda Tan 00fca06b43 YARN-4582. Label-related invalid resource request exception should be able to properly handled by application. (Bibin A Chundatt via wangda) 
(cherry picked from commit 9e792da014)
 2016-01-12 12:56:12 +08:00
Akira Ajisaka 3fced13eae HADOOP-12584. Disable browsing the static directory in HttpServer2. Contributed by Robert Kanter. 
(cherry picked from commit 9c89bcd04212543ae279d34938ec2ad319e5ba6d)
(cherry picked from commit 6f80be98758ff5d53a192f86ada6cc4062fcdef8)
 2016-01-12 13:35:03 +09:00
Jian He 8172921f6d Missing file for YARN-4580 2016-01-11 17:00:17 -08:00
Jian He e73bf71e10 YARN-4537. Pull out priority comparison from fifocomparator and use compound comparator for FifoOrdering policy. Contributed by Rohith Sharma K S 
(cherry picked from commit b8942be888)
 2016-01-11 16:45:48 -08:00
Jian He 17d29b515f YARN-4479. Change CS LeafQueue pendingOrderingPolicy to hornor recovered apps. Contributed by Rohith Sharma K S 
(cherry picked from commit 109e528ef5)
 2016-01-08 15:52:03 -08:00
Akira Ajisaka 852033ca62 YARN-4544. All the log messages about rolling monitoring interval are shown with WARN level. Contributed by Takashi Ohnishi. 
(cherry picked from commit 52b77577c4)
(cherry picked from commit 3905435d3d)
 2016-01-08 01:13:44 +09:00
Junping Du 49ba43e05b YARN-4546. ResourceManager crash due to scheduling opportunity overflow. Contributed by Jason Lowe. 
(cherry picked from commit c1462a67ff)
(cherry picked from commit 1cc001db4c)
 2016-01-06 05:52:21 -08:00
Junping Du a5eb8bb8f5 YARN-1382. Remove unusableRMNodesConcurrentSet (never used) in NodeListManager to get rid of memory leak. Contributed by Rohith Sharma K S. 
(cherry picked from commit 223ce323bb)
(cherry picked from commit 401d1e6e8a)
 2015-12-30 07:53:55 -08:00
Jian He 9b11e7bd6d YARN-4098. Document ApplicationPriority feature. Contributed by Rohith Sharma K S 2015-12-29 16:52:30 -08:00
Wangda Tan c7fe0bac77 YARN-4315. NaN in Queue percentage for cluster apps page. (Bibin A Chundatt via wangda) 
(cherry picked from commit 561abb9fee)
(cherry picked from commit 3efbde5378)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-12-29 13:31:43 -08:00
Junping Du ce74b17028 YARN-4400. AsyncDispatcher.waitForDrained should be final. Contributed by Daniel Templeton. 
(cherry picked from commit bb5df272b9)
(cherry picked from commit dbc4c79b44)
 2015-12-23 08:56:37 -08:00
Junping Du 630b637ff3 YARN-4234. New put APIs in TimelineClient for ats v1.5. Contributed by Xuan Gong. 
(cherry picked from commit 882f2f0464)
(cherry picked from commit 6e97a3c968)
 2015-12-23 05:31:16 -08:00
Arun Suresh 546178a5de YARN-4477. FairScheduler: Handle condition which can result in an infinite loop in attemptScheduling. (Tao Jie via asuresh) 
(cherry picked from commit e88422df45)
 2015-12-21 22:42:42 -08:00
cnauroth eb5f46e34f YARN-3458. CPU resource monitoring in Windows. Contributed by Inigo Goiri. 
(cherry picked from commit 114b590955)
(cherry picked from commit fdfcffb3d1)
 2015-12-21 16:59:24 -08:00
Wangda Tan e507ed1f90 YARN-4290. Add -showDetails option to YARN Nodes CLI to print all nodes reports information. (Sunil G via wangda) 
(cherry picked from commit a0de702851)
 2015-12-21 11:38:15 -08:00
Wangda Tan 01c813d210 YARN-4454. NM to nodelabel mapping going wrong after RM restart. (Bibin A Chundatt via wangda) 
(cherry picked from commit bc038b382c)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-12-21 11:31:54 -08:00
Uma Mahesh 6f30919336 YARN-4480. Clean up some inappropriate imports. (Kai Zheng via umamahesh) 
(cherry picked from commit 0f82b5d878)
 2015-12-19 23:28:51 -08:00
Wangda Tan b74391ce14 YARN-4416. Deadlock due to synchronised get Methods in AbstractCSQueue. (Naganarasimha G R via wangda) 
(cherry picked from commit 9b856d9787)
(cherry picked from commit 5060fd1840)
 2015-12-18 16:34:45 -08:00
Wangda Tan c4b1386b4e YARN-4225. Add preemption status to yarn queue -status for capacity scheduler. (Eric Payne via wangda) 
(cherry picked from commit 7faa406f27)
(cherry picked from commit 2f5435acbd)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-12-18 16:34:45 -08:00
Wangda Tan 9ed9eb265f YARN-4293. ResourceUtilization should be a part of yarn node CLI. (Sunil G via wangda) 
(cherry picked from commit 79c41b1d83)
(cherry picked from commit 9f4a6ec663)
 2015-12-18 16:34:45 -08:00
Zhihai Xu e4f638d815 YARN-4440. FSAppAttempt#getAllowedLocalityLevelByTime should init the lastScheduler time. Contributed by Lin Yiqun 
(cherry picked from commit 2aaed10327)
(cherry picked from commit be50df687b)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-12-18 16:34:45 -08:00
Wangda Tan 1d1c6c1f8d YARN-4418. AM Resource Limit per partition can be updated to ResourceUsage as well. (Sunil G via wangda) 
(cherry picked from commit 07b0fb996a)
(cherry picked from commit c915a658c8)
 2015-12-18 16:34:44 -08:00
Wangda Tan c7e10d70de YARN-4309. Add container launch related debug information to container logs when a container fails. (Varun Vasudev via wangda) 
(cherry picked from commit dfcbbddb09)
(cherry picked from commit 9f979107c2)
 2015-12-18 16:34:44 -08:00
Wangda Tan 8ee83765f4 YARN-3946. Update exact reason as to why a submitted app is in ACCEPTED state to app's diagnostic message. (Naganarasimha G R via wangda) 
(cherry picked from commit 6cb0af3c39)
(cherry picked from commit a1a723fdff)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-12-18 16:34:44 -08:00
Xuan 41986d5373 YARN-3623-Addendum: Improve the description for Timeline Service Version configuration in yarn-default.xml 
(cherry picked from commit 21daa6c68a)
(cherry picked from commit 455c6f39c5)
 2015-12-18 16:34:43 -08:00
Wangda Tan a4ba6c8525 YARN-4424. Fix deadlock in RMAppImpl. (Jian he via wangda) 
(cherry picked from commit 7e4715186d)

Conflicts:
	hadoop-yarn-project/CHANGES.txt

(cherry picked from commit 7013f9d6cd)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-12-18 16:34:43 -08:00
Jonathan Eagles c4ae007fd4 YARN-4422. Generic AHS sometimes doesn't show started, node, or logs on App page (Eric Payne via jeagles) 
(cherry picked from commit 4ff973f96a)
(cherry picked from commit 7c604c3d06)
 2015-12-18 16:34:43 -08:00
Xuan 46cb47c03e YARN-4392. ApplicationCreatedEvent event time resets after RM 
restart/failover. Contributed by Naganarasimha G R and Xuan Gong

(cherry picked from commit 4546c7582b)
(cherry picked from commit 7decba83ef)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-12-18 16:34:43 -08:00
Jian He 12772f5a45 Add missing file for YARN-4419 
(cherry picked from commit 8a97ca4b00)
 2015-12-18 16:34:42 -08:00
Jian He f355645ca4 YARN-4405. Support node label store in non-appendable file system. Contributed by Wangda Tan 
(cherry picked from commit 755dda8dd8)

(cherry picked from commit be404e1680)
 2015-12-18 16:34:42 -08:00
Wangda Tan 15b3ff1971 YARN-4292. ResourceUtilization should be a part of NodeInfo REST API. (Sunil G via wangda) 
(cherry picked from commit a2c3bfc8c1)
(cherry picked from commit 51e448d295)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-12-18 16:33:53 -08:00
Jian He 9c22910570 YARN-4398. Remove unnecessary synchronization in RMStateStore. Contributed by Ning Ding 
(cherry picked from commit 48cbde9083)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-12-18 16:33:53 -08:00
Junping Du 8812be3dc1 YARN-3226. UI changes for decommissioning node. Contributed by Sunil G. 
(cherry picked from commit 1de56b0448)
(cherry picked from commit 6d2914a697)
 2015-12-17 15:22:37 -08:00
Junping Du 92de57bdca YARN-4452. NPE when submit Unmanaged application. Contributed by Naganarasimha G R. 
(cherry picked from commit 50bd067e1d)
(cherry picked from commit 1a2ef845b5)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/SystemMetricsPublisher.java
(cherry picked from commit 0f9dd48842)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2015-12-17 06:16:08 -08:00
Jason Lowe d939c23cc1 YARN-4461. Redundant nodeLocalityDelay log in LeafQueue. Contributed by Eric Payne 
(cherry picked from commit 91828fef6b)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2015-12-16 23:24:51 +00:00
Siddharth Seth 7415870d13 YARN-4207. Add a non-judgemental YARN app completion status. Contributed by Rich Haase. 
(cherry picked from commit 0f708d465f)
(cherry picked from commit 433212ef87)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-12-16 11:39:37 -08:00
Xuan f22d0046de YARN-4439. Clarify NMContainerStatus#toString method. Contributed by 
Jian He

(cherry picked from commit d8a45425eb)
(cherry picked from commit 992f65dd8b)
 2015-12-14 17:16:07 -08:00
Jian He bc7603e393 YARN-4402. TestNodeManagerShutdown And TestNodeManagerResync fails with bind exception. Contributed by Brahma Reddy Battula 2015-12-14 15:06:13 -08:00
Jian He 16f7ac48d4 YARN-4403. (AM/NM/Container)LivelinessMonitor should use monotonic time when calculating period. Contributed by Junping Du 2015-12-14 13:52:33 -08:00
Arun Suresh c2e788a7b5 YARN-4358 addendum patch to fix javadoc error 
(cherry picked from commit 7fb212e5e6)
 2015-12-12 22:26:35 -08:00
Junping Du 5adf7d851c YARN-3623. Add a config to indicate the Timeline Service version. Contributed by Xuan Gong. 
(cherry picked from commit f910e4f639)
(cherry picked from commit 0b5e76e9ff)
 2015-12-10 02:58:28 -08:00
Akira Ajisaka 59c892683f YARN-4434. NodeManager Disk Checker parameter documentation is not correct. Contributed by Weiwei Yang. 
(cherry picked from commit 16dc65811d4e0e7ab61c61295149848fe53ddd89)
(cherry picked from commit 925bb94d87db7e916ed380d4512225e796f9e03d)
 2015-12-09 16:33:12 +09:00
rohithsharmaks 5bc33c5ae4 YARN-4421. Remove dead code in RmAppImpl.RMAppRecoveredTransition. (Daniel Templeton via rohithsharmaks) 
(cherry picked from commit a5e2e1ecb0)
 2015-12-09 11:35:02 +05:30
rohithsharmaks 6a6bbc51c4 YARN-4431. Not necessary to do unRegisterNM() if NM get stop due to failed to connect to RM. (Junpin Du via rohithsharmaks) 2015-12-09 11:06:31 +05:30
Chris Douglas 3d100163de YARN-4248. Followup patch adding asf-licence exclusions for json test files 
(cherry picked from commit 9f50e13d5d)
(cherry picked from commit bfe4796802)
 2015-12-08 12:08:53 -08:00
= 3205fe3da8 YARN-4248. REST API for submit/update/delete Reservations. (curino) 
(cherry picked from commit c25a635459)
(cherry picked from commit 4ac1564418)
 2015-12-07 13:46:37 -08:00
rohithsharmaks 03e615ba55 YARN-3456. Improve handling of incomplete TimelineEntities. (Varun Saxena via rohithsharmaks) 
(cherry picked from commit 01a641bc44)
 2015-12-07 15:20:50 +05:30
Arun Suresh db69c5f744 YARN-4358. Reservation System: Improve relationship between SharingPolicy and ReservationAgent. (Carlo Curino via asuresh) 
(cherry picked from commit 742632e346)
 2015-12-05 21:35:32 -08:00
Junping Du 6af5255b02 YARN-4408. Fix issue that NodeManager still reports negative running containers. Contributed by Robert Kanter. 
(cherry picked from commit 62e9348bc1)
(cherry picked from commit e76ba91fa5)
 2015-12-03 06:39:07 -08:00
Allen Wittenauer 81a7ad45c9 HADOOP-12366. expose calculated paths (aw) 2015-12-01 08:15:57 -08:00
Tsuyoshi Ozawa d76b523b02 YARN-4380. TestResourceLocalizationService.testDownloadingResourcesOnContainerKill fails intermittently. Contributed by Varun Saxena. 
(cherry picked from commit 0656d2dc83)
 2015-11-26 01:10:41 +09:00
Wangda Tan 7b9f7eaf5c YARN-4384. updateNodeResource CLI should not accept negative values for resource. (Junping Du via wangda) 
(cherry picked from commit 23c625ec57)
 2015-11-24 16:36:30 -08:00
Jason Lowe 3a2f5d6329 YARN-4365. FileSystemNodeLabelStore should check for root dir existence on startup. Contributed by Kuhu Shukla 
(cherry picked from commit f5acf94eca)
 2015-11-24 23:47:54 +00:00
Jason Lowe 9e54433c6c YARN-4132. Separate configs for nodemanager to resourcemanager connection timeout and retries. Contributed by Chang Li 
(cherry picked from commit 4ac6799d4a)
 2015-11-24 22:36:19 +00:00
Tsuyoshi Ozawa 9e458c3e9e YARN-4387. Fix typo in FairScheduler log message. Contributed by Xin Wang. 
(cherry picked from commit 28dfe721b8)
 2015-11-24 22:10:31 +09:00
Akira Ajisaka ac0ddc4fec YARN-4298. Fix findbugs warnings in hadoop-yarn-common. Contributed by Sunil G. 
(cherry picked from commit 8c0133f3d4)
 2015-11-24 18:40:56 +09:00
Karthik Kambatla 49ed955c91 YARN-3980. Plumb resource-utilization info in node heartbeat through to the scheduler. (Inigo Goiri via kasha) 
(cherry picked from commit 52948bb20b)
 2015-11-24 13:52:29 +05:30
Jian He b339a4b8d6 YARN-4349. Support CallerContext in YARN. Contributed by Wangda Tan 
(cherry picked from commit 8676a118a1)
 2015-11-23 17:20:24 -08:00
Jason Lowe 9d303edbfb YARN-4344. NMs reconnecting with changed capabilities can lead to wrong cluster resource calculations. Contributed by Varun Vasudev 
(cherry picked from commit d36b6e045f)
 2015-11-23 20:31:50 +00:00
Arun Suresh cbbdbe3cdd YARN-3454. Add efficient merge operation to RLESparseResourceAllocation (Carlo Curino via asuresh) 
(cherry picked from commit da1016365a)
 2015-11-21 10:01:10 -08:00
Wangda Tan cdca460ad4 YARN-3769. Consider user limit when calculating total pending resource for preemption policy in Capacity Scheduler. (Eric Payne via wangda) 
(cherry picked from commit 2346fa3141)
 2015-11-20 15:56:17 -08:00
Jason Lowe f3bca8bc1a YARN-4374. RM capacity scheduler UI rounds user limit factor. Contributed by Chang Li 
(cherry picked from commit 060cdcbe5d)
 2015-11-20 23:13:27 +00:00
Xuan be65d319fa YARN-2859.addendum: fix the remaining issue from the previous patch 
(cherry picked from commit f114e728da)
 2015-11-19 10:09:48 -08:00
Steve Loughran a0b3e43a6d YARN-4279. Mark ApplicationId and ApplicationAttemptId static methods as @Public, @Unstable. (stevel) 2015-11-18 21:41:24 +00:00
Arun Suresh d8e60080e9 YARN-4184. Remove update reservation state api from state store as its not used by ReservationSystem (Sean Po via asuresh) 
(cherry picked from commit 6a61928fb7)
 2015-11-17 15:51:59 -08:00
Jian He 2cf084ac76 Revert "YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan" 
This reverts commit a4ff03b200.

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-11-16 20:19:31 -08:00
Junping Du 1d107d8058 YARN-4354. Public resource localization fails with NPE. Contributed by Jason Lowe. 
(cherry picked from commit 855d52927b)
 2015-11-15 04:44:59 -08:00
Wangda Tan 875aec3177 YARN-1509. Make AMRMClient support send increase container request and get increased/decreased containers. (Meng Ding via wangda) 
(cherry picked from commit 7ff280fca9)
 2015-11-12 11:50:41 -08:00
Wangda Tan 13f8c7093b YARN-4347. Resource manager fails with Null pointer exception. (Jian He via wangda) 
(cherry picked from commit 7f55a18071)
 2015-11-12 11:24:04 -08:00
Wangda Tan 0c1fb15b29 YARN-4287. Capacity Scheduler: Rack Locality improvement (Nathan Roberts via wangda) 
(cherry picked from commit 796638d9bc)
 2015-11-12 11:12:11 -08:00
Jason Lowe 9050078005 YARN-4345. yarn rmadmin -updateNodeResource doesn't work. Contributed by Junping Du 
(cherry picked from commit 06a70311b9)
 2015-11-12 14:49:55 +00:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) bb5dd7f772 YARN-4183. Reverting the patch to fix behaviour change. 
Revert "YARN-4183. Enabling generic application history forces every job to get a timeline service delegation token (jeagles)"

This reverts commit 42bc525acb.
 2015-11-11 10:42:52 -08:00
Akira Ajisaka f6f128c867 YARN-4241. Fix typo of property name in yarn-default.xml. Contributed by Anthony Rojas. 
(cherry picked from commit 23d0db551c)
 2015-11-11 16:13:50 +09:00
Wangda Tan 8a8ac50a5b YARN-1510. Make NMClient support change container resources. (Meng Ding via wangda) 
(cherry picked from commit c99925d6dd)
 2015-11-10 13:17:23 -08:00
Vinod Kumar Vavilapalli b742ea7fe6 HADOOP-12451. Setting HADOOP_HOME explicitly should be allowed. Fixing bugs in 
the first patch committed. (Karthik Kambatla via vinodkv)
 2015-11-10 12:16:52 -08:00
Jian He a4ff03b200 YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan 
(cherry picked from commit 8fbea531d7)
 2015-11-09 11:05:12 -08:00
Wangda Tan 3190446df9 YARN-4326. Fix TestDistributedShell timeout as AHS in MiniYarnCluster no longer binds to default port 8188. (Meng Ding via wangda) 
(cherry picked from commit 0783184f4b)
 2015-11-03 11:36:11 -08:00
Tsuyoshi Ozawa 1c80473a41 YARN-4320. TestJobHistoryEventHandler fails as AHS in MiniYarnCluster no longer binds to default port 8188. Contributed by Varun Saxena. 
(cherry picked from commit ce31b22739)
 2015-10-30 18:00:35 +09:00
Jian He 2b197bbffd YARN-4127. RM fail with noAuth error if switched from failover to non-failover. Contributed by Varun Saxena 
(cherry picked from commit e5b1733e04)
 2015-10-29 15:43:26 -07:00
Jonathan Eagles 42bc525acb YARN-4183. Enabling generic application history forces every job to get a timeline service delegation token (jeagles) 
(cherry picked from commit c293c58954)
 2015-10-29 16:42:49 -05:00
Arun Suresh 8943abfb08 YARN-4310. FairScheduler: Log skipping reservation messages at DEBUG level (asuresh) 
(cherry picked from commit 58d1df585c)
 2015-10-29 13:43:43 -07:00
Jason Lowe c75d8b164f YARN-2902. Killing a container that is localizing can orphan resources in the DOWNLOADING state. Contributed by Varun Saxena 
(cherry picked from commit e2267de207)
 2015-10-29 16:39:55 +00:00
Jian He 7d109ef5b5 YARN-4288. Fixed RMProxy to retry on IOException from local host. Contributed by Junping Du 
(cherry picked from commit c41699965e)
 2015-10-29 00:01:20 -07:00
Rohith Sharma K S e3fcb711d6 YARN-4130. Duplicate declaration of ApplicationId in RMAppManager#submitApplication method. (Kai Sasaki via rohithsharmaks) 
(cherry picked from commit 656c8f9527)
 2015-10-29 12:25:08 +05:30
Xuan 9ce5069d16 YARN-2859. ApplicationHistoryServer binds to default port 8188 in 
MiniYARNCluster. Contributed by Vinod Kumar Vavilapalli

(cherry picked from commit 27414dac66)
 2015-10-28 10:33:56 -07:00
Tsuyoshi Ozawa d29bff86c4 YARN-4251. TestAMRMClientOnRMRestart#testAMRMClientOnAMRMTokenRollOverOnRMRestart is failing. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 9f4dfdf4eb)
 2015-10-28 17:39:26 +09:00
Tsuyoshi Ozawa 3138b43f16 YARN-3573. MiniMRYarnCluster constructor that starts the timeline server using a boolean should be marked deprecated. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 96677bef00)
 2015-10-27 12:35:20 +09:00
Akira Ajisaka d9156c1245 YARN-4300. [JDK8] Fix javadoc errors caused by wrong tags. (aajisaka) 
(cherry picked from commit 8a68630dd1)
 2015-10-27 10:28:30 +09:00
Wangda Tan 1499af38eb YARN-3216. Max-AM-Resource-Percentage should respect node labels. (Sunil G via wangda) 
(cherry picked from commit 56e4f6237a)
 2015-10-26 16:45:20 -07:00
Wangda Tan 9bf1e74171 YARN-4169. Fix racing condition of TestNodeStatusUpdaterForLabels. (Naganarasimha G R via wangda) 
(cherry picked from commit 6f606214e7)
 2015-10-26 16:38:39 -07:00
Wangda Tan baa2998e93 YARN-4285. Display resource usage as percentage of queue and cluster in the RM UI (Varun Vasudev via wangda) 
(cherry picked from commit 3cc73773eb)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-10-26 13:12:56 -07:00
Jason Lowe 7d66e2e666 YARN-4284. condition for AM blacklisting is too narrow. Contributed by Sangjin Lee 
(cherry picked from commit 33a03af3c3)
 2015-10-26 19:54:16 +00:00
Rohith Sharma K S 33455fa415 YARN-2729. Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup. (Naganarasimha G R via rohithsharmaks) 
(cherry picked from commit 5acdde4744)
 2015-10-26 15:45:22 +05:30
Rohith Sharma K S 06ecc37e82 YARN-4223. Fixed findbugs warnings in hadoop-yarn-server-nodemanager project. (varun saxena via rohithsharmaks) 
(cherry picked from commit 092883b34a)
 2015-10-26 15:26:08 +05:30
Tsuyoshi Ozawa e85ebf0825 YARN-3528. Tests with 12345 as hard-coded port break jenkins. Contributed by Brahma Reddy Battula. 
Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeManagerShutdown.java
 2015-10-26 17:06:46 +09:00
Rohith Sharma K S e75799d5fd YARN-4246. NPE while listing app attempt. (Nijel S F via rohithsharmaks) 
(cherry picked from commit b57f08c0d2)
 2015-10-26 12:07:48 +05:30
Allen Wittenauer 599653ad92 YARN-3724. Use POSIX nftw(3) instead of fts(3) (Alan Burlison via aw) 2015-10-25 21:43:51 -07:00
Arun Suresh f0580dcac0 YARN-3738. Add support for recovery of reserved apps running under dynamic queues (subru via asuresh) 
(cherry picked from commit ab8eb8770c)
 2015-10-24 22:54:45 -07:00
Steve Loughran fc5a433adb YARN-4296. DistributedShell Log.info is not friendly. (Xiaowei Wang via stevel) 2015-10-24 12:54:48 +01:00
Steve Loughran 65b928d8c7 YARN-4289. TestDistributedShell failing with bind exception. (Brahma Reddy Battula via stevel) 2015-10-24 12:48:03 +01:00
Akira Ajisaka 356e922632 YARN-4294. [JDK8] Fix javadoc errors caused by wrong reference and illegal tag. (aajisaka) 
(cherry picked from commit 7781fe1b9e)
 2015-10-24 11:56:37 +09:00
Jason Lowe 235e84bf3d YARN-4041. Slow delegation token renewal can severely prolong RM recovery. Contributed by Sunil G 
(cherry picked from commit d3a34a4f38)
 2015-10-23 20:57:53 +00:00
Jonathan Eagles 6db7bfbcfd YARN-4009. CORS support for ResourceManager REST API. ( Varun Vasudev via jeagles) 
(cherry picked from commit f8adeb712d)
 2015-10-23 10:43:59 -05:00
Ming Ma 4bb7e68eb6 YARN-2913. Fair scheduler should have ability to set MaxResourceDefault for each queue. (Siqi Li via mingma) 
(cherry picked from commit 934d96a334)
 2015-10-23 08:37:46 -07:00
Junping Du d5145b58f6 YARN-4243. Add retry on establishing Zookeeper conenction in EmbeddedElectorService#serviceInit. Contributed by Xuan Gong. 
(cherry picked from commit 0fce5f9a49)
 2015-10-22 13:42:12 -07:00
Zhihai Xu 9fe7ccc61c YARN-4256. YARN fair scheduler vcores with decimal values. Contributed by Jun Gong 
(cherry picked from commit 960201b79b)
 2015-10-22 12:31:28 -07:00
Anubhav Dhoot c44401f362 YARN-3739. Add reservation system recovery to RM recovery process. Contributed by Subru Krishnan. 
(cherry picked from commit 2798723a54)
 2015-10-22 06:52:41 -07:00
Jonathan Eagles cc8093e5ac YARN-2513. Host framework UIs in YARN for use with the ATS (jeagles) 
(cherry picked from commit 25f8f801d1)
 2015-10-21 15:38:48 -05:00
Arun Suresh f2009dc89c YARN-3985. Make ReservationSystem persist state using RMStateStore reservation APIs. (adhoot via asuresh) 
(cherry picked from commit 506d1b1dbc)
 2015-10-20 16:48:58 -07:00
Arun Suresh acc0e718d6 YARN-4270. Limit application resource reservation on nodes for non-node/rack specific requests (asuresh) 
(cherry picked from commit 7e2837f830)
 2015-10-19 20:02:10 -07:00
Varun Vasudev e4429de86f YARN-4267. Add additional logging to container launch implementations in container-executor. Contributed by Sidharta Seethana. 
(cherry picked from commit 151eab2a12)
 2015-10-19 20:56:43 +05:30
Varun Vasudev 196d549e2b YARN-4262. Allow whitelisted users to run privileged docker containers. Contributed by Sidharta Seethana. 
(cherry picked from commit e39ae0e676)
 2015-10-19 20:21:48 +05:30
Steve Loughran 390f29ab99 YARN-4155. TestLogAggregationService.testLogAggregationServiceWithInterval failing. (Bibin A Chundatt via stevel) 2015-10-18 12:03:46 +01:00
Jian He 1ba73169c7 YARN-4170. AM need to be notified with priority in AllocateResponse. Contributed by Sunil G 
(cherry picked from commit f9da5cdb2b)
 2015-10-16 15:26:46 -07:00
Wangda Tan 204ee6a40f YARN-4162. CapacityScheduler: Add resource usage by partition and queue capacity by partition to REST API. (Naganarasimha G R via wangda) 
(cherry picked from commit 4337b263aa)
 2015-10-16 15:10:08 -07:00
Jian He 2862057f11 YARN-4000. RM crashes with NPE if leaf queue becomes parent queue during restart. Contributed by Varun Saxena 
(cherry picked from commit cf23f2c2b5)
 2015-10-15 17:13:09 -07:00
Varun Vasudev 342870b70f YARN-4258. Add support for controlling capabilities for docker containers. Contributed by Sidharta Seethana. 
(cherry picked from commit 63020c54c1)
 2015-10-15 13:41:58 +05:30
Steve Loughran 0e4fb329ed HADOOP-12478. Shell.getWinUtilsPath() has been renamed Shell.getWinutilsPath(). (stevel) 2015-10-14 20:25:33 +01:00
rohithsharmaks 526be695c0 YARN-4250. NPE in AppSchedulingInfo#isRequestLabelChanged. (Brahma Reddy Battula via rohithsharmaks) 
(cherry picked from commit d6c8bad869)
 2015-10-14 16:13:26 +05:30
Varun Vasudev 2dd9c475c4 YARN-4255. container-executor does not clean up docker operation command files. Contributed by Sidharta Seethana. 
(cherry picked from commit da1ee078f9)
 2015-10-14 15:27:18 +05:30
Varun Vasudev 0aeedfd572 YARN-4252. Log container-executor invocation details when exit code is non-zero. Contributed by Sidharta Seethana. 
(cherry picked from commit fb9c1519b3)
 2015-10-14 15:19:19 +05:30
Varun Vasudev a9b03b5afb YARN-4253. Standardize on using PrivilegedOperationExecutor for all invocations of container-executor in LinuxContainerExecutor. Contributed by Sidharta Seethana. 
(cherry picked from commit  8d59293089)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LinuxContainerExecutor.java
 2015-10-14 15:04:22 +05:30
Steve Loughran 5eab51a24f HADOOP-10775. Shell operations to fail with meaningful errors on windows if winutils.exe not found. (stevel) 2015-10-13 21:47:54 +01:00
Varun Vasudev 34b8142802 YARN-4017. container-executor overuses PATH_MAX. Contributed by Sidharta Seethana 
(cherry picked from commit 5b6bae0094)
 2015-10-13 12:50:28 +05:30
Jian He ee1ce393f5 YARN-4230. RM crashes with NPE when increasing container resource if there is no headroom left. Contributed by Meng Ding 
(cherry picked from commit 9849c8b386)
 2015-10-12 11:52:04 -07:00
Zhihai Xu 9988b57e71 YARN-4201. AMBlacklist does not work for minicluster. Contributed by Jun Gong. 
(cherry picked from commit 049c6e8dc0)
 2015-10-12 00:16:40 -07:00
Devaraj K 28edc7b129 YARN-3964. Support NodeLabelsProvider at Resource Manager side. 
Contributed by Dian Fu.

(cherry picked from commit db93047881)
 2015-10-11 11:24:05 +05:30
Wangda Tan 37642abb83 YARN-4140. RM container allocation delayed incase of app submitted to Nodelabel partition. (Bibin A Chundatt via wangda) 
(cherry picked from commit def374e666)
 2015-10-09 16:40:10 -07:00
Karthik Kambatla 2d8f9e3fd9 MAPREDUCE-6302. Incorrect headroom can lead to a deadlock between map and reduce allocations. (kasha) 
(cherry picked from commit 4aa9b3e75c)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java
 2015-10-09 07:54:47 -07:00
Jason Lowe 0139517811 YARN-261. Ability to fail AM attempts. Contributed by Andrey Klochkov and Rohith Sharma K S 
(cherry picked from commit a0bca2b5ad)
 2015-10-09 14:19:41 +00:00
Rohith Sharma K S 21609e8e9a YARN-4235. FairScheduler PrimaryGroup does not handle empty groups returned for a user. (Anubhav Dhoot via rohithsharmaks) 
(cherry picked from commit 8f195387a4)
 2015-10-09 10:11:53 +05:30
Jason Lowe ec74b0cd4e YARN-3943. Use separate threshold configurations for disk-full detection and disk-not-full detection. Contributed by Zhihai Xu 
(cherry picked from commit 8d226225d0)
 2015-10-08 22:26:35 +00:00
Rohith Sharma K S 99126714a3 YARN-4228. FileSystemRMStateStore use IOUtils#close instead of fs#close. (Bibin A Chundatt via rohithsharmaks) 
(cherry picked from commit 3793cbe4c3)
 2015-10-07 10:20:36 +05:30
Rohith Sharma K S 053fcf4c0d YARN-4209. RMStateStore FENCED state doesn’t work due to updateFencedState called by stateMachine.doTransition. (Zhihai Xu via rohithsharmaks) 
(cherry picked from commit 9156fc60c6)
 2015-10-07 09:37:14 +05:30
Wangda Tan 5453a63612 YARN-4215. RMNodeLabels Manager Need to verify and replace node labels for the only modified Node Label Mappings in the request. (Naganarasimha G R via wangda) 
(cherry picked from commit 29a582ada0)
 2015-10-06 11:59:10 -07:00
Wangda Tan d24a3b9a3c YARN-4176. Resync NM nodelabels with RM periodically for distributed nodelabels. (Bibin A Chundatt via wangda) 
(cherry picked from commit 30ac69c6bd)
 2015-10-05 15:48:40 -07:00
Vinod Kumar Vavilapalli b876babdc7 HADOOP-12441. Fixed shell-kill command behaviour to work correctly on some Linux distributions after HADOOP-12317. Contributed by Wangda Tan. 
(cherry picked from commit f1c19b9365)
 2015-10-05 10:57:55 -07:00
Harsh J 21b4ba48ce HADOOP-12458. Retries is typoed to spell Retires in parts of hadoop-yarn and hadoop-common. Contributed by Neelesh Srinivas Salian. 
(cherry picked from commit c918f7be5e)
 2015-10-03 18:39:22 +05:30
Xuan 53bddc410f YARN-1897. CLI and core support for signal container functionality. Contributed by Ming Ma 
(cherry picked from commit 8f08532bde)
 2015-10-02 18:53:04 -07:00
Jason Lowe 3e3733437f YARN-3619. ContainerMetrics unregisters during getMetrics and leads to ConcurrentModificationException. Contributed by Zhihai Xu 
(cherry picked from commit fdf02d1f26)
 2015-10-02 20:21:32 +00:00
Jason Lowe d645ee1d62 YARN-3727. For better error recovery, check if the directory exists before using it for localization. Contributed by Zhihai Xu 
(cherry picked from commit 854d25b0c3)
 2015-09-30 15:01:01 +00:00
Karthik Kambatla 64a7617eb9 YARN-4066. Large number of queues choke fair scheduler. (Johan Gustavsson via kasha) 
(cherry picked from commit a0b5a0a419)
 2015-09-29 08:13:25 -07:00
Anubhav Dhoot 22f2501476 YARN-4180. AMLauncher does not retry on failures when talking to NM. (adhoot) 
(cherry picked from commit 9735afe967)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationMasterLauncher.java
 2015-09-28 16:54:06 -07:00
Jason Lowe 600022ae69 YARN-4141. Runtime Application Priority change should not throw exception for applications at finishing states. Contributed by Sunil G 
(cherry picked from commit 9f53a95ff6)
 2015-09-28 22:56:07 +00:00
Anubhav Dhoot 4d8b99423e YARN-4204. ConcurrentModificationException in FairSchedulerQueueInfo. (adhoot) 
(cherry picked from commit fb2e525c07)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-09-28 09:39:52 -07:00
Xuan bf3f08cc63 YARN-3624. ApplicationHistoryServer should not reverse the order of the 
filters it gets. Contributed by Mit Desai

(cherry picked from commit d1b9b85244)
 2015-09-24 15:03:05 -07:00
Jian He ea2b6ee9e9 YARN-4171. Fix findbugs warnings in YARN-1197 branch. Contributed by Wangda Tan 
(cherry picked from commit b3f6b641dc)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-09-24 10:30:49 -07:00
Jian He d458805690 YARN-1651. CapacityScheduler side changes to support container resize. Contributed by Wangda Tan 
(cherry picked from commit 89cab1ba5f)

Conflicts:
	hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/SLSCapacityScheduler.java
 2015-09-24 10:30:04 -07:00
Jian He d592aa115b YARN-3868. Recovery support for container resizing. Contributed by Meng Ding 
(cherry picked from commit c57eac5dfe)
 2015-09-24 10:24:13 -07:00
Jian He b6c594c6ea YARN-1644. RM-NM protocol changes and NodeStatusUpdater implementation to support container resizing. Contributed by Meng Ding 
(cherry picked from commit c3dc1af072)
 2015-09-24 10:24:04 -07:00
Jian He b8955d81c5 YARN-1643. Make ContainersMonitor support changing monitoring size of an allocated container. Contributed by Meng Ding and Wangda Tan 
(cherry picked from commit c59ae4eeb1)
 2015-09-24 10:23:55 -07:00
Jian He fa0a554ae0 YARN-3867. ContainerImpl changes to support container resizing. Contributed by Meng Ding 
(cherry picked from commit 5f5a968d65)
 2015-09-24 10:23:46 -07:00
Jian He afe4afd0c9 YARN-1645. ContainerManager implementation to support container resizing. Contributed by Meng Ding & Wangda Tan 
(cherry picked from commit ffd820c27a)
 2015-09-24 10:23:35 -07:00
Jian He c19e8ffdc7 YARN-1449. AM-NM protocol changes to support container resizing. Contributed by Meng Ding & Wangda Tan) 
(cherry picked from commit 83a18add10)
 2015-09-24 10:23:25 -07:00
Jian He 89dc37450c YARN-3866. AM-RM protocol changes to support container resizing. Contributed by Meng Ding 
(cherry picked from commit dfe2cb849f)
 2015-09-24 10:23:13 -07:00
Rohith Sharma K S fbb1d923f4 YARN-4044. Running applications information changes such as movequeue is not published to TimeLine server. (Sunil G via rohithsharmaks) 2015-09-24 12:17:55 +05:30
Rohith Sharma K S ace4d26936 YARN-4152. NodeManager crash with NPE when LogAggregationService#stopContainer called for absent container. (Bibin A Chundatt via rohithsharmaks) 2015-09-24 11:31:47 +05:30
Jason Lowe b7111449a1 YARN-3975. WebAppProxyServlet should not redirect to RM page if AHS is enabled. Contributed by Mit Desai 
(cherry picked from commit 692d51c09d)
 2015-09-23 16:39:50 +00:00
Jason Lowe c829be5133 YARN-4095. Avoid sharing AllocatorPerContext object in LocalDirAllocator between ShuffleHandler and LocalDirsHandlerService. Contributed by Zhihai Xu 
(cherry picked from commit c890c51a91)
 2015-09-23 15:44:13 +00:00
Tsuyoshi Ozawa 52cc2ca488 HADOOP-12428. Fix inconsistency between log-level guards and statements. Contributed by Jagadesh Kiran N and Jackie Chang. 
(cherry picked from commit dfd807afab)
 2015-09-22 12:54:44 +09:00
Chris Douglas b39b61e356 YARN-4188. Make MoveApplicationAcrossQueues abstract, newInstance static 
Contributed by Giovanni Matteo Fumarola
(cherry picked from commit 8e01b0d97a)
 2015-09-21 17:54:24 -07:00
Wangda Tan fafcb4ac4e YARN-4113. RM should respect retry-interval when uses RetryPolicies.RETRY_FOREVER. (Sunil G via wangda) 
(cherry picked from commit b00392dd9c)
 2015-09-21 11:15:02 -07:00
Rohith Sharma K S 52f343598e YARN-4167. NPE on RMActiveServices#serviceStop when store is null. (Bibin A Chundatt via rohithsharmaks) 
(cherry picked from commit c9cb6a5960)
 2015-09-21 10:06:39 +05:30
Arun Suresh b70c7b0e51 YARN-3920. FairScheduler container reservation on a node should be configurable to limit it to large containers (adhoot via asuresh) 
(cherry picked from commit 94dec5a916)
 2015-09-18 14:04:03 -07:00
Wangda Tan 4a657e9326 YARN-3212. RMNode State Transition Update with DECOMMISSIONING state. (Junping Du via wangda) 
(cherry picked from commit 9bc913a35c)
 2015-09-18 10:05:04 -07:00
Steve Loughran 22dee29857 YARN-2597 MiniYARNCluster should propagate reason for AHS not starting 2015-09-18 09:45:17 +01:00
Rohith Sharma K S d94c092ad1 YARN-4135. Improve the assertion message in MockRM while failing after waiting for the state.(Nijel S F via rohithsharmaks) 
(cherry picked from commit 723c31d45b)
 2015-09-18 08:46:36 +05:30
Jian He 5d2f85021f YARN-4034. Render cluster Max Priority in scheduler metrics in RM web UI. Contributed by Rohith Sharma K S 
(cherry picked from commit 6c6e734f0b)
 2015-09-17 14:56:32 +08:00
Steve Loughran ab09573fcb YARN-3433. Jersey tests failing with Port in Use -again. (Brahma Reddy Battula) 2015-09-16 10:30:49 +01:00
Jian He 9eda3ce3fa YARN-4078. Add getPendingResourceRequestForAttempt in YarnScheduler interface. Contributed by Naganarasimha G R 
(cherry picked from commit 452079af8b)
 2015-09-16 15:01:23 +08:00
Xuan f3c0a210b6 YARN-4149. yarn logs -am should provide an option to fetch all the log 
files. Contributed by Varun Vasudev

(cherry picked from commit 34ef1a092b)
 2015-09-15 14:37:59 -07:00
Jason Lowe 181bda0421 YARN-4158. Remove duplicate close for LogWriter in AppLogAggregatorImpl#uploadLogsForContainers. Contributed by Zhihai Xu 
(cherry picked from commit 8c1cdb17a0)
 2015-09-15 20:22:23 +00:00
Wangda Tan c31fe79979 YARN-3717. Expose app/am/queue's node-label-expression to RM web UI / CLI / REST-API. (Naganarasimha G R via wangda) 
(cherry picked from commit ae5308fe1d)
 2015-09-15 11:41:57 -07:00
Junping Du df714e25aa YARN-313. Add Admin API for supporting node resource configuration in command line. (Contributed by Inigo Goiri, Kenji Kikushima and Junping Du) 
(cherry picked from commit 73e3a49eb0)
 2015-09-15 07:58:47 -07:00
Jian He eacc18677a YARN-3635. Refactored current queue mapping implementation in CapacityScheduler to use a generic PlacementManager framework. Contributed by Wangda Tan 
(cherry picked from commit 5468baa80a)
 2015-09-15 19:02:17 +08:00
Wangda Tan e552775545 YARN-4151. Fix findbugs errors in hadoop-yarn-server-common module. (Meng Ding via wangda) 
(cherry picked from commit e2a0270217)
 2015-09-14 16:04:08 -07:00
Jian He b705d9d0ac YARN-4126. RM should not issue delegation tokens in unsecure mode. Contributed by Bibin A Chundatt 
(cherry picked from commit e1b1d7e4ae)
 2015-09-14 14:15:04 +08:00
Karthik Kambatla 1f9f219062 YARN-3697. FairScheduler: ContinuousSchedulingThread can fail to shutdown. (Zhihai Xu via kasha) 
(cherry picked from commit 332b520a48)
 2015-09-13 18:08:03 -07:00
Karthik Kambatla 11e2fa151c YARN-2005. Blacklisting support for scheduling AMs. (Anubhav Dhoot via kasha) 
(cherry picked from commit 81df7b586a)
 2015-09-13 17:06:04 -07:00
Steve Loughran 988749e6ea HADOOP-12087. [JDK8] Fix javadoc errors caused by incorrect or illegal tags. (Akira AJISAKA via stevel). 2015-09-13 14:25:06 +01:00
Robert Kanter 43f051a09a YARN-4145. Make RMHATestBase abstract so its not run when running all tests under that namespace (adhoot via rkanter) 
(cherry picked from commit ea4bb2749f)
 2015-09-11 11:46:35 -07:00
Robert Kanter d07eef8c87 YARN-4115. Reduce loglevel of ContainerManagementProtocolProxy to Debug (adhoot via rkanter) 
(cherry picked from commit b84fb41bb6)
 2015-09-11 11:46:35 -07:00
Varun Vasudev a4288f1262 YARN-4136. LinuxContainerExecutor loses info when forwarding ResourceHandlerException. Contributed by Bibin A Chundatt. 
(cherry picked from commit 486d5cb803)
 2015-09-11 14:39:01 +05:30
Wangda Tan 5decd8f9f7 YARN-4106. NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM. (Bibin A Chundatt via wangda) 
(cherry picked from commit 77666105b4)
 2015-09-10 09:30:34 -07:00
Robert Kanter ea64a9b277 YARN-4086. Allow Aggregated Log readers to handle HAR files (rkanter) 
(cherry picked from commit 6dd6ca442a)
 2015-09-09 18:03:47 -07:00
Zhihai Xu 214338bd68 YARN-4096. App local logs are leaked if log aggregation fails to initialize for the app. Contributed by Jason Lowe. 
(cherry picked from commit 16b9037dc1)
 2015-09-08 12:33:29 -07:00
Varun Vasudev 1c53cd2b0d YARN-4121. Fix typos in capacity scheduler documentation. Contributed by Kai Sasaki. 
(cherry picked from commit 435f935ba7)
 2015-09-08 12:39:39 +05:30
Jian He f23be93dd1 YARN-2884. Added a proxy service in NM to proxy the the communication between AM and RM. Contributed by Kishore Chaliparambil 
(cherry picked from commit 6f72f1e600)
 2015-09-08 11:24:07 +08:00
Xuan a0b7ef15d0 YARN-4087. Followup fixes after YARN-2019 regarding RM behavior when 
state-store error occurs. Contributed by Jian He

(cherry picked from commit 9b78e6e33d)
 2015-09-07 17:46:48 -07:00
Varun Vasudev 70575286b7 YARN-3591. Resource localization on a bad disk causes subsequent containers failure. Contributed by Lavkesh Lahngir. 
(cherry picked from commit 1dbd8e34a7)
 2015-09-07 11:34:14 +05:30
Wangda Tan 9f97b86816 YARN-4024. YARN RM should avoid unnecessary resolving IP when NMs doing heartbeat. (Hong Zhiguo via wangda) 
(cherry picked from commit bcc85e3bab)
 2015-09-04 15:14:34 -07:00
Jason Lowe 6a5220cab3 YARN-4105. Capacity Scheduler headroom for DRF is wrong. Contributed by Chang Li 
(cherry picked from commit 6eaca2e363)
 2015-09-04 15:31:37 +00:00
Varun Vasudev 64c2e8011e YARN-4103. RM WebServices missing scheme for appattempts logLinks. Contributed by Jonathan Eagles. 
(cherry picked from commit 1b2e2ca9104dfc05e9aa7f93954ae2e00b3e62c4)
 2015-09-04 14:32:07 +05:30
Varun Vasudev a53054c853 YARN-3970. Add REST api support for Application Priority. Contributed by Naganarasimha G R. 
(cherry picked from commit 223a306da925ad031bc398f2ea5e56f0e6521edf)
 2015-09-03 16:40:42 +05:30
Jian He cd82fa2f83 YARN-4101. RM should print alert messages if Zookeeper and Resourcemanager gets connection issue. Contributed by Xuan Gong 
(cherry picked from commit 09c64ba1ba)
 2015-09-02 17:46:04 -07:00
Rohith Sharma K S 6e41aa64a5 YARN-3893. Both RM in active state when Admin#transitionToActive failure from refeshAll() (Bibin A Chundatt via rohithsharmaks) 
(cherry picked from commit 7d6687fe76)
 2015-09-02 15:27:33 +05:30
Rohith Sharma K S 7ecdf51441 YARN-4073. Removed unused ApplicationACLsManager in ContainerManagerImpl constructor. (Naganarasimha G R via rohithsharmaks) 
(cherry picked from commit 095ab9ab5f)
 2015-09-02 14:28:50 +05:30
Andrew Wang 7d833a3058 HADOOP-12369. Point hadoop-project/pom.xml java.security.krb5.conf within target folder. 
(cherry picked from commit 0eb9b19325)
 2015-09-01 10:59:39 -07:00
Tsuyoshi Ozawa d3d65f62b3 YARN-2801. Add documentation for node labels feature. Contributed by Wangda Tan and Naganarasimha G R. 
(cherry picked from commit faa38e1aa4)
 2015-09-01 17:55:53 +09:00
Varun Vasudev 855e0f8b00 YARN-4082. Container shouldn't be killed when node's label updated. Contributed by Wangda Tan. 
(cherry picked from commit bf669b6d9f)
 2015-09-01 14:20:54 +05:30
Jian He 48f5161cd5 YARN-4092. Fixed UI redirection to print useful messages when both RMs are in standby mode. Contributed by Xuan Gong 
(cherry picked from commit a3fd2ccc86)
 2015-08-31 17:34:49 -07:00
Junping Du 792b9c0eaf YARN-1556. NPE getting application report with a null appId. Contributed by Weiwei Yang. 
(cherry picked from commit beb65c9465)
 2015-08-28 05:58:45 -07:00
Jian He 306c535395 YARN-3250. Support admin cli interface in for Application Priority. Contributed by Rohith Sharma K S 
(cherry picked from commit a9c8ea71aa)
 2015-08-27 13:27:34 -07:00
Andrew Wang 7a668981f5 HADOOP-12362. Set hadoop.tmp.dir and hadoop.log.dir in pom. Contributed by Charlie Helin. 
(cherry picked from commit 90fe7bcc3b)
 2015-08-27 10:17:04 -07:00
Jian He 387076894f YARN-4014. Support user cli interface in for Application Priority. Contributed by Rohith Sharma K S 
(cherry picked from commit 57c7ae1aff)
 2015-08-24 20:37:18 -07:00
Rohith Sharma K S 6dc732f2f7 YARN-3896. RMNode transitioned from RUNNING to REBOOTED because its response id has not been reset synchronously. (Jun Gong via rohithsharmaks) 
(cherry picked from commit feaf034994)
 2015-08-24 11:30:46 +05:30
Xuan 85363ea4bd YARN-221. Addendum patch to compilation issue which is caused by missing 
AllContainerLogAggregationPolicy. Contributed by Xuan Gong

(cherry picked from commit b71c6006f5)
 2015-08-23 16:50:02 -07:00
Xuan 205b98c059 YARN-221. NM should provide a way for AM to tell it not to aggregate 
logs. Contributed by Ming Ma

(cherry picked from commit 37e1c3d82a)
 2015-08-22 16:26:39 -07:00
Rohith Sharma K S 27ced8253b YARN-3986. getTransferredContainers in AbstractYarnScheduler should be present in YarnScheduler interface 
(cherry picked from commit 22de7c1dca)
 2015-08-21 10:56:45 +05:30
Vinod Kumar Vavilapalli 9da81a3955 MAPREDUCE-6454. Fixed MapReduce to modify HADOOP_CLASSPATH to have distributed cache files so that child processes running hadoop scripts can access these files. Contributed by Junping Du. 2015-08-20 17:35:24 -07:00
Wangda Tan 6452b31760 YARN-2923. Support configuration based NodeLabelsProvider Service in Distributed Node Label Configuration Setup. (Naganarasimha G R) 
(cherry picked from commit fc07464d1a)
 2015-08-20 16:52:57 -07:00
Karthik Kambatla b567aa2b4f YARN-4055. Report node resource utilization in heartbeat. (Inigo Goiri via kasha) 
(cherry picked from commit 13604bd5f1)
 2015-08-20 16:52:33 -07:00
Xuan fc6b176e70 YARN-4028. AppBlock page key update and diagnostics value null on 
recovery. Contributed by Bibin A Chundatt

(cherry picked from commit 22dc5fc209)
 2015-08-18 22:54:01 -07:00
Zhihai Xu 051a622b57 YARN-4057. If ContainersMonitor is not enabled, only print related log info one time. Contributed by Jun Gong. 
(cherry picked from commit 14215c8ef8)
 2015-08-18 11:47:41 -07:00
Zhihai Xu 8f2eb99d71 YARN-3857: Memory leak in ResourceManager with SIMPLE mode. Contributed by mujunchao. 
(cherry picked from commit 3a76a010b8)
 2015-08-18 10:53:16 -07:00
Karthik Kambatla d2ff763533 YARN-3534. Collect memory/cpu usage on the node. (Inigo Goiri via kasha) 
(cherry picked from commit def12933b3)
 2015-08-16 06:26:21 -07:00
Jian He b2e869f2a8 YARN-3987. Send AM container completed msg to NM once AM finishes. Contributed by sandflee 
(cherry picked from commit 0a030546e2)
 2015-08-13 16:23:30 -07:00
Jian He 703fa1b141 YARN-4047. ClientRMService getApplications has high scheduler lock contention. Contributed by Jason Lowe 
(cherry picked from commit 7a445fcfab)
 2015-08-13 16:13:28 -07:00
Jian He 4a5fdb7c23 YARN-4005. Completed container whose app is finished is possibly not removed from NMStateStore. Contributed by Jun Gong 
(cherry picked from commit 38aed1a94e)
 2015-08-13 14:46:34 -07:00
Robert Kanter fbcfca6f91 YARN-4031. Add JvmPauseMonitor to ApplicationHistoryServer and WebAppProxyServer (djp via rkanter) 
(cherry picked from commit dc2340c60e)
 2015-08-12 15:48:14 -07:00
Jian He 65d22b3686 YARN-4026. Refactored ContainerAllocator to accept a list of priorites rather than a single priority. Contributed by Wangda Tan 
(cherry picked from commit e5003be907)
 2015-08-12 15:09:46 -07:00
rohithsharmaks c5b20e0db6 YARN-4023. Publish Application Priority to TimelineServer. (Sunil G via rohithsharmaks) 
(cherry picked from commit 1c12adb71f)
 2015-08-12 14:47:23 +05:30
Xuan 2ebdf5bfce YARN-3999. RM hangs on draing events. Contributed by Jian He 
(cherry picked from commit 3ae716fa69)
 2015-08-11 18:26:00 -07:00
Jian He df97e3265b YARN-3887. Support changing Application priority during runtime. Contributed by Sunil G 
(cherry picked from commit fa1d84ae27)
 2015-08-10 20:52:18 -07:00
Wangda Tan 380d401da5 YARN-3873. PendingApplications in LeafQueue should also use OrderingPolicy. (Sunil G via wangda) 
(cherry picked from commit cf9d3c9256)
 2015-08-10 14:58:19 -07:00
Wangda Tan cdb61b5fb2 YARN-3966. Fix excessive loggings in CapacityScheduler. (Jian He via wangda) 
(cherry picked from commit 4bc42d76e7)
 2015-08-07 09:53:42 -07:00