Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,239 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

2758 Commits

Author SHA1 Message Date
Naganarasimha 5f386967a1 YARN-5720. Update document for "rmadmin -replaceLabelOnNode". Contributred by Tao Jie 
(cherry picked from commit 0e75496049)
 2016-11-08 10:37:34 +05:30
Varun Vasudev 3603e525b1 YARN-4884. Fix missing documentation about rmadmin command regarding node labels. Contributed by Kai Sasaki. 
(cherry picked from commit f1b8f6b2c1)
 2016-11-08 10:22:59 +05:30
Sunil ee5dd2c388 YARN-5802. updateApplicationPriority api in scheduler should ensure to re-insert app to correct ordering policy. Contributed by Bibin A Chundatt 2016-11-05 13:06:31 +05:30
Jason Lowe 4cb4e4c0a1 YARN-5837. NPE when getting node status of a decommissioned node after an RM restart. Contributed by Robert Kanter 
(cherry picked from commit 6bb741ff0e)
 2016-11-04 22:25:35 +00:00
Rohith Sharma K S c89d462761 YARN-5608. TestAMRMClient.setup() fails with ArrayOutOfBoundsException. Contributed by Daniel Templeton. 
(cherry picked from commit f9557127b3)
 2016-11-04 13:52:11 +00:00
Naganarasimha 605e9bf3bf YARN-5697. Use CliParser to parse options in RMAdminCLI. Contributed by Tao Jie. 2016-11-03 10:31:30 +05:30
Varun Saxena 8602e2cb0f YARN-5815. Random failure of TestApplicationPriority.testOrderOfActivatingThePriorityApplicationOnRMRestart (Bibin A Chundatt via Varun Saxena) 2016-11-03 00:43:28 +05:30
Jason Lowe 4a70bd86da YARN-5001. Aggregated Logs root directory is created with wrong group if nonexistent. Contributed by Haibo Chen 
(cherry picked from commit 76893a4100)
 2016-11-01 20:24:44 +00:00
Naganarasimha b8ae7a8ee4 YARN-4498. Application level node labels stats to be available in REST. Contributed by Bibin A Chundatt. 
(cherry picked from commit e4023f0d81)
 2016-10-31 19:09:53 +05:30
Varun Saxena 367cb029d4 YARN-5773. RM recovery too slow due to LeafQueue#activateApplications (Bibin A Chundatt via Varun Saxena) 2016-10-30 11:50:54 +05:30
Jason Lowe 93165c1ade YARN-4963. capacity scheduler: Make number of OFF_SWITCH assignments per heartbeat configurable. Contributed by Nathan Roberts 2016-10-28 17:44:56 +00:00
Jason Lowe 9cc7ab4e96 YARN-5767. Fix the order that resources are cleaned up from the local Public/Private caches. Contributed by Chris Trezzo 
(cherry picked from commit 1b79c417dc)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ResourceLocalizationService.java
 2016-10-28 16:11:59 +00:00
Jason Lowe 83bb428e6c YARN-5027. NM should clean up app log dirs after NM restart. Contributed by sandflee 
(cherry picked from commit 7146359bfd)
 2016-10-28 15:50:22 +00:00
Jason Lowe 558e53b10b HADOOP-13770. Shell.checkIsBashSupported swallowed an interrupted exception. Contributed by Wei-Chiu Chuang 
(cherry picked from commit c017171da0)
 2016-10-28 15:06:57 +00:00
Carlo Curino b32f14d806 YARN-3432. Cluster metrics have wrong Total Memory when there is reserved memory on CS. (Brahma Reddy Battula via curino) 
(cherry picked from commit 892a8348fc)
(cherry picked from commit 58ac40b552)
 2016-10-27 15:23:04 -07:00
Subru Krishnan 55ba220721 YARN-3568. TestAMRMTokens should use some random port. (Takashi Ohnishi via Subru). 
(cherry picked from commit 79ae78dcbe)
(cherry picked from commit 4274600b95)
 2016-10-27 15:19:01 -07:00
Jason Lowe d21dc8aa6c YARN-4831. Recovered containers will be killed after NM stateful restart. Contributed by Siqi Li 
(cherry picked from commit 7e3c327d31)
 2016-10-27 20:45:18 +00:00
Naganarasimha 3b9df97158 YARN-3848. TestNodeLabelContainerAllocation is timing out. Contributed by Varun Saxena 
(cherry picked from commit 6c8830992c)
 2016-10-27 17:16:34 +05:30
Jian He dab611e605 Revert YARN-4126. RM should not issue delegation tokens in unsecure mode. 
(cherry picked from commit bb8214cab8)
 2016-10-23 16:56:12 -07:00
Rohith Sharma K S 34023ca418 YARN-4763. RMApps Page crashes with NPE. (Bibin A Chundatt via rohithsharmaks) 
(cherry picked from commit 19ee185907)
 2016-10-19 17:40:42 +00:00
Varun Saxena 28d75482ff YARN-5599. Publish AM launch command to ATS (Rohith Sharma K S via Varun Saxena) 2016-10-14 23:36:39 +05:30
Varun Vasudev d1841b4750 YARN-5704. Provide config knobs to control enabling/disabling new/work in progress features in container-executor. Contributed by Sidharta Seethana. 2016-10-09 13:22:01 +05:30
Varun Vasudev 50409404bf YARN-4245. Generalize config file handling in container-executor. Contributed by Sidharta Seethana. 
(cherry picked from commit 8ed2e060e8)
(cherry picked from commit 78919f8c34)
 2016-10-09 11:53:03 +05:30
Junping Du 2ff6ffe0ef YARN-5659. getPathFromYarnURL should use standard methods. Contributed by Sergey Shelukhin. 
(cherry picked from commit 459a4833a9)
(cherry picked from commit 8bee319130)
 2016-10-07 07:51:01 -07:00
Rohith Sharma K S 319b101b7e YARN-5101. YARN_APPLICATION_UPDATED event is parsed in ApplicationHistoryManagerOnTimelineStore#convertToApplicationReport with reversed order. Contributed by Sunil G. 
(cherry picked from commit 4d2f380d78)
 2016-10-06 20:44:39 +05:30
Varun Saxena 3656b84085 YARN-5491. Fix random failure of TestCapacityScheduler#testCSQueueBlocked (Bibin A Chundatt via Varun Saxena) 
(cherry picked from commit d677b68c25)
 2016-10-04 20:57:01 +00:00
Naganarasimha 90443140a5 YARN-4855. Should check if node exists when replace nodelabels. Contributeed by Tao Jie 2016-10-03 13:29:18 -04:00
Jason Lowe 515727f46f YARN-4543. Fix random test failure in TestNodeStatusUpdater.testStopReentrant. (Akihiro Suda via rohithsharmaks) 
(cherry picked from commit ac68666803)
 2016-10-03 16:06:17 +00:00
Wangda Tan 69f91d8c48 HADOOP-13544. JDiff reports unncessarily show unannotated APIs and cause confusion while our javadocs only show annotated and public APIs. (vinodkv via wangda) 
(cherry picked from commit 875062b5bc)
(cherry picked from commit 269401dc83)
 2016-09-27 11:46:49 -07:00
Rohith Sharma K S 4f6facb90d YARN-5631. Missing refreshClusterMaxPriority usage in rmadmin help message. Contributed by Kai Sasaki. 2016-09-27 23:07:43 +05:30
Naganarasimha 63ca9f5441 YARN-5660. Wrong audit constants are used in Get/Put of priority in RMWebService. Contributed by Rohith Sharma K S 
(cherry picked from commit ebf528c443)
 2016-09-27 16:46:52 +05:30
Akira Ajisaka ece3ca0cba YARN-5663. Small refactor in ZKRMStateStore. Contributed by Oleksii Dymytrov. 
(cherry picked from commit 14a696f369)
(cherry picked from commit 74f2df16a9)
 2016-09-26 15:01:39 +09:00
Naganarasimha 4c2b20ca3e YARN-3877. YarnClientImpl.submitApplication swallows exceptions. Contributed by Varun Saxena 
(cherry picked from commit e4e72db5f9)
 2016-09-25 17:42:08 +05:30
Naganarasimha 41169b7c2b YARN-5664. Fix Yarn documentation to link to correct versions. Contributed by Xiao Chen 
(cherry picked from commit 4d16d2d72f)
 2016-09-24 23:24:56 +05:30
Naganarasimha fa2025316d YARN-3692. Allow REST API to set a user generated message when killing an application. Contributed by Rohith Sharma K S 2016-09-24 22:42:28 +05:30
Kihwal Lee 9269225522 YARN-5622. TestYarnCLI.testGetContainers fails due to mismatched date formats. Contributed by Eric Badger. 
(cherry picked from commit 6e849cb658)
 2016-09-23 14:15:31 -05:00
Varun Saxena 42f6f68c78 YARN-5539. TimelineClient failed to retry on java.net.SocketTimeoutException: Read timed out (Junping Du via Varun Saxena) 2016-09-23 13:34:18 +05:30
Wangda Tan 6f6e62ad86 YARN-4591. YARN Web UIs should provide a robots.txt. (Sidharta Seethana via wangda) 
(cherry picked from commit 5a58bfee30)
 2016-09-20 17:29:27 -07:00
Jason Lowe 4782390eb1 YARN-5655. TestContainerManagerSecurity#testNMTokens is asserting. Contributed by Robert Kanter 
(cherry picked from commit c6d1d742e7)
 2016-09-20 14:17:26 +00:00
Jason Lowe 564d9e6101 YARN-5540. Scheduler spends too much time looking at empty priorities. Contributed by Jason Lowe 2016-09-19 20:34:46 +00:00
Naganarasimha a1cc90bca1 YARN-4232. TopCLI console support for HA mode. Contributed by Bibin A Chundatt 
(cherry picked from commit ade7c2bc9c)
 2016-09-17 10:01:11 +05:30
Jason Lowe 1ac8206a85 YARN-5630. NM fails to start after downgrade from 2.8 to 2.7. Contributed by Jason Lowe 
(cherry picked from commit e793309735)
 2016-09-13 14:43:55 +00:00
Varun Vasudev d5ea508ca2 YARN-5555. Scheduler UI: "% of Queue" is inaccurate if leaf queue is hierarchically nested. Contributed by Eric Payne. 2016-09-10 18:28:38 +00:00
Jason Lowe 1742022390 YARN-5632. UPDATE_EXECUTION_TYPE causes UpdateContainerRequestPBImpl to throw. Contributed by Jason Lowe 2016-09-09 22:40:17 +00:00
Karthik Kambatla 7a27b2a82f YARN-5566. Client-side NM graceful decom is not triggered when jobs finish. Addendum to fix test flakiness. (Robert Kanter via kasha) 2016-09-08 21:11:20 -07:00
Karthik Kambatla f9016dfec3 YARN-5566. Client-side NM graceful decom is not triggered when jobs finish. (Robert Kanter via kasha) 2016-09-06 16:23:06 -07:00
Ray Chiang 02272a6a77 YARN-5549. AMLauncher#createAMContainerLaunchContext() should not log the command to be launched indiscriminately. (Daniel Templeton via rchiang) 
(cherry picked from commit 4ee1729cb0)
 2016-09-02 15:00:24 -07:00
Arun Suresh 979b29a03c YARN-5221. Expose UpdateResourceRequest API to allow AM to request for change in container properties. (asuresh) 
(cherry picked from commit d6d9cff21b)
(cherry picked from commit b279f42d79)
 2016-08-31 20:06:49 -07:00
Zhe Zhang 765139e4da YARN-5550. TestYarnCLI#testGetContainers should format according to CONTAINER_PATTERN. Contributed by Jonathan Hung. 
(cherry picked from commit ed6ff5cd78)
(cherry picked from commit 3ad4f0823c)
 2016-08-29 14:51:40 -07:00
Jason Lowe 09e585e30a YARN-5560. Clean up bad exception catching practices in TestYarnClient. Contributed by Sean Po 
(cherry picked from commit 4cbe61407d)
 2016-08-29 16:20:04 +00:00
Naganarasimha b952ce854b YARN-3940. Application moveToQueue should check NodeLabel permission. Contributed by Bibin A Chundatt 
(cherry picked from commit 46e02ab719)
 2016-08-26 20:24:37 +05:30
Jason Lowe 8657dece8b YARN-5389. TestYarnClient#testReservationDelete fails. Contributed by Sean Po 
(cherry picked from commit 4037b22b7c)
 2016-08-25 15:11:47 +00:00
Rohith Sharma K S 71a3b01d51 YARN-5544. TestNodeBlacklistingOnAMFailures fails on trunk. Contributed by Sunil G. 
(cherry picked from commit 0d5997d2b9)
 2016-08-23 14:40:05 +05:30
Wangda Tan 2e9d84d24c YARN-4685. Disable AM blacklisting by default to mitigate situations that application get hanged. (Rohith Sharma K S via wangda) 
(cherry picked from commit 2da32a6ef9)
(cherry picked from commit 3e511f39c2)
 2016-08-19 16:55:40 -07:00
Wangda Tan 9bf420b3bc YARN-4837. User facing aspects of 'AM blacklisting' feature need fixing. (vinodkv via wangda) 2016-08-19 16:55:34 -07:00
Wangda Tan 3bf2e16f76 YARN-3388. Allocation in LeafQueue could get stuck because DRF calculator isn't well supported when computing user-limit. (Nathan Roberts via wangda) 
(cherry picked from commit 444b2ea7af)
(cherry picked from commit c7d782d2f6)
 2016-08-19 16:54:03 -07:00
Varun Vasudev 5b49bcaee3 YARN-4307. Display blacklisted nodes for AM container in the RM web UI. Contributed by Naganarasimha G R. 
(cherry picked from commit 308d63f382)
(cherry picked from commit 99966b1912)
 2016-08-18 12:33:52 -07:00
Karthik Kambatla 42a88f9c58 YARN-3446. FairScheduler headroom calculation should exclude nodes in the blacklist. (Zhihai Xu via kasha) 
(cherry picked from commit 9d04f26d4c)
(cherry picked from commit f0923819c3)
 2016-08-18 12:30:39 -07:00
Jason Lowe 74831973cb YARN-4566. Fix test failure in TestMiniYarnClusterNodeUtilization. (Takashi Ohnishi via rohithsharmaks) 
(cherry picked from commit e0b14f26f5)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2016-08-15 19:45:45 +00:00
Junping Du 1a38dd9cee YARN-4916. TestNMProxy.tesNMProxyRPCRetry fails. Contributed by Tibor Kiss. 
(cherry picked from commit 0005816743)
 2016-08-15 19:18:36 +00:00
Varun Saxena ca139a3f87 YARN-5476. Non existent application reported as ACCEPTED by YarnClientImpl (Junping Du via Varun Saxena) 2016-08-13 00:27:17 +05:30
Rohith Sharma K S f7cc0af611 YARN-5492. TestSubmitApplicationWithRMHA is failing sporadically during precommit builds. Contributed by Vrushali C. 
(cherry picked from commit 5199db387d)
 2016-08-11 11:52:44 +05:30
Jason Lowe 69217ba813 YARN-5382. RM does not audit log kill request for active applications. Contributed by Vrushali C 
(cherry picked from commit 5c95bb315b)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAuditLogger.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMAuditLogger.java
 2016-08-10 19:15:12 +00:00
Jason Lowe e1845faeda YARN-5483. Optimize RMAppAttempt#pullJustFinishedContainers. Contributed by sandflee 
(cherry picked from commit 3d401206cf)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/RMAppAttemptImpl.java
 2016-08-10 18:09:11 +00:00
Rohith Sharma K S 3b4d6ece1a YARN-5262. Optimize sending RMNodeFinishedContainersPulledByAMEvent for every AM heartbeat. Contributed by Rohith Sharma K S 
(cherry picked from commit 26b5e6116f)
 2016-08-09 22:19:17 +00:00
Naganarasimha 16c8fd9dca YARN-5287. LinuxContainerExecutor fails to set proper permission. Contributed by Ying Zhang. 2016-08-09 17:34:48 +05:30
Naganarasimha e6a994badf YARN-4624. NPE in PartitionQueueCapacitiesInfo while accessing Schduler UI. Contributed by Brahma Reddy Battula 
(cherry picked from commit d81b8163b4)
(cherry picked from commit 991823482f)
 2016-08-06 08:19:56 +05:30
Jason Lowe 470ea3da1f YARN-4573. Fix test failure in TestRMAppTransitions#testAppRunningKill and testAppKilledKilled. (Takashi Ohnishi via rohithsharmaks) 
(cherry picked from commit c01bee0108)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2016-08-05 20:10:04 +00:00
Jason Lowe eef24f5d93 YARN-5469. Increase timeout of TestAmFilter.testFilter. Contributed by Eric Badger 
(cherry picked from commit db4a61dc61)
 2016-08-03 19:54:00 +00:00
Jason Lowe 5883718eea YARN-4717. TestResourceLocalizationService.testPublicResourceInitializesLocalDir fails Intermittently due to IllegalArgumentException from cleanup (templedf via rkanter) 
(cherry picked from commit 22ca176dfe)
 2016-08-03 19:33:58 +00:00
Jason Lowe c972498b74 YARN-5462. TestNodeStatusUpdater.testNodeStatusUpdaterRetryAndNMShutdown fails intermittently. Contributed by Eric Badger 
(cherry picked from commit db646540f0)
 2016-08-03 19:19:47 +00:00
Jason Lowe aca7eea611 YARN-4280. CapacityScheduler reservations may not prevent indefinite postponement on a busy cluster. Contributed by Kuhu Shukla 2016-08-03 19:01:14 +00:00
Wangda Tan c322e749d6 YARN-5342. Improve non-exclusive node partition resource allocation in Capacity Scheduler. (Sunil G via wangda) 2016-08-03 11:29:12 -07:00
Junping Du bba06f4526 YARN-5434. Add -client|server argument for graceful decommmission. Contributed by Robert Kanter. 
(cherry picked from commit 95f2b98597)
(cherry picked from commit 47dd871f96)
 2016-07-29 10:31:17 -07:00
Jason Lowe 95fb2fe394 YARN-5438. TimelineClientImpl leaking FileSystem Instances causing Long running services like HiverServer2 daemon going OOM. Contributed by Rohith Sharma K S 
(cherry picked from commit a1890c32c5)
 2016-07-28 21:49:21 +00:00
Junping Du 08358c8d4e YARN-5432. Lock already held by another process while LevelDB cache store creation for dag. Contributed by Li Lu. 
(cherry picked from commit 7f3c306e2e)
(cherry picked from commit 932bf79244)
 2016-07-28 06:56:00 -07:00
mattf-horton c96cb3fd48 HADOOP-13382. Remove unneeded commons-httpclient dependencies from POM files in Hadoop and sub-projects, while adding it in to the only place it is still needed, hadoop-openstack. Contributed by Matt Foley. 2016-07-21 16:01:05 -07:00
Rohith Sharma K S 7234fce25b YARN-5092. TestRMDelegationTokens fails intermittently. Contributed by Jason Lowe. 
(cherry picked from commit 557a245d83)
 2016-07-21 12:49:52 +05:30
Rohith Sharma K S cdcd131d5f YARN-1126. Add validation of users input nodes-states options to nodes CLI. Contributed by Wei Yan. 
(cherry picked from commit be34b2a8fd)
 2016-07-21 12:25:14 +05:30
Akira Ajisaka 66f3056090 YARN-4883. Make consistent operation name in AdminService. Contributed by Kai Sasaki. 
(cherry picked from commit c63afdbe14)
(cherry picked from commit 2bf5c4eb1a)
 2016-07-20 16:52:22 -07:00
Vinod Kumar Vavilapalli 8c5101ea6d YARN-5340. Fixed a race condition in RollingLevelDBTimelineStore that caused loss of Timeline events. Contributed by Li Lu. 
(cherry picked from commit 1c9d2ab503)
 2016-07-20 08:39:19 -07:00
Varun Vasudev 478a2bb185 YARN-5309. Fix SSLFactory truststore reloader thread leak in TimelineClientImpl. Contributed by Weiwei Yang. 2016-07-20 13:02:34 +05:30
Wangda Tan 343633a6e8 YARN-4484. Available Resource calculation for a queue is not correct when used with labels. (Sunil G via wangda) 
(cherry picked from commit 24db9167f1)
(cherry picked from commit e34e1aa4fe)
 2016-07-15 11:46:17 -07:00
Akira Ajisaka 2f7857baea YARN-5159. Wrong Javadoc tag in MiniYarnCluster. Contributed by Andras Bokor. 
(cherry picked from commit 6cf017558a)
(cherry picked from commit 6664077727)
 2016-07-14 10:46:11 -07:00
Varun Saxena f87866446d YARN-5353. ResourceManager can leak delegation tokens when they are shared across apps. (Jason Lowe via Varun Saxena). 2016-07-13 07:40:10 +05:30
Jason Lowe 0a94c6d9da YARN-5317. testAMRestartNotLostContainerCompleteMsg may fail. Contributed by sandflee 
(cherry picked from commit 10b704c594)
 2016-07-12 20:39:09 +00:00
Jason Lowe 9d08ca1ed7 YARN-4393. Fix intermittent test failure for TestResourceLocalizationService#testFailedDirsResourceRelease (Varun Saxana via rohithsharmaks) 
(cherry picked from commit 791c1639ae)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2016-07-12 19:25:07 +00:00
Junping Du a0ddc5b3a5 New addendum to YARN-5080. Cannot obtain logs using YARN CLI -am for either KILLED or RUNNING AM. 
(cherry picked from commit 7705812c7e)
(cherry picked from commit 54812841e2)
 2016-07-12 08:37:47 -07:00
Jian He 23eb3c7ceb YARN-5270. Solve miscellaneous issues caused by YARN-4844. Contributed by Wangda Tan 2016-07-11 22:38:35 -07:00
Varun Saxena f685973e21 YARN-5037. Fix random failure of TestRMRestart#testQueueMetricsOnRMRestart (sandflee via Varun Saxena). 2016-07-11 09:39:06 +05:30
Junping Du 25ea7cc373 Fix OOM issue in DistributedShell. Contributed by Jian He. 
(cherry picked from commit 9bdb5bebea)
(cherry picked from commit 7db796562c)
 2016-07-09 05:12:21 -07:00
Varun Saxena 845daee463 YARN-5318. Fix intermittent test failure of TestRMAdminService#testRefreshNodesResourceWithFileSystemBasedConfigurationProvider. (Jun Gong via Varun Saxena). 2016-07-09 01:54:22 +05:30
Junping Du 8645e4ac3b YARN-4939. The decommissioning Node should keep alive during NM restart. Contributed by sandflee. 
(cherry picked from commit 30ee57ceb1)
(cherry picked from commit 5c0386d535)
 2016-07-08 06:06:56 -07:00
Vinod Kumar Vavilapalli e4a62a233d YARN-5314. Fixed a ConcurrentModificationException in ATS v1.5 EntityGroupFSTimelineStore. Contributed by Li Lu. 
(cherry picked from commit 673e5e02fe)
 2016-07-07 22:42:19 -07:00
Varun Saxena 22ca536b0c YARN-5286. Add RPC port info in RM web service's response when getting app status. (Jun Gong via Varun Saxena). 2016-07-06 09:31:48 +05:30
Vinod Kumar Vavilapalli 4ea87cb38c YARN-5214. Fixed locking in DirectoryCollection to avoid hanging NMs when various code-paths hit slow disks. Contributed by Junping Du. 
(cherry picked from commit ce9c006430)
 2016-07-05 17:12:37 -07:00
Jian He 4437e6f336 YARN-5023. TestAMRestart#testShouldNotCountFailureToMaxAttemptRetry fails. Contributed by sandflee 
(cherry picked from commit c35a5a7a8d)
 2016-07-01 14:29:47 -07:00
Rohith Sharma K S 33e6986ec9 YARN-5197. RM leaks containers if running container disappears from node update. Contributed by Jason Lowe. 2016-06-21 10:12:01 +05:30
Wangda Tan 93b406b66d YARN-4844. Addendum patch, revert few changes to public and evolving APIs to avoid breaking downstream projects as much as possible. 
(cherry picked from commit 7eb524ddee)
 2016-06-16 11:10:46 -07:00
Junping Du 810470508b YARN-5237. Fix missing log files issue in rolling log aggregation. Contributed by Xuan Gong. 2016-06-16 07:18:36 -07:00
Wangda Tan d838c6443d YARN-1942. Deprecate toString/fromString methods from ConverterUtils and move them to records classes like ContainerId/ApplicationId, etc. (wangda) 2016-06-14 15:21:41 -07:00
Wangda Tan 9436548152 YARN-3426. Add jdiff support to YARN. (vinodkv via wangda) 2016-06-10 09:56:18 -07:00
Junping Du 2be48e7d15 YARN-5199. Close LogReader in NMWebServices#getLogs. Contributed by Xuan Gong. 2016-06-09 12:29:25 -07:00
Xuan 11b4d1e486 Revert "YARN-4920. ATS/NM should support a link to dowload/get the logs in text format. Contributed by Xuan Gong." 
This reverts commit 22ac37615a.
 2016-06-08 11:23:12 -07:00
Vinod Kumar Vavilapalli 593d2c66c4 YARN-5080. Addendum fix to the original patch to fix YARN logs CLI. Contributed by Xuan Gong 
(cherry picked from commit 5a43583c0b)
 2016-06-08 09:52:37 -07:00
Jason Lowe fb31aaef63 YARN-5206. RegistrySecurity includes id:pass in exception text if considered invalid. Contributed by Steve Loughran 
(cherry picked from commit 8c8a377cac)
 2016-06-08 14:13:48 +00:00
Wangda Tan 19e578870d YARN-4844. Add getMemorySize/getVirtualCoresSize to o.a.h.y.api.records.Resource. (wangda) 2016-06-07 12:41:50 -07:00
Arun Suresh 51a16b1ff0 YARN-4525. Fix bug in RLESparseResourceAllocation.getRangeOverlapping(). (Ishai Menache and Carlo Curino via asuresh) 
(cherry picked from commit 3a154f75ed)
(cherry picked from commit d3655f4123)
 2016-06-06 21:19:48 -07:00
Arun Suresh 934bd8989b YARN-5185. StageAllocaterGreedyRLE: Fix NPE in corner case. (Carlo Curino via asuresh) 
(cherry picked from commit 7a9b7372a1)
(cherry picked from commit f0a869b52a)
 2016-06-06 21:08:52 -07:00
Ming Ma ec4f9a14f9 MAPREDUCE-5044. Have AM trigger jstack on task attempts that timeout before killing them. (Eric Payne and Gera Shegalov via mingma) 
(cherry picked from commit 4a1cedc010)
(cherry picked from commit 74e2b5efa2)
 2016-06-06 14:49:43 -07:00
Arun Suresh 6963d50e2f YARN-5165. Fix NoOvercommitPolicy to take advantage of RLE representation of plan. (Carlo Curino via asuresh) 
(cherry picked from commit db54670e83)
(cherry picked from commit 8ebd89dcfb)
 2016-06-03 14:52:45 -07:00
Vinod Kumar Vavilapalli c87b9c1471 YARN-5098. Fixed ResourceManager's DelegationTokenRenewer to replace expiring system-tokens if RM stops and only restarts after a long time. Contributed by Jian He. 
Made one minor edit for branch-2 patch.

(cherry picked from commit f10ebc67f5)
(cherry picked from commit 029888871f)
 2016-06-03 13:15:44 -07:00
Jian He 3c2bd19fa5 YARN-5190. Registering/unregistering container metrics in ContainerMonitorImpl and ContainerImpl causing uncaught exception in ContainerMonitorImpl. Contributed by Junping Du 
(cherry picked from commit 99cc439e29)
 2016-06-03 11:11:49 -07:00
Jian He 01a3f7899c YARN-1815. Work preserving recovery of Unmanged AMs. Contributed by Subru Krishnan 
(cherry picked from commit 097baaaeba)
 2016-06-03 10:50:12 -07:00
Wangda Tan 2f3e1d965d Revert "YARn-4844. Add getMemoryLong/getVirtualCoreLong to o.a.h.y.api.records.Resource. Contributed by Wangda Tan." 
This reverts commit 457884737f.
 2016-05-31 22:16:53 -07:00
Varun Vasudev 457884737f YARn-4844. Add getMemoryLong/getVirtualCoreLong to o.a.h.y.api.records.Resource. Contributed by Wangda Tan. 2016-05-29 20:57:56 +05:30
Karthik Kambatla e5b1fb2a91 YARN-5132. Exclude generated protobuf sources from YARN Javadoc build. (Subru Krishnan via kasha) 
(cherry picked from commit f5ff05cc8a)
(cherry picked from commit b3f5337bb3)
 2016-05-28 08:12:56 -07:00
Akira Ajisaka 01e7c2bbe4 YARN-5166. javadoc:javadoc goal fails on hadoop-yarn-client. Contributed by Andras Bokor. 
(cherry picked from commit e4022debf7)
(cherry picked from commit ec811909e4)
 2016-05-27 23:16:00 +09:00
Junping Du 6b41fceb69 YARN-4987. Read cache concurrency issue between read and evict in EntityGroupFS timeline store. Contributed by Li Lu. 
(cherry picked from commit 705286ccae)
(cherry picked from commit dea79a7dd6)
 2016-05-27 07:03:18 -07:00
Karthik Kambatla 427e3f995e YARN-5035. FairScheduler: Adjust maxAssign dynamically when assignMultiple is turned on. (kasha) 
(cherry picked from commit 04ded558b0)
(cherry picked from commit 59335b4d7a)
 2016-05-26 16:54:38 -07:00
Carlo Curino 7982933c09 YARN-4957. Add getNewReservation in ApplicationClientProtocol (Sean Po via curino) 
(cherry picked from commit 013532a95e)
(cherry picked from commit c656977961)
 2016-05-25 17:02:22 -07:00
Jason Lowe fe10caee8d YARN-4459. container-executor should only kill process groups. Contributed by Jun Gong 
(cherry picked from commit 1ba31fe9e9)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
 2016-05-25 21:37:31 +00:00
Karthik Kambatla b491d24bb6 YARN-3344. Fix warning - procfs stat file is not in the expected format. (Ravindra Kumar Naik and Akira Ajisaka via kasha) 
(cherry picked from commit 77d5ce9069)
(cherry picked from commit 0ac8c098e8)
 2016-05-25 10:53:01 -07:00
Varun Vasudev 35456bb7c9 YARN-857. Localization failures should be available in container diagnostics. Contributed by Vinod Kumar Vavilapalli. 
(cherry picked from commit f440a9d8c4a177bc5062d21d4b4bc4d9b2944344)
(cherry picked from commit 36f2ae0692)
 2016-05-25 19:03:33 +05:30
Naganarasimha 943cefc81e YARN-5114. Add additional tests in TestRMWebServicesApps and rectify testInvalidAppAttempts failure in 2.8. Contributed by Bibin A Chundatt 2016-05-25 06:27:44 +08:00
Naganarasimha abab31b8f2 YARN-3971. Skip RMNodeLabelsManager#checkRemoveFromClusterNodeLabelsOfQueue on nodelabel recovery. (addendum patch). Contributed by Bibin A chundatt 
(cherry picked from commit b4078bd17b)
 2016-05-24 08:19:26 +08:00
Jason Lowe 0bfad9675e YARN-5055. max apps per user can be larger than max per queue. Contributed by Eric Badger 
(cherry picked from commit ac954486c5)
 2016-05-23 15:56:33 +00:00
Jason Lowe 8e404b4321 YARN-5103. With NM recovery enabled, restarting NM multiple times results in AM restart. Contributed by Junping Du 
(cherry picked from commit d1df0266cf)
 2016-05-23 15:17:26 +00:00
Jian He dbde956bf5 YARN-5020. Fix Documentation for Yarn Capacity Scheduler on Resource Calculator. Contributed by Takashi Ohnishi 
(cherry picked from commit d364ceac85)
(cherry picked from commit 6e0522ee1b)
 2016-05-20 11:36:40 -07:00
Rohith Sharma K S f102c4f6c8 YARN-5068. Expose scheduler queue to application master. (Harish Jaiprakash via rohithsharmaks) 2016-05-20 08:53:21 +05:30
Jian He 61fa9256c5 YARN-4002. Make ResourceTrackerService#nodeHeartbeat more concurrent. Contributed by Rohith Sharma K S & Zhiguo Hong 
(cherry picked from commit feb90ffcca)
 2016-05-19 13:02:24 -07:00
Junping Du c8843cac03 YARN-5100. The YarnApplicationState is always running in ATS even application is finished. Contributed by Xuan Gong. 
(cherry picked from commit 141873ca7d)
(cherry picked from commit 416274b53e)
 2016-05-19 09:54:03 -07:00
Steve Loughran 7a001ae19b HADOOP-12767. Update apache httpclient version to 4.5.2; httpcore to 4.4.4. Artem Aliev via stevel. 2016-05-19 14:47:46 +01:00
Rohith Sharma K S 726c1f14b8 YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999). Contributed by Mohammad Shahid Khan and Varun Saxena 2016-05-19 10:50:32 +05:30
Naganarasimha 0fb6a05577 YARN-4925. ContainerRequest in AMRMClient, application should be able to specify nodes/racks together with nodeLabelExpression. Contributed by Bibin A Chundatt 
(cherry picked from commit f04c81c9ce)
 2016-05-19 01:40:06 +08:00
Eric Payne a552932b38 YARN-5069. TestFifoScheduler.testResourceOverCommit race condition. Contributed by Eric Badger. 
(cherry picked from commit 1217c8f6b4)
 2016-05-16 20:44:41 +00:00
Jason Lowe 70faa87ccf YARN-4325. Nodemanager log handlers fail to send finished/failed events in some cases. Contributed by Junping Du 
(cherry picked from commit 81effb7dcd)
 2016-05-16 15:43:42 +00:00
Xuan 4eeec4e35b YARN-5080. Cannot obtain logs using YARN CLI -am for either KILLED or 
RUNNING AM. Contributed by Xuan Gong

(cherry picked from commit 7be53b65f9)
(cherry picked from commit 3fe4107909)
 2016-05-14 12:22:33 -07:00
Karthik Kambatla a9b0041abc YARN-4995. FairScheduler: Display per-queue demand on the scheduler page. (xupeng via kasha) 
(cherry picked from commit 4b4e4c6ba8)
(cherry picked from commit f96b2b6fac)
 2016-05-11 17:40:57 -07:00
Junping Du 6d1300880f YARN-5029. RM needs to send update event with YarnApplicationState as Running to ATS/AHS. Contributed by Xuan Gong. 
(cherry picked from commit 39f2bac38b)
(cherry picked from commit 9dc3683d87)
 2016-05-11 09:40:51 -07:00
Naganarasimha b47259caf0 YARN-4926. Change nodelabel rest API invalid reponse status to 400. Contributed by Bibin A Chundatt 
(cherry picked from commit 2750fb900f)
 2016-05-08 23:06:15 +05:30
Yongjun Zhang 5ae5b869aa YARN-5048. DelegationTokenRenewer#skipTokenRenewal may throw NPE (Jian He via Yongjun Zhang) 
(cherry picked from commit 47c41e7ac7)
 2016-05-06 22:43:44 -07:00
Jason Lowe da24fe95f8 YARN-4747. AHS error 500 due to NPE when container start event is missing. Contributed by Varun Saxena 
(cherry picked from commit b2ed6ae731)
 2016-05-06 23:00:53 +00:00
Karthik Kambatla 367e757d8d YARN-4947. Test timeout is happening for TestRMWebServicesNodes. Contributed by Bibin A Chundatt 2016-05-06 14:28:24 +05:30
Wangda Tan 3620d0e623 YARN-4984. LogAggregationService shouldn't swallow exception in handling createAppDir() which cause thread leak. (Junping Du via wangda) 
(cherry picked from commit 7bd418e48c)
 2016-05-05 10:08:37 -07:00
Jason Lowe b452ecf4a0 YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla 
(cherry picked from commit ee86cef2fe)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
 2016-05-05 14:40:35 +00:00
Junping Du 22ac37615a YARN-4920. ATS/NM should support a link to dowload/get the logs in text format. Contributed by Xuan Gong. 
(cherry picked from commit 3cf223166d452a0f58f92676837a9edb8ddc1139)
(cherry picked from commit c79dc07dc193904f2586a5d64ea2f4e56d2396b8)
 2016-05-04 09:49:08 -07:00
Jason Lowe 8013529746 YARN-4834. ProcfsBasedProcessTree doesn't track daemonized processes. Contributed by Nathan Roberts 
(cherry picked from commit c6b4839168)
 2016-05-03 22:15:31 +00:00
Junping Du 4f1c198881 YARN-4851. Metric improvements for ATS v1.5 storage components. Li Lu via junping_du. 
(cherry picked from commit 06413da72e)
(cherry picked from commit 47f67ae447)
 2016-05-03 04:59:57 -07:00
rohithsharmaks 9ea567da42 YARN-4109. Exception on RM scheduler page loading with labels. (Mohammad Shahid Khan via rohithsharmaks) 
(cherry picked from commit 8c180a13c8)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2016-05-03 10:31:32 +05:30