Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,296 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

2939 Commits

Author SHA1 Message Date
Robert Kanter 4f8c7f819c YARN-5400. addendum patch to fix JDK 7 issue 2016-09-28 16:01:18 -07:00
Robert Kanter bc91e33d5e YARN-5400. Light cleanup in ZKRMStateStore (templedf via rkanter) 
(cherry picked from commit bcb2528a51)
 2016-09-28 14:57:14 -07:00
Varun Vasudev 3f1d3e3177 YARN-5662. Provide an option to enable ContainerMonitor. Contributed by Jian He. 
(cherry picked from commit bc2656f09f)
 2016-09-28 15:27:09 +05:30
Wangda Tan 379c2b3e5b YARN-3142. Improve locks in AppSchedulingInfo. (Varun Saxena via wangda) 
(cherry picked from commit 1831be8e73)
 2016-09-27 12:05:00 -07:00
Wangda Tan 269401dc83 HADOOP-13544. JDiff reports unncessarily show unannotated APIs and cause confusion while our javadocs only show annotated and public APIs. (vinodkv via wangda) 
(cherry picked from commit 875062b5bc)
 2016-09-27 11:27:25 -07:00
Naganarasimha 86ec99e82d YARN-5660. Wrong audit constants are used in Get/Put of priority in RMWebService. Contributed by Rohith Sharma K S 
(cherry picked from commit ebf528c443)
 2016-09-27 16:42:21 +05:30
Arun Suresh 7484d0b1b9 YARN-5609. Expose upgrade and restart API in ContainerManagementProtocol. Contributed by Arun Suresh 2016-09-26 23:55:13 +08:00
Akira Ajisaka 74f2df16a9 YARN-5663. Small refactor in ZKRMStateStore. Contributed by Oleksii Dymytrov. 
(cherry picked from commit 14a696f369)
 2016-09-26 15:01:13 +09:00
Naganarasimha eec0afa098 YARN-3877. YarnClientImpl.submitApplication swallows exceptions. Contributed by Varun Saxena 
(cherry picked from commit e4e72db5f9)
 2016-09-25 17:38:01 +05:30
Naganarasimha 733bfa552a YARN-5664. Fix Yarn documentation to link to correct versions. Contributed by Xiao Chen 
(cherry picked from commit 4d16d2d72f)
 2016-09-24 23:22:50 +05:30
Naganarasimha 0e4a5611ff YARN-3692. Allow REST API to set a user generated message when killing an application. Contributed by Rohith Sharma K S 2016-09-24 21:13:01 +05:30
Kihwal Lee 9dec68c269 YARN-5622. TestYarnCLI.testGetContainers fails due to mismatched date formats. Contributed by Eric Badger. 
(cherry picked from commit 6e849cb658)
 2016-09-23 14:10:46 -05:00
Varun Saxena b58651d8d4 YARN-5539. TimelineClient failed to retry on java.net.SocketTimeoutException: Read timed out (Junping Du via Varun Saxena) 2016-09-23 13:30:56 +05:30
Robert Kanter 0e4f2a0edf YARN-4973. YarnWebParams next.fresh.interval should be next.refresh.interval (templedf via rkanter) 
(cherry picked from commit 5ffd4b7c1e)
 2016-09-22 16:46:27 -07:00
Wangda Tan b9b6396e09 YARN-4591. YARN Web UIs should provide a robots.txt. (Sidharta Seethana via wangda) 
(cherry picked from commit 5a58bfee30)
 2016-09-20 17:23:01 -07:00
Arun Suresh 2a9f809c96 YARN-5656. Fix ReservationACLsTestBase. (Sean Po via asuresh) 
(cherry picked from commit 9f03b403ec)
 2016-09-20 12:27:50 -07:00
Wangda Tan 679652fcbd Addendum patch for fix javadocs failure which is caused by YARN-3141. (wangda) 
(cherry picked from commit e45307c9a0)
 2016-09-20 11:21:47 -07:00
Jason Lowe 7231000149 YARN-5655. TestContainerManagerSecurity#testNMTokens is asserting. Contributed by Robert Kanter 
(cherry picked from commit c6d1d742e7)
 2016-09-20 14:16:47 +00:00
Jian He 3acd30df71 YARN-3140. Improve locks in AbstractCSQueue/LeafQueue/ParentQueue. Contributed by Wangda Tan 
(cherry picked from commit 2b66d9ec5b)
 2016-09-20 15:05:54 +08:00
Jason Lowe 035f5f8f1d YARN-5540. Scheduler spends too much time looking at empty priorities. Contributed by Jason Lowe 
(cherry picked from commit 7558dbbb48)
 2016-09-19 20:33:59 +00:00
Jian He 9942ca2bf0 YARN-3141. Improve locks in SchedulerApplicationAttempt/FSAppAttempt/FiCaSchedulerApp. Contributed by Wangda Tan 
(cherry picked from commit b8a30f2f17)
 2016-09-19 17:08:57 +08:00
Arun Suresh dfaac56433 YARN-5637. Changes in NodeManager to support Container rollback and commit. (asuresh) 
(cherry picked from commit 3552c2b99d)
 2016-09-18 11:06:38 -07:00
Arun Suresh 00759f50be YARN-5657. Fix TestDefaultContainerExecutor. (asuresh) 
(cherry picked from commit f67237cbe7)
 2016-09-17 09:32:55 -07:00
Naganarasimha 20729ead91 YARN-5642. Typos in 9 log messages. Contributed by Mehran Hassani 2016-09-17 10:37:42 +05:30
Naganarasimha 656b520dc6 YARN-4232. TopCLI console support for HA mode. Contributed by Bibin A Chundatt 
(cherry picked from commit ade7c2bc9c)
 2016-09-17 09:54:29 +05:30
Wangda Tan ee101e49fa YARN-4091. Add REST API to retrieve scheduler activity. (Chen Ge via wangda) 2016-09-15 14:49:03 -07:00
Arun Suresh 16aecc699d YARN-5620. Core changes in NodeManager to support re-initialization of Containers with new launchContext. (asuresh) 
(cherry picked from commit 40b5a59b72)
 2016-09-15 08:11:14 -07:00
Jason Lowe 4f2f0eb0bb YARN-5630. NM fails to start after downgrade from 2.8 to 2.7. Contributed by Jason Lowe 
(cherry picked from commit e793309735)
 2016-09-13 14:43:21 +00:00
Rohith Sharma K S 8c8ff0cfa3 YARN-5631. Missing refreshClusterMaxPriority usage in rmadmin help message. Contributed by Kai Sasaki. 
(cherry picked from commit 729de3e6b6)
 2016-09-13 15:27:20 +05:30
Karthik Kambatla a064375e68 YARN-5616. Clean up WeightAdjuster. (Yufei Gu via kasha) 
(cherry picked from commit c0e492e50f)
 2016-09-06 22:40:49 -07:00
Varun Vasudev ec3a651b17 YARN-5576. Allow resource localization while container is running. Contributed by Jian He. 2016-09-06 20:07:54 +05:30
Rohith Sharma K S de9f046598 YARN-5608. TestAMRMClient.setup() fails with ArrayOutOfBoundsException. Contributed by Daniel Templeton. 
(cherry picked from commit f9557127b3)
 2016-09-06 14:13:41 +05:30
Karthik Kambatla a5748eec7b YARN-5264. Store all queue-specific information in FSQueue. (Yufei Gu via kasha) 
(cherry picked from commit f6ea9be547)
 2016-09-02 15:02:20 -07:00
Ray Chiang 4ee1729cb0 YARN-5549. AMLauncher#createAMContainerLaunchContext() should not log the command to be launched indiscriminately. (Daniel Templeton via rchiang) 2016-09-02 14:57:05 -07:00
Varun Vasudev 1222433729 YARN-5555. Scheduler UI: "% of Queue" is inaccurate if leaf queue is hierarchically nested. Contributed by Eric Payne. 
(cherry picked from commit 05f5c0f631)

 Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/common/fica/FiCaSchedulerApp.java
 2016-09-02 16:16:31 +05:30
Karthik Kambatla 736dd1646d YARN-5566. Client-side NM graceful decom is not triggered when jobs finish. (Robert Kanter via kasha) 
(cherry picked from commit 74f4bae455)
 2016-09-01 14:44:41 -07:00
Varun Vasudev 34f9330651 YARN-5596. Fix failing unit test in TestDockerContainerRuntime. Contributed by Sidharta Seethana. 
(cherry picked from commit 08f55ccbb0)
 2016-09-01 14:10:18 +05:30
Arun Suresh b279f42d79 YARN-5221. Expose UpdateResourceRequest API to allow AM to request for change in container properties. (asuresh) 
(cherry picked from commit d6d9cff21b)
 2016-08-30 16:33:31 -07:00
Zhe Zhang 3ad4f0823c YARN-5550. TestYarnCLI#testGetContainers should format according to CONTAINER_PATTERN. Contributed by Jonathan Hung. 
(cherry picked from commit ed6ff5cd78)
 2016-08-29 14:49:03 -07:00
Jason Lowe eeb18b0a46 YARN-5560. Clean up bad exception catching practices in TestYarnClient. Contributed by Sean Po 
(cherry picked from commit 4cbe61407d)
 2016-08-29 16:15:55 +00:00
Wangda Tan 07e1527cc1 YARN-4889. Changes in AMRMClient for identifying resource-requests explicitly. (Arun Suresh via wangda) 
(cherry picked from commit 19c743c1bb)
 2016-08-26 17:23:55 -07:00
Subru Krishnan 3ad7cb3e1c YARN-5327. API changes required to support recurring reservations in the YARN ReservationSystem. (Sangeetha Abdu Jyothi via Subru). 
(cherry picked from commit b930dc3ec0)
 2016-08-26 17:01:37 -07:00
Karthik Kambatla d9b42248de YARN-5373. NPE listing wildcard directory in containerLaunch. (Daniel Templeton via kasha) 
(cherry picked from commit cde3a00526)
 2016-08-26 11:08:16 -07:00
Junping Du af1502fcab YARN-5557. Add localize API to the ContainerManagementProtocol. Contributed by Jian He. 
(cherry picked from commit 9ef632f3b0)
 2016-08-26 09:05:33 -07:00
Naganarasimha 026027166e YARN-3940. Application moveToQueue should check NodeLabel permission. Contributed by Bibin A Chundatt 
(cherry picked from commit 46e02ab719)
 2016-08-26 20:23:01 +05:30
Varun Vasudev 49a97a76f5 YARN-5430. Return container's ip and host from NM ContainerStatus call. Contributed by Jian He. 2016-08-26 16:33:21 +05:30
Naganarasimha ee3358402a YARN-5564. Fix typo in RM_SCHEDULER_RESERVATION_THRESHOLD_INCREMENT_MULTIPLE. Contributed by Ray Chiang 
(cherry picked from commit 27c3b86252)
 2016-08-26 08:53:42 +05:30
Jason Lowe 4037b22b7c YARN-5389. TestYarnClient#testReservationDelete fails. Contributed by Sean Po 2016-08-25 15:10:23 +00:00
Jason Lowe 6de262cdc4 Revert "YARN-5389. TestYarnClient#testReservationDelete fails. Contributed by Sean Po" 
This reverts commit 782aeabcd5.
 2016-08-25 14:59:20 +00:00
Jason Lowe 782aeabcd5 YARN-5389. TestYarnClient#testReservationDelete fails. Contributed by Sean Po 
(cherry picked from commit 3d86110a5c)
 2016-08-25 14:47:40 +00:00
Varun Vasudev 952c309dd9 YARN-5042. Mount /sys/fs/cgroup into Docker containers as read only mount. Contributed by luhuichun. 
(cherry picked from commit 42d9876471ff0bf0ea240cd48fe483cda8aa1ec7)
 2016-08-25 14:18:12 +05:30
Varun Saxena 42feab2436 YARN-5537. Fix intermittent failure of TestAMRMClient#testAMRMClientWithContainerResourceChange (Bibin A Chundatt via Varun Saxena) 2016-08-25 14:12:28 +05:30
Arun Suresh bd348d20b2 Revert "YARN-5049. Extend NMStateStore to save queued container information. (Konstantinos Karanasos via asuresh)" 
This reverts commit 307cda70db.
 2016-08-24 10:23:06 -07:00
Naganarasimha 5fa241daa0 YARN-4491. yarn list command to support filtering by tags. Contributed by Varun Saxena 
(cherry picked from commit 143c59e4c5)
 2016-08-24 01:55:24 +05:30
Rohith Sharma K S a19859c3f5 YARN-5544. TestNodeBlacklistingOnAMFailures fails on trunk. Contributed by Sunil G. 
(cherry picked from commit 0d5997d2b9)
 2016-08-23 14:39:19 +05:30
Wangda Tan 3e511f39c2 YARN-4685. Disable AM blacklisting by default to mitigate situations that application get hanged. (Rohith Sharma K S via wangda) 
(cherry picked from commit 2da32a6ef9)
 2016-08-19 16:47:42 -07:00
Wangda Tan c7d782d2f6 YARN-3388. Allocation in LeafQueue could get stuck because DRF calculator isn't well supported when computing user-limit. (Nathan Roberts via wangda) 
(cherry picked from commit 444b2ea7af)
 2016-08-19 16:47:03 -07:00
Varun Saxena 928cccb9f9 YARN-5533. JMX AM Used metrics for queue wrong when app submited to nodelabel partition (Bibin A Chundatt via Varun Saxena) 2016-08-19 17:32:23 +05:30
Varun Saxena a746775e9a Revert "YARN-5533. JMX AM Used metrics for queue wrong when app submited to nodelabel partition (Bibin A Chundatt via Varun Saxena)" 
This reverts commit 613048a8a2.
 2016-08-19 16:40:48 +05:30
Varun Saxena 613048a8a2 YARN-5533. JMX AM Used metrics for queue wrong when app submited to nodelabel partition (Bibin A Chundatt via Varun Saxena) 2016-08-19 15:18:42 +05:30
Junping Du d464483bf7 YARN-4676. Automatic and Asynchronous Decommissioning Nodes Status Tracking. Contributed by Diniel Zhi. 2016-08-18 07:23:29 -07:00
Varun Saxena 5fa7a14da9 YARN=5526. DrainDispacher#serviceStop blocked if setDrainEventsOnStop invoked (sandflee via Varun Saxena) 2016-08-18 12:23:34 +05:30
Karthik Kambatla 07d5ab16df YARN-4702. FairScheduler: Allow setting maxResources for ad hoc queues. (Daniel Templeton via kasha) 
(cherry picked from commit 20f0eb871c)
 2016-08-17 17:54:28 -07:00
Varun Saxena 9dcb7197d4 YARN-5523. Yarn running container log fetching causes OutOfMemoryError (Xuan Gong via Varun Saxena) 2016-08-18 01:53:55 +05:30
Varun Vasudev 3ee1fb774e YARN-5455. Update Javadocs for LinuxContainerExecutor. Contributed by Daniel Templeton. 
(cherry picked from commit 7f05ff7a4e)
 2016-08-17 15:35:23 +05:30
Varun Saxena 96e9f8bec9 YARN-5475. Fix test failure of TestAggregatedLogFormat#testReadAcontainerLogs1 (Jun Gong via Varun Saxena) 2016-08-16 20:28:02 +05:30
Varun Saxena da70d6f25d YARN-5514. Clarify DecommissionType.FORCEFUL comment (Vrushali C via Varun Saxena) 2016-08-16 14:09:47 +05:30
Varun Saxena 354afbb9df YARN-5521. Fix random failure of TestCapacityScheduler#testKillAllAppsInQueue (sandflee via Varun Saxena) 2016-08-16 00:04:40 +05:30
Varun Saxena 3c7cf5cd77 YARN-5491. Fix random failure of TestCapacityScheduler#testCSQueueBlocked (Bibin A Chundatt via Varun Saxena) 2016-08-15 03:56:31 +05:30
Varun Saxena 918a0f12fa YARN-5476. Non existent application reported as ACCEPTED by YarnClientImpl (Junping Du via Varun Saxena) 2016-08-13 00:19:00 +05:30
Naganarasimha b63d3dcac9 YARN-4833. For Queue AccessControlException client retries multiple times on both RM. Contributed by Bibin A Chundatt 2016-08-12 20:16:15 +05:30
Rohith Sharma K S 0937715f7f YARN-5492. TestSubmitApplicationWithRMHA is failing sporadically during precommit builds. Contributed by Vrushali C. 
(cherry picked from commit 5199db387d)
 2016-08-11 11:51:56 +05:30
Rohith Sharma K S 05b384271c YARN-2398. TestResourceTrackerOnHA crashes. Contributed by Ajith S. 
(cherry picked from commit 666ad0b67b)
 2016-08-11 11:41:36 +05:30
Ray Chiang 372f6f8456 YARN-5137. Make DiskChecker pluggable in NodeManager. (Yufei Gu via rchiang) 
(cherry picked from commit dbe9e70cc084220ea1f68da850cdb092281b5e96)
 2016-08-10 19:26:49 -07:00
Jason Lowe 2b31f6fb2c YARN-5382. RM does not audit log kill request for active applications. Contributed by Vrushali C 
(cherry picked from commit 5c95bb315b)
 2016-08-10 18:27:25 +00:00
Jason Lowe 3d401206cf YARN-5483. Optimize RMAppAttempt#pullJustFinishedContainers. Contributed by sandflee 2016-08-10 18:06:56 +00:00
Karthik Kambatla 447ba47640 YARN-5343. TestContinuousScheduling#testSortedNodes fails intermittently. (Yufei Gu via kasha) 
(cherry picked from commit 7992c0b42c)
 2016-08-09 16:51:27 -07:00
Varun Vasudev c599f6215e YARN-5394. Remove bind-mount /etc/passwd for Docker containers. Contributed by Zhankun Tang. 
(cherry picked from commit 522ddbde79)
 2016-08-09 16:05:03 +05:30
Arun Suresh 5f7edb79d1 YARN-5457. Refactor DistributedScheduling framework to pull out common functionality. (asuresh) 
(cherry picked from commit 82c9e06101)
 2016-08-09 00:46:08 -07:00
Naganarasimha 93b768d0d1 YARN-5287. LinuxContainerExecutor fails to set proper permission. Contributed by Ying Zhang. 2016-08-09 06:48:39 +05:30
Vinod Kumar Vavilapalli (I am also known as @tshooter.) 31bb302c91 YARN-5470. Addedum to differentiate exactly matching of log-files with regex in yarn log CLI. Contributed by Xuan Gong. 
(cherry picked from commit 83a2ffec40)
 2016-08-08 14:29:24 -07:00
Varun Saxena ce8df272f4 YARN-4910. Fix incomplete log info in ResourceLocalizationService (Jun Gong via Varun Saxena) 2016-08-08 22:14:56 +05:30
Naganarasimha 991823482f YARN-4624. NPE in PartitionQueueCapacitiesInfo while accessing Schduler UI. Contributed by Brahma Reddy Battula 
(cherry picked from commit d81b8163b4)
 2016-08-06 07:04:30 +05:30
Varun Saxena 0e48f29937 YARN-5429. Fix return related javadoc warnings in yarn-api (Vrushali C via Varun Saxena) 2016-08-06 01:13:30 +05:30
Junping Du adcecd417b YARN-5470. Differentiate exactly match with regex in yarn log CLI. Contributed by Xuan Gong. 
(cherry picked from commit e605d47df0)
 2016-08-05 11:00:13 -07:00
Wangda Tan a73e118437 YARN-4888. Changes in scheduler to identify resource-requests explicitly by allocation-id. (Subru Krishnan via wangda) 
(cherry picked from commit 3f100d76ff)
 2016-08-05 10:44:07 -07:00
Rohith Sharma K S 69da77c357 YARN-5333. Some recovered apps are put into default queue when RM HA. Contributed by Jun Gong. 
(cherry picked from commit d9a354c2f3)
 2016-08-05 21:37:05 +05:30
Varun Vasudev 19d894538d YARN-5459. Add support for docker rm. Contributed by Shane Kumpf. 
(cherry picked from commit 08e33381f5)
 2016-08-04 14:18:01 +05:30
Jason Lowe 65a91c8527 YARN-5469. Increase timeout of TestAmFilter.testFilter. Contributed by Eric Badger 
(cherry picked from commit db4a61dc61)
 2016-08-03 19:53:28 +00:00
Jason Lowe 1ea03411c2 YARN-5462. TestNodeStatusUpdater.testNodeStatusUpdaterRetryAndNMShutdown fails intermittently. Contributed by Eric Badger 
(cherry picked from commit db646540f0)
 2016-08-03 19:18:50 +00:00
Jason Lowe 9a0ac56a5c YARN-4280. CapacityScheduler reservations may not prevent indefinite postponement on a busy cluster. Contributed by Kuhu Shukla 
(cherry picked from commit 4d92aefd35)
 2016-08-03 19:00:06 +00:00
Varun Vasudev 1087745d79 YARN-5460. Change container runtime type logging in DelegatingLinuxContainerRuntime to debug. Contributed by Shane Kumpf. 
(cherry picked from commit d848184e90)
 2016-08-03 12:03:31 +05:30
Junping Du 7e213844a7 YARN-5226. Remove AHS enable check from LogsCLI#fetchAMContainerLogs. Contributed by Xuan Gong. 
(cherry picked from commit 3818393297)
 2016-08-02 10:42:31 -07:00
Varun Vasudev b63e6c8027 YARN-5458. Rename DockerStopCommandTest to TestDockerStopCommand. Contributed by Shane Kumpf. 
(cherry picked from commit 7fc70c6422)
 2016-08-02 17:02:17 +05:30
Varun Vasudev 4e5409a2bb YARN-5443. Add support for docker inspect command. Contributed by Shane Kumpf. 
(cherry picked from commit 2e7c2a13a8)
 2016-08-02 17:01:59 +05:30
Varun Vasudev 7ecb1635b4 YARN-5444. Fix failing unit tests in TestLinuxContainerExecutorWithMocks. Contributed by Yufei Gu. 
(cherry picked from commit 4455aa7c86a36b6e017e288479a7d4ced889faa0)
 2016-08-01 16:01:10 +05:30
Arun Suresh f197378f81 YARN-5113. Refactoring and other clean-up for distributed scheduling. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit e5766b1dbe)
 2016-07-31 11:49:09 -07:00
Junping Du 47dd871f96 YARN-5434. Add -client|server argument for graceful decommmission. Contributed by Robert Kanter. 
(cherry picked from commit 95f2b98597)
 2016-07-29 10:27:11 -07:00
Li Lu 0b5d96abb5 YARN-5436. Race in AsyncDispatcher can cause random test failures in Tez (probably YARN also). (Zhiyuan Yang via gtcarrera9) 
(cherry picked from commit 7086fc72ee)
 2016-07-28 16:52:21 -07:00
Subru Krishnan 2f4df2b851 YARN-5203.Return ResourceRequest JAXB object in ResourceManager Cluster Applications REST API. Contributed by Ellen Hui. 
(cherry picked from commit 4e756d7271)
 2016-07-28 16:04:30 -07:00