d3bb69a667
YARN-5575. Many classes use bare yarn. properties instead of the defined constants. Contributed by Daniel Templeton.
2016-10-26 15:32:07 +09:00
de6faae97c
YARN-5388. Deprecate and remove DockerContainerExecutor. (Daniel Templeton via kasha)
2016-10-25 13:35:47 -07:00
a71fc81655
YARN-5754. Null check missing for earliest in FifoPolicy. (Yufei Gu via kasha)
2016-10-24 22:19:23 -07:00
44eb2bd7ae
YARN-5747. Application timeline metric aggregation in timeline v2 will lose last round aggregation when an application finishes (Li Lu via Varun Saxena)
2016-10-22 01:14:49 +05:30
754cb4e30f
YARN-5047. Refactor nodeUpdate across schedulers. (Ray Chiang via kasha)
2016-10-20 21:17:48 -07:00
a064865abf
YARN-4911. Bad placement policy in FairScheduler causes the RM to crash
2016-10-20 20:57:04 -07:00
e9c4616b5e
YARN-5561. [Atsv2] : Support for ability to retrieve apps/app-attempt/containers and entities via REST. Contributed by Rohith Sharma K S.
2016-10-19 09:45:23 -07:00
b733a6f862
YARN-5718. TimelineClient (and other places in YARN) shouldn't over-write HDFS client retry settings which could cause unexpected behavior. Contributed by Junping Du.
2016-10-18 11:06:47 -07:00
b154d3edce
YARN-5743. [Atsv2] Publish queue name and RMAppMetrics to ATS (Rohith Sharma K S via Varun Saxena)
2016-10-18 23:32:52 +05:30
f5d9235914
YARN-5466. DefaultContainerExecutor needs JavaDocs (templedf via rkanter)
2016-10-17 14:29:09 -07:00
1f304b0c7f
YARN-5699. Retrospect yarn entity fields which are publishing in events info fields. Contributed by Rohith Sharma K S.
2016-10-15 13:54:40 -07:00
cf3f43e95b
YARN-5717. Add tests for container-executor is_feature_enabled. Contributed by Sidharta Seethana
2016-10-13 20:49:07 -07:00
6476934ae5
YARN-5677. RM should transition to standby when connection is lost for an extended period. (Daniel Templeton via kasha)
2016-10-11 22:07:10 -07:00
6378845f9e
YARN-4464. Lower the default max applications stored in the RM and store. (Daniel Templeton via kasha)
2016-10-11 21:42:08 -07:00
0773ffd0f8
YARN-5057. Resourcemanager.security.TestDelegationTokenRenewer fails in trunk. Contributed by Jason Lowe.
2016-10-10 18:04:47 -04:00
4d2f380d78
YARN-5101. YARN_APPLICATION_UPDATED event is parsed in ApplicationHistoryManagerOnTimelineStore#convertToApplicationReport with reversed order. Contributed by Sunil G.
2016-10-06 20:42:36 +05:30
272a21747e
MAPREDUCE-6789. Fix TestAMWebApp failure. Contributed by Daniel Templeton.
2016-10-06 15:57:15 +09:00
0992708d79
YARN-5704. Provide config knobs to control enabling/disabling new/work in progress features in container-executor. Contributed by Sidharta Seethana.
2016-10-05 15:31:38 +05:30
31f8da22d0
YARN-3139. Improve locks in AbstractYarnScheduler/CapacityScheduler/FairScheduler. Contributed by Wangda Tan
2016-10-04 17:23:13 -07:00
736d33cddd
YARN-4767. Network issues can cause persistent RM UI outage. (Daniel Templeton via kasha)
2016-10-03 14:35:57 -07:00
0da54e8848
YARN-5672. FairScheduler: Wrong queue name in log when adding application. (Wilfred Spiegelenburg via kasha)
2016-10-03 06:03:46 -07:00
6e130c308c
YARN-4855. Should check if node exists when replace nodelabels. Contributeed by Tao Jie
2016-10-03 02:02:26 -04:00
82857037b6
YARN-5678. Log demand as demand in FSLeafQueue and FSParentQueue. (Yufei Gu via kasha)
2016-10-02 22:09:43 -07:00
3a3697deab
YARN-5384. Expose priority in ReservationSystem submission APIs. (Sean Po via Subru).
2016-09-30 19:41:43 -07:00
10be45986c
YARN-5486. Update OpportunisticContainerAllocatorAMService::allocate method to handle OPPORTUNISTIC container requests. (Konstantinos Karanasos via asuresh)
2016-09-29 15:11:41 -07:00
2ae5a3a5bf
YARN-4205. Add a service for monitoring application life time out. Contributed by Rohith Sharma K S
2016-09-29 22:00:31 +08:00
bcb2528a51
YARN-5400. Light cleanup in ZKRMStateStore (templedf via rkanter)
2016-09-28 14:56:41 -07:00
9b0fd01d2e
YARN-5599. Publish AM launch command to ATS (Rohith Sharma K S via Varun Saxena)
2016-09-28 16:10:10 +05:30
bc2656f09f
YARN-5662. Provide an option to enable ContainerMonitor. Contributed by Jian He.
2016-09-28 15:18:18 +05:30
1831be8e73
YARN-3142. Improve locks in AppSchedulingInfo. (Varun Saxena via wangda)
2016-09-27 11:54:55 -07:00
ebf528c443
YARN-5660. Wrong audit constants are used in Get/Put of priority in RMWebService. Contributed by Rohith Sharma K S
2016-09-27 16:37:59 +05:30
4815d024c5
YARN-5609. Expose upgrade and restart API in ContainerManagementProtocol. Contributed by Arun Suresh
2016-09-26 08:46:54 -07:00
2f163cd5cf
Revert "YARN-5609. Expose upgrade and restart API in ContainerManagementProtocol. Contributed by Arun Suresh"
...
This reverts commit fe644bafe7
2016-09-26 08:36:59 -07:00
fe644bafe7
YARN-5609. Expose upgrade and restart API in ContainerManagementProtocol. Contributed by Arun Suresh
2016-09-26 22:41:16 +08:00
14a696f369
YARN-5663. Small refactor in ZKRMStateStore. Contributed by Oleksii Dymytrov.
2016-09-26 15:00:01 +09:00
d0372dc613
YARN-3692. Allow REST API to set a user generated message when killing an application. Contributed by Rohith Sharma K S
2016-09-23 06:30:49 +05:30
9f03b403ec
YARN-5656. Fix ReservationACLsTestBase. (Sean Po via asuresh)
2016-09-20 12:27:17 -07:00
e45307c9a0
Addendum patch for fix javadocs failure which is caused by YARN-3141. (wangda)
2016-09-20 11:21:01 -07:00
c6d1d742e7
YARN-5655. TestContainerManagerSecurity#testNMTokens is asserting. Contributed by Robert Kanter
2016-09-20 14:15:06 +00:00
2b66d9ec5b
YARN-3140. Improve locks in AbstractCSQueue/LeafQueue/ParentQueue. Contributed by Wangda Tan
2016-09-20 15:03:31 +08:00
7558dbbb48
YARN-5540. Scheduler spends too much time looking at empty priorities. Contributed by Jason Lowe
2016-09-19 20:31:35 +00:00
b8a30f2f17
YARN-3141. Improve locks in SchedulerApplicationAttempt/FSAppAttempt/FiCaSchedulerApp. Contributed by Wangda Tan
2016-09-19 17:08:01 +08:00
3552c2b99d
YARN-5637. Changes in NodeManager to support Container rollback and commit. (asuresh)
2016-09-18 10:55:18 -07:00
f67237cbe7
YARN-5657. Fix TestDefaultContainerExecutor. (asuresh)
2016-09-17 09:32:05 -07:00
4174b9756c
YARN-5642. Typos in 9 log messages. Contributed by Mehran Hassani
2016-09-17 10:35:39 +05:30
58bae35447
YARN-5163. Migrate TestClientToAMTokens and TestClientRMTokens tests from the old RPC engine. Contributed by Wei Zhou and Kai Zheng
2016-09-18 08:43:36 +08:00
40b5a59b72
YARN-5620. Core changes in NodeManager to support re-initialization of Containers with new launchContext. (asuresh)
2016-09-15 07:15:11 -07:00
e793309735
YARN-5630. NM fails to start after downgrade from 2.8 to 2.7. Contributed by Jason Lowe
2016-09-13 14:41:27 +00:00
c0e492e50f
YARN-5616. Clean up WeightAdjuster. (Yufei Gu via kasha)
2016-09-06 22:40:20 -07:00
e6fcfe28e3
YARN-5576. Allow resource localization while container is running. Contributed by Jian He.
2016-09-06 20:01:45 +05:30
f6ea9be547
YARN-5264. Store all queue-specific information in FSQueue. (Yufei Gu via kasha)
2016-09-02 14:56:29 -07:00
378f624a39
YARN-5549. AMLauncher#createAMContainerLaunchContext() should not log the command to be launched indiscriminately. (Daniel Templeton via rchiang)
2016-09-02 11:14:35 -07:00
05f5c0f631
YARN-5555. Scheduler UI: "% of Queue" is inaccurate if leaf queue is hierarchically nested. Contributed by Eric Payne.
2016-09-02 16:02:01 +05:30
74f4bae455
YARN-5566. Client-side NM graceful decom is not triggered when jobs finish. (Robert Kanter via kasha)
2016-09-01 14:44:01 -07:00
08f55ccbb0
YARN-5596. Fix failing unit test in TestDockerContainerRuntime. Contributed by Sidharta Seethana.
2016-09-01 14:08:51 +05:30
d6d9cff21b
YARN-5221. Expose UpdateResourceRequest API to allow AM to request for change in container properties. (asuresh)
2016-08-30 15:52:29 -07:00
b930dc3ec0
YARN-5327. API changes required to support recurring reservations in the YARN ReservationSystem. (Sangeetha Abdu Jyothi via Subru).
2016-08-26 16:58:47 -07:00
cde3a00526
YARN-5373. NPE listing wildcard directory in containerLaunch. (Daniel Templeton via kasha)
2016-08-26 11:04:33 -07:00
9ef632f3b0
YARN-5557. Add localize API to the ContainerManagementProtocol. Contributed by Jian He.
2016-08-26 09:04:44 -07:00
46e02ab719
YARN-3940. Application moveToQueue should check NodeLabel permission. Contributed by Bibin A Chundatt
2016-08-26 20:19:11 +05:30
bfb4d95059
YARN-5430. Return container's ip and host from NM ContainerStatus call. Contributed by Jian He.
2016-08-26 16:30:18 +05:30
27c3b86252
YARN-5564. Fix typo in RM_SCHEDULER_RESERVATION_THRESHOLD_INCREMENT_MULTIPLE. Contributed by Ray Chiang
2016-08-26 08:47:21 +05:30
525d52bf7c
YARN-5042. Mount /sys/fs/cgroup into Docker containers as read only mount. Contributed by luhuichun.
2016-08-25 14:18:26 +05:30
0d5997d2b9
YARN-5544. TestNodeBlacklistingOnAMFailures fails on trunk. Contributed by Sunil G.
2016-08-23 14:37:39 +05:30
444b2ea7af
YARN-3388. Allocation in LeafQueue could get stuck because DRF calculator isn't well supported when computing user-limit. (Nathan Roberts via wangda)
2016-08-19 16:28:32 -07:00
091dd19e86
YARN-5533. JMX AM Used metrics for queue wrong when app submited to nodelabel partition (Bibin A Chundatt via Varun Saxena)
2016-08-19 17:30:17 +05:30
8aed374182
Revert "YARN-5533. JMX AM Used metrics for queue wrong when app submited to nodelabel partition (Bibin A Chundatt via Varun Saxena)"
...
This reverts commit 59557e85a4
2016-08-19 16:14:16 +05:30
59557e85a4
YARN-5533. JMX AM Used metrics for queue wrong when app submited to nodelabel partition (Bibin A Chundatt via Varun Saxena)
2016-08-19 15:01:48 +05:30
0da69c324d
YARN-4676. Automatic and Asynchronous Decommissioning Nodes Status Tracking. Contributed by Diniel Zhi.
...
(cherry picked from commit d464483bf7
2016-08-18 07:27:23 -07:00
20f0eb871c
YARN-4702. FairScheduler: Allow setting maxResources for ad hoc queues. (Daniel Templeton via kasha)
2016-08-17 17:40:20 -07:00
7f05ff7a4e
YARN-5455. Update Javadocs for LinuxContainerExecutor. Contributed by Daniel Templeton.
2016-08-17 15:34:58 +05:30
24249115bf
YARN-5521. Fix random failure of TestCapacityScheduler#testKillAllAppsInQueue (sandflee via Varun Saxena)
2016-08-16 00:03:29 +05:30
d677b68c25
YARN-5491. Fix random failure of TestCapacityScheduler#testCSQueueBlocked (Bibin A Chundatt via Varun Saxena)
2016-08-15 03:31:21 +05:30
23c6e3c4e4
YARN-5476. Non existent application reported as ACCEPTED by YarnClientImpl (Junping Du via Varun Saxena)
2016-08-12 20:37:58 +05:30
874577a67d
YARN-4833. For Queue AccessControlException client retries multiple times on both RM. Contributed by Bibin A Chundatt
2016-08-12 01:09:41 +05:30
5199db387d
YARN-5492. TestSubmitApplicationWithRMHA is failing sporadically during precommit builds. Contributed by Vrushali C.
2016-08-11 11:50:46 +05:30
aea3e65749
YARN-5137. Make DiskChecker pluggable in NodeManager. (Yufei Gu via rchiang)
2016-08-10 19:23:29 -07:00
89c0bff719
YARN-5495. Remove import wildcard in CapacityScheduler. Contributed by Ray Chiang.
2016-08-11 00:35:19 +05:30
5c95bb315b
YARN-5382. RM does not audit log kill request for active applications. Contributed by Vrushali C
2016-08-10 18:25:54 +00:00
e0b570dffb
YARN-5483. Optimize RMAppAttempt#pullJustFinishedContainers. Contributed by sandflee
2016-08-10 18:04:18 +00:00
7992c0b42c
YARN-5343. TestContinuousScheduling#testSortedNodes fails intermittently. (Yufei Gu via kasha)
2016-08-09 16:51:03 -07:00
522ddbde79
YARN-5394. Remove bind-mount /etc/passwd for Docker containers. Contributed by Zhankun Tang.
2016-08-09 16:04:09 +05:30
82c9e06101
YARN-5457. Refactor DistributedScheduling framework to pull out common functionality. (asuresh)
2016-08-09 00:42:29 -07:00
37d939a453
YARN-4910. Fix incomplete log info in ResourceLocalizationService (Jun Gong via Varun Saxena)
2016-08-08 22:12:53 +05:30
131d58a24e
YARN-5287. LinuxContainerExecutor fails to set proper permission. Contributed by Ying Zhang
2016-08-07 01:08:13 +05:30
d81b8163b4
YARN-4624. NPE in PartitionQueueCapacitiesInfo while accessing Schduler UI. Contributed by Brahma Reddy Battula
2016-08-06 01:13:36 +05:30
3f100d76ff
YARN-4888. Changes in scheduler to identify resource-requests explicitly by allocation-id. (Subru Krishnan via wangda)
2016-08-05 10:43:35 -07:00
e0d131f055
YARN-4091. Add REST API to retrieve scheduler activity. (Chen Ge and Sunil G via wangda)
2016-08-05 10:27:34 -07:00
d9a354c2f3
YARN-5333. Some recovered apps are put into default queue when RM HA. Contributed by Jun Gong.
2016-08-05 21:35:49 +05:30
08e33381f5
YARN-5459. Add support for docker rm. Contributed by Shane Kumpf.
2016-08-04 14:08:34 +05:30
db4a61dc61
YARN-5469. Increase timeout of TestAmFilter.testFilter. Contributed by Eric Badger
2016-08-03 19:51:44 +00:00
db646540f0
YARN-5462. TestNodeStatusUpdater.testNodeStatusUpdaterRetryAndNMShutdown fails intermittently. Contributed by Eric Badger
2016-08-03 19:17:25 +00:00
4d92aefd35
YARN-4280. CapacityScheduler reservations may not prevent indefinite postponement on a busy cluster. Contributed by Kuhu Shukla
2016-08-03 18:53:14 +00:00
d848184e90
YARN-5460. Change container runtime type logging in DelegatingLinuxContainerRuntime to debug. Contributed by Shane Kumpf.
2016-08-03 12:02:39 +05:30
b913677365
YARN-5456. container-executor support for FreeBSD, NetBSD, and others if conf path is absolute. Contributed by Allen Wittenauer.
2016-08-02 22:24:34 -07:00
7fc70c6422
YARN-5458. Rename DockerStopCommandTest to TestDockerStopCommand. Contributed by Shane Kumpf.
2016-08-02 16:56:45 +05:30
2e7c2a13a8
YARN-5443. Add support for docker inspect command. Contributed by Shane Kumpf.
2016-08-02 16:55:10 +05:30
95694b70cd
YARN-5444. Fix failing unit tests in TestLinuxContainerExecutorWithMocks. Contributed by Yufei Gu.
2016-08-01 16:08:38 +05:30
e5766b1dbe
YARN-5113. Refactoring and other clean-up for distributed scheduling. (Konstantinos Karanasos via asuresh)
2016-07-31 11:48:25 -07:00
ef501b1a0b
YARN-5121. fix some container-executor portability issues. Contributed by Allen Wittenauer.
2016-07-30 08:26:19 -07:00
4e756d7271
YARN-5203.Return ResourceRequest JAXB object in ResourceManager Cluster Applications REST API. Contributed by Ellen Hui.
2016-07-28 16:03:24 -07:00
7f3c306e2e
YARN-5432. Lock already held by another process while LevelDB cache store creation for dag. Contributed by Li Lu.
2016-07-28 06:35:24 -07:00
d2cbfd7de3
YARN-5441. Fixing minor Scheduler test case failures
2016-07-27 18:21:58 -07:00
54fe17a607
HADOOP-9427. Use JUnit assumptions to skip platform-specific tests. Contributed by Gergely Novák.
2016-07-27 19:41:09 +09:00
d62e121ffc
YARN-5195. RM intermittently crashed with NPE while handling APP_ATTEMPT_REMOVED event when async-scheduling enabled in CapacityScheduler. (sandflee via wangda)
2016-07-26 21:22:59 -07:00
49969b16cd
YARN-5342. Improve non-exclusive node partition resource allocation in Capacity Scheduler. (Sunil G via wangda)
2016-07-26 18:14:09 -07:00
5aace38b74
YARN-5392. Replace use of Priority in the Scheduling infrastructure with an opaque ShedulerRequestKey. (asuresh and subru)
2016-07-26 14:54:03 -07:00
d383bfdcd4
YARN-5164. Use plan RLE to improve CapacityOverTimePolicy efficiency
2016-07-25 16:37:50 -07:00
557a245d83
YARN-5092. TestRMDelegationTokens fails intermittently. Contributed by Jason Lowe.
2016-07-21 12:47:27 +05:30
c63afdbe14
YARN-4883. Make consistent operation name in AdminService. Contributed by Kai Sasaki.
2016-07-20 16:51:01 -07:00
1c9d2ab503
YARN-5340. Fixed a race condition in RollingLevelDBTimelineStore that caused loss of Timeline events. Contributed by Li Lu.
2016-07-20 08:36:36 -07:00
8fbe6ece24
YARN-5350. Distributed Scheduling: Ensure sort order of allocatable nodes returned by the RM is not lost. (asuresh)
2016-07-19 23:03:58 -07:00
cda0a280dd
YARN-5181. ClusterNodeTracker: add method to get list of nodes matching a specific resourceName. (kasha via asuresh)
2016-07-19 10:43:37 -07:00
5f2d33a551
Revert "YARN=5181. ClusterNodeTracker: add method to get list of nodes matching a specific resourceName. (kasha via asuresh)"
...
This reverts commit e905a42a2c
2016-07-19 10:43:19 -07:00
fe20494a72
YARN-4996. Make TestNMReconnect.testCompareRMNodeAfterReconnect() scheduler agnostic (Kai Sasaki via Varun Saxena)
2016-07-19 16:03:28 +05:30
da456ffd62
Preparing for 3.0.0-alpha2 development
2016-07-15 19:04:17 -07:00
f5f1c81e7d
YARN-5272. Handle queue names consistently in FairScheduler. (Wilfred Spiegelenburg via rchiang)
2016-07-15 14:38:50 -07:00
e905a42a2c
YARN=5181. ClusterNodeTracker: add method to get list of nodes matching a specific resourceName. (kasha via asuresh)
2016-07-15 14:35:12 -07:00
24db9167f1
YARN-4484. Available Resource calculation for a queue is not correct when used with labels. (Sunil G via wangda)
2016-07-15 11:40:12 -07:00
a72cb3825a
YARN-5383. Fix findbugs and checkstyle issues in ContainerExecutor. (Vrushali C via Varun Saxena)
2016-07-15 23:17:04 +05:30
b5ee7dbd8d
YARN-5380. NMTimelinePublisher should use getMemorySize instead of getMemory. Contributed by Vrushali C
2016-07-15 14:34:48 +05:30
e549a9af31
YARN-5379. TestHBaseTimelineStorage. testWriteApplicationToHBase() fails intermittently (Vrushali C via sjlee)
2016-07-14 15:19:03 -07:00
6cf017558a
YARN-5159. Wrong Javadoc tag in MiniYarnCluster. Contributed by Andras Bokor.
2016-07-14 10:45:11 -07:00
54bf14f80b
YARN-5303. Clean up ContainerExecutor JavaDoc. Contributed by Daniel Templeton.
2016-07-14 19:28:11 +05:30
e5e558b0a3
YARN-4759. Fix signal handling for docker containers. Contributed by Shane Kumpf.
2016-07-14 19:27:16 +05:30
58e1850801
YARN-5298. Mount usercache and NM filecache directories into Docker container. Contributed by Sidharta Seethana.
2016-07-14 16:11:07 +05:30
dbe97aa768
YARN-5299. Log Docker run command when container fails. Contributed by Varun Vasudev.
2016-07-14 11:48:10 +05:30
af8f480c24
YARN-5364. timelineservice modules have indirect dependencies on mapreduce artifacts. Contributed by Sangjin Lee
2016-07-14 01:43:37 +05:30
56142171b9
YARN-5359. FileSystemTimelineReader/Writer uses unix-specific default storage path (Sangjin Lee via Varun Saxena)
2016-07-13 21:15:41 +05:30
d6d41e820a
YARN-5362. TestRMRestart#testFinishedAppRemovalAfterRMRestart can fail. Contributed by sandflee.
2016-07-13 19:12:35 +05:30
06c56ff79b
YARN-5353. ResourceManager can leak delegation tokens when they are shared across apps. (Jason Lowe via Varun Saxena).
2016-07-13 07:55:34 +05:30
10b704c594
YARN-5317. testAMRestartNotLostContainerCompleteMsg may fail. Contributed by sandflee
2016-07-12 20:27:41 +00:00
819224dcf9
YARN-5270. Solve miscellaneous issues caused by YARN-4844. Contributed by Wangda Tan
2016-07-11 22:36:20 -07:00
0fd3980a1f
YARN-5037. Fix random failure of TestRMRestart#testQueueMetricsOnRMRestart (sandflee via Varun Saxena).
2016-07-10 21:28:52 +05:30
6cf6ab7b78
Made a number of miscellaneous fixes for javac, javadoc, and checstyle warnings.
2016-07-10 08:46:05 -07:00
c5dbde0cc4
Fixed the build after the rebase with trunk.
2016-07-10 08:46:05 -07:00
6d943038f6
Cleanup changes during rebase with trunk (Vrushali C)
2016-07-10 08:46:04 -07:00
1a227744ac
YARN-5070. upgrade HBase version for first merge (Vrushali C via sjlee)
2016-07-10 08:46:04 -07:00
1ff6833bba
YARN-5243. fix several rebase and other miscellaneous issues before merge. (Sangjin Lee via Varun Saxena)
2016-07-10 08:46:03 -07:00
892b193bd7
YARN-5170. Eliminate singleton converters and static method access. (Joep Rottinghuis via Varun Saxena)
2016-07-10 08:46:03 -07:00
c65020075e
YARN-5167. Escape occurences of encodedValues. (Sangjin Lee via Varun Saxena)
2016-07-10 08:46:02 -07:00
9c926cf432
YARN-5142. fix findbugs warnings/errors for hadoop-yarn-server-timelineservice-hbase-tests. (Vrushali C via Varun Saxena)
2016-07-10 08:46:02 -07:00
0a9b085f05
YARN-5189. Make HBaseTimeline[Reader|Writer]Impl default and move FileSystemTimeline*Impl. (Joep Rottinghuis and Sangjin Lee via gtcarrera9)
2016-07-10 08:46:01 -07:00
c81a2e1d19
YARN-5015. entire time series is returned for YARN container system metrics (CPU and memory) (Varun Saxena via sjlee)
2016-07-10 08:46:01 -07:00
7b8cfa5c2f
YARN-5109. timestamps are stored unencoded causing parse errors (Varun Saxena via sjlee)
2016-07-10 08:46:01 -07:00
d729e8211b
YARN-5138. fix "no findbugs output file" error for hadoop-yarn-server-timelineservice-hbase-tests. (Vrushali C via gtcarrera9)
2016-07-10 08:46:00 -07:00
702236129b
YARN-5095. flow activities and flow runs are populated with wrong timestamp when RM restarts w/ recovery enabled (Varun Saxena via sjlee)
2016-07-10 08:46:00 -07:00
831a3ffd6e
YARN-5097. NPE in Separator.joinEncoded() (Vrushali C via sjlee)
2016-07-10 08:46:00 -07:00
a1b6d7456f
YARN-5018. Online aggregation logic should not run immediately after collectors got started (Li Lu via sjlee)
2016-07-10 08:45:59 -07:00
8c8183e515
YARN-5093. created time shows 0 in most REST output (Varun Saxena via sjlee)
2016-07-10 08:45:59 -07:00
4a7011a45f
YARN-5096 addendum. Turned another logging statement to debug. Contributed by Sangjin Lee.
2016-07-10 08:45:59 -07:00
bcdb382600
YARN-5096. timelinereader has a lot of logging that's not useful. (Sangjin Lee via Varun Saxena)
2016-07-10 08:45:58 -07:00
bc6f978cd3
YARN-5045. hbase unit tests fail due to dependency issues. (Sangjin Lee via varunsaxena)
2016-07-10 08:45:58 -07:00
089caf49fe
YARN-4447. Provide a mechanism to represent complex filters and parse them at the REST layer (Varun Saxena via sjlee)
2016-07-10 08:45:57 -07:00
c2055a97d5
YARN-3150. Documenting the timeline service v2. (Sangjin Lee and Vrushali C via gtcarrera9)
2016-07-10 08:45:57 -07:00
69dc561b61
YARN-4986. Add a check in the coprocessor for table to operated on (Vrushali C via sjlee)
2016-07-10 08:45:56 -07:00
39cce4e629
YARN-3816. [Aggregation] App-level aggregation and accumulation for YARN system metrics (Li Lu via sjlee)
2016-07-10 08:45:56 -07:00
c2efdc415a
YARN-3863. Support complex filters in TimelineReader (Varun Saxena via sjlee)
2016-07-10 08:45:55 -07:00
a3cf40e532
YARN-3461. Consolidate flow name/version/run defaults. (Sangjin Lee via Varun Saxena)
2016-07-10 08:45:55 -07:00
84c35ac6c4
YARN-4711. NM is going down with NPE's due to single thread processing of events by Timeline client (Naganarasimha G R via sjlee)
2016-07-10 08:45:55 -07:00
6f6cc647d6
YARN-4712. CPU Usage Metric is not captured properly in YARN-2928. (Naganarasimha G R via Varun Saxena)
2016-07-10 08:45:54 -07:00
9bdd455dce
YARN-4062. Add the flush and compaction functionality via coprocessors and scanners for flow run table (Vrushali C via sjlee)
2016-07-10 08:45:54 -07:00
d56dde490b
YARN-4700. ATS storage has one extra record each time the RM got restarted. (Naganarasimha G R via Varun Saxena)
2016-07-10 08:45:53 -07:00
0d02ab8729
YARN-3367. Replace starting a separate thread for post entity with event loop in TimelineClient (Naganarasimha G R via sjlee)
2016-07-10 08:45:53 -07:00
960af7d471
YARN-4409. Fix javadoc and checkstyle issues in timelineservice code (Varun Saxena via sjlee)
2016-07-10 08:45:53 -07:00
9cb1287e9b
YARN-4446. Refactor reader API for better extensibility (Varun Saxena via sjlee)
2016-07-10 08:45:52 -07:00
06f0b50a28
YARN-4644. TestRMRestart fails and findbugs issue in YARN-2928 branch (Varun Saxena via Naganarasimha G R)
2016-07-10 08:45:52 -07:00
9d40d9d34c
YARN-4224. Support fetching entities by UID and change the REST
...
interface to conform to current REST APIs' in YARN. (Varun Saxena via
gtcarrera9)
2016-07-10 08:45:52 -07:00
6934b05c71
YARN-4238. createdTime and modifiedTime is not reported while publishing entities to ATSv2. (Varun Saxena via Naganarasimha G R)
2016-07-10 08:45:52 -07:00
34f02f07d5
Rebase to latest trunk
2016-07-10 08:45:51 -07:00
88f0294114
YARN-4200. Refactor reader classes in storage to nest under hbase
...
specific package name. Contributed by Li Lu.
2016-07-10 08:45:51 -07:00
cc16683cef
YARN-3995. Some of the NM events are not getting published due race condition when AM container finishes in NM (Naganarasimha G R via sjlee)
2016-07-10 08:45:50 -07:00
829cceebc0
YARN-3586. RM to only get back addresses of Collectors that NM needs to know.
...
(Junping Du via Varun Saxena).
2016-07-10 08:45:50 -07:00
eb0ac8efb1
YARN-4350. TestDistributedShell fails for V2 scenarios. (Naganarasimha G R via Varun Saxena)
2016-07-10 08:45:50 -07:00
8ef546c1ee
YARN-4445. Unify the term flowId and flowName in timeline v2 codebase.
...
Contributed by Zhan Zhang.
2016-07-10 08:45:49 -07:00
c4d7bbda5c
YARN-4460. [Bug fix] RM fails to start when SMP is enabled. (Li Lu via Varun Saxena)
2016-07-10 08:45:49 -07:00
2e2dbf59d1
YARN-4392. ApplicationCreatedEvent event time resets after RM
...
restart/failover. Contributed by Naganarasimha G R and Xuan Gong
(cherry picked from commit 4546c7582b
2016-07-10 08:45:49 -07:00
89e5c44f9e
YARN-4356. Ensure the timeline service v.2 is disabled cleanly and has no
...
impact when it's turned off. Contributed by Sangjin Lee.
2016-07-10 08:45:48 -07:00
1f710484e5
YARN-3862. Support for fetching specific configs and metrics based on prefixes (Varun Saxena via sjlee)
2016-07-10 08:45:48 -07:00
ae72f1dc77
YARN-4297. TestJobHistoryEventHandler and TestRMContainerAllocator failing on YARN-2928 branch (Varun Saxena via sjlee)
2016-07-10 08:45:47 -07:00
51254a6b51
YARN-4053. Change the way metric values are stored in HBase Storage (Varun Saxena via sjlee)
2016-07-10 08:45:47 -07:00
09649005ca
YARN-4221. Store user in app to flow table (Varun Saxena via sjlee)
2016-07-10 08:45:47 -07:00
10ec5586fb
YARN-4129. Refactor the SystemMetricPublisher in RM to better support newer events (Naganarasimha G R via sjlee)
2016-07-10 08:45:46 -07:00
e3e857866d
YARN-4179. [reader implementation] support flow activity queries based on time (Varun Saxena via sjlee)
2016-07-10 08:45:46 -07:00
d014f2ffd2
YARN-4237 Support additional queries for ATSv2 Web UI. Contributed by
...
Varun Saxena.
2016-07-10 08:45:46 -07:00
b51d0fef56
YARN-4178. [storage implementation] app id as string in row keys can cause incorrect ordering (Varun Saxena via sjlee)
2016-07-10 08:45:45 -07:00
0f44b5508d
YARN-3864. Implement support for querying single app and all apps for a flow run (Varun Saxena via sjlee)
2016-07-10 08:45:45 -07:00
708fa8b1ae
YARN-4210. HBase reader throws NPE if Get returns no rows (Varun Saxena via vrushali)
2016-07-10 08:45:45 -07:00
da2b7bd08e
YARN-4203. Add request/response logging & timing for each REST endpoint call (Varun Saxena via vrushali)
2016-07-10 08:45:45 -07:00
d95dc89a02
YARN-4075 [reader REST API] implement support for querying for flows and flow runs (Varun Saxena via vrushali)
2016-07-10 08:45:44 -07:00
10fa6da7d8
YARN-4074. [timeline reader] implement support for querying for flows and flow runs (sjlee via vrushali)
2016-07-10 08:45:44 -07:00
a68e383921
YARN-3901. Populate flow run data in the flow_run & flow activity tables (Vrushali C via sjlee)
2016-07-10 08:45:44 -07:00
ee081018e4
YARN-4102. Add a "skip existing table" mode for timeline schema creator (Li Lu via sjlee)
2016-07-10 08:45:43 -07:00
8d9476ec5f
YARN-4058. Miscellaneous issues in NodeManager project (Naganarasimha G R via sjlee)
2016-07-10 08:45:43 -07:00
451c13b1de
YARN-3814. REST API implementation for getting raw entities in TimelineReader (Varun Saxena via sjlee)
2016-07-10 08:45:43 -07:00
c7224a285c
YARN-4064. build is broken at TestHBaseTimelineWriterImpl.java (sjlee)
2016-07-10 08:45:43 -07:00
7a41b5501e
YARN-4025. Deal with byte representations of Longs in writer code. Contributed by Sangjin Lee and Vrushali C.
2016-07-10 08:45:42 -07:00
477a30f536
YARN-3045. Implement NM writing container lifecycle events to Timeline Service v2. Contributed by Naganarasimha G R.
2016-07-10 08:45:42 -07:00
102b56ee96
YARN-3904. Refactor timelineservice.storage to add support to online and offline aggregation writers (Li Lu via sjlee)
2016-07-10 08:45:42 -07:00
5088f6c76a
MAPREDUCE-6337. Added a mode to replay MR job history files and put them into the timeline service v2. Contributed by Sangjin Lee.
...
(cherry picked from commit 463e070a8e7c882706a96eaa20ea49bfe9982875)
2016-07-10 08:45:41 -07:00
00e85e7a2b
YARN-3906. Split the application table from the entity table. Contributed by Sangjin Lee.
...
(cherry picked from commit bcd755eba9466ce277d3c14192c31da6462c4ab3)
2016-07-10 08:45:41 -07:00
9e5155be36
YARN-3049. [Storage Implementation] Implement storage reader interface to fetch raw data from HBase backend (Zhijie Shen via sjlee)
...
(cherry picked from commit 07433c2ad52df9e844dbd90020c277d3df844dcd)
2016-07-10 08:45:41 -07:00
9422d9b50d
YARN-3984. Adjusted the event column key schema and avoided missing empty event. Contributed by Vrushali C.
...
(cherry picked from commit 895ccfa1ab9e701f2908586e323249f670fe5544)
2016-07-10 08:45:40 -07:00
57e2498cd4
YARN-3993. Changed to use the AM flag in ContainerContext determine AM container in TestPerNodeTimelineCollectorsAuxService. Contributed by Sunil G.
...
(cherry picked from commit 9e48f9ff2ce08f3dcdd8d60bacb697664b92196f)
2016-07-10 08:45:40 -07:00
a9fab9b644
YARN-3908. Fixed bugs in HBaseTimelineWriterImpl. Contributed by Vrushali C and Sangjin Lee.
...
(cherry picked from commit df0ec473a84871b0effd7ca6faac776210d7df09)
2016-07-10 08:45:40 -07:00
8603736ef2
YARN-3949. Ensure timely flush of timeline writes. Contributed by Sangjin Lee.
...
(cherry picked from commit 967bef7e0396d857913caa2574afb103a5f0b81b)
2016-07-10 08:45:39 -07:00
e27642abf4
YARN-3047. [Data Serving] Set up ATS reader with basic request serving structure and lifecycle (Varun Saxena via sjlee)
...
(cherry picked from commit 4c5f88fb0f04b7919738d07598b0f006a9ff91f2)
2016-07-10 08:45:39 -07:00
2d59bc4458
YARN-3051. Created storage oriented reader interface for fetching raw entity data and made the filesystem based implementation. Contributed by Varun Saxena.
...
(cherry picked from commit 499ce52c7b645ec0b1cc8ac62dc9a3127b987a20)
2016-07-10 08:45:38 -07:00
22e7ae5771
YARN-3792. Test case failures in TestDistributedShell and some issue fixes related to ATSV2 (Naganarasimha G R via sjlee)
...
(cherry picked from commit 84f37f1c7eefec6d139cbf091c50d6c06f734323)
2016-07-10 08:45:38 -07:00
92d90c3a24
YARN-3706. Generalize native HBase writer for additional tables (Joep Rottinghuis via sjlee)
...
(cherry picked from commit 9137aeae0dec83f9eff40d12cae712dfd508c0c5)
2016-07-10 08:45:38 -07:00
f3c661e8dd
YARN-3044. Made RM write app, attempt and optional container lifecycle events to timeline service v2. Contributed by Naganarasimha G R.
2016-07-10 08:45:37 -07:00
f668212529
YARN-3721. build is broken on YARN-2928 branch due to possible dependency cycle (Li Lu via sjlee)
...
(cherry picked from commit a9738ceb17b50cce8844fd42bb800c7f83f15caf)
2016-07-10 08:45:37 -07:00
db891f51d0
YARN-3726. Fix TestHBaseTimelineWriterImpl unit test failure by fixing its test data (Vrushali C via sjlee)
...
(cherry picked from commit 4c0b6d73914f2e249795deb292f508177ea54884)
2016-07-10 08:45:36 -07:00
5a4278ccbd
YARN-3411. [Storage implementation] explore the native HBase write schema for storage (Vrushali C via sjlee)
...
(cherry picked from commit 7a3068854d27eadae1c57545988f5b2029bf119a)
2016-07-10 08:45:36 -07:00
d275677e24
YARN-3634. TestMRTimelineEventHandling and TestApplication are broken. Contributed by Sangjin Lee.
...
(cherry picked from commit b059dd4882fd759e4762cc11c019be4b68fb74c1)
2016-07-10 08:45:36 -07:00
51d092faef
YARN-3529. Added mini HBase cluster and Phoenix support to timeline service v2 unit tests. Contributed by Li Lu.
2016-07-10 08:45:36 -07:00
41fb5c7381
YARN-3134. Implemented Phoenix timeline writer to access HBase backend. Contributed by Li Lu.
...
(cherry picked from commit b3b791be466be79e4e964ad068f7a6ec701e22e1)
2016-07-10 08:45:35 -07:00
dc1f306fdc
YARN-3562. unit tests failures and issues found from findbug from earlier ATS checkins (Naganarasimha G R via sjlee)
2016-07-10 08:45:35 -07:00
fc8485d893
YARN-3551. Consolidate data model change according to the backend implementation (Zhijie Shen via sale)
...
(cherry picked from commit 557a3950bddc837469244835f5577899080115d8)
2016-07-10 08:45:35 -07:00
2bdefbc4a0
YARN-3431. Sub resources of timeline entity needs to be passed to a separate endpoint. Contributed By Zhijie Shen.
...
(cherry picked from commit fa5cc75245a6dba549620a8b26c7b4a8aed9838e)
2016-07-10 08:45:34 -07:00
11e8905d8d
YARN-3390. Reuse TimelineCollectorManager for RM (Zhijie Shen via sjlee)
...
(cherry picked from commit 58221188811e0f61d842dac89e1f4ad4fd8aa182)
2016-07-10 08:45:33 -07:00
47f35a30bb
YARN-3391. Clearly define flow ID/ flow run / flow version in API and storage. Contributed by Zhijie Shen
...
(cherry picked from commit 68c6232f8423e55b4d152ef3d1d66aeb2d6a555e)
2016-07-10 08:45:33 -07:00
5712b8f9fd
YARN-3334. NM uses timeline client to publish container metrics to new timeline service. Contributed by Junping Du.
2016-07-10 08:45:33 -07:00
42e49399ce
YARN-3374. Collector's web server should randomly bind an available port. Contributed by Zhijie Shen
...
(cherry picked from commit 3aa898e734a1e4368ddf1d0bbd31f9b4de53ceba)
2016-07-10 08:45:32 -07:00
d67c9bdb4d
YARN-3040. Make putEntities operation be aware of the app's context. Contributed by Zhijie Shen
2016-07-10 08:45:32 -07:00
5e3d9a477b
YARN-3034. Implement RM starting its timeline collector. Contributed by Naganarasimha G R
2016-07-10 08:45:32 -07:00
19c1132146
YARN-3377. Fixed test failure in TestTimelineServiceClientIntegration. Contributed by Sangjin Lee.
...
(cherry picked from commit 04de2ceccd02edb00bd671a63e04855c132e9735)
2016-07-10 08:45:31 -07:00
2188a07e5b
YARN-3333. Rename TimelineAggregator etc. to TimelineCollector. Contributed by Sangjin Lee
2016-07-10 08:45:31 -07:00
9b56364080
YARN-3039. Implemented the app-level timeline aggregator discovery service. Contributed by Junping Du.
2016-07-10 08:45:31 -07:00
f0e752c14b
YARN-3264. Created backing storage write interface and a POC only FS based storage implementation. Contributed by Vrushali C.
...
(cherry picked from commit 821b68d05d246fd57d7b7286eb2ccc075ed1eae8)
2016-07-10 08:45:30 -07:00
bf54d32750
YARN-3210. Refactored timeline aggregator according to new code organization proposed in YARN-3166. Contributed by Li Lu.
...
(cherry picked from commit d3ff7f06cbc66d3a23c2551e7d4c752689f46afe)
2016-07-10 08:45:30 -07:00
d45ff878c4
YARN-3125. Made the distributed shell use timeline service next gen and add an integration test for it. Contributed by Junping Du and Li Lu.
...
(cherry picked from commit bf08f7f0ed4900ce52f98137297dd1a47ba2a536)
2016-07-10 08:45:30 -07:00
9d57c9c015
YARN-3087. Made the REST server of per-node aggregator work alone in NM daemon. Conntributed by Li Lu.
...
(cherry picked from commit 41a08ad404d4278fe598d6c222b2ae0e84bae0df)
2016-07-10 08:45:29 -07:00
4f0c7eaff3
YARN-3240. Implement client API to put generic entities. Contributed by Zhijie Shen
...
(cherry picked from commit 4487da249f448d5c67b712cd0aa723e764eed77d)
2016-07-10 08:45:29 -07:00
7c8abec0a8
YARN-3030. Set up TS aggregator with basic request serving structure and lifecycle. Contributed by Sangjin Lee.
...
(cherry picked from commit f26941b39028ac30c77547e2be2d657bb5bf044a)
2016-07-10 08:45:28 -07:00
0d2da08f35
YARN-3063. Bootstrapping TimelineServer next generation module. Contributed by Zhijie Shen.
...
(cherry picked from commit fe7d0e7c8fc88e0721b18e1531b0899d9dbed296)
2016-07-10 08:45:28 -07:00
c04c5ec501
YARN-5318. Fix intermittent test failure of TestRMAdminService#testRefreshNodesResourceWithFileSystemBasedConfigurationProvider. (Jun Gong via Varun Saxena).
2016-07-09 01:13:18 +05:30
5252562edf
YARN-5297. Avoid printing a stack trace when recovering an app after the RM restarts. (Junping Du via Varun Saxena).
2016-07-09 00:09:25 +05:30
30ee57ceb1
YARN-4939. The decommissioning Node should keep alive during NM restart. Contributed by sandflee.
2016-07-08 04:14:53 -07:00
673e5e02fe
YARN-5314. Fixed a ConcurrentModificationException in ATS v1.5 EntityGroupFSTimelineStore. Contributed by Li Lu.
2016-07-07 22:37:29 -07:00
8a9d293dd6
YARN-5233. Support for specifying a path for ATS plugin jars. Contributed by Li Lu
2016-07-06 16:44:46 -07:00
4c9e1aeb94
YARN-5224. Added new web-services /containers/{containerid}/logs & /containers/{containerid}/logs/{filename} and using them in "yarn logs" CLI to get logs of finished containers of a running application. Contributed by Xuan Gong.
2016-07-06 14:10:54 -07:00
04f6ebb66a
YARN-5294. Pass remote ip address down to YarnAuthorizationProvider. (Jian He via wangda)
2016-07-06 10:36:48 -07:00
d792a90206
YARN-5296. NMs going OutOfMemory because ContainerMetrics leak in ContainerMonitorImpl. Contributed by Junping Du
2016-07-05 18:05:16 -07:00
ce9c006430
YARN-5214. Fixed locking in DirectoryCollection to avoid hanging NMs when various code-paths hit slow disks. Contributed by Junping Du.
2016-07-05 16:07:28 -07:00
8e672e3c71
YARN-5286. Add RPC port info in RM web service's response when getting app status. (Jun Gong via Varun Saxena).
2016-07-05 22:56:07 +05:30
0a5def155e
YARN-4568. Fix message when NodeManager runs into errors initializing the recovery directory. (rchiang)
2016-07-01 15:53:46 -07:00
c35a5a7a8d
YARN-5023. TestAMRestart#testShouldNotCountFailureToMaxAttemptRetry fails. Contributed by sandflee
2016-07-01 14:29:03 -07:00
abe7fc22c1
YARN-5182. MockNodes.newNodes creates one more node per rack than requested. (Karthik Kambatla via Varun Saxena).
2016-06-30 00:13:28 +05:30
26b5e6116f
YARN-5262. Optimize sending RMNodeFinishedContainersPulledByAMEvent for every AM heartbeat.
2016-06-29 10:08:30 +05:30
a8a48c9125
YARN-5278. Remove unused argument in TestRMWebServicesForCSWithPartitions#setupQueueConfiguration. Contributed by Tao Jie.
2016-06-23 14:28:12 +09:00
99e5dd68d0
YARN-5171. Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node. (Inigo Goiri via asuresh)
2016-06-22 19:04:54 -07:00
5d58858bb6
HADOOP-9613. [JDK8] Update jersey version to latest 1.x release.
2016-06-21 08:05:32 +09:00
5107a967fa
YARN-4958. The file localization process should allow for wildcards to reduce the application footprint in the state store (Daniel Templeton via sjlee)
2016-06-20 09:56:53 -07:00
d0162f2040
YARN-5246. NMWebAppFilter web redirects drop query parameters. Contributed by Varun Vasudev.
2016-06-19 17:44:54 -07:00
20f2799938
YARN-5077. Fix FSLeafQueue#getFairShare() for queues with zero fairshare. (Yufei Gu via kasha)
2016-06-17 22:24:42 -07:00
fbbe0bb627
YARN-5082. Limit ContainerId increase in fair scheduler if the num of node app reserved reached the limit. Addendum to fix javac warning. (Arun Suresh via kasha)
2016-06-17 22:12:50 -07:00
c35fa4a0e5
YARN-5251. Yarn CLI to obtain App logs for last 'n' bytes fails. Contributed by Xuan Gong.
2016-06-17 08:24:24 -07:00
5dfc38ff57
YARN-5237. Fix missing log files issue in rolling log aggregation. Contributed by Xuan Gong.
2016-06-15 16:17:54 -07:00
c77a1095dc
YARN-1942. Deprecate toString/fromString methods from ConverterUtils and move them to records classes like ContainerId/ApplicationId, etc. (wangda)
2016-06-14 15:06:38 -07:00
28b66ae919
YARN-4989. TestWorkPreservingRMRestart#testCapacitySchedulerRecovery fails intermittently. Contributed by Ajith S.
2016-06-13 11:09:32 +05:30
7dae2b3bc4
YARN-5212. Run existing ContainerManager tests using QueuingContainerManagerImpl. (Konstantinos Karanasos via asuresh)
2016-06-12 10:05:37 -07:00
5279af7cd4
YARN-5082. Limit ContainerId increase in fair scheduler if the num of node app reserved reached the limit (sandflee via asuresh)
2016-06-10 22:33:42 -07:00
e0f4620cc7
YARN-5197. RM leaks containers if running container disappears from node update. Contributed by Jason Lowe.
2016-06-11 10:22:27 +05:30
03fc6b1bb0
YARN-3426. Add jdiff support to YARN. (vinodkv via wangda)
2016-06-10 09:51:09 -07:00
244506f9c8
YARN-5208. Run TestAMRMClient TestNMClient TestYarnClient TestClientRMTokens TestAMAuthorization tests with hadoop.security.token.service.use_ip enabled. (Rohith Sharma K S via wangda)
2016-06-10 09:34:32 -07:00
9378d9428f
YARN-5191. Renamed the newly added “download=true” option for getting logs via NMWebServices and AHSWebServices to be a better "format" option. (Xuan Gong via vinodkv)
2016-06-09 12:30:58 -07:00
1500a0a300
YARN-4308. ContainersAggregated CPU resource utilization reports negative usage in first few heartbeats. Contributed by Sunil G
2016-06-09 05:41:09 +05:30
3344ba70e0
YARN-5204. Properly report status of killed/stopped queued containers. (Konstantinos Karanasos via asuresh)
2016-06-08 08:31:32 -07:00
76f0800c21
YARN-5176. More test cases for queuing of containers at the NM. (Konstantinos Karanasos via asuresh)
2016-06-07 17:16:18 -07:00
58be55b6e0
YARN-5199. Close LogReader in in AHSWebServices#getStreamingOutput and
...
FileInputStream in NMWebServices#getLogs. Contributed by Xuan Gong
2016-06-07 16:07:02 -07:00
620325e816
YARN-4837. User facing aspects of 'AM blacklisting' feature need fixing. (vinodkv via wangda)
2016-06-07 15:06:42 -07:00
bddea5fe5f
YARN-5118. Tests fails with localizer port bind exception. Contributed by Brahma Reddy Battula.
2016-06-07 11:20:15 +05:30
3a154f75ed
YARN-4525. Fix bug in RLESparseResourceAllocation.getRangeOverlapping(). (Ishai Menache and Carlo Curino via asuresh)
2016-06-06 21:18:32 -07:00
7a9b7372a1
YARN-5185. StageAllocaterGreedyRLE: Fix NPE in corner case. (Carlo Curino via asuresh)
2016-06-06 21:06:52 -07:00
4a1cedc010
MAPREDUCE-5044. Have AM trigger jstack on task attempts that timeout before killing them. (Eric Payne and Gera Shegalov via mingma)
2016-06-06 14:30:51 -07:00
db54670e83
YARN-5165. Fix NoOvercommitPolicy to take advantage of RLE representation of plan. (Carlo Curino via asuresh)
2016-06-03 14:49:32 -07:00
f10ebc67f5
YARN-5098. Fixed ResourceManager's DelegationTokenRenewer to replace expiring system-tokens if RM stops and only restarts after a long time. Contributed by Jian He.
2016-06-03 13:00:07 -07:00
99cc439e29
YARN-5190. Registering/unregistering container metrics in ContainerMonitorImpl and ContainerImpl causing uncaught exception in ContainerMonitorImpl. Contributed by Junping Du
2016-06-03 11:10:42 -07:00
097baaaeba
YARN-1815. Work preserving recovery of Unmanged AMs. Contributed by Subru Krishnan
2016-06-03 10:49:30 -07:00
dc26601d8f
YARN-5180. Allow ResourceRequest to specify an enforceExecutionType flag. (asuresh)
2016-06-02 09:01:02 -07:00
0bc05e40fa
YARN-5088. Improve "yarn log" command-line to read the last K bytes for the log files. Contributed by Xuan Gong
2016-06-01 13:44:21 -07:00
86fb58b7dc
YARN-4007. Add support for different network setups when launching the docker container. Contributed by Sidharta Seethana.
2016-05-29 21:32:51 +05:30
42f90ab885
YARN-4844. Add getMemorySize/getVirtualCoresSize to o.a.h.y.api.records.Resource. Contributed by Wangda Tan.
2016-05-29 21:24:16 +05:30
4fc09a897b
YARN-5117. QueuingContainerManager does not start GUARANTEED Container even if Resources are available. (Konstantinos Karanasos via asuresh)
2016-05-27 17:49:15 -07:00
aa975bc781
YARN-5127. Expose ExecutionType in Container api record. (Hitesh Sharma via asuresh)
2016-05-27 14:06:32 -07:00
705286ccae
YARN-4987. Read cache concurrency issue between read and evict in EntityGroupFS timeline store. Contributed by Li Lu.
2016-05-27 06:58:32 -07:00
916140604f
HADOOP-12911. Upgrade Hadoop MiniKDC with Kerby. Contributed by Jiajia Li
2016-05-28 14:23:39 +08:00
0a544f8a3e
YARN-5005. TestRMWebServices#testDumpingSchedulerLogs fails randomly. Contributed by Bibin A Chundatt.
2016-05-27 10:44:35 +05:30
5b41b288d0
YARN-5162. Fix Exceptions thrown during in registerAM call when Distributed Scheduling is Enabled (Hitesh Sharma via asuresh)
2016-05-26 14:56:37 -07:00
04ded558b0
YARN-5035. FairScheduler: Adjust maxAssign dynamically when assignMultiple is turned on. (kasha)
2016-05-26 14:41:07 -07:00
4f513a4a8e
YARN-4866. FairScheduler: AMs can consume all vcores leading to a livelock when using FAIR policy. (Yufei Gu via kasha)
2016-05-25 22:13:27 -07:00
013532a95e
YARN-4957. Add getNewReservation in ApplicationClientProtocol (Sean Po via curino)
2016-05-25 16:55:49 -07:00
1ba31fe9e9
YARN-4459. container-executor should only kill process groups. Contributed by Jun Gong
2016-05-25 21:24:55 +00:00
e07519b8db
YARN-4766. NM should not aggregate logs older than the retention policy (haibochen via rkanter)
2016-05-25 10:25:44 -07:00
9a31e5dfef
YARN-5115. Avoid setting CONTENT-DISPOSITION header in the container-logs web-service. Contributed by Xuan Gong.
2016-05-25 19:01:12 +05:30
c380a22031
YARN-857. Localization failures should be available in container diagnostics. Contributed by Vinod Kumar Vavilapalli.
2016-05-25 19:01:12 +05:30
28bd63e92b
YARN-5024. TestContainerResourceUsage#testUsageAfterAMRestartWithMultipleContainers random failure. Contributed by Bibin A Chundatt
2016-05-25 10:15:50 +05:30
edd716e99c
YARN-5114. Add additional tests in TestRMWebServicesApps and rectify testInvalidAppAttempts failure in 2.8. Contributed by Bibin A Chundatt
2016-05-25 06:11:38 +08:00
f979d779e1
YARN-4878. Expose scheduling policy and max running apps over JMX for Yarn queues. (Yufei Gu via kasha)
2016-05-24 10:54:11 -07:00
b4078bd17b
YARN-3971. Skip RMNodeLabelsManager#checkRemoveFromClusterNodeLabelsOfQueue on nodelabel recovery. (addendum patch). Contributed by Bibin A chundatt
2016-05-24 08:06:53 +08:00
6d043aa4cf
YARN-4979. FSAppAttempt demand calculation considers demands at multiple locality levels different. (Zhihai Xu via kasha)
2016-05-23 14:29:28 -07:00
ac954486c5
YARN-5055. max apps per user can be larger than max per queue. Contributed by Eric Badger
2016-05-23 15:54:42 +00:00
d1df0266cf
YARN-5103. With NM recovery enabled, restarting NM multiple times results in AM restart. Contributed by Junping Du
2016-05-23 15:15:27 +00:00
6161d9ba52
YARN-5112. Excessive log warnings for directory permission issue on NM recovery. Contributed by Jian He.
...
(cherry picked from commit 867cd2f5a2
2016-05-22 16:04:02 -07:00
0287c49107
YARN-5016. Add support for a minimum retry interval for container retries. Contributed by Jun Gong.
2016-05-20 12:28:14 +05:30
22fcd819f0
YARN-5076. YARN web interfaces lack XFS protection. Contributed by Jonathan Maron.
...
(cherry picked from commit 2703ec6871
2016-05-19 14:15:21 -07:00
feb90ffcca
YARN-4002. Make ResourceTrackerService#nodeHeartbeat more concurrent. Contributed by Rohith Sharma K S & Zhiguo Hong
2016-05-19 13:01:36 -07:00
141873ca7d
YARN-5100. The YarnApplicationState is always running in ATS even application is finished. Contributed by Xuan Gong.
2016-05-19 09:13:29 -07:00
22ff9e68d1
HADOOP-12767. Update apache httpclient version to 4.5.2; httpcore to 4.4.4. Artem Aliev via stevel.
2016-05-19 14:48:18 +01:00
d4274c64bc
YARN-5107. TestContainerMetrics fails. (aajisaka)
2016-05-19 19:53:59 +09:00
1597630681
YARN-5110. Fix OpportunisticContainerAllocator to insert complete HostAddress in issued ContainerTokenIds. (Konstantinos Karanasos via asuresh)
2016-05-18 18:46:00 -07:00
8a9ecb7584
YARN-5090. Add End-to-End test-cases for DistributedScheduling using MiniYarnCluster. (asuresh)
2016-05-17 19:01:29 -07:00
0c6726e20d
Addendum patch for YARN-4832. Contributed by Junping Du
2016-05-17 14:50:30 -07:00
fa3bc3405d
YARN-4832. NM side resource value should get updated if change applied in RM side. Contributed by Junping Du
2016-05-17 12:52:19 -07:00
ccc93e7812
YARN-5075. Fix findbugs warnings in hadoop-yarn-common module. (asuresh)
2016-05-16 23:22:01 -07:00
1217c8f6b4
YARN-5069. TestFifoScheduler.testResourceOverCommit race condition. Contributed by Eric Badger.
2016-05-16 20:28:04 +00:00
81effb7dcd
YARN-4325. Nodemanager log handlers fail to send finished/failed events in some cases. Contributed by Junping Du
2016-05-16 15:40:23 +00:00
f45bc5a83e
YARN-4738. Notify the RM about the status of OPPORTUNISTIC containers (Konstantinos Karanasos via asuresh)
2016-05-15 17:54:34 -07:00
f0ac18d001
YARN-2888. Corrective mechanisms for rebalancing NM container queues. (asuresh)
2016-05-13 13:38:36 -07:00
3c5c57af28
HADOOP-13142. Change project version from 3.0.0 to 3.0.0-alpha1.
2016-05-12 18:27:28 -07:00
ca5613af91
Revert "Update project version to 3.0.0-alpha1-SNAPSHOT."
...
This reverts commit 6b53802cba
2016-05-12 15:32:45 -07:00
013000fbc2
YARN-5053. More informative diagnostics when applications killed by a user. Contributed by Eric Badger
2016-05-12 20:28:36 +00:00
6b53802cba
Update project version to 3.0.0-alpha1-SNAPSHOT.
2016-05-12 11:05:05 -07:00
0bbe01f8d5
YARN-4577. Enable aux services to have their own custom classpath/jar file (Xuan Gong via sale)
2016-05-12 10:11:23 -07:00
b7ac85259c
YARN-5068. Expose scheduler queue to application master. (Harish Jaiprakash via rohithsharmaks)
2016-05-12 15:17:49 +05:30
d464f4d1c4
YARN-5049. Extend NMStateStore to save queued container information. (Konstantinos Karanasos via asuresh)
2016-05-11 19:10:17 -07:00
4b4e4c6ba8
YARN-4995. FairScheduler: Display per-queue demand on the scheduler page. (xupeng via kasha)
2016-05-11 17:36:21 -07:00
39f2bac38b
YARN-5029. RM needs to send update event with YarnApplicationState as Running to ATS/AHS. Contributed by Xuan Gong.
2016-05-11 09:28:35 -07:00
d971bf252f
YARN-5073. Refactor startContainerInternal() in ContainerManager to remove unused parameter. (Konstantinos Karanasos via asuresh)
2016-05-10 22:31:38 -07:00
2750fb900f
YARN-4926. Change nodelabel rest API invalid reponse status to 400. Contributed by Bibin A Chundatt
2016-05-08 22:49:25 +05:30
47c41e7ac7
YARN-5048. DelegationTokenRenewer#skipTokenRenewal may throw NPE (Jian He via Yongjun Zhang)
2016-05-06 21:50:09 -07:00
b2ed6ae731
YARN-4747. AHS error 500 due to NPE when container start event is missing. Contributed by Varun Saxena
2016-05-06 22:59:39 +00:00
23248f63aa
getApplicationReport call may raise NPE for removed queues. (Jian He via wangda)
2016-05-06 15:30:45 -07:00
bb62e05925
YARN-4390. Do surgical preemption based on reserved container in CapacityScheduler. Contributed by Wangda Tan
2016-05-05 12:56:21 -07:00
d0da13229c
YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla
2016-05-05 14:07:54 +00:00
72b047715c
YARN-4595. Add support for configurable read-only mounts when launching Docker containers. Contributed by Billie Rinaldi.
2016-05-05 13:01:54 +05:30
7bd418e48c
YARN-4984. LogAggregationService shouldn't swallow exception in handling createAppDir() which cause thread leak. (Junping Du via wangda)
2016-05-04 11:38:55 -07:00
e61d431275
YARN-4920. ATS/NM should support a link to dowload/get the logs in text format. Contributed by Xuan Gong.
2016-05-04 10:35:49 -07:00
75e0450593
YARN-4947. Test timeout is happening for TestRMWebServicesNodes. Contributed by Bibin A Chundatt
2016-05-04 09:58:26 +05:30
ed54f5f1ff
YARN-5003. Add container resource to RM audit log. Contributed by Nathan Roberts
2016-05-03 20:03:41 +00:00
06413da72e
YARN-4851. Metric improvements for ATS v1.5 storage components. Li Lu via junping_du.
2016-05-03 04:16:21 -07:00
9d4d30243b
Remove parent's env vars from child processes
2016-04-29 09:25:51 -07:00
0f25a1bb52
YARN-3998. Add support in the NodeManager to re-launch containers. Contributed by Jun Gong.
2016-04-29 16:09:07 +05:30
4a8508501b
YARN-5009. NMLeveldbStateStoreService database can grow substantially leading to longer recovery times. Contributed by Jason Lowe
2016-04-28 21:54:11 -07:00
dd80042c42
YARN-5008. LeveldbRMStateStore database can grow substantially leading to long recovery times. Contributed by Jason Lowe
2016-04-28 21:27:25 -07:00
f16722d2ef
YARN-4956. findbug issue on LevelDBCacheTimelineStore. (Zhiyuan Yang via gtcarrera9)
2016-04-27 10:57:10 -07:00
185c3d4de1
YARN-4807. MockAM#waitForState sleep duration is too long. (Yufei Gu via kasha)
2016-04-27 09:43:23 -07:00
4beff01354
YARN-4983. JVM and UGI metrics disappear after RM transitioned to standby mode
2016-04-26 21:00:17 -07:00
341888a0aa
YARN-4412. Create ClusterMonitor to compute ordered list of preferred NMs for OPPORTUNITIC containers. (asuresh)
2016-04-26 20:12:12 -07:00
1a3f1482e2
YARN-4795. ContainerMetrics drops records. (Daniel Templeton via kasha)
2016-04-26 06:15:36 -07:00
4b1dcbbe0c
YARN-1297. FairScheduler: Move some logs to debug and check if debug logging is enabled
2016-04-26 05:10:09 -07:00
c282a08f38
YARN-2885. Create AMRMProxy request interceptor and ContainerAllocator to distribute OPPORTUNISTIC containers to appropriate Nodes (asuresh)
...
(cherry picked from commit 2bf025278a318b0452fdc9ece4427b4c42124e39)
2016-04-24 22:38:33 -07:00
63e5412f1a
HDFS-9427. HDFS should not default to ephemeral ports. Contributed by Xiaobing Zhou.
2016-04-22 15:14:40 -07:00
7cb3a3da96
YARN-4846. Fix random failures for TestCapacitySchedulerPreemption#testPreemptionPolicyShouldRespectAlreadyMarkedKillableContainers. (Bibin A Chundatt via wangda)
2016-04-22 11:40:32 -07:00
3dce486d88
YARN-4556. TestFifoScheduler.testResourceOverCommit fails. Contributed by Akihiro Suda
2016-04-21 21:16:47 +00:00
7c6339f66a
YARN-4968. A couple of AM retry unit tests need to wait SchedulerApplicationAttempt stopped. (Wangda Tan via gtcarrera9)
2016-04-21 13:25:33 -07:00
170c4fd4cd
YARN-4784. Fairscheduler: defaultQueueSchedulingPolicy should not accept FIFO. (Yufei Gu via kasha)
2016-04-20 23:58:12 -07:00
33fd95a99c
YARN-4890. Unit test intermittent failure: TestNodeLabelContainerAllocation#testQueueUsedCapacitiesUpdate. (Sunil G via wangda)
2016-04-20 17:37:38 -07:00
c8172f5f14
YARN-2883. Queuing of container requests in the NM. (Konstantinos Karanasos and Arun Suresh via kasha)
2016-04-20 09:55:50 -07:00
fdc46bfb37
YARN-4934. Reserved Resource for QueueMetrics needs to be handled correctly in few cases. (Sunil G via wangda)
2016-04-16 22:47:41 -07:00
69f3d428d5
YARN-4940. yarn node -list -all failed if RM start with decommissioned node. Contributed by sandflee
2016-04-15 20:36:45 +00:00
3150ae8108
YARN-4924. NM recovery race can lead to container not cleaned up. Contributed by sandflee
2016-04-14 19:17:14 +00:00
0d9194df00
YARN-4541. Change log message in LocalizedResource#handle() to DEBUG (rchiang via rkanter)
2016-04-13 17:45:18 -07:00
e0cb426758
YARN-4886. Add HDFS caller context for EntityGroupFSTimelineStore. Contributed by Li Lu
2016-04-13 10:38:17 -07:00
437e9d6475
YARN-4810. NM applicationpage cause internal error 500. Contributed by Bibin A Chundatt.
2016-04-12 17:59:46 +05:30
44bbc50d91
YARN-4168. Fixed a failing test TestLogAggregationService.testLocalFileDeletionOnDiskFull. Contributed by Takashi Ohnishi.
2016-04-11 12:11:14 -07:00
2a5da97f81
Revert "YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla"
...
This reverts commit 1cbcd4a491
2016-04-11 15:51:01 +00:00
08ddb3ac6d
YARN-4928. Some yarn.server.timeline.* tests fail on Windows attempting to use a test root path containing a colon. Contributed by Gergely Novák.
2016-04-11 08:48:32 -07:00
1ff27f9d12
YARN-4630. Remove useless boxing/unboxing code. Contributed by Kousuke Saruta.
2016-04-11 14:55:03 +09:00
1b78b2ba17
YARN-4938. MiniYarnCluster should not request transitionToActive to RM on non-HA environment. Contributed by Eric Badger.
2016-04-11 01:31:49 +09:00
ff95fd547b
YARN-4927. TestRMHA#testTransitionedToActiveRefreshFail fails with FairScheduler. (Bibin A Chundatt via kasha)
2016-04-09 10:31:02 -07:00
ec06957941
YARN-3215. Respect labels in CapacityScheduler when computing headroom. (Naganarasimha G R via wangda)
2016-04-08 15:33:04 -07:00
9cb0c963d2
YARN-4740. AM may not receive the container complete msg when it restarts. Contributed by Jun Gong
2016-04-08 11:20:35 -07:00
e82f961a39
YARN-4756. Unnecessary wait in Node Status Updater during reboot. (Eric Badger via kasha)
2016-04-07 17:05:29 -07:00
93bacda08b
YARN-4769. Add support for CSRF header in the dump capacity scheduler logs and kill app buttons in RM web UI. Contributed by Varun Vasudev
2016-04-06 16:13:47 -07:00
b41e65e5bc
YARN-4906. Capture container start/finish time in container metrics. Contributed by Jian He.
2016-04-06 13:41:33 +05:30
21eb428448
YARN-4699. Scheduler UI and REST o/p is not in sync when -replaceLabelsOnNode is used to change label of a node. (Sunil G via wangda)
2016-04-05 16:24:11 -07:00
0005816743
YARN-4916. TestNMProxy.tesNMProxyRPCRetry fails. Contributed by Tibor Kiss.
2016-04-05 09:01:08 -07:00
6be28bcc46
YARN-4893. Fix some intermittent test failures in TestRMAdminService. Contributed by Brahma Reddy Battula.
2016-04-05 06:57:54 -07:00
1cbcd4a491
YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla
2016-04-05 13:40:19 +00:00
776b549e2a
YARN-4609. RM Nodes list page takes too much time to load. Contributed by Bibin A Chundatt
2016-04-05 14:47:25 +05:30
552237d4a3
YARN-4880. Running TestZKRMStateStorePerf with real zookeeper cluster throws NPE. Contributed by Sunil G
2016-04-05 14:26:19 +05:30
5092c94195
YARN-4746. yarn web services should convert parse failures of appId, appAttemptId and containerId to 400. Contributed by Bibin A Chundatt
2016-04-04 16:25:03 +05:30
1e6f92977d
YARN-4607. Pagination support for AppAttempt page TotalOutstandingResource Requests table. Contributed by Bibin A Chundatt
2016-04-04 08:09:29 +05:30
12b11e2e68
YARN-4634. Scheduler UI/Metrics need to consider cases like non-queue label mappings. (Sunil G via wangda)
2016-03-31 14:35:18 -07:00
0dd9bcab97
YARN-4811. Generate histograms in ContainerMetrics for actual container resource usage
2016-03-31 14:28:13 -07:00
7a021471c3
YARN-4639. Remove dead code in TestDelegationTokenRenewer added in YARN-3055 (templedf via rkanter)
2016-03-31 13:09:09 -07:00
0a74610d1c
HADOOP-11393. Revert HADOOP_PREFIX, go back to HADOOP_HOME (aw)
2016-03-31 07:51:05 -07:00
60e4116bf1
YARN-4822. Refactor existing Preemption Policy of CS for easier adding new approach to select preemption candidates. Contributed by Wangda Tan
2016-03-30 12:43:52 -07:00
fc055a3cbe
YARN-4865. Track Reserved resources in ResourceUsage and QueueCapacities. (Sunil G via wangda)
2016-03-29 17:07:55 -07:00
80182809ae
YARN-4863. AHS Security login should be in serviceInit() instead of serviceStart(). Contributed by Junping Du
2016-03-28 22:17:45 -07:00
948b758070
YARN-4773. Log aggregation performs extraneous filesystem operations when rolling log aggregation is disabled. Contributed by Jun Gong
2016-03-28 23:00:56 +00:00
524bc3c33a
YARN-998. Keep NM resource updated through dynamic resource config for RM/NM restart. Contributed by Junping Du
2016-03-28 11:12:33 -07:00
55ae143923
YARN-4117. End to end unit test with mini YARN cluster for AMRMProxy Service. Contributed by Giovanni Matteo Fumarola
2016-03-27 20:22:12 -07:00
49ff54c860
YARN-4805. Don't go through all schedulers in ParameterizedTestBase. (kasha)
2016-03-26 21:45:13 -07:00
00bebb7e58
YARN-4823. Refactor the nested reservation id field in listReservation to simple string field. (subru via asuresh)
2016-03-25 15:54:38 -07:00
d82e797b65
YARN-4825. Remove redundant code in ClientRMService::listReservations. (subru via asuresh)
2016-03-24 09:59:55 -07:00
b1394d6307
YARN-4850. test-fair-scheduler.xml isn't valid xml (Yufei Gu via aw)
2016-03-24 08:15:58 -07:00
19b645c938
YARN-4820. ResourceManager web redirects in HA mode drops query parameters. Contributed by Varun Vasudev.
2016-03-23 19:34:30 -07:00
Akira Ajisaka
Karthik Kambatla
Karthik Kambatla
Varun Saxena
Sangjin Lee
Xuan
Robert Kanter
Chris Douglas
Naganarasimha
Rohith Sharma K S
Varun Vasudev
Jian He
Subru Krishnan
Arun Suresh
Wangda Tan
Jason Lowe
Kai Zheng
Ray Chiang
Subru Krishnan
Junping Du
Chris Nauroth
Vinod Kumar Vavilapalli
Andrew Wang
Vrushali
Li Lu
Naganarasimha
Vrushali Channapattan
Zhijie Shen
Ray Chiang
Tsuyoshi Ozawa
Sangjin Lee
Ming Ma
Carlo Curino
Steve Loughran
Eric Payne
Yongjun Zhang
Jing Zhao
Allen Wittenauer