ac0ddc4fec
YARN-4298. Fix findbugs warnings in hadoop-yarn-common. Contributed by Sunil G.
...
(cherry picked from commit 8c0133f3d4
2015-11-24 18:40:56 +09:00
49ed955c91
YARN-3980. Plumb resource-utilization info in node heartbeat through to the scheduler. (Inigo Goiri via kasha)
...
(cherry picked from commit 52948bb20b
2015-11-24 13:52:29 +05:30
b339a4b8d6
YARN-4349. Support CallerContext in YARN. Contributed by Wangda Tan
...
(cherry picked from commit 8676a118a1
2015-11-23 17:20:24 -08:00
d13b646c61
Update CHANGES.txt for recent commits to branch-2.6.
...
(cherry picked from commit aa2009c37b
2015-11-23 21:58:57 +00:00
9d303edbfb
YARN-4344. NMs reconnecting with changed capabilities can lead to wrong cluster resource calculations. Contributed by Varun Vasudev
...
(cherry picked from commit d36b6e045f
2015-11-23 20:31:50 +00:00
cbbdbe3cdd
YARN-3454. Add efficient merge operation to RLESparseResourceAllocation (Carlo Curino via asuresh)
...
(cherry picked from commit da1016365a
2015-11-21 10:01:10 -08:00
08727287de
move YARN-4326/YARN-3849 from 2.8.0 to 2.7.3
2015-11-20 16:42:11 -08:00
cdca460ad4
YARN-3769. Consider user limit when calculating total pending resource for preemption policy in Capacity Scheduler. (Eric Payne via wangda)
...
(cherry picked from commit 2346fa3141
2015-11-20 15:56:17 -08:00
f3bca8bc1a
YARN-4374. RM capacity scheduler UI rounds user limit factor. Contributed by Chang Li
...
(cherry picked from commit 060cdcbe5d
2015-11-20 23:13:27 +00:00
8347b1048d
YARN-2859 addendum: move the entry from 2.6.2 to 2.6.3 in hadoop-yarn/CHANGES.txt.
...
(cherry picked from commit 866dce4e8e
2015-11-19 10:17:20 -08:00
be65d319fa
YARN-2859.addendum: fix the remaining issue from the previous patch
...
(cherry picked from commit f114e728da
2015-11-19 10:09:48 -08:00
3504d0e795
move fix version of YARN-4287 from 2.8.0 to 2.7.3
2015-11-18 14:01:02 -08:00
d3f9508096
YARN-4367. SLS webapp doesn't load. (kasha)
2015-11-18 13:45:00 -08:00
a0b3e43a6d
YARN-4279. Mark ApplicationId and ApplicationAttemptId static methods as @Public, @Unstable. (stevel)
2015-11-18 21:41:24 +00:00
d8e60080e9
YARN-4184. Remove update reservation state api from state store as its not used by ReservationSystem (Sean Po via asuresh)
...
(cherry picked from commit 6a61928fb7
2015-11-17 15:51:59 -08:00
2cf084ac76
Revert "YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan"
...
This reverts commit a4ff03b200
2015-11-16 20:19:31 -08:00
1d107d8058
YARN-4354. Public resource localization fails with NPE. Contributed by Jason Lowe.
...
(cherry picked from commit 855d52927b
2015-11-15 04:44:59 -08:00
875aec3177
YARN-1509. Make AMRMClient support send increase container request and get increased/decreased containers. (Meng Ding via wangda)
...
(cherry picked from commit 7ff280fca9
2015-11-12 11:50:41 -08:00
13f8c7093b
YARN-4347. Resource manager fails with Null pointer exception. (Jian He via wangda)
...
(cherry picked from commit 7f55a18071
2015-11-12 11:24:04 -08:00
0c1fb15b29
YARN-4287. Capacity Scheduler: Rack Locality improvement (Nathan Roberts via wangda)
...
(cherry picked from commit 796638d9bc
2015-11-12 11:12:11 -08:00
9050078005
YARN-4345. yarn rmadmin -updateNodeResource doesn't work. Contributed by Junping Du
...
(cherry picked from commit 06a70311b9
2015-11-12 14:49:55 +00:00
bb5dd7f772
YARN-4183. Reverting the patch to fix behaviour change.
...
Revert "YARN-4183. Enabling generic application history forces every job to get a timeline service delegation token (jeagles)"
This reverts commit 42bc525acb
2015-11-11 10:42:52 -08:00
f6f128c867
YARN-4241. Fix typo of property name in yarn-default.xml. Contributed by Anthony Rojas.
...
(cherry picked from commit 23d0db551c
2015-11-11 16:13:50 +09:00
8a8ac50a5b
YARN-1510. Make NMClient support change container resources. (Meng Ding via wangda)
...
(cherry picked from commit c99925d6dd
2015-11-10 13:17:23 -08:00
a4ff03b200
YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan
...
(cherry picked from commit 8fbea531d7
2015-11-09 11:05:12 -08:00
8334d86472
Add 2.7.3 release to CHANGES.txt
...
(cherry picked from commit 0383a3973b
2015-11-04 16:27:28 +00:00
3190446df9
YARN-4326. Fix TestDistributedShell timeout as AHS in MiniYarnCluster no longer binds to default port 8188. (Meng Ding via wangda)
...
(cherry picked from commit 0783184f4b
2015-11-03 11:36:11 -08:00
b2d6ea5e18
Updated the 2.6.2 final release date.
...
(cherry picked from commit a4a6b5b4b4
2015-10-30 18:49:21 -07:00
8dce482f61
Creating 2.6.3 entries in CHANGES.txt files.
...
(cherry picked from commit eadf7b3096
2015-10-30 14:52:06 +00:00
1c80473a41
YARN-4320. TestJobHistoryEventHandler fails as AHS in MiniYarnCluster no longer binds to default port 8188. Contributed by Varun Saxena.
...
(cherry picked from commit ce31b22739
2015-10-30 18:00:35 +09:00
dd144035a0
Add an entry of YARN-4312 to CHANGES.txt
...
(cherry picked from commit d21214ce33
2015-10-30 17:57:29 +09:00
c522a9ffc2
Move YARN-3580 in CHANGES.txt from 2.8 to 2.7.2.
...
(cherry picked from commit d2e01f4ed8
2015-10-30 15:51:04 +09:00
b505bf9e58
YARN-4313. Race condition in MiniMRYarnCluster when getting history
...
server address. Contributed by Jian He
(cherry picked from commit 7412ff48ee
2015-10-29 17:37:24 -07:00
2b197bbffd
YARN-4127. RM fail with noAuth error if switched from failover to non-failover. Contributed by Varun Saxena
...
(cherry picked from commit e5b1733e04
2015-10-29 15:43:26 -07:00
42bc525acb
YARN-4183. Enabling generic application history forces every job to get a timeline service delegation token (jeagles)
...
(cherry picked from commit c293c58954
2015-10-29 16:42:49 -05:00
8943abfb08
YARN-4310. FairScheduler: Log skipping reservation messages at DEBUG level (asuresh)
...
(cherry picked from commit 58d1df585c
2015-10-29 13:43:43 -07:00
c75d8b164f
YARN-2902. Killing a container that is localizing can orphan resources in the DOWNLOADING state. Contributed by Varun Saxena
...
(cherry picked from commit e2267de207
2015-10-29 16:39:55 +00:00
7d109ef5b5
YARN-4288. Fixed RMProxy to retry on IOException from local host. Contributed by Junping Du
...
(cherry picked from commit c41699965e
2015-10-29 00:01:20 -07:00
e3fcb711d6
YARN-4130. Duplicate declaration of ApplicationId in RMAppManager#submitApplication method. (Kai Sasaki via rohithsharmaks)
...
(cherry picked from commit 656c8f9527
2015-10-29 12:25:08 +05:30
16db203a83
Set the release date for 2.6.2
2015-10-28 20:48:49 -07:00
9ce5069d16
YARN-2859. ApplicationHistoryServer binds to default port 8188 in
...
MiniYARNCluster. Contributed by Vinod Kumar Vavilapalli
(cherry picked from commit 27414dac66
2015-10-28 10:33:56 -07:00
d29bff86c4
YARN-4251. TestAMRMClientOnRMRestart#testAMRMClientOnAMRMTokenRollOverOnRMRestart is failing. Contributed by Brahma Reddy Battula.
...
(cherry picked from commit 9f4dfdf4eb
2015-10-28 17:39:26 +09:00
c5bf1cb7af
YARN-4302. SLS not able start due to NPE in SchedulerApplicationAttempt. Contributed by Bibin A Chundatt.
...
(cherry picked from commit c28e16b40c
2015-10-27 20:56:45 +05:30
3138b43f16
YARN-3573. MiniMRYarnCluster constructor that starts the timeline server using a boolean should be marked deprecated. Contributed by Brahma Reddy Battula.
...
(cherry picked from commit 96677bef00
2015-10-27 12:35:20 +09:00
d9156c1245
YARN-4300. [JDK8] Fix javadoc errors caused by wrong tags. (aajisaka)
...
(cherry picked from commit 8a68630dd1
2015-10-27 10:28:30 +09:00
1499af38eb
YARN-3216. Max-AM-Resource-Percentage should respect node labels. (Sunil G via wangda)
...
(cherry picked from commit 56e4f6237a
2015-10-26 16:45:20 -07:00
9bf1e74171
YARN-4169. Fix racing condition of TestNodeStatusUpdaterForLabels. (Naganarasimha G R via wangda)
...
(cherry picked from commit 6f606214e7
2015-10-26 16:38:39 -07:00
baa2998e93
YARN-4285. Display resource usage as percentage of queue and cluster in the RM UI (Varun Vasudev via wangda)
...
(cherry picked from commit 3cc73773eb
2015-10-26 13:12:56 -07:00
7d66e2e666
YARN-4284. condition for AM blacklisting is too narrow. Contributed by Sangjin Lee
...
(cherry picked from commit 33a03af3c3
2015-10-26 19:54:16 +00:00
33455fa415
YARN-2729. Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup. (Naganarasimha G R via rohithsharmaks)
...
(cherry picked from commit 5acdde4744
2015-10-26 15:45:22 +05:30
06ecc37e82
YARN-4223. Fixed findbugs warnings in hadoop-yarn-server-nodemanager project. (varun saxena via rohithsharmaks)
...
(cherry picked from commit 092883b34a
2015-10-26 15:26:08 +05:30
e85ebf0825
YARN-3528. Tests with 12345 as hard-coded port break jenkins. Contributed by Brahma Reddy Battula.
...
Conflicts:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeManagerShutdown.java
2015-10-26 17:06:46 +09:00
e75799d5fd
YARN-4246. NPE while listing app attempt. (Nijel S F via rohithsharmaks)
...
(cherry picked from commit b57f08c0d2
2015-10-26 12:07:48 +05:30
599653ad92
YARN-3724. Use POSIX nftw(3) instead of fts(3) (Alan Burlison via aw)
2015-10-25 21:43:51 -07:00
f0580dcac0
YARN-3738. Add support for recovery of reserved apps running under dynamic queues (subru via asuresh)
...
(cherry picked from commit ab8eb8770c
2015-10-24 22:54:45 -07:00
fc5a433adb
YARN-4296. DistributedShell Log.info is not friendly. (Xiaowei Wang via stevel)
2015-10-24 12:54:48 +01:00
65b928d8c7
YARN-4289. TestDistributedShell failing with bind exception. (Brahma Reddy Battula via stevel)
2015-10-24 12:48:03 +01:00
356e922632
YARN-4294. [JDK8] Fix javadoc errors caused by wrong reference and illegal tag. (aajisaka)
...
(cherry picked from commit 7781fe1b9e
2015-10-24 11:56:37 +09:00
235e84bf3d
YARN-4041. Slow delegation token renewal can severely prolong RM recovery. Contributed by Sunil G
...
(cherry picked from commit d3a34a4f38
2015-10-23 20:57:53 +00:00
6db7bfbcfd
YARN-4009. CORS support for ResourceManager REST API. ( Varun Vasudev via jeagles)
...
(cherry picked from commit f8adeb712d
2015-10-23 10:43:59 -05:00
4bb7e68eb6
YARN-2913. Fair scheduler should have ability to set MaxResourceDefault for each queue. (Siqi Li via mingma)
...
(cherry picked from commit 934d96a334
2015-10-23 08:37:46 -07:00
d5145b58f6
YARN-4243. Add retry on establishing Zookeeper conenction in EmbeddedElectorService#serviceInit. Contributed by Xuan Gong.
...
(cherry picked from commit 0fce5f9a49
2015-10-22 13:42:12 -07:00
9fe7ccc61c
YARN-4256. YARN fair scheduler vcores with decimal values. Contributed by Jun Gong
...
(cherry picked from commit 960201b79b
2015-10-22 12:31:28 -07:00
c44401f362
YARN-3739. Add reservation system recovery to RM recovery process. Contributed by Subru Krishnan.
...
(cherry picked from commit 2798723a54
2015-10-22 06:52:41 -07:00
cc8093e5ac
YARN-2513. Host framework UIs in YARN for use with the ATS (jeagles)
...
(cherry picked from commit 25f8f801d1
2015-10-21 15:38:48 -05:00
e2f027dddc
Moved CHANGES.txt entries to 2.6.2
...
YARN-2019, YARN-4092, YARN-4101, YARN-4087, YARN-3727, YARN-4005,
YARN-3780, YARN-3802, YARN-3896, YARN-3554, YARN-3194, YARN-3798
(cherry picked from commit 14f0e2787c
2015-10-21 23:25:43 +09:00
f2009dc89c
YARN-3985. Make ReservationSystem persist state using RMStateStore reservation APIs. (adhoot via asuresh)
...
(cherry picked from commit 506d1b1dbc
2015-10-20 16:48:58 -07:00
acc0e718d6
YARN-4270. Limit application resource reservation on nodes for non-node/rack specific requests (asuresh)
...
(cherry picked from commit 7e2837f830
2015-10-19 20:02:10 -07:00
e4429de86f
YARN-4267. Add additional logging to container launch implementations in container-executor. Contributed by Sidharta Seethana.
...
(cherry picked from commit 151eab2a12
2015-10-19 20:56:43 +05:30
196d549e2b
YARN-4262. Allow whitelisted users to run privileged docker containers. Contributed by Sidharta Seethana.
...
(cherry picked from commit e39ae0e676
2015-10-19 20:21:48 +05:30
390f29ab99
YARN-4155. TestLogAggregationService.testLogAggregationServiceWithInterval failing. (Bibin A Chundatt via stevel)
2015-10-18 12:03:46 +01:00
180efe6677
YARN-2556. Tool to measure the performance of the timeline server (Chang Li via sjlee)
...
(cherry picked from commit 58590fef49
2015-10-16 16:57:32 -07:00
1ba73169c7
YARN-4170. AM need to be notified with priority in AllocateResponse. Contributed by Sunil G
...
(cherry picked from commit f9da5cdb2b
2015-10-16 15:26:46 -07:00
204ee6a40f
YARN-4162. CapacityScheduler: Add resource usage by partition and queue capacity by partition to REST API. (Naganarasimha G R via wangda)
...
(cherry picked from commit 4337b263aa
2015-10-16 15:10:08 -07:00
2862057f11
YARN-4000. RM crashes with NPE if leaf queue becomes parent queue during restart. Contributed by Varun Saxena
...
(cherry picked from commit cf23f2c2b5
2015-10-15 17:13:09 -07:00
342870b70f
YARN-4258. Add support for controlling capabilities for docker containers. Contributed by Sidharta Seethana.
...
(cherry picked from commit 63020c54c1
2015-10-15 13:41:58 +05:30
526be695c0
YARN-4250. NPE in AppSchedulingInfo#isRequestLabelChanged. (Brahma Reddy Battula via rohithsharmaks)
...
(cherry picked from commit d6c8bad869
2015-10-14 16:13:26 +05:30
2dd9c475c4
YARN-4255. container-executor does not clean up docker operation command files. Contributed by Sidharta Seethana.
...
(cherry picked from commit da1ee078f9
2015-10-14 15:27:18 +05:30
0aeedfd572
YARN-4252. Log container-executor invocation details when exit code is non-zero. Contributed by Sidharta Seethana.
...
(cherry picked from commit fb9c1519b3
2015-10-14 15:19:19 +05:30
a9b03b5afb
YARN-4253. Standardize on using PrivilegedOperationExecutor for all invocations of container-executor in LinuxContainerExecutor. Contributed by Sidharta Seethana.
...
(cherry picked from commit 8d59293089
2015-10-14 15:04:22 +05:30
34b8142802
YARN-4017. container-executor overuses PATH_MAX. Contributed by Sidharta Seethana
...
(cherry picked from commit 5b6bae0094
2015-10-13 12:50:28 +05:30
ee1ce393f5
YARN-4230. RM crashes with NPE when increasing container resource if there is no headroom left. Contributed by Meng Ding
...
(cherry picked from commit 9849c8b386
2015-10-12 11:52:04 -07:00
9988b57e71
YARN-4201. AMBlacklist does not work for minicluster. Contributed by Jun Gong.
...
(cherry picked from commit 049c6e8dc0
2015-10-12 00:16:40 -07:00
28edc7b129
YARN-3964. Support NodeLabelsProvider at Resource Manager side.
...
Contributed by Dian Fu.
(cherry picked from commit db93047881
2015-10-11 11:24:05 +05:30
37642abb83
YARN-4140. RM container allocation delayed incase of app submitted to Nodelabel partition. (Bibin A Chundatt via wangda)
...
(cherry picked from commit def374e666
2015-10-09 16:40:10 -07:00
0139517811
YARN-261. Ability to fail AM attempts. Contributed by Andrey Klochkov and Rohith Sharma K S
...
(cherry picked from commit a0bca2b5ad
2015-10-09 14:19:41 +00:00
21609e8e9a
YARN-4235. FairScheduler PrimaryGroup does not handle empty groups returned for a user. (Anubhav Dhoot via rohithsharmaks)
...
(cherry picked from commit 8f195387a4
2015-10-09 10:11:53 +05:30
ec74b0cd4e
YARN-3943. Use separate threshold configurations for disk-full detection and disk-not-full detection. Contributed by Zhihai Xu
...
(cherry picked from commit 8d226225d0
2015-10-08 22:26:35 +00:00
1caa05de4a
Fixup CHANGES.txt to reflect recent commits to 2.7.2 and 2.6.2
...
(cherry picked from commit 0841940a21
2015-10-08 16:47:30 +00:00
99126714a3
YARN-4228. FileSystemRMStateStore use IOUtils#close instead of fs#close. (Bibin A Chundatt via rohithsharmaks)
...
(cherry picked from commit 3793cbe4c3
2015-10-07 10:20:36 +05:30
053fcf4c0d
YARN-4209. RMStateStore FENCED state doesn’t work due to updateFencedState called by stateMachine.doTransition. (Zhihai Xu via rohithsharmaks)
...
(cherry picked from commit 9156fc60c6
2015-10-07 09:37:14 +05:30
5453a63612
YARN-4215. RMNodeLabels Manager Need to verify and replace node labels for the only modified Node Label Mappings in the request. (Naganarasimha G R via wangda)
...
(cherry picked from commit 29a582ada0
2015-10-06 11:59:10 -07:00
d24a3b9a3c
YARN-4176. Resync NM nodelabels with RM periodically for distributed nodelabels. (Bibin A Chundatt via wangda)
...
(cherry picked from commit 30ac69c6bd
2015-10-05 15:48:40 -07:00
53bddc410f
YARN-1897. CLI and core support for signal container functionality. Contributed by Ming Ma
...
(cherry picked from commit 8f08532bde
2015-10-02 18:53:04 -07:00
3e3733437f
YARN-3619. ContainerMetrics unregisters during getMetrics and leads to ConcurrentModificationException. Contributed by Zhihai Xu
...
(cherry picked from commit fdf02d1f26
2015-10-02 20:21:32 +00:00
d645ee1d62
YARN-3727. For better error recovery, check if the directory exists before using it for localization. Contributed by Zhihai Xu
...
(cherry picked from commit 854d25b0c3
2015-09-30 15:01:01 +00:00
64a7617eb9
YARN-4066. Large number of queues choke fair scheduler. (Johan Gustavsson via kasha)
...
(cherry picked from commit a0b5a0a419
2015-09-29 08:13:25 -07:00
22f2501476
YARN-4180. AMLauncher does not retry on failures when talking to NM. (adhoot)
...
(cherry picked from commit 9735afe967
2015-09-28 16:54:06 -07:00
600022ae69
YARN-4141. Runtime Application Priority change should not throw exception for applications at finishing states. Contributed by Sunil G
...
(cherry picked from commit 9f53a95ff6
2015-09-28 22:56:07 +00:00
4d8b99423e
YARN-4204. ConcurrentModificationException in FairSchedulerQueueInfo. (adhoot)
...
(cherry picked from commit fb2e525c07
2015-09-28 09:39:52 -07:00
Akira Ajisaka
Karthik Kambatla
Jian He
Jason Lowe
Arun Suresh
Wangda Tan
Xuan
Steve Loughran
Junping Du
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
Sangjin Lee
Tsuyoshi Ozawa
Jonathan Eagles
Rohith Sharma K S
Varun Vasudev
Allen Wittenauer
Ming Ma
Zhihai Xu
Anubhav Dhoot
Devaraj K