3be2659f83
YARN-6694. Add certain envs to the default yarn.nodemanager.env-whitelist. Contributed by Jian He
2017-06-29 20:10:35 -07:00
af2773f609
Updating version for 3.0.0-beta1 development
2017-06-29 17:57:40 -07:00
ec97519779
YARN-6751. Display reserved resources in web UI per queue
...
(Contributed by Abdullah Yousufi via Daniel Templeton)
2017-06-29 16:53:50 -07:00
c1edca101c
YARN-6280. Introduce deselect query param to skip ResourceRequest from getApp/getApps REST API. Contributed by Lantao Jin.
2017-06-28 15:40:58 -07:00
4e3eebc943
YARN-5311. Document graceful decommission CLI and usage. Contributed by Elek, Marton.
2017-06-28 15:32:04 -07:00
25d891a784
YARN-6743. yarn.resourcemanager.zk-max-znode-size.bytes description needs spaces in yarn-default.xml
...
(Contributed by Lori Loberg via Daniel Templeton)
2017-06-28 13:17:58 -07:00
63ce1593c5
YARN-6738. LevelDBCacheTimelineStore should reuse ObjectMapper instances. Contributed by Zoltan Haindrich
2017-06-27 17:12:42 -05:00
0b77262890
YARN-5876. TestResourceTrackerService#testGracefulDecommissionWithApp fails intermittently on trunk. (Robert Kanter via Yufei Gu)
2017-06-23 13:26:03 -07:00
ca13b224b2
YARN-5892. Support user-specific minimum user limit percentage in Capacity Scheduler. Contributed by Eric Payne.
2017-06-22 23:50:57 -07:00
740204b292
YARN-5006. ResourceManager quit due to ApplicationStateData exceed the limit size of znode in zk. Contributed by Bibin A Chundatt.
2017-06-23 07:52:41 +05:30
49aa60e50d
YARN-6127. Add support for work preserving NM restart when AMRMProxy is enabled. (Botong Huang via asuresh).
2017-06-22 11:27:13 -07:00
099dfe92a9
YARN-6713. Fix dead link in the Javadoc of FairSchedulerEventLog.java. Contributed by Weiwei Yang.
2017-06-20 17:20:27 +09:00
ee89ac84e6
YARN-6680. Avoid locking overhead for NO_LABEL lookups. Contributed by Daryn Sharp
2017-06-19 22:42:45 +05:30
e9c2aa1bc3
YARN-6467. CSQueueMetrics needs to update the current metrics for default partition only. Contributed by Manikandan R.
2017-06-19 21:31:40 +05:30
7582dedad1
YARN-6517. Fix warnings from Spotbugs in hadoop-yarn-common(addendum). Contributed by Weiwei Yang.
2017-06-18 19:53:42 +05:30
036a24bab3
MAPREDUCE-6896. Document wrong spelling in usage of MapredTestDriver tools.
...
(Contributed by LiXin Ge via Daniel Templeton)
2017-06-13 13:21:23 -07:00
d64c842743
YARN-6703. RM startup failure with old state store due to version mismatch. Contributed by Varun Saxena
2017-06-12 18:07:43 +05:30
5578af8603
YARN-6585. RM fails to start when upgrading from 2.7 for clusters with node labels. Contributed by Sunil G.
2017-06-09 16:16:34 -05:00
99634d10ed
YARN-6682. Improve performance of AssignmentInformation datastructures. Contributed by Daryn Sharp.
2017-06-09 11:10:13 +05:30
a062374c39
YARN-6679. Reduce Resource instance overhead via non-PBImpl
...
(Contributed by Daryn Sharp via Daniel Templeton)
2017-06-08 10:08:08 -07:00
547cb53e8a
YARN-6634. [API] Refactor ResourceManager WebServices to make API explicit. (Giovanni Matteo Fumarola via curino)
2017-06-07 13:41:06 -07:00
0b7bff706e
YARN-6604. Allow metric TTL for Application table to be specified through cmd (Haibo Chen via Varun Saxena)
2017-06-07 21:51:07 +05:30
19ef3a81f8
HADOOP-14485. Redundant 'final' modifier in try-with-resources statement. Contributed by wenxin he.
2017-06-06 23:11:47 +08:00
7311015ace
YARN-6683. Invalid event: COLLECTOR_UPDATE at KILLED. Contributed by Rohith Sharma K S
2017-06-05 13:16:57 -07:00
46f7e91980
YARN-6458. Use yarn package manager to lock down dependency versions for new web UI. Contributed by Sreenath Somarajapuram.
2017-06-04 22:05:14 +05:30
73ecb19312
YARN-6208. Improve the log when FinishAppEvent sent to the NodeManager which didn't run the application
...
(Contributed by Akira Ajisaka via Daniel Templeton)
2017-06-02 08:50:19 -07:00
d95c82cb79
YARN-6316 Provide help information and documentation for TimelineSchemaCreator (Contributed by Haibo Chen via Vrushali C)
2017-06-01 18:30:23 -07:00
d5b71e4175
YARN-6246. Identifying starved apps does not need the scheduler writelock
...
(Contributed by Karthik Kambatla via Daniel Templeton)
2017-05-31 15:48:04 -07:00
4369690ce6
YARN-6649. RollingLevelDBTimelineServer throws RuntimeException if object decoding ever fails runtime exception. Contributed by Jon Eagles.
2017-05-31 11:32:32 -05:00
cbfed0e82f
YARN-6497. Method length of ResourceManager#serviceInit() is too long (Gergely Novák via jeagles)
2017-05-31 10:18:42 -05:00
547f18cb96
YARN-6366. Refactor the NodeManager DeletionService to support additional DeletionTask types. Contributed by Shane Kumpf.
2017-05-31 16:15:35 +05:30
4b4a6524f2
YARN-6477. Dispatcher no longer needs the raw types suppression. (Maya Wexler via Haibo Chen)
2017-05-30 16:58:15 -07:00
af03c33346
YARN-6635. Refactor yarn-app pages in new YARN UI. Contributed by Akhil PB.
2017-05-30 13:52:40 +05:30
16ad896d5c
Update maven version for 3.0.0-alpha4 development
2017-05-26 14:09:44 -07:00
d81372dfad
YARN-6646. Modifier 'static' is redundant for inner enums
...
(Contributed by ZhangBing Lin via Daniel Templeton)
2017-05-26 12:05:48 -07:00
aea42930bb
YARN-6641. Non-public resource localization on a bad disk causes subsequent containers failure. Contributed by Kuhu Shukla
2017-05-26 09:37:56 -05:00
47474fffac
YARN-6555. Store application flow context in NM state store for work-preserving restart. (Rohith Sharma K S via Haibo Chen)
2017-05-25 21:15:27 -07:00
87590090c8
YARN-6582. FSAppAttempt demand can be updated atomically in updateDemand(). (Karthik Kambatla via Yufei Gu)
2017-05-25 14:22:13 -07:00
3fd6a2da4e
YARN-6643. TestRMFailover fails rarely due to port conflict. Contributed by Robert Kanter
2017-05-25 16:07:52 -05:00
1a56a3db59
Addendum patch to fix Docker sanitization.
2017-05-25 14:53:57 +05:30
bc28da65fb
YARN-6141. ppc64le on Linux doesn't trigger __linux get_executable codepath. Contributed by Sonia Garudi and Ayappan.
2017-05-25 17:06:26 +09:00
8bf1949c0e
YARN-6615. AmIpFilter drops query parameters on redirect. Contributed by Wilfred Spiegelenburg
2017-05-24 11:22:42 -05:00
8e0f83e49a
YARN-6493. Print requested node partition in assignContainer logs. (Jonathan Hung via wangda)
...
Change-Id: Ib20326daac10acad363befb0ae0998441fa5b32d
2017-05-22 14:28:55 -07:00
c583ab02c7
YARN-2113. Add cross-user preemption within CapacityScheduler's leaf-queue. (Sunil G via wangda)
...
Change-Id: I9b19f69788068be05b3295247cdd7b972f8a573c
2017-05-22 14:26:13 -07:00
b6f66b0da1
YARN-6584. Correct license headers in hadoop-common, hdfs, yarn and mapreduce. Contributed by Yeliang Cang.
2017-05-22 14:10:06 +05:30
46d9c4ca7a
YARN-5705. Show timeline data from ATS v2 in new web UI. Contributed by Akhil PB.
2017-05-21 18:09:13 +05:30
e135c0b207
YARN-6627. Use deployed webapp folder to launch new YARN UI. Contributed by Sunil G
2017-05-19 20:14:03 -07:00
9855225a79
YARN-6602. Impersonation does not work if standby RM is contacted first (rkanter)
2017-05-19 15:43:22 -07:00
66bba8c024
YARN-6249. TestFairSchedulerPreemption fails inconsistently. (Tao Jie via Yufei Gu)
2017-05-19 14:53:27 -07:00
e82da2b4f4
YARN-6540. Resource Manager is spelled "Resource Manger" in ResourceManagerRestart.md and ResourceManagerHA.md. (Grant Sohn via Yufei Gu)
2017-05-19 14:16:00 -07:00
19482e0d09
YARN-6618. TestNMLeveldbStateStoreService#testCompactionCycle can fail if compaction occurs more than once. Contributed by Jason Lowe.
2017-05-19 11:24:54 -05:00
b46cd315f7
Sanitize arguments before launching Docker containers.
2017-05-18 10:37:16 +05:30
e120ee865a
Validate docker image name before launching container.
2017-05-18 10:29:34 +05:30
b23fcc86c6
YARN-6577. Remove unused ContainerLocalization classes. Contributed by ZhangBing Lin
2017-05-17 20:40:06 -07:00
d87a63a901
HADOOP-14412. HostsFileReader#getHostDetails is very expensive on large clusters. Contributed by Jason Lowe.
2017-05-17 08:27:45 +05:30
18c494a00c
YARN-6447. Provide container sandbox policies for groups (gphillips via rkanter)
2017-05-16 18:02:39 -07:00
8236130b2c
YARN-6306. NMClient API change for container upgrade. Contributed by Arun Suresh
2017-05-16 10:48:46 -07:00
489f85933c
YARN-6603. NPE in RMAppsBlock. Contributed by Jason Lowe
2017-05-16 09:26:44 -05:00
c48f2976a3
YARN-6598. History server getApplicationReport NPE when fetching report for pre-2.8 job (Jason Lowe via jeagles)
2017-05-15 10:32:01 -05:00
6600abbb5c
YARN-6587. Refactor of ResourceManager#startWebApp in a Util class. (Giovanni Matteo Fumarola via curino).
2017-05-12 17:48:55 -07:00
6c35001b9f
YARN-6580. Incorrect logger for FairSharePolicy. (Vrushali C via Haibo Chen)
2017-05-12 13:05:40 -07:00
90cb5b4635
YARN-6380. FSAppAttempt keeps redundant copy of the queue
2017-05-11 13:47:02 -07:00
2ada100da7
YARN-5543. ResourceManager SchedulingMonitor could potentially terminate the preemption checker thread. Contributed by Min Shen.
2017-05-11 13:25:31 -07:00
4aae2d40a3
HADOOP-14410. Correct spelling of 'beginning' and variants. Contributed By Dongtao Zhang
2017-05-11 23:33:45 +05:30
5cb6e3e082
YARN-6473. Create ReservationInvariantChecker to validate ReservationSystem + Scheduler operations.
2017-05-10 14:49:25 -07:00
e7654c4a1f
YARN-6571. Fix JavaDoc issues in SchedulingPolicy
...
(Contributed by Weiwei Yang via Daniel Templeton)
2017-05-10 14:34:13 -07:00
74a61438ca
YARN-6475. Fix some long function checkstyle issues
...
(Contributed by Soumabrata Chakraborty via Daniel Templeton)
2017-05-10 10:46:50 -07:00
6099deebcb
YARN-6552. Increase YARN test timeouts from 1 second to 10 seconds. Contributed by Eric Badger
2017-05-10 08:47:48 -05:00
ab2bb93a2a
YARN-6561. Update exception message during timeline collector aux service initialization. (Vrushali C via Haibo Chen)
2017-05-09 21:37:30 -07:00
461ee44d28
YARN-6435. [ATSv2] Can't retrieve more than 1000 versions of metrics in time series. (Rohith Sharma K S via Haibo Chen)
2017-05-09 21:12:57 -07:00
166be0ee95
YARN-3742. YARN RM will shut down if ZKClient creation times out. (Daniel Templeton via kasha)
2017-05-09 14:44:16 -07:00
a2f680493f
YARN-5301. NM mount cpu cgroups failed on some systems
...
(Contributed by Miklos Szegedi via Daniel Templeton)
2017-05-09 12:05:46 -07:00
7dd258d8f4
YARN-6563 ConcurrentModificationException in TimelineCollectorManager while stopping RM (Contributed by Haibo Chen via Vrushali C)
2017-05-09 11:18:12 -07:00
543aac9f28
HADOOP-14386. Rewind trunk from Guava 21.0 back to Guava 11.0.2.
2017-05-09 09:22:53 -07:00
749e5c09b9
YARN-6281. Cleanup when AMRMProxy fails to initialize a new interceptor chain. (Botong Huang via Subru)
...
(cherry picked from commit 57a9afbd45b7ef8e6021cc58f96bd0074bf1389d)
2017-05-08 16:55:47 -07:00
cd9ff27ffc
YARN-6234. Support multiple attempts on the node when AMRMProxy is enabled. (Giovanni Matteo Fumarola via Subru).
2017-05-08 16:41:30 -07:00
1769b12a77
YARN-6457. Allow custom SSL configuration to be supplied in WebApps. (Sanjay M Pujare via Haibo Chen)
2017-05-08 15:46:12 -07:00
424887ecb7
YARN-3839. Quit throwing NMNotYetReadyException. Contributed by Manikandan R
2017-05-08 17:14:37 -05:00
d6eed5acca
YARN-6565. Fix memory leak and finish app trigger in AMRMProxy. (Botong Huang via Subru).
2017-05-05 16:27:49 -07:00
e4f34ecb04
HADOOP-14390. Correct spelling of 'succeed' and variants. Contributed by Dongtao Zhang
2017-05-05 12:10:50 -07:00
54e2b9e876
YARN-6375 App level aggregation should not consider metric values reported in the previous aggregation cycle (Varun Saxena via Vrushali C)
2017-05-04 15:25:56 -07:00
fd5cb2c946
YARN-6374. Improve test coverage and add utility classes for common Docker operations. Contributed by Shane Kumpf
2017-05-03 14:49:30 -07:00
9f0aea0ee2
YARN-6481. Yarn top shows negative container number in FS
...
(Contributed by Tao Jie via Daniel Templeton)
2017-05-02 13:06:47 -07:00
6bf42e48ef
YARN-5331. Extend RLESparseResourceAllocation with period for supporting recurring reservations in YARN ReservationSystem. (Sangeetha Abdu Jyothi via Subru).
2017-05-01 18:48:36 -07:00
e514fc432a
YARN-6542. Fix the logger in TestAlignedPlanner and TestGreedyReservationAgent.
2017-05-01 18:10:07 -07:00
a3a615eeab
YARN-4359. Update LowCost agents logic to take advantage of YARN-4358. (Jonathan Yaniv and Ishai Menache via Subru).
2017-05-01 16:01:07 -07:00
30fc580196
YARN-6519. Fix warnings from Spotbugs in hadoop-yarn-server-resourcemanager. Contributed by Weiwei Yang.
2017-05-01 20:15:27 +05:30
64f68cb0b8
YARN-6520. Fix warnings from Spotbugs in hadoop-yarn-client. Contributed by Weiwei Yang.
2017-05-01 18:38:22 +05:30
0f1af3178e
YARN-6518. Fix warnings from Spotbugs in hadoop-yarn-server-timelineservice. Contributed by Weiwei Yang.
2017-05-01 16:49:30 +05:30
4b5bd73ac5
YARN-6517. Fix warnings from Spotbugs in hadoop-yarn-common. Contributed by Weiwei Yang
2017-05-01 16:31:16 +05:30
2e52789edf
YARN-2962. ZKRMStateStore: Limit the number of znodes under a znode (Contributed by Varun Sexena via Daniel Templeton)
2017-04-28 13:30:28 -07:00
fdf5192bbb
YARN-6536. TestAMRMClient.testAMRMClientWithSaslEncryption fails intermittently. Contributed by Jason Lowe.
2017-04-28 15:20:35 -05:00
68e45f554b
YARN-6472. Improve Java sandbox regex (gphillips via rkanter)
2017-04-28 11:01:50 -07:00
371b6467dc
YARN-5894. fixed license warning caused by de.ruedigermoeller:fst:jar:2.24 (haibochen via rkanter)
2017-04-27 17:49:38 -07:00
94607214ce
YARN-6534. ResourceManager failed due to TimelineClient try to init SSLFactory even https is not enabled. Contributed by Rohith Sharma K S.
...
(cherry picked from commit 09e36ff59c05b8bd54d346d1103de0a7300875c6)
2017-04-27 13:11:32 -07:00
793bbf216d
YARN-6455. Enhance the timelinewriter.flush() race condition fix (Haibo Chen via Varun Saxena)
2017-04-27 15:01:58 +05:30
4f3ca0396a
YARN-6510. Fix profs stat file warning caused by process names that includes parenthesis. (Wilfred Spiegelenburg via Haibo Chen)
2017-04-26 11:46:55 -07:00
8a99eba96d
Revert "YARN-5007. Remove deprecated constructors of MiniYARNCluster and MiniMRYarnCluster. Contributed by Andras Bokor."
...
This reverts commit 34ab8e73d4
2017-04-26 15:34:23 +09:00
2ba21d6376
YARN-6392. Add submit time to Application Summary log. (Zhihai Xu via wangda)
2017-04-24 13:17:35 -07:00
8ac50e1322
YARN-6500. Do not mount inaccessible cgroups directories in CgroupsLCEResourcesHandler. (Miklos Szegedi via Haibo Chen)
2017-04-24 11:37:52 -07:00
52adf71914
YARN-5617. AMs only intended to run one attempt can be run more than once? Contributed by Jason Lowe.
2017-04-24 12:03:28 -05:00
561718e05d
YARN-6291. Introduce query parameters (sort, filter, etc.) for tables to keep state on refresh/navigation in new YARN UI. Contributed by Gergely Novák.
2017-04-24 13:28:57 +05:30
f356f0f4cf
YARN-6453. fairscheduler-statedump.log gets generated regardless of service
...
(Contributed by Yufei Gu via Daniel Templeton)
2017-04-20 14:13:07 -07:00
c0cf11e107
YARN-6402. Move 'Long Running Services' to an independent tab at top level for new Yarn UI. Contributed by Akhil PB.
2017-04-20 12:33:53 +05:30
46940d92e2
YARN-6302. Fail the node if Linux Container Executor is not configured properly
...
(Contributed by Miklos Szegedi via Daniel Templeton)
2017-04-19 12:23:49 -07:00
dd43b895c2
YARN-6202. Configuration item Dispatcher.DISPATCHER_EXIT_ON_ERROR_KEY is disregarded
...
(Contributed by Yufei Gu via Daniel Templeton)
2017-04-19 11:44:55 -07:00
41ac190dba
YARN-6438. Code can be improved in ContainersMonitorImpl.java
...
(Contributed by Miklos Szegedi via Daniel Templeton)
2017-04-19 10:53:08 -07:00
15acdbdba2
YARN-3427. Remove deprecated methods from ResourceCalculatorProcessTree
...
(Contributed by Miklos Szegedi via Daniel Templeton)
2017-04-19 10:31:14 -07:00
9ff25408bc
YARN-6164. Expose Queue Configurations per Node Label through YARN client api. Contributed by Benson Qiu.
2017-04-19 20:39:09 +05:30
af8e9842d2
YARN-6451. Add RM monitor validating metrics invariants. Contributed by Carlo Curino
2017-04-18 10:29:44 -07:00
6ed9d36224
YARN-6304. Skip rm.transitionToActive call to RM if RM is already active. Contributed by Rohith Sharma K S.
2017-04-17 15:51:10 -07:00
4168805502
YARN-6480. Timeout is too aggressive for TestAMRestart.testPreemptedAMRestartOnRMRestart. Contributed by Eric Badger
2017-04-14 17:15:48 -05:00
8a1d7480f7
YARN-6433. Only accessible cgroup mount directories should be selected for a controller. (Miklos Szegedi via kasha)
2017-04-14 15:07:14 -07:00
a41f8dd58e
YARN-5994. TestCapacityScheduler.testAMLimitUsage fails intermittently. Contributed by Eric Badger
2017-04-14 10:53:09 -05:00
c3375175d6
YARN-6432. FairScheduler: Reserve preempted resources for corresponding applications. (Miklos Szegedi via kasha)
2017-04-12 14:21:20 -07:00
9d9087a67a
YARN-3760. FSDataOutputStream leak in AggregatedLogFormat.LogWriter.close(). Contributed by Haibo Chen.
...
(cherry picked from commit c26ccf1adb3a72df3f68e1150b86b813c691203a)
2017-04-12 13:43:18 -07:00
a16ab2be91
YARN-6450. TestContainerManagerWithLCE requires override for each new test added to ContainerManagerTest? Contributed by Jason Lowe.
2017-04-12 10:20:08 -05:00
f48e968bec
YARN-6421. Upgrade frontend-maven-plugin to 1.1 to fix new YARN UI build error in ppc64le. Contributed by Sonia Garudi.
2017-04-12 12:01:47 +05:30
4d4ad0ebb7
YARN-6439. Fix ReservationSystem creation of default ReservationQueue. (Carlo Curino via wangda)
2017-04-11 14:56:18 -07:00
7d873c465b
YARN-6463. Correct spelling mistake in FileSystemRMStateStore. Contributed by Yeliang Cang.
2017-04-11 22:42:08 +05:30
90d97372ed
YARN-6461. TestRMAdminCLI has very low test timeouts. Contributed by Eric Badger
2017-04-11 09:16:11 -05:00
0e065f2ede
YARN-6195. Export UsedCapacity and AbsoluteUsedCapacity to JMX. Contributed by Benson Qiu
2017-04-11 08:44:18 -05:00
7999318af1
YARN-6344. Add parameter for rack locality delay in CapacityScheduler. (kkaranasos)
2017-04-10 15:34:44 -07:00
e9ac61cc0e
YARN-6372. Add default value for NM disk validator (Contributed by Yufei Gu via Daniel Templeton)
2017-04-10 14:56:42 -07:00
14a39907c6
YARN-6343. Docker docs MR example is broken (Contributed by Prashant Jha via Daniel Templeton)
2017-04-10 09:59:53 -07:00
443aa51bc1
YARN-5153. Add a toggle button to switch between timeline view / table view for containers and application-attempts in new YARN UI. Contributed by Akhil PB.
2017-04-10 13:35:08 +05:30
2aa8967809
YARN-6298. Metric preemptCall is not used in new preemption. (Yufei Gu via kasha)
2017-04-07 17:31:59 -07:00
63f7322522
YARN-6368. Decommissioning an NM results in a -1 exit code (miklos.szegedi@cloudera.com via rkanter)
2017-04-07 14:28:07 -07:00
96cbb4fce2
HADOOP-14202. fix jsvc/secure user var inconsistencies
...
Signed-off-by: John Zhuge <jzhuge@apache.org>
2017-04-07 08:59:21 -07:00
ad24464be8
YARN-6258. localBaseAddress for CORS proxy configuration is not working when suffixed with forward slash in new YARN UI. Contributed by Gergely Novák.
2017-04-07 11:42:50 +05:30
0116c3c957
YARN-5797. Add metrics to the node manager for cleaning the PUBLIC and PRIVATE caches. (Chris Trezzo via mingma)
2017-04-06 16:54:43 -07:00
1b081ca27e
YARN-6288. Exceptions during aggregated log writes are mishandled. Contributed by Akira Ajisaka
2017-04-06 16:24:36 -05:00
1a9439e299
YARN-6424. TimelineCollector is not stopped when an app finishes in RM. Contributed by Varun Saxena.
2017-04-06 10:15:22 +05:30
a2c57bb70d
YARN-6381. FSAppAttempt has several variables that should be final
...
(Contributed by Ameet Zaveri via Daniel Templeton)
2017-04-05 16:07:24 -07:00
539ef5aa2e
HFDS-11596. hadoop-hdfs-client jar is in the wrong directory in release tarball. Contributed by Yuanbo Liu.
2017-04-05 16:04:09 -07:00
b4c4f36594
YARN-6448. Continuous scheduling thread crashes while sorting nodes. (Yufei Gu via kasha)
2017-04-05 15:43:14 -07:00
7d963c477a
YARN-6437. TestSignalContainer#testSignalRequestDeliveryToNM fails intermittently (Jason Lowe via Varun Saxena)
2017-04-06 01:20:41 +05:30
87e2ef8c98
YARN-6406. Remove SchedulerRequestKeys when no more pending ResourceRequest. (Arun Suresh via wangda)
2017-04-05 11:28:30 -07:00
e8071aa249
YARN-6403. Invalid local resource request can raise NPE and make NM exit. Contributed by Tao Yang
2017-04-05 12:30:45 -05:00
34ab8e73d4
YARN-5007. Remove deprecated constructors of MiniYARNCluster and MiniMRYarnCluster. Contributed by Andras Bokor.
2017-04-05 17:33:07 +09:00
19b89c4c7b
YARN-6420. RM startup failure due to wrong order in nodelabel editlog (Bibin A Chundatt via Varun Saxena)
2017-04-05 09:38:35 +05:30
2d5c09b848
YARN-6004. Refactor TestResourceLocalizationService#testDownloadingResourcesOnContainer so that it is less than 150 lines. (Chris Trezzo via mingma)
2017-04-04 17:56:21 -07:00
9cc04b4708
YARN-6436. TestSchedulingPolicy#testParseSchedulingPolicy timeout is too low. (Eric Badger via kasha)
2017-04-04 17:08:33 -07:00
3fdae0a2b6
YARN-6109. Add an ability to convert ChildQueue to ParentQueue. (Xuan Gong via wangda)
2017-04-04 14:39:01 -07:00
6eba79232f
HADOOP-14271. Correct spelling of 'occurred' and variants. Contributed by Yeliang Cang
2017-04-03 20:13:14 -07:00
a4b5aa8493
YARN-6377. NMTimelinePublisher#serviceStop does not stop timeline clients (Haibo Chen via Varun Saxena)
2017-04-02 04:54:12 +05:30
b8838578ce
YARN-6414. ATSv2 HBase related tests fail due to guava version upgrade (Haibo Chen via Varun Saxena)
2017-04-02 04:37:34 +05:30
318bfb01bc
YARN-6354. LeveldbRMStateStore can parse invalid keys when recovering reservations. Contributed by Jason Lowe
2017-03-31 12:30:35 -05:00
4d1fac5df2
YARN-6411. Clean up the overwrite of createDispatcher() in subclass of MockRM. Contributed by Yufei Gu
2017-03-31 10:05:34 -05:00
b58777a9c9
YARN-6376. Exceptions caused by synchronous putEntities requests can be swallowed (Haibo Chen via Varun Saxena)
2017-03-31 02:17:20 +05:30
7c2bc444b3
YARN-6342. Make TimelineV2Client's drain timeout after stop configurable (Haibo Chen via Varun Saxena)
2017-03-31 02:02:57 +05:30
7297e82cb4
HADOOP-14250. Correct spelling of 'separate' and variants. Contributed by Doris Gu
2017-03-30 10:17:11 -07:00
640ba1d23f
YARN-5685. RM configuration allows all failover methods to disabled when automatic failover is enabled
2017-03-29 12:38:11 -07:00
82fb9ce8df
YARN-6329. Remove unnecessary TODO comment from AppLogAggregatorImpl.java. Contributed by victor bertschinger.
2017-03-29 10:36:24 +09:00
063b513b1c
YARN-6357. Implement putEntitiesAsync API in TimelineCollector (Haibo Chen via Varun Saxena)
2017-03-29 03:48:03 +05:30
01aca54a22
YARN-5368. Memory leak in timeline server (Jonathan Eagles via Varun Saxena)
2017-03-29 01:53:20 +05:30
fdf8f8ebca
YARN-6359. TestRM#testApplicationKillAtAcceptedState fails rarely due to race condition. Contributed by Robert Kanter
2017-03-28 09:33:26 -05:00
9bae6720cb
YARN-6050. AMs can't be scheduled on racks or nodes (rkanter)
2017-03-27 17:23:13 -07:00
cd014d57aa
YARN-6339. Improve performance for createAndGetApplicationReport. (Yunjiong Zhao via wangda)
2017-03-27 13:29:09 -07:00
b63ae854f9
YARN-6379. Remove unused argument in ClientRMService. Contributed by Kai Sasaki.
2017-03-27 11:23:41 +09:00
6df029db36
Add COMMON/MAPREDUCE/YARN jdiff of 2.8.0 release.
2017-03-25 14:28:48 -07:00
cffea251be
YARN-5956. Refactor ClientRMService for unify error handling across apis. Contributed by Kai Sasaki.
2017-03-25 22:26:19 +05:30
84ddedc0b2
HADOOP-10101. Update guava dependency to the latest version. (ozawa)
2017-03-25 10:06:52 +09:00
33815af424
YARN-6334. TestRMFailover#testAutomaticFailover always passes even when it should fail
...
(Contributed by Yufei Gu via Daniel Templeton)
2017-03-24 11:42:54 -07:00
d4f73e7e27
HDFS-11574. Spelling mistakes in the Java source. Contributed by Hu Xiaodong.
2017-03-24 09:38:17 -07:00
f462e1ff68
YARN-5934. Fix TestTimelineWebServices.testPrimaryFilterNumericString (Akira Ajisaka via Varun Saxena)
2017-03-22 04:11:53 +05:30
0a05c5c598
YARN-6284. hasAlreadyRun should be final in ResourceManager.StandByTransitionRunnable
...
(Contributed by Laura Adams via Daniel Templeton)
2017-03-21 14:15:40 -07:00
c01d15ab27
YARN-6367. YARN logs CLI needs alway check containerLogsInfo/containerLogInfo before parse the JSON object from NMWebService. Contributed by Xuan Gong.
2017-03-21 10:53:27 -07:00
1a98a279f4
YARN-6362. Use frontend-maven-plugin 0.0.22 version for new yarn ui. Contributed by Kai Sasaki.
2017-03-21 13:46:31 +05:30
49efd5d204
YARN-6309. Fair scheduler docs should have the queue and queuePlacementPolicy
...
elements listed in bold so that they're easier to see
(Contributed by Esmaeil Mirzaee via Daniel Templeton)
2017-03-20 14:25:00 -07:00
35034653d0
YARN-6353. Clean up OrderingPolicy javadoc (Daniel Templeton via Varun Saxena)
2017-03-20 23:29:09 +05:30
4a8e304502
YARN-6217. TestLocalCacheDirectoryManager test timeout is too aggressive. Contributed by Miklos Szegedi
2017-03-17 14:06:35 -05:00
7114baddb6
YARN-4051. ContainerKillEvent lost when container is still recovering and application finishes. Contributed by sandflee
2017-03-16 09:30:10 -05:00
6d95866dcf
YARN-6332. Make RegistrySecurity use short user names for ZK ACLs. Contributed by Billie Rinaldi
2017-03-16 12:59:55 +08:00
221783b35b
YARN-6294. ATS client should better handle Socket closed case. Contributed by Li Lu.
2017-03-15 11:48:17 -07:00
bb6a214cd3
YARN-6336. Jenkins report YARN new UI build failure. Contributed by Sunil G.
2017-03-15 15:54:09 +05:30
2d0e24eb01
YARN-6328. Fix a spelling mistake in CapacityScheduler. Contributed by Jin Yibo. This closes #202
2017-03-15 14:48:05 +05:30
4c66a8d19b
YARN-6331. Fix flakiness in TestFairScheduler#testDumpState. (Yufei Gu via rchiang)
2017-03-14 15:09:47 -07:00
871dc420f8
YARN-6313. YARN logs cli should provide logs for a completed container even when application is still running. Contributed by Xuan Gong.
...
(cherry picked from commit b88f5e0f7858d1d89b79dfd325b767c34416052d)
2017-03-14 12:58:12 -07:00
0a3aa40fe7
YARN-6327. Removing queues from CapacitySchedulerQueueManager and ParentQueue should be done with iterator. Contributed by Jonathan Hung.
2017-03-15 01:22:25 +05:30
34424e98a6
YARN-6314. Potential infinite redirection on YARN log redirection web service. Contributed by Xuan Gong.
...
(cherry picked from commit 5a9dda796f0e73060ada794ad5752cc6a237ab2e)
2017-03-14 02:58:07 -07:00
e5f2eedcbf
YARN-5496. Make Node Heatmap Chart categories clickable in new YARN UI. Contributed by Gergely Novák.
2017-03-14 11:47:11 +05:30
4db9cc70d0
YARN-6042. Dump scheduler and queue state information into FairScheduler DEBUG log. (Yufei Gu via rchiang)
2017-03-10 16:13:31 -08:00
229c7c9f89
YARN-6069. CORS support in timeline v2 (Rohith Sharma K S via Varun Saxena)
2017-03-11 04:09:01 +05:30
9649c27864
YARN-6321. TestResources test timeouts are too aggressive. Contributed by Eric Badger
2017-03-10 13:06:54 -06:00
4478273e5f
HADOOP-14062. ApplicationMasterProtocolPBClientImpl.allocate fails with EOFException when RPC privacy is enabled. Contributed by Steven Rand
2017-03-10 09:25:58 -08:00
deb9f56946
YARN-6310. OutputStreams in AggregatedLogFormat.LogWriter can be left open upon exceptions. Contributed by Haibo Chen
2017-03-10 11:08:33 -06:00
e06ff18ab6
YARN-6196. Improve Resource Donut chart with better label in Node page of new YARN UI. Contributed by Akhil PB.
2017-03-10 16:17:48 +05:30
a96afae125
YARN-6264. AM not launched when a single vcore is available on the cluster. (Yufei Gu via kasha)
2017-03-09 23:11:54 -08:00
846a0cd678
YARN-1047. Expose # of pre-emptions as a queue counter (Contributed by Karthik Kambatla via Daniel Templeton)
2017-03-09 17:51:47 -08:00
e96a0b8c92
YARN-5669. Add support for docker pull command (Contribtued by luhuichun)
2017-03-09 16:22:19 -08:00
822a74f2ae
YARN-6300. NULL_UPDATE_REQUESTS is redundant in TestFairScheduler (Contributed by Yuanbo Liu via Daniel Templeton)
2017-03-09 12:14:33 -08:00
d7762a5511
YARN-6165. Intra-queue preemption occurs even when preemption is turned off for a specific queue. Contributed by Eric Payne
2017-03-08 16:46:09 -06:00
2be8947d12
Revert "HADOOP-14062. ApplicationMasterProtocolPBClientImpl.allocate fails with EOFException when RPC privacy is enabled. Contributed by Steven Rand"
...
This reverts commit 241c1cc05b
2017-03-08 13:20:01 -08:00
241c1cc05b
HADOOP-14062. ApplicationMasterProtocolPBClientImpl.allocate fails with EOFException when RPC privacy is enabled. Contributed by Steven Rand
2017-03-08 10:48:27 -08:00
287ba4ffa6
YARN-6297. TestAppLogAggregatorImp.verifyFilesUploaded() should check # of filed uploaded with that of files expected (haibochen via rkanter)
2017-03-08 10:45:33 -08:00
1eb8186703
YARN-6207. Move application across queues should handle delayed event processing. Contributed by Bibin A Chundatt.
2017-03-08 12:04:30 +05:30
e0c239cdbd
YARN-6287. RMCriticalThreadUncaughtExceptionHandler.rmContext should be final (Contributed by Corey Barker via Daniel Templeton)
2017-03-07 11:58:48 -08:00
c1386a90d7
YARN-6248. user is not removed from UsersManager’s when app is killed with pending container requests. Contributed by Eric Payne.
2017-03-04 13:46:35 +05:30
c61bc2d843
YARN-6278. Enforce to use correct node and npm version in new YARN-UI build. (Sunil G via wangda)
2017-03-03 23:43:25 -08:00
279d187f72
YARN-6270. WebUtils.getRMWebAppURLWithScheme() needs to honor RM HA setting. Contributed by Xuan Gong
2017-03-03 15:24:04 -08:00
8db7a8c3ae
YARN-6271. yarn rmadin -getGroups returns information from standby RM. Contributed by Jian He.
2017-03-03 14:39:31 -08:00
2148b83993
YARN-6218. Fix TestAMRMClient when using FairScheduler. (Miklos Szegedi via rchiang)
2017-03-03 13:04:51 -08:00
e58fc76030
YARN-6263. NMTokenSecretManagerInRM.createAndGetNMToken is not thread safe. Contributed by Haibo Chen
2017-03-03 08:23:08 -06:00
6f6dfe0202
YARN-5280. Allow YARN containers to run with Java Security Manager (gphillips via rkanter)
2017-03-01 14:53:47 -08:00
235203dffd
YARN-6153. KeepContainer does not work when AM retry window is set. Contributed by kyungwan nam
2017-02-28 13:23:36 -08:00
e0bb867c3f
YARN-6189: Improve application status log message when RM restarted when
...
app is in NEW state. Contributed by Junping Du
2017-02-28 11:04:56 -08:00
eac6b4c35c
YARN-6216. Unify Container Resizing code paths with Container Updates making it scheduler agnostic. (Arun Suresh via wangda)
2017-02-28 10:35:50 -08:00
480b4dd574
YARN-1728. Regression test for guice-undecoded pathInfo in YARN WebApp. (Yuanbo Liu via gera)
2017-02-28 09:27:22 -08:00
0f35443bf2
YARN-6239. Fix javadoc warnings in YARN that caused by deprecated FileSystem APIs. Contributed by Yiqun Lin.
2017-02-28 18:51:56 +08:00
ef488044d0
YARN-5335. Use em-table in app/nodes pages for new YARN UI. Contributed by Sunil G.
2017-02-28 13:49:19 +05:30
f187d63816
YARN-6231. FairSchedulerTestBase helper methods should call scheduler.update to avoid flakiness. (kasha)
2017-02-27 22:10:57 -08:00
5f5b031d1f
YARN-5703. ReservationAgents are not correctly configured. Contributed by Manikandan R.
2017-02-27 20:38:29 +05:30
fbfe86deea
YARN-6172. FSLeafQueue demand update needs to be atomic. (Miklos Szegedi via kasha)
2017-02-26 20:36:33 -08:00
815d53506f
YARN-6215. FairScheduler preemption and update should not run concurrently. (Tao Jie via kasha)
2017-02-26 20:16:36 -08:00
53d372a255
YARN-6228: EntityGroupFSTimelineStore should allow configurable cache
...
stores. Contributed by Li Lu
2017-02-24 10:59:35 -08:00
694e680d20
YARN-6222. TestFairScheduler.testReservationMetrics is flaky. (Yufei Gu via kasha)
2017-02-23 15:21:52 -08:00
a207aa9930
YARN-6211. Synchronization improvement for moveApplicationAcrossQueues and updateApplicationPriority. Contributed by Bibin A Chundatt.
2017-02-23 14:19:07 +05:30
b10e962224
YARN-6194. Cluster capacity in SchedulingPolicy is updated only on allocation file reload. (Yufei Gu via kasha)
2017-02-22 15:58:49 -08:00
718ad9f6ee
YARN-6210. FairScheduler: Node reservations can interfere with preemption. (kasha)
2017-02-22 15:46:07 -08:00
1a6ca75f38
YARN-6143. Fix incompatible issue caused by YARN-3583. (Sunil G via wangda)
2017-02-22 11:17:09 -08:00
f1c9cafefc
YARN-6184. Introduce loading icon in each page of new YARN UI. Contributed by Akhil PB.
2017-02-22 11:54:32 +05:30
6ba61d20d3
YARN-6159. Documentation changes for TimelineV2Client (Naganarasimha G R via Varun Saxena)
2017-02-21 12:25:37 +05:30
172b23af33
HADOOP-14077. Add ability to access jmx via proxy. Contributed by Yuanbo Liu.
2017-02-18 18:34:13 -08:00
dbbfcf74ab
YARN-6193. FairScheduler might not trigger preemption when using DRF. (kasha)
2017-02-17 14:07:31 -08:00
c7a36e6130
YARN-6125. The application attempt's diagnostic message should have a maximum size
...
(Contributed by Andras Piros via Daniel Templeton)
2017-02-17 13:40:58 -08:00
9a92837786
YARN-6188. Fix OOM issue with decommissioningNodesWatcher in the case of clusters with
...
large number of nodes (Contributed by Ajay Jadhav via Daniel Templeton)
2017-02-17 13:13:46 -08:00
a77f432449
YARN-6171. ConcurrentModificationException on FSAppAttempt.containersToPreempt. (Miklos Szegedi via kasha)
2017-02-16 14:54:58 -08:00
5d339c46f5
YARN-6177. Yarn client should exit with an informative error message if an incompatible Jersey library is used at client. Contributed by Weiwei Yang.
2017-02-16 13:41:42 -08:00
4fa1afdb88
YARN-4675. Reorganize TimelineClient and TimelineClientImpl into separate classes for ATSv1.x and ATSv2. Contributed by Naganarasimha G R.
2017-02-16 11:41:04 -08:00
e63a7814d2
YARN-6200. addendum to fix compilation error caused by reverting YARN-5068.
2017-02-16 18:19:57 +05:30
a393e84c6f
YARN-6200. Reverting since the same functionality achieved by YARN-1623.
...
Revert "YARN-5068. Expose scheduler queue to application master. (Harish Jaiprakash via rohithsharmaks)"
This reverts commit b7ac85259c
2017-02-16 18:00:25 +05:30
74dd142250
YARN-5798. Set UncaughtExceptionHandler for all FairScheduler threads. (Yufei Gu via kasha)
2017-02-16 00:03:09 -08:00
11be3f70e0
YARN-4212. FairScheduler: Can't create a DRF queue under a FAIR policy queue. (Yufei Gu via kasha)
2017-02-15 23:51:22 -08:00
6c25dbcdc0
YARN-6163. FS Preemption is a trickle for severely starved applications. (kasha)
2017-02-15 23:16:12 -08:00
0fc6f38379
Addendum patch for YARN-6174 - remove deleted file.
2017-02-15 09:07:49 -08:00
ce2d5bfa5f
YARN-6174. Log files pattern should be same for both running and finished container. Contributed by Xuan Gong.
2017-02-15 09:05:14 -08:00
ff75132355
YARN-6183. Few missing informations in Application and Application Attempt pages for new YARN UI. Contributed by Akhil PB.
2017-02-15 15:52:57 +05:30
b7613e0f40
YARN-6156. AM blacklisting to consider node label partition (Bibin A Chundatt via Varun Saxena)
2017-02-15 14:48:17 +05:30
fbc0c2bd76
YARN-4753. Use doxia macro to generate in-page TOC of YARN site documentation. (iwasakims)
2017-02-15 13:09:10 +09:00
353a9b2d91
YARN-6061. Addendum. Remove extraneous change.
2017-02-14 15:19:52 -08:00
652679aa8a
YARN-6061. Add an UncaughtExceptionHandler for critical threads in RM. (Yufei Gu via kasha)
2017-02-14 13:39:41 -08:00
1fa084c425
YARN-5912. Fix breadcrumb issues for various pages in new YARN UI. Contributed by Akhil P B.
2017-02-14 22:29:21 +05:30
aaf106fde3
YARN-5966. AMRMClient changes to support ExecutionType update. (asuresh)
2017-02-14 06:09:10 -08:00
646c6d6509
YARN-3933. FairScheduler: Multiple calls to completedContainer are not safe. (Shiwei Guo and Miklos Szegedi via kasha)
2017-02-13 11:26:30 -08:00
464ff479ce
YARN-6113. Re-direct NM Web Service to get container logs for finished applications. Contributed by Xuan Gong.
2017-02-13 06:12:54 -08:00
649deb72fb
YARN-6170. TimelineReaderServer should wait to join with HttpServer2 (Sangjin Lee via Varun Saxena)
2017-02-11 19:21:45 +05:30
1238fa71f8
YARN-6166. Unnecessary INFO logs in AMRMClientAsyncImpl$CallbackHandlerThread.run (Contributed by Grant Whiteheart via Daniel Templeton)
2017-02-10 09:46:11 -08:00
3a0a0a485a
YARN-6118. Add javadoc for Resources.isNone. Contributed by Andres Perez.
2017-02-10 15:48:46 +05:30
08f93978f3
YARN-6144. FairScheduler: preempted resources can become negative. (Miklos Szegedi via kasha)
2017-02-09 14:45:04 -08:00
9b85053583
YARN-6112. UpdateCallDuration is calculated only when debug logging is enabled. (Yufei Gu via kasha)
2017-02-09 13:09:32 -08:00
5fb723bb77
YARN-5889. Improve and refactor user-limit calculation in Capacity Scheduler. (Sunil G via wangda)
2017-02-09 10:23:50 -08:00
37b4acf7ce
YARN-6137. Yarn client implicitly invoke ATS client which accesses HDFS. Contributed by Li Lu
2017-02-08 14:59:09 -06:00
eec52e158b
YARN-6145. Improve log message on fail over. Contributed by Jian He.
2017-02-08 02:11:19 -08:00
a87e3850b9
YARN-6135. Node manager REST API documentation is not up to date (Contributed by Miklos Szegedi via Daniel Templeton)
2017-02-07 16:41:02 -08:00
307ec58ffb
YARN-5258. Document Use of Docker with LinuxContainerExecutor (Contributed by Daniel Templeton)
2017-02-07 16:29:21 -08:00
7a8f3f237e
YARN-5917. Make navigation link active when selecting sub tabs in 'Applications' and 'Nodes' page for new UI. Contributed by Kai Sasaki.
2017-02-07 21:45:40 +05:30
9dbfab1284
YARN-6108. Improve AHS webservice to accept NM address as a parameter to get container logs. Contributed by Xuan Gong.
2017-02-07 02:13:14 -08:00
cce35c3815
HADOOP-12097. Allow port range to be specified while starting webapp. Contributed by Varun Saxena.
2017-02-05 19:42:11 -08:00
327c9980aa
YARN-6100. Improve YARN webservice to output aggregated container logs. Contributed by Xuan Gong.
2017-02-02 00:41:18 -08:00
3e06475307
YARN-4658. Typo in o.a.h.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler comment (Contributed by Udai Potluri via Daniel Templeton)
2017-01-31 16:25:33 -08:00
4c6bae5a22
YARN-5866. Fix few issues reported by jshint in new YARN UI. Contributed by Akhil P B.
2017-01-31 14:18:31 +05:30
87852b6ef4
YARN-6103. Log updates for ZKRMStateStore (Contributed by Daniel Sturman via Daniel Templeton)
2017-01-30 21:44:48 -08:00
11e44bdda7
YARN-6131. FairScheduler: Lower update interval for faster tests. (kasha)
2017-01-30 13:35:23 -08:00
1a16431bd0
YARN-6126. Obtaining app logs for Running application fails with json parse error. Contributed by Xuan Gong.
2017-01-27 06:31:10 -08:00
165f07f51a
YARN-6123. [YARN-5864] Add a test to make sure queues of orderingPolicy will be updated when childQueues is added or removed. Contributed by Wangda Tan.
2017-01-27 18:37:51 +05:30
f85b74ccf9
YARN-4975. Fair Scheduler: exception thrown when a parent queue marked 'parent' has configured child queues
...
(Contributed by Yufei Gu via Daniel Templeton)
2017-01-26 10:31:09 -08:00
425a7e5028
YARN-3637. Handle localization sym-linking correctly at the YARN level. Contributed by Chris Trezzo.
2017-01-25 15:51:36 -08:00
9e19f758c1
YARN-5641. Localizer leaves behind tarballs after container is complete. Contributed by Eric Badger
2017-01-25 21:41:43 +00:00
abedb8a9d8
YARN-5830. FairScheduler: Avoid preempting AM containers. (Yufei Gu via kasha)
2017-01-25 12:17:28 -08:00
8528d85a68
YARN-6099. Improve webservice to list aggregated log files. Contributed by Xuan Gong.
2017-01-24 15:26:53 -08:00
a33ce45e35
YARN-5547. NMLeveldbStateStore should be more tolerant of unknown keys. Contributed by Ajith S
2017-01-24 16:17:36 +00:00
0101267d9d
YARN-6082. Invalid REST api response for getApps since queueUsagePercentage is coming as INF. Contributed by Sunil G.
2017-01-24 10:59:55 +05:30
dc6ec97048
YARN-6117. SharedCacheManager does not start up. Contributed by Chris Trezzo.
2017-01-23 21:07:25 -08:00
a2c50127d1
YARN-6012. Remove node label (removeFromClusterNodeLabels) document is missing (Contributed by Ying Zhang via Daniel Templeton)
2017-01-23 17:42:54 -08:00
ce832059db
YARN-5864. Capacity Scheduler - Queue Priorities. (wangda)
2017-01-23 10:52:14 -08:00
69fa81679f
YARN-5910. Support for multi-cluster delegation tokens. Contributed by Jian He
2017-01-23 17:12:51 +00:00
3fa0d540df
YARN-6031. Application recovery has failed when node label feature is turned off during RM recovery. Contributed by Ying Zhang.
2017-01-23 12:19:35 +05:30
9bab85cadf
YARN-6106. Document FairScheduler 'allowPreemptionFrom' queue property. (Yufei Gu via rchiang)
2017-01-20 14:08:25 -08:00
b01514f65b
YARN-5928. Move ATSv2 HBase backend code into a new module that is only dependent at runtime by yarn servers. Contributed by Haibo Chen.
2017-01-19 20:52:55 -08:00
5d8b80ea9b
Preparing for 3.0.0-alpha3 development
2017-01-19 15:50:07 -08:00
efc8faa1ba
YARN-6104. RegistrySecurity overrides zookeeper sasl system properties. Contributed by Billie Rinaldi
2017-01-19 10:18:59 -08:00
7e8d32147c
YARN-6110. Fix opportunistic containers documentation. (Akira Ajisaka via asuresh)
2017-01-19 06:14:28 -08:00
0eb4b513b7
HADOOP-13673. Update scripts to be smarter when running with privilege
...
Signed-off-by: Andrew Wang <wang@apache.org>
Signed-off-by: Ravi Prakash <raviprak@apache.org>
2017-01-18 14:39:05 -08:00
72054a817d
YARN-5556. CapacityScheduler: Support deleting queues without requiring a RM restart. (Naganarasimha G R via wangda)
2017-01-18 13:31:33 -08:00
e224c96234
YARN-5831. FairScheduler: Propagate allowPreemptionFrom flag all the way down to the app. (Yufei Gu via kasha)
2017-01-17 17:01:31 -08:00
4d1f3d9020
YARN-6016. Fix minor bugs in handling of local AMRMToken in AMRMProxy. (Botong Huang via Subru).
2017-01-17 14:48:03 -08:00
695f110f93
YARN-6071. Fix incompatible API change on AM-RM protocol due to YARN-3866 (trunk only)
...
(Contributed by Wangda Tan via Daniel Templeton)
2017-01-17 12:55:47 -08:00
a9d946d4df
YARN-6057. yarn.scheduler.minimum-allocation-* descriptions are incorrect about behavior when a request is out of bounds. Contributed by Julia Sommer.
2017-01-17 14:16:00 +05:30
e407449ddb
HADOOP-13933. Add haadmin -getAllServiceState option to get the HA state of all the NameNodes/ResourceManagers. Contributed by Surendra Singh Lilhore.
2017-01-17 10:10:23 +09:00
cf695577aa
YARN-6011. Add a new web service to list the files on a container in AHSWebService. Contributed by Xuan Gong.
2017-01-16 16:20:24 -08:00
d3170f9eba
YARN-6081. LeafQueue#getTotalPendingResourcesConsideringUserLimit should deduct reserved from pending to avoid unnecessary preemption of reserved container. Contributed by Wangda Tan.
2017-01-13 18:22:29 +05:30
a6b06f7179
YARN-6072. RM unable to start in secure mode. Contributed by Ajith S.
2017-01-12 06:40:41 +05:30
e6f13fe5d1
YARN-5849. Automatically create YARN control group for pre-mounted cgroups (Contributed by Miklos Szegedi via Daniel Templeton)
2017-01-11 16:00:01 -08:00
7979939428
YARN-5554. MoveApplicationAcrossQueues does not check user permission on the target queue
...
(Contributed by Wilfred Spiegelenburg via Daniel Templeton)
2017-01-11 14:34:10 -08:00
357eab9566
YARN-5416. TestRMRestart#testRMRestartWaitForPreviousAMToFinish failed intermittently due to not wait SchedulerApplicationAttempt to be stopped. Contributed by Junping Du
2017-01-11 17:13:37 +00:00
4db119b7b5
YARN-6079. Fix simple spelling errors in yarn test code. Contributed by vijay.
2017-01-10 18:20:27 -08:00
4c431a6940
YARN-6054. TimelineServer fails to start when some LevelDb state files are missing. Contributed by Ravi Prakash.
2017-01-10 15:54:16 +05:30
41db07d532
YARN-5937. stop-yarn.sh is not able to gracefully stop node managers. Contributed by Weiwei Yang.
2017-01-10 15:35:01 +05:30
9594c35dcb
YARN-5899. Debug log in AbstractCSQueue#canAssignToThisQueue needs improvement. Contributed by Ying Zhang.
2017-01-10 14:08:01 +05:30
6332a318bc
YARN-6073. Misuse of format specifier in Preconditions.checkArgument (Contributed by Yuanbo Liu via Daniel Templeton)
2017-01-09 22:12:58 -08:00
945db55f2e
YARN-4148. When killing app, RM releases app's resource before they are released by NM. Contributed by Jason Lowe.
2017-01-09 18:14:46 -08:00
287d3d6804
YARN-3955. Support for application priority ACLs in queues of CapacityScheduler. (Sunil G via wangda)
2017-01-09 08:40:39 -08:00
db490eccce
YARN-6074. FlowRunEntity does not deserialize long values correctly (Rohith Sharma K S via Varun Saxena)
2017-01-09 18:54:22 +05:30
f59e36b4ce
YARN-6068. Log aggregation get failed when NM restart even with recovery (Junping Du via Varun Saxena)
2017-01-09 11:17:19 +05:30
85826f6ca5
YARN-6066. Opportunistic containers Minor fixes : API annotations, parameter name changes, checkstyles. (asuresh)
...
(cherry picked from commit 4985217de4
2017-01-08 09:04:20 -08:00
ac16400e1f
YARN-6022. Revert changes of AbstractResourceRequest (Contributed by Wangda Tan via Daniel Templeton)
2017-01-07 08:24:30 -08:00
a59df15757
YARN-6015. AsyncDispatcher thread name can be set to improved debugging. Contributed by Ajith S.
2017-01-07 08:11:58 +05:30
020316458d
YARN-6009. Skip validating app timeout value on recovery. Contributed by Rohith Sharma K S
2017-01-06 18:19:09 -08:00
71a4acf74b
YARN-5991. Yarn Distributed Shell does not print throwable t to App Master When failed to start container
...
(Contributed by Jim Frankola via Daniel Templeton)
2017-01-06 14:26:08 -08:00
2977bc6a14
YARN-6040. Introduce api independent PendingAsk to replace usage of ResourceRequest within Scheduler classes. (Wangda Tan via asuresh)
2017-01-06 09:59:57 -08:00
0a55bd841e
YARN-5959. RM changes to support change of container ExecutionType. (Arun Suresh via wangda)
2017-01-05 10:31:05 -08:00
e49e0a6e37
YARN-5988. RM unable to start in secure setup. Contributed by Ajith S.
2017-01-04 12:09:29 +05:30
8fadd69047
YARN-5923. Unable to access logs for a running application if YARN_ACL_ENABLE is enabled. Contributed by Xuan Gong.
2017-01-03 15:03:38 -08:00
f69a107aec
YARN-6025. Fix synchronization issues of AbstractYarnScheduler#nodeUpdate and its implementations. (Naganarasimha G R via wangda)
...
(cherry picked from commit e0f2379312
2017-01-03 14:53:36 -08:00
ab1faa4ba8
Revert "Fix synchronization issues of AbstractYarnScheduler#nodeUpdate and its implementations. (Naganarasimha G R via wangda)" to add correct JIRA number
...
This reverts commit e0f2379312
2017-01-03 14:53:13 -08:00
e0f2379312
Fix synchronization issues of AbstractYarnScheduler#nodeUpdate and its implementations. (Naganarasimha G R via wangda)
2017-01-03 14:46:41 -08:00
165d01a73e
YARN-5931. Document timeout interfaces CLI and REST APIs (Contributed by Rohith Sharma K S via Daniel Templeton)
2016-12-30 08:41:52 -08:00
f216276d21
YARN-4882. Change the log level to DEBUG for recovering completed applications (templedf via rkanter)
2016-12-28 15:21:52 -08:00
9ca54f4810
YARN-5906. Update AppSchedulingInfo to use SchedulingPlacementSet. Contributed by Wangda Tan.
2016-12-28 22:48:19 +05:30
972da46cb4
YARN-5719. Enforce a C standard for native container-executor. Contributed by Chris Douglas.
2016-12-28 14:59:57 +05:30
f6715b26b6
YARN-6024. Capacity Scheduler 'continuous reservation looking' doesn't work when sum of queue's used and reserved resources is equal to max. Contributed by Wangda Tan.
2016-12-28 12:33:02 +05:30
e297be74ed
YARN-6001. Improve moveApplicationQueues command line. Contributed by Sunil G.
2016-12-28 12:16:11 +05:30
0840b4329b
YARN-5756. Add state-machine implementation for scheduler queues. (Xuan Gong via wangda)
2016-12-27 21:18:33 -08:00
9262797e86
YARN-5257. Fix unreleased resources and null dereferences (yufeigu via rkanter)
2016-12-27 14:14:08 -08:00
1bbd023275
YARN-5962. Spelling errors in logging and exceptions for resource manager code (gsohn via rkanter)
2016-12-27 13:35:34 -08:00
e216e8e233
HADOOP-13932. Fix indefinite article in comments (Contributed by LiXin Ge via Daniel Templeton)
2016-12-27 13:03:16 -08:00
ac1e5d4f77
YARN-5938. Refactoring OpportunisticContainerAllocator to use SchedulerRequestKey instead of Priority and other misc fixes (asuresh)
2016-12-27 12:40:44 -08:00
c3973e7080
YARN-5969. FairShareComparator: Cache value of getResourceUsage for better performance. (zhangshilong via kasha)
2016-12-27 12:30:35 -08:00
ded2d08f33
HADOOP-13940. Document the missing envvars commands (Contributed by Yiqun Lin via Daniel Templeton)
2016-12-27 07:18:08 -08:00
c721f78a3c
YARN-6026. A couple of spelling errors in the docs. Contributed by Grant Sohn.
2016-12-24 05:38:09 +05:30
4d3f73acc0
YARN-6000. Make AllocationFileLoaderService.Listener public. (Tao Jie via kasha)
2016-12-23 11:40:56 -08:00
38e66d4d64
YARN-5903. Fix race condition in TestResourceManagerAdministrationProtocolPBClientImpl beforeclass setup method (Haibo Chen via Varun Saxena)
2016-12-22 23:08:33 +05:30
ae401539ea
YARN-4994. Use MiniYARNCluster with try-with-resources in tests. Contributed by Andras Bokor.
2016-12-22 14:32:24 +09:00
736f54b727
YARN-4990. Re-direction of a particular log file within in a container in NM UI does not redirect properly. Contributed by Xuan Gong.
2016-12-21 14:14:42 -08:00
8b042bc1e6
YARN-5976. Update hbase version to 1.2. Contributed by Vrushali C.
2016-12-21 09:43:17 -08:00
575773a357
YARN-5877. Allow all env's from yarn.nodemanager.env-whitelist to get overridden during launch. Contributed by Bibin A Chundatt.
2016-12-20 11:39:06 +05:30
358341398a
YARN-5524. Yarn live log aggregation does not throw if command line arg is wrong. Contributed by Xuan Gong.
2016-12-20 06:36:25 +05:30
ef2dd7b78c
YARN-5650. Render Application Timeout value in web UI. Contributed by Akhil PB.
2016-12-19 11:44:03 +05:30
2273a74c1f
YARN-5646. Add documentation and update config parameter names for scheduling of OPPORTUNISTIC containers. (Konstantinos Karanasos via asuresh)
2016-12-16 08:14:34 -08:00
64a2d5be91
YARN-5999. AMRMClientAsync will stop if any exceptions thrown on allocate call. Contributed by Jian He
2016-12-14 14:33:23 -08:00
ada876cd1d
Revert YARN-4126. RM should not issue delegation tokens in unsecure mode.
2016-12-13 20:49:54 -08:00
f66f61892a
HADOOP-13852 hadoop build to allow hadoop version property to be explicitly set. Contriibuted by Steve Loughran
2016-12-12 13:58:27 +00:00
4b149a1e77
YARN-4457. Cleanup unchecked types for EventHandler (templedf via rkanter)
2016-12-09 17:28:25 -08:00
a6410a542e
YARN-5709. Cleanup leader election configs and pluggability. Contribtued by Karthik Kambatla
2016-12-09 17:00:37 -08:00
b0aace21b1
YARN-5982. Simplify opportunistic container parameters and metrics. (Konstantinos Karanasos via asuresh)
2016-12-09 16:41:25 -08:00
55f5886ea2
YARN-5925. Extract hbase-backend-exclusive utility methods from TimelineStorageUtil. Contributed by Haibo Chen.
2016-12-09 16:17:24 -08:00
7d8e440eee
Revert "HADOOP-13852 hadoop build to allow hadoop version property to be explicitly set. Contriibuted by Steve Loughran"
...
This reverts commit c265515725
2016-12-09 12:49:27 +09:00
a5a55a54ab
YARN-5922. Remove direct references of HBaseTimelineWriter/Reader in core ATS classes. Contributed by Haibo Chen.
2016-12-08 12:31:12 -08:00
c265515725
HADOOP-13852 hadoop build to allow hadoop version property to be explicitly set. Contriibuted by Steve Loughran
2016-12-08 17:57:15 +00:00
74d0066d33
YARN-5970. Validate application update timeout request parameters. Contributed by Rohith Sharma K S.
2016-12-08 15:53:56 +05:30
72fe546841
YARN-5963. Spelling errors in logging and exceptions for node manager, client, web-proxy, common, and app history code (gsohn via rkanter)
2016-12-07 15:07:25 -08:00
9f5d2c4fff
YARN-5136. Error in handling event type APP_ATTEMPT_REMOVED to the scheduler
...
(Contributed by Wilfred Spiegelenburg via Daniel Templeton)
2016-12-07 11:12:14 -08:00
ab923a53fc
YARN-5965. Retrospect ApplicationReport#getApplicationTimeouts. Contributed by Rohith Sharma K S
2016-12-08 00:27:25 +05:30
563480dccd
YARN-5932. Retrospect moveApplicationToQueue in align with YARN-5611. Contributed by Sunil G.
2016-12-07 10:39:14 +05:30
a7288da595
YARN-5184. Fix up incompatible changes introduced on ContainerStatus and NodeReport. Contributed by Sangjin Lee.
2016-12-06 14:36:41 -08:00
c8d0a049b0
HADOOP-13859. TestConfigurationFieldsBase fails for fields that are DEFAULT values of skipped properties. (Haibo Chen via kasha)
2016-12-06 14:10:58 -08:00
f3b8ff54ab
YARN-5921. Incorrect synchronization in RMContextImpl#setHAServiceState/getHAServiceState. Contributed by Varun Saxena
2016-12-06 06:53:38 +05:30
43ebff2e35
YARN-5559. Analyse 2.8.0/3.0.0 jdiff reports and fix any issues. Contributed by Akira Ajisaka & Wangda Tan
2016-12-05 11:39:34 -08:00
f885160f4a
YARN-5746. The state of the parentQueue and its childQueues should be synchronized. Contributed by Xuan Gong
2016-12-02 16:17:31 -08:00
5bd18c49bd
YARN-5929. Missing scheduling policy in the FS queue metric. (Contributed by Yufei Gu via Daniel Templeton)
2016-12-02 13:55:42 -08:00
f304ccae3c
YARN-5915. ATS 1.5 FileSystemTimelineWriter causes flush() to be called after every event write. Contributed by Atul Sikaria
2016-12-02 16:54:15 +00:00
c87b3a448a
MAPREDUCE-6787. Allow job_conf.xml to be downloadable on the job overview page in JHS (haibochen via rkanter)
2016-12-01 17:29:38 -08:00
2d77dc727d
YARN-5901. Fix race condition in TestGetGroups beforeclass setup() (Contributed by Haibo Chen via Daniel Templeton)
2016-12-01 15:57:39 -08:00
69fb70c31a
YARN-5761. Separate QueueManager from Scheduler. (Xuan Gong via gtcarrera9)
2016-11-30 13:38:42 -08:00
4fca94fbda
YARN-5942. "Overridden" is misspelled as "overriden" in FairScheduler.md
...
(Contributed by Heather Sutherland via Daniel Templeton)
2016-11-30 11:23:51 -08:00
b3befc021b
YARN-4997. Update fair scheduler to use pluggable auth provider (Contributed by Tao Jie via Daniel Templeton)
2016-11-30 09:50:33 -08:00
cfd8076f81
Revert due to an error "HDFS-10994. Support an XOR policy XOR-2-1-64k in HDFS. Contributed by Sammi Chen"
...
This reverts commit 5614f847b2ef2a5b70bd9a06edc4eba06174c6.
2016-11-30 15:44:52 +08:00
dd9a96cb17
YARN-5851. TestContainerManagerSecurity testContainerManager[1] failed (Contributed by Haibo Chen via Daniel Templeton)
2016-11-29 14:22:19 -08:00
3b9d3acd20
YARN-5890. FairScheduler should log information about AM-resource-usage and max-AM-share for queues
...
(Contributed by Yufei Gu via Daniel Templeton)
2016-11-29 12:46:05 -08:00
25f9872be6
YARN-5774. MR Job stuck in ACCEPTED status without any progress in Fair Scheduler
...
if set yarn.scheduler.minimum-allocation-mb to 0. (Contributed by Yufei Gu via Daniel Templeton)
2016-11-29 09:40:49 -08:00
62b42ef5dd
YARN-5725. Test uncaught exception in TestContainersMonitorResourceChange.testContainersResourceChange
...
when setting IP and host (Contributed by Miklos Szegedi via Daniel Templeton)
2016-11-28 15:19:52 -08:00
5d5614f847
HDFS-10994. Support an XOR policy XOR-2-1-64k in HDFS. Contributed by Sammi Chen
2016-11-28 14:34:44 +08:00
209e805430
HADOOP-13506. Redundant groupid warning in child projects. Contributed by Kai Sasaki.
2016-11-28 14:34:57 +09:00
c06114d6a3
YARN-5572. HBaseTimelineWriterImpl appears to reference a bad property name. Contributed by Varun Saxena.
2016-11-27 23:35:53 +05:30
e2f77ca77b
Revert "YARN-5921. Incorrect synchronization in RMContextImpl#setHAServiceState/getHAServiceState. Contributed by Varun Saxena."
...
This reverts commit 9f2960be19
2016-11-27 23:32:59 +05:30
9f2960be19
YARN-5921. Incorrect synchronization in RMContextImpl#setHAServiceState/getHAServiceState. Contributed by Varun Saxena.
2016-11-27 23:31:27 +05:30
eb0a483ed0
YARN-4206. Add Application timeouts in Application report and CLI. Contributed by Rohith Sharma K S.
2016-11-24 18:18:42 +05:30
e15c20edba
YARN-5920. Fix deadlock in TestRMHA.testTransitionedToStandbyShouldNotHang. Contributed by Varun Saxena.
2016-11-24 12:18:38 +05:30
10468529a9
YARN-4752. Improved preemption in FairScheduler. (kasha)
...
Contains:
YARN-5605. Preempt containers (all on one node) to meet the requirement of starved applications
YARN-5821. Drop left-over preemption-related code and clean up method visibilities in the Schedulable hierarchy
YARN-5783. Verify identification of starved applications.
YARN-5819. Verify fairshare and minshare preemption
YARN-5885. Cleanup YARN-4752 branch for merge
Change-Id: Iee0962377d019dd64dc69a020725d2eaf360858c
2016-11-23 19:48:59 -10:00
1f12867a69
YARN-5649. Add REST endpoints for updating application timeouts. Contributed by Rohith Sharma K S
2016-11-23 16:25:39 -08:00
005850b28f
YARN-5918. Handle Opportunistic scheduling allocate request failure when NM is lost. (Bibin A Chundatt via asuresh)
2016-11-23 09:53:31 -08:00
3541ed8068
YARN-4330. MiniYARNCluster is showing multiple Failed to instantiate default resource calculator warning messages. Contributed by Varun Saxena
2016-11-23 14:12:23 +05:30
4667564162
YARN-5911. DrainDispatcher does not drain all events on stop even if setDrainEventsOnStop is true. Contributed by Varun Saxena.
2016-11-23 08:49:48 +05:30
83cc7263af
YARN-5722. FairScheduler hides group resolution exceptions when assigning queue (Contributed by Wilfred Spiegelenburg via Daniel Templeton)
2016-11-22 11:16:00 -08:00
a926f895c1
YARN-5865. Retrospect updateApplicationPriority api to handle state store exception in align with YARN-5611. Contributed by Sunil G.
2016-11-22 14:49:15 +05:30
6f8074298d
YARN-5713. Update jackson from 1.9.13 to 2.x in hadoop-yarn.
2016-11-22 10:20:31 +09:00
009452bb6d
YARN-5859. TestResourceLocalizationService#testParallelDownloadAttemptsForPublicResource sometimes fails. Contributed by Eric Badger
2016-11-21 16:36:08 +00:00
09520cb439
YARN-5271. ATS client doesn't work with Jersey 2 on the classpath. Contributed by Weiwei Yang.
2016-11-17 22:17:23 -06:00
140b9939da
YARN-5904. Reduce the number of default server threads for AMRMProxyService.
2016-11-17 18:26:25 -08:00
f05a9ceb4a
YARN-3538. TimelineWebService doesn't catch runtime exception. Contributed by Steve Loughran and Tsuyoshi Ozawa.
2016-11-17 16:55:39 -08:00
ce2847e794
YARN-5870. Expose getApplications API in YarnClient with GetApplicationsRequest parameter. Contributed by Jian He.
2016-11-17 08:48:35 +05:30
59bfcbf357
YARN-5836. Malicious AM can kill containers of other apps running in any node its containers are running. Contributed by Botong Huang
2016-11-16 22:21:03 +00:00
6a11877ab5
YARN-5891. yarn rmadmin -help contains a misspelled ResourceManager (Grant Sohn via Varun Saxena)
2016-11-17 03:26:13 +05:30
d65603517e
YARN-5375. invoke MockRM#drainEvents implicitly in MockRM methods to reduce test failures. Contributed by sandflee.
2016-11-16 15:14:00 +05:30
f121d0b036
YARN-5875. TestTokenClientRMService#testTokenRenewalWrongUser fails. Contributed by Gergely Novák.
2016-11-15 13:58:11 -08:00
264ddb13ff
YARN-5736 Addendum. Fixes segfault due to unterminated string. (Contributed by Miklos Szegedi via Daniel Templeton)
2016-11-15 13:01:57 -08:00
5af572b644
HADOOP-13427. Eliminate needless uses of FileSystem#{exists(), isFile(), isDirectory()}. Contributed by Steve Loughran and Mingliang Liu
2016-11-15 10:57:00 -08:00
3219b7b4ac
YARN-4597. Introduce ContainerScheduler and a SCHEDULED state to NodeManager container lifecycle. (asuresh)
2016-11-15 07:56:25 -08:00
7ffb9943b8
YARN-4355. NPE while processing localizer heartbeat. Contributed by Varun Saxena & Jonathan Hung.
2016-11-15 15:41:56 +05:30
Xuan
Andrew Wang
Daniel Templeton
Sunil G
Junping Du
Jason Lowe
Yufei Gu
Naganarasimha
Arun Suresh
Akira Ajisaka
bibinchundatt
Eric Payne
Carlo Curino
Varun Saxena
Brahma Reddy Battula
Jian He
vrushali
Nathan Roberts
Jonathan Eagles
Varun Vasudev
Haibo Chen
Wangda Tan
Robert Kanter
Yufei Gu
Chris Douglas
Rohith Sharma K S
Haibo Chen
Min Shen
Karthik Kambatla
Subru Krishnan
Sidharta S
Karthik Kambatla
Konstantinos Karanasos
Allen Wittenauer
Ming Ma
Tsuyoshi Ozawa
Ravi Prakash
Ray Chiang
Gera Shegalov
Yiqun Lin
Eric Yang
Li Lu
Sangjin Lee
Masatake Iwasaki
Jason Lowe
Junping Du
Steve Loughran
Kai Zheng
Wei-Chiu Chuang
Xiao Chen
Mingliang Liu