04f6ebb66a
YARN-5294. Pass remote ip address down to YarnAuthorizationProvider. (Jian He via wangda)
2016-07-06 10:36:48 -07:00
d792a90206
YARN-5296. NMs going OutOfMemory because ContainerMetrics leak in ContainerMonitorImpl. Contributed by Junping Du
2016-07-05 18:05:16 -07:00
ce9c006430
YARN-5214. Fixed locking in DirectoryCollection to avoid hanging NMs when various code-paths hit slow disks. Contributed by Junping Du.
2016-07-05 16:07:28 -07:00
8e672e3c71
YARN-5286. Add RPC port info in RM web service's response when getting app status. (Jun Gong via Varun Saxena).
2016-07-05 22:56:07 +05:30
0a5def155e
YARN-4568. Fix message when NodeManager runs into errors initializing the recovery directory. (rchiang)
2016-07-01 15:53:46 -07:00
c35a5a7a8d
YARN-5023. TestAMRestart#testShouldNotCountFailureToMaxAttemptRetry fails. Contributed by sandflee
2016-07-01 14:29:03 -07:00
8ade81228e
YARN-5282. Fix typos in CapacityScheduler documentation. (Ray Chiang via Varun Saxena).
2016-07-01 11:38:32 +05:30
abe7fc22c1
YARN-5182. MockNodes.newNodes creates one more node per rack than requested. (Karthik Kambatla via Varun Saxena).
2016-06-30 00:13:28 +05:30
26b5e6116f
YARN-5262. Optimize sending RMNodeFinishedContainersPulledByAMEvent for every AM heartbeat.
2016-06-29 10:08:30 +05:30
23c3ff85a9
HADOOP-13209. replace slaves with workers (John Smith via aw)
2016-06-28 05:53:03 -07:00
a8a48c9125
YARN-5278. Remove unused argument in TestRMWebServicesForCSWithPartitions#setupQueueConfiguration. Contributed by Tao Jie.
2016-06-23 14:28:12 +09:00
6ab5aa1c1f
YARN-5266. Wrong exit code while trying to get app logs using regex via CLI. Contributed by Xuan Gong
2016-06-22 21:48:49 -07:00
99e5dd68d0
YARN-5171. Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node. (Inigo Goiri via asuresh)
2016-06-22 19:04:54 -07:00
5d58858bb6
HADOOP-9613. [JDK8] Update jersey version to latest 1.x release.
2016-06-21 08:05:32 +09:00
5107a967fa
YARN-4958. The file localization process should allow for wildcards to reduce the application footprint in the state store (Daniel Templeton via sjlee)
2016-06-20 09:56:53 -07:00
d0162f2040
YARN-5246. NMWebAppFilter web redirects drop query parameters. Contributed by Varun Vasudev.
2016-06-19 17:44:54 -07:00
20f2799938
YARN-5077. Fix FSLeafQueue#getFairShare() for queues with zero fairshare. (Yufei Gu via kasha)
2016-06-17 22:24:42 -07:00
fbbe0bb627
YARN-5082. Limit ContainerId increase in fair scheduler if the num of node app reserved reached the limit. Addendum to fix javac warning. (Arun Suresh via kasha)
2016-06-17 22:12:50 -07:00
c35fa4a0e5
YARN-5251. Yarn CLI to obtain App logs for last 'n' bytes fails. Contributed by Xuan Gong.
2016-06-17 08:24:24 -07:00
e14ee0d3b5
YARN-5083. YARN CLI for AM logs does not give any error message if entered invalid am value. Contributed by Jian He.
2016-06-16 08:55:56 -07:00
5dfc38ff57
YARN-5237. Fix missing log files issue in rolling log aggregation. Contributed by Xuan Gong.
2016-06-15 16:17:54 -07:00
25064fb2fb
YARN-5223. Container line in yarn logs output for a live application should include the hostname for the container. Contributed by Xuan Gong.
2016-06-15 12:42:44 +05:30
d6284428e8
YARN-5122. "yarn logs" for running containers should print an explicit footer saying that the log may be incomplete. Contributed by Jian He.
2016-06-15 12:35:17 +05:30
c77a1095dc
YARN-1942. Deprecate toString/fromString methods from ConverterUtils and move them to records classes like ContainerId/ApplicationId, etc. (wangda)
2016-06-14 15:06:38 -07:00
eec835ec17
YARN-4887. Add allocation request ID to AM-RM protocol for identifying resource-requests explicitly. (Subru Krishnan via wangda)
2016-06-13 21:57:33 -07:00
28b66ae919
YARN-4989. TestWorkPreservingRMRestart#testCapacitySchedulerRecovery fails intermittently. Contributed by Ajith S.
2016-06-13 11:09:32 +05:30
7dae2b3bc4
YARN-5212. Run existing ContainerManager tests using QueuingContainerManagerImpl. (Konstantinos Karanasos via asuresh)
2016-06-12 10:05:37 -07:00
5143277958
YARN-5124. Modify AMRMClient to set the ExecutionType in the ResourceRequest. (asuresh)
2016-06-12 09:42:38 -07:00
5279af7cd4
YARN-5082. Limit ContainerId increase in fair scheduler if the num of node app reserved reached the limit (sandflee via asuresh)
2016-06-10 22:33:42 -07:00
e0f4620cc7
YARN-5197. RM leaks containers if running container disappears from node update. Contributed by Jason Lowe.
2016-06-11 10:22:27 +05:30
03fc6b1bb0
YARN-3426. Add jdiff support to YARN. (vinodkv via wangda)
2016-06-10 09:51:09 -07:00
244506f9c8
YARN-5208. Run TestAMRMClient TestNMClient TestYarnClient TestClientRMTokens TestAMAuthorization tests with hadoop.security.token.service.use_ip enabled. (Rohith Sharma K S via wangda)
2016-06-10 09:34:32 -07:00
9378d9428f
YARN-5191. Renamed the newly added “download=true” option for getting logs via NMWebServices and AHSWebServices to be a better "format" option. (Xuan Gong via vinodkv)
2016-06-09 12:30:58 -07:00
1500a0a300
YARN-4308. ContainersAggregated CPU resource utilization reports negative usage in first few heartbeats. Contributed by Sunil G
2016-06-09 05:41:09 +05:30
5a43583c0b
YARN-5080. Addendum fix to the original patch to fix YARN logs CLI. Contributed by Xuan Gong
2016-06-08 09:49:55 -07:00
3344ba70e0
YARN-5204. Properly report status of killed/stopped queued containers. (Konstantinos Karanasos via asuresh)
2016-06-08 08:31:32 -07:00
8c8a377cac
YARN-5206. RegistrySecurity includes id:pass in exception text if considered invalid. Contributed by Steve Loughran
2016-06-08 14:11:25 +00:00
76f0800c21
YARN-5176. More test cases for queuing of containers at the NM. (Konstantinos Karanasos via asuresh)
2016-06-07 17:16:18 -07:00
58be55b6e0
YARN-5199. Close LogReader in in AHSWebServices#getStreamingOutput and
...
FileInputStream in NMWebServices#getLogs. Contributed by Xuan Gong
2016-06-07 16:07:02 -07:00
8554aee1be
Addendum patch for YARN-5180 updating findbugs-exclude.xml
2016-06-07 15:59:43 -07:00
620325e816
YARN-4837. User facing aspects of 'AM blacklisting' feature need fixing. (vinodkv via wangda)
2016-06-07 15:06:42 -07:00
bddea5fe5f
YARN-5118. Tests fails with localizer port bind exception. Contributed by Brahma Reddy Battula.
2016-06-07 11:20:15 +05:30
3a154f75ed
YARN-4525. Fix bug in RLESparseResourceAllocation.getRangeOverlapping(). (Ishai Menache and Carlo Curino via asuresh)
2016-06-06 21:18:32 -07:00
7a9b7372a1
YARN-5185. StageAllocaterGreedyRLE: Fix NPE in corner case. (Carlo Curino via asuresh)
2016-06-06 21:06:52 -07:00
4a1cedc010
MAPREDUCE-5044. Have AM trigger jstack on task attempts that timeout before killing them. (Eric Payne and Gera Shegalov via mingma)
2016-06-06 14:30:51 -07:00
db54670e83
YARN-5165. Fix NoOvercommitPolicy to take advantage of RLE representation of plan. (Carlo Curino via asuresh)
2016-06-03 14:49:32 -07:00
f10ebc67f5
YARN-5098. Fixed ResourceManager's DelegationTokenRenewer to replace expiring system-tokens if RM stops and only restarts after a long time. Contributed by Jian He.
2016-06-03 13:00:07 -07:00
99cc439e29
YARN-5190. Registering/unregistering container metrics in ContainerMonitorImpl and ContainerImpl causing uncaught exception in ContainerMonitorImpl. Contributed by Junping Du
2016-06-03 11:10:42 -07:00
097baaaeba
YARN-1815. Work preserving recovery of Unmanged AMs. Contributed by Subru Krishnan
2016-06-03 10:49:30 -07:00
dc26601d8f
YARN-5180. Allow ResourceRequest to specify an enforceExecutionType flag. (asuresh)
2016-06-02 09:01:02 -07:00
0bc05e40fa
YARN-5088. Improve "yarn log" command-line to read the last K bytes for the log files. Contributed by Xuan Gong
2016-06-01 13:44:21 -07:00
4e1f56e111
YARN-5141. Get Container logs for the Running application from Yarn Logs CommandLine. Contributed by Xuan Gong.
2016-05-29 21:53:43 +05:30
86fb58b7dc
YARN-4007. Add support for different network setups when launching the docker container. Contributed by Sidharta Seethana.
2016-05-29 21:32:51 +05:30
42f90ab885
YARN-4844. Add getMemorySize/getVirtualCoresSize to o.a.h.y.api.records.Resource. Contributed by Wangda Tan.
2016-05-29 21:24:16 +05:30
f5ff05cc8a
YARN-5132. Exclude generated protobuf sources from YARN Javadoc build. (Subru Krishnan via kasha)
2016-05-28 08:09:46 -07:00
4fc09a897b
YARN-5117. QueuingContainerManager does not start GUARANTEED Container even if Resources are available. (Konstantinos Karanasos via asuresh)
2016-05-27 17:49:15 -07:00
aa975bc781
YARN-5127. Expose ExecutionType in Container api record. (Hitesh Sharma via asuresh)
2016-05-27 14:06:32 -07:00
e4022debf7
YARN-5166. javadoc:javadoc goal fails on hadoop-yarn-client. Contributed by Andras Bokor.
2016-05-27 23:15:32 +09:00
705286ccae
YARN-4987. Read cache concurrency issue between read and evict in EntityGroupFS timeline store. Contributed by Li Lu.
2016-05-27 06:58:32 -07:00
bde819abbb
YARN-5089. Improve "yarn log" command-line "logFiles" option to support
...
regex. Contributed by Xuan Gong
2016-05-26 23:49:20 -07:00
916140604f
HADOOP-12911. Upgrade Hadoop MiniKDC with Kerby. Contributed by Jiajia Li
2016-05-28 14:23:39 +08:00
0a544f8a3e
YARN-5005. TestRMWebServices#testDumpingSchedulerLogs fails randomly. Contributed by Bibin A Chundatt.
2016-05-27 10:44:35 +05:30
1b0cfc84e7
YARN-5116. Failed to execute "yarn application". Contributed by Jun Gong
2016-05-26 23:43:08 +00:00
5b41b288d0
YARN-5162. Fix Exceptions thrown during in registerAM call when Distributed Scheduling is Enabled (Hitesh Sharma via asuresh)
2016-05-26 14:56:37 -07:00
04ded558b0
YARN-5035. FairScheduler: Adjust maxAssign dynamically when assignMultiple is turned on. (kasha)
2016-05-26 14:41:07 -07:00
4f513a4a8e
YARN-4866. FairScheduler: AMs can consume all vcores leading to a livelock when using FAIR policy. (Yufei Gu via kasha)
2016-05-25 22:13:27 -07:00
013532a95e
YARN-4957. Add getNewReservation in ApplicationClientProtocol (Sean Po via curino)
2016-05-25 16:55:49 -07:00
1ba31fe9e9
YARN-4459. container-executor should only kill process groups. Contributed by Jun Gong
2016-05-25 21:24:55 +00:00
77d5ce9069
YARN-3344. Fix warning - procfs stat file is not in the expected format. (Ravindra Kumar Naik and Akira Ajisaka via kasha)
2016-05-25 10:50:54 -07:00
e07519b8db
YARN-4766. NM should not aggregate logs older than the retention policy (haibochen via rkanter)
2016-05-25 10:25:44 -07:00
9a31e5dfef
YARN-5115. Avoid setting CONTENT-DISPOSITION header in the container-logs web-service. Contributed by Xuan Gong.
2016-05-25 19:01:12 +05:30
c380a22031
YARN-857. Localization failures should be available in container diagnostics. Contributed by Vinod Kumar Vavilapalli.
2016-05-25 19:01:12 +05:30
48c931331c
YARN-5131. Distributed shell AM fails when extra container arrives during finishing. Contributed by Wangda Tan.
2016-05-25 06:18:11 -07:00
28bd63e92b
YARN-5024. TestContainerResourceUsage#testUsageAfterAMRestartWithMultipleContainers random failure. Contributed by Bibin A Chundatt
2016-05-25 10:15:50 +05:30
edd716e99c
YARN-5114. Add additional tests in TestRMWebServicesApps and rectify testInvalidAppAttempts failure in 2.8. Contributed by Bibin A Chundatt
2016-05-25 06:11:38 +08:00
f979d779e1
YARN-4878. Expose scheduling policy and max running apps over JMX for Yarn queues. (Yufei Gu via kasha)
2016-05-24 10:54:11 -07:00
b4078bd17b
YARN-3971. Skip RMNodeLabelsManager#checkRemoveFromClusterNodeLabelsOfQueue on nodelabel recovery. (addendum patch). Contributed by Bibin A chundatt
2016-05-24 08:06:53 +08:00
6d043aa4cf
YARN-4979. FSAppAttempt demand calculation considers demands at multiple locality levels different. (Zhihai Xu via kasha)
2016-05-23 14:29:28 -07:00
ac954486c5
YARN-5055. max apps per user can be larger than max per queue. Contributed by Eric Badger
2016-05-23 15:54:42 +00:00
d1df0266cf
YARN-5103. With NM recovery enabled, restarting NM multiple times results in AM restart. Contributed by Junping Du
2016-05-23 15:15:27 +00:00
6161d9ba52
YARN-5112. Excessive log warnings for directory permission issue on NM recovery. Contributed by Jian He.
...
(cherry picked from commit 867cd2f5a2
2016-05-22 16:04:02 -07:00
d364ceac85
YARN-5020. Fix Documentation for Yarn Capacity Scheduler on Resource Calculator. Contributed by Takashi Ohnishi
2016-05-20 11:35:26 -07:00
0287c49107
YARN-5016. Add support for a minimum retry interval for container retries. Contributed by Jun Gong.
2016-05-20 12:28:14 +05:30
22fcd819f0
YARN-5076. YARN web interfaces lack XFS protection. Contributed by Jonathan Maron.
...
(cherry picked from commit 2703ec6871
2016-05-19 14:15:21 -07:00
feb90ffcca
YARN-4002. Make ResourceTrackerService#nodeHeartbeat more concurrent. Contributed by Rohith Sharma K S & Zhiguo Hong
2016-05-19 13:01:36 -07:00
141873ca7d
YARN-5100. The YarnApplicationState is always running in ATS even application is finished. Contributed by Xuan Gong.
2016-05-19 09:13:29 -07:00
22ff9e68d1
HADOOP-12767. Update apache httpclient version to 4.5.2; httpcore to 4.4.4. Artem Aliev via stevel.
2016-05-19 14:48:18 +01:00
d4274c64bc
YARN-5107. TestContainerMetrics fails. (aajisaka)
2016-05-19 19:53:59 +09:00
1597630681
YARN-5110. Fix OpportunisticContainerAllocator to insert complete HostAddress in issued ContainerTokenIds. (Konstantinos Karanasos via asuresh)
2016-05-18 18:46:00 -07:00
f04c81c9ce
YARN-4925. ContainerRequest in AMRMClient, application should be able to specify nodes/racks together with nodeLabelExpression. Contributed by Bibin A Chundatt
2016-05-19 01:30:48 +08:00
ef1757790d
YARN-4913. Yarn logs should take a -out option to write to a directory. Contributed by Xuan Gong.
2016-05-18 22:44:33 +05:30
8a9ecb7584
YARN-5090. Add End-to-End test-cases for DistributedScheduling using MiniYarnCluster. (asuresh)
2016-05-17 19:01:29 -07:00
0c6726e20d
Addendum patch for YARN-4832. Contributed by Junping Du
2016-05-17 14:50:30 -07:00
fa3bc3405d
YARN-4832. NM side resource value should get updated if change applied in RM side. Contributed by Junping Du
2016-05-17 12:52:19 -07:00
ccc93e7812
YARN-5075. Fix findbugs warnings in hadoop-yarn-common module. (asuresh)
2016-05-16 23:22:01 -07:00
730bc746f9
HADOOP-12930. Dynamic subcommands for hadoop shell scripts (aw)
...
This commit contains the following JIRA issues:
HADOOP-12931. bin/hadoop work for dynamic subcommands
HADOOP-12932. bin/yarn work for dynamic subcommands
HADOOP-12933. bin/hdfs work for dynamic subcommands
HADOOP-12934. bin/mapred work for dynamic subcommands
HADOOP-12935. API documentation for dynamic subcommands
HADOOP-12936. modify hadoop-tools to take advantage of dynamic subcommands
HADOOP-13086. enable daemonization of dynamic commands
HADOOP-13087. env var doc update for dynamic commands
HADOOP-13088. fix shellprofiles in hadoop-tools to allow replacement
HADOOP-13089. hadoop distcp adds client opts twice when dynamic
HADOOP-13094. hadoop-common unit tests for dynamic commands
HADOOP-13095. hadoop-hdfs unit tests for dynamic commands
HADOOP-13107. clean up how rumen is executed
HADOOP-13108. dynamic subcommands need a way to manipulate arguments
HADOOP-13110. add a streaming subcommand to mapred
HADOOP-13111. convert hadoop gridmix to be dynamic
HADOOP-13115. dynamic subcommand docs should talk about exit vs. continue program flow
HADOOP-13117. clarify daemonization and security vars for dynamic commands
HADOOP-13120. add a --debug message when dynamic commands have been used
HADOOP-13121. rename sub-project shellprofiles to match the rest of Hadoop
HADOOP-13129. fix typo in dynamic subcommand docs
HADOOP-13151. Underscores should be escaped in dynamic subcommands document
HADOOP-13153. fix typo in debug statement for dynamic subcommands
2016-05-16 17:54:45 -07:00
1217c8f6b4
YARN-5069. TestFifoScheduler.testResourceOverCommit race condition. Contributed by Eric Badger.
2016-05-16 20:28:04 +00:00
81effb7dcd
YARN-4325. Nodemanager log handlers fail to send finished/failed events in some cases. Contributed by Junping Du
2016-05-16 15:40:23 +00:00
f45bc5a83e
YARN-4738. Notify the RM about the status of OPPORTUNISTIC containers (Konstantinos Karanasos via asuresh)
2016-05-15 17:54:34 -07:00
7be53b65f9
YARN-5080. Cannot obtain logs using YARN CLI -am for either KILLED or
...
RUNNING AM. Contributed by Xuan Gong
2016-05-14 12:19:23 -07:00
f0ac18d001
YARN-2888. Corrective mechanisms for rebalancing NM container queues. (asuresh)
2016-05-13 13:38:36 -07:00
3c5c57af28
HADOOP-13142. Change project version from 3.0.0 to 3.0.0-alpha1.
2016-05-12 18:27:28 -07:00
ca5613af91
Revert "Update project version to 3.0.0-alpha1-SNAPSHOT."
...
This reverts commit 6b53802cba
2016-05-12 15:32:45 -07:00
013000fbc2
YARN-5053. More informative diagnostics when applications killed by a user. Contributed by Eric Badger
2016-05-12 20:28:36 +00:00
6b53802cba
Update project version to 3.0.0-alpha1-SNAPSHOT.
2016-05-12 11:05:05 -07:00
0bbe01f8d5
YARN-4577. Enable aux services to have their own custom classpath/jar file (Xuan Gong via sale)
2016-05-12 10:11:23 -07:00
b7ac85259c
YARN-5068. Expose scheduler queue to application master. (Harish Jaiprakash via rohithsharmaks)
2016-05-12 15:17:49 +05:30
d464f4d1c4
YARN-5049. Extend NMStateStore to save queued container information. (Konstantinos Karanasos via asuresh)
2016-05-11 19:10:17 -07:00
4b4e4c6ba8
YARN-4995. FairScheduler: Display per-queue demand on the scheduler page. (xupeng via kasha)
2016-05-11 17:36:21 -07:00
39f2bac38b
YARN-5029. RM needs to send update event with YarnApplicationState as Running to ATS/AHS. Contributed by Xuan Gong.
2016-05-11 09:28:35 -07:00
d971bf252f
YARN-5073. Refactor startContainerInternal() in ContainerManager to remove unused parameter. (Konstantinos Karanasos via asuresh)
2016-05-10 22:31:38 -07:00
87f5e35133
YARN-4842. Fixed "yarn logs" command to guess (and thus not require) the appOwner argument when viewing another user's logs. Contributed by Ram Venkatesh and Xuan Gong.
2016-05-09 22:41:18 -07:00
2750fb900f
YARN-4926. Change nodelabel rest API invalid reponse status to 400. Contributed by Bibin A Chundatt
2016-05-08 22:49:25 +05:30
47c41e7ac7
YARN-5048. DelegationTokenRenewer#skipTokenRenewal may throw NPE (Jian He via Yongjun Zhang)
2016-05-06 21:50:09 -07:00
b2ed6ae731
YARN-4747. AHS error 500 due to NPE when container start event is missing. Contributed by Varun Saxena
2016-05-06 22:59:39 +00:00
23248f63aa
getApplicationReport call may raise NPE for removed queues. (Jian He via wangda)
2016-05-06 15:30:45 -07:00
bb62e05925
YARN-4390. Do surgical preemption based on reserved container in CapacityScheduler. Contributed by Wangda Tan
2016-05-05 12:56:21 -07:00
d0da13229c
YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla
2016-05-05 14:07:54 +00:00
72b047715c
YARN-4595. Add support for configurable read-only mounts when launching Docker containers. Contributed by Billie Rinaldi.
2016-05-05 13:01:54 +05:30
9e37fe3b7a
YARN-4905. Improved "yarn logs" command-line to optionally show log metadata also. Contributed by Xuan Gong.
2016-05-04 14:16:03 -07:00
7bd418e48c
YARN-4984. LogAggregationService shouldn't swallow exception in handling createAppDir() which cause thread leak. (Junping Du via wangda)
2016-05-04 11:38:55 -07:00
e61d431275
YARN-4920. ATS/NM should support a link to dowload/get the logs in text format. Contributed by Xuan Gong.
2016-05-04 10:35:49 -07:00
75e0450593
YARN-4947. Test timeout is happening for TestRMWebServicesNodes. Contributed by Bibin A Chundatt
2016-05-04 09:58:26 +05:30
ed54f5f1ff
YARN-5003. Add container resource to RM audit log. Contributed by Nathan Roberts
2016-05-03 20:03:41 +00:00
c6b4839168
YARN-4834. ProcfsBasedProcessTree doesn't track daemonized processes. Contributed by Nathan Roberts
2016-05-03 17:27:28 +00:00
06413da72e
YARN-4851. Metric improvements for ATS v1.5 storage components. Li Lu via junping_du.
2016-05-03 04:16:21 -07:00
9d4d30243b
Remove parent's env vars from child processes
2016-04-29 09:25:51 -07:00
0f25a1bb52
YARN-3998. Add support in the NodeManager to re-launch containers. Contributed by Jun Gong.
2016-04-29 16:09:07 +05:30
4a8508501b
YARN-5009. NMLeveldbStateStoreService database can grow substantially leading to longer recovery times. Contributed by Jason Lowe
2016-04-28 21:54:11 -07:00
dd80042c42
YARN-5008. LeveldbRMStateStore database can grow substantially leading to long recovery times. Contributed by Jason Lowe
2016-04-28 21:27:25 -07:00
f16722d2ef
YARN-4956. findbug issue on LevelDBCacheTimelineStore. (Zhiyuan Yang via gtcarrera9)
2016-04-27 10:57:10 -07:00
185c3d4de1
YARN-4807. MockAM#waitForState sleep duration is too long. (Yufei Gu via kasha)
2016-04-27 09:43:23 -07:00
66b07d8374
YARN-4966. Improve yarn logs to fetch container logs without specifying nodeId. Contributed by Xuan Gong.
2016-04-27 15:56:36 +05:30
4beff01354
YARN-4983. JVM and UGI metrics disappear after RM transitioned to standby mode
2016-04-26 21:00:17 -07:00
341888a0aa
YARN-4412. Create ClusterMonitor to compute ordered list of preferred NMs for OPPORTUNITIC containers. (asuresh)
2016-04-26 20:12:12 -07:00
1a3f1482e2
YARN-4795. ContainerMetrics drops records. (Daniel Templeton via kasha)
2016-04-26 06:15:36 -07:00
4b1dcbbe0c
YARN-1297. FairScheduler: Move some logs to debug and check if debug logging is enabled
2016-04-26 05:10:09 -07:00
a5fed8bbfe
YARN-4991. Fix ContainerRequest Constructor to set nodelabelExpression correctly. (Bibin A Chundatt via asuresh)
2016-04-25 23:26:21 -07:00
c282a08f38
YARN-2885. Create AMRMProxy request interceptor and ContainerAllocator to distribute OPPORTUNISTIC containers to appropriate Nodes (asuresh)
...
(cherry picked from commit 2bf025278a318b0452fdc9ece4427b4c42124e39)
2016-04-24 22:38:33 -07:00
b2a654c5ee
YARN-4335. Allow ResourceRequests to specify ExecutionType of a request ask (kkaranasos via asuresh)
...
(cherry picked from commit 8ffabfdf4f
2016-04-22 23:20:54 -07:00
63e5412f1a
HDFS-9427. HDFS should not default to ephemeral ports. Contributed by Xiaobing Zhou.
2016-04-22 15:14:40 -07:00
7cb3a3da96
YARN-4846. Fix random failures for TestCapacitySchedulerPreemption#testPreemptionPolicyShouldRespectAlreadyMarkedKillableContainers. (Bibin A Chundatt via wangda)
2016-04-22 11:40:32 -07:00
3dce486d88
YARN-4556. TestFifoScheduler.testResourceOverCommit fails. Contributed by Akihiro Suda
2016-04-21 21:16:47 +00:00
7c6339f66a
YARN-4968. A couple of AM retry unit tests need to wait SchedulerApplicationAttempt stopped. (Wangda Tan via gtcarrera9)
2016-04-21 13:25:33 -07:00
95a5046607
YARN-4976. Missing NullPointer check in ContainerLaunchContextPBImpl. Contributed by Giovanni Matteo Fumarola
2016-04-21 10:45:38 -07:00
170c4fd4cd
YARN-4784. Fairscheduler: defaultQueueSchedulingPolicy should not accept FIFO. (Yufei Gu via kasha)
2016-04-20 23:58:12 -07:00
6e297836d6
YARN-4935. TestYarnClient#testSubmitIncorrectQueue fails when FairScheduler is the default. (Yufei Gu via kasha)
2016-04-20 23:40:46 -07:00
33fd95a99c
YARN-4890. Unit test intermittent failure: TestNodeLabelContainerAllocation#testQueueUsedCapacitiesUpdate. (Sunil G via wangda)
2016-04-20 17:37:38 -07:00
c8172f5f14
YARN-2883. Queuing of container requests in the NM. (Konstantinos Karanasos and Arun Suresh via kasha)
2016-04-20 09:55:50 -07:00
477003730e
Fixed TimelineClient to retry SocketTimeoutException too. Contributed by Xuan Gong.
2016-04-18 11:47:06 -07:00
fdc46bfb37
YARN-4934. Reserved Resource for QueueMetrics needs to be handled correctly in few cases. (Sunil G via wangda)
2016-04-16 22:47:41 -07:00
e6c0742012
YARN-4965. Distributed shell AM failed due to ClientHandlerException thrown by jersey. Contributed by Junping Du
2016-04-16 19:39:18 -07:00
cab9cbaa0a
YARN-4468. Document the general ReservationSystem functionality, and the REST API. (subru and carlo via asuresh)
2016-04-15 16:58:49 -07:00
69f3d428d5
YARN-4940. yarn node -list -all failed if RM start with decommissioned node. Contributed by sandflee
2016-04-15 20:36:45 +00:00
fdbafbc9e5
YARN-4909. Fix intermittent failures of TestRMWebServices And TestRMWithCSRFFilter. Contributed by Bibin A Chundatt
2016-04-15 23:37:05 +05:30
3150ae8108
YARN-4924. NM recovery race can lead to container not cleaned up. Contributed by sandflee
2016-04-14 19:17:14 +00:00
0d9194df00
YARN-4541. Change log message in LocalizedResource#handle() to DEBUG (rchiang via rkanter)
2016-04-13 17:45:18 -07:00
e0cb426758
YARN-4886. Add HDFS caller context for EntityGroupFSTimelineStore. Contributed by Li Lu
2016-04-13 10:38:17 -07:00
437e9d6475
YARN-4810. NM applicationpage cause internal error 500. Contributed by Bibin A Chundatt.
2016-04-12 17:59:46 +05:30
4ee7f0a3e7
YARN-4897. dataTables_wrapper change min height. Contributed by Bibin A Chundatt.
2016-04-12 10:25:56 +05:30
ff722bbbdf
YARN-4794. Deadlock in NMClientImpl. Contributed by Jian He.
2016-04-12 09:59:33 +05:30
44bbc50d91
YARN-4168. Fixed a failing test TestLogAggregationService.testLocalFileDeletionOnDiskFull. Contributed by Takashi Ohnishi.
2016-04-11 12:11:14 -07:00
2a5da97f81
Revert "YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla"
...
This reverts commit 1cbcd4a491
2016-04-11 15:51:01 +00:00
08ddb3ac6d
YARN-4928. Some yarn.server.timeline.* tests fail on Windows attempting to use a test root path containing a colon. Contributed by Gergely Novák.
2016-04-11 08:48:32 -07:00
1ff27f9d12
YARN-4630. Remove useless boxing/unboxing code. Contributed by Kousuke Saruta.
2016-04-11 14:55:03 +09:00
07920acc16
YARN-4562. YARN WebApp ignores the configuration passed to it for keystore settings. Contributed by Sergey Shelukhin.
2016-04-11 09:39:57 +05:30
1b78b2ba17
YARN-4938. MiniYarnCluster should not request transitionToActive to RM on non-HA environment. Contributed by Eric Badger.
2016-04-11 01:31:49 +09:00
ff95fd547b
YARN-4927. TestRMHA#testTransitionedToActiveRefreshFail fails with FairScheduler. (Bibin A Chundatt via kasha)
2016-04-09 10:31:02 -07:00
fcb3fcd4c6
YARN-4826. Document configuration of ReservationSystem for CapacityScheduler. (Subru Krishnan via wangda)
2016-04-08 16:21:04 -07:00
ec06957941
YARN-3215. Respect labels in CapacityScheduler when computing headroom. (Naganarasimha G R via wangda)
2016-04-08 15:33:04 -07:00
9cb0c963d2
YARN-4740. AM may not receive the container complete msg when it restarts. Contributed by Jun Gong
2016-04-08 11:20:35 -07:00
e82f961a39
YARN-4756. Unnecessary wait in Node Status Updater during reboot. (Eric Badger via kasha)
2016-04-07 17:05:29 -07:00
93bacda08b
YARN-4769. Add support for CSRF header in the dump capacity scheduler logs and kill app buttons in RM web UI. Contributed by Varun Vasudev
2016-04-06 16:13:47 -07:00
b41e65e5bc
YARN-4906. Capture container start/finish time in container metrics. Contributed by Jian He.
2016-04-06 13:41:33 +05:30
21eb428448
YARN-4699. Scheduler UI and REST o/p is not in sync when -replaceLabelsOnNode is used to change label of a node. (Sunil G via wangda)
2016-04-05 16:24:11 -07:00
500e5a5952
YARN-4917. Fix typos in documentation of Capacity Scheduler. (Takashi Ohnishi via iwasakims)
2016-04-06 04:00:31 +09:00
30206346cf
YARN-4915. Fix typo in YARN Secure Containers documentation (Takashi Ohnishi via iwasakims)
2016-04-06 03:47:22 +09:00
0005816743
YARN-4916. TestNMProxy.tesNMProxyRPCRetry fails. Contributed by Tibor Kiss.
2016-04-05 09:01:08 -07:00
6be28bcc46
YARN-4893. Fix some intermittent test failures in TestRMAdminService. Contributed by Brahma Reddy Battula.
2016-04-05 06:57:54 -07:00
1cbcd4a491
YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla
2016-04-05 13:40:19 +00:00
776b549e2a
YARN-4609. RM Nodes list page takes too much time to load. Contributed by Bibin A Chundatt
2016-04-05 14:47:25 +05:30
552237d4a3
YARN-4880. Running TestZKRMStateStorePerf with real zookeeper cluster throws NPE. Contributed by Sunil G
2016-04-05 14:26:19 +05:30
f61de41736
YARN-4706. UI Hosting Configuration in TimelineServer doc is broken. (Akira AJISAKA via gtcarrera9)
2016-04-04 14:40:27 -07:00
5092c94195
YARN-4746. yarn web services should convert parse failures of appId, appAttemptId and containerId to 400. Contributed by Bibin A Chundatt
2016-04-04 16:25:03 +05:30
1e6f92977d
YARN-4607. Pagination support for AppAttempt page TotalOutstandingResource Requests table. Contributed by Bibin A Chundatt
2016-04-04 08:09:29 +05:30
81d04cae41
YARN-4657. Javadoc comment is broken for Resources.multiplyByAndAddTo(). (Daniel Templeton via kasha)
2016-04-01 16:20:00 -07:00
5686caa9fc
Missing file for YARN-4895.
2016-04-01 15:58:13 -07:00
82621e38a0
YARN-4895. Add subtractFrom method to ResourceUtilization class. Contributed by Konstantinos Karanasos.
2016-04-01 14:57:06 -07:00
3488c4f2c9
Revert "YARN-4857. Add missing default configuration regarding preemption of CapacityScheduler. Contributed by Kai Sasaki."
...
This reverts commit 0064cba169
2016-04-01 12:20:40 +05:30
12b11e2e68
YARN-4634. Scheduler UI/Metrics need to consider cases like non-queue label mappings. (Sunil G via wangda)
2016-03-31 14:35:18 -07:00
0dd9bcab97
YARN-4811. Generate histograms in ContainerMetrics for actual container resource usage
2016-03-31 14:28:13 -07:00
7a021471c3
YARN-4639. Remove dead code in TestDelegationTokenRenewer added in YARN-3055 (templedf via rkanter)
2016-03-31 13:09:09 -07:00
6d67420dbc
YARN-4183. Clarify the behavior of timeline service config properties (Naganarasimha G R via sjlee)
2016-03-31 10:49:03 -07:00
0a74610d1c
HADOOP-11393. Revert HADOOP_PREFIX, go back to HADOOP_HOME (aw)
2016-03-31 07:51:05 -07:00
0064cba169
YARN-4857. Add missing default configuration regarding preemption of CapacityScheduler. Contributed by Kai Sasaki.
2016-03-31 14:05:49 +05:30
f1b8f6b2c1
YARN-4884. Fix missing documentation about rmadmin command regarding node labels. Contributed by Kai Sasaki.
2016-03-31 14:01:48 +05:30
60e4116bf1
YARN-4822. Refactor existing Preemption Policy of CS for easier adding new approach to select preemption candidates. Contributed by Wangda Tan
2016-03-30 12:43:52 -07:00
8286270466
YARN-4436. DistShell ApplicationMaster.ExecBatScripStringtPath is misspelled (mattlamantia via rkanter)
2016-03-29 17:21:29 -07:00
fc055a3cbe
YARN-4865. Track Reserved resources in ResourceUsage and QueueCapacities. (Sunil G via wangda)
2016-03-29 17:07:55 -07:00
80182809ae
YARN-4863. AHS Security login should be in serviceInit() instead of serviceStart(). Contributed by Junping Du
2016-03-28 22:17:45 -07:00
948b758070
YARN-4773. Log aggregation performs extraneous filesystem operations when rolling log aggregation is disabled. Contributed by Jun Gong
2016-03-28 23:00:56 +00:00
524bc3c33a
YARN-998. Keep NM resource updated through dynamic resource config for RM/NM restart. Contributed by Junping Du
2016-03-28 11:12:33 -07:00
8831b18c65
Missing file for YARN-4117.
2016-03-28 09:59:47 -07:00
55ae143923
YARN-4117. End to end unit test with mini YARN cluster for AMRMProxy Service. Contributed by Giovanni Matteo Fumarola
2016-03-27 20:22:12 -07:00
49ff54c860
YARN-4805. Don't go through all schedulers in ParameterizedTestBase. (kasha)
2016-03-26 21:45:13 -07:00
00bebb7e58
YARN-4823. Refactor the nested reservation id field in listReservation to simple string field. (subru via asuresh)
2016-03-25 15:54:38 -07:00
e8fc81f9c8
YARN-4814. ATS 1.5 timelineclient impl call flush after every event write. Contributed by Xuan Gong
2016-03-25 20:15:49 +00:00
2e1d0ff4e9
YARN-4687. Document Reservation ACLs (Contributed by Sean Po)
2016-03-24 10:40:51 -07:00
d820975c1a
YARN-4683. Document the List Reservations REST API. (Contributed by Sean Po)
2016-03-24 10:30:29 -07:00
d82e797b65
YARN-4825. Remove redundant code in ClientRMService::listReservations. (subru via asuresh)
2016-03-24 09:59:55 -07:00
b1394d6307
YARN-4850. test-fair-scheduler.xml isn't valid xml (Yufei Gu via aw)
2016-03-24 08:15:58 -07:00
19b645c938
YARN-4820. ResourceManager web redirects in HA mode drops query parameters. Contributed by Varun Vasudev.
2016-03-23 19:34:30 -07:00
738155063e
HADOOP-12857. rework hadoop-tools (aw)
2016-03-23 13:46:38 -07:00
af1d125f9c
YARN-4814. ATS 1.5 timelineclient impl call flush after every event write. Contributed by Xuan Gong.
2016-03-23 08:57:16 -07:00
7fae4c68e6
YARN-4732. *ProcessTree classes have too many whitespace issues
2016-03-20 10:47:38 -07:00
92b7e0d413
YARN-4686. MiniYARNCluster.start() returns before cluster is completely started. Contributed by Eric Badger.
2016-03-18 16:12:47 +00:00
ca8106d2dd
YARN-4785. inconsistent value type of the type field for LeafQueueInfo in response of RM REST API.
2016-03-17 09:04:41 -07:00
f84af8bd58
YARN-4812. TestFairScheduler#testContinuousScheduling fails intermittently. (kasha)
2016-03-17 05:54:06 -07:00
ae14e5d07f
YARN-4108. CapacityScheduler: Improve preemption to only kill containers that would satisfy the incoming request. (Wangda Tan)
...
(cherry picked from commit 7e8c9beb41
2016-03-16 17:02:33 -07:00
fa7a43529d
Revert "CapacityScheduler: Improve preemption to only kill containers that would satisfy the incoming request. (Wangda Tan)"
...
This reverts commit 7e8c9beb41
2016-03-16 17:02:10 -07:00
7e8c9beb41
CapacityScheduler: Improve preemption to only kill containers that would satisfy the incoming request. (Wangda Tan)
2016-03-16 16:59:59 -07:00
3ef5500783
YARN-4560. Make scheduler error checking message more user friendly. (Ray Chiang via kasha)
2016-03-15 23:45:01 -07:00
a888b5bc53
Revert "YARN-4814. ATS 1.5 timelineclient impl call flush after every event write. (Xuan Gong via gtcarrera9)"
...
This reverts commit 3a0b695441
2016-03-15 22:10:39 -07:00
5176a69868
YARN-4817. Change Log Level to DEBUG for putDomain call in ATS 1.5. (Xuan Gong via gtcarrera9)
2016-03-15 11:47:39 -07:00
3a0b695441
YARN-4814. ATS 1.5 timelineclient impl call flush after every event write. (Xuan Gong via gtcarrera9)
2016-03-15 11:40:54 -07:00
22ca176dfe
TestResourceLocalizationService.testPublicResourceInitializesLocalDir fails Intermittently due to IllegalArgumentException from cleanup (templedf via rkanter)
2016-03-15 10:05:10 -07:00
eba66a64d2
YARN-4816. Fix incompatible change in SystemClock.
2016-03-14 20:14:33 -07:00
20d389ce61
YARN-4719. Add a helper library to maintain node state and allows common queries. (kasha)
2016-03-14 14:19:05 -07:00
f291d82cd4
YARN-4545. Allow YARN distributed shell to use ATS v1.5 APIs. Li Lu via junping_du
2016-03-14 08:28:38 -07:00
017d2c127b
YARN-4696. Improving EntityGroupFSTimelineStore on exception handling, test setup, and concurrency.
...
This commit amends commit d49cfb3504
2016-03-10 13:02:28 -08:00
d49cfb3504
YARN-4696. Improving EntityGroupFSTimelineStore on exception handling, test setup, and concurrency. (Steve Loughran via gtcarrera9)
2016-03-10 10:51:55 -08:00
89b16d27e2
HADOOP-12905. Clean up CHANGES.txt RAT exclusions from pom.xml files. Contributed by Chris Nauroth.
2016-03-08 21:22:06 -08:00
0233d4e0ee
YARN-4465. SchedulerUtils#validateRequest for Label check should happen only when nodelabel enabled. (Bibin A Chundatt via wangda)
2016-03-08 14:27:03 -08:00
3c33158d1c
YARN-4764. Application submission fails when submitted queue is not available in scheduler xml. Contributed by Bibin A Chundatt
2016-03-08 13:07:57 -08:00
f86850b544
HADOOP-12860. Expand section "Data Encryption on HTTP" in SecureMode documentation. Contributed by Wei-Chiu Chuang.
2016-03-08 14:28:47 +09:00
b2661765a5
YARN-4762. Fixed CgroupHandler's creation and usage to avoid NodeManagers crashing when LinuxContainerExecutor is enabled. (Sidharta Seethana via vinodkv)
2016-03-07 11:08:17 -08:00
4163e36c2b
YARN-4760. proxy redirect to history server uses wrong URL. Contributed by Eric Badger
2016-03-07 15:56:33 +00:00
059caf9989
YARN-4744. Too many signal to container failure in case of LCE. Contributed by Sidharta Seethana
2016-03-07 15:40:01 +00:00
8ed2e060e8
YARN-4245. Generalize config file handling in container-executor. Contributed by Sidharta Seethana.
2016-03-07 16:18:35 +05:30
e51a8c1056
YARN-4737. Add CSRF filter support in YARN. Contributed by Jonathan Maron.
2016-03-07 15:26:44 +05:30
e1ccc9622b
YARN-4761. NMs reconnecting with changed capabilities can lead to wrong cluster resource calculations on fair scheduler. Contributed by Sangjin Lee
2016-03-06 19:46:09 -08:00
19ee185907
YARN-4763. RMApps Page crashes with NPE. (Bibin A Chundatt via rohithsharmaks)
2016-03-05 13:02:57 +05:30
cbd31328a6
HADOOP-12470. In-page TOC of documentation should be automatically generated by doxia macro (iwasakims)
2016-03-04 14:11:36 +09:00
1bc454615e
HADOOP-11792. Remove all of the CHANGES.txt files.
2016-03-03 21:02:31 -08:00
5c465df904
YARN-4671. There is no need to acquire CS lock when completing a container. Contributed by Meng Ding
2016-03-01 13:14:12 -08:00
d93c22ec27
YARN-4748. ApplicationHistoryManagerOnTimelineStore should not swallow exceptions on generateApplicationReport. Contributed by Li Lu
2016-02-29 18:19:09 -08:00
9dafaaaf0d
YARN-4704. TestResourceManager#testResourceAllocation() fails when using FairScheduler. (Yufei Gu via kasha)
2016-02-29 16:10:12 -08:00
0fa54d45b1
HADOOP-12813. Migrate TestRPC and related codes to rebase on ProtobufRpcEngine. Contributed by Kai Zheng.
2016-02-29 11:41:00 -08:00
c58a6d53c5
YARN-4731. container-executor should not follow symlinks in recursive_unlink_children. Contributed by Colin Patrick McCabe
2016-02-29 15:24:35 +00:00
e0b14f26f5
YARN-4566. Fix test failure in TestMiniYarnClusterNodeUtilization. (Takashi Ohnishi via rohithsharmaks)
2016-02-29 10:38:14 +08:00
Wangda Tan
Jian He
Vinod Kumar Vavilapalli
Varun Saxena
Ray Chiang
Rohith Sharma K S
Allen Wittenauer
Akira Ajisaka
Xuan
Arun Suresh
Tsuyoshi Ozawa
Sangjin Lee
Junping Du
Karthik Kambatla
Varun Vasudev
Naganarasimha
Jason Lowe
Ming Ma
Kai Zheng
Eric Payne
Karthik Kambatla
Carlo Curino
Robert Kanter
Steve Loughran
Andrew Wang
Sangjin Lee
Yongjun Zhang
Li Lu
Jing Zhao
Chris Douglas
Masatake Iwasaki
naganarasimha
Siddharth Seth
Chris Nauroth
Zhihai Xu
Haohui Mai