Rohith Sharma K S
33455fa415
YARN-2729. Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup. (Naganarasimha G R via rohithsharmaks)
...
(cherry picked from commit 5acdde4744
)
2015-10-26 15:45:22 +05:30
Tsuyoshi Ozawa
e85ebf0825
YARN-3528. Tests with 12345 as hard-coded port break jenkins. Contributed by Brahma Reddy Battula.
...
Conflicts:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeManagerShutdown.java
2015-10-26 17:06:46 +09:00
Allen Wittenauer
599653ad92
YARN-3724. Use POSIX nftw(3) instead of fts(3) (Alan Burlison via aw)
2015-10-25 21:43:51 -07:00
Jonathan Eagles
6db7bfbcfd
YARN-4009. CORS support for ResourceManager REST API. ( Varun Vasudev via jeagles)
...
(cherry picked from commit f8adeb712d
)
2015-10-23 10:43:59 -05:00
Varun Vasudev
e4429de86f
YARN-4267. Add additional logging to container launch implementations in container-executor. Contributed by Sidharta Seethana.
...
(cherry picked from commit 151eab2a12
)
2015-10-19 20:56:43 +05:30
Varun Vasudev
196d549e2b
YARN-4262. Allow whitelisted users to run privileged docker containers. Contributed by Sidharta Seethana.
...
(cherry picked from commit e39ae0e676
)
2015-10-19 20:21:48 +05:30
Steve Loughran
390f29ab99
YARN-4155. TestLogAggregationService.testLogAggregationServiceWithInterval failing. (Bibin A Chundatt via stevel)
2015-10-18 12:03:46 +01:00
Varun Vasudev
342870b70f
YARN-4258. Add support for controlling capabilities for docker containers. Contributed by Sidharta Seethana.
...
(cherry picked from commit 63020c54c1
)
2015-10-15 13:41:58 +05:30
Steve Loughran
0e4fb329ed
HADOOP-12478. Shell.getWinUtilsPath() has been renamed Shell.getWinutilsPath(). (stevel)
2015-10-14 20:25:33 +01:00
Varun Vasudev
2dd9c475c4
YARN-4255. container-executor does not clean up docker operation command files. Contributed by Sidharta Seethana.
...
(cherry picked from commit da1ee078f9
)
2015-10-14 15:27:18 +05:30
Varun Vasudev
0aeedfd572
YARN-4252. Log container-executor invocation details when exit code is non-zero. Contributed by Sidharta Seethana.
...
(cherry picked from commit fb9c1519b3
)
2015-10-14 15:19:19 +05:30
Varun Vasudev
a9b03b5afb
YARN-4253. Standardize on using PrivilegedOperationExecutor for all invocations of container-executor in LinuxContainerExecutor. Contributed by Sidharta Seethana.
...
(cherry picked from commit 8d59293089
)
Conflicts:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LinuxContainerExecutor.java
2015-10-14 15:04:22 +05:30
Steve Loughran
5eab51a24f
HADOOP-10775. Shell operations to fail with meaningful errors on windows if winutils.exe not found. (stevel)
2015-10-13 21:47:54 +01:00
Varun Vasudev
34b8142802
YARN-4017. container-executor overuses PATH_MAX. Contributed by Sidharta Seethana
...
(cherry picked from commit 5b6bae0094
)
2015-10-13 12:50:28 +05:30
Jason Lowe
0139517811
YARN-261. Ability to fail AM attempts. Contributed by Andrey Klochkov and Rohith Sharma K S
...
(cherry picked from commit a0bca2b5ad
)
2015-10-09 14:19:41 +00:00
Jason Lowe
ec74b0cd4e
YARN-3943. Use separate threshold configurations for disk-full detection and disk-not-full detection. Contributed by Zhihai Xu
...
(cherry picked from commit 8d226225d0
)
2015-10-08 22:26:35 +00:00
Wangda Tan
d24a3b9a3c
YARN-4176. Resync NM nodelabels with RM periodically for distributed nodelabels. (Bibin A Chundatt via wangda)
...
(cherry picked from commit 30ac69c6bd
)
2015-10-05 15:48:40 -07:00
Vinod Kumar Vavilapalli
b876babdc7
HADOOP-12441. Fixed shell-kill command behaviour to work correctly on some Linux distributions after HADOOP-12317. Contributed by Wangda Tan.
...
(cherry picked from commit f1c19b9365
)
2015-10-05 10:57:55 -07:00
Xuan
53bddc410f
YARN-1897. CLI and core support for signal container functionality. Contributed by Ming Ma
...
(cherry picked from commit 8f08532bde
)
2015-10-02 18:53:04 -07:00
Jason Lowe
3e3733437f
YARN-3619. ContainerMetrics unregisters during getMetrics and leads to ConcurrentModificationException. Contributed by Zhihai Xu
...
(cherry picked from commit fdf02d1f26
)
2015-10-02 20:21:32 +00:00
Jason Lowe
d645ee1d62
YARN-3727. For better error recovery, check if the directory exists before using it for localization. Contributed by Zhihai Xu
...
(cherry picked from commit 854d25b0c3
)
2015-09-30 15:01:01 +00:00
Jian He
d592aa115b
YARN-3868. Recovery support for container resizing. Contributed by Meng Ding
...
(cherry picked from commit c57eac5dfe
)
2015-09-24 10:24:13 -07:00
Jian He
b6c594c6ea
YARN-1644. RM-NM protocol changes and NodeStatusUpdater implementation to support container resizing. Contributed by Meng Ding
...
(cherry picked from commit c3dc1af072
)
2015-09-24 10:24:04 -07:00
Jian He
b8955d81c5
YARN-1643. Make ContainersMonitor support changing monitoring size of an allocated container. Contributed by Meng Ding and Wangda Tan
...
(cherry picked from commit c59ae4eeb1
)
2015-09-24 10:23:55 -07:00
Jian He
fa0a554ae0
YARN-3867. ContainerImpl changes to support container resizing. Contributed by Meng Ding
...
(cherry picked from commit 5f5a968d65
)
2015-09-24 10:23:46 -07:00
Jian He
afe4afd0c9
YARN-1645. ContainerManager implementation to support container resizing. Contributed by Meng Ding & Wangda Tan
...
(cherry picked from commit ffd820c27a
)
2015-09-24 10:23:35 -07:00
Jian He
c19e8ffdc7
YARN-1449. AM-NM protocol changes to support container resizing. Contributed by Meng Ding & Wangda Tan)
...
(cherry picked from commit 83a18add10
)
2015-09-24 10:23:25 -07:00
Rohith Sharma K S
ace4d26936
YARN-4152. NodeManager crash with NPE when LogAggregationService#stopContainer called for absent container. (Bibin A Chundatt via rohithsharmaks)
2015-09-24 11:31:47 +05:30
Jason Lowe
c829be5133
YARN-4095. Avoid sharing AllocatorPerContext object in LocalDirAllocator between ShuffleHandler and LocalDirsHandlerService. Contributed by Zhihai Xu
...
(cherry picked from commit c890c51a91
)
2015-09-23 15:44:13 +00:00
Xuan
f3c0a210b6
YARN-4149. yarn logs -am should provide an option to fetch all the log
...
files. Contributed by Varun Vasudev
(cherry picked from commit 34ef1a092b
)
2015-09-15 14:37:59 -07:00
Jason Lowe
181bda0421
YARN-4158. Remove duplicate close for LogWriter in AppLogAggregatorImpl#uploadLogsForContainers. Contributed by Zhihai Xu
...
(cherry picked from commit 8c1cdb17a0
)
2015-09-15 20:22:23 +00:00
Varun Vasudev
a4288f1262
YARN-4136. LinuxContainerExecutor loses info when forwarding ResourceHandlerException. Contributed by Bibin A Chundatt.
...
(cherry picked from commit 486d5cb803
)
2015-09-11 14:39:01 +05:30
Wangda Tan
5decd8f9f7
YARN-4106. NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM. (Bibin A Chundatt via wangda)
...
(cherry picked from commit 77666105b4
)
2015-09-10 09:30:34 -07:00
Zhihai Xu
214338bd68
YARN-4096. App local logs are leaked if log aggregation fails to initialize for the app. Contributed by Jason Lowe.
...
(cherry picked from commit 16b9037dc1
)
2015-09-08 12:33:29 -07:00
Jian He
f23be93dd1
YARN-2884. Added a proxy service in NM to proxy the the communication between AM and RM. Contributed by Kishore Chaliparambil
...
(cherry picked from commit 6f72f1e600
)
2015-09-08 11:24:07 +08:00
Varun Vasudev
70575286b7
YARN-3591. Resource localization on a bad disk causes subsequent containers failure. Contributed by Lavkesh Lahngir.
...
(cherry picked from commit 1dbd8e34a7
)
2015-09-07 11:34:14 +05:30
Rohith Sharma K S
7ecdf51441
YARN-4073. Removed unused ApplicationACLsManager in ContainerManagerImpl constructor. (Naganarasimha G R via rohithsharmaks)
...
(cherry picked from commit 095ab9ab5f
)
2015-09-02 14:28:50 +05:30
Xuan
85363ea4bd
YARN-221. Addendum patch to compilation issue which is caused by missing
...
AllContainerLogAggregationPolicy. Contributed by Xuan Gong
(cherry picked from commit b71c6006f5
)
2015-08-23 16:50:02 -07:00
Xuan
205b98c059
YARN-221. NM should provide a way for AM to tell it not to aggregate
...
logs. Contributed by Ming Ma
(cherry picked from commit 37e1c3d82a
)
2015-08-22 16:26:39 -07:00
Wangda Tan
6452b31760
YARN-2923. Support configuration based NodeLabelsProvider Service in Distributed Node Label Configuration Setup. (Naganarasimha G R)
...
(cherry picked from commit fc07464d1a
)
2015-08-20 16:52:57 -07:00
Karthik Kambatla
b567aa2b4f
YARN-4055. Report node resource utilization in heartbeat. (Inigo Goiri via kasha)
...
(cherry picked from commit 13604bd5f1
)
2015-08-20 16:52:33 -07:00
Zhihai Xu
051a622b57
YARN-4057. If ContainersMonitor is not enabled, only print related log info one time. Contributed by Jun Gong.
...
(cherry picked from commit 14215c8ef8
)
2015-08-18 11:47:41 -07:00
Karthik Kambatla
d2ff763533
YARN-3534. Collect memory/cpu usage on the node. (Inigo Goiri via kasha)
...
(cherry picked from commit def12933b3
)
2015-08-16 06:26:21 -07:00
Jian He
4a5fdb7c23
YARN-4005. Completed container whose app is finished is possibly not removed from NMStateStore. Contributed by Jun Gong
...
(cherry picked from commit 38aed1a94e
)
2015-08-13 14:46:34 -07:00
Junping Du
7af531d2e3
YARN-4019. Add JvmPauseMonitor to ResourceManager and NodeManager. Contributed by Robert Kanter.
...
(cherry picked from commit cfee02b3bd
)
2015-08-06 06:50:40 -07:00
Xuan
89ccfd9fcc
YARN-4004. container-executor should print output of docker logs if the
...
docker container exits with non-0 exit status. Contributed by Varun
Vasudev
(cherry picked from commit c3364ca8e7
)
2015-08-03 18:11:16 -07:00
Jason Lowe
b264217bc3
YARN-3965. Add startup timestamp to nodemanager UI. Contributed by Hong Zhiguo
...
(cherry picked from commit 469cfcd695
)
2015-08-03 15:55:04 +00:00
Xuan
2ec008eb38
YARN-3982. container-executor parsing of container-executor.cfg broken
...
in trunk and branch-2. Contributed by Varun Vasudev
(cherry picked from commit f170934215
)
2015-07-27 23:47:31 -07:00
Varun Vasudev
9da487e0fd
YARN-3853. Add docker container runtime support to LinuxContainterExecutor. Contributed by Sidharta Seethana.
...
(cherry picked from commit 3e6fce91a4
)
Conflicts:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LinuxContainerExecutor.java
2015-07-27 12:03:36 -07:00
Varun Vasudev
ec0f801f52
YARN-3852. Add docker container support to container-executor. Contributed by Abin Shahab.
...
(cherry picked from commit f36835ff9b
)
2015-07-27 10:18:06 -07:00
Jason Lowe
2cf714123a
YARN-3925. ContainerLogsUtils#getContainerLogFile fails to read container log files from full disks. Contributed by zhihai xu
...
(cherry picked from commit ff9c13e0a7
)
2015-07-24 22:15:44 +00:00
Wangda Tan
3800e25d6e
YARN-3956. Fix TestNodeManagerHardwareUtils fails on Mac (Varun Vasudev via wangda)
...
(cherry picked from commit 06e5dd2c84
)
Conflicts:
hadoop-yarn-project/CHANGES.txt
2015-07-22 12:03:14 -07:00
Colin Patrick Mccabe
aff955a14a
YARN-3844. Make hadoop-yarn-project Native code -Wall-clean (Alan Burlison via Colin P. McCabe)
...
(cherry picked from commit 419c51d233
)
2015-07-17 12:05:24 -07:00
Akira Ajisaka
0ba2defb9e
YARN-3381. Fix typo InvalidStateTransitonException. Contributed by Brahma Reddy Battula.
...
(cherry picked from commit 19295b36d9
)
2015-07-13 17:53:27 +09:00
Zhijie Shen
37a93c2d78
YARN-3116. RM notifies NM whether a container is an AM container or normal task container. Contributed by Giovanni Matteo Fumarola.
...
(cherry picked from commit 1ea36299a4
)
2015-07-10 19:02:13 -07:00
Karthik Kambatla
d15f69457b
YARN-1012. Report NM aggregated container resource utilization in heartbeat. (Inigo Goiri via kasha)
...
(cherry picked from commit 527c40e4d6
)
2015-07-09 09:35:39 -07:00
Varun Vasudev
d17a7bbf6e
YARN-2194. Addendum patch to fix failing unit test in TestPrivilegedOperationExecutor. Contributed by Sidharta Seethana.
...
(cherry picked from ad5b904175a1f89951c7c32f744a4e20fafb51b5)
2015-07-09 11:52:57 +05:30
Varun Vasudev
16ca0fcf99
YARN-2194. Fix bug causing CGroups functionality to fail on RHEL7. Contributed by Wei Yan.
...
(cherry-picked from commit 7e3ebc0fc59dbd1d7768b5a6c05af18b8c8fea4d)
2015-07-07 17:05:06 +05:30
Jason Lowe
a721b9d812
YARN-3793. Several NPEs when deleting local files on NM recovery. Contributed by Varun Saxena
...
(cherry picked from commit b5cdf78e8e
)
2015-07-01 21:14:35 +00:00
Colin Patrick Mccabe
b1da58d23d
YARN-3827. Migrate YARN native build to new CMake framework (Alan Burlison via Colin P. McCabe)
...
(cherry picked from commit d0cc0380b5
)
2015-06-30 16:24:44 -07:00
Jian He
0100995c5e
YARN-3695. ServerProxy (NMProxy, etc.) shouldn't retry forever for non network exception. Contributed by Raju Bairishetti
...
(cherry picked from commit 62e583c7dc
)
2015-06-29 13:38:04 -07:00
Jason Lowe
0221d19f4e
YARN-3850. NM fails to read files from full disks which can lead to container logs being lost and other issues. Contributed by Varun Saxena
...
(cherry picked from commit 40b256949a
)
2015-06-26 15:48:05 +00:00
Jason Lowe
15b1800b12
YARN-3832. Resource Localization fails on a cluster due to existing cache directories. Contributed by Brahma Reddy Battula
...
(cherry picked from commit 8d58512d6e
)
2015-06-24 16:38:22 +00:00
Karthik Kambatla
9656ee4ee7
YARN-3842. NMProxy should retry on NMNotYetReadyException. (Robert Kanter via kasha)
...
(cherry picked from commit 5ebf2817e5
)
2015-06-22 17:48:07 -07:00
Xuan
c3254a63a9
YARN-3834. Scrub debug logging of tokens during resource localization. Contributed by Chris Nauroth
...
(cherry picked from commit 6c7a9d502a
)
2015-06-21 17:14:59 -07:00
Devaraj K
8c643e3bf9
YARN-3747. TestLocalDirsHandlerService should delete the created test
...
directory logDir2. Contributed by David Moore.
(cherry picked from commit 126321eded
)
2015-06-08 15:33:38 +05:30
Junping Du
d8c7ee1b53
YARN-41. The RM should handle the graceful shutdown of the NM. Contributed by Devaraj K.
...
(cherry picked from commit d7e7f6aa03
)
2015-06-04 05:00:32 -07:00
Jason Lowe
752caa95a4
YARN-3585. NodeManager cannot exit on SHUTDOWN event triggered and NM recovery is enabled. Contributed by Rohith Sharmaks
...
(cherry picked from commit e13b671aa5
)
2015-06-03 19:45:18 +00:00
Robert Kanter
bde4031a02
YARN-3713. Remove duplicate function call storeContainerDiagnostics in ContainerDiagnosticsUpdateTransition (zxu via rkanter)
...
(cherry picked from commit 6aec13cb33
)
2015-05-29 15:35:08 -07:00
Andrew Wang
bdaa0fed13
HADOOP-11885. hadoop-dist dist-layout-stitching.sh does not work with dash. (wang)
...
(cherry picked from commit 7673d4f205
)
Conflicts:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/pom.xml
hadoop-mapreduce-project/pom.xml
2015-05-29 14:55:52 -07:00
cnauroth
6ed8989a6f
YARN-3626. On Windows localized resources are not moved to the front of the classpath when they should be. Contributed by Craig Welch.
...
(cherry picked from commit 7bc33b63c78c6ee9a04c0c1511ef16e63f7f3074)
2015-05-27 14:21:15 -07:00
Vinod Kumar Vavilapalli
a67cb4826b
YARN-160. Enhanced NodeManager to automatically obtain cpu/memory values from underlying OS when configured to do so. Contributed by Varun Vasudev.
...
(cherry picked from commit 500a1d9c76
)
2015-05-26 11:39:40 -07:00
Junping Du
5fb61d71b1
YARN-3594. WintuilsProcessStubExecutor.startStreamReader leaks streams. Contributed by Lars Francke.
...
(cherry picked from commit 132d909d4a
)
2015-05-22 04:25:18 -07:00
Vinod Kumar Vavilapalli
673bd9708f
YARN-3684. Changed ContainerExecutor's primary lifecycle methods to use a more extensible mechanism of context objects. Contributed by Sidharta Seethana.
...
(cherry picked from commit 53fafcf061
)
2015-05-21 15:51:19 -07:00
Jian He
7c3895c8f6
YARN-3654. ContainerLogsPage web UI should not have meta-refresh. Contributed by Xuan Gong
...
(cherry picked from commit 6329bd00fa
)
2015-05-20 17:21:19 -07:00
Wangda Tan
7b97f049ee
YARN-3565. NodeHeartbeatRequest/RegisterNodeManagerRequest should use NodeLabel object instead of String. (Naganarasimha G R via wangda)
...
(cherry picked from commit b37da52a1c
)
2015-05-19 16:35:23 -07:00
Colin Patrick Mccabe
9534e533dc
HADOOP-11970. Replace uses of ThreadLocal<Random> with JDK7 ThreadLocalRandom (Sean Busbey via Colin P. McCabe)
...
(cherry picked from commit 470c87dbc6
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicy.java
hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/JspHelper.java
2015-05-19 11:15:04 -07:00
Ravi Prakash
084e453629
YARN-3302. TestDockerContainerExecutor should run automatically if it can detect docker in the usual place (Ravindra Kumar Naik via raviprak)
...
(cherry picked from commit c97f32e7b9
)
2015-05-19 10:30:24 -07:00
Junping Du
cc671349bc
YARN-3505 addendum: fix an issue in previous patch.
...
(cherry picked from commit 03a293aed6
)
2015-05-15 06:40:28 -07:00
Ravi Prakash
4caadaa8bb
YARN-1519. Check in container-executor if sysconf is implemented before using it (Radim Kolar and Eric Payne via raviprak)
...
(cherry picked from commit 53fe4eff09
)
2015-05-14 15:56:36 -07:00
Junping Du
bc13c7d84b
YARN-3505. Node's Log Aggregation Report with SUCCEED should not cached in RMApps. Contributed by Xuan Gong.
...
(cherry picked from commit 15ccd967ee
)
2015-05-14 10:59:48 -07:00
Jason Lowe
a81ad81461
YARN-3641. NodeManager: stopRecoveryStore() shouldn't be skipped when exceptions happen in stopping NM's sub-services. Contributed by Junping Du
...
(cherry picked from commit 711d77cc54
)
2015-05-13 21:08:45 +00:00
Xuan
487d9b0f3f
YARN-3626. On Windows localized resources are not moved to the front of the classpath when they should be. Contributed by Craig Welch
...
(cherry picked from commit 0f95921447
)
2015-05-13 13:12:04 -07:00
Devaraj K
5fceebd696
YARN-3629. NodeID is always printed as "null" in node manager
...
initialization log. Contributed by nijel.
(cherry picked from commit 5c2f05cd9b
)
2015-05-12 22:22:40 +05:30
Devaraj K
f887243f88
YARN-3513. Remove unused variables in ContainersMonitorImpl and add debug
...
log for overall resource usage by all containers. Contributed by
Naganarasimha G R.
(cherry picked from commit 8badd82ce2
)
2015-05-12 16:56:26 +05:30
Karthik Kambatla
e8dd0bc4fd
YARN-1287. Consolidate MockClocks. (Sebastian Wong and Anubhav Dhoot via kasha)
...
(cherry picked from commit 70fb37cd79
)
2015-05-09 14:35:13 -07:00
Xuan
acb342b77c
YARN-1912. ResourceLocalizer started without any jvm memory control.
...
Contributed by Masatake Iwasaki
(cherry picked from commit 6471d18bc7
)
2015-05-08 20:02:23 -07:00
Xuan
fc980247a6
YARN-3602. TestResourceLocalizationService.testPublicResourceInitializesLocalDir fails Intermittently due to IOException from cleanup. Contributed by zhihai xu
...
(cherry picked from commit 333f9a896d
)
2015-05-08 17:54:46 -07:00
Jason Lowe
743ca1a056
YARN-3476. Nodemanager can fail to delete local logs if log aggregation fails. Contributed by Rohith
...
(cherry picked from commit 25e2b02122
)
2015-05-08 22:46:34 +00:00
Xuan
8d07116266
YARN-2331. Distinguish shutdown during supervision vs. shutdown for
...
rolling upgrade. Contributed by Jason Lowe
(cherry picked from commit 088156de43
)
2015-05-08 15:11:55 -07:00
Robert (Bobby) Evans
28e0593b96
YARN-644: Basic null check is not performed on passed in arguments before using them in ContainerManagerImpl.startContainer
...
(cherry picked from commit bcf2890502
)
2015-05-08 11:11:53 -05:00
Devaraj K
bf8be56292
YARN-2784. Make POM project names consistent. Contributed by Rohith.
2015-05-08 19:26:11 +05:30
Akira Ajisaka
170f1b0afd
YARN-1832. Fix wrong MockLocalizerStatus#equals implementation. Contributed by Hong Zhiguo.
...
(cherry picked from commit b167fe7605
)
2015-05-08 11:15:28 +09:00
Akira Ajisaka
67bebabb21
YARN-3577. Misspelling of threshold in log4j.properties for tests. Contributed by Brahma Reddy Battula.
...
(cherry picked from commit 995f1b5831fc33042de90267a689333dc110c4c3)
2015-05-07 13:33:17 +09:00
Robert Kanter
28b40aca98
YARN-3491. PublicLocalizer#addResource is too slow. (zxu via rkanter)
...
(cherry picked from commit b72507810a
)
Conflicts:
hadoop-yarn-project/CHANGES.txt
2015-05-06 14:20:08 -07:00
Junping Du
6ea007ba91
YARN-3396. Handle URISyntaxException in ResourceLocalizationService. (Contributed by Brahma Reddy Battula)
...
(cherry picked from commit 3810242062
)
2015-05-05 10:19:16 -07:00
Wangda Tan
4e77ee2f6f
YARN-3375. NodeHealthScriptRunner.shouldRun() check is performing 3 times for starting NodeHealthScriptRunner (Devaraj K via wangda)
...
(cherry picked from commit 71f4de220c
)
2015-05-04 16:02:57 -07:00
Allen Wittenauer
02e650248d
YARN-2980. Move health check script related functionality to hadoop-common (Varun Saxena via aw)
...
(cherry picked from commit d4ac6822e1
)
Conflicts:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
2015-05-04 16:00:20 -07:00
Jason Lowe
99b49a8181
YARN-3097. Logging of resource recovery on NM restart has redundancies. Contributed by Eric Payne
...
(cherry picked from commit 8f65c793f2
)
2015-05-04 10:32:33 -05:00
Robert Kanter
32b3b8e2ef
YARN-3363. add localization and container launch time to ContainerMetrics at NM to show these timing information for each active container. (zxu via rkanter)
...
(cherry picked from commit ac7d152901
)
2015-05-01 16:39:59 -07:00
Vinod Kumar Vavilapalli
90f6e8c90f
YARN-2619. Added NodeManager support for disk io isolation through cgroups. Contributed by Varun Vasudev and Wei Yan.
...
(cherry picked from commit 1b3b9e5c31
)
2015-04-30 21:43:22 -07:00
Karthik Kambatla
4045c41afe
YARN-3464. Race condition in LocalizerRunner kills localizer before localizing all resources. (Zhihai Xu via kasha)
...
(cherry picked from commit 47279c3228
)
2015-04-26 09:14:12 -07:00
Jason Lowe
cf4154676b
YARN-3537. NPE when NodeManager.serviceInit fails and stopRecoveryStore invoked. Contributed by Brahma Reddy Battula
...
(cherry picked from commit 5e093f0d40
)
2015-04-24 22:04:03 +00:00
Xuan
8f6053ae51
YARN-3516. killing ContainerLocalizer action doesn't take effect when
...
private localizer receives FETCH_FAILURE status. Contributed by zhihai
xu
(cherry picked from commit 0b3f8957a8
)
2015-04-23 16:41:52 -07:00
Vinod Kumar Vavilapalli
04783b0402
YARN-3366. Enhanced NodeManager to support classifying/shaping outgoing network bandwidth traffic originating from YARN containers Contributed by Sidharta Seethana.
...
(cherry picked from commit a100be685c
)
2015-04-22 17:36:55 -07:00
Jian He
613a783380
YARN-3503. Expose disk utilization percentage and bad local and log dir counts in NM metrics. Contributed by Varun Vasudev
...
(cherry picked from commit 674c7ef649
)
Conflicts:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
2015-04-21 21:06:06 -07:00
Junping Du
b244701fad
YARN-1402. Update related Web UI and CLI with exposing client API to check log aggregation status. Contributed by Xuan Gong.
...
(cherry picked from commit 1db355a875
)
2015-04-17 13:20:27 -07:00
Jian He
6fed2c2a79
YARN-3354. Add node label expression in ContainerTokenIdentifier to support RM recovery. Contributed by Wangda Tan
...
(cherry picked from commit 1b89a3e173
)
2015-04-15 14:03:29 -07:00
Harsh J
01af29106a
YARN-3462. Patches applied for YARN-2424 are inconsistent between trunk and branch-2. Contributed by Naganarasimha G R.
2015-04-15 09:41:43 +05:30
Junping Du
7c072bf092
YARN-3443. Create a 'ResourceHandler' subsystem to ease addition of support for new resource types on the NM. Contributed by Sidharta Seethana.
...
(cherry picked from commit 838b06ac87
)
2015-04-13 18:37:39 -07:00
Junping Du
0891de0712
YARN-1376. NM need to notify the log aggregation status to RM through Node heartbeat. Contributed by Xuan Gong.
...
(cherry picked from commit 92431c9617
)
2015-04-10 08:58:07 -07:00
Karthik Kambatla
53e0bf5c17
YARN-3465. Use LinkedHashMap to preserve order of resource requests. (Zhihai Xu via kasha)
...
(cherry picked from commit 6495940eae
)
2015-04-09 00:10:23 -07:00
Tsuyoshi Ozawa
ac32fa187c
YARN-3457. NPE when NodeManager.serviceInit fails and stopRecoveryStore called. Contributed by Bibin A Chundatt.
...
(cherry picked from commit dd852f5b8c
)
2015-04-08 15:56:33 +09:00
Wangda Tan
405aead879
YARN-2901. Add errors and warning metrics page to RM, NM web UI. (Varun Vasudev via wangda)
...
(cherry picked from commit bad070fe15
)
2015-04-02 17:23:44 -07:00
Vinod Kumar Vavilapalli
d8e17c58bc
YARN-3365. Enhanced NodeManager to support using the 'tc' tool via container-executor for outbound network traffic control. Contributed by Sidharta Seethana.
...
(cherry picked from commit b21c72777a
)
2015-04-02 16:55:00 -07:00
Tsuyoshi Ozawa
80755edb7a
YARN-3424. Change logs for ContainerMonitorImpl's resourse monitoring from info to debug. Contributed by Anubhav Dhoot.
...
(cherry picked from commit c69ba81497
)
2015-04-01 17:44:40 +09:00
Karthik Kambatla
93b29d962d
YARN-3428. Debug log resources to be localized for a container. (kasha)
...
(cherry picked from commit 2daa478a64
)
2015-03-31 17:35:11 -07:00
Wangda Tan
cba4ed1678
YARN-2495. Allow admin specify labels from each NM (Distributed configuration for node label). (Naganarasimha G R via wangda)
...
(cherry picked from commit 2a945d24f7
)
2015-03-30 12:05:54 -07:00
Vinod Kumar Vavilapalli
c5bc48946d
YARN-3304. Cleaning up ResourceCalculatorProcessTree APIs for public use and removing inconsistencies in the default values. Contributed by Junping Du and Karthik Kambatla.
...
(cherry picked from commit c358368f51
)
2015-03-30 10:11:12 -07:00
Ravi Prakash
b1b4951452
YARN-3288. Document and fix indentation in the DockerContainerExecutor code
...
(cherry picked from commit e0ccea33c9
)
2015-03-28 08:01:26 -07:00
Tsuyoshi Ozawa
cbacf20755
YARN-3384. TestLogAggregationService.verifyContainerLogs fails after YARN-2777. Contributed by Naganarasimha G R.
...
(cherry picked from commit 82eda771e0
)
2015-03-24 00:25:52 +09:00
Junping Du
f40f17489c
YARN-3269. Yarn.nodemanager.remote-app-log-dir could not be configured to fully qualified path. Contributed by Xuan Gong
...
(cherry picked from commit d81109e588
)
2015-03-20 13:42:31 -07:00
Karthik Kambatla
05627dd96d
YARN-3351. AppMaster tracking URL is broken in HA. (Anubhav Dhoot via kasha)
...
(cherry picked from commit 20b49224eb
)
2015-03-18 16:33:57 -07:00
Ravi Prakash
9f227ad696
YARN-3339. TestDockerContainerExecutor should pull a single image and not the entire centos repository. (Ravindra Kumar Naik via raviprak)
...
(cherry picked from commit 56085203c4
)
2015-03-16 16:18:42 -07:00
Tsuyoshi Ozawa
d6e3ef2506
YARN-1453. [JDK8] Fix Javadoc errors caused by incorrect or illegal tags in doc comments. Contributed by Akira AJISAKA, Andrew Purtell, and Allen Wittenauer.
...
(cherry picked from commit 3da9a97cfb
)
2015-03-16 23:20:17 +09:00
Vinod Kumar Vavilapalli
53aa3a4d1f
YARN-3154. Added additional APIs in LogAggregationContext to avoid aggregating running logs of application when rolling is enabled. Contributed by Xuan Gong.
...
(cherry picked from commit 863079bb87
)
2015-03-12 13:33:42 -07:00
Vinod Kumar Vavilapalli
d9416317a9
Preparing for 2.8.0 development. Changing SNAPSHOT version to be 2.8 on branch-2.
2015-03-08 20:32:51 -07:00
Jian He
6cef2c16de
YARN-2190. Added CPU and memory limit options to the default container executor for Windows containers. Contributed by Chuan Liu
...
(cherry picked from commit 21101c01f2
)
2015-03-06 14:18:56 -08:00
Karthik Kambatla
1749094fab
YARN-3122. Metrics for container's actual CPU usage. (Anubhav Dhoot via kasha)
...
(cherry picked from commit 53947f37c7
)
2015-03-04 17:39:39 -08:00
Konstantin V Shvachko
9e67f2cb0d
YARN-3255. RM, NM, JobHistoryServer, and WebAppProxyServer's main() should support generic options. Contributed by Konstantin Shvachko.
2015-02-26 17:26:08 -08:00
Xuan
9cbe9bcbaa
YARN-3237. AppLogAggregatorImpl fails to log error cause. Contributed by
...
Rushabh S Shah
(cherry picked from commit f56c65bb3e
)
2015-02-20 14:03:46 -08:00
Junping Du
4a87a61fe9
YARN-2799. Cleanup TestLogAggregationService based on the change in YARN-90. Contributed by Zhihai Xu
...
(cherry picked from commit c33ae271c2
)
2015-02-20 09:44:31 -08:00
Tsuyoshi Ozawa
fd8f7e2c7d
YARN-3203. Correct a log message in AuxServices. Contributed by Brahma Reddy Battula.
...
(cherry picked from commit 447bd7b5a6
)
2015-02-16 23:56:32 +09:00
Junping Du
eee89bd18c
YARN-2749. Fix some testcases from TestLogAggregationService fails in trunk. (Contributed by Xuan Gong)
...
(cherry picked from commit ab0b958a52
)
2015-02-15 06:48:29 -08:00
cnauroth
48302e687a
YARN-2899. Run TestDockerContainerExecutorWithMocks on Linux only. Contributed by Ming Ma.
...
(cherry picked from commit 6804d68901
)
2015-02-13 21:59:14 -08:00
Jason Lowe
c9cd58d711
YARN-2847. Linux native container executor segfaults if default banned user detected. Contributed by Olaf Flebbe
...
(cherry picked from commit 1a0f508b63
)
2015-02-13 20:22:12 +00:00
Akira Ajisaka
9d8dbf8ea4
YARN-3191. Log object should be initialized with its own class. Contributed by Rohith.
...
(cherry picked from commit 99c7f2b8fa417303e8f42ea3641c302135879280)
2015-02-12 17:59:04 -08:00
Junping Du
380cc4dbed
YARN-2079. Recover NonAggregatingLogHandler state upon nodemanager restart. (Contributed by Jason Lowe)
...
(cherry picked from commit 04f5ef18f7
)
2015-02-12 11:48:24 -08:00
Jason Lowe
38333c8f29
YARN-3074. Nodemanager dies when localizer runner tries to write to a full disk. Contributed by Varun Saxena
...
(cherry picked from commit b379972ab3
)
2015-02-11 16:34:42 +00:00
Jason Lowe
ca11ffa5de
YARN-2809. Implement workaround for linux kernel panic when removing cgroup. Contributed by Nathan Roberts
...
(cherry picked from commit 3f5431a22f
)
2015-02-10 17:28:18 +00:00
Jason Lowe
9ffd985dc0
YARN-3090. DeletionService can silently ignore deletion task failures. Contributed by Varun Saxena
...
(cherry picked from commit 4eb5f7fa32
)
2015-02-10 16:55:21 +00:00
Jason Lowe
83449a4e4d
YARN-3089. LinuxContainerExecutor does not handle file arguments to deleteAsUser. Contributed by Eric Payne
...
(cherry picked from commit 4c484320b4
)
2015-02-06 20:40:33 +00:00
Arun C. Murthy
92ff524182
YARN-1537. Fix race condition in TestLocalResourcesTrackerImpl.testLocalResourceCache. Contributed by Xuan Gong.
2015-02-05 23:59:34 -08:00
Xuan
c22dcdd191
YARN-3056. Add verification for containerLaunchDuration in
...
TestNodeManagerMetrics. Contributed by zhihai Xu
(cherry picked from commit b73e776abc
)
2015-02-03 15:14:23 -08:00
Robert Kanter
410830fe8c
YARN-3022. Expose Container resource information from NodeManager for monitoring (adhoot via ranter)
...
(cherry picked from commit f7a77819a1
)
2015-02-03 10:39:51 -08:00
Akira Ajisaka
1081b4c4a7
HADOOP-9907. Webapp http://hostname:port/metrics link is not working. (aajisaka)
...
(cherry picked from commit 342efa110a
)
2015-01-30 02:50:46 +09:00
Jian He
8100c8a68c
YARN-3011. Possible IllegalArgumentException in ResourceLocalizationService might lead NM to crash. Contributed by Varun Saxena
...
(cherry picked from commit 4e15fc0841
)
2015-01-27 13:31:48 -08:00
Jason Lowe
07fe6a36cb
YARN-3088. LinuxContainerExecutor.deleteAsUser can throw NPE if native executor returns an error. Contributed by Eric Payne
...
(cherry picked from commit 902c6ea7e4
)
2015-01-26 15:41:23 +00:00
Xuan
a7696b3fbf
YARN-3024. LocalizerRunner should give DIE action when all resources are
...
localized. Contributed by Chengbing Liu
(cherry picked from commit 0d6bd62102
)
2015-01-25 19:39:52 -08:00
Tsuyoshi Ozawa
ff627d94e7
YARN-3082. Non thread safe access to systemCredentials in NodeHeartbeatResponse processing. Contributed by Anubhav Dhoot.
...
(cherry picked from commit 3aab354e66
)
2015-01-23 16:05:05 +09:00
Karthik Kambatla
4d8fa9615f
YARN-2984. Metrics for container's actual memory usage. (kasha)
...
(cherry picked from commit 84198564ba
)
2015-01-17 06:26:47 +05:30
Junping Du
7cddec31d7
YARN-3064. TestRMRestart/TestContainerResourceUsage/TestNodeManagerResync failure with allocation timeout. (Contributed by Jian He)
...
(cherry picked from commit 5d1cca34fa
)
2015-01-16 00:10:36 -08:00
Jian He
e7e6173049
YARN-2997. Fixed NodeStatusUpdater to not send alreay-sent completed container statuses on heartbeat. Contributed by Chengbing Liu
...
(cherry picked from commit cc2a745f7e
)
2015-01-08 11:28:24 -08:00
Zhijie Shen
f02bd6683a
YARN-2937. Fixed new findbugs warnings in hadoop-yarn-nodemanager. Contributed by Varun Saxena.
...
(cherry picked from commit 41a548a916
)
2014-12-23 20:35:32 -08:00
Jian He
9180d11b3b
YARN-2952. Fixed incorrect version check in StateStore. Contributed by Rohith Sharmaks
...
(cherry picked from commit 808cba3821
)
2014-12-19 16:56:55 -08:00
Karthik Kambatla
b4e8ae591d
YARN-2675. containersKilled metrics is not updated when the container is killed during localization. (Zhihai Xu via kasha)
...
(cherry picked from commit 954fb8581e
)
2014-12-19 16:03:02 -08:00
cnauroth
36068768d8
HADOOP-11321. copyToLocal cannot save a file to an SMB share unless the user has Full Control permissions. Contributed by Chris Nauroth.
...
(cherry picked from commit e996a1bfd4
)
2014-12-16 15:32:23 -08:00
Steve Loughran
a858d726c8
YARN-2912 Jersey Tests failing with port in use. (varun saxena via stevel)
2014-12-12 17:10:54 +00:00
Karthik Kambatla
9d72b0282f
YARN-2931. PublicLocalizer may fail until directory is initialized by LocalizeRunner. (Anubhav Dhoot via kasha)
...
(cherry picked from commit db73cc9124
)
2014-12-08 22:26:44 -08:00
Harsh J
ee25c0d890
YARN-2891. Failed Container Executor does not provide a clear error message. Contributed by Dustin Cote. (harsh)
...
(cherry picked from commit 4b13318dea7a1cbbbfc1f84207af829cbe2f720e)
2014-12-04 03:19:01 +05:30
Jason Lowe
db723a8499
MAPREDUCE-5932. Provide an option to use a dedicated reduce-side shuffle log. Contributed by Gera Shegalov
...
(cherry picked from commit 03ab24aa01
)
Conflicts:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
2014-12-03 17:12:47 +00:00
Junping Du
86535ff65f
YARN-1156. Enhance NodeManager AllocatedGB and AvailableGB metrics for aggregation of decimal values. (Contributed by Tsuyoshi OZAWA)
...
(cherry picked from commit e65b7c5ff6
)
2014-12-03 04:12:35 -08:00
Karthik Kambatla
af0b54a4ee
YARN-2679. Add metric for container launch duration. (Zhihai Xu via kasha)
...
(cherry picked from commit 233b61e495
)
2014-11-21 14:22:53 -08:00
Jason Lowe
ad140d1fc8
YARN-2816. NM fail to start with NPE during container recovery. Contributed by Zhihai Xu
...
(cherry picked from commit 49c38898b0
)
2014-11-14 21:27:16 +00:00
Jason Lowe
6fd547da8f
YARN-2846. Incorrect persist exit code for running containers in reacquireContainer() that interrupted by NodeManager restart. Contributed by Junping Du
...
(cherry picked from commit 33ea5ae92b
)
2014-11-13 16:17:08 +00:00
Zhijie Shen
8405ec0895
YARN-2794. Fixed log messages about distributing system-credentials. Contributed by Jian He.
...
(cherry picked from commit be7bf956e9
)
2014-11-12 11:09:34 -08:00
Karthik Kambatla
ff1b13ded5
YARN-2236. [YARN-1492] Shared Cache uploader service on the Node Manager. (Chris Trezzo and Sanjin Lee via kasha)
...
(cherry picked from commit a04143039e
)
2014-11-12 09:31:30 -08:00
Ravi Prakash
d863f54f57
YARN-1964. Create Docker analog of the LinuxContainerExecutor in YARN
2014-11-11 21:29:27 -08:00
Karthik Kambatla
f030959849
YARN-2735. diskUtilizationPercentageCutoff and diskUtilizationSpaceCutoff are initialized twice in DirectoryCollection. (Zhihai Xu via kasha)
...
(cherry picked from commit 061bc293c8
)
2014-11-11 10:31:56 -08:00
Xuan
b3badf935a
YARN-2841. RMProxy should retry EOFException. Contributed by Jian He
...
(cherry picked from commit 5c9a51f140
)
2014-11-10 18:26:32 -08:00
Arun C. Murthy
adfb830a2b
Preparing to release hadoop-2.6.0: Set version in branch-2 to 2.7.0-SNAPSHOT.
2014-11-09 19:19:02 -08:00
Arun C. Murthy
175d222bfc
YARN-2830. Add backwords compatible ContainerId.newInstance constructor. Contributed by Jonathan Eagles.
...
(cherry picked from commit 43cd07b408
)
2014-11-09 15:03:59 -08:00
Jason Lowe
a5764cb783
YARN-2825. Container leak on NM. Contributed by Jian He
...
(cherry picked from commit c3d475070a
)
2014-11-07 23:17:34 +00:00
cnauroth
c16f718293
YARN-2803. MR distributed cache not working correctly on Windows after NodeManager privileged account changes. Contributed by Craig Welch.
...
(cherry picked from commit 06b797947c
)
2014-11-07 12:36:42 -08:00
Zhijie Shen
e06c23a6c9
YARN-2752. Made ContainerExecutor append "nice -n" arg only when priority adjustment flag is set. Contributed by Xuan Gong.
2014-11-04 15:50:10 -08:00
Haohui Mai
aa09c4c342
HADOOP-10563. Remove the dependency of jsp in trunk. Contributed by Haohui Mai.
2014-11-04 14:52:51 -08:00
Vinod Kumar Vavilapalli
9c76dcadaf
YARN-1922. Fixed NodeManager to kill process-trees correctly in the presence of races between the launch and the stop-container call and when root processes crash. Contributed by Billie Rinaldi.
...
(cherry picked from commit c5a46d4c8c
)
2014-11-03 16:40:37 -08:00
Vinod Kumar Vavilapalli
715c81ef6d
YARN-2788. Fixed backwards compatiblity issues with log-aggregation feature that were caused when adding log-upload-time via YARN-2703. Contributed by Xuan Gong.
...
(cherry picked from commit 58e9f24e0f
)
2014-11-03 13:19:34 -08:00
Jason Lowe
8091ea54d5
YARN-2730. DefaultContainerExecutor runs only one localizer at a time. Contributed by Siqi Li
...
(cherry picked from commit 6157ace547
)
2014-11-03 20:39:14 +00:00
Vinod Kumar Vavilapalli
6627f67bf5
YARN-2790. Fixed a NodeManager bug that was causing log-aggregation to fail beyond HFDS delegation-token expiry even when RM is a proxy-user (YARN-2704). Contributed by Jian He.
...
(cherry picked from commit 5c0381c96a
)
2014-11-01 16:33:35 -07:00
Zhijie Shen
d9ac25454c
YARN-2711. Fixed TestDefaultContainerExecutor#testContainerLaunchError failure on Windows. Contributed by Varun Vasudev.
...
(cherry picked from commit 1cd088fd9d
)
2014-10-31 17:45:05 -07:00
Xuan
902566741e
YARN-2701. Addendum patch. Potential race condition in startLocalizer when using LinuxContainerExecutor. Contributed by Xuan Gong
2014-10-31 14:39:06 -07:00
Jason Lowe
3e8544c5f2
YARN-2755. NM fails to clean up usercache_DEL_<timestamp> dirs after YARN-661. Contributed by Siqi Li
...
(cherry picked from commit 73e626ad91
)
2014-10-30 15:11:57 +00:00
Zhijie Shen
f40389ae08
YARN-2741. Made NM web UI serve logs on the drive other than C: on Windows. Contributed by Craig Welch.
...
(cherry picked from commit 8984e9b177
)
2014-10-28 14:12:09 -07:00
Vinod Kumar Vavilapalli
0ad33e1483
YARN-2704. Changed ResourceManager to optionally obtain tokens itself for the sake of localization and log-aggregation for long-running services. Contributed by Jian He.
...
(cherry picked from commit a16d022ca4
)
2014-10-27 15:50:51 -07:00
Zhijie Shen
1b81105143
YARN-2703. Added logUploadedTime into LogValue for better display. Contributed by Xuan Gong.
...
(cherry picked from commit f81dc3f995
)
2014-10-24 14:12:17 -07:00
Jian He
1c235a4448
YARN-2198. Remove the need to run NodeManager as privileged account for Windows Secure Container Executor. Contributed by Remus Rusanu
...
(cherry picked from commit 3b12fd6cfb
)
2014-10-22 15:58:26 -07:00
cnauroth
a066134277
YARN-2720. Windows: Wildcard classpath variables not expanded against resources contained in archives. Contributed by Craig Welch.
...
(cherry picked from commit 6637e3cf95
)
2014-10-21 12:34:58 -07:00
Jason Lowe
3820bf055e
YARN-90. NodeManager should identify failed disks becoming good again. Contributed by Varun Vasudev
...
(cherry picked from commit 6f2028bd15
)
2014-10-21 17:33:34 +00:00
Jian He
e9564e729f
Missing file for YARN-2701
...
(cherry picked from commit 4fa1fb3193
)
2014-10-20 19:58:21 -07:00
Jian He
3c8ae89050
YARN-2701. Potential race condition in startLocalizer when using LinuxContainerExecutor. Contributed by Xuan Gong
...
(cherry picked from commit 2839365f23
)
2014-10-20 19:54:10 -07:00
Jian He
a78f1f8da8
YARN-2682. Updated WindowsSecureContainerExecutor to not use DefaultContainerExecutor#getFirstApplicationDir and use getWorkingDir() instead. Contributed by Zhihai Xu
...
(cherry picked from commit 0fd0ebae64
)
2014-10-16 18:15:43 -07:00
Jian He
f93d2ea27e
YARN-2312. Deprecated old ContainerId#getId API and updated MapReduce to use ContainerId#getContainerId instead. Contributed by Tsuyoshi OZAWA
2014-10-15 15:28:26 -07:00
Karthik Kambatla
88455173e8
YARN-2566. DefaultContainerExecutor should pick a working directory randomly. (Zhihai Xu via kasha)
...
(cherry picked from commit cc93e7e683
)
2014-10-13 16:32:42 -07:00
Jason Lowe
6b5741d423
YARN-2377. Localization exception stack traces are not passed as diagnostic info. Contributed by Gera Shegalov
...
(cherry picked from commit a56ea01002
)
2014-10-13 18:32:38 +00:00
Zhijie Shen
e51ae64761
YARN-2651. Spun off LogRollingInterval from LogAggregationContext. Contributed by Xuan Gong.
...
(cherry picked from commit 4aed2d8e91
)
2014-10-13 10:55:09 -07:00
Zhijie Shen
1e6d81a886
YARN-2583. Modified AggregatedLogDeletionService to be able to delete rolling aggregated logs. Contributed by Xuan Gong.
...
(cherry picked from commit cb81bac002
)
2014-10-10 00:16:34 -07:00
cnauroth
b81641a310
YARN-2662. TestCgroupsLCEResourcesHandler leaks file descriptors. Contributed by Chris Nauroth.
...
(cherry picked from commit d3afd730ac
)
2014-10-09 22:47:04 -07:00
Vinod Kumar Vavilapalli
7ed61e150c
YARN-2468. Enhanced NodeManager to support log handling APIs (YARN-2569) for use by long running services. Contributed by Xuan Gong.
...
(cherry picked from commit 34cdcaad71
)
2014-10-03 12:17:03 -07:00
Jason Lowe
531c1fd00a
YARN-2624. Resource Localization fails on a cluster due to existing cache directories. Contributed by Anubhav Dhoot
...
(cherry picked from commit 29f520052e
)
2014-10-02 17:40:44 +00:00
Jian He
61c7ceaf82
YARN-2617. Fixed NM to not send duplicate container status whose app is not running. Contributed by Jun Gong
...
(cherry picked from commit 3ef1cf187f
)
2014-10-02 10:04:42 -07:00
junping_du
6483342a61
YARN-1979. TestDirectoryCollection fails when the umask is unusual. (Contributed by Vinod Kumar Vavilapalli and Tsuyoshi OZAWA)
...
(cherry picked from commit c7cee9b455
)
2014-10-02 08:04:25 -07:00
Vinod Kumar Vavilapalli
3326fba382
YARN-1972. Added a secure container-executor for Windows. Contributed by Remus Rusanu.
...
commit ba7f31c2ee
is the corresponding trunk commit, this is a slightly different patch for branch-2.
2014-10-01 17:07:21 -07:00
junping_du
625456746c
YARN-2613. Support retry in NMClient for rolling-upgrades. (Contributed by Jian He)
2014-10-01 17:08:55 -07:00
Zhijie Shen
4b50e23271
YARN-2630. Prevented previous AM container status from being acquired by the current restarted AM. Contributed by Jian He.
...
(cherry picked from commit 52bbe0f11b
)
2014-10-01 15:39:36 -07:00
Jian He
cb08ed1484
YARN-668. Changed NMTokenIdentifier/AMRMTokenIdentifier/ContainerTokenIdentifier to use protobuf object as the payload. Contributed by Junping Du.
...
(cherry picked from commit 5391919b09
)
2014-09-26 17:53:35 -07:00
Zhijie Shen
3a2e400377
YARN-2581. Passed LogAggregationContext to NM via ContainerTokenIdentifier. Contributed by Xuan Gong.
...
(cherry picked from commit c86674a3a4
)
2014-09-24 17:51:54 -07:00
Allen Wittenauer
e5a531a8a7
YARN-2161. Fix build on macosx: YARN parts (Binglin Chang via aw)
2014-09-24 08:48:42 -07:00
junping_du
34ee084c3d
YARN-2584. TestContainerManagerSecurity fails on trunk. (Contributed by Jian He)
...
(cherry picked from commit a9a55db065
)
2014-09-22 22:47:59 -07:00
Jian He
3ce97a9efd
YARN-1372. Ensure all completed containers are reported to the AMs across RM restart. Contributed by Anubhav Dhoot
...
(cherry picked from commit 0a641496c7
)
2014-09-22 10:32:44 -07:00
Vinod Kumar Vavilapalli
9d34dc87e1
YARN-2531. Added a configuration for admins to be able to override app-configs and enforce/not-enforce strict control of per-container cpu usage. Contributed by Varun Vasudev.
...
(cherry picked from commit 9f6891d9ef
)
2014-09-16 10:15:37 -07:00
cnauroth
aaef460e7a
YARN-2549. TestContainerLaunch fails due to classpath problem with hamcrest classes. Contributed by Chris Nauroth.
...
(cherry picked from commit 9d4ec97c95
)
2014-09-15 11:54:41 -07:00
Vinod Kumar Vavilapalli
a2a61eec6d
YARN-2440. Enabled Nodemanagers to limit the aggregate cpu usage across all containers to a preconfigured limit. Contributed by Varun Vasudev.
...
(cherry picked from commit 4be95175cd
)
2014-09-10 19:24:14 -07:00
Jason Lowe
04d325afff
YARN-2431. NM restart: cgroup is not removed for reacquired containers. Contributed by Jason Lowe
...
(cherry picked from commit 3fa5f728c4
)
2014-09-04 21:14:20 +00:00
Jason Lowe
b61b78e5c6
YARN-2462. TestNodeManagerResync#testBlockNewContainerRequestsOnStartAndResync should have a test timeout. Contributed by Eric Payne
...
(cherry picked from commit 9ecda8f4c7e10d825b884e35c994d241b9fc8907)
2014-08-29 20:18:49 +00:00
Hitesh Shah
aeb8667a06
YARN-2450. Fix typos in log messages. Contributed by Ray Chiang.
...
(cherry picked from commit 3de66011c2
)
2014-08-29 11:18:54 -07:00
Allen Wittenauer
5d965f2f3c
YARN-2424. LCE should support non-cgroups, non-secure mode (Chris Douglas via aw)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1619424 13f79535-47bb-0310-9956-ffa450edef68
2014-08-21 14:57:53 +00:00
Junping Du
e8d20ad77c
Merge r1617448 from trunk: YARN-1337. Recover containers upon nodemanager restart. (Contributed by Jason Lowe)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1617450 13f79535-47bb-0310-9956-ffa450edef68
2014-08-12 11:02:38 +00:00
Junping Du
fc5bb235f2
Merge r1615550 from trunk: YARN-1354. Recover applications upon nodemanager restart. (Contributed by Jason Lowe)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1615554 13f79535-47bb-0310-9956-ffa450edef68
2014-08-04 13:35:49 +00:00
Jian He
2417753952
Merge r1615270 from trunk. YARN-2343. Improve NMToken expire exception message. Contributed by Li Lu
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1615271 13f79535-47bb-0310-9956-ffa450edef68
2014-08-01 23:46:26 +00:00
Xuan Gong
ecc257e384
YARN-1994. Expose YARN/MR endpoints on multiple interfaces. Contributed by Craig Welch, Milan Potocnik and Arpit Agarwal
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1615010 13f79535-47bb-0310-9956-ffa450edef68
2014-07-31 21:29:01 +00:00
Zhijie Shen
f52092be46
YARN-2347. Consolidated RMStateVersion and NMDBSchemaVersion into Version in yarn-server-common. Contributed by Junping Du.
...
svn merge --ignore-ancestry -c 1614838 ../../trunk/
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1614839 13f79535-47bb-0310-9956-ffa450edef68
2014-07-31 09:31:22 +00:00
Aaron Myers
5d0172fdf5
YARN-1796. container-executor shouldn't require o-r permissions. Contributed by Aaron T. Myers.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1613549 13f79535-47bb-0310-9956-ffa450edef68
2014-07-26 01:52:25 +00:00
Devarajulu K
087a2acb8b
YARN-1342. Recover container tokens upon nodemanager restart. Contributed by Jason Lowe.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1612997 13f79535-47bb-0310-9956-ffa450edef68
2014-07-24 05:02:46 +00:00
Junping Du
f6b932fe48
Merget r1612449 from trunk: YARN-2013. The diagnostics is always the ExitCodeException stack when the container crashes. (Contributed by Tsuyoshi OZAWA)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1612450 13f79535-47bb-0310-9956-ffa450edef68
2014-07-22 03:04:22 +00:00
Jason Darrell Lowe
3fc91db99f
svn merge -c 1612411 FIXES: YARN-2321. NodeManager web UI can incorrectly report Pmem enforcement. Contributed by Leitao Guo
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1612414 13f79535-47bb-0310-9956-ffa450edef68
2014-07-21 22:08:12 +00:00
Jason Darrell Lowe
f57b6946d7
svn merge -c 1612285 FIXES: YARN-2045. Data persisted in NM should be versioned. Contributed by Junping Du
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1612289 13f79535-47bb-0310-9956-ffa450edef68
2014-07-21 14:49:38 +00:00
Junping Du
f81b04df50
Merge r1611512 from trunk: YARN-1341. Recover NMTokens upon nodemanager restart. (Contributed by Jason Lowe)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1611514 13f79535-47bb-0310-9956-ffa450edef68
2014-07-17 23:38:36 +00:00
Jian He
77a94b73b2
Merge r1608334 from trunk. YARN-1367. Changed NM to not kill containers on NM resync if RM work-preserving restart is enabled. Contributed by Anubhav Dhoot
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1608336 13f79535-47bb-0310-9956-ffa450edef68
2014-07-07 04:40:36 +00:00
Karthik Kambatla
9133e3142c
Preparing for 2.6.0 development. Updated versions.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1607541 13f79535-47bb-0310-9956-ffa450edef68
2014-07-03 06:11:57 +00:00
Steve Loughran
4cfc90590c
YARN-2065 AM cannot create new containers after restart
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1607440 13f79535-47bb-0310-9956-ffa450edef68
2014-07-02 18:34:09 +00:00
Karthik Kambatla
14858cd6f7
YARN-2204. Explicitly enable vmem check in TestContainersMonitor#testContainerKillOnMemoryOverflow. (Anubhav Dhoot via kasha)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1607233 13f79535-47bb-0310-9956-ffa450edef68
2014-07-02 02:07:48 +00:00
Vinod Kumar Vavilapalli
a2e2c8ad97
YARN-2152. Added missing information into ContainerTokenIdentifier so that NodeManagers can report the same to RM when RM restarts. Contributed Jian He.
...
svn merge --ignore-ancestry -c 1605205 ../../trunk/
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1605206 13f79535-47bb-0310-9956-ffa450edef68
2014-06-24 21:44:00 +00:00
Thomas Graves
1c2052e200
YARN-2072. RM/NM UIs and webservices are missing vcore information. (Nathan Roberts via tgraves)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1605166 13f79535-47bb-0310-9956-ffa450edef68
2014-06-24 19:41:56 +00:00
Jason Darrell Lowe
0873304b90
svn merge -c 160639 FIXES: YARN-2167. LeveldbIterator should get closed in NMLeveldbStateStoreService#loadLocalizationState() within finally block. Contributed by Junping Du
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1603040 13f79535-47bb-0310-9956-ffa450edef68
2014-06-17 02:14:29 +00:00
Junping Du
771e157b66
Merge r1603036 from trunk: YARN-1339. Recover DeletionService state upon nodemanager restart. (Contributed by Jason Lowe)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1603037 13f79535-47bb-0310-9956-ffa450edef68
2014-06-17 01:10:49 +00:00
Jian He
d79919f175
Merge r1603028 from trunk. YARN-1885. Fixed a bug that RM may not send application-clean-up signal to NMs where the completed applications previously ran in case of RM restart. Contributed by Wangda Tan
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1603030 13f79535-47bb-0310-9956-ffa450edef68
2014-06-17 00:02:57 +00:00
Bikas Saha
dc5ee5ff7c
Merge 1601762 from trunk to branch-2 for YARN-2091. Add more values to ContainerExitStatus and pass it from NM to RM and then to app masters (Tsuyoshi OZAWA via bikas)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1601763 13f79535-47bb-0310-9956-ffa450edef68
2014-06-10 20:13:23 +00:00
Vinod Kumar Vavilapalli
a73447fa07
YARN-2115. Replaced RegisterNodeManagerRequest's ContainerStatus with a new NMContainerStatus which has more information that is needed for work-preserving RM-restart. Contributed by Jian He.
...
svn merge --ignore-ancestry -c 1598790 ../../trunk/
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1598791 13f79535-47bb-0310-9956-ffa450edef68
2014-05-31 00:21:47 +00:00
Junping Du
9f76296358
Merge r1598640 from trunk: YARN-1338. Recover localized resource cache state upon nodemanager restart (Contributed by Jason Lowe)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1598652 13f79535-47bb-0310-9956-ffa450edef68
2014-05-30 16:09:36 +00:00
Junping Du
ffb0d24fef
Merge r1594421 from trunk: YARN-1362. Distinguish between nodemanager shutdown for decommission vs shutdown for restart. (Contributed by Jason Lowe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1594422 13f79535-47bb-0310-9956-ffa450edef68
2014-05-14 00:25:08 +00:00
Junping Du
4b27c6882a
Merge r1593660 from trunk: YARN-766. TestNodeManagerShutdown in branch-2 should use Shell to form the output path and a format issue in trunk. (Contributed by Siddharth Seth)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1593661 13f79535-47bb-0310-9956-ffa450edef68
2014-05-10 03:47:43 +00:00
Ivan Mitic
2fb649a668
YARN-1865 Merging change r1588693 from trunk.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1588696 13f79535-47bb-0310-9956-ffa450edef68
2014-04-19 19:00:42 +00:00
Jason Darrell Lowe
ccec38b801
svn merge -c 1588546 FIXES: YARN-1940. deleteAsUser() terminates early without deleting more files on error. Contributed by Rushabh S Shah
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1588549 13f79535-47bb-0310-9956-ffa450edef68
2014-04-18 19:36:18 +00:00
Junping Du
8c23c3295a
Merge r1588343 from trunk: YARN-1750. TestNodeStatusUpdater#testNMRegistration is incorrect in test case. (Wangda Tan via junping_du)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1588347 13f79535-47bb-0310-9956-ffa450edef68
2014-04-17 19:13:17 +00:00
Zhijie Shen
7c7d00d779
YARN-1892. Improved some logs in the scheduler. Contributed by Jian He.
...
svn merge --ignore-ancestry -c 1587717 ../../trunk/
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1587718 13f79535-47bb-0310-9956-ffa450edef68
2014-04-15 20:40:53 +00:00
Vinod Kumar Vavilapalli
2595a27092
YARN-1933. Fixed test issues with TestAMRestart and TestNodeHealthService. Contributed by Jian He.
...
svn merge --ignore-ancestry -c 1587104 ../../trunk/
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1587105 13f79535-47bb-0310-9956-ffa450edef68
2014-04-13 21:52:09 +00:00
Jian He
9df6ddd282
Merge r1586522 from trunk. YARN-1903. Set exit code and diagnostics when container is killed at NEW/LOCALIZING state. Contributed by Zhijie Shen
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1586523 13f79535-47bb-0310-9956-ffa450edef68
2014-04-11 01:28:22 +00:00
Karthik Kambatla
a2cdf208dd
YARN-1757. NM Recovery. Auxiliary service support. (Jason Lowe via kasha)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1585784 13f79535-47bb-0310-9956-ffa450edef68
2014-04-08 17:17:59 +00:00
Vinod Kumar Vavilapalli
08a194fb55
YARN-1775. Enhanced ProcfsBasedProcessTree to optionally add the ability to use smaps for obtaining used memory information. Contributed by Rajesh Balamohan.
...
svn merge --ignore-ancestry -c 1580087 ../../trunk/
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1580088 13f79535-47bb-0310-9956-ffa450edef68
2014-03-22 00:02:26 +00:00
Jian He
69835b9651
Merge r1578614 from trunk. Fixed AM container log to show on NM web page after application finishes if log-aggregation is disabled. Contributed by Rohith Sharmaks
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1578618 13f79535-47bb-0310-9956-ffa450edef68
2014-03-17 21:52:41 +00:00
Jonathan Turner Eagles
8e38068076
YARN-1136. Replace junit.framework.Assert with org.junit.Assert (Chen He via jeagles)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1578546 13f79535-47bb-0310-9956-ffa450edef68
2014-03-17 20:20:22 +00:00
Vinod Kumar Vavilapalli
d470c7b71a
YARN-1824. Improved NodeManager and clients to be able to handle cross platform application submissions. Contributed by Jian He.
...
MAPREDUCE-4052. Improved MapReduce clients to use NodeManagers' ability to handle cross platform application submissions. Contributed by Jian He.
svn merge --ignore-ancestry -c 1578135 ../../trunk/ with a couple of minor edits for working in branch-2.
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1578139 13f79535-47bb-0310-9956-ffa450edef68
2014-03-16 19:13:16 +00:00
Christopher Douglas
9695bc7af6
YARN-1771. Reduce the number of NameNode operations during localization of
...
public resources using a cache. Contributed by Sangjin Lee
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1577392 13f79535-47bb-0310-9956-ffa450edef68
2014-03-14 00:31:08 +00:00
Vinod Kumar Vavilapalli
d5120ccc6b
YARN-1800. Fixed NodeManager to gracefully handle RejectedExecutionException in the public-localizer thread-pool. Contributed by Varun Vasudev.
...
svn merge --ignore-ancestry -c 1576545 ../../trunk/
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1576546 13f79535-47bb-0310-9956-ffa450edef68
2014-03-11 23:34:20 +00:00
Vinod Kumar Vavilapalli
2fbec50fed
YARN-1781. Modified NodeManagers to allow admins to specify max disk utilization for local disks so as to be able to offline full disks. Contributed by Varun Vasudev.
...
svn merge --ignore-ancestry -c 1575463 ../../trunk/
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1575464 13f79535-47bb-0310-9956-ffa450edef68
2014-03-08 00:52:32 +00:00
Vinod Kumar Vavilapalli
78f1a475c8
YARN-1783. Fixed a bug in NodeManager's status-updater that was losing completed container statuses when NodeManager is forced to resync by the ResourceManager. Contributed by Jian He.
...
svn merge --ignore-ancestry -c 1575437 ../../trunk/
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1575438 13f79535-47bb-0310-9956-ffa450edef68
2014-03-07 22:37:12 +00:00
Vinod Kumar Vavilapalli
ae456f408a
YARN-1686. Fixed NodeManager to properly handle any errors during re-registration after a RESYNC and thus avoid hanging. Contributed by Rohith Sharma.
...
svn merge --ignore-ancestry -c 1571474 ../../trunk/
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1571475 13f79535-47bb-0310-9956-ffa450edef68
2014-02-24 22:42:00 +00:00
Arun Murthy
1ccccb8534
Branching for hadoop-2.4 releases, also set version to 2.5.0-SNAPSHOT in branch-2.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1569186 13f79535-47bb-0310-9956-ffa450edef68
2014-02-18 04:41:01 +00:00
Vinod Kumar Vavilapalli
7e252cdc9d
YARN-1553. Modified YARN and MR to stop using HttpConfig.isSecure() and
...
instead rely on the http policy framework. And also fix some bugs related
to https handling in YARN web-apps. Contributed by Haohui Mai.
svn merge --ignore-ancestry -c 1568501 ../../trunk/
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1568503 13f79535-47bb-0310-9956-ffa450edef68
2014-02-14 20:01:32 +00:00
Sanford Ryza
5bc592d88d
YARN-1697. NodeManager reports negative running containers (Sandy Ryza)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1567381 13f79535-47bb-0310-9956-ffa450edef68
2014-02-11 20:50:22 +00:00
Karthik Kambatla
49389403d6
YARN-1672. YarnConfiguration is missing a default for yarn.nodemanager.log.retain-seconds (Naren Koneru via kasha)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1565867 13f79535-47bb-0310-9956-ffa450edef68
2014-02-08 01:56:10 +00:00
Jason Darrell Lowe
4fe4190b1d
svn merge -c 1561110 FIXES: YARN-1575. Public localizer crashes with "Localized unkown resource". Contributed by Jason Lowe
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1561112 13f79535-47bb-0310-9956-ffa450edef68
2014-01-24 18:59:54 +00:00
Jason Darrell Lowe
0fa22b8120
svn merge -c 1558948 FIXES: MAPREDUCE-5672. Provide optional RollingFileAppender for container log4j (syslog). Contributed by Gera Shegalov
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1558957 13f79535-47bb-0310-9956-ffa450edef68
2014-01-16 23:13:17 +00:00
Karthik Kambatla
7eb1ca53b3
YARN-1608. LinuxContainerExecutor has a few DEBUG messages at INFO level (kasha)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1558875 13f79535-47bb-0310-9956-ffa450edef68
2014-01-16 18:44:35 +00:00
Vinod Kumar Vavilapalli
f35f702426
YARN-888. Cleaned up POM files so that non-leaf modules don't include any dependencies and thus compact the dependency list for leaf modules. Contributed by Alejandro Abdelnur.
...
svn merge --ignore-ancestry -c 1557801 ../../trunk/
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1557802 13f79535-47bb-0310-9956-ffa450edef68
2014-01-13 18:34:39 +00:00
Jian He
5c47b8d78a
Merge r1556318 from trunk. YARN-1293. Fixed TestContainerLaunch#testInvalidEnvSyntaxDiagnostics failure caused by non-English system locale. Contributed by Tsuyoshi OZAWA.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1556319 13f79535-47bb-0310-9956-ffa450edef68
2014-01-07 19:03:50 +00:00
Jason Darrell Lowe
b8f59ebeaa
svn merge -c 1556282 FIXES: YARN-1409. NonAggregatingLogHandler can throw RejectedExecutionException. Contributed by Tsuyoshi OZAWA
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1556284 13f79535-47bb-0310-9956-ffa450edef68
2014-01-07 17:23:50 +00:00
Jing Zhao
88d09130e8
HDFS-5545. Merge change r1546151 from trunk.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1551703 13f79535-47bb-0310-9956-ffa450edef68
2013-12-17 20:59:27 +00:00
Vinod Kumar Vavilapalli
f7f79f93ab
YARN-1145. Fixed a potential file-handle leak in the web interface for displaying aggregated logs. Contributed by Rohith Sharma.
...
svn merge --ignore-ancestry -c 1551326 ../../trunk/
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1551327 13f79535-47bb-0310-9956-ffa450edef68
2013-12-16 19:28:20 +00:00
Arun Murthy
2c217482e8
Merge -c 1548336 from trunk to branch-2 to fix YARN-1438. Ensure container diagnostics includes exception from container launch. Contributed by Steve Loughran.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1548337 13f79535-47bb-0310-9956-ffa450edef68
2013-12-06 00:26:03 +00:00
Arun Murthy
439a690ad1
Setting version to 2.4.0-SNAPSHOT in branch-2.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1547165 13f79535-47bb-0310-9956-ffa450edef68
2013-12-02 19:40:42 +00:00
Bikas Saha
eb97badbcd
Merge r1543973 from trunk to branch-2 for YARN-1053. Diagnostic message from ContainerExitEvent is ignored in ContainerImpl (Omkar Vinit Joshi via bikas)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1543974 13f79535-47bb-0310-9956-ffa450edef68
2013-11-20 22:29:16 +00:00
Vinod Kumar Vavilapalli
8613e7ef3d
YARN-1210. Changed RM to start new app-attempts on RM restart only after ensuring that previous AM exited or after expiry time. Contributed by Omkar Vinit Joshi.
...
svn merge --ignore-ancestry -c 1543310 ../../trunk/
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1543311 13f79535-47bb-0310-9956-ffa450edef68
2013-11-19 05:17:56 +00:00
Sanford Ryza
cfcb4a716f
YARN-1401. With zero sleep-delay-before-sigkill.ms, no signal is ever sent (Gera Shegalov via Sandy Ryza)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1542047 13f79535-47bb-0310-9956-ffa450edef68
2013-11-14 19:59:05 +00:00
Jonathan Turner Eagles
1884c3a0b9
YARN-1386. NodeManager mistakenly loses resources and relocalizes them (Jason Lowe via jeagles)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1541376 13f79535-47bb-0310-9956-ffa450edef68
2013-11-13 03:26:20 +00:00
Chris Nauroth
f6bb3ce621
YARN-1357. Merging change r1537293 from trunk to branch-2.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1537296 13f79535-47bb-0310-9956-ffa450edef68
2013-10-30 20:53:16 +00:00
Sanford Ryza
67365f3172
YARN-1109. Demote NodeManager "Sending out status for container" logs to debug (haosdent via Sandy Ryza)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1536574 13f79535-47bb-0310-9956-ffa450edef68
2013-10-29 00:57:03 +00:00
Sanford Ryza
3b0a6e6edd
YARN-1295. In UnixLocalWrapperScriptBuilder, using bash -c can cause Text file busy errors. (Sandy Ryza)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1532533 13f79535-47bb-0310-9956-ffa450edef68
2013-10-15 21:15:38 +00:00
Sanford Ryza
86bacb6b43
Add missing file TestCgroupsLCEResourcesHandler for YARN-1284.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1530495 13f79535-47bb-0310-9956-ffa450edef68
2013-10-09 05:12:26 +00:00
Sanford Ryza
b57b4252bf
YARN-1284. LCE: Race condition leaves dangling cgroups entries for killed containers. (Alejandro Abdelnur via Sandy Ryza)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1530494 13f79535-47bb-0310-9956-ffa450edef68
2013-10-09 05:08:17 +00:00
Vinod Kumar Vavilapalli
4b32d6ad9a
YARN-1277. Added a policy based configuration for http/https in common HttpServer and using the same in YARN - related
...
to per project https config support via HADOOP-10022. Contributed by Suresh Srinivas and Omkar Vinit Joshi.
svn merge --ignore-ancestry -c 1529662 ../../trunk/
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1529663 13f79535-47bb-0310-9956-ffa450edef68
2013-10-06 18:44:27 +00:00
Vinod Kumar Vavilapalli
763521305d
YARN-1278. Fixed NodeManager to not delete local resources for apps on resync command from RM - a bug caused by YARN-1149. Contributed by Hitesh Shah.
...
svn merge --ignore-ancestry -c 1529657 ../../trunk/
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1529658 13f79535-47bb-0310-9956-ffa450edef68
2013-10-06 18:33:37 +00:00
Vinod Kumar Vavilapalli
85bed752d5
YARN-1274. Fixed NodeManager's LinuxContainerExecutor to create user, app-dir and log-dirs correctly even when there are no resources to localize for the container. Contributed by Siddharth Seth.
...
svn merge --ignore-ancestry -c 1529555 ../../trunk/
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1529556 13f79535-47bb-0310-9956-ffa450edef68
2013-10-06 00:15:51 +00:00
Vinod Kumar Vavilapalli
c91addb743
YARN-1271. Addendum patch to remove unused method in ContainerExecutor. Contributed by Sandy Ryza.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1529386 13f79535-47bb-0310-9956-ffa450edef68
2013-10-05 05:14:05 +00:00
Vinod Kumar Vavilapalli
ea3c2d28f7
YARN-1254. Fixed NodeManager to not pollute container's credentials. Contributed by Omkar Vinit Joshi.
...
svn merge --ignore-ancestry -c 1529382 ../../trunk/
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1529383 13f79535-47bb-0310-9956-ffa450edef68
2013-10-05 04:26:37 +00:00
Alejandro Abdelnur
4de1616b82
YARN-1253. Changes to LinuxContainerExecutor to run containers as a single dedicated user in non-secure mode. (rvs via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1529326 13f79535-47bb-0310-9956-ffa450edef68
2013-10-04 22:00:01 +00:00
Sanford Ryza
c03521fd86
YARN-1271. Text file busy errors launching containers again (Sandy Ryza)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1529060 13f79535-47bb-0310-9956-ffa450edef68
2013-10-04 01:31:26 +00:00
Bikas Saha
7b2cd4b411
Merge r1529039 and r1529048 from trunk to branch-2 for YARN-1256. NM silently ignores non-existent service in StartContainerRequest (Xuan Gong via bikas)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1529049 13f79535-47bb-0310-9956-ffa450edef68
2013-10-04 01:04:26 +00:00
Hitesh Shah
75aeb6070a
Merge 1529043 from trunk to branch-2 for YARN-1149. NM throws InvalidStateTransitonException: Invalid event: APPLICATION_LOG_HANDLING_FINISHED at RUNNING. Contributed by Xuan Gong.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1529044 13f79535-47bb-0310-9956-ffa450edef68
2013-10-04 00:46:18 +00:00
Vinod Kumar Vavilapalli
80dd306450
YARN-1070. Fixed race conditions in NodeManager during container-kill. Contributed by Zhijie Shen.
...
svn merge --ignore-ancestry -c 1527827 ../../trunk/
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1527828 13f79535-47bb-0310-9956-ffa450edef68
2013-10-01 00:18:42 +00:00
Alejandro Abdelnur
15ee08ef6c
YARN-1247. test-container-executor has gotten out of sync with the changes to container-executor. (rvs via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1527814 13f79535-47bb-0310-9956-ffa450edef68
2013-09-30 22:55:51 +00:00
Jonathan Turner Eagles
e33f5dc85d
YARN-819. ResourceManager and NodeManager should check for a minimum allowed version (Robert Parker via jeagles)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1526665 13f79535-47bb-0310-9956-ffa450edef68
2013-09-26 20:06:47 +00:00
Siddharth Seth
e23d5672a7
merge YARN-1229 from trunk. Define constraints on Auxiliary Service names. Change ShuffleHandler service name from mapreduce.shuffle to mapreduce_shuffle. Contributed by Xuan Gong.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1526066 13f79535-47bb-0310-9956-ffa450edef68
2013-09-25 00:37:14 +00:00
Vinod Kumar Vavilapalli
2daddb610e
YARN-1204. Added separate configuration properties for https for RM and NM without which servers enabled with https will also start on http ports. Contributed by Omkar Vinit Joshi.
...
MAPREDUCE-5523. Added separate configuration properties for https for JHS without which even when https is enabled, it starts on http port itself. Contributed by Omkar Vinit Joshi.
svn merge --ignore-ancestry -c 1525947 ../../trunk/
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1525948 13f79535-47bb-0310-9956-ffa450edef68
2013-09-24 17:20:48 +00:00
Arun Murthy
c061a0da55
Merge -c 1523652 from trunk to branch-2 to fix YARN-1170. YARN & MapReduce proto definitions fixed to specify protobuf package as hadoop.yarn and hadoop.mapreduce respectively. Contributed by Binglin Chang.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1523653 13f79535-47bb-0310-9956-ffa450edef68
2013-09-16 14:06:27 +00:00
Alejandro Abdelnur
460fd2053b
YARN-1137. Add support whitelist for system users to Yarn container-executor.c. (rvs via tucu)
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1523589 13f79535-47bb-0310-9956-ffa450edef68
2013-09-16 11:02:13 +00:00
Jason Darrell Lowe
d5b84644f0
svn merge -c 1523158 FIXES: YARN-1189. NMTokenSecretManagerInNM is not being told when applications have finished. Contributed by Omkar Vinit Joshi
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1523159 13f79535-47bb-0310-9956-ffa450edef68
2013-09-14 00:23:20 +00:00
Jason Darrell Lowe
039c10e0d9
svn merge -c 1522968 FIXES: YARN-1194. TestContainerLogsPage fails with native builds. Contributed by Roman Shaposhnik
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1522971 13f79535-47bb-0310-9956-ffa450edef68
2013-09-13 15:20:38 +00:00
Chris Nauroth
399df2cb08
YARN-1078. Merging change r1522644 from trunk to branch-2.
...
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/branch-2@1522645 13f79535-47bb-0310-9956-ffa450edef68
2013-09-12 16:04:41 +00:00