Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,913 Commits 326 Branches 371 Tags
Commit Graph

543 Commits

Author SHA1 Message Date
Jian He
7c3895c8f6 YARN-3654. ContainerLogsPage web UI should not have meta-refresh. Contributed by Xuan Gong 
(cherry picked from commit 6329bd00fa1f17cc9555efa496ea7607ad93e0ce)
 2015-05-20 17:21:19 -07:00
Wangda Tan
7b97f049ee YARN-3565. NodeHeartbeatRequest/RegisterNodeManagerRequest should use NodeLabel object instead of String. (Naganarasimha G R via wangda) 
(cherry picked from commit b37da52a1c4fb3da2bd21bfadc5ec61c5f953a59)
 2015-05-19 16:35:23 -07:00
Colin Patrick Mccabe
9534e533dc HADOOP-11970. Replace uses of ThreadLocal<Random> with JDK7 ThreadLocalRandom (Sean Busbey via Colin P. McCabe) 
(cherry picked from commit 470c87dbc6c24dd3b370f1ad9e7ab1f6dabd2080)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicy.java
        hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/JspHelper.java
 2015-05-19 11:15:04 -07:00
Ravi Prakash
084e453629 YARN-3302. TestDockerContainerExecutor should run automatically if it can detect docker in the usual place (Ravindra Kumar Naik via raviprak) 
(cherry picked from commit c97f32e7b9d9e1d4c80682cc01741579166174d1)
 2015-05-19 10:30:24 -07:00
Junping Du
cc671349bc YARN-3505 addendum: fix an issue in previous patch. 
(cherry picked from commit 03a293aed6de101b0cae1a294f506903addcaa75)
 2015-05-15 06:40:28 -07:00
Ravi Prakash
4caadaa8bb YARN-1519. Check in container-executor if sysconf is implemented before using it (Radim Kolar and Eric Payne via raviprak) 
(cherry picked from commit 53fe4eff09fdaeed75a8cad3a26156bf963a8d37)
 2015-05-14 15:56:36 -07:00
Junping Du
bc13c7d84b YARN-3505. Node's Log Aggregation Report with SUCCEED should not cached in RMApps. Contributed by Xuan Gong. 
(cherry picked from commit 15ccd967ee3e7046a50522089f67ba01f36ec76a)
 2015-05-14 10:59:48 -07:00
Jason Lowe
a81ad81461 YARN-3641. NodeManager: stopRecoveryStore() shouldn't be skipped when exceptions happen in stopping NM's sub-services. Contributed by Junping Du 
(cherry picked from commit 711d77cc54a64b2c3db70bdacc6bf2245c896a4b)
 2015-05-13 21:08:45 +00:00
Xuan
487d9b0f3f YARN-3626. On Windows localized resources are not moved to the front of the classpath when they should be. Contributed by Craig Welch 
(cherry picked from commit 0f95921447ea547bdf9caf18f7fde46bc66031f8)
 2015-05-13 13:12:04 -07:00
Devaraj K
5fceebd696 YARN-3629. NodeID is always printed as "null" in node manager 
initialization log. Contributed by nijel.

(cherry picked from commit 5c2f05cd9bad9bf9beb0f4ca18f4ae1bc3e84499)
 2015-05-12 22:22:40 +05:30
Devaraj K
f887243f88 YARN-3513. Remove unused variables in ContainersMonitorImpl and add debug 
log for overall resource usage by all containers.  Contributed by
Naganarasimha G R.

(cherry picked from commit 8badd82ce256e4dc8c234961120d62a88358ab39)
 2015-05-12 16:56:26 +05:30
Karthik Kambatla
e8dd0bc4fd YARN-1287. Consolidate MockClocks. (Sebastian Wong and Anubhav Dhoot via kasha) 
(cherry picked from commit 70fb37cd79a7eba6818313960624380bacfe0bb2)
 2015-05-09 14:35:13 -07:00
Xuan
acb342b77c YARN-1912. ResourceLocalizer started without any jvm memory control. 
Contributed by Masatake Iwasaki

(cherry picked from commit 6471d18bc72bc6c83ce31a03b5c5f5737847bb6d)
 2015-05-08 20:02:23 -07:00
Xuan
fc980247a6 YARN-3602. TestResourceLocalizationService.testPublicResourceInitializesLocalDir fails Intermittently due to IOException from cleanup. Contributed by zhihai xu 
(cherry picked from commit 333f9a896d8a4407ce69cfd0dc8314587a339233)
 2015-05-08 17:54:46 -07:00
Jason Lowe
743ca1a056 YARN-3476. Nodemanager can fail to delete local logs if log aggregation fails. Contributed by Rohith 
(cherry picked from commit 25e2b02122c4ed760227ab33c49d3445c23b9276)
 2015-05-08 22:46:34 +00:00
Xuan
8d07116266 YARN-2331. Distinguish shutdown during supervision vs. shutdown for 
rolling upgrade. Contributed by Jason Lowe

(cherry picked from commit 088156de43abb07bec590a3fcd1a5af2feb02cd2)
 2015-05-08 15:11:55 -07:00
Robert (Bobby) Evans
28e0593b96 YARN-644: Basic null check is not performed on passed in arguments before using them in ContainerManagerImpl.startContainer 
(cherry picked from commit bcf2890502fbd11dd394048fe30d67c92aeec4fa)
 2015-05-08 11:11:53 -05:00
Devaraj K
bf8be56292 YARN-2784. Make POM project names consistent. Contributed by Rohith. 2015-05-08 19:26:11 +05:30
Akira Ajisaka
170f1b0afd YARN-1832. Fix wrong MockLocalizerStatus#equals implementation. Contributed by Hong Zhiguo. 
(cherry picked from commit b167fe7605deb29ec533047d79d036eb65328853)
 2015-05-08 11:15:28 +09:00
Akira Ajisaka
67bebabb21 YARN-3577. Misspelling of threshold in log4j.properties for tests. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 995f1b5831fc33042de90267a689333dc110c4c3)
 2015-05-07 13:33:17 +09:00
Robert Kanter
28b40aca98 YARN-3491. PublicLocalizer#addResource is too slow. (zxu via rkanter) 
(cherry picked from commit b72507810aece08e17ab4b5aae1f7eae1fe98609)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-05-06 14:20:08 -07:00
Junping Du
6ea007ba91 YARN-3396. Handle URISyntaxException in ResourceLocalizationService. (Contributed by Brahma Reddy Battula) 
(cherry picked from commit 38102420621308f5ba91cdeb6a18a63aa5acf640)
 2015-05-05 10:19:16 -07:00
Wangda Tan
4e77ee2f6f YARN-3375. NodeHealthScriptRunner.shouldRun() check is performing 3 times for starting NodeHealthScriptRunner (Devaraj K via wangda) 
(cherry picked from commit 71f4de220c74bf2c90630bd0442979d92380d304)
 2015-05-04 16:02:57 -07:00
Allen Wittenauer
02e650248d YARN-2980. Move health check script related functionality to hadoop-common (Varun Saxena via aw) 
(cherry picked from commit d4ac6822e1c5dfac504ced48f10ab57a55b49e93)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
 2015-05-04 16:00:20 -07:00
Jason Lowe
99b49a8181 YARN-3097. Logging of resource recovery on NM restart has redundancies. Contributed by Eric Payne 
(cherry picked from commit 8f65c793f2930bfd16885a2ab188a9970b754974)
 2015-05-04 10:32:33 -05:00
Robert Kanter
32b3b8e2ef YARN-3363. add localization and container launch time to ContainerMetrics at NM to show these timing information for each active container. (zxu via rkanter) 
(cherry picked from commit ac7d152901e29b1f444507fe4e421eb6e1402b5a)
 2015-05-01 16:39:59 -07:00
Vinod Kumar Vavilapalli
90f6e8c90f YARN-2619. Added NodeManager support for disk io isolation through cgroups. Contributed by Varun Vasudev and Wei Yan. 
(cherry picked from commit 1b3b9e5c31c38388c1ce4208c65e8dd5f956da82)
 2015-04-30 21:43:22 -07:00
Karthik Kambatla
4045c41afe YARN-3464. Race condition in LocalizerRunner kills localizer before localizing all resources. (Zhihai Xu via kasha) 
(cherry picked from commit 47279c3228185548ed09c36579b420225e4894f5)
 2015-04-26 09:14:12 -07:00
Jason Lowe
cf4154676b YARN-3537. NPE when NodeManager.serviceInit fails and stopRecoveryStore invoked. Contributed by Brahma Reddy Battula 
(cherry picked from commit 5e093f0d400f82f67d9b2d24253c79e4a5abacf9)
 2015-04-24 22:04:03 +00:00
Xuan
8f6053ae51 YARN-3516. killing ContainerLocalizer action doesn't take effect when 
private localizer receives FETCH_FAILURE status. Contributed by zhihai
xu

(cherry picked from commit 0b3f8957a87ada1a275c9904b211fdbdcefafb02)
 2015-04-23 16:41:52 -07:00
Vinod Kumar Vavilapalli
04783b0402 YARN-3366. Enhanced NodeManager to support classifying/shaping outgoing network bandwidth traffic originating from YARN containers Contributed by Sidharta Seethana. 
(cherry picked from commit a100be685cc4521e9949589948219231aa5d2733)
 2015-04-22 17:36:55 -07:00
Jian He
613a783380 YARN-3503. Expose disk utilization percentage and bad local and log dir counts in NM metrics. Contributed by Varun Vasudev 
(cherry picked from commit 674c7ef64916fabbe59c8d6cdd50ca19cf7ddb7c)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
 2015-04-21 21:06:06 -07:00
Junping Du
b244701fad YARN-1402. Update related Web UI and CLI with exposing client API to check log aggregation status. Contributed by Xuan Gong. 
(cherry picked from commit 1db355a875c3ecc40a244045c6812e00c8d36ef1)
 2015-04-17 13:20:27 -07:00
Jian He
6fed2c2a79 YARN-3354. Add node label expression in ContainerTokenIdentifier to support RM recovery. Contributed by Wangda Tan 
(cherry picked from commit 1b89a3e173f8e905074ed6714a7be5c003c0e2c4)
 2015-04-15 14:03:29 -07:00
Harsh J
01af29106a YARN-3462. Patches applied for YARN-2424 are inconsistent between trunk and branch-2. Contributed by Naganarasimha G R. 2015-04-15 09:41:43 +05:30
Junping Du
7c072bf092 YARN-3443. Create a 'ResourceHandler' subsystem to ease addition of support for new resource types on the NM. Contributed by Sidharta Seethana. 
(cherry picked from commit 838b06ac87339494cea706d4a97e7f5383bdc442)
 2015-04-13 18:37:39 -07:00
Junping Du
0891de0712 YARN-1376. NM need to notify the log aggregation status to RM through Node heartbeat. Contributed by Xuan Gong. 
(cherry picked from commit 92431c961741747b5d6442f4025016d48d9a6863)
 2015-04-10 08:58:07 -07:00
Karthik Kambatla
53e0bf5c17 YARN-3465. Use LinkedHashMap to preserve order of resource requests. (Zhihai Xu via kasha) 
(cherry picked from commit 6495940eae09418a939882a8955845f9241a6485)
 2015-04-09 00:10:23 -07:00
Tsuyoshi Ozawa
ac32fa187c YARN-3457. NPE when NodeManager.serviceInit fails and stopRecoveryStore called. Contributed by Bibin A Chundatt. 
(cherry picked from commit dd852f5b8c8fe9e52d15987605f36b5b60f02701)
 2015-04-08 15:56:33 +09:00
Wangda Tan
405aead879 YARN-2901. Add errors and warning metrics page to RM, NM web UI. (Varun Vasudev via wangda) 
(cherry picked from commit bad070fe15a642cc6f3a165612fbd272187e03cb)
 2015-04-02 17:23:44 -07:00
Vinod Kumar Vavilapalli
d8e17c58bc YARN-3365. Enhanced NodeManager to support using the 'tc' tool via container-executor for outbound network traffic control. Contributed by Sidharta Seethana. 
(cherry picked from commit b21c72777ae664b08fd1a93b4f88fa43f2478d94)
 2015-04-02 16:55:00 -07:00
Tsuyoshi Ozawa
80755edb7a YARN-3424. Change logs for ContainerMonitorImpl's resourse monitoring from info to debug. Contributed by Anubhav Dhoot. 
(cherry picked from commit c69ba81497ae4da329ddb34ba712a64a7eec479f)
 2015-04-01 17:44:40 +09:00
Karthik Kambatla
93b29d962d YARN-3428. Debug log resources to be localized for a container. (kasha) 
(cherry picked from commit 2daa478a6420585dc13cea2111580ed5fe347bc1)
 2015-03-31 17:35:11 -07:00
Wangda Tan
cba4ed1678 YARN-2495. Allow admin specify labels from each NM (Distributed configuration for node label). (Naganarasimha G R via wangda) 
(cherry picked from commit 2a945d24f7de1a7ae6e7bd6636188ce3b55c7f52)
 2015-03-30 12:05:54 -07:00
Vinod Kumar Vavilapalli
c5bc48946d YARN-3304. Cleaning up ResourceCalculatorProcessTree APIs for public use and removing inconsistencies in the default values. Contributed by Junping Du and Karthik Kambatla. 
(cherry picked from commit c358368f511963ad8e35f030b9babee541e1bd01)
 2015-03-30 10:11:12 -07:00
Ravi Prakash
b1b4951452 YARN-3288. Document and fix indentation in the DockerContainerExecutor code 
(cherry picked from commit e0ccea33c9e12f6930b2867e14b1b37569fed659)
 2015-03-28 08:01:26 -07:00
Tsuyoshi Ozawa
cbacf20755 YARN-3384. TestLogAggregationService.verifyContainerLogs fails after YARN-2777. Contributed by Naganarasimha G R. 
(cherry picked from commit 82eda771e05cf2b31788ee1582551e65f1c0f9aa)
 2015-03-24 00:25:52 +09:00
Junping Du
f40f17489c YARN-3269. Yarn.nodemanager.remote-app-log-dir could not be configured to fully qualified path. Contributed by Xuan Gong 
(cherry picked from commit d81109e588493cef31e68508a3d671203bd23e12)
 2015-03-20 13:42:31 -07:00
Karthik Kambatla
05627dd96d YARN-3351. AppMaster tracking URL is broken in HA. (Anubhav Dhoot via kasha) 
(cherry picked from commit 20b49224eb90c796f042ac4251508f3979fd4787)
 2015-03-18 16:33:57 -07:00
Ravi Prakash
9f227ad696 YARN-3339. TestDockerContainerExecutor should pull a single image and not the entire centos repository. (Ravindra Kumar Naik via raviprak) 
(cherry picked from commit 56085203c43b8f2561bf3745910e03f8ac176a67)
 2015-03-16 16:18:42 -07:00