Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
7,585 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

1298 Commits

Author SHA1 Message Date
Vinod Kumar Vavilapalli 0b73606b1c YARN-1922. Fixed NodeManager to kill process-trees correctly in the presence of races between the launch and the stop-container call and when root processes crash. Contributed by Billie Rinaldi. 
(cherry picked from commit c5a46d4c8c)
 2014-11-03 16:41:21 -08:00
Vinod Kumar Vavilapalli f2ef8c7b48 YARN-2795. Fixed ResourceManager to not crash loading node-label data from HDFS in secure mode. Contributed by Wangda Tan. 
(cherry picked from commit ec6cbece8e)
 2014-11-03 13:46:05 -08:00
Vinod Kumar Vavilapalli c205841d49 YARN-2788. Fixed backwards compatiblity issues with log-aggregation feature that were caused when adding log-upload-time via YARN-2703. Contributed by Xuan Gong. 
(cherry picked from commit 58e9f24e0f)
 2014-11-03 13:20:01 -08:00
Vinod Kumar Vavilapalli dd82a1eb75 YARN-2798. Fixed YarnClient to populate the renewer correctly for Timeline delegation tokens. Contributed by Zhijie Shen. 
(cherry picked from commit 71fbb474f5)
 2014-11-03 12:51:30 -08:00
Jason Lowe 2098c68acb YARN-2730. DefaultContainerExecutor runs only one localizer at a time. Contributed by Siqi Li 
(cherry picked from commit 6157ace547)
 2014-11-03 20:40:02 +00:00
Zhijie Shen 3e41828639 YARN-2785. Fixed intermittent TestContainerResourceUsage failure. Contributed by Varun Vasudev. 
(cherry picked from commit 27715ec63b)
 2014-11-02 15:24:07 -08:00
Vinod Kumar Vavilapalli 5a0aac5506 YARN-2790. Fixed a NodeManager bug that was causing log-aggregation to fail beyond HFDS delegation-token expiry even when RM is a proxy-user (YARN-2704). Contributed by Jian He. 
(cherry picked from commit 5c0381c96a)
 2014-11-01 16:34:11 -07:00
Zhijie Shen 5492370a4c YARN-2711. Fixed TestDefaultContainerExecutor#testContainerLaunchError failure on Windows. Contributed by Varun Vasudev. 
(cherry picked from commit 1cd088fd9d)
 2014-10-31 17:51:03 -07:00
Jason Lowe 239c4cc572 YARN-2707. Potential null dereference in FSDownload. Contributed by Gera Shegalov 
(cherry picked from commit 7bc179f9f5)
 2014-10-31 22:55:06 +00:00
Vinod Kumar Vavilapalli f71d940e42 YARN-2789. Re-instated the NodeReport.newInstance private unstable API modified in YARN-2698 so that tests in YARN frameworks don't break. Contributed by Wangda Tan. 
(cherry picked from commit 6ce32f593b)
 2014-10-31 15:34:07 -07:00
Jian He a859adcc23 YARN-2770. Added functionality to renew/cancel TimeLineDelegationToken. Contributed by Zhijie Shen 
(cherry picked from commit 1b4be91866)
 2014-10-31 13:17:45 -07:00
Vinod Kumar Vavilapalli 16511ef404 YARN-2778. Moved node-lables' reports to the yarn nodes CLI from the admin CLI. Contributed by Wangda Tan. 
(cherry picked from commit b6c1188b85)
 2014-10-31 10:20:08 -07:00
Vinod Kumar Vavilapalli a3f032031d YARN-2779. Fixed ResourceManager to not require delegation tokens for communicating with Timeline Service. Contributed by Zhijie Shen. 
(cherry picked from commit d1828d9443)
 2014-10-30 23:17:50 -07:00
Vinod Kumar Vavilapalli 85fcdaf8d8 YARN-2698. Moved some node label APIs to be correctly placed in client protocol. Contributed by Wangda Tan. 
(cherry picked from commit e0233c16eb)
 2014-10-30 23:03:02 -07:00
Steve Loughran b69ebef49e YARN-2677 registry punycoding of usernames doesn't fix all usernames to be DNS-valid (stevel) 2014-10-30 22:32:08 +00:00
Jason Lowe 773ba8b606 YARN-2755. NM fails to clean up usercache_DEL_<timestamp> dirs after YARN-661. Contributed by Siqi Li 
(cherry picked from commit 73e626ad91)
 2014-10-30 15:12:35 +00:00
Zhijie Shen 2f62716441 YARN-2769. Fixed the problem that timeline domain is not set in distributed shell AM when using shell_command on Windows. Contributed by Varun Vasudev. 
(cherry picked from commit 72496311def320987c31822776e8fbb73dfcc726)
 2014-10-29 12:15:54 -07:00
Zhijie Shen dcb484fac0 YARN-2747. Fixed the test failure of TestAggregatedLogFormat when native I/O is enabled. Contributed by Xuan Gong. 
(cherry picked from commit ec63a3ffbd)
 2014-10-28 22:47:05 -07:00
Jian He e181498a67 YARN-2503. Added node lablels in web UI. Contributed by Wangda Tan 
(cherry picked from commit d5e0a09721)
(cherry picked from commit 720de7eb4c)
 2014-10-28 18:02:13 -07:00
Zhijie Shen ae62fe2ebd YARN-2741. Made NM web UI serve logs on the drive other than C: on Windows. Contributed by Craig Welch. 
(cherry picked from commit 8984e9b177)
 2014-10-28 14:13:31 -07:00
Xuan 126d1b9089 YARN-2758. Update TestApplicationHistoryClientService to use the new generic history store. Contributed by Zhijie Shen 2014-10-28 13:32:42 -07:00
Xuan 615af9e454 YARN-2279. Add UTs to cover timeline server authentication. Contributed by Zhijie Shen 2014-10-28 11:20:46 -07:00
Karthik Kambatla f5361a8354 YARN-2760. Remove 'experimental' from FairScheduler docs. (Harsh J via kasha) 
(cherry picked from commit ade3727ecb)
 2014-10-28 08:50:04 -07:00
Jian He 8d4c93aabf YARN-2502. Changed DistributedShell to support node labels. Contributed by Wangda Tan 
(cherry picked from commit f6b963fdfc)
 2014-10-27 20:14:30 -07:00
Vinod Kumar Vavilapalli 0af8cc9ca2 YARN-2704. Changed ResourceManager to optionally obtain tokens itself for the sake of localization and log-aggregation for long-running services. Contributed by Jian He. 
(cherry picked from commit a16d022ca4)
 2014-10-27 15:51:20 -07:00
Jian He ba1e549145 YARN-2591. Fixed AHSWebServices to return FORBIDDEN(403) if the request user doesn't have access to the history data. Contributed by Zhijie Shen 
(cherry picked from commit c05b581a55)
 2014-10-27 10:46:59 -07:00
Xuan 55c5612caf YARN-2726. CapacityScheduler should explicitly log when an accessible label has no capacity. Contributed by Wangda Tan 2014-10-27 10:30:09 -07:00
Zhijie Shen ab7a21569c YARN-2734. Skipped sub-folders in the local log dir when aggregating logs. Contributed by Xuan Gong. 
(cherry picked from commit caecd9fffe)
 2014-10-26 13:00:26 -07:00
Vinod Kumar Vavilapalli 259db7a726 YARN-2743. Fixed a bug in ResourceManager that was causing RMDelegationToken identifiers to be tampered and thus causing app submission failures in secure mode. Contributed by Jian He. 
(cherry picked from commit 0186645505)
 2014-10-26 11:29:06 -07:00
Xuan cccfc27e0d YARN-2723. Fix rmadmin -replaceLabelsOnNode does not correctly parse port. Contributed by Naganarasimha G R 2014-10-25 12:49:05 -07:00
Jian He c8d48d2981 YARN-2314. Disable ContainerManagementProtocolProxy cache by default to prevent creating thousands of threads in a large cluster. Contributed by Jason Lowe 
(cherry picked from commit f44cf99599)
 2014-10-24 23:06:38 -07:00
Jian He 728cc2394c YARN-1915. Fixed a race condition that client could use the ClientToAMToken to contact with AM before AM actually receives the ClientToAMTokenMasterKey. Contributed by Jason Lowe 
(cherry picked from commit 5864dd99a4)
 2014-10-24 22:49:19 -07:00
Zhijie Shen a41371e173 YARN-2703. Added logUploadedTime into LogValue for better display. Contributed by Xuan Gong. 
(cherry picked from commit f81dc3f995)
 2014-10-24 14:13:28 -07:00
Zhijie Shen 38fa39aa17 YARN-2724. Skipped uploading a local log file to HDFS if exception is raised when opening it. Contributed by Xuan Gong. 
(cherry picked from commit e31f0a6558)
 2014-10-24 11:16:44 -07:00
Zhijie Shen 09e422302f YARN-2209. Replaced AM resync/shutdown command with corresponding exceptions and made related MR changes. Contributed by Jian He. 
(cherry picked from commit 0f3b6900be)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2014-10-23 22:18:27 -07:00
Zhijie Shen 78efd4fd5c YARN-2732. Fixed syntax error in SecureContainer.apt.vm. Contributed by Jian He. 
(cherry picked from commit b94b8b30f2)
 2014-10-22 18:42:27 -07:00
Jian He 29d0164ee7 YARN-2198. Remove the need to run NodeManager as privileged account for Windows Secure Container Executor. Contributed by Remus Rusanu 
(cherry picked from commit 3b12fd6cfb)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2014-10-22 16:22:08 -07:00
Jian He a057552468 YARN-2682. Updated WindowsSecureContainerExecutor to not use DefaultContainerExecutor#getFirstApplicationDir and use getWorkingDir() instead. Contributed by Zhihai Xu 
(cherry picked from commit 0fd0ebae64)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java
 2014-10-22 16:19:40 -07:00
Vinod Kumar Vavilapalli e282805f1c YARN-1063. Augmented Hadoop common winutils to have the ability to create containers as domain users. Contributed by Remus Rusanu. 
Committed as a YARN patch even though all the code changes are in common.
(cherry picked from commit 5ca97f1e60)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2014-10-22 16:10:49 -07:00
Steve Loughran ada23f3dd3 YARN-2700 TestSecureRMRegistryOperations failing on windows: auth problems 2014-10-22 21:12:52 +01:00
Steve Loughran 8fdff5b791 YARN-2692 ktutil test hanging on some machines/ktutil versions (stevel) 2014-10-22 14:34:57 +01:00
Zhijie Shen b1ca12bc0b YARN-2721. Suppress NodeExist exception thrown by ZKRMStateStore when it retries creating znode. Contributed by Jian He. 
(cherry picked from commit 7e3b5e6f5c)
 2014-10-21 21:53:27 -07:00
Vinod Kumar Vavilapalli a1c5d4c51f YARN-2715. Fixed ResourceManager to respect common configurations for proxy users/groups beyond just the YARN level config. Contributed by Zhijie Shen. 
(cherry picked from commit c0e034336c)
 2014-10-21 20:11:14 -07:00
Zhijie Shen 7df8cbdd0f YARN-2709. Made timeline client getDelegationToken API retry if ConnectException happens. Contributed by Li Lu. 
(cherry picked from commit b2942762d7)
 2014-10-21 16:09:23 -07:00
cnauroth 8727f9e1b0 YARN-2720. Windows: Wildcard classpath variables not expanded against resources contained in archives. Contributed by Craig Welch. 
(cherry picked from commit 6637e3cf95)
(cherry picked from commit a066134277)
 2014-10-21 12:36:43 -07:00
Jason Lowe 413fe4cb02 YARN-90. NodeManager should identify failed disks becoming good again. Contributed by Varun Vasudev 
(cherry picked from commit 6f2028bd15)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestDirectoryCollection.java
 2014-10-21 17:43:22 +00:00
Zhijie Shen fcdeba204c YARN-2717. Avoided duplicate logging when container logs are not found. Contributed by Xuan Gong. 
(cherry picked from commit 171f2376d2)
 2014-10-20 22:24:19 -07:00
Jian He c7db30522c YARN-2701. Potential race condition in startLocalizer when using LinuxContainerExecutor. Contributed by Xuan Gong 
(cherry picked from commit 2839365f23)
 2014-10-20 20:53:28 -07:00
Vinod Kumar Vavilapalli debb3686d3 YARN-1972. Added a secure container-executor for Windows. Contributed by Remus Rusanu. 
commit ba7f31c2ee is the corresponding trunk commit, this is a slightly different patch for branch-2.
(cherry picked from commit 3326fba382)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2014-10-20 20:47:49 -07:00
Zhijie Shen 2af7f48625 YARN-2582. Fixed Log CLI and Web UI for showing aggregated logs of LRS. Contributed Xuan Gong. 
(cherry picked from commit e90718fa5a)
 2014-10-20 13:43:03 -07:00