Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
7,678 Commits 326 Branches 371 Tags
Commit Graph

1384 Commits

Author SHA1 Message Date
Jian He
a859adcc23 YARN-2770. Added functionality to renew/cancel TimeLineDelegationToken. Contributed by Zhijie Shen 
(cherry picked from commit 1b4be918664b09272b120bc42de3e5fc02d79047)
 2014-10-31 13:17:45 -07:00
Vinod Kumar Vavilapalli
16511ef404 YARN-2778. Moved node-lables' reports to the yarn nodes CLI from the admin CLI. Contributed by Wangda Tan. 
(cherry picked from commit b6c1188b855d636586cd8fd0fb6d8e984bbfb0f5)
 2014-10-31 10:20:08 -07:00
Vinod Kumar Vavilapalli
a3f032031d YARN-2779. Fixed ResourceManager to not require delegation tokens for communicating with Timeline Service. Contributed by Zhijie Shen. 
(cherry picked from commit d1828d94435eca21761b0ba8458f9de2f125d012)
 2014-10-30 23:17:50 -07:00
Vinod Kumar Vavilapalli
85fcdaf8d8 YARN-2698. Moved some node label APIs to be correctly placed in client protocol. Contributed by Wangda Tan. 
(cherry picked from commit e0233c16ebd06bb3aabeb523cd17259008e243ef)
 2014-10-30 23:03:02 -07:00
Steve Loughran
b69ebef49e YARN-2677 registry punycoding of usernames doesn't fix all usernames to be DNS-valid (stevel) 2014-10-30 22:32:08 +00:00
Jason Lowe
773ba8b606 YARN-2755. NM fails to clean up usercache_DEL_<timestamp> dirs after YARN-661. Contributed by Siqi Li 
(cherry picked from commit 73e626ad91cd5c06a005068d8432fd16e06fe6a0)
 2014-10-30 15:12:35 +00:00
Zhijie Shen
2f62716441 YARN-2769. Fixed the problem that timeline domain is not set in distributed shell AM when using shell_command on Windows. Contributed by Varun Vasudev. 
(cherry picked from commit 72496311def320987c31822776e8fbb73dfcc726)
 2014-10-29 12:15:54 -07:00
Zhijie Shen
dcb484fac0 YARN-2747. Fixed the test failure of TestAggregatedLogFormat when native I/O is enabled. Contributed by Xuan Gong. 
(cherry picked from commit ec63a3ffbd9413e7434594682fdbbd36eef7413c)
 2014-10-28 22:47:05 -07:00
Jian He
e181498a67 YARN-2503. Added node lablels in web UI. Contributed by Wangda Tan 
(cherry picked from commit d5e0a09721a5156fa2ee51ac1c32fbfd9905b8fb)
(cherry picked from commit 720de7eb4ca310e8d800938fbb56fd4de06fc6a5)
 2014-10-28 18:02:13 -07:00
Zhijie Shen
ae62fe2ebd YARN-2741. Made NM web UI serve logs on the drive other than C: on Windows. Contributed by Craig Welch. 
(cherry picked from commit 8984e9b1774033e379b57da1bd30a5c81888c7a3)
 2014-10-28 14:13:31 -07:00
Xuan
126d1b9089 YARN-2758. Update TestApplicationHistoryClientService to use the new generic history store. Contributed by Zhijie Shen 2014-10-28 13:32:42 -07:00
Xuan
615af9e454 YARN-2279. Add UTs to cover timeline server authentication. Contributed by Zhijie Shen 2014-10-28 11:20:46 -07:00
Karthik Kambatla
f5361a8354 YARN-2760. Remove 'experimental' from FairScheduler docs. (Harsh J via kasha) 
(cherry picked from commit ade3727ecb092935dcc0f1291c1e6cf43d764a03)
 2014-10-28 08:50:04 -07:00
Jian He
8d4c93aabf YARN-2502. Changed DistributedShell to support node labels. Contributed by Wangda Tan 
(cherry picked from commit f6b963fdfc517429149165e4bb6fb947be6e3c99)
 2014-10-27 20:14:30 -07:00
Vinod Kumar Vavilapalli
0af8cc9ca2 YARN-2704. Changed ResourceManager to optionally obtain tokens itself for the sake of localization and log-aggregation for long-running services. Contributed by Jian He. 
(cherry picked from commit a16d022ca4313a41425c8e97841c841a2d6f2f54)
 2014-10-27 15:51:20 -07:00
Jian He
ba1e549145 YARN-2591. Fixed AHSWebServices to return FORBIDDEN(403) if the request user doesn't have access to the history data. Contributed by Zhijie Shen 
(cherry picked from commit c05b581a5522eed499d3ba16af9fa6dc694563f6)
 2014-10-27 10:46:59 -07:00
Xuan
55c5612caf YARN-2726. CapacityScheduler should explicitly log when an accessible label has no capacity. Contributed by Wangda Tan 2014-10-27 10:30:09 -07:00
Zhijie Shen
ab7a21569c YARN-2734. Skipped sub-folders in the local log dir when aggregating logs. Contributed by Xuan Gong. 
(cherry picked from commit caecd9fffe7c6216be31f3ab65349182045451fa)
 2014-10-26 13:00:26 -07:00
Vinod Kumar Vavilapalli
259db7a726 YARN-2743. Fixed a bug in ResourceManager that was causing RMDelegationToken identifiers to be tampered and thus causing app submission failures in secure mode. Contributed by Jian He. 
(cherry picked from commit 018664550507981297fd9f91e29408e6b7801ea9)
 2014-10-26 11:29:06 -07:00
Xuan
cccfc27e0d YARN-2723. Fix rmadmin -replaceLabelsOnNode does not correctly parse port. Contributed by Naganarasimha G R 2014-10-25 12:49:05 -07:00
Jian He
c8d48d2981 YARN-2314. Disable ContainerManagementProtocolProxy cache by default to prevent creating thousands of threads in a large cluster. Contributed by Jason Lowe 
(cherry picked from commit f44cf99599119b5e989be724eeab447b2dc4fe53)
 2014-10-24 23:06:38 -07:00
Jian He
728cc2394c YARN-1915. Fixed a race condition that client could use the ClientToAMToken to contact with AM before AM actually receives the ClientToAMTokenMasterKey. Contributed by Jason Lowe 
(cherry picked from commit 5864dd99a419b8a0fb973a4984c0f1d1e02ccf16)
 2014-10-24 22:49:19 -07:00
Zhijie Shen
a41371e173 YARN-2703. Added logUploadedTime into LogValue for better display. Contributed by Xuan Gong. 
(cherry picked from commit f81dc3f995579c1b94b11d60e9fc6da56c8a9496)
 2014-10-24 14:13:28 -07:00
Zhijie Shen
38fa39aa17 YARN-2724. Skipped uploading a local log file to HDFS if exception is raised when opening it. Contributed by Xuan Gong. 
(cherry picked from commit e31f0a6558b106662c83e1f797216e412b6689a9)
 2014-10-24 11:16:44 -07:00
Zhijie Shen
09e422302f YARN-2209. Replaced AM resync/shutdown command with corresponding exceptions and made related MR changes. Contributed by Jian He. 
(cherry picked from commit 0f3b6900be1a3b2e4624f31f84656f4a32dadce9)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2014-10-23 22:18:27 -07:00
Zhijie Shen
78efd4fd5c YARN-2732. Fixed syntax error in SecureContainer.apt.vm. Contributed by Jian He. 
(cherry picked from commit b94b8b30f282563ee2ecdd25761b2345aaf06c9b)
 2014-10-22 18:42:27 -07:00
Jian He
29d0164ee7 YARN-2198. Remove the need to run NodeManager as privileged account for Windows Secure Container Executor. Contributed by Remus Rusanu 
(cherry picked from commit 3b12fd6cfbf4cc91ef8e8616c7aafa9de006cde5)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2014-10-22 16:22:08 -07:00
Jian He
a057552468 YARN-2682. Updated WindowsSecureContainerExecutor to not use DefaultContainerExecutor#getFirstApplicationDir and use getWorkingDir() instead. Contributed by Zhihai Xu 
(cherry picked from commit 0fd0ebae645e671699f6a6a56a012ebe6dfb5b2a)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java
 2014-10-22 16:19:40 -07:00
Steve Loughran
ada23f3dd3 YARN-2700 TestSecureRMRegistryOperations failing on windows: auth problems 2014-10-22 21:12:52 +01:00
Steve Loughran
8fdff5b791 YARN-2692 ktutil test hanging on some machines/ktutil versions (stevel) 2014-10-22 14:34:57 +01:00
Zhijie Shen
b1ca12bc0b YARN-2721. Suppress NodeExist exception thrown by ZKRMStateStore when it retries creating znode. Contributed by Jian He. 
(cherry picked from commit 7e3b5e6f5cb4945b4fab27e8a83d04280df50e17)
 2014-10-21 21:53:27 -07:00
Vinod Kumar Vavilapalli
a1c5d4c51f YARN-2715. Fixed ResourceManager to respect common configurations for proxy users/groups beyond just the YARN level config. Contributed by Zhijie Shen. 
(cherry picked from commit c0e034336c85296be6f549d88d137fb2b2b79a15)
 2014-10-21 20:11:14 -07:00
Zhijie Shen
7df8cbdd0f YARN-2709. Made timeline client getDelegationToken API retry if ConnectException happens. Contributed by Li Lu. 
(cherry picked from commit b2942762d7f76d510ece5621c71116346a6b12f6)
 2014-10-21 16:09:23 -07:00
cnauroth
8727f9e1b0 YARN-2720. Windows: Wildcard classpath variables not expanded against resources contained in archives. Contributed by Craig Welch. 
(cherry picked from commit 6637e3cf95b3a9be8d6b9cd66bc849a0607e8ed5)
(cherry picked from commit a066134277716a9709b63cfb11b4be5627a9bd3f)
 2014-10-21 12:36:43 -07:00
Jason Lowe
413fe4cb02 YARN-90. NodeManager should identify failed disks becoming good again. Contributed by Varun Vasudev 
(cherry picked from commit 6f2028bd1514d90b831f889fd0ee7f2ba5c15000)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestDirectoryCollection.java
 2014-10-21 17:43:22 +00:00
Zhijie Shen
fcdeba204c YARN-2717. Avoided duplicate logging when container logs are not found. Contributed by Xuan Gong. 
(cherry picked from commit 171f2376d23d51b61b9c9b3804ee86dbd4de033a)
 2014-10-20 22:24:19 -07:00
Jian He
852ffea84e Missing file for YARN-1879 
(cherry picked from commit 4a78a752286effbf1a0d8695325f9d7464a09fb4)
 2014-10-20 21:15:30 -07:00
Jian He
23b4af488e Missing file for YARN-2701 
(cherry picked from commit 4fa1fb3193bf39fcb1bd7f8f8391a78f69c3c302)
 2014-10-20 20:53:28 -07:00
Jian He
c7db30522c YARN-2701. Potential race condition in startLocalizer when using LinuxContainerExecutor. Contributed by Xuan Gong 
(cherry picked from commit 2839365f230165222f63129979ea82ada79ec56e)
 2014-10-20 20:53:28 -07:00
Vinod Kumar Vavilapalli
debb3686d3 YARN-1972. Added a secure container-executor for Windows. Contributed by Remus Rusanu. 
commit ba7f31c2ee8d23ecb183f88920ef06053c0b9769 is the corresponding trunk commit, this is a slightly different patch for branch-2.
(cherry picked from commit 3326fba382350db88ccaa67625f92fa1dc5fd3e3)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2014-10-20 20:47:49 -07:00
Zhijie Shen
2af7f48625 YARN-2582. Fixed Log CLI and Web UI for showing aggregated logs of LRS. Contributed Xuan Gong. 
(cherry picked from commit e90718fa5a0e7c18592af61534668acebb9db51b)
 2014-10-20 13:43:03 -07:00
Zhijie Shen
6d78dc3ee4 YARN-2673. Made timeline client put APIs retry if ConnectException happens. Contributed by Li Lu. 
(cherry picked from commit 89427419a3c5eaab0f73bae98d675979b9efab5f)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
 2014-10-20 12:31:33 -07:00
Vinod Kumar Vavilapalli
88b7e5ca65 YARN-2504. Enhanced RM Admin CLI to support management of node-labels. Contribyted by Wangda Tan. 
(cherry picked from commit 82567664988b673f1b819a42a4baf31cb0dcb331)
 2014-10-18 12:09:02 -07:00
Vinod Kumar Vavilapalli
24b0529a9f YARN-2676. Enhanced Timeline auth-filter to support proxy users. Contributed by Zhijie Shen. 
(cherry picked from commit 39063cd36f96e351e4a6bf0bc2b6185711d4b059)
 2014-10-17 22:04:14 -07:00
Vinod Kumar Vavilapalli
ce42210651 YARN-2705. Fixed bugs in ResourceManager node-label manager that were causing test-failures: added a dummy in-memory labels-manager. Contributed by Wangda Tan. 
(cherry picked from commit e9c66e8fd2ccb658db2848e1ab911f1502de4de5)
 2014-10-17 18:27:34 -07:00
Jian He
4d03915be9 YARN-1879. Marked Idempotent/AtMostOnce annotations to ApplicationMasterProtocol for RM fail over. Contributed by Tsuyoshi OZAWA 
(cherry picked from commit c3de2412eb7633ff16c67e71e73bbe27a982d984)
 2014-10-17 16:36:45 -07:00
Jian He
b7504f0394 YARN-2588. Standby RM fails to transitionToActive if previous transitionToActive failed with ZK exception. Contributed by Rohith Sharmaks 
(cherry picked from commit a6aa6e42cacdbfcc1c2b7c19e7239204fe9ff654)
 2014-10-17 10:55:29 -07:00
Vinod Kumar Vavilapalli
778fbcd5d3 YARN-2699. Fixed a bug in CommonNodeLabelsManager that caused tests to fail when using ephemeral ports on NodeIDs. Contributed by Wangda Tan. 
(cherry picked from commit abae63caf9c53b404f2f2db7d482555484eaeaf8)
 2014-10-17 08:59:49 -07:00
Jian He
6be8330606 YARN-2621. Simplify the output when the user doesn't have the access for getDomain(s). Contributed by Zhijie Shen 
(cherry picked from commit 233d446be1bc1bc77c7c1c45386086a732897afd)
 2014-10-16 16:38:27 -07:00
Steve Loughran
b0a72f354b YARN-2689 TestSecureRMRegistryOperations failing on windows: secure ZK won't start 2014-10-16 14:21:38 -07:00