Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
7,854 Commits 326 Branches 371 Tags
Commit Graph

1466 Commits

Author SHA1 Message Date
Vinod Kumar Vavilapalli
259db7a726 YARN-2743. Fixed a bug in ResourceManager that was causing RMDelegationToken identifiers to be tampered and thus causing app submission failures in secure mode. Contributed by Jian He. 
(cherry picked from commit 018664550507981297fd9f91e29408e6b7801ea9)
 2014-10-26 11:29:06 -07:00
Xuan
cccfc27e0d YARN-2723. Fix rmadmin -replaceLabelsOnNode does not correctly parse port. Contributed by Naganarasimha G R 2014-10-25 12:49:05 -07:00
Jian He
c8d48d2981 YARN-2314. Disable ContainerManagementProtocolProxy cache by default to prevent creating thousands of threads in a large cluster. Contributed by Jason Lowe 
(cherry picked from commit f44cf99599119b5e989be724eeab447b2dc4fe53)
 2014-10-24 23:06:38 -07:00
Jian He
728cc2394c YARN-1915. Fixed a race condition that client could use the ClientToAMToken to contact with AM before AM actually receives the ClientToAMTokenMasterKey. Contributed by Jason Lowe 
(cherry picked from commit 5864dd99a419b8a0fb973a4984c0f1d1e02ccf16)
 2014-10-24 22:49:19 -07:00
Zhijie Shen
a41371e173 YARN-2703. Added logUploadedTime into LogValue for better display. Contributed by Xuan Gong. 
(cherry picked from commit f81dc3f995579c1b94b11d60e9fc6da56c8a9496)
 2014-10-24 14:13:28 -07:00
Zhijie Shen
38fa39aa17 YARN-2724. Skipped uploading a local log file to HDFS if exception is raised when opening it. Contributed by Xuan Gong. 
(cherry picked from commit e31f0a6558b106662c83e1f797216e412b6689a9)
 2014-10-24 11:16:44 -07:00
Zhijie Shen
09e422302f YARN-2209. Replaced AM resync/shutdown command with corresponding exceptions and made related MR changes. Contributed by Jian He. 
(cherry picked from commit 0f3b6900be1a3b2e4624f31f84656f4a32dadce9)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2014-10-23 22:18:27 -07:00
Zhijie Shen
78efd4fd5c YARN-2732. Fixed syntax error in SecureContainer.apt.vm. Contributed by Jian He. 
(cherry picked from commit b94b8b30f282563ee2ecdd25761b2345aaf06c9b)
 2014-10-22 18:42:27 -07:00
Jian He
29d0164ee7 YARN-2198. Remove the need to run NodeManager as privileged account for Windows Secure Container Executor. Contributed by Remus Rusanu 
(cherry picked from commit 3b12fd6cfbf4cc91ef8e8616c7aafa9de006cde5)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2014-10-22 16:22:08 -07:00
Jian He
a057552468 YARN-2682. Updated WindowsSecureContainerExecutor to not use DefaultContainerExecutor#getFirstApplicationDir and use getWorkingDir() instead. Contributed by Zhihai Xu 
(cherry picked from commit 0fd0ebae645e671699f6a6a56a012ebe6dfb5b2a)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java
 2014-10-22 16:19:40 -07:00
Steve Loughran
ada23f3dd3 YARN-2700 TestSecureRMRegistryOperations failing on windows: auth problems 2014-10-22 21:12:52 +01:00
Steve Loughran
8fdff5b791 YARN-2692 ktutil test hanging on some machines/ktutil versions (stevel) 2014-10-22 14:34:57 +01:00
Zhijie Shen
b1ca12bc0b YARN-2721. Suppress NodeExist exception thrown by ZKRMStateStore when it retries creating znode. Contributed by Jian He. 
(cherry picked from commit 7e3b5e6f5cb4945b4fab27e8a83d04280df50e17)
 2014-10-21 21:53:27 -07:00
Vinod Kumar Vavilapalli
a1c5d4c51f YARN-2715. Fixed ResourceManager to respect common configurations for proxy users/groups beyond just the YARN level config. Contributed by Zhijie Shen. 
(cherry picked from commit c0e034336c85296be6f549d88d137fb2b2b79a15)
 2014-10-21 20:11:14 -07:00
Zhijie Shen
7df8cbdd0f YARN-2709. Made timeline client getDelegationToken API retry if ConnectException happens. Contributed by Li Lu. 
(cherry picked from commit b2942762d7f76d510ece5621c71116346a6b12f6)
 2014-10-21 16:09:23 -07:00
cnauroth
8727f9e1b0 YARN-2720. Windows: Wildcard classpath variables not expanded against resources contained in archives. Contributed by Craig Welch. 
(cherry picked from commit 6637e3cf95b3a9be8d6b9cd66bc849a0607e8ed5)
(cherry picked from commit a066134277716a9709b63cfb11b4be5627a9bd3f)
 2014-10-21 12:36:43 -07:00
Jason Lowe
413fe4cb02 YARN-90. NodeManager should identify failed disks becoming good again. Contributed by Varun Vasudev 
(cherry picked from commit 6f2028bd1514d90b831f889fd0ee7f2ba5c15000)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestDirectoryCollection.java
 2014-10-21 17:43:22 +00:00
Zhijie Shen
fcdeba204c YARN-2717. Avoided duplicate logging when container logs are not found. Contributed by Xuan Gong. 
(cherry picked from commit 171f2376d23d51b61b9c9b3804ee86dbd4de033a)
 2014-10-20 22:24:19 -07:00
Jian He
852ffea84e Missing file for YARN-1879 
(cherry picked from commit 4a78a752286effbf1a0d8695325f9d7464a09fb4)
 2014-10-20 21:15:30 -07:00
Jian He
23b4af488e Missing file for YARN-2701 
(cherry picked from commit 4fa1fb3193bf39fcb1bd7f8f8391a78f69c3c302)
 2014-10-20 20:53:28 -07:00
Jian He
c7db30522c YARN-2701. Potential race condition in startLocalizer when using LinuxContainerExecutor. Contributed by Xuan Gong 
(cherry picked from commit 2839365f230165222f63129979ea82ada79ec56e)
 2014-10-20 20:53:28 -07:00
Vinod Kumar Vavilapalli
debb3686d3 YARN-1972. Added a secure container-executor for Windows. Contributed by Remus Rusanu. 
commit ba7f31c2ee8d23ecb183f88920ef06053c0b9769 is the corresponding trunk commit, this is a slightly different patch for branch-2.
(cherry picked from commit 3326fba382350db88ccaa67625f92fa1dc5fd3e3)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2014-10-20 20:47:49 -07:00
Zhijie Shen
2af7f48625 YARN-2582. Fixed Log CLI and Web UI for showing aggregated logs of LRS. Contributed Xuan Gong. 
(cherry picked from commit e90718fa5a0e7c18592af61534668acebb9db51b)
 2014-10-20 13:43:03 -07:00
Zhijie Shen
6d78dc3ee4 YARN-2673. Made timeline client put APIs retry if ConnectException happens. Contributed by Li Lu. 
(cherry picked from commit 89427419a3c5eaab0f73bae98d675979b9efab5f)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
 2014-10-20 12:31:33 -07:00
Vinod Kumar Vavilapalli
88b7e5ca65 YARN-2504. Enhanced RM Admin CLI to support management of node-labels. Contribyted by Wangda Tan. 
(cherry picked from commit 82567664988b673f1b819a42a4baf31cb0dcb331)
 2014-10-18 12:09:02 -07:00
Vinod Kumar Vavilapalli
24b0529a9f YARN-2676. Enhanced Timeline auth-filter to support proxy users. Contributed by Zhijie Shen. 
(cherry picked from commit 39063cd36f96e351e4a6bf0bc2b6185711d4b059)
 2014-10-17 22:04:14 -07:00
Vinod Kumar Vavilapalli
ce42210651 YARN-2705. Fixed bugs in ResourceManager node-label manager that were causing test-failures: added a dummy in-memory labels-manager. Contributed by Wangda Tan. 
(cherry picked from commit e9c66e8fd2ccb658db2848e1ab911f1502de4de5)
 2014-10-17 18:27:34 -07:00
Jian He
4d03915be9 YARN-1879. Marked Idempotent/AtMostOnce annotations to ApplicationMasterProtocol for RM fail over. Contributed by Tsuyoshi OZAWA 
(cherry picked from commit c3de2412eb7633ff16c67e71e73bbe27a982d984)
 2014-10-17 16:36:45 -07:00
Jian He
b7504f0394 YARN-2588. Standby RM fails to transitionToActive if previous transitionToActive failed with ZK exception. Contributed by Rohith Sharmaks 
(cherry picked from commit a6aa6e42cacdbfcc1c2b7c19e7239204fe9ff654)
 2014-10-17 10:55:29 -07:00
Vinod Kumar Vavilapalli
778fbcd5d3 YARN-2699. Fixed a bug in CommonNodeLabelsManager that caused tests to fail when using ephemeral ports on NodeIDs. Contributed by Wangda Tan. 
(cherry picked from commit abae63caf9c53b404f2f2db7d482555484eaeaf8)
 2014-10-17 08:59:49 -07:00
Jian He
6be8330606 YARN-2621. Simplify the output when the user doesn't have the access for getDomain(s). Contributed by Zhijie Shen 
(cherry picked from commit 233d446be1bc1bc77c7c1c45386086a732897afd)
 2014-10-16 16:38:27 -07:00
Steve Loughran
b0a72f354b YARN-2689 TestSecureRMRegistryOperations failing on windows: secure ZK won't start 2014-10-16 14:21:38 -07:00
Vinod Kumar Vavilapalli
8a09fdccb3 YARN-2685. Fixed a bug in CommonNodeLabelsManager that caused wrong resource tracking per label when a host runs multiple node-managers. Contributed by Wangda Tan. 
(cherry picked from commit b3056c266a628a65cf7ceb61b55ab3bd0a09baf2)
 2014-10-15 18:49:06 -07:00
Vinod Kumar Vavilapalli
fe80fd5e88 YARN-2496. Enhanced Capacity Scheduler to have basic support for allocating resources based on node-labels. Contributed by Wangda Tan. 
YARN-2500. Ehnaced ResourceManager to support schedulers allocating resources based on node-labels. Contributed by Wangda Tan.

(cherry picked from commit f2ea555ac6c06a3f2f6559731f48711fff05d3f1)
 2014-10-15 18:35:34 -07:00
Jian He
bd45dba807 YARN-2312. Deprecated old ContainerId#getId API and updated MapReduce to use ContainerId#getContainerId instead. Contributed by Tsuyoshi OZAWA 
(cherry picked from commit f93d2ea27e7554272845fb11d97b3564704834b3)
 2014-10-15 15:31:57 -07:00
Zhijie Shen
eb055eadd7 YARN-2656. Made RM web services authentication filter support proxy user. Contributed by Varun Vasudev and Zhijie Shen. 
(cherry picked from commit 1220bb72d452521c6f09cebe1dd77341054ee9dd)
 2014-10-14 21:58:12 -07:00
Zhijie Shen
e71fa82ee5 HADOOP-11181. Generalized o.a.h.s.t.d.DelegationTokenManager to handle all sub-classes of AbstractDelegationTokenIdentifier. Contributed by Zhijie Shen. 
(cherry picked from commit cdce88376a60918dfe2f3bcd82a7666d74992a19)
 2014-10-14 11:39:06 -07:00
Karthik Kambatla
e6102b1828 YARN-2566. DefaultContainerExecutor should pick a working directory randomly. (Zhihai Xu via kasha) 
(cherry picked from commit cc93e7e683fa74eb1a7aa2b357a36667bd21086a)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java
 2014-10-13 16:42:47 -07:00
Jian He
29496c9be2 YARN-2308. Changed CapacityScheduler to explicitly throw exception if the queue 
to which the apps were submitted is changed across RM restart. Contributed by Craig Welch & Chang Li
(cherry picked from commit f9680d9a160ee527c8f2c1494584abf1a1f70f82)
 2014-10-13 14:16:57 -07:00
Jason Lowe
9d73c1b15f YARN-2377. Localization exception stack traces are not passed as diagnostic info. Contributed by Gera Shegalov 
(cherry picked from commit a56ea0100215ecf2e1471a18812b668658197239)
 2014-10-13 18:34:28 +00:00
Zhijie Shen
6da637ec12 YARN-2651. Spun off LogRollingInterval from LogAggregationContext. Contributed by Xuan Gong. 
(cherry picked from commit 4aed2d8e91c7dccc78fbaffc409d3076c3316289)
 2014-10-13 10:56:22 -07:00
Jason Lowe
eb5faa161a YARN-2667. Fix the release audit warning caused by hadoop-yarn-registry. Contributed by Yi Liu 
(cherry picked from commit 344a10ad5e26c25abd62eda65eec2820bb808a74)
 2014-10-13 15:46:14 +00:00
Steve Loughran
752f724d52 YARN-2668 yarn-registry JAR won't link against ZK 3.4.5. (stevel) 2014-10-11 18:32:47 -07:00
Vinod Kumar Vavilapalli
8dacf89b3e YARN-2501. Enhanced AMRMClient library to support requests against node labels. Contributed by Wangda Tan. 
(cherry picked from commit a5ec3d080978a67837946a991843a081ea712539)
 2014-10-10 19:59:30 -07:00
Vinod Kumar Vavilapalli
bb6c79f76c YARN-2494. Added NodeLabels Manager internal API and implementation. Contributed by Wangda Tan. 
(cherry picked from commit db7f1653198b950e89567c06898d64f6b930a0ee)
 2014-10-10 11:46:10 -07:00
Zhijie Shen
16d7144337 YARN-2583. Modified AggregatedLogDeletionService to be able to delete rolling aggregated logs. Contributed by Xuan Gong. 
(cherry picked from commit cb81bac0029fce3a9726df3523f0b692cd3375b8)
 2014-10-10 00:18:24 -07:00
cnauroth
281041a57e YARN-2662. TestCgroupsLCEResourcesHandler leaks file descriptors. Contributed by Chris Nauroth. 
(cherry picked from commit d3afd730acfa380ab5032be5ee296c5d73744518)
(cherry picked from commit b81641a31050b6a5cf5db3af636125e0617ffd56)
 2014-10-09 22:49:00 -07:00
Zhijie Shen
647874206c YARN-2617. Fixed ApplicationSubmissionContext to still set resource for backward compatibility. Contributed by Wangda Tan. 
(cherry picked from commit e532ed8faa8db4b008a5b8d3f82b48a1b314fa6c)
 2014-10-09 18:01:13 -07:00
Vinod Kumar Vavilapalli
967c758373 YARN-2544. Added admin-API objects for using node-labels. Contributed by Wangda Tan. 
(cherry picked from commit 596702a02501e9cb09aabced168027189eaf02ba)
 2014-10-09 15:47:51 -07:00
Zhijie Shen
a4522e90f6 YARN-2629. Made the distributed shell use the domain-based timeline ACLs. Contributed by Zhijie Shen. 
(cherry picked from commit 1d4612f5ad9678c952b416e798dccd20c88f96ef)
 2014-10-09 13:03:03 -07:00