Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
7,500 Commits 326 Branches 371 Tags
Commit Graph

1307 Commits

Author SHA1 Message Date
Jian He
a057552468 YARN-2682. Updated WindowsSecureContainerExecutor to not use DefaultContainerExecutor#getFirstApplicationDir and use getWorkingDir() instead. Contributed by Zhihai Xu 
(cherry picked from commit 0fd0ebae645e671699f6a6a56a012ebe6dfb5b2a)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java
 2014-10-22 16:19:40 -07:00
Steve Loughran
ada23f3dd3 YARN-2700 TestSecureRMRegistryOperations failing on windows: auth problems 2014-10-22 21:12:52 +01:00
Steve Loughran
8fdff5b791 YARN-2692 ktutil test hanging on some machines/ktutil versions (stevel) 2014-10-22 14:34:57 +01:00
Zhijie Shen
b1ca12bc0b YARN-2721. Suppress NodeExist exception thrown by ZKRMStateStore when it retries creating znode. Contributed by Jian He. 
(cherry picked from commit 7e3b5e6f5cb4945b4fab27e8a83d04280df50e17)
 2014-10-21 21:53:27 -07:00
Vinod Kumar Vavilapalli
a1c5d4c51f YARN-2715. Fixed ResourceManager to respect common configurations for proxy users/groups beyond just the YARN level config. Contributed by Zhijie Shen. 
(cherry picked from commit c0e034336c85296be6f549d88d137fb2b2b79a15)
 2014-10-21 20:11:14 -07:00
Zhijie Shen
7df8cbdd0f YARN-2709. Made timeline client getDelegationToken API retry if ConnectException happens. Contributed by Li Lu. 
(cherry picked from commit b2942762d7f76d510ece5621c71116346a6b12f6)
 2014-10-21 16:09:23 -07:00
cnauroth
8727f9e1b0 YARN-2720. Windows: Wildcard classpath variables not expanded against resources contained in archives. Contributed by Craig Welch. 
(cherry picked from commit 6637e3cf95b3a9be8d6b9cd66bc849a0607e8ed5)
(cherry picked from commit a066134277716a9709b63cfb11b4be5627a9bd3f)
 2014-10-21 12:36:43 -07:00
Jason Lowe
413fe4cb02 YARN-90. NodeManager should identify failed disks becoming good again. Contributed by Varun Vasudev 
(cherry picked from commit 6f2028bd1514d90b831f889fd0ee7f2ba5c15000)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestDirectoryCollection.java
 2014-10-21 17:43:22 +00:00
Zhijie Shen
fcdeba204c YARN-2717. Avoided duplicate logging when container logs are not found. Contributed by Xuan Gong. 
(cherry picked from commit 171f2376d23d51b61b9c9b3804ee86dbd4de033a)
 2014-10-20 22:24:19 -07:00
Jian He
852ffea84e Missing file for YARN-1879 
(cherry picked from commit 4a78a752286effbf1a0d8695325f9d7464a09fb4)
 2014-10-20 21:15:30 -07:00
Jian He
23b4af488e Missing file for YARN-2701 
(cherry picked from commit 4fa1fb3193bf39fcb1bd7f8f8391a78f69c3c302)
 2014-10-20 20:53:28 -07:00
Jian He
c7db30522c YARN-2701. Potential race condition in startLocalizer when using LinuxContainerExecutor. Contributed by Xuan Gong 
(cherry picked from commit 2839365f230165222f63129979ea82ada79ec56e)
 2014-10-20 20:53:28 -07:00
Vinod Kumar Vavilapalli
debb3686d3 YARN-1972. Added a secure container-executor for Windows. Contributed by Remus Rusanu. 
commit ba7f31c2ee8d23ecb183f88920ef06053c0b9769 is the corresponding trunk commit, this is a slightly different patch for branch-2.
(cherry picked from commit 3326fba382350db88ccaa67625f92fa1dc5fd3e3)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2014-10-20 20:47:49 -07:00
Zhijie Shen
2af7f48625 YARN-2582. Fixed Log CLI and Web UI for showing aggregated logs of LRS. Contributed Xuan Gong. 
(cherry picked from commit e90718fa5a0e7c18592af61534668acebb9db51b)
 2014-10-20 13:43:03 -07:00
Zhijie Shen
6d78dc3ee4 YARN-2673. Made timeline client put APIs retry if ConnectException happens. Contributed by Li Lu. 
(cherry picked from commit 89427419a3c5eaab0f73bae98d675979b9efab5f)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
 2014-10-20 12:31:33 -07:00
Vinod Kumar Vavilapalli
88b7e5ca65 YARN-2504. Enhanced RM Admin CLI to support management of node-labels. Contribyted by Wangda Tan. 
(cherry picked from commit 82567664988b673f1b819a42a4baf31cb0dcb331)
 2014-10-18 12:09:02 -07:00
Vinod Kumar Vavilapalli
24b0529a9f YARN-2676. Enhanced Timeline auth-filter to support proxy users. Contributed by Zhijie Shen. 
(cherry picked from commit 39063cd36f96e351e4a6bf0bc2b6185711d4b059)
 2014-10-17 22:04:14 -07:00
Vinod Kumar Vavilapalli
ce42210651 YARN-2705. Fixed bugs in ResourceManager node-label manager that were causing test-failures: added a dummy in-memory labels-manager. Contributed by Wangda Tan. 
(cherry picked from commit e9c66e8fd2ccb658db2848e1ab911f1502de4de5)
 2014-10-17 18:27:34 -07:00
Jian He
4d03915be9 YARN-1879. Marked Idempotent/AtMostOnce annotations to ApplicationMasterProtocol for RM fail over. Contributed by Tsuyoshi OZAWA 
(cherry picked from commit c3de2412eb7633ff16c67e71e73bbe27a982d984)
 2014-10-17 16:36:45 -07:00
Jian He
b7504f0394 YARN-2588. Standby RM fails to transitionToActive if previous transitionToActive failed with ZK exception. Contributed by Rohith Sharmaks 
(cherry picked from commit a6aa6e42cacdbfcc1c2b7c19e7239204fe9ff654)
 2014-10-17 10:55:29 -07:00
Vinod Kumar Vavilapalli
778fbcd5d3 YARN-2699. Fixed a bug in CommonNodeLabelsManager that caused tests to fail when using ephemeral ports on NodeIDs. Contributed by Wangda Tan. 
(cherry picked from commit abae63caf9c53b404f2f2db7d482555484eaeaf8)
 2014-10-17 08:59:49 -07:00
Jian He
6be8330606 YARN-2621. Simplify the output when the user doesn't have the access for getDomain(s). Contributed by Zhijie Shen 
(cherry picked from commit 233d446be1bc1bc77c7c1c45386086a732897afd)
 2014-10-16 16:38:27 -07:00
Steve Loughran
b0a72f354b YARN-2689 TestSecureRMRegistryOperations failing on windows: secure ZK won't start 2014-10-16 14:21:38 -07:00
Vinod Kumar Vavilapalli
8a09fdccb3 YARN-2685. Fixed a bug in CommonNodeLabelsManager that caused wrong resource tracking per label when a host runs multiple node-managers. Contributed by Wangda Tan. 
(cherry picked from commit b3056c266a628a65cf7ceb61b55ab3bd0a09baf2)
 2014-10-15 18:49:06 -07:00
Vinod Kumar Vavilapalli
fe80fd5e88 YARN-2496. Enhanced Capacity Scheduler to have basic support for allocating resources based on node-labels. Contributed by Wangda Tan. 
YARN-2500. Ehnaced ResourceManager to support schedulers allocating resources based on node-labels. Contributed by Wangda Tan.

(cherry picked from commit f2ea555ac6c06a3f2f6559731f48711fff05d3f1)
 2014-10-15 18:35:34 -07:00
Jian He
bd45dba807 YARN-2312. Deprecated old ContainerId#getId API and updated MapReduce to use ContainerId#getContainerId instead. Contributed by Tsuyoshi OZAWA 
(cherry picked from commit f93d2ea27e7554272845fb11d97b3564704834b3)
 2014-10-15 15:31:57 -07:00
Zhijie Shen
eb055eadd7 YARN-2656. Made RM web services authentication filter support proxy user. Contributed by Varun Vasudev and Zhijie Shen. 
(cherry picked from commit 1220bb72d452521c6f09cebe1dd77341054ee9dd)
 2014-10-14 21:58:12 -07:00
Zhijie Shen
e71fa82ee5 HADOOP-11181. Generalized o.a.h.s.t.d.DelegationTokenManager to handle all sub-classes of AbstractDelegationTokenIdentifier. Contributed by Zhijie Shen. 
(cherry picked from commit cdce88376a60918dfe2f3bcd82a7666d74992a19)
 2014-10-14 11:39:06 -07:00
Karthik Kambatla
e6102b1828 YARN-2566. DefaultContainerExecutor should pick a working directory randomly. (Zhihai Xu via kasha) 
(cherry picked from commit cc93e7e683fa74eb1a7aa2b357a36667bd21086a)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java
 2014-10-13 16:42:47 -07:00
Jian He
29496c9be2 YARN-2308. Changed CapacityScheduler to explicitly throw exception if the queue 
to which the apps were submitted is changed across RM restart. Contributed by Craig Welch & Chang Li
(cherry picked from commit f9680d9a160ee527c8f2c1494584abf1a1f70f82)
 2014-10-13 14:16:57 -07:00
Jason Lowe
9d73c1b15f YARN-2377. Localization exception stack traces are not passed as diagnostic info. Contributed by Gera Shegalov 
(cherry picked from commit a56ea0100215ecf2e1471a18812b668658197239)
 2014-10-13 18:34:28 +00:00
Zhijie Shen
6da637ec12 YARN-2651. Spun off LogRollingInterval from LogAggregationContext. Contributed by Xuan Gong. 
(cherry picked from commit 4aed2d8e91c7dccc78fbaffc409d3076c3316289)
 2014-10-13 10:56:22 -07:00
Jason Lowe
eb5faa161a YARN-2667. Fix the release audit warning caused by hadoop-yarn-registry. Contributed by Yi Liu 
(cherry picked from commit 344a10ad5e26c25abd62eda65eec2820bb808a74)
 2014-10-13 15:46:14 +00:00
Steve Loughran
752f724d52 YARN-2668 yarn-registry JAR won't link against ZK 3.4.5. (stevel) 2014-10-11 18:32:47 -07:00
Vinod Kumar Vavilapalli
8dacf89b3e YARN-2501. Enhanced AMRMClient library to support requests against node labels. Contributed by Wangda Tan. 
(cherry picked from commit a5ec3d080978a67837946a991843a081ea712539)
 2014-10-10 19:59:30 -07:00
Vinod Kumar Vavilapalli
bb6c79f76c YARN-2494. Added NodeLabels Manager internal API and implementation. Contributed by Wangda Tan. 
(cherry picked from commit db7f1653198b950e89567c06898d64f6b930a0ee)
 2014-10-10 11:46:10 -07:00
Zhijie Shen
16d7144337 YARN-2583. Modified AggregatedLogDeletionService to be able to delete rolling aggregated logs. Contributed by Xuan Gong. 
(cherry picked from commit cb81bac0029fce3a9726df3523f0b692cd3375b8)
 2014-10-10 00:18:24 -07:00
cnauroth
281041a57e YARN-2662. TestCgroupsLCEResourcesHandler leaks file descriptors. Contributed by Chris Nauroth. 
(cherry picked from commit d3afd730acfa380ab5032be5ee296c5d73744518)
(cherry picked from commit b81641a31050b6a5cf5db3af636125e0617ffd56)
 2014-10-09 22:49:00 -07:00
Zhijie Shen
647874206c YARN-2617. Fixed ApplicationSubmissionContext to still set resource for backward compatibility. Contributed by Wangda Tan. 
(cherry picked from commit e532ed8faa8db4b008a5b8d3f82b48a1b314fa6c)
 2014-10-09 18:01:13 -07:00
Vinod Kumar Vavilapalli
967c758373 YARN-2544. Added admin-API objects for using node-labels. Contributed by Wangda Tan. 
(cherry picked from commit 596702a02501e9cb09aabced168027189eaf02ba)
 2014-10-09 15:47:51 -07:00
Zhijie Shen
a4522e90f6 YARN-2629. Made the distributed shell use the domain-based timeline ACLs. Contributed by Zhijie Shen. 
(cherry picked from commit 1d4612f5ad9678c952b416e798dccd20c88f96ef)
 2014-10-09 13:03:03 -07:00
Mayank Bansal
cf0ad82c5a YARN-2598 GHS should show N/A instead of null for the inaccessible information (Zhijie Shen via mayank) 2014-10-09 11:39:25 -07:00
Vinod Kumar Vavilapalli
ad7baad21e YARN-2493. Added user-APIs for using node-labels. Contributed by Wangda Tan. 
(cherry picked from commit 180afa2f86f9854c536c3d4ff7476880e41ac48d)
 2014-10-09 11:27:49 -07:00
Steve Loughran
088ae9c5bf YARN-913 service registry: YARN-2652 add hadoop-yarn-registry package under hadoop-yarn 2014-10-08 12:54:37 -07:00
Jian He
7ec65c8617 YARN-2649. Fixed TestAMRMRPCNodeUpdates test failure. Contributed by Ming Ma 
(cherry picked from commit e16e25ab1beac89c8d8be4e9f2a7fbefe81d35f3)
 2014-10-08 11:03:28 -07:00
Jian He
5ff984f33f YARN-1857. CapacityScheduler headroom doesn't account for other AM's running. Contributed by Chen He and Craig Welch 
(cherry picked from commit 30d56fdbb40d06c4e267d6c314c8c767a7adc6a3)
 2014-10-07 13:48:46 -07:00
Jian He
efc482d1f7 YARN-2644. Fixed CapacityScheduler to return up-to-date headroom when AM allocates. Contributed by Craig Welch 
(cherry picked from commit 519e5a7dd2bd540105434ec3c8939b68f6c024f8)
 2014-10-06 15:52:05 -07:00
Jian He
73a1b77223 YARN-2615. Changed ClientToAMTokenIdentifier/RM(Timeline)DelegationTokenIdentifier to use protobuf as payload. Contributed by Junping Du 
(cherry picked from commit ea26cc0b4ac02b8af686dfda80f540e5aa70c358)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestClientToAMTokens.java
 2014-10-06 10:52:20 -07:00
Chris Douglas
0b2dedc42d YARN-1051. Add a system for creating reservations of cluster capacity. 
Contributed by Subru Krishnan and Carlo Curino.
(cherry picked from commit c8212bacb1b2a7e6ee83cc56f72297465ce99390)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
(cherry picked from commit ebf4aae60f7a641ac7406e27d43528cace20d4ff)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2014-10-06 10:30:04 -07:00
subru
114f09226e YARN-2611. Fixing jenkins findbugs warning and TestRMWebServicesCapacitySched for branch YARN-1051. Contributed by Subru Krishnan and Carlo Curino. 
(cherry picked from commit c47464aba407d1dafe10be23fe454f0489cc4367)
(cherry picked from commit a2986234be4e02f9ccb589f9ff5f7ffb28bc6400)
(cherry picked from commit fb5e9df7fd09a5cbc2c98bc9c875faf6a3b8f7c0)
 2014-10-06 10:29:14 -07:00