Commit Graph

1016 Commits

Author SHA1 Message Date
Jason Lowe 3e8544c5f2 YARN-2755. NM fails to clean up usercache_DEL_<timestamp> dirs after YARN-661. Contributed by Siqi Li
(cherry picked from commit 73e626ad91)
2014-10-30 15:11:57 +00:00
Karthik Kambatla d2ba115f06 YARN-2712. TestWorkPreservingRMRestart: Augment FS tests with queue and headroom checks. (Tsuyoshi Ozawa via kasha)
(cherry picked from commit 179cab81e0)
2014-10-30 00:39:34 -07:00
Karthik Kambatla e88832dfb3 YARN-2742. FairSchedulerConfiguration should allow extra spaces between value and unit. (Wei Yan via kasha)
(cherry picked from commit 782971ae7a)
2014-10-29 10:25:29 -07:00
Jian He 720de7eb4c YARN-2503. Added node lablels in web UI. Contributed by Wangda Tan
(cherry picked from commit d5e0a09721)
2014-10-28 18:00:59 -07:00
Zhijie Shen f40389ae08 YARN-2741. Made NM web UI serve logs on the drive other than C: on Windows. Contributed by Craig Welch.
(cherry picked from commit 8984e9b177)
2014-10-28 14:12:09 -07:00
Xuan 43b1878f86 YARN-2758. Update TestApplicationHistoryClientService to use the new generic history store. Contributed by Zhijie Shen 2014-10-28 13:30:55 -07:00
Xuan 43a53f2b65 YARN-2279. Add UTs to cover timeline server authentication. Contributed by Zhijie Shen 2014-10-28 11:18:35 -07:00
Vinod Kumar Vavilapalli 0ad33e1483 YARN-2704. Changed ResourceManager to optionally obtain tokens itself for the sake of localization and log-aggregation for long-running services. Contributed by Jian He.
(cherry picked from commit a16d022ca4)
2014-10-27 15:50:51 -07:00
Jian He 2d3c64943d YARN-2591. Fixed AHSWebServices to return FORBIDDEN(403) if the request user doesn't have access to the history data. Contributed by Zhijie Shen
(cherry picked from commit c05b581a55)
2014-10-27 10:46:37 -07:00
Xuan 3a77838535 YARN-2726. CapacityScheduler should explicitly log when an accessible label has no capacity. Contributed by Wangda Tan 2014-10-27 10:28:45 -07:00
Vinod Kumar Vavilapalli 36d3f1fcca YARN-2743. Fixed a bug in ResourceManager that was causing RMDelegationToken identifiers to be tampered and thus causing app submission failures in secure mode. Contributed by Jian He.
(cherry picked from commit 0186645505)
2014-10-26 11:15:45 -07:00
Karthik Kambatla acc6f622e0 YARN-2183. [YARN-1492] Cleaner service for cache manager. (Chris Trezzo and Sangjin Lee via kasha)
(cherry picked from commit c51e53d7aa)
2014-10-25 11:50:06 -07:00
Jian He 96a6e02d16 YARN-1915. Fixed a race condition that client could use the ClientToAMToken to contact with AM before AM actually receives the ClientToAMTokenMasterKey. Contributed by Jason Lowe
(cherry picked from commit 5864dd99a4)
2014-10-24 22:48:48 -07:00
Zhijie Shen 1b81105143 YARN-2703. Added logUploadedTime into LogValue for better display. Contributed by Xuan Gong.
(cherry picked from commit f81dc3f995)
2014-10-24 14:12:17 -07:00
Zhijie Shen 3b03ea6b50 YARN-2209. Replaced AM resync/shutdown command with corresponding exceptions and made related MR changes. Contributed by Jian He.
(cherry picked from commit 0f3b6900be)
2014-10-23 22:03:30 -07:00
Jian He 1c235a4448 YARN-2198. Remove the need to run NodeManager as privileged account for Windows Secure Container Executor. Contributed by Remus Rusanu
(cherry picked from commit 3b12fd6cfb)
2014-10-22 15:58:26 -07:00
Zhijie Shen 4b9de2cdfe YARN-2721. Suppress NodeExist exception thrown by ZKRMStateStore when it retries creating znode. Contributed by Jian He.
(cherry picked from commit 7e3b5e6f5c)
2014-10-21 21:52:01 -07:00
Vinod Kumar Vavilapalli 77dc9afe86 YARN-2715. Fixed ResourceManager to respect common configurations for proxy users/groups beyond just the YARN level config. Contributed by Zhijie Shen.
(cherry picked from commit c0e034336c)
2014-10-21 20:10:45 -07:00
cnauroth a066134277 YARN-2720. Windows: Wildcard classpath variables not expanded against resources contained in archives. Contributed by Craig Welch.
(cherry picked from commit 6637e3cf95)
2014-10-21 12:34:58 -07:00
Jason Lowe 3820bf055e YARN-90. NodeManager should identify failed disks becoming good again. Contributed by Varun Vasudev
(cherry picked from commit 6f2028bd15)
2014-10-21 17:33:34 +00:00
Jian He e9564e729f Missing file for YARN-2701
(cherry picked from commit 4fa1fb3193)
2014-10-20 19:58:21 -07:00
Jian He 3c8ae89050 YARN-2701. Potential race condition in startLocalizer when using LinuxContainerExecutor. Contributed by Xuan Gong
(cherry picked from commit 2839365f23)
2014-10-20 19:54:10 -07:00
Vinod Kumar Vavilapalli 9915d52185 YARN-2504. Enhanced RM Admin CLI to support management of node-labels. Contribyted by Wangda Tan.
(cherry picked from commit 8256766498)
2014-10-18 12:08:28 -07:00
Vinod Kumar Vavilapalli ea841eb25d YARN-2676. Enhanced Timeline auth-filter to support proxy users. Contributed by Zhijie Shen.
(cherry picked from commit 39063cd36f)
2014-10-17 22:03:43 -07:00
Vinod Kumar Vavilapalli 1c52b6551b YARN-2705. Fixed bugs in ResourceManager node-label manager that were causing test-failures: added a dummy in-memory labels-manager. Contributed by Wangda Tan.
(cherry picked from commit e9c66e8fd2)
2014-10-17 18:27:04 -07:00
Jian He 78a57087bf YARN-1879. Marked Idempotent/AtMostOnce annotations to ApplicationMasterProtocol for RM fail over. Contributed by Tsuyoshi OZAWA
(cherry picked from commit c3de2412eb)
2014-10-17 16:36:15 -07:00
Jian He 79428e021f YARN-2588. Standby RM fails to transitionToActive if previous transitionToActive failed with ZK exception. Contributed by Rohith Sharmaks
(cherry picked from commit a6aa6e42ca)
2014-10-17 10:55:04 -07:00
Vinod Kumar Vavilapalli 99ce4277a8 YARN-2699. Fixed a bug in CommonNodeLabelsManager that caused tests to fail when using ephemeral ports on NodeIDs. Contributed by Wangda Tan.
(cherry picked from commit abae63caf9)
2014-10-17 08:59:22 -07:00
Jian He a78f1f8da8 YARN-2682. Updated WindowsSecureContainerExecutor to not use DefaultContainerExecutor#getFirstApplicationDir and use getWorkingDir() instead. Contributed by Zhihai Xu
(cherry picked from commit 0fd0ebae64)
2014-10-16 18:15:43 -07:00
Jian He caba212f12 YARN-2621. Simplify the output when the user doesn't have the access for getDomain(s). Contributed by Zhijie Shen
(cherry picked from commit 233d446be1)
2014-10-16 16:38:00 -07:00
Vinod Kumar Vavilapalli 76cf2250de YARN-2685. Fixed a bug in CommonNodeLabelsManager that caused wrong resource tracking per label when a host runs multiple node-managers. Contributed by Wangda Tan.
(cherry picked from commit b3056c266a)
2014-10-15 18:48:36 -07:00
Vinod Kumar Vavilapalli e8e3a36213 YARN-2496. Enhanced Capacity Scheduler to have basic support for allocating resources based on node-labels. Contributed by Wangda Tan.
YARN-2500. Ehnaced ResourceManager to support schedulers allocating resources based on node-labels. Contributed by Wangda Tan.

(cherry picked from commit f2ea555ac6)
2014-10-15 18:34:19 -07:00
Jian He f93d2ea27e YARN-2312. Deprecated old ContainerId#getId API and updated MapReduce to use ContainerId#getContainerId instead. Contributed by Tsuyoshi OZAWA 2014-10-15 15:28:26 -07:00
Zhijie Shen bec165e163 YARN-2656. Made RM web services authentication filter support proxy user. Contributed by Varun Vasudev and Zhijie Shen.
(cherry picked from commit 1220bb72d4)
2014-10-14 21:56:44 -07:00
Zhijie Shen 8aeda20b25 HADOOP-11181. Generalized o.a.h.s.t.d.DelegationTokenManager to handle all sub-classes of AbstractDelegationTokenIdentifier. Contributed by Zhijie Shen.
(cherry picked from commit cdce88376a)
2014-10-14 11:37:28 -07:00
Karthik Kambatla 88455173e8 YARN-2566. DefaultContainerExecutor should pick a working directory randomly. (Zhihai Xu via kasha)
(cherry picked from commit cc93e7e683)
2014-10-13 16:32:42 -07:00
Karthik Kambatla a1116b56a4 YARN-2641. Decommission nodes on -refreshNodes instead of next NM-RM heartbeat. (Zhihai Xu via kasha)
(cherry picked from commit da709a2eac)
2014-10-13 16:24:00 -07:00
Jian He 536254f5e7 YARN-2308. Changed CapacityScheduler to explicitly throw exception if the queue
to which the apps were submitted is changed across RM restart. Contributed by Craig Welch & Chang Li
(cherry picked from commit f9680d9a16)
2014-10-13 14:16:12 -07:00
Jason Lowe 6b5741d423 YARN-2377. Localization exception stack traces are not passed as diagnostic info. Contributed by Gera Shegalov
(cherry picked from commit a56ea01002)
2014-10-13 18:32:38 +00:00
Zhijie Shen e51ae64761 YARN-2651. Spun off LogRollingInterval from LogAggregationContext. Contributed by Xuan Gong.
(cherry picked from commit 4aed2d8e91)
2014-10-13 10:55:09 -07:00
Vinod Kumar Vavilapalli 8ef83a9480 YARN-2494. Added NodeLabels Manager internal API and implementation. Contributed by Wangda Tan.
(cherry picked from commit db7f165319)
2014-10-10 11:45:15 -07:00
Zhijie Shen 1e6d81a886 YARN-2583. Modified AggregatedLogDeletionService to be able to delete rolling aggregated logs. Contributed by Xuan Gong.
(cherry picked from commit cb81bac002)
2014-10-10 00:16:34 -07:00
cnauroth b81641a310 YARN-2662. TestCgroupsLCEResourcesHandler leaks file descriptors. Contributed by Chris Nauroth.
(cherry picked from commit d3afd730ac)
2014-10-09 22:47:04 -07:00
Karthik Kambatla 7e20187f91 YARN-2180. [YARN-1492] In-memory backing store for cache manager. (Chris Trezzo via kasha)
(cherry picked from commit 4f426fe223)
2014-10-09 17:16:45 -07:00
Mayank Bansal c1a86d105e YARN-2598 GHS should show N/A instead of null for the inaccessible information (Zhijie Shen via mayank) 2014-10-09 13:43:36 -07:00
Jian He 5e6f86e3aa YARN-2649. Fixed TestAMRMRPCNodeUpdates test failure. Contributed by Ming Ma
(cherry picked from commit e16e25ab1b)
2014-10-08 11:03:00 -07:00
Jian He 9ff7c0c96c YARN-1857. CapacityScheduler headroom doesn't account for other AM's running. Contributed by Chen He and Craig Welch
(cherry picked from commit 30d56fdbb4)
2014-10-07 13:48:03 -07:00
Jian He 5c33e91229 YARN-2644. Fixed CapacityScheduler to return up-to-date headroom when AM allocates. Contributed by Craig Welch
(cherry picked from commit 519e5a7dd2)
2014-10-06 15:51:38 -07:00
Jian He 43358be60b YARN-2615. Changed ClientToAMTokenIdentifier/RM(Timeline)DelegationTokenIdentifier to use protobuf as payload. Contributed by Junping Du
(cherry picked from commit ea26cc0b4a)
2014-10-06 10:48:27 -07:00
Chris Douglas 4ba102bdc3 YARN-1051. Add a system for creating reservations of cluster capacity.
Contributed by Subru Krishnan and Carlo Curino.
(cherry picked from commit c8212bacb1)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
2014-10-06 10:30:53 -07:00