715c81ef6d
YARN-2788. Fixed backwards compatiblity issues with log-aggregation feature that were caused when adding log-upload-time via YARN-2703. Contributed by Xuan Gong.
...
(cherry picked from commit 58e9f24e0f
2014-11-03 13:19:34 -08:00
52c58115d2
YARN-2798. Fixed YarnClient to populate the renewer correctly for Timeline delegation tokens. Contributed by Zhijie Shen.
...
(cherry picked from commit 71fbb474f5
2014-11-03 12:50:39 -08:00
8091ea54d5
YARN-2730. DefaultContainerExecutor runs only one localizer at a time. Contributed by Siqi Li
...
(cherry picked from commit 6157ace547
2014-11-03 20:39:14 +00:00
caae0a0a08
YARN-2785. Fixed intermittent TestContainerResourceUsage failure. Contributed by Varun Vasudev.
...
(cherry picked from commit 27715ec63b
2014-11-02 15:22:54 -08:00
6627f67bf5
YARN-2790. Fixed a NodeManager bug that was causing log-aggregation to fail beyond HFDS delegation-token expiry even when RM is a proxy-user (YARN-2704). Contributed by Jian He.
...
(cherry picked from commit 5c0381c96a
2014-11-01 16:33:35 -07:00
d9ac25454c
YARN-2711. Fixed TestDefaultContainerExecutor#testContainerLaunchError failure on Windows. Contributed by Varun Vasudev.
...
(cherry picked from commit 1cd088fd9d
2014-10-31 17:45:05 -07:00
ada40aa933
YARN-2707. Potential null dereference in FSDownload. Contributed by Gera Shegalov
...
(cherry picked from commit 7bc179f9f5
2014-10-31 22:54:49 +00:00
6d4a04a033
YARN-2789. Re-instated the NodeReport.newInstance private unstable API modified in YARN-2698 so that tests in YARN frameworks don't break. Contributed by Wangda Tan.
...
(cherry picked from commit 6ce32f593b
2014-10-31 15:33:41 -07:00
a87f827ae4
YARN-2770. Added functionality to renew/cancel TimeLineDelegationToken. Contributed by Zhijie Shen
...
(cherry picked from commit 1b4be91866
2014-10-31 13:17:19 -07:00
268af259b5
YARN-2186. [YARN-1492] Node Manager uploader service for cache manager. (Chris Trezzo and Sangjin Lee via kasha)
...
(cherry picked from commit 256697acd5
2014-10-31 13:01:25 -07:00
9b879d713a
YARN-2778. Moved node-lables' reports to the yarn nodes CLI from the admin CLI. Contributed by Wangda Tan.
...
(cherry picked from commit b6c1188b85
2014-10-31 10:19:14 -07:00
97694e78ad
YARN-2779. Fixed ResourceManager to not require delegation tokens for communicating with Timeline Service. Contributed by Zhijie Shen.
...
(cherry picked from commit d1828d9443
2014-10-30 23:17:16 -07:00
0ba8d8c19c
YARN-2698. Moved some node label APIs to be correctly placed in client protocol. Contributed by Wangda Tan.
...
(cherry picked from commit e0233c16eb
2014-10-30 23:02:06 -07:00
cd24075343
YARN-2677 registry punycoding of usernames doesn't fix all usernames to be DNS-valid (stevel)
2014-10-30 22:32:20 +00:00
3e8544c5f2
YARN-2755. NM fails to clean up usercache_DEL_<timestamp> dirs after YARN-661. Contributed by Siqi Li
...
(cherry picked from commit 73e626ad91
2014-10-30 15:11:57 +00:00
d2ba115f06
YARN-2712. TestWorkPreservingRMRestart: Augment FS tests with queue and headroom checks. (Tsuyoshi Ozawa via kasha)
...
(cherry picked from commit 179cab81e0
2014-10-30 00:39:34 -07:00
d2a5c386c1
YARN-2769. Fixed the problem that timeline domain is not set in distributed shell AM when using shell_command on Windows. Contributed by Varun Vasudev.
...
(cherry picked from commit 72496311def320987c31822776e8fbb73dfcc726)
2014-10-29 12:14:48 -07:00
e88832dfb3
YARN-2742. FairSchedulerConfiguration should allow extra spaces between value and unit. (Wei Yan via kasha)
...
(cherry picked from commit 782971ae7a
2014-10-29 10:25:29 -07:00
734cbe7de2
YARN-2747. Fixed the test failure of TestAggregatedLogFormat when native I/O is enabled. Contributed by Xuan Gong.
...
(cherry picked from commit ec63a3ffbd
2014-10-28 22:45:20 -07:00
720de7eb4c
YARN-2503. Added node lablels in web UI. Contributed by Wangda Tan
...
(cherry picked from commit d5e0a09721
2014-10-28 18:00:59 -07:00
f40389ae08
YARN-2741. Made NM web UI serve logs on the drive other than C: on Windows. Contributed by Craig Welch.
...
(cherry picked from commit 8984e9b177
2014-10-28 14:12:09 -07:00
43b1878f86
YARN-2758. Update TestApplicationHistoryClientService to use the new generic history store. Contributed by Zhijie Shen
2014-10-28 13:30:55 -07:00
43a53f2b65
YARN-2279. Add UTs to cover timeline server authentication. Contributed by Zhijie Shen
2014-10-28 11:18:35 -07:00
3f7edeb055
YARN-2760. Remove 'experimental' from FairScheduler docs. (Harsh J via kasha)
...
(cherry picked from commit ade3727ecb
2014-10-28 08:49:34 -07:00
a1c7a707ca
YARN-2502. Changed DistributedShell to support node labels. Contributed by Wangda Tan
...
(cherry picked from commit f6b963fdfc
2014-10-27 20:13:53 -07:00
0ad33e1483
YARN-2704. Changed ResourceManager to optionally obtain tokens itself for the sake of localization and log-aggregation for long-running services. Contributed by Jian He.
...
(cherry picked from commit a16d022ca4
2014-10-27 15:50:51 -07:00
2d3c64943d
YARN-2591. Fixed AHSWebServices to return FORBIDDEN(403) if the request user doesn't have access to the history data. Contributed by Zhijie Shen
...
(cherry picked from commit c05b581a55
2014-10-27 10:46:37 -07:00
3a77838535
YARN-2726. CapacityScheduler should explicitly log when an accessible label has no capacity. Contributed by Wangda Tan
2014-10-27 10:28:45 -07:00
cb3974524c
YARN-2734. Skipped sub-folders in the local log dir when aggregating logs. Contributed by Xuan Gong.
...
(cherry picked from commit caecd9fffe
2014-10-26 12:59:29 -07:00
36d3f1fcca
YARN-2743. Fixed a bug in ResourceManager that was causing RMDelegationToken identifiers to be tampered and thus causing app submission failures in secure mode. Contributed by Jian He.
...
(cherry picked from commit 0186645505
2014-10-26 11:15:45 -07:00
f862999a7e
YARN-2723. Fix rmadmin -replaceLabelsOnNode does not correctly parse
...
port. Contributed by Naganarasimha G R
2014-10-25 12:47:09 -07:00
acc6f622e0
YARN-2183. [YARN-1492] Cleaner service for cache manager. (Chris Trezzo and Sangjin Lee via kasha)
...
(cherry picked from commit c51e53d7aa
2014-10-25 11:50:06 -07:00
9bd149978d
YARN-2314. Disable ContainerManagementProtocolProxy cache by default to prevent creating thousands of threads in a large cluster. Contributed by Jason Lowe
...
(cherry picked from commit f44cf99599
2014-10-24 23:06:00 -07:00
96a6e02d16
YARN-1915. Fixed a race condition that client could use the ClientToAMToken to contact with AM before AM actually receives the ClientToAMTokenMasterKey. Contributed by Jason Lowe
...
(cherry picked from commit 5864dd99a4
2014-10-24 22:48:48 -07:00
9af3cfc7eb
YARN-2713. "RM Home" link in NM should point to one of the RMs in an HA setup. (kasha)
...
(cherry picked from commit 683897fd02
2014-10-24 15:00:37 -07:00
1b81105143
YARN-2703. Added logUploadedTime into LogValue for better display. Contributed by Xuan Gong.
...
(cherry picked from commit f81dc3f995
2014-10-24 14:12:17 -07:00
13c60bac57
YARN-2724. Skipped uploading a local log file to HDFS if exception is raised when opening it. Contributed by Xuan Gong.
...
(cherry picked from commit e31f0a6558
2014-10-24 11:15:19 -07:00
d57f2b9dea
Updated CHANGES.txt to move YARN-2682 to branch-2.6
...
(cherry picked from commit 071c925c7d
2014-10-23 22:28:23 -07:00
3b03ea6b50
YARN-2209. Replaced AM resync/shutdown command with corresponding exceptions and made related MR changes. Contributed by Jian He.
...
(cherry picked from commit 0f3b6900be
2014-10-23 22:03:30 -07:00
36515435ff
YARN-2732. Fixed syntax error in SecureContainer.apt.vm. Contributed by Jian He.
...
(cherry picked from commit b94b8b30f2
2014-10-22 18:41:05 -07:00
1c235a4448
YARN-2198. Remove the need to run NodeManager as privileged account for Windows Secure Container Executor. Contributed by Remus Rusanu
...
(cherry picked from commit 3b12fd6cfb
2014-10-22 15:58:26 -07:00
5e6494ea95
YARN-2700 TestSecureRMRegistryOperations failing on windows: auth problems
2014-10-22 21:13:32 +01:00
a64ac5501c
YARN-2692 ktutil test hanging on some machines/ktutil versions (stevel)
2014-10-22 14:35:13 +01:00
4b9de2cdfe
YARN-2721. Suppress NodeExist exception thrown by ZKRMStateStore when it retries creating znode. Contributed by Jian He.
...
(cherry picked from commit 7e3b5e6f5c
2014-10-21 21:52:01 -07:00
77dc9afe86
YARN-2715. Fixed ResourceManager to respect common configurations for proxy users/groups beyond just the YARN level config. Contributed by Zhijie Shen.
...
(cherry picked from commit c0e034336c
2014-10-21 20:10:45 -07:00
296bbd8971
YARN-2709. Made timeline client getDelegationToken API retry if ConnectException happens. Contributed by Li Lu.
...
(cherry picked from commit b2942762d7
2014-10-21 16:08:18 -07:00
a066134277
YARN-2720. Windows: Wildcard classpath variables not expanded against resources contained in archives. Contributed by Craig Welch.
...
(cherry picked from commit 6637e3cf95
2014-10-21 12:34:58 -07:00
3820bf055e
YARN-90. NodeManager should identify failed disks becoming good again. Contributed by Varun Vasudev
...
(cherry picked from commit 6f2028bd15
2014-10-21 17:33:34 +00:00
fd7ba56f6a
YARN-2717. Avoided duplicate logging when container logs are not found. Contributed by Xuan Gong.
...
(cherry picked from commit 171f2376d2
2014-10-20 22:23:12 -07:00
3c8ae89050
YARN-2701. Potential race condition in startLocalizer when using LinuxContainerExecutor. Contributed by Xuan Gong
...
(cherry picked from commit 2839365f23
2014-10-20 19:54:10 -07:00
f75570734b
YARN-2582. Fixed Log CLI and Web UI for showing aggregated logs of LRS. Contributed Xuan Gong.
...
(cherry picked from commit e90718fa5a
2014-10-20 13:42:22 -07:00
42ceed8f20
YARN-2673. Made timeline client put APIs retry if ConnectException happens. Contributed by Li Lu.
...
(cherry picked from commit 89427419a3
2014-10-20 12:23:10 -07:00
9915d52185
YARN-2504. Enhanced RM Admin CLI to support management of node-labels. Contribyted by Wangda Tan.
...
(cherry picked from commit 8256766498
2014-10-18 12:08:28 -07:00
ea841eb25d
YARN-2676. Enhanced Timeline auth-filter to support proxy users. Contributed by Zhijie Shen.
...
(cherry picked from commit 39063cd36f
2014-10-17 22:03:43 -07:00
1c52b6551b
YARN-2705. Fixed bugs in ResourceManager node-label manager that were causing test-failures: added a dummy in-memory labels-manager. Contributed by Wangda Tan.
...
(cherry picked from commit e9c66e8fd2
2014-10-17 18:27:04 -07:00
78a57087bf
YARN-1879. Marked Idempotent/AtMostOnce annotations to ApplicationMasterProtocol for RM fail over. Contributed by Tsuyoshi OZAWA
...
(cherry picked from commit c3de2412eb
2014-10-17 16:36:15 -07:00
79428e021f
YARN-2588. Standby RM fails to transitionToActive if previous transitionToActive failed with ZK exception. Contributed by Rohith Sharmaks
...
(cherry picked from commit a6aa6e42ca
2014-10-17 10:55:04 -07:00
99ce4277a8
YARN-2699. Fixed a bug in CommonNodeLabelsManager that caused tests to fail when using ephemeral ports on NodeIDs. Contributed by Wangda Tan.
...
(cherry picked from commit abae63caf9
2014-10-17 08:59:22 -07:00
10110b6eae
Moved YARN-2682 from 2.6 to 2.7 in CHANGES.txt
...
(cherry picked from commit 72093fd8cb
2014-10-16 19:03:29 -07:00
a78f1f8da8
YARN-2682. Updated WindowsSecureContainerExecutor to not use DefaultContainerExecutor#getFirstApplicationDir and use getWorkingDir() instead. Contributed by Zhihai Xu
...
(cherry picked from commit 0fd0ebae64
2014-10-16 18:15:43 -07:00
caba212f12
YARN-2621. Simplify the output when the user doesn't have the access for getDomain(s). Contributed by Zhijie Shen
...
(cherry picked from commit 233d446be1
2014-10-16 16:38:00 -07:00
fddbf52caa
YARN-2689 TestSecureRMRegistryOperations failing on windows: secure ZK won't start
2014-10-16 14:21:49 -07:00
76cf2250de
YARN-2685. Fixed a bug in CommonNodeLabelsManager that caused wrong resource tracking per label when a host runs multiple node-managers. Contributed by Wangda Tan.
...
(cherry picked from commit b3056c266a
2014-10-15 18:48:36 -07:00
e8e3a36213
YARN-2496. Enhanced Capacity Scheduler to have basic support for allocating resources based on node-labels. Contributed by Wangda Tan.
...
YARN-2500. Ehnaced ResourceManager to support schedulers allocating resources based on node-labels. Contributed by Wangda Tan.
(cherry picked from commit f2ea555ac6
2014-10-15 18:34:19 -07:00
f93d2ea27e
YARN-2312. Deprecated old ContainerId#getId API and updated MapReduce to use ContainerId#getContainerId instead. Contributed by Tsuyoshi OZAWA
2014-10-15 15:28:26 -07:00
bec165e163
YARN-2656. Made RM web services authentication filter support proxy user. Contributed by Varun Vasudev and Zhijie Shen.
...
(cherry picked from commit 1220bb72d4
2014-10-14 21:56:44 -07:00
88455173e8
YARN-2566. DefaultContainerExecutor should pick a working directory randomly. (Zhihai Xu via kasha)
...
(cherry picked from commit cc93e7e683
2014-10-13 16:32:42 -07:00
a1116b56a4
YARN-2641. Decommission nodes on -refreshNodes instead of next NM-RM heartbeat. (Zhihai Xu via kasha)
...
(cherry picked from commit da709a2eac
2014-10-13 16:24:00 -07:00
31a4bf7321
Missing Changes.txt for YARN-2308
...
(cherry picked from commit 178bc505da
2014-10-13 14:20:18 -07:00
6b5741d423
YARN-2377. Localization exception stack traces are not passed as diagnostic info. Contributed by Gera Shegalov
...
(cherry picked from commit a56ea01002
2014-10-13 18:32:38 +00:00
e51ae64761
YARN-2651. Spun off LogRollingInterval from LogAggregationContext. Contributed by Xuan Gong.
...
(cherry picked from commit 4aed2d8e91
2014-10-13 10:55:09 -07:00
9ed9ea8c52
YARN-2667. Fix the release audit warning caused by hadoop-yarn-registry. Contributed by Yi Liu
...
(cherry picked from commit 344a10ad5e
2014-10-13 15:40:16 +00:00
38dedc2f07
YARN-2668 yarn-registry JAR won't link against ZK 3.4.5. (stevel)
2014-10-11 18:33:29 -07:00
fff6a1f77e
YARN-2501. Enhanced AMRMClient library to support requests against node labels. Contributed by Wangda Tan.
...
(cherry picked from commit a5ec3d0809
2014-10-10 19:58:42 -07:00
8ef83a9480
YARN-2494. Added NodeLabels Manager internal API and implementation. Contributed by Wangda Tan.
...
(cherry picked from commit db7f165319
2014-10-10 11:45:15 -07:00
1e6d81a886
YARN-2583. Modified AggregatedLogDeletionService to be able to delete rolling aggregated logs. Contributed by Xuan Gong.
...
(cherry picked from commit cb81bac002
2014-10-10 00:16:34 -07:00
b81641a310
YARN-2662. TestCgroupsLCEResourcesHandler leaks file descriptors. Contributed by Chris Nauroth.
...
(cherry picked from commit d3afd730ac
2014-10-09 22:47:04 -07:00
79204dced9
YARN-2671. Fix the Jira number in the change log.
...
(cherry picked from commit 5b12df6587
2014-10-09 20:59:37 -07:00
ad47a27dbf
YARN-2617. Fixed ApplicationSubmissionContext to still set resource for backward compatibility. Contributed by Wangda Tan.
...
(cherry picked from commit e532ed8faa
2014-10-09 18:00:01 -07:00
7e20187f91
YARN-2180. [YARN-1492] In-memory backing store for cache manager. (Chris Trezzo via kasha)
...
(cherry picked from commit 4f426fe223
2014-10-09 17:16:45 -07:00
d3125f3de5
YARN-2544. Added admin-API objects for using node-labels. Contributed by Wangda Tan.
...
(cherry picked from commit 596702a025
2014-10-09 15:47:20 -07:00
c1a86d105e
YARN-2598 GHS should show N/A instead of null for the inaccessible information (Zhijie Shen via mayank)
2014-10-09 13:43:36 -07:00
4d170d3db6
YARN-2629. Made the distributed shell use the domain-based timeline ACLs. Contributed by Zhijie Shen.
...
(cherry picked from commit 1d4612f5ad
2014-10-09 13:01:44 -07:00
b42cd54d7a
YARN-2493. Added user-APIs for using node-labels. Contributed by Wangda Tan.
...
(cherry picked from commit 180afa2f86
2014-10-09 11:27:17 -07:00
d218ab58fc
YARN-913 service registry: YARN-2652 add hadoop-yarn-registry package under hadoop-yarn
2014-10-08 12:55:47 -07:00
5e6f86e3aa
YARN-2649. Fixed TestAMRMRPCNodeUpdates test failure. Contributed by Ming Ma
...
(cherry picked from commit e16e25ab1b
2014-10-08 11:03:00 -07:00
9ff7c0c96c
YARN-1857. CapacityScheduler headroom doesn't account for other AM's running. Contributed by Chen He and Craig Welch
...
(cherry picked from commit 30d56fdbb4
2014-10-07 13:48:03 -07:00
5c33e91229
YARN-2644. Fixed CapacityScheduler to return up-to-date headroom when AM allocates. Contributed by Craig Welch
...
(cherry picked from commit 519e5a7dd2
2014-10-06 15:51:38 -07:00
43358be60b
YARN-2615. Changed ClientToAMTokenIdentifier/RM(Timeline)DelegationTokenIdentifier to use protobuf as payload. Contributed by Junping Du
...
(cherry picked from commit ea26cc0b4a
2014-10-06 10:48:27 -07:00
4ba102bdc3
YARN-1051. Add a system for creating reservations of cluster capacity.
...
Contributed by Subru Krishnan and Carlo Curino.
(cherry picked from commit c8212bacb1
2014-10-06 10:30:53 -07:00
7ed61e150c
YARN-2468. Enhanced NodeManager to support log handling APIs (YARN-2569) for use by long running services. Contributed by Xuan Gong.
...
(cherry picked from commit 34cdcaad71
2014-10-03 12:17:03 -07:00
b6ce0a1f69
YARN-2635. TestRM, TestRMRestart, TestClientToAMTokens should run with both CS and FS. (Wei Yan and kasha via kasha)
...
(cherry picked from commit 80d11eb68e
2014-10-03 11:52:05 -07:00
1c58ab9206
YARN-2562. Changed ContainerId#toString() to be more readable. Contributed by Tsuyoshi OZAWA
...
(cherry picked from commit eb6ce5e97c
2014-10-03 11:04:46 -07:00
6f4c77409b
YARN-2628. Capacity scheduler with DominantResourceCalculator carries out reservation even though slots are free. Contributed by Varun Vasudev
...
(cherry picked from commit 054f285526
2014-10-02 15:16:07 -07:00
ecaae67ab3
YARN-2527. Fixed the potential NPE in ApplicationACLsManager and added test cases for it. Contributed by Benoy Antony.
...
(cherry picked from commit 1c93025a1b
2014-10-02 14:57:27 -07:00
531c1fd00a
YARN-2624. Resource Localization fails on a cluster due to existing cache directories. Contributed by Anubhav Dhoot
...
(cherry picked from commit 29f520052e
2014-10-02 17:40:44 +00:00
b6f4d1ef42
YARN-2254. TestRMWebServicesAppsModification should run against both CS and FS. (Zhihai Xu via kasha)
...
(cherry picked from commit 5e0b49da9c
2014-10-02 10:15:41 -07:00
61c7ceaf82
YARN-2617. Fixed NM to not send duplicate container status whose app is not running. Contributed by Jun Gong
...
(cherry picked from commit 3ef1cf187f
2014-10-02 10:04:42 -07:00
6483342a61
YARN-1979. TestDirectoryCollection fails when the umask is unusual. (Contributed by Vinod Kumar Vavilapalli and Tsuyoshi OZAWA)
...
(cherry picked from commit c7cee9b455
2014-10-02 08:04:25 -07:00
34dd2d1d1c
YARN-2446. Augmented Timeline service APIs to start taking in domains as a parameter while posting entities and events. Contributed by Zhijie Shen.
...
(cherry picked from commit 9e40de6af7
2014-10-01 17:54:12 -07:00
Vinod Kumar Vavilapalli
Jason Lowe
Zhijie Shen
Jian He
Karthik Kambatla
Steve Loughran
Xuan
cnauroth
Mayank Bansal
Chris Douglas
junping_du