Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
7,894 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

1605 Commits

Author SHA1 Message Date
Karthik Kambatla 6e954bc25c YARN-2910. FSLeafQueue can throw ConcurrentModificationException. (Wilfred Spiegelenburg via kasha) 
(cherry picked from commit a2e07a5456)
(cherry picked from commit 1986ea8dd2)
(cherry picked from commit 2b827a18d7b4eb41dc0095ea7277239273e7e396)
 2015-08-30 13:44:52 -07:00
Karthik Kambatla 7d686eccc3 YARN-2874. Dead lock in DelegationTokenRenewer which blocks RM to execute any further apps. (Naganarasimha G R via kasha) 
(cherry picked from commit 799353e2c7)

(cherry picked from commit 25be97808b)
(cherry picked from commit d82bf536d44c6e7ba06a01105545b3979b731d80)
 2015-08-30 13:25:37 -07:00
Jian He 3600f30c35 YARN-2894. Fixed a bug regarding application view acl when RM fails over. Contributed by Rohith Sharmaks 
(cherry picked from commit 392c3aaea8)

(cherry picked from commit d6f3d4893d)
(cherry picked from commit 61efbc1cba0c4a81b8aafb1d45c2f7b3cf7857d8)
 2015-08-30 13:08:42 -07:00
Zhijie Shen 8a47d1aa55 YARN-2890. MiniYARNCluster should start the timeline server based on the configuration. Contributed by Mit Desai. 
(cherry picked from commit 51af8d367d)
(cherry picked from commit d21ef79707)
(cherry picked from commit db325c053c6ef3ee8731d7273d1d92da7e5deee7)
 2015-08-27 19:47:43 -07:00
Jason Lowe 2f0a34b5ba YARN-2905. AggregatedLogsBlock page can infinitely loop if the aggregated log file is corrupted. Contributed by Varun Saxena 
(cherry picked from commit 0f9528b99a)

(cherry picked from commit 38ea1419f6)
(cherry picked from commit 3877166754956bf66a8b1c81440dba2d279a1e03)
 2015-08-27 19:27:24 -07:00
Jian He 888ab4a6e7 YARN-2906. CapacitySchedulerPage shows HTML tags for a queue's Active Users. Contributed by Jason Lowe 
(cherry picked from commit 8a7ca13b13)

(cherry picked from commit ae35b0e14d)
(cherry picked from commit 65acee3e19a147e5c5a8688319ab75357bdf51b5)
 2015-08-27 19:10:43 -07:00
Jian He 7f97189bcf YARN-2865. Fixed RM to always create a new RMContext when transtions from StandBy to Active. Contributed by Rohith Sharmaks 
(cherry picked from commit 9cb8b75ba5)

(cherry picked from commit db31ef7e7f)
(cherry picked from commit e669974ae94c03914c9181a4481b4879fd4acc0d)
 2015-08-27 18:56:22 -07:00
Jason Lowe f307b426f3 YARN-2414. RM web UI: app page will crash if app is failed before any attempt has been created. Contributed by Wangda Tan 
(cherry picked from commit 81c9d17af8)

(cherry picked from commit 242fd0e39a)
(cherry picked from commit a9d5acd898b34e1050a78f2d70ed62fdb82948a6)
 2015-08-27 18:36:50 -07:00
Jason Lowe f83d898944 YARN-2816. NM fail to start with NPE during container recovery. Contributed by Zhihai Xu 
(cherry picked from commit 49c38898b0)

(cherry picked from commit ad140d1fc8)
(cherry picked from commit 85b23c323c80c5303bd0b7bdb066258792ca67d8)
 2015-08-27 18:32:59 -07:00
Jian He 81ba30211e YARN-2856. Fixed RMAppImpl to handle ATTEMPT_KILLED event at ACCEPTED state on app recovery. Contributed by Rohith Sharmaks 
(cherry picked from commit d005404ef7)

(cherry picked from commit beb184ac58)
(cherry picked from commit 325bb33988743d60cb333002f9da60314241632e)
 2015-08-27 18:29:39 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) 4239513ec0 Adding release 2.6.1 to CHANGES.txt 
(cherry picked from commit 7c796fd12c)
 2015-08-11 13:23:27 -07:00
Wangda Tan 881084fe5c YARN-3251. Fixed a deadlock in CapacityScheduler when computing absoluteMaxAvailableCapacity in LeafQueue (Craig Welch via wangda) 2015-02-26 17:05:25 -08:00
Tsuyoshi Ozawa 07b3329e3f Revert "CHANGES.txt: add YARN-2977" 
This reverts commit 3419091631.
 2014-12-21 12:47:30 +09:00
Tsuyoshi Ozawa ca121a78fd Revert "YARN-2977. Fixed intermittent TestNMClient failure. (Contributed by Junping Du)" 
This reverts commit d79f63120e.
 2014-12-21 12:47:28 +09:00
Tsuyoshi Ozawa 3419091631 CHANGES.txt: add YARN-2977 
(cherry picked from commit 76b0370a27)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2014-12-20 23:11:04 +09:00
Tsuyoshi Ozawa d79f63120e YARN-2977. Fixed intermittent TestNMClient failure. (Contributed by Junping Du) 
(cherry picked from commit cf7fe583d1)
 2014-12-20 23:09:58 +09:00
Karthik Kambatla 5913c5143b Update release date for 2.5.2 
(cherry picked from commit 1e9a3f42ff)
 2014-11-21 14:35:06 -08:00
Jonathan Eagles a9b76813a2 YARN-2375. Allow enabling/disabling timeline server per framework. (Mit Desai via jeagles) 
(cherry picked from commit c298a9a845)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2014-11-20 23:54:59 -06:00
Arun C. Murthy 98a4538cf4 Preparing to release hadoop-2.6.0-rc1. 
(cherry picked from commit 9ff7315cbe)

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-mapreduce-project/CHANGES.txt
	hadoop-yarn-project/CHANGES.txt
 2014-11-13 13:05:50 -08:00
Arun C. Murthy 4e9b46da11 YARN-1964. Adding changelog back in. 2014-11-13 11:39:35 -08:00
Arun C. Murthy 32032ace21 YARN-2853. Merging to branch-2.6 for hadoop-2.6.0-rc1. 
(cherry picked from commit d648e60eba)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2014-11-13 10:40:33 -08:00
Vinod Kumar Vavilapalli 56020955fd YARN-2853. Fixed a bug in ResourceManager causing apps to hang when the user kill request races with ApplicationMaster finish. Contributed by Jian He. 
(cherry picked from commit 3651fe1b08)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2014-11-13 10:40:09 -08:00
Arun C. Murthy f8892660a8 YARN-2635. Merging to branch-2.6 for hadoop-2.6.0-rc1. 
(cherry picked from commit 81dc0ac6dc)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2014-11-13 10:36:58 -08:00
Karthik Kambatla b579c3d405 YARN-2635. TestRM, TestRMRestart, TestClientToAMTokens should run with both CS and FS. (Wei Yan and kasha via kasha) 
(cherry picked from commit 80d11eb68e)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestClientToAMTokens.java
 2014-11-13 10:36:13 -08:00
Jason Lowe 685790e027 YARN-2846. Incorrect persist exit code for running containers in reacquireContainer() that interrupted by NodeManager restart. Contributed by Junping Du 
(cherry picked from commit 33ea5ae92b)
 2014-11-13 16:12:30 +00:00
Zhijie Shen 1535e7211e YARN-2794. Fixed log messages about distributing system-credentials. Contributed by Jian He. 
(cherry picked from commit be7bf956e9)
 2014-11-12 11:10:47 -08:00
Arun C. Murthy de0073cbc4 YARN-2843. Merging to branch-2.6 for hadoop-2.6.0-rc1. 
(cherry picked from commit 51957ca818)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2014-11-12 10:44:43 -08:00
Vinod Kumar Vavilapalli 7a8a413eb2 YARN-2843. Fixed NodeLabelsManager to trim inputs for hosts and labels so as to make them work correctly. Contributed by Wangda Tan. 
(cherry picked from commit 0fd97f9c19)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2014-11-12 10:44:08 -08:00
Xuan a690629ffc YARN-2841. RMProxy should retry EOFException. Contributed by Jian He 2014-11-12 10:28:55 -08:00
Ravi Prakash 9613a57e83 YARN-1964. Create Docker analog of the LinuxContainerExecutor in YARN 2014-11-12 09:33:42 -08:00
Karthik Kambatla ba01844260 Set the release date for 2.5.2 
(cherry picked from commit 68a0508728)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

(cherry picked from commit 92de44c2c5)
 2014-11-10 15:24:14 -08:00
Arun C. Murthy 60584c732f Preparing to release hadoop-2.6.0: Set version in branch-2.6 to 2.6.1-SNAPSHOT. 2014-11-09 19:21:11 -08:00
Vinod Kumar Vavilapalli 6a9534e9cf YARN-2834. Fixed ResourceManager to ignore token-renewal failures on recovery consistent with the (somewhat incorrect) behaviour in the non-recovery case. Contributed by Jian He. 
Fixed a minor import issue in the test during cherry-pick from trunk.

(cherry picked from commit e76faebc95)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestWorkPreservingRMRestart.java
 2014-11-09 19:09:48 -08:00
Arun C. Murthy be31db82f7 Updated CHANGES.txt to reflect hadoop-2.6.0 release dates. 
(cherry picked from commit 770cc14442)
 2014-11-09 18:53:55 -08:00
Karthik Kambatla e9f7079c58 Fix broken download of directories 
(cherry picked from commit 745c9a0124)
 2014-11-09 18:38:57 -08:00
Arun C. Murthy c2cc879f01 YARN-2830. Add backwords compatible ContainerId.newInstance constructor. Contributed by Jonathan Eagles. 
(cherry picked from commit 43cd07b408)
 2014-11-09 15:04:55 -08:00
Vinod Kumar Vavilapalli f098b261fe YARN-2607. Fixed issues in TestDistributedShell. Contributed by Wangda Tan. 
(cherry picked from commit 737d9284c1)
 2014-11-08 11:02:55 -08:00
Zhijie Shen 33da0f8ecf YARN-2505. Supported get/add/remove/change labels in RM REST API. Contributed by Craig Welch. 
(cherry picked from commit 9a4e0d343e)
 2014-11-07 20:40:56 -08:00
Vinod Kumar Vavilapalli 6ff765bea2 YARN-2826. Fixed user-groups mappings' refresh bug caused by YARN-2826. Contributed by Wangda Tan. 
(cherry picked from commit df36edf751)
 2014-11-07 19:45:23 -08:00
Xuan 629274c09b YARN-2819. NPE in ATS Timeline Domains when upgrading from 2.4 to 2.6. Contributed by Zhijie Shen 
(cherry picked from commit 4a114dd67a)
(cherry picked from commit 7d26cffb0c)
 2014-11-07 16:13:38 -08:00
Jason Lowe dcc1ab15ab YARN-2632. Document NM Restart feature. Contributed by Junping Du and Vinod Kumar Vavilapalli 
(cherry picked from commit 1e215e8ba2)
 2014-11-07 23:42:26 +00:00
Jason Lowe e5c0fa2c95 YARN-2825. Container leak on NM. Contributed by Jian He 
(cherry picked from commit c3d475070a)
 2014-11-07 23:17:49 +00:00
Vinod Kumar Vavilapalli 6593aaf117 YARN-2753. Fixed a bunch of bugs in the NodeLabelsManager classes. Contributed by Zhihai xu. 
(cherry picked from commit 4cfd5bc7c1)
 2014-11-07 14:17:23 -08:00
cnauroth 9adb31b6a8 YARN-2803. MR distributed cache not working correctly on Windows after NodeManager privileged account changes. Contributed by Craig Welch. 
(cherry picked from commit 06b797947c)
(cherry picked from commit c16f718293)
 2014-11-07 12:45:25 -08:00
Vinod Kumar Vavilapalli 326c722eb7 YARN-2827. Fixed bugs in "yarn queue" CLI. Contributed by Wangda Tan. 
(cherry picked from commit 7b19079657)
 2014-11-07 12:01:33 -08:00
Vinod Kumar Vavilapalli 71a18a5303 YARN-2824. Fixed Capacity Scheduler to not crash when some node-labels are not mapped to queues by making default capacities per label to be zero. Contributed by Wangda Tan. 
(cherry picked from commit 2ac1be7dec)
 2014-11-07 10:45:01 -08:00
Xuan 09955ea2c3 YARN-2810. TestRMProxyUsersConf fails on Windows VMs. Contributed by Varun Vasudev 
(cherry picked from commit 1e97f2f094)
(cherry picked from commit bf79541868)
 2014-11-07 09:46:53 -08:00
Vinod Kumar Vavilapalli 9f7396be55 YARN-2823. Fixed ResourceManager app-attempt state machine to inform schedulers about previous finished attempts of a running appliation to avoid expectation mismatch w.r.t transferred containers. Contributed by Jian He. 
(cherry picked from commit a5657182a7)
 2014-11-07 09:30:31 -08:00
Vinod Kumar Vavilapalli 8c3a0d5a98 YARN-2647. Added a queue CLI for getting queue information. Contributed by Sunil Govind. 
(cherry picked from commit 61effcbebc)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/bin/yarn

(cherry picked from commit b1521c7c3c)
 2014-11-06 22:22:16 -08:00
Vinod Kumar Vavilapalli 21ef5afafa YARN-2744. Fixed CapacityScheduler to validate node-labels correctly against queues. Contributed by Wangda Tan. 
(cherry picked from commit a3839a9fbf)
 2014-11-06 17:29:39 -08:00
Steve Loughran e333584ca6 YARN-2768 Improved Yarn Registry service record structure (stevel) 2014-11-06 20:21:49 +00:00
Vinod Kumar Vavilapalli b557f689b4 YARN-2818. Removed the now unnecessary user entity injection from Timeline service given we now have domains. Contributed by Zhijie Shen. 
(cherry picked from commit f5b19bed7d)
 2014-11-06 11:50:12 -08:00
Xuan 43854764a9 YARN-2812. TestApplicationHistoryServer is likely to fail on less powerful machine. Contributed by Zhijie Shen 
(cherry picked from commit b0b52c4e11)
(cherry picked from commit 4aa98d5991)
 2014-11-05 20:45:09 -08:00
Xuan 888fc4af4e YARN-2813. Fixed NPE from MemoryTimelineStore.getDomains. Contributed by Zhijie Shen 2014-11-05 18:27:41 -08:00
Jian He e29e864c51 YARN-2579. Fixed a deadlock issue when EmbeddedElectorService and FatalEventDispatcher try to transition RM to StandBy at the same time. Contributed by Rohith Sharmaks 
(cherry picked from commit 395275af86)
 2014-11-05 17:03:26 -08:00
Vinod Kumar Vavilapalli 812ddc3991 YARN-2805. Fixed ResourceManager to load HA configs correctly before kerberos login. Contributed by Wangda Tan. 
(cherry picked from commit 834e931d8e)
 2014-11-05 15:32:49 -08:00
Zhijie Shen a1764e4d33 YARN-2767. Added a test case to verify that http static user cannot kill or submit apps in the secure mode. Contributed by Varun Vasudev. 
(cherry picked from commit 7a4c92a9d55fcecef066053ac30dff0fcd4ec90c)
 2014-11-05 11:04:38 -08:00
Vinod Kumar Vavilapalli 87e880b580 YARN-2804. Fixed Timeline service to not fill the logs with JAXB bindings exceptions. Contributed by Zhijie Shen. 
(cherry picked from commit b76179895d)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2014-11-04 18:04:37 -08:00
Karthik Kambatla 36993e39d0 YARN-2010. Handle app-recovery failures gracefully. (Jian He and Karthik Kambatla via kasha) 
(cherry picked from commit b2cd269802)
 2014-11-04 17:49:56 -08:00
Karthik Kambatla d76756dfe1 Adding release 2.5.2 to CHANGES.txt 
(cherry picked from commit 4de56d27fb)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2014-11-04 16:52:56 -08:00
Zhijie Shen 1550617e48 YARN-2752. Made ContainerExecutor append "nice -n" arg only when priority adjustment flag is set. Contributed by Xuan Gong. 
(cherry picked from commit e06c23a6c9)
 2014-11-04 15:54:28 -08:00
Vinod Kumar Vavilapalli 0b73606b1c YARN-1922. Fixed NodeManager to kill process-trees correctly in the presence of races between the launch and the stop-container call and when root processes crash. Contributed by Billie Rinaldi. 
(cherry picked from commit c5a46d4c8c)
 2014-11-03 16:41:21 -08:00
Vinod Kumar Vavilapalli f2ef8c7b48 YARN-2795. Fixed ResourceManager to not crash loading node-label data from HDFS in secure mode. Contributed by Wangda Tan. 
(cherry picked from commit ec6cbece8e)
 2014-11-03 13:46:05 -08:00
Vinod Kumar Vavilapalli c205841d49 YARN-2788. Fixed backwards compatiblity issues with log-aggregation feature that were caused when adding log-upload-time via YARN-2703. Contributed by Xuan Gong. 
(cherry picked from commit 58e9f24e0f)
 2014-11-03 13:20:01 -08:00
Vinod Kumar Vavilapalli dd82a1eb75 YARN-2798. Fixed YarnClient to populate the renewer correctly for Timeline delegation tokens. Contributed by Zhijie Shen. 
(cherry picked from commit 71fbb474f5)
 2014-11-03 12:51:30 -08:00
Jason Lowe 2098c68acb YARN-2730. DefaultContainerExecutor runs only one localizer at a time. Contributed by Siqi Li 
(cherry picked from commit 6157ace547)
 2014-11-03 20:40:02 +00:00
Zhijie Shen 3e41828639 YARN-2785. Fixed intermittent TestContainerResourceUsage failure. Contributed by Varun Vasudev. 
(cherry picked from commit 27715ec63b)
 2014-11-02 15:24:07 -08:00
Vinod Kumar Vavilapalli 5a0aac5506 YARN-2790. Fixed a NodeManager bug that was causing log-aggregation to fail beyond HFDS delegation-token expiry even when RM is a proxy-user (YARN-2704). Contributed by Jian He. 
(cherry picked from commit 5c0381c96a)
 2014-11-01 16:34:11 -07:00
Zhijie Shen 5492370a4c YARN-2711. Fixed TestDefaultContainerExecutor#testContainerLaunchError failure on Windows. Contributed by Varun Vasudev. 
(cherry picked from commit 1cd088fd9d)
 2014-10-31 17:51:03 -07:00
Jason Lowe 239c4cc572 YARN-2707. Potential null dereference in FSDownload. Contributed by Gera Shegalov 
(cherry picked from commit 7bc179f9f5)
 2014-10-31 22:55:06 +00:00
Vinod Kumar Vavilapalli f71d940e42 YARN-2789. Re-instated the NodeReport.newInstance private unstable API modified in YARN-2698 so that tests in YARN frameworks don't break. Contributed by Wangda Tan. 
(cherry picked from commit 6ce32f593b)
 2014-10-31 15:34:07 -07:00
Xuan aa13977001 YARN-2701. Addendum patch. Potential race condition in startLocalizer when using LinuxContainerExecutor. Contributed by Xuan Gong 2014-10-31 14:39:49 -07:00
Jian He a859adcc23 YARN-2770. Added functionality to renew/cancel TimeLineDelegationToken. Contributed by Zhijie Shen 
(cherry picked from commit 1b4be91866)
 2014-10-31 13:17:45 -07:00
Vinod Kumar Vavilapalli 16511ef404 YARN-2778. Moved node-lables' reports to the yarn nodes CLI from the admin CLI. Contributed by Wangda Tan. 
(cherry picked from commit b6c1188b85)
 2014-10-31 10:20:08 -07:00
Vinod Kumar Vavilapalli a3f032031d YARN-2779. Fixed ResourceManager to not require delegation tokens for communicating with Timeline Service. Contributed by Zhijie Shen. 
(cherry picked from commit d1828d9443)
 2014-10-30 23:17:50 -07:00
Vinod Kumar Vavilapalli 85fcdaf8d8 YARN-2698. Moved some node label APIs to be correctly placed in client protocol. Contributed by Wangda Tan. 
(cherry picked from commit e0233c16eb)
 2014-10-30 23:03:02 -07:00
Steve Loughran b69ebef49e YARN-2677 registry punycoding of usernames doesn't fix all usernames to be DNS-valid (stevel) 2014-10-30 22:32:08 +00:00
Jason Lowe 773ba8b606 YARN-2755. NM fails to clean up usercache_DEL_<timestamp> dirs after YARN-661. Contributed by Siqi Li 
(cherry picked from commit 73e626ad91)
 2014-10-30 15:12:35 +00:00
Zhijie Shen 2f62716441 YARN-2769. Fixed the problem that timeline domain is not set in distributed shell AM when using shell_command on Windows. Contributed by Varun Vasudev. 
(cherry picked from commit 72496311def320987c31822776e8fbb73dfcc726)
 2014-10-29 12:15:54 -07:00
Zhijie Shen dcb484fac0 YARN-2747. Fixed the test failure of TestAggregatedLogFormat when native I/O is enabled. Contributed by Xuan Gong. 
(cherry picked from commit ec63a3ffbd)
 2014-10-28 22:47:05 -07:00
Jian He e181498a67 YARN-2503. Added node lablels in web UI. Contributed by Wangda Tan 
(cherry picked from commit d5e0a09721)
(cherry picked from commit 720de7eb4c)
 2014-10-28 18:02:13 -07:00
Zhijie Shen ae62fe2ebd YARN-2741. Made NM web UI serve logs on the drive other than C: on Windows. Contributed by Craig Welch. 
(cherry picked from commit 8984e9b177)
 2014-10-28 14:13:31 -07:00
Xuan 126d1b9089 YARN-2758. Update TestApplicationHistoryClientService to use the new generic history store. Contributed by Zhijie Shen 2014-10-28 13:32:42 -07:00
Xuan 615af9e454 YARN-2279. Add UTs to cover timeline server authentication. Contributed by Zhijie Shen 2014-10-28 11:20:46 -07:00
Karthik Kambatla f5361a8354 YARN-2760. Remove 'experimental' from FairScheduler docs. (Harsh J via kasha) 
(cherry picked from commit ade3727ecb)
 2014-10-28 08:50:04 -07:00
Jian He 8d4c93aabf YARN-2502. Changed DistributedShell to support node labels. Contributed by Wangda Tan 
(cherry picked from commit f6b963fdfc)
 2014-10-27 20:14:30 -07:00
Vinod Kumar Vavilapalli 0af8cc9ca2 YARN-2704. Changed ResourceManager to optionally obtain tokens itself for the sake of localization and log-aggregation for long-running services. Contributed by Jian He. 
(cherry picked from commit a16d022ca4)
 2014-10-27 15:51:20 -07:00
Jian He ba1e549145 YARN-2591. Fixed AHSWebServices to return FORBIDDEN(403) if the request user doesn't have access to the history data. Contributed by Zhijie Shen 
(cherry picked from commit c05b581a55)
 2014-10-27 10:46:59 -07:00
Xuan 55c5612caf YARN-2726. CapacityScheduler should explicitly log when an accessible label has no capacity. Contributed by Wangda Tan 2014-10-27 10:30:09 -07:00
Zhijie Shen ab7a21569c YARN-2734. Skipped sub-folders in the local log dir when aggregating logs. Contributed by Xuan Gong. 
(cherry picked from commit caecd9fffe)
 2014-10-26 13:00:26 -07:00
Vinod Kumar Vavilapalli 259db7a726 YARN-2743. Fixed a bug in ResourceManager that was causing RMDelegationToken identifiers to be tampered and thus causing app submission failures in secure mode. Contributed by Jian He. 
(cherry picked from commit 0186645505)
 2014-10-26 11:29:06 -07:00
Xuan cccfc27e0d YARN-2723. Fix rmadmin -replaceLabelsOnNode does not correctly parse port. Contributed by Naganarasimha G R 2014-10-25 12:49:05 -07:00
Jian He c8d48d2981 YARN-2314. Disable ContainerManagementProtocolProxy cache by default to prevent creating thousands of threads in a large cluster. Contributed by Jason Lowe 
(cherry picked from commit f44cf99599)
 2014-10-24 23:06:38 -07:00
Jian He 728cc2394c YARN-1915. Fixed a race condition that client could use the ClientToAMToken to contact with AM before AM actually receives the ClientToAMTokenMasterKey. Contributed by Jason Lowe 
(cherry picked from commit 5864dd99a4)
 2014-10-24 22:49:19 -07:00
Zhijie Shen a41371e173 YARN-2703. Added logUploadedTime into LogValue for better display. Contributed by Xuan Gong. 
(cherry picked from commit f81dc3f995)
 2014-10-24 14:13:28 -07:00
Zhijie Shen 38fa39aa17 YARN-2724. Skipped uploading a local log file to HDFS if exception is raised when opening it. Contributed by Xuan Gong. 
(cherry picked from commit e31f0a6558)
 2014-10-24 11:16:44 -07:00
Zhijie Shen 09e422302f YARN-2209. Replaced AM resync/shutdown command with corresponding exceptions and made related MR changes. Contributed by Jian He. 
(cherry picked from commit 0f3b6900be)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2014-10-23 22:18:27 -07:00
Zhijie Shen 78efd4fd5c YARN-2732. Fixed syntax error in SecureContainer.apt.vm. Contributed by Jian He. 
(cherry picked from commit b94b8b30f2)
 2014-10-22 18:42:27 -07:00
Jian He 29d0164ee7 YARN-2198. Remove the need to run NodeManager as privileged account for Windows Secure Container Executor. Contributed by Remus Rusanu 
(cherry picked from commit 3b12fd6cfb)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2014-10-22 16:22:08 -07:00
Jian He a057552468 YARN-2682. Updated WindowsSecureContainerExecutor to not use DefaultContainerExecutor#getFirstApplicationDir and use getWorkingDir() instead. Contributed by Zhihai Xu 
(cherry picked from commit 0fd0ebae64)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java
 2014-10-22 16:19:40 -07:00
Vinod Kumar Vavilapalli e282805f1c YARN-1063. Augmented Hadoop common winutils to have the ability to create containers as domain users. Contributed by Remus Rusanu. 
Committed as a YARN patch even though all the code changes are in common.
(cherry picked from commit 5ca97f1e60)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2014-10-22 16:10:49 -07:00
Steve Loughran ada23f3dd3 YARN-2700 TestSecureRMRegistryOperations failing on windows: auth problems 2014-10-22 21:12:52 +01:00
Steve Loughran 8fdff5b791 YARN-2692 ktutil test hanging on some machines/ktutil versions (stevel) 2014-10-22 14:34:57 +01:00
Zhijie Shen b1ca12bc0b YARN-2721. Suppress NodeExist exception thrown by ZKRMStateStore when it retries creating znode. Contributed by Jian He. 
(cherry picked from commit 7e3b5e6f5c)
 2014-10-21 21:53:27 -07:00
Vinod Kumar Vavilapalli a1c5d4c51f YARN-2715. Fixed ResourceManager to respect common configurations for proxy users/groups beyond just the YARN level config. Contributed by Zhijie Shen. 
(cherry picked from commit c0e034336c)
 2014-10-21 20:11:14 -07:00
Zhijie Shen 7df8cbdd0f YARN-2709. Made timeline client getDelegationToken API retry if ConnectException happens. Contributed by Li Lu. 
(cherry picked from commit b2942762d7)
 2014-10-21 16:09:23 -07:00
cnauroth 8727f9e1b0 YARN-2720. Windows: Wildcard classpath variables not expanded against resources contained in archives. Contributed by Craig Welch. 
(cherry picked from commit 6637e3cf95)
(cherry picked from commit a066134277)
 2014-10-21 12:36:43 -07:00
Jason Lowe 413fe4cb02 YARN-90. NodeManager should identify failed disks becoming good again. Contributed by Varun Vasudev 
(cherry picked from commit 6f2028bd15)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestDirectoryCollection.java
 2014-10-21 17:43:22 +00:00
Zhijie Shen fcdeba204c YARN-2717. Avoided duplicate logging when container logs are not found. Contributed by Xuan Gong. 
(cherry picked from commit 171f2376d2)
 2014-10-20 22:24:19 -07:00
Jian He 852ffea84e Missing file for YARN-1879 
(cherry picked from commit 4a78a75228)
 2014-10-20 21:15:30 -07:00
Jian He 23b4af488e Missing file for YARN-2701 
(cherry picked from commit 4fa1fb3193)
 2014-10-20 20:53:28 -07:00
Jian He c7db30522c YARN-2701. Potential race condition in startLocalizer when using LinuxContainerExecutor. Contributed by Xuan Gong 
(cherry picked from commit 2839365f23)
 2014-10-20 20:53:28 -07:00
Vinod Kumar Vavilapalli debb3686d3 YARN-1972. Added a secure container-executor for Windows. Contributed by Remus Rusanu. 
commit ba7f31c2ee is the corresponding trunk commit, this is a slightly different patch for branch-2.
(cherry picked from commit 3326fba382)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2014-10-20 20:47:49 -07:00
Zhijie Shen 2af7f48625 YARN-2582. Fixed Log CLI and Web UI for showing aggregated logs of LRS. Contributed Xuan Gong. 
(cherry picked from commit e90718fa5a)
 2014-10-20 13:43:03 -07:00
Zhijie Shen 6d78dc3ee4 YARN-2673. Made timeline client put APIs retry if ConnectException happens. Contributed by Li Lu. 
(cherry picked from commit 89427419a3)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
 2014-10-20 12:31:33 -07:00
Vinod Kumar Vavilapalli 88b7e5ca65 YARN-2504. Enhanced RM Admin CLI to support management of node-labels. Contribyted by Wangda Tan. 
(cherry picked from commit 8256766498)
 2014-10-18 12:09:02 -07:00
Vinod Kumar Vavilapalli 24b0529a9f YARN-2676. Enhanced Timeline auth-filter to support proxy users. Contributed by Zhijie Shen. 
(cherry picked from commit 39063cd36f)
 2014-10-17 22:04:14 -07:00
Vinod Kumar Vavilapalli ce42210651 YARN-2705. Fixed bugs in ResourceManager node-label manager that were causing test-failures: added a dummy in-memory labels-manager. Contributed by Wangda Tan. 
(cherry picked from commit e9c66e8fd2)
 2014-10-17 18:27:34 -07:00
Jian He 4d03915be9 YARN-1879. Marked Idempotent/AtMostOnce annotations to ApplicationMasterProtocol for RM fail over. Contributed by Tsuyoshi OZAWA 
(cherry picked from commit c3de2412eb)
 2014-10-17 16:36:45 -07:00
Jian He b7504f0394 YARN-2588. Standby RM fails to transitionToActive if previous transitionToActive failed with ZK exception. Contributed by Rohith Sharmaks 
(cherry picked from commit a6aa6e42ca)
 2014-10-17 10:55:29 -07:00
Vinod Kumar Vavilapalli 778fbcd5d3 YARN-2699. Fixed a bug in CommonNodeLabelsManager that caused tests to fail when using ephemeral ports on NodeIDs. Contributed by Wangda Tan. 
(cherry picked from commit abae63caf9)
 2014-10-17 08:59:49 -07:00
Jian He 6be8330606 YARN-2621. Simplify the output when the user doesn't have the access for getDomain(s). Contributed by Zhijie Shen 
(cherry picked from commit 233d446be1)
 2014-10-16 16:38:27 -07:00
Steve Loughran b0a72f354b YARN-2689 TestSecureRMRegistryOperations failing on windows: secure ZK won't start 2014-10-16 14:21:38 -07:00
Vinod Kumar Vavilapalli 8a09fdccb3 YARN-2685. Fixed a bug in CommonNodeLabelsManager that caused wrong resource tracking per label when a host runs multiple node-managers. Contributed by Wangda Tan. 
(cherry picked from commit b3056c266a)
 2014-10-15 18:49:06 -07:00
Vinod Kumar Vavilapalli fe80fd5e88 YARN-2496. Enhanced Capacity Scheduler to have basic support for allocating resources based on node-labels. Contributed by Wangda Tan. 
YARN-2500. Ehnaced ResourceManager to support schedulers allocating resources based on node-labels. Contributed by Wangda Tan.

(cherry picked from commit f2ea555ac6)
 2014-10-15 18:35:34 -07:00
Jian He bd45dba807 YARN-2312. Deprecated old ContainerId#getId API and updated MapReduce to use ContainerId#getContainerId instead. Contributed by Tsuyoshi OZAWA 
(cherry picked from commit f93d2ea27e)
 2014-10-15 15:31:57 -07:00
Zhijie Shen eb055eadd7 YARN-2656. Made RM web services authentication filter support proxy user. Contributed by Varun Vasudev and Zhijie Shen. 
(cherry picked from commit 1220bb72d4)
 2014-10-14 21:58:12 -07:00
Zhijie Shen e71fa82ee5 HADOOP-11181. Generalized o.a.h.s.t.d.DelegationTokenManager to handle all sub-classes of AbstractDelegationTokenIdentifier. Contributed by Zhijie Shen. 
(cherry picked from commit cdce88376a)
 2014-10-14 11:39:06 -07:00
Karthik Kambatla e6102b1828 YARN-2566. DefaultContainerExecutor should pick a working directory randomly. (Zhihai Xu via kasha) 
(cherry picked from commit cc93e7e683)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java
 2014-10-13 16:42:47 -07:00
Jian He da3e84beb9 Missing Changes.txt for YARN-2308 
(cherry picked from commit 178bc505da)
 2014-10-13 14:21:01 -07:00
Jian He 29496c9be2 YARN-2308. Changed CapacityScheduler to explicitly throw exception if the queue 
to which the apps were submitted is changed across RM restart. Contributed by Craig Welch & Chang Li
(cherry picked from commit f9680d9a16)
 2014-10-13 14:16:57 -07:00
Jason Lowe 9d73c1b15f YARN-2377. Localization exception stack traces are not passed as diagnostic info. Contributed by Gera Shegalov 
(cherry picked from commit a56ea01002)
 2014-10-13 18:34:28 +00:00
Zhijie Shen 6da637ec12 YARN-2651. Spun off LogRollingInterval from LogAggregationContext. Contributed by Xuan Gong. 
(cherry picked from commit 4aed2d8e91)
 2014-10-13 10:56:22 -07:00
Jason Lowe eb5faa161a YARN-2667. Fix the release audit warning caused by hadoop-yarn-registry. Contributed by Yi Liu 
(cherry picked from commit 344a10ad5e)
 2014-10-13 15:46:14 +00:00
Steve Loughran 752f724d52 YARN-2668 yarn-registry JAR won't link against ZK 3.4.5. (stevel) 2014-10-11 18:32:47 -07:00
Vinod Kumar Vavilapalli 8dacf89b3e YARN-2501. Enhanced AMRMClient library to support requests against node labels. Contributed by Wangda Tan. 
(cherry picked from commit a5ec3d0809)
 2014-10-10 19:59:30 -07:00
Vinod Kumar Vavilapalli bb6c79f76c YARN-2494. Added NodeLabels Manager internal API and implementation. Contributed by Wangda Tan. 
(cherry picked from commit db7f165319)
 2014-10-10 11:46:10 -07:00
Zhijie Shen 16d7144337 YARN-2583. Modified AggregatedLogDeletionService to be able to delete rolling aggregated logs. Contributed by Xuan Gong. 
(cherry picked from commit cb81bac002)
 2014-10-10 00:18:24 -07:00
cnauroth 281041a57e YARN-2662. TestCgroupsLCEResourcesHandler leaks file descriptors. Contributed by Chris Nauroth. 
(cherry picked from commit d3afd730ac)
(cherry picked from commit b81641a310)
 2014-10-09 22:49:00 -07:00
Zhijie Shen ce704583a7 YARN-2671. Fix the Jira number in the change log. 
(cherry picked from commit 5b12df6587)
 2014-10-09 21:00:38 -07:00
Zhijie Shen 647874206c YARN-2617. Fixed ApplicationSubmissionContext to still set resource for backward compatibility. Contributed by Wangda Tan. 
(cherry picked from commit e532ed8faa)
 2014-10-09 18:01:13 -07:00
Vinod Kumar Vavilapalli 967c758373 YARN-2544. Added admin-API objects for using node-labels. Contributed by Wangda Tan. 
(cherry picked from commit 596702a025)
 2014-10-09 15:47:51 -07:00
Zhijie Shen a4522e90f6 YARN-2629. Made the distributed shell use the domain-based timeline ACLs. Contributed by Zhijie Shen. 
(cherry picked from commit 1d4612f5ad)
 2014-10-09 13:03:03 -07:00
Mayank Bansal cf0ad82c5a YARN-2598 GHS should show N/A instead of null for the inaccessible information (Zhijie Shen via mayank) 2014-10-09 11:39:25 -07:00
Vinod Kumar Vavilapalli ad7baad21e YARN-2493. Added user-APIs for using node-labels. Contributed by Wangda Tan. 
(cherry picked from commit 180afa2f86)
 2014-10-09 11:27:49 -07:00
Steve Loughran 088ae9c5bf YARN-913 service registry: YARN-2652 add hadoop-yarn-registry package under hadoop-yarn 2014-10-08 12:54:37 -07:00
Jian He 7ec65c8617 YARN-2649. Fixed TestAMRMRPCNodeUpdates test failure. Contributed by Ming Ma 
(cherry picked from commit e16e25ab1b)
 2014-10-08 11:03:28 -07:00
Jian He 5ff984f33f YARN-1857. CapacityScheduler headroom doesn't account for other AM's running. Contributed by Chen He and Craig Welch 
(cherry picked from commit 30d56fdbb4)
 2014-10-07 13:48:46 -07:00
Jian He efc482d1f7 YARN-2644. Fixed CapacityScheduler to return up-to-date headroom when AM allocates. Contributed by Craig Welch 
(cherry picked from commit 519e5a7dd2)
 2014-10-06 15:52:05 -07:00
Jian He 73a1b77223 YARN-2615. Changed ClientToAMTokenIdentifier/RM(Timeline)DelegationTokenIdentifier to use protobuf as payload. Contributed by Junping Du 
(cherry picked from commit ea26cc0b4a)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestClientToAMTokens.java
 2014-10-06 10:52:20 -07:00