Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
8,118 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

1461 Commits

Author SHA1 Message Date
Jian He 878350a0e8 YARN-2762. Fixed RMAdminCLI to trim and check node-label related arguments before sending to RM. Contributed by Rohith Sharmaks 
(cherry picked from commit c65f1b382e)
 2014-12-16 11:01:24 -08:00
Karthik Kambatla b1ccf68db3 YARN-2914. [YARN-1492] Potential race condition in Singleton implementation of SharedCacheUploaderMetrics, CleanerMetrics, ClientSCMMetrics. (Varun Saxena via kasha) 
(cherry picked from commit e597249d36)
 2014-12-15 12:00:46 -08:00
Jian He d2356a552a Revert "YARN-2890. MiniYARNCluster should start the timeline server based on the configuration. Contributed by Mit Desai." 
This reverts commit 51af8d367d.

Conflicts:
	hadoop-yarn-project/CHANGES.txt
(cherry picked from commit a4f2995b9e)
 2014-12-15 11:08:44 -08:00
Devaraj K 4c6760bc6b YARN-2356. yarn status command for non-existent application/application 
attempt/container is too verbose. Contributed by Sunil G.

(cherry picked from commit fae3e8614f)
 2014-12-15 14:46:27 +05:30
Harsh J da36252077 YARN-2950. Change message to mandate, not suggest JS requirement on UI. Contributed by Dustin Cote. 
(cherry picked from commit 0e37bbc8e3)
 2014-12-13 07:27:37 +05:30
Steve Loughran a858d726c8 YARN-2912 Jersey Tests failing with port in use. (varun saxena via stevel) 2014-12-12 17:10:54 +00:00
Devaraj K 10ff600b35 YARN-2243. Order of arguments for Preconditions.checkNotNull() is wrong in 
SchedulerApplicationAttempt ctor. Contributed by Devaraj K.

(cherry picked from commit bda748ac3a)
 2014-12-12 12:37:13 +05:30
Jian He 41f0d20fcb YARN-2917. Fixed potential deadlock when system.exit is called in AsyncDispatcher. Contributed by Rohith Sharmaks 
(cherry picked from commit 614b6afea4)
 2014-12-11 11:18:45 -08:00
Jian He b31dfb17d8 YARN-2924. Fixed RMAdminCLI to not convert node labels to lower case. Contributed by Wangda Tan 
(cherry picked from commit 437322afca)
 2014-12-09 17:56:41 -08:00
Jian He 0f47e4e745 YARN-2930. Fixed TestRMRestart#testRMRestartRecoveringNodeLabelManager intermittent failure. Contributed by Wangda Tan 
(cherry picked from commit 2ed90a57fd)
 2014-12-09 16:48:46 -08:00
Karthik Kambatla 1986ea8dd2 YARN-2910. FSLeafQueue can throw ConcurrentModificationException. (Wilfred Spiegelenburg via kasha) 
(cherry picked from commit a2e07a5456)
 2014-12-09 14:00:52 -08:00
Karthik Kambatla 9d72b0282f YARN-2931. PublicLocalizer may fail until directory is initialized by LocalizeRunner. (Anubhav Dhoot via kasha) 
(cherry picked from commit db73cc9124)
 2014-12-08 22:26:44 -08:00
Karthik Kambatla d02cb9c512 YARN-2927. [YARN-1492] InMemorySCMStore properties are inconsistent. (Ray Chiang via kasha) 
(cherry picked from commit 120e1decd7)
 2014-12-07 22:28:57 -08:00
Jian He 4cc0abe4fe YARN-2869. CapacityScheduler should trim sub queue names when parse configuration. Contributed by Wangda Tan 
(cherry picked from commit e69af836f3)
 2014-12-05 17:34:15 -08:00
Jason Lowe b72fb6c774 YARN-2056. Disable preemption at Queue level. Contributed by Eric Payne 
(cherry picked from commit 4b13082199)
 2014-12-05 21:08:14 +00:00
Robert Kanter 2ebd1bc654 YARN-2461. Fix PROCFS_USE_SMAPS_BASED_RSS_ENABLED property in YarnConfiguration. (rchiang via rkanter) 
(cherry picked from commit 3c72f54ef5)
 2014-12-05 12:10:20 -08:00
Karthik Kambatla aadd0c392b YARN-2189. [YARN-1492] Admin service for cache manager. (Chris Trezzo via kasha) 
(cherry picked from commit 78968155d7)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/bin/yarn
 2014-12-04 19:48:18 -08:00
Jian He 1d1e7682c9 YARN-2301. Improved yarn container command. Contributed by Naganarasimha G R 
(cherry picked from commit 258623ff8b)
 2014-12-04 12:53:52 -08:00
Jian He 58c971164c YARN-2880. Added a test to make sure node labels will be recovered if RM restart is enabled. Contributed by Rohith Sharmaks 
(cherry picked from commit 73fbb3c66b)
 2014-12-03 17:17:40 -08:00
Harsh J ee25c0d890 YARN-2891. Failed Container Executor does not provide a clear error message. Contributed by Dustin Cote. (harsh) 
(cherry picked from commit 4b13318dea7a1cbbbfc1f84207af829cbe2f720e)
 2014-12-04 03:19:01 +05:30
Karthik Kambatla 25be97808b YARN-2874. Dead lock in DelegationTokenRenewer which blocks RM to execute any further apps. (Naganarasimha G R via kasha) 
(cherry picked from commit 799353e2c7)
 2014-12-03 13:45:03 -08:00
Junping Du 86535ff65f YARN-1156. Enhance NodeManager AllocatedGB and AvailableGB metrics for aggregation of decimal values. (Contributed by Tsuyoshi OZAWA) 
(cherry picked from commit e65b7c5ff6)
 2014-12-03 04:12:35 -08:00
Jian He d6f3d4893d YARN-2894. Fixed a bug regarding application view acl when RM fails over. Contributed by Rohith Sharmaks 
(cherry picked from commit 392c3aaea8)
 2014-12-02 17:17:16 -08:00
Jian He 5067ac098b YARN-2136. Changed RMStateStore to ignore store opearations when fenced. Contributed by Varun Saxena 
(cherry picked from commit 52bcefca8b)
 2014-12-02 11:02:00 -08:00
Zhijie Shen d21ef79707 YARN-2890. MiniYARNCluster should start the timeline server based on the configuration. Contributed by Mit Desai. 
(cherry picked from commit 51af8d367d)
 2014-12-01 19:08:31 -08:00
Jian He d208c90148 YARN-2765. Added leveldb-based implementation for RMStateStore. Contributed by Jason Lowe 
(cherry picked from commit a7fba0bc28)
 2014-12-01 16:45:17 -08:00
Jason Lowe 38ea1419f6 YARN-2905. AggregatedLogsBlock page can infinitely loop if the aggregated log file is corrupted. Contributed by Varun Saxena 
(cherry picked from commit 0f9528b99a)
 2014-12-01 22:30:20 +00:00
Junping Du c7bd22974a YARN-2907. SchedulerNode#toString should print all resource detail instead of only memory. (Contributed by Rohith) 
(cherry picked from commit c732ed760e)
 2014-12-01 05:41:56 -08:00
Zhijie Shen a9adc1295c YARN-2165. Added the sanity check for the numeric configuration values of the timeline service. Contributed by Vasanth kumar RJ. 
(cherry picked from commit 8f1454cc6d)
 2014-11-26 12:12:42 -08:00
Karthik Kambatla 3d4536af82 YARN-2188. [YARN-1492] Client service for cache manager. (Chris Trezzo and Sangjin Lee via kasha) 
(cherry picked from commit 834b3b98e7ecc4bfe93c7dd1a9e928e9653cf138)
 2014-11-25 16:22:14 -08:00
Jian He ae35b0e14d YARN-2906. CapacitySchedulerPage shows HTML tags for a queue's Active Users. Contributed by Jason Lowe 
(cherry picked from commit 8a7ca13b13)
 2014-11-25 16:18:02 -08:00
Jian He 2863056530 YARN-2404. Removed ApplicationAttemptState and ApplicationState class in RMStateStore. Contributed by Tsuyoshi OZAWA 
(cherry picked from commit 5805a81efb)
 2014-11-25 12:51:03 -08:00
Zhijie Shen 57d62d4ded YARN-2697. Remove useless RMAuthenticationHandler. Contributed by Haosong Huang. 
(cherry picked from commit e37a4ff0c1)
 2014-11-24 15:06:44 -08:00
Jason Lowe 89ef49fb08 YARN-1984. LeveldbTimelineStore does not handle db exceptions properly. Contributed by Varun Saxena 
(cherry picked from commit 1ce4d33c2d)
 2014-11-24 22:38:09 +00:00
Sandy Ryza 3e2e8eac22 YARN-2669. FairScheduler: queue names shouldn't allow periods (Wei Yan via Sandy Ryza) 2014-11-21 16:08:22 -08:00
Karthik Kambatla e0df934ed1 Update release date for 2.5.2 
(cherry picked from commit 1e9a3f42ff)
 2014-11-21 14:34:47 -08:00
Karthik Kambatla af0b54a4ee YARN-2679. Add metric for container launch duration. (Zhihai Xu via kasha) 
(cherry picked from commit 233b61e495)
 2014-11-21 14:22:53 -08:00
Karthik Kambatla e9db0aa35c YARN-2604. Scheduler should consider max-allocation-* in conjunction with the largest node. (Robert Kanter via kasha) 
(cherry picked from commit 3114d4731d)
 2014-11-21 10:42:47 -08:00
Jonathan Eagles 6570f7fd72 YARN-2375. Allow enabling/disabling timeline server per framework. (Mit Desai via jeagles) 
(cherry picked from commit c298a9a845)
 2014-11-20 23:40:26 -06:00
Karthik Kambatla 7ba5bb0c5b YARN-2315. FairScheduler: Set current capacity in addition to capacity. (Zhihai Xu via kasha) 
(cherry picked from commit a9a0cc3679)
 2014-11-19 20:19:54 -08:00
Karthik Kambatla 564deb75a8 YARN-2802. ClusterMetrics to include AM launch and register delays. (Zhihai Xu via kasha) 
(cherry picked from commit f9f8e750edbe6db453f0a845e2ed49ede66e0e8a)
 2014-11-19 19:50:35 -08:00
Jian He db31ef7e7f YARN-2865. Fixed RM to always create a new RMContext when transtions from StandBy to Active. Contributed by Rohith Sharmaks 
(cherry picked from commit 9cb8b75ba5)
 2014-11-19 19:49:44 -08:00
Jian He c359e2a6a4 YARN-2878. Fix DockerContainerExecutor.apt.vm formatting. Contributed by Abin Shahab 
(cherry picked from commit bc4ee5e06f)
 2014-11-19 14:45:01 -08:00
Zhijie Shen 610e123772 YARN-2870. Updated the command to run the timeline server it the document. Contributed by Masatake Iwasaki. 
(cherry picked from commit ef38fb9758)
 2014-11-18 16:17:07 -08:00
Jian He 1412e712b4 YARN-2157. Added YARN metrics in the documentaion. Contributed by Akira AJISAKA 
(cherry picked from commit 90a968d675)
 2014-11-18 16:13:08 -08:00
Karthik Kambatla 8df4c04c3a YARN-2690. [YARN-2574] Make ReservationSystem and its dependent classes independent of Scheduler type. (Anubhav Dhoot via kasha) 
(cherry picked from commit 2fce6d6141)
 2014-11-17 16:46:18 -08:00
Jason Lowe 242fd0e39a YARN-2414. RM web UI: app page will crash if app is failed before any attempt has been created. Contributed by Wangda Tan 
(cherry picked from commit 81c9d17af8)
 2014-11-17 21:16:54 +00:00
Vinod Kumar Vavilapalli 41275755c7 YARN-1703. Fixed ResourceManager web-proxy to close connections correctly. Contributed by Rohith Sharma. 
(cherry picked from commit 198fb58776)
 2014-11-15 12:06:53 -08:00
Jian He 21054c5d14 YARN-2432. RMStateStore should process the pending events before close. Contributed by Varun Saxena 
(cherry picked from commit 9b860661a5)
 2014-11-14 17:25:21 -08:00
Sandy Ryza c4c77669f0 YARN-2811. In Fair Scheduler, reservation fulfillments shouldn't ignore max share (Siqi Li via Sandy Ryza) 2014-11-14 15:20:37 -08:00
Jason Lowe ad140d1fc8 YARN-2816. NM fail to start with NPE during container recovery. Contributed by Zhihai Xu 
(cherry picked from commit 49c38898b0)
 2014-11-14 21:27:16 +00:00
Jason Lowe 00d135d75a YARN-2857. ConcurrentModificationException in ContainerLogAppender. Contributed by Mohammad Kamrul Islam 
(cherry picked from commit f2fe8a800e)
 2014-11-14 20:48:51 +00:00
Jian He beb184ac58 YARN-2856. Fixed RMAppImpl to handle ATTEMPT_KILLED event at ACCEPTED state on app recovery. Contributed by Rohith Sharmaks 
(cherry picked from commit d005404ef7)
 2014-11-13 15:34:56 -08:00
Zhijie Shen 53d6c91df9 YARN-2766. Made ApplicationHistoryManager return a sorted list of apps, attempts and containers. Contributed by Robert Kanter. 
(cherry picked from commit 3648cb57c9)
 2014-11-13 13:54:30 -08:00
Vinod Kumar Vavilapalli d35bcf2074 Revert "YARN-2603. ApplicationConstants missing HADOOP_MAPRED_HOME (Ray Chiang via aw)" 
This reverts commit b0a9cd37ab.

Conflicts:
	hadoop-yarn-project/CHANGES.txt

(cherry picked from commit 4ae9780e6a)
 2014-11-13 13:21:46 -08:00
Arun C. Murthy fc324d7062 Preparing to release hadoop-2.6.0-rc1. 
(cherry picked from commit 9ff7315cbe)
 2014-11-13 13:03:13 -08:00
Arun C. Murthy d0285d1ad6 YARN-2635. Merging to branch-2.6 for hadoop-2.6.0-rc1. 
(cherry picked from commit 81dc0ac6dc)
 2014-11-13 10:28:10 -08:00
Arun C. Murthy 906e0a1862 YARN-2853. Merging to branch-2.6 for hadoop-2.6.0-rc1. 
(cherry picked from commit d648e60eba)
 2014-11-13 10:05:07 -08:00
Jason Lowe 6fd547da8f YARN-2846. Incorrect persist exit code for running containers in reacquireContainer() that interrupted by NodeManager restart. Contributed by Junping Du 
(cherry picked from commit 33ea5ae92b)
 2014-11-13 16:17:08 +00:00
Vinod Kumar Vavilapalli ed3e5cb164 YARN-2853. Fixed a bug in ResourceManager causing apps to hang when the user kill request races with ApplicationMaster finish. Contributed by Jian He. 
(cherry picked from commit 3651fe1b08)
 2014-11-13 08:13:57 -08:00
Allen Wittenauer a6de8f2e30 YARN-2603. ApplicationConstants missing HADOOP_MAPRED_HOME (Ray Chiang via aw) 2014-11-12 17:04:24 -08:00
Zhijie Shen 8405ec0895 YARN-2794. Fixed log messages about distributing system-credentials. Contributed by Jian He. 
(cherry picked from commit be7bf956e9)
 2014-11-12 11:09:34 -08:00
Arun C. Murthy f7f78cf934 YARN-2843. Merging to branch-2.6 for hadoop-2.6.0-rc1. 2014-11-12 10:41:44 -08:00
Arun C. Murthy cd8559e099 YARN-2841. Merging to branch-2.6 for hadoop-2.6.0-rc1. 2014-11-12 10:37:40 -08:00
Karthik Kambatla ff1b13ded5 YARN-2236. [YARN-1492] Shared Cache uploader service on the Node Manager. (Chris Trezzo and Sanjin Lee via kasha) 
(cherry picked from commit a04143039e)
 2014-11-12 09:31:30 -08:00
Jason Lowe 6984d899e3 YARN-2780. Log aggregated resource allocation in rm-appsummary.log. Contributed by Eric Payne 
(cherry picked from commit f8aefa5e9c)
 2014-11-12 17:02:30 +00:00
Ravi Prakash d863f54f57 YARN-1964. Create Docker analog of the LinuxContainerExecutor in YARN 2014-11-11 21:29:27 -08:00
Karthik Kambatla 6cbd3aa2e8 YARN-570. Time strings are formated in different timezone. (Akira Ajisaka and Peng Zhang via kasha) 
(cherry picked from commit 840b13110d63d7e062cefd4b993cc7dea44aed8d)
 2014-11-11 13:23:29 -08:00
Vinod Kumar Vavilapalli 9d6c0d7c9f YARN-2843. Fixed NodeLabelsManager to trim inputs for hosts and labels so as to make them work correctly. Contributed by Wangda Tan. 
(cherry picked from commit 0fd97f9c19)
 2014-11-11 12:45:59 -08:00
Karthik Kambatla f030959849 YARN-2735. diskUtilizationPercentageCutoff and diskUtilizationSpaceCutoff are initialized twice in DirectoryCollection. (Zhihai Xu via kasha) 
(cherry picked from commit 061bc293c8)
 2014-11-11 10:31:56 -08:00
Xuan b390579c15 YARN-2841: Correct fix version from branch-2.6 to branch-2.7 in the 
change log.

(cherry picked from commit 58e9bf4b90)
 2014-11-10 18:35:59 -08:00
Xuan b3badf935a YARN-2841. RMProxy should retry EOFException. Contributed by Jian He 
(cherry picked from commit 5c9a51f140)
 2014-11-10 18:26:32 -08:00
Karthik Kambatla 92de44c2c5 Set the release date for 2.5.2 
(cherry picked from commit 68a0508728)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-11-10 15:23:43 -08:00
Vinod Kumar Vavilapalli d76fc94b21 YARN-2834. Fixed ResourceManager to ignore token-renewal failures on recovery consistent with the (somewhat incorrect) behaviour in the non-recovery case. Contributed by Jian He. 
(cherry picked from commit e76faebc95)
 2014-11-09 18:57:48 -08:00
Arun C. Murthy cff08dab5e Updated CHANGES.txt to reflect hadoop-2.6.0 release dates. 
(cherry picked from commit 770cc14442)
 2014-11-09 18:48:01 -08:00
Arun C. Murthy 175d222bfc YARN-2830. Add backwords compatible ContainerId.newInstance constructor. Contributed by Jonathan Eagles. 
(cherry picked from commit 43cd07b408)
 2014-11-09 15:03:59 -08:00
Vinod Kumar Vavilapalli 0a5d95f705 YARN-2607. Fixed issues in TestDistributedShell. Contributed by Wangda Tan. 
(cherry picked from commit 737d9284c1)
 2014-11-08 11:02:17 -08:00
Zhijie Shen 1880a5a7c3 YARN-2505. Supported get/add/remove/change labels in RM REST API. Contributed by Craig Welch. 
(cherry picked from commit 9a4e0d343e)
 2014-11-07 20:39:53 -08:00
Vinod Kumar Vavilapalli b865a95f06 YARN-2826. Fixed user-groups mappings' refresh bug caused by YARN-2826. Contributed by Wangda Tan. 
(cherry picked from commit df36edf751)
 2014-11-07 19:44:52 -08:00
Xuan 7d26cffb0c YARN-2819. NPE in ATS Timeline Domains when upgrading from 2.4 to 2.6. Contributed by Zhijie Shen 
(cherry picked from commit 4a114dd67a)
 2014-11-07 16:12:45 -08:00
Jason Lowe 9447235527 YARN-2632. Document NM Restart feature. Contributed by Junping Du and Vinod Kumar Vavilapalli 
(cherry picked from commit 1e215e8ba2)
 2014-11-07 23:41:29 +00:00
Jason Lowe a5764cb783 YARN-2825. Container leak on NM. Contributed by Jian He 
(cherry picked from commit c3d475070a)
 2014-11-07 23:17:34 +00:00
Vinod Kumar Vavilapalli a50345f654 YARN-2753. Fixed a bunch of bugs in the NodeLabelsManager classes. Contributed by Zhihai xu. 
(cherry picked from commit 4cfd5bc7c1)
 2014-11-07 14:16:54 -08:00
cnauroth c16f718293 YARN-2803. MR distributed cache not working correctly on Windows after NodeManager privileged account changes. Contributed by Craig Welch. 
(cherry picked from commit 06b797947c)
 2014-11-07 12:36:42 -08:00
Vinod Kumar Vavilapalli 7b19079657 YARN-2827. Fixed bugs in "yarn queue" CLI. Contributed by Wangda Tan. 2014-11-07 12:00:49 -08:00
Vinod Kumar Vavilapalli d5d2905262 YARN-2824. Fixed Capacity Scheduler to not crash when some node-labels are not mapped to queues by making default capacities per label to be zero. Contributed by Wangda Tan. 
(cherry picked from commit 2ac1be7dec)
 2014-11-07 10:44:25 -08:00
Xuan bf79541868 YARN-2810. TestRMProxyUsersConf fails on Windows VMs. Contributed by Varun Vasudev 
(cherry picked from commit 1e97f2f094)
 2014-11-07 09:46:04 -08:00
Vinod Kumar Vavilapalli eb01c602ed YARN-2823. Fixed ResourceManager app-attempt state machine to inform schedulers about previous finished attempts of a running appliation to avoid expectation mismatch w.r.t transferred containers. Contributed by Jian He. 
(cherry picked from commit a5657182a7)
 2014-11-07 09:30:04 -08:00
Vinod Kumar Vavilapalli b1521c7c3c YARN-2647. Added a queue CLI for getting queue information. Contributed by Sunil Govind. 
(cherry picked from commit 61effcbebc)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/bin/yarn
 2014-11-06 20:57:55 -08:00
Vinod Kumar Vavilapalli 52e57a95d9 YARN-2744. Fixed CapacityScheduler to validate node-labels correctly against queues. Contributed by Wangda Tan. 
(cherry picked from commit a3839a9fbf)
 2014-11-06 17:29:12 -08:00
Steve Loughran 5924e74d55 YARN-2768 Improved Yarn Registry service record structure (stevel) 2014-11-06 20:21:25 +00:00
Vinod Kumar Vavilapalli 0f9199fb07 YARN-2818. Removed the now unnecessary user entity injection from Timeline service given we now have domains. Contributed by Zhijie Shen. 
(cherry picked from commit f5b19bed7d)
 2014-11-06 11:49:44 -08:00
Xuan 4aa98d5991 YARN-2812. TestApplicationHistoryServer is likely to fail on less powerful machine. Contributed by Zhijie Shen 
(cherry picked from commit b0b52c4e11)
 2014-11-05 20:44:07 -08:00
Xuan e26be1d85c YARN-2813. Fixed NPE from MemoryTimelineStore.getDomains. Contributed by Zhijie Shen 2014-11-05 18:26:07 -08:00
Jian He f92ff24f5e YARN-2579. Fixed a deadlock issue when EmbeddedElectorService and FatalEventDispatcher try to transition RM to StandBy at the same time. Contributed by Rohith Sharmaks 
(cherry picked from commit 395275af86)
 2014-11-05 17:03:01 -08:00
Vinod Kumar Vavilapalli 3bc31e35a6 YARN-2805. Fixed ResourceManager to load HA configs correctly before kerberos login. Contributed by Wangda Tan. 
(cherry picked from commit 834e931d8e)
 2014-11-05 15:31:41 -08:00
Zhijie Shen f3ed97de3a YARN-2767. Added a test case to verify that http static user cannot kill or submit apps in the secure mode. Contributed by Varun Vasudev. 
(cherry picked from commit 7a4c92a9d55fcecef066053ac30dff0fcd4ec90c)
 2014-11-05 10:59:05 -08:00
Vinod Kumar Vavilapalli 56ab333161 YARN-2804. Fixed Timeline service to not fill the logs with JAXB bindings exceptions. Contributed by Zhijie Shen. 
(cherry picked from commit b76179895d)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2014-11-04 18:03:17 -08:00
Karthik Kambatla 3e4b280de7 YARN-2010. Handle app-recovery failures gracefully. (Jian He and Karthik Kambatla via kasha) 
(cherry picked from commit b2cd269802)
 2014-11-04 17:48:49 -08:00
Karthik Kambatla 6f9484221e Adding release 2.5.2 to CHANGES.txt 
(cherry picked from commit 4de56d27fb)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2014-11-04 16:51:54 -08:00