Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
8,468 Commits 326 Branches 371 Tags
Commit Graph

1608 Commits

Author SHA1 Message Date
Karthik Kambatla
4d8fa9615f YARN-2984. Metrics for container's actual memory usage. (kasha) 
(cherry picked from commit 84198564ba6028d51c1fcf9cdcb87f6ae6e08513)
 2015-01-17 06:26:47 +05:30
Zhijie Shen
dcd6eed13a YARN-2815. Excluded transitive dependency of JLine in hadoop-yarn-server-common. Contributed by Ferdinand Xu. 
(cherry picked from commit 43302f6f44f97d67069eefdda986b6da2933393e)
 2015-01-16 16:38:36 -08:00
Junping Du
7cddec31d7 YARN-3064. TestRMRestart/TestContainerResourceUsage/TestNodeManagerResync failure with allocation timeout. (Contributed by Jian He) 
(cherry picked from commit 5d1cca34fa14706586f8bbe7f2a2264757fdcc11)
 2015-01-16 00:10:36 -08:00
Jian He
ed5a0d0aa8 YARN-2861. Fixed Timeline DT secret manager to not reuse RM's configs. Contributed by Zhijie Shen 
(cherry picked from commit 9e33116d1d8944a393937337b3963e192b9c74d1)
 2015-01-15 11:08:19 -08:00
Akira Ajisaka
b349cd9d0b YARN-3005. [JDK7] Use switch statement for String instead of if-else statement in RegistrySecurity.java (Contributed by Kengo Seki) 
(cherry picked from commit 533e551eb42af188535aeb0ab35f8ebf150a0da1)
 2015-01-15 21:30:26 +09:00
Karthik Kambatla
d98b8f4bd3 YARN-2217. [YARN-1492] Shared cache client side changes. (Chris Trezzo via kasha) 
(cherry picked from commit ba5116ec8e0c075096c6f84a8c8a1c6ce8297cf2)
 2015-01-15 14:32:35 +05:30
Xuan
79c8f4dcba YARN-2807. Option "--forceactive" not works as described in usage of 
"yarn rmadmin -transitionToActive". Contributed by Masatake Iwasaki

(cherry picked from commit d15cbae73c7ae22d5d60d8cba16cba565e8e8b20)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/YarnCommands.apt.vm
 2015-01-14 11:18:10 -08:00
Junping Du
0119e717ae YARN-3019. Make work-preserving-recovery the default mechanism for RM recovery. (Contributed by Jian He) 
(cherry picked from commit f92e5038000a012229c304bc6e5281411eff2883)
 2015-01-13 18:23:32 -08:00
Jian He
4931600030 YARN-2637. Fixed max-am-resource-percent calculation in CapacityScheduler when activating applications. Contributed by Craig Welch 
(cherry picked from commit c53420f58364b11fbda1dace7679d45534533382)
 2015-01-13 17:32:36 -08:00
Robert Kanter
13cdcf28e0 YARN-2643. Don't create a new DominantResourceCalculator on every FairScheduler.allocate call. (kasha via rkanter) 
(cherry picked from commit 51881535e659940b1b332d0c5952ee1f9958cc7f)
 2015-01-12 17:51:45 -08:00
Robert Kanter
7c4e5ff660 YARN-2957. Create unit test to automatically compare YarnConfiguration and yarn-default.xml. (rchiang via rkanter) 
(cherry picked from commit f45163191583eadcfbe0df233a3185fd1b2b78f3)
 2015-01-12 12:00:52 -08:00
Robert Kanter
e5059b91bb YARN-3027. Scheduler should use totalAvailable resource from node instead of availableResource for maxAllocation. (adhoot via rkanter) 
(cherry picked from commit ae7bf31fe1c63f323ba5271e50fd0e4425a7510f)
 2015-01-12 10:48:45 -08:00
Jian He
36b3dcaab2 YARN-3014. Replaces labels on a host should update all NM's labels on that host. Contributed by Wangda Tan 
(cherry picked from commit a2604062681230442eefae79815db5d77ccc4c2e)
 2015-01-09 17:50:18 -08:00
Jian He
68709cb9be YARN-2956. Added missing links in YARN documentation. Contributed by Masatake Iwasaki 
(cherry picked from commit 27a489cba63d9473e4cbcaef018b66108fb6c122)
 2015-01-09 17:47:00 -08:00
cnauroth
9a7c763b3f HADOOP-11464. Reinstate support for launching Hadoop processes on Windows using Cygwin. Contributed by Chris Nauroth. 2015-01-09 15:01:33 -08:00
Jian He
e7e6173049 YARN-2997. Fixed NodeStatusUpdater to not send alreay-sent completed container statuses on heartbeat. Contributed by Chengbing Liu 
(cherry picked from commit cc2a745f7e82c9fa6de03242952347c54c52dccc)
 2015-01-08 11:28:24 -08:00
Zhijie Shen
d2fbba790a YARN-2996. Improved synchronization and I/O operations of FS- and Mem- RMStateStore. Contributed by Yi Liu. 
(cherry picked from commit dc2eaa26b20cfbbcdd5784bb8761d08a42f29605)
 2015-01-08 09:49:27 -08:00
Jian He
fe664b8b8d YARN-2936. Changed YARNDelegationTokenIdentifier to set proto fields on getProto method. Contributed by Varun Saxena 
(cherry picked from commit 2638f4d0f0da375b0dd08f3188057637ed0f32d5)
 2015-01-07 16:15:03 -08:00
Jian He
d3b26454e4 YARN-3010. Fixed findbugs warning in AbstractYarnScheduler. Contributed by Yi Liu 
(cherry picked from commit e13a484a2be64fb781c5eca5ae7056cbe194ac5e)
 2015-01-07 14:12:54 -08:00
Jian He
492717cbf5 YARN-2230. Fixed few configs description in yarn-default.xml. Contributed by Vijay Bhat 
(cherry picked from commit fe8d2bd74175e7ad521bc310c41a367c0946d6ec)
 2015-01-07 10:57:41 -08:00
Zhijie Shen
8a81312062 YARN-2427. Added the API of moving apps between queues in RM web services. Contributed by Varun Vasudev. 
(cherry picked from commit 60103fca04dc713183e4ec9e12f961642e7d1001)
 2015-01-06 14:39:44 -08:00
Jian He
c61e8a7bfa YARN-2978. Fixed potential NPE while getting queue info. Contributed by Varun Saxena 
(cherry picked from commit dd57c2047bfd21910acc38c98153eedf1db75169)
 2015-01-06 14:16:43 -08:00
Karthik Kambatla
7adffad2bb YARN-2881. [YARN-2574] Implement PlanFollower for FairScheduler. (Anubhav Dhoot via kasha) 
(cherry picked from commit 0c4b11267717eb451fa6ed4c586317f2db32fbd5)
 2015-01-06 04:42:55 +05:30
Karthik Kambatla
d070597a0e YARN-2360. Fair Scheduler: Display dynamic fair share for queues on the scheduler page. (Ashwin Shankar and Wei Yan via kasha) 
(cherry picked from commit 270a271f53f52a1f33e63a0520ae6032536dd4c6)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-01-06 03:28:48 +05:30
Zhijie Shen
c6cf748985 YARN-2958. Made RMStateStore not update the last sequence number when updating the delegation token. Contributed by Varun Saxena. 
(cherry picked from commit 562a701945be3a672f9cb5a52cc6db2c1589ba2b)
 2015-01-05 13:35:09 -08:00
Tsuyoshi Ozawa
c116743bdd YARN-2922. ConcurrentModificationException in CapacityScheduler's LeafQueue. Contributed by Rohith Sharmaks. 
(cherry picked from commit ddc5be48fc35868abf7f59088f747c636e76a42a)
 2015-01-05 00:11:01 +09:00
Zhijie Shen
efe6260762 YARN-2991. Fixed DrainDispatcher to reuse the draining code path in AsyncDispatcher. Contributed by Rohith Sharmaks. 
(cherry picked from commit 947578c1c1413f9043ceb1e87df6a97df048e854)
 2015-01-02 11:10:46 -08:00
Karthik Kambatla
798ab51289 YARN-2998. Abstract out scheduler independent PlanFollower components. (Anubhav Dhoot via kasha) 
(cherry picked from commit e7257acd8a7adb74d81cd1d009d4a99f023ed844)
 2014-12-30 19:55:50 -08:00
Jian He
cde5bfe3ec YARN-2987. Fixed ClientRMService#getQueueInfo to check against queue and app ACLs. Contributed by Varun Saxena 
(cherry picked from commit e2351c7ae24cea9b217af4174512d279c55e8efd)
 2014-12-30 17:16:05 -08:00
Jian He
143e48c25a YARN-2493. Added node-labels page on RM web UI. Contributed by Wangda Tan 
(cherry picked from commit b7442bf92eb6e1ae64a0f9644ffc2eee4597aad5)
 2014-12-30 16:49:59 -08:00
Zhijie Shen
e5cb52ec1f YARN-2938. Fixed new findbugs warnings in hadoop-yarn-resourcemanager and hadoop-yarn-applicationhistoryservice. Contributed by Varun Saxena. 
(cherry picked from commit 241d3b3a50c6af92f023d8b2c24598f4813f4674)
 2014-12-29 10:07:54 -08:00
Jian He
ca0349b87a YARN-2992. ZKRMStateStore crashes due to session expiry. Contributed by Karthik Kambatla 
(cherry picked from commit 1454efe5d4fe4214ec5ef9142d55dbeca7dab953)
 2014-12-26 18:00:32 -08:00
Junping Du
fc3e810a19 YARN-2993. Several fixes (missing acl check, error log msg ...) and some refinement in AdminService. (Contributed by Yi Liu) 
(cherry picked from commit 40ee4bff65b2bfdabfd16ee7d9be3382a0476565)
 2014-12-26 07:01:44 -08:00
Tsuyoshi Ozawa
4ca49c8634 YARN-2988. Graph#save() may leak file descriptors. (Ted Yu via ozawa) 2014-12-25 13:09:51 +09:00
Jian He
60530a6c4f YARN-2946. Fixed potential deadlock in RMStateStore. Contributed by Rohith Sharmaks 
(cherry picked from commit 4f18018b7abb405f38d269da096ec9fd80c81d9a)
 2014-12-23 22:15:28 -08:00
Zhijie Shen
f02bd6683a YARN-2937. Fixed new findbugs warnings in hadoop-yarn-nodemanager. Contributed by Varun Saxena. 
(cherry picked from commit 41a548a916d4248164cb9495320f123ec215d70e)
 2014-12-23 20:35:32 -08:00
Junping Du
361fab14be YARN-2940. Fix new findbugs warnings in rest of the hadoop-yarn components. (Contributed by Li Lu) 
(cherry picked from commit d468c9aaf15e977f12e6214b47996412c907e069)
 2014-12-23 20:06:53 -08:00
Jian He
e3a54b1b8b YARN-2837. Support TimeLine server to recover delegation token when restarting. Contributed by Zhijie Shen 
(cherry picked from commit 149512a83743715bb100feb6747b7e88301018ab)
 2014-12-23 18:26:37 -08:00
Junping Du
ae3faab13e YARN-2970. NodeLabel operations in RMAdmin CLI get missing in help command. (Contributed by Varun Saxena) 
(cherry picked from commit e1ee0d45ea3315e7fbe1d28a7a35a0ce0c54483b)
 2014-12-23 18:24:45 -08:00
Jian He
8b398a66ca YARN-2340. Fixed NPE when queue is stopped during RM restart. Contributed by Rohith Sharmaks 
(cherry picked from commit 0d89859b51157078cc504ac81dc8aa75ce6b1782)
 2014-12-22 21:55:06 -08:00
Jian He
411836b74c YARN-2920. Changed CapacityScheduler to kill containers on nodes where node labels are changed. Contributed by Wangda Tan 
(cherry picked from commit fdf042dfffa4d2474e3cac86cfb8fe9ee4648beb)
 2014-12-22 16:52:15 -08:00
Junping Du
6f252f7f36 YARN-2939. Fix new findbugs warnings in hadoop-yarn-common. (Li Lu via junping_du) 
(cherry picked from commit a696fbb001b946ae75f3b8e962839c2fd3decfa1)
 2014-12-22 03:08:25 -08:00
Karthik Kambatla
2abec14ec6 YARN-2975. FSLeafQueue app lists are accessed without required locks. (kasha) 
(cherry picked from commit 24ee9e3431d27811530ffa01d8d241133fd643fe)
 2014-12-20 12:20:29 -08:00
Tsuyoshi Ozawa
31753da961 YARN-2977. Fixed intermittent TestNMClient failure. (Contributed by Junping Du) 2014-12-20 22:43:47 +09:00
Jian He
9180d11b3b YARN-2952. Fixed incorrect version check in StateStore. Contributed by Rohith Sharmaks 
(cherry picked from commit 808cba3821d5bc4267f69d14220757f01cd55715)
 2014-12-19 16:56:55 -08:00
Karthik Kambatla
b4e8ae591d YARN-2675. containersKilled metrics is not updated when the container is killed during localization. (Zhihai Xu via kasha) 
(cherry picked from commit 954fb8581ec6d7d389ac5d6f94061760a29bc309)
 2014-12-19 16:03:02 -08:00
Karthik Kambatla
8ee40a1580 YARN-2738. [YARN-2574] Add FairReservationSystem for FairScheduler. (Anubhav Dhoot via kasha) 
(cherry picked from commit a22ffc318801698e86cd0e316b4824015f2486ac)
 2014-12-19 15:38:27 -08:00
Jason Lowe
173664d70f YARN-2964. RM prematurely cancels tokens for jobs that submit jobs (oozie). Contributed by Jian He 
(cherry picked from commit 0402bada1989258ecbfdc437cb339322a1f55a97)
 2014-12-18 23:31:11 +00:00
Junping Du
cec9300201 YARN-2949. Add documentation for CGroups. (Contributed by Varun Vasudev) 
(cherry picked from commit 389f881d423c1f7c2bb90ff521e59eb8c7d26214)
 2014-12-18 06:26:53 -08:00
Junping Du
11e4ec52a1 YARN-2972. DelegationTokenRenewer thread pool never expands. Contributed by Jason Lowe 
(cherry picked from commit 2b4b0e8847048850206f091c6870a02e08cfe836)
 2014-12-17 17:41:21 -08:00