Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
7,895 Commits 326 Branches 371 Tags
Commit Graph

7895 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
41d19f4740 Release process for 2.6.1: Set the release date for 2.6.1 release-2.6.1 2015-09-23 15:15:36 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
e8c4544d8e YARN-3248. Moving it to improvements section in CHANGES.txt. 
(cherry picked from commit df52ff92ae42977bc743630191d1cbe1e17d00cf)
 2015-09-23 15:04:39 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
4697d94c6a CHANGES.txt: Dropping YARN-3379 as only the refactor part of it exists in branch-2.6.1. 2015-09-23 13:52:16 -07:00
Vinod Kumar Vavilapalli
b4d876d837 Set the release date for 2.6.1-RC1 release-2.6.1-RC1 2015-09-16 12:04:57 -07:00
Hitesh Shah
dba2b60fdd YARN-2890. MiniYarnCluster should turn on timeline service if configured to do so. Contributed by Mit Desai. 
(cherry picked from commit 265ed1fe804743601a8b62cabc1e4dc2ec8e502f)
(cherry picked from commit 55b794e7fa205df655c19bbfe1de99091fa9dc64)
 2015-09-15 17:30:15 -07:00
Vinod Kumar Vavilapalli
9c4a6e1270 Revert "YARN-2890. MiniYARNCluster should start the timeline server based on the configuration. Contributed by Mit Desai." 
This reverts commit 8a47d1aa559c380869861c070af9d1e7a8732607.
 2015-09-15 17:30:15 -07:00
Zhijie Shen
d57c3f0a26 YARN-3544. Got back AM logs link on the RM web UI for a completed app. Contributed by Xuan Gong. 
(cherry picked from commit 21bf2cdcb77f69abc906e6cd401a8fb221f250e9)
(cherry picked from commit c9ee316045b83b18cb068aa4de739a1f4b50f02a)
 2015-09-15 17:30:15 -07:00
Xuan
7af5d6b4ba YARN-3248. Display count of nodes blacklisted by apps in the web UI. 
Contributed by Varun Vasudev

(cherry picked from commit 4728bdfa15809db4b8b235faa286c65de4a48cf6)
(cherry picked from commit e26b6e55e96b763063dfbd39977096367eafc1e3)
 2015-09-15 17:30:06 -07:00
Jian He
e914220ab9 YARN-3379. Fixed missing data in localityTable and ResourceRequests table in RM WebUI. Contributed by Xuan Gong 
(cherry picked from commit 4e886eb9cbd2dcb128bbfd17309c734083093a4c)

(cherry picked from commit 3f0c9e5fe36d201de021d989b23ebaeb2d9a027b)
 2015-09-14 12:54:01 -07:00
Zhijie Shen
3ab820e696 YARN-3740. Fixed the typo in the configuration name: APPLICATION_HISTORY_PREFIX_MAX_APPS. Contributed by Xuan Gong. 
(cherry picked from commit eb6bf91eeacf97afb4cefe590f75ba94f3187d2b)
(cherry picked from commit 68cddb894a95a1ee63291af162849a23cb4eeb5f)
 2015-09-11 11:53:51 -07:00
Xuan
85ec6eb37a YARN-3171. Sort by Application id, AppAttempt and ContainerID doesn't 
work in ATS / RM web ui. Contributed by Naganarasimha G R

(cherry picked from commit 3ff1ba2a7b00fdf06270d00b2193bde4b56b06b3)
(cherry picked from commit b6eb36dbdc1c75f412616426cc7599f485e9b86e)
 2015-09-11 11:49:11 -07:00
Zhijie Shen
f4154bdee8 YARN-1884. Added nodeHttpAddress into ContainerReport and fixed the link to NM web page. Contributed by Xuan Gong. 
(cherry picked from commit 85f6d67fa78511f255fcfa810afc9a156a7b483b)
(cherry picked from commit 426535007bcdc67331f7a37b5d69cc20b37c26e0)
 2015-09-11 11:45:29 -07:00
Vinod Kumar Vavilapalli
075d899d10 Set the release date for 2.6.1-RC0 release-2.6.1-RC0 2015-09-09 15:39:02 -07:00
Vinod Kumar Vavilapalli
c86d9bd10b Preparing for release 2.6.1: Posting release notes generated. 2015-09-09 15:31:14 -07:00
Vinod Kumar Vavilapalli
3462a00dd2 Preparing for release 2.6.1: mvn versions:set -DnewVersion=2.6.1 2015-09-09 15:29:57 -07:00
Vinod Kumar Vavilapalli
cfb012a255 Preparing for release 2.6.1: Updating site update page. 2015-09-09 15:29:03 -07:00
Colin Patrick Mccabe
adc3405321 HDFS-7552. Change FsVolumeList toString() to fix TestDataNodeVolumeFailureToleration (Liang Xie via Colin P. McCabe) 
(cherry picked from commit a4876c130f1627e59ef055e586640d1933fc49af)

(cherry picked from commit 22f72c79462c424e47610470fd48e871887ac326)
 2015-09-09 14:59:31 -07:00
yliu
0adcfe6c7e HDFS-8863. The remaining space check in BlockPlacementPolicyDefault is flawed. (Kihwal Lee via yliu) 
(cherry picked from commit 146db49f7ff0dec82cd51f366311030404b770d7)
 2015-09-08 22:57:36 -07:00
Jian He
2b526ba757 YARN-4047. ClientRMService getApplications has high scheduler lock contention. Contributed by Jason Lowe 
(cherry picked from commit 7a445fcfabcf9c6aae219051f65d3f6cb8feb87c)

(cherry picked from commit 703fa1b141a98449746bd6fb3b144e74d964d1f5)
 2015-09-08 22:57:35 -07:00
Xuan
d59bf81e08 YARN-3999. RM hangs on draing events. Contributed by Jian He 
(cherry picked from commit 3ae716fa696b87e849dae40225dc59fb5ed114cb)
(cherry picked from commit 2ebdf5bfcee9ede80681a5266df225885d830883)
 2015-09-08 22:57:35 -07:00
Jonathan Eagles
6ed2486c7e YARN-3978. Configurably turn off the saving of container info in Generic AHS (Eric Payne via jeagles) 
(cherry picked from commit 3cd02b95224e9d43fd63a4ef9ac5c44f113f710d)
(cherry picked from commit 899df5bce03ea4f94487e48c1d38bd30ae10c26f)
 2015-09-08 22:57:34 -07:00
Jian He
92742b4402 YARN-2301. Improved yarn container command. Contributed by Naganarasimha G R 
(cherry picked from commit 258623ff8bb1a1057ae3501d4f20982d5a59ea34)

(cherry picked from commit 1d1e7682c9cad6a2f819b390ca3368dfa29c7097)
 2015-09-08 22:57:28 -07:00
cnauroth
c3928b3068 HDFS-7570. SecondaryNameNode need twice memory when calling reloadFromImageFile. Contributed by zhaoyunjiong. 
(cherry picked from commit 85aec75ce53445e1abf840076d2e10f1e3c6d69b)
(cherry picked from commit 27eecaef10feb49c4160ad289fd301bba6608973)
 2015-09-08 17:21:58 -07:00
Vinod Kumar Vavilapalli
33537078a8 HDFS-8384. Allow NN to startup if there are files having a lease but are not under construction. Contributed by Jing Zhao. 2015-09-08 16:45:38 -07:00
Colin Patrick Mccabe
a976acc02d HDFS-8846. Add a unit test for INotify functionality across a layout version upgrade (Zhe Zhang via Colin P. McCabe) 
(cherry picked from commit a4d9acc51d1a977bc333da17780c00c72e8546f1)
(cherry picked from commit 9264b7e119efb70fb355904652beeb97e7ad90b9)
 2015-09-08 16:18:30 -07:00
Colin Patrick Mccabe
2e86d1f254 HDFS-8480. Fix performance and timeout issues in HDFS-7929 by using hard-links to preserve old edit logs, instead of copying them. (Zhe Zhang via Colin P. McCabe) 
(cherry picked from commit 7b424f938c3c306795d574792b086d84e4f06425)
(cherry picked from commit cbd11681ce8a51d187d91748b67a708681e599de)
 2015-09-08 16:12:45 -07:00
Colin Patrick Mccabe
4363145128 HDFS-7446. HDFS inotify should have the ability to determine what txid it has read up to (cmccabe) 
(cherry picked from commit 75a326aaff8c92349701d9b3473c3070b8c2be44)

(cherry picked from commit 06552a15d5172a2b0ad3d61aa7f9a849857385aa)
 2015-09-08 15:32:17 -07:00
Tsz-Wo Nicholas Sze
8ed162bcbf HDFS-7980. Incremental BlockReport will dramatically slow down namenode startup. Contributed by Walter Su 
(cherry picked from commit 4e1f2eb3955a97a70cf127dc97ae49201a90f5e0)
 2015-09-07 11:46:33 -07:00
cnauroth
c76f5b3188 HADOOP-11934. Use of JavaKeyStoreProvider in LdapGroupsMapping causes infinite loop. Contributed by Larry McCay. 
(cherry picked from commit 860b8373c3a851386b8cd2d4265dd35e5aabc941)
(cherry picked from commit 33648268ce0f79bf51facafa3d151612e3d00ddb)
 2015-09-07 11:09:20 -07:00
Jian He
2336264900 YARN-2918. RM should not fail on startup if queue's configured labels do not exist in cluster-node-labels. Contributed by Wangda Tan 
(cherry picked from commit f489a4ec969f3727d03c8e85d51af1018fc0b2a1)

(cherry picked from commit d817fbb34d6e34991c6e512c20d71387750a98f4)
 2015-09-06 14:15:33 -07:00
Jian He
ee2b6bc248 YARN-3124. Fixed CS LeafQueue/ParentQueue to use QueueCapacities to track capacities-by-label. Contributed by Wangda Tan 
(cherry picked from commit 18a594257e052e8f10a03e5594e6cc6901dc56be)

(cherry picked from commit 1be2d64ddddaa6322909073cfaf7f2f2eb46e18d)
 2015-09-06 11:54:40 -07:00
Jian He
637e7f9e39 YARN-2694. Ensure only single node label specified in ResourceRequest. Contributed by Wangda Tan 
(cherry picked from commit c1957fef29b07fea70938e971b30532a1e131fd0)

(cherry picked from commit 3ddafaa7c854dcf21ecc790c276927e7c869e62c)
 2015-09-05 21:07:51 -07:00
Jian He
4c94f07140 YARN-3098. Created common QueueCapacities class in Capacity Scheduler to track capacities-by-labels of queues. Contributed by Wangda Tan 
(cherry picked from commit 21d80b3dd90a8e33e51701887c8d9369ed4ab17d)

(cherry picked from commit c0b1311a93614becc4a255af48fb7b697d491b80)
 2015-09-05 20:54:20 -07:00
Jian He
d9281fbbab YARN-3099. Capacity Scheduler LeafQueue/ParentQueue should use ResourceUsage to track used-resources-by-label. Contributed by Wangda Tan 
(cherry picked from commit 86358221fc85a7743052a0b4c1647353508bf308)

(cherry picked from commit cabf97ae4f2dad53c7b9e3d10a67876b16d94074)
 2015-09-05 20:54:20 -07:00
Jian He
b0ad553841 YARN-3092. Created a common ResourceUsage class to track labeled resource usages in Capacity Scheduler. Contributed by Wangda Tan 
(cherry picked from commit 6f9fe76918bbc79109653edc6cde85df05148ba3)

(cherry picked from commit 61b4116b4b3c0eec8f514f079debd88bc757b28e)
 2015-09-05 20:54:19 -07:00
Jian He
419e18cb37 YARN-2978. Fixed potential NPE while getting queue info. Contributed by Varun Saxena 
(cherry picked from commit dd57c2047bfd21910acc38c98153eedf1db75169)

(cherry picked from commit c61e8a7bfa7236e354f859a889083fab3d7ca9eb)
 2015-09-05 20:54:19 -07:00
Jian He
88f022da24 YARN-2920. Changed CapacityScheduler to kill containers on nodes where node labels are changed. Contributed by Wangda Tan 
(cherry picked from commit fdf042dfffa4d2474e3cac86cfb8fe9ee4648beb)

(cherry picked from commit 411836b74c6c02c0b5aebbbce29c209d93db1de2)
 2015-09-05 20:54:18 -07:00
Wangda Tan
2073fc0f84 Add missing test file of YARN-3733 
(cherry picked from commit 405bbcf68c32d8fd8a83e46e686eacd14e5a533c)
(cherry picked from commit 344b7509153cdd993218cd5104c7e5c07cd35d3c)
 2015-09-03 17:43:03 -07:00
Wangda Tan
85d92721a4 YARN-3733. Fix DominantRC#compare() does not work as expected if cluster resource is empty. (Rohith Sharmaks via wangda) 
(cherry picked from commit ebd797c48fe236b404cf3a125ac9d1f7714e291e)
(cherry picked from commit 78d626fa892415023827e35ad549636e2a83275d)
 2015-09-03 17:43:01 -07:00
Jian He
f1b35ffd4c YARN-2637. Fixed max-am-resource-percent calculation in CapacityScheduler when activating applications. Contributed by Craig Welch 
(cherry picked from commit c53420f58364b11fbda1dace7679d45534533382)

(cherry picked from commit 4931600030e13d9332d9a0e588487cb8684c667d)
 2015-09-03 17:40:24 -07:00
Colin Patrick Mccabe
3bd9b7459b HADOOP-12280. Skip unit tests based on maven profile rather than NativeCodeLoader.isNativeCodeLoaded (Masatake Iwasaki via Colin P. McCabe) 
(cherry picked from commit 6f83274afc1eba1159427684d72d8f13778c5a88)
(cherry picked from commit e92107b18f82b3501deaa6170d322a0fb512ec71)
 2015-09-03 15:02:33 -07:00
Jian He
d8ddfea491 HADOOP-11932. MetricsSinkAdapter may hang when being stopped. Contributed by Brahma Reddy Battula 
(cherry picked from commit f59612edd74d1bef2b60870c24c1f67c56b2b3cb)

(cherry picked from commit 5950c1f6f8ac6f514f8d2e8bfbd1f71747b097de)
 2015-09-03 14:45:04 -07:00
Jason Lowe
ca7fe71000 YARN-3990. AsyncDispatcher may overloaded with RMAppNodeUpdateEvent when Node is connected/disconnected. Contributed by Bibin A Chundatt 
(cherry picked from commit 32e490b6c035487e99df30ce80366446fe09bd6c)

(cherry picked from commit c31e3ba92132f232bd56b257f3854ffe430fbab9)
(cherry picked from commit 07d31d4c0808a169f4770187d655f38aa105255c)
 2015-09-03 14:40:20 -07:00
Jason Lowe
fe5877a49e YARN-3850. NM fails to read files from full disks which can lead to container logs being lost and other issues. Contributed by Varun Saxena 
(cherry picked from commit 40b256949ad6f6e0dbdd248f2d257b05899f4332)

(cherry picked from commit 0221d19f4e398c386f4ca3990b0893562aa8dacf)
(cherry picked from commit 87d2204f28f192a964c04a5fa1e2e31644d74b59)
 2015-09-03 14:35:01 -07:00
Harsh J
34739fc91e HADOOP-8151. Error handling in snappy decompressor throws invalid exceptions. Contributed by Matt Foley. (harsh) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1389006 13f79535-47bb-0310-9956-ffa450edef68
(cherry picked from commit ac31d6a4485d7ff9074fb5dade7a6cf5292bb347)

Conflicts:

	hadoop-common-project/hadoop-common/CHANGES.txt

(cherry picked from commit 55427fb66c6d52ce98b4d68a29b592a734014c28)
 2015-09-03 14:30:35 -07:00
Jason Lowe
f21fb808f1 YARN-3832. Resource Localization fails on a cluster due to existing cache directories. Contributed by Brahma Reddy Battula 
(cherry picked from commit 8d58512d6e6d9fe93784a9de2af0056bcc316d96)

(cherry picked from commit 15b1800b1289d239cbebc5cfd66cfe156d45a2d3)
(cherry picked from commit 38400507e3352d83c2a1f364de137366249b7983)
 2015-09-03 14:26:43 -07:00
Vinayakumar B
4f53c98ca4 HADOOP-7139. Allow appending to existing SequenceFiles (Contributed by kanaka kumar avvaru) 
(cherry picked from commit 295d678be8853a52c3ec3da43d9265478d6632b3)
(cherry picked from commit 80697e4f324948ec32b4cad3faccba55287be652)
(cherry picked from commit b3546b60340e085c5abd8f8f0990d45c7445fe07)

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt

(cherry picked from commit e9c8d8c58516aa64589cd44e9e5dd0a23ba72a17)
 2015-09-03 14:15:23 -07:00
Jason Lowe
193d8d3667 YARN-3585. NodeManager cannot exit on SHUTDOWN event triggered and NM recovery is enabled. Contributed by Rohith Sharmaks 
(cherry picked from commit e13b671aa510f553f4a6a232b4694b6a4cce88ae)

(cherry picked from commit 752caa95a40d899e1bf98bc907e91aec2bb57073)
(cherry picked from commit 13c4db632b0e7f19dcfa883c2492431c2c7d0799)
 2015-09-03 14:09:16 -07:00
Vinayakumar B
a2434dcf2a HDFS-8270. create() always retried with hardcoded timeout when file already exists with open lease (Contributed by J.Andreina) 
(cherry picked from commit 54f83d9bd917e8641e902c5f0695e65ded472f9a)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/NameNodeProxies.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileCreation.java

(cherry picked from commit 8090a6ee63c414ac5b76c21df1f6b5a188e873d6)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

(cherry picked from commit 04a7000c8d955a134d86967f4b4622ea920e1ea7)
 2015-09-03 14:06:09 -07:00
Wangda Tan
ae0fac3efa YARN-3725. App submission via REST API is broken in secure mode due to Timeline DT service address is empty. (Zhijie Shen via wangda) 
(cherry picked from commit 5cc3fced957a8471733e0e9490878bd68429fe24)
(cherry picked from commit a3734f67d35e714690ecdf21d80bce8a355381e3)
(cherry picked from commit 9ccc22e2ac89990f3e7997f1d89594523c66e76a)
 2015-09-03 13:50:14 -07:00