Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
7,894 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

7894 Commits

Author SHA1 Message Date
Vinod Kumar Vavilapalli (I am also known as @tshooter.) e8c4544d8e YARN-3248. Moving it to improvements section in CHANGES.txt. 
(cherry picked from commit df52ff92ae)
 2015-09-23 15:04:39 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) 4697d94c6a CHANGES.txt: Dropping YARN-3379 as only the refactor part of it exists in branch-2.6.1. 2015-09-23 13:52:16 -07:00
Vinod Kumar Vavilapalli b4d876d837 Set the release date for 2.6.1-RC1 2015-09-16 12:04:57 -07:00
Hitesh Shah dba2b60fdd YARN-2890. MiniYarnCluster should turn on timeline service if configured to do so. Contributed by Mit Desai. 
(cherry picked from commit 265ed1fe80)
(cherry picked from commit 55b794e7fa)
 2015-09-15 17:30:15 -07:00
Vinod Kumar Vavilapalli 9c4a6e1270 Revert "YARN-2890. MiniYARNCluster should start the timeline server based on the configuration. Contributed by Mit Desai." 
This reverts commit 8a47d1aa55.
 2015-09-15 17:30:15 -07:00
Zhijie Shen d57c3f0a26 YARN-3544. Got back AM logs link on the RM web UI for a completed app. Contributed by Xuan Gong. 
(cherry picked from commit 21bf2cdcb77f69abc906e6cd401a8fb221f250e9)
(cherry picked from commit c9ee316045)
 2015-09-15 17:30:15 -07:00
Xuan 7af5d6b4ba YARN-3248. Display count of nodes blacklisted by apps in the web UI. 
Contributed by Varun Vasudev

(cherry picked from commit 4728bdfa15)
(cherry picked from commit e26b6e55e9)
 2015-09-15 17:30:06 -07:00
Jian He e914220ab9 YARN-3379. Fixed missing data in localityTable and ResourceRequests table in RM WebUI. Contributed by Xuan Gong 
(cherry picked from commit 4e886eb9cb)

(cherry picked from commit 3f0c9e5fe3)
 2015-09-14 12:54:01 -07:00
Zhijie Shen 3ab820e696 YARN-3740. Fixed the typo in the configuration name: APPLICATION_HISTORY_PREFIX_MAX_APPS. Contributed by Xuan Gong. 
(cherry picked from commit eb6bf91eea)
(cherry picked from commit 68cddb894a)
 2015-09-11 11:53:51 -07:00
Xuan 85ec6eb37a YARN-3171. Sort by Application id, AppAttempt and ContainerID doesn't 
work in ATS / RM web ui. Contributed by Naganarasimha G R

(cherry picked from commit 3ff1ba2a7b)
(cherry picked from commit b6eb36dbdc)
 2015-09-11 11:49:11 -07:00
Zhijie Shen f4154bdee8 YARN-1884. Added nodeHttpAddress into ContainerReport and fixed the link to NM web page. Contributed by Xuan Gong. 
(cherry picked from commit 85f6d67fa7)
(cherry picked from commit 426535007b)
 2015-09-11 11:45:29 -07:00
Vinod Kumar Vavilapalli 075d899d10 Set the release date for 2.6.1-RC0 2015-09-09 15:39:02 -07:00
Vinod Kumar Vavilapalli c86d9bd10b Preparing for release 2.6.1: Posting release notes generated. 2015-09-09 15:31:14 -07:00
Vinod Kumar Vavilapalli 3462a00dd2 Preparing for release 2.6.1: mvn versions:set -DnewVersion=2.6.1 2015-09-09 15:29:57 -07:00
Vinod Kumar Vavilapalli cfb012a255 Preparing for release 2.6.1: Updating site update page. 2015-09-09 15:29:03 -07:00
Colin Patrick Mccabe adc3405321 HDFS-7552. Change FsVolumeList toString() to fix TestDataNodeVolumeFailureToleration (Liang Xie via Colin P. McCabe) 
(cherry picked from commit a4876c130f)

(cherry picked from commit 22f72c7946)
 2015-09-09 14:59:31 -07:00
yliu 0adcfe6c7e HDFS-8863. The remaining space check in BlockPlacementPolicyDefault is flawed. (Kihwal Lee via yliu) 
(cherry picked from commit 146db49f7f)
 2015-09-08 22:57:36 -07:00
Jian He 2b526ba757 YARN-4047. ClientRMService getApplications has high scheduler lock contention. Contributed by Jason Lowe 
(cherry picked from commit 7a445fcfab)

(cherry picked from commit 703fa1b141)
 2015-09-08 22:57:35 -07:00
Xuan d59bf81e08 YARN-3999. RM hangs on draing events. Contributed by Jian He 
(cherry picked from commit 3ae716fa69)
(cherry picked from commit 2ebdf5bfce)
 2015-09-08 22:57:35 -07:00
Jonathan Eagles 6ed2486c7e YARN-3978. Configurably turn off the saving of container info in Generic AHS (Eric Payne via jeagles) 
(cherry picked from commit 3cd02b9522)
(cherry picked from commit 899df5bce0)
 2015-09-08 22:57:34 -07:00
Jian He 92742b4402 YARN-2301. Improved yarn container command. Contributed by Naganarasimha G R 
(cherry picked from commit 258623ff8b)

(cherry picked from commit 1d1e7682c9)
 2015-09-08 22:57:28 -07:00
cnauroth c3928b3068 HDFS-7570. SecondaryNameNode need twice memory when calling reloadFromImageFile. Contributed by zhaoyunjiong. 
(cherry picked from commit 85aec75ce5)
(cherry picked from commit 27eecaef10)
 2015-09-08 17:21:58 -07:00
Vinod Kumar Vavilapalli 33537078a8 HDFS-8384. Allow NN to startup if there are files having a lease but are not under construction. Contributed by Jing Zhao. 2015-09-08 16:45:38 -07:00
Colin Patrick Mccabe a976acc02d HDFS-8846. Add a unit test for INotify functionality across a layout version upgrade (Zhe Zhang via Colin P. McCabe) 
(cherry picked from commit a4d9acc51d)
(cherry picked from commit 9264b7e119)
 2015-09-08 16:18:30 -07:00
Colin Patrick Mccabe 2e86d1f254 HDFS-8480. Fix performance and timeout issues in HDFS-7929 by using hard-links to preserve old edit logs, instead of copying them. (Zhe Zhang via Colin P. McCabe) 
(cherry picked from commit 7b424f938c)
(cherry picked from commit cbd11681ce)
 2015-09-08 16:12:45 -07:00
Colin Patrick Mccabe 4363145128 HDFS-7446. HDFS inotify should have the ability to determine what txid it has read up to (cmccabe) 
(cherry picked from commit 75a326aaff)

(cherry picked from commit 06552a15d5)
 2015-09-08 15:32:17 -07:00
Tsz-Wo Nicholas Sze 8ed162bcbf HDFS-7980. Incremental BlockReport will dramatically slow down namenode startup. Contributed by Walter Su 
(cherry picked from commit 4e1f2eb395)
 2015-09-07 11:46:33 -07:00
cnauroth c76f5b3188 HADOOP-11934. Use of JavaKeyStoreProvider in LdapGroupsMapping causes infinite loop. Contributed by Larry McCay. 
(cherry picked from commit 860b8373c3)
(cherry picked from commit 33648268ce)
 2015-09-07 11:09:20 -07:00
Jian He 2336264900 YARN-2918. RM should not fail on startup if queue's configured labels do not exist in cluster-node-labels. Contributed by Wangda Tan 
(cherry picked from commit f489a4ec96)

(cherry picked from commit d817fbb34d)
 2015-09-06 14:15:33 -07:00
Jian He ee2b6bc248 YARN-3124. Fixed CS LeafQueue/ParentQueue to use QueueCapacities to track capacities-by-label. Contributed by Wangda Tan 
(cherry picked from commit 18a594257e)

(cherry picked from commit 1be2d64ddd)
 2015-09-06 11:54:40 -07:00
Jian He 637e7f9e39 YARN-2694. Ensure only single node label specified in ResourceRequest. Contributed by Wangda Tan 
(cherry picked from commit c1957fef29)

(cherry picked from commit 3ddafaa7c8)
 2015-09-05 21:07:51 -07:00
Jian He 4c94f07140 YARN-3098. Created common QueueCapacities class in Capacity Scheduler to track capacities-by-labels of queues. Contributed by Wangda Tan 
(cherry picked from commit 21d80b3dd9)

(cherry picked from commit c0b1311a93)
 2015-09-05 20:54:20 -07:00
Jian He d9281fbbab YARN-3099. Capacity Scheduler LeafQueue/ParentQueue should use ResourceUsage to track used-resources-by-label. Contributed by Wangda Tan 
(cherry picked from commit 86358221fc)

(cherry picked from commit cabf97ae4f)
 2015-09-05 20:54:20 -07:00
Jian He b0ad553841 YARN-3092. Created a common ResourceUsage class to track labeled resource usages in Capacity Scheduler. Contributed by Wangda Tan 
(cherry picked from commit 6f9fe76918)

(cherry picked from commit 61b4116b4b)
 2015-09-05 20:54:19 -07:00
Jian He 419e18cb37 YARN-2978. Fixed potential NPE while getting queue info. Contributed by Varun Saxena 
(cherry picked from commit dd57c2047b)

(cherry picked from commit c61e8a7bfa)
 2015-09-05 20:54:19 -07:00
Jian He 88f022da24 YARN-2920. Changed CapacityScheduler to kill containers on nodes where node labels are changed. Contributed by Wangda Tan 
(cherry picked from commit fdf042dfff)

(cherry picked from commit 411836b74c)
 2015-09-05 20:54:18 -07:00
Wangda Tan 2073fc0f84 Add missing test file of YARN-3733 
(cherry picked from commit 405bbcf68c)
(cherry picked from commit 344b750915)
 2015-09-03 17:43:03 -07:00
Wangda Tan 85d92721a4 YARN-3733. Fix DominantRC#compare() does not work as expected if cluster resource is empty. (Rohith Sharmaks via wangda) 
(cherry picked from commit ebd797c48f)
(cherry picked from commit 78d626fa89)
 2015-09-03 17:43:01 -07:00
Jian He f1b35ffd4c YARN-2637. Fixed max-am-resource-percent calculation in CapacityScheduler when activating applications. Contributed by Craig Welch 
(cherry picked from commit c53420f583)

(cherry picked from commit 4931600030)
 2015-09-03 17:40:24 -07:00
Colin Patrick Mccabe 3bd9b7459b HADOOP-12280. Skip unit tests based on maven profile rather than NativeCodeLoader.isNativeCodeLoaded (Masatake Iwasaki via Colin P. McCabe) 
(cherry picked from commit 6f83274afc)
(cherry picked from commit e92107b18f)
 2015-09-03 15:02:33 -07:00
Jian He d8ddfea491 HADOOP-11932. MetricsSinkAdapter may hang when being stopped. Contributed by Brahma Reddy Battula 
(cherry picked from commit f59612edd7)

(cherry picked from commit 5950c1f6f8)
 2015-09-03 14:45:04 -07:00
Jason Lowe ca7fe71000 YARN-3990. AsyncDispatcher may overloaded with RMAppNodeUpdateEvent when Node is connected/disconnected. Contributed by Bibin A Chundatt 
(cherry picked from commit 32e490b6c0)

(cherry picked from commit c31e3ba921)
(cherry picked from commit 07d31d4c0808a169f4770187d655f38aa105255c)
 2015-09-03 14:40:20 -07:00
Jason Lowe fe5877a49e YARN-3850. NM fails to read files from full disks which can lead to container logs being lost and other issues. Contributed by Varun Saxena 
(cherry picked from commit 40b256949a)

(cherry picked from commit 0221d19f4e)
(cherry picked from commit 87d2204f28f192a964c04a5fa1e2e31644d74b59)
 2015-09-03 14:35:01 -07:00
Harsh J 34739fc91e HADOOP-8151. Error handling in snappy decompressor throws invalid exceptions. Contributed by Matt Foley. (harsh) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1389006 13f79535-47bb-0310-9956-ffa450edef68
(cherry picked from commit ac31d6a448)

Conflicts:

	hadoop-common-project/hadoop-common/CHANGES.txt

(cherry picked from commit 55427fb66c)
 2015-09-03 14:30:35 -07:00
Jason Lowe f21fb808f1 YARN-3832. Resource Localization fails on a cluster due to existing cache directories. Contributed by Brahma Reddy Battula 
(cherry picked from commit 8d58512d6e)

(cherry picked from commit 15b1800b12)
(cherry picked from commit 38400507e3352d83c2a1f364de137366249b7983)
 2015-09-03 14:26:43 -07:00
Vinayakumar B 4f53c98ca4 HADOOP-7139. Allow appending to existing SequenceFiles (Contributed by kanaka kumar avvaru) 
(cherry picked from commit 295d678be8)
(cherry picked from commit 80697e4f32)
(cherry picked from commit b3546b6034)

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt

(cherry picked from commit e9c8d8c58516aa64589cd44e9e5dd0a23ba72a17)
 2015-09-03 14:15:23 -07:00
Jason Lowe 193d8d3667 YARN-3585. NodeManager cannot exit on SHUTDOWN event triggered and NM recovery is enabled. Contributed by Rohith Sharmaks 
(cherry picked from commit e13b671aa5)

(cherry picked from commit 752caa95a4)
(cherry picked from commit 13c4db632b0e7f19dcfa883c2492431c2c7d0799)
 2015-09-03 14:09:16 -07:00
Vinayakumar B a2434dcf2a HDFS-8270. create() always retried with hardcoded timeout when file already exists with open lease (Contributed by J.Andreina) 
(cherry picked from commit 54f83d9bd9)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/NameNodeProxies.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileCreation.java

(cherry picked from commit 8090a6ee63)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

(cherry picked from commit 04a7000c8d955a134d86967f4b4622ea920e1ea7)
 2015-09-03 14:06:09 -07:00
Wangda Tan ae0fac3efa YARN-3725. App submission via REST API is broken in secure mode due to Timeline DT service address is empty. (Zhijie Shen via wangda) 
(cherry picked from commit 5cc3fced95)
(cherry picked from commit a3734f67d3)
(cherry picked from commit 9ccc22e2ac89990f3e7997f1d89594523c66e76a)
 2015-09-03 13:50:14 -07:00
Xuan 1c6a287bf5 YARN-2900. Application (Attempt and Container) Not Found in AHS results 
in Internal Server Error (500). Contributed by Zhijie Shen and Mit Desai

(cherry picked from commit 06f8e9caba)
(cherry picked from commit 4fee8b3202)
(cherry picked from commit 6c7b625138ce3b262a8c8aa28077074b553638ed)
 2015-09-03 13:45:06 -07:00