Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
10,038 Commits 326 Branches 371 Tags
Commit Graph

10038 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Haohui Mai
bc0487c29c HDFS-8696. Make the lower and higher watermark in the DN Netty server configurable. Contributed by Xiaobing Zhou. 2015-09-29 14:59:35 -07:00
Haohui Mai
baa6ef0207 HDFS-9174. Fix findbugs warnings in FSOutputSummer.tracer and DirectoryScanner$ReportCompiler.currentThread. Contributed by Yi Liu. 2015-09-29 14:56:13 -07:00
Haohui Mai
cf6fb09162 HDFS-9166. Move hftp / hsftp filesystem to hfds-client. Contributed by Mingliang Liu. 2015-09-29 10:07:19 -07:00
Haohui Mai
c705eab221 HDFS-9165. Move entries in META-INF/services/o.a.h.fs.FileSystem to hdfs-client. Contributed by Mingliang Liu. 2015-09-29 09:51:30 -07:00
Colin Patrick Mccabe
171bd1cb2b HADOOP-12447. Clean up some htrace integration issues (cmccabe) 
(cherry picked from commit 850d679acb935a6a6b0e6cb6f69d998e99395468)
 2015-09-29 09:38:10 -07:00
Karthik Kambatla
64a7617eb9 YARN-4066. Large number of queues choke fair scheduler. (Johan Gustavsson via kasha) 
(cherry picked from commit a0b5a0a419dfc07b7ac45c06b11b4c8dc7e79958)
 2015-09-29 08:13:25 -07:00
yliu
28805cb8d7 HDFS-9141. Thread leak in Datanode#refreshVolumes. (Uma Maheswara Rao G via yliu) 2015-09-29 22:06:20 +08:00
yliu
9c47ab32b1 HDFS-8859. Improve DataNode ReplicaMap memory footprint to save about 45%. (yliu) 2015-09-29 16:23:50 +08:00
Yongjun Zhang
1a5f3e93c3 HDFS-9092. Nfs silently drops overlapping write requests and causes data copying to fail. Contributed by Yongjun Zhang. 
(cherry picked from commit 151fca5032719e561226ef278e002739073c23ec)
 2015-09-28 18:47:47 -07:00
Ming Ma
ac7d85efb2 HADOOP-12440. TestRPC#testRPCServerShutdown did not produce the desired thread states before shutting down. (Xiao Chen via mingma) 
(cherry picked from commit 5c3b663bf95551d1cf36a2a39849e0676893fa1d)
 2015-09-28 18:14:29 -07:00
Anubhav Dhoot
22f2501476 YARN-4180. AMLauncher does not retry on failures when talking to NM. (adhoot) 
(cherry picked from commit 9735afe967a660f356e953348cb6c34417f41055)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationMasterLauncher.java
 2015-09-28 16:54:06 -07:00
Jason Lowe
600022ae69 YARN-4141. Runtime Application Priority change should not throw exception for applications at finishing states. Contributed by Sunil G 
(cherry picked from commit 9f53a95ff624f66a774fe3defeea4a3454f4c4af)
 2015-09-28 22:56:07 +00:00
Kihwal Lee
95b8f1f3a6 HADOOP-12446. Undeprecate createNonRecursive(). Contributed by Ted Yu. 
(cherry picked from commit 3abbdc929bde05f8819f5410cef1eaeb8940203f)
 2015-09-28 17:33:20 -05:00
Jason Lowe
3c4c52a095 MAPREDUCE-6492. AsyncDispatcher exit with NPE on TaskAttemptImpl#sendJHStartEventForAssignedFailTask. Contributed by Bibin A Chundatt 
(cherry picked from commit ab11085b81353e1617875deb10f3c8e2a8b91a1e)
 2015-09-28 22:13:03 +00:00
Lei Xu
213ff984ee HDFS-9147. Fix the setting of visibleLength in ExternalBlockReader. (Colin P. McCabe via Lei (Eddy) Xu) 
(cherry picked from commit e5992ef4df63fbc6a6b8e357b32c647e7837c662)
 2015-09-28 13:43:52 -07:00
Lei Xu
74092f2144 HDFS-9148. Incorrect assert message in TestWriteToReplica#testWriteToTemporary (Tony Wu via Lei (Eddy) Xu) 
(cherry picked from commit 50741cb568d4da30b92d4954928bc3039e583b22)
 2015-09-28 13:28:08 -07:00
Kihwal Lee
468b330feb HDFS-9106. Transfer failure during pipeline recovery causes permanent write failures. Contributed by Kihwal Lee. 
(cherry picked from commit 4c9497cbf02ecc82532a4e79e18912d8e0eb4731)
 2015-09-28 14:30:06 -05:00
Anubhav Dhoot
4d8b99423e YARN-4204. ConcurrentModificationException in FairSchedulerQueueInfo. (adhoot) 
(cherry picked from commit fb2e525c0775ccf218c8980676e9fb4005a406a6)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-09-28 09:39:52 -07:00
Colin Patrick Mccabe
3b9d32627f HDFS-9080. Update htrace version to 4.0.1 (cmccabe) 
(cherry picked from commit 892ade689f9bcce76daae8f66fc00a49bee8548e)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
        hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/BlockStorageLocationUtil.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
 2015-09-28 08:15:00 -07:00
Harsh J
03e370e306 MAPREDUCE-6471. Document distcp incremental copy. Contributed by Neelesh Srinivas Salian. 
(cherry picked from commit 66dad854c0aea8c137017fcf198b165cc1bd8bdd)
 2015-09-28 13:14:20 +05:30
Haohui Mai
504d6fd950 HDFS-8740. Move DistributedFileSystem to hadoop-hdfs-client. Contributed by Mingliang Liu. 2015-09-27 10:54:53 -07:00
Vinayakumar B
1c7b3b98c1 HADOOP-11984. Enable parallel JUnit tests in pre-commit (Contributed by Chris Nauroth) 
(cherry picked from commit f0f984e4e63d0dbafe93062a122ee051330db301)
 2015-09-27 14:12:34 +05:30
Colin Patrick Mccabe
2dca55be00 HDFS-9087. Add some jitter to DataNode.checkDiskErrorThread (Elliott Clark via Colin P. McCabe) 
(cherry picked from commit 0b31c237f2622e256726fc5d7698f0f195dbdbc1)
 2015-09-26 20:37:54 -07:00
Haohui Mai
94cbb6d164 HDFS-8053. Move DFSIn/OutputStream and related classes to hadoop-hdfs-client. Contributed by Mingliang Liu. 2015-09-26 11:16:50 -07:00
Vinayakumar B
b46e4ceafd HADOOP-12442. Display help if the command option to 'hdfs dfs' is not valid (Contributed by nijel) 
(cherry picked from commit 861b52db242f238d7e36ad75c158025be959a696)
 2015-09-26 21:37:24 +05:30
Colin Patrick Mccabe
e5dc723d48 HDFS-8873. Allow the directoryScanner to be rate-limited (Daniel Templeton via Colin P. McCabe) 
(cherry picked from commit 7a3c381b39887a02e944fa98287afd0eb4db3560)
 2015-09-26 04:21:23 -07:00
cnauroth
85c0cb0075 HADOOP-11918. Listing an empty s3a root directory throws FileNotFound. Contributed by Lei (Eddy) Xu. 
(cherry picked from commit 7fe521b1dd49f81ae325f78cf531cfff15be6641)
 2015-09-25 22:34:10 -07:00
Lei Xu
82d4c05b2d HDFS-9133. ExternalBlockReader and ReplicaAccessor need to return -1 on read when at EOF. (Colin Patrick McCabe via Lei (Eddy) Xu) 
(cherry picked from commit 67b0e967f0e13eb6bed123fc7ba4cce0dcca198f)
 2015-09-25 16:02:27 -07:00
Lei Xu
1a2870f6ff HDFS-9132. Pass genstamp to ReplicaAccessorBuilder. (Colin Patrick McCabe via Lei (Eddy) Xu) 2015-09-25 15:51:29 -07:00
Colin Patrick Mccabe
a6166aa552 HDFS-9107. Prevent NN's unrecoverable death spiral after full GC (Daryn Sharp via Colin P. McCabe) 
(cherry-picked from 4e7c6a653f108d44589f84d78a03d92ee0e8a3c3)
 2015-09-25 15:33:02 -07:00
Robert Kanter
d8a5d2b2fc MAPREDUCE-6480. archive-logs tool may miss applications (rkanter) 
(cherry picked from commit 8fec882cfaac047af1892615fe32d8c3ca02e7f9)
 2015-09-25 15:00:07 -07:00
Jing Zhao
aca014ea67 HDFS-9112. Improve error message for Haadmin when multiple name service IDs are configured. Contributed by Anu Engineer. 
(cherry picked from commit 83e99d06d0e5a71888aab33e9ae47460e9f1231f)
 2015-09-25 14:21:52 -07:00
Yongjun Zhang
bc5ee80794 HDFS-5795. RemoteBlockReader2#checkSuccess() shoud print error status. (Xiao Chen via Yongjun Zhang) 
(cherry picked from commit 83e65c5fe84819b6c6da015b269fb4e46a88d105)
 2015-09-24 22:15:28 -07:00
Yongjun Zhang
16ae3f5d46 HDFS-9123. Copying from the root to a subdirectory should be forbidden. (Wei-Chiu Chuang via Yongjun Zhang) 
(cherry picked from commit 3187dbd7a8b6376134a00e5c9c4693d8eb762c9d)
 2015-09-24 21:41:48 -07:00
Haohui Mai
85c73d159e HDFS-9134. Move LEASE_{SOFTLIMIT,HARDLIMIT}_PERIOD constants from HdfsServerConstants to HdfsConstants. Contributed by Mingliang Liu. 2015-09-24 20:29:52 -07:00
Xuan
bf3f08cc63 YARN-3624. ApplicationHistoryServer should not reverse the order of the 
filters it gets. Contributed by Mit Desai

(cherry picked from commit d1b9b852442cb93b413eff8a137db6213e730d9c)
 2015-09-24 15:03:05 -07:00
Zhihai Xu
4ba0febd58 HADOOP-12252. LocalDirAllocator should not throw NPE with empty string configuration. Contributed by Zhihai Xu 
(cherry picked from commit 52c1f272ecb6c29c81898a1ff50d03a1296df1f7)

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
 2015-09-24 12:40:39 -07:00
Harsh J
9bb6fba759 HADOOP-8436. NPE In getLocalPathForWrite ( path, conf ) when the required context item is not configured. Contributed by Brahma Reddy Battula. (harsh) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1389799 13f79535-47bb-0310-9956-ffa450edef68
(cherry picked from commit a7e450c7cc0adef3ff832368a2a041bb51396945)

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
 2015-09-24 12:30:55 -07:00
Arpit Agarwal
3c06162259 HADOOP-12437. Allow SecurityUtil to lookup alternate hostnames. (Contributed by Arpit Agarwal) 2015-09-24 11:41:58 -07:00
Jian He
ea2b6ee9e9 YARN-4171. Fix findbugs warnings in YARN-1197 branch. Contributed by Wangda Tan 
(cherry picked from commit b3f6b641dccb0d59df78855e2951d2cae7dff8ad)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-09-24 10:30:49 -07:00
Jian He
d458805690 YARN-1651. CapacityScheduler side changes to support container resize. Contributed by Wangda Tan 
(cherry picked from commit 89cab1ba5f0671f8ef30dbe7432079c18362b434)

Conflicts:
	hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/SLSCapacityScheduler.java
 2015-09-24 10:30:04 -07:00
Jian He
d592aa115b YARN-3868. Recovery support for container resizing. Contributed by Meng Ding 
(cherry picked from commit c57eac5dfe277845ab4522a1188023a73ee41539)
 2015-09-24 10:24:13 -07:00
Jian He
b6c594c6ea YARN-1644. RM-NM protocol changes and NodeStatusUpdater implementation to support container resizing. Contributed by Meng Ding 
(cherry picked from commit c3dc1af072574f5890a8d43e4d60526951b4b8bc)
 2015-09-24 10:24:04 -07:00
Jian He
b8955d81c5 YARN-1643. Make ContainersMonitor support changing monitoring size of an allocated container. Contributed by Meng Ding and Wangda Tan 
(cherry picked from commit c59ae4eeb17e52e8fc659b9962d20628719fc621)
 2015-09-24 10:23:55 -07:00
Jian He
fa0a554ae0 YARN-3867. ContainerImpl changes to support container resizing. Contributed by Meng Ding 
(cherry picked from commit 5f5a968d65c44a831176764439e00db9203999ed)
 2015-09-24 10:23:46 -07:00
Jian He
afe4afd0c9 YARN-1645. ContainerManager implementation to support container resizing. Contributed by Meng Ding & Wangda Tan 
(cherry picked from commit ffd820c27a4f8cf4676ad8758696ed89fde80218)
 2015-09-24 10:23:35 -07:00
Jian He
c19e8ffdc7 YARN-1449. AM-NM protocol changes to support container resizing. Contributed by Meng Ding & Wangda Tan) 
(cherry picked from commit 83a18add10ee937a04e833a66e0a4642e776e510)
 2015-09-24 10:23:25 -07:00
Jian He
89dc37450c YARN-3866. AM-RM protocol changes to support container resizing. Contributed by Meng Ding 
(cherry picked from commit dfe2cb849fcb273d32e840d8562eaaee904af054)
 2015-09-24 10:23:13 -07:00
Haohui Mai
13bdc20d2b HDFS-7529. Consolidate encryption zone related implementation into a single class. Contributed by Rakesh R. 2015-09-24 08:46:44 -07:00
Zhihai Xu
e5ba1a0521 MAPREDUCE-6484. Yarn Client uses local address instead of RM address as token renewer in a secure cluster when RM HA is enabled. Contributed by Zhihai Xu 
(cherry picked from commit 97a08807ec328a6f455d941fe3d9a469d885caa6)
 2015-09-24 08:14:11 -07:00