Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,732 Commits 326 Branches 371 Tags
Commit Graph

3289 Commits

Author SHA1 Message Date
Vinayakumar B
4e099c113b HDFS-8096. DatanodeMetrics#blocksReplicated will get incremented early and even for failed transfers (Contributed by Vinayakumar B) 
(cherry picked from commit 9d8952f97f638ede27e4336b9601507d7bb1de7b)
 2015-04-09 11:59:05 +05:30
Andrew Wang
351fac25a9 HDFS-7979. Initialize block report IDs with a random number. 
(cherry picked from commit b1e059089d6a5b2b7006d7d384c6df81ed268bd9)
 2015-04-08 21:43:57 -07:00
Haohui Mai
88ef75f167 HDFS-8089. Move o.a.h.hdfs.web.resources.* to the client jars. Contributed by Haohui Mai. 2015-04-08 16:30:57 -07:00
Andrew Wang
069366e1be HDFS-8025. Addendum fix for HDFS-3087 Decomissioning on NN restart can complete without blocks being replicated. Contributed by Ming Ma. 2015-04-08 16:09:28 -07:00
Haohui Mai
6d2eca081d Revert HDFS-7813. 2015-04-08 16:03:20 -07:00
Haohui Mai
35816c438d Revert HDFS-7808. 2015-04-08 16:00:10 -07:00
Andrew Wang
8104d52269 HDFS-7725. Incorrect 'nodes in service' metrics caused all writes to fail. Contributed by Ming Ma. 2015-04-08 15:52:15 -07:00
Andrew Wang
9792500c54 HDFS-8076. Code cleanup for DFSInputStream: use offset instead of LocatedBlock when possible. Contributed by Zhe Zhang. 
(cherry picked from commit a42bb1cd915abe5dc33eda3c01e8c74c64f35748)
 2015-04-08 15:41:54 -07:00
Kihwal Lee
7e622076d4 HDFS-8046. Allow better control of getContentSummary. Contributed by Kihwal Lee. 
(cherry picked from commit 285b31e75e51ec8e3a796c2cb0208739368ca9b8)
 2015-04-08 15:39:25 -05:00
Tsz-Wo Nicholas Sze
de7f9a8bcc HDFS-8079. Move CorruptFileBlockIterator to a new hdfs.client.impl package. 2015-04-08 11:51:43 -07:00
Arpit Agarwal
f0324738c9 HDFS-8072. Reserved RBW space is not released if client terminates while writing block. (Arpit Agarwal) 2015-04-08 11:38:30 -07:00
Haohui Mai
f9fbde3074 HDFS-8080. Separate JSON related routines used by WebHdfsFileSystem to a package local class. Contributed by Haohui Mai. 2015-04-07 21:30:23 -07:00
Vinayakumar B
3cf7ac181b HDFS-8073. Split BlockPlacementPolicyDefault.chooseTarget(..) so it can be easily overrided. (Contributed by Walter Su) 
(cherry picked from commit d505c8acd30d6f40d0632fe9c93c886a4499a9fc)
 2015-04-08 09:57:49 +05:30
Tsz-Wo Nicholas Sze
eb3ca514fb HDFS-8079. Move dfs.client.retry.* confs from DFSConfigKeys to HdfsClientConfigKeys.Retry. 2015-04-07 19:53:30 -07:00
Yongjun Zhang
b76c321e94 HDFS-5215. dfs.datanode.du.reserved is not considered while computing 
available space ( Brahma Reddy Battula via Yongjun Zhang)

(cherry picked from commit 66763bb06f107f0e072c773a5feb25903c688ddc)
 2015-04-07 19:36:02 -07:00
cnauroth
8921526440 HDFS-8038. PBImageDelimitedTextWriter#getEntry output HDFS path in platform-specific format. Contributed by Xiaoyu Yao. 
(cherry picked from commit 672ed462965ea7da10ce3df48c02e2a3bd13f0ae)
 2015-04-07 14:23:29 -07:00
Tsz-Wo Nicholas Sze
a4f0eea261 HDFS-8049. Add @InterfaceAudience.Private annotation to hdfs client implementation. Contributed by Takuya Fukudome 2015-04-07 14:01:55 -07:00
Konstantin V Shvachko
81b522790b HDFS-8071. Redundant checkFileProgress() in PART II of getAdditionalBlock(). Contributed by Konstantin Shvachko. 2015-04-06 22:23:33 -07:00
Colin Patrick Mccabe
a827089905 HDFS-7999. FsDatasetImpl#createTemporary sometimes holds the FSDatasetImpl lock for a very long time (sinago via cmccabe) 
(cherry picked from commit 28bebc81db8bb6d1bc2574de7564fe4c595cfe09)
 2015-04-06 09:10:15 -07:00
Vinod Kumar Vavilapalli
00854d2c2f HADOOP-11776. Fixed the broken JDiff support in Hadoop 2. Contributed by Li Lu. 
(cherry picked from commit 4b3948ea365db07df7a9369a271009fafd1ba8f5)
 2015-04-04 13:52:51 -07:00
Vinod Kumar Vavilapalli
4eec6e09d0 HADOOP-11377. Added Null.java without which jdiff completely flops. Contributed by Tsuyoshi Ozawa. 
(cherry picked from commit 5370e7128b4b78dabff79986a92151f1de39eeed)
 2015-04-04 13:48:14 -07:00
Colin Patrick Mccabe
b26ba22a90 HDFS-8051. FsVolumeList#addVolume should release volume reference if not put it into BlockScanner. (Lei (Eddy) Xu via Colin P. McCabe) 
(cherry picked from commit ef591b1d6a08f08358b19763a874de6010227307)
 2015-04-03 16:56:25 -07:00
Colin Patrick Mccabe
0c5069c432 HDFS-7996. After swapping a volume, BlockReceiver reports ReplicaNotFoundException (Lei (Eddy) Xu via Colin P. McCabe) 
(cherry picked from commit 023133cef9a7ca05364cefbcead57c921589eda7)
 2015-04-03 14:26:45 -07:00
cnauroth
28e0602e7b HDFS-8039. Fix TestDebugAdmin#testRecoverLease and testVerfiyBlockChecksumCommand on Windows. Contributed by Xiaoyu Yao. 
(cherry picked from commit db80e428919e4e763f56b9344c748e7f8fb46491)
 2015-04-03 10:44:32 -07:00
cnauroth
0dde8c079d HDFS-8034. Fix TestDFSClientRetries#testDFSClientConfigurationLocateFollowingBlockInitialDelay for Windows. Contributed by Xiaoyu Yao. 
(cherry picked from commit dc5596c36a261626fafda7f30638a000839d8392)
 2015-04-03 10:10:29 -07:00
Jing Zhao
2c1f33d178 HDFS-7811. Avoid recursive call getStoragePolicyID in INodeFile#computeQuotaUsage. Contributed by Xiaoyu Yao and Jing Zhao. 
(cherry picked from commit 72f6bd4893dcf10d6dad24753f9be99505a87a1f)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-04-02 21:22:43 -07:00
Haohui Mai
ddb5bb8fcf Addendum for HDFS-7893 in branch-2. 2015-04-02 16:17:14 -07:00
Haohui Mai
1278310c21 HDFS-7893. Update the POM to create a separate hdfs-client jar. Contributed by Haohui Mai. 2015-04-02 15:39:40 -07:00
Haohui Mai
6e36dbf03a HDFS-8035. Move checkBlocksProperlyReplicated() in FSNamesystem to BlockManager. Contributed by Haohui Mai. 2015-04-02 11:38:02 -07:00
Tsz-Wo Nicholas Sze
81ed3a2bb8 HDFS-7954. TestBalancer#testBalancerWithPinnedBlocks should not be executed on Windows. Contributed by Xiaoyu Yao 2015-04-02 11:12:27 -07:00
Tsz-Wo Nicholas Sze
ce3f32590d HDFS-7888. Change DFSOutputStream and DataStreamer for convenience of subclassing. Contributed by Li Bo 2015-04-02 11:00:47 -07:00
Vinayakumar B
beb0fd0d60 HDFS-7916. 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop (Contributed by Vinayakumar B) 
(cherry picked from commit 867d5d2675b8fb73c40fac1e581b02b005459d95)
 2015-04-02 08:12:51 +05:30
Brandon Li
27970b3f46 HDFS-8001 RpcProgramNfs3 : wrong parsing of dfs.blocksize. Contributed by Remi Catherinot 
(cherry picked from commit 4d14816c269f110445e1ad3e03ac53b0c1cdb58b)
 2015-04-01 17:25:28 -07:00
Haohui Mai
cfcf795492 HDFS-8008. Support client-side back off when the datanodes are congested. Contributed by Haohui Mai. 2015-04-01 16:54:53 -07:00
Haohui Mai
9e114ee607 HDFS-8036. Use snapshot path as source when using snapshot diff report in DistCp. Contributed by Jing Zhao. 2015-04-01 16:51:09 -07:00
Colin Patrick Mccabe
6f753da4a9 HDFS-7922. ShortCircuitCache#close is not releasing ScheduledThreadPoolExecutors (Rakesh R via Colin P. McCabe) 
(cherry picked from commit 3c7adaaf3571c91fee80585472d2a81402a53e2b)
 2015-04-01 16:11:44 -07:00
Colin Patrick Mccabe
4b74aa7182 HDFS-8026. Trace FSOutputSummer#writeChecksumChunks rather than DFSOutputStream#writeChunk (cmccabe) 
(cherry picked from commit c94d594a57806dec515e2a2053a1221f8ce48cc4)
 2015-04-01 14:10:25 -07:00
Andrew Wang
b17d8a53fa HDFS-7978. Add LOG.isDebugEnabled() guard for some LOG.debug(..). Contributed by Walter Su. 
(cherry picked from commit ed72daa5df97669906234e8ac9a406d78136b206)
 2015-04-01 12:54:46 -07:00
Haohui Mai
2ca2515b33 Add the missing files for HDFS-8009. 2015-04-01 10:58:41 -07:00
Haohui Mai
a5bcfe0d33 HDFS-8009. Signal congestion on the DataNode. Contributed by Haohui Mai. 2015-04-01 10:57:07 -07:00
Akira Ajisaka
b85bbca745 HDFS-6945. BlockManager should remove a block from excessReplicateMap and decrement ExcessBlocks metric when the block is removed. (aajisaka) 
(cherry picked from commit 18a91fe4df0448d9f7de91602646ecf5a51c52e4)
 2015-04-01 09:08:22 +09:00
Akira Ajisaka
7f6811ad73 HDFS-7671. hdfs user guide should point to the common rack awareness doc. Contributed by Kai Sasaki. 
(cherry picked from commit 859cab2f2273f563fd70e3e616758edef91ccf41)
 2015-04-01 00:26:54 +09:00
yliu
e4cd67e9b0 HDFS-7997. The first non-existing xattr should also throw IOException. (zhouyingchao via yliu) 2015-03-31 21:16:13 +08:00
Arpit Agarwal
312e50f9c4 HDFS-7944. Minor cleanup of BlockPoolManager#getAllNamenodeThreads. (Arpit Agarwal) 2015-03-30 21:24:59 -07:00
Harsh J
32766b6563 HDFS-3918. EditLogTailer shouldn't log WARN when other node is in standby mode. Contributed by Todd Lipcon. 
(cherry picked from commit cce66ba3c9ec293e8ba1afd0eb518c7ca0bbc7c9)
 2015-03-31 08:17:22 +05:30
Arpit Agarwal
abf3ad988d HDFS-7645. Rolling upgrade is restoring blocks from trash multiple times (Contributed by Vinayakumar B and Keisuke Ogiwara) 2015-03-30 15:25:26 -07:00
Haohui Mai
02fcf622fb Addendum for HDFS-7748. 2015-03-30 13:11:24 -07:00
Haohui Mai
dd5b2dac5a HDFS-7748. Separate ECN flags from the Status in the DataTransferPipelineAck. Contributed by Anu Engineer and Haohui Mai. 2015-03-30 11:59:32 -07:00
Colin Patrick Mccabe
02ed22cd2d HDFS-7261. storageMap is accessed without synchronization in DatanodeDescriptor#updateHeartbeatState() (Brahma Reddy Battula via Colin P. McCabe) 
(cherry picked from commit 1feb9569f366a29ecb43592d71ee21023162c18f)
 2015-03-30 10:53:57 -07:00
Akira Ajisaka
d4bb9b2146 HDFS-8002. Website refers to /trash directory. Contributd by Brahma Reddy Battula. 
(cherry picked from commit e7ea2a8e8f0a7b428ef10552885757b99b59e4dc)
 2015-03-31 00:28:27 +09:00