Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
8,778 Commits 326 Branches 371 Tags
Commit Graph

2589 Commits

Author SHA1 Message Date
Colin Patrick Mccabe
a827089905 HDFS-7999. FsDatasetImpl#createTemporary sometimes holds the FSDatasetImpl lock for a very long time (sinago via cmccabe) 
(cherry picked from commit 28bebc81db8bb6d1bc2574de7564fe4c595cfe09)
 2015-04-06 09:10:15 -07:00
Colin Patrick Mccabe
b26ba22a90 HDFS-8051. FsVolumeList#addVolume should release volume reference if not put it into BlockScanner. (Lei (Eddy) Xu via Colin P. McCabe) 
(cherry picked from commit ef591b1d6a08f08358b19763a874de6010227307)
 2015-04-03 16:56:25 -07:00
Colin Patrick Mccabe
0c5069c432 HDFS-7996. After swapping a volume, BlockReceiver reports ReplicaNotFoundException (Lei (Eddy) Xu via Colin P. McCabe) 
(cherry picked from commit 023133cef9a7ca05364cefbcead57c921589eda7)
 2015-04-03 14:26:45 -07:00
cnauroth
28e0602e7b HDFS-8039. Fix TestDebugAdmin#testRecoverLease and testVerfiyBlockChecksumCommand on Windows. Contributed by Xiaoyu Yao. 
(cherry picked from commit db80e428919e4e763f56b9344c748e7f8fb46491)
 2015-04-03 10:44:32 -07:00
cnauroth
0dde8c079d HDFS-8034. Fix TestDFSClientRetries#testDFSClientConfigurationLocateFollowingBlockInitialDelay for Windows. Contributed by Xiaoyu Yao. 
(cherry picked from commit dc5596c36a261626fafda7f30638a000839d8392)
 2015-04-03 10:10:29 -07:00
Jing Zhao
2c1f33d178 HDFS-7811. Avoid recursive call getStoragePolicyID in INodeFile#computeQuotaUsage. Contributed by Xiaoyu Yao and Jing Zhao. 
(cherry picked from commit 72f6bd4893dcf10d6dad24753f9be99505a87a1f)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-04-02 21:22:43 -07:00
Haohui Mai
6e36dbf03a HDFS-8035. Move checkBlocksProperlyReplicated() in FSNamesystem to BlockManager. Contributed by Haohui Mai. 2015-04-02 11:38:02 -07:00
Tsz-Wo Nicholas Sze
81ed3a2bb8 HDFS-7954. TestBalancer#testBalancerWithPinnedBlocks should not be executed on Windows. Contributed by Xiaoyu Yao 2015-04-02 11:12:27 -07:00
Tsz-Wo Nicholas Sze
ce3f32590d HDFS-7888. Change DFSOutputStream and DataStreamer for convenience of subclassing. Contributed by Li Bo 2015-04-02 11:00:47 -07:00
Vinayakumar B
beb0fd0d60 HDFS-7916. 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop (Contributed by Vinayakumar B) 
(cherry picked from commit 867d5d2675b8fb73c40fac1e581b02b005459d95)
 2015-04-02 08:12:51 +05:30
Brandon Li
27970b3f46 HDFS-8001 RpcProgramNfs3 : wrong parsing of dfs.blocksize. Contributed by Remi Catherinot 
(cherry picked from commit 4d14816c269f110445e1ad3e03ac53b0c1cdb58b)
 2015-04-01 17:25:28 -07:00
Haohui Mai
cfcf795492 HDFS-8008. Support client-side back off when the datanodes are congested. Contributed by Haohui Mai. 2015-04-01 16:54:53 -07:00
Haohui Mai
9e114ee607 HDFS-8036. Use snapshot path as source when using snapshot diff report in DistCp. Contributed by Jing Zhao. 2015-04-01 16:51:09 -07:00
Colin Patrick Mccabe
6f753da4a9 HDFS-7922. ShortCircuitCache#close is not releasing ScheduledThreadPoolExecutors (Rakesh R via Colin P. McCabe) 
(cherry picked from commit 3c7adaaf3571c91fee80585472d2a81402a53e2b)
 2015-04-01 16:11:44 -07:00
Colin Patrick Mccabe
4b74aa7182 HDFS-8026. Trace FSOutputSummer#writeChecksumChunks rather than DFSOutputStream#writeChunk (cmccabe) 
(cherry picked from commit c94d594a57806dec515e2a2053a1221f8ce48cc4)
 2015-04-01 14:10:25 -07:00
Andrew Wang
b17d8a53fa HDFS-7978. Add LOG.isDebugEnabled() guard for some LOG.debug(..). Contributed by Walter Su. 
(cherry picked from commit ed72daa5df97669906234e8ac9a406d78136b206)
 2015-04-01 12:54:46 -07:00
Haohui Mai
a5bcfe0d33 HDFS-8009. Signal congestion on the DataNode. Contributed by Haohui Mai. 2015-04-01 10:57:07 -07:00
Akira Ajisaka
b85bbca745 HDFS-6945. BlockManager should remove a block from excessReplicateMap and decrement ExcessBlocks metric when the block is removed. (aajisaka) 
(cherry picked from commit 18a91fe4df0448d9f7de91602646ecf5a51c52e4)
 2015-04-01 09:08:22 +09:00
Akira Ajisaka
7f6811ad73 HDFS-7671. hdfs user guide should point to the common rack awareness doc. Contributed by Kai Sasaki. 
(cherry picked from commit 859cab2f2273f563fd70e3e616758edef91ccf41)
 2015-04-01 00:26:54 +09:00
yliu
e4cd67e9b0 HDFS-7997. The first non-existing xattr should also throw IOException. (zhouyingchao via yliu) 2015-03-31 21:16:13 +08:00
Arpit Agarwal
312e50f9c4 HDFS-7944. Minor cleanup of BlockPoolManager#getAllNamenodeThreads. (Arpit Agarwal) 2015-03-30 21:24:59 -07:00
Harsh J
32766b6563 HDFS-3918. EditLogTailer shouldn't log WARN when other node is in standby mode. Contributed by Todd Lipcon. 
(cherry picked from commit cce66ba3c9ec293e8ba1afd0eb518c7ca0bbc7c9)
 2015-03-31 08:17:22 +05:30
Arpit Agarwal
abf3ad988d HDFS-7645. Rolling upgrade is restoring blocks from trash multiple times (Contributed by Vinayakumar B and Keisuke Ogiwara) 2015-03-30 15:25:26 -07:00
Haohui Mai
dd5b2dac5a HDFS-7748. Separate ECN flags from the Status in the DataTransferPipelineAck. Contributed by Anu Engineer and Haohui Mai. 2015-03-30 11:59:32 -07:00
Colin Patrick Mccabe
02ed22cd2d HDFS-7261. storageMap is accessed without synchronization in DatanodeDescriptor#updateHeartbeatState() (Brahma Reddy Battula via Colin P. McCabe) 
(cherry picked from commit 1feb9569f366a29ecb43592d71ee21023162c18f)
 2015-03-30 10:53:57 -07:00
Akira Ajisaka
d4bb9b2146 HDFS-8002. Website refers to /trash directory. Contributd by Brahma Reddy Battula. 
(cherry picked from commit e7ea2a8e8f0a7b428ef10552885757b99b59e4dc)
 2015-03-31 00:28:27 +09:00
Kihwal Lee
c4cedfc1d6 HDFS-7742. Favoring decommissioning node for replication can cause a block to stay 
underreplicated for long periods. Contributed by Nathan Roberts.
(cherry picked from commit 04ee18ed48ceef34598f954ff40940abc9fde1d2)
 2015-03-30 10:11:25 -05:00
Harsh J
c58357939f HDFS-4396. Add START_MSG/SHUTDOWN_MSG for ZKFC. Contributed by Liang Xie. 
(cherry picked from commit ae3e8c61ff4c926ef3e71c782433ed9764d21478)
 2015-03-30 15:22:57 +05:30
Vinayakumar B
9f49b3e93d HDFS-7890. Improve information on Top users for metrics in RollingWindowsManager and lower log level (Contributed by J.Andreina) 
(cherry picked from commit 1ed9fb76645ecd195afe0067497dca10a3fb997d)
 2015-03-30 10:04:11 +05:30
Akira Ajisaka
7aded71f3a HDFS-6408. Remove redundant definitions in log4j.properties. Contributed by Abhiraj Butala. 
(cherry picked from commit 232eca944a721c62f37e9012546a7fa814da6e01)
 2015-03-30 11:26:11 +09:00
Akira Ajisaka
a79a38d096 HDFS-6263. Remove DRFA.MaxBackupIndex config from log4j.properties. Contributed by Abhiraj Butala. 
(cherry picked from commit 257c77f895e8e4c3d8748909ebbd3ba7e7f880fc)
 2015-03-30 10:52:45 +09:00
Harsh J
b679dc5a8e HDFS-7501. TransactionsSinceLastCheckpoint can be negative on SBNs. Contributed by Gautam Gopalakrishnan. 
(cherry picked from commit 3d9132d434c39e9b6e142e5cf9fd7a8afa4190a6)
 2015-03-29 00:46:22 +05:30
Arpit Agarwal
d0bc2eff1a HDFS-7700. Document quota support for storage types. (Contributed by Xiaoyu Yao) 2015-03-27 19:49:33 -07:00
Arun Suresh
dadda2edca HDFS-8004. Use KeyProviderCryptoExtension#warmUpEncryptedKeys when creating an encryption zone. (awang via asuresh) 
(cherry picked from commit e97f8e44af9dffc42c030278425cffe0c9da723b)
 2015-03-27 19:25:32 -07:00
Kihwal Lee
577ea08865 HDFS-7990. IBR delete ack should not be delayed. Contributed by Daryn Sharp. 
(cherry picked from commit 60882ab26d49f05cbf0686944af6559f86b3417d)
 2015-03-27 09:06:23 -05:00
Colin Patrick Mccabe
2a750c9aa6 HDFS-7410. Support CreateFlags with append() to support hsync() for appending streams (Vinayakumar B via Colin P. McCabe) 
(cherry picked from commit 61df1b27a797efd094328c7d9141b9e157e01bf4)
 2015-03-26 13:28:58 -07:00
Arpit Agarwal
c035106c05 HDFS-7824. GetContentSummary API and its namenode implementation for Storage Type Quota/Usage. (Contributed by Xiaoyu Yao) 2015-03-26 10:47:43 -07:00
Kihwal Lee
071277b9ce HDFS-7963. Fix expected tracing spans in TestTracing. Contributed by Masatake Iwasaki. 2015-03-26 08:42:45 -05:00
Kihwal Lee
be4eabdcd4 HDFS-7928. Scanning blocks from disk during rolling upgrade startup takes a lot of time if disks are busy. Contributed by Rushabh Shah. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/BlockPoolSlice.java
 2015-03-25 14:51:00 -05:00
Jitendra Pandey
456cec127b HDFS-6826. Plugin interface to enable delegation of HDFS authorization assertions. Contributed by Arun Suresh. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
 2015-03-24 16:02:21 -07:00
Haohui Mai
161dae898f HDFS-7713. Implement mkdirs in the HDFS Web UI. Contributed by Ravi Prakash. 2015-03-24 15:49:08 -07:00
Jing Zhao
483f77b75b HDFS-7854. Separate class DataStreamer out of DFSOutputStream. Contributed by Li Bo. 
(cherry picked from commit a16bfff71bd7f00e06e1f59bfe5445a154bb8c66)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
 2015-03-24 11:21:10 -07:00
Brandon Li
260dbe96c3 HDFS-7977. NFS couldn't take percentile intervals. Contributed by Brandon Li 
(cherry picked from commit 570a83ae80faf2076966acf30588733803327844)
 2015-03-24 10:50:50 -07:00
Harsh J
608ad6c2cd HDFS-7875. Improve log message when wrong value configured for dfs.datanode.failed.volumes.tolerated. Contributed by Nijel. 
(cherry picked from commit 2da3d2ed2ff2e9b48dbda7e029aa58261c729d35)
 2015-03-24 23:03:56 +05:30
Brandon Li
95bde88986 HDFS-7976. Update NFS user guide for mount option 'sync' to minimize or avoid reordered writes. Contributed by Brandon Li 
(cherry picked from commit 51f1f4937402c63392854443ccde18c1579e3348)
 2015-03-24 10:29:27 -07:00
Andrew Wang
e39c58fd9d HDFS-7961. Trigger full block report after hot swapping disk. Contributed by Eddy Xu. 
(cherry picked from commit 6413d34986f3399023426c89c9a0d401c9557716)
 2015-03-24 09:07:24 -07:00
Vinayakumar B
c3e2c20c99 HDFS-3325. When configuring 'dfs.namenode.safemode.threshold-pct' to a value greater or equal to 1 there is mismatch in the UI report (Contributed by J.Andreina) 
(cherry picked from commit c6c396fcd69514ba93583268b2633557c3d74a47)
 2015-03-24 12:13:21 +05:30
Plamen Jeliazkov
75d6d9e022 HDFS-7956. Improve logging for DatanodeRegistration. Contributed by Plamen Jeliazkov. 2015-03-23 23:07:01 -07:00
Andrew Wang
2f46ee50bd HDFS-7960. The full block report should prune zombie storages even if they're not empty. Contributed by Colin McCabe and Eddy Xu. 
(cherry picked from commit 50ee8f4e67a66aa77c5359182f61f3e951844db6)
 2015-03-23 22:00:44 -07:00
Tsz-Wo Nicholas Sze
fe693b72de HDFS-7884. Fix NullPointerException in BlockSender when the generation stamp provided by the client is larger than the one stored in the datanode. Contributed by Brahma Reddy Battula 2015-03-24 13:51:31 +09:00