Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,757 Commits 326 Branches 371 Tags
Commit Graph

4168 Commits

Author SHA1 Message Date
Kihwal Lee
d03240e0bf HDFS-10956. Remove rename/delete performance penalty when not using snapshots. Contributed by Daryn Sharp. 
(cherry picked from commit 44f48ee96ee6b2a3909911c37bfddb0c963d5ffc)
 2016-10-04 15:29:39 -05:00
Kihwal Lee
33cafc9fcd HDFS-10878. TestDFSClientRetries#testIdempotentAllocateBlockAndClose throws ConcurrentModificationException. Contributed by Rushabh S Shah. 
(cherry picked from commit 382307cbdd94107350fe6fad1acf87d63c9be9d6)
 2016-10-04 11:13:39 -05:00
Brahma Reddy Battula
253e2442e4 HDFS-10947. Correct the API name for truncate in webhdfs document. Contributed by Surendra Singh Lilhore 
(cherry picked from commit ef7f06f7d1561db13bd3b07a5f62815ec29e1cdf)
 2016-10-04 18:53:12 +05:30
Brahma Reddy Battula
1ae35b6fff HDFS-10944. Correct the javadoc of dfsadmin#disallowSnapshot. Contributed by Jagadesh Kiran N 
(cherry picked from commit 5ea0210c7fea1324973e7f69256bc53f31ee4cf7)
 2016-10-04 18:39:31 +05:30
Mingliang Liu
438402a710 HDFS-10934. TestDFSShell#testStat fails intermittently. Contributed by Eric Badger 
(cherry picked from commit f61e3d13e9610cbd09886359553f27d6480f6735)
 2016-10-03 22:44:22 -04:00
Mingliang Liu
11ea5a59c2 HDFS-10810. setreplication removing block from underconstrcution temporarily. Contributed by Brahma Reddy Battula 
(cherry picked from commit d963ecb9185b8fcfba86bec8420a8daf7140a071)
 2016-10-03 22:04:49 -04:00
Xiao Chen
bf043e63f9 HDFS-10918. Add a tool to get FileEncryptionInfo from CLI. Contributed by Xiao Chen. 
(cherry picked from commit 853d65a157362661ccab10379c2d82e780382f83)
 2016-10-03 16:02:11 -07:00
Xiaoyu Yao
b0b29de8ab HDFS-10690. Optimize insertion/removal of replica in ShortCircuitCache. Contributed by Fenghua Hu. 
(cherry picked from commit 607705c488fa5263d851cee578a2d319e6e52ecd)
 2016-10-03 11:05:58 -07:00
Kihwal Lee
52c534cd0f HDFS-10940. Reduce performance penalty of block caching when not used. Contributed by Daryn Sharp. 
(cherry picked from commit 03b797a6ac2b1f99c6baebf4729bae22aa267692)
 2016-10-03 11:42:06 -05:00
Kihwal Lee
84b20d1c18 HDFS-10945. Fix the Findbugwaring FSNamesystem#renameTo() in branch-2. Contributed by Brahma Reddy Battula. 2016-10-03 11:09:40 -05:00
Kihwal Lee
3a89a8814f HDFS-10619. Cache path in InodesInPath. Contributed by Daryn Sharp. 
(cherry picked from commit 90020624b05230ad4a7fbd666d0177ecb107a4d6)
 2016-10-03 09:26:41 -05:00
Arpit Agarwal
c31d23db35 Revert "HDFS-10923. Make InstrumentedLock require ReentrantLock." 
This reverts commit 9d4e0168d5f0e9211049252981f0f8e705e98fc7.
 2016-09-30 23:12:16 -07:00
Arpit Agarwal
9d4e0168d5 HDFS-10923. Make InstrumentedLock require ReentrantLock. 2016-09-30 23:04:30 -07:00
Kihwal Lee
bfc7f0630b HDFS-10851. FSDirStatAndListingOp: stop passing path as string. Contributed by Daryn Sharp. 
(cherry picked from commit 2551ff80b7eb3e13194ec47db64cf9ee940b9ab0)
 2016-09-30 16:02:26 -05:00
Zhe Zhang
345debf22e HDFS-10896. Move lock logging logic from FSNamesystem into FSNamesystemLock. Contributed by Erik Krogen. 
(cherry-picked from commit 434c5ea75dc3d87513e49290ac9999148ff5163c)
 2016-09-30 13:43:49 -07:00
Kihwal Lee
f5b193b482 HDFS-10850. getEZForPath should NOT throw FNF. Contributed by Andrew Wang. 
(cherry picked from commit 0670149c88852cd7c4d6774bff06c7c588558739)
 2016-09-30 09:16:13 -05:00
Mingliang Liu
7467628d43 HDFS-10892. Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'. Contributed by Mingliang Liu 
(cherry picked from commit 84c626407925e03ee2ef11faba9324d5c55b8e93)
 2016-09-28 16:09:42 -07:00
Kihwal Lee
5294eec729 HDFS-10779. Rename does not need to re-solve destination. Contributed by Daryn Sharp. 
(cherry picked from commit 5f34402adae191232fe78e62990396ca07f314bb)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
 2016-09-28 18:03:58 -05:00
Andrew Wang
c4f81f199f HDFS-10914. Move remnants of oah.hdfs.client to hadoop-hdfs-client. 
(cherry picked from commit 92e5e9159850c01635091ea6ded0d8ee76691a9a)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/HdfsAdmin.java

(cherry picked from commit 9ddaa29301ea8ae2e3cb0f1e031343d5b91f92ee)
 2016-09-28 16:03:02 -07:00
Arpit Agarwal
76f92277c3 HDFS-10824. MiniDFSCluster#storageCapacities has no effects on real capacity. Contributed by Xiaobing Zhou. 2016-09-28 15:56:39 -07:00
Brahma Reddy Battula
5f754e8638 HDFS-9444. Add utility to find set of available ephemeral ports to ServerSocketUtil. Contributed by Masatake Iwasaki 
(cherry picked from commit e9a34ae29c7390f3ffcbeee02dc5faa26fca482a)
 2016-09-28 11:16:40 +05:30
Mingliang Liu
6da9a3c603 HDFS-10915. Fix time measurement bug in TestDatanodeRestart. Contributed by Xiaobing Zhou 
(cherry picked from commit 6437ba18c5c26bc271a63aff5ea03756f43dd9a3)
 2016-09-27 20:47:04 -07:00
Wangda Tan
69f91d8c48 HADOOP-13544. JDiff reports unncessarily show unannotated APIs and cause confusion while our javadocs only show annotated and public APIs. (vinodkv via wangda) 
(cherry picked from commit 875062b5bc789158290bf93dadc71b5328ca4fee)
(cherry picked from commit 269401dc83699e4de7b99e794eeb7646a7f3fc93)
 2016-09-27 11:46:49 -07:00
Arpit Agarwal
3f5482bb69 HDFS-10828. Fix usage of FsDatasetImpl object lock in ReplicaMap. (Arpit Agarwal) 2016-09-27 10:02:27 -07:00
Masatake Iwasaki
756dbc505e HDFS-10426. TestPendingInvalidateBlock failed in trunk. Contributed by Yiqun Lin. 
(cherry picked from commit f55eb981dd408ca61d49488f83c99c1fc1f83134)
 2016-09-28 01:47:31 +09:00
Arpit Agarwal
a349c54603 HDFS-10713. Throttle FsNameSystem lock warnings. Contributed by Hanisha Koneru. 2016-09-27 09:05:53 -07:00
Brahma Reddy Battula
5bc297c58f HDFS-10889. Remove outdated Fault Injection Framework documentaion. Contributed by Brahma Reddy Battula 
(cherry picked from commit df1d0f5d56f375e82b0219a19675a7b6052aaf30)
 2016-09-27 19:35:57 +05:30
Wei-Chiu Chuang
f0f3b6a66a HDFS-10609. Uncaught InvalidEncryptionKeyException during pipeline recovery may abort downstream applications. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 3ae652f82110a52bf239f3c1849b48981558eb19)
(cherry picked from commit bde787db23dd38388dac045d421394006ba63bed)
 2016-09-26 14:46:02 -07:00
Brahma Reddy Battula
17b66c894b HDFS-10869. Remove the unused method InodeId#checkId(). Contributed by Jagadesh Kiran N 
(cherry picked from commit 3e37e243ee041e843e060b17c622ab50c8f9ff11)
 2016-09-25 11:07:21 +05:30
Wei-Chiu Chuang
367d5b2a7e HDFS-10876. Dispatcher#dispatch should log IOException stacktrace. Contributed by Manoj Govindassamy. 
(cherry picked from commit 74b3dd514c86b46197e2e19d9824a423715cab30)
(cherry picked from commit 6c9845572893c406cbbe2e4ea3867585e3a7780f)
 2016-09-23 13:28:59 -07:00
Konstantin V Shvachko
20540df80d HDFS-10843. Update space quota when a UC block is completed rather than committed. Contributed by Erik Krogen. 
(cherry picked from commit a5bb88c8e0fd4bd19b6d377fecbe1d2d441514f6)
 2016-09-23 10:49:26 -07:00
Brahma Reddy Battula
aa967869a5 HDFS-10886. Replace fs.default.name with fs.defaultFS in viewfs document. 
(cherry picked from commit bbdf350ff9fb624fe736c1eb9271c5dcb4e14b06)
 2016-09-23 19:35:34 +05:30
Masatake Iwasaki
1524e63206 HDFS-9333. Some tests using MiniDFSCluster errored complaining port in use. (iwasakims) 
(cherry picked from commit 964e546ab1dba5f5d53b209ec6c9a70a85654765)

 Conflicts:
  hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockTokenWithDFSStriped.java

(cherry picked from commit 1b91ebb7145860ecda394d764da5fc7063406fee)
 2016-09-21 10:52:00 +09:00
Xiao Chen
11ed4f5d40 HDFS-10879. TestEncryptionZonesWithKMS#testReadWrite fails intermittently. Contributed by Xiao Chen. 
(cherry picked from commit d31bef575478fe78c0600fb602e117e4b81c9887)
 2016-09-20 16:57:44 -07:00
Xiao Chen
e7daeea711 HDFS-10875. Optimize du -x to cache intermediate result. Contributed by Xiao Chen. 
(cherry picked from commit e52d6e7a46ceef74dd8d8a3d49c49420e3271365)
 2016-09-19 21:45:09 -07:00
Xiao Chen
900dc70a01 HDFS-10489. Deprecate dfs.encryption.key.provider.uri for HDFS encryption zones. Contributed by Xiao Chen. 
(cherry picked from commit ea839bd48e4478fc7b6d0a69e0eaeae2de5e0f0d)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/HdfsConfiguration.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml

(cherry picked from commit 20585ade1d9d7fbd8b9df8b1e3db5a74fd4441a6)
 2016-09-19 16:38:00 -07:00
Andrew Wang
8c24388af0 HDFS-10832. Propagate ACL bit and isEncrypted bit in HttpFS FileStatus permissions. 
(cherry picked from commit cba973f036a2c8fcc21a5826b8306247ec154c7b)
(cherry picked from commit f448ce2a89aff506c7aa99d18c47a6db81411a04)
 2016-09-19 16:38:00 -07:00
Akira Ajisaka
ab57c869e2 HDFS-10862. Typos in 4 log messages. Contributed by Mehran Hassani. 
(cherry picked from commit b09a03cd7d26cf96ec26a81ba11f00778241eb3e)
(cherry picked from commit 214b6d16d5822d2be861ae8ee5f2782ddeeef49b)
 2016-09-16 16:13:00 +09:00
Arpit Agarwal
bdc71bd608 HDFS-10805. Reduce runtime for append test. Contributed by Gergely Novak. 2016-09-14 09:31:35 -07:00
Akira Ajisaka
c06cf0f89c HDFS-10856. Update the comment of BPServiceActor$Scheduler#scheduleNextBlockReport. Contributed by Yiqun Lin. 
(cherry picked from commit f0876b8b60c19aa25e0417ac0f419a3a82bf210b)
(cherry picked from commit 764cf431ca1fdcba92378dc35d4565e079a63911)
 2016-09-13 14:51:29 +09:00
Yongjun Zhang
b96e2cbd20 HDFS-10657. testAclCLI.xml setfacl test should expect mask r-x. (John Zhuge via Yongjun Zhang) 
(cherry picked from commit d2466ec3e01b5ef2a0bde738232c5ad6d2d956eb)
 2016-09-12 22:13:17 -07:00
Xiao Chen
76a18915bf HDFS-9781. FsDatasetImpl#getBlockReports can occasionally throw NullPointerException. Contributed by Manoj Govindassamy. 
(cherry picked from commit bb6d8662070984c7a3be6d1b8f3e1cfdc787b60f)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
 2016-09-10 23:14:22 -07:00
Arpit Agarwal
ed35f99045 HDFS-10830. FsDatasetImpl#removeVolumes crashes with IllegalMonitorStateException when vol being removed is in use. (Arpit Agarwal and Manoj Govindassamy) 2016-09-10 18:34:28 -07:00
Arpit Agarwal
04f620c4d0 HDFS-10742. Measure lock time in FsDatasetImpl. Contributed by Chen Liang. 2016-09-10 18:01:37 -07:00
Andrew Wang
912631a226 HDFS-10831. Add log when URLConnectionFactory.openConnection failed. Contributed by yunjiong zhao. 
(cherry picked from commit b07c266dca7f303c793b432a0738d593728cf2b3)
(cherry picked from commit 72ea64146810bb5010280145ee7cad9fb9375aad)
 2016-09-08 18:30:36 -07:00
Arpit Agarwal
162ee0f0a4 HDFS-9038. DFS reserved space is erroneously counted towards non-DFS used. (Brahma Reddy Battula) 2016-09-06 13:52:05 -07:00
Xiao Chen
e84b5c5e3e HDFS-10841. Remove duplicate or unused variable in appendFile(). Contributed by Kihwal Lee. 
(cherry picked from commit f6c0b7543f612de756ff0c03e9a2c6e33b496a36)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirAppendOp.java

(cherry picked from commit 3025992bd358518349c1f80d03adecabe0ac386e)
 2016-09-06 11:54:02 -07:00
Andrew Wang
9b7e079d4b HDFS-10833. Fix JSON errors in WebHDFS.md examples. 
(cherry picked from commit cbd909ce2a5ac1da258f756fa1f93e84dd20b926)
(cherry picked from commit 40eab775629f5968348983ac5bc1d2cf5b9b228b)
 2016-09-02 14:51:12 -07:00
Kihwal Lee
16c166249c HDFS-10772. Reduce byte/string conversions for get listing. Contributed by Daryn Sharp. 
(cherry picked from commit a1f3293762dddb0ca953d1145f5b53d9086b25b8)
 2016-09-02 12:32:34 -05:00
Kihwal Lee
1c946a2665 HDFS-10768. Optimize mkdir ops. Contributed by Daryn Sharp. 
(cherry picked from commit 442dfd1bbe0d6c6fe836bd0e0b0151b52faf8582)
 2016-09-02 12:28:11 -05:00