Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,444 Commits 326 Branches 371 Tags
Commit Graph

4331 Commits

Author SHA1 Message Date
Xiao Chen
4e376f162f HDFS-10875. Optimize du -x to cache intermediate result. Contributed by Xiao Chen. 
(cherry picked from commit e52d6e7a46ceef74dd8d8a3d49c49420e3271365)
 2016-09-19 21:45:05 -07:00
Xiao Chen
b6a60d715d HDFS-10489. Deprecate dfs.encryption.key.provider.uri for HDFS encryption zones. Contributed by Xiao Chen. 
(cherry picked from commit ea839bd48e4478fc7b6d0a69e0eaeae2de5e0f0d)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/HdfsConfiguration.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
 2016-09-19 16:38:18 -07:00
Andrew Wang
b03a0be7a3 HDFS-10823. Implement HttpFSFileSystem#listStatusIterator. 
(cherry picked from commit 8a40953058d50d421d62b71067a13b626b3cba1f)
 2016-09-16 15:37:49 -07:00
Akira Ajisaka
214b6d16d5 HDFS-10862. Typos in 4 log messages. Contributed by Mehran Hassani. 
(cherry picked from commit b09a03cd7d26cf96ec26a81ba11f00778241eb3e)
 2016-09-16 16:12:19 +09:00
Arpit Agarwal
853e1e6946 HDFS-10805. Reduce runtime for append test. Contributed by Gergely Novak. 2016-09-14 09:31:24 -07:00
Andrew Wang
abaad976cd HDFS-10837. Standardize serializiation of WebHDFS DirectoryListing. 
(cherry picked from commit db6d243cf89d25fefbffd4c8721e14d9246b5a16)
 2016-09-13 11:02:43 -07:00
Akira Ajisaka
764cf431ca HDFS-10856. Update the comment of BPServiceActor$Scheduler#scheduleNextBlockReport. Contributed by Yiqun Lin. 
(cherry picked from commit f0876b8b60c19aa25e0417ac0f419a3a82bf210b)
 2016-09-13 14:51:04 +09:00
Yongjun Zhang
223ce108a1 HDFS-10657. testAclCLI.xml setfacl test should expect mask r-x. (John Zhuge via Yongjun Zhang) 
(cherry picked from commit d2466ec3e01b5ef2a0bde738232c5ad6d2d956eb)
 2016-09-12 21:46:09 -07:00
Arpit Agarwal
6948691ad1 HDFS-10830. FsDatasetImpl#removeVolumes crashes with IllegalMonitorStateException when vol being removed is in use. (Arpit Agarwal and Manoj Govindassamy) 2016-09-10 18:22:15 -07:00
Arpit Agarwal
012b266e5e HDFS-10742. Measure lock time in FsDatasetImpl. Contributed by Chen Liang. 2016-09-10 18:04:00 -07:00
Arpit Agarwal
ad0ac6cced HDFS-10682. Replace FsDatasetImpl object lock with a separate lock object. (Contributed by Chen Liang) 2016-09-10 17:48:27 -07:00
Xiao Chen
bb6d866207 HDFS-9781. FsDatasetImpl#getBlockReports can occasionally throw NullPointerException. Contributed by Manoj Govindassamy. 2016-09-09 18:25:35 -07:00
Andrew Wang
f448ce2a89 HDFS-10832. Propagate ACL bit and isEncrypted bit in HttpFS FileStatus permissions. 
(cherry picked from commit cba973f036a2c8fcc21a5826b8306247ec154c7b)
 2016-09-09 12:16:08 -07:00
Andrew Wang
72ea641468 HDFS-10831. Add log when URLConnectionFactory.openConnection failed. Contributed by yunjiong zhao. 
(cherry picked from commit b07c266dca7f303c793b432a0738d593728cf2b3)
 2016-09-08 18:30:26 -07:00
Akira Ajisaka
c0756dfef6 HDFS-10847. Complete the document for FileDistribution processor in OfflineImageViewer. Contributed by Yiqun Lin. 
(cherry picked from commit 20a20c2f6e1b3b8aa6a58a824ad3aadc349dc761)
 2016-09-09 01:35:36 +09:00
Akira Ajisaka
fe6dd66174 HDFS-10778. Add -format option to make the output of FileDistribution processor human-readable in OfflineImageViewer. 
(cherry picked from commit 63f594892ecd4687e37a99790288e36eb278849f)
 2016-09-08 15:18:25 +09:00
Arpit Agarwal
f9415b7a7e HDFS-9038. DFS reserved space is erroneously counted towards non-DFS used. (Brahma Reddy Battula) 2016-09-06 13:44:22 -07:00
Xiao Chen
3025992bd3 HDFS-10841. Remove duplicate or unused variable in appendFile(). Contributed by Kihwal Lee. 
(cherry picked from commit f6c0b7543f612de756ff0c03e9a2c6e33b496a36)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirAppendOp.java
 2016-09-06 11:52:30 -07:00
Andrew Wang
40eab77562 HDFS-10833. Fix JSON errors in WebHDFS.md examples. 
(cherry picked from commit cbd909ce2a5ac1da258f756fa1f93e84dd20b926)
 2016-09-02 14:50:41 -07:00
Lei Xu
d6bfb2176c HDFS-10822. Log DataNodes in the write pipeline. John Zhuge via Lei Xu 
(cherry picked from commit 5a8c5064d1a1d596b1f5c385299a86ec6ab9ad1e)
 2016-09-02 11:02:38 -07:00
Allen Wittenauer
b1466b8124 HDFS-9877. HDFS Namenode UI: Fix browsing directories that need to be encoded (Ravi Prakash via aw) 
(cherry picked from commit 15f018434c5b715729488fd0b03a11f1bc943470)
 2016-09-01 14:24:13 -07:00
Allen Wittenauer
b9eedc24a6 HDFS-7987. Allow files / directories to be moved (Ravi Prakash via aw) 
(cherry picked from commit d44f4745b4a186dd06dd6837a85ad90a237d7d97)
 2016-09-01 14:23:42 -07:00
Allen Wittenauer
743c0ebd67 HDFS-7767. Use the noredirect flag in WebHDFS to allow web browsers to upload files via the NN UI (Ravi Prakash via aw) 
(cherry picked from commit 99a771cd7a3f792a76ac89c406b82a983c059d28)
 2016-09-01 14:22:23 -07:00
Zhe Zhang
776eccd873 Addendum fix for HDFS-10817 to fix failure of the added testFSReadLockLongHoldingReport in branch-2. 
(cherry picked from commit 6c600360ca469d5fe0f017d681585db06c80c9cc)
 2016-08-31 23:44:54 -07:00
Zhe Zhang
f87e2f5d05 HDFS-10817. Add Logging for Long-held NN Read Locks. Contributed by Erik Krogen. 
(cherry picked from commit 6f4b0d33ca339e3724623a1d23c101f8cfd3cdd5)
 2016-08-31 15:54:03 -07:00
Andrew Wang
72d90cbe32 HDFS-10784. Implement WebHdfsFileSystem#listStatusIterator. 
(cherry picked from commit 85bab5fb572194fda38854f1f21c670925058009)
 2016-08-31 14:33:14 -07:00
Allen Wittenauer
72334a4be3 HDFS-7766. Add a flag to WebHDFS op=CREATE to not respond with a 307 redirect (Ravi Prakash via aw) 
(cherry picked from commit 4b0f55b6ea1665e2118fd573f72a6fcd1fce20d6)
 2016-08-31 14:33:10 -07:00
Kihwal Lee
34042ccbb9 HDFS-10729. Improve log message for edit loading failures caused by FS limit checks. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 01721dd88ee532d20eda841254437da4dfd69db5)
 2016-08-31 14:09:23 -05:00
Ming Ma
56c9a96a76 HDFS-9392. Admins support for maintenance state. Contributed by Ming Ma. 
(cherry picked from commit 9dcbdbdb5a34d85910707f81ebc1bb1f81c99978)
 2016-08-30 14:09:18 -07:00
Wei-Chiu Chuang
abbd95b790 HDFS-10760. DataXceiver#run() should not log InvalidToken exception as an error. Contributed by Pan Yuxuan. 
(cherry picked from commit c4ee6915a14e00342755d7cdcbf2d61518f306aa)
 2016-08-30 10:44:27 -07:00
Zhe Zhang
320be1b3c6 HDFS-10814. Add assertion for getNumEncryptionZones when no EZ is created. Contributed by Vinitha Reddy Gankidi. 
(cherry picked from commit 4bd45f54eedd449a98a90540698c6ceb47454fec)
 2016-08-29 23:42:46 -07:00
Xiao Chen
72863f0049 HDFS-4210. Throw helpful exception when DNS entry for JournalNode cannot be resolved. Contributed by Charles Lamb and John Zhuge. 
(cherry picked from commit 046f6bf0f944311fc70de3a96fc6ab8211cbbbd8)
 2016-08-29 17:48:49 -07:00
Yongjun Zhang
4bed2a4b9c HDFS-10625. VolumeScanner to report why a block is found bad. Contributed by Rushabh S Shah and Yiqun Lin. 
(cherry picked from commit 5d1609ddf275e4907bd224bf618e2aad4b262888)
 2016-08-29 14:08:36 -07:00
Mingliang Liu
875c14004d HDFS-10807. Doc about upgrading to a version of HDFS with snapshots may be confusing. (Contributed by Mingliang Liu) 
(cherry picked from commit 6742fb6e68d349055f985eb640d845e689d75384)
 2016-08-29 10:19:33 -07:00
Yongjun Zhang
e813a3ea4a HDFS-10652. Add a unit test for HDFS-4660. Contributed by Vinayakumar B., Wei-Chiu Chuang, Yongjun Zhang. 
(cherry picked from commit c25817159af17753b398956cfe6ff14984801b01)
 2016-08-27 22:59:19 -07:00
Andrew Wang
1c47a356d8 HDFS-10793. Fix HdfsAuditLogger binary incompatibility introduced by HDFS-9184. Contributed by Manoj Govindassamy. 
(cherry picked from commit a445b82baaa58dcaaa3831e724a9915e9dde57c5)
 2016-08-26 15:40:05 -07:00
Zhe Zhang
f3a4cbcc60 HDFS-10798. Make the threshold of reporting FSNamesystem lock contention configurable. Contributed by Erik Krogen. 
(cherry picked from commit 407b519fb14f79f19ebc4fbdf08204336a7acf77)
 2016-08-26 14:20:48 -07:00
Kihwal Lee
442dfd1bbe HDFS-10768. Optimize mkdir ops. Contributed by Daryn Sharp. 
(cherry picked from commit 8b7adf4ddf420a93c586c4b2eac27dd0f649682e)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirWriteFileOp.java
 2016-08-26 15:49:56 -05:00
Kihwal Lee
2a6497cc7c HDFS-8915. TestFSNamesystem.testFSLockGetWaiterCount fails intermittently. Contributed by Masatake Iwasaki. 
(cherry picked from commit 13fb1b50e608558b2970184908ee5b9fcd7eb7b6)
 2016-08-26 10:00:36 -05:00
Xiaoyu Yao
fc67501393 HDFS-10748. TestFileTruncate#testTruncateWithDataNodesRestart runs sometimes timeout. Contributed by Yiqun Lin. 
(cherry picked from commit 4da5000dd33cf013e7212848ed2c44f1e60e860e)
 2016-08-25 09:22:02 -07:00
Kihwal Lee
ba3257baf5 HDFS-10772. Reduce byte/string conversions for get listing. Contributed by Daryn Sharp. 
(cherry picked from commit a1f3293762dddb0ca953d1145f5b53d9086b25b8)
 2016-08-24 15:25:33 -05:00
Wei-Chiu Chuang
d19e29bd68 HDFS-8986. Add option to -du to calculate directory space usage excluding snapshots. Contributed by Xiao Chen. 2016-08-24 08:49:54 -07:00
Daryn Sharp
a30f6a68fa HDFS-10762. Pass IIP for file status related methods 
(cherry picked from commit ec252ce0fc0998ce13f31af3440c08a236328e5a)
 2016-08-24 08:51:05 -05:00
Zhe Zhang
dfcbc12026 HADOOP-12765. HttpServer2 should switch to using the non-blocking SslSelectChannelConnector to prevent performance degradation when handling SSL connections. Contributed by Min Shen. Branch-2 patch contributed by Wei-Chiu Chuang. 2016-08-23 14:46:08 -07:00
Jason Lowe
8cd3996246 HDFS-9745. TestSecureNNWithQJM#testSecureMode sometimes fails with timeouts. Contributed by Xiao Chen 
(cherry picked from commit 126d165efd80e266a8309241f3cf059e358f5019)
 2016-08-23 14:48:35 +00:00
Akira Ajisaka
94a844b4f6 HDFS-10783. The option '-maxSize' and '-step' fail in OfflineImageViewer. Contributed by Yiqun Lin. 
(cherry picked from commit e90f3359de299ef5e3a54ca71070e3dfe1dbb98c)
 2016-08-23 19:58:13 +09:00
Eric Yang
9faacbec2c HDFS-8312. Added permission check for moving file to Trash. (Weiwei Yang via Eric Yang) 2016-08-22 18:32:06 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
8a6a7c82f2 HDFS-10692. Update JDiff report's base version for HDFS from 2.6.0 to 2.7.2. Contributed by Wangda Tan. 
(cherry picked from commit dc7a1c54f4447fb01980ae61d5d67e90c4f52f00)
 2016-08-22 11:36:38 -07:00
Akira Ajisaka
62c7d09efa HDFS-10645. Make block report size as a metric and add this metric to datanode web ui. Contributed by Yuanbo Liu. 2016-08-22 14:22:37 +09:00
Jing Zhao
fff0418458 HDFS-10764. Fix INodeFile#getBlocks to not return null. Contributed by Arpit Agarwal. 
(cherry picked from commit 0faee62a0c8c1b8fd83227babfd00fbc2b26bddf)
 2016-08-19 22:23:55 -07:00