25c84d279b
HDFS-11412. Maintenance minimum replication config value allowable range should be [0, DefaultReplication]. (Manoj Govindassamy via mingma)
2017-03-01 20:23:52 -08:00
555d0c3995
HDFS-11478. Update EC commands in HDFSCommands.md. Contributed by Yiqun Lin.
2017-03-02 12:10:24 +08:00
82ef9accaf
HDFS-11428. Change setErasureCodingPolicy to take a required string EC policy name. Contributed by Andrew Wang.
2017-03-01 16:06:02 +05:30
dcd03df9f9
HDFS-11432. Federation : Support fully qualified path for Quota/Snapshot/cacheadmin/cryptoadmin commands. Contributed by Brahma Reddy Battula.
2017-03-01 10:45:56 +05:30
d269b488a7
HDFS-11466. Change dfs.namenode.write-lock-reporting-threshold-ms default from 1000ms to 5000ms. Contributed by Andrew Wang.
2017-02-28 09:00:49 -08:00
258342e76c
HADOOP-14118. move jets3t into a dependency on hadoop-aws JAR.
2017-02-28 13:47:44 +09:00
55c07bbed2
HDFS-11382. Persist Erasure Coding Policy ID in a new optional field in INodeFile in FSImage. Contributed by Manoj Govindassamy.
2017-02-27 17:07:45 -08:00
05391c1845
HDFS-10506. OIV's ReverseXML processor cannot reconstruct some snapshot details. Contributed by Akira Ajisaka.
2017-02-25 14:38:50 -08:00
120bef7de8
HDFS-11462. Fix occasional BindException in TestNameNodeMetricsLogger.
2017-02-24 19:53:39 -08:00
4a58870a04
HDFS-11433. Document missing usages of OfflineEditsViewer processors. Contributed by Yiqun Lin.
2017-02-25 10:00:05 +08:00
e24ed47d9a
HDFS-11421. Make WebHDFS' ACLs RegEx configurable. Contributed by Harsh J.
2017-02-24 16:49:46 -08:00
d2b3ba9b8f
HDFS-11295. Check storage remaining instead of node remaining in BlockPlacementPolicyDefault.chooseReplicaToDelete(). Contributed by Marton Elek.
2017-02-24 15:44:11 -08:00
c1a52b04d0
HDFS-11427. Rename rs-default to rs.
2017-02-24 10:58:45 -08:00
132f758e3d
HDFS-11426. Refactor EC CLI to be similar to storage policies CLI.
2017-02-23 16:00:11 -08:00
13d4bcfe35
HDFS-4025. QJM: Sychronize past log segments to JNs that missed them. Contributed by Hanisha Koneru.
2017-02-22 16:33:38 -08:00
d150f061f4
HDFS-11438. Fix typo in error message of StoragePolicyAdmin tool. Contributed by Alison Yu.
2017-02-22 15:16:09 -08:00
cfcd527323
HDFS-11411. Avoid OutOfMemoryError in TestMaintenanceState test runs. (Manoj Govindassamy via mingma)
2017-02-22 09:41:07 -08:00
003ae00693
HDFS-11430. Separate class InnerNode from class NetworkTopology and make it extendable. Contributed by Tsz Wo Nicholas Sze
2017-02-21 15:32:46 -08:00
8ef7ebbc71
HDFS-11406. Remove unused getStartInstance and getFinalizeInstance in FSEditLogOp. Contributed by Alison Yu.
2017-02-21 14:54:20 -08:00
fc9ad3ce3a
HDFS-11405. Rename "erasurecode" CLI subcommand to "ec". Contributed by Manoj Govindassamy.
2017-02-21 13:55:27 -08:00
a07ddef101
HDFS-11404. Increase timeout on TestShortCircuitLocalRead.testDeprecatedGetBlockLocalPathInfoRpc. Contributed by Eric Badger
2017-02-21 12:04:25 -06:00
4804050630
HDFS-11177. 'storagepolicies -getStoragePolicy' command should accept URI based path. (Contributed by Surendra Singh Lilhore)
2017-02-21 18:13:19 +05:30
02c549484a
HDFS-11410. Use the cached instance when edit logging SetAclOp, SetXAttrOp and RemoveXAttrOp.
2017-02-16 18:07:55 -08:00
5690b51ef7
HDFS-11100. Recursively deleting file protected by sticky bit should fail. Contributed by John Zhuge.
2017-02-16 05:39:37 -08:00
a136936d01
HDFS-11265. Extend visualization for Maintenance Mode under Datanode tab in the NameNode UI. (Marton Elek via mingma)
2017-02-15 20:24:07 -08:00
0741dd3b9a
HDFS-11375. Display the volume storage type in datanode UI. Contributed by Surendra Singh Lilhore
2017-02-15 11:37:26 -08:00
627da6f717
HDFS-8498. Blocks can be committed with wrong size. Contributed by Jing Zhao.
2017-02-15 10:44:37 -08:00
859bd159ae
HDFS-11333. Print a user friendly error message when plugins are not found. Contributed by Wei-Chiu Chuang.
2017-02-15 02:50:35 -08:00
8acb376c9c
HDFS-11238. Fix checkstyle warnings in NameNode#createNameNode. Contributed by Ethan Li.
2017-02-15 16:53:50 +09:00
8e53f2b9b0
HDFS-11391. Numeric usernames do no work with WebHDFS FS write access. (Pierre Villard via Yongjun Zhang)
2017-02-14 13:40:53 -08:00
aaf2713235
HDFS-11409. DatanodeInfo getNetworkLocation and setNetworkLocation shoud use volatile instead of synchronized. Contributed by Chen Liang.
2017-02-14 12:52:34 -08:00
0cf5993712
HDFS-11084. Add a regression test for sticky bit support of OIV ReverseXML processor. Contributed by Wei-Chiu Chuang.
2017-02-14 09:11:55 -08:00
4164a2032a
HDFS-11408. The config name of balance bandwidth is out of date. Contributed by Yiqun Lin.
2017-02-14 18:57:20 +08:00
719df99c05
HDFS-11407. Document the missing usages of OfflineImageViewer processors. Contributed by Yiqun Lin.
2017-02-14 18:48:08 +08:00
4ed33e9ca3
HDFS-11026. Convert BlockTokenIdentifier to use Protobuf. Contributed by Ewan Higgs.
2017-02-13 11:29:18 -08:00
07a5184f74
HDFS-11379. DFSInputStream may infinite loop requesting block locations. Contributed by Daryn Sharp.
2017-02-10 12:27:08 -06:00
2007e0cf2a
HDFS-11210. Enhance key rolling to guarantee new KeyVersion is returned from generateEncryptedKeys after a key is rolled.
2017-02-07 20:36:11 -08:00
69b23632c4
HDFS-10860. Switch HttpFS from Tomcat to Jetty. Contributed by John Zhuge.
2017-02-07 09:36:55 -08:00
663e683adf
HDFS-10219. Change the default value for dfs.namenode.reconstruction.pending.timeout-sec from -1 to 300. Contributed by Yiqun Lin.
2017-02-06 19:19:30 +08:00
9cbbd1eae8
HDFS-11377. Balancer hung due to no available mover threads. Contributed by yunjiong zhao.
2017-02-06 13:15:16 +08:00
e0235842a7
HDFS-11387. Socket reuse address option is not honored in PrivilegedNfsGatewayStarter. Contributed by Mukul Kumar Singh.
2017-02-03 12:28:05 -08:00
0914fcca31
HDFS-11363. Need more diagnosis info when seeing Slow waitForAckedSeqno.
2017-02-02 10:08:27 -08:00
3433f572fa
HDFS-11353. Improve the unit tests relevant to DataNode volume failure testing. Contributed by Yiqun Lin.
2017-02-02 19:38:17 +08:00
6aa09dc28a
HDFS-11112. Journal Nodes should refuse to format non-empty directories. Contributed by Yiqun Lin.
2017-02-01 16:51:58 -08:00
b6f290d5b6
HDFS-11370. Optimize NamenodeFsck#getReplicaInfo. Contributed Takanobu Asanuma.
2017-02-01 11:21:35 -08:00
59c5f18784
HADOOP-13895. Make FileStatus Serializable
2017-02-01 10:19:36 -08:00
bec9b7aa1d
HDFS-11335. Remove HdfsClientConfigKeys.DFS_CLIENT_SLOW_IO_WARNING_THRESHOLD_KEY usage from DNConf. (Manoj Govindassamy via lei)
2017-02-01 14:42:51 +08:00
312b36d113
HDFS-11378. Verify multiple DataNodes can be decommissioned/maintenance at the same time. (Manoj Govindassamy via mingma)
2017-01-27 16:16:42 -08:00
7bc333ad41
HDFS-11374. Skip FSync in CreateEditsLog to speed up edit log generation. Contributed by Hanisha Koneru.
2017-01-26 16:21:25 -08:00
55c9f6d747
HDFS-9884. Use doxia macro to generate in-page TOC of HDFS site documentation. (iwasakims)
2017-01-27 08:06:39 +09:00
88da9f6b67
Revert "HDFS-8377. Support HTTP/2 in datanode. Contributed by Duo Zhang."
...
This reverts commit ada233b7cd
2017-01-26 13:42:50 -08:00
44606aa850
HDFS-11364. Add a test to verify Audit log entries for setfacl/getfacl commands over FS shell. Contributed by Manoj Govindassamy.
2017-01-26 10:48:26 -08:00
7c1cc30b3c
HDFS-11369. Change exception message in StorageLocationChecker.
2017-01-26 09:12:38 -08:00
ff02bdfe65
Add CHANGES, RELEASENOTES, and jdiff for 3.0.0-alpha2 release.
2017-01-25 15:39:40 -08:00
b782bf2156
HDFS-11124. Report blockIds of internal blocks for EC files in Fsck. Contributed by Takanobu Asanuma.
2017-01-25 11:16:17 -08:00
18e1d68209
HDFS-10534. NameNode WebUI should display DataNode usage histogram. Contributed by Kai Sasaki.
2017-01-25 09:58:39 -08:00
5a5652098c
HDFS-11365. Log portnumber in PrivilegedNfsGatewayStarter. Contributed by Mukul Kumar Singh.
2017-01-24 21:58:02 -08:00
b57368b6f8
HDFS-11194. Maintain aggregated peer performance metrics on NameNode.
2017-01-24 16:58:20 -08:00
355b907c09
HDFS-11121. Add assertions to BlockInfo#addStorage to protect from breaking reportedBlock-blockGroup mapping. Contributed by Takanobu Asanuma.
2017-01-21 05:56:22 +08:00
f3fb94be05
HDFS-11296. Maintenance state expiry should be an epoch time and not jvm monotonic. (Manoj Govindassamy via mingma)
2017-01-19 22:31:15 -08:00
5d8b80ea9b
Preparing for 3.0.0-alpha3 development
2017-01-19 15:50:07 -08:00
78ae2aed8f
HDFS-11259. Update fsck to display maintenance state info. (Manoj Govindassamy via lei)
2017-01-19 16:28:31 +08:00
a8f1c7f542
HDFS-110268. Correctly reconstruct erasure coding file from FSImage. Contributed by SammiChen.
2017-01-19 00:24:11 -08:00
63320d1daa
HDFS-11287. Storage class member storageDirs should be private to avoid unprotected access by derived classes. (Manoj Govindassamy via lei)
2017-01-19 15:38:20 +08:00
724bb86351
HDFS-11316. TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailure fails in trunk. Contributed by Yiqun Lin.
2017-01-19 12:05:13 +08:00
0eb4b513b7
HADOOP-13673. Update scripts to be smarter when running with privilege
...
Signed-off-by: Andrew Wang <wang@apache.org>
Signed-off-by: Ravi Prakash <raviprak@apache.org>
2017-01-18 14:39:05 -08:00
a2a5d7b5bc
HDFS-10759. Change fsimage bool isStriped from boolean to an enum. Contributed by Ewan Higgs.
2017-01-18 13:31:05 -08:00
8a0fa0f7e8
HDFS-10733. NameNode terminated after full GC thinking QJM is unresponsive. Contributed by Vinitha Gankidi.
2017-01-18 12:46:32 -08:00
9130af3f74
HADOOP-13955. Replace deprecated HttpServer2 and SSLFactory constants. Contributed by John Zhuge.
2017-01-18 18:11:33 +09:00
b1a9ec856b
HDFS-11290. TestFSNameSystemMBean should wait until JMX cache is cleared. Contributed by Erik Krogen.
2017-01-18 17:54:33 +09:00
b1fce2b8b1
HDFS-11209. SNN can't checkpoint when rolling upgrade is not finalized. Contributed by Xiaoyu Yao.
2017-01-17 14:33:26 -08:00
89bb05d92b
HDFS-11339. Fix import.
2017-01-16 23:05:28 -08:00
e407449ddb
HADOOP-13933. Add haadmin -getAllServiceState option to get the HA state of all the NameNodes/ResourceManagers. Contributed by Surendra Singh Lilhore.
2017-01-17 10:10:23 +09:00
d1d0b3e1fd
HDFS-11307. The rpc to portmap service for NFS has hardcoded timeout. Contributed by Mukul Kumar Singh.
2017-01-16 14:53:53 -08:00
79e939d0b8
HDFS-11339. Support File IO sampling for Datanode IO profiling hooks. Contributed by Hanisha Koneru.
2017-01-16 14:43:29 -08:00
a853b4e1b5
HDFS-11342. Fix FileInputStream leak in loadLastPartialChunkChecksum. Contributed by Chen Liang.
2017-01-16 13:45:22 -08:00
2f8e9b7e4b
Revert "HDFS-11259. Update fsck to display maintenance state info. (Manoj Govindassamy via lei)"
...
This reverts commit c18590fce2
2017-01-16 15:11:53 +08:00
2604e82eba
HDFS-11344. The default value of the setting dfs.disk.balancer.block.tolerance.percent is different. Contributed by Yiqun Lin.
2017-01-13 22:38:46 -08:00
1cde954a4f
HDFS-11306. Print remaining edit logs from buffer if edit log can't be rolled. Contributed by Wei-Chiu Chuang.
2017-01-13 11:46:30 -08:00
5f336512d0
HADOOP-13961. Fix compilation failure from missing hadoop-kms test jar. Contributed by Sangjin Lee and John Zhuge.
2017-01-11 15:48:50 -08:00
d51f8ba808
HDFS-11312. Fix incompatible tag number change for nonDfsUsed in DatanodeInfoProto. Contributed by Sean Mackrory.
2017-01-11 15:13:27 -08:00
a5ec1e31a3
HDFS-9935. Remove LEASE_{SOFTLIMIT,HARDLIMIT}_PERIOD and unused import from HdfsServerConstants. Contributed Yiqun Lin.
2017-01-11 19:34:01 +05:30
467f5f1735
HDFS-9391. Update webUI/JMX to display maintenance state info. (Manoj Govindassamy via mingma)
2017-01-10 20:12:42 -08:00
e69231658d
HDFS-11072. Add ability to unset and change directory EC policy. Contributed by Sammi Chen.
2017-01-10 11:32:48 -08:00
4046794a53
HDFS-11299. Support multiple Datanode File IO hooks. Contributed by Hanisha Koneru.
2017-01-10 10:43:11 -08:00
c18590fce2
HDFS-11259. Update fsck to display maintenance state info. (Manoj Govindassamy via lei)
2017-01-10 19:37:58 +08:00
7ec609b289
HDFS-11273. Move TransferFsImage#doGetUrl function to a Util class. Contributed by Hanisha Koneru.
2017-01-09 18:05:33 -08:00
603cbcd513
HDFS-11292. log lastWrittenTxId etc info in logSyncAll. Contributed by Yongjun Zhang.
2017-01-09 15:44:42 -08:00
91bf504440
HDFS-11301. Double wrapping over RandomAccessFile in LocalReplicaInPipeline#createStreams. Contributed by Hanisha Koneru.
2017-01-09 10:32:18 -08:00
4c8f9e1302
HDFS-9483. Documentation does not cover use of "swebhdfs" as URL scheme for SSL-secured WebHDFS. Contributed by Surendra Singh Lilhore.
2017-01-05 15:04:47 -08:00
a605ff36a5
HDFS-11280. Allow WebHDFS to reuse HTTP connections to NN. Contributed by Zheng Shao.
2017-01-04 21:01:23 -08:00
87bb1c49bb
HDFS-11279. Cleanup unused DataNode#checkDiskErrorAsync(). Contributed by Hanisha Koneru
2017-01-03 18:25:46 -08:00
7fcc73fc0d
HDFS-11156. Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang.
2017-01-03 09:58:00 -08:00
b31e1951e0
Revert "HDFS-11280. Allow WebHDFS to reuse HTTP connections to NN. Contributed by Zheng Shao."
...
This reverts commit b811a1c14d
2017-01-03 19:19:29 +05:30
b811a1c14d
HDFS-11280. Allow WebHDFS to reuse HTTP connections to NN. Contributed by Zheng Shao.
2016-12-30 22:17:49 -08:00
e769957556
HDFS-11275. Check groupEntryIndex and throw a helpful exception on failures when removing ACL.
2016-12-29 10:36:50 -08:00
a4f66655ec
HDFS-11267. Avoid redefinition of storageDirs in NNStorage and cleanup its accessors in Storage. (Manoj Govindassamy via lei)
2016-12-29 16:57:40 +08:00
e9f1396834
HDFS-11251. ConcurrentModificationException during DataNode#refreshVolumes. (Manoj Govindassamy via lei)
2016-12-29 15:11:25 +08:00
603f3ef138
HDFS-11274. Datanode should only check the failed volume upon IO errors. Contributed by Xiaoyu Yao.
2016-12-28 22:08:13 -08:00
ce3613c965
HDFS-11278. Add missing @Test annotation for TestSafeMode.testSafeModeUtils() (Contributed by Lukas Majercak via Daniel Templeton)
2016-12-28 20:09:00 -08:00
0ddb8defad
HDFS-11252. TestFileTruncate#testTruncateWithDataNodesRestartImmediately can fail with BindException. Contributed by Yiqun Lin.
2016-12-28 10:34:33 +05:30
e216e8e233
HADOOP-13932. Fix indefinite article in comments (Contributed by LiXin Ge via Daniel Templeton)
2016-12-27 13:03:16 -08:00
c0e0ef2969
HDFS-11270. Document the missing options of NameNode bootstrap command. Contributed by Yiqun Lin
2016-12-27 11:21:10 -08:00
ea547529cb
HDFS-11272. Refine the assert messages in TestFSDirAttrOp. Contributed by Jimmy Xiang.
2016-12-26 17:15:45 +09:00
8f218ea284
HDFS-11250. Fix a typo in ReplicaUnderRecovery#setRecoveryID. Contributed by Yiqun Lin.
2016-12-26 16:51:29 +09:00
483cd06ad4
HDFS-11271. Typo in NameNode UI. Contributed by Wei-Chiu Chuang.
2016-12-26 16:36:54 +09:00
4e9029653d
HDFS-10917. Collect peer performance statistics on DataNode. Contributed by Xiaobing Zhou.
2016-12-22 23:46:58 -08:00
56a13a6a59
HDFS-11216. Add remoteBytesRead counter metrics for erasure coding reconstruction task. Contributed by Sammi Chen
2016-12-22 14:18:54 +08:00
f6e80acd68
HDFS-11258. File mtime change could not save to editlog. Contributed by Jimmy Xiang.
2016-12-21 13:04:03 -08:00
f678080dbd
HDFS-11182. Update DataNode to use DatasetVolumeChecker. Contributed by Arpit Agarwal.
2016-12-20 13:53:32 -08:00
5daa8d8631
HDFS-10913. Introduce fault injectors to simulate slow mirrors. Contributed by Xiaobing Zhou.
2016-12-20 13:17:52 -08:00
4af66b1d66
HDFS-11247. Add a test to verify NameNodeMXBean#getDecomNodes() and Live/Dead Decom Nodes shown in NameNode WebUI. Contributed by Manoj Govindassamy.
2016-12-20 12:42:13 -08:00
5b7acdd206
HDFS-11195. Return error when appending files by webhdfs rest api fails. Contributed by Yuanbo Liu.
2016-12-20 12:24:00 -08:00
1b401f6a73
HDFS-11263. ClassCastException when we use Bzipcodec for Fsimage compression. Contributed by Brahma Reddy Battula.
2016-12-20 20:54:03 +05:30
fe4ff64a4a
HDFS-10959. Adding per disk IO statistics and metrics in DataNode. Contributed by Xiaoyu Yao.
2016-12-19 22:42:54 -08:00
d2026ae9e6
HDFS-11262. Remove unused variables in FSImage.java. Contributed by Jagadesh Kiran N.
2016-12-20 14:34:35 +09:00
a95639068c
HDFS-9911. TestDataNodeLifeline Fails intermittently. Contributed by Yiqun Lin
2016-12-16 09:46:21 -08:00
cee0c468b0
HDFS-11188. Change min supported DN and NN versions back to 2.x. Contributed by Andrew Wang.
2016-12-15 16:48:50 -08:00
aebb9127ba
HDFS-11160. VolumeScanner reports write-in-progress replicas as corrupt incorrectly. Contributed by Wei-Chiu Chuang and Yongjun Zhang.
2016-12-15 16:32:50 -08:00
6fce191549
HDFS-11253. FileInputStream leak on failure path in BlockSender. Contributed by Arpit Agarwal
2016-12-15 15:39:15 -08:00
ac0d402c6a
HDFS-11094. Send back HAState along with NamespaceInfo during a versionRequest as an optional parameter. Contributed by Eric Badger
2016-12-15 14:53:53 -08:00
36947f79ba
HDFS-11249. Redundant toString() in DFSConfigKeys.java. Contributed by Jagadesh Kiran N.
2016-12-16 00:33:27 +09:00
6ba9587d37
HDFS-10958. Add instrumentation hooks around Datanode disk IO.
2016-12-14 11:18:58 -08:00
72bff192cd
HDFS-11204. Document the missing options of hdfs zkfc command. Contributed by Yiqun Lin.
2016-12-14 18:50:43 +09:00
1f14f6d038
HDFS-8411. Add bytes count metrics to datanode for ECWorker. Contributed by Sammi Chen and Andrew Wang
2016-12-14 14:50:50 +08:00
fbdbbd57cd
HDFS-10684. WebHDFS DataNode calls fail without parameter createparent. Contributed by John Zhuge.
2016-12-13 18:01:31 -08:00
e24a923db5
HDFS-11164: Mover should avoid unnecessary retries if the block is pinned. Contributed by Rakesh R
2016-12-13 17:09:58 -08:00
754f15bae6
HDFS-11226. cacheadmin, cryptoadmin and storagepolicyadmin should support generic options. Contributed by Brahma Reddy Battula
2016-12-12 18:21:15 -08:00
2d4731c067
HDFS-11233. Fix javac warnings related to the deprecated APIs after upgrading Jackson. Contributed by Yiqun Lin.
2016-12-13 11:11:15 +09:00
2a28e8cf04
HDFS-11229. HDFS-11056 failed to close meta file. Contributed by Wei-Chiu Chuang.
2016-12-09 16:02:42 -08:00
5bd7dece92
HDFS-11217. Annotate NameNode and DataNode MXBean interfaces as Private/Stable. Contributed by Jagadesh Kiran N.
2016-12-09 12:05:42 -08:00
d1d4aba71b
HDFS-11224. Lifeline message should be ignored for dead nodes (Contributed by Vinayakumar B)
2016-12-09 14:53:50 +05:30
13d8e552d2
Revert "HDFS-5079. Cleaning up NNHAStatusHeartbeat.State from DatanodeProtocolProtos. Contributed by Tao Luo."
...
This reverts commit 4c8db60092
2016-12-08 15:41:49 -06:00
9f8344db50
Revert "HDFS-11094. Send back HAState along with NamespaceInfo during a versionRequest as an optional parameter. Contributed by Eric Badger"
...
This reverts commit 8c4680852b
2016-12-08 15:36:08 -06:00
401c731872
HDFS-11197. Listing encryption zones fails when deleting a EZ that is on a snapshotted directory. Contributed by Wellington Chevreuil.
2016-12-08 12:40:20 -08:00
4c2cf5560f
HDFS-11223. Fix typos in HttpFs documentations. Contributed by Yiqun Lin.
2016-12-08 20:52:24 +09:00
9ef89ede2f
HDFS-11140. Directory Scanner should log startup message time correctly. Contributed by Yiqun Lin.
2016-12-08 13:38:02 +09:00
deb4743b2c
Revert "HDFS-11140. Directory Scanner should log startup message time correctly. Contributed by Yiqun Lin."
...
This reverts commit 0857641f62
2016-12-08 13:37:06 +09:00
0857641f62
HDFS-11140. Directory Scanner should log startup message time correctly. Contributed by Yiqun Lin.
2016-12-08 13:34:45 +09:00
ea2895f4ed
HDFS-8630. WebHDFS : Support get/set/unset StoragePolicy. Contributed by Surendra Singh Lilhore.
2016-12-07 15:52:16 -08:00
c73e08a6da
HDFS-10206. Datanodes not sorted properly by distance when the reader isn't a datanode. (Nandakumar via mingma)
2016-12-07 08:26:09 -08:00
c8d0a049b0
HADOOP-13859. TestConfigurationFieldsBase fails for fields that are DEFAULT values of skipped properties. (Haibo Chen via kasha)
2016-12-06 14:10:58 -08:00
f53b481ffe
HDFS-11172. Support an erasure coding policy using RS 10 + 4. Contributed by Wei Zhou.
2016-12-06 12:49:38 -08:00
df983b524a
HDFS-10930. Refactor: Wrap Datanode IO related operations. Contributed by Xiaoyu Yao.
2016-12-06 11:05:47 -08:00
43cb1678cc
HDFS-11211. Add a time unit to the DataNode client trace format. Contributed by Jagadesh Kiran N
2016-12-06 09:12:11 -08:00
ed89856755
HDFS-11178. TestAddStripedBlockInFBR#testAddBlockInFullBlockReport fails frequently in trunk. Contributed By Yiqun Lin.
2016-12-06 22:30:43 +05:30
4dd4f3a4bb
HDFS-11198. NN UI should link DN web address using hostnames. Contributed by Weiwei Yang.
2016-12-06 08:06:50 -06:00
08a7253bc0
Revert "HDFS-11156. Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang"
...
This reverts commit c7ff34f8dc
2016-12-05 23:09:35 -08:00
8e63fa98ea
HDFS-10581. Hide redundant table on NameNode WebUI when no nodes are decomissioning. Contributed by Weiwei Yang.
2016-12-05 18:13:53 -08:00
dcedb72af4
Revert "HADOOP-10930. Refactor: Wrap Datanode IO related operations. Contributed by Xiaoyu Yao."
...
This reverts commit aeecfa24f4
2016-12-05 12:44:20 -08:00
8c4680852b
HDFS-11094. Send back HAState along with NamespaceInfo during a versionRequest as an optional parameter. Contributed by Eric Badger
2016-12-05 11:48:58 -08:00
1b5cceaffb
Revert "HDFS-11201. Spelling errors in the logging, help, assertions and exception messages. Contributed by Grant Sohn."
...
This reverts commit b9522e86a5
2016-12-05 10:54:43 -08:00
b9522e86a5
HDFS-11201. Spelling errors in the logging, help, assertions and exception messages. Contributed by Grant Sohn.
2016-12-05 10:48:25 -08:00
c51bfd29cd
HDFS-11181. Fuse wrapper has a typo. Contributed by Wei-Chiu Chuang.
2016-12-05 08:44:40 -08:00
c7ff34f8dc
HDFS-11156. Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang
2016-12-02 11:10:13 -08:00
96c574927a
HDFS-8674. Improve performance of postponed block scans. Contributed by Daryn Sharp.
2016-12-01 12:11:27 -06:00
e0fa49234f
HDFS-11180. Intermittent deadlock in NameNode when failover happens.
2016-12-01 23:08:59 +09:00
7226a71b1f
HDFS-5517. Lower the default maximum number of blocks per file. Contributed by Aaron T. Myers and Andrew Wang.
2016-11-30 15:58:31 -08:00
625df87c7b
HDFS-8678. Bring back the feature to view chunks of files in the HDFS file browser. Contributed by Ivo Udelsmann.
2016-11-30 09:12:15 -08:00
51e6c1cc3f
HDFS-10994. Support an XOR policy XOR-2-1-64k in HDFS. Contributed by Sammi Chen
2016-11-30 15:52:56 +08:00
cfd8076f81
Revert due to an error "HDFS-10994. Support an XOR policy XOR-2-1-64k in HDFS. Contributed by Sammi Chen"
...
This reverts commit 5614f847b2ef2a5b70bd9a06edc4eba06174c6.
2016-11-30 15:44:52 +08:00
aeecfa24f4
HADOOP-10930. Refactor: Wrap Datanode IO related operations. Contributed by Xiaoyu Yao.
2016-11-29 20:52:36 -08:00
eaaa32950c
HDFS-11149. Support for parallel checking of FsVolumes.
2016-11-29 20:31:02 -08:00
b407d53195
Revert "HDFS-5517. Lower the default maximum number of blocks per file. Contributed by Aaron T. Myers."
...
This reverts commit 09451252fa
2016-11-29 10:50:10 -08:00
00096dcc0c
HADOOP-13812. Upgrade Tomcat to 6.0.48. Contributed by John Zhuge.
2016-11-29 19:47:02 +09:00
09451252fa
HDFS-5517. Lower the default maximum number of blocks per file. Contributed by Aaron T. Myers.
2016-11-29 00:12:35 -08:00
47ca9e26fb
HADOOP-13838. KMSTokenRenewer should close providers (xiaochen via rkanter)
2016-11-28 18:08:09 -08:00
a2b1ff0257
HADOOP-13828. Implement getFileChecksum(path, length) for ViewFileSystem. Contributed by Manoj Govindassamy.
2016-11-28 11:54:43 -08:00
5d5614f847
HDFS-10994. Support an XOR policy XOR-2-1-64k in HDFS. Contributed by Sammi Chen
2016-11-28 14:34:44 +08:00
209e805430
HADOOP-13506. Redundant groupid warning in child projects. Contributed by Kai Sasaki.
2016-11-28 14:34:57 +09:00
bb1b5e083b
HDFS-11174. Wrong HttpFS test command in doc. Contributed by John Zhuge.
2016-11-25 21:28:29 +09:00
636f7547b7
HDFS-11175. Document uppercase key names are not supported in TransparentEncryption.md. Contributed by Yiqun Lin.
2016-11-25 17:48:54 +09:00
de4894936a
HDFS-11368. Erasure Coding: Deprecate replication-related config keys. Contributed by Rakesh R.
2016-11-23 16:42:06 -08:00
dd98a8005a
HDFS-11058. Implement 'hadoop fs -df' command for ViewFileSystem. Contributed by Manoj Govindassamy.
2016-11-23 16:40:39 -08:00
3a09e5970d
HDFS-11145. Implement getTrashRoot() for ViewFileSystem. (Manoj Govindassamy via lei)
2016-11-22 13:52:09 -08:00
613b902b98
HDFS-11148. Update DataNode to use StorageLocationChecker at startup.
2016-11-22 10:50:25 -08:00
afcf8d38e7
HADOOP-13646. Remove outdated overview.html. Contributed By Brahma Reddy Battula.
2016-11-22 19:45:45 +05:30
49a09179e3
HDFS-10966. Enhance Dispatcher logic on deciding when to give up a source DataNode. Contributed by Mark Wagner and Zhe Zhang.
2016-11-21 10:13:24 -06:00
f922067748
HDFS-11101. TestDFSShell#testMoveWithTargetPortEmpty fails intermittently. Contributed By Brahma Reddy Battula.
2016-11-21 19:34:52 +05:30
c68dad18ab
HDFS-11144. TestFileCreationDelete#testFileCreationDeleteParent fails wind bind exception. Contributed By Brahma Reddy Battula.
2016-11-21 10:58:34 +05:30
d232625f73
HDFS-11113. Document dfs.client.read.striped configuration in hdfs-default.xml. Contributed by Rakesh R.
2016-11-21 14:05:15 +09:00
f6ffa11635
HDFS-11117. Refactor striped file tests to allow flexibly test erasure coding policy. Contributed by Sammi Chen
2016-11-18 16:08:59 +06:00
c0b1a44f6c
HADOOP-13660. Upgrade commons-configuration version. Contributed by Sean Mackrory.
2016-11-17 22:48:35 -06:00
b2d4b7b1b9
HDFS-11134. Fix bind exception threw in TestRenameWhileOpen. Contributed By Yiqun Lin.
2016-11-17 10:22:30 +05:30
04a024b683
HDFS-11147. Remove confusing log output in FsDatasetImpl#getInitialVolumeFailureInfos. Contributed by Chen Liang.
2016-11-16 14:47:34 -08:00
c90891e7b3
HDFS-11105. TestRBWBlockInvalidation#testRWRInvalidation fails intermittently. Contributed by Yiqun Lin
2016-11-16 13:02:10 -08:00
4fcea8a0c8
HDFS-8870. Lease is leaked on write failure. Contributed by Kuhu Shukla.
2016-11-15 14:47:53 -06:00
5af572b644
HADOOP-13427. Eliminate needless uses of FileSystem#{exists(), isFile(), isDirectory()}. Contributed by Steve Loughran and Mingliang Liu
2016-11-15 10:57:00 -08:00
ff0b99eafe
HDFS-10872. Add MutableRate metrics for FSNamesystemLock operations. Contributed by Erik Krogen.
2016-11-14 11:05:29 -08:00
6efb8c9c60
HDFS-11135. The tests in TestBalancer run fails due to NPE. Contributed By Yiqun Lin
2016-11-14 20:39:39 +05:30
96f43921ba
HADOOP-13813. TestDelegationTokenFetcher#testDelegationTokenWithoutRenewer is failing. Contributed by Mingliang Liu
2016-11-12 10:35:22 -08:00
2ee18fc15e
HDFS-11129. TestAppendSnapshotTruncate fails with bind exception. Contributed by Brahma Reddy Battula
2016-11-11 22:12:27 -08:00
3d26717777
HDFS-11119. Support for parallel checking of StorageLocations on DataNode startup.
...
This closes #155 .
2016-11-11 15:02:52 -08:00
4484b48498
HDFS-10941. Improve BlockManager#processMisReplicatesAsync log. Contributed by Chen Liang.
2016-11-11 14:31:23 -08:00
aa6010ccca
HDFS-11122. TestDFSAdmin#testReportCommand fails due to timed out. Contributed by Yiqun Lin
2016-11-11 12:29:47 -08:00
1ae57f0f75
HDFS-11128. CreateEditsLog throws NullPointerException. Contributed by Hanisha Koneru.
2016-11-11 20:08:15 +05:30
8848a8a76c
HDFS-11116. Fix javac warnings caused by deprecation of APIs in TestViewFsDefaultValue. Contributed by Yiqun Lin.
2016-11-11 13:45:30 +09:00
ca68f9cb5b
HDFS-9337. Validate required params for WebHDFS requests (Contributed by Jagadesh Kiran N)
2016-11-10 16:51:33 +05:30
71adf44c3f
HDFS-11120. TestEncryptionZones should waitActive. Contributed by John Zhuge.
2016-11-09 17:16:07 -08:00
c619e9b43f
HDFS-11056. Concurrent append and read operations lead to checksum error. Contributed by Wei-Chiu Chuang.
2016-11-09 09:16:50 -08:00
ed0bebabaa
HDFS-9482. Replace DatanodeInfo constructors with a builder pattern. Contributed by Brahma Reddy Battula.
2016-11-08 18:17:07 -08:00
62d8c17dfd
HDFS-11083. Add unit test for DFSAdmin -report command. Contributed by Xiaobing Zhou
2016-11-08 17:04:30 -08:00
e1c6ef2efa
HADOOP-13789. Hadoop Common includes generated test protos in both jar and test-jar. Contributed by Sean Busbey.
2016-11-08 16:33:55 -08:00
3fff158587
HDFS-11114. Support for running async disk checks in DataNode.
...
This closes #153 .
2016-11-07 18:45:53 -08:00
049e7d27be
HDFS-10970. Update jackson from 1.9.13 to 2.x in hadoop-hdfs.
2016-11-07 11:16:31 +09:00
d8bab3dcb6
HDFS-10756. Expose getTrashRoot to HTTPFS and WebHDFS. Contributed by Yuanbo Liu.
2016-11-04 18:06:55 -07:00
0c0ab102ab
HDFS-11085. Add unit test for NameNode failing to start when name dir is unwritable. Contributed by Xiaobing Zhou
2016-11-04 14:34:49 -07:00
69dd5fa2d4
HDFS-11098. Datanode in tests cannot start in Windows after HDFS-10638 (Contributed by Vinayakumar B)
2016-11-04 08:31:43 +05:30
5cad93d5c7
HADOOP-11088. Quash unnecessary safemode WARN message during NameNode startup. Contributed by Yiqun Lin.
2016-11-03 15:31:07 -07:00
7534aee09a
HDFS-11076. Add unit test for extended Acls. Contributed by Chen Liang
2016-11-03 13:49:10 -07:00
b71907b2ae
HDFS-11097. Fix warnings for deprecated StorageReceivedDeletedBlocks constructor. Contributed by Yiqun Lin.
2016-11-03 10:27:52 -07:00
7e521c5a49
HDFS-11080. Update HttpFS to use ConfigRedactor. Contributed by Sean Mackrory.
2016-11-02 19:11:05 -07:00
730cb0cff6
HDFS-10566. Submit plan request should throw exception if Datanode is in non-REGULAR status. Contributed by Xiaobing Zhou.
2016-11-02 17:46:56 -07:00
b59206190e
HDFS-10909. De-duplicate code in ErasureCodingWorker#initializeStripedReadThreadPool and DFSClient#initThreadsNumForStripedReads. (Manoj Govindassamy via lei)
2016-11-02 16:45:25 -07:00
9b0c17f852
HDFS-11095. BlockManagerSafeMode should respect extension period default config value (30s). Contributed by Mingliang Liu
2016-11-02 14:58:24 -07:00
f2800f523d
Revert "HADOOP-13717. Normalize daemonization behavior of the diskbalancer with balancer and mover."
...
This reverts commit bd7f5911c7
2016-11-02 13:46:33 -07:00
cb5cc0dc53
HDFS-11031. Add additional unit test for DataNode startup behavior when volumes fail. Contributed by Mingliang Liu.
2016-11-02 10:35:47 +05:30
bd7f5911c7
HADOOP-13717. Normalize daemonization behavior of the diskbalancer with balancer and mover.
2016-11-01 16:48:45 -07:00
fa1512a740
HDFS-11065. Add space quota tests for heterogenous storages. Contributed by Xiaobing Zhou
2016-11-01 14:42:03 -07:00
34173a42d4
HDFS-10633. DiskBalancer : Add the description for the new setting dfs.disk.balancer.plan.threshold.percent in HDFSDiskbalancer.md. Contributed by Yiqun Lin.
2016-11-01 11:14:03 -07:00
808f5d8985
HDFS-10499. TestNameNodeMetadataConsistency#testGenerationStampInFuture Fails Intermittently. Contributed by Yiqun Lin.
2016-11-01 21:32:52 +05:30
0c49f73a6c
HDFS-11030. TestDataNodeVolumeFailure#testVolumeFailure is flaky (though passing). Contributed by Mingliang Liu
2016-10-31 16:00:33 -07:00
a9d68d2e8e
HDFS-11074. Remove unused method FsDatasetSpi#getFinalizedBlocksOnPersistentStorage. Contributed by Hanisha Koneru.
2016-10-31 12:39:30 -07:00
e9c7a97089
HDFS-10455. Logging the username when deny the setOwner operation. Contributed by Rakesh R.
2016-10-31 00:22:51 +05:30
8a9388e5f6
HDFS-11048. Audit Log should escape control characters. Contributed by Eric Badger
2016-10-28 10:53:14 -07:00
8fa2195697
HDFS-10926. Update staled configuration properties related to erasure coding. Contributed by Sammi Chen
2016-10-28 12:37:30 +06:00
57187fdb93
HDFS-11064. Mention the default NN rpc ports in hdfs-default.xml. Contributed by Yiqun Lin.
2016-10-27 18:13:06 -07:00
5877f20f9c
HADOOP-10075. Update jetty dependency to version 9 (rkanter)
2016-10-27 16:09:00 -07:00
9e03ee5279
HDFS-11047. Remove deep copies of FinalizedReplica to alleviate heap consumption on DataNode. Contributed by Xiaobing Zhou
2016-10-27 16:00:27 -07:00
f3ac1f41b8
HDFS-9500. Fix software version counts for DataNodes during rolling upgrade. Contributed by Erik Krogen.
2016-10-27 15:58:25 -07:00
31ff42b510
HDFS-11055. Update default-log4j.properties for httpfs to imporve test logging. Contributed by Wei-Chiu Chuang.
2016-10-27 13:37:00 -07:00
ae48c496dc
HDFS-11069. Tighten the authorization of datanode RPC. Contributed by Kihwal Lee
2016-10-27 14:17:16 -05:00
db41965996
Revert "HDFS-10455. Logging the username when deny the setOwner operation. Contributed by Tianyin Xiu"
...
This reverts commit ac35ee9393
2016-10-27 14:07:28 -05:00
ac35ee9393
HDFS-10455. Logging the username when deny the setOwner operation. Contributed by Tianyin Xiu
2016-10-27 20:20:56 +05:30
1cf6ec4ad4
HDFS-8492. DN should notify NN when client requests a missing block (Contributed by Walter Su)
2016-10-27 16:44:00 +05:30
77f2684dd3
HDFS-10769. BlockIdManager.clear doesn't reset the counter for blockGroupIdGenerator (Contributed by Yiqun Lin and Rakesh R)
2016-10-27 14:16:03 +05:30
9c680d0b05
HDFS-11049. The description of dfs.block.replicator.classname is not clear. Contributed by Yiqun Lin.
2016-10-27 16:30:57 +09:00
a895c66255
HDFS-9929. Duplicate keys in NAMENODE_SPECIFIC_KEYS (Contributed by Akira Ajisaka)
2016-10-27 12:20:15 +05:30
0bdd263d82
HADOOP-13017. Implementations of InputStream.read(buffer, offset, bytes) to exit 0 if bytes==0. Contributed by Steve Loughran.
2016-10-27 15:46:59 +09:00
9f32364d28
HDFS-11038. DiskBalancer: support running multiple commands in single test. Contributed by Xiaobing Zhou.
2016-10-26 19:39:02 -07:00
55e1fb8e32
HDFS-10921. TestDiskspaceQuotaUpdate doesn't wait for NN to get out of safe mode. Contributed by Eric Badger
2016-10-26 14:11:38 -07:00
f209e93566
HDFS-10638. Modifications to remove the assumption that StorageLocation is associated with java.io.File in Datanode. (Virajith Jalaparti via lei)
2016-10-26 10:32:35 -07:00
e90af4a89b
HDFS-11053. Unnecessary superuser check in versionRequest(). Contributed by Kihwal Lee.
2016-10-26 09:16:13 -05:00
ce6bbfb23c
HDFS-11050. Change log level to 'warn' when ssl initialization fails and defaults to DEFAULT_TIMEOUT_CONN_CONFIGURATOR. Contributed by Kuhu Shukla.
2016-10-26 08:07:53 -05:00
287effff93
HDFS-10935. TestFileChecksum fails in some cases. Contributed by Sammi Chen
2016-10-26 09:40:31 +06:00
9dce9011ad
HDFS-11054. Suppress verbose log message in BlockPlacementPolicyDefault. Contributed by Chen Liang.
2016-10-25 19:50:56 -07:00
9a8a386458
HDFS-11011. Add unit tests for HDFS command 'dfsadmin -set/clrSpaceQuota'. Contributed by Xiaobing Zhou.
2016-10-25 11:24:03 -07:00
f6367c5f44
HDFS-11015. Enforce timeout in balancer. Contributed by Kihwal Lee.
2016-10-25 10:19:13 -07:00
dc3272bfe8
HDFS-11046. Duplicate '-' in the daemon log name.
2016-10-25 10:07:43 +09:00
9d175853b0
HDFS-10997. Reduce number of path resolving methods. Contributed by Daryn Sharp.
2016-10-24 17:14:51 -05:00
b18f35f097
HDFS-11042. Add missing cleanupSSLConfig() call for tests that use setupSSLConfig(). Contributed by Kuhu Shukla.
2016-10-24 08:28:44 -05:00
25438526ed
HDFS-11033. Add documents for native raw erasure coder in XOR codes. Contributed by SammiChen.
2016-10-21 16:20:07 -07:00
df857f0d10
HDFS-10975. fsck -list-corruptfileblocks does not report corrupt EC files. Contributed by Takanobu Asanuma.
2016-10-21 15:43:12 -07:00
61e30cf83c
HDFS-8410. Add computation time metrics to datanode for ECWorker. Contributed by SammiChen.
2016-10-21 13:12:41 -07:00
ae8bccd509
HADOOP-13702. Add instrumented ReadWriteLock. Contributed by Jingcheng Du
2016-10-21 12:59:54 -07:00
f63cd78f60
HDFS-10730. Fix some failed tests due to BindException. Contributed by Yiqun Lin
2016-10-21 18:16:39 +05:30
d7d87deece
HDFS-10998. Add unit tests for HDFS command 'dfsadmin -fetchImage' in HA. Contributed by Xiaobing Zhou
2016-10-20 19:51:48 -07:00
5e83a21cb6
HDFS-10976. Report erasure coding policy of EC files in Fsck. Contributed by Wei-Chiu Chuang.
2016-10-20 13:06:43 -07:00
6d2da38d16
HDFS-11018. Incorrect check and message in FsDatasetImpl#invalidate. Contributed by Yiqun Lin.
2016-10-20 10:56:39 -07:00
f872c6bc03
HADOOP-13236. truncate will fail when we use viewfilesystem. Contributed by Brahma Reddy Battula
2016-10-20 21:47:17 +05:30
6fb6b651e8
HDFS-10699. Log object instance get incorrectly in TestDFSAdmin. Contributed by Yiqun Lin
2016-10-20 21:28:16 +05:30
5c0bffddc0
HDFS-10627. Volume Scanner marks a block as "suspect" even if the exception is network-related. Contributed by Rushabh S Shah.
2016-10-20 10:48:35 -05:00
4c73be135c
HDFS-9480. Expose nonDfsUsed via StorageTypeStats. Contributed by Brahma Reddy Battula
2016-10-20 20:45:45 +05:30
73504b1bdc
HDFS-11025. TestDiskspaceQuotaUpdate fails in trunk due to Bind exception. Contributed by Yiqun Lin
2016-10-20 10:41:18 +05:30
b4564103e4
HDFS-10752. Several log refactoring/improvement suggestion in HDFS. Contributed by Hanisha Koneru.
2016-10-19 17:20:07 -07:00
c5573e6a75
HDFS-11009. Add a tool to reconstruct block meta file from CLI.
2016-10-18 22:42:28 -07:00
efdf810cf9
HADOOP-7352. FileSystem#listStatus should throw IOE upon access error. Contributed by John Zhuge.
2016-10-18 18:18:43 -07:00
c62ae7107f
HDFS-10906. Add unit tests for Trash with HDFS encryption zones. Contributed by Hanisha Koneru.
2016-10-18 14:05:43 -07:00
d26a1bb9d6
HDFS-10920. TestStorageMover#testNoSpaceDisk is failing intermittently. Contributed by Rakesh R
2016-10-18 14:51:08 +06:00
c023c74886
HADOOP-13061. Refactor erasure coders. Contributed by Kai Sasaki
2016-10-18 12:02:53 +06:00
bedfec0c10
HADOOP-13522. Add %A and %a formats for fs -stat command to print permissions. Contributed by Alex Garbarini.
2016-10-18 15:00:44 +09:00
b61fb267b9
HDFS-9390. Block management for maintenance states.
2016-10-17 17:45:41 -07:00
8fd4c37c45
HDFS-10922. Adding additional unit tests for Trash (II). Contributed by Weiwei Yang.
2016-10-17 14:21:36 -07:00
5f4ae85bd8
HDFS-11003. Expose XmitsInProgress through DataNodeMXBean. Contributed By Brahma Reddy Battula
2016-10-15 22:28:33 +05:30
5ad037df25
HDFS-11012. Unnecessary INFO logging on DFSClients for InvalidToken. Contributed by Harsh J.
...
This closes #142
2016-10-15 22:14:24 +09:00
391ce535a7
HDFS-10301. Remove FBR tracking state to fix false zombie storage detection for interleaving block reports. Contributed by Vinitha Gankidi.
2016-10-14 18:13:54 -07:00
30bb1970cc
HDFS-10558. DiskBalancer: Print the full path to plan file. Contributed by Xiaobing Zhou.
2016-10-14 17:07:59 -07:00
76cc84e6d4
HDFS-11008. Change unit test for testing parsing "-source" parameter in Balancer CLI. Contributed by Mingliang Liu
2016-10-14 14:29:02 -07:00
8c520a27cb
HDFS-10960. TestDataNodeHotSwapVolumes#testRemoveVolumeBeingWritten fails at disk error verification after volume remove. (Manoj Govindassamy via lei)
2016-10-14 13:41:59 -07:00
adb96e109f
HDFS-10827. When there are unrecoverable ec block groups, Namenode Web UI doesn't show the block names. Contributed by Takanobu Asanuma.
2016-10-14 13:21:53 -07:00
0007360c33
HDFS-10883. 's behavior is not consistent in DFS after enabling EZ. Contributed by Yuanbo Liu.
2016-10-14 11:41:29 -07:00
701c27a776
HDFS-10735 Distcp using webhdfs on secure HA clusters fails with StandbyException
2016-10-14 10:26:39 -07:00
8a9f6635a3
HDFS-10990. TestPendingInvalidateBlock should wait for IBRs. Contributed by Yiqun Lin.
2016-10-14 11:38:48 -05:00
8c721aa00a
Revert "HDFS-10990. TestPendingInvalidateBlock should wait for IBRs. Contributed by Yiqun Lin."
...
This reverts commit fdce515091
2016-10-13 13:23:28 -07:00
332a61fd74
HDFS-10987. Make Decommission less expensive when lot of blocks present. Contributed by Brahma Reddy Battula.
2016-10-13 13:52:49 -05:00
fdce515091
HDFS-10990. TestPendingInvalidateBlock should wait for IBRs. Contributed by Yiqun Lin.
2016-10-13 11:41:37 -07:00
b371c56365
HDFS-10949. DiskBalancer: deprecate TestDiskBalancer#setVolumeCapacity. Contributed by Xiaobing Zhou.
2016-10-13 10:26:07 -07:00
1291254042
HDFS-10986. DFSAdmin should log detailed error message if any. Contributed by MingLiang Liu
2016-10-13 22:05:00 +05:30
9454dc5e80
HDFS-11000. webhdfs PUT does not work if requests are routed to call queue. Contributed by Kihwal Lee.
2016-10-13 08:47:15 -05:00
901eca004d
HDFS-11002. Fix broken attr/getfattr/setfattr links in ExtendedAttributes.md. Contributed by Mingliang Liu.
2016-10-13 14:29:30 +09:00
85cd06f663
HDFS-10789. Route webhdfs through the RPC call queue. Contributed by Daryn Sharp and Rushabh S Shah.
2016-10-12 15:11:42 -05:00
7ba7092bbc
HDFS-10965. Add unit test for HDFS command 'dfsadmin -printTopology'. Contributed by Xiaobing Zhou
2016-10-11 17:23:54 -07:00
3c9a01062e
HDFS-10903. Replace config key literal strings with config key names II: hadoop hdfs. Contributed by Chen Liang
2016-10-11 16:29:30 -07:00
61f0490a73
HDFS-10984. Expose nntop output as metrics. Contributed by Siddharth Wagle.
2016-10-11 15:55:02 -07:00
dacd3ec66b
HDFS-10991. Export hdfsTruncateFile symbol in libhdfs. Contributed by Surendra Singh Lilhore.
2016-10-11 15:07:14 -07:00
809cfd27a3
HDFS-10916. Switch from "raw" to "system" xattr namespace for erasure coding policy. (Andrew Wang via lei)
2016-10-11 10:04:46 -07:00
96b12662ea
HDFS-10637. Modifications to remove the assumption that FsVolumes are backed by java.io.File. (Virajith Jalaparti via lei)
2016-10-10 15:30:03 -07:00
b963818621
HDFS-10988. Refactor TestBalancerBandwidth. Contributed by Brahma Reddy Battula
2016-10-10 13:19:17 -07:00
3441c746b5
HDFS-10972. Add unit test for HDFS command 'dfsadmin -getDatanodeInfo'. Contributed by Xiaobing Zhou
2016-10-10 11:33:37 -07:00
af50da3298
HDFS-10895. Update HDFS Erasure Coding doc to add how to use ISA-L based coder. Contributed by Sammi Chen
2016-10-10 11:55:49 +06:00
4d106213c0
HDFS-10968. BlockManager#isInNewRack should consider decommissioning nodes. Contributed by Jing Zhao.
2016-10-07 22:44:54 -07:00
6a38d118d8
HDFS-10797. Disk usage summary of snapshots causes renamed blocks to get counted twice. Contributed by Sean Mackrory.
2016-10-07 17:37:15 -07:00
e57fa81d95
HDFS-10980. Optimize check for existence of parent directory. Contributed by Daryn Sharp.
2016-10-07 17:20:15 -05:00
3565c9af17
HDFS-10979. Pass IIP for FSDirDeleteOp methods. Contributed by Daryn Sharp.
2016-10-07 14:15:59 -05:00
3059b251d8
HDFS-10933. Refactor TestFsck. Contributed by Takanobu Asanuma.
2016-10-07 10:17:50 -07:00
be3cb10f53
HDFS-10969. Fix typos in hdfs-default.xml Contributed by Yiqun Lin
2016-10-07 22:18:40 +05:30
48b9d5fd2a
HDFS-10955. Pass IIP for FSDirAttr methods. Contributed by Daryn Sharp.
2016-10-06 16:33:46 -05:00
1d330fbaf6
HADOOP-13150. Avoid use of toString() in output of HDFS ACL shell commands. Contributed by Chris Nauroth.
2016-10-06 13:19:16 -07:00
f32e9fc8f7
HDFS-10939. Reduce performance penalty of encryption zones. Contributed by Daryn sharp.
2016-10-06 15:11:14 -05:00
72a2ae6452
HADOOP-13688. Stop bundling HTML source code in javadoc JARs.
2016-10-06 11:19:38 -07:00
311954883f
HDFS-10957. Retire BKJM from trunk (Vinayakumar B)
2016-10-06 19:28:25 +05:30
b90fc70d67
HDFS-10963. Reduce log level when network topology cannot find enough datanodes. Contributed by Xiao chen
2016-10-06 17:47:31 +05:30
202325485d
HDFS-10893. Refactor TestDFSShell by setting up MiniDFSCluser once for all commands test. Contributed by Mingliang Liu
2016-10-05 22:33:29 -07:00
c5ca216915
HDFS-10683. Make class Token$PrivateToken private. Contributed by John Zhuge.
2016-10-05 17:36:50 -07:00
e68c7b96c7
HDFS-10962. TestRequestHedgingProxyProvider is flaky.
2016-10-05 15:08:56 -07:00
8867762256
HDFS-10826. Correctly report missing EC blocks in FSCK. Contributed by Takanobu Asanuma.
2016-10-05 10:52:50 -07:00
44f48ee96e
HDFS-10956. Remove rename/delete performance penalty when not using snapshots. Contributed by Daryn Sharp.
2016-10-04 15:05:09 -05:00
382307cbdd
HDFS-10878. TestDFSClientRetries#testIdempotentAllocateBlockAndClose throws ConcurrentModificationException. Contributed by Rushabh S Shah.
2016-10-04 11:09:03 -05:00
ef7f06f7d1
HDFS-10947. Correct the API name for truncate in webhdfs document. Contributed by Surendra Singh Lilhore
2016-10-04 18:46:42 +05:30
5ea0210c7f
HDFS-10944. Correct the javadoc of dfsadmin#disallowSnapshot. Contributed by Jagadesh Kiran N
2016-10-04 18:33:22 +05:30
f61e3d13e9
HDFS-10934. TestDFSShell#testStat fails intermittently. Contributed by Eric Badger
2016-10-03 22:34:41 -04:00
8078a5efd0
HDFS-10810. setreplication removing block from underconstrcution temporarily. Contributed by Brahma Reddy Battula
2016-10-03 21:50:16 -04:00
853d65a157
HDFS-10918. Add a tool to get FileEncryptionInfo from CLI. Contributed by Xiao Chen.
2016-10-03 16:01:54 -07:00
607705c488
HDFS-10690. Optimize insertion/removal of replica in ShortCircuitCache. Contributed by Fenghua Hu.
2016-10-03 10:53:21 -07:00
744208431f
HDFS-10940. Reduce performance penalty of block caching when not used. Contributed by Daryn Sharp.
2016-10-03 11:27:23 -05:00
90020624b0
HDFS-10619. Cache path in InodesInPath. Contributed by Daryn Sharp.
2016-10-03 09:13:04 -05:00
fe9ebe20ab
Revert "HDFS-10923. Make InstrumentedLock require ReentrantLock."
...
This reverts commit c7ce6fdc20
2016-09-30 23:11:51 -07:00
c7ce6fdc20
HDFS-10923. Make InstrumentedLock require ReentrantLock.
2016-09-30 23:00:06 -07:00
434c5ea75d
HDFS-10896. Move lock logging logic from FSNamesystem into FSNamesystemLock. Contributed by Erik Krogen.
2016-09-30 13:16:08 -07:00
7fad1221d6
HDFS-10907. Fix Erasure Coding documentation. Contributed by Manoj Govindassamy.
2016-09-30 12:51:27 -07:00
ee33a02234
HDFS-10910. HDFS Erasure Coding doc should state its currently supported erasure coding policies. Contributed by Yiqun Lin.
2016-09-30 12:48:11 -07:00
2ab1ef15c5
HDFS-10908. Improve StripedBlockReader#createBlockReader error logging. Contributed by Manoj Govindassamy.
2016-09-30 12:35:47 -07:00
a0730aa5ce
HDFS-10851. FSDirStatAndListingOp: stop passing path as string. Contributed by Daryn Sharp.
2016-09-30 13:03:24 -05:00
0670149c88
HDFS-10850. getEZForPath should NOT throw FNF. Contributed by Andrew Wang.
2016-09-30 08:44:18 -05:00
84c6264079
HDFS-10892. Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'. Contributed by Mingliang Liu
2016-09-28 16:03:51 -07:00
92e5e91598
HDFS-10914. Move remnants of oah.hdfs.client to hadoop-hdfs-client.
2016-09-28 16:01:03 -07:00
5f34402ada
HDFS-10779. Rename does not need to re-solve destination. Contributed by Daryn Sharp.
2016-09-28 17:57:23 -05:00
c3b235e565
HDFS-10824. MiniDFSCluster#storageCapacities has no effects on real capacity. Contributed by Xiaobing Zhou.
2016-09-28 11:47:37 -07:00
03f519a757
HDFS-9850. DiskBalancer: Explore removing references to FsVolumeSpi. Contributed by Manoj Govindassamy.
2016-09-27 21:35:06 -07:00
6437ba18c5
HDFS-10915. Fix time measurement bug in TestDatanodeRestart. Contributed by Xiaobing Zhou
2016-09-27 20:40:17 -07:00
9c9736463b
HDFS-10900. DiskBalancer: Complete the documents for the report command. Contributed by Yiqun Lin.
2016-09-27 19:28:41 -07:00
2acfb1e1e4
HDFS-10376. Enhance setOwner testing. (John Zhuge via Yongjun Zhang)
2016-09-27 14:55:28 -07:00
875062b5bc
HADOOP-13544. JDiff reports unncessarily show unannotated APIs and cause confusion while our javadocs only show annotated and public APIs. (vinodkv via wangda)
2016-09-27 11:26:45 -07:00
8ae4729107
HDFS-10828. Fix usage of FsDatasetImpl object lock in ReplicaMap. (Arpit Agarwal)
2016-09-27 10:02:15 -07:00
f55eb981dd
HDFS-10426. TestPendingInvalidateBlock failed in trunk. Contributed by Yiqun Lin.
2016-09-28 01:29:24 +09:00
df1d0f5d56
HDFS-10889. Remove outdated Fault Injection Framework documentaion. Contributed by Brahma Reddy Battula
2016-09-27 19:33:10 +05:30
edf0d0f8b2
HDFS-9444. Add utility to find set of available ephemeral ports to ServerSocketUtil. Contributed by Masatake Iwasaki
2016-09-27 07:04:37 +05:30
059058f961
HDFS-10713. Throttle FsNameSystem lock warnings. Contributed by Hanisha Koneru.
2016-09-26 17:09:47 -07:00
8e06d865c4
Addendum patch for HDFS-10609. Uncaught InvalidEncryptionKeyException during pipeline recovery may abort downstream applications.
2016-09-26 14:41:04 -07:00
3ae652f821
HDFS-10609. Uncaught InvalidEncryptionKeyException during pipeline recovery may abort downstream applications. Contributed by Wei-Chiu Chuang.
2016-09-26 13:11:32 -07:00
3e37e243ee
HDFS-10869. Remove the unused method InodeId#checkId(). Contributed by Jagadesh Kiran N
2016-09-25 11:02:46 +05:30
6eb700ee74
HDFS-10866. Fix Eclipse Java compile errors related to generic parameters. Contributed by Konstantin Shvachko.
2016-09-23 14:10:11 -07:00
74b3dd514c
HDFS-10876. Dispatcher#dispatch should log IOException stacktrace. Contributed by Manoj Govindassamy.
2016-09-23 13:26:57 -07:00
a5bb88c8e0
HDFS-10843. Update space quota when a UC block is completed rather than committed. Contributed by Erik Krogen.
2016-09-23 10:37:46 -07:00
bbdf350ff9
HDFS-10886. Replace fs.default.name with fs.defaultFS in viewfs document.
2016-09-23 19:21:34 +05:30
c4480f7f42
HDFS-10894. Remove the redundant charactors for command -saveNamespace in HDFSCommands.md Contributed by Yiqun Lin.
2016-09-23 19:11:02 +05:30
d85d9b2e7b
HDFS-10871. DiskBalancerWorkItem should not import jackson relocated by htrace. Contributed by Manoj Govindassamy.
2016-09-22 19:36:16 -07:00
40acacee08
HDFS-10877. Make RemoteEditLogManifest.committedTxnId optional in Protocol Buffers. Contributed by Sean Mackrory.
2016-09-22 11:43:11 -07:00
964e546ab1
HDFS-9333. Some tests using MiniDFSCluster errored complaining port in use. (iwasakims)
2016-09-21 10:35:25 +09:00
0e918dff59
HDFS-10879. TestEncryptionZonesWithKMS#testReadWrite fails intermittently. Contributed by Xiao Chen.
2016-09-20 16:56:52 -07:00
734d54c1a8
HDFS-10861. Refactor StripeReaders and use ECChunk version decode API. Contributed by Sammi Chen
2016-09-21 21:34:48 +08:00
e52d6e7a46
HDFS-10875. Optimize du -x to cache intermediate result. Contributed by Xiao Chen.
2016-09-19 21:44:42 -07:00
c54f6ef30f
HDFS-10868. Remove stray references to DFS_HDFS_BLOCKS_METADATA_ENABLED.
2016-09-19 11:17:03 -07:00
ea839bd48e
HDFS-10489. Deprecate dfs.encryption.key.provider.uri for HDFS encryption zones. Contributed by Xiao Chen.
2016-09-17 22:25:39 -07:00
8a40953058
HDFS-10823. Implement HttpFSFileSystem#listStatusIterator.
2016-09-16 15:37:36 -07:00
b09a03cd7d
HDFS-10862. Typos in 4 log messages. Contributed by Mehran Hassani.
2016-09-16 16:08:47 +09:00
ec3ea18875
HDFS-9895. Remove unnecessary conf cache from DataNode. Contributed by Xiaobing Zhou.
2016-09-15 15:25:47 -07:00
2a8f55a0cf
HDFS-10805. Reduce runtime for append test. Contributed by Gergely Novak.
2016-09-14 09:31:17 -07:00
ea0c2b8b05
HADOOP-13218. Migrate other Hadoop side tests to prepare for removing WritableRPCEngine. Contributed by Wei Zhou and Kai Zheng
2016-09-15 10:46:00 +08:00
86c9862bec
HDFS-10636. Modify ReplicaInfo to remove the assumption that replica metadata and data are stored in java.io.File. (Virajith Jalaparti via lei)
2016-09-13 12:54:14 -07:00
1c0d18f322
HDFS-10562. DiskBalancer: update documentation on how to report issues and debug. Contributed by Anu Engineer.
2016-09-13 12:42:10 -07:00
db6d243cf8
HDFS-10837. Standardize serializiation of WebHDFS DirectoryListing.
2016-09-13 11:02:36 -07:00
e3f7f58a5f
HDFS-10599. DiskBalancer: Execute CLI via Shell. Contributed by Manoj Govindassamy.
2016-09-13 09:38:12 -07:00
f0876b8b60
HDFS-10856. Update the comment of BPServiceActor$Scheduler#scheduleNextBlockReport. Contributed by Yiqun Lin.
2016-09-13 14:50:03 +09:00
d2466ec3e0
HDFS-10657. testAclCLI.xml setfacl test should expect mask r-x. (John Zhuge via Yongjun Zhang)
2016-09-12 21:25:06 -07:00
72dfb048a9
HDFS-10858. FBR processing may generate incorrect reportedBlock-blockGroup mapping. Contributed by Jing Zhao.
2016-09-12 16:40:11 -07:00
8a93f45a80
HDFS-10821. DiskBalancer: Report command support with multiple nodes. Contributed by Yiqun Lin.
2016-09-12 15:45:26 -07:00
58ed4fa544
HADOOP-13341. Deprecate HADOOP_SERVERNAME_OPTS; replace with (command)_(subcommand)_OPTS
...
This commit includes the following changes:
HADOOP-13356. Add a function to handle command_subcommand_OPTS
HADOOP-13355. Handle HADOOP_CLIENT_OPTS in a function
HADOOP-13554. Add an equivalent of hadoop_subcmd_opts for secure opts
HADOOP-13562. Change hadoop_subcommand_opts to use only uppercase
HADOOP-13358. Modify HDFS to use hadoop_subcommand_opts
HADOOP-13357. Modify common to use hadoop_subcommand_opts
HADOOP-13359. Modify YARN to use hadoop_subcommand_opts
HADOOP-13361. Modify hadoop_verify_user to be consistent with hadoop_subcommand_opts (ie more granularity)
HADOOP-13564. modify mapred to use hadoop_subcommand_opts
HADOOP-13563. hadoop_subcommand_opts should print name not actual content during debug
HADOOP-13360. Documentation for HADOOP_subcommand_OPTS
This closes apache/hadoop#126
2016-09-12 11:10:00 -07:00
cc01ed7026
HDFS-10855. Fix typos in HDFS documents. Contributed by Yiqun Lin.
2016-09-10 23:24:49 -07:00
a99bf26a08
HDFS-10830. FsDatasetImpl#removeVolumes crashes with IllegalMonitorStateException when vol being removed is in use. (Arpit Agarwal and Manoj Govindassamy)
2016-09-10 18:22:25 -07:00
bee9f57f5c
HDFS-10808. DiskBalancer does not execute multi-steps plan-redux. Contributed by Anu Engineer.
2016-09-09 15:00:39 -07:00
cba973f036
HDFS-10832. Propagate ACL bit and isEncrypted bit in HttpFS FileStatus permissions.
2016-09-09 12:12:29 -07:00
a0b0383677
Addendum patch for HDFS-9781. FsDatasetImpl#getBlockReports can occasionally throw NullPointerException. Contributed by Manoj Govindassamy.
2016-09-09 10:54:14 -07:00
baab48922a
HDFS-9849. DiskBalancer: reduce lock path in shutdown code. Contributed by Yuanbo Liu.
2016-09-08 20:00:42 -07:00
35c5943b8b
HDFS-10553. DiskBalancer: Rename Tools/DiskBalancer class to Tools/DiskBalancerCLI. Contributed by Manoj Govindassamy.
2016-09-08 19:26:56 -07:00
b07c266dca
HDFS-10831. Add log when URLConnectionFactory.openConnection failed. Contributed by yunjiong zhao.
2016-09-08 18:30:18 -07:00
011f3b24d4
HDFS-10742. Measure lock time in FsDatasetImpl. Contributed by Chen Liang
2016-09-08 17:53:40 -07:00
b6d839a60c
HDFS-10845. Change defaults in hdfs-site.xml to match timeunit type. Contributed by Yiqun Lin
2016-09-08 13:29:30 -07:00
401db4fc65
HDFS-8901. Use ByteBuffer in striping positional read. Contributed by Sammi Chen and Kai Zheng.
2016-09-08 11:54:33 -07:00
20a20c2f6e
HDFS-10847. Complete the document for FileDistribution processor in OfflineImageViewer. Contributed by Yiqun Lin.
2016-09-09 01:34:34 +09:00
2d1bf53c7e
HDFS-10844. test_libhdfs_threaded_hdfs_static and test_libhdfs_zerocopy_hdfs_static are failing.
2016-09-09 00:49:22 +09:00
63f594892e
HDFS-10778. Add -format option to make the output of FileDistribution processor human-readable in OfflineImageViewer.
2016-09-08 15:13:43 +09:00
d355573f56
Revert "HADOOP-13218. Migrate other Hadoop side tests to prepare for removing WritableRPCEngine. Contributed by Wei Zhou and Kai Zheng"
...
This reverts commit 62a9667136
2016-09-08 05:50:17 +08:00
5f23abfa30
HDFS-9038. DFS reserved space is erroneously counted towards non-DFS used. (Brahma Reddy Battula)
2016-09-06 13:37:21 -07:00
f6c0b7543f
HDFS-10841. Remove duplicate or unused variable in appendFile(). Contributed by Kihwal Lee.
2016-09-06 11:48:35 -07:00
7c213c341d
HDFS-10835. Fix typos in httpfs.sh. Contributed by John Zhuge.
2016-09-06 11:44:26 -07:00
f0d5382ff3
HDFS-6962. ACL inheritance conflicts with umaskmode. Contributed by Chris Nauroth.
2016-09-06 11:02:39 -07:00
d37dc5d1b8
HDFS-9847. HDFS configuration should accept time units. Contributed by Yiqun Lin
2016-09-06 10:38:04 -07:00
62a9667136
HADOOP-13218. Migrate other Hadoop side tests to prepare for removing WritableRPCEngine. Contributed by Wei Zhou and Kai Zheng
2016-09-07 17:05:33 +08:00
07650bc37a
HDFS-9781. FsDatasetImpl#getBlockReports can occasionally throw NullPointerException. Contributed by Manoj Govindassamy.
2016-09-02 15:33:11 -07:00
Ming Ma
Yiqun Lin
Rakesh Radhakrishnan
Brahma Reddy Battula
Zhe Zhang
Akira Ajisaka
Andrew Wang
Wei-Chiu Chuang
Arpit Agarwal
Xiao Chen
Jing Zhao
Mingliang Liu
Eric Payne
Vinayakumar B
Yongjun Zhang
Xiaoyu Yao
Chris Douglas
Kihwal Lee
Lei Xu
Masatake Iwasaki
Jitendra Pandey
Allen Wittenauer
Vinitha Reddy Gankidi
Anu Engineer
Chris Nauroth
Haohui Mai
Daniel Templeton
Kai Zheng
Uma Maheswara Rao G
Karthik Kambatla
Ravi Prakash
Robert Kanter
Robert Kanter
Erik Krogen
Benoy Antony
Wangda Tan
Konstantin V Shvachko