ca6bcc3c76
HDFS-11866. JournalNode Sync should be off by default in hdfs-default.xml. Contributed by Hanisha Koneru.
2017-05-22 17:53:47 -07:00
9cab42cc79
HDFS-11849. JournalNode startup failure exception should be logged in log file. Contributed By Surendra Singh Lilhore.
2017-05-23 02:39:19 +08:00
b6f66b0da1
YARN-6584. Correct license headers in hadoop-common, hdfs, yarn and mapreduce. Contributed by Yeliang Cang.
2017-05-22 14:10:06 +05:30
44e9ef2e90
HDFS-11842. TestDataNodeOutlierDetectionViaMetrics UT fails intermittently. Controbuted by Hanisha Koneru.
2017-05-17 19:33:37 -07:00
ec21ce425f
HDFS-11827. NPE is thrown when log level changed in BlockPlacementPolicyDefault#chooseRandom() method. Contributed by xupeng.
2017-05-16 19:51:04 -07:00
89a8edc014
HDFS-11696. Fix warnings from Spotbugs in hadoop-hdfs. Contributed by Yiqun Lin.
2017-05-16 12:41:59 -04:00
9b90e52f1e
HDFS-11641. Reduce cost of audit logging by using FileStatus instead of HdfsFileStatus. Contributed by Daryn Sharp.
2017-05-16 11:28:46 -05:00
1d1c52b42f
HDFS-11833. HDFS architecture documentation descibes outdated placement policy. Contributed by Chen Liang.
2017-05-16 11:52:55 -04:00
2397a2626e
HDFS-11818. TestBlockManager.testSufficientlyReplBlocksUsesNewRack fails intermittently. Contributed by Nathan Roberts
2017-05-12 17:42:47 -05:00
1411612aa4
HDFS-11674. reserveSpaceForReplicas is not released if append request failed due to mirror down and replica recovered (Contributed by Vinayakumar B)
2017-05-12 07:38:18 +05:30
0d5c8ed8e0
HADOOP-14401. maven-project-info-reports-plugin can be removed. Contributed by Andras Bokor.
2017-05-11 16:37:32 -05:00
83dd14aa84
HDFS-11757. Query StreamCapabilities when creating balancer's lock file. Contributed by SammiChen.
2017-05-11 14:09:49 -07:00
51b671ef18
HDFS-11681. DatanodeStorageInfo#getBlockIterator() should return an iterator to an unmodifiable set Contributed by Virajith Jalaparti
2017-05-10 22:25:28 -07:00
ad1e3e4d9f
HDFS-11755. Underconstruction blocks can be considered missing. Contributed by Nathan Roberts.
2017-05-10 14:15:57 -05:00
fab73748a1
HDFS-11745. Increase HDFS test timeouts from 1 second to 10 seconds. Contributed by Eric Badger
2017-05-10 13:02:31 -05:00
543aac9f28
HADOOP-14386. Rewind trunk from Guava 21.0 back to Guava 11.0.2.
2017-05-09 09:22:53 -07:00
54fd0e44b7
HDFS-11644. Support for querying outputstream capabilities. Contributed by Manoj Govindassamy.
2017-05-08 21:59:49 -07:00
cef2815cf4
HDFS-11702. Remove indefinite caching of key provider uri in DFSClient. Contributed by Rushabh S Shah.
2017-05-08 08:27:37 -05:00
a9a3d219fe
HDFS-9342. Erasure coding: client should update and commit block based on acknowledged size. Contributed by SammiChen.
2017-05-07 14:45:26 -07:00
e4f34ecb04
HADOOP-14390. Correct spelling of 'succeed' and variants. Contributed by Dongtao Zhang
2017-05-05 12:10:50 -07:00
a3954ccab1
HDFS-9807. Add an optional StorageID to writes. Contributed by Ewan Higgs
2017-05-05 12:01:26 -07:00
97c2e576c9
HDFS-11530. Use HDFS specific network topology to choose datanode in BlockPlacementPolicyDefault. Contributed by Yiqun Lin and Chen Liang.
2017-05-05 11:54:50 +08:00
07761af357
HDFS-11448. JN log segment syncing should support HA upgrade. Contributed by Hanisha Koneru.
2017-05-04 15:57:44 -07:00
25f5d9ad5e
HDFS-11687. Add new public encryption APIs required by Hive. (lei)
...
Change-Id: I4a23a00de63ad18022312ceb1f306a87d032d07c
2017-05-04 12:06:50 -07:00
c2a52ef9c2
HDFS-11643. Add shouldReplicate option to create builder. Contributed by SammiChen.
2017-05-04 11:39:14 -07:00
81092b1f11
HDFS-11722. Change Datanode file IO profiling sampling to percentage. Contributed by Hanisha Koneru.
2017-05-03 16:29:30 -07:00
30cd265134
Revert "HDFS-11488. JN log segment syncing should support HA upgrade. Contributed by Hanisha Koneru."
...
This reverts commit 08fb82d6d1
2017-05-03 16:28:47 -07:00
08fb82d6d1
HDFS-11488. JN log segment syncing should support HA upgrade. Contributed by Hanisha Koneru.
2017-05-03 15:46:08 -07:00
83dded556d
HDFS-11739. Fix regression in tests caused by YARN-679. Contributed by Steve Loughran
2017-05-03 11:22:44 -07:00
d9014bda93
HDFS-11717. Add unit test for HDFS-11709 StandbyCheckpointer should handle non-existing legacyOivImageDir gracefully. Contributed by Erik Krogen.
2017-05-02 18:34:11 -07:00
b0f54ea035
HDFS-11710. hadoop-hdfs-native-client build fails in trunk in Windows after HDFS-11529 (Contributed by Sailesh Mukil)
2017-05-02 10:32:38 +05:30
1058b40843
HDFS-11593. Change SimpleHttpProxyHandler#exceptionCaught log level from info to debug. Contributed by Xiaobing Zhou.
2017-05-01 16:12:51 -07:00
4cfc866436
HDFS-11714. Newly added NN storage directory won't get initialized and cause space exhaustion. Contributed by Kihwal Lee.
2017-05-01 17:29:25 -05:00
343948ca79
HDFS-11724. libhdfs compilation is broken on OS X. Contributed by John Zhuge.
2017-05-01 13:37:55 -07:00
07b98e7830
HDFS-11609. Some blocks can be permanently lost if nodes are decommissioned while dead. Contributed by Kihwal Lee.
2017-05-01 14:19:02 -05:00
b7d769d020
HADOOP-14359. Remove unnecessary shading of commons-httpclient. Contributed by Wei-Chiu Chuang.
2017-05-01 15:22:28 +09:00
19a7e94ee4
HDFS-11718. DFSStripedOutputStream hsync/hflush should not throw UnsupportedOperationException. (Manoj Govindassamy via lei)
...
Change-Id: I4cc226b80c64a0d900a3b1ce71e51f051cd29c22
2017-04-28 17:06:14 -07:00
cb672a45a0
HDFS-11605. Allow user to customize new erasure code policies. Contributed by Huafeng Wang
2017-04-28 13:18:50 +08:00
bbf8cac14d
HADOOP-13200. Implement customizable and configurable erasure coders. Contributed by Tim Yao.
2017-04-27 12:39:47 -07:00
ddaeb3e497
Revert "HADOOP-13200. Implement customizable and configurable erasure coders. Contributed by Tim Yao."
...
This reverts commit 872088c6e7
2017-04-27 12:25:47 -07:00
872088c6e7
HADOOP-13200. Implement customizable and configurable erasure coders. Contributed by Tim Yao.
2017-04-27 11:59:33 -07:00
d8a3309830
HDFS-11709. StandbyCheckpointer should handle an non-existing legacyOivImageDir gracefully. Contributed by Erik Krogen.
2017-04-27 09:50:30 -07:00
249cb210f5
HDFS-11697. Add javadoc for storage policy and erasure coding policy. Contributed by Kai Sasaki.
2017-04-27 16:12:27 +09:00
28eb2aabeb
HDFS-11384. Balancer disperses getBlocks calls to avoid NameNode's rpc queue saturation. Contributed by Konstantin V Shvachko.
2017-04-26 17:28:49 -07:00
93fa48fcf2
HDFS-11417. Add datanode admin command to get the storage info. Contributed by Surendra Singh Lilhore.
2017-04-26 17:43:29 +09:00
2f73396b59
HDFS-6708. StorageType should be encoded in the block token. Contributed by Ewan Higgs
2017-04-25 23:57:00 -07:00
4ea2778acb
HADOOP-14207. "dfsadmin -refreshCallQueue" fails with DecayRpcScheduler. Contributed by Surendra Singh Lihore.
2017-04-25 18:45:56 -07:00
e4321ec843
HDFS-11691. Add a proper scheme to the datanode links in NN web UI. Contributed by Kihwal Lee
2017-04-25 17:30:42 -05:00
fda86ef2a3
HDFS-11529. Add libHDFS API to return last exception. Contributed by Sailesh Mukil.
2017-04-22 02:56:48 -07:00
20e3ae260b
HDFS-11402. HDFS Snapshots should capture point-in-time copies of OPEN files. (Manoj Govindassamy via Yongjun Zhang)
2017-04-21 20:35:48 -07:00
5078df7be3
HDFS-11689. New exception thrown by DFSClient%isHDFSEncryptionEnabled broke hacky hive code. Contributed by Yongjun Zhang.
2017-04-21 13:36:31 -07:00
b0803388fc
HDFS-11604. Define and parse erasure code policies. Contributed by Lin Zeng
2017-04-21 13:33:33 +08:00
74a723852d
HDFS-11660. TestFsDatasetCache#testPageRounder fails intermittently with AssertionError.
2017-04-19 18:10:04 -07:00
c1549352cf
HDFS-11671. TestReconstructStripedBlocks#test2RecoveryTasksForSameBlockGroup fails.
2017-04-19 13:49:56 -07:00
8c81a16a1f
HDFS-11531. Expose hedged read metrics via libHDFS API. Contributed by Sailesh Mukil.
2017-04-18 21:11:07 -07:00
ca2488c009
HDFS-11665. HttpFSServerWebServer$deprecateEnv may leak secret. Contributed by John Zhuge.
2017-04-18 09:28:52 -07:00
8dfcd95d58
HDFS-11634. Optimize BlockIterator when interating starts in the middle. Contributed by Konstantin V Shvachko.
2017-04-17 16:56:24 -07:00
c0ca785dbb
HDFS-11652. Improve ECSchema and ErasureCodingPolicy toString, hashCode, equals.
2017-04-17 16:30:25 -07:00
ad49098eb3
HDFS-11615. FSNamesystemLock metrics can be inaccurate due to millisecond precision. Contributed by Erik Krogen.
2017-04-17 16:22:33 -07:00
a7312715a6
HDFS-10996. Ability to specify per-file EC policy at create time. Contributed by SammiChen.
2017-04-12 12:27:34 -07:00
966b1b5b44
HDFS-11565. Use compact identifiers for built-in ECPolicies in HdfsFileStatus.
2017-04-12 12:24:32 -07:00
abce613356
HDFS-11645. DataXceiver thread should log the actual error when getting InvalidMagicNumberException. Contributed by Chen Liang.
2017-04-12 11:40:58 -07:00
d4c01dde49
HDFS-11648. Lazy construct the IIP pathname. Contributed by Daryn Sharp.
2017-04-12 13:29:24 -05:00
23b1a7bdf1
HDFS-11163. Mover should move the file blocks to default storage once policy is unset. Contributed by Surendra Singh Lilhore.
2017-04-11 22:03:09 -07:00
62e4573efb
HDFS-11630. TestThrottledAsyncCheckerTimeout fails intermittently in Jenkins builds. Contributed by Hanisha Koneru.
2017-04-11 21:36:24 -07:00
3a91376707
HDFS-11558. BPServiceActor thread name is too long. Contributed by Xiaobing Zhou
2017-04-11 15:29:38 -07:00
f050afb578
HADOOP-13665. Erasure Coding codec should support fallback coder. Contributed by Kai Sasaki.
2017-04-11 07:32:13 -07:00
2fd568fdd4
HDFS-11637. Fix javac warning caused by the deprecated key used in TestDFSClientRetries#testFailuresArePerOperation. Contributed by Yiqun Lin.
2017-04-11 19:40:09 +08:00
a5e57df3c5
HADOOP-14225. Remove xmlenc dependency
2017-04-10 11:48:40 -07:00
e8bdad7385
HDFS-11623. Move system erasure coding policies into hadoop-hdfs-client.
2017-04-07 16:46:28 -07:00
203edc026c
HDFS-11633. FSImage failover disables all erasure coding policies. Contributed by Wei-Chiu Chuang.
2017-04-07 15:46:11 -07:00
96cbb4fce2
HADOOP-14202. fix jsvc/secure user var inconsistencies
...
Signed-off-by: John Zhuge <jzhuge@apache.org>
2017-04-07 08:59:21 -07:00
0eacd4c13b
HDFS-11608. HDFS write crashed with block size greater than 2 GB. Contributed by Xiaobing Zhou.
2017-04-06 16:11:55 -07:00
a49fac5302
HDFS-11362. StorageDirectory should initialize a non-null default StorageDirType. Contribute by Hanisha Koneru.
2017-04-06 14:33:16 -07:00
8c57aeb5b4
HDFS-11131. TestThrottledAsyncChecker#testCancellation is flaky.
2017-04-05 17:54:30 -07:00
539ef5aa2e
HFDS-11596. hadoop-hdfs-client jar is in the wrong directory in release tarball. Contributed by Yuanbo Liu.
2017-04-05 16:04:09 -07:00
3db8d68d63
HDFS-11628. Clarify the behavior of HDFS Mover in documentation. Contributed by Xiaobing Zhou.
2017-04-05 15:33:08 -07:00
9e0e430f18
HDFS-11538. Move ClientProtocol HA proxies into hadoop-hdfs-client. Contributed by Huafeng Wang.
2017-04-04 23:05:24 -07:00
8cb7325527
HDFS-9651. Add missing robots.txt files.
2017-04-05 12:47:45 +09:00
09e6caec3d
HDFS-9651. All web UIs should include a robots.txt file. Contributed by Lars Francke.
2017-04-05 12:45:28 +09:00
18432130a7
HADOOP-14104. Client should always ask namenode for kms provider path. Contributed by Rushabh S Shah.
2017-04-04 13:38:25 -07:00
6eba79232f
HADOOP-14271. Correct spelling of 'occurred' and variants. Contributed by Yeliang Cang
2017-04-03 20:13:14 -07:00
bbd68478d5
HDFS-11598. Improve -setrep for Erasure Coded files. Contributed by Yiqun Lin.
2017-04-03 07:57:28 -07:00
bc7aff7cec
HDFS-11515. -du throws ConcurrentModificationException. Contributed by Istvan Fajth, Wei-Chiu Chuang.
2017-04-03 07:35:09 -07:00
73835c73e2
HDFS-11560. Expose slow disks via NameNode JMX. Contributed by Hanisha Koneru.
2017-03-31 13:50:29 -07:00
5485d93bda
HDFS-11603. Improve slow mirror/disk warnings in BlockReceiver.
2017-03-31 12:10:27 -07:00
28cdc5a8dc
HDFS-11551. Handle SlowDiskReport from DataNode at the NameNode. Contributed by Hanisha Koneru.
2017-03-30 22:41:26 -07:00
bf3fb585aa
HADOOP-11794. Enable distcp to copy blocks in parallel. Contributed by Yongjun Zhang, Wei-Chiu Chuang, Xiao Chen, Rosie Li.
2017-03-30 17:38:56 -07:00
144f1cf765
Revert "HADOOP-11794. Enable distcp to copy blocks in parallel. Contributed by Yongjun Zhang, Wei-Chiu Chuang, Xiao Chen."
...
This reverts commit 064c8b25ec
2017-03-30 17:38:18 -07:00
064c8b25ec
HADOOP-11794. Enable distcp to copy blocks in parallel. Contributed by Yongjun Zhang, Wei-Chiu Chuang, Xiao Chen.
2017-03-30 17:01:15 -07:00
e5f0622a6f
HDFS-11592. Closing a file has a wasteful preconditions in NameNode. Contributed by Eric Badger
2017-03-30 15:44:21 -07:00
8c591b8d19
HDFS-10974. Document replication factor for EC files. Contributed by Yiqun Lin.
2017-03-30 11:16:05 -07:00
7297e82cb4
HADOOP-14250. Correct spelling of 'separate' and variants. Contributed by Doris Gu
2017-03-30 10:17:11 -07:00
4966a6e26e
HADOOP-14223. Extend FileStatus#toString() to include details like Erasure Coding and Encryption. Contributed by Manoj Govindassamy.
2017-03-29 14:37:21 -07:00
15e3873dc3
HDFS-11571. Typo in DataStorage exception message (Contributed by Anna Budai via Daniel Templeton)
2017-03-29 10:56:36 -07:00
84d787b9d5
HDFS-11541. Call RawErasureEncoder and RawErasureDecoder release() methods. Contributed by SammiChen.
2017-03-29 11:41:48 +05:30
6b09336438
HDFS-11577. Combine the old and the new chooseRandom for better performance. Contributed by Chen Liang.
2017-03-28 23:02:07 +08:00
64ea62c1cc
HDFS-11486. Client close() should not fail fast if the last block is being decommissioned. Contributed by Wei-Chiu Chuang and Yiqun Lin.
2017-03-28 09:13:44 +09:00
46d37a65cf
HDFS-10506. Addendum patch to include missing changes. Contributed by Akira Ajisaka.
2017-03-27 05:23:45 -07:00
96e2ab8144
HDFS-11555. Fix typos in class OfflineImageReconstructor. Contributed by Yiqun Lin.
2017-03-27 19:23:37 +08:00
d174c06b01
Add HDFS jdiff for 2.8.0 release.
2017-03-25 14:05:20 -07:00
84ddedc0b2
HADOOP-10101. Update guava dependency to the latest version. (ozawa)
2017-03-25 10:06:52 +09:00
1f66524432
HDFS-11570. Unit test for NameNodeStatusMXBean. Contributed by Hanisha Koneru.
2017-03-24 14:44:25 -07:00
1168ece596
HDFS-11534. Add counters for number of blocks in pending IBR. Contributed by Xiaobing Zhou.
2017-03-24 14:33:44 -07:00
332a997e10
HDFS-11170. Add builder-based create API to FileSystem. Contributed by SammiChen and Wei Zhou.
2017-03-24 12:56:46 -07:00
52b00600df
HADOOP-13715. Add isErasureCoded() API to FileStatus class. Contributed by Manoj Govindassamy.
2017-03-24 11:44:46 -07:00
d4f73e7e27
HDFS-11574. Spelling mistakes in the Java source. Contributed by Hu Xiaodong.
2017-03-24 09:38:17 -07:00
59d69257a8
HADOOP-9631. ViewFs should use underlying FileSystem's server side defaults. Contributed by Lohit Vijayarenu and Erik Krogen.
2017-03-22 22:22:22 -07:00
ed2d03f2a5
HDFS-11561. HttpFS doc errors. Contributed by Yuanbo Liu.
2017-03-22 22:15:24 -07:00
3b908f71c5
HDFS-11358. DiskBalancer: Report command supports reading nodes from host file. Contributed by Yiqun Lin.
2017-03-21 17:44:17 +08:00
e7c8da614c
HDFS-11545. Propagate DataNode's slow disks info to the NameNode via Heartbeats. Contributed by Hanisha Koneru.
2017-03-20 21:54:58 -07:00
b104f3a282
HDFS-6648. Order of namenodes in ConfiguredFailoverProxyProvider is undefined. Contributed by Inigo Goiri
2017-03-20 17:15:13 -07:00
ffa160ddb8
HDFS-11547. Add logs for slow BlockReceiver while writing data to disk. Contributed by Xiaobing Zhou.
2017-03-17 19:02:24 -07:00
7f8e928400
HDFS-11517. Expose slow disks via DataNode JMX. Contributed by Hanisha Koneru
2017-03-17 15:42:25 -07:00
92ea6d74ec
HDFS-11516. Admin command line should print message to stderr in failure case. Contributed by Kai Sasaki.
2017-03-17 11:26:02 -07:00
7536815039
HDFS-10394. move declaration of okhttp version from hdfs-client to hadoop-project POM. Contributed by Xiaobing Zhou.
2017-03-16 18:13:43 -07:00
4812518b23
HDFS-10530. BlockManager reconstruction work scheduling should correctly adhere to EC block placement policy. Contributed by Manoj Govindassamy and Rui Gao.
2017-03-16 15:07:38 -07:00
ba62b50eba
HDFS-10601. Improve log message to include hostname when the NameNode is in safemode. Contributed by Kuhu Shukla.
2017-03-16 10:54:59 -05:00
d69a82c89c
HDFS-11511. Support Timeout when checking single disk. Contributed by Hanisha Koneru.
2017-03-15 18:01:45 -07:00
615ac09499
HDFS-11419. DFSTopologyNodeImpl#chooseRandom optimizations. Contributed by Chen Liang.
2017-03-15 12:28:22 -07:00
4416a07c9c
HDFS-11420. Edit file should not be processed by the same type processor in OfflineEditsViewer. Contributed by Yiqun Lin.
2017-03-15 18:05:03 +08:00
cc1292e73a
HDFS-9705. Refine the behaviour of getFileChecksum when length = 0. Contributed by Kai Zheng and SammiChen.
2017-03-14 16:41:10 -07:00
7515e75103
HDFS-11505. Do not enable any erasure coding policies by default. Contributed by Manoj Govindassamy.
2017-03-14 11:47:25 -07:00
023b941e3b
HDFS-11526. Fix confusing block recovery message. Contributed by Yiqun Lin.
2017-03-14 17:49:48 +08:00
9832ae0ed8
HDFS-11482. Add storage type demand to into DFSNetworkTopology#chooseRandom. Contributed by Chen Liang.
2017-03-13 17:30:10 -07:00
55796a0946
HDFS-11395. RequestHedgingProxyProvider#RequestHedgingInvocationHandler hides the Exception thrown from NameNode. Contributed by Nandakumar.
2017-03-13 14:24:51 -07:00
79924266f8
HDFS-11512. Increase timeout on TestShortCircuitLocalRead#testSkipWithVerifyChecksum. Contributed by Eric Badger.
2017-03-13 18:22:30 +08:00
6d356b6b4d
HDFS-11340. DataNode reconfigure for disks doesn't remove the failed volumes. (Manoj Govindassamy via lei)
2017-03-10 14:37:13 -08:00
819808a016
HDFS-11506. Move ErasureCodingPolicyManager#getSystemDefaultPolicy to test code. Contributed by Manoj Govindassamy.
2017-03-09 17:29:11 -08:00
385d2cb777
HDFS-11499. Decommissioning stuck because of failing recovery. Contributed by Lukas Majercak and Manoj Govindassamy.
2017-03-09 21:13:50 +09:00
33a38a5341
HDFS-11314. Enforce set of enabled EC policies on the NameNode.
2017-03-08 16:41:44 -08:00
5ca6ef0c26
HDFS-10983. OIV tool should make an EC file explicit. Contributed by Manoj Govindassamy.
2017-03-08 15:36:19 -08:00
5addacb1e3
HDFS-11152. Start erasure coding policy ID number from 1 instead of 0 to void potential unexpected errors. Contributed by SammiChen.
2017-03-08 08:47:38 -08:00
959940b0ab
HDFS-11477. Simplify file IO profiling configuration. Contributed by Hanisha Koneru.
2017-03-07 10:12:35 -08:00
b5adc5c301
HDFS-10838. Last full block report received time for each DN should be easily discoverable. Contributed by Surendra Singh Lilhore.
2017-03-06 16:39:53 -08:00
5e74196ede
HDFS-11498. Make RestCsrfPreventionHandler and WebHdfsHandler compatible with Netty 4.0.
2017-03-06 15:04:13 -08:00
3536ce031c
HDFS-8741. Proper error msg to be printed when invalid operation type is given to WebHDFS operations. Contributed by Surendra Singh Lilhore.
2017-03-06 19:04:03 +08:00
0f336bab9c
HDFS-11494. Log message when DN is not selected for block replication. Contributed by Yiqun Lin
2017-03-03 18:47:13 -08:00
ac5ae0065a
HDFS-11476. Fix NPE in FsDatasetImpl#checkAndUpdate. Contributed by Xiaobing Zhou.
2017-03-03 13:31:20 -08:00
3085a60430
HDFS-8112. Relax permission checking for EC related operations.
2017-03-03 13:00:22 -08:00
3749152b66
HDFS-11416. Refactor out system default erasure coding policy. Contributed by Andrew Wang.
2017-03-03 09:28:03 +05:30
8f4817f2c5
Revert "HDFS-11477. Simplify file IO profiling configuration. Contributed by Hanisha Koneru."
...
This reverts commit e61491d476
2017-03-02 16:20:48 -08:00
e61491d476
HDFS-11477. Simplify file IO profiling configuration. Contributed by Hanisha Koneru.
2017-03-02 15:38:25 -08:00
b3ec531f40
HDFS-11461. DataNode Disk Outlier Detection. Contributed by Hanisha Koneru.
2017-03-02 12:45:48 -08:00
eeca8b0c4e
HDFS-11419. HDFS specific network topology classes with storage type info included. Contributed by Chen Liang.
2017-03-02 09:21:27 -08:00
4e14eaded2
Merge branch 'trunk' of https://git-wip-us.apache.org/repos/asf/hadoop into trunk
2017-03-01 20:24:49 -08:00
25c84d279b
HDFS-11412. Maintenance minimum replication config value allowable range should be [0, DefaultReplication]. (Manoj Govindassamy via mingma)
2017-03-01 20:23:52 -08:00
555d0c3995
HDFS-11478. Update EC commands in HDFSCommands.md. Contributed by Yiqun Lin.
2017-03-02 12:10:24 +08:00
82ef9accaf
HDFS-11428. Change setErasureCodingPolicy to take a required string EC policy name. Contributed by Andrew Wang.
2017-03-01 16:06:02 +05:30
dcd03df9f9
HDFS-11432. Federation : Support fully qualified path for Quota/Snapshot/cacheadmin/cryptoadmin commands. Contributed by Brahma Reddy Battula.
2017-03-01 10:45:56 +05:30
d269b488a7
HDFS-11466. Change dfs.namenode.write-lock-reporting-threshold-ms default from 1000ms to 5000ms. Contributed by Andrew Wang.
2017-02-28 09:00:49 -08:00
258342e76c
HADOOP-14118. move jets3t into a dependency on hadoop-aws JAR.
2017-02-28 13:47:44 +09:00
55c07bbed2
HDFS-11382. Persist Erasure Coding Policy ID in a new optional field in INodeFile in FSImage. Contributed by Manoj Govindassamy.
2017-02-27 17:07:45 -08:00
05391c1845
HDFS-10506. OIV's ReverseXML processor cannot reconstruct some snapshot details. Contributed by Akira Ajisaka.
2017-02-25 14:38:50 -08:00
120bef7de8
HDFS-11462. Fix occasional BindException in TestNameNodeMetricsLogger.
2017-02-24 19:53:39 -08:00
4a58870a04
HDFS-11433. Document missing usages of OfflineEditsViewer processors. Contributed by Yiqun Lin.
2017-02-25 10:00:05 +08:00
e24ed47d9a
HDFS-11421. Make WebHDFS' ACLs RegEx configurable. Contributed by Harsh J.
2017-02-24 16:49:46 -08:00
d2b3ba9b8f
HDFS-11295. Check storage remaining instead of node remaining in BlockPlacementPolicyDefault.chooseReplicaToDelete(). Contributed by Marton Elek.
2017-02-24 15:44:11 -08:00
c1a52b04d0
HDFS-11427. Rename rs-default to rs.
2017-02-24 10:58:45 -08:00
132f758e3d
HDFS-11426. Refactor EC CLI to be similar to storage policies CLI.
2017-02-23 16:00:11 -08:00
13d4bcfe35
HDFS-4025. QJM: Sychronize past log segments to JNs that missed them. Contributed by Hanisha Koneru.
2017-02-22 16:33:38 -08:00
d150f061f4
HDFS-11438. Fix typo in error message of StoragePolicyAdmin tool. Contributed by Alison Yu.
2017-02-22 15:16:09 -08:00
cfcd527323
HDFS-11411. Avoid OutOfMemoryError in TestMaintenanceState test runs. (Manoj Govindassamy via mingma)
2017-02-22 09:41:07 -08:00
003ae00693
HDFS-11430. Separate class InnerNode from class NetworkTopology and make it extendable. Contributed by Tsz Wo Nicholas Sze
2017-02-21 15:32:46 -08:00
8ef7ebbc71
HDFS-11406. Remove unused getStartInstance and getFinalizeInstance in FSEditLogOp. Contributed by Alison Yu.
2017-02-21 14:54:20 -08:00
fc9ad3ce3a
HDFS-11405. Rename "erasurecode" CLI subcommand to "ec". Contributed by Manoj Govindassamy.
2017-02-21 13:55:27 -08:00
a07ddef101
HDFS-11404. Increase timeout on TestShortCircuitLocalRead.testDeprecatedGetBlockLocalPathInfoRpc. Contributed by Eric Badger
2017-02-21 12:04:25 -06:00
4804050630
HDFS-11177. 'storagepolicies -getStoragePolicy' command should accept URI based path. (Contributed by Surendra Singh Lilhore)
2017-02-21 18:13:19 +05:30
02c549484a
HDFS-11410. Use the cached instance when edit logging SetAclOp, SetXAttrOp and RemoveXAttrOp.
2017-02-16 18:07:55 -08:00
5690b51ef7
HDFS-11100. Recursively deleting file protected by sticky bit should fail. Contributed by John Zhuge.
2017-02-16 05:39:37 -08:00
a136936d01
HDFS-11265. Extend visualization for Maintenance Mode under Datanode tab in the NameNode UI. (Marton Elek via mingma)
2017-02-15 20:24:07 -08:00
0741dd3b9a
HDFS-11375. Display the volume storage type in datanode UI. Contributed by Surendra Singh Lilhore
2017-02-15 11:37:26 -08:00
627da6f717
HDFS-8498. Blocks can be committed with wrong size. Contributed by Jing Zhao.
2017-02-15 10:44:37 -08:00
859bd159ae
HDFS-11333. Print a user friendly error message when plugins are not found. Contributed by Wei-Chiu Chuang.
2017-02-15 02:50:35 -08:00
8acb376c9c
HDFS-11238. Fix checkstyle warnings in NameNode#createNameNode. Contributed by Ethan Li.
2017-02-15 16:53:50 +09:00
8e53f2b9b0
HDFS-11391. Numeric usernames do no work with WebHDFS FS write access. (Pierre Villard via Yongjun Zhang)
2017-02-14 13:40:53 -08:00
aaf2713235
HDFS-11409. DatanodeInfo getNetworkLocation and setNetworkLocation shoud use volatile instead of synchronized. Contributed by Chen Liang.
2017-02-14 12:52:34 -08:00
0cf5993712
HDFS-11084. Add a regression test for sticky bit support of OIV ReverseXML processor. Contributed by Wei-Chiu Chuang.
2017-02-14 09:11:55 -08:00
4164a2032a
HDFS-11408. The config name of balance bandwidth is out of date. Contributed by Yiqun Lin.
2017-02-14 18:57:20 +08:00
719df99c05
HDFS-11407. Document the missing usages of OfflineImageViewer processors. Contributed by Yiqun Lin.
2017-02-14 18:48:08 +08:00
4ed33e9ca3
HDFS-11026. Convert BlockTokenIdentifier to use Protobuf. Contributed by Ewan Higgs.
2017-02-13 11:29:18 -08:00
07a5184f74
HDFS-11379. DFSInputStream may infinite loop requesting block locations. Contributed by Daryn Sharp.
2017-02-10 12:27:08 -06:00
2007e0cf2a
HDFS-11210. Enhance key rolling to guarantee new KeyVersion is returned from generateEncryptedKeys after a key is rolled.
2017-02-07 20:36:11 -08:00
69b23632c4
HDFS-10860. Switch HttpFS from Tomcat to Jetty. Contributed by John Zhuge.
2017-02-07 09:36:55 -08:00
663e683adf
HDFS-10219. Change the default value for dfs.namenode.reconstruction.pending.timeout-sec from -1 to 300. Contributed by Yiqun Lin.
2017-02-06 19:19:30 +08:00
9cbbd1eae8
HDFS-11377. Balancer hung due to no available mover threads. Contributed by yunjiong zhao.
2017-02-06 13:15:16 +08:00
e0235842a7
HDFS-11387. Socket reuse address option is not honored in PrivilegedNfsGatewayStarter. Contributed by Mukul Kumar Singh.
2017-02-03 12:28:05 -08:00
0914fcca31
HDFS-11363. Need more diagnosis info when seeing Slow waitForAckedSeqno.
2017-02-02 10:08:27 -08:00
3433f572fa
HDFS-11353. Improve the unit tests relevant to DataNode volume failure testing. Contributed by Yiqun Lin.
2017-02-02 19:38:17 +08:00
6aa09dc28a
HDFS-11112. Journal Nodes should refuse to format non-empty directories. Contributed by Yiqun Lin.
2017-02-01 16:51:58 -08:00
b6f290d5b6
HDFS-11370. Optimize NamenodeFsck#getReplicaInfo. Contributed Takanobu Asanuma.
2017-02-01 11:21:35 -08:00
59c5f18784
HADOOP-13895. Make FileStatus Serializable
2017-02-01 10:19:36 -08:00
bec9b7aa1d
HDFS-11335. Remove HdfsClientConfigKeys.DFS_CLIENT_SLOW_IO_WARNING_THRESHOLD_KEY usage from DNConf. (Manoj Govindassamy via lei)
2017-02-01 14:42:51 +08:00
312b36d113
HDFS-11378. Verify multiple DataNodes can be decommissioned/maintenance at the same time. (Manoj Govindassamy via mingma)
2017-01-27 16:16:42 -08:00
7bc333ad41
HDFS-11374. Skip FSync in CreateEditsLog to speed up edit log generation. Contributed by Hanisha Koneru.
2017-01-26 16:21:25 -08:00
55c9f6d747
HDFS-9884. Use doxia macro to generate in-page TOC of HDFS site documentation. (iwasakims)
2017-01-27 08:06:39 +09:00
88da9f6b67
Revert "HDFS-8377. Support HTTP/2 in datanode. Contributed by Duo Zhang."
...
This reverts commit ada233b7cd
2017-01-26 13:42:50 -08:00
44606aa850
HDFS-11364. Add a test to verify Audit log entries for setfacl/getfacl commands over FS shell. Contributed by Manoj Govindassamy.
2017-01-26 10:48:26 -08:00
7c1cc30b3c
HDFS-11369. Change exception message in StorageLocationChecker.
2017-01-26 09:12:38 -08:00
ff02bdfe65
Add CHANGES, RELEASENOTES, and jdiff for 3.0.0-alpha2 release.
2017-01-25 15:39:40 -08:00
b782bf2156
HDFS-11124. Report blockIds of internal blocks for EC files in Fsck. Contributed by Takanobu Asanuma.
2017-01-25 11:16:17 -08:00
18e1d68209
HDFS-10534. NameNode WebUI should display DataNode usage histogram. Contributed by Kai Sasaki.
2017-01-25 09:58:39 -08:00
5a5652098c
HDFS-11365. Log portnumber in PrivilegedNfsGatewayStarter. Contributed by Mukul Kumar Singh.
2017-01-24 21:58:02 -08:00
b57368b6f8
HDFS-11194. Maintain aggregated peer performance metrics on NameNode.
2017-01-24 16:58:20 -08:00
355b907c09
HDFS-11121. Add assertions to BlockInfo#addStorage to protect from breaking reportedBlock-blockGroup mapping. Contributed by Takanobu Asanuma.
2017-01-21 05:56:22 +08:00
f3fb94be05
HDFS-11296. Maintenance state expiry should be an epoch time and not jvm monotonic. (Manoj Govindassamy via mingma)
2017-01-19 22:31:15 -08:00
5d8b80ea9b
Preparing for 3.0.0-alpha3 development
2017-01-19 15:50:07 -08:00
78ae2aed8f
HDFS-11259. Update fsck to display maintenance state info. (Manoj Govindassamy via lei)
2017-01-19 16:28:31 +08:00
a8f1c7f542
HDFS-110268. Correctly reconstruct erasure coding file from FSImage. Contributed by SammiChen.
2017-01-19 00:24:11 -08:00
63320d1daa
HDFS-11287. Storage class member storageDirs should be private to avoid unprotected access by derived classes. (Manoj Govindassamy via lei)
2017-01-19 15:38:20 +08:00
724bb86351
HDFS-11316. TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailure fails in trunk. Contributed by Yiqun Lin.
2017-01-19 12:05:13 +08:00
0eb4b513b7
HADOOP-13673. Update scripts to be smarter when running with privilege
...
Signed-off-by: Andrew Wang <wang@apache.org>
Signed-off-by: Ravi Prakash <raviprak@apache.org>
2017-01-18 14:39:05 -08:00
a2a5d7b5bc
HDFS-10759. Change fsimage bool isStriped from boolean to an enum. Contributed by Ewan Higgs.
2017-01-18 13:31:05 -08:00
8a0fa0f7e8
HDFS-10733. NameNode terminated after full GC thinking QJM is unresponsive. Contributed by Vinitha Gankidi.
2017-01-18 12:46:32 -08:00
9130af3f74
HADOOP-13955. Replace deprecated HttpServer2 and SSLFactory constants. Contributed by John Zhuge.
2017-01-18 18:11:33 +09:00
b1a9ec856b
HDFS-11290. TestFSNameSystemMBean should wait until JMX cache is cleared. Contributed by Erik Krogen.
2017-01-18 17:54:33 +09:00
b1fce2b8b1
HDFS-11209. SNN can't checkpoint when rolling upgrade is not finalized. Contributed by Xiaoyu Yao.
2017-01-17 14:33:26 -08:00
89bb05d92b
HDFS-11339. Fix import.
2017-01-16 23:05:28 -08:00
e407449ddb
HADOOP-13933. Add haadmin -getAllServiceState option to get the HA state of all the NameNodes/ResourceManagers. Contributed by Surendra Singh Lilhore.
2017-01-17 10:10:23 +09:00
d1d0b3e1fd
HDFS-11307. The rpc to portmap service for NFS has hardcoded timeout. Contributed by Mukul Kumar Singh.
2017-01-16 14:53:53 -08:00
79e939d0b8
HDFS-11339. Support File IO sampling for Datanode IO profiling hooks. Contributed by Hanisha Koneru.
2017-01-16 14:43:29 -08:00
a853b4e1b5
HDFS-11342. Fix FileInputStream leak in loadLastPartialChunkChecksum. Contributed by Chen Liang.
2017-01-16 13:45:22 -08:00
2f8e9b7e4b
Revert "HDFS-11259. Update fsck to display maintenance state info. (Manoj Govindassamy via lei)"
...
This reverts commit c18590fce2
2017-01-16 15:11:53 +08:00
2604e82eba
HDFS-11344. The default value of the setting dfs.disk.balancer.block.tolerance.percent is different. Contributed by Yiqun Lin.
2017-01-13 22:38:46 -08:00
1cde954a4f
HDFS-11306. Print remaining edit logs from buffer if edit log can't be rolled. Contributed by Wei-Chiu Chuang.
2017-01-13 11:46:30 -08:00
5f336512d0
HADOOP-13961. Fix compilation failure from missing hadoop-kms test jar. Contributed by Sangjin Lee and John Zhuge.
2017-01-11 15:48:50 -08:00
d51f8ba808
HDFS-11312. Fix incompatible tag number change for nonDfsUsed in DatanodeInfoProto. Contributed by Sean Mackrory.
2017-01-11 15:13:27 -08:00
a5ec1e31a3
HDFS-9935. Remove LEASE_{SOFTLIMIT,HARDLIMIT}_PERIOD and unused import from HdfsServerConstants. Contributed Yiqun Lin.
2017-01-11 19:34:01 +05:30
467f5f1735
HDFS-9391. Update webUI/JMX to display maintenance state info. (Manoj Govindassamy via mingma)
2017-01-10 20:12:42 -08:00
e69231658d
HDFS-11072. Add ability to unset and change directory EC policy. Contributed by Sammi Chen.
2017-01-10 11:32:48 -08:00
4046794a53
HDFS-11299. Support multiple Datanode File IO hooks. Contributed by Hanisha Koneru.
2017-01-10 10:43:11 -08:00
c18590fce2
HDFS-11259. Update fsck to display maintenance state info. (Manoj Govindassamy via lei)
2017-01-10 19:37:58 +08:00
7ec609b289
HDFS-11273. Move TransferFsImage#doGetUrl function to a Util class. Contributed by Hanisha Koneru.
2017-01-09 18:05:33 -08:00
603cbcd513
HDFS-11292. log lastWrittenTxId etc info in logSyncAll. Contributed by Yongjun Zhang.
2017-01-09 15:44:42 -08:00
91bf504440
HDFS-11301. Double wrapping over RandomAccessFile in LocalReplicaInPipeline#createStreams. Contributed by Hanisha Koneru.
2017-01-09 10:32:18 -08:00
4c8f9e1302
HDFS-9483. Documentation does not cover use of "swebhdfs" as URL scheme for SSL-secured WebHDFS. Contributed by Surendra Singh Lilhore.
2017-01-05 15:04:47 -08:00
a605ff36a5
HDFS-11280. Allow WebHDFS to reuse HTTP connections to NN. Contributed by Zheng Shao.
2017-01-04 21:01:23 -08:00
87bb1c49bb
HDFS-11279. Cleanup unused DataNode#checkDiskErrorAsync(). Contributed by Hanisha Koneru
2017-01-03 18:25:46 -08:00
7fcc73fc0d
HDFS-11156. Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang.
2017-01-03 09:58:00 -08:00
b31e1951e0
Revert "HDFS-11280. Allow WebHDFS to reuse HTTP connections to NN. Contributed by Zheng Shao."
...
This reverts commit b811a1c14d
2017-01-03 19:19:29 +05:30
b811a1c14d
HDFS-11280. Allow WebHDFS to reuse HTTP connections to NN. Contributed by Zheng Shao.
2016-12-30 22:17:49 -08:00
e769957556
HDFS-11275. Check groupEntryIndex and throw a helpful exception on failures when removing ACL.
2016-12-29 10:36:50 -08:00
a4f66655ec
HDFS-11267. Avoid redefinition of storageDirs in NNStorage and cleanup its accessors in Storage. (Manoj Govindassamy via lei)
2016-12-29 16:57:40 +08:00
e9f1396834
HDFS-11251. ConcurrentModificationException during DataNode#refreshVolumes. (Manoj Govindassamy via lei)
2016-12-29 15:11:25 +08:00
603f3ef138
HDFS-11274. Datanode should only check the failed volume upon IO errors. Contributed by Xiaoyu Yao.
2016-12-28 22:08:13 -08:00
ce3613c965
HDFS-11278. Add missing @Test annotation for TestSafeMode.testSafeModeUtils() (Contributed by Lukas Majercak via Daniel Templeton)
2016-12-28 20:09:00 -08:00
0ddb8defad
HDFS-11252. TestFileTruncate#testTruncateWithDataNodesRestartImmediately can fail with BindException. Contributed by Yiqun Lin.
2016-12-28 10:34:33 +05:30
e216e8e233
HADOOP-13932. Fix indefinite article in comments (Contributed by LiXin Ge via Daniel Templeton)
2016-12-27 13:03:16 -08:00
c0e0ef2969
HDFS-11270. Document the missing options of NameNode bootstrap command. Contributed by Yiqun Lin
2016-12-27 11:21:10 -08:00
ea547529cb
HDFS-11272. Refine the assert messages in TestFSDirAttrOp. Contributed by Jimmy Xiang.
2016-12-26 17:15:45 +09:00
8f218ea284
HDFS-11250. Fix a typo in ReplicaUnderRecovery#setRecoveryID. Contributed by Yiqun Lin.
2016-12-26 16:51:29 +09:00
483cd06ad4
HDFS-11271. Typo in NameNode UI. Contributed by Wei-Chiu Chuang.
2016-12-26 16:36:54 +09:00
4e9029653d
HDFS-10917. Collect peer performance statistics on DataNode. Contributed by Xiaobing Zhou.
2016-12-22 23:46:58 -08:00
56a13a6a59
HDFS-11216. Add remoteBytesRead counter metrics for erasure coding reconstruction task. Contributed by Sammi Chen
2016-12-22 14:18:54 +08:00
f6e80acd68
HDFS-11258. File mtime change could not save to editlog. Contributed by Jimmy Xiang.
2016-12-21 13:04:03 -08:00
f678080dbd
HDFS-11182. Update DataNode to use DatasetVolumeChecker. Contributed by Arpit Agarwal.
2016-12-20 13:53:32 -08:00
5daa8d8631
HDFS-10913. Introduce fault injectors to simulate slow mirrors. Contributed by Xiaobing Zhou.
2016-12-20 13:17:52 -08:00
4af66b1d66
HDFS-11247. Add a test to verify NameNodeMXBean#getDecomNodes() and Live/Dead Decom Nodes shown in NameNode WebUI. Contributed by Manoj Govindassamy.
2016-12-20 12:42:13 -08:00
5b7acdd206
HDFS-11195. Return error when appending files by webhdfs rest api fails. Contributed by Yuanbo Liu.
2016-12-20 12:24:00 -08:00
1b401f6a73
HDFS-11263. ClassCastException when we use Bzipcodec for Fsimage compression. Contributed by Brahma Reddy Battula.
2016-12-20 20:54:03 +05:30
fe4ff64a4a
HDFS-10959. Adding per disk IO statistics and metrics in DataNode. Contributed by Xiaoyu Yao.
2016-12-19 22:42:54 -08:00
d2026ae9e6
HDFS-11262. Remove unused variables in FSImage.java. Contributed by Jagadesh Kiran N.
2016-12-20 14:34:35 +09:00
a95639068c
HDFS-9911. TestDataNodeLifeline Fails intermittently. Contributed by Yiqun Lin
2016-12-16 09:46:21 -08:00
cee0c468b0
HDFS-11188. Change min supported DN and NN versions back to 2.x. Contributed by Andrew Wang.
2016-12-15 16:48:50 -08:00
aebb9127ba
HDFS-11160. VolumeScanner reports write-in-progress replicas as corrupt incorrectly. Contributed by Wei-Chiu Chuang and Yongjun Zhang.
2016-12-15 16:32:50 -08:00
6fce191549
HDFS-11253. FileInputStream leak on failure path in BlockSender. Contributed by Arpit Agarwal
2016-12-15 15:39:15 -08:00
ac0d402c6a
HDFS-11094. Send back HAState along with NamespaceInfo during a versionRequest as an optional parameter. Contributed by Eric Badger
2016-12-15 14:53:53 -08:00
36947f79ba
HDFS-11249. Redundant toString() in DFSConfigKeys.java. Contributed by Jagadesh Kiran N.
2016-12-16 00:33:27 +09:00
6ba9587d37
HDFS-10958. Add instrumentation hooks around Datanode disk IO.
2016-12-14 11:18:58 -08:00
72bff192cd
HDFS-11204. Document the missing options of hdfs zkfc command. Contributed by Yiqun Lin.
2016-12-14 18:50:43 +09:00
1f14f6d038
HDFS-8411. Add bytes count metrics to datanode for ECWorker. Contributed by Sammi Chen and Andrew Wang
2016-12-14 14:50:50 +08:00
fbdbbd57cd
HDFS-10684. WebHDFS DataNode calls fail without parameter createparent. Contributed by John Zhuge.
2016-12-13 18:01:31 -08:00
e24a923db5
HDFS-11164: Mover should avoid unnecessary retries if the block is pinned. Contributed by Rakesh R
2016-12-13 17:09:58 -08:00
754f15bae6
HDFS-11226. cacheadmin, cryptoadmin and storagepolicyadmin should support generic options. Contributed by Brahma Reddy Battula
2016-12-12 18:21:15 -08:00
2d4731c067
HDFS-11233. Fix javac warnings related to the deprecated APIs after upgrading Jackson. Contributed by Yiqun Lin.
2016-12-13 11:11:15 +09:00
2a28e8cf04
HDFS-11229. HDFS-11056 failed to close meta file. Contributed by Wei-Chiu Chuang.
2016-12-09 16:02:42 -08:00
5bd7dece92
HDFS-11217. Annotate NameNode and DataNode MXBean interfaces as Private/Stable. Contributed by Jagadesh Kiran N.
2016-12-09 12:05:42 -08:00
d1d4aba71b
HDFS-11224. Lifeline message should be ignored for dead nodes (Contributed by Vinayakumar B)
2016-12-09 14:53:50 +05:30
13d8e552d2
Revert "HDFS-5079. Cleaning up NNHAStatusHeartbeat.State from DatanodeProtocolProtos. Contributed by Tao Luo."
...
This reverts commit 4c8db60092
2016-12-08 15:41:49 -06:00
9f8344db50
Revert "HDFS-11094. Send back HAState along with NamespaceInfo during a versionRequest as an optional parameter. Contributed by Eric Badger"
...
This reverts commit 8c4680852b
2016-12-08 15:36:08 -06:00
401c731872
HDFS-11197. Listing encryption zones fails when deleting a EZ that is on a snapshotted directory. Contributed by Wellington Chevreuil.
2016-12-08 12:40:20 -08:00
4c2cf5560f
HDFS-11223. Fix typos in HttpFs documentations. Contributed by Yiqun Lin.
2016-12-08 20:52:24 +09:00
9ef89ede2f
HDFS-11140. Directory Scanner should log startup message time correctly. Contributed by Yiqun Lin.
2016-12-08 13:38:02 +09:00
deb4743b2c
Revert "HDFS-11140. Directory Scanner should log startup message time correctly. Contributed by Yiqun Lin."
...
This reverts commit 0857641f62
2016-12-08 13:37:06 +09:00
0857641f62
HDFS-11140. Directory Scanner should log startup message time correctly. Contributed by Yiqun Lin.
2016-12-08 13:34:45 +09:00
ea2895f4ed
HDFS-8630. WebHDFS : Support get/set/unset StoragePolicy. Contributed by Surendra Singh Lilhore.
2016-12-07 15:52:16 -08:00
c73e08a6da
HDFS-10206. Datanodes not sorted properly by distance when the reader isn't a datanode. (Nandakumar via mingma)
2016-12-07 08:26:09 -08:00
c8d0a049b0
HADOOP-13859. TestConfigurationFieldsBase fails for fields that are DEFAULT values of skipped properties. (Haibo Chen via kasha)
2016-12-06 14:10:58 -08:00
f53b481ffe
HDFS-11172. Support an erasure coding policy using RS 10 + 4. Contributed by Wei Zhou.
2016-12-06 12:49:38 -08:00
df983b524a
HDFS-10930. Refactor: Wrap Datanode IO related operations. Contributed by Xiaoyu Yao.
2016-12-06 11:05:47 -08:00
43cb1678cc
HDFS-11211. Add a time unit to the DataNode client trace format. Contributed by Jagadesh Kiran N
2016-12-06 09:12:11 -08:00
ed89856755
HDFS-11178. TestAddStripedBlockInFBR#testAddBlockInFullBlockReport fails frequently in trunk. Contributed By Yiqun Lin.
2016-12-06 22:30:43 +05:30
4dd4f3a4bb
HDFS-11198. NN UI should link DN web address using hostnames. Contributed by Weiwei Yang.
2016-12-06 08:06:50 -06:00
08a7253bc0
Revert "HDFS-11156. Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang"
...
This reverts commit c7ff34f8dc
2016-12-05 23:09:35 -08:00
8e63fa98ea
HDFS-10581. Hide redundant table on NameNode WebUI when no nodes are decomissioning. Contributed by Weiwei Yang.
2016-12-05 18:13:53 -08:00
dcedb72af4
Revert "HADOOP-10930. Refactor: Wrap Datanode IO related operations. Contributed by Xiaoyu Yao."
...
This reverts commit aeecfa24f4
2016-12-05 12:44:20 -08:00
8c4680852b
HDFS-11094. Send back HAState along with NamespaceInfo during a versionRequest as an optional parameter. Contributed by Eric Badger
2016-12-05 11:48:58 -08:00
1b5cceaffb
Revert "HDFS-11201. Spelling errors in the logging, help, assertions and exception messages. Contributed by Grant Sohn."
...
This reverts commit b9522e86a5
2016-12-05 10:54:43 -08:00
b9522e86a5
HDFS-11201. Spelling errors in the logging, help, assertions and exception messages. Contributed by Grant Sohn.
2016-12-05 10:48:25 -08:00
c51bfd29cd
HDFS-11181. Fuse wrapper has a typo. Contributed by Wei-Chiu Chuang.
2016-12-05 08:44:40 -08:00
c7ff34f8dc
HDFS-11156. Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang
2016-12-02 11:10:13 -08:00
96c574927a
HDFS-8674. Improve performance of postponed block scans. Contributed by Daryn Sharp.
2016-12-01 12:11:27 -06:00
e0fa49234f
HDFS-11180. Intermittent deadlock in NameNode when failover happens.
2016-12-01 23:08:59 +09:00
7226a71b1f
HDFS-5517. Lower the default maximum number of blocks per file. Contributed by Aaron T. Myers and Andrew Wang.
2016-11-30 15:58:31 -08:00
625df87c7b
HDFS-8678. Bring back the feature to view chunks of files in the HDFS file browser. Contributed by Ivo Udelsmann.
2016-11-30 09:12:15 -08:00
51e6c1cc3f
HDFS-10994. Support an XOR policy XOR-2-1-64k in HDFS. Contributed by Sammi Chen
2016-11-30 15:52:56 +08:00
cfd8076f81
Revert due to an error "HDFS-10994. Support an XOR policy XOR-2-1-64k in HDFS. Contributed by Sammi Chen"
...
This reverts commit 5614f847b2ef2a5b70bd9a06edc4eba06174c6.
2016-11-30 15:44:52 +08:00
aeecfa24f4
HADOOP-10930. Refactor: Wrap Datanode IO related operations. Contributed by Xiaoyu Yao.
2016-11-29 20:52:36 -08:00
eaaa32950c
HDFS-11149. Support for parallel checking of FsVolumes.
2016-11-29 20:31:02 -08:00
b407d53195
Revert "HDFS-5517. Lower the default maximum number of blocks per file. Contributed by Aaron T. Myers."
...
This reverts commit 09451252fa
2016-11-29 10:50:10 -08:00
00096dcc0c
HADOOP-13812. Upgrade Tomcat to 6.0.48. Contributed by John Zhuge.
2016-11-29 19:47:02 +09:00
09451252fa
HDFS-5517. Lower the default maximum number of blocks per file. Contributed by Aaron T. Myers.
2016-11-29 00:12:35 -08:00
47ca9e26fb
HADOOP-13838. KMSTokenRenewer should close providers (xiaochen via rkanter)
2016-11-28 18:08:09 -08:00
a2b1ff0257
HADOOP-13828. Implement getFileChecksum(path, length) for ViewFileSystem. Contributed by Manoj Govindassamy.
2016-11-28 11:54:43 -08:00
5d5614f847
HDFS-10994. Support an XOR policy XOR-2-1-64k in HDFS. Contributed by Sammi Chen
2016-11-28 14:34:44 +08:00
209e805430
HADOOP-13506. Redundant groupid warning in child projects. Contributed by Kai Sasaki.
2016-11-28 14:34:57 +09:00
bb1b5e083b
HDFS-11174. Wrong HttpFS test command in doc. Contributed by John Zhuge.
2016-11-25 21:28:29 +09:00
636f7547b7
HDFS-11175. Document uppercase key names are not supported in TransparentEncryption.md. Contributed by Yiqun Lin.
2016-11-25 17:48:54 +09:00
de4894936a
HDFS-11368. Erasure Coding: Deprecate replication-related config keys. Contributed by Rakesh R.
2016-11-23 16:42:06 -08:00
dd98a8005a
HDFS-11058. Implement 'hadoop fs -df' command for ViewFileSystem. Contributed by Manoj Govindassamy.
2016-11-23 16:40:39 -08:00
3a09e5970d
HDFS-11145. Implement getTrashRoot() for ViewFileSystem. (Manoj Govindassamy via lei)
2016-11-22 13:52:09 -08:00
613b902b98
HDFS-11148. Update DataNode to use StorageLocationChecker at startup.
2016-11-22 10:50:25 -08:00
afcf8d38e7
HADOOP-13646. Remove outdated overview.html. Contributed By Brahma Reddy Battula.
2016-11-22 19:45:45 +05:30
49a09179e3
HDFS-10966. Enhance Dispatcher logic on deciding when to give up a source DataNode. Contributed by Mark Wagner and Zhe Zhang.
2016-11-21 10:13:24 -06:00
f922067748
HDFS-11101. TestDFSShell#testMoveWithTargetPortEmpty fails intermittently. Contributed By Brahma Reddy Battula.
2016-11-21 19:34:52 +05:30
c68dad18ab
HDFS-11144. TestFileCreationDelete#testFileCreationDeleteParent fails wind bind exception. Contributed By Brahma Reddy Battula.
2016-11-21 10:58:34 +05:30
d232625f73
HDFS-11113. Document dfs.client.read.striped configuration in hdfs-default.xml. Contributed by Rakesh R.
2016-11-21 14:05:15 +09:00
f6ffa11635
HDFS-11117. Refactor striped file tests to allow flexibly test erasure coding policy. Contributed by Sammi Chen
2016-11-18 16:08:59 +06:00
c0b1a44f6c
HADOOP-13660. Upgrade commons-configuration version. Contributed by Sean Mackrory.
2016-11-17 22:48:35 -06:00
b2d4b7b1b9
HDFS-11134. Fix bind exception threw in TestRenameWhileOpen. Contributed By Yiqun Lin.
2016-11-17 10:22:30 +05:30
04a024b683
HDFS-11147. Remove confusing log output in FsDatasetImpl#getInitialVolumeFailureInfos. Contributed by Chen Liang.
2016-11-16 14:47:34 -08:00
c90891e7b3
HDFS-11105. TestRBWBlockInvalidation#testRWRInvalidation fails intermittently. Contributed by Yiqun Lin
2016-11-16 13:02:10 -08:00
4fcea8a0c8
HDFS-8870. Lease is leaked on write failure. Contributed by Kuhu Shukla.
2016-11-15 14:47:53 -06:00
5af572b644
HADOOP-13427. Eliminate needless uses of FileSystem#{exists(), isFile(), isDirectory()}. Contributed by Steve Loughran and Mingliang Liu
2016-11-15 10:57:00 -08:00
ff0b99eafe
HDFS-10872. Add MutableRate metrics for FSNamesystemLock operations. Contributed by Erik Krogen.
2016-11-14 11:05:29 -08:00
6efb8c9c60
HDFS-11135. The tests in TestBalancer run fails due to NPE. Contributed By Yiqun Lin
2016-11-14 20:39:39 +05:30
96f43921ba
HADOOP-13813. TestDelegationTokenFetcher#testDelegationTokenWithoutRenewer is failing. Contributed by Mingliang Liu
2016-11-12 10:35:22 -08:00
2ee18fc15e
HDFS-11129. TestAppendSnapshotTruncate fails with bind exception. Contributed by Brahma Reddy Battula
2016-11-11 22:12:27 -08:00
3d26717777
HDFS-11119. Support for parallel checking of StorageLocations on DataNode startup.
...
This closes #155 .
2016-11-11 15:02:52 -08:00
4484b48498
HDFS-10941. Improve BlockManager#processMisReplicatesAsync log. Contributed by Chen Liang.
2016-11-11 14:31:23 -08:00
aa6010ccca
HDFS-11122. TestDFSAdmin#testReportCommand fails due to timed out. Contributed by Yiqun Lin
2016-11-11 12:29:47 -08:00
1ae57f0f75
HDFS-11128. CreateEditsLog throws NullPointerException. Contributed by Hanisha Koneru.
2016-11-11 20:08:15 +05:30
8848a8a76c
HDFS-11116. Fix javac warnings caused by deprecation of APIs in TestViewFsDefaultValue. Contributed by Yiqun Lin.
2016-11-11 13:45:30 +09:00
ca68f9cb5b
HDFS-9337. Validate required params for WebHDFS requests (Contributed by Jagadesh Kiran N)
2016-11-10 16:51:33 +05:30
71adf44c3f
HDFS-11120. TestEncryptionZones should waitActive. Contributed by John Zhuge.
2016-11-09 17:16:07 -08:00
c619e9b43f
HDFS-11056. Concurrent append and read operations lead to checksum error. Contributed by Wei-Chiu Chuang.
2016-11-09 09:16:50 -08:00
ed0bebabaa
HDFS-9482. Replace DatanodeInfo constructors with a builder pattern. Contributed by Brahma Reddy Battula.
2016-11-08 18:17:07 -08:00
62d8c17dfd
HDFS-11083. Add unit test for DFSAdmin -report command. Contributed by Xiaobing Zhou
2016-11-08 17:04:30 -08:00
e1c6ef2efa
HADOOP-13789. Hadoop Common includes generated test protos in both jar and test-jar. Contributed by Sean Busbey.
2016-11-08 16:33:55 -08:00
3fff158587
HDFS-11114. Support for running async disk checks in DataNode.
...
This closes #153 .
2016-11-07 18:45:53 -08:00
049e7d27be
HDFS-10970. Update jackson from 1.9.13 to 2.x in hadoop-hdfs.
2016-11-07 11:16:31 +09:00
d8bab3dcb6
HDFS-10756. Expose getTrashRoot to HTTPFS and WebHDFS. Contributed by Yuanbo Liu.
2016-11-04 18:06:55 -07:00
0c0ab102ab
HDFS-11085. Add unit test for NameNode failing to start when name dir is unwritable. Contributed by Xiaobing Zhou
2016-11-04 14:34:49 -07:00
69dd5fa2d4
HDFS-11098. Datanode in tests cannot start in Windows after HDFS-10638 (Contributed by Vinayakumar B)
2016-11-04 08:31:43 +05:30
5cad93d5c7
HADOOP-11088. Quash unnecessary safemode WARN message during NameNode startup. Contributed by Yiqun Lin.
2016-11-03 15:31:07 -07:00
7534aee09a
HDFS-11076. Add unit test for extended Acls. Contributed by Chen Liang
2016-11-03 13:49:10 -07:00
b71907b2ae
HDFS-11097. Fix warnings for deprecated StorageReceivedDeletedBlocks constructor. Contributed by Yiqun Lin.
2016-11-03 10:27:52 -07:00
7e521c5a49
HDFS-11080. Update HttpFS to use ConfigRedactor. Contributed by Sean Mackrory.
2016-11-02 19:11:05 -07:00
730cb0cff6
HDFS-10566. Submit plan request should throw exception if Datanode is in non-REGULAR status. Contributed by Xiaobing Zhou.
2016-11-02 17:46:56 -07:00
b59206190e
HDFS-10909. De-duplicate code in ErasureCodingWorker#initializeStripedReadThreadPool and DFSClient#initThreadsNumForStripedReads. (Manoj Govindassamy via lei)
2016-11-02 16:45:25 -07:00
9b0c17f852
HDFS-11095. BlockManagerSafeMode should respect extension period default config value (30s). Contributed by Mingliang Liu
2016-11-02 14:58:24 -07:00
f2800f523d
Revert "HADOOP-13717. Normalize daemonization behavior of the diskbalancer with balancer and mover."
...
This reverts commit bd7f5911c7
2016-11-02 13:46:33 -07:00
cb5cc0dc53
HDFS-11031. Add additional unit test for DataNode startup behavior when volumes fail. Contributed by Mingliang Liu.
2016-11-02 10:35:47 +05:30
bd7f5911c7
HADOOP-13717. Normalize daemonization behavior of the diskbalancer with balancer and mover.
2016-11-01 16:48:45 -07:00
fa1512a740
HDFS-11065. Add space quota tests for heterogenous storages. Contributed by Xiaobing Zhou
2016-11-01 14:42:03 -07:00
34173a42d4
HDFS-10633. DiskBalancer : Add the description for the new setting dfs.disk.balancer.plan.threshold.percent in HDFSDiskbalancer.md. Contributed by Yiqun Lin.
2016-11-01 11:14:03 -07:00
808f5d8985
HDFS-10499. TestNameNodeMetadataConsistency#testGenerationStampInFuture Fails Intermittently. Contributed by Yiqun Lin.
2016-11-01 21:32:52 +05:30
0c49f73a6c
HDFS-11030. TestDataNodeVolumeFailure#testVolumeFailure is flaky (though passing). Contributed by Mingliang Liu
2016-10-31 16:00:33 -07:00
a9d68d2e8e
HDFS-11074. Remove unused method FsDatasetSpi#getFinalizedBlocksOnPersistentStorage. Contributed by Hanisha Koneru.
2016-10-31 12:39:30 -07:00
e9c7a97089
HDFS-10455. Logging the username when deny the setOwner operation. Contributed by Rakesh R.
2016-10-31 00:22:51 +05:30
8a9388e5f6
HDFS-11048. Audit Log should escape control characters. Contributed by Eric Badger
2016-10-28 10:53:14 -07:00
8fa2195697
HDFS-10926. Update staled configuration properties related to erasure coding. Contributed by Sammi Chen
2016-10-28 12:37:30 +06:00
57187fdb93
HDFS-11064. Mention the default NN rpc ports in hdfs-default.xml. Contributed by Yiqun Lin.
2016-10-27 18:13:06 -07:00
5877f20f9c
HADOOP-10075. Update jetty dependency to version 9 (rkanter)
2016-10-27 16:09:00 -07:00
9e03ee5279
HDFS-11047. Remove deep copies of FinalizedReplica to alleviate heap consumption on DataNode. Contributed by Xiaobing Zhou
2016-10-27 16:00:27 -07:00
f3ac1f41b8
HDFS-9500. Fix software version counts for DataNodes during rolling upgrade. Contributed by Erik Krogen.
2016-10-27 15:58:25 -07:00
31ff42b510
HDFS-11055. Update default-log4j.properties for httpfs to imporve test logging. Contributed by Wei-Chiu Chuang.
2016-10-27 13:37:00 -07:00
ae48c496dc
HDFS-11069. Tighten the authorization of datanode RPC. Contributed by Kihwal Lee
2016-10-27 14:17:16 -05:00
db41965996
Revert "HDFS-10455. Logging the username when deny the setOwner operation. Contributed by Tianyin Xiu"
...
This reverts commit ac35ee9393
2016-10-27 14:07:28 -05:00
ac35ee9393
HDFS-10455. Logging the username when deny the setOwner operation. Contributed by Tianyin Xiu
2016-10-27 20:20:56 +05:30
1cf6ec4ad4
HDFS-8492. DN should notify NN when client requests a missing block (Contributed by Walter Su)
2016-10-27 16:44:00 +05:30
77f2684dd3
HDFS-10769. BlockIdManager.clear doesn't reset the counter for blockGroupIdGenerator (Contributed by Yiqun Lin and Rakesh R)
2016-10-27 14:16:03 +05:30
9c680d0b05
HDFS-11049. The description of dfs.block.replicator.classname is not clear. Contributed by Yiqun Lin.
2016-10-27 16:30:57 +09:00
a895c66255
HDFS-9929. Duplicate keys in NAMENODE_SPECIFIC_KEYS (Contributed by Akira Ajisaka)
2016-10-27 12:20:15 +05:30
0bdd263d82
HADOOP-13017. Implementations of InputStream.read(buffer, offset, bytes) to exit 0 if bytes==0. Contributed by Steve Loughran.
2016-10-27 15:46:59 +09:00
9f32364d28
HDFS-11038. DiskBalancer: support running multiple commands in single test. Contributed by Xiaobing Zhou.
2016-10-26 19:39:02 -07:00
55e1fb8e32
HDFS-10921. TestDiskspaceQuotaUpdate doesn't wait for NN to get out of safe mode. Contributed by Eric Badger
2016-10-26 14:11:38 -07:00
f209e93566
HDFS-10638. Modifications to remove the assumption that StorageLocation is associated with java.io.File in Datanode. (Virajith Jalaparti via lei)
2016-10-26 10:32:35 -07:00
e90af4a89b
HDFS-11053. Unnecessary superuser check in versionRequest(). Contributed by Kihwal Lee.
2016-10-26 09:16:13 -05:00
ce6bbfb23c
HDFS-11050. Change log level to 'warn' when ssl initialization fails and defaults to DEFAULT_TIMEOUT_CONN_CONFIGURATOR. Contributed by Kuhu Shukla.
2016-10-26 08:07:53 -05:00
287effff93
HDFS-10935. TestFileChecksum fails in some cases. Contributed by Sammi Chen
2016-10-26 09:40:31 +06:00
9dce9011ad
HDFS-11054. Suppress verbose log message in BlockPlacementPolicyDefault. Contributed by Chen Liang.
2016-10-25 19:50:56 -07:00
9a8a386458
HDFS-11011. Add unit tests for HDFS command 'dfsadmin -set/clrSpaceQuota'. Contributed by Xiaobing Zhou.
2016-10-25 11:24:03 -07:00
f6367c5f44
HDFS-11015. Enforce timeout in balancer. Contributed by Kihwal Lee.
2016-10-25 10:19:13 -07:00
dc3272bfe8
HDFS-11046. Duplicate '-' in the daemon log name.
2016-10-25 10:07:43 +09:00
9d175853b0
HDFS-10997. Reduce number of path resolving methods. Contributed by Daryn Sharp.
2016-10-24 17:14:51 -05:00
b18f35f097
HDFS-11042. Add missing cleanupSSLConfig() call for tests that use setupSSLConfig(). Contributed by Kuhu Shukla.
2016-10-24 08:28:44 -05:00
25438526ed
HDFS-11033. Add documents for native raw erasure coder in XOR codes. Contributed by SammiChen.
2016-10-21 16:20:07 -07:00
df857f0d10
HDFS-10975. fsck -list-corruptfileblocks does not report corrupt EC files. Contributed by Takanobu Asanuma.
2016-10-21 15:43:12 -07:00
61e30cf83c
HDFS-8410. Add computation time metrics to datanode for ECWorker. Contributed by SammiChen.
2016-10-21 13:12:41 -07:00
ae8bccd509
HADOOP-13702. Add instrumented ReadWriteLock. Contributed by Jingcheng Du
2016-10-21 12:59:54 -07:00
f63cd78f60
HDFS-10730. Fix some failed tests due to BindException. Contributed by Yiqun Lin
2016-10-21 18:16:39 +05:30
d7d87deece
HDFS-10998. Add unit tests for HDFS command 'dfsadmin -fetchImage' in HA. Contributed by Xiaobing Zhou
2016-10-20 19:51:48 -07:00
5e83a21cb6
HDFS-10976. Report erasure coding policy of EC files in Fsck. Contributed by Wei-Chiu Chuang.
2016-10-20 13:06:43 -07:00
6d2da38d16
HDFS-11018. Incorrect check and message in FsDatasetImpl#invalidate. Contributed by Yiqun Lin.
2016-10-20 10:56:39 -07:00
f872c6bc03
HADOOP-13236. truncate will fail when we use viewfilesystem. Contributed by Brahma Reddy Battula
2016-10-20 21:47:17 +05:30
6fb6b651e8
HDFS-10699. Log object instance get incorrectly in TestDFSAdmin. Contributed by Yiqun Lin
2016-10-20 21:28:16 +05:30
5c0bffddc0
HDFS-10627. Volume Scanner marks a block as "suspect" even if the exception is network-related. Contributed by Rushabh S Shah.
2016-10-20 10:48:35 -05:00
4c73be135c
HDFS-9480. Expose nonDfsUsed via StorageTypeStats. Contributed by Brahma Reddy Battula
2016-10-20 20:45:45 +05:30
73504b1bdc
HDFS-11025. TestDiskspaceQuotaUpdate fails in trunk due to Bind exception. Contributed by Yiqun Lin
2016-10-20 10:41:18 +05:30
b4564103e4
HDFS-10752. Several log refactoring/improvement suggestion in HDFS. Contributed by Hanisha Koneru.
2016-10-19 17:20:07 -07:00
c5573e6a75
HDFS-11009. Add a tool to reconstruct block meta file from CLI.
2016-10-18 22:42:28 -07:00
efdf810cf9
HADOOP-7352. FileSystem#listStatus should throw IOE upon access error. Contributed by John Zhuge.
2016-10-18 18:18:43 -07:00
c62ae7107f
HDFS-10906. Add unit tests for Trash with HDFS encryption zones. Contributed by Hanisha Koneru.
2016-10-18 14:05:43 -07:00
d26a1bb9d6
HDFS-10920. TestStorageMover#testNoSpaceDisk is failing intermittently. Contributed by Rakesh R
2016-10-18 14:51:08 +06:00
c023c74886
HADOOP-13061. Refactor erasure coders. Contributed by Kai Sasaki
2016-10-18 12:02:53 +06:00
bedfec0c10
HADOOP-13522. Add %A and %a formats for fs -stat command to print permissions. Contributed by Alex Garbarini.
2016-10-18 15:00:44 +09:00
b61fb267b9
HDFS-9390. Block management for maintenance states.
2016-10-17 17:45:41 -07:00
8fd4c37c45
HDFS-10922. Adding additional unit tests for Trash (II). Contributed by Weiwei Yang.
2016-10-17 14:21:36 -07:00
5f4ae85bd8
HDFS-11003. Expose XmitsInProgress through DataNodeMXBean. Contributed By Brahma Reddy Battula
2016-10-15 22:28:33 +05:30
5ad037df25
HDFS-11012. Unnecessary INFO logging on DFSClients for InvalidToken. Contributed by Harsh J.
...
This closes #142
2016-10-15 22:14:24 +09:00
391ce535a7
HDFS-10301. Remove FBR tracking state to fix false zombie storage detection for interleaving block reports. Contributed by Vinitha Gankidi.
2016-10-14 18:13:54 -07:00
30bb1970cc
HDFS-10558. DiskBalancer: Print the full path to plan file. Contributed by Xiaobing Zhou.
2016-10-14 17:07:59 -07:00
76cc84e6d4
HDFS-11008. Change unit test for testing parsing "-source" parameter in Balancer CLI. Contributed by Mingliang Liu
2016-10-14 14:29:02 -07:00
8c520a27cb
HDFS-10960. TestDataNodeHotSwapVolumes#testRemoveVolumeBeingWritten fails at disk error verification after volume remove. (Manoj Govindassamy via lei)
2016-10-14 13:41:59 -07:00
adb96e109f
HDFS-10827. When there are unrecoverable ec block groups, Namenode Web UI doesn't show the block names. Contributed by Takanobu Asanuma.
2016-10-14 13:21:53 -07:00
0007360c33
HDFS-10883. 's behavior is not consistent in DFS after enabling EZ. Contributed by Yuanbo Liu.
2016-10-14 11:41:29 -07:00
701c27a776
HDFS-10735 Distcp using webhdfs on secure HA clusters fails with StandbyException
2016-10-14 10:26:39 -07:00
8a9f6635a3
HDFS-10990. TestPendingInvalidateBlock should wait for IBRs. Contributed by Yiqun Lin.
2016-10-14 11:38:48 -05:00
8c721aa00a
Revert "HDFS-10990. TestPendingInvalidateBlock should wait for IBRs. Contributed by Yiqun Lin."
...
This reverts commit fdce515091
2016-10-13 13:23:28 -07:00
332a61fd74
HDFS-10987. Make Decommission less expensive when lot of blocks present. Contributed by Brahma Reddy Battula.
2016-10-13 13:52:49 -05:00
fdce515091
HDFS-10990. TestPendingInvalidateBlock should wait for IBRs. Contributed by Yiqun Lin.
2016-10-13 11:41:37 -07:00
b371c56365
HDFS-10949. DiskBalancer: deprecate TestDiskBalancer#setVolumeCapacity. Contributed by Xiaobing Zhou.
2016-10-13 10:26:07 -07:00
1291254042
HDFS-10986. DFSAdmin should log detailed error message if any. Contributed by MingLiang Liu
2016-10-13 22:05:00 +05:30
9454dc5e80
HDFS-11000. webhdfs PUT does not work if requests are routed to call queue. Contributed by Kihwal Lee.
2016-10-13 08:47:15 -05:00
901eca004d
HDFS-11002. Fix broken attr/getfattr/setfattr links in ExtendedAttributes.md. Contributed by Mingliang Liu.
2016-10-13 14:29:30 +09:00
85cd06f663
HDFS-10789. Route webhdfs through the RPC call queue. Contributed by Daryn Sharp and Rushabh S Shah.
2016-10-12 15:11:42 -05:00
7ba7092bbc
HDFS-10965. Add unit test for HDFS command 'dfsadmin -printTopology'. Contributed by Xiaobing Zhou
2016-10-11 17:23:54 -07:00
3c9a01062e
HDFS-10903. Replace config key literal strings with config key names II: hadoop hdfs. Contributed by Chen Liang
2016-10-11 16:29:30 -07:00
61f0490a73
HDFS-10984. Expose nntop output as metrics. Contributed by Siddharth Wagle.
2016-10-11 15:55:02 -07:00
dacd3ec66b
HDFS-10991. Export hdfsTruncateFile symbol in libhdfs. Contributed by Surendra Singh Lilhore.
2016-10-11 15:07:14 -07:00
809cfd27a3
HDFS-10916. Switch from "raw" to "system" xattr namespace for erasure coding policy. (Andrew Wang via lei)
2016-10-11 10:04:46 -07:00
96b12662ea
HDFS-10637. Modifications to remove the assumption that FsVolumes are backed by java.io.File. (Virajith Jalaparti via lei)
2016-10-10 15:30:03 -07:00
b963818621
HDFS-10988. Refactor TestBalancerBandwidth. Contributed by Brahma Reddy Battula
2016-10-10 13:19:17 -07:00
3441c746b5
HDFS-10972. Add unit test for HDFS command 'dfsadmin -getDatanodeInfo'. Contributed by Xiaobing Zhou
2016-10-10 11:33:37 -07:00
af50da3298
HDFS-10895. Update HDFS Erasure Coding doc to add how to use ISA-L based coder. Contributed by Sammi Chen
2016-10-10 11:55:49 +06:00
4d106213c0
HDFS-10968. BlockManager#isInNewRack should consider decommissioning nodes. Contributed by Jing Zhao.
2016-10-07 22:44:54 -07:00
6a38d118d8
HDFS-10797. Disk usage summary of snapshots causes renamed blocks to get counted twice. Contributed by Sean Mackrory.
2016-10-07 17:37:15 -07:00
e57fa81d95
HDFS-10980. Optimize check for existence of parent directory. Contributed by Daryn Sharp.
2016-10-07 17:20:15 -05:00
3565c9af17
HDFS-10979. Pass IIP for FSDirDeleteOp methods. Contributed by Daryn Sharp.
2016-10-07 14:15:59 -05:00
3059b251d8
HDFS-10933. Refactor TestFsck. Contributed by Takanobu Asanuma.
2016-10-07 10:17:50 -07:00
be3cb10f53
HDFS-10969. Fix typos in hdfs-default.xml Contributed by Yiqun Lin
2016-10-07 22:18:40 +05:30
48b9d5fd2a
HDFS-10955. Pass IIP for FSDirAttr methods. Contributed by Daryn Sharp.
2016-10-06 16:33:46 -05:00
1d330fbaf6
HADOOP-13150. Avoid use of toString() in output of HDFS ACL shell commands. Contributed by Chris Nauroth.
2016-10-06 13:19:16 -07:00
f32e9fc8f7
HDFS-10939. Reduce performance penalty of encryption zones. Contributed by Daryn sharp.
2016-10-06 15:11:14 -05:00
72a2ae6452
HADOOP-13688. Stop bundling HTML source code in javadoc JARs.
2016-10-06 11:19:38 -07:00
311954883f
HDFS-10957. Retire BKJM from trunk (Vinayakumar B)
2016-10-06 19:28:25 +05:30
b90fc70d67
HDFS-10963. Reduce log level when network topology cannot find enough datanodes. Contributed by Xiao chen
2016-10-06 17:47:31 +05:30
202325485d
HDFS-10893. Refactor TestDFSShell by setting up MiniDFSCluser once for all commands test. Contributed by Mingliang Liu
2016-10-05 22:33:29 -07:00
c5ca216915
HDFS-10683. Make class Token$PrivateToken private. Contributed by John Zhuge.
2016-10-05 17:36:50 -07:00
e68c7b96c7
HDFS-10962. TestRequestHedgingProxyProvider is flaky.
2016-10-05 15:08:56 -07:00
8867762256
HDFS-10826. Correctly report missing EC blocks in FSCK. Contributed by Takanobu Asanuma.
2016-10-05 10:52:50 -07:00
44f48ee96e
HDFS-10956. Remove rename/delete performance penalty when not using snapshots. Contributed by Daryn Sharp.
2016-10-04 15:05:09 -05:00
382307cbdd
HDFS-10878. TestDFSClientRetries#testIdempotentAllocateBlockAndClose throws ConcurrentModificationException. Contributed by Rushabh S Shah.
2016-10-04 11:09:03 -05:00
ef7f06f7d1
HDFS-10947. Correct the API name for truncate in webhdfs document. Contributed by Surendra Singh Lilhore
2016-10-04 18:46:42 +05:30
5ea0210c7f
HDFS-10944. Correct the javadoc of dfsadmin#disallowSnapshot. Contributed by Jagadesh Kiran N
2016-10-04 18:33:22 +05:30
f61e3d13e9
HDFS-10934. TestDFSShell#testStat fails intermittently. Contributed by Eric Badger
2016-10-03 22:34:41 -04:00
8078a5efd0
HDFS-10810. setreplication removing block from underconstrcution temporarily. Contributed by Brahma Reddy Battula
2016-10-03 21:50:16 -04:00
853d65a157
HDFS-10918. Add a tool to get FileEncryptionInfo from CLI. Contributed by Xiao Chen.
2016-10-03 16:01:54 -07:00
607705c488
HDFS-10690. Optimize insertion/removal of replica in ShortCircuitCache. Contributed by Fenghua Hu.
2016-10-03 10:53:21 -07:00
744208431f
HDFS-10940. Reduce performance penalty of block caching when not used. Contributed by Daryn Sharp.
2016-10-03 11:27:23 -05:00
90020624b0
HDFS-10619. Cache path in InodesInPath. Contributed by Daryn Sharp.
2016-10-03 09:13:04 -05:00
fe9ebe20ab
Revert "HDFS-10923. Make InstrumentedLock require ReentrantLock."
...
This reverts commit c7ce6fdc20
2016-09-30 23:11:51 -07:00
c7ce6fdc20
HDFS-10923. Make InstrumentedLock require ReentrantLock.
2016-09-30 23:00:06 -07:00
434c5ea75d
HDFS-10896. Move lock logging logic from FSNamesystem into FSNamesystemLock. Contributed by Erik Krogen.
2016-09-30 13:16:08 -07:00
7fad1221d6
HDFS-10907. Fix Erasure Coding documentation. Contributed by Manoj Govindassamy.
2016-09-30 12:51:27 -07:00
ee33a02234
HDFS-10910. HDFS Erasure Coding doc should state its currently supported erasure coding policies. Contributed by Yiqun Lin.
2016-09-30 12:48:11 -07:00
2ab1ef15c5
HDFS-10908. Improve StripedBlockReader#createBlockReader error logging. Contributed by Manoj Govindassamy.
2016-09-30 12:35:47 -07:00
a0730aa5ce
HDFS-10851. FSDirStatAndListingOp: stop passing path as string. Contributed by Daryn Sharp.
2016-09-30 13:03:24 -05:00
0670149c88
HDFS-10850. getEZForPath should NOT throw FNF. Contributed by Andrew Wang.
2016-09-30 08:44:18 -05:00
84c6264079
HDFS-10892. Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'. Contributed by Mingliang Liu
2016-09-28 16:03:51 -07:00
92e5e91598
HDFS-10914. Move remnants of oah.hdfs.client to hadoop-hdfs-client.
2016-09-28 16:01:03 -07:00
5f34402ada
HDFS-10779. Rename does not need to re-solve destination. Contributed by Daryn Sharp.
2016-09-28 17:57:23 -05:00
c3b235e565
HDFS-10824. MiniDFSCluster#storageCapacities has no effects on real capacity. Contributed by Xiaobing Zhou.
2016-09-28 11:47:37 -07:00
03f519a757
HDFS-9850. DiskBalancer: Explore removing references to FsVolumeSpi. Contributed by Manoj Govindassamy.
2016-09-27 21:35:06 -07:00
6437ba18c5
HDFS-10915. Fix time measurement bug in TestDatanodeRestart. Contributed by Xiaobing Zhou
2016-09-27 20:40:17 -07:00
9c9736463b
HDFS-10900. DiskBalancer: Complete the documents for the report command. Contributed by Yiqun Lin.
2016-09-27 19:28:41 -07:00
2acfb1e1e4
HDFS-10376. Enhance setOwner testing. (John Zhuge via Yongjun Zhang)
2016-09-27 14:55:28 -07:00
875062b5bc
HADOOP-13544. JDiff reports unncessarily show unannotated APIs and cause confusion while our javadocs only show annotated and public APIs. (vinodkv via wangda)
2016-09-27 11:26:45 -07:00
8ae4729107
HDFS-10828. Fix usage of FsDatasetImpl object lock in ReplicaMap. (Arpit Agarwal)
2016-09-27 10:02:15 -07:00
f55eb981dd
HDFS-10426. TestPendingInvalidateBlock failed in trunk. Contributed by Yiqun Lin.
2016-09-28 01:29:24 +09:00
df1d0f5d56
HDFS-10889. Remove outdated Fault Injection Framework documentaion. Contributed by Brahma Reddy Battula
2016-09-27 19:33:10 +05:30
edf0d0f8b2
HDFS-9444. Add utility to find set of available ephemeral ports to ServerSocketUtil. Contributed by Masatake Iwasaki
2016-09-27 07:04:37 +05:30
059058f961
HDFS-10713. Throttle FsNameSystem lock warnings. Contributed by Hanisha Koneru.
2016-09-26 17:09:47 -07:00
8e06d865c4
Addendum patch for HDFS-10609. Uncaught InvalidEncryptionKeyException during pipeline recovery may abort downstream applications.
2016-09-26 14:41:04 -07:00
3ae652f821
HDFS-10609. Uncaught InvalidEncryptionKeyException during pipeline recovery may abort downstream applications. Contributed by Wei-Chiu Chuang.
2016-09-26 13:11:32 -07:00
3e37e243ee
HDFS-10869. Remove the unused method InodeId#checkId(). Contributed by Jagadesh Kiran N
2016-09-25 11:02:46 +05:30
6eb700ee74
HDFS-10866. Fix Eclipse Java compile errors related to generic parameters. Contributed by Konstantin Shvachko.
2016-09-23 14:10:11 -07:00
74b3dd514c
HDFS-10876. Dispatcher#dispatch should log IOException stacktrace. Contributed by Manoj Govindassamy.
2016-09-23 13:26:57 -07:00
a5bb88c8e0
HDFS-10843. Update space quota when a UC block is completed rather than committed. Contributed by Erik Krogen.
2016-09-23 10:37:46 -07:00
bbdf350ff9
HDFS-10886. Replace fs.default.name with fs.defaultFS in viewfs document.
2016-09-23 19:21:34 +05:30
c4480f7f42
HDFS-10894. Remove the redundant charactors for command -saveNamespace in HDFSCommands.md Contributed by Yiqun Lin.
2016-09-23 19:11:02 +05:30
d85d9b2e7b
HDFS-10871. DiskBalancerWorkItem should not import jackson relocated by htrace. Contributed by Manoj Govindassamy.
2016-09-22 19:36:16 -07:00
40acacee08
HDFS-10877. Make RemoteEditLogManifest.committedTxnId optional in Protocol Buffers. Contributed by Sean Mackrory.
2016-09-22 11:43:11 -07:00
964e546ab1
HDFS-9333. Some tests using MiniDFSCluster errored complaining port in use. (iwasakims)
2016-09-21 10:35:25 +09:00
0e918dff59
HDFS-10879. TestEncryptionZonesWithKMS#testReadWrite fails intermittently. Contributed by Xiao Chen.
2016-09-20 16:56:52 -07:00
734d54c1a8
HDFS-10861. Refactor StripeReaders and use ECChunk version decode API. Contributed by Sammi Chen
2016-09-21 21:34:48 +08:00
e52d6e7a46
HDFS-10875. Optimize du -x to cache intermediate result. Contributed by Xiao Chen.
2016-09-19 21:44:42 -07:00
c54f6ef30f
HDFS-10868. Remove stray references to DFS_HDFS_BLOCKS_METADATA_ENABLED.
2016-09-19 11:17:03 -07:00
ea839bd48e
HDFS-10489. Deprecate dfs.encryption.key.provider.uri for HDFS encryption zones. Contributed by Xiao Chen.
2016-09-17 22:25:39 -07:00
8a40953058
HDFS-10823. Implement HttpFSFileSystem#listStatusIterator.
2016-09-16 15:37:36 -07:00
b09a03cd7d
HDFS-10862. Typos in 4 log messages. Contributed by Mehran Hassani.
2016-09-16 16:08:47 +09:00
ec3ea18875
HDFS-9895. Remove unnecessary conf cache from DataNode. Contributed by Xiaobing Zhou.
2016-09-15 15:25:47 -07:00
2a8f55a0cf
HDFS-10805. Reduce runtime for append test. Contributed by Gergely Novak.
2016-09-14 09:31:17 -07:00
ea0c2b8b05
HADOOP-13218. Migrate other Hadoop side tests to prepare for removing WritableRPCEngine. Contributed by Wei Zhou and Kai Zheng
2016-09-15 10:46:00 +08:00
86c9862bec
HDFS-10636. Modify ReplicaInfo to remove the assumption that replica metadata and data are stored in java.io.File. (Virajith Jalaparti via lei)
2016-09-13 12:54:14 -07:00
1c0d18f322
HDFS-10562. DiskBalancer: update documentation on how to report issues and debug. Contributed by Anu Engineer.
2016-09-13 12:42:10 -07:00
db6d243cf8
HDFS-10837. Standardize serializiation of WebHDFS DirectoryListing.
2016-09-13 11:02:36 -07:00
e3f7f58a5f
HDFS-10599. DiskBalancer: Execute CLI via Shell. Contributed by Manoj Govindassamy.
2016-09-13 09:38:12 -07:00
f0876b8b60
HDFS-10856. Update the comment of BPServiceActor$Scheduler#scheduleNextBlockReport. Contributed by Yiqun Lin.
2016-09-13 14:50:03 +09:00
d2466ec3e0
HDFS-10657. testAclCLI.xml setfacl test should expect mask r-x. (John Zhuge via Yongjun Zhang)
2016-09-12 21:25:06 -07:00
72dfb048a9
HDFS-10858. FBR processing may generate incorrect reportedBlock-blockGroup mapping. Contributed by Jing Zhao.
2016-09-12 16:40:11 -07:00
8a93f45a80
HDFS-10821. DiskBalancer: Report command support with multiple nodes. Contributed by Yiqun Lin.
2016-09-12 15:45:26 -07:00
58ed4fa544
HADOOP-13341. Deprecate HADOOP_SERVERNAME_OPTS; replace with (command)_(subcommand)_OPTS
...
This commit includes the following changes:
HADOOP-13356. Add a function to handle command_subcommand_OPTS
HADOOP-13355. Handle HADOOP_CLIENT_OPTS in a function
HADOOP-13554. Add an equivalent of hadoop_subcmd_opts for secure opts
HADOOP-13562. Change hadoop_subcommand_opts to use only uppercase
HADOOP-13358. Modify HDFS to use hadoop_subcommand_opts
HADOOP-13357. Modify common to use hadoop_subcommand_opts
HADOOP-13359. Modify YARN to use hadoop_subcommand_opts
HADOOP-13361. Modify hadoop_verify_user to be consistent with hadoop_subcommand_opts (ie more granularity)
HADOOP-13564. modify mapred to use hadoop_subcommand_opts
HADOOP-13563. hadoop_subcommand_opts should print name not actual content during debug
HADOOP-13360. Documentation for HADOOP_subcommand_OPTS
This closes apache/hadoop#126
2016-09-12 11:10:00 -07:00
cc01ed7026
HDFS-10855. Fix typos in HDFS documents. Contributed by Yiqun Lin.
2016-09-10 23:24:49 -07:00
a99bf26a08
HDFS-10830. FsDatasetImpl#removeVolumes crashes with IllegalMonitorStateException when vol being removed is in use. (Arpit Agarwal and Manoj Govindassamy)
2016-09-10 18:22:25 -07:00
bee9f57f5c
HDFS-10808. DiskBalancer does not execute multi-steps plan-redux. Contributed by Anu Engineer.
2016-09-09 15:00:39 -07:00
cba973f036
HDFS-10832. Propagate ACL bit and isEncrypted bit in HttpFS FileStatus permissions.
2016-09-09 12:12:29 -07:00
a0b0383677
Addendum patch for HDFS-9781. FsDatasetImpl#getBlockReports can occasionally throw NullPointerException. Contributed by Manoj Govindassamy.
2016-09-09 10:54:14 -07:00
baab48922a
HDFS-9849. DiskBalancer: reduce lock path in shutdown code. Contributed by Yuanbo Liu.
2016-09-08 20:00:42 -07:00
35c5943b8b
HDFS-10553. DiskBalancer: Rename Tools/DiskBalancer class to Tools/DiskBalancerCLI. Contributed by Manoj Govindassamy.
2016-09-08 19:26:56 -07:00
b07c266dca
HDFS-10831. Add log when URLConnectionFactory.openConnection failed. Contributed by yunjiong zhao.
2016-09-08 18:30:18 -07:00
011f3b24d4
HDFS-10742. Measure lock time in FsDatasetImpl. Contributed by Chen Liang
2016-09-08 17:53:40 -07:00
b6d839a60c
HDFS-10845. Change defaults in hdfs-site.xml to match timeunit type. Contributed by Yiqun Lin
2016-09-08 13:29:30 -07:00
401db4fc65
HDFS-8901. Use ByteBuffer in striping positional read. Contributed by Sammi Chen and Kai Zheng.
2016-09-08 11:54:33 -07:00
20a20c2f6e
HDFS-10847. Complete the document for FileDistribution processor in OfflineImageViewer. Contributed by Yiqun Lin.
2016-09-09 01:34:34 +09:00
2d1bf53c7e
HDFS-10844. test_libhdfs_threaded_hdfs_static and test_libhdfs_zerocopy_hdfs_static are failing.
2016-09-09 00:49:22 +09:00
63f594892e
HDFS-10778. Add -format option to make the output of FileDistribution processor human-readable in OfflineImageViewer.
2016-09-08 15:13:43 +09:00
d355573f56
Revert "HADOOP-13218. Migrate other Hadoop side tests to prepare for removing WritableRPCEngine. Contributed by Wei Zhou and Kai Zheng"
...
This reverts commit 62a9667136
2016-09-08 05:50:17 +08:00
5f23abfa30
HDFS-9038. DFS reserved space is erroneously counted towards non-DFS used. (Brahma Reddy Battula)
2016-09-06 13:37:21 -07:00
f6c0b7543f
HDFS-10841. Remove duplicate or unused variable in appendFile(). Contributed by Kihwal Lee.
2016-09-06 11:48:35 -07:00
7c213c341d
HDFS-10835. Fix typos in httpfs.sh. Contributed by John Zhuge.
2016-09-06 11:44:26 -07:00
f0d5382ff3
HDFS-6962. ACL inheritance conflicts with umaskmode. Contributed by Chris Nauroth.
2016-09-06 11:02:39 -07:00
d37dc5d1b8
HDFS-9847. HDFS configuration should accept time units. Contributed by Yiqun Lin
2016-09-06 10:38:04 -07:00
62a9667136
HADOOP-13218. Migrate other Hadoop side tests to prepare for removing WritableRPCEngine. Contributed by Wei Zhou and Kai Zheng
2016-09-07 17:05:33 +08:00
07650bc37a
HDFS-9781. FsDatasetImpl#getBlockReports can occasionally throw NullPointerException. Contributed by Manoj Govindassamy.
2016-09-02 15:33:11 -07:00
cbd909ce2a
HDFS-10833. Fix JSON errors in WebHDFS.md examples.
2016-09-02 14:50:34 -07:00
5a8c5064d1
HDFS-10822. Log DataNodes in the write pipeline. John Zhuge via Lei Xu
2016-09-02 11:01:13 -07:00
0690f0969e
HDFS-10820. Reuse closeResponder to reset the response variable in DataStreamer#run. Contributed by Yiqun Lin.
2016-09-01 15:35:25 -07:00
6c600360ca
Addendum fix for HDFS-10817 to fix failure of the added testFSReadLockLongHoldingReport in branch-2.
2016-08-31 23:43:59 -07:00
6f4b0d33ca
HDFS-10817. Add Logging for Long-held NN Read Locks. Contributed by Erik Krogen.
2016-08-31 15:40:09 -07:00
85bab5fb57
HDFS-10784. Implement WebHdfsFileSystem#listStatusIterator.
2016-08-31 14:29:37 -07:00
01721dd88e
HDFS-10729. Improve log message for edit loading failures caused by FS limit checks. Contributed by Wei-Chiu Chuang.
2016-08-31 14:02:37 -05:00
20ae1fa259
HDFS-10813. DiskBalancer: Add the getNodeList method in Command. Contributed by Yiqun Lin.
2016-08-30 18:42:55 -07:00
9dcbdbdb5a
HDFS-9392. Admins support for maintenance state. Contributed by Ming Ma.
2016-08-30 14:00:13 -07:00
c4ee6915a1
HDFS-10760. DataXceiver#run() should not log InvalidToken exception as an error. Contributed by Pan Yuxuan.
2016-08-30 10:43:20 -07:00
4bd45f54ee
HDFS-10814. Add assertion for getNumEncryptionZones when no EZ is created. Contributed by Vinitha Reddy Gankidi.
2016-08-29 23:37:26 -07:00
cd5e10ccca
HDFS-4210. Throw helpful exception when DNS entry for JournalNode cannot be resolved. Contributed by Charles Lamb and John Zhuge.
2016-08-29 17:48:08 -07:00
5d1609ddf2
HDFS-10625. VolumeScanner to report why a block is found bad. Contributed by Rushabh S Shah and Yiqun Lin.
2016-08-29 13:59:54 -07:00
6742fb6e68
HDFS-10807. Doc about upgrading to a version of HDFS with snapshots may be confusing. (Contributed by Mingliang Liu)
2016-08-29 10:15:34 -07:00
c25817159a
HDFS-10652. Add a unit test for HDFS-4660. Contributed by Vinayakumar B., Wei-Chiu Chuang, Yongjun Zhang.
2016-08-27 22:51:31 -07:00
e806db7190
HDFS-10584. Allow long-running Mover tool to login with keytab. Contributed by Rakesh R.
2016-08-26 16:43:45 -07:00
a445b82baa
HDFS-10793. Fix HdfsAuditLogger binary incompatibility introduced by HDFS-9184. Contributed by Manoj Govindassamy.
2016-08-26 15:39:18 -07:00
407b519fb1
HDFS-10798. Make the threshold of reporting FSNamesystem lock contention configurable. Contributed by Erik Krogen.
2016-08-26 14:19:55 -07:00
8b7adf4ddf
HDFS-10768. Optimize mkdir ops. Contributed by Daryn Sharp.
2016-08-26 15:39:56 -05:00
13fb1b50e6
HDFS-8915. TestFSNamesystem.testFSLockGetWaiterCount fails intermittently. Contributed by Masatake Iwasaki.
2016-08-26 09:53:10 -05:00
f4a21d3aba
HDFS-10795. Fix an error in ReaderStrategy#ByteBufferStrategy. Contributed by Sammi Chen
2016-08-27 10:54:25 +08:00
4da5000dd3
HDFS-10748. TestFileTruncate#testTruncateWithDataNodesRestart runs sometimes timeout. Contributed by Yiqun Lin.
2016-08-25 09:00:44 -07:00
a1f3293762
HDFS-10772. Reduce byte/string conversions for get listing. Contributed by Daryn Sharp.
2016-08-24 15:21:08 -05:00
793447f799
HDFS-8905. Refactor DFSInputStream#ReaderStrategy. Contributed by Kai Zheng and Sammi Chen
2016-08-24 21:57:23 +08:00
ec252ce0fc
HDFS-10762. Pass IIP for file status related methods
2016-08-24 08:46:47 -05:00
126d165efd
HDFS-9745. TestSecureNNWithQJM#testSecureMode sometimes fails with timeouts. Contributed by Xiao Chen
2016-08-23 14:46:58 +00:00
f0efea490e
HDFS-8986. Add option to -du to calculate directory space usage excluding snapshots. Contributed by Xiao Chen.
2016-08-23 04:14:24 -07:00
e90f3359de
HDFS-10783. The option '-maxSize' and '-step' fail in OfflineImageViewer. Contributed by Yiqun Lin.
2016-08-23 19:57:23 +09:00
c49333becf
HDFS-8312. Added permission check for moving file to Trash. (Weiwei Yang via Eric Yang)
2016-08-22 18:29:56 -07:00
3ca4d6ddfd
Revert "HDFS-10762. Pass IIP for file status related methods. Contributed by Daryn Sharp."
...
This reverts commit 22fc46d765
2016-08-22 16:57:45 -05:00
22fc46d765
HDFS-10762. Pass IIP for file status related methods. Contributed by Daryn Sharp.
2016-08-22 15:37:02 -05:00
dc7a1c54f4
HDFS-10692. Update JDiff report's base version for HDFS from 2.6.0 to 2.7.2. Contributed by Wangda Tan.
2016-08-22 11:35:18 -07:00
0faee62a0c
HDFS-10764. Fix INodeFile#getBlocks to not return null. Contributed by Arpit Agarwal.
2016-08-19 22:13:36 -07:00
03a9343d57
HADOOP-12765. HttpServer2 should switch to using the non-blocking SslSelectChannelConnector to prevent performance degradation when handling SSL connections. Contributed by Min Shen.
2016-08-19 09:22:49 -07:00
2550371f66
HDFS-10711. Optimize FSPermissionChecker group membership check. Contributed by Daryn Sharp.
2016-08-19 09:12:17 -05:00
8179f9a493
HDFS-10645. Make block report size as a metric and add this metric to datanode web ui. Contributed by Yuanbo Liu.
2016-08-19 16:15:00 +09:00
ca13e7971d
HDFS-10773. BlockSender should not synchronize on the dataset object. (Contributed by Chen Liang)
2016-08-17 16:29:08 -07:00
2aa5e2c403
HDFS-10549. Correctly revoke file leases when closing files. Contributed by Yiqun Lin.
2016-08-17 15:52:38 -07:00
869393643d
HDFS-10745. Directly resolve paths into INodesInPath. Contributed by Daryn Sharp.
2016-08-17 15:53:03 -05:00
b047bc7270
HDFS-10560. DiskBalancer: Reuse ObjectMapper instance to improve the performance. Contributed by Yiqun Lin.
2016-08-16 10:20:08 -07:00
4d4d95fdd5
HDFS-10641. TestBlockManager#testBlockReportQueueing fails intermittently. (Contributed by Daryn Sharp)
2016-08-15 20:28:40 -07:00
ef55fe1716
HDFS-10747. o.a.h.hdfs.tools.DebugAdmin usage message is misleading. (Contributed by Mingliang Liu)
2016-08-15 20:23:47 -07:00
382d615260
HDFS-10678. Documenting NNThroughputBenchmark tool. (Contributed by Mingliang Liu)
2016-08-15 20:22:14 -07:00
5628b36c08
HDFS-10559. DiskBalancer: Use SHA1 for Plan ID. Contributed by Xiaobing Zhou.
2016-08-15 20:10:21 -07:00
02abd131b8
HDFS-10567. Improve plan command help message. Contributed by Xiaobing Zhou.
2016-08-15 19:58:57 -07:00
864f878d59
HDFS-10763. Open files can leak permanently due to inconsistent lease update. Contributed by Kihwal Lee.
2016-08-15 17:28:09 -05:00
03dea65e0b
HDFS-10744. Internally optimize path component resolution. Contributed by Daryn Sharp.
2016-08-15 16:45:44 -05:00
bed69d18e6
HDFS-10580. DiskBalancer: Make use of unused methods in GreedyPlanner to print debug info. Contributed by Yiqun Lin
2016-08-15 12:40:29 -07:00
83e57e083f
HDFS-9696. Garbage snapshot records linger forever. Contributed by Kihwal Lee
2016-08-15 13:01:23 -05:00
9f29f423e4
HDFS-10737. disk balancer add volume path to report command. Contributed by Yuanbo Liu.
2016-08-15 09:47:30 -07:00
9019606b69
HDFS-10731. FSDirectory#verifyMaxDirItems does not log path name. Contributed by Wei-Chiu Chuang.
2016-08-12 05:58:13 -07:00
b5af9be72c
HDFS-8668. Erasure Coding: revisit buffer used for encoding and decoding. Contributed by Sammi Chen
2016-08-13 13:52:37 +08:00
a428d4f50e
HDFS-8897. Balancer should handle fs.defaultFS trailing slash in HA. Contributed by John Zhuge.
2016-08-10 22:17:35 -07:00
e7e8aed208
HDFS-10720. Fix intermittent test failure of TestDataNodeErasureCodingMetrics. Contributed by Rakesh R
2016-08-12 11:15:05 +08:00
ec289bbece
HDFS-10643. Namenode should use loginUser(hdfs) to generateEncryptedKey. Contributed by Xiaoyu Yao.
2016-08-10 15:49:36 -07:00
fdafd0cb4f
Revert "Namenode should use loginUser(hdfs) to generateEncryptedKey. Contributed by Xiaoyu Yao."
...
This reverts commit 9f816d7ca7
2016-08-10 15:47:55 -07:00
9f816d7ca7
Namenode should use loginUser(hdfs) to generateEncryptedKey. Contributed by Xiaoyu Yao.
2016-08-10 15:14:38 -07:00
10e84c6a6e
HDFS-10694. processReport() should print blockReportId in each log message. Contributed by Yuanbo Liu.
2016-08-10 11:07:42 -07:00
d00d3add9e
HDFS-8224. Schedule a block for scanning if its metadata file is corrupt. Contributed by Rushabh S Shah.
2016-08-09 18:11:34 -07:00
9c6a4383ca
HDFS-10681. DiskBalancer: query command should report Plan file path apart from PlanID. (Manoj Govindassamy via lei)
2016-08-09 15:59:22 -07:00
cc48251bfd
HDFS-10457. DataNode should not auto-format block pool directory if VERSION is missing. (Wei-Chiu Chuang via lei)
2016-08-09 15:51:19 -07:00
0f701f433d
HDFS-10738. Fix TestRefreshUserMappings.testRefreshSuperUserGroupsConfiguration test failure. Contributed by Rakesh R.
2016-08-09 14:54:11 -05:00
b10c936020
HDFS-10342. BlockManager#createLocatedBlocks should not check corrupt replicas if none are corrupt. Contributed by Kuhu Shukla.
2016-08-09 11:56:21 -05:00
8c0638471f
HADOOP-10682. Replace FsDatasetImpl object lock with a separate lock object. (Chen Liang)
2016-08-08 12:02:53 -07:00
4d3af47f27
HDFS-10693. metaSave should print blocks, not LightWeightHashSet. Contributed by Yuanbo Liu.
2016-08-07 14:29:26 -07:00
ef432579a7
HDFS-10715. NPE when applying AvailableSpaceBlockPlacementPolicy. Contributed by Guangbin Zhu.
2016-08-05 17:31:11 +09:00
18d9e6ec0b
HDFS-10588. False alarm in datanode log - ERROR - Disk Balancer is not enabled. Contributed by Weiwei Yang.
2016-08-04 19:40:33 -07:00
438a9f047e
HDFS-10673. Optimize FSPermissionChecker's internal path usage. Contributed by Daryn Sharp.
2016-08-04 16:14:13 -05:00
57369a678c
HDFS-10343. BlockManager#createLocatedBlocks may return blocks on failed storages. Contributed by Kuhu Shukla.
2016-08-04 14:55:21 -05:00
4a8e6dc02f
HDFS-10722. Fix race condition in TestEditLog#testBatchedSyncWithClosedLogs. Contributed by Daryn Sharp.
2016-08-04 11:53:44 -05:00
8c9b44eaff
In Balancer, the target task should be removed when its size < 0. Contributed by Yiqun Lin
2016-08-04 09:45:40 -07:00
6ae39199da
HDFS-10662. Optimize UTF8 string/byte conversions. Contributed by Daryn Sharp.
2016-08-04 09:07:34 -05:00
6f63566694
HDFS-10569. A bug causes OutOfIndex error in BlockListAsLongs. Contributed by Weiwei Yang.
2016-08-03 15:20:20 -05:00
f4ba5ff1d7
HDFS-10710. In BlockManager#rescanPostponedMisreplicatedBlocks(), postponed misreplicated block counts should be retrieved within the NN lock protection.Contributed by GAO Rui.
2016-08-03 12:42:05 -07:00
22ef5286bc
HDFS-10674. Optimize creating a full path from an inode. Contributed by Daryn Sharp.
2016-08-03 13:12:43 -05:00
58db263e93
HDFS-742. A down DataNode makes Balancer to hang on repeatingly asking NameNode its partial block list. Contributed by Mit Desai.
2016-08-03 12:14:59 -05:00
bebf10d245
HDFS-10656. Optimize conversion of byte arrays back to path string. Contributed by Daryn Sharp.
2016-08-03 11:53:41 -05:00
5e5b8793fb
HDFS-5805. TestCheckpoint.testCheckpoint fails intermittently on branch-2. Contributed by Eric Badger.
2016-08-02 15:37:36 -05:00
a5fb298e56
HDFS-10707. Replace org.apache.commons.io.Charsets with java.nio.charset.StandardCharsets. Contributed by Vincent Poon.
2016-08-02 17:07:59 +09:00
c4463f2ef2
Revert "HDFS-10301. Interleaving processing of storages from repeated block reports causes false zombie storage detection, removes valid blocks. Contributed by Vinitha Gankidi."
...
This reverts commit 85a20508bd
2016-08-01 22:34:40 -07:00
9f473cf903
HDFS-10655. Fix path related byte array conversion bugs. (daryn)
2016-08-01 10:14:28 -05:00
8d32bd884d
HDFS-10703. HA NameNode Web UI should show last checkpoint time. (John Zhuge via Yongjun Zhang)
2016-07-29 23:20:14 -07:00
204a2055b1
HDFS-10691. FileDistribution fails in hdfs oiv command due to ArrayIndexOutOfBoundsException. Contributed by Yiqun Lin.
2016-07-29 15:39:48 +09:00
d9aae22fdf
HDFS-9276. Failed to Update HDFS Delegation Token for long running application in HA mode. Contributed by Liangliang Gu and John Zhuge
2016-07-28 16:29:40 -07:00
ce3d68e9c3
HDFS-10676. Add namenode metric to measure time spent in generating EDEKs. Contributed by Hanisha Koneru.
2016-07-28 16:07:45 -07:00
a3d0cba810
HDFS-10689. Hdfs dfs chmod should reset sticky bit permission when the bit is omitted in the octal mode. (Manoj Govindassamy via Lei Xu)
2016-07-28 13:52:14 -07:00
67406460f0
HDFS-4176. EditLogTailer should call rollEdits with a timeout. (Lei Xu)
2016-07-28 13:32:00 -07:00
328c855a57
HDFS-10650. DFSClient#mkdirs and DFSClient#primitiveMkdir should use default directory permission. Contributed by John Zhuge.
2016-07-28 13:15:02 -07:00
414fbfab41
HDFS-10696. TestHDFSCLI fails. Contributed by Kai Sasaki.
2016-07-28 16:58:23 +09:00
098ec2b11f
HDFS-10519. Add a configuration option to enable in-progress edit log tailing. Contributed by Jiayi Zhou.
2016-07-27 17:55:41 -07:00
eb7ff0c992
HDFS-10667. Report more accurate info about data corruption location. (Yuanbo Liu via Yongjun Zhang)
2016-07-27 13:58:04 -07:00
54fe17a607
HADOOP-9427. Use JUnit assumptions to skip platform-specific tests. Contributed by Gergely Novák.
2016-07-27 19:41:09 +09:00
55d5993a8e
HDFS-10668. Fix intermittently failing UT TestDataNodeMXBean#testDataNodeMXBeanBlockCount. Contributed by Mingliang Liu.
2016-07-27 10:26:42 +05:30
d84ab8a578
HDFS-10598. DiskBalancer does not execute multi-steps plan. Contributed by Lei (Eddy) Xu.
2016-07-26 16:01:50 -07:00
d2cf8b54c5
HDFS-10642. TestLazyPersistReplicaRecovery#testDnRestartWithSavedReplicas fails intermittently. (Contributed by Mingliang Liu)
2016-07-26 12:27:46 -07:00
25f0a9b3ab
HDFS-9937. Update dfsadmin command line help and HdfsQuotaAdminGuide. Contributed by Kai Sasaki.
2016-07-26 08:01:42 -07:00
2aa18e1925
Revert "HDFS-10668. Fix intermittently failing UT TestDataNodeMXBean#testDataNodeMXBeanBlockCount. Contributed by Mingliang Liu."
...
This reverts commit 7cac7655fd
2016-07-26 14:30:02 +01:00
7cac7655fd
HDFS-10668. Fix intermittently failing UT TestDataNodeMXBean#testDataNodeMXBeanBlockCount. Contributed by Mingliang Liu.
2016-07-26 10:54:24 +05:30
85a20508bd
HDFS-10301. Interleaving processing of storages from repeated block reports causes false zombie storage detection, removes valid blocks. Contributed by Vinitha Gankidi.
2016-07-25 18:50:59 -07:00
59466b8c18
HDFS-10671. Fix typo in HdfsRollingUpgrade.md. Contributed by Yiqun Lin.
2016-07-26 10:48:21 +09:00
0cde9e12a7
HDFS-10688. BPServiceActor may run into a tight loop for sending block report when hitting IOException. Contributed by Chen Liang.
2016-07-25 18:41:13 -07:00
b79ba4fe96
HDFS-10651. Clean up some configuration related codes about legacy block reader. Contributed by Youwei Wang
2016-07-23 17:45:33 +08:00
c6e3a0020f
HDFS-10660. Expose storage policy apis via HDFSAdmin interface. (Contributed by Rakesh R)
2016-07-22 12:33:50 -07:00
77ac04efe4
HDFS-10565: Erasure Coding: Document about the current allowed storage policies for EC Striped mode files
2016-07-22 11:08:20 -07:00
132deb4cac
HDFS-10225. DataNode hot swap drives should disallow storage type changes. Contributed by Lei (Eddy) Xu.
2016-07-21 16:41:02 -07:00
fcde6940e0
HDFS-10287. MiniDFSCluster should implement AutoCloseable. Contributed by Andras Bokor.
2016-07-21 11:26:08 -07:00
bd3dcf46e2
HDFS-10653. Optimize conversion from path string to components. Contributed by Daryn Sharp.
2016-07-21 11:14:39 -07:00
38128baff4
HDFS-10425. Clean up NNStorage and TestSaveNamespace. Contributed by Andras Bokor.
2016-07-20 10:44:11 -07:00
f7dabe3add
HDFS-10620. StringBuilder created and appended even if logging is disabled. Contributed by Staffan Friberg.
2016-07-19 15:13:01 -07:00
fc570b55b9
HDFS-10603. Fix flaky tests in org.apache.hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot. Contributed by Yiqun Lin.
2016-07-19 13:49:24 -07:00
5b4a708704
HDFS-10639. Fix typos in HDFSDiskbalancer.md. Contributed by Yiqun Lin.
2016-07-15 21:41:57 -07:00
da456ffd62
Preparing for 3.0.0-alpha2 development
2016-07-15 19:04:17 -07:00
c2f9cd584c
HADOOP-13368. DFSOpsCountStatistics$OpType#fromSymbol and s3a.Statistic#fromSymbol should be O(1) operation. Contributed by Mingliang Liu.
2016-07-15 14:28:53 -07:00
c48e9d608c
HDFS-10628. Log HDFS Balancer exit message to its own log. Contributed by Jiayi Zhou.
2016-07-15 11:42:44 -07:00
5537c6b234
HADOOP-13351. TestDFSClientSocketSize buffer size tests are flaky. Contributed by Aaron Fabbri and Mingliang Liu.
2016-07-14 14:40:58 -07:00
382dff7475
HDFS-10600. PlanCommand#getThrsholdPercentage should not use throughput value. (Yiqun Lin via lei)
2016-07-14 10:40:34 -07:00
1fa1fab695
HDFS-10570. Remove classpath conflicts of netty-all jar in hadoop-hdfs-client. Contributed by Vinayakumar B.
2016-07-14 10:20:18 +01:00
2bbc3ea1b5
HDFS-10617. PendingReconstructionBlocks.size() should be synchronized. Contributed by Eric Badger.
2016-07-13 21:59:35 -05:00
438b7c5935
HDFS-10590: Fix TestReconstructStripedBlocks.testCountLiveReplicas test failures. Contributed by Rakesh R
2016-07-13 00:07:11 -07:00
087290e6b1
HDFS-10544. Balancer doesn't work with IPFailoverProxyProvider.
2016-07-12 23:18:37 -07:00
d5f5a6475b
HDFS-10387. DataTransferProtocol#writeBlock missing some javadocs (John Zhuge via lei)
2016-07-12 16:16:03 -07:00
c447efebdb
HDFS-10579. HDFS web interfaces lack configs for X-FRAME-OPTIONS protection. Contributed by Anu Engineer.
2016-07-11 14:55:33 -07:00
da6f1b88dd
HDFS-10512. VolumeScanner may terminate due to NPE in DataNode.reportBadBlocks. Contributed by Wei-Chiu Chuang and Yiqun Lin.
2016-07-08 19:40:44 -07:00
932aed64d7
HDFS-10336. TestBalancer failing intermittently because of not reseting UserGroupInformation completely. Contributed by Yiqun Lin.
2016-07-08 14:39:11 -07:00
6e597600f7
HADOOP-13283. Support reset operation for new global storage statistics and per FS storage stats. Contributed by Mingliang Liu.
2016-07-07 14:58:19 -07:00
ab092c56c2
HDFS-10592:Fix intermittent test failure of TestNameNodeResourceChecker#testCheckThatNameNodeResourceMonitorIsRunning. Contributed by Rakesh R
2016-07-07 13:52:35 -07:00
b8f93cd275
HDFS-10488. Update WebHDFS documentation regarding CREATE and MKDIR default permissions. Contributed by Wellington Chevreuil.
2016-07-07 10:36:55 -07:00
a3f93beaa1
HDFS-10582. Change deprecated configuration fs.checkpoint.dir to dfs.namenode.checkpoint.dir in HDFS Commands Doc. Contributed by Pan Yuxuan.
2016-07-06 17:51:42 -07:00
8b4b5259d5
HDFS-10589. Javadoc for HAState#HAState and HAState#setStateInternal contains non-existent params. Contributed by shenyinjie.
2016-07-04 07:15:04 -07:00
2e835d986e
HDFS-10572. Fix TestOfflineEditsViewer#testGenerated. Contributed by Surendra Singh Lilhore.
2016-07-03 21:11:56 -07:00
8b281bce85
HDFS-10548. Remove the long deprecated BlockReaderRemote. Contributed by Kai Zheng
2016-07-03 11:56:23 +08:00
aa42c7a6dd
HADOOP-13305. Define common statistics names across schemes. Contributed by Mingliang Liu.
2016-07-01 15:34:03 -07:00
e4a2545620
HDFS-9805. Add server-side configuration for enabling TCP_NODELAY for DataTransferProtocol and default it to true (Gary Helmling via cmccabe)
2016-06-29 12:41:42 -07:00
991c946593
HADOOP-10568. Reuse ObjectMapper instance in CombinedHostsFileReader and CombinedHostsFileWriter. Contributed by Yiqun Lin.
2016-06-29 10:10:19 -07:00
c0829f4493
HDFS-6434. Default permission for creating file should be 644 for WebHdfs/HttpFS. Contributed by Wellington Chevreuil.
2016-06-28 13:55:26 -07:00
23c3ff85a9
HADOOP-13209. replace slaves with workers (John Smith via aw)
2016-06-28 05:53:03 -07:00
2a0082c51d
HDFS-10440. Improve DataNode web UI (Contributed by Weiwei Yang)
2016-06-28 16:49:39 +05:30
73615a789d
HDFS-10536. Standby NN can not trigger log roll after EditLogTailer thread failed 3 times in EditLogTailer.triggerActiveLogRoll method (Contributed by XingFeng Shen
2016-06-26 15:45:32 +05:30
bf74dbf80d
HDFS-7959. WebHdfs logging is missing on Datanode (Kihwal Lee via sjlee)
2016-06-24 14:44:15 -07:00
22be604fa3
Revert "HDFS-10534. NameNode WebUI should display DataNode usage rate with a certain percentile. Contributed by Kai Sasaki."
...
This reverts commit 0424056a77
2016-06-24 09:36:03 -07:00
0424056a77
HDFS-10534. NameNode WebUI should display DataNode usage rate with a certain percentile. Contributed by Kai Sasaki.
2016-06-24 08:58:29 -07:00
b2584bee45
HDFS-10571. TestDiskBalancerCommand#testPlanNode failed with IllegalArgumentException. Contributed by Xiaobing Zhou.
2016-06-23 18:27:02 -07:00
8a6e354122
HDFS-10552. DiskBalancer "-query" results in NPE if no plan for the node. Contributed by Anu Engineer.
2016-06-23 18:27:02 -07:00
e8de28181a
HDFS-10551. o.a.h.h.s.diskbalancer.command.Command does not actually verify options as expected. Contributed by Anu Engineer.
2016-06-23 18:27:02 -07:00
66fa34c839
HDFS-10557. Fix handling of the -fs Generic option. (Arpit Agarwal)
2016-06-23 18:27:01 -07:00
7b23ad1ef7
HDFS-10550. DiskBalancer: fix issue of order dependency in iteration in ReportCommand test. Contributed by Xiaobing Zhou.
2016-06-23 18:27:01 -07:00
c6ed54808d
HDFS-10547. DiskBalancer: fix whitespace issue in doc files. Contributed by Anu Engineer.
2016-06-23 18:27:01 -07:00
0774412e41
HDFS-10545. DiskBalancer: PlanCommand should use -fs instead of -uri to be consistent with other hdfs commands. Contributed by Anu Engineer.
2016-06-23 18:27:01 -07:00
5b8e1c26d7
HDFS-10541. Diskbalancer: When no actions in plan, error message says "Plan was generated more than 24 hours ago". Contributed by Anu Engineer.
2016-06-23 18:27:00 -07:00
cb68e5b3bd
HDFS-10540. Diskbalancer: The CLI error message for disk balancer is not enabled is not clear. Contributed by Anu Engineer.
2016-06-23 18:27:00 -07:00
3225c24e0e
HDFS-10520. DiskBalancer: Fix Checkstyle issues in test code. Contributed by Anu Engineer.
2016-06-23 18:27:00 -07:00
7e2be5c4a0
HDFS-10518. DiskBalancer: Pretty-print json in Query command. Contributed by Anu Engineer.
2016-06-23 18:21:08 -07:00
af11ab34d0
HDFS-10517. DiskBalancer: Support help command. Contributed by Anu Engineer.
2016-06-23 18:21:08 -07:00
b502102bb1
HDFS-9461. DiskBalancer: Add Report Command. Contributed by Xiaobing Zhou.
2016-06-23 18:21:08 -07:00
121142cf95
HDFS-10501. DiskBalancer: Use the default datanode port if port is not provided. Contributed by Anu Engineer.
2016-06-23 18:21:08 -07:00
78a1032b71
HDFS-10500. Diskbalancer: Print out information when a plan is not generated. Contributed by Anu Engineer.
2016-06-23 18:21:08 -07:00
d2ff793ae4
HDFS-10496. DiskBalancer: ExecuteCommand checks planFile in a wrong way. Contributed by Lei (Eddy) Xu.
2016-06-23 18:21:08 -07:00
64ccb232cc
HDFS-10478. DiskBalancer: resolve volume path names. Contributed by Anu Engineer.
2016-06-23 18:21:08 -07:00
47dcb0f952
HDFS-10476. DiskBalancer: Plan command output directory should be a sub-directory. Contributed by Anu Engineer.
2016-06-23 18:21:08 -07:00
06a9799d84
HDFS-9547. DiskBalancer: Add user documentation. Contributed by Anu Engineer.
2016-06-23 18:21:08 -07:00
43eee50966
HDFS-10403. DiskBalancer: Add cancel command. Contributed by Anu Engineer.
2016-06-23 18:21:08 -07:00
9e5fcb5e40
HDFS-10402. DiskBalancer: Add QueryStatus command. (Contributed by Anu Engineer)
2016-06-23 18:21:08 -07:00
5df2d2b8fd
HDFS-10399. DiskBalancer: Add JMX for DiskBalancer. Contributed by Anu Engineer.
2016-06-23 18:21:08 -07:00
1b39b283c7
HDFS-9546: DiskBalancer: Add Execute command. Contributed by Anu Engineer.
2016-06-23 18:21:08 -07:00
75882ec0b0
HDFS-9545: DiskBalancer: Add Plan Command. Contributed by Anu Engineer.
2016-06-23 18:21:08 -07:00
1594b472bb
HDFS-9543. DiskBalancer: Add Data mover. Contributed by Anu Engineer.
2016-06-23 18:20:24 -07:00
7820737cfa
HDFS-9735. DiskBalancer : Refactor moveBlockAcrossStorage to be used by disk balancer. Contributed by Anu Engineer.
2016-06-23 18:20:24 -07:00
050677077b
HDFS-9720. DiskBalancer : Add configuration parameters. Contributed by Anu Engineer.
2016-06-23 18:18:48 -07:00
6c606bf5c8
HDFS-9709. DiskBalancer : Add tests for disk balancer using a Mock Mover class. Contributed by Anu Engineer.
2016-06-23 18:18:48 -07:00
9be9703716
Fix a build break in HDFS-1312
2016-06-23 18:18:48 -07:00
ec601673a8
HDFS-10170. DiskBalancer: Force rebase diskbalancer branch. Contributed by Anu Engineer
2016-06-23 18:18:48 -07:00
75a711a2d5
HDFS-9703. DiskBalancer: getBandwidth implementation. (Contributed by Anu Engineer)
2016-06-23 18:18:48 -07:00
918722bdd2
HDFS-9702. DiskBalancer: getVolumeMap implementation. (Contributed by Anu Engineer)
2016-06-23 18:18:48 -07:00
4b93ddae07
HDFS-8821. Stop tracking CHANGES.txt in the HDFS-1312 feature branch. Contributed by Xiaobing Zhou
2016-06-23 18:18:48 -07:00
747227e9de
HDFS-9817. Use SLF4J in new classes. Contributed by Anu Engineer
2016-06-23 18:18:48 -07:00
3df0781aa7
HDFS-9856. Suppress Jenkins warning for sample JSON file. Contributed by Xiaobing Zhou
2016-06-23 18:18:48 -07:00
9847640603
HDFS-9683. DiskBalancer: Add cancelPlan implementation. (Contributed by Anu Engineer)
2016-06-23 18:18:48 -07:00
e646c2eb50
HDFS-9681. DiskBalancer: Add QueryPlan implementation. (Contributed by Anu Engineer)
2016-06-23 18:18:48 -07:00
2b1b2faf76
HDFS-9671. DiskBalancer: SubmitPlan implementation. (Contributed by Anu Engineer)
2016-06-23 18:18:48 -07:00
66f0bb646d
HDFS-9647. DiskBalancer: Add getRuntimeSettings. (Contributed by Anu Engineer)
2016-06-23 18:18:48 -07:00
96fe685b7a
HDFS-9645. DiskBalancer: Add Query RPC. (Contributed by Anu Engineer)
2016-06-23 18:18:48 -07:00
0501d430e2
HDFS-9595. DiskBalancer: Add cancelPlan RPC. (Contributed by Anu Engineer)
2016-06-23 18:18:48 -07:00
7100c0da35
HDFS-9588. DiskBalancer: Add submitDiskbalancer RPC. (Anu Engineer via Arpit Agarwal)
2016-06-23 18:18:48 -07:00
5724a10316
HDFS-9469. DiskBalancer: Add Planner. (Contributed by Anu Engineer)
2016-06-23 18:18:48 -07:00
e325c6ade9
HDFS-9611. DiskBalancer : Replace htrace json imports with jackson. (Contributed by Anu Engineer)
2016-06-23 18:18:48 -07:00
599eca07b4
HDFS-9526. Fix jackson annotation imports. Contributed by Xiaobing Zhou
2016-06-23 18:18:48 -07:00
30c6ebd699
HDFS-9449. DiskBalancer: Add connectors. Contributed by Anu Engineer
2016-06-23 18:18:48 -07:00
91a5c48143
HDFS-9420. Add DataModels for DiskBalancer. Contributed by Anu Engineer
2016-06-23 18:18:48 -07:00
dca298d79e
HDFS-10469. Add number of active xceivers to datanode metrics. Contributed by Hanisha Koneru.
2016-06-23 14:13:39 -07:00
e6cb07520f
HDFS-10460. Recompute block checksum for a particular range less than file size on the fly by reconstructing missed block. Contributed by Rakesh R
2016-06-24 17:39:32 +08:00
ff07b10803
HDFS-10561. test_native_mini_dfs fails by NoClassDefFoundError. (aajisaka)
2016-06-23 15:48:18 +09:00
01643020d2
HDFS-10555: Unable to loadFSEdits due to a failure in readCachePoolInfo. Contributed by Uma Maheswara Rao G
2016-06-22 22:06:13 -07:00
17eae9ebb3
HDFS-10473: Allow only suitable storage policies to be set on striped files. Contributed by Uma Maheswara Rao G
2016-06-22 11:17:43 -07:00
d433b16ce6
HDFS-7597. DelegationTokenIdentifier should cache the TokenIdentifier to UGI mapping. Contributed by Daryn Sharp, Bob Hansen, and Xiao Chen.
2016-06-22 11:00:16 +09:00
f2ac132d6a
HDFS-9530. ReservedSpace is not cleared for abandoned Blocks (Contributed by Brahma Reddy Battula)
2016-06-21 15:42:28 +05:30
46f1602e89
HDFS-10448. CacheManager#addInternal tracks bytesNeeded incorrectly when dealing with replication factors other than 1 (Yiqun Lin via cmccabe)
2016-06-20 18:25:09 -07:00
7adc4d1769
HDFS-10538. Remove AsyncDistributedFileSystem. Contributed by Xiaobing Zhou and Jitendra Pandey.
2016-06-20 17:22:55 -07:00
b7c4cf7129
HADOOP-13291. Probing stats in DFSOpsCountStatistics/S3AStorageStatistics should be correctly implemented. Contributed by Mingliang Liu.
2016-06-20 16:25:30 -07:00
aa1b583cf9
HDFS-10423. Increase default value of httpfs maxHttpHeaderSize. Contributed by Nicolae Popa.
2016-06-20 13:46:56 -07:00
5f6b4157a4
HDFS-10328. Add per-cache-pool default replication num configuration (xupeng via cmccabe)
2016-06-20 10:42:50 -07:00
0319d73c3b
HDFS-10437. ReconfigurationProtocol not covered by HDFSPolicyProvider. (Arpit Agarwal)
2016-06-18 16:16:54 -07:00
280069510b
HDFS-10493. Add links to datanode web UI in namenode datanodes page. Contributed by Weiwei Yang.
2016-06-17 11:05:44 -05:00
09e82acaf9
HADOOP-12943. Add -w -r options in dfs -test command. Contributed by Weiwei Yang.
2016-06-17 16:20:38 +09:00
e983eafb44
HDFS-10532. Typo in RollingUpgrade docs. (Contributed by Yiqun Lin)
2016-06-16 10:10:33 -07:00
cc6c265171
HDFS-10256. Use GenericTestUtils.getTestDir method in tests for temporary directories (Contributed by Vinayakumar B)
2016-06-16 16:47:06 +05:30
2ca73445f5
HDFS-10525. Fix NPE in CacheReplicationMonitor#rescanCachedBlockMap (Xiao Chen via cmccabe)
2016-06-15 22:47:01 -07:00
2449db507d
HDFS-10505. OIV's ReverseXML processor should support ACLs (Surendra Singh Lilhore via cmccabe)
2016-06-15 22:35:19 -07:00
b48f27e794
HDFS-9922. Upgrade Domain placement policy status marks a good block in violation when there are decommissioned nodes. (Chris Trezzo via mingma)
2016-06-15 22:00:52 -07:00
7d521a29ee
HDFS-9016. Display upgrade domain information in fsck. (mingma)
2016-06-14 20:05:50 -07:00
709a814fe0
HDFS-10516. Fix bug when warming up EDEK cache of more than one encryption zone. Contributed by Xiao Chen.
2016-06-13 11:50:19 -07:00
d44f4745b4
HDFS-7987. Allow files / directories to be moved (Ravi Prakash via aw)
2016-06-10 09:02:28 -07:00
8ea9bbce26
HDFS-10508. DFSInputStream should set thread's interrupt status after catching InterruptException from sleep. Contributed by Jing Zhao.
2016-06-09 14:52:29 +09:00
ae047655f4
HDFS-10220. A large number of expired leases can make namenode unresponsive and cause failover (Nicolas Fraison via raviprak)
2016-06-08 13:44:22 -07:00
1ee9ea0026
Remove redundant TestMiniDFSCluster.testDualClusters. Contributed by Jiayi Zhou.
2016-06-08 12:58:56 -07:00
be34e85e68
HDFS-10468. HDFS read ends up ignoring an interrupt. Contributed by Jing Zhao
2016-06-07 10:48:21 -07:00
e620530301
HDFS-10485. Fix findbugs warning in FSEditLog.java. (aajisaka)
2016-06-07 17:52:55 +09:00
6de9213df1
HDFS-10458. getFileEncryptionInfo should return quickly for non-encrypted cluster.
2016-06-06 15:52:39 -07:00
35f255b03b
Revert "Revert "HADOOP-13226 Support async call retry and failover.""
...
This reverts commit 5360da8bd9
2016-06-06 16:31:43 +08:00
7e7b1ae037
Revert "Revert "HDFS-10430. Reuse FileSystem#access in TestAsyncDFS. Contributed by Xiaobing Zhou.""
...
This reverts commit 8cf47d8589
2016-06-06 16:31:38 +08:00
db41e6d285
Revert "Revert "HDFS-10431 Refactor and speedup TestAsyncDFSRename. Contributed by Xiaobing Zhou""
...
This reverts commit 5ee5912ebd
2016-06-06 16:31:34 +08:00
b3d81f38da
Revert "Revert "HDFS-10390. Implement asynchronous setAcl/getAclStatus for DistributedFileSystem. Contributed by Xiaobing Zhou""
...
This reverts commit b82c74b910
2016-06-06 16:31:30 +08:00
574dcd34c0
Revert "Revert "HADOOP-13168. Support Future.get with timeout in ipc async calls.""
...
This reverts commit e4450d47f1
2016-06-06 16:31:23 +08:00
cba9a01889
Revert "Revert "HDFS-10346. Implement asynchronous setPermission/setOwner for DistributedFileSystem. Contributed by Xiaobing Zhou""
...
This reverts commit f23d5dfc60
2016-06-06 16:29:38 +08:00
aa20fa150d
Revert "Revert "HADOOP-12957. Limit the number of outstanding async calls. Contributed by Xiaobing Zhou""
...
This reverts commit 4d36b221a2
2016-06-06 16:28:47 +08:00
eded3d109e
Revert "Revert "HDFS-10224. Implement asynchronous rename for DistributedFileSystem. Contributed by Xiaobing Zhou""
...
This reverts commit 106234d873
2016-06-06 16:28:21 +08:00
106234d873
Revert "HDFS-10224. Implement asynchronous rename for DistributedFileSystem. Contributed by Xiaobing Zhou"
...
This reverts commit fc94810d3f
2016-06-03 18:09:55 -07:00
4d36b221a2
Revert "HADOOP-12957. Limit the number of outstanding async calls. Contributed by Xiaobing Zhou"
...
This reverts commit 1b9f18623a
2016-06-03 18:09:18 -07:00
f23d5dfc60
Revert "HDFS-10346. Implement asynchronous setPermission/setOwner for DistributedFileSystem. Contributed by Xiaobing Zhou"
...
This reverts commit 7251bb922b
2016-06-03 18:09:17 -07:00
e4450d47f1
Revert "HADOOP-13168. Support Future.get with timeout in ipc async calls."
...
This reverts commit 42c22f7e3d
2016-06-03 18:09:16 -07:00
b82c74b910
Revert "HDFS-10390. Implement asynchronous setAcl/getAclStatus for DistributedFileSystem. Contributed by Xiaobing Zhou"
...
This reverts commit 02d4e478a3
2016-06-03 18:09:14 -07:00
5ee5912ebd
Revert "HDFS-10431 Refactor and speedup TestAsyncDFSRename. Contributed by Xiaobing Zhou"
...
This reverts commit f4b9bcd87c
2016-06-03 18:09:13 -07:00
8cf47d8589
Revert "HDFS-10430. Reuse FileSystem#access in TestAsyncDFS. Contributed by Xiaobing Zhou."
...
This reverts commit 21890c4239
2016-06-03 18:09:12 -07:00
5360da8bd9
Revert "HADOOP-13226 Support async call retry and failover."
...
This reverts commit 83f2f78c11
2016-06-03 18:09:09 -07:00
47e0321ee9
HDFS-10481. HTTPFS server should correctly impersonate as end user to open file. Contributed by Xiao Chen.
2016-06-03 17:21:17 -07:00
99a771cd7a
HDFS-7767. Use the noredirect flag in WebHDFS to allow web browsers to upload files via the NN UI (Ravi Prakash via aw)
2016-06-03 17:07:39 -07:00
15f018434c
HDFS-9877. HDFS Namenode UI: Fix browsing directories that need to be encoded (Ravi Prakash via aw)
2016-06-03 17:06:29 -07:00
713cb71820
HADOOP-13155. Implement TokenRenewer to renew and cancel delegation tokens in KMS. Contributed by Xiao Chen.
2016-06-03 16:48:54 -07:00
97e2449477
HDFS-10341. Add a metric to expose the timeout number of pending replication blocks. (Contributed by Akira Ajisaka)
2016-06-02 13:14:45 -07:00
1df6f5735c
HDFS-10471. DFSAdmin#SetQuotaCommand's help msg is not correct. Contributed by Yiqun Lin.
2016-06-03 04:10:32 +09:00
aadb77e412
HDFS-10367. TestDFSShell.testMoveWithTargetPortEmpty fails with Address bind exception. Contributed by Brahma Reddy Battula.
2016-06-02 20:55:04 +09:00
69555fca06
HDFS-9476. TestDFSUpgradeFromImage#testUpgradeFromRel1BBWImage occasionally fail. Contributed by Masatake Iwasaki.
2016-06-02 18:52:47 +09:00
d749cf65e1
HDFS-9833. Erasure coding: recomputing block checksum on the fly by reconstructing the missed/corrupt block data. Contributed by Rakesh R.
2016-06-02 12:56:21 +08:00
8ceb06e239
HADOOP-13137. TraceAdmin should support Kerberized cluster (Wei-Chiu Chuang via cmccabe)
2016-05-31 17:54:34 -07:00
c7921c9bdd
HDFS-9466. TestShortCircuitCache#testDataXceiverCleansUpSlotsOnFailure is flaky (Wei-Chiu Chuang via cmccabe)
2016-05-31 17:37:52 -07:00
29d6cadc52
HDFS-10415. TestDistributedFileSystem#MyDistributedFileSystem attempts to set up statistics before initialize() is called (Mingliang Liu via cmccabe)
2016-05-31 16:54:52 -07:00
83f2f78c11
HADOOP-13226 Support async call retry and failover.
2016-05-31 16:30:11 -07:00
93d8a7f2a2
Revert "HADOOP-12579. Deprecate and remove WriteableRPCEngine. Contributed by Kai Zheng"
...
This reverts commit a6c79f92d5
2016-06-01 08:41:15 +08:00
21890c4239
HDFS-10430. Reuse FileSystem#access in TestAsyncDFS. Contributed by Xiaobing Zhou.
2016-05-27 14:20:18 -07:00
5ea6fd85c7
HDFS-10276. HDFS should not expose path info that user has no permission to see. (Yuanbo Liu via Yongjun Zhang)
2016-05-27 10:02:02 -07:00
916140604f
HADOOP-12911. Upgrade Hadoop MiniKDC with Kerby. Contributed by Jiajia Li
2016-05-28 14:23:39 +08:00
8c84a2a93c
HDFS-10236. Erasure Coding: Rename replication-based names in BlockManager to more generic [part-3]. Contributed by Rakesh R.
2016-05-26 16:50:55 -07:00
55c3e2de3d
HDFS-10463. TestRollingFileSystemSinkWithHdfs needs some cleanup. (Daniel Templeton via kasha)
2016-05-26 14:24:31 -07:00
f4b9bcd87c
HDFS-10431 Refactor and speedup TestAsyncDFSRename. Contributed by Xiaobing Zhou
2016-05-26 12:15:17 -07:00
77202fa103
HADOOP-13010. Refactor raw erasure coders. Contributed by Kai Zheng
2016-05-27 13:23:34 +08:00
f69f5ab3b6
HDFS-10434. Fix intermittent test failure of TestDataNodeErasureCodingMetrics. Contributed by Rakesh R
2016-05-27 04:42:53 +08:00
a6c79f92d5
HADOOP-12579. Deprecate and remove WriteableRPCEngine. Contributed by Kai Zheng
2016-05-27 04:29:42 +08:00
2dab64da2f
HDFS-10217. show 'blockScheduled' tooltip in datanodes table.
2016-05-25 12:34:52 +05:30
02d4e478a3
HDFS-10390. Implement asynchronous setAcl/getAclStatus for DistributedFileSystem. Contributed by Xiaobing Zhou
2016-05-24 14:51:27 -07:00
15ed080e36
HDFS-9365. Balaner does not work with the HDFS-6376 HA setup.
2016-05-24 12:49:48 -07:00
57c31a3fef
HDFS-9782. RollingFileSystemSink should have configurable roll interval. (Daniel Templeton via kasha)
2016-05-24 10:37:00 -07:00
4b0f55b6ea
HDFS-7766. Add a flag to WebHDFS op=CREATE to not respond with a 307 redirect (Ravi Prakash via aw)
2016-05-23 15:52:01 -07:00
12eb5a7b11
HDFS-10439. Update setOwner doc in HdfsPermissionsGuide. Contributed by John Zhuge.
2016-05-22 16:53:30 +09:00
500e946729
HDFS-10438. When NameNode HA is configured to use the lifeline RPC server, it should log the address of that server. Contributed by Chris Nauroth.
2016-05-20 13:01:43 -07:00
bcde1562d2
HDFS-10424. DatanodeLifelineProtocol not able to use under security cluster. Contributed by Chris Nauroth.
2016-05-20 12:47:41 -07:00
42c22f7e3d
HADOOP-13168. Support Future.get with timeout in ipc async calls.
2016-05-19 15:34:04 -07:00
204de430bf
HDFS-10417. Improve error message from checkBlockLocalPathAccess. Contributed by Tianyin Xu.
2016-05-19 15:52:32 -05:00
010e6ac328
HDFS-2173. saveNamespace should not throw IOE when only one storage directory fails to write VERSION file. Contributed by Andras Bokor.
2016-05-18 14:13:41 -07:00
cf552aa87b
HDFS-10360. DataNode may format directory and lose blocks if current/VERSION is missing. (Wei-Chiu Chuang via lei)
2016-05-18 09:38:37 -07:00
f4d8fde822
HDFS-9926. MiniDFSCluster leaks dependency Mockito via DataNodeTestUtils. (Contributed by Josh Elser)
2016-05-18 09:20:57 -07:00
dd99f5fc79
HDFS-10383. Safely close resources in DFSTestUtil. Contributed by Mingliang Liu.
2016-05-17 16:12:00 -07:00
16c07cc68a
HDFS-10381, DataStreamer DataNode exclusion log message should be warning. (John Zhuge via Yongjun Zhang)
2016-05-17 15:57:33 -07:00
f6ef876fe1
MAPREDUCE-6657. Job history server can fail on startup when NameNode is in start phase. Contributed by Haibo Chen.
2016-05-17 14:41:47 -07:00
e24fe2641b
HDFS-9732, Improve DelegationTokenIdentifier.toString() for better logging. Contributed by Yongjun Zhang.
2016-05-17 14:03:08 -07:00
1356cbe994
Revert "HDFS-10417. Improve error message from checkBlockLocalPathAccess. Contributed by Tianyin Xu."
...
This reverts commit 0942954e8a
2016-05-17 15:20:03 -05:00
0942954e8a
HDFS-10417. Improve error message from checkBlockLocalPathAccess. Contributed by Tianyin Xu.
2016-05-17 15:06:25 -05:00
7cd5ae62f6
HDFS-10404. Fix formatting of CacheAdmin command usage help text (Yiqun Lin via cmccabe)
2016-05-17 11:10:48 -07:00
4a5819dae2
HDFS-10303. DataStreamer#ResponseProcessor calculates packet ack latency incorrectly. Contributed by Surendra Singh Lilhore.
2016-05-17 08:53:31 -05:00
9fe5828f05
HDFS-10242. Cannot create space quota of zero. Contributed by Takashi Ohnishi.
2016-05-17 15:53:09 +09:00
61f46be071
HDFS-10208. Addendum for HDFS-9579: to handle the case when client machine can't resolve network path (Ming Ma via sjlee)
2016-05-16 18:49:47 -07:00
730bc746f9
HADOOP-12930. Dynamic subcommands for hadoop shell scripts (aw)
...
This commit contains the following JIRA issues:
HADOOP-12931. bin/hadoop work for dynamic subcommands
HADOOP-12932. bin/yarn work for dynamic subcommands
HADOOP-12933. bin/hdfs work for dynamic subcommands
HADOOP-12934. bin/mapred work for dynamic subcommands
HADOOP-12935. API documentation for dynamic subcommands
HADOOP-12936. modify hadoop-tools to take advantage of dynamic subcommands
HADOOP-13086. enable daemonization of dynamic commands
HADOOP-13087. env var doc update for dynamic commands
HADOOP-13088. fix shellprofiles in hadoop-tools to allow replacement
HADOOP-13089. hadoop distcp adds client opts twice when dynamic
HADOOP-13094. hadoop-common unit tests for dynamic commands
HADOOP-13095. hadoop-hdfs unit tests for dynamic commands
HADOOP-13107. clean up how rumen is executed
HADOOP-13108. dynamic subcommands need a way to manipulate arguments
HADOOP-13110. add a streaming subcommand to mapred
HADOOP-13111. convert hadoop gridmix to be dynamic
HADOOP-13115. dynamic subcommand docs should talk about exit vs. continue program flow
HADOOP-13117. clarify daemonization and security vars for dynamic commands
HADOOP-13120. add a --debug message when dynamic commands have been used
HADOOP-13121. rename sub-project shellprofiles to match the rest of Hadoop
HADOOP-13129. fix typo in dynamic subcommand docs
HADOOP-13151. Underscores should be escaped in dynamic subcommands document
HADOOP-13153. fix typo in debug statement for dynamic subcommands
2016-05-16 17:54:45 -07:00
6a6e74acf5
HDFS-10410. RedundantEditLogInputStream.LOG is set to wrong class. (John Zhuge via lei)
2016-05-16 17:05:46 -07:00
ad9441122f
HDFS-8449. Add tasks count metrics to datanode for ECWorker. Contributed by Bo Li.
2016-05-15 23:39:09 -07:00
45788204ae
HDFS-10333. Intermittent org.apache.hadoop.hdfs.TestFileAppend failure in trunk. Contributed by Yiqun Lin.
2016-05-15 22:15:05 -07:00
3c5c57af28
HADOOP-13142. Change project version from 3.0.0 to 3.0.0-alpha1.
2016-05-12 18:27:28 -07:00
a2774debf7
HDFS-9389. Add maintenance states to AdminStates. (Ming Ma via lei)
2016-05-12 15:42:54 -07:00
ca5613af91
Revert "Update project version to 3.0.0-alpha1-SNAPSHOT."
...
This reverts commit 6b53802cba
2016-05-12 15:32:45 -07:00
6b53802cba
Update project version to 3.0.0-alpha1-SNAPSHOT.
2016-05-12 11:05:05 -07:00
7251bb922b
HDFS-10346. Implement asynchronous setPermission/setOwner for DistributedFileSystem. Contributed by Xiaobing Zhou
2016-05-11 17:53:40 -07:00
687233f20d
HADOOP-13065. Add a new interface for retrieving FS and FC Statistics (Mingliang Liu via cmccabe)
2016-05-11 13:45:39 -07:00
307db61aac
HDFS-10172. hdfs erasurecode command should remove the redundant -usage option. Contributed by Yuanbo Liu.
2016-05-10 15:20:17 -07:00
f0252ad252
HDFS-10377. CacheReplicationMonitor shutdown log message should use INFO level (Yiqun Lin via cmccabe)
2016-05-10 11:15:05 -07:00
b9e5a32fa1
HDFS-10372. Fix for failing TestFsDatasetImpl#testCleanShutdownOfVolume. Contributed by Rushabh Shah.
2016-05-09 15:50:09 -05:00
411fb4b2b7
HDFS-10337. OfflineEditsViewer stats option should print 0 instead of null for the count of operations. Contributed by Yiqun Lin.
2016-05-08 15:49:13 -07:00
6957e45699
HDFS-10374. typo in HDFSHighAvailabilityWithQJM doc. Contributed by Takashi Ohnishi.
2016-05-07 11:26:42 +09:00
2835f146b5
HDFS-2043. TestHFlush failing intermittently. Contributed by Lin Yiqun.
2016-05-06 15:36:47 +09:00
dacd1f50fe
HDFS-10324. Trash directory in an encryption zone should be pre-created with correct permissions. Contributed by Wei-Chiu Chuang.
2016-05-05 18:08:47 -07:00
d8faf47f32
HADOOP-13051. Test for special characters in path being respected during globPaths. Contributed by Harsh J.
2016-05-05 14:18:39 -07:00
35cf503149
HADOOP-13077. Handle special characters in passwords in httpfs.sh (Xiao Chen via aw)
2016-05-05 11:33:06 -07:00
1268cf5fbe
HDFS-10320. Rack failures may result in NN terminate. (Xiao Chen via mingma)
2016-05-04 17:02:26 -07:00
6d77d6eab7
HDFS-9902. Support different values of dfs.datanode.du.reserved per storage type. (Contributed by Brahma Reddy Battula)
2016-05-03 16:52:43 -07:00
45a753ccf7
HDFS-10344. DistributedFileSystem#getTrashRoots should skip encryption zone that does not have .Trash. Contributed by Xiaoyu Yao.
2016-05-02 19:30:47 -07:00
3ff0510ffd
Fix hadoop-hdfs-native-client compilation on Windows. Contributed by Brahma Reddy Battula.
2016-05-02 18:15:51 -07:00
1b9f18623a
HADOOP-12957. Limit the number of outstanding async calls. Contributed by Xiaobing Zhou
2016-05-02 11:16:30 -07:00
2753185a01
HADOOP-12563. Updated utility (dtutil) to create/modify token files. Contributed by Matthew Paduano
2016-04-29 22:32:27 -07:00
7da540d03e
HDFS-10347. Namenode report bad block method doesn't log the bad block or datanode. Contributed by Rushabh Shah.
2016-04-29 15:50:43 -05:00
4da6f69ca1
HDFS-10335 Mover$Processor#chooseTarget() always chooses the first matching target storage group. Contributed by Mingliang Liu
2016-04-29 11:22:33 -07:00
af9b000535
HDFS-10260. TestFsDatasetImpl#testCleanShutdownOfVolume often fails. Contributed by Rushabh Shah.
2016-04-29 09:58:44 -05:00
6243eabb48
HDFS-9958. BlockManager#createLocatedBlocks can throw NPE for corruptBlocks on failed storages. Contributed by Kuhu Shukla
2016-04-28 12:44:53 -05:00
8bd0bca0b1
HDFS-3702. Fix missing imports from HDFS-3702 trunk patch.
2016-04-27 15:37:41 -07:00
0a152103f1
HDFS-3702. Add an option for NOT writing the blocks locally if there is a datanode on the same box as the client. (Contributed by Lei (Eddy) Xu)
2016-04-27 14:22:51 -07:00
7b1c37a13a
HADOOP-12892. fix/rewrite create-release (aw)
2016-04-27 08:38:22 -07:00
919a1d824a
HDFS-10330. Add Corrupt Blocks Information in Metasave output. Contributed by Kuhu Shukla.
2016-04-27 08:19:48 -05:00
fc94810d3f
HDFS-10224. Implement asynchronous rename for DistributedFileSystem. Contributed by Xiaobing Zhou
2016-04-26 17:10:13 -07:00
6be22ddbf1
HDFS-10297. Increase default balance bandwidth and concurrent moves. Contributed by John Zhuge.
2016-04-26 13:04:18 -07:00
c6a2430b47
HDFS-10329. Bad initialisation of StringBuffer in RequestHedgingProxyProvider. Contributed by Lin Yiqun.
2016-04-26 11:45:29 -05:00
5865fe2bf0
HDFS-9869. Erasure Coding: Rename replication-based names in BlockManager to more generic [part-2]. Contributed by Rakesh R.
2016-04-25 22:01:54 -07:00
8eadd7145e
HDFS-10325. TestDFSShell#testMoveWithTargetPortEmpty fails. Contributed by Takanobu Asanuma.
2016-04-26 07:11:13 +09:00
f308561f1d
HDFS-8057 Move BlockReader implementation to the client implementation package. Contributed by Takanobu Asanuma
2016-04-25 12:01:48 -07:00
10f0f7851a
HDFS-10318. TestJMXGet hides the real error in case of test failure. Contributed by Andras Bokor.
2016-04-25 11:38:14 -05:00
6fcde2e38d
HDFS-9905. WebHdfsFileSystem#runWithRetry should display original stack trace on error. (Wei-Chiu Chuang via iwasakims)
2016-04-23 23:37:56 +09:00
63e5412f1a
HDFS-9427. HDFS should not default to ephemeral ports. Contributed by Xiaobing Zhou.
2016-04-22 15:14:40 -07:00
d6402faded
Revert "HADOOP-12563. Updated utility (dtutil) to create/modify token files. Contributed by Matthew Paduano"
...
This reverts commit 4838b735f0
2016-04-22 10:17:20 -07:00
5d4255a801
HDFS-9555. LazyPersistFileScrubber should still sleep if there are errors in the clear progress. Contributed by Phil Yang.
2016-04-22 11:08:44 -05:00
4a043b3fec
HDFS-9894. Add unsetStoragePolicy API to FileContext/AbstractFileSystem and derivatives. Contributed by Xiaobing Zhou.
2016-04-21 20:17:27 -07:00
14ab7a81e2
HDFS-10309 Balancer doesn't honor dfs.blocksize value defined with suffix k(kilo), m(mega), g(giga). Contributed by Amit Anand
2016-04-21 15:41:50 -07:00
bbce1d525e
HDFS-10319. Balancer should not try to pair storages with different types.
2016-04-21 15:20:12 -07:00
4838b735f0
HADOOP-12563. Updated utility (dtutil) to create/modify token files. Contributed by Matthew Paduano
2016-04-21 11:32:39 -07:00
b4be288c5d
HDFS-10207. Support enable Hadoop IPC backoff without namenode restart. Contributed by Xiaobing Zhou.
2016-04-21 10:18:48 -07:00
5bd7b592e5
HADOOP-12985. Support MetricsSource interface for DecayRpcScheduler Metrics. Contributed by Xiaoyu Yao.
2016-04-20 23:10:52 -07:00
63ac2db59a
HDFS-10312. Large block reports may fail to decode at NameNode due to 64 MB protobuf maximum length restriction. Contributed by Chris Nauroth.
2016-04-20 13:39:44 -07:00
ad36fa6f42
HDFS-10308. TestRetryCacheWithHA#testRetryCacheOnStandbyNN failing (Rakesh R via cmccabe)
2016-04-20 11:33:11 -07:00
af9bdbe447
HDFS-10264. Logging improvements in FSImageFormatProtobuf.Saver. (Contributed by Xiaobing Zhou)
2016-04-19 11:26:30 -07:00
90d1af1416
HDFS-9744. TestDirectoryScanner#testThrottling occasionally time out after 300 seconds. Contributed by Lin Yiqun.
2016-04-19 18:54:11 +08:00
4b4df2cd48
HDFS-10284. o.a.h.hdfs.server.blockmanagement.TestBlockManagerSafeMode.testCheckSafeMode fails intermittently. Contributed by Mingliang Liu.
2016-04-19 18:07:46 +08:00
438efc3634
HDFS-10291 TestShortCircuitLocalRead failing (stevel)
...
(cherry picked from commit dd64769699
2016-04-19 17:48:20 +08:00
b5d4c7dc76
HADOOP-12924. Configure raw erasure coders for supported codecs. Contributed by Rui Li.
2016-04-20 15:47:01 +08:00
be0bce1b71
HDFS-10306. SafeModeMonitor should not leave safe mode if name system is starting active service. Contributed by Mingliang Liu.
2016-04-18 18:05:49 -07:00
cb3ca460ef
HDFS-10265. OEV tool fails to read edit xml file if OP_UPDATE_BLOCKS has no BLOCK tag (Wan Chang via cmccabe)
2016-04-18 13:47:56 -07:00
d8b729e16f
HDFS-10302. BlockPlacementPolicyDefault should use default replication considerload value. Contributed by Lin Yiqun.
2016-04-18 07:58:55 -05:00
ab903029a9
HDFS-10275. TestDataNodeMetrics failing intermittently due to TotalWriteTime counted incorrectly. Contributed by Lin Yiqun.
2016-04-18 20:29:29 +08:00
67523ffcf4
HDFS-9412. getBlocks occupies FSLock and takes too long to complete. Contributed by He Tianyi.
2016-04-18 09:28:02 +08:00
55e19b7f0c
HDFS-10293. StripedFileTestUtil#readAll flaky. Contributed by Mingliang Liu.
2016-04-15 10:53:40 -07:00
89a838769f
HDFS-10283. o.a.h.hdfs.server.namenode.TestFSImageWithSnapshot#testSaveLoadImageWithAppending fails intermittently. Contributed by Mingliang Liu.
2016-04-15 10:49:21 -07:00
b9c9d03591
HDFS-10281. TestPendingCorruptDnMessages fails intermittently. Contributed by Mingliang Liu.
2016-04-14 15:24:39 -05:00
2c155afe27
HDFS-10292. Add block id when client got Unable to close file exception. Contributed by Brahma Reddy Battula.
2016-04-14 14:25:11 -05:00
809226752d
HDFS-10286. Fix TestDFSAdmin#testNameNodeGetReconfigurableProperties. Contributed by Xiaobing Zhou.
2016-04-14 10:57:02 -07:00
c970f1d005
HDFS-10280. Document new dfsadmin command -evictWriters. Contributed by Wei-Chiu Chuang.
2016-04-14 12:45:47 -05:00
0d1c1152f1
HDFS-10282. The VolumeScanner should warn about replica files which are misplaced. Contributed by Colin Patrick McCabe.
2016-04-14 07:58:24 -05:00
5566177c9a
HDFS-10209. Support enable caller context in HDFS namenode audit log without restart namenode. Contributed by Xiaobing Zhou.
2016-04-13 16:51:13 -07:00
4895c73dd4
Revert "HDFS-10209. Support enable caller context in HDFS namenode audit log without restart namenode"
...
This reverts commit 192112d5a2
2016-04-13 16:45:18 -07:00
314aa21a89
HDFS-10279. Improve validation of the configured number of tolerated failed volumes. Contributed by Lin Yiqun.
2016-04-13 16:39:50 -07:00
192112d5a2
HDFS-10209. Support enable caller context in HDFS namenode audit log without restart namenode
2016-04-13 15:34:41 -07:00
d2f3bbc290
HDFS-10270. TestJMXGet:testNameNode() fails. Contributed by Gergely Novák
2016-04-13 11:22:36 -05:00
903428bf94
HDFS-9772. TestBlockReplacement#testThrottler doesn't work as expected. Contributed by Lin Yiqun.
2016-04-13 17:49:45 +08:00
35f0770555
HADOOP-12973. Make DU pluggable. (Elliott Clark via cmccabe)
2016-04-12 16:28:42 -07:00
6ef42873a0
HDFS-9918. Erasure Coding: Sort located striped blocks based on decommissioned states. Contributed by Rakesh R.
2016-04-12 13:38:58 -07:00
600d129bb8
HDFS-10273. Remove duplicate logSync() and log message in FSN#enterSafemode() (Vinayakumar B via cmccabe)
2016-04-12 10:52:39 -07:00
209303be3a
HDFS-8356. Document missing properties in hdfs-default.xml. Contributed by Ray Chiang.
2016-04-12 15:26:50 +09:00
a9a607f8fc
HDFS-10271. Extra bytes are getting released from reservedSpace for append (Contributed by Brahma Reddy Battula)
2016-04-12 11:44:34 +05:30
a409508b3f
HDFS-10277. PositionedReadable test testReadFullyZeroByteFile failing in HDFS. Contributed by Steve Loughran.
2016-04-12 14:55:01 +09:00
843ee8d59d
HADOOP-12994. Specify PositionedReadable, add contract tests, fix problems. Contributed by Steve Loughran.
2016-04-08 13:36:58 -07:00
9c32f8785e
HDFS-10266. Remove unused properties dfs.client.file-block-storage-locations.num-threads and dfs.client.file-block-storage-locations.timeout.millis. Contributed by Ray Chiang.
2016-04-08 15:32:13 +09:00
3c18a53cbd
HDFS-9719. Refactoring ErasureCodingWorker into smaller reusable constructs. Contributed by Kai Zheng.
2016-04-06 22:50:24 -07:00
654cd1d0c0
HDFS-10186. DirectoryScanner: Improve logs by adding full path of both actual and expected block directories. Contributed by Rakesh R
2016-04-07 12:38:42 +08:00
4bd7cbc29d
HDFS-10267. Extra "synchronized" on FsDatasetImpl#recoverAppend and FsDatasetImpl#recoverClose
2016-04-06 21:07:31 -07:00
aede8c10ec
HDFS-9945. Datanode command for evicting writers. Contributed by Kihwal Lee
2016-04-06 20:20:14 +00:00
188f65287d
HDFS-6520. hdfs fsck passes invalid length value when creating BlockReader (Xiao Chen via cmccabe)
2016-04-06 11:28:34 -07:00
221b3a8722
HDFS-10192. Namenode safemode not coming out during failover. Contributed by Brahma Reddy Battula.
2016-04-06 10:42:59 -07:00
9ba1e5af06
HDFS-10261. TestBookKeeperHACheckpoints doesn't handle ephemeral HTTP ports. Contributed by Eric Badger.
2016-04-05 16:26:18 -05:00
0cd320a846
HDFS-10235. Last contact for Live Nodes should be relative time. Contributed by Brahma Reddy Battula.
2016-04-05 13:41:19 -07:00
85ec5573eb
HADOOP-12672. RPC timeout should not override IPC ping interval (iwasakims)
2016-04-06 03:22:48 +09:00
917464505c
HDFS-10239. Fsshell mv fails if port usage doesn't match in src and destination paths. Contributed by Kuhu Shukla.
2016-04-05 09:07:24 -05:00
818d6b799e
HDFS-9917. IBR accumulate more objects when SNN was down for sometime. (Contributed by Brahma Reddy Battula)
2016-04-05 09:49:39 +08:00
f6b1a81812
HDFS-8496. Calling stopWriter() with FSDatasetImpl lock held may block other threads (cmccabe)
2016-04-04 18:02:15 -07:00
a7d1fb0cd2
HDFS-10178. Permanent write failures can happen if pipeline recoveries occur for the first packet. Contributed by Kihwal Lee.
2016-04-04 16:40:00 -05:00
154d2532cf
HDFS-9599. TestDecommissioningStatus.testDecommissionStatus occasionally fails (Lin Yiqun via iwasakims)
2016-04-05 03:19:48 +09:00
54b2e78fd2
HDFS-10253. Fix TestRefreshCallQueue failure (Contributed by Xiaoyu Yao)
2016-04-03 13:27:49 +05:30
0a74610d1c
HADOOP-11393. Revert HADOOP_PREFIX, go back to HADOOP_HOME (aw)
2016-03-31 07:51:05 -07:00
32c0c3ecdf
HDFS-10221. Add .json to the rat exclusions. Contributed by Ming Ma.
2016-03-31 09:06:14 +09:00
37e23ce45c
HDFS-10223. peerFromSocketAndKey performs SASL exchange before setting connection timeouts (cmccabe)
2016-03-30 13:37:37 -07:00
09d63d5a19
HDFS-5177. blocksScheduled count should be decremented for abandoned blocks (Contributed by Vinayakumar B)
2016-03-30 14:22:00 +08:00
ddfe6774c2
HDFS-9439. Support reconfiguring fs.protected.directories without NN restart. (Contributed by Xiaobing Zhou)
2016-03-29 13:55:00 -07:00
c9307e48b7
HDFS-10228. TestHDFSCLI fails. (Contributed by Akira AJISAKA)
2016-03-29 13:50:08 -07:00
f2aec4eb82
HDFS-10197. TestFsDatasetCache failing intermittently due to timeout. Contributed by Lin Yiqun.
2016-03-29 11:54:42 -07:00
1f004b3367
HDFS-9871. "Bytes Being Moved" -ve(-1 B) when cluster was already balanced. (Contributed by Brahma Reddy Battulla)
2016-03-29 11:25:41 +08:00
d8383c687c
HDFS-10182. Hedged read might overwrite user's buf. Contributed by zhouyingchao.
2016-03-28 15:44:25 +08:00
3a4ff7776e
HDFS-9694. Make existing DFSClient#getFileChecksum() work for striped blocks. Contributed by Kai Zheng
2016-03-26 19:58:09 -07:00
a337ceb74e
Revert "HDFS-9694. Make existing DFSClient#getFileChecksum() work for striped blocks. Contributed by Kai Zheng"
...
This reverts commit e5ff0ea7ba
2016-03-26 09:20:01 -07:00
e5ff0ea7ba
HDFS-9694. Make existing DFSClient#getFileChecksum() work for striped blocks. Contributed by Kai Zheng
2016-03-26 00:52:50 -07:00
fde8ac5d85
Add missing files from HDFS-9005. (lei)
2016-03-25 17:11:35 -07:00
4fcfea71bf
HDFS-9005. Provide support for upgrade domain script. (Ming Ma via Lei Xu)
2016-03-25 17:09:12 -07:00
738155063e
HADOOP-12857. rework hadoop-tools (aw)
2016-03-23 13:46:38 -07:00
8f85e5d212
HDFS-10200. Docs for WebHDFS still describe GETDELEGATIONTOKENS operation. Contributed by Wellington Chevreuil.
2016-03-23 12:55:15 -07:00
0d19a0ce98
HDFS-10193. fuse_dfs segfaults if uid cannot be resolved to a username (John Thiltges via cmccabe)
2016-03-23 11:19:18 -07:00
a7d8f2b396
HDFS-10189. PacketResponder#toString should include the downstreams for PacketResponderType.HAS_DOWNSTREAM_IN_PIPELINE (Joe Pallas via cmccabe)
2016-03-22 14:56:02 -07:00
680716f31e
HDFS-9951. Use string constants for XML tags in OfflineImageReconstructor (Lin Yiqun via cmccabe)
2016-03-21 11:40:37 -07:00
e3bb38d625
HDFS-9405. Warmup NameNode EDEK caches in background thread. Contributed by Xiao Chen.
2016-03-21 11:39:05 -07:00
8a3f0cb255
HDFS-7166. SbNN Web UI shows #Under replicated blocks and #pending deletion blocks. Contributed by Wei-Chiu Chuang.
2016-03-20 17:21:25 -07:00
cd8b6889a7
HDFS-9579. Provide bytes-read-by-network-distance metrics at FileSystem.Statistics level (Ming Ma via sjlee)
2016-03-19 14:02:04 -07:00
9b623fbaf7
HDFS-3677. dfs.namenode.edits.dir.required is missing from hdfs-default.xml. Contributed by Mark Yang.
2016-03-19 01:11:15 +09:00
63c966a3fb
HDFS-9874. Long living DataXceiver threads cause volume shutdown to block. Contributed by Rushabh Shah.
2016-03-18 10:24:59 -05:00
dc951e606f
HDFS-9949. Add a test case to ensure that the DataNode does not regenerate its UUID when a storage directory is cleared (Harsh J via cmccabe)
2016-03-17 10:37:42 -07:00
02a250db9f
HDFS-10173. Typo in DataXceiverServer. Contributed by Michael Han.
2016-03-16 18:49:57 -07:00
32d043d9c5
HDFS-9857. Erasure Coding: Rename replication-based names in BlockManager to more generic [part-1]. Contributed by Rakesh R.
2016-03-16 16:53:58 -07:00
Arpit Agarwal
Brahma Reddy Battula
Sunil G
Akira Ajisaka
Kihwal Lee
Jason Lowe
Vinayakumar B
Andrew Wang
Chris Douglas
Yiqun Lin
Lei Xu
Mingliang Liu
Erik Krogen
Xiaoyu Yao
John Zhuge
Kai Zheng
Wei-Chiu Chuang
Zhe Zhang
Konstantin V Shvachko
Yongjun Zhang
Anu Engineer
Chris Nauroth
Hanisha Koneru
Allen Wittenauer
Xiaobing Zhou
Daniel Templeton
Rakesh Radhakrishnan
Masatake Iwasaki
Junping Du
Tsuyoshi Ozawa
Ravi Prakash
Chen Liang
Chen Liang
Jing Zhao
Ming Ma
Xiao Chen
Eric Payne
Jitendra Pandey
Vinitha Reddy Gankidi
Haohui Mai
Uma Maheswara Rao G
Karthik Kambatla
Robert Kanter
Robert Kanter
Benoy Antony
Wangda Tan
Daryn Sharp
Jason Lowe
Eric Yang
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
Yuanbo Liu
Tsz-Wo Nicholas Sze
Steve Loughran
Colin Patrick Mccabe
Sangjin Lee
Aaron T. Myers
Karthik Kambatla
Ravi Prakash
Walter Su
Xiaobing Zhou
Zhe Zhang