343ce8a8c9
HDFS-15906. Close FSImage and FSNamesystem after formatting is complete ( #2788 )
2021-03-22 20:29:17 +09:00
03a3a04c06
HDFS-15787. Remove unnecessary Lease Renew in FSNamesystem#internalReleaseLease. Contributed by Lisheng Sun.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2021-03-22 18:37:00 +08:00
261191cbc0
HDFS-15904 : De-flake TestBalancer#testBalancerWithSortTopNodes() ( #2785 )
...
Contributed by Viraj Jasani.
Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2021-03-19 20:37:44 -07:00
f5f3fc6fe9
HDFS-15868. Possible Resource Leak in EditLogFileOutputStream ( #2736 ). Contributed by Narges Shadab.
...
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
2021-03-19 16:36:48 +08:00
9c43b60348
HDFS-15895 : Remove redundant String#format in DFSAdmin#printOpenFiles ( #2772 )
2021-03-17 15:12:36 +09:00
9ba60c33d8
HDFS-15890. Improve the Logs for File Concat Operation. Contributed by Bhavik Patel.
2021-03-17 14:14:26 +09:00
25390babc7
HDFS-13975. TestBalancer#testMaxIterationTime fails sporadically ( #2726 )
...
Co-authored-by: toshihiko.uchida <toshihiko.uchida@linecorp.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-03-17 12:00:05 +09:00
0d42104119
HDFS-15898. Test case TestOfflineImageViewer fails ( #2779 )
2021-03-17 09:11:30 +08:00
47620f8821
HDFS-15843. Make write cross-platform ( #2710 )
2021-03-16 09:06:51 -07:00
7025f39944
HDFS-15809. DeadNodeDetector does not remove live nodes from dead node set. Contributed by Jinglun.
2021-03-15 11:34:13 +08:00
970455c917
HDFS-15816. Fix shouldAvoidStaleDataNodesForWrite returns when no stale node in cluster. Contributed by Yang Yun.
2021-03-14 17:48:18 +08:00
e565b05c80
HDFS-15884. RBF: Remove unused method getCreateLocation in RouterRpcServer ( #2754 ). Contributed by tomscut.
2021-03-14 04:07:31 +05:30
6921ec8b40
HDFS-15882. Fix incorrectly initializing RandomAccessFile based on configuration options ( #2751 ). Contributed by Xie Lei.
...
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
2021-03-13 14:22:11 +08:00
7cf5969f0b
HADOOP-17581. Fix reference to LOG is ambiguous after HADOOP-17482. ( #2764 )
2021-03-12 11:27:53 -08:00
ecd3335187
HDFS-15848. Snapshot Operations: Add debug logs at the entry point. Contributed by Bhavik Patel.
2021-03-12 15:42:15 +09:00
23b343aed1
HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin ( #2753 )
...
Removed findbugs from the hadoop build images and added spotbugs instead.
Upgraded SpotBugs to 4.2.2 and spotbugs-maven-plugin to 4.2.0.
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
2021-03-11 10:56:07 +09:00
6a55baeee4
HDFS-15875. Check whether file is being truncated before truncate ( #2746 )
2021-03-10 14:11:29 +08:00
6a922f9896
HDFS-15873. Add namenode address in logs for block report ( #2743 )
2021-03-08 11:50:51 -05:00
4a0b7f7ebe
HDFS-15384. Document getLocatedBlocks(String src, long start) of DFSClient only return partial blocks. Contributed by Yang Yun.
2021-03-08 12:18:46 +08:00
9cb51bf106
HDFS-15808. Add metrics for FSNamesystem read/write lock hold long time. ( #2668 ) Contributed by tomscut.
2021-03-06 13:40:44 -08:00
cdba06e380
HDFS-15870. Remove unused configuration dfs.namenode.stripe.min ( #2739 )
...
Co-authored-by: tomscut <litao@bigo.sg>
2021-03-03 22:34:02 +09:00
8af56de1fa
HADOOP-17560. Fix some spelling errors ( #2730 )
...
Co-authored-by: jiaguodong5 <jiaguodong5@jd.com>
2021-03-03 11:41:05 +09:00
1f1a1ef52d
HDFS-15856: Make write pipeline retry times configurable. ( #2721 ). Contributed by Qi Zhu
...
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
2021-03-02 13:16:11 +08:00
9501c698f4
HDFS-15849. ExpiredHeartbeats metric should be of Type.COUNTER. Contributed by Qi Zhu.
2021-03-01 17:17:23 -08:00
32353eb38a
HDFS-15854. Make some parameters configurable for SlowDiskTracker and SlowPeerTracker ( #2718 )
...
Authored-by: tomscut <litao@bigo.sg>
2021-03-01 07:52:59 -08:00
c3b3b36dee
HDFS-14013. Skip any credentials stored in HDFS when starting ZKFC. Contributed by Stephen O'Donnell
2021-03-01 11:36:41 +00:00
9c583f36d5
HDFS-15422. Reported IBR is partially replaced with stored info when queuing. Contributed by Stephen O'Donnell and Kihwal Lee
2021-02-24 10:29:36 +00:00
940c780feb
HDFS-15781. Add metrics for how blocks are moved in replaceBlock. ( #2704 )
2021-02-23 22:29:48 -08:00
647d23b611
HDFS-15847. create client protocol: add ecPolicyName & storagePolicy param to debug statement string. Contributed by Bhavik Patel.
2021-02-23 19:08:38 +05:30
a57282540f
HDFS-15845. RBF: Router fails to start due to NoClassDefFoundError for hadoop-federation-balance. ( #2714 )
2021-02-23 15:12:06 +09:00
1e32ab7d2f
HDFS-15734. [READ] DirectoryScanner#scan need not check StorageType.PROVIDED ( #2559 )
...
Co-authored-by: wangyuxuan <wangyuxuan@xiaomi.com>
2021-02-22 18:13:36 +08:00
f8059d0bdf
Revert "first commit ( #2559 )"
...
This reverts commit 1b94569d89
2021-02-22 18:13:06 +08:00
1b94569d89
first commit ( #2559 )
...
Co-authored-by: wangyuxuan <wangyuxuan@xiaomi.com>
2021-02-22 02:10:39 -08:00
98ffd65d8b
HDFS-15826. Solve the problem of incorrect progress of delegation tokens when loading FsImage. ( #2686 )
...
Co-authored-by: zhujianghua <zhujianghua@zhujianghuadeMacBook-Pro.local>
2021-02-21 23:22:32 -08:00
ff84a57483
HDFS-15806. DeadNodeDetector should close all the threads when it is closed. Contributed by Jinglun.
2021-02-20 19:48:00 +05:30
2ce5752fa8
HDFS-15839. RBF: Cannot get method setBalancerBandwidth on Router Client. Contributed by Yang Yun.
2021-02-20 19:34:29 +05:30
25af8901a9
HDFS-15835. Erasure coding: Add/remove logs for the better readability/debugging. Contributed by Bhavik Patel.
2021-02-19 14:44:58 +09:00
9a298d180d
Revert "HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin ( #2454 )"
...
This reverts commit 4cf3531583
2021-02-19 11:09:10 +09:00
2970bd93f3
HDFS-15830. Support to make dfs.image.parallel.load reconfigurable ( #2694 )
2021-02-19 09:07:22 +08:00
e391844e8e
HDFS-15815. if required storageType are unavailable, log the failed reason during choosing Datanode. Contributed by Yang Yun.
2021-02-18 01:39:04 +05:30
0ed9e36282
HDFS-15834. Remove the usage of org.apache.log4j.Level ( #2696 )
2021-02-17 01:59:24 -05:00
4cf3531583
HADOOP-16870. Use spotbugs-maven-plugin instead of findbugs-maven-plugin ( #2454 )
...
Use spotbugs instead of findbugs. Removed findbugs from the hadoop build images,
and added spotbugs in the images instead.
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Dinesh Chitlangia <dineshc@apache.org>
2021-02-17 10:38:20 +09:00
2b3c5b1733
HDFS-15836. RBF: Fix contract tests after HADOOP-13327 ( #2702 )
...
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
2021-02-16 14:50:50 +09:00
07a4220cd2
HDFS-15821. Add metrics for in-service datanodes ( #2690 ). Contributed by Zehao Chen.
2021-02-15 11:14:32 -06:00
c3134ab3a9
HDFS-15814. Make some parameters configurable for DataNodeDiskMetrics ( #2676 )
2021-02-14 00:12:24 -05:00
798df6d699
HADOOP-13327 Output Stream Specification. ( #2587 )
...
This defines what output streams and especially those which implement
Syncable are meant to do, and documents where implementations (HDFS; S3)
don't. With tests.
The file:// FileSystem now supports Syncable if an application calls
FileSystem.setWriteChecksum(false) before creating a file -checksumming
and Syncable.hsync() are incompatible.
Contributed by Steve Loughran.
2021-02-10 10:28:59 +00:00
9434c1eccc
HDFS-15818. Fix TestFsDatasetImpl.testReadLockCanBeDisabledByConfig. Contributed by Leon Gao ( #2679 )
2021-02-09 10:49:28 +00:00
2df2dfb9ed
HDFS-15817. Rename snapshots while marking them deleted. ( #2677 )
2021-02-09 10:27:51 +05:30
0e2b3086e3
HDFS-15683. Allow configuring DISK/ARCHIVE capacity for individual volumes. ( #2625 )
2021-02-08 11:01:21 -08:00
19ae0faacc
HDFS-15820. Ensure snapshot root trash provisioning happens only post safe mode exit ( #2682 )
2021-02-06 08:26:12 -08:00
c4918fb298
[HDFS-15813] DataStreamer: keep sending heartbeat packets during flush. Contributed by Daryn Sharp and Jim Brennan
2021-02-05 21:20:30 +00:00
7ef2875c89
HDFS-15798. EC: Reconstruct task failed, and It would be XmitsInProgress of DN has negative number. Contributed by huhaiyang
2021-02-05 11:01:49 +08:00
b99dba38b9
HDFS-15819. Fix import style #2681
2021-02-05 10:11:27 +08:00
15a1f7adfc
HDFS-15779. EC: fix NPE caused by StripedWriter.clearBuffers during reconstruct block. Contributed by Hongbing Wang
2021-02-04 09:19:51 +08:00
182623d2bc
HDFS-15799. Make DisallowedDatanodeException terse. Contributed by Richard Ross.
2021-02-03 08:55:50 -06:00
66ecee333e
HDFS-15803. EC: Remove unnecessary method (getWeight) in StripedReconstructionInfo. Contributed by huhaiyang
2021-02-03 20:00:45 +08:00
394b9f7a5c
HDFS-15624. fix the function of setting quota by storage type ( #2377 )
...
1. puts NVDIMM to the end of storage type enum to make sure compatibility.
2. adds check to make sure the software layout version is satisfied
Co-authored-by: su xu <kevinbrandon@163.com>
Co-authored-by: huangtianhua <huangtianhua223@gmail.com>
Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Vinayakumar B <vinayakumarb@apache.org>
2021-02-02 22:44:34 -08:00
18978f2e20
HDFS-15795. EC: Wrong checksum when reconstruction was failed by exception. Contributed by Yushi Hayasaka ( #2657 )
2021-02-02 09:02:09 +00:00
1b893e10ae
HDFS-15792. Addendum: ClasscastException while loading FSImage. Contributed by Renukaprasad C.
2021-02-02 13:50:54 +08:00
21a3fc3d2d
HADOOP-17482: Remove Commons Logger from FileSystem Class ( #2633 )
2021-02-01 09:40:01 -08:00
b54134661b
HDFS-15792. ClasscastException while loading FSImage. Contributed by Renukaprasad C.
2021-02-01 23:28:29 +08:00
115623a6ee
HDFS-15791. Possible Resource Leak in FSImageFormatProtobuf. ( #2652 )
2021-02-01 01:56:59 -08:00
1a205cc3ad
HADOOP-17424. Replace HTrace with No-Op tracer ( #2645 )
2021-02-01 13:42:44 +09:00
ad483fd66e
HDFS-15740. Add x-platform utilities ( #2567 )
2021-01-29 10:11:38 -08:00
f565476939
[HDFS-15789] Lease renewal does not require namesystem lock. Contributed by Daryn Sharp and Jim Brennan.
2021-01-28 18:17:15 +00:00
6184ab07f6
[HDFS-10498] Intermittent test failure TestSnapshotFileLength.testSnapshotFileLength. Contributed by Jim Brennan.
2021-01-28 16:47:52 +00:00
f8769e0f4b
HDFS-15661. The DeadNodeDetector should not be shared by different DFSClients. Contributed by Jinglun.
2021-01-28 10:10:39 +08:00
e40f99f6d5
HDFS-15767. RBF: Router federation rename of directory. Contributed by Jinglun.
2021-01-26 14:25:27 +05:30
45b3a84709
HDFS-15783. Speed up BlockPlacementPolicyRackFaultTolerant#verifyBlockPlacement ( #2626 )
2021-01-21 10:07:46 +09:00
97f843de3a
HADOOP-16947. Stale record should be remove when MutableRollingAverages generating aggregate data. Contributed by Haibin Huang.
2021-01-16 14:06:56 +08:00
7743d40ac5
HDFS-15549. Use Hardlink to move replica between DISK and ARCHIVE storage if on same filesystem mount ( #2583 ). Contributed by Leon Gao.
2021-01-15 16:28:11 -08:00
630f8ddd2c
HDFS-15672. TestBalancerWithMultipleNameNodes#testBalancingBlockpoolsWithBlockPoolPolicy fails on trunk. ( #2475 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-01-14 16:19:35 +09:00
8ec824f2ba
HDFS-15762. TestMultipleNNPortQOP#testMultipleNNPortOverwriteDownStream fails intermittently ( #2598 )
...
Co-authored-by: Toshihiko Uchida <toshihiko.uchida@linecorp.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2021-01-13 11:23:07 +09:00
0d7ac54510
HDFS-14558. RBF: Isolation/Fairness documentation. Contributed by Fengnan Li.
2021-01-12 23:38:55 +08:00
32abc0af49
HDFS-15702. Fix intermittent falilure of TestDecommission#testAllocAndIBRWhileDecommission. ( #2507 )
2021-01-11 11:59:32 -08:00
ec22850dbe
Make upstream aware of 3.2.2 release.
2021-01-09 18:06:08 +08:00
77435a025e
HADOOP-17408. Optimize NetworkTopology sorting block locations. ( #2601 ). Contributed by Ahmed Hussein and Daryn Sharp.
2021-01-08 13:10:09 -06:00
2ba7ec2b48
HDFS-15766. RBF: MockResolver.getMountPoints() breaks the semantic of FileSubclusterResolver. Contributed by Jinglun.
2021-01-08 16:27:54 +05:30
87bd4d2aca
HDFS-15754. Add DataNode packet metrics ( #2578 )
...
Contributed by Fengnan Li.
2021-01-07 23:46:23 -08:00
2b4febcf57
HDFS-15719. [Hadoop 3] Both NameNodes can crash simultaneously due to the short JN socket timeout ( #2533 )
2021-01-04 20:54:44 -08:00
77299ae992
HDFS-15748. RBF: Move the router related part from hadoop-federation-balance module to hadoop-hdfs-rbf. Contributed by Jinglun.
2021-01-05 00:05:03 +05:30
66ee0a6df0
HADOOP-17371. Bump Jetty to the latest version 9.4.34. Contributed by Wei-Chiu Chuang. ( #2453 )
2021-01-04 09:43:58 -08:00
2825d060cf
HDFS-15648. TestFileChecksum should be parameterized. ( #2501 )
2021-01-04 14:00:57 +09:00
1cd96e8dd8
HDFS-15745. Make DataNodePeerMetrics#LOW_THRESHOLD_MS and MIN_OUTLIER_DETECTION_NODES configurable. Contributed by Haibin Huang.
2021-01-01 11:37:52 +05:30
3b77cf4446
HDFS-15749. Make size of editPendingQ can be configurable ( #2572 )
2020-12-28 14:18:18 +08:00
16a20503ca
HDFS-15569. Speed up the Storage#doRecover during datanode rolling upgrade. Contributed by Hemanth Boyina.
2020-12-22 17:27:31 +05:30
4ae561bcb4
HDFS-15739. Add missing Javadoc for a param in method chooseRandomWithStorageType ( #2566 )
2020-12-22 09:25:11 +08:00
ba496f61f8
HDFS-15308. TestReconstructStripedFile#testNNSendsErasureCodingTasks fails intermittently. Contributed by Hemanth Boyina.
2020-12-22 02:16:00 +05:30
5ab1db9e21
HDFS-15380. RBF: Could not fetch real remote IP in RouterWebHdfsMethods. Contributed by tomscut.
2020-12-21 09:58:30 +05:30
2aea43bf4f
HDFS-15655. Add option to make balancer prefer to get cold blocks. Contributed by Yang Yun.
2020-12-20 23:06:19 +05:30
3ecd3628d4
HDFS-15733. Add seqno in log when BlockReceiver receive packet. Contributed by Haibin Huang.
2020-12-20 15:18:25 +05:30
70789fc6b9
HDFS-15116. Correct spelling of comments for NNStorage.setRestoreFailedStorage. Contributed by Xudong Cao.
2020-12-20 15:12:32 +05:30
c2672bb234
HDFS-15704. Mitigate lease monitor's rapid infinite loop. ( #2511 ). Contributed by Daryn Sharp and Ahmed Hussein
2020-12-17 11:33:29 -06:00
0da00fc743
HDFS-15170. EC: Block gets marked as CORRUPT in case of failover and pipeline recovery. Contributed by Ayush Saxena.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2020-12-14 20:42:23 -08:00
3363cc380b
HDFS-15728. Update description of dfs.datanode.handler.count in hdfs-default.xml. Contributed by liuyan.( #2534 )
2020-12-15 10:06:11 +05:30
9ed737001c
HDFS-15725. Lease Recovery never completes for a committed block which the DNs never finalize. Contributed by Stephen O'Donnell
2020-12-11 20:53:10 +00:00
be35fa186c
HDFS-15717. Improve fsck logging. ( #2529 ) Contributed by Kihwal Lee and Ahmed Hussein
2020-12-11 11:02:42 -06:00
9bd3c9bc50
HDFS-15720 namenode audit async logger should add some log4j config ( #2532 )
2020-12-10 13:47:00 -08:00
3ec01b1bb3
HDFS-15711. Add Metrics to HttpFS Server. ( #2521 ) Contributed by Ahmed Hussein and Kihwal Lee
2020-12-10 13:09:53 -06:00
01383a2172
HDFS-15716. WaitforReplication in TestUpgradeDomainBlockPlacementPolicy ( #2528 )
2020-12-08 13:25:24 -06:00
4ffec79b7c
HDFS-15712. Upgrade googletest to 1.10.0 ( #2523 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2020-12-08 17:08:14 +09:00
40f7543a6d
HDFS-15709. Socket file descriptor leak in StripedBlockChecksumReconstructor. ( #2518 )
2020-12-07 15:49:45 -08:00
32099e36dd
HDFS-15707. NNTop counts don't add up as expected. ( #2516 )
2020-12-07 14:22:06 -06:00
da1ea2530f
HDFS-15660. StorageTypeProto is not compatiable between 3.x and 2.6. Contributed by Ryan Wu.
2020-12-07 18:52:12 +08:00
ad40715690
HDFS-15221. Add checking of effective filesystem during initializing storage locations. Contributed by Yang Yun.
2020-12-07 15:00:19 +05:30
7dda804a1a
HDFS-14090. RBF: Improved isolation for downstream name nodes. {Static}. Contributed by Fengnan Li and CR Hota.
2020-12-04 22:56:37 +05:30
e2c1268ebd
HDFS-15240. Erasure Coding: dirty buffer causes reconstruction block error. Contributed by HuangTao.
2020-12-04 09:20:09 +08:00
07655a7040
HDFS-15706. HttpFS: Log more information on request failures. ( #2515 )
2020-12-03 15:14:40 -08:00
42a29199c0
HDFS-15705. Fix a typo in SecondaryNameNode.java. Contributed by Sixiang Ma.
2020-12-02 21:32:02 -08:00
6ff2409b31
HDFS-14904. Add Option to let Balancer prefer highly utilized nodes in each iteration ( #2483 ). Contributed by Leon Gao.
2020-12-02 15:53:09 -08:00
60201cbf69
HDFS-15703. Don't generate edits for set operations that are no-op ( #2508 ). Contributed by Daryn Sharp and Ahmed Hussein
2020-12-02 15:38:20 -06:00
2b5b556dd7
HDFS-15695. NN should not let the balancer run in safemode ( #2489 ). Contributed by Daryn Sharp and Ahmed Hussein
2020-12-02 13:59:00 -06:00
918ba9e82f
HDFS-15694. Avoid calling UpdateHeartBeatState inside DataNodeDescriptor. ( #2487 ) Contributed by Kuhu Shukla and Ahmed Hussein
2020-12-01 15:02:53 -06:00
6a1d7d9ed2
HDFS-15677. TestRouterRpcMultiDestination#testGetCachedDatanodeReport fails on trunk. ( #2503 )
2020-12-01 08:06:52 +09:00
68442b48c8
HDFS-15698. Fix the typo of dfshealth.html after HDFS-15358 ( #2495 )
2020-11-28 12:19:57 +08:00
2ce2198287
HADOOP-17394. [JDK 11] Fix error in mvn package -Pdocs ( #2488 )
...
Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>
2020-11-26 11:32:10 +09:00
65002c9486
Revert "[JDK 11] Fix error in mvn package -Pdocs ( #2488 )" because JIRA ID is missing.
...
This reverts commit ce5b3d7975
2020-11-26 11:30:53 +09:00
ce5b3d7975
[JDK 11] Fix error in mvn package -Pdocs ( #2488 )
...
Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>
2020-11-26 11:27:30 +09:00
235947e282
HDFS-15689. allow/disallowSnapshot on EZ roots shouldn't fail due to trash provisioning/emptiness check ( #2472 )
2020-11-25 11:01:04 -08:00
641d8856d2
HDFS-15684. EC: Call recoverLease on DFSStripedOutputStream close exception. Contributed by Hongbing Wang.
...
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Reviewed-by: Hui Fei <ferhui@apache.org>
2020-11-23 11:26:52 +08:00
d73029463c
HDFS-15690. Add lz4-java as test dependency ( #2481 )
2020-11-21 17:49:56 -08:00
747883af4d
HDFS-15659. MiniDFSCluster dfs.namenode.redundancy.considerLoad default to false ( #2443 ). Contributed by Ahmed Hussein.
2020-11-20 18:03:45 +05:30
425996eb4a
HDFS-15674. TestBPOfferService#testMissBlocksWhenReregister fails on trunk. ( #2467 )
2020-11-18 16:11:09 +09:00
b57f04cd5b
HDFS-15685. [JDK 14] TestConfiguredFailoverProxyProvider#testResolveDomainNameUsingDNS fails. ( #2465 )
2020-11-17 10:56:56 +09:00
f56cd88d7d
HDFS-15538. Fix the documentation for dfs.namenode.replication.max-streams in hdfs-default.xml. Contributed by Xieming Li.
2020-11-13 11:02:26 +09:00
f5e6be337b
HDFS-15545 - Allow WebHdfsFileSystem to read a new delegation token f… ( #2255 )
...
Co-authored-by: Issac Buenrostro <ibuenros@linkedin.com>
2020-11-12 14:57:26 -08:00
cd0490e8c6
Revert "YARN-10425. Replace the legacy placement engine in CS with the new one. Contributed by Gergely Pollak."
...
This reverts commit b0ab222a6c
2020-11-11 17:06:12 +01:00
b0ab222a6c
YARN-10425. Replace the legacy placement engine in CS with the new one. Contributed by Gergely Pollak.
2020-11-11 17:03:16 +01:00
e6d2dccbef
HDFS-15469. Dynamically configure the size of PacketReceiver#MAX_PACKET_SIZE. ( #2138 )
2020-11-10 16:34:17 -08:00
fbd2220167
HDFS-15607. Addendum: Create trash dir when allowing snapshottable dir ( #2448 )
2020-11-10 12:13:15 -08:00
82de07c794
HADOOP-17142. Fix outdated properties of JournalNode when performing rollback. Contributed by Deegue.
2020-11-10 09:11:31 -08:00
f1d5652359
HDFS-15668. RBF: Fix RouterRPCMetrics annocation and document misplaced error. Contributed by Hongbing Wang.
2020-11-10 13:09:03 +08:00
95c96605b3
HDFS-15667. Audit log record the unexpected allowed result when delete ( #2437 )
2020-11-10 13:01:10 +08:00
9a9ab5b48e
HDFS-15548. Allow configuring DISK/ARCHIVE storage types on same device mount ( #2288 ). Contributed by Leon Gao.
2020-11-09 15:06:16 -08:00
3067a25fa1
HDFS-15651. Client could not obtain block when DN CommandProcessingThread exit. Contributed by Aiphago.
...
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Reviewed-by: Yiqun Lin <yqlin@apache.org>
2020-11-04 13:53:46 +08:00
7ac7840fc3
HDFS-15643. EC: Fix checksum computation in case of native encoders. ( #2424 ). Contributed by Ayush Saxena.
2020-11-04 09:28:44 +05:30
d07dc7afb4
HDFS-15665. Balancer logging improvements. Contributed by Konstantin V Shvachko.
2020-11-03 12:02:40 -08:00
324879127a
HDFS-15654. TestBPOfferService#testMissBlocksWhenReregister fails intermittently ( #2419 )
2020-10-28 16:24:34 -07:00
b3ba74d72d
HDFS-15657. RBF: TestRouter#testNamenodeHeartBeatEnableDefault fails by BindException ( #2418 )
2020-10-28 19:12:23 +09:00
d0c786db4d
HDFS-15460. TestFileCreation#testServerDefaultsWithMinimalCaching fails intermittently. ( #2406 )
2020-10-27 18:56:40 -07:00
98097b8f19
HDFS-15457. TestFsDatasetImpl fails intermittently ( #2407 )
2020-10-27 18:52:56 -07:00
8a6d5b9151
HDFS-15652. Make block size from NNThroughputBenchmark configurable ( #2416 )
2020-10-28 09:13:25 +08:00
ae74407ac4
HDFS-9776. testMultipleAppendsDuringCatchupTailing is flaky ( #2410 )
...
Co-authored-by: ahussein <ahmed.hussein@verizonmedia.com>
2020-10-27 20:45:00 +09:00
3848b44b37
HDFS-15580. [JDK 12] DFSTestUtil#addDataNodeLayoutVersion fails ( #2309 )
2020-10-27 19:18:08 +09:00
afaab3d332
HDFS-15461. TestDFSClientRetries testGetFileChecksum fails ( #2404 )
...
Co-authored-by: ahussein <ahmed.hussein@verizonmedia.com>
2020-10-27 16:56:30 +09:00
12c908c827
HDFS-15641. DataNode could meet deadlock if invoke refreshNameNode. Contributed by Hongbing Wang.
2020-10-26 22:12:27 +08:00
74634eb002
HDFS-15644. Failed volumes can cause DNs to stop block reporting. Contributed by Ahmed Hussein.
2020-10-23 12:05:53 -07:00
e7aa4da0a2
HDFS-15459. TestBlockTokenWithDFSStriped fails intermittently. Contributed by Ahmed Hussein.
2020-10-23 09:47:10 -07:00
264c948e60
HDFS-15630. RBF: Fix wrong client IP info in CallerContext when requests mount points with multi-destinations. Contributed by Chengwei Wang
2020-10-23 14:06:22 +08:00
da1b6e3cc2
HDFS-15622. Deleted blocks linger in the replications queue. Contributed by Ahmed Hussein.
2020-10-22 16:59:49 -05:00
88a9f42f32
HDFS-15618. Improve datanode shutdown latency. Contributed by Ahmed Hussein.
2020-10-21 00:59:35 -05:00
30f06e0c74
HDFS-15639. [JDK 11] Fix Javadoc errors in hadoop-hdfs-client. ( #2394 )
2020-10-20 19:08:59 +09:00
2e8cafac3b
HDFS-14383. Compute datanode load based on StoragePolicy. Contributed by Ayush Saxena.
2020-10-19 10:48:47 +05:30
173310e2f5
HDFS-15629. Add seqno when warning slow mirror/disk in BlockReceiver. Contributed by Haibin Huang.
2020-10-19 10:34:17 +05:30
1e3a6efcef
HADOOP-17288. Use shaded guava from thirdparty. ( #2342 ). Contributed by Ayush Saxena.
2020-10-17 12:01:18 +05:30
740a2c4635
HDFS-15627. Log delete audits before collecting blocks.
2020-10-16 11:13:41 -05:00
cc57eebe45
HDFS-15633. Avoid redundant RPC calls for getDiskStatus. ( #2386 ). Contributed by Ayush Saxena.
2020-10-16 10:36:24 +05:30
e45407128d
HDFS-15628. HttpFS server throws NPE if a file is a symlink. Contributed by Ahmed Hussein.
2020-10-14 17:26:04 -05:00
a308a1ec22
HDFS-15614. Initialize snapshot trash root during NameNode startup if enabled ( #2370 )
2020-10-13 10:59:42 -07:00
59f01a548e
HADOOP-16878. FileUtil.copy() to throw IOException if the source and destination are the same
...
Contributed by Gabor Bota.
2020-10-13 16:17:44 +01:00
0507c4160f
HDFS-15626. TestWebHDFS.testLargeDirectory failing ( #2380 )
...
Fixes the regression caused by HADOOP-17281, where the WebHDFS client
listStatusIterator (correctly) throws NoSuchElementException when next()
runs out of values.
Contributed by Mukund Thakur.
Change-Id: I6cec41c20467920cf21f169653553535414b2680
2020-10-13 13:31:00 +01:00
d606ddc12c
Revert "HADOOP-17303. TestWebHDFS.testLargeDirectory failing ( #2380 )"
...
(reverting to change the JIRA number)
This reverts commit 3e1b1dfd77
2020-10-13 13:30:38 +01:00
3e1b1dfd77
HADOOP-17303. TestWebHDFS.testLargeDirectory failing ( #2380 )
...
Fixes the regression caused by HADOOP-17281, where the WebHDFS client
listStatusIterator (correctly) throws NoSuchElementException when next()
runs out of values.
Contributed by Mukund Thakur.
2020-10-13 13:27:55 +01:00
b3786d6c3c
HDFS-15567. [SBN Read] HDFS should expose msync() API to allow downstream applications call it explicitly. Contributed by Konstantin V Shvachko.
2020-10-12 17:26:49 -07:00
b76b36ebbc
HDFS-15625: Namenode trashEmptier should not init ViewFs on startup ( #2378 ). Contributed by Uma Maheswara Rao G.
2020-10-12 14:22:44 -07:00
69ef9b1ee8
HDFS-15620. RBF: Fix test failures after HADOOP-17281 ( #2375 )
2020-10-12 09:52:12 +09:00
de055ec8aa
HDFS-15456. TestExternalStoragePolicySatisfier fails intermittently ( #2299 )
2020-10-10 22:08:36 +05:30
518a212cff
HDFS-13293. RBF: The RouterRPCServer should transfer client IP via CallerContext to NamenodeRpcServer ( #2363 )
2020-10-09 16:12:22 +08:00
735e85a1d2
HDFS-15610 Reduced datanode upgrade/hardlink thread from 12 to 6 ( #2365 )
2020-10-08 12:51:04 +05:30
df4006eb81
HDFS-15253 Default checkpoint transfer speed, 50mb per second ( #2366 )
2020-10-07 22:09:04 +05:30
16aea11c94
HDFS-15611. Add list Snapshot command in WebHDFS. ( #2355 )
2020-10-07 10:34:32 +05:30
074f0d46af
HDFS-15613. RBF: Router FSCK fails after HDFS-14442. ( #2360 )
...
* Support getHAServiceState in DFSRouter
2020-10-07 13:37:45 +09:00
921ca1f554
HDFS-15543. RBF: Write Should allow, when a subcluster is unavailable for RANDOM mount points with fault Tolerance enabled. Contributed by Hemanth Boyina.
2020-10-07 09:58:53 +05:30
1cfe5916e2
HADOOP-16990. Update Mockserver. Contributed by Attila Doroszlai.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2020-10-06 14:58:42 -07:00
6ece6407e1
HDFS-15458. TestNameNodeRetryCacheMetrics fails intermittently ( #2356 ). Contributed by Hui Fei.
2020-10-05 22:11:24 +05:30
43b0c0b054
HDFS-15607. Create trash dir when allowing snapshottable dir ( #2352 )
2020-10-05 05:02:00 -07:00
a89ca56a1b
HDFS-15600 Fix TestRouterQuota#testStorageTypeQuota ( #2347 )
2020-09-29 19:00:55 -07:00
d686a655bc
HDFS-15603. RBF: Fix getLocationsForPath twice in create operation. Contributed by wangzhaohui.
2020-09-29 00:03:11 +05:30
fc8a6dd8a9
HDFS-15591. RBF: Fix webHdfs file display error. Contributed by wangzhaohui.
2020-09-28 23:55:31 +05:30
984875c624
HDFS-15604. Fix Typo for HdfsDataNodeAdminGuide doc ( #2345 )
2020-09-28 09:54:01 -07:00
eae0035a2d
HDFS-15530. RBF: Fix typo in DFS_ROUTER_QUOTA_CACHE_UPDATE_INTERVAL var definition. Contributed by Sha Fanghao.
2020-09-28 09:40:39 -07:00
d89af79507
HDFS-15577. Refactor TestTracing. ( #2302 )
...
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
2020-09-28 18:27:41 +09:00
82b86e3754
HDFS-15098. Add SM4 encryption method for HDFS. Contributed by liusheng
2020-09-27 19:27:13 +05:30
899dea2a21
HDFS-15598: ViewHDFS#canonicalizeUri should not be restricted to DFS only API. ( #2339 ). Contributed by Uma Maheswara Rao G.
...
Co-authored-by: Uma Maheswara Rao G <umagangumalla@cloudera.com>
2020-09-25 21:21:01 -07:00
00c4de63cf
HDFS-15594. Lazy calculate live datanodes in safe mode tip ( #2332 )
2020-09-25 09:47:54 -07:00
3ccc962b99
HDFS-15596: ViewHDFS#create(f, permission, cflags, bufferSize, replication, blockSize, progress, checksumOpt) should not be restricted to DFS only. ( #2333 ). Contributed by Uma Maheswara Rao G.
...
Co-authored-by: Uma Maheswara Rao G <umagangumalla@cloudera.com>
2020-09-24 07:07:48 -07:00
486ddb73f9
HADOOP-17283. Hadoop - Upgrade to jQuery 3.5.1 ( #2330 )
...
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2020-09-24 21:15:55 +09:00
ff59fbb8b0
HDFS-15025. Applying NVDIMM storage media to HDFS ( #2189 )
...
* HDFS-15025. Applying NVDIMM storage media to HDFS
Co-authored-by: YaYun-Wang <yywangyayun@163.com>
Co-authored-by: YaYun-Wang <34060507+YaYun-Wang@users.noreply.github.com>
2020-09-24 14:27:04 +05:30
368f2f637e
HDFS-15590. namenode fails to start when ordered snapshot deletion feature is disabled ( #2326 )
2020-09-24 14:00:41 +05:30
ae089f2db7
HDFS-15557. Log the reason why a storage log file can't be deleted ( #2274 )
2020-09-22 13:23:20 -07:00
474fa80bfb
HADOOP-17277. Correct spelling errors for separator ( #2322 )
...
Contributed by Hui Fei.
2020-09-22 17:10:33 +01:00
dfc2682213
HDFS-15581. Access Controlled HttpFS Server. Contributed by Richard Ross.
2020-09-22 10:53:04 -05:00
3e8b1e7426
HDFS-15554. RBF: force router check file existence in destinations before adding/updating mount points ( #2266 ). Contributed by Fengnan Li.
2020-09-21 23:24:13 +05:30
7a6265ac42
HDFS-14811. RBF: TestRouterRpc#testErasureCoding is flaky. Contributed by Chen Zhang.
2020-09-21 08:50:08 +05:30
8eba922bb3
HDFS-15579. RBF: The constructor of PathLocation may got some misunderstanding. Contributed by Janus Chow.
2020-09-20 08:54:25 +05:30
2377df38ad
HDFS-15438. Setting dfs.disk.balancer.max.disk.errors = 0 will fail the block copy. Contributed by AMC-team.
2020-09-18 17:28:11 +05:30
7bba4c609c
HDFS-15585: ViewDFS#getDelegationToken should not throw UnsupportedOperationException. ( #2312 ). Contributed by Uma Maheswara Rao G.
2020-09-18 15:18:10 +05:30
20a0e6278d
HDFS-15415. Reduce locking in Datanode DirectoryScanner. Contributed by Stephen O'Donnell
2020-09-17 14:11:42 +01:00
425f48799c
HDFS-15568. namenode start failed to start when dfs.namenode.max.snapshot.limit set. ( #2296 )
2020-09-17 14:50:08 +05:30
e4cb0d3514
HDFS-15578: Fix the rename issues with fallback fs enabled ( #2305 ). Contributed by Uma Maheswara Rao G.
...
Co-authored-by: Uma Maheswara Rao G <umagangumalla@cloudera.com>
2020-09-16 22:43:00 -07:00
5c5b2ed7c7
HDFS-15576. Erasure Coding: Add rs and rs-legacy codec test for addPolicies. Contributed by Fei Hui.
2020-09-16 01:36:46 +09:00
f4ed9f3f91
HDFS-15574. Remove unnecessary sort of block list in DirectoryScanner. Contributed by Stephen O'Donnell.
2020-09-15 10:10:21 +05:30
924959088e
HDFS-15559. Complement initialize member variables in TestHdfsConfigFields#initializeMemberVariables. Contributed by Lisheng Sun.
2020-09-15 12:05:04 +08:00
6d3312de47
HDFS-15539. When disallowing snapshot on a dir, throw exception if its trash root is not empty ( #2258 )
2020-09-14 13:31:34 -07:00
c78d18023d
HDFS-15555. RBF: Refresh cacheNS when SocketException occurs. ( #2267 )
2020-09-14 11:34:52 +09:00
d2779de3f5
HDFS-15532: listFiles on root/InternalDir will fail if fallback root has file. ( #2298 ). Contributed by Uma Maheswara Rao G.
2020-09-12 17:06:39 -07:00
f59f7f2175
HDFS-15573. Only log warning if considerLoad and considerStorageType are both true. Contributed by Stephen O'Donnell
2020-09-12 01:41:38 -07:00
2a6fdcedeb
HDFS-15564. Add Test annotation for TestPersistBlocks#testRestartDfsWithSync. Contributed by Fei Hui.
2020-09-11 18:21:25 +05:30
89428f142f
HDFS-15551. Tiny Improve for DeadNode detector ( #2265 )
...
Contributed by imbajin.
Reviewed-by: leosunli <lisheng.sun08@gmail.com>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2020-09-11 15:44:03 +08:00
56ebabd426
HADOOP-17222. Create socket address leveraging URI cache ( #2241 )
...
Contributed by fanrui.
Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2020-09-10 22:30:52 -07:00
12a316cdf9
HADOOP-15891. provide Regex Based Mount Point In Inode Tree ( #2185 ). Contributed by Zhenzhao Wang.
...
Co-authored-by: Zhenzhao Wang <zhenzhaowang@gmail.com>
2020-09-10 21:20:32 -07:00
43c52d6495
HDFS-15563. Incorrect getTrashRoot return value when a non-snapshottable dir prefix matches the path of a snapshottable dir ( #2295 )
...
Change-Id: Iefcbc14ca939b61f0c1befc9a94d686be1684ddc
2020-09-10 15:55:23 +05:30
1d6d0d8207
HDFS-14694. Call recoverLease on DFSOutputStream close exception. Contributed by Lisheng Sun.
...
Co-authored-by: Chen Zhang <chzhang1987@gmail.com>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
2020-09-09 20:09:13 +08:00
ac7d4623ae
HDFS-15558: ViewDistributedFileSystem#recoverLease should call super.recoverLease when there are no mounts configured ( #2275 ) Contributed by Uma Maheswara Rao G.
2020-09-07 11:36:13 -07:00
b3660d0147
HDFS-15529: getChildFilesystems should include fallback fs as well ( #2234 ). Contributed by Uma Maheswara Rao G.
2020-09-03 11:06:20 -07:00
60de592a88
HDFS-15550. Remove unused imports from TestFileTruncate.java. Contributed by Ravuri Sushma sree.
2020-08-31 14:49:12 +08:00
2d03209a06
HDFS-15542. Add identified snapshot corruption tests for ordered snapshot deletion ( #2251 )
2020-08-31 11:29:48 +05:30
c392d9022a
HDFS-15471. TestHDFSContractMultipartUploader failing ( #2252 )
...
Contributed by Steve Loughran
(Was: broken by Steve Loughran)
2020-08-28 15:39:09 +01:00
06793da100
HDFS-15531. Namenode UI: List snapshots in separate table for each snapshottable directory ( #2230 )
2020-08-27 11:36:30 -07:00
41182a9b6d
HDFS-15500. In-order deletion of snapshots: Diff lists must be update only in the last snapshot. ( #2233 )
2020-08-27 14:54:52 +05:30
2ffe00fc46
HDFS-15540. Directories protected from delete can still be moved to the trash. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2020-08-26 23:05:32 -07:00
ca8e7a7725
HDFS-15510. RBF: Quota and Content Summary was not correct in Multiple Destinations. Contributed by Hemanth Boyina.
2020-08-27 12:10:39 +09:00
6e618b6a7e
HDFS-15536. RBF: Clear Quota in Router was not consistent.
2020-08-26 13:03:08 +05:30
6a49bf9bff
HDFS-14852. Removing from LowRedundancyBlocks does not remove the block from all queues. Contributed by Fei Hui.
2020-08-25 15:18:36 +01:00
a7830423c5
HDFS-15448. Remove duplicate BlockPoolManager starting when run DataNode. Contriubted by jianghua zhu.
2020-08-24 21:16:00 +08:00
83fd15b412
HDFS-14504. Rename with Snapshots does not honor quota limit.
2020-08-23 16:10:47 +05:30
f734455e5d
HDFS-15290. NPE in HttpServer during NameNode startup. Contributed by Simbarashe Dzinamarira.
2020-08-20 15:25:10 -07:00
4813a37023
HDFS-15535. RBF: Fix Namespace path to snapshot path resolution for snapshot API. Contributed by Ayush Saxena.
2020-08-20 23:54:35 +05:30
dd013f2fdf
HDFS-15533: Provide DFS API compatible class, but use ViewFileSystemOverloadScheme inside. ( #2229 ). Contributed by Uma Maheswara Rao G.
2020-08-19 09:30:41 -07:00
b65e43fe38
HDFS-15525. Make trash root inside each snapshottable directory for WebHDFS ( #2220 )
2020-08-18 19:28:19 +09:00
092bfe7c8e
HDFS-15483. Ordered snapshot deletion: Disallow rename between two snapshottable directories. ( #2172 )
...
* HDFS-15483. Ordered snapshot deletion: Disallow rename between two snapshottable directories.
* Addressed review comments.
* Rebased to latest trunk and added snapshotTrashRoot config check.
* Addressed review comments.
* Removede empty line added in SnapshotManager.Java.
* Addressed whitespace issues.
2020-08-17 12:56:13 +05:30
e3d1966f58
HDFS-15439. Setting dfs.mover.retry.max.attempts to negative value will retry forever. Contributed by AMC-team.
2020-08-15 15:07:58 +05:30
b93dd7c281
HDFS-15519. Check inaccessible INodes in FsImageValidation. ( #2224 )
2020-08-14 10:10:01 -07:00
15a76e8d65
HDFS-15524. Add edit log entry for Snapshot deletion GC thread snapshot deletion. ( #2219 )
2020-08-14 20:46:48 +05:30
cb50e3fcf7
HDFS-15496. Add UI for deleted snapshots ( #2212 )
2020-08-13 22:36:15 +05:30
e592ec5f8b
HDFS-15518. Wrong operation name in FsNamesystem for listSnapshots. ( #2217 )
...
* HDFS-15518. Fixed String operationName = ListSnapshot.
* HDFS-15518. Changed ListSnapshot to operationName.
Co-authored-by: Aryan Gupta <aryan.gupta@cloudera.com>
2020-08-13 00:09:38 +05:30
10716040a8
HDFS-15493. Update block map and name cache in parallel while loading fsimage. Contributed by Chengwei Wang
2020-08-12 09:02:47 +01:00
11cec9ab94
HDFS-15523. Fix findbugs warnings from HDFS-15520. ( #2218 )
2020-08-11 21:41:28 -07:00
3fd3aeb621
HDFS-15492. Make trash root inside each snapshottable directory ( #2176 )
2020-08-11 08:52:16 -07:00
909f1e82d3
HADOOP-17196. Fix C/C++ standard warnings ( #2208 )
...
* Passing C/C++ standard flags -std is
not cross-compiler friendly as not all
compilers support all values.
* Thus, we need to make use of the
appropriate flags provided by CMake in
order to specify the C/C++ standards.
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2020-08-11 16:35:08 +09:00
8955a6ceb7
HDFS-15515: mkdirs on fallback should throw IOE out instead of suppressing and returning false ( #2205 )
...
* HDFS-15515: mkdirs on fallback should throw IOE out instead of suppressing and returning false
* Used LambdaTestUtils#intercept in test
2020-08-11 00:01:58 -07:00
592127bdc2
HDFS-15520 Use visitor pattern to visit namespace tree ( #2203 )
2020-08-11 12:30:30 +05:30
32895f4f7e
HDFS-15507. [JDK 11] Fix javadoc errors in hadoop-hdfs-client module. Contributed by Xieming Li.
2020-08-11 13:49:56 +09:00
7938ebfb9d
HDFS-15506. [JDK 11] Fix javadoc errors in hadoop-hdfs module. Contributed by Xieming Li.
2020-08-11 11:14:58 +09:00
9062814bab
HDFS-15508. [JDK 11] Fix javadoc errors in hadoop-hdfs-rbf module
2020-08-11 10:58:56 +09:00
c2a17659d1
HDFS-15443. Setting dfs.datanode.max.transfer.threads to a very small value can cause strange failure. Contributed by AMC-team.
2020-08-08 14:33:53 +05:30
975b6024dd
HDFS-15514. Remove useless dfs.webhdfs.enabled. Contributed by Fei Hui.
2020-08-07 22:19:17 +05:30
1d5ccc790b
HDFS-15512. Remove smallBufferSize in DFSClient. ( #2191 )
2020-08-06 22:47:12 +05:30
58def7cecb
HDFS-15499. Clean up httpfs/pom.xml to remove aws-java-sdk-s3 exclusion ( #2188 )
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2020-08-04 20:48:45 -07:00
e072d33327
HDFS-15497. Make snapshot limit on global as well per snapshot root directory configurable ( #2175 )
2020-08-04 14:10:29 +05:30
82f3ffcd64
HDFS-15503. File and directory permissions are not able to be modified from WebUI.
2020-08-03 23:21:01 +05:30
c40cbc57fa
HADOOP-17091. [JDK11] Fix Javadoc errors ( #2098 )
2020-08-03 10:46:51 +09:00
d8a2df25ad
HDFS-15498. Show snapshots deletion status in snapList cmd. ( #2181 )
2020-08-02 21:59:56 +05:30
528a799a78
HDFS-15229. Truncate info should be logged at INFO level. Contributed by Ravuri Sushma sree.
2020-08-02 12:02:37 +05:30
e756fe3590
HDFS-14950. fix missing libhdfspp lib in dist-package ( #1947 )
...
libhdfspp.{a,so} are missed in dist-package.
This patch fixed this by copying these libs to the right directory
Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>
2020-07-31 16:49:49 +09:00
05b3337a46
HDFS-15481. Ordered snapshot deletion: garbage collect deleted snapshots ( #2165 )
2020-07-30 23:06:51 +05:30
68287371cc
HDFS-15488. Add a command to list all snapshots for a snaphottable root with snapshot Ids. ( #2166 )
2020-07-29 21:33:25 +05:30
026dce5334
HDFS-15465. Support WebHDFS accesses to the data stored in secure Datanode through insecure Namenode. ( #2135 )
2020-07-27 09:55:11 -07:00
2d12496643
HDFS-15480. Ordered snapshot deletion: record snapshot deletion in XAttr ( #2163 )
2020-07-22 16:16:27 -07:00
ac9a07b51a
HDFS-15478: When Empty mount points, we are assigning fallback link to self. But it should not use full URI for target fs. ( #2160 ). Contributed by Uma Maheswara Rao G.
2020-07-21 23:29:10 -07:00
1b29c9bfee
HADOOP-17138. Fix spotbugs warnings surfaced after upgrade to 4.0.6. ( #2155 )
2020-07-22 13:40:20 +09:00
8b7695bb26
HDFS-15246. ArrayIndexOfboundsException in BlockManager CreateLocatedBlock. Contributed by Hemanth Boyina.
2020-07-21 09:00:07 -07:00
d57462f2da
HDFS-15479. Ordered snapshot deletion: make it a configurable feature ( #2156 )
2020-07-21 11:36:24 +05:30
d9441f95c3
HDFS-15470. Added more unit tests to validate rename behaviour across snapshots. Contributed by Shashikant Banerjee.
2020-07-21 11:13:05 +05:30
3833c616e0
HDFS-15404. ShellCommandFencer should expose info about source. Contributed by Chen Liang.
2020-07-20 12:49:58 -07:00
6cbd8854ee
HDFS-15381. Fix typos corrputBlocksFiles to corruptBlocksFiles. Contributed by bianqi.
2020-07-20 21:49:05 +05:30
2cec50cf16
HDFS-15463. Add a tool to validate FsImage. ( #2140 )
2020-07-20 11:44:30 +05:30
4101b0c0ed
HDFS-15476 Make AsyncStream executor private ( #2151 )
2020-07-19 23:26:54 +05:30
8a9a674ef1
HDFS-15198. RBF: Add test for MountTableRefresherService failed to refresh other router MountTableEntries in secure mode. Contributed by zhengchenyu.
2020-07-18 15:36:55 +05:30
6bcb24d269
HADOOP-17100. Replace Guava Supplier with Java8+ Supplier in Hadoop. Contributed by Ahmed Hussein.
2020-07-18 14:33:43 +05:30
1f71c4ae71
HADOOP-17099. Replace Guava Predicate with Java8+ Predicate
...
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
2020-07-15 11:39:06 -05:00
98fcffe93f
HADOOP-17101. Replace Guava Function with Java8+ Function
...
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
2020-07-15 09:53:18 -05:00
85d4718ed7
HDFS-15319. Fix INode#isInLatestSnapshot() API. Contributed by Shashikant Banerjee.
2020-07-15 10:32:25 +05:30
cce5a6f609
HDFS-15385 Upgrade boost library to 1.72 ( #2051 )
...
* Removed the asio-1.10.2 that was
checked in under libhdfspp/third_party
directory.
2020-07-15 13:39:48 +09:00
bdce75d737
HDFS-15371. Nonstandard characters exist in NameNode.java ( #2032 )
...
Contributed by zhaoyim
2020-07-14 14:42:12 +01:00
b9fa5e0182
HDFS-13934. Multipart uploaders to be created through FileSystem/FileContext.
...
Contributed by Steve Loughran.
Change-Id: Iebd34140c1a0aa71f44a3f4d0fee85f6bdf123a3
2020-07-13 13:30:02 +01:00
b97fea65e7
HDFS-14498 LeaseManager can loop forever on the file for which create has failed. Contributed by Stephen O'Donnell.
2020-07-13 14:12:48 +08:00
84b74b335c
HDFS-15447 RBF: Add top real owners metrics for delegation tokens ( #2110 )
2020-07-12 00:10:12 -07:00
3e70006639
HDFS-15464: ViewFsOverloadScheme should work when -fs option pointing to remote cluster without mount links ( #2132 ). Contributed by Uma Maheswara Rao G.
2020-07-11 23:50:04 -07:00
f91a8ad88b
HADOOP-17079. Optimize UGI#getGroups by adding UGI#getGroupsSet. ( #2085 )
2020-07-09 11:33:37 -07:00
4f26454a7d
HDFS-15425. Review Logging of DFSClient. Contributed by Hongbing Wang.
2020-07-08 00:02:39 +08:00
f77bbc2123
HDFS-15312. Apply umask when creating directory by WebHDFS ( #2096 )
2020-07-06 19:01:46 -07:00
dc0626b5f2
HDFS-15449. Optionally ignore port number in mount-table name when picking from initialized uri. Contributed by Uma Maheswara Rao G.
2020-07-06 18:50:03 -07:00
e820baa6e6
HDFS-15417. RBF: Get the datanode report from cache for federation WebHDFS operations ( #2080 )
2020-07-06 16:17:09 -07:00
834372f404
HDFS-15451. Do not discard non-initial block report for provided storage. ( #2119 ). Contributed by Shanyu Zhao.
...
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2020-07-06 23:43:34 +08:00
55a2ae80dc
HDFS-15450. Fix NN trash emptier to work if ViewFSOveroadScheme enabled. Contributed by Uma Maheswara Rao G.
2020-07-04 13:45:49 -07:00
f86f15cf20
HDFS-15446. CreateSnapshotOp fails during edit log loading for /.reserved/raw/path with error java.io.FileNotFoundException: Directory does not exist: /.reserved/raw/path. Contributed by Stephen O'Donnell.
2020-07-04 13:00:43 +05:30
1f2a80b5e5
HDFS-15430. create should work when parent dir is internalDir and fallback configured. Contributed by Uma Maheswara Rao G.
2020-07-04 00:12:10 -07:00
9ac498e300
HDFS-15416. Improve DataStorage#addStorageLocations() for empty locations. Contibuted by jianghua zhu.
2020-07-01 12:38:07 +08:00
2a67e2b1a0
HDFS-15160. ReplicaMap, Disk Balancer, Directory Scanner and various FsDatasetImpl methods should use datanode readlock. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2020-06-30 07:10:17 -07:00
c71ce7ac33
HDFS-15421. IBR leak causes standby NN to be stuck in safe mode.
2020-06-28 16:02:47 +09:00
8db38c98a6
HDFS-15378. TestReconstructStripedFile#testErasureCodingWorkerXmitsWeight is failing on trunk. Contributed by hemanthboyina.
2020-06-27 19:13:01 +05:30
bed0a3a374
HDFS-15436. Default mount table name used by ViewFileSystem should be configurable ( #2100 )
...
* HDFS-15436. Default mount table name used by ViewFileSystem should be configurable
* Replace Constants.CONFIG_VIEWFS_DEFAULT_MOUNT_TABLE use in tests
* Address Uma's comments on PR#2100
* Sort lists in test to match without concern to order
* Address comments, fix checkstyle and fix failing tests
* Fix checkstyle
2020-06-26 13:19:16 -07:00
d5e1bb6155
HDFS-15429. mkdirs should work when parent dir is an internalDir and fallback configured. Contributed by Uma Maheswara Rao G.
2020-06-26 01:29:38 -07:00
84110d850e
HDFS-15383. RBF: Add support for router delegation token without watch ( #2047 )
...
Improving router's performance for delegation tokens related operations. It achieves the goal by removing watchers from router on tokens since based on our experience. The huge number of watches inside Zookeeper is degrading Zookeeper's performance pretty hard. The current limit is about 1.2-1.5 million.
2020-06-23 13:12:29 -07:00
7c02d1889b
HDFS-15427. Merged ListStatus with Fallback target filesystem and InternalDirViewFS. Contributed by Uma Maheswara Rao G.
2020-06-23 01:42:25 -07:00
17ffcab5f6
HDFS-14546. Document block placement policies. Contributed by Amithsha.
2020-06-21 17:27:28 +05:30
b27810aa60
HDFS-15418. ViewFileSystemOverloadScheme should represent mount links as non symlinks. Contributed by Uma Maheswara Rao G.
2020-06-20 00:32:02 -07:00
d50e93ce7b
HDFS-15372. Files in snapshots no longer see attribute provider permissions. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2020-06-18 06:45:15 -07:00
edf716a5c3
Revert "HDFS-15372. Files in snapshots no longer see attribute provider permissions. Contributed by Stephen O'Donnell."
...
This reverts commit 730a39d138
2020-06-18 06:45:15 -07:00
litao
He Xiaoqiao
Viraj Jasani
Narges Shadab
Takanobu Asanuma
touchida
Hui Fei
Gautham B A
sunlisheng
lamberken
Xiaoyu Yao
Akira Ajisaka
tomscut
litao
JJiaguodong
zhuqi
Konstantin V Shvachko
S O'Donnell
LeonGao
Ayush Saxena
Wang Yuxuan
Wei-Chiu Chuang
jianghuazhu
Zehao Chen
Steve Loughran
bshashikant
Jim Brennan
fermi
maobaolong
Kihwal Lee
YaYun-Wang
crossfire
belugabehr
Siyao Meng
Masatake Iwasaki
Yiqun Lin
Ahmed Hussein
lfengnan
maobaolong
hemanthboyina
langlaile1221
liuyanpunk
Neil
Liang-Chi Hsieh
Issac Buenrostro
Peter Bacsko
Yizhong Zhang
Inigo Goiri
Gabor Bota
Mukund Thakur
Uma Maheswara Rao G
Hui Fei
Karthik Palanisamy
Doroszlai, Attila
huangtianhua
Vinayakumar B
Ye Ni
Aryan Gupta
Mingliang Liu
imbajin
1996fanrui
zz
Vivek Ratnavel Subramanian
Tsz-Wo Nicholas Sze
Stephen O'Donnell
Chen Liang
Yuan
Suraj Naik
Ahmed Hussein
zhaoyim
Shanyu Zhao
Virajith Jalaparti