d8313b2274
HDFS-14850. Optimize FileSystemAccessService#getFileSystemConfiguration. Contributed by Lisheng Sun.
2019-09-28 17:20:44 -07:00
2200871607
HDFS-14876. Remove unused imports from TestBlockMissingException.java and TestClose.java. Contributed by Lisheng Sun.
2019-09-28 22:03:53 +05:30
c4c8d5fd0e
HDFS-14849. Erasure Coding: the internal block is replicated many times when datanode is decommissioning. Contributed by HuangTao.
2019-09-28 21:44:44 +05:30
0d5d0b914a
Revert "HDFS-14849. Erasure Coding: the internal block is replicated many times when datanode is decommissioning. Contributed by HuangTao."
...
This reverts commit ce58c05f1d
2019-09-28 21:32:15 +05:30
13b427fc05
HDFS-14564: Add libhdfs APIs for readFully; add readFully to ByteBufferPositionedReadable ( #963 ) Contributed by Sahil Takiar.
...
Reviewed-by: Siyao Meng <smeng@cloudera.com>
2019-09-27 12:25:03 -07:00
ce58c05f1d
HDFS-14849. Erasure Coding: the internal block is replicated many times when datanode is decommissioning. Contributed by HuangTao.
2019-09-27 18:58:19 +05:30
2870668cfa
Make upstream aware of 3.1.3 release.
2019-09-27 16:44:07 +08:00
2adcc3c932
HDFS-14785. [SBN read] Change client logging to be less aggressive. Contributed by Chen Liang.
2019-09-26 15:48:19 -07:00
b1e55cfb55
HDFS-14461. RBF: Fix intermittently failing kerberos related unit test. Contributed by Xiaoqiao He.
2019-09-26 15:26:23 -07:00
1a2a352ecd
HDFS-11934. Add assertion to TestDefaultNameNodePort#testGetAddressFromConf. Contributed by Nikhil Navadiya.
2019-09-26 20:45:37 +05:30
a3f68933d7
HDFS-14873. Fix dfsadmin doc for triggerBlockReport. Contributed by Fei Hui.
2019-09-26 19:54:07 +05:30
bdaaa3bbf2
HDFS-14832. RBF: Add Icon for ReadOnly False. Contributed by hemanthboyina
2019-09-25 23:48:49 +09:00
e346e3638c
HADOOP-15691 Add PathCapabilities to FileSystem and FileContext.
...
Contributed by Steve Loughran.
This complements the StreamCapabilities Interface by allowing applications to probe for a specific path on a specific instance of a FileSystem client
to offer a specific capability.
This is intended to allow applications to determine
* Whether a method is implemented before calling it and dealing with UnsupportedOperationException.
* Whether a specific feature is believed to be available in the remote store.
As well as a common set of capabilities defined in CommonPathCapabilities,
file systems are free to add their own capabilities, prefixed with
fs. + schema + .
The plan is to identify and document more capabilities -and for file systems which add new features, for a declaration of the availability of the feature to always be available.
Note
* The remote store is not expected to be checked for the feature;
It is more a check of client API and the client's configuration/knowledge
of the state of the remote system.
* Permissions are not checked.
Change-Id: I80bfebe94f4a8bdad8f3ac055495735b824968f5
2019-09-25 12:16:41 +01:00
3f89084ac7
HDFS-14845. Ignore AuthenticationFilterInitializer for HttpFSServerWebServer and honor hadoop.http.authentication configs.
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-09-25 10:19:30 +09:00
66400c1cbb
HDFS-14808. EC: Improper size values for corrupt ec block in LOG. Contributed by Ayush Saxena.
2019-09-25 01:31:15 +05:30
eb96a3093e
HDFS-14655. [SBN Read] Namenode crashes if one of The JN is down. Contributed by Ayush Saxena.
2019-09-25 01:16:30 +05:30
afa1006a53
HDFS-14843. Double Synchronization in BlockReportLeaseManager. Contributed by David Mollitor.
2019-09-24 09:58:42 -07:00
816d3cb908
HDFS-14837. Review of Block.java. Contributed by David Mollitor.
2019-09-24 09:54:09 -07:00
43203b466d
HDFS-14868. RBF: Fix typo in TestRouterQuota. Contributed by Jinglun.
...
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-09-24 09:38:58 -07:00
c30e495557
HDFS-14853. NPE in DFSNetworkTopology#chooseRandomWithStorageType() when the excludedNode is not present. Contributed by Ranith Sardar.
2019-09-23 21:22:50 +05:30
07c81e9bfc
HADOOP-16558. [COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes ( #1494 ). Contributed by Vinayakumar B.
2019-09-23 12:37:47 +05:30
4c0a7a9e13
Make upstream aware of 3.2.1 release.
2019-09-23 06:20:54 +00:00
659c88801d
HDFS-14818. Check native pmdk lib by 'hadoop checknative' command. Contributed by Feilong He.
2019-09-22 22:02:54 +05:30
aa938662f9
HDFS-14833. RBF: Router Update Doesn't Sync Quota. Contributed by Ayush Saxena.
2019-09-21 09:53:13 +05:30
3f223bebfa
HDFS-14844. Make buffer of BlockReaderRemote#newBlockReader#BufferedOutputStream configurable. Contributed by Lisheng Sun.
2019-09-20 11:45:01 -07:00
1654497f98
HADOOP-16557. [pb-upgrade] Upgrade protobuf.version to 3.7.1 ( #1432 )
...
HADOOP-16557. [pb-upgrade] Upgrade protobuf.version to 3.7.1. Contributed by Vinayakumar B.
2019-09-20 16:08:30 +05:30
a79f286c6f
HDFS-14609. RBF: Security should use common AuthenticationFilter. Contributed by Chen Zhang.
2019-09-19 11:00:10 -07:00
56248f9d87
HADOOP-16556. Fix some alerts raised by LGTM.
...
Contributed by Malcolm Taylor.
Change-Id: Ic60c3f4681dd9d48b3afcba7520bd1e4d3cc4231
2019-09-19 16:00:05 +01:00
4ed0aefe9f
HDFS-14822. [SBN read] Revisit GlobalStateIdContext locking when getting server state id. Contributed by Chen Liang.
2019-09-18 16:47:46 -07:00
5dd859a8a0
HDFS-14836. FileIoProvider should not increase FileIoErrors metric in datanode volume metric. Contributed by Aiphago.
...
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
2019-09-18 09:34:01 -07:00
3cf6e4272f
HDFS-14846: libhdfs tests are failing on trunk due to jni usage bugs
...
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-09-17 15:51:39 -07:00
f580a87079
HDFS-14795. Add Throttler for writing block. Contributed by Lisheng Sun.
2019-09-17 14:55:34 -07:00
e04b8a46c3
HDFS-14799. Do Not Call Map containsKey In Conjunction with get. Contributed by hemanthboyina.
2019-09-14 10:39:20 +05:30
06ad6540b3
HDFS-14821. Make HDFS-14617 (fsimage sub-sections) off by default. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-09-13 10:36:45 -07:00
4852a90e4b
HDFS-14754. Erasure Coding : The number of Under-Replicated Blocks never reduced. Contributed by hemanthboyina.
2019-09-13 00:17:13 +05:30
2ff2a7f612
HDFS-14798. Synchronize invalidateBlocks in DatanodeDescriptor. Contributed by hemanthboyina.
2019-09-12 19:43:18 +05:30
d1c303a497
HDFS-14699. Erasure Coding: Storage not considered in live replica when replication streams hard limit reached to threshold. Contributed by Zhao Yi Ming.
2019-09-12 19:11:50 +05:30
68612a0410
HDFS-14840. Use Java Conccurent Instead of Synchronization in BlockPoolTokenSecretManager. Contributed by David Mollitor.
2019-09-12 12:41:57 +09:00
c255333e20
HDFS-14838. RBF: Display RPC (instead of HTTP) Port Number in RBF web UI. Contributed by Xieming Li
2019-09-11 16:54:08 +09:00
524b553a5f
HDFS-14835. RBF: Secured Router should not run when it can't initialize DelegationTokenSecretManager. ( #1414 )
2019-09-11 10:32:07 +09:00
c92a3e94d8
HADOOP-15565. Add an inner FS cache to ViewFileSystem, separate from the global cache, to avoid file system leaks. Contributed by Jinglun.
2019-09-06 10:22:28 -07:00
d98c54816d
HDFS-13913. LazyPersistFileScrubber.run() should log meaningful warn message. Contributed by Daniel Green.
2019-09-06 18:12:42 +05:30
fa7f03fc56
HDFS-14826. dfs.ha.zkfc.port property duplicated in hdfs-default.xml. Contributed by Renukaprasad C.
2019-09-06 17:55:30 +05:30
494d75eb2b
HDFS-14784. Add more methods to WebHdfsTestUtil to support tests outside of package. Contributed by Chen Zhang.
2019-09-05 21:15:17 -07:00
72d8b92ba5
HDFS-12904. Add DataTransferThrottler to the Datanode transfers. Contributed by Lisheng Sun.
2019-09-05 11:44:02 -07:00
2f70b52a5b
HDFS-14812. RBF: MountTableRefresherService should load cache when refresh. Contributed by xuzq.
2019-09-05 08:29:56 +05:30
05704754a0
HDFS-14777. RBF: Set ReadOnly is failing for mount Table but actually readonly succeed to set. Contributed by Ranith Sardar.
2019-09-04 15:09:09 +05:30
3c117163a3
HDFS-14633. The StorageType quota and consume in QuotaFeature is not handled for rename. Contributed by Jinglun.
2019-09-03 09:29:58 -07:00
91b01a1db7
HDFS-14807. SetTimes updates all negative values apart from -1. Contributed by Ayush Saxena.
2019-09-03 18:08:42 +05:30
5ff76cb8bc
HDFS-14630. Configuration.getTimeDurationHelper() should not log time unit warning in info log. Contributed by hemanthboyina.
2019-09-03 12:37:09 +05:30
915cbc91c0
HDFS-14706. Checksums are not checked if block meta file is less than 7 bytes. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-09-02 09:47:04 -07:00
d207aba026
Revert "HDFS-14706. Checksums are not checked if block meta file is less than 7 bytes. Contributed by Stephen O'Donnell."
...
This reverts commit 7bebad61d9
2019-09-02 09:43:44 -07:00
040f6e93bb
HDFS-14654. RBF: TestRouterRpc#testNamenodeMetrics is flaky. Contributed by Chen Zhang.
2019-09-02 13:01:52 +05:30
c3abfcefdd
HDFS-13843. RBF: Add optional parameter -d for detailed listing of mount points. Contributed by Ayush Saxena.
2019-09-02 07:24:04 +05:30
18d74fe41c
HDFS-14711. RBF: RBFMetrics throws NullPointerException if stateStore disabled. Contributed by Chen Zhang.
2019-09-01 12:53:10 +05:30
96f7dc1992
HDFS-14796. Define LOG instead of BlockManager.LOG in ErasureCodingWork/ReplicationWork. Contributed by Fei Hui.
2019-08-30 14:31:05 +05:30
e220dac15c
HDFS-12212. Options.Rename.To_TRASH is considered even when Options.Rename.NONE is specified. Contributed by Vinayakumar B.
2019-08-30 08:40:13 +05:30
915271245b
Make DataNodePeerMetrics#minOutlierDetectionSamples configurable ( #1314 ). Contributed by Lisheng Sun.
...
Signed-off-by: sunlisheng <sunlisheng@xiaomi.com>
2019-08-29 18:35:26 -07:00
7bebad61d9
HDFS-14706. Checksums are not checked if block meta file is less than 7 bytes. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-29 17:38:07 -07:00
fcb7884bfc
HDFS-8178. QJM doesn't move aside stale inprogress edits files. Contributed by Istvan Fajth.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-29 16:56:14 -07:00
3b22fcd377
HDFS-14256. Review Logging of NameNode Class. Contributed by David Mollitor.
2019-08-29 14:21:09 -07:00
ffca734c62
HDFS-14104. Review getImageTxIdToRetain. Contributed by David Mollitor.
2019-08-29 14:18:35 -07:00
f600fbb6c4
HDFS-11246. FSNameSystem#logAuditEvent should be called outside the read or write locks. Contributed by He Xiaoqiao, Kuhu Shukla.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Co-authored-by: Kuhu Shukla <kshukla@apache.org>
2019-08-29 10:10:52 -07:00
8e779a151e
HDFS-14721. RBF: ProxyOpComplete is not accurate in FederationRPCPerformanceMonitor. Contributed by xuzq.
2019-08-29 20:08:38 +05:30
29bd6f3fc3
HDFS-8631. WebHDFS : Support setQuota. Contributed by Chao Sun.
2019-08-28 23:58:23 +05:30
48cb583906
HDFS-14710. RBF: Improve some RPC performance by using previous block. Contributed by xuzq.
2019-08-28 10:48:00 -07:00
6e37d65b03
HDFS-14760. Log INFO mode if snapshot usage and actual usage differ. Contributed by CR Hota.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-27 16:20:25 -07:00
dde9399b37
HDFS-14497. Addendum: Write lock held by metasave impact following RPC processing.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-27 15:26:59 -07:00
ce8eb1283a
Revert "HDDS-1610. applyTransaction failure should not be lost on restart. Contributed by Shashikant Banerjee."
...
This reverts commit 62445021d5
2019-08-27 23:23:44 +05:30
8ab7020e64
HDFS-14779. Fix logging error in TestEditLog#testMultiStreamsLoadEditWithConfMaxTxns
2019-08-27 10:30:31 -07:00
b69ac575a1
HDFS-14772. RBF: hdfs-rbf-site.xml can't be loaded automatically. Contributed by Yuxuan Wang
2019-08-27 14:32:32 +09:00
07e3cf952e
HDFS-2470. NN should automatically set permissions on dfs.namenode.*.dir. Contributed by Siddharth Wagle.
2019-08-26 15:57:55 -07:00
d2225c8ca8
HDFS-14722. RBF: GetMountPointStatus should return mountTable information when getFileInfoAll throw IOException. Contributed by xuzq.
2019-08-24 08:47:43 +05:30
0b796754b9
HDFS-14766. RBF: MountTableStoreImpl#getMountTableEntries returns extra entry. Contributed by Chen Zhang.
2019-08-23 14:53:57 -07:00
ebef99dcf4
HDFS-14674. [SBN read] Got an unexpected txid when tail editlog. Contributed by wangzhaohui.
2019-08-23 12:18:43 -07:00
d699022fce
HDFS-13977. Override shouldForceSync in QuorumOutputStream to allow for proper auto-sync behavior. Contributed by Erik Krogen.
2019-08-23 11:33:21 -07:00
894e2300d6
HDFS-14761. RBF: MountTableResolver cannot invalidate cache correctly ( #1334 )
...
HDFS-14761. RBF: MountTableResolver cannot invalidate cache correctly
2019-08-23 10:46:39 -07:00
c92de8209d
HDFS-14747. RBF: IsFileClosed should be return false when the file is open in multiple destination. Contributed by xuzq.
2019-08-23 19:08:07 +05:30
bd7baea5a5
HDFS-14396. Failed to load image from FSImageFile when downgrade from 3.x to 2.x. Contributed by Fei Hui.
2019-08-23 15:28:27 +09:00
abc8fde4ca
HDFS-13596. NN restart fails after RollingUpgrade from 2.x to 3.x. Contributed by Fei Hui.
2019-08-23 14:44:34 +09:00
b67812ea21
HDFS-14617. Improve fsimage load time by writing sub-sections to the fsimage index ( #1028 ). Contributed by Stephen O'Donnell.
...
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
2019-08-22 17:09:57 -07:00
93daf69f90
HDFS-14675. Increase Balancer Defaults Further. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Gabor Bota <gabota@apache.org>
2019-08-22 15:21:33 -07:00
e04dcfdc57
HDFS-14583. FileStatus#toString() will throw IllegalArgumentException. Contributed by xuzq.
2019-08-22 10:22:38 -07:00
76790a1e67
HDFS-14358. Provide LiveNode and DeadNode filter in DataNode UI. Contributed by hemanthboyina.
2019-08-22 12:27:45 +05:30
52c77bc160
HDFS-14741. RBF: RecoverLease should be return false when the file is open in multiple destination. Contributed by xuzq
2019-08-22 08:57:22 +05:30
f9029c4070
HDFS-14744. RBF: Non secured routers should not log in error mode when UGI is default. Contributed by CR Hota.
2019-08-22 02:40:39 +05:30
0f598aed13
HDFS-14276. [SBN read] Reduce tailing overhead. Contributed by Wei-Chiu Chuang.
2019-08-22 02:25:06 +05:30
93595febaa
HDFS-14756. RBF: getQuotaUsage may ignore some folders. Contributed by Chen Zhang.
2019-08-21 09:39:57 -07:00
5eeb6da2d4
HDFS-14714. RBF: implement getReplicatedBlockStats interface. Contributed by Chen Zhang.
2019-08-21 09:38:17 -07:00
57f7370174
Revert "HDFS-14476. lock too long when fix inconsistent blocks between disk and in-memory. Contributed by Sean Chow."
...
This reverts commit b58a35f374
2019-08-21 08:59:49 -07:00
8aaf5e1a14
HDFS-14759. HDFS cat logs an info message.
...
Contributed by Eric Badger.
2019-08-20 20:24:19 -07:00
3a145e2918
HDFS-14582. Failed to start DN with ArithmeticException when NULL checksum used. Contributed by Surendra Singh Lilhore.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-20 15:55:26 -07:00
269b543367
HADOOP-16523. Minor spell mistake in comment : Line number 101 ( #388 )
2019-08-20 13:36:23 -07:00
4cb22cd867
HDFS-14311. Multi-threading conflict at layoutVersion when loading block pool storage. Contributed by Yicong Cai.
2019-08-20 10:37:11 -07:00
aa6995fde2
HDFS-13201. Fix prompt message in testPolicyAndStateCantBeNull. Contributed by chencan.
2019-08-20 10:32:55 -07:00
bd92462321
HDFS-14729. Upgrade Bootstrap and jQuery versions used in HDFS UIs. Contributed by Vivek Ratnavel Subramanian. This closes #1297
2019-08-20 18:42:19 +05:30
62445021d5
HDDS-1610. applyTransaction failure should not be lost on restart. Contributed by Shashikant Banerjee.
2019-08-20 14:37:01 +05:30
360a96f342
HDFS-13709. Report bad block to NN when transfer block encounter EIO exception. Contributed by Chen Zhang.
2019-08-19 13:08:55 -07:00
abae6ff2a2
HDFS-14746. Trivial test code update after HDFS-14687. Contributed by kevin su.
...
Reviewed-by: Surendra Singh Lilhore <surendralilhore@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-19 11:14:51 -07:00
b8db5b9a98
HDFS-14687. Standby Namenode never come out of safemode when EC files are being written. Contributed by Surendra Singh Lilhore.
...
Reviewed-by: Siyao Meng <smeng@cloudera.com>
Reviewed-by: Wei-Chiu CHuang <weichiu@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-18 18:13:35 -07:00
b58a35f374
HDFS-14476. lock too long when fix inconsistent blocks between disk and in-memory. Contributed by Sean Chow.
2019-08-18 17:46:00 -07:00
d873ddd656
HDFS-14127. Add a description about the observer read configuration. Contributed by xiangheng.
2019-08-17 04:39:22 -07:00
528378784f
HDFS-12012. Fix spelling mistakes in BPServiceActor.java. Contributed by chencan.
2019-08-17 04:37:05 -07:00
a38b9e137e
HDFS-14456:HAState#prepareToEnterState neednt a lock ( #770 ) Contributed by hunshenshi.
2019-08-16 14:53:06 -07:00
9a1d8cfaf5
HDFS-14678. Allow triggerBlockReport to a specific namenode. ( #1252 ). Contributed by Leon Gao.
2019-08-16 08:00:51 -07:00
34681643e9
HDFS-14719. Correct the safemode threshold value in BlockManagerSafeMode. Contributed by hemanthboyina.
2019-08-15 12:12:18 +05:30
0a85af959c
HDFS-13101. Yet another fsimage corruption related to snapshot. Contributed by Shashikant Banerjee.
2019-08-15 10:16:25 +05:30
b06c2345ef
HDFS-14713. RBF: RouterAdmin supports refreshRouterArgs command but not on display. Contributed by wangzhaohui.
2019-08-14 20:44:31 +05:30
3c0382f1b9
HDFS-14595. HDFS-11848 breaks API compatibility. Contributed by Siyao Meng.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
2019-08-14 07:30:35 -07:00
c13ec7ab66
HDFS-14491. More Clarity on Namenode UI Around Blocks and Replicas. Contributed by Siyao Meng.
2019-08-13 17:15:26 -07:00
633b7c1cfe
HDFS-14625. Make DefaultAuditLogger class in FSnamesystem to Abstract. Contributed by hemanthboyina.
2019-08-13 16:50:49 -07:00
da0006fe04
HDFS-14423. Percent (%) and plus (+) characters no longer work in WebHDFS.
...
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
2019-08-14 08:39:40 +09:00
6ae8bc3a4a
HDFS-14665. HttpFS: LISTSTATUS response is missing HDFS-specific fields ( #1267 ) Contributed by Siyao Meng.
2019-08-13 16:27:57 -07:00
e9b6b81de4
HDFS-13505. Turn on HDFS ACLs by default. Contributed by Siyao Meng.
2019-08-13 19:17:10 +05:30
454420e4f2
HDFS-14708. TestLargeBlockReport#testBlockReportSucceedsWithLargerLengthLimit fails in trunk. Contributed by Lisheng Sun.
2019-08-13 08:27:20 +05:30
c92b49876a
HDFS-14148. HDFS OIV ReverseXML SnapshotSection parser throws exception when there are more than one snapshottable directory ( #1274 ) Contributed by Siyao Meng.
2019-08-12 17:26:20 -07:00
8a77a224c7
HDFS-13359. DataXceiver hung due to the lock in FsDatasetImpl#getBlockInputStream. Contributed by Yiqun Lin.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-09 18:40:28 -07:00
e02ffed1b1
HDFS-12125. Document the missing EC removePolicy command ( #1258 ) Contributed by Wenxin He, Siyao Meng.
...
Co-authored-by: Wenxin He <wenxin.he@gmail.com>
2019-08-09 18:00:22 -07:00
ce3c5a3e3b
HDFS-14623. In NameNode Web UI, for Head the file (first 32K) old data is showing. Contributed by hemanthboyina.
2019-08-09 15:41:37 -07:00
865021b8c9
HDFS-14195. OIV: print out storage policy id in oiv Delimited output. Contributed by Wang, Xinglong.
2019-08-09 15:37:29 -07:00
f6fa865d6f
HDFS-14700. Clean up pmem cache before setting pmem cache capacity. Contributed by Feilong He.
2019-08-09 14:07:54 +05:30
6ad9a11494
HDFS-14693. NameNode should log a warning when EditLog IPC logger's pending size exceeds limit. Contributed by Xudong Cao.
2019-08-08 13:50:30 -07:00
2265872c2d
HDFS-14705. Remove unused configuration dfs.min.replication. Contributed by CR Hota.
2019-08-08 13:48:29 -07:00
28a848412c
HDFS-14701. Change Log Level to warn in SlotReleaser. Contributed by Lisheng Sun.
2019-08-08 13:46:31 -07:00
23f91f68b8
HDFS-14662. Document the usage of the new Balancer "asService" parameter. Contributed by Chen Zhang.
2019-08-08 13:45:29 -07:00
b0799148cf
HDFS-14459. ClosedChannelException silently ignored in FsVolumeList.addBlockPool(). Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-08 13:36:39 -07:00
11f750e6a7
HDFS-14616. Add the warn log when the volume available space isn't enough. Contributed by liying.
2019-08-08 00:49:58 +05:30
de64253892
HDFS-14608. DataNode#DataTransfer should be named. Contributed by Inigo Goiri.
2019-08-08 00:37:56 +05:30
827dbb11e2
HDFS-14370. Add exponential backoff to the edit log tailer to avoid spinning on empty edit tail requests. Contributed by Erik Krogen.
2019-08-07 09:25:58 -07:00
a5bb1e8ee8
HDFS-14313. Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du. Contributed by Lisheng Sun.
2019-08-07 10:18:11 +08:00
954ff36360
HDFS-14692. Upload button should not encode complete url. Contributed by Lokesh Jain.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-06 15:30:50 -07:00
35e0a01d7b
HDFS-14557. JournalNode error: Can't scan a pre-transactional edit log. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-06 07:57:46 -07:00
11272159bb
HDFS-14679. Failed to add erasure code policies with example template. Contributed by Yuan Zhou.
2019-08-06 08:42:15 +05:30
8f40856f76
HDFS-14669. TestDirectoryScanner#testDirectoryScannerInFederatedCluster fails intermittently in trunk. Contributed by qiang Liu.
2019-08-03 12:15:41 +05:30
e503db5f44
HDFS-12826. Document Saying the RPC port, But it's required IPC port in HDFS Federation Document. Contributed by usharani.
2019-08-03 12:05:46 +05:30
e7a0b8aa83
HDFS-14462 Ensure WebHDFS client throws the correct exception during writes. Contributed by Simbarashe Dzinamarira.
2019-08-02 11:48:31 -07:00
b964b81f85
HDFS-14685. DefaultAuditLogger doesn't print CallerContext. Contributed by xuzq.
2019-08-02 10:54:32 -07:00
c2d00c8450
HDFS-13131. Modifying testcase testEnableAndDisableErasureCodingPolicy. Contributed by chencan.
2019-08-01 19:11:43 -07:00
17e8cf501b
HDFS-14686. HttpFS: HttpFSFileSystem#getErasureCodingPolicy always returns null ( #1192 ) Contributed by Siyao Meng.
2019-08-01 17:15:22 -07:00
99bf1dc9eb
HDFS-14683. WebHDFS: Add erasureCodingPolicy field to GETCONTENTSUMMARY response ( #1189 ) Contributed by Siyao Meng.
2019-08-01 17:14:07 -07:00
32607dbd98
HDFS-14631.The DirectoryScanner doesn't fix the wrongly placed replica. Contributed by Jinglun.
2019-08-01 15:48:36 -07:00
89b102f916
HDFS-14661. RBF: updateMountTableEntry shouldn't update mountTableEntry if targetPath not exist. Contributed by xuzq.
2019-08-01 08:43:39 +05:30
b008072044
HDFS-14681. RBF: TestDisableRouterQuota failed because port 8888 was occupied.
2019-07-31 10:24:48 -07:00
99f88c30cb
HDFS-14419. Avoid repeated calls to the listOpenFiles function. Contributed by HuangTao.
2019-07-30 16:50:06 -07:00
3ae775d740
HDFS-14034. Support getQuotaUsage API in WebHDFS. Contributed by Chao Sun.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-07-30 16:01:17 -07:00
1f26cc8705
HDFS-13783. Add an option to the Balancer to make it run as a long-running service. Contributed by Chen Zhang.
2019-07-30 15:42:55 -07:00
7849bdcf70
HDFS-14449. Expose total number of DT in JMX for Namenode. Contributed by Fengnan Li.
2019-07-30 13:45:27 -07:00
8deced0073
HDFS-14677. TestDataNodeHotSwapVolumes#testAddVolumesConcurrently fails intermittently in trunk. Contributed by Chen Zhang.
2019-07-30 22:17:39 +05:30
8053085388
HDFS-14429. Block remain in COMMITTED but not COMPLETE caused by Decommission. Contributed by Yicong Cai.
2019-07-29 14:31:34 -07:00
611718f59f
HDFS-14670: RBF: Create secret manager instance using FederationUtil#newInstance.
2019-07-29 11:00:22 -07:00
02bd02b5af
HDFS-14660. [SBN Read] ObserverNameNode should throw StandbyException for requests not from ObserverProxyProvider. Contributed by Chao Sun.
2019-07-28 08:41:42 +05:30
2fe450cb5e
HDFS-14425. Native build fails on macos due to jlong in hdfs.c ( #741 )
2019-07-26 15:33:12 -07:00
ecc8acfd24
HDFS-14673. The console log is noisy when using DNSDomainNameResolver to resolve NameNode.
2019-07-26 11:08:46 -07:00
ce99cc31e9
HDFS-14303. Addendum: check block directory logic not correct when there is only meta file, print no meaning warn log. Contributed by qiang Liu.
2019-07-26 08:31:00 +05:30
6b8107ad97
HDFS-14135. TestWebHdfsTimeouts Fails intermittently in trunk. Contributed by Ayush Saxena.
...
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
2019-07-26 09:35:23 +09:00
62deab17a3
HDFS-14647. NPE during secure namenode startup. Contributed by Fengnan Li.
2019-07-25 06:51:07 +05:30
377f95bbe8
HDFS-13693. Remove unnecessary search in INodeDirectory.addChild during image loading. Contributed by Lisheng Sun.
2019-07-23 08:37:55 +05:30
6282c02d7d
HDFS-14577. RBF: FederationUtil#newInstance should allow constructor without context. Contributed by CR Hota.
2019-07-19 17:44:13 +05:30
9792f58662
HDFS-14653. RBF: Correct the default value for dfs.federation.router.namenode.heartbeat.enable. Contributed by Ayush Saxena.
2019-07-18 10:16:20 +09:00
5e6cc6fe8a
HDFS-12979. [SBN read] StandbyNode should upload FsImage to ObserverNode after checkpointing. Contributed by Chen Liang.
2019-07-17 14:37:16 -07:00
af0665c7f5
HDFS-13647. Fix the description of storageType option for space quota. Contributed by Takanobu Asanuma.
2019-07-17 23:22:32 +05:30
f77d54c243
HDFS-14642. processMisReplicatedBlocks does not return correct processed count. Contributed by Stephen O'Donnell.
2019-07-16 08:14:27 +05:30
64d4abf489
HDFS-14593. RBF: Implement deletion feature for expired records in State Store. Contributed by Takanobu Asanuma.
2019-07-15 22:38:00 +05:30
30a8f840f1
HDFS-14357. Update documentation for HDFS cache on SCM support. Contributed by Feilong He.
2019-07-15 13:18:23 +05:30
e98adb00b7
HDFS-14458. Report pmem stats to namenode. Contributed by Feilong He.
2019-07-15 13:02:37 +05:30
f9fab9f22a
HDFS-14499. Misleading REM_QUOTA value with snapshot and trash feature enabled for a directory. Contributed by Shashikant Banerjee.
2019-07-12 15:42:06 +05:30
00dd843a1a
HDFS-14466. Add a regression test for HDFS-14323.
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-07-12 10:04:55 +09:00
8fb5ca3f40
HDFS-14611. Move handshake secret field from Token to BlockAccessToken. Contributed by Chen Liang.
2019-07-11 13:23:25 -07:00
5747f6cff5
HDFS-14624. When decommissioning a node, log remaining blocks to replicate periodically. Contributed by Stephen O'Donnell.
2019-07-11 08:55:44 -07:00
eccc9a40de
HDFS-12703. Exceptions are fatal to decommissioning monitor. Contributed by He Xiaoqiao.
2019-07-10 11:11:52 -07:00
cdb20adfcc
HDDS-1611. Evaluate ACL on volume bucket key and prefix to authorize access. Contributed by Ajay Kumar. ( #973 )
2019-07-10 11:03:58 -07:00
993dc8726b
HDFS-14632. Reduce useless #getNumLiveDataNodes call in SafeModeMonitor. Contributed by He Xiaoqiao.
2019-07-10 10:53:34 +08:00
4632708148
HDFS-14547. Improve memory efficiency of quotas when storage type quotas are not set. Contributed by Jinglun.
2019-07-08 14:46:25 -07:00
ec851e4db2
HDFS-12862. CacheDirective becomes invalid when NN restart or failover. Contributed by Wang XL.
2019-07-08 15:23:46 +08:00
9c90729486
HDFS-13694. Making md5 computing being in parallel with image loading.
2019-07-05 10:17:37 -07:00
c3ca348b81
HDFS-14620. RBF: Fix 'not a super user' error when disabling a namespace in kerberos with superuser principal. Contributed by luhuachao.
2019-07-04 11:21:55 -07:00
1c254a8ec7
HDFS-14629. Property value Hard Coded in DNConf.java. Contributed by hemanthboyina.
2019-07-04 23:20:42 +09:00
729cb3aefe
HDFS-12748. NameNode memory leak when accessing webhdfs GETHOMEDIRECTORY. Contributed by Weiwei Yang.
2019-07-04 09:37:24 +08:00
d8bac50e12
HDFS-14610. HashMap is not thread safe. Field storageMap is typically synchronized by storageMap. However, in one place, field storageMap is not protected with synchronized. ( #1015 )
2019-07-01 13:54:49 -07:00
d203045c30
HDFS-14618. Incorrect synchronization of ArrayList field (ArrayList is thread-unsafe). Contributed by Paul Ward.
2019-06-28 16:06:43 -07:00
4a212242d9
HDFS-14599. HDFS-12487 breaks test TestDiskBalancer.testDiskBalancerWithFedClusterWithOneNameServiceEmpty. Contributed by He Xiaoqiao.
2019-06-27 10:00:05 -07:00
041e0c0564
HDFS-14036. RBF: Add hdfs-rbf-default.xml to HdfsConfiguration by default. Contributed by Takanobu Asanuma.
2019-06-27 09:57:52 -07:00
041e7a7dee
HDFS-14598. Findbugs warning caused by HDFS-12487.
...
Contributed by He Xiaoqiao.
2019-06-24 19:07:22 -07:00
b76b843c8b
HDFS-13371. NPE for FsServerDefaults.getKeyProviderUri() for clientProtocol communication between 2.7 and 3.X. Contributed by Sherwood Zheng.
2019-06-24 17:52:33 -07:00
38a560c6f1
Merge pull request #977 from leosunli/trunk
...
HDFS-14541. When evictableMmapped or evictable size is zero, do not throw NoSuchElementException.
2019-06-24 17:44:54 -07:00
95c94dcca7
HADOOP-16350. Ability to tell HDFS client not to request KMS Information from NameNode. Ccontributed by Greg Senia, Ajay Kumar.
2019-06-24 11:59:07 -07:00
719d57bf46
HDFS-14545. RBF: Router should support GetUserMappingsProtocol. Contributed by Ayush Saxena.
2019-06-24 22:03:04 +05:30
8627f6532b
HDFS-14550. RBF: Failed to get statistics from NameNodes before 2.9.0. Contributed by He Xiaoqiao.
2019-06-24 22:03:04 +05:30
ba221dc50e
HDFS-14526. RBF: Update the document of RBF related metrics. Contributed by Takanobu Asanuma.
2019-06-24 22:03:04 +05:30
ade8d3b60e
HDFS-14508. RBF: Clean-up and refactor UI components. Contributed by Takanobu Asanuma.
2019-06-24 22:03:04 +05:30
6915d7e13c
HDFS-13480. RBF: Separate namenodeHeartbeat and routerHeartbeat to different config key. Contributed by Ayush Saxena.
2019-06-24 22:03:04 +05:30
ddbe08db33
HDFS-13955. RBF: Support secure Namenode in NamenodeHeartbeatService. Contributed by CR Hota.
2019-06-24 22:03:04 +05:30
b062dd462d
HDFS-14475. RBF: Expose router security enabled status on the UI. Contributed by CR Hota.
2019-06-24 22:03:04 +05:30
b6fff8c81e
HDFS-13787. RBF: Add Snapshot related ClientProtocol APIs. Contributed by Inigo Goiri.
2019-06-24 22:03:04 +05:30
d240eec136
HDFS-14516. RBF: Create hdfs-rbf-site.xml for RBF specific properties. Contributed by Takanobu Asanuma.
2019-06-24 22:03:04 +05:30
9b197c2893
HDFS-13909. RBF: Add Cache pools and directives related ClientProtocol APIs. Contributed by Ayush Saxena.
2019-06-24 22:03:04 +05:30
ffbb6b6557
HDFS-13255. RBF: Fail when try to remove mount point paths. Contributed by Akira Ajisaka.
2019-06-24 22:03:04 +05:30
8e4267650f
HDFS-14440. RBF: Optimize the file write process in case of multiple destinations. Contributed by Ayush Saxena.
2019-06-24 22:03:04 +05:30
2636a54ffd
HDFS-13995. RBF: Security documentation. Contributed by CR Hota.
2019-06-24 22:03:04 +05:30
393f15176d
HDFS-14447. RBF: Router should support RefreshUserMappingsProtocol. Contributed by Shen Yinjie.
2019-06-24 22:03:04 +05:30
3deb5d345f
HDFS-14490. RBF: Remove unnecessary quota checks. Contributed by Ayush Saxena.
2019-06-24 22:03:04 +05:30
62fa53a01d
HDFS-14210. RBF: ACL commands should work over all the destinations. Contributed by Ayush Saxena.
2019-06-24 22:03:04 +05:30
32841178ba
HDFS-14426. RBF: Add delegation token total count as one of the federation metrics. Contributed by Fengnan Li.
2019-06-24 22:03:04 +05:30
203664e6b2
HDFS-14454. RBF: getContentSummary() should allow non-existing folders. Contributed by Inigo Goiri.
2019-06-24 22:03:04 +05:30
b522b52bb1
HDFS-14457. RBF: Add order text SPACE in CLI command 'hdfs dfsrouteradmin'. Contributed by luhuachao.
2019-06-24 22:03:04 +05:30
021a43b1a4
HDFS-13972. RBF: Support for Delegation Token (WebHDFS). Contributed by CR Hota.
2019-06-24 22:03:04 +05:30
de7da9b69e
HDFS-14422. RBF: Router shouldn't allow READ operations in safe mode. Contributed by Inigo Goiri.
2019-06-24 22:03:04 +05:30
0f9b8d7a75
HDFS-14369. RBF: Fix trailing / for webhdfs. Contributed by Akira Ajisaka.
2019-06-24 22:03:04 +05:30
dd8c2b92df
HDFS-13853. RBF: RouterAdmin update cmd is overwriting the entry not updating the existing. Contributed by Ayush Saxena.
2019-06-24 22:03:04 +05:30
6c42d40504
HDFS-14316. RBF: Support unavailable subclusters for mount points with multiple destinations. Contributed by Inigo Goiri.
2019-06-24 22:03:04 +05:30
0dbd87874a
HDFS-14388. RBF: Prevent loading metric system when disabled. Contributed by Inigo Goiri.
2019-06-24 22:03:03 +05:30
9a9fbbe145
HDFS-14351. RBF: Optimize configuration item resolving for monitor namenode. Contributed by He Xiaoqiao and Inigo Goiri.
2019-06-24 22:03:03 +05:30
f539e2a4ee
HDFS-14343. RBF: Fix renaming folders spread across multiple subclusters. Contributed by Ayush Saxena.
2019-06-24 22:03:03 +05:30
2a2d5eb441
HDFS-14334. RBF: Use human readable format for long numbers in the Router UI. Contributed by Inigo Goiri.
2019-06-24 22:03:03 +05:30
fcabc8f0e4
HDFS-14335. RBF: Fix heartbeat typos in the Router. Contributed by CR Hota.
2019-06-24 22:03:03 +05:30
6cdf8db55c
HDFS-14331. RBF: IOE While Removing Mount Entry. Contributed by Ayush Saxena.
2019-06-24 22:03:03 +05:30
1ce25e702b
HDFS-14329. RBF: Add maintenance nodes to federation metrics. Contributed by Ayush Saxena.
2019-06-24 22:03:03 +05:30
7bbe01a196
HDFS-14259. RBF: Fix safemode message for Router. Contributed by Ranith Sadar.
2019-06-24 22:03:03 +05:30
1c7ab59be3
HDFS-14322. RBF: Security manager should not load if security is disabled. Contributed by CR Hota.
2019-06-24 22:03:03 +05:30
c4b1fa91fa
HDFS-14052. RBF: Use Router keytab for WebHDFS. Contributed by CR Hota.
2019-06-24 22:03:03 +05:30
de719b08b5
HDFS-14307. RBF: Update tests to use internal Whitebox instead of Mockito. Contributed by CR Hota.
2019-06-24 22:03:03 +05:30
8b8ff5ccbc
HDFS-14249. RBF: Tooling to identify the subcluster location of a file. Contributed by Inigo Goiri.
2019-06-24 22:03:03 +05:30
50aee18a84
HDFS-14268. RBF: Fix the location of the DNs in getDatanodeReport(). Contributed by Inigo Goiri.
2019-06-24 22:03:03 +05:30
e2a3c4494b
HDFS-14226. RBF: Setting attributes should set on all subclusters' directories. Contributed by Ayush Saxena.
2019-06-24 22:03:03 +05:30
75f8b6ccfa
HDFS-13358. RBF: Support for Delegation Token (RPC). Contributed by CR Hota.
2019-06-24 22:03:03 +05:30
7e63e37dc5
HDFS-14230. RBF: Throw RetriableException instead of IOException when no namenodes available. Contributed by Fei Hui.
2019-06-24 22:03:03 +05:30
912b90f91e
HDFS-14252. RBF : Exceptions are exposing the actual sub cluster path. Contributed by Ayush Saxena.
2019-06-24 22:03:03 +05:30
9c4e55685b
HDFS-14225. RBF : MiniRouterDFSCluster should configure the failover proxy provider for namespace. Contributed by Ranith Sardar.
2019-06-24 22:03:03 +05:30
9eed3a49df
HDFS-14215. RBF: Remove dependency on availability of default namespace. Contributed by Ayush Saxena.
2019-06-24 22:03:03 +05:30
acdf911c01
HDFS-14224. RBF: NPE in getContentSummary() for getEcPolicy() in case of multiple destinations. Contributed by Ayush Saxena.
2019-06-24 22:03:03 +05:30
8b9b58b58a
HDFS-14223. RBF: Add configuration documents for using multiple sub-clusters. Contributed by Takanobu Asanuma.
2019-06-24 22:03:03 +05:30
020f83f51f
HDFS-14209. RBF: setQuota() through router is working for only the mount Points under the Source column in MountTable. Contributed by Shubham Dewan.
2019-06-24 22:03:03 +05:30
235406d904
HDFS-14156. RBF: rollEdit() command fails with Router. Contributed by Shubham Dewan.
2019-06-24 22:03:03 +05:30
c012b09fb6
HDFS-14193. RBF: Inconsistency with the Default Namespace. Contributed by Ayush Saxena.
2019-06-24 22:03:03 +05:30
7b61cbf672
HDFS-14129. addendum to HDFS-14129. Contributed by Ranith Sardar.
2019-06-24 22:03:03 +05:30
f40e10b349
HDFS-14129. RBF: Create new policy provider for router. Contributed by Ranith Sardar.
2019-06-24 22:03:03 +05:30
221f24cbdc
HDFS-14206. RBF: Cleanup quota modules. Contributed by Inigo Goiri.
2019-06-24 22:03:02 +05:30
f4e2bfce58
HDFS-13856. RBF: RouterAdmin should support dfsrouteradmin -refreshRouterArgs command. Contributed by yanghuafeng.
2019-06-24 22:03:02 +05:30
b8bcbd0ed2
HDFS-14191. RBF: Remove hard coded router status from FederationMetrics. Contributed by Ranith Sardar.
2019-06-24 22:03:02 +05:30
4244653e43
HDFS-14150. RBF: Quotas of the sub-cluster should be removed when removing the mount point. Contributed by Takanobu Asanuma.
2019-06-24 22:03:02 +05:30
f3cbf0eb9a
HDFS-14161. RBF: Throw StandbyException instead of IOException so that client can retry when can not get connection. Contributed by Fei Hui.
2019-06-24 22:03:02 +05:30
1dc01e59af
HDFS-14167. RBF: Add stale nodes to federation metrics. Contributed by Inigo Goiri.
2019-06-24 22:03:02 +05:30
8f6f9d9c83
HDFS-13443. RBF: Update mount table cache immediately after changing (add/update/remove) mount table entries. Contributed by Mohammad Arshad.
2019-06-24 22:03:02 +05:30
3d97142dff
HDFS-14151. RBF: Make the read-only column of Mount Table clearly understandable.
2019-06-24 22:03:02 +05:30
bbe859177d
HDFS-13869. RBF: Handle NPE for NamenodeBeanMetrics#getFederationMetrics. Contributed by Ranith Sardar.
2019-06-24 22:03:02 +05:30
01b4126b4e
HDFS-14152. RBF: Fix a typo in RouterAdmin usage. Contributed by Ayush Saxena.
2019-06-24 22:03:02 +05:30
94a8dec168
HDFS-14114. RBF: MIN_ACTIVE_RATIO should be configurable. Contributed by Fei Hui.
2019-06-24 22:03:02 +05:30
0ca7142c11
Revert "HDFS-14114. RBF: MIN_ACTIVE_RATIO should be configurable. Contributed by Fei Hui."
...
This reverts commit 7c0d6f65fde12ead91ed7c706521ad1d3dc995f8.
2019-06-24 22:03:02 +05:30
6aa7aabff8
HDFS-14114. RBF: MIN_ACTIVE_RATIO should be configurable. Contributed by Fei Hui.
2019-06-24 22:03:02 +05:30
b320caecb3
HDFS-14085. RBF: LS command for root shows wrong owner and permission information. Contributed by Ayush Saxena.
2019-06-24 22:03:02 +05:30
19088e1b49
HDFS-14089. RBF: Failed to specify server's Kerberos pricipal name in NamenodeHeartbeatService. Contributed by Ranith Sardar.
2019-06-24 22:03:02 +05:30
f2355c7063
HDFS-13776. RBF: Add Storage policies related ClientProtocol APIs. Contributed by Dibyendu Karmakar.
2019-06-24 22:03:02 +05:30
f4bd1114ff
HDFS-14082. RBF: Add option to fail operations when a subcluster is unavailable. Contributed by Inigo Goiri.
2019-06-24 22:03:02 +05:30
fa55eacd35
HDFS-13834. RBF: Connection creator thread should catch Throwable. Contributed by CR Hota.
2019-06-24 22:03:02 +05:30
04caaba488
HDFS-13852. RBF: The DN_REPORT_TIME_OUT and DN_REPORT_CACHE_EXPIRE should be configured in RBFConfigKeys. Contributed by yanghuafeng.
2019-06-24 22:03:02 +05:30
ebfd2d8a4e
HDFS-12284. addendum to HDFS-12284. Contributed by Inigo Goiri.
2019-06-24 22:03:02 +05:30
6f2c871b05
HDFS-12284. RBF: Support for Kerberos authentication. Contributed by Sherwood Zheng and Inigo Goiri.
2019-06-24 22:03:02 +05:30
7b0bc492ed
HDFS-14024. RBF: ProvidedCapacityTotal json exception in NamenodeHeartbeatService. Contributed by CR Hota.
2019-06-24 22:03:02 +05:30
c5065bf20b
HDFS-13845. RBF: The default MountTableResolver should fail resolving multi-destination paths. Contributed by yanghuafeng.
2019-06-24 22:03:02 +05:30
b3fee1d2bf
HDFS-14011. RBF: Add more information to HdfsFileStatus for a mount point. Contributed by Akira Ajisaka.
2019-06-24 22:03:02 +05:30
41c94a636b
HDFS-13906. RBF: Add multiple paths for dfsrouteradmin 'rm' and 'clrquota' commands. Contributed by Ayush Saxena.
2019-06-24 22:03:02 +05:30
05145404d5
HDFS-14339. Inconsistent log level practices in RpcProgramNfs3.java. Contributed by Anuhan Torgonshar.
2019-06-24 08:30:48 -07:00
daa1e14745
when evictableMmapped or evictable size is zero, do not throw NoSuchElementException
...
Signed-off-by: sunlisheng <sunlisheng@xiaomi.com>
2019-06-24 20:17:22 +08:00
645d67bc4f
HDFS-14074. DataNode runs async disk checks maybe throws NullPointerException, and DataNode failed to register to NameSpace. Contributed by guangyi lu.
2019-06-21 18:27:03 -07:00
1524e2e6c5
HDFS-12487. FsDatasetSpi.isValidBlock() lacks null pointer check inside and neither do the callers. Contributed by liumi.
2019-06-21 18:17:36 -07:00
272b96d243
HDFS-13893. DiskBalancer: no validations for Disk balancer commands. Contributed by Lokesh Jain.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-20 20:23:05 -07:00
98d2065643
HDFS-12564. Add the documents of swebhdfs configurations on the client side. Contributed by Takanobu Asanuma.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-20 20:17:24 -07:00
5962a518bd
HDFS-14581. Appending to EC files crashes NameNode. Contributed by Surendra Singh Lilhore.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-20 09:40:29 -07:00
71ecd2e411
HDFS-14303. check block directory logic not correct when there is only meta file, print no meaning warn log. Contributed by qiang Liu.
2019-06-19 10:27:53 -07:00
f5ecc0bc08
HDFS-14103. Review Logging of BlockPlacementPolicyDefault. Contributed by David Mollitor.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-19 10:06:40 -07:00
d3ac516665
Revert "HDFS-13287. TestINodeFile#testGetBlockType results in NPE when run alone. Contributed by Virajith Jalaparti."
...
This reverts commit a1c3868c4f
2019-06-19 06:27:02 -07:00
48e564f7e2
HDFS-14537. Journaled Edits Cache is not cleared when formatting the JN. Contributed by Ranith Sardar.
2019-06-18 22:28:21 -07:00
a1c3868c4f
HDFS-13287. TestINodeFile#testGetBlockType results in NPE when run alone. Contributed by Virajith Jalaparti.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-18 22:17:49 -07:00
f9ee97de98
HDFS-14398. Update HAState.java to fix typos. Contributed by Nikhil Navadiya.
2019-06-18 21:58:37 -07:00
81ec909411
HDFS-14078. Admin helper fails to prettify NullPointerExceptions. Contributed by Elek, Marton.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-18 12:01:26 -07:00
3c1a1ceea9
HDFS-14487. Missing Space in Client Error Message (Contributed by Shweta Yakkali via Daniel Templeton)
...
Change-Id: I0f8ce74a35ab24fe94fd0e57d8247bb3fa575e6f
2019-06-18 10:21:22 -07:00
3ab77d9bc9
HDFS-14201. Ability to disallow safemode NN to become active. Contributed by Xiao Liang and He Xiaoqiao.
2019-06-18 09:58:29 -07:00
335c1c9938
HDFS-14010. Pass correct DF usage to ReservedSpaceCalculator builder. Contributed by Virajith Jalaparti.
2019-06-18 00:35:48 -07:00
54cdde38c7
HDFS-12314. Typo in the TestDataNodeHotSwapVolumes.testAddOneNewVolume(). Contributed by Oleg Danilov.
2019-06-17 23:11:25 -07:00
d41310a15d
HDFS-12315. Use Path instead of String to check closedFiles set. Contributed by Oleg Danilov.
2019-06-17 23:06:02 -07:00
098c325a78
HDFS-14340. Lower the log level when can't get postOpAttr. Contributed by Anuhan Torgonshar.
2019-06-17 22:45:52 -07:00
e3172221ea
HDFS-12770. Add doc about how to disable client socket cache. Contributed by Weiwei Yang.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-17 21:48:16 -07:00
62ad9885ea
HDFS-13730. BlockReaderRemote.sendReadResult throws NPE. Contributed by Yuanbo Liu.
2019-06-17 20:18:53 -07:00
a95e87d827
HDFS-11950. Disable libhdfs zerocopy test on Mac. Contributed by Akira Ajisaka.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-17 17:03:25 -07:00
6822193ee6
HDFS-12914. Block report leases cause missing blocks until next report. Contributed by Santosh Marella, He Xiaoqiao.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Co-authored-by: He Xiaoqiao <hexiaoqiao@apache.org>
2019-06-17 16:20:30 -07:00
7314185c4a
Revert "HDFS-12914. Block report leases cause missing blocks until next report. Contributed by Santosh Marella, He Xiaoqiao."
...
This reverts commit ae4143a529
2019-06-17 16:18:48 -07:00
a50c35bb81
Revert "HDFS-12914. Addendum patch. Block report leases cause missing blocks until next report. Contributed by Santosh Marella, He Xiaoqiao."
...
This reverts commit cdc5de6448
2019-06-17 16:18:37 -07:00
f9a7b442fd
HDFS-14465. When the Block expected replications is larger than the number of DataNodes, entering maintenance will never exit. Contributed by Yicong Cai.
2019-06-17 15:18:17 -07:00
cc1630288e
HDFS-14556: Spelling Mistake "gloablly" ( #938 )
2019-06-16 18:32:42 -07:00
8370a0ae16
HDFS-14203. Refactor OIV Delimited output entry building mechanism. Contributed by Adam Antal.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-15 06:47:10 -07:00
b24efa11ea
HDFS-14549. EditLogTailer shouldn't output full stack trace when interrupted. Contributed by Inigo Goiri.
2019-06-14 13:37:23 -07:00
cdc5de6448
HDFS-12914. Addendum patch. Block report leases cause missing blocks until next report. Contributed by Santosh Marella, He Xiaoqiao.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Co-authored-by: He Xiaoqiao <hexiaoqiao@apache.org>
2019-06-14 13:01:39 -07:00
ae4143a529
HDFS-12914. Block report leases cause missing blocks until next report. Contributed by Santosh Marella, He Xiaoqiao.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Co-authored-by: He Xiaoqiao <hexiaoqiao@apache.org>
2019-06-14 10:42:54 -07:00
9ebbda342f
HADOOP-16372. Fix typo in DFSUtil getHttpPolicy method
...
Closes #967
2019-06-14 16:09:37 +02:00
4f455290b1
HDFS-14560. Allow block replication parameters to be refreshable. Contributed by Stephen O'Donnell.
2019-06-13 18:26:53 -07:00
50de0874d0
HDFS-13231. Extend visualization for Decommissioning, Maintenance Mode under Datanode tab in the NameNode UI. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-12 10:30:23 -07:00
4ea6c2f457
HADOOP-16354. Enable AuthFilter as default for WebHDFS.
...
Contributed by Prabhu Joseph
2019-06-11 18:41:08 -04:00
5740eea081
HDFS-14513. FSImage which is saving should be clean while NameNode shutdown. Contributed by He Xiaoqiao.
2019-06-11 11:48:35 -07:00
101d5b5f86
HDFS-14234. Limit WebHDFS to specifc user, host, directory triples.
...
Contributed by Clay B.
2019-06-10 17:55:16 -07:00
bd46bdf9f9
HDFS-14553. Make queue size of BlockReportProcessingThread configurable. Contributed by He Xiaoqiao.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-10 17:21:52 -07:00
0d160a0ba8
HDFS-10210. Remove the defunct startKdc profile from hdfs. Contributed by Wei-Chiu Chuang, Akira Ajisaka.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Co-authored-by: Akira Ajisaka <aajisaka@apache.org>
2019-06-10 14:13:53 -07:00
7217494f40
HDFS-10659. Namenode crashes after Journalnode re-installation in an HA cluster due to missing paxos directory. Contributed by star, Hanisha Koneru.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-10 13:45:37 -07:00
d6d95d2686
HDFS-14494. Move Server logging of StatedId inside receiveRequestState(). Contributed by Shweta Yakkali.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-10 10:23:18 -07:00
e1dfc060f8
HDFS-14486. The exception classes in some throw statements do not accurately describe why they are thrown. Contributed by Ayush Saxena.
2019-06-06 11:59:53 -07:00
944adc61b1
HDFS-14527. Stop all DataNodes may result in NN terminate. Contributed by He Xiaoqiao.
2019-06-06 10:20:28 -07:00
294695dd57
HADOOP-16314. Make sure all web end points are covered by the same authentication filter.
...
Contributed by Prabhu Joseph
2019-06-05 18:55:13 -04:00
d1aad44490
HDFS-14356. Implement HDFS cache on SCM with native PMDK libs. Contributed by Feilong He.
2019-06-05 21:33:00 +08:00
ea3b0a1844
HDFS-14535. The default 8KB buffer in requestFileDescriptors#BufferedOutputStream is causing lots of heap allocation in HBase when using short-circut read
2019-06-04 08:48:05 -07:00
35f1014b3e
HDFS-13654. Use a random secret when a secret file doesn't exist in HttpFS. This should be default.
2019-05-31 10:29:24 +09:00
9122b9b649
HDFS-14521. Suppress setReplication logging. Contributed by Kihwal Lee
2019-05-30 17:28:45 -05:00
33c62f8f4e
HDFS-14497. Write lock held by metasave impact following RPC processing. Contributed by He Xiaoqiao.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-05-30 13:30:35 -07:00
6f5a36c13c
HADOOP-13656. fs -expunge to take a filesystem. Contributed by Shweta.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-05-30 13:21:58 -07:00
c1caab40f2
HDFS-14512. ONE_SSD policy will be violated while write data with DistributedFileSystem.create(....favoredNodes). Contributed by Ayush Saxena.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-05-29 20:56:38 -07:00
afd844059c
HADOOP-16331. Fix ASF License check in pom.xml
...
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2019-05-29 17:25:13 +09:00
d78854b928
HDFS-14434. Ignore user.name query parameter in secure WebHDFS.
...
Contributed by KWON BYUNGCHANG
2019-05-28 17:31:35 -04:00
9f933e6446
HADOOP-16323. https everywhere in Maven settings.
2019-05-27 15:24:59 +09:00
37900c5639
HDFS-14402. Use FileChannel.transferTo() method for transferring block to SCM cache. Contributed by Feilong He.
2019-05-26 14:30:11 +05:30
55e0c134f0
HDFS-14500. NameNode StartupProgress should not allow new steps in an already-completed phase. Contributed by Erik Krogen.
2019-05-24 13:12:59 -07:00
c31b7b8d52
HDFS-14502. keepResults option in NNThroughputBenchmark should call saveNamespace(). Contributed by Konstantin V Shvachko.
2019-05-24 12:34:29 -07:00
c9393dd17f
Revert "HDFS-14353. Erasure Coding: metrics xmitsInProgress become to negative. Contributed by maobaolong."
...
This reverts commit 1cb2eb0df3
2019-05-24 10:23:47 -07:00
f96a2df38d
HADOOP-16266. Add more fine-grained processing time metrics to the RPC layer. Contributed by Christopher Gregorian.
2019-05-23 10:28:37 -07:00
9dff6eff81
HDFS-14507. Document -blockingDecommission option for hdfs dfsadmin -listOpenFiles
...
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2019-05-22 13:22:26 +09:00
1cb2eb0df3
HDFS-14353. Erasure Coding: metrics xmitsInProgress become to negative. Contributed by maobaolong.
2019-05-20 17:22:58 -07:00
3e5e5b028a
HDFS-14323. Distcp fails in Hadoop 3.x when 2.x source webhdfs url has special characters in hdfs file path. Contributed by Srinivasu Majeti.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-05-17 19:20:28 +02:00
77170e70d1
HDFS-14390. Provide kerberos support for AliasMap service used by Provided storage. Contributed by Ashvin Agrawal
2019-05-15 12:15:47 -07:00
e2dfdaee7c
HDFS-14482: Crash when using libhdfs with bad classpath
2019-05-14 11:04:06 -07:00
389e640f0c
HADOOP-16161. NetworkTopology#getWeightUsingNetworkLocation return unexpected result. Contributed by He Xiaoqiao.
2019-05-13 11:46:16 -07:00
5847e00143
HDFS-14245. [SBN read] Enable ObserverReadProxyProvider to work with non-ClientProtocol proxy types. Contributed by Erik Krogen.
2019-05-13 08:39:59 -07:00
9b0aace1e6
HDFS-14401. Refine the implementation for HDFS cache on SCM. Contributed by Feilong He.
2019-05-08 17:20:21 +05:30
69b903bbd8
HDFS-14372. NPE while DN is shutting down. Contributed by lujie.
2019-05-05 16:33:52 +05:30
e424392a62
HDFS-14438. Fix typo in OfflineEditsVisitorFactory. Contributed by bianqi.
2019-05-05 13:38:09 +05:30
1d59cc490c
HADOOP-16292. Refactor checkTrustAndSend in SaslDataTransferClient to make it cleaner. Contributed by Sherwood Zheng.
2019-05-03 10:14:17 -07:00
f1875b205e
HADOOP-16059. Use SASL Factories Cache to Improve Performance. Contributed by Ayush Saxena.
2019-05-03 11:22:14 +05:30
d6b7609c96
HDFS-14453. Improve Bad Sequence Number Error Message. Contributed by Shweta.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-05-02 14:25:02 -07:00
7a3188d054
HADOOP-16282. Avoid FileStream to improve performance. Contributed by Ayush Saxena.
2019-05-02 12:58:42 -07:00
865c328930
HDFS-14460. DFSUtil#getNamenodeWebAddr should return HTTPS address based on policy configured. Contributed by CR Hota.
2019-05-02 10:09:21 -07:00
7cb46f035a
HDFS-14463. Add Log Level link under NameNode and DataNode Web UI Utilities dropdown. Contributed by Siyao Meng.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-05-01 16:48:44 -07:00
4877f0aa51
HDFS-3246: pRead equivalent for direct read path ( #597 )
...
HDFS-3246: pRead equivalent for direct read path
Contributed by Sahil Takiar
2019-04-30 14:52:16 -07:00
4b4200f1f8
HDFS-13677. Dynamic refresh Disk configuration results in overwriting VolumeMap. Contributed by xuzq.
2019-04-29 14:49:35 -07:00
a703dae25e
HADOOP-16222. Fix new deprecations after guava 27.0 update in trunk. Contributed by Gabor Bota.
2019-04-24 10:39:00 -06:00
174b7d3126
HDFS-14435. [SBN Read] Enable ObserverReadProxyProvider to gracefully handle StandbyException when fetching HAServiceState. Contributed by Erik Krogen.
2019-04-22 13:29:56 -07:00
5321235fe8
HDFS-14445. TestTrySendErrorReportWhenNNThrowsIOException fails in trunk. Contributed by Ayush Saxena.
2019-04-22 13:23:42 -07:00
685cb83e4c
HDFS-14433. Remove the extra empty space in the DataStreamer logging. Contributed by Yishuang Lu. ( #747 )
2019-04-17 10:38:48 -07:00
13907d8479
HDFS-14432. dfs.datanode.shared.file.descriptor.paths duplicated in hdfs-default.xml
...
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
2019-04-17 21:39:06 +09:00
be6c8014e6
HDFS-14418. Remove redundant super user priveledge checks from namenode. Contributed by Ayush Saxena.
2019-04-16 10:34:31 -07:00
2382f63fc0
HADOOP-14747. S3AInputStream to implement CanUnbuffer.
...
Author: Sahil Takiar <stakiar@cloudera.com>
2019-04-12 18:12:02 -07:00
626fec652b
HDFS-13699. Add DFSClient sending handshake token to DataNode, and allow DataNode overwrite downstream QOP. Contributed by Chen Liang.
2019-04-12 17:37:51 -07:00
586826fe99
HDFS-12245. Fix INodeId javadoc
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-04-11 10:01:47 +09:00
e9b859f749
HDFS-14371. Improve Logging in FSNamesystem by adding parameterized logging. Contributed by Shweta.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-04-05 16:09:00 -07:00
260d843b25
HDFS-14416. Fix TestHdfsConfigFields for field dfs.client.failover.resolver.useFQDN. Contributed by Fengnan Li.
2019-04-05 09:26:06 -07:00
Inigo Goiri
Ayush Saxena
Sahil Takiar
Zhankun Tang
Chen Liang
Takanobu Asanuma
Steve Loughran
Prabhu Joseph
Wei-Chiu Chuang
Vinayakumar B
Rohith Sharma K S
Rakesh Radhakrishnan
Malcolm Taylor
Sahil Takiar
Stephen O'Donnell
Surendra Singh Lilhore
Akira Ajisaka
Erik Krogen
Xiaoyu Yao
leosunli
Stephen O'Donnell
Istvan Fajth
He Xiaoqiao
CR Hota
Shashikant Banerjee
Jonathan Hung
Arpit Agarwal
Wang Yuxuan
Anu Engineer
Sandeep Kumar
Sunil G
HUAN-PING SU
hunshenshi
LeonGao91
Siyao Meng
Masatake Iwasaki
Siyao Meng
Yiqun Lin
Lokesh Jain
Chao Sun
Ajay Yadav
Weiwei Yang
paulward24
Inigo Goiri
Ajay Kumar
Brahma Reddy Battula
Giovanni Matteo Fumarola
sunlisheng
David Mollitor
Virajith Jalaparti
Bharat Viswanadham
Elek, Marton
Shweta Yakkali
Oleg Danilov
Santosh Marella
mpicker90
Adam Antal
dineshchitlangia
Eric Yang
Hanisha Koneru
Sammi Chen
huzheng
Kihwal Lee
Konstantin V Shvachko
Christopher Gregorian
Srinivasu Majeti
Sahil Takiar
Sean Mackrory
lys0716
Puleya7