dc32f583af
HDFS-14934. [SBN Read] Standby NN throws many InterruptedExceptions when dfs.ha.tail-edits.period is 0. Contributed by Ayush Saxena.
2019-12-28 21:32:15 +09:00
8730a7bf60
HDFS-15003. RBF: Make Router support storage type quota. Contributed by Jinglun.
2019-12-27 09:22:58 +05:30
80f91d14ab
HDFS-14997. Addendum: BPServiceActor processes commands from NameNode asynchronously. Contributed by Xiaoqiao He.
2019-12-27 09:12:44 +05:30
300505c562
HDFS-15054. Delete Snapshot not updating new modification time. Contributed by hemanthboyina.
2019-12-25 12:42:50 +05:30
df622cf4a3
HDFS-12999. When reach the end of the block group, it may not need to flush all the data packets(flushAllInternals) twice. Contributed by lufei and Fei Hui.
2019-12-25 11:07:25 +05:30
d8cd7098b4
HDFS-15073. Replace curator-shaded guava import with the standard one ( #1782 )
2019-12-25 10:50:01 +09:00
b98ac2a3af
HDFS-15076. Fix tests that hold FSDirectory lock, without holding FSNamesystem lock. Contributed by Konstantin V Shvachko.
2019-12-24 12:08:34 -08:00
52d7b745c6
HDFS-15062. Add LOG when sendIBRs failed. Contributed by Fei Hui.
2019-12-19 09:42:17 -08:00
b86895485d
HDFS-14997. BPServiceActor processes commands from NameNode asynchronously. Contributed by Xiaoqiao He.
2019-12-19 09:34:43 -08:00
fdd96e46d1
HDFS-15012. NN fails to parse Edit logs after applying HDFS-13101. Contributed by Shashikant Banerjee.
2019-12-18 22:50:46 +05:30
24080666e5
HDFS-14908. LeaseManager should check parent-child relationship when filter open files. Contributed by Jinglun.
2019-12-16 18:41:45 -08:00
dc6cf17b34
HDFS-15048. Fix findbug in DirectoryScanner.
2019-12-16 19:00:57 +09:00
7a87007545
HDFS-15038. TestFsck testFsckListCorruptSnapshotFiles is failing in trunk. Contributed by hemanthboyina.
2019-12-15 00:33:32 +05:30
72aee114f8
HDFS-15053. RBF: Add permission check for safemode operation. Contributed by Xiaoqiao He.
2019-12-14 17:57:23 +05:30
7fe924b1c0
HDFS-15016. RBF: getDatanodeReport() should return the latest update. Contributed by Inigo Goiri.
2019-12-13 10:51:14 -08:00
65c4660bcd
HDFS-15036. Active NameNode should not silently fail the image transfer. Contributed by Chen Liang.
2019-12-12 10:22:05 -08:00
bdd00f10b4
HDFS-15047. Document the new decommission monitor (HDFS-14854). ( #1755 )
2019-12-13 00:35:41 +09:00
0e28cd8f63
HDFS-15050. Optimize log information when DFSInputStream meet CannotObtainBlockLengthException. Contributed by Xiaoqiao He.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-12-12 02:23:32 -08:00
93bb368094
HDFS-14983. RBF: Add dfsrouteradmin -refreshSuperUserGroupsConfiguration command option. Contributed by Xieming Li
2019-12-12 10:56:19 +09:00
c174d50b30
HDFS-15032. Properly handle InvocationTargetExceptions in the proxy created by ProxyCombiner. This fixes a bug encountered by the HDFS balancer when used with Observer Nodes. Contributed by Erik Krogen.
2019-12-11 09:39:37 -08:00
c2e9783d5f
HDFS-15045. DataStreamer#createBlockOutputStream() should log exception in warn. Contributed by Ravuri Sushma Sree.
2019-12-11 08:52:55 +05:30
c93cb6790e
HDFS-14854. Create improved decommission monitor implementation. Contributed by Stephen O'Donnell.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-12-10 17:16:43 -08:00
c4733377d0
HDFS-15040. RBF: Secured Router should not run when SecretManager is not running. ( #1745 )
2019-12-10 16:03:32 +09:00
9f09852051
HDFS-15043. RBF: The detail of the Exception is not shown in ZKDelegationTokenSecretManagerImpl ( #1748 )
2019-12-10 16:00:20 +09:00
4dffd81bb7
HDFS-14522. Allow compact property description in xml in httpfs. ( #1737 )
2019-12-10 10:30:57 +09:00
11cd5b6e39
HDFS-15028. Keep the capacity of volume and reduce a system call. Contributed by Yang Yun.
...
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
2019-12-07 08:46:45 +09:00
ecd461f940
HDFS-14751. Synchronize on diffs in DirectoryScanner. Contributed by Lisheng Sun.
2019-12-06 15:10:40 -08:00
313b76f8e9
HDFS-14476. lock too long when fix inconsistent blocks between disk and in-memory. Contributed by Sean Chow.
2019-12-06 15:10:40 -08:00
705b172b95
HDFS-14998. [SBN read] Update Observer Namenode doc for ZKFC after HDFS-14130. Contributed by Fei Hui.
2019-12-06 23:04:09 +05:30
83a14559e5
HDFS-15023. [SBN read] ZKFC should check the state before joining the election. Contributed by Fei Hui.
2019-12-05 18:52:01 +05:30
47fdae7904
HDFS-13811. RBF: Race condition between router admin quota update and periodic quota update service. Contributed by Jinglun.
2019-12-04 18:19:11 +08:00
0c217feed8
HDFS-15026. TestPendingReconstruction#testPendingReconstruction() fail in trunk. Contributed by Lisheng Sun.
2019-12-03 09:18:09 +05:30
4ede8bce28
HDFS-9695. HTTPFS - CHECKACCESS operation missing. Contributed by hemanthboyina
2019-12-03 09:38:36 +09:00
6b2d6d4aaf
HDFS-15009. FSCK -list-corruptfileblocks return Invalid Entries. Contributed by hemanthboyina.
2019-11-29 23:55:37 +05:30
44f7b9159d
HDFS-15013. Reduce NameNode overview tab response time. Contributed by HuangTao.
2019-11-29 00:42:31 +05:30
0384687811
HDFS-15010. BlockPoolSlice#addReplicaThreadPool static pool should be initialized by static method. Contributed by Surendra Singh Lilhore.
2019-11-28 23:49:35 +05:30
46166bd8d1
HDFS-14961. [SBN read] Prevent ZKFC changing Observer Namenode state. Contributed by Ayush Saxena.
2019-11-28 18:28:20 +05:30
c3659f8f94
HDFS-15019. Refactor the unit test of TestDeadNodeDetection. Contributed by Lisheng Sun.
2019-11-28 14:41:49 +08:00
2b452b4e60
HDFS-14986. ReplicaCachingGetSpaceUsed throws ConcurrentModificationException. Contributed by Aiphago.
2019-11-28 10:43:35 +08:00
c8bef4d6a6
HDFS-14649. Add suspect probe for DeadNodeDetector. Contributed by Lisheng Sun.
2019-11-27 10:57:20 +08:00
049940e77b
HDFS-14519. NameQuota is not update after concat operation, so namequota is wrong. Contributed by Ranith Sardar.
2019-11-23 02:30:02 +05:30
c422e36397
HDFS-13842. RBF: Exceptions are conflicting when creating the same mount entry twice. Contributed by Ranith Sardar.
2019-11-22 11:42:19 +05:30
b25e94ce29
HDFS-14924. RenameSnapshot not updating new modification time. Contributed by hemanthboyina
2019-11-22 13:48:02 +09:00
9b6906fe91
HDFS-14651. DeadNodeDetector checks dead node periodically. Contributed by Lisheng Sun.
2019-11-22 10:53:55 +08:00
b89fd4dfe9
HDFS-15002. RBF: Fix annotation in RouterAdmin. Contributed by Jinglun.
2019-11-22 01:08:11 +05:30
26270196a2
HDFS-14940. HDFS Balancer : Do not allow to set balancer maximum network bandwidth more than 1TB. Contributed by hemanthboyina.
2019-11-22 00:31:25 +05:30
98d249dcda
HDFS-14996. RBF: GetFileStatus fails for directory with EC policy set in case of multiple destinations. Contributed by Ayush Saxena.
2019-11-21 12:08:42 +05:30
3037762b2c
HDFS-14949. Add getServerDefaults() support to HttpFS. Contributed by hemanthboyina.
2019-11-20 10:51:58 -08:00
fd264b8265
HDFS-14995. Use log variable directly instead of passing as argument in InvalidateBlocks#printBlockDeletionTime(). Contributed by Lisheng Sun.
2019-11-21 00:00:42 +05:30
0b50aa29fd
HDFS-14952. Skip safemode if blockTotal is 0 in new NN. Contributed by Xiaoqiao He.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Mukul Kumar Singh <msingh@apache.org>
2019-11-19 12:16:57 -08:00
c870514740
HDFS-14992. TestOfflineEditsViewer is failing in Trunk. Contributed by hemanthboyina.
2019-11-20 00:07:43 +05:30
34cb595d2e
HDFS-14967. TestWebHDFS fails in Windows. Contributed by Renukaprasad C.
2019-11-18 18:49:58 +05:30
12617fad2e
HDFS-14955. RBF: getQuotaUsage() on mount point should return global quota. Contributed by Jinglun.
2019-11-18 16:04:52 +05:30
3b5a0e86c1
HDFS-14974. RBF: Make tests use free ports. Contributed by Inigo Goiri.
2019-11-18 15:49:36 +05:30
b3119b9ab6
HDFS-14648. Implement DeadNodeDetector basic model. Contributed by Lisheng Sun.
2019-11-16 11:32:41 +08:00
67f2c491fe
HDFS-14802. The feature of protect directories should be used in RenameOp ( #1669 )
2019-11-15 12:35:37 -08:00
c892a879dd
HDFS-14882. Consider DataNode load when #getBlockLocation. Contributed by Xiaoqiao He.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Istvan Fajth <pifta@cloudera.com>
2019-11-15 12:16:31 -08:00
b2cc8b6b4a
HDFS-14973. More strictly enforce Balancer/Mover/SPS throttling of getBlocks RPCs to NameNodes. Contributed by Erik Krogen.
2019-11-15 10:10:37 -08:00
586defe711
HDFS-14979 Allow Balancer to submit getBlocks calls to Observer Nodes when possible. Contributed by Erik Krogen.
2019-11-13 11:46:35 -08:00
97ec34e117
HDFS-14959: [SBNN read] access time should be turned off ( #1706 ) Contributed by Chao Sun.
2019-11-12 11:35:50 -08:00
40150da1e1
HDFS-14922. Prevent snapshot modification time got change on startup. Contributed by hemanthboyina.
2019-11-12 10:53:54 -08:00
77934bc07b
HDFS-14975. Add CR for SetECPolicyCommand usage. Contributed by Fei Hui.
2019-11-11 11:05:34 +05:30
b25a37c322
HDFS-14962. RBF: ConnectionPool#newConnection() error log wrong protocol class ( #1699 ). Contributed by Yuxuan Wang.
2019-11-11 10:48:38 +05:30
6663d6a5c2
HDFS-14928. UI: unifying the WebUI across different components. Contributed by Xieming Li
2019-11-11 14:04:57 +09:00
320008bb7c
HDFS-14720. DataNode shouldn't report block as bad block if the block length is Long.MAX_VALUE. Contributed by hemanthboyina.
2019-11-09 22:14:35 +05:30
42fc8884ab
HDFS-14815. RBF: Update the quota in MountTable when calling setQuota on a MountTable src. Contributed by Jinglun.
2019-11-08 08:48:52 +05:30
247584eb63
HDFS-14958. TestBalancerWithNodeGroup is not using NetworkTopologyWithNodeGroup. Contributed by Jim Brennan.
2019-11-07 22:49:01 +05:30
dd900259c4
HDFS-14941. Potential editlog race condition can cause corrupted file. Contributed by Chen Liang and Konstantin Shvachko.
2019-11-06 09:56:19 -08:00
9d0d580031
HDFS-14806. Bootstrap standby may fail if with in-progress tailing. Contributed by Chen Liang and Erik Krogen.
2019-11-06 09:30:04 -08:00
9e287054a8
HDFS-14942. Change Log Level to debug in JournalNodeSyncer#syncWithJournalAtIndex. Contributed by Lisheng Sun.
2019-11-06 22:02:03 +05:30
dcf55838ae
HDFS-14880. Correct the sequence of statistics & exit message in balencer. Contributed by Renukaprasad C.
2019-11-06 21:45:03 +05:30
c36014165c
HDFS-14384. When lastLocatedBlock token expire, it will take 1~3s second to refetch it. Contributed by Surendra Singh Lilhore.
2019-11-06 19:28:55 +05:30
bfb8f28cc9
HDFS-14775. Add Timestamp for longest FSN write/read lock held log. Contributed by Chen Zhang.
2019-11-05 09:19:52 -08:00
7901062707
HDFS-14891. RBF: namenode links in NameFederation Health page (federationhealth.html) cannot use https scheme. Contributed by Xieming Li
2019-11-05 15:03:09 +09:00
b643a1cbe8
HDFS-14938. Add check if excludedNodes contain scope in DFSNetworkTopology#chooseRandomWithStorageType(). Contributed by Lisheng Sun.
2019-11-05 09:23:20 +05:30
2ffec347eb
HDFS-14946. Erasure Coding: Block recovery failed during decommissioning. Contributed by Fei Hui.
2019-11-05 01:37:15 +05:30
eb73ba6ed5
HDFS-14945. Revise PacketResponder's log. Contributed by Xudong Cao.
2019-11-04 09:40:42 -08:00
de6b8b0c0b
HDFS-14925. Rename operation should check nest snapshot ( #1670 )
...
If the src directory or any of its descendant is snapshottable
and the dst directory or any of its ancestors is snapshottable,
we consider this as nested snapshot, which should be denied.
Reviewed-by: Shashikant Banerjee <shashikant@apache.org>
2019-11-01 16:37:53 -07:00
7d7acb004a
HDFS-13736. BlockPlacementPolicyDefault can not choose favored nodes when 'dfs.namenode.block-placement-policy.default.prefer-local-node' set to false. Contributed by hu xiaodong.
2019-11-01 22:50:10 +05:30
f18bbdd9d8
HDFS-14927. RBF: Add metrics for async callers thread pool. Contributed by Leon Gao.
2019-11-01 10:14:31 -07:00
02009c3bb7
HDFS-14768. EC : Busy DN replica should be consider in live replica check. Contributed by guojh.
2019-11-01 22:15:05 +05:30
9d25ae7669
HDFS-14920. Erasure Coding: Decommission may hang If one or more datanodes are out of service during decommission. Contributed by Fei Hui.
2019-10-31 23:49:37 +05:30
d9fbedc4ae
HDFS-14936. Add getNumOfChildren() for interface InnerNode. Contributed by Lisheng Sun.
2019-10-31 23:39:27 +05:30
e6137d0e50
Make upstream aware of 2.10.0 release
2019-10-30 20:58:32 -07:00
fa4904cdca
HDFS-14935. Refactor DFSNetworkTopology#isNodeInScope. Contributed by Lisheng Sun.
2019-10-29 12:30:24 +05:30
30ed24a421
HDFS-14730. Removed unused configuration dfs.web.authentication.filter.
...
Contributed by Chen Zhang
2019-10-28 19:27:32 -04:00
9ef6ed9c1c
HDFS-14931. hdfs crypto commands limit column width. Contributed by Eric Badger
2019-10-28 16:37:04 +00:00
30db895b59
HDFS-14308. DFSStripedInputStream curStripeBuf is not freed by unbuffer() ( #1667 )
...
Reviewed-by: Aravindan Vijayan <avijayan@cloudera.com>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-10-25 13:09:14 -07:00
862526530a
HDFS-14933. Fixing a typo in documentation of Observer NameNode. Contributed by Xieming Li
2019-10-25 15:23:05 +09:00
0db0f1e399
HDFS-14917. Change the ICON of "Decommissioned & dead" datanode on "dfshealth.html". Contributed by Xieming Li
2019-10-25 10:27:38 +09:00
a1b4eebcc9
HDFS-14910. Rename Snapshot with Pre Descendants Fail With IllegalArgumentException. Contributed by Wei-Chiu Chuang. ( #1672 )
...
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
2019-10-24 13:09:52 -07:00
ac6b6a6a85
HADOOP-16614. Add aarch64 support for dependent leveldbjni.
...
Contributed by liusheng
close #1546
2019-10-24 11:45:57 -04:00
ee699dc26c
HDFS-14921. Remove SuperUser Check in Setting Storage Policy in FileStatus During Listing. Contributed by Ayush Saxena.
2019-10-24 12:14:09 +05:30
a901405ad8
HDFS-14884. Add sanity check that zone key equals feinfo key while setting Xattrs. Contributed by Mukul Kumar Singh.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-10-22 13:31:15 -07:00
6020505943
HDFS-14915. Move Superuser Check Before Taking Lock For Encryption API. Contributed by Ayush Saxena.
2019-10-22 18:44:22 +05:30
19f35cfd57
HDFS-14918. Remove useless getRedundancyThread from BlockManagerTestUtil. Contributed by Fei Hui.
2019-10-22 18:34:02 +05:30
72003b19bf
HDFS-13901. INode access time is ignored because of race between open and rename. Contributed by Jinglun.
2019-10-21 17:31:43 -07:00
c4e27ef773
HDFS-12749. DN may not send block report to NN after NN restart. Contributed by Xiaoqiao He.
2019-10-21 16:33:15 -05:00
74c2329fc3
HDFS-14913. Correct the value of available count in DFSNetworkTopology#chooseRandomWithStorageType(). Contributed by Ayush Saxena.
2019-10-21 18:35:12 +05:30
ff6a492d66
HDFS-14916. RBF: line breaks are missing from the output of 'hdfs dfsrouteradmin -ls'. Contributed by Xieming Li.
2019-10-21 18:20:00 +05:30
447f46d962
HDFS-14847. Erasure Coding: Blocks are over-replicated while EC decommissioning. Contributed by Fei Hui.
2019-10-20 06:10:59 +05:30
e6f95eb0f7
HDFS-14887. RBF: In Router Web UI, Observer Namenode Information displaying as Unavailable. Contributed by hemanthboyina.
2019-10-19 13:24:58 -07:00
3d41f33018
HADOOP-16152. Upgrade Eclipse Jetty version to 9.4.x. Contributed by Yuming Wang, Siyao Meng.
...
Co-authored-By: Siyao Meng <smeng@cloudera.com>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-10-18 08:25:02 -07:00
54dc6b7d72
HDFS-14909. DFSNetworkTopology#chooseRandomWithStorageType() should not decrease storage count for excluded node which is already part of excluded scope. Contributed by Surendra Singh Lilhore.
2019-10-17 22:28:47 +05:30
5527d79adb
HDFS-14810. Review FSNameSystem editlog sync. Contributed by Xiaoqiao He.
2019-10-17 21:56:30 +05:30
375224edeb
HDFS-14739. RBF: LS command for mount point shows wrong owner and permission information. Contributed by Jinglun.
2019-10-16 19:26:01 +05:30
336abbd873
HDFS-14886. In NameNode Web UI's Startup Progress page, Loading edits always shows 0 sec. Contributed by hemanthboyina.
2019-10-14 23:31:43 +05:30
fabd41fa48
HDFS-14856. Fetch file ACLs while mounting external store. ( #1478 )
2019-10-14 09:44:56 -07:00
5f4641a120
HDFS-14238. A log in NNThroughputBenchmark should change log level to INFO instead of ERROR. Contributed by Shen Yinjie.
2019-10-12 21:06:23 +05:30
6e5cd5273f
HDFS-14899. Use Relative URLS in Hadoop HDFS RBF. Contributed by David Mollitor.
2019-10-12 13:26:47 +05:30
eeb58a07e2
HDFS-14898. Use Relative URLS in Hadoop HDFS HTTP FS. Contributed by David Mollitor.
2019-10-10 01:03:33 +05:30
d76e2655ac
HDFS-14754. Erasure Coding : The number of Under-Replicated Blocks never reduced(addendum). Contributed by Surendra Singh Lilhore.
2019-10-09 23:20:28 +05:30
72ae371e7a
HDFS-14509. DN throws InvalidToken due to inequality of password when upgrade NN 2.x to 3.x. Contributed by Yuxuan Wang and Konstantin Shvachko.
2019-10-08 11:56:52 -07:00
91320b4461
HDFS-14859. Prevent unnecessary evaluation of costly operation getNumLiveDataNodes when dfs.namenode.safemode.min.datanodes is not zero. Contributed by Srinivasu Majeti.
2019-10-08 14:14:14 +05:30
761594549e
HDFS-14814. RBF: RouterQuotaUpdateService supports inherited rule. Contributed by Jinglun.
2019-10-08 14:01:44 +05:30
382967be51
HDFS-14373. EC : Decoding is failing when block group last incomplete cell fall in to AlignedStripe. Contributed by Surendra Singh Lilhore.
2019-10-08 00:14:30 +05:30
aa24add8f0
HDFS-14890. Fixed namenode and journalnode startup on Windows.
...
Contributed by Siddharth Wagle
2019-10-04 13:13:10 -04:00
b23bdaf085
HDFS-14879. Header was wrong in Snapshot web UI. Contributed by hemanthboyina
2019-10-04 16:47:06 +09:00
c99a12167f
HDFS-14637. Namenode may not replicate blocks to meet the policy after enabling upgradeDomain. Contributed by Stephen O'Donnell.
...
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-10-03 22:13:50 -07:00
844b766da5
HDFS-14889. Ability to check if a block has a replica on provided storage. Contributed by Ashvin Agrawal. ( #1573 )"
2019-10-03 21:32:48 -07:00
a3fe404244
HDFS-14881. Safemode 'forceExit' option, doesn’t shown in help message. Contributed by Renukaprasad C.
2019-10-03 22:34:55 +05:30
5a7483ca5c
HDFS-14888. RBF: Enable Parallel Test Profile for builds. Contributed by Ayush Saxena.
2019-10-03 19:21:27 +05:30
1303255aee
HDFS-14858. [SBN read] Allow configurably enable/disable AlignmentContext on NameNode. Contributed by Chen Liang.
2019-10-02 16:38:20 -07:00
3df733c250
HDFS-14885. UI: Fix a typo on WebUI of DataNode. Contributed by Xieming Li.
2019-10-02 11:03:16 +09:00
6ef6594c7e
HDFS-14492. Snapshot memory leak. Contributed by Wei-Chiu Chuang. ( #1370 )
...
* HDFS-14492. Snapshot memory leak. Contributed by Wei-Chiu Chuang.
Change-Id: I9e5e450c07ad70aa1905973896c4f627042dbd37
* Fix checkstyle
Change-Id: I16d4bd4f03a971e1ed36cf57d89dc42357ef8fbf
2019-10-01 21:16:46 +05:30
b3275ab1f2
HDFS-14305. Fix serial number calculation in BlockTokenSecretManager to avoid token key ID overlap between NameNodes. Contributed by Konstantin V Shvachko.
2019-09-30 16:48:10 -07:00
760b523e58
Revert "HDFS-14305. Fix serial number calculation in BlockTokenSecretManager to avoid token key ID overlap between NameNodes. Contributed by He Xiaoqiao."
...
This reverts commit 0feba4396f
2019-09-29 13:11:31 -07:00
d8313b2274
HDFS-14850. Optimize FileSystemAccessService#getFileSystemConfiguration. Contributed by Lisheng Sun.
2019-09-28 17:20:44 -07:00
2200871607
HDFS-14876. Remove unused imports from TestBlockMissingException.java and TestClose.java. Contributed by Lisheng Sun.
2019-09-28 22:03:53 +05:30
c4c8d5fd0e
HDFS-14849. Erasure Coding: the internal block is replicated many times when datanode is decommissioning. Contributed by HuangTao.
2019-09-28 21:44:44 +05:30
0d5d0b914a
Revert "HDFS-14849. Erasure Coding: the internal block is replicated many times when datanode is decommissioning. Contributed by HuangTao."
...
This reverts commit ce58c05f1d
2019-09-28 21:32:15 +05:30
13b427fc05
HDFS-14564: Add libhdfs APIs for readFully; add readFully to ByteBufferPositionedReadable ( #963 ) Contributed by Sahil Takiar.
...
Reviewed-by: Siyao Meng <smeng@cloudera.com>
2019-09-27 12:25:03 -07:00
ce58c05f1d
HDFS-14849. Erasure Coding: the internal block is replicated many times when datanode is decommissioning. Contributed by HuangTao.
2019-09-27 18:58:19 +05:30
2870668cfa
Make upstream aware of 3.1.3 release.
2019-09-27 16:44:07 +08:00
2adcc3c932
HDFS-14785. [SBN read] Change client logging to be less aggressive. Contributed by Chen Liang.
2019-09-26 15:48:19 -07:00
b1e55cfb55
HDFS-14461. RBF: Fix intermittently failing kerberos related unit test. Contributed by Xiaoqiao He.
2019-09-26 15:26:23 -07:00
1a2a352ecd
HDFS-11934. Add assertion to TestDefaultNameNodePort#testGetAddressFromConf. Contributed by Nikhil Navadiya.
2019-09-26 20:45:37 +05:30
a3f68933d7
HDFS-14873. Fix dfsadmin doc for triggerBlockReport. Contributed by Fei Hui.
2019-09-26 19:54:07 +05:30
bdaaa3bbf2
HDFS-14832. RBF: Add Icon for ReadOnly False. Contributed by hemanthboyina
2019-09-25 23:48:49 +09:00
e346e3638c
HADOOP-15691 Add PathCapabilities to FileSystem and FileContext.
...
Contributed by Steve Loughran.
This complements the StreamCapabilities Interface by allowing applications to probe for a specific path on a specific instance of a FileSystem client
to offer a specific capability.
This is intended to allow applications to determine
* Whether a method is implemented before calling it and dealing with UnsupportedOperationException.
* Whether a specific feature is believed to be available in the remote store.
As well as a common set of capabilities defined in CommonPathCapabilities,
file systems are free to add their own capabilities, prefixed with
fs. + schema + .
The plan is to identify and document more capabilities -and for file systems which add new features, for a declaration of the availability of the feature to always be available.
Note
* The remote store is not expected to be checked for the feature;
It is more a check of client API and the client's configuration/knowledge
of the state of the remote system.
* Permissions are not checked.
Change-Id: I80bfebe94f4a8bdad8f3ac055495735b824968f5
2019-09-25 12:16:41 +01:00
3f89084ac7
HDFS-14845. Ignore AuthenticationFilterInitializer for HttpFSServerWebServer and honor hadoop.http.authentication configs.
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-09-25 10:19:30 +09:00
66400c1cbb
HDFS-14808. EC: Improper size values for corrupt ec block in LOG. Contributed by Ayush Saxena.
2019-09-25 01:31:15 +05:30
eb96a3093e
HDFS-14655. [SBN Read] Namenode crashes if one of The JN is down. Contributed by Ayush Saxena.
2019-09-25 01:16:30 +05:30
afa1006a53
HDFS-14843. Double Synchronization in BlockReportLeaseManager. Contributed by David Mollitor.
2019-09-24 09:58:42 -07:00
816d3cb908
HDFS-14837. Review of Block.java. Contributed by David Mollitor.
2019-09-24 09:54:09 -07:00
43203b466d
HDFS-14868. RBF: Fix typo in TestRouterQuota. Contributed by Jinglun.
...
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-09-24 09:38:58 -07:00
c30e495557
HDFS-14853. NPE in DFSNetworkTopology#chooseRandomWithStorageType() when the excludedNode is not present. Contributed by Ranith Sardar.
2019-09-23 21:22:50 +05:30
07c81e9bfc
HADOOP-16558. [COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes ( #1494 ). Contributed by Vinayakumar B.
2019-09-23 12:37:47 +05:30
4c0a7a9e13
Make upstream aware of 3.2.1 release.
2019-09-23 06:20:54 +00:00
659c88801d
HDFS-14818. Check native pmdk lib by 'hadoop checknative' command. Contributed by Feilong He.
2019-09-22 22:02:54 +05:30
aa938662f9
HDFS-14833. RBF: Router Update Doesn't Sync Quota. Contributed by Ayush Saxena.
2019-09-21 09:53:13 +05:30
3f223bebfa
HDFS-14844. Make buffer of BlockReaderRemote#newBlockReader#BufferedOutputStream configurable. Contributed by Lisheng Sun.
2019-09-20 11:45:01 -07:00
1654497f98
HADOOP-16557. [pb-upgrade] Upgrade protobuf.version to 3.7.1 ( #1432 )
...
HADOOP-16557. [pb-upgrade] Upgrade protobuf.version to 3.7.1. Contributed by Vinayakumar B.
2019-09-20 16:08:30 +05:30
a79f286c6f
HDFS-14609. RBF: Security should use common AuthenticationFilter. Contributed by Chen Zhang.
2019-09-19 11:00:10 -07:00
56248f9d87
HADOOP-16556. Fix some alerts raised by LGTM.
...
Contributed by Malcolm Taylor.
Change-Id: Ic60c3f4681dd9d48b3afcba7520bd1e4d3cc4231
2019-09-19 16:00:05 +01:00
4ed0aefe9f
HDFS-14822. [SBN read] Revisit GlobalStateIdContext locking when getting server state id. Contributed by Chen Liang.
2019-09-18 16:47:46 -07:00
5dd859a8a0
HDFS-14836. FileIoProvider should not increase FileIoErrors metric in datanode volume metric. Contributed by Aiphago.
...
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
2019-09-18 09:34:01 -07:00
3cf6e4272f
HDFS-14846: libhdfs tests are failing on trunk due to jni usage bugs
...
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-09-17 15:51:39 -07:00
f580a87079
HDFS-14795. Add Throttler for writing block. Contributed by Lisheng Sun.
2019-09-17 14:55:34 -07:00
e04b8a46c3
HDFS-14799. Do Not Call Map containsKey In Conjunction with get. Contributed by hemanthboyina.
2019-09-14 10:39:20 +05:30
06ad6540b3
HDFS-14821. Make HDFS-14617 (fsimage sub-sections) off by default. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-09-13 10:36:45 -07:00
4852a90e4b
HDFS-14754. Erasure Coding : The number of Under-Replicated Blocks never reduced. Contributed by hemanthboyina.
2019-09-13 00:17:13 +05:30
2ff2a7f612
HDFS-14798. Synchronize invalidateBlocks in DatanodeDescriptor. Contributed by hemanthboyina.
2019-09-12 19:43:18 +05:30
d1c303a497
HDFS-14699. Erasure Coding: Storage not considered in live replica when replication streams hard limit reached to threshold. Contributed by Zhao Yi Ming.
2019-09-12 19:11:50 +05:30
68612a0410
HDFS-14840. Use Java Conccurent Instead of Synchronization in BlockPoolTokenSecretManager. Contributed by David Mollitor.
2019-09-12 12:41:57 +09:00
c255333e20
HDFS-14838. RBF: Display RPC (instead of HTTP) Port Number in RBF web UI. Contributed by Xieming Li
2019-09-11 16:54:08 +09:00
524b553a5f
HDFS-14835. RBF: Secured Router should not run when it can't initialize DelegationTokenSecretManager. ( #1414 )
2019-09-11 10:32:07 +09:00
c92a3e94d8
HADOOP-15565. Add an inner FS cache to ViewFileSystem, separate from the global cache, to avoid file system leaks. Contributed by Jinglun.
2019-09-06 10:22:28 -07:00
d98c54816d
HDFS-13913. LazyPersistFileScrubber.run() should log meaningful warn message. Contributed by Daniel Green.
2019-09-06 18:12:42 +05:30
fa7f03fc56
HDFS-14826. dfs.ha.zkfc.port property duplicated in hdfs-default.xml. Contributed by Renukaprasad C.
2019-09-06 17:55:30 +05:30
494d75eb2b
HDFS-14784. Add more methods to WebHdfsTestUtil to support tests outside of package. Contributed by Chen Zhang.
2019-09-05 21:15:17 -07:00
72d8b92ba5
HDFS-12904. Add DataTransferThrottler to the Datanode transfers. Contributed by Lisheng Sun.
2019-09-05 11:44:02 -07:00
2f70b52a5b
HDFS-14812. RBF: MountTableRefresherService should load cache when refresh. Contributed by xuzq.
2019-09-05 08:29:56 +05:30
05704754a0
HDFS-14777. RBF: Set ReadOnly is failing for mount Table but actually readonly succeed to set. Contributed by Ranith Sardar.
2019-09-04 15:09:09 +05:30
3c117163a3
HDFS-14633. The StorageType quota and consume in QuotaFeature is not handled for rename. Contributed by Jinglun.
2019-09-03 09:29:58 -07:00
91b01a1db7
HDFS-14807. SetTimes updates all negative values apart from -1. Contributed by Ayush Saxena.
2019-09-03 18:08:42 +05:30
5ff76cb8bc
HDFS-14630. Configuration.getTimeDurationHelper() should not log time unit warning in info log. Contributed by hemanthboyina.
2019-09-03 12:37:09 +05:30
915cbc91c0
HDFS-14706. Checksums are not checked if block meta file is less than 7 bytes. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-09-02 09:47:04 -07:00
d207aba026
Revert "HDFS-14706. Checksums are not checked if block meta file is less than 7 bytes. Contributed by Stephen O'Donnell."
...
This reverts commit 7bebad61d9
2019-09-02 09:43:44 -07:00
040f6e93bb
HDFS-14654. RBF: TestRouterRpc#testNamenodeMetrics is flaky. Contributed by Chen Zhang.
2019-09-02 13:01:52 +05:30
c3abfcefdd
HDFS-13843. RBF: Add optional parameter -d for detailed listing of mount points. Contributed by Ayush Saxena.
2019-09-02 07:24:04 +05:30
18d74fe41c
HDFS-14711. RBF: RBFMetrics throws NullPointerException if stateStore disabled. Contributed by Chen Zhang.
2019-09-01 12:53:10 +05:30
96f7dc1992
HDFS-14796. Define LOG instead of BlockManager.LOG in ErasureCodingWork/ReplicationWork. Contributed by Fei Hui.
2019-08-30 14:31:05 +05:30
e220dac15c
HDFS-12212. Options.Rename.To_TRASH is considered even when Options.Rename.NONE is specified. Contributed by Vinayakumar B.
2019-08-30 08:40:13 +05:30
915271245b
Make DataNodePeerMetrics#minOutlierDetectionSamples configurable ( #1314 ). Contributed by Lisheng Sun.
...
Signed-off-by: sunlisheng <sunlisheng@xiaomi.com>
2019-08-29 18:35:26 -07:00
7bebad61d9
HDFS-14706. Checksums are not checked if block meta file is less than 7 bytes. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-29 17:38:07 -07:00
fcb7884bfc
HDFS-8178. QJM doesn't move aside stale inprogress edits files. Contributed by Istvan Fajth.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-29 16:56:14 -07:00
3b22fcd377
HDFS-14256. Review Logging of NameNode Class. Contributed by David Mollitor.
2019-08-29 14:21:09 -07:00
ffca734c62
HDFS-14104. Review getImageTxIdToRetain. Contributed by David Mollitor.
2019-08-29 14:18:35 -07:00
f600fbb6c4
HDFS-11246. FSNameSystem#logAuditEvent should be called outside the read or write locks. Contributed by He Xiaoqiao, Kuhu Shukla.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Co-authored-by: Kuhu Shukla <kshukla@apache.org>
2019-08-29 10:10:52 -07:00
8e779a151e
HDFS-14721. RBF: ProxyOpComplete is not accurate in FederationRPCPerformanceMonitor. Contributed by xuzq.
2019-08-29 20:08:38 +05:30
29bd6f3fc3
HDFS-8631. WebHDFS : Support setQuota. Contributed by Chao Sun.
2019-08-28 23:58:23 +05:30
48cb583906
HDFS-14710. RBF: Improve some RPC performance by using previous block. Contributed by xuzq.
2019-08-28 10:48:00 -07:00
6e37d65b03
HDFS-14760. Log INFO mode if snapshot usage and actual usage differ. Contributed by CR Hota.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-27 16:20:25 -07:00
dde9399b37
HDFS-14497. Addendum: Write lock held by metasave impact following RPC processing.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-27 15:26:59 -07:00
ce8eb1283a
Revert "HDDS-1610. applyTransaction failure should not be lost on restart. Contributed by Shashikant Banerjee."
...
This reverts commit 62445021d5
2019-08-27 23:23:44 +05:30
8ab7020e64
HDFS-14779. Fix logging error in TestEditLog#testMultiStreamsLoadEditWithConfMaxTxns
2019-08-27 10:30:31 -07:00
b69ac575a1
HDFS-14772. RBF: hdfs-rbf-site.xml can't be loaded automatically. Contributed by Yuxuan Wang
2019-08-27 14:32:32 +09:00
07e3cf952e
HDFS-2470. NN should automatically set permissions on dfs.namenode.*.dir. Contributed by Siddharth Wagle.
2019-08-26 15:57:55 -07:00
d2225c8ca8
HDFS-14722. RBF: GetMountPointStatus should return mountTable information when getFileInfoAll throw IOException. Contributed by xuzq.
2019-08-24 08:47:43 +05:30
0b796754b9
HDFS-14766. RBF: MountTableStoreImpl#getMountTableEntries returns extra entry. Contributed by Chen Zhang.
2019-08-23 14:53:57 -07:00
ebef99dcf4
HDFS-14674. [SBN read] Got an unexpected txid when tail editlog. Contributed by wangzhaohui.
2019-08-23 12:18:43 -07:00
d699022fce
HDFS-13977. Override shouldForceSync in QuorumOutputStream to allow for proper auto-sync behavior. Contributed by Erik Krogen.
2019-08-23 11:33:21 -07:00
894e2300d6
HDFS-14761. RBF: MountTableResolver cannot invalidate cache correctly ( #1334 )
...
HDFS-14761. RBF: MountTableResolver cannot invalidate cache correctly
2019-08-23 10:46:39 -07:00
c92de8209d
HDFS-14747. RBF: IsFileClosed should be return false when the file is open in multiple destination. Contributed by xuzq.
2019-08-23 19:08:07 +05:30
bd7baea5a5
HDFS-14396. Failed to load image from FSImageFile when downgrade from 3.x to 2.x. Contributed by Fei Hui.
2019-08-23 15:28:27 +09:00
abc8fde4ca
HDFS-13596. NN restart fails after RollingUpgrade from 2.x to 3.x. Contributed by Fei Hui.
2019-08-23 14:44:34 +09:00
b67812ea21
HDFS-14617. Improve fsimage load time by writing sub-sections to the fsimage index ( #1028 ). Contributed by Stephen O'Donnell.
...
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
2019-08-22 17:09:57 -07:00
93daf69f90
HDFS-14675. Increase Balancer Defaults Further. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Gabor Bota <gabota@apache.org>
2019-08-22 15:21:33 -07:00
e04dcfdc57
HDFS-14583. FileStatus#toString() will throw IllegalArgumentException. Contributed by xuzq.
2019-08-22 10:22:38 -07:00
76790a1e67
HDFS-14358. Provide LiveNode and DeadNode filter in DataNode UI. Contributed by hemanthboyina.
2019-08-22 12:27:45 +05:30
52c77bc160
HDFS-14741. RBF: RecoverLease should be return false when the file is open in multiple destination. Contributed by xuzq
2019-08-22 08:57:22 +05:30
f9029c4070
HDFS-14744. RBF: Non secured routers should not log in error mode when UGI is default. Contributed by CR Hota.
2019-08-22 02:40:39 +05:30
0f598aed13
HDFS-14276. [SBN read] Reduce tailing overhead. Contributed by Wei-Chiu Chuang.
2019-08-22 02:25:06 +05:30
93595febaa
HDFS-14756. RBF: getQuotaUsage may ignore some folders. Contributed by Chen Zhang.
2019-08-21 09:39:57 -07:00
5eeb6da2d4
HDFS-14714. RBF: implement getReplicatedBlockStats interface. Contributed by Chen Zhang.
2019-08-21 09:38:17 -07:00
57f7370174
Revert "HDFS-14476. lock too long when fix inconsistent blocks between disk and in-memory. Contributed by Sean Chow."
...
This reverts commit b58a35f374
2019-08-21 08:59:49 -07:00
8aaf5e1a14
HDFS-14759. HDFS cat logs an info message.
...
Contributed by Eric Badger.
2019-08-20 20:24:19 -07:00
3a145e2918
HDFS-14582. Failed to start DN with ArithmeticException when NULL checksum used. Contributed by Surendra Singh Lilhore.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-20 15:55:26 -07:00
269b543367
HADOOP-16523. Minor spell mistake in comment : Line number 101 ( #388 )
2019-08-20 13:36:23 -07:00
4cb22cd867
HDFS-14311. Multi-threading conflict at layoutVersion when loading block pool storage. Contributed by Yicong Cai.
2019-08-20 10:37:11 -07:00
aa6995fde2
HDFS-13201. Fix prompt message in testPolicyAndStateCantBeNull. Contributed by chencan.
2019-08-20 10:32:55 -07:00
bd92462321
HDFS-14729. Upgrade Bootstrap and jQuery versions used in HDFS UIs. Contributed by Vivek Ratnavel Subramanian. This closes #1297
2019-08-20 18:42:19 +05:30
62445021d5
HDDS-1610. applyTransaction failure should not be lost on restart. Contributed by Shashikant Banerjee.
2019-08-20 14:37:01 +05:30
360a96f342
HDFS-13709. Report bad block to NN when transfer block encounter EIO exception. Contributed by Chen Zhang.
2019-08-19 13:08:55 -07:00
abae6ff2a2
HDFS-14746. Trivial test code update after HDFS-14687. Contributed by kevin su.
...
Reviewed-by: Surendra Singh Lilhore <surendralilhore@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-19 11:14:51 -07:00
b8db5b9a98
HDFS-14687. Standby Namenode never come out of safemode when EC files are being written. Contributed by Surendra Singh Lilhore.
...
Reviewed-by: Siyao Meng <smeng@cloudera.com>
Reviewed-by: Wei-Chiu CHuang <weichiu@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-18 18:13:35 -07:00
b58a35f374
HDFS-14476. lock too long when fix inconsistent blocks between disk and in-memory. Contributed by Sean Chow.
2019-08-18 17:46:00 -07:00
d873ddd656
HDFS-14127. Add a description about the observer read configuration. Contributed by xiangheng.
2019-08-17 04:39:22 -07:00
528378784f
HDFS-12012. Fix spelling mistakes in BPServiceActor.java. Contributed by chencan.
2019-08-17 04:37:05 -07:00
a38b9e137e
HDFS-14456:HAState#prepareToEnterState neednt a lock ( #770 ) Contributed by hunshenshi.
2019-08-16 14:53:06 -07:00
9a1d8cfaf5
HDFS-14678. Allow triggerBlockReport to a specific namenode. ( #1252 ). Contributed by Leon Gao.
2019-08-16 08:00:51 -07:00
34681643e9
HDFS-14719. Correct the safemode threshold value in BlockManagerSafeMode. Contributed by hemanthboyina.
2019-08-15 12:12:18 +05:30
0a85af959c
HDFS-13101. Yet another fsimage corruption related to snapshot. Contributed by Shashikant Banerjee.
2019-08-15 10:16:25 +05:30
b06c2345ef
HDFS-14713. RBF: RouterAdmin supports refreshRouterArgs command but not on display. Contributed by wangzhaohui.
2019-08-14 20:44:31 +05:30
3c0382f1b9
HDFS-14595. HDFS-11848 breaks API compatibility. Contributed by Siyao Meng.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
2019-08-14 07:30:35 -07:00
c13ec7ab66
HDFS-14491. More Clarity on Namenode UI Around Blocks and Replicas. Contributed by Siyao Meng.
2019-08-13 17:15:26 -07:00
633b7c1cfe
HDFS-14625. Make DefaultAuditLogger class in FSnamesystem to Abstract. Contributed by hemanthboyina.
2019-08-13 16:50:49 -07:00
da0006fe04
HDFS-14423. Percent (%) and plus (+) characters no longer work in WebHDFS.
...
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
2019-08-14 08:39:40 +09:00
6ae8bc3a4a
HDFS-14665. HttpFS: LISTSTATUS response is missing HDFS-specific fields ( #1267 ) Contributed by Siyao Meng.
2019-08-13 16:27:57 -07:00
e9b6b81de4
HDFS-13505. Turn on HDFS ACLs by default. Contributed by Siyao Meng.
2019-08-13 19:17:10 +05:30
454420e4f2
HDFS-14708. TestLargeBlockReport#testBlockReportSucceedsWithLargerLengthLimit fails in trunk. Contributed by Lisheng Sun.
2019-08-13 08:27:20 +05:30
c92b49876a
HDFS-14148. HDFS OIV ReverseXML SnapshotSection parser throws exception when there are more than one snapshottable directory ( #1274 ) Contributed by Siyao Meng.
2019-08-12 17:26:20 -07:00
8a77a224c7
HDFS-13359. DataXceiver hung due to the lock in FsDatasetImpl#getBlockInputStream. Contributed by Yiqun Lin.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-09 18:40:28 -07:00
e02ffed1b1
HDFS-12125. Document the missing EC removePolicy command ( #1258 ) Contributed by Wenxin He, Siyao Meng.
...
Co-authored-by: Wenxin He <wenxin.he@gmail.com>
2019-08-09 18:00:22 -07:00
ce3c5a3e3b
HDFS-14623. In NameNode Web UI, for Head the file (first 32K) old data is showing. Contributed by hemanthboyina.
2019-08-09 15:41:37 -07:00
865021b8c9
HDFS-14195. OIV: print out storage policy id in oiv Delimited output. Contributed by Wang, Xinglong.
2019-08-09 15:37:29 -07:00
f6fa865d6f
HDFS-14700. Clean up pmem cache before setting pmem cache capacity. Contributed by Feilong He.
2019-08-09 14:07:54 +05:30
6ad9a11494
HDFS-14693. NameNode should log a warning when EditLog IPC logger's pending size exceeds limit. Contributed by Xudong Cao.
2019-08-08 13:50:30 -07:00
2265872c2d
HDFS-14705. Remove unused configuration dfs.min.replication. Contributed by CR Hota.
2019-08-08 13:48:29 -07:00
28a848412c
HDFS-14701. Change Log Level to warn in SlotReleaser. Contributed by Lisheng Sun.
2019-08-08 13:46:31 -07:00
23f91f68b8
HDFS-14662. Document the usage of the new Balancer "asService" parameter. Contributed by Chen Zhang.
2019-08-08 13:45:29 -07:00
b0799148cf
HDFS-14459. ClosedChannelException silently ignored in FsVolumeList.addBlockPool(). Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-08 13:36:39 -07:00
11f750e6a7
HDFS-14616. Add the warn log when the volume available space isn't enough. Contributed by liying.
2019-08-08 00:49:58 +05:30
de64253892
HDFS-14608. DataNode#DataTransfer should be named. Contributed by Inigo Goiri.
2019-08-08 00:37:56 +05:30
827dbb11e2
HDFS-14370. Add exponential backoff to the edit log tailer to avoid spinning on empty edit tail requests. Contributed by Erik Krogen.
2019-08-07 09:25:58 -07:00
a5bb1e8ee8
HDFS-14313. Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du. Contributed by Lisheng Sun.
2019-08-07 10:18:11 +08:00
954ff36360
HDFS-14692. Upload button should not encode complete url. Contributed by Lokesh Jain.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-06 15:30:50 -07:00
35e0a01d7b
HDFS-14557. JournalNode error: Can't scan a pre-transactional edit log. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-06 07:57:46 -07:00
11272159bb
HDFS-14679. Failed to add erasure code policies with example template. Contributed by Yuan Zhou.
2019-08-06 08:42:15 +05:30
8f40856f76
HDFS-14669. TestDirectoryScanner#testDirectoryScannerInFederatedCluster fails intermittently in trunk. Contributed by qiang Liu.
2019-08-03 12:15:41 +05:30
e503db5f44
HDFS-12826. Document Saying the RPC port, But it's required IPC port in HDFS Federation Document. Contributed by usharani.
2019-08-03 12:05:46 +05:30
e7a0b8aa83
HDFS-14462 Ensure WebHDFS client throws the correct exception during writes. Contributed by Simbarashe Dzinamarira.
2019-08-02 11:48:31 -07:00
b964b81f85
HDFS-14685. DefaultAuditLogger doesn't print CallerContext. Contributed by xuzq.
2019-08-02 10:54:32 -07:00
c2d00c8450
HDFS-13131. Modifying testcase testEnableAndDisableErasureCodingPolicy. Contributed by chencan.
2019-08-01 19:11:43 -07:00
17e8cf501b
HDFS-14686. HttpFS: HttpFSFileSystem#getErasureCodingPolicy always returns null ( #1192 ) Contributed by Siyao Meng.
2019-08-01 17:15:22 -07:00
99bf1dc9eb
HDFS-14683. WebHDFS: Add erasureCodingPolicy field to GETCONTENTSUMMARY response ( #1189 ) Contributed by Siyao Meng.
2019-08-01 17:14:07 -07:00
32607dbd98
HDFS-14631.The DirectoryScanner doesn't fix the wrongly placed replica. Contributed by Jinglun.
2019-08-01 15:48:36 -07:00
89b102f916
HDFS-14661. RBF: updateMountTableEntry shouldn't update mountTableEntry if targetPath not exist. Contributed by xuzq.
2019-08-01 08:43:39 +05:30
b008072044
HDFS-14681. RBF: TestDisableRouterQuota failed because port 8888 was occupied.
2019-07-31 10:24:48 -07:00
99f88c30cb
HDFS-14419. Avoid repeated calls to the listOpenFiles function. Contributed by HuangTao.
2019-07-30 16:50:06 -07:00
3ae775d740
HDFS-14034. Support getQuotaUsage API in WebHDFS. Contributed by Chao Sun.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-07-30 16:01:17 -07:00
1f26cc8705
HDFS-13783. Add an option to the Balancer to make it run as a long-running service. Contributed by Chen Zhang.
2019-07-30 15:42:55 -07:00
7849bdcf70
HDFS-14449. Expose total number of DT in JMX for Namenode. Contributed by Fengnan Li.
2019-07-30 13:45:27 -07:00
8deced0073
HDFS-14677. TestDataNodeHotSwapVolumes#testAddVolumesConcurrently fails intermittently in trunk. Contributed by Chen Zhang.
2019-07-30 22:17:39 +05:30
8053085388
HDFS-14429. Block remain in COMMITTED but not COMPLETE caused by Decommission. Contributed by Yicong Cai.
2019-07-29 14:31:34 -07:00
611718f59f
HDFS-14670: RBF: Create secret manager instance using FederationUtil#newInstance.
2019-07-29 11:00:22 -07:00
02bd02b5af
HDFS-14660. [SBN Read] ObserverNameNode should throw StandbyException for requests not from ObserverProxyProvider. Contributed by Chao Sun.
2019-07-28 08:41:42 +05:30
2fe450cb5e
HDFS-14425. Native build fails on macos due to jlong in hdfs.c ( #741 )
2019-07-26 15:33:12 -07:00
ecc8acfd24
HDFS-14673. The console log is noisy when using DNSDomainNameResolver to resolve NameNode.
2019-07-26 11:08:46 -07:00
ce99cc31e9
HDFS-14303. Addendum: check block directory logic not correct when there is only meta file, print no meaning warn log. Contributed by qiang Liu.
2019-07-26 08:31:00 +05:30
6b8107ad97
HDFS-14135. TestWebHdfsTimeouts Fails intermittently in trunk. Contributed by Ayush Saxena.
...
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
2019-07-26 09:35:23 +09:00
62deab17a3
HDFS-14647. NPE during secure namenode startup. Contributed by Fengnan Li.
2019-07-25 06:51:07 +05:30
377f95bbe8
HDFS-13693. Remove unnecessary search in INodeDirectory.addChild during image loading. Contributed by Lisheng Sun.
2019-07-23 08:37:55 +05:30
6282c02d7d
HDFS-14577. RBF: FederationUtil#newInstance should allow constructor without context. Contributed by CR Hota.
2019-07-19 17:44:13 +05:30
9792f58662
HDFS-14653. RBF: Correct the default value for dfs.federation.router.namenode.heartbeat.enable. Contributed by Ayush Saxena.
2019-07-18 10:16:20 +09:00
5e6cc6fe8a
HDFS-12979. [SBN read] StandbyNode should upload FsImage to ObserverNode after checkpointing. Contributed by Chen Liang.
2019-07-17 14:37:16 -07:00
af0665c7f5
HDFS-13647. Fix the description of storageType option for space quota. Contributed by Takanobu Asanuma.
2019-07-17 23:22:32 +05:30
f77d54c243
HDFS-14642. processMisReplicatedBlocks does not return correct processed count. Contributed by Stephen O'Donnell.
2019-07-16 08:14:27 +05:30
64d4abf489
HDFS-14593. RBF: Implement deletion feature for expired records in State Store. Contributed by Takanobu Asanuma.
2019-07-15 22:38:00 +05:30
30a8f840f1
HDFS-14357. Update documentation for HDFS cache on SCM support. Contributed by Feilong He.
2019-07-15 13:18:23 +05:30
e98adb00b7
HDFS-14458. Report pmem stats to namenode. Contributed by Feilong He.
2019-07-15 13:02:37 +05:30
f9fab9f22a
HDFS-14499. Misleading REM_QUOTA value with snapshot and trash feature enabled for a directory. Contributed by Shashikant Banerjee.
2019-07-12 15:42:06 +05:30
00dd843a1a
HDFS-14466. Add a regression test for HDFS-14323.
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-07-12 10:04:55 +09:00
8fb5ca3f40
HDFS-14611. Move handshake secret field from Token to BlockAccessToken. Contributed by Chen Liang.
2019-07-11 13:23:25 -07:00
5747f6cff5
HDFS-14624. When decommissioning a node, log remaining blocks to replicate periodically. Contributed by Stephen O'Donnell.
2019-07-11 08:55:44 -07:00
eccc9a40de
HDFS-12703. Exceptions are fatal to decommissioning monitor. Contributed by He Xiaoqiao.
2019-07-10 11:11:52 -07:00
cdb20adfcc
HDDS-1611. Evaluate ACL on volume bucket key and prefix to authorize access. Contributed by Ajay Kumar. ( #973 )
2019-07-10 11:03:58 -07:00
993dc8726b
HDFS-14632. Reduce useless #getNumLiveDataNodes call in SafeModeMonitor. Contributed by He Xiaoqiao.
2019-07-10 10:53:34 +08:00
4632708148
HDFS-14547. Improve memory efficiency of quotas when storage type quotas are not set. Contributed by Jinglun.
2019-07-08 14:46:25 -07:00
ec851e4db2
HDFS-12862. CacheDirective becomes invalid when NN restart or failover. Contributed by Wang XL.
2019-07-08 15:23:46 +08:00
9c90729486
HDFS-13694. Making md5 computing being in parallel with image loading.
2019-07-05 10:17:37 -07:00
c3ca348b81
HDFS-14620. RBF: Fix 'not a super user' error when disabling a namespace in kerberos with superuser principal. Contributed by luhuachao.
2019-07-04 11:21:55 -07:00
1c254a8ec7
HDFS-14629. Property value Hard Coded in DNConf.java. Contributed by hemanthboyina.
2019-07-04 23:20:42 +09:00
729cb3aefe
HDFS-12748. NameNode memory leak when accessing webhdfs GETHOMEDIRECTORY. Contributed by Weiwei Yang.
2019-07-04 09:37:24 +08:00
d8bac50e12
HDFS-14610. HashMap is not thread safe. Field storageMap is typically synchronized by storageMap. However, in one place, field storageMap is not protected with synchronized. ( #1015 )
2019-07-01 13:54:49 -07:00
d203045c30
HDFS-14618. Incorrect synchronization of ArrayList field (ArrayList is thread-unsafe). Contributed by Paul Ward.
2019-06-28 16:06:43 -07:00
4a212242d9
HDFS-14599. HDFS-12487 breaks test TestDiskBalancer.testDiskBalancerWithFedClusterWithOneNameServiceEmpty. Contributed by He Xiaoqiao.
2019-06-27 10:00:05 -07:00
041e0c0564
HDFS-14036. RBF: Add hdfs-rbf-default.xml to HdfsConfiguration by default. Contributed by Takanobu Asanuma.
2019-06-27 09:57:52 -07:00
041e7a7dee
HDFS-14598. Findbugs warning caused by HDFS-12487.
...
Contributed by He Xiaoqiao.
2019-06-24 19:07:22 -07:00
b76b843c8b
HDFS-13371. NPE for FsServerDefaults.getKeyProviderUri() for clientProtocol communication between 2.7 and 3.X. Contributed by Sherwood Zheng.
2019-06-24 17:52:33 -07:00
38a560c6f1
Merge pull request #977 from leosunli/trunk
...
HDFS-14541. When evictableMmapped or evictable size is zero, do not throw NoSuchElementException.
2019-06-24 17:44:54 -07:00
95c94dcca7
HADOOP-16350. Ability to tell HDFS client not to request KMS Information from NameNode. Ccontributed by Greg Senia, Ajay Kumar.
2019-06-24 11:59:07 -07:00
719d57bf46
HDFS-14545. RBF: Router should support GetUserMappingsProtocol. Contributed by Ayush Saxena.
2019-06-24 22:03:04 +05:30
8627f6532b
HDFS-14550. RBF: Failed to get statistics from NameNodes before 2.9.0. Contributed by He Xiaoqiao.
2019-06-24 22:03:04 +05:30
ba221dc50e
HDFS-14526. RBF: Update the document of RBF related metrics. Contributed by Takanobu Asanuma.
2019-06-24 22:03:04 +05:30
ade8d3b60e
HDFS-14508. RBF: Clean-up and refactor UI components. Contributed by Takanobu Asanuma.
2019-06-24 22:03:04 +05:30
6915d7e13c
HDFS-13480. RBF: Separate namenodeHeartbeat and routerHeartbeat to different config key. Contributed by Ayush Saxena.
2019-06-24 22:03:04 +05:30
ddbe08db33
HDFS-13955. RBF: Support secure Namenode in NamenodeHeartbeatService. Contributed by CR Hota.
2019-06-24 22:03:04 +05:30
b062dd462d
HDFS-14475. RBF: Expose router security enabled status on the UI. Contributed by CR Hota.
2019-06-24 22:03:04 +05:30
b6fff8c81e
HDFS-13787. RBF: Add Snapshot related ClientProtocol APIs. Contributed by Inigo Goiri.
2019-06-24 22:03:04 +05:30
d240eec136
HDFS-14516. RBF: Create hdfs-rbf-site.xml for RBF specific properties. Contributed by Takanobu Asanuma.
2019-06-24 22:03:04 +05:30
9b197c2893
HDFS-13909. RBF: Add Cache pools and directives related ClientProtocol APIs. Contributed by Ayush Saxena.
2019-06-24 22:03:04 +05:30
ffbb6b6557
HDFS-13255. RBF: Fail when try to remove mount point paths. Contributed by Akira Ajisaka.
2019-06-24 22:03:04 +05:30
8e4267650f
HDFS-14440. RBF: Optimize the file write process in case of multiple destinations. Contributed by Ayush Saxena.
2019-06-24 22:03:04 +05:30
2636a54ffd
HDFS-13995. RBF: Security documentation. Contributed by CR Hota.
2019-06-24 22:03:04 +05:30
393f15176d
HDFS-14447. RBF: Router should support RefreshUserMappingsProtocol. Contributed by Shen Yinjie.
2019-06-24 22:03:04 +05:30
3deb5d345f
HDFS-14490. RBF: Remove unnecessary quota checks. Contributed by Ayush Saxena.
2019-06-24 22:03:04 +05:30
62fa53a01d
HDFS-14210. RBF: ACL commands should work over all the destinations. Contributed by Ayush Saxena.
2019-06-24 22:03:04 +05:30
32841178ba
HDFS-14426. RBF: Add delegation token total count as one of the federation metrics. Contributed by Fengnan Li.
2019-06-24 22:03:04 +05:30
203664e6b2
HDFS-14454. RBF: getContentSummary() should allow non-existing folders. Contributed by Inigo Goiri.
2019-06-24 22:03:04 +05:30
b522b52bb1
HDFS-14457. RBF: Add order text SPACE in CLI command 'hdfs dfsrouteradmin'. Contributed by luhuachao.
2019-06-24 22:03:04 +05:30
021a43b1a4
HDFS-13972. RBF: Support for Delegation Token (WebHDFS). Contributed by CR Hota.
2019-06-24 22:03:04 +05:30
de7da9b69e
HDFS-14422. RBF: Router shouldn't allow READ operations in safe mode. Contributed by Inigo Goiri.
2019-06-24 22:03:04 +05:30
0f9b8d7a75
HDFS-14369. RBF: Fix trailing / for webhdfs. Contributed by Akira Ajisaka.
2019-06-24 22:03:04 +05:30
dd8c2b92df
HDFS-13853. RBF: RouterAdmin update cmd is overwriting the entry not updating the existing. Contributed by Ayush Saxena.
2019-06-24 22:03:04 +05:30
6c42d40504
HDFS-14316. RBF: Support unavailable subclusters for mount points with multiple destinations. Contributed by Inigo Goiri.
2019-06-24 22:03:04 +05:30
0dbd87874a
HDFS-14388. RBF: Prevent loading metric system when disabled. Contributed by Inigo Goiri.
2019-06-24 22:03:03 +05:30
9a9fbbe145
HDFS-14351. RBF: Optimize configuration item resolving for monitor namenode. Contributed by He Xiaoqiao and Inigo Goiri.
2019-06-24 22:03:03 +05:30
f539e2a4ee
HDFS-14343. RBF: Fix renaming folders spread across multiple subclusters. Contributed by Ayush Saxena.
2019-06-24 22:03:03 +05:30
2a2d5eb441
HDFS-14334. RBF: Use human readable format for long numbers in the Router UI. Contributed by Inigo Goiri.
2019-06-24 22:03:03 +05:30
fcabc8f0e4
HDFS-14335. RBF: Fix heartbeat typos in the Router. Contributed by CR Hota.
2019-06-24 22:03:03 +05:30
6cdf8db55c
HDFS-14331. RBF: IOE While Removing Mount Entry. Contributed by Ayush Saxena.
2019-06-24 22:03:03 +05:30
1ce25e702b
HDFS-14329. RBF: Add maintenance nodes to federation metrics. Contributed by Ayush Saxena.
2019-06-24 22:03:03 +05:30
7bbe01a196
HDFS-14259. RBF: Fix safemode message for Router. Contributed by Ranith Sadar.
2019-06-24 22:03:03 +05:30
1c7ab59be3
HDFS-14322. RBF: Security manager should not load if security is disabled. Contributed by CR Hota.
2019-06-24 22:03:03 +05:30
c4b1fa91fa
HDFS-14052. RBF: Use Router keytab for WebHDFS. Contributed by CR Hota.
2019-06-24 22:03:03 +05:30
de719b08b5
HDFS-14307. RBF: Update tests to use internal Whitebox instead of Mockito. Contributed by CR Hota.
2019-06-24 22:03:03 +05:30
8b8ff5ccbc
HDFS-14249. RBF: Tooling to identify the subcluster location of a file. Contributed by Inigo Goiri.
2019-06-24 22:03:03 +05:30
50aee18a84
HDFS-14268. RBF: Fix the location of the DNs in getDatanodeReport(). Contributed by Inigo Goiri.
2019-06-24 22:03:03 +05:30
e2a3c4494b
HDFS-14226. RBF: Setting attributes should set on all subclusters' directories. Contributed by Ayush Saxena.
2019-06-24 22:03:03 +05:30
75f8b6ccfa
HDFS-13358. RBF: Support for Delegation Token (RPC). Contributed by CR Hota.
2019-06-24 22:03:03 +05:30
7e63e37dc5
HDFS-14230. RBF: Throw RetriableException instead of IOException when no namenodes available. Contributed by Fei Hui.
2019-06-24 22:03:03 +05:30
912b90f91e
HDFS-14252. RBF : Exceptions are exposing the actual sub cluster path. Contributed by Ayush Saxena.
2019-06-24 22:03:03 +05:30
9c4e55685b
HDFS-14225. RBF : MiniRouterDFSCluster should configure the failover proxy provider for namespace. Contributed by Ranith Sardar.
2019-06-24 22:03:03 +05:30
9eed3a49df
HDFS-14215. RBF: Remove dependency on availability of default namespace. Contributed by Ayush Saxena.
2019-06-24 22:03:03 +05:30
acdf911c01
HDFS-14224. RBF: NPE in getContentSummary() for getEcPolicy() in case of multiple destinations. Contributed by Ayush Saxena.
2019-06-24 22:03:03 +05:30
8b9b58b58a
HDFS-14223. RBF: Add configuration documents for using multiple sub-clusters. Contributed by Takanobu Asanuma.
2019-06-24 22:03:03 +05:30
020f83f51f
HDFS-14209. RBF: setQuota() through router is working for only the mount Points under the Source column in MountTable. Contributed by Shubham Dewan.
2019-06-24 22:03:03 +05:30
235406d904
HDFS-14156. RBF: rollEdit() command fails with Router. Contributed by Shubham Dewan.
2019-06-24 22:03:03 +05:30
c012b09fb6
HDFS-14193. RBF: Inconsistency with the Default Namespace. Contributed by Ayush Saxena.
2019-06-24 22:03:03 +05:30
7b61cbf672
HDFS-14129. addendum to HDFS-14129. Contributed by Ranith Sardar.
2019-06-24 22:03:03 +05:30
f40e10b349
HDFS-14129. RBF: Create new policy provider for router. Contributed by Ranith Sardar.
2019-06-24 22:03:03 +05:30
221f24cbdc
HDFS-14206. RBF: Cleanup quota modules. Contributed by Inigo Goiri.
2019-06-24 22:03:02 +05:30
f4e2bfce58
HDFS-13856. RBF: RouterAdmin should support dfsrouteradmin -refreshRouterArgs command. Contributed by yanghuafeng.
2019-06-24 22:03:02 +05:30
b8bcbd0ed2
HDFS-14191. RBF: Remove hard coded router status from FederationMetrics. Contributed by Ranith Sardar.
2019-06-24 22:03:02 +05:30
4244653e43
HDFS-14150. RBF: Quotas of the sub-cluster should be removed when removing the mount point. Contributed by Takanobu Asanuma.
2019-06-24 22:03:02 +05:30
f3cbf0eb9a
HDFS-14161. RBF: Throw StandbyException instead of IOException so that client can retry when can not get connection. Contributed by Fei Hui.
2019-06-24 22:03:02 +05:30
1dc01e59af
HDFS-14167. RBF: Add stale nodes to federation metrics. Contributed by Inigo Goiri.
2019-06-24 22:03:02 +05:30
8f6f9d9c83
HDFS-13443. RBF: Update mount table cache immediately after changing (add/update/remove) mount table entries. Contributed by Mohammad Arshad.
2019-06-24 22:03:02 +05:30
3d97142dff
HDFS-14151. RBF: Make the read-only column of Mount Table clearly understandable.
2019-06-24 22:03:02 +05:30
bbe859177d
HDFS-13869. RBF: Handle NPE for NamenodeBeanMetrics#getFederationMetrics. Contributed by Ranith Sardar.
2019-06-24 22:03:02 +05:30
01b4126b4e
HDFS-14152. RBF: Fix a typo in RouterAdmin usage. Contributed by Ayush Saxena.
2019-06-24 22:03:02 +05:30
94a8dec168
HDFS-14114. RBF: MIN_ACTIVE_RATIO should be configurable. Contributed by Fei Hui.
2019-06-24 22:03:02 +05:30
0ca7142c11
Revert "HDFS-14114. RBF: MIN_ACTIVE_RATIO should be configurable. Contributed by Fei Hui."
...
This reverts commit 7c0d6f65fde12ead91ed7c706521ad1d3dc995f8.
2019-06-24 22:03:02 +05:30
6aa7aabff8
HDFS-14114. RBF: MIN_ACTIVE_RATIO should be configurable. Contributed by Fei Hui.
2019-06-24 22:03:02 +05:30
b320caecb3
HDFS-14085. RBF: LS command for root shows wrong owner and permission information. Contributed by Ayush Saxena.
2019-06-24 22:03:02 +05:30
19088e1b49
HDFS-14089. RBF: Failed to specify server's Kerberos pricipal name in NamenodeHeartbeatService. Contributed by Ranith Sardar.
2019-06-24 22:03:02 +05:30
f2355c7063
HDFS-13776. RBF: Add Storage policies related ClientProtocol APIs. Contributed by Dibyendu Karmakar.
2019-06-24 22:03:02 +05:30
f4bd1114ff
HDFS-14082. RBF: Add option to fail operations when a subcluster is unavailable. Contributed by Inigo Goiri.
2019-06-24 22:03:02 +05:30
fa55eacd35
HDFS-13834. RBF: Connection creator thread should catch Throwable. Contributed by CR Hota.
2019-06-24 22:03:02 +05:30
04caaba488
HDFS-13852. RBF: The DN_REPORT_TIME_OUT and DN_REPORT_CACHE_EXPIRE should be configured in RBFConfigKeys. Contributed by yanghuafeng.
2019-06-24 22:03:02 +05:30
ebfd2d8a4e
HDFS-12284. addendum to HDFS-12284. Contributed by Inigo Goiri.
2019-06-24 22:03:02 +05:30
6f2c871b05
HDFS-12284. RBF: Support for Kerberos authentication. Contributed by Sherwood Zheng and Inigo Goiri.
2019-06-24 22:03:02 +05:30
7b0bc492ed
HDFS-14024. RBF: ProvidedCapacityTotal json exception in NamenodeHeartbeatService. Contributed by CR Hota.
2019-06-24 22:03:02 +05:30
c5065bf20b
HDFS-13845. RBF: The default MountTableResolver should fail resolving multi-destination paths. Contributed by yanghuafeng.
2019-06-24 22:03:02 +05:30
b3fee1d2bf
HDFS-14011. RBF: Add more information to HdfsFileStatus for a mount point. Contributed by Akira Ajisaka.
2019-06-24 22:03:02 +05:30
41c94a636b
HDFS-13906. RBF: Add multiple paths for dfsrouteradmin 'rm' and 'clrquota' commands. Contributed by Ayush Saxena.
2019-06-24 22:03:02 +05:30
05145404d5
HDFS-14339. Inconsistent log level practices in RpcProgramNfs3.java. Contributed by Anuhan Torgonshar.
2019-06-24 08:30:48 -07:00
daa1e14745
when evictableMmapped or evictable size is zero, do not throw NoSuchElementException
...
Signed-off-by: sunlisheng <sunlisheng@xiaomi.com>
2019-06-24 20:17:22 +08:00
645d67bc4f
HDFS-14074. DataNode runs async disk checks maybe throws NullPointerException, and DataNode failed to register to NameSpace. Contributed by guangyi lu.
2019-06-21 18:27:03 -07:00
1524e2e6c5
HDFS-12487. FsDatasetSpi.isValidBlock() lacks null pointer check inside and neither do the callers. Contributed by liumi.
2019-06-21 18:17:36 -07:00
272b96d243
HDFS-13893. DiskBalancer: no validations for Disk balancer commands. Contributed by Lokesh Jain.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-20 20:23:05 -07:00
98d2065643
HDFS-12564. Add the documents of swebhdfs configurations on the client side. Contributed by Takanobu Asanuma.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-20 20:17:24 -07:00
5962a518bd
HDFS-14581. Appending to EC files crashes NameNode. Contributed by Surendra Singh Lilhore.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-20 09:40:29 -07:00
71ecd2e411
HDFS-14303. check block directory logic not correct when there is only meta file, print no meaning warn log. Contributed by qiang Liu.
2019-06-19 10:27:53 -07:00
f5ecc0bc08
HDFS-14103. Review Logging of BlockPlacementPolicyDefault. Contributed by David Mollitor.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-19 10:06:40 -07:00
d3ac516665
Revert "HDFS-13287. TestINodeFile#testGetBlockType results in NPE when run alone. Contributed by Virajith Jalaparti."
...
This reverts commit a1c3868c4f
2019-06-19 06:27:02 -07:00
48e564f7e2
HDFS-14537. Journaled Edits Cache is not cleared when formatting the JN. Contributed by Ranith Sardar.
2019-06-18 22:28:21 -07:00
a1c3868c4f
HDFS-13287. TestINodeFile#testGetBlockType results in NPE when run alone. Contributed by Virajith Jalaparti.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-18 22:17:49 -07:00
f9ee97de98
HDFS-14398. Update HAState.java to fix typos. Contributed by Nikhil Navadiya.
2019-06-18 21:58:37 -07:00
81ec909411
HDFS-14078. Admin helper fails to prettify NullPointerExceptions. Contributed by Elek, Marton.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-18 12:01:26 -07:00
3c1a1ceea9
HDFS-14487. Missing Space in Client Error Message (Contributed by Shweta Yakkali via Daniel Templeton)
...
Change-Id: I0f8ce74a35ab24fe94fd0e57d8247bb3fa575e6f
2019-06-18 10:21:22 -07:00
3ab77d9bc9
HDFS-14201. Ability to disallow safemode NN to become active. Contributed by Xiao Liang and He Xiaoqiao.
2019-06-18 09:58:29 -07:00
335c1c9938
HDFS-14010. Pass correct DF usage to ReservedSpaceCalculator builder. Contributed by Virajith Jalaparti.
2019-06-18 00:35:48 -07:00
54cdde38c7
HDFS-12314. Typo in the TestDataNodeHotSwapVolumes.testAddOneNewVolume(). Contributed by Oleg Danilov.
2019-06-17 23:11:25 -07:00
d41310a15d
HDFS-12315. Use Path instead of String to check closedFiles set. Contributed by Oleg Danilov.
2019-06-17 23:06:02 -07:00
098c325a78
HDFS-14340. Lower the log level when can't get postOpAttr. Contributed by Anuhan Torgonshar.
2019-06-17 22:45:52 -07:00
e3172221ea
HDFS-12770. Add doc about how to disable client socket cache. Contributed by Weiwei Yang.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-17 21:48:16 -07:00
62ad9885ea
HDFS-13730. BlockReaderRemote.sendReadResult throws NPE. Contributed by Yuanbo Liu.
2019-06-17 20:18:53 -07:00
a95e87d827
HDFS-11950. Disable libhdfs zerocopy test on Mac. Contributed by Akira Ajisaka.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-17 17:03:25 -07:00
6822193ee6
HDFS-12914. Block report leases cause missing blocks until next report. Contributed by Santosh Marella, He Xiaoqiao.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Co-authored-by: He Xiaoqiao <hexiaoqiao@apache.org>
2019-06-17 16:20:30 -07:00
7314185c4a
Revert "HDFS-12914. Block report leases cause missing blocks until next report. Contributed by Santosh Marella, He Xiaoqiao."
...
This reverts commit ae4143a529
2019-06-17 16:18:48 -07:00
a50c35bb81
Revert "HDFS-12914. Addendum patch. Block report leases cause missing blocks until next report. Contributed by Santosh Marella, He Xiaoqiao."
...
This reverts commit cdc5de6448
2019-06-17 16:18:37 -07:00
f9a7b442fd
HDFS-14465. When the Block expected replications is larger than the number of DataNodes, entering maintenance will never exit. Contributed by Yicong Cai.
2019-06-17 15:18:17 -07:00
cc1630288e
HDFS-14556: Spelling Mistake "gloablly" ( #938 )
2019-06-16 18:32:42 -07:00
8370a0ae16
HDFS-14203. Refactor OIV Delimited output entry building mechanism. Contributed by Adam Antal.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-15 06:47:10 -07:00
b24efa11ea
HDFS-14549. EditLogTailer shouldn't output full stack trace when interrupted. Contributed by Inigo Goiri.
2019-06-14 13:37:23 -07:00
cdc5de6448
HDFS-12914. Addendum patch. Block report leases cause missing blocks until next report. Contributed by Santosh Marella, He Xiaoqiao.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Co-authored-by: He Xiaoqiao <hexiaoqiao@apache.org>
2019-06-14 13:01:39 -07:00
ae4143a529
HDFS-12914. Block report leases cause missing blocks until next report. Contributed by Santosh Marella, He Xiaoqiao.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Co-authored-by: He Xiaoqiao <hexiaoqiao@apache.org>
2019-06-14 10:42:54 -07:00
9ebbda342f
HADOOP-16372. Fix typo in DFSUtil getHttpPolicy method
...
Closes #967
2019-06-14 16:09:37 +02:00
4f455290b1
HDFS-14560. Allow block replication parameters to be refreshable. Contributed by Stephen O'Donnell.
2019-06-13 18:26:53 -07:00
50de0874d0
HDFS-13231. Extend visualization for Decommissioning, Maintenance Mode under Datanode tab in the NameNode UI. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-12 10:30:23 -07:00
4ea6c2f457
HADOOP-16354. Enable AuthFilter as default for WebHDFS.
...
Contributed by Prabhu Joseph
2019-06-11 18:41:08 -04:00
5740eea081
HDFS-14513. FSImage which is saving should be clean while NameNode shutdown. Contributed by He Xiaoqiao.
2019-06-11 11:48:35 -07:00
101d5b5f86
HDFS-14234. Limit WebHDFS to specifc user, host, directory triples.
...
Contributed by Clay B.
2019-06-10 17:55:16 -07:00
bd46bdf9f9
HDFS-14553. Make queue size of BlockReportProcessingThread configurable. Contributed by He Xiaoqiao.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-10 17:21:52 -07:00
0d160a0ba8
HDFS-10210. Remove the defunct startKdc profile from hdfs. Contributed by Wei-Chiu Chuang, Akira Ajisaka.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Co-authored-by: Akira Ajisaka <aajisaka@apache.org>
2019-06-10 14:13:53 -07:00
7217494f40
HDFS-10659. Namenode crashes after Journalnode re-installation in an HA cluster due to missing paxos directory. Contributed by star, Hanisha Koneru.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-10 13:45:37 -07:00
d6d95d2686
HDFS-14494. Move Server logging of StatedId inside receiveRequestState(). Contributed by Shweta Yakkali.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-10 10:23:18 -07:00
e1dfc060f8
HDFS-14486. The exception classes in some throw statements do not accurately describe why they are thrown. Contributed by Ayush Saxena.
2019-06-06 11:59:53 -07:00
944adc61b1
HDFS-14527. Stop all DataNodes may result in NN terminate. Contributed by He Xiaoqiao.
2019-06-06 10:20:28 -07:00
294695dd57
HADOOP-16314. Make sure all web end points are covered by the same authentication filter.
...
Contributed by Prabhu Joseph
2019-06-05 18:55:13 -04:00
d1aad44490
HDFS-14356. Implement HDFS cache on SCM with native PMDK libs. Contributed by Feilong He.
2019-06-05 21:33:00 +08:00
ea3b0a1844
HDFS-14535. The default 8KB buffer in requestFileDescriptors#BufferedOutputStream is causing lots of heap allocation in HBase when using short-circut read
2019-06-04 08:48:05 -07:00
35f1014b3e
HDFS-13654. Use a random secret when a secret file doesn't exist in HttpFS. This should be default.
2019-05-31 10:29:24 +09:00
9122b9b649
HDFS-14521. Suppress setReplication logging. Contributed by Kihwal Lee
2019-05-30 17:28:45 -05:00
33c62f8f4e
HDFS-14497. Write lock held by metasave impact following RPC processing. Contributed by He Xiaoqiao.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-05-30 13:30:35 -07:00
6f5a36c13c
HADOOP-13656. fs -expunge to take a filesystem. Contributed by Shweta.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-05-30 13:21:58 -07:00
c1caab40f2
HDFS-14512. ONE_SSD policy will be violated while write data with DistributedFileSystem.create(....favoredNodes). Contributed by Ayush Saxena.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-05-29 20:56:38 -07:00
afd844059c
HADOOP-16331. Fix ASF License check in pom.xml
...
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2019-05-29 17:25:13 +09:00
d78854b928
HDFS-14434. Ignore user.name query parameter in secure WebHDFS.
...
Contributed by KWON BYUNGCHANG
2019-05-28 17:31:35 -04:00
9f933e6446
HADOOP-16323. https everywhere in Maven settings.
2019-05-27 15:24:59 +09:00
37900c5639
HDFS-14402. Use FileChannel.transferTo() method for transferring block to SCM cache. Contributed by Feilong He.
2019-05-26 14:30:11 +05:30
55e0c134f0
HDFS-14500. NameNode StartupProgress should not allow new steps in an already-completed phase. Contributed by Erik Krogen.
2019-05-24 13:12:59 -07:00
c31b7b8d52
HDFS-14502. keepResults option in NNThroughputBenchmark should call saveNamespace(). Contributed by Konstantin V Shvachko.
2019-05-24 12:34:29 -07:00
c9393dd17f
Revert "HDFS-14353. Erasure Coding: metrics xmitsInProgress become to negative. Contributed by maobaolong."
...
This reverts commit 1cb2eb0df3
2019-05-24 10:23:47 -07:00
f96a2df38d
HADOOP-16266. Add more fine-grained processing time metrics to the RPC layer. Contributed by Christopher Gregorian.
2019-05-23 10:28:37 -07:00
9dff6eff81
HDFS-14507. Document -blockingDecommission option for hdfs dfsadmin -listOpenFiles
...
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2019-05-22 13:22:26 +09:00
1cb2eb0df3
HDFS-14353. Erasure Coding: metrics xmitsInProgress become to negative. Contributed by maobaolong.
2019-05-20 17:22:58 -07:00
Takanobu Asanuma
Ayush Saxena
Chandra Sanivarapu
Konstantin V Shvachko
Inigo Goiri
Shashikant Banerjee
Masatake Iwasaki
Chen Liang
He Xiaoqiao
Erik Krogen
Surendra Singh Lilhore
Stephen O'Donnell
Akira Ajisaka
Wei-Chiu Chuang
Yiqun Lin
Hui Fei
Chao Sun
Wang Yuxuan
Zhao Junwang
Jonathan Hung
Eric Yang
Eric Badger
zhaoyim
Vinayakumar B
Mukul Kumar Singh
Kihwal Lee
Yuming Wang
Ashvin
Virajith Jalaparti
Sahil Takiar
Zhankun Tang
Steve Loughran
Prabhu Joseph
Rohith Sharma K S
Rakesh Radhakrishnan
Malcolm Taylor
Sahil Takiar
Xiaoyu Yao
leosunli
Stephen O'Donnell
Istvan Fajth
CR Hota
Jonathan Hung
Arpit Agarwal
Anu Engineer
Sandeep Kumar
Sunil G
HUAN-PING SU
hunshenshi
LeonGao91
Siyao Meng
Siyao Meng
Lokesh Jain
Ajay Yadav
Weiwei Yang
paulward24
Inigo Goiri
Ajay Kumar
Brahma Reddy Battula
Giovanni Matteo Fumarola
sunlisheng
David Mollitor
Bharat Viswanadham
Elek, Marton
Shweta Yakkali
Oleg Danilov
Santosh Marella
mpicker90
Adam Antal
dineshchitlangia
Hanisha Koneru
Sammi Chen
huzheng
Christopher Gregorian