Surendra Singh Lilhore
c870514740
HDFS-14992. TestOfflineEditsViewer is failing in Trunk. Contributed by hemanthboyina.
2019-11-20 00:07:43 +05:30
Ayush Saxena
34cb595d2e
HDFS-14967. TestWebHDFS fails in Windows. Contributed by Renukaprasad C.
2019-11-18 18:49:58 +05:30
Ayush Saxena
12617fad2e
HDFS-14955. RBF: getQuotaUsage() on mount point should return global quota. Contributed by Jinglun.
2019-11-18 16:04:52 +05:30
Ayush Saxena
3b5a0e86c1
HDFS-14974. RBF: Make tests use free ports. Contributed by Inigo Goiri.
2019-11-18 15:49:36 +05:30
Yiqun Lin
b3119b9ab6
HDFS-14648. Implement DeadNodeDetector basic model. Contributed by Lisheng Sun.
2019-11-16 11:32:41 +08:00
Hui Fei
67f2c491fe
HDFS-14802. The feature of protect directories should be used in RenameOp ( #1669 )
2019-11-15 12:35:37 -08:00
He Xiaoqiao
c892a879dd
HDFS-14882. Consider DataNode load when #getBlockLocation. Contributed by Xiaoqiao He.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: Istvan Fajth <pifta@cloudera.com>
2019-11-15 12:16:31 -08:00
Erik Krogen
b2cc8b6b4a
HDFS-14973. More strictly enforce Balancer/Mover/SPS throttling of getBlocks RPCs to NameNodes. Contributed by Erik Krogen.
2019-11-15 10:10:37 -08:00
Erik Krogen
586defe711
HDFS-14979 Allow Balancer to submit getBlocks calls to Observer Nodes when possible. Contributed by Erik Krogen.
2019-11-13 11:46:35 -08:00
Chao Sun
97ec34e117
HDFS-14959: [SBNN read] access time should be turned off ( #1706 ) Contributed by Chao Sun.
2019-11-12 11:35:50 -08:00
Inigo Goiri
40150da1e1
HDFS-14922. Prevent snapshot modification time got change on startup. Contributed by hemanthboyina.
2019-11-12 10:53:54 -08:00
Ayush Saxena
77934bc07b
HDFS-14975. Add CR for SetECPolicyCommand usage. Contributed by Fei Hui.
2019-11-11 11:05:34 +05:30
Wang Yuxuan
b25a37c322
HDFS-14962. RBF: ConnectionPool#newConnection() error log wrong protocol class ( #1699 ). Contributed by Yuxuan Wang.
2019-11-11 10:48:38 +05:30
Takanobu Asanuma
6663d6a5c2
HDFS-14928. UI: unifying the WebUI across different components. Contributed by Xieming Li
2019-11-11 14:04:57 +09:00
Surendra Singh Lilhore
320008bb7c
HDFS-14720. DataNode shouldn't report block as bad block if the block length is Long.MAX_VALUE. Contributed by hemanthboyina.
2019-11-09 22:14:35 +05:30
Ayush Saxena
42fc8884ab
HDFS-14815. RBF: Update the quota in MountTable when calling setQuota on a MountTable src. Contributed by Jinglun.
2019-11-08 08:48:52 +05:30
Ayush Saxena
247584eb63
HDFS-14958. TestBalancerWithNodeGroup is not using NetworkTopologyWithNodeGroup. Contributed by Jim Brennan.
2019-11-07 22:49:01 +05:30
Chen Liang
dd900259c4
HDFS-14941. Potential editlog race condition can cause corrupted file. Contributed by Chen Liang and Konstantin Shvachko.
2019-11-06 09:56:19 -08:00
Chen Liang
9d0d580031
HDFS-14806. Bootstrap standby may fail if with in-progress tailing. Contributed by Chen Liang and Erik Krogen.
2019-11-06 09:30:04 -08:00
Ayush Saxena
9e287054a8
HDFS-14942. Change Log Level to debug in JournalNodeSyncer#syncWithJournalAtIndex. Contributed by Lisheng Sun.
2019-11-06 22:02:03 +05:30
Ayush Saxena
dcf55838ae
HDFS-14880. Correct the sequence of statistics & exit message in balencer. Contributed by Renukaprasad C.
2019-11-06 21:45:03 +05:30
Surendra Singh Lilhore
c36014165c
HDFS-14384. When lastLocatedBlock token expire, it will take 1~3s second to refetch it. Contributed by Surendra Singh Lilhore.
2019-11-06 19:28:55 +05:30
Inigo Goiri
bfb8f28cc9
HDFS-14775. Add Timestamp for longest FSN write/read lock held log. Contributed by Chen Zhang.
2019-11-05 09:19:52 -08:00
Takanobu Asanuma
7901062707
HDFS-14891. RBF: namenode links in NameFederation Health page (federationhealth.html) cannot use https scheme. Contributed by Xieming Li
2019-11-05 15:03:09 +09:00
Ayush Saxena
b643a1cbe8
HDFS-14938. Add check if excludedNodes contain scope in DFSNetworkTopology#chooseRandomWithStorageType(). Contributed by Lisheng Sun.
2019-11-05 09:23:20 +05:30
Ayush Saxena
2ffec347eb
HDFS-14946. Erasure Coding: Block recovery failed during decommissioning. Contributed by Fei Hui.
2019-11-05 01:37:15 +05:30
Wei-Chiu Chuang
eb73ba6ed5
HDFS-14945. Revise PacketResponder's log. Contributed by Xudong Cao.
2019-11-04 09:40:42 -08:00
Zhao Junwang
de6b8b0c0b
HDFS-14925. Rename operation should check nest snapshot ( #1670 )
...
If the src directory or any of its descendant is snapshottable
and the dst directory or any of its ancestors is snapshottable,
we consider this as nested snapshot, which should be denied.
Reviewed-by: Shashikant Banerjee <shashikant@apache.org>
2019-11-01 16:37:53 -07:00
Ayush Saxena
7d7acb004a
HDFS-13736. BlockPlacementPolicyDefault can not choose favored nodes when 'dfs.namenode.block-placement-policy.default.prefer-local-node' set to false. Contributed by hu xiaodong.
2019-11-01 22:50:10 +05:30
Inigo Goiri
f18bbdd9d8
HDFS-14927. RBF: Add metrics for async callers thread pool. Contributed by Leon Gao.
2019-11-01 10:14:31 -07:00
Surendra Singh Lilhore
02009c3bb7
HDFS-14768. EC : Busy DN replica should be consider in live replica check. Contributed by guojh.
2019-11-01 22:15:05 +05:30
Ayush Saxena
9d25ae7669
HDFS-14920. Erasure Coding: Decommission may hang If one or more datanodes are out of service during decommission. Contributed by Fei Hui.
2019-10-31 23:49:37 +05:30
Ayush Saxena
d9fbedc4ae
HDFS-14936. Add getNumOfChildren() for interface InnerNode. Contributed by Lisheng Sun.
2019-10-31 23:39:27 +05:30
Jonathan Hung
e6137d0e50
Make upstream aware of 2.10.0 release
2019-10-30 20:58:32 -07:00
Ayush Saxena
fa4904cdca
HDFS-14935. Refactor DFSNetworkTopology#isNodeInScope. Contributed by Lisheng Sun.
2019-10-29 12:30:24 +05:30
Eric Yang
30ed24a421
HDFS-14730. Removed unused configuration dfs.web.authentication.filter.
...
Contributed by Chen Zhang
2019-10-28 19:27:32 -04:00
Eric Badger
9ef6ed9c1c
HDFS-14931. hdfs crypto commands limit column width. Contributed by Eric Badger
2019-10-28 16:37:04 +00:00
zhaoyim
30db895b59
HDFS-14308. DFSStripedInputStream curStripeBuf is not freed by unbuffer() ( #1667 )
...
Reviewed-by: Aravindan Vijayan <avijayan@cloudera.com>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-10-25 13:09:14 -07:00
Takanobu Asanuma
862526530a
HDFS-14933. Fixing a typo in documentation of Observer NameNode. Contributed by Xieming Li
2019-10-25 15:23:05 +09:00
Takanobu Asanuma
0db0f1e399
HDFS-14917. Change the ICON of "Decommissioned & dead" datanode on "dfshealth.html". Contributed by Xieming Li
2019-10-25 10:27:38 +09:00
Wei-Chiu Chuang
a1b4eebcc9
HDFS-14910. Rename Snapshot with Pre Descendants Fail With IllegalArgumentException. Contributed by Wei-Chiu Chuang. ( #1672 )
...
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
2019-10-24 13:09:52 -07:00
Eric Yang
ac6b6a6a85
HADOOP-16614. Add aarch64 support for dependent leveldbjni.
...
Contributed by liusheng
close #1546
2019-10-24 11:45:57 -04:00
Vinayakumar B
ee699dc26c
HDFS-14921. Remove SuperUser Check in Setting Storage Policy in FileStatus During Listing. Contributed by Ayush Saxena.
2019-10-24 12:14:09 +05:30
Mukul Kumar Singh
a901405ad8
HDFS-14884. Add sanity check that zone key equals feinfo key while setting Xattrs. Contributed by Mukul Kumar Singh.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-10-22 13:31:15 -07:00
Ayush Saxena
6020505943
HDFS-14915. Move Superuser Check Before Taking Lock For Encryption API. Contributed by Ayush Saxena.
2019-10-22 18:44:22 +05:30
Ayush Saxena
19f35cfd57
HDFS-14918. Remove useless getRedundancyThread from BlockManagerTestUtil. Contributed by Fei Hui.
2019-10-22 18:34:02 +05:30
Wei-Chiu Chuang
72003b19bf
HDFS-13901. INode access time is ignored because of race between open and rename. Contributed by Jinglun.
2019-10-21 17:31:43 -07:00
Kihwal Lee
c4e27ef773
HDFS-12749. DN may not send block report to NN after NN restart. Contributed by Xiaoqiao He.
2019-10-21 16:33:15 -05:00
Ayush Saxena
74c2329fc3
HDFS-14913. Correct the value of available count in DFSNetworkTopology#chooseRandomWithStorageType(). Contributed by Ayush Saxena.
2019-10-21 18:35:12 +05:30
Ayush Saxena
ff6a492d66
HDFS-14916. RBF: line breaks are missing from the output of 'hdfs dfsrouteradmin -ls'. Contributed by Xieming Li.
2019-10-21 18:20:00 +05:30
Ayush Saxena
447f46d962
HDFS-14847. Erasure Coding: Blocks are over-replicated while EC decommissioning. Contributed by Fei Hui.
2019-10-20 06:10:59 +05:30
Inigo Goiri
e6f95eb0f7
HDFS-14887. RBF: In Router Web UI, Observer Namenode Information displaying as Unavailable. Contributed by hemanthboyina.
2019-10-19 13:24:58 -07:00
Yuming Wang
3d41f33018
HADOOP-16152. Upgrade Eclipse Jetty version to 9.4.x. Contributed by Yuming Wang, Siyao Meng.
...
Co-authored-By: Siyao Meng <smeng@cloudera.com>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-10-18 08:25:02 -07:00
Surendra Singh Lilhore
54dc6b7d72
HDFS-14909. DFSNetworkTopology#chooseRandomWithStorageType() should not decrease storage count for excluded node which is already part of excluded scope. Contributed by Surendra Singh Lilhore.
2019-10-17 22:28:47 +05:30
Ayush Saxena
5527d79adb
HDFS-14810. Review FSNameSystem editlog sync. Contributed by Xiaoqiao He.
2019-10-17 21:56:30 +05:30
Ayush Saxena
375224edeb
HDFS-14739. RBF: LS command for mount point shows wrong owner and permission information. Contributed by Jinglun.
2019-10-16 19:26:01 +05:30
Surendra Singh Lilhore
336abbd873
HDFS-14886. In NameNode Web UI's Startup Progress page, Loading edits always shows 0 sec. Contributed by hemanthboyina.
2019-10-14 23:31:43 +05:30
Ashvin
fabd41fa48
HDFS-14856. Fetch file ACLs while mounting external store. ( #1478 )
2019-10-14 09:44:56 -07:00
Ayush Saxena
5f4641a120
HDFS-14238. A log in NNThroughputBenchmark should change log level to INFO instead of ERROR. Contributed by Shen Yinjie.
2019-10-12 21:06:23 +05:30
Ayush Saxena
6e5cd5273f
HDFS-14899. Use Relative URLS in Hadoop HDFS RBF. Contributed by David Mollitor.
2019-10-12 13:26:47 +05:30
Ayush Saxena
eeb58a07e2
HDFS-14898. Use Relative URLS in Hadoop HDFS HTTP FS. Contributed by David Mollitor.
2019-10-10 01:03:33 +05:30
Surendra Singh Lilhore
d76e2655ac
HDFS-14754. Erasure Coding : The number of Under-Replicated Blocks never reduced(addendum). Contributed by Surendra Singh Lilhore.
2019-10-09 23:20:28 +05:30
Chen Liang
72ae371e7a
HDFS-14509. DN throws InvalidToken due to inequality of password when upgrade NN 2.x to 3.x. Contributed by Yuxuan Wang and Konstantin Shvachko.
2019-10-08 11:56:52 -07:00
Ayush Saxena
91320b4461
HDFS-14859. Prevent unnecessary evaluation of costly operation getNumLiveDataNodes when dfs.namenode.safemode.min.datanodes is not zero. Contributed by Srinivasu Majeti.
2019-10-08 14:14:14 +05:30
Ayush Saxena
761594549e
HDFS-14814. RBF: RouterQuotaUpdateService supports inherited rule. Contributed by Jinglun.
2019-10-08 14:01:44 +05:30
Surendra Singh Lilhore
382967be51
HDFS-14373. EC : Decoding is failing when block group last incomplete cell fall in to AlignedStripe. Contributed by Surendra Singh Lilhore.
2019-10-08 00:14:30 +05:30
Eric Yang
aa24add8f0
HDFS-14890. Fixed namenode and journalnode startup on Windows.
...
Contributed by Siddharth Wagle
2019-10-04 13:13:10 -04:00
Takanobu Asanuma
b23bdaf085
HDFS-14879. Header was wrong in Snapshot web UI. Contributed by hemanthboyina
2019-10-04 16:47:06 +09:00
Stephen O'Donnell
c99a12167f
HDFS-14637. Namenode may not replicate blocks to meet the policy after enabling upgradeDomain. Contributed by Stephen O'Donnell.
...
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-10-03 22:13:50 -07:00
Virajith Jalaparti
844b766da5
HDFS-14889. Ability to check if a block has a replica on provided storage. Contributed by Ashvin Agrawal. ( #1573 )"
2019-10-03 21:32:48 -07:00
Ayush Saxena
a3fe404244
HDFS-14881. Safemode 'forceExit' option, doesn’t shown in help message. Contributed by Renukaprasad C.
2019-10-03 22:34:55 +05:30
Ayush Saxena
5a7483ca5c
HDFS-14888. RBF: Enable Parallel Test Profile for builds. Contributed by Ayush Saxena.
2019-10-03 19:21:27 +05:30
Chen Liang
1303255aee
HDFS-14858. [SBN read] Allow configurably enable/disable AlignmentContext on NameNode. Contributed by Chen Liang.
2019-10-02 16:38:20 -07:00
Akira Ajisaka
3df733c250
HDFS-14885. UI: Fix a typo on WebUI of DataNode. Contributed by Xieming Li.
2019-10-02 11:03:16 +09:00
Wei-Chiu Chuang
6ef6594c7e
HDFS-14492. Snapshot memory leak. Contributed by Wei-Chiu Chuang. ( #1370 )
...
* HDFS-14492. Snapshot memory leak. Contributed by Wei-Chiu Chuang.
Change-Id: I9e5e450c07ad70aa1905973896c4f627042dbd37
* Fix checkstyle
Change-Id: I16d4bd4f03a971e1ed36cf57d89dc42357ef8fbf
2019-10-01 21:16:46 +05:30
Konstantin V Shvachko
b3275ab1f2
HDFS-14305. Fix serial number calculation in BlockTokenSecretManager to avoid token key ID overlap between NameNodes. Contributed by Konstantin V Shvachko.
2019-09-30 16:48:10 -07:00
Konstantin V Shvachko
760b523e58
Revert "HDFS-14305. Fix serial number calculation in BlockTokenSecretManager to avoid token key ID overlap between NameNodes. Contributed by He Xiaoqiao."
...
This reverts commit 0feba4396f
.
2019-09-29 13:11:31 -07:00
Inigo Goiri
d8313b2274
HDFS-14850. Optimize FileSystemAccessService#getFileSystemConfiguration. Contributed by Lisheng Sun.
2019-09-28 17:20:44 -07:00
Ayush Saxena
2200871607
HDFS-14876. Remove unused imports from TestBlockMissingException.java and TestClose.java. Contributed by Lisheng Sun.
2019-09-28 22:03:53 +05:30
Ayush Saxena
c4c8d5fd0e
HDFS-14849. Erasure Coding: the internal block is replicated many times when datanode is decommissioning. Contributed by HuangTao.
2019-09-28 21:44:44 +05:30
Ayush Saxena
0d5d0b914a
Revert "HDFS-14849. Erasure Coding: the internal block is replicated many times when datanode is decommissioning. Contributed by HuangTao."
...
This reverts commit ce58c05f1d
.
2019-09-28 21:32:15 +05:30
Sahil Takiar
13b427fc05
HDFS-14564: Add libhdfs APIs for readFully; add readFully to ByteBufferPositionedReadable ( #963 ) Contributed by Sahil Takiar.
...
Reviewed-by: Siyao Meng <smeng@cloudera.com>
2019-09-27 12:25:03 -07:00
Ayush Saxena
ce58c05f1d
HDFS-14849. Erasure Coding: the internal block is replicated many times when datanode is decommissioning. Contributed by HuangTao.
2019-09-27 18:58:19 +05:30
Zhankun Tang
2870668cfa
Make upstream aware of 3.1.3 release.
2019-09-27 16:44:07 +08:00
Chen Liang
2adcc3c932
HDFS-14785. [SBN read] Change client logging to be less aggressive. Contributed by Chen Liang.
2019-09-26 15:48:19 -07:00
Inigo Goiri
b1e55cfb55
HDFS-14461. RBF: Fix intermittently failing kerberos related unit test. Contributed by Xiaoqiao He.
2019-09-26 15:26:23 -07:00
Ayush Saxena
1a2a352ecd
HDFS-11934. Add assertion to TestDefaultNameNodePort#testGetAddressFromConf. Contributed by Nikhil Navadiya.
2019-09-26 20:45:37 +05:30
Ayush Saxena
a3f68933d7
HDFS-14873. Fix dfsadmin doc for triggerBlockReport. Contributed by Fei Hui.
2019-09-26 19:54:07 +05:30
Takanobu Asanuma
bdaaa3bbf2
HDFS-14832. RBF: Add Icon for ReadOnly False. Contributed by hemanthboyina
2019-09-25 23:48:49 +09:00
Steve Loughran
e346e3638c
HADOOP-15691 Add PathCapabilities to FileSystem and FileContext.
...
Contributed by Steve Loughran.
This complements the StreamCapabilities Interface by allowing applications to probe for a specific path on a specific instance of a FileSystem client
to offer a specific capability.
This is intended to allow applications to determine
* Whether a method is implemented before calling it and dealing with UnsupportedOperationException.
* Whether a specific feature is believed to be available in the remote store.
As well as a common set of capabilities defined in CommonPathCapabilities,
file systems are free to add their own capabilities, prefixed with
fs. + schema + .
The plan is to identify and document more capabilities -and for file systems which add new features, for a declaration of the availability of the feature to always be available.
Note
* The remote store is not expected to be checked for the feature;
It is more a check of client API and the client's configuration/knowledge
of the state of the remote system.
* Permissions are not checked.
Change-Id: I80bfebe94f4a8bdad8f3ac055495735b824968f5
2019-09-25 12:16:41 +01:00
Prabhu Joseph
3f89084ac7
HDFS-14845. Ignore AuthenticationFilterInitializer for HttpFSServerWebServer and honor hadoop.http.authentication configs.
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-09-25 10:19:30 +09:00
Ayush Saxena
66400c1cbb
HDFS-14808. EC: Improper size values for corrupt ec block in LOG. Contributed by Ayush Saxena.
2019-09-25 01:31:15 +05:30
Ayush Saxena
eb96a3093e
HDFS-14655. [SBN Read] Namenode crashes if one of The JN is down. Contributed by Ayush Saxena.
2019-09-25 01:16:30 +05:30
Inigo Goiri
afa1006a53
HDFS-14843. Double Synchronization in BlockReportLeaseManager. Contributed by David Mollitor.
2019-09-24 09:58:42 -07:00
Inigo Goiri
816d3cb908
HDFS-14837. Review of Block.java. Contributed by David Mollitor.
2019-09-24 09:54:09 -07:00
Wei-Chiu Chuang
43203b466d
HDFS-14868. RBF: Fix typo in TestRouterQuota. Contributed by Jinglun.
...
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-09-24 09:38:58 -07:00
Ayush Saxena
c30e495557
HDFS-14853. NPE in DFSNetworkTopology#chooseRandomWithStorageType() when the excludedNode is not present. Contributed by Ranith Sardar.
2019-09-23 21:22:50 +05:30
Vinayakumar B
07c81e9bfc
HADOOP-16558. [COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes ( #1494 ). Contributed by Vinayakumar B.
2019-09-23 12:37:47 +05:30
Rohith Sharma K S
4c0a7a9e13
Make upstream aware of 3.2.1 release.
2019-09-23 06:20:54 +00:00
Rakesh Radhakrishnan
659c88801d
HDFS-14818. Check native pmdk lib by 'hadoop checknative' command. Contributed by Feilong He.
2019-09-22 22:02:54 +05:30
Ayush Saxena
aa938662f9
HDFS-14833. RBF: Router Update Doesn't Sync Quota. Contributed by Ayush Saxena.
2019-09-21 09:53:13 +05:30
Inigo Goiri
3f223bebfa
HDFS-14844. Make buffer of BlockReaderRemote#newBlockReader#BufferedOutputStream configurable. Contributed by Lisheng Sun.
2019-09-20 11:45:01 -07:00
Vinayakumar B
1654497f98
HADOOP-16557. [pb-upgrade] Upgrade protobuf.version to 3.7.1 ( #1432 )
...
HADOOP-16557. [pb-upgrade] Upgrade protobuf.version to 3.7.1. Contributed by Vinayakumar B.
2019-09-20 16:08:30 +05:30
Inigo Goiri
a79f286c6f
HDFS-14609. RBF: Security should use common AuthenticationFilter. Contributed by Chen Zhang.
2019-09-19 11:00:10 -07:00
Malcolm Taylor
56248f9d87
HADOOP-16556. Fix some alerts raised by LGTM.
...
Contributed by Malcolm Taylor.
Change-Id: Ic60c3f4681dd9d48b3afcba7520bd1e4d3cc4231
2019-09-19 16:00:05 +01:00
Chen Liang
4ed0aefe9f
HDFS-14822. [SBN read] Revisit GlobalStateIdContext locking when getting server state id. Contributed by Chen Liang.
2019-09-18 16:47:46 -07:00
Wei-Chiu Chuang
5dd859a8a0
HDFS-14836. FileIoProvider should not increase FileIoErrors metric in datanode volume metric. Contributed by Aiphago.
...
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
2019-09-18 09:34:01 -07:00
Sahil Takiar
3cf6e4272f
HDFS-14846: libhdfs tests are failing on trunk due to jni usage bugs
...
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-09-17 15:51:39 -07:00
Inigo Goiri
f580a87079
HDFS-14795. Add Throttler for writing block. Contributed by Lisheng Sun.
2019-09-17 14:55:34 -07:00
Ayush Saxena
e04b8a46c3
HDFS-14799. Do Not Call Map containsKey In Conjunction with get. Contributed by hemanthboyina.
2019-09-14 10:39:20 +05:30
Stephen O'Donnell
06ad6540b3
HDFS-14821. Make HDFS-14617 (fsimage sub-sections) off by default. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-09-13 10:36:45 -07:00
Surendra Singh Lilhore
4852a90e4b
HDFS-14754. Erasure Coding : The number of Under-Replicated Blocks never reduced. Contributed by hemanthboyina.
2019-09-13 00:17:13 +05:30
Surendra Singh Lilhore
2ff2a7f612
HDFS-14798. Synchronize invalidateBlocks in DatanodeDescriptor. Contributed by hemanthboyina.
2019-09-12 19:43:18 +05:30
Surendra Singh Lilhore
d1c303a497
HDFS-14699. Erasure Coding: Storage not considered in live replica when replication streams hard limit reached to threshold. Contributed by Zhao Yi Ming.
2019-09-12 19:11:50 +05:30
Akira Ajisaka
68612a0410
HDFS-14840. Use Java Conccurent Instead of Synchronization in BlockPoolTokenSecretManager. Contributed by David Mollitor.
2019-09-12 12:41:57 +09:00
Takanobu Asanuma
c255333e20
HDFS-14838. RBF: Display RPC (instead of HTTP) Port Number in RBF web UI. Contributed by Xieming Li
2019-09-11 16:54:08 +09:00
Takanobu Asanuma
524b553a5f
HDFS-14835. RBF: Secured Router should not run when it can't initialize DelegationTokenSecretManager. ( #1414 )
2019-09-11 10:32:07 +09:00
Erik Krogen
c92a3e94d8
HADOOP-15565. Add an inner FS cache to ViewFileSystem, separate from the global cache, to avoid file system leaks. Contributed by Jinglun.
2019-09-06 10:22:28 -07:00
Surendra Singh Lilhore
d98c54816d
HDFS-13913. LazyPersistFileScrubber.run() should log meaningful warn message. Contributed by Daniel Green.
2019-09-06 18:12:42 +05:30
Surendra Singh Lilhore
fa7f03fc56
HDFS-14826. dfs.ha.zkfc.port property duplicated in hdfs-default.xml. Contributed by Renukaprasad C.
2019-09-06 17:55:30 +05:30
Inigo Goiri
494d75eb2b
HDFS-14784. Add more methods to WebHdfsTestUtil to support tests outside of package. Contributed by Chen Zhang.
2019-09-05 21:15:17 -07:00
Inigo Goiri
72d8b92ba5
HDFS-12904. Add DataTransferThrottler to the Datanode transfers. Contributed by Lisheng Sun.
2019-09-05 11:44:02 -07:00
Ayush Saxena
2f70b52a5b
HDFS-14812. RBF: MountTableRefresherService should load cache when refresh. Contributed by xuzq.
2019-09-05 08:29:56 +05:30
Surendra Singh Lilhore
05704754a0
HDFS-14777. RBF: Set ReadOnly is failing for mount Table but actually readonly succeed to set. Contributed by Ranith Sardar.
2019-09-04 15:09:09 +05:30
Xiaoyu Yao
3c117163a3
HDFS-14633. The StorageType quota and consume in QuotaFeature is not handled for rename. Contributed by Jinglun.
2019-09-03 09:29:58 -07:00
Ayush Saxena
91b01a1db7
HDFS-14807. SetTimes updates all negative values apart from -1. Contributed by Ayush Saxena.
2019-09-03 18:08:42 +05:30
Surendra Singh Lilhore
5ff76cb8bc
HDFS-14630. Configuration.getTimeDurationHelper() should not log time unit warning in info log. Contributed by hemanthboyina.
2019-09-03 12:37:09 +05:30
Stephen O'Donnell
915cbc91c0
HDFS-14706. Checksums are not checked if block meta file is less than 7 bytes. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-09-02 09:47:04 -07:00
Wei-Chiu Chuang
d207aba026
Revert "HDFS-14706. Checksums are not checked if block meta file is less than 7 bytes. Contributed by Stephen O'Donnell."
...
This reverts commit 7bebad61d9
.
2019-09-02 09:43:44 -07:00
Ayush Saxena
040f6e93bb
HDFS-14654. RBF: TestRouterRpc#testNamenodeMetrics is flaky. Contributed by Chen Zhang.
2019-09-02 13:01:52 +05:30
Ayush Saxena
c3abfcefdd
HDFS-13843. RBF: Add optional parameter -d for detailed listing of mount points. Contributed by Ayush Saxena.
2019-09-02 07:24:04 +05:30
Ayush Saxena
18d74fe41c
HDFS-14711. RBF: RBFMetrics throws NullPointerException if stateStore disabled. Contributed by Chen Zhang.
2019-09-01 12:53:10 +05:30
Surendra Singh Lilhore
96f7dc1992
HDFS-14796. Define LOG instead of BlockManager.LOG in ErasureCodingWork/ReplicationWork. Contributed by Fei Hui.
2019-08-30 14:31:05 +05:30
Ayush Saxena
e220dac15c
HDFS-12212. Options.Rename.To_TRASH is considered even when Options.Rename.NONE is specified. Contributed by Vinayakumar B.
2019-08-30 08:40:13 +05:30
leosunli
915271245b
Make DataNodePeerMetrics#minOutlierDetectionSamples configurable ( #1314 ). Contributed by Lisheng Sun.
...
Signed-off-by: sunlisheng <sunlisheng@xiaomi.com>
2019-08-29 18:35:26 -07:00
Stephen O'Donnell
7bebad61d9
HDFS-14706. Checksums are not checked if block meta file is less than 7 bytes. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-29 17:38:07 -07:00
Istvan Fajth
fcb7884bfc
HDFS-8178. QJM doesn't move aside stale inprogress edits files. Contributed by Istvan Fajth.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-29 16:56:14 -07:00
Inigo Goiri
3b22fcd377
HDFS-14256. Review Logging of NameNode Class. Contributed by David Mollitor.
2019-08-29 14:21:09 -07:00
Inigo Goiri
ffca734c62
HDFS-14104. Review getImageTxIdToRetain. Contributed by David Mollitor.
2019-08-29 14:18:35 -07:00
He Xiaoqiao
f600fbb6c4
HDFS-11246. FSNameSystem#logAuditEvent should be called outside the read or write locks. Contributed by He Xiaoqiao, Kuhu Shukla.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Co-authored-by: Kuhu Shukla <kshukla@apache.org>
2019-08-29 10:10:52 -07:00
Ayush Saxena
8e779a151e
HDFS-14721. RBF: ProxyOpComplete is not accurate in FederationRPCPerformanceMonitor. Contributed by xuzq.
2019-08-29 20:08:38 +05:30
Surendra Singh Lilhore
29bd6f3fc3
HDFS-8631. WebHDFS : Support setQuota. Contributed by Chao Sun.
2019-08-28 23:58:23 +05:30
Inigo Goiri
48cb583906
HDFS-14710. RBF: Improve some RPC performance by using previous block. Contributed by xuzq.
2019-08-28 10:48:00 -07:00
CR Hota
6e37d65b03
HDFS-14760. Log INFO mode if snapshot usage and actual usage differ. Contributed by CR Hota.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-27 16:20:25 -07:00
He Xiaoqiao
dde9399b37
HDFS-14497. Addendum: Write lock held by metasave impact following RPC processing.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-27 15:26:59 -07:00
Shashikant Banerjee
ce8eb1283a
Revert "HDDS-1610. applyTransaction failure should not be lost on restart. Contributed by Shashikant Banerjee."
...
This reverts commit 62445021d5
as it has unintended changes in DirectoryWithSnapshotFeature class..
2019-08-27 23:23:44 +05:30
Jonathan Hung
8ab7020e64
HDFS-14779. Fix logging error in TestEditLog#testMultiStreamsLoadEditWithConfMaxTxns
2019-08-27 10:30:31 -07:00
Takanobu Asanuma
b69ac575a1
HDFS-14772. RBF: hdfs-rbf-site.xml can't be loaded automatically. Contributed by Yuxuan Wang
2019-08-27 14:32:32 +09:00
Arpit Agarwal
07e3cf952e
HDFS-2470. NN should automatically set permissions on dfs.namenode.*.dir. Contributed by Siddharth Wagle.
2019-08-26 15:57:55 -07:00
Ayush Saxena
d2225c8ca8
HDFS-14722. RBF: GetMountPointStatus should return mountTable information when getFileInfoAll throw IOException. Contributed by xuzq.
2019-08-24 08:47:43 +05:30
Inigo Goiri
0b796754b9
HDFS-14766. RBF: MountTableStoreImpl#getMountTableEntries returns extra entry. Contributed by Chen Zhang.
2019-08-23 14:53:57 -07:00
Chen Liang
ebef99dcf4
HDFS-14674. [SBN read] Got an unexpected txid when tail editlog. Contributed by wangzhaohui.
2019-08-23 12:18:43 -07:00
Erik Krogen
d699022fce
HDFS-13977. Override shouldForceSync in QuorumOutputStream to allow for proper auto-sync behavior. Contributed by Erik Krogen.
2019-08-23 11:33:21 -07:00
Wang Yuxuan
894e2300d6
HDFS-14761. RBF: MountTableResolver cannot invalidate cache correctly ( #1334 )
...
HDFS-14761. RBF: MountTableResolver cannot invalidate cache correctly
2019-08-23 10:46:39 -07:00
Ayush Saxena
c92de8209d
HDFS-14747. RBF: IsFileClosed should be return false when the file is open in multiple destination. Contributed by xuzq.
2019-08-23 19:08:07 +05:30
Akira Ajisaka
bd7baea5a5
HDFS-14396. Failed to load image from FSImageFile when downgrade from 3.x to 2.x. Contributed by Fei Hui.
2019-08-23 15:28:27 +09:00
Akira Ajisaka
abc8fde4ca
HDFS-13596. NN restart fails after RollingUpgrade from 2.x to 3.x. Contributed by Fei Hui.
2019-08-23 14:44:34 +09:00
Stephen O'Donnell
b67812ea21
HDFS-14617. Improve fsimage load time by writing sub-sections to the fsimage index ( #1028 ). Contributed by Stephen O'Donnell.
...
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
2019-08-22 17:09:57 -07:00
Stephen O'Donnell
93daf69f90
HDFS-14675. Increase Balancer Defaults Further. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Gabor Bota <gabota@apache.org>
2019-08-22 15:21:33 -07:00
Inigo Goiri
e04dcfdc57
HDFS-14583. FileStatus#toString() will throw IllegalArgumentException. Contributed by xuzq.
2019-08-22 10:22:38 -07:00
Surendra Singh Lilhore
76790a1e67
HDFS-14358. Provide LiveNode and DeadNode filter in DataNode UI. Contributed by hemanthboyina.
2019-08-22 12:27:45 +05:30
Ayush Saxena
52c77bc160
HDFS-14741. RBF: RecoverLease should be return false when the file is open in multiple destination. Contributed by xuzq
2019-08-22 08:57:22 +05:30
Ayush Saxena
f9029c4070
HDFS-14744. RBF: Non secured routers should not log in error mode when UGI is default. Contributed by CR Hota.
2019-08-22 02:40:39 +05:30
Ayush Saxena
0f598aed13
HDFS-14276. [SBN read] Reduce tailing overhead. Contributed by Wei-Chiu Chuang.
2019-08-22 02:25:06 +05:30
Inigo Goiri
93595febaa
HDFS-14756. RBF: getQuotaUsage may ignore some folders. Contributed by Chen Zhang.
2019-08-21 09:39:57 -07:00
Inigo Goiri
5eeb6da2d4
HDFS-14714. RBF: implement getReplicatedBlockStats interface. Contributed by Chen Zhang.
2019-08-21 09:38:17 -07:00
Wei-Chiu Chuang
57f7370174
Revert "HDFS-14476. lock too long when fix inconsistent blocks between disk and in-memory. Contributed by Sean Chow."
...
This reverts commit b58a35f374
.
2019-08-21 08:59:49 -07:00
Anu Engineer
8aaf5e1a14
HDFS-14759. HDFS cat logs an info message.
...
Contributed by Eric Badger.
2019-08-20 20:24:19 -07:00
Surendra Singh Lilhore
3a145e2918
HDFS-14582. Failed to start DN with ArithmeticException when NULL checksum used. Contributed by Surendra Singh Lilhore.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-20 15:55:26 -07:00
Sandeep Kumar
269b543367
HADOOP-16523. Minor spell mistake in comment : Line number 101 ( #388 )
2019-08-20 13:36:23 -07:00
Wei-Chiu Chuang
4cb22cd867
HDFS-14311. Multi-threading conflict at layoutVersion when loading block pool storage. Contributed by Yicong Cai.
2019-08-20 10:37:11 -07:00
Wei-Chiu Chuang
aa6995fde2
HDFS-13201. Fix prompt message in testPolicyAndStateCantBeNull. Contributed by chencan.
2019-08-20 10:32:55 -07:00
Sunil G
bd92462321
HDFS-14729. Upgrade Bootstrap and jQuery versions used in HDFS UIs. Contributed by Vivek Ratnavel Subramanian. This closes #1297
2019-08-20 18:42:19 +05:30
Shashikant Banerjee
62445021d5
HDDS-1610. applyTransaction failure should not be lost on restart. Contributed by Shashikant Banerjee.
2019-08-20 14:37:01 +05:30
Wei-Chiu Chuang
360a96f342
HDFS-13709. Report bad block to NN when transfer block encounter EIO exception. Contributed by Chen Zhang.
2019-08-19 13:08:55 -07:00
HUAN-PING SU
abae6ff2a2
HDFS-14746. Trivial test code update after HDFS-14687. Contributed by kevin su.
...
Reviewed-by: Surendra Singh Lilhore <surendralilhore@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-19 11:14:51 -07:00
Surendra Singh Lilhore
b8db5b9a98
HDFS-14687. Standby Namenode never come out of safemode when EC files are being written. Contributed by Surendra Singh Lilhore.
...
Reviewed-by: Siyao Meng <smeng@cloudera.com>
Reviewed-by: Wei-Chiu CHuang <weichiu@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-18 18:13:35 -07:00
Wei-Chiu Chuang
b58a35f374
HDFS-14476. lock too long when fix inconsistent blocks between disk and in-memory. Contributed by Sean Chow.
2019-08-18 17:46:00 -07:00
Wei-Chiu Chuang
d873ddd656
HDFS-14127. Add a description about the observer read configuration. Contributed by xiangheng.
2019-08-17 04:39:22 -07:00
Wei-Chiu Chuang
528378784f
HDFS-12012. Fix spelling mistakes in BPServiceActor.java. Contributed by chencan.
2019-08-17 04:37:05 -07:00
hunshenshi
a38b9e137e
HDFS-14456:HAState#prepareToEnterState neednt a lock ( #770 ) Contributed by hunshenshi.
2019-08-16 14:53:06 -07:00
LeonGao91
9a1d8cfaf5
HDFS-14678. Allow triggerBlockReport to a specific namenode. ( #1252 ). Contributed by Leon Gao.
2019-08-16 08:00:51 -07:00
Surendra Singh Lilhore
34681643e9
HDFS-14719. Correct the safemode threshold value in BlockManagerSafeMode. Contributed by hemanthboyina.
2019-08-15 12:12:18 +05:30
Shashikant Banerjee
0a85af959c
HDFS-13101. Yet another fsimage corruption related to snapshot. Contributed by Shashikant Banerjee.
2019-08-15 10:16:25 +05:30
Ayush Saxena
b06c2345ef
HDFS-14713. RBF: RouterAdmin supports refreshRouterArgs command but not on display. Contributed by wangzhaohui.
2019-08-14 20:44:31 +05:30
Siyao Meng
3c0382f1b9
HDFS-14595. HDFS-11848 breaks API compatibility. Contributed by Siyao Meng.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
2019-08-14 07:30:35 -07:00
Wei-Chiu Chuang
c13ec7ab66
HDFS-14491. More Clarity on Namenode UI Around Blocks and Replicas. Contributed by Siyao Meng.
2019-08-13 17:15:26 -07:00
Wei-Chiu Chuang
633b7c1cfe
HDFS-14625. Make DefaultAuditLogger class in FSnamesystem to Abstract. Contributed by hemanthboyina.
2019-08-13 16:50:49 -07:00
Masatake Iwasaki
da0006fe04
HDFS-14423. Percent (%) and plus (+) characters no longer work in WebHDFS.
...
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
2019-08-14 08:39:40 +09:00
Siyao Meng
6ae8bc3a4a
HDFS-14665. HttpFS: LISTSTATUS response is missing HDFS-specific fields ( #1267 ) Contributed by Siyao Meng.
2019-08-13 16:27:57 -07:00
Ayush Saxena
e9b6b81de4
HDFS-13505. Turn on HDFS ACLs by default. Contributed by Siyao Meng.
2019-08-13 19:17:10 +05:30
Ayush Saxena
454420e4f2
HDFS-14708. TestLargeBlockReport#testBlockReportSucceedsWithLargerLengthLimit fails in trunk. Contributed by Lisheng Sun.
2019-08-13 08:27:20 +05:30
Siyao Meng
c92b49876a
HDFS-14148. HDFS OIV ReverseXML SnapshotSection parser throws exception when there are more than one snapshottable directory ( #1274 ) Contributed by Siyao Meng.
2019-08-12 17:26:20 -07:00
Yiqun Lin
8a77a224c7
HDFS-13359. DataXceiver hung due to the lock in FsDatasetImpl#getBlockInputStream. Contributed by Yiqun Lin.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-09 18:40:28 -07:00
Siyao Meng
e02ffed1b1
HDFS-12125. Document the missing EC removePolicy command ( #1258 ) Contributed by Wenxin He, Siyao Meng.
...
Co-authored-by: Wenxin He <wenxin.he@gmail.com>
2019-08-09 18:00:22 -07:00
Wei-Chiu Chuang
ce3c5a3e3b
HDFS-14623. In NameNode Web UI, for Head the file (first 32K) old data is showing. Contributed by hemanthboyina.
2019-08-09 15:41:37 -07:00
Wei-Chiu Chuang
865021b8c9
HDFS-14195. OIV: print out storage policy id in oiv Delimited output. Contributed by Wang, Xinglong.
2019-08-09 15:37:29 -07:00
Rakesh Radhakrishnan
f6fa865d6f
HDFS-14700. Clean up pmem cache before setting pmem cache capacity. Contributed by Feilong He.
2019-08-09 14:07:54 +05:30
Wei-Chiu Chuang
6ad9a11494
HDFS-14693. NameNode should log a warning when EditLog IPC logger's pending size exceeds limit. Contributed by Xudong Cao.
2019-08-08 13:50:30 -07:00
Wei-Chiu Chuang
2265872c2d
HDFS-14705. Remove unused configuration dfs.min.replication. Contributed by CR Hota.
2019-08-08 13:48:29 -07:00
Wei-Chiu Chuang
28a848412c
HDFS-14701. Change Log Level to warn in SlotReleaser. Contributed by Lisheng Sun.
2019-08-08 13:46:31 -07:00
Wei-Chiu Chuang
23f91f68b8
HDFS-14662. Document the usage of the new Balancer "asService" parameter. Contributed by Chen Zhang.
2019-08-08 13:45:29 -07:00
Stephen O'Donnell
b0799148cf
HDFS-14459. ClosedChannelException silently ignored in FsVolumeList.addBlockPool(). Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-08 13:36:39 -07:00
Ayush Saxena
11f750e6a7
HDFS-14616. Add the warn log when the volume available space isn't enough. Contributed by liying.
2019-08-08 00:49:58 +05:30
Ayush Saxena
de64253892
HDFS-14608. DataNode#DataTransfer should be named. Contributed by Inigo Goiri.
2019-08-08 00:37:56 +05:30
Erik Krogen
827dbb11e2
HDFS-14370. Add exponential backoff to the edit log tailer to avoid spinning on empty edit tail requests. Contributed by Erik Krogen.
2019-08-07 09:25:58 -07:00
Yiqun Lin
a5bb1e8ee8
HDFS-14313. Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du. Contributed by Lisheng Sun.
2019-08-07 10:18:11 +08:00
Lokesh Jain
954ff36360
HDFS-14692. Upload button should not encode complete url. Contributed by Lokesh Jain.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-06 15:30:50 -07:00
Stephen O'Donnell
35e0a01d7b
HDFS-14557. JournalNode error: Can't scan a pre-transactional edit log. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-06 07:57:46 -07:00
Ayush Saxena
11272159bb
HDFS-14679. Failed to add erasure code policies with example template. Contributed by Yuan Zhou.
2019-08-06 08:42:15 +05:30
Ayush Saxena
8f40856f76
HDFS-14669. TestDirectoryScanner#testDirectoryScannerInFederatedCluster fails intermittently in trunk. Contributed by qiang Liu.
2019-08-03 12:15:41 +05:30
Ayush Saxena
e503db5f44
HDFS-12826. Document Saying the RPC port, But it's required IPC port in HDFS Federation Document. Contributed by usharani.
2019-08-03 12:05:46 +05:30
Erik Krogen
e7a0b8aa83
HDFS-14462 Ensure WebHDFS client throws the correct exception during writes. Contributed by Simbarashe Dzinamarira.
2019-08-02 11:48:31 -07:00
Wei-Chiu Chuang
b964b81f85
HDFS-14685. DefaultAuditLogger doesn't print CallerContext. Contributed by xuzq.
2019-08-02 10:54:32 -07:00
Wei-Chiu Chuang
c2d00c8450
HDFS-13131. Modifying testcase testEnableAndDisableErasureCodingPolicy. Contributed by chencan.
2019-08-01 19:11:43 -07:00
Siyao Meng
17e8cf501b
HDFS-14686. HttpFS: HttpFSFileSystem#getErasureCodingPolicy always returns null ( #1192 ) Contributed by Siyao Meng.
2019-08-01 17:15:22 -07:00
Siyao Meng
99bf1dc9eb
HDFS-14683. WebHDFS: Add erasureCodingPolicy field to GETCONTENTSUMMARY response ( #1189 ) Contributed by Siyao Meng.
2019-08-01 17:14:07 -07:00
Wei-Chiu Chuang
32607dbd98
HDFS-14631.The DirectoryScanner doesn't fix the wrongly placed replica. Contributed by Jinglun.
2019-08-01 15:48:36 -07:00
Ayush Saxena
89b102f916
HDFS-14661. RBF: updateMountTableEntry shouldn't update mountTableEntry if targetPath not exist. Contributed by xuzq.
2019-08-01 08:43:39 +05:30
Chao Sun
b008072044
HDFS-14681. RBF: TestDisableRouterQuota failed because port 8888 was occupied.
2019-07-31 10:24:48 -07:00
Wei-Chiu Chuang
99f88c30cb
HDFS-14419. Avoid repeated calls to the listOpenFiles function. Contributed by HuangTao.
2019-07-30 16:50:06 -07:00
Chao Sun
3ae775d740
HDFS-14034. Support getQuotaUsage API in WebHDFS. Contributed by Chao Sun.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-07-30 16:01:17 -07:00
Erik Krogen
1f26cc8705
HDFS-13783. Add an option to the Balancer to make it run as a long-running service. Contributed by Chen Zhang.
2019-07-30 15:42:55 -07:00
Inigo Goiri
7849bdcf70
HDFS-14449. Expose total number of DT in JMX for Namenode. Contributed by Fengnan Li.
2019-07-30 13:45:27 -07:00
Ayush Saxena
8deced0073
HDFS-14677. TestDataNodeHotSwapVolumes#testAddVolumesConcurrently fails intermittently in trunk. Contributed by Chen Zhang.
2019-07-30 22:17:39 +05:30
Wei-Chiu Chuang
8053085388
HDFS-14429. Block remain in COMMITTED but not COMPLETE caused by Decommission. Contributed by Yicong Cai.
2019-07-29 14:31:34 -07:00
CR Hota
611718f59f
HDFS-14670: RBF: Create secret manager instance using FederationUtil#newInstance.
2019-07-29 11:00:22 -07:00
Ayush Saxena
02bd02b5af
HDFS-14660. [SBN Read] ObserverNameNode should throw StandbyException for requests not from ObserverProxyProvider. Contributed by Chao Sun.
2019-07-28 08:41:42 +05:30
hunshenshi
2fe450cb5e
HDFS-14425. Native build fails on macos due to jlong in hdfs.c ( #741 )
2019-07-26 15:33:12 -07:00
HUAN-PING SU
ecc8acfd24
HDFS-14673. The console log is noisy when using DNSDomainNameResolver to resolve NameNode.
2019-07-26 11:08:46 -07:00
Ayush Saxena
ce99cc31e9
HDFS-14303. Addendum: check block directory logic not correct when there is only meta file, print no meaning warn log. Contributed by qiang Liu.
2019-07-26 08:31:00 +05:30
Masatake Iwasaki
6b8107ad97
HDFS-14135. TestWebHdfsTimeouts Fails intermittently in trunk. Contributed by Ayush Saxena.
...
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
2019-07-26 09:35:23 +09:00
Ayush Saxena
62deab17a3
HDFS-14647. NPE during secure namenode startup. Contributed by Fengnan Li.
2019-07-25 06:51:07 +05:30
Ayush Saxena
377f95bbe8
HDFS-13693. Remove unnecessary search in INodeDirectory.addChild during image loading. Contributed by Lisheng Sun.
2019-07-23 08:37:55 +05:30
Ayush Saxena
6282c02d7d
HDFS-14577. RBF: FederationUtil#newInstance should allow constructor without context. Contributed by CR Hota.
2019-07-19 17:44:13 +05:30
Takanobu Asanuma
9792f58662
HDFS-14653. RBF: Correct the default value for dfs.federation.router.namenode.heartbeat.enable. Contributed by Ayush Saxena.
2019-07-18 10:16:20 +09:00
Chen Liang
5e6cc6fe8a
HDFS-12979. [SBN read] StandbyNode should upload FsImage to ObserverNode after checkpointing. Contributed by Chen Liang.
2019-07-17 14:37:16 -07:00
Ayush Saxena
af0665c7f5
HDFS-13647. Fix the description of storageType option for space quota. Contributed by Takanobu Asanuma.
2019-07-17 23:22:32 +05:30
Ayush Saxena
f77d54c243
HDFS-14642. processMisReplicatedBlocks does not return correct processed count. Contributed by Stephen O'Donnell.
2019-07-16 08:14:27 +05:30
Ayush Saxena
64d4abf489
HDFS-14593. RBF: Implement deletion feature for expired records in State Store. Contributed by Takanobu Asanuma.
2019-07-15 22:38:00 +05:30
Rakesh Radhakrishnan
30a8f840f1
HDFS-14357. Update documentation for HDFS cache on SCM support. Contributed by Feilong He.
2019-07-15 13:18:23 +05:30
Rakesh Radhakrishnan
e98adb00b7
HDFS-14458. Report pmem stats to namenode. Contributed by Feilong He.
2019-07-15 13:02:37 +05:30
Shashikant Banerjee
f9fab9f22a
HDFS-14499. Misleading REM_QUOTA value with snapshot and trash feature enabled for a directory. Contributed by Shashikant Banerjee.
2019-07-12 15:42:06 +05:30
Masatake Iwasaki
00dd843a1a
HDFS-14466. Add a regression test for HDFS-14323.
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-07-12 10:04:55 +09:00
Chen Liang
8fb5ca3f40
HDFS-14611. Move handshake secret field from Token to BlockAccessToken. Contributed by Chen Liang.
2019-07-11 13:23:25 -07:00
Inigo Goiri
5747f6cff5
HDFS-14624. When decommissioning a node, log remaining blocks to replicate periodically. Contributed by Stephen O'Donnell.
2019-07-11 08:55:44 -07:00
Inigo Goiri
eccc9a40de
HDFS-12703. Exceptions are fatal to decommissioning monitor. Contributed by He Xiaoqiao.
2019-07-10 11:11:52 -07:00
Ajay Yadav
cdb20adfcc
HDDS-1611. Evaluate ACL on volume bucket key and prefix to authorize access. Contributed by Ajay Kumar. ( #973 )
2019-07-10 11:03:58 -07:00
Yiqun Lin
993dc8726b
HDFS-14632. Reduce useless #getNumLiveDataNodes call in SafeModeMonitor. Contributed by He Xiaoqiao.
2019-07-10 10:53:34 +08:00
Erik Krogen
4632708148
HDFS-14547. Improve memory efficiency of quotas when storage type quotas are not set. Contributed by Jinglun.
2019-07-08 14:46:25 -07:00
Wei-Chiu Chuang
ec851e4db2
HDFS-12862. CacheDirective becomes invalid when NN restart or failover. Contributed by Wang XL.
2019-07-08 15:23:46 +08:00
leosunli
9c90729486
HDFS-13694. Making md5 computing being in parallel with image loading.
2019-07-05 10:17:37 -07:00
Inigo Goiri
c3ca348b81
HDFS-14620. RBF: Fix 'not a super user' error when disabling a namespace in kerberos with superuser principal. Contributed by luhuachao.
2019-07-04 11:21:55 -07:00
Masatake Iwasaki
1c254a8ec7
HDFS-14629. Property value Hard Coded in DNConf.java. Contributed by hemanthboyina.
2019-07-04 23:20:42 +09:00
Weiwei Yang
729cb3aefe
HDFS-12748. NameNode memory leak when accessing webhdfs GETHOMEDIRECTORY. Contributed by Weiwei Yang.
2019-07-04 09:37:24 +08:00
paulward24
d8bac50e12
HDFS-14610. HashMap is not thread safe. Field storageMap is typically synchronized by storageMap. However, in one place, field storageMap is not protected with synchronized. ( #1015 )
2019-07-01 13:54:49 -07:00
paulward24
d203045c30
HDFS-14618. Incorrect synchronization of ArrayList field (ArrayList is thread-unsafe). Contributed by Paul Ward.
2019-06-28 16:06:43 -07:00
Inigo Goiri
4a212242d9
HDFS-14599. HDFS-12487 breaks test TestDiskBalancer.testDiskBalancerWithFedClusterWithOneNameServiceEmpty. Contributed by He Xiaoqiao.
2019-06-27 10:00:05 -07:00
Inigo Goiri
041e0c0564
HDFS-14036. RBF: Add hdfs-rbf-default.xml to HdfsConfiguration by default. Contributed by Takanobu Asanuma.
2019-06-27 09:57:52 -07:00
Anu Engineer
041e7a7dee
HDFS-14598. Findbugs warning caused by HDFS-12487.
...
Contributed by He Xiaoqiao.
2019-06-24 19:07:22 -07:00
Inigo Goiri
b76b843c8b
HDFS-13371. NPE for FsServerDefaults.getKeyProviderUri() for clientProtocol communication between 2.7 and 3.X. Contributed by Sherwood Zheng.
2019-06-24 17:52:33 -07:00
Inigo Goiri
38a560c6f1
Merge pull request #977 from leosunli/trunk
...
HDFS-14541. When evictableMmapped or evictable size is zero, do not throw NoSuchElementException.
2019-06-24 17:44:54 -07:00
Ajay Kumar
95c94dcca7
HADOOP-16350. Ability to tell HDFS client not to request KMS Information from NameNode. Ccontributed by Greg Senia, Ajay Kumar.
2019-06-24 11:59:07 -07:00
Inigo Goiri
719d57bf46
HDFS-14545. RBF: Router should support GetUserMappingsProtocol. Contributed by Ayush Saxena.
2019-06-24 22:03:04 +05:30
Akira Ajisaka
8627f6532b
HDFS-14550. RBF: Failed to get statistics from NameNodes before 2.9.0. Contributed by He Xiaoqiao.
2019-06-24 22:03:04 +05:30
Ayush Saxena
ba221dc50e
HDFS-14526. RBF: Update the document of RBF related metrics. Contributed by Takanobu Asanuma.
2019-06-24 22:03:04 +05:30
Ayush Saxena
ade8d3b60e
HDFS-14508. RBF: Clean-up and refactor UI components. Contributed by Takanobu Asanuma.
2019-06-24 22:03:04 +05:30
Ayush Saxena
6915d7e13c
HDFS-13480. RBF: Separate namenodeHeartbeat and routerHeartbeat to different config key. Contributed by Ayush Saxena.
2019-06-24 22:03:04 +05:30
Ayush Saxena
ddbe08db33
HDFS-13955. RBF: Support secure Namenode in NamenodeHeartbeatService. Contributed by CR Hota.
2019-06-24 22:03:04 +05:30
Inigo Goiri
b062dd462d
HDFS-14475. RBF: Expose router security enabled status on the UI. Contributed by CR Hota.
2019-06-24 22:03:04 +05:30
Ayush Saxena
b6fff8c81e
HDFS-13787. RBF: Add Snapshot related ClientProtocol APIs. Contributed by Inigo Goiri.
2019-06-24 22:03:04 +05:30
Ayush Saxena
d240eec136
HDFS-14516. RBF: Create hdfs-rbf-site.xml for RBF specific properties. Contributed by Takanobu Asanuma.
2019-06-24 22:03:04 +05:30
Ayush Saxena
9b197c2893
HDFS-13909. RBF: Add Cache pools and directives related ClientProtocol APIs. Contributed by Ayush Saxena.
2019-06-24 22:03:04 +05:30
Ayush Saxena
ffbb6b6557
HDFS-13255. RBF: Fail when try to remove mount point paths. Contributed by Akira Ajisaka.
2019-06-24 22:03:04 +05:30
Ayush Saxena
8e4267650f
HDFS-14440. RBF: Optimize the file write process in case of multiple destinations. Contributed by Ayush Saxena.
2019-06-24 22:03:04 +05:30
Brahma Reddy Battula
2636a54ffd
HDFS-13995. RBF: Security documentation. Contributed by CR Hota.
2019-06-24 22:03:04 +05:30
Giovanni Matteo Fumarola
393f15176d
HDFS-14447. RBF: Router should support RefreshUserMappingsProtocol. Contributed by Shen Yinjie.
2019-06-24 22:03:04 +05:30
Giovanni Matteo Fumarola
3deb5d345f
HDFS-14490. RBF: Remove unnecessary quota checks. Contributed by Ayush Saxena.
2019-06-24 22:03:04 +05:30
Ayush Saxena
62fa53a01d
HDFS-14210. RBF: ACL commands should work over all the destinations. Contributed by Ayush Saxena.
2019-06-24 22:03:04 +05:30
Giovanni Matteo Fumarola
32841178ba
HDFS-14426. RBF: Add delegation token total count as one of the federation metrics. Contributed by Fengnan Li.
2019-06-24 22:03:04 +05:30
Ayush Saxena
203664e6b2
HDFS-14454. RBF: getContentSummary() should allow non-existing folders. Contributed by Inigo Goiri.
2019-06-24 22:03:04 +05:30
Ayush Saxena
b522b52bb1
HDFS-14457. RBF: Add order text SPACE in CLI command 'hdfs dfsrouteradmin'. Contributed by luhuachao.
2019-06-24 22:03:04 +05:30
Brahma Reddy Battula
021a43b1a4
HDFS-13972. RBF: Support for Delegation Token (WebHDFS). Contributed by CR Hota.
2019-06-24 22:03:04 +05:30
Ayush Saxena
de7da9b69e
HDFS-14422. RBF: Router shouldn't allow READ operations in safe mode. Contributed by Inigo Goiri.
2019-06-24 22:03:04 +05:30
Ayush Saxena
0f9b8d7a75
HDFS-14369. RBF: Fix trailing / for webhdfs. Contributed by Akira Ajisaka.
2019-06-24 22:03:04 +05:30
Ayush Saxena
dd8c2b92df
HDFS-13853. RBF: RouterAdmin update cmd is overwriting the entry not updating the existing. Contributed by Ayush Saxena.
2019-06-24 22:03:04 +05:30
Ayush Saxena
6c42d40504
HDFS-14316. RBF: Support unavailable subclusters for mount points with multiple destinations. Contributed by Inigo Goiri.
2019-06-24 22:03:04 +05:30
Ayush Saxena
0dbd87874a
HDFS-14388. RBF: Prevent loading metric system when disabled. Contributed by Inigo Goiri.
2019-06-24 22:03:03 +05:30
Ayush Saxena
9a9fbbe145
HDFS-14351. RBF: Optimize configuration item resolving for monitor namenode. Contributed by He Xiaoqiao and Inigo Goiri.
2019-06-24 22:03:03 +05:30
Giovanni Matteo Fumarola
f539e2a4ee
HDFS-14343. RBF: Fix renaming folders spread across multiple subclusters. Contributed by Ayush Saxena.
2019-06-24 22:03:03 +05:30
Giovanni Matteo Fumarola
2a2d5eb441
HDFS-14334. RBF: Use human readable format for long numbers in the Router UI. Contributed by Inigo Goiri.
2019-06-24 22:03:03 +05:30
Inigo Goiri
fcabc8f0e4
HDFS-14335. RBF: Fix heartbeat typos in the Router. Contributed by CR Hota.
2019-06-24 22:03:03 +05:30
Inigo Goiri
6cdf8db55c
HDFS-14331. RBF: IOE While Removing Mount Entry. Contributed by Ayush Saxena.
2019-06-24 22:03:03 +05:30
Inigo Goiri
1ce25e702b
HDFS-14329. RBF: Add maintenance nodes to federation metrics. Contributed by Ayush Saxena.
2019-06-24 22:03:03 +05:30
Inigo Goiri
7bbe01a196
HDFS-14259. RBF: Fix safemode message for Router. Contributed by Ranith Sadar.
2019-06-24 22:03:03 +05:30
Inigo Goiri
1c7ab59be3
HDFS-14322. RBF: Security manager should not load if security is disabled. Contributed by CR Hota.
2019-06-24 22:03:03 +05:30
Brahma Reddy Battula
c4b1fa91fa
HDFS-14052. RBF: Use Router keytab for WebHDFS. Contributed by CR Hota.
2019-06-24 22:03:03 +05:30
Inigo Goiri
de719b08b5
HDFS-14307. RBF: Update tests to use internal Whitebox instead of Mockito. Contributed by CR Hota.
2019-06-24 22:03:03 +05:30
Giovanni Matteo Fumarola
8b8ff5ccbc
HDFS-14249. RBF: Tooling to identify the subcluster location of a file. Contributed by Inigo Goiri.
2019-06-24 22:03:03 +05:30
Giovanni Matteo Fumarola
50aee18a84
HDFS-14268. RBF: Fix the location of the DNs in getDatanodeReport(). Contributed by Inigo Goiri.
2019-06-24 22:03:03 +05:30
Inigo Goiri
e2a3c4494b
HDFS-14226. RBF: Setting attributes should set on all subclusters' directories. Contributed by Ayush Saxena.
2019-06-24 22:03:03 +05:30
Brahma Reddy Battula
75f8b6ccfa
HDFS-13358. RBF: Support for Delegation Token (RPC). Contributed by CR Hota.
2019-06-24 22:03:03 +05:30
Inigo Goiri
7e63e37dc5
HDFS-14230. RBF: Throw RetriableException instead of IOException when no namenodes available. Contributed by Fei Hui.
2019-06-24 22:03:03 +05:30
Giovanni Matteo Fumarola
912b90f91e
HDFS-14252. RBF : Exceptions are exposing the actual sub cluster path. Contributed by Ayush Saxena.
2019-06-24 22:03:03 +05:30
Surendra Singh Lilhore
9c4e55685b
HDFS-14225. RBF : MiniRouterDFSCluster should configure the failover proxy provider for namespace. Contributed by Ranith Sardar.
2019-06-24 22:03:03 +05:30
Inigo Goiri
9eed3a49df
HDFS-14215. RBF: Remove dependency on availability of default namespace. Contributed by Ayush Saxena.
2019-06-24 22:03:03 +05:30
Brahma Reddy Battula
acdf911c01
HDFS-14224. RBF: NPE in getContentSummary() for getEcPolicy() in case of multiple destinations. Contributed by Ayush Saxena.
2019-06-24 22:03:03 +05:30
Brahma Reddy Battula
8b9b58b58a
HDFS-14223. RBF: Add configuration documents for using multiple sub-clusters. Contributed by Takanobu Asanuma.
2019-06-24 22:03:03 +05:30
Yiqun Lin
020f83f51f
HDFS-14209. RBF: setQuota() through router is working for only the mount Points under the Source column in MountTable. Contributed by Shubham Dewan.
2019-06-24 22:03:03 +05:30
Inigo Goiri
235406d904
HDFS-14156. RBF: rollEdit() command fails with Router. Contributed by Shubham Dewan.
2019-06-24 22:03:03 +05:30
Vinayakumar B
c012b09fb6
HDFS-14193. RBF: Inconsistency with the Default Namespace. Contributed by Ayush Saxena.
2019-06-24 22:03:03 +05:30
Surendra Singh Lilhore
7b61cbf672
HDFS-14129. addendum to HDFS-14129. Contributed by Ranith Sardar.
2019-06-24 22:03:03 +05:30
Surendra Singh Lilhore
f40e10b349
HDFS-14129. RBF: Create new policy provider for router. Contributed by Ranith Sardar.
2019-06-24 22:03:03 +05:30
Yiqun Lin
221f24cbdc
HDFS-14206. RBF: Cleanup quota modules. Contributed by Inigo Goiri.
2019-06-24 22:03:02 +05:30
Inigo Goiri
f4e2bfce58
HDFS-13856. RBF: RouterAdmin should support dfsrouteradmin -refreshRouterArgs command. Contributed by yanghuafeng.
2019-06-24 22:03:02 +05:30
Surendra Singh Lilhore
b8bcbd0ed2
HDFS-14191. RBF: Remove hard coded router status from FederationMetrics. Contributed by Ranith Sardar.
2019-06-24 22:03:02 +05:30
Yiqun Lin
4244653e43
HDFS-14150. RBF: Quotas of the sub-cluster should be removed when removing the mount point. Contributed by Takanobu Asanuma.
2019-06-24 22:03:02 +05:30
Inigo Goiri
f3cbf0eb9a
HDFS-14161. RBF: Throw StandbyException instead of IOException so that client can retry when can not get connection. Contributed by Fei Hui.
2019-06-24 22:03:02 +05:30
Inigo Goiri
1dc01e59af
HDFS-14167. RBF: Add stale nodes to federation metrics. Contributed by Inigo Goiri.
2019-06-24 22:03:02 +05:30
Yiqun Lin
8f6f9d9c83
HDFS-13443. RBF: Update mount table cache immediately after changing (add/update/remove) mount table entries. Contributed by Mohammad Arshad.
2019-06-24 22:03:02 +05:30
Takanobu Asanuma
3d97142dff
HDFS-14151. RBF: Make the read-only column of Mount Table clearly understandable.
2019-06-24 22:03:02 +05:30
Yiqun Lin
bbe859177d
HDFS-13869. RBF: Handle NPE for NamenodeBeanMetrics#getFederationMetrics. Contributed by Ranith Sardar.
2019-06-24 22:03:02 +05:30
Takanobu Asanuma
01b4126b4e
HDFS-14152. RBF: Fix a typo in RouterAdmin usage. Contributed by Ayush Saxena.
2019-06-24 22:03:02 +05:30
Yiqun Lin
94a8dec168
HDFS-14114. RBF: MIN_ACTIVE_RATIO should be configurable. Contributed by Fei Hui.
2019-06-24 22:03:02 +05:30
Yiqun Lin
0ca7142c11
Revert "HDFS-14114. RBF: MIN_ACTIVE_RATIO should be configurable. Contributed by Fei Hui."
...
This reverts commit 7c0d6f65fde12ead91ed7c706521ad1d3dc995f8.
2019-06-24 22:03:02 +05:30
Yiqun Lin
6aa7aabff8
HDFS-14114. RBF: MIN_ACTIVE_RATIO should be configurable. Contributed by Fei Hui.
2019-06-24 22:03:02 +05:30
Surendra Singh Lilhore
b320caecb3
HDFS-14085. RBF: LS command for root shows wrong owner and permission information. Contributed by Ayush Saxena.
2019-06-24 22:03:02 +05:30
Brahma Reddy Battula
19088e1b49
HDFS-14089. RBF: Failed to specify server's Kerberos pricipal name in NamenodeHeartbeatService. Contributed by Ranith Sardar.
2019-06-24 22:03:02 +05:30
Brahma Reddy Battula
f2355c7063
HDFS-13776. RBF: Add Storage policies related ClientProtocol APIs. Contributed by Dibyendu Karmakar.
2019-06-24 22:03:02 +05:30
Yiqun Lin
f4bd1114ff
HDFS-14082. RBF: Add option to fail operations when a subcluster is unavailable. Contributed by Inigo Goiri.
2019-06-24 22:03:02 +05:30
Inigo Goiri
fa55eacd35
HDFS-13834. RBF: Connection creator thread should catch Throwable. Contributed by CR Hota.
2019-06-24 22:03:02 +05:30
Inigo Goiri
04caaba488
HDFS-13852. RBF: The DN_REPORT_TIME_OUT and DN_REPORT_CACHE_EXPIRE should be configured in RBFConfigKeys. Contributed by yanghuafeng.
2019-06-24 22:03:02 +05:30
Brahma Reddy Battula
ebfd2d8a4e
HDFS-12284. addendum to HDFS-12284. Contributed by Inigo Goiri.
2019-06-24 22:03:02 +05:30
Brahma Reddy Battula
6f2c871b05
HDFS-12284. RBF: Support for Kerberos authentication. Contributed by Sherwood Zheng and Inigo Goiri.
2019-06-24 22:03:02 +05:30
Inigo Goiri
7b0bc492ed
HDFS-14024. RBF: ProvidedCapacityTotal json exception in NamenodeHeartbeatService. Contributed by CR Hota.
2019-06-24 22:03:02 +05:30
Brahma Reddy Battula
c5065bf20b
HDFS-13845. RBF: The default MountTableResolver should fail resolving multi-destination paths. Contributed by yanghuafeng.
2019-06-24 22:03:02 +05:30
Yiqun Lin
b3fee1d2bf
HDFS-14011. RBF: Add more information to HdfsFileStatus for a mount point. Contributed by Akira Ajisaka.
2019-06-24 22:03:02 +05:30
Vinayakumar B
41c94a636b
HDFS-13906. RBF: Add multiple paths for dfsrouteradmin 'rm' and 'clrquota' commands. Contributed by Ayush Saxena.
2019-06-24 22:03:02 +05:30
Wei-Chiu Chuang
05145404d5
HDFS-14339. Inconsistent log level practices in RpcProgramNfs3.java. Contributed by Anuhan Torgonshar.
2019-06-24 08:30:48 -07:00
sunlisheng
daa1e14745
when evictableMmapped or evictable size is zero, do not throw NoSuchElementException
...
Signed-off-by: sunlisheng <sunlisheng@xiaomi.com>
2019-06-24 20:17:22 +08:00
Wei-Chiu Chuang
645d67bc4f
HDFS-14074. DataNode runs async disk checks maybe throws NullPointerException, and DataNode failed to register to NameSpace. Contributed by guangyi lu.
2019-06-21 18:27:03 -07:00
Wei-Chiu Chuang
1524e2e6c5
HDFS-12487. FsDatasetSpi.isValidBlock() lacks null pointer check inside and neither do the callers. Contributed by liumi.
2019-06-21 18:17:36 -07:00
Lokesh Jain
272b96d243
HDFS-13893. DiskBalancer: no validations for Disk balancer commands. Contributed by Lokesh Jain.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-20 20:23:05 -07:00
Takanobu Asanuma
98d2065643
HDFS-12564. Add the documents of swebhdfs configurations on the client side. Contributed by Takanobu Asanuma.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-20 20:17:24 -07:00
Surendra Singh Lilhore
5962a518bd
HDFS-14581. Appending to EC files crashes NameNode. Contributed by Surendra Singh Lilhore.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-20 09:40:29 -07:00
Wei-Chiu Chuang
71ecd2e411
HDFS-14303. check block directory logic not correct when there is only meta file, print no meaning warn log. Contributed by qiang Liu.
2019-06-19 10:27:53 -07:00
David Mollitor
f5ecc0bc08
HDFS-14103. Review Logging of BlockPlacementPolicyDefault. Contributed by David Mollitor.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-19 10:06:40 -07:00
Wei-Chiu Chuang
d3ac516665
Revert "HDFS-13287. TestINodeFile#testGetBlockType results in NPE when run alone. Contributed by Virajith Jalaparti."
...
This reverts commit a1c3868c4f
.
2019-06-19 06:27:02 -07:00
Wei-Chiu Chuang
48e564f7e2
HDFS-14537. Journaled Edits Cache is not cleared when formatting the JN. Contributed by Ranith Sardar.
2019-06-18 22:28:21 -07:00
Virajith Jalaparti
a1c3868c4f
HDFS-13287. TestINodeFile#testGetBlockType results in NPE when run alone. Contributed by Virajith Jalaparti.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-18 22:17:49 -07:00
Bharat Viswanadham
f9ee97de98
HDFS-14398. Update HAState.java to fix typos. Contributed by Nikhil Navadiya.
2019-06-18 21:58:37 -07:00
Elek, Marton
81ec909411
HDFS-14078. Admin helper fails to prettify NullPointerExceptions. Contributed by Elek, Marton.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-18 12:01:26 -07:00
Shweta Yakkali
3c1a1ceea9
HDFS-14487. Missing Space in Client Error Message (Contributed by Shweta Yakkali via Daniel Templeton)
...
Change-Id: I0f8ce74a35ab24fe94fd0e57d8247bb3fa575e6f
2019-06-18 10:21:22 -07:00
Inigo Goiri
3ab77d9bc9
HDFS-14201. Ability to disallow safemode NN to become active. Contributed by Xiao Liang and He Xiaoqiao.
2019-06-18 09:58:29 -07:00
Wei-Chiu Chuang
335c1c9938
HDFS-14010. Pass correct DF usage to ReservedSpaceCalculator builder. Contributed by Virajith Jalaparti.
2019-06-18 00:35:48 -07:00
Oleg Danilov
54cdde38c7
HDFS-12314. Typo in the TestDataNodeHotSwapVolumes.testAddOneNewVolume(). Contributed by Oleg Danilov.
2019-06-17 23:11:25 -07:00
Oleg Danilov
d41310a15d
HDFS-12315. Use Path instead of String to check closedFiles set. Contributed by Oleg Danilov.
2019-06-17 23:06:02 -07:00
Wei-Chiu Chuang
098c325a78
HDFS-14340. Lower the log level when can't get postOpAttr. Contributed by Anuhan Torgonshar.
2019-06-17 22:45:52 -07:00
Weiwei Yang
e3172221ea
HDFS-12770. Add doc about how to disable client socket cache. Contributed by Weiwei Yang.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-17 21:48:16 -07:00
Wei-Chiu Chuang
62ad9885ea
HDFS-13730. BlockReaderRemote.sendReadResult throws NPE. Contributed by Yuanbo Liu.
2019-06-17 20:18:53 -07:00
Akira Ajisaka
a95e87d827
HDFS-11950. Disable libhdfs zerocopy test on Mac. Contributed by Akira Ajisaka.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-17 17:03:25 -07:00
Santosh Marella
6822193ee6
HDFS-12914. Block report leases cause missing blocks until next report. Contributed by Santosh Marella, He Xiaoqiao.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Co-authored-by: He Xiaoqiao <hexiaoqiao@apache.org>
2019-06-17 16:20:30 -07:00
Wei-Chiu Chuang
7314185c4a
Revert "HDFS-12914. Block report leases cause missing blocks until next report. Contributed by Santosh Marella, He Xiaoqiao."
...
This reverts commit ae4143a529
.
2019-06-17 16:18:48 -07:00
Wei-Chiu Chuang
a50c35bb81
Revert "HDFS-12914. Addendum patch. Block report leases cause missing blocks until next report. Contributed by Santosh Marella, He Xiaoqiao."
...
This reverts commit cdc5de6448
.
2019-06-17 16:18:37 -07:00
Wei-Chiu Chuang
f9a7b442fd
HDFS-14465. When the Block expected replications is larger than the number of DataNodes, entering maintenance will never exit. Contributed by Yicong Cai.
2019-06-17 15:18:17 -07:00
mpicker90
cc1630288e
HDFS-14556: Spelling Mistake "gloablly" ( #938 )
2019-06-16 18:32:42 -07:00
Adam Antal
8370a0ae16
HDFS-14203. Refactor OIV Delimited output entry building mechanism. Contributed by Adam Antal.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-15 06:47:10 -07:00
Giovanni Matteo Fumarola
b24efa11ea
HDFS-14549. EditLogTailer shouldn't output full stack trace when interrupted. Contributed by Inigo Goiri.
2019-06-14 13:37:23 -07:00
Santosh Marella
cdc5de6448
HDFS-12914. Addendum patch. Block report leases cause missing blocks until next report. Contributed by Santosh Marella, He Xiaoqiao.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Co-authored-by: He Xiaoqiao <hexiaoqiao@apache.org>
2019-06-14 13:01:39 -07:00
Santosh Marella
ae4143a529
HDFS-12914. Block report leases cause missing blocks until next report. Contributed by Santosh Marella, He Xiaoqiao.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Co-authored-by: He Xiaoqiao <hexiaoqiao@apache.org>
2019-06-14 10:42:54 -07:00
dineshchitlangia
9ebbda342f
HADOOP-16372. Fix typo in DFSUtil getHttpPolicy method
...
Closes #967
2019-06-14 16:09:37 +02:00
Inigo Goiri
4f455290b1
HDFS-14560. Allow block replication parameters to be refreshable. Contributed by Stephen O'Donnell.
2019-06-13 18:26:53 -07:00
Stephen O'Donnell
50de0874d0
HDFS-13231. Extend visualization for Decommissioning, Maintenance Mode under Datanode tab in the NameNode UI. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-12 10:30:23 -07:00
Eric Yang
4ea6c2f457
HADOOP-16354. Enable AuthFilter as default for WebHDFS.
...
Contributed by Prabhu Joseph
2019-06-11 18:41:08 -04:00
Inigo Goiri
5740eea081
HDFS-14513. FSImage which is saving should be clean while NameNode shutdown. Contributed by He Xiaoqiao.
2019-06-11 11:48:35 -07:00
Anu Engineer
101d5b5f86
HDFS-14234. Limit WebHDFS to specifc user, host, directory triples.
...
Contributed by Clay B.
2019-06-10 17:55:16 -07:00
He Xiaoqiao
bd46bdf9f9
HDFS-14553. Make queue size of BlockReportProcessingThread configurable. Contributed by He Xiaoqiao.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-10 17:21:52 -07:00
Wei-Chiu Chuang
0d160a0ba8
HDFS-10210. Remove the defunct startKdc profile from hdfs. Contributed by Wei-Chiu Chuang, Akira Ajisaka.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Co-authored-by: Akira Ajisaka <aajisaka@apache.org>
2019-06-10 14:13:53 -07:00
Hanisha Koneru
7217494f40
HDFS-10659. Namenode crashes after Journalnode re-installation in an HA cluster due to missing paxos directory. Contributed by star, Hanisha Koneru.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-10 13:45:37 -07:00
Shweta Yakkali
d6d95d2686
HDFS-14494. Move Server logging of StatedId inside receiveRequestState(). Contributed by Shweta Yakkali.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-10 10:23:18 -07:00
Inigo Goiri
e1dfc060f8
HDFS-14486. The exception classes in some throw statements do not accurately describe why they are thrown. Contributed by Ayush Saxena.
2019-06-06 11:59:53 -07:00
Inigo Goiri
944adc61b1
HDFS-14527. Stop all DataNodes may result in NN terminate. Contributed by He Xiaoqiao.
2019-06-06 10:20:28 -07:00
Eric Yang
294695dd57
HADOOP-16314. Make sure all web end points are covered by the same authentication filter.
...
Contributed by Prabhu Joseph
2019-06-05 18:55:13 -04:00
Sammi Chen
d1aad44490
HDFS-14356. Implement HDFS cache on SCM with native PMDK libs. Contributed by Feilong He.
2019-06-05 21:33:00 +08:00
huzheng
ea3b0a1844
HDFS-14535. The default 8KB buffer in requestFileDescriptors#BufferedOutputStream is causing lots of heap allocation in HBase when using short-circut read
2019-06-04 08:48:05 -07:00
Takanobu Asanuma
35f1014b3e
HDFS-13654. Use a random secret when a secret file doesn't exist in HttpFS. This should be default.
2019-05-31 10:29:24 +09:00
Kihwal Lee
9122b9b649
HDFS-14521. Suppress setReplication logging. Contributed by Kihwal Lee
2019-05-30 17:28:45 -05:00
He Xiaoqiao
33c62f8f4e
HDFS-14497. Write lock held by metasave impact following RPC processing. Contributed by He Xiaoqiao.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-05-30 13:30:35 -07:00
Shweta Yakkali
6f5a36c13c
HADOOP-13656. fs -expunge to take a filesystem. Contributed by Shweta.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-05-30 13:21:58 -07:00
Ayush Saxena
c1caab40f2
HDFS-14512. ONE_SSD policy will be violated while write data with DistributedFileSystem.create(....favoredNodes). Contributed by Ayush Saxena.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-05-29 20:56:38 -07:00
Akira Ajisaka
afd844059c
HADOOP-16331. Fix ASF License check in pom.xml
...
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2019-05-29 17:25:13 +09:00
Eric Yang
d78854b928
HDFS-14434. Ignore user.name query parameter in secure WebHDFS.
...
Contributed by KWON BYUNGCHANG
2019-05-28 17:31:35 -04:00
Akira Ajisaka
9f933e6446
HADOOP-16323. https everywhere in Maven settings.
2019-05-27 15:24:59 +09:00
Rakesh Radhakrishnan
37900c5639
HDFS-14402. Use FileChannel.transferTo() method for transferring block to SCM cache. Contributed by Feilong He.
2019-05-26 14:30:11 +05:30
Erik Krogen
55e0c134f0
HDFS-14500. NameNode StartupProgress should not allow new steps in an already-completed phase. Contributed by Erik Krogen.
2019-05-24 13:12:59 -07:00
Konstantin V Shvachko
c31b7b8d52
HDFS-14502. keepResults option in NNThroughputBenchmark should call saveNamespace(). Contributed by Konstantin V Shvachko.
2019-05-24 12:34:29 -07:00
Inigo Goiri
c9393dd17f
Revert "HDFS-14353. Erasure Coding: metrics xmitsInProgress become to negative. Contributed by maobaolong."
...
This reverts commit 1cb2eb0df3
.
2019-05-24 10:23:47 -07:00
Christopher Gregorian
f96a2df38d
HADOOP-16266. Add more fine-grained processing time metrics to the RPC layer. Contributed by Christopher Gregorian.
2019-05-23 10:28:37 -07:00
Siyao Meng
9dff6eff81
HDFS-14507. Document -blockingDecommission option for hdfs dfsadmin -listOpenFiles
...
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2019-05-22 13:22:26 +09:00
Inigo Goiri
1cb2eb0df3
HDFS-14353. Erasure Coding: metrics xmitsInProgress become to negative. Contributed by maobaolong.
2019-05-20 17:22:58 -07:00
Srinivasu Majeti
3e5e5b028a
HDFS-14323. Distcp fails in Hadoop 3.x when 2.x source webhdfs url has special characters in hdfs file path. Contributed by Srinivasu Majeti.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-05-17 19:20:28 +02:00
Virajith Jalaparti
77170e70d1
HDFS-14390. Provide kerberos support for AliasMap service used by Provided storage. Contributed by Ashvin Agrawal
2019-05-15 12:15:47 -07:00
Sahil Takiar
e2dfdaee7c
HDFS-14482: Crash when using libhdfs with bad classpath
2019-05-14 11:04:06 -07:00
Inigo Goiri
389e640f0c
HADOOP-16161. NetworkTopology#getWeightUsingNetworkLocation return unexpected result. Contributed by He Xiaoqiao.
2019-05-13 11:46:16 -07:00
Erik Krogen
5847e00143
HDFS-14245. [SBN read] Enable ObserverReadProxyProvider to work with non-ClientProtocol proxy types. Contributed by Erik Krogen.
2019-05-13 08:39:59 -07:00
Rakesh Radhakrishnan
9b0aace1e6
HDFS-14401. Refine the implementation for HDFS cache on SCM. Contributed by Feilong He.
2019-05-08 17:20:21 +05:30
Surendra Singh Lilhore
69b903bbd8
HDFS-14372. NPE while DN is shutting down. Contributed by lujie.
2019-05-05 16:33:52 +05:30
Surendra Singh Lilhore
e424392a62
HDFS-14438. Fix typo in OfflineEditsVisitorFactory. Contributed by bianqi.
2019-05-05 13:38:09 +05:30
Chen Liang
1d59cc490c
HADOOP-16292. Refactor checkTrustAndSend in SaslDataTransferClient to make it cleaner. Contributed by Sherwood Zheng.
2019-05-03 10:14:17 -07:00
Vinayakumar B
f1875b205e
HADOOP-16059. Use SASL Factories Cache to Improve Performance. Contributed by Ayush Saxena.
2019-05-03 11:22:14 +05:30
Shweta
d6b7609c96
HDFS-14453. Improve Bad Sequence Number Error Message. Contributed by Shweta.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-05-02 14:25:02 -07:00
Giovanni Matteo Fumarola
7a3188d054
HADOOP-16282. Avoid FileStream to improve performance. Contributed by Ayush Saxena.
2019-05-02 12:58:42 -07:00
Inigo Goiri
865c328930
HDFS-14460. DFSUtil#getNamenodeWebAddr should return HTTPS address based on policy configured. Contributed by CR Hota.
2019-05-02 10:09:21 -07:00
Siyao Meng
7cb46f035a
HDFS-14463. Add Log Level link under NameNode and DataNode Web UI Utilities dropdown. Contributed by Siyao Meng.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-05-01 16:48:44 -07:00
Sahil Takiar
4877f0aa51
HDFS-3246: pRead equivalent for direct read path ( #597 )
...
HDFS-3246: pRead equivalent for direct read path
Contributed by Sahil Takiar
2019-04-30 14:52:16 -07:00
Arpit Agarwal
4b4200f1f8
HDFS-13677. Dynamic refresh Disk configuration results in overwriting VolumeMap. Contributed by xuzq.
2019-04-29 14:49:35 -07:00
Sean Mackrory
a703dae25e
HADOOP-16222. Fix new deprecations after guava 27.0 update in trunk. Contributed by Gabor Bota.
2019-04-24 10:39:00 -06:00
Erik Krogen
174b7d3126
HDFS-14435. [SBN Read] Enable ObserverReadProxyProvider to gracefully handle StandbyException when fetching HAServiceState. Contributed by Erik Krogen.
2019-04-22 13:29:56 -07:00
Inigo Goiri
5321235fe8
HDFS-14445. TestTrySendErrorReportWhenNNThrowsIOException fails in trunk. Contributed by Ayush Saxena.
2019-04-22 13:23:42 -07:00
lys0716
685cb83e4c
HDFS-14433. Remove the extra empty space in the DataStreamer logging. Contributed by Yishuang Lu. ( #747 )
2019-04-17 10:38:48 -07:00
Puleya7
13907d8479
HDFS-14432. dfs.datanode.shared.file.descriptor.paths duplicated in hdfs-default.xml
...
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
2019-04-17 21:39:06 +09:00
Inigo Goiri
be6c8014e6
HDFS-14418. Remove redundant super user priveledge checks from namenode. Contributed by Ayush Saxena.
2019-04-16 10:34:31 -07:00
Sahil Takiar
2382f63fc0
HADOOP-14747. S3AInputStream to implement CanUnbuffer.
...
Author: Sahil Takiar <stakiar@cloudera.com>
2019-04-12 18:12:02 -07:00
Chen Liang
626fec652b
HDFS-13699. Add DFSClient sending handshake token to DataNode, and allow DataNode overwrite downstream QOP. Contributed by Chen Liang.
2019-04-12 17:37:51 -07:00
Adam Antal
586826fe99
HDFS-12245. Fix INodeId javadoc
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-04-11 10:01:47 +09:00
Shweta
e9b859f749
HDFS-14371. Improve Logging in FSNamesystem by adding parameterized logging. Contributed by Shweta.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-04-05 16:09:00 -07:00
Inigo Goiri
260d843b25
HDFS-14416. Fix TestHdfsConfigFields for field dfs.client.failover.resolver.useFQDN. Contributed by Fengnan Li.
2019-04-05 09:26:06 -07:00
Akira Ajisaka
67020f0950
HDFS-14407. Fix misuse of SLF4j logging API in DatasetVolumeChecker#checkAllVolumes. Contributed by Wanqiang Ji.
2019-04-05 12:29:11 +09:00
Stephen O'Donnell
d2637cb176
HDFS-14389. getAclStatus returns incorrect permissions and owner when an iNodeAttributeProvider is configured. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-04-04 04:07:18 -07:00
Inigo Goiri
7b5b783f66
HDFS-14327. Using FQDN instead of IP to access servers with DNS resolving. Contributed by Fengnan Li.
2019-04-03 16:11:13 -07:00
Gabor Bota
d7979079ea
HADOOP-16210. Update guava to 27.0-jre in hadoop-project trunk. Contributed by Gabor Bota.
2019-04-03 12:59:39 -06:00
Wei-Chiu Chuang
be488b6070
HDFS-10477. Stop decommission a rack of DataNodes caused NameNode fail over to standby. Contributed by yunjiong zhao and Wei-Chiu Chuang.
2019-04-03 11:00:59 -07:00
Lokesh Jain
cf268114c9
HDFS-13960. hdfs dfs -checksum command should optionally show block size in output. Contributed by Lokesh Jain.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-04-02 12:24:55 -07:00
Gabor Bota
53a86e2b8e
HADOOP-16220. Add findbugs ignores for unjustified issues during update to guava to 27.0-jre in hadoop-project
...
This closes #665
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-04-01 13:49:37 +09:00
Uma Maheswara Rao G
35ff31dd94
HDFS-14355 : Implement HDFS cache on SCM by using pure java mapped byte buffer. Contributed by Feilong He.
2019-03-30 23:33:25 -07:00
Rakesh Radhakrishnan
f3f51284d5
HDFS-14393. Refactor FsDatasetCache for SCM cache implementation. Contributed by Rakesh R
2019-03-29 00:18:15 +05:30
Giovanni Matteo Fumarola
49b02d4a9b
HDFS-14395. Remove WARN Logging From Interrupts. Contributed by David Mollitor.
2019-03-28 11:16:01 -07:00
Inigo Goiri
15d38b1bf9
HDFS-14295. Add Threadpool for DataTransfers. Contributed by David Mollitor.
2019-03-28 03:37:33 -07:00
Sahil Takiar
18c57cf046
HDFS-14304: High lock contention on hdfsHashMutex in libhdfs
...
This closes #595
Signed-off-by: Todd Lipcon <todd@apache.org>
2019-03-26 20:32:51 -07:00
Sahil Takiar
fe29b3901b
HDFS-14348: Fix JNI exception handling issues in libhdfs
...
This closes #600
Signed-off-by: Todd Lipcon <todd@apache.org>
2019-03-26 16:33:34 -07:00
Takanobu Asanuma
55fb3c32fb
HDFS-14037. Fix SSLFactory truststore reloader thread leak in URLConnectionFactory.
2019-03-27 03:27:02 +09:00
Stephen O'Donnell
3f6d6d2811
HDFS-14359. Inherited ACL permissions masked when parent directory does not exist (mkdir -p)
...
(Contributed by Stephen O'Donnell via Daniel Templeton)
Change-Id: Ia83f799a8f56aa8057a967b234f184683395fa41
2019-03-25 16:27:04 -07:00
Ajay Kumar
399563fec6
HDFS-14176. Replace incorrect use of system property user.name. Contributed by Dinesh Chitlangia.
2019-03-20 11:20:45 -07:00
Erik Krogen
55b3a718e9
HDFS-14211. [SBN Read]. Add a configurable flag to enable always-msync mode to ObserverReadProxyProvider. Contributed by Erik Krogen.
2019-03-19 08:14:49 -07:00
Inigo Goiri
ff06ef0631
HDFS-14366. Improve HDFS append performance. Contributed by Chao Sun.
2019-03-15 11:06:42 -07:00
Uma Maheswara Rao G
ba50a36a3e
HDFS-14354: Refactor MappableBlock to align with the implementation of SCM cache. Contributed by Feilong He.
2019-03-14 22:21:08 -07:00
Erik Krogen
66357574ae
HDFS-14346. Add better time precision to Configuration#getTimeDuration, allowing return unit and default unit to be specified independently. Contributed by Chao Sun.
2019-03-13 13:15:56 -07:00
Stephen O'Donnell
34b14061b3
HDFS-14333. Datanode fails to start if any disk has errors during Namenode registration. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-03-12 10:17:25 -07:00
Erik Krogen
1bc282e0b3
HDFS-14317. Ensure checkpoints are created when in-progress edit log tailing is enabled with a period shorter than the log roll period. Contributed by Ekanth Sethuramalingam.
2019-03-07 08:41:23 -08:00
Sahil Takiar
618e009ac0
HDFS-14111. hdfsOpenFile on HDFS causes unnecessary IO from file offset 0. Contributed by Sahil Takiar.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-03-06 15:04:06 -08:00
Wei-Chiu Chuang
6192c1fe3b
Revert "HDFS-14111. hdfsOpenFile on HDFS causes unnecessary IO from file offset 0. Contributed by Sahil Takiar."
...
This reverts commit f5a4b43a49
.
2019-03-06 15:02:18 -08:00
Sahil Takiar
f5a4b43a49
HDFS-14111. hdfsOpenFile on HDFS causes unnecessary IO from file offset 0. Contributed by Sahil Takiar.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-03-06 14:58:45 -08:00
Inigo Goiri
945b504c25
HDFS-14326. Add CorruptFilesCount to JMX. Contributed by Danny Becker.
2019-03-05 17:39:52 -08:00
Vinayakumar B
f940ab242d
HDFS-7663. Erasure Coding: Append on striped file. Contributed by Ayush Saxena.
2019-03-05 19:26:42 +05:30
Inigo Goiri
4b7313e640
HDFS-14336. Fix checkstyle for NameNodeMXBean. Contributed by Danny Becker.
2019-03-04 23:03:32 -08:00
Sahil Takiar
cb0fa0ce3f
HDFS-14321. Fix -Xcheck:jni issues in libhdfs, run ctest with -Xcheck:jni enabled. Contributed by Sahil Takiar.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-03-04 12:37:57 -08:00
Wei-Chiu Chuang
387dbe587a
HDFS-14314. fullBlockReportLeaseId should be reset after registering to NN. Contributed by star.
2019-03-04 10:43:44 -08:00
Yiqun Lin
18ea0c1493
HDFS-14182. Datanode usage histogram is clicked to show ip list. Contributed by fengchuang.
2019-03-04 17:34:24 +08:00
Wei-Chiu Chuang
6c4d566955
Revert "HDFS-14261. Kerberize JournalNodeSyncer unit test. Contributed by Siyao Meng."
...
This reverts commit 5c10630ad8
.
2019-03-04 16:59:20 +09:00
Erik Krogen
fc17ba172b
HDFS-14272. [SBN read] Make ObserverReadProxyProvider initialize its state ID against the active NN on startup. Contributed by Erik Krogen.
2019-03-04 16:52:04 +09:00
Inigo Goiri
80b77deb42
HDFS-14247. Repeat adding node description into network topology. Contributed by HuangTao.
2019-03-01 09:18:51 -08:00
Brahma Reddy Battula
7a0db2f92b
HDFS-13997. Secondary NN Web UI displays nothing, and the console log shows moment is not defined. Contributed by Ayush Saxena
2019-03-01 12:24:52 +05:30
Inigo Goiri
05e4ddeee6
HDFS-14324. Fix TestDataNodeVolumeFailure. Contributed by Ayush Saxena.
2019-02-28 10:03:05 -08:00
Erik Krogen
0feba4396f
HDFS-14305. Fix serial number calculation in BlockTokenSecretManager to avoid token key ID overlap between NameNodes. Contributed by He Xiaoqiao.
2019-02-28 08:48:10 -08:00
Konstantin V Shvachko
a6ab37192a
HDFS-14130. [SBN read] Make ZKFC ObserverNode aware. Contributed by xiangheng and Konstantin Shvachko.
2019-02-25 14:35:02 -08:00
Vinayakumar B
f4ae00c530
HDFS-7133. Support clearing namespace quota on '/'. Contributed by Ayush Saxena."
2019-02-26 00:36:14 +05:30
Yongjun Zhang
f7a27cdee4
HDFS-14118. Support using DNS to resolve nameservices to IP addresses. Contributed by Fengnan Li.
2019-02-23 09:35:36 -08:00
Sahil Takiar
f19c844e75
HDFS-14285. libhdfs hdfsRead copies entire array even if its only partially filled. Contributed by Sahil Takiar.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-02-22 17:54:32 -08:00
Erik Krogen
bad3ffd290
HDFS-14279. [SBN read] Fix race condition in ObserverReadProxyProvider. Contributed by Erik Krogen.
2019-02-22 13:58:49 -08:00
Surendra Singh Lilhore
7d3b567194
HDFS-14298. Improve log messages of ECTopologyVerifier. Contributed by Kitti Nanasi.
2019-02-23 01:08:15 +05:30
Surendra Singh Lilhore
92b53c40f0
HDFS-14216. NullPointerException happens in NamenodeWebHdfs. Contributed by lujie.
2019-02-21 20:36:34 +05:30
Shweta Yakkali
1bea785020
HDFS-14081. hdfs dfsadmin -metasave metasave_test results NPE. Contributed by Shweta Yakkali.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-02-20 14:28:37 -08:00
Inigo Goiri
f5b4e0f971
HDFS-14302. Refactor NameNodeWebHdfsMethods#generateDelegationToken() to allow better extensibility. Contributed by CR Hota.
2019-02-20 13:55:13 -08:00
Sahil Takiar
a30059bb61
HDFS-14267. Add test_libhdfs_ops to libhdfs tests, mark libhdfs_read/write.c as examples. Contributed by Sahil Takiar.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-02-20 11:38:37 -08:00
Surendra Singh Lilhore
41e18feda3
HDFS-14235. Handle ArrayIndexOutOfBoundsException in DataNodeDiskMetrics#slowDiskDetectionDaemon. Contributed by Ranith Sardar.
2019-02-20 16:56:10 +05:30
Bharat Viswanadham
51950f149e
Logging stale datanode information. Contributed by Karthik Palanisamy.
2019-02-19 17:01:07 -08:00
Kitti Nanasi
14282e311b
HDFS-14188. Make hdfs ec -verifyClusterSetup command accept an erasure coding policy as a parameter. Contributed by Kitti Nanasi.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-02-19 12:04:56 -08:00
Inigo Goiri
7587f97127
HDFS-14296. Prefer ArrayList over LinkedList in VolumeScanner. Contributed by BELUGA BEHR.
2019-02-18 17:55:43 -08:00
Inigo Goiri
1de25d134f
HDFS-9596. Remove Shuffle Method From DFSUtil. Contributed by BELUGA BEHR.
2019-02-18 15:30:38 -08:00
Inigo Goiri
235e3da90a
HDFS-14287. DataXceiverServer May Double-Close PeerServer. Contributed by BELUGA BEHR.
2019-02-18 11:00:04 -08:00
Akira Ajisaka
0f2b65c3da
HADOOP-16116. Fix Spelling Mistakes - DECOMISSIONED. Contributed by BELUGA BEHR.
2019-02-18 15:21:46 +09:00
Inigo Goiri
dde0ab55aa
HDFS-14258. Introduce Java Concurrent Package To DataXceiverServer Class. Contributed by BELUGA BEHR.
2019-02-15 16:32:27 -08:00
Anu Engineer
75e15cc0c4
HDDS-1103.Fix rat/findbug/checkstyle errors in ozone/hdds projects.
...
Contributed by Elek, Marton.
2019-02-14 23:33:25 -08:00
Erik Krogen
64f28f9efa
HDFS-14162. [SBN read] Allow Balancer to work with Observer node. Add a new ProxyCombiner allowing for multiple related protocols to be combined. Allow AlignmentContext to be passed in NameNodeProxyFactory. Contributed by Erik Krogen.
2019-02-14 11:22:04 -08:00
Surendra Singh Lilhore
0d7a5ac5f5
HDFS-13209. DistributedFileSystem.create should allow an option to provide StoragePolicy. Contributed by Ayush Saxena.
2019-02-14 22:13:14 +05:30
Shweta Yakkali
080a421911
HDFS-14262. [SBN read] Make Log.WARN message in GlobalStateIdContext more informative. Contributed by Shweta Yakkali.
2019-02-14 08:05:06 -08:00
Chen Liang
024c87291c
HDFS-13617. Allow wrapping NN QOP into token in encrypted message. Contributed by Chen Liang
2019-02-13 12:40:31 -08:00
Surendra Singh Lilhore
29b411d5f0
HDFS-14263. Remove unnecessary block file exists check from FsDatasetImpl#getBlockInputStream(). Contributed by Surendra Singh Lilhore
2019-02-13 23:41:27 +05:30
Surendra Singh Lilhore
35d4f32b32
HDFS-14274. EC: NPE While Listing EC Policy For A Directory Following Replication Policy. Contributed by Ayush Saxena.
2019-02-13 23:06:46 +05:30
Vinayakumar B
7806403842
HDFS-14266. EC : Fsck -blockId shows null for EC Blocks if One Block Is Not Available. Contributed by Ayush Saxena.
2019-02-12 21:57:57 +05:30
Siyao Meng
5c10630ad8
HDFS-14261. Kerberize JournalNodeSyncer unit test. Contributed by Siyao Meng.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-02-11 11:02:32 -08:00
Inigo Goiri
0ceb1b70f3
HDFS-14260. Replace synchronized method in BlockReceiver with atomic value. Contributed by BELUGA BEHR.
2019-02-11 10:09:44 -08:00
Yiqun Lin
177131793a
HDFS-14172. Avoid NPE when SectionName#fromString returns null. Contributed by Xiang Li.
2019-02-08 20:51:30 +08:00
Surendra Singh Lilhore
4be87353e3
HDFS-14140. JournalNodeSyncer authentication is failing in secure cluster. Contributed by Surendra Singh Lilhore.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-02-07 16:51:59 -08:00
Steve Loughran
668817a6ce
Revert "HADOOP-15954. ABFS: Enable owner and group conversion for MSI and login user using OAuth."
...
(accidentally mixed in two patches)
This reverts commit fa8cd1bf28
.
2019-02-07 21:57:22 +00:00
Siyao Meng
6aa63452b3
HDFS-14242. OIV WebImageViewer: NPE when param op is not specified. Contributed by Siyao Meng.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-02-06 12:19:21 -08:00
Erik Krogen
d3de8e162b
HDFS-14250. [SBN read]. msync should always direct to active NameNode to get latest stateID. Contributed by Chao Sun.
2019-02-06 09:54:47 -08:00
Kitti Nanasi
49ddd8a6ed
HDFS-14231. DataXceiver#run() should not log exceptions caused by InvalidToken exception as an error. Contributed by Kitti Nanasi.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-02-05 15:14:19 -08:00
Wangda Tan
308f3168fa
Make upstream aware of 3.1.2 release
...
Change-Id: I397bc6ef75498726df4763bd07a8bf8fe1c38365
2019-02-05 14:03:18 -08:00
Da Zhou
fa8cd1bf28
HADOOP-15954. ABFS: Enable owner and group conversion for MSI and login user using OAuth.
...
Contributed by Da Zhou and Junhua Gu.
2019-02-05 19:23:15 +00:00