8aaf5e1a14
HDFS-14759. HDFS cat logs an info message.
...
Contributed by Eric Badger.
2019-08-20 20:24:19 -07:00
aa6995fde2
HDFS-13201. Fix prompt message in testPolicyAndStateCantBeNull. Contributed by chencan.
2019-08-20 10:32:55 -07:00
9a1d8cfaf5
HDFS-14678. Allow triggerBlockReport to a specific namenode. ( #1252 ). Contributed by Leon Gao.
2019-08-16 08:00:51 -07:00
3c0382f1b9
HDFS-14595. HDFS-11848 breaks API compatibility. Contributed by Siyao Meng.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
2019-08-14 07:30:35 -07:00
da0006fe04
HDFS-14423. Percent (%) and plus (+) characters no longer work in WebHDFS.
...
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
2019-08-14 08:39:40 +09:00
28a848412c
HDFS-14701. Change Log Level to warn in SlotReleaser. Contributed by Lisheng Sun.
2019-08-08 13:46:31 -07:00
e7a0b8aa83
HDFS-14462 Ensure WebHDFS client throws the correct exception during writes. Contributed by Simbarashe Dzinamarira.
2019-08-02 11:48:31 -07:00
17e8cf501b
HDFS-14686. HttpFS: HttpFSFileSystem#getErasureCodingPolicy always returns null ( #1192 ) Contributed by Siyao Meng.
2019-08-01 17:15:22 -07:00
3ae775d740
HDFS-14034. Support getQuotaUsage API in WebHDFS. Contributed by Chao Sun.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-07-30 16:01:17 -07:00
ecc8acfd24
HDFS-14673. The console log is noisy when using DNSDomainNameResolver to resolve NameNode.
2019-07-26 11:08:46 -07:00
8fb5ca3f40
HDFS-14611. Move handshake secret field from Token to BlockAccessToken. Contributed by Chen Liang.
2019-07-11 13:23:25 -07:00
729cb3aefe
HDFS-12748. NameNode memory leak when accessing webhdfs GETHOMEDIRECTORY. Contributed by Weiwei Yang.
2019-07-04 09:37:24 +08:00
041e0c0564
HDFS-14036. RBF: Add hdfs-rbf-default.xml to HdfsConfiguration by default. Contributed by Takanobu Asanuma.
2019-06-27 09:57:52 -07:00
b76b843c8b
HDFS-13371. NPE for FsServerDefaults.getKeyProviderUri() for clientProtocol communication between 2.7 and 3.X. Contributed by Sherwood Zheng.
2019-06-24 17:52:33 -07:00
38a560c6f1
Merge pull request #977 from leosunli/trunk
...
HDFS-14541. When evictableMmapped or evictable size is zero, do not throw NoSuchElementException.
2019-06-24 17:44:54 -07:00
95c94dcca7
HADOOP-16350. Ability to tell HDFS client not to request KMS Information from NameNode. Ccontributed by Greg Senia, Ajay Kumar.
2019-06-24 11:59:07 -07:00
50aee18a84
HDFS-14268. RBF: Fix the location of the DNs in getDatanodeReport(). Contributed by Inigo Goiri.
2019-06-24 22:03:03 +05:30
f40e10b349
HDFS-14129. RBF: Create new policy provider for router. Contributed by Ranith Sardar.
2019-06-24 22:03:03 +05:30
daa1e14745
when evictableMmapped or evictable size is zero, do not throw NoSuchElementException
...
Signed-off-by: sunlisheng <sunlisheng@xiaomi.com>
2019-06-24 20:17:22 +08:00
3c1a1ceea9
HDFS-14487. Missing Space in Client Error Message (Contributed by Shweta Yakkali via Daniel Templeton)
...
Change-Id: I0f8ce74a35ab24fe94fd0e57d8247bb3fa575e6f
2019-06-18 10:21:22 -07:00
62ad9885ea
HDFS-13730. BlockReaderRemote.sendReadResult throws NPE. Contributed by Yuanbo Liu.
2019-06-17 20:18:53 -07:00
cc1630288e
HDFS-14556: Spelling Mistake "gloablly" ( #938 )
2019-06-16 18:32:42 -07:00
ea3b0a1844
HDFS-14535. The default 8KB buffer in requestFileDescriptors#BufferedOutputStream is causing lots of heap allocation in HBase when using short-circut read
2019-06-04 08:48:05 -07:00
afd844059c
HADOOP-16331. Fix ASF License check in pom.xml
...
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2019-05-29 17:25:13 +09:00
d78854b928
HDFS-14434. Ignore user.name query parameter in secure WebHDFS.
...
Contributed by KWON BYUNGCHANG
2019-05-28 17:31:35 -04:00
9f933e6446
HADOOP-16323. https everywhere in Maven settings.
2019-05-27 15:24:59 +09:00
3e5e5b028a
HDFS-14323. Distcp fails in Hadoop 3.x when 2.x source webhdfs url has special characters in hdfs file path. Contributed by Srinivasu Majeti.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-05-17 19:20:28 +02:00
5847e00143
HDFS-14245. [SBN read] Enable ObserverReadProxyProvider to work with non-ClientProtocol proxy types. Contributed by Erik Krogen.
2019-05-13 08:39:59 -07:00
1d59cc490c
HADOOP-16292. Refactor checkTrustAndSend in SaslDataTransferClient to make it cleaner. Contributed by Sherwood Zheng.
2019-05-03 10:14:17 -07:00
f1875b205e
HADOOP-16059. Use SASL Factories Cache to Improve Performance. Contributed by Ayush Saxena.
2019-05-03 11:22:14 +05:30
d6b7609c96
HDFS-14453. Improve Bad Sequence Number Error Message. Contributed by Shweta.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-05-02 14:25:02 -07:00
7a3188d054
HADOOP-16282. Avoid FileStream to improve performance. Contributed by Ayush Saxena.
2019-05-02 12:58:42 -07:00
4877f0aa51
HDFS-3246: pRead equivalent for direct read path ( #597 )
...
HDFS-3246: pRead equivalent for direct read path
Contributed by Sahil Takiar
2019-04-30 14:52:16 -07:00
174b7d3126
HDFS-14435. [SBN Read] Enable ObserverReadProxyProvider to gracefully handle StandbyException when fetching HAServiceState. Contributed by Erik Krogen.
2019-04-22 13:29:56 -07:00
685cb83e4c
HDFS-14433. Remove the extra empty space in the DataStreamer logging. Contributed by Yishuang Lu. ( #747 )
2019-04-17 10:38:48 -07:00
626fec652b
HDFS-13699. Add DFSClient sending handshake token to DataNode, and allow DataNode overwrite downstream QOP. Contributed by Chen Liang.
2019-04-12 17:37:51 -07:00
260d843b25
HDFS-14416. Fix TestHdfsConfigFields for field dfs.client.failover.resolver.useFQDN. Contributed by Fengnan Li.
2019-04-05 09:26:06 -07:00
7b5b783f66
HDFS-14327. Using FQDN instead of IP to access servers with DNS resolving. Contributed by Fengnan Li.
2019-04-03 16:11:13 -07:00
49b02d4a9b
HDFS-14395. Remove WARN Logging From Interrupts. Contributed by David Mollitor.
2019-03-28 11:16:01 -07:00
55fb3c32fb
HDFS-14037. Fix SSLFactory truststore reloader thread leak in URLConnectionFactory.
2019-03-27 03:27:02 +09:00
55b3a718e9
HDFS-14211. [SBN Read]. Add a configurable flag to enable always-msync mode to ObserverReadProxyProvider. Contributed by Erik Krogen.
2019-03-19 08:14:49 -07:00
66357574ae
HDFS-14346. Add better time precision to Configuration#getTimeDuration, allowing return unit and default unit to be specified independently. Contributed by Chao Sun.
2019-03-13 13:15:56 -07:00
618e009ac0
HDFS-14111. hdfsOpenFile on HDFS causes unnecessary IO from file offset 0. Contributed by Sahil Takiar.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-03-06 15:04:06 -08:00
6192c1fe3b
Revert "HDFS-14111. hdfsOpenFile on HDFS causes unnecessary IO from file offset 0. Contributed by Sahil Takiar."
...
This reverts commit f5a4b43a49
2019-03-06 15:02:18 -08:00
f5a4b43a49
HDFS-14111. hdfsOpenFile on HDFS causes unnecessary IO from file offset 0. Contributed by Sahil Takiar.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-03-06 14:58:45 -08:00
f940ab242d
HDFS-7663. Erasure Coding: Append on striped file. Contributed by Ayush Saxena.
2019-03-05 19:26:42 +05:30
fc17ba172b
HDFS-14272. [SBN read] Make ObserverReadProxyProvider initialize its state ID against the active NN on startup. Contributed by Erik Krogen.
2019-03-04 16:52:04 +09:00
f7a27cdee4
HDFS-14118. Support using DNS to resolve nameservices to IP addresses. Contributed by Fengnan Li.
2019-02-23 09:35:36 -08:00
bad3ffd290
HDFS-14279. [SBN read] Fix race condition in ObserverReadProxyProvider. Contributed by Erik Krogen.
2019-02-22 13:58:49 -08:00
64f28f9efa
HDFS-14162. [SBN read] Allow Balancer to work with Observer node. Add a new ProxyCombiner allowing for multiple related protocols to be combined. Allow AlignmentContext to be passed in NameNodeProxyFactory. Contributed by Erik Krogen.
2019-02-14 11:22:04 -08:00
0d7a5ac5f5
HDFS-13209. DistributedFileSystem.create should allow an option to provide StoragePolicy. Contributed by Ayush Saxena.
2019-02-14 22:13:14 +05:30
024c87291c
HDFS-13617. Allow wrapping NN QOP into token in encrypted message. Contributed by Chen Liang
2019-02-13 12:40:31 -08:00
d3de8e162b
HDFS-14250. [SBN read]. msync should always direct to active NameNode to get latest stateID. Contributed by Chao Sun.
2019-02-06 09:54:47 -08:00
f365957c63
HADOOP-15229. Add FileSystem builder-based openFile() API to match createFile();
...
S3A to implement S3 Select through this API.
The new openFile() API is asynchronous, and implemented across FileSystem and FileContext.
The MapReduce V2 inputs are moved to this API, and you can actually set must/may
options to pass in.
This is more useful for setting things like s3a seek policy than for S3 select,
as the existing input format/record readers can't handle S3 select output where
the stream is shorter than the file length, and splitting plain text is suboptimal.
Future work is needed there.
In the meantime, any/all filesystem connectors are now free to add their own filesystem-specific
configuration parameters which can be set in jobs and used to set filesystem input stream
options (seek policy, retry, encryption secrets, etc).
Contributed by Steve Loughran
2019-02-05 11:51:02 +00:00
0ab7fc9200
HDFS-14187. Make warning message more clear when there are not enough data nodes for EC write. Contributed by Kitti Nanasi.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-01-31 16:07:24 -08:00
1129288cf5
HADOOP-14178. Move Mockito up to version 2.23.4. Contributed by Akira Ajisaka and Masatake Iwasaki.
2019-01-29 18:29:56 -08:00
45caeee6cf
HDFS-14228. Incorrect getSnapshottableDirListing() javadoc. Contributed by Dinesh Chitlangia.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-01-25 07:37:01 -08:00
221e308cb5
HDFS-14153. [SPS] : Add Support for Storage Policy Satisfier in WEBHDFS. Contributed by Ayush Saxena.
2019-01-23 17:00:37 +05:30
f048512bb8
HDFS-14192. Track missing DFS operations in Statistics and StorageStatistics. Contributed by Ayush Saxena.
2019-01-16 10:14:22 -08:00
fb8932a727
HADOOP-16029. Consecutive StringBuilder.append can be reused. Contributed by Ayush Saxena.
2019-01-11 10:54:49 -08:00
4ab5260b7e
HDFS-14132. Add BlockLocation.isStriped() to determine if block is replicated or Striped
...
(Contributed by Shweta Yakkali via Daniel Templeton)
Change-Id: I0ed8996a0bae2ad2c7d3513143195533f7191af8
2019-01-08 17:04:57 -08:00
bd2a59e508
HDFS-14170. [SBN read] Fix checkstyle warnings related to SBN reads. Contributed by Konstantin V Shvachko.
2018-12-24 09:39:20 -08:00
c9d7737431
HDFS-14160. [SBN read] ObserverReadInvocationHandler should implement RpcInvocationHandler. Contributed by Konstantin V Shvachko.
2018-12-24 09:34:01 -08:00
fa8550337d
HDFS-14149. [SBN read] Fix annotations on new interfaces/classes for SBN reads. Contributed by Chao Sun.
2018-12-24 09:34:01 -08:00
b73fb70f97
HDFS-13873. [SBN read] ObserverNode should reject read requests when it is too far behind. Contributed by Konstantin Shvachko.
2018-12-24 09:34:01 -08:00
b8ad6c85a5
HDFS-14142. Move ipfailover config key out of HdfsClientConfigKeys. Contributed by Chen Liang.
2018-12-24 09:34:01 -08:00
71cf66e29b
HDFS-14120. [SBN read] ORFPP should also clone DT for the virtual IP. Contributed by Chen Liang.
2018-12-24 09:34:01 -08:00
3bb92a1d9a
HDFS-14094. [SBN read] Fix the order of logging arguments in ObserverReadProxyProvider. Contributed by Ayush Saxena.
2018-12-24 09:34:01 -08:00
a3aab48df0
HDFS-14017. [SBN read] ObserverReadProxyProviderWithIPFailover should work with HA configuration. Contributed by Chen Liang.
2018-12-24 09:34:01 -08:00
652b257478
HDFS-14035. NN status discovery does not leverage delegation token. Contributed by Chen Liang.
2018-12-24 09:34:01 -08:00
1a9ba9616a
HDFS-14016. [SBN read] ObserverReadProxyProvider should enable observer read by default. Contributed by Chen Liang.
2018-12-24 09:34:01 -08:00
8c49135078
HDFS-13924. [SBN read] Handle BlockMissingException when reading from observer. Contributed by Chao Sun.
2018-12-24 09:34:01 -08:00
25b63e8da8
HDFS-13749. [SBN read] Use getServiceStatus to discover observer namenodes. Contributed by Chao Sun.
2018-12-24 09:34:00 -08:00
d2e85b0b6d
HDFS-13880. Add mechanism to allow certain RPC calls to bypass sync. Contributed by Chen Liang.
2018-12-24 09:34:00 -08:00
aa42fb0db7
HDFS-13779. [SBN read] Implement proper failover and observer failure handling logic for for ObserverReadProxyProvider. Contributed by Erik Krogen.
2018-12-24 09:34:00 -08:00
f9fc01cd7f
HDFS-13782. ObserverReadProxyProvider should work with IPFailoverProxyProvider. Contributed by Konstantin Shvachko.
2018-12-24 09:34:00 -08:00
25d8e39b5c
HDFS-13851. Remove AlignmentContext from AbstractNNFailoverProxyProvider. Contributed by Konstantin Shvachko.
2018-12-24 09:34:00 -08:00
66ec3a67e8
HDFS-13767. Add msync server implementation. Contributed by Chen Liang.
2018-12-24 09:34:00 -08:00
eae0a5d54a
HDFS-13688. [SBN read] Introduce msync API call. Contributed by Chen Liang.
2018-12-24 09:34:00 -08:00
64b7cf59bd
HDFS-12976. [SBN read] Introduce ObserverReadProxyProvider. Contributed by Chao Sun.
2018-12-24 09:33:59 -08:00
0105d7f597
HDFS-13706. [SBN read] Rename client context to ClientGSIContext. Contributed by Konstantin Shvachko.
2018-12-24 09:33:59 -08:00
e880660a20
HDFS-13399. [SBN read] Make Client field AlignmentContext non-static. Contributed by Plamen Jeliazkov.
2018-12-24 09:33:59 -08:00
a109f2b32f
HDFS-13578. [SBN read] Add ReadOnly annotation to methods in ClientProtocol. Contributed by Chao Sun.
2018-12-24 09:33:59 -08:00
9bf0696c73
HDFS-13331. [SBN read] Add lastSeenStateId to RpcRequestHeader. Contributed by Plamen Jeliazkov.
2018-12-24 09:33:59 -08:00
c9d73437e8
HDFS-12977. [SBN read] Add stateId to RPC headers. Contributed by Plamen Jeliazkov.
2018-12-24 09:33:59 -08:00
788e7473a4
HDFS-14096. [SPS] : Add Support for Storage Policy Satisfier in ViewFs. Contributed by Ayush Saxena.
2018-12-17 11:24:57 +05:30
f858f18554
HDFS-14088. RequestHedgingProxyProvider can throw NullPointerException when failover due to no lock on currentUsedProxy. Contributed by Yuxuan Wang.
2018-12-14 11:04:36 -08:00
aa12859890
HDFS-14121. Log message about the old hosts file format is misleading
...
(Contributed by Zsolt Venczel via Daniel Templeton)
Change-Id: I7ff548f6c82e0aeb08a7a50ca7c2c827db8726bb
2018-12-14 13:54:57 +01:00
4aa0609fb0
HDFS-13985. Clearer error message for ReplicaNotFoundException
...
(Contributed by Adam Antal via Daniel Templeton)
Change-Id: I68ae7a5bedecbc204c161fc3b3ce8878fe5d493d
2018-12-13 16:21:45 +01:00
39dc7345b8
HDFS-14124. EC : Support EC Commands (set/get/unset EcPolicy) via WebHdfs. Contributed by Ayush Saxena.
2018-12-11 17:59:04 +05:30
f534736867
HDFS-14095. EC: Track Erasure Coding commands in DFS statistics. Contributed by Ayush Saxena.
2018-11-30 00:18:27 +05:30
892b33e054
HDFS-14064. WEBHDFS: Support Enable/Disable EC Policy. Contributed by Ayush Saxena.
2018-11-22 00:13:56 +05:30
0b6625a973
MAPREDUCE-7148. Fast fail jobs when exceeds dfs quota limitation. Contributed by Wang Yan
2018-11-07 08:20:49 -06:00
887244de4a
HDFS-14048. DFSOutputStream close() throws exception on subsequent call after DataNode restart. Contributed by Erik Krogen.
2018-11-06 11:18:15 -08:00
d98b881e9a
HDFS-14008. NN should log snapshotdiff report. Contributed by Pranay Singh.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2018-11-01 17:26:00 -07:00
db7e636824
HDFS-14027. DFSStripedOutputStream should implement both hsync methods.
2018-10-29 19:06:15 -07:00
635786a511
HDFS-13566. Add configurable additional RPC listener to NameNode. Contributed by Chen Liang.
2018-10-23 14:53:45 -07:00
88cce32551
HDFS-9872. HDFS bytes-default configurations should accept multiple size units. Contributed by Yiqun Lin.
2018-10-19 10:38:04 -07:00
b6fc72a025
HADOOP-14445. Addendum: Use DelegationTokenIssuer to create KMS delegation tokens that can authenticate to all KMS instances.
2018-10-15 10:51:55 -07:00
b60ca37914
Fix potential FSImage corruption. Contributed by Daryn Sharp.
2018-10-15 15:48:26 +05:30
5ec86b445c
HADOOP-14445. Use DelegationTokenIssuer to create KMS delegation tokens that can authenticate to all KMS instances.
...
Contributed by Daryn Sharp, Xiao Chen, Rushabh S Shah.
2018-10-12 09:35:52 -07:00
bf3d591f0c
HDFS-13956. iNotify should include information to identify a file as either replicated or erasure coded. Contributed by Hrishikesh Gadre.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2018-10-10 10:23:07 -07:00
3ead525c71
HDFS-12459. Fix revert: Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2018-10-10 10:11:10 -07:00
10185d9a77
HDFS-13882. Set a maximum delay for retrying locateFollowingBlock. Contributed by Kitti Nanasi.
...
Signed-off-by: Xiao Chen <xiao@apache.org>
2018-10-10 08:55:32 -07:00
08bb6c49a5
HDFS-13926. ThreadLocal aggregations for FileSystem.Statistics are incorrect with striped reads.
...
Contributed by Xiao Chen, Hrishikesh Gadre.
Signed-off-by: Xiao Chen <xiao@apache.org>
2018-10-08 20:31:57 -07:00
396ce0d9f4
HDFS-13877. HttpFS: Implement GETSNAPSHOTDIFF. Contributed by Siyao Meng.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2018-10-04 15:17:47 -07:00
58fa96b697
Changed version in trunk to 3.3.0-SNAPSHOT.
2018-10-02 22:41:41 +05:30
f6c5ef9903
HDFS-13943. [JDK10] Fix javadoc errors in hadoop-hdfs-client module. Contributed by Akira Ajisaka.
2018-10-02 09:49:48 +09:00
28ceb34a72
HDFS-13868. WebHDFS: GETSNAPSHOTDIFF API NPE when param "snapshotname" is given but "oldsnapshotname" is not. Contributed by Pranay Singh.
2018-09-19 03:12:20 -07:00
a1de8cbac5
HDFS-13838. WebHdfsFileSystem.getFileStatus() won't return correct "snapshot enabled" status. Contributed by Siyao Meng.
2018-09-14 05:22:56 +08:00
30eceec342
HDFS-13806. EC: No error message for unsetting EC policy of the directory inherits the erasure coding policy from an ancestor directory. Contributed by Ayush Saxena.
2018-09-10 09:10:51 +05:30
eca1a4bfe9
HDFS-13695. Move logging to slf4j in HDFS package. Contributed by Ian Pickering.
2018-09-06 14:48:00 -07:00
6e5ffb74dd
HDFS-13885. Add debug logs in dfsclient around decrypting EDEK. Contributed by Kitti Nanasi.
...
Signed-off-by: Xiao Chen <xiao@apache.org>
2018-09-03 22:37:37 -07:00
8aa6c4f079
Revert "HDFS-13838. WebHdfsFileSystem.getFileStatus() won't return correct "snapshot enabled" status. Contributed by Siyao Meng."
...
This reverts commit 26c2a97c56
2018-08-30 11:44:20 -07:00
26c2a97c56
HDFS-13838. WebHdfsFileSystem.getFileStatus() won't return correct "snapshot enabled" status. Contributed by Siyao Meng.
2018-08-27 16:02:35 -07:00
a4121c71c2
HDFS-13848. Refactor NameNode failover proxy providers. Contributed by Konstantin Shvachko.
2018-08-24 18:27:30 -07:00
c67b0650ea
HDFS-13747. Statistic for list_located_status is incremented incorrectly by listStatusIterator. Contributed by Antal Mihalyi.
2018-08-16 23:14:21 -07:00
39ed3a66db
HDFS-13076: [SPS]: Cleanup work for HDFS-10285 merge. Contributed by Rakesh R.
2018-08-12 03:06:06 -07:00
4402f3f855
HDFS-13097: [SPS]: Fix the branch review comments(Part1). Contributed by Surendra Singh.
2018-08-12 03:06:04 -07:00
3b83110d5e
HDFS-13057: [SPS]: Revisit configurations to make SPS service modes internal/external/none. Contributed by Rakesh R.
2018-08-12 03:06:04 -07:00
c561cb316e
HDFS-12106: [SPS]: Improve storage policy satisfier configurations. Contributed by Surendra Singh Lilhore.
2018-08-12 03:06:02 -07:00
68017e3349
HDFS-12310: [SPS]: Provide an option to track the status of in progress requests. Contributed by Surendra Singh Lilhore.
2018-08-12 03:06:02 -07:00
6215e35bb6
HDFS-11150: [SPS]: Provide persistence when satisfying storage policy. Contributed by Yuanbo Liu
2018-08-12 03:05:57 -07:00
cd5262aba0
HDFS-10885. [SPS]: Mover tool should not be allowed to run when Storage Policy Satisfier is on. Contributed by Wei Zhou
2018-08-12 03:05:55 -07:00
b67ae6d9d7
HDFS-10802. [SPS]: Add satisfyStoragePolicy API in HdfsAdmin. Contributed by Yuanbo Liu
2018-08-12 03:05:54 -07:00
9499df7b81
HDFS-13658. Expose HighestPriorityLowRedundancy blocks statistics. Contributed by Kitti Nanasi.
2018-08-08 10:40:20 -07:00
849c45db18
HDFS-13448. HDFS Block Placement - Ignore Locality for First Block Replica
...
(Contributed by BELUGA BEHR via Daniel Templeton)
Change-Id: I965d1cfa642ad24296038b83e3d5c9983545267d
2018-07-24 16:05:27 -07:00
0247cb6318
HDFS-13121. NPE when request file descriptors when SC read. Contributed by Zsolt Venczel.
2018-07-06 14:59:49 -07:00
344f324710
HDFS-13712. BlockReaderRemote.read() logging improvement. Contributed by Gergo Repas.
2018-07-03 11:07:45 +02:00
1804a31515
HDFS-13536. [PROVIDED Storage] HA for InMemoryAliasMap. Contributed by Virajith Jalaparti.
2018-07-02 10:48:20 -07:00
fef20a446f
HDFS-13635. Incorrect message when block is not found. Contributed by Gabor Bota.
2018-07-02 09:41:15 -07:00
5d748bd056
HDFS-13702. Remove HTrace hooks from DFSClient to reduce CPU usage. Contributed by Todd Lipcon.
2018-07-02 12:11:06 +02:00
6ba9974108
HDFS-13703. Avoid allocation of CorruptedBlocks hashmap when no corrupted blocks are hit. Contributed by Todd Lipcon.
2018-07-02 12:02:19 +02:00
7a3c6e9c3c
HADOOP-15550. Avoid static initialization of ObjectMappers
2018-06-25 15:36:45 -07:00
fba9d7cd74
HDFS-13621. Upgrade commons-lang version to 3.7 in hadoop-hdfs-project. Contributed by Takanobu Asanuma.
2018-06-18 10:17:07 -07:00
980031bb04
HADOOP-13186. Multipart Uploader API. Contributed by Ewan Higgs
2018-06-17 11:54:26 -07:00
2e5cfe6df3
HDFS-13653. Make dfs.client.failover.random.order a per nameservice configuration. Contributed by Ekanth Sethuramalingam.
2018-06-11 11:20:29 -07:00
000a678396
HDFS-12670. can't renew HDFS tokens with only the hdfs client jar. Contributed by Arpit Agarwal.
2018-06-08 17:57:12 -07:00
fba1c42adc
HDFS-13664. Refactor ConfiguredFailoverProxyProvider to make inheritance easier. Contributed by Chao Sun.
2018-06-08 16:36:42 -07:00
774c1f199e
HDFS-13511. Provide specialized exception when block length cannot be obtained. Contributed by Gabor Bota.
2018-06-05 21:25:08 -07:00
ba4011d64f
HADOOP-15217. FsUrlConnection does not handle paths with spaces. Contributed by Joseph Fourny and Zsolt Venczel.
2018-06-05 21:18:31 -07:00
6d5e87aec2
HADOOP-15507. Add MapReduce counters about EC bytes read.
2018-06-04 21:13:47 -07:00
c95b9b5c07
HDFS-13440. Log HDFS file name when client fails to connect. Contributed by Gabor Bota.
2018-05-31 15:20:59 -07:00
1361030e59
HDFS-13582. Improve backward compatibility for HDFS-13176 (WebHdfs file path gets truncated when having semicolon (;) inside). Contributed by Zsolt Venczel.
2018-05-31 07:59:21 -06:00
c9b63deb53
HDFS-13611. Unsafe use of Text as a ConcurrentHashMap key in PBHelperClient.
2018-05-24 09:56:23 -07:00
bc6d9d4c79
HDFS-13589: Add dfsAdmin command to query if upgrade is finalized. Contributed by Hanisha Koneru
2018-05-23 10:15:40 -07:00
34e8b9f9a8
HDFS-13540. DFSStripedInputStream should only allocate new buffers when reading. Contributed by Xiao Chen.
2018-05-23 19:10:09 +08:00
1d2640b613
HDFS-13601. Optimize ByteString conversions in PBHelper.
2018-05-22 23:55:20 -07:00
0fc988e6a3
HDFS-13512. WebHdfs getFileStatus doesn't return ecPolicy. Contributed by Ajay Kumar.
2018-05-16 11:28:39 -07:00
960940e0e0
HDFS-13539. DFSStripedInputStream NPE when reportCheckSumFailure.
2018-05-14 09:28:39 -07:00
84ecfe3ceb
HADOOP-14188. Remove the usage of org.mockito.internal.util.reflection.Whitebox
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-04-27 19:21:37 +09:00
a8e428b2dc
HDFS-13468. Add erasure coding metrics into ReadStatistics. (Contributed by Lei (Eddy) Xu)
2018-04-26 13:54:40 -07:00
6f655aed3b
HADOOP-15404. Remove multibyte characters in DataNodeUsageReportUtil
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-04-26 13:54:40 -07:00
0ec88ea42b
HDFS-13301. Ozone: Remove containerPort, ratisPort and ozoneRestPort from DatanodeID and DatanodeIDProto. Contributed by Shashikant Banerjee.
2018-04-26 05:36:04 -07:00
4e61bc431e
HDFS-13258. Ozone: restructure Hdsl/Ozone code to separated maven subprojects.
...
Contributed by Elek Marton, Mukul Kumar Singh, Xiaoyu Yao, Ajay Kumar, Anu Engineer, Lokesh Jain, Nanda Kumar.
2018-04-26 05:36:04 -07:00
5e7164c614
HDFS-13137. Ozone: Ozonefs read fails because ChunkGroupInputStream#read does not iterate through all the blocks in the key.
...
Contributed by Mukul Kumar Singh.
2018-04-26 05:36:04 -07:00
9a914126a7
HDFS-13008. Ozone: Add DN container open/close state to container report. Contributed by Xiaoyu Yao.
2018-04-26 05:36:04 -07:00
b4a3cf1476
HDFS-13070. Ozone: SCM: Support for container replica reconciliation - 1. Contributed by Nanda kumar.
2018-04-26 05:36:04 -07:00
4c10a849e8
HDFS-13078. Ozone: Update Ratis on Ozone to 0.1.1-alpha-8fd74ed-SNAPSHOT.
...
To fix large chunk reads (>4M) from Datanodes. Contributed by Mukul Kumar Singh.
2018-04-26 05:36:04 -07:00
9272e1021d
HDFS-12636. Ozone: OzoneFileSystem: Implement seek functionality for rpc client. Contributed by Lokesh Jain.
2018-04-26 05:36:04 -07:00
377b31ffa1
HDFS-13022. Block Storage: Kubernetes dynamic persistent volume provisioner. Contributed by Elek, Marton.
2018-04-26 05:36:04 -07:00
ee5495456e
HDFS-13116. Ozone: Refactor Pipeline to have transport and container specific information. Contributed by Mukul Kumar Singh.
2018-04-26 05:36:04 -07:00
3cf07b43bc
HDFS-13127. Fix TestContainerStateManager and TestOzoneConfigurationFields. Contributed by Mukul Kumar Singh.
2018-04-26 05:36:04 -07:00
443425a5d9
HDFS-12522. Ozone: Remove the Priority Queues used in the Container State Manager. Contributed by Anu Engineer.
2018-04-26 05:36:04 -07:00
6b03fca37d
HDFS-12868. Ozone: Service Discovery API. Contributed by Nanda Kumar.
2018-04-26 05:36:04 -07:00
fb09d75379
HDFS-13018. Block Storage: make the iscsi target addres configurable for discovery. Contributed by Elek, Marton.
2018-04-26 05:36:04 -07:00
ba4d5a52a8
HDFS-12986. Ozone: Update ozone to latest ratis snapshot build (0.1.1-alpha-0f7169d-SNAPSHOT). Contributed by Lokesh Jain
2018-04-26 05:36:04 -07:00
94c0346f35
HDFS-13024. Ozone: ContainerStateMachine should synchronize operations between createContainer and writeChunk. Contributed by Mukul Kumar Singh.
2018-04-26 05:36:04 -07:00
28b87af51b
HDFS-13013. Fix closeContainer API with the right container state change. Contributed by Xiaoyu Yao.
2018-04-26 05:36:04 -07:00
98d62e55c4
Revert "HDFS-12794. Ozone: Parallelize ChunkOutputSream Writes to container. Contributed by Shashikant Banerjee."
...
This reverts commit 6ce5ec6761
2018-04-26 05:36:04 -07:00
98eeabbdb3
HDFS-12794. Ozone: Parallelize ChunkOutputSream Writes to container. Contributed by Shashikant Banerjee.
2018-04-26 05:36:04 -07:00
98973cd0c4
HDFS-12966. Ozone: owner name should be set properly when the container allocation happens. Contributed by Shashikant Banerjee.
2018-04-26 05:36:04 -07:00
3965f1ec99
HDFS-12853. Ozone: Optimize chunk writes for Ratis by avoiding double writes. Contributed by Mukul Kumar Singh
2018-04-26 05:36:04 -07:00
5766792cad
HDFS-12870. Ozone: Service Discovery: REST endpoint in KSM for getServiceList. Contributed by Nanda kumar.
2018-04-26 05:36:04 -07:00
5f10c2d8ce
HDFS-12980. Ozone: SCM: Restructuring container state transition and event. Contributed by Nanda kumar.
2018-04-26 05:36:04 -07:00
87b7774488
HDFS-12958. Ozone: remove setAllocatedBytes method in ContainerInfo. Contributed by Chen Liang.
2018-04-26 05:36:04 -07:00
ce19f09bd0
HDFS-12908. Ozone: write chunk call fails because of Metrics registry exception. Contributed by Mukul Kumar Singh.
2018-04-26 05:36:04 -07:00
4b9f66a19e
HDFS-12869. Ozone: Service Discovery: RPC endpoint in KSM for getServiceList. Contributed by Nanda kumar.
2018-04-26 05:36:04 -07:00
966853894f
HDFS-12795. Ozone: SCM: Support for Container LifeCycleState PENDING_CLOSE and LifeCycleEvent FULL_CONTAINER. Contributed by Nandakumar.
2018-04-26 05:36:04 -07:00
26e270b908
HDFS-12751. Ozone: SCM: update container allocated size to container db for all the open containers in ContainerStateManager#close. Contributed by Chen Liang.
2018-04-26 05:36:04 -07:00
fd09c2ce5b
HDFS-12890 . Ozone: XceiverClient should have upper bound on async requests. Contributed by Shashikant Banerjee.
2018-04-26 05:36:04 -07:00
2f3068bc72
HDFS-12698. Ozone: Use time units in the Ozone configuration values. Contributed by Elek, Marton.
2018-04-26 05:36:04 -07:00
8ff98e2af3
HDFS-12000. Ozone: Container : Add key versioning support-1. Contributed by Chen Liang.
2018-04-26 05:36:04 -07:00
c0c87dea9b
HDFS-12626. Ozone : delete open key entries that will no longer be closed. Contributed by Chen Liang.
2018-04-26 05:36:04 -07:00
74362cb503
HDFS-12901. Ozone: SCM: Expose StorageContainerManager#getScmId through container protocol. Contributed by Nanda kumar.
2018-04-26 05:36:04 -07:00
7bba3943f0
HDFS-12876. Ozone: moving NodeType from OzoneConsts to Ozone.proto. Contributed by Nanda kumar.
2018-04-26 05:36:04 -07:00
8023ba50b5
HDFS-12807. Ozone: Expose RockDB stats via JMX for Ozone metadata stores. Contributed by Elek, Marton.
2018-04-26 05:36:04 -07:00
86802a461c
HDFS-12851. Ozone: Upgrade to latest ratis build. Contributed by Mukul Kumar Singh
2018-04-26 05:36:04 -07:00
f9c11d952c
HDFS-12740. SCM should support a RPC to share the cluster Id with KSM and DataNodes. Contributed by Shashikant Banerjee.
2018-04-26 05:36:04 -07:00
4460ac9c57
HDFS-12756. Ozone: Add datanodeID to heartbeat responses and container protocol. Contributed by Anu Engineer.
2018-04-26 05:36:04 -07:00
740a06cdd7
HDFS-12719. Ozone: Fix checkstyle, javac, whitespace issues in HDFS-7240 branch. Contributed by Mukul Kumar Singh.
2018-04-26 05:36:04 -07:00
e395244e95
HDFS-12758. Ozone: Correcting assertEquals argument order in test cases. Contributed by Bharat Viswanadham.
2018-04-26 05:36:04 -07:00
cec96b296f
HDFS-12549. Ozone: OzoneClient: Support for REST protocol. Contributed by Nanda Kumar.
2018-04-26 05:36:04 -07:00
9734f505ea
HDFS-12739. Add Support for SCM --init command. Contributed by Shashikant Banerjee.
2018-04-26 05:36:04 -07:00
709d56fdc9
HDFS-12474. Ozone: SCM: Handling container report with key count and container usage. Contributed by Nanda kumar.
2018-04-26 05:36:04 -07:00
2ec24a7542
HDFS-12774. Ozone: OzoneClient: Moving OzoneException from hadoop-hdfs to hadoop-hdfs-client. Contributed by Nanda kumar.
2018-04-26 05:36:04 -07:00
7ebe79e879
HDFS-12720. Ozone: Ratis options are not passed from KSM Client protobuf helper correctly. Contributed by Mukul Kumar Singh.
2018-04-26 05:36:04 -07:00
76d34bca62
HDFS-12521. Ozone: SCM should read all Container info into memory when booting up. Contributed by Lokesh Jain.
2018-04-26 05:36:04 -07:00
6291ca1e32
HDFS-12663. Ozone: OzoneClient: Remove protobuf classes exposed to clients through OzoneBucket. Contributed by Nandakumar.
2018-04-26 05:36:04 -07:00
1b56a73a4f
HDFS-12680. Ozone: SCM: Lease support for container creation. Contributed by Nandakumar.
2018-04-26 05:36:04 -07:00
Anu Engineer
Wei-Chiu Chuang
LeonGao91
Siyao Meng
Masatake Iwasaki
Erik Krogen
Siyao Meng
Chao Sun
HUAN-PING SU
Chen Liang
Weiwei Yang
Inigo Goiri
Inigo Goiri
Ajay Kumar
Giovanni Matteo Fumarola
Surendra Singh Lilhore
sunlisheng
Shweta Yakkali
mpicker90
huzheng
Akira Ajisaka
Eric Yang
Srinivasu Majeti
Vinayakumar B
Sahil Takiar
lys0716
Takanobu Asanuma
Sahil Takiar
Yongjun Zhang
Steve Loughran
Kitti Nanasi
Dinesh Chitlangia
Konstantin V Shvachko
Plamen Jeliazkov
Zsolt Venczel
Daniel Templeton
Brahma Reddy Battula
Jason Lowe
Pranay Singh
Xiao Chen
Hrishikesh Gadre
Sunil G
Uma Maheswara Rao G
Rakesh Radhakrishnan
Andrew Wang
Todd Lipcon
Chris Douglas
Arpit Agarwal
Sean Mackrory
Bharat Viswanadham
Sammi Chen
Ewan Higgs
Lei Xu
Takanobu Asanuma
Nanda kumar
Mukul Kumar Singh
Xiaoyu Yao
Tsz-Wo Nicholas Sze