770adc5d4a
HADOOP-16578 : Avoid FileSystem API calls when FileSystem already exists
2019-10-01 17:38:11 -07:00
d947ded053
HADOOP-16619. Upgrade jackson and jackson-databind to 2.9.10 ( #1554 ). Contributed by Siyao Meng.
2019-10-01 12:46:40 -07:00
1921e94292
HADOOP-16458. LocatedFileStatusFetcher.getFileStatuses failing intermittently with S3
...
Contributed by Steve Loughran.
Includes
-S3A glob scans don't bother trying to resolve symlinks
-stack traces don't get lost in getFileStatuses() when exceptions are wrapped
-debug level logging of what is up in Globber
-Contains HADOOP-13373. Add S3A implementation of FSMainOperationsBaseTest.
-ITestRestrictedReadAccess tests incomplete read access to files.
This adds a builder API for constructing globbers which other stores can use
so that they too can skip symlink resolution when not needed.
Change-Id: I23bcdb2783d6bd77cf168fdc165b1b4b334d91c7
2019-10-01 18:11:05 +01:00
918b470deb
HDDS-2166. Some RPC metrics are missing from SCM prometheus endpoint
...
Closes #1505
2019-10-01 18:01:51 +02:00
6ef6594c7e
HDFS-14492. Snapshot memory leak. Contributed by Wei-Chiu Chuang. ( #1370 )
...
* HDFS-14492. Snapshot memory leak. Contributed by Wei-Chiu Chuang.
Change-Id: I9e5e450c07ad70aa1905973896c4f627042dbd37
* Fix checkstyle
Change-Id: I16d4bd4f03a971e1ed36cf57d89dc42357ef8fbf
2019-10-01 21:16:46 +05:30
99cd7572f1
YARN-9801. SchedConfCli does not work wiwith https mode. Contributed by Prabhu Joseph
2019-10-01 20:06:21 +05:30
425a6c89ae
YARN-9858. Optimize RMContext getExclusiveEnforcedPartitions. Contributed by Jonathan Hung.
2019-10-01 16:02:26 +05:30
8efd25b33a
HDDS-1615. ManagedChannel references are being leaked in ReplicationSupervisor.java. Contributed by Mukul Kumar Singh. ( #1547 )
2019-10-01 14:31:15 +05:30
137546a78a
YARN-9864. Format CS Configuration present in Configuration Store. Contributeed by Prabhu Joseph
2019-10-01 09:08:41 +05:30
b3275ab1f2
HDFS-14305. Fix serial number calculation in BlockTokenSecretManager to avoid token key ID overlap between NameNodes. Contributed by Konstantin V Shvachko.
2019-09-30 16:48:10 -07:00
e5bba592a8
HDDS-2205. checkstyle.sh reports wrong failure count
...
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-09-30 16:32:38 -07:00
4d3c580b03
YARN-9859. Refactoring of OpportunisticContainerAllocator. Contributed by Abhishek Modi.
2019-09-30 23:40:15 +05:30
98ca07ebed
HDDS-2207. Update Ratis to latest snapshot. Contributed by Shashikant Bannerjee. ( #1550 )
2019-09-30 20:36:27 +05:30
b46d82339f
HDDS-2202. Remove unused import in OmUtils
...
Closes #1543
2019-09-30 16:02:30 +02:00
a530ac3f50
HDDS-2153. Add a config to tune max pending requests in Ratis leader
...
Closes #1474
2019-09-30 15:42:04 +02:00
d6b0a8da77
HDDS-2183. Container and pipline subcommands of scmcli should be grouped
...
Closes #1532
2019-09-30 14:38:01 +02:00
760b523e58
Revert "HDFS-14305. Fix serial number calculation in BlockTokenSecretManager to avoid token key ID overlap between NameNodes. Contributed by He Xiaoqiao."
...
This reverts commit 0feba4396f
2019-09-29 13:11:31 -07:00
c0edc848a8
HADOOP-16548 : Disable Flush() over config
2019-09-28 20:39:42 -07:00
d8313b2274
HDFS-14850. Optimize FileSystemAccessService#getFileSystemConfiguration. Contributed by Lisheng Sun.
2019-09-28 17:20:44 -07:00
2200871607
HDFS-14876. Remove unused imports from TestBlockMissingException.java and TestClose.java. Contributed by Lisheng Sun.
2019-09-28 22:03:53 +05:30
c4c8d5fd0e
HDFS-14849. Erasure Coding: the internal block is replicated many times when datanode is decommissioning. Contributed by HuangTao.
2019-09-28 21:44:44 +05:30
0d5d0b914a
Revert "HDFS-14849. Erasure Coding: the internal block is replicated many times when datanode is decommissioning. Contributed by HuangTao."
...
This reverts commit ce58c05f1d
2019-09-28 21:32:15 +05:30
14b4fbc019
HDDS-1146. Adding container related metrics in SCM. ( #1541 )
2019-09-27 15:24:30 -07:00
13b427fc05
HDFS-14564: Add libhdfs APIs for readFully; add readFully to ByteBufferPositionedReadable ( #963 ) Contributed by Sahil Takiar.
...
Reviewed-by: Siyao Meng <smeng@cloudera.com>
2019-09-27 12:25:03 -07:00
ce58c05f1d
HDFS-14849. Erasure Coding: the internal block is replicated many times when datanode is decommissioning. Contributed by HuangTao.
2019-09-27 18:58:19 +05:30
a93a139b5d
HDDS-2185. createmrenv failure not reflected in acceptance test result
...
Closes #1533
2019-09-27 12:47:25 +02:00
8a9ede5cff
HADOOP-15616. Incorporate Tencent Cloud COS File System Implementation. Contributed by Yang Yu.
2019-09-27 16:53:13 +08:00
2870668cfa
Make upstream aware of 3.1.3 release.
2019-09-27 16:44:07 +08:00
9bf7a6e5b2
HDDS-2149. Replace findbugs with spotbugs
...
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-09-26 22:56:05 -07:00
0371e953ac
HDDS-2179. ConfigFileGenerator fails with Java 10 or newer
...
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-09-26 21:59:36 -07:00
b6ef8cc120
HDD-2193. Adding container related metrics in SCM.
...
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-09-26 21:42:20 -07:00
c55ac6a1c7
HDDS-2174. Delete GDPR Encryption Key from metadata when a Key is deleted
...
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-09-26 21:10:34 -07:00
2adcc3c932
HDFS-14785. [SBN read] Change client logging to be less aggressive. Contributed by Chen Liang.
2019-09-26 15:48:19 -07:00
b1e55cfb55
HDFS-14461. RBF: Fix intermittently failing kerberos related unit test. Contributed by Xiaoqiao He.
2019-09-26 15:26:23 -07:00
06998a1126
HDDS-2180. Add Object ID and update ID on VolumeList Object. ( #1526 )
2019-09-26 09:41:03 -07:00
18a8c2404e
YARN-9857. TestDelegationTokenRenewer throws NPE but tests pass. Contributed by Ahmed Hussein
2019-09-26 16:15:08 +00:00
1a2a352ecd
HDFS-11934. Add assertion to TestDefaultNameNodePort#testGetAddressFromConf. Contributed by Nikhil Navadiya.
2019-09-26 20:45:37 +05:30
a3f68933d7
HDFS-14873. Fix dfsadmin doc for triggerBlockReport. Contributed by Fei Hui.
2019-09-26 19:54:07 +05:30
7b6219a8d1
HDDS-2182. Fix checkstyle violations introduced by HDDS-1738
...
Closes #1529
2019-09-26 12:09:28 +02:00
587a8eeec8
HDFS-14874. Fix TestHDFSCLI and TestDFSShell test break because of logging change in mkdir ( #1522 ). Contributed by Gabor Bota.
2019-09-26 10:15:13 +05:30
606e341c1a
Addendum to YARN-9730. Support forcing configured partitions to be exclusive based on app node label
2019-09-25 17:54:53 -07:00
f647185905
HDDS-2067. Create generic service facade with tracing/metrics/logging support
...
Signed-off-by: Anu Engineer <aengineer@apache.org>
Co-Authored-By: Doroszlai, Attila <6454655+adoroszlai@users.noreply.github.com>
2019-09-25 17:19:22 -07:00
bdaaa3bbf2
HDFS-14832. RBF: Add Icon for ReadOnly False. Contributed by hemanthboyina
2019-09-25 23:48:49 +09:00
c89d22d13a
HADOOP-16602. mvn package fails in hadoop-aws.
...
Contributed by Xieming Li.
Follow-up to HADOOP-16445
Change-Id: I72c62d55b734a0f67556844f398ef4a50d9ea585
2019-09-25 14:15:35 +01:00
16f626f7f0
HDDS-2165. Freon fails if bucket does not exists
...
Closes #1503
2019-09-25 13:31:35 +02:00
e346e3638c
HADOOP-15691 Add PathCapabilities to FileSystem and FileContext.
...
Contributed by Steve Loughran.
This complements the StreamCapabilities Interface by allowing applications to probe for a specific path on a specific instance of a FileSystem client
to offer a specific capability.
This is intended to allow applications to determine
* Whether a method is implemented before calling it and dealing with UnsupportedOperationException.
* Whether a specific feature is believed to be available in the remote store.
As well as a common set of capabilities defined in CommonPathCapabilities,
file systems are free to add their own capabilities, prefixed with
fs. + schema + .
The plan is to identify and document more capabilities -and for file systems which add new features, for a declaration of the availability of the feature to always be available.
Note
* The remote store is not expected to be checked for the feature;
It is more a check of client API and the client's configuration/knowledge
of the state of the remote system.
* Permissions are not checked.
Change-Id: I80bfebe94f4a8bdad8f3ac055495735b824968f5
2019-09-25 12:16:41 +01:00
e6fb6ee94d
HDDS-1738. Add nullable annotation for OMResponse classes
...
Closes #1499
2019-09-25 12:14:07 +02:00
8baebb54e1
HDDS-2171. Dangling links in test report due to incompatible realpath
...
Closes #1515
2019-09-25 12:02:42 +02:00
c72457787d
YARN-6715. Fix documentation about NodeHealthScriptRunner. Contributed by Peter Bacsko
2019-09-25 10:36:22 +02:00
bec0864394
YARN-9808. Zero length files in container log output haven't got a header. Contributed by Adam Antal
2019-09-25 10:28:34 +02:00
Sneha Vijayarajan
Siyao Meng
Steve Loughran
Márton Elek
Wei-Chiu Chuang
Sunil G
bibinchundatt
Mukul Kumar Singh
Konstantin V Shvachko
Doroszlai, Attila
Abhishek Modi
bshashikant
dchitlangia
cxorm
Inigo Goiri
Ayush Saxena
Bharat Viswanadham
Sahil Takiar
Sammi Chen
Zhankun Tang
Chen Liang
Anu Engineer
Eric Badger
Vivek Ratnavel Subramanian
Gabor Bota
Jonathan Hung
Takanobu Asanuma
Xieming Li
Szilard Nemeth