7b219778e0
HADOOP-16433. S3Guard: Filter expired entries and tombstones when listing with MetadataStore.listChildren().
...
Contributed by Gabor Bota.
This pulls the tracking of the lastUpdated timestamp of metadata entries up from the DDB metastore into all s3guard stores, and then uses this to filter out expired tombstones from listings.
Change-Id: I80f121236b49c75a024116f65a3ef29d3580b462
2019-07-24 18:11:43 +01:00
cf9ff08f4b
HDDS-1848. Fix TestOzoneManagerHA and TestOzoneManagerSnapShotProvider. ( #1148 )
2019-07-23 20:23:06 -07:00
a1251addff
HADOOP-16431. Remove useless log in IOUtils.java and ExceptionDiags.java.
...
This closes #1091
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-07-24 10:04:39 +09:00
ecb9f817ac
HDDS-1845. Fix OMVolumeSetQuota|OwnerRequest#validateAndUpdateCache return response. ( #1141 )
2019-07-23 14:45:47 -07:00
fbe2fc6580
HDDS-1846. Default value for checksum bytes is different in ozone-site.xml and code.
...
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-07-23 10:42:16 -07:00
278390f5bf
HDDS-1710. Fix checkstyle errors
...
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-07-23 07:56:30 -07:00
4317d33232
HADOOP-16380. S3Guard to determine empty directory status for all non-root directories.
...
Contributed by Steve Loughran and Gabor Bota.
This
* Asks S3Guard to determine the empty directory status.
* Has S3A's root directory rm("/") command to always return false (as abfs does)
* Documents that object stores MAY do this
* Overloads ContractTestUtils.assertDeleted to let assertions declare that the source directory does not need to exist. This stops inconsistencies in directory listings failing a root test.
It avoids a recent regression (HADOOP-16279) where if there was a tombstone above the first element found in a directory listing, the directory would be considered empty, when in fact there were child entries. That could downgrade an rm(path, recursive) to a no-op, while also confusing rename(src, dest), as dest could be mistaken for an empty directory and so permit the copy above it, rather than reject it "destination path exists and is not empty".
Change-Id: I136a3d1a5a48a67e6155d790a40ff558d0d2c108
2019-07-23 14:52:03 +01:00
eb36b09cb7
HADOOP-16443. Improve help text for setfacl --set option.
...
Contributed by S O'Donnell.
Change-Id: I1da46c4c414a5d2b07ee15867508f0799440a413
2019-07-23 10:24:07 +01:00
377f95bbe8
HDFS-13693. Remove unnecessary search in INodeDirectory.addChild during image loading. Contributed by Lisheng Sun.
2019-07-23 08:37:55 +05:30
ee87e9a42e
MAPREDUCE-7076. TestNNBench#testNNBenchCreateReadAndDelete failing in our internal build
...
This closes #1089
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-07-23 10:34:10 +09:00
c533b79c32
HDDS-1710. Publish JVM metrics via Hadoop metrics
...
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-07-22 16:27:50 -07:00
d59f2711e0
HDDS-1803. shellcheck.sh does not work on Mac
...
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-07-22 16:21:42 -07:00
d70ec4b5fa
HDDS-1799. Add goofyfs to the ozone-runner docker image
...
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-07-22 15:53:18 -07:00
f3f2f3419d
HDDS-1585. Add LICENSE.txt and NOTICE.txt to Ozone Recon Web
...
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-07-22 15:26:23 -07:00
2ea71d953b
HDDS-1686. Remove check to get from openKeyTable in acl implementatio… ( #966 )
2019-07-22 15:11:10 -07:00
c958eddcf4
HDDS-1811. Prometheus metrics are broken.
...
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-07-22 14:12:06 -07:00
cdc36fe286
HDDS-1649. On installSnapshot notification from OM leader, download checkpoint and reload OM state ( #948 )
2019-07-22 12:06:55 -07:00
340bbaf8bf
HDDS-1840. Fix TestSecureOzoneContainer. ( #1135 )
2019-07-22 10:23:48 -07:00
acdb0a1276
HDDS-1841. Fix TestSecureContainerServer. ( #1136 )
2019-07-20 08:14:23 -07:00
4aa76e327d
HDDS-1805. Implement S3 Initiate MPU request to use Cache and DoubleBuffer. ( #1108 )
2019-07-20 08:11:36 -07:00
e60f5e2572
HADOOP-16440. Distcp can not preserve timestamp with -delete option. Contributed by ludun.
2019-07-20 13:11:14 +05:30
0b45293abb
HADOOP-16404. ABFS default blocksize change(256MB from 512MB)
...
Contributed by: Arun Singh
2019-07-19 20:21:28 -07:00
f5b2f7538c
HDDS-1795. Implement S3 Delete Bucket request to use Cache and DoubleBuffer. ( #1097 )
2019-07-19 17:43:23 -07:00
69a46a95bb
HDDS-1713. ReplicationManager fail to find proper node topology based… ( #1112 )
2019-07-19 15:45:26 -07:00
7f1b76ca35
HADOOP-13868. [s3a] New default for S3A multi-part configuration ( #1125 )
2019-07-19 09:49:59 -06:00
cd967c75a7
HADOOP-15847. S3Guard testConcurrentTableCreations to set R/W capacity == 0
...
Contributed by lqjaclee
Change-Id: I4a4d5b29f2677c188799479e4db38f07fa0591d1
2019-07-19 14:46:55 +01:00
6282c02d7d
HDFS-14577. RBF: FederationUtil#newInstance should allow constructor without context. Contributed by CR Hota.
2019-07-19 17:44:13 +05:30
4e66cb9333
HDDS-1653. Add option to "ozone scmcli printTopology" to order the output acccording to topology layer. Contributed by Xiaoyu Yao. ( #1067 )
...
* HDDS-1653. Add option to "ozone scmcli printTopology" to order the output acccording to topology layer. Contributed by Xiaoyu Yao.
* use ip/hostname instead of network name for -o output and add smoke test
2019-07-19 11:00:49 +08:00
d545f9c290
HADOOP-16437 documentation typo fix: fs.s3a.experimental.input.fadvise
...
Fix fs.s3a.experimental.fadvise to fs.s3a.experimental.input.fadvise
Contributed by: Josh Rosen
2019-07-18 23:19:38 +01:00
d5ef38b093
HDDS-1822. NPE in SCMCommonPolicy.chooseDatanodes ( #1120 )
2019-07-18 13:28:03 -07:00
9838a47d44
YARN-6046. Fixed documentation error in YarnApplicationSecurity.
...
Contributed by Yousef Abu-Salah
2019-07-18 12:39:18 -04:00
f22818d0d6
HDDS-1820. Fix numKeys metrics in OM HA. ( #1116 )
2019-07-18 09:36:15 -07:00
c34ceb5fde
YARN-9568. Fixed NPE in MiniYarnCluster during FileSystemNodeAttributeStore.recover.
...
Contributed by Steve Loughran
2019-07-18 12:30:53 -04:00
8ef2365ffd
HDDS-1779. TestWatchForCommit tests are flaky.Contributed by Shashikant Banerjee. ( #1071 )
2019-07-18 19:57:12 +05:30
79f6118dcc
MAPREDUCE-6521. MiniMRYarnCluster should not create /tmp/hadoop-yarn/staging on local filesystem in unit test.
...
Contributed by Masatake Iwasaki.
Change-Id: Id74b90eb7cfb9d676188fc5aa47249ee8904a3d5
2019-07-18 13:15:18 +01:00
20cf50c6d0
HDDS-1654. Ensure container state on datanode gets synced to disk whennever state change happens. Cotributed by Shashikant Banerjee. ( #923 )
2019-07-18 17:09:05 +05:30
ccceedb432
HDDS-1780. TestFailureHandlingByClient tests are flaky. Contributed by Shashikant Banerjee. ( #1073 )
2019-07-18 16:01:58 +05:30
23e9bebe13
HDDS-1767: ContainerStateMachine should have its own executors for executing applyTransaction calls ( #1087 )
2019-07-18 14:48:13 +05:30
53a4c22b40
HDDS-1481: Cleanup BasicOzoneFileSystem#mkdir ( #1114 )
2019-07-18 14:40:45 +05:30
3dc256ef81
HDDS-1689. Implement S3 Create Bucket request to use Cache and DoubleBuffer. ( #1088 )
2019-07-17 21:46:27 -07:00
73e6ffce69
HDDS-1802. Add Eviction policy for table cache. ( #1100 )
2019-07-17 18:19:06 -07:00
9792f58662
HDFS-14653. RBF: Correct the default value for dfs.federation.router.namenode.heartbeat.enable. Contributed by Ayush Saxena.
2019-07-18 10:16:20 +09:00
5e6cc6fe8a
HDFS-12979. [SBN read] StandbyNode should upload FsImage to ObserverNode after checkpointing. Contributed by Chen Liang.
2019-07-17 14:37:16 -07:00
0541322c9a
HDDS-1675. Cleanup Volume Request 2 phase old code. ( #964 )
2019-07-17 14:13:06 -07:00
303a7f8a39
HDDS-1721 : Client Metrics are not being pushed to the configured sink while running a hadoop command to write to Ozone. ( #1034 )
2019-07-17 12:05:43 -07:00
af0665c7f5
HDFS-13647. Fix the description of storageType option for space quota. Contributed by Takanobu Asanuma.
2019-07-17 23:22:32 +05:30
eb0849b547
HDDS-1801. Make Topology Aware Replication/Read non-default for ozone 0.4.1. Contributed by Xiaoyu Yao.
...
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-07-17 10:32:59 -07:00
c58e11bf52
HADOOP-16383. Pass ITtlTimeProvider instance in initialize method in MetadataStore interface. Contributed by Gabor Bota. ( #1009 )
2019-07-17 16:24:39 +02:00
256fcc160e
HDDS-1785. OOM error in Freon due to the concurrency handling
...
Closes #1085
2019-07-17 15:02:15 +02:00
b4466a3b0a
HADOOP-16341. ShutDownHookManager: Regressed performance on Hook removals after HADOOP-15679
...
Contributed by Gopal V and Atilla Magyar.
Change-Id: I066d5eece332a1673594de0f9b484443f95530ec
2019-07-17 13:50:02 +01:00
Gabor Bota
Bharat Viswanadham
sunlisheng
Doroszlai, Attila
Steve Loughran
S O'Donnell
Ayush Saxena
pingsutw
Márton Elek
Vivek Ratnavel Subramanian
Hanisha Koneru
Arun Singh
Sammi Chen
Sean Mackrory
lqjaclee
Xiaoyu Yao
Josh Rosen
Doroszlai, Attila
Eric Yang
Shashikant Banerjee
Masatake Iwasaki
Lokesh Jain
Takanobu Asanuma
Chen Liang
avijayanhwx
Gopal V