f59f7f2175
HDFS-15573. Only log warning if considerLoad and considerStorageType are both true. Contributed by Stephen O'Donnell
2020-09-12 01:41:38 -07:00
958cab804e
Revert "HADOOP-17244. S3A directory delete tombstones dir markers prematurely. ( #2280 )"
...
This reverts commit 9960c01a25
2020-09-11 18:07:49 +01:00
9afec2ed17
YARN-10390: LeafQueue: retain user limits cache across assignContainers() calls. Contributed by Samir Khan (samkhan).
2020-09-11 13:29:26 +00:00
2a6fdcedeb
HDFS-15564. Add Test annotation for TestPersistBlocks#testRestartDfsWithSync. Contributed by Fei Hui.
2020-09-11 18:21:25 +05:30
89428f142f
HDFS-15551. Tiny Improve for DeadNode detector ( #2265 )
...
Contributed by imbajin.
Reviewed-by: leosunli <lisheng.sun08@gmail.com>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2020-09-11 15:44:03 +08:00
56ebabd426
HADOOP-17222. Create socket address leveraging URI cache ( #2241 )
...
Contributed by fanrui.
Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2020-09-10 22:30:52 -07:00
12a316cdf9
HADOOP-15891. provide Regex Based Mount Point In Inode Tree ( #2185 ). Contributed by Zhenzhao Wang.
...
Co-authored-by: Zhenzhao Wang <zhenzhaowang@gmail.com>
2020-09-10 21:20:32 -07:00
9960c01a25
HADOOP-17244. S3A directory delete tombstones dir markers prematurely. ( #2280 )
...
This changes directory tree deletion so that only files are incrementally deleted
from S3Guard after the objects are deleted; the directories are left alone
until metadataStore.deleteSubtree(path) is invoke.
This avoids directory tombstones being added above files/child directories,
which stop the treewalk and delete phase from working.
Also:
* Callback to delete objects splits files and dirs so that
any problems deleting the dirs doesn't trigger s3guard updates
* New statistic to measure #of objects deleted, alongside request count.
* Callback listFilesAndEmptyDirectories renamed listFilesAndDirectoryMarkers
to clarify behavior.
* Test enhancements to replicate the failure and verify the fix
Contributed by Steve Loughran
2020-09-10 17:03:52 +01:00
43c52d6495
HDFS-15563. Incorrect getTrashRoot return value when a non-snapshottable dir prefix matches the path of a snapshottable dir ( #2295 )
...
Change-Id: Iefcbc14ca939b61f0c1befc9a94d686be1684ddc
2020-09-10 15:55:23 +05:30
e5fe326270
HADOOP-17165. Implement service-user feature in DecayRPCScheduler. ( #2240 )
2020-09-09 09:56:58 -07:00
85119267be
HADOOP-17166. ABFS: configure output stream thread pool ( #2179 )
...
Adds the options to control the size of the per-output-stream threadpool
when writing data through the abfs connector
* fs.azure.write.max.concurrent.requests
* fs.azure.write.max.requests.to.queue
Contributed by Bilahari T H
2020-09-09 16:41:36 +01:00
773ac799c6
YARN-10387. Implement logic which returns MappingRule objects based on mapping rules. Contributed by Peter Bacsko
2020-09-09 15:12:58 +02:00
1d6d0d8207
HDFS-14694. Call recoverLease on DFSOutputStream close exception. Contributed by Lisheng Sun.
...
Co-authored-by: Chen Zhang <chzhang1987@gmail.com>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
2020-09-09 20:09:13 +08:00
43572fc7f8
YARN-10397. SchedulerRequest should be forwarded to scheduler if custom scheduler supports placement constraints. Contributed by Bilwa S T.
2020-09-09 17:08:13 +05:30
2029556dbb
HADOOP-17181. Handle transient stream read failures in FileSystem contract tests ( #2286 )
...
Contributed by Steve Loughran.
* Fixes AbstractContractSeekTest test to use readFully
* Doesn't do this to AbstractContractUnbufferTest test as it changes the test too much.
Instead just notes in the error that this may be transient
The issue is that read(buffer) doesn't guarantee that the buffer is filled, only that it will
read up to a point, and that may be just be the amount of data left in the TCP packet.
readFully corrects for this, but using it in the unbuffer test runs the risk that what
is tested for in terms of unbuffering doesn't actually get validated.
2020-09-09 11:58:26 +01:00
aba4a506d6
YARN-10424. Adapt existing AppName and UserGroupMapping unittests to ensure backwards compatibility. Contributed by Benjamin Teke.
2020-09-09 09:41:11 +02:00
0d855159f0
HADOOP-17229. No updation of bytes received counter value after response failure occurs in ABFS ( #2264 )
...
Contributed by Mehakmeet Singh
2020-09-08 10:14:23 +01:00
84ed6adccc
HADOOP-17158. Test timeout for ITestAbfsInputStreamStatistics#testReadAheadCounters ( #2272 )
...
Contributed by: Mehakmeet Singh.
2020-09-08 10:11:06 +01:00
c4fb4044b2
YARN-10415. Create a group matcher which checks ALL groups of the user. Contributed by Gergely Pollak.
2020-09-08 10:57:00 +02:00
ac7d4623ae
HDFS-15558: ViewDistributedFileSystem#recoverLease should call super.recoverLease when there are no mounts configured ( #2275 ) Contributed by Uma Maheswara Rao G.
2020-09-07 11:36:13 -07:00
34fe74da0e
YARN-10332. RESOURCE_UPDATE event was repeatedly registered in DECOMMISSIONING state. Contributed by yehuanhuan
2020-09-07 11:44:09 +02:00
360bbcd3bc
YARN-10411. Create an allowCreate flag for MappingRuleAction. Contributed by Gergely Pollak.
2020-09-07 11:39:03 +02:00
dd6c66556e
YARN-9136. getNMResourceInfo NodeManager REST API method is not documented
2020-09-07 10:29:20 +02:00
1841a5bb03
YARN-10376. Create a class that covers the functionality of UserGroupMappingPlacementRule and AppNameMappingPlacementRule using the new mapping rules. Contributed by Gergely Pollak.
2020-09-05 10:00:33 +02:00
8a3952436c
HADOOP-17245. Add OzoneFileSystem classes to core-default.xml. ( #2276 )
2020-09-04 18:38:48 -07:00
5346cc3263
HADOOP-17227. S3A Marker Tool tuning ( #2254 )
...
Contributed by Steve Loughran.
2020-09-04 14:58:03 +01:00
696e4fe50e
Typos in YarnScheduler#allocate method's doc comment. Contributed by Siddharth Ahuja
2020-09-04 09:59:54 +02:00
139a43e98e
HADOOP-17167 ITestS3AEncryptionWithDefaultS3Settings failing ( #2187 )
...
Now skips ITestS3AEncryptionWithDefaultS3Settings.testEncryptionOverRename
when server side encryption is not set to sse:kms
Contributed by Mukund Thakur
2020-09-03 19:35:24 +01:00
b3660d0147
HDFS-15529: getChildFilesystems should include fallback fs as well ( #2234 ). Contributed by Uma Maheswara Rao G.
2020-09-03 11:06:20 -07:00
5c15815773
YARN-10375. CS Mapping rule config parser should return MappingRule objects. Contributed by Gergely Pollak.
2020-09-03 18:54:22 +02:00
288dab7563
YARN-10372. Create MappingRule class to represent each CS mapping rule. Contributed by Gergely Pollak
2020-09-03 15:02:48 +02:00
5e12dc51f0
YARN-10419. Javadoc error in hadoop-yarn-server-common module. ( #2271 )
2020-09-03 16:17:52 +09:00
0207f5cf46
HADOOP-17240. Fix wrong command line for setting up CentOS 8. ( #2268 )
2020-09-02 15:12:03 +09:00
c7dab2b23e
YARN-10374. Create Actions for CS mapping rules. Contributed by Gergely Pollak.
2020-09-01 17:38:49 +02:00
73a0d149e2
YARN-10386. Addendum patch: Fixing ASF Warnings. Contributed by Peter Bacsko.
2020-09-01 14:39:24 +02:00
7e671f3b65
HADOOP-17235. Erasure Coding: Remove dead code from common side. Contributed by Fei Hui.
2020-09-01 11:35:55 +08:00
658b1a6cde
YARN-10407. Add phantomjsdriver.log to gitignore. ( #2244 )
2020-09-01 10:44:55 +09:00
4ffe26f9b8
[UI1] Provide a way to hide Tools section in Web UIv1. Contributed by Andras Gyori
2020-08-31 16:00:39 +02:00
60de592a88
HDFS-15550. Remove unused imports from TestFileTruncate.java. Contributed by Ravuri Sushma sree.
2020-08-31 14:49:12 +08:00
2d03209a06
HDFS-15542. Add identified snapshot corruption tests for ordered snapshot deletion ( #2251 )
2020-08-31 11:29:48 +05:30
56a5c360a1
YARN-10373. Create Matchers for CS mapping rules. Contributed by Gergely Pollak
2020-08-29 21:34:55 +02:00
f4f872b778
YARN-10371. Create variable context class for CS queue mapping rules. Contributed by Gergely Pollak
2020-08-29 17:31:48 +02:00
a888d580d8
YARN-10409. Improve MockQueueHierarchyBuilder to detect queue ambiguity. Contributed by Gergely Pollak
2020-08-29 15:24:50 +02:00
44542863f4
YARN-10408. Extract MockQueueHierarchyBuilder to a separate class. Contributed by Gergely Pollak
2020-08-28 17:35:45 +02:00
6e3d7050a9
YARN-10386. Create new JSON schema for Placement Rules. Contributed by Peter Bacsko and Benjamin Teke
2020-08-28 17:09:31 +02:00
c392d9022a
HDFS-15471. TestHDFSContractMultipartUploader failing ( #2252 )
...
Contributed by Steve Loughran
(Was: broken by Steve Loughran)
2020-08-28 15:39:09 +01:00
bfd99f31fd
HADOOP-17232. Erasure Coding: Typo in document. Contributed by Fei Hui.
2020-08-28 17:38:23 +05:30
30b1b1cc04
HADOOP-17234. Add .asf.yaml to allow Github to Jira integration. ( #2253 ). Contributed by Ayush Saxena.
2020-08-28 17:22:46 +05:30
06793da100
HDFS-15531. Namenode UI: List snapshots in separate table for each snapshottable directory ( #2230 )
2020-08-27 11:36:30 -07:00
d1c60a53f6
HADOOP-17194. Adding Context class for AbfsClient in ABFS ( #2216 )
...
Contributed by Mehakmeet Singh.
2020-08-27 11:27:00 +01:00
Mingliang Liu
Steve Loughran
Eric E Payne
hemanthboyina
imbajin
1996fanrui
zz
Siyao Meng
Takanobu Asanuma
bilaharith
Szilard Nemeth
He Xiaoqiao
Brahma Reddy Battula
Peter Bacsko
Mehakmeet Singh
Uma Maheswara Rao G
Adam Antal
Bharat Viswanadham
Mukund Thakur
Masatake Iwasaki
tkuramoto33
bshashikant
Ayush Saxena
Vivek Ratnavel Subramanian