3dc9db3aed
HDFS-15406. Improve the speed of Datanode Block Scan. Contributed by hemanthboyina
...
(cherry picked from commit 123777823e
2020-06-18 12:29:12 +01:00
d1ba6c963d
HADOOP-17020. Improve RawFileSystem Performance ( #2063 )
...
Contributed by : Mehakmeet Singh
Co-authored-by: Rajesh Balamohan
Co-authored-by: Mehakmeet Singh
2020-06-17 16:16:30 +01:00
ec913398a9
YARN-10281. Redundant QueuePath usage in UserGroupMappingPlacementRule and AppNameMappingPlacementRule. Contributed by Gergely Pollak
2020-06-17 14:36:08 +02:00
c1ef247dc6
YARN-10314. YarnClient throws NoClassDefFoundError for WebSocketException with only shaded client jars ( #2075 )
2020-06-17 09:29:49 +05:30
120ee793fc
HDFS-15387. FSUsage#DF should consider ViewFSOverloadScheme in processPath. Contributed by Uma Maheswara Rao G.
...
(cherry picked from commit 785b1def95
2020-06-16 20:02:44 -07:00
bee2846bee
HDFS-15389. DFSAdmin should close filesystem and dfsadmin -setBalancerBandwidth should work with ViewFSOverloadScheme. Contributed by Ayush Saxena
...
(cherry picked from commit cc671b16f7
2020-06-16 16:54:40 -07:00
418580446b
HDFS-15330. Document the ViewFSOverloadScheme details in ViewFS guide. Contributed by Uma Maheswara Rao G.
...
(cherry picked from commit 76fa0222f0
2020-06-16 16:54:01 -07:00
0b5e202614
HDFS-15321. Make DFSAdmin tool to work with ViewFileSystemOverloadScheme. Contributed by Uma Maheswara Rao G.
...
(cherry picked from commit ed83c865dd
2020-06-16 16:53:38 -07:00
8e71e85af7
HDFS-15322. Make NflyFS to work when ViewFsOverloadScheme's scheme and target uris schemes are same. Contributed by Uma Maheswara Rao G.
...
(cherry picked from commit 4734c77b4b
2020-06-16 16:53:10 -07:00
5b248de42d
HADOOP-17024. ListStatus on ViewFS root (ls "/") should list the linkFallBack root (configured target root). Contributed by Abhishek Das.
...
(cherry picked from commit ce4ec74453
2020-06-16 16:52:29 -07:00
544996c857
HDFS-15306. Make mount-table to read from central place ( Let's say from HDFS). Contributed by Uma Maheswara Rao G.
...
(cherry picked from commit ac4a2e11d9
2020-06-16 16:50:57 -07:00
0b9e5ea592
HDFS-15372. Files in snapshots no longer see attribute provider permissions. Contributed by Stephen O'Donnell.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 730a39d138
2020-06-16 16:02:07 -07:00
8be302a3b8
YARN-10274. Merge QueueMapping and QueueMappingEntity. Contributed by Gergely Pollak
2020-06-16 18:25:47 +02:00
52efe48d79
YARN-10292. FS-CS converter: add an option to enable asynchronous scheduling in CapacityScheduler. Contributed by Benjamin Teke
2020-06-16 18:01:39 +02:00
d73cdb1c86
SPNEGO TLS verification
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 81d8a887b0
2020-06-15 11:12:08 +09:00
3b8418250f
HDFS-15403. NPE in FileIoProvider#transferToSocketFully. Contributed by hemanthboyina.
...
(cherry picked from commit f41a144077
2020-06-15 09:17:40 +09:00
534b15caf9
HADOOP-17046. Support downstreams' existing Hadoop-rpc implementations using non-shaded protobuf classes ( #2026 )
2020-06-12 23:20:10 +05:30
fcd7ce53b5
YARN-10312. Add support for yarn logs -logFile to retain backward compatibility.
...
Contributed by Jim Brennan.
(cherry picked from commit fed6fecd3a
2020-06-11 21:11:20 +00:00
e35f619841
YARN-10296. Make ContainerPBImpl#getId/setId synchronized. Contributed by Benjamin Teke
2020-06-10 18:00:21 +02:00
043628dcf1
HDFS-15398. EC: hdfs client hangs due to exception during addBlock. Contributed by Hongbing Wang.
2020-06-10 12:09:34 +05:30
a7526ba9f7
YARN-10300: appMasterHost not set in RM ApplicationSummary when AM fails before first heartbeat. Contributed by Eric Badger (ebadger).
...
(cherry picked from commit 56247db302
2020-06-09 18:51:46 +00:00
5e290e702f
HADOOP-17050. S3A to support additional token issuers
...
Contributed by Steve Loughran.
S3A delegation token providers will be asked for any additional
token issuers, an array can be returned,
each one will be asked for tokens when DelegationTokenIssuer collects
all the tokens for a filesystem.
Change-Id: I1bd3035bbff98cbd8e1d1ac7fc615d937e6bb7bb
2020-06-09 14:43:02 +01:00
2d4faa39e8
HDFS-15211. EC: File write hangs during close in case of Exception during updatePipeline. Contributed by Ayush Saxena.
...
*Added missed test file.
2020-06-09 18:47:24 +05:30
890617c7ac
Revert "MAPREDUCE-7277. IndexCache totalMemoryUsed differs from cache contents. Contributed by Jon Eagles (jeagles)."
...
This reverts commit 741fcf2c63
2020-06-08 20:25:02 +00:00
fa723aa7f8
HADOOP-17047. TODO comment exist in trunk while related issue HADOOP-6223 is already fixed. Contributed by Rungroj Maipradit
2020-06-08 11:31:42 -07:00
543075b845
HADOOP-17059. ArrayIndexOfboundsException in ViewFileSystem#listStatus. Contributed by hemanthboyina
2020-06-08 10:38:17 -07:00
ac307fe20d
YARN-10284. Add lazy initialization of LogAggregationFileControllerFactory in LogServlet. Contributed by Adam Antal
2020-06-05 12:40:57 +02:00
ec8f3714e0
HDFS-15386. ReplicaNotFoundException keeps happening in DN after removing multiple DN's data directories ( #2052 )
...
Contributed by Toshihiro Suzuki.
(cherry picked from commit 545a0a147c
2020-06-05 11:17:13 +01:00
a266e32d82
YARN-10286. PendingContainers bugs in the scheduler outputs. Contributed by Andras Gyori
2020-06-05 09:50:43 +02:00
9cfc0e50fa
HADOOP-17056. Addendum patch: Fix typo
...
(cherry picked from commit 5157118bd7
2020-06-04 16:35:27 +09:00
0b25913384
HADOOP-17062. Fix shelldocs path in Jenkinsfile ( #2049 )
...
(cherry picked from commit 704409d53b
2020-06-04 06:05:51 +09:00
1714589609
HADOOP-17016. Adding Common Counters in ABFS ( #1991 ).
...
Contributed by: Mehakmeet Singh.
Change-Id: Ib84e7a42f28e064df4c6204fcce33e573360bf42
2020-06-03 20:02:44 +01:00
8a642caca8
HADOOP-16568. S3A FullCredentialsTokenBinding fails if local credentials are unset. ( #1441 )
...
Contributed by Steve Loughran.
Move the loading to deployUnbonded (where they are required) and add a safety check when a new DT is requested
Change-Id: I03c69aa2e16accfccddca756b2771ff832e7dd58
2020-06-03 17:08:52 +01:00
cf84bec6e3
HADOOP-14566. Add seek support for SFTP FileSystem. ( #1999 )
...
Contributed by Mikhail Pryakhin
2020-06-03 11:38:49 +01:00
c88bf7acc1
HADOOP-17056. shelldoc fails in hadoop-common. ( #2045 )
...
In the docker build image, skip GPG verification when downloading
Yetus tarball via yetus-wrapper.
(cherry picked from commit 9c290c08db
2020-06-03 18:03:03 +09:00
f65f64e8ae
YARN-10254. CapacityScheduler incorrect User Group Mapping after leaf queue change. Contributed by Gergely Pollak
2020-06-02 18:32:06 +02:00
910d88eeed
HADOOP-17052. NetUtils.connect() throws unchecked exception (UnresolvedAddressException) causing clients to abort ( #2036 )
...
Contributed by Dhiraj Hegde.
Signed-off-by: Mingliang Liu <liuml07@apache.org>
2020-06-01 10:50:22 -07:00
07b8963aa3
YARN-10259. Fix reservation logic in Multi Node Placement.
...
Reviewed by Wangda Tan.
(cherry picked from commit 6ce295b787
2020-06-01 18:24:38 +05:30
b0c9e4f1b5
HADOOP-16900. Very large files can be truncated when written through the S3A FileSystem.
...
Contributed by Mukund Thakur and Steve Loughran.
This patch ensures that writes to S3A fail when more than 10,000 blocks are
written. That upper bound still exists. To write massive files, make sure
that the value of fs.s3a.multipart.size is set to a size which is large
enough to upload the files in fewer than 10,000 blocks.
Change-Id: Icec604e2a357ffd38d7ae7bc3f887ff55f2d721a
2020-06-01 12:01:13 +01:00
7a323a45aa
YARN-6492. Generate queue metrics for each partition. Contributed by Manikandan R
...
(cherry picked from commit c30c23cb66
2020-05-29 09:25:08 -07:00
90f57965e9
HADOOP-7002. Wrong description of copyFromLocal and copyToLocal in documentation. Contributed by Andras Bokor.
...
(cherry picked from commit 19f26a020e
2020-05-29 14:49:40 +01:00
b803efbdce
HADOOP-14698. Make copyFromLocals -t option available for put as well. Contributed by Andras Bokor.
2020-05-29 11:44:48 +01:00
9b84a637b7
HADOOP-17055. Remove residual code of Ozone ( #2039 )
...
(cherry picked from commit d9838f2d42
2020-05-29 16:50:10 +09:00
6b7040a1cb
HDFS-13183. Addendum: Standby NameNode process getBlocks request to reduce Active load. Contributed by Xiaoqiao He.
2020-05-28 13:04:22 +05:30
586d9427e3
MAPREDUCE-7278. Speculative execution behavior is observed even when mapreduce.map.speculative and mapreduce.reduce.speculative are false
...
Contributed by Tarun Parimi.
(cherry picked from commit 10db97df1c
2020-05-28 13:36:21 +10:00
94358dbc3c
YARN-10108. FS-CS converter: nestedUserQueue with default rule results in invalid queue mapping. Contributed by Gergely Pollak
...
(cherry picked from commit ac4540dd8e
2020-05-27 13:07:17 +02:00
2c60bcb1c2
Revert "FS-CS converter: nestedUserQueue with default rule results in invalid queue mapping. Contributed by Gergely Pollak"
...
This reverts commit 9ce913a7bf
2020-05-27 13:07:01 +02:00
718c8a5868
HDFS-15369. Refactor method VolumeScanner#runLoop(). Contributed by Yang Yun.
2020-05-24 18:06:19 +05:30
c44d8b8bdf
HDFS-15363. BlockPlacementPolicyWithNodeGroup should validate if it is initialized by NetworkTopologyWithNodeGroup. Contributed by hemanthboyina.
...
(cherry picked from commit 4d22d1c58f
2020-05-23 17:33:39 +09:00
9ce913a7bf
FS-CS converter: nestedUserQueue with default rule results in invalid queue mapping. Contributed by Gergely Pollak
...
(cherry picked from commit ac4540dd8e
2020-05-22 14:48:53 +02:00
S O'Donnell
Mehakmeet Singh
Szilard Nemeth
Vinayakumar B
Uma Maheswara Rao G
Ayush Saxena
Abhishek Das
Stephen O'Donnell
Eric Yang
Takanobu Asanuma
Eric Badger
Eric E Payne
Steve Loughran
Mingliang Liu
Toshihiro Suzuki
Akira Ajisaka
Mehakmeet Singh
Mike
Dhiraj
Prabhu Joseph
Mukund Thakur
Jonathan Hung
Wanqiang Ji
Wilfred Spiegelenburg