Commit Graph

24168 Commits

Author SHA1 Message Date
Ayush Saxena 8525874561 HDFS-15211. EC: File write hangs during close in case of Exception during updatePipeline. Contributed by Ayush Saxena.
*Added missed test file.
2020-06-09 18:53:33 +05:30
Eric Badger fbb8775430 Revert "MAPREDUCE-7277. IndexCache totalMemoryUsed differs from cache contents. Contributed by Jon Eagles (jeagles)."
This reverts commit e2322e1117.
2020-06-08 20:35:27 +00:00
Mingliang Liu 0c25131ca4
HADOOP-17047. TODO comment exist in trunk while related issue HADOOP-6223 is already fixed. Contributed by Rungroj Maipradit 2020-06-08 11:28:56 -07:00
Mingliang Liu 9f242c215e
HADOOP-17059. ArrayIndexOfboundsException in ViewFileSystem#listStatus. Contributed by hemanthboyina 2020-06-08 10:11:30 -07:00
Ayush Saxena a8610c15c4
HDFS-15396. Fix TestViewFileSystemOverloadSchemeHdfsFileSystemContract#testListStatusRootDir. Contributed by Ayush Saxena. 2020-06-07 13:29:10 -07:00
Uma Maheswara Rao G 3ca15292c5
HDFS-15394. Add all available fs.viewfs.overload.scheme.target.<scheme>.impl classes in core-default.xml bydefault. Contributed by Uma Maheswara Rao G. 2020-06-06 08:11:57 -07:00
Ayush Saxena cc671b16f7
HDFS-15389. DFSAdmin should close filesystem and dfsadmin -setBalancerBandwidth should work with ViewFSOverloadScheme. Contributed by Ayush Saxena 2020-06-05 22:19:38 -07:00
Abhishek Das e7dd02768b
HADOOP-17029. Return correct permission and owner for listing on internal directories in ViewFs. Contributed by Abhishek Das. 2020-06-05 14:56:51 -07:00
Uma Maheswara Rao G 76fa0222f0
HDFS-15330. Document the ViewFSOverloadScheme details in ViewFS guide. Contributed by Uma Maheswara Rao G. 2020-06-05 10:58:21 -07:00
Ayush Saxena 2326123705 HDFS-15359. EC: Allow closing a file with committed blocks. Contributed by Ayush Saxena. 2020-06-05 19:15:05 +05:30
Szilard Nemeth 8abff5151a YARN-10292. FS-CS converter: add an option to enable asynchronous scheduling in CapacityScheduler. Contributed by Benjamin Teke 2020-06-05 15:00:39 +02:00
Toshihiro Suzuki 545a0a147c
HDFS-15386 ReplicaNotFoundException keeps happening in DN after removing multiple DN's data directories (#2052)
Contributed by Toshihiro Suzuki.
2020-06-05 11:11:49 +01:00
Szilard Nemeth 8b146c17b3 YARN-10274. Merge QueueMapping and QueueMappingEntity. Contributed by Gergely Pollak 2020-06-05 11:38:32 +02:00
Szilard Nemeth a4835db95a YARN-10296. Make ContainerPBImpl#getId/setId synchronized. Contributed by Benjamin Teke 2020-06-05 10:11:40 +02:00
Akira Ajisaka 5157118bd7
HADOOP-17056. Addendum patch: Fix typo 2020-06-04 16:34:46 +09:00
Akira Ajisaka 704409d53b
HADOOP-17062. Fix shelldocs path in Jenkinsfile (#2049) 2020-06-04 06:04:26 +09:00
Steve Loughran 40d63e02f0
HADOOP-16568. S3A FullCredentialsTokenBinding fails if local credentials are unset. (#1441)
Contributed by Steve Loughran.

Move the loading to deployUnbonded (where they are required) and add a safety check when a new DT is requested
2020-06-03 17:07:00 +01:00
Mike 97c98ce531
HADOOP-14566. Add seek support for SFTP FileSystem. (#1999)
Contributed by Mikhail Pryakhin
2020-06-03 11:37:40 +01:00
Akira Ajisaka 9c290c08db
HADOOP-17056. shelldoc fails in hadoop-common. (#2045)
In the docker build image, skip GPG verification when downloading
Yetus tarball via yetus-wrapper.
2020-06-03 18:01:37 +09:00
Ayush Saxena e8cb2ae409 HDFS-11041. Unable to unregister FsDatasetState MBean if DataNode is shutdown twice. Contributed by Wei-Chiu Chuang. 2020-06-03 12:47:15 +05:30
Ayush Saxena f6453244ab HDFS-14960. TestBalancerWithNodeGroup should not succeed with DFSNetworkTopology. Contributed by Jim Brennan. 2020-06-03 12:16:36 +05:30
Xiaoyu Yao 6288e15118 HADOOP-16828. Zookeeper Delegation Token Manager fetch sequence number by batch. Contributed by Fengnan Li. 2020-06-02 11:53:32 -07:00
Uma Maheswara Rao G ed83c865dd
HDFS-15321. Make DFSAdmin tool to work with ViewFileSystemOverloadScheme. Contributed by Uma Maheswara Rao G. 2020-06-02 11:09:26 -07:00
Mehakmeet Singh 7f486f0258
HADOOP-17016. Adding Common Counters in ABFS (#1991).
Contributed by: Mehakmeet Singh.

Change-Id: Ib84e7a42f28e064df4c6204fcce33e573360bf42
2020-06-02 18:31:35 +01:00
Szilard Nemeth b5efdea4fd YARN-10254. CapacityScheduler incorrect User Group Mapping after leaf queue change. Contributed by Gergely Pollak 2020-06-02 18:29:15 +02:00
Szilard Nemeth aa6d13455b YARN-10284. Add lazy initialization of LogAggregationFileControllerFactory in LogServlet. Contributed by Adam Antal 2020-06-02 15:53:48 +02:00
Szilard Nemeth e0a0741ac8 YARN-10286. PendingContainers bugs in the scheduler outputs. Contributed by Andras Gyori 2020-06-02 15:42:39 +02:00
Dhiraj 9fe4c37c25
HADOOP-17052. NetUtils.connect() throws unchecked exception (UnresolvedAddressException) causing clients to abort (#2036)
Contributed by Dhiraj Hegde.

Signed-off-by: Mingliang Liu <liuml07@apache.org>
2020-06-01 10:49:17 -07:00
Ayush Saxena ae13a5ccbe HDFS-10792. RedundantEditLogInputStream should log caught exceptions. Contributed by Wei-Chiu Chuang. 2020-05-31 17:10:59 +05:30
S O'Donnell 19f26a020e HADOOP-7002. Wrong description of copyFromLocal and copyToLocal in documentation. Contributed by Andras Bokor. 2020-05-29 14:36:56 +01:00
S O'Donnell d9e8046a1a HADOOP-14698. Make copyFromLocals -t option available for put as well. Contributed by Andras Bokor. 2020-05-29 10:32:37 +01:00
Wanqiang Ji d9838f2d42
HADOOP-17055. Remove residual code of Ozone (#2039) 2020-05-29 16:49:18 +09:00
Karthik Amarnath b2200a33a6
HDFS-15168: ABFS enhancement to translate AAD to Linux identities. (#1978) 2020-05-28 19:00:23 -07:00
Ayush Saxena a838d871a7 HDFS-15368. TestBalancerWithHANameNodes#testBalancerWithObserver failed occasionally. Contributed by Xiaoqiao He. 2020-05-28 20:52:58 +05:30
Ayush Saxena 9b38be43c6 HDFS-13183. Addendum: Standby NameNode process getBlocks request to reduce Active load. Contributed by Xiaoqiao He. 2020-05-28 13:00:04 +05:30
Wilfred Spiegelenburg 10db97df1c
MAPREDUCE-7278. Speculative execution behavior is observed even when mapreduce.map.speculative and mapreduce.reduce.speculative are false
Contributed by Tarun Parimi.
2020-05-28 13:23:01 +10:00
Sneha Vijayarajan 4c5cd751e3
HADOOP-17053. ABFS: Fix Account-specific OAuth config setting parsing
Contributed by Sneha Vijayarajan
2020-05-27 13:56:09 -07:00
Sneha Vijayarajan 53b993e604
HADOOP-16852: Report read-ahead error back
Contributed by Sneha Vijayarajan
2020-05-27 13:51:42 -07:00
Inigo Goiri 2148a8fe64 HDFS-15362. FileWithSnapshotFeature#updateQuotaAndCollectBlocks should collect all distinct blocks. Contributed by hemanthboyina. 2020-05-27 11:06:13 -07:00
Szilard Nemeth 593af878c0 YARN-10108. FS-CS converter: nestedUserQueue with default rule results in invalid queue mapping. Contributed by Gergely Pollak 2020-05-27 13:05:52 +02:00
Szilard Nemeth fb030f3239 Revert "FS-CS converter: nestedUserQueue with default rule results in invalid queue mapping. Contributed by Gergely Pollak"
This reverts commit ac4540dd8e.
Reverting because of missing Jira-ID in the commit message.
2020-05-27 13:04:48 +02:00
Jonathan Hung c30c23cb66 YARN-6492. Generate queue metrics for each partition. Contributed by Manikandan R 2020-05-26 16:29:42 -07:00
Sneha Vijayarajan 37b1b4799d
HADOOP-17054. ABFS: Fix test AbfsClient authentication instance
Contributed by Sneha Vijayarajan
2020-05-26 15:26:28 -07:00
Ayush Saxena 6c9f75cf16 HDFS-15373. Fix number of threads in IPCLoggerChannel#createParallelExecutor. Contributed by Ayush Saxena. 2020-05-26 16:26:57 +05:30
Ayush Saxena f43a152b97 HDFS-15369. Refactor method VolumeScanner#runLoop(). Contributed by Yang Yun. 2020-05-24 18:03:41 +05:30
Ayush Saxena f4901d0778 HDFS-15355. Make the default block storage policy ID configurable. Contributed by Yang Yun. 2020-05-24 17:59:16 +05:30
Inigo Goiri 6e04b00df1 HDFS-12288. Fix DataNode's xceiver count calculation. Contributed by Lisheng Sun. 2020-05-23 09:58:19 -07:00
Ayush Saxena e0ae232f66 HDFS-15093. RENAME.TO_TRASH is ignored When RENAME.OVERWRITE is specified. Contributed by Ayush Saxena. 2020-05-23 18:38:32 +05:30
Ayush Saxena f99fcb26ab HDFS-15288. Add Available Space Rack Fault Tolerant BPP. Contributed by Ayush Saxena. 2020-05-23 18:29:31 +05:30
Takanobu Asanuma 4d22d1c58f HDFS-15363. BlockPlacementPolicyWithNodeGroup should validate if it is initialized by NetworkTopologyWithNodeGroup. Contributed by hemanthboyina. 2020-05-23 17:28:38 +09:00