Commit Graph

23954 Commits

Author SHA1 Message Date
Inigo Goiri 35010120fb YARN-9017. PlacementRule order is not maintained in CS. Contributed by Bilwa S T. 2020-05-06 13:22:54 -07:00
Inigo Goiri 130f89e068 HDFS-15332. Quota Space consumed was wrong in truncate with Snapshots. Contributed by hemanthboyina. 2020-05-06 13:18:32 -07:00
Ahmed Hussein 92e3ebb401 YARN-8959. TestContainerResizing fails randomly (Ahmed Hussein via jeagles)
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
2020-05-06 12:27:17 -05:00
Ayush Saxena 7fddf4855e HDFS-14283. DFSInputStream to prefer cached replica. Contributed by Lisheng Sun. 2020-05-06 16:55:04 +05:30
Wei-Chiu Chuang adecdb8b53
HDFS-15334. INodeAttributeProvider's new API checkPermissionWithContext not getting called in for authorization. (#1998)
Reviewed-by: Arpit Agarwal <arp@apache.org>
2020-05-05 16:04:54 -07:00
Wanqiang Ji e877118454
HDFS-15331. Remove invalid exclusions that minicluster dependency on HDFS (#1996). Contributed by Wanqiang Ji 2020-05-06 02:06:38 +05:30
Szilard Nemeth cb6399c109 YARN-10257. FS-CS converter: skip increment properties for mem/vcores and fix DRF check. Contributed by Peter Bacsko 2020-05-05 16:00:43 +02:00
Szilard Nemeth 0debe55d6c YARN-10160. Add auto queue creation related configs to RMWebService#CapacitySchedulerQueueInfo. Contributed by Prabhu Joseph 2020-05-05 13:11:28 +02:00
Uma Maheswara Rao G 9c8236d04d
HDFS-15305. Extend ViewFS and provide ViewFileSystemOverloadScheme implementation with scheme configurable. Contributed by Uma Maheswara Rao G. 2020-05-04 17:55:40 -07:00
Babneet Singh 1996351b0b
HDFS-15270. Account for *env == NULL in hdfsThreadDestructor (#1951)
OpenJ9 JVM properly terminates the thread before hdfsThreadDestructor is
invoked. JNIEnv is a mirror of J9VMThread in OpenJ9. After proper thread
termination, accessing JNIEnv in hdfsThreadDestructor (*env)->GetJavaVM,
yields a SIGSEGV since *env is NULL after thread cleanup is performed.

The main purpose of hdfsThreadDestructor is to invoke
DetachCurrentThread, which performs thread cleanup in OpenJ9. Since
OpenJ9 performs thread cleanup before hdfsThreadDestructor is invoked,
hdfsThreadDestructor should account for *env == NULL and skip
DetachCurrentThread.

Signed-off-by: Babneet Singh <sbabneet@ca.ibm.com>
2020-05-04 13:08:53 -07:00
Konstantin V Shvachko ebb878bab9 HDFS-15323. StandbyNode fails transition to active due to insufficient transaction tailing. Contributed by Konstantin V Shvachko. 2020-05-04 10:29:50 -07:00
Ahmed Hussein f5081a9a5d YARN-10256. Refactor TestContainerSchedulerQueuing.testContainerUpdateExecTypeGuaranteedToOpportunistic (Ahmed Hussein via jeagles)
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
2020-05-04 10:13:51 -05:00
Ayush Saxena 8dace8ff3a HDFS-15328. Use DFSConfigKeys MONITOR_CLASS_DEFAULT constant. Contributed by bianqi. 2020-05-03 23:59:17 +05:30
Ayush Saxena d6fc482a54 Erasure Coding: metrics xmitsInProgress become to negative. Contributed by maobaolong and Toshihiko Uchida. 2020-05-03 19:09:14 +05:30
Mingliang Liu 44de193bec
HDFS-15325. TestRefreshCallQueue is failing due to changed CallQueue constructor. Contributed by Fengnan Li 2020-05-02 20:56:29 -07:00
Akira Ajisaka f40dacdc2e
HDFS-15320. StringIndexOutOfBoundsException in HostRestrictingAuthorizationFilter (#1992)
Signed-off-by: Mingliang Liu <liuml07@apache.org>
2020-05-02 13:02:27 -07:00
Eric Yang ba66f3b454
SPNEGO TLS verification
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2020-05-02 21:50:13 +09:00
Surendra Singh Lilhore 257b04ce4e YARN-6553. Replace MockResourceManagerFacade with MockRM for AMRMProxy/Router tests. Contributed by Bilwa S T. 2020-05-02 12:43:20 +05:30
Shashikant Banerjee 82343790ee HDFS-15313. Ensure inodes in active filesytem are not deleted during snapshot delete. Contributed by Shashikant Banerjee. 2020-05-01 12:16:02 +05:30
Mingliang Liu 263c76b678
HADOOP-17011. Tolerate leading and trailing spaces in fs.defaultFS. Contributed by Ctest
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2020-04-30 14:15:28 -07:00
Ayush Saxena 6bdab3723e HADOOP-16957. NodeBase.normalize doesn't removing all trailing slashes. Contributed by Ayush Saxena. 2020-04-30 19:55:20 +05:30
Akira Ajisaka b5b45c53a4
HDFS-15309. Remove redundant String.valueOf method on ExtendedBlockId.java. Contributed by bianqi. 2020-04-30 15:46:46 +09:00
Inigo Goiri 31b2f687ef HDFS-15265. HttpFS: validate content-type in HttpFSUtils. Contributed by hemanthboyina. 2020-04-29 13:41:36 -07:00
Inigo Goiri d125d39108 YARN-6973. Adding RM Cluster Id in ApplicationReport. Contributed by Bilwa S T. 2020-04-29 11:23:58 -07:00
S O'Donnell 9ca6298a9a HDFS-15285. The same distance and load nodes don't shuffle when consider DataNode load. Contributed by Lisheng Sun. 2020-04-29 15:56:06 +01:00
Szilard Nemeth 410c605aec YARN-10247. Application priority queue ACLs are not respected. Contributed by Sunil G 2020-04-29 15:53:30 +02:00
Surendra Singh Lilhore db6252b6c3 HDFS-15210. EC : File write hanged when DN is shutdown by admin command. Contributed by Surendra Singh Lilhore. 2020-04-29 10:58:35 +05:30
Aryan Gupta 816042e62b
HDFS-15301. statfs function in hdfs-fuse not working. Contributed by Aryan Gupta.(#1980) 2020-04-29 09:19:59 +05:30
lfengnan 4202750040
HADOOP-17010. Add queue capacity support for FairCallQueue (#1977) 2020-04-28 16:14:55 -07:00
Szilard Nemeth ab36429559 YARN-10215. Endpoint for obtaining direct URL for the logs. Contributed by Andras Gyori 2020-04-28 18:53:28 +02:00
Akira Ajisaka 5e0eda5d5f
HDFS-15286. Concat on a same file deleting the file. Contributed by hemanthboyina. 2020-04-28 06:17:14 +09:00
Akira Ajisaka 6ef01646ba
YARN-9848. Revert YARN-4946. Contributed by Steven Rand. 2020-04-28 05:35:36 +09:00
Eric E Payne e2322e1117 MAPREDUCE-7277. IndexCache totalMemoryUsed differs from cache contents. Contributed by Jon Eagles (jeagles). 2020-04-27 19:10:00 +00:00
Mike 18d7dfbf35
HDFS-1820. FTPFileSystem attempts to close the outputstream even when it is not initialised. (#1952)
Contributed by Mikhail Pryakhin.
2020-04-27 14:43:51 +01:00
Prabhu Joseph 9224568b0e YARN-10156. Destroy Jersey Client in TimelineConnector.
Contributed by Tanu Ajmera.
2020-04-27 18:49:15 +05:30
Toshihiro Suzuki 62c26b91fd
HDFS-15298 Fix the findbugs warnings introduced in HDFS-15217 (#1979) 2020-04-27 17:20:47 +09:00
Szilard Nemeth f91e21ac10 YARN-10194. YARN RMWebServices /scheduler-conf/validate leaks ZK Connections. Contributed by Prabhu Joseph 2020-04-27 08:26:11 +02:00
Ayush Saxena 85516a8af7 HADOOP-17007. hadoop-cos fails to build. Contributed by Yang Yu. 2020-04-26 12:46:53 +05:30
Mingliang Liu 74d4a18735
HDFS-15297. TestNNHandlesBlockReportPerStorage::blockReport_02 fails intermittently. Contributed by Ayush Saxena 2020-04-25 15:43:02 -07:00
Dhiraj 453c9c9a67
HDFS-15281. Make sure ZKFC uses dfs.namenode.rpc-address to bind to host address (#1964)
Contributed by Dhiraj Hegde.

Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Inigo Goiri <inigoiri@apache.org>
2020-04-25 13:04:32 -07:00
Akira Ajisaka 81d8b71534
HADOOP-16054. Update Dockerfile to use Bionic (#1966) 2020-04-26 02:54:45 +09:00
Ayush Saxena ef9a6e775c HADOOP-16886. Add hadoop.http.idle_timeout.ms to core-default.xml. Contributed by Lisheng Sun. 2020-04-25 13:07:32 +05:30
Wei-Chiu Chuang 453771f170 HDFS-15295. AvailableSpaceBlockPlacementPolicy should use chooseRandomWithStorageTypeTwoTrial() for better performance. Contributed by Jinglun. 2020-04-24 11:53:24 -07:00
Szilard Nemeth d4874585f4 YARN-10237. Add isAbsoluteResource config for queue in scheduler response. Contributed by Prabhu Joseph 2020-04-24 17:04:47 +02:00
bilaharith 30ef8d0f1a
HADOOP-17002. ABFS: Adding config to determine if the account is HNS enabled or not
Contributed by Bilahari T H.
2020-04-23 17:46:18 -07:00
Akira Ajisaka 9827ff2961
YARN-10223. Remove jersey-test-framework-core dependency from yarn-server-common. (#1939) 2020-04-24 09:32:25 +09:00
Mehakmeet Singh 459eb2ad6d
HADOOP-16914 Adding Output Stream Counters in ABFS (#1899)
Contributed by Mehakmeet Singh.There
2020-04-23 13:35:39 +01:00
Mingliang Liu 5958af4dbc
HDFS-15276. Concat on INodeRefernce fails with illegal state exception. Contributed by hemanthboyina 2020-04-22 13:31:02 -07:00
Mingliang Liu af85971a58
HADOOP-17001. The suffix name of the unified compression class. Contributed by bianqi 2020-04-22 12:36:33 -07:00
Sneha Vijayarajan 3d69383c26
Hadoop 16857. ABFS: Stop CustomTokenProvider retry logic to depend on AbfsRestOp retry policy
Contributed by Sneha Vijayarajan
2020-04-21 21:39:48 -07:00