Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,780 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

17780 Commits

Author SHA1 Message Date
Siyao Meng 6a7bf9f850 HDFS-14009. HttpFS: FileStatus#setSnapShotEnabledFlag throws InvocationTargetException when attribute set is emptySet. Contributed by Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2018-10-18 17:02:38 -07:00
drankye b34c650a41 HDFS-13668. FSPermissionChecker may throws AIOOE when check inode permission. Contributed by He Xiaoqiao. 
(cherry picked from commit 975d60685e)
 2018-10-18 08:31:12 -07:00
Inigo Goiri ea19a36132 HDFS-14000. RBF: Documentation should reflect right scripts for v3.0 and above. Contributed by CR Hota. 
(cherry picked from commit 7347fa2df3)
 2018-10-17 16:43:05 -07:00
Jason Lowe 0aee3a0c3f HADOOP-15859. ZStandardDecompressor.c mistakes a class for an instance. Contributed by Jason Lowe 
(cherry picked from commit 9abda83947)
 2018-10-17 14:44:39 -05:00
Steve Loughran ae42d59eba
HADOOP-15861. Move DelegationTokenIssuer to the right path. 
Contributed by Wei-Chiu Chuang.

(cherry picked from commit 41b3603b5b)
 2018-10-17 11:04:05 +01:00
Xiao Chen 99b447f612 MAPREDUCE-7132. JobSplitWriter prints unnecessary warnings if EC(RS10,4) is used. Contributed by Peter Bacsko. 
(cherry picked from commit 25f8fcb064)
 2018-10-16 10:24:33 -07:00
Xiao Chen b9f65d717c HADOOP-14445. Addendum: Use DelegationTokenIssuer to create KMS delegation tokens that can authenticate to all KMS instances. 
(cherry picked from commit b6fc72a025)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java
 2018-10-15 11:00:39 -07:00
Xiao Chen ff7ca472d2 HADOOP-14445. Use DelegationTokenIssuer to create KMS delegation tokens that can authenticate to all KMS instances. 
Contributed by Daryn Sharp, Xiao Chen, Rushabh S Shah.
 2018-10-12 12:08:38 -07:00
Inigo Goiri 53b522af6d HDFS-13802. RBF: Remove FSCK from Router Web UI. Contributed by Fei Hui. 
(cherry picked from commit f63ee083db)
 2018-10-12 10:12:07 -07:00
Steve Loughran 2abc4ab60a
HADOOP-15837. DynamoDB table Update can fail S3A FS init. 
Contributed by Steve Loughran.

(cherry picked from commit f1fbc56371)
 2018-10-12 09:49:02 +01:00
Wangda Tan b67dc5e1d1 YARN-7274. Ability to disable elasticity at leaf queue level. (Zian Chen via wangda) 
Change-Id: Ic8d43e297f0f5de788b562f7eff8106c5c35e8d2
(cherry picked from commit 74665e3a7d)
 2018-10-11 16:00:10 +00:00
Jonathan Eagles 2b6d8c1e29 HADOOP-15835. Reuse Object Mapper in KMSJSONWriter (jeagles) 2018-10-10 10:32:43 -05:00
Hrishikesh Gadre b170de8be5 HDFS-13926. ThreadLocal aggregations for FileSystem.Statistics are incorrect with striped reads. 
Contributed by Xiao Chen, Hrishikesh Gadre.

Signed-off-by: Xiao Chen <xiao@apache.org>
(cherry picked from commit 323b76bccf)
 2018-10-09 19:54:56 -07:00
Weiwei Yang af85ce6ae4 YARN-8858. CapacityScheduler should respect maximum node resource when per-queue maximum-allocation is being used. Contributed by Wangda Tan. 
(cherry picked from commit edce866489)
 2018-10-10 09:56:16 +08:00
Inigo Goiri 8a9f61be4e HDFS-13964. RBF: TestRouterWebHDFSContractAppend fails with No Active Namenode under nameservice. Contributed by Ayush Saxena. 
(cherry picked from commit 3bfff0f79e)
 2018-10-06 10:42:51 -07:00
Jason Lowe bbceae10b9 HADOOP-15820. ZStandardDecompressor native code sets an integer field as a long. Contributed by Jason Lowe 
(cherry picked from commit f13e231025)
 2018-10-05 09:29:41 -05:00
Takanobu Asanuma 4bad89802b HADOOP-15816. Upgrade Apache Zookeeper version due to security concerns. Contributed by Akira Ajisaka. 
(cherry picked from commit 241cbec2da)
 2018-10-05 15:11:31 +09:00
Yiqun Lin d993a1fc3d HADOOP-15817. Reuse Object Mapper in KMSJSONReader. Contributed by Jonathan Eagles. 
(cherry picked from commit 81f635f47f)
 2018-10-04 10:36:49 +08:00
Eric E Payne b412bb2240 YARN-8774. Memory leak when CapacityScheduler allocates from reserved container with non-default label. Contributed by Tao Yang. 
(cherry picked from commit 8598b498bc)
 2018-09-28 15:48:51 +00:00
Jason Lowe 17583e690a YARN-8804. resourceLimits may be wrongly calculated when leaf-queue is blocked in cluster with 3+ level queues. Contributed by Tao Yang 
(cherry picked from commit 6b988d821e)
 2018-09-26 16:54:29 -07:00
Brahma Reddy Battula b28dacf60c HDFS-13840. RBW Blocks which are having less GS should be added to Corrupt. Contributed by Brahma Reddy Battula 
(cherry picked from commit e0ff8e2c10)
 2018-09-26 23:55:02 +05:30
Sammi Chen 85e00477b8 HADOOP-15671. AliyunOSS: Support Assume Roles in AliyunOSS. Contributed by Jinhu Wu. 
(cherry picked from commit 2b635125fb)
(cherry picked from commit 5da3e83597)
 2018-09-25 19:53:03 +08:00
Siyao Meng 2449795b8e HDFS-13830. Backport HDFS-13141 to branch-3.0: WebHDFS: Add support for getting snasphottable directory list. Contributed by Siyao Meng, Lokesh Jain. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2018-09-21 12:41:03 -07:00
Steve Loughran fc6d851f70
HADOOP-15748. S3 listing inconsistency can raise NPE in globber. 
Contributed by Steve Loughran.

(cherry picked from commit 646874c326)
 2018-09-20 13:06:20 +01:00
Vinayakumar B d2655ea60c HADOOP-15736. Trash : Negative Value For Deletion Interval Leads To Abnormal Behaviour. Contributed by Ayush Saxena. 
(cherry picked from commit 7ad27e97f0)
(cherry picked from commit 2aa3854633)
 2018-09-20 09:34:37 +05:30
Chen Liang 78e0c0a1bb HADOOP-15726. Create utility to limit frequency of log statements. Contributed by Erik Krogen. 2018-09-19 14:51:23 -07:00
Arpit Agarwal ffef819ee8 HADOOP-15772. Remove the 'Path ... should be specified as a URI' warnings on startup. Contributed by Ayush Saxena. 
(cherry picked from commit 2eb597b151)
 2018-09-19 13:49:57 -07:00
Inigo Goiri 3fc3a5c01c HADOOP-15684. triggerActiveLogRoll stuck on dead name node, when ConnectTimeoutException happens. Contributed by Rong Tang. 
(cherry picked from commit 7f9a89e1b54a9712af50ffef70bed7cfb91ed34e)
 2018-09-19 12:59:42 -07:00
Inigo Goiri b93aeb43a4 HDFS-13908. TestDataNodeMultipleRegistrations is flaky. Contributed by Ayush Saxena. 
(cherry picked from commit 56e0d635e0)
 2018-09-19 09:31:58 -07:00
Weiwei Yang 4edcde055b Revert "YARN-8771. CapacityScheduler fails to unreserve when cluster resource contains empty resource type. Contributed by Tao Yang." 
This reverts commit f56e36935d.
 2018-09-19 19:47:06 +08:00
Weiwei Yang f56e36935d YARN-8771. CapacityScheduler fails to unreserve when cluster resource contains empty resource type. Contributed by Tao Yang. 
(cherry picked from commit 0712537e79)
 2018-09-19 19:41:07 +08:00
Wei-Chiu Chuang 823b1fceee HDFS-13868. WebHDFS: GETSNAPSHOTDIFF API NPE when param "snapshotname" is given but "oldsnapshotname" is not. Contributed by Pranay Singh. 
(cherry picked from commit 28ceb34a72)
(cherry picked from commit 7442b7941d)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
 2018-09-19 03:20:26 -07:00
Wei-Chiu Chuang 3a32e84d3e HDFS-13886. HttpFSFileSystem.getFileStatus() doesn't return "snapshot enabled" bit. Contributed by Siyao Meng. 
(cherry picked from commit 44857476fa)
(cherry picked from commit 595ce94570)
 2018-09-18 15:48:05 -07:00
Wei-Chiu Chuang 2ef3911f7c HDFS-13838. WebHdfsFileSystem.getFileStatus() won't return correct "snapshot enabled" status. Contributed by Siyao Meng. 
(cherry picked from commit a1de8cbac5)
(cherry picked from commit 829399a9f3)
 2018-09-18 15:48:04 -07:00
Jason Lowe f5ebb2ccf5 MAPREDUCE-7138. ThrottledContainerAllocator in MRAppBenchmark should implement RMHeartbeatHandler. Contributed by Oleksandr Shevchenko 
(cherry picked from commit 8382b860d4)
 2018-09-18 17:15:17 -05:00
Jason Lowe 4a2a7d37dd MAPREDUCE-7137. MRAppBenchmark.benchmark1() fails with NullPointerException. Contributed by Oleksandr Shevchenko 
(cherry picked from commit 34b2237e42)
 2018-09-18 17:13:00 -05:00
Jason Lowe f2eeba5c5b HADOOP-15755. StringUtils#createStartupShutdownMessage throws NPE when args is null. Contributed by Lokesh Jain and Dinesh Chitlangia 
(cherry picked from commit e71f61ecb8)

Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestStringUtils.java
 2018-09-18 16:03:42 -05:00
Eric E Payne 373c7a0ceb MAPREDUCE-3801: org.apache.hadoop.mapreduce.v2.app.TestRuntimeEstimators.testExponentialEstimator fails intermittently. Contributed by Jason Lowe 
(cherry picked from commit 2cf8927683)
 2018-09-18 20:42:35 +00:00
Sammi Chen 61847f4ea0 HADOOP-15759. AliyunOSS: Update oss-sdk version to 3.0.0. Contributed by Jinhu Wu. 
(cherry picked from commit e4fca6aae4)
(cherry picked from commit 53528d5ec5)
 2018-09-18 18:45:49 +08:00
Inigo Goiri f0537b5161 HDFS-13844. Fix the fmt_bytes function in the dfs-dust.js. Contributed by yanghuafeng. 
(cherry picked from commit a9c1c55cafc0338bb35d2991f217d4157463f1d3)
 2018-09-17 14:39:31 -07:00
Weiwei Yang 14b9e64d80 YARN-8782. Fix exception message in Resource.throwExceptionWhenArrayOutOfBound. Contributed by Gergely Pollak. 
(cherry picked from commit 9a265fa673)
 2018-09-17 22:27:59 +08:00
Weiwei Yang ef17728c58 YARN-8720. CapacityScheduler does not enforce max resource allocation check at queue level. Contributed by Tarun Parimi. 
(cherry picked from commit f1a893fdbc)
 2018-09-14 16:43:28 +08:00
Jason Lowe f918e4d894 MAPREDUCE-7133. History Server task attempts REST API returns invalid data. Contributed by Oleksandr Shevchenko 
(cherry picked from commit 2886024ac3)
 2018-09-13 14:47:37 -05:00
Inigo Goiri 1314dc2397 HDFS-13914. Fix DN UI logs link broken when https is enabled after HDFS-13902. Contributed by Jianfei Jiang. 
(cherry picked from commit e1b242a984)
 2018-09-13 09:18:11 -07:00
Weiwei Yang 4e105a4b9b YARN-8729. Node status updater thread could be lost after it is restarted. Contributed by Tao Yang. 
(cherry picked from commit 39c1ea1ed4)
 2018-09-13 23:18:16 +08:00
Brahma Reddy Battula 0c8aab690c HDFS-13902. Add JMX, conf and stacks menus to the datanode page. Contributed by fengchuang. 
(cherry picked from commit b2432d254c)
 2018-09-12 06:35:29 +05:30
Xiao Chen a5da4df73c HDFS-13051. Fix dead lock during async editlog rolling if edit queue is full. Contributed by Daryn Sharp. 
(cherry picked from commit 8e54da1511)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLogRace.java

(cherry picked from commit 2dd27c999b)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLogRace.java
 2018-09-10 23:00:41 -07:00
Eric E Payne 48dc8de28b YARN-8709: CS preemption monitor always fails since one under-served queue was deleted. Contributed by Tao Yang. 
(cherry picked from commit 987d8191ad)
 2018-09-10 20:18:14 +00:00
Giovanni Matteo Fumarola 37032a3aca HADOOP-15731. TestDistributedShell fails on Windows. Contributed by Botong Huang. 2018-09-07 16:37:15 -07:00
Inigo Goiri a9392bdb17 HDFS-13836. RBF: Handle mount table znode with null value. Contributed by yanghuafeng. 
(cherry picked from commit 527288ef89)
 2018-09-06 16:49:14 -07:00