Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,516 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

17516 Commits

Author SHA1 Message Date
Jason Lowe 85c611ad7d YARN-7813. Capacity Scheduler Intra-queue Preemption should be configurable for each queue. Contributed by Eric Payne 2018-02-19 14:38:49 -06:00
Rohith Sharma K S 313c38b502 YARN-7937. Fix http method name in Cluster Application Timeout Update API example request. Contributed by Charan Hebri. 
(cherry picked from commit 87bdde6943)
 2018-02-18 14:06:29 +05:30
Jason Lowe f173e9cc04 HADOOP-15206. BZip2 drops and duplicates records when input split size is small. Contributed by Aki Tanaka 
(cherry picked from commit 0898ff42e9)
 2018-02-16 14:54:55 -06:00
Eric Payne cf933e17bc MAPREDUCE-7053: Timed out tasks can fail to produce thread dump. Contributed by Jason Lowe. 
(cherry picked from commit 82f029f7b5)
 2018-02-16 08:25:52 -06:00
Kihwal Lee 7b83fca453 HDFS-13112. Token expiration edits may cause log corruption or deadlock. Contributed by Daryn Sharp. 
(cherry picked from commit 47473952e5)
 2018-02-15 15:35:27 -06:00
Jason Lowe b9ce463fbd MAPREDUCE-7052. TestFixedLengthInputFormat#testFormatCompressedIn is flaky. Contributed by Peter Bacsko 
(cherry picked from commit a53d62ab26)
 2018-02-15 15:21:05 -06:00
Kihwal Lee 11ff6327af xattr api cleanup 
(cherry picked from commit da59acd8ca)
 2018-02-15 11:14:42 -06:00
Steve Loughran b302f728f8 HADOOP-13972. ADLS to support per-store configuration. 
Contributed by Sharad Sonker.

(cherry picked from commit 050f5287b7)
 2018-02-15 16:29:30 +00:00
Steve Loughran 1888b94806 HADOOP-15090. Add ADL troubleshooting doc. 
Contributed by Steve Loughran.

(cherry picked from commit 58a2120e8a)
 2018-02-15 14:26:18 +00:00
Anu Engineer 4b4157441f HADOOP-15204. Add Configuration API for parsing storage sizes. Contributed by Anu Engineer. 
(cherry picked from commit 8f66affd62)
 2018-02-14 13:30:33 -08:00
Eric Payne 55ded70716 Revert "YARN-7813: Capacity Scheduler Intra-queue Preemption should be configurable for each queue" 
This reverts commit 36c9dda07f.
 2018-02-14 14:38:31 -06:00
Xiao Chen 2f3415a4d6 Revert "HADOOP-12897. KerberosAuthenticator.authenticate to include URL on IO failures. Contributed by Ajay Kumar." 
This reverts commit 16b2cad8e8.
 2018-02-14 10:25:15 -08:00
Eric Payne 36c9dda07f YARN-7813: Capacity Scheduler Intra-queue Preemption should be configurable for each queue 2018-02-14 08:58:28 -06:00
Aaron Fabbri e6f6c0bfc2
HADOOP-15040 Upgrade AWS SDK to 1.11.271: NPE bug spams logs w/ Yarn Log Aggregation 2018-02-13 20:20:46 -08:00
Arpit Agarwal 16b2cad8e8 HADOOP-12897. KerberosAuthenticator.authenticate to include URL on IO failures. Contributed by Ajay Kumar. 2018-02-13 10:14:29 -08:00
Jason Lowe bc2d4ba507 MAPREDUCE-7048. Uber AM can crash due to unknown task in statusUpdate. Contributed by Peter Bacsko 
(cherry picked from commit 87e2570a14)
 2018-02-12 14:49:35 -06:00
Brahma Reddy Battula 055d5e0b85 HDFS-8693. Addendum patch to execute the command using UGI. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 35c17351ca)
 2018-02-12 22:17:35 +05:30
Arpit Agarwal 41cf2b8e7c HDFS-10453. ReplicationMonitor thread could stuck for long time due to the race between replication and delete of same file in a large cluster.. Contributed by He Xiaoqiao. 2018-02-12 07:16:48 -08:00
Yiqun Lin 38febc0ae2 HDFS-13099. RBF: Use the ZooKeeper as the default State Store. Contributed by Yiqun Lin. 
(cherry picked from commit 543f3abbee)
 2018-02-09 13:59:38 +08:00
Xiaoyu Yao 49ab09f00b HDFS-13120. Snapshot diff could be corrupted after concat. Contributed by Xiaoyu Yao. 
(cherry picked from commit 8faf0b50d4)
 2018-02-08 10:18:35 -08:00
Manoj Govindassamy bef769e436 HDFS-12985. NameNode crashes during restart after an OpenForWrite file present in the Snapshot got deleted. 
(cherry picked from commit 73ff09b79a)

(cherry picked from commit 821729905e)
 2018-02-07 19:49:27 -08:00
Yongjun Zhang 5105b60614 HDFS-13115. In getNumUnderConstructionBlocks(), ignore the inodeIds for which the inodes have been deleted. Contributed by Yongjun Zhang. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/LeaseManager.java

(cherry picked from commit 60d82ac61f)
 2018-02-07 16:49:26 -08:00
Anu Engineer 2d6ab20007 HDFS-12990. Change default NameNode RPC port back to 8020. Contributed by Xiao Chen. 
(cherry picked from commit 4304fcd5bd)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/ViewFs.md
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestGetConf.java

(cherry picked from commit 9264f10bb3)
 2018-02-07 11:16:53 -08:00
Brahma Reddy Battula c6349c3204 HDFS-12935. Get ambiguous result for DFSAdmin command in HA mode when only one namenode is up. Contributed by Jianfei Jiang. 
(cherry picked from commit 01bd6ab18f)
 2018-02-07 23:27:24 +05:30
Billie Rinaldi 0ec45f73a1 Revert "YARN-6078. Containers stuck in Localizing state. Contributed by Billie Rinaldi." 
This reverts commit ef212b8550.
 2018-02-07 07:51:56 -08:00
Xiao Chen 49de91446f HDFS-12933. Improve logging when DFSStripedOutputStream failed to write some blocks. Contributed by chencan. 
(cherry picked from commit e5c2fdddd6)
 2018-02-06 22:57:15 -08:00
Wei-Chiu Chuang 29a3b64ec0 HDFS-11187. Optimize disk access for last partial chunk checksum of Finalized replica. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 2021f4bdce)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/ReplicaBuilder.java
 2018-02-06 22:57:15 -08:00
Kihwal Lee d436c403b2 HADOOP-15212. Add independent secret manager method for logging expired tokens. Contributed by Daryn Sharp. 
(cherry picked from commit 2dd960de98)
 2018-02-06 15:34:52 -06:00
Billie Rinaldi 615bd15072 YARN-7890. NPE during container relaunch. Contributed by Jason Lowe 
(cherry picked from commit a196ee9362)
 2018-02-06 11:34:49 -08:00
Lei Xu 7b377f43d0 Preparing for 3.0.2 development 2018-02-06 10:50:37 -08:00
Jason Lowe 07f779483c YARN-7849. TestMiniYarnClusterNodeUtilization#testUpdateNodeUtilization fails due to heartbeat sync error. Contributed by Botong Huang 
(cherry picked from commit 60656bceca)
 2018-02-05 17:17:15 -06:00
Yongjun Zhang 35247cca87 HDFS-13100. Handle IllegalArgumentException when GETSERVERDEFAULTS is not implemented in webhdfs. Contributed by Aaron T. Myers and Yongjun Zhang. 
(cherry picked from commit 4e9a59ce16)

(cherry picked from commit 0524fac21e)
 2018-02-04 22:53:09 -08:00
Yiqun Lin 712e9381d2 HDFS-13068. RBF: Add router admin option to manage safe mode. Contributed by Yiqun Lin. 2018-02-02 17:34:12 +08:00
Akira Ajisaka 01661d4f90
HDFS-13048. LowRedundancyReplicatedBlocks metric can be negative 
(cherry picked from commit 4aef8bd2ef)
 2018-02-02 14:35:11 +09:00
Xiao Chen abdf8abf1a HADOOP-15197. Remove tomcat from the Hadoop-auth test bundle. 
(cherry picked from commit 09dd709d6e)
 2018-02-01 15:35:27 -08:00
Hanisha Koneru 992d7b5739 HDFS-13062. Provide support for JN to use separate journal disk per namespace. Contributed by Bharat Viswanadham. 
(cherry picked from commit dd50f53997)
 2018-02-01 12:32:47 -08:00
Xiaoyu Yao 784b454122 HDFS-12997. Move logging to slf4j in BlockPoolSliceStorage and Storage. Contributed by Ajay Kumar. 
(cherry picked from commit b3ae11d597)
 2018-02-01 12:21:29 -08:00
Inigo Goiri 7491aaeb8f HDFS-13043. RBF: Expose the state of the Routers in the federation. Contributed by Inigo Goiri. 
(cherry picked from commit 6ca7204ceb)
 2018-02-01 10:37:56 -08:00
Jason Lowe 84297f69d8 Revert "YARN-7677. Docker image cannot set HADOOP_CONF_DIR. Contributed by Jim Brennan" 
This reverts commit 874bdbc401.
 2018-02-01 12:17:23 -06:00
Xiao Chen 4010262ae0 HDFS-12897. getErasureCodingPolicy should handle .snapshot dir better. Contributed by LiXin Ge. 
(cherry picked from commit ae2177d296)
 2018-01-31 22:50:52 -08:00
Xiaoyu Yao e260a0af24 HDFS-13060. Adding a BlacklistBasedTrustedChannelResolver for TrustedChannelResolver. Contributed by Ajay Kumar. 
(cherry picked from commit b3ac6f7ce3f5e014af7ad6bf882431a97716e020)
 2018-01-31 22:40:13 -08:00
Eric Payne 1d53c9d3a9 MAPREDUCE-7033: Map outputs implicitly rely on permissive umask for shuffle. Contributed by Jason Lowe 
(cherry picked from commit 5a725bb886)
 2018-01-31 16:01:46 -06:00
Hanisha Koneru b71d775f57 HDFS-13092. Reduce verbosity for ThrottledAsyncChecker#schedule. Contributed by Mukul Kumar Singh. 
(cherry picked from commit 3ce2190b58)
 2018-01-31 11:07:39 -08:00
Xiaoyu Yao 02790eb633 HDFS-13061. SaslDataTransferClient#checkTrustAndSend should not trust a partially trusted channel. Contributed by Ajay Kumar. 
(cherry picked from commit f4bcae10d306ef2b0083e1de17253ec732b5b5c1)
 2018-01-31 10:47:46 -08:00
Jason Lowe 874bdbc401 YARN-7677. Docker image cannot set HADOOP_CONF_DIR. Contributed by Jim Brennan 
(cherry picked from commit 12eaae383a)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/DockerLinuxContainerRuntime.java
 2018-01-31 11:45:57 -06:00
Inigo Goiri 961dcb693d HDFS-13044. RBF: Add a safe mode for the Router. Contributed by Inigo Goiri. 2018-01-31 09:08:47 -08:00
Inigo Goiri 0ce4851ee2 HDFS-13083. RBF: Fix doc error setting up client. Contributed by tartarus. 
(cherry picked from commit 5206b2c7ca)
 2018-01-30 18:27:51 -08:00
Weiwei Yang 7f62d2a473 HDFS-12528. Add an option to not disable short-circuit reads on failures. Contributed by Xiao Chen. 2018-01-31 07:35:54 +08:00
Sammi Chen 91184299c5 HADOOP-15027. AliyunOSS: Support multi-thread pre-read to improve sequential read from Hadoop to Aliyun OSS performance. (Contributed by Jinhu Wu) 
(cherry picked from commit 9195a6e302)
 2018-01-30 15:21:27 +08:00
Kihwal Lee 673200ac1e HDFS-12574. Add CryptoInputStream to WebHdfsFileSystem read call. Contributed by Rushabh S Shah 
(cherry picked from commit fde95d463c)
 2018-01-29 17:25:30 -06:00