Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,265 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

14265 Commits

Author SHA1 Message Date
Steve Loughran 717295e464 HADOOP-13972. ADLS to support per-store configuration. 
Contributed by Sharad Sonker.

(cherry picked from commit 050f5287b7)
(cherry picked from commit 07c7df4b26)
 2018-02-28 14:54:17 -08:00
Inigo Goiri 0d7a3f48bc HDFS-13199. RBF: Fix the hdfs router page missing label icon issue. Contributed by maobaolong. 
(cherry picked from commit d86f301d46)
 2018-02-27 17:43:07 -08:00
Yiqun Lin 13e86a7ab8 HDFS-13184. RBF: Improve the unit test TestRouterRPCClientRetries. Contributed by Yiqun Lin. 
(cherry picked from commit 1e85a995d1)
 2018-02-27 10:55:39 +08:00
Chris Douglas 6024b3a2ad Backport HADOOP-13514 (surefire upgrade) to branch-2 
(cherry picked from commit 762125b864)
 2018-02-26 16:32:19 -08:00
Wei Yan 39e1f963bb HDFS-13187. RBF: Fix Routers information shown in the web UI. Contributed by Inigo Goiri and Wei Yan. 
(cherry picked from commit 6ce9f79cc9)
 2018-02-26 15:36:21 -08:00
Kihwal Lee a6343ff808 HDFS-12070. Failed block recovery leaves files open indefinitely and at risk for data loss. Contributed by Kihwal Lee. 
(cherry picked from commit 4b43f2aa56)
 2018-02-26 10:59:47 -06:00
Arpit Agarwal 627a32375c HDFS-12781. After Datanode down, In Namenode UI Datanode tab is throwing warning message.. Contributed by Brahma Reddy Battula. 2018-02-24 15:13:02 -08:00
Xiao Chen 16e14b9969 HDFS-13164. File not closed if streamer fail with DSQuotaExceededException. 
(cherry picked from commit 51088d3233)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestQuota.java

(cherry picked from commit 80f7165374)
 2018-02-23 14:01:19 -08:00
Jason Lowe 76d4d5c759 YARN-7945. Java Doc error in UnmanagedAMPoolManager for branch-2. Contributed by Botong Huang 
(cherry picked from commit 7a6aa7bf99)
 2018-02-22 11:01:27 -06:00
Brahma Reddy Battula b32cc32ee2 HADOOP-14903. Add json-smart explicitly to pom.xml. Contributed by Brahma Reddy Battula and Ray Chiang 
(cherry picked from commit 7307a89ec1)
 2018-02-22 18:53:29 +05:30
Ray Chiang ed17197305 HADOOP-14799. Update nimbus-jose-jwt to 4.41.1. (rchiang) 
(cherry picked from commit 556812c179)
 2018-02-22 18:11:20 +05:30
Sunil G d9d8fa1d3a YARN-7947. Capacity Scheduler intra-queue preemption can NPE for non-schedulable apps. Contributed by Eric Payne. 
(cherry picked from commit bdd2a184d7)
 2018-02-21 15:32:54 +05:30
Yiqun Lin 14ddac3174 HDFS-13119. RBF: Manage unavailable clusters. Contributed by Yiqun Lin. 
(cherry picked from commit 8896d20b91)
 2018-02-20 13:03:55 -08:00
Xiao Chen c16b91fde0 HDFS-11187. Optimize disk access for last partial chunk checksum of Finalized replica. Contributed by Gabor Bota, Wei-Chiu Chuang. 
(cherry picked from commit 49ed7d7fc9)
 2018-02-19 19:19:02 -08:00
Jason Lowe c753df4b69 YARN-7813. Capacity Scheduler Intra-queue Preemption should be configurable for each queue. Contributed by Eric Payne 
(cherry picked from commit 51ad923f51)
 2018-02-19 16:07:19 -06:00
Jason Lowe 03a4d9a7b4 HADOOP-15206. BZip2 drops and duplicates records when input split size is small. Contributed by Aki Tanaka 
(cherry picked from commit 0898ff42e9)
 2018-02-16 15:08:43 -06:00
Eric Payne ebf183fa56 MAPREDUCE-7053: Timed out tasks can fail to produce thread dump. Contributed by Jason Lowe. 
(cherry picked from commit 904a6bf263)
 2018-02-16 08:51:12 -06:00
Kihwal Lee 01fc51fca7 HDFS-13112. Token expiration edits may cause log corruption or deadlock. Contributed by Daryn Sharp. 
(cherry picked from commit 47473952e5)
 2018-02-15 15:39:23 -06:00
Jason Lowe 1871e6844e MAPREDUCE-7052. TestFixedLengthInputFormat#testFormatCompressedIn is flaky. Contributed by Peter Bacsko 
(cherry picked from commit a53d62ab26)
 2018-02-15 15:31:50 -06:00
Kihwal Lee 961876e94f xattr api cleanup 
(cherry picked from commit da59acd8ca)
 2018-02-15 11:29:29 -06:00
Kihwal Lee 5b4725fb04 Revert "HADOOP-14799. Update nimbus-jose-jwt to 4.41.1. (rchiang)" 
This reverts commit 557d6f2814.
 2018-02-14 15:22:05 -06:00
Ray Chiang 557d6f2814 HADOOP-14799. Update nimbus-jose-jwt to 4.41.1. (rchiang) 
(cherry picked from commit 556812c179)
 2018-02-14 00:24:14 +05:30
Jason Lowe b8b67e4b8d MAPREDUCE-7048. Uber AM can crash due to unknown task in statusUpdate. Contributed by Peter Bacsko 
(cherry picked from commit 08feac4c83)
 2018-02-12 14:33:32 -06:00
Brahma Reddy Battula ed512645b2 HDFS-8693. Addendum patch to execute the command using UGI. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 35c17351ca)
(cherry picked from commit 24af0bc144)
 2018-02-12 22:27:21 +05:30
Arpit Agarwal 41d434d08e HDFS-10453. ReplicationMonitor thread could stuck for long time due to the race between replication and delete of same file in a large cluster.. Contributed by He Xiaoqiao. 2018-02-12 07:12:02 -08:00
Yiqun Lin 050aa531ba HDFS-13099. RBF: Use the ZooKeeper as the default State Store. Contributed by Yiqun Lin. 
(cherry picked from commit 543f3abbee)
 2018-02-09 14:03:13 +08:00
Xiaoyu Yao 77c6439c33 HDFS-13120. Snapshot diff could be corrupted after concat. Contributed by Xiaoyu Yao. 
(cherry picked from commit 8faf0b50d4)
 2018-02-08 10:20:55 -08:00
Brahma Reddy Battula 3487f2b722 HDFS-12935. Get ambiguous result for DFSAdmin command in HA mode when only one namenode is up. Contributed by Jianfei Jiang. 
(cherry picked from commit 9873eb63a7)
 2018-02-08 18:44:34 +05:30
Billie Rinaldi 52281fd893 Revert "YARN-6078. Containers stuck in Localizing state. Contributed by Billie Rinaldi." 
This reverts commit 9229fb3523.
 2018-02-07 09:57:46 -08:00
Kihwal Lee 39963d53df HADOOP-15212. Add independent secret manager method for logging expired tokens. Contributed by Daryn Sharp. 
(cherry picked from commit 2dd960de98)
 2018-02-06 15:38:32 -06:00
Billie Rinaldi 43ea9a42d7 YARN-7890. NPE during container relaunch. Contributed by Jason Lowe 
(cherry picked from commit a196ee9362)
 2018-02-06 12:06:25 -08:00
Jason Lowe e73e820d2a YARN-7849. TestMiniYarnClusterNodeUtilization#testUpdateNodeUtilization fails due to heartbeat sync error. Contributed by Botong Huang 
(cherry picked from commit 60656bceca)
 2018-02-05 17:29:49 -06:00
Haibo Chen e78a701687 YARN-6868. Add test scope to certain entries in hadoop-yarn-server-resourcemanager pom.xml. (Ray Chiang via Haibo Chen) 
(cherry picked from commit a20e7105ea)
(cherry picked from commit c25c082961)
 2018-02-05 10:51:39 -08:00
Yongjun Zhang 1ed6ae7b12 HDFS-13100. Handle IllegalArgumentException when GETSERVERDEFAULTS is not implemented in webhdfs. Contributed by Aaron T. Myers and Yongjun Zhang. 
(cherry picked from commit 4e9a59ce16)
 2018-02-02 23:45:28 -08:00
Yiqun Lin 024127e471 HDFS-13068. RBF: Add router admin option to manage safe mode. Contributed by Yiqun Lin. 
(cherry picked from commit 712e9381d2)
 2018-02-02 17:37:14 +08:00
Inigo Goiri 6769c78cdd HDFS-13043. RBF: Expose the state of the Routers in the federation. Contributed by Inigo Goiri. 
(cherry picked from commit 6ca7204ceb)
 2018-02-01 10:39:25 -08:00
Inigo Goiri 38c3e17117 HDFS-13044. RBF: Add a safe mode for the Router. Contributed by Inigo Goiri. 
(cherry picked from commit 961dcb693d)
 2018-01-31 09:17:40 -08:00
Inigo Goiri ffba476192 HDFS-13083. RBF: Fix doc error setting up client. Contributed by tartarus. 
(cherry picked from commit 5206b2c7ca)
 2018-01-30 18:28:54 -08:00
Sammi Chen e30abdaaae HADOOP-15027. AliyunOSS: Support multi-thread pre-read to improve sequential read from Hadoop to Aliyun OSS performance. (Contributed by Jinhu Wu) 
(cherry picked from commit 9195a6e302)
(cherry picked from commit 91184299c5)
(cherry picked from commit 2816bd1f43)
 2018-01-30 17:15:12 +08:00
Kihwal Lee a15df67f65 HDFS-12574. Add CryptoInputStream to WebHdfsFileSystem read call. Contributed by Rushabh S Shah. 
(cherry picked from commit eda786ea12)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestEncryptionZones.java
 2018-01-29 17:57:54 -06:00
Jinjiang Ling d70ca99594
YARN-7698. A misleading variable's name in ApplicationAttemptEventDispatcher 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 3400d0c535)
(cherry picked from commit 987a8972ac)
 2018-01-30 00:13:26 +09:00
Steve Loughran d9875b046f HADOOP-15039/HADOOP-15189. Move SemaphoredDelegatingExecutor to hadoop-common 
Contributed by Genmao Yu

(cherry picked from commit 312c571694)
 2018-01-27 21:00:25 -08:00
Jason Lowe d69df0b596 MAPREDUCE-7020. Task timeout in uber mode can crash AM. Contributed by Peter Bacsko 
(cherry picked from commit e3a1647477)
 2018-01-26 15:51:34 -06:00
Chris Douglas e062e2b08c MapFile.fix creates a wrong index file in case of block-compressed data file. Contributed by Grigori Rybkine 
(cherry picked from commit 91db424c4360d7556660e8c57ac9a266e6688e01)
(cherry picked from commit 7f35487788)
(cherry picked from commit 9bd439e2c5)
 2018-01-26 09:16:08 -08:00
Jason Lowe 98499bb098 YARN-7102. NM heartbeat stuck when responseId overflows MAX_INT. Contributed by Botong Huang 
(cherry picked from commit 4d7bbee0a1)
 2018-01-25 18:00:41 -06:00
Wei Yan 9951c2d04f HDFS-13049. RBF: Inconsistent Router OPTS config in branch-2 and branch-3. Contributed by Wei Yan. 
(cherry picked from commit cefe4b7bb6)
 2018-01-25 12:59:36 -08:00
Sunil G b354357733 YARN-7728. Expose container preemptions related information in Capacity Scheduler queue metrics. Contributed by Eric Payne. 
(cherry picked from commit 82cc6f6968)
 2018-01-25 15:55:45 +05:30
Yiqun Lin 44cfee9af4 HDFS-13042. RBF: Heartbeat Router State. Contributed by Inigo Goiri. 
(cherry picked from commit 7721fff744)
 2018-01-25 15:55:24 +08:00
Jianfei Jiang 1b5f0e819e
HDFS-13063. Fix the incorrect spelling in HDFSHighAvailabilityWithQJM.md 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit eb2dd08688)
(cherry picked from commit f4656e2150)
 2018-01-25 15:32:36 +09:00
Jian He cbaf6d242d YARN-7801. AmFilterInitializer should addFilter after fill all parameters. Contributed by Wangda Tan 2018-01-24 16:18:36 -08:00