Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,274 Commits 326 Branches 371 Tags
Commit Graph

14274 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Inigo Goiri
44abf77d9b HDFS-13232. RBF: ConnectionManager's cleanup task will compare each pool's own active conns with its total conns. Contributed by Chao Sun. 
(cherry picked from commit 0c2b969e0161a068bf9ae013c4b95508dfb90a8a)
 2018-03-08 09:34:07 -08:00
Yiqun Lin
e3c80d7391 HDFS-13214. RBF: Complete document of Router configuration. Contributed by Yiqun Lin. 
(cherry picked from commit 253ef33f80afa38614e018c059777f6a81edf2cb)
 2018-03-07 16:09:23 +08:00
Xiaoyu Yao
321766bcc8 HDFS-13109. Support fully qualified hdfs path in EZ commands. Contributed by Hanisha Koneru. 
(cherry picked from commit edf9445708ffb7a9e59cb933e049b540f99add1e)
 2018-03-06 18:33:11 -08:00
Allen Wittenauer
ef54f78530
HADOOP-15279. increase maven heap size recommendations 
Signed-off-by: Jason Lowe <jlowe@apache.org>
Signed-off-by: Chris Douglas <cdouglas@apache.org>
(cherry picked from commit d4cc50f62279dc4d607cd05c16de480a76f05880)
(cherry picked from commit dbd69cb2ce2a060e36280b222cac05fc05e065d1)
 2018-03-06 13:59:01 -08:00
Sen Zhao
2ee654acfe
YARN-7736. Fix itemization in YARN federation document 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 245751ffdc4229715a0c031f57f20748ed16d8a6)
(cherry picked from commit bda2a9c2a0ed77a000a31b9531b86cad1d0383ff)
 2018-03-05 11:55:31 -08:00
Akira Ajisaka
9c7be95470
HADOOP-15283. Upgrade from findbugs 3.0.1 to spotbugs 3.1.2 in branch-2 to fix docker image build. 
(cherry picked from commit 8ad3ec6aabd265cae01242100729e3d02d794256)
 2018-03-04 07:03:24 -08:00
Gergely Novák
bea5dff7b3
MAPREDUCE-7023. TestHadoopArchiveLogs.testCheckFilesAndSeedApps fails on rerun 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit e99a1e77671fa0e74a652bc2221bbe01d1a579d3)
(cherry picked from commit 278676508f8a56f3582aaabf7b60099864dc7587)
 2018-03-02 16:22:49 -08:00
Jason Lowe
2d313c8264 YARN-7511. NPE in ContainerLocalizer when localization failed for running container. Contributed by Tao Yang 
(cherry picked from commit 83798f15f8602ef580a7885876de114b2425da89)
 2018-03-02 10:17:31 -06:00
Nanda kumar
2599b97697
HADOOP-15275. Incorrect javadoc for return type of RetryPolicy#shouldRetry 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 55d04a6db10d0e58cfe0ff82fa646bb6ba57aaac)
(cherry picked from commit 45ec3e18e33bd93a26b24a33c95d13afbe0a313d)
 2018-03-01 19:05:22 +09:00
Steve Loughran
717295e464 HADOOP-13972. ADLS to support per-store configuration. 
Contributed by Sharad Sonker.

(cherry picked from commit 050f5287b79324b7f6231b879c0bfc608203b980)
(cherry picked from commit 07c7df4b261f23e567d58936b78aee4ab73cb5fc)
 2018-02-28 14:54:17 -08:00
Inigo Goiri
0d7a3f48bc HDFS-13199. RBF: Fix the hdfs router page missing label icon issue. Contributed by maobaolong. 
(cherry picked from commit d86f301d464683f8d392dad50e83f50d823e862e)
 2018-02-27 17:43:07 -08:00
Yiqun Lin
13e86a7ab8 HDFS-13184. RBF: Improve the unit test TestRouterRPCClientRetries. Contributed by Yiqun Lin. 
(cherry picked from commit 1e85a995d1c7fe3dbf4b36a481feb5fdeeb6015b)
 2018-02-27 10:55:39 +08:00
Chris Douglas
6024b3a2ad Backport HADOOP-13514 (surefire upgrade) to branch-2 
(cherry picked from commit 762125b864ab812512bad9a59344ca79af7f43ac)
 2018-02-26 16:32:19 -08:00
Wei Yan
39e1f963bb HDFS-13187. RBF: Fix Routers information shown in the web UI. Contributed by Inigo Goiri and Wei Yan. 
(cherry picked from commit 6ce9f79cc9b2107e5953a39d05b22966aff0b7ff)
 2018-02-26 15:36:21 -08:00
Kihwal Lee
a6343ff808 HDFS-12070. Failed block recovery leaves files open indefinitely and at risk for data loss. Contributed by Kihwal Lee. 
(cherry picked from commit 4b43f2aa566322317a7f3163027bf5fd0a247207)
 2018-02-26 10:59:47 -06:00
Arpit Agarwal
627a32375c HDFS-12781. After Datanode down, In Namenode UI Datanode tab is throwing warning message.. Contributed by Brahma Reddy Battula. 2018-02-24 15:13:02 -08:00
Xiao Chen
16e14b9969 HDFS-13164. File not closed if streamer fail with DSQuotaExceededException. 
(cherry picked from commit 51088d323359587dca7831f74c9d065c2fccc60d)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestQuota.java

(cherry picked from commit 80f716537425254362c14c7cf81e845f6dea8ffe)
 2018-02-23 14:01:19 -08:00
Jason Lowe
76d4d5c759 YARN-7945. Java Doc error in UnmanagedAMPoolManager for branch-2. Contributed by Botong Huang 
(cherry picked from commit 7a6aa7bf99d667e277ef24eba0ca1bde648aa01e)
 2018-02-22 11:01:27 -06:00
Brahma Reddy Battula
b32cc32ee2 HADOOP-14903. Add json-smart explicitly to pom.xml. Contributed by Brahma Reddy Battula and Ray Chiang 
(cherry picked from commit 7307a89ec14d6e9812486f8d78ff4e8f4f597c31)
 2018-02-22 18:53:29 +05:30
Ray Chiang
ed17197305 HADOOP-14799. Update nimbus-jose-jwt to 4.41.1. (rchiang) 
(cherry picked from commit 556812c179aa094c21acf610439a8d69fe6420ab)
 2018-02-22 18:11:20 +05:30
Sunil G
d9d8fa1d3a YARN-7947. Capacity Scheduler intra-queue preemption can NPE for non-schedulable apps. Contributed by Eric Payne. 
(cherry picked from commit bdd2a184d78379d99c802a43ebec7d2cef0bbaf7)
 2018-02-21 15:32:54 +05:30
Yiqun Lin
14ddac3174 HDFS-13119. RBF: Manage unavailable clusters. Contributed by Yiqun Lin. 
(cherry picked from commit 8896d20b91520053a6bbfb680adb345cd24f4142)
 2018-02-20 13:03:55 -08:00
Xiao Chen
c16b91fde0 HDFS-11187. Optimize disk access for last partial chunk checksum of Finalized replica. Contributed by Gabor Bota, Wei-Chiu Chuang. 
(cherry picked from commit 49ed7d7fc9cd9ea280460cc11738df81c492be68)
 2018-02-19 19:19:02 -08:00
Jason Lowe
c753df4b69 YARN-7813. Capacity Scheduler Intra-queue Preemption should be configurable for each queue. Contributed by Eric Payne 
(cherry picked from commit 51ad923f51db30609d738479e8fcbc4f5ea98461)
 2018-02-19 16:07:19 -06:00
Jason Lowe
03a4d9a7b4 HADOOP-15206. BZip2 drops and duplicates records when input split size is small. Contributed by Aki Tanaka 
(cherry picked from commit 0898ff42e9e5c53f2fce7ccdeb4e1cd7d0f123b3)
 2018-02-16 15:08:43 -06:00
Eric Payne
ebf183fa56 MAPREDUCE-7053: Timed out tasks can fail to produce thread dump. Contributed by Jason Lowe. 
(cherry picked from commit 904a6bf26311750d09b2d8e662fe97b30cdb95b4)
 2018-02-16 08:51:12 -06:00
Kihwal Lee
01fc51fca7 HDFS-13112. Token expiration edits may cause log corruption or deadlock. Contributed by Daryn Sharp. 
(cherry picked from commit 47473952e56b0380147d42f4110ad03c2276c961)
 2018-02-15 15:39:23 -06:00
Jason Lowe
1871e6844e MAPREDUCE-7052. TestFixedLengthInputFormat#testFormatCompressedIn is flaky. Contributed by Peter Bacsko 
(cherry picked from commit a53d62ab26e170a0338f93e228718da52e9196e4)
 2018-02-15 15:31:50 -06:00
Kihwal Lee
961876e94f xattr api cleanup 
(cherry picked from commit da59acd8ca9ab5b49b988ffca64e8cce91c5f741)
 2018-02-15 11:29:29 -06:00
Kihwal Lee
5b4725fb04 Revert "HADOOP-14799. Update nimbus-jose-jwt to 4.41.1. (rchiang)" 
This reverts commit 557d6f28145e46c8ff21b291362808e88010bbde.
 2018-02-14 15:22:05 -06:00
Ray Chiang
557d6f2814 HADOOP-14799. Update nimbus-jose-jwt to 4.41.1. (rchiang) 
(cherry picked from commit 556812c179aa094c21acf610439a8d69fe6420ab)
 2018-02-14 00:24:14 +05:30
Jason Lowe
b8b67e4b8d MAPREDUCE-7048. Uber AM can crash due to unknown task in statusUpdate. Contributed by Peter Bacsko 
(cherry picked from commit 08feac4c83939f60289f8540d72ffcabb7852e2a)
 2018-02-12 14:33:32 -06:00
Brahma Reddy Battula
ed512645b2 HDFS-8693. Addendum patch to execute the command using UGI. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 35c17351cab645dcc72e0d2ae1608507aa787ffb)
(cherry picked from commit 24af0bc144b7e89fc2ea8d2f49b2c8eeb5a2b725)
 2018-02-12 22:27:21 +05:30
Arpit Agarwal
41d434d08e HDFS-10453. ReplicationMonitor thread could stuck for long time due to the race between replication and delete of same file in a large cluster.. Contributed by He Xiaoqiao. 2018-02-12 07:12:02 -08:00
Yiqun Lin
050aa531ba HDFS-13099. RBF: Use the ZooKeeper as the default State Store. Contributed by Yiqun Lin. 
(cherry picked from commit 543f3abbee79d7ec70353f0cdda6397ee001324e)
 2018-02-09 14:03:13 +08:00
Xiaoyu Yao
77c6439c33 HDFS-13120. Snapshot diff could be corrupted after concat. Contributed by Xiaoyu Yao. 
(cherry picked from commit 8faf0b50d435039f69ea35f592856ca04d378809)
 2018-02-08 10:20:55 -08:00
Brahma Reddy Battula
3487f2b722 HDFS-12935. Get ambiguous result for DFSAdmin command in HA mode when only one namenode is up. Contributed by Jianfei Jiang. 
(cherry picked from commit 9873eb63a7525301ab601a0ae65f7e615d1a6bce)
 2018-02-08 18:44:34 +05:30
Billie Rinaldi
52281fd893 Revert "YARN-6078. Containers stuck in Localizing state. Contributed by Billie Rinaldi." 
This reverts commit 9229fb3523fbf1e98bd85a98530e3df9079d32b2.
 2018-02-07 09:57:46 -08:00
Kihwal Lee
39963d53df HADOOP-15212. Add independent secret manager method for logging expired tokens. Contributed by Daryn Sharp. 
(cherry picked from commit 2dd960de983a30bf0d9ee957bdb09f825f9d40a3)
 2018-02-06 15:38:32 -06:00
Billie Rinaldi
43ea9a42d7 YARN-7890. NPE during container relaunch. Contributed by Jason Lowe 
(cherry picked from commit a196ee9362a1b35e5de20ee519f7c544ab1588e1)
 2018-02-06 12:06:25 -08:00
Jason Lowe
e73e820d2a YARN-7849. TestMiniYarnClusterNodeUtilization#testUpdateNodeUtilization fails due to heartbeat sync error. Contributed by Botong Huang 
(cherry picked from commit 60656bcecadd80e28c81bc943b44abf13d20abae)
 2018-02-05 17:29:49 -06:00
Haibo Chen
e78a701687 YARN-6868. Add test scope to certain entries in hadoop-yarn-server-resourcemanager pom.xml. (Ray Chiang via Haibo Chen) 
(cherry picked from commit a20e7105ea9d4e38d7f8f9fd48035e342bb22f1c)
(cherry picked from commit c25c082961a837b94016dcf0250bdc6d5f54e915)
 2018-02-05 10:51:39 -08:00
Yongjun Zhang
1ed6ae7b12 HDFS-13100. Handle IllegalArgumentException when GETSERVERDEFAULTS is not implemented in webhdfs. Contributed by Aaron T. Myers and Yongjun Zhang. 
(cherry picked from commit 4e9a59ce16e81b4bd6fae443a997ef24d588a6e8)
 2018-02-02 23:45:28 -08:00
Yiqun Lin
024127e471 HDFS-13068. RBF: Add router admin option to manage safe mode. Contributed by Yiqun Lin. 
(cherry picked from commit 712e9381d2e3592953a7f9d66540864441842cbf)
 2018-02-02 17:37:14 +08:00
Inigo Goiri
6769c78cdd HDFS-13043. RBF: Expose the state of the Routers in the federation. Contributed by Inigo Goiri. 
(cherry picked from commit 6ca7204cebf4b7060696a07b1777723a6dfa7d9d)
 2018-02-01 10:39:25 -08:00
Inigo Goiri
38c3e17117 HDFS-13044. RBF: Add a safe mode for the Router. Contributed by Inigo Goiri. 
(cherry picked from commit 961dcb693d128a7416720d9bdf4ecf51a98c2dd1)
 2018-01-31 09:17:40 -08:00
Inigo Goiri
ffba476192 HDFS-13083. RBF: Fix doc error setting up client. Contributed by tartarus. 
(cherry picked from commit 5206b2c7ca479dd53f614d75bab594043a9866e1)
 2018-01-30 18:28:54 -08:00
Sammi Chen
e30abdaaae HADOOP-15027. AliyunOSS: Support multi-thread pre-read to improve sequential read from Hadoop to Aliyun OSS performance. (Contributed by Jinhu Wu) 
(cherry picked from commit 9195a6e302028ed3921d1016ac2fa5754f06ebf0)
(cherry picked from commit 91184299c54df18540f841cca0efe0131d05b882)
(cherry picked from commit 2816bd1f43bca3734a8eaae0b228aaa72b575792)
 2018-01-30 17:15:12 +08:00
Kihwal Lee
a15df67f65 HDFS-12574. Add CryptoInputStream to WebHdfsFileSystem read call. Contributed by Rushabh S Shah. 
(cherry picked from commit eda786ea12db6b3ca8d6b0565c4ebdeab28b3cf6)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestEncryptionZones.java
 2018-01-29 17:57:54 -06:00
Jinjiang Ling
d70ca99594
YARN-7698. A misleading variable's name in ApplicationAttemptEventDispatcher 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 3400d0c535aeb151c3f283cc41111b15d66990e5)
(cherry picked from commit 987a8972acf4201642881dc3dcda21f6cfcf3bc7)
 2018-01-30 00:13:26 +09:00