Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,374 Commits 326 Branches 371 Tags
Commit Graph

17374 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Xiao Chen
8f4bfb3ed5 HDFS-13239. Fix non-empty dir warning message when setting default EC policy. Contributed by Bharat Viswanadham. 
(cherry picked from commit d6931c30c5a643ca192109b05a9c44da42a6318c)
 2018-03-13 10:08:09 -07:00
Xiao Chen
752e116f41 HDFS-12505. Extend TestFileStatusWithECPolicy with a random EC policy. Contributed by Takanobu Asanuma. 
(cherry picked from commit 84c10955863eca1e300aeeac1d9cd7a1186144b6)
 2018-03-13 09:58:50 -07:00
Xiaoyu Yao
2a93ba690c HDFS-12455. WebHDFS - Adding "snapshot enabled" status to ListStatus query result. Contributed by Ajay Kumar. 
(cherry picked from commit 107c177782a24a16c66113841f2fc5144f56207b)
 2018-03-12 20:23:52 -07:00
Yiqun Lin
840c83bf7a HDFS-13226. RBF: Throw the exception if mount table entry validated failed. Contributed by maobaolong. 2018-03-13 11:13:15 +08:00
Chris Douglas
a03c8ea61f HADOOP-14696. parallel tests don't work for Windows. Contributed by Allen Wittenauer 
(cherry picked from commit 45d1b0fdcc04a86be91a9b72073cdc30bec04d3b)
(cherry picked from commit 09940b1eb3b7ed764149f4a993c1857e9c6ad938)
 2018-03-12 20:06:22 -07:00
Yiqun Lin
430cdfefc0 HDFS-13253. RBF: Quota management incorrect parent-child relationship judgement. Contributed by Yiqun Lin. 2018-03-13 10:41:44 +08:00
Akira Ajisaka
89105a623b
HDFS-12156. TestFSImage fails without -Pnative 
(cherry picked from commit 319defafc105c0d0b69b83828b578d9c453036f5)
 2018-03-13 11:27:46 +09:00
Inigo Goiri
177dd6580d HDFS-13241. RBF: TestRouterSafemode failed if the port 8888 is in use. Contributed by maobaolong. 
(cherry picked from commit 91c82c90f05ea75fe59c6ffad3dc3fcac1429e9e)
 2018-03-12 17:30:00 -07:00
Chris Douglas
2210593dad HADOOP-14742. Document multi-URI replication Inode for ViewFS. Contributed by Gera Shegalov 
(cherry picked from commit ddb67ca707de896cd0ba5cda3c0d1a2d9edca968)
(cherry picked from commit c9364b3bce30946a6c0154974d0adc6accb3bea3)
 2018-03-12 13:55:10 -07:00
Inigo Goiri
29cf29a853 HDFS-11399. Many tests fails in Windows due to injecting disk failures. Contributed by Yiqun Lin. 
(cherry picked from commit ac627f561f0946e98a650850fb507536cbd2f2c4)
 2018-03-12 10:05:43 -07:00
Wangda Tan
8034faa888 Updated timeline reader to use AuthenticationFilter 
Change-Id: I961771589180c1eb377d36c37a79aa23754effbf
(cherry picked from commit 837338788eb903d0e8bbb1230694782a707891be)
 2018-03-10 13:36:09 -08:00
Owen O'Malley
0f3b6102ef Revert "HADOOP-13119. Add ability to secure log servlet using proxy users. Contribute by Yuanbo Liu." 
This reverts commit a847903b6e64c6edb11d852b91f2c816b1253eb3.

Change-Id: I3122a2142f5bdf8507dece930e447556a43cd9ae
(cherry picked from commit 8fad3ec76070ccfcd3ed80feaba4355077bc6f5c)

Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestHttpServerWithSpengo.java
 2018-03-10 13:35:47 -08:00
Owen O'Malley
0f106df4d6 Revert "HADOOP-14077. Add ability to access jmx via proxy. Contributed by Yuanbo Liu." 
This reverts commit 172b23af33554b7d58fd41b022d983bcc2433da7.

(cherry picked from commit d0d2d4c51e9534e08893ae14cf3fff7b2ee70b1d)
 2018-03-10 13:33:52 -08:00
Yiqun Lin
2e607f2103 HDFS-13240. RBF: Update some inaccurate document descriptions. Contributed by Yiqun Lin. 
(cherry picked from commit 4743d4a2c70a213a41804a24c776e6db00e1b90d)
 2018-03-10 11:32:12 +08:00
Inigo Goiri
465c3f0e24 HDFS-13232. RBF: ConnectionPool should return first usable connection. Contributed by Ekanth S. 
(cherry picked from commit 8133cd5305d7913453abb2d48da12f672c0ce334)
 2018-03-09 18:27:17 -08:00
Inigo Goiri
c453fcb23f HDFS-13212. RBF: Fix router location cache issue. Contributed by Weiwei Wu. 
(cherry picked from commit afe1a3ccd56a12fec900360a8a2855c080728e65)
 2018-03-09 17:20:29 -08:00
Hanisha Koneru
bd2ebf031c HDFS-13244. Add stack, conf, metrics links to utilities dropdown in NN webUI. Contributed by Bharat Viswanadham. 
(cherry picked from commit 4eeff62f6925991bca725b1ede5308055817de80)
 2018-03-09 15:49:09 -08:00
Lokesh Jain
80aabd72c2
HDFS-13190. Document WebHDFS support for snapshot diff 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
Signed-off-by: Xiaoyu Yao <xyao@apache.org>
(cherry picked from commit 7b0dc310208ee5bc191c9accb3d1312513145653)
 2018-03-09 15:07:34 -08:00
Hanisha Koneru
dec53bb9e1 HDFS-11394. Support for getting erasure coding policy through WebHDFS#FileStatus. 
(cherry picked from commit 9a082fbe6e302df7139b65a23be9a39acd87715d)
 2018-03-09 13:39:49 -08:00
Jason Lowe
9dacb9fb27 MAPREDUCE-6930. mapreduce.map.cpu.vcores and mapreduce.reduce.cpu.vcores are both present twice in mapred-default.xml. Contributed by Sen Zhao 
(cherry picked from commit 32fa3a63e0e7d8bfb3d3b9b3c500ecb3a4874ecf)
 2018-03-09 10:44:49 -06:00
Steve Loughran
ca5c4d454e HADOOP-15277. Remove .FluentPropertyBeanIntrospector from CLI operation log output. 
Contributed by Steve Loughran.
 2018-03-09 10:50:04 +00:00
Steve Loughran
1771af2320 HADOOP-15273.distcp can't handle remote stores with different checksum algorithms. 
Contributed by Steve Loughran.

(cherry picked from commit 7ef4d942dd96232b0743a40ed25f77065254f94d)
 2018-03-09 10:49:11 +00:00
Yiqun Lin
f879504fe1 HDFS-13233. RBF: MountTableResolver doesn't return the correct mount point of the given path. Contributed by wangzhiyuan. 2018-03-09 15:48:36 +08:00
Inigo Goiri
68cff8c235 HDFS-13232. RBF: ConnectionManager's cleanup task will compare each pool's own active conns with its total conns. Contributed by Chao Sun. 
(cherry picked from commit 0c2b969e0161a068bf9ae013c4b95508dfb90a8a)
 2018-03-08 09:33:24 -08:00
Yiqun Lin
7a9aea17a2 HADOOP-15296. Fix a wrong link for RBF in the top page. Contributed by Takanobu Asanuma. 
(cherry picked from commit 4cc9a6d9bb34329d6de30706d5432c7cb675bb88)
 2018-03-08 16:05:53 +08:00
Hanisha Koneru
0b1e9665df HDFS-13225. StripeReader#checkMissingBlocks() 's IOException info is incomplete. Contributed by lufei. 
(cherry picked from commit 19ae4429f9a2807548bb4b9ff296521fbdbbb2bc)
 2018-03-07 14:22:01 -08:00
Steve Loughran
cc0f14c13c HADOOP-15267. S3A multipart upload fails when SSE-C encryption is enabled. 
Contributed by Anis Elleuch.

(cherry picked from commit 1dedc68f9d8d8544d715e67ee77cd3f017c21699)
 2018-03-07 19:20:53 +00:00
Yiqun Lin
253ef33f80 HDFS-13214. RBF: Complete document of Router configuration. Contributed by Yiqun Lin. 2018-03-07 16:07:05 +08:00
Xiaoyu Yao
b74df5523c HDFS-13109. Support fully qualified hdfs path in EZ commands. Contributed by Hanisha Koneru. 
(cherry picked from commit edf9445708ffb7a9e59cb933e049b540f99add1e)
 2018-03-06 18:30:10 -08:00
Allen Wittenauer
5aaf88d401
HADOOP-15279. increase maven heap size recommendations 
Signed-off-by: Jason Lowe <jlowe@apache.org>
Signed-off-by: Chris Douglas <cdouglas@apache.org>
(cherry picked from commit d4cc50f62279dc4d607cd05c16de480a76f05880)
 2018-03-06 13:56:40 -08:00
Xiao Chen
283d48a4f7 HDFS-13170. Port webhdfs unmaskedpermission parameter to HTTPFS. Contributed by Stephen O'Donnell. 
(cherry picked from commit 92c15e19272a8bb59f8704c44d4cd713d3a58bcb)
 2018-03-06 09:58:09 -08:00
Steve Loughran
a5cb7f3616 HADOOP-15289. FileStatus.readFields() assertion incorrect. 
Contributed by Steve Loughran.

(cherry picked from commit 12ecb55ffee2b89108eea2565e7ad955ea1fabe7)
 2018-03-06 10:08:28 +00:00
Sen Zhao
4df200bc6d
YARN-7736. Fix itemization in YARN federation document 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 245751ffdc4229715a0c031f57f20748ed16d8a6)
 2018-03-05 11:55:01 -08:00
Xiao Chen
d28db463c2 HDFS-13040. Kerberized inotify client fails despite kinit properly. Contributed by Istvan Fajth, Wei-Chiu Chuang, Xiao Chen. 
(cherry picked from commit c75105f07b4cdbc2773435fc1125446233113c15)
 2018-03-02 16:47:36 -08:00
Gergely Novák
8b9a7b2e48
MAPREDUCE-7023. TestHadoopArchiveLogs.testCheckFilesAndSeedApps fails on rerun 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit e99a1e77671fa0e74a652bc2221bbe01d1a579d3)
 2018-03-02 16:22:06 -08:00
Jason Lowe
7c224656ce YARN-7511. NPE in ContainerLocalizer when localization failed for running container. Contributed by Tao Yang 
(cherry picked from commit 83798f15f8602ef580a7885876de114b2425da89)
 2018-03-02 10:03:51 -06:00
Weiwei Yang
5db733fb09 MAPREDUCE-7059. Downward Compatibility issue: MR job fails because of unknown setErasureCodingPolicy method from 3.x client to HDFS 2.x cluster. Contributed by Jiandan Yang. 2018-03-02 10:43:03 +08:00
Steve Loughran
c960ae0b9a MAPREDUCE-7060. Cherry Pick PathOutputCommitter class/factory to branch-3.0. 
Contributed by Steve Loughran.
 2018-03-01 18:39:54 +00:00
Nanda kumar
7193c95584
HADOOP-15275. Incorrect javadoc for return type of RetryPolicy#shouldRetry 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 55d04a6db10d0e58cfe0ff82fa646bb6ba57aaac)
 2018-03-01 19:04:41 +09:00
Xiaoyu Yao
908ca63c03 HDFS-13081. Datanode#checkSecureConfig should allow SASL and privileged HTTP. Contributed by Ajay Kumar. 
(cherry picked from commit f20e10b2dd59f99d9af009bbbb60a067b9893e69)
 2018-02-28 09:48:12 -08:00
Xiaoyu Yao
ce356dcbf6 HDFS-13114. CryptoAdmin#ReencryptZoneCommand should resolve Namespace info from path. Contributed by Hanisha Koneru. 
(cherry picked from commit 90fbe5c6834a6685e65542fa69e396fad2678927)
 2018-02-28 08:44:14 -08:00
Steve Loughran
3c03672e87 HADOOP-15263. hadoop cloud-storage module to mark hadoop-common as provided; add azure-datalake. 
Contributed by Steve Loughran

(cherry picked from commit edc9f1451b4af79069cd4b1d608b0fa68648ee72)
(cherry picked from commit abfe1eaf29a31d96f63022a59e29392b773e6bc1)
 2018-02-28 15:33:21 +00:00
Steve Loughran
0dcd0ead91 HADOOP-15264. AWS "shaded" SDK 1.271 is pulling in netty 4.1.17. 
Contributed by Steve Loughran.

(cherry picked from commit b580196a16dcafe26d6069267a5e2242917aad84)
 2018-02-28 15:33:08 +00:00
Inigo Goiri
0eff245d3d HDFS-13199. RBF: Fix the hdfs router page missing label icon issue. Contributed by maobaolong. 
(cherry picked from commit d86f301d464683f8d392dad50e83f50d823e862e)
 2018-02-27 17:41:42 -08:00
Arpit Agarwal
e22a731070 MAPREDUCE-7061. SingleCluster setup document needs to be updated. Contributed by Bharat Viswanadham. 2018-02-27 10:28:03 -08:00
Yiqun Lin
09017aee90 HDFS-13184. RBF: Improve the unit test TestRouterRPCClientRetries. Contributed by Yiqun Lin. 
(cherry picked from commit 1e85a995d1c7fe3dbf4b36a481feb5fdeeb6015b)
 2018-02-27 10:52:49 +08:00
Chao Sun
26395aef67 HDFS-13145. SBN crash when transition to ANN with in-progress edit tailing enabled. Contributed by Chao Sun. 
(cherry picked from commit ae290a4bb4e514e2fe9b40d28426a7589afe2a3f)
 2018-02-26 16:18:51 -08:00
Wei Yan
1fb87df87e HDFS-13187. RBF: Fix Routers information shown in the web UI. Contributed by Inigo Goiri and Wei Yan. 
(cherry picked from commit 6ce9f79cc9b2107e5953a39d05b22966aff0b7ff)
 2018-02-26 15:29:11 -08:00
Kihwal Lee
1087b9af8c HDFS-12070. Failed block recovery leaves files open indefinitely and at risk for data loss. Contributed by Kihwal Lee. 
(cherry picked from commit 451265a83d8798624ae2a144bc58fa41db826704)
 2018-02-26 10:30:50 -06:00
Xiaoyu Yao
21d4b5fd24 HDFS-13052. WebHDFS: Add support for snasphot diff. Contributed by Lokesh Jain. 
(cherry picked from commit 1e84e46f1621fe694f806bfc41d3b2a06c9500b6)
 2018-02-23 19:41:28 -08:00