Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
16,715 Commits 326 Branches 371 Tags
Commit Graph

16715 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Wangda Tan
03b6e95fa5 YARN-6977. Node information is not provided for non am containers in RM logs. (Suma Shivaprasad via wangda) 
Change-Id: I0c44d09a560446dee2ba68c2b9ae69fce0ec1d3e
(cherry picked from commit 8a42e922fad613f3cf1cc6cb0f3fa72546a9cc56)
 2017-09-17 21:19:20 -07:00
Arpit Agarwal
31cdd260f8 HDFS-12472. Add JUNIT timeout to TestBlockStatsMXBean. Contributed by Bharat Viswanadham. 2017-09-16 10:09:36 -07:00
Andrew Wang
7ea10e9f96 HDFS-10701. TestDFSStripedOutputStreamWithFailure#testBlockTokenExpired occasionally fails. Contributed by SammiChen. 
(cherry picked from commit ef8cd5dc565f901b4954befe784675e130e84c3c)
 2017-09-15 16:20:39 -07:00
Junping Du
21f6aa77ab YARN-7174. Add retry logic in LogsCLI when fetch running application logs. Contributed by Xuan Gong. 
(cherry picked from commit 1a84c24b0cf6674fa755403971fa57d8e412b320)
 2017-09-15 15:34:30 -07:00
Robert Kanter
b27ce4d599 HADOOP-14853. hadoop-mapreduce-client-app is not a client module (haibochen via rkanter) 
(cherry picked from commit b9b607daa74322a2928f7671a985a60388b9b9c2)
 2017-09-15 13:53:27 -07:00
Lei Xu
2787766b41 HDFS-12349. Improve log message when it could not alloc enough blocks for EC. (Lei (Eddy) Xu) 
(cherry picked from commit fbe06b58805aac4861fb27dfa273914b69e8bdc6)
 2017-09-15 12:15:49 -07:00
Robert Kanter
293ab432e1 YARN-7162. Remove XML excludes file format (rkanter) 
(cherry picked from commit 3a8d57a0a2e047b34be82f602a2b6cf5593d2125)
 2017-09-15 12:01:41 -07:00
Nathan Roberts
944b315dd1 HDFS-12441. Supress UnresolvedPathException in namenode log. Contributed by Kihwal Lee. 
(cherry picked from commit de197fc1562c4d1c39d24738c7cf8a8869dcec9c)
 2017-09-15 13:29:14 -05:00
Yufei Gu
6362a9503d YARN-6799. Remove the duplicated code in CGroupsHandlerImp.java. (Contributed by Weiyuan via Yufei Gu) 
(cherry picked from commit 5ab21dfe9c8e14585db71e02bcb021a526507f98)
 2017-09-15 09:51:50 -07:00
Robert Kanter
d3d32d3580 YARN-7162. Remove XML excludes file format (rkanter) 2017-09-15 09:48:59 -07:00
Steve Loughran
9f6b08f840
HADOOP-14553. Add (parallelized) integration tests to hadoop-azure 
Contributed by Steve Loughran

(cherry picked from commit 2d2d97fa7d4224369b3c13bc4a45e8cc9e29afb1)
 2017-09-15 17:04:43 +01:00
Steve Loughran
b5e9982355
MAPREDUCE-6956 FileOutputCommitter to gain abstract superclass PathOutputCommitter. 
Contributed by Steve Loughran

(cherry picked from commit 11390c2d111910b01d9c4d3e39dee49babae272f)
 2017-09-15 17:00:16 +01:00
Kihwal Lee
ba26097ea7 HDFS-12424. Datatable sorting on the Datanode Information page in the Namenode UI is broken. Contributed by Shawna Martell. 
(cherry picked from commit 78bdf10ae40300a5f2f3b95491d280802edc1df2)
 2017-09-15 09:55:04 -05:00
Kai Zheng
8e23929ad8 HADOOP-14869. Upgrade Apache Kerby version to v1.0.1. Contributed by Wei Zhou 2017-09-15 19:51:05 +08:00
Arpit Agarwal
a3d6ce25a3 HDFS-12456. TestNamenodeMetrics.testSyncAndBlockReportMetric fails. Contributed by Bharat Viswanadham. 2017-09-14 22:23:49 -07:00
Kai Zheng
754b27f17b HDFS-12395. Support erasure coding policy operations in namenode edit log. Contributed by Sammi Chen 2017-09-15 09:42:26 +08:00
Kai Zheng
02c8807312 HDFS-7859. Erasure Coding: Persist erasure coding policies in NameNode. Contributed by Sammi Chen 2017-09-15 09:09:31 +08:00
Lei Xu
e140489147 HDFS-12378. TestClientProtocolForPipelineRecovery#testZeroByteBlockRecovery fails on trunk. (Lei (Eddy) Xu) 2017-09-14 17:04:29 -07:00
Aaron Fabbri
62e8a5ceed
HADOOP-14738 Remove S3N and obsolete bits of S3A; rework docs. Contributed by Steve Loughran. 2017-09-14 16:12:29 -07:00
Yufei Gu
8d49c2fc3b YARN-7057. FSAppAttempt#getResourceUsage doesn't need to consider resources queued for preemption. (Contributed by Karthik via Yufei Gu) 
(cherry picked from commit 82c5dd1d508292ed88eda0f5356776437ba67d4c)
 2017-09-14 15:48:48 -07:00
Andrew Wang
61bc2c05af Revert "HDFS-11156. Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang." 
This reverts commit 7fcc73fc0d248aae1edbd4e1514c5818f6198928.

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/WebHDFS.md
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFS.java

(cherry picked from commit 390c2b5df0c4e25a2156d8daefd4670efe82b191)
 2017-09-14 15:12:42 -07:00
Andrew Wang
4da321233e Revert "HDFS-6874. Add GETFILEBLOCKLOCATIONS operation to HttpFS. Contributed by Weiwei Yang" 
This reverts commit 931a49800ef05ee0a6fdc143be1799abb228735d.

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/client/HttpFSFileSystem.java
	hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/server/FSOperations.java
	hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/client/BaseTestHttpFSWith.java

(cherry picked from commit 2f867115a819327a50cf9c8caa56e0e5a85cabb9)
 2017-09-14 15:12:41 -07:00
John Zhuge
9e1e81396e HADOOP-14864. FSDataInputStream#unbuffer UOE should include stream class name. Contributed by Bharat Viswanadham. 
(cherry picked from commit 7ee02d1065268511ec10a8b8852d0d6327419b9c)
 2017-09-14 11:28:58 -07:00
Arpit Agarwal
5f6abb2803 Revert "HDFS-10391. Always enable NameNode service RPC port. Contributed by Gergely Novak." 
This reverts commit b4dc2fa2474088e105d8ef0d4a87b1ff1fc2549a.
 2017-09-14 11:17:23 -07:00
Steve Loughran
ecad184808
Revert "HADOOP-14654. Update httpclient version to 4.5.3. (rchiang)" 
This reverts commit 14bec7043a622a4226b0b267818efae32accd1bf.
 2017-09-14 13:35:57 +01:00
Kai Zheng
3238947a1f HDFS-12438. Rename dfs.datanode.ec.reconstruction.stripedblock.threads.size to dfs.datanode.ec.reconstruction.threads. Contributed by Andrew Wang 2017-09-14 15:41:03 +08:00
Kai Zheng
b873b1395e HDFS-12414. Ensure to use CLI command to enable/disable erasure coding policy. Contributed by Sammi Chen 2017-09-14 09:22:00 +08:00
Andrew Wang
ad0fe570d9 HADOOP-14089. Automated checking for malformed client. Contributed by Sean Busbey. 
(cherry picked from commit c3f35c422bbb7fe9c8e6509063896de549b127d1)
 2017-09-13 16:57:53 -07:00
Robert Kanter
aa21b01fcb YARN-7146. Many RM unit tests failing with FairScheduler (rkanter) 
(cherry picked from commit bb34ae955496c1aa595dc1186153d605a41f5378)
 2017-09-13 15:45:28 -07:00
Jason Lowe
12e9558ed1 MAPREDUCE-6957. shuffle hangs after a node manager connection timeout. Contributed by Jooseong Kim 
(cherry picked from commit 4d98936eec1b5d196053426c70d455cf8f83f84f)
 2017-09-13 17:22:29 -05:00
Arpit Agarwal
78e9a98c4a HADOOP-14867. Update HDFS Federation setup document, for incorrect property name for secondary name node http address. Contributed by Bharat Viswanadham. 2017-09-13 12:54:11 -07:00
Eric Payne
4f6e8e08dc YARN-4727. Unable to override the $HADOOP_CONF_DIR env variable for container. Contributed by Jason Lowe. 
(cherry picked from commit 3860be7961580ac20dd505d665b580f0a04ac4f8)
 2017-09-13 14:46:04 -05:00
Andrew Wang
7b6b2dc93a HADOOP-14857. Fix downstream shaded client integration test. Contributed by Sean Busbey. 
(cherry picked from commit 8277fab2be3b0898ba326d15e4cb641da2ac51ce)
 2017-09-13 10:59:08 -07:00
Chen Liang
d1fcab3ea9 HADOOP-14804. correct wrong parameters format order in core-default.xml. Contributed by Chen Hongfei. 2017-09-13 10:55:13 -07:00
Andrew Wang
4687d3d6f6 HDFS-12222. Document and test BlockLocation for erasure-coded files. Contributed by Huafeng Wang. 
(cherry picked from commit f4b6267465d139bfdaf75e25761672eaf61d8a11)
 2017-09-12 17:35:33 -07:00
Arun Suresh
783db4792c YARN-7185. [Addendum patch] Minor javadoc and checkstyle fix. 
(cherry picked from commit a1c398e0dc85e424b45c50a3c3a0a784c04ed737)
 2017-09-12 17:04:25 -07:00
Arun Suresh
1230b60fa6 YARN-7185. ContainerScheduler should only look at availableResource for GUARANTEED containers when OPPORTUNISTIC container queuing is enabled. (Wangda Tan via asuresh) 
(cherry picked from commit 2ae72692fc370267141a1ee55ef372ff62302b54)
 2017-09-12 16:38:58 -07:00
Arpit Agarwal
e5a65ebc28 HDFS-12407. Journal node fails to shutdown cleanly if JournalNodeHttpServer or JournalNodeRpcServer fails to start. Contributed by Ajay Kumar. 2017-09-12 16:22:19 -07:00
Jason Lowe
4e147f7e96 HADOOP-14843. Improve FsPermission symbolic parsing unit test coverage. Contributed by Bharat Viswanadham 
(cherry picked from commit 86f4d1c66c8b541465ff769e5d951305c41c715c)
 2017-09-12 16:52:39 -05:00
Ray Chiang
99e7b26443 HADOOP-14856. Fix AWS, Jetty, HBase, Ehcache entries for NOTICE.txt. (rchiang) 
(cherry picked from commit 2ffe93ab609dfb54c3d1a53273ac2bc5ad15a5dd)
 2017-09-12 13:38:12 -07:00
Ray Chiang
5f6e387ba9 HADOOP-14797. Update re2j version to 1.1. (rchiang) 
(cherry picked from commit 288c3b12d46fccdb9638d823b1b6b0cbc076ec16)
 2017-09-12 11:35:15 -07:00
Andrew Wang
4e458e93f3 HDFS-12417. Disable flaky TestDFSStripedOutputStreamWithFailure. 
(cherry picked from commit 80ee89b2871ce2b4b7387de6e410f53d798dfd0b)
 2017-09-12 11:20:59 -07:00
Ray Chiang
619d61080c HADOOP-14653. Update joda-time version to 2.9.9. (rchiang) 
(cherry picked from commit a6432ba5a177ec3d3a95fa79e313a9bbc531a1e7)
 2017-09-12 11:13:21 -07:00
Ray Chiang
85db3b2aeb HADOOP-14648. Bump commons-configuration2 to 2.1.1. (rchiang) 
(cherry picked from commit 39818259c3b28a52b2cd7ee65b2422212d584664)
 2017-09-12 10:54:10 -07:00
Ray Chiang
fb5d12524c HADOOP-14796. Update json-simple version to 1.1.1. (rchiang) 
(cherry picked from commit af45cd1be63c3eee968f0c820e7d70230a7bf246)
 2017-09-12 10:36:28 -07:00
Ray Chiang
656c6d1acc HADOOP-14799. Update nimbus-jose-jwt to 4.41.1. (rchiang) 
(cherry picked from commit 556812c179aa094c21acf610439a8d69fe6420ab)
 2017-09-12 10:19:58 -07:00
Ray Chiang
f811306c86 HADOOP-14798. Update sshd-core and related mina-core library versions. (rchiang) 
(cherry picked from commit 3876fe4e500efcb88fff93a86acf9626d961c8c7)
 2017-09-12 10:00:25 -07:00
Wangda Tan
d2e3818823 YARN-7173. Container update RM-NM communication fix for backward compatibility. (Arun Suresh via wangda) 
Change-Id: I1c39ed5c59dee739ba5044b61b3ef5ed203b79c1
(cherry picked from commit e74d1be04be47969943b0501a4f335b0b5188287)
 2017-09-12 08:09:47 -07:00
Haibo Chen
13310f91a5 YARN-7132. FairScheduler.initScheduler() contains a surprising unary plus. (Yeliang Cang via Haibo Chen) 
(cherry picked from commit fa531788fd7c9eee70808818a2f6aec9bdc3bec7)
 2017-09-11 20:49:33 -07:00
Haibo Chen
4f58533389 YARN-7128. The error message in TimelineSchemaCreator is not enough to find out the error. (Jinjiang Ling via Haibo Chen) 
(cherry picked from commit 661f5eb0c6791148f2d15d5730635ccb668601e3)
 2017-09-11 16:22:06 -07:00