Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,301 Commits 326 Branches 371 Tags
Commit Graph

12301 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Arpit Agarwal
a071e33a91 HDFS-11054. Suppress verbose log message in BlockPlacementPolicyDefault. Contributed by Chen Liang. 2016-10-25 19:51:32 -07:00
Karthik Kambatla
e4ba92abee YARN-5388. Deprecate and remove DockerContainerExecutor. (Daniel Templeton via kasha) 2016-10-25 15:26:27 -07:00
Karthik Kambatla
aedd5c4c1b YARN-5677. RM should transition to standby when connection is lost for an extended period. (Daniel Templeton via kasha) 2016-10-25 13:15:58 -07:00
Yongjun Zhang
ce72314657 HDFS-11040. Add documentation for HDFS-9820 distcp improvement. Contributed by Yongjun Zhang. 
(cherry picked from commit 0f0c15f7a5ea33ced781978bea971f3750883f41)
 2016-10-25 12:29:28 -07:00
Yongjun Zhang
7bc170ba26 HDFS-9820. Improve distcp to support efficient restore to an earlier snapshot. Contributed by Yongjun Zhang. 
(cherry picked from commit 8650cc84f20e7d8c32dcdcd91c94372d476e2276)

Conflicts
	hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptions.java
	hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestDistCpOptions.java
	hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java
 2016-10-25 11:32:26 -07:00
Xiaoyu Yao
590d19c0a8 HDFS-11011. Add unit tests for HDFS command 'dfsadmin -set/clrSpaceQuota'. Contributed by Xiaobing Zhou. 
(cherry picked from commit 85f6fec0370b3cd94d6c2f19920c0b6d33f127c6)
 2016-10-25 11:24:27 -07:00
Junping Du
432657780d MAPREDUCE-6792. Allow user's full principal name as owner of MapReduce staging directory in JobSubmissionFiles#JobStagingDir(). Contributed by Santhosh G Nayak. 
(cherry picked from commit 9b7ce948a52f4ef433dc958ec891b669a669d7c2)
 2016-10-25 11:21:33 -07:00
Zhe Zhang
ff806cbfc7 HDFS-11015. Enforce timeout in balancer. Contributed by Kihwal Lee. 
(cherry picked from commit f6367c5f44a88cb5eb7edffb015b10b657504a61)
 2016-10-25 10:45:31 -07:00
Xiao Chen
4c5c6c8c44 HADOOP-13669. Addendum patch 2 for KMS Server should log exceptions before throwing. 
(cherry picked from commit 09ef97dccbd4c11aaeedf7503d414f999435ad91)
 2016-10-25 09:59:45 -07:00
Chris Nauroth
05d772e297 HADOOP-13309. Document S3A known limitations in file ownership and permission model. Contributed by Chris Nauroth. 
(cherry picked from commit 309a43925c078ff51cdb6bd1273e6f91f43311cb)
 2016-10-25 09:03:11 -07:00
Benoy Antony
b913b62aaf HADOOP-12082. Support multiple authentication schemes via AuthenticationFilter. Contributed by Hrishikesh Gadre. 2016-10-25 08:30:02 -07:00
Akira Ajisaka
6bb23a14b6 HADOOP-13514. Upgrade maven surefire plugin to 2.19.1. Contributed by Ewan Higgs. 
(cherry picked from commit dbd205762ef2cba903b9bd9335bb9a5964d51f74)
 2016-10-25 15:58:19 +09:00
Karthik Kambatla
ab9105d814 YARN-5754. Null check missing for earliest in FifoPolicy. (Yufei Gu via kasha) 
(cherry picked from commit a71fc81655cd5382d354674dd06570ba49753688)
 2016-10-24 22:19:56 -07:00
Chris Nauroth
5c2f67bdae HADOOP-12774. s3a should use UGI.getCurrentUser.getShortname() for username. Contributed by Steve Loughran. 
(cherry picked from commit 3372e940303149d6258e0b72c54d72f080f0daa2)
 2016-10-24 21:54:16 -07:00
Chris Nauroth
5b7cbb5a3c HADOOP-13727. S3A: Reduce high number of connections to EC2 Instance Metadata Service caused by InstanceProfileCredentialsProvider. Contributed by Chris Nauroth. 
(cherry picked from commit d8fa1cfa6722cbf7a4ec3d6b9c44b034da9aa351)
 2016-10-24 21:22:46 -07:00
Subru Krishnan
81595a127f YARN-5711. Propogate exceptions back to client when using hedging RM failover provider. 
(cherry picked from commit 0a166b13472213db0a0cd2dfdaddb2b1746b3957)
 2016-10-24 19:02:03 -07:00
Robert Kanter
c76eb90f0e MAPREDUCE-6728. Give fetchers hint when ShuffleHandler rejects a shuffling connection (haibochen via rkanter) 2016-10-24 17:47:09 -07:00
Kihwal Lee
086577c67a HDFS-11042. Add missing cleanupSSLConfig() call for tests that use setupSSLConfig(). Contributed by Kuhu Shukla. 
(cherry picked from commit b18f35f097a3ee33f449e506676961f9b9456bbd)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java
 2016-10-24 08:56:14 -05:00
Jian He
bb8214cab8 Revert YARN-4126. RM should not issue delegation tokens in unsecure mode. 2016-10-23 15:56:22 -07:00
Xiaoyu Yao
48c9064b33 HADOOP-13749. KMSClientProvider combined with KeyProviderCache can result in wrong UGI being used. Contributed by Xiaoyu Yao. 
(cherry picked from commit d0a347984da175948b553a675dc357491df2fd0f)
 2016-10-23 10:59:01 -07:00
Xiaoyu Yao
68d91e0200 Revert "HDFS-10757. KMSClientProvider combined with KeyProviderCache can result in wrong UGI being used. Contributed by Xiaoyu Yao." 
This reverts commit 0205ad5fa81c4001b4d5189dce88b4546a98b2eb.
 2016-10-23 08:26:01 -07:00
Xiaoyu Yao
0205ad5fa8 HDFS-10757. KMSClientProvider combined with KeyProviderCache can result in wrong UGI being used. Contributed by Xiaoyu Yao. 
(cherry picked from commit be7237224819e2491aef91cd4f055c7efcf7b90d)
 2016-10-21 14:25:29 -07:00
Chris Douglas
25f4327f0b HADOOP-13702. Add instrumented ReadWriteLock. Contributed by Jingcheng Du 
(cherry picked from commit ae8bccd5090d8b42dae9a8e0c13a9766a7c42ecb)
 2016-10-21 13:01:08 -07:00
Steve Loughran
385c1daa46 HADOOP-13716. Add LambdaTestUtils class for tests; fix eventual consistency problem in contract test setup. Contributed by Steve Loughran. 2016-10-21 19:11:31 +01:00
Brahma Reddy Battula
2ab149f0e4 HADOOP-13236. truncate will fail when we use viewfilesystem. Contributed by Brahma Reddy Battula 
(cherry picked from commit f872c6bc0390415f13e95b99749b0b1a690991b7)
 2016-10-21 16:19:18 +05:30
Karthik Kambatla
440a6326d1 YARN-5047. Refactor nodeUpdate across schedulers. (Ray Chiang via kasha) 
(cherry picked from commit 754cb4e30fac1c5fe8d44626968c0ddbfe459335)
 2016-10-20 21:18:13 -07:00
Karthik Kambatla
c2cabce2e8 YARN-4911. Bad placement policy in FairScheduler causes the RM to crash 
(cherry picked from commit a064865abf7dceee46d3c42eca67a04a25af9d4e)
 2016-10-20 20:57:31 -07:00
Mingliang Liu
c37c00b061 HDFS-10998. Add unit tests for HDFS command 'dfsadmin -fetchImage' in HA. Contributed by Xiaobing Zhou 
(cherry picked from commit d7d87deece66333c188e9b7c10b4b56ddb529ce9)
 2016-10-20 19:54:35 -07:00
Steve Loughran
f0af3dee25 Revert "HADOOP-13716. Add LambdaTestUtils class for tests; fix eventual consistency problem in contract test setup. Contributed by Steve Loughran." 
This reverts commit 42f8a1d6eb5add3180094b126c4af2b3ddbc7cae.
 2016-10-20 22:48:26 +01:00
Anu Engineer
1f384b617d HADOOP-13737. Cleanup DiskChecker interface. Contributed by Arpit Agarwal. 2016-10-20 13:40:01 -07:00
Aaron T. Myers
0f616cefeb HDFS-10423. Increase default value of httpfs maxHttpHeaderSize. Contributed by Nicolae Popa. 
(cherry picked from commit aa1b583cf99d1a7cfe554d1769fc4af252374663)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/libexec/httpfs-config.sh
 2016-10-20 13:34:52 -07:00
Anu Engineer
42f8a1d6eb HADOOP-13716. Add LambdaTestUtils class for tests; fix eventual consistency problem in contract test setup. Contributed by Steve Loughran. 2016-10-20 12:47:57 -07:00
Wei-Chiu Chuang
6f4192d77d HDFS-11018. Incorrect check and message in FsDatasetImpl#invalidate. Contributed by Yiqun Lin. 
(cherry picked from commit 6d2da38d16cebe9b82f1048f87127eecee33664c)

Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
 2016-10-20 11:58:46 -07:00
Steve Loughran
ab36519b6f Revert "HADOOP-13236. truncate will fail when we use viewfilesystem. Contributed by Brahma Reddy Battula" 
This reverts commit bf6379fdd2eef2fed30414559190afa8ccd963d5.
 2016-10-20 19:32:10 +01:00
Brahma Reddy Battula
bf6379fdd2 HADOOP-13236. truncate will fail when we use viewfilesystem. Contributed by Brahma Reddy Battula 
(cherry picked from commit f872c6bc0390415f13e95b99749b0b1a690991b7)
 2016-10-20 21:48:23 +05:30
Kihwal Lee
cacf83b6c3 HDFS-10627. Volume Scanner marks a block as "suspect" even if the exception is network-related. Contributed by Rushabh S Shah. 
(cherry picked from commit 5c0bffddc0cb824a8a2751bcd0dc3e15ce081727)
 2016-10-20 11:02:02 -05:00
Brahma Reddy Battula
69a4b0c7d2 HDFS-10699. Log object instance get incorrectly in TestDFSAdmin. Contributed by Yiqun Lin 
(cherry picked from commit f5e401de3e7ad6e73b07fef1cb5915753b08fbdb)
 2016-10-20 21:08:39 +05:30
Brahma Reddy Battula
df5671db74 HDFS-9480. Expose nonDfsUsed via StorageTypeStats. Contributed by Brahma Reddy Battula 
(cherry picked from commit 4c73be135ca6ee2ba0b075a507097900db206b09)
 2016-10-20 20:47:37 +05:30
Steve Loughran
c640ba5fc9 HADOOP-13735 ITestS3AFileContextStatistics.testStatistics() failing. Contributed by Pieter Reuse 2016-10-20 14:50:30 +01:00
Brahma Reddy Battula
5a37668d1b HDFS-11025. TestDiskspaceQuotaUpdate fails in trunk due to Bind exception. Contributed by Yiqun Lin 
(cherry picked from commit 73504b1bdc4b93c64741de5eb9d022817fdfa22f)
 2016-10-20 10:42:34 +05:30
Arpit Agarwal
a3cbaf0c04 HDFS-10752. Several log refactoring/improvement suggestion in HDFS. Contributed by Hanisha Koneru. 2016-10-19 17:20:15 -07:00
Xiao Chen
ad7d3c4db8 HDFS-11009. Add a tool to reconstruct block meta file from CLI. 
(cherry picked from commit bc4a32aea50e86819730312e89315c0244ce64bf)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetUtil.java
 2016-10-18 22:40:58 -07:00
Xiaoyu Yao
74f1c49d76 HDFS-10906. Add unit tests for Trash with HDFS encryption zones. Contributed by Hanisha Koneru. 
(cherry picked from commit 0c79d5af74944363f3d04f7e1702411db564bbfd)
 2016-10-18 14:07:57 -07:00
Steve Loughran
bc176961e6 HADOOP-13560. S3ABlockOutputStream to support huge (many GB) file writes. Contributed by Steve Loughran 2016-10-18 19:33:38 +01:00
Kai Zheng
3972bb3bba HDFS-10920. TestStorageMover#testNoSpaceDisk is failing intermittently. Contributed by Rakesh R 2016-10-18 15:08:15 +06:00
Akira Ajisaka
b2618685fb HADOOP-13522. Add %A and %a formats for fs -stat command to print permissions. Contributed by Alex Garbarini. 
(cherry picked from commit bedfec0c10144087168bc79501ffd5ab4fa52606)
 2016-10-18 15:03:33 +09:00
Vinitha Reddy Gankidi
278eeb1056 HDFS-10712. TestDataNodeVolumeFailure should pass not null BlockReportContext. Contributed by Vinitha Gankidi. 2016-10-17 18:55:26 -07:00
Ming Ma
d55a7f8935 HDFS-9390. Block management for maintenance states. 2016-10-17 17:46:29 -07:00
Allen Wittenauer
a5a56c3564 HADOOP-13034. Log message about input options in distcp lacks some items (Takashi Ohnishi via aw) 
(cherry picked from commit 422c73a8657d8699920f7db13d4be200e16c4272)
 2016-10-17 16:35:32 -07:00
Robert Kanter
e02c756f16 YARN-5466. DefaultContainerExecutor needs JavaDocs (templedf via rkanter) 
(cherry picked from commit f5d92359145dfb820a9521e00e2d44c4ee96e67e)
 2016-10-17 14:29:42 -07:00