Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,012 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

18012 Commits

Author SHA1 Message Date
Jason Lowe f3e6682ba7 YARN-8865. RMStateStore contains large number of expired RMDelegationToken. Contributed by Wilfred Spiegelenburg 
(cherry picked from commit ab6aa4c726)
 2018-11-06 08:59:47 -06:00
Weiwei Yang ddb349ceb3 YARN-8970. Improve the debug message in CS#allocateContainerOnSingleNode. Contributed by Zhankun Tang. 
(cherry picked from commit 5d6554c722)
 2018-11-06 14:54:33 +08:00
Weiwei Yang c3a0f07db0 YARN-8969. AbstractYarnScheduler#getNodeTracker should return generic type to avoid type casting. Contributed by Wanqiang Ji. 
(cherry picked from commit c7fcca0d7e)
 2018-11-06 13:43:24 +08:00
Inigo Goiri 4b47daa640 HDFS-14043. Tolerate corrupted seen_txid file. Contributed by Lukas Majercak. 
(cherry picked from commit f3296501e0)
(cherry picked from commit 9bf4f3d614)
 2018-11-05 16:53:28 -08:00
Akira Ajisaka aee665f2db
HADOOP-15900. Update JSch versions in LICENSE.txt. 
(cherry picked from commit d43cc5db0f)
 2018-11-05 17:52:36 +09:00
Akira Ajisaka 869b1cee67
HADOOP-15899. Update AWS Java SDK versions in NOTICE.txt. 
(cherry picked from commit 73d8f8761e)
 2018-11-05 15:32:23 +09:00
Jonathan Hung 68ec0fa0a8 YARN-7225. Add queue and partition info to RM audit log. Contributed by Eric Payne 
(cherry picked from commit 2ab611d48b)
 2018-11-01 14:31:57 -07:00
Akira Ajisaka 2ae21d5387
Addendum: Fix potential FSImage corruption. Contributed by Xiao Chen. 
(cherry picked from commit dd70b1f538)
 2018-10-31 17:03:47 +09:00
Xiao Chen 20c9a12bc7 HDFS-14027. DFSStripedOutputStream should implement both hsync methods. 
(cherry picked from commit db7e636824)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStream.java

(cherry picked from commit 399645ebc1)
 2018-10-29 19:14:12 -07:00
Wei-Chiu Chuang 0f34ff7722 Revert "HADOOP-15864. Job submitter / executor fail when SBN domain name can not resolved. Contributed by He Xiaoqiao." 
This reverts commit fa01a8fa0e.
 2018-10-28 17:07:29 -07:00
Anu Engineer c5b704a34c HDFS-14028. HDFS OIV temporary dir deletes folder. 
Contributed by Adam Antal.

(cherry picked from commit 4f10d7e23f)
 2018-10-26 16:55:55 -07:00
Sunil G d91cb0a5e7 HADOOP-15815. Upgrade Eclipse Jetty version to 9.3.24. Cotributed by Boris Vulikh. 
(cherry picked from commit e28c00c290)
 2018-10-26 13:48:23 +05:30
Sammi Chen c5a227062f HADOOP-15868. AliyunOSS: update document for properties of multiple part download, multiple part upload and directory copy. Contributed by Jinhu Wu. 
(cherry picked from commit 7574d18538)
(cherry picked from commit 366541d834)
 2018-10-26 15:30:06 +08:00
Arpit Agarwal f6469adbb1 HDFS-14029. Sleep in TestLazyPersistFiles should be put into a loop. Contributed by Adam Antal. 
(cherry picked from commit 9b899f1ebd)
 2018-10-25 15:14:39 -07:00
Bharat Viswanadham 96bdaf3b4a HADOOP-15882. Upgrade maven-shade-plugin from 2.4.3 to 3.2.0. Contributed by Takanobu Asanuma. 
(cherry picked from commit 34b2521f5e)
 2018-10-25 13:42:50 -07:00
Wei-Chiu Chuang fa01a8fa0e HADOOP-15864. Job submitter / executor fail when SBN domain name can not resolved. Contributed by He Xiaoqiao. 
(cherry picked from commit fb2b72e6fc)
(cherry picked from commit fd02c501c0)
(cherry picked from commit ef9f8ca13d)
 2018-10-25 09:38:26 -07:00
Ajay Kumar 2caf69debd HDFS-13941. make storageId in BlockPoolTokenSecretManager.checkAccess optional. Contributed by Wei-Chiu Chuang. 2018-10-24 22:35:06 -07:00
Inigo Goiri 1d5390679e HDFS-14003. Fix findbugs warning in trunk for FSImageFormatPBINode. Contributed by Yiqun Lin. 
(cherry picked from commit 0e56c883cd)
 2018-10-24 16:14:22 -07:00
Xiao Chen e8832418bf Fix potential FSImage corruption. Contributed by Daryn Sharp and Vinayakumar B. 
(cherry picked from commit f1996ccbaee734d423caa9d47a571cfff98ef42c)
(cherry picked from commit 96cedb87b9)
 2018-10-24 16:10:51 -07:00
Inigo Goiri 6aef514795 HDFS-14021. TestReconstructStripedBlocksWithRackAwareness#testReconstructForNotEnoughRacks fails intermittently. Contributed by Xiao Chen. 
(cherry picked from commit c1874046e2)
 2018-10-24 09:38:48 -07:00
Akira Ajisaka e402791a51
HADOOP-15822. zstd compressor can fail with a small output buffer. Contributed by Jason Lowe. 
(cherry picked from commit 8f97d6f2cd)

Conflicts:
	dev-support/docker/Dockerfile
 2018-10-24 13:56:19 +09:00
Jason Lowe a9bc3e3708 YARN-8904. TestRMDelegationTokens can fail in testRMDTMasterKeyStateOnRollingMasterKey. Contributed by Wilfred Spiegelenburg 
(cherry picked from commit 93fb3b4b9c)
 2018-10-23 12:57:49 -05:00
Wei-Chiu Chuang e4464f9598 Revert "HDFS-13941. make storageId in BlockPoolTokenSecretManager.checkAccess optional. Contributed by Ajay Kumar." 
This reverts commit d8cd07d5f5.
 2018-10-23 05:58:10 -07:00
Rohith Sharma K S 6ff7056b58 YARN-8826. Fix lingering timeline collector after serviceStop in TimelineCollectorManager. Contributed by Prabha Manepalli. 
(cherry picked from commit 0b62983c5a)
 2018-10-23 14:09:18 +05:30
Arpit Agarwal d8cd07d5f5 HDFS-13941. make storageId in BlockPoolTokenSecretManager.checkAccess optional. Contributed by Ajay Kumar. 
(cherry picked from commit c58811c77d)
 2018-10-22 14:44:42 -07:00
Steve Loughran a069dade5c
HADOOP-15866. Renamed HADOOP_SECURITY_GROUP_SHELL_COMMAND_TIMEOUT keys break compatibility. 
Contributed by Wei-Chiu Chuang.

(cherry picked from commit cb8d11e7bf5055508949a6a4f074efacc8cb1305)
 2018-10-22 11:09:04 +01:00
Sunil G 78cf8c8099 YARN-7502. Nodemanager restart docs should describe nodemanager supervised property. Contributed by Suma Shivaprasad. 
(cherry picked from commit e50334513c)
 2018-10-22 14:17:29 +05:30
Wei-Chiu Chuang 8360d15abc HADOOP-15850. CopyCommitter#concatFileChunks should check that the blocks per chunk is not 0. Contributed by Ted Yu. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit e2cecb681e)
(cherry picked from commit a7dd244a49)

	modified:   hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyCommitter.java
(cherry picked from commit 84726679f0)
 2018-10-19 13:56:23 -07:00
Wei-Chiu Chuang 8754c0b65e Revert "HADOOP-15850. CopyCommitter#concatFileChunks should check that the blocks per chunk is not 0. Contributed by Ted Yu." 
This reverts commit 437c5187cd.
 2018-10-19 13:33:39 -07:00
Ted Yu 437c5187cd HADOOP-15850. CopyCommitter#concatFileChunks should check that the blocks per chunk is not 0. Contributed by Ted Yu. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit e2cecb681e)
(cherry picked from commit a7dd244a49)
(cherry picked from commit 84726679f0)
 2018-10-19 13:23:02 -07:00
Takanobu Asanuma 115c039495 HADOOP-15804. upgrade to commons-compress 1.18. Contributed by Akira Ajisaka. 
(cherry picked from commit 9bd18324c7)
 2018-10-19 21:41:10 +09:00
Weiwei Yang 37af9580c8 YARN-8907. Fix incorrect logging message in TestCapacityScheduler. Contributed by Zhankun Tang. 
(cherry picked from commit 13cc0f50ea)
 2018-10-19 10:03:39 +08:00
Siyao Meng 6a7bf9f850 HDFS-14009. HttpFS: FileStatus#setSnapShotEnabledFlag throws InvocationTargetException when attribute set is emptySet. Contributed by Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2018-10-18 17:02:38 -07:00
drankye b34c650a41 HDFS-13668. FSPermissionChecker may throws AIOOE when check inode permission. Contributed by He Xiaoqiao. 
(cherry picked from commit 975d60685e)
 2018-10-18 08:31:12 -07:00
Inigo Goiri ea19a36132 HDFS-14000. RBF: Documentation should reflect right scripts for v3.0 and above. Contributed by CR Hota. 
(cherry picked from commit 7347fa2df3)
 2018-10-17 16:43:05 -07:00
Jason Lowe 0aee3a0c3f HADOOP-15859. ZStandardDecompressor.c mistakes a class for an instance. Contributed by Jason Lowe 
(cherry picked from commit 9abda83947)
 2018-10-17 14:44:39 -05:00
Steve Loughran ae42d59eba
HADOOP-15861. Move DelegationTokenIssuer to the right path. 
Contributed by Wei-Chiu Chuang.

(cherry picked from commit 41b3603b5b)
 2018-10-17 11:04:05 +01:00
Xiao Chen 99b447f612 MAPREDUCE-7132. JobSplitWriter prints unnecessary warnings if EC(RS10,4) is used. Contributed by Peter Bacsko. 
(cherry picked from commit 25f8fcb064)
 2018-10-16 10:24:33 -07:00
Xiao Chen b9f65d717c HADOOP-14445. Addendum: Use DelegationTokenIssuer to create KMS delegation tokens that can authenticate to all KMS instances. 
(cherry picked from commit b6fc72a025)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java
 2018-10-15 11:00:39 -07:00
Xiao Chen ff7ca472d2 HADOOP-14445. Use DelegationTokenIssuer to create KMS delegation tokens that can authenticate to all KMS instances. 
Contributed by Daryn Sharp, Xiao Chen, Rushabh S Shah.
 2018-10-12 12:08:38 -07:00
Inigo Goiri 53b522af6d HDFS-13802. RBF: Remove FSCK from Router Web UI. Contributed by Fei Hui. 
(cherry picked from commit f63ee083db)
 2018-10-12 10:12:07 -07:00
Steve Loughran 2abc4ab60a
HADOOP-15837. DynamoDB table Update can fail S3A FS init. 
Contributed by Steve Loughran.

(cherry picked from commit f1fbc56371)
 2018-10-12 09:49:02 +01:00
Wangda Tan b67dc5e1d1 YARN-7274. Ability to disable elasticity at leaf queue level. (Zian Chen via wangda) 
Change-Id: Ic8d43e297f0f5de788b562f7eff8106c5c35e8d2
(cherry picked from commit 74665e3a7d)
 2018-10-11 16:00:10 +00:00
Jonathan Eagles 2b6d8c1e29 HADOOP-15835. Reuse Object Mapper in KMSJSONWriter (jeagles) 2018-10-10 10:32:43 -05:00
Hrishikesh Gadre b170de8be5 HDFS-13926. ThreadLocal aggregations for FileSystem.Statistics are incorrect with striped reads. 
Contributed by Xiao Chen, Hrishikesh Gadre.

Signed-off-by: Xiao Chen <xiao@apache.org>
(cherry picked from commit 323b76bccf)
 2018-10-09 19:54:56 -07:00
Weiwei Yang af85ce6ae4 YARN-8858. CapacityScheduler should respect maximum node resource when per-queue maximum-allocation is being used. Contributed by Wangda Tan. 
(cherry picked from commit edce866489)
 2018-10-10 09:56:16 +08:00
Inigo Goiri 8a9f61be4e HDFS-13964. RBF: TestRouterWebHDFSContractAppend fails with No Active Namenode under nameservice. Contributed by Ayush Saxena. 
(cherry picked from commit 3bfff0f79e)
 2018-10-06 10:42:51 -07:00
Jason Lowe bbceae10b9 HADOOP-15820. ZStandardDecompressor native code sets an integer field as a long. Contributed by Jason Lowe 
(cherry picked from commit f13e231025)
 2018-10-05 09:29:41 -05:00
Takanobu Asanuma 4bad89802b HADOOP-15816. Upgrade Apache Zookeeper version due to security concerns. Contributed by Akira Ajisaka. 
(cherry picked from commit 241cbec2da)
 2018-10-05 15:11:31 +09:00
Yiqun Lin d993a1fc3d HADOOP-15817. Reuse Object Mapper in KMSJSONReader. Contributed by Jonathan Eagles. 
(cherry picked from commit 81f635f47f)
 2018-10-04 10:36:49 +08:00