Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,819 Commits 326 Branches 371 Tags
Commit Graph

18819 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Jason Lowe
afbcb51ac1 MAPREDUCE-7156. NullPointerException when reaching max shuffle connections. Contributed by Peter Bacsko 
(cherry picked from commit ba1f9d66d94ed0b85084d7c40c09a87478b3a05a)
 2018-11-06 18:07:57 -06:00
Inigo Goiri
f65324aa47 HDFS-14048. DFSOutputStream close() throws exception on subsequent call after DataNode restart. Contributed by Erik Krogen. 
(cherry picked from commit 887244de4adebe27693ed4ad3296a6f700cfa8c1)
 2018-11-06 11:20:33 -08:00
Jason Lowe
7335d940de YARN-8865. RMStateStore contains large number of expired RMDelegationToken. Contributed by Wilfred Spiegelenburg 
(cherry picked from commit ab6aa4c7265db5bcbb446c2f779289023d454b81)
 2018-11-06 08:52:29 -06:00
Weiwei Yang
631b31110c YARN-8970. Improve the debug message in CS#allocateContainerOnSingleNode. Contributed by Zhankun Tang. 
(cherry picked from commit 5d6554c722f08f79bce904e021243605ee75bae3)
 2018-11-06 14:53:28 +08:00
Weiwei Yang
71999f4464 YARN-8969. AbstractYarnScheduler#getNodeTracker should return generic type to avoid type casting. Contributed by Wanqiang Ji. 
(cherry picked from commit c7fcca0d7ec9e31d43ef3040ecd576ec808f1f8b)
 2018-11-06 13:23:42 +08:00
Inigo Goiri
9bf4f3d614 HDFS-14043. Tolerate corrupted seen_txid file. Contributed by Lukas Majercak. 
(cherry picked from commit f3296501e09fa7f1e81548dfcefa56f20fe337ca)
 2018-11-05 16:52:43 -08:00
Giovanni Matteo Fumarola
a1321d020a HDFS-14042. Fix NPE when PROVIDED storage is missing. Contributed by Virajith Jalaparti. 2018-11-05 11:39:15 -08:00
Akira Ajisaka
b732ed3553
HADOOP-15900. Update JSch versions in LICENSE.txt. 
(cherry picked from commit d43cc5db0ff80958ca873df1dc2fa00054e37175)
 2018-11-05 17:52:21 +09:00
Akira Ajisaka
73d8f8761e
HADOOP-15899. Update AWS Java SDK versions in NOTICE.txt. 2018-11-05 15:31:53 +09:00
Xiaoyu Yao
0577c89015 HADOOP-15339. Support additional key/value propereties in JMX bean registration. Contributed by Elek Marton. 2018-11-02 10:44:41 +01:00
Jonathan Hung
221494a75c YARN-7225. Add queue and partition info to RM audit log. Contributed by Eric Payne 
(cherry picked from commit 2ab611d48b7669b31bd2c9b918f47251da77d0f6)
 2018-11-01 14:31:22 -07:00
Steve Loughran
cd641f3218
HADOOP-15781. S3A assumed role tests failing due to changed error text in AWS exceptions. 
Contributed by Steve Loughran.

Missed final change to ITestAssumeRole
 2018-11-01 12:55:19 +00:00
Steve Loughran
3f35946c6e
HADOOP-15781. S3A assumed role tests failing due to changed error text in AWS exceptions. 
Contributed by Steve Loughran

This is a subset of c07715e37895a2e406 to track changes in the AWS STS error messages.
If the later AWS SDK is ever backported to older branches, the rest of that patch will need
cherry-picking
 2018-11-01 12:52:48 +00:00
Akira Ajisaka
dd70b1f538
Addendum: Fix potential FSImage corruption. Contributed by Xiao Chen. 2018-10-31 17:02:15 +09:00
Steve Loughran
e060e90baa
HADOOP-15855. Review hadoop credential doc, including object store details. 
Contributed by Steve Loughran.

(cherry picked from commit 62d98ca92aee15d1790d169bfdf0043b05b748ce)
 2018-10-30 15:59:00 +00:00
Rohith Sharma K S
d9a494b1e0 YARN-8950. Fix compilation issue due to dependency convergence error for hbase.profile=2.0. 
(cherry picked from commit 4ec4ec69711180d642c5b56cd3d3dbdf44d3c61f)
 2018-10-30 11:50:56 +05:30
Xiao Chen
399645ebc1 HDFS-14027. DFSStripedOutputStream should implement both hsync methods. 
(cherry picked from commit db7e636824a36b90ba1c8e9b2fba1162771700fe)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStream.java
 2018-10-29 19:13:51 -07:00
Weiwei Yang
7dd8eafe33 YARN-8915. Update the doc about the default value of maximum-container-assignments for capacity scheduler. Contributed by Zhankun Tang. 
(cherry picked from commit 78ea897b6701c82799a6231103ca7ee1f8b4efde)
 2018-10-29 12:06:52 +08:00
Weiwei Yang
70efe253f3 YARN-8944. TestContainerAllocation.testUserLimitAllocationMultipleContainers failure after YARN-8896. Contributed by Wilfred Spiegelenburg. 
(cherry picked from commit 1d90a0dd23c5d1bd52d04c303506806d0dc61cd2)
 2018-10-29 11:56:31 +08:00
Wei-Chiu Chuang
35e09fe594 Revert "HADOOP-15864. Job submitter / executor fail when SBN domain name can not resolved. Contributed by He Xiaoqiao." 
This reverts commit ef9f8ca13d8dca8031c045bad801fceb3e00aa15.
 2018-10-28 17:06:40 -07:00
Anu Engineer
4a1be70099 HDFS-14028. HDFS OIV temporary dir deletes folder. 
Contributed by Adam Antal.

(cherry picked from commit 4f10d7e23fc0b757a9e94bc448187a6211b90f10)
 2018-10-26 16:51:52 -07:00
Sunil G
1408f46834 HADOOP-15815. Upgrade Eclipse Jetty version to 9.3.24. Cotributed by Boris Vulikh. 
(cherry picked from commit e28c00c290a06813d156a845b93e806f71413dbf)
 2018-10-26 13:47:49 +05:30
Sammi Chen
366541d834 HADOOP-15868. AliyunOSS: update document for properties of multiple part download, multiple part upload and directory copy. Contributed by Jinhu Wu. 
(cherry picked from commit 7574d18538e838f40581519080d7c8621c65e53b)
 2018-10-26 15:28:20 +08:00
Arpit Agarwal
8b4f9b3e2c HDFS-14029. Sleep in TestLazyPersistFiles should be put into a loop. Contributed by Adam Antal. 
(cherry picked from commit 9b899f1ebd5126a756fceb43459164165488b203)
 2018-10-25 15:14:48 -07:00
Bharat Viswanadham
e3bf7f29af HADOOP-15882. Upgrade maven-shade-plugin from 2.4.3 to 3.2.0. Contributed by Takanobu Asanuma. 
(cherry picked from commit 34b2521f5e7388e6a4d76af537ef147206fd9b72)
 2018-10-25 13:53:24 -07:00
Wei-Chiu Chuang
ef9f8ca13d HADOOP-15864. Job submitter / executor fail when SBN domain name can not resolved. Contributed by He Xiaoqiao. 
(cherry picked from commit fb2b72e6fce019130e10964a644b94cddbab1c06)
(cherry picked from commit fd02c501c0cea3ec55956e11b390111519cedc00)
 2018-10-25 09:37:31 -07:00
Inigo Goiri
4b1f2ecd4c HDFS-14003. Fix findbugs warning in trunk for FSImageFormatPBINode. Contributed by Yiqun Lin. 
(cherry picked from commit 0e56c883cd2310f3ff9d62afb306b1ab27419c36)
 2018-10-24 16:14:14 -07:00
Xiao Chen
96cedb87b9 Fix potential FSImage corruption. Contributed by Daryn Sharp and Vinayakumar B. 
(cherry picked from commit f1996ccbaee734d423caa9d47a571cfff98ef42c)
 2018-10-24 16:10:25 -07:00
bibinchundatt
4167275e89 MAPREDUCE-7151. RMContainerAllocator#handleJobPriorityChange expects application_priority always. Contributed by Bilwa S T. 
(cherry picked from commit 74a5e683fe65ee624a094de77940a42777ed7a26)
 2018-10-25 00:27:40 +05:30
Inigo Goiri
2e43966b56 HDFS-14021. TestReconstructStripedBlocksWithRackAwareness#testReconstructForNotEnoughRacks fails intermittently. Contributed by Xiao Chen. 
(cherry picked from commit c1874046e2f9275ba330090fcf12c5611c6d6fc4)
 2018-10-24 09:38:40 -07:00
Akira Ajisaka
080e9d0f9b
HADOOP-15822. zstd compressor can fail with a small output buffer. Contributed by Jason Lowe. 
(cherry picked from commit 8f97d6f2cdfccefba5457ae3d561e9ce0109da3f)

Conflicts:
	dev-support/docker/Dockerfile
 2018-10-24 13:54:30 +09:00
Jason Lowe
3be72b7aa2 YARN-8904. TestRMDelegationTokens can fail in testRMDTMasterKeyStateOnRollingMasterKey. Contributed by Wilfred Spiegelenburg 
(cherry picked from commit 93fb3b4b9cbc63e65d590676a5a318d555a25904)
 2018-10-23 12:55:48 -05:00
Rohith Sharma K S
3e3b088856 YARN-8826. Fix lingering timeline collector after serviceStop in TimelineCollectorManager. Contributed by Prabha Manepalli. 
(cherry picked from commit 0b62983c5a9361eb832784f134f140f9926c9ec6)
 2018-10-23 14:08:06 +05:30
Sunil G
0cb184d6e9 YARN-8868. Set HTTPOnly attribute to Cookie. Contributed by Chandni Singh. 
(cherry picked from commit 2202e00ba8a44ad70f0a90e6c519257e3ae56a36)
 2018-10-23 09:58:05 +05:30
Arpit Agarwal
8548864ccb HDFS-13941. make storageId in BlockPoolTokenSecretManager.checkAccess optional. Contributed by Ajay Kumar. 
(cherry picked from commit c58811c77d5c0442c404a5b2876e09eaf6d16073)
 2018-10-22 14:44:58 -07:00
Steve Loughran
f8c7073bd0
HADOOP-15866. Renamed HADOOP_SECURITY_GROUP_SHELL_COMMAND_TIMEOUT keys break compatibility. 
Contributed by Wei-Chiu Chuang.

(cherry picked from commit cb8d11e7bf5055508949a6a4f074efacc8cb1305)
 2018-10-22 11:08:47 +01:00
Sunil G
450f231a53 YARN-7502. Nodemanager restart docs should describe nodemanager supervised property. Contributed by Suma Shivaprasad. 
(cherry picked from commit e50334513c781a10368e7353bec6674f89eb0a7b)
 2018-10-22 14:16:54 +05:30
Ted Yu
84726679f0 HADOOP-15850. CopyCommitter#concatFileChunks should check that the blocks per chunk is not 0. Contributed by Ted Yu. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit e2cecb681e2aab8b7c5465719cac53dce407a64c)
(cherry picked from commit a7dd244a4995cbf20feb054530fc5913ccf384ac)
 2018-10-19 13:22:31 -07:00
Eric Yang
e86efa8712 YARN-8910. Fixed misleading log statement when container max retries is infinite. 
Contributed by Chandni Singh

(cherry picked from commit 47ad98b2e1b4e9ad9304aca4d452da428778e7ee)
 2018-10-19 13:50:32 -04:00
Wangda Tan
c11d2341af YARN-8908. Fix errors in yarn-default.xml related to GPU/FPGA. (Zhankun Tang via wangda) 
Change-Id: I46f511a9b51b65b069e15d5b827530110c41edab
(cherry picked from commit 071b9e19aa2c17bb9fa046c6ddabd6b8f0cc6d14)
 2018-10-19 09:55:27 -07:00
Takanobu Asanuma
bad5d81ac1 HADOOP-15804. upgrade to commons-compress 1.18. Contributed by Akira Ajisaka. 
(cherry picked from commit 9bd18324c7801472409d9ad69ea365aa7a33a9c4)
 2018-10-19 21:40:59 +09:00
Weiwei Yang
beca90ece8 YARN-8907. Fix incorrect logging message in TestCapacityScheduler. Contributed by Zhankun Tang. 
(cherry picked from commit 13cc0f50ea1c5f8978455e34f49716ddb0e3a143)
 2018-10-19 10:02:46 +08:00
Giovanni Matteo Fumarola
92c6bde6be HDFS-14005. RBF: Web UI update to bootstrap-3.3.7. Contributed by Inigo Goiri. 2018-10-18 14:28:19 -07:00
Wangda Tan
d381f32219 YARN-6098. Add documentation for Delete Queue. (Suma Shivaprasad via wangda) 
Change-Id: I23bf7ca0acbe1168a7f805ceccdd62fe41bfeb35
(cherry picked from commit bfb88b10f46a265aa38ab3e1d87b6a0a99d94be8)
 2018-10-18 12:29:28 -07:00
Wangda Tan
46baafedf1 YARN-8896. Limit the maximum number of container assignments per heartbeat. (Zhankun Tang via wangda) 
Change-Id: I6e72f8362bd7f5c2a844cb9e3c4732492314e9f1
(cherry picked from commit 780be14f07df2a3ed6273b96ae857c278fd72718)
 2018-10-18 12:29:19 -07:00
Eric Yang
3cb52958b2 YARN-8687. Update YARN service file type in documentation. 
Contributed by Suma Shivaprasad
 2018-10-18 12:04:27 -04:00
Inigo Goiri
16e1ab4c99 HDFS-14000. RBF: Documentation should reflect right scripts for v3.0 and above. Contributed by CR Hota. 
(cherry picked from commit 7347fa2df3cfd48857f4ded3539a60101d026daf)
 2018-10-17 16:42:32 -07:00
Jason Lowe
65b27f8ed2 HADOOP-15859. ZStandardDecompressor.c mistakes a class for an instance. Contributed by Jason Lowe 
(cherry picked from commit 9abda83947a5babfe5a650b3409ad952f6782105)
 2018-10-17 14:43:19 -05:00
Steve Loughran
9c350785d4
HADOOP-15861. Move DelegationTokenIssuer to the right path. 
Contributed by Wei-Chiu Chuang.

(cherry picked from commit 41b3603b5bcb74a7d78a314a4a5c177d941af27f)
 2018-10-17 11:03:22 +01:00
Akira Ajisaka
ddd7da52c9
HADOOP-15802. start-build-env.sh creates an invalid /etc/sudoers.d/hadoop-build-${USER_ID} file entry. Contributed by Jon Boone. 
(cherry picked from commit e3342a1abaff71823ebd952baf24a6143e711b99)
 2018-10-17 10:14:49 +09:00