Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
20,285 Commits 326 Branches 371 Tags
Commit Graph

20285 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Weiwei Yang
8720814349 HADOOP-15907. Add missing maven modules in BUILDING.txt. Contributed Wanqiang Ji. 
(cherry picked from commit e6444f1c640cda86e1c97fbfebf68de92a162c95)
 2018-11-07 16:46:41 +08:00
Weiwei Yang
4ff7255d6e YARN-8976. Remove redundant modifiers in interface ApplicationConstants. Contributed by Zhankun Tang. 
(cherry picked from commit 482716e5a4d1edfd3aa6a1ae65a58652f89375f1)
 2018-11-07 10:49:09 +08:00
Akira Ajisaka
41c4e9583d
YARN-8233. NPE in CapacityScheduler#tryCommit when handling allocate/reserve proposal whose allocatedOrReservedContainer is null. Contributed by Tao Yang. 
(cherry picked from commit 951c98f89059d64fda8456366f680eff4a7a6785)
 2018-11-07 11:18:48 +09:00
Jason Lowe
ff5a2cb5c9 MAPREDUCE-7156. NullPointerException when reaching max shuffle connections. Contributed by Peter Bacsko 
(cherry picked from commit ba1f9d66d94ed0b85084d7c40c09a87478b3a05a)
 2018-11-06 17:57:09 -06:00
Inigo Goiri
e7fba62e6a HDFS-14048. DFSOutputStream close() throws exception on subsequent call after DataNode restart. Contributed by Erik Krogen. 
(cherry picked from commit 887244de4adebe27693ed4ad3296a6f700cfa8c1)
 2018-11-06 11:20:06 -08:00
Jason Lowe
9265934201 YARN-8865. RMStateStore contains large number of expired RMDelegationToken. Contributed by Wilfred Spiegelenburg 
(cherry picked from commit ab6aa4c7265db5bcbb446c2f779289023d454b81)
 2018-11-06 08:47:30 -06:00
Weiwei Yang
f00125e2d9 YARN-8970. Improve the debug message in CS#allocateContainerOnSingleNode. Contributed by Zhankun Tang. 
(cherry picked from commit 5d6554c722f08f79bce904e021243605ee75bae3)
 2018-11-06 14:52:10 +08:00
Weiwei Yang
b474239e0b YARN-8969. AbstractYarnScheduler#getNodeTracker should return generic type to avoid type casting. Contributed by Wanqiang Ji. 
(cherry picked from commit c7fcca0d7ec9e31d43ef3040ecd576ec808f1f8b)
 2018-11-06 13:17:19 +08:00
Inigo Goiri
378f189c4f HDFS-14043. Tolerate corrupted seen_txid file. Contributed by Lukas Majercak. 
(cherry picked from commit f3296501e09fa7f1e81548dfcefa56f20fe337ca)
 2018-11-05 16:49:12 -08:00
Giovanni Matteo Fumarola
6e1fad299a HDFS-14042. Fix NPE when PROVIDED storage is missing. Contributed by Virajith Jalaparti. 2018-11-05 11:33:32 -08:00
Akira Ajisaka
fcd221fed7
HADOOP-15900. Update JSch versions in LICENSE.txt. 
(cherry picked from commit d43cc5db0ff80958ca873df1dc2fa00054e37175)
 2018-11-05 17:52:04 +09:00
Akira Ajisaka
ea6b9d6c61
HADOOP-15899. Update AWS Java SDK versions in NOTICE.txt. 
(cherry picked from commit a5e21cd6a450990d00eb7515d579b51d2a0bff82)
 2018-11-05 15:29:06 +09:00
Lars Francke
a15bcf97a4
HADOOP-15687. Credentials class should allow access to aliases. 
Author:    Lars Francke <lars.francke@gmail.com>

(cherry picked from commit cb8d679c95642842efacc5d38ccf2a61b043c689)
 2018-11-03 16:29:56 +00:00
Jonathan Hung
b9a3c988c0 YARN-7225. Add queue and partition info to RM audit log. Contributed by Eric Payne 
(cherry picked from commit 2ab611d48b7669b31bd2c9b918f47251da77d0f6)
 2018-11-01 14:31:09 -07:00
Sunil G
856dd1208e HDFS-14033. [libhdfs++] Disable libhdfs++ build on systems that do not support thread_local. Contributed by Anatoli Shein. 
(cherry picked from commit 9c438abe52d4ee0b25345a4b7ec1697dd66f85e9)
 2018-10-31 12:33:55 +05:30
Steve Loughran
d79ea3844c
HADOOP-15855. Review hadoop credential doc, including object store details. 
Contributed by Steve Loughran.

(cherry picked from commit 62d98ca92aee15d1790d169bfdf0043b05b748ce)
 2018-10-30 15:58:44 +00:00
Rohith Sharma K S
7920f79049 YARN-8950. Fix compilation issue due to dependency convergence error for hbase.profile=2.0. 
(cherry picked from commit 4ec4ec69711180d642c5b56cd3d3dbdf44d3c61f)
 2018-10-30 11:50:33 +05:30
Xiao Chen
8788489df4 HDFS-14027. DFSStripedOutputStream should implement both hsync methods. 
(cherry picked from commit db7e636824a36b90ba1c8e9b2fba1162771700fe)
 2018-10-29 19:13:43 -07:00
Weiwei Yang
288bc3f1eb YARN-8915. Update the doc about the default value of maximum-container-assignments for capacity scheduler. Contributed by Zhankun Tang. 
(cherry picked from commit 78ea897b6701c82799a6231103ca7ee1f8b4efde)
 2018-10-29 12:05:45 +08:00
Weiwei Yang
5db4272d91 YARN-8944. TestContainerAllocation.testUserLimitAllocationMultipleContainers failure after YARN-8896. Contributed by Wilfred Spiegelenburg. 
(cherry picked from commit 1d90a0dd23c5d1bd52d04c303506806d0dc61cd2)
 2018-10-29 11:55:22 +08:00
Wei-Chiu Chuang
5f8dd40647 Revert "HADOOP-15864. Job submitter / executor fail when SBN domain name can not resolved. Contributed by He Xiaoqiao." 
This reverts commit fd02c501c0cea3ec55956e11b390111519cedc00.
 2018-10-28 17:05:45 -07:00
Anu Engineer
3cbb028dd2 HDFS-14028. HDFS OIV temporary dir deletes folder. 
Contributed by Adam Antal.

(cherry picked from commit 4f10d7e23fc0b757a9e94bc448187a6211b90f10)
 2018-10-26 16:47:53 -07:00
Sunil G
c61b00f307 HADOOP-15815. Upgrade Eclipse Jetty version to 9.3.24. Cotributed by Boris Vulikh. 
(cherry picked from commit e28c00c290a06813d156a845b93e806f71413dbf)
 2018-10-26 13:46:57 +05:30
Sammi Chen
ca22bf175f HADOOP-15868. AliyunOSS: update document for properties of multiple part download, multiple part upload and directory copy. Contributed by Jinhu Wu. 
(cherry picked from commit 7574d18538e838f40581519080d7c8621c65e53b)
(cherry picked from commit 366541d834f70fd6f8d4c5296a9e844236c6fd74)
(cherry picked from commit c5a227062fae33ce3137161802d6f090a6af010e)
 2018-10-26 15:33:05 +08:00
Arpit Agarwal
642b613a7d HDFS-14029. Sleep in TestLazyPersistFiles should be put into a loop. Contributed by Adam Antal. 
(cherry picked from commit 9b899f1ebd5126a756fceb43459164165488b203)
 2018-10-25 15:14:58 -07:00
Bharat Viswanadham
3d6a6ce019 HADOOP-15882. Upgrade maven-shade-plugin from 2.4.3 to 3.2.0. Contributed by Takanobu Asanuma. 
(cherry picked from commit 34b2521f5e7388e6a4d76af537ef147206fd9b72)
 2018-10-25 14:06:56 -07:00
Wei-Chiu Chuang
fd02c501c0 HADOOP-15864. Job submitter / executor fail when SBN domain name can not resolved. Contributed by He Xiaoqiao. 
(cherry picked from commit fb2b72e6fce019130e10964a644b94cddbab1c06)
 2018-10-25 09:36:28 -07:00
Inigo Goiri
5b995f85f5 HDFS-14003. Fix findbugs warning in trunk for FSImageFormatPBINode. Contributed by Yiqun Lin. 
(cherry picked from commit 0e56c883cd2310f3ff9d62afb306b1ab27419c36)
 2018-10-24 16:14:06 -07:00
bibinchundatt
aed71316e5 MAPREDUCE-7151. RMContainerAllocator#handleJobPriorityChange expects application_priority always. Contributed by Bilwa S T. 
(cherry picked from commit 74a5e683fe65ee624a094de77940a42777ed7a26)
 2018-10-25 00:18:39 +05:30
Inigo Goiri
b6937e8e70 HDFS-14021. TestReconstructStripedBlocksWithRackAwareness#testReconstructForNotEnoughRacks fails intermittently. Contributed by Xiao Chen. 
(cherry picked from commit c1874046e2f9275ba330090fcf12c5611c6d6fc4)
 2018-10-24 09:38:32 -07:00
Eric Yang
3665c46176 YARN-8938. Updated YARN service upgrade document. 
Contributed by Chandni Singh

(cherry picked from commit bbc6dcd3d0976932a49d8650804fb0a4018b3a02)
 2018-10-24 11:52:50 -04:00
Akira Ajisaka
63396beab4
HADOOP-15822. zstd compressor can fail with a small output buffer. Contributed by Jason Lowe. 
(cherry picked from commit 8f97d6f2cdfccefba5457ae3d561e9ce0109da3f)
 2018-10-24 13:50:30 +09:00
Jason Lowe
7097755925 YARN-8904. TestRMDelegationTokens can fail in testRMDTMasterKeyStateOnRollingMasterKey. Contributed by Wilfred Spiegelenburg 
(cherry picked from commit 93fb3b4b9cbc63e65d590676a5a318d555a25904)
 2018-10-23 12:54:04 -05:00
Rohith Sharma K S
660fff3138 YARN-8826. Fix lingering timeline collector after serviceStop in TimelineCollectorManager. Contributed by Prabha Manepalli. 
(cherry picked from commit 0b62983c5a9361eb832784f134f140f9926c9ec6)
 2018-10-23 14:06:13 +05:30
Sunil G
30998fea28 YARN-8868. Set HTTPOnly attribute to Cookie. Contributed by Chandni Singh. 
(cherry picked from commit 2202e00ba8a44ad70f0a90e6c519257e3ae56a36)
 2018-10-23 09:57:26 +05:30
Arpit Agarwal
28a1cabb72 HDFS-13941. make storageId in BlockPoolTokenSecretManager.checkAccess optional. Contributed by Ajay Kumar. 
(cherry picked from commit c58811c77d5c0442c404a5b2876e09eaf6d16073)
 2018-10-22 14:45:11 -07:00
Steve Loughran
85bf008405
HADOOP-15866. Renamed HADOOP_SECURITY_GROUP_SHELL_COMMAND_TIMEOUT keys break compatibility. 
Contributed by Wei-Chiu Chuang.

(cherry picked from commit cb8d11e7bf5055508949a6a4f074efacc8cb1305)
 2018-10-22 11:08:30 +01:00
Sunil G
8724c2900f YARN-7502. Nodemanager restart docs should describe nodemanager supervised property. Contributed by Suma Shivaprasad. 
(cherry picked from commit e50334513c781a10368e7353bec6674f89eb0a7b)
 2018-10-22 14:16:24 +05:30
Ted Yu
a7dd244a49 HADOOP-15850. CopyCommitter#concatFileChunks should check that the blocks per chunk is not 0. Contributed by Ted Yu. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit e2cecb681e2aab8b7c5465719cac53dce407a64c)
 2018-10-19 13:22:01 -07:00
Eric Yang
04600cf5da YARN-8910. Fixed misleading log statement when container max retries is infinite. 
Contributed by Chandni Singh

(cherry picked from commit 47ad98b2e1b4e9ad9304aca4d452da428778e7ee)
 2018-10-19 13:49:51 -04:00
Wangda Tan
9ed9e185d7 YARN-8916. Define a constant docker string in ContainerRuntimeConstants.java for better maintainability. (Zhankun Tang via wangda) 
Change-Id: I1349e740037f81afdbe30edbe741f20e88fd0a90
(cherry picked from commit 5e02b4915b876eab943b30cf728cfc26ba832188)
 2018-10-19 09:50:18 -07:00
Wangda Tan
bd08554aa4 YARN-8908. Fix errors in yarn-default.xml related to GPU/FPGA. (Zhankun Tang via wangda) 
Change-Id: I46f511a9b51b65b069e15d5b827530110c41edab
(cherry picked from commit 071b9e19aa2c17bb9fa046c6ddabd6b8f0cc6d14)
 2018-10-19 09:50:12 -07:00
Takanobu Asanuma
ef073547dc HADOOP-15804. upgrade to commons-compress 1.18. Contributed by Akira Ajisaka. 
(cherry picked from commit 9bd18324c7801472409d9ad69ea365aa7a33a9c4)
 2018-10-19 21:40:32 +09:00
Weiwei Yang
042f2df19b YARN-8907. Fix incorrect logging message in TestCapacityScheduler. Contributed by Zhankun Tang. 
(cherry picked from commit 13cc0f50ea1c5f8978455e34f49716ddb0e3a143)
 2018-10-19 10:01:45 +08:00
Giovanni Matteo Fumarola
0b6a7b5f16 HDFS-14007. Incompatible layout when generating FSImage. Contributed by Inigo Goiri. 2018-10-18 17:26:22 -07:00
Inigo Goiri
c0f618f3d5 HDFS-14002. TestLayoutVersion#testNameNodeFeatureMinimumCompatibleLayoutVersions fails. Contributed by Takanobu Asanuma. 
(cherry picked from commit be1cffb0854cb28967beb062e9db7d61e6eeff1d)
 2018-10-18 17:14:56 -07:00
Eric Yang
91c916e209 YARN-8618. Added detection for non-upgradable service. 
Contributed by Chandni Singh

(cherry picked from commit 66f059ed1db93c0a86e86c64636013001169a677)
 2018-10-18 20:00:50 -04:00
Giovanni Matteo Fumarola
1810f2392f HDFS-14005. RBF: Web UI update to bootstrap-3.3.7. Contributed by Inigo Goiri. 2018-10-18 14:35:20 -07:00
Wangda Tan
7252f8e117 YARN-8896. Limit the maximum number of container assignments per heartbeat. (Zhankun Tang via wangda) 
Change-Id: I6e72f8362bd7f5c2a844cb9e3c4732492314e9f1
(cherry picked from commit 780be14f07df2a3ed6273b96ae857c278fd72718)
 2018-10-18 12:12:19 -07:00
Wangda Tan
5f7ed043c8 YARN-8456. Fix a configuration handling bug when user leave FPGA discover executable path configuration default but set OpenCL SDK path environment variable. (Zhankun Tang via wangda) 
Change-Id: Iff150ea98ba0c60d448474fd940eb121afce6965
(cherry picked from commit a457a8951a1b35f06811c40443ca44bb9c698c30)
 2018-10-18 12:12:14 -07:00