Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,842 Commits 326 Branches 371 Tags
Commit Graph

18842 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Xiao Chen
96cedb87b9 Fix potential FSImage corruption. Contributed by Daryn Sharp and Vinayakumar B. 
(cherry picked from commit f1996ccbaee734d423caa9d47a571cfff98ef42c)
 2018-10-24 16:10:25 -07:00
bibinchundatt
4167275e89 MAPREDUCE-7151. RMContainerAllocator#handleJobPriorityChange expects application_priority always. Contributed by Bilwa S T. 
(cherry picked from commit 74a5e683fe65ee624a094de77940a42777ed7a26)
 2018-10-25 00:27:40 +05:30
Inigo Goiri
2e43966b56 HDFS-14021. TestReconstructStripedBlocksWithRackAwareness#testReconstructForNotEnoughRacks fails intermittently. Contributed by Xiao Chen. 
(cherry picked from commit c1874046e2f9275ba330090fcf12c5611c6d6fc4)
 2018-10-24 09:38:40 -07:00
Akira Ajisaka
080e9d0f9b
HADOOP-15822. zstd compressor can fail with a small output buffer. Contributed by Jason Lowe. 
(cherry picked from commit 8f97d6f2cdfccefba5457ae3d561e9ce0109da3f)

Conflicts:
	dev-support/docker/Dockerfile
 2018-10-24 13:54:30 +09:00
Jason Lowe
3be72b7aa2 YARN-8904. TestRMDelegationTokens can fail in testRMDTMasterKeyStateOnRollingMasterKey. Contributed by Wilfred Spiegelenburg 
(cherry picked from commit 93fb3b4b9cbc63e65d590676a5a318d555a25904)
 2018-10-23 12:55:48 -05:00
Rohith Sharma K S
3e3b088856 YARN-8826. Fix lingering timeline collector after serviceStop in TimelineCollectorManager. Contributed by Prabha Manepalli. 
(cherry picked from commit 0b62983c5a9361eb832784f134f140f9926c9ec6)
 2018-10-23 14:08:06 +05:30
Sunil G
0cb184d6e9 YARN-8868. Set HTTPOnly attribute to Cookie. Contributed by Chandni Singh. 
(cherry picked from commit 2202e00ba8a44ad70f0a90e6c519257e3ae56a36)
 2018-10-23 09:58:05 +05:30
Arpit Agarwal
8548864ccb HDFS-13941. make storageId in BlockPoolTokenSecretManager.checkAccess optional. Contributed by Ajay Kumar. 
(cherry picked from commit c58811c77d5c0442c404a5b2876e09eaf6d16073)
 2018-10-22 14:44:58 -07:00
Steve Loughran
f8c7073bd0
HADOOP-15866. Renamed HADOOP_SECURITY_GROUP_SHELL_COMMAND_TIMEOUT keys break compatibility. 
Contributed by Wei-Chiu Chuang.

(cherry picked from commit cb8d11e7bf5055508949a6a4f074efacc8cb1305)
 2018-10-22 11:08:47 +01:00
Sunil G
450f231a53 YARN-7502. Nodemanager restart docs should describe nodemanager supervised property. Contributed by Suma Shivaprasad. 
(cherry picked from commit e50334513c781a10368e7353bec6674f89eb0a7b)
 2018-10-22 14:16:54 +05:30
Ted Yu
84726679f0 HADOOP-15850. CopyCommitter#concatFileChunks should check that the blocks per chunk is not 0. Contributed by Ted Yu. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit e2cecb681e2aab8b7c5465719cac53dce407a64c)
(cherry picked from commit a7dd244a4995cbf20feb054530fc5913ccf384ac)
 2018-10-19 13:22:31 -07:00
Eric Yang
e86efa8712 YARN-8910. Fixed misleading log statement when container max retries is infinite. 
Contributed by Chandni Singh

(cherry picked from commit 47ad98b2e1b4e9ad9304aca4d452da428778e7ee)
 2018-10-19 13:50:32 -04:00
Wangda Tan
c11d2341af YARN-8908. Fix errors in yarn-default.xml related to GPU/FPGA. (Zhankun Tang via wangda) 
Change-Id: I46f511a9b51b65b069e15d5b827530110c41edab
(cherry picked from commit 071b9e19aa2c17bb9fa046c6ddabd6b8f0cc6d14)
 2018-10-19 09:55:27 -07:00
Takanobu Asanuma
bad5d81ac1 HADOOP-15804. upgrade to commons-compress 1.18. Contributed by Akira Ajisaka. 
(cherry picked from commit 9bd18324c7801472409d9ad69ea365aa7a33a9c4)
 2018-10-19 21:40:59 +09:00
Weiwei Yang
beca90ece8 YARN-8907. Fix incorrect logging message in TestCapacityScheduler. Contributed by Zhankun Tang. 
(cherry picked from commit 13cc0f50ea1c5f8978455e34f49716ddb0e3a143)
 2018-10-19 10:02:46 +08:00
Giovanni Matteo Fumarola
92c6bde6be HDFS-14005. RBF: Web UI update to bootstrap-3.3.7. Contributed by Inigo Goiri. 2018-10-18 14:28:19 -07:00
Wangda Tan
d381f32219 YARN-6098. Add documentation for Delete Queue. (Suma Shivaprasad via wangda) 
Change-Id: I23bf7ca0acbe1168a7f805ceccdd62fe41bfeb35
(cherry picked from commit bfb88b10f46a265aa38ab3e1d87b6a0a99d94be8)
 2018-10-18 12:29:28 -07:00
Wangda Tan
46baafedf1 YARN-8896. Limit the maximum number of container assignments per heartbeat. (Zhankun Tang via wangda) 
Change-Id: I6e72f8362bd7f5c2a844cb9e3c4732492314e9f1
(cherry picked from commit 780be14f07df2a3ed6273b96ae857c278fd72718)
 2018-10-18 12:29:19 -07:00
Eric Yang
3cb52958b2 YARN-8687. Update YARN service file type in documentation. 
Contributed by Suma Shivaprasad
 2018-10-18 12:04:27 -04:00
Inigo Goiri
16e1ab4c99 HDFS-14000. RBF: Documentation should reflect right scripts for v3.0 and above. Contributed by CR Hota. 
(cherry picked from commit 7347fa2df3cfd48857f4ded3539a60101d026daf)
 2018-10-17 16:42:32 -07:00
Jason Lowe
65b27f8ed2 HADOOP-15859. ZStandardDecompressor.c mistakes a class for an instance. Contributed by Jason Lowe 
(cherry picked from commit 9abda83947a5babfe5a650b3409ad952f6782105)
 2018-10-17 14:43:19 -05:00
Steve Loughran
9c350785d4
HADOOP-15861. Move DelegationTokenIssuer to the right path. 
Contributed by Wei-Chiu Chuang.

(cherry picked from commit 41b3603b5bcb74a7d78a314a4a5c177d941af27f)
 2018-10-17 11:03:22 +01:00
Akira Ajisaka
ddd7da52c9
HADOOP-15802. start-build-env.sh creates an invalid /etc/sudoers.d/hadoop-build-${USER_ID} file entry. Contributed by Jon Boone. 
(cherry picked from commit e3342a1abaff71823ebd952baf24a6143e711b99)
 2018-10-17 10:14:49 +09:00
Eric Yang
7c40f9b37c YARN-8810. Fixed a YARN service bug in comparing ConfigFile object. 
Contributed by Chandni Singh
 2018-10-16 18:58:28 -04:00
Wangda Tan
3e1894c376 YARN-8892. YARN UI2 doc changes to update security status (verified under security environment). (Sunil G via wangda) 
Change-Id: I8bc8622936861b8d6de3e42a0b75af86ad8a3961
(cherry picked from commit 538250db26ce0b261bb74053348cddfc2d65cf52)
 2018-10-16 14:36:36 -07:00
Xiao Chen
30526f2c5c MAPREDUCE-7132. JobSplitWriter prints unnecessary warnings if EC(RS10,4) is used. Contributed by Peter Bacsko. 
(cherry picked from commit 25f8fcb06476938826cdc92858a61124b18cd98d)
 2018-10-16 10:24:24 -07:00
Xiao Chen
d1749fbaae HADOOP-14445. Addendum: Use DelegationTokenIssuer to create KMS delegation tokens that can authenticate to all KMS instances. 
(cherry picked from commit b6fc72a0250ac3f2341ebe8a14d19b073e6224c8)
 2018-10-15 11:00:30 -07:00
Sunil G
38e187384b YARN-8869. YARN Service Client might not work correctly with RM REST API for Kerberos authentication. Contributed by Eric Yang. 
(cherry picked from commit fa94d370b6e3cba9c7560c09b517583e6652f103)
 2018-10-15 21:23:07 +05:30
Xiao Chen
6a1ce74fb1 HADOOP-14445. Use DelegationTokenIssuer to create KMS delegation tokens that can authenticate to all KMS instances. 
Contributed by Daryn Sharp, Xiao Chen, Rushabh S Shah.

(cherry picked from commit 5ec86b445cc492f52c33639efb6a09a0d2f27475)
(cherry picked from commit e93e401f492c7fc112da3d0c63b3b186c1b196ec)

 Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/kms/KMSClientProvider.java
 2018-10-12 12:08:26 -07:00
Inigo Goiri
6342a7cb98 HDFS-13802. RBF: Remove FSCK from Router Web UI. Contributed by Fei Hui. 
(cherry picked from commit f63ee083db7c261207b726f0f44be2e5a92124f3)
 2018-10-12 10:11:24 -07:00
Vinayakumar B
bed2f407a9 HDFS-13945. TestDataNodeVolumeFailure is Flaky. Contributed by Ayush Saxena. 
(cherry picked from commit 6e0e6daaf3215213c32cdffa79f3730d40e981ea)
 2018-10-12 17:36:03 +05:30
Steve Loughran
f1fbc56371
HADOOP-15837. DynamoDB table Update can fail S3A FS init. 
Contributed by Steve Loughran.
 2018-10-12 09:48:17 +01:00
Sunil G
52d270ef8c YARN-8753. [UI2] Lost nodes representation missing from Nodemanagers Chart. Contributed by Yesha Vora. 
(cherry picked from commit 8a379839909197690af170afc13c49ef7aa54286)
 2018-10-11 16:16:26 +05:30
Sunil G
fd9cd5f3c1 YARN-8666. [UI2] Remove application tab from YARN Queue Page. Contributed by Yesha Vora. 
(cherry picked from commit d3da741fa27c14f3d1391a76d6ade5d1a8365aa5)
 2018-10-11 16:07:30 +05:30
Steve Loughran
2a305efd4e
HADOOP-15839. Review + update cloud store sensitive keys in hadoop.security.sensitive-config-keys. 
Contributed by Steve Loughran.

(cherry picked from commit cdc4350718055189fef8c70e31314607001d4009)
 2018-10-10 17:03:06 +01:00
Weiwei Yang
a0060cf8ee Revert "YARN-8468. Enable the use of queue based maximum container allocation limit and implement it in FairScheduler. Contributed by Antal Bálint Steinbach." 
This reverts commit ce4a0898df5e9891f0388aa65a754f0ff85b56ac.
 2018-10-10 21:41:00 +08:00
Hrishikesh Gadre
323b76bccf HDFS-13926. ThreadLocal aggregations for FileSystem.Statistics are incorrect with striped reads. 
Contributed by Xiao Chen, Hrishikesh Gadre.

Signed-off-by: Xiao Chen <xiao@apache.org>
 2018-10-09 19:54:34 -07:00
Weiwei Yang
3968ce1073 YARN-8858. CapacityScheduler should respect maximum node resource when per-queue maximum-allocation is being used. Contributed by Wangda Tan. 
(cherry picked from commit edce866489d83744f3f47a3b884b0c6136885e4a)
 2018-10-10 09:48:56 +08:00
Eric Yang
5813c1de56 YARN-8845. Removed unused hadoop.registry.rm reference. 
Contributed by Íñigo Goiri

(cherry picked from commit bf04f194568f9e81f5481b25a84ad903e3c307cf)
 2018-10-09 15:56:38 -04:00
Todd Lipcon
89d4481020 HADOOP-15550. Avoid static initialization of ObjectMappers 
(cherry picked from commit 7a3c6e9c3cd9ffdc71946fd12f5c3d59718c4939)
 2018-10-09 13:50:00 -05:00
Sunil G
be0ce74c71 YARN-8853. [UI2] Application Attempts tab is not shown correctly when there are no attempts. Contributed by Akhil PB. 
(cherry picked from commit 6f11919beb71f3bab7a49f2866c03bca5a4afc3b)
 2018-10-09 22:06:21 +05:30
Weiwei Yang
ce4a0898df YARN-8468. Enable the use of queue based maximum container allocation limit and implement it in FairScheduler. Contributed by Antal Bálint Steinbach. 2018-10-09 22:30:42 +08:00
Yiqun Lin
665036c5f7 HDFS-13962. Add null check for add-replica pool to avoid lock acquiring. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit 1043795f7fe44c98a34f8ea3cea708c801c3043b)
 2018-10-09 10:36:45 +08:00
Sunil G
dd445e0368 YARN-7825. [UI2] Maintain constant horizontal application info bar for Application Attempt page. Contributed by Akhil PB. 
(cherry picked from commit 228dc192f1108438e2a221ec2ed7b6fd44080295)
 2018-10-08 11:56:28 +05:30
Inigo Goiri
9bbf494c7f HDFS-13964. RBF: TestRouterWebHDFSContractAppend fails with No Active Namenode under nameservice. Contributed by Ayush Saxena. 
(cherry picked from commit 3bfff0f79e7246cbb2989c720c14cfa2566a9d1a)
 2018-10-06 10:41:49 -07:00
Jason Lowe
df7ec58ced HADOOP-15820. ZStandardDecompressor native code sets an integer field as a long. Contributed by Jason Lowe 
(cherry picked from commit f13e231025333ebf80b30bbdce1296cef554943b)
 2018-10-05 09:27:23 -05:00
Sunil G
f3f0a9a2ca YARN-8797. [UI2] Improve error pages in new YARN UI. Contributed by Akhil PB. 
(cherry picked from commit 94d82f416258b65c8ab7849a459af78abebdf7d0)
 2018-10-05 14:34:55 +05:30
Sunil G
6407f2a0ed YARN-7957. [UI2] YARN service delete option disappears after stopping application. Contributed by Akhil PB. 
(cherry picked from commit 751f626e506b6cf8cc87ca65b65550871dca74fb)
 2018-10-05 14:10:57 +05:30
Sunil G
fae24b4747 Revert "YARN-7957. [UI2] YARN service delete option disappears after stopping application. Contributed by Akhil PB." 
This reverts commit dd70029255e9621675f2bcf1083d787d24896733.
 2018-10-05 13:38:21 +05:30
Sunil G
dd70029255 YARN-7957. [UI2] YARN service delete option disappears after stopping application. Contributed by Akhil PB. 
(cherry picked from commit 751f626e506b6cf8cc87ca65b65550871dca74fb)
 2018-10-05 12:29:33 +05:30