Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,162 Commits 326 Branches 371 Tags
Commit Graph

17162 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Sunil G
6d814ae068 YARN-7383. Node resource is not parsed correctly for resource names containing dot. Contributed by Gergely Novák. 
(cherry picked from commit 10fc8d2a7dc975ce70de63f88a2674389314f197)
 2017-12-13 22:01:28 +05:30
Weiwei Yang
415c69b36f YARN-7647. NM print inappropriate error log when node-labels is enabled. Contributed by Yang Wang. 2017-12-13 13:18:54 +08:00
Jason Lowe
472c7edc62 YARN-7595. Container launching code suppresses close exceptions after writes. Contributed by Jim Brennan 
(cherry picked from commit 2abab1d7c53e64c160384fd5a3ac4cd8ffa57af4)
 2017-12-12 16:07:47 -06:00
Wei-Chiu Chuang
8062a7428d HDFS-12891. Do not invalidate blocks if toInvalidate is empty. Contributed by Zsolt Venczel. 
(cherry picked from commit 55fc2d6485702a99c6d4bb261a720d1f0498af2b)
 2017-12-11 16:43:42 -08:00
Inigo Goiri
6a43fbe724 HDFS-12875. RBF: Complete logic for -readonly option of dfsrouteradmin add command. Contributed by Inigo Goiri. 
(cherry picked from commit 5cd1056ad77a2ebb0466e7bc5933337b57f6fe30)
 2017-12-11 15:16:19 -08:00
Weiwei Yang
3dc1ea6050 YARN-7608. Incorrect sTarget column causing DataTable warning on RM application and scheduler web page. Contributed by Gergely Novák. 2017-12-11 10:33:46 +08:00
Wangda Tan
41796ea60a YARN-7591. NPE in async-scheduling mode of CapacityScheduler. (Tao Yang via wangda) 
Change-Id: I67e086d6d35c3166ec0f6305490ea6527bd6d799
 2017-12-08 21:43:28 -08:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
ba874d6787 HADOOP-15059. Undoing the switch of Credentials to PB format as default - done via HADOOP-12563 for supporting 2.x to 3.x upgrades. 
(cherry picked from commit f19638333b11da6dcab9a964e73a49947b8390fd)
 2017-12-08 08:03:23 -08:00
Sammi Chen
abaabb5dec HADOOP-15024. AliyunOSS: Support user agent configuration and include that & Hadoop version information to oss server. 
(cherry picked from commit c326fc89b06a8fe0978306378ba217748c7f2054)
 2017-12-08 21:28:19 +08:00
Sammi Chen
fb809e05dc HADOOP-14993. AliyunOSS: Override listFiles and listLocatedStatus. Contributed Genmao Yu. 
(cherry picked from commit 18621af7ae8f8ed703245744f8f2a770d07bbfb9)
 2017-12-08 20:40:14 +08:00
John Zhuge
fc9e156484 HADOOP-14872. CryptoInputStream should implement unbuffer. Contributed by John Zhuge. 
(cherry picked from commit 6c32ddad30240a251caaefdf7fec9ff8ad177a7c)
 2017-12-07 21:08:16 -08:00
Xiao Chen
1d1401ad38 HADOOP-15056. Fix TestUnbuffer#testUnbufferException failure. Contributed by Jack Bearden. 
(cherry picked from commit 19e089420999dd9d97d981dcd0abd64b6166152d)
 2017-12-07 21:06:25 -08:00
John Zhuge
9942952dd8 HADOOP-15012. Add readahead, dropbehind, and unbuffer to StreamCapabilities. Contributed by John Zhuge. 
(cherry picked from commit bf6a660232b01642b07697a289c773ea5b97217c)
 2017-12-07 21:06:25 -08:00
Sean Mackrory
63d81cbbe7 HADOOP-15098. TestClusterTopology#testChooseRandom fails intermittently. Contributed by Zsolt Venczel. 
(cherry picked from commit acb92904d04053551d3010937e133f74450043df)
 2017-12-07 13:24:12 -07:00
Lei Xu
f3143d225a HDFS-12840. Creating a file with non-default EC policy in a EC zone is not correctly serialized in the editlog. Contributed by Lei (Eddy) Xu. 
(cherry picked from commit 67662e2ac9e68f32b725c8118cf2be79a662fca5)
 2017-12-07 11:20:19 -08:00
Sammi Chen
afcbfbf7f4 HADOOP-15080. Aliyun OSS: update oss sdk from 2.8.1 to 2.8.3 to remove its dependency on Cat-x json-lib 2017-12-07 23:15:15 +08:00
Weiwei Yang
27c7a1f221 YARN-7607. Remove the trailing duplicated timestamp in container diagnostics message. Contributed by Weiwei Yang. 2017-12-07 17:36:31 +08:00
Akira Ajisaka
a8be4b559a
HDFS-12889. Addendum patch to add missing file. 
(cherry picked from commit 6cca5b3bcb440095f12d3eda88101fa250ac000a)
 2017-12-07 13:23:58 +09:00
Robert Kanter
cdf305c337 YARN-6483. addendum to skip failing test (asuresh via rkanter) 2017-12-06 12:10:59 -08:00
Weiwei Yang
8bfd30b921 YARN-7610. Extend Distributed Shell to support launching job with opportunistic containers. Contributed by Weiwei Yang. 
(cherry picked from commit 40b0045ebe0752cd3d1d09be00acbabdea983799)
 2017-12-06 19:10:34 +08:00
Xiao Chen
bf79246c5d HDFS-12872. EC Checksum broken when BlockAccessToken is enabled. 
(cherry picked from commit 56b1ff80dd9fbcde8d21a604eff0babb3a16418f)
 2017-12-05 20:48:46 -08:00
Weiwei Yang
48af34571f YARN-7611. Node manager web UI should display container type in containers page. Contributed by Weiwei Yang. 
(cherry picked from commit 05c347fe51c01494ed8110f8f116a01c90205f13)
 2017-12-06 12:24:00 +08:00
Akira Ajisaka
c81057d936
HDFS-12889. Router UI is missing robots.txt file. Contributed by Bharat Viswanadham. 
(cherry picked from commit 44b06d34a537f8b558007cc92a5d1a8e59b5d86b)
 2017-12-06 11:41:40 +09:00
Wangda Tan
5cb1b97905 YARN-7381. Enable the configuration: yarn.nodemanager.log-container-debug-info.enabled by default in yarn-default.xml. (Xuan Gong via wangda) 
Change-Id: I1ed58dafad5cc276eea5c0b0813cf04f57d73a87
(cherry picked from commit 0311cf05358cd75388f48f048c44fba52ec90f00)
 2017-12-05 13:10:15 -08:00
Akira Ajisaka
0a886461b5
HADOOP-14985. Remove subversion related code from VersionInfoMojo.java. Contributed by Ajay Kumar. 
(cherry picked from commit 9f1bdafedb60f83598819c6a682f659d6e168eb0)
 2017-12-05 14:31:21 +09:00
Sean Mackrory
486311cdc4 HADOOP-15058. create-release site build outputs dummy shaded jars due to skipShade. Contributed by Andrew Wang. 
(cherry picked from commit 5533648241b11f547ac10b64a126f3ae7a2f54c4)
 2017-12-04 15:43:20 -07:00
Xiao Chen
55fa509eb6 HDFS-12396. Webhdfs file system should get delegation token from kms provider. Contributed by Rushabh S Shah. 
(cherry picked from commit 404eab4dc0582e0384b93664ea6ee77ccd5eeebc)
 2017-12-04 10:49:58 -08:00
Sunil G
f034668b91 YARN-6907. Node information page in the old web UI should report resource types. Contributed by Gergely Novák. 
(cherry picked from commit 81f6e46b2fb54659a08864677ad14f80fe4e452d)
 2017-12-04 12:19:27 +05:30
Arun Suresh
6f123aae41 YARN-7587. Skip dispatching opportunistic containers to nodes whose queue is already full. (Weiwei Yang via asuresh) 
(cherry picked from commit 37ca4169508c3003dbe9044fefd37eb8cd8c0503)
 2017-12-03 22:23:46 -08:00
Chris Douglas
2399c96ee7 HDFS-11576. Block recovery will fail indefinitely if recovery time > heartbeat interval. Contributed by Lukas Majercak 
(cherry picked from commit 42307e3c3abbfe0b83d9a2581deba327435b910f)
 2017-12-02 23:32:44 -08:00
Jason Lowe
5db3ca39dc YARN-7455. quote_and_append_arg can overflow buffer. Contributed by Jim Brennan 
(cherry picked from commit 60f95fb719f00a718b484c36a823ec5aa8b099f4)
 2017-12-01 15:48:33 -06:00
Jason Lowe
f222b9d362 YARN-7589. TestPBImplRecords fails with NullPointerException. Contributed by Daniel Templeton 
(cherry picked from commit 25df5054216a6a76d09d9c49984f8075ebc6a197)
 2017-12-01 15:38:57 -06:00
Jason Lowe
0dd7f1f4e8 MAPREDUCE-5124. AM lacks flow control for task events. Contributed by Peter Bacsko 
(cherry picked from commit 21d36273551fa45c4130e5523b6724358cf34b1e)
 2017-12-01 14:10:02 -06:00
Wei-Chiu Chuang
83b62a8eed HDFS-12836. startTxId could be greater than endTxId when tailing in-progress edit log. Contributed by Chao Sun. 
(cherry picked from commit 0faf50624580b86b64a828cdbbb630ae8994e2cd)
 2017-12-01 12:03:04 -08:00
Chris Douglas
6266c52680 Revert "HDFS-11576. Block recovery will fail indefinitely if recovery time > heartbeat interval. Contributed by Lukas Majercak" 
This reverts commit 482fd5a880994f37fc3ad9e0cc2d127737b70aef.
 2017-12-01 11:19:17 -08:00
Chris Douglas
482fd5a880 HDFS-11576. Block recovery will fail indefinitely if recovery time > heartbeat interval. Contributed by Lukas Majercak 
(cherry picked from commit 5304698dc8c5667c33e6ed9c4a827ef57172a723)
 2017-12-01 10:30:08 -08:00
Konstantin V Shvachko
def87db9cb HDFS-12638. Delete copy-on-truncate block along with the original block, when deleting a file being truncated. Contributed by Konstantin Shvachko. 
(cherry picked from commit 60fd0d7fd73198fd610e59d1a4cd007c5fcc7205)
 2017-11-30 18:27:16 -08:00
Junping Du
58d849b858 YARN-7558. yarn logs command fails to get logs for running containers if UI authentication is enabled. Contributed by Xuan Gong. 
(cherry picked from commit a409425986fc128bb54f656b05373201545f7213)
 2017-11-30 13:49:00 -08:00
Jason Lowe
f072caa0bc YARN-7495. Improve robustness of the AggregatedLogDeletionService. Contributed by Jonathan Eagles 
(cherry picked from commit 5cfaee2e6db8b2ac55708de0968ff5539ee3bd76)
 2017-11-30 12:40:45 -06:00
Daniel Templeton
40372be7bb YARN-7541. Node updates don't update the maximum cluster capability for resources other than CPU and memory 
(cherry picked from commit 8498d287cd3beddcf8fe19625227e09982ec4be2)
 2017-11-30 09:26:25 -08:00
Robert Kanter
2c2ff7da05 HADOOP-13493. Compatibility Docs should clarify the policy for what takes precedence when a conflict is found (templedf via rkanter) 
(cherry picked from commit 75a3ab88f5f4ea6abf0a56cb8058e17b5a5fe403)
 2017-11-30 07:39:51 -08:00
Kihwal Lee
ee4525e071 HDFS-11754. Make FsServerDefaults cache configurable. Contributed by Mikhail Erofeev. 
(cherry picked from commit 53509f295b5274059541565d7216bf98aa35347d)
 2017-11-29 15:12:47 -06:00
Wei Yan
4b1a2152eb HDFS-12835. Fix the javadoc errors in Router-based federation. 
(cherry picked from commit 301641811d93ac22dc6fe1a05f18c1f266cc5e54)
 2017-11-29 09:59:17 -08:00
Konstantin V Shvachko
4cbd5ea42b HDFS-12832. INode.getFullPathName may throw ArrayIndexOutOfBoundsException lead to NameNode exit. Contribuited by Konstantin Shvachko. 
(cherry picked from commit d331762f24b3f22f609366740c9c4f449edc61ac)
 2017-11-28 17:18:21 -08:00
Jason Lowe
a4f1e3036d YARN-6647. RM can crash during transitionToStandby due to InterruptedException. Contributed by Bibin A Chundatt 
(cherry picked from commit a2c7a73e33045ce42cce19aacbe45c0421a61994)
 2017-11-28 11:19:43 -06:00
Steve Loughran
8bf74f325a HADOOP-15042. Azure PageBlobInputStream.skip() can return negative value when numberOfPagesRemaining is 0. 
Contributed by Rajesh Balamohan

(cherry picked from commit 0ea182d0faa35c726dcb37249d48786bfc8ca04c)
 2017-11-28 11:57:12 +00:00
Yiqun Lin
72d0be964d HDFS-12858. Add router admin commands usage in HDFS commands reference doc. Contributed by Yiqun Lin. 
(cherry picked from commit 94bed5047113fb148194380853ff01e92897a91f)
 2017-11-28 11:52:29 +08:00
Yufei Gu
c0ab3c8407 YARN-7363. ContainerLocalizer don't have a valid log4j config in case of Linux container executor. (Contributed by Yufei Gu) 
(cherry picked from commit d8923cdbf1567aee10a54f144fef734d1465ebed)
 2017-11-27 14:33:13 -08:00
Yufei Gu
f335d509d3 YARN-7290. Method canContainerBePreempted can return true when it shouldn't. (Contributed by Steven Rand) 
(cherry picked from commit 2bde3aedf139368fc71f053d8dd6580b498ff46d)
 2017-11-24 23:39:03 -08:00
Wangda Tan
6e038165a6 YARN-7509. AsyncScheduleThread and ResourceCommitterService are still running after RM is transitioned to standby. (Tao Yang via wangda) 
Change-Id: I7477fe355419fd4a0a6e2bdda7319abad4c4c748
(cherry picked from commit 834e91ee91d22d74866afbf6252107e969bf8370)
 2017-11-23 20:02:34 -08:00