483d54f1a7
MAPREDUCE-7256. Fix javadoc error in SimpleExponentialSmoothing. ( #1804 )
...
(cherry picked from commit a2fdd7c2b5
2020-01-15 12:09:59 +09:00
1d2c5dffa8
HADOOP-16683. Disable retry of FailoverOnNetworkExceptionRetry in case of wrapped AccessControlException. Contributed by Adam Antal
2020-01-14 11:18:54 +01:00
aa3c96474f
HDFS-15097. Purge log in KMS and HttpFS. Contributed by Doris Gu.
...
(cherry picked from commit 6b86a5110e
2020-01-13 10:49:26 -08:00
6a7dfb3bf3
YARN-10026. Pull out common code pieces from ATS v1.5 and v2. Contributed by Adam Antal
2020-01-12 13:54:08 +01:00
d3c4c8be6a
HDFS-14578. AvailableSpaceBlockPlacementPolicy always prefers local node. Contributed by Ayush Saxena.
2020-01-11 10:34:01 +05:30
1ee90d6731
HDFS-15099. [SBN Read] checkOperation(WRITE) should throw ObserverRetryOnActiveException for ObserverNode. Contributed by Chen Liang.
...
(cherry picked from commit 26a969ec73
2020-01-10 17:17:11 -08:00
59260a0e0e
HDFS-15095. Fix TestDecommissioningStatus. Contributed by Ahmed Hussein.
...
(cherry picked from commit 5fb901ac40
2020-01-10 12:46:49 -06:00
57cb180267
HADOOP-16749. Configuration parsing of CDATA values are blank
...
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit c4fb43c942
2020-01-10 10:32:42 -06:00
3ba0fd1e50
YARN-9018. Add functionality to AuxiliaryLocalPathHandler to return all locations to read for a given path. Contributed by Kuhu Shukla (kshukla)
...
(cherry picked from commit 93233a7d6e
2020-01-09 17:22:10 +00:00
6304f94bc0
HDFS-14957. INodeReference Space Consumed was not same in QuotaUsage and ContentSummary. Contributed by hemanthboyina.
2020-01-09 12:07:23 +05:30
58db04ce15
YARN-8672. TestContainerManager#testLocalingResourceWhileContainerRunning occasionally times out. Contributed by Chandni Singh and Jim Brennan.
2020-01-08 19:44:43 +00:00
5e2be81fcb
YARN-7387: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestIncreaseAllocationExpirer fails intermittently. Contributed by Jim Brennan (Jim_Brennan)
...
(cherry picked from commit b1e07d27cc
2020-01-08 19:30:09 +00:00
b20ce118da
YARN-10072: TestCSAllocateCustomResource failures. Contributed by Jim Brennan (Jim_Brennan)
...
(cherry picked from commit 6899be5a17
2020-01-08 17:42:41 +00:00
ee7c56c6de
MAPREDUCE-7252. Handling 0 progress in SimpleExponential task runtime estimator
...
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit cdd6efd3ab
2020-01-08 11:09:58 -06:00
e21cb8f96e
HADOOP-16785. Improve wasb and abfs resilience on double close() calls.
...
This hardens the wasb and abfs output streams' resilience to being invoked
in/after close().
wasb:
Explicity raise IOEs on operations invoked after close,
rather than implicitly raise NPEs.
This ensures that invocations which catch and swallow IOEs will perform as
expected.
abfs:
When rethrowing an IOException in the close() call, explicitly wrap it
with a new instance of the same subclass.
This is needed to handle failures in try-with-resources clauses, where
any exception in closed() is added as a suppressed exception to the one
thrown in the try {} clause
*and you cannot attach the same exception to itself*
Contributed by Steve Loughran.
Change-Id: Ic44b494ff5da332b47d6c198ceb67b965d34dd1b
2020-01-08 12:04:11 +00:00
d7f9a90e2c
HDFS-15080. Fix the issue in reading persistent memory cached data with an offset. Contributed by Feilong He.
...
(cherry picked from commit 7030722e5d
2020-01-08 14:29:05 +05:30
3e17b471b6
HDFS-15077. Fix intermittent failure of TestDFSClientRetries#testLeaseRenewSocketTimeout. ( #1797 )
...
(cherry picked from commit aba3f6c3e1
2020-01-08 17:05:20 +09:00
3e32f5476a
HDFS-15072. HDFS MiniCluster fails to start when run in directory path with a %. ( #1775 )
...
(cherry picked from commit a43c177f1d
2020-01-08 11:29:23 +09:00
250cd9f575
HADOOP-16772. Extract version numbers to head of pom.xml (addendum) ( #1774 ). Contributed by Tamas Penzes.
...
Follow up task of HADOOP-16729, extract even more version numbers for branch-3.2.
Change-Id: I0546fc8ec8f68baa4bcfa9db0df4d8293b078913
2020-01-06 14:12:28 +01:00
d7cc897bdc
MAPREDUCE-7255. Fix typo in MapReduce documentaion example ( #1793 )
...
(cherry picked from commit b343e1533b
2020-01-06 12:40:29 +09:00
3a624cad37
HADOOP-16784. Update the year to 2020. Contributed by Ayush Saxena.
2020-01-02 21:38:49 +05:30
5410732cff
HADOOP-16775. DistCp reuses the same temp file within the task for different files.
...
Contributed by Amir Shenavandeh.
This avoids overwrite consistency issues with S3 and other stores -though
given S3's copy operation is O(data), you are still best of using -direct
when distcp-ing to it.
Change-Id: I8dc9f048ad0cc57ff01543b849da1ce4eaadf8c3
2020-01-02 15:37:55 +00:00
ad98a30810
YARN-10053. Use Shared Group Mapping Service in Placement Rules. Contributed by Wilfred Spiegelenburg.
...
(cherry Picked from commit 217b56ffdd
2020-01-02 14:30:01 +05:30
a27f9bd28e
HDFS-14740. Recover data blocks from persistent memory read cache during datanode restarts. Contributed by Feilong He.
...
(cherry picked from commit d79cce20ab
2020-01-02 11:57:02 +05:30
5d9cbca31e
HDFS-15074. DataNode.DataTransfer thread should catch all the expception and log it. Contributed by hemanthboyina.
2019-12-29 11:21:33 +05:30
ad11b25df2
HDFS-14934. [SBN Read] Standby NN throws many InterruptedExceptions when dfs.ha.tail-edits.period is 0. Contributed by Ayush Saxena.
...
(cherry picked from commit dc32f583af
2019-12-28 21:34:19 +09:00
8ec1461b25
HADOOP-16042. Update the link to HadoopJavaVersion ( #1784 )
...
(cherry picked from commit d80d86264d
2019-12-27 17:20:26 +09:00
440b7abc04
HADOOP-16610. Upgrade to yetus 0.11.1 and use emoji vote on github pre commit ( #1527 ). Contributed by Duo Zhang.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
(cherry picked from commit 3cecb2a469
2019-12-25 19:48:15 +09:00
0d94e0b1a0
HDFS-15073. Replace curator-shaded guava import with the standard one ( #1782 )
...
(cherry picked from commit d8cd7098b4
2019-12-25 11:12:03 +09:00
82957562ea
HDFS-15076. Fix tests that hold FSDirectory lock, without holding FSNamesystem lock. Contributed by Konstantin V Shvachko.
...
(cherry picked from commit b98ac2a3af
2019-12-24 12:50:57 -08:00
85da5cb870
YARN-10055. bower install fails. ( #1778 )
...
(cherry picked from commit 34ff7dbaf5
2019-12-23 20:26:40 +09:00
355ec33416
YARN-10009. In Capacity Scheduler, DRC can treat minimum user limit percent as a max when custom resource is defined. Contributed by Eric Payne
2019-12-20 19:32:36 +00:00
11c5396b4a
Revert "HDFS-15062. Add LOG when sendIBRs failed. Contributed by Fei Hui."
...
This reverts commit 59cdf18d5a
2019-12-20 19:04:45 +00:00
7201384e1b
HADOOP-16771. Update checkstyle to 8.26 and maven-checkstyle-plugin to 3.1.0. Contributed by Andras Bokor.
...
(cherry picked from commit f777cd398f
2019-12-20 13:12:02 +09:00
59cdf18d5a
HDFS-15062. Add LOG when sendIBRs failed. Contributed by Fei Hui.
...
(cherry picked from commit 52d7b745c6
2019-12-19 09:43:40 -08:00
0707d0a0ae
YARN-9894. CapacitySchedulerPerf test for measuring hundreds of apps in a large number of queues. Contributed by Eric Payne
...
(cherry picked from commit 7b93575b92
2019-12-18 13:18:22 -08:00
01e2c996ff
YARN-10039. Allow disabling app submission from REST endpoints
...
(cherry picked from commit fddc3d55c3
2019-12-18 10:51:57 -08:00
afe00a1ca5
HDFS-15012. NN fails to parse Edit logs after applying HDFS-13101. Contributed by Shashikant Banerjee.
...
(cherry picked from commit fdd96e46d1
2019-12-18 23:10:00 +05:30
18b515322c
YARN-10033. TestProportionalCapacityPreemptionPolicy not initializing vcores for effective max resources. Contributed by Eric Payne.
...
(cherry picked from commit f47dcf2d4c
2019-12-17 17:36:52 +00:00
d4a6901c42
HDFS-15036. Active NameNode should not silently fail the image transfer. Contributed by Chen Liang.
...
(cherry picked from commit 65c4660bcd
2019-12-12 15:57:15 -08:00
8ba4e13dcc
HDFS-15050. Optimize log information when DFSInputStream meet CannotObtainBlockLengthException. Contributed by Xiaoqiao He.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 99af49878cc47f5750b6c0edadc8a4c2fcd7a7c6)
2019-12-11 16:34:28 -08:00
d19981fe48
HADOOP-16758. Refine testing.md to tell user better how to use auth-keys.xml ( #1753 )
...
Contributed by Mingliang Liu
2019-12-11 11:54:12 -08:00
3402c87353
HDFS-15032. Properly handle InvocationTargetExceptions in the proxy created by ProxyCombiner. This fixes a bug encountered by the HDFS balancer when used with Observer Nodes. Contributed by Erik Krogen.
...
(cherry picked from c174d50b30
2019-12-11 09:40:36 -08:00
0f25cbbb25
HADOOP-16754. Fix docker failed to build yetus/hadoop.
...
Contributed by Kevin Su.
Change-Id: Ie9cf27b7a7ba5923405f24d39b95b0248f3718b7
2019-12-11 16:57:30 +00:00
a3cdba7713
YARN-9985. Unsupported transitionToObserver option displaying for rmadmin command. Contributed by Ayush Saxena.
...
(cherry picked from commit dc66de7448
2019-12-09 18:38:47 +09:00
aa9cd0a2d6
HADOOP-16660. ABFS: Make RetryCount in ExponentialRetryPolicy Configurable.
...
Contributed by Sneha Vijayarajan.
2019-12-08 21:32:13 -08:00
c225efe237
HADOOP-16455. ABFS: Implement FileSystem.access() method.
...
Contributed by Bilahari T H.
2019-12-08 21:32:02 -08:00
b1e748f45b
HADOOP-16612. Track Azure Blob File System client-perceived latency
...
Contributed by Jeetesh Mangwani.
This add the ability to track the end-to-end performance of ADLS Gen 2 REST APIs by measuring latency in the Hadoop ABFS driver.
The latency information is sent back to the ADLS Gen 2 REST API endpoints in the subsequent requests.
2019-12-08 21:31:51 -08:00
ffeb6d8ece
HADOOP-16587. Make ABFS AAD endpoints configurable.
...
Contributed by Bilahari T H.
This also addresses HADOOP-16498: AzureADAuthenticator cannot authenticate
in China.
Change-Id: I2441dd48b50b59b912b0242f7f5a4418cf94a87c
2019-12-08 21:31:39 -08:00
8b2c7e0c4d
HADOOP-16578 : Avoid FileSystem API calls when FileSystem already exists
2019-12-08 21:31:24 -08:00
Masatake Iwasaki
Szilard Nemeth
Wei-Chiu Chuang
Ayush Saxena
Chen Liang
Kihwal Lee
Daryn Sharp
Eric E Payne
Surendra Singh Lilhore
Eric Badger
Ahmed Hussein
Steve Loughran
Rakesh Radhakrishnan
Tamás Pénzes
Sergey Pogorelov
Prabhu Joseph
Takanobu Asanuma
Chandra Sanivarapu
Duo Zhang
Konstantin V Shvachko
Akira Ajisaka
Inigo Goiri
Jonathan Hung
Shashikant Banerjee
He Xiaoqiao
Mingliang Liu
Erik Krogen
Kevin Su
Sneha Vijayarajan
bilaharith
Jeetesh Mangwani
bilaharith