Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
15,299 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

15299 Commits

Author SHA1 Message Date
Kihwal Lee 8d4671b611 HADOOP-16683. Disable retry of FailoverOnNetworkExceptionRetry in case of wrapped AccessControlException. Contributed by Adam Antal 
(cherry picked from commit 3d249301f4)
 2020-01-24 11:08:24 -06:00
Kihwal Lee 2873a705f4 HADOOP-16580. Disable retry of FailoverOnNetworkExceptionRetry in case of AccessControlException. Contributed by Adam Antal 
(cherry picked from commit c79a5f2d99)
 2020-01-24 11:07:06 -06:00
Abhishek Modi 296786a647 YARN-9790. Failed to set default-application-lifetime if maximum-application-lifetime is less than or equal to zero. Contributed by kyungwan nam. 
(cherry picked from commit d2d963f3d4)
 2020-01-23 17:12:25 +00:00
Kihwal Lee 0ead38b59a HDFS-14968. Add ability to log stale datanodes. Contributed by Ahmed Hussein. 
(cherry picked from commit bd03053ea2)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/HeartbeatManager.java

(cherry picked from commit 4842708320)
 2020-01-22 09:30:41 -06:00
Kihwal Lee c5d43b65a9 HDFS-15125. Pull back HDFS-11353, HDFS-13993, HDFS-13945, and HDFS-14324 
to branch-2.10. Contributed by Jim Brennan.
 2020-01-21 09:59:14 -06:00
sunlisheng dc010b9844 HADOOP-16793. Redefine log level when ipc connection interrupted in Client#handleSaslConnectionFailure(). 
Signed-off-by: sunlisheng <sunlisheng@xiaomi.com>
(cherry picked from commit d887e49dd4)
 2020-01-21 18:56:41 +09:00
Akira Ajisaka 0c0be3b9f0
HADOOP-16808. Use forkCount and reuseForks parameters instead of forkMode in the config of maven surefire plugin. Contributed by Xieming Li. 
(cherry picked from commit f6d20daf40)
 2020-01-21 18:05:46 +09:00
kevinzhao1661 df16cea24f
MAPREDUCE-7247. Modify HistoryServerRest.html content,change The job attempt id‘s datatype from string to int (#1684) 
(cherry picked from commit b8518e6dab)
 2020-01-16 12:58:57 +09:00
Wei-Chiu Chuang d5f7aa2285 HDFS-13339. Volume reference can't be released and may lead to deadlock when DataXceiver does a check volume. Contributed by Jim Brennan, Zsolt Venczel. 
(cherry picked from commit c675719c3f)
 2020-01-15 11:06:15 -08:00
Masatake Iwasaki 7a309f1f91
MAPREDUCE-7256. Fix javadoc error in SimpleExponentialSmoothing. (#1804) 
(cherry picked from commit a2fdd7c2b5)
 2020-01-15 12:13:12 +09:00
Chen Liang 6deb6be252 HDFS-15099. [SBN Read] checkOperation(WRITE) should throw ObserverRetryOnActiveException for ObserverNode. Contributed by Chen Liang. 
(cherry picked from commit 26a969ec73)
 2020-01-10 17:19:59 -08:00
Daryn Sharp 23a77050e0 HADOOP-16749. Configuration parsing of CDATA values are blank 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
 2020-01-10 10:48:45 -06:00
Eric E Payne c296e3c0aa YARN-9018. Add functionality to AuxiliaryLocalPathHandler to return all locations to read for a given path. Contributed by Kuhu Shukla (kshukla) 
(cherry picked from commit 93233a7d6e)
 2020-01-09 17:52:26 +00:00
Ahmed Hussein 203f55fccc MAPREDUCE-7252. Handling 0 progress in SimpleExponential task runtime estimator 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
 2020-01-09 09:52:26 -06:00
Eric Badger ab29b54b19 YARN-8672. TestContainerManager#testLocalingResourceWhileContainerRunning occasionally times out. Contributed by Chandni Singh and Jim Brennan. 2020-01-08 20:13:13 +00:00
Eric E Payne 5cca5ca81b YARN-7387: org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestIncreaseAllocationExpirer fails intermittently. Contributed by Jim Brennan (Jim_Brennan) 
(cherry picked from commit b1e07d27cc)
 2020-01-08 19:59:13 +00:00
Eric E Payne 2ae1b3568b YARN-10072: TestCSAllocateCustomResource failures. Contributed by Jim Brennan (Jim_Brennan) 
(cherry picked from commit 6899be5a17)
 2020-01-08 18:04:12 +00:00
Chen Liang 82bc4778ef HADOOP-16789. In TestZKFailoverController, restore changes from HADOOP-11149 that were dropped by HDFS-6440. Contributed by Jim Brennan. 2020-01-03 16:07:56 -08:00
Ayush Saxena 02642a6181 HADOOP-16784. Update the year to 2020. Contributed by Ayush Saxena. 2020-01-02 21:39:46 +05:30
Takanobu Asanuma 01a29f12c0 HDFS-14934. [SBN Read] Standby NN throws many InterruptedExceptions when dfs.ha.tail-edits.period is 0. Contributed by Ayush Saxena. 
(cherry picked from commit dc32f583af)
 2019-12-28 22:24:28 +09:00
Duo Zhang 7db97a2326
HADOOP-16610. Upgrade to yetus 0.11.1 and use emoji vote on github pre commit (#1527). Contributed by Duo Zhang. 
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
(cherry picked from commit 3cecb2a469)
(cherry picked from commit 440b7abc04)
 2019-12-25 19:49:07 +09:00
Akira Ajisaka eda4f5ed7c
HADOOP-16609. Add Jenkinsfile for all active branches (#1557) 
* Backport HADOOP-16035. Jenkinsfile for Hadoop
* Use Yetus 0.10.0

(cherry picked from commit ae8ca70fc0)
 2019-12-25 17:26:06 +09:00
Konstantin V Shvachko 199c7cc62c HDFS-15076. Fix tests that hold FSDirectory lock, without holding FSNamesystem lock. Contributed by Konstantin V Shvachko. 
(cherry picked from commit b98ac2a3af)
 2019-12-24 12:48:10 -08:00
Eric Badger cb5b80d6cb YARN-10009. In Capacity Scheduler, DRC can treat minimum user limit percent as a max when custom resource is defined. Contributed by Eric Payne. 2019-12-20 19:40:55 +00:00
Jonathan Hung 041fe5fb57 YARN-9894. CapacitySchedulerPerf test for measuring hundreds of apps in a large number of queues. Contributed by Eric Payne 
(cherry picked from commit 7b93575b92)
(cherry picked from commit 0707d0a0ae)
(cherry picked from commit 750fb4c321)
 2019-12-18 13:31:52 -08:00
Jonathan Hung c4a8c834e5 YARN-10039. Allow disabling app submission from REST endpoints 
(cherry picked from commit 406c35dd12)
 2019-12-18 11:01:29 -08:00
Shashikant Banerjee abbf2a4446 HDFS-15012. NN fails to parse Edit logs after applying HDFS-13101. Contributed by Shashikant Banerjee. 
(cherry picked from commit fdd96e46d1)
 2019-12-18 23:30:34 +05:30
Ayush Saxena 0927041589 HDFS-14519. NameQuota is not update after concat operation, so namequota is wrong. Contributed by Ranith Sardar. 2019-12-17 21:55:58 +05:30
He Xiaoqiao b91fda7f1d HDFS-15050. Optimize log information when DFSInputStream meet CannotObtainBlockLengthException. Contributed by Xiaoqiao He. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 99af49878cc47f5750b6c0edadc8a4c2fcd7a7c6)
(cherry picked from commit 8ba4e13dcc)
(cherry picked from commit 974706c5fd)
 2019-12-12 03:25:28 -08:00
Xiao Chen ba4de45e75 HDFS-13511. Provide specialized exception when block length cannot be obtained. Contributed by Gabor Bota. 
(cherry picked from commit 774c1f199e)
(cherry picked from commit 3b646a9836)
 2019-12-12 03:25:28 -08:00
pingsutw a969cad0a1
HADOOP-16754. Fix docker failed to build yetus/hadoop 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2019-12-12 11:05:43 +09:00
Mingliang Liu a8044c75ad
HADOOP-16758. Refine testing.md to tell user better how to use auth-keys.xml (#1753) 
Contributed by Mingliang Liu
 2019-12-11 11:56:46 -08:00
Erik Krogen a22698fa52 HDFS-15032. Properly handle InvocationTargetExceptions in the proxy created by ProxyCombiner. This fixes a bug encountered by the HDFS balancer when used with Observer Nodes. Contributed by Erik Krogen. 
(cherry picked from c174d50b30)
(cherry picked from 3402c87353)
(cherry picked from b06965ae32)
 2019-12-11 09:41:40 -08:00
Jonathan Hung d74f64aa99 Setting version to 2.10.1-SNAPSHOT after branch-2 to branch-2.10 rename 2019-12-09 11:49:59 -08:00
Jonathan Hung b04f1df4cf YARN-10012. Guaranteed and max capacity queue metrics for custom resources. Contributed by Manikandan R 
(cherry picked from commit 92bce918dc)
(cherry picked from commit 9228e3f0ad)
(cherry picked from commit 9b4368a62f)
 2019-12-08 16:44:38 -08:00
Sunil G 5401d7f024 YARN-4901. QueueMetrics needs to be cleared before MockRM is initialized. Contributed by Peter Bacsko. 
(cherry picked from commit 002dcc4ebf)
(cherry picked from commit 69dc329acc)
(cherry picked from commit 67cf1f94cd)
 2019-12-08 14:44:57 -08:00
Chao Sun 9a385e00a4 HDFS-15017. Remove redundant import of AtomicBoolean in NameNodeConnector. Contributed by Chao Sun. 2019-12-07 13:45:17 -08:00
Chao Sun c6ce4011d7 HDFS-15005. Backport HDFS-12300 to branch-2. Contributed by Chao Sun. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-12-06 15:25:30 -08:00
Mingliang Liu cdb0cafff5
HADOOP-16735. Make it clearer in config default that EnvironmentVariableCredentialsProvider supports AWS_SESSION_TOKEN. Contributed by Mingliang Liu 
This closes #1733
 2019-12-05 17:56:02 -08:00
Erik Krogen 0ac6dc79b0 HADOOP-16700. Track response time separately in Server.Call to avoid negative RpcQueueTime values. Contributed by xuzq. 
(cherry picked from be77231452)
(cherry picked from c1ed247459)
(cherry picked from 23f25418e8)
 2019-12-03 14:19:27 -08:00
Weiwei Yang e7ecb4f51f YARN-9205. When using custom resource type, application will fail to run due to the CapacityScheduler throws InvalidResourceRequestException(GREATER_THEN_MAX_ALLOCATION). Contributed by Zhankun Tang. 
(cherry picked from commit bc6374f282)
 2019-12-03 13:56:54 -08:00
Yiqun Lin fc2cdb6cca HDFS-14986. ReplicaCachingGetSpaceUsed throws ConcurrentModificationException. Contributed by Aiphago. 
(cherry picked from commit 2b452b4e60)
 2019-11-28 10:45:25 +08:00
Erik Krogen 403892264b HDFS-14973. More strictly enforce Balancer/Mover/SPS throttling of getBlocks RPCs to NameNodes. Contributed by Erik Krogen. 
(cherry picked from b2cc8b6b4a)
(cherry picked from 60655bfe54)
(cherry picked from 17779adb32)
 2019-11-26 16:46:38 -08:00
Eric E Payne 3a7121b0b6 YARN-9915: Fix FindBug issue in QueueMetrics. Contributed by Prabhu Joseph. 
(cherry picked from commit 83d148074f)
 2019-11-22 19:03:42 +00:00
Eric E Payne bfb76e60cf YARN-9773: Add QueueMetrics for Custom Resources. Contributed by Manikandan R. 
(cherry picked from commit a5034c7988)
 2019-11-22 18:57:05 +00:00
Tao Yang 5b2a1b641a YARN-9838. Fix resource inconsistency for queues when moving app with reserved container to another queue. Contributed by jiulongzhu. 
(cherry picked from commit 8afabe41e4)
 2019-11-22 16:18:59 +08:00
Jason Lowe dbbe23f269 YARN-7589. TestPBImplRecords fails with NullPointerException. Contributed by Daniel Templeton 
(cherry picked from commit 25df505421)
 2019-11-21 11:55:20 -08:00
Eric E Payne a9f51e0372 YARN-8842. Expose metrics for custom resource types in QueueMetrics. (Contributed by Szilard Nemeth) (refactored patch for branch-2) 2019-11-21 19:23:48 +00:00
Wangda Tan 835bc686e0 YARN-7411. Inter-Queue preemption's computeFixpointAllocation need to handle absolute resources while computing normalizedGuarantee. (Sunil G via wangda) 
Change-Id: I41b1d7558c20fc4eb2050d40134175a2ef6330cb
(cherry picked from commit 034b312d9f)

Conflicts:
  hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ResourcePBImpl.java
  I did not backport the changes to ResourcePBImpl.java because the code was removed by mistake and soon added back as part of YARN-7483 (Eric Payne).
 2019-11-19 22:39:14 +00:00
He Xiaoqiao ae7469cb67 HDFS-14952. Skip safemode if blockTotal is 0 in new NN. Contributed by Xiaoqiao He. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Mukul Kumar Singh <msingh@apache.org>
(cherry picked from commit 0b50aa29fd)
(cherry picked from commit 1d383436f09f8e490171c8f6af4605971ad2f904)
(cherry picked from commit 7909c5df572a5cbc729ebc6c7c6e59f17803d50f)
 2019-11-19 12:23:01 -08:00