Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
19,615 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

19615 Commits

Author SHA1 Message Date
Inigo Goiri 27555620fa HDFS-15062. Add LOG when sendIBRs failed. Contributed by Fei Hui. 
(cherry picked from commit 52d7b745c6)
 2019-12-19 09:47:30 -08:00
Jonathan Hung 750fb4c321 YARN-9894. CapacitySchedulerPerf test for measuring hundreds of apps in a large number of queues. Contributed by Eric Payne 
(cherry picked from commit 7b93575b92)
(cherry picked from commit 0707d0a0ae)
 2019-12-18 13:25:30 -08:00
Jonathan Hung 406c35dd12 YARN-10039. Allow disabling app submission from REST endpoints 2019-12-18 10:58:32 -08:00
Shashikant Banerjee 280cd6dd1e HDFS-15012. NN fails to parse Edit logs after applying HDFS-13101. Contributed by Shashikant Banerjee. 
(cherry picked from commit fdd96e46d1)
 2019-12-18 23:11:18 +05:30
Eric Badger 86120637a4 YARN-10033. TestProportionalCapacityPreemptionPolicy not initializing vcores for effective max resources. Contributed by Eric Payne. 
(cherry picked from commit f47dcf2d4c)
 2019-12-17 17:39:51 +00:00
Chen Liang ea0eeb8f1a HDFS-15036. Active NameNode should not silently fail the image transfer. Contributed by Chen Liang. 
(cherry picked from commit 65c4660bcd)
(cherry picked from commit d4a6901c42)
 2019-12-12 15:58:09 -08:00
He Xiaoqiao 974706c5fd HDFS-15050. Optimize log information when DFSInputStream meet CannotObtainBlockLengthException. Contributed by Xiaoqiao He. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 99af49878cc47f5750b6c0edadc8a4c2fcd7a7c6)
(cherry picked from commit 8ba4e13dcc)
 2019-12-12 02:23:22 -08:00
pingsutw 70a0ef5d4a
HADOOP-16754. Fix docker failed to build yetus/hadoop 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2019-12-12 11:03:51 +09:00
Mingliang Liu a2336d35f6
HADOOP-16758. Refine testing.md to tell user better how to use auth-keys.xml (#1753) 
Contributed by Mingliang Liu
 2019-12-11 11:54:56 -08:00
Erik Krogen b06965ae32 HDFS-15032. Properly handle InvocationTargetExceptions in the proxy created by ProxyCombiner. This fixes a bug encountered by the HDFS balancer when used with Observer Nodes. Contributed by Erik Krogen. 
(cherry picked from c174d50b30)
(cherry picked from 3402c87353)
 2019-12-11 09:41:11 -08:00
dependabot[bot] 935854e0ca
Bump nimbus-jose-jwt from 4.41.1 to 7.9 (#1682) 
Bumps [nimbus-jose-jwt](https://bitbucket.org/connect2id/nimbus-jose-jwt) from 4.41.1 to 7.9.
- [Changelog](https://bitbucket.org/connect2id/nimbus-jose-jwt/src/master/CHANGELOG.txt)
- [Commits](https://bitbucket.org/connect2id/nimbus-jose-jwt/branches/compare/7.9..4.41.1)

Signed-off-by: dependabot[bot] <support@github.com>
(cherry picked from commit c1d393a156)
 2019-12-09 10:23:00 +09:00
Jonathan Hung 9b4368a62f YARN-10012. Guaranteed and max capacity queue metrics for custom resources. Contributed by Manikandan R 
(cherry picked from commit 92bce918dc)
(cherry picked from commit 9228e3f0ad)
 2019-12-08 16:38:21 -08:00
Sunil G 67cf1f94cd YARN-4901. QueueMetrics needs to be cleared before MockRM is initialized. Contributed by Peter Bacsko. 
(cherry picked from commit 002dcc4ebf)
(cherry picked from commit 69dc329acc)
 2019-12-08 14:43:20 -08:00
Masatake Iwasaki 80a84ad8f5 HADOOP-16744. Fix building instruction to enable zstd. (#1736) 
(cherry picked from commit 18059acb6a)
 2019-12-06 15:28:44 +09:00
Mingliang Liu 941a98eb28
HADOOP-16735. Make it clearer in config default that EnvironmentVariableCredentialsProvider supports AWS_SESSION_TOKEN. Contributed by Mingliang Liu 
This closes #1733
 2019-12-05 17:52:25 -08:00
Aravindan Vijayan e73fa45e5b HADOOP-16718. Allow disabling Server Name Indication (SNI) for Jetty. Contributed by Aravindan Vijayan. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Siyao Meng <smeng@cloudera.com>
(cherry picked from commit f1ab7f18c4)
(cherry picked from commit f0c1403ec3)
 2019-12-03 15:48:25 -08:00
Erik Krogen 23f25418e8 HADOOP-16700. Track response time separately in Server.Call to avoid negative RpcQueueTime values. Contributed by xuzq. 
(cherry picked from be77231452)
(cherry picked from c1ed247459)
 2019-12-03 14:02:00 -08:00
Surendra Singh Lilhore 5eb21b83df HDFS-15010. BlockPoolSlice#addReplicaThreadPool static pool should be initialized by static method. Contributed by Surendra Singh Lilhore. 2019-11-29 00:07:45 +05:30
prabhujoseph a97f7776bd MAPREDUCE-7249. Fix Invalid event TA_TOO_MANY_FETCH_FAILURE at SUCCESS_CONTAINER_CLEANUP causes job failure. 
Contributed by Wilfred Spiegelenburg.
 2019-11-28 14:34:52 +05:30
Sunil G 57c499ef19 YARN-9949. Add missing queue configs for root queue in RMWebService#CapacitySchedulerInfo. 
Contributed by Prabhu Joseph.
 2019-11-27 23:01:16 +05:30
Szilard Nemeth 1172ebd6e8 YARN-9937. Add missing queue configs in RMWebService#CapacitySchedulerQueueInfo. 
Contributed by Prabhu Joseph.
 2019-11-27 22:30:10 +05:30
prabhujoseph 6a4966f336 MAPREDUCE-7240. Fix Invalid event: TA_TOO_MANY_FETCH_FAILURE at SUCCESS_FINISHING_CONTAINER. 
Contributed by Huachao and Peter Bacsko. Reviewed by Wilfred Spiegelenburg.
 2019-11-27 16:51:47 +05:30
Szilard Nemeth 80a97e6ac2 YARN-9011. Race condition during decommissioning. Contributed by Peter Bacsko 2019-11-26 14:27:55 +01:00
HUAN-PING SU fb2df5234d
YARN-9966. Code duplication in UserGroupMappingPlacementRule (#1709) 
(cherry picked from commit f8e36e03b4)
 2019-11-25 15:58:09 +09:00
Ayush Saxena 35f0478042 HDFS-14519. NameQuota is not update after concat operation, so namequota is wrong. Contributed by Ranith Sardar. 2019-11-23 02:47:59 +05:30
Szilard Nemeth df157211c9 YARN-9968. Public Localizer is exiting in NodeManager due to NullPointerException. Contributed by Tarun Parimi 2019-11-22 13:00:18 +01:00
Tao Yang c95e772d5d YARN-9838. Fix resource inconsistency for queues when moving app with reserved container to another queue. Contributed by jiulongzhu. 2019-11-22 16:10:07 +08:00
Surendra Singh Lilhore 8551c04726 HDFS-14940. HDFS Balancer : Do not allow to set balancer maximum network bandwidth more than 1TB. Contributed by hemanthboyina. 2019-11-22 00:45:55 +05:30
Wei-Chiu Chuang 0807cb2485 HADOOP-15686. Supress bogus AbstractWadlGeneratorGrammarGenerator in KMS stderr. Contributed by Wei-Chiu Chuang. 
Reviewed-by: Xiaoyu Yao <xyao@apache.org>
(cherry picked from commit 9664b9c7a6)
(cherry picked from commit bd2f79cfd8)
 2019-11-20 13:08:52 -08:00
Wanqiang Ji c7d0e0813a
HADOOP-16719. Remove the disallowed element config within maven-checkstyle-plugin (#1724) 
(cherry picked from commit dfc61d8ea5)
(cherry picked from commit 4c79e2a02b)
 2019-11-20 16:23:21 +09:00
He Xiaoqiao d101b74814 HDFS-14952. Skip safemode if blockTotal is 0 in new NN. Contributed by Xiaoqiao He. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Mukul Kumar Singh <msingh@apache.org>
(cherry picked from commit 0b50aa29fd)
(cherry picked from commit 1d383436f09f8e490171c8f6af4605971ad2f904)
 2019-11-19 12:22:54 -08:00
Szilard Nemeth 0a4e6b7b62 YARN-9836. General usability improvements in showSimulationTrace.html. Contributed by Adam Antal 2019-11-19 21:22:01 +01:00
Siyao Meng df13f6d485 HADOOP-16703. Backport HADOOP-16152 to branch-3.1. Contributed by Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-11-19 12:14:27 -08:00
Eric Yang 1669a5c1cb YARN-9983. Fixed typo in YARN Service overview. 
Contributed by Denes Gerencser
 2019-11-19 14:18:38 -05:00
Sunil G 9a51380208 YARN-9873. Version Number for each Scheduler Config Change. Contributed by Prabhu Joseph. 
(cherry picked from commit be901f4962)
 2019-11-19 17:17:45 +05:30
Abhishek Modi 96e07b7e97 YARN-9791. Queue Mutation API does not allow to remove a config. Contributed by Prabhu Joseph. 
(cherry picked from commit 751b5a1ac8)
 2019-11-19 16:41:04 +05:30
Sunil G dab4cc6320 YARN-9909. Offline Format of YarnConfigurationStore. Contributed by Prabhu Joseph. 2019-11-19 16:19:19 +05:30
Sunil G 9b17fa7091 YARN-9950. Unset Ordering Policy of Leaf/Parent queue converted from Parent/Leaf queue respectively. Contributed by Prabhu Joseph. 
(cherry picked from commit 51e7d1b37e)
 2019-11-19 15:43:19 +05:30
Sunil G f24d1fac35 YARN-9984. FSPreemptionThread can cause NullPointerException while app is unregistered with containers running on a node. Contributed by Wilfred Spiegelenburg. 
(cherry picked from commit 215f2052fc)
 2019-11-19 14:05:51 +05:30
Ayush Saxena b4f0c63a6a HDFS-14942. Change Log Level to debug in JournalNodeSyncer#syncWithJournalAtIndex. Contributed by Lisheng Sun. 
(cherry picked from commit 9e287054a8)
(cherry picked from commit d16441460215155b25fa649a7ca733e579d97929)
 2019-11-18 10:23:48 -08:00
Ayush Saxena 7d3c67b4a7 HADOOP-16712. Config ha.failover-controller.active-standby-elector.zk.op.retries is not in core-default.xml. Contributed by Xieming Li. 2019-11-18 19:59:18 +05:30
Prabhu Joseph d271d174ca YARN-9900. Revert Invalid Config and Refresh Support in SchedulerConfig Format. 
Signed-off-by: prabhujoseph <pjoseph@cloudera.com>
 2019-11-18 15:02:59 +05:30
Lukas Majercak 50d98a6568 HADOOP-16705. MBeanInfoBuilder puts unnecessary memory pressure on the system with a debug log. 
MBeanInfoBuilder's get() method DEBUG logs all the MBeanAttributeInfo attributes that it gathered. This can have a high memory churn that can be easily avoided.

(cherry picked from commit c73334a924)
 2019-11-15 13:29:29 -08:00
Erik Krogen 17779adb32 HDFS-14973. More strictly enforce Balancer/Mover/SPS throttling of getBlocks RPCs to NameNodes. Contributed by Erik Krogen. 
(cherry picked from b2cc8b6b4a)
(cherry picked from 60655bfe54)
 2019-11-15 11:14:49 -08:00
Akira Ajisaka b9e4fd5f4c
HADOOP-15097. AbstractContractDeleteTest::testDeleteNonEmptyDirRecursive with misleading path. Contributed by Xieming Li. 
(cherry picked from commit 92c28c100e)
 2019-11-15 13:51:35 +09:00
Erik Krogen 5cf36aa2b1 HDFS-14979 Allow Balancer to submit getBlocks calls to Observer Nodes when possible. Contributed by Erik Krogen. 
(cherry picked from 586defe711)
(cherry picked from dec765b329)
 2019-11-13 11:57:59 -08:00
Xudong Cao b4f39c2a8a HADOOP-16677. Recalculate the remaining timeout millis correctly while throwing an InterupptedException in SocketIOWithTimeout. (#1687) 
(cherry picked from commit df6b3162c1)
(cherry picked from commit 975f669141)
 2019-11-12 14:21:59 -08:00
Chao Sun 7e0f1eabd8 HDFS-14959: [SBNN read] access time should be turned off (#1706) Contributed by Chao Sun. 
(cherry picked from commit 97ec34e117)
(cherry picked from commit 0c12d633b9)
 2019-11-12 11:41:09 -08:00
Surendra Singh Lilhore 9179046368 HDFS-14720. DataNode shouldn't report block as bad block if the block length is Long.MAX_VALUE. Contributed by hemanthboyina. 
(cherry picked from commit 320008bb7c)
 2019-11-09 22:36:48 +05:30
Ayush Saxena 9aa59c0629 HDFS-14958. TestBalancerWithNodeGroup is not using NetworkTopologyWithNodeGroup. Contributed by Jim Brennan. 2019-11-07 22:42:07 +05:30