Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
19,517 Commits 326 Branches 371 Tags
Commit Graph

19517 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Eric Badger
ef00db4ce9 Revert "HDFS-15062. Add LOG when sendIBRs failed. Contributed by Fei Hui." 
This reverts commit 27555620fae3791d9fd24168e346714de8467a1d.
 2019-12-20 19:16:22 +00:00
Akira Ajisaka
39c6849d34
HADOOP-16771. Update checkstyle to 8.26 and maven-checkstyle-plugin to 3.1.0. Contributed by Andras Bokor. 
(cherry picked from commit f777cd398f1d48898ddc4a9a5ab4e7e310e3027a)
 2019-12-20 13:12:46 +09:00
Inigo Goiri
27555620fa HDFS-15062. Add LOG when sendIBRs failed. Contributed by Fei Hui. 
(cherry picked from commit 52d7b745c6d95e799542d6409dac30d0418ce8a8)
 2019-12-19 09:47:30 -08:00
Jonathan Hung
750fb4c321 YARN-9894. CapacitySchedulerPerf test for measuring hundreds of apps in a large number of queues. Contributed by Eric Payne 
(cherry picked from commit 7b93575b92e8bad889c1ef15e0baaade6de6de4d)
(cherry picked from commit 0707d0a0ae36456f3467cbb408c3a9a0073c70f7)
 2019-12-18 13:25:30 -08:00
Jonathan Hung
406c35dd12 YARN-10039. Allow disabling app submission from REST endpoints 2019-12-18 10:58:32 -08:00
Shashikant Banerjee
280cd6dd1e HDFS-15012. NN fails to parse Edit logs after applying HDFS-13101. Contributed by Shashikant Banerjee. 
(cherry picked from commit fdd96e46d1f89f0ecdb9b1836dc7fca9fbb954fd)
 2019-12-18 23:11:18 +05:30
Eric Badger
86120637a4 YARN-10033. TestProportionalCapacityPreemptionPolicy not initializing vcores for effective max resources. Contributed by Eric Payne. 
(cherry picked from commit f47dcf2d4cf437dd67d14dd05c60648d9ff87843)
 2019-12-17 17:39:51 +00:00
Chen Liang
ea0eeb8f1a HDFS-15036. Active NameNode should not silently fail the image transfer. Contributed by Chen Liang. 
(cherry picked from commit 65c4660bcd897e139fc175ca438cff75ec0c6be8)
(cherry picked from commit d4a6901c429a24dffdb5c884b1ba8e6492c233fe)
 2019-12-12 15:58:09 -08:00
He Xiaoqiao
974706c5fd HDFS-15050. Optimize log information when DFSInputStream meet CannotObtainBlockLengthException. Contributed by Xiaoqiao He. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 99af49878cc47f5750b6c0edadc8a4c2fcd7a7c6)
(cherry picked from commit 8ba4e13dcc4ad16e8483d33a31ed52e1b8d4e686)
 2019-12-12 02:23:22 -08:00
pingsutw
70a0ef5d4a
HADOOP-16754. Fix docker failed to build yetus/hadoop 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2019-12-12 11:03:51 +09:00
Mingliang Liu
a2336d35f6
HADOOP-16758. Refine testing.md to tell user better how to use auth-keys.xml (#1753) 
Contributed by Mingliang Liu
 2019-12-11 11:54:56 -08:00
Erik Krogen
b06965ae32 HDFS-15032. Properly handle InvocationTargetExceptions in the proxy created by ProxyCombiner. This fixes a bug encountered by the HDFS balancer when used with Observer Nodes. Contributed by Erik Krogen. 
(cherry picked from c174d50b30abc08a4642614fb35165e79792608b)
(cherry picked from 3402c87353c18730cb659ac1d8b43b4e21c634f6)
 2019-12-11 09:41:11 -08:00
dependabot[bot]
935854e0ca
Bump nimbus-jose-jwt from 4.41.1 to 7.9 (#1682) 
Bumps [nimbus-jose-jwt](https://bitbucket.org/connect2id/nimbus-jose-jwt) from 4.41.1 to 7.9.
- [Changelog](https://bitbucket.org/connect2id/nimbus-jose-jwt/src/master/CHANGELOG.txt)
- [Commits](https://bitbucket.org/connect2id/nimbus-jose-jwt/branches/compare/7.9..4.41.1)

Signed-off-by: dependabot[bot] <support@github.com>
(cherry picked from commit c1d393a1567cac1bcf71e2e5f252cddffa0f97cc)
 2019-12-09 10:23:00 +09:00
Jonathan Hung
9b4368a62f YARN-10012. Guaranteed and max capacity queue metrics for custom resources. Contributed by Manikandan R 
(cherry picked from commit 92bce918dc5d03560169642b71636800680a3292)
(cherry picked from commit 9228e3f0ad4e94cedd59c5b02a87ce6c267db5b5)
 2019-12-08 16:38:21 -08:00
Sunil G
67cf1f94cd YARN-4901. QueueMetrics needs to be cleared before MockRM is initialized. Contributed by Peter Bacsko. 
(cherry picked from commit 002dcc4ebf79bbaa5e603565640d8289991d781f)
(cherry picked from commit 69dc329acc0174b133e8b68f16b3e894329a6fd0)
 2019-12-08 14:43:20 -08:00
Masatake Iwasaki
80a84ad8f5 HADOOP-16744. Fix building instruction to enable zstd. (#1736) 
(cherry picked from commit 18059acb6ae16e72a6cdd08795f6281cda122bff)
 2019-12-06 15:28:44 +09:00
Mingliang Liu
941a98eb28
HADOOP-16735. Make it clearer in config default that EnvironmentVariableCredentialsProvider supports AWS_SESSION_TOKEN. Contributed by Mingliang Liu 
This closes #1733
 2019-12-05 17:52:25 -08:00
Aravindan Vijayan
e73fa45e5b HADOOP-16718. Allow disabling Server Name Indication (SNI) for Jetty. Contributed by Aravindan Vijayan. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Siyao Meng <smeng@cloudera.com>
(cherry picked from commit f1ab7f18c423a9cfc59292d25fa178e73715b85b)
(cherry picked from commit f0c1403ec382a7a8c25b0311db0c88749576c308)
 2019-12-03 15:48:25 -08:00
Erik Krogen
23f25418e8 HADOOP-16700. Track response time separately in Server.Call to avoid negative RpcQueueTime values. Contributed by xuzq. 
(cherry picked from be77231452baee2c3dd68f7c0991411cae2eea1b)
(cherry picked from c1ed2474595c672d95237c42a6e2a51913f14d25)
 2019-12-03 14:02:00 -08:00
Surendra Singh Lilhore
5eb21b83df HDFS-15010. BlockPoolSlice#addReplicaThreadPool static pool should be initialized by static method. Contributed by Surendra Singh Lilhore. 2019-11-29 00:07:45 +05:30
prabhujoseph
a97f7776bd MAPREDUCE-7249. Fix Invalid event TA_TOO_MANY_FETCH_FAILURE at SUCCESS_CONTAINER_CLEANUP causes job failure. 
Contributed by Wilfred Spiegelenburg.
 2019-11-28 14:34:52 +05:30
Sunil G
57c499ef19 YARN-9949. Add missing queue configs for root queue in RMWebService#CapacitySchedulerInfo. 
Contributed by Prabhu Joseph.
 2019-11-27 23:01:16 +05:30
Szilard Nemeth
1172ebd6e8 YARN-9937. Add missing queue configs in RMWebService#CapacitySchedulerQueueInfo. 
Contributed by Prabhu Joseph.
 2019-11-27 22:30:10 +05:30
prabhujoseph
6a4966f336 MAPREDUCE-7240. Fix Invalid event: TA_TOO_MANY_FETCH_FAILURE at SUCCESS_FINISHING_CONTAINER. 
Contributed by Huachao and Peter Bacsko. Reviewed by Wilfred Spiegelenburg.
 2019-11-27 16:51:47 +05:30
Szilard Nemeth
80a97e6ac2 YARN-9011. Race condition during decommissioning. Contributed by Peter Bacsko 2019-11-26 14:27:55 +01:00
HUAN-PING SU
fb2df5234d
YARN-9966. Code duplication in UserGroupMappingPlacementRule (#1709) 
(cherry picked from commit f8e36e03b4e65f173bf2330715dceafdb6111190)
 2019-11-25 15:58:09 +09:00
Ayush Saxena
35f0478042 HDFS-14519. NameQuota is not update after concat operation, so namequota is wrong. Contributed by Ranith Sardar. 2019-11-23 02:47:59 +05:30
Szilard Nemeth
df157211c9 YARN-9968. Public Localizer is exiting in NodeManager due to NullPointerException. Contributed by Tarun Parimi 2019-11-22 13:00:18 +01:00
Tao Yang
c95e772d5d YARN-9838. Fix resource inconsistency for queues when moving app with reserved container to another queue. Contributed by jiulongzhu. 2019-11-22 16:10:07 +08:00
Surendra Singh Lilhore
8551c04726 HDFS-14940. HDFS Balancer : Do not allow to set balancer maximum network bandwidth more than 1TB. Contributed by hemanthboyina. 2019-11-22 00:45:55 +05:30
Wei-Chiu Chuang
0807cb2485 HADOOP-15686. Supress bogus AbstractWadlGeneratorGrammarGenerator in KMS stderr. Contributed by Wei-Chiu Chuang. 
Reviewed-by: Xiaoyu Yao <xyao@apache.org>
(cherry picked from commit 9664b9c7a65d943e815c119e94234d3bf0b68dd4)
(cherry picked from commit bd2f79cfd8da9279acca826a86ea7bfcc8590bd1)
 2019-11-20 13:08:52 -08:00
Wanqiang Ji
c7d0e0813a
HADOOP-16719. Remove the disallowed element config within maven-checkstyle-plugin (#1724) 
(cherry picked from commit dfc61d8ea5971e9742af77fcae2cae28011ae0c9)
(cherry picked from commit 4c79e2a02bc51f7f57b36198da948ebaca653db2)
 2019-11-20 16:23:21 +09:00
He Xiaoqiao
d101b74814 HDFS-14952. Skip safemode if blockTotal is 0 in new NN. Contributed by Xiaoqiao He. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Mukul Kumar Singh <msingh@apache.org>
(cherry picked from commit 0b50aa29fd5dc114b3e0fc54b5c393bbc9f3102e)
(cherry picked from commit 1d383436f09f8e490171c8f6af4605971ad2f904)
 2019-11-19 12:22:54 -08:00
Szilard Nemeth
0a4e6b7b62 YARN-9836. General usability improvements in showSimulationTrace.html. Contributed by Adam Antal 2019-11-19 21:22:01 +01:00
Siyao Meng
df13f6d485 HADOOP-16703. Backport HADOOP-16152 to branch-3.1. Contributed by Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-11-19 12:14:27 -08:00
Eric Yang
1669a5c1cb YARN-9983. Fixed typo in YARN Service overview. 
Contributed by Denes Gerencser
 2019-11-19 14:18:38 -05:00
Sunil G
9a51380208 YARN-9873. Version Number for each Scheduler Config Change. Contributed by Prabhu Joseph. 
(cherry picked from commit be901f49628a553cfe6a3942478cb528cce2b266)
 2019-11-19 17:17:45 +05:30
Abhishek Modi
96e07b7e97 YARN-9791. Queue Mutation API does not allow to remove a config. Contributed by Prabhu Joseph. 
(cherry picked from commit 751b5a1ac871734c46cb934d100a5367f1dbf791)
 2019-11-19 16:41:04 +05:30
Sunil G
dab4cc6320 YARN-9909. Offline Format of YarnConfigurationStore. Contributed by Prabhu Joseph. 2019-11-19 16:19:19 +05:30
Sunil G
9b17fa7091 YARN-9950. Unset Ordering Policy of Leaf/Parent queue converted from Parent/Leaf queue respectively. Contributed by Prabhu Joseph. 
(cherry picked from commit 51e7d1b37eb06165657befdae82c015055c58334)
 2019-11-19 15:43:19 +05:30
Sunil G
f24d1fac35 YARN-9984. FSPreemptionThread can cause NullPointerException while app is unregistered with containers running on a node. Contributed by Wilfred Spiegelenburg. 
(cherry picked from commit 215f2052fc3b7e366e8bd1bd332663966fa9206c)
 2019-11-19 14:05:51 +05:30
Ayush Saxena
b4f0c63a6a HDFS-14942. Change Log Level to debug in JournalNodeSyncer#syncWithJournalAtIndex. Contributed by Lisheng Sun. 
(cherry picked from commit 9e287054a8aa0725643bc5c90601645302fffade)
(cherry picked from commit d16441460215155b25fa649a7ca733e579d97929)
 2019-11-18 10:23:48 -08:00
Ayush Saxena
7d3c67b4a7 HADOOP-16712. Config ha.failover-controller.active-standby-elector.zk.op.retries is not in core-default.xml. Contributed by Xieming Li. 2019-11-18 19:59:18 +05:30
Prabhu Joseph
d271d174ca YARN-9900. Revert Invalid Config and Refresh Support in SchedulerConfig Format. 
Signed-off-by: prabhujoseph <pjoseph@cloudera.com>
 2019-11-18 15:02:59 +05:30
Lukas Majercak
50d98a6568 HADOOP-16705. MBeanInfoBuilder puts unnecessary memory pressure on the system with a debug log. 
MBeanInfoBuilder's get() method DEBUG logs all the MBeanAttributeInfo attributes that it gathered. This can have a high memory churn that can be easily avoided.

(cherry picked from commit c73334a924d2009d136a8ee62278aaac53418a24)
 2019-11-15 13:29:29 -08:00
Erik Krogen
17779adb32 HDFS-14973. More strictly enforce Balancer/Mover/SPS throttling of getBlocks RPCs to NameNodes. Contributed by Erik Krogen. 
(cherry picked from b2cc8b6b4a78f31cdd937dc4d1a2255f80c5881e)
(cherry picked from 60655bfe54e138957ef5bbf480a4541bd83152fd)
 2019-11-15 11:14:49 -08:00
Akira Ajisaka
b9e4fd5f4c
HADOOP-15097. AbstractContractDeleteTest::testDeleteNonEmptyDirRecursive with misleading path. Contributed by Xieming Li. 
(cherry picked from commit 92c28c100ee1aa414948cd510321ad13cb8639bc)
 2019-11-15 13:51:35 +09:00
Erik Krogen
5cf36aa2b1 HDFS-14979 Allow Balancer to submit getBlocks calls to Observer Nodes when possible. Contributed by Erik Krogen. 
(cherry picked from 586defe7113ed246ed0275bb3833882a3d873d70)
(cherry picked from dec765b329d3947f30273c0e7f0c4eb607ec42c9)
 2019-11-13 11:57:59 -08:00
Xudong Cao
b4f39c2a8a HADOOP-16677. Recalculate the remaining timeout millis correctly while throwing an InterupptedException in SocketIOWithTimeout. (#1687) 
(cherry picked from commit df6b3162c11987ba5299c69cb251332228dacf36)
(cherry picked from commit 975f669141add885d631685d693e81c44c190751)
 2019-11-12 14:21:59 -08:00
Chao Sun
7e0f1eabd8 HDFS-14959: [SBNN read] access time should be turned off (#1706) Contributed by Chao Sun. 
(cherry picked from commit 97ec34e117af71e1a9950b8002131c45754009c7)
(cherry picked from commit 0c12d633b9797f2cb14246fe020993b2e9123e82)
 2019-11-12 11:41:09 -08:00