Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
19,449 Commits 326 Branches 371 Tags
Commit Graph

19449 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Erik Krogen
23f25418e8 HADOOP-16700. Track response time separately in Server.Call to avoid negative RpcQueueTime values. Contributed by xuzq. 
(cherry picked from be77231452baee2c3dd68f7c0991411cae2eea1b)
(cherry picked from c1ed2474595c672d95237c42a6e2a51913f14d25)
 2019-12-03 14:02:00 -08:00
Surendra Singh Lilhore
5eb21b83df HDFS-15010. BlockPoolSlice#addReplicaThreadPool static pool should be initialized by static method. Contributed by Surendra Singh Lilhore. 2019-11-29 00:07:45 +05:30
prabhujoseph
a97f7776bd MAPREDUCE-7249. Fix Invalid event TA_TOO_MANY_FETCH_FAILURE at SUCCESS_CONTAINER_CLEANUP causes job failure. 
Contributed by Wilfred Spiegelenburg.
 2019-11-28 14:34:52 +05:30
Sunil G
57c499ef19 YARN-9949. Add missing queue configs for root queue in RMWebService#CapacitySchedulerInfo. 
Contributed by Prabhu Joseph.
 2019-11-27 23:01:16 +05:30
Szilard Nemeth
1172ebd6e8 YARN-9937. Add missing queue configs in RMWebService#CapacitySchedulerQueueInfo. 
Contributed by Prabhu Joseph.
 2019-11-27 22:30:10 +05:30
prabhujoseph
6a4966f336 MAPREDUCE-7240. Fix Invalid event: TA_TOO_MANY_FETCH_FAILURE at SUCCESS_FINISHING_CONTAINER. 
Contributed by Huachao and Peter Bacsko. Reviewed by Wilfred Spiegelenburg.
 2019-11-27 16:51:47 +05:30
Szilard Nemeth
80a97e6ac2 YARN-9011. Race condition during decommissioning. Contributed by Peter Bacsko 2019-11-26 14:27:55 +01:00
HUAN-PING SU
fb2df5234d
YARN-9966. Code duplication in UserGroupMappingPlacementRule (#1709) 
(cherry picked from commit f8e36e03b4e65f173bf2330715dceafdb6111190)
 2019-11-25 15:58:09 +09:00
Ayush Saxena
35f0478042 HDFS-14519. NameQuota is not update after concat operation, so namequota is wrong. Contributed by Ranith Sardar. 2019-11-23 02:47:59 +05:30
Szilard Nemeth
df157211c9 YARN-9968. Public Localizer is exiting in NodeManager due to NullPointerException. Contributed by Tarun Parimi 2019-11-22 13:00:18 +01:00
Tao Yang
c95e772d5d YARN-9838. Fix resource inconsistency for queues when moving app with reserved container to another queue. Contributed by jiulongzhu. 2019-11-22 16:10:07 +08:00
Surendra Singh Lilhore
8551c04726 HDFS-14940. HDFS Balancer : Do not allow to set balancer maximum network bandwidth more than 1TB. Contributed by hemanthboyina. 2019-11-22 00:45:55 +05:30
Wei-Chiu Chuang
0807cb2485 HADOOP-15686. Supress bogus AbstractWadlGeneratorGrammarGenerator in KMS stderr. Contributed by Wei-Chiu Chuang. 
Reviewed-by: Xiaoyu Yao <xyao@apache.org>
(cherry picked from commit 9664b9c7a65d943e815c119e94234d3bf0b68dd4)
(cherry picked from commit bd2f79cfd8da9279acca826a86ea7bfcc8590bd1)
 2019-11-20 13:08:52 -08:00
Wanqiang Ji
c7d0e0813a
HADOOP-16719. Remove the disallowed element config within maven-checkstyle-plugin (#1724) 
(cherry picked from commit dfc61d8ea5971e9742af77fcae2cae28011ae0c9)
(cherry picked from commit 4c79e2a02bc51f7f57b36198da948ebaca653db2)
 2019-11-20 16:23:21 +09:00
He Xiaoqiao
d101b74814 HDFS-14952. Skip safemode if blockTotal is 0 in new NN. Contributed by Xiaoqiao He. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Mukul Kumar Singh <msingh@apache.org>
(cherry picked from commit 0b50aa29fd5dc114b3e0fc54b5c393bbc9f3102e)
(cherry picked from commit 1d383436f09f8e490171c8f6af4605971ad2f904)
 2019-11-19 12:22:54 -08:00
Szilard Nemeth
0a4e6b7b62 YARN-9836. General usability improvements in showSimulationTrace.html. Contributed by Adam Antal 2019-11-19 21:22:01 +01:00
Siyao Meng
df13f6d485 HADOOP-16703. Backport HADOOP-16152 to branch-3.1. Contributed by Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-11-19 12:14:27 -08:00
Eric Yang
1669a5c1cb YARN-9983. Fixed typo in YARN Service overview. 
Contributed by Denes Gerencser
 2019-11-19 14:18:38 -05:00
Sunil G
9a51380208 YARN-9873. Version Number for each Scheduler Config Change. Contributed by Prabhu Joseph. 
(cherry picked from commit be901f49628a553cfe6a3942478cb528cce2b266)
 2019-11-19 17:17:45 +05:30
Abhishek Modi
96e07b7e97 YARN-9791. Queue Mutation API does not allow to remove a config. Contributed by Prabhu Joseph. 
(cherry picked from commit 751b5a1ac871734c46cb934d100a5367f1dbf791)
 2019-11-19 16:41:04 +05:30
Sunil G
dab4cc6320 YARN-9909. Offline Format of YarnConfigurationStore. Contributed by Prabhu Joseph. 2019-11-19 16:19:19 +05:30
Sunil G
9b17fa7091 YARN-9950. Unset Ordering Policy of Leaf/Parent queue converted from Parent/Leaf queue respectively. Contributed by Prabhu Joseph. 
(cherry picked from commit 51e7d1b37eb06165657befdae82c015055c58334)
 2019-11-19 15:43:19 +05:30
Sunil G
f24d1fac35 YARN-9984. FSPreemptionThread can cause NullPointerException while app is unregistered with containers running on a node. Contributed by Wilfred Spiegelenburg. 
(cherry picked from commit 215f2052fc3b7e366e8bd1bd332663966fa9206c)
 2019-11-19 14:05:51 +05:30
Ayush Saxena
b4f0c63a6a HDFS-14942. Change Log Level to debug in JournalNodeSyncer#syncWithJournalAtIndex. Contributed by Lisheng Sun. 
(cherry picked from commit 9e287054a8aa0725643bc5c90601645302fffade)
(cherry picked from commit d16441460215155b25fa649a7ca733e579d97929)
 2019-11-18 10:23:48 -08:00
Ayush Saxena
7d3c67b4a7 HADOOP-16712. Config ha.failover-controller.active-standby-elector.zk.op.retries is not in core-default.xml. Contributed by Xieming Li. 2019-11-18 19:59:18 +05:30
Prabhu Joseph
d271d174ca YARN-9900. Revert Invalid Config and Refresh Support in SchedulerConfig Format. 
Signed-off-by: prabhujoseph <pjoseph@cloudera.com>
 2019-11-18 15:02:59 +05:30
Lukas Majercak
50d98a6568 HADOOP-16705. MBeanInfoBuilder puts unnecessary memory pressure on the system with a debug log. 
MBeanInfoBuilder's get() method DEBUG logs all the MBeanAttributeInfo attributes that it gathered. This can have a high memory churn that can be easily avoided.

(cherry picked from commit c73334a924d2009d136a8ee62278aaac53418a24)
 2019-11-15 13:29:29 -08:00
Erik Krogen
17779adb32 HDFS-14973. More strictly enforce Balancer/Mover/SPS throttling of getBlocks RPCs to NameNodes. Contributed by Erik Krogen. 
(cherry picked from b2cc8b6b4a78f31cdd937dc4d1a2255f80c5881e)
(cherry picked from 60655bfe54e138957ef5bbf480a4541bd83152fd)
 2019-11-15 11:14:49 -08:00
Akira Ajisaka
b9e4fd5f4c
HADOOP-15097. AbstractContractDeleteTest::testDeleteNonEmptyDirRecursive with misleading path. Contributed by Xieming Li. 
(cherry picked from commit 92c28c100ee1aa414948cd510321ad13cb8639bc)
 2019-11-15 13:51:35 +09:00
Erik Krogen
5cf36aa2b1 HDFS-14979 Allow Balancer to submit getBlocks calls to Observer Nodes when possible. Contributed by Erik Krogen. 
(cherry picked from 586defe7113ed246ed0275bb3833882a3d873d70)
(cherry picked from dec765b329d3947f30273c0e7f0c4eb607ec42c9)
 2019-11-13 11:57:59 -08:00
Xudong Cao
b4f39c2a8a HADOOP-16677. Recalculate the remaining timeout millis correctly while throwing an InterupptedException in SocketIOWithTimeout. (#1687) 
(cherry picked from commit df6b3162c11987ba5299c69cb251332228dacf36)
(cherry picked from commit 975f669141add885d631685d693e81c44c190751)
 2019-11-12 14:21:59 -08:00
Chao Sun
7e0f1eabd8 HDFS-14959: [SBNN read] access time should be turned off (#1706) Contributed by Chao Sun. 
(cherry picked from commit 97ec34e117af71e1a9950b8002131c45754009c7)
(cherry picked from commit 0c12d633b9797f2cb14246fe020993b2e9123e82)
 2019-11-12 11:41:09 -08:00
Surendra Singh Lilhore
9179046368 HDFS-14720. DataNode shouldn't report block as bad block if the block length is Long.MAX_VALUE. Contributed by hemanthboyina. 
(cherry picked from commit 320008bb7cc558b1300398178bd2f48cbf0b6c80)
 2019-11-09 22:36:48 +05:30
Ayush Saxena
9aa59c0629 HDFS-14958. TestBalancerWithNodeGroup is not using NetworkTopologyWithNodeGroup. Contributed by Jim Brennan. 2019-11-07 22:42:07 +05:30
Akira Ajisaka
3ac564ddb1 HDFS-14590. [SBN Read] Add the document link to the top page. Contributed by Takanobu Asanuma. 
(cherry picked from commit 48d7f00da8aab25707dc08219122541867edfa65)
 2019-11-07 22:30:58 +09:00
Chen Liang
76ca864406 HDFS-14806. Bootstrap standby may fail if with in-progress tailing. Contributed by Chen Liang and Erik Krogen. 2019-11-06 16:21:41 -08:00
Chen Liang
c6cbf67b82 HDFS-14941. Potential editlog race condition can cause corrupted file. Contributed by Chen Liang and Konstantin Shvachko. 2019-11-06 15:11:09 -08:00
Ayush Saxena
5841d0b2f9 HDFS-14946. Erasure Coding: Block recovery failed during decommissioning. Contributed by Fei Hui. 2019-11-06 22:17:11 +05:30
Surendra Singh Lilhore
4690142e86 HDFS-14384. When lastLocatedBlock token expire, it will take 1~3s second to refetch it. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit c36014165c212b26d75268ee3659aa2cadcff349)
 2019-11-06 19:48:16 +05:30
Ahmed Hussein
cbd501636c MAPREDUCE-7208. Tuning TaskRuntimeEstimator. (Ahmed Hussein via jeagles) 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit ed302f1fed6d124d682486d24dae958946dba9be)
 2019-11-05 15:29:19 -06:00
Chen Liang
7b4f44573f HADOOP-16013. DecayRpcScheduler decay thread should run as a daemon. Contributed by Erik Krogen. 
(cherry picked from commit 01cb958af44b2376bcf579cc65d90566530f733d)
(cherry picked from commit a58f466d074becff11b0757b1b3cf9c8f6bb57a8)
 2019-11-04 17:18:37 -08:00
Wei-Chiu Chuang
c9ff4b7a1b HDFS-14945. Revise PacketResponder's log. Contributed by Xudong Cao. 
(cherry picked from commit eb73ba6ed5f7c5500cc0ef36ca22aae4e71046fa)
(cherry picked from commit 240b5ee3802550950707c91f22154386d915fffb)
 2019-11-04 09:41:05 -08:00
Zhao Junwang
cda8b2ac69 HDFS-14925. Rename operation should check nest snapshot (#1670) 
If the src directory or any of its descendant is snapshottable
and the dst directory or any of its ancestors is snapshottable,
we consider this as nested snapshot, which should be denied.

Reviewed-by: Shashikant Banerjee <shashikant@apache.org>
(cherry picked from commit de6b8b0c0b1933aab2af3e8adc50a2091d428238)
(cherry picked from commit c9fc118991104acf9109aa769fe1a0e84f425a61)
 2019-11-01 16:39:29 -07:00
Ayush Saxena
9956d18129 HDFS-13736. BlockPlacementPolicyDefault can not choose favored nodes when 'dfs.namenode.block-placement-policy.default.prefer-local-node' set to false. Contributed by hu xiaodong. 2019-11-01 22:52:12 +05:30
Jonathan Hung
f29481f046 Add 2.10.0 release notes for HDFS-12943 
(cherry picked from commit ef9d12df24c0db76fd37a95551db7920d27d740c)
(cherry picked from commit 9ccfc2d6d3f3819da58ea9f0ff056ed03970a6e1)
 2019-10-31 20:35:20 -07:00
Ayush Saxena
98e3c6af2c HDFS-14920. Erasure Coding: Decommission may hang If one or more datanodes are out of service during decommission. Contributed by Fei Hui. 2019-10-31 23:53:37 +05:30
Duo Zhang
07f59c023c HADOOP-16598. Backport "HADOOP-16558 [COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes" to all active branches 
Signed-off-by: stack <stack@apache.org>
 2019-10-30 21:30:34 -07:00
Jonathan Hung
4d274f60bc Make upstream aware of 2.10.0 release 
(cherry picked from commit 7663db59c097c82eeed2df7a91168a4d7123c96b)
(cherry picked from commit 5d2ffcc7aac608746946dc2ed5711549a01a2055)
 2019-10-30 21:00:55 -07:00
Steve Loughran
ad881214bc HADOOP-16669. TestRawLocalFileSystemContract.testPermission fails if no native library. 
Contributed by Steve Loughran.

Change-Id: I622e36f17a63c775c5e33e93b8b26e461d0bc451
 2019-10-29 17:11:25 +00:00
Eric Badger
50c1f932c3 HDFS-14931. hdfs crypto commands limit column width. Contributed by Eric Badger 
(cherry picked from commit 9ef6ed9c1c83b9752e772ece7a716a33045752bf)
 2019-10-28 16:54:05 +00:00