Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,217 Commits 326 Branches 371 Tags
Commit Graph

4737 Commits

Author SHA1 Message Date
Brahma Reddy Battula
7bad748091 HDFS-11445. FSCK shows overall health stauts as corrupt even one replica is corrupt. Contributed by Brahma Reddy Battula. 2017-05-25 22:38:12 +08:00
Yongjun Zhang
49dc547252 HADOOP-11794. Enable distcp to copy blocks in parallel. Contributed by Yongjun Zhang, Wei-Chiu Chuang, Xiao Chen, Rosie Li. 2017-05-24 18:54:51 -07:00
Yongjun Zhang
97317b1aaa Revert "HADOOP-11794. Enable distcp to copy blocks in parallel. Contributed by Yongjun Zhang, Wei-Chiu Chuang, Xiao Chen, Rosie Li." 
This reverts commit c411adac32cda42433b15979253248336487d977.
 2017-05-24 18:54:51 -07:00
Wei-Chiu Chuang
bd0138ea0a Revert "HDFS-10797. Disk usage summary of snapshots causes renamed blocks to get counted twice. Contributed by Sean Mackrory." 
This reverts commit 5c96ef365de1a7ffc6a13f2d64cda9a26d59e4aa.
 2017-05-24 18:08:30 -07:00
Wei-Chiu Chuang
fe2c28df70 Revert "HDFS-11515. -du throws ConcurrentModificationException. Contributed by Istvan Fajth, Wei-Chiu Chuang." 
This reverts commit 4f1f16f1a67dcb6db705a73bf7fcf3f0f9d7094c.
 2017-05-24 18:08:10 -07:00
Sunil G
ec346d6377 YARN-6584. Correct license headers in hadoop-common, hdfs, yarn and mapreduce. Contributed by Yeliang Cang. 2017-05-23 07:57:36 +05:30
Brahma Reddy Battula
ec2468ced1 HDFS-11849. JournalNode startup failure exception should be logged in log file. Contributed By Surendra Singh Lilhore. 
(cherry picked from commit 9cab42cc797986081fef184748044f1790a4f039)
 2017-05-23 02:40:11 +08:00
John Zhuge
85f7b7e8e4 HDFS-11579. Make HttpFS Tomcat SSL property sslEnabledProtocols and clientAuth configurable. Contributed by John Zhuge. 2017-05-18 18:04:05 -07:00
Arpit Agarwal
3db695d400 HDFS-11842. TestDataNodeOutlierDetectionViaMetrics UT fails intermittently. Controbuted by Hanisha Koneru. 2017-05-17 19:38:54 -07:00
Arpit Agarwal
0e95ff78c3 HDFS-11827. NPE is thrown when log level changed in BlockPlacementPolicyDefault#chooseRandom() method. Contributed by xupeng. 2017-05-16 19:51:17 -07:00
Kihwal Lee
82ea3f4545 HDFS-11641. Reduce cost of audit logging by using FileStatus instead of HdfsFileStatus. Contributed by Daryn Sharp. 2017-05-16 15:45:41 -05:00
Akira Ajisaka
c17cb03a23
HDFS-11833. HDFS architecture documentation descibes outdated placement policy. Contributed by Chen Liang. 
(cherry picked from commit 1d1c52b42feae5a4271ef4b771d0d8de43e83c15)
 2017-05-16 11:53:55 -04:00
John Zhuge
da7c900fe3 HDFS-11816. Update default SSL cipher list for HttpFS. Contributed by John Zhuge. 2017-05-15 16:26:39 -07:00
Andrew Wang
18b89e902b HDFS-11644. Support for querying outputstream capabilities. Contributed by Manoj Govindassamy. 2017-05-13 21:41:11 -07:00
Jason Lowe
aeca471ec1 HDFS-11818. TestBlockManager.testSufficientlyReplBlocksUsesNewRack fails intermittently. Contributed by Nathan Roberts 2017-05-12 17:47:17 -05:00
Vinayakumar B
53d9f56a18 HDFS-11674. reserveSpaceForReplicas is not released if append request failed due to mirror down and replica recovered (Contributed by Vinayakumar B) 
(cherry picked from commit 1411612aa4e70c704b941723217ed4efd8a0125b)
 2017-05-12 07:44:11 +05:30
Kihwal Lee
dfaaa14a47 HDFS-11755. Underconstruction blocks can be considered missing. Contributed by Nathan Roberts. 2017-05-10 14:20:26 -05:00
Jason Lowe
0a4021387f HDFS-11745. Increase HDFS test timeouts from 1 second to 10 seconds. Contributed by Eric Badger 
(cherry picked from commit fab73748a1ce982904d332e6ddd46169f186ad54)
 2017-05-10 13:03:33 -05:00
Kihwal Lee
0d9322aa5b HDFS-11702. Remove indefinite caching of key provider uri in DFSClient. Contributed by Rushabh S Shah. 
(cherry picked from commit cef2815cf48154fe82f44082dcbdce6373c81284)
 2017-05-08 08:29:03 -05:00
Lei Xu
1f30a8a105 HDFS-11687. Add new public encryption APIs required by Hive. (lei) 
Change-Id: I4a23a00de63ad18022312ceb1f306a87d032d07c
(cherry picked from commit 25f5d9ad5ee5ead349d259a99b49541a70b1604d)
 2017-05-04 12:07:29 -07:00
Arpit Agarwal
89baea96d5 HDFS-11722. Change Datanode file IO profiling sampling to percentage. Contributed by Hanisha Koneru. 2017-05-03 16:30:01 -07:00
Arpit Agarwal
4b7d1ff326 Revert "HDFS-11488. JN log segment syncing should support HA upgrade. Contributed by Hanisha Koneru." 
This reverts commit dfc6ef42c4d96133c4d6245c959ee8540bd12f91.
 2017-05-03 16:29:44 -07:00
Arpit Agarwal
dfc6ef42c4 HDFS-11488. JN log segment syncing should support HA upgrade. Contributed by Hanisha Koneru. 2017-05-03 15:53:43 -07:00
Mingliang Liu
616413ecb2 HDFS-11739. Fix regression in tests caused by YARN-679. Contributed by Steve Loughran 
(cherry picked from commit 83dded556dc179fcff078451fb80533065e116f0)
 2017-05-03 11:23:15 -07:00
Erik Krogen
9d6b873d3c HDFS-11717. Add unit test for HDFS-11709 StandbyCheckpointer should handle non-existing legacyOivImageDir gracefully. Contributed by Erik Krogen. 
(cherry picked from commit d9014bda93760f223789d2ec9f5e35f40de157d4)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestStandbyCheckpoints.java
 2017-05-02 18:39:37 -07:00
Xiaoyu Yao
3e7f97fb0b HDFS-11593. Change SimpleHttpProxyHandler#exceptionCaught log level from info to debug. Contributed by Xiaobing Zhou. 
(cherry picked from commit 1058b4084392fbc18522ba126313d4a77c09d797)
 2017-05-01 16:13:30 -07:00
Kihwal Lee
f77d138725 HDFS-11714. Newly added NN storage directory won't get initialized and cause space exhaustion. Contributed by Kihwal Lee. 2017-05-01 17:33:24 -05:00
Kihwal Lee
6a6d0ece99 HDFS-11609. Some blocks can be permanently lost if nodes are decommissioned while dead. Contributed by Kihwal Lee. 2017-05-01 14:20:37 -05:00
Zhe Zhang
fc46f398d9 HDFS-11709. StandbyCheckpointer should handle an non-existing legacyOivImageDir gracefully. Contributed by Erik Krogen. 
(cherry picked from commit d8a33098309f17dfb0e3a000934f68394de44bf7)
 2017-04-27 09:51:04 -07:00
Konstantin V Shvachko
4cbf5c5c41 HDFS-11384. Balancer disperses getBlocks calls to avoid NameNode's rpc queue saturation. Contributed by Konstantin V Shvachko. 
(cherry picked from commit 28eb2aabebd15c15a357d86e23ca407d3c85211c)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java
 2017-04-26 19:02:48 -07:00
Akira Ajisaka
cc66153e12
HDFS-11417. Add datanode admin command to get the storage info. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit 93fa48fcf243dc759db1736af145633da760f937)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/ClientDatanodeProtocol.java
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientDatanodeProtocolTranslatorPB.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientDatanodeProtocolServerSideTranslatorPB.java
 2017-04-26 18:08:12 +09:00
Xiaoyu Yao
fc0e6de929 HADOOP-14207. "dfsadmin -refreshCallQueue" fails with DecayRpcScheduler. Contributed by Surendra Singh Lihore. 
(cherry picked from commit 918dee10f56861d15bb6644edcefb4246a3e00a4)
 2017-04-25 18:49:44 -07:00
Jason Lowe
31af85c18e HDFS-11691. Add a proper scheme to the datanode links in NN web UI. Contributed by Kihwal Lee 
(cherry picked from commit e4321ec84321672a714419278946fe1012daac71)
 2017-04-25 17:31:31 -05:00
Yongjun Zhang
31d7d47070 HDFS-11689. New exception thrown by DFSClient%isHDFSEncryptionEnabled broke hacky hive code. Contributed by Yongjun Zhang. 
(cherry picked from commit 5078df7be317e635615c05c5da3285798993ff1f)
 2017-04-21 14:01:47 -07:00
John Zhuge
5046a45507 HDFS-11685. TestDistributedFileSystem.java fails to compile. Contributed by John Zhuge. 2017-04-19 22:10:18 -07:00
Andrew Wang
efb790b0d6 HDFS-11660. TestFsDatasetCache#testPageRounder fails intermittently with AssertionError. 
(cherry picked from commit 74a723852d9ef265d0fd46334f2f885bfb247a3a)
 2017-04-19 18:10:16 -07:00
Yiqun Lin
e278eb3ab5 HDFS-11316. TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailure fails in trunk. Contributed by Yiqun Lin. 
(cherry picked from commit 724bb8635144dd16fc18cf95a78db6c3d5d1c468)
 2017-04-19 12:15:52 -05:00
Arpit Agarwal
5c092097b6 HDFS-11603. Improve slow mirror/disk warnings in BlockReceiver. 2017-04-18 16:20:43 -07:00
Konstantin V Shvachko
c3840bde54 HDFS-11634. Optimize BlockIterator when interating starts in the middle. Contributed by Konstantin V Shvachko. 
(cherry picked from commit 8dfcd95d580bb090af7f40af0a57061518c18c8c)
 2017-04-17 17:06:18 -07:00
Zhe Zhang
051ab77806 HDFS-11615. FSNamesystemLock metrics can be inaccurate due to millisecond precision. Contributed by Erik Krogen. 
(cherry picked from commit ad49098eb324e238d97db68d7239ed2c4d84afa0)
 2017-04-17 16:28:58 -07:00
Yongjun Zhang
c411adac32 HADOOP-11794. Enable distcp to copy blocks in parallel. Contributed by Yongjun Zhang, Wei-Chiu Chuang, Xiao Chen, Rosie Li. 2017-04-14 10:14:02 -07:00
Hanisha Koneru
107f685f64 HDFS-11630. TestThrottledAsyncCheckerTimeout fails intermittently in Jenkins builds. Contributed by Hanisha Koneru. 2017-04-12 17:11:36 -07:00
Hanisha Koneru
ed8bb51f59 HDFS-11560. Expose slow disks via NameNode JMX. Contributed by Hanisha Koneru. 2017-04-12 16:59:37 -07:00
Kihwal Lee
8ed230c805 HDFS-11648. Lazy construct the IIP pathname. Contributed by Daryn Sharp. 2017-04-12 13:28:26 -05:00
Chris Nauroth
d5e2bd4096 HDFS-11163. Mover should move the file blocks to default storage once policy is unset. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit 00ed21a6fedb45a7c8992b8d45adaa83f14af34c)
 2017-04-11 21:18:47 -07:00
Mingliang Liu
f1a85b2be3 HDFS-11558. BPServiceActor thread name is too long. Contributed by Xiaobing Zhou 2017-04-11 18:09:43 -07:00
Hanisha Koneru
f16bb79957 HDFS-11551. Support Timeout when checking single disk. Contributed by Hanisha Koneru. 2017-04-11 16:54:28 -07:00
Xiaoyu Yao
0391c92022 HDFS-11608. HDFS write crashed with block size greater than 2 GB. Contributed by Xiaobing Zhou. 
(cherry picked from commit 0eacd4c13be9bad0fbed9421a6539c64bbda4df1)
 2017-04-06 16:15:56 -07:00
Andrew Wang
ed8f738be5 HADOOP-14104. Client should always ask namenode for kms provider path. Contributed by Rushabh S Shah. 2017-04-06 14:38:22 -07:00
Xiaoyu Yao
325be6e720 HDFS-11362. StorageDirectory should initialize a non-null default StorageDirType. Contribute by Hanisha Koneru. 
(cherry picked from commit dde3bbf351ef3121ae80a96d06668d8fe33c788f)
 2017-04-06 14:33:30 -07:00