Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
13,198 Commits 326 Branches 371 Tags
Commit Graph

271 Commits

Author SHA1 Message Date
Vinayakumar B
9a26120809 HDFS-11708. Positional read will fail if replicas moved to different DNs after stream is opened. Contributed by Vinayakumar B. 2017-06-07 10:57:47 +05:30
Brahma Reddy Battula
ac2af66e77 HDFS-11905. Fix license header inconsistency in hdfs. Contributed by Yeliang Cang. 2017-06-02 22:22:43 +08:00
Yiqun Lin
870b5e39d2 HDFS-11359. DFSAdmin report command supports displaying maintenance state datanodes. Contributed by Yiqun Lin. 2017-06-02 12:53:31 +08:00
Andrew Wang
d0a4e7e95d HDFS-11383. Intern strings in BlockLocation and ExtendedBlock. Contributed by Misha Dmitriev. 
(cherry picked from commit 7101477e4726a70ab0eab57c2d4480a04446a8dc)
 2017-06-01 15:20:22 -07:00
Kihwal Lee
212a566088 HDFS-11856. Ability to re-add Upgrading nodes to pipeline for future pipeline updates. Contributed by Vinayakumar B. 2017-05-31 12:45:33 -05:00
Xiao Chen
54971c4195 HDFS-11421. Make WebHDFS' ACLs RegEx configurable. Contributed by Harsh J. 2017-05-25 21:01:39 -07:00
Sunil G
ec346d6377 YARN-6584. Correct license headers in hadoop-common, hdfs, yarn and mapreduce. Contributed by Yeliang Cang. 2017-05-23 07:57:36 +05:30
Andrew Wang
18b89e902b HDFS-11644. Support for querying outputstream capabilities. Contributed by Manoj Govindassamy. 2017-05-13 21:41:11 -07:00
Kihwal Lee
0d9322aa5b HDFS-11702. Remove indefinite caching of key provider uri in DFSClient. Contributed by Rushabh S Shah. 
(cherry picked from commit cef2815cf48154fe82f44082dcbdce6373c81284)
 2017-05-08 08:29:03 -05:00
Lei Xu
1f30a8a105 HDFS-11687. Add new public encryption APIs required by Hive. (lei) 
Change-Id: I4a23a00de63ad18022312ceb1f306a87d032d07c
(cherry picked from commit 25f5d9ad5ee5ead349d259a99b49541a70b1604d)
 2017-05-04 12:07:29 -07:00
Akira Ajisaka
cc66153e12
HDFS-11417. Add datanode admin command to get the storage info. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit 93fa48fcf243dc759db1736af145633da760f937)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/ClientDatanodeProtocol.java
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientDatanodeProtocolTranslatorPB.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientDatanodeProtocolServerSideTranslatorPB.java
 2017-04-26 18:08:12 +09:00
Yongjun Zhang
31d7d47070 HDFS-11689. New exception thrown by DFSClient%isHDFSEncryptionEnabled broke hacky hive code. Contributed by Yongjun Zhang. 
(cherry picked from commit 5078df7be317e635615c05c5da3285798993ff1f)
 2017-04-21 14:01:47 -07:00
Arpit Agarwal
5c092097b6 HDFS-11603. Improve slow mirror/disk warnings in BlockReceiver. 2017-04-18 16:20:43 -07:00
Chris Nauroth
d5e2bd4096 HDFS-11163. Mover should move the file blocks to default storage once policy is unset. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit 00ed21a6fedb45a7c8992b8d45adaa83f14af34c)
 2017-04-11 21:18:47 -07:00
Hanisha Koneru
f16bb79957 HDFS-11551. Support Timeout when checking single disk. Contributed by Hanisha Koneru. 2017-04-11 16:54:28 -07:00
Xiaoyu Yao
0391c92022 HDFS-11608. HDFS write crashed with block size greater than 2 GB. Contributed by Xiaobing Zhou. 
(cherry picked from commit 0eacd4c13be9bad0fbed9421a6539c64bbda4df1)
 2017-04-06 16:15:56 -07:00
Andrew Wang
ed8f738be5 HADOOP-14104. Client should always ask namenode for kms provider path. Contributed by Rushabh S Shah. 2017-04-06 14:38:22 -07:00
Andrew Wang
e154893d3b HDFS-11538. Move ClientProtocol HA proxies into hadoop-hdfs-client. Contributed by Huafeng Wang. 2017-04-05 12:52:34 -07:00
Hanisha Koneru
4cdda4ba87 HDFS-11545. Propagate DataNode's slow disks info to the NameNode via Heartbeat. Contributed by Hanisha Koneru. 2017-04-05 07:35:09 -07:00
Andrew Wang
82b4a9c3d0 HDFS-9705. Refine the behaviour of getFileChecksum when length = 0. Contributed by SammiChen and Kai Zheng. 2017-03-28 21:40:53 -07:00
Andrew Wang
10deafee82 HDFS-11170. Add builder-based create API to FileSystem. Contributed by SammiChen and Wei Zhou. 2017-03-27 14:35:03 -07:00
Zhe Zhang
f98f4bb6ca HADOOP-9631. ViewFs should use underlying FileSystem's server side defaults. Contributed by Lohit Vijayarenu and Erik Krogen. 
(cherry picked from commit 59d69257a888347f0fb9c51bb000afc986b64f98)
 2017-03-22 22:32:07 -07:00
Chris Douglas
69f34e812e HDFS-6648. Order of namenodes in ConfiguredFailoverProxyProvider is undefined. Contributed by Inigo Goiri 
(cherry picked from commit b104f3a282ab8e15314446e71189eb3a3b5dd9a6)
 2017-03-20 17:16:15 -07:00
Xiaobing Zhou
dbdd97d057 HDFS-10394. move declaration of okhttp version from hdfs-client to hadoop-project POM. Contributed by Xiaobing Zhou. 2017-03-16 18:13:55 -07:00
Arpit Agarwal
8dc8ecbb91 HDFS-10838. Last full block report received time for each DN should be easily discoverable. Contributed by Surendra Singh Lilhore. 2017-03-06 16:40:00 -08:00
Yiqun Lin
5b49a96328 HDFS-8741. Proper error msg to be printed when invalid operation type is given to WebHDFS operations. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit 3536ce031ca780d6de83cf67779f571a0142ccc8)
 2017-03-06 19:05:43 +08:00
Wei-Chiu Chuang
f3cdf29af4 HDFS-8498. Blocks can be committed with wrong size. Contributed by Jing Zhao. 2017-02-25 21:13:51 -08:00
Xiaoyu Yao
d72f1c5976 HDFS-11409. DatanodeInfo getNetworkLocation and setNetworkLocation shoud use volatile instead of synchronized. Contributed by Chen Liang. 
(cherry picked from commit aaf27132350547fcde1fdb372f19626838f44bc4)
 2017-02-14 12:55:49 -08:00
Kihwal Lee
33c62d2d19 HDFS-11379. DFSInputStream may infinite loop requesting block locations. Contributed by Daryn Sharp. 2017-02-10 12:52:14 -06:00
Xiao Chen
b0a1703f10 HDFS-11363. Need more diagnosis info when seeing Slow waitForAckedSeqno. 
(cherry picked from commit 0914fcca312b5e9d20bcf1b6633bc13c9034ba46)
 2017-02-02 10:09:20 -08:00
Arpit Agarwal
b4078e1d0e HDFS-11194. Maintain aggregated peer performance metrics on NameNode. 2017-01-24 19:34:33 -08:00
Ming Ma
bed700e98f HDFS-11296. Maintenance state expiry should be an epoch time and not jvm monotonic. (Manoj Govindassamy via mingma) 
(cherry picked from commit f3fb94be05a61a4c4c06ab279897e5de2b181b0e)
 2017-01-19 22:33:43 -08:00
Arpit Agarwal
954dae26cd HDFS-11337. (HDFS-10958 backport). Add instrumentation hooks around Datanode disk IO. 2017-01-16 13:53:30 -08:00
Andrew Wang
b089e122b1 HDFS-11312. Fix incompatible tag number change for nonDfsUsed in DatanodeInfoProto. Contributed by Sean Mackrory. 
(cherry picked from commit d51f8ba80816c20cd97ba5aa839a84b0bb6a7291)
 2017-01-11 15:13:41 -08:00
Haohui Mai
20486730e8 HDFS-11280. Allow WebHDFS to reuse HTTP connections to NN. Contributed by Zheng Shao. 2017-01-04 21:03:05 -08:00
Andrew Wang
d4ca1c5226 HDFS-11156. Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang. 2017-01-04 15:10:15 -08:00
Andrew Wang
2b5b62f9cc Revert "HDFS-11156. Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang." 
This reverts commit 80e664084894bbff9b1cc3102e71b161873fb6ef.
 2017-01-03 11:12:53 -08:00
Andrew Wang
80e6640848 HDFS-11156. Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang. 
(cherry picked from commit 7fcc73fc0d248aae1edbd4e1514c5818f6198928)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/WebHDFS.md
 2017-01-03 09:58:45 -08:00
Brahma Reddy Battula
029b6fbe71 Revert "HDFS-11280. Allow WebHDFS to reuse HTTP connections to NN. Contributed by Zheng Shao." 
This reverts commit 1bc9b316ba60b1ecc8d6fca8d78642933d2c186b.
 2017-01-03 19:26:44 +05:30
Haohui Mai
1bc9b316ba HDFS-11280. Allow WebHDFS to reuse HTTP connections to NN. Contributed by Zheng Shao. 2016-12-30 22:59:24 -08:00
Andrew Wang
6b795c34d0 HDFS-10684. WebHDFS DataNode calls fail without parameter createparent. Contributed by John Zhuge. 
(cherry picked from commit fbdbbd57cdc3d8c778fca9266a7cadf298c8ff6c)
 2016-12-13 18:01:35 -08:00
Vinayakumar B
c6fb9f0d0b HDFS-10570. Remove classpath conflicts of netty-all jar in hadoop-hdfs-client (Contributed by Vinayakumar B) 2016-12-12 12:11:11 +05:30
Andrew Wang
215abbdf5f HDFS-8630. WebHDFS : Support get/set/unset StoragePolicy. Contributed by Surendra Singh Lilhore. 2016-12-09 14:29:22 -08:00
Vinayakumar B
4b21b47f56 HDFS-9057. allow/disallow snapshots via webhdfs (Contributed by Brahma Reddy Battula) 
(cherry picked from commit 6d2332ae375e26d024358c6e75fdb3c68a781a66)
 2016-12-07 15:58:15 -08:00
Andrew Wang
6a2f239d9e Revert "HDFS-11156. Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang" 
This reverts commit be969e591883aa6cdd69bb62cea4e8904ece65f1.
 2016-12-05 23:09:14 -08:00
Mingliang Liu
be969e5918 HDFS-11156. Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang 
(cherry picked from commit c7ff34f8dcca3a2024230c5383abd9299daa1b20)
 2016-12-02 11:36:11 -08:00
Kihwal Lee
f08e227671 HDFS-8870. Lease is leaked on write failure. Contributed by Kuhu Shukla. 
(cherry picked from commit 4fcea8a0c8019d6d9a5e6f315c83659938b93a40)
 2016-11-15 14:51:34 -06:00
Mingliang Liu
4e0fcff8ab HADOOP-13427. Eliminate needless uses of FileSystem#{exists(), isFile(), isDirectory()}. Contributed by Steve Loughran and Mingliang Liu 
(cherry picked from commit 5af572b6443715b7a741296c1bd520a1840f9a7c)
 2016-11-15 11:36:08 -08:00
Brahma Reddy Battula
53ff3c9e03 HDFS-9482. Replace DatanodeInfo constructors with a builder pattern. Contributed by Brahma Reddy Battula. 2016-11-09 09:58:36 +05:30
Andrew Wang
4f25fd975e HADOOP-13789. Hadoop Common includes generated test protos in both jar and test-jar. Contributed by Sean Busbey. 
(cherry picked from commit e1c6ef2efa9d87fdfc7474ca63998a13a3929874)
 2016-11-08 16:33:58 -08:00