Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
13,282 Commits 326 Branches 371 Tags
Commit Graph

277 Commits

Author SHA1 Message Date
Wei-Chiu Chuang
6917c19167 HDFS-11881. NameNode consumes a lot of memory for snapshot diff report generation. Contributed by Manoj Govindassamy. 
(cherry picked from commit 16c8dbde574f49827fde5ee9add1861ee65d4645)
 2017-06-29 06:40:24 -07:00
Ravi Prakash
ef491f4637 HDFS-11993. Add log info when connect to datanode socket address failed. Contributed by chencan 
(cherry picked from commit a9d3412b4ce40f5ab5a18756ede7e0606b653171)
 2017-06-26 13:25:32 -07:00
Andrew Wang
1ec1aebd35 HDFS-10480. Add an admin command to list currently open files. Contributed by Manoj Govindassamy. 2017-06-19 09:52:07 -07:00
Lei Xu
33afa1fdca HADOOP-14395. Provide Builder pattern for DistributedFileSystem.append. Contributed by Lei (Eddy) Xu. 
(cherry picked from commit 6460df21a09a7fcc29eceb8dc3859d6298da6882)

    Conflicts:
    hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
 2017-06-16 17:32:29 -07:00
Xiao Chen
2afe9722af HADOOP-14524. Make CryptoCodec Closeable so it can be cleaned up proactively. 2017-06-16 09:49:41 -07:00
Lei Xu
abac844c90 HADOOP-14394. Provide Builder pattern for DistributedFileSystem.create. (lei) 
(cherry picked from commit 5fbec46525d6d49837d934556b59ba77bd2301a8)

    Conflicts:
    hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
    hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/NameNodeConnector.java
    hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDistributedFileSystem.java
    hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestErasureCodingPolicies.java
 2017-06-15 10:59:11 -07:00
Vinayakumar B
9a26120809 HDFS-11708. Positional read will fail if replicas moved to different DNs after stream is opened. Contributed by Vinayakumar B. 2017-06-07 10:57:47 +05:30
Brahma Reddy Battula
ac2af66e77 HDFS-11905. Fix license header inconsistency in hdfs. Contributed by Yeliang Cang. 2017-06-02 22:22:43 +08:00
Yiqun Lin
870b5e39d2 HDFS-11359. DFSAdmin report command supports displaying maintenance state datanodes. Contributed by Yiqun Lin. 2017-06-02 12:53:31 +08:00
Andrew Wang
d0a4e7e95d HDFS-11383. Intern strings in BlockLocation and ExtendedBlock. Contributed by Misha Dmitriev. 
(cherry picked from commit 7101477e4726a70ab0eab57c2d4480a04446a8dc)
 2017-06-01 15:20:22 -07:00
Kihwal Lee
212a566088 HDFS-11856. Ability to re-add Upgrading nodes to pipeline for future pipeline updates. Contributed by Vinayakumar B. 2017-05-31 12:45:33 -05:00
Xiao Chen
54971c4195 HDFS-11421. Make WebHDFS' ACLs RegEx configurable. Contributed by Harsh J. 2017-05-25 21:01:39 -07:00
Sunil G
ec346d6377 YARN-6584. Correct license headers in hadoop-common, hdfs, yarn and mapreduce. Contributed by Yeliang Cang. 2017-05-23 07:57:36 +05:30
Andrew Wang
18b89e902b HDFS-11644. Support for querying outputstream capabilities. Contributed by Manoj Govindassamy. 2017-05-13 21:41:11 -07:00
Kihwal Lee
0d9322aa5b HDFS-11702. Remove indefinite caching of key provider uri in DFSClient. Contributed by Rushabh S Shah. 
(cherry picked from commit cef2815cf48154fe82f44082dcbdce6373c81284)
 2017-05-08 08:29:03 -05:00
Lei Xu
1f30a8a105 HDFS-11687. Add new public encryption APIs required by Hive. (lei) 
Change-Id: I4a23a00de63ad18022312ceb1f306a87d032d07c
(cherry picked from commit 25f5d9ad5ee5ead349d259a99b49541a70b1604d)
 2017-05-04 12:07:29 -07:00
Akira Ajisaka
cc66153e12
HDFS-11417. Add datanode admin command to get the storage info. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit 93fa48fcf243dc759db1736af145633da760f937)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/ClientDatanodeProtocol.java
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientDatanodeProtocolTranslatorPB.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientDatanodeProtocolServerSideTranslatorPB.java
 2017-04-26 18:08:12 +09:00
Yongjun Zhang
31d7d47070 HDFS-11689. New exception thrown by DFSClient%isHDFSEncryptionEnabled broke hacky hive code. Contributed by Yongjun Zhang. 
(cherry picked from commit 5078df7be317e635615c05c5da3285798993ff1f)
 2017-04-21 14:01:47 -07:00
Arpit Agarwal
5c092097b6 HDFS-11603. Improve slow mirror/disk warnings in BlockReceiver. 2017-04-18 16:20:43 -07:00
Chris Nauroth
d5e2bd4096 HDFS-11163. Mover should move the file blocks to default storage once policy is unset. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit 00ed21a6fedb45a7c8992b8d45adaa83f14af34c)
 2017-04-11 21:18:47 -07:00
Hanisha Koneru
f16bb79957 HDFS-11551. Support Timeout when checking single disk. Contributed by Hanisha Koneru. 2017-04-11 16:54:28 -07:00
Xiaoyu Yao
0391c92022 HDFS-11608. HDFS write crashed with block size greater than 2 GB. Contributed by Xiaobing Zhou. 
(cherry picked from commit 0eacd4c13be9bad0fbed9421a6539c64bbda4df1)
 2017-04-06 16:15:56 -07:00
Andrew Wang
ed8f738be5 HADOOP-14104. Client should always ask namenode for kms provider path. Contributed by Rushabh S Shah. 2017-04-06 14:38:22 -07:00
Andrew Wang
e154893d3b HDFS-11538. Move ClientProtocol HA proxies into hadoop-hdfs-client. Contributed by Huafeng Wang. 2017-04-05 12:52:34 -07:00
Hanisha Koneru
4cdda4ba87 HDFS-11545. Propagate DataNode's slow disks info to the NameNode via Heartbeat. Contributed by Hanisha Koneru. 2017-04-05 07:35:09 -07:00
Andrew Wang
82b4a9c3d0 HDFS-9705. Refine the behaviour of getFileChecksum when length = 0. Contributed by SammiChen and Kai Zheng. 2017-03-28 21:40:53 -07:00
Andrew Wang
10deafee82 HDFS-11170. Add builder-based create API to FileSystem. Contributed by SammiChen and Wei Zhou. 2017-03-27 14:35:03 -07:00
Zhe Zhang
f98f4bb6ca HADOOP-9631. ViewFs should use underlying FileSystem's server side defaults. Contributed by Lohit Vijayarenu and Erik Krogen. 
(cherry picked from commit 59d69257a888347f0fb9c51bb000afc986b64f98)
 2017-03-22 22:32:07 -07:00
Chris Douglas
69f34e812e HDFS-6648. Order of namenodes in ConfiguredFailoverProxyProvider is undefined. Contributed by Inigo Goiri 
(cherry picked from commit b104f3a282ab8e15314446e71189eb3a3b5dd9a6)
 2017-03-20 17:16:15 -07:00
Xiaobing Zhou
dbdd97d057 HDFS-10394. move declaration of okhttp version from hdfs-client to hadoop-project POM. Contributed by Xiaobing Zhou. 2017-03-16 18:13:55 -07:00
Arpit Agarwal
8dc8ecbb91 HDFS-10838. Last full block report received time for each DN should be easily discoverable. Contributed by Surendra Singh Lilhore. 2017-03-06 16:40:00 -08:00
Yiqun Lin
5b49a96328 HDFS-8741. Proper error msg to be printed when invalid operation type is given to WebHDFS operations. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit 3536ce031ca780d6de83cf67779f571a0142ccc8)
 2017-03-06 19:05:43 +08:00
Wei-Chiu Chuang
f3cdf29af4 HDFS-8498. Blocks can be committed with wrong size. Contributed by Jing Zhao. 2017-02-25 21:13:51 -08:00
Xiaoyu Yao
d72f1c5976 HDFS-11409. DatanodeInfo getNetworkLocation and setNetworkLocation shoud use volatile instead of synchronized. Contributed by Chen Liang. 
(cherry picked from commit aaf27132350547fcde1fdb372f19626838f44bc4)
 2017-02-14 12:55:49 -08:00
Kihwal Lee
33c62d2d19 HDFS-11379. DFSInputStream may infinite loop requesting block locations. Contributed by Daryn Sharp. 2017-02-10 12:52:14 -06:00
Xiao Chen
b0a1703f10 HDFS-11363. Need more diagnosis info when seeing Slow waitForAckedSeqno. 
(cherry picked from commit 0914fcca312b5e9d20bcf1b6633bc13c9034ba46)
 2017-02-02 10:09:20 -08:00
Arpit Agarwal
b4078e1d0e HDFS-11194. Maintain aggregated peer performance metrics on NameNode. 2017-01-24 19:34:33 -08:00
Ming Ma
bed700e98f HDFS-11296. Maintenance state expiry should be an epoch time and not jvm monotonic. (Manoj Govindassamy via mingma) 
(cherry picked from commit f3fb94be05a61a4c4c06ab279897e5de2b181b0e)
 2017-01-19 22:33:43 -08:00
Arpit Agarwal
954dae26cd HDFS-11337. (HDFS-10958 backport). Add instrumentation hooks around Datanode disk IO. 2017-01-16 13:53:30 -08:00
Andrew Wang
b089e122b1 HDFS-11312. Fix incompatible tag number change for nonDfsUsed in DatanodeInfoProto. Contributed by Sean Mackrory. 
(cherry picked from commit d51f8ba80816c20cd97ba5aa839a84b0bb6a7291)
 2017-01-11 15:13:41 -08:00
Haohui Mai
20486730e8 HDFS-11280. Allow WebHDFS to reuse HTTP connections to NN. Contributed by Zheng Shao. 2017-01-04 21:03:05 -08:00
Andrew Wang
d4ca1c5226 HDFS-11156. Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang. 2017-01-04 15:10:15 -08:00
Andrew Wang
2b5b62f9cc Revert "HDFS-11156. Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang." 
This reverts commit 80e664084894bbff9b1cc3102e71b161873fb6ef.
 2017-01-03 11:12:53 -08:00
Andrew Wang
80e6640848 HDFS-11156. Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang. 
(cherry picked from commit 7fcc73fc0d248aae1edbd4e1514c5818f6198928)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/WebHDFS.md
 2017-01-03 09:58:45 -08:00
Brahma Reddy Battula
029b6fbe71 Revert "HDFS-11280. Allow WebHDFS to reuse HTTP connections to NN. Contributed by Zheng Shao." 
This reverts commit 1bc9b316ba60b1ecc8d6fca8d78642933d2c186b.
 2017-01-03 19:26:44 +05:30
Haohui Mai
1bc9b316ba HDFS-11280. Allow WebHDFS to reuse HTTP connections to NN. Contributed by Zheng Shao. 2016-12-30 22:59:24 -08:00
Andrew Wang
6b795c34d0 HDFS-10684. WebHDFS DataNode calls fail without parameter createparent. Contributed by John Zhuge. 
(cherry picked from commit fbdbbd57cdc3d8c778fca9266a7cadf298c8ff6c)
 2016-12-13 18:01:35 -08:00
Vinayakumar B
c6fb9f0d0b HDFS-10570. Remove classpath conflicts of netty-all jar in hadoop-hdfs-client (Contributed by Vinayakumar B) 2016-12-12 12:11:11 +05:30
Andrew Wang
215abbdf5f HDFS-8630. WebHDFS : Support get/set/unset StoragePolicy. Contributed by Surendra Singh Lilhore. 2016-12-09 14:29:22 -08:00
Vinayakumar B
4b21b47f56 HDFS-9057. allow/disallow snapshots via webhdfs (Contributed by Brahma Reddy Battula) 
(cherry picked from commit 6d2332ae375e26d024358c6e75fdb3c68a781a66)
 2016-12-07 15:58:15 -08:00