Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,137 Commits 326 Branches 371 Tags
Commit Graph

4709 Commits

Author SHA1 Message Date
Arpit Agarwal
09290659e7 HDFS-12530. Processor argument in Offline Image Viewer should be case insensitive. Contributed by Hanisha Koneru. 2017-09-22 09:48:10 -07:00
Arpit Agarwal
18e5f2068b HDFS-12496. Make QuorumJournalManager timeout properties configurable. Contributed by Ajay Kumar. 2017-09-21 08:53:27 -07:00
Kihwal Lee
9640bfb808 HDFS-12371. BlockVerificationFailures and BlocksVerified show up as 0 in Datanode JMX. Contributed by Hanisha Koneru. 
(cherry picked from commit 6bf921a5c3152a307b5c0903056d73ce07775a08)
 2017-09-21 08:48:04 -05:00
Akira Ajisaka
3527327f34
HDFS-12526. FSDirectory should use Time.monotonicNow for durations. Contributed by Bharat Viswanadham. 
(cherry picked from commit 50849ec9eb9bad90586a95d2b2380ee6e8724d6b)
 2017-09-21 19:28:49 +09:00
Ming Ma
3bb23f4be9 HDFS-11035. Better documentation for maintenace mode and upgrade domain. 
(cherry picked from commit ce943eb17a4218d8ac1f5293c6726122371d8442)
 2017-09-20 09:42:22 -07:00
Ming Ma
7dd662eafd HDFS-12473. Change hosts JSON file format. 2017-09-20 09:08:41 -07:00
Brahma Reddy Battula
eddc581860 HDFS-11799. Introduce a config to allow setting up write pipeline with fewer nodes than replication factor. Contributed by Brahma Reddy Battula 
(cherry picked from commit fda1221c55101d97ac62e1ee4e3ddf9a915d5363)
 2017-09-19 11:31:22 +05:30
Arpit Agarwal
f42705b957 HDFS-12472. Add JUNIT timeout to TestBlockStatsMXBean. Contributed by Bharat Viswanadham. 2017-09-16 10:14:39 -07:00
Erik Krogen
791e20c244 HDFS-12323. NameNode terminates after full GC thinking QJM unresponsive if full GC is much longer than timeout. Contributed by Erik Krogen. 
(cherry picked from commit 90894c7262df0243e795b675f3ac9f7b322ccd11)
 2017-09-15 14:03:10 -07:00
Nathan Roberts
bd14e8d8b0 HDFS-12441. Supress UnresolvedPathException in namenode log. Contributed by Kihwal Lee. 
(cherry picked from commit de197fc1562c4d1c39d24738c7cf8a8869dcec9c)
 2017-09-15 14:47:07 -05:00
Kihwal Lee
07dee7fd63 HDFS-12424. Datatable sorting on the Datanode Information page in the Namenode UI is broken. Contributed by Shawna Martell. 
(cherry picked from commit 78bdf10ae40300a5f2f3b95491d280802edc1df2)
 2017-09-15 09:55:59 -05:00
John Zhuge
eaef8446f6 HADOOP-14864. FSDataInputStream#unbuffer UOE should include stream class name. Contributed by Bharat Viswanadham. 
(cherry picked from commit 7ee02d1065268511ec10a8b8852d0d6327419b9c)
 2017-09-14 11:29:34 -07:00
Arpit Agarwal
ecf352d645 HADOOP-14867. Update HDFS Federation setup document, for incorrect property name for secondary name node http address. Contributed by Bharat Viswanadham. 2017-09-13 12:54:21 -07:00
Xiao Chen
0a8ed025e4 Revert "HADOOP-14521. KMS client needs retry logic. Contributed by Rushabh S Shah." 
This reverts commit 5337c2511df5f75d950b713740c4d61bffc6034d.
 2017-09-12 19:53:14 -07:00
Arpit Agarwal
ac20d7e246 HDFS-12407. Journal node fails to shutdown cleanly if JournalNodeHttpServer or JournalNodeRpcServer fails to start. Contributed by Ajay Kumar. 2017-09-12 16:18:56 -07:00
Manoj Govindassamy
796404c3ed HDFS-12404. Rename hdfs config authorization.provider.bypass.users to attributes.provider.bypass.users. 
(cherry picked from commit 3b3be355b35d08a78d9dcd647650812a2d28207b)
 2017-09-07 17:21:33 -07:00
Xiao Chen
c6a1e3facb HDFS-12369. Edit log corruption due to hard lease recovery of not-closed file which has snapshots. 
(cherry picked from commit 3964b131f3f0131a663377685950d1b7bd3fe63d)
(cherry picked from commit 8b7d597d3a8d728aa05302ccd769af84a1c9ec0f)
 2017-09-07 16:30:38 -07:00
Yongjun Zhang
32e8c84614 HDFS-12357. Let NameNode to bypass external attribute provider for configured users. Contributed by Yongjun Zhang, Arun Suresh. 
(cherry picked from commit d77ed238a911fc85d6f4bbce606cac7ec44f557f)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java
 2017-09-07 10:22:37 -07:00
Wei-Chiu Chuang
3a2f3e78ff HDFS-12182. BlockManager.metaSave does not distinguish between "under replicated" and "missing" blocks. Contributed by Wellington Chevreuil. 2017-09-05 14:58:08 -07:00
Xiao Chen
858715d054 HDFS-12336. Listing encryption zones still fails when deleted EZ is not a direct child of snapshottable directory. Contributed by Wellington Chevreuil. 2017-08-30 09:43:12 -07:00
Yongjun Zhang
870ef0b2fd HDFS-12191. Provide option to not capture the accessTime change of a file to snapshot if no other modification has been done to this file. Contributed by Yongjun Zhang. 
(cherry picked from commit cf93d60d3f032000e5b78a08d320793d78799f3d)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSDirAttrOp.java
 2017-08-29 21:57:25 -07:00
Arpit Agarwal
e19f004f65 HDFS-11998. Enable DFSNetworkTopology as default. Contributed by Chen Liang. 2017-08-28 16:22:42 -07:00
Arpit Agarwal
98ba68b05d HDFS-12293. DataNode should log file name on disk error. Contributed by Ajay Kumar. 2017-08-28 15:50:06 -07:00
Yiqun Lin
d63e98ad5f HDFS-11530. Use HDFS specific network topology to choose datanode in BlockPlacementPolicyDefault. Contributed by Chen Liang. 2017-08-28 20:47:38 +08:00
Lei Xu
c22b5108df HDFS-12215. DataNode.transferBlock does not create its daemon in the xceiver thread group. (Lei Xu) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
 2017-08-25 16:06:34 -07:00
Kihwal Lee
5a83ffa396 HDFS-12299. Race Between update pipeline and DN Re-Registration. Contributed by Brahma Reddy Battula. 2017-08-25 14:53:53 -05:00
Arpit Agarwal
2cae387402 HDFS-12319. DirectoryScanner will throw IllegalStateException when Multiple BP's are present. Contributed by Brahma Reddy Battula. 2017-08-25 10:41:59 -07:00
Arpit Agarwal
ae69c99068 HDFS-11577. Combine the old and the new chooseRandom for better performance. Contributed by Chen Liang. 2017-08-24 11:17:36 -07:00
Arpit Agarwal
8b2d5a4faa HDFS-11514. DFSTopologyNodeImpl#chooseRandom optimizations. Contributed by Chen Liang. 2017-08-23 12:49:13 -07:00
John Zhuge
c54c3500ed HDFS-11738. Hedged pread takes more time when block moved from initial locations. Contributed by Vinayakumar B. 
(cherry picked from commit b6bfb2fcb2391d51b8de97c01c1290880779132e)
 2017-08-23 19:44:52 +05:30
Arpit Agarwal
b3ea11dfdb HDFS-11482. Add storage type demand to into DFSNetworkTopology#chooseRandom. Contributed by Chen Liang. 2017-08-21 14:07:59 -07:00
Manoj Govindassamy
89fc7fe670 HDFS-11988. Verify HDFS Snapshots with open files captured are consistent across truncates and appends to current version file. 2017-08-21 11:14:44 -07:00
John Zhuge
76fbeed997 HDFS-12117. HttpFS does not seem to support SNAPSHOT related methods for WebHDFS REST Interface. Contributed by Wellington Chevreuil. 2017-08-18 22:06:02 -07:00
Arpit Agarwal
d8d2a221f8 HDFS-11450. HDFS specific network topology classes with storage type info included. Contributed by Chen Liang. 2017-08-18 11:43:17 -07:00
Manoj Govindassamy
a5bc30170c HDFS-12316. Verify HDFS snapshot deletion doesn't crash the ongoing file writes. 
(cherry picked from commit 4230872dd66d748172903b1522885b03f34bbf9b)
 2017-08-17 16:25:15 -07:00
Mingliang Liu
03d15fa58e HDFS-11430. Separate class InnerNode from class NetworkTopology and make it extendable. Contributed by Tsz Wo Nicholas Sze 2017-08-16 16:06:13 -07:00
Ravi Prakash
895a35eac0 HDFS-12301. NN File Browser UI: Navigate to a path when enter is pressed 
(cherry picked from commit f34646d652310442cb5339aabbbb269f10dfa838)
 2017-08-15 15:45:37 -07:00
John Zhuge
ed4d6aa2c1 HDFS-11303. Hedged read might hang infinitely if read data from all DN failed . Contributed by Chen Zhang, Wei-chiu Chuang, and John Zhuge. 
(cherry picked from commit 8b242f09a61a7536d2422546bfa6c2aaf1d57ed6)
 2017-08-11 21:55:23 -07:00
Kihwal Lee
306abf4980 HDFS-12278. LeaseManager operations are inefficient in 2.8. Contributed by Rushabh S Shah. 
(cherry picked from commit b5c02f95b5a2fcb8931d4a86f8192caa18009ea9)
 2017-08-09 16:48:55 -05:00
Kihwal Lee
74bcc8d1f2 HDFS-12157. Do fsyncDirectory(..) outside of FSDataset lock. Contributed by Vinayakumar B. 2017-08-09 09:31:53 -05:00
Haohui Mai
3e2753daa1 HADOOP-14598. Blacklist Http/HttpsFileSystem in FsUrlStreamHandlerFactory. Contributed by Steve Loughran. 2017-08-08 16:33:54 -07:00
Haohui Mai
21d76f710e HDFS-10326. Disable setting tcp socket send/receive buffers for write pipelines. Contributed by Daryn Sharp. 2017-08-08 14:58:44 -07:00
Andrew Wang
2da20df1dd HDFS-12131. Add some of the FSNamesystem JMX values as metrics. Contributed by Erik Krogen. 2017-08-03 15:45:21 -07:00
Manoj Govindassamy
065a906be7 HDFS-9388. Decommission related code to support Maintenance State for datanodes. 2017-08-02 16:58:54 -07:00
Arpit Agarwal
45cde760b0 HDFS-12043. Add counters for block re-replication. Contributed by Chen Liang. 2017-08-02 16:45:08 -07:00
Manoj Govindassamy
a77ae7abdb HDFS-12217. HDFS snapshots doesn't capture all open files when one of the open files is deleted. 
(cherry picked from commit 52d7bafcf49916887197436ddb0f08f021d248d9)
 2017-08-02 10:16:30 -07:00
Jason Lowe
f34c01b92a Revert "HDFS-12217. HDFS snapshots doesn't capture all open files when one of the open files is deleted." 
This reverts commit b0330c15da8483302007d1df797f4398b7129cdb.
 2017-08-02 11:15:58 -05:00
Manoj Govindassamy
b0330c15da HDFS-12217. HDFS snapshots doesn't capture all open files when one of the open files is deleted. 
(cherry picked from commit 52d7bafcf49916887197436ddb0f08f021d248d9)
 2017-08-01 17:15:29 -07:00
Manoj Govindassamy
70c511598f HDFS-11402. HDFS Snapshots should capture point-in-time copies of OPEN files. 2017-08-01 17:11:27 -07:00
Akira Ajisaka
72bde1dad5
HADOOP-14245. Use Mockito.when instead of Mockito.stub. Contributed by Andras Bokor. 
(cherry picked from commit b38a1eea8e2917989d83d169a7b5773163e6832e)
 2017-08-01 15:16:21 +09:00