Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,552 Commits 326 Branches 371 Tags
Commit Graph

2838 Commits

Author SHA1 Message Date
Konstantin V Shvachko
6d657054eb Update CHANGES.txt for HDFS-11377. 2017-05-23 15:02:30 -07:00
Brahma Reddy Battula
7ae1bc3c20 HDFS-11864. Document Metrics to track usage of memory for writes. Contributed by Yiqun Lin. 
(cherry picked from commit 52661e0912a79d1e851afc2b46c941ce952ca63f)
 2017-05-24 00:04:25 +08:00
Brahma Reddy Battula
0b53e9f9af Update CHANGES.txt for HDFS-11849 2017-05-23 23:31:03 +08:00
Akira Ajisaka
8a0e5e3b70 HDFS-10715. NPE when applying AvailableSpaceBlockPlacementPolicy. Contributed by Guangbin Zhu. 
(cherry picked from commit ef432579a7763cc0e482fe049027c6e5325eb034)
(cherry picked from commit 68638661271e6e69c220b19fec7cbc7ae5e6208e)

# Conflicts:
#	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/AvailableSpaceBlockPlacementPolicy.java
 2017-05-21 17:47:51 -07:00
Kihwal Lee
9440ec2771 HDFS-8131. Implement a space balanced block placement policy. Contributed by Liu Shaohui. 
(cherry picked from commit de30d66b2673d0344346fb985e786247ca682317)

# Conflicts:
#	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
 2017-05-21 17:19:54 -07:00
Konstantin V Shvachko
321f730ca8 HDFS-9412. getBlocks occupies FSLock and takes too long to complete. Contributed by He Tianyi. Backport HDFS-11855 by Vinitha Reddy Gankidi. 
(cherry picked from commit 67523ffcf491f4f2db5335899c00a174d0caaa9b)
(cherry picked from commit a6d0e9e56a7eef226c8a891cf741ab22d8205e1c)
 2017-05-21 13:56:44 -07:00
Konstantin V Shvachko
eecc68d4d8 HDFS-9726. Refactor IBR code to a new class. Contributed by Tsz-Wo Nicholas Sze. Backport HDFS-11839 by Vinitha Reddy Gankidi. 
(cherry picked from commit 494b6c7c4b5e0a861fcbdcfb89cfc4869247e710)
 2017-05-18 18:09:36 -07:00
Konstantin V Shvachko
2c3f6aedf1 HDFS-7990. IBR delete ack should not be delayed. Contributed by Daryn Sharp. Backport HDFS-11838 by Vinitha Gankidi. 
(cherry picked from commit 60882ab26d49f05cbf0686944af6559f86b3417d)
 2017-05-17 23:58:27 -07:00
Konstantin V Shvachko
0823fb72a9 HDFS-8498. Blocks can be committed with wrong size. Contributed by Jing Zhao. Backport HDFS-11732 by Zhe Zhang. 2017-05-16 11:03:12 -07:00
Brahma Reddy Battula
3004358639 HDFS-8312. Trash does not descent into child directories to check for permissions. Contributed By Weiwei Yang via Eric Yang. 2017-05-16 23:27:57 +05:30
Brahma Reddy Battula
b95d63cdac HDFS-10987. Make Decommission less expensive when lot of blocks present. Contributed By Brahma Reddy Battula. 2017-05-16 10:50:51 +05:30
Vinayakumar B
b0b6de38c9 HDFS-11674. reserveSpaceForReplicas is not released if append request failed due to mirror down and replica recovered (Contributed by Vinayakumar B) 2017-05-12 14:26:28 +05:30
Brahma Reddy Battula
764fd3ed49 HDFS-9902. Support different values of dfs.datanode.du.reserved per storage type. Contributed By Brahma Reddy Battula. 2017-05-11 10:22:59 +05:30
Akira Ajisaka
f052976ff1
HDFS-11795. Fix ASF License warnings in branch-2.7. Contributed by Yiqun Lin. 2017-05-10 14:17:00 -05:00
Brahma Reddy Battula
48c5e37abf HDFS-11373. Backport HDFS-11258 and HDFS-11272(File mtime change could not save to editlog). (Contributed by Jimmy Xiang, backported by Akira Ajisaka) 2017-05-10 11:06:57 +05:30
Akira Ajisaka
6599e57a79
HDFS-11766. Fix findbugs warning in branch-2.7. Contributed by Chen Liang. 2017-05-09 11:22:57 -05:00
Konstantin V Shvachko
c7c410898e Update CHANGES.txt for HDFS-11717 branch-2.7. 2017-05-02 22:30:18 -07:00
Kihwal Lee
130a0e2156 HDFS-11714. Newly added NN storage directory won't get initialized and cause space exhaustion. Contributed by Kihwal Lee. 
Added a CHANGES.txt entry
(cherry picked from commit f77d138725662197ca0be874e9ec4dc430db2a4f)
 2017-05-01 17:36:58 -05:00
Kihwal Lee
139327d34b HDFS-11609. Some blocks can be permanently lost if nodes are decommissioned while dead. Contributed by Kihwal Lee. 2017-05-01 14:31:59 -05:00
Zhe Zhang
8ad9efbe14 HDFS-8873. Allow the directoryScanner to be rate-limited. Contributed by Daniel Templeton. 2017-04-27 15:04:10 -07:00
Konstantin V Shvachko
5e67eb7560 Update CHANGES.txt for HDFS-11384 branch-2.7. 2017-04-27 12:23:41 -07:00
Zhe Zhang
bec23593b5 HDFS-11709. StandbyCheckpointer should handle an non-existing legacyOivImageDir gracefully. Contributed by Erik Krogen. 
(cherry picked from commit d8a33098309f17dfb0e3a000934f68394de44bf7)
(cherry picked from commit fc46f398d9192e83c7197115589eca7805c49943)
(cherry picked from commit 70f18911c71c61f9b3918e195bfb9a99d22d230c)
 2017-04-27 09:56:14 -07:00
Brahma Reddy Battula
ac12063eeb HDFS-10455. Logging the username when deny the setOwner operation. Contributed by Rakesh R. 
(cherry picked from commit 1c5708f6ded6ba9fc007eed93c1f65ba44451241)
(cherry picked from commit c895f0893621fd6196c42ed46b97cb99cc2e9bee)
 2017-04-26 15:59:21 -07:00
Konstantin V Shvachko
54bbdb4c02 Update CHANGES.txt for HDFS-11634. 2017-04-18 13:20:21 -07:00
Zhe Zhang
ccd24f0917 HDFS-11615. FSNamesystemLock metrics can be inaccurate due to millisecond precision. Contributed by Erik Krogen. 
(cherry picked from commit ad49098eb324e238d97db68d7239ed2c4d84afa0)
(cherry picked from commit 051ab778065d3659c5284b5d95d04156c4ee7ffa)
(cherry picked from commit 07047335a3bc12b719ca608742bef1e565151035)
 2017-04-17 16:55:24 -07:00
Masatake Iwasaki
0ece3c8dd9 HDFS-11486. Client close() should not fail fast if the last block is being decommissioned. Contributed by Wei-Chiu Chuang and Yiqun Lin. 
(cherry picked from commit ca3ede5fda60f317c24be8a378e7450ae581f0c5)
 2017-03-28 18:17:45 +09:00
Wei-Chiu Chuang
830a602375 HDFS-11499. Decommissioning stuck because of failing recovery. Contributed by Lukas Majercak and Manoj Govindassamy. 
(cherry picked from commit 385d2cb777a0272ac20c62336c944fad295d5d12)

 Conflicts:
    hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java

(cherry picked from commit 60be2e5d8a1a6a8921c68f8b0f428b55152d05db)

 Conflicts:
    hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
 2017-03-13 13:45:12 -07:00
Zhe Zhang
33f4263345 HDFS-11466. Change dfs.namenode.write-lock-reporting-threshold-ms default from 1000ms to 5000ms. Contributed by Andrew Wang. 
(cherry picked from commit d269b488a71a158d3ddcbdea96992abe29725c69)
(cherry picked from commit 5fb785e154b429beb0b2e392c39be91dae65bb5c)
(cherry picked from commit 265ddb20c3aa99c2d91e2a85e82106d4960f5a7b)
 2017-02-28 09:07:23 -08:00
Xiao Chen
c2a4ce7717 HDFS-11379. DFSInputStream may infinite loop requesting block locations. Contributed by Daryn Sharp. 2017-02-17 15:38:55 -08:00
Wei-Chiu Chuang
f133e0161c HDFS-11333. Print a user friendly error message when plugins are not found. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 859bd159ae554174200334b5eb1d7e8dbef958ad)
(cherry picked from commit a8531d5d52fb219c2adb5b6025cfc3fad264e8aa)
(cherry picked from commit 4c47cb68e88505d5fdc6830a4789e78a974cb9cf)
 2017-02-15 03:05:01 -08:00
Zhe Zhang
03a67e1e62 HDFS-10534. NameNode WebUI should display DataNode usage histogram. Contributed by Kai Sasaki. 2017-01-26 10:34:00 -08:00
Akira Ajisaka
ab62484240 HDFS-11352. Add missing entry in CHANGES.txt. 2017-01-24 14:57:21 +09:00
Akira Ajisaka
b0b3966851 HDFS-11290. TestFSNameSystemMBean should wait until JMX cache is cleared. Contributed by Erik Krogen. 
(cherry picked from commit b1a9ec856b572894e769f052aea2340fc3f23c78)
(cherry picked from commit db47bd15dd80a4a5727607832c16635cb0ec7726)
 2017-01-24 14:19:21 +09:00
Brahma Reddy Battula
3e0bd9951f HDFS-11263. ClassCastException when we use Bzipcodec for Fsimage compression. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 1b401f6a734df4e23a79b3bd89c816a1fc0de574)
 2016-12-20 21:30:18 +05:30
Wei-Chiu Chuang
bbf380a493 HDFS-11160. VolumeScanner reports write-in-progress replicas as corrupt incorrectly. Contributed by Wei-Chiu Chuang and Yongjun Zhang. 
(cherry picked from commit 0cb99db9d91d113e7fbe229f90a61a33433cecb9)

Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockScanner.java

(cherry picked from commit 18be0447cde622dfaaad27f7c2b9cccb30469fef)

Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/FsVolumeSpi.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsVolumeImpl.java
 2016-12-16 13:40:20 -08:00
Wei-Chiu Chuang
2123b664ca HDFS-11229. HDFS-11056 failed to close meta file. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 292bd78b4494ba1a1073b32b3f89d95e72f3204d)
(cherry picked from commit fa3b2bee618c72bae57890f011b475522fe2ff47)
 2016-12-10 09:34:05 -08:00
Akira Ajisaka
6d8df4e81e HDFS-11180. Intermittent deadlock in NameNode when failover happens. 2016-12-03 11:11:50 +09:00
Akira Ajisaka
c594904eca HDFS-11174. Wrong HttpFS test command in doc. Contributed by John Zhuge. 
(cherry picked from commit bb1b5e083b6f32646412b292d061ce5721f5dbe1)
(cherry picked from commit 6fe5ffd05d41727552834e54695d69a26eddd8d8)
 2016-11-25 21:31:11 +09:00
Kihwal Lee
01ad9dee08 HDFS-10966. Enhance Dispatcher logic on deciding when to give up a source DataNode. Contributed by Mark Wagner and Zhe Zhang. 2016-11-21 16:58:12 -06:00
Konstantin V Shvachko
f6e517c6cc Updated CHANGES.txt for HDFS-11087. 2016-11-15 11:10:44 -08:00
Zhe Zhang
ec0f854ba0 Add CHANGES.txt entry for HDFS-10872. 2016-11-14 23:14:35 -08:00
Wei-Chiu Chuang
81bf6f2834 HDFS-11056. Concurrent append and read operations lead to checksum error. Contributed by Wei-Chiu Chuang. 2016-11-14 11:57:51 -08:00
Anu Engineer
95edf265b3 HDFS-8307. Spurious DNS Queries from hdfs shell. Contributed by Andres Perez 2016-11-10 14:38:53 -08:00
Zhe Zhang
49f059125a Addendum of HDFS-8709: update CHANGES.TXT. 2016-10-31 15:17:21 -07:00
Andrew Wang
85708329c8 HDFS-8709. Clarify automatic sync in FSEditLog#logEdit. 
(cherry picked from commit 5fddc5177ddad07a735d49c15a63cfc5f74d0891)
(cherry picked from commit efe9ae9fc68e4eec4b7388b971c520328e95fe76)
(cherry picked from commit 3b8cc7c21615d07e62251fd1b9c80f696f590a8f)
 2016-10-31 15:08:52 -07:00
Erik Krogen
d002e4d10b HDFS-9500. Fix software version counts for DataNodes during rolling upgrade. Contributed by Erik Krogen. 
(cherry picked from commit f3ac1f41b8fa82a0ac87a207d7afa2061d90a9bd)
 2016-10-27 16:10:26 -07:00
Kihwal Lee
a49510f697 HDFS-11069. Tighten the authorization of datanode RPC. Contributed by Kihwal Lee 
Updated CHANGES.txt
(cherry picked from commit ae48c496dce8d0eae4571fc64e6850d602bae688)
 2016-10-27 14:23:20 -05:00
Kihwal Lee
fdd645be6b HDFS-11053. Unnecessary superuser check in versionRequest(). Contributed by Kihwal Lee. 
Updated CHANGES.txt
(cherry picked from commit e90af4a89ba2741aeca20ead47699587b14f0de8)
 2016-10-26 09:19:45 -05:00
Zhe Zhang
c05d701161 HDFS-11015. Enforce timeout in balancer. Contributed by Kihwal Lee. 
(cherry picked from commit f6367c5f44a88cb5eb7edffb015b10b657504a61)
(cherry picked from commit ff806cbfc7f5d1bdccd51a1802b6d69f4777219e)
(cherry picked from commit 262518fa5b2f0e06c9d9158ec8b37a2ac3cc6981)
 2016-10-25 11:50:51 -07:00
Kihwal Lee
6ae7b9f800 HDFS-10627. Volume Scanner marks a block as "suspect" even if the exception is network-related. Contributed by Rushabh S Shah. 
Updated CHANGES.txt
(cherry picked from commit 5c0bffddc0cb824a8a2751bcd0dc3e15ce081727)
 2016-10-20 11:09:20 -05:00