Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,594 Commits 326 Branches 371 Tags
Commit Graph

3288 Commits

Author SHA1 Message Date
Xiao Chen
187bb77ddb HDFS-11741. Long running balancer may fail due to expired DataEncryptionKey. Contributed by Wei-Chiu Chuang and Xiao Chen. 
(cherry picked from commit 068e23b896c63b0f817e6b91d73c994be1551eb2)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/security/token/block/BlockTokenSecretManager.java
(cherry picked from commit 6f2391e0ad6fbaa6786cc9f0e3a09b955e0cb21c)

(cherry picked from commit ec10a81bdbc4f5799ae40cd9fe5f6109dfbfa9ae)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/security/token/block/BlockTokenSecretManager.java
 2017-07-11 16:59:25 -07:00
Konstantin V Shvachko
53907b66cb HDFS-9503. Use generic option -fs for NNThroughputBenchmark instead of -namenode. Contributed by Mingliang Liu. 
(cherry picked from commit 81ef1a1f55ee0104f267316e44596463a78f87ea)
 2017-07-03 12:23:00 -07:00
Xiaoyu Yao
2af5548212 HDFS-9421. NNThroughputBenchmark replication test NPE with -namenode option. Contributed by Mingliang Liu. 
(cherry picked from commit a4f62a2d58d4e00cda3632411c6abda6eaa12a0e)
 2017-07-03 11:42:59 -07:00
Colin Patrick Mccabe
51f012aec4 HDFS-7847. Modify NNThroughputBenchmark to be able to operate on a remote NameNode (Charles Lamb via Colin P. McCabe) 
(cherry picked from commit ffce9a3413277a69444fcb890460c885de56db69)
 2017-07-03 11:32:45 -07:00
Akira Ajisaka
f36da00c1f
HDFS-11583. Parent spans are not initialized to NullScope for every DFSPacket. Contributed by Masatake Iwasaki. 2017-06-15 14:15:36 +09:00
Akira Ajisaka
59fab4ea04
HDFS-11736. OIV tests should not write outside 'target' directory. Contributed by Yiqun Lin. 2017-06-15 13:49:53 +09:00
Zhe Zhang
a91a1031e3 HDFS-11743. Revert the incompatible fsck reporting output in HDFS-7933 from branch-2.7. 2017-06-07 08:59:28 -07:00
Vinayakumar B
d2ec34d556 HDFS-11708. Positional read will fail if replicas moved to different DNs after stream is opened (Contributed by Vinayakumar B) 2017-06-07 10:53:01 +05:30
Andrew Wang
db81fbbab1 HDFS-8549. Abort the balancer if an upgrade is in progress. Contributed by Andrew Wang. Backport HDFS-11808 by Akira Ajisaka. 
(cherry picked from commit a7a7768341f1b7d3a8f2686e2f4d00c57f2e1d4f)
 2017-06-05 17:27:52 -07:00
Kihwal Lee
d3b86234b2 HDFS-11856. Ability to re-add upgrading nodes to pipeline for future pipeline updates. Contributed by Vinayakumar B. 2017-06-01 16:45:55 -05:00
Kihwal Lee
01cdea732f HDFS-5042. Completed files lost after power failure. Contributed by Vinayakumar B. 2017-06-01 12:41:00 -05:00
Brahma Reddy Battula
38bc12c683 Updating CHANGES.txt for HDFS-11078 2017-06-02 00:21:12 +08:00
Brahma Reddy Battula
2b19c7b3ff HDFS-11893. Fix TestDFSShell.testMoveWithTargetPortEmpty failure. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 0dcf843c008f2b9cece8c0a0cef78140398ac464)
 2017-06-01 22:53:06 +08:00
Kihwal Lee
1903665b23 HDFS-11648. Lazy construct the IIP pathname. Contributed by Daryn Sharp. 
(cherry picked from commit 8ed230c805625549b1cecc830e909a7027bb4961)
(cherry picked from commit 9dfe0b35158cc0046a82cbfea2218a4ad61f329e)
 2017-05-31 08:50:26 -07:00
Kihwal Lee
81712e84fb HDFS-10619. Cache path in InodesInPath. Contributed by Daryn Sharp. 
(cherry picked from commit 90020624b05230ad4a7fbd666d0177ecb107a4d6)
(cherry picked from commit 3a89a8814f8e443a5f7ceddcecd6d953cfc2c6f3)
 2017-05-31 08:50:26 -07:00
Kihwal Lee
cafee1109c HDFS-8674. Improve performance of postponed block scans. Contributed by Daryn Sharp. 
(cherry picked from commit 0d8a35bd6de5d2a5a9b816ca98f31975e94bd7c6)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
 2017-05-26 19:00:03 -07:00
Inigo Goiri
948d0ac72d Fix NPE in LazyPersistFileScrubber. Contributed by Inigo Goiri. 
(cherry picked from commit 303c8dc9b6c853c0939ea9ba14388897cc258071)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
 2017-05-26 13:36:34 -07:00
Brahma Reddy Battula
724a5f3dbc HDFS-11445. FSCK shows overall health stauts as corrupt even one replica is corrupt. Contributed by Brahma Reddy Battula. 2017-05-25 23:00:35 +08:00
Steve Loughran
f225b5514f HADOOP-13026 Should not wrap IOExceptions into a AuthenticationException in KerberosAuthenticator. Xuan Gong via stevel 2017-05-24 19:00:18 -07:00
Vinayakumar B
b6afb4a951 HDFS-6291. FSImage may be left unclosed in BootstrapStandby#doRun() ( Contributed by Sanghyun Yun) 
(cherry picked from commit e5e492a9631ff78302fccedcb64d7b64b9407991)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2017-05-24 18:47:31 -07:00
Konstantin V Shvachko
2b067895a1 Commit missing file TestBatchIbr for HDFS-9710 backport. 2017-05-24 12:21:48 -07:00
Konstantin V Shvachko
ac1b8ff78f HDFS-9710. DN can be configured to send block receipt IBRs in batches. Contributed by Tsz-Wo Nicholas Sze. 
Backport HDFS-11837 by Vinitha Reddy Gankidi

(cherry picked from commit 5b95971f8a6dee09d1143c6cf121afa22fa6c16e)
 2017-05-24 11:42:52 -07:00
Konstantin V Shvachko
6d657054eb Update CHANGES.txt for HDFS-11377. 2017-05-23 15:02:30 -07:00
Brahma Reddy Battula
7ae1bc3c20 HDFS-11864. Document Metrics to track usage of memory for writes. Contributed by Yiqun Lin. 
(cherry picked from commit 52661e0912a79d1e851afc2b46c941ce952ca63f)
 2017-05-24 00:04:25 +08:00
Brahma Reddy Battula
0b53e9f9af Update CHANGES.txt for HDFS-11849 2017-05-23 23:31:03 +08:00
Yiqun Lin
f46781261f HDFS-11377. Balancer hung due to no available mover threads. Contributed by yunjiong zhao. 
(cherry picked from commit 9cbbd1eae893b21212c9bc9e6745c6859317a667)
 2017-05-22 17:44:12 -07:00
Brahma Reddy Battula
cecedcf558 HDFS-11849. JournalNode startup failure exception should be logged in log file. Contributed By Surendra Singh Lilhore. 
(cherry picked from commit 9cab42cc797986081fef184748044f1790a4f039)
 2017-05-23 02:52:41 +08:00
Akira Ajisaka
8a0e5e3b70 HDFS-10715. NPE when applying AvailableSpaceBlockPlacementPolicy. Contributed by Guangbin Zhu. 
(cherry picked from commit ef432579a7763cc0e482fe049027c6e5325eb034)
(cherry picked from commit 68638661271e6e69c220b19fec7cbc7ae5e6208e)

# Conflicts:
#	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/AvailableSpaceBlockPlacementPolicy.java
 2017-05-21 17:47:51 -07:00
Kihwal Lee
9440ec2771 HDFS-8131. Implement a space balanced block placement policy. Contributed by Liu Shaohui. 
(cherry picked from commit de30d66b2673d0344346fb985e786247ca682317)

# Conflicts:
#	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
 2017-05-21 17:19:54 -07:00
Konstantin V Shvachko
321f730ca8 HDFS-9412. getBlocks occupies FSLock and takes too long to complete. Contributed by He Tianyi. Backport HDFS-11855 by Vinitha Reddy Gankidi. 
(cherry picked from commit 67523ffcf491f4f2db5335899c00a174d0caaa9b)
(cherry picked from commit a6d0e9e56a7eef226c8a891cf741ab22d8205e1c)
 2017-05-21 13:56:44 -07:00
Konstantin V Shvachko
eecc68d4d8 HDFS-9726. Refactor IBR code to a new class. Contributed by Tsz-Wo Nicholas Sze. Backport HDFS-11839 by Vinitha Reddy Gankidi. 
(cherry picked from commit 494b6c7c4b5e0a861fcbdcfb89cfc4869247e710)
 2017-05-18 18:09:36 -07:00
Konstantin V Shvachko
2c3f6aedf1 HDFS-7990. IBR delete ack should not be delayed. Contributed by Daryn Sharp. Backport HDFS-11838 by Vinitha Gankidi. 
(cherry picked from commit 60882ab26d49f05cbf0686944af6559f86b3417d)
 2017-05-17 23:58:27 -07:00
Konstantin V Shvachko
0823fb72a9 HDFS-8498. Blocks can be committed with wrong size. Contributed by Jing Zhao. Backport HDFS-11732 by Zhe Zhang. 2017-05-16 11:03:12 -07:00
Brahma Reddy Battula
3004358639 HDFS-8312. Trash does not descent into child directories to check for permissions. Contributed By Weiwei Yang via Eric Yang. 2017-05-16 23:27:57 +05:30
Brahma Reddy Battula
b95d63cdac HDFS-10987. Make Decommission less expensive when lot of blocks present. Contributed By Brahma Reddy Battula. 2017-05-16 10:50:51 +05:30
Vinayakumar B
b0b6de38c9 HDFS-11674. reserveSpaceForReplicas is not released if append request failed due to mirror down and replica recovered (Contributed by Vinayakumar B) 2017-05-12 14:26:28 +05:30
Brahma Reddy Battula
764fd3ed49 HDFS-9902. Support different values of dfs.datanode.du.reserved per storage type. Contributed By Brahma Reddy Battula. 2017-05-11 10:22:59 +05:30
Akira Ajisaka
f052976ff1
HDFS-11795. Fix ASF License warnings in branch-2.7. Contributed by Yiqun Lin. 2017-05-10 14:17:00 -05:00
Brahma Reddy Battula
48c5e37abf HDFS-11373. Backport HDFS-11258 and HDFS-11272(File mtime change could not save to editlog). (Contributed by Jimmy Xiang, backported by Akira Ajisaka) 2017-05-10 11:06:57 +05:30
Akira Ajisaka
6599e57a79
HDFS-11766. Fix findbugs warning in branch-2.7. Contributed by Chen Liang. 2017-05-09 11:22:57 -05:00
Konstantin V Shvachko
c7c410898e Update CHANGES.txt for HDFS-11717 branch-2.7. 2017-05-02 22:30:18 -07:00
Erik Krogen
a00cef6079 HDFS-11717. Add unit test for HDFS-11709 StandbyCheckpointer should handle non-existing legacyOivImageDir gracefully. Contributed by Erik Krogen. 
(cherry picked from commit d9014bda93760f223789d2ec9f5e35f40de157d4)
(cherry picked from commit 9d6b873d3c249aa76420ab03ceefc6180295915f)
 2017-05-02 18:46:58 -07:00
Kihwal Lee
130a0e2156 HDFS-11714. Newly added NN storage directory won't get initialized and cause space exhaustion. Contributed by Kihwal Lee. 
Added a CHANGES.txt entry
(cherry picked from commit f77d138725662197ca0be874e9ec4dc430db2a4f)
 2017-05-01 17:36:58 -05:00
Kihwal Lee
139327d34b HDFS-11609. Some blocks can be permanently lost if nodes are decommissioned while dead. Contributed by Kihwal Lee. 2017-05-01 14:31:59 -05:00
Zhe Zhang
8ad9efbe14 HDFS-8873. Allow the directoryScanner to be rate-limited. Contributed by Daniel Templeton. 2017-04-27 15:04:10 -07:00
Konstantin V Shvachko
5e67eb7560 Update CHANGES.txt for HDFS-11384 branch-2.7. 2017-04-27 12:23:41 -07:00
Konstantin V Shvachko
cf5abf8a7c HDFS-11384. Balancer disperses getBlocks calls to avoid NameNode's rpc queue saturation. Contributed by Konstantin V Shvachko. 
(cherry picked from commit 28eb2aabebd15c15a357d86e23ca407d3c85211c)
# Conflicts:
#	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Dispatcher.java
#	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java
 2017-04-27 11:57:36 -07:00
Zhe Zhang
bec23593b5 HDFS-11709. StandbyCheckpointer should handle an non-existing legacyOivImageDir gracefully. Contributed by Erik Krogen. 
(cherry picked from commit d8a33098309f17dfb0e3a000934f68394de44bf7)
(cherry picked from commit fc46f398d9192e83c7197115589eca7805c49943)
(cherry picked from commit 70f18911c71c61f9b3918e195bfb9a99d22d230c)
 2017-04-27 09:56:14 -07:00
Brahma Reddy Battula
ac12063eeb HDFS-10455. Logging the username when deny the setOwner operation. Contributed by Rakesh R. 
(cherry picked from commit 1c5708f6ded6ba9fc007eed93c1f65ba44451241)
(cherry picked from commit c895f0893621fd6196c42ed46b97cb99cc2e9bee)
 2017-04-26 15:59:21 -07:00
Konstantin V Shvachko
54bbdb4c02 Update CHANGES.txt for HDFS-11634. 2017-04-18 13:20:21 -07:00