Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,887 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

4841 Commits

Author SHA1 Message Date
Konstantin V Shvachko 272a198175 HDFS-13706. [SBN read] Rename client context to ClientGSIContext. Contributed by Konstantin Shvachko. 2019-08-09 15:49:55 -07:00
Erik Krogen 85f81fe458 HDFS-13609. [SBN read] Edit Tail Fast Path Part 3: NameNode-side changes to support tailing edits via RPC. Contributed by Erik Krogen. 
HDFS-10519. Add a configuration option to enable in-progress edit log tailing. Contributed by Jiayi Zhou.
 2019-08-09 15:49:55 -07:00
Erik Krogen 88d65af8a1 HDFS-13608. [SBN read] Edit Tail Fast Path Part 2: Add ability for JournalNode to serve edits via RPC. Contributed by Erik Krogen. 2019-08-09 15:49:55 -07:00
Erik Krogen befe41074c HDFS-13607. [SBN read] Edit Tail Fast Path Part 1: Enhance JournalNode with an in-memory cache of recent edit transactions. Contributed by Erik Krogen. 2019-08-09 15:49:55 -07:00
Plamen Jeliazkov 3ed8d45660 HDFS-13399. [SBN read] Make Client field AlignmentContext non-static. Contributed by Plamen Jeliazkov. 2019-08-09 15:49:55 -07:00
Erik Krogen d5da529acf HDFS-13286. [SBN read] Add haadmin commands to transition between standby and observer. Contributed by Chao Sun. 2019-08-09 15:49:55 -07:00
Erik Krogen d0c85f83da HDFS-13331. [SBN read] Add lastSeenStateId to RpcRequestHeader. Contributed by Plamen Jeliazkov. 2019-08-09 15:49:55 -07:00
Plamen Jeliazkov dd70e6d364 HDFS-12977. [SBN read] Add stateId to RPC headers. Contributed by Plamen Jeliazkov. 2019-08-09 15:49:55 -07:00
Chao Sun 61b510815c HDFS-12975. [SBN read] Changes to the NameNode to support reads from standby. Contributed by Chao Sun. 2019-08-09 15:49:55 -07:00
Siyao Meng 15062b6d28 HDFS-14696. Backport HDFS-11273 to branch-2 (Move TransferFsImage#doGetUrl function to a Util class) (#1251) Contributed by Siyao Meng. 2019-08-08 14:55:41 -07:00
Erik Krogen ce12c8fc54 HDFS-14034. Support getQuotaUsage API in WebHDFS. Contributed by Chao Sun. 2019-08-08 10:18:00 -07:00
Erik Krogen 97600cd916 HDFS-14631. The DirectoryScanner doesn't fix the wrongly placed replica. Contributed by Jinglun. 2019-08-07 13:46:56 -07:00
Yiqun Lin 63531be505 HDFS-14313. Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du. Contributed by Lisheng Sun. 2019-08-07 23:01:51 +08:00
Weiwei Yang 567e1178d8 HDFS-12914. Block report leases cause missing blocks until next report. Contributed by Santosh Marella, He Xiaoqiao. 2019-08-06 14:30:17 -07:00
Erik Krogen ee1547a807 HDFS-14697. Backport of HDFS-14513. FSImage which is saving should be clean while NameNode shutdown. Contributed by He Xiaoqiao. 2019-08-06 09:15:24 -07:00
Erik Krogen a503662d6c HDFS-14462 Ensure WebHDFS client throws the correct exception during writes. Contributed by Simbarashe Dzinamarira. 
(cherry picked from e7a0b8aa83)
(cherry picked from ba4ca7d996cafa1b1786249cae2b207f0eaa1070)
(cherry picked from da548cf263)
(cherry picked from 055f824bcb)
 2019-08-02 11:53:34 -07:00
Erik Krogen 61581a86b2 HDFS-12703. Exceptions are fatal to decommissioning monitor. Contributed by He Xiaoqiao. 2019-07-29 14:36:57 -07:00
Wei-Chiu Chuang 27ead06a13 HDFS-14429. Block remain in COMMITTED but not COMPLETE caused by Decommission. Contributed by Yicong Cai. 2019-07-29 14:34:31 -07:00
Erik Krogen 653951e1f7 Revert "HDFS-14135. TestWebHdfsTimeouts Fails intermittently in trunk. Contributed by Ayush Saxena." 
This reverts commit 3bfae20e94.
 2019-07-26 08:27:36 -07:00
Masatake Iwasaki 3bfae20e94 HDFS-14135. TestWebHdfsTimeouts Fails intermittently in trunk. Contributed by Ayush Saxena. 
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit 6b8107ad97)
 2019-07-26 09:54:27 +09:00
Erik Krogen cf7b9a5bd5 HDFS-14547. Improve memory efficiency of quotas when storage type quotas are not set. Contributed by Jinglun. 2019-07-16 07:22:45 -07:00
Weiwei Yang 3f112e7347 HDFS-12748. NameNode memory leak when accessing webhdfs GETHOMEDIRECTORY. Contributed by Weiwei Yang. 
(cherry picked from commit 6c8cfd135e)
 2019-07-11 10:07:47 +08:00
Yiqun Lin 78aebee5c5 HDFS-14632. Reduce useless #getNumLiveDataNodes call in SafeModeMonitor. Contributed by He Xiaoqiao. 
(cherry picked from commit 993dc8726b)
 2019-07-10 10:58:59 +08:00
stack a7c6d25dc3 HDFS-14483 Backport HDFS-3246,HDFS-14111 ByteBuffer pread interface to branch-2.9"" 
This is a revert of a revert, i.e. a reapplication of the HDFS-14483
patch. Original commit was missing the JIRA number.

Revert "Revert "Backport HDFS-3246,HDFS-14111 ByteBuffer pread interface to branch-2.9""

This reverts commit b26ad8a4c3.
 2019-07-09 08:58:16 -07:00
stack b26ad8a4c3 Revert "Backport HDFS-3246,HDFS-14111 ByteBuffer pread interface to branch-2.9" 
This reverts commit 4f09a6bcfd.
 2019-07-09 08:57:57 -07:00
sunlisheng 4f09a6bcfd Backport HDFS-3246,HDFS-14111 ByteBuffer pread interface to branch-2.9 
Signed-off-by: stack <stack@apache.org>
 2019-07-09 08:54:37 -07:00
leosunli 0ff5db746e HDFS-13694. Making md5 computing being in parallel with image loading. 
(cherry picked from commit 9c90729486)
 2019-07-07 20:04:08 -07:00
Masatake Iwasaki e61e52805f HDFS-14629. Property value Hard Coded in DNConf.java. Contributed by hemanthboyina. 
(cherry picked from commit 1c254a8ec7)
 2019-07-04 23:55:13 +09:00
Inigo Goiri 0272480e9f HDFS-14247. Repeat adding node description into network topology. Contributed by HuangTao. 
(cherry picked from commit 80b77deb42)
(cherry picked from commit 9637124535)
(cherry picked from commit 90b88db35d)
(cherry picked from commit 9daa45f646)
 2019-06-24 21:17:38 -07:00
Wei-Chiu Chuang bf63475e7a HDFS-14303. check block directory logic not correct when there is only meta file, print no meaning warn log. Contributed by qiang Liu. 2019-06-20 21:06:06 -07:00
Wei-Chiu Chuang f876e82abc HDFS-14465. When the Block expected replications is larger than the number of DataNodes, entering maintenance will never exit. Contributed by Yicong Cai. 2019-06-20 19:36:44 -07:00
Zsolt Venczel a68de43957 HDFS-14101. Random failure of testListCorruptFilesCorruptedBlock. Contributed by Zsolt Venczel, Nikhil Navadia. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Co-authored-by: Nikhil Navadia <nikhil.navadiya@cloudera.com>
(cherry picked from commit 7c00756aff)
(cherry picked from commit 4ea3b04bf3)
(cherry picked from commit 76faa41f18)
 2019-06-18 14:38:42 -07:00
Kitti Nanasi abd11f9db1 HDFS-13770. dfsadmin -report does not always decrease "missing blocks (with replication factor 1)" metrics when file is deleted. Contributed by Kitti Nanasi. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-06-18 00:25:53 -07:00
Kihwal Lee 500dc05924 HDFS-14521. Suppress setReplication logging. Contributed by Kihwal Lee 
(cherry picked from commit 9122b9b649)
(cherry picked from commit fd8b8c7494)
 2019-05-30 17:37:38 -05:00
Ayush Saxena c63049dfd4 HDFS-14512. ONE_SSD policy will be violated while write data with DistributedFileSystem.create(....favoredNodes). Contributed by Ayush Saxena. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit c1caab40f2)
(cherry picked from commit 2cd7577a5a)
(cherry picked from commit 90cc5ddd49)
(cherry picked from commit 942c5b04f2)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDistributedFileSystem.java
 2019-05-29 21:11:15 -07:00
Siyao Meng cecba551aa HDFS-14514. Actual read size of open file in encryption zone still larger than listing size even after enabling HDFS-11402 in Hadoop 2. Contributed by Siyao Meng, Stephen O'Donnell. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Co-authored-by: Stephen O'Donnell <sodonnell@cloudera.com>
 2019-05-29 17:28:49 -07:00
Takanobu Asanuma e3d4f6bfed HADOOP-16331. Fix ASF License check in pom.xml. Contributed by Akira Ajisaka. 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2019-05-29 17:41:21 +09:00
Akira Ajisaka cea1b0000c
HADOOP-16323. https everywhere in Maven settings. 2019-05-27 15:28:54 +09:00
Erik Krogen f024d27cd9 HDFS-14500. NameNode StartupProgress should not allow new steps in an already-completed phase. Contributed by Erik Krogen. 
(cherry-picked from 55e0c134f0)
(cherry-picked from c26cf22d6b)
(cherry-picked from 9295e95174)
 2019-05-24 13:39:52 -07:00
Konstantin V Shvachko 2afc770443 HDFS-14502. keepResults option in NNThroughputBenchmark should call saveNamespace(). Contributed by Konstantin V Shvachko. 2019-05-24 12:43:12 -07:00
Wei-Chiu Chuang 380289a167 HDFS-13087. Snapshotted encryption zone information should be immutable. Contributed by LiXin Ge, Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Co-authored-by: Siyao Meng <smeng@cloudera.com>
 2019-05-24 18:36:53 +02:00
Puleya7 8579ab13cb HDFS-14432. dfs.datanode.shared.file.descriptor.paths duplicated in hdfs-default.xml 
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit 13907d8479)
 2019-04-17 21:58:43 +09:00
Wei-Chiu Chuang 8af767b069 HDFS-10477. Stop decommission a rack of DataNodes caused NameNode fail over to standby. Contributed by yunjiong zhao and Wei-Chiu Chuang. 2019-04-10 14:36:07 -07:00
Dinesh Chitlangia 66ac003128 HDFS-14414. Clean up findbugs warning in branch-2. Contributed by Dinesh Chitlangia. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-04-08 10:47:13 -07:00
Chen Liang dce76e0355 HDFS-14415. Backport HDFS-13799 to branch-2. Contributed by Chao Sun. 2019-04-05 11:27:37 -07:00
Akira Ajisaka 2abc7927d1
HDFS-14407. Fix misuse of SLF4j logging API in DatasetVolumeChecker#checkAllVolumes. Contributed by Wanqiang Ji. 
(cherry picked from commit 67020f0950)
 2019-04-05 12:32:07 +09:00
Chen Liang a5bca767f7 HDFS-14397. Backport HADOOP-15684 to branch-2. Contributed by Chao Sun. 2019-04-03 17:32:56 -07:00
Chen Liang d6083e9920 HDFS-14399. Backport HDFS-10536 to branch-2. Contributed by Chao Sun. 2019-04-01 12:47:02 -07:00
Chen Liang 812e6d5c81 HDFS-14392. Backport HDFS-9787 to branch-2. Contributed by Chao Sun. 2019-03-28 19:07:58 -07:00
Chen Liang 99e41293ee HDFS-14391. Backport HDFS-9659 to branch-2. Contributed by Chao Sun. 2019-03-28 10:39:45 -07:00