Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,311 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

4367 Commits

Author SHA1 Message Date
Brahma Reddy Battula 9a168ae884 HDFS-11896. Non-dfsUsed will be doubled on dead node re-registration. Contributed by Brahma Reddy Battula. 
(cherry picked from commit c4a85c694f)
 2017-07-27 12:37:36 -07:00
Kihwal Lee c586b7ab7d HADOOP-13933. Add haadmin -getAllServiceState option to get the HA state of all the NameNodes/ResourceManagers. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit e407449ddb)
(cherry picked from commit 2d6be7ea23)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestRMAdminCLI.java
 2017-07-26 14:15:39 -05:00
Kihwal Lee 38c56bbb46 HDFS-12171. Reduce IIP object allocations for inode lookup. Contributed by Daryn Sharp. 
(cherry picked from commit 3d8c7d701b)
 2017-07-25 11:08:57 -05:00
Kihwal Lee 7be52c9c79 HDFS-12143. Improve performance of getting and removing inode features. Contributed by Daryn Sharp. 
(cherry picked from commit 1a79dcfc45)
 2017-07-25 10:33:09 -05:00
Brahma Reddy Battula 587d47cfec HDFS-8312. Trash does not descent into child directories to check for permissions. Contributed By Weiwei Yang via Eric Yang. 
(cherry picked from commit 820496fbbc)
 2017-07-23 16:09:10 +08:00
Kihwal Lee e229ffee64 HDFS-11742. Improve balancer usability after HDFS-8818. Contributed by Kihwal Lee 
(cherry picked from commit 8e3a992ecc)
 2017-07-21 09:22:28 -05:00
Brahma Reddy Battula f65dc6ee95 HDFS-12177. NameNode exits due to setting BlockPlacementPolicy loglevel to Debug. Contributed by Jiandan Yang. 2017-07-21 21:13:20 +08:00
Konstantin V Shvachko f3bf1a6eb9 HDFS-11472. Fix inconsistent replica size after a data pipeline failure. Contributed by Erik Krogen and Wei-Chiu Chuang. 
(cherry picked from commit 2a5a313539)
 2017-07-20 16:09:56 -07:00
Anu Engineer 6ed569df21 HDFS-12158. Secondary Namenode's web interface lack configs for X-FRAME-OPTIONS protection. Contributed by Mukul Kumar Singh. 
(cherry picked from commit 413b23eb04)
(cherry picked from commit e0297ffbc8)
 2017-07-19 10:46:21 -07:00
Brahma Reddy Battula 6039a2a55f HDFS-12112. TestBlockManager#testBlockManagerMachinesArray sometimes fails with NPE. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit b778887af5)
 2017-07-15 11:12:56 +08:00
Xiao Chen 8b7d1df766 HADOOP-14521. KMS client needs retry logic. Contributed by Rushabh S Shah. 2017-07-14 22:23:13 -07:00
Kihwal Lee e28c74102d HDFS-12140. Remove BPOfferService lock contention to get block pool id. Contributed by Daryn Sharp. 
(cherry picked from commit e7d187a1b6)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPOfferService.java
 2017-07-14 16:17:44 -05:00
Kihwal Lee 561b72970f HDFS-12137. DN dataset lock should be fair. Contributed by Daryn Sharp. 
(cherry picked from commit a47820d72a)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
 2017-07-14 15:50:18 -05:00
Xiaoyu Yao c9adca70e6 HDFS-11502. Datanode UI should display hostname based on JMX bean instead of window.location.hostname. Contributed by Jeffrey E Rodriguez and Yuanbo Liu. 
(cherry picked from commit e15e2713e1)
 2017-07-12 16:39:22 -07:00
Akira Ajisaka d1fb4948d2
HADOOP-14629. Improve exception checking in FileContext related JUnit tests. Contributed by Andras Bokor. 
(cherry picked from commit 9144fd9e9b)
(cherry picked from commit 2653cf4466)
 2017-07-12 11:44:15 +09:00
Mingliang Liu db6e8ff747 HDFS-12089. Fix ambiguous NN retry log message in WebHDFS. Contributed by Eric Badger 
(cherry picked from commit 6436768baf)
 2017-07-05 11:12:41 -07:00
Akira Ajisaka 9f9d554edf
HDFS-12078. Add time unit to the description of property dfs.namenode.stale.datanode.interval in hdfs-default.xml. Contributed by Weiwei Yang. 
(cherry picked from commit 186650d21d)
(cherry picked from commit 063f2132ef)
 2017-07-04 14:53:02 +09:00
Wei-Chiu Chuang 8027246306 HDFS-12042. Lazy initialize AbstractINodeDiffList#diffs for snapshots to reduce memory consumption. Contributed by Misha Dmitriev. 
(cherry picked from commit bcba844d11)
(cherry picked from commit 94bc5cdbb3)
 2017-06-30 11:43:41 -07:00
Wei-Chiu Chuang 7c076f917f HDFS-11881. NameNode consumes a lot of memory for snapshot diff report generation. Contributed by Manoj Govindassamy. 
(cherry picked from commit 16c8dbde57)
(cherry picked from commit 6917c19167)
 2017-06-29 06:44:37 -07:00
Kihwal Lee a560104fab HDFS-11960. Successfully closed files can stay under-replicated. Contributed by Kihwal Lee. 
(cherry picked from commit 8f533ca76b)
 2017-06-20 12:12:33 -05:00
Kihwal Lee e87bf17129 HDFS-9754. Avoid unnecessary getBlockCollection calls in BlockManager. Contributed by Jing Zhao. 
(cherry picked from commit 972782d956)
(cherry picked from commit f8c9c0ff0e)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/Namesystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMetadataConsistency.java
 2017-06-20 12:10:43 -05:00
Kihwal Lee 1f9255672d Revert "HDFS-11960. Successfully closed files can stay under-replicated. Contributed by Kihwal Lee." 
This reverts commit 40e3cd24b0.
 2017-06-20 09:22:25 -05:00
Kihwal Lee 40e3cd24b0 HDFS-11960. Successfully closed files can stay under-replicated. Contributed by Kihwal Lee. 
(cherry picked from commit 8f533ca76b)
 2017-06-20 09:20:02 -05:00
Brahma Reddy Battula a526c3b0bf HADOOP-14533. Size of args cannot be less than zero in TraceAdmin#run as its linkedlist. Contributed by Weisen Han. 
(cherry picked from commit 2e9daa2e27)
 2017-06-20 20:46:21 +08:00
Akira Ajisaka 189e39a90d
HDFS-11345. Document the configuration key for FSNamesystem lock fairness. Contributed by Erik Krogen. 
(cherry picked from commit 2c769167db)
(cherry picked from commit 6b3c13d543)
 2017-06-20 16:41:10 +09:00
Akira Ajisaka 6a4a63b0fd
HDFS-11736. OIV tests should not write outside 'target' directory. Contributed by Yiqun Lin. 
(cherry picked from commit 71626fd0b9)
 2017-06-20 13:56:20 +09:00
Masatake Iwasaki 6032511239 HDFS-11995. HDFS Architecture documentation incorrectly describes writing to a local temporary file. Contributed by Nandakumar. 
(cherry picked from commit d954a64730)
 2017-06-20 09:25:05 +09:00
Akira Ajisaka 4eb577c9f0
Revert "HDFS-11736. OIV tests should not write outside 'target' directory. Contributed by Yiqun Lin." 
This reverts commit a52315d2c3.
 2017-06-16 00:22:24 +09:00
Akira Ajisaka a52315d2c3
HDFS-11736. OIV tests should not write outside 'target' directory. Contributed by Yiqun Lin. 
(cherry picked from commit 315f07700d)
(cherry picked from commit 88fa17c238)
 2017-06-15 13:47:40 +09:00
Arpit Agarwal 6ca195997b HDFS-11967. TestJMXGet fails occasionally. Contributed by Arpit Agarwal. 2017-06-12 16:04:02 -07:00
Mingliang Liu 72d895a11f HDFS-11945. Internal lease recovery may not be retried for a long time. Contributed by Kihwal Lee 
(cherry picked from commit facb6812d9)
 2017-06-08 14:49:37 -07:00
wenxin he 2c12102edd
HADOOP-14208. Fix typo in the top page in branch-2.8 
This closes #230

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2017-06-08 11:36:24 +09:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) 396de0fd8f Preparing for 2.8.3 development 2017-06-07 15:38:20 -07:00
Brahma Reddy Battula 8a16846f5d HDFS-11711. DN should not delete the block On "Too many open files" Exception. Contributed by Brahma Reddy Battula. 
(cherry picked from commit fca08f8362)
 2017-06-07 16:59:26 +08:00
Vinayakumar B 4a391c72d4 HDFS-11708. Positional read will fail if replicas moved to different DNs after stream is opened. Contributed by Vinayakumar B. 2017-06-07 10:59:04 +05:30
Arpit Agarwal 5fc4b8567b HDFS-11932. BPServiceActor thread name is not correctly set. Contributed by Chen Liang. 2017-06-06 13:52:20 -07:00
Kihwal Lee 1d3fa87011 HDFS-10816. TestComputeInvalidateWork#testDatanodeReRegistration fails due to race between test and replication monitor. Contributed by Eric Badger. 
(cherry picked from commit e2ea601231)
 2017-06-05 15:28:01 -05:00
Xiao Chen ec10a81bdb HDFS-11741. Long running balancer may fail due to expired DataEncryptionKey. Contributed by Wei-Chiu Chuang and Xiao Chen. 
(cherry picked from commit 068e23b896c63b0f817e6b91d73c994be1551eb2)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/security/token/block/BlockTokenSecretManager.java
(cherry picked from commit 6f2391e0ad6fbaa6786cc9f0e3a09b955e0cb21c)
 2017-06-01 14:05:31 -07:00
Kihwal Lee 89d59c292f HDFS-5042. Completed files lost after power failure. Contributed by Vinayakumar B. 2017-06-01 12:25:27 -05:00
Brahma Reddy Battula 0cba282261 HDFS-11893. Fix TestDFSShell.testMoveWithTargetPortEmpty failure. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 0dcf843c00)
 2017-06-01 22:40:43 +08:00
Kihwal Lee fe6de5e594 HDFS-11856. Ability to re-add upgrading nodes to pipeline for future pipeline updates. Contributed by Vinayakumar B. 2017-05-31 13:08:51 -05:00
Inigo Goiri a20789f299 Fix NPE in LazyPersistFileScrubber. Contributed by Inigo Goiri. 
(cherry picked from commit 303c8dc9b6)
 2017-05-26 13:42:55 -07:00
Kihwal Lee bbef16b84e HDFS-11817. A faulty node can cause a lease leak and NPE on accessing data. Contributed by Kihwal Lee. 
(Updated TestBlockUnderConstruction for nextGenerationStamp method)
(cherry picked from commit 2cb63433ab)
 2017-05-25 17:34:56 -05:00
Brahma Reddy Battula 4f5846f1e3 HDFS-11445. FSCK shows overall health stauts as corrupt even one replica is corrupt. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 7bad748091)
 2017-05-25 22:40:11 +08:00
Wei-Chiu Chuang caad191c0e Revert "HDFS-10797. Disk usage summary of snapshots causes renamed blocks to get counted twice. Contributed by Sean Mackrory." 
This reverts commit 998e44a972.
 2017-05-24 18:12:03 -07:00
Wei-Chiu Chuang 6c50341ea8 Revert "HDFS-11515. -du throws ConcurrentModificationException. Contributed by Istvan Fajth, Wei-Chiu Chuang." 
This reverts commit f9d3bb3b83.
 2017-05-24 18:11:42 -07:00
Brahma Reddy Battula a7f063576c HDFS-11849. JournalNode startup failure exception should be logged in log file. Contributed By Surendra Singh Lilhore. 
(cherry picked from commit 9cab42cc79)
 2017-05-24 23:45:57 +08:00
Yiqun Lin 94a80974d3 HDFS-11377. Balancer hung due to no available mover threads. Contributed by yunjiong zhao. 
(cherry picked from commit 9cbbd1eae8)
 2017-05-22 17:11:52 -07:00
Kihwal Lee 95e8f4c80d HDFS-11641. Reduce cost of audit logging by using FileStatus instead of HdfsFileStatus. Contributed by Daryn Sharp. 
(cherry picked from commit 82ea3f4545)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
 2017-05-16 15:55:53 -05:00
Akira Ajisaka 9b595ba40b
HDFS-11833. HDFS architecture documentation descibes outdated placement policy. Contributed by Chen Liang. 
(cherry picked from commit 1d1c52b42f)
(cherry picked from commit c17cb03a23)
 2017-05-16 11:54:27 -04:00