Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
8,968 Commits 326 Branches 371 Tags
Commit Graph

2969 Commits

Author SHA1 Message Date
Jing Zhao
e1ce0700eb HDFS-7934. Update RollingUpgrade rollback documentation: should use bootstrapstandby for standby NN. Contributed by J. Andreina. 
(cherry picked from commit b172d03595d1591e7f542791224607d8c5fce3e2)
(cherry picked from commit 725621d95a833f813e2164e32b33984e1c1ee29a)
 2015-04-15 13:11:45 -07:00
Jing Zhao
9e11ae684a HDFS-8151. Always use snapshot path as source when invalid snapshot names are used for diff based distcp. Contributed by Jing Zhao. 
(cherry picked from commit 4c097e473bb1f18d1510deb61bae2bcb8c156f18)
(cherry picked from commit d4dd97eabd6691eedeeb9fb7685060dfb192ff21)
 2015-04-15 12:41:34 -07:00
Jing Zhao
50c8d06314 HDFS-8127. NameNode Failover during HA upgrade can cause DataNode to finalize upgrade. Contributed by Jing Zhao. 
(cherry picked from commit fddd55279d0bdd08b3b40aba6fe2ded1d2e0d846)
(cherry picked from commit 38b031d6bab8527698bd186887d301bd6a63cf01)
 2015-04-14 15:15:36 -07:00
Arun Suresh
0a95b4f8aa HDFS-7931. DistributedFileSystem should not look for keyProvider in cache if Encryption is disabled (asuresh) 
(cherry picked from commit 7660da95cb67cbfe034aa8fa2a5bf0f8c9fdf41a)
 2015-04-10 11:43:49 -07:00
Konstantin V Shvachko
e964bbc856 HDFS-8081. Split getAdditionalBlock() into two methods. Contributed by Konstantin Shvachko 2015-04-09 22:04:48 -07:00
Vinod Kumar Vavilapalli
a7fa46c2e0 Preparing for 2.7.1 development: mvn versions:set 2015-04-09 14:13:30 -07:00
Vinod Kumar Vavilapalli
2b44fcaa53 Adding release 2.7.1 to CHANGES.txt 
(cherry picked from commit 623fd46c1eac057ea9bc4b503a47ad1a831cdacf)
 2015-04-09 14:01:23 -07:00
Colin Patrick Mccabe
4d5b1fbdec HDFS-8063: Fix intermittent test failures in TestTracing (Masatake Iwasaki via Colin P. McCabe) 
(cherry picked from commit 61dc2ea3fee4085b19cd2d01de9eacdc4c42e21f)
(cherry picked from commit 8dac245920f41709dd6343873682f172fdbeb0a1)
 2015-04-09 11:42:29 -07:00
Kihwal Lee
9a111fcd1d HDFS-8025. Addendum fix for HDFS-3087 Decomissioning on NN restart can complete without blocks being replicated. Contributed by Ming Ma. 
(cherry picked from commit 5a540c3d3107199f4632e2ad7ee8ff913b107a04)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-04-09 10:01:44 -05:00
Haohui Mai
84ba4db2a0 Revert HDFS-7813. 2015-04-08 16:03:35 -07:00
Haohui Mai
eb53d6b967 Revert HDFS-7808. 2015-04-08 16:00:26 -07:00
Arpit Agarwal
12739b541b HDFS-8072. Reserved RBW space is not released if client terminates while writing block. (Arpit Agarwal) 2015-04-08 11:38:42 -07:00
cnauroth
5021f52f7b HDFS-8038. PBImageDelimitedTextWriter#getEntry output HDFS path in platform-specific format. Contributed by Xiaoyu Yao. 
(cherry picked from commit 672ed462965ea7da10ce3df48c02e2a3bd13f0ae)
(cherry picked from commit 950dc1b1995191327e6b919ee17a87e59b749264)
 2015-04-07 13:33:38 -07:00
Konstantin V Shvachko
2cb9dac9a3 HDFS-8071. Redundant checkFileProgress() in PART II of getAdditionalBlock(). Contributed by Konstantin Shvachko. 2015-04-06 23:35:01 -07:00
Colin Patrick Mccabe
3f1d1d0b48 HDFS-7999. FsDatasetImpl#createTemporary sometimes holds the FSDatasetImpl lock for a very long time (sinago via cmccabe) 
(cherry picked from commit 28bebc81db8bb6d1bc2574de7564fe4c595cfe09)
(cherry picked from commit a827089905524e10638c783ba908a895d621911d)
 2015-04-06 09:10:34 -07:00
Vinod Kumar Vavilapalli
5387a289fc HADOOP-11776. Fixed the broken JDiff support in Hadoop 2. Contributed by Li Lu. 
(cherry picked from commit 4b3948ea365db07df7a9369a271009fafd1ba8f5)
 2015-04-04 13:53:34 -07:00
Vinod Kumar Vavilapalli
996b0ca173 HADOOP-11377. Added Null.java without which jdiff completely flops. Contributed by Tsuyoshi Ozawa. 
(cherry picked from commit 5370e7128b4b78dabff79986a92151f1de39eeed)
 2015-04-04 13:48:58 -07:00
Colin Patrick Mccabe
aa5eab8ec4 HDFS-8051. Fix missing import statement from backport. 2015-04-03 19:07:25 -07:00
Colin Patrick Mccabe
54b88943ff HDFS-8051. FsVolumeList#addVolume should release volume reference if not put it into BlockScanner. (Lei (Eddy) Xu via Colin P. McCabe) 
(cherry picked from commit ef591b1d6a08f08358b19763a874de6010227307)
(cherry picked from commit b26ba22a9023ac2ae058abf509db67aa8ef64b41)
 2015-04-03 16:57:06 -07:00
Colin Patrick Mccabe
520043b438 HDFS-7996. After swapping a volume, BlockReceiver reports ReplicaNotFoundException (Lei (Eddy) Xu via Colin P. McCabe) 
(cherry picked from commit 023133cef9a7ca05364cefbcead57c921589eda7)
(cherry picked from commit 0c5069c4329305db876f0c0f5bd8e983d46f854e)
 2015-04-03 14:27:09 -07:00
cnauroth
6855bbc6ca HDFS-8039. Fix TestDebugAdmin#testRecoverLease and testVerfiyBlockChecksumCommand on Windows. Contributed by Xiaoyu Yao. 
(cherry picked from commit db80e428919e4e763f56b9344c748e7f8fb46491)
(cherry picked from commit 28e0602e7bf20581e3ce477512c60a854a4612d3)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-04-03 10:45:15 -07:00
Jing Zhao
b08bc43c9b HDFS-7811. Avoid recursive call getStoragePolicyID in INodeFile#computeQuotaUsage. Contributed by Xiaoyu Yao and Jing Zhao. 
(cherry picked from commit 72f6bd4893dcf10d6dad24753f9be99505a87a1f)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

(cherry picked from commit 2c1f33d17853bc5e9bd00872d221067f1fa7c9c7)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-04-02 21:27:32 -07:00
Tsz-Wo Nicholas Sze
acfe44e5b0 HDFS-7954. TestBalancer#testBalancerWithPinnedBlocks should not be executed on Windows. Contributed by Xiaoyu Yao 2015-04-02 11:14:04 -07:00
Brandon Li
a445f8b675 HDFS-8001 RpcProgramNfs3 : wrong parsing of dfs.blocksize. Contributed by Remi Catherinot 
(cherry picked from commit 4d14816c269f110445e1ad3e03ac53b0c1cdb58b)
(cherry picked from commit 27970b3f46674ef7824a4478141d968ef2c16ebb)
 2015-04-01 17:30:00 -07:00
Haohui Mai
8ebbbc6ebd HDFS-8036. Use snapshot path as source when using snapshot diff report in DistCp. Contributed by Jing Zhao. 2015-04-01 16:51:25 -07:00
Haohui Mai
86c0c6b044 HDFS-7748. Separate ECN flags from the Status in the DataTransferPipelineAck. Contributed by Anu Engineer and Haohui Mai. 2015-03-30 12:16:25 -07:00
Kihwal Lee
a0ed29a058 HDFS-7742. Favoring decommissioning node for replication can cause a block to stay 
underreplicated for long periods. Contributed by Nathan Roberts.
(cherry picked from commit 04ee18ed48ceef34598f954ff40940abc9fde1d2)
 2015-03-30 10:11:47 -05:00
Colin Patrick Mccabe
9411cf9b65 HDFS-7410. Support CreateFlags with append() to support hsync() for appending streams (Vinayakumar B via Colin P. McCabe) 
(cherry picked from commit 61df1b27a797efd094328c7d9141b9e157e01bf4)
(cherry picked from commit 2a750c9aa68ae4f1b1b3ade082ec9362e1f1c5d2)
 2015-03-26 13:29:17 -07:00
Arpit Agarwal
c808d5fe55 HDFS-7824. GetContentSummary API and its namenode implementation for Storage Type Quota/Usage. (Contributed by Xiaoyu Yao) 2015-03-26 10:48:03 -07:00
Kihwal Lee
370c91461f HDFS-7963. Fix expected tracing spans in TestTracing. Contributed by Masatake Iwasaki. 2015-03-26 08:45:39 -05:00
Jitendra Pandey
d286673c60 HDFS-6826. Plugin interface to enable delegation of HDFS authorization assertions. Contributed by Arun Suresh. 
(cherry picked from commit 456cec127b23b9195784dd4b35b75a2b69ad2a4a)
 2015-03-24 16:17:06 -07:00
Brandon Li
14d1cccfad HDFS-7977. NFS couldn't take percentile intervals. Contributed by Brandon Li 
(cherry picked from commit 570a83ae80faf2076966acf30588733803327844)
(cherry picked from commit 260dbe96c3d21ea372c728b2cd0f4a6aeb970010)
 2015-03-24 10:53:20 -07:00
Brandon Li
cc5922607a HDFS-7976. Update NFS user guide for mount option 'sync' to minimize or avoid reordered writes. Contributed by Brandon Li 
(cherry picked from commit 51f1f4937402c63392854443ccde18c1579e3348)
(cherry picked from commit 95bde8898636156a442949535fd776ccd638e337)
 2015-03-24 10:30:19 -07:00
Andrew Wang
a7f447d460 HDFS-7961. Trigger full block report after hot swapping disk. Contributed by Eddy Xu. 
(cherry picked from commit 6413d34986f3399023426c89c9a0d401c9557716)
 2015-03-24 09:07:27 -07:00
Plamen Jeliazkov
da318cbbde HDFS-7956. Improve logging for DatanodeRegistration. Contributed by Plamen Jeliazkov. 2015-03-23 23:09:52 -07:00
Andrew Wang
af0af28afc HDFS-7960. The full block report should prune zombie storages even if they're not empty. Contributed by Colin McCabe and Eddy Xu. 
(cherry picked from commit 50ee8f4e67a66aa77c5359182f61f3e951844db6)
 2015-03-23 22:01:37 -07:00
Tsz-Wo Nicholas Sze
87079cde7d HDFS-7884. Fix NullPointerException in BlockSender when the generation stamp provided by the client is larger than the one stored in the datanode. Contributed by Brahma Reddy Battula 2015-03-24 13:52:18 +09:00
cnauroth
8e1c33e703 HDFS-7917. Use file to replace data dirs in test to simulate a disk failure. Contributed by Lei (Eddy) Xu. 
(cherry picked from commit 2c238ae4e00371ef76582b007bb0e20ac8455d9c)
(cherry picked from commit 01c0bcb176e22ddefbc8086e382dd1ebd105f9c6)
 2015-03-23 16:30:48 -07:00
Akira Ajisaka
2742f12b58 HDFS-7881. TestHftpFileSystem#testSeek fails in branch-2. Contributed by Brahma Reddy Battula. 
(cherry picked from commit fad8c78173c4b7c55324033720f04a09943deac7)
 2015-03-24 06:25:21 +09:00
Brandon Li
6b9f2d9f39 HDFS-7942. NFS: support regexp grouping in nfs.exports.allowed.hosts. Contributed by Brandon Li 
(cherry picked from commit 36af4a913c97113bd0486c48e1cb864c5cba46fd)
(cherry picked from commit 503d8e4164ff3da29fcaf56436fe6fab6a450105)
 2015-03-23 10:13:11 -07:00
Kihwal Lee
08f2f7ed34 HDFS-6841. Use Time.monotonicNow() wherever applicable instead of Time.now(). Contributed by Vinayakumar B 
(cherry picked from commit 99a8dcd19528b265d4fda9ae09a17e4af52f2782)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/DatanodeInfo.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
(cherry picked from commit 63b7870aec007081c82427587cb6e1d38e2c70f8)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
 2015-03-20 14:06:02 -05:00
Jing Zhao
03fb9b4ce0 HDFS-7957. Truncate should verify quota before making changes. Contributed by Jing Zhao. 
(cherry picked from commit d368d3647a858644b9fcd3be33d9fea2a6962f69)
(cherry picked from commit 6be52e42a9eb2069cedb12a8311bc36131a3804f)
 2015-03-20 11:56:39 -07:00
yliu
faeb8b9b53 HDFS-7962. Remove duplicated logs in BlockManager. (yliu) 2015-03-20 05:32:19 +08:00
yliu
0da3fcb4b0 HDFS-7930. commitBlockSynchronization() does not remove locations. (yliu) 2015-03-19 23:26:26 +08:00
Kihwal Lee
4e967d0bcd HDFS-7932. Speed up the shutdown of datanode during rolling upgrade. Contributed by Kihwal Lee. 
(cherry picked from commit 61a4c7fc9891def0e85edf7e41d74c6b92c85fdb)
 2015-03-19 12:29:16 -05:00
Vinayakumar B
76ce2e8ea0 HDFS.7849. Update documentation for enabling a new feature in rolling upgrade ( Contributed by J.Andreina ) 
(cherry picked from commit 5be5129d93481e9caae6a05b2361c463211a6d55)
 2015-03-19 19:10:23 +05:30
Kihwal Lee
b8f269af9d HDFS-7816. Unable to open webhdfs paths with "+". Contributed by Haohui Mai 
(cherry picked from commit e79be0ee123d05104eb34eb854afcf9fa78baef2)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
(cherry picked from commit ceb39c1cc6b52178172c879d96cd743e0dc7a650)
 2015-03-19 08:04:19 -05:00
Colin Patrick Mccabe
9e61835678 HDFS-7929. inotify unable fetch pre-upgrade edit log segments once upgrade starts (Zhe Zhang via Colin P. McCabe) 
(cherry picked from commit 43b41f22411439c5e23629197fb2fde45dcf0f0f)
(cherry picked from commit 219eb22c1571f76df32967a930049d983cbf5024)
 2015-03-18 19:08:53 -07:00
Jing Zhao
0bf549728d HDFS-7587. Edit log corruption can happen if append fails with a quota violation. Contributed by Jing Zhao. 
(cherry picked from commit c7c71cdba50cb7d8282622cd496cc913c80cff54)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

(cherry picked from commit 5a5b2446485531f12d37f3d4ca791672b9921872)
 2015-03-18 19:00:06 -07:00
Jing Zhao
4b8de154aa HDFS-7943. Append cannot handle the last block with length greater than the preferred block size. Contributed by Jing Zhao. 
(cherry picked from commit bee5a6a64a1c037308fa4d52249be39c82791590)
(cherry picked from commit 6dcc79507d70ff70a1efee6475795430f519d66b)
 2015-03-18 18:47:21 -07:00