Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,292 Commits 326 Branches 371 Tags
Commit Graph

11292 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Brahma Reddy Battula
aa8f4cc48c HDFS-9530. ReservedSpace is not cleared for abandoned Blocks (Contributed by Brahma Reddy Battula) 
(cherry picked from commit f2ac132d6a21c215093b7f87acf2843ac8123716)
(cherry picked from commit ee0f389ecd0c479aa38dae7ee6fd02ed66a64e4e)
 2016-06-21 15:55:57 +05:30
Rohith Sharma K S
33e6986ec9 YARN-5197. RM leaks containers if running container disappears from node update. Contributed by Jason Lowe. 2016-06-21 10:12:01 +05:30
Colin Patrick Mccabe
9b51420ea5 HDFS-10448. CacheManager#addInternal tracks bytesNeeded incorrectly when dealing with replication factors other than 1 (Yiqun Lin via cmccabe) 
(cherry picked from commit 46f1602e896273b308fbd5df6c75f6c142828227)
(cherry picked from commit 4e11f33ccc849355eaeb0694dbee702e34340a74)
 2016-06-20 18:32:00 -07:00
Jitendra Pandey
29c3ee9744 HDFS-10538. Remove AsyncDistributedFileSystem. Contributed by Xiaobing Zhou and Jitendra Pandey. 2016-06-20 18:05:32 -07:00
Jitendra Pandey
2f95d22b17 HADOOP-13291. Probing stats in DFSOpsCountStatistics/S3AStorageStatistics should be correctly implemented. Contributed by Mingliang Liu. 2016-06-20 16:31:54 -07:00
Ravi Prakash
588ea6d431 HDFS-10220. A large number of expired leases can make namenode unresponsive and cause failover (Nicolas Fraison via raviprak) 
(cherry picked from commit ae047655f4355288406cd5396fb4e3ea7c307b14)
(cherry picked from commit 9c5f7f290eb05808fe89835a15bee0947b91d1a0)
 2016-06-20 14:38:05 -07:00
Colin Patrick Mccabe
c5dafb4ad7 HADOOP-13288. Guard null stats key in FileSystemStorageStatistics (Mingliang Liu via Colin P. McCabe) 
(cherry picked from commit 8c1f81d4bf424bdc421cf4952b230344e39a7b68)
(cherry picked from commit 14df17383c20cebcdb27da1f8bce0b494d47b392)
 2016-06-20 14:30:51 -07:00
Colin Patrick Mccabe
c610b60331 HADOOP-13280. FileSystemStorageStatistics#getLong(“readOps“) should return readOps + largeReadOps (Mingliang Liu via cmccabe) 
(cherry picked from commit 5370a6ffaec5227c0978f10c86a5811155271933)
(cherry picked from commit 7db1e57cc160a902a688db7068b6e692d8263a65)

Conflicts:
    hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileSystemStorageStatistics.java
 2016-06-20 10:53:14 -07:00
Akira Ajisaka
615a023be3 HADOOP-13192. org.apache.hadoop.util.LineReader cannot handle multibyte delimiters correctly. Contributed by binde. 
(cherry picked from commit fc6b50cc574e144fd314dea6c11987c6a384bfa6)
(cherry picked from commit 39ea0891d2b1369ec8c1ea4312489309e1a85227)
 2016-06-20 17:08:19 +09:00
Brahma Reddy Battula
9d27530c36 HDFS-10474. hftp copy fails when file name with Chinese+special char in branch-2 (Contributed by Brahma Reddy Battula) 
(cherry picked from commit 3f27f405039926bbee4b36a60d6cecdb6322fff5)
 2016-06-20 12:40:11 +05:30
Chris Nauroth
55658dcaa3 Revert "HADOOP-12718. Incorrect error message by fs -put local dir without permission. (John Zhuge via Yongjun Zhang)" 
This reverts commit 62206dd2edcc53dbfbae30a524c21e7eb828bd0e.

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
 2016-06-17 23:51:27 -07:00
Andrew Wang
3f2d2cd0a1 Fix hadoop-hdfs-native-client compilation on Windows. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 3ff0510ffdc20e44f06aec591782f8875b686327)
(cherry picked from commit 94759470b8a922d410797b8b76e055fd2fc6dd69)
 2016-06-17 15:55:43 -07:00
Chris Nauroth
c8b0ec5363 HADOOP-13149. Windows distro build fails on dist-copynativelibs. Contributed by Chris Nauroth. 
(cherry picked from commit b9dbef94177dfe0d8ea9526722691f861f47bb4e)
 2016-06-17 15:55:41 -07:00
Kihwal Lee
edd5373095 HDFS-10493. Add links to datanode web UI in namenode datanodes page. Contributed by Weiwei Yang. 
(cherry picked from commit 280069510b71d81a150b2b28d9fa987891d82774)
 2016-06-17 11:13:12 -05:00
Akira Ajisaka
242263c56d HADOOP-12943. Add -w -r options in dfs -test command. Contributed by Weiwei Yang. 
(cherry picked from commit 09e82acaf9a6d7663bc51bbca0cdeca4b582b535)
(cherry picked from commit a36aa920f437f9e21772e8067e5db86fbffe7895)
 2016-06-17 16:21:31 +09:00
Xiaoyu Yao
e84a9c976a HADOOP-13255. KMSClientProvider should check and renew tgt when doing delegation token operations. Contributed by Xiao Chen. 
(cherry picked from commit ddf66427ff92a3886f94954e21d59e998412059b)
 2016-06-16 20:12:55 -07:00
Konstantin V Shvachko
408848d1e9 HADOOP-13189. FairCallQueue makes callQueue larger than the configured capacity. Contributed by Vinitha Gankidi. 2016-06-16 18:33:05 -07:00
Colin Patrick Mccabe
04682cc6b0 HADOOP-12975. Add jitter to CachingGetSpaceUsed's thread (Elliott Clark via Colin P. McCabe) 
(cherry picked from commit bf780406f2b30e627bdf36ac07973f6931f81106)
(cherry picked from commit 56c997a1a61188693ed306b3ceddb13a2f2e6456)
 2016-06-16 17:03:52 -07:00
Colin Patrick Mccabe
b07cd9c6a9 HADOOP-13284. FileSystemStorageStatistics must not attempt to read non-existent rack-aware read stats in branch-2.8 (Mingliang Liu via cmccabe) 2016-06-16 16:45:41 -07:00
Colin Patrick Mccabe
cc53ca8d69 HDFS-9466. TestShortCircuitCache#testDataXceiverCleansUpSlotsOnFailure is flaky (Wei-Chiu Chuang via cmccabe) 
(cherry picked from commit c7921c9bddb79c9db5059b6c3f7a3a586a3cd95b)
(cherry picked from commit 49c0d9b325a21fc0a05da58bb9d4c76d82181a5d)
 2016-06-16 13:53:55 -07:00
Ravi Prakash
605443c913 HADOOP-3733. "s3x:" URLs break when Secret Key contains a slash, even if encoded. Contributed by Steve Loughran. 2016-06-16 20:45:59 +01:00
Chris Nauroth
eddefbd01b HADOOP-13241. document s3a better. Contributed by Steve Loughran. 
(cherry picked from commit 127d2c7281917f23bce17afa6098a2d678a16441)
(cherry picked from commit e4b63ed7b4a001b9a41f68fc4846e971a9a8eb51)
 2016-06-16 11:19:00 -07:00
Wangda Tan
93b406b66d YARN-4844. Addendum patch, revert few changes to public and evolving APIs to avoid breaking downstream projects as much as possible. 
(cherry picked from commit 7eb524ddee4af80c87b134070945977bf73e950f)
 2016-06-16 11:10:46 -07:00
Arpit Agarwal
394dae17ac HDFS-10532. Typo in RollingUpgrade docs. (Contributed by Yiqun Lin) 2016-06-16 10:11:05 -07:00
Junping Du
810470508b YARN-5237. Fix missing log files issue in rolling log aggregation. Contributed by Xuan Gong. 2016-06-16 07:18:36 -07:00
Colin Patrick Mccabe
ea2e7321d6 HDFS-10525. Fix NPE in CacheReplicationMonitor#rescanCachedBlockMap (Xiao Chen via cmccabe) 
(cherry picked from commit 2ca73445f5c2929d9c2ff4232dca58a63a0570a0)
(cherry picked from commit 8280e3bcc21adbbbb21e091bd4ec786bf3beb72b)
 2016-06-15 22:51:27 -07:00
Colin Patrick Mccabe
ef08eb9031 HDFS-10505. OIV's ReverseXML processor should support ACLs (Surendra Singh Lilhore via cmccabe) 
(cherry picked from commit 2449db507d84b1c4fac70a800fb2ad8905cf3db7)
(cherry picked from commit 17301ab3958d0b36fc6179efe1090a3b12b34772)
 2016-06-15 22:44:41 -07:00
Akira Ajisaka
b35c7cdd14 HADOOP-12892. fix/rewrite create-release. Contributed by Allen Wittenauer. 2016-06-16 14:06:08 +09:00
Akira Ajisaka
9128137142 HADOOP-12893. Verify LICENSE.txt and NOTICE.txt. Contributed by Xiao Chen, Andrew Wang, and Akira Ajisaka. 
(cherry picked from commit 6ffb557521af2ea0c21cfc90d8edbd7cb7305f0f)
 2016-06-15 13:33:10 +09:00
Ming Ma
03ed3f9b66 Revert "HDFS-9016. Display upgrade domain information in fsck. (mingma)" 
This reverts commit 68ab0aa9d0b9bd03768dcd7c422ef201ad3f58f6.
 2016-06-14 20:14:00 -07:00
Ming Ma
68ab0aa9d0 HDFS-9016. Display upgrade domain information in fsck. (mingma) 
(cherry picked from commit b7436f44682442e4d431ec3c97e4753e03435f3e)
 2016-06-14 20:08:29 -07:00
Wangda Tan
d838c6443d YARN-1942. Deprecate toString/fromString methods from ConverterUtils and move them to records classes like ContainerId/ApplicationId, etc. (wangda) 2016-06-14 15:21:41 -07:00
Masatake Iwasaki
3c9d31c55a HADOOP-13244. o.a.h.ipc.Server#Server should honor handlerCount when queueSizePerHandler is specified in consturctor. Contributed by Kai Sasaki. 
(cherry picked from commit 20b13d109a1f66d8911936d26bdde38d81a1c4c6)
 2016-06-14 19:52:48 +09:00
Vinayakumar B
3bf5379adb HADOOP-13179. GenericOptionsParser is not thread-safe because commons-cli OptionBuilder is not thread-safe. (Contributed by hongbin ma) 
(cherry picked from commit 5cd19e429e7aef8751599dc5f54ffe29a9dc3ea9)
(cherry picked from commit 06b76f15869e072261ba10b3e9059bbafae8c7f8)
 2016-06-14 15:59:36 +05:30
Akira Ajisaka
8295351e54 HADOOP-13270. BZip2CompressionInputStream finds the same compression marker twice in corner case, causing duplicate data blocks. Contributed by Kai Sasaki. 
(cherry picked from commit e3ba9ad3f116306910f74645ded91506345b9f6e)
(cherry picked from commit d219550e8bf4c24be68a74f7b8032b6a8b8af0fc)
 2016-06-14 10:24:53 +09:00
Andrew Wang
56e29d2711 HDFS-10516. Fix bug when warming up EDEK cache of more than one encryption zone. Contributed by Xiao Chen. 
(cherry picked from commit 709a814fe0153e86a37806796ea27c8252d9c6d1)
(cherry picked from commit ff4466fe15c338e56bab2e4afd61c46b634843b3)
 2016-06-13 11:50:27 -07:00
Andrew Wang
08ff8d7482 HADOOP-13079. Add -q option to Ls to print ? instead of non-printable characters. Contributed by John Zhuge. 
(cherry picked from commit 0accc3306d830c3f2b16c4b8abf68729c7aba6cb)
(cherry picked from commit 6a8d3233fc248e3606930ecbdb07d3d926f550a7)
 2016-06-13 11:44:08 -07:00
Akira Ajisaka
f5cdf94725 HADOOP-13213. Small Documentation bug with AuthenticatedURL in hadoop-auth. Contributed by Tom Ellis. 
This closes #97.

(cherry picked from commit 8a1dcceccea5cfe9fb0c94daa2517cd64a16b3c0)
(cherry picked from commit 035eea4d126f2dfe4b01f47f8ee670fd58127320)
 2016-06-11 03:33:40 +09:00
Wangda Tan
9436548152 YARN-3426. Add jdiff support to YARN. (vinodkv via wangda) 2016-06-10 09:56:18 -07:00
Akira Ajisaka
3bea925cea MAPREDUCE-6741. Refactor UncompressedSplitLineReader.fillBuffer(). Contributed by Daniel Templeton. 
(cherry picked from commit 0b7b8a377611b2a3041a2995504a437c36dfa6e6)
(cherry picked from commit db2d96b378d6a4ba9a504b4daaca5d8ff6f5f227)
 2016-06-10 19:17:26 +09:00
Steve Loughran
faf9a9f4e0 HADOOP-12537 S3A to support Amazon STS temporary credentials. Contributed by Sean Mackrory. 2016-06-09 20:58:30 +01:00
Junping Du
2be48e7d15 YARN-5199. Close LogReader in NMWebServices#getLogs. Contributed by Xuan Gong. 2016-06-09 12:29:25 -07:00
Steve Loughran
7e09601a90 HADOOP-13237: s3a initialization against public bucket fails if caller lacks any credentials. Contributed by Chris Nauroth 2016-06-09 16:36:51 +01:00
Allen Wittenauer
3b2a25bc4b HADOOP-12743. Fix git environment check during test-patch (aw) 
(cherry picked from commit d323639686eab28f1510031e52e4390f82d78989)
(cherry picked from commit a086fd90408c0dca13d22d66ab528a68b4f57f42)
 2016-06-09 23:27:23 +09:00
Andrew Wang
f583fa5f4e Remove redundant TestMiniDFSCluster.testDualClusters. Contributed by Jiayi Zhou. 
(cherry picked from commit 1ee9ea002609971ad58082bf525d57fca8a37035)
(cherry picked from commit 9319665461201ca84c7fa007c09f087f5753c8cc)
 2016-06-08 12:59:08 -07:00
Xuan
11b4d1e486 Revert "YARN-4920. ATS/NM should support a link to dowload/get the logs in text format. Contributed by Xuan Gong." 
This reverts commit 22ac37615a933f9cee8cf19ad0182586a037b690.
 2016-06-08 11:23:12 -07:00
Vinod Kumar Vavilapalli
593d2c66c4 YARN-5080. Addendum fix to the original patch to fix YARN logs CLI. Contributed by Xuan Gong 
(cherry picked from commit 5a43583c0bbb9650ea6a9f48d9544ec3ec24b580)
 2016-06-08 09:52:37 -07:00
Jason Lowe
fb31aaef63 YARN-5206. RegistrySecurity includes id:pass in exception text if considered invalid. Contributed by Steve Loughran 
(cherry picked from commit 8c8a377cac10b086a7ff37ee366b79e6b04d2738)
 2016-06-08 14:13:48 +00:00
Wangda Tan
19e578870d YARN-4844. Add getMemorySize/getVirtualCoresSize to o.a.h.y.api.records.Resource. (wangda) 2016-06-07 12:41:50 -07:00
Junping Du
333490ba45 HADOOP-10048. LocalDirAllocator should avoid holding locks while accessing the filesystem. Contributed by Jason Lowe. 
(cherry picked from commit c14c1b298e29e799f7c8f15ff24d7eba6e0cd39b)
(cherry picked from commit 19eb997f675a843a2bfe21094ef524f8c5870919)
 2016-06-07 09:26:05 -07:00