Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,989 Commits 326 Branches 371 Tags
Commit Graph

3195 Commits

Author SHA1 Message Date
Tsz-Wo Nicholas Sze
ce788c207f HADOOP-13227. AsyncCallHandler should use an event driven architecture to handle async calls. 2016-06-25 15:51:45 +08:00
Jing Zhao
e251654ee4 HADOOP-13249. RetryInvocationHandler need wrap InterruptedException in IOException when call Thread.sleep. Contributed by Zhihai Xu. 
(cherry picked from commit 0bbb4ddd793063c87927035969884a34f60f2076)
 2016-06-25 15:51:35 +08:00
Colin Patrick Mccabe
2d681a9246 Fix accidental revert of HADOOP-13072: WindowsGetSpaceUsed constructor should be public (Vinayakumar B via cmccabe) 
(cherry picked from commit 975786492c42b8aebc6dc247bdcdb4768d99d641)
(cherry picked from commit e751bb4497879d66ec771e6f9e08937a3164516b)
 2016-06-24 09:47:41 -07:00
Andrew Wang
f28c270c96 Revert "HADOOP-13228. Add delegation token to the connection in DelegationTokenAuthenticator. Contributed by Xiao Chen." 
This reverts commit 4ad9ca85fbde69369264f089c0195f90232dc3bd.
 2016-06-23 11:09:06 -07:00
Allen Wittenauer
d4474d7d09 HADOOP-13245. Fix up some misc create-release issues (aw) 
(cherry picked from commit e2f640942b722e35490cf146c0268517da5a28b1)
(cherry picked from commit 2a956baaabe4a86a63fc181c3a07b2f4c4ee2a75)
 2016-06-23 15:16:33 +09:00
Masatake Iwasaki
9c46f99cc6 HADOOP-12588. addendum patch to fix intermittent failure of TestGangliaMetrics. (iwasakims) 
(cherry picked from commit 67089875f0ea2a47cca0c237fccf239bf6fe2d53)
 2016-06-22 23:49:04 +09:00
Steve Loughran
1eb45180a6 HADOOP-13203 S3A: Support fadvise "random" mode for high performance readPositioned() reads. Contributed by Rajesh Balamohan and stevel. 2016-06-22 15:41:52 +01:00
Akira Ajisaka
6fefb8f4a4 HADOOP-13296. Cleanup javadoc for Path. Contributed by Daniel Templeton. 
(cherry picked from commit e15cd43369eb6d478844f25897e4a86065c62168)
(cherry picked from commit 65b4f2682e4b1803a4fd7348bd518b8723724686)
 2016-06-21 20:17:42 +09:00
Colin Patrick Mccabe
c5dafb4ad7 HADOOP-13288. Guard null stats key in FileSystemStorageStatistics (Mingliang Liu via Colin P. McCabe) 
(cherry picked from commit 8c1f81d4bf424bdc421cf4952b230344e39a7b68)
(cherry picked from commit 14df17383c20cebcdb27da1f8bce0b494d47b392)
 2016-06-20 14:30:51 -07:00
Colin Patrick Mccabe
c610b60331 HADOOP-13280. FileSystemStorageStatistics#getLong(“readOps“) should return readOps + largeReadOps (Mingliang Liu via cmccabe) 
(cherry picked from commit 5370a6ffaec5227c0978f10c86a5811155271933)
(cherry picked from commit 7db1e57cc160a902a688db7068b6e692d8263a65)

Conflicts:
    hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileSystemStorageStatistics.java
 2016-06-20 10:53:14 -07:00
Akira Ajisaka
615a023be3 HADOOP-13192. org.apache.hadoop.util.LineReader cannot handle multibyte delimiters correctly. Contributed by binde. 
(cherry picked from commit fc6b50cc574e144fd314dea6c11987c6a384bfa6)
(cherry picked from commit 39ea0891d2b1369ec8c1ea4312489309e1a85227)
 2016-06-20 17:08:19 +09:00
Brahma Reddy Battula
9d27530c36 HDFS-10474. hftp copy fails when file name with Chinese+special char in branch-2 (Contributed by Brahma Reddy Battula) 
(cherry picked from commit 3f27f405039926bbee4b36a60d6cecdb6322fff5)
 2016-06-20 12:40:11 +05:30
Chris Nauroth
55658dcaa3 Revert "HADOOP-12718. Incorrect error message by fs -put local dir without permission. (John Zhuge via Yongjun Zhang)" 
This reverts commit 62206dd2edcc53dbfbae30a524c21e7eb828bd0e.

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
 2016-06-17 23:51:27 -07:00
Akira Ajisaka
242263c56d HADOOP-12943. Add -w -r options in dfs -test command. Contributed by Weiwei Yang. 
(cherry picked from commit 09e82acaf9a6d7663bc51bbca0cdeca4b582b535)
(cherry picked from commit a36aa920f437f9e21772e8067e5db86fbffe7895)
 2016-06-17 16:21:31 +09:00
Xiaoyu Yao
e84a9c976a HADOOP-13255. KMSClientProvider should check and renew tgt when doing delegation token operations. Contributed by Xiao Chen. 
(cherry picked from commit ddf66427ff92a3886f94954e21d59e998412059b)
 2016-06-16 20:12:55 -07:00
Konstantin V Shvachko
408848d1e9 HADOOP-13189. FairCallQueue makes callQueue larger than the configured capacity. Contributed by Vinitha Gankidi. 2016-06-16 18:33:05 -07:00
Colin Patrick Mccabe
04682cc6b0 HADOOP-12975. Add jitter to CachingGetSpaceUsed's thread (Elliott Clark via Colin P. McCabe) 
(cherry picked from commit bf780406f2b30e627bdf36ac07973f6931f81106)
(cherry picked from commit 56c997a1a61188693ed306b3ceddb13a2f2e6456)
 2016-06-16 17:03:52 -07:00
Colin Patrick Mccabe
b07cd9c6a9 HADOOP-13284. FileSystemStorageStatistics must not attempt to read non-existent rack-aware read stats in branch-2.8 (Mingliang Liu via cmccabe) 2016-06-16 16:45:41 -07:00
Chris Nauroth
eddefbd01b HADOOP-13241. document s3a better. Contributed by Steve Loughran. 
(cherry picked from commit 127d2c7281917f23bce17afa6098a2d678a16441)
(cherry picked from commit e4b63ed7b4a001b9a41f68fc4846e971a9a8eb51)
 2016-06-16 11:19:00 -07:00
Masatake Iwasaki
3c9d31c55a HADOOP-13244. o.a.h.ipc.Server#Server should honor handlerCount when queueSizePerHandler is specified in consturctor. Contributed by Kai Sasaki. 
(cherry picked from commit 20b13d109a1f66d8911936d26bdde38d81a1c4c6)
 2016-06-14 19:52:48 +09:00
Vinayakumar B
3bf5379adb HADOOP-13179. GenericOptionsParser is not thread-safe because commons-cli OptionBuilder is not thread-safe. (Contributed by hongbin ma) 
(cherry picked from commit 5cd19e429e7aef8751599dc5f54ffe29a9dc3ea9)
(cherry picked from commit 06b76f15869e072261ba10b3e9059bbafae8c7f8)
 2016-06-14 15:59:36 +05:30
Akira Ajisaka
8295351e54 HADOOP-13270. BZip2CompressionInputStream finds the same compression marker twice in corner case, causing duplicate data blocks. Contributed by Kai Sasaki. 
(cherry picked from commit e3ba9ad3f116306910f74645ded91506345b9f6e)
(cherry picked from commit d219550e8bf4c24be68a74f7b8032b6a8b8af0fc)
 2016-06-14 10:24:53 +09:00
Andrew Wang
08ff8d7482 HADOOP-13079. Add -q option to Ls to print ? instead of non-printable characters. Contributed by John Zhuge. 
(cherry picked from commit 0accc3306d830c3f2b16c4b8abf68729c7aba6cb)
(cherry picked from commit 6a8d3233fc248e3606930ecbdb07d3d926f550a7)
 2016-06-13 11:44:08 -07:00
Akira Ajisaka
f5cdf94725 HADOOP-13213. Small Documentation bug with AuthenticatedURL in hadoop-auth. Contributed by Tom Ellis. 
This closes #97.

(cherry picked from commit 8a1dcceccea5cfe9fb0c94daa2517cd64a16b3c0)
(cherry picked from commit 035eea4d126f2dfe4b01f47f8ee670fd58127320)
 2016-06-11 03:33:40 +09:00
Wangda Tan
9436548152 YARN-3426. Add jdiff support to YARN. (vinodkv via wangda) 2016-06-10 09:56:18 -07:00
Steve Loughran
faf9a9f4e0 HADOOP-12537 S3A to support Amazon STS temporary credentials. Contributed by Sean Mackrory. 2016-06-09 20:58:30 +01:00
Steve Loughran
7e09601a90 HADOOP-13237: s3a initialization against public bucket fails if caller lacks any credentials. Contributed by Chris Nauroth 2016-06-09 16:36:51 +01:00
Junping Du
333490ba45 HADOOP-10048. LocalDirAllocator should avoid holding locks while accessing the filesystem. Contributed by Jason Lowe. 
(cherry picked from commit c14c1b298e29e799f7c8f15ff24d7eba6e0cd39b)
(cherry picked from commit 19eb997f675a843a2bfe21094ef524f8c5870919)
 2016-06-07 09:26:05 -07:00
Tsz-Wo Nicholas Sze
49535c0683 Revert "Revert "HADOOP-13226 Support async call retry and failover."" 
This reverts commit 886e2396062ef076b3ea4aa93395cc90287efd4f.
 2016-06-06 16:54:02 +08:00
Tsz-Wo Nicholas Sze
c949001eee Revert "Revert "HADOOP-13168. Support Future.get with timeout in ipc async calls."" 
This reverts commit 308d28640d8c797f07aa41a8adc5dcd9420f1d0a.
 2016-06-06 16:53:41 +08:00
Tsz-Wo Nicholas Sze
8ccc8aeb6c Revert "Revert "HADOOP-12957. Limit the number of outstanding async calls. Contributed by Xiaobing Zhou"" 
This reverts commit 8a07026e7792e43e692409c3ec998a306b7733ff.
 2016-06-06 16:53:31 +08:00
Tsz-Wo Nicholas Sze
2cf663bb59 Revert "Revert "HDFS-10224. Implement asynchronous rename for DistributedFileSystem. Contributed by Xiaobing Zhou"" 
This reverts commit d740a90260d9d26a67688b004e5fcda60cc30723.
 2016-06-06 16:53:24 +08:00
Andrew Wang
d740a90260 Revert "HDFS-10224. Implement asynchronous rename for DistributedFileSystem. Contributed by Xiaobing Zhou" 
This reverts commit bdc45bef646cafdc04a59c19e23dcba3bb16b20c.
 2016-06-03 18:17:33 -07:00
Andrew Wang
8a07026e77 Revert "HADOOP-12957. Limit the number of outstanding async calls. Contributed by Xiaobing Zhou" 
This reverts commit a1ba6eee38420ae9436f9e9ecc3db83702fbe762.
 2016-06-03 18:17:33 -07:00
Andrew Wang
308d28640d Revert "HADOOP-13168. Support Future.get with timeout in ipc async calls." 
This reverts commit fa7c7f25105bc157c652a2f44ac49620fc61c0f4.
 2016-06-03 18:12:31 -07:00
Andrew Wang
886e239606 Revert "HADOOP-13226 Support async call retry and failover." 
This reverts commit a8941d7790b2209ac779c9372298b833ededd132.
 2016-06-03 18:12:30 -07:00
Andrew Wang
d712b2ee3b HADOOP-13155. Implement TokenRenewer to renew and cancel delegation tokens in KMS. Contributed by Xiao Chen. 
(cherry picked from commit 713cb71820ad94a5436f35824d07aa12fcba5cc6)

 Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/KeyProviderDelegationTokenExtension.java
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSUtilClient.java

(cherry picked from commit 03c4724c884b133eeb1114e417f7d49d239f4783)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSUtilClient.java
 2016-06-03 17:03:40 -07:00
Chris Nauroth
e54f073cfe HADOOP-13105. Support timeouts in LDAP queries in LdapGroupsMapping. Contributed by Mingliang Liu. 
(cherry picked from commit d82bc8501869be78780fc09752dbf7af918c14af)
(cherry picked from commit 5e6ee5aafb9b9f200d906444e4731cfc60ac6ebb)

Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestLdapGroupsMapping.java
 2016-06-03 16:40:48 -07:00
Jian He
3c2bd19fa5 YARN-5190. Registering/unregistering container metrics in ContainerMonitorImpl and ContainerImpl causing uncaught exception in ContainerMonitorImpl. Contributed by Junping Du 
(cherry picked from commit 99cc439e29794f8e61bebe03b2a7ca4b6743ec92)
 2016-06-03 11:11:49 -07:00
Chris Nauroth
b8216c10d8 HADOOP-13171. Add StorageStatistics to S3A; instrument some more operations. Contributed by Steve Loughran. 2016-06-03 08:56:36 -07:00
Akira Ajisaka
d0dc5aaa2d HDFS-10341. Add a metric to expose the timeout number of pending replication blocks. (aajisaka) 
(cherry picked from commit b6d5546e2414009311ca7e92203b7b4a6b29e165)
 2016-06-03 16:04:40 +09:00
Masatake Iwasaki
5b3d9ca0cd HDFS-10367. TestDFSShell.testMoveWithTargetPortEmpty fails with Address bind exception. Contributed by Brahma Reddy Battula. 
(cherry picked from commit aadb77e412ab9d4ad05a0bd8b37d547ba5adad03)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java

(cherry picked from commit 37409744135186a9b90865436a731366b259ea81)
 2016-06-02 21:04:48 +09:00
Akira Ajisaka
f7e1a1eaa7 HADOOP-13232. Typo in exception in ValueQueue.java. Contributed by Jiayi Zhou. 
(cherry picked from commit 99675e00df65e84dcc02082b795f1f93c90b794e)
(cherry picked from commit 6843e552087d77bade52599a10c59ff55e1e3ed7)
 2016-06-02 19:15:57 +09:00
Andrew Wang
4ad9ca85fb HADOOP-13228. Add delegation token to the connection in DelegationTokenAuthenticator. Contributed by Xiao Chen. 
(cherry picked from commit 35356de1ba1cad0fa469ff546263290109c61b77)
(cherry picked from commit 55eda958ccbf3941986d6da63ad7ee16646d7067)
 2016-06-01 13:13:38 -07:00
Steve Loughran
809e359d2d HADOOP-13162. Consider reducing number of getFileStatus calls in S3AFileSystem.mkdirs. (Rajesh Balamohan via stevel) 2016-06-01 14:17:18 +01:00
Colin Patrick Mccabe
6fe1c67fa1 HADOOP-13137. TraceAdmin should support Kerberized cluster (Wei-Chiu Chuang via cmccabe) 
(cherry picked from commit 8ceb06e2392763726210f96bb1c176e6a9fe7b53)
(cherry picked from commit a64e1425ed12edc7434b2f40d111111caf1db274)
 2016-05-31 18:26:37 -07:00
Tsz-Wo Nicholas Sze
a8941d7790 HADOOP-13226 Support async call retry and failover. 2016-05-31 17:16:44 -07:00
Jing Zhao
03c4491d89 HADOOP-13146. Refactor RetryInvocationHandler. Contributed by Tsz Wo Nicholas Sze. 2016-05-31 17:16:26 -07:00
Andrew Wang
63f73d78a1 HADOOP-13132. Handle ClassCastException on AuthenticationException in LoadBalancingKMSClientProvider. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit bca31fe276ccf7d02b13f25d43c81cca0b0b905b)
 2016-05-31 09:35:30 -07:00
Xiaoyu Yao
1d2928ece7 HADOOP-13197. Add non-decayed call metrics for DecayRpcScheduler. Contributed by Xiaoyu Yao. 
(cherry picked from commit 4ca8859583839761663fc1fc1de1b3ce2e3fc5b5)
(cherry picked from commit 001c8f5d7b54812bb992c7d73db0a08cab2267ec)
 2016-05-27 18:51:46 -07:00