Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,573 Commits 326 Branches 371 Tags
Commit Graph

11573 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Akira Ajisaka
4fba09d7e8 HDFS-10582. Change deprecated configuration fs.checkpoint.dir to dfs.namenode.checkpoint.dir in HDFS Commands Doc. Contributed by Pan Yuxuan. 
(cherry picked from commit a3f93beaa19d9acb7b1db2958102d0004b0eee46)
(cherry picked from commit 8a9440298db2c3943cbe399c68e247adeefa28b0)
 2016-07-06 17:54:44 -07:00
Varun Saxena
22ca536b0c YARN-5286. Add RPC port info in RM web service's response when getting app status. (Jun Gong via Varun Saxena). 2016-07-06 09:31:48 +05:30
Vinod Kumar Vavilapalli
4ea87cb38c YARN-5214. Fixed locking in DirectoryCollection to avoid hanging NMs when various code-paths hit slow disks. Contributed by Junping Du. 
(cherry picked from commit ce9c006430d13a28bc1ca57c5c70cc1b7cba1692)
 2016-07-05 17:12:37 -07:00
Akira Ajisaka
4b263b30c2 HDFS-10589. Javadoc for HAState#HAState and HAState#setStateInternal contains non-existent params. Contributed by shenyinjie. 
(cherry picked from commit 8b4b5259d5088360476b4582c4cf7b0e7973eff3)
(cherry picked from commit 0401adcaca5f2e9680bc2dc3602af8696d204f21)
 2016-07-04 07:15:54 -07:00
Jitendra Pandey
bc7fd76a1f HADOOP-13305. Define common statistics names across schemes. Contributed by Mingliang Liu. 2016-07-01 15:37:32 -07:00
Jian He
4437e6f336 YARN-5023. TestAMRestart#testShouldNotCountFailureToMaxAttemptRetry fails. Contributed by sandflee 
(cherry picked from commit c35a5a7a8d85b42498e6981a6b1f09f2bdd56459)
 2016-07-01 14:29:47 -07:00
Vinayakumar B
2c6e7f221e HADOOP-13314. Remove 'package-info.java' from 'test\java\org\apache\hadoop\fs\shell\' to remove eclipse compile error (Contributed by Vinayakumar B) 
(cherry picked from commit c25021fb7196f498ccf1319dbd0c7f948f8518c1)
(cherry picked from commit 1458e66593511acfe4c2cd36853339bccc6fd93d)
 2016-07-01 12:23:23 +05:30
Andrew Wang
ae2ab601c0 HADOOP-12345. Pad hostname correctly in CredentialsSys.java. Contributed by Pradeep Nayak Udupi Kadbet. 
(cherry picked from commit 846ada2de33f9840d6e60fd7339ecc2d71f277c7)
(cherry picked from commit af581c0e2306970f4965f64cbbc2255c8bfc2191)
 2016-06-29 17:32:52 -07:00
Masatake Iwasaki
b5dbd3b5ed HADOOP-12242. Add in-page TOC to filesystem specification pages. (iwasakims) 
(cherry picked from commit 5c07c573db06481c89f1c9ad062e2d2aa19feaa0)
 2016-06-30 07:30:37 +09:00
Steve Loughran
52dbafecc5 HADOOP-12229 Fix inconsistent subsection titles in filesystem.md. Contributed by Masatake Iwasaki 2016-06-29 14:31:13 +01:00
Steve Loughran
f4f0c5074d HADOOP-13239 Deprecate s3:// in branch-2 2016-06-29 13:16:10 +01:00
Yongjun Zhang
4a57c03f30 HDFS-10396. Using -diff option with DistCp may get "Comparison method violates its general contract" exception. Contributed by Yongjun Zhang. 
(cherry picked from commit 8113855b3af913e6dcd09626953c64d43168efbd)
 2016-06-28 23:34:44 -07:00
Vinayakumar B
f44d7782f6 HDFS-10440. Improve DataNode web UI (Contributed by Weiwei Yang) 
(cherry picked from commit 2a0082c51da7cbe2770eddb5f72cd7f8d72fa5f6)
(cherry picked from commit 1e347631817d882353bfb91d68f109cb8232e8c4)
 2016-06-28 16:55:57 +05:30
Andrew Wang
47c13c4f7d HADOOP-13316. Enforce Kerberos authentication for required ops in DelegationTokenAuthenticator. Contributed by Xiao Chen. 
(cherry picked from commit 7d2070493e07198896bc49135bc84ef00499a375)
(cherry picked from commit 22eb53eb715deb95cb3fcc16cdf327e1406b9c64)
 2016-06-27 18:46:59 -07:00
Andrew Wang
59a70977c3 HADOOP-13251. Authenticate with Kerberos credentials when renewing KMS delegation token. Contributed by Xiao Chen. 
(cherry picked from commit 771f798edf97b27ae003395118c0317b484df6ee)

 Conflicts:
	hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMS.java

(cherry picked from commit d8a69c873725b681b06b116cdd10bdb3c2082963)
 2016-06-27 18:29:58 -07:00
Akira Ajisaka
a0f6109907 MAPREDUCE-6542. HistoryViewer uses SimpleDateFormat, but SimpleDateFormat is not threadsafe. Contributed by zhangyubiao. 2016-06-27 15:19:21 -07:00
Arpit Agarwal
f7bd302550 HADOOP-13263. Reload cached groups in background after expiry. (Contributed bt Stephen O'Donnell) 2016-06-27 09:51:16 -07:00
Tsz-Wo Nicholas Sze
ce788c207f HADOOP-13227. AsyncCallHandler should use an event driven architecture to handle async calls. 2016-06-25 15:51:45 +08:00
Jing Zhao
e251654ee4 HADOOP-13249. RetryInvocationHandler need wrap InterruptedException in IOException when call Thread.sleep. Contributed by Zhihai Xu. 
(cherry picked from commit 0bbb4ddd793063c87927035969884a34f60f2076)
 2016-06-25 15:51:35 +08:00
Akira Ajisaka
5af2af15ef HADOOP-13067. cleanup the dockerfile. Contributed by Allen Wittenauer. 
(cherry picked from commit b59b8b7351c8212a7ea05777fbe0eb7afb53bf62)
 2016-06-25 10:42:54 +09:00
Colin Patrick Mccabe
2d681a9246 Fix accidental revert of HADOOP-13072: WindowsGetSpaceUsed constructor should be public (Vinayakumar B via cmccabe) 
(cherry picked from commit 975786492c42b8aebc6dc247bdcdb4768d99d641)
(cherry picked from commit e751bb4497879d66ec771e6f9e08937a3164516b)
 2016-06-24 09:47:41 -07:00
Akira Ajisaka
ae5dc10ecd MAPREDUCE-6725. Javadoc for CLI#listEvents() contains no-existent param. Contributed by shenyinjie. 
(cherry picked from commit 0f23cd497b0baa0899c7565cd94e2f27dbc7c0ff)
(cherry picked from commit 2830ee5f0fdbdf8123d0dd0858baf2c059a3f96f)
 2016-06-25 00:27:35 +09:00
Andrew Wang
f28c270c96 Revert "HADOOP-13228. Add delegation token to the connection in DelegationTokenAuthenticator. Contributed by Xiao Chen." 
This reverts commit 4ad9ca85fbde69369264f089c0195f90232dc3bd.
 2016-06-23 11:09:06 -07:00
Akira Ajisaka
5c07aee0b0 HADOOP-13307. add rsync to Dockerfile so that precommit archive works. Contributed by Allen Wittenauer. 
(cherry picked from commit 85209cc59152bcd7070b0a654af7d64105d36a6b)
(cherry picked from commit fb13ab07270067062998381b238ba9def87aa304)
 2016-06-23 15:24:22 +09:00
Allen Wittenauer
d4474d7d09 HADOOP-13245. Fix up some misc create-release issues (aw) 
(cherry picked from commit e2f640942b722e35490cf146c0268517da5a28b1)
(cherry picked from commit 2a956baaabe4a86a63fc181c3a07b2f4c4ee2a75)
 2016-06-23 15:16:33 +09:00
Jing Zhao
d17265f467 HDFS-10556. DistCpOptions should be validated automatically. Contributed by Mingliang Liu. 
(cherry picked from commit 856bc4e28b356f31d3c48eed2c352fb9010232d7)
 2016-06-22 15:22:54 -07:00
Masatake Iwasaki
9c46f99cc6 HADOOP-12588. addendum patch to fix intermittent failure of TestGangliaMetrics. (iwasakims) 
(cherry picked from commit 67089875f0ea2a47cca0c237fccf239bf6fe2d53)
 2016-06-22 23:49:04 +09:00
Steve Loughran
1eb45180a6 HADOOP-13203 S3A: Support fadvise "random" mode for high performance readPositioned() reads. Contributed by Rajesh Balamohan and stevel. 2016-06-22 15:41:52 +01:00
Akira Ajisaka
700dacf8c4 HDFS-7597. DelegationTokenIdentifier should cache the TokenIdentifier to UGI mapping. Contributed by Daryn Sharp, Bob Hansen, and Xiao Chen. 
(cherry picked from commit d433b16ce6d74f1a44bc29446c74b1cb5f8a10fa)
(cherry picked from commit ad50a64ded1ba0c32d7bc89e5100f3b4d86a790c)
 2016-06-22 11:02:48 +09:00
Chris Nauroth
e30b7112ed HADOOP-13287. TestS3ACredentials#testInstantiateFromURL fails if AWS secret key contains +. Contributed by Chris Nauroth. 
(cherry picked from commit b2c596cdda7c129951074bc53b4b9ecfedbf080a)
(cherry picked from commit e6ebe9ab4e38651b39c926d98b1d3d01e6a9ed71)
 2016-06-21 11:34:46 -07:00
Akira Ajisaka
6fefb8f4a4 HADOOP-13296. Cleanup javadoc for Path. Contributed by Daniel Templeton. 
(cherry picked from commit e15cd43369eb6d478844f25897e4a86065c62168)
(cherry picked from commit 65b4f2682e4b1803a4fd7348bd518b8723724686)
 2016-06-21 20:17:42 +09:00
Brahma Reddy Battula
aa8f4cc48c HDFS-9530. ReservedSpace is not cleared for abandoned Blocks (Contributed by Brahma Reddy Battula) 
(cherry picked from commit f2ac132d6a21c215093b7f87acf2843ac8123716)
(cherry picked from commit ee0f389ecd0c479aa38dae7ee6fd02ed66a64e4e)
 2016-06-21 15:55:57 +05:30
Rohith Sharma K S
33e6986ec9 YARN-5197. RM leaks containers if running container disappears from node update. Contributed by Jason Lowe. 2016-06-21 10:12:01 +05:30
Colin Patrick Mccabe
9b51420ea5 HDFS-10448. CacheManager#addInternal tracks bytesNeeded incorrectly when dealing with replication factors other than 1 (Yiqun Lin via cmccabe) 
(cherry picked from commit 46f1602e896273b308fbd5df6c75f6c142828227)
(cherry picked from commit 4e11f33ccc849355eaeb0694dbee702e34340a74)
 2016-06-20 18:32:00 -07:00
Jitendra Pandey
29c3ee9744 HDFS-10538. Remove AsyncDistributedFileSystem. Contributed by Xiaobing Zhou and Jitendra Pandey. 2016-06-20 18:05:32 -07:00
Jitendra Pandey
2f95d22b17 HADOOP-13291. Probing stats in DFSOpsCountStatistics/S3AStorageStatistics should be correctly implemented. Contributed by Mingliang Liu. 2016-06-20 16:31:54 -07:00
Ravi Prakash
588ea6d431 HDFS-10220. A large number of expired leases can make namenode unresponsive and cause failover (Nicolas Fraison via raviprak) 
(cherry picked from commit ae047655f4355288406cd5396fb4e3ea7c307b14)
(cherry picked from commit 9c5f7f290eb05808fe89835a15bee0947b91d1a0)
 2016-06-20 14:38:05 -07:00
Colin Patrick Mccabe
c5dafb4ad7 HADOOP-13288. Guard null stats key in FileSystemStorageStatistics (Mingliang Liu via Colin P. McCabe) 
(cherry picked from commit 8c1f81d4bf424bdc421cf4952b230344e39a7b68)
(cherry picked from commit 14df17383c20cebcdb27da1f8bce0b494d47b392)
 2016-06-20 14:30:51 -07:00
Colin Patrick Mccabe
c610b60331 HADOOP-13280. FileSystemStorageStatistics#getLong(“readOps“) should return readOps + largeReadOps (Mingliang Liu via cmccabe) 
(cherry picked from commit 5370a6ffaec5227c0978f10c86a5811155271933)
(cherry picked from commit 7db1e57cc160a902a688db7068b6e692d8263a65)

Conflicts:
    hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFileSystemStorageStatistics.java
 2016-06-20 10:53:14 -07:00
Akira Ajisaka
615a023be3 HADOOP-13192. org.apache.hadoop.util.LineReader cannot handle multibyte delimiters correctly. Contributed by binde. 
(cherry picked from commit fc6b50cc574e144fd314dea6c11987c6a384bfa6)
(cherry picked from commit 39ea0891d2b1369ec8c1ea4312489309e1a85227)
 2016-06-20 17:08:19 +09:00
Brahma Reddy Battula
9d27530c36 HDFS-10474. hftp copy fails when file name with Chinese+special char in branch-2 (Contributed by Brahma Reddy Battula) 
(cherry picked from commit 3f27f405039926bbee4b36a60d6cecdb6322fff5)
 2016-06-20 12:40:11 +05:30
Chris Nauroth
55658dcaa3 Revert "HADOOP-12718. Incorrect error message by fs -put local dir without permission. (John Zhuge via Yongjun Zhang)" 
This reverts commit 62206dd2edcc53dbfbae30a524c21e7eb828bd0e.

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
 2016-06-17 23:51:27 -07:00
Andrew Wang
3f2d2cd0a1 Fix hadoop-hdfs-native-client compilation on Windows. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 3ff0510ffdc20e44f06aec591782f8875b686327)
(cherry picked from commit 94759470b8a922d410797b8b76e055fd2fc6dd69)
 2016-06-17 15:55:43 -07:00
Chris Nauroth
c8b0ec5363 HADOOP-13149. Windows distro build fails on dist-copynativelibs. Contributed by Chris Nauroth. 
(cherry picked from commit b9dbef94177dfe0d8ea9526722691f861f47bb4e)
 2016-06-17 15:55:41 -07:00
Kihwal Lee
edd5373095 HDFS-10493. Add links to datanode web UI in namenode datanodes page. Contributed by Weiwei Yang. 
(cherry picked from commit 280069510b71d81a150b2b28d9fa987891d82774)
 2016-06-17 11:13:12 -05:00
Akira Ajisaka
242263c56d HADOOP-12943. Add -w -r options in dfs -test command. Contributed by Weiwei Yang. 
(cherry picked from commit 09e82acaf9a6d7663bc51bbca0cdeca4b582b535)
(cherry picked from commit a36aa920f437f9e21772e8067e5db86fbffe7895)
 2016-06-17 16:21:31 +09:00
Xiaoyu Yao
e84a9c976a HADOOP-13255. KMSClientProvider should check and renew tgt when doing delegation token operations. Contributed by Xiao Chen. 
(cherry picked from commit ddf66427ff92a3886f94954e21d59e998412059b)
 2016-06-16 20:12:55 -07:00
Konstantin V Shvachko
408848d1e9 HADOOP-13189. FairCallQueue makes callQueue larger than the configured capacity. Contributed by Vinitha Gankidi. 2016-06-16 18:33:05 -07:00
Colin Patrick Mccabe
04682cc6b0 HADOOP-12975. Add jitter to CachingGetSpaceUsed's thread (Elliott Clark via Colin P. McCabe) 
(cherry picked from commit bf780406f2b30e627bdf36ac07973f6931f81106)
(cherry picked from commit 56c997a1a61188693ed306b3ceddb13a2f2e6456)
 2016-06-16 17:03:52 -07:00
Colin Patrick Mccabe
b07cd9c6a9 HADOOP-13284. FileSystemStorageStatistics must not attempt to read non-existent rack-aware read stats in branch-2.8 (Mingliang Liu via cmccabe) 2016-06-16 16:45:41 -07:00