Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
8,672 Commits 326 Branches 371 Tags
Commit Graph

8672 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Tsz-Wo Nicholas Sze
657b027bb2 HDFS-7308. Change the packet chunk size computation in DFSOutputStream in order to enforce packet size <= 64kB. Contributed by Takuya Fukudome 2015-02-27 23:46:42 +08:00
Vinayakumar B
bc60404eaf HDFS-6753. Initialize checkDisk when DirectoryScanner not able to get files list for scanning (Contributed by J.Andreina) 
(cherry picked from commit 4f75b15628a76881efc39054612dc128e23d27be)
 2015-02-27 16:37:03 +05:30
Tsz-Wo Nicholas Sze
d223a4a594 Revert "HDFS-7769. TestHDFSCLI should not create files in hdfs project root dir." 
This reverts commit acc172e3718e23ff6808ddcc01543212f1334a27.

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-02-27 18:27:19 +08:00
Tsuyoshi Ozawa
02df51497f HADOOP-11569. Provide Merge API for MapFile to merge multiple similar MapFiles to one MapFile. Contributed by Vinayakumar B. 
(cherry picked from commit 48c7ee7553af94a57952bca03b49c04b9bbfab45)
 2015-02-27 17:46:29 +09:00
Konstantin V Shvachko
9e67f2cb0d YARN-3255. RM, NM, JobHistoryServer, and WebAppProxyServer's main() should support generic options. Contributed by Konstantin Shvachko. 2015-02-26 17:26:08 -08:00
Akira Ajisaka
8aa0c73294 MAPREDUCE-5612. Add javadoc for TaskCompletionEvent.Status. Contributed by Chris Palmer. 
(cherry picked from commit bfbf076b7dcb71b59d85b05d0a6da88195ea9902)
 2015-02-26 15:06:26 -08:00
cnauroth
5dfb793b05 HDFS-7774. Unresolved symbols error while compiling HDFS on Windows 7/32 bit. Contributed by Kiran Kumar M R. 
(cherry picked from commit c6d5b37a1c4c3acc190f5f4e27109594efb7be8c)
 2015-02-26 13:37:58 -08:00
cnauroth
264c96777e HADOOP-9922. hadoop windows native build will fail in 32 bit machine. Contributed by Kiran Kumar M R. 
(cherry picked from commit 2214dab60ff11b8de74c9d661585452a078fe0c1)
 2015-02-26 12:41:48 -08:00
Colin Patrick Mccabe
8b3b9568b6 HDFS-7819. Log WARN message for the blocks which are not in Block ID based layout (Rakesh R via Colin P. McCabe) 
(cherry picked from commit f0c980abed3843923e0eb16b626fa27334195eda)
 2015-02-26 12:03:11 -08:00
Tsuyoshi Ozawa
dc348f4894 YARN-3217. Remove httpclient dependency from hadoop-yarn-server-web-proxy. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 773b6515ac51af3484824bd6f57685a9726a1e70)
 2015-02-27 00:25:12 +09:00
Devaraj K
f83d2e4410 YARN-3256. TestClientToAMTokens#testClientTokenRace is not running against 
all Schedulers even when using ParameterizedSchedulerTestBase. Contributed
by Anubhav Dhoot.

(cherry picked from commit 0d4296f0e0f545267f2e39a868d4ffefc9844db8)
 2015-02-26 15:47:32 +05:30
Vinayakumar B
f0f3d1937d HDFS-7832. Show 'Last Modified' in Namenode's 'Browse Filesystem' (Contributed by Vinayakumar B) 
(cherry picked from commit 166eecf687765f369cd5bf210cc6d7bf493121ea)
 2015-02-26 14:37:03 +05:30
Andrew Wang
0512e50d6e HADOOP-11620. Add support for load balancing across a group of KMS for HA. Contributed by Arun Suresh. 
(cherry picked from commit 71385f9b70e22618db3f3d2b2c6dca3b1e82c317)
 2015-02-25 21:17:44 -08:00
Tsz-Wo Nicholas Sze
22b13a0ddb HDFS-7537. Add "UNDER MIN REPL'D BLOCKS" count to fsck. Contributed by GAO Rui 2015-02-26 11:47:04 +08:00
Tsz-Wo Nicholas Sze
12fe3afcd3 HDFS-7843. A truncated file is corrupted after rollback from a rolling upgrade. 2015-02-26 10:15:49 +08:00
Benoy Antony
4e400030f6 HDFS-7467. Provide storage tier information for a directory via fsck. (Benoy Antony) 2015-02-25 16:46:41 -08:00
Colin Patrick Mccabe
d389a1ae98 HDFS-7495. Remove updatePosition argument from DFSInputStream#getBlockAt() (cmccabe) 
(cherry picked from commit caa42adf208bfb5625d1b3ef665fbf334ffcccd9)
 2015-02-25 13:39:02 -08:00
cnauroth
1bc4c6808d HADOOP-11629. WASB filesystem should not start BandwidthGaugeUpdater if fs.azure.skip.metrics set to true. Contributed by Shanyu Zhao. 
(cherry picked from commit 5731c0e0d08c3048fafdf62a14ca7611be4df5d7)
 2015-02-25 09:09:15 -08:00
Jason Lowe
257087417e YARN-3239. WebAppProxy does not support a final tracking url which has query fragments and params. Contributed by Jian He 
(cherry picked from commit 1a68fc43464d3948418f453bb2f80df7ce773097)
 2015-02-25 16:15:35 +00:00
Tsuyoshi Ozawa
75fd43be02 HADOOP-11632. Cleanup Find.java to remove SupressWarnings annotations. Contributed by Akira AJISAKA. 
(cherry picked from commit ad8ed3e802782a7a3fb3d21c5862673a8f695372)
 2015-02-25 16:25:21 +09:00
Tsuyoshi Ozawa
13e213c41a YARN-3247. TestQueueMappings should use CapacityScheduler explicitly. Contributed by Zhihai Xu. 
(cherry picked from commit 6cbd9f1113fca9ff86fd6ffa783ecd54b147e0db)
 2015-02-25 11:23:34 +09:00
Akira Ajisaka
e97613b429 HADOOP-11480. Typo in hadoop-aws/index.md uses wrong scheme for test.fs.s3.name. Contributed by Ted Yu. 
(cherry picked from commit 1a625b8158ab1cf765fbda962ba725503409d9fe)
 2015-02-24 17:13:35 -08:00
Colin Patrick Mccabe
5c0073e706 HDFS-7668. Backport "Convert site documentation from apt to markdown" to branch-2 (Masatake Iwasaki via Colin P. McCabe) 
(backported from commit 2f1e5dc6288972004b5bed335c4a8d038aaedcf4)

Conflicts:
     hadoop-hdfs-project/hadoop-hdfs/src/site/apt/ExtendedAttributes.apt.vm
     hadoop-hdfs-project/hadoop-hdfs/src/site/apt/Federation.apt.vm
     hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HDFSCommands.apt.vm
     hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HDFSHighAvailabilityWithNFS.apt.vm
     hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HDFSHighAvailabilityWithQJM.apt.vm
     hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HdfsDesign.apt.vm
     hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HdfsNfsGateway.apt.vm
     hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HdfsQuotaAdminGuide.apt.vm
 2015-02-24 16:18:18 -08:00
Colin Patrick Mccabe
343cffb0ea HADOOP-11495. Backport "convert site documentation from apt to markdown" to branch-2 (Masatake Iwasaki via Colin P. McCabe) 
(cherry-picked from commit b6fc1f3e4355be913b7d4f6ccd48c0c26b66d039)

Conflicts:
    hadoop-common-project/hadoop-common/src/site/apt/ClusterSetup.apt.vm
    hadoop-common-project/hadoop-common/src/site/apt/CommandsManual.apt.vm
    hadoop-common-project/hadoop-common/src/site/apt/FileSystemShell.apt.vm
    hadoop-common-project/hadoop-common/src/site/apt/RackAwareness.apt.vm
    hadoop-common-project/hadoop-common/src/site/apt/SingleCluster.apt.vm
    hadoop-common-project/hadoop-common/src/site/apt/FileSystemShell.apt.vm
    hadoop-common-project/hadoop-common/src/site/apt/SecureMode.apt.vm
    hadoop-common-project/hadoop-common/src/site/apt/Tracing.apt.vm
    hadoop-project/src/site/site.xml
 2015-02-24 15:48:58 -08:00
Andrew Wang
efb7e287f4 HDFS-7763. fix zkfc hung issue due to not catching exception in a corner case. Contributed by Liang Xie. 
(cherry picked from commit 7105ebaa9f370db04962a1e19a67073dc080433b)
 2015-02-24 15:31:18 -08:00
Akira Ajisaka
5f0d1f10db MAPREDUCE-2815. JavaDoc does not generate correctly for MultithreadedMapRunner. Contributed by Chris Palmer. 
(cherry picked from commit 9a37247a6747d5d1a8749750698a1ae2f7b68414)
 2015-02-24 12:11:22 -08:00
Akira Ajisaka
5e686dae44 MAPREDUCE-6264. Remove httpclient dependency from hadoop-mapreduce-client. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 3f665aa4d5ac2e737e29ff63b3837c083a49ea80)
 2015-02-24 11:33:05 -08:00
Jing Zhao
8346427929 HDFS-7831. Fix the starting index and end condition of the loop in FileDiffList.findEarlierSnapshotBlocks(). Contributed by Konstantin Shvachko. 
(cherry picked from commit 73bcfa99af61e5202f030510db8954c17cba43cc)
 2015-02-24 10:34:42 -08:00
Tsuyoshi Ozawa
280dda5bcd HDFS-7008. xlator should be closed upon exit from DFSAdmin#genericRefresh(). (ozawa) 
(cherry picked from commit b53fd7163bc3a4eef4632afb55e5513c7c592fcf)
 2015-02-24 23:59:59 +09:00
Devaraj K
b9e82e27f5 YARN-2797. Add -help to yarn logs and nodes CLI command. Contributed by 
Jagadesh Kiran N.

(cherry picked from commit b610c68d4423a5a1ab342dc490cd0064f8983c07)
 2015-02-24 12:08:50 +05:30
Gera Shegalov
62e0e67d49 HADOOP-11619. FTPFileSystem should override getDefaultPort. (Brahma Reddy Battula via gera) 
(cherry picked from commit 1dba57271fa56a7383139deb0b89a61c58eedf25)
 2015-02-23 21:19:57 -08:00
Colin Patrick Mccabe
453b210d0a HDFS-7805. NameNode recovery prompt should be printed on console (Surendra Singh Lilhore via Colin P. McCabe) 
(cherry picked from commit faaddb6ecb44cdc9ef82a2ab392f64fc2561e938)
 2015-02-23 19:06:01 -08:00
Colin Patrick Mccabe
f4da6f7193 HDFS-7807. libhdfs htable.c: fix htable resizing, add unit test (cmccabe) 
(cherry picked from commit 585768667e443f56c2f97068276ec8768dc49cf8)
 2015-02-23 18:45:44 -08:00
Akira Ajisaka
6bbb051d38 HADOOP-8642. Document that io.native.lib.available only controls native bz2 and zlib compression codecs. (aajisaka) 
(cherry picked from commit ab5976161f3afaaf2ace60bab400e0d8dbc61923)
 2015-02-23 17:35:46 -08:00
Haohui Mai
92acb0e4f4 HADOOP-10478. Fix new findbugs warnings in hadoop-maven-plugins. Contributed by Li Lu. 2015-02-23 16:38:10 -08:00
cnauroth
657a6e389b HDFS-7009. Active NN and standby NN have different live nodes. Contributed by Ming Ma. 
(cherry picked from commit 769507bd7a501929d9a2fd56c72c3f50673488a4)
 2015-02-23 15:12:36 -08:00
Xuan
0ca974d0b0 YARN-2797. TestWorkPreservingRMRestart should use 
ParametrizedSchedulerTestBase. Contributed by Karthik Kambatla

(cherry picked from commit fe7a302473251b7310105a936edf220e401c613f)
 2015-02-21 19:19:08 -08:00
Xuan
f8435c5d3d YARN-3236. Cleanup RMAuthenticationFilter#AUTH_HANDLER_PROPERTY. 
Contributed by zhihai xu

(cherry picked from commit e3d290244c8a39edc37146d992cf34e6963b6851)
 2015-02-21 16:20:33 -08:00
Xuan
fefeba4ac8 YARN-3238. Connection timeouts to nodemanagers are retried at multiple 
levels. Contributed by Jason Lowe

(cherry picked from commit 92d67ace3248930c0c0335070cc71a480c566a36)
 2015-02-21 16:07:28 -08:00
Arpit Agarwal
83f779e49a HDFS-7806. Refactor: move StorageType from hadoop-hdfs to hadoop-common. (Contributed by Xiaoyu Yao) 2015-02-21 15:48:59 -08:00
Steve Loughran
a36cad0522 HADOOP-11584 s3a file block size set to 0 in getFileStatus. (Brahma Reddy Battula via stevel) 2015-02-21 12:02:41 +00:00
yliu
eca1588db8 HDFS-7740. Test truncate with DataNodes restarting. (yliu) 2015-02-21 06:33:58 +08:00
Robert Kanter
199a853ed7 HADOOP-11612. Workaround for Curator's ChildReaper requiring Guava 15+. (rkanter) 
(cherry picked from commit 6f0133039a064ca82363ac6f29fb255506f31b8a)
 2015-02-20 19:51:35 -08:00
cnauroth
9f06fb3aef HDFS-7813. TestDFSHAAdminMiniCluster#testFencer testcase is failing frequently. Contributed by Rakesh R. 
(cherry picked from commit 0d6af574e0056fc627461eb91ed0c365b026b470)
 2015-02-20 17:01:22 -08:00
Xuan
9cbe9bcbaa YARN-3237. AppLogAggregatorImpl fails to log error cause. Contributed by 
Rushabh S Shah

(cherry picked from commit f56c65bb3eb9436b67de2df63098e26589e70e56)
 2015-02-20 14:03:46 -08:00
cnauroth
187e081d5a HADOOP-11604. Prevent ConcurrentModificationException while closing domain sockets during shutdown of DomainSocketWatcher thread. Contributed by Chris Nauroth. 
(cherry picked from commit 3c5ff0759c4f4e10c97c6d9036add00edb8be2b5)
 2015-02-20 13:07:36 -08:00
Steve Loughran
4276f8b315 HADOOP-11607. Reduce log spew in S3AFileSystem. (Lei (Eddy) Xu via stevel) 2015-02-20 20:51:00 +00:00
cnauroth
e09ba94f28 HDFS-7773. Additional metrics in HDFS to be accessed via jmx. Contributed by Anu Engineer. 2015-02-20 12:38:34 -08:00
cnauroth
ad77dfc3e4 HDFS-7814. Fix usage string of storageType parameter for "dfsadmin -setSpaceQuota/clrSpaceQuota". Contributed by Xiaoyu Yao. 
(cherry picked from commit 8c6ae0d6199efa327d8f2761f2ad2163a60e5508)
 2015-02-20 12:22:00 -08:00
Wangda Tan
a1963968d2 YARN-3230. Clarify application states on the web UI. (Jian He via wangda) 
(cherry picked from commit ce5bf927c3d9f212798de1bf8706e5e9def235a1)
 2015-02-20 10:40:04 -08:00