Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
15,936 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

672 Commits

Author SHA1 Message Date
Steve Loughran 025219b12f HADOOP-12982 Document missing S3A and S3 properties. (Wei-Chiu Chuang via stevel) 2016-05-10 21:37:22 +01:00
Jian He 996a210ab0 YARN-4778. Support specifying resources for task containers in SLS. Contributed by Wangda Tan 2016-05-09 14:49:41 -07:00
Allen Wittenauer 1c5bbf6499 HADOOP-12866. add a subcommand for gridmix (Kai Sasaki via aw) 2016-05-06 14:07:02 -07:00
Jason Lowe d0da13229c YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla 2016-05-05 14:07:54 +00:00
Jing Zhao af942585a1 HADOOP-12469. distcp should not ignore the ignoreFailures option. Contributed by Mingliang Liu. 2016-05-04 10:23:04 -07:00
Masatake Iwasaki ef0870ad03 HADOOP-12378. Fix findbugs warnings in hadoop-tools module. Contributed by Akira AJISAKA. 2016-04-29 20:25:52 +09:00
Arun Suresh 341888a0aa YARN-4412. Create ClusterMonitor to compute ordered list of preferred NMs for OPPORTUNITIC containers. (asuresh) 2016-04-26 20:12:12 -07:00
Yongjun Zhang 959a28dd12 HDFS-10313. Distcp need to enforce the order of snapshot names passed to -diff. (Lin Yiqun via Yongjun Zhang) 2016-04-26 16:08:03 -07:00
Akira Ajisaka 02c51c27d9 HDFS-10298. Document the usage of distcp -diff option. Contributed by Takashi Ohnishi. 2016-04-25 22:33:09 +09:00
Jing Zhao 63e5412f1a HDFS-9427. HDFS should not default to ephemeral ports. Contributed by Xiaobing Zhou. 2016-04-22 15:14:40 -07:00
Steve Loughran 19f0f9608e HADOOP-12891. S3AFileSystem should configure Multipart Copy threshold and chunk size. (Andrew Olson via stevel) 2016-04-22 11:25:03 +01:00
Yongjun Zhang a749ba0cea HDFS-9670. DistCp throws NPE when source is root. (John Zhuge via Yongjun Zhang) 2016-04-21 12:17:17 -07:00
Jing Zhao 404f57f328 HDFS-10216. Distcp -diff throws exception when handling relative path. Contributed by Takashi Ohnishi. 2016-04-14 10:35:22 -07:00
Steve Loughran df18b6e984 HADOOP-12963 Allow using path style addressing for accessing the s3 endpoint. (Stephen Montgomery via stevel) 2016-04-14 12:44:55 +01:00
Jason Lowe 2a5da97f81 Revert "YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla" 
This reverts commit 1cbcd4a491.
 2016-04-11 15:51:01 +00:00
Steve Loughran b9e3eff62a HADOOP-12444 Support lazy seek in S3AInputStream. Rajesh Balamohan via stevel 2016-04-09 11:25:31 +01:00
Chris Nauroth 843ee8d59d HADOOP-12994. Specify PositionedReadable, add contract tests, fix problems. Contributed by Steve Loughran. 2016-04-08 13:36:58 -07:00
Steve Loughran ce5b4812f0 HADOOP-12753 S3A JUnit tests failing if using HTTP proxy. (Zoran Rajic via stevel) 2016-04-08 17:10:00 +01:00
Akira Ajisaka 9c32f8785e HDFS-10266. Remove unused properties dfs.client.file-block-storage-locations.num-threads and dfs.client.file-block-storage-locations.timeout.millis. Contributed by Ray Chiang. 2016-04-08 15:32:13 +09:00
Jason Lowe 1cbcd4a491 YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla 2016-04-05 13:40:19 +00:00
Steve Loughran 89c9347511 HADOOP-12169 ListStatus on empty dir in S3A lists itself instead of returning an empty list. author: Pieter Reuse. - omitted new S3A subclass 2016-04-04 17:02:04 +01:00
Steve Loughran 0ecdd4cffa HADOOP-12169 ListStatus on empty dir in S3A lists itself instead of returning an empty list. author: Pieter Reuse. 2016-04-03 16:40:19 +01:00
Harsh J 256c82fe29 HADOOP-11687. Ignore x-* and response headers when copying an Amazon S3 object. Contributed by Aaron Peterson and harsh. 2016-04-01 14:18:10 +05:30
Allen Wittenauer 0a74610d1c HADOOP-11393. Revert HADOOP_PREFIX, go back to HADOOP_HOME (aw) 2016-03-31 07:51:05 -07:00
Akira Ajisaka 0050fa5f1c MAPREDUCE-6543. Migrate MR client test cases part 2. Contributed by Dustin Cote. 2016-03-29 18:17:52 +09:00
Akira Ajisaka 115be193df Revert "MAPREDUCE-6543. Migrate MR client test cases part 2. Contributed by Dustin Cote." 
This reverts commit 2c268cc936.
 2016-03-28 16:05:35 +09:00
Akira Ajisaka 18c7e58283 HDFS-9640. Remove hsftp from DistCp in trunk. Contributed by Wei-Chiu Chuang. 2016-03-28 15:32:38 +09:00
Akira Ajisaka 2c268cc936 MAPREDUCE-6543. Migrate MR client test cases part 2. Contributed by Dustin Cote. 2016-03-25 15:17:27 +09:00
Allen Wittenauer 738155063e HADOOP-12857. rework hadoop-tools (aw) 2016-03-23 13:46:38 -07:00
Chris Nauroth c50aad0f85 HADOOP-12717. NPE when trying to rename a directory in Windows Azure Storage FileSystem. Contributed by Robert Yokota and Gaurav Kanade. 2016-03-04 22:57:43 -08:00
Masatake Iwasaki cbd31328a6 HADOOP-12470. In-page TOC of documentation should be automatically generated by doxia macro (iwasakims) 2016-03-04 14:11:36 +09:00
Masatake Iwasaki 33a412e8a4 HDFS-9048. DistCp documentation is out-of-dated (Daisuke Kobayashi via iwasakims) 2016-03-03 18:57:23 +09:00
Chris Nauroth f98dff329b HADOOP-12853. Change WASB documentation regarding page blob support. Contributed by Madhumita Chakraborty. 2016-03-01 13:18:42 -08:00
Chris Nauroth d251e55415 HADOOP-12851. S3AFileSystem Uptake of ProviderUtils.excludeIncompatibleCredentialProviders. Contributed by Larry McCay. 2016-02-29 21:59:52 -08:00
Masatake Iwasaki 809ebc0b14 HADOOP-12552. Fix undeclared/unused dependency to httpclient (iwasakims) 2016-02-29 18:20:14 +09:00
Chris Nauroth 7634d404b7 HADOOP-12846. Credential Provider Recursive Dependencies. Contributed by Larry McCay. 2016-02-28 11:22:55 -08:00
cnauroth 3369a4f691 HADOOP-12535. Run FileSystem contract tests with hadoop-azure. Contributed by Madhumita Chakraborty. 2016-02-23 21:37:50 -08:00
cnauroth 27b77751c1 HADOOP-12555. WASB to read credentials from a credential provider. Contributed by Larry McCay. 2016-02-22 17:30:13 -08:00
cnauroth d4f5fc23b2 HADOOP-11613. Remove commons-httpclient dependency from hadoop-azure. Contributed by Masatake Iwasaki. 2016-02-22 13:39:38 -08:00
Yongjun Zhang ba1c9d484a HDFS-9764. DistCp doesn't print value for several arguments including -numListstatusThreads. (Wei-Chiu Chuang via Yongjun Zhang) 2016-02-19 10:17:37 -08:00
cnauroth 8ab7658025 HADOOP-11031. Design Document for Credential Provider API. Contributed by Larry McCay. 2016-02-18 14:06:38 -08:00
cnauroth 76fab26c5c HADOOP-12548. Read s3a creds from a Credential Provider. Contributed by Larry McCay. 2016-02-17 12:19:59 -08:00
cnauroth 91a96eaa53 HADOOP-12780. During WASB atomic rename handle crash when one directory has been renamed but not file under it. Contributed by Madhumita Chakraborty. 2016-02-12 15:50:10 -08:00
Steve Loughran 29ae258013 HADOOP-12292. Make use of DeleteObjects optional. (Thomas Demoor via stevel) 2016-02-06 15:05:16 +00:00
Akira Ajisaka 700a176fa7 HADOOP-12757. Findbug compilation fails for 'Kafka Library support'. (aajisaka) 2016-02-02 23:15:39 +09:00
Yongjun Zhang eddd823cd6 HDFS-9638. Improve DistCp Help and documentation. (Wei-Chiu Chuang via Yongjun Zhang) 2016-01-29 12:11:55 -08:00
cnauroth 41da9a0773 MAPREDUCE-6563. Streaming documentation contains a stray % character. Contributed by Chris Nauroth. 2016-01-28 14:46:01 -08:00
Xuan 4efdf3a979 YARN-4612. Fix rumen and scheduler load simulator handle killed tasks 
properly. Contributed by Ming Ma.
 2016-01-26 18:17:12 -08:00
Akira Ajisaka 736eb17a79 HADOOP-12731. Remove useless boxing/unboxing code. Contributed by Kousuke Saruta. 2016-01-25 13:47:29 +09:00
Akira Ajisaka 0bae506c22 HADOOP-12730. Hadoop streaming -mapper and -reducer options are wrongly documented as required. Contributed by Kengo Seki. 2016-01-22 21:08:31 +09:00
Xuan 1708a4cd23 YARN-4611. Fix scheduler load simulator to support multi-layer network 
location. Contributed by Ming Ma.
 2016-01-20 15:41:50 -08:00
Ravi Prakash 1acc509b45 HADOOP-12696. Add tests for S3FileSystem Contract. Contributed by Matt Paduano 2016-01-19 13:57:08 -08:00
Wangda Tan 89d1fd5dac HADOOP-12356. Fix computing CPU usage statistics on Windows. (Inigo Goiri via wangda) 2016-01-19 21:27:38 +08:00
Wangda Tan a44ce3f14f YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda) 2016-01-19 09:30:04 +08:00
Wangda Tan 150f5ae034 Revert "YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda)" 
This reverts commit 3fe5728563.

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2016-01-19 09:27:36 +08:00
cnauroth 8bc93db2e7 HADOOP-12635. Adding Append API support for WASB. Contributed by Dushyanth. 2016-01-18 09:08:58 -08:00
Wangda Tan 3fe5728563 YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda) 
(cherry picked from commit 805a9ed85e)
 2016-01-18 17:06:05 +08:00
Wangda Tan adf260a728 Revert "YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda)" 
This reverts commit 805a9ed85e.
 2016-01-18 16:50:45 +08:00
Wangda Tan 805a9ed85e YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda) 2016-01-18 11:04:25 +08:00
Yongjun Zhang a9c69ebeb7 HDFS-9612. DistCp worker threads are not terminated after jobs are done. (Wei-Chiu Chuang via Yongjun Zhang) 2016-01-15 10:03:09 -08:00
Colin Patrick Mccabe b1ed28fa77 HADOOP-8887. Use a Maven plugin to build the native code using CMake (cmccabe) 2016-01-14 11:02:34 -08:00
Colin Patrick Mccabe 8315582c4f HDFS-9517. Fix missing @Test annotation on TestDistCpUtils.testUnpackAttributes (Wei-Chiu Chuang via cmccabe) 2016-01-13 16:28:06 -08:00
Lei Xu 126705f67e HADOOP-11262. Enable YARN to use S3A. (Pieter Reuse via lei) 2016-01-12 12:19:53 -08:00
Xiaoyu Yao c2e2e13455 HDFS-8584. NPE in distcp when ssl configuration file does not exist in class path. Contributed by Surendra Singh Lilhore. 2016-01-11 17:08:26 -08:00
Zhe Zhang 95f32015ad HDFS-9630. DistCp minor refactoring and clean up. Contributed by Kai Zheng. 
Change-Id: I363c4ffcac32116ddcdc0a22fac3db92f14a0db0
 2016-01-11 09:46:56 -08:00
cnauroth 0e76f1fcea HADOOP-12551. Introduce FileNotFoundException for WASB FileSystem API. Contributed by Dushyanth. 2016-01-09 22:18:11 -08:00
cnauroth f0fa6d869b HADOOP-12678. Handle empty rename pending metadata file during atomic rename in redo path. Contributed by Madhumita Chakraborty. 2016-01-08 20:18:59 -08:00
cnauroth 978bbdfeb2 HADOOP-12634. Change Lazy Rename Pending Operation Completion of WASB to address case of potential data loss due to partial copy. Contributed by Gaurav Kanade. 2016-01-06 11:15:59 -08:00
Ravi Prakash 2d16f40dab HADOOP-12689. S3 filesystem operations stopped working correctly 2016-01-05 23:26:03 -08:00
Wangda Tan a9594c61bb YARN-4510. Fix SLS startup failure caused by NPE. (Bibin A Chundatt via wangda) 
(cherry picked from commit 6eefae1b33)
 2015-12-30 15:39:27 -08:00
Wangda Tan 8310b2e9ff YARN-4522. Queue acl can be checked at app submission. (Jian He via wangda) 2015-12-30 15:30:12 -08:00
Wangda Tan 79c41b1d83 YARN-4293. ResourceUtilization should be a part of yarn node CLI. (Sunil G via wangda) 2015-12-16 13:18:19 -08:00
Robert Kanter 6d84cc16b3 MAPREDUCE-6550. archive-logs tool changes log ownership to the Yarn user when using DefaultContainerExecutor (rkanter) 2015-11-25 17:12:40 -08:00
Akira Ajisaka bab03f3ee6 HADOOP-12598. Add XML namespace declarations for some hadoop/tools modules. Contributed by Xin Wang. 2015-11-26 01:21:20 +09:00
Karthik Kambatla 52948bb20b YARN-3980. Plumb resource-utilization info in node heartbeat through to the scheduler. (Inigo Goiri via kasha) 2015-11-24 13:47:17 +05:30
Haohui Mai dc46c46b91 HADOOP-10465. Fix use of generics within SortedMapWritable. Contributed by Bertrand Dechoux. 2015-11-22 18:10:08 -08:00
Haohui Mai 0d27917042 HADOOP-12098. Remove redundant test dependencies in Hadoop Archives. Contributed by Varun Saxena. 2015-11-22 17:25:22 -08:00
Haohui Mai 176ff5ce90 HDFS-9402. Switch DataNode.LOG to use slf4j. Contributed by Walter Su. 2015-11-22 15:54:41 -08:00
Akira Ajisaka f94d892704 HADOOP-12545. Hadoop javadoc has broken links for AccessControlList, ImpersonationProvider, DefaultImpersonationProvider, and DistCp. Contributed by Arshad Mohammad. 2015-11-13 21:52:57 +09:00
Akira Ajisaka 3ae82863f0 HADOOP-12553. [JDK8] Fix javadoc error caused by illegal tag. (aajisaka) 2015-11-08 19:17:57 +09:00
Lei Xu bff7c90a56 HADOOP-11684. S3a to use thread pool that blocks clients. (Thomas Demoor and Aaron Fabbri via lei) 2015-11-05 18:35:15 -08:00
Allen Wittenauer 19a0c2660c MAPREDUCE-1125. SerialUtils.cc: deserializeFloat is out of sync with SerialUtils.hh (Simone Leo via aw) 2015-11-05 16:57:22 -08:00
Allen Wittenauer fc7cd46faf MAPREDUCE-3914. Mismatched free() / delete / delete [] in HadoopPipes (Joe Mudd via aw) 2015-11-05 16:53:32 -08:00
cnauroth 0fb1867fd6 HADOOP-12540. TestAzureFileSystemInstrumentation#testClientErrorMetrics fails intermittently due to assumption that a lease error will be thrown. Contributed by Gaurav Kanade. 2015-11-04 10:28:44 -08:00
Varun Vasudev 73b9c7b82b HADOOP-10787. Rename/remove non-HADOOP_*, etc from the shell scripts. Contributed by Allen Wittenauer. 2015-11-04 15:56:17 +05:30
cnauroth 9e7dcab185 HADOOP-12508. delete fails with exception when lease is held on blob. Contributed by Gaurav Kanade. 2015-11-02 10:21:39 -08:00
cnauroth 3ce0a6502e HADOOP-12533. Introduce FileNotFoundException in WASB for read and seek API. Contributed by Dushyanth. 2015-11-02 10:17:41 -08:00
Kihwal Lee b24fe06483 Addendum to MAPREDUCE-6451 2015-10-30 16:05:23 -05:00
Kihwal Lee 2868ca0328 MAPREDUCE-6451. DistCp has incorrect chunkFilePath for multiple jobs when strategy is dynamic. Contributed by Kuhu Shukla. 2015-10-30 14:56:41 -05:00
Gábor Lipták 18727c63da HADOOP-12133 Add schemas to Maven Assembly XMLs 
Signed-off-by: Allen Wittenauer <aw@apache.org>
 2015-10-30 11:36:52 -07:00
cnauroth 2d10cb8e00 HADOOP-12519. hadoop-azure tests should avoid creating a metrics configuration fiale in the module root directory. Contributed by Chris Nauroth. 2015-10-28 09:44:10 -07:00
cnauroth 1f7ecb0c84 HADOOP-11685. StorageException complaining " no lease ID" during HBase distributed log splitting. Contributed by Duo Xu. 2015-10-27 22:56:22 -07:00
cnauroth 73822de7c3 HADOOP-12520. Use XInclude in hadoop-azure test configuration to isolate Azure Storage account keys for service integration tests. Contributed by Chris Nauroth. 2015-10-27 22:48:56 -07:00
Varun Vasudev c28e16b40c YARN-4302. SLS not able start due to NPE in SchedulerApplicationAttempt. Contributed by Bibin A Chundatt. 2015-10-27 20:56:00 +05:30
Akira Ajisaka bcb2386e39 HADOOP-12515. Mockito dependency is missing in hadoop-kafka module. Contributed by Kai Zheng. 2015-10-27 20:17:16 +09:00
cnauroth cb282d5b89 HADOOP-12484. Single File Rename Throws Incorrectly In Potential Race Condition Scenarios. Contributed by Gaurav Kanade. 2015-10-22 14:29:57 -07:00
cnauroth 47641fcbc9 HADOOP-12334. Change Mode Of Copy Operation of HBase WAL Archiving to bypass Azure Storage Throttling after retries. Contributed by Gaurav Kanade. 2015-10-22 12:21:32 -07:00
Robert Kanter 0c4af0f998 MAPREDUCE-6495. Docs for archive-logs tool (rkanter) 2015-10-20 17:34:34 -07:00
Haohui Mai da8441d0fe HDFS-9167. Update pom.xml in other modules to depend on hdfs-client instead of hdfs. Contributed by Mingliang Liu. 2015-10-13 15:24:24 -07:00
Haohui Mai 7e2c971fed Revert "HADOOP-12469. distcp shout not ignore the ignoreFailures option. Contributed by Mingliang Liu." 
This reverts commit fa5a54cac7.
 2015-10-10 03:24:59 -07:00
Haohui Mai fa5a54cac7 HADOOP-12469. distcp shout not ignore the ignoreFailures option. Contributed by Mingliang Liu. 2015-10-09 22:38:06 -07:00
Robert Kanter 1bca1bb517 MAPREDUCE-6503. archive-logs tool should use HADOOP_PREFIX instead of HADOOP_HOME (rkanter) 2015-10-06 16:49:50 -07:00
cnauroth 5f6edb30c2 HADOOP-12350. WASB Logging: Improve WASB Logging around deletes, reads and writes. Contributed by Dushyanth. 2015-10-05 20:11:41 -07:00
Allen Wittenauer b59e434116 HADOOP-10949. metrics2 sink plugin for Apache Kafka (Babak Behzad via aw) 2015-10-05 14:02:07 -07:00
Akira Ajisaka ecbfd68974 HADOOP-10296. Incorrect null check in SwiftRestClient#buildException(). Contributed by Rahul Palamuttam and Kanaka Kumar Avvaru. 2015-10-01 18:25:31 +02:00
Robert Kanter 5db371f52f MAPREDUCE-6494. Permission issue when running archive-logs tool as different users (rkanter) 2015-09-30 17:33:53 -07:00
Kihwal Lee 3abbdc929b HADOOP-12446. Undeprecate createNonRecursive(). Contributed by Ted Yu. 2015-09-28 17:31:51 -05:00
Harsh J 66dad854c0 MAPREDUCE-6471. Document distcp incremental copy. Contributed by Neelesh Srinivas Salian. 2015-09-28 13:12:43 +05:30
cnauroth 7fe521b1dd HADOOP-11918. Listing an empty s3a root directory throws FileNotFound. Contributed by Lei (Eddy) Xu. 2015-09-25 22:33:53 -07:00
Robert Kanter d3c49e7662 MAPREDUCE-6480. archive-logs tool may miss applications (rkanter) 2015-09-25 15:02:42 -07:00
Jian He 89cab1ba5f YARN-1651. CapacityScheduler side changes to support container resize. Contributed by Wangda Tan 2015-09-23 13:29:38 -07:00
Harsh J cc2b473990 MAPREDUCE-5045. UtilTest#isCygwin method appears to be unused. Contributed by Neelesh Srinivas Salian. 2015-09-22 21:38:06 +05:30
Tsuyoshi Ozawa dfd807afab HADOOP-12428. Fix inconsistency between log-level guards and statements. Contributed by Jagadesh Kiran N and Jackie Chang. 2015-09-22 12:54:29 +09:00
Steve Loughran 7269906254 HADOOP-12087. [JDK8] Fix javadoc errors caused by incorrect or illegal tags. (Akira AJISAKA via stevel). 2015-09-13 14:25:26 +01:00
Karthik Kambatla 119cc75e7e MAPREDUCE-6415. Create a tool to combine aggregated logs into HAR files. (Robert Kanter via kasha) 2015-09-09 17:45:19 -07:00
Tsuyoshi Ozawa 355eaaa33d HADOOP-10318. Incorrect reference to nodeFile in RumenToSLSConverter error message. Contributed by Wei Yan. 2015-09-03 22:48:53 +09:00
Lei Xu 6ab2d19f5c HADOOP-12346. Increase some default timeouts / retries for S3a connector. (Sean Mackrory via Lei (Eddy) Xu) 2015-08-29 09:59:30 -07:00
Rohith Sharma K S feaf034994 YARN-3896. RMNode transitioned from RUNNING to REBOOTED because its response id has not been reset synchronously. (Jun Gong via rohithsharmaks) 2015-08-24 11:25:07 +05:30
Yongjun Zhang 0bc15cb6e6 HDFS-8828. Utilize Snapshot diff report to build diff copy list in distcp. (Yufei Gu via Yongjun Zhang) 2015-08-20 08:02:54 -07:00
cnauroth 3da0bedaec HADOOP-12304. Applications using FileContext fail with the default file system configured to be wasb/s3/etc. Contributed by Chris Nauroth. 2015-08-05 14:56:41 -07:00
Lei Xu d5403747b5 HADOOP-12269. Update aws-sdk dependency to 1.10.6 (Thomas Demoor via Lei (Eddy) Xu) 2015-08-04 18:51:52 -07:00
cnauroth efa97243ec HADOOP-12239. StorageException complaining " no lease ID" when updating FolderLastModifiedTime in WASB. Contributed by Duo Xu. 2015-07-22 11:16:49 -07:00
Vinayakumar B 94c6a4aa85 HADOOP-12017. Hadoop archives command should use configurable replication factor when closing (Contributed by Bibin A Chundatt) 2015-07-22 10:25:49 +05:30
Allen Wittenauer 68d1f4bfe8 HADOOP-11762. Enable swift distcp to secure HDFS (Chen He via aw) 2015-07-21 11:19:29 -07:00
Wangda Tan c39ca541f4 YARN-2003. Support for Application priority : Changes in RM and Capacity Scheduler. (Sunil G via wangda) 2015-07-21 09:57:23 -07:00
Vinayakumar B 87f29c6b8a HADOOP-12214. Parse 'HadoopArchive' commandline using cli Options. (Contributed by Vinayakumar B) 2015-07-21 13:12:46 +05:30
Steve Loughran 05130e94c5 HADOOP-12235 hadoop-openstack junit & mockito dependencies should be "provided". (Ted Yu via stevel) 2015-07-20 11:22:39 +01:00
Chris Douglas a431ed9075 HADOOP-12211. Collect disk usage on the node. Contributed by Robert Grandl 2015-07-13 15:36:11 -07:00
Ming Ma 08244264c0 YARN-3445. Cache runningApps in RMNode for getting running apps on given NodeId. (Junping Du via mingma) 2015-07-10 08:30:10 -07:00
Chris Douglas 1a0752d85a HADOOP-12210. Collect network usage on the node. Contributed by Robert Grandl 2015-07-09 17:48:43 -07:00
Robert Kanter aaafa0b2ee HADOOP-12159. Move DistCpUtils#compareFs() to org.apache.hadoop.fs.FileUtil and fix for HA namespaces (rchiang via rkanter) 2015-06-30 16:49:27 -07:00
cnauroth 460e98f7b3 HADOOP-12089. StorageException complaining " no lease ID" when updating FolderLastModifiedTime in WASB. Contributed by Duo Xu. 2015-06-29 13:48:02 -07:00
Colin Patrick Mccabe 5a27c3fd76 HADOOP-12104. Migrate Hadoop Pipes native build to new CMake framework (Alan Burlison via Colin P. McCabe) 2015-06-29 12:31:26 -07:00
Amareshwari Sriramadasu 8ef07f767f HADOOP-11203. Allow ditscp to accept bandwitdh in fraction MegaBytes. Contributed by Raju Bairishetti 2015-06-26 09:52:06 +05:30
cnauroth c45784bc90 HADOOP-12073. Azure FileSystem PageBlobInputStream does not return -1 on EOF. Contributed by Ivan Mitic. 2015-06-08 22:42:14 -07:00
Andrew Wang 2dbc40e608 HADOOP-12059. S3Credentials should support use of CredentialProvider. Contributed by Sean Busbey. 2015-06-05 13:11:01 -07:00
Robert Kanter cc70df98e7 MAPREDUCE-5965. Hadoop streaming throws error if list of input files is high. Error is: "error=7, Argument list too long at if number of input file is high" (wilfreds via rkanter) 2015-06-03 18:41:45 -07:00
cnauroth 94e7d49a6d HADOOP-11959. WASB should configure client side socket timeout in storage client blob request options. Contributed by Ivan Mitic. 2015-05-28 12:31:06 -07:00
Colin Patrick Mccabe 7dba7005b7 HADOOP-11969. ThreadLocal initialization in several classes is not thread safe (Sean Busbey via Colin P. McCabe) 2015-05-26 12:15:46 -07:00
Vinod Kumar Vavilapalli 500a1d9c76 YARN-160. Enhanced NodeManager to automatically obtain cpu/memory values from underlying OS when configured to do so. Contributed by Varun Vasudev. 2015-05-26 11:38:35 -07:00
Colin Patrick Mccabe f34638399a HADOOP-11927. Fix "undefined reference to dlopen" error when compiling libhadooppipes (Xianyin Xin via Colin P. McCabe) 2015-05-22 12:51:03 -07:00
Akira Ajisaka 4aa730ce85 HADOOP-11698. Remove DistCpV1 and Logalyzer. Contributed by Brahma Reddy Battula. 2015-05-20 19:25:45 +09:00
Jing Zhao 0790275f05 HADOOP-1540. Support file exclusion list in distcp. Contributed by Rich Haase. 2015-05-18 13:24:35 -07:00
Akira Ajisaka f5c48238d5 HADOOP-11939. Deprecate DistCpV1 and Logalyzer. Contributed by Brahma Reddy Battula. 2015-05-18 16:13:16 +09:00
cnauroth cb8e69a80c HADOOP-11960. Enable Azure-Storage Client Side logging. Contributed by Dushyanth. 2015-05-14 22:22:24 -07:00
Akira Ajisaka 92c38e41e1 HADOOP-9723. Improve error message when hadoop archive output path already exists. Contributed by Jean-Baptiste Onofré and Yongjun Zhang. 2015-05-13 17:28:57 +09:00
Allen Wittenauer ed0f4db0dd HADOOP-11775. Fix Javadoc typos in hadoop-openstack module (Yanjun Wang via aw) 2015-05-08 17:01:08 -07:00
Akira Ajisaka c5afe444df HDFS-8349. Remove .xml and documentation references to dfs.webhdfs.enabled. Contributed by Ray Chiang. 2015-05-08 15:57:13 +09:00
Vinayakumar B f33efbc827 HADOOP-11922. Misspelling of threshold in log4j.properties for tests in hadoop-tools (Contributed by Gabor Liptak) 2015-05-08 10:22:11 +05:30
Jonathan Eagles 2d7363b273 HADOOP-11889. Make checkstyle runnable from root project (Gera Shegalov via jeagles) 2015-05-01 13:11:50 -05:00
Akira Ajisaka f384a063a6 HADOOP-11821. Fix findbugs warnings in hadoop-sls. Contributed by Brahma Reddy Battula. 2015-04-30 19:34:41 +09:00
Allen Wittenauer 618ba707f0 MAPREDUCE-6057. Remove obsolete entries from mapred-default.xml (Ray Chiang via aw) 2015-04-26 20:31:40 -07:00
Ravi Prakash a287d2fb77 HADOOP-11876. Refactor code to make it more readable, minor maybePrintStats bug (Zoran Dimitrijevic via raviprak) 2015-04-24 13:39:07 -07:00
Steve Loughran 19262d99eb HADOOP-11730. Regression: s3n read failure recovery broken. (Takenori Sato via stevel) 2015-04-23 21:41:51 +01:00
Ravi Prakash cfba355052 HADOOP-11827. Speed-up distcp buildListing() using threadpool (Zoran Dimitrijevic via raviprak) 2015-04-21 16:49:37 -07:00
Jing Zhao 4c097e473b HDFS-8151. Always use snapshot path as source when invalid snapshot names are used for diff based distcp. Contributed by Jing Zhao. 2015-04-15 12:37:20 -07:00
Akira Ajisaka 83979e61a4 MAPREDUCE-6307. Remove property mapreduce.tasktracker.taskmemorymanager.monitoringinterval. Contributed by J.Andreina. 2015-04-10 20:21:00 +09:00
Colin Patrick Mccabe 932730df7d HADOOP-11785. Reduce the number of listStatus operation in distcp buildListing (Zoran Dimitrijevic via Colin P. McCabe) 2015-04-03 14:08:25 -07:00
Haohui Mai 75cb1d42ab HDFS-8036. Use snapshot path as source when using snapshot diff report in DistCp. Contributed by Jing Zhao. 2015-04-01 16:50:59 -07:00
Harsh J 27d49e6714 MAPREDUCE-6291. Correct mapred queue usage command. Contributed by Brahma Reddy Battula. 2015-03-28 11:58:17 +05:30
Tsuyoshi Ozawa e074952bd6 HADOOP-11760. Fix typo of javadoc in DistCp. Contributed by Brahma Reddy Battula. 2015-03-27 23:15:51 +09:00
Yongjun Zhang 44809b8081 HADOOP-11724. DistCp throws NPE when the target directory is root. (Lei Eddy Xu via Yongjun Zhang) 2015-03-25 15:45:45 -07:00
Harsh J b6dea9776b MAPREDUCE-579. Streaming slowmatch documentation. 2015-03-25 14:39:00 +05:30
Harsh J e5370477c2 MAPREDUCE-4653. TestRandomAlgorithm has an unused import statement. Contributed by Amir Sanjar. 2015-03-17 14:11:54 +05:30
Tsuyoshi Ozawa ef9946cd52 HADOOP-11720. [JDK8] Fix javadoc errors caused by incorrect or illegal tags in hadoop-tools. Contributed by Akira AJISAKA. 2015-03-17 16:09:21 +09:00
Tsuyoshi Ozawa 79426f3334 HADOOP-11558. Fix dead links to doc of hadoop-tools. Contributed by Jean-Pierre Matsumoto. 2015-03-15 14:29:49 +09:00
Tsuyoshi Ozawa b308a8d181 Revert "HADOOP-11558. Fix dead links to doc of hadoop-tools. Contributed by Masatake Iwasaki." 
This reverts commit 7da136ecca.
 2015-03-15 14:27:38 +09:00
Tsuyoshi Ozawa 7da136ecca HADOOP-11558. Fix dead links to doc of hadoop-tools. Contributed by Masatake Iwasaki. 2015-03-15 14:17:35 +09:00
cnauroth 7a346bcb4f HADOOP-11693. Azure Storage FileSystem rename operations are throttled too aggressively to complete HBase WAL archiving. Contributed by Duo Xu. 2015-03-11 14:36:51 -07:00
Steve Loughran 64443490d7 HADOOP-11670. Regression: s3a auth setup broken. (Adam Budde via stevel) 2015-03-08 11:22:16 -07:00
cnauroth 608ebd52ba HADOOP-11642. Upgrade azure sdk version from 0.6.0 to 2.0.0. Contributed by Shashank Khandelwal and Ivan Mitic. 2015-03-06 15:25:01 -08:00
Jing Zhao ed70fa142c HDFS-7535. Utilize Snapshot diff report for distcp. Contributed by Jing Zhao. 2015-03-04 10:30:53 -08:00
Vinod Kumar Vavilapalli 5af693fde2 MAPREDUCE-6248. Exposed the internal MapReduce job's information as a public API in DistCp. Contributed by Jing Zhao. 2015-03-03 16:28:41 -08:00
Steve Loughran 15b7076ad5 HADOOP-11183. Memory-based S3AOutputstream. (Thomas Demoor via stevel) 2015-03-03 16:18:51 -08:00
Tsuyoshi Ozawa 9ae7f9eb7b MAPREDUCE-5657. Fix Javadoc errors caused by incorrect or illegal tags in doc comments. Contributed by Akira AJISAKA. 2015-03-03 18:06:26 +09:00
Tsuyoshi Ozawa d1c6accb6f HADOOP-11602. Fix toUpperCase/toLowerCase to use Locale.ENGLISH. (ozawa) 2015-03-03 14:17:52 +09:00
Allen Wittenauer 039366e3b4 MAPREDUCE-5653. DistCp does not honour config-overrides for mapreduce.[map,reduce].memory.mb (Ratandeep Ratti via aw) 2015-02-28 22:53:38 -08:00
Allen Wittenauer dce8b9c4d0 HADOOP-11637. bash location hard-coded in shell scripts (aw) 2015-02-26 09:29:16 -08:00
cnauroth 5731c0e0d0 HADOOP-11629. WASB filesystem should not start BandwidthGaugeUpdater if fs.azure.skip.metrics set to true. Contributed by Shanyu Zhao. 2015-02-25 09:08:55 -08:00
Akira Ajisaka 1a625b8158 HADOOP-11480. Typo in hadoop-aws/index.md uses wrong scheme for test.fs.s3.name. Contributed by Ted Yu. 2015-02-24 17:11:46 -08:00
Tsuyoshi Ozawa 9cedad11d8 Revert "HADOOP-11602. Fix toUpperCase/toLowerCase to use Locale.ENGLISH. (ozawa)" 
This reverts commit 946456c6d8.

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/QuotaByStorageTypeEntry.java
 2015-02-25 00:32:04 +09:00
Steve Loughran 709ff99cff HADOOP-11584 s3a file block size set to 0 in getFileStatus. (Brahma Reddy Battula via stevel) 2015-02-21 12:03:03 +00:00
Steve Loughran aa1c437b6a HADOOP-11607. Reduce log spew in S3AFileSystem. (Lei (Eddy) Xu via stevel) 2015-02-20 20:51:24 +00:00
Tsuyoshi Ozawa 946456c6d8 HADOOP-11602. Fix toUpperCase/toLowerCase to use Locale.ENGLISH. (ozawa) 2015-02-19 13:06:53 +09:00
Allen Wittenauer b6fc1f3e43 HADOOP-11593. Convert site documentation from apt to markdown (stragglers) (Masatake Iwasaki via aw) 2015-02-17 21:30:24 -10:00
Karthik Kambatla 409113d8f9 MAPREDUCE-6234. TestHighRamJob fails due to the change in MAPREDUCE-5785. (Masatake Iwasaki via kasha) 2015-02-17 14:38:00 -08:00
Steve Loughran 00b80958d8 HADOOP-11521. Make connection timeout configurable in s3a. (Thomas Demoor via stevel) 2015-02-17 20:06:27 +00:00
Steve Loughran 78a7e8d3a6 HADOOP-11522. Update S3A Documentation. (Thomas Demoor via stevel) 2015-02-17 18:15:02 +00:00
Steve Loughran 826267f789 HADOOP-11570. S3AInputStream.close() downloads the remaining bytes of the object from S3. (Dan Hecht via stevel). 2015-02-17 16:36:44 +00:00
Allen Wittenauer 7776847a59 HADOOP-11437. Remove the version and author information from distcp's README file (Brahma Reddy Battula via aw) 2015-02-11 15:47:36 -08:00
Steve Loughran 17165d3df9 HADOOP-11520. Clean incomplete multi-part uploads in S3A tests. (Thomas Demoor via stevel) 2015-02-06 21:01:41 +00:00
Steve Loughran 4e7ad4f0a8 HADOOP-11463 Replace method-local TransferManager object with S3AFileSystem#transfers. (Ted Yu via stevel) 2015-02-05 12:20:23 +00:00
Allen Wittenauer 9112f093cd HADOOP-10976. moving the source code of hadoop-tools docs to the directory under hadoop-tools (Masatake Iwasaki via aw) 2015-02-04 17:57:34 -08:00
Allen Wittenauer f990e9d229 HADOOP-11346. Rewrite sls/rumen to use new shell framework (John Smith via aw) 2015-02-04 17:27:45 -08:00
Akira Ajisaka 34fe11c987 MAPREDUCE-6243. Fix findbugs warnings in hadoop-rumen. Contributed by Masatake Iwasaki. 2015-02-04 09:25:44 -08:00
Tsuyoshi Ozawa ba58da29f4 HADOOP-11546. Checkstyle failing: Unable to instantiate DoubleCheckedLockingCheck. (ozawa) 2015-02-05 00:42:14 +09:00
Akira Ajisaka bd69fb2d44 MAPREDUCE-5800. Use Job#getInstance instead of deprecated constructors. (aajisaka) 2015-02-03 14:30:09 -08:00
Karthik Kambatla c559df2219 YARN-1393. SLS: Add how-to-use instructions. (Wei Yan via kasha) 2015-02-03 13:20:41 -08:00
Haohui Mai c89977f89c HADOOP-11529. Fix findbugs warnings in hadoop-archives. Contributed by Masatake Iwasaki. 2015-02-03 10:53:17 -08:00
cnauroth 1c09ca2ba4 HADOOP-11442. hadoop-azure: Create test jar. Contributed by Shashank Khandelwal. 2015-02-02 11:50:43 -08:00
Harsh J ffc75d6ebe HADOOP-11488. Difference in default connection timeout for S3A FS. Contributed by Daisuke Kobayashi. 2015-02-01 00:17:04 +05:30
Allen Wittenauer 12e883007c MAPREDUCE-6151. Update document of GridMix (Masatake Iwasaki via aw) 2015-01-30 11:08:20 -08:00
cnauroth f2c91098c4 HADOOP-11523. StorageException complaining " no lease ID" when updating FolderLastModifiedTime in WASB. Contributed by Duo Xu. 2015-01-29 17:02:38 -08:00
Allen Wittenauer 3f982c5c26 MAPREDUCE-6150. Update document of Rumen (Masatake Iwasaki via aw) 2015-01-29 14:17:44 -08:00
cnauroth ad55083f75 HADOOP-11441. Hadoop-azure: Change few methods scope to public. Contributed by Shashank Khandelwal. 2015-01-29 13:07:51 -08:00
Allen Wittenauer 0a05ae1782 HDFS-7566. Remove obsolete entries from hdfs-default.xml (Ray Chiang via aw) 2015-01-27 14:31:27 -08:00
Tsuyoshi Ozawa 7b82c4ab4e HADOOP-11450. Cleanup DistCpV1 not to use deprecated methods and fix javadocs. Contributed by Varun Saxena. 2015-01-26 12:58:38 +09:00
Allen Wittenauer 42715547f0 HADOOP-11256. Some site docs have inconsistent appearance (Masatake Iwasaki via aw) 2015-01-21 08:44:22 -08:00
Steve Loughran 2908fe4ec5 HADOOP-11171 Enable using a proxy server to connect to S3a. (Thomas Demoor via stevel) 2015-01-17 18:40:50 +00:00
Steve Loughran c6c0f4eb25 HADOOP-10542 Potential null pointer dereference in Jets3tFileSystemStore retrieveBlock(). (Ted Yu via stevel) 2015-01-17 18:26:26 +00:00
Steve Loughran 000ca83ea3 HADOOP-11261 Set custom endpoint for S3A. (Thomas Demoor via stevel) 2015-01-16 10:15:51 +00:00
Arpit Agarwal c4cba6165a HADOOP-11465. Fix findbugs warnings in hadoop-gridmix. (Contributed by Varun Saxena) 2015-01-12 21:22:58 -08:00
Steve Loughran 27d8395867 HADOOP-11446 S3AOutputStream should use shared thread pool to avoid OutOfMemoryError 2015-01-05 13:00:01 +00:00
Karthik Kambatla e7257acd8a YARN-2998. Abstract out scheduler independent PlanFollower components. (Anubhav Dhoot via kasha) 2014-12-30 19:55:24 -08:00
Tsuyoshi Ozawa a164ce2ac9 HADOOP-11283. SequenceFile.Writer can leak file descriptors in DistCpV1#setup(). (Contributed by Varun Saxena via ozawa) 2014-12-25 23:30:45 +09:00
Jian He fdf042dfff YARN-2920. Changed CapacityScheduler to kill containers on nodes where node labels are changed. Contributed by Wangda Tan 2014-12-22 16:51:15 -08:00
Haohui Mai 02b21b7131 HADOOP-11429. Findbugs warnings in hadoop extras. Contributed by Varun Saxena. 2014-12-21 14:57:34 -08:00
arp c1f857b0b4 HADOOP-11395. Add site documentation for Azure Storage FileSystem integration. (Contributed by Chris Nauroth) 2014-12-19 18:54:22 -08:00
Haohui Mai efe6357748 HADOOP-11431. Clean up redundant maven-site-plugin configuration. Contributed by Herve Boutemy. 2014-12-18 16:23:35 -08:00
Steve Loughran 07bb0b0bbb HADOOP-11412 POMs mention "The Apache Software License" rather than "Apache License". (Herve Boutemy via stevel) 2014-12-16 11:04:36 +00:00
cnauroth 9458cd5bce HADOOP-11394. hadoop-aws documentation missing. Contributed by Chris Nauroth. 2014-12-12 23:29:11 -08:00
Haohui Mai 2e98ad34ce HADOOP-11381. Fix findbugs warnings in hadoop-distcp, hadoop-aws, hadoop-azure, and hadoop-openstack. Contributed by Li Lu. 2014-12-09 20:45:21 -08:00
Haohui Mai 7bceb13ba9 HADOOP-11367. Fix warnings from findbugs 3.0 in hadoop-streaming. Contributed by Li Lu. 2014-12-09 10:41:35 -08:00
Jing Zhao 57cb43be50 HADOOP-11354. ThrottledInputStream doesn't perform effective throttling. Contributed by Ted Yu. 2014-12-08 11:08:39 -08:00
cnauroth 79301e80d7 HADOOP-11201. Hadoop Archives should support globs resolving to files. Contributed by Gera Shegalov. 2014-11-18 17:05:48 -08:00
Aaron T. Myers 6ba52d88ec HADOOP-10714. AmazonS3Client.deleteObjects() need to be limited to 1000 entries per call. Contributed by Juan Yu. 2014-11-05 17:17:04 -08:00
cnauroth 734eeb4f35 HADOOP-11248. Add hadoop configuration to disable Azure Filesystem metrics collection. Contributed by Shanyu Zhao. 2014-11-03 14:29:18 -08:00
Zhijie Shen 260ab6d5f4 HADOOP-11241. Fixed intermittent TestNMSimulator failure due to timing issue. Contributed by Varun Vasudev. 2014-10-31 18:02:39 -07:00
Haohui Mai f1a149e916 HADOOP-11253. Hadoop streaming test TestStreamXmlMultipleRecords fails on Windows. Contributed by Varun Vasudev. 2014-10-31 11:54:33 -07:00
Kihwal Lee b056048114 MAPREDUCE-6022. map_input_file is missing from streaming job 
environment. Contributed by Jason Lowe.
 2014-10-29 12:29:07 -05:00
Jian He b8f7966c7a MAPREDUCE-6126. Fixed Rumen JobBuilder to ignore NormalizedResourceEvent. Contributed by Junping Du 2014-10-22 10:43:20 -07:00
Vinod Kumar Vavilapalli f2ea555ac6 YARN-2496. Enhanced Capacity Scheduler to have basic support for allocating resources based on node-labels. Contributed by Wangda Tan. 
YARN-2500. Ehnaced ResourceManager to support schedulers allocating resources based on node-labels. Contributed by Wangda Tan.
 2014-10-15 18:33:06 -07:00
cnauroth f4b7e99f4e HADOOP-11188. hadoop-azure: automatically expand page blobs when they become full. Contributed by Eric Hanson. 2014-10-10 15:05:52 -07:00
cnauroth 2217e2f8ff HADOOP-10809. hadoop-azure: page blob support. Contributed by Dexter Bradshaw, Mostafa Elhemali, Eric Hanson, and Mike Liddell. 2014-10-08 14:20:23 -07:00
Kihwal Lee c8617ff136 Missing files from the previous commit of HADOOP-11007 2014-10-07 16:30:53 -05:00
Kihwal Lee 99d7a452be HADOOP-11007. Reinstate building of ant tasks support. Contributed by 
Jason Lowe.
 2014-10-07 16:20:16 -05:00
Allen Wittenauer 83264cf457 HADOOP-10954. Adding site documents of hadoop-tools (Masatake Iwasaki via aw) 2014-09-26 11:59:45 -07:00
Andrew Wang 4ea77efa3a HADOOP-11140. hadoop-aws only need test-scoped dependency on hadoop-common's tests jar. Contributed by Juan Yu. 2014-09-25 21:56:26 -07:00
Allen Wittenauer f614cb71d4 MAPREDUCE-6109. Fix minor typo in distcp -p usage text (Charles Lamb via aw) 2014-09-25 11:27:33 -07:00
Allen Wittenauer 3cde37c991 HADOOP-11009. Add Timestamp Preservation to DistCp (Gary Steelman via aw) 2014-09-24 15:38:30 -07:00
Allen Wittenauer aa1052c34b YARN-2460. Remove obsolete entries from yarn-default.xml (Ray Chiang via aw) 2014-09-19 14:07:38 -07:00
Allen Wittenauer 9f03a7c018 HADOOP-10946. Fix a bunch of typos in log messages (Ray Chiang via aw) 2014-09-19 11:33:07 -07:00
Colin Patrick Mccabe 0ac760a58d HADOOP-11091. Eliminate old configuration parameter names from s3a (dsw via cmccabe) 2014-09-15 16:49:15 -07:00
Aaron T. Myers 24d920b80e HADOOP-10400. Incorporate new S3A FileSystem implementation. Contributed by Jordan Mendelson and Dave Wang. 2014-09-15 08:27:07 -07:00
Colin Patrick Mccabe 5ec7fcd9dd HADOOP-11074. Move s3-related FS connector code to hadoop-aws. (David S. Wang via Colin Patrick McCabe) 2014-09-10 16:14:53 -07:00
Karthik Kambatla 28d99db992 YARN-2526. SLS can deadlock when all the threads are taken by AMSimulators. (Wei Yan via kasha) 2014-09-09 15:13:16 -07:00
Allen Wittenauer d989ac0444 MAPREDUCE-5972. Fix typo 'programatically' in job.xml (and a few other places) (Akira AJISAKA via aw) 2014-09-08 13:15:46 -07:00
Steve Loughran 59384dfb71 HADOOP-10373 create tools/hadoop-amazon for aws/EMR support 2014-09-02 20:11:13 +01:00
Jian He 5c14bc426b YARN-1506. Changed RMNode/SchedulerNode to update resource with event notification. Contributed by Junping Du 2014-08-29 23:05:51 -07:00
Andrew Wang ea1c6f31c2 HADOOP-11021. Configurable replication factor in the hadoop archive command. Contributed by Zhe Zhang. 2014-08-29 14:44:37 -07:00
Sanford Ryza 5109157ed1 MAPREDUCE-5130. Add missing job config options to mapred-default.xml (Ray Chiang via Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1619626 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-21 23:28:44 +00:00
Andrew Wang d2a39b61aa Merge from trunk to branch. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1619018 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-20 01:22:42 +00:00
Charles Lamb 14b01dd046 HDFS-6872. Fix TestOptionsParser. (clamb) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1618974 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-19 20:41:15 +00:00
Zhijie Shen f6a778c372 YARN-2249. Avoided AM release requests being lost on work preserving RM restart. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618972 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-19 20:33:49 +00:00
Andrew Wang 0cc08f6da4 Merge from trunk to branch. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1618700 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-18 18:41:31 +00:00
Allen Wittenauer 399e428deb HADOOP-10121. Fix javadoc spelling for HadoopArchives#writeTopLevelDirs (Akira AJISAKA via aw) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618044 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-14 20:28:36 +00:00
Colin McCabe e90d940699 HADOOP-10843. TestGridmixRecord unit tests failure on PowerPC (Jinghui Wang via Colin Patrick McCabe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617860 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-14 01:36:12 +00:00
Karthik Kambatla 486e718fc1 YARN-2399. FairScheduler: Merge AppSchedulable and FSSchedulerApp into FSAppAttempt. (kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1617600 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-12 21:43:27 +00:00
Charles Lamb 041b8326a1 MAPREDUCE-6007. Add support to distcp to preserve raw.* namespace extended attributes. (clamb) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/branches/fs-encryption@1616657 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-08 01:32:18 +00:00
Steve Loughran 96fc3b3ecd HADOOP-10931 compile error on tools/hadoop-openstack 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1616482 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-07 12:38:35 +00:00
Karthik Kambatla 4625792c20 YARN-1726. Add missing files. ResourceSchedulerWrapper broken due to AbstractYarnScheduler. (Wei Yan via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1613552 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-26 02:03:18 +00:00
Karthik Kambatla b0c2c85619 YARN-1726. ResourceSchedulerWrapper broken due to AbstractYarnScheduler. (Wei Yan via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1613547 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-26 01:29:25 +00:00
Karthik Kambatla 77363b9d83 YARN-2335. Annotate all hadoop-sls APIs as @Private. (Wei Yan via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1613478 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-25 17:12:22 +00:00
Chris Nauroth 0a02b5a19b HADOOP-10840. Fix OutOfMemoryError caused by metrics system in Azure File System. Contributed by Shanyu Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1611247 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-17 05:04:53 +00:00
Andrew Wang e8a1ff7efe MAPREDUCE-5971. Move the default options for distcp -p to DistCpOptionSwitch. Contributed by Charles Lamb. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1611217 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-16 23:42:34 +00:00
Steve Loughran a10a4f72ab HADOOP-9361: Strictly define FileSystem APIs - OpenStack portion 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1607600 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-03 12:12:52 +00:00
Steve Loughran becc23fb65 HADOOP-9361: Strictly define FileSystem APIs - OpenStack portion 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1607599 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-03 12:10:52 +00:00
Chris Nauroth 0d91576ec3 HADOOP-10728. Metrics system for Windows Azure Storage Filesystem. Contributed by Dexter Bradshaw, Mostafa Elhemali, Xi Fang, Johannes Klein, David Lao, Mike Liddell, Chuan Liu, Lengning Liu, Ivan Mitic, Michael Rys, Alexander Stojanovic, Brian Swan, and Min Wei. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605187 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-24 20:52:44 +00:00
Chris Nauroth 61bf9f7799 HADOOP-10690. Lack of synchronization on access to InputStream in NativeAzureFileSystem#NativeAzureFsInputStream#close(). Contributed by Chen He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1604236 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-20 18:36:11 +00:00
Chris Nauroth 8300b9fb38 HADOOP-10689. InputStream is not closed in AzureNativeFileSystemStore#retrieve(). Contributed by Chen He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1604233 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-20 18:30:49 +00:00
Haohui Mai 22cbcd16cb HADOOP-10717. Revert r1603571. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603620 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-18 20:37:25 +00:00
Steve Loughran bae5837d3b HADOOP-10717. Missing JSP support in Jetty 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603534 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-18 17:08:45 +00:00
Andrew Wang 50cd2a6bed HADOOP-10666. Remove Copyright /d/d/d/d Apache Software Foundation from the source files license header. Contributed by Henry Saputra. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603025 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-16 23:23:33 +00:00
Chris Nauroth 81bc395deb HADOOP-9629. Support Windows Azure Storage - Blob as a file system in Hadoop. Contributed by Dexter Bradshaw, Mostafa Elhemali, Xi Fang, Johannes Klein, David Lao, Mike Liddell, Chuan Liu, Lengning Liu, Ivan Mitic, Michael Rys, Alexander Stojanovic, Brian Swan, and Min Wei. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601781 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-10 22:26:45 +00:00
Vinod Kumar Vavilapalli 6480fdfd33 MAPREDUCE-5834. Increased test-timeouts in TestGridMixClasses to avoid occassional failures. Contributed by Mit Desai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601764 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-10 20:24:57 +00:00
Steve Loughran 3d077726d0 HADOOP-10647. String Format Exception in SwiftNativeFileSystemStore.java. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1600971 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-06 18:00:52 +00:00
Uma Maheswara Rao G f81c7b0252 MAPREDUCE-5898. distcp to support preserving HDFS extended attributes(XAttrs). Contributed by Yi Liu. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1600900 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-06 14:45:39 +00:00
Karthik Kambatla a4ba451802 YARN-1474. Make schedulers services. (Tsuyoshi Ozawa via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598908 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-31 19:33:09 +00:00
Chris Nauroth 806f4d11f2 HADOOP-10602. Documentation has broken "Go Back" hyperlinks. Contributed by Akira AJISAKA. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598337 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-29 17:26:50 +00:00
Jing Zhao 3671a5e16f MAPREDUCE-5899. Support incremental data copy in DistCp. Contributed by Jing Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1596931 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-22 18:17:11 +00:00
Vinod Kumar Vavilapalli 82f3454f5a YARN-2017. Merged some of the common scheduler code. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1596753 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-22 05:32:26 +00:00
Chris Nauroth 11be7334c4 MAPREDUCE-5809. Enhance distcp to support preserving HDFS ACLs. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1595283 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-16 18:25:23 +00:00
Tsz-wo Sze 03db13206f MAPREDUCE-5402. In DynamicInputFormat, change MAX_CHUNKS_TOLERABLE, MAX_CHUNKS_IDEAL, MIN_RECORDS_PER_CHUNK and SPLIT_RATIO to be configurable. Contributed by Tsuyoshi OZAWA 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1592703 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-06 10:24:10 +00:00
Jonathan Turner Eagles a059eadbe9 MAPREDUCE-5639. Port DistCp2 document to trunk (Akira AJISAKA via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1590058 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-25 15:23:42 +00:00
Ravi Prakash a93be7b46c MAPREDUCE-5846. Rumen doesn't understand JobQueueChangedEvent (Nathan Roberts via raviprak) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1589595 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-24 05:59:28 +00:00
Chris Nauroth 06c15b6a3e MAPREDUCE-5852. Prepare MapReduce codebase for JUnit 4.11. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1589006 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-21 23:44:41 +00:00
Chris Nauroth 84388525a3 YARN-1970. Prepare YARN codebase for JUnit 4.11. Contributed by Chris Nauroth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1589001 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-21 23:31:18 +00:00
Haohui Mai 0862ee6520 HADOOP-10485. Remove dead classes in hadoop-streaming. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1586059 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-09 18:10:36 +00:00
Steve Loughran b8f250a99d HADOOP-10104. Update jackson to 1.9.13 (Akira Ajisaka via stevel) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1585932 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-09 11:11:20 +00:00
Haohui Mai 8ca32df08e HADOOP-10474. Move o.a.h.record to hadoop-streaming. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1585886 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-09 04:02:46 +00:00
Aaron Myers 640a097533 HADOOP-10459. distcp V2 doesn't preserve root dir's attributes when -p is specified. Contributed by Yongjun Zhang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1584227 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-03 00:32:25 +00:00
Haohui Mai fb1d7fb596 HDFS-5570. Deprecate hftp / hsftp and replace them with webhdfs / swebhdfs. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1584100 13f79535-47bb-0310-9956-ffa450edef68
 2014-04-02 17:28:08 +00:00
Vinod Kumar Vavilapalli 86ef362a7d MAPREDUCE-5810. Removed the faulty and failing streaming test TestStreamingTaskLog. Contributed by Akira Ajisaka. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1582896 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-28 22:16:55 +00:00
Jing Zhao 0b3ee72af8 HDFS-6090. Use MiniDFSCluster.Builder instead of deprecated constructors. Contributed by Akira AJISAKA. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1578669 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-17 23:53:48 +00:00
Jonathan Turner Eagles a8f302a1e6 MAPREDUCE-5765. Update hadoop-pipes examples README (Mit Desai via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1577188 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-13 15:13:07 +00:00
Sanford Ryza 1079c37128 YARN-1504. RM changes for moving apps between queues (Sandy Ryza) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1563371 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-01 04:03:36 +00:00
Alejandro Abdelnur 5a2428d39f HADOOP-10311. Cleanup vendor names from the code base. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1563239 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-31 20:11:29 +00:00
Jing Zhao 067d52b98c HADOOP-10295. Allow distcp to automatically identify the checksum type of source files and use it for the target. Contributed by Jing Zhao and Laurent Goujon. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1563019 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-30 23:53:23 +00:00
Konstantin Boudnik 4c197b5d56 HADOOP-10167. Mark hadoop-common source as UTF-8 in Maven pom files / refactoring. Contributed by Mikhail Antonov. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1560831 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-23 22:29:23 +00:00
Vinod Kumar Vavilapalli 1393581bce YARN-1490. Introduced the ability to make ResourceManager optionally not kill all containers when an ApplicationMaster exits. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1557143 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-10 15:04:39 +00:00