Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
25,145 Commits 326 Branches 371 Tags 1.5 GiB
Java 92.7%
C++ 2.9%
C 1.9%
JavaScript 1.2%
Shell 0.5%
Other 0.5%
Go to file
Steve Loughran ee466d4b40
HADOOP-17628. Distcp contract test is really slow with ABFS and S3A; timing out. (#3240) 
This patch cuts down the size of directory trees used for
distcp contract tests against object stores, so making
them much faster against distant/slow stores.

On abfs, the test only runs with -Dscale (as was the case for s3a already),
and has the larger scale test timeout.

After every test case, the FileSystem IOStatistics are logged,
to provide information about what IO is taking place and
what it's performance is.

There are some test cases which upload files of 1+ MiB; you can
increase the size of the upload in the option
"scale.test.distcp.file.size.kb" 
Set it to zero and the large file tests are skipped.

Contributed by Steve Loughran.
 		2021-08-02 11:36:43 +01:00
.github HADOOP-15184. Add GitHub pull request template. (#1419) 2019-09-11 11:10:11 +09:00
dev-support HADOOP-17815. Run CI for Centos 7 (#3231) 2021-07-29 10:50:57 -07:00
hadoop-assemblies HDFS-15346. FedBalance tool implementation. Contributed by Jinglun. 2020-06-18 13:33:25 +08:00
hadoop-build-tools HADOOP-17813. Checkstyle - Allow line length: 100 2021-07-22 13:45:49 -05:00
hadoop-client-modules HADOOP-16822. Provide source artifacts for hadoop-client-api. Contributed by Karel Kolman. 2021-05-13 20:30:00 +08:00
hadoop-cloud-storage-project HADOOP-17753. Keep restrict-imports-enforcer-rule for Guava Lists in top level hadoop-main pom (#3087) 2021-06-11 12:15:52 +09:00
hadoop-common-project HADOOP-17628. Distcp contract test is really slow with ABFS and S3A; timing out. (#3240) 2021-08-02 11:36:43 +01:00
hadoop-dist Preparing for 3.4.0 development 2020-03-29 23:24:25 +05:30
hadoop-hdfs-project HDFS-14529. SetTimes to throw FileNotFoundException if inode is not found (#3243) 2021-07-30 08:29:56 -07:00
hadoop-mapreduce-project MAPREDUCE-7258. HistoryServerRest.html#Task_Counters_API, modify the jobTaskCounters's itemName from taskcounterGroup to taskCounterGroup (#1808) 2021-08-02 15:36:53 +09:00
hadoop-maven-plugins HADOOP-17777. Update clover-maven-plugin version from 3.3.0 to 4.4.1 (#3152) 2021-06-29 18:32:16 -07:00
hadoop-minicluster HDFS-15331. Remove invalid exclusions that minicluster dependency on HDFS (#1996). Contributed by Wanqiang Ji 2020-05-06 02:06:38 +05:30
hadoop-project HADOOP-17317. [JDK 11] Upgrade dnsjava to remove illegal access warnings (#2442) 2021-07-24 14:48:25 +09:00
hadoop-project-dist Make upstream aware of 3.3.1 release 2021-06-15 15:43:36 +00:00
hadoop-tools HADOOP-17628. Distcp contract test is really slow with ABFS and S3A; timing out. (#3240) 2021-08-02 11:36:43 +01:00
hadoop-yarn-project YARN-10854. Support marking inactive node as untracked without configured include path. Contributed by Tao Yang. 2021-08-02 18:23:33 +08:00
licenses HADOOP-17144. Update Hadoop's lz4 to v1.9.2. Contributed by Hemanth Boyina. 2020-10-18 18:37:46 +05:30
licenses-binary HADOOP-15993. Upgrade Kafka to 2.4.0 in hadoop-kafka module. (#1796) 2020-01-09 16:24:58 +09:00
.asf.yaml HADOOP-17234. Add .asf.yaml to allow Github to Jira integration. (#2253). Contributed by Ayush Saxena. 2020-08-28 17:22:46 +05:30
.gitattributes HADOOP-13598. Add eol=lf for unix format files in .gitattributes. Contributed by Yiqun Lin. 2016-09-14 11:14:31 +09:00
.gitignore YARN-10407. Add phantomjsdriver.log to gitignore. (#2244) 2020-09-01 10:44:55 +09:00
BUILDING.txt HADOOP-17777. Update clover-maven-plugin version from 3.3.0 to 4.4.1 (#3152) 2021-06-29 18:32:16 -07:00
LICENSE-binary HADOOP-16272. Upgrade HikariCP to 4.0.3 (#3204) 2021-07-16 12:17:17 +09:00
LICENSE.txt HADOOP-17283. Hadoop - Upgrade to jQuery 3.5.1 (#2330) 2020-09-24 21:15:55 +09:00
NOTICE-binary HADOOP-15958. Revisiting LICENSE and NOTICE files. 2019-08-27 13:47:12 +09:00
NOTICE.txt HADOOP-15958. Revisiting LICENSE and NOTICE files. 2019-08-27 13:47:12 +09:00
README.txt HADOOP-15958. Revisiting LICENSE and NOTICE files. 2019-08-27 13:47:12 +09:00
pom.xml HADOOP-17777. Update clover-maven-plugin version from 3.3.0 to 4.4.1 (#3152) 2021-06-29 18:32:16 -07:00
start-build-env.sh HADOOP-17298. Backslash in username causes build failure in the environment started by start-build-env.sh. (#2367) 2020-10-20 10:09:03 +09:00

README.txt 

For the latest information about Hadoop, please visit our website at:

   http://hadoop.apache.org/

and our wiki, at:

   https://cwiki.apache.org/confluence/display/HADOOP/