Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
24,568 Commits 326 Branches 371 Tags 1.5 GiB
Java 92.7%
C++ 2.9%
C 1.9%
JavaScript 1.2%
Shell 0.5%
Other 0.5%
Go to file
Steve Loughran 26514b6534 HADOOP-17628. Distcp contract test is really slow with ABFS and S3A; timing out. (#3240) 
This patch cuts down the size of directory trees used for
distcp contract tests against object stores, so making
them much faster against distant/slow stores.

On abfs, the test only runs with -Dscale (as was the case for s3a already),
and has the larger scale test timeout.

After every test case, the FileSystem IOStatistics are logged,
to provide information about what IO is taking place and
what it's performance is.

There are some test cases which upload files of 1+ MiB; you can
increase the size of the upload in the option
"scale.test.distcp.file.size.kb" 
Set it to zero and the large file tests are skipped.

Contributed by Steve Loughran.
 		2021-08-02 12:58:37 +01:00
.github HADOOP-15184. Add GitHub pull request template. (#1419) 2019-09-11 11:10:11 +09:00
dev-support HADOOP-17775. Remove JavaScript package from Docker environment. (#3182) 2021-07-07 17:49:19 +09:00
hadoop-assemblies Preparing for 3.3.2 development 2021-05-19 21:52:37 +08:00
hadoop-build-tools HADOOP-17813. Checkstyle - Allow line length: 100 2021-07-22 15:02:58 -05:00
hadoop-client-modules Preparing for 3.3.2 development 2021-05-19 21:52:37 +08:00
hadoop-cloud-storage-project Preparing for 3.3.2 development 2021-05-19 21:52:37 +08:00
hadoop-common-project HADOOP-17628. Distcp contract test is really slow with ABFS and S3A; timing out. (#3240) 2021-08-02 12:58:37 +01:00
hadoop-dist Preparing for 3.3.2 development 2021-05-19 21:52:37 +08:00
hadoop-hdfs-project HDFS-15175. Multiple CloseOp shared block instance causes the standby namenode to crash when rolling editlog. Contributed by Wan Chang. 2021-07-29 19:04:08 +08:00
hadoop-mapreduce-project MAPREDUCE-7258. HistoryServerRest.html#Task_Counters_API, modify the jobTaskCounters's itemName from taskcounterGroup to taskCounterGroup (#1808) 2021-08-02 15:39:38 +09:00
hadoop-maven-plugins HADOOP-17777. Update clover-maven-plugin version from 3.3.0 to 4.4.1 (#3152) 2021-06-30 09:35:36 +08:00
hadoop-minicluster Preparing for 3.3.2 development 2021-05-19 21:52:37 +08:00
hadoop-project HADOOP-17796. Upgrade jetty version to 9.4.43 (#3208) 2021-07-22 16:37:39 +08:00
hadoop-project-dist Preparing for 3.3.2 development 2021-05-19 21:52:37 +08:00
hadoop-tools HADOOP-17628. Distcp contract test is really slow with ABFS and S3A; timing out. (#3240) 2021-08-02 12:58:37 +01:00
hadoop-yarn-project YARN-6221. Entities missing from ATS when summary log file info got returned to the ATS before the domain log. Contributed by Xiaomin Zhang 2021-07-31 14:53:30 +02:00
licenses HADOOP-17666. Update LICENSE for 3.3.1 (#3011) 2021-05-21 18:15:48 -07:00
licenses-binary HADOOP-17666. Update LICENSE for 3.3.1 (#3011) 2021-05-21 18:15:48 -07:00
.gitattributes HADOOP-13598. Add eol=lf for unix format files in .gitattributes. Contributed by Yiqun Lin. 2016-09-14 11:14:31 +09:00
.gitignore YARN-10407. Add phantomjsdriver.log to gitignore. (#2244) 2021-02-17 10:28:17 +09:00
BUILDING.txt HADOOP-17777. Update clover-maven-plugin version from 3.3.0 to 4.4.1 (#3152) 2021-06-30 09:35:36 +08:00
LICENSE-binary HADOOP-17769. Upgrade JUnit to 4.13.2. fixes TestBlockRecovery (#3130). Contributed by Ahmed Hussein. (#3138) 2021-06-25 22:48:56 +05:30
LICENSE-binary-yarn-applications-catalog-webapp HADOOP-17666. Update LICENSE for 3.3.1 (#3011) 2021-05-21 18:15:48 -07:00
LICENSE-binary-yarn-ui HADOOP-17666. Update LICENSE for 3.3.1 (#3011) 2021-05-21 18:15:48 -07:00
LICENSE.txt HADOOP-17666. Update LICENSE for 3.3.1 (#3011) 2021-05-21 18:15:48 -07:00
NOTICE-binary HADOOP-17666. Update LICENSE for 3.3.1 (#3011) 2021-05-21 18:15:48 -07:00
NOTICE.txt HADOOP-15958. Revisiting LICENSE and NOTICE files. 2019-08-27 13:47:12 +09:00
README.txt HADOOP-15958. Revisiting LICENSE and NOTICE files. 2019-08-27 13:47:12 +09:00
pom.xml HADOOP-17777. Update clover-maven-plugin version from 3.3.0 to 4.4.1 (#3152) 2021-06-30 09:35:36 +08:00
start-build-env.sh HADOOP-17298. Backslash in username causes build failure in the environment started by start-build-env.sh. (#2367) 2020-10-20 10:11:18 +09:00

README.txt 

For the latest information about Hadoop, please visit our website at:

   http://hadoop.apache.org/

and our wiki, at:

   https://cwiki.apache.org/confluence/display/HADOOP/