Go to file
Steve Loughran 75950e47e7
HADOOP-16202. Enhanced openFile(): hadoop-common changes. (#2584/1)
This defines standard option and values for the
openFile() builder API for opening a file:

fs.option.openfile.read.policy
 A list of the desired read policy, in preferred order.
 standard values are
 adaptive, default, random, sequential, vector, whole-file

fs.option.openfile.length
 How long the file is.

fs.option.openfile.split.start
 start of a task's split

fs.option.openfile.split.end
 end of a task's split

These can be used by filesystem connectors to optimize their
reading of the source file, including but not limited to
* skipping existence/length probes when opening a file
* choosing a policy for prefetching/caching data

The hadoop shell commands which read files all declare "whole-file"
and "sequential", as appropriate.

Contributed by Steve Loughran.

Change-Id: Ia290f79ea7973ce8713d4f90f1315b24d7a23da1
2022-04-27 19:23:10 +01:00
.github HADOOP-15184. Add GitHub pull request template. (#1419) 2019-09-11 11:10:11 +09:00
dev-support HADOOP-18202. create-release fails fatal: unsafe repository (#4188) 2022-04-18 19:28:44 +01:00
hadoop-assemblies HADOOP-18198. Preparing for 3.3.4 development 2022-04-12 14:09:08 +01:00
hadoop-build-tools HADOOP-18198. Preparing for 3.3.4 development 2022-04-12 14:09:08 +01:00
hadoop-client-modules HADOOP-18198. Preparing for 3.3.4 development 2022-04-12 14:09:08 +01:00
hadoop-cloud-storage-project HADOOP-18198. Preparing for 3.3.4 development 2022-04-12 14:09:08 +01:00
hadoop-common-project HADOOP-16202. Enhanced openFile(): hadoop-common changes. (#2584/1) 2022-04-27 19:23:10 +01:00
hadoop-dist HADOOP-18198. Preparing for 3.3.4 development 2022-04-12 14:09:08 +01:00
hadoop-hdfs-project HADOOP-17956. Replace all default Charset usage with UTF-8 (#3529) 2022-04-27 10:30:07 +01:00
hadoop-mapreduce-project HADOOP-17956. Replace all default Charset usage with UTF-8 (#3529) 2022-04-27 10:30:07 +01:00
hadoop-maven-plugins HADOOP-17956. Replace all default Charset usage with UTF-8 (#3529) 2022-04-27 10:30:07 +01:00
hadoop-minicluster HADOOP-18198. Preparing for 3.3.4 development 2022-04-12 14:09:08 +01:00
hadoop-project HADOOP-17650. Bump solr to unblock build failure with Maven 3.8.1 (#2939) 2022-04-20 16:36:51 +01:00
hadoop-project-dist HADOOP-18198. Preparing for 3.3.4 development 2022-04-12 14:09:08 +01:00
hadoop-tools HADOOP-17682. ABFS: Support FileStatus input to OpenFileWithOptions() via OpenFileParameters (#2975) 2022-04-27 19:22:49 +01:00
hadoop-yarn-project HADOOP-17956. Replace all default Charset usage with UTF-8 (#3529) 2022-04-27 10:30:07 +01:00
licenses HADOOP-17666. Update LICENSE for 3.3.1 (#3011) 2021-05-21 18:15:48 -07:00
licenses-binary HADOOP-17666. Update LICENSE for 3.3.1 (#3011) 2021-05-21 18:15:48 -07:00
.gitattributes HADOOP-13598. Add eol=lf for unix format files in .gitattributes. Contributed by Yiqun Lin. 2016-09-14 11:14:31 +09:00
.gitignore YARN-10407. Add phantomjsdriver.log to gitignore. (#2244) 2021-02-17 10:28:17 +09:00
BUILDING.txt HADOOP-18214. Update BUILDING.txt (#3811) 2022-04-21 18:39:51 +01:00
LICENSE-binary HADOOP-18178. Upgrade jackson to 2.13.2 and jackson-databind to 2.13.2.2 (#4147) 2022-04-11 14:58:28 +09:00
LICENSE-binary-yarn-applications-catalog-webapp HADOOP-17666. Update LICENSE for 3.3.1 (#3011) 2021-05-21 18:15:48 -07:00
LICENSE-binary-yarn-ui HADOOP-17666. Update LICENSE for 3.3.1 (#3011) 2021-05-21 18:15:48 -07:00
LICENSE.txt HADOOP-18044. Hadoop - Upgrade to jQuery 3.6.0 (#3791) 2022-02-11 23:18:25 +08:00
NOTICE-binary HADOOP-18068. upgrade AWS SDK to 1.12.132 (#3864) 2022-01-18 12:20:12 +00:00
NOTICE.txt HADOOP-15958. Revisiting LICENSE and NOTICE files. 2019-08-27 13:47:12 +09:00
README.txt HADOOP-15958. Revisiting LICENSE and NOTICE files. 2019-08-27 13:47:12 +09:00
pom.xml HADOOP-17551. Upgrade maven-site-plugin to 3.11.0 (#4196) 2022-04-21 22:16:58 +09:00
start-build-env.sh HADOOP-18052. Support Apple Silicon in start-build-env.sh (#3817) 2021-12-23 18:14:16 +09:00

README.txt

For the latest information about Hadoop, please visit our website at:

   http://hadoop.apache.org/

and our wiki, at:

   https://cwiki.apache.org/confluence/display/HADOOP/