Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

HADOOP-18026. Fix default value of Magic committer (#3723) 

Contributed by guophilipse
This commit is contained in:
GuoPhilipse 2021-11-29 23:50:30 +08:00 committed by GitHub
parent 932a78fe38
commit c65c87f211
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/committers.md
View File

@ -547,7 +547,7 @@ Conflict management is left to the execution engine itself.
|--------|---------|---------| |--------|---------|---------|
| `mapreduce.fileoutputcommitter.marksuccessfuljobs` | Write a `_SUCCESS` file on the successful completion of the job. | `true` | | `mapreduce.fileoutputcommitter.marksuccessfuljobs` | Write a `_SUCCESS` file on the successful completion of the job. | `true` |
| `fs.s3a.buffer.dir` | Local filesystem directory for data being written and/or staged. | `${hadoop.tmp.dir}/s3a` | | `fs.s3a.buffer.dir` | Local filesystem directory for data being written and/or staged. | `${hadoop.tmp.dir}/s3a` |
| `fs.s3a.committer.magic.enabled` | Enable "magic committer" support in the filesystem. | `false` | | `fs.s3a.committer.magic.enabled` | Enable "magic committer" support in the filesystem. | `true` |
| `fs.s3a.committer.abort.pending.uploads` | list and abort all pending uploads under the destination path when the job is committed or aborted. | `true` | | `fs.s3a.committer.abort.pending.uploads` | list and abort all pending uploads under the destination path when the job is committed or aborted. | `true` |
| `fs.s3a.committer.threads` | Number of threads in committers for parallel operations on files. | 8 | | `fs.s3a.committer.threads` | Number of threads in committers for parallel operations on files. | 8 |
| `fs.s3a.committer.generate.uuid` | Generate a Job UUID if none is passed down from Spark | `false` | | `fs.s3a.committer.generate.uuid` | Generate a Job UUID if none is passed down from Spark | `false` |