HDFS-12251. Add document for StreamCapabilities. (Lei (Eddy) Xu)

This commit is contained in:
Lei Xu 2017-08-04 11:21:58 -07:00
parent a6fdeb8a87
commit fe3341786a
2 changed files with 43 additions and 0 deletions

View File

@ -1210,3 +1210,27 @@ try {
It is notable that this is *not* done in the Hadoop codebase. This does not imply
that robust loops are not recommended —more that the concurrency
problems were not considered during the implementation of these loops.
## <a name="StreamCapability"></a> interface `StreamCapabilities`
The `StreamCapabilities` provides a way to programmatically query the
capabilities that an `OutputStream` supports.
```java
public interface StreamCapabilities {
boolean hasCapability(String capability);
}
```
### `boolean hasCapability(capability)`
Return true if the `OutputStream` has the desired capability.
The caller can query the capabilities of a stream using a string value.
It currently supports to query:
* `StreamCapabilties.HFLUSH` ("*hflush*"): the capability to flush out the data
in client's buffer.
* `StreamCapabilities.HSYNC` ("*hsync*"): capability to flush out the data in
client's buffer and the disk device.

View File

@ -199,3 +199,22 @@ Below are the details about each command.
* `[-disablePolicy -policy <policyName>]`
Disable an erasure coding policy.
Limitations
-----------
Certain HDFS file write operations, i.e., `hflush`, `hsync` and `append`,
are not supported on erasure coded files due to substantial technical
challenges.
* `append()` on an erasure coded file will throw `IOException`.
* `hflush()` and `hsync()` on `DFSStripedOutputStream` are no-op. Thus calling
`hflush()` or `hsync()` on an erasure coded file can not guarantee data
being persistent.
A client can use [`StreamCapabilities`](../hadoop-common/filesystem/filesystem.html#interface_StreamCapabilities)
API to query whether a `OutputStream` supports `hflush()` and `hsync()`.
If the client desires data persistence via `hflush()` and `hsync()`, the current
remedy is creating such files as regular 3x replication files in a
non-erasure-coded directory, or using `FSDataOutputStreamBuilder#replicate()`
API to create 3x replication files in an erasure-coded directory.