HDFS-12251. Add document for StreamCapabilities. (Lei (Eddy) Xu)
This commit is contained in:
parent
a6fdeb8a87
commit
fe3341786a
|
@ -1210,3 +1210,27 @@ try {
|
|||
It is notable that this is *not* done in the Hadoop codebase. This does not imply
|
||||
that robust loops are not recommended —more that the concurrency
|
||||
problems were not considered during the implementation of these loops.
|
||||
|
||||
|
||||
## <a name="StreamCapability"></a> interface `StreamCapabilities`
|
||||
|
||||
The `StreamCapabilities` provides a way to programmatically query the
|
||||
capabilities that an `OutputStream` supports.
|
||||
|
||||
```java
|
||||
public interface StreamCapabilities {
|
||||
boolean hasCapability(String capability);
|
||||
}
|
||||
```
|
||||
|
||||
### `boolean hasCapability(capability)`
|
||||
|
||||
Return true if the `OutputStream` has the desired capability.
|
||||
|
||||
The caller can query the capabilities of a stream using a string value.
|
||||
It currently supports to query:
|
||||
|
||||
* `StreamCapabilties.HFLUSH` ("*hflush*"): the capability to flush out the data
|
||||
in client's buffer.
|
||||
* `StreamCapabilities.HSYNC` ("*hsync*"): capability to flush out the data in
|
||||
client's buffer and the disk device.
|
|
@ -199,3 +199,22 @@ Below are the details about each command.
|
|||
* `[-disablePolicy -policy <policyName>]`
|
||||
|
||||
Disable an erasure coding policy.
|
||||
|
||||
Limitations
|
||||
-----------
|
||||
|
||||
Certain HDFS file write operations, i.e., `hflush`, `hsync` and `append`,
|
||||
are not supported on erasure coded files due to substantial technical
|
||||
challenges.
|
||||
|
||||
* `append()` on an erasure coded file will throw `IOException`.
|
||||
* `hflush()` and `hsync()` on `DFSStripedOutputStream` are no-op. Thus calling
|
||||
`hflush()` or `hsync()` on an erasure coded file can not guarantee data
|
||||
being persistent.
|
||||
|
||||
A client can use [`StreamCapabilities`](../hadoop-common/filesystem/filesystem.html#interface_StreamCapabilities)
|
||||
API to query whether a `OutputStream` supports `hflush()` and `hsync()`.
|
||||
If the client desires data persistence via `hflush()` and `hsync()`, the current
|
||||
remedy is creating such files as regular 3x replication files in a
|
||||
non-erasure-coded directory, or using `FSDataOutputStreamBuilder#replicate()`
|
||||
API to create 3x replication files in an erasure-coded directory.
|
||||
|
|
Loading…
Reference in New Issue