Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

HADOOP-16027. [DOC] Effective use of FS instances during S3A integration tests. Contributed by Gabor Bota.

This commit is contained in:
Sean Mackrory 2019-01-09 10:57:58 -07:00
parent 709ddb10e5
commit 3420e26ae5
1 changed files with 14 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

15
hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/testing.md
View File

@ -570,10 +570,23 @@ rather than write new tests. When doing this, make sure that the new predicates
fail with meaningful diagnostics, so any new problems can be easily debugged fail with meaningful diagnostics, so any new problems can be easily debugged
from test logs. from test logs.
***Effective use of FS instances during S3A integration tests.*** Tests using
`FileSystem` instances are fastest if they can recycle the existing FS
instance from the same JVM.
If you do that, you MUST NOT close or do unique configuration on them.
If you want a guarantee of 100% isolation or an instance with unique config,
create a new instance which you MUST close in the teardown to avoid leakage
of resources.
Do NOT add `FileSystem` instances manually
(with e.g `org.apache.hadoop.fs.FileSystem#addFileSystemForTesting`) to the
cache that will be modified or closed during the test runs. This can cause
other tests to fail when using the same modified or closed FS instance.
For more details see HADOOP-15819.
## <a name="requirements"></a> Requirements of new Tests ## <a name="requirements"></a> Requirements of new Tests
This is what we expect from new tests; they're an extension of the normal This is what we expect from new tests; they're an extension of the normal
Hadoop requirements, based on the need to work with remote servers whose Hadoop requirements, based on the need to work with remote servers whose
use requires the presence of secret credentials, where tests may be slow, use requires the presence of secret credentials, where tests may be slow,