From 3420e26ae57f5946a913278a8a62ae82e930df88 Mon Sep 17 00:00:00 2001 From: Sean Mackrory Date: Wed, 9 Jan 2019 10:57:58 -0700 Subject: [PATCH] HADOOP-16027. [DOC] Effective use of FS instances during S3A integration tests. Contributed by Gabor Bota. --- .../src/site/markdown/tools/hadoop-aws/testing.md | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/testing.md b/hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/testing.md index fb141ddecc4..d6dbdc4c8f7 100644 --- a/hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/testing.md +++ b/hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/testing.md @@ -570,10 +570,23 @@ rather than write new tests. When doing this, make sure that the new predicates fail with meaningful diagnostics, so any new problems can be easily debugged from test logs. +***Effective use of FS instances during S3A integration tests.*** Tests using +`FileSystem` instances are fastest if they can recycle the existing FS +instance from the same JVM. + +If you do that, you MUST NOT close or do unique configuration on them. +If you want a guarantee of 100% isolation or an instance with unique config, +create a new instance which you MUST close in the teardown to avoid leakage +of resources. + +Do NOT add `FileSystem` instances manually +(with e.g `org.apache.hadoop.fs.FileSystem#addFileSystemForTesting`) to the +cache that will be modified or closed during the test runs. This can cause +other tests to fail when using the same modified or closed FS instance. +For more details see HADOOP-15819. ## Requirements of new Tests - This is what we expect from new tests; they're an extension of the normal Hadoop requirements, based on the need to work with remote servers whose use requires the presence of secret credentials, where tests may be slow,