Apache
/
nifi
mirror of https://github.com/apache/nifi.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

NIFI-4944: Guard against race condition in Snappy for PutHiveStreaming 

NIFI-4944: Removed unnecessary synchronized block, added more comments
This closes #2519
This commit is contained in:
Matthew Burgess 2018-03-07 15:19:39 -05:00 committed by Matt Gilman
parent 32a8db2253
commit d4632bdd5d
No known key found for this signature in database
GPG Key ID: DF61EC19432AEE37
1 changed files with 12 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
nifi-nar-bundles/nifi-hive-bundle/nifi-hive-processors/src/main/java/org/apache/nifi/processors/hive/PutHiveStreaming.java
View File

@ -63,6 +63,7 @@ import org.apache.nifi.util.hive.HiveConfigurator;
import org.apache.nifi.util.hive.HiveOptions;
import org.apache.nifi.util.hive.HiveUtils;
import org.apache.nifi.util.hive.HiveWriter;
import org.xerial.snappy.Snappy;
import java.io.ByteArrayOutputStream;
import java.io.File;
@ -137,6 +138,17 @@ public class PutHiveStreaming extends AbstractSessionFactoryProcessor {
private static final Set<String> RESERVED_METADATA;
static {
// This is used to prevent a race condition in Snappy 1.0.5 where two classloaders could
// try to define the native loader class at the same time, causing an error. Make a no-op
// call here to ensure Snappy's static initializers are called. Note that this block is
// called once by the extensions loader before any actual processor instances are created,
// so the race condition will not occur, and for each other instance, this is a no-op
try {
Snappy.compress("");
} catch (IOException ioe) {
// Do nothing here, should never happen as it is intended to be a no-op
}
Set<String> reservedMetadata = new HashSet<>();
reservedMetadata.add("avro.schema");
reservedMetadata.add("avro.codec");