- NIFI-9129 Refactored as much of nifi-kudu-bundle to use JUnit 5 as possible without breaking the integration tests which use test libraries from the Kudu project that depend on JUnit 4
- NIFI-9109 Refactored nifi-extension-utils to use JUnit 5. Left two tests with JUnit 4 because they used APIs that don't appear to have analogues in JUnit 5
- NIFI-9112 Refactored nifi-groovyx-bundle to use JUnit 5
- NIFI-9142 Refactored nifi-ranger-bundle to use JUnit 5
- NIFI-9136 Refactored nifi-network-bundle to use JUnit 5
This closes#5794
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Replaced nifi-processor-utils with nifi-utils where necessary
- Added direct dependencies on commons-lang3 where necessary
Signed-off-by: Joe Gresock <jgresock@gmail.com>
This closes#5693.
- Implemented override for Timestamp Record Field Type format handling to add support for optional microseconds
- Added FieldConverter and ObjectTimestampFieldConverter implementation for generalized Timestamp parsing using DateTimeFormatter
- Updated PutKudu unit tests for standard Timestamp and Timestamp with microseconds
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#5589.
* NIFI-9382: Created a new ClassloaderIsolationKey mechanism by which Hadoop related processors (and potentially others) can indicate that they need full classloaders to be cloned but can share with other instances in certain circumstances
- Added system tests
* NIFI-9382: Renamed interface based on review feedback
* NIFI-9382: Removed ReentrantKerberosUser.
NIFI-8974 Integrate KerberosUserService with HDFS processors
NIFI-8980 Integrate KerberosUserService with Kafka 2.6 processors
- Introduced SelfContainerKerberosUserService to restrict which impls can be used with Kafka
- Add variations of KerberosUser doAs that allow setting the context ClassLoader
- Add additional unit tests for configurations
This closes#5277
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Updated PutKudu to use DataTypeUtils.toLocalDate() for DATE fields
- Updated PutDatabaseRecord to remove convertDateToLocalTZ() since convertType() uses toLocalDate()
- Updated PutElasticsearchHttpRecord to use default time zone format for DATE fields
- Updated WriteXMLResult to use default time zone format for DATE fields
- Updated WriteJsonResult to use default time zone format for DATE fields
- Updated AvroTypeUtil to use toLocalDate() for logical DATE fields
- Updated JdbcCommon to avoid conversion to UTC for logical DATE fields
- Updated Processor and RecordReader unit tests for consistency in DATE comparison
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#5210
- Added getDateFormat() using default time zone instead of GMT time zone from DataTypeUtils.getDateFormat()
NIFI-8748 Adjusted Date Format to use DataType.getFormat()
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#5194.
As of KUDU-1884, Kudu supports custom Kerberos principals on server-side
and custom SASL protocol (service) names on client-side which must match
the SPN base, i.e. if the SPN is kudu/_HOST, SASL protocol name *must*
be "kudu" in the client to be able to connect to the cluster.
This patch adds the ability to configure this in the PutKudu processor.
Signed-off-by: Attila Bukor <abukor@apache.org>
- Implemented custom ThreadPoolExecutor with maximum pool size based on Worker Count property
- Refactored processing methods to ensure KuduSession is always closed
- Added SystemResourceConsideration for Memory
- Removed duplicate dependency on nifi-security-kerberos
- Adjusted method naming to clarify functionality
- Reverted addition of defaultAdminOperationTimeoutMs()
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#5020.
- Created nifi-security-socket-ssl
- Created nifi-security-kerberos
- Removed nifi-security-utils dependency from nifi-processor-utils
- Updated modules to reference new dependencies
- Eliminated unnecessary transitive dependencies on bcprov-jdk15on from over 30 modules
Signed-off-by: Nathan Gough <thenatog@gmail.com>
This closes#4881.
The Kudu versions for the controller service and processors
currently doesn’t match. This patch unifies the version in
the parent bundle pom to ensure they match going forward.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#4832.
- update Kudu dependencies to Kudu 1.13.0
- add support for passing java.sql.Date for Kudu DATE columns
- add tests for passing java.sql.Date to DATE columns
more about DATE type support in Kudu:
https://issues.apache.org/jira/browse/KUDU-2632
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#4549.
- update Kudu dependencies to Kudu 1.12.0
- add VARCHAR to Kudu Lookup Service and Processor
- add tests for VARCHAR columns
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#4347.
NIFI-7527 Fixed StackOverFlowError due to pacing issue (recursive login before loggedIn flag is set).
NIFI-7527 Refactor: removed redundant kudu client creation.
This closes#4330.
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
NIFI-7453 Creating a new Kudu client when refreshing TGT in KerberosPasswordUser as well. (Applied to KerberosKeytabUser only before.)
NIFI-7453 Safely closing old Kudu client before creating a new one.
NIFI-7453 Visibility adjustment.
This closes#4276.
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
Uses `DataTypeUtils.toTimestamp` when writing to Kudu
timestamp (`UNIXTIME_MICROS`) columns. This allows
us to use the `row.addTimestamp` API and get much more
intuitive and predictable timestamp ingest behavior.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#4054.
This patch adds validation to the PutKudu operation type property.
It also improves the description to include the valid values and
adjusts the inputs to be case insensitive.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#4063.
Adds a boolean property to the PutKudu processor to optionally
enable automatic schema drift handling.
If set to true, when fields with names that are not in the target
Kudu table are encountered, the Kudu table will be altered to
include new columns for those fields.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#4053.
Calls to `trigger()` may be called concurrently from different threads,
however the PutKudu processor is storing the `kuduSession`
in a class level field. This can result in the logging issue reported in
NIFI-6895 and likely other unusual anomolies including performace
issues depending on the processor configuration.
Additionally the `operationType` was also stored in a class level field
and could be set concurrently resulting in the incorrect operation type
used.
This patch fixes the issue by moving both kuduSession and operationType
to be local. Additionaly some minor code cleanup was included.
An integration test, ITPutKudu, was added and used to manual verify the
logging issue existed and is fixed by this patch. I ran the test using
`mvn -Pintegration-tests verify -Dtest=ITPutKudu`
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#3910.
Joe Witt
Mike Thomsen
exceptionfactory
markap14
Bryan Bende
Attila Bukor
Grant Henke
Greg Solovyev
abrown
Tamas Palfy
Bence Simon
Bryan Bende
kevinmccarthy