Commit Graph

25 Commits

Author SHA1 Message Date
Lehel 7e97c1ce19
NIFI-11045: Sensitive dynamic property support for parameterized queries in ExecuteSQL and ExecuteSQLRecord
This closes #6853.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2023-02-22 12:37:07 +01:00
exceptionfactory 1156f4cbc5
NIFI-11165 This closes #6944. Removed optional jigsaw build profiles
- Removed jigsaw build profiles
- Removed Java 11 directories from assemblies

Signed-off-by: Joe Witt <joewitt@apache.org>
2023-02-13 13:01:58 -05:00
dan-s1 e86786433a
NIFI-11108 Further changes to upgrade from JUnit 4 to JUnit 5.
Signed-off-by: Chris Sampson <chris.sampson82@gmail.com>

This closes #6919
2023-02-02 12:27:42 +00:00
Matthew Burgess 5f1d93f977 NIFI-5819: Support SQLServer sql_variant type
This closes #6699.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2022-11-24 16:57:50 +01:00
Matthew Burgess 639ab328c9
NIFI-10057: Add support for SQL Server's DATETIMEOFFSET type to ExecuteSQL processors
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #6080.
2022-06-03 10:33:41 +02:00
Mike Thomsen ae1731c79e
NIFI-9134 nifi-metrics-reporting-bundle to use JUnit 5
- NIFI-9129 Refactored as much of nifi-kudu-bundle to use JUnit 5 as possible without breaking the integration tests which use test libraries from the Kudu project that depend on JUnit 4
- NIFI-9109 Refactored nifi-extension-utils to use JUnit 5. Left two tests with JUnit 4 because they used APIs that don't appear to have analogues in JUnit 5
- NIFI-9112 Refactored nifi-groovyx-bundle to use JUnit 5
- NIFI-9142 Refactored nifi-ranger-bundle to use JUnit 5
- NIFI-9136 Refactored nifi-network-bundle to use JUnit 5

This closes #5794

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-02-26 08:18:08 -06:00
Matthew Burgess bcc8d03314
NIFI-9585 Upgraded H2 from 1.4 to 2.1.210
- Added nifi-h2-database module shading H2 1.4.200
- Implemented version checking and migration

This closes #5724

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-02-25 19:20:48 -06:00
Peter Gyori 38d26dafa3
NIFI-9192: ResultSetRecordSet considers value of useLogicalTypes flag when determining the object's schema.
NIFI-9192: ResultSetRecordSet consider value of useLogicalType flag when sqlType is not handled separately (default) and readerSchema is not null

Signed-off-by: Peter Gyori <peter.gyori.dev@gmail.com>

NIFI-9192: Unit tests added to test the use of useLogicalType flag

Signed-off-by: Peter Gyori <peter.gyori.dev@gmail.com>

NIFI-9192: ResultSetRecordSet consider not using logical type when sqlType is OTHER

NIFI-9192: Fixed checkstyle violations

Signed-off-by: Peter Gyori <peter.gyori.dev@gmail.com>

NIFI-9192: Renamed JdbcCommon.getUseLogicalTypes to isUseLogicalTypes

Signed-off-by: Peter Gyori <peter.gyori.dev@gmail.com>

NIFI-9192: Added unit tests for Array types with and without using logical types

Signed-off-by: Peter Gyori <peter.gyori.dev@gmail.com>

NIFI-9192: Review comments considered and unit tests refactored

NIFI-9192: Added further unit tests

NIFI-9192: Minor modifications based on review comments

Signed-off-by: Peter Gyori <peter.gyori.dev@gmail.com>
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5388
2021-09-27 16:27:39 -04:00
exceptionfactory 05d7867a16
NIFI-8768 Added toLocalDate() for convertType() handling of DATE fields
- Updated PutKudu to use DataTypeUtils.toLocalDate() for DATE fields
- Updated PutDatabaseRecord to remove convertDateToLocalTZ() since convertType() uses toLocalDate()
- Updated PutElasticsearchHttpRecord to use default time zone format for DATE fields
- Updated WriteXMLResult to use default time zone format for DATE fields
- Updated WriteJsonResult to use default time zone format for DATE fields
- Updated AvroTypeUtil to use toLocalDate() for logical DATE fields
- Updated JdbcCommon to avoid conversion to UTC for logical DATE fields
- Updated Processor and RecordReader unit tests for consistency in DATE comparison

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5210
2021-07-13 16:17:36 -04:00
Denes Arvay 4fde9648dd
NIFI-8759 ExecuteSQL and ExecuteSQLRecord unnecessarily fall back to default decimal scale
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5212
2021-07-13 13:27:58 -04:00
Matthew Burgess a5dbf56114
NIFI-8368: If decimal scale > precision, set precision = scale
This closes #4938

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-03-29 10:53:46 -05:00
Matthew Burgess 29ebec071e
NIFI-4974: Add try/catch around blob.free() to support JDBC spec
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4861.
2021-03-07 18:51:07 +04:00
Christian Gumpert f2555f27f1
NIFI-8119: properly free database resources when done with the processing of BLOBs
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4791
2021-02-23 16:47:32 -05:00
Peter Turcsanyi 67d06003b7 NIFI-8023: Convert java.sql.Date between UTC/local time zone normalized forms before/after database operations
This closes #4781

Signed-off-by: David Handermann <exceptionfactory@gmail.com>
2021-01-26 14:39:02 -06:00
Matthew Burgess 382439c1d0 NIFI-7973: Add default precision and scale properties to remaining SQL-based components
This closes #4682.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2021-01-13 08:41:12 -05:00
zhangcheng e18b4f0c75
NIFI-7410 Update JdbcCommon.java when javaSqlType is CLOB or NCLOB in convertToAvroStream method, use the DataTypeUtils.toString(clob/nClob,(String) null, StandardCharsets.UTF_8)) method to get the String
NIFI-7410 Update JdbcCommon.java when javaSqlType is CLOB or NCLOB in convertToAvroStream method, use the CharacterStream rto read the value of CLOB
NIFI-7410 Add a unit test. validate if it's unreadable when the clob value is Chinese, Japanese, and Korean.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4243.
2020-07-17 14:39:30 +02:00
Matthew Burgess ace5754524 NIFI-7208: Restore default timezone in JdbcCommon 2020-03-13 15:00:24 -04:00
Matthew Burgess 74b1b2fc59 NIFI-7208: Fixed PutSQL/JdbcCommon handling of timestamps (nanoseconds, e.g.) 2020-03-04 11:46:47 -05:00
Atul Mahajan e3f72c5ed0
NIFI-6771 : Added fix to support Oracle Binary float and Binary double conversion to Avro type
Fixed CheckStyle errors

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3815
2019-11-06 08:57:31 -05:00
Atul Mahajan 7ac41576e3
NIFI-6775 Added fix to support Oracle Float type conversion to Avro
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3812
2019-11-05 16:52:20 -05:00
Mark Payne bcf373a049
NIFI-6382: Allow Parameters to have null values
Signed-off-by: Bryan Bende <bbende@apache.org>
2019-08-16 16:47:27 -04:00
Jeff Storck b938fc20a3
NIFI-6479 Fixes timezone issues in TestJdbcCommon
Dates calculated for method testConvertToAvroStreamForDateTimeAsLogicalType now handle timezone correctly
Added debug logging of date/time calculations and expectations

This closes #3630.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2019-08-09 16:29:40 +09:00
Jeff Storck 1d560e2b02 NIFI-6360 Updated Mockito to 2.28.2, PowerMock to 2.0.2
Fixed test failures in nifi-couchbase-processors, BinaryDocument matcher replaced with ByteArrayDocument
Fixed test failures in nifi-riemann-processors, anyInt() matcher replaced with anyLong() matcher, calling method passes a long, not int
Removed unnecessary method mocks from nifi-toolkit-tls tests, TlsCertificateAuthorityServiceHandlerTest and TlsCertificateSigningRequestPerformerTest, since those were flagged by Mockito as unnecessary (they're unused)
Removed explicit mockito dependency version in nifi-gcp-processors pom to inherit version from nifi's pom.xml
Updated ArgumentMatchers in Kafka 0.10, 0.11, 1.0, and 2.0 processor tests, since in Mockito 2.x, the "any" matchers no longer allow nulls
Updated ArgumentMatchers in nifi-jolt-transform-json-ui, since in Mockito 2.x, the "any" matchers no longer allow nulls
Removed unnecessary method mocks from MetricsReportingTaskTest
Updated TestStandardRemoteGroupPort to return Long instead of Integer for test flowfile.size() invocations
Updated AbstractCassandraProcessor to include keyspaceProperty.getValue() in null check
Updated SimpleProcessLogger and TestSimpleProcessLogger, vararg matching does not work the same in Java 8 and 11
Updated TestStandardProcessScheduler to allow null values during mock invocations, Mockito 2.x no longer allows nulls in those matchers
Updated TestPutHiveStreaming to allow null values during mock invocations, Mockito 2.x no longer allows nulls in those matchers
Updated FetchParquetTest to allow null values during mock invocations, Mockito 2.x no longer allows nulls in those matchers
Updated ControllerSearchServiceTest to allow null values during mock invocations, Mockito 2.x no longer allows nulls in those matchers
Removed usage of Whitebox from GetAzureEventHubTest due to Mockito 2.x, replaced with FieldUtils
Removed usage of Whitebox from StandardOidcIdentityProviderTest due to Mockito 2.x, replaced with FieldUtils
Updated apache-rat-plugin configuration in root POM to make use of useIdeaDefaultExcludes which makes the rat plugin exclude IntelliJ artifacts
Updated several modules to use mockito-core instead of mockito-all (discontinued in Mockito 2.x)
Updated nifi-site-to-site-reporting-task tests to be compatible with Mockito 2.x
Ignored TestPutJMS tests; the tests need to be refactored to work with Mockito 2.x, but the processor is deprecated.  Refactor may be done in a separate PR.
Adjusted several mock interaction iterations to 0 for TestPublishKafkaRecord_* tests.  Mockito 2.x flagged several interactions as unused and were adjusted to 0 interactions.
Updated PowerMock and Mockito dependencies to exclude transitive dependency on bytebuddy, added explicit dependency on bytebuddy 1.9.10 so that PowerMock and Mockito use the same version.  Bytebuddy 1.9.3 (used by PowerMock 2.0.2) did not allow for the mocking of final/private classes, bytebuddy 1.9.10 (used by Mockito 2.28.2) does.
Updated TestSiteToSiteProvenanceReportingTask use of InvocationOnMock.getArgument to use objects for the resulting object rather than primitives
Removed unnecessary stubs from evtx tests, Mockito 2.x defaults to strict mocks
Fixed classloader issue with tests in nifi-windows-event-log-processors module that use JNAJUnitRunner when Mockito mocked JNA classes (Kernel32)
Addressed Mockito-related deprecation warnings
Import cleanup

This closes #3533

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2019-06-17 12:21:07 -04:00
Aaron Leon 81ddd02ca8 NIFI-6100 Use setBytes in JdbcCommon for binary types
This closes #3410

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2019-05-21 14:22:56 -04:00
Matthew Burgess ca76fe178c NIFI-6082: Added DatabaseRecordLookupService, refactored common DB utils
NIFI-6082: Added SimpleDatabaseLookupService

NIFI-6082: Merged Koji's improvements, incorporated review comments

This closes #3341.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2019-03-14 09:33:30 +09:00