Commit Graph

151 Commits

Author SHA1 Message Date
Mark Payne ba599d29c2
NIFI-12480 Updated MapRecord's toString() method to use the SerializedForm of the record when available and fixed bugs around ensuring that the serialized form is properly set
This closes #8132

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-12-18 08:25:09 -06:00
Chris Sampson aac71c5aa1 NIFI-8135 allow CHOICE data types in conversion of Records to Java Maps
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #7746
2023-10-19 13:13:58 -04:00
Mark Payne 3ae0eedee6
NIFI-12124: This closes #7791. Added a new RenameRecordField processor. In testing, also noticed that the Descendant Wildcard operator (//*) and Descendant Field Path Operator (//name, for instance) did not properly account for array of records or map elements, so addressed those concerns.
Signed-off-by: Joseph Witt <joewitt@apache.org>
2023-10-13 14:02:55 -07:00
Matt Burgess e7da2ab5b8
NIFI-11177: Add defensive code for null values for Iceberg
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #7777.
2023-09-26 11:30:08 +02:00
Pierre Villard 5405a3b08b
NIFI-12102 Fixed deprecation warnings when instantiating primitive types
This closes #7769

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-09-21 13:41:30 -05:00
Tamas Neumer 80e71068be NIFI-11860 fixed bug in ResultSetRecordSet affecting decimalScale when resultSetScale was zero.
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #7545
2023-08-02 12:22:55 -04:00
Chris Sampson 3f5ed23504
NIFI-9206 Added RemoveRecordField Processor
This closes #6816

Co-authored-by: Peter Gyori <peter.gyori.dev@gmail.com>
Co-authored-by: Chris Sampson <chris.sampson82@gmail.com>
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-06-26 09:50:41 -05:00
Matt Burgess f3e8651522 NIFI-11647: Support UUID type in DataTypeUtils.getSQLTypeValue
This closes #7355

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2023-06-08 11:13:07 -04:00
krisztina-zsihovszki a36040fded
NIFI-11367 Database DATE value displayed incorrectly in record output if timezone is GMT-X
This closes #7103.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2023-04-02 22:47:42 +02:00
Joe Witt 43eab4c81d
NIFI-11103 prepping for 2.0.0 line 2023-02-09 15:32:53 -07:00
dan-s1 53371844a4
NIFI-11035 Replaced remaining JUnit 4 assertions in nifi-commons with JUnit 5
- Replaced Groovy asserts with JUnit 5 assertions and Groovy shouldFail method Junit 5 with assertThrow method

This closes #6880

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-01-24 15:30:53 -06:00
Lehel Boér ed6ba53724 NIFI-10832: Create PutSalesforceRecord processor
This closes #6670.

Signed-off-by: Tamas Palfy <tpalfy@apache.org>
2022-12-13 18:59:41 +01:00
Matt Burgess b744fac479
NIFI-10956: Fix inference issues with mixed arrays (#6763) 2022-12-08 09:28:19 -05:00
Joe Witt 75e7c9e47c
NIFI-10854-RC1 prepare for next development iteration 2022-11-22 19:22:47 -07:00
Joe Witt ec87bf93ad
NIFI-10854-RC1 prepare release nifi-1.19.0-RC1 2022-11-22 19:22:45 -07:00
Matthew Burgess a76abef270 NIFI-10635: Fix handling of enums in PutDatabaseRecord
Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #6518.
2022-10-17 11:29:53 -04:00
Mark Bathori e87bced147
NIFI-10442: Create PutIceberg processor
This closes #6368.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2022-10-11 21:51:18 +02:00
Joe Witt c9ebdd2025
NIFI-10521-RC4 prepare for next development iteration 2022-10-03 10:59:36 -07:00
Joe Witt 109e54cd58
NIFI-10521-RC4 prepare release nifi-1.18.0-RC4 2022-10-03 10:59:34 -07:00
Csaba Bejan 35fb66f50f
NIFI-10403 Add processor supporting the new BigQuery Write API
Signed-off-by: Arpad Boda <aboda@apache.org>

This closes #6344
2022-09-19 14:32:11 +02:00
Mark Payne ec09c56e93
NIFI-10508: When inferring data types for values, allow float and double to encapsulate byte/short/int/long values
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6421
2022-09-16 17:58:19 -04:00
Joe Witt 8c66bf948c
NIFI-10272-RC2 prepare for next development iteration 2022-07-27 13:24:09 -07:00
Joe Witt 8d256784d8
NIFI-10272-RC2 prepare release nifi-1.17.0-RC2 2022-07-27 13:24:06 -07:00
Mike Thomsen a3e8048b2d
NIFI-9981 Added support for Avro UUID types
This closes #6013

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-06-23 08:35:31 -05:00
exceptionfactory 5febd47c72
NIFI-10109 Changed HashMap to LinkedHashMap in JsonTreeRowRecordReader (#6112)
- Added unit test to confirm preservation of field ordering
2022-06-21 17:00:34 -04:00
Matthew Burgess 639ab328c9
NIFI-10057: Add support for SQL Server's DATETIMEOFFSET type to ExecuteSQL processors
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #6080.
2022-06-03 10:33:41 +02:00
Joe Witt 0419dc2939
NIFI-9780-RC3 prepare for next development iteration 2022-03-21 11:58:27 -07:00
Joe Witt b019a9191f
NIFI-9780-RC3 prepare release nifi-1.16.0-RC3 2022-03-21 11:58:09 -07:00
在红尘中成仙 8d5de6b292
NIFI-9818 fix flaky tests
This closes #5883

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-03-21 08:23:40 -04:00
Mark Payne 6a1c7c72d5
NIFI-9781: Fix handling when selecting array element via QueryRecord
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5853.
2022-03-09 13:07:15 -05:00
zhangcheng a85cafe771
NIFI-9064:Support Oracle timestamp when `Use Avro Logical Types` is true for ExecuteSQLRecord and QueryDatabaseTableRecord
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5807.
2022-03-02 11:18:58 -05:00
Mike Thomsen 4b41570d17
NIFI-9714 Added overloaded toMap to MapRecord that can convert sub-records into maps.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5784
2022-02-28 09:12:51 -05:00
Mark Payne ce0122bd25
NIFI-9691: Added ForkEnrichment, JoinEnrichment processors
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5772.
2022-02-22 18:54:32 -05:00
Pierre Villard 90930ca197
NIFI-9476 - Fix QueryRecord when no result and with array type column
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5594
2021-12-15 17:37:24 -05:00
exceptionfactory b7ad1f924d
NIFI-9457 Support microseconds for String Timestamps in PutKudu
- Implemented override for Timestamp Record Field Type format handling to add support for optional microseconds
- Added FieldConverter and ObjectTimestampFieldConverter implementation for generalized Timestamp parsing using DateTimeFormatter
- Updated PutKudu unit tests for standard Timestamp and Timestamp with microseconds

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5589.
2021-12-10 10:31:52 +01:00
Matthew Burgess fe4161b1c2
NIFI-8392: Translate JDBC CHAR type to RecordFieldType STRING
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5506.
2021-11-14 16:03:17 +01:00
Joe Witt c055895952
NIFI-9344-RC3 prepare for next development iteration 2021-11-03 08:53:37 -07:00
Joe Witt 7fdc07cccd
NIFI-9344-RC3 prepare release nifi-1.15.0-RC3 2021-11-03 08:53:32 -07:00
Mike Thomsen 8d513c5ed3
NIFI-9080 Converted nifi-commons to use JUnit 5
This closes #5332

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-10-25 21:07:54 -05:00
Peter Gyori 38d26dafa3
NIFI-9192: ResultSetRecordSet considers value of useLogicalTypes flag when determining the object's schema.
NIFI-9192: ResultSetRecordSet consider value of useLogicalType flag when sqlType is not handled separately (default) and readerSchema is not null

Signed-off-by: Peter Gyori <peter.gyori.dev@gmail.com>

NIFI-9192: Unit tests added to test the use of useLogicalType flag

Signed-off-by: Peter Gyori <peter.gyori.dev@gmail.com>

NIFI-9192: ResultSetRecordSet consider not using logical type when sqlType is OTHER

NIFI-9192: Fixed checkstyle violations

Signed-off-by: Peter Gyori <peter.gyori.dev@gmail.com>

NIFI-9192: Renamed JdbcCommon.getUseLogicalTypes to isUseLogicalTypes

Signed-off-by: Peter Gyori <peter.gyori.dev@gmail.com>

NIFI-9192: Added unit tests for Array types with and without using logical types

Signed-off-by: Peter Gyori <peter.gyori.dev@gmail.com>

NIFI-9192: Review comments considered and unit tests refactored

NIFI-9192: Added further unit tests

NIFI-9192: Minor modifications based on review comments

Signed-off-by: Peter Gyori <peter.gyori.dev@gmail.com>
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5388
2021-09-27 16:27:39 -04:00
Matthew Burgess 98e3b465c3
NIFI-8376: Gracefully handle SQL exceptions in ResultSetRecordSet
This closes #4951

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2021-09-08 09:11:26 -04:00
exceptionfactory 20804ff2b1
NIFI-8749 Removed implicit time zone conversion to GMT
- Updated DataTypeUtils.getDateFormat() to use system default time zone
- Updated Record Path Guide to match Expression Language Guide regarding default time zone
- Updated impacted unit tests to expect localized dates instead of dates converted to GMT

NIFI-8749 Set user.timezone in TestRecordPath to avoid unexpected date conversions

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5361
2021-08-31 16:07:53 -04:00
Matthew Burgess dfa6592a47
NIFI-8749 Revert "Removed implicit time zone conversion to GMT"
This reverts commit 758e1850ed.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>
2021-08-31 13:33:53 -04:00
exceptionfactory 758e1850ed
NIFI-8749 Removed implicit time zone conversion to GMT
- Updated DataTypeUtils.getDateFormat() to use system default time zone
- Updated Record Path Guide to match Expression Language Guide regarding default time zone
- Updated impacted unit tests to expect localized dates instead of dates converted to GMT

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5305
2021-08-31 10:40:57 -04:00
Joe Witt 7356332852
Merge branch 'NIFI-8767-RC2' 2021-07-14 09:07:22 -05:00
exceptionfactory 05d7867a16
NIFI-8768 Added toLocalDate() for convertType() handling of DATE fields
- Updated PutKudu to use DataTypeUtils.toLocalDate() for DATE fields
- Updated PutDatabaseRecord to remove convertDateToLocalTZ() since convertType() uses toLocalDate()
- Updated PutElasticsearchHttpRecord to use default time zone format for DATE fields
- Updated WriteXMLResult to use default time zone format for DATE fields
- Updated WriteJsonResult to use default time zone format for DATE fields
- Updated AvroTypeUtil to use toLocalDate() for logical DATE fields
- Updated JdbcCommon to avoid conversion to UTC for logical DATE fields
- Updated Processor and RecordReader unit tests for consistency in DATE comparison

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5210
2021-07-13 16:17:36 -04:00
Joe Witt 97feacc181
NIFI-8767-RC2 prepare for next development iteration 2021-07-10 12:17:09 -07:00
Joe Witt fcbf1d5f97
NIFI-8767-RC2 prepare release nifi-1.14.0-RC2 2021-07-10 12:17:05 -07:00
Sandro Tolaini fdda3582e3
NIFI-8470: Fix DataTypeUtils for CHAR() sequences
NIFI-8470: Added unit test

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5027
2021-05-11 17:46:38 -04:00
Matthew Burgess 30fc26647e
NIFI-6061: Fix CLOB/BLOB handling in PutDatabaseRecord
NIFI-6061: Force getBytes() in BLOB handling to use UTF-8 charset

NIFI-6061: Use setClob(), added unit tests, incorporated review comments

This closes #5049

Co-authored-by: zhangcheng <zhangchengk@foxmail.com>
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-05-04 14:58:14 -05:00