Commit Graph

36 Commits

Author SHA1 Message Date
EndzeitBegins 53bb995e4e
NIFI-12613 Renamed asDescribedValue() to asAllowableValue()
This closes #8260

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-01-17 22:16:29 -06:00
Mark Bathori c92034cda6 NIFI-12596: PutIceberg is missing case-insensitive Record type handling in List and Map types
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #8234
2024-01-16 17:30:17 -05:00
David Handermann 250fe90b34
NIFI-9458 Replaced SimpleDateFormat with DateTimeFormatter (#8248)
- Added FieldConverterRegistry and Standard implementation
- Moved DateTypeUtils methods to FieldConverter implementations
2024-01-16 16:39:40 -05:00
exceptionfactory 42547eb60c
NIFI-11703 Upgraded to Spring Framework 6 and Jetty 12
- Upgraded Spring Framework from 5.3.31 to 6.0.15
- Upgraded Spring Security from 5.8.7 to 6.2.0
- Upgraded Spring Vault from 2.3.4 to 3.1.0
- Upgraded Jetty from 10.0.18 to 12.0.5 with EE 10
- Upgraded Jersey from 2.41 to 3.1.4
- Upgraded JAXB from 2.3.9 to 4.0.4
- Upgraded AspectJ from 1.9.20.1 to 1.9.21
- Upgraded JMS API from 2.0.1 to 3.1.0
- Upgraded ActiveMQ Broker from 5.18.2 to 6.0.1 for JMS 3
- Upgraded JJWT from 0.9.1 to 0.12.3
- Replaced jackson-module-jaxb-annotations with jackson-module-jakarta-xmlbind-annotations
- Replaced maven-jaxb2-plugin with hisrc-higherjaxb40-maven-plugin 2.1.1
- Replaced kongchen swagger-maven-plugin with swagger-codegen-maven-plugin from Swagger 3
- Replaced com.nickwongdev AspectJ Plugin with Codehaus 1.14.0 for newer Java versions
- Removed unused cglib-nodep
- Removed references to javax.validation
- Removed custom Jetty ALPN Processor not required for Java 21
- Removed several tests depending on older Jetty and Jakarta libraries
- Removed unnecessary webdefault.xml configurations
- Replaced unsupported cross-context servlet forwarding with HTTP forwarding
- Replaced javax.servlet references with jakarta.servlet
- Replaced javax.xml.bind references with jakarta.xml.bind
- Replaced javax.ws references with jakarata.ws
- Updated Spring Security CSRF implementation for Spring Security 6
- Updated web.xml versions to 6.0
- Updated REST API templates using new swagger-codegen variables
- Removed VALIDATE_DATA property from ParseCEF based on library compatibility issue with javax.validation
- Added application URL logging to NiFi JettyServer

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #8197.
2024-01-04 14:01:32 +04:00
EndzeitBegins 499b63e544
NIFI-12545 Updated nifi-iceberg-bundle using current API methods
This closes #8186

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-12-26 16:46:04 -06:00
exceptionfactory e5e76d0161
NIFI-12393 Upgraded OWASP Check from 8.4.2 to 8.4.3
- Upgraded Azure SDK BOM from 1.2.17 to 1.2.18
- Upgraded Reactor Netty HTTP from 1.0.34 to 1.0.39 for Azure Identity
- Upgraded MSAL4J from 1.13.10 to 1.14.0
- Upgraded Box Java SDK from 4.4.0 to 4.6.1
- Relocated Apache Ant managed versions to bundle parent modules
- Added okio-fakefilesystem to managed dependencies
- Suppressed vulnerability for Picocli misidentified as LINE library
- Added managed dependencies to nifi-code-coverage to avoid false positives due to different parent modules

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #8054.
2023-11-21 12:16:12 +01:00
exceptionfactory 0e1ae2bd6f
NIFI-12276 Addressed Dependency Check Findings
- Added dependency-check GitHub workflow
- Upgraded Janino Commons Compiler from 3.1.9 to 3.1.10
- Upgraded Azure SDK BOM from 1.2.16 to 1.2.17
- Upgraded GCP SDK BOM from 26.17.0 to 26.25.0
- Upgraded AWS SDK from 1.12.550 to 1.12.573
- Upgraded Hazelcast from 5.3.2 to 5.3.5
- Upgraded Jersey from 2.40 to 2.41
- Upgraded Camel Salesforce from 3.14.5 to 3.14.9
- Unified ZooKeeper versioning on 3.9.1
- Applied Groovy 2.4.21 to Hive 3 and Iceberg components
- Applied gRPC version 1.59.0 to Asana components
- Applied Jettison 1.5.4 to Atlas and Hive 3 components
- Managed JUnit 4 version to 4.13.2 for MockWebServer
- Excluded HBase libraries from Hive 3 following Iceberg approach
- Excluded Htrace from HBase components
- Upgraded OWASP Dependency Check from 8.4.0 to 8.4.2
- Removed non-applicable dependency check suppressions
- Added dependency check suppressions for non-applicable findings

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #7948.
2023-10-27 16:21:38 -07:00
Mark Bathori 4fa0299f8b NIFI-12130: Ability to configure snapshot properties via dynamic attributes in PutIceberg
Fix dynamic field expression language handling

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #7849
2023-10-26 19:26:24 -04:00
Matt Burgess 3f7b1de6b8
NIFI-11739 - Add ability to ignore missing fields in PutIceberg
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #7421.
2023-10-03 14:41:12 +03:00
Matt Burgess e7da2ab5b8
NIFI-11177: Add defensive code for null values for Iceberg
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #7777.
2023-09-26 11:30:08 +02:00
Joseph Witt bd2276260e
NIFI-12051 Upgraded Groovy from 3.0.19 to 4.0.15
- Adjusted Groovy Maven coordinates to org.apache.groovy
- Adjusted build configuration and tests for Groovy 4

This closes #7692

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-09-15 13:38:24 -05:00
Mark Bathori ea4c2055d6
NIFI-12054: PutIceberg should produce a provenance send event
This closes #7690.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2023-09-14 16:53:32 +02:00
krisztina-zsihovszki 0446990d54
NIFI-11916 Iceberg processor extensibility improvement
This closes #7583.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2023-08-08 22:31:52 +02:00
exceptionfactory 16c4004d58 NIFI-10728 Upgraded Apache Derby from 10.14.2.0 to 10.16.1.1
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #7497
2023-07-18 18:29:20 -04:00
Mark Bathori b1be71f918
NIFI-11334: Fixed PutIceberg processor instance interference due to same class loader usage
This closes #7449.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2023-06-30 20:36:33 +02:00
Matthew Burgess 3ebe8f2983
NIFI-11538: This closes #7239. Fix primitive type conversion for PutIceberg 2023-06-01 12:29:38 -07:00
Matthew Burgess adb8420b48
NIFI-11552: Support FlowFile Attributes in some PutIceberg proeprties
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #7268.
2023-05-24 11:40:48 +02:00
krisztina-zsihovszki 3051b69a6c
NIFI-11544 Fixed REL_FAILURE usage in AbstractIcebergProcessor
This closes #7241.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2023-05-16 09:43:47 +02:00
Mark Bathori 44a7f7f38b
NIFI-11440 Speed up Iceberg Hive Metastore Tests
This closes #7170

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-04-14 12:52:22 -05:00
exceptionfactory dde89c0b15
NIFI-11358 Upgraded Hadoop from 3.3.4 to 3.3.5
- Upgraded Ranger Hadoop dependencies from 3.3.3 to 3.3.5
- Aligned Iceberg Hadoop version with project Hadoop version
- Updated Atlas dependencies to align hadoop-hdfs-client version with hadoop-common
- Updated Ranger hadoop-auth version to align with other Hadoop dependencies
- Updated Spark Livy bundle to align with project Hadoop version
- Removed unnecessary dependencies from Hive Test Utilities
- Updated HBase 2 Woodstox Core from 5.3.0 to 5.4.0
- Suppressed false positive vulnerabilities for HBase client libraries

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #7097.
2023-03-29 13:12:57 +02:00
exceptionfactory 5bdee9a714
NIFI-11347 This closes #7089. Upgraded OWASP Dependency Check from 8.0.2 to 8.2.1
- Updated suppression configuration
- Upgraded Solr from 8.6.3 to 8.11.1 for Ranger
- Excluded Apache Ivy from Hive and Janus Graph dependencies
- Excluded Groovy from Hive tests

Signed-off-by: Joe Witt <joewitt@apache.org>
2023-03-28 08:41:11 -07:00
exceptionfactory 623bcfd500
NIFI-11345 Adjusted Iceberg test to avoid expensive duplicative runs
This closes #7086

Signed-off-by: Nandor Soma Abonyi <nsabonyi@apache.org>
2023-03-28 01:35:48 +02:00
Mark Bathori b50c8101cf
NIFI-11215: Add custom validation for KerberosUserService in PutIceberg
This closes #6985.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2023-03-24 08:41:33 +01:00
Mark Bathori e370292d7f
NIFI-11204: Add configurable retry logic for table commits in PutIceberg processor
This closes #6976.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2023-03-20 19:20:43 +01:00
Mark Bathori 2873575fce
NIFI-11263: Add case-insensitive and order independent field mapping to PutIceberg record converter
This closes #7026.

Signed-off-by: Nandor Soma Abonyi <nsabonyi@apache.org>
2023-03-17 20:03:21 +01:00
Joe Witt 43eab4c81d
NIFI-11103 prepping for 2.0.0 line 2023-02-09 15:32:53 -07:00
Mark Bathori 869e2b3d66
NIFI-11110: Create processor for triggering HMS events
Review comment fixes

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6902
2023-02-02 11:31:44 -05:00
dan-s1 e86786433a
NIFI-11108 Further changes to upgrade from JUnit 4 to JUnit 5.
Signed-off-by: Chris Sampson <chris.sampson82@gmail.com>

This closes #6919
2023-02-02 12:27:42 +00:00
Mark Bathori 4b24bcd2f8
NIFI-10677: Add Choice data type handling to Iceberg record converter
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #6563.
2022-12-25 17:58:17 +01:00
Joe Witt 75e7c9e47c
NIFI-10854-RC1 prepare for next development iteration 2022-11-22 19:22:47 -07:00
Joe Witt ec87bf93ad
NIFI-10854-RC1 prepare release nifi-1.19.0-RC1 2022-11-22 19:22:45 -07:00
Mark Bathori 594e93d97e
NIFI-10812: Remove duplicate record template creation from Iceberg record converter
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6659
2022-11-20 17:26:49 -05:00
Mark Bathori 4083e07447
NIFI-10801: Fix hive metastore based tests in Iceberg processors
This closes #6656.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2022-11-15 20:12:37 +01:00
Joe Witt c155d29658
NIFI-10614 Upgraded Bouncy Castle from 1.70 to 1.71
- Blocked Bouncy Castle jdk15on dependencies
- Forced jdk18on dependencies and validated across entire build

This closes #6498

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-10-20 19:35:02 -05:00
exceptionfactory fcefd051d2
NIFI-10652 Corrected dependency scopes for nifi-iceberg-processors
- Marked nifi-iceberg-services-api as provided
- Marked multiple Hadoop and Hive dependencies as provided to avoid duplicate inclusion in nifi-iceberg-processors-nar

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6535
2022-10-18 13:25:11 -04:00
Mark Bathori e87bced147
NIFI-10442: Create PutIceberg processor
This closes #6368.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2022-10-11 21:51:18 +02:00