5811 Commits

Author SHA1 Message Date
aglotero
d80a19e230 NIFI-5642: QueryCassandra processor : output FlowFiles as soon fetch_size is reached
NIFI-5642: QueryCassandra processor : output FlowFiles as soon fetch_size is reached
Fixed checkstyle error
Delete build.sh
Delete local build file
NIFI-5642 : letting fetch_size to control the Cassandra data flow creating a new MAX_ROWS_PER_FLOW_FILE parameter
Fixed checkstyle error: no more import java.util.*
Fixed missing imports
NIFI-5642: added REL_ORIGINAL relationship in order to allow incremental commit
Addressing comments from code review
Adjustments on timestamp datatype formatting
Created the OUTPUT_BATCH_SIZE property
Code review adjustments
NIFI-5642: update after rebase
NIFI-5642: addressing PR comments
NIFI-5642: adding in integration test, fixing race condition
NIFI-5642: remove log4j2

This closes #6848

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2023-04-05 16:01:54 -04:00
Mark Bathori
d15eeb445b
NIFI-11342: HDFS processors fail to get ClassloaderIsolationKey at startup
This closes #7121.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2023-04-05 15:37:28 +02:00
exceptionfactory
dffad44a03
NIFI-11377 This closes #7113. Upgraded Apache Iceberg from 1.1.0 to 1.2.0
Signed-off-by: Joe Witt <joewitt@apache.org>
2023-04-03 14:10:02 -07:00
Nandor Soma Abonyi
32df0fa484
NIFI-11137 Add record support to Consume/PublishJMS (#6987) 2023-04-03 21:42:27 +02:00
exceptionfactory
e4f0508c90
NIFI-11370 Corrected JWK Set retrieval for NIFI Trust Strategy (#7108)
- Added StandardOidcIdTokenDecoderFactory based on Spring Security OidcIdTokenDecoderFactory with custom REST Operations

Merged #7108 into main.
2023-03-31 16:26:02 -07:00
Matthew Burgess
59eebb9906
NIFI-8710: Pass details instead of source ID for Provenance per the interface doc
This closes #5224

Signed-off-by: Nandor Soma Abonyi <nsabonyi@apache.org>
2023-03-31 18:43:22 +02:00
exceptionfactory
75eb449a31
NIFI-11365 Corrected OIDC Redirect URI resolution for reverse proxies (#7104)
- Added Authorization Request Resolver with support for building the base redirect URI using allowed proxy headers

This closes #7104
2023-03-30 16:26:35 -04:00
Nandor Soma Abonyi
2b9f2071ed
NIFI-11270 Refactoring of the overly Paho-specific MQTT interface
This closes #7032.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2023-03-29 23:20:06 +02:00
exceptionfactory
8952f7c0cf NIFI-11354 Removed Neo4j 3 Service and modules
This closes #7093

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2023-03-29 15:41:24 -04:00
exceptionfactory
d66b29853e
NIFI-11359 Upgraded woodstox-core to 6.5.0 for Solr 8.11.1
Signed-off-by: Joe Witt <joewitt@apache.org>
2023-03-29 08:04:35 -07:00
Mark Payne
23d6d6ede4
NIFI-11251: Fixed case in which changing Flow Version was improperly attempting to use a Temporary Funnel destination for a connection due to differences in Group ID by instead checking if the destination is reachable; fixed issue in which processors in an inner versioned group were not stopped even though they changed, when a higher-level parent (that was itself under version control) had its version changed. Wrote system tests to verify
- Fixed system tests so that they work properly in Clustered version of RegistryClientIT
- Fixed system test - ensure that we wait for processors to become valid before attempting to start them; also added an additional system test around Controller Services in versioned flows

This closes #7095

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-29 08:30:18 -05:00
exceptionfactory
dde89c0b15
NIFI-11358 Upgraded Hadoop from 3.3.4 to 3.3.5
- Upgraded Ranger Hadoop dependencies from 3.3.3 to 3.3.5
- Aligned Iceberg Hadoop version with project Hadoop version
- Updated Atlas dependencies to align hadoop-hdfs-client version with hadoop-common
- Updated Ranger hadoop-auth version to align with other Hadoop dependencies
- Updated Spark Livy bundle to align with project Hadoop version
- Removed unnecessary dependencies from Hive Test Utilities
- Updated HBase 2 Woodstox Core from 5.3.0 to 5.4.0
- Suppressed false positive vulnerabilities for HBase client libraries

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #7097.
2023-03-29 13:12:57 +02:00
exceptionfactory
7cb20d6f7f
NIFI-11356 This closes #7096. Upgraded Nimbus JOSE JWT to 9.31
- Upgraded NiFi Framework from 9.24.3
- Upgraded NiFi Registry from 8.20
- Upgraded transitive msal4j dependency from 9.22
- Upgraded transitive hadoop-auth dependency from 9.8.1

Signed-off-by: Joe Witt <joewitt@apache.org>
2023-03-28 14:22:12 -07:00
exceptionfactory
5214097c59
NIFI-11355 Upgraded Couchbase Client from 2.5.8 to 2.7.23
- Suppressed vulnerability findings related to Couchbase Server

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #7094.
2023-03-28 22:05:55 +02:00
exceptionfactory
2dbb8f25f3
NIFI-11349 Upgraded HBase from 2.2.2 to 2.5.3 with Hadoop 3
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #7091.
2023-03-28 22:04:00 +02:00
exceptionfactory
158b5ffbcc
NIFI-11346 This closes #7088. Upgraded Parquet from 1.12.0 to 1.12.3
Signed-off-by: Joe Witt <joewitt@apache.org>
2023-03-28 08:41:45 -07:00
exceptionfactory
5bdee9a714
NIFI-11347 This closes #7089. Upgraded OWASP Dependency Check from 8.0.2 to 8.2.1
- Updated suppression configuration
- Upgraded Solr from 8.6.3 to 8.11.1 for Ranger
- Excluded Apache Ivy from Hive and Janus Graph dependencies
- Excluded Groovy from Hive tests

Signed-off-by: Joe Witt <joewitt@apache.org>
2023-03-28 08:41:11 -07:00
exceptionfactory
04cacad52a
NIFI-11348 This closes #7090. Upgraded JRuby from 9.3.9.0 to 9.4.2.0
Signed-off-by: Joe Witt <joewitt@apache.org>
2023-03-28 08:40:51 -07:00
exceptionfactory
26400fcbe9
NIFI-4890 Refactor OIDC with support for Refresh Tokens (#7013)
* NIFI-4890 Refactored OIDC with support for Refresh Tokens

- Implemented OIDC Authorization Code Grant Flow using Spring Security Filters
- Implemented OIDC RP-Initiated Logout 1.0
- Implemented OAuth2 Token Revocation RFC 7009 for Refresh Tokens
- Added OIDC Bearer Token Refresh Filter for updating application Bearer Tokens from Refresh Token exchanges
- Added configurable Token Refresh Window to application properties
- Removed original implementation and supporting classes

* NIFI-4890 Set Bearer Token expiration based on Access Token

* NIFI-4890 Corrected spelling and naming issues based on feedback

This closes #7013
2023-03-28 08:35:10 -04:00
exceptionfactory
623bcfd500
NIFI-11345 Adjusted Iceberg test to avoid expensive duplicative runs
This closes #7086

Signed-off-by: Nandor Soma Abonyi <nsabonyi@apache.org>
2023-03-28 01:35:48 +02:00
Mark Payne
94ae926c42
NIFI-11333: Do not allow components to be removed while a node is disconnected
This closes #7085

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-27 17:43:38 -05:00
Mark Payne
969fc50778
NIFI-11341 Fixed OutputStream.close() handling for Content Claims
Fixed issue in StandardContentClaimWriteCache in which inner OutputStream class did not have an idempotent close() method; as a result, the stream could be written to while already in use for another active FlowFile; fixed bug in ContentClaimInputStream in which skip() method ignored its own BufferedInputStream - this was discovered because it was causing failures in StandardProcessSessionIT; fixed bug in StandardProcessSessionIT in which the length of StandardContentClaim was being doubled because the OutputStream was setting the claim length but that is already handled at a lower level.

This closes #7087

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-27 16:05:50 -05:00
Christian Wahl
6bd893da16
NIFI-11232 Fixed buffer handling in ContentClaimInputStream
This closes #6996

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-27 16:02:27 -05:00
krisztina-zsihovszki
fe2721786c
NIFI-11266 PutGoogleDrive, ListGoogleDrive, FetchGoogleDrive can't access a SharedDrive
This closes #7058

Reviewed-by: Mark Bathori <bathori.mark@gmail.com>

Signed-off-by: Nandor Soma Abonyi <nsabonyi@apache.org>
2023-03-27 16:01:04 +02:00
Matt Burgess
1a9acb1f4b
NIFI-11337: Update PutDatabaseRecord's Max Batch Size property documentation to reflect actual behavior (#7081) 2023-03-24 10:22:56 -04:00
exceptionfactory
2698000a85
NIFI-11254 Upgraded SnakeYAML from 1.33 to 2.0
- Updated MiNiFi and EasyRules references to use required arguments

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #7084.
2023-03-24 11:11:37 +01:00
Mark Bathori
b50c8101cf
NIFI-11215: Add custom validation for KerberosUserService in PutIceberg
This closes #6985.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2023-03-24 08:41:33 +01:00
Bryan Bende
5386095083
NIFI-11332 Close Stream from Files.lines in StandardRuntimeManifestService
This closes #7079

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-23 14:29:17 -05:00
Joe Gresock
1b84927ed5
NIFI-11336 Added CapabilityDescription to PutAccumuloRecord
This closes #7080

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-23 13:57:23 -05:00
Emilio Setiadarma
cac76e339d
NIFI-9196 Corrected FetchSFTP Move File destination formatting
- Changed FetchSFTP to use String.format() instead of  File.getAbsolutePath() for defining the destination path as part of the Move File completion strategy

This closes #7072

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-23 13:57:23 -05:00
Joe Witt
58517e4dd0
NIFI-11329 Updated NAR Plugin to 1.5.0 and Commons Compress to 1.23.0
This closes #7077

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-23 12:28:40 -05:00
Mark Payne
016d58612c
NIFI-11290 Run Component Primary Node State changes in background thread
- Ensure that components are notified that primary node has changed in a background thread instead of the Leader Election thread and activate/deactivate the thread in the case of Processors so that they can be viewed in the UI and terminated

- Fixed system tests that would fail intermittently because they did not wait for node disconnection to complete and did not properly switch the client to look at the connected node before checking cluster status

This closes #7052

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-23 12:20:37 -05:00
Joe Gresock
9de93f19a4
NIFI-11330: Restarting service referenced by parameter only when service is active (#7078) 2023-03-23 13:13:22 -04:00
dependabot[bot]
24bbe51b54
NIFI-11328 Upgraded Jettison from 1.5.2 to 1.5.4
- Upgraded direct dependencies in Framework and Registry modules as well as Spark bundle

This closes #7073
This closes #7074
This closes #7075

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-23 10:48:20 -05:00
exceptionfactory
8154fcce49
NIFI-11325 Upgraded Mockito from 3.12.4 to 4.11.0
- Replaced deprecated Matchers references with ArgumentMatchers
- Removed unnecessary Mockito versions for Registry
- Refactored test configuration to Java for mocking

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #7071.
2023-03-23 09:56:15 +01:00
Matt Burgess
3bf1195f4b
NIFI-11305: Fixed out-of-order handling of binlog client shutdown in CaptureChangeMySQL (#7068)
NIFI-11305: Fixed out-of-order handling of binlog client shutdown in CaptureChangeMySQL
- Changed to allow 'unexpected' events to still be processed
2023-03-22 09:47:27 -04:00
Mark Payne
6adfb6131d
NIFI-11310: Provide appropriate classpath resources to the ReloadComponent when a processor is terminated
NIFI-11310: Fixed META-INF/services file that was mistakenly listing an extra extension point, due to rebase
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #7061
2023-03-21 14:29:44 -04:00
krisztina-zsihovszki
63e9365449
NIFI-4651 Add error message and code to PutSQL FlowFile attributes
NIFI-4651 Review comments

Signed-off-by: Matthew Burgess <mattyb149@apache.org>
2023-03-21 11:19:33 -04:00
Nandor Soma Abonyi
4e602ce40f
NIFI-11264 Added debug logs for List Processor yielding
- Corrected ListFile Processor description

This closes #7027

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-21 08:04:59 -05:00
Matt Burgess
0c7de2c748
NIFI-11305: Fix logic to correctly stop CaptureChangeMySQL (#7059) 2023-03-20 16:04:19 -04:00
Mark Bathori
e370292d7f
NIFI-11204: Add configurable retry logic for table commits in PutIceberg processor
This closes #6976.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2023-03-20 19:20:43 +01:00
simonbence
7954ff355c
NIFI-11213 Showing version change in older (pre 1.18.0) contained version flows properly (#7017) 2023-03-20 11:52:07 -04:00
Dustin Rodrigues
cb6e530a3c
NIFI-11301 Fixed typos in ISPEnrichIP and DistributeLoad
This closes #7056

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-20 08:54:01 -05:00
Peter Turcsanyi
4bec56ed81
NIFI-11296 Improved code reusability of GRPC modules
This closes #7053

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-17 15:25:58 -05:00
Mark Bathori
2873575fce
NIFI-11263: Add case-insensitive and order independent field mapping to PutIceberg record converter
This closes #7026.

Signed-off-by: Nandor Soma Abonyi <nsabonyi@apache.org>
2023-03-17 20:03:21 +01:00
mr1716
4d8c13b8e1
NIFI-10769 Updated Accumulo from 2.0.1 to 2.1.0
- Updated exclusions with Log4j 2 dependencies
- Removed exclusions no longer required based on Accumulo 2.1.0

This closes #6964

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-16 15:20:51 -05:00
johnAlJunior
40be70762a
NIFI-11269 Added snappy-java to nifi-avro-processors
This closes #7035

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-16 08:53:03 -05:00
Joe Gresock
8227c19b97
NIFI-11291: Ensuring exceptional EventIterators are closed in PartitionedEventStore
This closes #7047

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-16 08:26:32 -05:00
dan-s1
6c64050a29
NIFI-10792 Updated ConvertExcelToCSV to handle files over 10 MB
- Refactored to reduce the cognitive complexity in the onTrigger method
- Added log4j-to-slf4j library and adjusted formatting
- Removed unnecessary graphics2d dependency

This closes #7016

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-15 15:03:44 -05:00
Mark Payne
450a46bc58
NIFI-11289: Avoid obtaining read locks on queues when fetching Group Status, except in those few specific situations where it's needed.
This closes #7046

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-15 14:18:22 -05:00