Commit Graph

402 Commits

Author SHA1 Message Date
Bence Simon 7340bb8153
NIFI-11924 Closing FileSystem after using in HDFSExternalResourceProvider
This closes #7588.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2023-08-22 13:18:10 +02:00
exceptionfactory 84326ba4b2 NIFI-11920 Improved JDBC and JNDI JMS Connection URL Validation
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #7586
2023-08-09 13:00:55 -04:00
Mike Thomsen 5cb15b484c
NIFI-11913 Added enum handling to StandardSchemaValidator
This closes #7576

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-08-07 13:58:43 -05:00
exceptionfactory 5c577be946
NIFI-11889 Added Record-oriented Transmission to PutTCP
This closes #7554
Signed-off-by: Paul Grey <greyp@apache.org>
2023-08-04 11:25:23 -04:00
Pierre Villard 319c974e7c
NIFI-11828 - Confluent Schema Encoding Access Strategy - Schema ID versus Schema Version ID (#7495)
* NIFI-11828 - Confluent Schema Encoding Access Strategy - Schema ID versus Schema Version ID
* removed hard coded version 1
2023-07-25 09:42:50 -04:00
exceptionfactory 16c4004d58 NIFI-10728 Upgraded Apache Derby from 10.14.2.0 to 10.16.1.1
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #7497
2023-07-18 18:29:20 -04:00
Peter Turcsanyi 437995b75a
NIFI-11758: Added FileResourceService and used it in PutAzure*Storage processors for local file upload
- Renamed classes from DataUpload to ResourceTransfer and updated references
- Disabled testNonReadableFile() on Windows due to Posix permissions
- Replaced utility methods with functional handling of FileResource
- Corrected FlowFile InputStream access using Optional.orElseGet()

This closes: #7458

Co-authored-by: David Handermann <exceptionfactory@apache.org>
Signed-off-by: Nandor Soma Abonyi <nsabonyi@apache.org>
2023-07-14 23:48:27 +02:00
Chris Sampson 3f5ed23504
NIFI-9206 Added RemoveRecordField Processor
This closes #6816

Co-authored-by: Peter Gyori <peter.gyori.dev@gmail.com>
Co-authored-by: Chris Sampson <chris.sampson82@gmail.com>
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-06-26 09:50:41 -05:00
exceptionfactory ca6c7f7041 NIFI-11532 Removed JUnit 4 and Groovy Test from default dependencies
- Removed hamcrest-all from default dependencies
- Added groovy-test to selected modules with Groovy test classes
- Added junit-vintage-engine to selected modules with JUnit 4 test classes
- Corrected references to JUnit 4 assertions in JUnit 5 test classes
- Removed several unnecessary test classes from nifi-socket-utils
- Removed duplicative Registry toolkit test classes
- Removed Kudu integration tests

NIFI-11532 Corrected scope for junit-vintage-engine for Elasticsearch

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #7233
2023-06-13 15:36:23 -04:00
exceptionfactory 4faf3ea598
NIFI-11653 This closes #7349. Added Connection URL Validation for Database Services
Signed-off-by: Joe Witt <joewitt@apache.org>
2023-06-06 09:29:48 -07:00
Mark Payne 50811660d0
NIFI-11603 Refactored Socket-based tests to use Dynamic Ports
- Removed NetworkUtils methods for getting available ports
- Updated Socket-based components to support using 0 to listen on a random available port for improved test reliability

This closes #7299

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-06-01 13:33:06 -05:00
Mark Payne ac810671c5 NIFI-11621: Handle the case of CHOICE fields when inferring the type of ARRAY elements. E.g., support ARRAY<CHOICE<STRING, NULL>>
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #7322
2023-05-31 17:07:39 -04:00
Peter Turcsanyi 3bd4b49abe NIFI-11369: Fixed Defragment strategy with optional fragment.count attribute in MergeContent 2023-05-30 16:25:28 +02:00
Lehel 1a38bf003e
NIFI-11547 Fixed DBCPConnectionPool verification
- Added unit test for AbstractDBCPConnectionPool

This closes #7249

Co-authored-by: David Handermann <exceptionfactory@apache.org>
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-05-17 10:51:29 -05:00
Pierre Villard 7fb2468f4b NIFI-11502 - Upgrade json-path to 2.8.0
NIFI-11502 fixing unit tests
NIFI-11502 review

This closes #7208

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2023-04-29 16:03:16 -04:00
Mark Payne 7f7cccfdc2
NIFI-11241 Initial implementation of Python-based Processor API with Py4J
- Updated GitHub workflow so that system tests include Python 3.9
- Updated GitHub actions to build necessary modules for system tests

This closes #7003

Co-authored-by: David Handermann <exceptionfactory@apache.org>
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-04-14 15:03:34 -05:00
Pierre Villard b0997f843d
NIFI-11416 Upgraded Caffeine from 2.8.1 to 3.1.6
This closes #7147

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-04-10 08:45:12 -05:00
Mark Bathori d15eeb445b
NIFI-11342: HDFS processors fail to get ClassloaderIsolationKey at startup
This closes #7121.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2023-04-05 15:37:28 +02:00
Joe Witt 58517e4dd0
NIFI-11329 Updated NAR Plugin to 1.5.0 and Commons Compress to 1.23.0
This closes #7077

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-23 12:28:40 -05:00
Nandor Soma Abonyi 4e602ce40f
NIFI-11264 Added debug logs for List Processor yielding
- Corrected ListFile Processor description

This closes #7027

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-21 08:04:59 -05:00
exceptionfactory 512155ba2f
NIFI-10975 Add Kubernetes Leader Election and State Provider (#6779)
* NIFI-10975 Added Kubernetes Leader Election and State Provider
- Added Kubernetes Leader Election Manager based on Kubernetes Leases
- Added Kubernetes State Provider based on Kubernetes ConfigMaps
- Added nifi-kubernetes-client for generalized access to Fabric8 Kubernetes Client
- Added nifi.cluster.leader.election.implementation Property defaulting to CuratorLeaderElectionManager
- Refactored LeaderElectionManager to nifi-framework-api for Extension Discovering Manager
- Refactored shared ZooKeeper configuration to nifi-framework-cluster-zookeeper

* NIFI-10975 Updated Kubernetes Client and StateMap
- Upgraded Kubernetes Client from 6.2.0 to 6.3.0
- Added getStateVersion to StateMap and deprecated getVersion
- Updated Docker start.sh with additional properties

* NIFI-10975 Corrected MockStateManager.assertStateSet()
* NIFI-10975 Upgraded Kubernetes Client from 6.3.0 to 6.3.1
* NIFI-10975 Corrected unregister leader and disabled release on cancel

* NIFI-10975 Corrected findLeader handling of Lease expiration
- Changed LeaderElectionManager.getLeader() return to Optional String

* NIFI-10975 Corrected StandardNiFiServiceFacade handling of Optional Leader
* NIFI-10975 Changed getLeader() to call findLeader() to avoid stale cached values
* NIFI-10975 Updated LeaderElectionCommand to run LeaderElector in loop
* NIFI-10975 Rebased on project version 2.0.0-SNAPSHOT

* NIFI-10975 Corrected Gson and AspectJ versions
- Updated versions to match current main branch and avoid reverting
2023-03-07 14:14:24 -05:00
Lehel Boér 13d343d5ee
NIFI-11151: Improving code reusability of DBCP services
This closes #6935.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2023-03-07 14:41:34 +01:00
Paul Grey 288031f5f6
NIFI-11210 Added read timeout to DistributedMapCacheClientService
This closes #6994

Co-authored-by: David Handermann <exceptionfactory@apache.org>
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-03 15:00:28 -06:00
exceptionfactory 48689a2567
NIFI-11195 Refactored Identity Mapping to nifi-security-identity
- Moved StringUtils from nifi-properties to nifi-property-utils
- Moved Peer Identity methods from CertificateUtils to specific Site-to-Site classes

Signed-off-by: Joe Gresock <jgresock@gmail.com>
This closes #6977.
2023-02-23 10:33:45 -05:00
Lehel 7e97c1ce19
NIFI-11045: Sensitive dynamic property support for parameterized queries in ExecuteSQL and ExecuteSQLRecord
This closes #6853.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2023-02-22 12:37:07 +01:00
exceptionfactory 1156f4cbc5
NIFI-11165 This closes #6944. Removed optional jigsaw build profiles
- Removed jigsaw build profiles
- Removed Java 11 directories from assemblies

Signed-off-by: Joe Witt <joewitt@apache.org>
2023-02-13 13:01:58 -05:00
Joe Witt 43eab4c81d
NIFI-11103 prepping for 2.0.0 line 2023-02-09 15:32:53 -07:00
Mark Bathori b375f4b138
NIFI-11124: Add hadoop.file.url attribute to HDFS processors
This closes #6916.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2023-02-07 23:10:16 +01:00
exceptionfactory 6877e84931
NIFI-11135 Upgraded Commons CSV from 1.9.0 to 1.10.0
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #6921.
2023-02-03 12:26:47 +01:00
dan-s1 e86786433a
NIFI-11108 Further changes to upgrade from JUnit 4 to JUnit 5.
Signed-off-by: Chris Sampson <chris.sampson82@gmail.com>

This closes #6919
2023-02-02 12:27:42 +00:00
krisztina-zsihovszki 96a6594680
NIFI-11068 Introduced ConflictResolutionStrategy enum
This closes #6861

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-01-26 15:34:43 -06:00
Peter Turcsanyi cb77a96059
NIFI-11088: This closes #6878. updated copyright year to 2023 in NOTICE files
Signed-off-by: Joe Witt <joewitt@apache.org>
2023-01-23 09:42:44 -07:00
Matthew Burgess 6adbc1aeb0
NIFI-11036: Add Cluster Summary Metrics to Prometheus endpoint
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #6834.
2023-01-17 14:44:33 +01:00
Lehel Boér ed6ba53724 NIFI-10832: Create PutSalesforceRecord processor
This closes #6670.

Signed-off-by: Tamas Palfy <tpalfy@apache.org>
2022-12-13 18:59:41 +01:00
sopan98 ee5d557d2a
NIFI-10891 Replaced HashMap with LinkedHashMap in BinFiles
- Resolved non-deterministic behavior in TestMergeContent.testDefragmentMultipleMingledSegments

This closes #6727

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-11-29 14:29:08 -06:00
sopan98 ab8e9aa8df
NIFI-10890 Replaced HashMap with LinkedHashMap in JsonRowRecordReader
- Updated unit test based on deterministic behavior of LinkedHashMap

This closes #6726

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-11-29 14:26:58 -06:00
Joe Witt 1046b2c308
Merge branch 'NIFI-10854-RC1' 2022-11-28 08:43:56 -07:00
Matthew Burgess 5f1d93f977 NIFI-5819: Support SQLServer sql_variant type
This closes #6699.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2022-11-24 16:57:50 +01:00
Joe Witt 75e7c9e47c
NIFI-10854-RC1 prepare for next development iteration 2022-11-22 19:22:47 -07:00
Joe Witt ec87bf93ad
NIFI-10854-RC1 prepare release nifi-1.19.0-RC1 2022-11-22 19:22:45 -07:00
Mark Payne 2126dbbe29
NIFI-10660: When converting Avro GenericRecord to a java Map, check GenericRecord's schema to see if field exists before calling Record.get().
NIFI-10660: Added unit tests; fixed bug to ensure that we use the desired field name when checking if it exists in schema

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6688
2022-11-21 15:09:31 -05:00
Robert Kalmar 89426d167b
NIFI-10370 Create PutSnowflake processor using Snowpipe ingest
This closes #6584.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2022-11-18 16:19:03 +01:00
Mark Payne d7f2eb7c26
NIFI-10818: Fixed bug in which oldest bin in MergeContent was not being evicted in order to make room for a new bin when using a correlation attribute. Added additional tests for JoinEnrichment to ensure that the changes had no adverse effects on that processor. Found a bug when running that test in MockProcessSession so also addressed the bug (ConcurrentModificationException).
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6668
2022-11-16 16:49:29 -05:00
Mark Payne bda624823b
NIFI-10817: Moved the calls in StandardProcessSession to 'resetState()' and 'acknowledgeRecords()' from the outer commit(boolean) to the inner commit(Checkpoint, boolean). By moving the call here, the logic of StandardProcessSession is unaffected. But the StatelessProcessSession that inherits from it now has the benefit of having the state cleaned up when calling super.commit(Checkpoint, boolean).
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6663
2022-11-15 15:47:32 -05:00
exceptionfactory 588e04cd07 NIFI-10780 Improved Event Server to avoid dropping messages
- Improved ByteArrayMessageChannelHandler to call BlockingQueue.offer() while the server is not shutting down
- Improved NettyEventServer to use ChannelFuture.await() for closing the channel to allow shutdown to proceed
- Added test method for EventDroppedException handling

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #6634.
2022-11-10 15:38:43 -05:00
exceptionfactory e60cbd4bbb
NIFI-10768 Filter empty messages in ListenSyslog (#6624)
- Added FilteringStrategy argument to Server Factory for ListenSyslog
- Set quick shutdown quiet period for ListenSyslog and TestListenSyslog
- Added test method for batched messages with empty message included
2022-11-07 15:05:40 -05:00
mr1716 438d8ec1eb NIFI-10688 Update javax.json-api & javax.json
This closes #6572

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-10-28 08:02:28 -04:00
mr1716 f23318a361
NIFI-10670 Update commons-csv from 1.8 to 1.9.0
This closes #6554

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-10-21 15:18:00 -05:00
Peter Turcsanyi 2be5c26f28
NIFI-10659: Extracted DBCP common code to nifi-dbcp-base module for use in Snowflake NAR
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6543
2022-10-18 17:04:14 -04:00
exceptionfactory 0dc02e26d2 NIFI-10648 Upgraded Apache Commons Text to 1.10.0
- Added org.apache.commons.text.version property in root Maven configuration
- Removed direct version references in favor of managed dependency version

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #6531.
2022-10-14 14:07:23 -04:00