Commit Graph

1335 Commits

Author SHA1 Message Date
Mark Bean 02653143d8
NIFI-11914 Support Expression Language for SegmentContent Size
This closes #7578

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-08-09 20:25:01 -05:00
Matt Burgess 88b6b587be
NIFI-11922: Honor catalog/schema field in UpdateDatabaseTable
This closes #7585

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-08-09 17:30:20 -05:00
Mark Payne b3c620411a
NIFI-11556: Added ability to use a Process Group as a Stateless Flow
When shutting down FlowController, wait up to gracefulShutdownSeconds seconds for the components to stop before shutting down thread pools. This allows for asynchronous operations such as disableControllerServicesAsync to complete during shutdown. Updated StandardStatelessFlow so that on shutdown it catches more general Exception to ensure that shutdown succeeds

Ensure that Max Concurrent Tasks cannot be set less than 1 for stateless group; fixed typo in ProcessGroupDTO's docs; on shutdown, we may need to disable controller services asynchronously. At that point, the thread pool used to do so may already be shutdown. If so, catch this and create a new single-thread pool, disable the service, and immediately shutdown the pool. Also, if we fail to disable services on shutdown of a stateless flow, instead of throwing an Exception, just log it and move on - it doesn't make much sense for shutdown() to throw an Exception in that case.

Updated system tests so that when emptying a queue, we check the result and if the queue still has data (because a Processor hasn't acknowledged the data, for example) then continue issuing request until the queue fully becomes empty.

When shutting down input/output ports for stateless group, ensure that we wait for the ports' active threads to complete before returning

When stopping StatelessGroupNode, ensure that all the Processors, Controller Services (even those not executed by the Stateless Engine) are stopped/disabled before considering the Stateless Group to be fully STOPPED.

This closes #7253

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-08-09 12:08:22 -05:00
Mark Payne 964f56043c
NIFI-11910: Clarified some points in docs
This closes #7574

Signed-off-by: Chris Sampson <chris.sampson82@gmail.com>
2023-08-08 08:10:49 +01:00
exceptionfactory 5c577be946
NIFI-11889 Added Record-oriented Transmission to PutTCP
This closes #7554
Signed-off-by: Paul Grey <greyp@apache.org>
2023-08-04 11:25:23 -04:00
Matt Burgess 63c72bd7e2
NIFI-11898 Handle commit based on driver capabilities in PutDatabaseRecord
This closes #7561

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-08-03 16:18:45 -05:00
exceptionfactory 3a078c0c31
NIFI-11780 Added Certificate SAN Attributes to HandleHttpRequest
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #7521.
2023-07-25 16:09:25 +02:00
exceptionfactory 8d5a6ce8d2
NIFI-11807 Reverted inadvertent changes to HandleHttpRequest
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-07-21 17:21:08 -05:00
Paul Grey 7db956fea7
NIFI-11825 Fixed QueryRecord closing of resources
This closes #7496

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-07-19 12:48:15 -05:00
Matt Burgess f6a14bc475
NIFI-11807 Added ExtractRecordSchema Processor
This closes #7482

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-07-19 12:32:59 -05:00
exceptionfactory 16c4004d58 NIFI-10728 Upgraded Apache Derby from 10.14.2.0 to 10.16.1.1
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #7497
2023-07-18 18:29:20 -04:00
Pierre Villard 150bfbe990
NIFI-11813 Removed Event Driven Scheduling Strategy
This closes #7483

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-07-18 12:22:52 -05:00
Joe Witt d2c70d1d2f
NIFI-11434 added support for optionally allowing zip entries with dat… (#7462)
* NIFI-11434 added support for optionally allowing zip entries with data descriptors
* set default value and ensured all zip tests use various configs
2023-07-12 17:09:03 -04:00
Matt Burgess f69b457757
NIFI-11770: Replace java-faker with Data Faker for GenerateRecord
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #7454.
2023-07-03 11:33:50 +02:00
Chris Sampson 3f5ed23504
NIFI-9206 Added RemoveRecordField Processor
This closes #6816

Co-authored-by: Peter Gyori <peter.gyori.dev@gmail.com>
Co-authored-by: Chris Sampson <chris.sampson82@gmail.com>
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-06-26 09:50:41 -05:00
exceptionfactory 1f1c5df5a3
NIFI-11673 Removed Legacy TLS Configuration Versions
This closes #7367

Signed-off-by: Nandor Soma Abonyi <nsabonyi@apache.org>
2023-06-26 11:10:39 +02:00
exceptionfactory 7d6af0dbb0 NIFI-11748 Upgraded Apache Commons Codec from 1.15 to 1.16.0
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #7428
2023-06-23 18:27:36 -04:00
exceptionfactory bdd7ef3671 NIFI-11744 Added Required Permission to Reference Remote Resources
- Applied Required Permission to components supporting URL Resource References

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #7426
2023-06-23 18:07:16 -04:00
mr1716 f62e1eb1e9
NIFI-11732 Upgraded Saxon from 12.1 to 12.2
- Adjusted spacing in test HTML to match Saxon 12.2 serialization

This closes #7414

Co-authored-by: David Handermann <exceptionfactory@apache.org>
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-06-21 11:01:45 -05:00
Matt Burgess 8a696c1dfa
NIFI-11715 Removed unused user-defined property support from PutDatabaseRecord and DistributedMapCacheLookupService
This closes #7395

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-06-16 16:23:43 -05:00
Matt Burgess bc6ac4b700
NIFI-11682 Correct Quoting for UPSERT and INSERT_IGNORE keys in PutDatabaseRecord
This closes #7385

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-06-16 15:05:03 -05:00
Matt Burgess 760949922c
NIFI-11691 Support VARBINARY and LONGVARBINARY in PutDatabaseRecord
This closes #7380

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-06-14 21:01:02 -05:00
Matthew Burgess 991e5e24de
NIFI-4957 Add Resource File Support for Jolt Specifications
This closes #4044

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-06-14 16:01:02 -05:00
mr1716 4b6209ed20
NIFI-11684 Upgraded Brotli4j from 1.11.0 to 1.12.0
This closes #7377

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-06-14 14:27:26 -05:00
Jeyassri Balchandran 8382b0a7a3
NIFI-11540 Removed legacy segment attributes from MergeContent
This closes #7305

Co-authored-by: David Handermann <exceptionfactory@apache.org>
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-06-13 18:37:48 -05:00
exceptionfactory ca6c7f7041 NIFI-11532 Removed JUnit 4 and Groovy Test from default dependencies
- Removed hamcrest-all from default dependencies
- Added groovy-test to selected modules with Groovy test classes
- Added junit-vintage-engine to selected modules with JUnit 4 test classes
- Corrected references to JUnit 4 assertions in JUnit 5 test classes
- Removed several unnecessary test classes from nifi-socket-utils
- Removed duplicative Registry toolkit test classes
- Removed Kudu integration tests

NIFI-11532 Corrected scope for junit-vintage-engine for Elasticsearch

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #7233
2023-06-13 15:36:23 -04:00
Paul Grey 45e8226f89
NIFI-11672 Adjusted ListSFTP to handle incomplete File Types
- Modified file handling to include both REGULAR and UNKNOWN File Types returned from an SFTP server

This closes #7365

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-06-12 11:25:22 -05:00
Matt Burgess 5e6f3abdd2
NIFI-11655 Fixed float and double handling in GenerateRecord
This closes #7356

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-06-08 16:18:44 -05:00
Matt Burgess f3e8651522 NIFI-11647: Support UUID type in DataTypeUtils.getSQLTypeValue
This closes #7355

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2023-06-08 11:13:07 -04:00
exceptionfactory ec525b0437
NIFI-11629 This closes #7327. Added Socket Write Timeout to InvokeHTTP
Signed-off-by: Joe Witt <joewitt@apache.org>
2023-06-02 08:04:25 -07:00
Mark Payne 50811660d0
NIFI-11603 Refactored Socket-based tests to use Dynamic Ports
- Removed NetworkUtils methods for getting available ports
- Updated Socket-based components to support using 0 to listen on a random available port for improved test reliability

This closes #7299

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-06-01 13:33:06 -05:00
Peter Turcsanyi 3bd4b49abe NIFI-11369: Fixed Defragment strategy with optional fragment.count attribute in MergeContent 2023-05-30 16:25:28 +02:00
Mark Bathori cfd62c9511
NIFI-11590: Missing Enum data type handling in FlowFileTable
This closes #7294.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2023-05-26 11:08:37 +02:00
Peter Turcsanyi de62c6261f Revert "NIFI-11590: Missing Enum data type handling in FlowFileTable"
This reverts commit 4e304ac586.
2023-05-26 11:07:45 +02:00
Mark Bathori 4e304ac586
NIFI-11590: Missing Enum data type handling in FlowFileTable
This closes #7294.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2023-05-26 10:57:33 +02:00
dan-s1 d6600e67cb
NIFI-2964 Added JSON Handling Strategy property to AttributesToJSON
This closes #7231

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-05-25 15:51:02 -05:00
Mark Payne ca2a829d47
NIFI-11584 Improved MergeContent stream handling
- Allow ProcessSession to manage its own input streams and deprecated method that reads from FlowFile without allowing it

This closes #7286

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-05-24 11:09:42 -05:00
Lehel Boér 6c70471cc6
NIFI-5151: Add UPSERT support for Apache Phoenix
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #7263
2023-05-18 15:54:08 -04:00
Nandor Soma Abonyi 2049d5068b
NIFI-11524 Improved ExecuteStreamCommand documentation and configuration
This closes #7228

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-05-11 14:52:21 -05:00
dan-s1 e4cdb90a75 [NIFI-11509] Added configuration to allow for parsing comments in JSON.
This closes #7220

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2023-05-02 08:18:47 -04:00
dan-s1 71c7d38c75 [NIFI-11511] Added Provenance route events for all the relationships.
This closes #7221

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2023-05-02 08:03:49 -04:00
Chris Sampson 9274d0c9b0 NIFI-10865 allow RecordPath's unescapeJson to convert de-serialised JSON Objects into Records
NIFI-10865 allow UpdateRecord to replace the Record root for relative paths, e.g. when a RecordPath function is used to modify selected field(s)

This closes #6708

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2023-04-29 12:26:07 -04:00
Aakash Chauhan afdbed5749
NIFI-8971 Removed legacy FlowFile name and path attributes
This closes #6003

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-04-26 16:37:48 -05:00
Daniel Aaron Salwerowicz d3f2bf1c16
NIFI-11472 Check working directory after making directory in PutFTP
- Multiple threads can attempt to create a remote directory when PutFTP has multiple concurrent tasks.

This closes #7184

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-04-24 11:33:09 -05:00
mr1716 29bb652623
NIFI-11456 Upgraded brotli4j from 1.8.0 to 1.11.0
This closes #7178

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-04-17 18:08:27 -05:00
Pierre Villard 2b06306b07
NIFI-11418 Upgraded bcrypt from 0.9.0 to 0.10.2
This closes #7149

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-04-10 08:45:55 -05:00
Pierre Villard 4a0b38f4dd
NIFI-11405 Upgraded multiple drivers and test dependencies
This closes #7134

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-04-07 15:54:51 -05:00
Matt Burgess 1a9acb1f4b
NIFI-11337: Update PutDatabaseRecord's Max Batch Size property documentation to reflect actual behavior (#7081) 2023-03-24 10:22:56 -04:00
Emilio Setiadarma cac76e339d
NIFI-9196 Corrected FetchSFTP Move File destination formatting
- Changed FetchSFTP to use String.format() instead of  File.getAbsolutePath() for defining the destination path as part of the Move File completion strategy

This closes #7072

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-23 13:57:23 -05:00
krisztina-zsihovszki 63e9365449
NIFI-4651 Add error message and code to PutSQL FlowFile attributes
NIFI-4651 Review comments

Signed-off-by: Matthew Burgess <mattyb149@apache.org>
2023-03-21 11:19:33 -04:00