1131 Commits

Author SHA1 Message Date
exceptionfactory
1eede30974
NIFI-9732 Upgraded Zip4j from 2.8.0 to 2.9.1
This closes #5811

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-02-27 06:09:14 -05:00
markobean
4e3871fec7
NIFI-9072: improvements to ValidateXML including validate XML in attr… (#5324)
NIFI-9072: Improvements to ValidateXML including validate XML in attributes and non-schema, well-formed XML only validation, and additional documentation
- Updated description and added System Resource Consideration to discourage using large XML documents in attributes
- Migrate from XML DocumentBuilderFactory to SafeXMLConfiguration
2022-02-25 14:57:04 -05:00
Mark Payne
39483b9c12
NIFI-9716, NIFI-9577: Addressed issue in the PathFilter for GetFile / ListFile. For any file that is found in the Input Directory directly, it was previously being listed/fetched even if it didn't match the PathFilter. Additionally, updated the code to create a new File FIlter for every invocation of onTrigger. This was necessary for NIFI-9577 because the directory to monitor supports Expression Language and as a result may change from invocation to invocation, if using a function such as now() but the PathFilter would always relativize the path based on the value that was obtained when the processor was scheduled.
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5788.
2022-02-24 09:13:16 -05:00
Mark Payne
ce0122bd25
NIFI-9691: Added ForkEnrichment, JoinEnrichment processors
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5772.
2022-02-22 18:54:32 -05:00
levilentz
dfbf2e3cea
NIFI-9286: JOLT Expression Language
Fixes NIFI-6213 and adds in functionality to use expression language in class and module specification
NIFI-9286: adding JOLT unit tests
NIFI-9286: addressing PR feedback
Fixes a problem with the scope of the EL for module directory
NIFI-9286: alignment of JOLT processors
NIFI-9286: fix checkstyle

This closes #5444

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-02-19 14:11:44 -05:00
Matthew Burgess
03165ad817 NIFI-9607: Honor Update Keys when Quoting Identifiers in PutDatabaseRecord
NIFI-9607: Fixed wrong column name in WHERE clause for generateUpdate

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5701.
2022-02-16 16:07:34 -05:00
Gabor Gyimesi
6788076536
NIFI-9058 Corrected AttributesToJSON Core Attributes filtering
- Avoiding filtering Core Attributes from configured Attributes List

This closes #5312

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-02-15 15:51:39 -06:00
exceptionfactory
e584d3cf04
NIFI-9660 Upgraded Apache Tika to 2.3.0
- Upgraded tika-core from 1.27 to 2.3.0
- Upgraded tika-parsers to tika-parsers-standard-package in nifi-media-processors
- Updated Tika metadata property references

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5754.
2022-02-09 11:11:46 +01:00
Noel Baek
bbc78f1547
NIFI-9621: Added Ignore Reserved Characters to FlattenJson
- Upgraded to json-flattener from 0.12.0 to 0.13.0

This closes #5704

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-02-08 21:02:55 -06:00
exceptionfactory
ce66cf41e2
NIFI-6699 Corrected SFTP symbolic link handling (#5744)
- Added SFTP stat() request on symbolic links to check directory status
- Refactored and renamed TestServerSFTPTransfer class
2022-02-04 18:43:42 -05:00
exceptionfactory
ba0bf2ede0
NIFI-9650 Upgraded OkHttp from 4.9.2 to 4.9.3
- Added okhttp-bom to root Maven configuration
- Removed repeated okhttp.version property from specific dependencies

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5737.
2022-02-02 11:45:18 +01:00
exceptionfactory
43748a5523 NIFI-9638 Refactored Google Guava references
- Refactored nifi-framework and nifi-standard modules
- Replaced Google Cache with Caffeine Cache
- Replaced Google collections classes with standard Java collections

This closes #5730.

Signed-off-by: Kevin Doran <kdoran@apache.org>
2022-02-01 15:17:50 -05:00
exceptionfactory
dc7d9510cd NIFI-9610 Refactored nifi-processor-utils to separate modules
- Added nifi-bin-manager
- Added nifi-event-listen
- Added nifi-event-put
- Added nifi-listed-entity
- Added nifi-put-pattern
- Removed nifi-processor-utils

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5729.
2022-02-01 12:59:16 -05:00
Bence Simon
49978cdd91
NIFI-9341 Added CEF RecordReader
This closes #5530

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-01-28 12:42:51 -06:00
exceptionfactory
95abc35463 NIFI-6266 Corrected proxy FTP connect handling
- Added ProxyFTPClient with connect method supporting unresolved hosts
- Added FTPClientProvider with implementation handling configuration and connection
- Added support for SOCKS with authentication in FTP components
- Refactored FTPTransfer using FTPClientProvider

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5655.
2022-01-27 17:36:27 -05:00
Nathan Gough
08153b8260
NIFI-9453 Refactored ListenBeats using Netty
This closes #5669

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-01-26 14:40:32 -06:00
Peter Turcsanyi
96b53d8943
NIFI-9589: Support initial loading from the current max values in QueryDatabaseTable* processors
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5676.
2022-01-25 18:12:14 +01:00
Tamas Palfy
aa61494fc3
NIFI-9065 Add support for OAuth2AccessTokenProvider in InvokeHTTP
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5319.
2022-01-19 13:29:11 +01:00
exceptionfactory
405934dcd2
NIFI-9571 Corrected Session commit handling in PutTCP
- Added generic type to AbstractPutEventProcessor for compiler checking of event types
- Refactored createTransitUri to shared method in AbstractPutEventProcessor

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5658.
2022-01-18 15:31:36 -05:00
Pierre Villard
4847926a4b
NIFI-9544 - LookupRecord - fixed behavior when no matching value in the LRS
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5638.
2022-01-13 16:13:24 -05:00
exceptionfactory
7c50f1429e
NIFI-9563 Enabled ListenTCP Pool Receive Buffers property
- Removed deprecation from ListenTCP Pool Receive Buffers property
- Added BufferAllocator configuration property for NettyEventServerFactory

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5653.
2022-01-13 19:50:44 +01:00
Pierre Villard
d1599eebcf
NIFI-9545 - Fix in-place replacement for LookupRecord processor
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5637.
2022-01-13 10:59:02 -05:00
exceptionfactory
64495e99e9
NIFI-7835 Added authenticated SOCKS proxy support for SFTP
NIFI-7749 Added authenticated HTTP proxy support for SFTP

- Added StandardSocketFactoryProvider to return SocketFactory based on credentials

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5624.
2022-01-12 10:50:01 -05:00
Mark Payne
72e54f4fab NIFI-9390: Updates to MergeContent / MergeRecord so that they play nicely within Stateless
NIFI-9390: Addressed underlying condition in stateless framework that caused Merge-related processors and similar to not properly be triggered as necessary. Added several system tests to verify different configurations.

NIFI-9390: Simplified the logic for how to iterate over the components in a Stateless flow that are ready to be triggered

This closes #5634.

Co-authored-by: Peter Turcsanyi <turcsanyi@apache.org>
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2022-01-11 09:45:00 +01:00
exceptionfactory
6039095625
NIFI-9267 Replaced nifi-standard-web-test-utils with mockwebserver
- Replaced instances of custom Jetty TestServer with OkHttp MockWebServer
- Removed integration tests referencing nifi-standard-web-test-utils SslContextUtils
- Removed nifi-standard-web-test-utils

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5620.
2022-01-09 05:25:32 -05:00
exceptionfactory
ca8bc17c0a
NIFI-7089 Upgraded Apache SSHD from 1.7.0 to 2.8.0
- Replaced fake-sftp-server-rule with test Apache SSHD Server

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5646.
2022-01-08 11:02:12 +01:00
markobean
42626adab8
NIFI-8962: Add overflow strategy to DistributeLoad (#5267) 2022-01-07 14:45:18 -05:00
Joe Gresock
287e41d8a6
NIFI-9547: Allowing more datatypes in LookupRecord value substitution
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5640.
2022-01-07 12:30:13 +01:00
Mark Payne
56593ad12f
NIFI-9546: Updated PutTCP and PutUDP with SupportsBatching annotations
- Removed incorrect usage of TriggerWhenEmpty
- Allow for 0 seconds of gracefully waiting for incoming threads on shutdown since we won't have any
- Updated unit tests to no longer have arbitrary sleep statements

This closes #5639

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-01-06 15:52:37 -06:00
Peter Turcsanyi
4346dd8faf
NIFI-9391: Modified MergeRecord to process FlowFiles within a loop in… (#5550)
* NIFI-9391: Modified MergeRecord to process FlowFiles within a loop in a single onTrigger

MergeRecord processed the FlowFiles in multiple onTrigger-s and it needed an extra onTrigger call
(with no incoming FFs) to realize that no more FFs are available and it is time to send the merged FF downstream.
It was not compatible with Stateless Runtime which does not trigger the flow any more if no FFs available.

Also changed "unschedule" logic in StandardProcessorTestRunner: @OnUnscheduled methods were called immediately after
the 1st FlowFile was processed. Unschedule the processor only at the end of the execution (onTrigger finished)
and only if stopOnFinish has been requested by the test case.
2022-01-04 11:36:56 -05:00
Vibhath Ileperuma
0ac8f1b32c
NIFI-8605 Adding a new property for ExecuteSQL and ExecuteSQLRecord processors to enable/disable auto committing
change the default value of auto commit function to true
Changed the auto commit property name and add more details in the description
If the auto commit is set to false, commit() is called for consistency
adds unit tests
Fix the check style issue of having more than 200 characters in single line

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5554
2021-12-30 18:35:50 -05:00
exceptionfactory
9747d6a410
NIFI-9509 This closes #5618. Added PatchedSFTPEngine to correct SFTP rename flags handling
- SSHJ 0.32.0 introduced support for rename flags in SFTP commands without checking the protocol version
- PatchedSFTPEngine overrides the rename method to check the SFTP protocol version

Signed-off-by: Joe Witt <joewitt@apache.org>
2021-12-21 12:14:23 -07:00
exceptionfactory
4f28b28ba2
NIFI-9507 This closes #5617. Corrected SSH Client handling on connect failures
- Refactored SSH Client configuration and connection to SSHClientProvider
- Implemented exception handling for configuration and connection failures
- Named SSH keep-alive thread for improved runtime tracking
- Closed SSH Client and interrupted keep-alive thread on configuration failures
- Added missing Compression Property to ListSFTP
- Corrected Hostname and Port property descriptors in ListSFTP

Signed-off-by: Joe Witt <joewitt@apache.org>
2021-12-20 20:23:31 -07:00
exceptionfactory
7c6bdcb035
NIFI-9491 Excluded commons-logging and added jcl-over-slf4j references
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5608
2021-12-17 07:21:03 -05:00
Nathan Gough
5e9c86885c
NIFI-9478 Moved Netty Log Exception Handler to end of pipeline
- Changed display name of Max Number of TCP Connections to Max Number of Worker Threads for ListenTCP
- Set Netty Socket Receive Buffer using Max Socket Buffer Size in ListenTCP

This closes #5599

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-12-15 16:59:07 -06:00
Pierre Villard
90930ca197
NIFI-9476 - Fix QueryRecord when no result and with array type column
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5594
2021-12-15 17:37:24 -05:00
Pierre Villard
e6b573e79e
NIFI-9419 ParseCEF - Upgraded parcefone and supported empty extensions
- Upgraded com.fluenda:parcefone from 2.0.0 to 2.1.0
- Added Accept empty extensions property to ParseCEF

This closes #5555

Co-authored-by: David Handermann <exceptionfactory@apache.org>
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-11-30 11:27:50 -06:00
Paul Grey
d6b6dd1eb4
NIFI-9389 - NPE guard for ExecuteProcess when not triggered
- Added support for TestRunner.run() with zero operations

This closes #5544

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-11-29 09:27:04 -06:00
exceptionfactory
c033debdf3
NIFI-9399 This closes #5542. Apply Secure Processing to TransformXml XSLT
- Added XML Stream Reader processing for XSLT with external entities disabled
- Removed unused XsltValidator
- Upgraded Saxon-HE from 9.6.0-5 to 10.6

Signed-off-by: Joe Witt <joewitt@apache.org>
2021-11-22 09:18:57 -07:00
exceptionfactory
0cf515c9c0 NIFI-9384 Corrected usage and generics in ListenTCP
- Addressed compiler warnings in ListenTCP and EventBatcher
- Adjusted ListenTCP property order to match previous version

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5526.
2021-11-17 11:54:51 -05:00
Nathan Gough
f7466fd0f1
NIFI-8806 Refactored ListenTCP using Netty
- Deprecated Max Number of Receiving Message Handler Threads property
- Deprecated Pool Receive Buffers property
- Updated TestListenTCP using Netty EventSender

This closes #5493

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-11-16 11:27:38 -06:00
Nandor Soma Abonyi
6fd1f03bd6
NIFI-830 Added FlowFile Naming Strategy to InvokeHTTP
This closes #5475

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-11-15 07:46:37 -06:00
Roberto Santos
e906c1d70f
NIFI-9194: Upsert for Oracle12+
Fixes pr #5366.

Fixes pr #5366. Replace tabchars fot whitespaces.

Fixes pr #5366. Replaced tabchars for whitespaces. Removed unnecessary comments.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5366
2021-11-08 20:45:25 -05:00
Joe Witt
c055895952
NIFI-9344-RC3 prepare for next development iteration 2021-11-03 08:53:37 -07:00
Joe Witt
7fdc07cccd
NIFI-9344-RC3 prepare release nifi-1.15.0-RC3 2021-11-03 08:53:32 -07:00
Pierre Villard
decfad394d
NIFI-9349 - add attribute with request duration in InvokeHTTP (#5501)
This closes #9349

Signed-off-by: Otto Fowler <ottobackwards@gmail.com>
2021-11-03 09:25:57 -04:00
exceptionfactory
16e6045d13 NIFI-9346 Added closing of EventSender to TestListenRELP
Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5492.
2021-10-28 13:10:04 -04:00
exceptionfactory
67ccdf6159
NIFI-9326 Added Socket Keep Alive property to ListenSyslog
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5479.
2021-10-26 19:44:56 +02:00
Lehel
d7e41e2005
NIFI-9328: Transfer cleanup and reuse added to FetchFileTransfer in case of FileNotFound and PermissionDenied exceptions.
- Corrects connection handling in FetchFTP and FetchSFTP

This closes #5478

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-10-26 11:17:33 -05:00
Nathan Gough
309ab8f4ab
NIFI-8792 - Modified ListenRELP to use Netty
- Refactored RELP encoders and decoders

This closes #5398

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-10-25 15:53:58 -05:00