1433 Commits

Author SHA1 Message Date
dan-s1
942d13c118
NIFI-12785 Corrected InvokeHTTP URL handling to avoid double encoding
This closes #8458

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-03-08 08:20:07 -06:00
Mark Payne
ae423fc6ba
NIFI-12832 Removed unnecessary dependencies from nifi-mock
- Moved StandardValidators to nifi-api
- Moved URL creation method from UriUtils to URLValidator
- Separated FormatUtils into FormatUtils and DurationFormat classes
- Added DurationFormat to nifi-api

This closes #8442

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-02-23 15:13:21 -06:00
Joseph Witt
9e7e39fc2a
NIFI-12826 Added timing lag in TestFTP method for improved stability
This closes #8443

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-02-22 10:58:07 -06:00
dan-s1
0a2ba317c0
NIFI-12784 Set Path Not Found as a Dependent Property in EvaluateJsonPath
This closes #8420

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-02-20 22:14:52 -06:00
dependabot[bot]
47290eb39f
NIFI-12827 Upgraded PostgreSQL JDBC test driver from 42.7.1 to 42.7.2
This closes #8437

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-02-20 21:28:47 -06:00
Joseph Witt
3f2f5a02e0
NIFI-12768 Removed unstable assertions from TestListFile
Additional assertion removals which assumed ordering that isn't guaranteed or enforced by the tests

This closes #8438

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-02-20 21:09:07 -06:00
Tom Brisland
89b618cc05
NIFI-12772 Expose REMOTE_POLL_BATCH_SIZE property for ListSFTP
This closes #8390.

revert max results optimisation + unnecessary import

Signed-off-by: Joseph Witt <joewitt@apache.org>
2024-02-20 12:52:35 -07:00
Pierre Villard
abc08eb842
NIFI-12796 - PutDatabaseRecord statement type should support u/c/d for Debezium
This closes #8412.

Signed-off-by: Joseph Witt <joewitt@apache.org>
2024-02-20 09:36:55 -07:00
David Handermann
e93fb17b66
NIFI-12768 Removed unstable filename assertion in TestListFile (#8383) 2024-02-09 16:00:22 -05:00
Mark Payne
5f534dcc42
NIFI-12710 Support microsecond precision for Timestamp Record fields
- PutDatabaseRecordIT supports operating systems with either nanosecond or microsecond precision

This closes #8332

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-02-09 08:19:51 -06:00
Umar Hussain
e00d2b6d5e
NIFI-12708: add option in UnpackContent to specify encoding charset for filenames in zip unpacking
This closes #8350

The processor can now take a filename encoding parameter and pass it to zip unpacking. This will allow
user to unzip files with specific encoding to get correct filenames in output.

This for example help with zip files created on Windows which by default uses Cp437 for filename encoding.
If the filename contains special character like German alphabet ä, ü etc., decoding this with Linux's
default encoding usually UTF8 output will contain `?` in it. When the same file is processed with property
set with `Cp437`, the processor outputs correct filenames with special characters preserved.

Signed-off-by: Joseph Witt <joewitt@apache.org>
2024-02-07 09:56:21 -07:00
mr1716
00e5b454ab
NIFI-12728 Upgraded brotli4j from 1.13.0 to 1.16.0
This closes #8343

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-02-02 16:31:02 -06:00
Mark Payne
e16286ef40
NIFI-12707 Allow LookupRecord to operate on multiple child-records
- In order to accommodate this, also needed to improve DataTypeUtils so that it knows that Record A is wider than Record B if Record A contains all fields of Record B and more.

- Removed unit tests and resources that are overly complex and no longer applicable
- Fixed issue in unit test based on different line endings between operating systems

This closes #8331

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-02-02 16:24:43 -06:00
jsteinebrey
59ff1b6561 NIFI-9677 Fixed issue that an empty JSON array causes flow file to be considered unmatched even though it should be considered as a match.
Refactored to avoid streaming over the list twice.

Tweaked unit test, so it works in NiFi 1.x also.

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #8266
2024-01-26 21:54:43 -05:00
exceptionfactory
103e350943
NIFI-12676 Fixed Servlet Registration in HandleHttpRequest
This closes #8304.

Signed-off-by: Joseph Witt <joewitt@apache.org>
2024-01-25 16:40:43 -07:00
dan-s1
f76d7884ba
NIFI-12554 Moved JoltTransformJSON and JoltTransformRecord to nifi-jolt-nar
This closes #8249

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-01-19 17:30:37 -06:00
Mark Payne
345b8bd059
NIFI-12647 Added MultiProcessorUseCase for ListFile/FetchFile together
This closes #8276

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-01-19 16:11:53 -06:00
nathluu
6649115a34
NIFI-12402 Added Wait for Activity to MonitorActivity
This closes #8063

Signed-off-by: David Handermann <exceptionfactory@apache.org>
Co-authored-by: Vuong Nguyen Van <vanvuong24072001@gmail.com>
Co-authored-by: nathluu <luuhoangtanbis@gmail.com>
2024-01-19 11:48:37 -06:00
Mark Payne
2212afe482
NIFI-12638 Add Use Case on how to use QueryRecord as a router
This closes #8271

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-01-18 19:00:12 -06:00
Mark Payne
bf1dfd0615
NIFI-12637 Handle migrating Proxy properties for InvokeHTTP
This closes #8270

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-01-18 18:59:31 -06:00
EndzeitBegins
359ffa5b4f
NIFI-12627 Extract nifi-file-transfer from nifi-standard-processors
This closes #8262

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-01-17 22:21:07 -06:00
EndzeitBegins
53bb995e4e
NIFI-12613 Renamed asDescribedValue() to asAllowableValue()
This closes #8260

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-01-17 22:16:29 -06:00
David Handermann
250fe90b34
NIFI-9458 Replaced SimpleDateFormat with DateTimeFormatter (#8248)
- Added FieldConverterRegistry and Standard implementation
- Moved DateTypeUtils methods to FieldConverter implementations
2024-01-16 16:39:40 -05:00
Pierre Villard
dd45e7edcb
NIFI-12593 Added Include all violations property to ValidateCsv
This closes #8229

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-01-16 09:18:58 -06:00
EndzeitBegins
4efabdcf51
NIFI-12561 Fixed MergeContent DELIMITER_STRATEGY_NONE Handling
This closes #8222

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2024-01-12 15:00:13 -06:00
Matt Burgess
31d04c8575
NIFI-12530: Support CREATE TABLE in Oracle database adapters
Signed-off-by: Joe Gresock <jgresock@gmail.com>
This closes #8175.
2024-01-11 21:11:36 -05:00
Pierre Villard
415505ea5d NIFI-12587 - improve error message in ValidateCSV
review - removed row source from logging

Update nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateCsv.java

Co-authored-by: dan-s1 <dstieg1@gmail.com>

Update nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ValidateCsv.java

Co-authored-by: dan-s1 <dstieg1@gmail.com>

Update nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestValidateCsv.java

Co-authored-by: dan-s1 <dstieg1@gmail.com>

Update nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestValidateCsv.java

Signed-off-by: Matt Burgess <mattyb149@apache.org>
This closes #8224

Co-authored-by: dan-s1 <dstieg1@gmail.com>
2024-01-09 17:56:50 -05:00
exceptionfactory
42547eb60c
NIFI-11703 Upgraded to Spring Framework 6 and Jetty 12
- Upgraded Spring Framework from 5.3.31 to 6.0.15
- Upgraded Spring Security from 5.8.7 to 6.2.0
- Upgraded Spring Vault from 2.3.4 to 3.1.0
- Upgraded Jetty from 10.0.18 to 12.0.5 with EE 10
- Upgraded Jersey from 2.41 to 3.1.4
- Upgraded JAXB from 2.3.9 to 4.0.4
- Upgraded AspectJ from 1.9.20.1 to 1.9.21
- Upgraded JMS API from 2.0.1 to 3.1.0
- Upgraded ActiveMQ Broker from 5.18.2 to 6.0.1 for JMS 3
- Upgraded JJWT from 0.9.1 to 0.12.3
- Replaced jackson-module-jaxb-annotations with jackson-module-jakarta-xmlbind-annotations
- Replaced maven-jaxb2-plugin with hisrc-higherjaxb40-maven-plugin 2.1.1
- Replaced kongchen swagger-maven-plugin with swagger-codegen-maven-plugin from Swagger 3
- Replaced com.nickwongdev AspectJ Plugin with Codehaus 1.14.0 for newer Java versions
- Removed unused cglib-nodep
- Removed references to javax.validation
- Removed custom Jetty ALPN Processor not required for Java 21
- Removed several tests depending on older Jetty and Jakarta libraries
- Removed unnecessary webdefault.xml configurations
- Replaced unsupported cross-context servlet forwarding with HTTP forwarding
- Replaced javax.servlet references with jakarta.servlet
- Replaced javax.xml.bind references with jakarta.xml.bind
- Replaced javax.ws references with jakarata.ws
- Updated Spring Security CSRF implementation for Spring Security 6
- Updated web.xml versions to 6.0
- Updated REST API templates using new swagger-codegen variables
- Removed VALIDATE_DATA property from ParseCEF based on library compatibility issue with javax.validation
- Added application URL logging to NiFi JettyServer

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #8197.
2024-01-04 14:01:32 +04:00
exceptionfactory
b8c8eafd03 NIFI-12559 Upgraded SSHJ from 0.37.0 to 0.38.0
- Removed Bouncy Castle JDK 1.5 version exclusions no longer required in SSHJ 0.38.0

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #8200
2024-01-02 13:49:31 -05:00
krisztina-zsihovszki
1a34208f65
NIFI-12531 Parameter references are removed after property migration
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #8176.
2024-01-02 21:42:56 +04:00
dan-s1
3e62e1e7f2
NIFI-12517 Updated isJson function to improve space handling
- The isJson Expression Language function returns true regardless of leading or trailing spaces

This closes #8165

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-12-29 21:58:09 -06:00
exceptionfactory
a06fe5905c
NIFI-11627 Corrected TestValidateJson exception evaluation
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-12-26 18:57:34 -06:00
dan-s1
3b8ff2299f
NIFI-12513 Added UriUtils to nifi-utils and updated InvokeHTTP
The UriUtils class allows the construction of a valid java.net.URI with a single string even though there maybe illegal characters in the path, query and/or fragment sections(s) of the URI. The create method uses regular expressions from Spring Framework UriComponentsBuilder and provides capabilities closer to the deprecated constructor for java.net.URL.

This closes #8189

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-12-26 16:53:29 -06:00
dan-s1
6bca79cb37
NIFI-11627 Added JsonSchemaRegistry for ValidateJson
- Added nifi-json-schema-api to nifi-commons
- Added StandardJsonSchemaRegistry implementation of JsonSchemaRegistry
- Added strategy configuration properties to ValidateJson

This closes #8005

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-12-26 16:28:30 -06:00
dan-s1
1ddb5c185a
NIFI-12408 Added Pretty Print Property to AttributesToJSON
This closes #8156

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-12-16 14:45:30 -06:00
EndzeitBegins
4f399c9bb9
NIFI-12446 Refactor FilterAttribute to align with code conventions
This closes #8161

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-12-16 12:18:39 -06:00
EndzeitBegins
c272574dcc
NIFI-12452 Improved support for DescribedValue in Descriptors
- Removed calls to Builder.defaultValue(null) in several Components

This closes #8102

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-12-13 13:45:45 -06:00
exceptionfactory
34aebc1f69
NIFI-12473 Deleted removeByPattern Methods from Cache Services
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #8124.
2023-12-07 12:42:08 +01:00
Matt Burgess
eeb2b1a644
NIFI-12318: Fixed byte array generation in GenerateRecord
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #8109.
2023-12-05 14:41:22 +01:00
Mark Payne
283abadf97
NIFI-12457: Added additional use case based documentation and fixed some typos/made some clarifications in related docs
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #8108.
2023-12-04 14:59:53 +01:00
EndzeitBegins
edca4cd347 NIFI-12386 Adds processor FilterAttribute
attribute uuid is not removed by removeAttribute(s) in MockProcessSession

Signed-off-by: Mike Moser <mosermw@apache.org>

This closes #8049
2023-11-30 19:21:08 +00:00
mr1716
0430974783
NIFI-12436 Upgraded datafaker from 2.0.1 to 2.0.2
This closes #8084

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-11-29 22:45:26 -06:00
Mark Payne
d0c7ebb196
NIFI-11671: Allow FlowFile attributes to be referenced in SQL for JoinEnrichment
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #8059.
2023-11-22 14:37:18 +01:00
Ferenc Gerlits
ceeb0deb3d
NIFI-12381 Fixed CR and LF documentation in SplitText
This closes #8041

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-11-16 17:44:53 -06:00
Mark Payne
e1942a8346
NIFI-12374: Added use case based documentation to explain how to perform incremental / full loads from databases
This closes #8030

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-11-15 19:28:10 -06:00
EndzeitBegins
6c333cdb7e
NIFI-1874 Added Character Set Detection to IdentifyMimeType
NIFI-4550 New Processor not required based on improvements to IdentifyMimeType

- Added mime.charset FlowFile attribute when not null for text MIME types

This closes #8011

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-11-15 09:11:30 -06:00
mr1716
db827b88b2
NIFI-12360 Upgraded brotli4j from 1.12.0 to 1.13.0
This closes #8020

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-11-13 13:58:08 -06:00
Nissim Shiman
013ca6da7d
NIFI-11463 Added Config Strategy to IdentifyMimeType
This closes #7781

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-11-07 09:39:48 -06:00
Mark Payne
a978406c83
NIFI-12326: When a Python Processor is created, it attempts to download dependencies in the background and then load the processor code. If that fails, it previously gave up; now it will log the exception and keep trying
This closes #7990

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-11-07 09:07:59 -06:00
exceptionfactory
9b29fffbc6 NIFI-12305 Optimized Regular Expression patterns
- Updated SampleRecord range validation to split on comma and validate individual ranges
- Updated AccessPolicyEndpointMerger to restrict the set of characters matched

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #7967
2023-11-05 19:26:04 -05:00