1
0
mirror of https://github.com/apache/nifi.git synced 2025-03-05 00:50:09 +00:00

1412 Commits

Author SHA1 Message Date
exceptionfactory
4f423a59ba
NIFI-9952 Upgraded Jackson BOM to 2.13.2.20220328
- Removed unnecessary references to jackson.version property
- Removed unnecessary dependency management references to Jackson libraries

This closes 

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-04-23 08:23:39 -04:00
exceptionfactory
4b7f8cbe1f
NIFI-9949 This closes . Corrected intermittent failures in TestPutSFTP
- Changed SSH server to start and stop after each method
- Replaced queued file with string FlowFile contents
- Refactored TestPutSFTP using JUnit 5

Signed-off-by: Joe Witt <joewitt@apache.org>
2022-04-21 12:23:07 -07:00
exceptionfactory
b288810316
NIFI-9943 Added Transform Provider to nifi-xml-processing
- Refactored TransformerFactory references using StandardTransformerProvider

This closes 
Signed-off-by: Paul Grey <greyp@apache.org>
2022-04-21 15:14:54 -04:00
exceptionfactory
1e345db649
NIFI-9942 Removed load test from TestPutUDP
- Refactored TestPutUDP using JUnit 5
- Standardized test method naming

This closes 
Signed-off-by: Paul Grey <greyp@apache.org>
2022-04-21 09:45:00 -04:00
exceptionfactory
e98b68cc3a
NIFI-9939 Upgraded SSHJ from 0.32.0 to 0.33.0
- Removed PatchedSFTPEngine with resolution of SFTP renaming in SSHJ 0.33.0

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes .
2022-04-20 15:21:22 +02:00
Paul Grey
0a94762119
NIFI-9925 Adjust PutUDP and SNMP tests to run without preferring IPv4
This closes 

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-04-19 14:13:00 -05:00
exceptionfactory
ecf0a93124
NIFI-9935 Upgraded Zip4j from 2.9.1 to 2.10.0
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes .
2022-04-19 09:36:29 +02:00
Paul Grey
4310df0544
NIFI-9924 Corrected text encoding in PutEmail filenames and TestFTP
This closes 

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-04-14 14:58:22 -05:00
exceptionfactory
15f7590f7a
NIFI-9901 Added nifi-xml-processing to nifi-commons
- Refactored XML parsing to use providers from nifi-xml-processing
- Configured spotbugs-maven-plugin with findsecbugs-plugin in nifi-xml-processing
- Disabled Validate DTD in default configuration for EvaluateXPath and EvaluateXQuery
- Replaced configuration of DocumentBuilder and streaming XML Readers with shared components
- Removed XML utilities from nifi-security-utils
- Moved Commons Configuration classes to nifi-lookup-services

This closes 
Signed-off-by: Paul Grey <greyp@apache.org>
2022-04-13 19:56:57 -04:00
Mark Payne
05f3d7510f
NIFI-9903: This closes . When using the 'success' relationship only for LookupRecord, lookup all records until a match is found, in order to determine the resultant schema. Refactored code to eliminate AbstractRouteRecord, because LookupRecord is the last processor that extended from it. Refactored code to use an inner interface to clean up code.
Signed-off-by: Joe Witt <joewitt@apache.org>
2022-04-13 13:49:26 -07:00
Joe Witt
873d25585c
NIFI-9780 Merge branch 'NIFI-9780-RC3' 2022-04-04 08:35:23 -07:00
exceptionfactory
cf21bc47cd NIFI-9844 Refactored Encryptor tests using JUnit 5
- Refactored Keyed and Password Based Encryptor tests from Groovy to Java

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes .
2022-03-29 17:53:13 -04:00
Nathan Gough
2a6e521549
NIFI-9702 ListSFTP will set mime.type when Record Writer configured
This closes 

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-03-28 16:03:00 -05:00
Peter Gyori
6ade47ac4f
NIFI-9838: Added Client Certificate attributes to ListenTCPRecord FlowFiles
This closes 

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-03-28 13:36:16 -05:00
Emilio Setiadarma
9566d3fa78
NIFI-9841 Improved Encryptor test reliability
- Changed expected exception to ProcessException in KeyedEncryptorGroovyTest and PasswordBasedEncryptorGroovyTest to avoid intermittent failures

This closes 

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-03-28 13:28:28 -05:00
Nathan Gough
2fbe10b4bc
NIFI-9701 - Corrected No Tracking strategy to create one flow file when using a Record Writer
- Corrected No Tracking strategy Record Writer handling for ListSFTP
- Updated temporary test files to have last modified time of epoch to avoid intermittent issue with Minimum Age filtering
- Refactored MockCacheService to separate reusable class

This closes 

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-03-28 12:47:30 -05:00
Mark Payne
63f02c99bf
NIFI-9835: Fixed threading bug in which NioAsyncLoadBalanceClient calls LoadBalanceSession.isComplete() followed by LoadBalanceSession.isCanceled() but it's possible for the complete flag to change before the canceled flag (they are not updated atomically). So changed to use a single LoadBalanceSessionState enum that represents the state. Also made the private StandardProcessSession.commit(boolean) method synchronized. When a processor is terminated (as is the case in Offload), we roll back sessions and both the commit() and rollback() need to be synchronized. Only the public commit() method was synchronized, and now with commitAsync() happening, we had the ability to commit without any synchronization. This addresses that concern. Also fixed a typo in docs for MergeRecord.
This closes 

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-03-25 12:42:03 -05:00
Marcus Ely
72435c3c6d
NIFI-8124 Added Cookie Strategy property to InvokeHTTP
This closes 

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-03-23 15:55:40 -05:00
Emilio Setiadarma
772adbc709
NIFI-1468 Added tests to handle invalid cipher streams missing Salt/IV
- Updated PasswordBasedEncryptorGroovyTest and KeyedEncryptorGroovyTest

This closes 

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-03-23 13:30:15 -05:00
Joe Witt
0419dc2939
NIFI-9780-RC3 prepare for next development iteration 2022-03-21 11:58:27 -07:00
Joe Witt
b019a9191f
NIFI-9780-RC3 prepare release nifi-1.16.0-RC3 2022-03-21 11:58:09 -07:00
Matthew Burgess
c84f438782
NIFI-9800: Unwrap SQLException in PutDatabaseRecord when table does not exist
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes 
2022-03-21 14:00:13 -04:00
Mike Thomsen
2bb1a06a8c
NIFI-9788 Updated commons-codec to 1.15 across all modules.
This closes 

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-03-21 09:13:05 -05:00
exceptionfactory
412e21be97
NIFI-9281 Enabled building on Java 17
- Added Ubuntu Zulu JDK 17 GitHub build
- Adjusted MiNiFi C2 FileSystemConfigurationCache test to avoid using environment variables
- Adjusted MiNiFi StatusLogger and StatusLoggerTest to avoid overriding private logger
- Adjusted failure reason attribute check in TestGetIgniteCache
- Adjusted TestRangerAuthorizer and TestRangerNiFiAuthorizer to avoid checking nested exceptions
- Adjusted encrypt-config TestUtil to avoid unnecessary comparison of different types
- Disabled Javascript tests on Java 15 and higher
- Disabled several Hive 3 tests on Java 17 for StringInternUtils illegal access
- Refactored nifi-enrich-processors tests to use Mockito without Powermock
- Refactored nifi-toolkit-tls tests to avoid illegal reflective access
- Removed deprecated X509Certificate test in CertificateUtilsTest
- Removed kryo serialization from nifi-site-to-site-client test
- Updated TestHashContent to use SHA-1 instead of SHA for hash algorithm
- Upgraded maven-war-plugin from 2.5 to 3.3.2
- Upgraded nifi-graph-bundle dependencies from Groovy 2.5.14 to 3.0.8
- Upgraded QuestDB from 4.2.1 to 6.2.1 in nifi-framework-core

This closes 

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-03-17 14:08:38 -04:00
Mike Thomsen
df00cc6cb5
NIFI-6047 Cleaned up code to allow tests to run against 1.13.0-snapshot
Removed DMC.
NIFI-6047 Started integrating changes from NIFI-6014.
NIFI-6047 Added DMC tests.
NIFI-6047 Added cache identifier recordpath test.
NIFI-6047 Added additional details.
NIFI-6047 Removed old additional details.
NIFI-6047 made some changes requested in a follow up review.
NIFI-6047 latest.
NIFI-6047 Finished updates
First round of code review cleanup
Latest
Removed EL from the dynamic properties.
Finished code review requested refactoring.
Checkstyle fix.
Removed a Java 11 API
NIFI-6047 Renamed processor to DeduplicateRecord

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes 
2022-03-09 19:06:44 -05:00
Adam
23132fb89f
NIFI-6047
Added NiFi DetectDuplicateRecord standard processor.
Adding some documentation and PR review tweaks.
Exposing processor
Documentation updates, exception handling consolidation, added support for record path field variables.
Added tests.
Build bump.
Migrated cache service to groovy folder.
Moved declarations for properties to @BeforeClass lifecycle method.
Adding some documentation and PR review tweaks.
Documentation updates, exception handling consolidation, added support for record path field variables.
Added tests.
Build bump.
Migrated cache service to groovy folder.
Fixed variable type bug.
Fixed mapping of test params to usage.
Fixed potential illegal state exception bug.
2022-03-09 19:06:08 -05:00
Mark Payne
6a1c7c72d5
NIFI-9781: Fix handling when selecting array element via QueryRecord
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes .
2022-03-09 13:07:15 -05:00
Mark Payne
65dd62716a
NIFI-9748: Added new property for Output Format to LogAttribute. Also made the FlowFile Properties (file size, entry date, lineage start date) optional and renamed from 'Standard FlowFile Attributes' to 'FlowFile Properties' because this has led to confusion many times in the past, around users wanting to reference these things as attributes via EL but they are not actually attributes.
This closes 

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-03-04 08:24:39 -05:00
Mark Payne
0d0912d34b
NIFI-9400: Ensure that we always use the CollectionUsage metrics in the mbeans instead of the Usage metrics.
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes .
2022-03-01 14:17:32 -05:00
exceptionfactory
ddecfa201b
NIFI-9736 Improved TestRouteText to avoid intermittent failures ()
- Refactored testPatternCache to avoid testing maximum cache size reached
2022-02-28 14:26:19 -05:00
exceptionfactory
1eede30974
NIFI-9732 Upgraded Zip4j from 2.8.0 to 2.9.1
This closes 

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-02-27 06:09:14 -05:00
Matthew Burgess
bcc8d03314
NIFI-9585 Upgraded H2 from 1.4 to 2.1.210
- Added nifi-h2-database module shading H2 1.4.200
- Implemented version checking and migration

This closes 

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-02-25 19:20:48 -06:00
markobean
4e3871fec7
NIFI-9072: improvements to ValidateXML including validate XML in attr… ()
NIFI-9072: Improvements to ValidateXML including validate XML in attributes and non-schema, well-formed XML only validation, and additional documentation
- Updated description and added System Resource Consideration to discourage using large XML documents in attributes
- Migrate from XML DocumentBuilderFactory to SafeXMLConfiguration
2022-02-25 14:57:04 -05:00
Mark Payne
39483b9c12
NIFI-9716, NIFI-9577: Addressed issue in the PathFilter for GetFile / ListFile. For any file that is found in the Input Directory directly, it was previously being listed/fetched even if it didn't match the PathFilter. Additionally, updated the code to create a new File FIlter for every invocation of onTrigger. This was necessary for NIFI-9577 because the directory to monitor supports Expression Language and as a result may change from invocation to invocation, if using a function such as now() but the PathFilter would always relativize the path based on the value that was obtained when the processor was scheduled.
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes .
2022-02-24 09:13:16 -05:00
Mark Payne
ce0122bd25
NIFI-9691: Added ForkEnrichment, JoinEnrichment processors
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes .
2022-02-22 18:54:32 -05:00
levilentz
dfbf2e3cea
NIFI-9286: JOLT Expression Language
Fixes NIFI-6213 and adds in functionality to use expression language in class and module specification
NIFI-9286: adding JOLT unit tests
NIFI-9286: addressing PR feedback
Fixes a problem with the scope of the EL for module directory
NIFI-9286: alignment of JOLT processors
NIFI-9286: fix checkstyle

This closes 

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-02-19 14:11:44 -05:00
Matthew Burgess
03165ad817 NIFI-9607: Honor Update Keys when Quoting Identifiers in PutDatabaseRecord
NIFI-9607: Fixed wrong column name in WHERE clause for generateUpdate

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes .
2022-02-16 16:07:34 -05:00
exceptionfactory
fe1139b8bb
NIFI-9692 Upgraded Apache Commons Lang3 to 3.12.0
- Moved commons-lang3 version management to root Maven configuration
- Refactored limited usage of StringUtils is nifi-reporting-utils to remove commons-lang3
- Refactored limited usage of StringUtils in nifi-websocket-processors to remove commons-lang3

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes 
2022-02-16 09:28:21 -05:00
Gabor Gyimesi
6788076536
NIFI-9058 Corrected AttributesToJSON Core Attributes filtering
- Avoiding filtering Core Attributes from configured Attributes List

This closes 

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-02-15 15:51:39 -06:00
exceptionfactory
7dfbf99d5f
NIFI-9527 Upgraded snappy-java to 1.1.8.4
- Set managed version in root Maven configuration

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes 
2022-02-15 16:12:36 -05:00
exceptionfactory
a2f6420f43
NIFI-9638 Refactored Google Guava usage in extensions
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes 
2022-02-09 16:39:53 -05:00
exceptionfactory
e584d3cf04
NIFI-9660 Upgraded Apache Tika to 2.3.0
- Upgraded tika-core from 1.27 to 2.3.0
- Upgraded tika-parsers to tika-parsers-standard-package in nifi-media-processors
- Updated Tika metadata property references

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes .
2022-02-09 11:11:46 +01:00
Noel Baek
bbc78f1547
NIFI-9621: Added Ignore Reserved Characters to FlattenJson
- Upgraded to json-flattener from 0.12.0 to 0.13.0

This closes 

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-02-08 21:02:55 -06:00
exceptionfactory
ce66cf41e2
NIFI-6699 Corrected SFTP symbolic link handling ()
- Added SFTP stat() request on symbolic links to check directory status
- Refactored and renamed TestServerSFTPTransfer class
2022-02-04 18:43:42 -05:00
exceptionfactory
ba0bf2ede0
NIFI-9650 Upgraded OkHttp from 4.9.2 to 4.9.3
- Added okhttp-bom to root Maven configuration
- Removed repeated okhttp.version property from specific dependencies

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes .
2022-02-02 11:45:18 +01:00
exceptionfactory
43748a5523 NIFI-9638 Refactored Google Guava references
- Refactored nifi-framework and nifi-standard modules
- Replaced Google Cache with Caffeine Cache
- Replaced Google collections classes with standard Java collections

This closes .

Signed-off-by: Kevin Doran <kdoran@apache.org>
2022-02-01 15:17:50 -05:00
exceptionfactory
dc7d9510cd NIFI-9610 Refactored nifi-processor-utils to separate modules
- Added nifi-bin-manager
- Added nifi-event-listen
- Added nifi-event-put
- Added nifi-listed-entity
- Added nifi-put-pattern
- Removed nifi-processor-utils

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes .
2022-02-01 12:59:16 -05:00
Bence Simon
49978cdd91
NIFI-9341 Added CEF RecordReader
This closes 

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-01-28 12:42:51 -06:00
exceptionfactory
95abc35463 NIFI-6266 Corrected proxy FTP connect handling
- Added ProxyFTPClient with connect method supporting unresolved hosts
- Added FTPClientProvider with implementation handling configuration and connection
- Added support for SOCKS with authentication in FTP components
- Refactored FTPTransfer using FTPClientProvider

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes .
2022-01-27 17:36:27 -05:00
Nathan Gough
08153b8260
NIFI-9453 Refactored ListenBeats using Netty
This closes 

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-01-26 14:40:32 -06:00