Commit Graph

336 Commits

Author SHA1 Message Date
Peter Gyori 3f16a41ca1
NIFI-10010: ListenTCP adds Certificate Subject and Issuer FlowFile attributes
This closes #6032

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-05-13 15:48:12 -05:00
exceptionfactory 0c8305950c
NIFI-9987 Standardize Commons IO on version 2.11.0
- Removed version declarations from multiple modules
- Adjusted PutDynamoDB to remove catch for IOException not thrown in Commons IO 2.11.0

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #6015.
2022-05-09 12:06:37 +02:00
Mike Thomsen 7271e8cea7
NIFI-7234 Standardized on Avro 1.11.0
NIFI-7234 Replaced Jackson 1.X references with Jackson 2.X references in various classes.
NIFI-7234 Added jackson-annotations to nifi-hdfs-processors.
NIFI-7234 Various updates to bring our test cases into better alignment with the Avro specification as of 1.11.
Fixed a checkstyle issue.

NIFI-7234 Made changes requested in a review.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5900
2022-04-29 17:48:59 -04:00
exceptionfactory 4f423a59ba
NIFI-9952 Upgraded Jackson BOM to 2.13.2.20220328
- Removed unnecessary references to jackson.version property
- Removed unnecessary dependency management references to Jackson libraries

This closes #5992

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-04-23 08:23:39 -04:00
Joe Witt 84340962d4
NIFI-9927 Initial Commit to clean up all extraneous version/group declarations.
This closes #5969

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-04-16 12:05:07 -04:00
Peter Turcsanyi 47f75651bb
NIFI-9911: Fixed checking HDFS URI in ListHDFS' Directory property
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5957
2022-04-12 08:50:49 -04:00
Paul Grey 900a34971c
NIFI-9902 - Improve handling of charsets when performing string/byte conversions
This closes #5954

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-04-11 13:22:13 -04:00
Joe Witt 873d25585c
NIFI-9780 Merge branch 'NIFI-9780-RC3' 2022-04-04 08:35:23 -07:00
Nathan Gough 2a6e521549
NIFI-9702 ListSFTP will set mime.type when Record Writer configured
This closes #5891

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-03-28 16:03:00 -05:00
Peter Gyori 6ade47ac4f
NIFI-9838: Added Client Certificate attributes to ListenTCPRecord FlowFiles
This closes #5908

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-03-28 13:36:16 -05:00
Nathan Gough 2fbe10b4bc
NIFI-9701 - Corrected No Tracking strategy to create one flow file when using a Record Writer
- Corrected No Tracking strategy Record Writer handling for ListSFTP
- Updated temporary test files to have last modified time of epoch to avoid intermittent issue with Minimum Age filtering
- Refactored MockCacheService to separate reusable class

This closes #5885

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-03-28 12:47:30 -05:00
Joe Witt 0419dc2939
NIFI-9780-RC3 prepare for next development iteration 2022-03-21 11:58:27 -07:00
Joe Witt b019a9191f
NIFI-9780-RC3 prepare release nifi-1.16.0-RC3 2022-03-21 11:58:09 -07:00
Nathan Gough 713e2fd03c
NIFI-9645 - Updated PutSplunk to allow idle connection timeouts
This closes #5841

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-03-14 11:15:16 -05:00
Bence Simon b86466b4a5 NIFI-9669 Adding PutDynamoDBRecord processor
This closes #5761.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2022-03-09 16:20:33 +01:00
Mike Thomsen ae1731c79e
NIFI-9134 nifi-metrics-reporting-bundle to use JUnit 5
- NIFI-9129 Refactored as much of nifi-kudu-bundle to use JUnit 5 as possible without breaking the integration tests which use test libraries from the Kudu project that depend on JUnit 4
- NIFI-9109 Refactored nifi-extension-utils to use JUnit 5. Left two tests with JUnit 4 because they used APIs that don't appear to have analogues in JUnit 5
- NIFI-9112 Refactored nifi-groovyx-bundle to use JUnit 5
- NIFI-9142 Refactored nifi-ranger-bundle to use JUnit 5
- NIFI-9136 Refactored nifi-network-bundle to use JUnit 5

This closes #5794

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-02-26 08:18:08 -06:00
Matthew Burgess bcc8d03314
NIFI-9585 Upgraded H2 from 1.4 to 2.1.210
- Added nifi-h2-database module shading H2 1.4.200
- Implemented version checking and migration

This closes #5724

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-02-25 19:20:48 -06:00
Matthew Burgess ab4cadc204
NIFI-9721: Support enum types in AvroTypeUtil.buildAvroSchema()
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5793.
2022-02-23 12:44:57 -05:00
Mark Payne ce0122bd25
NIFI-9691: Added ForkEnrichment, JoinEnrichment processors
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5772.
2022-02-22 18:54:32 -05:00
Mark Payne 4d8c79d7f3 NIFI-9698: When creating an Avro schema, ensure that any default value is converted from what is returned by RecordField.getDefaultValue() to what Avro requires.
Signed-off-by: Chris Sampson <chriss@apache.org>

This closes #5776
2022-02-16 20:37:50 +00:00
exceptionfactory fe1139b8bb
NIFI-9692 Upgraded Apache Commons Lang3 to 3.12.0
- Moved commons-lang3 version management to root Maven configuration
- Refactored limited usage of StringUtils is nifi-reporting-utils to remove commons-lang3
- Refactored limited usage of StringUtils in nifi-websocket-processors to remove commons-lang3

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5773
2022-02-16 09:28:21 -05:00
exceptionfactory bab4309905
NIFI-9655 Add Queue Logging to ListenUDP (#5747)
* NIFI-9655 Added Queue Logging to ListenUDP

- Added TrackingLinkedBlockingQueue to track the largest queue size
- Updated AbstractListenEventProcessor to write debug log at most once per minute
- AbstractListenEventProcessor updates support both ListenUDP and ListenUDPRecord
2022-02-04 18:41:38 -05:00
exceptionfactory dc7d9510cd NIFI-9610 Refactored nifi-processor-utils to separate modules
- Added nifi-bin-manager
- Added nifi-event-listen
- Added nifi-event-put
- Added nifi-listed-entity
- Added nifi-put-pattern
- Removed nifi-processor-utils

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5729.
2022-02-01 12:59:16 -05:00
Mark Payne 038811d0dd
NIFI-9629: Ensure that when we are setting default values on Avro GenericRecord objects that we convert from the schema's default value to the proper type
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5716.
2022-01-29 13:56:53 +01:00
Nathan Gough 8f2a9f94fe
NIFI-9632 - Removed nifi-lumberjack-bundle
- Removed several unused socket classes from nifi-processor-utils

This closes #5722

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-01-27 14:43:19 -06:00
Timea Barna 3ccc9d29b6
NIFI-9455 Added aggregated predictions to Prometheus Flow Metrics
This closes #5582

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-01-27 10:07:01 -06:00
exceptionfactory e2243500d3
NIFI-9618 Upgraded Checkstyle to 9.2.1
- Corrected several missing return statements
- Increased Maven heap size to 3 GB for ci-workflow

This closes #5700

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-01-21 20:08:16 -06:00
Mark Payne 1d353435a5
NIFI-9594: When converting Record to Avro GenericRecord, ensure that any default values that are defined in the GenericRecord's schema get applied, regardless of whether or not the field exists in the associated RecordSchema.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5677.
2022-01-19 19:20:53 +01:00
Joe Witt 2f95eed8cb
NIFI-9575 This closes #5667. updating copyright year to 2022
Signed-off-by: Joe Witt <joewitt@apache.org>
2022-01-19 09:14:09 -07:00
exceptionfactory 405934dcd2
NIFI-9571 Corrected Session commit handling in PutTCP
- Added generic type to AbstractPutEventProcessor for compiler checking of event types
- Refactored createTransitUri to shared method in AbstractPutEventProcessor

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5658.
2022-01-18 15:31:36 -05:00
exceptionfactory 7c50f1429e
NIFI-9563 Enabled ListenTCP Pool Receive Buffers property
- Removed deprecation from ListenTCP Pool Receive Buffers property
- Added BufferAllocator configuration property for NettyEventServerFactory

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5653.
2022-01-13 19:50:44 +01:00
Mark Payne 72e54f4fab NIFI-9390: Updates to MergeContent / MergeRecord so that they play nicely within Stateless
NIFI-9390: Addressed underlying condition in stateless framework that caused Merge-related processors and similar to not properly be triggered as necessary. Added several system tests to verify different configurations.

NIFI-9390: Simplified the logic for how to iterate over the components in a Stateless flow that are ready to be triggered

This closes #5634.

Co-authored-by: Peter Turcsanyi <turcsanyi@apache.org>
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2022-01-11 09:45:00 +01:00
Tamas Palfy 990285ba1c NIFI-8676 Added 'Tracking Entities' listing strategy to 'ListS3' and 'ListGCSBucket'
This closes #5413.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2022-01-10 20:35:04 +01:00
exceptionfactory 5832dff25e
NIFI-9556 Upgraded Apache HttpClient to 4.5.13
- Upgraded Apache HttpCore to 4.4.15
- Added dependency management declarations in root Maven configuration for HttpClient and HttpCore
- Removed version numbers from multiple modules

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5647.
2022-01-08 11:06:07 +01:00
Mark Payne 56593ad12f
NIFI-9546: Updated PutTCP and PutUDP with SupportsBatching annotations
- Removed incorrect usage of TriggerWhenEmpty
- Allow for 0 seconds of gracefully waiting for incoming threads on shutdown since we won't have any
- Updated unit tests to no longer have arbitrary sleep statements

This closes #5639

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-01-06 15:52:37 -06:00
exceptionfactory 12ba579b8f NIFI-9539 Upgraded Netty from 4.1.69 to 4.1.72
- Added Netty 4 BOM to root Maven configuration
- Removed individual Netty 4 dependency overrides
- Removed commented dependencies and unnecessary declarations from MiNiFi

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5635.
2022-01-05 18:11:03 -05:00
exceptionfactory 7c6bdcb035
NIFI-9491 Excluded commons-logging and added jcl-over-slf4j references
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5608
2021-12-17 07:21:03 -05:00
Nathan Gough 5e9c86885c
NIFI-9478 Moved Netty Log Exception Handler to end of pipeline
- Changed display name of Max Number of TCP Connections to Max Number of Worker Threads for ListenTCP
- Set Netty Socket Receive Buffer using Max Socket Buffer Size in ListenTCP

This closes #5599

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-12-15 16:59:07 -06:00
Mark Payne 97198e35a0
NIFI-9382: This closes #5584. Added system test that replicates issue in which a closed shared classloader causes issues when used again
NIFI-9382: Fixed issue with SharedInstanceClassLoader where the classloader may get closed but then get used again. When the SharedInstanceClassLoader is closed, we will now ensure that we don't use anymore and instead create a new one.

Signed-off-by: Joe Witt <joewitt@apache.org>
2021-12-09 12:41:57 -07:00
Tamas Palfy ff864266f5
NIFI-9436 - In AbstractPutHDFSRecord make sure the record writers use the FileSystem object the processor already has.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5565
2021-12-02 14:24:12 -05:00
markap14 839fbf7d19
NIFI-9382: Created a new ClassloaderIsolationKey mechanism by which H… (#5533)
* NIFI-9382: Created a new ClassloaderIsolationKey mechanism by which Hadoop related processors (and potentially others) can indicate that they need full classloaders to be cloned but can share with other instances in certain circumstances
- Added system tests

* NIFI-9382: Renamed interface based on review feedback

* NIFI-9382: Removed ReentrantKerberosUser.
2021-11-22 14:55:19 -05:00
exceptionfactory 0cf515c9c0 NIFI-9384 Corrected usage and generics in ListenTCP
- Addressed compiler warnings in ListenTCP and EventBatcher
- Adjusted ListenTCP property order to match previous version

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5526.
2021-11-17 11:54:51 -05:00
noblenumbat360 f410c8df0a
NIFI-8272 Delete stale metrics from REST API Prometheus endpoint.
Added <scope>test</scope> tag to the nifi-web-api pom.xml and corrected imports.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5447
2021-11-08 20:26:30 -05:00
Joe Witt 9e845d87ff
Merge branch 'NIFI-9344-RC3' 2021-11-07 16:23:10 -07:00
Mark Payne 705c65c86c
NIFI-9362: Ensure that we update the StateMap in AbstractListProcessor to hold any files whose date matches the latest before setting cluster-wide state
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5507
2021-11-04 13:46:41 -04:00
Joe Witt c055895952
NIFI-9344-RC3 prepare for next development iteration 2021-11-03 08:53:37 -07:00
Joe Witt 7fdc07cccd
NIFI-9344-RC3 prepare release nifi-1.15.0-RC3 2021-11-03 08:53:32 -07:00
Bence Simon 75d1d7f6e7
NIFI-9265 Fixing path handling for HDFS processors when there are multiplied separators in the path
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5437.
2021-10-28 10:47:53 +02:00
exceptionfactory 60d6d469bf
NIFI-9342 Upgraded to Netty 3.10.6 and 4.1.69
- Replaced Netty 3.6.9 and 3.7.1 with 3.10.6
- Replaced Netty 4.1 with 4.1.69

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5490.
2021-10-28 09:34:31 +02:00
exceptionfactory 67ccdf6159
NIFI-9326 Added Socket Keep Alive property to ListenSyslog
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5479.
2021-10-26 19:44:56 +02:00