63 Commits

Author SHA1 Message Date
Paul Grey
75c661bbbe
NIFI-12194 Added Yield on Exceptions in Kafka Processors
- Catching KafkaException and yielding for publisher lease requests improves behavior when the Processor is unable to connect to Kafka Brokers

This closes #7955

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-11-01 11:08:44 -05:00
exceptionfactory
bd4ba34339
NIFI-12266 Added Standard Shared NAR and BOM (#7925)
- Added nifi-standard-shared-nar with common dependencies
- Added nifi-standard-shared-bom with provided scope for parent module references
2023-10-25 13:22:26 -04:00
Bence Simon
3f13604c36
NIFI-11772 Removed flow.xml.gz support
- Created NIFI-12203 to evaluate issues with flow comparison surfaced in JoinClusterWithDifferentFlow

This closes #7661

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-10-11 13:53:01 -05:00
exceptionfactory
98cebc6687
NIFI-12185 Added nifi-bom Bill of Materials (#7852)
- Imported nifi-bom to nifi-commons and nifi-nar-bundles parent modules
- Removed explicit version and scope properties for dependencies declared in nifi-bom
2023-10-10 13:55:22 -04:00
Mark Payne
db727aa419
NIFI-12142: This closes #7806. Deleted many methods, classes, and references that were deprecated.
Signed-off-by: Joseph Witt <joewitt@apache.org>
2023-09-28 14:07:48 -07:00
Pierre Villard
43cc5ad122
NIFI-12079 This closes #7750. Remove Variables / Variable Registry
Signed-off-by: Joseph Witt <joewitt@apache.org>
2023-09-25 09:39:53 -07:00
Peter Turcsanyi
ebd0d60e49
NIFI-12075 Deprecated ComponentLog methods with Object[] and Throwable
- Updated component references to remove use of Object[] wrapping for log methods

This closes #7748

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-09-19 08:53:00 -05:00
Paul Grey
5cad5838c5
NIFI-11840 Add Kafka Partition and Topic Attributes before handoff to RecordReader
This closes #7510

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-07-24 13:40:22 -05:00
Matthew Burgess
df04c60e01
NIFI-8326: Send records as individual messages in Kafka RecordSinks
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4901.
2023-05-24 10:43:30 +02:00
Nandor Soma Abonyi
ed1cee4520
NIFI-11202 Removed deprecated Kerberos properties from Kafka 2_6
This closes #6978

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-03-09 16:04:58 -06:00
exceptionfactory
1156f4cbc5
NIFI-11165 This closes #6944. Removed optional jigsaw build profiles
- Removed jigsaw build profiles
- Removed Java 11 directories from assemblies

Signed-off-by: Joe Witt <joewitt@apache.org>
2023-02-13 13:01:58 -05:00
Joe Witt
43eab4c81d
NIFI-11103 prepping for 2.0.0 line 2023-02-09 15:32:53 -07:00
Paul Grey
b426226207
NIFI-10993 - PublishKafkaRecord should use correct record schema
This closes #6833.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2023-01-23 21:40:29 +01:00
Nandor Soma Abonyi
eb5d172693
NIFI-10991 Add AWS MSK IAM support to Kafka processors
This closes #6846.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2023-01-23 12:33:49 +01:00
greyp9
282c56b5ce
NIFI-10901 - PublishKafka headers not sent in ProducerRecord (#6731) 2022-11-30 13:23:23 -05:00
Joe Witt
75e7c9e47c
NIFI-10854-RC1 prepare for next development iteration 2022-11-22 19:22:47 -07:00
Joe Witt
ec87bf93ad
NIFI-10854-RC1 prepare release nifi-1.19.0-RC1 2022-11-22 19:22:45 -07:00
exceptionfactory
8e417c890a
NIFI-10819 Restricted JndiLoginModule for Kafka JAAS Config
- Refactored property configuration and validation to nifi-kafka-shared

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #6690.
2022-11-22 16:51:23 -05:00
greyp9
30facedc43
NIFI-9822 - ConsumeKafkaRecord allows writing out Kafka record key (#6131)
* NIFI-9822 - ConsumeKafkaRecord allows writing out Kafka record key
* publisher wrapper record; property naming and display order; doc updates
2022-11-16 16:07:43 -05:00
Matt Gilman
e638bcee2f
NIFI-10617:
- Adding dynamic property annotation on a number of extensions that were missing it.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6501
2022-10-10 10:32:50 -04:00
Joe Witt
c9ebdd2025
NIFI-10521-RC4 prepare for next development iteration 2022-10-03 10:59:36 -07:00
Joe Witt
109e54cd58
NIFI-10521-RC4 prepare release nifi-1.18.0-RC4 2022-10-03 10:59:34 -07:00
Daniel Urban
387ae7d323
NIFI-10199: PublishKafka InFlightMessageTracker should not log batch level errors on each record
Currently, when a batch level error occurs (e.g. delivery timeout), the same error is logged for each record of the batch, needlessly flooding the logs.
InFlightMessageTracker now only logs an exception when a flow file encounters it for the first time.

This closes #6185
Signed-off-by: Paul Grey <greyp@apache.org>
2022-08-05 14:12:43 -04:00
Joe Witt
8c66bf948c
NIFI-10272-RC2 prepare for next development iteration 2022-07-27 13:24:09 -07:00
Joe Witt
8d256784d8
NIFI-10272-RC2 prepare release nifi-1.17.0-RC2 2022-07-27 13:24:06 -07:00
exceptionfactory
0c8305950c
NIFI-9987 Standardize Commons IO on version 2.11.0
- Removed version declarations from multiple modules
- Adjusted PutDynamoDB to remove catch for IOException not thrown in Commons IO 2.11.0

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #6015.
2022-05-09 12:06:37 +02:00
Ravinarayan Singh
28c7583bdb
fixed error message reporting (#5979) 2022-04-21 09:12:02 -04:00
Nandor Soma Abonyi
9774bb90ca
NIFI-9917 fix the tooltip of "delivery guarantee/best effort" to a more accurate one
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5963.
2022-04-20 16:25:42 +02:00
Joe Witt
0419dc2939
NIFI-9780-RC3 prepare for next development iteration 2022-03-21 11:58:27 -07:00
Joe Witt
b019a9191f
NIFI-9780-RC3 prepare release nifi-1.16.0-RC3 2022-03-21 11:58:09 -07:00
Mark Payne
e6229ab938
NIFI-9771: When a Kafka record is obtained during config verification, we should produce an invalid response if the Record Reader is not able to produce any records from it
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5847.
2022-03-08 06:42:27 -05:00
Mike Thomsen
51cf0fb0d1
NIFI-9133 Refactored nifi-media-bundle to use JUnit 5
- NIFI-9130 Refactored nifi-language-translation-bundle to use JUnit 5
- NIFI-9126 Refactored nifi-kafka-bundle to use JUnit 5
- NIFI-9125 Refactored nifi-jolt-record-bundle to use JUnit 5

This closes #5792

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-02-23 21:18:25 -06:00
exceptionfactory
9075fc2cd8
NIFI-9611 Removed unnecessary references to nifi-processor-utils
- Replaced nifi-processor-utils with nifi-utils where necessary
- Added direct dependencies on commons-lang3 where necessary

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5693.
2022-01-26 11:52:08 -05:00
Pierre Villard
8638b7c0d8
NIFI-9466 - PublishKafka 2.x - change default delivery guarantee from best effort to replicated
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5586
2021-12-16 15:10:32 -05:00
Pierre Villard
03cf976c07
NIFI-9467 - Update description for DefaultPartitioner in PublishKafka 2.6 to mention sticky partitioning
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5587
2021-12-16 15:07:55 -05:00
Joe Witt
c055895952
NIFI-9344-RC3 prepare for next development iteration 2021-11-03 08:53:37 -07:00
Joe Witt
7fdc07cccd
NIFI-9344-RC3 prepare release nifi-1.15.0-RC3 2021-11-03 08:53:32 -07:00
markap14
90ae271692
NIFI-9310: Addressed issues found while testing the component verification features. Added a supportsVerification flag to the ConfigAnalysis DTO (#5469)
This closes #5469
2021-10-20 11:07:17 -04:00
Bryan Bende
4bcfc61563
NIFI-9011 Correct tags in publish Kafka 2.6 processors
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5464.
2021-10-19 16:48:15 +02:00
Mark Payne
607480c1a0
NIFI-9239: Updated Consume/Publish Kafka processors to support Exactly Once Semantics. Added ExecuteStateless processor/bundle
- A few minor updates to Stateless in order to surface some concepts from the stateless engine up to the caller, such as bulletins, counters, etc.

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5412.
2021-10-04 15:32:21 -04:00
Mark Payne
baf29e59f1
NIFI-9009: Created VerifiableProcessor, VerifiableControllerService, VerifiableReportingTask components; implemented backend work to call the methods. Added REST APIs and created/updated data models for component configuration verification
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5288
2021-09-22 19:00:23 -04:00
exceptionfactory
f5a421ccb1
NIFI-9067 Upgraded JAXB Runtime from 2.3.0 to 2.3.5
- Replaced old com.sun.xml.bind:jaxb-impl and jaxb-core with current org.glassfish.jaxb:jaxb-runtime
- Replaced old javax.xml.bind:jaxb-api with current jakarta.xml.bind-api
- Removed unnecessary dependency references to javax.activation-api

This closes #5320

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2021-09-07 13:27:53 -04:00
Bryan Bende
21c2fb95d3
NIFI-8973 Implement KerberosUserService API and keytab, password, and ticket cache implementations
NIFI-8974 Integrate KerberosUserService with HDFS processors

NIFI-8980 Integrate KerberosUserService with Kafka 2.6 processors
- Introduced SelfContainerKerberosUserService to restrict which impls can be used with Kafka
- Add variations of KerberosUser doAs that allow setting the context ClassLoader
- Add additional unit tests for configurations

This closes #5277

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-08-18 19:54:56 -05:00
Joe Witt
97feacc181
NIFI-8767-RC2 prepare for next development iteration 2021-07-10 12:17:09 -07:00
Joe Witt
fcbf1d5f97
NIFI-8767-RC2 prepare release nifi-1.14.0-RC2 2021-07-10 12:17:05 -07:00
exceptionfactory
2efc2b6b8b
NIFI-8718 Upgraded Apache Commons IO to 2.10.0
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5169
2021-06-21 11:29:27 -04:00
Mark Payne
ecacfdaa4c NIFI-8469: Introduced ProcessSession.commitAsync and updated processors to use it. Deprecated ProcessSession.commit()
- Updated Mock Framework to now fail tests that use ProcessSession.commit() unless they first call TestRunner.setAllowSynchronousSessionCommits(true)
- Updated stateless nifi in order to make use of async session commits
- Fixed bug that caused stateless to not properly handle Additional Classpath URLs and bug that caused warnings about validation to get generated when a flow that used controller services was initialized. While this is not really in scope of NIFI-8469, it was found when testing and blocked further progress so addresssed here.
- If Processor fails to progress when run from stateless, trigger from start of flow until that is no longer the case
- Introduced notion of TransactionThresholds that can limit the amount of data that a flow will bring in for a given invocation of stateless dataflow
- Several new system-level tests
2021-05-19 08:51:16 -04:00
exceptionfactory
6776765a92
NIFI-8538 Upgraded Apache Commons IO to 2.8.0
- Upgraded direct dependencies from 2.6 to 2.8.0
- Added dependency management configuration to use 2.8.0 for some modules
- Updated scripted Groovy tests to avoid copying unnecessary files

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5073
2021-05-14 09:31:48 -04:00
Mohammed Nadeem
7d5903c323
NIFI-8412: This closes #4993. Use proper log messages in PublishKafka(Record) processors
incase of 'Rollback on Failure'

Signed-off-by: Joe Witt <joewitt@apache.org>
2021-04-13 07:25:50 -07:00
Mark Payne
7d1d536da6
NIFI-8206: Added identifiesExternalResource() method to PropertyDescriptor.Builder and implemented functionality.
- Updated components to make use of new feature

NIFI-8206: Added a ResourceType of TEXT. This requires that the ResourceReferenceFactory know which types are allowed in order to create the ResourceReference. PropertyValue needs to then have the PropertyDescriptor available to it. This resulted in highlighting many bugs in unit tests where components were not exposing property descriptors via getSupportedPropertyDescriptors() or were evaluating Expression Language using the wrong scope, so fixed many unit tests/components to properly declare Expression Language scope when using it

NIFI-8206: Removed problematic unit test that required directory names with special characters that are not allowed on some operating systems

This closes #4890.

Signed-off-by: Bryan Bende <bbende@apache.org>
2021-04-12 11:48:04 -04:00