4698 Commits

Author SHA1 Message Date
Pierre Villard
37c0527a72
NIFI-9304 - Adding PublishGCPubSubLite and ConsumeGCPubSubLite processors
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5460.
2021-11-08 09:25:35 -05:00
Bryan Bende
c96809012b
NIFI-9350 Add NiFi Registry NarProvider implementation
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5497.
2021-11-08 09:18:10 -05:00
Chris Sampson
f86fe0d61a
NIFI-9366 Create and remove provenance repository directory in test methods
- Updated nifi-persistent-provenance-repository tests

This closes #5510

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-11-08 07:40:05 -06:00
Joe Witt
9e845d87ff
Merge branch 'NIFI-9344-RC3' 2021-11-07 16:23:10 -07:00
Bence Simon
9b724df6d9 NIFI-9260 Making the 'write and rename' behaviour optional for PutHDFS
This closes #5423.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2021-11-04 20:19:26 +01:00
Mark Payne
705c65c86c
NIFI-9362: Ensure that we update the StateMap in AbstractListProcessor to hold any files whose date matches the latest before setting cluster-wide state
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5507
2021-11-04 13:46:41 -04:00
shadon
a9685b36fc
optimize import. (#5323)
Co-authored-by: pengxd <pengxd@hua-cloud.com.cn>
2021-11-04 09:29:11 -04:00
Tamas Palfy
2bd752d868
NIFI-9202 Improve Allowable Values merging to handle cases when different nodes have different set of Allowable Values.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5380
2021-11-03 15:09:37 -04:00
Joe Witt
c055895952
NIFI-9344-RC3 prepare for next development iteration 2021-11-03 08:53:37 -07:00
Joe Witt
7fdc07cccd
NIFI-9344-RC3 prepare release nifi-1.15.0-RC3 2021-11-03 08:53:32 -07:00
Pierre Villard
decfad394d
NIFI-9349 - add attribute with request duration in InvokeHTTP (#5501)
This closes #9349

Signed-off-by: Otto Fowler <ottobackwards@gmail.com>
2021-11-03 09:25:57 -04:00
Paul Grey
362a243e0f
NIFI-9360 Update PutHDFS to handle filesystems which do not support getAclStatus()
This closes #5505

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-11-02 21:57:03 -05:00
Mark Payne
09feed9df6
NIFI-9352: Consider all property values in component references
- Ensure that we consider all property values to determine if a component references a variable, not just 'supported' (i.e., well-known/non-user-defined) properties

This closes #5499

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-10-30 11:16:57 -05:00
exceptionfactory
1c4ee93e68
NIFI-9351 Excluded test dependencies from nifi-scripting-nar
NIFI-9351 Excluded Groovy test dependencies

- Updated nifi-scripting-nar
- Updated nifi-groovyx-nar

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5498
2021-10-29 17:25:14 -04:00
exceptionfactory
16e6045d13 NIFI-9346 Added closing of EventSender to TestListenRELP
Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5492.
2021-10-28 13:10:04 -04:00
exceptionfactory
9865ea2bfb NIFI-9322 Refactored OIDC and SAML Access Resources
- Removed parent AccessResource from OIDCAccessResource and SAMLAccessResource to avoid unexpected inherited methods
- Moved Token Expiration validation from AccessResource to StandardBearerTokenProvider

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5489.
2021-10-28 12:12:57 -04:00
Joe Gresock
28cd5d1300
NIFI-9345: Resolving NPE in ProcessorEntityMerger (#5491)
NIFI-9345: Resolving NPE in ProcessorEntityMerger, Streamlining forEach calls
2021-10-28 12:06:56 -04:00
Bence Simon
75d1d7f6e7
NIFI-9265 Fixing path handling for HDFS processors when there are multiplied separators in the path
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5437.
2021-10-28 10:47:53 +02:00
exceptionfactory
60d6d469bf
NIFI-9342 Upgraded to Netty 3.10.6 and 4.1.69
- Replaced Netty 3.6.9 and 3.7.1 with 3.10.6
- Replaced Netty 4.1 with 4.1.69

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5490.
2021-10-28 09:34:31 +02:00
Mark Payne
3114bdb701
NIFI-9311: When determining property values, be sure to fetch the property descriptor from the component itself, rather than using the PropertyDescriptor in the Map. This allows us to ensure that if the definition of the PropertyDescriptor changes, the most up-to-date definition is picked up
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5488
2021-10-27 17:08:06 -04:00
Lehel Boér
059f14fd62
NIFI-8385: Add FlowFiles from logging to bulletins (#4952) 2021-10-27 16:06:30 -04:00
Matthew Burgess
104078868e
NIFI-7012: Refactored OnConfigurationRestored to support sensitive property validation (#5415) 2021-10-27 14:40:17 -04:00
Joe Gresock
ca530f40d8
NIFI-9317: Updating config verification for ListS3 (#5485) 2021-10-27 13:47:58 -04:00
Mark Payne
0eee70c4b9 NIFI-9339: Fixed headers to include appropriate Reqest-Token header for all requests to the server for JoltTransformJSON UI
Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5487.
2021-10-27 11:40:30 -04:00
Lance Kinley
30efcd35e4
NIFI-8208: Upgrade MongoDB driver to 4.3 series
- Add AllowableValue objects for write concern values
- Remove warnings for using deprecated write concerns in PutMongo

This closes #5392

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-10-26 16:14:55 -05:00
exceptionfactory
67ccdf6159
NIFI-9326 Added Socket Keep Alive property to ListenSyslog
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5479.
2021-10-26 19:44:56 +02:00
Lehel
d7e41e2005
NIFI-9328: Transfer cleanup and reuse added to FetchFileTransfer in case of FileNotFound and PermissionDenied exceptions.
- Corrects connection handling in FetchFTP and FetchSFTP

This closes #5478

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-10-26 11:17:33 -05:00
Mark Payne
1191e511a5
NIFI-9335: Updated AvroTypeUtil#createAvroRecord to ensure that if the given Avro Schema contains a field whose value is defaulted, the produced Avro Record has that value populated. Also added a unit test to verify behavior. Ran performance test (which is igonred and must be manually enabled) - about 2.1 seconds before the change and about 2.5 seconds after the change for each iteration.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5483
2021-10-26 12:15:54 -04:00
omkadmi
8506a6012f
NIFI-9337: added missing enum value SINGLE_BATCH_PER_NODE of FlowFileConcurrencyType (#5484) 2021-10-26 11:25:52 -04:00
exceptionfactory
7b2f364cf5
NIFI-9331 Corrected ThreadUtils using Objects.equals() for LockedStackFrame
- Removed unused arguments from ThreadUtils.createStackTrace()

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5480.
2021-10-26 09:56:28 +02:00
Nathan Gough
309ab8f4ab
NIFI-8792 - Modified ListenRELP to use Netty
- Refactored RELP encoders and decoders

This closes #5398

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-10-25 15:53:58 -05:00
Pierre Villard
8fe7f372d6
NIFI-9329 - Expose event validation in ParseCEF processor (#5477) 2021-10-25 14:37:39 +02:00
markap14
5766d33fce
NIFI-9309: Include a uiOnly flag when requesting flow for a given pro… (#5462)
NIFI-9309: Include a uiOnly flag when requesting flow for a given process group. In that case, do not include the property descriptors, property values, etc. for Processors. When fetching Variable Registry, improved logic to cache the VariableImpact for each property value instead of parsing/recreating it every time. When fetching bulletins for a component, avoid filtering through all components' bulletins and instead only look at bulletins that might pertain to the appropriate component
2021-10-23 19:42:31 -04:00
Matthew Burgess
f7e4e52caf
NIFI-9318: Downgrade Cassandra driver to 3.10.2 for max compatibility
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5473.
2021-10-22 09:52:17 +02:00
Joe Gresock
207da405ee NIFI-9263: Using secret-per-context model for Stateless Vault PVP
Rebasing after PVP refactor

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5427.
2021-10-21 16:43:53 -04:00
Joe Gresock
d2f8f97b10
NIFI-9276: Adding config verification to AbstractListProcessor subclasses (#5453) 2021-10-21 12:22:50 -04:00
Matt Gilman
1bec905890 NIFI-9288:
- Allowing the user to submit a verification request for Processors, Controller Services, and Reporting Tasks.
- Tracking progress of verification requests.
- Showing the verification results.

NIFI-9288:
- Fixing class name which prevented styles from being applied.

NIFI-9288:
- Ensuring that previously entered referenced attribute values take precedence.

NIFI-9288:
- Positioning the property listing and verification results based on percentages instead of fixed values.
- Removing the additional dialog height.

NIFI-9288:
- Allowing attribute value entry to be skipped when appropriate.

NIFI-9288:
- Working around an issue caused by css minification.

NIFI-9288:
- Adding some padding to the verifying progress dialog.

This closes #5461

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
2021-10-21 10:06:58 -04:00
Peter Turcsanyi
1e2783ba63
NIFI-9305: Upgraded Azure dependencies
Also fixed ITPutAzureDataLakeStorage (was broken due to a former dependency upgrade).

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5472.
2021-10-21 10:55:54 +02:00
Matt Gilman
51c6910510 NIFI-9237:
- Similar to NIFI-9215, converting integer identifiers to strings to ensure the items are successfully retrieved when attempting to apply a tooltip.

This closes #5454

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
2021-10-20 16:47:21 -04:00
markap14
90ae271692
NIFI-9310: Addressed issues found while testing the component verification features. Added a supportsVerification flag to the ConfigAnalysis DTO (#5469)
This closes #5469
2021-10-20 11:07:17 -04:00
Bryan Bende
a380fb2964
NIFI-9297 Correct typo in documentation for listing strategy in AbstractListProcessor
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5466.
2021-10-19 17:25:17 +02:00
Bryan Bende
4bcfc61563
NIFI-9011 Correct tags in publish Kafka 2.6 processors
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5464.
2021-10-19 16:48:15 +02:00
exceptionfactory
da1b1dfd4b
NIFI-7322 Added SignContentPGP and VerifyContentPGP
- Added Decryption Strategy property to DecryptContentPGP
- Added OpenPGP Packet detection to EncryptContentPGP to avoid unnecessary packaging
- Refactored shared processing to EncodingStreamCallback

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5457.
2021-10-19 06:47:21 -04:00
exceptionfactory
70274ebcfc
NIFI-9306 Upgraded Spring Framework to 5.3.11
- Upgraded Spring Security from 5.5.1 to 5.5.2
- Upgraded Spring Boot from 2.5.2 to 2.5.5 in Registry
- Upgraded Spring Data Redis from 2.5.3 to 2.5.5 in Redis modules

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5463.
2021-10-19 11:26:37 +02:00
Mark Payne
be2b80d909
NIFI-9292: Added getInputPortNames() and getOutputPortNames() to StatelessDataflowDefinition and getReadOnlyExtensionsDirectories() to StatelessEngineConfiguration
- Addressed issue identified in system test where a flow can fail due to FailurePortEncounteredException but then purge the flow in the background after the call to trigger completes

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5450.
2021-10-18 12:22:12 -04:00
Denes Arvay
d722b820a1
NIFI-9248 Use hive-exec:core instead of the "regular" hive-exec dependency
- Excluded org.apache.hive:hive-exec and added the "core" classifier version
- Added the removed dependencies explicitly

NIFI-9248 Add additional log4j excludes

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5414
2021-10-18 11:55:43 -04:00
Mark Payne
151a9361fa
NIFI-9288: Updated data model to use for cnofig verification requests so that only properties are necessary not the full component/config dtos. Also added endpoint necessary for determining which attributes are referenced by a component's properties and removed the referencedAttributes field from the compnoents themselves, since there's now a new endpoint for it. Also fixed a bug that was encountered where the VerifyConfigRequestDTO's complete flag was incorrect in case of failures
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5452.
2021-10-18 11:42:47 -04:00
Lehel
207894ebe0 NIFI-9277: Add Record Reader and Writer to ListenHTTP.
This closes #5446.

Signed-off-by: Tamas Palfy <tamas.bertalan.palfy@gmail.com>
2021-10-15 18:27:53 +02:00
exceptionfactory
0147b15e37
NIFI-9303 Upgraded SSHJ from 0.31 to 0.32
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5459.
2021-10-15 11:36:45 +02:00
Lehel Boér
32c7d856c8
NIFI-9300: Fix AWSCredentialsService EL attribute evaluation. (#5456) 2021-10-14 10:46:15 -04:00