Commit Graph

7176 Commits

Author SHA1 Message Date
Mark Payne 73356ea448
NIFI-9783: This closes #5855. When migrating FlowFiles from one ProcessSession to another, if any FlowFile had already been transferred, and the Relationship to which it was transferred was auto-terminated, we were updating the wrong member variable, which threw off our stats for the processor. Fixed that.
Signed-off-by: Joe Witt <joewitt@apache.org>
2022-03-09 13:26:32 -07:00
exceptionfactory c73573b325
NIFI-9761 Correct PeerChannel processing for TLS 1.3 (#5836)
* NIFI-9761 Corrected PeerChannel processing for TLS 1.3
- Added TestPeerChannel with methods for TLS 1.2 and TLS 1.3
- Updated PeerChannel.close() to process SSLEngine close notification
- Improved logging and corrected handling after decryption
2022-03-09 15:15:52 -05:00
Mark Payne 6a1c7c72d5
NIFI-9781: Fix handling when selecting array element via QueryRecord
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5853.
2022-03-09 13:07:15 -05:00
Peter Turcsanyi 3219c105eb
NIFI-9728: Added support for User Assigned Managed Identity authentication for Azure ADLS and Blob_v12 processors
Also bumped Azure dependencies.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5846.
2022-03-09 16:50:38 +01:00
Bence Simon b86466b4a5 NIFI-9669 Adding PutDynamoDBRecord processor
This closes #5761.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2022-03-09 16:20:33 +01:00
exceptionfactory c8f8747dbc
NIFI-9774 Upgraded Netty from 4.1.73 to 4.1.74
This closes #5851

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-03-09 08:51:00 -05:00
Bence Simon 3b8afee9d6
NIFI-9778 Fixing additional details for ScriptedPartitionRecord
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5852.
2022-03-09 14:49:28 +01:00
Matt Gilman 227dcf5456
NIFI-9777: (#5850) - Adding support to remove attributes from verification requests.
This closes #5850
2022-03-08 16:25:32 -05:00
Mark Payne e6229ab938
NIFI-9771: When a Kafka record is obtained during config verification, we should produce an invalid response if the Record Reader is not able to produce any records from it
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5847.
2022-03-08 06:42:27 -05:00
Mike Thomsen f35f010deb
NIFI-9085 Refactored the Elasticsearch bundle to use JUnit 5
- NIFI-9086 Refactored nifi-accumulo-bundle to use JUnit 5
- NIFI-9087 Updated nifi-ambari-bundle to use JUnit 5
- NIFI-9088 Refactored nifi-amqp-bundle to use JUnit 5
- NIFI-9089 Updated nifi-asn1-bundle to use JUnit 5
- NIFI-9090 Refactored nifi-atlas-bundle to use JUnit 5
- NIFI-9091 Refactored nifi-avro-bundle to use JUnit 5
- NIFI-9092 Refactored nifi-aws-bundle to use JUnit 5
- NIFI-9095 Refactored nifi-azure-bundle to use JUnit 5
- NIFI-9145 Refactored nifi-rethinkdb-bundle to use JUnit 5
- NIFI-9143 Refactored nifi-redis-bundle to use JUnit 5

This closes #5350

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-03-07 21:49:23 -06:00
Matthew Burgess 43bf2b5975
NIFI-9763: Additional escaped VALUE column due to H2 changes
This closes #5848

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-03-07 19:18:02 -06:00
Joe Gresock 743020eeb4
NIFI-9762: Adding DBCPConnectionPool config verification
Relaxing MockPropertyValue validation to allow for variables to be passed to config verification

Fixing underlying framework issue with config verification: wrong variable registry was being used

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5843
2022-03-07 14:43:11 -05:00
Paul Grey fc5c810de7
NIFI-9750 - Logging Improvements to support LoadBalanceProtocol troubleshooting
This closes #5828

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-03-07 13:41:07 -06:00
Eduardo Fontes d062029258
NIFI-8927 - Add option to start/stop all controllers (#5247)
* Update nf-context-menu.js

Include enable/disable all controllers menu item

* Update nf-actions.js

Create enableAllControllers and disableAllControllers actions

* Rename ...Controllers to ...ControllerServices

* Rename *controllers to *controller services

* Adjusts proposed by reviewer.

* Refactor enable and disable AllControllerServices

This closes #5247
2022-03-07 13:57:35 -05:00
exceptionfactory 7d046fe3de
NIFI-9763 Escaped VALUE column for Configure Details Auditing
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5838
2022-03-07 08:53:08 -05:00
Andrew Lim 9782a51564
NIFI-9756 Add documentation for framework-level retry in Processors and update processor tab images
This closes #5837

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-03-06 08:27:06 -05:00
Chris Sampson 12685a4af4
NIFI-9766 avoid intermittent SearchElasticsearchTest failures in CI pipeline
This closes #5842

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-03-05 19:05:02 -05:00
Mark Payne 65dd62716a
NIFI-9748: Added new property for Output Format to LogAttribute. Also made the FlowFile Properties (file size, entry date, lineage start date) optional and renamed from 'Standard FlowFile Attributes' to 'FlowFile Properties' because this has led to confusion many times in the past, around users wanting to reference these things as attributes via EL but they are not actually attributes.
This closes #5825

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-03-04 08:24:39 -05:00
Mark Payne 8959226b50
NIFI-9754: Introduced VersionedExternalFlow
- Updated stateless and StandardProcessGroup, etc. to make use of VersionedExternalFlow
- Updated StatelessDataflowDefinition to use ExternalVersionedFlow instead of generic type
- Updated Stateless Bootstrap to avoid loading stateless engine libs from root class path but instead use a NarClassLoader to load the statelss nar

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5832.
2022-03-04 07:19:38 -05:00
exceptionfactory 7438bc9244
NIFI-9759 Upgraded Spring Framework from 5.3.15 to 5.3.16
- Upgraded Spring Security from 5.6.1 to 5.6.2
- Upgraded Spring Boot from 2.6.3 to 2.6.4 in NiFi Registry
- Upgraded Spring Data Redis from 2.6.0 to 2.6.2
- Upgraded Spring Integration from 5.5.7 to 5.5.9

This closes #5835

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-03-04 06:23:51 -05:00
exceptionfactory 923e9f4e4f
NIFI-9757 Upgraded SLF4J from 1.7.35 to 1.7.36
This closes #5834

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-03-04 06:18:09 -05:00
Shane Ardell 6cd30e3d0a
NIFI-9543: Add bring-to-front functionality to labels (#5765)
* NIFI-9543: add bring-to-front functionality to labels

* NIFI-9543: update zIndex changes to persist after shutdown

* NIFI-9543: remove commented code

* NIFI-9543: update flow comparator and snippet to account for zIndex

* NIFI-9543: changed add function to sort labels when a new label is added

* NIFI-9543: remove unnecessary call to position

* NIFI-9543: only register difference if zIndex is not default value

* NIFI-9543: add null checks to prevent primitive Long

This closes #5765
2022-03-03 18:50:15 -05:00
Adam Kocsis 6f5fda54e5
[NIFI-9746] NiFi Registy Bucket Side Nav no longer fails to open (#5831)
- all ViewCild queries are decorated with {static: true} which is needed because of new angular version
none of these ViewChild under *ng directives so they are available during init with static:true

Merged #5831 into main.
2022-03-03 14:45:35 -08:00
Matthew Burgess c91f543c2b NIFI-9425: Added auto-load NAR capability to MiNiFi
NIFI-9425: Added autoload directory property and defensive code for stop() called twice

This closes #5762.

Signed-off-by: Kevin Doran <kdoran@apache.org>
2022-03-03 16:30:47 -05:00
Paul Grey d2e90dd160
NIFI-9751 - Poll as needed during system-tests to ensure expected state
This closes #5829

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-03-03 15:24:52 -06:00
Adam Kocsis 7a16619969
[NIFI-9690] Display icon on Connection Label to show when head of the queue is penalized (#5812)
This closes #5812
2022-03-03 09:13:00 -05:00
Paul Grey 6bc85155f9
NIFI-9749 - Capture additional logging for system-test workflow runs
This closes #5827

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-03-02 18:52:43 -06:00
Paul Grey 272f412254
NIFI-9747 - Track PID in nifi-bootstrap logging on shutdown
This closes #5826

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-03-02 18:49:34 -06:00
Bryan Bende db7690769d
NIFI-9745 - Prevent insertion of revisions in NiFi registry when revision feature is disabled
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5823.
2022-03-02 19:12:32 -05:00
exceptionfactory fee7c16732
NIFI-9679 Added access-environment-credentials permission
- Applied new permission restrictions to AWSCredentialsProviderControllerService and GCPCredentialsControllerService

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5796.
2022-03-02 12:31:09 -05:00
zhangcheng a85cafe771
NIFI-9064:Support Oracle timestamp when `Use Avro Logical Types` is true for ExecuteSQLRecord and QueryDatabaseTableRecord
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5807.
2022-03-02 11:18:58 -05:00
Chris Sampson 4cc20e6c06
NIFI-9678 update Elasticsearch REST API processor integration-tests for Elasticsearch 8.x
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5822.
2022-03-02 10:04:34 -05:00
exceptionfactory cf7af1b463
NIFI-9743 Upgraded Jetty from 9.4.44 to 9.4.45
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5819.
2022-03-02 12:48:15 +01:00
Mark Payne 53a35ae4c9
NIFI-9689: When checking FlowFile Availability, consider swap queue and trigger data to be swapped in, since calling poll() will no longer happen if no data is available
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5821.
2022-03-02 06:22:37 -05:00
Mark Payne a327ba478a
NIFI-9741: Make the close() method of WriteAvroResultWithExternalSchema idempotent
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5818
2022-03-01 17:52:02 -05:00
exceptionfactory d17fcf6427 NIFI-9735 Corrected Jetty Duplicate Mapping Warning
- Removed duplicate default servlet mapping from addDocsServlets()

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5817.
2022-03-01 15:49:23 -05:00
Shane Ardell 4719f78ef4
NIFI-9580: UI work for framework-level retry in Processors (#5666)
* NIFI-9580: initial ui work for framework-level retry in processors

* NIFI-9580: disable retry controls when retry not selected

* NIFI-9580: refactor marshalling to be one function

* NIFI-9580: style tweaks and documentation update

* NIFI-9580: update tooltip content

* NIFI-9580: change tooltip copy for clarity

* NIFI-9580: only submit retry configs if selected

* NIFI-9580: add relationship details to read-only dialog

* NIFI-9580: change styling based on feedback

This closes #5666
2022-03-01 14:57:18 -05:00
Mark Payne 0d0912d34b
NIFI-9400: Ensure that we always use the CollectionUsage metrics in the mbeans instead of the Usage metrics.
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5798.
2022-03-01 14:17:32 -05:00
Mark Payne 9d3788ff05
NIFI-9730: Consider a change in value for retry-related fields from 'null' to the default value as an environmental change so that it's not flagged as a Local Modification, which would prevent users from updating the version of the Process Group that they are using
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5809.
2022-03-01 13:49:47 -05:00
Joe Gresock db28c91cdb
NIFI-9733: Fixing StandardConnection.verifySourceStoppedOrFunnel infinite recursion (#5815) 2022-03-01 09:57:51 -05:00
Nathan Gough dd3c9be847
NIFI-9641 - Adjusted the extraction of the chroot suffix for solr client connections.
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5727.
2022-03-01 06:51:16 -05:00
exceptionfactory 9bb34188c9
NIFI-9734 Standardized exception cause message formatting
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5813.
2022-03-01 06:49:34 -05:00
Paul Grey 875f589548
NIFI-9738 - VersionedComponent data members should derive from Object
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5816.
2022-03-01 06:06:48 -05:00
exceptionfactory 52f21390db
NIFI-9688 Improve Logback shutdown handling (#5805)
* NIFI-9688 Improved Logback shutdown handling
- Added environment variable to disable Logback shutdown on web application termination
- Added shutdown hook to Logback configuration
- Added Logback shutdown hook configuration to system tests
2022-02-28 14:29:44 -05:00
exceptionfactory ddecfa201b
NIFI-9736 Improved TestRouteText to avoid intermittent failures (#5814)
- Refactored testPatternCache to avoid testing maximum cache size reached
2022-02-28 14:26:19 -05:00
Mark Payne 649e2d2556
NIFI-9731: Updated to use a shorter, simpler output format for FlowFiles when creating bulletins
NIFI-9731: Only simplify flowfile arg in bulletin after extracting the flowfile uuid
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5808
2022-02-28 13:21:10 -05:00
Mike Thomsen 589a1c24d1
NIFI-9124 Refactored nifi-jms-bundle to use JUnit 5
- NIFI-9122 Refactored nifi-influxdb-bundle to use JUnit 5
- NIFI-9121 Refactored nifi-ignite-bundle to use JUnit 5
- NIFI-9120 Refactored nifi-html-bundle to use JUnit 5
- NIFI-9119 Refactored nifi-hl7-bundle to use JUnit 5
- NIFI-9118 Refactored nifi-hive-bundle to use JUnit 5
- NIFI-9117 Refactored nifi-hbase-bundle to use JUnit 5
- NIFI-9116 Refactored nifi-hazelcast-bundle to use JUnit 5
- NIFI-9114 Refactored nifi-hadoop-bundle to use JUnit 5
- Added DisabledOnOs(OS.WINDOWS) to Hive tests with @TempDir from JUnit 5 due to Windows-specific problems in JUnit 5

This closes #5791

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-02-28 10:39:47 -06:00
Mark Payne 483968bac3
NIFI-6390, NIFI-1825: When we write to a FlowFile and that results in a 0-byte FlowFile, remove the content claim all together. This is more efficient to process, but far more importantly it prevents a 0-byte FlowFile from holding content in the Content Repository. Also fixed issue in which a Provenance Event cannot be replayed if there is no ContentClaim.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5810
2022-02-28 10:24:32 -05:00
Mike Thomsen 4b41570d17
NIFI-9714 Added overloaded toMap to MapRecord that can convert sub-records into maps.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5784
2022-02-28 09:12:51 -05:00
exceptionfactory 1eede30974
NIFI-9732 Upgraded Zip4j from 2.8.0 to 2.9.1
This closes #5811

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-02-27 06:09:14 -05:00