Apache
/
nifi
mirror of https://github.com/apache/nifi.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
7,251 Commits 146 Branches 201 Tags 406 MiB
Commit Graph

7251 Commits

Author SHA1 Message Date
Adam Kocsis 7a16619969
[NIFI-9690] Display icon on Connection Label to show when head of the queue is penalized (#5812) 
This closes #5812
 2022-03-03 09:13:00 -05:00
Paul Grey 6bc85155f9
NIFI-9749 - Capture additional logging for system-test workflow runs 
This closes #5827

Signed-off-by: David Handermann <exceptionfactory@apache.org>
 2022-03-02 18:52:43 -06:00
Paul Grey 272f412254
NIFI-9747 - Track PID in nifi-bootstrap logging on shutdown 
This closes #5826

Signed-off-by: David Handermann <exceptionfactory@apache.org>
 2022-03-02 18:49:34 -06:00
Bryan Bende db7690769d
NIFI-9745 - Prevent insertion of revisions in NiFi registry when revision feature is disabled 
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5823.
 2022-03-02 19:12:32 -05:00
exceptionfactory fee7c16732
NIFI-9679 Added access-environment-credentials permission 
- Applied new permission restrictions to AWSCredentialsProviderControllerService and GCPCredentialsControllerService

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5796.
 2022-03-02 12:31:09 -05:00
zhangcheng a85cafe771
NIFI-9064:Support Oracle timestamp when `Use Avro Logical Types` is true for ExecuteSQLRecord and QueryDatabaseTableRecord 
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5807.
 2022-03-02 11:18:58 -05:00
Chris Sampson 4cc20e6c06
NIFI-9678 update Elasticsearch REST API processor integration-tests for Elasticsearch 8.x 
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5822.
 2022-03-02 10:04:34 -05:00
exceptionfactory cf7af1b463
NIFI-9743 Upgraded Jetty from 9.4.44 to 9.4.45 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5819.
 2022-03-02 12:48:15 +01:00
Mark Payne 53a35ae4c9
NIFI-9689: When checking FlowFile Availability, consider swap queue and trigger data to be swapped in, since calling poll() will no longer happen if no data is available 
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5821.
 2022-03-02 06:22:37 -05:00
Mark Payne a327ba478a
NIFI-9741: Make the close() method of WriteAvroResultWithExternalSchema idempotent 
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5818
 2022-03-01 17:52:02 -05:00
exceptionfactory d17fcf6427 NIFI-9735 Corrected Jetty Duplicate Mapping Warning 
- Removed duplicate default servlet mapping from addDocsServlets()

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5817.
 2022-03-01 15:49:23 -05:00
Shane Ardell 4719f78ef4
NIFI-9580: UI work for framework-level retry in Processors (#5666) 
* NIFI-9580: initial ui work for framework-level retry in processors

* NIFI-9580: disable retry controls when retry not selected

* NIFI-9580: refactor marshalling to be one function

* NIFI-9580: style tweaks and documentation update

* NIFI-9580: update tooltip content

* NIFI-9580: change tooltip copy for clarity

* NIFI-9580: only submit retry configs if selected

* NIFI-9580: add relationship details to read-only dialog

* NIFI-9580: change styling based on feedback

This closes #5666
 2022-03-01 14:57:18 -05:00
Mark Payne 0d0912d34b
NIFI-9400: Ensure that we always use the CollectionUsage metrics in the mbeans instead of the Usage metrics. 
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5798.
 2022-03-01 14:17:32 -05:00
Mark Payne 9d3788ff05
NIFI-9730: Consider a change in value for retry-related fields from 'null' to the default value as an environmental change so that it's not flagged as a Local Modification, which would prevent users from updating the version of the Process Group that they are using 
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5809.
 2022-03-01 13:49:47 -05:00
Joe Gresock db28c91cdb
NIFI-9733: Fixing StandardConnection.verifySourceStoppedOrFunnel infinite recursion (#5815) 2022-03-01 09:57:51 -05:00
Nathan Gough dd3c9be847
NIFI-9641 - Adjusted the extraction of the chroot suffix for solr client connections. 
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5727.
 2022-03-01 06:51:16 -05:00
exceptionfactory 9bb34188c9
NIFI-9734 Standardized exception cause message formatting 
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5813.
 2022-03-01 06:49:34 -05:00
Paul Grey 875f589548
NIFI-9738 - VersionedComponent data members should derive from Object 
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5816.
 2022-03-01 06:06:48 -05:00
exceptionfactory 52f21390db
NIFI-9688 Improve Logback shutdown handling (#5805) 
* NIFI-9688 Improved Logback shutdown handling
- Added environment variable to disable Logback shutdown on web application termination
- Added shutdown hook to Logback configuration
- Added Logback shutdown hook configuration to system tests
 2022-02-28 14:29:44 -05:00
exceptionfactory ddecfa201b
NIFI-9736 Improved TestRouteText to avoid intermittent failures (#5814) 
- Refactored testPatternCache to avoid testing maximum cache size reached
 2022-02-28 14:26:19 -05:00
Mark Payne 649e2d2556
NIFI-9731: Updated to use a shorter, simpler output format for FlowFiles when creating bulletins 
NIFI-9731: Only simplify flowfile arg in bulletin after extracting the flowfile uuid
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5808
 2022-02-28 13:21:10 -05:00
Mike Thomsen 589a1c24d1
NIFI-9124 Refactored nifi-jms-bundle to use JUnit 5 
- NIFI-9122 Refactored nifi-influxdb-bundle to use JUnit 5
- NIFI-9121 Refactored nifi-ignite-bundle to use JUnit 5
- NIFI-9120 Refactored nifi-html-bundle to use JUnit 5
- NIFI-9119 Refactored nifi-hl7-bundle to use JUnit 5
- NIFI-9118 Refactored nifi-hive-bundle to use JUnit 5
- NIFI-9117 Refactored nifi-hbase-bundle to use JUnit 5
- NIFI-9116 Refactored nifi-hazelcast-bundle to use JUnit 5
- NIFI-9114 Refactored nifi-hadoop-bundle to use JUnit 5
- Added DisabledOnOs(OS.WINDOWS) to Hive tests with @TempDir from JUnit 5 due to Windows-specific problems in JUnit 5

This closes #5791

Signed-off-by: David Handermann <exceptionfactory@apache.org>
 2022-02-28 10:39:47 -06:00
Mark Payne 483968bac3
NIFI-6390, NIFI-1825: When we write to a FlowFile and that results in a 0-byte FlowFile, remove the content claim all together. This is more efficient to process, but far more importantly it prevents a 0-byte FlowFile from holding content in the Content Repository. Also fixed issue in which a Provenance Event cannot be replayed if there is no ContentClaim. 
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5810
 2022-02-28 10:24:32 -05:00
Mike Thomsen 4b41570d17
NIFI-9714 Added overloaded toMap to MapRecord that can convert sub-records into maps. 
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5784
 2022-02-28 09:12:51 -05:00
exceptionfactory 1eede30974
NIFI-9732 Upgraded Zip4j from 2.8.0 to 2.9.1 
This closes #5811

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
 2022-02-27 06:09:14 -05:00
Mike Thomsen ae1731c79e
NIFI-9134 nifi-metrics-reporting-bundle to use JUnit 5 
- NIFI-9129 Refactored as much of nifi-kudu-bundle to use JUnit 5 as possible without breaking the integration tests which use test libraries from the Kudu project that depend on JUnit 4
- NIFI-9109 Refactored nifi-extension-utils to use JUnit 5. Left two tests with JUnit 4 because they used APIs that don't appear to have analogues in JUnit 5
- NIFI-9112 Refactored nifi-groovyx-bundle to use JUnit 5
- NIFI-9142 Refactored nifi-ranger-bundle to use JUnit 5
- NIFI-9136 Refactored nifi-network-bundle to use JUnit 5

This closes #5794

Signed-off-by: David Handermann <exceptionfactory@apache.org>
 2022-02-26 08:18:08 -06:00
Matthew Burgess bcc8d03314
NIFI-9585 Upgraded H2 from 1.4 to 2.1.210 
- Added nifi-h2-database module shading H2 1.4.200
- Implemented version checking and migration

This closes #5724

Signed-off-by: David Handermann <exceptionfactory@apache.org>
 2022-02-25 19:20:48 -06:00
markobean 4e3871fec7
NIFI-9072: improvements to ValidateXML including validate XML in attr… (#5324) 
NIFI-9072: Improvements to ValidateXML including validate XML in attributes and non-schema, well-formed XML only validation, and additional documentation
- Updated description and added System Resource Consideration to discourage using large XML documents in attributes
- Migrate from XML DocumentBuilderFactory to SafeXMLConfiguration
 2022-02-25 14:57:04 -05:00
Mark Payne 6cea5ea520
NIFI-9729: When restarting components in the VersionedFlowSynchronizer, first filter out any components that are intended to be stopped. 
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5806.
 2022-02-25 14:29:38 -05:00
Paul Grey e74991e705
NIFI-9727 - IndexOutOfBoundsException in CorrelationAttributePartitioner 
This closes #5804

Signed-off-by: David Handermann <exceptionfactory@apache.org>
 2022-02-25 11:42:02 -05:00
exceptionfactory 49d1c747ca
NIFI-9724 Added set-sensitive-properties-algorithm command 
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5801.
 2022-02-25 11:03:55 -05:00
exceptionfactory a6aba3bf8e
NIFI-9726 Removed duplicate nifi-utils dependencies from graph modules 
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5803
 2022-02-25 10:40:24 -05:00
Mark Payne 432f98714f
NIFI-9725: On shutdown, instead of spawning a background thread to shutdown Cluster Coordinator, do so in the calling thread. This avoids a race condition whereby the cluster coordinator cannot be determined because the other thread has shutdown the FlowController. 
This closes #5800

Signed-off-by: David Handermann <exceptionfactory@apache.org>
 2022-02-24 22:16:01 -06:00
Mike Thomsen 13be48cfe3
NIFI-9113 Refactored nifi-grpc-bundle to use JUnit 5 
- NIFI-9111 Refactored nifi-gcp-bundle to use JUnit 5
- NIFI-9110 Refactored nifi-flume-bundle to use JUnit 5
- NIFI-9108 Refactored nifi-evtx-bundle to use JUnit 5
- NIFI-9106 Refactored nifi-email-bundle to use JUnit 5
- NIFI-9105 Refactored nifi-easyrules-bundle to use JUnit 5
- NIFI-9104 Refactored nifi-druid-bundle to use JUnit 5

This closes #5790

Signed-off-by: David Handermann <exceptionfactory@apache.org>
 2022-02-24 16:59:44 -06:00
Mark Payne c453d2ba8f
NIFI-9723: When we add controller-level Controller Services on restart of NiFi, ensure that all Controller Services are updated to include their property values, etc. Also ensure that for these services and reporting tasks we decrypt the property values. 
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5797.
 2022-02-24 15:13:14 -05:00
Hsin-Ying Lee c331f9cb9a
NIFI-9227 Run Once not working when scheduling strategy is CRON or Event driven (#5445) 2022-02-24 12:27:01 -05:00
Mark Payne 39483b9c12
NIFI-9716, NIFI-9577: Addressed issue in the PathFilter for GetFile / ListFile. For any file that is found in the Input Directory directly, it was previously being listed/fetched even if it didn't match the PathFilter. Additionally, updated the code to create a new File FIlter for every invocation of onTrigger. This was necessary for NIFI-9577 because the directory to monitor supports Expression Language and as a result may change from invocation to invocation, if using a function such as now() but the PathFilter would always relativize the path based on the value that was obtained when the processor was scheduled. 
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5788.
 2022-02-24 09:13:16 -05:00
Mark Payne 9237b4e923
NIFI-9722: Do not throw an Exception from verifyCanUpdateProperties when property descriptor & parameter descriptor's sensitivities don't match - instead allow the set to happen and let processor become invalid. Also, allow values such as abc#{param} for ghost processors. 
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5795.
 2022-02-24 08:58:54 -05:00
Mike Thomsen 51cf0fb0d1
NIFI-9133 Refactored nifi-media-bundle to use JUnit 5 
- NIFI-9130 Refactored nifi-language-translation-bundle to use JUnit 5
- NIFI-9126 Refactored nifi-kafka-bundle to use JUnit 5
- NIFI-9125 Refactored nifi-jolt-record-bundle to use JUnit 5

This closes #5792

Signed-off-by: David Handermann <exceptionfactory@apache.org>
 2022-02-23 21:18:25 -06:00
Mike Thomsen 8b62ebeb76
NIFI-9103 Refactored nifi-datadog-bundle to use JUnit 5 
- NIFI-9102 Refactored nifi-cybersecurity-bundle to use JUnit 5
- NIFI-9101 Refactored nifi-couchbase-bundle to use JUnit 5
- NIFI-9100 Refactored nifi-confluent-platform-bundle to use JUnit 5
- NIFI-9099 Refactored nifi-cdc to use JUnit 5
- NIFI-9098 Refactored nifi-ccda-bundle to use JUnit 5
- NIFI-9097 Refactored nifi-cassandra-bundle to use JUnit 5
- NIFI-9096 Refactored nifi-beats-bundle to use JUnit 5

This closes #5789

Signed-off-by: David Handermann <exceptionfactory@apache.org>
 2022-02-23 21:03:43 -06:00
Matthew Burgess ab4cadc204
NIFI-9721: Support enum types in AvroTypeUtil.buildAvroSchema() 
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5793.
 2022-02-23 12:44:57 -05:00
Mark Payne 3511ce3d13
NIFI-9548: When disabling RPG transmission, wait for the ports to complete in a background thread instead of blocking the web thread. Also moved the RPG initialization logic into flow controller instead of flow service and added a delay in order to reduce likelihood of ConnectException happening when pointing to nodes in the same cluster 
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5641.
 2022-02-23 11:29:26 -05:00
Timea Barna a3e1f32cae NIFI-9657 Create MoveADLS processor 
This closes #5752.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
 2022-02-23 16:09:31 +01:00
Mark Payne ce0122bd25
NIFI-9691: Added ForkEnrichment, JoinEnrichment processors 
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5772.
 2022-02-22 18:54:32 -05:00
Joe Gresock dc1117d104
NIFI-9707: Resolved duplicate JLine dependency in nifi-toolkit 
- Excluded groovy-groovysh from several toolkit modules

This closes #5782

Signed-off-by: David Handermann <exceptionfactory@apache.org>
 2022-02-22 14:55:43 -06:00
exceptionfactory 45f8795177
NIFI-9711 Added support for flow.json.gz in SetSensitivePropertiesKey 
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5783.
 2022-02-22 12:58:30 -05:00
Mark Payne d0a23bc26b
NIFI-9704: Updated the ContentRepositoryScanTask to show details of how much content in the content repo is retained by each queue in the dataflow. Changed default for nifi.content.claim.max.appendable.size property from 1 MB to 50 KB. Updated docs to reflect the new default value and explain what the property does and how it's used. 
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5780.
 2022-02-22 12:06:54 -05:00
Chris Sampson 04d061a7bd
NIFI-9581 add PutElasticsearchRecord relationship for output of successful Records sent to Elasticsearch 
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5665.
 2022-02-22 11:42:01 -05:00
xinyuwu4 22b47ccc7b
NIFI-9672: fix flaky tests caused by the use of HashMap. 
This closes #5758

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
 2022-02-22 08:06:52 -05:00
lawye1973 0ade040b76
NIFI-9696 : DeleteS3Object don't have provenance 
Changed from AmazonS3 to AmazonS3client.
Add REMOTE_INVOCATION provenance event with URL and "Object deleted"

This closes #5778

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
 2022-02-22 08:01:59 -05:00