2128 Commits

Author SHA1 Message Date
Mark Payne
53a35ae4c9
NIFI-9689: When checking FlowFile Availability, consider swap queue and trigger data to be swapped in, since calling poll() will no longer happen if no data is available
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5821.
2022-03-02 06:22:37 -05:00
exceptionfactory
d17fcf6427 NIFI-9735 Corrected Jetty Duplicate Mapping Warning
- Removed duplicate default servlet mapping from addDocsServlets()

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5817.
2022-03-01 15:49:23 -05:00
Shane Ardell
4719f78ef4
NIFI-9580: UI work for framework-level retry in Processors (#5666)
* NIFI-9580: initial ui work for framework-level retry in processors

* NIFI-9580: disable retry controls when retry not selected

* NIFI-9580: refactor marshalling to be one function

* NIFI-9580: style tweaks and documentation update

* NIFI-9580: update tooltip content

* NIFI-9580: change tooltip copy for clarity

* NIFI-9580: only submit retry configs if selected

* NIFI-9580: add relationship details to read-only dialog

* NIFI-9580: change styling based on feedback

This closes #5666
2022-03-01 14:57:18 -05:00
Mark Payne
9d3788ff05
NIFI-9730: Consider a change in value for retry-related fields from 'null' to the default value as an environmental change so that it's not flagged as a Local Modification, which would prevent users from updating the version of the Process Group that they are using
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5809.
2022-03-01 13:49:47 -05:00
Joe Gresock
db28c91cdb
NIFI-9733: Fixing StandardConnection.verifySourceStoppedOrFunnel infinite recursion (#5815) 2022-03-01 09:57:51 -05:00
exceptionfactory
9bb34188c9
NIFI-9734 Standardized exception cause message formatting
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5813.
2022-03-01 06:49:34 -05:00
Paul Grey
875f589548
NIFI-9738 - VersionedComponent data members should derive from Object
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5816.
2022-03-01 06:06:48 -05:00
exceptionfactory
52f21390db
NIFI-9688 Improve Logback shutdown handling (#5805)
* NIFI-9688 Improved Logback shutdown handling
- Added environment variable to disable Logback shutdown on web application termination
- Added shutdown hook to Logback configuration
- Added Logback shutdown hook configuration to system tests
2022-02-28 14:29:44 -05:00
Mark Payne
649e2d2556
NIFI-9731: Updated to use a shorter, simpler output format for FlowFiles when creating bulletins
NIFI-9731: Only simplify flowfile arg in bulletin after extracting the flowfile uuid
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5808
2022-02-28 13:21:10 -05:00
Mark Payne
483968bac3
NIFI-6390, NIFI-1825: When we write to a FlowFile and that results in a 0-byte FlowFile, remove the content claim all together. This is more efficient to process, but far more importantly it prevents a 0-byte FlowFile from holding content in the Content Repository. Also fixed issue in which a Provenance Event cannot be replayed if there is no ContentClaim.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5810
2022-02-28 10:24:32 -05:00
Matthew Burgess
bcc8d03314
NIFI-9585 Upgraded H2 from 1.4 to 2.1.210
- Added nifi-h2-database module shading H2 1.4.200
- Implemented version checking and migration

This closes #5724

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-02-25 19:20:48 -06:00
Mark Payne
6cea5ea520
NIFI-9729: When restarting components in the VersionedFlowSynchronizer, first filter out any components that are intended to be stopped.
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5806.
2022-02-25 14:29:38 -05:00
Paul Grey
e74991e705
NIFI-9727 - IndexOutOfBoundsException in CorrelationAttributePartitioner
This closes #5804

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-02-25 11:42:02 -05:00
exceptionfactory
49d1c747ca
NIFI-9724 Added set-sensitive-properties-algorithm command
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5801.
2022-02-25 11:03:55 -05:00
Mark Payne
432f98714f
NIFI-9725: On shutdown, instead of spawning a background thread to shutdown Cluster Coordinator, do so in the calling thread. This avoids a race condition whereby the cluster coordinator cannot be determined because the other thread has shutdown the FlowController.
This closes #5800

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-02-24 22:16:01 -06:00
Mark Payne
c453d2ba8f
NIFI-9723: When we add controller-level Controller Services on restart of NiFi, ensure that all Controller Services are updated to include their property values, etc. Also ensure that for these services and reporting tasks we decrypt the property values.
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5797.
2022-02-24 15:13:14 -05:00
Hsin-Ying Lee
c331f9cb9a
NIFI-9227 Run Once not working when scheduling strategy is CRON or Event driven (#5445) 2022-02-24 12:27:01 -05:00
Mark Payne
9237b4e923
NIFI-9722: Do not throw an Exception from verifyCanUpdateProperties when property descriptor & parameter descriptor's sensitivities don't match - instead allow the set to happen and let processor become invalid. Also, allow values such as abc#{param} for ghost processors.
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5795.
2022-02-24 08:58:54 -05:00
Mark Payne
3511ce3d13
NIFI-9548: When disabling RPG transmission, wait for the ports to complete in a background thread instead of blocking the web thread. Also moved the RPG initialization logic into flow controller instead of flow service and added a delay in order to reduce likelihood of ConnectException happening when pointing to nodes in the same cluster
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5641.
2022-02-23 11:29:26 -05:00
Mark Payne
ce0122bd25
NIFI-9691: Added ForkEnrichment, JoinEnrichment processors
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5772.
2022-02-22 18:54:32 -05:00
Mark Payne
d0a23bc26b
NIFI-9704: Updated the ContentRepositoryScanTask to show details of how much content in the content repo is retained by each queue in the dataflow. Changed default for nifi.content.claim.max.appendable.size property from 1 MB to 50 KB. Updated docs to reflect the new default value and explain what the property does and how it's used.
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5780.
2022-02-22 12:06:54 -05:00
Nathan Gough
7ef2fd2986
NIFI-7333 Added OIDC trust store strategy property
This closes #5753

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-02-18 15:19:22 -05:00
Roberto Marturano
a8fd5993eb NIFI-7192 Added systemd reload to nifi.sh install on systemd servers
This closes #4079

Signed-off-by: Joey Frazee <jfrazee@apache.org>
2022-02-18 10:40:52 -08:00
Mark Payne
2aa6bd1e13
NIFI-9689: When all FlowFiles in a FlowFile Queue are penalized, do not schedule the destination to run. Also expose this fact via the ConnectionStatusSnapshotDTO, as this allows the front-end to render this information to the user in order to avoid confusion when it appears that the Processor has data but does nothing
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5771
2022-02-16 15:30:00 -05:00
exceptionfactory
fe1139b8bb
NIFI-9692 Upgraded Apache Commons Lang3 to 3.12.0
- Moved commons-lang3 version management to root Maven configuration
- Refactored limited usage of StringUtils is nifi-reporting-utils to remove commons-lang3
- Refactored limited usage of StringUtils in nifi-websocket-processors to remove commons-lang3

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5773
2022-02-16 09:28:21 -05:00
timeabarna
63a543cd67
NIFI-9475 Provide Framework-Level Retries for NiFi Relationships (#5593) 2022-02-15 15:40:41 -05:00
tpalfy
84b2484fd4
NIFI-9684 Fix: When starting/stopping selected process group, it sends the parent process group id to the REST interface that is responsible to enable/disable transmission for all remote process groups within a process group. Need to send the id of the select process group instead. (#5764) 2022-02-15 14:55:17 -05:00
greyp9
87cfd43f6f
NIFI-9233 - Improve reliability of system integration tests (#5749)
* NIFI-9233 - Improve reliability of system integration tests
2022-02-15 13:14:01 -05:00
Steve Lawrence
4815df29db
NIFI-6740: Add configuration options to specify NiFi/Bootstrap communication ports
The NiFi and NiFi Bootstrap processes both bind to random ephemeral
ports to allow for inter-process communication (e.g. shutdown, port,
ping, etc.). However, the randomness of these ephemeral ports can pose
challenges for some security policies and firewall rules.

This adds two configuration options, nifi.bootstrap.listen.port and
nifi.listener.bootstrap.port, that allow an administrator to define
which ports the two processes should bind to for this communication,
making it easier to define security policies. The options default to
zero to maintain the current ephemeral port behavior.

NIFI-6740: Add configuration options to specify NiFi/Bootstrap communication ports

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5746
2022-02-15 11:46:21 -05:00
Scott Aslan
b8ec6d38dc [NIFI-9554] upgrade node and npm
update lintin
update font awesome path
bump nifi-fds version
update nifi deps
update frontend maven plugin
change CI npm cache restore keys so that we will not use cached directories for older versions of node

This closes #5751.

Signed-off-by: Kevin Doran <kdoran@apache.org>
2022-02-14 14:24:02 -05:00
exceptionfactory
e584d3cf04
NIFI-9660 Upgraded Apache Tika to 2.3.0
- Upgraded tika-core from 1.27 to 2.3.0
- Upgraded tika-parsers to tika-parsers-standard-package in nifi-media-processors
- Updated Tika metadata property references

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5754.
2022-02-09 11:11:46 +01:00
Mark Payne
94668dee6e
NIFI-9639: Determine how long it takes to find cluster coordinator and perform DNS lookup when sending heartbeats and include in the logs
This closes #5748

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-02-04 20:19:07 -06:00
markap14
230ed9c98d
NIFI-9628: Added a uiOnly flag when requesting Controller Service det… (#5712)
* NIFI-9628: Added a uiOnly flag when requesting Controller Service details and the list of Controller Services. This allows us to return much less data when retrieving these resources.

* NIFI-9628: Addressed review feedback; added uiOnly flag for controller service run-status and references also

* NIFI-9628: Fixed checkstyle issues by removing unused imports

This closes #5712
2022-02-03 15:53:23 -05:00
greyp9
9dbd066238
NIFI-9233 - Improve reliability of system integration tests (#5733) 2022-02-03 10:20:55 -05:00
exceptionfactory
ba0bf2ede0
NIFI-9650 Upgraded OkHttp from 4.9.2 to 4.9.3
- Added okhttp-bom to root Maven configuration
- Removed repeated okhttp.version property from specific dependencies

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5737.
2022-02-02 11:45:18 +01:00
exceptionfactory
43748a5523 NIFI-9638 Refactored Google Guava references
- Refactored nifi-framework and nifi-standard modules
- Replaced Google Cache with Caffeine Cache
- Replaced Google collections classes with standard Java collections

This closes #5730.

Signed-off-by: Kevin Doran <kdoran@apache.org>
2022-02-01 15:17:50 -05:00
greyp9
27ceda38bf
NIFI-9233 - Improve reliability of system integration tests (#5720) 2022-01-31 09:20:54 -05:00
Timea Barna
3ccc9d29b6
NIFI-9455 Added aggregated predictions to Prometheus Flow Metrics
This closes #5582

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-01-27 10:07:01 -06:00
Mark Payne
3ea9faccc6
NIFI-8040: When changing version of a flow, stop processors that have a state of Starting in addition to those with a state of Running
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5718.
2022-01-27 15:06:40 +01:00
Adam Kocsis
3eef7e0a3e
NIFI-9578 NiFi user list no longer breaks if a user and group have the same id (#5695)
- ids of the entities are prefixed by their types to avoid id conflict in 3rd party table library
   For some authorizers the UUID calculation for users and groups based only on the name of the entity and a global seed.
   This results the same UUID for a group and a user with the same name. The same ids are no longer causing any issue on the UI.
- group icon alignment is fixed for edit user dialog

This closes #5695
2022-01-25 14:52:07 -05:00
exceptionfactory
e2243500d3
NIFI-9618 Upgraded Checkstyle to 9.2.1
- Corrected several missing return statements
- Increased Maven heap size to 3 GB for ci-workflow

This closes #5700

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-01-21 20:08:16 -06:00
Bryan Bende
6ef6e71d05
NIFI-9501 Added REST end-point to retrieve a RuntimeManifest
- Switch ExtensionManifestParser implementation to use JAXB instead of Jackson XML
- Fix handling of older NARs that don't have newer fields in ExtensionManifest
- Add response merging for runtime-manifest REST end-point, remove commons-lang3 from c2 dependencies

This closes #5630

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-01-21 15:37:04 -06:00
Timea Barna
ccd47de6dc
NIFI-9587 Added JSON format for Prometheus Flow Metrics
This closes #5673

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-01-20 11:14:01 -06:00
Adam Kocsis
3c2793be09
[NIFI-8006] Additional options on UI to leave current process group (#5678)
- 'Leave group' action button is added to 'Navigation'
- 'Leave group' action works with 'esc' hotkey if no modal, context menu, etc. is open
- 'esc' key closes context menu if it is open
- user guide is updated with new navigation options

This closes #5678
2022-01-20 10:58:05 -05:00
exceptionfactory
fc27b3138b
NIFI-9481 Excluded Data Transfer REST methods from DoSFilter
- Added DataTransferDoSFilter with request URI evaluation
- Added RequestFilterProvider and implementations to abstract Jetty Filter configuration

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5670.
2022-01-19 16:11:53 -05:00
Joe Witt
2f95eed8cb
NIFI-9575 This closes #5667. updating copyright year to 2022
Signed-off-by: Joe Witt <joewitt@apache.org>
2022-01-19 09:14:09 -07:00
exceptionfactory
dd7131b257
NIFI-9435 Added registries and names include parameters to Flow Metrics
- Added optional includedRegistries query parameter to Flow Metrics Resource method supporting one or more registries
- Added optional includedNames query parameter to Flow Metrics Resource method supporting one or more metric family names
- Added sampleName and sampleLabelValue optional pattern parameters
- Added FilteringMetricFamilySamplesEnumeration to support streamed filtering
- Added PrometheusMetricsWriter and TextFormat implementation

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5571.
2022-01-13 15:59:00 -05:00
exceptionfactory
2ffd4a5a9a
NIFI-9438 Refactored sensitive-property-provider to multiple modules
- Added nifi-property-protection-api with provider interfaces
- Added nifi-property-protection-factory with implementation references
- Added ProtectionSchemeResolver for abstracting conversion from command arguments
- Refactored PropertyProtectionScheme to package private visibility
- Refactored multiple unit test and removed provider integration tests
- Renamed AESSensitivePropertyProvider to AesGcmSensitivePropertyProvider
- Added getSupportedProtectionSchemes() to StandardProtectionSchemeResolver
- Updated command argument descriptions for protection schemes to include supported values

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5650.
2022-01-13 13:18:04 -05:00
exceptionfactory
5832dff25e
NIFI-9556 Upgraded Apache HttpClient to 4.5.13
- Upgraded Apache HttpCore to 4.4.15
- Added dependency management declarations in root Maven configuration for HttpClient and HttpCore
- Removed version numbers from multiple modules

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5647.
2022-01-08 11:06:07 +01:00
exceptionfactory
81e57c9651
NIFI-9535 Upgraded Spring Framework to 5.3.14
- Centralized Spring Framework and Spring Security versions using BOM dependencies
- Upgraded Spring Security from 5.5.2 to 5.6.1
- Upgraded Spring Boot from 2.5.5 to 2.5.8 in Registry
- Upgraded Spring Integration from 5.5.2 to 5.5.7

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5631.
2022-01-05 15:30:03 +01:00