Commit Graph

1135 Commits

Author SHA1 Message Date
Bryan Bende 546f986603
NIFI-9775 Create RuntimeManifestService
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5849.
2022-03-10 07:52:16 -05:00
Matt Gilman 227dcf5456
NIFI-9777: (#5850) - Adding support to remove attributes from verification requests.
This closes #5850
2022-03-08 16:25:32 -05:00
Joe Gresock 743020eeb4
NIFI-9762: Adding DBCPConnectionPool config verification
Relaxing MockPropertyValue validation to allow for variables to be passed to config verification

Fixing underlying framework issue with config verification: wrong variable registry was being used

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5843
2022-03-07 14:43:11 -05:00
Eduardo Fontes d062029258
NIFI-8927 - Add option to start/stop all controllers (#5247)
* Update nf-context-menu.js

Include enable/disable all controllers menu item

* Update nf-actions.js

Create enableAllControllers and disableAllControllers actions

* Rename ...Controllers to ...ControllerServices

* Rename *controllers to *controller services

* Adjusts proposed by reviewer.

* Refactor enable and disable AllControllerServices

This closes #5247
2022-03-07 13:57:35 -05:00
Mark Payne 8959226b50
NIFI-9754: Introduced VersionedExternalFlow
- Updated stateless and StandardProcessGroup, etc. to make use of VersionedExternalFlow
- Updated StatelessDataflowDefinition to use ExternalVersionedFlow instead of generic type
- Updated Stateless Bootstrap to avoid loading stateless engine libs from root class path but instead use a NarClassLoader to load the statelss nar

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5832.
2022-03-04 07:19:38 -05:00
Shane Ardell 6cd30e3d0a
NIFI-9543: Add bring-to-front functionality to labels (#5765)
* NIFI-9543: add bring-to-front functionality to labels

* NIFI-9543: update zIndex changes to persist after shutdown

* NIFI-9543: remove commented code

* NIFI-9543: update flow comparator and snippet to account for zIndex

* NIFI-9543: changed add function to sort labels when a new label is added

* NIFI-9543: remove unnecessary call to position

* NIFI-9543: only register difference if zIndex is not default value

* NIFI-9543: add null checks to prevent primitive Long

This closes #5765
2022-03-03 18:50:15 -05:00
Adam Kocsis 7a16619969
[NIFI-9690] Display icon on Connection Label to show when head of the queue is penalized (#5812)
This closes #5812
2022-03-03 09:13:00 -05:00
exceptionfactory d17fcf6427 NIFI-9735 Corrected Jetty Duplicate Mapping Warning
- Removed duplicate default servlet mapping from addDocsServlets()

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5817.
2022-03-01 15:49:23 -05:00
Shane Ardell 4719f78ef4
NIFI-9580: UI work for framework-level retry in Processors (#5666)
* NIFI-9580: initial ui work for framework-level retry in processors

* NIFI-9580: disable retry controls when retry not selected

* NIFI-9580: refactor marshalling to be one function

* NIFI-9580: style tweaks and documentation update

* NIFI-9580: update tooltip content

* NIFI-9580: change tooltip copy for clarity

* NIFI-9580: only submit retry configs if selected

* NIFI-9580: add relationship details to read-only dialog

* NIFI-9580: change styling based on feedback

This closes #5666
2022-03-01 14:57:18 -05:00
exceptionfactory 52f21390db
NIFI-9688 Improve Logback shutdown handling (#5805)
* NIFI-9688 Improved Logback shutdown handling
- Added environment variable to disable Logback shutdown on web application termination
- Added shutdown hook to Logback configuration
- Added Logback shutdown hook configuration to system tests
2022-02-28 14:29:44 -05:00
Mark Payne 3511ce3d13
NIFI-9548: When disabling RPG transmission, wait for the ports to complete in a background thread instead of blocking the web thread. Also moved the RPG initialization logic into flow controller instead of flow service and added a delay in order to reduce likelihood of ConnectException happening when pointing to nodes in the same cluster
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5641.
2022-02-23 11:29:26 -05:00
Nathan Gough 7ef2fd2986
NIFI-7333 Added OIDC trust store strategy property
This closes #5753

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-02-18 15:19:22 -05:00
Mark Payne 2aa6bd1e13
NIFI-9689: When all FlowFiles in a FlowFile Queue are penalized, do not schedule the destination to run. Also expose this fact via the ConnectionStatusSnapshotDTO, as this allows the front-end to render this information to the user in order to avoid confusion when it appears that the Processor has data but does nothing
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5771
2022-02-16 15:30:00 -05:00
timeabarna 63a543cd67
NIFI-9475 Provide Framework-Level Retries for NiFi Relationships (#5593) 2022-02-15 15:40:41 -05:00
tpalfy 84b2484fd4
NIFI-9684 Fix: When starting/stopping selected process group, it sends the parent process group id to the REST interface that is responsible to enable/disable transmission for all remote process groups within a process group. Need to send the id of the select process group instead. (#5764) 2022-02-15 14:55:17 -05:00
Scott Aslan b8ec6d38dc [NIFI-9554] upgrade node and npm
update lintin
update font awesome path
bump nifi-fds version
update nifi deps
update frontend maven plugin
change CI npm cache restore keys so that we will not use cached directories for older versions of node

This closes #5751.

Signed-off-by: Kevin Doran <kdoran@apache.org>
2022-02-14 14:24:02 -05:00
exceptionfactory e584d3cf04
NIFI-9660 Upgraded Apache Tika to 2.3.0
- Upgraded tika-core from 1.27 to 2.3.0
- Upgraded tika-parsers to tika-parsers-standard-package in nifi-media-processors
- Updated Tika metadata property references

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5754.
2022-02-09 11:11:46 +01:00
markap14 230ed9c98d
NIFI-9628: Added a uiOnly flag when requesting Controller Service det… (#5712)
* NIFI-9628: Added a uiOnly flag when requesting Controller Service details and the list of Controller Services. This allows us to return much less data when retrieving these resources.

* NIFI-9628: Addressed review feedback; added uiOnly flag for controller service run-status and references also

* NIFI-9628: Fixed checkstyle issues by removing unused imports

This closes #5712
2022-02-03 15:53:23 -05:00
exceptionfactory 43748a5523 NIFI-9638 Refactored Google Guava references
- Refactored nifi-framework and nifi-standard modules
- Replaced Google Cache with Caffeine Cache
- Replaced Google collections classes with standard Java collections

This closes #5730.

Signed-off-by: Kevin Doran <kdoran@apache.org>
2022-02-01 15:17:50 -05:00
Timea Barna 3ccc9d29b6
NIFI-9455 Added aggregated predictions to Prometheus Flow Metrics
This closes #5582

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-01-27 10:07:01 -06:00
Mark Payne 3ea9faccc6
NIFI-8040: When changing version of a flow, stop processors that have a state of Starting in addition to those with a state of Running
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5718.
2022-01-27 15:06:40 +01:00
Adam Kocsis 3eef7e0a3e
NIFI-9578 NiFi user list no longer breaks if a user and group have the same id (#5695)
- ids of the entities are prefixed by their types to avoid id conflict in 3rd party table library
   For some authorizers the UUID calculation for users and groups based only on the name of the entity and a global seed.
   This results the same UUID for a group and a user with the same name. The same ids are no longer causing any issue on the UI.
- group icon alignment is fixed for edit user dialog

This closes #5695
2022-01-25 14:52:07 -05:00
exceptionfactory e2243500d3
NIFI-9618 Upgraded Checkstyle to 9.2.1
- Corrected several missing return statements
- Increased Maven heap size to 3 GB for ci-workflow

This closes #5700

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-01-21 20:08:16 -06:00
Bryan Bende 6ef6e71d05
NIFI-9501 Added REST end-point to retrieve a RuntimeManifest
- Switch ExtensionManifestParser implementation to use JAXB instead of Jackson XML
- Fix handling of older NARs that don't have newer fields in ExtensionManifest
- Add response merging for runtime-manifest REST end-point, remove commons-lang3 from c2 dependencies

This closes #5630

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-01-21 15:37:04 -06:00
Timea Barna ccd47de6dc
NIFI-9587 Added JSON format for Prometheus Flow Metrics
This closes #5673

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-01-20 11:14:01 -06:00
Adam Kocsis 3c2793be09
[NIFI-8006] Additional options on UI to leave current process group (#5678)
- 'Leave group' action button is added to 'Navigation'
- 'Leave group' action works with 'esc' hotkey if no modal, context menu, etc. is open
- 'esc' key closes context menu if it is open
- user guide is updated with new navigation options

This closes #5678
2022-01-20 10:58:05 -05:00
exceptionfactory fc27b3138b
NIFI-9481 Excluded Data Transfer REST methods from DoSFilter
- Added DataTransferDoSFilter with request URI evaluation
- Added RequestFilterProvider and implementations to abstract Jetty Filter configuration

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5670.
2022-01-19 16:11:53 -05:00
Joe Witt 2f95eed8cb
NIFI-9575 This closes #5667. updating copyright year to 2022
Signed-off-by: Joe Witt <joewitt@apache.org>
2022-01-19 09:14:09 -07:00
exceptionfactory dd7131b257
NIFI-9435 Added registries and names include parameters to Flow Metrics
- Added optional includedRegistries query parameter to Flow Metrics Resource method supporting one or more registries
- Added optional includedNames query parameter to Flow Metrics Resource method supporting one or more metric family names
- Added sampleName and sampleLabelValue optional pattern parameters
- Added FilteringMetricFamilySamplesEnumeration to support streamed filtering
- Added PrometheusMetricsWriter and TextFormat implementation

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5571.
2022-01-13 15:59:00 -05:00
exceptionfactory 81e57c9651
NIFI-9535 Upgraded Spring Framework to 5.3.14
- Centralized Spring Framework and Spring Security versions using BOM dependencies
- Upgraded Spring Security from 5.5.2 to 5.6.1
- Upgraded Spring Boot from 2.5.5 to 2.5.8 in Registry
- Upgraded Spring Integration from 5.5.2 to 5.5.7

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5631.
2022-01-05 15:30:03 +01:00
exceptionfactory 7c6bdcb035
NIFI-9491 Excluded commons-logging and added jcl-over-slf4j references
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5608
2021-12-17 07:21:03 -05:00
Shane Ardell 293919f881
NIFI-9460: Update Lodash to 4.17.21 (#5605)
* NIFI-9460: update lodash transitive dep in registry

* NIFI-9460: use lodash instead of lodash-core

Merged #5605 into main.
2021-12-16 14:28:02 -08:00
M Tien 3d5f357de8
[NIFI-9423-NIFI-9429]: Show icon and tooltip for Parameters with leading and/or trailing whitespace (#5569)
* NIFI-9423 - Show icon and tooltip for Parameter values that have leading and/or trailing whitespaces

NIFI-9429 - Parameters should allow blank values that are non-null (only whitespace)

* - Update areas to clean up tooltips in parameter values
- Show whitespaces and ellipsis in parameter and property values and tooltips
- Update serializeValue to accommodate for blank values

* - Address review findings

* - Remove commented out code

* - Add multiline check for ellipsis

* NIFI-9459 - Empty string checked will disable Edit Parameter value field on dialog open

* - Add multi-line style to parameter and property table

* - Safely insert title attribute content

* - Fix Edit Parameter bug that clears textarea for sensitive and empty string values on dialog open

This closes #5569
2021-12-13 16:01:42 -05:00
Mark Payne 90b39b593a
NIFI-9069 Changed framework dataflow serialization to support JSON
- Changed framework so that it serializes the dataflow into a VersionedDataflow using JSON as well as XML, and prefers the JSON representation on load, if it's available. This also results in the need for the cluster protocol to exchange its representation of the dataflow to using JSON. Rather than re-implementing all of the complex logic of Flow Fingerprinting, updated to just inherit the cluster's flow.
- Moved logic to synchronize Process Group with Versioned Process Group into a new ProcessGroupSynchronizer class instead of having all of the logic within StandardProcessGroup
- Reworked versioned components to use an instance id.
- Renamed StandardFlowSynchronizer to XmlFlowSynchronizer; introduced new StandardFlowSynchronizer that delegates to the appropriate (Xml or Versioned)FlowSynchronzer
- Updated to allow import of VersionedProcessGroup even if not all bundles are available - will now use ghost components
- Introduced a VersionedDataflow object to hold controller-level services, reporting tasks, parameter contexts, templates, etc.
- Allow mutable requests to be made while nodes are disconnected. Also fixed issue in AbstractPolicyBasedAuthorizer that caused ClassNotFoundException / NoClassDefFoundError if the authorizations were changed and then a node attempts to rejoin the cluster. The Authorizer was attempting to use XmlUtils, which is in nifi-security-utils and so so by madking nifi-security-utils a provided dependency of nifi-framework-api, but this doesn't work, because nifi-framework-api is loaded by a higher-level classloader, so the classloader that loads AbstractPolicyBasedAuthorizer will never have the appropriate classloader to provide nifi-security-utils. Addressed this by copying the code for creating a safe document builder from XmlUtils to AbstractPolicyBasedAuthorizer.
- Fixed bug that occurred when importing a Process Group that has 2 parameter contexts, one inheriting from another, where neither is pre-defined in the existing flow
- Fixed bug that was encountered when Updating a Versioned Process Group where one version had a disabled processor and the other had the processor running.
- Increased system-tests workflow timeout to 120 minutes
- Added additional exception handling to system tests

This closes #5514

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-12-02 15:10:38 -06:00
M Tien 2b415de912
NIFI-9336 - Show icon for property values with leading or trailing whitespace (#5559)
* NIFI-9336 - Show icon in processor and controller services configurations when property values contain leading or trailing whitespace

* - Address PR feedback

* - Fix a bug to clean up tooltips to prevent a DOM leak

This closes #5559
2021-12-02 13:31:19 -05:00
exceptionfactory 6a6b755aaa
NIFI-9394 Removed RequestLogger and TimerFilter
- Removed logger references from logback.xml

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5543.
2021-11-29 12:55:39 -05:00
Joe Gresock a5581562bf
NIFI-9416: Fixing NPE when updating param context without inheritedParameterContexts (#5553) 2021-11-29 11:29:33 -05:00
Joe Gresock dfd8adfd8e
NIFI-9411: Adding missing 'setSubtitle' call for Controller Services modal
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5551.
2021-11-24 18:01:48 +01:00
markap14 839fbf7d19
NIFI-9382: Created a new ClassloaderIsolationKey mechanism by which H… (#5533)
* NIFI-9382: Created a new ClassloaderIsolationKey mechanism by which Hadoop related processors (and potentially others) can indicate that they need full classloaders to be cloned but can share with other instances in certain circumstances
- Added system tests

* NIFI-9382: Renamed interface based on review feedback

* NIFI-9382: Removed ReentrantKerberosUser.
2021-11-22 14:55:19 -05:00
Timea Barna 848a22eb29
Revert "NIFI-9385 Add Flow Metrics producer for Cloudera Manager"
This reverts commit 785021dade.

This closes #5540

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-11-19 08:52:20 -06:00
Timea Barna 785021dade
NIFI-9385 Add Flow Metrics producer for Cloudera Manager
This closes #5528

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-11-18 11:51:53 -06:00
exceptionfactory 605346ae0f
NIFI-9291 Added NiFi HTTP request logging
- Added nifi.web.request.log.format property
- Added Filters to set and retrieve authenticated username for logging

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5527.
2021-11-18 10:30:50 -05:00
M Tien 5016eac010
NIFI-9205 - Update prioritizer configuration (#5516)
* NIFI-9205 - Update prioritizer configuration

* - Address review feedback

This closes #5516
2021-11-17 13:55:02 -05:00
noblenumbat360 f410c8df0a
NIFI-8272 Delete stale metrics from REST API Prometheus endpoint.
Added <scope>test</scope> tag to the nifi-web-api pom.xml and corrected imports.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5447
2021-11-08 20:26:30 -05:00
Joe Witt c055895952
NIFI-9344-RC3 prepare for next development iteration 2021-11-03 08:53:37 -07:00
Joe Witt 7fdc07cccd
NIFI-9344-RC3 prepare release nifi-1.15.0-RC3 2021-11-03 08:53:32 -07:00
exceptionfactory 9865ea2bfb NIFI-9322 Refactored OIDC and SAML Access Resources
- Removed parent AccessResource from OIDCAccessResource and SAMLAccessResource to avoid unexpected inherited methods
- Moved Token Expiration validation from AccessResource to StandardBearerTokenProvider

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5489.
2021-10-28 12:12:57 -04:00
Matthew Burgess 104078868e
NIFI-7012: Refactored OnConfigurationRestored to support sensitive property validation (#5415) 2021-10-27 14:40:17 -04:00
markap14 5766d33fce
NIFI-9309: Include a uiOnly flag when requesting flow for a given pro… (#5462)
NIFI-9309: Include a uiOnly flag when requesting flow for a given process group. In that case, do not include the property descriptors, property values, etc. for Processors. When fetching Variable Registry, improved logic to cache the VariableImpact for each property value instead of parsing/recreating it every time. When fetching bulletins for a component, avoid filtering through all components' bulletins and instead only look at bulletins that might pertain to the appropriate component
2021-10-23 19:42:31 -04:00
Matt Gilman 1bec905890 NIFI-9288:
- Allowing the user to submit a verification request for Processors, Controller Services, and Reporting Tasks.
- Tracking progress of verification requests.
- Showing the verification results.

NIFI-9288:
- Fixing class name which prevented styles from being applied.

NIFI-9288:
- Ensuring that previously entered referenced attribute values take precedence.

NIFI-9288:
- Positioning the property listing and verification results based on percentages instead of fixed values.
- Removing the additional dialog height.

NIFI-9288:
- Allowing attribute value entry to be skipped when appropriate.

NIFI-9288:
- Working around an issue caused by css minification.

NIFI-9288:
- Adding some padding to the verifying progress dialog.

This closes #5461

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
2021-10-21 10:06:58 -04:00