Commit Graph

205 Commits

Author SHA1 Message Date
Timea Barna 42910e80d1
NIFI-11658 Streamline using single Parameter Context for nested PGs
This closes #7353

Co-authored-by: Matt Gilman <mcgilman@apache.org>
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-07-27 15:02:54 -05:00
Pierre Villard 150bfbe990
NIFI-11813 Removed Event Driven Scheduling Strategy
This closes #7483

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-07-18 12:22:52 -05:00
Bence Simon cadf2fb6f2
NIFI-11706 Add option to create dedicated Parameter Contexts for Imported Flows
This closes #7401

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-06-30 12:56:27 -05:00
timeabarna ba6797bb94
NIFI-3065 Per Process Group logging (#7315)
* NIFI-3065 Per Process Group logging
2023-06-23 21:14:34 -04:00
Bence Simon 007bf3bcec
NIFI-11596 Adjusting Swagger contract to cover RegistryClients as possible reference type for Controller Services
This closes #7295

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-05-31 11:02:48 -05:00
Timea Barna 5811a9c579
NIFI-11385 Added JMX Metrics REST Resource for Diagnostics
This closes #7124

Co-authored-by: David Handermann <exceptionfactory@apache.org>
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-05-08 08:11:20 -05:00
Timea Barna a9274c12c5
NIFI-11079 Extend QueryNifiReportingTask and ProcessGroup Summary Table
Correcting EmbeddedQuestDbStatusHistoryRepositoryForComponentsTest

Correcting TestQueryNiFiReportingTask

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6869
2023-01-20 15:22:20 -05:00
dan-s1 e8d554d9b4
NIFI-10876 Converted some nifi-framework tests from JUnit 4 to JUnit 5
This closes #6720

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-11-28 14:47:17 -06:00
Bryan Bende 83d4e5bd04
NIFI-10771 Add SINGLE_BATCH_PER_NODE to allowable values for flow file concurrency in ProcessGroupDTO
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6627
2022-11-07 13:58:49 -05:00
Mark Payne d390a0becf
NIFI-10673: When a component is added to a ProcessGroup, we had a method called ensureUniqueVersionControlId but the method only took into accounts in that Process Group. Updated methods to now consider components in any Process Group that is part of the same Versioned Flow. Also added system test to verify the problem and the fix. Added Toolkit clients and additional methods as necessary in order to implement system tests
NIFI-10673: Addresed checkstyle issue

NIFI-10673: Removed TODO comment
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
2022-10-26 14:38:01 -04:00
simonbence 94fdc794b0
NIFI-10497 Making RegistryClient an extension point (#6433)
- NIFI-10497 Mae FlowRegistryClient an extension point
2022-09-26 12:16:21 -04:00
Joe Gresock ece83709f4
NIFI-10473: Removing referencing components check on param provider f… (#6388)
* NIFI-10473: Removing referencing components check on param provider fetch

* NIFI-10473: Adding parameter status DTO to ParameterProviderDTO

* Allowing parameterStatus to be populated even when no parameters were updated

* Adding ParameterStatus enum for parameter fetching

* Adding MISSING_BUT_REFERENCED ParameterStatus

This closes #6388
2022-09-23 13:44:52 -04:00
Joe Gresock de7793e69b
NIFI-9003 Added framework support for Parameter Providers
- Added Environment Variable and File Parameter Provider implementations
- Added Parameter Provider commands to CLI
- Added component search handling for Parameter Providers
- Added documentation for integrating Parameter Providers

This closes #5369

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-09-09 13:04:56 -05:00
Mark Payne a0c705715b
NIFI-10429: Added the ability to Replay latest provenance event for a given Processor.
This closes #6359

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-09-08 15:55:26 -05:00
Nathan Gough c77f85aafb
NIFI-10070 Updated status merging for ControllerService and ReportingTask entities
- Corrected node identifier selection in multiple Mergers

This closes #6154

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-07-27 14:48:02 -05:00
Otto Fowler cf4eed0224 NIFI-10207: ensure allowableValues matches SUPPORTED_STATE in ProcessorRunStatusEntity
This closes #6188.

Signed-off-by: Tamas Palfy <tpalfy@apache.org>
2022-07-11 21:34:27 +02:00
exceptionfactory 0de83292de NIFI-9849 Refactored SAML Support with Spring Security 5
- Updated SAML Authentication Configuration with Spring Security SAML 2 components
- Updated Administration Guide with REST Resources
- Replaced SAMLAccessResource methods with applicable Spring Security Filters
- Removed IDP Credential Service and supporting components
- Removed message.logging.enabled, metadata.signing.enabled, and signature.digest.algorithm properties
- Added Access Token Expiration resource method
- Removed Saml2AccessResource and replaced with Access Token Expiration to avoid unnecessary conflicts with SAML login consumer
- Corrected Resource URI handling to support proxy server access

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #6149.
2022-06-28 13:57:35 -04:00
Nissim Shiman db11961026
NIFI-9440 Allow Controller Services to have configurable Bulletins
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6035
2022-06-28 10:44:45 -04:00
exceptionfactory 99be62be67
NIFI-9958 Add Framework Support for Sensitive Dynamic Properties (#6057)
* NIFI-9958 Implemented Sensitive Dynamic Properties

- Added SupportsSensitiveDynamicProperties annotation for components
- Added optional sensitive query parameter to Property Descriptor REST Resource methods
- Added system tests for components supporting sensitive dynamic properties
- Updated REST Resources to support Sensitive Dynamic Property Names
- Updated Documentation Writer to indicate component support for Sensitive Dynamic Properties
- Updated InvokeHTTP to support Sensitive Dynamic Properties
- Updated Auditor components to handle masking Sensitive Dynamic Properties

* Refactored Property Descriptor REST method handling

- Corrected AbstractDocumentationWriter evaluation of support for sensitive dynamic properties
- Refactored Controller Service Dynamic Properties system tests to new class

* Updated AbstractComponentNode.getProperties() to get canonical descriptor
2022-05-23 17:20:54 -04:00
Shane Ardell 6cd30e3d0a
NIFI-9543: Add bring-to-front functionality to labels (#5765)
* NIFI-9543: add bring-to-front functionality to labels

* NIFI-9543: update zIndex changes to persist after shutdown

* NIFI-9543: remove commented code

* NIFI-9543: update flow comparator and snippet to account for zIndex

* NIFI-9543: changed add function to sort labels when a new label is added

* NIFI-9543: remove unnecessary call to position

* NIFI-9543: only register difference if zIndex is not default value

* NIFI-9543: add null checks to prevent primitive Long

This closes #5765
2022-03-03 18:50:15 -05:00
Mark Payne 2aa6bd1e13
NIFI-9689: When all FlowFiles in a FlowFile Queue are penalized, do not schedule the destination to run. Also expose this fact via the ConnectionStatusSnapshotDTO, as this allows the front-end to render this information to the user in order to avoid confusion when it appears that the Processor has data but does nothing
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5771
2022-02-16 15:30:00 -05:00
timeabarna 63a543cd67
NIFI-9475 Provide Framework-Level Retries for NiFi Relationships (#5593) 2022-02-15 15:40:41 -05:00
markap14 230ed9c98d
NIFI-9628: Added a uiOnly flag when requesting Controller Service det… (#5712)
* NIFI-9628: Added a uiOnly flag when requesting Controller Service details and the list of Controller Services. This allows us to return much less data when retrieving these resources.

* NIFI-9628: Addressed review feedback; added uiOnly flag for controller service run-status and references also

* NIFI-9628: Fixed checkstyle issues by removing unused imports

This closes #5712
2022-02-03 15:53:23 -05:00
Bryan Bende 6ef6e71d05
NIFI-9501 Added REST end-point to retrieve a RuntimeManifest
- Switch ExtensionManifestParser implementation to use JAXB instead of Jackson XML
- Fix handling of older NARs that don't have newer fields in ExtensionManifest
- Add response merging for runtime-manifest REST end-point, remove commons-lang3 from c2 dependencies

This closes #5630

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-01-21 15:37:04 -06:00
markap14 90ae271692
NIFI-9310: Addressed issues found while testing the component verification features. Added a supportsVerification flag to the ConfigAnalysis DTO (#5469)
This closes #5469
2021-10-20 11:07:17 -04:00
Mark Payne 151a9361fa
NIFI-9288: Updated data model to use for cnofig verification requests so that only properties are necessary not the full component/config dtos. Also added endpoint necessary for determining which attributes are referenced by a component's properties and removed the referencedAttributes field from the compnoents themselves, since there's now a new endpoint for it. Also fixed a bug that was encountered where the VerifyConfigRequestDTO's complete flag was incorrect in case of failures
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5452.
2021-10-18 11:42:47 -04:00
Matt Gilman be57a210ff
NIFI-8491: Adding support for configuring parameter context inheritance (#5371)
* NIFI-8491:
- Adding support for configuring parameter context inheritance.

* NIFI-8491:
- Allowing changes to the parameter context inheritance to drive Apply disabled state.

* NIFI-8491: Updating StandardParameterContext#isAuthorized check

* NIFI-8491:
- Showing selected inherited parameter contexts in ready only form when appropriate.
- Allowing available parameter contexts to be inherited by double clicking.
- Removing support for rendering unauthorized inherited parameter contexts as they can no longer be opened.

* NIFI-8491: Adding inherited param context verification earlier

* NIFI-8491:
- Addressing CI failures by rolling back to some order JS language spec to allow yui-compress to minify and compress.

* NIFI-8491:
- Ensuring selected context sort order is honored.
- Ensuring the Apply button is correctly enabled.
- Showing Pending Apply message when selected Parameter Context changes.
- Ensuring the Parameter's tab is selected now that there is a third tab.

* Updates to inherited param context verification

* Improving validation between parameters/inherited parameters

* NIFI-8491:
- Ensuring the available parameter contexts are loaded whether the edit dialog is opened from the listing or outside of the listing.

* NIFI-8491:
- Fixing conditions we check if the parameter context listing is currently open.

* NIFI-8491:
- Waiting for the parameter contexts to load prior to rendering the parameter context inheritance tab and showing the dialog.

* NIFI-8491:
- Fixing pending apply message clipping.
- Hiding pending apply message after clicking Apply.

Co-authored-by: Joe Gresock <jgresock@gmail.com>

This closes #5371
2021-09-29 17:04:35 -04:00
Mark Payne baf29e59f1
NIFI-9009: Created VerifiableProcessor, VerifiableControllerService, VerifiableReportingTask components; implemented backend work to call the methods. Added REST APIs and created/updated data models for component configuration verification
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5288
2021-09-22 19:00:23 -04:00
markap14 73f88b3239
NIFI-9061: Eliminated the nifi.cluster.node.protocol.threads property in favor of nifi.cluster.node.protocol.max.threads property so that we can properly scale out the number of threads used for HTTP request replication. Implementing a caching mechanism for creating the DateTimeFormatter used by TimeAdapter in order to improve performance when parsing timestamps in web requests. Implementing caching logic for caching the number of characters that can rendered without needing an ellipsis for some components in the UI (#5316)
This closes #5316
2021-09-13 14:36:35 -04:00
Joe Gresock 4a3e81531b
NIFI-8490: Adding inherited parameter contexts (#5072)
- Allowing inherited param contexts on creation, updating PC authorization
2021-08-30 10:44:31 -04:00
Mark Bean ea31634ea7
NIFI-8195: add default connection settings to process group configuration
- include new process group property support in NiFi Registry
  - updated documentation to describe and show new feature
  - added elements to XSD schema definition

NIFI-8195: update to DAO to fix PG move and copy/paste

update condition to not null vice null

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5192
2021-07-07 17:45:54 -04:00
Matthew Burgess 5e4f32663e Revert "NIFI-8195: add default connection settings to process group configuration"
This reverts commit 5ebbe0028b.
2021-06-24 18:52:17 -04:00
Mark Bean 5ebbe0028b
NIFI-8195: add default connection settings to process group configuration
NIFIDEVS-8195: fixed properties not properly inheriting from template/snippet values

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5094
2021-06-24 14:03:51 -04:00
Jose Luis Pedrosa f21c1be60f
NIFI-8374 Refactor readOnly to accessMode in ApiModelProperty usages
This closes #4945

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2021-04-02 14:22:07 -04:00
mtien b9a0991157
NIFI-8260 [WIP] Created Upload File front end and server side logic.
NIFI-8260 [WIP] Fixed server side logic to upload a flow file. Cleaned up the front end logic.
NIFI-8260 [WIP] Finished the server side upload logic.
Added a client ID parameter to the endpoint.
Added JSON parsing error response.
Fixed the client side file form to reset after submit.
Fixed the canvas to instantly update and show the process group after submitting the file.
Changed the Add Processor Group dialog UI based on design notes.
Changed the Upload File link to an icon and moved to the process group name input.
Changed the Registry Import link to say 'Import from Registry' and moved to the bottom of the dialog.
Display the filename when a file is selected.

NIFI-8260 [WIP] Added a cancel file button to the Process Group dialog.
Fixed some CSS styles.

NIFI-8260 - Removed accessing the snapshot metadata to avoid an NPE.
Added a title attribute to the html of the dialog file cancel button.

NIFI-8260 - Disabled the dialog 'Add' button.
Revised based on PR feedback.
Refactored the upload file endpoint and client side filename extraction methods.
Fixed some CSS.
Reverted some unnecessary changes.

NIFI-8260 - Revised based on PR feedback.
Refactored uploadProcessGroup.
Fixed some exception handling.
Hid the Upload File button when grouping components.
Refactored nf-ng-group-component.js replacing jquery selectors with variables.
Extracted the resetValues function to clear dialog values.

NIFI-8260 - Fixed the cluster replicate request.
Created a new endpoint to handle the cluster replicate request.
Created ProcessGroupUploadEntity.
Renamed positionX and positionY parameters.

NIFI-8260 - Fixed a checkstyle error.
Removed unnecessary httpServletRequest parameter.
Reverted some re-ordering of imports.

NIFI-8260 - Changed the ProcessGroupUploadEntity to be consistent with other Entity and DTO models.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4846.
2021-03-03 14:38:45 +04:00
Tamas Palfy aa726040c5 NIFI-8188 - Add 'Run Once' for processors in context menu.
Signed-off-by: Mark Payne <markap14@hotmail.com>
2021-02-18 14:51:28 -05:00
Mark Payne 4ff9cddf15
NIFI-8173, NIFI-8174: This closes #4784. Updated Parameter Contexts to allow for unsetting a parameter value / setting it to null. Allow Parameters to make use of Expression Language. Updated docs to illustrated how EL is evaluated
Signed-off-by: Joe Witt <joewitt@apache.org>
2021-01-26 21:55:18 -07:00
Nissim Shiman 3cc8d767b3 NIFI-7738 Reverse Provenance Query
This closes #4563.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2021-01-13 09:35:05 -05:00
Pierre Villard 14ec02f21d
NIFI-7981 - add support for enum type in avro schema
This closes #4648

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-11-05 18:19:55 -05:00
Mark Payne f7f336a4b0
NIFI-1121: Added API changes for having one Property depend on another
Signed-off-by: Bryan Bende <bbende@apache.org>
2020-11-03 15:37:08 -05:00
markap14 f8a730568f
NIFI-7627: Fixed typo that showed the wrong case for Run Statuses in the ProcessorRunStatusDetailsDTO (#4400) 2020-07-14 15:44:52 -04:00
Mark Payne c9d08a76b1 NIFI-7536: Fix to improve performance when determining the run status of processors when needing to wait for all processors to stop for updating parameter context, etc. 2020-06-19 10:14:12 -04:00
Mark Payne 359fd3ff29
NIFI-7476: Implemented FlowFileGating / FlowFileConcurrency at the ProcessGroup level
Added FlowFileOutboundPolicy to ProcessGroups and updated LocalPort to make use of it
Persisted FlowFile Concurrency and FlowFile Output Policy to flow.xml.gz and included in flow fingerprint
Added configuration for FlowFile concurrency and outbound policy to UI for configuration of Process Groups
Added system tests. Fixed a couple of bugs that were found
Fixed a couple of typos in the RecordPath guide

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4306.
2020-06-04 23:24:03 +02:00
Michael Hogue aeaf953e06
NIFI-5925: Added controller services to set of components that are searched
NIFI-5925: cleanup, add negative test

NIFI-5925: fixed checkstyle

This closes #4105

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-03-25 08:55:44 -04:00
MatthewKnight-NG acaf321af0
NIFI-5924 Labels should be searchable
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4070
2020-02-24 16:14:21 -05:00
Joe Ferner 62606ff89a NIFI-6873: Added support for replacing a process group
- decoupled flow update request behavior from VersionsResource into new abstract FlowUpdateResource
 - added replace process group functionality in ProcessGroupResource
 - parameterized FlowUpdateResource and created entity hierarchies to allow for maximum code sharing across different update types
 - refactored flow update methods to make use of commonality across different update types whenever possible
 - fixed issues in StandardProcessGroup verify update methods where same components existed in different ancestry chains but were considered a match when they shouldn't be
 - improved StandardProcessGroup to properly match up components on update using generated versioned component ids, when necessary to allow for update flow to efficiently match common components on flow import

This closes #4023.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2020-02-11 14:46:40 -05:00
Joe Ferner 058883091c
NIFI-6872:
- Added UI versioned flow supportsDownload functionality with download flow menu item
- Added VersionsResource endpoint for downloading versioned flow with registry-related info removed
- Added ProcessGroupResource endpoint for downloading current flow with registry-related info removed
- Added StandardNifiServiceFacade functionality for downloading both current and versioned flow
- Added XmlTransient markers on variables introduced by Instantiated model classes so they do not appear in serialized download
- Updated NiFiRegistryFlowMapper.mapParameterContexts to handle mapping nested parameter contexts for use in producing a complete VersionedFlowSnapshot
- Added ability for NiFiRegistryFlowMapper to map nested process groups ignoring versioning for use in producing a complete VersionedFlowSnapshot
- Added unit tests where helpful

NIFI-6872: PR response...
- Updated mapParameterContext to return a Map to handle uniqueness of contexts by name since ultimately everything converted it to a map anyway. The VersionedParameterContext class from the registry model doesn't support hashcode/equals currently so returning a Set wouldn't work.
- Updated assert calls to put expected value as first parameter and actual as second parameter
- Added one time password (OTP) support for flow download endpoint to support non cert based authentication

This closes #3931
2019-12-18 16:37:47 -05:00
Joe Ferner c947e57d5e
NIFI-6814:
- Restored JAXB Adapter

This closes #3880
2019-11-08 15:43:13 -05:00
Mark Payne 47f45abdf9
NIFI-6748: Fixed bug in Parameter Contexts' affected components where if a Controller Service referenced a Parameter, any component that references that service should also be considered an affected component but wasn't. Also fixed a bug in how we handled stopping a Processor that was in the STARTING phase.
This closes #3794.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-10-09 10:20:45 -04:00
Kevin Doran 2d90145842
NIFI-6749 This closes #3792. Fix Swagger for ControllerServiceReferencingComponentDTO
Signed-off-by: Joe Witt <joewitt@apache.org>
2019-10-08 11:48:10 -04:00