- Corrected Jetty AuthenticationState interface reference for authenticated user attribute mapping
- Added unit test verifying expected attribute values
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#8423.
* [NIFI-12761] refresh RPG
* address feedback
* stop polling on error
* fix linting errors introduced in this PR
* reload the connections
* refresh rpg, if first call does not have updated timestamp start polling
* rename action
* final touches
This closes#8401
Refactored so that when a Python Process dies, NiFi will detect this, respawn the process, recreate the Processors that exist in the process, re-initialize them, and restart them. In testing, found the PythonControllerInteractionIT had bugs that were causing Python Processors to be re-initialized many times; this resulted in threading issues that caused processors to be invalid, indicating that property descriptors didn't exist, etc. Addressed these concerns in the same commit, since they were necessary to properly run tests
Ensure that ClassLoader is consistently established for python processor proxies; ensure that if we re-initialize python processor the old initialization thread is stopped
This closes#8363
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Adjusted URIs for schemas Draft 4, Draft 6 and Draft 7 to match official values set on json-schema-validator 1.3.0
This closes#8405
Signed-off-by: David Handermann <exceptionfactory@apache.org>
* [NIFI-12790] update version of typescript and zone.js
* remove override of dependency
* upgrade to angular 17
* migration to control-flow - used "ng generate @angular/core:control-flow"
* update the version of node used to build with
* add engine.node requirement to package.json
This closes#8410
* [NIFI-12754] - Flow Configuration History
* support selection
* support pagination
* support sorting
* added time controls, also updated provenance to use them too
* allow for clearing of filters
* use date range filter
* more details dialog for flow config history
* support purge history
* review feedback - use snackbar error where intended, add padding between header and page content
* don't use route for flow config history item selection
* Address review feedback
* remove unused style
* Review feedback * initial query is for the last 2 weeks * added timezone to purge confirmation message * reset pagination state on filter, clear filter, and purge * only resubmit query when filter by changes IF there is a filter term specified
This closes#8399
* NIFI-12739 Import ProcessPoolExecutor to fix bug in python 3.9+ that causes
Exceptions to be raised incorrectly in multi-threaded applications
(https://bugs.python.org/issue42647)
* Removed extraneous whitespace.
* NIFI-12767:
- Error handling in Provenance and Lineage.
* NIFI-12767:
- Addressing review feedback.
* NIFI-12767:
- Restoring empty initial state for completed request that allows showing a banner error over an empty table instead of an empty page.
* NIFI-12767:
- Restoring error state to ensure that the loading spinner stops on error.
This closes#8386
- Refactored WebSocket Test Listener to avoid method visibility problem
This closes#8374
Co-authored-by: David Handermann <exceptionfactory@apache.org>
Signed-off-by: David Handermann <exceptionfactory@apache.org>
* [NIFI-12665] - fetch parameter provider parameters
* add permissions checks to parameter provider table actions for edit, delete, and fetch.
* error handling
* routing to fetch parameter provider dialog
* list the parameter groups
* parameter sensitivity selction/deselection
* show banner error if any affected/referencing components are not readable and writeable
* add indicators to parameters for changed, new, removed, ...
* refactored parameter-references component to the common area. leveraged it in the fetch dialog.
* validate the fetch form
* submit the fetch parameter provider parameters
* make the async update step completion icon color theme-aware
* add missing license header
* fixes for the initial round of review comments
* fixing issues found in review
* fix registry clients test
* stop polling when there is an api error
* use sort and join pipes in a couple of more places.
* protect references to parameter provider in the context for read permissions
* when full screen error is triggered, close any open dialog with the 'ROUTED' result to prevent unintended afterClosed actions taking place (like re-selection)
* handle fetch parameter provider error
* remove TODO comments
* call fullScreenError correctly
This closes#8367
- Error handling in Users.
- Error handling in Summary.
- Error handling in Queue Listing.
- Addressing review feedback.
- Dispatching delete success to ensure the active request is reset upon completion.
This closes#8366
Fixed a threading bug where Java makes a call to Python, Python create a Java object to return (ArrayList), Python returns the object and then cleans it up, notifying Java to remove it from 'bound objects', and then Java process the response and returns a null object because it's no longer bound
This closes#8356
Signed-off-by: David Handermann <exceptionfactory@apache.org>
RuleViolations (these objects only reside in memory only) now contain the type of the component that is responsible for the violation. This is used in StandardNiFiServiceFacade to fix and improve the authorization logic.
This closes#8318
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- PutDatabaseRecordIT supports operating systems with either nanosecond or microsecond precision
This closes#8332
Signed-off-by: David Handermann <exceptionfactory@apache.org>
* [NIFI-12748] apply panelClass to snackbars to apply lighter color to the snackbar action text
* hover color for snackbar action button
* updates based on https://github.com/angular/components/issues/11426
This closes#8365
* NIFI-12734:
- Import from Registry.
* NIFI-12734:
- Providing better guidance when there are no registry clients available based on user permissions.
- Showing form validation errors when there are no buckets or no flows.
This closes#8354
This closes#8350
The processor can now take a filename encoding parameter and pass it to zip unpacking. This will allow
user to unzip files with specific encoding to get correct filenames in output.
This for example help with zip files created on Windows which by default uses Cp437 for filename encoding.
If the filename contains special character like German alphabet ä, ü etc., decoding this with Linux's
default encoding usually UTF8 output will contain `?` in it. When the same file is processed with property
set with `Cp437`, the processor outputs correct filenames with special characters preserved.
Signed-off-by: Joseph Witt <joewitt@apache.org>
* [NIFI-12727] Detect theme based on OS setting, allow user override
* update storage service types, introduce theming service for user selection of theme settings
* review feedback, update menu option disaplay names, update storage service types, delete expired local storage items on init
* check for existence of window.matchmedia
* rebase and address review comments
This closes#8352
* NIFI-12737:
- Recording last canvas URL in local storage.
- Using last canvas URL in global menu Canvas item.
- Conditionally applying transitions when centering components.
- Always applying transitions during zoom events (1:1, fit, zoom in/out).
- Adding support to center more than one component.
* NIFI-12737:
- Fixing bug when attempting to click on the search result of the currently selected component.
- Handling centering of a single selection different from a bulk selection as it performs betters with Connections.
This closes#8355
- Updating property editing so that Shift-Enter inserts a new line and Enter commits the edit.
- Fixing bug that prevented deleting a Property and then re-adding a new Property with the same name.
This closes#8358
- In order to accommodate this, also needed to improve DataTypeUtils so that it knows that Record A is wider than Record B if Record A contains all fields of Record B and more.
- Removed unit tests and resources that are overly complex and no longer applicable
- Fixed issue in unit test based on different line endings between operating systems
This closes#8331
Signed-off-by: David Handermann <exceptionfactory@apache.org>
* [NIFI-12698] dark mode, [NIFI-12687] status history axis label fix
* prettier
* lint fix
* update input form field styles to support dark mode
* adjust colors of flow status icons, and PG status icons, and softened the warn palette. Fix background of status history charts in dark mode
* fix provenance text color in dark mode
* leave the toggle for dark mode always visible to encourage testing
This closes#8320
Fixed bug that caused custom Relationships not to work on Python Processors. Added unit test to verify. Also addressed issue in PythonControllerInteractionIT where it did not wait for Processors to become valid (originally this wasn't necessary but when we refactored Processors to initialize in the background this was overlooked).
This closes#8316
Signed-off-by: David Handermann <exceptionfactory@apache.org>
* NIFI-12684:
- Handling error responses in the service listing on the canvas.
- Handling error responses when converting a Property value to a Parameter.
* NIFI-12684:
- Introducing parameter helper to remove duplicated logic and error handling when fetching parameters and converting properties to parameters.
This closes#8321
valid because it had cached property descriptors before the processor was fully initialized, so updated code to ensure that we do not cache values before initialization is completed.
This closes#8315
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Added Spring Security PKCE OAuth2 Request Customizer to standard Resolver
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#8324.
* [NIFI-12686] eslint configuration for ts files
* add linting to the default maven build
* updates for RAT plugin
* remove .eslitrc as it wasn't needed. updated RAT config
This closes#8313
* [NIFI-12655] theme nifi following Material Design and leveraging Angular Material themes
* update accent palette usage to $nifi-canvas-accent-light-palette
* fix layout of create process group dialog
This closes#8294
* NIFI-12663:
- Handling API error responses in the Management Controller Services page.
* NIFI-12679:
- Renaming components based on review feedback.
- Using ng-content in page-content component.
- Removing the problematic route when navigating to the error page.
- Fixing logic when handling service loading errors.
- Handling errors in the Property Table Helper service.
* NIFI-12679:
- Addressing review feedback.
Refactored to avoid streaming over the list twice.
Tweaked unit test, so it works in NiFi 1.x also.
Signed-off-by: Matt Burgess <mattyb149@apache.org>
This closes#8266
* [NIFI-12622] - Parameter Providers - listing table
* Add a new Parameter Provider
* Delete parameter provider
* refactor to reduce duplicate code when creating new properties
* support editing parameter providers
* refactored inline service creation into PropertyTableHelperService
* added parameter provider linking to access policies.
* fix bugs in flow reducer not setting state properly when starting/stopping components and run once.
* address review feedback
* refactored nifiCommon to provide stripProtocol method that was implemented in loads of places. replaced all occurrences to the nifiCommon implementation.
This closes#8298
The empty string allows controller service configuration to be saved without flow
configuration history recording a configuration change for comments.
This closes#8275
Signed-off-by: David Handermann <exceptionfactory@apache.org>
This closes#8063
Signed-off-by: David Handermann <exceptionfactory@apache.org>
Co-authored-by: Vuong Nguyen Van <vanvuong24072001@gmail.com>
Co-authored-by: nathluu <luuhoangtanbis@gmail.com>
- Updated KubernetesConfigMapStateProvider and KubernetesLeaderElectionManager to ignore blank prefix values as provided in default configuration files
- Added some Use Case docs for Python processors and updated Runtime Manifests to include Python based processors as well as Use Case/MultiProcessorUseCase documentation elements. Refactored/cleaned up some of the Python code and added unit tests.
- Added python-unit-tests profile and enabled on Ubuntu and macOS GitHub workflows
This closes#8253
Signed-off-by: David Handermann <exceptionfactory@apache.org>
This closes#8013
Signed-off-by: David Handermann <exceptionfactory@apache.org>
Co-authored-by: Peter Turcsanyi <turcsanyip@apache.org>
Co-authored-by: Malthe Borch <mborch@gmail.com>
When syncing to a VersionedFlow, make sure processor references to new controller services are valid after the processor migrates the configuration of its properties
This closes#8184
Signed-off-by: David Handermann <exceptionfactory@apache.org>
* NIFI-12604:
- Empty Queue.
- Empty All Queues.
* NIFI-12611:
- Ensuring dialog is closed when an error occurs.
- Migrating from withLatestFrom to concatLatestFrom.
* NIFI-12611:
- Adding header/nav to the controller service listing page.
This closes#8246
Used migrateProperties() for migrating old flows to the new property structure.
Moved common properties to AzureStorageUtils and also updated/consolidated some property descriptions
This closes#8205
Signed-off-by: David Handermann <exceptionfactory@apache.org>
* [NIFI-12568] - start/stop/run once from the context menu. start/stop from operation panel.
* remove excess arguments from menu condition and action definitions
* review feedback: * reload flow if current pg is started/stopped. * reload pg if pg is started/stopped.
* another round of review feedback: catchErrors, filter order, reuse Revision interface from shared, unused imports
* renaming action and related interfaces from reloadProcessGroup to loadChildProcessGroup
* remove commented out code
* remove extranious calls to canOperate
* update conditions when start/stop/enable/disable are presented to the user
* fix vertical alignment for some icons in context menu
* more icon alignment
* review feedback
* updates
This closes#8216
* NIFI-12586:
- Fixing the handling of getParameters since the supplied input could be undefined in some instances.
* NIFI-12586:
- Removing unused event.
- Removed iotdb-server dependencies and integration tests based on banned SNAPSHOT dependency versions
Signed-off-by: Matt Burgess <mattyb149@apache.org>
This closes#8207
- Upgraded Spring Framework from 5.3.31 to 6.0.15
- Upgraded Spring Security from 5.8.7 to 6.2.0
- Upgraded Spring Vault from 2.3.4 to 3.1.0
- Upgraded Jetty from 10.0.18 to 12.0.5 with EE 10
- Upgraded Jersey from 2.41 to 3.1.4
- Upgraded JAXB from 2.3.9 to 4.0.4
- Upgraded AspectJ from 1.9.20.1 to 1.9.21
- Upgraded JMS API from 2.0.1 to 3.1.0
- Upgraded ActiveMQ Broker from 5.18.2 to 6.0.1 for JMS 3
- Upgraded JJWT from 0.9.1 to 0.12.3
- Replaced jackson-module-jaxb-annotations with jackson-module-jakarta-xmlbind-annotations
- Replaced maven-jaxb2-plugin with hisrc-higherjaxb40-maven-plugin 2.1.1
- Replaced kongchen swagger-maven-plugin with swagger-codegen-maven-plugin from Swagger 3
- Replaced com.nickwongdev AspectJ Plugin with Codehaus 1.14.0 for newer Java versions
- Removed unused cglib-nodep
- Removed references to javax.validation
- Removed custom Jetty ALPN Processor not required for Java 21
- Removed several tests depending on older Jetty and Jakarta libraries
- Removed unnecessary webdefault.xml configurations
- Replaced unsupported cross-context servlet forwarding with HTTP forwarding
- Replaced javax.servlet references with jakarta.servlet
- Replaced javax.xml.bind references with jakarta.xml.bind
- Replaced javax.ws references with jakarata.ws
- Updated Spring Security CSRF implementation for Spring Security 6
- Updated web.xml versions to 6.0
- Updated REST API templates using new swagger-codegen variables
- Removed VALIDATE_DATA property from ParseCEF based on library compatibility issue with javax.validation
- Added application URL logging to NiFi JettyServer
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#8197.
* [NIFI-12552] - Support client-side pagination on Summary tables
* updated the filter control to only show the label indicating matches when a filter is active. similar to how it is done for the events table
* aligned cells like Run Status so the icon and text are vertically centered in the table cell
* moving System Diagnostics out from the Summary page to the main application menu
* reset summary listing table selection when pagination or filter changes.
* remove area chart icon from the System Diagnostics menu option as it duplicates the Node Status History option
* refactor selection clearing to its own action
This closes#8192
* [NIFI-12553] - Context menu option for View Status History
* Added View Status History to the canvas context menu for applicable component types
* Added support for Node Status History from the flow menu
* remove unused imports
This closes#8193
- Removed Bouncy Castle JDK 1.5 version exclusions no longer required in SSHJ 0.38.0
Signed-off-by: Matt Burgess <mattyb149@apache.org>
This closes#8200
* NIFI-12543:
- Users/User Groups.
* NIFI-12543:
- Users/User Groups Deletion.
- Establishing routes for selection, editing, and access policies.
* NIFI-12543:
- User/User Group Creation.
- User/User Group Configuration.
- Renaming existing User State to Current User State.
* NIFI-12543:
- User access policies table.
* NIFI-12543:
- Sorting users/groups in the edit dialog.
* NIFI-12543:
- Addressing review feedback.
This closes#8191
- The isJson Expression Language function returns true regardless of leading or trailing spaces
This closes#8165
Signed-off-by: David Handermann <exceptionfactory@apache.org>
* NIFI-12529:
- Enable/Disable Controller Services.
- Resetting state when destroying Controller Service listings, General, Registry Clients, and Reporting Tasks.
* NIFI-12529:
- Addressing review feedback.
This closes#8185
- Set the width of the Property editor. This ensures the width looks correct and the overflow scroll policy works as expected.
- Fixed table layout issues where width: 100% from listing table wasn't correct.
This closes#8188
The UriUtils class allows the construction of a valid java.net.URI with a single string even though there maybe illegal characters in the path, query and/or fragment sections(s) of the URI. The create method uses regular expressions from Spring Framework UriComponentsBuilder and provides capabilities closer to the deprecated constructor for java.net.URL.
This closes#8189
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Added nifi-json-schema-api to nifi-commons
- Added StandardJsonSchemaRegistry implementation of JsonSchemaRegistry
- Added strategy configuration properties to ValidateJson
This closes#8005
Signed-off-by: David Handermann <exceptionfactory@apache.org>
* [NIFI-12504] process group summary tab
* ellipsify data/headers in table
* input and output port summary tabs
* connections summary tab
* rpg summary tab
* address review feedback including a filtering approach that doesn't rely on splitting strings by some arbitray character.
* remove unnecessary try/catch blocks
This closes#8174
* [NIFI-12519] - deselect parameter when it is deleted. also, enable goto controller service linking
* Added confirmation dialog in controller service edit when the form is dirty and the user attempts to route away using a referencing component link
* Reset state when closing the summary and counter pages.
This closes#8166
* NIFI-12502:
- Go To Service.
- Save before navigating if form dirty.
- Go To Parameter.
- Convert To Parameter.
* NIFI-12502:
- Ensuring tests are still bootstrapped correctly.
* NIFI-12502:
- Addressing review feedback.
This closes#8159
- Updated OidcBearerTokenRefreshFilter to maintain current Identity Provider Groups when generating refreshed application Bearer Tokens
- Refactored LoginAuthenticationToken to remove unnecessary optional constructors and use java.time.Instant for expiration
- Added Issuer Provider with implementation for Bearer Token Issuer based on host and port properties
- Updated Expression Language Guide to note the available system property providing the current framework version according to the Framework NAR version
This closes#8101
Signed-off-by: David Handermann <exceptionfactory@apache.org>
* [NIFI-12437] - Summary
* Processors Status Snapshot Listing
* initial processors status snapshot table
* sorting
* goto processor
* multi-valued sort for processors status listing summary
* add filtering to the processors status snapshot tab of the summary
* created a re-usable summary-table-filter componennt
* moved status history to common location
* status history
* status history chart
* resize
* display insufficient data message if there isn't enough data to render the history
* moved status history chart into its own component
* update missing licenses
* review feedback
* removing use of <label> for non-form elements in status-history component, also updated vertical spacing
* review feedback
* remove unused items from processor-status-listing.component.ts
* fixed tests. added npm script
* fixed routing to processor after initial load of the processors summary table
* turn of debug route tracing
This closes#8143
The headers in the map that come back when replicating a request used to be in the case given; however they can be lowercased in the OkHttp Headers object when using HTTP/2 instead of HTTP/1.1. As a result, we need to ensure that we always use lower-case header names or check the map case-insensitive.
This closes#8163
Signed-off-by: David Handermann <exceptionfactory@apache.org>
* NIFI-12486:
- Registry Clients.
- General authorization guard.
- Additional authorization checks in the existing Settings tabs.
* NIFI-12486:
- Adding authorization guard to /counters.
* NIFI-12486:
- Enabling some debug build out to attempt to track down a sporadic build failure.
* NIFI-12486:
- Addressing review feedback.
* NIFI-12486:
- Fixing unit test and running prettier.
This closes#8142
exceptionfactory
Scott Aslan
Matt Gilman
Mark Payne
dan-s1
Rob Fellows
Pierre Villard
Alex Ethier
Peter Turcsanyi
lehelb
tpalfy
Bence Simon
Rajmund Takacs
Umar Hussain
mr1716
Matt Burgess
Mark Bean
Balázs Gerner
EndzeitBegins
jsteinebrey
Nissim Shiman
dependabot[bot]
Freedom9339
nathluu
Timea Barna
Juldrixx
Mike Moser
Tanmaya Panda
Mark Bathori
NissimShiman
p-kimberley
tpalfy
krisztina-zsihovszki
annanys23
Mikhail Sapozhnikov
Emilio Setiadarma
Peter Gyori