1291 Commits

Author SHA1 Message Date
Joe Witt
0419dc2939
NIFI-9780-RC3 prepare for next development iteration 2022-03-21 11:58:27 -07:00
Joe Witt
b019a9191f
NIFI-9780-RC3 prepare release nifi-1.16.0-RC3 2022-03-21 11:58:09 -07:00
Mike Thomsen
2bb1a06a8c
NIFI-9788 Updated commons-codec to 1.15 across all modules.
This closes #5881

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-03-21 09:13:05 -05:00
exceptionfactory
7823156606
NIFI-9815 Corrected log message formatting in multiple classes
This closes #5879

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-03-18 20:11:05 -04:00
exceptionfactory
412e21be97
NIFI-9281 Enabled building on Java 17
- Added Ubuntu Zulu JDK 17 GitHub build
- Adjusted MiNiFi C2 FileSystemConfigurationCache test to avoid using environment variables
- Adjusted MiNiFi StatusLogger and StatusLoggerTest to avoid overriding private logger
- Adjusted failure reason attribute check in TestGetIgniteCache
- Adjusted TestRangerAuthorizer and TestRangerNiFiAuthorizer to avoid checking nested exceptions
- Adjusted encrypt-config TestUtil to avoid unnecessary comparison of different types
- Disabled Javascript tests on Java 15 and higher
- Disabled several Hive 3 tests on Java 17 for StringInternUtils illegal access
- Refactored nifi-enrich-processors tests to use Mockito without Powermock
- Refactored nifi-toolkit-tls tests to avoid illegal reflective access
- Removed deprecated X509Certificate test in CertificateUtilsTest
- Removed kryo serialization from nifi-site-to-site-client test
- Updated TestHashContent to use SHA-1 instead of SHA for hash algorithm
- Upgraded maven-war-plugin from 2.5 to 3.3.2
- Upgraded nifi-graph-bundle dependencies from Groovy 2.5.14 to 3.0.8
- Upgraded QuestDB from 4.2.1 to 6.2.1 in nifi-framework-core

This closes #5870

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-03-17 14:08:38 -04:00
Nathan Gough
885c475f90
NIFI-9699 - Updated oidcCallback method to handle error cases. Added some unit tests.
This closes #5824

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-03-11 11:40:47 -06:00
Bryan Bende
546f986603
NIFI-9775 Create RuntimeManifestService
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5849.
2022-03-10 07:52:16 -05:00
Matt Gilman
227dcf5456
NIFI-9777: (#5850) - Adding support to remove attributes from verification requests.
This closes #5850
2022-03-08 16:25:32 -05:00
Joe Gresock
743020eeb4
NIFI-9762: Adding DBCPConnectionPool config verification
Relaxing MockPropertyValue validation to allow for variables to be passed to config verification

Fixing underlying framework issue with config verification: wrong variable registry was being used

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5843
2022-03-07 14:43:11 -05:00
Eduardo Fontes
d062029258
NIFI-8927 - Add option to start/stop all controllers (#5247)
* Update nf-context-menu.js

Include enable/disable all controllers menu item

* Update nf-actions.js

Create enableAllControllers and disableAllControllers actions

* Rename ...Controllers to ...ControllerServices

* Rename *controllers to *controller services

* Adjusts proposed by reviewer.

* Refactor enable and disable AllControllerServices

This closes #5247
2022-03-07 13:57:35 -05:00
Mark Payne
8959226b50
NIFI-9754: Introduced VersionedExternalFlow
- Updated stateless and StandardProcessGroup, etc. to make use of VersionedExternalFlow
- Updated StatelessDataflowDefinition to use ExternalVersionedFlow instead of generic type
- Updated Stateless Bootstrap to avoid loading stateless engine libs from root class path but instead use a NarClassLoader to load the statelss nar

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5832.
2022-03-04 07:19:38 -05:00
Shane Ardell
6cd30e3d0a
NIFI-9543: Add bring-to-front functionality to labels (#5765)
* NIFI-9543: add bring-to-front functionality to labels

* NIFI-9543: update zIndex changes to persist after shutdown

* NIFI-9543: remove commented code

* NIFI-9543: update flow comparator and snippet to account for zIndex

* NIFI-9543: changed add function to sort labels when a new label is added

* NIFI-9543: remove unnecessary call to position

* NIFI-9543: only register difference if zIndex is not default value

* NIFI-9543: add null checks to prevent primitive Long

This closes #5765
2022-03-03 18:50:15 -05:00
Adam Kocsis
7a16619969
[NIFI-9690] Display icon on Connection Label to show when head of the queue is penalized (#5812)
This closes #5812
2022-03-03 09:13:00 -05:00
exceptionfactory
d17fcf6427 NIFI-9735 Corrected Jetty Duplicate Mapping Warning
- Removed duplicate default servlet mapping from addDocsServlets()

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5817.
2022-03-01 15:49:23 -05:00
Shane Ardell
4719f78ef4
NIFI-9580: UI work for framework-level retry in Processors (#5666)
* NIFI-9580: initial ui work for framework-level retry in processors

* NIFI-9580: disable retry controls when retry not selected

* NIFI-9580: refactor marshalling to be one function

* NIFI-9580: style tweaks and documentation update

* NIFI-9580: update tooltip content

* NIFI-9580: change tooltip copy for clarity

* NIFI-9580: only submit retry configs if selected

* NIFI-9580: add relationship details to read-only dialog

* NIFI-9580: change styling based on feedback

This closes #5666
2022-03-01 14:57:18 -05:00
exceptionfactory
52f21390db
NIFI-9688 Improve Logback shutdown handling (#5805)
* NIFI-9688 Improved Logback shutdown handling
- Added environment variable to disable Logback shutdown on web application termination
- Added shutdown hook to Logback configuration
- Added Logback shutdown hook configuration to system tests
2022-02-28 14:29:44 -05:00
Mark Payne
3511ce3d13
NIFI-9548: When disabling RPG transmission, wait for the ports to complete in a background thread instead of blocking the web thread. Also moved the RPG initialization logic into flow controller instead of flow service and added a delay in order to reduce likelihood of ConnectException happening when pointing to nodes in the same cluster
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5641.
2022-02-23 11:29:26 -05:00
Nathan Gough
7ef2fd2986
NIFI-7333 Added OIDC trust store strategy property
This closes #5753

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-02-18 15:19:22 -05:00
Mark Payne
2aa6bd1e13
NIFI-9689: When all FlowFiles in a FlowFile Queue are penalized, do not schedule the destination to run. Also expose this fact via the ConnectionStatusSnapshotDTO, as this allows the front-end to render this information to the user in order to avoid confusion when it appears that the Processor has data but does nothing
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5771
2022-02-16 15:30:00 -05:00
timeabarna
63a543cd67
NIFI-9475 Provide Framework-Level Retries for NiFi Relationships (#5593) 2022-02-15 15:40:41 -05:00
tpalfy
84b2484fd4
NIFI-9684 Fix: When starting/stopping selected process group, it sends the parent process group id to the REST interface that is responsible to enable/disable transmission for all remote process groups within a process group. Need to send the id of the select process group instead. (#5764) 2022-02-15 14:55:17 -05:00
Scott Aslan
b8ec6d38dc [NIFI-9554] upgrade node and npm
update lintin
update font awesome path
bump nifi-fds version
update nifi deps
update frontend maven plugin
change CI npm cache restore keys so that we will not use cached directories for older versions of node

This closes #5751.

Signed-off-by: Kevin Doran <kdoran@apache.org>
2022-02-14 14:24:02 -05:00
exceptionfactory
e584d3cf04
NIFI-9660 Upgraded Apache Tika to 2.3.0
- Upgraded tika-core from 1.27 to 2.3.0
- Upgraded tika-parsers to tika-parsers-standard-package in nifi-media-processors
- Updated Tika metadata property references

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5754.
2022-02-09 11:11:46 +01:00
markap14
230ed9c98d
NIFI-9628: Added a uiOnly flag when requesting Controller Service det… (#5712)
* NIFI-9628: Added a uiOnly flag when requesting Controller Service details and the list of Controller Services. This allows us to return much less data when retrieving these resources.

* NIFI-9628: Addressed review feedback; added uiOnly flag for controller service run-status and references also

* NIFI-9628: Fixed checkstyle issues by removing unused imports

This closes #5712
2022-02-03 15:53:23 -05:00
exceptionfactory
43748a5523 NIFI-9638 Refactored Google Guava references
- Refactored nifi-framework and nifi-standard modules
- Replaced Google Cache with Caffeine Cache
- Replaced Google collections classes with standard Java collections

This closes #5730.

Signed-off-by: Kevin Doran <kdoran@apache.org>
2022-02-01 15:17:50 -05:00
Timea Barna
3ccc9d29b6
NIFI-9455 Added aggregated predictions to Prometheus Flow Metrics
This closes #5582

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-01-27 10:07:01 -06:00
Mark Payne
3ea9faccc6
NIFI-8040: When changing version of a flow, stop processors that have a state of Starting in addition to those with a state of Running
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5718.
2022-01-27 15:06:40 +01:00
Adam Kocsis
3eef7e0a3e
NIFI-9578 NiFi user list no longer breaks if a user and group have the same id (#5695)
- ids of the entities are prefixed by their types to avoid id conflict in 3rd party table library
   For some authorizers the UUID calculation for users and groups based only on the name of the entity and a global seed.
   This results the same UUID for a group and a user with the same name. The same ids are no longer causing any issue on the UI.
- group icon alignment is fixed for edit user dialog

This closes #5695
2022-01-25 14:52:07 -05:00
exceptionfactory
e2243500d3
NIFI-9618 Upgraded Checkstyle to 9.2.1
- Corrected several missing return statements
- Increased Maven heap size to 3 GB for ci-workflow

This closes #5700

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-01-21 20:08:16 -06:00
Bryan Bende
6ef6e71d05
NIFI-9501 Added REST end-point to retrieve a RuntimeManifest
- Switch ExtensionManifestParser implementation to use JAXB instead of Jackson XML
- Fix handling of older NARs that don't have newer fields in ExtensionManifest
- Add response merging for runtime-manifest REST end-point, remove commons-lang3 from c2 dependencies

This closes #5630

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-01-21 15:37:04 -06:00
Timea Barna
ccd47de6dc
NIFI-9587 Added JSON format for Prometheus Flow Metrics
This closes #5673

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-01-20 11:14:01 -06:00
Adam Kocsis
3c2793be09
[NIFI-8006] Additional options on UI to leave current process group (#5678)
- 'Leave group' action button is added to 'Navigation'
- 'Leave group' action works with 'esc' hotkey if no modal, context menu, etc. is open
- 'esc' key closes context menu if it is open
- user guide is updated with new navigation options

This closes #5678
2022-01-20 10:58:05 -05:00
exceptionfactory
fc27b3138b
NIFI-9481 Excluded Data Transfer REST methods from DoSFilter
- Added DataTransferDoSFilter with request URI evaluation
- Added RequestFilterProvider and implementations to abstract Jetty Filter configuration

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5670.
2022-01-19 16:11:53 -05:00
Joe Witt
2f95eed8cb
NIFI-9575 This closes #5667. updating copyright year to 2022
Signed-off-by: Joe Witt <joewitt@apache.org>
2022-01-19 09:14:09 -07:00
exceptionfactory
dd7131b257
NIFI-9435 Added registries and names include parameters to Flow Metrics
- Added optional includedRegistries query parameter to Flow Metrics Resource method supporting one or more registries
- Added optional includedNames query parameter to Flow Metrics Resource method supporting one or more metric family names
- Added sampleName and sampleLabelValue optional pattern parameters
- Added FilteringMetricFamilySamplesEnumeration to support streamed filtering
- Added PrometheusMetricsWriter and TextFormat implementation

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5571.
2022-01-13 15:59:00 -05:00
exceptionfactory
81e57c9651
NIFI-9535 Upgraded Spring Framework to 5.3.14
- Centralized Spring Framework and Spring Security versions using BOM dependencies
- Upgraded Spring Security from 5.5.2 to 5.6.1
- Upgraded Spring Boot from 2.5.5 to 2.5.8 in Registry
- Upgraded Spring Integration from 5.5.2 to 5.5.7

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5631.
2022-01-05 15:30:03 +01:00
exceptionfactory
7c6bdcb035
NIFI-9491 Excluded commons-logging and added jcl-over-slf4j references
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5608
2021-12-17 07:21:03 -05:00
Shane Ardell
293919f881
NIFI-9460: Update Lodash to 4.17.21 (#5605)
* NIFI-9460: update lodash transitive dep in registry

* NIFI-9460: use lodash instead of lodash-core

Merged #5605 into main.
2021-12-16 14:28:02 -08:00
M Tien
3d5f357de8
[NIFI-9423-NIFI-9429]: Show icon and tooltip for Parameters with leading and/or trailing whitespace (#5569)
* NIFI-9423 - Show icon and tooltip for Parameter values that have leading and/or trailing whitespaces

NIFI-9429 - Parameters should allow blank values that are non-null (only whitespace)

* - Update areas to clean up tooltips in parameter values
- Show whitespaces and ellipsis in parameter and property values and tooltips
- Update serializeValue to accommodate for blank values

* - Address review findings

* - Remove commented out code

* - Add multiline check for ellipsis

* NIFI-9459 - Empty string checked will disable Edit Parameter value field on dialog open

* - Add multi-line style to parameter and property table

* - Safely insert title attribute content

* - Fix Edit Parameter bug that clears textarea for sensitive and empty string values on dialog open

This closes #5569
2021-12-13 16:01:42 -05:00
Mark Payne
90b39b593a
NIFI-9069 Changed framework dataflow serialization to support JSON
- Changed framework so that it serializes the dataflow into a VersionedDataflow using JSON as well as XML, and prefers the JSON representation on load, if it's available. This also results in the need for the cluster protocol to exchange its representation of the dataflow to using JSON. Rather than re-implementing all of the complex logic of Flow Fingerprinting, updated to just inherit the cluster's flow.
- Moved logic to synchronize Process Group with Versioned Process Group into a new ProcessGroupSynchronizer class instead of having all of the logic within StandardProcessGroup
- Reworked versioned components to use an instance id.
- Renamed StandardFlowSynchronizer to XmlFlowSynchronizer; introduced new StandardFlowSynchronizer that delegates to the appropriate (Xml or Versioned)FlowSynchronzer
- Updated to allow import of VersionedProcessGroup even if not all bundles are available - will now use ghost components
- Introduced a VersionedDataflow object to hold controller-level services, reporting tasks, parameter contexts, templates, etc.
- Allow mutable requests to be made while nodes are disconnected. Also fixed issue in AbstractPolicyBasedAuthorizer that caused ClassNotFoundException / NoClassDefFoundError if the authorizations were changed and then a node attempts to rejoin the cluster. The Authorizer was attempting to use XmlUtils, which is in nifi-security-utils and so so by madking nifi-security-utils a provided dependency of nifi-framework-api, but this doesn't work, because nifi-framework-api is loaded by a higher-level classloader, so the classloader that loads AbstractPolicyBasedAuthorizer will never have the appropriate classloader to provide nifi-security-utils. Addressed this by copying the code for creating a safe document builder from XmlUtils to AbstractPolicyBasedAuthorizer.
- Fixed bug that occurred when importing a Process Group that has 2 parameter contexts, one inheriting from another, where neither is pre-defined in the existing flow
- Fixed bug that was encountered when Updating a Versioned Process Group where one version had a disabled processor and the other had the processor running.
- Increased system-tests workflow timeout to 120 minutes
- Added additional exception handling to system tests

This closes #5514

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-12-02 15:10:38 -06:00
M Tien
2b415de912
NIFI-9336 - Show icon for property values with leading or trailing whitespace (#5559)
* NIFI-9336 - Show icon in processor and controller services configurations when property values contain leading or trailing whitespace

* - Address PR feedback

* - Fix a bug to clean up tooltips to prevent a DOM leak

This closes #5559
2021-12-02 13:31:19 -05:00
exceptionfactory
6a6b755aaa
NIFI-9394 Removed RequestLogger and TimerFilter
- Removed logger references from logback.xml

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5543.
2021-11-29 12:55:39 -05:00
Joe Gresock
a5581562bf
NIFI-9416: Fixing NPE when updating param context without inheritedParameterContexts (#5553) 2021-11-29 11:29:33 -05:00
Joe Gresock
dfd8adfd8e
NIFI-9411: Adding missing 'setSubtitle' call for Controller Services modal
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5551.
2021-11-24 18:01:48 +01:00
markap14
839fbf7d19
NIFI-9382: Created a new ClassloaderIsolationKey mechanism by which H… (#5533)
* NIFI-9382: Created a new ClassloaderIsolationKey mechanism by which Hadoop related processors (and potentially others) can indicate that they need full classloaders to be cloned but can share with other instances in certain circumstances
- Added system tests

* NIFI-9382: Renamed interface based on review feedback

* NIFI-9382: Removed ReentrantKerberosUser.
2021-11-22 14:55:19 -05:00
Timea Barna
848a22eb29
Revert "NIFI-9385 Add Flow Metrics producer for Cloudera Manager"
This reverts commit 785021dade215fdbf9b6ffcce2817d4509cadeac.

This closes #5540

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-11-19 08:52:20 -06:00
Timea Barna
785021dade
NIFI-9385 Add Flow Metrics producer for Cloudera Manager
This closes #5528

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-11-18 11:51:53 -06:00
exceptionfactory
605346ae0f
NIFI-9291 Added NiFi HTTP request logging
- Added nifi.web.request.log.format property
- Added Filters to set and retrieve authenticated username for logging

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5527.
2021-11-18 10:30:50 -05:00
M Tien
5016eac010
NIFI-9205 - Update prioritizer configuration (#5516)
* NIFI-9205 - Update prioritizer configuration

* - Address review feedback

This closes #5516
2021-11-17 13:55:02 -05:00
noblenumbat360
f410c8df0a
NIFI-8272 Delete stale metrics from REST API Prometheus endpoint.
Added <scope>test</scope> tag to the nifi-web-api pom.xml and corrected imports.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5447
2021-11-08 20:26:30 -05:00