- Updated Admin Guide with section on Logging Configuration and Deprecation Logging
- Updated Developer Guide with section on Deprecating Components and Features
- Added Deprecation Logging when adding components marked with DeprecationNotice to the Flow Configuration
- Added Deprecation Logging on deprecated methods in standard Controller Service implementations
- Removed integration tests for deprecated PersistentProvenanceRepository
- Updated logging and added documentation on minor upgrade steps
- Updated logging for HDFSNarProvider and RocksDB Repository
This closes#6390
Signed-off-by: Paul Grey <greyp@apache.org>
- Updated standard user interface error handler to clear the Access-Token-Expiration item from Session Storage when receiving an error with the WWW-Authenticate Header indicating a problem with the Bearer Token
Signed-off-by: Nathan Gough <thenatog@gmail.com>
This closes#6334.
- Added nifi-deprecation-log module with interface and implementation using SLF4J
- Updated standard logback.xml with nifi-deprecation.log appender
- Updated NiFiLegacyCipherProvider with deprecation logging
- Set Size, Time Policy, and Total Size Limit for Deprecation Log
This closes#6300
Signed-off-by: Paul Grey <greyp@apache.org>
Fixed bug where ProcessGroup would inadvertently set the wrong component's Versioned Component ID to null when there was an ID conflict
This closes#6314
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Added managed dependency in root Maven configuration
- Removed different versions from other Maven configurations
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#6284.
- Appended root path to Cookie path attribute when removing Bearer Tokens as part of unauthorized response handling
- Updated Saml2AuthenticationSuccessHandler to follow standard Cookie path building strategy
Signed-off-by: Nathan Gough <thenatog@gmail.com>
This closes#6278.
NIFI-10277 Changed failure test to throw SocketException
NIFI-10277 Increased timeout to 30 seconds and moved verify method
Signed-off-by: Joe Witt <joewitt@apache.org>
- Added Standard AuthenticationEntryPoint
- Configured AuthenticationEntryPoint for SecurityFilterChain and BearerTokenAuthenticationFilter
Signed-off-by: Nathan Gough <thenatog@gmail.com>
This closes#6233.
- Migrate tests in nifi-framework-mark-loading-utils to JUnit5
- Annotate tests that use x86_64 native binaries to be conditional on x86_64 os.arch
This closes#6215
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Removed extension of deprecated WebSecurityConfigurerAdapter
- Moved Filter bean configuration associated configuration classes
- Set default Spring Security log level to INFO
- Adjusted CSRF Token Repository to leverage simplified RequestUriBuilder for retrieving allowed context paths
Signed-off-by: Nathan Gough <thenatog@gmail.com>
This closes#6195
- Bump version to 6.29.5 for arm64 compatibility
- Extract RocksDBFlowFileRepository and supporting code to its own module
- Mark RocksDBFlowFileRepository as deprecated, both in code and documentation
- Log deprecation warning at startup if RocksDBFlowFileRepository is used
- Move native RocksDB info logs to NiFi debug level logs
This closes#6155
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Corrected handling of documentation for allowable values
NIFI-10162 Updated Multipart Form-Data Name description
This closes#6163.
Signed-off-by: Kevin Doran <kdoran@apache.org>
- Increased bootstrap heap size from 24 MB to 48 to avoid heap constraints when using HTTP notifications
This closes#6161
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Updated SAML Authentication Configuration with Spring Security SAML 2 components
- Updated Administration Guide with REST Resources
- Replaced SAMLAccessResource methods with applicable Spring Security Filters
- Removed IDP Credential Service and supporting components
- Removed message.logging.enabled, metadata.signing.enabled, and signature.digest.algorithm properties
- Added Access Token Expiration resource method
- Removed Saml2AccessResource and replaced with Access Token Expiration to avoid unnecessary conflicts with SAML login consumer
- Corrected Resource URI handling to support proxy server access
Signed-off-by: Nathan Gough <thenatog@gmail.com>
This closes#6149.
- Removed direct dependency on metadata-extractor to inherit newer transitive version
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#6140.
- Implemented MiNiFi C2 client using OkHttp
- Refactored MiNiFi bootstrap command handling and socket communication
- Added C2 Client Service to nifi-framework-core
This closes#6075
Co-authored-by: Matthew Burgess <mattyb149@apache.org>
Co-authored-by: Csaba Bejan <bejan.csaba@gmail.com>
Co-authored-by: Ferenc Erdei <ferdei@cloudera.com>
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Added nifi.web.https.application.protocols property
- Set default protocol to HTTP/1.1 and provided documentation for enabling HTTP/2
- Changed StandardALPNProcessor handshakeFailed log to debug
Signed-off-by: Joe Gresock <jgresock@gmail.com>
This closes#6093.
* NIFI-9960 Added documentation for Sensitive Dynamic Properties
- Updated User Guide with Add Property details
- Updated Developer Guide with annotation configuration details
- Updated Document Writer to indicate Supports Sensitive Dynamic Properties status
* NIFI-9960 Adjusted User Guide wording based on feedback
* NIFI-9959 Added UI Support for Sensitive Dynamic Properties
- Added SupportsSensitiveDynamicProperties to DBCPConnectionPool and ScriptedReportingTask
* NIFI-9959 Added sensitive parameter argument for Controller Service descriptors
* NIFI-9959 Adjusted sensitive property descriptor handling to support changing status
* NIFI-9959 Added info icon for Sensitive Value field
* NIFI-9959 Corrected handling of descriptor for existing dynamic properties
* NIFI-9959 Cleaning up dialog markup.
Co-authored-by: Matt Gilman <matt.c.gilman@gmail.com>
This closes#6073
* NIFI-9958 Implemented Sensitive Dynamic Properties
- Added SupportsSensitiveDynamicProperties annotation for components
- Added optional sensitive query parameter to Property Descriptor REST Resource methods
- Added system tests for components supporting sensitive dynamic properties
- Updated REST Resources to support Sensitive Dynamic Property Names
- Updated Documentation Writer to indicate component support for Sensitive Dynamic Properties
- Updated InvokeHTTP to support Sensitive Dynamic Properties
- Updated Auditor components to handle masking Sensitive Dynamic Properties
* Refactored Property Descriptor REST method handling
- Corrected AbstractDocumentationWriter evaluation of support for sensitive dynamic properties
- Refactored Controller Service Dynamic Properties system tests to new class
* Updated AbstractComponentNode.getProperties() to get canonical descriptor
* NIFI-10001: Fixed issue in which some components may fail to update the scheduled state when comparing flows
* NIFI-10001: Fixed bugs that caused some components to not have their scheduled state updated. When comparing two flows, now allow specifying how to determine a VersionedComponent's ID for comparison. When comparing local flow against flow from registry, use Versioned Component ID. But when comparing two instantiated flows, such as local flow vs. cluster flow, use the VersionedComponent's Instance ID instead. This ensures that we can properly compare two components even if there are several instances of a given flow
- Added Map and Set Cache Servers based on nifi-event-transport components
- Removed custom servers and unused socket stream components
- Reduced duplication on protocol classes
- Added checks for readable bytes
- Added mark and reset handling for buffer reads
This closes#6040
Signed-off-by: Paul Grey <greyp@apache.org>
* NIFI-9895 Allow parameter to reference controller service. Check read and write authorization for both previous and newly set controller service. Authorization done for both property or parameter change. Import/export handled by switching between instance id and versioned id.
* NIFI-10001: When enabling a collection of Controller Services, changed logic. Instead of enabling dependent services and waiting for them to complete enablement before starting a given service, just start the services given. The previous logic was necessary long ago because we couldn't enable a service unless all dependent services were fully enabled. But that changed a while ago. Now, we can enable a service when it's invalid. It'll just keep trying to enable until it becomes valid. At that point, it will complete its transition from ENABLING to ENABLED.
* NIFI-10001: Restored previous implementation for StandardControllerServiceProvider, as the changes were not ultimately what we needed. Changed StandardProcessGroup to use a ConcurrentHashMap for controller services instead of a HashMap with readLock. This was causing a deadlock when we enable a Controller Service that references another service during flow synchronization. Flow Synchronization was happening within a write lock and enabling the service required a read lock on the group. Eventually the thread holding the write lock would timeout and release the write lock. But this caused significant delays on startup. By changing to a ConcurrentHashMap, we alleviate the need for the Read Lock. Also noticed in testing that the StandardNiFiServiceFacade did not save flow changes when enabling dependent services so added call to controllerFacade.save().
- Removed version declarations from multiple modules
- Adjusted PutDynamoDB to remove catch for IOException not thrown in Commons IO 2.11.0
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#6015.
- Replaced nifi-framework-bundle managed dependency to root managed dependency
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#6005.
- Removed unnecessary references to jackson.version property
- Removed unnecessary dependency management references to Jackson libraries
This closes#5992
Signed-off-by: Mike Thomsen <mthomsen@apache.org>
* NIFI-9883 Refactored property protection to isolated ClassLoader
- Added nifi-property-protection-loader for abstracting access to implementation classes using ServiceLoader
- Updated Authorizer and Login Identity Provider configuration using isolated ClassLoader
- Updated NiFi Properties Loader using isolated ClassLoader
- Updated nifi-assembly to place property protection dependencies in lib/properties directory
- Updated and refactored unit tests
- Corrected LoginIdentityProviderFactoryBean getObject() Type
- Changed log methods that accept a String and Object array to replace the last Throwable argument with a formatted summary of causes when calling LogRepository.addLogMessage()
Signed-off-by: Joe Gresock <jgresock@gmail.com>
This closes#5965.
- Refactored XML parsing to use providers from nifi-xml-processing
- Configured spotbugs-maven-plugin with findsecbugs-plugin in nifi-xml-processing
- Disabled Validate DTD in default configuration for EvaluateXPath and EvaluateXQuery
- Replaced configuration of DocumentBuilder and streaming XML Readers with shared components
- Removed XML utilities from nifi-security-utils
- Moved Commons Configuration classes to nifi-lookup-services
This closes#5962
Signed-off-by: Paul Grey <greyp@apache.org>
- Refactored SimpleProcessLogger to avoid sending stack trace causes to SLF4J Logger
- Refactored SimpleProcessLogger to minimize duplication of component message formatting
- Updated ConnectableTask logging to avoid repeating Throwable class in message string
- Refactored TestSimpleProcessLogger to improve coverage and confirm consistent argument handling
- Corrected handling of exception strings in argument arrays
- Replaced individual Azure dependencies with azure-sdk-bom in nifi-property-protection-azure
- Removed woodstox-core and stax2-api exclusions
- Refactored TemplateDeserializer test class to avoid dependency on specific XML implementation
This closes#5929
Signed-off-by: Mike Thomsen <mthomsen@apache.org>
* NIFI-9853: Refactored StandardProcessGroupSynchronizer to make use of State Lookups and Compoennt Schedulers to ensure that we properly synchronize states when starting up, when exporting flow definitions, and when importing Flow Definitions
* NIFI-9853: Fixed NPE
* NIFI-9847: Switched LifecycleState to use a WeakHashMap to track ActiveProcessSessionFactory instances, instead of a regular Set that removed the instance after calling onTrigger. This was necessary for processors such as MergeRecord that may stash away an ActiveProcessSessionFactory for later use, as we need to be able to force rollback on processor termination
* NIFI-9847: Fixed checkstyle violation
* NIFI-9787: Increasing visibility of selected slickgrid row with 1px borders on the top and bottom plus a slightly darker yellow background-color value.
* NIFI-9787: Added missing units to the padding values.
Merged #5857 into main.
- Added Ubuntu Zulu JDK 17 GitHub build
- Adjusted MiNiFi C2 FileSystemConfigurationCache test to avoid using environment variables
- Adjusted MiNiFi StatusLogger and StatusLoggerTest to avoid overriding private logger
- Adjusted failure reason attribute check in TestGetIgniteCache
- Adjusted TestRangerAuthorizer and TestRangerNiFiAuthorizer to avoid checking nested exceptions
- Adjusted encrypt-config TestUtil to avoid unnecessary comparison of different types
- Disabled Javascript tests on Java 15 and higher
- Disabled several Hive 3 tests on Java 17 for StringInternUtils illegal access
- Refactored nifi-enrich-processors tests to use Mockito without Powermock
- Refactored nifi-toolkit-tls tests to avoid illegal reflective access
- Removed deprecated X509Certificate test in CertificateUtilsTest
- Removed kryo serialization from nifi-site-to-site-client test
- Updated TestHashContent to use SHA-1 instead of SHA for hash algorithm
- Upgraded maven-war-plugin from 2.5 to 3.3.2
- Upgraded nifi-graph-bundle dependencies from Groovy 2.5.14 to 3.0.8
- Upgraded QuestDB from 4.2.1 to 6.2.1 in nifi-framework-core
This closes#5870
Signed-off-by: Mike Thomsen <mthomsen@apache.org>
* NIFI-9761 Corrected PeerChannel processing for TLS 1.3
- Added TestPeerChannel with methods for TLS 1.2 and TLS 1.3
- Updated PeerChannel.close() to process SSLEngine close notification
- Improved logging and corrected handling after decryption
Relaxing MockPropertyValue validation to allow for variables to be passed to config verification
Fixing underlying framework issue with config verification: wrong variable registry was being used
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#5843
* Update nf-context-menu.js
Include enable/disable all controllers menu item
* Update nf-actions.js
Create enableAllControllers and disableAllControllers actions
* Rename ...Controllers to ...ControllerServices
* Rename *controllers to *controller services
* Adjusts proposed by reviewer.
* Refactor enable and disable AllControllerServices
This closes#5247
- Updated stateless and StandardProcessGroup, etc. to make use of VersionedExternalFlow
- Updated StatelessDataflowDefinition to use ExternalVersionedFlow instead of generic type
- Updated Stateless Bootstrap to avoid loading stateless engine libs from root class path but instead use a NarClassLoader to load the statelss nar
Signed-off-by: Joe Gresock <jgresock@gmail.com>
This closes#5832.
* NIFI-9543: add bring-to-front functionality to labels
* NIFI-9543: update zIndex changes to persist after shutdown
* NIFI-9543: remove commented code
* NIFI-9543: update flow comparator and snippet to account for zIndex
* NIFI-9543: changed add function to sort labels when a new label is added
* NIFI-9543: remove unnecessary call to position
* NIFI-9543: only register difference if zIndex is not default value
* NIFI-9543: add null checks to prevent primitive Long
This closes#5765
NIFI-9425: Added autoload directory property and defensive code for stop() called twice
This closes#5762.
Signed-off-by: Kevin Doran <kdoran@apache.org>
* NIFI-9580: initial ui work for framework-level retry in processors
* NIFI-9580: disable retry controls when retry not selected
* NIFI-9580: refactor marshalling to be one function
* NIFI-9580: style tweaks and documentation update
* NIFI-9580: update tooltip content
* NIFI-9580: change tooltip copy for clarity
* NIFI-9580: only submit retry configs if selected
* NIFI-9580: add relationship details to read-only dialog
* NIFI-9580: change styling based on feedback
This closes#5666
NIFI-9731: Only simplify flowfile arg in bulletin after extracting the flowfile uuid
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#5808
- Added nifi-h2-database module shading H2 1.4.200
- Implemented version checking and migration
This closes#5724
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Moved commons-lang3 version management to root Maven configuration
- Refactored limited usage of StringUtils is nifi-reporting-utils to remove commons-lang3
- Refactored limited usage of StringUtils in nifi-websocket-processors to remove commons-lang3
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#5773
The NiFi and NiFi Bootstrap processes both bind to random ephemeral
ports to allow for inter-process communication (e.g. shutdown, port,
ping, etc.). However, the randomness of these ephemeral ports can pose
challenges for some security policies and firewall rules.
This adds two configuration options, nifi.bootstrap.listen.port and
nifi.listener.bootstrap.port, that allow an administrator to define
which ports the two processes should bind to for this communication,
making it easier to define security policies. The options default to
zero to maintain the current ephemeral port behavior.
NIFI-6740: Add configuration options to specify NiFi/Bootstrap communication ports
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#5746
update lintin
update font awesome path
bump nifi-fds version
update nifi deps
update frontend maven plugin
change CI npm cache restore keys so that we will not use cached directories for older versions of node
This closes#5751.
Signed-off-by: Kevin Doran <kdoran@apache.org>
- Upgraded tika-core from 1.27 to 2.3.0
- Upgraded tika-parsers to tika-parsers-standard-package in nifi-media-processors
- Updated Tika metadata property references
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#5754.
* NIFI-9628: Added a uiOnly flag when requesting Controller Service details and the list of Controller Services. This allows us to return much less data when retrieving these resources.
* NIFI-9628: Addressed review feedback; added uiOnly flag for controller service run-status and references also
* NIFI-9628: Fixed checkstyle issues by removing unused imports
This closes#5712
- Added okhttp-bom to root Maven configuration
- Removed repeated okhttp.version property from specific dependencies
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#5737.
- Refactored nifi-framework and nifi-standard modules
- Replaced Google Cache with Caffeine Cache
- Replaced Google collections classes with standard Java collections
This closes#5730.
Signed-off-by: Kevin Doran <kdoran@apache.org>
- ids of the entities are prefixed by their types to avoid id conflict in 3rd party table library
For some authorizers the UUID calculation for users and groups based only on the name of the entity and a global seed.
This results the same UUID for a group and a user with the same name. The same ids are no longer causing any issue on the UI.
- group icon alignment is fixed for edit user dialog
This closes#5695
- Corrected several missing return statements
- Increased Maven heap size to 3 GB for ci-workflow
This closes#5700
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Switch ExtensionManifestParser implementation to use JAXB instead of Jackson XML
- Fix handling of older NARs that don't have newer fields in ExtensionManifest
- Add response merging for runtime-manifest REST end-point, remove commons-lang3 from c2 dependencies
This closes#5630
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- 'Leave group' action button is added to 'Navigation'
- 'Leave group' action works with 'esc' hotkey if no modal, context menu, etc. is open
- 'esc' key closes context menu if it is open
- user guide is updated with new navigation options
This closes#5678
- Added DataTransferDoSFilter with request URI evaluation
- Added RequestFilterProvider and implementations to abstract Jetty Filter configuration
Signed-off-by: Joe Gresock <jgresock@gmail.com>
This closes#5670.
Joe Gresock
exceptionfactory
Csaba Bejan
mr1716
Peter Turcsanyi
Bence Simon
Mark Payne
Nathan Gough
Lehel Boér
Matthew Burgess
Hsin-Ying Lee
Malthe Borch
Emilio Setiadarma
Joe Witt
Kevin Doran
Paul Grey
Otto Fowler
Ferenc Erdei
timeabarna
Nissim Shiman
Bryan Bende
Mark Bean
Nandor Soma Abonyi
tpalfy
Chris Sampson
Matt Gilman
simonbence
greyp9
dependabot[bot]
Bence Simon
Hervé Boutemy
Jonathan Conti-Vock
Jon Conti-Vock
Mike Thomsen
Eduardo Fontes
Shane Ardell
Adam Kocsis
Roberto Marturano
Steve Lawrence
Scott Aslan
Timea Barna