- Methods for getBranches, getDefaultBranch, and createBranch in flow registry client
- Add branch as a field to all relevant objects
- Add optional branch argument to REST API for retrieving buckets
- Add new REST API for retrieving branches for a given registry client
- Enforce NiFi Registry can only receive default branch
This closes#8726
Signed-off-by: David Handermann <exceptionfactory@apache.org>
Ensure nifi-py4j integration tests have proper logback libs
This closes#8747
Signed-off-by: Joseph Witt <joewitt@apache.org>
Signed-off-by: Chris Sampson <chris.sampson82@gmail.com>
Simplified dependency management into Bill of Material dependencies as parents that align to classloader structures.
- Moved nifi-framework-bundle to root level
- Moved nifi-nar-bundle modules to root nifi-extension-bundles
- Added nifi-extension-bom for internal and external extensions
- Added nifi-framework-nar-bom and nifi-server-nar-bom for managing framework dependencies
- Renamed nifi-jetty-bundle to nifi-jetty-nar
This closes#8677
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Updated user interfaces to support String version numbers
- Also updated new UI to clear out the buckets and flows when selections change 'above' them in the form when importing from registry
This closes#8674
Co-authored-by: Rob Fellows <rob.fellows@gmail.com>
Signed-off-by: David Handermann <exceptionfactory@apache.org>
* NIFI-11443 Routed Python Framework Logging to SLF4J
- Changed Python logging to use standard output stream
- Adjusted Python logging format for simplified processing
- Updated PythonProcess to pipe standard error and standard output streams to reader
- Added Log Reader command with Virtual Thread for each Python Process
- Removed Python log properties from NiFi Properties configuration
- Moved StandardValidators to nifi-api
- Moved URL creation method from UriUtils to URLValidator
- Separated FormatUtils into FormatUtils and DurationFormat classes
- Added DurationFormat to nifi-api
This closes#8442
Signed-off-by: David Handermann <exceptionfactory@apache.org>
Ensured that if a Process Group doesn't have a Versioned Component ID we use the ComponentIdLookup to create one based on its Instance ID in the same way that is done when serializing the flow; this ensures matching ID's when we synchronize flows across the cluster. Also included some code cleanup around failure handling on startup
This closes#8406
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Upgraded Spring Framework from 5.3.31 to 6.0.15
- Upgraded Spring Security from 5.8.7 to 6.2.0
- Upgraded Spring Vault from 2.3.4 to 3.1.0
- Upgraded Jetty from 10.0.18 to 12.0.5 with EE 10
- Upgraded Jersey from 2.41 to 3.1.4
- Upgraded JAXB from 2.3.9 to 4.0.4
- Upgraded AspectJ from 1.9.20.1 to 1.9.21
- Upgraded JMS API from 2.0.1 to 3.1.0
- Upgraded ActiveMQ Broker from 5.18.2 to 6.0.1 for JMS 3
- Upgraded JJWT from 0.9.1 to 0.12.3
- Replaced jackson-module-jaxb-annotations with jackson-module-jakarta-xmlbind-annotations
- Replaced maven-jaxb2-plugin with hisrc-higherjaxb40-maven-plugin 2.1.1
- Replaced kongchen swagger-maven-plugin with swagger-codegen-maven-plugin from Swagger 3
- Replaced com.nickwongdev AspectJ Plugin with Codehaus 1.14.0 for newer Java versions
- Removed unused cglib-nodep
- Removed references to javax.validation
- Removed custom Jetty ALPN Processor not required for Java 21
- Removed several tests depending on older Jetty and Jakarta libraries
- Removed unnecessary webdefault.xml configurations
- Replaced unsupported cross-context servlet forwarding with HTTP forwarding
- Replaced javax.servlet references with jakarta.servlet
- Replaced javax.xml.bind references with jakarta.xml.bind
- Replaced javax.ws references with jakarata.ws
- Updated Spring Security CSRF implementation for Spring Security 6
- Updated web.xml versions to 6.0
- Updated REST API templates using new swagger-codegen variables
- Removed VALIDATE_DATA property from ParseCEF based on library compatibility issue with javax.validation
- Added application URL logging to NiFi JettyServer
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#8197.
- Upgraded Spring from 5.3.30 to 6.0.13
- Upgraded Spring Boot from 2.7.16 to 3.1.5
- Upgraded Spring Security from 5.8.7 to 6.1.5
- Upgraded Jetty from 10.0.18 to 11.0.18
- Upgraded Servlet API from 3.1.0 to 5.0.0
- Upgraded JAX-RS from 2.1.1 to 3.1.0
- Upgraded Jersey from 2.41 to 3.1.3
- Upgraded JAXB from 2.3.2 to 4.0.4
- Upgraded Java Validation from 2.0.1 to 3.0.2
- Upgraded Flyway from 8.5.13 to 9.22.3
- Upgraded ASM from 9.3 to 9.6 for Java 21
- Upgraded NiFi Toolkit CLI to JAX RS 3 and Jersey 3
- Set Jersey 3 and JAX RS 3 dependency versions in nifi-flow-registry-client-bundle to align with updates to nifi-registry-client classes
Signed-off-by: Joseph Witt <joewitt@apache.org>
When calling migrateProperties, provide the properties that were configured in the VersionedComponent, rather than creating a new property map based on the component's new properties
This closes#7964
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Added nar-packaging and war-packaging profiles to root Maven configuration
- Removed individual properties from NAR module configurations
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#7947.
- Corrected system test failures for PythonProcessorIT and Stateless CreateFlowFileIT
This closes#7935
Signed-off-by: Chris Sampson <chris.sampson82@gmail.com>
- Added ability to get raw property values from PropertyConfiguration instead of just effective values
- Updated TestRunner to allow for testing these migration methods
- Auto-enable newly created controller services if they are valid
- Eliminated Proxy properties in all AWS processors and instead just make use of the Proxy Configuration controller service
- Eliminated authentication properties from AWS processors and migrated all processors to using Controller Service or authentication
This closes#7874
Signed-off-by: David Handermann <exceptionfactory@apache.org>
* NIFI-12206 Refactored Flow History using JetBrains Xodus
- Replaced H2 Database Engine with JetBrains Xodus for persistent storage of FlowConfigurationHistory
- Added EntityStoreAuditService implementation using Xodus PersistentEntityStore
- Removed nifi.h2.url.append from properties
- Ensure that after restoring nars in the lib/ directory we restart NiFi so that they take effect. This is important if this test is not the last one to run
This closes#7837
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Created NIFI-12203 to evaluate issues with flow comparison surfaced in JoinClusterWithDifferentFlow
This closes#7661
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Imported nifi-bom to nifi-commons and nifi-nar-bundles parent modules
- Removed explicit version and scope properties for dependencies declared in nifi-bom
Fixed issue in logic of LocalComponentLifecycle, which was waiting for all PGs to reach desired stateless run schedule, even when the group itself was not stateless
Fixed JavaScript to specify correct value for recursivity when updating process group parameter context
- Added nifi-security-cert for reusable components without dependencies
- Added nifi-security-cert-builder for certificate generation
Signed-off-by: Matt Burgess <mattyb149@apache.org>
This closes#7651
- Added new extensible Component Type: FlowAnalysisRule
- Added DisallowComponentType Rule implementation
- Flow Analysis Rules can be managed from the UI under Controller Settings -> Flow Analysis Rules
- Flow Analysis Rules can be set up with an enforcement policy of WARN or ENFORCE
- Flow Analysis Rules can evaluate an individual Component or a Process Group
This closes#7191
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Timing issues in system integration tests could sometimes fail with a NullPointerException when calling the poison method multiple times
This closes#7618
Signed-off-by: David Handermann <exceptionfactory@apache.org>
When shutting down FlowController, wait up to gracefulShutdownSeconds seconds for the components to stop before shutting down thread pools. This allows for asynchronous operations such as disableControllerServicesAsync to complete during shutdown. Updated StandardStatelessFlow so that on shutdown it catches more general Exception to ensure that shutdown succeeds
Ensure that Max Concurrent Tasks cannot be set less than 1 for stateless group; fixed typo in ProcessGroupDTO's docs; on shutdown, we may need to disable controller services asynchronously. At that point, the thread pool used to do so may already be shutdown. If so, catch this and create a new single-thread pool, disable the service, and immediately shutdown the pool. Also, if we fail to disable services on shutdown of a stateless flow, instead of throwing an Exception, just log it and move on - it doesn't make much sense for shutdown() to throw an Exception in that case.
Updated system tests so that when emptying a queue, we check the result and if the queue still has data (because a Processor hasn't acknowledged the data, for example) then continue issuing request until the queue fully becomes empty.
When shutting down input/output ports for stateless group, ensure that we wait for the ports' active threads to complete before returning
When stopping StatelessGroupNode, ensure that all the Processors, Controller Services (even those not executed by the Stateless Engine) are stopped/disabled before considering the Stateless Group to be fully STOPPED.
This closes#7253
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Ensure that testComponentsRecreatedOnRestart only counts components in its own group
This closes#7466
Signed-off-by: David Handermann <exceptionfactory@apache.org>
tpalfy
Matt Gilman
dan-s1
exceptionfactory
Pierre Villard
Bryan Bende
Joseph Witt
Mark Payne
Bence Simon
Joe Gresock
Nandor Soma Abonyi
Timea Barna
Tamas Palfy
Robert Kalmar