- Added ConsumeKafka and PublishKafka Processors with Kafka3ConnectionService
- Added Kafka 3 integration tests
This closes#8463
Co-authored-by: David Handermann <exceptionfactory@apache.org>
Signed-off-by: David Handermann <exceptionfactory@apache.org>
This closes#8978
- Removed nifi-property-protection-api and implementation modules
- Removed nifi-toolkit-encrypt-config and minifi-toolkit-encrypt-config modules
- Removed extra bootstrap.conf configuration files for property protection implementations
Signed-off-by: Joseph Witt <joewitt@apache.org>
- Fixed logging logic for tests vs runtime scope
- Restoring behavior to testProcessUnparsableRecordWithRawOutputWithCheckpoint that changed in NIFI-12998 presumably because of log dependency changes. Now that we have a single provider the test behavior appears to be back to normal.
- Upgraded AWS SDK 1 from 1.12.686 to 1.12.710
- Upgraded AWS SDK 2 from 2.25.16 to 2.25.40
- Upgraded Azure SDK BOM from 1.2.21 to 1.2.23
- Upgraded GCP Libraries from 26.34.0 to 26.37.0
- Upgraded Fabric8 Kubernetes Client from 6.10.0 to 6.12.1
- Upgraded Commons CLI from 1.6.0 to 1.7.0
- Upgraded Commons Codec from 1.16.1 to 1.17.0
- Upgraded zstd-jni from 1.5.6-1 to 1.5.6-3
- Upgraded Commons Text from 1.11.0 to 1.12.0
- Upgraded Test Containers from 1.19.4 to 1.19.7
- Upgraded JAXB Runtime from 4.0.4 to 4.0.5
- Upgraded JAXB API from 4.0.1 to 4.0.2
- Upgraded Jersey from 3.1.4 to 3.1.6
- Upgraded Log4j 2 from 2.23.0 to 2.23.1
- Upgraded Logback from 1.5.5 to 1.5.6
- Upgraded Mockito from 5.8.0 to 5.11.0
- Upgraded Swagger Annotations from 2.2.20 to 2.2.21
- Upgraded jsoup from 1.17.1 to 1.17.2
- Upgraded JUnit Platform Commons from 1.10.0 to 1.10.2
- Upgraded Checkstyle from 9.3 to 10.15.0
- Upgraded ASM from 9.6 to 9.7
This closes#8708
Signed-off-by: David Handermann <exceptionfactory@apache.org>
Simplified dependency management into Bill of Material dependencies as parents that align to classloader structures.
- Moved nifi-framework-bundle to root level
- Moved nifi-nar-bundle modules to root nifi-extension-bundles
- Added nifi-extension-bom for internal and external extensions
- Added nifi-framework-nar-bom and nifi-server-nar-bom for managing framework dependencies
- Renamed nifi-jetty-bundle to nifi-jetty-nar
This closes#8677
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Moved nifi-hive-test-utils to nifi-iceberg-test-utils
- Removed nifi-hive-bundle and include-hive3 profile
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#8588.
- Upgraded Spring Framework from 5.3.31 to 6.0.15
- Upgraded Spring Security from 5.8.7 to 6.2.0
- Upgraded Spring Vault from 2.3.4 to 3.1.0
- Upgraded Jetty from 10.0.18 to 12.0.5 with EE 10
- Upgraded Jersey from 2.41 to 3.1.4
- Upgraded JAXB from 2.3.9 to 4.0.4
- Upgraded AspectJ from 1.9.20.1 to 1.9.21
- Upgraded JMS API from 2.0.1 to 3.1.0
- Upgraded ActiveMQ Broker from 5.18.2 to 6.0.1 for JMS 3
- Upgraded JJWT from 0.9.1 to 0.12.3
- Replaced jackson-module-jaxb-annotations with jackson-module-jakarta-xmlbind-annotations
- Replaced maven-jaxb2-plugin with hisrc-higherjaxb40-maven-plugin 2.1.1
- Replaced kongchen swagger-maven-plugin with swagger-codegen-maven-plugin from Swagger 3
- Replaced com.nickwongdev AspectJ Plugin with Codehaus 1.14.0 for newer Java versions
- Removed unused cglib-nodep
- Removed references to javax.validation
- Removed custom Jetty ALPN Processor not required for Java 21
- Removed several tests depending on older Jetty and Jakarta libraries
- Removed unnecessary webdefault.xml configurations
- Replaced unsupported cross-context servlet forwarding with HTTP forwarding
- Replaced javax.servlet references with jakarta.servlet
- Replaced javax.xml.bind references with jakarta.xml.bind
- Replaced javax.ws references with jakarata.ws
- Updated Spring Security CSRF implementation for Spring Security 6
- Updated web.xml versions to 6.0
- Updated REST API templates using new swagger-codegen variables
- Removed VALIDATE_DATA property from ParseCEF based on library compatibility issue with javax.validation
- Added application URL logging to NiFi JettyServer
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#8197.
- Added nifi-framework-status-history-shared module for common components
- Added include-questdb build profile for including the QuestDB NAR
- Upgraded QuestDB from 7.2 to 7.3.7 for NIFI-12435
Signed-off-by: Bence Simon <bsimon@apache.org>
This closes#8141
Created new python processors for text embeddings, inserting into Chroma, querying Chroma, querying ChatGPT, inserting into and querying Pinecone. Fixed some bugs in the Python framework. Added Python extensions to assembly. Also added ability to load dependencies from a requirements.txt as that was important for making the different vectorstore implementations play more nicely together.
Excluded nifi-python-extensions-bundle from GitHub build because it requires Maven to use unpack-resources goal, which will not work in GitHub because it uses mvn compile instead of mvn install
- ParseDocument
- ChunkDocument
- PromptChatGPT
- PutChroma
- PutPinecone
- QueryChroma
- QueryPinecone
NIFI-12195 Added support for requirements.txt to define Python dependencies
This closes#7894
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Moved Hadoop and Parquet NARs to include-hadoop
- Moved HBase NARs to include-hbase
- Moved Kudu NAR to include-kudu
- Moved Solr NAR to include-solr
This closes#7942.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
- Added ListenOTLP Processor supporting OpenTelemetry OTLP 1.0.0 Specification with gRPC and HTTP
- Updated nifi-event-transport to support configurable SSLParameters for configurable Cipher Suites
Signed-off-by: Joseph Witt <joewitt@apache.org>
- Renamed classes from DataUpload to ResourceTransfer and updated references
- Disabled testNonReadableFile() on Windows due to Posix permissions
- Replaced utility methods with functional handling of FileResource
- Corrected FlowFile InputStream access using Optional.orElseGet()
This closes: #7458
Co-authored-by: David Handermann <exceptionfactory@apache.org>
Signed-off-by: Nandor Soma Abonyi <nsabonyi@apache.org>
- Added nifi-compress-bundle with nifi-compress-nar
This closes#7180
Co-authored-by: David Handermann <exceptionfactory@apache.org>
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Updated GitHub workflow so that system tests include Python 3.9
- Updated GitHub actions to build necessary modules for system tests
This closes#7003
Co-authored-by: David Handermann <exceptionfactory@apache.org>
Signed-off-by: David Handermann <exceptionfactory@apache.org>
* NIFI-10975 Added Kubernetes Leader Election and State Provider
- Added Kubernetes Leader Election Manager based on Kubernetes Leases
- Added Kubernetes State Provider based on Kubernetes ConfigMaps
- Added nifi-kubernetes-client for generalized access to Fabric8 Kubernetes Client
- Added nifi.cluster.leader.election.implementation Property defaulting to CuratorLeaderElectionManager
- Refactored LeaderElectionManager to nifi-framework-api for Extension Discovering Manager
- Refactored shared ZooKeeper configuration to nifi-framework-cluster-zookeeper
* NIFI-10975 Updated Kubernetes Client and StateMap
- Upgraded Kubernetes Client from 6.2.0 to 6.3.0
- Added getStateVersion to StateMap and deprecated getVersion
- Updated Docker start.sh with additional properties
* NIFI-10975 Corrected MockStateManager.assertStateSet()
* NIFI-10975 Upgraded Kubernetes Client from 6.3.0 to 6.3.1
* NIFI-10975 Corrected unregister leader and disabled release on cancel
* NIFI-10975 Corrected findLeader handling of Lease expiration
- Changed LeaderElectionManager.getLeader() return to Optional String
* NIFI-10975 Corrected StandardNiFiServiceFacade handling of Optional Leader
* NIFI-10975 Changed getLeader() to call findLeader() to avoid stale cached values
* NIFI-10975 Updated LeaderElectionCommand to run LeaderElector in loop
* NIFI-10975 Rebased on project version 2.0.0-SNAPSHOT
* NIFI-10975 Corrected Gson and AspectJ versions
- Updated versions to match current main branch and avoid reverting
exceptionfactory
Joseph Witt
Paul Grey
Bryan Bende
Mark Bathori
Bence Simon
dan-s1
Matt Burgess
lehelb
Mark Payne
Peter Turcsanyi
timeabarna