Created new python processors for text embeddings, inserting into Chroma, querying Chroma, querying ChatGPT, inserting into and querying Pinecone. Fixed some bugs in the Python framework. Added Python extensions to assembly. Also added ability to load dependencies from a requirements.txt as that was important for making the different vectorstore implementations play more nicely together.
Excluded nifi-python-extensions-bundle from GitHub build because it requires Maven to use unpack-resources goal, which will not work in GitHub because it uses mvn compile instead of mvn install
- ParseDocument
- ChunkDocument
- PromptChatGPT
- PutChroma
- PutPinecone
- QueryChroma
- QueryPinecone
NIFI-12195 Added support for requirements.txt to define Python dependencies
This closes#7894
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Moved Hadoop and Parquet NARs to include-hadoop
- Moved HBase NARs to include-hbase
- Moved Kudu NAR to include-kudu
- Moved Solr NAR to include-solr
This closes#7942.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
- Added ListenOTLP Processor supporting OpenTelemetry OTLP 1.0.0 Specification with gRPC and HTTP
- Updated nifi-event-transport to support configurable SSLParameters for configurable Cipher Suites
Signed-off-by: Joseph Witt <joewitt@apache.org>
- Renamed classes from DataUpload to ResourceTransfer and updated references
- Disabled testNonReadableFile() on Windows due to Posix permissions
- Replaced utility methods with functional handling of FileResource
- Corrected FlowFile InputStream access using Optional.orElseGet()
This closes: #7458
Co-authored-by: David Handermann <exceptionfactory@apache.org>
Signed-off-by: Nandor Soma Abonyi <nsabonyi@apache.org>
- Added nifi-compress-bundle with nifi-compress-nar
This closes#7180
Co-authored-by: David Handermann <exceptionfactory@apache.org>
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Updated GitHub workflow so that system tests include Python 3.9
- Updated GitHub actions to build necessary modules for system tests
This closes#7003
Co-authored-by: David Handermann <exceptionfactory@apache.org>
Signed-off-by: David Handermann <exceptionfactory@apache.org>
* NIFI-10975 Added Kubernetes Leader Election and State Provider
- Added Kubernetes Leader Election Manager based on Kubernetes Leases
- Added Kubernetes State Provider based on Kubernetes ConfigMaps
- Added nifi-kubernetes-client for generalized access to Fabric8 Kubernetes Client
- Added nifi.cluster.leader.election.implementation Property defaulting to CuratorLeaderElectionManager
- Refactored LeaderElectionManager to nifi-framework-api for Extension Discovering Manager
- Refactored shared ZooKeeper configuration to nifi-framework-cluster-zookeeper
* NIFI-10975 Updated Kubernetes Client and StateMap
- Upgraded Kubernetes Client from 6.2.0 to 6.3.0
- Added getStateVersion to StateMap and deprecated getVersion
- Updated Docker start.sh with additional properties
* NIFI-10975 Corrected MockStateManager.assertStateSet()
* NIFI-10975 Upgraded Kubernetes Client from 6.3.0 to 6.3.1
* NIFI-10975 Corrected unregister leader and disabled release on cancel
* NIFI-10975 Corrected findLeader handling of Lease expiration
- Changed LeaderElectionManager.getLeader() return to Optional String
* NIFI-10975 Corrected StandardNiFiServiceFacade handling of Optional Leader
* NIFI-10975 Changed getLeader() to call findLeader() to avoid stale cached values
* NIFI-10975 Updated LeaderElectionCommand to run LeaderElector in loop
* NIFI-10975 Rebased on project version 2.0.0-SNAPSHOT
* NIFI-10975 Corrected Gson and AspectJ versions
- Updated versions to match current main branch and avoid reverting
- Replaced OSUtils.getProcessId() with java.lang.Process.pid() available on Java 9 and following
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#6989.
This closes#6975
- Corrected Fetch Size property evaluation in QueryIoTDBRecord for FlowFile attributes
Signed-off-by: Mike Thomsen <mthomsen@apache.org>
- Upgraded Logback from 1.2.11 to 1.3.5
- Updated Logback DelayingShutdownHook to DefaultShutdownHook
- Disabled Spring Boot Logging System in favor of standard Logback initialization
- Excluded logback-classic from ZooKeeper and other dependencies to avoid conflicts when running tests
- Excluded spring-boot-starter-logging to avoid failures related to Logback 1.2 and Spring Boot 2.7
- Removed ZooKeeperMigratorTest.groovy based on Apache Curator test server usage of Logback 1.2
NIFI-10580 Added logback-core as explicit dependency
- Set logback-core as provided in root configuration
- Added logback-core as compile dependency in assembly configurations
Signed-off-by: Joe Witt <joewitt@apache.org>
- Update manifests assembly to include the additionaDetails.html files
- Update C2 ExtensionDefinition to add a new field for additional details content
- Update manifest generation to read the additional details and pass through to C2 runtime manifest
- Switch to boolean to indicate presence of additional details, instead of including content
Add additionalDetails files to nifi-runtime-manifest artifact, organized by group/artifact/version/type
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#6767
- Defined PrivateKeyService in nifi-key-service-api
- Implemented StandardPrivateKeyService using Bouncy Castle library
NIFI-10667 Improved custom validation to avoid repetitive reads
NIFI-10667 Added onPropertyModified() to clear Key Reference for validation
Signed-off-by: Nathan Gough <thenatog@gmail.com>
This closes#6553.
- Upgraded Azure Event Hubs components from legacy version 3 to current version 5
- Refactored Azure dependencies using azure-sdk-bom
- Merged nifi-azure-record-sink-nar with nifi-azure-nar
- Refactored PutAzureCosmosDBRecordTest to use Jackson instead of Gson
This closes#6319.
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
lehelb
Mark Payne
exceptionfactory
Joe Witt
Peter Turcsanyi
timeabarna
Matthew Burgess
Nandor Soma Abonyi
Pierre Villard
Chris Sampson
Rajmund Takacs
Bryan Bende
Robert Kalmar
Mark Bathori
Ferenc Kis
simonbence
Tamas Palfy
Ferenc Erdei