- Added Spring Security PKCE OAuth2 Request Customizer to standard Resolver
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#8324.
- Added some Use Case docs for Python processors and updated Runtime Manifests to include Python based processors as well as Use Case/MultiProcessorUseCase documentation elements. Refactored/cleaned up some of the Python code and added unit tests.
- Added python-unit-tests profile and enabled on Ubuntu and macOS GitHub workflows
This closes#8253
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Updated Expression Language Guide to note the available system property providing the current framework version according to the Framework NAR version
This closes#8101
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Added nifi-framework-status-history-shared module for common components
- Added include-questdb build profile for including the QuestDB NAR
- Upgraded QuestDB from 7.2 to 7.3.7 for NIFI-12435
Signed-off-by: Bence Simon <bsimon@apache.org>
This closes#8141
Documented difference between Active Threads in Status bar versus as shown by Node in Cluster Summary
This closes#7978
Signed-off-by: David Handermann <exceptionfactory@apache.org>
Created new python processors for text embeddings, inserting into Chroma, querying Chroma, querying ChatGPT, inserting into and querying Pinecone. Fixed some bugs in the Python framework. Added Python extensions to assembly. Also added ability to load dependencies from a requirements.txt as that was important for making the different vectorstore implementations play more nicely together.
Excluded nifi-python-extensions-bundle from GitHub build because it requires Maven to use unpack-resources goal, which will not work in GitHub because it uses mvn compile instead of mvn install
- ParseDocument
- ChunkDocument
- PromptChatGPT
- PutChroma
- PutPinecone
- QueryChroma
- QueryPinecone
NIFI-12195 Added support for requirements.txt to define Python dependencies
This closes#7894
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Removed support for Year field in Cron Expressions
- Changed numeric Day of Week field from starting with 1 to starting with 0
- Removed Quartz and C3P0 from Notice files
This closes#7951
Signed-off-by: Mike Moser <mosermw@apache.org>
- Moved Hadoop and Parquet NARs to include-hadoop
- Moved HBase NARs to include-hbase
- Moved Kudu NAR to include-kudu
- Moved Solr NAR to include-solr
This closes#7942.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
- Replaced Groovy implementation with Java classes
- Added StandardEncryptConfig and RegistryEncryptConfig command implementations
- Added FileTransformer interface with implementations for configuration files
- Changed -A to -u for output authorizers.xml when targeting Registry
- Removed --newFlowProvider option not used
- Removed --translateCli option
- Removed --decrypt option
Signed-off-by: Joe Gresock <jgresock@gmail.com>
This closes#7901.
* NIFI-12206 Refactored Flow History using JetBrains Xodus
- Replaced H2 Database Engine with JetBrains Xodus for persistent storage of FlowConfigurationHistory
- Added EntityStoreAuditService implementation using Xodus PersistentEntityStore
- Removed nifi.h2.url.append from properties
- Created NIFI-12203 to evaluate issues with flow comparison surfaced in JoinClusterWithDifferentFlow
This closes#7661
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Added new extensible Component Type: FlowAnalysisRule
- Added DisallowComponentType Rule implementation
- Flow Analysis Rules can be managed from the UI under Controller Settings -> Flow Analysis Rules
- Flow Analysis Rules can be set up with an enforcement policy of WARN or ENFORCE
- Flow Analysis Rules can evaluate an individual Component or a Process Group
This closes#7191
Signed-off-by: David Handermann <exceptionfactory@apache.org>
When shutting down FlowController, wait up to gracefulShutdownSeconds seconds for the components to stop before shutting down thread pools. This allows for asynchronous operations such as disableControllerServicesAsync to complete during shutdown. Updated StandardStatelessFlow so that on shutdown it catches more general Exception to ensure that shutdown succeeds
Ensure that Max Concurrent Tasks cannot be set less than 1 for stateless group; fixed typo in ProcessGroupDTO's docs; on shutdown, we may need to disable controller services asynchronously. At that point, the thread pool used to do so may already be shutdown. If so, catch this and create a new single-thread pool, disable the service, and immediately shutdown the pool. Also, if we fail to disable services on shutdown of a stateless flow, instead of throwing an Exception, just log it and move on - it doesn't make much sense for shutdown() to throw an Exception in that case.
Updated system tests so that when emptying a queue, we check the result and if the queue still has data (because a Processor hasn't acknowledged the data, for example) then continue issuing request until the queue fully becomes empty.
When shutting down input/output ports for stateless group, ensure that we wait for the ports' active threads to complete before returning
When stopping StatelessGroupNode, ensure that all the Processors, Controller Services (even those not executed by the Stateless Engine) are stopped/disabled before considering the Stateless Group to be fully STOPPED.
This closes#7253
Signed-off-by: David Handermann <exceptionfactory@apache.org>
Added documentation to indicate how to debug Python side of nifi framework, as well as debugging Python processors themselves using VSCode's Remote debugger.
This also provides the ability to launch the Controller process in such a way that it will listen to incoming remote debug connections.
This closes#7469
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Added methods to enumerate Stored Component Identifiers on State Provider interface and implementations
- Added nifi.state.management.provider.cluster.previous to nifi.properties
- Updated State Manager Provider to restore Cluster State from Previous Cluster Provider
- Updated Configuring State Providers documentation for new property
NIFI-10865 allow UpdateRecord to replace the Record root for relative paths, e.g. when a RecordPath function is used to modify selected field(s)
This closes#6708
Signed-off-by: Mike Thomsen <mthomsen@apache.org>
Bence Simon
lehelb
exceptionfactory
Mark Payne
Pierre Villard
Joe Gresock
theBestAndrew
Alex Ethier
annanys23
Timea Barna
Paul Grey
Joseph Witt
Justin
Timea Barna
Tamas Palfy
Mike Thomsen
Nandor Soma Abonyi
Arek Burdach
Chris Sampson