- Upgraded cache from 3 to 4
- Upgraded CodeQL from 2 to 3
- Upgraded Codecov from 3 to 4
- Upgraded upload-artifact from 3 to 4
- Upgraded stale from 6 to 9
This closes#8373
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Upgraded setup-java action from v3 to v4
- Upgraded setup-python action from v4 to v5
- Updated system-tests to use Python 3.10 instead of Python 3.9
This closes#8371
Signed-off-by: David Handermann <exceptionfactory@apache.org>
Signed-off-by: Joe Witt <joewitt@apache.org>
- Added some Use Case docs for Python processors and updated Runtime Manifests to include Python based processors as well as Use Case/MultiProcessorUseCase documentation elements. Refactored/cleaned up some of the Python code and added unit tests.
- Added python-unit-tests profile and enabled on Ubuntu and macOS GitHub workflows
This closes#8253
Signed-off-by: David Handermann <exceptionfactory@apache.org>
* [NIFI-12419] - Only include the new UI module when building if the 'include-new-ui' profile is active. Also update the ci-workflow.yaml to activate that profile.
* add the profile activation to the Maven Build command
This closes#8069
- Added report-code-coverage profile to nifi-assembly for aggregation
- Added Codecov reporting to Ubuntu Linux Job in ci-workflow
- Added Codecov badge to README.md
This closes#7995
Signed-off-by: Chris Sampson <chris.sampson82@gmail.com>
- Updated StandardPythonBridge.onProcessorRemoved to avoid throwing exceptions when called with a Processor Type and Version that is not registered
- Updated system-tests workflow to run on changes in the nifi-py4j-bundle
Created new python processors for text embeddings, inserting into Chroma, querying Chroma, querying ChatGPT, inserting into and querying Pinecone. Fixed some bugs in the Python framework. Added Python extensions to assembly. Also added ability to load dependencies from a requirements.txt as that was important for making the different vectorstore implementations play more nicely together.
Excluded nifi-python-extensions-bundle from GitHub build because it requires Maven to use unpack-resources goal, which will not work in GitHub because it uses mvn compile instead of mvn install
- ParseDocument
- ChunkDocument
- PromptChatGPT
- PutChroma
- PutPinecone
- QueryChroma
- QueryPinecone
NIFI-12195 Added support for requirements.txt to define Python dependencies
This closes#7894
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Added dependency-check GitHub workflow
- Upgraded Janino Commons Compiler from 3.1.9 to 3.1.10
- Upgraded Azure SDK BOM from 1.2.16 to 1.2.17
- Upgraded GCP SDK BOM from 26.17.0 to 26.25.0
- Upgraded AWS SDK from 1.12.550 to 1.12.573
- Upgraded Hazelcast from 5.3.2 to 5.3.5
- Upgraded Jersey from 2.40 to 2.41
- Upgraded Camel Salesforce from 3.14.5 to 3.14.9
- Unified ZooKeeper versioning on 3.9.1
- Applied Groovy 2.4.21 to Hive 3 and Iceberg components
- Applied gRPC version 1.59.0 to Asana components
- Applied Jettison 1.5.4 to Atlas and Hive 3 components
- Managed JUnit 4 version to 4.13.2 for MockWebServer
- Excluded HBase libraries from Hive 3 following Iceberg approach
- Excluded Htrace from HBase components
- Upgraded OWASP Dependency Check from 8.4.0 to 8.4.2
- Removed non-applicable dependency check suppressions
- Added dependency check suppressions for non-applicable findings
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#7948.
Use Java Zulu distribution for all docker-tests and integration-tests Github Workflows
This closes#7858
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Replaced Maven Wagon properties to Maven Native HTTP properties as required for Maven 3.9
- Removed the deprecated disableXmlReport argument
Signed-off-by: Matt Burgess <mattyb149@apache.org>
This closes#7573
When waiting for a Processor to stop completely in system tests, also consider any terminated threads. Consider the processor stopped only when all active threads and all terminated threads have gone to 0. Additionally, when stopping all components in a Process Group, wait for the PG to reflect that there are no running components. Previously, we checked for processor active thread counts. However, this was problematic because the flow that was fetched was not fetched recursively and as a result, processors running in embedded groups were not always stopped when the waitForProcessorsStopped(String groupId) method returned. Finally, removed the step for gathering surefire-reports in failed system tests because it was not working as expected and was unneeded because the tests' logs were already gathered into the diagnostic dump that is uploaded.
Added fix for BatchFlowBetweenGroupsIT. The test would intermittently fail, based on timing. The test assumed that when Input Port in Group B was scheduled to run, it would trigger before FlowFiles in Group A finished processing. The Sleep processor caused a 10 ms delay for each of 5 FlowFiles which gives the Input Port about 50 ms to trigger. If it took longer to trigger the first time, the test failed. Adjusted the Sleep processor to use 2 seconds. So now instead of 50 milliseconds to start, it has 10 seconds. Overall this adds about 30-45 seconds to the run time of the test but ensures more-than-adequate time for the test to run, even on extremely slow hardware
This closes#7289
Signed-off-by: David Handermann <exceptionfactory@apache.org>
NIFI-11557: Added an additional system test and updated github actions to include surefire-report in order to help diagnose problem that occurred in one of the last system-test runs in Github. Could not replicate problem locally
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#7265
- Updated GitHub workflow so that system tests include Python 3.9
- Updated GitHub actions to build necessary modules for system tests
This closes#7003
Co-authored-by: David Handermann <exceptionfactory@apache.org>
Signed-off-by: David Handermann <exceptionfactory@apache.org>
* NIFI-11257 Enabled Maven HTTP pooling and additional retries
- Enabled HTTP connection pooling for Maven Wagon
- Configured 30 second timeout for HTTP pooled connection lifespan
- Enabled 5 retries for HTTP connections
- Set maximum connection per route to 5 instead of 20
- Enabled retry for sent HTTP requests
- Updated GitHub workflows from Java 8 to 11
- Updated Maven Compiler Source and Target version from 1.8 to 11
- Disabled JASN1 tests failing on Windows with Java 11
This closes#6940
Co-authored-by: David Handermann <exceptionfactory@apache.org>
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- NIFI-10981 Ensure NarAutoLoader starts after provider retrieves NARs, and ensure the auto-loader attempts to process any NARs available before starting to watch for new files
- Add system test with NAR Provider that copies NARs from a local directory and verifies the NARs were loaded and linked correctly
- Exclude new assembly from CI test-compile
- Upgraded stale from 3 to 6
- Upgraded checkout from 2 to 3 in system-tests
- Upgraded setup-java from 2 to 3 in system-tests
- Upgraded upload-artifact from 2 to 3 in system-tests
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#6542.
This closes#6469
Signed-off-by: David Handermann <exceptionfactory@apache.org>
Co-authored-by: Ashish Kurmi <akurmi@stepsecurity.io>
Co-authored-by: Alex <aleksandrosansan@gmail.com>
- Upgraded maven-enforcer-plugin from 3.0.0 to 3.1.0
- Upgraded maven-dependency-plugin from 3.2.0 to 3.3.0
- Upgraded checkstyle from 9.2.1 to 9.3
- Upgraded buildnumber-maven-plugin from 1.4 to 3.0.0
This closes#6222.
Signed-off-by: Kevin Doran <kdoran@apache.org>
- Upgraded multiple GitHub Actions from Version 2 to 3
- Replaced Maven Cache step with cache feature in Setup Java Action
- Moved checkstyle and license checks from verify to validate phase in Maven execution
- Removed unnecessary contrib-check profile from nifi-registry
This closes#6202
Signed-off-by: Paul Grey <greyp@apache.org>
- Removed italicized formatting for example summary paragraph and updated example issue number
Signed-off-by: Joe Gresock <jgresock@gmail.com>
This closes#6025.
- Added Summary, Tracking, and Verification section headings
- Reworded checklist to declarative statements instead of questions
- Added JDK 17 to list of build platforms
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#6002.
- Adjusted Java 17 build locale settings
- Added test to verify match between System properties and build environment variables
This closes#5980
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Added Ubuntu Zulu JDK 17 GitHub build
- Adjusted MiNiFi C2 FileSystemConfigurationCache test to avoid using environment variables
- Adjusted MiNiFi StatusLogger and StatusLoggerTest to avoid overriding private logger
- Adjusted failure reason attribute check in TestGetIgniteCache
- Adjusted TestRangerAuthorizer and TestRangerNiFiAuthorizer to avoid checking nested exceptions
- Adjusted encrypt-config TestUtil to avoid unnecessary comparison of different types
- Disabled Javascript tests on Java 15 and higher
- Disabled several Hive 3 tests on Java 17 for StringInternUtils illegal access
- Refactored nifi-enrich-processors tests to use Mockito without Powermock
- Refactored nifi-toolkit-tls tests to avoid illegal reflective access
- Removed deprecated X509Certificate test in CertificateUtilsTest
- Removed kryo serialization from nifi-site-to-site-client test
- Updated TestHashContent to use SHA-1 instead of SHA for hash algorithm
- Upgraded maven-war-plugin from 2.5 to 3.3.2
- Upgraded nifi-graph-bundle dependencies from Groovy 2.5.14 to 3.0.8
- Upgraded QuestDB from 4.2.1 to 6.2.1 in nifi-framework-core
This closes#5870
Signed-off-by: Mike Thomsen <mthomsen@apache.org>
update lintin
update font awesome path
bump nifi-fds version
update nifi deps
update frontend maven plugin
change CI npm cache restore keys so that we will not use cached directories for older versions of node
This closes#5751.
Signed-off-by: Kevin Doran <kdoran@apache.org>
- Corrected several missing return statements
- Increased Maven heap size to 3 GB for ci-workflow
This closes#5700
Signed-off-by: David Handermann <exceptionfactory@apache.org>
exceptionfactory
Joseph Witt
Mark Payne
Rob Fellows
Chris Sampson
Paul Grey
Nandor Soma Abonyi
Pierre Villard
Bryan Bende
Ashish Kurmi
Scott Aslan