- Prefixing endpoint paths with a forward slash ensures correct HTTP request formatting required for some deployments with a forwarding proxy
This closes#6058
Signed-off-by: David Handermann <exceptionfactory@apache.org>
* NIFI-10001: Fixed issue in which some components may fail to update the scheduled state when comparing flows
* NIFI-10001: Fixed bugs that caused some components to not have their scheduled state updated. When comparing two flows, now allow specifying how to determine a VersionedComponent's ID for comparison. When comparing local flow against flow from registry, use Versioned Component ID. But when comparing two instantiated flows, such as local flow vs. cluster flow, use the VersionedComponent's Instance ID instead. This ensures that we can properly compare two components even if there are several instances of a given flow
- Added Map and Set Cache Servers based on nifi-event-transport components
- Removed custom servers and unused socket stream components
- Reduced duplication on protocol classes
- Added checks for readable bytes
- Added mark and reset handling for buffer reads
This closes#6040
Signed-off-by: Paul Grey <greyp@apache.org>
* NIFI-9895 Allow parameter to reference controller service. Check read and write authorization for both previous and newly set controller service. Authorization done for both property or parameter change. Import/export handled by switching between instance id and versioned id.
* NIFI-10001: When enabling a collection of Controller Services, changed logic. Instead of enabling dependent services and waiting for them to complete enablement before starting a given service, just start the services given. The previous logic was necessary long ago because we couldn't enable a service unless all dependent services were fully enabled. But that changed a while ago. Now, we can enable a service when it's invalid. It'll just keep trying to enable until it becomes valid. At that point, it will complete its transition from ENABLING to ENABLED.
* NIFI-10001: Restored previous implementation for StandardControllerServiceProvider, as the changes were not ultimately what we needed. Changed StandardProcessGroup to use a ConcurrentHashMap for controller services instead of a HashMap with readLock. This was causing a deadlock when we enable a Controller Service that references another service during flow synchronization. Flow Synchronization was happening within a write lock and enabling the service required a read lock on the group. Eventually the thread holding the write lock would timeout and release the write lock. But this caused significant delays on startup. By changing to a ConcurrentHashMap, we alleviate the need for the Read Lock. Also noticed in testing that the StandardNiFiServiceFacade did not save flow changes when enabling dependent services so added call to controllerFacade.save().
- Removed version declarations from multiple modules
- Adjusted PutDynamoDB to remove catch for IOException not thrown in Commons IO 2.11.0
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#6015.
- Replaced google-cloud-bom 0.172.0 with libraries-bom 25.2.0 in nifi-gcp-bundle
- Removed specific versions from Google dependencies in nifi-gcp-processors and nifi-gcp-services-api
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#6010.
NIFI-7234 Replaced Jackson 1.X references with Jackson 2.X references in various classes.
NIFI-7234 Added jackson-annotations to nifi-hdfs-processors.
NIFI-7234 Various updates to bring our test cases into better alignment with the Avro specification as of 1.11.
Fixed a checkstyle issue.
NIFI-7234 Made changes requested in a review.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#5900
- Replaced nifi-framework-bundle managed dependency to root managed dependency
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#6005.
- Resolves build failures on Java 17 where the original user.timezone property returns null from System.getProperty()
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#6000
NIFI-9918: Fixed review findings
NIFI-9918: Example added to XMLReader's additionalDetails.html on the new property. Minor documentation fixes.
Signed-off-by: Nathan Gough <thenatog@gmail.com>
This closes#5964.
This fixes the occasional "403 Forbidden" bug that we have seen, simply because
the signature ends up being invalid due to an invalid datetime format used.
Issue: NIFI-9888
Reference: https://stackoverflow.com/a/51636763/647151
Use statically defined formatter and explain why
This closes#5943.
Signed-off-by: Kevin Doran <kdoran@apache.org>
- Removed unnecessary references to jackson.version property
- Removed unnecessary dependency management references to Jackson libraries
This closes#5992
Signed-off-by: Mike Thomsen <mthomsen@apache.org>
- Changed SSH server to start and stop after each method
- Replaced queued file with string FlowFile contents
- Refactored TestPutSFTP using JUnit 5
Signed-off-by: Joe Witt <joewitt@apache.org>
NIFI-9861: Fixed stateless-processor-tests assembly to ensure that all necessary libraries were included; removed BlockListClassLoader
NIFI-9861: Fixed issue in which we would list .class files as files that we allow through the AllowListClassLoader but didn't allow them.
This closes#5925.
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
- Removed PatchedSFTPEngine with resolution of SFTP renaming in SSHJ 0.33.0
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#5982.
* NIFI-9883 Refactored property protection to isolated ClassLoader
- Added nifi-property-protection-loader for abstracting access to implementation classes using ServiceLoader
- Updated Authorizer and Login Identity Provider configuration using isolated ClassLoader
- Updated NiFi Properties Loader using isolated ClassLoader
- Updated nifi-assembly to place property protection dependencies in lib/properties directory
- Updated and refactored unit tests
- Corrected LoginIdentityProviderFactoryBean getObject() Type
- Adjusted PrometheusServer configuration to use SSLContextService.createContext() instead of individual properties
This closes#5970
Signed-off-by: Paul Grey <greyp@apache.org>
- Corrected collection and database name properties to handle FlowFile attributes
This closes#5966
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Changed log methods that accept a String and Object array to replace the last Throwable argument with a formatted summary of causes when calling LogRepository.addLogMessage()
Signed-off-by: Joe Gresock <jgresock@gmail.com>
This closes#5965.
- Refactored XML parsing to use providers from nifi-xml-processing
- Configured spotbugs-maven-plugin with findsecbugs-plugin in nifi-xml-processing
- Disabled Validate DTD in default configuration for EvaluateXPath and EvaluateXQuery
- Replaced configuration of DocumentBuilder and streaming XML Readers with shared components
- Removed XML utilities from nifi-security-utils
- Moved Commons Configuration classes to nifi-lookup-services
This closes#5962
Signed-off-by: Paul Grey <greyp@apache.org>
- Refactored SimpleProcessLogger to avoid sending stack trace causes to SLF4J Logger
- Refactored SimpleProcessLogger to minimize duplication of component message formatting
- Updated ConnectableTask logging to avoid repeating Throwable class in message string
- Refactored TestSimpleProcessLogger to improve coverage and confirm consistent argument handling
- Corrected handling of exception strings in argument arrays
NIFI-9884 - JacksonCSVRecordReader ignores specified encoding; test case for ISO-8859-1
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#5941
Resolves performance issues that impact versions 4.4 and 4.3 of
the driver and adds support up through MongoDB 5.1
Add support for Java 17
This closes#5940
Signed-off-by: Mike Thomsen <mthomsen@apache.org>
- Replaced individual Azure dependencies with azure-sdk-bom in nifi-property-protection-azure
- Removed woodstox-core and stax2-api exclusions
- Refactored TemplateDeserializer test class to avoid dependency on specific XML implementation
This closes#5929
Signed-off-by: Mike Thomsen <mthomsen@apache.org>
- Upgraded direct dependencies in nifi-email-processors and nifi-poi-processors
- Upgraded transitive dependency in nifi-media-bundle
This closes#5927
Signed-off-by: Mike Thomsen <mthomsen@apache.org>
NIFI-9846 removing paging from ListAzureBlobStorage_v12 and ListAzureDataLakeStorage, adding = to filtering
This closes#5916.
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
* NIFI-9850 Added support for multiple expressions to GrokReader
- Updated Grok Expression property to support Resources
* NIFI-9850 Updated documentation for Fields from Grok Expression strategy
This closes#5918
Signed-off-by: Otto Fowler <otto@apache.org>
* NIFI-9853: Refactored StandardProcessGroupSynchronizer to make use of State Lookups and Compoennt Schedulers to ensure that we properly synchronize states when starting up, when exporting flow definitions, and when importing Flow Definitions
* NIFI-9853: Fixed NPE
* NIFI-9847: Switched LifecycleState to use a WeakHashMap to track ActiveProcessSessionFactory instances, instead of a regular Set that removed the instance after calling onTrigger. This was necessary for processors such as MergeRecord that may stash away an ActiveProcessSessionFactory for later use, as we need to be able to force rollback on processor termination
* NIFI-9847: Fixed checkstyle violation
- Replaced custom DatabaseReader with standard DatabaseReader implementation
This closes#5909
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Changed default Kudu Client Worker Count to number of runtime-reported available processors
Signed-off-by: Joe Gresock <jgresock@gmail.com>
This closes#5886.
* NIFI-9787: Increasing visibility of selected slickgrid row with 1px borders on the top and bottom plus a slightly darker yellow background-color value.
* NIFI-9787: Added missing units to the padding values.
Merged #5857 into main.
- Changed expected exception to ProcessException in KeyedEncryptorGroovyTest and PasswordBasedEncryptorGroovyTest to avoid intermittent failures
This closes#5911
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Corrected No Tracking strategy Record Writer handling for ListSFTP
- Updated temporary test files to have last modified time of epoch to avoid intermittent issue with Minimum Age filtering
- Refactored MockCacheService to separate reusable class
This closes#5885
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Added Ubuntu Zulu JDK 17 GitHub build
- Adjusted MiNiFi C2 FileSystemConfigurationCache test to avoid using environment variables
- Adjusted MiNiFi StatusLogger and StatusLoggerTest to avoid overriding private logger
- Adjusted failure reason attribute check in TestGetIgniteCache
- Adjusted TestRangerAuthorizer and TestRangerNiFiAuthorizer to avoid checking nested exceptions
- Adjusted encrypt-config TestUtil to avoid unnecessary comparison of different types
- Disabled Javascript tests on Java 15 and higher
- Disabled several Hive 3 tests on Java 17 for StringInternUtils illegal access
- Refactored nifi-enrich-processors tests to use Mockito without Powermock
- Refactored nifi-toolkit-tls tests to avoid illegal reflective access
- Removed deprecated X509Certificate test in CertificateUtilsTest
- Removed kryo serialization from nifi-site-to-site-client test
- Updated TestHashContent to use SHA-1 instead of SHA for hash algorithm
- Upgraded maven-war-plugin from 2.5 to 3.3.2
- Upgraded nifi-graph-bundle dependencies from Groovy 2.5.14 to 3.0.8
- Upgraded QuestDB from 4.2.1 to 6.2.1 in nifi-framework-core
This closes#5870
Signed-off-by: Mike Thomsen <mthomsen@apache.org>
Removed DMC.
NIFI-6047 Started integrating changes from NIFI-6014.
NIFI-6047 Added DMC tests.
NIFI-6047 Added cache identifier recordpath test.
NIFI-6047 Added additional details.
NIFI-6047 Removed old additional details.
NIFI-6047 made some changes requested in a follow up review.
NIFI-6047 latest.
NIFI-6047 Finished updates
First round of code review cleanup
Latest
Removed EL from the dynamic properties.
Finished code review requested refactoring.
Checkstyle fix.
Removed a Java 11 API
NIFI-6047 Renamed processor to DeduplicateRecord
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#4646
Added NiFi DetectDuplicateRecord standard processor.
Adding some documentation and PR review tweaks.
Exposing processor
Documentation updates, exception handling consolidation, added support for record path field variables.
Added tests.
Build bump.
Migrated cache service to groovy folder.
Moved declarations for properties to @BeforeClass lifecycle method.
Adding some documentation and PR review tweaks.
Documentation updates, exception handling consolidation, added support for record path field variables.
Added tests.
Build bump.
Migrated cache service to groovy folder.
Fixed variable type bug.
Fixed mapping of test params to usage.
Fixed potential illegal state exception bug.
* NIFI-9761 Corrected PeerChannel processing for TLS 1.3
- Added TestPeerChannel with methods for TLS 1.2 and TLS 1.3
- Updated PeerChannel.close() to process SSLEngine close notification
- Improved logging and corrected handling after decryption
- NIFI-9086 Refactored nifi-accumulo-bundle to use JUnit 5
- NIFI-9087 Updated nifi-ambari-bundle to use JUnit 5
- NIFI-9088 Refactored nifi-amqp-bundle to use JUnit 5
- NIFI-9089 Updated nifi-asn1-bundle to use JUnit 5
- NIFI-9090 Refactored nifi-atlas-bundle to use JUnit 5
- NIFI-9091 Refactored nifi-avro-bundle to use JUnit 5
- NIFI-9092 Refactored nifi-aws-bundle to use JUnit 5
- NIFI-9095 Refactored nifi-azure-bundle to use JUnit 5
- NIFI-9145 Refactored nifi-rethinkdb-bundle to use JUnit 5
- NIFI-9143 Refactored nifi-redis-bundle to use JUnit 5
This closes#5350
Signed-off-by: David Handermann <exceptionfactory@apache.org>
Relaxing MockPropertyValue validation to allow for variables to be passed to config verification
Fixing underlying framework issue with config verification: wrong variable registry was being used
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#5843
* Update nf-context-menu.js
Include enable/disable all controllers menu item
* Update nf-actions.js
Create enableAllControllers and disableAllControllers actions
* Rename ...Controllers to ...ControllerServices
* Rename *controllers to *controller services
* Adjusts proposed by reviewer.
* Refactor enable and disable AllControllerServices
This closes#5247
- Updated stateless and StandardProcessGroup, etc. to make use of VersionedExternalFlow
- Updated StatelessDataflowDefinition to use ExternalVersionedFlow instead of generic type
- Updated Stateless Bootstrap to avoid loading stateless engine libs from root class path but instead use a NarClassLoader to load the statelss nar
Signed-off-by: Joe Gresock <jgresock@gmail.com>
This closes#5832.
- Upgraded Spring Security from 5.6.1 to 5.6.2
- Upgraded Spring Boot from 2.6.3 to 2.6.4 in NiFi Registry
- Upgraded Spring Data Redis from 2.6.0 to 2.6.2
- Upgraded Spring Integration from 5.5.7 to 5.5.9
This closes#5835
Signed-off-by: Mike Thomsen <mthomsen@apache.org>
* NIFI-9543: add bring-to-front functionality to labels
* NIFI-9543: update zIndex changes to persist after shutdown
* NIFI-9543: remove commented code
* NIFI-9543: update flow comparator and snippet to account for zIndex
* NIFI-9543: changed add function to sort labels when a new label is added
* NIFI-9543: remove unnecessary call to position
* NIFI-9543: only register difference if zIndex is not default value
* NIFI-9543: add null checks to prevent primitive Long
This closes#5765
NIFI-9425: Added autoload directory property and defensive code for stop() called twice
This closes#5762.
Signed-off-by: Kevin Doran <kdoran@apache.org>
- Applied new permission restrictions to AWSCredentialsProviderControllerService and GCPCredentialsControllerService
Signed-off-by: Joe Gresock <jgresock@gmail.com>
This closes#5796.
* NIFI-9580: initial ui work for framework-level retry in processors
* NIFI-9580: disable retry controls when retry not selected
* NIFI-9580: refactor marshalling to be one function
* NIFI-9580: style tweaks and documentation update
* NIFI-9580: update tooltip content
* NIFI-9580: change tooltip copy for clarity
* NIFI-9580: only submit retry configs if selected
* NIFI-9580: add relationship details to read-only dialog
* NIFI-9580: change styling based on feedback
This closes#5666
NIFI-9731: Only simplify flowfile arg in bulletin after extracting the flowfile uuid
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#5808
- NIFI-9122 Refactored nifi-influxdb-bundle to use JUnit 5
- NIFI-9121 Refactored nifi-ignite-bundle to use JUnit 5
- NIFI-9120 Refactored nifi-html-bundle to use JUnit 5
- NIFI-9119 Refactored nifi-hl7-bundle to use JUnit 5
- NIFI-9118 Refactored nifi-hive-bundle to use JUnit 5
- NIFI-9117 Refactored nifi-hbase-bundle to use JUnit 5
- NIFI-9116 Refactored nifi-hazelcast-bundle to use JUnit 5
- NIFI-9114 Refactored nifi-hadoop-bundle to use JUnit 5
- Added DisabledOnOs(OS.WINDOWS) to Hive tests with @TempDir from JUnit 5 due to Windows-specific problems in JUnit 5
This closes#5791
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- NIFI-9129 Refactored as much of nifi-kudu-bundle to use JUnit 5 as possible without breaking the integration tests which use test libraries from the Kudu project that depend on JUnit 4
- NIFI-9109 Refactored nifi-extension-utils to use JUnit 5. Left two tests with JUnit 4 because they used APIs that don't appear to have analogues in JUnit 5
- NIFI-9112 Refactored nifi-groovyx-bundle to use JUnit 5
- NIFI-9142 Refactored nifi-ranger-bundle to use JUnit 5
- NIFI-9136 Refactored nifi-network-bundle to use JUnit 5
This closes#5794
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Added nifi-h2-database module shading H2 1.4.200
- Implemented version checking and migration
This closes#5724
Signed-off-by: David Handermann <exceptionfactory@apache.org>
NIFI-9072: Improvements to ValidateXML including validate XML in attributes and non-schema, well-formed XML only validation, and additional documentation
- Updated description and added System Resource Consideration to discourage using large XML documents in attributes
- Migrate from XML DocumentBuilderFactory to SafeXMLConfiguration
- NIFI-9111 Refactored nifi-gcp-bundle to use JUnit 5
- NIFI-9110 Refactored nifi-flume-bundle to use JUnit 5
- NIFI-9108 Refactored nifi-evtx-bundle to use JUnit 5
- NIFI-9106 Refactored nifi-email-bundle to use JUnit 5
- NIFI-9105 Refactored nifi-easyrules-bundle to use JUnit 5
- NIFI-9104 Refactored nifi-druid-bundle to use JUnit 5
This closes#5790
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- NIFI-9130 Refactored nifi-language-translation-bundle to use JUnit 5
- NIFI-9126 Refactored nifi-kafka-bundle to use JUnit 5
- NIFI-9125 Refactored nifi-jolt-record-bundle to use JUnit 5
This closes#5792
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- NIFI-9102 Refactored nifi-cybersecurity-bundle to use JUnit 5
- NIFI-9101 Refactored nifi-couchbase-bundle to use JUnit 5
- NIFI-9100 Refactored nifi-confluent-platform-bundle to use JUnit 5
- NIFI-9099 Refactored nifi-cdc to use JUnit 5
- NIFI-9098 Refactored nifi-ccda-bundle to use JUnit 5
- NIFI-9097 Refactored nifi-cassandra-bundle to use JUnit 5
- NIFI-9096 Refactored nifi-beats-bundle to use JUnit 5
This closes#5789
Signed-off-by: David Handermann <exceptionfactory@apache.org>
Changed from AmazonS3 to AmazonS3client.
Add REMOTE_INVOCATION provenance event with URL and "Object deleted"
This closes#5778
Signed-off-by: Mike Thomsen <mthomsen@apache.org>
Changed from AmazonS3 to AmazonS3client.
Add REMOTE_INVOCATION provenance event with URL and "Object tagged"
This closes#5785
Signed-off-by: Mike Thomsen <mthomsen@apache.org>
Chris Sampson
exceptionfactory
Mark Payne
Joe Gresock
tpalfy
Lehel
Matthew Burgess
Peter Gyori
Paul Grey
Pierre Villard
Tamas Palfy
Emilio Setiadarma
Matt Gilman
Juldrixx
Mike Thomsen
Malthe Borch
Ravinarayan Singh
simonbence
Nandor Soma Abonyi
Joe Witt
greyp9
dependabot[bot]
Peter Gyori
Peter Turcsanyi
Lance Kinley
Timea Barna
Bence Simon
Hervé Boutemy
Jonathan Conti-Vock
Jon Conti-Vock
Nathan Gough
Marcus Ely
Bryan Bende
Adam
Eduardo Fontes
Shane Ardell
Adam Kocsis
markobean
Hsin-Ying Lee
xinyuwu4
lawye1973