- Refactored stateless to use this when creating a dataflow so that Parameter Provider implementations can be externalized into NARs. Also updated ExtensionDiscoveringManager such that callers are able to provide a new type of class to be discovered (e.g., ParameterProvider) so that the extensions will be automatically discovered
- Put specific command-line overrides as highest precedence for parameter overrides
- Make ParameterOverrideProvider valid by allowing for dynamically added parameters
- Fixed bug in validation logic, added new system tests to verify proper handling of Required and Optional properties
- Addressed review feedback and fixed some bugs. Also added system test to verify Parameter Providers are working as expected
This closes#5113
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Added getDateFormat() using default time zone instead of GMT time zone from DataTypeUtils.getDateFormat()
NIFI-8748 Adjusted Date Format to use DataType.getFormat()
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#5194.
- Added jakarata.activation 2.0.1 to support jakarta.mail 2.0.1
- Clarified description of SMTP TLS property enabling STARTTLS
NIFI-8630 Added final to several variables and adjusted variable declaration
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#5179.
As of KUDU-1884, Kudu supports custom Kerberos principals on server-side
and custom SASL protocol (service) names on client-side which must match
the SPN base, i.e. if the SPN is kudu/_HOST, SASL protocol name *must*
be "kudu" in the client to be able to connect to the cluster.
This patch adds the ability to configure this in the PutKudu processor.
Signed-off-by: Attila Bukor <abukor@apache.org>
NIFIDEVS-8195: fixed properties not properly inheriting from template/snippet values
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#5094
- Updated secure FTP processors to configure which algorithms, ciphers and message authentication codes are allowed to be used by the SSH Client
- Included Expression Language Variable Registry support for properties
This closes#5061
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Handling additional FINISHED Handshake Status for TLS 1.3 Post-Handshake Messages per RFC 8446 Section 4.6
- Removed clearing buffers after handshake to avoid losing packets
- Updated read() method to check Handshake Status after SSLEngine.unwrap()
- Changed SSLSocketChannelSender to close SSLSocketChannel before other resources
- Added ChannelStatus enum and convenience logging methods for tracing status
- Added unit tests for TLS 1.2 and 1.3 using Netty server and client handlers
NIFI-8704 Updated netty-handler to 4.1.65.Final
NIFI-7468 Corrected SSLSocketChannel.read() to return byte read
NIFI-7468 Adjusted comment formatting
Signed-off-by: Nathan Gough <thenatog@gmail.com>
This closes#5152.
Added unit test that is ignored so that it can be manually run for testing performance before/after changes to AvroTypeUtil. Updated AvroTypeUtil to be more efficient by not using Record.getValue() and instead iterating over the Map of values directly. getValue() is less efficient here because we know the RecordField's we are iterating over exist in the schema since they are retrieved from there directly; as a result, any null values still have be looked up by aliaases, but that step can be skipped in this situation. Also avoided looking for fields that exist in Avro Schema and not in RecordSchema just to set default values on GenericRecord - there's no need to set them if they are default values.
This closes#5080
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Set default HTTPS Port to 9443
- Set default authorizer to single-user-authorizer
- Set default login-identity-provider to single-user-provider
- Updated README.md with authentication instructions using generated credentials
- Updated default URL and port information in Administration and User Guides
- Updated Getting Started Guide with authentication and URL changes
- Updated Docker images to set HTTPS as default configuration
- Updated default HTTPS port to 8443
- Set Cluster Protocol secure property in Docker start scripts
- Added set-single-user-credentials command
- Refactored shared classes to nifi-single-user-utils
- Updated Getting Started documentation and logging
- Updated documentation and TLS Toolkit default ports
- Updated Toolkit Guide and Administration Guide
- Updated README.md with HTTPS links
- Overriding Calcite incubating versions avoids dependencies from third party repositories
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#5142
NIFI-8658: Addressed issue where the RecordField that was provided from Function Filters were not accurate
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#5125
NIFI-8133 pass null/empty field handling from PutElasticsearchRecord to ElasticSearchClientImpl
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#4755
Also noticed a typo in the ControllerStatusReportingTask and found in comparing outputs
that it had a bug that caused it to log counters generated only by processors at the root level so fixed that.
This closes#5101
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Upgrade resolves issue unpacking Zip files with temporary spanning markers
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#5103.
NIFI-8528 Migrate NiFi Registry fully codebase into NiFi as a module. No changes except certain dependency scopes to preserve the NiFi Registry original by overriding the new parent (nifi).
- Version adjustments. Removed distinct checkstye rules form nifi-registry. (Using nifi's instead.)
- Made some tests Windows-compatible.
- Consolidated LICENSE, NOTICE and README.md.
- Fixed CryptoKeyLoaderGroovyTest.groovy.
- Disable frontend-maven-plugin on Windows.
- Skipping all goals of the frontend-maven-plugin on Windows.
- Registry integration tests not to run in github jobs (same as the original settings). Skip all registry tests (build and run) on Windows.
- Removed Husky from registry.
- Added nifi-event-transport module encapsulating Netty classes
- Refactored unit tests for PutSyslog and ListenSyslog
- Removed integration tests for PutSyslog and ListenSyslog
NIFI-8462 Added context.yield() in PutSyslog when no FlowFiles and addressed other issues
NIFI-8462 Removed unused import of ExpressionLanguageScope
Signed-off-by: Nathan Gough <thenatog@gmail.com>
This closes#5044.
Also allow to write them as such (byte-arrays) - again, instead of throwing an exception.
NIFI-8439 Fixed unit tests.
NIFI-8439 Allow writing parquet INT96 timestamps if they were read by the same parquet-avro library.
This closes#5006.
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
NIFI-8618: Allow for parameters to be set for environment variables. Updated README to indicate this and also corrected some outdated information.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#5087
- Unflattening a flattened json
- Preserving primitive arrays such as strings, numbers, booleans and null in a nested json
- Logging errors when failure
- Pretty printing resulted json
This closes#5083
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Implemented custom ThreadPoolExecutor with maximum pool size based on Worker Count property
- Refactored processing methods to ensure KuduSession is always closed
- Added SystemResourceConsideration for Memory
- Removed duplicate dependency on nifi-security-kerberos
- Adjusted method naming to clarify functionality
- Reverted addition of defaultAdminOperationTimeoutMs()
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#5020.
- Updated Mock Framework to now fail tests that use ProcessSession.commit() unless they first call TestRunner.setAllowSynchronousSessionCommits(true)
- Updated stateless nifi in order to make use of async session commits
- Fixed bug that caused stateless to not properly handle Additional Classpath URLs and bug that caused warnings about validation to get generated when a flow that used controller services was initialized. While this is not really in scope of NIFI-8469, it was found when testing and blocked further progress so addresssed here.
- If Processor fails to progress when run from stateless, trigger from start of flow until that is no longer the case
- Introduced notion of TransactionThresholds that can limit the amount of data that a flow will bring in for a given invocation of stateless dataflow
- Several new system-level tests
- FileBasedClusterNodeFirewallFactoryBean returns PermitAllClusterNodeFirewall instead of null to avoid having the Spring Framework return a NullBean in Spring Framework 5
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#5082
- Added totalActiveQueuedDuration and maxActiveQueuedDuration to the ConnectionStatus object
- Updated FlowFileQueue implementations and supporting code to properly calculate and provide the totalActiveQueuedDuration and maxActiveQueuedDuration for their active queues
- Fixing failing unit test. In examining this it appears that the unit test only accidentally passed in the past and that the object mocked to always throw an exception was not actually being used in the test.
- Adding UI component via ConnectionStatusDescriptor along with updates based on PR comments.
Signed-off-by: Mark Payne <markap14@hotmail.com>
- Upgraded direct dependencies from 2.6 to 2.8.0
- Added dependency management configuration to use 2.8.0 for some modules
- Updated scripted Groovy tests to avoid copying unnecessary files
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#5073
Mark Payne
Guillaume Schaer
exceptionfactory
Matthew Burgess
Nathan Gough
Pierre Villard
Lars Francke
Peter Turcsanyi
Joe Gresock
Attila Bukor
Mark Bean
Bence Simon
Jon Kessler
Lehel Boér
simonbence
Chris Sampson
rhavermans
Bernhard Geisberger
Abhishek Kumar
Kent Nguyen
Timea Barna
Peter Gyori
Paul Grey
Dominik Przybysz
s9514171
Chris Sampson
tpalfy
Tamas Palfy
Mohammed Nadeem
Wouter de Vries
Chris Sampson