- Replaced Netty 3.6.9 and 3.7.1 with 3.10.6
- Replaced Netty 4.1 with 4.1.69
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#5490.
- Removed GroovyTestCase super class
- Replaced shouldFail() with assertThrows()
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#5481.
- Upgraded Spring Security from 5.5.1 to 5.5.2
- Upgraded Spring Boot from 2.5.2 to 2.5.5 in Registry
- Upgraded Spring Data Redis from 2.5.3 to 2.5.5 in Redis modules
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#5463.
- Updated documentation with new properties
- Refactored cipher operations to common RepositoryEncryptor classes
- Abstracted record metadata serialization for better compatibility
Signed-off-by: Joe Gresock <jgresock@gmail.com>
This closes#5407.
- Added unit tests to reproduce issues with available() method
- Changed available() to return size of application buffer
- Removed unused isDataAvailable()
- Refactored unwrap handling to read from channel for buffer underflow
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#5421.
- Removed nifi.minifi.sensitive.props.provider Property from MiNiFi
- Removed property from example NiFi properties files
- Removed provider from MiNiFi SensitivePropsSchema
- Removed BC provider value from MiNiFi test cases
Signed-off-by: Joe Gresock <jgresock@gmail.com>
This closes#5422.
- Set NIFI_PBKDF2_AES_GCM_256 as property encryption method
- Replaced static default sensitive properties key with random UUID
- Added unit test for PropertiesFileEngineConfigurationParser
- Added random encryption key generation method
- Changed Stateless to use PropertyEncryptionMethod enum
Signed-off-by: Joe Gresock <jgresock@gmail.com>
This closes#5424
- Replaced use of Authorization header with custom Request-Token header for CSRF mitigation
- Added Request-Token cookie for CSRF mitigation
- Replaced session storage of JWT with expiration in seconds
- Removed and disabled CORS configuration
- Disabled HTTP OPTIONS method
- Refactored HTTP Proxy URI construction using RequestUriBuilder
Signed-off-by: Nathan Gough <thenatog@gmail.com>
This closes#5417.
NIFI-9192: ResultSetRecordSet consider value of useLogicalType flag when sqlType is not handled separately (default) and readerSchema is not null
Signed-off-by: Peter Gyori <peter.gyori.dev@gmail.com>
NIFI-9192: Unit tests added to test the use of useLogicalType flag
Signed-off-by: Peter Gyori <peter.gyori.dev@gmail.com>
NIFI-9192: ResultSetRecordSet consider not using logical type when sqlType is OTHER
NIFI-9192: Fixed checkstyle violations
Signed-off-by: Peter Gyori <peter.gyori.dev@gmail.com>
NIFI-9192: Renamed JdbcCommon.getUseLogicalTypes to isUseLogicalTypes
Signed-off-by: Peter Gyori <peter.gyori.dev@gmail.com>
NIFI-9192: Added unit tests for Array types with and without using logical types
Signed-off-by: Peter Gyori <peter.gyori.dev@gmail.com>
NIFI-9192: Review comments considered and unit tests refactored
NIFI-9192: Added further unit tests
NIFI-9192: Minor modifications based on review comments
Signed-off-by: Peter Gyori <peter.gyori.dev@gmail.com>
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#5388
- Implemented ApplicationCookieService for adding and retrieving HTTP Cookies
- Added getCookieResourceUri() leveraging allowed proxy headers to support optional Cookie Paths
- Refactored Access Resources to use ApplicationCookieService for processing
- Changed __Host- prefix to __Secure- prefix for Bearer Token cookie to support Cookie Path processing
- Removed unnecessary jetty-http dependency from nifi-web-api
- Corrected NiFi path references in JavaScript to support prefixed paths
Signed-off-by: Nathan Gough <thenatog@gmail.com>
This closes#5329.
- Added TemporaryKeyStoreBuilder with File.deleteOnExit() for KeyStore and TrustStore files
- Removed JKS files from nifi-security-utils tests
- Refactored usage of KeyStoreUtils.createKeyStoreAndGetX509Certificate() to TemporaryKeyStoreBuilder
- Removed unnecesary hadoop-minikdc test dependency in security-utils
- Replaced Mini KDC Hex utility with Bouncy Castle Hex utility in unit tests
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#5406
- Added EncodedSensitivePropertiesProvider with Base64 encoding methods
- Added ClientBasedEncodedSensitivePropertiesProvider with validate method
- Abstracted client configuration to ClientProvider interface and implementations
- Added unit tests for AWS and Azure Property Providers
NIFI-9184 Adjusted abstract provider class names and updated documentation
Signed-off-by: Nathan Gough <thenatog@gmail.com>
This closes#5363.
- Refactored multiple tests using KeyStoreUtils
- Removed static KeyStore and TrustStore files
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#5401
- Replaced old com.sun.xml.bind:jaxb-impl and jaxb-core with current org.glassfish.jaxb:jaxb-runtime
- Replaced old javax.xml.bind:jaxb-api with current jakarta.xml.bind-api
- Removed unnecessary dependency references to javax.activation-api
This closes#5320
Signed-off-by: Mike Thomsen <mthomsen@apache.org>
- Updated DataTypeUtils.getDateFormat() to use system default time zone
- Updated Record Path Guide to match Expression Language Guide regarding default time zone
- Updated impacted unit tests to expect localized dates instead of dates converted to GMT
NIFI-8749 Set user.timezone in TestRecordPath to avoid unexpected date conversions
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#5361
- Updated DataTypeUtils.getDateFormat() to use system default time zone
- Updated Record Path Guide to match Expression Language Guide regarding default time zone
- Updated impacted unit tests to expect localized dates instead of dates converted to GMT
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#5305
* NIFI-9082 Added nifi.zookeeper.jute.maxbuffer property
- Updated ZooKeeperStateProvider to check jute.maxbuffer instead of hard-coded number
- Updated internal ZooKeeperClientConfig class to include new juteMaxBuffer property
* NIFI-9082 Updated jute.maxbuffer documentation and added hexadecimal representation
- Renamed DefaultWatcher to NoOpWatcher
- Removed unnecessary validateDataSize() from createNode()
- Replaced per-user symmetric-key HS256 with shared and rotated RSA asymmetric-key RS512 implementation
- Added nifi.security.user.jws.key.rotation.period property for RSA Key Pair rotation
- Added JSON Web Tokens section to Administration Guide
- Implemented persistent storage of RSA Public Keys for verification using Local State Manager
- Implemented JWT revocation on logout with persistence using Local State Manager
- Refactored JWT implementation using Spring Security OAuth2 and Nimbus JWT
- Refactored Spring Security Provider configuration using Java instead of XML
- Removed H2 storage of per-user keys
- Upgraded nimbus-jose-jwt from 7.9 to 9.11.2
NIFI-8766 Corrected AuthenticationException handling in AccessResource.getAccessStatus
- Added nifi.user.security.jws.key.rotation.period to default nifi.properties
- Updated logging statements and clarified configuration and method documentation
NIFI-8766 Changed Algorithm to PS512 and updated documentation
Signed-off-by: Nathan Gough <thenatog@gmail.com>
This closes#5262.
NIFI-8974 Integrate KerberosUserService with HDFS processors
NIFI-8980 Integrate KerberosUserService with Kafka 2.6 processors
- Introduced SelfContainerKerberosUserService to restrict which impls can be used with Kafka
- Add variations of KerberosUser doAs that allow setting the context ClassLoader
- Add additional unit tests for configurations
This closes#5277
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Upgraded direct and transitive dependencies from 1.20 and earlier to 1.21
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#5273.
- Upgrades Spring Framework from 5.3.8 to 5.3.9
- Upgrades Spring Security from 5.4.6 to 5.5.1
- Upgrades Spring Boot from 2.5.1 to 2.5.2 in Registry
- Upgrades Spring Data Redis from 2.5.1 to 2.5.3
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#5252.
- Added Jetty DoSFilter configured for /access/token
- Added nifi.web.max.access.token.requests.per.second property with default value of 25
Signed-off-by: Nathan Gough <thenatog@gmail.com>
This closes#5215.
- Updated PutKudu to use DataTypeUtils.toLocalDate() for DATE fields
- Updated PutDatabaseRecord to remove convertDateToLocalTZ() since convertType() uses toLocalDate()
- Updated PutElasticsearchHttpRecord to use default time zone format for DATE fields
- Updated WriteXMLResult to use default time zone format for DATE fields
- Updated WriteJsonResult to use default time zone format for DATE fields
- Updated AvroTypeUtil to use toLocalDate() for logical DATE fields
- Updated JdbcCommon to avoid conversion to UTC for logical DATE fields
- Updated Processor and RecordReader unit tests for consistency in DATE comparison
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#5210
- KeyStoreKeyProvider supports PKCS12 and BCFKS
- Refactored KeyProvider and implementations to nifi-security-kms
- Updated Admin Guide and User Guide with KeyStoreKeyProvider details
NIFI-8511 Improved documentation and streamlined several methods
Signed-off-by: Nathan Gough <thenatog@gmail.com>
This closes#5110.
- Handling additional FINISHED Handshake Status for TLS 1.3 Post-Handshake Messages per RFC 8446 Section 4.6
- Removed clearing buffers after handshake to avoid losing packets
- Updated read() method to check Handshake Status after SSLEngine.unwrap()
- Changed SSLSocketChannelSender to close SSLSocketChannel before other resources
- Added ChannelStatus enum and convenience logging methods for tracing status
- Added unit tests for TLS 1.2 and 1.3 using Netty server and client handlers
NIFI-8704 Updated netty-handler to 4.1.65.Final
NIFI-7468 Corrected SSLSocketChannel.read() to return byte read
NIFI-7468 Adjusted comment formatting
Signed-off-by: Nathan Gough <thenatog@gmail.com>
This closes#5152.
- Set default HTTPS Port to 9443
- Set default authorizer to single-user-authorizer
- Set default login-identity-provider to single-user-provider
- Updated README.md with authentication instructions using generated credentials
- Updated default URL and port information in Administration and User Guides
- Updated Getting Started Guide with authentication and URL changes
- Updated Docker images to set HTTPS as default configuration
- Updated default HTTPS port to 8443
- Set Cluster Protocol secure property in Docker start scripts
- Added set-single-user-credentials command
- Refactored shared classes to nifi-single-user-utils
- Updated Getting Started documentation and logging
- Updated documentation and TLS Toolkit default ports
- Updated Toolkit Guide and Administration Guide
- Updated README.md with HTTPS links
NIFI-8658: Addressed issue where the RecordField that was provided from Function Filters were not accurate
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#5125
* NIFI-8137 Record Path EscapeJson/UnescapeJson functions
* Correct jackson-databind dependency version
* Add negative tests for RecordPath JSON handling; rename RecordPath JSON classes to better match existing functions
Signed-off-by: Otto Fowler <ottobackwards@gmail.com>
This closes#4756.
- Added nifi-event-transport module encapsulating Netty classes
- Refactored unit tests for PutSyslog and ListenSyslog
- Removed integration tests for PutSyslog and ListenSyslog
NIFI-8462 Added context.yield() in PutSyslog when no FlowFiles and addressed other issues
NIFI-8462 Removed unused import of ExpressionLanguageScope
Signed-off-by: Nathan Gough <thenatog@gmail.com>
This closes#5044.
- Upgraded direct dependencies from 2.6 to 2.8.0
- Added dependency management configuration to use 2.8.0 for some modules
- Updated scripted Groovy tests to avoid copying unnecessary files
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#5073
NIFI-6061: Force getBytes() in BLOB handling to use UTF-8 charset
NIFI-6061: Use setClob(), added unit tests, incorporated review comments
This closes#5049
Co-authored-by: zhangcheng <zhangchengk@foxmail.com>
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Retained legacy default Sensitive Properties Key in ConfigEncryptionTool to support migration
- Streamlined default file path and moved key generation conditional
- Refactored with getDefaultProperties()
- Cleared System Property in ConfigEncryptionToolTest
- Added checking and error handling for clustered status
- Added set-sensitive-properties-key command
- Refactored PropertyEncryptor classes to nifi-property-encryptor
- Added nifi-flow-encryptor
- Refactored ConfigEncryptionTool to use FlowEncryptor for supporting AEAD algorithms
- Added Admin Guide section Updating the Sensitive Properties Key
This closes#4857.
Signed-off-by: Mark Payne <markap14@hotmail.com>
- Refactored and consolidated KeyStoreUtils unit tests
- Corrected KeyStoreUtils.loadEmptyKeyStore() to use KeyStoreUtils.getKeyStore()
Signed-off-by: Nathan Gough <thenatog@gmail.com>
This closes#5015.
- AbstractJsonRowRecordReader - Handle (meaning log a warning and not fail completely) multi-array CHOICE type when data has extra fields (not defined by the schema) and can't determine correct type.
- AvroTypeUtil - Allow multiple different record types in avro union type. Minor refactors. Added documentation fro EqualsWrapper.
- Updated components to make use of new feature
NIFI-8206: Added a ResourceType of TEXT. This requires that the ResourceReferenceFactory know which types are allowed in order to create the ResourceReference. PropertyValue needs to then have the PropertyDescriptor available to it. This resulted in highlighting many bugs in unit tests where components were not exposing property descriptors via getSupportedPropertyDescriptors() or were evaluating Expression Language using the wrong scope, so fixed many unit tests/components to properly declare Expression Language scope when using it
NIFI-8206: Removed problematic unit test that required directory names with special characters that are not allowed on some operating systems
This closes#4890.
Signed-off-by: Bryan Bende <bbende@apache.org>
- Added nifi.web.request.ip.whitelist property to set DoSFilter.ipWhitelist
- Added nifi.web.request.timeout property to set DoSFilter.maxRequestMs with default of 60 seconds
This closes#4972
Signed-off-by: David Handermann <exceptionfactory@apache.org>
NIFI-8386: Addressed review feedback: removed unused call to determine permissions, null out bulletins in standalone mode if permissions not allowed. Also fixed automated tests that were failing due to changes
This closes#4955
- Added createTrustManager() on SSLContextService
- Removed nifi-security-utils and okhttp dependencies from nifi-web-utils
Signed-off-by: Nathan Gough <thenatog@gmail.com>
This closes#4869.
NIFI-8329 - Removed unnecessary jackson.version from azure bundle to use the global property instead.
NIFI-8329 - Updated jackson/jackson-databind version and removed the 'jackson-databind.version' pom property in favor of 'jackson.version'
Updated dependencies include the following:
- jackson-core
- jackson-databind
- icu4j
- snakeyaml
- spring-integration-mail
- spring-core and framework modules
- activemq-client
- activemq-broker
- xercesImpl
This closes#4911
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Created nifi-security-socket-ssl
- Created nifi-security-kerberos
- Removed nifi-security-utils dependency from nifi-processor-utils
- Updated modules to reference new dependencies
- Eliminated unnecessary transitive dependencies on bcprov-jdk15on from over 30 modules
Signed-off-by: Nathan Gough <thenatog@gmail.com>
This closes#4881.
exceptionfactory
Mike Thomsen
Nathan Gough
Joe Gresock
Paul Grey
Peter Gyori
Mark Payne
denes
Matthew Burgess
Emilio Setiadarma
Bryan Bende
Lehel Boér
Timea Barna
Bryan Bende
Joe Witt
Peter Turcsanyi
rhavermans
Chris Sampson
Bence Simon
Sandro Tolaini
Tamas Palfy
noblenumbat360
Mark Bean