NIFI-8081 Added new Listing Strategy to ListFTP and ListSFTP: Adjusted Time Window. User can specify the time zone or time difference (compared to where NiFi runs) of the system hosting the files and based on the calculates the current time there. Lists files modified before this adjusted current time (and after the last listing).
NIFI-8081 'Time Adjustment' validated not to be set if listing strategy is not 'Adjusted Time Window'. Extracted validator to a separate class. Added more tests. Minor refactor. Typo fix.
NIFI-8081 Improved validation.
NIFI-8081 'Time Adjustment' is not necessary - in fact it can cause problems. SFTP (and usually FTP - which has a more general bug at the moment) returns a timestamp that doesn't really need adjustment. (SFTP in particular returns the an 'epoch' time.) Everything remains the same - the new listing strategy relies on a sliding time window, but without the unnecessary option to adjust for the modification time.
NIFI-8081 Resolved conflicts after rebasing to main.
NIFI-8081 Renamed 'AbstractListProcessor.listByAdjustedSlidingTimeWindow' to 'listByTimeWindow'. Post main rebase correction.
NIFI-8081 Updated user doc for the BY_TIME_WINDOW strategy to warn user on it's reliance of accurate time.
This closes#4721.
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
Updated TestInvokeHTTP, TestInvokeHttpSSL, TestInvokeHttpTwoWaySSL, and TestListenHTTP to use new Keystore functionality.
NIFI-1355 Refactored and removed unnecessary unit tests in KeyStoreUtilsGroovyTest.
NIFI-1355 Added a password requirement when creating a new truststore.
Handled exception when loading a passwordless truststore type of Bouncy Castle PKCS12.
This closes#4801
Signed-off-by: David Handermann <exceptionfactory@apache.org>
- Removed references to StandardSSLContextService from nifi-standard-processors
- Removed TestGetHTTPGroovy and TestPostHTTPGroovy since these are testing deprecated processors
- Optimized HandleHttpRequest, GetHTTP, PostHTTP to use SSLContextService.createContext()
NIFI-8178 Changed TestGetHTTP to ITGetHTTP since GetHTTP is deprecated
NIFI-8178 Changed TestPostHTTP to ITPostHTTP since PostHTTP is deprecated
Signed-off-by: Joe Witt <joewitt@apache.org>
NIFI-8171 Increased response and idle timeouts for HTTP unit tests
NIFI-8171 Increased TestServer idle timeout to 45 seconds for HTTP unit tests
NIFI-8171 Adjusted timeout and sleep on TestPutTCPCommon.testPruneSenders
NIFI-8171 Increased TestServer idle timeout to 60 seconds and removed 500ms Thread.sleep() in TestInvokeHttpSSL
NIFI-8171 Optimized OkHttpClientUtils to avoid reading trust store twice during initialization
NIFI-8171 Added static variable for server startup sleep
NIFI-8171 Increased TestInvokeHTTP Connect Timeout and TestListenHTTP Response Timeout to 30 seconds
NIFI-8171 Refactored unit tests for InvokeHTTP and ListenHTTP to optimize SSLContext creation
NIFI-8171 Updated TestListenHTTP for static creation of SSLContext
NIFI-8171 Added started check for ListenHTTP Server in TestListenHTTP
NIFI-8171 Refactored TestPutTCP classes to optimize SSLContext creation
NIFI-8171 Increased TestListenHTTP timeout for server start to 120 seconds and added exception when not connected
NIFI-8171 Increased Connect and Read Timeouts for InvokeHTTP SSL unit tests
Signed-off-by: Joe Witt <joewitt@apache.org>
NIFI-7356 - Addresses PR feedback.
NIFI-7356 - Additional changes from PR feedback.
NIFI-7356 - Adding integration tests for ZooKeeperStateServer for TLS.
NIFI-7356 - TLS + Zookeeper now working with single and quorum. Needs code cleanup, need to fix IT tests and docs.
NIFI-7356 - Fixed up tests and removed some irrelevant ones. Refactored some of ZooKeeperStateServer. Tested successfully with a secure and insecure 3 node NiFi + Quorum.
NIFI-7356 - Checkstyle fixes.
NIFI-7356 - Updated administration guide with embedded ZooKeeper TLS configuration.
NIFI-7356 - Updated the way ZooKeeper TLS properties are set/mapped from NiFi properties.
NIFI-7356 - Updated per review, using NiFiProperties keystore strings, classname for ocnnection factory, adjusted TLS configuration checks in NiFiProperties.
NIFI-7356 - Updated configuration validation logic and added tests.
NIFI-7356 - Codestyle check fixes.
NIFI-7356 - Updated some of the log messages.
NIFI-7356 - Updated as per code review.
NIFI-7356 - Fixed max port number.
NIFI-7356 - Updated admin guide and updated small code issues as per code review.
Signed-off-by: Nathan Gough <thenatog@gmail.com>
This closes#4753.
NIFI-8146: Updated PutDatabaseRecord to avoid using the functional style framework that had previously been used. Doing so resulted in code that was difficult to understand and maintain. Added additional unit tests and improved MockRecordParser so that it could throw configurable types of exceptions
NIFI-8146: Fixed checkstyle issues
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#4763
NIFI-7989: Only rewrite records if a field name doesn't match a table column name exactly
NIFI-7989: Rewrite records for created tables if Update Field Names is set
This closes#4750.
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
NIFI-7906 Fixed bugs in unit tests.
NIFI-7906 Updated a few things based on a code review.
NIFI-7906: Fixed typo in record.count attribute
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#4739
- Added default property value for automatic determination of Client Authentication Policy based on SSLContextService Trust Store properties
- Added new ClientAuthentication enum with values specific to ListenHTTP
This closes#4749.
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
- Added SSLContextService.createContext() and refactored referencing components
- Removed references to ClientAuth from SslContextFactory methods
This closes#4737.
Signed-off-by: Mark Payne <markap14@hotmail.com>
NIFI-7964 Remove unused import
NIFI-7964 Reverted to using BufferedInputStream along with unmarkable
NIFI-7964 Made UnmarkableInputStream static and added factory method
NIFI-7964 Remove unnecessary factory method from UnmarkableInputStream
This closes#4632.
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
When using MonitorActivity, it would be interesting to use variables in "Threshold Duration", which will help maintain Version Control in ProcessGroups (Avoiding "breaking" versions when including the value manually).
I've included the option expressionLanguageSupported(ExpressionLanguageScope.VARIABLE_REGISTRY), so we can use variables in "Threshold Duration".
- Force the TailFile processor to recreate and reposition the reader
by setting it to null in case of a NulCharacterEncounteredException
- Updated the TestTailFile.testNULContent() to not initialize the
processor when calling the second run()
Update the reader's position instead of abandoning it.
This closes#4736.
Signed-off-by: Mark Payne <markap14@hotmail.com>
NIFI-7989: Add support for creating partitions, quote identifiers
NIFI-7989: Quote table name when getting description
This closes#4697.
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
NIFI-7913 Changed order of supported protocols to match existing comments in SSLContextService
This closes#4599
Signed-off-by: Nathan Gough <thenatog@gmail.com>
Mark Payne
Peter Gyori
Nathan Gough
Peter Turcsanyi
Matthew Burgess
Mark Bean
Jon Kessler
Tamas Palfy
Denes Arvay
mtien
Joe Witt
Bryan Bende
exceptionfactory
Grant Henke
Pierre Villard
snorlaxa
Waleed Al Aibani
Bence Simon
Troy Melhase
Mike Thomsen
Andrew Lim
s9514171
eduardofontes
Chris Sampson
Nissim Shiman
tlsmith
sjyang18
Joey Frazee
Otto Fowler
Kevin Barranco
Shane Ardell
Sushil Kumar
Peter Turcsanyi
DataMoTrain
r65535