93b1a05dc3
NIFI-8030: Added property to ReportLineageToAtlas to configure file/directory level logging of [hd]fs_path entities to Atlas
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4813 .
2021-02-25 18:37:19 +04:00
34fc94454f
NIFI-8249: Fixed issue with error during multiple FlowFile results in ExecuteSQL processors
2021-02-24 10:10:32 -05:00
2322b2cddf
NIFI-6113 Refactoring GetHDFSFileInfo to remove instance level variable to make it stateless.
...
This closes #4837 .
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2021-02-24 13:15:12 +01:00
2dedd8bf0f
NIFI-8253: Restore call to session.commit() in GenerateTableFetch
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4840 .
2021-02-24 15:57:16 +04:00
f101a2bba5
NIFI-8198: ConsumeAMQP detects if Queue is deleted on server
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4805 .
2021-02-24 12:04:52 +04:00
f2555f27f1
NIFI-8119: properly free database resources when done with the processing of BLOBs
...
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #4791
2021-02-23 16:47:32 -05:00
d049bca794
NIFI-8205: Documentation improvements for the Wait processor
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4808 .
2021-02-23 21:22:19 +04:00
528fce2407
NIFI-7646, NIFI-8222: Instead of having StandardProcessSession call ContentRepository.read(ContentClaim), introduced a new ContentRepository.read(ResourceClaim) and hold open the InputStream to the ResourceClaim. This can't be supported by EncryptedContentRepository, so introduced a method to allow using this or not. The benefit here is that when we have many FlowFiles read within a session, such as when using MergeContent/MergeRecord or a processor configured with a Run Duration, we can hold open a single InputStream instead of constantly opening FileInputStreams and seeking to the appropriate location. This is much faster.
...
- Instead of entering a 'synchronized' block for every provenance event, serialize up to 1 MB worth of data, and then enter synchronized block to write that data out. This avoids large amounts of lock contention and context switches
NIFI-7646: Removed TODO and unused Jackson dependency
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #4818
2021-02-23 10:57:54 -05:00
200c04c6d0
NIFI-8237: This closes #4835 . Added missing SQL types to getDataTypeFromSQLTypeValue(), added defensive code
...
Signed-off-by: Joe Witt <joewitt@apache.org>
2021-02-23 08:08:09 -07:00
9101160cbf
NIFI-8250 - This closes #4838 . UTF8 encoding for FTP processors
...
Signed-off-by: Joe Witt <joewitt@apache.org>
2021-02-23 08:05:32 -07:00
3cb26aec72
NIFI-8232 CSV Parsers optionally allow/reject duplicate header names
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4828 .
2021-02-23 09:50:18 +04:00
418e2cc2cb
NIFI-8132 Replaced framework uses of MD5 with SHA-256
...
NIFI-8132 Added FileDigestUtils in nifi-nar-utils to avoid dependency on nifi-utils
NIFI-8132 Removed unused imports from NarUnpacker
NIFI-8132 Removed MD5 references from FileUtils documentation
NIFI-8132 Replaced StringBuffer with StringBuilder and made new DigestUtils classes final
NIFI-8132 Replaced Collections.sort() with Stream.sorted()
Signed-off-by: Nathan Gough <thenatog@gmail.com>
This closes #4788 .
2021-02-22 17:52:24 -05:00
ebef823cb9
NIFI-8241 This closes #4833 . Set the SAML context provider to use the EmptyStorageFactory
...
Signed-off-by: Joe Witt <joewitt@apache.org>
2021-02-22 08:53:32 -07:00
703c3cfe63
NIFI-5906 - Removed FILE_EXISTS_VALIDATOR on JARs and Files to be added in the session
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4810 .
2021-02-22 13:04:03 +04:00
80da0cf47d
NIFI-7496: Regex pattern changed to match dashes in filters.
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4811 .
2021-02-22 12:59:03 +04:00
b08d6071dd
NIFI-8224: Add LoggingRecordSink controller service
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4820 .
2021-02-22 12:55:53 +04:00
6ed496c714
NIFI-8242 Ensure NiFi URL for redirects has a trailing slash in AccessResource
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4834 .
2021-02-22 12:48:42 +04:00
51e7bdf940
NIFI-3669 Add SSL Support to CaptureChangeMySQL
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4594 .
2021-02-22 11:29:59 +04:00
59ad21fbaa
NIFI-8200 Modified PutAzureDataLakeStorage to delete temp file if exception was thrown in uploadContent()
...
This closes #4815
Signed-off-by: Joey Frazee <jfrazee@apache.org>
2021-02-21 13:35:28 -08:00
662a35b7de
NIFI-8240: Unify Kudu versions
...
The Kudu versions for the controller service and processors
currently doesn’t match. This patch unifies the version in
the parent bundle pom to ensure they match going forward.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4832 .
2021-02-19 20:42:30 +04:00
cd6418cdaf
NIFI-8238: When FlowFile Concurrency/Outbound Policy is changed on child group, and child group is directly under version control, ensure that those fields are maintained on the generated VersionedProcessGroup when nested components are stripped out
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4831 .
2021-02-19 14:44:42 +04:00
f532b3ae1d
NIFI-5623 Upgraded OkHttp3 to 4.9.1 and updated unit tests
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4826 .
2021-02-19 14:42:16 +04:00
aa726040c5
NIFI-8188 - Add 'Run Once' for processors in context menu.
...
Signed-off-by: Mark Payne <markap14@hotmail.com>
2021-02-18 14:51:28 -05:00
6801704846
NIFI-8235 UnsupportedOpertationException in PutAccumuloRecord
2021-02-18 13:12:09 -05:00
042b34cb43
NIFI-8231 Clarify behavior of allowed values for Memory Pools in MonitorMemory
2021-02-18 09:58:48 -05:00
fd50d6f042
NIFI-3383 Check if usage operations are supported in MonitorMemory
...
This closes #4790
Signed-off-by: Joey Frazee <jfrazee@apache.org>
2021-02-16 17:25:28 -06:00
88fab00e29
NIFI-7873 merging release branch to latest and updating to 1.14.0-SNAPSHOT
2021-02-15 12:09:32 -07:00
4afb2ba743
NIFI-7873-RC4 prepare for next development iteration
2021-02-15 12:09:31 -07:00
487280bee9
NIFI-7873-RC4 prepare release nifi-1.13.0-RC4
2021-02-15 12:09:30 -07:00
a1c82a66a1
NIFI-7361 - increase actions column width for reporting tasks
...
This closes #4214
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-02-15 10:42:19 -06:00
d08f02428d
NIFI-8223: This closes #4819 . Use column datatype in PutDatabaseRecord when calling setObject()
...
Signed-off-by: Joe Witt <joewitt@apache.org>
2021-02-10 18:01:43 -07:00
8057f8f6c5
NIFI-8221 - Set the default HTTP listening interface to 127.0.0.1.
...
This closes #4817
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-02-10 14:46:37 -06:00
1d82fb8e01
NIFI-8218 This closes #4816 . Use proxy headers when available when getting request values while processing SAML responses
...
Signed-off-by: Joe Witt <joewitt@apache.org>
2021-02-10 13:34:57 -07:00
d5d520764d
NIFI-8210: When Index Reader/Searcher is used, do not allow the Lucene Index to be deleted until the reader/searcher is finsihed being used and closed.
...
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #4812
2021-02-09 15:32:15 -05:00
b9b131239c
NIFI-8215 --wait-for-init in nifi.sh refactored for POSIX sh
...
This fixes #4814
Signed-off-by: Joey Frazee <jfrazee@apache.org>
2021-02-09 11:10:15 -08:00
4ebeb4e146
NIFI-8181 - Added Disable HTTP/2 property to InvokeHTTP
...
This closes #4804
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-02-08 14:53:35 -06:00
40d8c41656
NIFI-8193: Fixed PutSQL reordering FlowFiles
...
Also removed sorting of transferred MockFlowFiles in test framework.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #4803
2021-02-05 16:17:10 -05:00
749d05840b
NIFI-8204, NIFI-7866: Send revision update count in heartbeats. If update count in heartbeat is greater than that of cluster coordinator, request that node reconnect to get most up-to-date revisions. Cannot check exact equality, as the values may change between the time a heartbeat is created and the time the cluster coordinator receives it. However, it should be safe to assume that the revision won't be greater than that of the cluster coordinator. There is a tiny window in which it could be, as the sending node may update its revision, create the heartbeat, send it, and cluster coordinator process it before updating its own revision. However, this window is incredibly small and would only result in the sending node reconnecting, which will resolve itself. Also, when testing this fix, encountered NIFI-7866 and addressed that NullPointerException.
...
This closes #4806 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2021-02-05 15:18:44 -05:00
03fd59eb2f
NIFI-8196: When node is reconnected to cluster, ensure that it re-registers for election of cluster coordinator / primary node. On startup, if cluster coordinator is already registered and is 'this node' then register silently as coordinator and do not join the cluster until there is no Cluster Coordinator or another node is elected. This allows the zookeeper session timeout to elapse.
...
Signed-off-by: Bryan Bende <bbende@apache.org>
2021-02-05 15:18:07 -05:00
b77dbd5030
NIFI-8172: Provide schema name to getPrimaryKeys call in PutDatabaseRecord
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4782 .
2021-02-06 00:00:58 +04:00
a9b8635ac9
NIFI-6522: Referencing Components link from Reporting Task Constroller Services links to proper Controller Settings tab
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4794 .
2021-02-05 23:53:29 +04:00
3f26e54521
NIFI-8194 Updated log level from WARN to DEBUG for various authentication protocols not being configured
2021-02-05 13:20:37 -05:00
b55998afc1
NIFI-8081 Added new Listing Strategy to ListFTP and ListSFTP: Time Window
...
NIFI-8081 Added new Listing Strategy to ListFTP and ListSFTP: Adjusted Time Window. User can specify the time zone or time difference (compared to where NiFi runs) of the system hosting the files and based on the calculates the current time there. Lists files modified before this adjusted current time (and after the last listing).
NIFI-8081 'Time Adjustment' validated not to be set if listing strategy is not 'Adjusted Time Window'. Extracted validator to a separate class. Added more tests. Minor refactor. Typo fix.
NIFI-8081 Improved validation.
NIFI-8081 'Time Adjustment' is not necessary - in fact it can cause problems. SFTP (and usually FTP - which has a more general bug at the moment) returns a timestamp that doesn't really need adjustment. (SFTP in particular returns the an 'epoch' time.) Everything remains the same - the new listing strategy relies on a sliding time window, but without the unnecessary option to adjust for the modification time.
NIFI-8081 Resolved conflicts after rebasing to main.
NIFI-8081 Renamed 'AbstractListProcessor.listByAdjustedSlidingTimeWindow' to 'listByTimeWindow'. Post main rebase correction.
NIFI-8081 Updated user doc for the BY_TIME_WINDOW strategy to warn user on it's reliance of accurate time.
This closes #4721 .
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2021-02-04 17:04:37 +01:00
c1f88ec740
NIFI-8183 TailFile intermittently creates records with ascii NULL after rollover
...
This closes #4792 .
Signed-off-by: Mark Payne <markap14@hotmail.com>
2021-02-03 13:09:00 -05:00
6e1f737c53
NIFI-1355 Implemented new methods in KeyStoreUtils to programmatically-generate certificates, Keystores, and Truststores and return it wrapped in a TLS configuration.
...
Updated TestInvokeHTTP, TestInvokeHttpSSL, TestInvokeHttpTwoWaySSL, and TestListenHTTP to use new Keystore functionality.
NIFI-1355 Refactored and removed unnecessary unit tests in KeyStoreUtilsGroovyTest.
NIFI-1355 Added a password requirement when creating a new truststore.
Handled exception when loading a passwordless truststore type of Bouncy Castle PKCS12.
This closes #4801
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-02-03 08:53:47 -06:00
5a8b18b12d
NIFI-8123: Added support for --wait-for-init when NiFi started with the 'start' command
...
This closes #4748 .
Signed-off-by: Mark Payne <markap14@hotmail.com>
2021-02-03 09:40:09 -05:00
d826416217
NIFI-8192 updating Copyright years for nifi things to include 2021
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4802 .
2021-02-03 12:56:28 +04:00
77eb8af275
NIFI-8187: Fixed PutSQL duplicating provenance events
...
Also fixed provenance event mocking in test framework.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #4796
2021-02-02 19:02:02 -05:00
e0a8b479fd
NIFI-8190 Protect against property that references missing controller service
2021-02-01 15:47:52 -05:00
b4e213cb2c
NIFI-8186 Excluded bcprov-ext-jdk15on from spring-security-saml2-core dependency ( #4793 )
2021-02-01 13:21:46 -05:00
29a1d6badc
NIFI-7987: Support ignore operations in the PutKudu processor
2021-02-01 12:10:19 -05:00
d72bfc76b5
NIFI-8155 - add banner text in page title
...
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #4768
2021-01-29 19:20:34 -05:00
5e84cf6ff9
NIFI-8175: Add WindowsEventLogReader controller service
...
This closes #4785
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-01-29 15:16:19 -06:00
11e9ff3773
NIFI-8178 This closes #4787 . Replaced StandardSSLContextService in unit tests with SslContextUtils
...
- Removed references to StandardSSLContextService from nifi-standard-processors
- Removed TestGetHTTPGroovy and TestPostHTTPGroovy since these are testing deprecated processors
- Optimized HandleHttpRequest, GetHTTP, PostHTTP to use SSLContextService.createContext()
NIFI-8178 Changed TestGetHTTP to ITGetHTTP since GetHTTP is deprecated
NIFI-8178 Changed TestPostHTTP to ITPostHTTP since PostHTTP is deprecated
Signed-off-by: Joe Witt <joewitt@apache.org>
2021-01-27 17:48:06 -07:00
8c1fe84f62
NIFI-8177 This closes #4786 . Added timeout to Sender Pool polling to avoid blocking indefinitely
...
Signed-off-by: Joe Witt <joewitt@apache.org>
2021-01-27 15:04:23 -07:00
04fb1aca47
NIFI-8142: Add "Insert Ignore" option to PutDatabaseRecord
...
NIFI-8142: Fix Checkstyle error
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #4778
2021-01-27 11:50:40 -05:00
813ac539b3
NIFI-7263 This closes #4167 . Add No Tracking Listing Strategy to ListFile/ListFTP processors
...
Signed-off-by: Joe Witt <joewitt@apache.org>
2021-01-26 22:13:26 -07:00
4ff9cddf15
NIFI-8173, NIFI-8174: This closes #4784 . Updated Parameter Contexts to allow for unsetting a parameter value / setting it to null. Allow Parameters to make use of Expression Language. Updated docs to illustrated how EL is evaluated
...
Signed-off-by: Joe Witt <joewitt@apache.org>
2021-01-26 21:55:18 -07:00
abb6ed3128
NIFI-8171 This closes #4779 . Upgraded Bouncy Castle libraries to 1.68 and centralized dependency version
...
NIFI-8171 Increased response and idle timeouts for HTTP unit tests
NIFI-8171 Increased TestServer idle timeout to 45 seconds for HTTP unit tests
NIFI-8171 Adjusted timeout and sleep on TestPutTCPCommon.testPruneSenders
NIFI-8171 Increased TestServer idle timeout to 60 seconds and removed 500ms Thread.sleep() in TestInvokeHttpSSL
NIFI-8171 Optimized OkHttpClientUtils to avoid reading trust store twice during initialization
NIFI-8171 Added static variable for server startup sleep
NIFI-8171 Increased TestInvokeHTTP Connect Timeout and TestListenHTTP Response Timeout to 30 seconds
NIFI-8171 Refactored unit tests for InvokeHTTP and ListenHTTP to optimize SSLContext creation
NIFI-8171 Updated TestListenHTTP for static creation of SSLContext
NIFI-8171 Added started check for ListenHTTP Server in TestListenHTTP
NIFI-8171 Refactored TestPutTCP classes to optimize SSLContext creation
NIFI-8171 Increased TestListenHTTP timeout for server start to 120 seconds and added exception when not connected
NIFI-8171 Increased Connect and Read Timeouts for InvokeHTTP SSL unit tests
Signed-off-by: Joe Witt <joewitt@apache.org>
2021-01-26 21:24:07 -07:00
2cdb0fb6a3
NIFI-8088 Removed deprecation warning log for PKCS12 trust stores
...
Signed-off-by: Nathan Gough <thenatog@gmail.com>
This closes #4727 .
2021-01-26 16:50:33 -05:00
67d06003b7
NIFI-8023: Convert java.sql.Date between UTC/local time zone normalized forms before/after database operations
...
This closes #4781
Signed-off-by: David Handermann <exceptionfactory@gmail.com>
2021-01-26 14:39:02 -06:00
0a10557dd5
NIFI-7801 Adding support for HTTP based Splunk put and indexed acknowledgement
...
This closes #4714 .
Signed-off-by: Mark Payne <markap14@hotmail.com>
2021-01-26 15:20:20 -05:00
f2a16cd02e
NIFI-8163: When counting number of components, we traverse into all Process Groups, but then call findAllRemoteProcessGroups, which is a recursive call, instead of calling getRemoteProcessGroups(). This results in counting the Process Groups many times. So fixed that.
...
Signed-off-by: Nathan Gough <thenatog@gmail.com>
This closes #4775 .
2021-01-21 15:09:42 -05:00
76648bdc0b
NIFI-7356 - Config TLS for embedded ZooKeeper when NiFi TLS enabled.
...
NIFI-7356 - Addresses PR feedback.
NIFI-7356 - Additional changes from PR feedback.
NIFI-7356 - Adding integration tests for ZooKeeperStateServer for TLS.
NIFI-7356 - TLS + Zookeeper now working with single and quorum. Needs code cleanup, need to fix IT tests and docs.
NIFI-7356 - Fixed up tests and removed some irrelevant ones. Refactored some of ZooKeeperStateServer. Tested successfully with a secure and insecure 3 node NiFi + Quorum.
NIFI-7356 - Checkstyle fixes.
NIFI-7356 - Updated administration guide with embedded ZooKeeper TLS configuration.
NIFI-7356 - Updated the way ZooKeeper TLS properties are set/mapped from NiFi properties.
NIFI-7356 - Updated per review, using NiFiProperties keystore strings, classname for ocnnection factory, adjusted TLS configuration checks in NiFiProperties.
NIFI-7356 - Updated configuration validation logic and added tests.
NIFI-7356 - Codestyle check fixes.
NIFI-7356 - Updated some of the log messages.
NIFI-7356 - Updated as per code review.
NIFI-7356 - Fixed max port number.
NIFI-7356 - Updated admin guide and updated small code issues as per code review.
Signed-off-by: Nathan Gough <thenatog@gmail.com>
This closes #4753 .
2021-01-21 13:47:53 -05:00
6741317cc4
NIFI-8162: Fixed outdated docs. Self-merging as this is a trivial docs fix.
...
Signed-off-by: Mark Payne <markap14@hotmail.com>
2021-01-21 12:52:30 -05:00
8c49f0688b
NIFI-7243 Upgraded com.hierynomus.sshj to 0.30.0 to resolve authentication race condition in version 0.27.0
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4774 .
2021-01-21 21:21:47 +04:00
525e4105eb
NIFI-8095: Created StatelessNiFi Sink Connector and Source Connector. Minor updates to stateless nifi api to accommodate.
...
Self-merging PR based on comments from @joewitt
2021-01-21 10:56:28 -05:00
93a5823f8a
NIFI-7890 - Added record support to ConsumeMQTT processor
...
This closes #4738 .
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2021-01-21 11:16:07 +00:00
953327cdf5
NIFI-8156 Fixed byte handling bug in cassandra.
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4771 .
2021-01-21 11:23:14 +04:00
fb2a8b5820
NIFI-8043: Quote update key column names in PutDatabaseRecord
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4772 .
2021-01-21 11:17:13 +04:00
27f57e6463
NIFI-8150 Change Download flow to Download flow definition for process groups ( #4766 )
2021-01-20 16:00:41 -05:00
803ba882aa
NIFI-8146: Ensure that we close the Connection/Statement/PreparedStatement objects in finally blocks or try-with-resources
...
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #4770
2021-01-20 13:48:57 -05:00
25ab050ed7
NIFI-8149: Updated Apache Calcite version from 1.21.0 to 1.26.0
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4765 .
2021-01-20 13:04:08 +04:00
88de03aebb
NIFI-7895 - Fix NPE in ConsumeMQTT with truststore only SSL CS
...
This closes #4587 .
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2021-01-20 00:29:30 +01:00
930e8d9e0e
NIFI-7698: Added a merge.reason attribute for flowfiles indicating why they were merged by MergeContent. Also updated logs to indicate the reason and added additional documentation for processor
...
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #4764
2021-01-19 17:20:06 -05:00
b4ddeb8997
NIFI-8116 The old peers do not be deleted when the URIs of the remote process group are changed
...
This closes #4745 .
Signed-off-by: Mark Payne <markap14@hotmail.com>
2021-01-19 16:25:09 -05:00
f02079c2fa
NIFI-8146: Added RecordPath for Statement Type and to specify where to find the data within the incoming Record. Also performed some minor code refactoring to avoid lots of large lambdas, as it results in code that is more readable and more maintainable. Updated some properties to use dependsOn() notation now that this is available.
...
NIFI-8146: Updated PutDatabaseRecord to avoid using the functional style framework that had previously been used. Doing so resulted in code that was difficult to understand and maintain. Added additional unit tests and improved MockRecordParser so that it could throw configurable types of exceptions
NIFI-8146: Fixed checkstyle issues
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #4763
2021-01-19 13:23:54 -05:00
3e72dfe421
NIFI-8109 - Update MongoDBControllerService to externalize username and password
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4759 .
2021-01-19 21:21:44 +04:00
40f193098b
NIFI-7726 correct versionedFlowState field name in SiteToSiteStatusReportingTask schema
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4762 .
2021-01-19 10:40:38 +04:00
097edf4f7c
NIFI-8136: Added getState/setState/replaceState/clearState methods to ProcessSession, updated processors to use these methods instead of StateManager version where appropriate
...
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #4757
2021-01-15 10:02:09 -05:00
b9076ca26e
NIFI-7989: Add Update Field Names and Record Writer to UpdateHiveTable processors
...
NIFI-7989: Only rewrite records if a field name doesn't match a table column name exactly
NIFI-7989: Rewrite records for created tables if Update Field Names is set
This closes #4750 .
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2021-01-13 23:26:44 +01:00
3cc8d767b3
NIFI-7738 Reverse Provenance Query
...
This closes #4563 .
Signed-off-by: Mark Payne <markap14@hotmail.com>
2021-01-13 09:35:05 -05:00
382439c1d0
NIFI-7973: Add default precision and scale properties to remaining SQL-based components
...
This closes #4682 .
Signed-off-by: Mark Payne <markap14@hotmail.com>
2021-01-13 08:41:12 -05:00
0b966cdb14
NIFI-7991 Flow Configuration History displays "annotation data not found/available" from "Advanced" changes
...
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #4668
2021-01-12 16:09:38 -05:00
2d69179184
NIFI-8138: Add Sensitive Property value to property tooltip
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4752 .
2021-01-13 00:44:03 +04:00
7d76bcd520
NIFI-8094 Added support for BCFKS Keystore Type
...
NIFI-8094 Updated Administration Guide to include BCFKS
Signed-off-by: Nathan Gough <thenatog@gmail.com>
This closes #4729 .
2021-01-12 10:52:49 -05:00
82c84492ce
NIFI-7906 Fixed a bug in how record paths were handled.
...
NIFI-7906 Fixed bugs in unit tests.
NIFI-7906 Updated a few things based on a code review.
NIFI-7906: Fixed typo in record.count attribute
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #4739
2021-01-12 10:32:08 -05:00
5f7558cecf
NIFI-8069 Set invalid status when Controller Services are enabling
...
This closes #4710 .
Signed-off-by: Mark Payne <markap14@hotmail.com>
2021-01-11 13:42:52 -05:00
84b561ad33
NIFI-8131: Support aws_s3_v2_directory in Atlas reporting task
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4751 .
2021-01-11 21:53:12 +04:00
d41f2e1d0a
NIFI-8121 Updated ListenHTTP with inferred Client Authentication Policy
...
- Added default property value for automatic determination of Client Authentication Policy based on SSLContextService Trust Store properties
- Added new ClientAuthentication enum with values specific to ListenHTTP
This closes #4749 .
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2021-01-11 18:02:09 +01:00
a4027e8e77
NIFI-8120 Added RuntimeException handling on HttpContextMap.complete()
...
NIFI-8120 Renamed exception variable and reordered log statements
This closes #4747 .
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2021-01-08 17:13:10 +00:00
f330078fff
NIFI-7924 Add fallback claims for identifying user to OIDC provider
...
This closes #4630
Signed-off-by: Joey Frazee <jfrazee@apache.org>
2021-01-07 14:57:57 -06:00
817f621d6f
NIFI-8096 Deprecated ClientAuth references in SSLContextService
...
- Added SSLContextService.createContext() and refactored referencing components
- Removed references to ClientAuth from SslContextFactory methods
This closes #4737 .
Signed-off-by: Mark Payne <markap14@hotmail.com>
2021-01-06 16:24:26 -05:00
783633cac5
NIFI-8021: Fixed bug in ConsumeKafka_2_6 and ConsumeKafkaRecord_2_6 where explicit partition assignment causes issues with more than 1 concurrent task. Also fixed bug that prevented more nifi nodes than partitions because it didn't properly handle empty string for the list of partitions
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4744 .
2021-01-06 19:28:09 +01:00
ea80dad16a
NIFI-8021: Fixed bug in ConsumeKafka_2_0 and ConsumeKafkaRecord_2_0 where explicit partition assignment causes issues with more than 1 concurrent task. Also fixed bug that prevented more nifi nodes than partitions because it didn't properly handle empty string for the list of partitions
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4744 .
2021-01-06 19:27:33 +01:00
7214dc0f85
NIFI-6242 PutFileTransfer generating incorrect provenance event
2021-01-06 12:10:20 -05:00
954c09d0c3
NIFI-7387: update ComponentLog API to accept Object varargs instead of array
...
This closes #4225 .
Signed-off-by: Mark Payne <markap14@hotmail.com>
2021-01-06 11:09:07 -05:00
80eb570ee1
NIFI-7894 - ConsumeMQTT - allow EL on Client ID property with shared subscription
...
This closes #4586 .
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2021-01-06 16:46:17 +01:00
c9d778a8ee
NIFI-7889 - ConsumeMQTT - use offer instead of add
...
This closes #4578 .
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2021-01-06 14:15:27 +01:00
1e227ca643
NIFI-7964 Force PutAzureBlobStorage to stream writes
...
NIFI-7964 Remove unused import
NIFI-7964 Reverted to using BufferedInputStream along with unmarkable
NIFI-7964 Made UnmarkableInputStream static and added factory method
NIFI-7964 Remove unnecessary factory method from UnmarkableInputStream
This closes #4632 .
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2021-01-06 13:47:05 +01:00
bb178f371b
NIFI-6576 Added Basic Authentication support to ConfluentSchemaRegistry
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4743 .
This closes #4508 .
This closes #4224 .
2021-01-06 09:34:43 +01:00
584adc3b91
NIFI-8067 - Fix 1-way SSL in GRPC processors
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4733 .
2021-01-05 11:31:00 +01:00
ceb9dff3b9
NIFI-7761 Allow HandleHttpRequest to add specified form data to FlowFile attributes
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4513 .
2021-01-05 09:42:41 +01:00
e7c6bdad42
NIFI-7937 Added StandardFlowFileMediaType enum to replace string references to FlowFile Media Types
2021-01-04 14:20:05 -05:00
0f667cb4f3
MonitorActivity THRESHOLD to use Variables
...
When using MonitorActivity, it would be interesting to use variables in "Threshold Duration", which will help maintain Version Control in ProcessGroups (Avoiding "breaking" versions when including the value manually).
I've included the option expressionLanguageSupported(ExpressionLanguageScope.VARIABLE_REGISTRY), so we can use variables in "Threshold Duration".
2021-01-04 14:14:29 -05:00
112b893bf2
NIFI-7225 FetchSFTP processor routing to not.found error given when Private Key Path property is invalid
2021-01-04 13:41:03 -05:00
5c79553af6
NIFI-8062: make tabs functional with keyboard
2021-01-04 11:58:47 -05:00
de88ce7a61
NIFI-7906: Implemented RecordSetWriter support for ExecuteGraphQueryRecord
...
This closes #4704
Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-12-23 12:41:42 -05:00
31c5dc5ab4
NIFI-8102 TailFile can skip characters if tailing a file on NFS mount
...
- Force the TailFile processor to recreate and reposition the reader
by setting it to null in case of a NulCharacterEncounteredException
- Updated the TestTailFile.testNULContent() to not initialize the
processor when calling the second run()
Update the reader's position instead of abandoning it.
This closes #4736 .
Signed-off-by: Mark Payne <markap14@hotmail.com>
2020-12-18 14:20:41 -05:00
942030579a
NIFI-7386 Added Azure Azurite emulator credentials service
...
This closes #4286
Signed-off-by: Joey Frazee <jfrazee@apache.org>
2020-12-16 18:11:06 -06:00
989287adaf
NIFI-8027 In AbstractExecuteSQL make 'SQL Pre-Query' and 'SQL Post-Query' properties support escaping semicolons to allow them be part of statements.
...
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #4732
2020-12-16 14:57:59 -05:00
02665d7561
NIFI-8092: Fix ITPutS3Object test failure due to Outposts storage class
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4728 .
2020-12-16 12:43:24 +01:00
f29d6a6046
NIFI-7989: Add support to UpdateHiveTable for creating external tables
...
NIFI-7989: Add support for creating partitions, quote identifiers
NIFI-7989: Quote table name when getting description
This closes #4697 .
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-12-15 22:05:53 +01:00
28ca7478d6
NIFI-8019 Added TlsPlatform to provide runtime TLS protocol configuration
...
NIFI-8019 Renamed getDefaultProtocols() to getSupportedProtocols()
Signed-off-by: Nathan Gough <thenatog@gmail.com>
This closes #4673 .
2020-12-14 19:34:58 -05:00
5bcb5abc86
NIFI-1930 Updated ListenHTTP to set TLS included protocols based on SSLContextService
...
NIFI-1930 Removed unused import in SSLContextServiceTest
Signed-off-by: Nathan Gough <thenatog@gmail.com>
This closes #4673 .
2020-12-14 14:31:08 -05:00
a73cd6a610
NIFI-7856: If a Provenance Event File is ready to be rolled over due to the maximum amount of time having elapsed, avoid rolling over unless there is at least one event written to the event file. Otherwise, we can have multiple RecordWriters / RecordWriterLeases pointing to the same file. This can result in data being overwritten, as well as failing to compress the event file upon rollover. Also added significant DEBUG/TRACE level logging.
...
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #4580
2020-12-14 11:49:47 -05:00
91f6b42985
NIFI-8085 Use poll(Duration) in ConsumeKafka_2_x processors
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4725 .
2020-12-13 10:23:35 +01:00
ec3d5f89f0
NIFI-8086: Fix file separator char in PutS3Object multipart state directory path
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4726 .
2020-12-13 10:18:32 +01:00
7bff64b3cf
NIFI-7913 Added getEnabledProtocols() to TlsConfiguration and updated ListenSMTP to set enabled protocols on SSL Sockets
...
NIFI-7913 Changed order of supported protocols to match existing comments in SSLContextService
This closes #4599
Signed-off-by: Nathan Gough <thenatog@gmail.com>
2020-12-10 11:53:20 -05:00
8e4948322e
NIFI-8074 Switched ReplaceText to use line by line evaluation as its default evaluation mode.
...
This closes #4711
Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-12-10 10:30:11 -05:00
1f8b4e4779
NIFI-8080: Move script compilation to configurator init method
...
This closes #4718
Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-12-09 21:05:44 -05:00
1d31434c48
NIFI-8055 Fix validation message class-name for AzureCosmosDBClientService
...
This closes #4695
Signed-off-by: Joey Frazee <jfrazee@apache.org>
2020-12-09 17:13:25 -06:00
8d2fd39bdb
NIFI-8066: Bump GRPC dependency versions ( #4716 )
2020-12-09 09:37:01 -05:00
0af736dc0b
NIFI-7885 Added Environment Variable to deny LFS access using Hadoop ( #4712 )
...
* NIFI-7885 Added Environment Variable to deny LFS access using Hadoop
* NIFI-7885 Changed Hadoop Processor validation to check LFS access
2020-12-09 09:29:42 -05:00
2c0671cf80
NIFI-8080: Compile Jython scripts before evaluating
...
This closes #4717
Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-12-09 08:57:21 -05:00
cfbcecc4c6
NIFI-7884 Added and applied Distributed File System permissions ( #4713 )
2020-12-08 15:56:46 -05:00
039a09923a
NIFI-8073 Correcting isBackPressureEnabled in PrometheusMetricsUtil
2020-12-07 10:03:16 -05:00
d7ca4c920c
NIFI-8073 corrected condition for label isBackPressureEnabled
...
Signed-off-by: Bryan Bende <bbende@apache.org>
2020-12-07 10:03:04 -05:00
405c393cb5
NIFI-8031: Add UPSERT capability for MySQL in PutDatabaseRecord
...
This closes #4678
Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-12-07 08:51:07 -05:00
2f99d6fce4
NIFI-4540 - Added FIFO options to PutSQS
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4705 .
2020-12-07 13:11:22 +01:00
37c2284d04
NIFI-8068: Ensure that when we determine the best of multiple possible types in a UNION that we handle Arrays of Records properly. Also refactored code to be a bit cleaner by extracting blocks of it into appropriately named methods
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4706 .
2020-12-04 19:31:23 +01:00
71d5162965
NIFI-6878: Added 'Use statement.type Attribute' to ConvertJSONToSQL
...
This closes #3893
Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-12-04 07:39:08 -05:00
29ea872f2c
NIFI-8042: Fixed bug that was escaping Expression Language references for use in a Regular Expression (i.e., Pattern.quote) even though it wasn't being used in a Regular Expression
...
This closes #4685
Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-12-04 07:30:09 -05:00
04aaf25131
NIFI-1121: Use display name for dependent property documentation
2020-12-03 16:39:46 -05:00
8055c47a84
NIFI-8058 Fixed a UI error to correctly delete dynamic properties while configuring processors.
...
Changed to check the length of all unfiltered properties instead of only filtered properties.
Added additional check if descriptor is a dynamic property.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #4707
2020-12-03 15:44:56 -05:00
312fa8e85e
NIFI-8064 Convert TestSecureClientZooKeeperFactory to integration test
2020-12-03 14:46:13 -05:00
2b1359a808
NIFI-8060 Addressed review feedback
2020-12-03 14:42:50 -05:00
8ac8a2bd1f
NIFI-8060 Added minimal VolatileProvenanceRepository to nifi-stateless-engine and remove dependency on nifi-volatile-provenance-repo module
2020-12-03 14:42:50 -05:00
b90a6e893d
NIFI-7906 This closes #4701 . Updated test case to fix a windows-centric bug.
...
NIFI-7906 Removed unused test code.
Signed-off-by: Joe Witt <joewitt@apache.org>
2020-12-03 07:22:40 -07:00
c29cced269
NIFI-7906: parameterized graph query
...
NIFI-7906: addressing PR concerns
NIFI-7906: code styling fixes
NIFI-7906: adding in license information to new files
+ enables processor in META-INF
NIFI-7906: exclude test files from RAT
NIFI-7906: PR refactor to streamline graph response
NIFI-7906: removing ERRORS output
Unused after refactor
Did a few cleanups for the contributor.
This closes #4638
Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-12-01 11:13:10 -05:00
64e3599f05
NIFI-7260: Fix error handling and re-evaluate Module Directory property on changed for scripted controller services
...
This closes #4147
Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-12-01 09:42:14 -05:00
1e13b62e78
NIFI-8047 Added documentation for sensitive DBCP properties
...
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #4696
2020-11-30 18:33:32 -05:00
aaa1452d04
NIFI-8054: Updated ReflectionUtils to use a WeakHashMap for the mapping of annotations to methods with that annotation. This way, the ReflectionUtils class will not hold a reference to Classes that are no longer referenced elsewhere. ( #4694 )
2020-11-30 16:14:12 -05:00
fe53f8090d
NIFI-8047 Added support for sensitive dynamic properties in DBCP
...
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #4692
2020-11-30 14:55:20 -05:00
39f8a008d4
NIFI-8039 Adding properties to ListenTCP in order to allow refine behaviour under higher load; Refining thread pool for better scalability
...
NIFI-8039 Review findings; refining thread pool to be able to scale down properly when not under load
NIFI-8039 Answers to PR comments
This closes #4689 .
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-11-30 10:17:15 +01:00
3625a73d91
NIFI-7821 Added Cassandra-based DMC.
...
NIFI-7821 Updated configuration documentation.
NIFI-7821 Fixed getAndPutIfAbsent and added int test.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #4635
2020-11-29 18:58:57 -05:00
fe950131c3
NIFI-8046: Fix issue with ResultSetRecordSet on DB2
...
This closes #8046 .
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-11-27 19:23:44 +01:00
5ea18f39b5
NIFI-8048 PutElasticsearchRecord should retry with 503 response
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4690 .
2020-11-26 10:49:52 +01:00
124cdbd3fe
NIFI-6403 and NIFI-6404: Elasticsearch 7 support
...
Addressed PR#4153 comments; removed ES Version property and made Type optional in all ES HTTP/Record processors, applying sensible default values where required; use _source queyr parameter instead of _source_include/s as it's compatible between ES versions
Fix unit test compilation to use JDK8-compatible library/method
Better optional type and id handling for PutElasticsearchRecord; update nifi-elasticsearch-client-service build dependencies to use latest versions of Elasticsearch in each supported major version (5/6/7); addressed several warnings in ElasticSearchClientServiceImpl
This closes #4667 .
Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2020-11-26 15:08:13 +09:00
a3d845a38f
NIFI-6403: Adding Elasticsearch7 support to HTTP processors
...
This closes #4153 .
Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2020-11-26 15:08:05 +09:00
Peter Turcsanyi
Matthew Burgess
Stan Antyufeev
Christian Gumpert
Peter Gyori
Mark Payne
Pierre Villard
Chris Sampson
exceptionfactory
Bryan Bende
Celso Marques
Lehel Boér
Timea Barna
Grant Henke
exceptionfactory
Tamas Palfy
Zsihovszki Krisztina
Joey Frazee
Joe Witt
Nathan Gough
Mark Bean
Jon Kessler
Denes Arvay
mtien
snorlaxa
Waleed Al Aibani
Bence Simon
Troy Melhase
Mike Thomsen
Andrew Lim
s9514171
eduardofontes
Nissim Shiman
tlsmith
sjyang18
Otto Fowler
Kevin Barranco
Shane Ardell
Sushil Kumar
Peter Turcsanyi
DataMoTrain
r65535
Levi Lentz
Koji Kawamura
Joe Gresock