93b1a05dc3
NIFI-8030: Added property to ReportLineageToAtlas to configure file/directory level logging of [hd]fs_path entities to Atlas
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4813 .
2021-02-25 18:37:19 +04:00
34fc94454f
NIFI-8249: Fixed issue with error during multiple FlowFile results in ExecuteSQL processors
2021-02-24 10:10:32 -05:00
2322b2cddf
NIFI-6113 Refactoring GetHDFSFileInfo to remove instance level variable to make it stateless.
...
This closes #4837 .
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2021-02-24 13:15:12 +01:00
2dedd8bf0f
NIFI-8253: Restore call to session.commit() in GenerateTableFetch
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4840 .
2021-02-24 15:57:16 +04:00
f101a2bba5
NIFI-8198: ConsumeAMQP detects if Queue is deleted on server
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4805 .
2021-02-24 12:04:52 +04:00
f2555f27f1
NIFI-8119: properly free database resources when done with the processing of BLOBs
...
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #4791
2021-02-23 16:47:32 -05:00
d049bca794
NIFI-8205: Documentation improvements for the Wait processor
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4808 .
2021-02-23 21:22:19 +04:00
528fce2407
NIFI-7646, NIFI-8222: Instead of having StandardProcessSession call ContentRepository.read(ContentClaim), introduced a new ContentRepository.read(ResourceClaim) and hold open the InputStream to the ResourceClaim. This can't be supported by EncryptedContentRepository, so introduced a method to allow using this or not. The benefit here is that when we have many FlowFiles read within a session, such as when using MergeContent/MergeRecord or a processor configured with a Run Duration, we can hold open a single InputStream instead of constantly opening FileInputStreams and seeking to the appropriate location. This is much faster.
...
- Instead of entering a 'synchronized' block for every provenance event, serialize up to 1 MB worth of data, and then enter synchronized block to write that data out. This avoids large amounts of lock contention and context switches
NIFI-7646: Removed TODO and unused Jackson dependency
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #4818
2021-02-23 10:57:54 -05:00
200c04c6d0
NIFI-8237: This closes #4835 . Added missing SQL types to getDataTypeFromSQLTypeValue(), added defensive code
...
Signed-off-by: Joe Witt <joewitt@apache.org>
2021-02-23 08:08:09 -07:00
9101160cbf
NIFI-8250 - This closes #4838 . UTF8 encoding for FTP processors
...
Signed-off-by: Joe Witt <joewitt@apache.org>
2021-02-23 08:05:32 -07:00
3cb26aec72
NIFI-8232 CSV Parsers optionally allow/reject duplicate header names
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4828 .
2021-02-23 09:50:18 +04:00
418e2cc2cb
NIFI-8132 Replaced framework uses of MD5 with SHA-256
...
NIFI-8132 Added FileDigestUtils in nifi-nar-utils to avoid dependency on nifi-utils
NIFI-8132 Removed unused imports from NarUnpacker
NIFI-8132 Removed MD5 references from FileUtils documentation
NIFI-8132 Replaced StringBuffer with StringBuilder and made new DigestUtils classes final
NIFI-8132 Replaced Collections.sort() with Stream.sorted()
Signed-off-by: Nathan Gough <thenatog@gmail.com>
This closes #4788 .
2021-02-22 17:52:24 -05:00
ebef823cb9
NIFI-8241 This closes #4833 . Set the SAML context provider to use the EmptyStorageFactory
...
Signed-off-by: Joe Witt <joewitt@apache.org>
2021-02-22 08:53:32 -07:00
703c3cfe63
NIFI-5906 - Removed FILE_EXISTS_VALIDATOR on JARs and Files to be added in the session
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4810 .
2021-02-22 13:04:03 +04:00
80da0cf47d
NIFI-7496: Regex pattern changed to match dashes in filters.
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4811 .
2021-02-22 12:59:03 +04:00
b08d6071dd
NIFI-8224: Add LoggingRecordSink controller service
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4820 .
2021-02-22 12:55:53 +04:00
6ed496c714
NIFI-8242 Ensure NiFi URL for redirects has a trailing slash in AccessResource
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4834 .
2021-02-22 12:48:42 +04:00
51e7bdf940
NIFI-3669 Add SSL Support to CaptureChangeMySQL
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4594 .
2021-02-22 11:29:59 +04:00
59ad21fbaa
NIFI-8200 Modified PutAzureDataLakeStorage to delete temp file if exception was thrown in uploadContent()
...
This closes #4815
Signed-off-by: Joey Frazee <jfrazee@apache.org>
2021-02-21 13:35:28 -08:00
662a35b7de
NIFI-8240: Unify Kudu versions
...
The Kudu versions for the controller service and processors
currently doesn’t match. This patch unifies the version in
the parent bundle pom to ensure they match going forward.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4832 .
2021-02-19 20:42:30 +04:00
cd6418cdaf
NIFI-8238: When FlowFile Concurrency/Outbound Policy is changed on child group, and child group is directly under version control, ensure that those fields are maintained on the generated VersionedProcessGroup when nested components are stripped out
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4831 .
2021-02-19 14:44:42 +04:00
f532b3ae1d
NIFI-5623 Upgraded OkHttp3 to 4.9.1 and updated unit tests
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4826 .
2021-02-19 14:42:16 +04:00
aa726040c5
NIFI-8188 - Add 'Run Once' for processors in context menu.
...
Signed-off-by: Mark Payne <markap14@hotmail.com>
2021-02-18 14:51:28 -05:00
6801704846
NIFI-8235 UnsupportedOpertationException in PutAccumuloRecord
2021-02-18 13:12:09 -05:00
042b34cb43
NIFI-8231 Clarify behavior of allowed values for Memory Pools in MonitorMemory
2021-02-18 09:58:48 -05:00
fd50d6f042
NIFI-3383 Check if usage operations are supported in MonitorMemory
...
This closes #4790
Signed-off-by: Joey Frazee <jfrazee@apache.org>
2021-02-16 17:25:28 -06:00
a69313e34c
NIFI-7873 updating docker files
2021-02-15 12:12:51 -07:00
88fab00e29
NIFI-7873 merging release branch to latest and updating to 1.14.0-SNAPSHOT
2021-02-15 12:09:32 -07:00
4afb2ba743
NIFI-7873-RC4 prepare for next development iteration
2021-02-15 12:09:31 -07:00
487280bee9
NIFI-7873-RC4 prepare release nifi-1.13.0-RC4
2021-02-15 12:09:30 -07:00
a1c82a66a1
NIFI-7361 - increase actions column width for reporting tasks
...
This closes #4214
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-02-15 10:42:19 -06:00
d08f02428d
NIFI-8223: This closes #4819 . Use column datatype in PutDatabaseRecord when calling setObject()
...
Signed-off-by: Joe Witt <joewitt@apache.org>
2021-02-10 18:01:43 -07:00
8057f8f6c5
NIFI-8221 - Set the default HTTP listening interface to 127.0.0.1.
...
This closes #4817
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-02-10 14:46:37 -06:00
1d82fb8e01
NIFI-8218 This closes #4816 . Use proxy headers when available when getting request values while processing SAML responses
...
Signed-off-by: Joe Witt <joewitt@apache.org>
2021-02-10 13:34:57 -07:00
d5d520764d
NIFI-8210: When Index Reader/Searcher is used, do not allow the Lucene Index to be deleted until the reader/searcher is finsihed being used and closed.
...
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #4812
2021-02-09 15:32:15 -05:00
b9b131239c
NIFI-8215 --wait-for-init in nifi.sh refactored for POSIX sh
...
This fixes #4814
Signed-off-by: Joey Frazee <jfrazee@apache.org>
2021-02-09 11:10:15 -08:00
13d6da73ac
NIFI-7818 - bump hadoop-client to 3.3.0
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4604 .
2021-02-09 21:22:59 +04:00
4ebeb4e146
NIFI-8181 - Added Disable HTTP/2 property to InvokeHTTP
...
This closes #4804
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-02-08 14:53:35 -06:00
40d8c41656
NIFI-8193: Fixed PutSQL reordering FlowFiles
...
Also removed sorting of transferred MockFlowFiles in test framework.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #4803
2021-02-05 16:17:10 -05:00
749d05840b
NIFI-8204, NIFI-7866: Send revision update count in heartbeats. If update count in heartbeat is greater than that of cluster coordinator, request that node reconnect to get most up-to-date revisions. Cannot check exact equality, as the values may change between the time a heartbeat is created and the time the cluster coordinator receives it. However, it should be safe to assume that the revision won't be greater than that of the cluster coordinator. There is a tiny window in which it could be, as the sending node may update its revision, create the heartbeat, send it, and cluster coordinator process it before updating its own revision. However, this window is incredibly small and would only result in the sending node reconnecting, which will resolve itself. Also, when testing this fix, encountered NIFI-7866 and addressed that NullPointerException.
...
This closes #4806 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2021-02-05 15:18:44 -05:00
03fd59eb2f
NIFI-8196: When node is reconnected to cluster, ensure that it re-registers for election of cluster coordinator / primary node. On startup, if cluster coordinator is already registered and is 'this node' then register silently as coordinator and do not join the cluster until there is no Cluster Coordinator or another node is elected. This allows the zookeeper session timeout to elapse.
...
Signed-off-by: Bryan Bende <bbende@apache.org>
2021-02-05 15:18:07 -05:00
b77dbd5030
NIFI-8172: Provide schema name to getPrimaryKeys call in PutDatabaseRecord
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4782 .
2021-02-06 00:00:58 +04:00
a9b8635ac9
NIFI-6522: Referencing Components link from Reporting Task Constroller Services links to proper Controller Settings tab
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4794 .
2021-02-05 23:53:29 +04:00
07dfdff389
NIFI-8191: Make the nifi-kafka-connector-assembly use attach=true, avoid appending assembly id, and stop producing the 'dir' format since it was not used
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4799 .
2021-02-05 23:46:54 +04:00
3f26e54521
NIFI-8194 Updated log level from WARN to DEBUG for various authentication protocols not being configured
2021-02-05 13:20:37 -05:00
b55998afc1
NIFI-8081 Added new Listing Strategy to ListFTP and ListSFTP: Time Window
...
NIFI-8081 Added new Listing Strategy to ListFTP and ListSFTP: Adjusted Time Window. User can specify the time zone or time difference (compared to where NiFi runs) of the system hosting the files and based on the calculates the current time there. Lists files modified before this adjusted current time (and after the last listing).
NIFI-8081 'Time Adjustment' validated not to be set if listing strategy is not 'Adjusted Time Window'. Extracted validator to a separate class. Added more tests. Minor refactor. Typo fix.
NIFI-8081 Improved validation.
NIFI-8081 'Time Adjustment' is not necessary - in fact it can cause problems. SFTP (and usually FTP - which has a more general bug at the moment) returns a timestamp that doesn't really need adjustment. (SFTP in particular returns the an 'epoch' time.) Everything remains the same - the new listing strategy relies on a sliding time window, but without the unnecessary option to adjust for the modification time.
NIFI-8081 Resolved conflicts after rebasing to main.
NIFI-8081 Renamed 'AbstractListProcessor.listByAdjustedSlidingTimeWindow' to 'listByTimeWindow'. Post main rebase correction.
NIFI-8081 Updated user doc for the BY_TIME_WINDOW strategy to warn user on it's reliance of accurate time.
This closes #4721 .
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2021-02-04 17:04:37 +01:00
c1f88ec740
NIFI-8183 TailFile intermittently creates records with ascii NULL after rollover
...
This closes #4792 .
Signed-off-by: Mark Payne <markap14@hotmail.com>
2021-02-03 13:09:00 -05:00
6e1f737c53
NIFI-1355 Implemented new methods in KeyStoreUtils to programmatically-generate certificates, Keystores, and Truststores and return it wrapped in a TLS configuration.
...
Updated TestInvokeHTTP, TestInvokeHttpSSL, TestInvokeHttpTwoWaySSL, and TestListenHTTP to use new Keystore functionality.
NIFI-1355 Refactored and removed unnecessary unit tests in KeyStoreUtilsGroovyTest.
NIFI-1355 Added a password requirement when creating a new truststore.
Handled exception when loading a passwordless truststore type of Bouncy Castle PKCS12.
This closes #4801
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-02-03 08:53:47 -06:00
5a8b18b12d
NIFI-8123: Added support for --wait-for-init when NiFi started with the 'start' command
...
This closes #4748 .
Signed-off-by: Mark Payne <markap14@hotmail.com>
2021-02-03 09:40:09 -05:00
d826416217
NIFI-8192 updating Copyright years for nifi things to include 2021
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #4802 .
2021-02-03 12:56:28 +04:00
Peter Turcsanyi
Matthew Burgess
Stan Antyufeev
Christian Gumpert
Peter Gyori
Mark Payne
Pierre Villard
Chris Sampson
exceptionfactory
Bryan Bende
Celso Marques
Lehel Boér
Timea Barna
Grant Henke
exceptionfactory
Tamas Palfy
Zsihovszki Krisztina
Joey Frazee
Joe Witt
Nathan Gough
Mark Bean
Jon Kessler
Denes Arvay
mtien