Commit Graph

4321 Commits

Author SHA1 Message Date
Mark Payne dc94ef568a
NIFI-8299: Because we now iterate over stored events in an unordered map, we need to update maxEventId to Math.max( <current value>, <next value> ) instead of just assigning the next value.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4873.
2021-03-09 11:44:51 +04:00
exceptionfactory 98ab42aebc NIFI-8302 Corrected FingerprintFactory to use decrypt() method removing wrapping
Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #4876.
2021-03-08 22:16:21 -05:00
Mark Payne b523698534
NIFI-8212: Refactored StandardExtensionDiscoveringManager to avoid using ServiceLoader
Instead, it will look at the ServiceLoader file and read the names of the classes but avoid instantiating all of the objects or loading the classes into memory.
- Updated Doc Generation so that if the documentation for a given NAR already exists, it doesn't delete it and re-generate it. This was necessary because we are no longer instantiating an instance of each component and instead lazily creating the components as necessary.
- Removed stateless version of extension registry because it's no longer necessary

This closes #4852

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-03-08 12:21:19 -06:00
Matthew Burgess 0965523bab NIFI-8266: Fix NPE when validating supported properties against validation context
- Removed schema properties from FreeFormTextRecordSetWriter causing validation issues

This closes #4851.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2021-03-08 11:30:13 -05:00
Pierre Villard c5b5be89a2
NIFI-8267 - Fix for HWX Schema Registry to use integers (#4850) 2021-03-08 09:12:37 -05:00
tpalfy f2a03fca2a
NIFI-8285 Prevent HBase client services to throw NPE in non-kerberized environment. (#4868)
* NIFI-8285 Prevent HBase client services to throw NPE in non-kerberized environment.

* NIFI-8285 Improve exception handling.
2021-03-08 09:10:12 -05:00
Matthew Burgess 29ebec071e
NIFI-4974: Add try/catch around blob.free() to support JDBC spec
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4861.
2021-03-07 18:51:07 +04:00
Mark Payne e2e137fced
NIFI-8291: Support escaping unicode characters for fields that previously allowed escaping only of \t, \n\, \r
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4871.
2021-03-03 21:26:18 +04:00
mtien b9a0991157
NIFI-8260 [WIP] Created Upload File front end and server side logic.
NIFI-8260 [WIP] Fixed server side logic to upload a flow file. Cleaned up the front end logic.
NIFI-8260 [WIP] Finished the server side upload logic.
Added a client ID parameter to the endpoint.
Added JSON parsing error response.
Fixed the client side file form to reset after submit.
Fixed the canvas to instantly update and show the process group after submitting the file.
Changed the Add Processor Group dialog UI based on design notes.
Changed the Upload File link to an icon and moved to the process group name input.
Changed the Registry Import link to say 'Import from Registry' and moved to the bottom of the dialog.
Display the filename when a file is selected.

NIFI-8260 [WIP] Added a cancel file button to the Process Group dialog.
Fixed some CSS styles.

NIFI-8260 - Removed accessing the snapshot metadata to avoid an NPE.
Added a title attribute to the html of the dialog file cancel button.

NIFI-8260 - Disabled the dialog 'Add' button.
Revised based on PR feedback.
Refactored the upload file endpoint and client side filename extraction methods.
Fixed some CSS.
Reverted some unnecessary changes.

NIFI-8260 - Revised based on PR feedback.
Refactored uploadProcessGroup.
Fixed some exception handling.
Hid the Upload File button when grouping components.
Refactored nf-ng-group-component.js replacing jquery selectors with variables.
Extracted the resetValues function to clear dialog values.

NIFI-8260 - Fixed the cluster replicate request.
Created a new endpoint to handle the cluster replicate request.
Created ProcessGroupUploadEntity.
Renamed positionX and positionY parameters.

NIFI-8260 - Fixed a checkstyle error.
Removed unnecessary httpServletRequest parameter.
Reverted some re-ordering of imports.

NIFI-8260 - Changed the ProcessGroupUploadEntity to be consistent with other Entity and DTO models.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4846.
2021-03-03 14:38:45 +04:00
Nathan Gough c5abf2ba54
NIFI-7127 - Allow choosing a SecureHasher for Flow Fingerprint checking
- Configuration based on Sensitive Properties Algorithm defaults to Argon2
- Added SensitiveValueEncoder interface
- Standard implementation uses existing approach with HmacSHA256

This closes #4867

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-03-02 21:38:30 -06:00
Matthew Burgess 0d473bacf6
NIFI-8270: Fixed issue to show property dependent on default value of another
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4860.
2021-03-02 12:41:50 +04:00
Peter Turcsanyi 7876f412fb
NIFI-8258: Add support for Service Principal authentication in ADLS processors
- Removed Expression Language support indicators from sensitive properties

This closes #4843

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-03-01 22:34:59 -06:00
Pierre Villard ea8727a278
NIFI-8274 - add EL consideration in XXEValidator
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4859
2021-03-01 16:34:24 -05:00
exceptionfactory bbd37b8db7
NIFI-8264 Replaced commons-logging with jcl-over-slf4j in framework modules
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4848
2021-03-01 15:18:42 -05:00
Peter Gyori 4a27b23b1f
NIFI-8263: Maximum Thread Pool Size property introduced in ListenHTTP
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4847.
2021-03-01 20:14:34 +04:00
Peter Gyori a54f739229 NIFI-7969: ValidateRecord enhanced with Force Types From Schema property
NIFI-7969: Documentation update
Clarified that the Force Types From Schema property applies to the data read, whereas the Strict Type Checking property applies to the validation.
NIFI-7969: Documentation update - updated the property name in additionalDetails.html

This closes #4825.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2021-03-01 16:04:11 +01:00
exceptionfactory 5608f4389a NIFI-7668 Implemented support for additional AEAD property encryption methods
- Added support for PBKDF2 and Scrypt property encryption methods in addition to Argon2
- Refactored StringEncryptor class to PropertyEncryptor interface with implementations
- Added PasswordBasedCipherPropertyEncryptor and KeyedCipherPropertyEncryptor
- Replaced direct instantiation of encryptor with PropertyEncryptorFactory
- Refactored applicable unit tests to use mocked PropertyEncryptor

NIFI-7668 Consolidated similar methods to CipherPropertyEncryptor

NIFI-7668 Updated AbstractTimeBasedSchedulingAgent with PropertyEncryptor

NIFI-7668 Added support for bcrypt secure hashing algorithm

NIFI-7668 Updated comments to clarify implementation of bcrypt key derivation

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #4809.
2021-02-25 17:00:32 -05:00
Bence Simon 7730777d66 NIFI-8113 Adding persistent status history repository backed by embedded QuestDB
Signed-off-by: Mark Payne <markap14@hotmail.com>
2021-02-25 15:59:01 -05:00
Mark Payne 82eedea659
NIFI-8261: When gathering the states of affected components, make sure that we properly obtain the state of Input Ports and Output Ports
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4844.
2021-02-25 22:21:03 +04:00
Koji Kawamura 68d38dd0a6
NIFI-6752 Add ASN.1 RecordReader
NIFI-6752 Refactored type and value conversion logic. Added support for more types. Added more tests.
Removed 'parent' from 'Recursive'. (Caused issues. The recursive nature is still there as it has a child with the same type).
Updated jasn1 1.11.2 to asn1bean 1.12.0. If an asn field name is a Java reserved keyword, the field gets a trailing "_" but the getter remains normal. In JASN1Utils adjusted logic when looking for the getter.
Added support for inherited types. OctetStrings are converted to Strings instead of byte arrays.
Service takes care of the compilation of the ASN files. Test sources are generated and removed from source control.

NIFI-6752 Removed obsolete TODOs.

NIFI-6752 Updated nifi-asn1-nar version to 1.13.0-SNAPSHOT. Fixed checkstyle violations (unused imports).

NIFI-6752 ASN.1 reader - ASN.1 bundle requires 'include-asn1' profile to be active to be part of assembly.

NIFI-6752 ASN.1 reader - Updated ASN1.xml template.

NIFI-6752 ASN.1 reader - Updated versions.

NIFI-6752 ASN.1 reader - Update example generator. Updated ASN1.xml template. Updated (fixed) nifi-asn1-nar version in pom.xml.

NIFI-6752 ASN.1 reader - Added missing license for ASN1.xml.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4577
2021-02-25 12:58:05 -05:00
Matthew Burgess 29cc3886fc NIFI-8259: Infer CSV field types as strings if no records are present 2021-02-25 12:45:12 -05:00
Peter Turcsanyi 93b1a05dc3
NIFI-8030: Added property to ReportLineageToAtlas to configure file/directory level logging of [hd]fs_path entities to Atlas
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4813.
2021-02-25 18:37:19 +04:00
Matthew Burgess 34fc94454f NIFI-8249: Fixed issue with error during multiple FlowFile results in ExecuteSQL processors 2021-02-24 10:10:32 -05:00
Stan Antyufeev 2322b2cddf NIFI-6113 Refactoring GetHDFSFileInfo to remove instance level variable to make it stateless.
This closes #4837.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2021-02-24 13:15:12 +01:00
Matthew Burgess 2dedd8bf0f
NIFI-8253: Restore call to session.commit() in GenerateTableFetch
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4840.
2021-02-24 15:57:16 +04:00
Peter Turcsanyi f101a2bba5
NIFI-8198: ConsumeAMQP detects if Queue is deleted on server
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4805.
2021-02-24 12:04:52 +04:00
Christian Gumpert f2555f27f1
NIFI-8119: properly free database resources when done with the processing of BLOBs
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4791
2021-02-23 16:47:32 -05:00
Peter Gyori d049bca794
NIFI-8205: Documentation improvements for the Wait processor
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4808.
2021-02-23 21:22:19 +04:00
Mark Payne 528fce2407
NIFI-7646, NIFI-8222: Instead of having StandardProcessSession call ContentRepository.read(ContentClaim), introduced a new ContentRepository.read(ResourceClaim) and hold open the InputStream to the ResourceClaim. This can't be supported by EncryptedContentRepository, so introduced a method to allow using this or not. The benefit here is that when we have many FlowFiles read within a session, such as when using MergeContent/MergeRecord or a processor configured with a Run Duration, we can hold open a single InputStream instead of constantly opening FileInputStreams and seeking to the appropriate location. This is much faster.
- Instead of entering a 'synchronized' block for every provenance event, serialize up to 1 MB worth of data, and then enter synchronized block to write that data out. This avoids large amounts of lock contention and context switches

NIFI-7646: Removed TODO and unused Jackson dependency

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4818
2021-02-23 10:57:54 -05:00
Matthew Burgess 200c04c6d0
NIFI-8237: This closes #4835. Added missing SQL types to getDataTypeFromSQLTypeValue(), added defensive code
Signed-off-by: Joe Witt <joewitt@apache.org>
2021-02-23 08:08:09 -07:00
Pierre Villard 9101160cbf
NIFI-8250 - This closes #4838. UTF8 encoding for FTP processors
Signed-off-by: Joe Witt <joewitt@apache.org>
2021-02-23 08:05:32 -07:00
Chris Sampson 3cb26aec72
NIFI-8232 CSV Parsers optionally allow/reject duplicate header names
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4828.
2021-02-23 09:50:18 +04:00
exceptionfactory 418e2cc2cb NIFI-8132 Replaced framework uses of MD5 with SHA-256
NIFI-8132 Added FileDigestUtils in nifi-nar-utils to avoid dependency on nifi-utils

NIFI-8132 Removed unused imports from NarUnpacker

NIFI-8132 Removed MD5 references from FileUtils documentation

NIFI-8132 Replaced StringBuffer with StringBuilder and made new DigestUtils classes final

NIFI-8132 Replaced Collections.sort() with Stream.sorted()

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #4788.
2021-02-22 17:52:24 -05:00
Bryan Bende ebef823cb9
NIFI-8241 This closes #4833. Set the SAML context provider to use the EmptyStorageFactory
Signed-off-by: Joe Witt <joewitt@apache.org>
2021-02-22 08:53:32 -07:00
Celso Marques 703c3cfe63
NIFI-5906 - Removed FILE_EXISTS_VALIDATOR on JARs and Files to be added in the session
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4810.
2021-02-22 13:04:03 +04:00
Lehel Boér 80da0cf47d
NIFI-7496: Regex pattern changed to match dashes in filters.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4811.
2021-02-22 12:59:03 +04:00
Matthew Burgess b08d6071dd
NIFI-8224: Add LoggingRecordSink controller service
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4820.
2021-02-22 12:55:53 +04:00
Bryan Bende 6ed496c714
NIFI-8242 Ensure NiFi URL for redirects has a trailing slash in AccessResource
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4834.
2021-02-22 12:48:42 +04:00
exceptionfactory 51e7bdf940
NIFI-3669 Add SSL Support to CaptureChangeMySQL
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4594.
2021-02-22 11:29:59 +04:00
Timea Barna 59ad21fbaa NIFI-8200 Modified PutAzureDataLakeStorage to delete temp file if exception was thrown in uploadContent()
This closes #4815

Signed-off-by: Joey Frazee <jfrazee@apache.org>
2021-02-21 13:35:28 -08:00
Grant Henke 662a35b7de
NIFI-8240: Unify Kudu versions
The Kudu versions for the controller service and processors
currently doesn’t match. This patch unifies the version in
the parent bundle pom to ensure they match going forward.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4832.
2021-02-19 20:42:30 +04:00
Mark Payne cd6418cdaf
NIFI-8238: When FlowFile Concurrency/Outbound Policy is changed on child group, and child group is directly under version control, ensure that those fields are maintained on the generated VersionedProcessGroup when nested components are stripped out
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4831.
2021-02-19 14:44:42 +04:00
exceptionfactory f532b3ae1d
NIFI-5623 Upgraded OkHttp3 to 4.9.1 and updated unit tests
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4826.
2021-02-19 14:42:16 +04:00
Tamas Palfy aa726040c5 NIFI-8188 - Add 'Run Once' for processors in context menu.
Signed-off-by: Mark Payne <markap14@hotmail.com>
2021-02-18 14:51:28 -05:00
Zsihovszki Krisztina 6801704846 NIFI-8235 UnsupportedOpertationException in PutAccumuloRecord 2021-02-18 13:12:09 -05:00
Joey Frazee 042b34cb43 NIFI-8231 Clarify behavior of allowed values for Memory Pools in MonitorMemory 2021-02-18 09:58:48 -05:00
Matthew Burgess fd50d6f042 NIFI-3383 Check if usage operations are supported in MonitorMemory
This closes #4790

Signed-off-by: Joey Frazee <jfrazee@apache.org>
2021-02-16 17:25:28 -06:00
Joe Witt 88fab00e29
NIFI-7873 merging release branch to latest and updating to 1.14.0-SNAPSHOT 2021-02-15 12:09:32 -07:00
Joe Witt 4afb2ba743
NIFI-7873-RC4 prepare for next development iteration 2021-02-15 12:09:31 -07:00
Joe Witt 487280bee9
NIFI-7873-RC4 prepare release nifi-1.13.0-RC4 2021-02-15 12:09:30 -07:00
Pierre Villard a1c82a66a1
NIFI-7361 - increase actions column width for reporting tasks
This closes #4214

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-02-15 10:42:19 -06:00
Matthew Burgess d08f02428d
NIFI-8223: This closes #4819. Use column datatype in PutDatabaseRecord when calling setObject()
Signed-off-by: Joe Witt <joewitt@apache.org>
2021-02-10 18:01:43 -07:00
Nathan Gough 8057f8f6c5
NIFI-8221 - Set the default HTTP listening interface to 127.0.0.1.
This closes #4817

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-02-10 14:46:37 -06:00
Bryan Bende 1d82fb8e01
NIFI-8218 This closes #4816. Use proxy headers when available when getting request values while processing SAML responses
Signed-off-by: Joe Witt <joewitt@apache.org>
2021-02-10 13:34:57 -07:00
Mark Payne d5d520764d
NIFI-8210: When Index Reader/Searcher is used, do not allow the Lucene Index to be deleted until the reader/searcher is finsihed being used and closed.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4812
2021-02-09 15:32:15 -05:00
Peter Gyori b9b131239c NIFI-8215 --wait-for-init in nifi.sh refactored for POSIX sh
This fixes #4814

Signed-off-by: Joey Frazee <jfrazee@apache.org>
2021-02-09 11:10:15 -08:00
Nathan Gough 4ebeb4e146
NIFI-8181 - Added Disable HTTP/2 property to InvokeHTTP
This closes #4804

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-02-08 14:53:35 -06:00
Peter Turcsanyi 40d8c41656
NIFI-8193: Fixed PutSQL reordering FlowFiles
Also removed sorting of transferred MockFlowFiles in test framework.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4803
2021-02-05 16:17:10 -05:00
Mark Payne 749d05840b
NIFI-8204, NIFI-7866: Send revision update count in heartbeats. If update count in heartbeat is greater than that of cluster coordinator, request that node reconnect to get most up-to-date revisions. Cannot check exact equality, as the values may change between the time a heartbeat is created and the time the cluster coordinator receives it. However, it should be safe to assume that the revision won't be greater than that of the cluster coordinator. There is a tiny window in which it could be, as the sending node may update its revision, create the heartbeat, send it, and cluster coordinator process it before updating its own revision. However, this window is incredibly small and would only result in the sending node reconnecting, which will resolve itself. Also, when testing this fix, encountered NIFI-7866 and addressed that NullPointerException.
This closes #4806.

Signed-off-by: Bryan Bende <bbende@apache.org>
2021-02-05 15:18:44 -05:00
Mark Payne 03fd59eb2f
NIFI-8196: When node is reconnected to cluster, ensure that it re-registers for election of cluster coordinator / primary node. On startup, if cluster coordinator is already registered and is 'this node' then register silently as coordinator and do not join the cluster until there is no Cluster Coordinator or another node is elected. This allows the zookeeper session timeout to elapse.
Signed-off-by: Bryan Bende <bbende@apache.org>
2021-02-05 15:18:07 -05:00
Matthew Burgess b77dbd5030
NIFI-8172: Provide schema name to getPrimaryKeys call in PutDatabaseRecord
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4782.
2021-02-06 00:00:58 +04:00
Mark Bean a9b8635ac9
NIFI-6522: Referencing Components link from Reporting Task Constroller Services links to proper Controller Settings tab
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4794.
2021-02-05 23:53:29 +04:00
Jon Kessler 3f26e54521 NIFI-8194 Updated log level from WARN to DEBUG for various authentication protocols not being configured 2021-02-05 13:20:37 -05:00
Tamas Palfy b55998afc1 NIFI-8081 Added new Listing Strategy to ListFTP and ListSFTP: Time Window
NIFI-8081 Added new Listing Strategy to ListFTP and ListSFTP: Adjusted Time Window. User can specify the time zone or time difference (compared to where NiFi runs) of the system hosting the files and based on the calculates the current time there. Lists files modified before this adjusted current time (and after the last listing).
NIFI-8081 'Time Adjustment' validated not to be set if listing strategy is not 'Adjusted Time Window'. Extracted validator to a separate class. Added more tests. Minor refactor. Typo fix.
NIFI-8081 Improved validation.
NIFI-8081 'Time Adjustment' is not necessary - in fact it can cause problems. SFTP (and usually FTP - which has a more general bug at the moment) returns a timestamp that doesn't really need adjustment. (SFTP in particular returns the an 'epoch' time.) Everything remains the same - the new listing strategy relies on a sliding time window, but without the unnecessary option to adjust for the modification time.
NIFI-8081 Resolved conflicts after rebasing to main.
NIFI-8081 Renamed 'AbstractListProcessor.listByAdjustedSlidingTimeWindow' to 'listByTimeWindow'. Post main rebase correction.
NIFI-8081 Updated user doc for the BY_TIME_WINDOW strategy to warn user on it's reliance of accurate time.

This closes #4721.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2021-02-04 17:04:37 +01:00
Denes Arvay c1f88ec740 NIFI-8183 TailFile intermittently creates records with ascii NULL after rollover
This closes #4792.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2021-02-03 13:09:00 -05:00
mtien 6e1f737c53
NIFI-1355 Implemented new methods in KeyStoreUtils to programmatically-generate certificates, Keystores, and Truststores and return it wrapped in a TLS configuration.
Updated TestInvokeHTTP, TestInvokeHttpSSL, TestInvokeHttpTwoWaySSL, and TestListenHTTP to use new Keystore functionality.

NIFI-1355 Refactored and removed unnecessary unit tests in KeyStoreUtilsGroovyTest.

NIFI-1355 Added a password requirement when creating a new truststore.
Handled exception when loading a passwordless truststore type of Bouncy Castle PKCS12.

This closes #4801

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-02-03 08:53:47 -06:00
Peter Gyori 5a8b18b12d NIFI-8123: Added support for --wait-for-init when NiFi started with the 'start' command
This closes #4748.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2021-02-03 09:40:09 -05:00
Joe Witt d826416217
NIFI-8192 updating Copyright years for nifi things to include 2021
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4802.
2021-02-03 12:56:28 +04:00
Peter Turcsanyi 77eb8af275
NIFI-8187: Fixed PutSQL duplicating provenance events
Also fixed provenance event mocking in test framework.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4796
2021-02-02 19:02:02 -05:00
Bryan Bende e0a8b479fd NIFI-8190 Protect against property that references missing controller service 2021-02-01 15:47:52 -05:00
exceptionfactory b4e213cb2c
NIFI-8186 Excluded bcprov-ext-jdk15on from spring-security-saml2-core dependency (#4793) 2021-02-01 13:21:46 -05:00
Grant Henke 29a1d6badc NIFI-7987: Support ignore operations in the PutKudu processor 2021-02-01 12:10:19 -05:00
Pierre Villard d72bfc76b5
NIFI-8155 - add banner text in page title
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4768
2021-01-29 19:20:34 -05:00
Matthew Burgess 5e84cf6ff9 NIFI-8175: Add WindowsEventLogReader controller service
This closes #4785

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-01-29 15:16:19 -06:00
exceptionfactory 11e9ff3773
NIFI-8178 This closes #4787. Replaced StandardSSLContextService in unit tests with SslContextUtils
- Removed references to StandardSSLContextService from nifi-standard-processors
- Removed TestGetHTTPGroovy and TestPostHTTPGroovy since these are testing deprecated processors
- Optimized HandleHttpRequest, GetHTTP, PostHTTP to use SSLContextService.createContext()
NIFI-8178 Changed TestGetHTTP to ITGetHTTP since GetHTTP is deprecated
NIFI-8178 Changed TestPostHTTP to ITPostHTTP since PostHTTP is deprecated

Signed-off-by: Joe Witt <joewitt@apache.org>
2021-01-27 17:48:06 -07:00
exceptionfactory 8c1fe84f62
NIFI-8177 This closes #4786. Added timeout to Sender Pool polling to avoid blocking indefinitely
Signed-off-by: Joe Witt <joewitt@apache.org>
2021-01-27 15:04:23 -07:00
snorlaxa 04fb1aca47
NIFI-8142: Add "Insert Ignore" option to PutDatabaseRecord
NIFI-8142: Fix Checkstyle error

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4778
2021-01-27 11:50:40 -05:00
Waleed Al Aibani 813ac539b3
NIFI-7263 This closes #4167. Add No Tracking Listing Strategy to ListFile/ListFTP processors
Signed-off-by: Joe Witt <joewitt@apache.org>
2021-01-26 22:13:26 -07:00
Mark Payne 4ff9cddf15
NIFI-8173, NIFI-8174: This closes #4784. Updated Parameter Contexts to allow for unsetting a parameter value / setting it to null. Allow Parameters to make use of Expression Language. Updated docs to illustrated how EL is evaluated
Signed-off-by: Joe Witt <joewitt@apache.org>
2021-01-26 21:55:18 -07:00
exceptionfactory abb6ed3128
NIFI-8171 This closes #4779. Upgraded Bouncy Castle libraries to 1.68 and centralized dependency version
NIFI-8171 Increased response and idle timeouts for HTTP unit tests
NIFI-8171 Increased TestServer idle timeout to 45 seconds for HTTP unit tests
NIFI-8171 Adjusted timeout and sleep on TestPutTCPCommon.testPruneSenders
NIFI-8171 Increased TestServer idle timeout to 60 seconds and removed 500ms Thread.sleep() in TestInvokeHttpSSL
NIFI-8171 Optimized OkHttpClientUtils to avoid reading trust store twice during initialization
NIFI-8171 Added static variable for server startup sleep
NIFI-8171 Increased TestInvokeHTTP Connect Timeout and TestListenHTTP Response Timeout to 30 seconds
NIFI-8171 Refactored unit tests for InvokeHTTP and ListenHTTP to optimize SSLContext creation
NIFI-8171 Updated TestListenHTTP for static creation of SSLContext
NIFI-8171 Added started check for ListenHTTP Server in TestListenHTTP
NIFI-8171 Refactored TestPutTCP classes to optimize SSLContext creation
NIFI-8171 Increased TestListenHTTP timeout for server start to 120 seconds and added exception when not connected
NIFI-8171 Increased Connect and Read Timeouts for InvokeHTTP SSL unit tests

Signed-off-by: Joe Witt <joewitt@apache.org>
2021-01-26 21:24:07 -07:00
exceptionfactory 2cdb0fb6a3 NIFI-8088 Removed deprecation warning log for PKCS12 trust stores
Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #4727.
2021-01-26 16:50:33 -05:00
Peter Turcsanyi 67d06003b7 NIFI-8023: Convert java.sql.Date between UTC/local time zone normalized forms before/after database operations
This closes #4781

Signed-off-by: David Handermann <exceptionfactory@gmail.com>
2021-01-26 14:39:02 -06:00
Bence Simon 0a10557dd5 NIFI-7801 Adding support for HTTP based Splunk put and indexed acknowledgement
This closes #4714.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2021-01-26 15:20:20 -05:00
Mark Payne f2a16cd02e NIFI-8163: When counting number of components, we traverse into all Process Groups, but then call findAllRemoteProcessGroups, which is a recursive call, instead of calling getRemoteProcessGroups(). This results in counting the Process Groups many times. So fixed that.
Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #4775.
2021-01-21 15:09:42 -05:00
Troy Melhase 76648bdc0b NIFI-7356 - Config TLS for embedded ZooKeeper when NiFi TLS enabled.
NIFI-7356 - Addresses PR feedback.

NIFI-7356 - Additional changes from PR feedback.

NIFI-7356 - Adding integration tests for ZooKeeperStateServer for TLS.

NIFI-7356 - TLS + Zookeeper now working with single and quorum. Needs code cleanup, need to fix IT tests and docs.

NIFI-7356 - Fixed up tests and removed some irrelevant ones. Refactored some of ZooKeeperStateServer. Tested successfully with a secure and insecure 3 node NiFi + Quorum.

NIFI-7356 - Checkstyle fixes.

NIFI-7356 - Updated administration guide with embedded ZooKeeper TLS configuration.

NIFI-7356 - Updated the way ZooKeeper TLS properties are set/mapped from NiFi properties.

NIFI-7356 - Updated per review, using NiFiProperties keystore strings, classname for ocnnection factory, adjusted TLS configuration checks in NiFiProperties.

NIFI-7356 - Updated configuration validation logic and added tests.

NIFI-7356 - Codestyle check fixes.

NIFI-7356 - Updated some of the log messages.

NIFI-7356 - Updated as per code review.

NIFI-7356 - Fixed max port number.

NIFI-7356 - Updated admin guide and updated small code issues as per code review.

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #4753.
2021-01-21 13:47:53 -05:00
Mark Payne 6741317cc4 NIFI-8162: Fixed outdated docs. Self-merging as this is a trivial docs fix.
Signed-off-by: Mark Payne <markap14@hotmail.com>
2021-01-21 12:52:30 -05:00
exceptionfactory 8c49f0688b
NIFI-7243 Upgraded com.hierynomus.sshj to 0.30.0 to resolve authentication race condition in version 0.27.0
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4774.
2021-01-21 21:21:47 +04:00
Mark Payne 525e4105eb NIFI-8095: Created StatelessNiFi Sink Connector and Source Connector. Minor updates to stateless nifi api to accommodate.
Self-merging PR based on comments from @joewitt
2021-01-21 10:56:28 -05:00
Pierre Villard 93a5823f8a NIFI-7890 - Added record support to ConsumeMQTT processor
This closes #4738.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2021-01-21 11:16:07 +00:00
Mike Thomsen 953327cdf5
NIFI-8156 Fixed byte handling bug in cassandra.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4771.
2021-01-21 11:23:14 +04:00
Matthew Burgess fb2a8b5820
NIFI-8043: Quote update key column names in PutDatabaseRecord
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4772.
2021-01-21 11:17:13 +04:00
Andrew Lim 27f57e6463
NIFI-8150 Change Download flow to Download flow definition for process groups (#4766) 2021-01-20 16:00:41 -05:00
Mark Payne 803ba882aa
NIFI-8146: Ensure that we close the Connection/Statement/PreparedStatement objects in finally blocks or try-with-resources
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4770
2021-01-20 13:48:57 -05:00
Mark Payne 25ab050ed7
NIFI-8149: Updated Apache Calcite version from 1.21.0 to 1.26.0
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4765.
2021-01-20 13:04:08 +04:00
Pierre Villard 88de03aebb NIFI-7895 - Fix NPE in ConsumeMQTT with truststore only SSL CS
This closes #4587.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2021-01-20 00:29:30 +01:00
Mark Payne 930e8d9e0e
NIFI-7698: Added a merge.reason attribute for flowfiles indicating why they were merged by MergeContent. Also updated logs to indicate the reason and added additional documentation for processor
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4764
2021-01-19 17:20:06 -05:00
s9514171 b4ddeb8997 NIFI-8116 The old peers do not be deleted when the URIs of the remote process group are changed
This closes #4745.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2021-01-19 16:25:09 -05:00
Mark Payne f02079c2fa
NIFI-8146: Added RecordPath for Statement Type and to specify where to find the data within the incoming Record. Also performed some minor code refactoring to avoid lots of large lambdas, as it results in code that is more readable and more maintainable. Updated some properties to use dependsOn() notation now that this is available.
NIFI-8146: Updated PutDatabaseRecord to avoid using the functional style framework that had previously been used. Doing so resulted in code that was difficult to understand and maintain. Added additional unit tests and improved MockRecordParser so that it could throw configurable types of exceptions

NIFI-8146: Fixed checkstyle issues

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4763
2021-01-19 13:23:54 -05:00
eduardofontes 3e72dfe421
NIFI-8109 - Update MongoDBControllerService to externalize username and password
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4759.
2021-01-19 21:21:44 +04:00
Chris Sampson 40f193098b
NIFI-7726 correct versionedFlowState field name in SiteToSiteStatusReportingTask schema
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4762.
2021-01-19 10:40:38 +04:00