5217 Commits

Author SHA1 Message Date
markap14
b03fdac1b5
NIFI-9839: Fixed bug in which Process Group would return true for isEmpty even if it has a Controller Service (#5910) 2022-03-28 16:58:45 -04:00
Peter Gyori
6ade47ac4f
NIFI-9838: Added Client Certificate attributes to ListenTCPRecord FlowFiles
This closes #5908

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-03-28 13:36:16 -05:00
Emilio Setiadarma
9566d3fa78
NIFI-9841 Improved Encryptor test reliability
- Changed expected exception to ProcessException in KeyedEncryptorGroovyTest and PasswordBasedEncryptorGroovyTest to avoid intermittent failures

This closes #5911

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-03-28 13:28:28 -05:00
Nathan Gough
2fbe10b4bc
NIFI-9701 - Corrected No Tracking strategy to create one flow file when using a Record Writer
- Corrected No Tracking strategy Record Writer handling for ListSFTP
- Updated temporary test files to have last modified time of epoch to avoid intermittent issue with Minimum Age filtering
- Refactored MockCacheService to separate reusable class

This closes #5885

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-03-28 12:47:30 -05:00
Mark Payne
63f02c99bf
NIFI-9835: Fixed threading bug in which NioAsyncLoadBalanceClient calls LoadBalanceSession.isComplete() followed by LoadBalanceSession.isCanceled() but it's possible for the complete flag to change before the canceled flag (they are not updated atomically). So changed to use a single LoadBalanceSessionState enum that represents the state. Also made the private StandardProcessSession.commit(boolean) method synchronized. When a processor is terminated (as is the case in Offload), we roll back sessions and both the commit() and rollback() need to be synchronized. Only the public commit() method was synchronized, and now with commitAsync() happening, we had the ability to commit without any synchronization. This addresses that concern. Also fixed a typo in docs for MergeRecord.
This closes #5902

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-03-25 12:42:03 -05:00
Mike
2ac90a6441
NIFI-7262 Added debug logging for the generated query in PutCassandraRecord. (#5898)
Signed-off-by: Otto Fowler <otto@apache.org>
2022-03-25 06:48:57 -04:00
Joe Gresock
a75381abbe NIFI-9827: Upgrading AWS Java SDK to 1.12.182 to pick up new AWS Regions
This closes #5892

Signed-off-by: Chris Sampson <chris.sampson82@gmail.com>
2022-03-24 13:24:28 +00:00
Marcus Ely
72435c3c6d
NIFI-8124 Added Cookie Strategy property to InvokeHTTP
This closes #5887

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-03-23 15:55:40 -05:00
Emilio Setiadarma
772adbc709
NIFI-1468 Added tests to handle invalid cipher streams missing Salt/IV
- Updated PasswordBasedEncryptorGroovyTest and KeyedEncryptorGroovyTest

This closes #5877

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-03-23 13:30:15 -05:00
Joe Witt
0419dc2939
NIFI-9780-RC3 prepare for next development iteration 2022-03-21 11:58:27 -07:00
Joe Witt
b019a9191f
NIFI-9780-RC3 prepare release nifi-1.16.0-RC3 2022-03-21 11:58:09 -07:00
Matthew Burgess
c84f438782
NIFI-9800: Unwrap SQLException in PutDatabaseRecord when table does not exist
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5871
2022-03-21 14:00:13 -04:00
Mike Thomsen
2bb1a06a8c
NIFI-9788 Updated commons-codec to 1.15 across all modules.
This closes #5881

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-03-21 09:13:05 -05:00
exceptionfactory
7823156606
NIFI-9815 Corrected log message formatting in multiple classes
This closes #5879

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-03-18 20:11:05 -04:00
exceptionfactory
ab0d2c2f72
NIFI-9807 Added Refresh Window Property to OAuth2 Token Provider
- Removed hard-coded expiry margin from AccessToken.isExpired() determination

This closes #5876

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-03-18 08:32:05 -04:00
Tamas Palfy
fc30b649cc
NIFI-9801 Fixed error in previous correction of AccessToken.isExpired() margin calculation
NIFI-9801 Stabilized shaky AccessTokenTest.

This closes #5872

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-03-17 20:01:32 -04:00
exceptionfactory
e17a436797
NIFI-9662 Remove unused mail-1.4.7 dependency from nifi-framework-bundle
This closes #5873

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-03-17 14:38:57 -04:00
exceptionfactory
412e21be97
NIFI-9281 Enabled building on Java 17
- Added Ubuntu Zulu JDK 17 GitHub build
- Adjusted MiNiFi C2 FileSystemConfigurationCache test to avoid using environment variables
- Adjusted MiNiFi StatusLogger and StatusLoggerTest to avoid overriding private logger
- Adjusted failure reason attribute check in TestGetIgniteCache
- Adjusted TestRangerAuthorizer and TestRangerNiFiAuthorizer to avoid checking nested exceptions
- Adjusted encrypt-config TestUtil to avoid unnecessary comparison of different types
- Disabled Javascript tests on Java 15 and higher
- Disabled several Hive 3 tests on Java 17 for StringInternUtils illegal access
- Refactored nifi-enrich-processors tests to use Mockito without Powermock
- Refactored nifi-toolkit-tls tests to avoid illegal reflective access
- Removed deprecated X509Certificate test in CertificateUtilsTest
- Removed kryo serialization from nifi-site-to-site-client test
- Updated TestHashContent to use SHA-1 instead of SHA for hash algorithm
- Upgraded maven-war-plugin from 2.5 to 3.3.2
- Upgraded nifi-graph-bundle dependencies from Groovy 2.5.14 to 3.0.8
- Upgraded QuestDB from 4.2.1 to 6.2.1 in nifi-framework-core

This closes #5870

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-03-17 14:08:38 -04:00
exceptionfactory
77c45cabc5 NIFI-9797 Corrected AccessToken.isExpired() margin calculation
Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5867.
2022-03-14 18:31:44 -04:00
markap14
21922af90c
NIFI-9794: If a node is OFFLOADING, do not allow connections to be deleted. Also if we fail to notify the node that it needs to offload its data, change its state back to DISCONNECTED. (#5865) 2022-03-14 15:45:02 -04:00
Nathan Gough
713e2fd03c
NIFI-9645 - Updated PutSplunk to allow idle connection timeouts
This closes #5841

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-03-14 11:15:16 -05:00
Mark Payne
7ff70706ee
NIFI-9790: This closes #5860. Fixed race condition in which SwappablePriorityQueue could attempt to access the 0th element of an empty collection
Signed-off-by: Joe Witt <joewitt@apache.org>
2022-03-14 08:38:39 -07:00
Nathan Gough
885c475f90
NIFI-9699 - Updated oidcCallback method to handle error cases. Added some unit tests.
This closes #5824

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-03-11 11:40:47 -06:00
Joe Witt
72fadf9e51
NIFI-9780 fixing many inconsistent pom project declarations and updated org apache parent version 2022-03-10 11:15:16 -07:00
Bryan Bende
546f986603
NIFI-9775 Create RuntimeManifestService
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5849.
2022-03-10 07:52:16 -05:00
Peter Turcsanyi
92202a5b95
NIFI-9764: Atlas reporting task sends 'unknown' hive_table when table is name not available
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5839.
2022-03-10 13:16:24 +01:00
Mike Thomsen
df00cc6cb5
NIFI-6047 Cleaned up code to allow tests to run against 1.13.0-snapshot
Removed DMC.
NIFI-6047 Started integrating changes from NIFI-6014.
NIFI-6047 Added DMC tests.
NIFI-6047 Added cache identifier recordpath test.
NIFI-6047 Added additional details.
NIFI-6047 Removed old additional details.
NIFI-6047 made some changes requested in a follow up review.
NIFI-6047 latest.
NIFI-6047 Finished updates
First round of code review cleanup
Latest
Removed EL from the dynamic properties.
Finished code review requested refactoring.
Checkstyle fix.
Removed a Java 11 API
NIFI-6047 Renamed processor to DeduplicateRecord

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4646
2022-03-09 19:06:44 -05:00
Adam
23132fb89f
NIFI-6047
Added NiFi DetectDuplicateRecord standard processor.
Adding some documentation and PR review tweaks.
Exposing processor
Documentation updates, exception handling consolidation, added support for record path field variables.
Added tests.
Build bump.
Migrated cache service to groovy folder.
Moved declarations for properties to @BeforeClass lifecycle method.
Adding some documentation and PR review tweaks.
Documentation updates, exception handling consolidation, added support for record path field variables.
Added tests.
Build bump.
Migrated cache service to groovy folder.
Fixed variable type bug.
Fixed mapping of test params to usage.
Fixed potential illegal state exception bug.
2022-03-09 19:06:08 -05:00
exceptionfactory
4a46b087b8
NIFI-9782 This closes #5854. Excluded H2 DB from nifi-druid-bundle
Signed-off-by: Joe Witt <joewitt@apache.org>
2022-03-09 13:26:54 -07:00
Mark Payne
73356ea448
NIFI-9783: This closes #5855. When migrating FlowFiles from one ProcessSession to another, if any FlowFile had already been transferred, and the Relationship to which it was transferred was auto-terminated, we were updating the wrong member variable, which threw off our stats for the processor. Fixed that.
Signed-off-by: Joe Witt <joewitt@apache.org>
2022-03-09 13:26:32 -07:00
exceptionfactory
c73573b325
NIFI-9761 Correct PeerChannel processing for TLS 1.3 (#5836)
* NIFI-9761 Corrected PeerChannel processing for TLS 1.3
- Added TestPeerChannel with methods for TLS 1.2 and TLS 1.3
- Updated PeerChannel.close() to process SSLEngine close notification
- Improved logging and corrected handling after decryption
2022-03-09 15:15:52 -05:00
Mark Payne
6a1c7c72d5
NIFI-9781: Fix handling when selecting array element via QueryRecord
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5853.
2022-03-09 13:07:15 -05:00
Peter Turcsanyi
3219c105eb
NIFI-9728: Added support for User Assigned Managed Identity authentication for Azure ADLS and Blob_v12 processors
Also bumped Azure dependencies.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5846.
2022-03-09 16:50:38 +01:00
Bence Simon
b86466b4a5 NIFI-9669 Adding PutDynamoDBRecord processor
This closes #5761.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2022-03-09 16:20:33 +01:00
Bence Simon
3b8afee9d6
NIFI-9778 Fixing additional details for ScriptedPartitionRecord
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5852.
2022-03-09 14:49:28 +01:00
Matt Gilman
227dcf5456
NIFI-9777: (#5850) - Adding support to remove attributes from verification requests.
This closes #5850
2022-03-08 16:25:32 -05:00
Mark Payne
e6229ab938
NIFI-9771: When a Kafka record is obtained during config verification, we should produce an invalid response if the Record Reader is not able to produce any records from it
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5847.
2022-03-08 06:42:27 -05:00
Mike Thomsen
f35f010deb
NIFI-9085 Refactored the Elasticsearch bundle to use JUnit 5
- NIFI-9086 Refactored nifi-accumulo-bundle to use JUnit 5
- NIFI-9087 Updated nifi-ambari-bundle to use JUnit 5
- NIFI-9088 Refactored nifi-amqp-bundle to use JUnit 5
- NIFI-9089 Updated nifi-asn1-bundle to use JUnit 5
- NIFI-9090 Refactored nifi-atlas-bundle to use JUnit 5
- NIFI-9091 Refactored nifi-avro-bundle to use JUnit 5
- NIFI-9092 Refactored nifi-aws-bundle to use JUnit 5
- NIFI-9095 Refactored nifi-azure-bundle to use JUnit 5
- NIFI-9145 Refactored nifi-rethinkdb-bundle to use JUnit 5
- NIFI-9143 Refactored nifi-redis-bundle to use JUnit 5

This closes #5350

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-03-07 21:49:23 -06:00
Matthew Burgess
43bf2b5975
NIFI-9763: Additional escaped VALUE column due to H2 changes
This closes #5848

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-03-07 19:18:02 -06:00
Joe Gresock
743020eeb4
NIFI-9762: Adding DBCPConnectionPool config verification
Relaxing MockPropertyValue validation to allow for variables to be passed to config verification

Fixing underlying framework issue with config verification: wrong variable registry was being used

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5843
2022-03-07 14:43:11 -05:00
Paul Grey
fc5c810de7
NIFI-9750 - Logging Improvements to support LoadBalanceProtocol troubleshooting
This closes #5828

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-03-07 13:41:07 -06:00
Eduardo Fontes
d062029258
NIFI-8927 - Add option to start/stop all controllers (#5247)
* Update nf-context-menu.js

Include enable/disable all controllers menu item

* Update nf-actions.js

Create enableAllControllers and disableAllControllers actions

* Rename ...Controllers to ...ControllerServices

* Rename *controllers to *controller services

* Adjusts proposed by reviewer.

* Refactor enable and disable AllControllerServices

This closes #5247
2022-03-07 13:57:35 -05:00
exceptionfactory
7d046fe3de
NIFI-9763 Escaped VALUE column for Configure Details Auditing
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5838
2022-03-07 08:53:08 -05:00
Chris Sampson
12685a4af4
NIFI-9766 avoid intermittent SearchElasticsearchTest failures in CI pipeline
This closes #5842

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-03-05 19:05:02 -05:00
Mark Payne
65dd62716a
NIFI-9748: Added new property for Output Format to LogAttribute. Also made the FlowFile Properties (file size, entry date, lineage start date) optional and renamed from 'Standard FlowFile Attributes' to 'FlowFile Properties' because this has led to confusion many times in the past, around users wanting to reference these things as attributes via EL but they are not actually attributes.
This closes #5825

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-03-04 08:24:39 -05:00
Mark Payne
8959226b50
NIFI-9754: Introduced VersionedExternalFlow
- Updated stateless and StandardProcessGroup, etc. to make use of VersionedExternalFlow
- Updated StatelessDataflowDefinition to use ExternalVersionedFlow instead of generic type
- Updated Stateless Bootstrap to avoid loading stateless engine libs from root class path but instead use a NarClassLoader to load the statelss nar

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5832.
2022-03-04 07:19:38 -05:00
exceptionfactory
7438bc9244
NIFI-9759 Upgraded Spring Framework from 5.3.15 to 5.3.16
- Upgraded Spring Security from 5.6.1 to 5.6.2
- Upgraded Spring Boot from 2.6.3 to 2.6.4 in NiFi Registry
- Upgraded Spring Data Redis from 2.6.0 to 2.6.2
- Upgraded Spring Integration from 5.5.7 to 5.5.9

This closes #5835

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-03-04 06:23:51 -05:00
Shane Ardell
6cd30e3d0a
NIFI-9543: Add bring-to-front functionality to labels (#5765)
* NIFI-9543: add bring-to-front functionality to labels

* NIFI-9543: update zIndex changes to persist after shutdown

* NIFI-9543: remove commented code

* NIFI-9543: update flow comparator and snippet to account for zIndex

* NIFI-9543: changed add function to sort labels when a new label is added

* NIFI-9543: remove unnecessary call to position

* NIFI-9543: only register difference if zIndex is not default value

* NIFI-9543: add null checks to prevent primitive Long

This closes #5765
2022-03-03 18:50:15 -05:00
Matthew Burgess
c91f543c2b NIFI-9425: Added auto-load NAR capability to MiNiFi
NIFI-9425: Added autoload directory property and defensive code for stop() called twice

This closes #5762.

Signed-off-by: Kevin Doran <kdoran@apache.org>
2022-03-03 16:30:47 -05:00
Adam Kocsis
7a16619969
[NIFI-9690] Display icon on Connection Label to show when head of the queue is penalized (#5812)
This closes #5812
2022-03-03 09:13:00 -05:00