39 Commits

Author SHA1 Message Date
exceptionfactory
83316736f8
NIFI-9959 Add UI Support for Sensitive Dynamic Properties (#6073)
* NIFI-9959 Added UI Support for Sensitive Dynamic Properties

- Added SupportsSensitiveDynamicProperties to DBCPConnectionPool and ScriptedReportingTask

* NIFI-9959 Added sensitive parameter argument for Controller Service descriptors

* NIFI-9959 Adjusted sensitive property descriptor handling to support changing status

* NIFI-9959 Added info icon for Sensitive Value field

* NIFI-9959 Corrected handling of descriptor for existing dynamic properties

* NIFI-9959 Cleaning up dialog markup.

Co-authored-by: Matt Gilman <matt.c.gilman@gmail.com>

This closes #6073
2022-05-25 21:36:31 -04:00
Joe Gresock
743020eeb4
NIFI-9762: Adding DBCPConnectionPool config verification
Relaxing MockPropertyValue validation to allow for variables to be passed to config verification

Fixing underlying framework issue with config verification: wrong variable registry was being used

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5843
2022-03-07 14:43:11 -05:00
Tamas Palfy
635824904d
NIFI-9609 Added nifi-snowflake-bundle with SnowflakeComputingConnectionPool
- Updated DBCPConnectionPool for extension

This closes #5692

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-02-21 19:38:47 -06:00
Mike Thomsen
766c4e392c
NIFI-9166 Refactored nifi-standard-services to use JUnit 5
This closes #5378

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-02-12 23:17:56 -06:00
exceptionfactory
9e06d13ad9
NIFI-9673 Improved DBCP and HikariCP test reliability
- Configured tests to use random temporary directory
- Added Derby shutdown handling and directory removal
- Upgraded DBCPServiceTest to JUnit 5
- Added pool name to HikariCP

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5759
2022-02-11 10:43:15 -05:00
Matthew Burgess
b8af44d81b
NIFI-6871: Added HikariCPConnectionPool controller service
This closes #3890

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-02-10 13:02:31 -06:00
Bryan Bende
2ae4f908f5
NIFI-8978 Add KerberosUserService to DBCPConnectionPool/HadoopDBCPConnectionPool
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5399.
2021-09-27 10:12:31 +02:00
Bryan Bende
e436381c3a
NIFI-8975 Integrate KerberosUserService into HBase processors/services
This closes #5322

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-08-27 17:41:58 -05:00
Bence Simon
f0ca63b2ab NIFI-8429 Changing DBCPConnectionPool driver manager in order to prevent from leaking
NIFI-8429 Refining error handling based on review comment

This closes #5003.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2021-04-16 09:26:50 +02:00
Mark Payne
7d1d536da6
NIFI-8206: Added identifiesExternalResource() method to PropertyDescriptor.Builder and implemented functionality.
- Updated components to make use of new feature

NIFI-8206: Added a ResourceType of TEXT. This requires that the ResourceReferenceFactory know which types are allowed in order to create the ResourceReference. PropertyValue needs to then have the PropertyDescriptor available to it. This resulted in highlighting many bugs in unit tests where components were not exposing property descriptors via getSupportedPropertyDescriptors() or were evaluating Expression Language using the wrong scope, so fixed many unit tests/components to properly declare Expression Language scope when using it

NIFI-8206: Removed problematic unit test that required directory names with special characters that are not allowed on some operating systems

This closes #4890.

Signed-off-by: Bryan Bende <bbende@apache.org>
2021-04-12 11:48:04 -04:00
exceptionfactory
02ea103c6b
NIFI-8379 This closes #4949. Improved reliability and performance of DBCPServiceTest
- Removed unreliable H2 network connection test methods
- Removed duplicate and ignored test methods
- Reduced duplication of setting common service properties

Signed-off-by: Joe Witt <joewitt@apache.org>
2021-03-30 07:24:35 -07:00
exceptionfactory
1e13b62e78
NIFI-8047 Added documentation for sensitive DBCP properties
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4696
2020-11-30 18:33:32 -05:00
exceptionfactory
fe53f8090d
NIFI-8047 Added support for sensitive dynamic properties in DBCP
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4692
2020-11-30 14:55:20 -05:00
Matthew Burgess
7e145142e1
NIFI-7799: Relogin with Kerberos on connect exception in DBCPConnectionPool (#4519) 2020-09-17 13:33:54 -04:00
Joe Witt
254b20fa5f
NIFI-7274 add time for test conditions to be met 2020-03-22 09:54:43 -04:00
Adam Taft
decb5d062e
NIFI-6742 Use JUnit TemporaryFolder when creating test databases
- Add @Rule for TemporaryFolder
- Replace use of previous target/db with TemporaryFolder/db
- Remove use of ~/test db (in home directory)
- Remove System.out lines

Signed-off-by: Marc Parisi <phrocker@apache.org>

This closes #4137.
2020-03-12 20:48:52 -04:00
Bryan Bende
e0fc75a963 NIFI-7019 Add kerberos principal and password properties to NiFi DBPCConnectionPool
This closes #4087.
2020-03-02 14:20:20 -05:00
Matthew Burgess
3a5a2da73c
NIFI-6954: Evaluate EL for DBCPConnectionPool properties
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3940.
2019-12-19 11:44:55 +01:00
Koji Kawamura
b3880a4a06
NIFI-5970 Handle multiple input FlowFiles at Put.initConnection
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3583
2019-11-19 16:48:14 -05:00
Tamas Palfy
84a05c8595 NIFI-6729 - Created AbstractSingleAttributeBasedControllerServiceLookup and updated DBCPConnectionPoolLookup and AzureStorageCredentialsControllerServiceLookup to inherit from it.
This closes #3774.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2019-11-05 14:02:03 +01:00
Matthew Burgess
ace23c35b5 NIFI-6780: Add QueryNiFiReportingTask, RecordSinkService, S2S and DB impls (#3826)
* NIFI-6780: Introduce RecordSinkService to separate format and destination, refactor common S2S utils

Added QueryNiFiReportingTask to SQL query NiFi status and metrics

Add PROCESSOR_STATUS and PROCESS_GROUP_STATUS

Add CONNECTION_STATUS_PREDICTIONS

check for null predictions

Add ConnectionStatusRecursiveIterator

Fix issue w/ duplicate iterator outputs

Refactored query interfaces, fixed assembly POM

Rebased v master, fixed isBackPressureEnabled and Checkstyle/RAT errors

Add DatabaseRecordSink service (#13)

* Add DatabaseRecordSink service

* Incorporated review comments

* NIFI-6780: Add/fix docs, cleanup warnings, fixed some table definitions

* Added bundle profile, remove predictions table if not enabled

* Added doc for which tables are available when

This closes #3826.
2019-10-22 08:56:42 -04:00
James Cheng
f1d35f46ce NIFI-6684 Add more property to Hive3ConnectionPool (#3763)
* NiFi-6684 Add more property to Hive3ConnectionPool

signed-off by: Peter Wicks <patricker@gmail.com>
2019-10-03 07:24:29 -06:00
Jeff Storck
8c8a9b4d53
NIFI-5985: Added capability for DBCPConnectionPool to use KerberosCredentialsService.
Refactored KerberosAction to return a result from execute()
Removed usage of ProcessContext.yield() from KerberosAction, which should instead be handled the component using the KerberosCredentialsService.
Updated SolrProcessor to yield a flowfile on error, rather than the KerberosAction invoking the yield.

NIFI-5985: Updated TestPutSolrContentStream.testUpdateWithKerberosAuth test case to match on PrivilegedExceptionAction instead of PrivilegedAction doAs arguments.

NIFI-5985: Moved kerberosUser logout after closing the datasource in the shutdown method.

NIFI-5985: Removed catching exceptions in DBCPConnectionPool.shutdown
Exception when closing the datasource is prioritized over an exception when logging out the kerberos principal
Added GroovyDBCPServiceTest tests to verify prioritizing datasource.close() exception over kerberosUser.logout() exception

This closes #3288.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-02-11 09:45:55 -05:00
SavtechSolutions
cf7ab0ce18 NIFI-5790 removed the last test as it's causing a race condition intermittently (#3260)
Signed-off-by: Peter Wicks <patricker@gmail.com>
2019-01-10 12:13:35 -07:00
Colin Dean
102a5288ef NIFI-5823: Fixes typo in min idle connections property name
So many people missed it :-(

This closes #3172.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2018-11-16 11:16:55 +09:00
Colin Dean
a628aced6b
NIFI-5790: Exposes 6 commons-dbcp options in DBCPConnectionPool
Signed-off-by: Peter Wicks <patricker@gmail.com>

This Closes #3133
2018-11-09 11:29:11 -07:00
Pierre Villard
c4d3b5e94f NIFI-5588 - Fix max wait time in DBCP Connection Pool
This closes #3022

Signed-off-by: zenfenan <zenfenan@apache.org>
2018-09-25 12:19:54 +05:30
Matthew Burgess
2ee0af9663 NIFI-5614: Update commons-dbcp to commons-dbcp2 for DBCPConnectionPool
This closes #3014

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-09-20 08:17:52 -04:00
Mark Bean
b279624398 NIFI-5368 controller services validated prior to enabling; referenced controller services must be enabled for referencing component to be valid (mock framework)
This closes #2873.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-07-10 11:23:40 -04:00
Bryan Bende
cf6089196f NIFI-5229 Adding a DBCPService implementation that can lookup other DBCPServices dynamically at runtime
This closes #2735

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-05-25 08:54:03 -04:00
Pierre Villard
4c787799ff NIFI-4149 - Indicate if EL is evaluated against FFs or not
- take into account input requirement for documentation rendering
- Renamed variable registry scope and added comments
- Doc + change in mock framework to check scope + update of components + UI
2018-04-06 12:40:54 -04:00
Koji Kawamura
b2290b24a6
NIFI-3107: Unit test failure for DBCPServiceTest
H2 and Kafka broker uses the same default port 9092.
If an user is running Kafka broker on the same machine, or run the unit
tests in parallel, DBCPServiceTest can fail since some of its test
methods connects to port 9092.

This closes #1504.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-02-13 23:09:07 -08:00
Matt Burgess
2d6d7710c7 NIFI-3426: Add dynamic property support to DBCPConnectionPool
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #1461.
2017-02-03 11:09:02 -08:00
Toivo Adams
4fc0e9c407 nifi-2381 Connection Pooling Service -Drop invalid connections and create new ones.
This closes #986.
2016-09-20 18:21:42 +02:00
Matt Burgess
0745990c2d NIFI-2604: Added validators and logic for multiple URLs/files/folders for DB driver location
This closes #912

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-08-25 23:35:41 -04:00
Jeff Storck
1ea49ad8cf NIFI-2407 Implements EL support on some properties of processors/services
- Added EL support to "Directory" property of ListHDFS and GetHDFS processors
 - Added EL support to "Destination" property for ConsumeJMS and PublishJMS processors
 - Added EL support to "MQ ConnectionFactory Implementation", "MQ client library path", "Broker URI" properties for the JMS Connection Factory Provider
 - Added EL support to "Database Connection URL", "Database Driver Class Name", "DB Driver jar url", "DB username", and "DB password" properties for the DBCP Connection Pool
 - Removed NotificationServiceManagerSpec.groovy, previous PR from Joe Witt covers the needed testing for Variable Registry/Value Lookup at this time
This closes #778.
2016-08-04 14:23:29 -04:00
joewitt
f987b21609 NIFI-1157 searched for and resolved all remaining references to deprecated items that were clearly addressable. 2016-07-14 09:32:35 -04:00
Oleg Zhurakousky
76690a8ee9 NIFI-1061 fixed deadlock caused by DBCPConnectionPool.onConfigured()
Current implementation of DBCPConnectionPool was attempting to test if connection could be obtained via dataSource.getConnection().
Such call is naturally a blocking call and the duration of the block is dependent on driver implementation. Some drivers (e.g., Phoenix - https://phoenix.apache.org/installation.html)
attempts numerous retries before failing creating a deadlock when attempt was made to disable DBCPConnectionPool which was still being enabled.

This fix removes the connection test from DBCPConnectionPool.onConfigured() operation returning successfully upon creation of DataSource.
For more details see comments in https://issues.apache.org/jira/browse/NIFI-1061
2015-11-11 08:33:51 -05:00
joewitt
aa99884782 NIFI-850 removed nifi parent, updated nifi pom, moved all nifi subdirs up one level, fixed readme. 2015-08-15 13:12:22 -04:00