Commit Graph

333 Commits

Author SHA1 Message Date
Dustin Rodrigues f0e8daf45d
NIFI-7317 - make .java files non-executable
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4181.
2020-04-06 12:19:46 +02:00
Joe Witt 992d990dd2
NIFI-7291 updated enforcement of dependency rules and build command
Reviewed by markap14
This closes #4166.

Signed-off-by: Joe Witt <joewitt@apache.org>
2020-03-27 16:34:59 -04:00
Bryan Bende 6412097eb2
NIFI-7257 Added HadoopDBCPConnectionPool
- Updated InstanceClassLoader to resolve files that are in the instance urls or additional urls
- Updated nifi-mock to support KerberosContext and removeProperty for ControllerServices
- Added unit test for HadoopDBCPConnectionPool
- Addressing review feedback

This closes #4149.
2020-03-24 11:05:18 -04:00
Joe Witt 254b20fa5f
NIFI-7274 add time for test conditions to be met 2020-03-22 09:54:43 -04:00
Bryan Bende 2feeb57159
NIFI-7221 Support v2 and v3 protocol version for Hortonworks Schema Registry
- Update nifi-nar-bundles/nifi-extension-utils/nifi-record-utils/nifi-avro-record-utils/src/main/java/org/apache/nifi/serialization/SchemaRegistryRecordSetWriter.java
- Addressing review feedback

This closes #4120.
2020-03-20 10:56:47 -04:00
Pierre Villard 1fe79021b5
NIFI-7221 Initial work 2020-03-20 10:56:32 -04:00
Joe Witt f694e6464f NIFI-7187 adding missing version strings from accumulo bundle pom
- Removed Cat X JSON.org dep inclusion which seems to not be necessary
- Updated a ton of easier/safer looking deps
- Updated tika due to CVE

This closes #4086

Signed-off-by: Mark Payne <markap14@hotmail.com>
2020-03-20 10:07:56 -04:00
Joe Witt 561be89a21
NIFI-7250 activate user.timezone appropriate to each region
NIFI-7250 fix a test which appears brittle at least on windows builds on slow environments
NIFI-7250 activated a timezone run for AU Australia/Melbourne which exposed a poor magic number and needless assertion but interesting results worth keeping

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4140.
2020-03-13 17:30:19 +01:00
Adam Taft decb5d062e
NIFI-6742 Use JUnit TemporaryFolder when creating test databases
- Add @Rule for TemporaryFolder
- Replace use of previous target/db with TemporaryFolder/db
- Remove use of ~/test db (in home directory)
- Remove System.out lines

Signed-off-by: Marc Parisi <phrocker@apache.org>

This closes #4137.
2020-03-12 20:48:52 -04:00
Joe Witt 97e250cdaa
NIFI-7244 Updated all tests which dont run well on windows to either work or be ignored on windows
Also dealt with unreliable tests which depend on timing by ignoring them or converting to IT.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4132.
2020-03-12 19:13:59 +01:00
Mark Payne 67676ba5b4 Fixed bug in JsonRowRecordReader when reading a 'raw' record with a schema that indicates that a field should be a Map. Also updated unit test to explicitly define schema, since schema inference will never return a Map but rather a Record 2020-03-11 14:29:44 -04:00
Pierre Villard cd83e70b91 unit test reproducing the issue 2020-03-11 14:29:44 -04:00
Bryan Bende e0fc75a963 NIFI-7019 Add kerberos principal and password properties to NiFi DBPCConnectionPool
This closes #4087.
2020-03-02 14:20:20 -05:00
Jeff Storck c74e71306e
Merge pull request #4071 from bbende/NIFI-7026
NIFI-7026 Add kerberos password property to NiFi HortonworksSchemaReg…
2020-03-02 12:50:20 -05:00
jstorck ab1658130d
NIFI-7024: Added Kerberos Password support to HBase_1_1_2_ClientService and HBase_2_ClientService
This closes #4103.
2020-03-02 11:43:13 -05:00
jstorck 614136ce51
NIFI-7018: Initial commit of processors extending AbstractHadoopProcessor supporting kerberos passwords
AbstractHadoopProcessor will always authenticate the principal with a KerberosUser implementation and a UGI will be acquired from the Subject associated with the KerberosUser implementation
AbstractHadoopProcessor's getUserGroupInformation method will now attempt to check the TGT and relogin if a KerberosUser impelmentation is available, otherwise it will return the UGI referenced in the HdfsResource instance
Updated AbstractHadoopProcessor's customValidate method to consider the provided password and updated validation failure explanations when a KerberosCredentialsService is specified together with a principal, password, or keytab
Added toString method override to AbstractKerberosUser
Updated Hive/HBase components to be compatible with the KerberosProperties.validatePrincipalWithKeytabOrPassword method
Fixed null ComponentLog in GetHDFSSequenceFileTest

Added package-protected accessor method (getAllowExplicitKeytabEnvironmentVariable) to AbstractHadoopProcessor for determining if the environment variable "NIFI_ALLOW_EXPLICIT_KEYTAB" has been set
AbstractHadoopProcessor will now only fail validation when the NIFI_ALLOW_EXPLICIT_KEYTAB environment variable is set to false if a keytab is provided to allow the user to specify a principal and password
Added AbstractHadoopProcessorSpec to verify validation of principal/keytab/password/kerberos credential service combinations

This closes #4095.
2020-02-28 10:10:19 -05:00
Yolanda M. Davis abf223d574
NIFI-7163 - added RulesEngine and RulesEngineProvider interfaces, enhanced easy rules to support provider interface and refactored to extract rules engine implementation
NIFI-7163 - updated documentation and comments

NIFI-7163 - fix checkstyle issues

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4081
2020-02-25 09:59:39 -05:00
Zoltan Kornel Torok e631851e0f
NIFI-7178 - Handle the case when schema is not available
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4067.
2020-02-22 11:07:31 -08:00
Matthew Burgess 1b0e3865e9
NIFI-7114: Fix file leaks in StandardCommsSession and S2S Reporting components
Signed-off-by: Joe Witt <joewitt@apache.org>
2020-02-21 17:37:54 -05:00
Bryan Bende 8fa855c8c3
NIFI-7026 Add kerberos password property to NiFi HortonworksSchemaRegistry 2020-02-21 14:56:33 -05:00
Troy Melhase 0de89452f1
NIFI-6927 Consolidate SSL context and trust managers for OkHttp on JDK9.
Fixes name conflicts.

This closes #4047.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2020-02-19 16:19:39 -08:00
Matthew Burgess dd6bcc7485
NIFI-7054: Add RecordSinkServiceLookup for dynamic sink routing (#4020)
* NIFI-7054: Add RecordSinkServiceLookup for dynamic sink routing

* NIFI-7054: Incorporated review comments
2020-02-03 09:45:08 -05:00
Joe Witt 3de77ebacc
NIFI-7021-RC3 prepare for next development iteration 2020-01-19 14:14:40 -05:00
Joe Witt 633408bce7
NIFI-7021-RC3 prepare release nifi-1.11.0-RC3 2020-01-19 14:14:38 -05:00
Nathan Gough 4ec9155cbc
NIFI-6770 - Set validator to Validator.VALID to allow empty password for truststores.
Added no-password keystore for tests
System NiFi truststore now allows a passwordless truststore. Added a unit test to prove this.
Forgot no-password-truststore.jks file for the unit test.
Refactored utility method from CertificateUtils to KeyStoreUtils.
Added utility methods to verify keystore and key passwords.
Added unit tests.
Implemented different keystore and truststore validation logic.
Refactored internal custom validation in StandardSSLContextService.
Added unit test resource for keystore with different key and keystore passwords.
Added unit test to generate passwordless truststore for https://nifi.apache.org for live testing.
Resolved NPE in SSLContext generation in StandardSSLContextService
Added unit test to generate passwordless truststore for localhost for InvokeHTTP testing.
Resolved TrustManagerFactoryImpl initialization error.
Fixed unit test without proper cleanup which caused RAT failures.

Co-authored-by: Andy LoPresto <alopresto@apache.org>

This closes #3823.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2020-01-17 20:01:06 -08:00
Joe Witt 23c8234586
NIFI-7031 updating copyright year on NOTICES 2020-01-15 16:10:31 -05:00
Matthew Burgess c604923c0b NIFI-6947: Add PutRecord processor using RecordSinkService (#3943)
* NIFI-6947: Add PutRecord processor using RecordSinkService

* NIFI-6947: Incorporated review comments
2020-01-08 04:25:14 -04:00
Pierre Villard ad636789f0
NIFI-6928 - add connect/read timeout to RestLookupService
This closes #3920

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2019-12-29 19:10:30 -05:00
Matthew Burgess 3a5a2da73c
NIFI-6954: Evaluate EL for DBCPConnectionPool properties
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3940.
2019-12-19 11:44:55 +01:00
Pierre Villard ac5bacccb8
NIFI-6839 - Upgrade jackson-databind direct dependencies
This closes #3870

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2019-11-25 10:58:22 -05:00
Bryan Bende 388683a5c5 NIFI-6808 Adding KeytabCredentialService to HortonworksSchemaRegistry and setting dynamic JAAS config property on client
This closes #3877.
2019-11-20 20:00:46 -05:00
Koji Kawamura b3880a4a06
NIFI-5970 Handle multiple input FlowFiles at Put.initConnection
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3583
2019-11-19 16:48:14 -05:00
Bryan Bende 5d9bba2f6f
NIFI-6857 Bump hbase-client from 2.1.x to 2.2.x
This closes #3878

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2019-11-12 09:53:30 -05:00
Tamas Palfy 84a05c8595 NIFI-6729 - Created AbstractSingleAttributeBasedControllerServiceLookup and updated DBCPConnectionPoolLookup and AzureStorageCredentialsControllerServiceLookup to inherit from it.
This closes #3774.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2019-11-05 14:02:03 +01:00
Joe Witt f8c3d877cf
NIFI-6733 updating to next release version for master branch 2019-11-04 13:31:39 -05:00
Joe Witt 4119279c03
Merge branch 'NIFI-6733-RC3' 2019-11-04 13:24:29 -05:00
Joe Witt 363b751310
Revert "NIFI-6733 bumping master to 1.11.0-SNAPSHOT"
This reverts commit a7c56cfd9a.
2019-11-04 13:24:13 -05:00
Joe Witt a7c56cfd9a
NIFI-6733 bumping master to 1.11.0-SNAPSHOT 2019-11-04 07:54:17 -05:00
Yolanda M. Davis 5b28f6dad9
NIFI-6803 - Initial commit for ActionHandler Controller Services
NIFI-6803 - updated to description

NIFI-6803 - add Spring reference to Notice

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3856
2019-11-01 10:41:25 -04:00
Joe Witt 418179f5b2
NIFI-6733-RC3 prepare for next development iteration 2019-10-28 15:13:13 -07:00
Joe Witt b217ae20ad
NIFI-6733-RC3 prepare release nifi-1.10.0-RC3 2019-10-28 15:12:57 -07:00
Matthew Burgess 8771c35f4a NIFI-6799: Add PrometheusRecordSink and reset() method to RecordSinkService (#3839)
This closes #3839
2019-10-24 11:53:28 -04:00
Matthew Burgess ace23c35b5 NIFI-6780: Add QueryNiFiReportingTask, RecordSinkService, S2S and DB impls (#3826)
* NIFI-6780: Introduce RecordSinkService to separate format and destination, refactor common S2S utils

Added QueryNiFiReportingTask to SQL query NiFi status and metrics

Add PROCESSOR_STATUS and PROCESS_GROUP_STATUS

Add CONNECTION_STATUS_PREDICTIONS

check for null predictions

Add ConnectionStatusRecursiveIterator

Fix issue w/ duplicate iterator outputs

Refactored query interfaces, fixed assembly POM

Rebased v master, fixed isBackPressureEnabled and Checkstyle/RAT errors

Add DatabaseRecordSink service (#13)

* Add DatabaseRecordSink service

* Incorporated review comments

* NIFI-6780: Add/fix docs, cleanup warnings, fixed some table definitions

* Added bundle profile, remove predictions table if not enabled

* Added doc for which tables are available when

This closes #3826.
2019-10-22 08:56:42 -04:00
Yolanda M. Davis 7c9d34f820
NIFI-6778 - Added rules engine service api and Easy Rules Implementation
NIFI-6778 - added comments and component description annotations

NIFI-6778 - added license and notice details

NIFI-6778 - added additional details section and updated description of service

NIFI-6778 - fixed checkstyle error

NIFI-6778 - addressed review comments for documentation and description corrections

NIFI-6778 - added include-rules profile for assembly build

NIFI-6778 - add MVEL and ASM to license and notice

NIFI-6778 - switch to use Jackson for NiFi Rules Json deserialization

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3824
2019-10-21 12:16:11 -04:00
Bryan Bende af81afce40
NIFI-5753 Make use of keyPassword optional and only used when keystore is used, bump registry client version
This closes #3126.
2019-10-10 15:09:28 -04:00
Grzegorz Kołakowski 5d65e6aba4
NIFI-5753 Add SSL support to HortonworksSchemaRegistry service
Signed-off-by: Bryan Bende <bbende@apache.org>
2019-10-10 15:09:16 -04:00
Mark Payne e394f6683a
NIFI-6753: Fixed bug where all values being provided to the CSV Writer were String objects, which resulted in the CSV Writer improperly quoting numeric values when the schema indicates that the value is a number. Now, we will only convert the value to a String if the value is not a Number and/or the schema does not indicate a numeric ty type
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3797
2019-10-09 09:54:45 -04:00
Joe Witt 0f02de6002
NIFI-6733 updating key apache commons dependencies and apache base dependency for build
This closes #3791.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2019-10-07 22:20:48 -04:00
Mike 02d3b7e92b NIFI-6656 Added a default visibility expression configuration item to… (#3723)
* NIFI-6656 Added a default visibility expression configuration item to the HBase map cache client services.

* NIFI-6656 Fixed validation bug.

* NIFI-6656 Added stylecheck fix.

* NIFI-6656 Deleted shared interface and moved property descriptor into both services.

* NIFI-6656 Removed dependency from hbase api.
2019-10-03 12:53:53 -04:00
James Cheng f1d35f46ce NIFI-6684 Add more property to Hive3ConnectionPool (#3763)
* NiFi-6684 Add more property to Hive3ConnectionPool

signed-off by: Peter Wicks <patricker@gmail.com>
2019-10-03 07:24:29 -06:00