Commit Graph

5947 Commits

Author SHA1 Message Date
Mark Payne 728bdec0f4 NIFI-7339: Fixed bug that caused Write Ahead Provenance Repository not to rollover event files after specified time. Code cleanup. Updated some default properties. 2020-04-09 13:32:34 -04:00
Andy LoPresto 1ec7e31f11
NIFI-7341 Updated certificate commands and source code formatting in Toolkit Guide. (#4196) 2020-04-09 10:13:53 -07:00
Matthew Burgess 84968e70d2
NIFI-7087: Use FlowManager.findAllConnections() when available
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4026
2020-04-09 12:59:01 -04:00
Mark Payne 683b1d9952
NIFI-6849: Reworked how nodes inherit cluster information when joining a cluster. Now, if there are conflicts, a local copy is made of the flow/authorizations/etc. and the cluster's flow is inherited.
- Refactored Flow Synchronization to make code cleaner
 - Updated Authorizers to forcibly inherit Users, Groups, and Access Policies if the local flow is empty.
 - Updated FlowFileRepositories to use SerializedRepositoryRecord instead of RepositoryRecord, so that we have the ability to read records without already knowing the Queue objects. Updated StandardFlowSynchronizer so that if the flow is not inheritable but the controller has not yet been initialized, the flow is backed up and replaced instead of NiFi failing to start
- Added system tests. Updated FlowController so that if it fails to inherit flow due to flow uninheritability that it notifies the cluster of this instead of remaining in the 'CONNECTING' state.
- Added additional log statements to aid in debugging

NIFI-6849: Rebased against master. Updated Admin Guide to describe new cluster flow inheritance behavior

NIFI-6849: Addressed review feedback

NIFI-6849: Addressed review feedback: Relocated logic for bundle compatibility into the BundleCompatibilityCheck class. Fixed logic that prevented users/groups/policies from being forcibly inherited during startup

This closes #3891
2020-04-09 10:42:05 -04:00
Peter Turcsanyi 08dcd4af14
NIFI-7345: Fixed Hive database and table names case insensitivity in Atlas reporting task
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4198.
2020-04-09 14:13:54 +02:00
muazmaz 63379c3520 NIFI-7259 Adding DeleteDataLakeStorage Processor to provide native support for Azure Data lake Gen 2 Storage.
Updated to remove unused variables
NIFI-7259 import and property description changes

This closes #4189.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-04-08 10:58:48 +02:00
Tamas Palfy 09cece8e99 NIFI-7314 HandleHttpRequest stops Jetty in OnUnscheduled instead of OnStopped. Also reject pending request and clean their queue when shutting down.
NIFI-7314 In HandleHttpRequest returning 503 when rejecting pending requests before shutdown.
NIFI-7314 In HandleHttpRequest add logs and better response message during cleanup.

This closes #4191.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-04-07 20:12:18 +02:00
Joe Witt 445efcfde6
NIFI-7326 updated URL to find splunk artifacts (#4188)
Signed-off-by: Andy LoPresto <alopresto@apache.org>
2020-04-06 20:36:56 -07:00
M Tien 59c756c72b
NIFI-7126 Increased test iterations to 10,000 in Argon2SecureHasherTe… (#4187)
* NIFI-7126 Increased test iterations to 10,000 in Argon2SecureHasherTest#testDefaultCostParamsShouldBeSufficient to avoid JVM warmup issues.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2020-04-06 18:26:32 -07:00
Dustin Rodrigues f0e8daf45d
NIFI-7317 - make .java files non-executable
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4181.
2020-04-06 12:19:46 +02:00
Matthew Burgess a093af2d42
NIFI-7273: Add flow metrics REST endpoint with for Prometheus scraping (#4156)
* NIFI-7273: Add flow metrics REST endpoint with for Prometheus scraping

* NIFI-7273: Changed method name, fix handling when analytics not enabled

* NIFI-7273: Removed attachment header from Prometheus metrics endpoint

* NIFI-7273: Removed unused variable
2020-04-03 16:12:53 -04:00
Bence Simon c0f5fcb484 NIFI-7188 Extending UI search with filters and refactoring existing solution
This closes #4123.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-04-02 21:49:40 +02:00
Joe Witt a9bed56f68
NIFI-7311 adding additional cleanup and output to github actions yml and not running nifi-system-test module
Self merging wo review as part of github ci actions stability.

Signed-off-by: Joe Witt <joewitt@apache.org>
2020-04-02 12:54:49 -04:00
Otto Fowler d9ac9e44a7
add validator for lists that ensure the element validator is called for empty entries
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4116
2020-04-02 09:29:06 -04:00
Mike Thomsen 4a2a91135e
NIFI-7293 Add in-memory janusgraph implementation of GraphClientService to help with live testing.
Added new in memory janus graph client for testing.
Added integration test to ExecuteGraphQuery.

NIFI-7293 Added missing getter.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4168
2020-04-01 10:00:41 -04:00
muazmaz daddf400a2 NIFI-7103 Adding PutDataLakeStorage Processor to provide native support for Azure Data Lake Storage Gen 2 Storage.
added data-lake dependency
NIFI-7103 fixed indentation
Update to add IllegalArgumentException
Fixed indentation and logging
nifi-7103 review changes
nifi-7103 root directory and exception change

This closes #4126.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-03-31 11:15:07 +02:00
KovalevIV f4a7aafe4a
NIFI-7297 add available()
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4172.
2020-03-31 09:47:01 +02:00
Mubashir Kazia fee1b8b8e0
NIFI-7294 Address deprecation issues in solrj and httpclient
Some calls to deprecated methods in httpclient were resulting in
UnsupportedOperationException. Use the new API calls in both httpclient
and solrj. Add an integration test to include test coverage for
org.apache.nifi.processors.solr.SolrUtils.createClient

This closes #4171.
2020-03-30 16:49:24 -04:00
Joe Witt 992d990dd2
NIFI-7291 updated enforcement of dependency rules and build command
Reviewed by markap14
This closes #4166.

Signed-off-by: Joe Witt <joewitt@apache.org>
2020-03-27 16:34:59 -04:00
Marc Parisi 1343b00e0b NIFI-7290: omit transitive that is causing the build to fail and not needed for the test scope 2020-03-27 13:12:13 -04:00
Matthew Burgess be2929b8b0
NIFI-7287: Move services-api dependency from Prometheus reporting task to its NAR
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4162
2020-03-27 08:48:41 -04:00
Tamas Palfy c79473baf5 NIFI-7286 ListenTCPRecord cleanup changed from @OnStopped to @OnUnscheduled 2020-03-26 11:45:09 -04:00
Troy Melhase 483f23a8aa
NIFI-7153 Adds ContentLengthFilter to enforce configurable maximum length on incoming HTTP requests.
Adds DoSFilter to enforce configurable maximum on incoming HTTP requests per second.
Redirected log messages for ContentLengthFilter to nifi-app.log in logback.xml.

This closes #4125.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2020-03-25 17:23:22 -07:00
Bryan Bende 975f2bdc4f
NIFI-7279 This closes #4160. Protect against NPE in RedisDistributedMapCacheClientService when value is null
Signed-off-by: Joe Witt <joewitt@apache.org>
2020-03-25 17:40:20 -04:00
Bryan Bende 5784888082
NIFI-7281 This closes #4159. Use BufferedInputStream in StandardSocketChannelRecordReader in order to support mark/reset
Signed-off-by: Joe Witt <joewitt@apache.org>
2020-03-25 17:39:44 -04:00
Joe Witt 271de92046
NIFI-7238 printing maven version info in same build command 2020-03-25 15:21:25 -04:00
Joe Witt 7d83622f05
NIFI-7238 remove no longer needed delete command 2020-03-25 12:01:45 -04:00
Joe Witt efe2bca4cb
NIFI-7238 Continue to improve Github Actions CI stability 2020-03-25 11:44:20 -04:00
Michael Hogue aeaf953e06
NIFI-5925: Added controller services to set of components that are searched
NIFI-5925: cleanup, add negative test

NIFI-5925: fixed checkstyle

This closes #4105

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-03-25 08:55:44 -04:00
Bryan Bende 5fd25d6235
NIFI-7278 Adding support for SCRAM-SHA-512 to Kafka 2.0 processors 2020-03-24 16:53:22 -04:00
Joe Witt 4d84c144d2
NIFI-7238 Improve Caching for Github CI and relax core usage to not max out cores all moving toward more stable builds.
Signed-off-by: Joe Witt <joewitt@apache.org>
2020-03-24 16:41:23 -04:00
Mubashir Kazia 9b4292024b
NIFI-7269 - Upgrade solrj version to 7 in nifi-solr-processors
Remove unused imports

Use the latest solrj version(8.4.1)

Setup default schemaFactory for tests
The default schemaFactory ManagedIndexSchemaFactory creates
additional files in test's resources directory. Change it to
ClassicIndexSchemaFactory for classic behavior.

This closes #4152.

Signed-off-by: Bryan Bende <bbende@apache.org>
2020-03-24 14:55:51 -04:00
Bryan Bende 6412097eb2
NIFI-7257 Added HadoopDBCPConnectionPool
- Updated InstanceClassLoader to resolve files that are in the instance urls or additional urls
- Updated nifi-mock to support KerberosContext and removeProperty for ControllerServices
- Added unit test for HadoopDBCPConnectionPool
- Addressing review feedback

This closes #4149.
2020-03-24 11:05:18 -04:00
Bryan Bende 7f32aa56db
NIFI-7271 Make command timeout configurable for ShellUserGroupProvider
- Changing ShellRunner to use a separate thread for reading the output of the process
- Removing unused member variable
- Addressing review feedback

This closes #4154.
2020-03-23 20:43:06 -04:00
Joe Witt 254b20fa5f
NIFI-7274 add time for test conditions to be met 2020-03-22 09:54:43 -04:00
Bryan Bende 2feeb57159
NIFI-7221 Support v2 and v3 protocol version for Hortonworks Schema Registry
- Update nifi-nar-bundles/nifi-extension-utils/nifi-record-utils/nifi-avro-record-utils/src/main/java/org/apache/nifi/serialization/SchemaRegistryRecordSetWriter.java
- Addressing review feedback

This closes #4120.
2020-03-20 10:56:47 -04:00
Pierre Villard 1fe79021b5
NIFI-7221 Initial work 2020-03-20 10:56:32 -04:00
Joe Witt f694e6464f NIFI-7187 adding missing version strings from accumulo bundle pom
- Removed Cat X JSON.org dep inclusion which seems to not be necessary
- Updated a ton of easier/safer looking deps
- Updated tika due to CVE

This closes #4086

Signed-off-by: Mark Payne <markap14@hotmail.com>
2020-03-20 10:07:56 -04:00
Otto Fowler 23fa2d3138
NIFI-7264 Make jsonPath Expression Logging More Reasonable
add special handling of PathNotFoundExceptions to log to debug
fix spelling error
wrap debug log in guard per review

This closes #4148

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-03-19 12:24:26 -04:00
Eduardo Fontes 943de310ad
NIFI-6293
Add support to Mongo Extended JSON v2
Add org.json lib
Replace evil json
Replace evil json for alternative
Include testExtendedJsonSupport
Style adjustment
Remove unecessary new JSON parser
Fix query in testExtendedJsonSupport
Parse with Jackson and BSON
Back to default MONGO_URI

This closes #4068

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-03-18 18:36:09 -04:00
M Tien f91d6c420d
NIFI-7268 Removed org.mindrot.jBcrypt library and replaced with at.fa… (#4151)
* NIFI-7268 Removed org.mindrot.jBcrypt library and replaced with at.favre.lib.bcrypt library.
Updated LICENSE and NOTICE files to reflect changes.
Updated unit tests.

Co-authored-by: Andy LoPresto <alopresto@apache.org>

* NIFI-7268 Fixed typo in Javadoc.

Co-authored-by: Andy LoPresto <alopresto@apache.org>
2020-03-17 19:49:15 -07:00
Pierre Villard 7105bf36a8
NIFI-7267 - Upgrade spring-data-redis in Redis bundle (#4150)
Signed-off-by: Andy LoPresto <alopresto@apache.org>
2020-03-17 15:51:28 -07:00
Pierre Villard 71bf12976f
NIFI-7258 - fix overflow in PutAzureEventHub when not configured correctly
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4146.
2020-03-16 21:28:12 +01:00
Andy LoPresto 3feb85a030
NIFI-7223 Resolved merge conflicts from additional test case for null key password.
This closes #4145.

Signed-off-by: Joe Witt <joewitt@apache.org>
2020-03-16 12:47:41 -04:00
Nathan Gough 69b6c231fd
NIFI-7223 - Added another test for when keyPasswd is not present. 2020-03-16 12:28:18 -04:00
Andy LoPresto f9d75056fa
NIFI-7223 [WIP] Resolved compilation issues in unit test on OpenJDK 11 by removing Sun security class references.
Added OkHttpReplicationClient#isTLSConfigured() method.
Added unit test.
NIFI-7223 Fixed remaining unit tests for TLS regression.
Renamed tests for clarity.
2020-03-16 12:28:01 -04:00
Nathan Gough 7374361b5c
NIFI-7223 - Fixed a minor issue where the OkHttpReplicationClient class loaded blank properties as empty string instead of an expected null value. Added a isNotBlank check. Added unit tests for replication client and HTTPNotificationService.
NIFI-7223 - Renamed some variables and methods.
NIFI-7223 - Removed unused dependency. Corrected security properties in administration-guide.
2020-03-16 12:26:50 -04:00
Mike 62c34d7e23
Merge pull request #4031 from jfrazee/NIFI-6791
NIFI-6791 Add UUID3 and UUID5 functions to Expression Language
2020-03-15 10:27:20 -04:00
Mike 60b6327757
Merge branch 'master' into NIFI-6791 2020-03-15 09:26:44 -04:00
Matthew Burgess ace5754524 NIFI-7208: Restore default timezone in JdbcCommon 2020-03-13 15:00:24 -04:00