Commit Graph

5754 Commits

Author SHA1 Message Date
Bence Simon 83400789f6
NIFI-7292 Preventing file listing from fail because of insufficient privileges
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4195.
2020-04-15 18:54:51 +02:00
Rob Fellows c263daf20b
[NIFI-7358] - Fix: Sorting on 'Estimated Time to Back Pressure' in the Connection summary table does not work properly
- fix style issues
- review feedback

This closes #4208
2020-04-15 10:06:38 -04:00
sjyang18 9df53e7204
NIFI-6977 - Change the reporting behavior of Azure Reporting task to report report the time when metrics are generated
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4211.
2020-04-15 12:53:18 +02:00
Mark Payne a61a4c2b58
NIFI-7347: Fixed NullPointerException that can happen if a bin is merged due to timeout and has no records
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4210.
2020-04-15 12:39:13 +02:00
Matthew Burgess c9f48e3788
NIFI-7354: Allow analytics properties to be set via environment variables in docker scripts
This closes #4203.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2020-04-13 09:15:05 -04:00
Andy LoPresto 07a8311b4c
NIFI-7319 Add walkthrough document (#4193)
* NIFI-7319 Added first draft of walkthroughs doc.

* NIFI-7319 Added instructions and screenshots for securing standalone NiFi instance.

* NIFI-7319 Added instructions and screenshots for instructing OS & browser to trust self-signed certificate.

* NIFI-7319 Added instructions and screenshots for securing NiFi with externally-provided certificates.

* NIFI-7319 Added instructions and screenshots for building NiFi from source.

* NIFI-7319 [WIP] Converting secure cluster instructions to match format.
Fixed instructions regarding embedded ZooKeeper configuration.

* NIFI-7319 Completed secure cluster walkthrough.

* NIFI-7319 Added walkthroughs to documentation navigation list.

* NIFI-7319 Incorporated PR feedback on broken links.

* NIFI-7319 Removed line number helpers from update sections.

* NIFI-7319 Incorporated final PR review items.

Co-authored-by: Sandra Pius <spiusapache@gmail.com>
2020-04-10 22:25:40 -07:00
Mark Payne c99a187018 NIFI-7346: Ensure that the Provenance Repository doesn't delete the Active Event File 2020-04-09 13:34:37 -04:00
Mark Payne 728bdec0f4 NIFI-7339: Fixed bug that caused Write Ahead Provenance Repository not to rollover event files after specified time. Code cleanup. Updated some default properties. 2020-04-09 13:32:34 -04:00
Andy LoPresto 1ec7e31f11
NIFI-7341 Updated certificate commands and source code formatting in Toolkit Guide. (#4196) 2020-04-09 10:13:53 -07:00
Matthew Burgess 84968e70d2
NIFI-7087: Use FlowManager.findAllConnections() when available
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4026
2020-04-09 12:59:01 -04:00
Mark Payne 683b1d9952
NIFI-6849: Reworked how nodes inherit cluster information when joining a cluster. Now, if there are conflicts, a local copy is made of the flow/authorizations/etc. and the cluster's flow is inherited.
- Refactored Flow Synchronization to make code cleaner
 - Updated Authorizers to forcibly inherit Users, Groups, and Access Policies if the local flow is empty.
 - Updated FlowFileRepositories to use SerializedRepositoryRecord instead of RepositoryRecord, so that we have the ability to read records without already knowing the Queue objects. Updated StandardFlowSynchronizer so that if the flow is not inheritable but the controller has not yet been initialized, the flow is backed up and replaced instead of NiFi failing to start
- Added system tests. Updated FlowController so that if it fails to inherit flow due to flow uninheritability that it notifies the cluster of this instead of remaining in the 'CONNECTING' state.
- Added additional log statements to aid in debugging

NIFI-6849: Rebased against master. Updated Admin Guide to describe new cluster flow inheritance behavior

NIFI-6849: Addressed review feedback

NIFI-6849: Addressed review feedback: Relocated logic for bundle compatibility into the BundleCompatibilityCheck class. Fixed logic that prevented users/groups/policies from being forcibly inherited during startup

This closes #3891
2020-04-09 10:42:05 -04:00
Peter Turcsanyi 08dcd4af14
NIFI-7345: Fixed Hive database and table names case insensitivity in Atlas reporting task
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4198.
2020-04-09 14:13:54 +02:00
muazmaz 63379c3520 NIFI-7259 Adding DeleteDataLakeStorage Processor to provide native support for Azure Data lake Gen 2 Storage.
Updated to remove unused variables
NIFI-7259 import and property description changes

This closes #4189.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-04-08 10:58:48 +02:00
Tamas Palfy 09cece8e99 NIFI-7314 HandleHttpRequest stops Jetty in OnUnscheduled instead of OnStopped. Also reject pending request and clean their queue when shutting down.
NIFI-7314 In HandleHttpRequest returning 503 when rejecting pending requests before shutdown.
NIFI-7314 In HandleHttpRequest add logs and better response message during cleanup.

This closes #4191.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-04-07 20:12:18 +02:00
Joe Witt 445efcfde6
NIFI-7326 updated URL to find splunk artifacts (#4188)
Signed-off-by: Andy LoPresto <alopresto@apache.org>
2020-04-06 20:36:56 -07:00
M Tien 59c756c72b
NIFI-7126 Increased test iterations to 10,000 in Argon2SecureHasherTe… (#4187)
* NIFI-7126 Increased test iterations to 10,000 in Argon2SecureHasherTest#testDefaultCostParamsShouldBeSufficient to avoid JVM warmup issues.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2020-04-06 18:26:32 -07:00
Dustin Rodrigues f0e8daf45d
NIFI-7317 - make .java files non-executable
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4181.
2020-04-06 12:19:46 +02:00
Matthew Burgess a093af2d42
NIFI-7273: Add flow metrics REST endpoint with for Prometheus scraping (#4156)
* NIFI-7273: Add flow metrics REST endpoint with for Prometheus scraping

* NIFI-7273: Changed method name, fix handling when analytics not enabled

* NIFI-7273: Removed attachment header from Prometheus metrics endpoint

* NIFI-7273: Removed unused variable
2020-04-03 16:12:53 -04:00
Bence Simon c0f5fcb484 NIFI-7188 Extending UI search with filters and refactoring existing solution
This closes #4123.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-04-02 21:49:40 +02:00
Joe Witt a9bed56f68
NIFI-7311 adding additional cleanup and output to github actions yml and not running nifi-system-test module
Self merging wo review as part of github ci actions stability.

Signed-off-by: Joe Witt <joewitt@apache.org>
2020-04-02 12:54:49 -04:00
Otto Fowler d9ac9e44a7
add validator for lists that ensure the element validator is called for empty entries
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4116
2020-04-02 09:29:06 -04:00
Mike Thomsen 4a2a91135e
NIFI-7293 Add in-memory janusgraph implementation of GraphClientService to help with live testing.
Added new in memory janus graph client for testing.
Added integration test to ExecuteGraphQuery.

NIFI-7293 Added missing getter.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4168
2020-04-01 10:00:41 -04:00
muazmaz daddf400a2 NIFI-7103 Adding PutDataLakeStorage Processor to provide native support for Azure Data Lake Storage Gen 2 Storage.
added data-lake dependency
NIFI-7103 fixed indentation
Update to add IllegalArgumentException
Fixed indentation and logging
nifi-7103 review changes
nifi-7103 root directory and exception change

This closes #4126.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-03-31 11:15:07 +02:00
KovalevIV f4a7aafe4a
NIFI-7297 add available()
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4172.
2020-03-31 09:47:01 +02:00
Mubashir Kazia fee1b8b8e0
NIFI-7294 Address deprecation issues in solrj and httpclient
Some calls to deprecated methods in httpclient were resulting in
UnsupportedOperationException. Use the new API calls in both httpclient
and solrj. Add an integration test to include test coverage for
org.apache.nifi.processors.solr.SolrUtils.createClient

This closes #4171.
2020-03-30 16:49:24 -04:00
Joe Witt 992d990dd2
NIFI-7291 updated enforcement of dependency rules and build command
Reviewed by markap14
This closes #4166.

Signed-off-by: Joe Witt <joewitt@apache.org>
2020-03-27 16:34:59 -04:00
Marc Parisi 1343b00e0b NIFI-7290: omit transitive that is causing the build to fail and not needed for the test scope 2020-03-27 13:12:13 -04:00
Matthew Burgess be2929b8b0
NIFI-7287: Move services-api dependency from Prometheus reporting task to its NAR
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4162
2020-03-27 08:48:41 -04:00
Tamas Palfy c79473baf5 NIFI-7286 ListenTCPRecord cleanup changed from @OnStopped to @OnUnscheduled 2020-03-26 11:45:09 -04:00
Troy Melhase 483f23a8aa
NIFI-7153 Adds ContentLengthFilter to enforce configurable maximum length on incoming HTTP requests.
Adds DoSFilter to enforce configurable maximum on incoming HTTP requests per second.
Redirected log messages for ContentLengthFilter to nifi-app.log in logback.xml.

This closes #4125.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2020-03-25 17:23:22 -07:00
Bryan Bende 975f2bdc4f
NIFI-7279 This closes #4160. Protect against NPE in RedisDistributedMapCacheClientService when value is null
Signed-off-by: Joe Witt <joewitt@apache.org>
2020-03-25 17:40:20 -04:00
Bryan Bende 5784888082
NIFI-7281 This closes #4159. Use BufferedInputStream in StandardSocketChannelRecordReader in order to support mark/reset
Signed-off-by: Joe Witt <joewitt@apache.org>
2020-03-25 17:39:44 -04:00
Joe Witt 271de92046
NIFI-7238 printing maven version info in same build command 2020-03-25 15:21:25 -04:00
Joe Witt 7d83622f05
NIFI-7238 remove no longer needed delete command 2020-03-25 12:01:45 -04:00
Joe Witt efe2bca4cb
NIFI-7238 Continue to improve Github Actions CI stability 2020-03-25 11:44:20 -04:00
Michael Hogue aeaf953e06
NIFI-5925: Added controller services to set of components that are searched
NIFI-5925: cleanup, add negative test

NIFI-5925: fixed checkstyle

This closes #4105

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-03-25 08:55:44 -04:00
Bryan Bende 5fd25d6235
NIFI-7278 Adding support for SCRAM-SHA-512 to Kafka 2.0 processors 2020-03-24 16:53:22 -04:00
Joe Witt 4d84c144d2
NIFI-7238 Improve Caching for Github CI and relax core usage to not max out cores all moving toward more stable builds.
Signed-off-by: Joe Witt <joewitt@apache.org>
2020-03-24 16:41:23 -04:00
Mubashir Kazia 9b4292024b
NIFI-7269 - Upgrade solrj version to 7 in nifi-solr-processors
Remove unused imports

Use the latest solrj version(8.4.1)

Setup default schemaFactory for tests
The default schemaFactory ManagedIndexSchemaFactory creates
additional files in test's resources directory. Change it to
ClassicIndexSchemaFactory for classic behavior.

This closes #4152.

Signed-off-by: Bryan Bende <bbende@apache.org>
2020-03-24 14:55:51 -04:00
Bryan Bende 6412097eb2
NIFI-7257 Added HadoopDBCPConnectionPool
- Updated InstanceClassLoader to resolve files that are in the instance urls or additional urls
- Updated nifi-mock to support KerberosContext and removeProperty for ControllerServices
- Added unit test for HadoopDBCPConnectionPool
- Addressing review feedback

This closes #4149.
2020-03-24 11:05:18 -04:00
Bryan Bende 7f32aa56db
NIFI-7271 Make command timeout configurable for ShellUserGroupProvider
- Changing ShellRunner to use a separate thread for reading the output of the process
- Removing unused member variable
- Addressing review feedback

This closes #4154.
2020-03-23 20:43:06 -04:00
Joe Witt 254b20fa5f
NIFI-7274 add time for test conditions to be met 2020-03-22 09:54:43 -04:00
Bryan Bende 2feeb57159
NIFI-7221 Support v2 and v3 protocol version for Hortonworks Schema Registry
- Update nifi-nar-bundles/nifi-extension-utils/nifi-record-utils/nifi-avro-record-utils/src/main/java/org/apache/nifi/serialization/SchemaRegistryRecordSetWriter.java
- Addressing review feedback

This closes #4120.
2020-03-20 10:56:47 -04:00
Pierre Villard 1fe79021b5
NIFI-7221 Initial work 2020-03-20 10:56:32 -04:00
Joe Witt f694e6464f NIFI-7187 adding missing version strings from accumulo bundle pom
- Removed Cat X JSON.org dep inclusion which seems to not be necessary
- Updated a ton of easier/safer looking deps
- Updated tika due to CVE

This closes #4086

Signed-off-by: Mark Payne <markap14@hotmail.com>
2020-03-20 10:07:56 -04:00
Otto Fowler 23fa2d3138
NIFI-7264 Make jsonPath Expression Logging More Reasonable
add special handling of PathNotFoundExceptions to log to debug
fix spelling error
wrap debug log in guard per review

This closes #4148

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-03-19 12:24:26 -04:00
Eduardo Fontes 943de310ad
NIFI-6293
Add support to Mongo Extended JSON v2
Add org.json lib
Replace evil json
Replace evil json for alternative
Include testExtendedJsonSupport
Style adjustment
Remove unecessary new JSON parser
Fix query in testExtendedJsonSupport
Parse with Jackson and BSON
Back to default MONGO_URI

This closes #4068

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-03-18 18:36:09 -04:00
M Tien f91d6c420d
NIFI-7268 Removed org.mindrot.jBcrypt library and replaced with at.fa… (#4151)
* NIFI-7268 Removed org.mindrot.jBcrypt library and replaced with at.favre.lib.bcrypt library.
Updated LICENSE and NOTICE files to reflect changes.
Updated unit tests.

Co-authored-by: Andy LoPresto <alopresto@apache.org>

* NIFI-7268 Fixed typo in Javadoc.

Co-authored-by: Andy LoPresto <alopresto@apache.org>
2020-03-17 19:49:15 -07:00
Pierre Villard 7105bf36a8
NIFI-7267 - Upgrade spring-data-redis in Redis bundle (#4150)
Signed-off-by: Andy LoPresto <alopresto@apache.org>
2020-03-17 15:51:28 -07:00
Pierre Villard 71bf12976f
NIFI-7258 - fix overflow in PutAzureEventHub when not configured correctly
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4146.
2020-03-16 21:28:12 +01:00