Commit Graph

5095 Commits

Author SHA1 Message Date
Mark Payne 38db4e97cd
NIFI-5464: Consider ports invalid unless they have both incoming and outgoing connections
This closes #3417.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-04-09 10:19:24 -04:00
Lars Francke 49e74b29db NIFI-6198 Infinite recursion in HBase*ClientService
This closes #3419.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2019-04-09 09:29:32 +09:00
Kourge cded30b3d2 NIFI-5953 Manage GetTwitter connection retries on '420 Enhance Your Calm' exceptions
NIFI-5953 Manage GetTwitter connection retries on '420 Enhance Your Calm' exceptions

Update "Max Client Error Retries" parameter name.
reintriduce client.reconnect() on HTTP_ERROR 420

This closes #3276.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2019-04-09 09:14:44 +09:00
Rob Fellows fb9f2af04a
NIFI-6136 - fix: UI - User and Template bug when page popped out
NIFI-6136 - return the appropriate value (false) when verifyDisconnectedCluster is called from a non-clustered environment.

This closes #3390
2019-04-08 15:45:19 -04:00
Matthew Burgess 8572852bdf
NIFI-6184: This closes #3413. Removed unnecessary test-jar dep from nifi-livy-processors causing race condition, added assertion to catch failures
Signed-off-by: Joe Witt <joewitt@apache.org>
2019-04-08 14:52:40 -04:00
Alexander T da593a2e8e Update WriteResourceToStream.java
Thread.currentThread().getContextClassLoader().getResourceAsStream(...) works in UnitTest, but not if the NAR is deployed in /extensions folder. If you want to use the processer the resource 'file.txt' is not found.
I changed this to 'getClass().getClassLoader()...' and its working as UnitTest and when deployed

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3381
2019-04-08 13:18:00 -04:00
Koji Kawamura 1c588f10b2 NIFI-5979 Add Line-by-Line Evaluation Mode to ReplaceText
Refactored to use functions to better handle strategy specific variables
via closure.
2019-04-08 16:19:48 +09:00
pushpavanthar d222f14a9e NIFI-5979 : enhanced ReplaceText processor with "Number of Occurrences" and "Occurrence offset" configurations
Fixed indentation errors to pass checkstyle-checks

Added Evaluation Modes as per discussion in PR thread

Adding exclusions of test files on rat plugin

Added new property 'Line-by-Line Evaluation Mode' and refactored common code

This closes #3375.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2019-04-08 16:19:27 +09:00
Bryan Bende 25cb29e109
NIFI-6177 Refactor HBaseListLookupService tests to remove use of TestRecordLookupProcessor
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3400.
2019-04-06 14:31:50 +02:00
bdesert 75217b33d0
NIFI-6181 FetchSFTP and FetchFTP File Not Found fix
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3407.
2019-04-06 14:27:39 +02:00
Alex Aversa 96279415a7 NIFI-6188 - Added 1s delay on Search control autocomplete query event to prevent premature results retrieval
This closes #3408.
Signed-off-by: Brandon Devries <devriesb@apache.org>
2019-04-05 12:32:40 -04:00
Mike Thomsen 4810f6d32e NIFI-6172 Fixed a bug that caused ElasticSearchStringLookupService to always return a value even when it should return Optional.empty().
NIFI-6172 Fixed broken integration test.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3399
2019-04-05 11:12:30 -04:00
Lars Francke d372318d47 NIFI-6191 HBase 2 ClientService documentation refers to HBase 1.x
This closes #3409.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2019-04-05 14:55:58 +09:00
Rob Fellows e374d2cc30
NIFI-6140 - Incorrect title on the read only version of the Reporting Task detail dialog
This closes #3385
2019-04-04 13:40:14 -04:00
dnsbtchr d36fa6a210 NIFI-6016 PutCassandraRecord batch size
Adds resetting the batch size to fix broken batch processing

Removes empty line

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3337
2019-04-04 11:26:08 -04:00
ambah c93ee5ad59 NIFI-6186 Resolve handling of module paths in JythonScriptEngineConfigurator
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3406
2019-04-04 11:13:18 -04:00
Rob Fellows cb4fdc2ae1
NIFI-6126 - throttle requests to refresh the canvas to prevent too many server requests when holding down CTRL-R
* Added lodash utility library to leverage its throttle capability (and many other useful functions in the future).
* Made lodash available in all JSP's so it could be imported into nf-common (or any component for that matter).
* Added a throttle function to nf-common that just wraps _.throttle

This closes #3393
2019-04-04 09:51:41 -04:00
Matthew Burgess 1b196ea213 NIFI-5338: Added SimpleScriptedLookupService controller service
This closes #3397

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2019-04-04 08:41:34 -04:00
Lars Francke 73571ae309 NIFI-6185: ListDatabaseTables processor doesn't close ResultSets
This closes #3405.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2019-04-04 12:24:47 +09:00
Endre Zoltan Kovacs 3a63de2ae2 NIFI-6180: exposing firehose grace period to DruidTranquilityController
this allows for configuring `druidBeam.firehoseGracePeriod`
(https://github.com/druid-io/tranquility/blob/master/docs/configuration.md#properties)

NIFI-6180: Corrected typo in DruidTranquilityController
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3403
2019-04-03 12:31:57 -04:00
Endre Zoltan Kovacs e907b689e8 NIFI-6179: minor changes to DruidTranquilityController
- hooking in property INDEX_RETRY_PERIOD to list of properties
- fixing property description

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3402
2019-04-03 12:24:32 -04:00
Yoshiaki Takahashi 3696b5bfcf
NIFI-6112: Add some useful commands to NiFi Toolkit for automating NiFi cluster construction.
- Supports both UUID and identity for user / user group specification
- Add UUID to toolkit command results
- Fix representation for type and bundle of ControllerServiceResult
- Make description more detailed
- Fix to print only in interactive mode or verbose mode in UpdateAccessPolicy. (with slight refactoring)
- Add list-templates command
- Add start/stop commands for reporting task and enable/disable commands for reporting task controller service.
- Also added "get-repoting-task" command (for single reporting task) to be consistent with controller service.

This closes #3366.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-03-29 11:45:52 -04:00
Mark Payne 76392ee862
NIFI-6155: Ensure that any task submitted to FlowEngine catches Throwable so that the task doesn't die just die silently in the case of an unexpected error/exception
This closes #3395.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-03-29 10:09:35 -04:00
Craig Davidson 4de51fd3d5 NIFI-6148 - Added support for aggregation option 'allowDiskUse'
This closes #3389

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2019-03-28 11:06:25 -04:00
Mark Payne d3d43262db
NIFI-6150: Moved logic of decrementing content claims from ProcessSession.commit to the FlowFile Repository. Also updated load-balancing protocol and queue drop logic to no longer decrement content claims since the flowfile repo will handle the logic. This allows us ts to ensure that the claimant counts are decremented only if the repo is successfully updated and still before checking if the claim is still in use or not.
This closes #3391.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-03-27 10:22:01 -04:00
Koji Kawamura cae01d555c NIFI-6117: Fix BIGINT handling in DataTypeUtils
Follow numeric type conversion convention used for other integral types.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3371
2019-03-26 10:39:35 -04:00
Matthew Burgess c2dc0910b7 NIFI-6117: Fix BIGINT handling in DataTypeUtils 2019-03-26 10:39:29 -04:00
Pierre Villard ead6a3b842 NIFI-6144 - Enable EL on listening port in HandleHttpRequest
This closes #3386.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2019-03-25 11:52:28 +09:00
Peter Turcsanyi c8eff590ef
NIFI-6142: Fixing some typos in user/dev guides
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3384.
2019-03-24 12:29:46 +01:00
Joe Witt 1bdab73201 NIFI-6121 Updated the easy dependencies to update
This closes #3374

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2019-03-22 09:01:21 -04:00
Matthew Burgess ce09b93ef1 NIFI-6139: Add fragment attributes to PartitionRecord
This closes #3382.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2019-03-22 10:41:10 +09:00
Ferenc Szabó 48a6c81fa2 NIFI-4735: ParseEVTX only outputs one event per chunk
This change is based on https://github.com/apache/nifi/pull/2489

I have reproduced the issue with some additional test cases and test files then applied the original fix.

commit message from the original change:
Updated the EVTX FileHeader class to correctly check if there are more chunks in the file. Previously this would not process the last chunk.

Updated the EVTX ChunkHeader class to correctly check if there are additional records in the chunk. Previously this would only process the first record of each chunk. It was using the fileLastRecordNumber where it should have been using the logLastRecordNumber value.

Updated the EVTX unit tests to have the correct expected number of events and use the logLastRecordNumber.

refactoring duplicated code and magic numbers

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2489
This closes #3379
2019-03-20 14:12:03 -04:00
Matt Gilman d35d15cdda
NIFI-6027
- Allowing user or group existence enforcement to be parameterized.
- Fixing error handling when loading user groups which may have resulted in stack trace leaking.

This closes #3377.

Signed-off-by: Kevin Doran <kdoran@apache.org>
2019-03-20 13:38:46 -04:00
Koji Kawamura cd3567873b NIFI-5918 Fix issue with MergeRecord when DefragmentStrategy is on
Added an unit test representing the fixed issue.
And updated existing testDefragment test to illustrate
the remaining FlowFiles those did not meet the threshold.
2019-03-20 12:36:08 +09:00
Andres Garagiola b97fbd2c89 NIFI-5918 Fix issue with MergeRecord when DefragmentStrategy is on
This closes #3334.

Signed-off-by: Andres Garagiola <andresgaragiola@gmail.com>
Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2019-03-20 12:15:40 +09:00
Bryan Bende 9d21a106ad NIFI-6120 Move nifi.registry.version property to root pom so it can be shared between nifi-framework and nifi-toolkit
This closes #3369.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2019-03-20 10:56:10 +09:00
Peter Turcsanyi 77b84edf5b NIFI-6122: PutS3Object supports national characters in filenames.
This closes #3373.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2019-03-20 10:51:39 +09:00
Matthew Burgess 06f41ac6f8 NIFI-6127: Fixed NPE in match() and find() EL functions when attribute doesn't exist
This closes #3378.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2019-03-20 10:30:55 +09:00
Rob Fellows 0cb15cfb1a
NIFI-6118 - Fix: UI - Not all calls to nfCanvasUtil.reloadConnectionSourceAndDestination are made with the proper component IDs
This closes #3368
2019-03-18 14:48:41 -04:00
Rob Fellows 6401e32a33
NIFI-6126 - Change nf-canvas to respond to keyup events rather than keydown
This closes #3376
2019-03-18 14:08:56 -04:00
Peter G. Horvath 43235724e2 NIFI-5318 Implement NiFi test harness: initial commit
NIFI-5318 Implement NiFi test harness: replaced original sample feed payload with synthetic content
NIFI-5318 Implement NiFi test harness: fixed test harness run crash issue; better reporting of paths
NIFI-5318 Implement NiFi test harness: added further states where NiFi version can be queried
NIFI-5318 Implement NiFi test harness: fixed incorrect class reference
NIFI-5318 Implement NiFi test harness: added type parameter bounding to setClassOfSingleProcessor to prevent configuring obviously incorrect classes
NIFI-5318 Updated project version.

This closes #3165

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2019-03-16 10:08:44 -04:00
Sandish Kumar 45ebeba846
NIFI-5267 Kafka support to replay by timestamp
This closes #3372.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-03-15 09:18:39 -04:00
Sandish Kumar 82e2c97782
NIFI-5956 Option BlockCache HBaseScanProcessor
This closes #3295.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-03-14 12:22:41 -04:00
Koji Kawamura 4db5446c87 NIFI-6082: Refactor the way to handle fields nullable
- Make enriched fields nullable at LookupRecord.
- Removed unnecessary AvroConversionOptions and reader schema creation,
because ResultSetRecordSet can generate NiFi Record Schema from RS
directly. No Avro schema is needed to do that.
2019-03-14 09:33:39 +09:00
Matthew Burgess ca76fe178c NIFI-6082: Added DatabaseRecordLookupService, refactored common DB utils
NIFI-6082: Added SimpleDatabaseLookupService

NIFI-6082: Merged Koji's improvements, incorporated review comments

This closes #3341.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2019-03-14 09:33:30 +09:00
Sandish Kumar 0e10e417df
NIFI-4358 This closes #3363. cassandra connection enable compression at resquest and response
Signed-off-by: Joe Witt <joewitt@apache.org>
2019-03-13 15:56:13 -04:00
Koji Kawamura 2846d3c3c6 NIFI-6105: Fix handling of arrays of records/maps in record utilities
Refactored to use the same check logic for Record and Map types

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3367
2019-03-13 11:49:00 -04:00
Matthew Burgess f91311da9d NIFI-6105: Fix handling of arrays of records/maps in record utilities
This closes #3353.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2019-03-13 11:47:55 -04:00
pushpavanthar 35d1cace08 Documentation : ConsumerKafka_2_0 - updated configuration details re… (#3360)
* Documentation :  ConsumerKafka_2_0 - updated configuration details related to SASL_PLAINTEXT - SCRAM

* Updated rest of the related documents with SCRAM configurations and few NOTEs

* incorporated review comments

* incorporated review comments
2019-03-13 11:12:00 -04:00
pushpavanthar 98ca28d50d
NIFI-6083 - Improving documentation for Kafka Service Name property
This closes #3365.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-03-12 12:29:22 -04:00