Apache
/
nifi
mirror of https://github.com/apache/nifi.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
5,139 Commits 147 Branches 203 Tags 406 MiB
Commit Graph

3282 Commits

Author SHA1 Message Date
Bryan Rosander d429470cc1 NIFI-6226: Returning -1 for EmptyRecordReader.getMaxEventId() to prevent potential provenance repo corruption 
This closes #3443.

Signed-off-by: Mark Payne <markap14@hotmail.com>
 2019-04-23 13:14:47 -04:00
Mark Payne 5e2559db42
NIFI-6236: Update VolatileFlowFileRepository to decrement claimant counts when FlowFiles are deleted 
This closes #3451.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2019-04-23 13:10:14 -04:00
Mark Payne 055b3cac54 NIFI-6220: If FlowFile is created by cloning a relationship, do not create an ATTRIBUTES_MODIFIED provenance event for it. 
NIFI-6220: Updated test name and fixed typo

This closes #3438.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
 2019-04-23 17:08:26 +09:00
Koji Kawamura c5d0643d1d NIFI-5792: Remember created versioned flow information 
- Before this fix, NiFi loses information about created versioned flow in case of subsequent snapshot creation failure, and NiFi API returned an error response
- This commit makes:
  - The created versioned Flow information is stored even if subsequent snapshot creation fails
  - NiFi API to return a successful 200 response in that case, but return versioned flow status as SYNC_FAILURE with an explanation. NiFi UI shows a popup error dialog with the explanation.
  - Versioned flow status will be LOCALLY_MODIFIED if the latest version is 0.

This closes #3134.

Signed-off-by: Mark Payne <markap14@hotmail.com>
 2019-04-22 13:27:40 -04:00
Sivaprasanna 94c2b1e76e NIFI-6049: Updated MoveHDFS to support FF Attribute for Output Directory (#3433) 
NIFI-6049: Updated MoveHDFS to support FF Attribute for Output Directory
 2019-04-22 10:12:16 -06:00
Dustin Rodrigues 161e4b5763 NIFI-6231 - fix source code permissions to be non-executable 
This closes #3449.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
 2019-04-22 10:13:07 +09:00
aeaversa 512555dbb0 NIFI-6221 Adding 'Download' and 'View' content buttons to List Queue (#3440) 
NIFI-6221 Adding 'Download' and 'View' content icon buttons to slick grid actions column to improve end user navigation.
 2019-04-19 16:25:15 -06:00
Rahul Patil 8411b6a46f
NIFI-6207: fixing 'partition' typos in AWS Kinesis processor and test as well as Kafka ConsumeLease classes. 
This closes #3437.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
 2019-04-16 11:00:48 -07:00
Matthew Burgess d0224f7897 NIFI-6093: Add count and id fragment attributes to original FlowFile in SplitRecord 
This closes #3435.

Signed-off-by: Mark Payne <markap14@hotmail.com>
 2019-04-16 09:43:10 -04:00
Andy LoPresto 22563fcb9a
NIFI-6201 Suppressed stacktraces for expected error conditions (connection timeout, missing component bundle, etc.) in application startup when the stacktrace provides no additional helpful data. 
NIFI-6201 Fixed existing checkstyle complaint on missing Javadoc element.

NIFI-6201 Restored "connection refused" text to relevant registry sync error message.

NIFI-6201 Simplified error logging for bundle not found issues and removed unused bundle coordinate methods.

This closes #3427.

Signed-off-by: Kevin Doran <kdoran@apache.org>
 2019-04-15 16:52:13 -04:00
Mark Payne 7b945182af NIFI-6215: Ensure that when we are swapping FlowFiles back into nifi, that we consider forward slashes or back slashes as the same path separator when determining filename 
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3434
 2019-04-15 15:54:41 -04:00
Brandon Devries cbc07eb663 NIFI-6157: Allowing configuration of concurrent tasks and transferred FlowFile count to Funnels and LocalPorts 
This closes #3412
Signed-off-by: Brandon <devriesb@apache.org>
 2019-04-15 14:13:11 +00:00
Rob Fellows b5ff622118
NIFI-4230 - Use a better default location when pasting via keystokes and the original copied item is not fully in view. 
- handle multi-select copy/paste better.
- remove dimension info from getOrigin function
- handle pasting better when pasted items should be centered and not offset from the original.
- fixed strict mode and removed console.logs
- Fixing pasting logic when trying to center the items on the canvas.

This closes #3383
 2019-04-12 15:27:26 -04:00
Matthew Burgess 97690275ac
NIFI-6210: Applied NIFI-5134 Kerberos TGT renewal to Hive3ConnectionPool 
This closes #3432.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2019-04-12 15:09:52 -04:00
Mark Payne 91b2283f49
NIFI-6183: Implemented base class for Framework-level integration tests that is responsible for instantiating the key components for integration tests, including FlowController, WriteAheadFlowFileRepository, WriteAheadProvenanceRepository, FileSystemRepository, FileSystemSwapManager, StandardFlowFileQueue, StandardProcessScheduler, etc. Implemented several integration tests to span different parts of the framework 
This closes #3423.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2019-04-10 16:47:08 -04:00
Matthew Burgess 8d92a29fb9
NIFI-5238: Replaced Shift-F with Shift-Ctrl-F in Jolt Advanced UI for auto-format 
This closes #3428
 2019-04-10 14:28:15 -04:00
Mark Payne 0f10010683
NIFI-6199: Fixed problematic unit tests that did not properly wait for async tasks to complete before making assertions 
This closes #3421.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2019-04-09 11:24:58 -04:00
Mark Payne bf7e70e4c3
NIFI-6194: Fixed verification logic to determine whether or not a variable's value can be changed. Previously, the logic had a bug that resulted in a failure if any child group contained a running processor that references an overridden variable and the user attempts to change the overridden variable at the higher level. We should not include any components of descendent groups if the descendent group overrides the variable. 
This closes #3420.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2019-04-09 11:15:45 -04:00
Mark Payne 38db4e97cd
NIFI-5464: Consider ports invalid unless they have both incoming and outgoing connections 
This closes #3417.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2019-04-09 10:19:24 -04:00
Lars Francke 49e74b29db NIFI-6198 Infinite recursion in HBase*ClientService 
This closes #3419.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
 2019-04-09 09:29:32 +09:00
Kourge cded30b3d2 NIFI-5953 Manage GetTwitter connection retries on '420 Enhance Your Calm' exceptions 
NIFI-5953 Manage GetTwitter connection retries on '420 Enhance Your Calm' exceptions

Update "Max Client Error Retries" parameter name.
reintriduce client.reconnect() on HTTP_ERROR 420

This closes #3276.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
 2019-04-09 09:14:44 +09:00
Rob Fellows fb9f2af04a
NIFI-6136 - fix: UI - User and Template bug when page popped out 
NIFI-6136 - return the appropriate value (false) when verifyDisconnectedCluster is called from a non-clustered environment.

This closes #3390
 2019-04-08 15:45:19 -04:00
Matthew Burgess 8572852bdf
NIFI-6184: This closes #3413. Removed unnecessary test-jar dep from nifi-livy-processors causing race condition, added assertion to catch failures 
Signed-off-by: Joe Witt <joewitt@apache.org>
 2019-04-08 14:52:40 -04:00
Koji Kawamura 1c588f10b2 NIFI-5979 Add Line-by-Line Evaluation Mode to ReplaceText 
Refactored to use functions to better handle strategy specific variables
via closure.
 2019-04-08 16:19:48 +09:00
pushpavanthar d222f14a9e NIFI-5979 : enhanced ReplaceText processor with "Number of Occurrences" and "Occurrence offset" configurations 
Fixed indentation errors to pass checkstyle-checks

Added Evaluation Modes as per discussion in PR thread

Adding exclusions of test files on rat plugin

Added new property 'Line-by-Line Evaluation Mode' and refactored common code

This closes #3375.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
 2019-04-08 16:19:27 +09:00
Bryan Bende 25cb29e109
NIFI-6177 Refactor HBaseListLookupService tests to remove use of TestRecordLookupProcessor 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3400.
 2019-04-06 14:31:50 +02:00
bdesert 75217b33d0
NIFI-6181 FetchSFTP and FetchFTP File Not Found fix 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3407.
 2019-04-06 14:27:39 +02:00
Alex Aversa 96279415a7 NIFI-6188 - Added 1s delay on Search control autocomplete query event to prevent premature results retrieval 
This closes #3408.
Signed-off-by: Brandon Devries <devriesb@apache.org>
 2019-04-05 12:32:40 -04:00
Mike Thomsen 4810f6d32e NIFI-6172 Fixed a bug that caused ElasticSearchStringLookupService to always return a value even when it should return Optional.empty(). 
NIFI-6172 Fixed broken integration test.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3399
 2019-04-05 11:12:30 -04:00
Lars Francke d372318d47 NIFI-6191 HBase 2 ClientService documentation refers to HBase 1.x 
This closes #3409.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
 2019-04-05 14:55:58 +09:00
Rob Fellows e374d2cc30
NIFI-6140 - Incorrect title on the read only version of the Reporting Task detail dialog 
This closes #3385
 2019-04-04 13:40:14 -04:00
dnsbtchr d36fa6a210 NIFI-6016 PutCassandraRecord batch size 
Adds resetting the batch size to fix broken batch processing

Removes empty line

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3337
 2019-04-04 11:26:08 -04:00
ambah c93ee5ad59 NIFI-6186 Resolve handling of module paths in JythonScriptEngineConfigurator 
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3406
 2019-04-04 11:13:18 -04:00
Rob Fellows cb4fdc2ae1
NIFI-6126 - throttle requests to refresh the canvas to prevent too many server requests when holding down CTRL-R 
* Added lodash utility library to leverage its throttle capability (and many other useful functions in the future).
* Made lodash available in all JSP's so it could be imported into nf-common (or any component for that matter).
* Added a throttle function to nf-common that just wraps _.throttle

This closes #3393
 2019-04-04 09:51:41 -04:00
Matthew Burgess 1b196ea213 NIFI-5338: Added SimpleScriptedLookupService controller service 
This closes #3397

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
 2019-04-04 08:41:34 -04:00
Lars Francke 73571ae309 NIFI-6185: ListDatabaseTables processor doesn't close ResultSets 
This closes #3405.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
 2019-04-04 12:24:47 +09:00
Endre Zoltan Kovacs 3a63de2ae2 NIFI-6180: exposing firehose grace period to DruidTranquilityController 
this allows for configuring `druidBeam.firehoseGracePeriod`
(https://github.com/druid-io/tranquility/blob/master/docs/configuration.md#properties)

NIFI-6180: Corrected typo in DruidTranquilityController
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3403
 2019-04-03 12:31:57 -04:00
Endre Zoltan Kovacs e907b689e8 NIFI-6179: minor changes to DruidTranquilityController 
- hooking in property INDEX_RETRY_PERIOD to list of properties
- fixing property description

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3402
 2019-04-03 12:24:32 -04:00
Mark Payne 76392ee862
NIFI-6155: Ensure that any task submitted to FlowEngine catches Throwable so that the task doesn't die just die silently in the case of an unexpected error/exception 
This closes #3395.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2019-03-29 10:09:35 -04:00
Craig Davidson 4de51fd3d5 NIFI-6148 - Added support for aggregation option 'allowDiskUse' 
This closes #3389

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
 2019-03-28 11:06:25 -04:00
Mark Payne d3d43262db
NIFI-6150: Moved logic of decrementing content claims from ProcessSession.commit to the FlowFile Repository. Also updated load-balancing protocol and queue drop logic to no longer decrement content claims since the flowfile repo will handle the logic. This allows us ts to ensure that the claimant counts are decremented only if the repo is successfully updated and still before checking if the claim is still in use or not. 
This closes #3391.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2019-03-27 10:22:01 -04:00
Pierre Villard ead6a3b842 NIFI-6144 - Enable EL on listening port in HandleHttpRequest 
This closes #3386.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
 2019-03-25 11:52:28 +09:00
Joe Witt 1bdab73201 NIFI-6121 Updated the easy dependencies to update 
This closes #3374

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
 2019-03-22 09:01:21 -04:00
Matthew Burgess ce09b93ef1 NIFI-6139: Add fragment attributes to PartitionRecord 
This closes #3382.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
 2019-03-22 10:41:10 +09:00
Ferenc Szabó 48a6c81fa2 NIFI-4735: ParseEVTX only outputs one event per chunk 
This change is based on https://github.com/apache/nifi/pull/2489

I have reproduced the issue with some additional test cases and test files then applied the original fix.

commit message from the original change:
Updated the EVTX FileHeader class to correctly check if there are more chunks in the file. Previously this would not process the last chunk.

Updated the EVTX ChunkHeader class to correctly check if there are additional records in the chunk. Previously this would only process the first record of each chunk. It was using the fileLastRecordNumber where it should have been using the logLastRecordNumber value.

Updated the EVTX unit tests to have the correct expected number of events and use the logLastRecordNumber.

refactoring duplicated code and magic numbers

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2489
This closes #3379
 2019-03-20 14:12:03 -04:00
Matt Gilman d35d15cdda
NIFI-6027 
- Allowing user or group existence enforcement to be parameterized.
- Fixing error handling when loading user groups which may have resulted in stack trace leaking.

This closes #3377.

Signed-off-by: Kevin Doran <kdoran@apache.org>
 2019-03-20 13:38:46 -04:00
Koji Kawamura cd3567873b NIFI-5918 Fix issue with MergeRecord when DefragmentStrategy is on 
Added an unit test representing the fixed issue.
And updated existing testDefragment test to illustrate
the remaining FlowFiles those did not meet the threshold.
 2019-03-20 12:36:08 +09:00
Andres Garagiola b97fbd2c89 NIFI-5918 Fix issue with MergeRecord when DefragmentStrategy is on 
This closes #3334.

Signed-off-by: Andres Garagiola <andresgaragiola@gmail.com>
Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
 2019-03-20 12:15:40 +09:00
Bryan Bende 9d21a106ad NIFI-6120 Move nifi.registry.version property to root pom so it can be shared between nifi-framework and nifi-toolkit 
This closes #3369.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
 2019-03-20 10:56:10 +09:00
Peter Turcsanyi 77b84edf5b NIFI-6122: PutS3Object supports national characters in filenames. 
This closes #3373.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
 2019-03-20 10:51:39 +09:00
Rob Fellows 0cb15cfb1a
NIFI-6118 - Fix: UI - Not all calls to nfCanvasUtil.reloadConnectionSourceAndDestination are made with the proper component IDs 
This closes #3368
 2019-03-18 14:48:41 -04:00
Rob Fellows 6401e32a33
NIFI-6126 - Change nf-canvas to respond to keyup events rather than keydown 
This closes #3376
 2019-03-18 14:08:56 -04:00
Sandish Kumar 45ebeba846
NIFI-5267 Kafka support to replay by timestamp 
This closes #3372.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2019-03-15 09:18:39 -04:00
Sandish Kumar 82e2c97782
NIFI-5956 Option BlockCache HBaseScanProcessor 
This closes #3295.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2019-03-14 12:22:41 -04:00
Koji Kawamura 4db5446c87 NIFI-6082: Refactor the way to handle fields nullable 
- Make enriched fields nullable at LookupRecord.
- Removed unnecessary AvroConversionOptions and reader schema creation,
because ResultSetRecordSet can generate NiFi Record Schema from RS
directly. No Avro schema is needed to do that.
 2019-03-14 09:33:39 +09:00
Matthew Burgess ca76fe178c NIFI-6082: Added DatabaseRecordLookupService, refactored common DB utils 
NIFI-6082: Added SimpleDatabaseLookupService

NIFI-6082: Merged Koji's improvements, incorporated review comments

This closes #3341.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
 2019-03-14 09:33:30 +09:00
Sandish Kumar 0e10e417df
NIFI-4358 This closes #3363. cassandra connection enable compression at resquest and response 
Signed-off-by: Joe Witt <joewitt@apache.org>
 2019-03-13 15:56:13 -04:00
pushpavanthar 35d1cace08 Documentation : ConsumerKafka_2_0 - updated configuration details re… (#3360) 
* Documentation :  ConsumerKafka_2_0 - updated configuration details related to SASL_PLAINTEXT - SCRAM

* Updated rest of the related documents with SCRAM configurations and few NOTEs

* incorporated review comments

* incorporated review comments
 2019-03-13 11:12:00 -04:00
pushpavanthar 98ca28d50d
NIFI-6083 - Improving documentation for Kafka Service Name property 
This closes #3365.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2019-03-12 12:29:22 -04:00
Ryan Bower 160ade9f9d
NIFI-5018 
- Added snap alignment for nf-label, label resize events, and nf-connection
- Shift key now disables snap alignment during the drag event.
- nf-connection load-balance-icon updated

This closes #3335
 2019-03-11 13:52:09 -04:00
Mark Payne c84f40ee36
NIFI-6111: Fixed bugs in the Status History values. If metrics are not available yet from all nodes for the last point on the graph, leave the cluster aggregate value off for the last point to prevent it from dropping significantly at the end of the chart. Fixed bug where counter values were not properly summed together in cluster view. Addressed issue with Average Task Duration 
This closes #3361.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2019-03-08 17:27:49 -05:00
Mark Payne 58a25cfa5a
NIFI-6110: Updated StandardProcessSession such that if we fail to update the FlowFile Repository, we do not decrement claimant counts for any FlowFiles that were removed. Doing so can cause an issue where a FlowFile is removed, then the FlowFileRepo update fails, resulting in the flowfile being rolled back, but after its claimant count is decremented. It will then be processed again, which can result in the same thing, and we'll end up decrementing the claimant count repeatedly. Also updated LengthDelimitedJournal so that if the overflow directory already exists, it does not fail when trying to create the directory and instead just moves on. Updated unit tests to test both of these new fixes and updated DummyRecordSerde to be thread-safe because the TestLengthDelimitedJournal now needs it to be thread safe due to the new unit test that was added. 
This closes #3358.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2019-03-08 10:32:38 -05:00
Peter Turcsanyi 79e05c9f58 NIFI-6078: Create PostSlack processor 
Processor for sending messages on Slack and optionally upload and attach
the FlowFile content (e.g. an image) to the message.

NIFI-6078: Remove username/icon properties.

NIFI-6078: Make Text property optional.

NIFI-6078: Documentation changes.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3339
 2019-03-07 13:00:31 -05:00
Ferenc Szabó d53cefa2a7 NIFI-3685 Azure EventHub processor tests fail 
Moved test classes into the proper folder.
Fixed outdated mocking and added assertions that actually tests the content.

This closes #3346.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
 2019-03-07 14:26:40 +09:00
Kevin Doran 1d06044e3b
NIFI-6108: Fix port names when imported from registry 
This closes #3356.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2019-03-06 17:07:18 -05:00
Matthew Burgess c32ea618c5
NIFI-6040: Fixed ExecuteSQL processors when Output Batch Size is set 
This closes #3355.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2019-03-06 13:44:33 -05:00
thenatog 36bbc77723 NIFI-6097 - Upgraded the fasterxml jackson version to 2.9.8. Ensure that the version is consistent across modules using a maven property defined in the root pom. 
This closes #3347

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
 2019-03-04 19:30:35 -05:00
Matthew Burgess 32bd7ed8b4 NIFI-6062: Add support for BLOB, CLOB, NCLOB in record handling 
This closes #3329

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
 2019-02-28 08:59:37 -05:00
Matthew Burgess e5fa18d63c NIFI-6088: Widen type inference for BIGINT and DOUBLE 
This closes #3342

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
 2019-02-28 05:51:33 -05:00
Arpad Boda 2a6c3c1686
NIFI-5977 - Add "Minimum/Maximum File Age" Parameter to ListSFTP 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3324.
 2019-02-28 09:15:59 +01:00
Otto Fowler 922da68a23
NIFI-6076 syslog5424 should support missing MSG 
- newer lib version and better test

This closes #3331.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2019-02-26 09:39:09 -05:00
Joe Percivall cd7edb1c04 NIFI-5172 Adding the ability to specify a record writer for PutElasticsearchHttpRecord in order to individually handle failed records 
Addressing PR feedback

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3299
 2019-02-25 17:15:39 -05:00
Bryan Bende be6cd4f0f2
NIFI-6069 Fixing checkstyle error 2019-02-25 16:52:41 -05:00
Mike Thomsen b0a93b473b
NIFI-5980 Added HBase_1_1_2_ListLookupService. 
NIFI-5980 Added HBase_2_ListLookupService.

This closes #3278.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2019-02-25 16:42:11 -05:00
Bryan Bende 4b45e85327 NIFI-6055 Improving error handling during auto-loading of NARs and ensuring only one Jetty NAR is loaded 
NIFI-6056 Fixing cassandra bundle so nifi-mock is a test scoped and removing incorrect usage of ProviderCreation exception from nifi-framework-api

This closes #3322.

Signed-off-by: Mark Payne <markap14@hotmail.com>
 2019-02-22 12:19:12 -05:00
Bryan Bende 26c00f1080 NIFI-6069 Fixing fingerprinting of connection relationships to be properly sorted 
This closes #3325.

Signed-off-by: Mark Payne <markap14@hotmail.com>
 2019-02-22 12:13:13 -05:00
Mark Payne 414bfc94fc NIFI-6068: Fixed unit test 2019-02-22 11:55:25 -05:00
Mark Payne e26bacf697 NIFI-6068, NIFI-6065: Updated StandardFunnel to avoid looping indefinitely and instead transfer no more than 10,000 FlowFiles before returning from onTrigger. Updated Local Ports to behavior in the same way. Updated Root Group Ports so that instead of blocking for up to 100 milliseconds for an incoming request, it blocks for up to 1 millisecond and if nothing is available yields for the 'bored yield duration' 
Signed-off-by: Brandon Devries <devriesb@apache.org>

This closes #3328.
 2019-02-22 10:12:20 -05:00
Pierre Villard 2336044b90 NIFI-6059 - Fixed NPE in GCS processors when project ID is not specified 
This closes #3327.

Signed-off-by: Mark Payne <markap14@hotmail.com>
 2019-02-22 09:45:13 -05:00
Mark Bean a0c28ee19e NIFI-5660: JMSPublisher should set some header properties in JmsTemplate instead of directly in the message 
NIFI-5660: Added lines to integration test to verify these header properties (added by Mike Moser)

Signed-off-by: Mike Moser <mosermw@apache.org>

This closes #3053
 2019-02-21 14:45:04 -05:00
Jan Hentschel 60b00a1e38 NIFI-2738 Removed references to nifi.cluster.request.replication.claim.timeout 
This closes #2860

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
 2019-02-21 06:34:27 -05:00
joewitt 25cc7b4a1e
NIFI-6029 merging nifi 1.9.0 release into master 2019-02-19 22:55:49 -05:00
Denes Arvay 76e92c8682 NIFI-6052 Update NOTICE files to reflect 2019 
This closes #3319

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
 2019-02-19 18:32:15 -05:00
Wietze d59a07245f NIFI-5500 Adding support for array fields when outputting to attributes when using QueryElasticsearchHttp 
NIFI-5500 Optimised array-to-string conversion

This closes #2942

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
 2019-02-19 12:41:57 -05:00
Corey Fritz af6039a5b0 NIFI-6045: Added support EL support to dyanmic properties for Kafka 2.0 processors 
This closes #3316

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
 2019-02-19 10:34:43 -05:00
Mike Thomsen 53ca9eb7e5 NIFI-5987 Fixed issue where an invalid query pulled from an attribute would cause GetMongo to not route to failure. 
Signed-off-by: zenfenan <zenfenan@apache.org>

This closes #3285
 2019-02-19 16:24:15 +05:30
joewitt 0e204f3576
NIFI-6029-RC2 prepare for next development iteration 2019-02-16 21:50:35 -05:00
joewitt 45bb53d2aa
NIFI-6029-RC2 prepare release nifi-1.9.0-RC2 2019-02-16 21:50:15 -05:00
Mark Payne 82f44155f6
NIFI-6044: This closes #3314. Retain the input data's order in the CSV Reader's inferred schema 
Signed-off-by: joewitt <joewitt@apache.org>
 2019-02-16 20:57:42 -05:00
Aleksandr Salatich 9e0687ab6b
NIFI-6018 Marking GetHTTP and PostHTTP deprecated. Adding info about deprecation to processors descriptions. Fix small mistake in developer-guide. 
Suppressed deprecation warnings in TestGetHTTP, TestPostHTTP, TestGetHTTPGroovy, and TestPostHTTPGroovy. The annotation suppresses some warnings, but until Java 9, explicit FQCN imports are needed to avoid all warnings.

This closes #3312.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
 2019-02-15 14:09:26 -08:00
Jeff Storck b7ca59f568
NIFI-6023: Deprecated the "Distributed Cache Service" property in ListHDFS 
Updated exception message thrown when accessing the ListHDFS state to refer to the State Manager rather than the "Distributed Cache Service"
Removed usage and references to MockCacheClient from TestListHDFS

NIFI-6023: Minor update of the "Distributed Cache Service" description

This closes #3313.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2019-02-15 14:12:04 -05:00
Mark Payne b508d6bfbc
NIFI-6033, NIFI-6034, NIFI-6035, NIFI-6036, NIFI-6037: Fixed bugs that were found during 1.9.0-RC1 validation. If multiple FlowFiles were written to same Content Claim, and a Processor attempted to read two of them wi within a single session, it would seek to the wrong part of the content or else throw a ContentNotFoundException. Updated logic for considering a processor to be 'running' / having 'active threads' if the processor is invalid upon NiFi restart but scheduled to run. Fixed NPE in FreeFormTextWriter. If MergeRecord reaches minimum number of records, flush writer after writing content out so that its minimum size can accurately be checked. 
This closes #3309.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2019-02-15 11:59:47 -05:00
Mike Moser f61947627e
NIFI-6038 Remove scope parameter from OIDC TokenRequest 
This closes #3308
 2019-02-14 16:43:15 -05:00
Aleksandr Salatich e598b30d6d NIFI-6000 Catch also IllegalArgumentException in ConvertAvroToORC hive processor. Added support for Avro null types. 
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3302
 2019-02-14 15:08:54 -05:00
Endre Zoltan Kovacs ae8a794ff0
NIFI-6031: allow OS level socket keep alive checking 
This closes #3307.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2019-02-14 13:29:00 -05:00
Patrice Freydiere 8a197e5a96 NIFI-4367 Fix on processor for permit deriving script classes from AbstractProcessor or other Records based base classes 
Improve UT

Add licence and fix import

improve filtering properties

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2201
 2019-02-14 12:19:13 -05:00
Alex Savitsky e7ae97797e NIFI-5943 support conversions from List to Avro ARRAY and from Map to Avro RECORD 
NIFI-5943 Added another unit test to verify list + map conversion to list of records. (Mike Thomsen)

This closes #3267

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
 2019-02-14 08:31:42 -05:00
Alex Savitsky 45b32e3bc1 NIFI-5947 Elasticsearch lookup service compatible with LookupAttribute 
NIFI-5947 Made the lookup key for ElasticSearchStringLookupService more unique to the service (was previously 'id'); change by Mike Thomsen

This closes #3268.

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
 2019-02-14 08:02:29 -05:00
Mike Thomsen 033b2a1940 NIFI-4975 Add GridFS processors 
NIFI-4975 Added changes requested in a code review.
NIFI-4975 Reverted some base Mongo changes.
NIFI-4975 Moved connection configuration to using Mongo client service.
NIFI-4975 Fixed a lot of style issues.
NIFI-4975 Removed an EL statement that was causing problems with the UI.
NIFI-4975 Added changes from code review.
NIFI-4975 Added additional details for FetchGridFS.
NIFI-4975 Added documentation for DeleteGridFS.
NIFI-4975 Added documentation for PutGridFS.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2546
 2019-02-13 16:51:17 -05:00
Jeff Storck 72244d09ff NIFI-5575: Added Hadoop configuration to MockFileSystem for umask tests in PutHDFSTest 
Minor update to Permissions umask property description
Minor code cleanup/formatting

This closes #3252
 2019-02-13 16:09:12 -05:00