3564 Commits

Author SHA1 Message Date
Mike Thomsen
4c79ff0576
NIFI-5248 Added new Elasticsearch processors.
NIFI-5248 Fixed a few stray 1.7.0 references.
NIFI-5248 Removed build helper plugin.
NIFI-5248 Made changes requested in a review.
NIFI-5248 Updated dummy service.
NIFI-5248 Made a few changes from a code review.
NIFI-5248 Added logic for removing nulls so record paths can be removed when no longer needed.
NIFI-5248 Switched from variable registry to flowfile level EL.
NIFI-5248 Added JsonPath code to remove index, id and type path statements.
NIFI-5248 Updated validation.
NIFI-5248 Set the field to null instead of empty string when nulling records.
NIFI-5248 Fixed TestElasticSearchClientService.
NIFI-5248 Removed high level client and switched over to low level client for everything.
NIFI-5248 Added profiles for ES 6 and ES 7 integration testing.
NIFI-5248 Updated integration tests to support 5 and 6.
NIFI-5248 Fixed some style check breaks.
NIFI-5248 Added create operation type.
NIFI-5248 Updated documentation.
NIFI-5248 Added error handling to PutElasticsearchRecord.
NIFI-5248 Added error logging to PutElasticsearchJson.
NIFI-5248 Added split failed records option to PutElasticsearchJson.
NIFI-5248 Added documentation for PutElasticsearchRecord.
NIFI-5248 Updated import to not use * import.
NIFI-5248 Removed processor that is no longer relevant due to schema inference.
NIFI-5248 Renamed ElasticSearch instances to Elasticsearch where we can within API guidelines.

NIFI-5248 Added groovy-json test dependency.

NIFI-5248 Updated PutElasticsearchRecord to only do index operations.

NIFI-5248 Added batch size property and refactored the way relationships and properties are added.

NIFI-5248 Added batch processing support.

NIFI-5248 Updated error handling.

NIFI-5248 Updated to 1.11.0-SNAPSHOT.

NIFI-5248 Made changes requested in a code review.

NIFI-5248 Made a few more changes from a code review.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2861
2019-11-19 13:53:34 -05:00
Shayne Burgess
c72a5618c0
NIFI-4012 Azure Event Hub UI typos and cleanup
This closes #3749.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2019-11-18 17:29:24 +09:00
Veli Kerim Celik
1dfbc97c07
NIFI-5929 Support for IBM MQ multi-instance queue managers
proper line break

more proper line break :)

Link to external javadocs and add some code comments

Test that properties is set on ConnectionFactory

cleanup

made two static properties optional and elaborated on 'Additional Details' page

minor corrections to user doc

open external links in new tab

Do broker accommodations only if property is set. Add fallback accommodation.

fix test

Add support for colon pair(s) for IBM MQ. In fallback broker handling use first pair if multiple given.

This closes #3246.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2019-11-18 16:55:23 +09:00
Mark Payne
ea31c1fcbb
NIFI-6868: Ensure that when a property value is obtained by calling ProcessContext.getPropertyValue(...).evaluateAttributeExpressions(...).getValue() that we still retain the appropriate value even if the property references a sensitive parameter
* Added additional unit tests and updated ParameterEvaluator to make sure that it follows the same logic as the ParameterExpression in that sensitive parameters cannot be access from within EL

This closes #3889
Signed-off-by: Rob Fellows <rob.fellows@gmail.com>
2019-11-14 15:37:39 -05:00
Matthew Burgess
55c334feb8 NIFI-6859: Add scripted versions of RecordSinkService, RulesEngineService, and ActionHandler (#3881)
* NIFI-6859: Add scripted versions of RecordSinkService, RulesEngineService, and ActionHandler

* Unit test updates
2019-11-14 14:20:58 -05:00
Koji Kawamura
86cae184ff
NIFI-6599 Fix MergeRecord failure in defragment mode
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3678.
2019-11-14 14:44:01 +01:00
Bryan Bende
5d9bba2f6f
NIFI-6857 Bump hbase-client from 2.1.x to 2.2.x
This closes #3878

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2019-11-12 09:53:30 -05:00
Joe Ferner
c947e57d5e
NIFI-6814:
- Restored JAXB Adapter

This closes #3880
2019-11-08 15:43:13 -05:00
Mark Payne
5f4ce8f431
NIFI-6707: Addition of nifi-system-tests module. Updates to EmbeddedNiFi, NiFi and SystemBundle to properly support classloader isolation that is needed for creating system-level tests and isolating the classpath of the tests themselves from the NiFi classpath. Fixed bootstrap listener to ensure that it always closes socket before calling shutdown(). Failure to do so can result in RunNiFi sometimes timing out while waiting for a response from NiFi
NIFI-6707: Added System Test to verify behavior when services depend on one another

NIFI-6707: Ensure that when tearing down flow after test, we wait for all processors and controller services to complete stop/disable before attempting to delete them

NIFI-6707: Fixed bug in RemoteProcessGroupIT that caused the test to fail intermittently based on timing. Improved logging for Process Group, Standard Process Group

NIFI-6707: Updated to include java11 subdirectory under lib/

NIFI-6707: Rebased against master and changed dependencies from 1.10.0-SNAPSHOT to 1.11.0-SNAPSHOT

NIFI-6707: Updates to see if it will help travis build correctly

NIFI-6707: Commenting out java commands in bootstrap.conf files

This closes #3831.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-11-08 12:33:16 -05:00
Yolanda M. Davis
a37b57e96b
NIFI-6854 - Added option to ignore condition errors in rules. Test correction
NIFI-6854 - change warning to debug

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3876
2019-11-08 11:24:18 -05:00
Yolanda M. Davis
c998a7259a
NIFI-6842 - Introduce MetricsEventReportingTask
NIFI-6842 - Added AlertHandler for bulletin reporting. Update ReportingTask meta data.

NIFI-6842 - corrected display names in action handlers, included metrics option for alert handlers, small refactor in reporting task

NIFI-6842 - updated docs and tags

NIFI-6842 - Added documentation for handlers.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3874
2019-11-08 10:25:54 -05:00
kevinmccarthy
eb366c8d0a
NIFI-6680: This closes #3748. Nifi PutKudu processor - Convert record field names to lowercase
Signed-off-by: Joe Witt <joewitt@apache.org>
2019-11-07 18:24:42 -05:00
Mark Payne
49b7a7cd6b
NIFI-6846: If an Exception is thrown while a Processor is writing to a FlowFile, but that Content Claim is not yet eligible for destruction, mark it as a transient claim on the RepositoryRecord so that if it's available when the FlowFile Repository is checkpointed, then it will be cleaned up then
This closes #3872
2019-11-06 09:53:25 -05:00
Atul Mahajan
e3f72c5ed0
NIFI-6771 : Added fix to support Oracle Binary float and Binary double conversion to Avro type
Fixed CheckStyle errors

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3815
2019-11-06 08:57:31 -05:00
Atul Mahajan
7ac41576e3
NIFI-6775 Added fix to support Oracle Float type conversion to Avro
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3812
2019-11-05 16:52:20 -05:00
Matthew Burgess
442a80bf1b NIFI-6819: Add KafkaRecordSink implementations for Kafka 1 and 2 (#3854)
* NIFI-6819: Add KafkaRecordSink implementations for Kafka 1 and 2

* Updated version to 1.11, corrected doc, added cleanup code
2019-11-05 16:44:37 -05:00
Andy LoPresto
12bb284453
NIFI-6841 Fixed checkstyle error introduced in a9db5a8. 2019-11-05 13:12:40 -08:00
Mark Payne
a9db5a8cb7
NIFI-6841: Fixed bug that resulted in the wrong number of 'Bytes Read' being reported by ByteCountingInputStream in the event that #skip was called between calls to #mark and #reset
This closes #3868
2019-11-05 13:06:06 -05:00
Tamas Palfy
84a05c8595 NIFI-6729 - Created AbstractSingleAttributeBasedControllerServiceLookup and updated DBCPConnectionPoolLookup and AzureStorageCredentialsControllerServiceLookup to inherit from it.
This closes #3774.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2019-11-05 14:02:03 +01:00
Ivan Ezequiel Rodriguez
7f96fa1d0d
NIFI-6395: Thread-safety bug fixed and added new flag property to handle the way to adjust the counters
Update CountText.java

Local variable changes by AtomicInteger

NIFI-6395 - Fix line is longer than 200 characters

This closes #3552.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2019-11-05 20:54:25 +09:00
Nicholas Zhan
af0777b2c9
NIFI-6834 Encoding attachment filename
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3865.
2019-11-05 11:10:15 +01:00
Jan Hentschel
d6b4bce668 NIFI-4758 Changed 'Name' to 'Flow Name' in Import Version dialog 2019-11-04 15:26:25 -05:00
mtien
7f1886adad NIFI-4757 Capitalized 'version' in tool tip. 2019-11-04 15:22:52 -05:00
Joe Witt
f8c3d877cf
NIFI-6733 updating to next release version for master branch 2019-11-04 13:31:39 -05:00
Joe Witt
eb046c4fab
NIFI-6733 updating after merging release branch 2019-11-04 13:28:39 -05:00
Joe Witt
4119279c03
Merge branch 'NIFI-6733-RC3' 2019-11-04 13:24:29 -05:00
Joe Witt
363b751310
Revert "NIFI-6733 bumping master to 1.11.0-SNAPSHOT"
This reverts commit a7c56cfd9a27f2be8809c87ad427dc2ba892da1f.
2019-11-04 13:24:13 -05:00
Joe Witt
a7c56cfd9a
NIFI-6733 bumping master to 1.11.0-SNAPSHOT 2019-11-04 07:54:17 -05:00
Mark Payne
0b3a60ae3e NIFI-6832: When receiving a 0-byte FlowFile via site-to-site, avoid writing to the FlowFile at all 2019-11-01 16:22:17 -04:00
Yolanda M. Davis
5b28f6dad9
NIFI-6803 - Initial commit for ActionHandler Controller Services
NIFI-6803 - updated to description

NIFI-6803 - add Spring reference to Notice

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3856
2019-11-01 10:41:25 -04:00
Kotaro Terada
245e055d5d
NIFI-6828 There is an unintentional string in "conf/logback.xml"
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3863.
2019-10-31 11:20:21 +01:00
mtien
3d56e2f26d
NIFI-4348 handled NPE in isExpressionLanguagePresent.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3857.
2019-10-30 09:58:55 +01:00
Mark Payne
ce5eae5b2c NIFI-6800: Added unit test to verify behavior of contribution 2019-10-29 10:17:27 -04:00
Ivan Ezequiel Rodriguez
67f1677212 NIFI-6800 - Fix hashmap counter overwritten in highThroughputSession
This closes #3837.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2019-10-29 10:16:49 -04:00
Joe Witt
418179f5b2
NIFI-6733-RC3 prepare for next development iteration 2019-10-28 15:13:13 -07:00
Joe Witt
b217ae20ad
NIFI-6733-RC3 prepare release nifi-1.10.0-RC3 2019-10-28 15:12:57 -07:00
David Mollitor
dccbde40f3
NIFI-5452: Allow ignore block locality in HDFS
Fixed formatting. Fallback to autoboxing

This closes #3652.
2019-10-28 14:23:44 -07:00
Peter Turcsanyi
5fd8df5780
NIFI-6818: This closes #3852. Set service api dependencies to provided in AWS and Azure bundles
The controller service api jars have their own bundles, they do not need to be
packaged into the processor nars.

Signed-off-by: Joe Witt <joewitt@apache.org>
2019-10-28 12:56:06 -07:00
Joe Witt
c6e8bf204c
NIFI-6794 resolving pom issue creating resolving merge conflict 2019-10-28 12:55:26 -07:00
Mike Thomsen
d5fbfd1a12
NIFI-6794 This closes #3828. Moved to the latest Elasticsearch 5.X client and upgraded Apache HttpClient to the latest 4.5.X release to try to get around a SSL bug reported on the mailing list. 2019-10-28 12:42:37 -07:00
Joe Witt
dda29cd90b
NIFI-6733 updating zk to latest incremental 2019-10-28 12:40:03 -07:00
mysunnytime
5c69faf9bb NIFI-6597: Upgrade Azure Event Hub version and code.
- Lazy instantiation for PutAzureEventHub. Also add explaination for thread pool needed for EventHubClient.
2019-10-28 15:33:32 -04:00
Jan Hentschel
ccf85777c4
NIFI-6816 Removed duplicated dependency declarations
This closes #3851

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2019-10-28 12:49:08 -04:00
Otto Fowler
cfcf2e698d
NIFI-6682 SplitJson does not work if the input is null
This closes #3772

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2019-10-25 17:43:13 -04:00
Matt Gilman
4f5718bf50
NIFI-6812: This closes #3847
- Establishing the L&N for the azure-services-api-nar.

Signed-off-by: Joe Witt <joewitt@apache.org>
2019-10-25 13:30:54 -04:00
Matt Gilman
9364f7d2e8
NIFI-6811:
- Allowing enterGroup to complete prior to loading the controller service table.

This closes #3846.
2019-10-25 12:07:03 -04:00
Mark Payne
3f270f184c
NIFI-6807: When a Controller Service's state is transitioned to ENABLING, complete the Future successfully, even if the Controller Service is not valid. The Controller Service will remain in the ENABLING state until it is made valid, at which point it will ENABLE (unless explicitly disabled first). This allows us to Enable a Controller Service and its referencing components, even if the referencing component is still invalid due to it not yet recognizing the the referenced service has been enabled.
This closes #3841
2019-10-25 11:38:34 -04:00
Mark Payne
81fc2768c9
NIFI-6798: When checking that referenced Controller Services are valid, use ValidationContext.getProperties() rather than ComponentNode.getSupportedPropertyDescriptors() because the latter does not include dynamic properties
This closes #3836.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-10-25 09:52:44 -04:00
Matt Gilman
2cef010b06
NIFI-6687:
- Using correct NiFi Ranger Service name in unit tests.

This closes #3842.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-10-25 09:31:34 -04:00
Koji Kawamura
87838e6c31
NIFI-6793 Fix NumberFormatException at CaptureChangeMySQL
This closes #3827

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2019-10-25 05:48:50 -04:00