4db5446c87
NIFI-6082: Refactor the way to handle fields nullable
...
- Make enriched fields nullable at LookupRecord.
- Removed unnecessary AvroConversionOptions and reader schema creation,
because ResultSetRecordSet can generate NiFi Record Schema from RS
directly. No Avro schema is needed to do that.
2019-03-14 09:33:39 +09:00
ca76fe178c
NIFI-6082: Added DatabaseRecordLookupService, refactored common DB utils
...
NIFI-6082: Added SimpleDatabaseLookupService
NIFI-6082: Merged Koji's improvements, incorporated review comments
This closes #3341 .
Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2019-03-14 09:33:30 +09:00
0e10e417df
NIFI-4358 This closes #3363 . cassandra connection enable compression at resquest and response
...
Signed-off-by: Joe Witt <joewitt@apache.org>
2019-03-13 15:56:13 -04:00
35d1cace08
Documentation : ConsumerKafka_2_0 - updated configuration details re… ( #3360 )
...
* Documentation : ConsumerKafka_2_0 - updated configuration details related to SASL_PLAINTEXT - SCRAM
* Updated rest of the related documents with SCRAM configurations and few NOTEs
* incorporated review comments
* incorporated review comments
2019-03-13 11:12:00 -04:00
98ca28d50d
NIFI-6083 - Improving documentation for Kafka Service Name property
...
This closes #3365 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2019-03-12 12:29:22 -04:00
160ade9f9d
NIFI-5018
...
- Added snap alignment for nf-label, label resize events, and nf-connection
- Shift key now disables snap alignment during the drag event.
- nf-connection load-balance-icon updated
This closes #3335
2019-03-11 13:52:09 -04:00
c84f40ee36
NIFI-6111: Fixed bugs in the Status History values. If metrics are not available yet from all nodes for the last point on the graph, leave the cluster aggregate value off for the last point to prevent it from dropping significantly at the end of the chart. Fixed bug where counter values were not properly summed together in cluster view. Addressed issue with Average Task Duration
...
This closes #3361 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2019-03-08 17:27:49 -05:00
58a25cfa5a
NIFI-6110: Updated StandardProcessSession such that if we fail to update the FlowFile Repository, we do not decrement claimant counts for any FlowFiles that were removed. Doing so can cause an issue where a FlowFile is removed, then the FlowFileRepo update fails, resulting in the flowfile being rolled back, but after its claimant count is decremented. It will then be processed again, which can result in the same thing, and we'll end up decrementing the claimant count repeatedly. Also updated LengthDelimitedJournal so that if the overflow directory already exists, it does not fail when trying to create the directory and instead just moves on. Updated unit tests to test both of these new fixes and updated DummyRecordSerde to be thread-safe because the TestLengthDelimitedJournal now needs it to be thread safe due to the new unit test that was added.
...
This closes #3358 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2019-03-08 10:32:38 -05:00
79e05c9f58
NIFI-6078: Create PostSlack processor
...
Processor for sending messages on Slack and optionally upload and attach
the FlowFile content (e.g. an image) to the message.
NIFI-6078: Remove username/icon properties.
NIFI-6078: Make Text property optional.
NIFI-6078: Documentation changes.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #3339
2019-03-07 13:00:31 -05:00
d53cefa2a7
NIFI-3685 Azure EventHub processor tests fail
...
Moved test classes into the proper folder.
Fixed outdated mocking and added assertions that actually tests the content.
This closes #3346 .
Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2019-03-07 14:26:40 +09:00
1d06044e3b
NIFI-6108: Fix port names when imported from registry
...
This closes #3356 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2019-03-06 17:07:18 -05:00
c32ea618c5
NIFI-6040: Fixed ExecuteSQL processors when Output Batch Size is set
...
This closes #3355 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2019-03-06 13:44:33 -05:00
36bbc77723
NIFI-6097 - Upgraded the fasterxml jackson version to 2.9.8. Ensure that the version is consistent across modules using a maven property defined in the root pom.
...
This closes #3347
Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2019-03-04 19:30:35 -05:00
32bd7ed8b4
NIFI-6062: Add support for BLOB, CLOB, NCLOB in record handling
...
This closes #3329
Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2019-02-28 08:59:37 -05:00
e5fa18d63c
NIFI-6088: Widen type inference for BIGINT and DOUBLE
...
This closes #3342
Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2019-02-28 05:51:33 -05:00
2a6c3c1686
NIFI-5977 - Add "Minimum/Maximum File Age" Parameter to ListSFTP
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #3324 .
2019-02-28 09:15:59 +01:00
922da68a23
NIFI-6076 syslog5424 should support missing MSG
...
- newer lib version and better test
This closes #3331 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2019-02-26 09:39:09 -05:00
cd7edb1c04
NIFI-5172 Adding the ability to specify a record writer for PutElasticsearchHttpRecord in order to individually handle failed records
...
Addressing PR feedback
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #3299
2019-02-25 17:15:39 -05:00
be6cd4f0f2
NIFI-6069 Fixing checkstyle error
2019-02-25 16:52:41 -05:00
b0a93b473b
NIFI-5980 Added HBase_1_1_2_ListLookupService.
...
NIFI-5980 Added HBase_2_ListLookupService.
This closes #3278 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2019-02-25 16:42:11 -05:00
4b45e85327
NIFI-6055 Improving error handling during auto-loading of NARs and ensuring only one Jetty NAR is loaded
...
NIFI-6056 Fixing cassandra bundle so nifi-mock is a test scoped and removing incorrect usage of ProviderCreation exception from nifi-framework-api
This closes #3322 .
Signed-off-by: Mark Payne <markap14@hotmail.com>
2019-02-22 12:19:12 -05:00
26c00f1080
NIFI-6069 Fixing fingerprinting of connection relationships to be properly sorted
...
This closes #3325 .
Signed-off-by: Mark Payne <markap14@hotmail.com>
2019-02-22 12:13:13 -05:00
414bfc94fc
NIFI-6068: Fixed unit test
2019-02-22 11:55:25 -05:00
e26bacf697
NIFI-6068, NIFI-6065: Updated StandardFunnel to avoid looping indefinitely and instead transfer no more than 10,000 FlowFiles before returning from onTrigger. Updated Local Ports to behavior in the same way. Updated Root Group Ports so that instead of blocking for up to 100 milliseconds for an incoming request, it blocks for up to 1 millisecond and if nothing is available yields for the 'bored yield duration'
...
Signed-off-by: Brandon Devries <devriesb@apache.org>
This closes #3328 .
2019-02-22 10:12:20 -05:00
2336044b90
NIFI-6059 - Fixed NPE in GCS processors when project ID is not specified
...
This closes #3327 .
Signed-off-by: Mark Payne <markap14@hotmail.com>
2019-02-22 09:45:13 -05:00
a0c28ee19e
NIFI-5660: JMSPublisher should set some header properties in JmsTemplate instead of directly in the message
...
NIFI-5660: Added lines to integration test to verify these header properties (added by Mike Moser)
Signed-off-by: Mike Moser <mosermw@apache.org>
This closes #3053
2019-02-21 14:45:04 -05:00
60b00a1e38
NIFI-2738 Removed references to nifi.cluster.request.replication.claim.timeout
...
This closes #2860
Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2019-02-21 06:34:27 -05:00
25cc7b4a1e
NIFI-6029 merging nifi 1.9.0 release into master
2019-02-19 22:55:49 -05:00
76e92c8682
NIFI-6052 Update NOTICE files to reflect 2019
...
This closes #3319
Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2019-02-19 18:32:15 -05:00
d59a07245f
NIFI-5500 Adding support for array fields when outputting to attributes when using QueryElasticsearchHttp
...
NIFI-5500 Optimised array-to-string conversion
This closes #2942
Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2019-02-19 12:41:57 -05:00
af6039a5b0
NIFI-6045: Added support EL support to dyanmic properties for Kafka 2.0 processors
...
This closes #3316
Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2019-02-19 10:34:43 -05:00
53ca9eb7e5
NIFI-5987 Fixed issue where an invalid query pulled from an attribute would cause GetMongo to not route to failure.
...
Signed-off-by: zenfenan <zenfenan@apache.org>
This closes #3285
2019-02-19 16:24:15 +05:30
0e204f3576
NIFI-6029-RC2 prepare for next development iteration
2019-02-16 21:50:35 -05:00
45bb53d2aa
NIFI-6029-RC2 prepare release nifi-1.9.0-RC2
2019-02-16 21:50:15 -05:00
82f44155f6
NIFI-6044: This closes #3314 . Retain the input data's order in the CSV Reader's inferred schema
...
Signed-off-by: joewitt <joewitt@apache.org>
2019-02-16 20:57:42 -05:00
9e0687ab6b
NIFI-6018 Marking GetHTTP and PostHTTP deprecated. Adding info about deprecation to processors descriptions. Fix small mistake in developer-guide.
...
Suppressed deprecation warnings in TestGetHTTP, TestPostHTTP, TestGetHTTPGroovy, and TestPostHTTPGroovy. The annotation suppresses some warnings, but until Java 9, explicit FQCN imports are needed to avoid all warnings.
This closes #3312 .
Signed-off-by: Andy LoPresto <alopresto@apache.org>
2019-02-15 14:09:26 -08:00
b7ca59f568
NIFI-6023: Deprecated the "Distributed Cache Service" property in ListHDFS
...
Updated exception message thrown when accessing the ListHDFS state to refer to the State Manager rather than the "Distributed Cache Service"
Removed usage and references to MockCacheClient from TestListHDFS
NIFI-6023: Minor update of the "Distributed Cache Service" description
This closes #3313 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2019-02-15 14:12:04 -05:00
b508d6bfbc
NIFI-6033, NIFI-6034, NIFI-6035, NIFI-6036, NIFI-6037: Fixed bugs that were found during 1.9.0-RC1 validation. If multiple FlowFiles were written to same Content Claim, and a Processor attempted to read two of them wi within a single session, it would seek to the wrong part of the content or else throw a ContentNotFoundException. Updated logic for considering a processor to be 'running' / having 'active threads' if the processor is invalid upon NiFi restart but scheduled to run. Fixed NPE in FreeFormTextWriter. If MergeRecord reaches minimum number of records, flush writer after writing content out so that its minimum size can accurately be checked.
...
This closes #3309 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2019-02-15 11:59:47 -05:00
f61947627e
NIFI-6038 Remove scope parameter from OIDC TokenRequest
...
This closes #3308
2019-02-14 16:43:15 -05:00
e598b30d6d
NIFI-6000 Catch also IllegalArgumentException in ConvertAvroToORC hive processor. Added support for Avro null types.
...
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #3302
2019-02-14 15:08:54 -05:00
ae8a794ff0
NIFI-6031: allow OS level socket keep alive checking
...
This closes #3307 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2019-02-14 13:29:00 -05:00
8a197e5a96
NIFI-4367 Fix on processor for permit deriving script classes from AbstractProcessor or other Records based base classes
...
Improve UT
Add licence and fix import
improve filtering properties
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #2201
2019-02-14 12:19:13 -05:00
e7ae97797e
NIFI-5943 support conversions from List to Avro ARRAY and from Map to Avro RECORD
...
NIFI-5943 Added another unit test to verify list + map conversion to list of records. (Mike Thomsen)
This closes #3267
Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2019-02-14 08:31:42 -05:00
45b32e3bc1
NIFI-5947 Elasticsearch lookup service compatible with LookupAttribute
...
NIFI-5947 Made the lookup key for ElasticSearchStringLookupService more unique to the service (was previously 'id'); change by Mike Thomsen
This closes #3268 .
Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2019-02-14 08:02:29 -05:00
033b2a1940
NIFI-4975 Add GridFS processors
...
NIFI-4975 Added changes requested in a code review.
NIFI-4975 Reverted some base Mongo changes.
NIFI-4975 Moved connection configuration to using Mongo client service.
NIFI-4975 Fixed a lot of style issues.
NIFI-4975 Removed an EL statement that was causing problems with the UI.
NIFI-4975 Added changes from code review.
NIFI-4975 Added additional details for FetchGridFS.
NIFI-4975 Added documentation for DeleteGridFS.
NIFI-4975 Added documentation for PutGridFS.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #2546
2019-02-13 16:51:17 -05:00
72244d09ff
NIFI-5575: Added Hadoop configuration to MockFileSystem for umask tests in PutHDFSTest
...
Minor update to Permissions umask property description
Minor code cleanup/formatting
This closes #3252
2019-02-13 16:09:12 -05:00
7a763d3c49
NIFI-5575 Make PutHDFS check fs.permissions.umask-mode if property "Permission umask" is empty.
2019-02-13 16:08:57 -05:00
2938454ae4
NIFI-6020: Fix NPE in getAccessPoliciesForUser
...
This closes #3304
2019-02-13 16:03:46 -05:00
3492313d0b
NIFI-5869 Support Reconnection for JMS
...
resets worker if it doesn't work anymore for any reason. this will add "reconnect" capabilities. Will solve issues for following use cases:
- authentication changed after successful connection
- JNDI mapping changed and requires recaching.
- JMS server isn't available anymore or restarted.
improved controller reset on exception
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #3261
2019-02-13 14:18:05 -05:00
da8c8a14a1
NIFI-6017 - ArrayIndexOutOfBounds Load Balancer CorrelationAttributePartitioner
...
Signed-off-by: Mark Payne <markap14@hotmail.com>
2019-02-13 08:53:19 -05:00
Koji Kawamura
Matthew Burgess
Sandish Kumar
pushpavanthar
Ryan Bower
Mark Payne
Peter Turcsanyi
Ferenc Szabó
Kevin Doran
thenatog
Arpad Boda
Otto Fowler
Joe Percivall
Bryan Bende
Mike Thomsen
Pierre Villard
Mark Bean
Jan Hentschel
joewitt
Denes Arvay
Wietze
Corey Fritz
Aleksandr Salatich
Jeff Storck
Mike Moser
Endre Zoltan Kovacs
Patrice Freydiere
Alex Savitsky
Key Miyauchi
Ed
Dorian Bugeja