Commit Graph

4787 Commits

Author SHA1 Message Date
Mark Payne 234ddb0fe1 NIFI-5745: When determining if backpressure should be applied across nodes for load balancing, only consider if the local partition has reached the threshold limits instead of considering the size of the entire queue
This closes #3108.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2018-10-26 16:04:46 +09:00
Mark Payne c7ff2fc5db NIFI-5746: Use Node Identifier's node address instead of getting from socket for RECEIVE prov events; make SEND prov events match syntax of RECEIVE prov events
NIFI-5746: Addressed issue found in review process

This closes #3109.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2018-10-26 10:24:12 +09:00
Pierre Villard 2201f7746f NIFI-5714 - Hive[3]ConnectionPool - Kerberos Authentication issue/misleading
add @Ignore on unit test...

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3086
2018-10-24 13:33:30 -04:00
Koji Kawamura d28b1172db NIFI-5739: Maintain CaptureChangeMySQL JDBC connection automatically
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3103.
2018-10-24 09:26:41 +02:00
Mark Payne ebead820f9
NIFI-5741: When returning a ConnectionFactory from the JndiJmsConnectionFactoryProvider, ensure that we wrap the ConnectionFactory so that any calls to the ConnectionFactory happen within the context of the Controller Service's Class Loader
This closes #3106.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-10-23 16:17:25 -04:00
Bryan Bende 10479a5a2a NIFI-5740 Ensuring permissions are restored after test completes in TestFetchFile
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3104
2018-10-23 15:26:33 -04:00
rednikotin c8928ce350 NIFI-5727: Added replace unnecessary row count with -1 stub value when paging is used
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3094
2018-10-23 14:46:14 -04:00
Mohit Garg e45584d0fb
NIFI-5706 Added ConvertAvroToParquet processor
- Refactored code : ParquetBuilderProperties merged in ParquetUtils

This closes #3079.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-10-23 14:08:07 -04:00
Matt Gilman 02261311b3 NIFI-5737:
- Removing needClientAuth property since cluster comms now requires two way ssl. Jetty client auth settings are based on configured features.
- Removing dead code.
- Updating documentation.
- Removing references to needClientAuth property in all test resources.
- Removing overloaded util method with strict parameter.

This closes #3102.
2018-10-22 20:02:47 -04:00
Pierre Villard c6106d1d88 NIFI-5525 - CSVRecordReader fails with StringIndexOutOfBoundsException when field is a double quote
review

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3092
2018-10-22 16:58:36 -04:00
Christian Englert 4fe7cb33e4
NIFI-5734 MQTTProcessors should have sensitive password
This closes #3099.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-10-22 15:19:06 -04:00
Bryan Bende ebaaf5797e
NIFI-5721 Fixing connection handling in MQTT processors
This closes #3096.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2018-10-22 15:13:14 -04:00
Mark Payne 5561c29ed3 NIFI-5736: Removed 'synchronized' keyword from SwappablePriorityQueue#getProperties - the method should not be synchronized, as it guards access to member variables via a ReadLock. I am guessing that the synchronized keyword is the result of refactoring from 'synchronized' to a read lock and inadvertently leaving the 'synchronized' keyword in.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3101
2018-10-22 12:22:05 -04:00
thenatog 08c54bc48d Revert "NIFI-4558 - Set JKS as the default keystore type and truststore type."
This reverts commit 2b9fdf8415.

This closes #3097.
2018-10-20 17:50:24 -04:00
Takanobu Asanuma ec8afd18d6 NIFI-5446: Specify the file encoding option to UTF-8 in pom.xml
This closes #2910.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2018-10-18 15:09:23 +09:00
Bryan Bende f3f7cdbab9 NIFI-5719 Ensuring FetchFile routes to failure if the move completion strategy can't be completed
This closes #3088.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2018-10-18 14:37:55 +09:00
Andrew Lim 5ec85299e7 NIFI-5653 Added default NiFi and Embedded Zookeeper port tables to Admin Guide
This closes #3089.
2018-10-17 15:29:11 -04:00
Matthew Burgess c752ec8394
NIFI-5705: Added Hive 3 attribution to nifi-assembly and top-level NOTICEs
This closes #3076
2018-10-17 12:24:32 -04:00
Mark Payne d6422e2d2c NIFI-5711: Replaced JavaDocs that appear also to be copied/pasted from BufferedReader, ran IDE code formatter to get consistent whitespace/formatting
This closes #3085.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-10-17 10:57:57 -04:00
patricker 95e4294673 NIFI-5711 NLKBufferedReader appears extend and copy portions of the JDK BufferedReader 2018-10-17 10:57:53 -04:00
Matt Gilman 87c6b3aa7c
NIFI-5715: Updating the allowable values for the runStatus.
This closes #3087.

Signed-off-by: Kevin Doran <kdoran@apache.org>
2018-10-17 10:33:53 -04:00
Andrew Lim 4039c21d67 NIFI-5701 Add documentation for Load Balancing connections to User Guide and add additional properties to Admin Guide
This closes #3080.
2018-10-16 21:00:38 -04:00
Bryan Bende 51ed618cf0 NIFI-5708 Fixing the creation of ValidationContextFactory for controller services so it uses the ComponentVariableRegistry and not the file-based instance
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3081
2018-10-16 15:38:32 -04:00
Mark Payne 4d21f9b34e
NIFI-5709, NIFI-5710: Addressed issue that causes NiFi to not be able to read provenance events when the a new FlowFile is created and then auto-terminated in the same session; minor bug fixes outlined in NIFI-5710
This closes #3083
2018-10-16 15:25:29 -04:00
Matt Gilman 11fd67cd1e
NIFI-375:
- Ensuring the run status endpoints are properly merged.

This closes #3082.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-10-16 14:59:27 -04:00
Andrew Lim fd5eccc593 NIFI-5659 Add documentation for Offloading Nodes functionality and new Node related CLI commands
This closes #3056
2018-10-15 18:02:14 -04:00
Mike Thomsen 11afb0115e NIFI-4811 This closes #2856. Removed no longer necessary license and updated NOTICE
Signed-off-by: joewitt <joewitt@apache.org>
2018-10-15 16:43:05 -04:00
Mark Payne 32db43b306 NIFI-5686: Updated StandardProcessScheduler so that if it fails to schedule a Reporting Task, it re-schedules the @OnScheduled task instead of looping and calling Thread.sleep. As it was, the single-threaded Process Scheduler was, when calling ProcessScheduler.unschedule(), the unschedule task was not executing because the schedule task was using the only thread. But switching the logic to schedule the task for later and return, instead of calling Thread.sleep and looping, we are able to avoid blocking the one thread in the thread pool. Also, performed some trivial code cleanup and updated erroneous links in Java-docs.
NIFI-5686: Fixed unit test in TestSocketLoadBalancedFlowFileQueue; renamed TestProcessorLifecycle to ProcessorLifecycleIT as it is testing integration between many components and largely focuses on high numbers of concurrent tasks to see if it can trigger any threading bugs that may get introduced

NIFI-5686: Extended unit test timeouts
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3062
2018-10-15 15:21:05 -04:00
Matthew Burgess 218063a0b5 NIFI-5678: Fixed MAP type support of MapRecord objects in StandardSchemaValidator
This closes #3060.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-10-15 14:33:21 -04:00
Matt Gilman f55204cb69
NIFI-4806:
- Bumping to tika 1.19.1.

This closes #3069
2018-10-15 13:52:49 -04:00
Endre Zoltan Kovacs d5f071b2fe NIFI-3469 multipart request support added to HandleHttpRequest
- introducing a in-memory-file-size-threashold, above which the incoming file is written to local disk
- using java.io.tmpdir for such file writes
- enhancing documentation
- documenting how to avoid premature HTTP response
- fix and UT for unsuccessful request registration

This closes #2991.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-10-15 13:14:31 -04:00
Matthew Burgess ce25ae5419
NIFI-5667: Add nested record support for PutORC
NIFI-5667: Fixed default table name

NIFI-5667: Fixed handling of binary types

NIFI-5667: Added backticks in Hive DDL generation

This closes #3057.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-10-15 10:10:47 -04:00
Jeff Storck 7cb39d636c NIFI-5696 Update references to default value for nifi.cluster.node.load.load.balance.port
This closes #3071.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2018-10-15 10:43:31 +09:00
zenfenan 0229a5c109 NIFI-5698: Fixed DeleteAzureBlobStorage bug
This closes #3073.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2018-10-15 10:27:15 +09:00
Brad Hards e30a21cfc5
[NIFI-5697] Trivial description fix for GenerateFlowFile processor
This closes #3072.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2018-10-13 05:53:30 -04:00
Mark Payne 270ce8570d
NIFI-5695: Fixed bug that caused ports to not properly map to their correct child group on Flow Import if the child group is independently versioned
This closes #3070.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-10-12 16:26:59 -04:00
thenatog 5eb5e96b16
NIFI-5665 - Changed netty versions to more closely match the original netty dependency version.
NIFI-5665 - Fixed version for nifi-spark-bundle.
NIFI-5665 - Fixing copy and paste error.

This closes #3067
2018-10-12 15:56:52 -04:00
Bryan Bende 02e0a16a68 NIFI-5680 Handling trailing slashes on URLs of registry clients
This closes #3065.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-10-12 14:18:42 -04:00
Matt Gilman 0f8880547f
NIFI-5691:
- Overriding the version of jackson in aws java sdk.

This closes #3066.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2018-10-12 13:19:43 -04:00
joewitt e25b26e9cf Revert "NIFI-5448 Added failure relationship to UpdateAttributes to handle bad expression language logic."
This reverts commit 32ee552ada.
2018-10-12 11:27:48 -04:00
joewitt 6b77e7dd89 Revert "NIFI-5448 Changed from 'stop' to 'penalize' in allowablevalue field to make the popup more consistent."
This reverts commit 9d2b698c1c.
2018-10-12 11:08:22 -04:00
Carl Gieringer a6b9364ebf NIFI-5664 Support ArrayList in DataTypeUtils#toArray
NIFI-5664 Generalize to handling List

This closes #3049

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-10-12 06:28:56 -04:00
Endre Zoltan Kovacs 5aa4263588 NIFI-1490: better field naming / displayname and description mix up fix
This closes #2994.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-10-11 15:41:38 -04:00
Endre Zoltan Kovacs c81a135161 NIFI-1490: multipart/form-data support for ListenHTTP processor
- introducing a in-memory-file-size-threashold, above which the incoming file is written to local disk
- using java.io.tmpdir for such file writes
- enhancing documentation
2018-10-11 15:41:19 -04:00
Mark Payne 8398ea77bc
NIFI-5688: Ensure that when we map our flow to a VersionedProcessGroup that we include the connections' Load Balance Compression flag
This closes #3064
2018-10-11 15:06:06 -04:00
Matt Gilman 8da403ce9e
NIFI-5661:
- Allowing load balance settings to be applied during creation.
- Clearing the load balance settings when the dialog is closed.
2018-10-11 13:21:20 -04:00
Matt Gilman 79c03caf4f
NIFI-5661:
- Allowing the load balance configuration to be shown/edited in both clustered and standalone mode.
2018-10-11 12:23:53 -04:00
thenatog 64de5c70e1
NIFI-5479 - Supressed the AnnotationParser logs using the logback.xml. Dependency changes can be look at in future.
NIFI-5479 - Updated comment.

This closes #3034
2018-10-11 12:11:44 -04:00
Koji Kawamura 8a751e8018
NIFI-5661: Adding Load Balance config UI
Incorporated review comments.
Move combo options to a common place.

This closes #3046
2018-10-11 11:36:28 -04:00
Koji Kawamura a6f722222a
NIFI-5645: Auto reconnect ConsumeWindowsEventLog
This commit also contains following refactoring:
- Catch URISyntaxException inside subscribe when constructing provenance
URI as it does not affect the core responsibility of this processor.
Even if it fails to be a proper URI, if the query works for consuming
logs, the processor should proceed forward.

Upgrade JNA version.

Do not update lastActivityTimestamp when subscribe failed.

This closes #3037
2018-10-11 10:05:26 -04:00