Commit Graph

6183 Commits

Author SHA1 Message Date
neptunesalt ee91341ec3
NIFI-7497 Adding support for AWS Credentials Assume Role to be able to set the STS Endpoint
NIFI-7497 Updating property description per comment

This closes #4309

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-07-10 09:19:58 -04:00
Chad Zobrisky b0494603da
NIFI-7610 Updated dockerhub readme to latest nifi version
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4394
2020-07-09 12:17:32 -04:00
Tamas Palfy ad95287e78
NIFI-6934 In PutDatabaseRecord added DatabaseAdapter-based UPSERT support for Postgres (9.5+)
NIFI-6934 Added more documentation and unit tests.

NIFI-6934 Added missing license for new test class.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4350
2020-07-09 11:26:41 -04:00
Andy LoPresto ca384b3f3b
NIFI-7620 Fixed failing unit test due to timeout. (#4395) 2020-07-09 09:14:34 -04:00
Mark Payne 6cece9cad7
NIFI-7572: Added ScriptedTransformRecord processor. Addressed a couple of minor bugs in mock classes that were encountered during testing.
NIFI-7572: Addressed review feedback
NIFI-7572: Fixed bug that resulted in constantly recompiling Jython script. Updated documentation showing performance difference. Fixed problematic unit tests for TestResizeImage

This closes #4374

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-07-08 15:52:06 -04:00
markap14 e17db80514
NIFI-7607: Fixed bug that caused the wrong Controller Service to be de-referenced when a component that used to reference a Controller Service is changed to reference a different Controller Service (#4391) 2020-07-07 16:38:42 -04:00
mtien 43fb57e7bb
NIFI-7332 Added method to log available claim names from the ID provider response when the OIDC Identifying User claim is not found. Revised log message to print available claims.
Added new StandardOidcIdentityProviderGroovyTest file.
Updated deprecated methods in StandardOidcIdentityProvider. Changed log output to print all available claim names from JWTClaimsSet. Added unit test.
Added comments in getAvailableClaims() method.
Fixed typos in NiFi Docs Admin Guide.
Added license to Groovy test.
Fixed a checkstyle error.
Refactor exchangeAuthorizationCode method.
Added unit tests.
Verified all unit tests added so far are passing.
Refactored code. Added unit tests.
Refactored OIDC provider to decouple constructor & network-dependent initialization.
Added unit tests.
Added unit tests.
Refactored OIDC provider to separately authorize the client. Added unit tests.
Added unit tests.

NIFI-7332 Refactored exchangeAuthorizationCode method to separately retrieve the NiFi JWT.

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #4344.
2020-07-07 15:54:32 -04:00
Matthew Burgess f040c6aadb
NIFI-5213: Allow AvroReader to process files w embedded schema even when the access strategy is explicit schema
NIFI-5213: Incorporated review comments

This closes #2718

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-07-07 14:43:57 -04:00
mtien f43dc8063f
NIFI-7542 Upgrade additional jackson-databind versions.
This close #4362

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-07-07 08:26:13 -04:00
Joe Witt f8e5be3c09
NIFI-7602 added verify phase so contrib-check can actually execute
This closes #4388

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-07-07 07:40:42 -04:00
KovalevIV ea02788de6
NIFI-7603 fix doc
This closes #4387

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-07-07 07:35:48 -04:00
mtien 74b4d5ac5f
NIFI-7595 Upgrade log4j-core dependency.
This closes #4382

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-07-07 07:28:45 -04:00
Tamas Palfy c396927299 NIFI-7594 In HandleHttpRequest deleting multipart file resources after processing.
This closes #4379.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-07-06 14:15:43 +02:00
Gardella Juan Pablo e1584207d1 NIFI-7563: Optimize the usage of JMS sessions and message producers
The introduced changes prevent creating unnecesary sessions and producers
in some scenarios.

This closes #4378.

Signed-off-by: Joey Frazee <jfrazee@apache.org>
2020-07-03 16:43:29 -05:00
VKadam 239a2e884c
NIFI-7513 Added custom DNS resolution steps to walkthrough (#4359) 2020-07-01 10:52:22 -07:00
Tamas Palfy 197df577ac NIFI-7590 In 'CassandraSessionProvider.onDisabled' setting Cassandra-related references properly to null after closing them so that they can be renewed in 'onEnabled' (which creates them only if set to 'null', leaving them closed otherwise).
NIFI-7590 Removed 'CassandraSessionProvider.onStopped'.

This closes #4373.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-06-30 21:56:14 +02:00
Andy LoPresto 045027892a
NIFI-7587 This closes #4372. Increased tolerance for non-deterministic unit test.
Signed-off-by: Joe Witt <joewitt@apache.org>
2020-06-30 09:27:43 -07:00
Javi Roman c221e4934d
NIFI-7578 nifi-toolkit CLI Process Group Create command
- Remove unused imports
- Fix checkstyle errors

This closes #4358.
2020-06-30 09:48:45 -04:00
Tamas Palfy c2f46c44ca NIFI-7586 In CassandraSesionProvider added properties to set socket-level read timeout and connect timeout.
In QueryCassandra when writing flowfile to the sesion it's done on the raw OutputStream.
Wrapped it in a BufferedOutputStream for better performance.

This closes #4368.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-06-30 15:21:03 +02:00
mtien 87ec8558a4 NIFI-7577 Update jquery usages.
This closes #4357

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
2020-06-26 15:01:30 -04:00
mtien 7df2421fe7 NIFI-7577 Upgrade angular version. 2020-06-26 14:39:47 -04:00
Makarov Vasily a2c88bc98f
NIFI-6163 Reporting task cannot be set to running when in INVALID state
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4334.
2020-06-26 16:48:55 +02:00
Tamás Bunth 91cd74d725 NIFI-7576 ListenHTTP: Honor primary node only
ListenHTTP processor now binds port and creates a HTTP connection only
if one of the following conditions apply:
- Primary node execution is 'false'
- Primary node execution is 'true' and node is elected as primary node.

Changes:
- Connection is established in 'onTrigger' annotated method instead of
  'onSchedule'. (This is similar to how handleHTTPRequest processor
  handles connections.)
- 'onPrimaryNodeStateChange' annotated method is introduced to tear down
  server on reelection of primary node

This closes #4356.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2020-06-23 14:26:45 -04:00
Peter Turcsanyi 327d73f5cd
NIFI-7523: Use SSL Context Service for Atlas HTTPS connection in Atla… (#4348)
* NIFI-7523: Use SSL Context Service for Atlas HTTPS connection in Atlas reporting task

Also fixing ControllerServiceDisabledException-s when validating the Kerberos config

* NIFI-7523: Fixed test failure on Windows

* NIFI-7523: Added license headers.

* NIFI-7523: Fixed another test failure on Windows

* NIFI-7523: Review changes
2020-06-23 14:22:50 -04:00
mtien 005d05f20b
NIFI-7542 Override jackson-databind version.
NIFI-7542 Override additional jackson-databind versions.
NIFI-7542 Upgrade jackson-databind dependency to 2.9.10.5 in the root pom.xml.

This closes #4343

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-06-22 17:37:57 -04:00
Andy LoPresto 94c98c019f
NIFI-7558 Fixed CatchAllFilter init logic by calling super.init().
Renamed legacy terms.
Updated documentation.

This closes #4351.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2020-06-22 12:20:28 -07:00
dennisjaheruddin 704260755a
NIFI-7501
Update nf-context-menu.js for an intuitive road to parameters
When rightclicking a process group the variables are shown, but parameters are not. This makes sense as they have a prerequisite, in the form of a parameter context. This change gives a more consistent experience for finding the functionality regarding parameters by ensuring the contextmenu shows the possibility to configure a parameter context. Once the paramater context has been created for a process group, the parameters text shows, so this is no longer visible. People would then need to click configure to change the context, just as they would be required to do now.

Added generateflowfile load tag and description
Added GenerateFlowFile load tag to be consistent with DuplicateFlowFile and updated description to refer to DuplicateFlowFile.

Revert "Update nf-context-menu.js for an intuitive road to parameters"

This reverts commit 3c44b1661f.

This closes #4333

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2020-06-22 11:12:22 -04:00
Matthew Burgess 32fa9ae51b
NIFI-7516: Catch and log SingularMatrixExceptions in OrdinaryLeastSquares model (#4323) 2020-06-22 09:15:41 -04:00
Mike Thomsen 64b544d5af NIFI-7529 Removed OS and Java information from InvokeHttp's UserAgent field so that it's removed regardless of whether or not this field is kept.
This closes #4332
2020-06-21 12:40:53 -04:00
Mark Payne 57c7883f64
NIFI-7566: Avoid using Thread.sleep() to wait for Site-to-Site connection to be handled. Instead, use TimeUnit.timedWait and use Object.notifyAll when setting the beingServiced flag. This significantly reduces latency and improves throughput for small-batch site-to-site communications
This closes #4353.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2020-06-19 15:37:02 -07:00
Mark Payne 0a16002076
NIFI-7509: Added optional Record Writer property to all List* Processors
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4315.
2020-06-19 17:30:40 +02:00
Mark Payne c9d08a76b1 NIFI-7536: Fix to improve performance when determining the run status of processors when needing to wait for all processors to stop for updating parameter context, etc. 2020-06-19 10:14:12 -04:00
Greg Solovyev a1b245e051
NIFI-7551 Add support for VARCHAR to Kudu NAR bundle
- update Kudu dependencies to Kudu 1.12.0
 - add VARCHAR to Kudu Lookup Service and Processor
 - add tests for VARCHAR columns

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4347.
2020-06-19 09:19:12 +02:00
thenatog 27b5bb7a20
NIFI-6094 - Added the X-Content-Type-Options header to all web responses. (#4307)
NIFI-6094 - Added the mime/content type for ttf files.
2020-06-17 17:15:18 -07:00
Nathan Gough 2b25ccc298 NIFI-7537 - Small fix to make the LDAP connection timeout property a String rather than a Long. Fixes cast error in LDAP libs. 2020-06-17 16:33:02 -04:00
Joey c18b27af18
NIFI-7540: Fix TestListenSMTP and TestListFile on macOS build environment (#4341)
* NIFI-7540: Fix TestListenSMTP and TestListFile on macOS build environment

This also fixes NIFI-4760.

* NIFI-7540: Remove duplicate mail.smtp.starttls.enable from TestListenSMTP

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2020-06-16 17:45:37 -07:00
Mark Payne 8b1a23a99c NIFI-7539: When capturing diagnostics information, capture a thread dump once and then provide this information to ProcessorNode when capturing active threads. Previously, each processor captured a thread dump itself. When this is done thousands of times it can result in a very long delay. 2020-06-16 14:48:14 -04:00
Макаров Василий Николаевич 9828e7dd14 NIFI-7035 The first curator connection issue is logged as ERROR until reconnect 2020-06-16 12:24:41 -04:00
Tamas Palfy e02ffdd99f NIFI-7527 AbstractKuduProcessorrefresh TGT deadlock fix: Redesigned locking.
NIFI-7527 Fixed StackOverFlowError due to pacing issue (recursive login before loggedIn flag is set).
NIFI-7527 Refactor: removed redundant kudu client creation.

This closes #4330.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-06-15 14:17:34 +02:00
Tamas Palfy a126d0a6b6
NIFI-7442 Added missing use cases (list users and user groups), made update-access-policy use case more in line with the NiFi side. Added some tests. Additional refactor, documentation revision.
This closes #4329.
2020-06-12 12:43:41 -04:00
Bence Simon d99983bdc7
NIFI-7442 Add CLI commands to the registry in order to support automatic registry setup 2020-06-12 12:43:27 -04:00
sjyang18 73008451d4 NIFI-7434: Add endpoint suffix to Azure storage processors
This closes #4265.

Signed-off-by: Joey Frazee <jfrazee@apache.org>
2020-06-10 09:45:04 -05:00
wanghq-c 3c757b6ba8
NIFI-7313:fix bug on 'Quote Table Identifiers'
NIFI-7313:add test by wanghongqi
NIFI-7313:edit test

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4185
2020-06-09 16:02:25 -04:00
Pierre Villard a2f521297f
NIFI-7490 - Add optional raw field to Syslog readers
review

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4299
2020-06-09 15:37:08 -04:00
Matt Gilman eb2ebefc46
NIFI-7514:
- Ensuring the group id is always set in the properties table when loading properties.
- Using a common approach to getting parameters in nfControllerService.
- Code clean up.
- Addressing review feedback.
- Ensuring the service dialog is closed when navigating to the parameter context dialog.

This closes #4322
2020-06-09 15:23:58 -04:00
Nathan Gough 01e42dfb32
NIFI-7385 Provided reverse-indexed TokenCache implementation.
Cleaned up code style.
Unit test was failing on Windows 1.8 GitHub Actions build but no other environment. Increased artificial delay to avoid timing issues.

Co-authored-by: Andy LoPresto <alopresto@apache.org>

This closes #4271.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2020-06-08 19:36:14 -07:00
Tamas Palfy aa7c5e2178 NIFI-7511 In ControllerServiceProxyWrapper extended documentation. Minor refactor in StandardControllerServiceInvocationHandler. Also removed an unused import from NiFiSystemIT.
This closes #4317.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2020-06-08 16:04:15 -04:00
Mark Payne 463d72117b
NIFI-7507: Added section to User Guide on configuring a Process Group
NIFI-7507: Fixed Flowfile Expiration header in doc

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4318
2020-06-08 14:03:00 -04:00
Wouter de Vries f2368a0dd1 NIFI-7393: Add max idle time and idle connections to InvokeHTTP
This closes #4233.

Signed-off-by: Joey Frazee <jfrazee@apache.org>
2020-06-08 09:29:10 -05:00
Andy LoPresto 845b66ab92
NIFI-7467 Refactored S2S peer selection logic.
Removed list structure for peer selection as it was unnecessary and often wasteful (most clusters are 3 - 7 nodes, the list was always 128 elements).
Changed integer percentages to double to allow for better normalization.
Removed 80% cap on remote peers as it was due to legacy requirements.
Added unit tests for non-deterministic distribution calculations.
Added unit tests for edge cases due to rounding errors, single valid remotes, unbalanced clusters, and peer queue consecutive selection tracking.
Migrated all legacy PeerSelector unit tests to new API.
Removed unused System time manipulation as tests no longer need it.
Added class-level Javadoc to PeerSelector.
Removed S2S details request replication, as the responses were not being merged, which led to incorrect ports being returned and breaking S2S peer retrieval.
Fixed copy/paste error where input ports were being listed as output ports during remote flow refresh.
Fixed comments and added unbalanced cluster test scenarios.
Removed unnecessary marker interface.
Removed commented code.
Changed weighting & penalization behavior.
Changed dependency scope to test.

This closes #4289.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2020-06-05 13:40:16 -07:00