Commit Graph

1453 Commits

Author SHA1 Message Date
Kemix Koo 5127cb209a
NIFI-6392: Avoid accessing the dir for /nifi-docs
Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2019-07-06 18:39:29 -04:00
Rob Fellows 846c9b0a3f
NIFI-6401 - Long List of Users Overflows onto Date
This closes #3567
2019-07-02 16:02:39 -04:00
Troy Melhase e973cacb2f
NIFI-5973 Adds ShellUserGroupProvider.
NIFI-5973 More comments and better defaults for the shell provider.
NIFI-5973 Fixed bug where user was being retrieved by identifier when identity was provided.
NIFI-5973 Fixed a formatting string in the OS X shell commands.
Updated testing conditions to run IT in OS X environment.
Changed unit test to provide identity rather than identifier.

This closes #3537.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2019-06-20 16:22:42 -07:00
Jeff Storck 1d560e2b02 NIFI-6360 Updated Mockito to 2.28.2, PowerMock to 2.0.2
Fixed test failures in nifi-couchbase-processors, BinaryDocument matcher replaced with ByteArrayDocument
Fixed test failures in nifi-riemann-processors, anyInt() matcher replaced with anyLong() matcher, calling method passes a long, not int
Removed unnecessary method mocks from nifi-toolkit-tls tests, TlsCertificateAuthorityServiceHandlerTest and TlsCertificateSigningRequestPerformerTest, since those were flagged by Mockito as unnecessary (they're unused)
Removed explicit mockito dependency version in nifi-gcp-processors pom to inherit version from nifi's pom.xml
Updated ArgumentMatchers in Kafka 0.10, 0.11, 1.0, and 2.0 processor tests, since in Mockito 2.x, the "any" matchers no longer allow nulls
Updated ArgumentMatchers in nifi-jolt-transform-json-ui, since in Mockito 2.x, the "any" matchers no longer allow nulls
Removed unnecessary method mocks from MetricsReportingTaskTest
Updated TestStandardRemoteGroupPort to return Long instead of Integer for test flowfile.size() invocations
Updated AbstractCassandraProcessor to include keyspaceProperty.getValue() in null check
Updated SimpleProcessLogger and TestSimpleProcessLogger, vararg matching does not work the same in Java 8 and 11
Updated TestStandardProcessScheduler to allow null values during mock invocations, Mockito 2.x no longer allows nulls in those matchers
Updated TestPutHiveStreaming to allow null values during mock invocations, Mockito 2.x no longer allows nulls in those matchers
Updated FetchParquetTest to allow null values during mock invocations, Mockito 2.x no longer allows nulls in those matchers
Updated ControllerSearchServiceTest to allow null values during mock invocations, Mockito 2.x no longer allows nulls in those matchers
Removed usage of Whitebox from GetAzureEventHubTest due to Mockito 2.x, replaced with FieldUtils
Removed usage of Whitebox from StandardOidcIdentityProviderTest due to Mockito 2.x, replaced with FieldUtils
Updated apache-rat-plugin configuration in root POM to make use of useIdeaDefaultExcludes which makes the rat plugin exclude IntelliJ artifacts
Updated several modules to use mockito-core instead of mockito-all (discontinued in Mockito 2.x)
Updated nifi-site-to-site-reporting-task tests to be compatible with Mockito 2.x
Ignored TestPutJMS tests; the tests need to be refactored to work with Mockito 2.x, but the processor is deprecated.  Refactor may be done in a separate PR.
Adjusted several mock interaction iterations to 0 for TestPublishKafkaRecord_* tests.  Mockito 2.x flagged several interactions as unused and were adjusted to 0 interactions.
Updated PowerMock and Mockito dependencies to exclude transitive dependency on bytebuddy, added explicit dependency on bytebuddy 1.9.10 so that PowerMock and Mockito use the same version.  Bytebuddy 1.9.3 (used by PowerMock 2.0.2) did not allow for the mocking of final/private classes, bytebuddy 1.9.10 (used by Mockito 2.28.2) does.
Updated TestSiteToSiteProvenanceReportingTask use of InvocationOnMock.getArgument to use objects for the resulting object rather than primitives
Removed unnecessary stubs from evtx tests, Mockito 2.x defaults to strict mocks
Fixed classloader issue with tests in nifi-windows-event-log-processors module that use JNAJUnitRunner when Mockito mocked JNA classes (Kernel32)
Addressed Mockito-related deprecation warnings
Import cleanup

This closes #3533

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2019-06-17 12:21:07 -04:00
thenatog 0e1a37fcb9
NIFI-6362 - Upgraded com.puppycrawl.tools:checkstyle to 8.18. Made small checkstyle changes to nifi-kafka.
NIFI-6362 - Upgraded both checkstyle plugins to latest. Added a checkstyleCache file and property.

NIFI-6362 - Downgraded maven-checkstyle-plugin to 3.0.0 instead as there was an issue with the latest version and @throws Javadoc checkstyle module. Fixed some checkstyle issues. Changed checkstyleCache file to .checkstyleCache. Added to .gitignore.

NIFI-6362 - Changed checkstyleCache file location into target/, and is no longer a hidden file. Removed .checkstyleCache from .gitignore as it is no longer required.

NIFI-6362 - Grep pattern no longer required.

This closes #3522.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2019-06-17 17:19:09 +09:00
Bryan Bende 2102d8a0bd
NIFI-6328 Allowing force-commit when in LOCALLY_MODIFIED_AND_STALE state
NIFI-6328 Addressing review feedback
NIFI-6328 Additional validation on action

This closes #3499
2019-06-13 10:42:13 -04:00
Mark Payne 8bed249f37
NIFI-6353: If LoadBalanceSession determines remote node's queue is full, mark session complete and penalize the Partition, rather than just penalizing the session. This allows the client to move on to another session, which avoids an issue where backpressure on one queue can prevent data from being transferred between nodes in another queue
This closes #3529.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-06-12 11:32:41 -04:00
Mark Payne 2465f67dfe
NIFI-6339: Only call @OnConfigurationRestored method on components when they are created if FlowController has been initialized
This closes #3530.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-06-12 10:56:04 -04:00
Peter Wicks 2d3fce81c1
NIFI-6350 Fixed misleading flow controller startup error message.
This closes #3516.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2019-06-05 19:25:13 -07:00
Mark Payne 60b5c13ce9
NIFI-6341: Addressed bug that can result in a deadlock if Thread 1 adds a Controller Service to a Process Group (which obtains group's write lock and AbstractComponent's Lock) while at the same time Thread 2 attempts to peform validation on the service (which obtains Controller Service's write lock, then AbstractComponent's Lock)
This closes #3510.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-06-04 09:55:26 -04:00
Bryan Bende e7f4af6801 NIFI-6315 Ensuring remote ports get tracked correctly when saving/retrieving versioned flows
NIFI-6315 Renaming public ports during import/change-version when name alreadye exists, and ignorning name change from flow diffs

NIFI-6315 Ensuring public ports get renamed during copy/paste

NIFI-6315 Set max concurrent tasks on Port based on value from VersionedPort

This closes #3488.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2019-06-03 11:25:04 +09:00
Andy LoPresto a9d1dd7b08 NIFI-6337 Resolved groovy unit test execution problems in nifi-properties-loader and fixed failing unit tests.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3508
2019-05-31 08:44:17 -04:00
Matt Gilman 999cfdcac9
NIFI-6302:
- Updating integration tests to verify pruned results.

This closes #3487.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2019-05-29 14:45:00 -07:00
Andy LoPresto e6c843f465
NIFI-6323 Changed URLs for repositories, project description, and mailing lists to use HTTPS.
NIFI-6323 Changed URLs for splunk.artifactoryonline.com to use HTTPS (certificate validity warning in browsers, but command-line connection using openssl s_client is successful).
NIFI-6323 Changed URLs for XMLNS schema locations to use HTTPS (the XMLNS and schema identifier remain http:// because they are not designed to be resolvable).
NIFI-6323 Fixed Maven XML schema descriptor URLs.

This closes #3497
2019-05-29 14:36:40 -04:00
Rob Fellows 0a31c38790
NIFI-6253 - Remove invalid/unnecessary argument in call to nfCanvasUtil.reload
This closes #3491
2019-05-24 11:02:09 -04:00
Rob Fellows 410854f53e
NIFI-6316 - Upgrade jQuery to the latest version (3.4.1).
This closes #3489
2019-05-24 09:52:02 -04:00
thenatog fe68d43e1d
NIFI-6280 - Broke out the matching for /access/knox/** and /access/oidc/** to allow the Jetty security filters to be applied in the /access/oidc/logout and /access/knox/logout cases.
NIFI-6280 - Updated terminology in JwtAuthenticationFilter to authentication instead of authorization. Added stricter token parsing using an explicit regex pattern. Added tests.
NIFI-6280 - Updated terminology from Authorization to Authentication.
NIFI-6280 - Updated the access logout method to use getNiFiUserIdentity(). Updated javascript logout method to handle errors.
NIFI-6280 - Fixing checkstyle issues.
NIFI-6280 - Added some javadoc comments and logging. Renamed some variables for clarity. Fixed handling of exception when JWT does not match expected format.
NIFI-6280 - Cleaned up checkstyle, increased log severity level for logout action, and cleaned up Groovy syntax in test.

This closes #3482.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2019-05-22 17:05:06 -07:00
Koji Kawamura 8a50cb10b2 NIFI-2933 Remote input/output ports at any PG
Specify remote access at port creation.
Incorporated comments, and finished refactoring.
Renamed RootGroupPort to PublicPort.
Fix error message for creating a connection from a child PG having only PublicPorts.
Enhanced ProcessGroup instances rendered in the parent ProcessGroup
Loosen Port move check, allow moving public port between PG.
Show 'Remote NiFi Instance' info on Connection dialogs
Make labels narrative.
'Within Remote Group'.
Fixed DTO (de)serialization.
Return null only if all values are null.

This closes #3351.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2019-05-22 15:44:16 -04:00
Matt Gilman a97766d62f NIFI-6302:
- Ensuring Process Group contents are pruned when appropriate.

This closes #3477.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2019-05-22 14:12:50 -04:00
Mark Payne cbafd8f834 NIFI-5922: Addressed checkstyle violations; added to README Updates to nifi-fn proposal:
- Separated into nifi-fn-core, nifi-fn-bootstrap, nifi-fn-nar, nifi-fn-assembly; fully unpacks nars and runs flows
- Rebased against master and updated to version 1.10.0-SNAPSHOT
- Removed dependency on nifi-framework-core
- Added LICENSE/NOTICE files
2019-05-21 11:19:17 -04:00
Mark Payne f08c2ee43f
NIFI-6285: Addressed issue that resulted in swapped data not being swapped back in if load balancing strategy changed while data was swapped out; added integration tests for swapping. In testing, also encountered an issue with data being swapped out while swap files were being recovered causing the queue size to be wrong and causing errors about not being able to swap data in, because it attempted to swap the data in twice.
This closes #3473.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-05-14 15:31:15 -04:00
Scott Aslan 95fd14866e
[NIFI-6292] update variables slickgrid table row ids
This closes #3469
2019-05-13 09:57:19 -04:00
Mark Bean a2bacde62c
NIFI-6225: bug fix for listing relationships in sorted order
This closes #3459
2019-05-10 15:00:45 -04:00
Jeff Storck 65c41ab917 NIFI-6196: Upgrade Jetty version to 9.4.15.v20190215
Updated NOTICE with current copyright year for Jetty dependencies
Updated Jetty SSLContextFactory usage, invoke setEndpointIdentificationAlgorithm(null) on server SslContextFactory instances
Updated TestInvokeHttpSSL to provide a separate client keystore, rather than reusing the server's keystore
Regenerated nifi-standard-processors keystore and truststore, added client keystore
Updated ITestHandleHttpRequest, TestInvokeHttpSSL, TestInvokeHttpTwoWaySSL, and TestListenHTTP to use a separate client keystore instead of reusing the server's keystore.  Also updated the tests to separately test one-way and two-way SSL

NIFI-6196 - Setting the endpoint identification algorithm to null for sockets to ensure certificates without SANs applied correctly still work. We can change this in a new NiFi release with other potentially breaking changes.

This closes #3426.
2019-05-09 12:05:12 -04:00
thenatog cf6f517250
NIFI-6085 - Added /access/logout endpoint to allow JWT auth tokens to be removed correctly. Added some tests. Found an error in the KeyDAO which did not allow key deletion.
NIFI-6085 - Updated logOut method to use NiFiUserUtils and updated tests.
NIFI-6085 - Added some more integration tests.
NIFI-6085 Suppressed stacktrace when token is used after being invalidated.

This closes #3362.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2019-05-08 17:45:39 -07:00
Jeff Storck 376c344edb NIFI-5574 Removed usage of Paths.get() due to TEST_RES_PATH being resolved to a string from a URI, which results in platform-specific path information (C:\) when tests are run on Windows.
This closes #2988

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2019-05-08 07:38:40 -04:00
Rob Fellows 4484565df1
NIFI-6253 - Fix: UI - Process Group ID display in Operate palette can get out of sync when process groups are removed from different sessions.
NIFI-6253 - simplifying loading of a new process group, handle failure.
NIFI-6253 - Expose nfCanvas.loadProcessGroup functionality through nfCanvasUtils.loadProcessGroup
NIFI-6253 - optional groupId approach to reload

This closes #3458
2019-05-07 15:34:12 -04:00
simonl 0650521eb2
NIFI-6171 always send email scope for OIDC
NIFI-6171 re-added lookupEmail() as fallback
NIFI-6171 additional OIDC scopes via nifi.properties
NIFI-6171 alternative user identification (instead of email)  via nifi.properties
NIFI-6171 changed lookupEmail() so that any configured claim can be fetched fro the UserInfo endpoint

This closes #3398
This closes #2346
2019-04-26 16:40:37 -04:00
Alex Aversa c5c9425989
NIFI-5986 Adding "Stop & Configure" button functionality to Processor Details modal.
NIFI-5986 Refactored Stop & Configure feature to enable bulletin/thread notifications and terminate capability in processor dialogs. Also added feature as a menu item to the canvas context menu.
NIFI-5986 Refactored Stop & Configure feature to enable a status bar in a dialog that conveys bulletin/thread notifications and buttons.
NIFI-5986 Refactored Stop & Configure feature to decouple status bar from modal component, updated styling and revised graph synchronization process.
NIFI-5986 Refactored Stop & Configure feature to improve status bar button hide/show functionality.
NIFI-5986: Rebased and resolved conflicts.
NIFI-5986 - Refactored Stop & Configure statusbar observer, as well as processor dialogs to remove duplicative code.

This closes #3281
2019-04-26 12:07:02 -04:00
Koji Kawamura 31463c5dad
NIFI-6160 Apply config error handling convention
What this PR changed:
- Standardized required name error message to 'The name of the xxxx must
be specified'
- Added nfErrorHandler.handleConfigurationUpdateAjaxError
    - Replaced existing custom error handling for 400 status code
    - Replaced handleAjxError with this where appropriate
- Standardized config error dialog title to 'Configuration Error'
- Close the configuration dialog when configuration is successfully
updated

Above convention applied to following components:
- ProcessGroup
- Input/OutputPort
- RemoteProcessGroup
    - RemoteGroupPort
- Template
- Connection
- ControllerService
- Label
- Processor
- ReportingTask
- NiFi Settings/GENERAL tab
- Variable
- User

NIFI-6170 Provide consistent UX at Connection config
Fixed Label.

Incorporated review comments
Removed unnecessary else block.
Disable canvas when the message pane is shown

This closes #3401
2019-04-24 16:47:11 -04:00
Peter Wicks 2b83b7d9e8
NIFI-6223 Expose Cluster Node Type to Controller Services
NIFI-6223 Enhance Mock Controller Service to allow specifying node type

This closes #3444.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-04-24 13:20:55 -04:00
Andrew Lim e8174fb99f NIFI-6002 Corrected duplicated CSS class and capitalized REST API 2019-04-24 10:05:21 -04:00
Mark Payne 5e2559db42
NIFI-6236: Update VolatileFlowFileRepository to decrement claimant counts when FlowFiles are deleted
This closes #3451.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-04-23 13:10:14 -04:00
Mark Payne 055b3cac54 NIFI-6220: If FlowFile is created by cloning a relationship, do not create an ATTRIBUTES_MODIFIED provenance event for it.
NIFI-6220: Updated test name and fixed typo

This closes #3438.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2019-04-23 17:08:26 +09:00
Koji Kawamura c5d0643d1d NIFI-5792: Remember created versioned flow information
- Before this fix, NiFi loses information about created versioned flow in case of subsequent snapshot creation failure, and NiFi API returned an error response
- This commit makes:
  - The created versioned Flow information is stored even if subsequent snapshot creation fails
  - NiFi API to return a successful 200 response in that case, but return versioned flow status as SYNC_FAILURE with an explanation. NiFi UI shows a popup error dialog with the explanation.
  - Versioned flow status will be LOCALLY_MODIFIED if the latest version is 0.

This closes #3134.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2019-04-22 13:27:40 -04:00
aeaversa 512555dbb0 NIFI-6221 Adding 'Download' and 'View' content buttons to List Queue (#3440)
NIFI-6221 Adding 'Download' and 'View' content icon buttons to slick grid actions column to improve end user navigation.
2019-04-19 16:25:15 -06:00
Andy LoPresto 22563fcb9a
NIFI-6201 Suppressed stacktraces for expected error conditions (connection timeout, missing component bundle, etc.) in application startup when the stacktrace provides no additional helpful data.
NIFI-6201 Fixed existing checkstyle complaint on missing Javadoc element.

NIFI-6201 Restored "connection refused" text to relevant registry sync error message.

NIFI-6201 Simplified error logging for bundle not found issues and removed unused bundle coordinate methods.

This closes #3427.

Signed-off-by: Kevin Doran <kdoran@apache.org>
2019-04-15 16:52:13 -04:00
Mark Payne 7b945182af NIFI-6215: Ensure that when we are swapping FlowFiles back into nifi, that we consider forward slashes or back slashes as the same path separator when determining filename
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3434
2019-04-15 15:54:41 -04:00
Brandon Devries cbc07eb663 NIFI-6157: Allowing configuration of concurrent tasks and transferred FlowFile count to Funnels and LocalPorts
This closes #3412
Signed-off-by: Brandon <devriesb@apache.org>
2019-04-15 14:13:11 +00:00
Rob Fellows b5ff622118
NIFI-4230 - Use a better default location when pasting via keystokes and the original copied item is not fully in view.
- handle multi-select copy/paste better.
- remove dimension info from getOrigin function
- handle pasting better when pasted items should be centered and not offset from the original.
- fixed strict mode and removed console.logs
- Fixing pasting logic when trying to center the items on the canvas.

This closes #3383
2019-04-12 15:27:26 -04:00
Mark Payne 91b2283f49
NIFI-6183: Implemented base class for Framework-level integration tests that is responsible for instantiating the key components for integration tests, including FlowController, WriteAheadFlowFileRepository, WriteAheadProvenanceRepository, FileSystemRepository, FileSystemSwapManager, StandardFlowFileQueue, StandardProcessScheduler, etc. Implemented several integration tests to span different parts of the framework
This closes #3423.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-04-10 16:47:08 -04:00
Mark Payne 0f10010683
NIFI-6199: Fixed problematic unit tests that did not properly wait for async tasks to complete before making assertions
This closes #3421.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-04-09 11:24:58 -04:00
Mark Payne bf7e70e4c3
NIFI-6194: Fixed verification logic to determine whether or not a variable's value can be changed. Previously, the logic had a bug that resulted in a failure if any child group contained a running processor that references an overridden variable and the user attempts to change the overridden variable at the higher level. We should not include any components of descendent groups if the descendent group overrides the variable.
This closes #3420.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-04-09 11:15:45 -04:00
Mark Payne 38db4e97cd
NIFI-5464: Consider ports invalid unless they have both incoming and outgoing connections
This closes #3417.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-04-09 10:19:24 -04:00
Rob Fellows fb9f2af04a
NIFI-6136 - fix: UI - User and Template bug when page popped out
NIFI-6136 - return the appropriate value (false) when verifyDisconnectedCluster is called from a non-clustered environment.

This closes #3390
2019-04-08 15:45:19 -04:00
Alex Aversa 96279415a7 NIFI-6188 - Added 1s delay on Search control autocomplete query event to prevent premature results retrieval
This closes #3408.
Signed-off-by: Brandon Devries <devriesb@apache.org>
2019-04-05 12:32:40 -04:00
Rob Fellows e374d2cc30
NIFI-6140 - Incorrect title on the read only version of the Reporting Task detail dialog
This closes #3385
2019-04-04 13:40:14 -04:00
Rob Fellows cb4fdc2ae1
NIFI-6126 - throttle requests to refresh the canvas to prevent too many server requests when holding down CTRL-R
* Added lodash utility library to leverage its throttle capability (and many other useful functions in the future).
* Made lodash available in all JSP's so it could be imported into nf-common (or any component for that matter).
* Added a throttle function to nf-common that just wraps _.throttle

This closes #3393
2019-04-04 09:51:41 -04:00
Mark Payne 76392ee862
NIFI-6155: Ensure that any task submitted to FlowEngine catches Throwable so that the task doesn't die just die silently in the case of an unexpected error/exception
This closes #3395.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-03-29 10:09:35 -04:00
Mark Payne d3d43262db
NIFI-6150: Moved logic of decrementing content claims from ProcessSession.commit to the FlowFile Repository. Also updated load-balancing protocol and queue drop logic to no longer decrement content claims since the flowfile repo will handle the logic. This allows us ts to ensure that the claimant counts are decremented only if the repo is successfully updated and still before checking if the claim is still in use or not.
This closes #3391.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-03-27 10:22:01 -04:00