Commit Graph

68 Commits

Author SHA1 Message Date
Oleg Zhurakousky 1bf10944ea NIFI-2366 - Fixed ID generation semantics in clustered environment
- added SnippetUtilsTest
- renamed TypeOneUUIDGenerator to ComponentIdGenerator

- changed lsb part of ComponentIdGenerator back to long
- Fixed 'isCopy' condition for clustered environments

This closes #718.
2016-07-31 15:24:02 -04:00
Matt Gilman da238b16ef NIFI-2122: - Merging responses for the current user and the flow configuration. - Returning whether NiFi is configured with a policy based authorizer in the flow configuration. - Only showing the users and policy icons when configured with a policy based authorizer. - Failing faster when invoking the users or policies endpoint when not configured with a configurable authorizer.
This closes #736

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-28 16:51:40 -04:00
Matt Gilman ae344806c0 NIFI-2224: - Ensuring the template form is reset when the upload template dialog is closed. NIFI-2175: - Not submitting the template form is a template isn't choosen. NIFI-2176: - Ensuring a template is specified during creation. NIFI-2223: - Ensuring templates with the same name cannot be added. NIFI-2296: - Updating the tooltip for the upload template browse button.
- Cleaning up un-used parameters to REST endpoints.

This closes #725

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-27 17:38:12 -04:00
Matt Gilman 4a4d60e6af NIFI-2307: - Enforcing connection permissions based on the source and destination comonent. - Removing connection specific access policies. NIFI-2265: - Filtering out sensitive details in component status and status history when appropriate. NIFI-1800: - Adding parent process group id to the Controller Services table. NIFI-2077: - Removing some old un-used icons following the UI refresh. NIFI-2242: - Requiring write permissions for all components in a selection. NIFI-2080: - Updating style of the name in the selection context to handle scroll bars and use available width. NIFI-2331: - Addressing issue when removing a user/group which was causing the tenant policy to be removed. NIFI-2335: - Ensuring the flow is saved after starting/stopping a process group. NIFI-2235: - Ensuring we use consistent conditions between the context menu and the operate palette.
- Allowing users with read only access to the tenants page.
- Fixing current user integration test.
- Ensuring schedule methods are locked appropriately.
- Addressing comments from PR.

This closes #698

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-21 23:52:01 -04:00
Oleg Zhurakousky 52a961873b NIFI-826 This closes #617. Added deterministic template support 2016-07-15 20:41:18 -04:00
Matt Gilman 5cd5a4ce78 NIFI-2260:
- Addressing issue searching provenance on a specific node.
- Fixing issues viewing content and replaying events.

This closes #658.
2016-07-15 12:14:24 -04:00
Matt Gilman 3373e18158 NIFI-1947 NIFI-2082:
- Making it more obviously when a nodes cluster state changes.
- Showing which node is the cluster coordinator.

This closes #651
2016-07-14 11:31:48 -04:00
Matt Gilman 6e5e4cf52b NIFI-2204:
- Move bulletins out of the controller status endpoint.
NIFI-2238:
- Ensuring the controller bulletins are rendered on screen.
NIFI-2246:
- Ensuring the correct number of bulletins are returned when clustered.
2016-07-14 08:50:26 -04:00
Matt Gilman 9e2f52c8b5 NIFI-2249:
- Making the URI accessibility outside of the component.
2016-07-13 09:01:05 -04:00
Matt Gilman e0c96794fa NIFI-2095:
- Adding a page for managing users and groups.
- Adding a page for managing access policies.
- Renaming accessPolicy in entity to permissions to avoid confusion with the accessPolicy model.
- Adding an Authorizable for access policies.
- Refactoring access policies endpoints.
NIFI-2022:
- Implementing site to site authorizations.
2016-07-12 15:45:13 -04:00
Matt Gilman ce5330330a NIFI-1781:
- Updating UI according to permissions through out the application.
- Shuffling provenance events, template, and cluster search REST APIs according to resources being authorized.
- Moving template upload controls.
- Removing username where appropriate.
- Addressing issues when authorizing flow configuration actions.
- Code clean up.
2016-07-01 15:10:27 -04:00
Mark Payne 8eb0a3882c NIFI-2150:
Cleanse more values from templates that are not necessary. Additionally, updated javadocs in ProcessorConfigDTO to provide further explanation of the getAutoTerminatedRelationships() method, since this was confusing
Removed additional unused fields from templates
Populating snippet response using actual components rather than the snippet contents.
This closes #593
2016-07-01 14:21:02 -04:00
Mark Payne ae9e2fdf0b NIFI-2123: Add authorization of provenance events; refactor core classes so that Authorizable is located within nifi-api. This closes #592 2016-06-30 07:57:17 -04:00
Jeff Storck 41f3253445 NIFI-1952 Update to revision-locking for users and groups
Adding user and group summary objects (TenantEntity)
Fixed ComponentEntity JSON mapping issues when the id field is null
Removing unecessary revision checking.
Fixing error message when checking user, group, and policy revision.
This closes #589
2016-06-28 15:38:01 -04:00
Jeff Storck 64719b6f9b NIFI-1952 Updated StandardPolicyBasedAuthorizerDAO to throw ResourceNotFoundExceptions when user/group/policy not found
Added spec for StandardPolicyBasedAuthorizerDAO
Added exception mapper for AuthorizationAccessException, added mapper to nifi-web-api-context.xml
Added rest endpoints to get all users and user groups
Merged UsersResource and UserGroupsResource into TenantsResource
This closes #582
2016-06-26 22:23:25 -04:00
Matt Gilman 82268afb0d NIFI-1554: - Updating cluster detection to support access through the flow resource. - Using the correct resource when authorizing a root group port during site to site.
This closes #579.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-06-24 16:59:06 -04:00
Matt Gilman f0811ca45a NIFI-1554:
- Addressing access controls for the Controller resource.
- Addressing access controls for RAW site to site clients.
- Addressing access controls for downloading content (from provenance and queue).
- Addressing access controls for accessing queues.
- Addressing access controls for cluster endpoints.
- Addressing access controls for counter endpoints.
- Removing redundant authorization calls.
NIFI-2044:
- Requiring revision when creating components.
- Requiring component creation over POST requests.
NIFI-1901
- Continuing to restore access control tests.
- Converting access control tests to itegration tests.
- Restoring contrib check to travis build.
- This closes #567
2016-06-23 17:09:54 -04:00
Jeff Storck f47be77b6a NIFI-1952 Create REST endpoints for user/group/policy management
created REST Resources for users, groups, and access policies
added Authorizables for users, groups, and access policies
added methods to DtoFactory and EntityFactory to create objects for users, groups, and access policies
extracted anonymous AuthorizableLookup impl in StandardNiFiServiceFacade.java to a protected class to make the lookup call mockable in tests
added methods to manage users/groups/access policies to StandardNiFiServiceFacade
added StandardNiFiServiceFacadeSpec to unit-test management of users/groups/access policies
added implementations for UserDAO, GroupDAO, AccessPolicyDAO.
added spring config for user/group/policy resources and daos
Updated user/group/policy creation via REST resources, no longer requires the use of the revision manager
updated StandardNiFiServiceFacadeSpec based on user/group/policy creation changes
condensed user/group/policy DAOs to a single DAO (StandardPolicyBasedAuthorizerDAO)
fixed spring config of user/group/policy REST resources
Updated to return ComponentEntity objects instead of just their IDs
mid-progress on updating tests
updated code and tests to return component entities from REST endpoints for users, groups, policies
This closes #526
2016-06-22 10:12:41 -04:00
Matt Gilman 0b437e09a7 NIFI-2007:
- Restoring bulletin functionality.
- Ensuring appropriate merging of bulletins in clustered responses.
2016-06-17 13:20:20 -04:00
Matt Gilman fbd299e885 NIFI-1901: - Building component based access control tests for Connections, Funnels, Labels, Input Ports, Output Ports, Processors, and Process Groups. - Restoring Access Token Endpoint tests.
This closes #518

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-06-17 10:16:09 -04:00
Koji Kawamura c120c4982d NIFI-1857: HTTPS Site-to-Site
- Enable HTTP(S) for Site-to-Site communication
- Support HTTP Proxy in the middle of local and remote NiFi
- Support BASIC and DIGEST auth with Proxy Server
- Provide 2-phase style commit same as existing socket version
- [WIP] Test with the latest cluster env (without NCM) hasn't tested yet

- Fixed Buffer handling issues at asyc http client POST
- Fixed JS error when applying Remote Process Group Port setting from UI
- Use compression setting from UI
- Removed already finished TODO comments

- Added additional buffer draining code after receiving EOF
- Added inspection and assert code to make sure Site-to-Site client has
  written data fully to output
stream
- Changed default nifi.remote.input.secure from true to false

This closes #497.
2016-06-09 15:09:57 -04:00
Mark Payne df0e4e7960 NIFI-1897:
- Refactoring to allow requests to be replicated from a node to other nodes
- Renaming cluster node connection/read timeout properties.
- Renaming NCM DN to Cluster Coordinator DN.
- Fixing default values in properties.
- Starting to fix Spring context to load correctly in standalone mode.
- Using the cluster protocol to handle connection failures instead of heartbeats.
- Ensured replicate call is returned from ControllerResource.
- Ensure the appropriate classloader when serializing templates.
- Handling when the flow contents are null.
- This closes #488
2016-06-03 15:29:16 -04:00
Matt Gilman f0f74fe404 NIFI-1928:
- Fixing UI action for bulk delete.
- Removing registration form.
- Fixing default visibility of the anonymous user warning.
2016-06-02 11:56:10 -04:00
Jeff Storck 893daf567d NIFI-1908 Added encoding-version attribute to TemplateDTO
added scaling of templates upon instantiation for placement on the canvas
added template-0.7.0.xml for live-testing the import of templates

Fixing issue with potentially uninitialized RemoteGroupPorts in copySnippet.

This closes #471
2016-05-30 14:40:52 -04:00
Matt Gilman a0ff2f7a9f NIFI-1554:
- Refactoring Cluster endpoints.
- Updating frontend to accomodate clustering endpoints.
- Remove the 'Make Primary' action.
- This closes #470
2016-05-26 14:28:24 -04:00
Matt Gilman 9152a9fdbb NIFI-1800:
- UI style updates to make the components stand out better.
- Reusing controller service table in different contexts (controller, process group, etc).
- This closes #469
2016-05-26 14:15:56 -04:00
Matt Gilman 4dd50c80a4 NIFI-1781:
- Including access policies in the breadcrumb's trail.
- Updating toolbox according to group access policies.
- Updating actions in palette based on selection access policies.
NIFI-1554:
- Introducing authorization during two phase commit.
- Introducing snippet authorization according to the encapsulated components and the action performed.
- This closes #461
2016-05-23 14:15:32 -04:00
Matt Gilman 8bb56fca68 NIFI-1554:
- Introducing ControllerService and ReportingTask authorization and revision.
- Removing the revision where appropriate.
NIFI-1883
- Restoring referencing components automatically.
- This closes #448
2016-05-17 10:32:55 -04:00
Mark Payne 270944ec69 NIFI-1801: Scope Templates to Process Groups. This closes #446. 2016-05-16 16:12:43 -04:00
Matt Gilman 3cc16d35ed NIFI-1742:
- Addressing issues when creating a new inline controller service.
- Ensuring controller service referencing components are updated.
- Including revisions and status with each component.
- Dynamically updating component and authorization states.
- This closes #435
2016-05-12 12:41:11 -04:00
Matt Gilman 9db1def6c6 NIFI-1781:
- Incorporating updated styles to reflect component level authorization.
- Updating canvas for new look and feel.
- This closes #417
2016-05-06 15:50:06 -04:00
Mark Payne 04c41c0654 NIFI-1678:
- Started refactoring heartbeating mechanism, using a new package: org.apache.nifi.cluster.coordination
- Added configuration options for ZooKeeper username & password for heartbeat management. Also addressed issue where nodes that were previously disconnected were asked to disconnect upon restart
- Ensure that ACL is set properly when creating heartbeat node. Removed unused ControllerStartupFailureMessage.java
- Changed ZooKeeper ACL's so that container nodes that would not be sensitive are wide open and removed the usage of username & password when communicating with ZooKeeper. This was done specifically because username/password combination is considered a 'testing' feature that should not be used in production and is not supported by Apache Curator
- Refactored CuratorHeartbeatMonitor into an abstract heartbeat monitor that is responsible for processing heartbeats and CuratorHeartbeatMonitor that is responsible for retrieving heartbeat information
- Refactored so that heartbeats are sent to Cluster Coordinator directly instead of to ZooKeeper. ZooKeeper is used to know which node is the cluster coordinator but heartbeats to the Cluster Coordinator provide additional information about the nodes.
- Started refactoring heartbeating mechanism, using a new package: org.apache.nifi.cluster.coordination
- Added configuration options for ZooKeeper username & password for heartbeat management. Also addressed issue where nodes that were previously disconnected were asked to disconnect upon restart
- Changed ZooKeeper ACL's so that container nodes that would not be sensitive are wide open and removed the usage of username & password when communicating with ZooKeeper. This was done specifically because username/password combination is considered a 'testing' feature that should not be used in production and is not supported by Apache Curator

NIFI-1727:
- Refactored logic for merging HTTP Requests that are federated across cluster

NIFI-1745:
- Refactoring how HTTP Requests are replicated to nodes
- Bug fixes and continuing to work on replication refactoring. Still need to handle cluster locking and revisions
- Begin work on RevisionManager
- Resolved some issues that resulted from rebase
- Fixed URIs to align with new URI's that will be used in 1.0.0
- This closes #413
2016-05-06 15:23:12 -04:00
Matt Gilman ff98d823e2 NIFI-1554:
- Populating component entities in the REST API to decouple key fields from the configuration DTOs.
- Added initial support for components in UI when access isn't allowed. Formal styling to come later.
2016-04-29 14:49:14 -04:00
Matt Gilman add298168d NIFI-1554:
- Introducing new REST endpoints to align with the authorizable resources.
- Additionally changes to support the new endpoints.
- Addressing comments in PR.
- This closes #374.
2016-04-21 17:29:58 -04:00
Matt Gilman 153f63ef43 NIFI-1551:
- Removing the AuthorityProvider.
- Refactoring REST API in preparation for introduction of the Authorizer.
- Updating UI accordingly.
- Removing unneeded properties from nifi.properties.
- Addressing comments from PR.
- This closes #359.
2016-04-15 16:03:00 -04:00
Matt Gilman 3f4ac3156c Revert "NIFI-1551:"
This reverts commit c4d06f203d. Accidently merged wrong branch.
2016-04-07 16:22:35 -04:00
Matt Gilman c4d06f203d NIFI-1551:
- Starting to remove the AuthorityProvider.
- This closes #330
2016-04-07 16:18:36 -04:00
Mark Payne 0d3bd2c401 NIFI-1563: - Federate requests and merge responses from nodes instead of storing bulletins and stats at NCM - Updating UI to support restructured status history DTO. - Return 'Insufficient History' message if aggregate stats don't have enough data points, even if all nodes do (which can be the case if the node performing the aggregation has a different value for the 'nifi.components.status.snapshot.frequency' property than the other nodes) - Bug fixes; code cleanup; replicate requests to bulletin board endpoint - Refactored the <Component>StatusDTO objects into <Component>StatusDTO, <Component>StatusSnapshotDTO, Node<Component>StatusSnapshotDTO objects - Introducing endpoints for accessing individual component status. - Wiring up new endpoints and updated core. - Code clean up. - Starting to handling status merging of individual components. - Nodewise breakdown has been added to Processors but the remaining components still need to be updated. - Refactor so that System Diagnostics requests are replicated to nodes instead of the information being pulled from Heartbeats - Replicate request for counters instead of pulling them from heartbeats - Removed the getCounters / setCounters method from HeartbeatPayload - Implementing component specific endpoints. - Removing unused endpoints. - Supporting nodewise breakdown for system diagnostics and counters. - Updating DTOs to use more consistent naming. - Code clean up. - Addressing contrib issues. - Removed ProcessGroupStatus from HeartbeatPayload - Removing nodewise from the system diagnostics endpoint. Had included it for testing that option but did not intend for it to be committed. - Addressing comments in PR #294. - This closes #294
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-04-04 11:42:39 -04:00
Matt Gilman 2de7f3f884 Updating versions to 1.0.0-SNAPSHOT. 2016-04-04 11:36:20 -04:00
Aldrin Piri e977729b56 NIFI-1634-rc2 prepare for next development iteration 2016-03-23 18:56:34 -04:00
Aldrin Piri 0b9bd20d31 NIFI-1634-rc2 prepare release nifi-0.6.0-RC2 2016-03-23 18:56:22 -04:00
Tony Kurc ad73b5c9d9 NIFI-1379-RC3 prepare for next development iteration 2016-02-12 17:28:10 -05:00
Tony Kurc 8309dba80b NIFI-1379-RC3 prepare release nifi-0.5.0-RC3 2016-02-12 17:27:59 -05:00
Matt Gilman f4487dd5f6 NIFI-1492: - Limiting the amount of state entries returned to a client. - Code clean up.
Signed-off-by: joewitt <joewitt@apache.org>
2016-02-09 20:48:56 -05:00
Tony Kurc 303f8eabf1 NIFI-1379: Move to 0.5.0-SNAPSHOT, add tkurc code signing key to keys 2016-02-06 08:49:48 -05:00
Mark Payne f6ec437bc7 Merge branch 'master' into NIFI-259 2016-01-21 09:32:02 -05:00
Matt Gilman 0d7edcb3ac NIFI-108:
- Removing sort from UI.
- Addressing issues with listing and flowfile retrieval when clustered.
- Making the context menu item available when source and destination are still running.
- Adding a refresh button to the queue listing table.
- Fixing the flowfile summary sorting in the cluster manager.
- Adding a message when the source or destination of a connection is actively running.
- Updating the documentation regarding queue interaction.
- Updating the error message when a flowfile is no longer in the active queue.
- Updated queue listing to allow listing to be done while source and destination are running but not sort or have ability to search
- Added heartbeat when we finish clearing queue
- Addressing comments from review.
2016-01-21 08:13:32 -05:00
Matt Gilman d05314c54b NIFI-259:
- Initial implementation of viewing and clearing state for a processor.
2016-01-13 13:35:24 -05:00
Matt Gilman bbd35a0258 NIFI-259:
- Adding endpoints for getting and clearing component state.
2016-01-12 10:43:30 -05:00
Mark Payne b330fd1601 NIFI-108: Built out 'skeleton' of the requests so that web tier can be written against it
NIFI-108: Implementing ability to list FlowFiles in a queue

NIFI-108:
- Starting to add support for endpoints that will listing flowfiles in a queue.

NIFI-108: Added merging of response for listing of flowfiles in cluster manager

NIFI-108:
- Starting to add support for endpoints that will listing flowfiles in a queue.

NIFI-108:
- Starting to add support for endpoints that will listing flowfiles in a queue.

NIFI-108:
- Adding checkstyle issues.

NIFI-108: Add clusterNodeId to FlowFileSummaryDTO

NIFI-108: Added unit tests; added verifyCanList method to queue; fixed bugs

NIFI-108:
- Adding compilation error for IOException from getFlowFile().
- Code clean up.
- Javadocs.

NIFI-108:
- Verifying two phase commit for queue listing.
- Fixing checkstyle.
- Ensuring drop and listing requests are merged when created when clustered.

NIFI-108:
- Adding initial listing capabilities.
- Passing through the sort column and direction.

NIFI-108:
- Removing Delete FlowFile button.
- Ensuring sort flags are being passed correctly.
- Setting column widths.
- Also including the cluster node address in the flowfile summaries.

NIFI-108:
- Including queue size statistics in listing request.
- Showing connection name.

NIFI-108:
- Including queue size statistics in listing request.
- Ensuring verifyCanList runs when appropriate.

NIFI-108:
- Adding initial support for viewing flowfile details dialog.
- Adding initial support for click to content.

NIFI-108:
- Allowing the flowfile details dialog to be draggable.

NIFI-108:
- Only showing the flowfile listing table when the listing is successful and the listing is not empty.

NIFI-108:
- Reseting the queue stats when closing the listing table.

NIFI-108: Implemented sorting when performing listing of FlowFiles

NIFI-108: Fixed bug that caused the listFlowFiles operation to wait on a readLock before returning and performing work asynchronously; fixed bug in Write-Ahead FlowFile Repository that caused ContentClaims to be queued up for destruction instead of ResourceClaims - this caused millions of ContentClaims to be queued up instead of a single ResourceClaim in some tests

NIFI-108:
- Ensured the column sort indicator is reset when a new listing is opened.
- Removing unused import.

NIFI-108:
- Addressed issues found during the review.
2016-01-06 16:20:23 -05:00