- Added NodeTypeProvider to expose flowController's isClustered and
isPrimaryNode so that processor can know if it's running on a cluster
and if it's a primary node.
- Added mechanism to simulate clustered or not, and primary or not, for testing
This closes#657
Signed-off-by: jpercivall <joepercivall@yahoo.com>
- Move bulletins out of the controller status endpoint.
NIFI-2238:
- Ensuring the controller bulletins are rendered on screen.
NIFI-2246:
- Ensuring the correct number of bulletins are returned when clustered.
- Adding a page for managing users and groups.
- Adding a page for managing access policies.
- Renaming accessPolicy in entity to permissions to avoid confusion with the accessPolicy model.
- Adding an Authorizable for access policies.
- Refactoring access policies endpoints.
NIFI-2022:
- Implementing site to site authorizations.
NIFI-2201 Add support for seeding cluster nodes in authorizations.xml
- Passing client address along in user context on authorization requests
- This closes#628
- Making FileAuthorizer not update the resource or action when updating an AccessPolicy
- Adding corresponding READ policies during initial seeding and legacy conversions
- Adding checks to FileAuthorizer to ensure only one policy per resource-action
- Removing merging of policies on legacy conversion since we have one action per policy now
- This closes#608
- Updating UI according to permissions through out the application.
- Shuffling provenance events, template, and cluster search REST APIs according to resources being authorized.
- Moving template upload controls.
- Removing username where appropriate.
- Addressing issues when authorizing flow configuration actions.
- Code clean up.
- Addressing access controls for the Controller resource.
- Addressing access controls for RAW site to site clients.
- Addressing access controls for downloading content (from provenance and queue).
- Addressing access controls for accessing queues.
- Addressing access controls for cluster endpoints.
- Addressing access controls for counter endpoints.
- Removing redundant authorization calls.
NIFI-2044:
- Requiring revision when creating components.
- Requiring component creation over POST requests.
NIFI-1901
- Continuing to restore access control tests.
- Converting access control tests to itegration tests.
- Restoring contrib check to travis build.
- This closes#567
created REST Resources for users, groups, and access policies
added Authorizables for users, groups, and access policies
added methods to DtoFactory and EntityFactory to create objects for users, groups, and access policies
extracted anonymous AuthorizableLookup impl in StandardNiFiServiceFacade.java to a protected class to make the lookup call mockable in tests
added methods to manage users/groups/access policies to StandardNiFiServiceFacade
added StandardNiFiServiceFacadeSpec to unit-test management of users/groups/access policies
added implementations for UserDAO, GroupDAO, AccessPolicyDAO.
added spring config for user/group/policy resources and daos
Updated user/group/policy creation via REST resources, no longer requires the use of the revision manager
updated StandardNiFiServiceFacadeSpec based on user/group/policy creation changes
condensed user/group/policy DAOs to a single DAO (StandardPolicyBasedAuthorizerDAO)
fixed spring config of user/group/policy REST resources
Updated to return ComponentEntity objects instead of just their IDs
mid-progress on updating tests
updated code and tests to return component entities from REST endpoints for users, groups, policies
This closes#526
Expose BackPressureObjectThreshold and BackPressureDataSizeThreshold to
ConnectionStatus
This closes#377.
Signed-off-by: Aldrin Piri <aldrin@apache.org>
- Removing Resources class from file authorizer and updating ResourceType enum
- Updating ResourceFactory to be in sync with ResourceType enum and adding additional required permissions to the auto-conversion
- Adding root process group to the seeding of the initial admin
- Improvement so that users that are already part of a read-write policy, won't end up in a read policy for the same resource
- Removing rootGroupId from authorization context and auto-detecting it from the flow provided through nifi.properties
- This closes#507
- Implementing CRUD operations and unit tests for Users
- Implementing CRUD operations and unit tests for Groups
- Implementing CRUD operations and unit tests for AccessPolicies
- Adding support for seeding with an initial admin user
- Fixing delete for user and group so it removes references from policies
- Adding example to authorizations.xml
- Adding back the old users schema in preparation for auto-converting to the new format, and providing the AuthorizationConfigurationContext with access to the root process group id
- Refactoring some of the FileAuthorizer to ensure thread safety
- Adding /groups to policies created for initial admin
- This closes#473
- Removing deprecated NiFiWebContext and related classes.
- Adding authorization to Custom UIs.
- Fixing issue when creating ControllerService inline.
- Addressing contentType issue when attempting to clear component state.
- This closes#489
- Refactoring to allow requests to be replicated from a node to other nodes
- Renaming cluster node connection/read timeout properties.
- Renaming NCM DN to Cluster Coordinator DN.
- Fixing default values in properties.
- Starting to fix Spring context to load correctly in standalone mode.
- Using the cluster protocol to handle connection failures instead of heartbeats.
- Ensured replicate call is returned from ControllerResource.
- Ensure the appropriate classloader when serializing templates.
- Handling when the flow contents are null.
- This closes#488
- Defining API for Users, Groups, and Policies
- Updating hashCode and equals methods
- Documenting that returned sets are unmodifiable
- Adding builders and unit tests
- Refactoring update methods in MutableAuthorizer to not take a string id
- Refactoring builders to use constructors for seeding
- Fixing toString() methods and cleaning up constructors that don't need to worry about null sets
- Changing MutableAuthorizer to an abstract class and providing implementation of authorize
- This closes#452
- Including access policies in the breadcrumb's trail.
- Updating toolbox according to group access policies.
- Updating actions in palette based on selection access policies.
NIFI-1554:
- Introducing authorization during two phase commit.
- Introducing snippet authorization according to the encapsulated components and the action performed.
- This closes#461
- Started refactoring heartbeating mechanism, using a new package: org.apache.nifi.cluster.coordination
- Added configuration options for ZooKeeper username & password for heartbeat management. Also addressed issue where nodes that were previously disconnected were asked to disconnect upon restart
- Ensure that ACL is set properly when creating heartbeat node. Removed unused ControllerStartupFailureMessage.java
- Changed ZooKeeper ACL's so that container nodes that would not be sensitive are wide open and removed the usage of username & password when communicating with ZooKeeper. This was done specifically because username/password combination is considered a 'testing' feature that should not be used in production and is not supported by Apache Curator
- Refactored CuratorHeartbeatMonitor into an abstract heartbeat monitor that is responsible for processing heartbeats and CuratorHeartbeatMonitor that is responsible for retrieving heartbeat information
- Refactored so that heartbeats are sent to Cluster Coordinator directly instead of to ZooKeeper. ZooKeeper is used to know which node is the cluster coordinator but heartbeats to the Cluster Coordinator provide additional information about the nodes.
- Started refactoring heartbeating mechanism, using a new package: org.apache.nifi.cluster.coordination
- Added configuration options for ZooKeeper username & password for heartbeat management. Also addressed issue where nodes that were previously disconnected were asked to disconnect upon restart
- Changed ZooKeeper ACL's so that container nodes that would not be sensitive are wide open and removed the usage of username & password when communicating with ZooKeeper. This was done specifically because username/password combination is considered a 'testing' feature that should not be used in production and is not supported by Apache Curator
NIFI-1727:
- Refactored logic for merging HTTP Requests that are federated across cluster
NIFI-1745:
- Refactoring how HTTP Requests are replicated to nodes
- Bug fixes and continuing to work on replication refactoring. Still need to handle cluster locking and revisions
- Begin work on RevisionManager
- Resolved some issues that resulted from rebase
- Fixed URIs to align with new URI's that will be used in 1.0.0
- This closes#413
- Populating component entities in the REST API to decouple key fields from the configuration DTOs.
- Added initial support for components in UI when access isn't allowed. Formal styling to come later.
- Removing the AuthorityProvider.
- Refactoring REST API in preparation for introduction of the Authorizer.
- Updating UI accordingly.
- Removing unneeded properties from nifi.properties.
- Addressing comments from PR.
- This closes#359.
* Simplified and cleaned StandardProcessScheduler.start/stopProcessor methods
* Added stop/start operations to ProcessorNode.
* Removed unnecessary synchronization blocks related to ScheduledState in favor of enforcing order and idempotency via CAS operations. Those synchronization blocks were causing intermittent deadlocks whenever @OnScheduled blocks indefinitely.
* Added support for stopping the service when @OnScheduled operation hangs.
* Fixed the order of life-cycle operation invocation ensuring that each operation can *only* be invoked at the appropriate time
* Removed unnecessary locks from StandardProcessNode since Atomic variables are used.
* Removed calls to @OnStopped from ContinuallyRunningProcessTask while ensuring that procesor's full shut down in implementation of StandardProcessorNode.stop() method.
* Removed dead code
* Added comprehensive tests suite that covers 95% of Processor's life-cycle operations within the scope of FlowController, StandardProcesssScheduler and StandardProcessNode
* Improved and added javadocs on covered operations with detailed explanations.
Add code to ContentViewerController to strip content type of any trailing parameters and lowercase the type and subtype.
Added function to ViewableContent to enable retrieving the original value of the content type if needed.
This closes#242
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
- Removing sort from UI.
- Addressing issues with listing and flowfile retrieval when clustered.
- Making the context menu item available when source and destination are still running.
- Adding a refresh button to the queue listing table.
- Fixing the flowfile summary sorting in the cluster manager.
- Adding a message when the source or destination of a connection is actively running.
- Updating the documentation regarding queue interaction.
- Updating the error message when a flowfile is no longer in the active queue.
- Updated queue listing to allow listing to be done while source and destination are running but not sort or have ability to search
- Added heartbeat when we finish clearing queue
- Addressing comments from review.
NIFI-108: Implementing ability to list FlowFiles in a queue
NIFI-108:
- Starting to add support for endpoints that will listing flowfiles in a queue.
NIFI-108: Added merging of response for listing of flowfiles in cluster manager
NIFI-108:
- Starting to add support for endpoints that will listing flowfiles in a queue.
NIFI-108:
- Starting to add support for endpoints that will listing flowfiles in a queue.
NIFI-108:
- Adding checkstyle issues.
NIFI-108: Add clusterNodeId to FlowFileSummaryDTO
NIFI-108: Added unit tests; added verifyCanList method to queue; fixed bugs
NIFI-108:
- Adding compilation error for IOException from getFlowFile().
- Code clean up.
- Javadocs.
NIFI-108:
- Verifying two phase commit for queue listing.
- Fixing checkstyle.
- Ensuring drop and listing requests are merged when created when clustered.
NIFI-108:
- Adding initial listing capabilities.
- Passing through the sort column and direction.
NIFI-108:
- Removing Delete FlowFile button.
- Ensuring sort flags are being passed correctly.
- Setting column widths.
- Also including the cluster node address in the flowfile summaries.
NIFI-108:
- Including queue size statistics in listing request.
- Showing connection name.
NIFI-108:
- Including queue size statistics in listing request.
- Ensuring verifyCanList runs when appropriate.
NIFI-108:
- Adding initial support for viewing flowfile details dialog.
- Adding initial support for click to content.
NIFI-108:
- Allowing the flowfile details dialog to be draggable.
NIFI-108:
- Only showing the flowfile listing table when the listing is successful and the listing is not empty.
NIFI-108:
- Reseting the queue stats when closing the listing table.
NIFI-108: Implemented sorting when performing listing of FlowFiles
NIFI-108: Fixed bug that caused the listFlowFiles operation to wait on a readLock before returning and performing work asynchronously; fixed bug in Write-Ahead FlowFile Repository that caused ContentClaims to be queued up for destruction instead of ResourceClaims - this caused millions of ContentClaims to be queued up instead of a single ResourceClaim in some tests
NIFI-108:
- Ensured the column sort indicator is reset when a new listing is opened.
- Removing unused import.
NIFI-108:
- Addressed issues found during the review.
- Refactoring web security to use Spring Security Java Configuration.
- Introducing security in Web UI in order to get JWT.
NIFI-655:
- Setting up the resources (js/css) for the login page.
NIFI-655:
- Adding support for configuring anonymous roles.
- Addressing checkstyle violations.
NIFI-655:
- Moving to token api to web-api.
- Creating an LoginProvider API for user/pass based authentication.
- Creating a module for funneling access to the authorized useres.
NIFI-655:
- Moving away from usage of DN to identity throughout the application (from the user db to the authorization provider).
- Updating the authorized users schema to support login users.
- Creating an extension point for authentication of users based on username/password.
NIFI-655:
- Creating an endpoint for returning the identity of the current user.
- Updating the LoginAuthenticationFilter.
NIFI-655:
- Moving NiFi registration to the login page.
- Running the authentication filters in a different order to ensure we can disambiguate each case.
- Starting to layout each case... Forbidden, Login, Create User, Create NiFi Account.
NIFI-655:
- Addressing checkstyle issues.
NIFI-655:
- Making nf-storage available in the login page.
- Requiring use of local storage.
- Ignoring security for GET requests when obtaining the login configuration.
NIFI-655:
- Adding a new endpoint to obtain the status of a user registration.
- Updated the login page loading to ensure all possible states work.
NIFI-655:
- Ensuring we know the necessary state before we attempt to render the login page.
- Building the proxy chain in the JWT authentication filter.
- Only rendering the login when appropriate.
NIFI-655:
- Starting to style the login page.
- Added simple 'login' support by identifying username/password. Issuing JWT token coming...
- Added logout support
- Rendering the username when appropriate.
NIFI-655:
- Extracting certificate validation into a utility class.
- Fixing checkstyle issues.
- Cleaning up the web security context.
- Removing proxy chain checking where possible.
NIFI-655:
- Starting to add support for registration.
- Creating registration form.
NIFI-655:
- Starting to implement the JWT service.
- Parsing JWT on client side in order to render who the user currently is when logged in.
NIFI-655:
- Allowing the user to link back to the log in page from the new account page.
- Renaming DN to identity where possible.
NIFI-655:
- Fixing checkstyle issues.
NIFI-655:
- Adding more/better support for logging out.
NIFI-655:
- Fixing checkstyle issues.
NIFI-655:
- Adding a few new exceptions for the login identity provider.
NIFI-655:
- Disabling log in by default initially.
- Restoring authorization service unit test.
NIFI-655:
- Fixing checkstyle issues.
NIFI-655:
- Updating packages for log in filters.
- Handling new registration exceptions.
- Code clean up.
NIFI-655:
- Removing registration support.
- Removing file based implementation.
NIFI-655:
- Removing file based implementation.
NIFI-655:
- Removing unused spring configuration files.
NIFI-655:
- Making the auto wiring more explicit.
NIFI-655:
- Removing unused dependencies.
NIFI-655:
- Removing unused filter.
NIFI-655:
- Updating the login API authenticate method to use a richer set of exceptions.
- UI code clean.
NIFI-655:
- Ensuring the login identity provider is able to switch context classloaders via the standard NAR mechanisms.
NIFI-655:
- Initial commit of the LDAP based identity providers.
- Fixed issue when attempting to log into a NiFi that does not support new account requests.
NIFI-655:
- Allowing the ldap provider to specify if client authentication is required/desired.
NIFI-655:
- Persisting keys to sign user tokens.
- Allowing the identity provider to specify the token expiration.
- Code clean up.
NIFI-655:
- Ensuring identities are unique in the key table.
NIFI-655:
- Adding support for specifying the user search base and user search filter in the active directory provider.
NIFI-655:
- Fixing checkstyle issues.
NIFI-655:
- Adding automatic client side token renewal.
NIFI-655:
- Ensuring the logout link is rendered when appropriate.
NIFI-655:
- Adding configuration options for referrals and connect/read timeouts
NIFI-655:
- Added an endpoint for access details including configuration, creating tokens, and checking status.
- Updated DTOs and client side to utilize new endpoints.
NIFI-655:
- Refactoring certificate extraction and validation.
- Refactoring how expiration is specified in the login identity providers.
- Adding unit tests for the access endpoints.
- Code clean up.
NIFI-655:
- Keeping token expiration between 1 minute and 12 hours.
NIFI-655:
- Using the user identity provided by the login identity provider.
NIFI-655: - Fixed typo in error message for unrecognized authentication strategy.
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
NIFI-655. - Added logback-test.xml configuration resource for nifi-web-security.
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
NIFI-655. - Added issuer field to LoginAuthenticationToken. - Updated AccessResource to pass identity provider class name when creating LoginAuthenticationTokens. - Began refactoring JWT logic from request parsing logic in JwtService. - Added unit tests for JWT logic.
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
NIFI-655. - Changed issuer field to use FQ class name because some classes return an empty string for getSimpleName(). - Finished refactoring JWT logic from request parsing logic in JwtService. - Updated AccessResource and JwtAuthenticationFilter to call new JwtService methods decoupled from request header parsing. - Added extensive unit tests for JWT logic.
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
NIFI-655:
- Refactoring key service to expose the key id.
- Handling client side expiration better.
- Removing specialized active directory provider and abstract ldap provider.
NIFI-655. - Updated JwtService and JwtServiceTest to use Key POJO instead of raw String key from KeyService.
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
NIFI-655:
- Fixing typo when loading the ldap connect timeout.
- Providing a better experience for session expiration.
- Using ellipsis for lengthly user name.
- Adding an issuer to the authentication response so the LIP can specify the appropriate value.
NIFI-655:
- Showing a logging in notification during the log in process.
NIFI-655:
- Removing unnecessary class.
NIFI-655:
- Fixing checkstyle issues.
- Showing the progress spinner while submitting account justification.
NIFI-655:
- Removing deprecated authentication strategy.
- Renaming TLS to START_TLS.
- Allowing the protocol to be configured.
NIFI-655:
- Fixing issue detecting the presence of DN column
NIFI-655:
- Pre-populating the login-identity-providers.xml file with necessary properties and documentation.
- Renaming the Authentication Duration property name.
NIFI-655:
- Updating documentation for the failure response codes.
NIFI-655:
- Ensuring the user identity is not too long.
NIFI-655:
- Updating default authentication expiration to 12 hours.
NIFI-655:
- Remaining on the login form when there is any unsuccessful login attempt.
- Fixing checkstyle issues.
connections are scheduled to run, even if the connections have no FlowFiles;
expose these details to processor developers; update documentation
Signed-off-by: Aldrin Piri <aldrin@apache.org>
Fixed the order of service state check in PropertyDescriptor
Encapsulated the check into private method for readability
Modified and documented test to validate correct behavior.
For more details please see comment in https://issues.apache.org/jira/browse/NIFI-1143
- Fixing empty java docs and adding sort by id asc to the history query
- Changing userDn to userIdentity in Action and FlowChangeAction
- Modifying NiFiAuditor to always save events locally, and implementing getFlowChanges for ClusteredEventAccess
joewitt
Koji Kawamura
Aldrin Piri
Matt Gilman
Mark Payne
Bryan Bende
Jeff Storck
Bryan Rosander
Yolanda M. Davis
Jeremy Dyer
Pierre Villard
Oleg Zhurakousky
Pasqualino Ferrentino
Richard Miskin
Tony Kurc
Sönke Liebau
Joseph Percivall