bb6c5d9d4e
NIFI-2777:
...
NIFI-2856:
- Only performing response merging when the node is the cluster cooridinator even if there is a single response.
- Fixing PropertyDescriptor merging to ensure the 'choosen' descriptor is included in map of all responses.
This closes #1095 .
2016-10-06 16:19:19 -04:00
3b408f5601
NIFI-2816 - Clean typos across the code - Part 2. This closes #1085
2016-10-05 13:07:57 -04:00
2c907c63af
NIFI-2858:
...
- getSystemLoadAverage of OperatingSystemMXBean sometimes returns a negative value.
- ProcessorLoadAverage sets -1.0. It added a defensive logic.
- When load average is zero(load average: 0.00, 0.02, 0.04), systemLoad is zero.
- This closes #1091
2016-10-05 08:59:26 -04:00
17a36c6fd5
NIFI-2825: Fix S2S getPeers flow file count
...
- Added ClusterWorkload message to retrieve workload information from a
cluster coordinator
- Use cluster workload to return queued flow file count to site-to-site
client so that it can calculate distribution of data transfer
This closes #1084 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2016-10-03 13:28:01 -04:00
446cd44702
NIFI-2816 - Clean typos across the code
...
This closes #1057 .
2016-09-26 17:47:31 +02:00
bc7c42efa5
NIFI-1966: Recreated issue that is outlined in JIRA (the reason for re-opening the ticket) that results in 'java.util.NoSuchElementException: No value present' in unit test - Resolved issue where two flows that are both empty but have different fingerprints (due to root group id being different) causes vote election to fail
...
This closes #995 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2016-09-08 10:48:24 -04:00
a7e76cc00a
NIFI-1966: When cluster is started up, do not assume that Cluster Coordinator has the golden copy of the flow but instead wait for some period of time or until the required number of nodes have connected, and then choose which flow is correct. This closes #977
2016-09-06 16:31:37 -04:00
97e2f406da
NIFI-2708 Added merging of SiteToSite details
...
NIFI-2708 Updated to include ports from clientDto for port merging.
This closes #981
2016-09-02 14:55:08 -04:00
1fe18a1567
NIFI-2676-rc1 prepare for next development iteration
2016-08-26 11:40:58 -04:00
74d5224783
NIFI-2676-rc1 prepare release nifi-1.0.0-RC1
2016-08-26 11:40:44 -04:00
a6133d4ce3
NIFI-2635:
...
- Fixing contrib check issues.
- Clean up pom.
- Addressing issue where reporting task property descriptor using wrong scope.
NIFI-2635:
- Fixing issue with revisions when creating users and user groups.
- Forwarding requests to the coordinator instead of replicating.
- Tweaking verbage in dialog for removing users and groups.
This closes #943
2016-08-25 13:08:35 -04:00
6fd1c17ca4
NIFI-2662 - Fix checkstyle violations introduced by NIFI-2635
...
Signed-off-by: Matt Burgess <mattyb149@apache.org>
This closes #940
2016-08-25 08:55:34 -04:00
c2bfc4ef24
NIFI-2635: - Re-using the original request during the second phase of the two phase commit. - Forwarding requests to the coordinator when received by a node.
...
This closes #933
Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-08-24 22:42:10 -04:00
c2ae7a6d7c
NIFI-2605: - Fixing a regression bug where nodes would potentially be elected leader for Cluster Coordinator role when they do not have the correct flow
...
- Ensure that we log which node is the cluster coordinator on startup instead of just indicating that there is one. If we later determine that there is none, ensure that we register for the role
This closes #900
Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-08-24 12:30:48 -04:00
c638191a47
NIFI-1831 Added internal logic and command-line tool to allow AES-encrypted sensitive configuration values in nifi.properties.
...
This closes #834 .
2016-08-23 20:34:31 -07:00
31ca8d02f7
NIFI-2488 Fixed BulletinDTOs not getting their node addresses set properly during bulletin merging. This closes #888
2016-08-19 08:32:00 -04:00
08735a4f57
NIFI-2488 Wrapping ActionDTO and BulletinDTO in entities to be able to provide readability permission to the client
2016-08-18 09:37:11 -04:00
6bf7e7f325
NIFI-2574 merging master with cluster changes to updated NiFiProperties approach
2016-08-17 01:23:54 -07:00
7d7401add4
NIFI-2574 Changed NiFiProperties to avoid static initializer and updated all references to it.
2016-08-17 00:10:07 -07:00
e42ea9ad45
NIFI-2566: Refactored to allow just the Leader Election Manager to be responsible for determining who is the Cluster Coordinator
...
NIFI-2566: Removed storage of cluster roles from heartbeats and NodeConnectionStatus; use LeaderElectionManager to determine roles instead
NIFI-2566: Updated Heartbeats so that if a node is out-of-sync with cluster topology, cluster coordinator will provide updated information back to the nodes
NIFI-2566: Fixed issue that prevented standalone instance from starting by creating a standalone-instance version of the Leader Election Manager. Also added Controller Service enabled/disabled state to fingerprint rather than attempting to update the state when joining the cluster, as the implementation was incorrect and the correct implementation will be a rather significant effort that doesn't have to happen for 1.0.0 release
This closes #866
Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-08-16 20:39:04 -04:00
6799bd4919
NIFI-1876 fixing counter URI pattern
2016-08-12 17:23:12 -04:00
bbdd087d77
NIFI-1876 Implemented PropertyDescriptorDTO merging
...
NIFI-1876 Updated PermissionsDTO to use boolean primitives instead of Boolean objects for read and write permission
Removed ISE throwing from EntityFactory, it will rely on null checks instead
This closes #694
Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-08-12 17:22:37 -04:00
572dfed78a
NIFI-1876 Implements merging of responses to successful requests based on authorization, returning the most restrictive response - Added StandardHttpResponseMergerSpec for testing response merging - Added Permissible interface - Added nifi-api/controller/archive to ProcessGroupEndpointMerger - Removed AbstractMultiEntityEndpoint.java, not used anymore - Implemented reponse merging for GET requests where there are some successful and problematic responses, returning most restrictive one. - Updated nf-settings.js with ControllerConfigurationEntity property rename from controllerConfiguration to component - Implemented merging of status DTOs based on read permission NIFI-2264 Implemented merging of status history results based on readability permissions - Added StatusHistoryEndpointMergerSpec to test merging of status history based on read permission
...
NIFI-1876 Added merging for labels, funnels, and controller service references.
NIFI-1876 Added Label and Funnel merging to FlowMerger.java
NIFI-1876 Added replication of request for process group controller services
- Updated merging code for Status and Status History based on read permissions
NIFI-1876 Fixed issue with node status snapshots all looking like they came from one node
Updated ProcessGroupStatusSnapshotDTO to contain status snapshot entities to retain readability permission
Added entity classes for ConnectionStatusSnapshotDTO, PortStatusSnapshotDTO, ProcessGroupStatusSnapshotDTO, ProcessorStatusSnapshotDTO, and RemoteProcessGroupStatusSnapshotDTO
Updated PropertyDescriptorDTO to contain AllowableValueEntity to retain readability permission
Added entity class for AllowableValueDTO
Moved AllowableValueDTO to its own top-level class
Updated DtoFactory to get permissions for status snapshot entities
Updated StatusMerger to merge status snapshot entities
Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-08-12 17:22:17 -04:00
76a4a2c48b
NIFI-2544: Created integration tests for clustering and addressed a few minor bugs that were found in doing so
...
This closes #832 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-11 10:43:38 -04:00
42df02f014
NIFI-2406 This closes #820 . Addressed regression introduced in NIFI-2406 where the cluster does not recognize a new Cluster Coordinator when the coordinator is shutdown
2016-08-09 15:19:49 -04:00
c1c052af71
NIFI-2406: Ensure that hearbeat monitor continues to run while instance is running. This way if a node sends heartbeat to this node as elected coordinator changes, we notify the node accordingly. Handle Exceptions more gracefully in leader election code. Tweaked some handling of how nodes reconnect to the cluster to ensure more stability with cluster
...
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>
This closes #729
2016-08-08 09:15:10 -04:00
fca59ff9d0
NIFI-2438:
...
- If a node reports forbidden, using an appropriate response failure exception.
This closes #760 .
2016-08-03 15:01:04 -04:00
16348b071d
NIFI-2452: This closes #771 . Ensure that we keep track of how many references we have to each lucene searcher and only close the underlying index reader if there are no references to the searcher. Also updated to prefer newer provenance events over older provenance events, and calculate FlowFile lineage based on an event id instead of a FlowFile UUID, as it's much more efficient
2016-08-03 08:05:26 -07:00
09840027a3
NIFI-2304: If node address is not set, default to localhost, as the admin guide says it should
...
This closes #688
Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2016-08-02 12:09:34 +09:00
1bf10944ea
NIFI-2366 - Fixed ID generation semantics in clustered environment
...
- added SnippetUtilsTest
- renamed TypeOneUUIDGenerator to ComponentIdGenerator
- changed lsb part of ComponentIdGenerator back to long
- Fixed 'isCopy' condition for clustered environments
This closes #718 .
2016-07-31 15:24:02 -04:00
a6d259ee01
NIFI-2245:
...
- If a single response exists using it, instead of attempting to merge anything.
This closes #743 .
2016-07-29 15:29:06 -04:00
da238b16ef
NIFI-2122: - Merging responses for the current user and the flow configuration. - Returning whether NiFi is configured with a policy based authorizer in the flow configuration. - Only showing the users and policy icons when configured with a policy based authorizer. - Failing faster when invoking the users or policies endpoint when not configured with a configurable authorizer.
...
This closes #736
Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-28 16:51:40 -04:00
52bc23f5db
NIFI-2316, NIFI-2318: Ensure that we do not save the flow before initializing the Run Status of components. Clarify the Node Event messages
...
This closes #678
Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-26 14:24:32 -04:00
7779af69b4
NIFI-2292: Funnel all cluster node status changes through the cluster coordinator instead of having each node broadcast changes to the whole cluster. This gives us the ability to increment the updateId consistently without race conditions.
...
This closes #717
Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-25 23:17:37 -04:00
b082858595
Revert "NIFI-2292: Funnel all cluster node status changes through the cluster coordinator instead of having each node broadcast changes to the whole cluster. This gives us the ability to increment the updateId consistently without race conditions."
...
This reverts commit 35ff0975bf
2016-07-25 12:35:43 -04:00
35ff0975bf
NIFI-2292: Funnel all cluster node status changes through the cluster coordinator instead of having each node broadcast changes to the whole cluster. This gives us the ability to increment the updateId consistently without race conditions.
2016-07-25 12:14:13 -04:00
6932a53ec9
NIFI-2360: Leave ZooKeeper running when a node is disconnected. Do not allow the last node in the cluster to be disconnected. Change ClusterProtocoLHeartbeater to use RetryNTime retry strategy instead of RetryForever because web requests could block on this
...
This closes #705
Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-22 23:57:00 -04:00
4a4d60e6af
NIFI-2307: - Enforcing connection permissions based on the source and destination comonent. - Removing connection specific access policies. NIFI-2265: - Filtering out sensitive details in component status and status history when appropriate. NIFI-1800: - Adding parent process group id to the Controller Services table. NIFI-2077: - Removing some old un-used icons following the UI refresh. NIFI-2242: - Requiring write permissions for all components in a selection. NIFI-2080: - Updating style of the name in the selection context to handle scroll bars and use available width. NIFI-2331: - Addressing issue when removing a user/group which was causing the tenant policy to be removed. NIFI-2335: - Ensuring the flow is saved after starting/stopping a process group. NIFI-2235: - Ensuring we use consistent conditions between the context menu and the operate palette.
...
- Allowing users with read only access to the tenants page.
- Fixing current user integration test.
- Ensuring schedule methods are locked appropriately.
- Addressing comments from PR.
This closes #698
Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-21 23:52:01 -04:00
10b91ffe6c
NIFI-2332: Ensure that bulletin node address is set when clustered but not overwritten if already set
...
This closes #692
Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-20 19:44:32 -04:00
7f2bda29d9
NIFI-2289: Ensure that we include Node ID's of all nodes when finding cluster coordinator, even if the node is currently still connecting or has not yet joined the cluster, which ccan be the case if all nodes in the cluster are restarting at the same time.
...
This closes #681
Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-20 16:50:25 -04:00
2a7f135f1c
NIFI-2319: Ensure that when we set cluster node id's and node addresses, that we do so only if they are not already populated
...
This closes #680
Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-20 11:39:57 -04:00
108c815988
NIFI-2324: Log number of requests per URI if we have too many outstanding requests to replicate
...
This closes #683
Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-20 11:22:24 -04:00
5c8636edf4
NIFI-2289: Directly ask ZooKeeper which node is cluster coordinator and add watches on the ZNode rather than relying on Node Status Updates over the cluster protocol because cluster protocol may get the events out-of-order
...
This closes #665 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2016-07-18 10:13:42 -04:00
52a961873b
NIFI-826 This closes #617 . Added deterministic template support
2016-07-15 20:41:18 -04:00
c81dc1959a
NIFI-1992:
...
- Updated site-to-site client and server to support clustered nifi instances
NIFI-2274:
- Ensuring we use the correct URI when updating a connection.
This closes #530
2016-07-15 16:13:59 -04:00
5cd5a4ce78
NIFI-2260:
...
- Addressing issue searching provenance on a specific node.
- Fixing issues viewing content and replaying events.
This closes #658 .
2016-07-15 12:14:24 -04:00
d1129706e2
NIFI-1896 This closes #650 . Refactored nifi-api into nifi-framework-api and other locations. The nifi-api is specific to that which is needed for intended extension points.
2016-07-14 18:24:48 -04:00
f6136b69e5
NIFI-1413: Rebased against master and updated templates endpoint merge to use template entities instead of template dto's
...
This closes #649
2016-07-14 15:19:47 -04:00
329192a7d3
NIFI-1413: Ensure that if a node's templates don't match the clusters that we take the following actions: -Local templates remain but aren't shown in the cluster's templates. -Any templates from the cluster that don't exist on the node are added to the node. -Any conflicting template definitions are replaced by those in the cluster
...
This closes #596
2016-07-14 15:01:46 -04:00
3373e18158
NIFI-1947 NIFI-2082:
...
- Making it more obviously when a nodes cluster state changes.
- Showing which node is the cluster coordinator.
This closes #651
2016-07-14 11:31:48 -04:00
Matt Gilman
Andre F de Miranda
Byunghwa Yun
Koji Kawamura
Mark Payne
Jeff Storck
Joseph Percivall
Andy LoPresto
joewitt
Oleg Zhurakousky
Aldrin Piri