185 Commits

Author SHA1 Message Date
Andy LoPresto
d42a1e8bf4
NIFI-5323-RC1 prepare for next development iteration 2018-06-19 20:02:21 -07:00
Andy LoPresto
99bcd1f88d
NIFI-5323-RC1 prepare release nifi-1.7.0-RC1 2018-06-19 20:02:01 -07:00
Mark Payne
ff00050266
NIFI-5241: Updated EventSumValue to use synchronized methods instead of many atomic values. This is more efficient and uses less heap. Also noticed that the Logger instance in ProcessorNode was not used so removed it, and in testing this also noticed that the default connection pool size for OkHttpReplicationClient was only 5, which can cause a lot of unnecessary HTTP connections to be created so adjusted the pool size
NIFI-5241: Extended timeout that Jetty uses before closing an active HTTP connection. Because the UI refreshes every 30 seconds by default, and the Jetty connection pool times out every 30 seconds by default, we very frequently saw new HTTP connections being created for the UI refreshes. This resulted in 4 new connections and 4 SSL handshakes occurring every 30 seconds. By extending the timeout, we now see those connections being reused and SSL Handshakes no longer occurring frequently
NIFI-5241: Set Jetty idle timeout to double the amount of time for browser to refresh
NIFI-5241: Fixed synchronization issue with EventSumValue
This closes #2752
2018-06-07 09:15:32 -04:00
Matt Gilman
568288dbcd NIFI-5186:
- Updating UI to support showing when a component is validating.
2018-05-22 11:05:28 -04:00
Mark Payne
604656fe88
NIFI-950: Make component validation asynchronous
NIFI-950: Still seeing some slow response times when instantiating a large template in cluster mode so making some minor tweaks based on the results of CPU profiling
NIFI-5112: Refactored FlowSerializer so that it creates the desired intermediate data model that can be serialized, separate from serializing. This allows us to hold the FlowController's Read Lock only while creating the data model, not while actually serializing the data. Configured Jersey Client in ThreadPoolRequestReplicator not to look for features using the Service Loader for every request. Updated Template object to hold a DOM Node that represents the template contents instead of having to serialize the DTO, then parse the serialized form as a DOM object each time that it needs to be serialized.
NIFI-5112: Change ThreadPoolRequestReplicator to use OkHttp client instead of Jersey Client
NIFI-5111: Ensure that if a node is no longer cluster coordinator, that it clears any stale heartbeats.
NIFI-5110: Notify StandardProcessScheduler when a component is removed so that it will clean up any resource related to component lifecycle.
NIFI-950: Avoid gathering the Status objects for entire flow when we don't need them; removed unnecessary code
NIFI-950: Bug fixes
NIFI-950: Bug fix; added validation status to ProcessorDTO, ControllerServiceDTO, ReportingTaskDTO; updated DebugFlow to allow for pause time to be set in the customValidate method for testing functionality
NIFI-950: Addressing test failures
NIFI-950: Bug fixes
NIFI-950: Addressing review feedback
NIFI-950: Fixed validation logic in mock framework
This closes #2693
2018-05-16 14:39:23 -04:00
Mark Payne
8acac9cba5
NIFI-5153: If a node is disconnected due to failure to complete mutable request, the node should be allowed to rejoin
This closes #2677.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-05-07 15:41:38 -04:00
Matt Gilman
af2513adf8 NIFI-1295:
- Adding UI controls for terminating hung threads.
- Showing current number of terminated threads.
- Fixing issue when replicating terminate threads request throughout the cluster.

This closes #2607.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-04-25 16:23:34 -04:00
joewitt
d511fe3e4b NIFI-4995-RC3 prepare for next development iteration 2018-04-03 08:28:34 -07:00
joewitt
f8466cb16d NIFI-4995-RC3 prepare release nifi-1.6.0-RC3 2018-04-03 08:28:15 -07:00
Mike Moser
c9f1c9789d NIFI-4973 Added license comment to new file RemoteProcessGroupEntityMergerTest.java 2018-03-16 19:15:52 +00:00
Matt Gilman
a1c917656e NIFI-4973:
- Fixing RPG port merging.
- Adding unit tests.
- Removing unecessary sorting that wasn't maintained while clustered.

This closes #2551.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-03-16 13:03:09 -04:00
Mark Payne
844da06344
NIFI-4849: Implemented REST Endpoint and associated backend code to generate a Diagnostics Report for a Processor
Implemented review feedback. Refactored data model to make the API cleaner and delineate more along the lines of what permissions are required in order to see which details
Implementing review feedback
Removed sensitive information from the diagnostics reports
Fixed bug in merging logic for GCDiagnosticsSnapshots
This closes #2468
2018-03-15 15:10:41 -04:00
Matt Gilman
b1217f529b NIFI-4885:
- Introducing more granular restricted component access policies.

This closes #2515.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-03-12 14:19:42 -04:00
joewitt
41ce788812 NIFI-4751 changed to next minor release version snapshot 2018-01-12 15:15:32 -05:00
joewitt
36405e888c NIFI-4751-RC1 prepare for next development iteration 2018-01-08 23:39:49 -07:00
joewitt
46d30c7e92 NIFI-4751-RC1 prepare release nifi-1.5.0-RC1 2018-01-08 23:39:32 -07:00
Matt Gilman
f48808b1f4
NIFI-4436:
- Updating buckets permissions based on new model.
- Adding check to ensure that flow name is non null before checking the length.
- Adding versioned flow state to the Process Group tab in the Summary table.
- Fixing issue with navigating to Controller Services from the local changes dialog.
2018-01-08 12:44:56 -05:00
Mark Payne
181d6809c1
NIFI-4436: Bug fixes; ensure correct Exception types are thrown
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2018-01-08 12:44:55 -05:00
Mark Payne
fe8b30bf26
NIFI-4436: Removed isCurrent, isModified from VersionControlInformation and associated DTO. Bug fixes & code refactoring
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2018-01-08 12:44:55 -05:00
Mark Payne
6a58d780d7
NIFI-4436:
- Initial checkpoint: able ot start version control and detect changes, in standalone mode, still 'crude' implementation
- Checkpoint: Can place flow under version control and can determine if modified
- Checkpoint: Change version working in some cases. Does not work if processor removed because COMPONENT_REMOVED type has ComponentA whose ID is the VersionedComponentID but we are trying to call ProcessorDAO.get() with this ID
- Checkpoint: Able to change flow from Version 1 to Version 2 and back. Not yet tested with controller services. Have not tried changing/removing connections. Not cluster-friendly yet. All inline, not in background. Have not taken into account ports, funnels, remote ports, etc. Have not tested with Labels yet
- Checkpoint after implementing ClusterReplicationComponentLifecycle instead of JerseyClientComponentLifecycle
- Checkpoint: Updated to allow starting version control and updating version in clustered mode
- Checkpoint: Updated versioning endpoint so that when version of a flow is updated, the bundle information is populated and the snapshot is replicated to the cluster.
- Checkpoint: Implemented endpoint for reverting to previously sync'ed version of a flow and updated version control endpoint so that Process Group can be pushed as a new version to existing flow instead of only creating a new flow
- Checkpoint: Updated so that if a Process Group is under Version Control and it has a child Process Group, which is also under Version Control, we can handle that gracefully. Not yet tested because it depends on updates to the nifi-registry module, which can't be compiled due to maven dependency conflicts
2018-01-08 12:44:52 -05:00
Matt Gilman
6b75921725 NIFI-4617:
- Ensuring a charset is specified when convert a string into bytes.
- Ensuring the aggregate snapshot is non null before attempting to promote the runStatus.

This closes #2278.
2017-11-20 09:39:23 -05:00
sbouchex
c89d793364
NIFI-4615
- processor status is set to the ProcessorStatusDTO object
- This closes #2276
2017-11-17 15:44:09 -05:00
Matt Gilman
6baea8ccff
NIFI-4444:
- Upgrading to Jersey 2.x.
- Updating NOTICE files where necessary.
- Fixing checkstyle issues.

This closes #2206.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-10-12 10:27:02 -07:00
Jeff Storck
a57911d3db NIFI-4412-RC2 prepare for next development iteration 2017-09-28 13:45:36 -04:00
Jeff Storck
e6508ba7d3 NIFI-4412-RC2 prepare release nifi-1.4.0-RC2 2017-09-28 13:45:21 -04:00
Matt Gilman
6c798d18ef NIFI-4382:
- Adding support for KnoxSSO.
- Updated the docs for nifi.security.user.knox.audiences.
- The KnoxSSO cookie is removed prior to request replication.

This closes #2177
2017-09-27 16:22:18 -04:00
Andy LoPresto
9e2c7be7d3
NIFI-4353
- Added XmlUtils class.
- Added unit test.
- Added XXE test resource.
- Refactored JAXB unmarshalling globally to prevent XXE attacks.
- Refactored duplicated/legacy code.
- Cleaned up commented code.
- Switched from FileInputStream back to StreamSource in AuthorizerFactoryBean.
- This closes #2134
2017-09-22 14:31:38 -04:00
Matt Gilman
eac47e90cb NIFI-4280:
- Adding support for the user to configure variables in the UI.
- Updating the endpoints for changing variables as necessary.
This closes #2135.
2017-09-14 11:12:54 -04:00
Pierre Villard
5957f6ee95
NIFI-4341 - add provenance repository storage usage in UI
- addressing review comments
- This closes #2119
2017-09-11 13:16:53 -04:00
Yolanda M. Davis
0eda71a9a6 NIFI-4310 - added changes to support detection of reporting tasks and controller services during isEmpty flow check. Added testing scenarios. This closes #2107. 2017-08-25 10:15:55 -04:00
Mark Payne
5cd8e93beb
NIFI-4224:
- Initial implementation of Process Group level Variable Registry
- Updated to incorporate PR Feedback
- Changed log message because slf4j-simple apparently has a memory leak; passing a String instead of passing in the Controller Service object as an argument addresses this.
- This closes #2051
2017-08-17 10:42:29 -04:00
Mark Payne
695e8aa98f
NIFI-106:
- Expose processors' counters in Stats History
- Only include counters in Processors' Status History if user has read access to corresponding Processor
- Addressed review feedback. Found and addressed bug where a counter is not present in all of the aggregate snaphot values for status history, resulting in the UI not rendering the chart properly
- This closes #1872
2017-07-21 14:18:11 -04:00
Pierre Villard
a3b72f1bb7
NIFI-4143 - externalize MAX_CONCURRENT_REQUESTS. This closes #1962 2017-07-06 15:38:55 -04:00
Mark Payne
ba56774fa1
NIFI-4151: Updated UpdateAttribute to only create JAXB Context once; Minor performance tweaks to standard validators and StatusMerge.prettyPrint; updated AbstractConfiguredComponent to not create a new ValidationContext each time that validate is called but only when needed; updated FlowController, StandardControllerServiceProvider, and StandardProcessGroup so that component lookups can be performed using a ConcurrentMap at FlowController level instead of having to perform a depth-first search through all ProcessGroups when calling findProcessor(), findProcessGroup(), findXYZ()
This closes #1979
2017-07-06 13:43:56 -04:00
Mark Payne
cff81c0cd2
NIFI-4153: Use a LinkedBlockingQueue instead of a SynchronousQueue for Request Replicator's thread pool so that requests will queue when all threads are active, instead of throwing an Exception. This closes #1980 2017-07-06 10:11:48 -04:00
Pierre Villard
47165afc0c
NIFI-4073 - fix duplicated stack trace
This closes #1916.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-06-23 11:20:07 -04:00
Matt Gilman
4ed7511bee
NIFI-3653: - Introducing UserGroup and Policy provider interfaces.
- Introducing FileUserGroupProvider and FileAccessPolicyProvider.
- Refactoring FileAuthorizer to utilize the file based implementations.
- Introducing the StandardManagedAuthorizer.
- Decorating the configured ManagedAuthorizer to ensure integrity checks are still performed.
- Loading user groups if possible to use during access decisions.
- Merging responses for requests for AccessPolicies, Users, and UserGroups.
- Adding unit tests as appropriate.
- Adding methods to the User, Group, and AccessPolicy builder that more easily supports generating UUIDs.
- Fixing typo when seeding policies during startup.
- Fixing type in documentation and error messages.

This closes #1897.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-06-09 13:54:10 -04:00
Matt Gilman
cc741d2be6
NIFI-3997:
- Bumping to next minor version.
2017-06-08 15:22:51 -04:00
Matt Gilman
6ee12e9b47
NIFI-3997-RC1prepare for next development iteration 2017-06-05 11:07:43 -04:00
Matt Gilman
ddb73612bd
NIFI-3997-RC1prepare release nifi-1.3.0-RC1 2017-06-05 11:07:28 -04:00
Mark Payne
5aa3baca79 NIFI-3568: This closes #1577. Use a cached thread pool in order to allow ThreadPoolRequestReplicator to scale up the number of threads to some configurable max
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-24 21:21:45 -04:00
Matt Gilman
ded396f0ef NIFI-3933:
- When monitoring heartbeats use the connected nodes as the basis for the check. This addresses the case when a node is terminated and no corresponding heartbeats exist.

This closes #1838.
2017-05-22 16:51:13 -04:00
Matt Gilman
d33c4c72d4 NIFI-3933:
- When monitoring heartbeats use the connected nodes as the basis for the check. This addresses the case when a node is terminated and no corresponding heartbeats exist.
2017-05-22 16:50:30 -04:00
Matt Gilman
6b71b4cbb8 NIFI-3868: - Ensuring we do not attempt to group bulletins that lack permissions. - Only group bulletins when all nodes report the same message. - Retain the most recent bulletin.
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1801
2017-05-16 16:54:30 -04:00
Matt Gilman
cf497c9cf9
NIFI-3865: - Obtaining the appropriate input stream after accessing content. The stream was being consumed into a buffer and was not longer available.
This closes #1780.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-10 15:51:00 -04:00
Bryan Bende
3af53419af
NIFI-3770-RC2 prepare for next development iteration 2017-05-05 20:50:28 -04:00
Bryan Bende
3a605af8e0
NIFI-3770-RC2 prepare release nifi-1.2.0-RC2 2017-05-05 20:50:14 -04:00
Matt Gilman
f3745065b0
NIFI-3800:
- Cleaning up the headers when replicating requests.

This closes #1752.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-05-04 12:05:24 -04:00
Yolanda M. Davis
c0f0462e8b
NIFI-3695 - created the nifi admin toolkit which includes shell scripts and classes to support notification and basic node management in standalone and clustered nifi.
This closes #1669.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-04-24 23:08:37 -07:00
Mark Payne
a88d3bfa3c NIFI-1280: Refactoring to make more generic so that other data types can be supported; created InputStreams to content on-demand so that multiple passes can be made over FlowFile content if required. Created new Controller Services for reading and writing specific data types
Signed-off-by: Matt Burgess <mattyb149@apache.org>
2017-04-11 19:29:04 -04:00