Mark Payne
78be613a0f
NIFI-10888: When inferring a schema using a Record Reader, buffer up to 1 MB of FlowFile content for the schema inference so that when we read the contents to obtain records we can use the buffered data. This helps in cases of small FlowFiles by not having to seek back to the beginning of the FlowFile every time.
...
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #6725
2022-12-14 12:09:32 -05:00
Mark Payne
2473683ce5
NIFI-10918: When fetching a flow from a Flow Registry, if it references any 'internal versioned flows' instead of requiring that we have a client configured for the appropriate URL, attempt to fetch the flow from each client. We will start with the clients that do report that they can handle the URL but will try others as well. As soon as we successfully fetch the flow, we stop.
...
NIFI-10918: Fixed checkstyle violations
This closes #6736
Signed-off-by: Bence Simon <bsimon@apache.org>
2022-12-02 14:24:40 +01:00
dan-s1
e8d554d9b4
NIFI-10876 Converted some nifi-framework tests from JUnit 4 to JUnit 5
...
This closes #6720
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-11-28 14:47:17 -06:00
Joe Witt
75e7c9e47c
NIFI-10854-RC1 prepare for next development iteration
2022-11-22 19:22:47 -07:00
Joe Witt
ec87bf93ad
NIFI-10854-RC1 prepare release nifi-1.19.0-RC1
2022-11-22 19:22:45 -07:00
sedadgn
009d641576
NIFI-10787 - Cannot commit flows to nifi registry after updating our nifi release to 1.18.0
...
NifiRegistryFlowRegistryClient defines the PropertyDescriptor PROPERTY_URL name as small case "url". The map bases on the name property of the PropertyDescriptor object. Here searching with uppercase value of "URL" causes the map lookup to fail and cause a NPE later on. Therefore, it is changed as "url"
NIFI-10787 - Added constant for property descriptor "url" in NiFiRegistryFlowMapper to make it more clear.
NIFI-10787 - Added change to unit test.
NIFI-10787 - Updated unit test to validate that NiFi registry url is being set and retrieved.
Signed-off-by: Nathan Gough <thenatog@gmail.com>
This closes #6655 .
2022-11-16 19:07:15 -05:00
Mark Payne
bda624823b
NIFI-10817: Moved the calls in StandardProcessSession to 'resetState()' and 'acknowledgeRecords()' from the outer commit(boolean) to the inner commit(Checkpoint, boolean). By moving the call here, the logic of StandardProcessSession is unaffected. But the StatelessProcessSession that inherits from it now has the benefit of having the state cleaned up when calling super.commit(Checkpoint, boolean).
...
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #6663
2022-11-15 15:47:32 -05:00
Paul Grey
25b6e2c9cd
NIFI-10815 Fixed initial internal yield duration when processor created
...
This closes #6664
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-11-15 14:14:50 -06:00
Bryan Bende
7de74ad3f0
NIFI-10802 Apply bundle updates before properties in component synchronizer ( #6654 )
...
This closes #6654
2022-11-14 09:09:44 -05:00
Bryan Bende
9c21e26e63
NIFI-10788 Ensure proposed service config is applied when component synchronizer adds a new service ( #6640 )
...
This closes #6640
2022-11-09 15:04:19 -05:00
Matt Gilman
b05bd988b6
NIFI-10700:
...
- Only restart ports if their scheduledState is Running.
- Adding unit tests.
- Fixing unit test to properly verify Port isn't restarted.
Signed-off-by: Nathan Gough <thenatog@gmail.com>
This closes #6582 .
2022-10-26 16:31:27 -04:00
Mark Payne
d390a0becf
NIFI-10673: When a component is added to a ProcessGroup, we had a method called ensureUniqueVersionControlId but the method only took into accounts in that Process Group. Updated methods to now consider components in any Process Group that is part of the same Versioned Flow. Also added system test to verify the problem and the fix. Added Toolkit clients and additional methods as necessary in order to implement system tests
...
NIFI-10673: Addresed checkstyle issue
NIFI-10673: Removed TODO comment
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
2022-10-26 14:38:01 -04:00
Nandor Soma Abonyi
2049268372
NIFI-10668 Persist proxy password in remote process group ( #6557 )
...
* NIFI-10668 Persist proxy password in remote process group
2022-10-21 11:44:12 -04:00
markap14
e83e3eb849
NIFI-10663: When ProcessorNode.stop() is called, it returns a Future. That future, however, may be completed when the processor is fully stopped or whenever it becomes stopping. Fixed this behavior by only completing the future whenever the processor has fully stopped. Also addressed an issue in the synchronizer in which we did not stop a processor when it had a state of STARTING. ( #6561 )
...
This closes #6561
2022-10-20 15:42:37 -04:00
Joe Gresock
2d7f86b265
NIFI-10641: Resolving typo on ParameterProvider warning message
...
This closes #6527
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-10-15 09:05:57 -05:00
Joe Witt
c9ebdd2025
NIFI-10521-RC4 prepare for next development iteration
2022-10-03 10:59:36 -07:00
Joe Witt
109e54cd58
NIFI-10521-RC4 prepare release nifi-1.18.0-RC4
2022-10-03 10:59:34 -07:00
Joe Gresock
10a5e9194c
NIFI-10572: Allowing variables to be deleted between versions, and considering ancestor variable additions to be environmental
...
This closes #6474
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-10-03 12:44:07 -05:00
Joe Gresock
094222260c
NIFI-9402: Adding DatabaseParameterProvider
...
Adding provided scope to api dependency in nifi-standard-parameter-providers module
Adding additional documentation, other minor code cleanup
Correcting error handling in StandardParameterProviderNode, updating additional details for DatabaseParameterProvider
Correcting null columm value handling
NIFI-9402: Fixed Checkstyle violation
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #6391
2022-09-28 17:41:57 -04:00
simonbence
94fdc794b0
NIFI-10497 Making RegistryClient an extension point ( #6433 )
...
- NIFI-10497 Mae FlowRegistryClient an extension point
2022-09-26 12:16:21 -04:00
Joe Gresock
43931b1504
NIFI-10518: Adding intended state to ScheduledStateChangeListener ( #6428 )
...
- NIFI-10518: Adding intended state to ScheduledStateChangeListener
- Notifying of scheduled state change when transitionComponentState is called
- Notifying scheduled state change when reporting task state is changed
- Notifying scheduledState changes for remote group port start/stop components calls
2022-09-19 10:38:00 -04:00
Mark Payne
eaa275a5e4
NIFI-10515: If propopsed controller service is disabled, ensure that we don't re-enable the service and references when synchronize(ControllerServiceNode...) is called
...
Signed-off-by: Joe Gresock <jgresock@gmail.com>
This closes #6425 .
2022-09-16 16:36:55 -04:00
Joe Gresock
51d01f874e
NIFI-10401: Adding ScheduledStateChangeListener to synchronizer ( #6341 )
...
NIFI-10401: Adding ScheduledStateChangeListener to synchronizer
2022-09-15 09:15:44 -04:00
Joe Gresock
de7793e69b
NIFI-9003 Added framework support for Parameter Providers
...
- Added Environment Variable and File Parameter Provider implementations
- Added Parameter Provider commands to CLI
- Added component search handling for Parameter Providers
- Added documentation for integrating Parameter Providers
This closes #5369
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-09-09 13:04:56 -05:00
Mark Payne
7a90137cef
NIFI-10431: Improved log messages and Exception messages to indicate which components they pertain to. Also added a toString() for StandardLabel to make it more clear which Label is being referenced
...
Signed-off-by: Joe Gresock <jgresock@gmail.com>
This closes #6361 .
2022-09-02 15:56:33 -04:00
Matthew Burgess
d698e7af1a
NIFI-10306: Add logging to StandardProcessGroup.addConnection() with more details
...
This closes #6264
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-08-25 07:50:55 -05:00
Mark Payne
9070cd0904
NIFI-10383: When importing/synching to a VersionedFlow, if the Versioned PropertyDescriptor indicates that a property references a Controller Service, treat it the same as if the PropertyDescriptor itself indicates that it references a service. This allows us to ensure that scripted components' property descriptors that reference controller services are properly mapped. Additionally updated StandardProcessorNode so that when we have this condition that we properly account for the Controller Service reference
...
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #6322
2022-08-23 09:59:07 -04:00
Mark Payne
b012e9aad2
NIFI-10371: When a component is moved between groups, ensure that its versioned component id is unique within the destination group. Also ensure that when adding a connection to a PG with the VersionedComponentSynchronizer that we prefer obtaining source/destination by instance id instead of versioned id.
...
Fixed bug where ProcessGroup would inadvertently set the wrong component's Versioned Component ID to null when there was an ID conflict
This closes #6314
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-08-22 08:07:27 -05:00
Hsin-Ying Lee
2685856c62
NIFI-10052 Avoid obtaining any locks when creating/sending heartbeats ( #6298 )
2022-08-17 14:05:55 -04:00
Joe Witt
8c66bf948c
NIFI-10272-RC2 prepare for next development iteration
2022-07-27 13:24:09 -07:00
Joe Witt
8d256784d8
NIFI-10272-RC2 prepare release nifi-1.17.0-RC2
2022-07-27 13:24:06 -07:00
Mark Payne
92750c2746
NIFI-10261: Ensure that when comparing Sensitive Parameters during flow sync we decrypt values for the comparison
...
This closes #6231
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-07-25 10:52:46 -05:00
Mark Payne
0497907829
NIFI-10262: Ensure that when an Exception is thrown from a Controller Service's @OnEnabled method that we properly handle that Exception and continue enabling the other services in the given collection of services
...
This closes #6236
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-07-25 10:02:21 -05:00
Mark Payne
bc4e3e850b
NIFI-10246: When syncing Controller Services, wait until any DISABLING Controller Service has fully disabled before attempting to synchronize them. Also performed some minor refactoring/cleanup for System Tests in order to make writing a system test for this simpler
...
This closes #6219
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-07-22 13:31:06 -05:00
Mark Payne
4c0555a476
NIFI-10203: Ensure that when a FlowFile is transferred and is not retried that we remove the retryCount.<uuid> attribute
...
This closes #6201
Signed-off-by: Paul Grey <greyp@apache.org>
2022-07-12 19:01:41 -04:00
Mark Payne
996d8faaf4
NIFI-10203: Fixed bug in which same FlowFile Builder was used repeatedly for multiple FlowFiles; this caused mingling of their attributes when a FlowFile is routed to a relationship that has more than 1 destination (i.e., many connections with the same Relationship)
...
This closes #6187
Signed-off-by: Paul Grey <greyp@apache.org>
2022-07-12 09:52:49 -04:00
Ferenc Erdei
d78c667c19
NIFI-10166 Improved MiNiFi bootstrap test coverage
...
This closes #6160
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-07-01 14:45:31 -05:00
exceptionfactory
1f18345929
NIFI-10163 Corrected StandardProcessSessionTest after NIFI-10167
2022-06-30 19:10:57 -05:00
exceptionfactory
943197c18b
NIFI-10163
...
- Corrected StandardProcessSession.exportTo() byte counting
- Removed bytesWritten incrementing from exportTo methods
Signed-off-by: Nathan Gough <thenatog@gmail.com>
This closes #6153 .
2022-06-30 18:42:44 -04:00
Mark Payne
5e83bda9e9
NIFI-10167: Added advanced timing metrics for processors, such as CPU Usage, time reading/writing content repo, process session commit time, etc. Exposed via nifi.sh diagnostics and made configurable via nifi.properties
...
This closes #6156
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-06-30 10:50:24 -05:00
timeabarna
d5b626f0e4
NIFI-10108 Processor scheduling via parameter ( #6115 )
...
- NIFI-10108 Processor scheduling via parameter
- Refactoring component referencing check in AbstractComponentNode and StandardParameterReferenceManager classes.
2022-06-30 10:11:13 -04:00
Nissim Shiman
db11961026
NIFI-9440 Allow Controller Services to have configurable Bulletins
...
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #6035
2022-06-28 10:44:45 -04:00
Joe Gresock
3dda694fd6
NIFI-10139: Adding fields to RemoteProcessGroupStatus ( #6137 )
2022-06-22 12:24:23 -04:00
Bryan Bende
1de9c72779
NIFI-10140 Handle deleted parameters in StandardVersionedComponentSynchronizer
...
Signed-off-by: Joe Gresock <jgresock@gmail.com>
This closes #6138 .
2022-06-21 10:43:38 -04:00
Joe Gresock
5e01d3de06
NIFI-10117: Allowing remote group ports to be synchronized from Stand… ( #6128 )
...
NIFI-10117: Allowing remote group ports to be synchronized from StandardVersionedComponentSynchronizer
2022-06-17 14:28:14 -04:00
Joe Gresock
314232ca6c
NIFI-10072: Adding LoadBalanceStatus to ConnectionStatus ( #6086 )
2022-05-31 16:27:23 -04:00
Bryan Bende
98ec2563e9
NIFI-10059 Remove incorrect removal of controller service from cache ( #6082 )
2022-05-26 17:13:28 -04:00
Mark Payne
35d8bb5ca9
NIFI-10051: Clear Controller Services' state in a background thread instead of the web thread.
...
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #6078
2022-05-26 10:31:55 -04:00
exceptionfactory
99be62be67
NIFI-9958 Add Framework Support for Sensitive Dynamic Properties ( #6057 )
...
* NIFI-9958 Implemented Sensitive Dynamic Properties
- Added SupportsSensitiveDynamicProperties annotation for components
- Added optional sensitive query parameter to Property Descriptor REST Resource methods
- Added system tests for components supporting sensitive dynamic properties
- Updated REST Resources to support Sensitive Dynamic Property Names
- Updated Documentation Writer to indicate component support for Sensitive Dynamic Properties
- Updated InvokeHTTP to support Sensitive Dynamic Properties
- Updated Auditor components to handle masking Sensitive Dynamic Properties
* Refactored Property Descriptor REST method handling
- Corrected AbstractDocumentationWriter evaluation of support for sensitive dynamic properties
- Refactored Controller Service Dynamic Properties system tests to new class
* Updated AbstractComponentNode.getProperties() to get canonical descriptor
2022-05-23 17:20:54 -04:00
markap14
88e53eae8c
NIFI-10001: Fixed bugs that caused some components to not have their scheduled state updated. ( #6049 )
...
* NIFI-10001: Fixed issue in which some components may fail to update the scheduled state when comparing flows
* NIFI-10001: Fixed bugs that caused some components to not have their scheduled state updated. When comparing two flows, now allow specifying how to determine a VersionedComponent's ID for comparison. When comparing local flow against flow from registry, use Versioned Component ID. But when comparing two instantiated flows, such as local flow vs. cluster flow, use the VersionedComponent's Instance ID instead. This ensures that we can properly compare two components even if there are several instances of a given flow
2022-05-18 13:53:36 -04:00
