Brandon Devries
cbc07eb663
NIFI-6157: Allowing configuration of concurrent tasks and transferred FlowFile count to Funnels and LocalPorts
...
This closes #3412
Signed-off-by: Brandon <devriesb@apache.org>
2019-04-15 14:13:11 +00:00
Rob Fellows
b5ff622118
NIFI-4230 - Use a better default location when pasting via keystokes and the original copied item is not fully in view.
...
- handle multi-select copy/paste better.
- remove dimension info from getOrigin function
- handle pasting better when pasted items should be centered and not offset from the original.
- fixed strict mode and removed console.logs
- Fixing pasting logic when trying to center the items on the canvas.
This closes #3383
2019-04-12 15:27:26 -04:00
Mark Payne
91b2283f49
NIFI-6183: Implemented base class for Framework-level integration tests that is responsible for instantiating the key components for integration tests, including FlowController, WriteAheadFlowFileRepository, WriteAheadProvenanceRepository, FileSystemRepository, FileSystemSwapManager, StandardFlowFileQueue, StandardProcessScheduler, etc. Implemented several integration tests to span different parts of the framework
...
This closes #3423 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2019-04-10 16:47:08 -04:00
Mark Payne
0f10010683
NIFI-6199: Fixed problematic unit tests that did not properly wait for async tasks to complete before making assertions
...
This closes #3421 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2019-04-09 11:24:58 -04:00
Mark Payne
bf7e70e4c3
NIFI-6194: Fixed verification logic to determine whether or not a variable's value can be changed. Previously, the logic had a bug that resulted in a failure if any child group contained a running processor that references an overridden variable and the user attempts to change the overridden variable at the higher level. We should not include any components of descendent groups if the descendent group overrides the variable.
...
This closes #3420 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2019-04-09 11:15:45 -04:00
Mark Payne
38db4e97cd
NIFI-5464: Consider ports invalid unless they have both incoming and outgoing connections
...
This closes #3417 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2019-04-09 10:19:24 -04:00
Rob Fellows
fb9f2af04a
NIFI-6136 - fix: UI - User and Template bug when page popped out
...
NIFI-6136 - return the appropriate value (false) when verifyDisconnectedCluster is called from a non-clustered environment.
This closes #3390
2019-04-08 15:45:19 -04:00
Alex Aversa
96279415a7
NIFI-6188 - Added 1s delay on Search control autocomplete query event to prevent premature results retrieval
...
This closes #3408 .
Signed-off-by: Brandon Devries <devriesb@apache.org>
2019-04-05 12:32:40 -04:00
Rob Fellows
e374d2cc30
NIFI-6140 - Incorrect title on the read only version of the Reporting Task detail dialog
...
This closes #3385
2019-04-04 13:40:14 -04:00
Rob Fellows
cb4fdc2ae1
NIFI-6126 - throttle requests to refresh the canvas to prevent too many server requests when holding down CTRL-R
...
* Added lodash utility library to leverage its throttle capability (and many other useful functions in the future).
* Made lodash available in all JSP's so it could be imported into nf-common (or any component for that matter).
* Added a throttle function to nf-common that just wraps _.throttle
This closes #3393
2019-04-04 09:51:41 -04:00
Mark Payne
76392ee862
NIFI-6155: Ensure that any task submitted to FlowEngine catches Throwable so that the task doesn't die just die silently in the case of an unexpected error/exception
...
This closes #3395 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2019-03-29 10:09:35 -04:00
Mark Payne
d3d43262db
NIFI-6150: Moved logic of decrementing content claims from ProcessSession.commit to the FlowFile Repository. Also updated load-balancing protocol and queue drop logic to no longer decrement content claims since the flowfile repo will handle the logic. This allows us ts to ensure that the claimant counts are decremented only if the repo is successfully updated and still before checking if the claim is still in use or not.
...
This closes #3391 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2019-03-27 10:22:01 -04:00
Joe Witt
1bdab73201
NIFI-6121 Updated the easy dependencies to update
...
This closes #3374
Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2019-03-22 09:01:21 -04:00
Matt Gilman
d35d15cdda
NIFI-6027
...
- Allowing user or group existence enforcement to be parameterized.
- Fixing error handling when loading user groups which may have resulted in stack trace leaking.
This closes #3377 .
Signed-off-by: Kevin Doran <kdoran@apache.org>
2019-03-20 13:38:46 -04:00
Rob Fellows
0cb15cfb1a
NIFI-6118 - Fix: UI - Not all calls to nfCanvasUtil.reloadConnectionSourceAndDestination are made with the proper component IDs
...
This closes #3368
2019-03-18 14:48:41 -04:00
Rob Fellows
6401e32a33
NIFI-6126 - Change nf-canvas to respond to keyup events rather than keydown
...
This closes #3376
2019-03-18 14:08:56 -04:00
Ryan Bower
160ade9f9d
NIFI-5018
...
- Added snap alignment for nf-label, label resize events, and nf-connection
- Shift key now disables snap alignment during the drag event.
- nf-connection load-balance-icon updated
This closes #3335
2019-03-11 13:52:09 -04:00
Mark Payne
c84f40ee36
NIFI-6111: Fixed bugs in the Status History values. If metrics are not available yet from all nodes for the last point on the graph, leave the cluster aggregate value off for the last point to prevent it from dropping significantly at the end of the chart. Fixed bug where counter values were not properly summed together in cluster view. Addressed issue with Average Task Duration
...
This closes #3361 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2019-03-08 17:27:49 -05:00
Mark Payne
58a25cfa5a
NIFI-6110: Updated StandardProcessSession such that if we fail to update the FlowFile Repository, we do not decrement claimant counts for any FlowFiles that were removed. Doing so can cause an issue where a FlowFile is removed, then the FlowFileRepo update fails, resulting in the flowfile being rolled back, but after its claimant count is decremented. It will then be processed again, which can result in the same thing, and we'll end up decrementing the claimant count repeatedly. Also updated LengthDelimitedJournal so that if the overflow directory already exists, it does not fail when trying to create the directory and instead just moves on. Updated unit tests to test both of these new fixes and updated DummyRecordSerde to be thread-safe because the TestLengthDelimitedJournal now needs it to be thread safe due to the new unit test that was added.
...
This closes #3358 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2019-03-08 10:32:38 -05:00
Kevin Doran
1d06044e3b
NIFI-6108: Fix port names when imported from registry
...
This closes #3356 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2019-03-06 17:07:18 -05:00
Bryan Bende
be6cd4f0f2
NIFI-6069 Fixing checkstyle error
2019-02-25 16:52:41 -05:00
Bryan Bende
4b45e85327
NIFI-6055 Improving error handling during auto-loading of NARs and ensuring only one Jetty NAR is loaded
...
NIFI-6056 Fixing cassandra bundle so nifi-mock is a test scoped and removing incorrect usage of ProviderCreation exception from nifi-framework-api
This closes #3322 .
Signed-off-by: Mark Payne <markap14@hotmail.com>
2019-02-22 12:19:12 -05:00
Bryan Bende
26c00f1080
NIFI-6069 Fixing fingerprinting of connection relationships to be properly sorted
...
This closes #3325 .
Signed-off-by: Mark Payne <markap14@hotmail.com>
2019-02-22 12:13:13 -05:00
Mark Payne
414bfc94fc
NIFI-6068: Fixed unit test
2019-02-22 11:55:25 -05:00
Mark Payne
e26bacf697
NIFI-6068, NIFI-6065: Updated StandardFunnel to avoid looping indefinitely and instead transfer no more than 10,000 FlowFiles before returning from onTrigger. Updated Local Ports to behavior in the same way. Updated Root Group Ports so that instead of blocking for up to 100 milliseconds for an incoming request, it blocks for up to 1 millisecond and if nothing is available yields for the 'bored yield duration'
...
Signed-off-by: Brandon Devries <devriesb@apache.org>
This closes #3328 .
2019-02-22 10:12:20 -05:00
Jan Hentschel
60b00a1e38
NIFI-2738 Removed references to nifi.cluster.request.replication.claim.timeout
...
This closes #2860
Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2019-02-21 06:34:27 -05:00
joewitt
25cc7b4a1e
NIFI-6029 merging nifi 1.9.0 release into master
2019-02-19 22:55:49 -05:00
Denes Arvay
76e92c8682
NIFI-6052 Update NOTICE files to reflect 2019
...
This closes #3319
Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2019-02-19 18:32:15 -05:00
joewitt
0e204f3576
NIFI-6029-RC2 prepare for next development iteration
2019-02-16 21:50:35 -05:00
joewitt
45bb53d2aa
NIFI-6029-RC2 prepare release nifi-1.9.0-RC2
2019-02-16 21:50:15 -05:00
Mark Payne
b508d6bfbc
NIFI-6033, NIFI-6034, NIFI-6035, NIFI-6036, NIFI-6037: Fixed bugs that were found during 1.9.0-RC1 validation. If multiple FlowFiles were written to same Content Claim, and a Processor attempted to read two of them wi within a single session, it would seek to the wrong part of the content or else throw a ContentNotFoundException. Updated logic for considering a processor to be 'running' / having 'active threads' if the processor is invalid upon NiFi restart but scheduled to run. Fixed NPE in FreeFormTextWriter. If MergeRecord reaches minimum number of records, flush writer after writing content out so that its minimum size can accurately be checked.
...
This closes #3309 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2019-02-15 11:59:47 -05:00
Mike Moser
f61947627e
NIFI-6038 Remove scope parameter from OIDC TokenRequest
...
This closes #3308
2019-02-14 16:43:15 -05:00
Kevin Doran
2938454ae4
NIFI-6020: Fix NPE in getAccessPoliciesForUser
...
This closes #3304
2019-02-13 16:03:46 -05:00
Dorian Bugeja
da8c8a14a1
NIFI-6017 - ArrayIndexOutOfBounds Load Balancer CorrelationAttributePartitioner
...
Signed-off-by: Mark Payne <markap14@hotmail.com>
2019-02-13 08:53:19 -05:00
Kevin Doran
a940ff6343
NIFI-5950: Use temp port names during flow updates
...
Use temporary values for port names when updated process groups as part of a Change Flow Version operation.
This avoids the potential for a name conflict between a ports during the update process.
Add a final step to the update process group logic to set the final name on all ports.
This closes #3301 .
2019-02-12 11:12:38 -05:00
Mark Payne
36c0a99e91
NIFI-5938: Added ability to infer record schema on read from JsonTreeReader, JsonPathReader, XML Reader, and CSV Reader.
...
- Updates to make UpdateRecord and RecordPath automatically update Record schema when performing update and perform the updates on the first record in UpdateRecord before obtaining Writer Schema. This allows the Writer to to inherit the Schema of the updated Record instead of the Schema of the Record as it was when it was read.
- Updated JoltTransformRecord so that schema is inferred on the first transformed object before passing the schema to the Record Writer, so that if writer inherits schema from record, the schema that is inherited is the trans transformed schema
- Updated LookupRecord to allow for Record fields to be arbitrarily added
- Implemented ContentClaimInputStream
- Added controller service for caching schemas
- UpdatedQueryRecord to cache schemas automatically up to some number of schemas, which will significantly inprove throughput in many cases, especially with inferred schemas.
NIFI-5938: Updated AvroTypeUtil so that if creating an Avro Schema using a field name that is not valid for Avro, it creates a Schema that uses a different, valid field name and adds an alias for the given field name so that the fields still are looked up appropriately. Fixed a bug in finding the appropriate Avro field when aliases are used. Updated ContentClaimInputStream so that if mark() is called followed by multiple calls to reset(), that each reset() call is successful instead of failing after the first one (the JavaDoc for InputStream appears to indicate that the InputStream is free to do either and in fact the InputStream is even free to allow reset() to reset to the beginning of file if mark() is not even called, if it chooses to do so instead of requiring a call to mark()).
NIFI-5938: Added another unit test for AvroTypeUtil
NIFI-5938: If using inferred schema in CSV Reader, do not consider first record as a header line. Also addressed a bug in StandardConfigurationContext that was exposed by CSVReader, in which calling getProperty(PropertyDescriptor) did not properly lookup the canonical representation of the Property Descriptor from the component before attempting to get a default value
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #3253
2019-02-11 12:56:50 -05:00
Mark Payne
3a4c3ccbaa
NIFI-6015: This closes #3300 . Updated unit test to sleep for 10 milliseconds before adding last FlowFile to queue so that the first and last FlowFiles will have different timestamps
...
NIFI-6015: Moved call to sleep() up in unit test to ensure that we sleep before creating the flowfile
Signed-off-by: joewitt <joewitt@apache.org>
2019-02-11 12:40:10 -05:00
Mark Payne
4e914cea1f
NIFI-6008: When a Processor is Terminated, call its #onPropertyModified method for properties whose value differs from the default value. Previously, it was called if the value differs from the current value (which it never will since we are only restoring the current values for the Processor). This results in ensuring that #onPr #onPropertyModified is correctly called when a component is reloaded
...
This closes #3296 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2019-02-08 10:52:04 -05:00
Bryan Bende
8c58d51857
NIFI-5990 This closes #3283 . Moving Developer docs section above Processors
...
Signed-off-by: joewitt <joewitt@apache.org>
2019-02-06 21:04:21 -05:00
thenatog
f81d6bd63b
NIFI-5968 - Added the X-XSS-Protection and Strict-Transport-Security HTTP headers using Jetty Filters. Added some tests.
...
Removed bad test.
Refactored filter creation method.
Ensure HSTS header is only applied if NiFi is secured with HTTPS
Small changes to header array list.
Fixed checkstyle errors.
This closes #3273 .
Signed-off-by: Andy LoPresto <alopresto@apache.org>
2019-02-07 11:13:28 +11:00
Peter Wicks
35147a620f
NIFI-5940 Cluster Node Offload Hangs if any RPG on flow is Disabled
...
This closes #3255
Signed-off-by: Mark Payne <markap14@hotmail.com>
2019-02-06 12:42:43 -05:00
Mark Payne
412c4908e2
NIFI-5997: Recover FlowFile Repository before swap files; then, when recovering swap files, ignore any that are unknown to the flowfile repo. This prevents us from incrementing the size of the flowfile queue for unknown swap files
...
This closes #3292 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2019-02-05 15:23:24 -05:00
Mark Payne
83ac191736
NIFI-5997: If we swap out data, ensure that we do not increment the size of the queue by the size of the data that we failed to swap out. Also, if the FlowFile Repo does not know about a given swap file, do not restore it on restart
...
This closes #3290 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2019-02-04 14:29:17 -05:00
Peter Wicks
ad1f2fb666
NIFI-5722 Expose Penalty Remaining Duration ( #3091 )
...
Signed-off-by: Koji Kawamura <ijokarumawak@gmail.com>
2019-02-01 13:04:51 -07:00
Bryan Bende
79a3696e5e
NIFI-5961 Fixing bug where the name of a versioned process group gets incorrectly reset
...
This closes #3271 .
Signed-off-by: Mark Payne <markap14@hotmail.com>
2019-01-30 11:14:53 -05:00
Mark Payne
fd3d69bc90
NIFI-5949: Addressed problematic error handling logic in PutKudu. Also removed AbstractKudu and moved all logic into PutKudu, as the abstract class was clearly designed to be extended only by PutKudu and no other processors exist for Kudu
...
Added explicit .toString() call on rowError to avoid IllegalStateException during logging of row errors
This closes #3262 .
2019-01-24 19:47:03 -05:00
Mark Payne
89adb039d2
NIFI-5944: If processor is no longer scheduled to run after a failed trigger of @OnScheduled, ensure that we set scheduled status to STOPPED
...
This closes #3263
2019-01-11 10:22:22 -05:00
Mark Payne
706cf7dcff
NIFI-5944: When components are started on NiFi startup, if they are invalid, don't fail immediately and give up. Instead, keep attempting to start the component when it becomes valid.
...
This closes #3259
2019-01-10 11:15:17 -05:00
Mark Payne
cf41c10546
NIFI-5879: Fixed bug in FileSystemRepository that can occur if an InputStream is obtained, then more data is written to the Content Claim - the InputStream would end before allowing the sequential data to be read. Also fixed bugs in LimitedInputStream related to available(), mark(), and reset() and the corresponding unit tests. Additionally, found that one call to StandardProcessSession.read() was not properly flushing the output of any Content Claim that has been written to before attempting to read it.
...
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes #3207
2019-01-07 11:06:17 -05:00
Mark Payne
fea17d0ca8
NIFI-5919: Addressed a race condition that can exist if adding FlowFiles to a FlowFileQueue before adjusting the size of the queue to account for the FlowFiles
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #3238 .
2019-01-03 11:35:23 +01:00