Commit Graph

94 Commits

Author SHA1 Message Date
Joe Witt 75e7c9e47c
NIFI-10854-RC1 prepare for next development iteration 2022-11-22 19:22:47 -07:00
Joe Witt ec87bf93ad
NIFI-10854-RC1 prepare release nifi-1.19.0-RC1 2022-11-22 19:22:45 -07:00
Mark Payne bda624823b
NIFI-10817: Moved the calls in StandardProcessSession to 'resetState()' and 'acknowledgeRecords()' from the outer commit(boolean) to the inner commit(Checkpoint, boolean). By moving the call here, the logic of StandardProcessSession is unaffected. But the StatelessProcessSession that inherits from it now has the benefit of having the state cleaned up when calling super.commit(Checkpoint, boolean).
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6663
2022-11-15 15:47:32 -05:00
Timea Barna 7c33516aee NIFI-9437 Flowfile Expiration cannot exceed 24 days
NIFI-9437 Addressing review comments, correcting setter in StatelessFlowQueue class

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #6558.
2022-10-24 14:46:56 -04:00
Joe Witt c9ebdd2025
NIFI-10521-RC4 prepare for next development iteration 2022-10-03 10:59:36 -07:00
Joe Witt 109e54cd58
NIFI-10521-RC4 prepare release nifi-1.18.0-RC4 2022-10-03 10:59:34 -07:00
simonbence 94fdc794b0
NIFI-10497 Making RegistryClient an extension point (#6433)
- NIFI-10497 Mae FlowRegistryClient an extension point
2022-09-26 12:16:21 -04:00
Mark Payne 3f0ef9e977
NIFI-10406: Updated Maven assemblies to ensure that we always use at least two of the GAVC parameters for includes and excludes. Updated Maven assembly plugin to version 3.4.2. Removed some includes and excludes from dependencies that previously were not matching and were generating Maven WARNING messages.
This closes #6415

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-09-15 11:02:08 -05:00
Joe Gresock de7793e69b
NIFI-9003 Added framework support for Parameter Providers
- Added Environment Variable and File Parameter Provider implementations
- Added Parameter Provider commands to CLI
- Added component search handling for Parameter Providers
- Added documentation for integrating Parameter Providers

This closes #5369

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-09-09 13:04:56 -05:00
Mark Payne a0c705715b
NIFI-10429: Added the ability to Replay latest provenance event for a given Processor.
This closes #6359

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-09-08 15:55:26 -05:00
Mark Payne 6b424c3fd3
NIFI-10375: If a class is not allowed in the AllowListClassLoader by classname, check the class's module and allow through anything in the java. or jdk. modules.
NIFI-10375: Addressed review feedback: removed loading of classes from JMODs in the StatelessBootstrap because it only was relevant if using JDK. Instead, just inspect the module as we do when using the JRE. Also addressed issue of allow NoClassDefFoundError fly when we should use ClassNotFoundException

This closes #6317.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2022-08-24 16:45:31 +02:00
Joe Witt 8c66bf948c
NIFI-10272-RC2 prepare for next development iteration 2022-07-27 13:24:09 -07:00
Joe Witt 8d256784d8
NIFI-10272-RC2 prepare release nifi-1.17.0-RC2 2022-07-27 13:24:06 -07:00
Mark Payne 5e83bda9e9
NIFI-10167: Added advanced timing metrics for processors, such as CPU Usage, time reading/writing content repo, process session commit time, etc. Exposed via nifi.sh diagnostics and made configurable via nifi.properties
This closes #6156

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-06-30 10:50:24 -05:00
timeabarna d5b626f0e4
NIFI-10108 Processor scheduling via parameter (#6115)
- NIFI-10108 Processor scheduling via parameter
- Refactoring component referencing check in AbstractComponentNode and StandardParameterReferenceManager classes.
2022-06-30 10:11:13 -04:00
Mark Payne 1442dcef23
NIFI-10111: Added option to unpack NARs into an Uber Jar and updated stateless nifi to always use this.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6117
2022-06-24 13:12:14 -04:00
Nathan Gough 62eb565daf
NIFI-5402 - Added more assembly options for different modules. Assemblies should build a zip by default, or a tar.gz with the -Ptargz profile
This closes #5694
Signed-off-by: Paul Grey <greyp@apache.org>
2022-06-23 13:52:41 -04:00
exceptionfactory bd45eb4995 NIFI-10100 Upgraded Jersey to 2.35
- Replaced individual version references with Jersey BOM dependency

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #6106.
2022-06-16 23:09:22 -04:00
exceptionfactory 99be62be67
NIFI-9958 Add Framework Support for Sensitive Dynamic Properties (#6057)
* NIFI-9958 Implemented Sensitive Dynamic Properties

- Added SupportsSensitiveDynamicProperties annotation for components
- Added optional sensitive query parameter to Property Descriptor REST Resource methods
- Added system tests for components supporting sensitive dynamic properties
- Updated REST Resources to support Sensitive Dynamic Property Names
- Updated Documentation Writer to indicate component support for Sensitive Dynamic Properties
- Updated InvokeHTTP to support Sensitive Dynamic Properties
- Updated Auditor components to handle masking Sensitive Dynamic Properties

* Refactored Property Descriptor REST method handling

- Corrected AbstractDocumentationWriter evaluation of support for sensitive dynamic properties
- Refactored Controller Service Dynamic Properties system tests to new class

* Updated AbstractComponentNode.getProperties() to get canonical descriptor
2022-05-23 17:20:54 -04:00
exceptionfactory 85d5e2071b
NIFI-10031 Downgraded Jakarta Mail from 2.0.1 to 1.6.7 (#6052)
- Downgraded Jakarta Activation from 2.0.1 to 1.2.2 in nifi-bootstrap
- Updated assembly configurations to include Jakarta Activation in java11 directory
2022-05-18 11:38:36 -04:00
Mark Payne 9311188785
NIFI-10004: Allow loading .class files that are relative to the NAR directory, in addition to the .jar / .nar files
NIFI-10004: When determining classname from a file, consider File.separator instead of just /
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #6026
2022-05-09 14:54:01 -04:00
exceptionfactory 4f423a59ba
NIFI-9952 Upgraded Jackson BOM to 2.13.2.20220328
- Removed unnecessary references to jackson.version property
- Removed unnecessary dependency management references to Jackson libraries

This closes #5992

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-04-23 08:23:39 -04:00
Mark Payne 5a3c7d1fbd
NIFI-9940: Renamed StandardProcessGroupSynchronizer to StandardVersionedComponentSynchronizer. Added synching for processors, input/output ports, connections, etc. Added unit tests.
This closes #5983
2022-04-21 10:28:21 -04:00
Mark Payne aeaec170d5 NIFI-9861: Removed BlockListClassLoader in favor of AllowListClassLoader
NIFI-9861: Fixed stateless-processor-tests assembly to ensure that all necessary libraries were included; removed BlockListClassLoader
NIFI-9861: Fixed issue in which we would list .class files as files that we allow through the AllowListClassLoader but didn't allow them.

This closes #5925.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2022-04-20 22:49:43 +02:00
Joe Witt 873d25585c
NIFI-9780 Merge branch 'NIFI-9780-RC3' 2022-04-04 08:35:23 -07:00
Mark Payne 2e66cf7114 NIFI-9854: Include slf4j-api as runtime scope for nifi-stateless-system-test-suite; When constructing the classes to block for the stateless ClassLoader, allow through any classes that come from Java Home's lib/ directory
This closes #5920.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2022-04-01 17:51:07 +02:00
markap14 940fd8e81c
NIFI-9847: Switched LifecycleState to use a WeakHashMap to track Acti… (#5917)
* NIFI-9847: Switched LifecycleState to use a WeakHashMap to track ActiveProcessSessionFactory instances, instead of a regular Set that removed the instance after calling onTrigger. This was necessary for processors such as MergeRecord that may stash away an ActiveProcessSessionFactory for later use, as we need to be able to force rollback on processor termination

* NIFI-9847: Fixed checkstyle violation
2022-03-30 15:25:14 -04:00
exceptionfactory 0fa4490a98
NIFI-9842 Refactored nifi-commons using JUnit 5
- Refactored nifi-bootstrap using JUnit 5
- Refactored nifi-maven-archetypes using JUnit 5
- Refactored nifi-stateless using JUnit 5

This closes #5912

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-03-28 16:00:23 -04:00
Mark Payne 16c9bd535b NIFI-9834: When calling ByteArrayContentRepository.read() on a null Content Claim, return an empty ByteArrayInputStream instead of throwing NullPointerException
This closes #5903.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2022-03-25 21:07:25 +01:00
Joe Witt 0419dc2939
NIFI-9780-RC3 prepare for next development iteration 2022-03-21 11:58:27 -07:00
Joe Witt b019a9191f
NIFI-9780-RC3 prepare release nifi-1.16.0-RC3 2022-03-21 11:58:09 -07:00
Joe Witt 72fadf9e51
NIFI-9780 fixing many inconsistent pom project declarations and updated org apache parent version 2022-03-10 11:15:16 -07:00
Mark Payne 8959226b50
NIFI-9754: Introduced VersionedExternalFlow
- Updated stateless and StandardProcessGroup, etc. to make use of VersionedExternalFlow
- Updated StatelessDataflowDefinition to use ExternalVersionedFlow instead of generic type
- Updated Stateless Bootstrap to avoid loading stateless engine libs from root class path but instead use a NarClassLoader to load the statelss nar

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5832.
2022-03-04 07:19:38 -05:00
Mark Payne d0a23bc26b
NIFI-9704: Updated the ContentRepositoryScanTask to show details of how much content in the content repo is retained by each queue in the dataflow. Changed default for nifi.content.claim.max.appendable.size property from 1 MB to 50 KB. Updated docs to reflect the new default value and explain what the property does and how it's used.
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5780.
2022-02-22 12:06:54 -05:00
Mark Payne 2aa6bd1e13
NIFI-9689: When all FlowFiles in a FlowFile Queue are penalized, do not schedule the destination to run. Also expose this fact via the ConnectionStatusSnapshotDTO, as this allows the front-end to render this information to the user in order to avoid confusion when it appears that the Processor has data but does nothing
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5771
2022-02-16 15:30:00 -05:00
Joe Gresock 4210e30047
NIFI-9620: Adding isStateful to StatelessDataflow (#5703)
* Adding isStateless method to reporting task and controller service
2022-02-16 13:30:40 -05:00
exceptionfactory ba0bf2ede0
NIFI-9650 Upgraded OkHttp from 4.9.2 to 4.9.3
- Added okhttp-bom to root Maven configuration
- Removed repeated okhttp.version property from specific dependencies

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5737.
2022-02-02 11:45:18 +01:00
Peter Turcsanyi ed5e90fb81
NIFI-9576: Allowed Stateless' BlockListClassLoader to load java11 jars/classes (#5726) 2022-01-28 13:37:33 -05:00
Joe Gresock 278967829a
NIFI-9626: Allowing Stateless NiFi to parse flow snapshots with unrecognized fields (#5709) 2022-01-26 12:15:27 -05:00
Mark Payne 1617406041 NIFI-9576: Introduced a BlockListClassLoader that can be used by stateless in order to isolate both the Stateless Engine and the NiFi extensions from extraneous classes that exist in the System ClassLoader
This closes #5705.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2022-01-26 17:28:15 +01:00
Mark Payne b5dd35431e NIFI-9293: Ensure that we properly set the scheduled flag in the LifecycleState when stopping processors. Added system test to verify that @OnScheduled, onTrigger, @OnUnscheduled, @OnStopped are all called and in the expected order
This closes #5706.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2022-01-24 22:07:31 +01:00
Joe Witt 2f95eed8cb
NIFI-9575 This closes #5667. updating copyright year to 2022
Signed-off-by: Joe Witt <joewitt@apache.org>
2022-01-19 09:14:09 -07:00
Mark Payne 72e54f4fab NIFI-9390: Updates to MergeContent / MergeRecord so that they play nicely within Stateless
NIFI-9390: Addressed underlying condition in stateless framework that caused Merge-related processors and similar to not properly be triggered as necessary. Added several system tests to verify different configurations.

NIFI-9390: Simplified the logic for how to iterate over the components in a Stateless flow that are ready to be triggered

This closes #5634.

Co-authored-by: Peter Turcsanyi <turcsanyi@apache.org>
Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2022-01-11 09:45:00 +01:00
Lehel Boér be138ab656
NIFI-9434: Added bulletin to alert FileSystemRepository backpressure (#5596) 2022-01-04 15:59:32 -05:00
exceptionfactory 7c6bdcb035
NIFI-9491 Excluded commons-logging and added jcl-over-slf4j references
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5608
2021-12-17 07:21:03 -05:00
Joe Gresock 429087c11d
NIFI-9386: Adding status task schedule to Stateless engine config
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5532.
2021-12-13 12:08:54 +01:00
Peter Turcsanyi dafa03a21a
NIFI-9421: Running NiFi Stateless with local NARs only
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5557.
2021-12-08 15:59:31 -05:00
markap14 839fbf7d19
NIFI-9382: Created a new ClassloaderIsolationKey mechanism by which H… (#5533)
* NIFI-9382: Created a new ClassloaderIsolationKey mechanism by which Hadoop related processors (and potentially others) can indicate that they need full classloaders to be cloned but can share with other instances in certain circumstances
- Added system tests

* NIFI-9382: Renamed interface based on review feedback

* NIFI-9382: Removed ReentrantKerberosUser.
2021-11-22 14:55:19 -05:00
Joe Witt 9e845d87ff
Merge branch 'NIFI-9344-RC3' 2021-11-07 16:23:10 -07:00
Mark Payne 373498445f
NIFI-9364: Ensure that we delegate calls to write(byte[]) and write(byte[], int, int) to the underlying OutputStream when writing to the file-based content repository for stateless
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5508.
2021-11-04 16:48:24 -04:00