142 Commits

Author SHA1 Message Date
Pierre Villard
7fedb49a9a
NIFI-12362-RC5 prepare for next development iteration 2023-11-23 11:03:53 +01:00
Pierre Villard
5241f43482
NIFI-12362-RC5 prepare release nifi-1.24.0-RC5 2023-11-23 11:03:47 +01:00
exceptionfactory
0a67620c0c
NIFI-12358 Corrected syntax for Java 8
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2023-11-15 17:07:48 -06:00
Mark Payne
a78c478721
NIFI-12358 Fixed NPE in HostHeaderHandler
Fixed NPE in ThreadDumpTask

This closes #8032

Signed-off-by: David Handermann <exceptionfactory@apache.org>

(cherry picked from commit b5c3e6f9aae9791b749ba3d908ce1b6672ee2124)
2023-11-15 16:59:59 -06:00
exceptionfactory
27a690a30a NIFI-11832-RC3 prepare for next development iteration 2023-07-25 14:03:12 -05:00
exceptionfactory
b7792f03b7 NIFI-11832-RC3 prepare release nifi-1.23.0-RC3 2023-07-25 14:03:11 -05:00
Joe Witt
f1d5a8c4de
NIFI-11654-RC1 prepare for next development iteration 2023-06-06 12:32:00 -07:00
Joe Witt
71e3ea9f2c
NIFI-11654-RC1 prepare release nifi-1.22.0-RC1 2023-06-06 12:31:58 -07:00
Joe Witt
9c27ddcb1c
NIFI-11357-RC2 prepare for next development iteration 2023-04-03 14:40:08 -07:00
Joe Witt
892f822107
NIFI-11357-RC2 prepare release nifi-1.21.0-RC2 2023-04-03 14:40:07 -07:00
Joe Witt
8300477ed1
NIFI-11103 preparing support/nifi-1.x line for any subsequent feature/maint releases starting with 1.21-snap 2023-02-09 15:40:01 -07:00
Joe Witt
896fdacbbd
NIFI-11103-RC1 prepare for next development iteration 2023-02-06 11:41:41 -07:00
Joe Witt
81296b5b69
NIFI-11103-RC1 prepare release nifi-1.20.0-RC1 2023-02-06 11:41:39 -07:00
Joe Witt
75e7c9e47c
NIFI-10854-RC1 prepare for next development iteration 2022-11-22 19:22:47 -07:00
Joe Witt
ec87bf93ad
NIFI-10854-RC1 prepare release nifi-1.19.0-RC1 2022-11-22 19:22:45 -07:00
Ferenc Erdei
86acde2fe2
NIFI-10750 Move C2NiFiClientService to minifi-framework
Signed-off-by: Csaba Bejan <bejan.csaba@gmail.com>

This closes #6612.
2022-11-04 08:53:53 +01:00
Timea Barna
7c33516aee NIFI-9437 Flowfile Expiration cannot exceed 24 days
NIFI-9437 Addressing review comments, correcting setter in StatelessFlowQueue class

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #6558.
2022-10-24 14:46:56 -04:00
Joe Witt
c9ebdd2025
NIFI-10521-RC4 prepare for next development iteration 2022-10-03 10:59:36 -07:00
Joe Witt
109e54cd58
NIFI-10521-RC4 prepare release nifi-1.18.0-RC4 2022-10-03 10:59:34 -07:00
simonbence
94fdc794b0
NIFI-10497 Making RegistryClient an extension point (#6433)
- NIFI-10497 Mae FlowRegistryClient an extension point
2022-09-26 12:16:21 -04:00
Joe Gresock
de7793e69b
NIFI-9003 Added framework support for Parameter Providers
- Added Environment Variable and File Parameter Provider implementations
- Added Parameter Provider commands to CLI
- Added component search handling for Parameter Providers
- Added documentation for integrating Parameter Providers

This closes #5369

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-09-09 13:04:56 -05:00
Mark Payne
a0c705715b
NIFI-10429: Added the ability to Replay latest provenance event for a given Processor.
This closes #6359

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-09-08 15:55:26 -05:00
Joe Witt
8c66bf948c
NIFI-10272-RC2 prepare for next development iteration 2022-07-27 13:24:09 -07:00
Joe Witt
8d256784d8
NIFI-10272-RC2 prepare release nifi-1.17.0-RC2 2022-07-27 13:24:06 -07:00
timeabarna
d5b626f0e4
NIFI-10108 Processor scheduling via parameter (#6115)
- NIFI-10108 Processor scheduling via parameter
- Refactoring component referencing check in AbstractComponentNode and StandardParameterReferenceManager classes.
2022-06-30 10:11:13 -04:00
Matthew Burgess
d89f98f5b4
NIFI-9666 Implemented MiNiFi C2 client and refactored bootstrap
- Implemented MiNiFi C2 client using OkHttp
- Refactored MiNiFi bootstrap command handling and socket communication
- Added C2 Client Service to nifi-framework-core

This closes #6075

Co-authored-by: Matthew Burgess <mattyb149@apache.org>
Co-authored-by: Csaba Bejan <bejan.csaba@gmail.com>
Co-authored-by: Ferenc Erdei <ferdei@cloudera.com>

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-06-17 09:23:52 -05:00
exceptionfactory
15f7590f7a
NIFI-9901 Added nifi-xml-processing to nifi-commons
- Refactored XML parsing to use providers from nifi-xml-processing
- Configured spotbugs-maven-plugin with findsecbugs-plugin in nifi-xml-processing
- Disabled Validate DTD in default configuration for EvaluateXPath and EvaluateXQuery
- Replaced configuration of DocumentBuilder and streaming XML Readers with shared components
- Removed XML utilities from nifi-security-utils
- Moved Commons Configuration classes to nifi-lookup-services

This closes #5962
Signed-off-by: Paul Grey <greyp@apache.org>
2022-04-13 19:56:57 -04:00
Joe Witt
0419dc2939
NIFI-9780-RC3 prepare for next development iteration 2022-03-21 11:58:27 -07:00
Joe Witt
b019a9191f
NIFI-9780-RC3 prepare release nifi-1.16.0-RC3 2022-03-21 11:58:09 -07:00
Mark Payne
d0a23bc26b
NIFI-9704: Updated the ContentRepositoryScanTask to show details of how much content in the content repo is retained by each queue in the dataflow. Changed default for nifi.content.claim.max.appendable.size property from 1 MB to 50 KB. Updated docs to reflect the new default value and explain what the property does and how it's used.
Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5780.
2022-02-22 12:06:54 -05:00
Mark Payne
2aa6bd1e13
NIFI-9689: When all FlowFiles in a FlowFile Queue are penalized, do not schedule the destination to run. Also expose this fact via the ConnectionStatusSnapshotDTO, as this allows the front-end to render this information to the user in order to avoid confusion when it appears that the Processor has data but does nothing
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5771
2022-02-16 15:30:00 -05:00
exceptionfactory
0da3994ac8
NIFI-9606 This closes #5688. Removed nifi-security-utils from nifi-framework-api
Signed-off-by: Joe Witt <joewitt@apache.org>
2022-01-20 10:53:41 -07:00
Lehel Boér
be138ab656
NIFI-9434: Added bulletin to alert FileSystemRepository backpressure (#5596) 2022-01-04 15:59:32 -05:00
Mark Payne
90b39b593a
NIFI-9069 Changed framework dataflow serialization to support JSON
- Changed framework so that it serializes the dataflow into a VersionedDataflow using JSON as well as XML, and prefers the JSON representation on load, if it's available. This also results in the need for the cluster protocol to exchange its representation of the dataflow to using JSON. Rather than re-implementing all of the complex logic of Flow Fingerprinting, updated to just inherit the cluster's flow.
- Moved logic to synchronize Process Group with Versioned Process Group into a new ProcessGroupSynchronizer class instead of having all of the logic within StandardProcessGroup
- Reworked versioned components to use an instance id.
- Renamed StandardFlowSynchronizer to XmlFlowSynchronizer; introduced new StandardFlowSynchronizer that delegates to the appropriate (Xml or Versioned)FlowSynchronzer
- Updated to allow import of VersionedProcessGroup even if not all bundles are available - will now use ghost components
- Introduced a VersionedDataflow object to hold controller-level services, reporting tasks, parameter contexts, templates, etc.
- Allow mutable requests to be made while nodes are disconnected. Also fixed issue in AbstractPolicyBasedAuthorizer that caused ClassNotFoundException / NoClassDefFoundError if the authorizations were changed and then a node attempts to rejoin the cluster. The Authorizer was attempting to use XmlUtils, which is in nifi-security-utils and so so by madking nifi-security-utils a provided dependency of nifi-framework-api, but this doesn't work, because nifi-framework-api is loaded by a higher-level classloader, so the classloader that loads AbstractPolicyBasedAuthorizer will never have the appropriate classloader to provide nifi-security-utils. Addressed this by copying the code for creating a safe document builder from XmlUtils to AbstractPolicyBasedAuthorizer.
- Fixed bug that occurred when importing a Process Group that has 2 parameter contexts, one inheriting from another, where neither is pre-defined in the existing flow
- Fixed bug that was encountered when Updating a Versioned Process Group where one version had a disabled processor and the other had the processor running.
- Increased system-tests workflow timeout to 120 minutes
- Added additional exception handling to system tests

This closes #5514

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-12-02 15:10:38 -06:00
Joe Witt
c055895952
NIFI-9344-RC3 prepare for next development iteration 2021-11-03 08:53:37 -07:00
Joe Witt
7fdc07cccd
NIFI-9344-RC3 prepare release nifi-1.15.0-RC3 2021-11-03 08:53:32 -07:00
Lehel Boér
f7df326521 NIFI-9183: Add a command-line option to save status history
Fixed typo error.

Fixed error.

Fixed review items.

Fix contrib-check. Added missing test file to excludes.

Fix review items.

Fix review items: error handling, input validation, added more unit tests.

Improved status history repository creation in HeadlessNiFiServer.
2021-10-12 16:35:20 +02:00
Mike Thomsen
6a55c6ca44
NIFI-9081 Updated nifi-framework-api tests to JUnit 5
- Updated nifi-external tests to JUnit 5
- Updated nifi-stateless to use JUnit 5
- Updated nifi-api to use JUnit 5

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-09-11 14:57:38 -05:00
Hsin-Ying Lee
59f0b6bac3
NIFI-8739 Penalized flowfiles should be able to be polled from the queue in some cases (#5189) 2021-08-30 11:26:02 -04:00
Mark Payne
fb4edfa0d6
NIFI-8950: Added ability to optionally use a file-system backed Content Repository for Stateless NiFi
This closes #5254

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-08-11 12:01:07 -05:00
Joe Witt
97feacc181
NIFI-8767-RC2 prepare for next development iteration 2021-07-10 12:17:09 -07:00
Joe Witt
fcbf1d5f97
NIFI-8767-RC2 prepare release nifi-1.14.0-RC2 2021-07-10 12:17:05 -07:00
Jon Kessler
2309d75d3d NIFI-8126
- Added totalActiveQueuedDuration and maxActiveQueuedDuration to the ConnectionStatus object
- Updated FlowFileQueue implementations and supporting code to properly calculate and provide the totalActiveQueuedDuration and maxActiveQueuedDuration for their active queues
- Fixing failing unit test. In examining this it appears that the unit test only accidentally passed in the past and that the object mocked to always throw an exception was not actually being used in the test.
- Adding UI component via ConnectionStatusDescriptor along with updates based on PR comments.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2021-05-18 17:09:53 -04:00
Mark Payne
935566ba23 NIFI-8433 Added ability to decommission a node in a cluster
This closes #5004

Signed-off-by: Joey Frazee <jfrazee@apache.org>
2021-05-01 12:45:48 -07:00
markap14
321e979673
NIFI-8353: When receiving data via load-balanced connection, throw an Exception (resulting in an ABORT_TRANSACTION status code) when attempting to add received FlowFiles to the FlowFile queue, if the node is not currently connected to cluster. In this case, ensure that we remove the received FlowFiles from the FlowFile Repository and emit a DROP event to the repository (with appropriate details) to coincide with the already-committed Provenance events. Also ensure that when a node is disconnected that we don't keep sending to that node until it reconnects. While testing the fixes via LoadBalancedQueueIT, also noticed that unit test logging was not working properly in nifi-framework-core because of the (erroneous) dependency on logback-class; removed this dependency and updated unit tests that were created that depended on its existence. (#4924) 2021-03-23 14:17:54 -04:00
Bence Simon
7730777d66 NIFI-8113 Adding persistent status history repository backed by embedded QuestDB
Signed-off-by: Mark Payne <markap14@hotmail.com>
2021-02-25 15:59:01 -05:00
Mark Payne
528fce2407
NIFI-7646, NIFI-8222: Instead of having StandardProcessSession call ContentRepository.read(ContentClaim), introduced a new ContentRepository.read(ResourceClaim) and hold open the InputStream to the ResourceClaim. This can't be supported by EncryptedContentRepository, so introduced a method to allow using this or not. The benefit here is that when we have many FlowFiles read within a session, such as when using MergeContent/MergeRecord or a processor configured with a Run Duration, we can hold open a single InputStream instead of constantly opening FileInputStreams and seeking to the appropriate location. This is much faster.
- Instead of entering a 'synchronized' block for every provenance event, serialize up to 1 MB worth of data, and then enter synchronized block to write that data out. This avoids large amounts of lock contention and context switches

NIFI-7646: Removed TODO and unused Jackson dependency

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4818
2021-02-23 10:57:54 -05:00
Joe Witt
88fab00e29
NIFI-7873 merging release branch to latest and updating to 1.14.0-SNAPSHOT 2021-02-15 12:09:32 -07:00
Joe Witt
4afb2ba743
NIFI-7873-RC4 prepare for next development iteration 2021-02-15 12:09:31 -07:00
Joe Witt
487280bee9
NIFI-7873-RC4 prepare release nifi-1.13.0-RC4 2021-02-15 12:09:30 -07:00