5023 Commits

Author SHA1 Message Date
Tamas Palfy
635824904d
NIFI-9609 Added nifi-snowflake-bundle with SnowflakeComputingConnectionPool
- Updated DBCPConnectionPool for extension

This closes #5692

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-02-21 19:38:47 -06:00
lawye1973
8d80fb2930
NIFI-9694 : Correct FetchS3Object.java provenance data URL
Current provenance data URL is incorrect as it's hardcoded.
Follow PutS3Object.java method, using AmazonS3Client (instead of AmazonS3) to use getResourceUrl(bucket,key) to build the correct URL for provenance data

This closes #5779

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-02-19 18:46:20 -05:00
levilentz
dfbf2e3cea
NIFI-9286: JOLT Expression Language
Fixes NIFI-6213 and adds in functionality to use expression language in class and module specification
NIFI-9286: adding JOLT unit tests
NIFI-9286: addressing PR feedback
Fixes a problem with the scope of the EL for module directory
NIFI-9286: alignment of JOLT processors
NIFI-9286: fix checkstyle

This closes #5444

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-02-19 14:11:44 -05:00
Nathan Gough
7ef2fd2986
NIFI-7333 Added OIDC trust store strategy property
This closes #5753

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-02-18 15:19:22 -05:00
Roberto Marturano
a8fd5993eb NIFI-7192 Added systemd reload to nifi.sh install on systemd servers
This closes #4079

Signed-off-by: Joey Frazee <jfrazee@apache.org>
2022-02-18 10:40:52 -08:00
Matthew Burgess
2f3380390d NIFI-8492: Addressed issue in DatabaseReader class of IPLookupService that was attempting to set values on the JSON returned by MaxMind. Instead of modifying the object directly, we should use an Injectable in the Reader so that the value read will have the appropriate values but we don't need to modify those objects returned by MaxMind. Similar solution of NIFI-5814
Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5777.
2022-02-17 12:55:12 -05:00
Mike Thomsen
a837397d56
NIFI-8209 Added Neo4J 4.X and 3.X clients by splitting the current controller service along release lines. This was necessary because Neo4J broke compatibility in their client drivers for Java between 3.X and 4.X at the Java API level.
NIFI-8209 Updated module name.
NIFI-8209 Updated parent module.
NIFI-8209 Renamed a few misnamed modules.
NIFI-8209 Updated 1.15.0-SNAPSHOT references in cypher v3 package.
NIFI-8209 Updated neo4j 4.x driver.

NIFI-8209 Made changes requested in a code review.

NIFI-8209: Fixed NAR module name
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5301
2022-02-17 10:28:12 -05:00
Matthew Burgess
03165ad817 NIFI-9607: Honor Update Keys when Quoting Identifiers in PutDatabaseRecord
NIFI-9607: Fixed wrong column name in WHERE clause for generateUpdate

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5701.
2022-02-16 16:07:34 -05:00
Mark Payne
4d8c79d7f3 NIFI-9698: When creating an Avro schema, ensure that any default value is converted from what is returned by RecordField.getDefaultValue() to what Avro requires.
Signed-off-by: Chris Sampson <chriss@apache.org>

This closes #5776
2022-02-16 20:37:50 +00:00
Mark Payne
2aa6bd1e13
NIFI-9689: When all FlowFiles in a FlowFile Queue are penalized, do not schedule the destination to run. Also expose this fact via the ConnectionStatusSnapshotDTO, as this allows the front-end to render this information to the user in order to avoid confusion when it appears that the Processor has data but does nothing
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5771
2022-02-16 15:30:00 -05:00
Joe Gresock
4210e30047
NIFI-9620: Adding isStateful to StatelessDataflow (#5703)
* Adding isStateless method to reporting task and controller service
2022-02-16 13:30:40 -05:00
exceptionfactory
fe1139b8bb
NIFI-9692 Upgraded Apache Commons Lang3 to 3.12.0
- Moved commons-lang3 version management to root Maven configuration
- Refactored limited usage of StringUtils is nifi-reporting-utils to remove commons-lang3
- Refactored limited usage of StringUtils in nifi-websocket-processors to remove commons-lang3

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5773
2022-02-16 09:28:21 -05:00
Gabor Gyimesi
6788076536
NIFI-9058 Corrected AttributesToJSON Core Attributes filtering
- Avoiding filtering Core Attributes from configured Attributes List

This closes #5312

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-02-15 15:51:39 -06:00
exceptionfactory
7dfbf99d5f
NIFI-9527 Upgraded snappy-java to 1.1.8.4
- Set managed version in root Maven configuration

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5760
2022-02-15 16:12:36 -05:00
timeabarna
63a543cd67
NIFI-9475 Provide Framework-Level Retries for NiFi Relationships (#5593) 2022-02-15 15:40:41 -05:00
tpalfy
84b2484fd4
NIFI-9684 Fix: When starting/stopping selected process group, it sends the parent process group id to the REST interface that is responsible to enable/disable transmission for all remote process groups within a process group. Need to send the id of the select process group instead. (#5764) 2022-02-15 14:55:17 -05:00
greyp9
87cfd43f6f
NIFI-9233 - Improve reliability of system integration tests (#5749)
* NIFI-9233 - Improve reliability of system integration tests
2022-02-15 13:14:01 -05:00
Lehel
51202d7c2d
NIFI-9686: Renamed SNMP integration tests correctly
This closes #5769

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-02-15 11:07:34 -06:00
Steve Lawrence
4815df29db
NIFI-6740: Add configuration options to specify NiFi/Bootstrap communication ports
The NiFi and NiFi Bootstrap processes both bind to random ephemeral
ports to allow for inter-process communication (e.g. shutdown, port,
ping, etc.). However, the randomness of these ephemeral ports can pose
challenges for some security policies and firewall rules.

This adds two configuration options, nifi.bootstrap.listen.port and
nifi.listener.bootstrap.port, that allow an administrator to define
which ports the two processes should bind to for this communication,
making it easier to define security policies. The options default to
zero to maintain the current ephemeral port behavior.

NIFI-6740: Add configuration options to specify NiFi/Bootstrap communication ports

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5746
2022-02-15 11:46:21 -05:00
exceptionfactory
b09a778d90
NIFI-9685 Upgraded JNA to 5.10.0
- Upgraded nifi-bootstrap-utils JNA from 4.4.0 to 5.10.0
- Upgraded nifi-windows-event-log-processors JNA from 4.5.2 to 5.10.0
- Upgraded nifi-toolkit-cli JLine from 3.5.2 to 3.21.0

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5766.
2022-02-15 10:52:54 +01:00
Scott Aslan
b8ec6d38dc [NIFI-9554] upgrade node and npm
update lintin
update font awesome path
bump nifi-fds version
update nifi deps
update frontend maven plugin
change CI npm cache restore keys so that we will not use cached directories for older versions of node

This closes #5751.

Signed-off-by: Kevin Doran <kdoran@apache.org>
2022-02-14 14:24:02 -05:00
exceptionfactory
f6f50e6b3a
NIFI-9681 Upgraded Apache Commons DBCP to 2.9.0
- Replaced module references to version 2.7.0 with managed dependency in nifi-nar-bundles

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5763.
2022-02-14 18:30:36 +01:00
Mike Thomsen
766c4e392c
NIFI-9166 Refactored nifi-standard-services to use JUnit 5
This closes #5378

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-02-12 23:17:56 -06:00
exceptionfactory
9e06d13ad9
NIFI-9673 Improved DBCP and HikariCP test reliability
- Configured tests to use random temporary directory
- Added Derby shutdown handling and directory removal
- Upgraded DBCPServiceTest to JUnit 5
- Added pool name to HikariCP

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5759
2022-02-11 10:43:15 -05:00
Matthew Burgess
b8af44d81b
NIFI-6871: Added HikariCPConnectionPool controller service
This closes #3890

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-02-10 13:02:31 -06:00
Robert Kalmar
357a8667f9
NIFI-9663 Setting the "csv-escape" property has no effect in SelectHive3QL
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5756.
2022-02-10 10:32:27 +01:00
exceptionfactory
a2f6420f43
NIFI-9638 Refactored Google Guava usage in extensions
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5742
2022-02-09 16:39:53 -05:00
exceptionfactory
e584d3cf04
NIFI-9660 Upgraded Apache Tika to 2.3.0
- Upgraded tika-core from 1.27 to 2.3.0
- Upgraded tika-parsers to tika-parsers-standard-package in nifi-media-processors
- Updated Tika metadata property references

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5754.
2022-02-09 11:11:46 +01:00
Noel Baek
bbc78f1547
NIFI-9621: Added Ignore Reserved Characters to FlattenJson
- Upgraded to json-flattener from 0.12.0 to 0.13.0

This closes #5704

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-02-08 21:02:55 -06:00
Lehel
72e6accc12 NIFI-9327: Added timewindow query to QueryNiFiReportingTask and MetricsEventReportingTask 2022-02-08 17:41:34 +01:00
Paul Grey
ad5b816626
NIFI-9644 Improved TestExecuteStateless increasing TestRunner.run() wait allowed
This closes #5741

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-02-04 21:37:59 -06:00
Mark Payne
94668dee6e
NIFI-9639: Determine how long it takes to find cluster coordinator and perform DNS lookup when sending heartbeats and include in the logs
This closes #5748

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-02-04 20:19:07 -06:00
exceptionfactory
ce66cf41e2
NIFI-6699 Corrected SFTP symbolic link handling (#5744)
- Added SFTP stat() request on symbolic links to check directory status
- Refactored and renamed TestServerSFTPTransfer class
2022-02-04 18:43:42 -05:00
exceptionfactory
bab4309905
NIFI-9655 Add Queue Logging to ListenUDP (#5747)
* NIFI-9655 Added Queue Logging to ListenUDP

- Added TrackingLinkedBlockingQueue to track the largest queue size
- Updated AbstractListenEventProcessor to write debug log at most once per minute
- AbstractListenEventProcessor updates support both ListenUDP and ListenUDPRecord
2022-02-04 18:41:38 -05:00
markap14
230ed9c98d
NIFI-9628: Added a uiOnly flag when requesting Controller Service det… (#5712)
* NIFI-9628: Added a uiOnly flag when requesting Controller Service details and the list of Controller Services. This allows us to return much less data when retrieving these resources.

* NIFI-9628: Addressed review feedback; added uiOnly flag for controller service run-status and references also

* NIFI-9628: Fixed checkstyle issues by removing unused imports

This closes #5712
2022-02-03 15:53:23 -05:00
greyp9
9dbd066238
NIFI-9233 - Improve reliability of system integration tests (#5733) 2022-02-03 10:20:55 -05:00
Matthew Burgess
9d8b39529e
NIFI-9596: Fix newline bug in JythonScriptRunner
NIFI-9596: Added comment to indicate why the Apache header is missing

This closes #5682

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-02-02 08:46:29 -05:00
exceptionfactory
ba0bf2ede0
NIFI-9650 Upgraded OkHttp from 4.9.2 to 4.9.3
- Added okhttp-bom to root Maven configuration
- Removed repeated okhttp.version property from specific dependencies

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5737.
2022-02-02 11:45:18 +01:00
exceptionfactory
43748a5523 NIFI-9638 Refactored Google Guava references
- Refactored nifi-framework and nifi-standard modules
- Replaced Google Cache with Caffeine Cache
- Replaced Google collections classes with standard Java collections

This closes #5730.

Signed-off-by: Kevin Doran <kdoran@apache.org>
2022-02-01 15:17:50 -05:00
exceptionfactory
dc7d9510cd NIFI-9610 Refactored nifi-processor-utils to separate modules
- Added nifi-bin-manager
- Added nifi-event-listen
- Added nifi-event-put
- Added nifi-listed-entity
- Added nifi-put-pattern
- Removed nifi-processor-utils

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5729.
2022-02-01 12:59:16 -05:00
greyp9
27ceda38bf
NIFI-9233 - Improve reliability of system integration tests (#5720) 2022-01-31 09:20:54 -05:00
Mark Payne
038811d0dd
NIFI-9629: Ensure that when we are setting default values on Avro GenericRecord objects that we convert from the schema's default value to the proper type
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5716.
2022-01-29 13:56:53 +01:00
exceptionfactory
43a24743cc
NIFI-9341 Corrected annotation syntax problem
Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-01-28 13:24:41 -06:00
Bence Simon
49978cdd91
NIFI-9341 Added CEF RecordReader
This closes #5530

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-01-28 12:42:51 -06:00
exceptionfactory
6fca4eb3ce
NIFI-9624 Removed JCenter Repository
- Removed moquette-broker and refactored tests
- Upgraded docker-compose-rule-junit4 to 1.7.0 for Maven Central retrieval
- Changed JSTL dependency groupId for Maven Central retrieval

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5719.
2022-01-28 10:11:34 -05:00
exceptionfactory
95abc35463 NIFI-6266 Corrected proxy FTP connect handling
- Added ProxyFTPClient with connect method supporting unresolved hosts
- Added FTPClientProvider with implementation handling configuration and connection
- Added support for SOCKS with authentication in FTP components
- Refactored FTPTransfer using FTPClientProvider

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5655.
2022-01-27 17:36:27 -05:00
Nathan Gough
8f2a9f94fe
NIFI-9632 - Removed nifi-lumberjack-bundle
- Removed several unused socket classes from nifi-processor-utils

This closes #5722

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-01-27 14:43:19 -06:00
Timea Barna
3ccc9d29b6
NIFI-9455 Added aggregated predictions to Prometheus Flow Metrics
This closes #5582

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-01-27 10:07:01 -06:00
Mark Payne
3ea9faccc6
NIFI-8040: When changing version of a flow, stop processors that have a state of Starting in addition to those with a state of Running
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5718.
2022-01-27 15:06:40 +01:00
Paul Grey
0eff249870
NIFI-9611 Removed duplicate nifi-utils dependency
This closes #5721

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-01-26 19:07:29 -06:00