7153 Commits

Author SHA1 Message Date
xinyuwu4
22b47ccc7b
NIFI-9672: fix flaky tests caused by the use of HashMap.
This closes #5758

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-02-22 08:06:52 -05:00
lawye1973
0ade040b76
NIFI-9696 : DeleteS3Object don't have provenance
Changed from AmazonS3 to AmazonS3client.
Add REMOTE_INVOCATION provenance event with URL and "Object deleted"

This closes #5778

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-02-22 08:01:59 -05:00
lawye1973
54b4209798
NIFI-9713 : TagS3Object do not have provenance data
Changed from AmazonS3 to AmazonS3client.
Add REMOTE_INVOCATION provenance event with URL and "Object tagged"

This closes #5785

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-02-22 07:58:45 -05:00
Chris Sampson
07131a66ea
NIFI-9715 add option to output empty FlowFile from Elasticsearch REST API Json Query processors when there are no hits from query
This closes #5786

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-02-22 06:32:56 -05:00
Mike Thomsen
4141ed29ec
NIFI-9647 Added ExtractDocumentText Processor
- Based on https://github.com/tspannhw/nifi-extracttext-processor

This closes #5732

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-02-21 20:01:25 -06:00
Tamas Palfy
635824904d
NIFI-9609 Added nifi-snowflake-bundle with SnowflakeComputingConnectionPool
- Updated DBCPConnectionPool for extension

This closes #5692

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-02-21 19:38:47 -06:00
lawye1973
8d80fb2930
NIFI-9694 : Correct FetchS3Object.java provenance data URL
Current provenance data URL is incorrect as it's hardcoded.
Follow PutS3Object.java method, using AmazonS3Client (instead of AmazonS3) to use getResourceUrl(bucket,key) to build the correct URL for provenance data

This closes #5779

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-02-19 18:46:20 -05:00
levilentz
dfbf2e3cea
NIFI-9286: JOLT Expression Language
Fixes NIFI-6213 and adds in functionality to use expression language in class and module specification
NIFI-9286: adding JOLT unit tests
NIFI-9286: addressing PR feedback
Fixes a problem with the scope of the EL for module directory
NIFI-9286: alignment of JOLT processors
NIFI-9286: fix checkstyle

This closes #5444

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-02-19 14:11:44 -05:00
Nathan Gough
7ef2fd2986
NIFI-7333 Added OIDC trust store strategy property
This closes #5753

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-02-18 15:19:22 -05:00
Roberto Marturano
a8fd5993eb NIFI-7192 Added systemd reload to nifi.sh install on systemd servers
This closes #4079

Signed-off-by: Joey Frazee <jfrazee@apache.org>
2022-02-18 10:40:52 -08:00
Matthew Burgess
2f3380390d NIFI-8492: Addressed issue in DatabaseReader class of IPLookupService that was attempting to set values on the JSON returned by MaxMind. Instead of modifying the object directly, we should use an Injectable in the Reader so that the value read will have the appropriate values but we don't need to modify those objects returned by MaxMind. Similar solution of NIFI-5814
Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5777.
2022-02-17 12:55:12 -05:00
Mike Thomsen
a837397d56
NIFI-8209 Added Neo4J 4.X and 3.X clients by splitting the current controller service along release lines. This was necessary because Neo4J broke compatibility in their client drivers for Java between 3.X and 4.X at the Java API level.
NIFI-8209 Updated module name.
NIFI-8209 Updated parent module.
NIFI-8209 Renamed a few misnamed modules.
NIFI-8209 Updated 1.15.0-SNAPSHOT references in cypher v3 package.
NIFI-8209 Updated neo4j 4.x driver.

NIFI-8209 Made changes requested in a code review.

NIFI-8209: Fixed NAR module name
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5301
2022-02-17 10:28:12 -05:00
Matthew Burgess
03165ad817 NIFI-9607: Honor Update Keys when Quoting Identifiers in PutDatabaseRecord
NIFI-9607: Fixed wrong column name in WHERE clause for generateUpdate

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5701.
2022-02-16 16:07:34 -05:00
Mark Payne
4d8c79d7f3 NIFI-9698: When creating an Avro schema, ensure that any default value is converted from what is returned by RecordField.getDefaultValue() to what Avro requires.
Signed-off-by: Chris Sampson <chriss@apache.org>

This closes #5776
2022-02-16 20:37:50 +00:00
Mark Payne
2aa6bd1e13
NIFI-9689: When all FlowFiles in a FlowFile Queue are penalized, do not schedule the destination to run. Also expose this fact via the ConnectionStatusSnapshotDTO, as this allows the front-end to render this information to the user in order to avoid confusion when it appears that the Processor has data but does nothing
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5771
2022-02-16 15:30:00 -05:00
Joe Gresock
4210e30047
NIFI-9620: Adding isStateful to StatelessDataflow (#5703)
* Adding isStateless method to reporting task and controller service
2022-02-16 13:30:40 -05:00
Andrew Lim
6ad28046c3
NIFI-9687 Add additional documentation for nifi.cluster.node.protocol.max.threads property to Admin Guide (#5770) 2022-02-16 13:18:36 -05:00
exceptionfactory
fe1139b8bb
NIFI-9692 Upgraded Apache Commons Lang3 to 3.12.0
- Moved commons-lang3 version management to root Maven configuration
- Refactored limited usage of StringUtils is nifi-reporting-utils to remove commons-lang3
- Refactored limited usage of StringUtils in nifi-websocket-processors to remove commons-lang3

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5773
2022-02-16 09:28:21 -05:00
Gabor Gyimesi
6788076536
NIFI-9058 Corrected AttributesToJSON Core Attributes filtering
- Avoiding filtering Core Attributes from configured Attributes List

This closes #5312

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-02-15 15:51:39 -06:00
exceptionfactory
7dfbf99d5f
NIFI-9527 Upgraded snappy-java to 1.1.8.4
- Set managed version in root Maven configuration

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5760
2022-02-15 16:12:36 -05:00
timeabarna
63a543cd67
NIFI-9475 Provide Framework-Level Retries for NiFi Relationships (#5593) 2022-02-15 15:40:41 -05:00
tpalfy
84b2484fd4
NIFI-9684 Fix: When starting/stopping selected process group, it sends the parent process group id to the REST interface that is responsible to enable/disable transmission for all remote process groups within a process group. Need to send the id of the select process group instead. (#5764) 2022-02-15 14:55:17 -05:00
greyp9
87cfd43f6f
NIFI-9233 - Improve reliability of system integration tests (#5749)
* NIFI-9233 - Improve reliability of system integration tests
2022-02-15 13:14:01 -05:00
Lehel
51202d7c2d
NIFI-9686: Renamed SNMP integration tests correctly
This closes #5769

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-02-15 11:07:34 -06:00
Steve Lawrence
4815df29db
NIFI-6740: Add configuration options to specify NiFi/Bootstrap communication ports
The NiFi and NiFi Bootstrap processes both bind to random ephemeral
ports to allow for inter-process communication (e.g. shutdown, port,
ping, etc.). However, the randomness of these ephemeral ports can pose
challenges for some security policies and firewall rules.

This adds two configuration options, nifi.bootstrap.listen.port and
nifi.listener.bootstrap.port, that allow an administrator to define
which ports the two processes should bind to for this communication,
making it easier to define security policies. The options default to
zero to maintain the current ephemeral port behavior.

NIFI-6740: Add configuration options to specify NiFi/Bootstrap communication ports

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5746
2022-02-15 11:46:21 -05:00
exceptionfactory
b09a778d90
NIFI-9685 Upgraded JNA to 5.10.0
- Upgraded nifi-bootstrap-utils JNA from 4.4.0 to 5.10.0
- Upgraded nifi-windows-event-log-processors JNA from 4.5.2 to 5.10.0
- Upgraded nifi-toolkit-cli JLine from 3.5.2 to 3.21.0

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5766.
2022-02-15 10:52:54 +01:00
Scott Aslan
b8ec6d38dc [NIFI-9554] upgrade node and npm
update lintin
update font awesome path
bump nifi-fds version
update nifi deps
update frontend maven plugin
change CI npm cache restore keys so that we will not use cached directories for older versions of node

This closes #5751.

Signed-off-by: Kevin Doran <kdoran@apache.org>
2022-02-14 14:24:02 -05:00
exceptionfactory
f6f50e6b3a
NIFI-9681 Upgraded Apache Commons DBCP to 2.9.0
- Replaced module references to version 2.7.0 with managed dependency in nifi-nar-bundles

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5763.
2022-02-14 18:30:36 +01:00
Mike Thomsen
766c4e392c
NIFI-9166 Refactored nifi-standard-services to use JUnit 5
This closes #5378

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-02-12 23:17:56 -06:00
exceptionfactory
9e06d13ad9
NIFI-9673 Improved DBCP and HikariCP test reliability
- Configured tests to use random temporary directory
- Added Derby shutdown handling and directory removal
- Upgraded DBCPServiceTest to JUnit 5
- Added pool name to HikariCP

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5759
2022-02-11 10:43:15 -05:00
Matthew Burgess
b8af44d81b
NIFI-6871: Added HikariCPConnectionPool controller service
This closes #3890

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-02-10 13:02:31 -06:00
Joe Gresock
a3fb61d9e6
NIFI-9668: Adding informative error when setting same value in 'set-param' CLI command
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5757.
2022-02-10 15:03:38 +01:00
Robert Kalmar
357a8667f9
NIFI-9663 Setting the "csv-escape" property has no effect in SelectHive3QL
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5756.
2022-02-10 10:32:27 +01:00
exceptionfactory
a2f6420f43
NIFI-9638 Refactored Google Guava usage in extensions
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #5742
2022-02-09 16:39:53 -05:00
exceptionfactory
e584d3cf04
NIFI-9660 Upgraded Apache Tika to 2.3.0
- Upgraded tika-core from 1.27 to 2.3.0
- Upgraded tika-parsers to tika-parsers-standard-package in nifi-media-processors
- Updated Tika metadata property references

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5754.
2022-02-09 11:11:46 +01:00
Noel Baek
bbc78f1547
NIFI-9621: Added Ignore Reserved Characters to FlattenJson
- Upgraded to json-flattener from 0.12.0 to 0.13.0

This closes #5704

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-02-08 21:02:55 -06:00
Lehel
72e6accc12 NIFI-9327: Added timewindow query to QueryNiFiReportingTask and MetricsEventReportingTask 2022-02-08 17:41:34 +01:00
exceptionfactory
61b867cda2
NIFI-9652 Upgraded jetty-schemas from 3.1 to 5.2 (#5750)
This closes #5750
2022-02-07 14:07:45 -05:00
Paul Grey
ad5b816626
NIFI-9644 Improved TestExecuteStateless increasing TestRunner.run() wait allowed
This closes #5741

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-02-04 21:37:59 -06:00
Mark Payne
94668dee6e
NIFI-9639: Determine how long it takes to find cluster coordinator and perform DNS lookup when sending heartbeats and include in the logs
This closes #5748

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-02-04 20:19:07 -06:00
Rob Fellows
8d143e8367 [NIFI-9623] - Attempt to look up messages locale file without country designation if it fails to find it with the country designation
Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5707.
2022-02-04 18:44:15 -05:00
exceptionfactory
ce66cf41e2
NIFI-6699 Corrected SFTP symbolic link handling (#5744)
- Added SFTP stat() request on symbolic links to check directory status
- Refactored and renamed TestServerSFTPTransfer class
2022-02-04 18:43:42 -05:00
exceptionfactory
bab4309905
NIFI-9655 Add Queue Logging to ListenUDP (#5747)
* NIFI-9655 Added Queue Logging to ListenUDP

- Added TrackingLinkedBlockingQueue to track the largest queue size
- Updated AbstractListenEventProcessor to write debug log at most once per minute
- AbstractListenEventProcessor updates support both ListenUDP and ListenUDPRecord
2022-02-04 18:41:38 -05:00
markap14
230ed9c98d
NIFI-9628: Added a uiOnly flag when requesting Controller Service det… (#5712)
* NIFI-9628: Added a uiOnly flag when requesting Controller Service details and the list of Controller Services. This allows us to return much less data when retrieving these resources.

* NIFI-9628: Addressed review feedback; added uiOnly flag for controller service run-status and references also

* NIFI-9628: Fixed checkstyle issues by removing unused imports

This closes #5712
2022-02-03 15:53:23 -05:00
greyp9
9dbd066238
NIFI-9233 - Improve reliability of system integration tests (#5733) 2022-02-03 10:20:55 -05:00
David Stewart
11bad7b4ed NIFI-9631 - Enable cli.sh to be used with a symbolic link
Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #5717.
2022-02-02 22:41:44 -05:00
Matthew Burgess
b669d08173
NIFI-9611: Restore commons-io to minifi-assembly
This closes #5739

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2022-02-02 20:30:11 -06:00
Matthew Burgess
9d8b39529e
NIFI-9596: Fix newline bug in JythonScriptRunner
NIFI-9596: Added comment to indicate why the Apache header is missing

This closes #5682

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2022-02-02 08:46:29 -05:00
Andrew Lim
0e53a1cb4e
NIFI-9642 Update Admin Guide and User Guide with correct nifi.properties default values
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5735.
2022-02-02 11:48:33 +01:00
exceptionfactory
ba0bf2ede0
NIFI-9650 Upgraded OkHttp from 4.9.2 to 4.9.3
- Added okhttp-bom to root Maven configuration
- Removed repeated okhttp.version property from specific dependencies

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #5737.
2022-02-02 11:45:18 +01:00