253 Commits

Author SHA1 Message Date
Joe Witt
6a0fb3e09e
NIFI-7873 Removed older/less commonly used nars from convenience assembly build to get back under max 1.5GB size
Signed-off-by: Joe Witt <joewitt@apache.org>
2021-01-26 23:12:10 -07:00
Mark Payne
3c9d8a7007
NIFI-7897: Refactoring to create a new nifi-framework-components module.
- Refactored nifi-stateless to make use of nifi-framework-components
- Removed requirement for nifi-framework-nar to be provided.
- Refactored stateless nifi into api, engine, nar, and bootstrap modules, with a parent 'bundle' module
- Creation of nifi-stateless-system-tests
- Added unit tests and logging
- Changed flow configuration to use properties file instead of json
- Allow for -p parameter to specify parameters on command line
- Moved implementations of Authorizer, NiFiUser, and UserGroupProviders to new module named nifi-framework-authorization-providers so that those that depend on nifi-framework-authorization don't have to bring in the providers. This way, we can have stateless not bring in those providers, as we otherwise get warnings on startup about the provider already being registered. Additionally, it avoids needing dependencies on spring-security-core
- Updated bin/nifi.sh script to run new stateless bootstrap
- Added Reporting Tasks to stateless.
- Download bundles as necessary on stateless nifi startup

NIFI-7897: Addressing review feedback

NIFI-7897: Fixed typos in README and also addressed issue that caused parameters with spaces not to be parsed properly

This closes #4669.

Signed-off-by: Bryan Bende <bbende@apache.org>
2020-11-24 11:37:45 -05:00
Peter Turcsanyi
3dd024fb66
NIFI-7825: Support native library loading via absolute path
Use an AspectJ aspect and agent to intercept the load native library calls
at runtime, copy the native library file to temp folder and proceed with
the newly created file in order to provide classloader isolation.

Remove AspectJ jars from lib directory, move the necessary jar to lib/aspectj subdirectory.

This closes #4540.

Signed-off-by: Bryan Bende <bbende@apache.org>
2020-10-26 16:21:16 -04:00
Bence Simon
b980a8ea8c NIFI-7549 Adding Hazelcast based DistributedMapCacheClient support
NIFI-7549 Refining documentation; Changing explicit HA mode; Smaller review comments
NIFI-7549 Code review responses about license, documentation and dependencies
NIFI-7549 Fixing issue when explicit HA; Some further review based adjustments
NIFI-7549 Response to code review comments
NIFI-7549 Adding extra serialization test
NIFI-7549 Minor changes based on review comments
NIFI-7549 Adding hook point to the shutdown

This closes #4510.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-10-22 19:22:49 +02:00
Matthew Burgess
325a49556c
NIFI-7592: Allow NiFi to be started without a GUI/REST interface (#4509)
* NIFI-7592: Allow NiFi to be started without a GUI/REST interface

* NIFI-7592: Enable all controller services when starting headless

* NIFI-7592: Marked duplicate dependencies as provided

* NIFI-7592: Incorporated additional review comments
2020-10-07 09:12:16 -04:00
Bryan Bende
f6d6e4adc9
NIFI-7798 Upgrade to apache ranger 2.1.0 client
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4518.
2020-09-11 16:51:43 +02:00
Joe Witt
8baa5c9940
NIFI-7692 updating for next dev release 1.13.0 2020-08-18 14:48:02 -07:00
Joe Witt
fb57bcbc11
NIFI-7692-RC1 prepare for next development iteration 2020-08-13 09:20:39 -07:00
Joe Witt
303d6c59ba
NIFI-7692-RC1 prepare release nifi-1.12.0-RC1 2020-08-13 09:20:36 -07:00
si
71d155e874
NIFI-7338 fixed provenance, top level dir,
chatty smbj log, pw validation
update accessmask to be more restrictive

Author:    si <si.sun@roche.com>

Signed-off-by: Joe Witt <joewitt@apache.org>
2020-08-10 09:43:43 -07:00
Joe Witt
58e324e9f1
NIFI-7522 bumping from Apache Kafka 2.5 to Apache Kafka 2.6
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4462.
2020-08-10 09:55:53 +02:00
Matthew Burgess
55adb11f06 Revert "NIFI-7592: Allow NiFi to be started without a GUI/REST interface"
This reverts commit 1b91a6caf32f60e08c2b4f0a7b2d95d9fb8ed22d.
2020-08-07 12:50:48 -04:00
Matthew Burgess
b8a1a4c578 Revert "NIFI-7592: Fixed typo in comment in pom.xml and changed headless profile to not active by default."
This reverts commit 2de4962ff1a46a0f1b9c9029ccce5c8206a04eb5.
2020-08-07 12:50:48 -04:00
Mathieu
1df7025ae8
NIFI-7522 : Add Kafka 2.5 Processors set 2020-08-04 16:59:39 -04:00
Mike Thomsen
255d0c0f19
NIFI-7604 Moved nifi-elasticsearch-5 NAR to a profile to save space.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4389
2020-07-31 16:12:20 -04:00
Mark Payne
2de4962ff1 NIFI-7592: Fixed typo in comment in pom.xml and changed headless profile to not active by default.
This closes #4376.
2020-07-31 14:11:48 -04:00
Matthew Burgess
1b91a6caf3 NIFI-7592: Allow NiFi to be started without a GUI/REST interface
- Rename MiNiFi references to Headless NiFi, added profile to build headless assembly

Signed-off-by: Mark Payne <markap14@hotmail.com>
2020-07-31 14:09:12 -04:00
Mike Thomsen
b8d51333d5
NIFI-7582 Added profile builds for kafka 0.9 and 0.10 so they can be removed from the convenience binary.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4364.
2020-07-17 11:38:42 +02:00
Mike Thomsen
915617dbe7 NIFI-7299 Add basic OAuth2 token provider service that can fetch access tokens when supplied with appropriate credentials.
Added skeleton of oauth2 provider.
Added copy of our code.
Refactored a few things.
Updated apis to better match flow descriptions.
Updated poms and other artifacts.
Updated copyright notice.
Updated LICENSE.

This closes #4173

Signed-off-by: Jeremy Dyer <jeremydyer@apache.org>
2020-06-04 08:43:27 -04:00
Bryan Bende
6412097eb2
NIFI-7257 Added HadoopDBCPConnectionPool
- Updated InstanceClassLoader to resolve files that are in the instance urls or additional urls
- Updated nifi-mock to support KerberosContext and removeProperty for ControllerServices
- Added unit test for HadoopDBCPConnectionPool
- Addressing review feedback

This closes #4149.
2020-03-24 11:05:18 -04:00
M Tien
f91d6c420d
NIFI-7268 Removed org.mindrot.jBcrypt library and replaced with at.fa… (#4151)
* NIFI-7268 Removed org.mindrot.jBcrypt library and replaced with at.favre.lib.bcrypt library.
Updated LICENSE and NOTICE files to reflect changes.
Updated unit tests.

Co-authored-by: Andy LoPresto <alopresto@apache.org>

* NIFI-7268 Fixed typo in Javadoc.

Co-authored-by: Andy LoPresto <alopresto@apache.org>
2020-03-17 19:49:15 -07:00
Matthew Burgess
dd6bcc7485
NIFI-7054: Add RecordSinkServiceLookup for dynamic sink routing (#4020)
* NIFI-7054: Add RecordSinkServiceLookup for dynamic sink routing

* NIFI-7054: Incorporated review comments
2020-02-03 09:45:08 -05:00
Joe Witt
3de77ebacc
NIFI-7021-RC3 prepare for next development iteration 2020-01-19 14:14:40 -05:00
Joe Witt
633408bce7
NIFI-7021-RC3 prepare release nifi-1.11.0-RC3 2020-01-19 14:14:38 -05:00
Kotaro Terada
b045c4ce0b
NIFI-6827 This closes #3862. Mismatch in "lib" directories of rpm and non-rpm build
Signed-off-by: Joe Witt <joewitt@apache.org>
2020-01-19 12:06:04 -05:00
Marc Parisi
de2a286a7a
NIFI-818: This closes #3926. Initial implementation of NiFi-Accumulo ( https://github.com/phrocker/nifi-accumulo ) with connectors to Apache Accumulo 2.x
Signed-off-by: Joe Witt <joewitt@apache.org>
2020-01-19 11:43:19 -05:00
Mike Thomsen
d87604a1cf Added nifi-other-graph-services-nar to the include-graph profile, since it was missing from that. 2019-11-11 18:32:10 -05:00
Yolanda M. Davis
c998a7259a
NIFI-6842 - Introduce MetricsEventReportingTask
NIFI-6842 - Added AlertHandler for bulletin reporting. Update ReportingTask meta data.

NIFI-6842 - corrected display names in action handlers, included metrics option for alert handlers, small refactor in reporting task

NIFI-6842 - updated docs and tags

NIFI-6842 - Added documentation for handlers.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3874
2019-11-08 10:25:54 -05:00
Joe Witt
f8c3d877cf
NIFI-6733 updating to next release version for master branch 2019-11-04 13:31:39 -05:00
Joe Witt
eb046c4fab
NIFI-6733 updating after merging release branch 2019-11-04 13:28:39 -05:00
Joe Witt
4119279c03
Merge branch 'NIFI-6733-RC3' 2019-11-04 13:24:29 -05:00
Joe Witt
363b751310
Revert "NIFI-6733 bumping master to 1.11.0-SNAPSHOT"
This reverts commit a7c56cfd9a27f2be8809c87ad427dc2ba892da1f.
2019-11-04 13:24:13 -05:00
Joe Witt
a7c56cfd9a
NIFI-6733 bumping master to 1.11.0-SNAPSHOT 2019-11-04 07:54:17 -05:00
Yolanda M. Davis
5b28f6dad9
NIFI-6803 - Initial commit for ActionHandler Controller Services
NIFI-6803 - updated to description

NIFI-6803 - add Spring reference to Notice

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3856
2019-11-01 10:41:25 -04:00
Joe Witt
418179f5b2
NIFI-6733-RC3 prepare for next development iteration 2019-10-28 15:13:13 -07:00
Joe Witt
b217ae20ad
NIFI-6733-RC3 prepare release nifi-1.10.0-RC3 2019-10-28 15:12:57 -07:00
Matthew Burgess
ace23c35b5 NIFI-6780: Add QueryNiFiReportingTask, RecordSinkService, S2S and DB impls (#3826)
* NIFI-6780: Introduce RecordSinkService to separate format and destination, refactor common S2S utils

Added QueryNiFiReportingTask to SQL query NiFi status and metrics

Add PROCESSOR_STATUS and PROCESS_GROUP_STATUS

Add CONNECTION_STATUS_PREDICTIONS

check for null predictions

Add ConnectionStatusRecursiveIterator

Fix issue w/ duplicate iterator outputs

Refactored query interfaces, fixed assembly POM

Rebased v master, fixed isBackPressureEnabled and Checkstyle/RAT errors

Add DatabaseRecordSink service (#13)

* Add DatabaseRecordSink service

* Incorporated review comments

* NIFI-6780: Add/fix docs, cleanup warnings, fixed some table definitions

* Added bundle profile, remove predictions table if not enabled

* Added doc for which tables are available when

This closes #3826.
2019-10-22 08:56:42 -04:00
Yolanda M. Davis
7c9d34f820
NIFI-6778 - Added rules engine service api and Easy Rules Implementation
NIFI-6778 - added comments and component description annotations

NIFI-6778 - added license and notice details

NIFI-6778 - added additional details section and updated description of service

NIFI-6778 - fixed checkstyle error

NIFI-6778 - addressed review comments for documentation and description corrections

NIFI-6778 - added include-rules profile for assembly build

NIFI-6778 - add MVEL and ASM to license and notice

NIFI-6778 - switch to use Jackson for NiFi Rules Json deserialization

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3824
2019-10-21 12:16:11 -04:00
Joe Witt
8abf330328
Revert NIFI-6703 as it creates startup issues or logs a confusing message to the user. Details on JIRA
This reverts commit 61baa41e7b5a86618e5b8773569c4805867f211b.
2019-10-01 15:33:07 -04:00
samhjelmfelt
61baa41e7b
NIFI-6703: This closes #3762. Add Stateless NiFi to CLI
NIFI-6703: Fixed extracted nar directory and marked api as experimental

Signed-off-by: Joe Witt <joewitt@apache.org>
2019-09-30 23:23:15 -04:00
Peter Turcsanyi
a4c3ca50dd
NIFI-6550: Create controller service for Azure Storage Credentials
Added AzureStorageCredentialsService controller service interface
with 2 implementations:
 - AzureStorageCredentialsControllerService: holds the credential properties
   (account name, account key, sas token)
 - AzureStorageCredentialsControllerServiceLookup: can be used to dynamically
   look up another AzureStorageCredentialsService (similar to
   DBCPConnectionPoolLookup)

The controller service can be used by the Azure Storage processors:
{List|Fetch|Put|Delete}AzureBlobStorage, {Get|Put}AzureQueueStorage

NIFI-6550: Review changes.

NIFI-6550: Review changes #2.

This closes #3742.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2019-09-27 01:36:28 +09:00
Aldrin Piri
be1acd07c5
NIFI-6604: This closes #3682. Removing large NARs from generated assemblies to reduce size.
Removed items consist of:
* nifi-kafka-0-8-nar
* nifi-flume-nar
* nifi-media-nar
* nifi-druid-nar
* nifi-druid-controller-service-api-nar
* nifi-other-graph-services-nar
* nifi-kite-nar
2019-09-19 23:39:03 -04:00
Mike Thomsen
2e554cbb67 NIFI-6588 Updated assembly pom file to move all graph components into a profile to save space in the convenience binary. 2019-08-29 10:58:25 -04:00
Mike Thomsen
82f177c632
NIFI-5537 Refactored to be named Cypher Bundle instead.
NIFI-5537 Added skeletons of services, apis, etc.
NIFI-5537 Renamed classes to reflect that they are for cypher and not neo4j specific
NIFI-5537 Added start of client API for doing streaming execution.
NIFI-5537 Completed refactor of unit and integration tests.
NIFI-5537 Added OpenCypherClientService.
NIFI-5537 Updated to gremlin-core 3.3.5 to get better SSL configuration options.
NIFI-5537 Added SSL support to OpenCypher client service.
NIFI-5537 Updated L&N.

NIFI-5537 Updated to 1.10.0-SNAPSHOT.

NIFI-5537 Removed Mac-specific documentation in int test.

NIFI-5537 Updated a few properties to add EL.

NIFI-5537 Added GremlinClientService to support gremlin as well.

NIFI-5537 refactored everything to be a graph bundle.

NIFI-5537 Updated documentation.

NIFI-5537 Fixed assembly.

NIFI-5537 Updated Jackson.

NIFI-5537 Moved MIT section per code review guidance.

NIFI-5537 Changed provenance event and made it more explicit.

NIFI-5537 Updated processor config file.

NIFI-5537 Made changes requested in a code review.

NIFI-5537 Added documentation about driver configuration.

NIFI-5537 Remove output flowfile if an exception is thrown.

NIFI-5537 Updated configuration properties.

NIFI-5537 Made port and path configurable in the tinkerpop family of components.
NIFI-5537 Upgraded to OpenCypher cypher-gremlin-neo4j-driver 0.9.13 because it's the most recent version that JanusGraph supports.

NIFI-5537 Added warning about Janus 0.3.X to int test.

NIFI-5537 Added TinkerPopClientService interface.

NIFI-5537 Changed to mockito-core from mockito-all.

NIFI-5537 Added documentation about Gremlin for first time users w/ NiFi.
2019-06-20 13:39:20 -04:00
mans2singh
1b28c7b45c
NIFI-5537 Create Neo4J cypher execution processor
NIFI-5537 Renamed bundle based on review comments

NIFI-5537 Updated neo4j setup and version documentation and default values based on review comments

NIFI-5537 - Changed exception, logging, and json parser based on review comments

NIFI-5537 - Check debug enabled before logging at debug level

NIFI-5537 - Updated property displayname/description and corrected test file names

NIFI-5537 - Added note about dummy password required in case authentication is disabled on server

NIFI-5537 - Use SSLContext
2019-06-20 13:38:59 -04:00
Satwik
8560eb995a NIFI-5435 Added Prometheus reporting task.
NIFI-5435 Fixed Rebase Issue
NIFI-5435 changes in capability description of reporting task
NIFI-5435 Changes in capability description of reporting task
NIFI-5435 Changes to build Property Descriptors in static constructor
NIFI-5435 Indentation changes
NIFI-5435 Test code changes
NIFI-5435 Minor code changes
NIFI-5435 Changes related to client authentication
NIFI-5435 Prometheus metrics changes
NIFI-5435 Added process group name and id to metrics

This closes #3257

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2019-05-30 08:08:07 -04:00
Andy LoPresto
e6c843f465
NIFI-6323 Changed URLs for repositories, project description, and mailing lists to use HTTPS.
NIFI-6323 Changed URLs for splunk.artifactoryonline.com to use HTTPS (certificate validity warning in browsers, but command-line connection using openssl s_client is successful).
NIFI-6323 Changed URLs for XMLNS schema locations to use HTTPS (the XMLNS and schema identifier remain http:// because they are not designed to be resolvable).
NIFI-6323 Fixed Maven XML schema descriptor URLs.

This closes #3497
2019-05-29 14:36:40 -04:00
Jeff Storck
2453c36710 NIFI-5820 NiFi built on Java 8 can run on Java 11
Updated RunNiFi.java to add libs needed to run on Java 11 when it is the detected runtime java version and grant access to the necessary module (java.xml.bind) when running on Java 9 or 10
Added dependencies/includes/excludes to nifi-assembly configurations for enabling NiFi to run on Java 11

This closes #3174.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2019-04-16 10:52:08 -04:00
joewitt
0e204f3576
NIFI-6029-RC2 prepare for next development iteration 2019-02-16 21:50:35 -05:00
joewitt
45bb53d2aa
NIFI-6029-RC2 prepare release nifi-1.9.0-RC2 2019-02-16 21:50:15 -05:00