Commit Graph

140 Commits

Author SHA1 Message Date
Mark Payne 607480c1a0
NIFI-9239: Updated Consume/Publish Kafka processors to support Exactly Once Semantics. Added ExecuteStateless processor/bundle
- A few minor updates to Stateless in order to surface some concepts from the stateless engine up to the caller, such as bulletins, counters, etc.

Signed-off-by: Joe Gresock <jgresock@gmail.com>

This closes #5412.
2021-10-04 15:32:21 -04:00
Joe Gresock 3df2fff6e0
NIFI-8732: Adding a HashiCorp Vault K/V Stateless Param Provider
This closes #5289

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-09-17 12:26:07 -05:00
Bryan Bende 21c2fb95d3
NIFI-8973 Implement KerberosUserService API and keytab, password, and ticket cache implementations
NIFI-8974 Integrate KerberosUserService with HDFS processors

NIFI-8980 Integrate KerberosUserService with Kafka 2.6 processors
- Introduced SelfContainerKerberosUserService to restrict which impls can be used with Kafka
- Add variations of KerberosUser doAs that allow setting the context ClassLoader
- Add additional unit tests for configurations

This closes #5277

Signed-off-by: David Handermann <exceptionfactory@apache.org>
2021-08-18 19:54:56 -05:00
Joe Witt 97feacc181
NIFI-8767-RC2 prepare for next development iteration 2021-07-10 12:17:09 -07:00
Joe Witt fcbf1d5f97
NIFI-8767-RC2 prepare release nifi-1.14.0-RC2 2021-07-10 12:17:05 -07:00
exceptionfactory 92bdc23adb NIFI-8251 Added EncryptContentPGP and DecryptContentPGP Processors
- Added PGPPrivateKeyService and PGPPublicKeyService interfaces with standard implementations
- NIFI-7396 EncryptContentPGP writes encryption metadata attributes
- NIFI-6708 Controller Services support ElGamal Public and Private Keys
- NIFI-5346 Controller Services support Keyring Files and ASCII Key properties
- NIFI-5335 Controller Services support multiple public or private keys from keyrings
- NIFI-2983 DecryptContentPGP finds and decrypts Encrypted Data Packets regardless of signing
- NIFI-1694 Controller Services support individual key files or keyrings

NIFI-8251 Refactored Public Key ID Property to Public Key Search

NIFI-8251 Corrected handling of multiple Encrypted Data packets in DecryptContentPGP

- Added unit tests for encryption and decryption with both password-based and public key
- Added PGP NAR dependencies to nifi-assembly

Signed-off-by: Nathan Gough <thenatog@gmail.com>

This closes #4842.
2021-04-22 15:48:49 -04:00
exceptionfactory 052c60dc53
NIFI-8363 Added Single User Login Identity Provider and Authorizer (#4968)
* NIFI-8363 Added Single User Login Identity Provider and Authorizer

- Reads and writes username and hashed password in login-identity-providers.xml
- Generates random username using java.util.UUID.randomUUID()
- Generates random password using java.security.SecureRandom with Base64 encoding
- Writes generated password hash using bcrypt

* NIFI-8363 Updated SingleUserAuthorizer to require SingleUserLoginIdentityProvider

* NIFI-8363 Added handling of null login identity provider property
2021-04-12 08:48:20 -04:00
Koji Kawamura 68d38dd0a6
NIFI-6752 Add ASN.1 RecordReader
NIFI-6752 Refactored type and value conversion logic. Added support for more types. Added more tests.
Removed 'parent' from 'Recursive'. (Caused issues. The recursive nature is still there as it has a child with the same type).
Updated jasn1 1.11.2 to asn1bean 1.12.0. If an asn field name is a Java reserved keyword, the field gets a trailing "_" but the getter remains normal. In JASN1Utils adjusted logic when looking for the getter.
Added support for inherited types. OctetStrings are converted to Strings instead of byte arrays.
Service takes care of the compilation of the ASN files. Test sources are generated and removed from source control.

NIFI-6752 Removed obsolete TODOs.

NIFI-6752 Updated nifi-asn1-nar version to 1.13.0-SNAPSHOT. Fixed checkstyle violations (unused imports).

NIFI-6752 ASN.1 reader - ASN.1 bundle requires 'include-asn1' profile to be active to be part of assembly.

NIFI-6752 ASN.1 reader - Updated ASN1.xml template.

NIFI-6752 ASN.1 reader - Updated versions.

NIFI-6752 ASN.1 reader - Update example generator. Updated ASN1.xml template. Updated (fixed) nifi-asn1-nar version in pom.xml.

NIFI-6752 ASN.1 reader - Added missing license for ASN1.xml.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #4577
2021-02-25 12:58:05 -05:00
Joe Witt 88fab00e29
NIFI-7873 merging release branch to latest and updating to 1.14.0-SNAPSHOT 2021-02-15 12:09:32 -07:00
Joe Witt 4afb2ba743
NIFI-7873-RC4 prepare for next development iteration 2021-02-15 12:09:31 -07:00
Joe Witt 487280bee9
NIFI-7873-RC4 prepare release nifi-1.13.0-RC4 2021-02-15 12:09:30 -07:00
Bence Simon b980a8ea8c NIFI-7549 Adding Hazelcast based DistributedMapCacheClient support
NIFI-7549 Refining documentation; Changing explicit HA mode; Smaller review comments
NIFI-7549 Code review responses about license, documentation and dependencies
NIFI-7549 Fixing issue when explicit HA; Some further review based adjustments
NIFI-7549 Response to code review comments
NIFI-7549 Adding extra serialization test
NIFI-7549 Minor changes based on review comments
NIFI-7549 Adding hook point to the shutdown

This closes #4510.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>
2020-10-22 19:22:49 +02:00
Matthew Burgess 325a49556c
NIFI-7592: Allow NiFi to be started without a GUI/REST interface (#4509)
* NIFI-7592: Allow NiFi to be started without a GUI/REST interface

* NIFI-7592: Enable all controller services when starting headless

* NIFI-7592: Marked duplicate dependencies as provided

* NIFI-7592: Incorporated additional review comments
2020-10-07 09:12:16 -04:00
Andy LoPresto 9370571131
NIFI-7804 Split nifi-security-utils into sub-module for nifi-security… (#4533)
* NIFI-7804 Split nifi-security-utils into sub-module for nifi-security-utils-api (no external dependencies).
Separated interface and implementation of TlsConfiguration.
Reabsorbed nifi-security-xml-config into nifi-security-utils.

* NIFI-7804 Resolved failing unit test on Java 8.
Removed accidental module dependency.

* NIFI-7804 Resolved failing unit test.

* NIFI-7804 Removed legacy dependency.

* NIFI-7804 Marked nifi-security-utils-api as provided and overrode with compile scope in specific modules which are not children of nifi-standard-services-api-nar.
2020-09-17 12:52:22 -04:00
Joe Witt 8baa5c9940
NIFI-7692 updating for next dev release 1.13.0 2020-08-18 14:48:02 -07:00
Joe Witt fb57bcbc11
NIFI-7692-RC1 prepare for next development iteration 2020-08-13 09:20:39 -07:00
Joe Witt 303d6c59ba
NIFI-7692-RC1 prepare release nifi-1.12.0-RC1 2020-08-13 09:20:36 -07:00
si 71d155e874
NIFI-7338 fixed provenance, top level dir,
chatty smbj log, pw validation
update accessmask to be more restrictive

Author:    si <si.sun@roche.com>

Signed-off-by: Joe Witt <joewitt@apache.org>
2020-08-10 09:43:43 -07:00
Matthew Burgess 55adb11f06 Revert "NIFI-7592: Allow NiFi to be started without a GUI/REST interface"
This reverts commit 1b91a6caf3.
2020-08-07 12:50:48 -04:00
Matthew Burgess 1b91a6caf3 NIFI-7592: Allow NiFi to be started without a GUI/REST interface
- Rename MiNiFi references to Headless NiFi, added profile to build headless assembly

Signed-off-by: Mark Payne <markap14@hotmail.com>
2020-07-31 14:09:12 -04:00
Joe Witt 3de77ebacc
NIFI-7021-RC3 prepare for next development iteration 2020-01-19 14:14:40 -05:00
Joe Witt 633408bce7
NIFI-7021-RC3 prepare release nifi-1.11.0-RC3 2020-01-19 14:14:38 -05:00
Marc Parisi de2a286a7a
NIFI-818: This closes #3926. Initial implementation of NiFi-Accumulo ( https://github.com/phrocker/nifi-accumulo ) with connectors to Apache Accumulo 2.x
Signed-off-by: Joe Witt <joewitt@apache.org>
2020-01-19 11:43:19 -05:00
Joe Witt f8c3d877cf
NIFI-6733 updating to next release version for master branch 2019-11-04 13:31:39 -05:00
Joe Witt 4119279c03
Merge branch 'NIFI-6733-RC3' 2019-11-04 13:24:29 -05:00
Joe Witt 363b751310
Revert "NIFI-6733 bumping master to 1.11.0-SNAPSHOT"
This reverts commit a7c56cfd9a.
2019-11-04 13:24:13 -05:00
Joe Witt a7c56cfd9a
NIFI-6733 bumping master to 1.11.0-SNAPSHOT 2019-11-04 07:54:17 -05:00
Yolanda M. Davis 5b28f6dad9
NIFI-6803 - Initial commit for ActionHandler Controller Services
NIFI-6803 - updated to description

NIFI-6803 - add Spring reference to Notice

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3856
2019-11-01 10:41:25 -04:00
Joe Witt 418179f5b2
NIFI-6733-RC3 prepare for next development iteration 2019-10-28 15:13:13 -07:00
Joe Witt b217ae20ad
NIFI-6733-RC3 prepare release nifi-1.10.0-RC3 2019-10-28 15:12:57 -07:00
Matthew Burgess ace23c35b5 NIFI-6780: Add QueryNiFiReportingTask, RecordSinkService, S2S and DB impls (#3826)
* NIFI-6780: Introduce RecordSinkService to separate format and destination, refactor common S2S utils

Added QueryNiFiReportingTask to SQL query NiFi status and metrics

Add PROCESSOR_STATUS and PROCESS_GROUP_STATUS

Add CONNECTION_STATUS_PREDICTIONS

check for null predictions

Add ConnectionStatusRecursiveIterator

Fix issue w/ duplicate iterator outputs

Refactored query interfaces, fixed assembly POM

Rebased v master, fixed isBackPressureEnabled and Checkstyle/RAT errors

Add DatabaseRecordSink service (#13)

* Add DatabaseRecordSink service

* Incorporated review comments

* NIFI-6780: Add/fix docs, cleanup warnings, fixed some table definitions

* Added bundle profile, remove predictions table if not enabled

* Added doc for which tables are available when

This closes #3826.
2019-10-22 08:56:42 -04:00
Yolanda M. Davis 7c9d34f820
NIFI-6778 - Added rules engine service api and Easy Rules Implementation
NIFI-6778 - added comments and component description annotations

NIFI-6778 - added license and notice details

NIFI-6778 - added additional details section and updated description of service

NIFI-6778 - fixed checkstyle error

NIFI-6778 - addressed review comments for documentation and description corrections

NIFI-6778 - added include-rules profile for assembly build

NIFI-6778 - add MVEL and ASM to license and notice

NIFI-6778 - switch to use Jackson for NiFi Rules Json deserialization

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3824
2019-10-21 12:16:11 -04:00
Mike Thomsen 82f177c632
NIFI-5537 Refactored to be named Cypher Bundle instead.
NIFI-5537 Added skeletons of services, apis, etc.
NIFI-5537 Renamed classes to reflect that they are for cypher and not neo4j specific
NIFI-5537 Added start of client API for doing streaming execution.
NIFI-5537 Completed refactor of unit and integration tests.
NIFI-5537 Added OpenCypherClientService.
NIFI-5537 Updated to gremlin-core 3.3.5 to get better SSL configuration options.
NIFI-5537 Added SSL support to OpenCypher client service.
NIFI-5537 Updated L&N.

NIFI-5537 Updated to 1.10.0-SNAPSHOT.

NIFI-5537 Removed Mac-specific documentation in int test.

NIFI-5537 Updated a few properties to add EL.

NIFI-5537 Added GremlinClientService to support gremlin as well.

NIFI-5537 refactored everything to be a graph bundle.

NIFI-5537 Updated documentation.

NIFI-5537 Fixed assembly.

NIFI-5537 Updated Jackson.

NIFI-5537 Moved MIT section per code review guidance.

NIFI-5537 Changed provenance event and made it more explicit.

NIFI-5537 Updated processor config file.

NIFI-5537 Made changes requested in a code review.

NIFI-5537 Added documentation about driver configuration.

NIFI-5537 Remove output flowfile if an exception is thrown.

NIFI-5537 Updated configuration properties.

NIFI-5537 Made port and path configurable in the tinkerpop family of components.
NIFI-5537 Upgraded to OpenCypher cypher-gremlin-neo4j-driver 0.9.13 because it's the most recent version that JanusGraph supports.

NIFI-5537 Added warning about Janus 0.3.X to int test.

NIFI-5537 Added TinkerPopClientService interface.

NIFI-5537 Changed to mockito-core from mockito-all.

NIFI-5537 Added documentation about Gremlin for first time users w/ NiFi.
2019-06-20 13:39:20 -04:00
mans2singh 1b28c7b45c
NIFI-5537 Create Neo4J cypher execution processor
NIFI-5537 Renamed bundle based on review comments

NIFI-5537 Updated neo4j setup and version documentation and default values based on review comments

NIFI-5537 - Changed exception, logging, and json parser based on review comments

NIFI-5537 - Check debug enabled before logging at debug level

NIFI-5537 - Updated property displayname/description and corrected test file names

NIFI-5537 - Added note about dummy password required in case authentication is disabled on server

NIFI-5537 - Use SSLContext
2019-06-20 13:38:59 -04:00
Satwik 8560eb995a NIFI-5435 Added Prometheus reporting task.
NIFI-5435 Fixed Rebase Issue
NIFI-5435 changes in capability description of reporting task
NIFI-5435 Changes in capability description of reporting task
NIFI-5435 Changes to build Property Descriptors in static constructor
NIFI-5435 Indentation changes
NIFI-5435 Test code changes
NIFI-5435 Minor code changes
NIFI-5435 Changes related to client authentication
NIFI-5435 Prometheus metrics changes
NIFI-5435 Added process group name and id to metrics

This closes #3257

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2019-05-30 08:08:07 -04:00
Andy LoPresto e6c843f465
NIFI-6323 Changed URLs for repositories, project description, and mailing lists to use HTTPS.
NIFI-6323 Changed URLs for splunk.artifactoryonline.com to use HTTPS (certificate validity warning in browsers, but command-line connection using openssl s_client is successful).
NIFI-6323 Changed URLs for XMLNS schema locations to use HTTPS (the XMLNS and schema identifier remain http:// because they are not designed to be resolvable).
NIFI-6323 Fixed Maven XML schema descriptor URLs.

This closes #3497
2019-05-29 14:36:40 -04:00
Bryan Bende 4b45e85327 NIFI-6055 Improving error handling during auto-loading of NARs and ensuring only one Jetty NAR is loaded
NIFI-6056 Fixing cassandra bundle so nifi-mock is a test scoped and removing incorrect usage of ProviderCreation exception from nifi-framework-api

This closes #3322.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2019-02-22 12:19:12 -05:00
joewitt 0e204f3576
NIFI-6029-RC2 prepare for next development iteration 2019-02-16 21:50:35 -05:00
joewitt 45bb53d2aa
NIFI-6029-RC2 prepare release nifi-1.9.0-RC2 2019-02-16 21:50:15 -05:00
zenfenan 9a1ab4c504 NIFI-5621: Added Cassandra connection provider service
NIFI-5621: Improvements on LICENSE & NOTICE files

This closes #3105

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-12-12 09:59:32 -05:00
Jeff Storck c0182294ed NIFI-5720-RC3 prepare for next development iteration 2018-10-22 22:16:43 -04:00
Jeff Storck 98aabf2c50 NIFI-5720-RC3 prepare release nifi-1.8.0-RC3 2018-10-22 22:16:23 -04:00
“PrashanthVenkatesan” 9161b1787a NIFI-5327 Adding Netflowv5 protocol parser
NIFI-5327 Formated Code to adhere NiFi checkstyle
NIFI-5327 Incorporated code review comments
NIFI-5327 Update additionalDetails docs
NIFI-5327 Format IPV4 fields & rename nar
NIFI-5327 Update dependency jar version
NIFI-5327 Updated Code review comments
NIFI-5327 Updated Code review comments
NIFI-5327 Patch for code checkstyle violations
NIFI-5327 commit for retriggering build

This closes #2820

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-09-19 13:15:41 -04:00
Matthew Burgess 53969adcc8 NIFI-5353: Add JoltTransformRecord processor 2018-08-28 14:32:27 -04:00
Andy LoPresto d42a1e8bf4
NIFI-5323-RC1 prepare for next development iteration 2018-06-19 20:02:21 -07:00
Andy LoPresto 99bcd1f88d
NIFI-5323-RC1 prepare release nifi-1.7.0-RC1 2018-06-19 20:02:01 -07:00
Koji Kawamura 2834fa4ce4 NIFI-4199: Added ProxyConfigurationService
- Added ProxyConfigurationService to manage centralized proxy
configurations
- Adopt ProxyConfigurationService at FTP and HTTP processors

NIFI-4175 - Add HTTP proxy support to *SFTP processors

This closes #2018.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>

NIFI-4199: Add ProxyConfigurationService to SFTP processors

- Fixed check style issue
- Use the same proxy related PropertyDescriptors from FTPTransfer and
SFTPTransfer
- Dropped FlowFile EL evaluation support to make it align with other
processors spec, Now it supports VARIABLE_REGISTRY
- Added ProxyConfigurationService to SFTP processors
- Added SOCKS proxy support to SFTP processors

NIFI-4199: Added ProxyConfigurationService to ElasticsearchHttp processors

- ElasticsearchHttp processors now support SOCKS proxy, too
- Added proxy support to PutElasticsearchHttpRecord
- Moved more common property descriptors to
AbstractElasticsearchHttpProcessor and just return static unmodifiable
property descriptor list at each implementation processors

NIFI-4196 - Expose AWS proxy authentication settings

NIFI-4196 - Fix jUnit errors

This closes #2016.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>

NIFI-4199: Add ProxyConfigService to AWS processors

- Applied ProxyConfigService to S3 processors
- Added proxy support to following processors:
  - PutKinesisFirehose, PutKinesisStream
  - PutDynamoDB, DeleteDynamoDB, GetDynamoDB
  - PutKinesisStream
- All AWS processors support HTTP proxy now

NIFI-4199: Add ProxyConfigService to Azure processors

NIFI-4199: More explicit validation and docs for Proxy spec

- Each processor has different supporting Proxy specs
- Show supported spec to ProxyConfigurationService property doc
- Validate not only Proxy type, but also with Authentication

NIFI-4199: Incorporated review comments

- Fixed TestListS3 property descriptor check
- Separate name and displayName

This closes #2016
This closes #2018
This closes #2704

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-05-20 17:57:47 -04:00
joewitt d511fe3e4b NIFI-4995-RC3 prepare for next development iteration 2018-04-03 08:28:34 -07:00
joewitt f8466cb16d NIFI-4995-RC3 prepare release nifi-1.6.0-RC3 2018-04-03 08:28:15 -07:00
Mark Payne 0b0aebe148
NIFI-4917: Externalize Keytab and Principal configuration from Processors to a Controller Service. This gives us the ability to allow users to interact with those Keytabs/Principals to which they've been given access without allowing them access to all Keytabs and Principals
- Addressed review feedback; rebased against master

This closes #2552.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-03-21 14:23:41 -04:00