Commit Graph

5438 Commits

Author SHA1 Message Date
Joe Witt eb046c4fab
NIFI-6733 updating after merging release branch 2019-11-04 13:28:39 -05:00
Joe Witt 4119279c03
Merge branch 'NIFI-6733-RC3' 2019-11-04 13:24:29 -05:00
Joe Witt 363b751310
Revert "NIFI-6733 bumping master to 1.11.0-SNAPSHOT"
This reverts commit a7c56cfd9a.
2019-11-04 13:24:13 -05:00
Joe Witt a7c56cfd9a
NIFI-6733 bumping master to 1.11.0-SNAPSHOT 2019-11-04 07:54:17 -05:00
Joe Witt b1f97708aa
NIFI-6733 updating docker version for next version 2019-11-04 07:42:47 -05:00
Manoj Mamidyala 6864f3eb25
NIFI-6792
added a function in Expression Language to
Evaluate Variable value containing expression Language
NIFI-6792
Build related changes
added license to newly added class StringElEvaluator.java
restyled the classes having maven-checkstyle related issues
NIFI-6792
made changes suggested by reviewer MikeThomsen

Expression Language function name changed from stringEL to evaluateELString

This closes #3829

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2019-11-02 10:23:07 -04:00
Mark Payne 0b3a60ae3e NIFI-6832: When receiving a 0-byte FlowFile via site-to-site, avoid writing to the FlowFile at all 2019-11-01 16:22:17 -04:00
Yolanda M. Davis 5b28f6dad9
NIFI-6803 - Initial commit for ActionHandler Controller Services
NIFI-6803 - updated to description

NIFI-6803 - add Spring reference to Notice

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3856
2019-11-01 10:41:25 -04:00
Kotaro Terada 245e055d5d
NIFI-6828 There is an unintentional string in "conf/logback.xml"
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3863.
2019-10-31 11:20:21 +01:00
mtien 3d56e2f26d
NIFI-4348 handled NPE in isExpressionLanguagePresent.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3857.
2019-10-30 09:58:55 +01:00
Mark Payne ce5eae5b2c NIFI-6800: Added unit test to verify behavior of contribution 2019-10-29 10:17:27 -04:00
Ivan Ezequiel Rodriguez 67f1677212 NIFI-6800 - Fix hashmap counter overwritten in highThroughputSession
This closes #3837.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2019-10-29 10:16:49 -04:00
Joe Witt 418179f5b2
NIFI-6733-RC3 prepare for next development iteration 2019-10-28 15:13:13 -07:00
Joe Witt b217ae20ad
NIFI-6733-RC3 prepare release nifi-1.10.0-RC3 2019-10-28 15:12:57 -07:00
David Mollitor dccbde40f3
NIFI-5452: Allow ignore block locality in HDFS
Fixed formatting. Fallback to autoboxing

This closes #3652.
2019-10-28 14:23:44 -07:00
Peter Turcsanyi 5fd8df5780
NIFI-6818: This closes #3852. Set service api dependencies to provided in AWS and Azure bundles
The controller service api jars have their own bundles, they do not need to be
packaged into the processor nars.

Signed-off-by: Joe Witt <joewitt@apache.org>
2019-10-28 12:56:06 -07:00
Joe Witt c6e8bf204c
NIFI-6794 resolving pom issue creating resolving merge conflict 2019-10-28 12:55:26 -07:00
Mike Thomsen d5fbfd1a12
NIFI-6794 This closes #3828. Moved to the latest Elasticsearch 5.X client and upgraded Apache HttpClient to the latest 4.5.X release to try to get around a SSL bug reported on the mailing list. 2019-10-28 12:42:37 -07:00
Joe Witt dda29cd90b
NIFI-6733 updating zk to latest incremental 2019-10-28 12:40:03 -07:00
mysunnytime 5c69faf9bb NIFI-6597: Upgrade Azure Event Hub version and code.
- Lazy instantiation for PutAzureEventHub. Also add explaination for thread pool needed for EventHubClient.
2019-10-28 15:33:32 -04:00
Jan Hentschel ccf85777c4
NIFI-6816 Removed duplicated dependency declarations
This closes #3851

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2019-10-28 12:49:08 -04:00
Otto Fowler cfcf2e698d
NIFI-6682 SplitJson does not work if the input is null
This closes #3772

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2019-10-25 17:43:13 -04:00
mtien 4d25b6341c
NIFI-1474 Added method to check p boundary. Added a unit test.
NIFI-1474 Added unit test for constructor p boundaries. Added p boundary conditions.
NIFI-1474 Updated documentation regarding p boundary.
NIFI-1474 Added r validation and unit tests. Added Javadoc for new methods.

This closes #3844.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2019-10-25 14:28:46 -07:00
Matt Gilman 4f5718bf50
NIFI-6812: This closes #3847
- Establishing the L&N for the azure-services-api-nar.

Signed-off-by: Joe Witt <joewitt@apache.org>
2019-10-25 13:30:54 -04:00
Matt Gilman 9364f7d2e8
NIFI-6811:
- Allowing enterGroup to complete prior to loading the controller service table.

This closes #3846.
2019-10-25 12:07:03 -04:00
Mark Payne 3f270f184c
NIFI-6807: When a Controller Service's state is transitioned to ENABLING, complete the Future successfully, even if the Controller Service is not valid. The Controller Service will remain in the ENABLING state until it is made valid, at which point it will ENABLE (unless explicitly disabled first). This allows us to Enable a Controller Service and its referencing components, even if the referencing component is still invalid due to it not yet recognizing the the referenced service has been enabled.
This closes #3841
2019-10-25 11:38:34 -04:00
Mark Payne 81fc2768c9
NIFI-6798: When checking that referenced Controller Services are valid, use ValidationContext.getProperties() rather than ComponentNode.getSupportedPropertyDescriptors() because the latter does not include dynamic properties
This closes #3836.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-10-25 09:52:44 -04:00
Matt Gilman 2cef010b06
NIFI-6687:
- Using correct NiFi Ranger Service name in unit tests.

This closes #3842.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-10-25 09:31:34 -04:00
Aldrin Piri 20f70357a3
NIFI-6809 Make use of ASF shortened URL to point to community SLack invite link.
This closes #3845

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2019-10-25 09:09:28 -04:00
Koji Kawamura 87838e6c31
NIFI-6793 Fix NumberFormatException at CaptureChangeMySQL
This closes #3827

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2019-10-25 05:48:50 -04:00
Mark Payne 8079d2531e
NIFI-6805, NIFI-6806: Updated toString() of StandardControllerServiceNode so that it includes the component's type and UUID, updated the more generic Exception Mappers to include stack traces regardless of the log level that is enabled
This closes #3840

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2019-10-24 19:27:30 -04:00
Matthew Burgess 8771c35f4a NIFI-6799: Add PrometheusRecordSink and reset() method to RecordSinkService (#3839)
This closes #3839
2019-10-24 11:53:28 -04:00
Yolanda M. Davis 147365285c
NIFI-6801 - fix to ensure unique model instance is created for each connection. Also increased default query interval.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3838
2019-10-24 08:51:50 -04:00
Matt Gilman c2da6e7aa2
NIFI-6687:
- Updating exclusions and overrides accordingly for Ranger 2.0.

This closes #3835.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-10-23 15:02:05 -04:00
Mark Payne 3543b9c42c
NIFI-6797: Add support for specifying Partition via EL or RecordPath for PublishKafka(Record)_1_0 and PublishKafka(Record)_2_0
This closes #3834.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-10-23 14:59:33 -04:00
Matthew Burgess ace23c35b5 NIFI-6780: Add QueryNiFiReportingTask, RecordSinkService, S2S and DB impls (#3826)
* NIFI-6780: Introduce RecordSinkService to separate format and destination, refactor common S2S utils

Added QueryNiFiReportingTask to SQL query NiFi status and metrics

Add PROCESSOR_STATUS and PROCESS_GROUP_STATUS

Add CONNECTION_STATUS_PREDICTIONS

check for null predictions

Add ConnectionStatusRecursiveIterator

Fix issue w/ duplicate iterator outputs

Refactored query interfaces, fixed assembly POM

Rebased v master, fixed isBackPressureEnabled and Checkstyle/RAT errors

Add DatabaseRecordSink service (#13)

* Add DatabaseRecordSink service

* Incorporated review comments

* NIFI-6780: Add/fix docs, cleanup warnings, fixed some table definitions

* Added bundle profile, remove predictions table if not enabled

* Added doc for which tables are available when

This closes #3826.
2019-10-22 08:56:42 -04:00
Andy LoPresto d148fb1854
NIFI-3834 This closes #3821. Added encrypted content repository implementation.
Added skeleton implementation of EncryptedFileSystemRepository.
Added new impl to META-INF registry.
Added investigation comments to FileSystemRepository.
Implemented RepositoryObject block and stream encryptors.
Added passing unit test for encryption and decryption of multiple content writes (large buffered file) for AES-CTR encryptor.
Refactored shared logic from AES CTR and G/CM encryptors to abstract parent.
Added working unit test for writing/reading via encrypted file system repository.
Added stream wrappers.
Added encryptor.
Added working unit test for writing/reading multiple pieces of content via encrypted file system repository.
Added unit test skeleton for writing/reading multiple pieces of content with different keys via encrypted file system repository.
Implemented key management skeleton for encrypted content repository.
Multiple content claims can now be encrypted with different keys on the same resource claim and retrieved.
Implemented validation on setting active key id.
Added content repository encryption properties to NiFiProperties.
Implemented configuration of encryption services from NiFiProperties.
Refactored NiFiPropertiesLoader functionality to CryptoUtils for availability in other modules.
Added RepositoryEncryptionConfiguration and repo-specific subclasses for data containers.
Continued refactoring of CryptoUtils and RepositoryEncryptorUtils library methods.
Exposed some internal state of FileSystemRepository via protected getters so encrypted implementation could access.
Refactored EncryptedFileSystemRepository to extend rather than duplicate FSR.
Refactored EFSR to use ECROS which now extends extracted ContentRepositoryOutputStream protected inner class in FSR.
Added unit test to encrypt & decrypt image resource.
Added smaller image resource for easier unit test debugging.
Added importFrom method to resolve issue where GetFile would not encrypt content persisted to repository.
Added text test resource for tests around exporting claim subsets.
Added exportTo methods to handle decrypting encrypted content.
Performed large unit test refactoring, moving shared logic to helper methods.
Added unit test for merged content claim with header/footer/demarcator.
Added unit test for merging content claims each encrypted with a different key.
Ignored non-deterministically failing firewall DNS test.
Added documentation to User and Admin Guide for Encrypted Content Repository.
Added image.
Added refactored utility method for shared ROEM extraction and validation logic in AbstractAESEncryptor.
Replaced ad-hoc generation of ciphertext stream and byte[] for testing with static initialization from pre-generated serialized form for performance.
Cleaned up unused test code.
Cleaned up Javadoc and code comments.
Refactored shared logic.
Fixed checkstyle issue.
Fixed test failure due to error message change.
Added experimental warning to repository implementation classes and User Guide documentation.

Signed-off-by: Joe Witt <joewitt@apache.org>
2019-10-21 13:03:59 -04:00
Yolanda M. Davis 7c9d34f820
NIFI-6778 - Added rules engine service api and Easy Rules Implementation
NIFI-6778 - added comments and component description annotations

NIFI-6778 - added license and notice details

NIFI-6778 - added additional details section and updated description of service

NIFI-6778 - fixed checkstyle error

NIFI-6778 - addressed review comments for documentation and description corrections

NIFI-6778 - added include-rules profile for assembly build

NIFI-6778 - add MVEL and ASM to license and notice

NIFI-6778 - switch to use Jackson for NiFi Rules Json deserialization

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3824
2019-10-21 12:16:11 -04:00
Matt Gilman 8b23929638
NIFI-6786:
- Upgrading nimbus-jose-jwt.

This closes #3830.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-10-21 11:41:29 -04:00
Andrew Lim e9717c76ac
NIFI-6553 Updated Admin Guide with a section that recommends antivirus scan exclusions
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3793.
2019-10-19 16:41:02 +02:00
Andrew Lim a30eb3a08a NIFI-6457 Add Upgrading NiFi section to Admin Guide (#3818)
Signed-off-by: Andy LoPresto <alopresto@apache.org>
2019-10-18 13:25:48 -07:00
samhjelmfelt ea1becac4f
NIFI-6703: Add Stateless NiFi to CLI
NIFI-6703: Fixed extracted nar directory and marked api as experimental
NIFI-6703: Moving nifi-stateless into nifi-framework
NIFI-6703: Refactored to fix jetty/spring issues
NIFI-6703: checkstyle fix
NIFI-6703: updated to mirror traditional NiFi's bootstrap process and java11 dependency management
NIFI-6703: minor changes
NIFI-6703: Documentation fixes

This closes #3795.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-10-17 15:46:32 -04:00
Mark Payne b004f1f94c
NIFI-6779: When resolving node identifier, if NodeIdentifier has a different value for hostname/port for web api or cluster protocol, assume that node is correct about itself instead of assuming that Cluster Coordinator knows best about the other node
NIFI-6779: Remove any conflicting Node Identifiers when a new Node ID is encountered

This closes #3819
2019-10-17 11:21:17 -04:00
Andrew Rodriguez cd90b1d3e1
NIFI-6773 This closes #3820. Adding lz4-framed option for compression/decompression
NIFI-6773 Adding two unit tests for lz4-framed compression/decompression
NIFI-6773 Adding compressed lz4 file for decompression test

Signed-off-by: Joe Witt <joewitt@apache.org>
2019-10-16 17:45:32 -04:00
Mark Bean c4ccd5f505
NIFI-6750: This closes #3810. add FETCH provenance event to FetchFile
Signed-off-by: Joe Witt <joewitt@apache.org>
2019-10-15 14:52:39 -04:00
Bryan Bende d570fe8154
NIFI-4820 This closes #3813. Improving security configuration for Kafka 2.0 processors
Signed-off-by: Joe Witt <joewitt@apache.org>
2019-10-15 14:05:25 -04:00
Matthew Burgess 77e9bda0ff
NIFI-6764: Recreate GroovyShell in ExecuteGroovyScript only when necessary
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3803.
2019-10-15 11:55:52 +02:00
Bahlul Haider 266a59d05a
NIFI-6755 Fix broken link to Quartz documentation
Update the links to the quartz documentation in the User Guide section
for cron trigger scheduling.

This closes #3802.

Signed-off-by: Kevin Doran <kdoran@apache.org>
2019-10-14 15:54:03 -04:00
Mark Payne 9dd0dda688
NIFI-6772: Improved the information that is emitted about Garbage Collection when a Diagnostics Dump is performed
This closes #3809
2019-10-14 11:32:47 -04:00
JF Beauvais 08fe648b1c
[NIFI-6374] ConsumeEWS fails when email attachment has no content type - ensure non null content type
This closes #3596.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2019-10-11 15:10:11 +09:00