Commit Graph

4026 Commits

Author SHA1 Message Date
Andrew Lim bac175a00f NIFI-4276 Add Write Ahead Provenance section to User Guide
Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #2074
2017-08-13 10:01:36 -04:00
Mark Payne 451f9cf124 NIFI-4142: This closes #2015. Refactored Record Reader/Writer to allow for reading/writing "raw records". Implemented ValidateRecord. Updated Record Reader to take two parameters for nextRecord: (boolean coerceTypes) and (boolean dropUnknownFields)
Signed-off-by: joewitt <joewitt@apache.org>
2017-08-11 22:01:46 -07:00
Mark Payne 84935d4f78 NIFI-4023, NIFI-4077 This closes #2075. Addressed issue where repository was aging off the wrong index. When it should age off Index 1, it was removing Index 2. As a result, the earliest index is never aged off, and the newest index could potentially be aged off before it is ready to be. Also addressed issue where a query that attempts to read an event that has aged off throws FileNotFoundException (NIFI-4077) instead of skipping over the event. The JavaDocs indicate that the EventIterator should skip records that it cannot find, but SelectiveRecordReaderEventIterator throw FileNotFoundException instead
Signed-off-by: joewitt <joewitt@apache.org>
2017-08-11 22:01:46 -07:00
Andy LoPresto ae940d8624 NIFI-4237 Added working test for StringEncryptor decryption of sensitive flow values in FlowFromDOMFactory.
NIFI-4237 Cleaned up unused alternate approaches.

NIFI-4237 Added failing unit test for better error message.

NIFI-4237 Added logic to capture unhelpful encryption exception and provide context in message. All tests pass.

This closes #2077
2017-08-11 23:57:15 -04:00
Bryan Bende 28d5a70ec0
NIFI-4288 Ignoring tests that intermittently fail in TestPutUDP and TestPutTCP
This closes #2076.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-08-11 17:27:55 -07:00
Mike Moser d6ebc53076 NIFI-3736 modify default nifi.content.claim.max.appendable.size in nifi.properties to 1 MB 2017-08-11 15:53:36 -04:00
Bryan Bende f7da7e67f4 NIFI-4275 Adding support for specifying the timestamp on PutHBase processors
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2070.
2017-08-11 18:04:13 +02:00
Andy LoPresto 760fd75bee NIFI-4187 Resolved issue where restart failed due to missing sensitive key file.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2073.
2017-08-11 13:02:38 +02:00
Wayne Steel a6e8f0afe3 NIFI-4177 This closes #2002. MergeContent - Tar - Save modification timestamp like Tar does
Signed-off-by: joewitt <joewitt@apache.org>
2017-08-10 19:35:46 -07:00
Pierre Villard 69f263c4d1
NIFI-2167 - include disabled state for processors when copy/paste/templates
This closes #2057.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-08-10 18:41:24 -07:00
Mark Payne 96f5407ed9 NIFI-3329 This closes #2053. Removed check for latest version of flowfile when migrating flowfile from 1 session to another because we now longer are requiring latest version of flowfile be passed to session anywhere else. It was intended to be removed from StandardProcessSession.migrate when it was removed from StandardProcessSession.validateRecordState but it was overlooked. 2017-08-10 18:13:28 -07:00
Matt Gilman ec0c8278b4
NIFI-4206:
- Updating admin guide to include instructions for running NiFi behind a proxy.
- Including a brief example proxy configuration for NiFi specific properties.

This closes #2023.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-08-10 17:08:59 -07:00
Matt Gilman 20d6596df0 NIFI-3232:
- Adding support for cascading sub context menus.

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #2072
2017-08-10 17:01:25 -04:00
Pierre Villard c03f4e731b
NIFI-968 - add @OnPrimaryNodeStateChange to dev guide
This closes #2027.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-08-10 12:21:04 -07:00
Pierre Villard 18c82eb6af
NIFI-4277 Fixed exception logging in StandardLogRepository
This closes #2068.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-08-10 11:54:32 -07:00
Matt Gilman 528b82634f
NIFI-4210:
- Introducing support for OpenId Connect.
- Updating REST API and UI to support the authorization code flow.
- Adding/fixing documentation.
- Implementing time constant equality checks where appropriate.
- Corrected error handling during startup and throughout the OIDC login sequence.
- Redacting the token values from the user log.
- Defaulting to RS256 when not preferred algorithm is specified.
- Marking the OIDC endpoints as non-guaranteed in to allow for minor adjustments if/when additional SSO techniques are introduced.

This closes #2047.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-08-10 11:15:35 -07:00
yuri1969 505e93065e NIFI-1027 - Treat label and node of a provenance graph event as one unit
The JIRA issue asks for treating both node and its label as one unit.
Described cursor issues seems to be fixed already.

However, there is an annoying dead space between a node and label
preventing displaying context menu, etc. Due to SVG group's nature
there has been added an opaque joint to remove the dead space.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2059.
2017-08-10 10:54:47 +02:00
Andy LoPresto 5150dff70b NIFI-4274 Removed EL evaluation logic from custom file validator.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2071.
2017-08-10 10:05:16 +02:00
Pierre Villard 6ff8321cf7
NIFI-4278 - add error message to invalid FFs in ValidateXml
This closes #2069.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-08-09 21:20:08 -07:00
Pierre Villard 9f1267e949
NIFI-4222 - Adding CN by default in SANs for generated certificates with tls-toolkit
This closes #2042.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-08-09 19:04:36 -07:00
Matt Burgess 9c4fdd4ef3 NIFI-4192: Add more Avro/metadata merge options to MergeContent
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2067.
2017-08-09 12:55:12 +02:00
Mike Thomsen 67819e5019
NIFI-4124 Added org.apache.nifi.mongo.PutMongoRecord.
Added changes based on code review. Changed:
* Put record reader instantiation inside of try-with.
* Put a batch size for the insert List.
* Ensured that session.transfer() to the success relationship will always happen.
Removed an unused import to fix the style check.

This closes #1945.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-08-07 22:10:27 -07:00
Bryan Bende 0029f025f8 NIFI-4152 Initial commit of ListenTCPRecord 2017-08-07 22:44:11 +02:00
yuri1969 e44f436bd8 NIFI-4271 - Revert double-click behavior back...
...to delete a bend point

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #2064
2017-08-07 15:58:48 -04:00
zhengsg ee3b58368d NIFI-4267 Change the class name for log in FileRemovalAction
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2061.
2017-08-07 21:42:46 +02:00
Wil Selwood f7f64a572c NIFI-4268: Add fetch tag to FetchFile processor.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2062.
2017-08-07 21:38:55 +02:00
Andy LoPresto 675d989003
NIFI-4139
- Moved key provider interface and implementations from nifi-data-provenance-utils module to nifi-security-utils module.
- Refactored duplicate byte[] concatenation methods from utility classes and removed deprecation warnings from CipherUtility.
- Created KeyProviderFactory to encapsulate key provider instantiation logic.
- Added logic to handle legacy package configuration values for key providers.
- Added unit tests.
- Added resource files for un/limited strength cryptography scenarios.
- Added ASL to test resources.
- Moved legacy FQCN handling logic to CryptUtils.
- Added unit tests to ensure application startup logic handles legacy FQCNs.
- Moved master key extraction/provision out of FBKP.
- Removed nifi-security-utils dependency on nifi-properties-loader module.
- Added unit tests.
2017-08-07 15:21:58 -04:00
Pierre Villard 8b54c2604c NIFI-4082 - Added EL on GetMongo properties
NIFI-4082 - Added EL on DB, URI and Collection

NIFI-4082 - Added UT for EL evaluation (URI, DB, Collection) and fixed ex. message for document validator.

This closes #1969
2017-08-07 13:40:27 -04:00
yuri1969 329b1caf82 NIFI-4266 - Allow double-click to display Event
...details

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #2058
2017-08-07 08:44:13 -04:00
Bryan Bende f8f1cc8d0d
NIFI-4024 Ensuring InputStream gets closed and cleaning up complex field handling
This closes #1961.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-08-04 14:54:05 -04:00
Mike Thomsen 496a32e12c
NIFI-4024 Added org.apache.nifi.hbase.PutHBaseRecord
Signed-off-by: Bryan Bende <bbende@apache.org>
2017-08-04 14:54:01 -04:00
Yolanda M. Davis afd4f9e034
NIFI-4022 - Initial update for SASL support for cluster management in Zookeeper
NIFI-4022 - adding sasl documentation update and update to test

This closes #2046.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-08-04 14:19:35 -04:00
Mark Payne 969bbe654c NIFI-4051: Ensure that we use the appropriate Lexical syntax for SQL query validation as we do when running
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2054.
2017-08-04 16:31:56 +02:00
Wesley-Lawrence 40cde0466a NIFI-4215 NiFi can now parse an Avro schema of a record that references an already defined record, including itself.
This closes #2052.
2017-08-03 15:13:07 -04:00
Matt Burgess 2d33582147 NIFI-4253: Add Oracle 12+ database adapter
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2048.
2017-08-03 09:59:29 +02:00
James Wing 2502b79bae NIFI-4215 Revert Complex Avro Schema Changes
This reverts commit cf49a58ee7.
2017-08-01 21:03:04 -07:00
yuri1969 bcf60aa556 Add double-click shortcut to RPG
Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #2009
2017-07-31 10:58:49 -04:00
yuri1969 ef9cb5be23 NIFI-1580 - Allow double-click to display config
* Added double-click shortcut opening config/details dialog to processors,
connections, ports and labels.
* Created a base for further default action selection, disabling, etc.
* Omitted default action configuration UI - that might be a separate JIRA ticket.
2017-07-31 10:36:37 -04:00
Wesley-Lawrence cf49a58ee7 NIFI-4215 Allow Complex Avro Schema Parsing
NiFi can now parse an Avro schema of a record that references an already defined record, including itself.

Signed-off-by: James Wing <jvwing@gmail.com>

This closes #2034.
2017-07-30 16:31:39 -07:00
mans2singh 0bb1411532 NIFI-4212 - RethinkDB Delete Processor
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #2030.
2017-07-29 14:49:22 -07:00
Leah Anderson 70878fe6d6 NIFI-4052 Add DynamicProperty annotation for LookupRecord
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2032.
2017-07-28 17:12:06 +02:00
Mark Payne 1d6b486b63 NIFI-4232: Ensure that we handle conversions to Avro Arrays properly. Also, if unable to convert a value to the expected object, include in the log message the (fully qualified) name of the field that is problematic
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2040.
2017-07-27 08:57:25 +02:00
Matt Burgess dc4006f423 NIFI-3335: Add initial.maxvalue support to GenerateTableFetch
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2039.
2017-07-27 08:43:00 +02:00
Pierre Villard 759f81bc1b
NIFI-4221 - Print app startup in human-readable time
This closes #2038.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-07-25 15:25:55 -07:00
Matt Burgess 9a5d4ff6b3 NIFI-4217 - Retain original DDL in CaptureChangeMySQL
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2036.
2017-07-25 17:54:11 +02:00
Matt Gilman 743c6b9c17 NIFI-4032: - Introducing the ManagedRangerAuthorizer. - Introducing the AuthorizationAuditor. - Updating authorization requests to utilize Authorizable where ever possible so allow for a singular place to audit resource not found as denied when the parent authorizable is null (no more inheritance). - Updating unit tests as appropriate. - Addressing issues with broken web-api integration tests.
NIFI-4032: - Generating the appropriate fingerprint for the ManagedRangerAuthorizer based on whether the UserGroupProvider is configurable. - Adding unit tests.

Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #2019
2017-07-24 15:51:27 -04:00
Pierre Villard 10692256d6 NIFI-917 Improve Rendering of Component Docs
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #2025.
2017-07-23 11:30:04 -07:00
Peter G. Horvath 082919f919 NIFI-4147 Refactor RunNiFi for Testability
Class org.apache.nifi.bootstrap.RunNiFi is not designed with extensibility
in mind: changes to allow RunNiFi to be subclassed in a test harness

Signed-off-by: James Wing <jvwing@gmail.com>

This closes #1972.
2017-07-22 11:11:24 -07:00
Leah Anderson 3371e915cc NIFI-4087 This closes #2026. Fix to allow exclusion of filename from tika criteria. 2017-07-21 16:56:56 -04:00
Mark Payne 695e8aa98f
NIFI-106:
- Expose processors' counters in Stats History
- Only include counters in Processors' Status History if user has read access to corresponding Processor
- Addressed review feedback. Found and addressed bug where a counter is not present in all of the aggregate snaphot values for status history, resulting in the UI not rendering the chart properly
- This closes #1872
2017-07-21 14:18:11 -04:00