Commit Graph

156 Commits

Author SHA1 Message Date
Mark Payne a7b97419e5 NIFI-1626: Throw an Exception proactively if too much state is attempting to be stored via ZooKeeperStateProvider
NIFI-1626: Updated State Management section of Developer Guide

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-03-16 16:12:00 -04:00
Oleg Zhurakousky 8e7e2916fa NIFI-1464 fixed admin guide 2016-03-11 12:54:50 -05:00
Oleg Zhurakousky 59fac58c96 NIFI-1464 ensured that OnUnscheduled is treated the same as OnScheduled
NIFI-1464 polished javadocs, error messages and docs
2016-03-11 12:54:50 -05:00
Oleg Zhurakousky 0c5b1c27f2 NIFI-1464, Refactored Processor's life-cycle operation sequence
* Simplified and cleaned StandardProcessScheduler.start/stopProcessor methods
* Added stop/start operations to ProcessorNode.
* Removed unnecessary synchronization blocks related to ScheduledState in favor of enforcing order and idempotency via CAS operations. Those synchronization blocks were causing intermittent deadlocks whenever @OnScheduled blocks indefinitely.
* Added support for stopping the service when @OnScheduled operation hangs.
* Fixed the order of life-cycle operation invocation ensuring that each operation can *only* be invoked at the appropriate time
* Removed unnecessary locks from StandardProcessNode since Atomic variables are used.
* Removed calls to @OnStopped from ContinuallyRunningProcessTask while ensuring that procesor's full shut down in implementation of StandardProcessorNode.stop() method.
* Removed dead code
* Added comprehensive tests suite that covers 95% of Processor's life-cycle operations within the scope of FlowController, StandardProcesssScheduler and StandardProcessNode
* Improved and added javadocs on covered operations with detailed explanations.
2016-03-11 12:54:50 -05:00
Richard Miskin 99c7fe3b44 NIFI-963 Update admin guide to cover configuration of multiple lib directories
This closes #250.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-27 10:01:38 -05:00
Matt Gilman bb2d2c3674 Clarifying embedded ZooKeeper on NCM is not possible. 2016-02-22 16:34:43 -05:00
Matt Gilman 8cff13e749 Minor changes to the state management section of the admin guide. 2016-02-22 15:21:56 -05:00
Tony Kurc 6c2ba997a0 Merge branch 'NIFI-1379-RC3' 2016-02-16 20:13:41 -05:00
Randy Gelhausen e4d0ec7e60 NIFI-1508: NiFi Site to Site doc tweak
This closes #215.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-16 15:17:28 -05:00
Tony Kurc ad73b5c9d9 NIFI-1379-RC3 prepare for next development iteration 2016-02-12 17:28:10 -05:00
Tony Kurc 8309dba80b NIFI-1379-RC3 prepare release nifi-0.5.0-RC3 2016-02-12 17:27:59 -05:00
Lars Francke d01449ee72 NIFI-1496: State Manager documentation link wrong. This closes #214
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-02-10 08:46:42 -05:00
Tony Kurc 5be83166ee NIFI-1485: Updated 'header' section of NiFi NOTICE files 2016-02-07 13:52:00 -05:00
Tony Kurc 303f8eabf1 NIFI-1379: Move to 0.5.0-SNAPSHOT, add tkurc code signing key to keys 2016-02-06 08:49:48 -05:00
Andy LoPresto b407379670 NIFI-1257 Resolved legacy compatibility issue with NiFi legacy KDF salt length dependent on cipher block size.
Replaced screenshot for NiFiLegacy salt encoding.
Added description of legacy salt length determination in admin guide.
Added logic for NiFiLegacyCipherProvider to generate and validate salts of the length determined by the cipher block size.
Changed EncryptContent to default to Bcrypt KDF.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-05 23:38:58 -05:00
Lars Francke 0d72969053 NIFI-1482 - DeveloperGuide :: Controller Service is using wrong method name
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-05 23:35:10 -05:00
Aldrin Piri 4df6512126 Merge branch 'NIFI-259' 2016-02-05 14:09:29 -05:00
Andy LoPresto 498b5023ce NIFI-1257 NIFI-1259
Added utility method to return the maximum acceptable password length for PBE ciphers on JVM with limited strength crypto because BC implementation is undocumented (based on empirical evidence).
Updated EncryptionMethod definitions to accurately reflect need for unlimited strength crypto according to algorithm key length.
Added processor logic to invoke keyed cipher.
Added EncryptContent processor property for raw hex key (always visible until NIFI-1121).
Added validations for KDF (keyed and PBE) and hex key.
Added utility method to return list of valid key lengths for algorithm.
Added description to allowable values for KDF and encryption method in EncryptContent processor.
Added IV read/write to KeyedCipherProvider and changed from interface to abstract class.
Added salt read/write logic to NifiLegacy and OpenSSL cipher providers.
Changed RandomIVPBECipherProvider from interface to abstract class.
Updated strong KDF implementations.
Renamed CipherFactory to CipherProviderFactory.
Added unit test for registered KDF resolution from factory.
Updated default iteration count for PBKDF2 cipher provider.
Implemented Scrypt cipher provider.
Added salt translator from mcrypt format to Java format.
Added unit tests for salt formatting and validation.
Added surefire block to groovy unit test profile to enforce 3072 MB heap for Scrypt test.
Added local Java implementation of Scrypt KDF (and underlying PBKDF2 KDF) from Will Glozer.
Defined interface for KeyedCipherProvider.
Implemented AES implementation for KeyedCipherProvider.
Added Ruby script to test/resources for external compatibility check.
Added key length check to PBKDF2 cipher provider.
Changed default PRF to SHA-512.
Added salt and key length check to PBKDF2 cipher provider.
Added utility method to check key length validity for cipher families.
Added Bcrypt implementation.
Implemented PBKDF2 cipher provider.
Added default constructor with strong choices for PBKDF2 cipher provider.
Implemented NiFiLegacyCipherProvider and added unit tests.
Added key length parameter to PBKDF2 cipher provider.
Added PRF resolution to PBKDF2 cipher provider.
Added RandomIVPBECipherProvider to allow for non-deterministic IVs.
Added new keyed encryption methods and added boolean field for compatibility with new KDFs.
Added CipherFactory.
Improved Javadoc in NiFi legacy cipher provider and OpenSSL cipher provider.
Added KeyedCipherProvider interface.
Added OpenSSL PKCS#5 v1.5 EVP_BytesToKey cipher provider and unit test.

This closes #201.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-04 19:40:55 -05:00
Matt Gilman adfa5dc0eb NIFI-259: - Moving the state management documentation to after the Clustering configuration.
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-03 10:13:23 -05:00
Matt Gilman 7711106d62 NIFI-259: - Adding additional documentation for embedded ZK.
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-03 10:13:19 -05:00
Aldrin Piri 8a668fd344 NIFI-259: Fixed bug that caused StateProvider.replace to return true if the value had never been set, instead of false. Fixed typos in administration-guide
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-02 23:03:31 -05:00
Matt Gilman 72c8467b9f NIFI-259:
- Addressing typo in documentation.
- Minor tweaks to admin guide.
- Adding support to stand up a ZooKeeperServer when a quorum peer is not distributed (ie supporting both embedded standalone and cluster).

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-01 14:07:38 -05:00
Aldrin Piri 6902812678 NIFI-259: Adjusting property description of the connect string to be more specific concerning socket tuple format and the associated default port if one is not specified. Providing minor cleanup and removing unused import. 2016-02-01 10:38:45 -05:00
Matt Gilman 4e50263581 NIFI-1445:
- Updating supported browsers sections to accurately reflect the capabilities the UI employs and the dependencies the UI uses.
2016-01-28 15:04:48 -05:00
Mark Payne 8a9a44c102 NIFI-1078: Fixed typo in docs and added additional examples 2016-01-28 09:35:05 -05:00
jpercivall 23364f554c NIFI-1078 Fixing the 'now' EL documentation
Signed-off-by: Mark Payne <markap14@hotmail.com>
2016-01-28 08:56:31 -05:00
Mark Payne 4d88aaedc5 NIFI-1258: Added a new function named getDelimitedField to the Expression Language and put together a guide that walks through how to add a new function
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-01-21 22:09:25 -08:00
Mark Payne f6ec437bc7 Merge branch 'master' into NIFI-259 2016-01-21 09:32:02 -05:00
Matt Gilman 0d7edcb3ac NIFI-108:
- Removing sort from UI.
- Addressing issues with listing and flowfile retrieval when clustered.
- Making the context menu item available when source and destination are still running.
- Adding a refresh button to the queue listing table.
- Fixing the flowfile summary sorting in the cluster manager.
- Adding a message when the source or destination of a connection is actively running.
- Updating the documentation regarding queue interaction.
- Updating the error message when a flowfile is no longer in the active queue.
- Updated queue listing to allow listing to be done while source and destination are running but not sort or have ability to search
- Added heartbeat when we finish clearing queue
- Addressing comments from review.
2016-01-21 08:13:32 -05:00
Mark Payne 593f1288d8 NIFI-259: Bug fixes, unit tests, documentation updates 2016-01-20 10:16:14 -05:00
Mark Payne 0cd6f80f36 NIFI-259: Bug fixes 2016-01-13 15:11:53 -05:00
Mark Payne 0151b1eed1 NIFI-259: Fixed checkstyle violations 2016-01-13 13:57:47 -05:00
Mark Payne bbce596d74 NIFI-259: Updated GetHBase to use new State Management; bug fixes; updated docs 2016-01-13 12:47:08 -05:00
Mark Payne 774c29a4da NIFI-259: Added Stateful annotation as described on ticket 2016-01-12 15:28:35 -05:00
Mark Payne 7a3e3efce1 NIFI-259: Updated documentation, added digest username/password 2016-01-11 16:43:46 -05:00
Mark Payne d2a969e3d6 NIFI-259: Initial implementation of State Management feature 2016-01-11 16:38:52 -05:00
Mark Payne 57dadb7286 NIFI-259: Initial implementation of State Management feature 2016-01-11 08:28:32 -05:00
Jenn Barnabee ad4019f9aa NIFI-1235: Added section on Bending Connections in the Connecting Components section of the User Guide.
Signed-off-by: joewitt <joewitt@apache.org>
2016-01-10 14:52:10 -05:00
joewitt 684f48ff92 NIFI-1374 updated admin guide to provide advice for permgen and codecache
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-01-10 14:48:20 -05:00
Jon Anderson 035562bb33 NIFI 1007 Updated Admin guide for stats configuration
Signed-off-by: joewitt <joewitt@apache.org>
2016-01-09 22:18:39 -05:00
Mark Payne b330fd1601 NIFI-108: Built out 'skeleton' of the requests so that web tier can be written against it
NIFI-108: Implementing ability to list FlowFiles in a queue

NIFI-108:
- Starting to add support for endpoints that will listing flowfiles in a queue.

NIFI-108: Added merging of response for listing of flowfiles in cluster manager

NIFI-108:
- Starting to add support for endpoints that will listing flowfiles in a queue.

NIFI-108:
- Starting to add support for endpoints that will listing flowfiles in a queue.

NIFI-108:
- Adding checkstyle issues.

NIFI-108: Add clusterNodeId to FlowFileSummaryDTO

NIFI-108: Added unit tests; added verifyCanList method to queue; fixed bugs

NIFI-108:
- Adding compilation error for IOException from getFlowFile().
- Code clean up.
- Javadocs.

NIFI-108:
- Verifying two phase commit for queue listing.
- Fixing checkstyle.
- Ensuring drop and listing requests are merged when created when clustered.

NIFI-108:
- Adding initial listing capabilities.
- Passing through the sort column and direction.

NIFI-108:
- Removing Delete FlowFile button.
- Ensuring sort flags are being passed correctly.
- Setting column widths.
- Also including the cluster node address in the flowfile summaries.

NIFI-108:
- Including queue size statistics in listing request.
- Showing connection name.

NIFI-108:
- Including queue size statistics in listing request.
- Ensuring verifyCanList runs when appropriate.

NIFI-108:
- Adding initial support for viewing flowfile details dialog.
- Adding initial support for click to content.

NIFI-108:
- Allowing the flowfile details dialog to be draggable.

NIFI-108:
- Only showing the flowfile listing table when the listing is successful and the listing is not empty.

NIFI-108:
- Reseting the queue stats when closing the listing table.

NIFI-108: Implemented sorting when performing listing of FlowFiles

NIFI-108: Fixed bug that caused the listFlowFiles operation to wait on a readLock before returning and performing work asynchronously; fixed bug in Write-Ahead FlowFile Repository that caused ContentClaims to be queued up for destruction instead of ResourceClaims - this caused millions of ContentClaims to be queued up instead of a single ResourceClaim in some tests

NIFI-108:
- Ensured the column sort indicator is reset when a new listing is opened.
- Removing unused import.

NIFI-108:
- Addressed issues found during the review.
2016-01-06 16:20:23 -05:00
Lemoda Laim 9bb77163cf NIFI-1334 fixed minor typos in expression-lang-guide documentation
This closes #152

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2015-12-29 12:20:21 -05:00
joewitt f4ac8d75c5 NIFI-1312-RC1 prepare for next development iteration 2015-12-19 00:41:04 -05:00
joewitt d624ea4866 NIFI-1312-RC1 prepare release nifi-0.4.1-RC1 2015-12-19 00:40:53 -05:00
Matt Burgess 04e9606770 NIFI-1290 Document the need for an explicit Validator for a PropertyDescriptor
Signed-off-by: joewitt <joewitt@apache.org>
2015-12-17 09:24:28 -08:00
joewitt d755e43ec8 NIFI-1122_nifi-0.4.0-RC2prepare for next development iteration 2015-12-08 13:00:10 -05:00
joewitt b66c029090 NIFI-1122_nifi-0.4.0-RC2prepare release nifi-0.4.0-RC2 2015-12-08 12:59:59 -05:00
joewitt 0f3a620154 NIFI-1250 removed erroneous ContentType reference and added docs for attributes to index
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2015-12-04 12:58:02 -05:00
Matt Gilman fbaf1c36de NIFI-1138: - Fixing typo in expression language documentation.
Signed-off-by: joewitt <joewitt@apache.org>
2015-12-01 16:06:48 -05:00
Matt Gilman 6bce858e4a NIFI-655:
- Updated user guide with screenshots of login process.
- Tweaked wording in admin guide.
- Triggering login on enter press in login form.
2015-11-25 16:42:22 -05:00
Matt Gilman c073253366 NIFI-655:
- Update admin guide with documentation for username/password authentication.
- Setting default anonymous roles to none.
- Making account status messages to users more clear.
- Deleting user keys when an admin revokes/deletes an account.
- Updating authentication filter to error back whenever authentication fails.
2015-11-25 14:17:23 -05:00
Jenn Barnabee b74617ba6a NIFI-1210: Fixes Copy and Paste descriptions and changes several references to canvas instead of graph to maintain consistency. Also capitalizes Site-to-Site in a couple of places for consistency.
Reviewed by Tony Kurc (tkurc@apache.org)
2015-11-22 08:29:14 -05:00
Jenn Barnabee 327342916e NIFI-1210: Makes various changes to the User Guide for the 0.4.0 version release.
Reviewed and amended (see ticket for details) by Tony Kurc (tkurc@apache.org)
2015-11-21 22:57:19 -05:00
Mark Payne d16392c618 NIFI-1114: Added documentation for nifi.kerberos.krb5.file property 2015-11-17 08:45:28 -05:00
Jenn Barnabee f98e335cde NIFI-1178: Updates to Administration Guide - System Properties section.
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-11-17 08:16:45 -05:00
joewitt 99629646fe NIFI-1122 moved to 0.4.0-SNAPSHOT 2015-11-06 23:41:15 -05:00
joewitt 92099ede2f NIFI-1058 2015-11-06 23:20:03 -05:00
Mark Payne 4c0cf7d72b NIFI-973: Created a Getting Started Guide
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-10-27 12:01:17 -04:00
Mark Payne 0636f0e731 NIFI-810: Merged master into branch 2015-10-25 11:02:40 -04:00
Aldrin Piri f6d3427955 NIFI-1024 Correcting references to nifi.cluster.manager.protocol.port 2015-10-09 14:22:00 -04:00
Mark Payne 4afd8f88f8 NIFI-810: Created RequiresInput annotation and ensure that processors are invalid if connections do not agree 2015-10-07 17:26:14 -04:00
Matt Gilman ded74ec94c NIFI-876 prepare for next development iteration 2015-09-14 21:48:11 -04:00
Matt Gilman 2ec735e350 NIFI-876 prepare release nifi-0.3.0-RC1 2015-09-14 21:48:00 -04:00
Mark Payne 992e841027 NIFI-948: Provide capability to register notifications for nifi lifecycle events 2015-09-13 19:55:12 -04:00
Joseph Percivall 9dae76b9c4 Initial commit of NIFI-867
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2015-09-09 15:09:33 -04:00
Joseph Percivall 59b7a71c8a NIFI-907: addressing comments to fix periods and link from expression lanaguage doc to developer doc
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2015-09-09 13:45:56 -04:00
Joseph Percivall 47e9a3cdee NIFI-907: Addressing comment on adding the value of the attribute within the enum
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2015-09-09 13:45:56 -04:00
Joseph Percivall 690e674a0d NIFI-907: Initial commit of core attributes in doc
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2015-09-09 13:45:56 -04:00
Matt Gilman 6f8eb9f5e3 NIFI-876:
- Updating the version of the nifi-nar-maven-plugin.
- Updating the minimum version of maven.
2015-08-23 12:32:52 -04:00
Mark Payne 4f92193159 NIFI-787: Ensure that Mock Framework passes a ProcessContext to methods using the @OnStopped annotation (if the method takes a ProcessContext object as an argument); updated developer guide to reflect that the object is allowed as an argument
Signed-off-by: joewitt <joewitt@apache.org>
2015-08-19 20:11:01 -04:00
Mark Payne 8ebf1f03c2 Merge branch 'nifi-868' of https://github.com/pukkaone/nifi into NIFI-868 2015-08-19 12:56:26 -04:00
Mark Payne 19f7db6986 NIFI-793: Added multi-threading to the indexing in the Persistent Provenance Repository 2015-08-19 12:24:25 -04:00
Chin Huang 8f98f80938 NIFI-868 Configure advertised host name of Remote Process Group Input Port
Before this change, the host given out to clients to connect to a Remote
Process Group Input Port is the host where the NiFi instance runs.
However, sometimes the binding host is different from the host that
clients connect to. For example, when a NiFi instance runs inside a
Docker container, a client on a separate machine must connect to the
Docker host which forwards the connection to the container.

Add a configuration property to specify the host name to give out to
clients to connect to a Remote Process Group Input Port. If the property
is not configured, then give out the name of host running the NiFi
instance.
2015-08-18 21:35:28 -07:00
Aldrin Piri c0dad8d1d8 NIFI-848 Extending the Administrator Guide to include guidance on providing multiple provenance and content repositories. 2015-08-17 10:46:13 -04:00
joewitt aa99884782 NIFI-850 removed nifi parent, updated nifi pom, moved all nifi subdirs up one level, fixed readme. 2015-08-15 13:12:22 -04:00
joewitt 300952a984 Reworked overall directory structure to make releasing nifi vs maven plugis easier 2015-01-15 21:11:07 -05:00
Jenn Barnabee a6f0948af1 NIFI-260 Replaced four screenshots so they have new logo
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2015-01-14 12:46:53 -05:00
Jenn Barnabee 78c069fb5b NIFI-162 Added a little to the admin guide
Signed-off-by: joewitt <joewitt@apache.org>
2015-01-12 07:56:32 -05:00
joewitt d996060dbf NIFI-162 added some content to the administrator guide 2015-01-09 13:33:34 -05:00
Jenn Barnabee d08cda1b6a NIFI-234 Updated User Guide to include Example Flow, description of prioritizers, updating connections, navigating within a flow (upstream and downstream), and touched on other management tools like Flow History, Flow Settings, and User Management, but said that is covered in Admin Guide.
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2015-01-08 14:34:49 -05:00
Mark Payne bcc271046e NIFI-218: Fixed typo that caused markup not to render properly for any and all matching attributes functions 2015-01-07 14:38:56 -05:00
Mark Payne e631c493d4 NIFI-218: Added isEmpty and replaceEmpty functions to doc 2015-01-07 14:05:05 -05:00
joewitt 4d11ff7830 NIFI-150 small mod to fix references title 2015-01-06 21:54:29 -05:00
Jenn Barnabee c27f7b0574 NIFI-162 In the User Guide, I added an overview (stolen from the web page), added the Data Provenance section, and copy-edited all the existing sections.
Signed-off-by: joewitt <joewitt@apache.org>
2015-01-06 21:31:04 -05:00
Matt Gilman 23dcdeb61b NIFI-216:
- Ignoring irrelevant whitespace in EL editor.
- Adding IDE style function details for each function in the auto complete.
2015-01-06 11:41:13 -05:00
Mark Payne 2d0b163961 NIFI-221: included controller service needed in nar-bundles pom 2015-01-06 10:16:00 -05:00
Jenn Barnabee 7e70fd53e9 NIFI-162 I copy-edited the Overview
Signed-off-by: joewitt <joewitt@apache.org>
2015-01-05 17:10:23 -05:00
Mark Payne 090ad38679 NIFI-218: Finished documenting the functions; changed formatting so that classnames can be applied to the functions, arguments, description, return type, and subject Types. This allows the values to be pulled out into the UI. 2015-01-05 16:37:17 -05:00
joewitt 9e35c7da9f NIFI-222 moved away from project.version and instead use explicit version declaration. Release plugin should take care of it anyway 2015-01-03 00:58:58 -05:00
Mark Payne 68b7ad78cc NIFI-218: Started Expression Language guide 2014-12-31 16:18:38 -05:00
joewitt ad90fbf24f NIFI-162 more improvements to overview 2014-12-31 12:05:00 -05:00
joewitt d6c02a5640 NIFI-162 added a couple new sections to the overview to address architecture of nifi and performance 2014-12-31 02:39:01 -05:00
Mark Payne 43991777da NIFI-150: Added information about Templates 2014-12-30 15:44:16 -05:00
Mark Payne 6f1c12f5ef Updated user guide 2014-12-29 16:17:48 -05:00
joewitt 87b07384a5 NIFI-162 filled in a basic overview. Definitely still needs work though. Too wordy. Too specific. 2014-12-28 15:23:45 -05:00
joewitt b6f2dd280c NIFI-162 all doc components prepped and integrated to build process. Now just to fill in content 2014-12-26 23:06:31 -05:00
joewitt 21182ac260 NIFI-150 updated to ensure docs get built and end up in the assembly under docs. 2014-12-19 16:34:25 -05:00
joewitt d850768765 NIFI-150 fixed error with inserting license header 2014-12-19 15:32:55 -05:00
joewitt 4031bd5495 NIFI-150 fixed error with inserting license header - still need to figure out the license header addition 2014-12-19 14:57:13 -05:00
Mark Payne 9c38f358b8 NIFI-150: Removed old formatted document and updated asciidoc version of user guide 2014-12-19 14:11:43 -05:00