Commit Graph

127 Commits

Author SHA1 Message Date
Chris McDermott abad7d805e NIFI-1660 - Enhance the expression language with jsonPath function 2016-06-07 15:34:36 -04:00
Mark Payne cd011731ab NIFI-1960: Update admin guide regarding documentation for clustering
NIFI-1960: Updates to guide as follow-up from PR review
2016-06-03 15:23:39 -04:00
Andy LoPresto 8127314975
NIFI-1919 Added replaceFirst() expression language method which accepts literal or pattern for replacement.
Reverted whitespace changes. (+8 squashed commits)
Squashed commits:
[329755c] NIFI-1919 Reverted import re-organization from IDE.
[cf73c2f] NIFI-1919 Updated expression language guide.
[d9a1455] NIFI-1919 Reverted changes to ReplaceEvaluator.
Added ReplaceFirstEvaluator.
Added replace first logic to Query buildFunctionEvaluator.
Added unit tests.
[e2eb880] NIFI-1919 Added replaceFirst to AttributeExpression lexer and parser grammar definitions.
[11fe913] NIFI-1919 Ignored demonstrative test for replaceAll as it behaves as expected.
[af97be1] NIFI-1919 Changed ReplaceEvaluator to use String#replaceFirst which interprets regex instead of compiling as literal.
Demonstrative unit test now passes but two existing unit tests fail. I am not sure these tests are correct.
[f24f17b] NIFI-1919 Added working unit test to illustrate fix.
[8a0d43b] NIFI-1919 Added Groovy unit test to demonstrate issue.
Added DelegatingMetaClass code to record it (test not complete).

This closes #474.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-06-01 16:39:58 -07:00
Pierre Villard 372ffb8aa0 NIFI-1811 Removed ProcessorLog and updated dependent interfaces
This closes #403.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-05-19 13:08:09 -04:00
James Wing 347b281b2d NIFI-1887 Updating default timeout in Admin Guide
This closes #447.

Signed-off-by: James Wing <jvwing@gmail.com>
2016-05-16 11:57:33 -07:00
Pierre Villard b7aa381ab4 NIFI-1826 Updated documentation
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #396
2016-05-11 10:02:11 -04:00
Andrew Lim dc4f983c7a
NIFI-1862 User Guide corrections/improvements
Made multiple edits to the User Guide documentation for correcting errors (spelling/grammatical) and improving readability.

This closes #427.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-05-09 10:31:33 -07:00
Yolanda M. Davis defb6f5b61 NIFI-361 - Create Processors to mutate JSON data
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #354
2016-05-03 14:38:11 -04:00
jpercivall 0557095613 NIFI-1028 initial commit of NiFi In Depth documentation
NIFI-1028 addressing review comments
This closes #339
2016-05-03 14:01:45 -04:00
Matt Burgess 106b0fa0fc NIFI-981: Added SelectHiveQL and PutHiveQL processors
This closes #410.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-05-03 13:51:38 -04:00
Aldrin Piri eded0de154 NIFI-1807 Adding information on volatile content repository configuration settings. 2016-04-23 14:53:51 -04:00
Randy Gelhausen b4309e86bd NIFI-1807 Update administration-guide.adoc adding flowfile, provenance, and content repository volatile implementation documentation.
This closes #378.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-04-23 14:53:27 -04:00
Chris McDermott 32398693d5 NIFI-1661 add random() function to expression language.
Signed-off-by: Joe Skora <jskora@apache.org>
2016-04-21 14:05:00 -04:00
Matt Gilman 153f63ef43 NIFI-1551:
- Removing the AuthorityProvider.
- Refactoring REST API in preparation for introduction of the Authorizer.
- Updating UI accordingly.
- Removing unneeded properties from nifi.properties.
- Addressing comments from PR.
- This closes #359.
2016-04-15 16:03:00 -04:00
Andrew Lim acfc01213c
Update getting-started.adoc
Corrected button, menu item and icon inconsistencies/errors.  Fixed bulleted list formatting error in "Working with Templates" section.

This closes #329.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-04-12 17:08:15 -07:00
andrewmlim e66315c4db Update getting-started.adoc
Corrected spelling/grammatical errors

This closes #327.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-04-05 10:47:52 -07:00
Matt Gilman 2de7f3f884 Updating versions to 1.0.0-SNAPSHOT. 2016-04-04 11:36:20 -04:00
Aldrin Piri e977729b56 NIFI-1634-rc2 prepare for next development iteration 2016-03-23 18:56:34 -04:00
Aldrin Piri 0b9bd20d31 NIFI-1634-rc2 prepare release nifi-0.6.0-RC2 2016-03-23 18:56:22 -04:00
Aldrin Piri b44b177039 NIFI-1605 Adjust documentation and resources to reflect nifi.provenance.repository.rollover.time default
This closes #263

Signed-off-by: Matt Burgess <mattyb149@apache.org>
2016-03-17 22:14:24 -04:00
Andy LoPresto 76f2d5702f NIFI-1274 Added Kerberos authentication mechanism.
NIFI-1274 Cleaned up TODO statements. (+3 squashed commits)
Squashed commits:
[fd101cd] Removed logic to check for presence of services to determine if token support is enabled when username/password authentication is enabled (Kerberos also requires tokens).
[c2ce29f] Reverted import changes to RulesResource.java.
[c269d72] Added Kerberos authentication mechanism.

Moved Kerberos service wiring from XML to Java to handle scenario where admin has not configured Kerberos (previously threw NullPointerException in FileSystemResource constructor). (+15 squashed commits)
Squashed commits:
[09fc694] Added Kerberos documentation to Admin Guide.
[ecfb864] Cleaned up unused logic.
[157efb3] Added logic to determine if client certificates are required for REST API (login, anonymous, and Kerberos service authentication all disabled).
Cleaned up KerberosService by moving logic to NiFiProperties.
[5438619] Added documentation for Kerberos login-identity-providers.xml.
[3332d9f] Added NiFi properties for Kerberos SSO.
[b14a557] Fixed canvas call to only attempt Kerberos login if JWT not present in local storage.
Added logic to handle ticket validation failure in AccessResource.
Changed wiring of Kerberos service beans to XML in nifi-web-security-context.xml for consistency.
[c31ae3d] Kerberos SPNEGO works without additional filter (new entry endpoint accepts Kerberos ticket in Authorization header and returns JWT so the rest of the application functions the same as LDAP).
[98460e7] Added check to only instantiate beans when Kerberos enabled to allow access control integration tests to pass.
[6ed0724] Renamed Kerberos discovery method to be explicit about service vs. credential login.
[ed67d2e] Removed temporary solution for Rules Resource access via Kerberos ticket.
[c8b2b01] Added temporary solution for Rules Resource access via Kerberos ticket.
[81ca80f] NIFI-1274 Added KerberosAuthenticationFilter to conduct SPNEGO authentication with local (client) Kerberos ticket.
Added properties and accessors for service principal and keytab location for NiFi app server.
Added KAF to NiFiWebApiSecurityConfiguration.
Added AlternateKerberosUserDetailsService to provide user lookup without dependency on extension bundle (nifi-kerberos-iaa-provider).
Added dependencies on spring-security-kerberos-core and -web modules to pom.xml.
[0605ba8] Added working configuration files to test/resources in kerberos module to document necessary config. This version requires the user to enter their Kerberos username (without realm) and password into the NiFi login screen and will authenticate them against the running KDC.
Also includes a sample keystore and root CA public key for configuring a secure instance.
[49236c8] Added kerberos module dependencies to nifi/pom.xml and nifi-assembly/pom.xml.
Added default properties to login-identity-providers.xml.
[928c52b] Added nifi-kerberos-iaa-providers-bundle module to nifi/pom.xml.
Added skeleton of Kerberos authenticator using Spring Security Kerberos plugin.
This closes #284

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-03-17 13:10:09 -04:00
Mark Payne a7b97419e5 NIFI-1626: Throw an Exception proactively if too much state is attempting to be stored via ZooKeeperStateProvider
NIFI-1626: Updated State Management section of Developer Guide

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-03-16 16:12:00 -04:00
Oleg Zhurakousky 8e7e2916fa NIFI-1464 fixed admin guide 2016-03-11 12:54:50 -05:00
Oleg Zhurakousky 59fac58c96 NIFI-1464 ensured that OnUnscheduled is treated the same as OnScheduled
NIFI-1464 polished javadocs, error messages and docs
2016-03-11 12:54:50 -05:00
Oleg Zhurakousky 0c5b1c27f2 NIFI-1464, Refactored Processor's life-cycle operation sequence
* Simplified and cleaned StandardProcessScheduler.start/stopProcessor methods
* Added stop/start operations to ProcessorNode.
* Removed unnecessary synchronization blocks related to ScheduledState in favor of enforcing order and idempotency via CAS operations. Those synchronization blocks were causing intermittent deadlocks whenever @OnScheduled blocks indefinitely.
* Added support for stopping the service when @OnScheduled operation hangs.
* Fixed the order of life-cycle operation invocation ensuring that each operation can *only* be invoked at the appropriate time
* Removed unnecessary locks from StandardProcessNode since Atomic variables are used.
* Removed calls to @OnStopped from ContinuallyRunningProcessTask while ensuring that procesor's full shut down in implementation of StandardProcessorNode.stop() method.
* Removed dead code
* Added comprehensive tests suite that covers 95% of Processor's life-cycle operations within the scope of FlowController, StandardProcesssScheduler and StandardProcessNode
* Improved and added javadocs on covered operations with detailed explanations.
2016-03-11 12:54:50 -05:00
Richard Miskin 99c7fe3b44 NIFI-963 Update admin guide to cover configuration of multiple lib directories
This closes #250.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-27 10:01:38 -05:00
Matt Gilman bb2d2c3674 Clarifying embedded ZooKeeper on NCM is not possible. 2016-02-22 16:34:43 -05:00
Matt Gilman 8cff13e749 Minor changes to the state management section of the admin guide. 2016-02-22 15:21:56 -05:00
Tony Kurc 6c2ba997a0 Merge branch 'NIFI-1379-RC3' 2016-02-16 20:13:41 -05:00
Randy Gelhausen e4d0ec7e60 NIFI-1508: NiFi Site to Site doc tweak
This closes #215.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-16 15:17:28 -05:00
Tony Kurc ad73b5c9d9 NIFI-1379-RC3 prepare for next development iteration 2016-02-12 17:28:10 -05:00
Tony Kurc 8309dba80b NIFI-1379-RC3 prepare release nifi-0.5.0-RC3 2016-02-12 17:27:59 -05:00
Lars Francke d01449ee72 NIFI-1496: State Manager documentation link wrong. This closes #214
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-02-10 08:46:42 -05:00
Tony Kurc 5be83166ee NIFI-1485: Updated 'header' section of NiFi NOTICE files 2016-02-07 13:52:00 -05:00
Tony Kurc 303f8eabf1 NIFI-1379: Move to 0.5.0-SNAPSHOT, add tkurc code signing key to keys 2016-02-06 08:49:48 -05:00
Andy LoPresto b407379670 NIFI-1257 Resolved legacy compatibility issue with NiFi legacy KDF salt length dependent on cipher block size.
Replaced screenshot for NiFiLegacy salt encoding.
Added description of legacy salt length determination in admin guide.
Added logic for NiFiLegacyCipherProvider to generate and validate salts of the length determined by the cipher block size.
Changed EncryptContent to default to Bcrypt KDF.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-05 23:38:58 -05:00
Lars Francke 0d72969053 NIFI-1482 - DeveloperGuide :: Controller Service is using wrong method name
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-05 23:35:10 -05:00
Aldrin Piri 4df6512126 Merge branch 'NIFI-259' 2016-02-05 14:09:29 -05:00
Andy LoPresto 498b5023ce NIFI-1257 NIFI-1259
Added utility method to return the maximum acceptable password length for PBE ciphers on JVM with limited strength crypto because BC implementation is undocumented (based on empirical evidence).
Updated EncryptionMethod definitions to accurately reflect need for unlimited strength crypto according to algorithm key length.
Added processor logic to invoke keyed cipher.
Added EncryptContent processor property for raw hex key (always visible until NIFI-1121).
Added validations for KDF (keyed and PBE) and hex key.
Added utility method to return list of valid key lengths for algorithm.
Added description to allowable values for KDF and encryption method in EncryptContent processor.
Added IV read/write to KeyedCipherProvider and changed from interface to abstract class.
Added salt read/write logic to NifiLegacy and OpenSSL cipher providers.
Changed RandomIVPBECipherProvider from interface to abstract class.
Updated strong KDF implementations.
Renamed CipherFactory to CipherProviderFactory.
Added unit test for registered KDF resolution from factory.
Updated default iteration count for PBKDF2 cipher provider.
Implemented Scrypt cipher provider.
Added salt translator from mcrypt format to Java format.
Added unit tests for salt formatting and validation.
Added surefire block to groovy unit test profile to enforce 3072 MB heap for Scrypt test.
Added local Java implementation of Scrypt KDF (and underlying PBKDF2 KDF) from Will Glozer.
Defined interface for KeyedCipherProvider.
Implemented AES implementation for KeyedCipherProvider.
Added Ruby script to test/resources for external compatibility check.
Added key length check to PBKDF2 cipher provider.
Changed default PRF to SHA-512.
Added salt and key length check to PBKDF2 cipher provider.
Added utility method to check key length validity for cipher families.
Added Bcrypt implementation.
Implemented PBKDF2 cipher provider.
Added default constructor with strong choices for PBKDF2 cipher provider.
Implemented NiFiLegacyCipherProvider and added unit tests.
Added key length parameter to PBKDF2 cipher provider.
Added PRF resolution to PBKDF2 cipher provider.
Added RandomIVPBECipherProvider to allow for non-deterministic IVs.
Added new keyed encryption methods and added boolean field for compatibility with new KDFs.
Added CipherFactory.
Improved Javadoc in NiFi legacy cipher provider and OpenSSL cipher provider.
Added KeyedCipherProvider interface.
Added OpenSSL PKCS#5 v1.5 EVP_BytesToKey cipher provider and unit test.

This closes #201.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-04 19:40:55 -05:00
Matt Gilman adfa5dc0eb NIFI-259: - Moving the state management documentation to after the Clustering configuration.
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-03 10:13:23 -05:00
Matt Gilman 7711106d62 NIFI-259: - Adding additional documentation for embedded ZK.
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-03 10:13:19 -05:00
Aldrin Piri 8a668fd344 NIFI-259: Fixed bug that caused StateProvider.replace to return true if the value had never been set, instead of false. Fixed typos in administration-guide
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-02 23:03:31 -05:00
Matt Gilman 72c8467b9f NIFI-259:
- Addressing typo in documentation.
- Minor tweaks to admin guide.
- Adding support to stand up a ZooKeeperServer when a quorum peer is not distributed (ie supporting both embedded standalone and cluster).

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-01 14:07:38 -05:00
Aldrin Piri 6902812678 NIFI-259: Adjusting property description of the connect string to be more specific concerning socket tuple format and the associated default port if one is not specified. Providing minor cleanup and removing unused import. 2016-02-01 10:38:45 -05:00
Matt Gilman 4e50263581 NIFI-1445:
- Updating supported browsers sections to accurately reflect the capabilities the UI employs and the dependencies the UI uses.
2016-01-28 15:04:48 -05:00
Mark Payne 8a9a44c102 NIFI-1078: Fixed typo in docs and added additional examples 2016-01-28 09:35:05 -05:00
jpercivall 23364f554c NIFI-1078 Fixing the 'now' EL documentation
Signed-off-by: Mark Payne <markap14@hotmail.com>
2016-01-28 08:56:31 -05:00
Mark Payne 4d88aaedc5 NIFI-1258: Added a new function named getDelimitedField to the Expression Language and put together a guide that walks through how to add a new function
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-01-21 22:09:25 -08:00
Mark Payne f6ec437bc7 Merge branch 'master' into NIFI-259 2016-01-21 09:32:02 -05:00
Matt Gilman 0d7edcb3ac NIFI-108:
- Removing sort from UI.
- Addressing issues with listing and flowfile retrieval when clustered.
- Making the context menu item available when source and destination are still running.
- Adding a refresh button to the queue listing table.
- Fixing the flowfile summary sorting in the cluster manager.
- Adding a message when the source or destination of a connection is actively running.
- Updating the documentation regarding queue interaction.
- Updating the error message when a flowfile is no longer in the active queue.
- Updated queue listing to allow listing to be done while source and destination are running but not sort or have ability to search
- Added heartbeat when we finish clearing queue
- Addressing comments from review.
2016-01-21 08:13:32 -05:00