Commit Graph

969 Commits

Author SHA1 Message Date
Mark Payne fb7b3fe4b8 NIFI-1678: Started refactoring heartbeating mechanism, using a new package: org.apache.nifi.cluster.coordination
Added configuration options for ZooKeeper username & password for heartbeat management. Also addressed issue where nodes that were previously disconnected were asked to disconnect upon restart

Ensure that ACL is set properly when creating heartbeat node. Removed unused ControllerStartupFailureMessage.java

Changed ZooKeeper ACL's so that container nodes that would not be sensitive are wide open and removed the usage of username & password when communicating with ZooKeeper. This was done specifically because username/password combination is considered a 'testing' feature that should not be used in production and is not supported by Apache Curator

Refactored CuratorHeartbeatMonitor into an abstract heartbeat monitor that is responsible for processing heartbeats and CuratorHeartbeatMonitor that is responsible for retrieving heartbeat information

Refactored so that heartbeats are sent to Cluster Coordinator directly instead of to ZooKeeper. ZooKeeper is used to know which node is the cluster coordinator but heartbeats to the Cluster Coordinator provide additional information about the nodes.

Code cleanup and incorporate comments from peer review

This closes #323
2016-04-22 15:01:04 -04:00
Andy LoPresto 9ea2275677
NIFI-1802 Separated /access REST API endpoints into separate section in API docs.
Fixed typo in DocGenerator comments.

This closes #376.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-04-21 21:18:14 -07:00
Matt Gilman add298168d NIFI-1554:
- Introducing new REST endpoints to align with the authorizable resources.
- Additionally changes to support the new endpoints.
- Addressing comments in PR.
- This closes #374.
2016-04-21 17:29:58 -04:00
Mark Payne dd8c26e35c NIFI-1790: Updated the URI patterns used in NCM for determining whether or not responses should be merged. This closes #369 2016-04-20 10:07:18 -04:00
Matt Gilman f54e5d4b1b NIFI-1551: - Ensuring the datum is included when getting the entity key in nf-actions.
This closes #364.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-04-19 10:23:10 -04:00
Matt Gilman 8c09a5c8d2 NIFI-1783: - Addressing mistake in exmample XML. - Ensuring the configured Authorizer is loaded correctly.
This closes #363.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-04-19 09:53:26 -04:00
Pierre Villard 5e55a543eb NIFI-1724 Added properties to configure log level when file not found and permission denied on FetchFile processor
NIFI-1724 Added unit test for logging with level
This closes #348
2016-04-18 08:15:27 -04:00
Pierre Villard f719cbf60c NIFI-1777 Prevent deleting a connection going to a running processor
NIFI-1777 Added unit tests to test processor removal
This closes #357
2016-04-18 07:55:59 -04:00
Scott Aslan 10986553aa [NIFI-1761] UI - Remove reminants of cluster-indicator. This closes #356 2016-04-15 16:12:51 -04:00
Matt Gilman 153f63ef43 NIFI-1551:
- Removing the AuthorityProvider.
- Refactoring REST API in preparation for introduction of the Authorizer.
- Updating UI accordingly.
- Removing unneeded properties from nifi.properties.
- Addressing comments from PR.
- This closes #359.
2016-04-15 16:03:00 -04:00
Scott Aslan 7db78e87a5 [NIFI-1761] UI - Introduce AngularJS application bootstrapping, Roboto fonts, and breadcrumbs directive. This closes #331 2016-04-15 09:46:06 -04:00
Mark Payne b59d3ece75 NIFI-1762: Changed Java dependency to 1.8 instead of 1.7 and refactored StandardNiFiServiceFacade to make use of Lambda expressions to simplify code base. Also had to address a unit test because changing to Java 8 results in calls to assertEquals to become ambiguous. This closes #352 2016-04-14 10:38:03 -04:00
Andy LoPresto 378ccf53c2
NIFI-1753 Replaced usage of javax.security.cert.X509Certificate with java.security.cert.X509Certificate and resolved user-reported ClassCastException when handling client certificates during TLS mutual authentication.
Fixed nifi-utils pom.xml comment about additional dependencies. (+5 squashed commits)
Squashed commits:
[965b766] NIFI-1753 Removed temporary work-around of duplicate certificate conversion util method and added nifi-security-utils as dependency of nifi-utils.
[cd35f9b] NIFI-1753 Replaced legacy X.509 certificate declarations with new declarations in SSLSocketChannel and EndpointConnectionPool.
Temporary work-around of duplicate certificate conversion util method because nifi-utils cannot depend on nifi-security-utils.
[6420897] NIFI-1753 Replaced legacy X.509 certificate declarations with new declarations in PostHTTP.
[b9868ef] NIFI-1753 Added convenience method for extracting DN from peer certificate chain in SSL socket (canonical implementation to reduce code duplication and references to legacy certificate implementations).
Refactored logic retrieving legacy X.509 certificates with reference to convenience method in NodeProtocolSenderImpl.
Replaced logic retrieving legacy X.509 certificates with reference to convenience method in SocketProtocolListener.
Cleaned up exception handling in SocketProtocolListener.
Replaced legacy X.509 certificate declarations with new declarations in HandleHttpRequest (needs manual test).
[e2d1c35] NIFI-1753 Added convenience methods for converting legacy X.509 certificates and abstract certificates to correct X.509 format.
Added unit tests for certificate manipulation.
Replaced logic retrieving legacy X.509 certificates with new logic in NodeProtocolSenderImpl.
Added bcpkix (Bouncy Castle PKI implementation) dependency to nifi-standard-processors pom.

This closes #346.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-04-13 18:30:21 -07:00
Mike Moser 5625686ea4 NIFI-1746 avoid exceptions if configuration dir is a symbolic link
Signed-off-by:  Joe Skora <jskora@apache.org>
2016-04-11 02:05:48 -04:00
Matt Gilman 5de40ccec3 NIFI-1553:
- Implementing a file based authorizer.
- Providing an example authorizations files.
- Address comments from PR.
- This closes #330
2016-04-07 16:28:42 -04:00
Matt Gilman 3f4ac3156c Revert "NIFI-1551:"
This reverts commit c4d06f203d. Accidently merged wrong branch.
2016-04-07 16:22:35 -04:00
Matt Gilman c4d06f203d NIFI-1551:
- Starting to remove the AuthorityProvider.
- This closes #330
2016-04-07 16:18:36 -04:00
Mark Payne 84b1c60d54 NIFI-1726: Addressed issue where we can run into an infinite loop if we are expiring data based on a timestamp instead of disk space usage and we have a file whose timestamp is exactly equal to our threshold for deletion
Signed-off-by: joewitt <joewitt@apache.org>
2016-04-06 11:27:53 -04:00
Matt Gilman 9aa69b242e NIFI-1552: - Introducing the Authorizer API and additional components necessary for discovery and creation of configured instances. - Minor refactoring of existing Authority Provider API code/configuration to avoid some xsd naming conflicts. These components will be removed in NIFI-1551. - Introducing a number of the resource definitions that the Authorizer will make access decisions on. This list is likely not finalized may see some changes in NIFI-1554. - Address comments from PR. - This closes #318.
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-04-04 11:47:43 -04:00
Mark Payne 1ac05266a5 NIFI-483: Use ZooKeeper's Leader Election to determine Primary Node. This closes #301
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-04-04 11:47:08 -04:00
Mark Payne 0d3bd2c401 NIFI-1563: - Federate requests and merge responses from nodes instead of storing bulletins and stats at NCM - Updating UI to support restructured status history DTO. - Return 'Insufficient History' message if aggregate stats don't have enough data points, even if all nodes do (which can be the case if the node performing the aggregation has a different value for the 'nifi.components.status.snapshot.frequency' property than the other nodes) - Bug fixes; code cleanup; replicate requests to bulletin board endpoint - Refactored the <Component>StatusDTO objects into <Component>StatusDTO, <Component>StatusSnapshotDTO, Node<Component>StatusSnapshotDTO objects - Introducing endpoints for accessing individual component status. - Wiring up new endpoints and updated core. - Code clean up. - Starting to handling status merging of individual components. - Nodewise breakdown has been added to Processors but the remaining components still need to be updated. - Refactor so that System Diagnostics requests are replicated to nodes instead of the information being pulled from Heartbeats - Replicate request for counters instead of pulling them from heartbeats - Removed the getCounters / setCounters method from HeartbeatPayload - Implementing component specific endpoints. - Removing unused endpoints. - Supporting nodewise breakdown for system diagnostics and counters. - Updating DTOs to use more consistent naming. - Code clean up. - Addressing contrib issues. - Removed ProcessGroupStatus from HeartbeatPayload - Removing nodewise from the system diagnostics endpoint. Had included it for testing that option but did not intend for it to be committed. - Addressing comments in PR #294. - This closes #294
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-04-04 11:42:39 -04:00
Matt Gilman 2de7f3f884 Updating versions to 1.0.0-SNAPSHOT. 2016-04-04 11:36:20 -04:00
Matt Gilman 16108467c1 NIFI-1695:
- Removing hidden control characters that were affecting graph styles. This closes #311
2016-04-04 11:26:55 -04:00
Bryan Bende 65b5c1a5f1 NIFI-1697 Ensuring FlowController appropriately wraps code with NarCloseable. This closes #312 2016-04-04 09:22:55 -04:00
Mark Payne 8000304e6a NIFI-1442: This closes #306. Use CircularFifoQueue instead of Set to store nodes' bulletins
Joint effort by Toivo Adams from PR306 and and Mark Payne

Signed-off-by: joewitt <joewitt@apache.org>
2016-03-28 13:35:36 -04:00
Aldrin Piri e977729b56 NIFI-1634-rc2 prepare for next development iteration 2016-03-23 18:56:34 -04:00
Aldrin Piri 0b9bd20d31 NIFI-1634-rc2 prepare release nifi-0.6.0-RC2 2016-03-23 18:56:22 -04:00
Mark Payne b95a82f4a5 NIFI-1676: Do not allow Processor to be started if state is STOPPING; ensure that it is STOPPED 2016-03-23 11:31:20 -04:00
Mark Payne 38c782c30b NIFI-1650: Ensure that proper size and offset are specified for Content Claim when viewing FlowFiles still in queue 2016-03-21 10:30:24 -04:00
Mark Payne d3578a7c03 NIFI-1650: Ensure that we seek to the appropriate offset within the Content Claim when downloading content of a FlowFile 2016-03-21 10:30:23 -04:00
Aldrin Piri b44b177039 NIFI-1605 Adjust documentation and resources to reflect nifi.provenance.repository.rollover.time default
This closes #263

Signed-off-by: Matt Burgess <mattyb149@apache.org>
2016-03-17 22:14:24 -04:00
Oleg Zhurakousky 138c71de82 NIFI-1464 This closes #287. removed e.printStackTrace() from onTrigger in StandarProcessScheduler
Signed-off-by: joewitt <joewitt@apache.org>
2016-03-17 17:18:56 -04:00
Andy LoPresto 76f2d5702f NIFI-1274 Added Kerberos authentication mechanism.
NIFI-1274 Cleaned up TODO statements. (+3 squashed commits)
Squashed commits:
[fd101cd] Removed logic to check for presence of services to determine if token support is enabled when username/password authentication is enabled (Kerberos also requires tokens).
[c2ce29f] Reverted import changes to RulesResource.java.
[c269d72] Added Kerberos authentication mechanism.

Moved Kerberos service wiring from XML to Java to handle scenario where admin has not configured Kerberos (previously threw NullPointerException in FileSystemResource constructor). (+15 squashed commits)
Squashed commits:
[09fc694] Added Kerberos documentation to Admin Guide.
[ecfb864] Cleaned up unused logic.
[157efb3] Added logic to determine if client certificates are required for REST API (login, anonymous, and Kerberos service authentication all disabled).
Cleaned up KerberosService by moving logic to NiFiProperties.
[5438619] Added documentation for Kerberos login-identity-providers.xml.
[3332d9f] Added NiFi properties for Kerberos SSO.
[b14a557] Fixed canvas call to only attempt Kerberos login if JWT not present in local storage.
Added logic to handle ticket validation failure in AccessResource.
Changed wiring of Kerberos service beans to XML in nifi-web-security-context.xml for consistency.
[c31ae3d] Kerberos SPNEGO works without additional filter (new entry endpoint accepts Kerberos ticket in Authorization header and returns JWT so the rest of the application functions the same as LDAP).
[98460e7] Added check to only instantiate beans when Kerberos enabled to allow access control integration tests to pass.
[6ed0724] Renamed Kerberos discovery method to be explicit about service vs. credential login.
[ed67d2e] Removed temporary solution for Rules Resource access via Kerberos ticket.
[c8b2b01] Added temporary solution for Rules Resource access via Kerberos ticket.
[81ca80f] NIFI-1274 Added KerberosAuthenticationFilter to conduct SPNEGO authentication with local (client) Kerberos ticket.
Added properties and accessors for service principal and keytab location for NiFi app server.
Added KAF to NiFiWebApiSecurityConfiguration.
Added AlternateKerberosUserDetailsService to provide user lookup without dependency on extension bundle (nifi-kerberos-iaa-provider).
Added dependencies on spring-security-kerberos-core and -web modules to pom.xml.
[0605ba8] Added working configuration files to test/resources in kerberos module to document necessary config. This version requires the user to enter their Kerberos username (without realm) and password into the NiFi login screen and will authenticate them against the running KDC.
Also includes a sample keystore and root CA public key for configuring a secure instance.
[49236c8] Added kerberos module dependencies to nifi/pom.xml and nifi-assembly/pom.xml.
Added default properties to login-identity-providers.xml.
[928c52b] Added nifi-kerberos-iaa-providers-bundle module to nifi/pom.xml.
Added skeleton of Kerberos authenticator using Spring Security Kerberos plugin.
This closes #284

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-03-17 13:10:09 -04:00
Mark Payne a7b97419e5 NIFI-1626: Throw an Exception proactively if too much state is attempting to be stored via ZooKeeperStateProvider
NIFI-1626: Updated State Management section of Developer Guide

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-03-16 16:12:00 -04:00
Oleg Zhurakousky e83429a08c NIFI-1631 This closes #283. increased timeouts on test
Signed-off-by: joewitt <joewitt@apache.org>
2016-03-16 11:06:21 -04:00
Oleg Zhurakousky cfeebfe7c7 NIFI-1464 fixed OnScheduled invocation to pass ProcessContext
NIFI-1464 removed unused imports

Signed-off-by: joewitt <joewitt@apache.org>
2016-03-15 12:54:13 -04:00
Mark Payne a4a7d53769 NIFI-1627: Do not call context.yield for funnels and local ports; use batch size of 100 instead of 10 for local ports and funnels
Signed-off-by: joewitt <joewitt@apache.org>
2016-03-15 12:54:03 -04:00
trkurc f22db218e3 NIFI-1481 fixed a couple bugs in bash script for cygwin and when JAVA_HOME not set
Reviewed by Aldrin Piri (aldrin@apache.org). This closes #277
2016-03-14 22:40:25 -04:00
puspendu.banerjee@gmail.com 76ce3f7520 NIFI-1481 support for dumping environment using nifi.sh env
Reviewed and amended (Amendments reviewed by Aldrin Piri (aldrin@apache.org)) by Tony Kurc (tkurc@apache.org). This closes #218
2016-03-14 19:24:21 -04:00
Oleg Zhurakousky c7df94e00f NIFI-1464 life-cycle refactoring part-2
Signed-off-by: joewitt <joewitt@apache.org>
2016-03-14 17:13:55 -04:00
Mark Payne 5a8b2cf7f1 NIFI-1606: Run the onComponentRemoved logic of state providers in a background thread
Signed-off-by: joewitt <joewitt@apache.org>
2016-03-13 14:08:24 -04:00
Mark Payne ce068ffc6c NIFI-1612: Do not ignore Remote Process Group's timeout configuration
Signed-off-by: joewitt <joewitt@apache.org>
2016-03-13 13:40:45 -04:00
Mark Payne 7400b6f7c5 NIFI-1622: Ensure that the Nar Context Class Loader is used when calling Processor lifecycle methods
Signed-off-by: joewitt <joewitt@apache.org>
2016-03-12 20:06:23 -05:00
Oleg Zhurakousky 59fac58c96 NIFI-1464 ensured that OnUnscheduled is treated the same as OnScheduled
NIFI-1464 polished javadocs, error messages and docs
2016-03-11 12:54:50 -05:00
Oleg Zhurakousky 1c22f3f012 NIFI-1464 refactored the latest commit 2016-03-11 12:54:50 -05:00
Oleg Zhurakousky 48af0bfbc5 NIFI-1464 addressed latest PR comments
NIFI-1464 polishing
2016-03-11 12:54:50 -05:00
Oleg Zhurakousky f53f45def3 NIFI-1464 addressed PR comments from @apiri and @markap14 2016-03-11 12:54:50 -05:00
Oleg Zhurakousky 0c5b1c27f2 NIFI-1464, Refactored Processor's life-cycle operation sequence
* Simplified and cleaned StandardProcessScheduler.start/stopProcessor methods
* Added stop/start operations to ProcessorNode.
* Removed unnecessary synchronization blocks related to ScheduledState in favor of enforcing order and idempotency via CAS operations. Those synchronization blocks were causing intermittent deadlocks whenever @OnScheduled blocks indefinitely.
* Added support for stopping the service when @OnScheduled operation hangs.
* Fixed the order of life-cycle operation invocation ensuring that each operation can *only* be invoked at the appropriate time
* Removed unnecessary locks from StandardProcessNode since Atomic variables are used.
* Removed calls to @OnStopped from ContinuallyRunningProcessTask while ensuring that procesor's full shut down in implementation of StandardProcessorNode.stop() method.
* Removed dead code
* Added comprehensive tests suite that covers 95% of Processor's life-cycle operations within the scope of FlowController, StandardProcesssScheduler and StandardProcessNode
* Improved and added javadocs on covered operations with detailed explanations.
2016-03-11 12:54:50 -05:00
Pierre Villard 8f0116544a NIFI-1491 Deprecated existing method and added new one throwing eception
Reviewed and amended (to add javadoc @deprecation link and to carry forward other deprecated methods) by Tony Kurc (tkurc@apache.org). This closes #227
2016-03-10 22:15:13 -05:00
Mark Payne d3367a7dc3 nNFI-1618: Updated unit test to use 30,000 FlowFiles to test dropping of swapped out FlowFiles intead of using 210,000. Thiss unit test creates a lot of objects and when reducing the number of FlowFiles generated to 30K instead of 210K, all appears to work okay 2016-03-10 16:01:29 -05:00
Pasqualino Ferrentino 73e168e954 NIFI-1003 A relationship can be auto-terminable. In this case the processor will auto-terminate the relationship and allow the user to run it even if he does not connect those relationships and does not terminate them.
This closes #217

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-03-10 14:56:46 -05:00
Mark Payne 2839a2f215 NIFI-15: Address issue where incomplete swap files can result in continually attempting to swap in data without ever being successful
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-03-09 23:46:53 -05:00
Mark Payne a2164136db Merge branch 'NIFI-1573' of https://github.com/rpmiskin/nifi into NIFI-1573 2016-03-09 09:40:37 -05:00
Mark Payne 9874d35b6b NIFI-1600: Ensure that if we move a RemoteProcessGroup, it also updates the Parent Process Group of the ports 2016-03-08 12:09:45 -05:00
Oleg Zhurakousky 4ce7b679e1 NIFI-1595 fixed ReflectionUtils to honor bridge methods
Refactored and simplified ReflectionUtils while at it
Added ReflectionUtilsTest

This closes #260.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
With adjustments to formatting and whitespace.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-03-07 15:27:35 -05:00
Mark Payne 6776060ac8 NIFI-1577: Close any streams that are left open for 'append' when the session is checkpointed, rather than waiting for it to be committed 2016-02-29 16:02:26 -05:00
Richard Miskin 738236a41d NIFI-1573 Allow programmatic access to a Processor's name 2016-02-27 15:18:22 +00:00
Tony Kurc c7e24c7569 NIFI-1513: fixed some easy to fix errors
Addressing checkstyle issue.

This closes #221

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-02-25 15:21:40 -05:00
Matt Gilman 0d13de0cf3 NIFI-1539: - Comparing octet stream content type by using starts with and ignores case. 2016-02-25 10:13:07 -05:00
Sönke Liebau fc92441981 NIFI-1539 - Add normalization of content type for content viewing
Add code to ContentViewerController to strip content type of any trailing parameters and lowercase the type and subtype.

Added function to ViewableContent to enable retrieving the original value of the content type if needed.

This closes #242

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-02-25 10:12:45 -05:00
Mark Payne 6af108c0ca NIFI-1557: Ensure that Reporting Tasks & controller services are ordered by ID before fingerprinting 2016-02-23 18:02:58 -05:00
James Wing 24a77755de NIFI-1548 Fixing Controller Service Usage Button. This closes #245
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-02-22 21:09:01 -05:00
Matt Gilman e7676ffae5 NIFI-1546: - Ensuring that the data reference query is not null before attempt to extract a cluster node id from it.
Signed-off-by: joewitt <joewitt@apache.org>
2016-02-22 12:39:37 -05:00
Mark Payne 122cba0ee7 NIFI-1527: Fixed issue that resulted in resource claims' claimant count getting incremented twice on restart
Signed-off-by: joewitt <joewitt@apache.org>
2016-02-22 12:36:08 -05:00
Matt Gilman a8edab2e79 NIFI-1497: - Introducing a one time use password service for use in query parameters when accessing UI extensions and downloading resources. - Using one time use tokens when accessing ui extensions and downloading resources. - Ensuring appropriate roles when accessing component details through the web context for custom UIs. - Addressing typo in class name. - Ensuring appropriate roles when accessing content through the content access. - Code clean up. - Refactoring some basic scripts for accessing JWT tokens so UI extensions can reuse common functionality.
Signed-off-by: Bryan Bende <bbende@apache.org>
2016-02-19 10:54:53 -05:00
joewitt 86ab4428f0 NIFI-1520 by default skip javadoc and source jar generation in nars and wars
Reviewed by Tony Kurc (tkurc@apache.org). This closes #234
2016-02-18 18:45:26 -05:00
Mark Payne 3bb18b9653 NIFI-1527: Ensure that we increment Claimant Counts for content claims that are referenced by Swapped-Out FlowFiles on restart of nifi 2016-02-18 16:25:57 -05:00
Tony Kurc 6c2ba997a0 Merge branch 'NIFI-1379-RC3' 2016-02-16 20:13:41 -05:00
Pierre Villard ef0018cf66 NIFI-1282 This closes #228. Fixed error message when attempting to start a disabled port
Fixed error message when attempting to start a disabled port

Signed-off-by: joewitt <joewitt@apache.org>
2016-02-16 09:38:56 -05:00
Tony Kurc ad73b5c9d9 NIFI-1379-RC3 prepare for next development iteration 2016-02-12 17:28:10 -05:00
Tony Kurc 8309dba80b NIFI-1379-RC3 prepare release nifi-0.5.0-RC3 2016-02-12 17:27:59 -05:00
joewitt f44eb643dd NIFI-1461 added run.as filtered property for RPM and removed extraneous properties 2016-02-10 21:21:19 -05:00
Matt Gilman 778229eb52 NIFI-1499: - Ensuring the universal keystroke capture is available to UI extensions.
Signed-off-by: joewitt <joewitt@apache.org>
2016-02-10 19:15:48 -05:00
Matt Gilman f4487dd5f6 NIFI-1492: - Limiting the amount of state entries returned to a client. - Code clean up.
Signed-off-by: joewitt <joewitt@apache.org>
2016-02-09 20:48:56 -05:00
Aldrin Piri 1c03fc7871 Removing unused import to remedy checkstyle error on previous documentation update. 2016-02-09 16:13:16 -05:00
Aldrin Piri a33289393d Correcting documentation on SnippetResource#createSnippet to reflect that it is creating a snippet. 2016-02-09 15:33:46 -05:00
Tony Kurc 5be83166ee NIFI-1485: Updated 'header' section of NiFi NOTICE files 2016-02-07 13:52:00 -05:00
Tony Kurc 303f8eabf1 NIFI-1379: Move to 0.5.0-SNAPSHOT, add tkurc code signing key to keys 2016-02-06 08:49:48 -05:00
Matt Gilman 7314af6177 NIFI-259: - Addressing issues that arose from a merge conflict.
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-05 17:47:56 -05:00
Aldrin Piri 4df6512126 Merge branch 'NIFI-259' 2016-02-05 14:09:29 -05:00
Andy LoPresto 498b5023ce NIFI-1257 NIFI-1259
Added utility method to return the maximum acceptable password length for PBE ciphers on JVM with limited strength crypto because BC implementation is undocumented (based on empirical evidence).
Updated EncryptionMethod definitions to accurately reflect need for unlimited strength crypto according to algorithm key length.
Added processor logic to invoke keyed cipher.
Added EncryptContent processor property for raw hex key (always visible until NIFI-1121).
Added validations for KDF (keyed and PBE) and hex key.
Added utility method to return list of valid key lengths for algorithm.
Added description to allowable values for KDF and encryption method in EncryptContent processor.
Added IV read/write to KeyedCipherProvider and changed from interface to abstract class.
Added salt read/write logic to NifiLegacy and OpenSSL cipher providers.
Changed RandomIVPBECipherProvider from interface to abstract class.
Updated strong KDF implementations.
Renamed CipherFactory to CipherProviderFactory.
Added unit test for registered KDF resolution from factory.
Updated default iteration count for PBKDF2 cipher provider.
Implemented Scrypt cipher provider.
Added salt translator from mcrypt format to Java format.
Added unit tests for salt formatting and validation.
Added surefire block to groovy unit test profile to enforce 3072 MB heap for Scrypt test.
Added local Java implementation of Scrypt KDF (and underlying PBKDF2 KDF) from Will Glozer.
Defined interface for KeyedCipherProvider.
Implemented AES implementation for KeyedCipherProvider.
Added Ruby script to test/resources for external compatibility check.
Added key length check to PBKDF2 cipher provider.
Changed default PRF to SHA-512.
Added salt and key length check to PBKDF2 cipher provider.
Added utility method to check key length validity for cipher families.
Added Bcrypt implementation.
Implemented PBKDF2 cipher provider.
Added default constructor with strong choices for PBKDF2 cipher provider.
Implemented NiFiLegacyCipherProvider and added unit tests.
Added key length parameter to PBKDF2 cipher provider.
Added PRF resolution to PBKDF2 cipher provider.
Added RandomIVPBECipherProvider to allow for non-deterministic IVs.
Added new keyed encryption methods and added boolean field for compatibility with new KDFs.
Added CipherFactory.
Improved Javadoc in NiFi legacy cipher provider and OpenSSL cipher provider.
Added KeyedCipherProvider interface.
Added OpenSSL PKCS#5 v1.5 EVP_BytesToKey cipher provider and unit test.

This closes #201.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-04 19:40:55 -05:00
Matt Gilman 6f4c3db186 NIFI-259:
- Fixing the line height to ensure the multiline ellipsis is consistent across browsers.
2016-02-04 08:59:26 -05:00
puspendu.banerjee@gmail.com 0690aee452 NIFI-1460 - Performance improvement. test Timeout Mitigation. less IO, less dependency on /dev/(u)random.
- Incorporated comments for UUID format conformance & optimized.
          - polished to satisfy this test-case. we don't need incrementAndGet
Reviewed by Tony Kurc (tkurc@apache.org). This closes #202
2016-02-03 21:50:24 -05:00
Matt Gilman 7711106d62 NIFI-259: - Adding additional documentation for embedded ZK.
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-03 10:13:19 -05:00
Aldrin Piri 8a668fd344 NIFI-259: Fixed bug that caused StateProvider.replace to return true if the value had never been set, instead of false. Fixed typos in administration-guide
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-02 23:03:31 -05:00
Matt Gilman 8cf34c3ea5 NIFI-1457:
- Update heartbeat bean when primary state changes.
2016-02-02 13:16:10 -05:00
Mark Payne 35d2b921ea NIFI-259: Distinguish between unable to communicate with ZK and 'bad version' when performing 'replace' method of ZooKeeperStateProvider
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-01 16:29:12 -05:00
Aldrin Piri a931e72787 NIFI-259: Expose the ZK exception code in the event of a KeeperException within the ZooKeeperStateProvider. 2016-02-01 16:29:12 -05:00
Aldrin Piri 447e401912 NIFI-259: Extending the StateProvider interface to provide a getSupportedScopes method and implemented this based on the capabilities of each of its implementations. Used supported scope to evaluated configurations at startup and prevent issues when trying to make use of state 2016-02-01 16:29:12 -05:00
Aldrin Piri 257eca9c46 NIFI-259 Removing StateProviderScope and refactoring to use Scope in its place. 2016-02-01 16:29:11 -05:00
Aldrin Piri f2c366cf9c NIFI-259: Correcting path to the ZooKeeperStateProvider. 2016-02-01 15:26:27 -05:00
Matt Gilman 72c8467b9f NIFI-259:
- Addressing typo in documentation.
- Minor tweaks to admin guide.
- Adding support to stand up a ZooKeeperServer when a quorum peer is not distributed (ie supporting both embedded standalone and cluster).

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-01 14:07:38 -05:00
Aldrin Piri f47168213b NIFI-259: Adjusting validation logic for Connect String to catch errors at startup. 2016-02-01 14:07:38 -05:00
Aldrin Piri 6902812678 NIFI-259: Adjusting property description of the connect string to be more specific concerning socket tuple format and the associated default port if one is not specified. Providing minor cleanup and removing unused import. 2016-02-01 10:38:45 -05:00
Matt Gilman dbe8ff3f44 NIFI-1426:
- Ensuring we aren't preventing default when focused in a textarea.
2016-02-01 08:28:18 -05:00
Oleg Zhurakousky 807eb8de6d NIFI-1447 modified SimpleProcessLogger Modified SimpleProcessLogger to ensure that whenever is called it prints full stack trace.
Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-01-29 12:39:27 -05:00
Matt Gilman b3990ecdcf NIFI-1426: - Introducing a universal capture for key events to ensure a consistent behavior throughout the application. - Allowing backspace to remove components from the canvas. - Introducing a more consistent behavior around the escape button.
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-01-28 12:23:02 -06:00
Oleg Zhurakousky f70f7e3447 NIFI-1333 fixed FlowController shutdown deadlock. put read lock back. This closes #148
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-01-28 08:31:30 -05:00
Andy LoPresto 93aac8cff3 NIFI-1365
Added Groovy support for unit tests to pom with skeleton test.
Added Groovy unit tests for OCSPCertificateValidator.
Implemented positive & negative unit tests with cache injection for valid/revoked OCSP certificate.
Modified pom.xml to support Groovy unit tests with custom variable.

mvn clean test -Dgroovy=test

Added local cache injection into Groovy tests for OCSP certificate validation (see NIFI-1324 and NIFI-1364).
Set Java version to 1.7 for Groovy test src/target.
Moved Groovy unit test profile from nifi-web-security to root pom.
Added null check for algorithm argument in PGPUtil.
Changed buffer length check from ">= 0" to "> -1" because it was confusing other developers.
Resolved contrib-check line length issues.
Fixed contrib-check issues in OpenPGPKeyBasedEncryptorTest.
This closes #163

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-01-26 17:37:59 -05:00
Oleg Zhurakousky 0395b90315 NIFI-1200 fixed CPU saturation in FileSystemRepository
added minimal interval
added WARN message if set interval is lower

NIFI-1200 fixed unused imports
2016-01-26 16:37:19 -05:00
Mark Payne 329e1fe2f4 NIFI-259: Construct QuorumPeer object instead of ZooKeeperServer object because we want to join part of a cluster when using embedded ZK Server 2016-01-25 14:33:22 -05:00
Matt Gilman d71266502c NIFI-1435: - Addressing typo when initializing and showing the reporting task property table which prevented dynamic properties to be created. 2016-01-25 10:42:37 -05:00
Matt Gilman a7d3f8d75f NIFI-1428: - Adding a button to link from a flowfile in a queue listing to a provenance search for that flowfile.
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-01-22 18:26:52 -05:00
Matt Gilman b25db650fd NIFI-1400: - Addressing sort issues with the controller service and reporting task tables. Specifically addressing bulletins, type, and state.
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-01-22 16:41:14 -05:00
Matt Gilman 2f5f7b8302 NIFI-1429: - Only attempting to verify disabling of a referenced controller service when it's active.
Signed-off-by: Mark Payne <markap14@hotmail.com>
2016-01-22 16:22:40 -05:00
Matt Gilman 8392b46597 NIFI-1397: - Only resetting the node availability when closing the new controller service or reporting task dialog.
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-01-22 13:20:58 -05:00
Matt Gilman fcf837b5af NIFI-259:
- Restoring the call to isListFlowFilesEndpoint when deciding if a response is interrupted.
2016-01-21 16:35:59 -05:00
Mark Payne 3e13996512 Merge branch 'master' into NIFI-259 2016-01-21 15:02:00 -05:00
Mark Payne b07e13a1d8 NIFI-259: Bug fixes 2016-01-21 13:44:44 -05:00
Matt Gilman 53322c99ac NIFI-1383: - Ensuring appropriate access and state prior to attempting an action with hot keys. - Fixing contrib-check issue. 2016-01-21 12:39:45 -05:00
Matt Gilman 561f5b740a NIFI-1383: - Ensuring that nodes are not kicked out of the cluster when failing to successfully process a mutable request (like copy/paste). - Showing a more descriptive error message when possible. - Ensuring we don't try to instantiate an incomplete flow snippet. 2016-01-21 12:39:28 -05:00
Mark Payne 8f9c0b9ca3 Merge branch 'NIFI-259' of https://git-wip-us.apache.org/repos/asf/nifi into NIFI-259 2016-01-21 12:00:44 -05:00
Mark Payne 09357297e2 NIFI-259: Ensured that thread pools were being shutdown properly 2016-01-21 12:00:40 -05:00
Matt Gilman c18f75c597 NIFI-259:
- Ensuring component state is sorted accordingly when loaded in the table.
2016-01-21 11:25:17 -05:00
Mark Payne f6ec437bc7 Merge branch 'master' into NIFI-259 2016-01-21 09:32:02 -05:00
Matt Gilman 0d7edcb3ac NIFI-108:
- Removing sort from UI.
- Addressing issues with listing and flowfile retrieval when clustered.
- Making the context menu item available when source and destination are still running.
- Adding a refresh button to the queue listing table.
- Fixing the flowfile summary sorting in the cluster manager.
- Adding a message when the source or destination of a connection is actively running.
- Updating the documentation regarding queue interaction.
- Updating the error message when a flowfile is no longer in the active queue.
- Updated queue listing to allow listing to be done while source and destination are running but not sort or have ability to search
- Added heartbeat when we finish clearing queue
- Addressing comments from review.
2016-01-21 08:13:32 -05:00
Mark Payne d696391f76 NIFI-259: Update ZooKeeperStateServer so that the thread starting it doesn't block indefinitely, as this was causing NiFi to not shut down properly; applied patch for NIFI-1415 2016-01-20 11:52:41 -05:00
Mark Payne 593f1288d8 NIFI-259: Bug fixes, unit tests, documentation updates 2016-01-20 10:16:14 -05:00
Mark Payne 6af2d4a342 NIFI-259: Fixed NullPointerException 2016-01-19 14:01:45 -05:00
Mark Payne 16dcf4595c Merge branch 'master' into NIFI-259 2016-01-18 08:48:30 -05:00
Mark Payne 95a3115f9b NIFI-259: Bug Fix 2016-01-17 20:32:04 -05:00
Mark Payne eba25ecaca NIFI-259, NIFI-1339: Added OnConfigurationRestored annotation, always invoke onPropertyModified even on restart when properties are changed from defaults, as was done previously 2016-01-17 14:40:27 -05:00
Matt Gilman 5b62ff0fc3 NIFI-259:
- Only providing the View State menu item for DFMs.
2016-01-14 15:36:57 -05:00
Matt Gilman 4236125f2b NIFI-259:
- Fixing Consumes for clear state endpoints.
2016-01-14 15:35:53 -05:00
Matt Gilman ae6f615365 NIFI-259:
- Code clean up.
- Adding component state actions to the controller service and reporting task tables.
2016-01-14 13:09:39 -05:00
Matt Gilman 65dfcd06a3 NIFI-259:
- Fixing contrib check issues.
2016-01-14 11:16:35 -05:00
Mark Payne e943c41a47 NIFI-259: Minor bug fixes 2016-01-14 09:26:26 -05:00
Oleg Zhurakousky 92062f9beb NIFI-1317: removed duplicate 'name' instance variable
Reviewed by Tony Kurc (tkurc@apache.org). This closes #169
2016-01-14 00:43:09 -05:00
Matt Gilman 1a7e6c735d NIFI-259:
- Exsuring the component state css file is included in aggregated builds.
2016-01-13 17:20:11 -05:00
Matt Gilman f0d8f73f26 NIFI-259:
- Addressing some minor layout issues with the view state dialog.
- Ensuring appropriate locking when attempting to clear state.
2016-01-13 16:57:59 -05:00
Mark Payne c65829f0cb Merge branch 'NIFI-259' of https://git-wip-us.apache.org/repos/asf/nifi into NIFI-259 2016-01-13 15:12:44 -05:00
Matt Gilman 3f4bd919a9 NIFI-259:
- Merging responses when clustered to populate node details.
- Fixed bug when clearing processor state when clustered.
- Cleared the table after successfully clearing state.
2016-01-13 15:12:17 -05:00
Mark Payne 0cd6f80f36 NIFI-259: Bug fixes 2016-01-13 15:11:53 -05:00
Mark Payne 06f525b674 Merge branch 'NIFI-259' of https://git-wip-us.apache.org/repos/asf/nifi into NIFI-259 2016-01-13 13:57:57 -05:00
Mark Payne 0151b1eed1 NIFI-259: Fixed checkstyle violations 2016-01-13 13:57:47 -05:00
Matt Gilman d05314c54b NIFI-259:
- Initial implementation of viewing and clearing state for a processor.
2016-01-13 13:35:24 -05:00
Mark Payne bbce596d74 NIFI-259: Updated GetHBase to use new State Management; bug fixes; updated docs 2016-01-13 12:47:08 -05:00
Mark Payne 774c29a4da NIFI-259: Added Stateful annotation as described on ticket 2016-01-12 15:28:35 -05:00
Matt Gilman bbd35a0258 NIFI-259:
- Adding endpoints for getting and clearing component state.
2016-01-12 10:43:30 -05:00
Mark Payne e559c68766 NIFI-259: Began updating processors to use new state management 2016-01-12 10:21:47 -05:00
Andy LoPresto ffbfffce6d NIFI-1324:
Changed Maven dependencies for BouncyCastle bcprov and bcpg from jdk16:1.46 to jdk15on:1.53 (kept nifi-web-security on jdk16:1.46 because jdk15on:1.53 splits OCSP logic into new module bcpkix).
Added individual unit tests for PGP public keyring validation.
Passes all legacy unit tests.
Added TODOs for customizable brick encryption and refactoring shared code.
Cleaned up magic numbers to constants.
Added unit tests for OpenPGPPasswordBasedEncryptor (internal consistency and legacy file decrypt).
Began refactoring shared encrypt code from OpenPGP* implementations.
Extracted encrypt utility method from OpenPGPPasswordBasedEncryptor to PGPUtil class.
Added test resources (signed and unsigned key-encrypted files).
Added unit tests for OpenPGPKeyBasedEncryptor (internal consistency and external file decrypt).
Changed BC dependency for nifi-web-security to bcprov-jdk15on:1.53 and bcpkix-jdk15on:1.53.
Updated OCSPValidator to use new BC logic for OCSP validation. This code compiles but should be fully audited, as the legacy OCSP validation was not completely implemented.
Added skeleton of OCSP validator unit tests with successful keypair and certificate generation and signing code.
Added further unit tests for issued certificates.
Annotated unimplemented unit tests with note about Groovy integration.
Refactored Jersey call in OCSPCertificateValidator to internal method.
Added toString() to NiFi local OcspRequest.
Implemented positive & negative unit tests with cache injection for valid/revoked OCSP certificate.
Resolved contrib-check issues.
Removed ignored code in unit test.

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-01-12 09:22:51 -05:00
Mark Payne 7a3e3efce1 NIFI-259: Updated documentation, added digest username/password 2016-01-11 16:43:46 -05:00
Mark Payne d2a969e3d6 NIFI-259: Initial implementation of State Management feature 2016-01-11 16:38:52 -05:00
Mark Payne 57dadb7286 NIFI-259: Initial implementation of State Management feature 2016-01-11 08:28:32 -05:00
Aldrin Piri 4f7a4e83b1 NIFI-1311 Avoid using sudo for nifi.sh commands when run.as is the same as the user invoking the script.
Signed-off-by: joewitt <joewitt@apache.org>
2016-01-10 15:22:45 -05:00
Matt Gilman fdef994ead NIFI-1346: - Always showing input and output content for a provenance event.
Signed-off-by: joewitt <joewitt@apache.org>
2016-01-10 15:08:09 -05:00
Richard Miskin 8a8ae5508f NIFI-1313 Allow deletion of ProcessGroups that contain nested ProcessGroups
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-01-08 16:22:55 -05:00
Matt Gilman 6e9175459f NIFI-1363: - Using the appropriate namespace to call the function to detect if the content viewer is configured.
Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-01-07 17:15:44 -05:00
Aldrin Piri 47453d2c47 NIFI-1145 Providing delays in tests where resources are being initialized to allow heavily taxed environments sufficient time. 2016-01-07 12:05:23 -05:00
Aldrin Piri aca114926b NIFI-1145
Increasing timeout values for TestStandardProcessScheduler#validateEnabledDisableMultiThread
Increasing timeout for testConcurrencyWithEnablingReferencingServicesGraph as 10s is not sufficient for overly taxed environments.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-01-07 12:05:23 -05:00
Mark Payne fba006c8e3 NIFI-1145: Extend timeout of unit test from 20 seconds to 120 seconds
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-01-07 12:05:23 -05:00
Aldrin Piri 3d487a5381 NIFI-1354 Using the newly generated keystore and truststore throughout the code base and updating an associated test to utilize the specified properties. 2016-01-06 18:16:37 -05:00
Mark Payne b330fd1601 NIFI-108: Built out 'skeleton' of the requests so that web tier can be written against it
NIFI-108: Implementing ability to list FlowFiles in a queue

NIFI-108:
- Starting to add support for endpoints that will listing flowfiles in a queue.

NIFI-108: Added merging of response for listing of flowfiles in cluster manager

NIFI-108:
- Starting to add support for endpoints that will listing flowfiles in a queue.

NIFI-108:
- Starting to add support for endpoints that will listing flowfiles in a queue.

NIFI-108:
- Adding checkstyle issues.

NIFI-108: Add clusterNodeId to FlowFileSummaryDTO

NIFI-108: Added unit tests; added verifyCanList method to queue; fixed bugs

NIFI-108:
- Adding compilation error for IOException from getFlowFile().
- Code clean up.
- Javadocs.

NIFI-108:
- Verifying two phase commit for queue listing.
- Fixing checkstyle.
- Ensuring drop and listing requests are merged when created when clustered.

NIFI-108:
- Adding initial listing capabilities.
- Passing through the sort column and direction.

NIFI-108:
- Removing Delete FlowFile button.
- Ensuring sort flags are being passed correctly.
- Setting column widths.
- Also including the cluster node address in the flowfile summaries.

NIFI-108:
- Including queue size statistics in listing request.
- Showing connection name.

NIFI-108:
- Including queue size statistics in listing request.
- Ensuring verifyCanList runs when appropriate.

NIFI-108:
- Adding initial support for viewing flowfile details dialog.
- Adding initial support for click to content.

NIFI-108:
- Allowing the flowfile details dialog to be draggable.

NIFI-108:
- Only showing the flowfile listing table when the listing is successful and the listing is not empty.

NIFI-108:
- Reseting the queue stats when closing the listing table.

NIFI-108: Implemented sorting when performing listing of FlowFiles

NIFI-108: Fixed bug that caused the listFlowFiles operation to wait on a readLock before returning and performing work asynchronously; fixed bug in Write-Ahead FlowFile Repository that caused ContentClaims to be queued up for destruction instead of ResourceClaims - this caused millions of ContentClaims to be queued up instead of a single ResourceClaim in some tests

NIFI-108:
- Ensured the column sort indicator is reset when a new listing is opened.
- Removing unused import.

NIFI-108:
- Addressed issues found during the review.
2016-01-06 16:20:23 -05:00
Mark Payne 8e031c987b NIFI-1164: Fixed contrib-check issues, log message 2015-12-31 15:37:00 -05:00
Oleg Zhurakousky 602c4a9639 NIFI-1164 addressed latest PR comments 2015-12-28 16:26:00 -05:00
Oleg Zhurakousky 0d09054d9f NIFI-1164 addressed PR comment
Added isActive check to the StandardControllerServiceNode:280 to ensure that
the IF statement can only have a chance to succeed if service is active. The service
will be indiscriminately deactivated as soon as disable(..) operation is invoked. This itself will
eliminate the race condition discovered by Mark

NIFI-1164 addressed PR comments
fixed the race condition described by Mark during disable call

NIFI-1164 polished javadoc
2015-12-28 15:16:32 -05:00
Oleg Zhurakousky 909c0decd6 NIFI-1164 Fixed race condition and refactored
Changed ControllerServiceNode by adding enable(..), disable(..) and isActive() operations. See javadocs for more details in both ControllerServiceNode and StandardControllerServiceNode

Refactored service enable/disable logic in StandardProcessScheduler and StandardControllerServiceNode . Below are some of the notes:
- No need for resetting class loader since its going to derive from the class loader of the service. In other words any classes that aren’t loaded and will be loaded within the scope of the already loaded service will be loaded by the class lower of that service
- No need to control 'scheduleState.isScheduled()’ since the logic has changed to use CAS operation on state update and the service state change is now atomic.
- Removed Thread.sleep(..) and while(true) loop in favor of rescheduling re-tries achieving better thread utilization since the thread that would normally block in Thread.sleep(..) is now reused.
- Added tests and validated that the race condition no longer happening

Added additional logic that allows the initiation of the service disabling while it is in ENABLING state. See javadoc of StandardProcessScheduler.enable/disable for more details.

NIFI-1164 polishing
2015-12-28 09:22:23 -05:00
Oleg Zhurakousky ebcefaac23 NIFI-1289 reverted new method of NiFiProperties in favor of the localized reflection call in test to refresh properties.
Reviewed and Amended (added comments) by Tony Kurc (tkurc@apache.org). This closes #150
2015-12-26 09:58:08 -05:00
Oleg Zhurakousky 2845e93812 NIFI-1289 added support for refreshing properties - Added _getNewInstance()_ operation to NiFiProperties to ensure there is a way to refresh/reload NiFi properties - Fixed javadocs
Signed-off-by: Bryan Bende <bbende@apache.org>

This closes #142
2015-12-23 14:30:08 -05:00
joewitt f4ac8d75c5 NIFI-1312-RC1 prepare for next development iteration 2015-12-19 00:41:04 -05:00
joewitt d624ea4866 NIFI-1312-RC1 prepare release nifi-0.4.1-RC1 2015-12-19 00:40:53 -05:00
Mark Payne 0a58ab47b9 NIFI-1310: Do not yield context when encountering IOException and fixed bug that caused the penalization period for a node not to get set
Signed-off-by: joewitt <joewitt@apache.org>
2015-12-18 17:35:46 -05:00
Matt Gilman 43c7ecd221 NIFI-1298: - Addressing too many significant digits in the processor load average and [non]heap utilization.
Signed-off-by: joewitt <joewitt@apache.org>
2015-12-18 17:02:40 -05:00
Matt Gilman 0436383f49 NIFI-1286: - Removing provenance option from the Label context menu
Signed-off-by: joewitt <joewitt@apache.org>
2015-12-18 17:02:33 -05:00
Matt Gilman 608287f9fe NIFI-1309:
- Addressing issues around remote process groups automatically issuing new account requests.
- Ensuring authorization issues are updated with status refresh.
2015-12-18 15:57:38 -05:00
Mark Payne 1869b44682 NIFI-1285: Used a BufferedInputStream when swapping in data
Signed-off-by: joewitt <joewitt@apache.org>
2015-12-18 12:08:23 -05:00
Mark Payne 9ca0f95d06 NIFI-1272: Added unit test to verify behavior 2015-12-18 11:51:42 -05:00
ianwww fa4c5314ca nifi-1272 Fixing a bug in StandardProcessSession.get(int) which returns incorrect number of flowfiles
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-12-18 11:50:39 -05:00
Mike Moser f239be289e NIFI-1302 find dynamic libraries with standard names
Signed-off-by: joewitt <joewitt@apache.org>
2015-12-18 09:12:32 -05:00
Matt Gilman b19ff7cf37 NIFI-1215:
- Only showing the run duration setting when applicable.
- Showing the user a warning that a source processor with a non 0 run duration could lose data when NiFi is restarted.
2015-12-16 10:53:15 -05:00
Matt Gilman 51b8ecd01b NIFI-1185:
- Using banners from the NCM rather than replicating to a node.
2015-12-15 12:51:14 -05:00
Matt Gilman c75b5cfcea NIFI-1119: - Addressing race condition that caused the revision to be checked before the flow was loaded.
Signed-off-by: jpercivall <joepercivall@yahoo.com>
2015-12-15 10:05:44 -05:00
Matt Gilman 17be1c2d9f NIFI-1206: - Only enabling the enable/disable toolbar icon when appropriate.
Signed-off-by: Joseph Percivall <joepercivall@yahoo.com>
2015-12-14 18:09:11 -05:00
Matt Gilman f9f04439fd NIFI-1119:
- Also refreshing flow revision when the user clicks Refresh status.
2015-12-14 12:41:45 -05:00
joewitt a7b09a57c5 NIFI-1122 release vote passess. Merge branch 'NIFI-1122_nifi-0.4.0-RC2' 2015-12-11 17:14:26 -05:00
Matt Gilman 8070a9f1cc NIFI-1104:
- Using the appropriate attributes based on the content direction.
2015-12-11 10:59:41 -05:00
joewitt d755e43ec8 NIFI-1122_nifi-0.4.0-RC2prepare for next development iteration 2015-12-08 13:00:10 -05:00
joewitt b66c029090 NIFI-1122_nifi-0.4.0-RC2prepare release nifi-0.4.0-RC2 2015-12-08 12:59:59 -05:00
Mark Payne fb65cf1235 NIFI-1271: Yield funnels and ports for nifi.bored.yield.duration amount of time if backpressure is applied, as we do when there are no input FlowFiles. Adjusting logic for ContinuallyRunProcessorTask#call in determining if there is appropriate availability for processor relationships.
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2015-12-08 10:54:29 -05:00
Matt Gilman 08d0f3e596 NIFI-956: - Initialize the filter controls prior to the new processor table. Because of this, items were being incorrectly filtered on load. - Clean up in web.xml.
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2015-12-07 20:12:44 -05:00
Matt Gilman ee7eeb0493 NIFI-1268: - In some environments it appears as though the row selection event is triggering when there is no items in the table. Verifying the item was found prior to continuing.
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2015-12-07 19:50:37 -05:00
Matt Gilman 4c7894bccc NIFI-1260:
- Not pre-compiling message-page as it seems to be causign some runtime errors in some environments.
2015-12-07 12:20:18 -05:00
Mark Payne ee14d8f9dd Merge branch 'NIFI-1249' 2015-12-04 16:38:49 -05:00
Mark Payne f378ee9021 NIFI-1249: Allow Processors to add their own variables to those referencable by Expression Language. Make ReplaceText allow users to reference back-references of regex matches 2015-12-04 13:17:37 -05:00
joewitt 0f3a620154 NIFI-1250 removed erroneous ContentType reference and added docs for attributes to index
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2015-12-04 12:58:02 -05:00
Matt Gilman dae73c52a7 NIFI-1248:
- Initializing the actions file which ensures the progress bar is initialized.
2015-12-03 13:49:56 -08:00
Oleg Zhurakousky 663b7f2561 NIFI-1243 This closes #137. added null check for 'currentReadClaimStream' . . .before it is being closed
Signed-off-by: joewitt <joewitt@apache.org>
2015-12-02 16:23:48 -05:00
Tony Kurc 3a7ddc6a35 NIFI-1054: Fixed DOS line endings in xml, java and js source files
Signed-off-by: joewitt <joewitt@apache.org>
2015-12-01 22:49:51 -05:00
Matt Gilman 7726d069cd Merge branch 'NIFI-655' 2015-12-01 11:20:56 -05:00
Matt Gilman e22b51f3a7 NIFI-655:
- Renaming spring tokens to avoid confusion over authentication and authorization.
2015-12-01 10:08:36 -05:00
Matt Gilman 85eb8defdd NIFI-655:
- Changing default expiration time to 12 hours.
2015-12-01 09:36:33 -05:00
Matt Gilman c100052dac NIFI-655:
- Adding additional logging when proceeding as an anonymous user.
2015-12-01 08:51:45 -05:00
Matt Gilman 2b0819a5f2 NIFI-655:
- Removing unused imports.
2015-11-30 16:33:11 -05:00
Matt Gilman 014b2ac4e8 NIFI-655:
- Removing proxied user chain as user details are already serialized.
2015-11-30 16:30:12 -05:00
Matt Gilman 774d626f88 NIFI-655:
- Adding documentation around the behavior of the authentication filters.
- Only passing along necessary parameters.
2015-11-30 15:07:40 -05:00
Matt Gilman c722b56335 NIFI-655:
- Ensuring the access token is not replicated when the user is already authenticated/authorized.
2015-11-30 14:57:38 -05:00
Matt Gilman a84e505bcd NIFI-655:
- Ensuring the access token is not replicated when the user is already authenticated/authorized.
2015-11-30 14:47:30 -05:00
Matt Gilman 64beeef593 NIFI-655:
- Ensuring anonymous user label and login links are rendered when appropriate.
- Ensuring responses are accurate when making requests with a token when user log in is not supported.
2015-11-27 14:13:40 -05:00
Matt Gilman c1cc165edb NIFI-655:
- Fixing issue with filter bean initialization when clustered.
2015-11-27 10:05:58 -05:00
Mark Payne 9aa9c27dbe NIFI-1216: Check if log level is enabled immediately in the SimpleProcessLogger before formatting log message
Signed-off-by: joewitt <joewitt@apache.org>
2015-11-26 00:41:15 -05:00
Matt Gilman 6bce858e4a NIFI-655:
- Updated user guide with screenshots of login process.
- Tweaked wording in admin guide.
- Triggering login on enter press in login form.
2015-11-25 16:42:22 -05:00
Mark Payne 5061e5fa0a NIFI-1222: Session.adjustCounter keeps track of local and global counters; it then call processContext.adjustCounter with each of them, but ProcessContext was changed a while back to automatically increment both 'lcoal' and 'global' counters each time, so our numbers are doubled; removed the 'localCounters' and 'globalCounters' from StandardProcessSession and replaced with just 'counters'
Signed-off-by: joewitt <joewitt@apache.org>
2015-11-25 16:26:59 -05:00
Mark Payne a29b7b3bf0 NIFI-1203: Do not count looping connections when determining validity based on incoming connections
Signed-off-by: joewitt <joewitt@apache.org>
2015-11-25 15:32:41 -05:00
Matt Gilman 0435911186 NIFI-1198: - Updating the connection source and destination when appropriate (deletion and (re)connection).
Signed-off-by: joewitt <joewitt@apache.org>
2015-11-25 14:50:56 -05:00
Matt Gilman c073253366 NIFI-655:
- Update admin guide with documentation for username/password authentication.
- Setting default anonymous roles to none.
- Making account status messages to users more clear.
- Deleting user keys when an admin revokes/deletes an account.
- Updating authentication filter to error back whenever authentication fails.
2015-11-25 14:17:23 -05:00
Matt Gilman 1312bde498 NIFI-655:
- Updating available links during login, registration, and account status review.
2015-11-24 00:37:47 -05:00
Matt Gilman a5754986e2 NIFI-655:
- Fixing the configuration property name for Authentication Expiration in the provided example configuration.
2015-11-23 15:55:24 -05:00
Matt Gilman aaf14c45c9 NIFI-655:
- Refactoring web security to use Spring Security Java Configuration.
- Introducing security in Web UI in order to get JWT.

NIFI-655:
- Setting up the resources (js/css) for the login page.

NIFI-655:
- Adding support for configuring anonymous roles.
- Addressing checkstyle violations.

NIFI-655:
- Moving to token api to web-api.
- Creating an LoginProvider API for user/pass based authentication.
- Creating a module for funneling access to the authorized useres.

NIFI-655:
- Moving away from usage of DN to identity throughout the application (from the user db to the authorization provider).
- Updating the authorized users schema to support login users.
- Creating an extension point for authentication of users based on username/password.

NIFI-655:
- Creating an endpoint for returning the identity of the current user.
- Updating the LoginAuthenticationFilter.

NIFI-655:
- Moving NiFi registration to the login page.
- Running the authentication filters in a different order to ensure we can disambiguate each case.
- Starting to layout each case... Forbidden, Login, Create User, Create NiFi Account.

NIFI-655:
- Addressing checkstyle issues.

NIFI-655:
- Making nf-storage available in the login page.
- Requiring use of local storage.
- Ignoring security for GET requests when obtaining the login configuration.

NIFI-655:
- Adding a new endpoint to obtain the status of a user registration.
- Updated the login page loading to ensure all possible states work.

NIFI-655:
- Ensuring we know the necessary state before we attempt to render the login page.
- Building the proxy chain in the JWT authentication filter.
- Only rendering the login when appropriate.

NIFI-655:
- Starting to style the login page.
- Added simple 'login' support by identifying username/password. Issuing JWT token coming...
- Added logout support
- Rendering the username when appropriate.

NIFI-655:
- Extracting certificate validation into a utility class.
- Fixing checkstyle issues.
- Cleaning up the web security context.
- Removing proxy chain checking where possible.

NIFI-655:
- Starting to add support for registration.
- Creating registration form.

NIFI-655:
- Starting to implement the JWT service.
- Parsing JWT on client side in order to render who the user currently is when logged in.

NIFI-655:
- Allowing the user to link back to the log in page from the new account page.
- Renaming DN to identity where possible.

NIFI-655:
- Fixing checkstyle issues.

NIFI-655:
- Adding more/better support for logging out.

NIFI-655:
- Fixing checkstyle issues.

NIFI-655:
- Adding a few new exceptions for the login identity provider.

NIFI-655:
- Disabling log in by default initially.
- Restoring authorization service unit test.

NIFI-655:
- Fixing checkstyle issues.

NIFI-655:
- Updating packages for log in filters.
- Handling new registration exceptions.
- Code clean up.

NIFI-655:
- Removing registration support.
- Removing file based implementation.

NIFI-655:
- Removing file based implementation.

NIFI-655:
- Removing unused spring configuration files.

NIFI-655:
- Making the auto wiring more explicit.

NIFI-655:
- Removing unused dependencies.

NIFI-655:
- Removing unused filter.

NIFI-655:
- Updating the login API authenticate method to use a richer set of exceptions.
- UI code clean.

NIFI-655:
- Ensuring the login identity provider is able to switch context classloaders via the standard NAR mechanisms.

NIFI-655:
- Initial commit of the LDAP based identity providers.
- Fixed issue when attempting to log into a NiFi that does not support new account requests.

NIFI-655:
- Allowing the ldap provider to specify if client authentication is required/desired.

NIFI-655:
- Persisting keys to sign user tokens.
- Allowing the identity provider to specify the token expiration.
- Code clean up.

NIFI-655:
- Ensuring identities are unique in the key table.

NIFI-655:
- Adding support for specifying the user search base and user search filter in the active directory provider.

NIFI-655:
- Fixing checkstyle issues.

NIFI-655:
- Adding automatic client side token renewal.

NIFI-655:
- Ensuring the logout link is rendered when appropriate.

NIFI-655:
- Adding configuration options for referrals and connect/read timeouts

NIFI-655:
- Added an endpoint for access details including configuration, creating tokens, and checking status.
- Updated DTOs and client side to utilize new endpoints.

NIFI-655:
- Refactoring certificate extraction and validation.
- Refactoring how expiration is specified in the login identity providers.
- Adding unit tests for the access endpoints.
- Code clean up.

NIFI-655:
- Keeping token expiration between 1 minute and 12 hours.

NIFI-655:
- Using the user identity provided by the login identity provider.

NIFI-655: - Fixed typo in error message for unrecognized authentication strategy.

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>

NIFI-655. - Added logback-test.xml configuration resource for nifi-web-security.

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>

NIFI-655. - Added issuer field to LoginAuthenticationToken. - Updated AccessResource to pass identity provider class name when creating LoginAuthenticationTokens. - Began refactoring JWT logic from request parsing logic in JwtService. - Added unit tests for JWT logic.

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>

NIFI-655. - Changed issuer field to use FQ class name because some classes return an empty string for getSimpleName(). - Finished refactoring JWT logic from request parsing logic in JwtService. - Updated AccessResource and JwtAuthenticationFilter to call new JwtService methods decoupled from request header parsing. - Added extensive unit tests for JWT logic.

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>

NIFI-655:
- Refactoring key service to expose the key id.
- Handling client side expiration better.
- Removing specialized active directory provider and abstract ldap provider.

NIFI-655. - Updated JwtService and JwtServiceTest to use Key POJO instead of raw String key from KeyService.

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>

NIFI-655:
- Fixing typo when loading the ldap connect timeout.
- Providing a better experience for session expiration.
- Using ellipsis for lengthly user name.
- Adding an issuer to the authentication response so the LIP can specify the appropriate value.

NIFI-655:
- Showing a logging in notification during the log in process.

NIFI-655:
- Removing unnecessary class.

NIFI-655:
- Fixing checkstyle issues.
- Showing the progress spinner while submitting account justification.

NIFI-655:
- Removing deprecated authentication strategy.
- Renaming TLS to START_TLS.
- Allowing the protocol to be configured.

NIFI-655:
- Fixing issue detecting the presence of DN column

NIFI-655:
- Pre-populating the login-identity-providers.xml file with necessary properties and documentation.
- Renaming the Authentication Duration property name.

NIFI-655:
- Updating documentation for the failure response codes.

NIFI-655:
- Ensuring the user identity is not too long.

NIFI-655:
- Updating default authentication expiration to 12 hours.

NIFI-655:
- Remaining on the login form when there is any unsuccessful login attempt.
- Fixing checkstyle issues.
2015-11-23 14:50:13 -05:00
Aldrin Piri 08d59e4374 NIFI-1196 Providing handling of FETCH provenance events for their "unique" property, transit URI, within the framework and UI.
Reviewed by Tony Kurc (tkurc@apache.org)
2015-11-19 17:42:15 -05:00
Mark Payne 69bce2c2db NIFI-1168: Ensure that processors with only looping
connections are scheduled to run, even if the connections have no FlowFiles;
 expose these details to processor developers; update documentation

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2015-11-18 14:53:30 -05:00
Mark Payne d88b6cb6bc NIFI-1173: Even if FlowFile Queue is empty, it needs to hold onto the Empty Queue request so that subsequent cancel/clear requests can reference it
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2015-11-18 10:56:38 -05:00
Mark Payne 93be753301 NIFI-1181: Ensure that a FlowFile's uuid cannot be modified by processors
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2015-11-18 10:35:37 -05:00
Matt Gilman 64369f67f9 NIFI-1160:
- Addressing possible NPE when removing a drop request.
- Only updating a connection status once it's been retrieved for the first time.
2015-11-16 08:34:04 -05:00
Mark Payne da9f40b3f0 NIFI-973: Updated main Help page to provide a link to the Getting Started Guide.
Reviewed by Aldrin Piri (aldrin@apache.org)
2015-11-15 08:38:51 -05:00
Tony Kurc 528dab78d6 NIFI-1073: Fixing coverity discovered errors. Resource leaks, and statics
Reviewed by Bryan Bende (bbende@apache.org)
2015-11-14 18:56:43 -05:00
joewitt 36d00a60f5 NIFI-1155 fixed contrib-check violation 2015-11-13 00:06:12 -05:00
Mark Payne 37d6b7350e NIFI-1155: Refactored StandardFlowFileQueue to update member variables more intelligently, using CAS operations instead of locks. This reduces code complexities because other optimizations that previously existed are no longer needed
Signed-off-by: joewitt <joewitt@apache.org>
2015-11-13 00:06:12 -05:00
Mark Payne 3ed0949c55 NIFI-1155: Ensure that when poll(FlowFileFilter, Set) is called, we properly update the indicator for whether or not queue is full
Signed-off-by: joewitt <joewitt@apache.org>
2015-11-13 00:06:12 -05:00
Mark Payne 854d203982 Merge branch 'NIFI-1143' of https://github.com/olegz/nifi into NIFI-1143 2015-11-11 14:14:54 -05:00
Oleg Zhurakousky 5baafa156a NIFI-1143 Fixed race condition which caused intermittent failures
Fixed the order of service state check in PropertyDescriptor
Encapsulated the check into private method for readability
Modified and documented test to validate correct behavior.
For more details please see comment in https://issues.apache.org/jira/browse/NIFI-1143
2015-11-11 14:06:08 -05:00
Oleg Zhurakousky 76690a8ee9 NIFI-1061 fixed deadlock caused by DBCPConnectionPool.onConfigured()
Current implementation of DBCPConnectionPool was attempting to test if connection could be obtained via dataSource.getConnection().
Such call is naturally a blocking call and the duration of the block is dependent on driver implementation. Some drivers (e.g., Phoenix - https://phoenix.apache.org/installation.html)
attempts numerous retries before failing creating a deadlock when attempt was made to disable DBCPConnectionPool which was still being enabled.

This fix removes the connection test from DBCPConnectionPool.onConfigured() operation returning successfully upon creation of DataSource.
For more details see comments in https://issues.apache.org/jira/browse/NIFI-1061
2015-11-11 08:33:51 -05:00
Mark Payne e6644b3747 Merge branch 'NIFI-1141' 2015-11-10 17:17:55 -05:00
Mark Payne 72008035b1 Merge branch 'NIFI-1000' 2015-11-10 15:02:02 -05:00
Mark Payne 11768cc388 NIFI-1141: Fixed the order of the arguments to the setCurrentContentClaim method 2015-11-10 11:35:14 -05:00
Mike Moser b75af5b344 Merge branch 'NIFI-1085' 2015-11-09 16:13:12 -05:00
Mark Payne 73c1671975 NIFI-1132: Limited number of Lineage Identifiers held to 100 and marked the getLineageIdentifiers() method as deprecated 2015-11-09 12:09:56 -05:00
joewitt 99629646fe NIFI-1122 moved to 0.4.0-SNAPSHOT 2015-11-06 23:41:15 -05:00
Mark Payne 0d6e81b54f Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/nifi 2015-11-06 09:16:12 -05:00
Bryan Bende ee87f33ce7 Fixing check-style violation in StandardFlowFileQueue 2015-11-06 09:10:10 -05:00
Mark Payne 1a388f957e NIFI-1111: We were subtracting values in the wrong order so that we always would 'yield' for a negative amount of time, which equated to not yielding at all. 2015-11-06 09:03:38 -05:00
Mark Payne 98f5a1ab77 NIFI-1110: Fixed bug that caused queue size to become negative when FlowFiles are expired
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-11-05 12:12:02 -05:00
Andrew Grande f1bd3d471d NIFI-1112 Fixed APIOperation Annotation on ReportingTaskResource.
Reviewed by Tony Kurc (tkurc@apache.org)
2015-11-04 18:31:34 -05:00
Mike Moser 6add372bc1 NIFI-1085 WebClusterManager fair locking and HeartbeatMonitoringTimerTask scheduling 2015-11-04 12:19:00 -05:00
Mark Payne 2b1d093120 NIFI-1105: Only trigger a processor that requires input to run if data is available for it process 2015-11-04 08:41:10 -05:00
Mark Payne dbf0c7893f NIFI-730: Completed merging of master 2015-11-02 14:31:28 -05:00
Mark Payne e4cebba3c7 Merge branch 'master' into NIFI-730 2015-11-02 14:02:16 -05:00
Mark Payne abf2c62884 NIFI-1051: Fixed checkstyle violations 2015-11-02 11:01:50 -05:00
Mark Payne 7a73867c46 Merge branch 'NIFI-1051' of https://github.com/olegz/nifi into NIFI-1051 2015-11-02 10:51:43 -05:00
Matt Gilman 5a04021dd7 NIFI-730:
- Updating logic now that original is guaranteed to be non null.
- Always reporting 100% once the drop request has completed.
2015-11-02 08:59:11 -05:00
Mark Payne f5727cfb0f NIFI-730: Ensure that we always populate queue counts when initiating a Drop FlowFile request 2015-11-02 08:36:50 -05:00
Mark Payne ad849c77df NIFI-1090: Fixed log message that was at info level but should have been debug level 2015-11-01 14:37:01 -05:00
Mark Payne 9c148da1c2 NIFI-869: Fixed checkstyle violations 2015-10-30 16:09:50 -04:00
Mark Payne ba72452f66 Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/nifi 2015-10-30 16:02:15 -04:00
Oleg Zhurakousky a3d43d23dc NIFI-869 Fixed SimpleProcessLogger to log correct messages Ensured that SimpleProcessLogger correctly interprets Throwable as discussed in JIRA Added tests
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-10-30 15:52:33 -04:00
Oleg Zhurakousky 90aea01350 NIFI-869 Fixed formatting issue Fixed formatting issue with printed error message which only apears when NiFi is cnfigured using Logback. Please see NIFI-869 for more details
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-10-30 15:52:33 -04:00
Oleg Zhurakousky 5c4042bd7c NIFI-1051 Allowed FileSystemRepository to skip un-readable entries.
The exception was caused due to basic file permissions. This fix overrides
'visitFileFailed' method of SimpleFileVisitor to log WARN message and allow
FileSystemRepository to continue.
2015-10-29 16:31:17 -04:00
Bryan Bende 5cc2b04b91 NIFI-986 Refactoring of action classes from nifi-user-actions to have interfaces in nifi-api, and adding getFlowChanges to EventAccess
- Fixing empty java docs and adding sort by id asc to the history query
- Changing userDn to userIdentity in Action and FlowChangeAction
- Modifying NiFiAuditor to always save events locally, and implementing getFlowChanges for ClusteredEventAccess
2015-10-29 16:28:36 -04:00
Mark Payne edf238e004 NIFI-730: Do not require a Read Lock in order to obtain backpressure configuration values for FlowFileQueue's 2015-10-28 10:27:58 -04:00
Joseph Percivall b885f955f4 NIFI-516 adding option to StandardProcessSession.read to close stream
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-10-26 20:23:13 -04:00
Mark Payne fc2aa2764c NIFI-10: Added FETCH and DOWNLOAD Provenance Events; updated FlowController to use DOWNLOAD event instead of SEND whenever a user downloads/views content via Provenance Event 2015-10-26 14:58:50 -04:00
Mark Payne 51f564024a NIFI-10: Added FETCH Provenance Event and updated processors to use this new event type 2015-10-26 14:58:50 -04:00
Mark Payne aec32a277c NIFI-1070: Added detailed debug-level logging about how FileSystemRepository is choosing to expire archived data 2015-10-26 14:36:03 -04:00
Matt Gilman f8c3377c84 NIFI-820:
- Creating a context menu item for accessing provenance directly for a selected component.
2015-10-26 10:16:01 -04:00
Mark Payne 0636f0e731 NIFI-810: Merged master into branch 2015-10-25 11:02:40 -04:00
Alex Moundalexis b809031195 NIFI-1046 shell clean up: +variable braces for consistency. Reviewed by Tony Kurc (tkurc@apache.org). This closes #106 2015-10-19 20:36:58 -04:00
Alex Moundalexis 9aa716b059 NIFI-1046 shell clean up: quoting prevents globbing & word splitting.
Ignoring the unquoted array expansions on L188/190; suspect they ought to be quoted per [SC2068](https://github.com/koalaman/shellcheck/wiki/SC2068), but need to confirm desired arguments.

Reviewed by Tony Kurc (tkurc@apache.org)
2015-10-19 20:36:07 -04:00
Alex Moundalexis 58d9b02517 NIFI-1046 shell clean up: remove backticks from nifi.sh. Reviewed by Tony Kurc (tkurc@apache.org) 2015-10-19 20:33:52 -04:00
Venkatesh Sellappa 9a8d763d8d NIFI-612 Remove FlowUnmarshaller. Builds, runs, could not find any latent references using reflection or other ways of loading classes without direct references. This closes #103. Signed off by Tony Kurc <tkurc@apache.org> 2015-10-17 09:43:10 -04:00
Matt Gilman 570202eb30 NIFI-730:
- Fixing capitalization to remain consistent.
2015-10-16 10:47:02 -04:00
Matt Gilman a872403831 NIFI-730:
- Updating the styling of the drop request status dialog.
- Rendering any errors that may have occurred.
2015-10-16 09:37:57 -04:00
Mark Payne 2b4999c018 NIFI-730: do not interrupt swap thread if drop flowfiles is canceled 2015-10-16 09:06:13 -04:00
Mark Payne b8dbd1018c NIFI-730: Updated details of Provenance Event for when a queue is emptied 2015-10-16 08:53:40 -04:00
Matt Gilman 39a050d2fd NIFI-730:
- Adding emptying a queue when clustered.
2015-10-14 17:47:06 -04:00
Mark Payne 09a3f6dadd NIFI-730: reordered states for drop flowfile request 2015-10-14 16:32:39 -04:00
Mark Payne 98a04eec74 NIFI-730: Set current queue size on drop flowfile request immediately when creating request 2015-10-14 13:48:59 -04:00
Matt Gilman cad0e7cf0f NIFI-730:
- Adding progress bar for showing current status of drop request.
- Allowing the user to cancel the current drop request.
2015-10-14 12:39:19 -04:00
Mark Payne a2ae99f899 NIFI-730: Make cancel request actually cancel 2015-10-14 11:11:48 -04:00
Mark Payne 5867193bc1 NIFI-730: Added additional testing logic to unit tests 2015-10-14 09:56:52 -04:00
Mark Payne 0af1acaafa NIFI-730: Return DropFlowFileStatus object when calling cancel 2015-10-14 09:46:21 -04:00
Mark Payne 77f7d7524c NIFI-730: bug fixes and code cleanup for swap manager and flowfile queue 2015-10-14 09:14:15 -04:00
Mark Payne afb76afcd0 NIFI-730: Added error messages if we fail to drop FlowFiles from queue 2015-10-13 15:57:18 -04:00
Mark Payne 72ff2a25d5 NIFI-730: Updated queue sizes appropriately during dropping of flowfiles 2015-10-13 15:46:56 -04:00
Matt Gilman 4b41aaab02 NIFI-730:
- Fixing checkstyle violations.
- Wiring endpoints and framework model.
- Lowering the max delay while polling from 8 seconds to 4 seconds.
2015-10-13 13:43:10 -04:00
Mark Payne af78354d84 NIFI-730: Added additional parameters to dropFlowFiles 2015-10-13 12:20:18 -04:00
Matt Gilman 09d6fe5cdb NIFI-730:
- Fixing checkstyle violations.
- Renaming context menu item for emptying a queue.
2015-10-13 10:12:58 -04:00
Mark Payne 9be37914dd Merge branch 'NIFI-730' of https://git-wip-us.apache.org/repos/asf/nifi into NIFI-730 2015-10-13 10:03:31 -04:00
Mark Payne 49a781df2d NIFI-730: Implemented swapping in and out on-demand by the FlowFileQueue rather than in a background thread 2015-10-13 10:03:03 -04:00
Matt Gilman e0ac7cde37 NIFI-730:
- Starting to add support for deleting flow files from a queue by creating endpoints and starting to wire everything together.
- Adding context menu item for initiating the request to drop flow files.
2015-10-12 10:00:54 -04:00
Mark Payne b8c51dc35d NIFI-730: Added methods for dropping queued flowfiles; refactored swap manager but have not yet started swapping flowfiles in or out from within the flowfile queue 2015-10-11 10:27:07 -04:00
Mike Moser f798127290 NIFI-1031 resolve reporting task error in flow.xml
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2015-10-09 10:53:23 -04:00
Matt Gilman 2583d7869a NIFI-442:
NIFI-828:
- Always selecting the first item in the new component table.
- Enabling adding the selected component by typing Enter.
- Removing the 'filter by' in the new component dialogs and instead just searching every field.
2015-10-07 19:11:55 -04:00
Mark Payne 8e2308b78d NIFI-810: Reworded validation errors pertaining ot upstream connections 2015-10-07 17:53:04 -04:00
Mark Payne ccfb57fe9f NIFI-810: Addressed several checkstyle violations 2015-10-07 17:48:51 -04:00
Mark Payne b974445ddd NIFI-810: rebased from master 2015-10-07 17:28:39 -04:00
Mark Payne 5ecdb1858e NIFI-810: Updated the wording on validation errors due to upstream connections 2015-10-07 17:26:14 -04:00
Matt Gilman 13edcfda2e NIFI-810:
- Adding basic support for preventing connection when appropriate.
- Updating validation when [dis]connecting processors.
2015-10-07 17:26:14 -04:00
Mark Payne 4afd8f88f8 NIFI-810: Created RequiresInput annotation and ensure that processors are invalid if connections do not agree 2015-10-07 17:26:14 -04:00
Matt Gilman 2215bc848b NIFI-810:
- Adding basic support for preventing connection when appropriate.
- Updating validation when [dis]connecting processors.
2015-09-25 17:46:58 -04:00
Mark Payne 034ee6de6b NIFI-810: Created RequiresInput annotation and ensure that processors are invalid if connections do not agree 2015-09-25 11:39:58 -04:00
Bryan Bende 600f91a262 NIFI-932 Adding hasIncomingConnection() and hasConnection(Relationship) to ProcessContext and updating ExecuteSQL 2015-09-18 17:05:31 -04:00
Matt Gilman ded74ec94c NIFI-876 prepare for next development iteration 2015-09-14 21:48:11 -04:00
Matt Gilman 2ec735e350 NIFI-876 prepare release nifi-0.3.0-RC1 2015-09-14 21:48:00 -04:00
Mark Payne 3d4ce34529 NIFI-938: If ResourceClaim is removed while a process has access to its stream, don't delete the claim 2015-09-14 11:47:02 -04:00
Mark Payne 992e841027 NIFI-948: Provide capability to register notifications for nifi lifecycle events 2015-09-13 19:55:12 -04:00
Mark Payne 3780159d01 NIFI-939: Fixed bug in equals and hashCode methods of StandardContentClaim; added unit test to TestStandardProcessSession in order to verify behavior of append() method 2015-09-11 12:50:39 -04:00
Joseph Percivall a83ed34f91 Commit for NIFI-836 to fix broken javadoc @link elements
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2015-09-10 12:05:47 -04:00
Matt Gilman d1206fe074 NIFI-926:
- Ensuring that nodes are not disconnected when the user attempts to reset a counter that does not exist on that node. This can happen when/if counters are adjusted conditionally.
2015-09-07 08:41:47 -04:00
Mark Payne 2bb7853001 Merge branch 'NIFI-905' 2015-08-31 11:14:17 -04:00
Mark Payne 3159cec782 NIFI-905: Ensure that when archive threshold is hit, archived data is destroyed and if no archived data exists that Processors aren't blocked from updating content repo 2015-08-28 14:15:35 -04:00
Mark Payne 5de37f63d9 NIFI-902: Ensure that if we get an IOException during rollover of WAL, we are able to recover 2015-08-28 10:04:58 -04:00
Mark Payne fbec28bad9 NIFI-893: Ensure that if session.commit fails, previously 'checkpointed' sessions are rolled back 2015-08-27 15:17:55 -04:00
Bryan Bende 310347fd66 Merge branch 'NIFI-842' 2015-08-24 18:58:56 -04:00
ricky 7fb6e884a7 NIFI-866: Add Kerberos Support for Hadoop
- Add krb5.conf to nifi.properties

	nifi.kerberos.krb5.file | path to krb5.conf

- Connections to secure Hadoop clusters will be determined by their config,
  that is, hadoop.security.authentication should be set to kerberos.

- Added two optional arguments to AbstractHadoopProcessor (principal and keytab),
  these are only required if the cluster you're connecting to is secured. Both of
  these options require the krb5.conf to be present in nifi.properties.

Signed-off-by: Bryan Bende <bbende@apache.org>
2015-08-24 15:36:22 -04:00
Bryan Bende 3bfe830afc NIFI-842 Updating org.apache.nifi.NiFi so it sends a started signal to bootstrap which can then be used to make retsart decisions 2015-08-24 12:00:21 -04:00
Mark Payne d351f3ed1b NIFI-882: When a ProcessGroup is removed from the graph, ensure that the removeXYZ method is called for each component in that Process Group so that resources are appropriately cleaned up 2015-08-21 15:55:04 -04:00
Mark Payne 1ddd736558 NIFI-854: Ensure that controller services are included in template if any processor (or other service) in the template references it, regardless of how many levels deep the component's ProcessGroup is 2015-08-21 14:46:20 -04:00
Mark Payne 15a8699dc4 NIFI-744: Addressed feedback from review, mostly adding documentation to a few points in the code 2015-08-21 11:08:34 -04:00
Mark Payne 68d94cc01b NIFI-744: Refactored ContentClaim into ContentClaim and ResourceClaim and allowed resource claim to be reused across sessions 2015-08-21 11:08:34 -04:00
Mark Payne 8ebf1f03c2 Merge branch 'nifi-868' of https://github.com/pukkaone/nifi into NIFI-868 2015-08-19 12:56:26 -04:00
Mark Payne 19f7db6986 NIFI-793: Added multi-threading to the indexing in the Persistent Provenance Repository 2015-08-19 12:24:25 -04:00
Mark Payne f171756a88 NIFI-862: Ensure that if we are unable to stop a controller service, we can still stop processors referencing it and the service itself 2015-08-19 10:12:57 -04:00
Mark Payne e409b6c5c1 NIFI-839: When disabling controller services, do so one-at-a-time in the process scheduler, so that we can ensure that all are disable-able and then disable them atomically 2015-08-19 08:55:15 -04:00
Chin Huang 8f98f80938 NIFI-868 Configure advertised host name of Remote Process Group Input Port
Before this change, the host given out to clients to connect to a Remote
Process Group Input Port is the host where the NiFi instance runs.
However, sometimes the binding host is different from the host that
clients connect to. For example, when a NiFi instance runs inside a
Docker container, a client on a separate machine must connect to the
Docker host which forwards the connection to the container.

Add a configuration property to specify the host name to give out to
clients to connect to a Remote Process Group Input Port. If the property
is not configured, then give out the name of host running the NiFi
instance.
2015-08-18 21:35:28 -07:00
joewitt 8e5347156f NIFI-841 fixed contrib check issues
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-08-18 20:06:42 -04:00
joewitt c3731703f3 NIFI-841 correct patch paths and fixed formatting issues
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-08-18 20:06:42 -04:00
Mark Payne 35439db347 NIFI-841: Ensure that IOExceptions on session commit are handled properly
Signed-off-by: joewitt <joewitt@apache.org>
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-08-18 20:06:41 -04:00
Matt Gilman ffa7ceb2b1 NIFI-861:
- Addressing issue with blank processor names when attempting to generate ellipsis on the canvas.
2015-08-17 15:23:36 -04:00
joewitt 6066557e66 NIFI-827 added headless entry
Signed-off-by: joewitt <joewitt@apache.org>
2015-08-17 09:16:34 -04:00
joewitt aa99884782 NIFI-850 removed nifi parent, updated nifi pom, moved all nifi subdirs up one level, fixed readme. 2015-08-15 13:12:22 -04:00