Apache
/
nifi
mirror of https://github.com/apache/nifi.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
2,328 Commits 146 Branches 201 Tags 403 MiB
Commit Graph

886 Commits

Author SHA1 Message Date
Scott Aslan 7db78e87a5 [NIFI-1761] UI - Introduce AngularJS application bootstrapping, Roboto fonts, and breadcrumbs directive. This closes #331 2016-04-15 09:46:06 -04:00
Mark Payne b59d3ece75 NIFI-1762: Changed Java dependency to 1.8 instead of 1.7 and refactored StandardNiFiServiceFacade to make use of Lambda expressions to simplify code base. Also had to address a unit test because changing to Java 8 results in calls to assertEquals to become ambiguous. This closes #352 2016-04-14 10:38:03 -04:00
Andy LoPresto 378ccf53c2
NIFI-1753 Replaced usage of javax.security.cert.X509Certificate with java.security.cert.X509Certificate and resolved user-reported ClassCastException when handling client certificates during TLS mutual authentication. 
Fixed nifi-utils pom.xml comment about additional dependencies. (+5 squashed commits)
Squashed commits:
[965b766] NIFI-1753 Removed temporary work-around of duplicate certificate conversion util method and added nifi-security-utils as dependency of nifi-utils.
[cd35f9b] NIFI-1753 Replaced legacy X.509 certificate declarations with new declarations in SSLSocketChannel and EndpointConnectionPool.
Temporary work-around of duplicate certificate conversion util method because nifi-utils cannot depend on nifi-security-utils.
[6420897] NIFI-1753 Replaced legacy X.509 certificate declarations with new declarations in PostHTTP.
[b9868ef] NIFI-1753 Added convenience method for extracting DN from peer certificate chain in SSL socket (canonical implementation to reduce code duplication and references to legacy certificate implementations).
Refactored logic retrieving legacy X.509 certificates with reference to convenience method in NodeProtocolSenderImpl.
Replaced logic retrieving legacy X.509 certificates with reference to convenience method in SocketProtocolListener.
Cleaned up exception handling in SocketProtocolListener.
Replaced legacy X.509 certificate declarations with new declarations in HandleHttpRequest (needs manual test).
[e2d1c35] NIFI-1753 Added convenience methods for converting legacy X.509 certificates and abstract certificates to correct X.509 format.
Added unit tests for certificate manipulation.
Replaced logic retrieving legacy X.509 certificates with new logic in NodeProtocolSenderImpl.
Added bcpkix (Bouncy Castle PKI implementation) dependency to nifi-standard-processors pom.

This closes #346.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
 2016-04-13 18:30:21 -07:00
ijokarumawak dfa27263d2
NIFI-1180 Adding more IT tests. 
This closes #336.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
 2016-04-12 17:44:29 -07:00
Oleg Zhurakousky bd1f0a82ae NIFI-1748 removed un-used test data 
This closes #344.

Signed-off-by: Joe Skora <jskora@apache.org>
 2016-04-11 19:50:22 -04:00
Aldrin Piri e76a75c26b NIFI-1749 This closes #342. Adjusting tests that fail without a connection to the Internet and removing attributable elements from test resources. 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-04-11 13:22:00 -04:00
Oleg Zhurakousky 95dda1d920 NIFI-1748 removed un-used test data 
This closes #341.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-04-11 02:24:45 -04:00
Mike Moser 5625686ea4 NIFI-1746 avoid exceptions if configuration dir is a symbolic link 
Signed-off-by:  Joe Skora <jskora@apache.org>
 2016-04-11 02:05:48 -04:00
Joe Skora 696a12e4e7 Revert "NIFI-1748 removed un-used test data" 
This reverts commit 47cd9ff22c.
 2016-04-11 02:05:24 -04:00
Oleg Zhurakousky 47cd9ff22c NIFI-1748 removed un-used test data 
This closes #341.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-04-10 13:21:37 -04:00
mans2singh e3155a8a4b NIFI-1516 Provide AWS DynamoDB Delete/Put/Get processors 
This closes #224.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-04-08 11:23:04 -04:00
Zachary Jaffee 41583e6dc0 NIFI-400 remove deprecated processor EvaluateRegularExpression 
This closes #77

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-04-08 09:11:20 -04:00
Matt Gilman 5de40ccec3 NIFI-1553: 
- Implementing a file based authorizer.
- Providing an example authorizations files.
- Address comments from PR.
- This closes #330
 2016-04-07 16:28:42 -04:00
Matt Gilman 3f4ac3156c Revert "NIFI-1551:" 
This reverts commit c4d06f203d. Accidently merged wrong branch.
 2016-04-07 16:22:35 -04:00
Matt Gilman c4d06f203d NIFI-1551: 
- Starting to remove the AuthorityProvider.
- This closes #330
 2016-04-07 16:18:36 -04:00
Adam Lamar 2bcc31330c
NIFI-1180: Modify PutS3Object to enable encryption 
This closes #246.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
 2016-04-07 11:19:22 -07:00
ijokarumawak 9235a28f82 NIFI-1736 Move kafka.StreamScanner to nifi-utils. This closes #333 2016-04-07 07:37:28 -04:00
Bryan Bende 3adb45eafc NIFI-1630 This closes #332. Making TestPutUDP select an open port for testing to avoid conflicts 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-04-06 23:39:30 -04:00
Mark Payne 84b1c60d54 NIFI-1726: Addressed issue where we can run into an infinite loop if we are expiring data based on a timestamp instead of disk space usage and we have a file whose timestamp is exactly equal to our threshold for deletion 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-04-06 11:27:53 -04:00
Matt Brown b0cc6ae7e8 NIFI-1630 PutUDP processor created. 
Signed-off-by: Bryan Bende <bbende@apache.org>
 2016-04-06 11:04:01 -04:00
Aldrin Piri 3effc6cca5 NIFI-1689 Correcting TimeUnit conversion to be milliseconds instead of the incorrect microseconds and using historical modified times to avoid issues with second granularity of HFS+ 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-04-05 09:38:48 -04:00
Aldrin Piri 0e9e9b36d9 NIFI-1728 Specifying charsets for messages sent to/received from Kafka in Kafka processor and related tests to remedy failures in Windows environments. Specifying EOF as an int instead of a byte. 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-04-05 09:38:39 -04:00
Mike Moser 1c025dd4be NIFI-1714 use standard HttpClient cookie policy instead of default 
Signed-off-by: jpercivall <joepercivall@yahoo.com>
 2016-04-04 13:13:17 -04:00
Matt Gilman 9aa69b242e NIFI-1552: - Introducing the Authorizer API and additional components necessary for discovery and creation of configured instances. - Minor refactoring of existing Authority Provider API code/configuration to avoid some xsd naming conflicts. These components will be removed in NIFI-1551. - Introducing a number of the resource definitions that the Authorizer will make access decisions on. This list is likely not finalized may see some changes in NIFI-1554. - Address comments from PR. - This closes #318. 
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
 2016-04-04 11:47:43 -04:00
Mark Payne 1ac05266a5 NIFI-483: Use ZooKeeper's Leader Election to determine Primary Node. This closes #301 
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
 2016-04-04 11:47:08 -04:00
Mark Payne 0d3bd2c401 NIFI-1563: - Federate requests and merge responses from nodes instead of storing bulletins and stats at NCM - Updating UI to support restructured status history DTO. - Return 'Insufficient History' message if aggregate stats don't have enough data points, even if all nodes do (which can be the case if the node performing the aggregation has a different value for the 'nifi.components.status.snapshot.frequency' property than the other nodes) - Bug fixes; code cleanup; replicate requests to bulletin board endpoint - Refactored the <Component>StatusDTO objects into <Component>StatusDTO, <Component>StatusSnapshotDTO, Node<Component>StatusSnapshotDTO objects - Introducing endpoints for accessing individual component status. - Wiring up new endpoints and updated core. - Code clean up. - Starting to handling status merging of individual components. - Nodewise breakdown has been added to Processors but the remaining components still need to be updated. - Refactor so that System Diagnostics requests are replicated to nodes instead of the information being pulled from Heartbeats - Replicate request for counters instead of pulling them from heartbeats - Removed the getCounters / setCounters method from HeartbeatPayload - Implementing component specific endpoints. - Removing unused endpoints. - Supporting nodewise breakdown for system diagnostics and counters. - Updating DTOs to use more consistent naming. - Code clean up. - Addressing contrib issues. - Removed ProcessGroupStatus from HeartbeatPayload - Removing nodewise from the system diagnostics endpoint. Had included it for testing that option but did not intend for it to be committed. - Addressing comments in PR #294. - This closes #294 
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
 2016-04-04 11:42:39 -04:00
Matt Gilman 2de7f3f884 Updating versions to 1.0.0-SNAPSHOT. 2016-04-04 11:36:20 -04:00
Matt Gilman 16108467c1 NIFI-1695: 
- Removing hidden control characters that were affecting graph styles. This closes #311
 2016-04-04 11:26:55 -04:00
Bryan Bende 04c6830937 NIFI-1698 Improving customValidate in AbstractHadoopProcessor and HBaseClient service to not reload Configuration unless it changed. This closes #313 2016-04-04 10:14:05 -04:00
Stephen Harper e02c79975e NIFI-1686 - NiFi is unable to populate over 1/4 of AMQP properties from flow properties 
This closes #305
 2016-04-04 10:07:50 -04:00
Oleg Zhurakousky 25290cedc4 NIFI-1701 fixed StreamScanner, added more tests 
NIFI-1701 additional refactoring, clean up and more tests
 2016-04-04 09:55:16 -04:00
Bryan Bende 65b5c1a5f1 NIFI-1697 Ensuring FlowController appropriately wraps code with NarCloseable. This closes #312 2016-04-04 09:22:55 -04:00
Matt Burgess 65b26e6f41 NIFI-1691: Add Fetch Size property to QueryDatabaseTable 
This closes #307

Signed-off-by: Bryan Bende <bbende@apache.org>
 2016-03-29 10:51:30 -04:00
Oleg Zhurakousky c3d54ab724 NIFI-1684 This closes #308. fixed ZKClient connection leak 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-29 09:56:24 -04:00
Oleg Zhurakousky 9912f18de5 NIFI-1684 This closes #302. fixed random partitioner initialization 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-29 09:56:08 -04:00
Oleg Zhurakousky 1292581ec8 NIFI-1684 added penalization on failure 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-29 09:56:04 -04:00
Oleg Zhurakousky 8d960f5243 NIFI-1684 fixed NPE in PutKafka when retrieving key attribute bytes 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-29 09:56:04 -04:00
Oleg Zhurakousky e35c40b0fd NIFI-1684 fixed NPE, added tests 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-29 09:56:04 -04:00
Mark Payne 8000304e6a NIFI-1442: This closes #306. Use CircularFifoQueue instead of Set to store nodes' bulletins 
Joint effort by Toivo Adams from PR306 and and Mark Payne

Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-28 13:35:36 -04:00
Aldrin Piri e977729b56 NIFI-1634-rc2 prepare for next development iteration 2016-03-23 18:56:34 -04:00
Aldrin Piri 0b9bd20d31 NIFI-1634-rc2 prepare release nifi-0.6.0-RC2 2016-03-23 18:56:22 -04:00
Mark Payne e4b7e47836 Merge branch 'NIFI-1676' 2016-03-23 12:14:35 -04:00
Mark Payne b95a82f4a5 NIFI-1676: Do not allow Processor to be started if state is STOPPING; ensure that it is STOPPED 2016-03-23 11:31:20 -04:00
Mark Payne f7ecb47e29 Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/nifi 2016-03-23 11:24:28 -04:00
Mark Payne 9e39653192 Merge branch 'NIFI-1645' of https://github.com/olegz/nifi into NIFI-1645 2016-03-23 10:31:57 -04:00
Oleg Zhurakousky e0e00ff282 NIFI-1645 refactored PutKafka 
- used newest API available in 0.8.* version
- added PutKafka integration tests
- Kafka module code coverage is at 85%

NIFI-1645 polishing

NIFI-1645 PR comments round 1

NIFI-1645 PR comments round 2

NIFI-1645 change to use async Kafka producer

NIFI-1645 polishing

NIFI-1645 polishing

NIFI-1645 polishing

NIFI-1645 changed from java.util.Scanner to custom StreamScanner

NIFI-1645 polishing

NIFI-1645 final polish
 2016-03-23 10:31:34 -04:00
Oleg Zhurakousky a68f87f96e NIFI-1665 This closes #296. fixed GetKafka to reset consumer in case of timeout 
NIFI-1665 polishing

Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-22 21:48:47 -06:00
Bryan Bende 423b333b71 NIFI-1420 Fixing minor bugs in GetSplunk 
- Adding a Time Zone property so the Managed time ranges use the provided time zone when formatting the date strings
- Adding a Time Field Strategy property to choose between searching event time or index time
- Making the next iteration use previousLastTime + 1 ms to avoid overlap
- Fixing bug where GetSplunk incorrectly cleared state on a restart of NiFi
- This closes #299
 2016-03-22 15:23:42 -04:00
Matt Burgess bbbc77707f NIFI-1666: Fixed bug with EL evaluation in PutElasticsearch processor 2016-03-22 15:10:27 -04:00
Aldrin Piri 4babd067c1 NIFI-1664 Preferring System.nanoTime to System.currentTimeMillis and providing explicit handling of timestamps for files in those tests that are testing other attributes of the ListFile process besides timestamp which could lead to erroneous transmissions depending on exactly when files were created. 
Adding unsalted_128_raw.enc and salted_128_raw.enc to the list of exclusions for the RAT plugin which caused issues in the Windows environment for contrib-checks.

This closes #297.
 2016-03-22 14:13:37 -04:00
Mark Payne 736896246c NIFI-1649: Addressed bug in SplitText that resulted in improper handling of input data if Remove Trailing Newlines is true and there is a FlowFile that is only made up of newlines 2016-03-21 13:26:32 -04:00
Mark Payne 38c782c30b NIFI-1650: Ensure that proper size and offset are specified for Content Claim when viewing FlowFiles still in queue 2016-03-21 10:30:24 -04:00
Mark Payne d3578a7c03 NIFI-1650: Ensure that we seek to the appropriate offset within the Content Claim when downloading content of a FlowFile 2016-03-21 10:30:23 -04:00
trkurc 9cde92da16 NIFI-1656 Added locale support to ConvertAvroSchema and fixed locale problems in unit tests 
Reviewed by Joe Witt (joewitt@apache.org). This closes #292
 2016-03-21 01:44:46 -04:00
trkurc 3921ab827b NIFI-1651 unit tests work on windows. removed contributor name/package from data and schema 
Reviewed by Aldrin Piri (aldrin@apache.org) and Mike Moser (mosermw@apache.org). This closes #291
 2016-03-20 15:01:57 -04:00
Oleg Zhurakousky bb738f978a NIFI-1647 This closes #288. fixed validators and config resolution 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-18 10:11:00 -04:00
Joey Frazee 3a4546c08a NIFI-1617 Add source filename metadata to IdentifyMimeType 
Signed-off-by: Matt Burgess <mattyb149@apache.org>
 2016-03-18 00:23:16 -04:00
Oleg Zhurakousky 8f40d2b181 NIFI-1571 initial commit of SpringContext support 
Reviewed by Tony Kurc (tkurc@apache.org) and Joe Witt (joewitt@apache.org). This closes #271
 2016-03-17 23:43:47 -04:00
Michal Klempa e12e7a55b7 NIFI-1562 ExecuteStreamCommand and ExecuteProcess now support empty command line arguments 
This closes #247

Signed-off-by: Matt Burgess <mattyb149@apache.org>
 2016-03-17 23:15:12 -04:00
Pierre Villard e7a254f78e NIFI-1332 Fixed HTTP-204 handling + unit test 
This closes #259.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-03-17 23:03:51 -04:00
Aldrin Piri b44b177039 NIFI-1605 Adjust documentation and resources to reflect nifi.provenance.repository.rollover.time default 
This closes #263

Signed-off-by: Matt Burgess <mattyb149@apache.org>
 2016-03-17 22:14:24 -04:00
Oleg Zhurakousky 138c71de82 NIFI-1464 This closes #287. removed e.printStackTrace() from onTrigger in StandarProcessScheduler 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-17 17:18:56 -04:00
Bryan Bende 8773ec3d3e NIFI-1488 Refactoring HBase Kerberos support 
- Storing UGI so we can support multiple HBaseClientServices with different configs
- Creating nifi-hadoop-utils to hold utility code shared between HDFS and HBase processors
- Incorporating KerberosProperties into existing hadoop processors

This closes #281
 2016-03-17 17:12:57 -04:00
Matt Burgess 94cf1beeba NIFI-901: Fixed unit test to verify IP address of Cassandra contact point This closes #286 
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
 2016-03-17 13:53:07 -04:00
Andy LoPresto 76f2d5702f NIFI-1274 Added Kerberos authentication mechanism. 
NIFI-1274 Cleaned up TODO statements. (+3 squashed commits)
Squashed commits:
[fd101cd] Removed logic to check for presence of services to determine if token support is enabled when username/password authentication is enabled (Kerberos also requires tokens).
[c2ce29f] Reverted import changes to RulesResource.java.
[c269d72] Added Kerberos authentication mechanism.

Moved Kerberos service wiring from XML to Java to handle scenario where admin has not configured Kerberos (previously threw NullPointerException in FileSystemResource constructor). (+15 squashed commits)
Squashed commits:
[09fc694] Added Kerberos documentation to Admin Guide.
[ecfb864] Cleaned up unused logic.
[157efb3] Added logic to determine if client certificates are required for REST API (login, anonymous, and Kerberos service authentication all disabled).
Cleaned up KerberosService by moving logic to NiFiProperties.
[5438619] Added documentation for Kerberos login-identity-providers.xml.
[3332d9f] Added NiFi properties for Kerberos SSO.
[b14a557] Fixed canvas call to only attempt Kerberos login if JWT not present in local storage.
Added logic to handle ticket validation failure in AccessResource.
Changed wiring of Kerberos service beans to XML in nifi-web-security-context.xml for consistency.
[c31ae3d] Kerberos SPNEGO works without additional filter (new entry endpoint accepts Kerberos ticket in Authorization header and returns JWT so the rest of the application functions the same as LDAP).
[98460e7] Added check to only instantiate beans when Kerberos enabled to allow access control integration tests to pass.
[6ed0724] Renamed Kerberos discovery method to be explicit about service vs. credential login.
[ed67d2e] Removed temporary solution for Rules Resource access via Kerberos ticket.
[c8b2b01] Added temporary solution for Rules Resource access via Kerberos ticket.
[81ca80f] NIFI-1274 Added KerberosAuthenticationFilter to conduct SPNEGO authentication with local (client) Kerberos ticket.
Added properties and accessors for service principal and keytab location for NiFi app server.
Added KAF to NiFiWebApiSecurityConfiguration.
Added AlternateKerberosUserDetailsService to provide user lookup without dependency on extension bundle (nifi-kerberos-iaa-provider).
Added dependencies on spring-security-kerberos-core and -web modules to pom.xml.
[0605ba8] Added working configuration files to test/resources in kerberos module to document necessary config. This version requires the user to enter their Kerberos username (without realm) and password into the NiFi login screen and will authenticate them against the running KDC.
Also includes a sample keystore and root CA public key for configuring a secure instance.
[49236c8] Added kerberos module dependencies to nifi/pom.xml and nifi-assembly/pom.xml.
Added default properties to login-identity-providers.xml.
[928c52b] Added nifi-kerberos-iaa-providers-bundle module to nifi/pom.xml.
Added skeleton of Kerberos authenticator using Spring Security Kerberos plugin.
This closes #284

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
 2016-03-17 13:10:09 -04:00
Bryan Bende b188b0abd6 NIFI-1420 Fixing bug where a FlowFile should route to failure when PutSplunk can't createa connection, defaulting PutSplunk to TCP 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-16 17:43:28 -04:00
Mark Payne a7b97419e5 NIFI-1626: Throw an Exception proactively if too much state is attempting to be stored via ZooKeeperStateProvider 
NIFI-1626: Updated State Management section of Developer Guide

Signed-off-by: jpercivall <joepercivall@yahoo.com>
 2016-03-16 16:12:00 -04:00
Matt Burgess 98395de74f NIFI-1575: Add QueryDatabaseTable processor 2016-03-16 15:55:57 -04:00
Oleg Zhurakousky 148b4497b4 NIFI-1629 This closes #282. downgraded Kafka back to 0.8 - added context.yield to PutKafka - added lifecycle hooks to defend from Kafka deadlocks 
NIFI-1629 changd thread pool implementation in Get/PutKafka

Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-16 15:38:46 -04:00
Mark Payne 68cfc8c612 NIFI-1632: Fixed NPE that occurs if a capturing group is optional and not matched; updated docs to illustrate how this is handled 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-16 15:19:17 -04:00
Matt Burgess 0f61079300 NIFI-901: Add QueryCassandra and PutCassandraQL processors 
This closes #237

Signed-off-by: jpercivall <joepercivall@yahoo.com>
 2016-03-16 14:09:15 -04:00
Oleg Zhurakousky e83429a08c NIFI-1631 This closes #283. increased timeouts on test 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-16 11:06:21 -04:00
mans2singh 19bc5ba999 NIFI-1495 Adding support for AWS Kinesis Firehose 
This closes #213

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-03-16 09:56:08 -04:00
joewitt 8ed64c9083 NIFI-1420 added missing properties to block javadocs/sources generation on nar 2016-03-15 21:43:06 -04:00
joewitt 2b0a0c0597 NIFI-614 added necessary pom element to skip javadocs/sources for nar pom itself in jms proc nar. 2016-03-15 21:40:08 -04:00
joewitt d78c7faf14 NIFI-614 added necessary pom element to skip javadocs/sources for nar pom itself. 2016-03-15 21:35:50 -04:00
Oleg Zhurakousky 812da19cad NIFI-614 Added initial support for new style JMS 
NIFI-614 finalized JMSConnectionFactoryProvider ControllerService

NIFI-614 finalized implementation of both Processors and ControllerService

NIFI-614 added initial documentation

NIFI-614 addressed PR comment with unused import and squashed

NIFI-614 added @OnDisabled method

NIFI-614 changed POMs to 0.6

NIFI-614 removed local .gitignore

NIFI-614 added support for parsing Tibco URL

NIFI-614 removed setting of jms message id

NIFI-614 addressed PR comments, fixed tests

NIFI-614 addressed latest PR comments

NIFI-614 second round of PR comments addressed

NIFI-614 3rd round of PR comments addressed

NIFI-614 finalizing on PR comments

NIFI-614 more PR comments

This closes #222

Signed-off-by: jpercivall <joepercivall@yahoo.com>
 2016-03-15 15:48:03 -04:00
Oleg Zhurakousky cfeebfe7c7 NIFI-1464 fixed OnScheduled invocation to pass ProcessContext 
NIFI-1464 removed unused imports

Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-15 12:54:13 -04:00
Mark Payne a4a7d53769 NIFI-1627: Do not call context.yield for funnels and local ports; use batch size of 100 instead of 10 for local ports and funnels 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-15 12:54:03 -04:00
trkurc f22db218e3 NIFI-1481 fixed a couple bugs in bash script for cygwin and when JAVA_HOME not set 
Reviewed by Aldrin Piri (aldrin@apache.org). This closes #277
 2016-03-14 22:40:25 -04:00
puspendu.banerjee@gmail.com 76ce3f7520 NIFI-1481 support for dumping environment using nifi.sh env 
Reviewed and amended (Amendments reviewed by Aldrin Piri (aldrin@apache.org)) by Tony Kurc (tkurc@apache.org). This closes #218
 2016-03-14 19:24:21 -04:00
Oleg Zhurakousky c7df94e00f NIFI-1464 life-cycle refactoring part-2 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-14 17:13:55 -04:00
Mark Payne 56f79e1e85 NIFI-1588: Reworked how ListHDFS store state so that only 2 timestamps are stored and no paths 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-14 15:59:24 -04:00
Mark Payne 8c488d7e8e NIFI-1587: Always poll state from State Manager when running ListHDFS instead of relying on local state over the cluster state 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-13 14:36:36 -04:00
Mark Payne 5a8b2cf7f1 NIFI-1606: Run the onComponentRemoved logic of state providers in a background thread 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-13 14:08:24 -04:00
Mark Payne ce068ffc6c NIFI-1612: Do not ignore Remote Process Group's timeout configuration 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-13 13:40:45 -04:00
joewitt 730d999d4b NIFI-627 minor formating adjustments and removed exception based handling logic. This closes #268. This closes #274. 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-13 13:28:48 -04:00
Mike Moser 0cb4047add NIFI-627 removed flowfile penalization which could skew behavior when processor's Time Duration was less than Penalty Duration, improved over throttle penalization NIFI-990 corrected failure path NIFI-1329 refactored using FlowFileFilter to avoid repeatedly returning flowfiles to the input queue, producing misleading stats and excessive Tasks/Time used 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-13 13:28:43 -04:00
Mark Payne 7400b6f7c5 NIFI-1622: Ensure that the Nar Context Class Loader is used when calling Processor lifecycle methods 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-12 20:06:23 -05:00
Mark Payne a85c119533 Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/nifi 2016-03-11 12:55:23 -05:00
Oleg Zhurakousky 59fac58c96 NIFI-1464 ensured that OnUnscheduled is treated the same as OnScheduled 
NIFI-1464 polished javadocs, error messages and docs
 2016-03-11 12:54:50 -05:00
Oleg Zhurakousky 1c22f3f012 NIFI-1464 refactored the latest commit 2016-03-11 12:54:50 -05:00
Oleg Zhurakousky 48af0bfbc5 NIFI-1464 addressed latest PR comments 
NIFI-1464 polishing
 2016-03-11 12:54:50 -05:00
Oleg Zhurakousky f53f45def3 NIFI-1464 addressed PR comments from @apiri and @markap14 2016-03-11 12:54:50 -05:00
Oleg Zhurakousky 0c5b1c27f2 NIFI-1464, Refactored Processor's life-cycle operation sequence 
* Simplified and cleaned StandardProcessScheduler.start/stopProcessor methods
* Added stop/start operations to ProcessorNode.
* Removed unnecessary synchronization blocks related to ScheduledState in favor of enforcing order and idempotency via CAS operations. Those synchronization blocks were causing intermittent deadlocks whenever @OnScheduled blocks indefinitely.
* Added support for stopping the service when @OnScheduled operation hangs.
* Fixed the order of life-cycle operation invocation ensuring that each operation can *only* be invoked at the appropriate time
* Removed unnecessary locks from StandardProcessNode since Atomic variables are used.
* Removed calls to @OnStopped from ContinuallyRunningProcessTask while ensuring that procesor's full shut down in implementation of StandardProcessorNode.stop() method.
* Removed dead code
* Added comprehensive tests suite that covers 95% of Processor's life-cycle operations within the scope of FlowController, StandardProcesssScheduler and StandardProcessNode
* Improved and added javadocs on covered operations with detailed explanations.
 2016-03-11 12:54:50 -05:00
Matt Burgess e1cf37fb89 NIFI-1619: Fix Elasticsearch processor bug when flow file missing ID attribute 
This closes #269

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-03-11 12:01:28 -05:00
Bryan Bende 1373bf6725 NIFI-899 Rewrite of ListenUDP to use new listener framework, includes the following changes: 
- Adding Network Interface property to AbstractListenEventProcessor and ListenSyslog
- Adding sending host and sending port to DatagramChannelDispatcher
- Creation of common base class AbstractListenEventBatchingProcessor
- Refactor of ListenUDP, ListenTCP, and ListenRELP to all extend from AbstractListenEventBatchingProcessor
- Changing DatagramChannelDispatcher, socket handlers, and RELP handler to use offer() when queueing instead of put(), and log an error if the offer failed

This closes #266
 2016-03-11 09:07:05 -05:00
Pierre Villard 8f0116544a NIFI-1491 Deprecated existing method and added new one throwing eception 
Reviewed and amended (to add javadoc @deprecation link and to carry forward other deprecated methods) by Tony Kurc (tkurc@apache.org). This closes #227
 2016-03-10 22:15:13 -05:00
Michal Klempa 784f2a2c20 NIFI-1518 InferAvroSchema note has an option to set CSV delimiter 
Reviewed (and amended based on acknowledgement in PR review) by Tony Kurc (tkurc@apache.org). This closes #235.
 2016-03-10 21:36:31 -05:00
Mark Payne d3367a7dc3 nNFI-1618: Updated unit test to use 30,000 FlowFiles to test dropping of swapped out FlowFiles intead of using 210,000. Thiss unit test creates a lot of objects and when reducing the number of FlowFiles generated to 30K instead of 210K, all appears to work okay 2016-03-10 16:01:29 -05:00
Pasqualino Ferrentino 73e168e954 NIFI-1003 A relationship can be auto-terminable. In this case the processor will auto-terminate the relationship and allow the user to run it even if he does not connect those relationships and does not terminate them. 
This closes #217

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-03-10 14:56:46 -05:00
Aldrin Piri da234abd76 NIFI-1488 Adjusting unused imports and adding license exclusions to RAT plugin for build. 2016-03-10 08:58:12 -05:00
Mark Payne 2839a2f215 NIFI-15: Address issue where incomplete swap files can result in continually attempting to swap in data without ever being successful 
Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-03-09 23:46:53 -05:00
Jeff Lord 7f15626af5 NIFI-1488: Add HBase Kerberos Support with UGI 
Added `kerberos-principal` and `kerberos-keytab` properties
to the HBase service.
 2016-03-09 15:51:30 -05:00
Mark Payne a2164136db Merge branch 'NIFI-1573' of https://github.com/rpmiskin/nifi into NIFI-1573 2016-03-09 09:40:37 -05:00
Mark Payne 0f9b55afb6 Merge branch 'nifi-1509' of https://github.com/mans2singh/nifi into NIFI-1509 2016-03-09 08:19:45 -05:00
mans2singh 1211996d35 updated attributes using hashmap rather than individually 2016-03-08 21:12:49 -08:00
Mark Payne 9874d35b6b NIFI-1600: Ensure that if we move a RemoteProcessGroup, it also updates the Parent Process Group of the ports 2016-03-08 12:09:45 -05:00
Oleg Zhurakousky 998ac1c830 NIFI-1534 removed SupportsBatching annotation from PutKafka 
Signed-off-by: Matt Burgess <mattyb149@gmail.com>

This closes #249
 2016-03-08 10:03:27 -05:00
Bryan Bende 6f5fb59479 NIFI-1420 Adding Splunk bundle containing PutSplunk, and GetSplunk, and adding a ListenTCP processor to standard processors. Refactored internal code from PutSyslog to create a generic AbstractPutEventProcessor which PutSplunk extends from. 
This closes #233
 2016-03-07 18:21:17 -05:00
Oleg Zhurakousky 4ce7b679e1 NIFI-1595 fixed ReflectionUtils to honor bridge methods 
Refactored and simplified ReflectionUtils while at it
Added ReflectionUtilsTest

This closes #260.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
With adjustments to formatting and whitespace.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-03-07 15:27:35 -05:00
Matt Burgess a7e6820fd9 NIFI-1596: ExecuteSQL Error Creating Schema 2016-03-07 10:42:21 -05:00
Bryan Bende 19e53962ca NIFI-1579 Performance improvements for ListenSyslog which include removing an unnecessary yield and exposing a configurable size for the internal queue used by the processor, changing ListenSyslog to use a 20ms poll and use a long poll when batching, also including same improvements for ListenRELP 2016-03-04 09:17:45 -05:00
Mark Payne 6776060ac8 NIFI-1577: Close any streams that are left open for 'append' when the session is checkpointed, rather than waiting for it to be committed 2016-02-29 16:02:26 -05:00
Richard Miskin 738236a41d NIFI-1573 Allow programmatic access to a Processor's name 2016-02-27 15:18:22 +00:00
Aldrin Piri 58e50ae39d NIFI-1511 Incorporating Groovy unit tests as part of the main build and providing explicit compile scope on the Groovy dependency for the scripting bundle. This closes #220 
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
 2016-02-26 10:25:45 -05:00
Mark Payne 28c2a3e5a6 NIFI-1256: Added a 'Satisfies Expression' option for Matching Strategy of RouteText 
Signed-off-by: jpercivall <joepercivall@yahoo.com>
 2016-02-25 15:59:57 -05:00
Tony Kurc c7e24c7569 NIFI-1513: fixed some easy to fix errors 
Addressing checkstyle issue.

This closes #221

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
 2016-02-25 15:21:40 -05:00
Jeremy Dyer 32f476aaa7 NIFI-1357 Add Snappy compression to "CompressContent" Processor 
This closes #164.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-02-25 12:47:01 -05:00
Matt Gilman 0d13de0cf3 NIFI-1539: - Comparing octet stream content type by using starts with and ignores case. 2016-02-25 10:13:07 -05:00
Sönke Liebau fc92441981 NIFI-1539 - Add normalization of content type for content viewing 
Add code to ContentViewerController to strip content type of any trailing parameters and lowercase the type and subtype.

Added function to ViewableContent to enable retrieving the original value of the content type if needed.

This closes #242

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
 2016-02-25 10:12:45 -05:00
Mark Payne 6af108c0ca NIFI-1557: Ensure that Reporting Tasks & controller services are ordered by ID before fingerprinting 2016-02-23 18:02:58 -05:00
mans2singh 6138b0332e corrected contrib failure 2016-02-22 20:12:24 -08:00
mans2singh 50da696cda updated capability documentation based on comments 2016-02-22 19:56:12 -08:00
James Wing 24a77755de NIFI-1548 Fixing Controller Service Usage Button. This closes #245 
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
 2016-02-22 21:09:01 -05:00
Joe Skora 5ee83574d8 NIFI-1545: Adjust integration tests ITDeleteS3Object, ITFetchS3Object, and ITPutS3Object to account for AWS new proxy properties. Correct typo in AbstractS3Processor.createClient() log message. 
Reviewed by Tony Kurc (tkurc@apache.org). This closes #243
 2016-02-22 19:16:55 -05:00
Oleg Zhurakousky a87230358d NIFI-1536 This closes #241. ensured that UserGroupInformation is re-initialized on each call to 'AbstractHadoopProcessor.resetHDFSResources(..)' 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-02-22 17:23:53 -05:00
Matt Gilman e7676ffae5 NIFI-1546: - Ensuring that the data reference query is not null before attempt to extract a cluster node id from it. 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-02-22 12:39:37 -05:00
Mark Payne 122cba0ee7 NIFI-1527: Fixed issue that resulted in resource claims' claimant count getting incremented twice on restart 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-02-22 12:36:08 -05:00
Joe Skora 4c5263220c NIFI-1542: Update how PutS3Object handles a 403 AccessDenied response to a S3 ListMultipartUploads request. 
* Change log message from error to warning and added note about S3 permissions.
 * Advance the ageoff check time by the check interval as if the request succeeded to prevent re-checking on every upload when permission does not exist.

Reviewed by Tony Kurc (tkurc@apache.org) (with minor amendments for whitespace and error message consistency). This closes #240
 2016-02-21 18:11:23 -05:00
Bryan Bende 9576f41603 NIFI-1529 Forcing hadoop-libraries-nar to version 4.2.5 of httpclient and httpcore since that is what hadoop-client 2.6.2 normally brings in - Removing dependencies from the nar pom that should be transitively brought in by hadoop-client - Updating NOTICE files in hadoop-libraries-nar and nifi-assembly according to dependency changes 
This closes #236
 2016-02-19 16:35:49 -05:00
Matt Gilman a8edab2e79 NIFI-1497: - Introducing a one time use password service for use in query parameters when accessing UI extensions and downloading resources. - Using one time use tokens when accessing ui extensions and downloading resources. - Ensuring appropriate roles when accessing component details through the web context for custom UIs. - Addressing typo in class name. - Ensuring appropriate roles when accessing content through the content access. - Code clean up. - Refactoring some basic scripts for accessing JWT tokens so UI extensions can reuse common functionality. 
Signed-off-by: Bryan Bende <bbende@apache.org>
 2016-02-19 10:54:53 -05:00
joewitt 86ab4428f0 NIFI-1520 by default skip javadoc and source jar generation in nars and wars 
Reviewed by Tony Kurc (tkurc@apache.org). This closes #234
 2016-02-18 18:45:26 -05:00
Mark Payne 3bb18b9653 NIFI-1527: Ensure that we increment Claimant Counts for content claims that are referenced by Swapped-Out FlowFiles on restart of nifi 2016-02-18 16:25:57 -05:00
mans2singh bdc1ef1f8c corrected discription 2016-02-17 21:36:53 -08:00
puspendu.banerjee@gmail.com 1c73d9090a NIFI-1501 : Test Monitor Activity has spurious failures. 
Added Timeout for the test. So, in case something goes South, threshold will never be more that that Timeout period.

Reviewed with amendments for whitespace by Tony Kurc (tkurc@apache.org). This closes #225
 2016-02-17 21:55:10 -05:00
Pierre Villard 502aebc66a NIFI-1063 Fixed exception logging 
Fixed logging of an exception in PutHDFS

Reviewed by Tony Kurc (tkurc@apache.org). This closes #226
 2016-02-17 20:59:31 -05:00
Michal Klempa c2998a71ec NIFI-1514 Fixed ExecuteStreamCommand when expressions in command arguments are used, contents are not split by command separator 
This closes #223.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-02-17 17:48:23 -05:00
Tony Kurc 6c2ba997a0 Merge branch 'NIFI-1379-RC3' 2016-02-16 20:13:41 -05:00
mans2singh b921bf602e updated comment 2016-02-16 13:33:48 -08:00
mans2singh ad39b1c41f renamed class and removed unused imports 2016-02-16 13:25:28 -08:00
mans2singh 36f1caad33 Adding support for using an HTTP proxy with AWS processors. 
This closes #209

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-02-16 14:51:06 -05:00
Pierre Villard ef0018cf66 NIFI-1282 This closes #228. Fixed error message when attempting to start a disabled port 
Fixed error message when attempting to start a disabled port

Signed-off-by: joewitt <joewitt@apache.org>
 2016-02-16 09:38:56 -05:00
Tony Kurc ad73b5c9d9 NIFI-1379-RC3 prepare for next development iteration 2016-02-12 17:28:10 -05:00
Tony Kurc 8309dba80b NIFI-1379-RC3 prepare release nifi-0.5.0-RC3 2016-02-12 17:27:59 -05:00
Tony Kurc cae5b109c0 NIFI-1507: Update bouncycastle license reference 2016-02-12 16:58:26 -05:00
mans2singh 528c4bc591 nifi-1509 catch recoverable service exceptions explicity 2016-02-12 09:22:41 -08:00
mans2singh cd05bedda8 Nifi-1509 - commented out integration test 2016-02-11 21:14:28 -08:00
mans2singh 180a90d12b first commit for aws lambda 
corrected attributes of flow file

added shutdown + provenence calls

minor formatting and unused imports correction

removed unused property

updated to populate exception attributes in flow file

updated write attributes
 2016-02-11 20:59:54 -08:00
joewitt f44eb643dd NIFI-1461 added run.as filtered property for RPM and removed extraneous properties 2016-02-10 21:21:19 -05:00
Matt Gilman 778229eb52 NIFI-1499: - Ensuring the universal keystroke capture is available to UI extensions. 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-02-10 19:15:48 -05:00
Aldrin Piri 1a512cd1e6 NIFI-1484 Making use of timestamps at various points of execution to provide listing of all but the latest files which are held until a subsequent execution. 
Correcting nifi-amqp-nar bundle's pom description.

This closes #212.
 2016-02-10 10:25:47 -05:00
Matt Gilman f4487dd5f6 NIFI-1492: - Limiting the amount of state entries returned to a client. - Code clean up. 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-02-09 20:48:56 -05:00
Oleg Zhurakousky 8a05f6880a NIFI-1493 This closes #211. removed Executor from InvokeScriptedProcessor 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-02-09 20:31:57 -05:00
Aldrin Piri 1c03fc7871 Removing unused import to remedy checkstyle error on previous documentation update. 2016-02-09 16:13:16 -05:00
Aldrin Piri a33289393d Correcting documentation on SnippetResource#createSnippet to reflect that it is creating a snippet. 2016-02-09 15:33:46 -05:00
ledor473 eb6f9f0fec NIFI-1423 Allow to penalize FlowFiles that are routed to No Retry relationship 
This closes #183

Signed-off-by: jpercivall <joepercivall@yahoo.com>
 2016-02-08 11:20:08 -05:00
Aldrin Piri 2673370cba NIFI-1483 Correcting logic in terms of when local persistence files are removed during the migration process. 
Reviewed by Tony Kurc (tkurc@apache.org). This closes #206
 2016-02-07 17:40:12 -05:00
joewitt ea5818c398 NIFI-1486 added commons io and lang to nifi-amqp-nar NOTICE 
Reviewed by Tony Kurc (tkurc@apace.org). This closes #208
 2016-02-07 17:27:53 -05:00
Tony Kurc 5be83166ee NIFI-1485: Updated 'header' section of NiFi NOTICE files 2016-02-07 13:52:00 -05:00
Tony Kurc 303f8eabf1 NIFI-1379: Move to 0.5.0-SNAPSHOT, add tkurc code signing key to keys 2016-02-06 08:49:48 -05:00
Andy LoPresto b407379670 NIFI-1257 Resolved legacy compatibility issue with NiFi legacy KDF salt length dependent on cipher block size. 
Replaced screenshot for NiFiLegacy salt encoding.
Added description of legacy salt length determination in admin guide.
Added logic for NiFiLegacyCipherProvider to generate and validate salts of the length determined by the cipher block size.
Changed EncryptContent to default to Bcrypt KDF.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-02-05 23:38:58 -05:00
Matt Gilman 7314af6177 NIFI-259: - Addressing issues that arose from a merge conflict. 
Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-02-05 17:47:56 -05:00
Aldrin Piri 4df6512126 Merge branch 'NIFI-259' 2016-02-05 14:09:29 -05:00
Aldrin Piri 16f1852453 NIFI-259 Updating logger message for AbstractListProcessor to reference State Manager in onTrigger instead of the distributed cache. 2016-02-05 12:55:06 -05:00
Andy LoPresto 498b5023ce NIFI-1257 NIFI-1259 
Added utility method to return the maximum acceptable password length for PBE ciphers on JVM with limited strength crypto because BC implementation is undocumented (based on empirical evidence).
Updated EncryptionMethod definitions to accurately reflect need for unlimited strength crypto according to algorithm key length.
Added processor logic to invoke keyed cipher.
Added EncryptContent processor property for raw hex key (always visible until NIFI-1121).
Added validations for KDF (keyed and PBE) and hex key.
Added utility method to return list of valid key lengths for algorithm.
Added description to allowable values for KDF and encryption method in EncryptContent processor.
Added IV read/write to KeyedCipherProvider and changed from interface to abstract class.
Added salt read/write logic to NifiLegacy and OpenSSL cipher providers.
Changed RandomIVPBECipherProvider from interface to abstract class.
Updated strong KDF implementations.
Renamed CipherFactory to CipherProviderFactory.
Added unit test for registered KDF resolution from factory.
Updated default iteration count for PBKDF2 cipher provider.
Implemented Scrypt cipher provider.
Added salt translator from mcrypt format to Java format.
Added unit tests for salt formatting and validation.
Added surefire block to groovy unit test profile to enforce 3072 MB heap for Scrypt test.
Added local Java implementation of Scrypt KDF (and underlying PBKDF2 KDF) from Will Glozer.
Defined interface for KeyedCipherProvider.
Implemented AES implementation for KeyedCipherProvider.
Added Ruby script to test/resources for external compatibility check.
Added key length check to PBKDF2 cipher provider.
Changed default PRF to SHA-512.
Added salt and key length check to PBKDF2 cipher provider.
Added utility method to check key length validity for cipher families.
Added Bcrypt implementation.
Implemented PBKDF2 cipher provider.
Added default constructor with strong choices for PBKDF2 cipher provider.
Implemented NiFiLegacyCipherProvider and added unit tests.
Added key length parameter to PBKDF2 cipher provider.
Added PRF resolution to PBKDF2 cipher provider.
Added RandomIVPBECipherProvider to allow for non-deterministic IVs.
Added new keyed encryption methods and added boolean field for compatibility with new KDFs.
Added CipherFactory.
Improved Javadoc in NiFi legacy cipher provider and OpenSSL cipher provider.
Added KeyedCipherProvider interface.
Added OpenSSL PKCS#5 v1.5 EVP_BytesToKey cipher provider and unit test.

This closes #201.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-02-04 19:40:55 -05:00
jpercivall 29fb9c9393 NIFI-259 Corrected some logic in AbstractListProcessor regarding being elected primary node and improved readability in GetHBase a smidge 2016-02-04 16:07:18 -05:00
Matt Gilman 6f4c3db186 NIFI-259: 
- Fixing the line height to ensure the multiline ellipsis is consistent across browsers.
 2016-02-04 08:59:26 -05:00
puspendu.banerjee@gmail.com 0690aee452 NIFI-1460 - Performance improvement. test Timeout Mitigation. less IO, less dependency on /dev/(u)random. 
- Incorporated comments for UUID format conformance & optimized.
          - polished to satisfy this test-case. we don't need incrementAndGet
Reviewed by Tony Kurc (tkurc@apache.org). This closes #202
 2016-02-03 21:50:24 -05:00
Joe Skora 769b044e30 NIFI-1107: Re-integrate Multipart Upload changes into PutS3Object. 1. add Multipart upload logic to allow resuming an upload after process/instance restart, 2. add local state management to track the part uploaded for a flowfile, 3. add configurable AWS S3 state management to abort orphaned uploads, and 4. adapt to IT test naming. 
Pull request updates: * Cleanup property description to reflect processor details, not code specifics. * Only resume an upload from local state if uploadID still exists in S3, otherwise delete local state and start a new upload. * Check that local state from was actually created before trying to delete it.

Re-integrate Multipart Upload changes into PutS3Object.
1. add Multipart upload logic to allow resuming an upload after process/instance restart,
2. add local state management to track the part uploaded for a flowfile,
3. add configurable AWS S3 state management to abort orphaned uploads, and
4. adapt to IT test naming.

Fixed test issues
 * added mock client for testing local state without matching S3 state,
 * updated regex for multipart etag pattern.

Reviewed by Tony Kurc (tkurc@apache.org). This closes #192
 2016-02-03 20:54:32 -05:00
jpercivall 88d4d2ce5f NIFI-259 Corrected GetHttp state managment and added a new unit test 2016-02-03 20:04:39 -05:00
Matt Gilman 55b77fe03b NIFI-1223: - Removing conf directory from unit test. 
Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-02-03 16:05:38 -05:00
Matt Burgess 478226451c NIFI-210: Add support for multiple module paths for scripting 2016-02-03 10:41:47 -05:00
Matt Gilman 7711106d62 NIFI-259: - Adding additional documentation for embedded ZK. 
Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-02-03 10:13:19 -05:00
Aldrin Piri 8a668fd344 NIFI-259: Fixed bug that caused StateProvider.replace to return true if the value had never been set, instead of false. Fixed typos in administration-guide 
Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-02-02 23:03:31 -05:00
joewitt 2d6fec5fa9 NIFI-865 mild cleanup for LICENSE NOTICE 2016-02-02 21:13:28 -05:00
Oleg Zhurakousky dc8b62c3a6 NIFI-865 Added processors to Get and Put to/from AMQP-based messaging systems 
This closes #200

Signed-off-by: Bryan Bende <bbende@apache.org>
 2016-02-02 20:57:10 -05:00
Matt Burgess 75af3a2ebe NIFI-1275: Added FetchElasticsearch processor and support for secure clusters 
This closes #180.

Signed-off-by: jpercivall <joepercivall@yahoo.com>
 2016-02-02 18:21:31 -05:00
scarpacci 0c137bc22d Added path home property and unit test to elasticsearch processor in support of the node client 2016-02-02 17:26:40 -05:00
Matt Burgess 943d0a6e53 Adding to PutElasticsearch - validators, unit tests, etc. 2016-02-02 17:26:40 -05:00
scarpacci e6cfcf40d0 Initial commit for the elasticsearch bundle to Nifi 2016-02-02 17:26:39 -05:00
Matt Gilman 8cf34c3ea5 NIFI-1457: 
- Update heartbeat bean when primary state changes.
 2016-02-02 13:16:10 -05:00
Matt Burgess 77ab5d3684 NIFI-210: This closes #185. Add ExecuteScript and InvokeScriptedProcessor 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-02-01 21:09:06 -05:00
Bryan Bende 1e56de9521 NIFI-1417 Exposing several connection settings on the Solr processors to help deal with timeouts and adding an @OnStopped method to close the SolrClient which didn't appear to be happening before 
This closes #194
 2016-02-01 18:42:00 -05:00
Mark Payne 35d2b921ea NIFI-259: Distinguish between unable to communicate with ZK and 'bad version' when performing 'replace' method of ZooKeeperStateProvider 
Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-02-01 16:29:12 -05:00
Aldrin Piri a931e72787 NIFI-259: Expose the ZK exception code in the event of a KeeperException within the ZooKeeperStateProvider. 2016-02-01 16:29:12 -05:00
Aldrin Piri 447e401912 NIFI-259: Extending the StateProvider interface to provide a getSupportedScopes method and implemented this based on the capabilities of each of its implementations. Used supported scope to evaluated configurations at startup and prevent issues when trying to make use of state 2016-02-01 16:29:12 -05:00
Aldrin Piri 257eca9c46 NIFI-259 Removing StateProviderScope and refactoring to use Scope in its place. 2016-02-01 16:29:11 -05:00
Aldrin Piri f2c366cf9c NIFI-259: Correcting path to the ZooKeeperStateProvider. 2016-02-01 15:26:27 -05:00
Matt Gilman 72c8467b9f NIFI-259: 
- Addressing typo in documentation.
- Minor tweaks to admin guide.
- Adding support to stand up a ZooKeeperServer when a quorum peer is not distributed (ie supporting both embedded standalone and cluster).

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-02-01 14:07:38 -05:00
Aldrin Piri f47168213b NIFI-259: Adjusting validation logic for Connect String to catch errors at startup. 2016-02-01 14:07:38 -05:00
Mark Payne ef80549d63 Merge branch 'NIFI-1093' of https://github.com/olegz/nifi into NIFI-1093 2016-02-01 11:51:27 -05:00
Daniel Cave 171b9c4e94 NIFI-1093 - Added settings for processing JSON documents when validating against database columns to fail, log a warning, or ignore unmatched columns. Added tests for new settings. 
Signed-off-by: Mark Payne <markap14@hotmail.com>
 2016-02-01 11:45:15 -05:00
Aldrin Piri 6902812678 NIFI-259: Adjusting property description of the connect string to be more specific concerning socket tuple format and the associated default port if one is not specified. Providing minor cleanup and removing unused import. 2016-02-01 10:38:45 -05:00
Matt Gilman dbe8ff3f44 NIFI-1426: 
- Ensuring we aren't preventing default when focused in a textarea.
 2016-02-01 08:28:18 -05:00
joewitt 1bf7a585b2 NIFI-1456 This closes #198. reduced tests to 10MB instead of 100MB datasets and resolved IO issues which impact test stability 2016-01-31 21:56:53 -05:00
Richard Miskin 8ef337b65e NIFI-1434 Prevent array index exception in PutEmail 
Signed-off-by: jpercivall <joepercivall@yahoo.com>
 2016-01-30 10:56:31 -05:00
Oleg Zhurakousky 0126800201 NIFI-1093 added support for handling non-required columns 
Mostly based on contribution from Daniel Cave
 2016-01-29 13:41:27 -05:00
Oleg Zhurakousky 807eb8de6d NIFI-1447 modified SimpleProcessLogger Modified SimpleProcessLogger to ensure that whenever is called it prints full stack trace. 
Signed-off-by: jpercivall <joepercivall@yahoo.com>
 2016-01-29 12:39:27 -05:00
jpercivall be27c07c7f NIFI-1382 Created FetchDistributedMapCache processor 2016-01-29 12:12:23 -05:00