Apache
/
nifi
mirror of https://github.com/apache/nifi.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
2,491 Commits 143 Branches 197 Tags 400 MiB
Commit Graph

1030 Commits

Author SHA1 Message Date
Oleg Zhurakousky 95dda1d920 NIFI-1748 removed un-used test data 
This closes #341.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-04-11 02:24:45 -04:00
Mike Moser 5625686ea4 NIFI-1746 avoid exceptions if configuration dir is a symbolic link 
Signed-off-by:  Joe Skora <jskora@apache.org>
 2016-04-11 02:05:48 -04:00
Joe Skora 696a12e4e7 Revert "NIFI-1748 removed un-used test data" 
This reverts commit 47cd9ff22c.
 2016-04-11 02:05:24 -04:00
Oleg Zhurakousky 47cd9ff22c NIFI-1748 removed un-used test data 
This closes #341.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-04-10 13:21:37 -04:00
mans2singh e3155a8a4b NIFI-1516 Provide AWS DynamoDB Delete/Put/Get processors 
This closes #224.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-04-08 11:23:04 -04:00
Zachary Jaffee 41583e6dc0 NIFI-400 remove deprecated processor EvaluateRegularExpression 
This closes #77

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-04-08 09:11:20 -04:00
Matt Gilman 5de40ccec3 NIFI-1553: 
- Implementing a file based authorizer.
- Providing an example authorizations files.
- Address comments from PR.
- This closes #330
 2016-04-07 16:28:42 -04:00
Matt Gilman 3f4ac3156c Revert "NIFI-1551:" 
This reverts commit c4d06f203d. Accidently merged wrong branch.
 2016-04-07 16:22:35 -04:00
Matt Gilman c4d06f203d NIFI-1551: 
- Starting to remove the AuthorityProvider.
- This closes #330
 2016-04-07 16:18:36 -04:00
Adam Lamar 2bcc31330c
NIFI-1180: Modify PutS3Object to enable encryption 
This closes #246.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
 2016-04-07 11:19:22 -07:00
ijokarumawak 9235a28f82 NIFI-1736 Move kafka.StreamScanner to nifi-utils. This closes #333 2016-04-07 07:37:28 -04:00
Bryan Bende 3adb45eafc NIFI-1630 This closes #332. Making TestPutUDP select an open port for testing to avoid conflicts 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-04-06 23:39:30 -04:00
Mark Payne 84b1c60d54 NIFI-1726: Addressed issue where we can run into an infinite loop if we are expiring data based on a timestamp instead of disk space usage and we have a file whose timestamp is exactly equal to our threshold for deletion 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-04-06 11:27:53 -04:00
Matt Brown b0cc6ae7e8 NIFI-1630 PutUDP processor created. 
Signed-off-by: Bryan Bende <bbende@apache.org>
 2016-04-06 11:04:01 -04:00
Aldrin Piri 3effc6cca5 NIFI-1689 Correcting TimeUnit conversion to be milliseconds instead of the incorrect microseconds and using historical modified times to avoid issues with second granularity of HFS+ 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-04-05 09:38:48 -04:00
Aldrin Piri 0e9e9b36d9 NIFI-1728 Specifying charsets for messages sent to/received from Kafka in Kafka processor and related tests to remedy failures in Windows environments. Specifying EOF as an int instead of a byte. 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-04-05 09:38:39 -04:00
Mike Moser 1c025dd4be NIFI-1714 use standard HttpClient cookie policy instead of default 
Signed-off-by: jpercivall <joepercivall@yahoo.com>
 2016-04-04 13:13:17 -04:00
Matt Gilman 9aa69b242e NIFI-1552: - Introducing the Authorizer API and additional components necessary for discovery and creation of configured instances. - Minor refactoring of existing Authority Provider API code/configuration to avoid some xsd naming conflicts. These components will be removed in NIFI-1551. - Introducing a number of the resource definitions that the Authorizer will make access decisions on. This list is likely not finalized may see some changes in NIFI-1554. - Address comments from PR. - This closes #318. 
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
 2016-04-04 11:47:43 -04:00
Mark Payne 1ac05266a5 NIFI-483: Use ZooKeeper's Leader Election to determine Primary Node. This closes #301 
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
 2016-04-04 11:47:08 -04:00
Mark Payne 0d3bd2c401 NIFI-1563: - Federate requests and merge responses from nodes instead of storing bulletins and stats at NCM - Updating UI to support restructured status history DTO. - Return 'Insufficient History' message if aggregate stats don't have enough data points, even if all nodes do (which can be the case if the node performing the aggregation has a different value for the 'nifi.components.status.snapshot.frequency' property than the other nodes) - Bug fixes; code cleanup; replicate requests to bulletin board endpoint - Refactored the <Component>StatusDTO objects into <Component>StatusDTO, <Component>StatusSnapshotDTO, Node<Component>StatusSnapshotDTO objects - Introducing endpoints for accessing individual component status. - Wiring up new endpoints and updated core. - Code clean up. - Starting to handling status merging of individual components. - Nodewise breakdown has been added to Processors but the remaining components still need to be updated. - Refactor so that System Diagnostics requests are replicated to nodes instead of the information being pulled from Heartbeats - Replicate request for counters instead of pulling them from heartbeats - Removed the getCounters / setCounters method from HeartbeatPayload - Implementing component specific endpoints. - Removing unused endpoints. - Supporting nodewise breakdown for system diagnostics and counters. - Updating DTOs to use more consistent naming. - Code clean up. - Addressing contrib issues. - Removed ProcessGroupStatus from HeartbeatPayload - Removing nodewise from the system diagnostics endpoint. Had included it for testing that option but did not intend for it to be committed. - Addressing comments in PR #294. - This closes #294 
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
 2016-04-04 11:42:39 -04:00
Matt Gilman 2de7f3f884 Updating versions to 1.0.0-SNAPSHOT. 2016-04-04 11:36:20 -04:00
Matt Gilman 16108467c1 NIFI-1695: 
- Removing hidden control characters that were affecting graph styles. This closes #311
 2016-04-04 11:26:55 -04:00
Bryan Bende 04c6830937 NIFI-1698 Improving customValidate in AbstractHadoopProcessor and HBaseClient service to not reload Configuration unless it changed. This closes #313 2016-04-04 10:14:05 -04:00
Stephen Harper e02c79975e NIFI-1686 - NiFi is unable to populate over 1/4 of AMQP properties from flow properties 
This closes #305
 2016-04-04 10:07:50 -04:00
Oleg Zhurakousky 25290cedc4 NIFI-1701 fixed StreamScanner, added more tests 
NIFI-1701 additional refactoring, clean up and more tests
 2016-04-04 09:55:16 -04:00
Bryan Bende 65b5c1a5f1 NIFI-1697 Ensuring FlowController appropriately wraps code with NarCloseable. This closes #312 2016-04-04 09:22:55 -04:00
Matt Burgess 65b26e6f41 NIFI-1691: Add Fetch Size property to QueryDatabaseTable 
This closes #307

Signed-off-by: Bryan Bende <bbende@apache.org>
 2016-03-29 10:51:30 -04:00
Oleg Zhurakousky c3d54ab724 NIFI-1684 This closes #308. fixed ZKClient connection leak 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-29 09:56:24 -04:00
Oleg Zhurakousky 9912f18de5 NIFI-1684 This closes #302. fixed random partitioner initialization 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-29 09:56:08 -04:00
Oleg Zhurakousky 1292581ec8 NIFI-1684 added penalization on failure 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-29 09:56:04 -04:00
Oleg Zhurakousky 8d960f5243 NIFI-1684 fixed NPE in PutKafka when retrieving key attribute bytes 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-29 09:56:04 -04:00
Oleg Zhurakousky e35c40b0fd NIFI-1684 fixed NPE, added tests 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-29 09:56:04 -04:00
Mark Payne 8000304e6a NIFI-1442: This closes #306. Use CircularFifoQueue instead of Set to store nodes' bulletins 
Joint effort by Toivo Adams from PR306 and and Mark Payne

Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-28 13:35:36 -04:00
Aldrin Piri e977729b56 NIFI-1634-rc2 prepare for next development iteration 2016-03-23 18:56:34 -04:00
Aldrin Piri 0b9bd20d31 NIFI-1634-rc2 prepare release nifi-0.6.0-RC2 2016-03-23 18:56:22 -04:00
Mark Payne e4b7e47836 Merge branch 'NIFI-1676' 2016-03-23 12:14:35 -04:00
Mark Payne b95a82f4a5 NIFI-1676: Do not allow Processor to be started if state is STOPPING; ensure that it is STOPPED 2016-03-23 11:31:20 -04:00
Mark Payne f7ecb47e29 Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/nifi 2016-03-23 11:24:28 -04:00
Mark Payne 9e39653192 Merge branch 'NIFI-1645' of https://github.com/olegz/nifi into NIFI-1645 2016-03-23 10:31:57 -04:00
Oleg Zhurakousky e0e00ff282 NIFI-1645 refactored PutKafka 
- used newest API available in 0.8.* version
- added PutKafka integration tests
- Kafka module code coverage is at 85%

NIFI-1645 polishing

NIFI-1645 PR comments round 1

NIFI-1645 PR comments round 2

NIFI-1645 change to use async Kafka producer

NIFI-1645 polishing

NIFI-1645 polishing

NIFI-1645 polishing

NIFI-1645 changed from java.util.Scanner to custom StreamScanner

NIFI-1645 polishing

NIFI-1645 final polish
 2016-03-23 10:31:34 -04:00
Oleg Zhurakousky a68f87f96e NIFI-1665 This closes #296. fixed GetKafka to reset consumer in case of timeout 
NIFI-1665 polishing

Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-22 21:48:47 -06:00
Bryan Bende 423b333b71 NIFI-1420 Fixing minor bugs in GetSplunk 
- Adding a Time Zone property so the Managed time ranges use the provided time zone when formatting the date strings
- Adding a Time Field Strategy property to choose between searching event time or index time
- Making the next iteration use previousLastTime + 1 ms to avoid overlap
- Fixing bug where GetSplunk incorrectly cleared state on a restart of NiFi
- This closes #299
 2016-03-22 15:23:42 -04:00
Matt Burgess bbbc77707f NIFI-1666: Fixed bug with EL evaluation in PutElasticsearch processor 2016-03-22 15:10:27 -04:00
Aldrin Piri 4babd067c1 NIFI-1664 Preferring System.nanoTime to System.currentTimeMillis and providing explicit handling of timestamps for files in those tests that are testing other attributes of the ListFile process besides timestamp which could lead to erroneous transmissions depending on exactly when files were created. 
Adding unsalted_128_raw.enc and salted_128_raw.enc to the list of exclusions for the RAT plugin which caused issues in the Windows environment for contrib-checks.

This closes #297.
 2016-03-22 14:13:37 -04:00
Mark Payne 736896246c NIFI-1649: Addressed bug in SplitText that resulted in improper handling of input data if Remove Trailing Newlines is true and there is a FlowFile that is only made up of newlines 2016-03-21 13:26:32 -04:00
Mark Payne 38c782c30b NIFI-1650: Ensure that proper size and offset are specified for Content Claim when viewing FlowFiles still in queue 2016-03-21 10:30:24 -04:00
Mark Payne d3578a7c03 NIFI-1650: Ensure that we seek to the appropriate offset within the Content Claim when downloading content of a FlowFile 2016-03-21 10:30:23 -04:00
trkurc 9cde92da16 NIFI-1656 Added locale support to ConvertAvroSchema and fixed locale problems in unit tests 
Reviewed by Joe Witt (joewitt@apache.org). This closes #292
 2016-03-21 01:44:46 -04:00
trkurc 3921ab827b NIFI-1651 unit tests work on windows. removed contributor name/package from data and schema 
Reviewed by Aldrin Piri (aldrin@apache.org) and Mike Moser (mosermw@apache.org). This closes #291
 2016-03-20 15:01:57 -04:00
Oleg Zhurakousky bb738f978a NIFI-1647 This closes #288. fixed validators and config resolution 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-18 10:11:00 -04:00
Joey Frazee 3a4546c08a NIFI-1617 Add source filename metadata to IdentifyMimeType 
Signed-off-by: Matt Burgess <mattyb149@apache.org>
 2016-03-18 00:23:16 -04:00
Oleg Zhurakousky 8f40d2b181 NIFI-1571 initial commit of SpringContext support 
Reviewed by Tony Kurc (tkurc@apache.org) and Joe Witt (joewitt@apache.org). This closes #271
 2016-03-17 23:43:47 -04:00
Michal Klempa e12e7a55b7 NIFI-1562 ExecuteStreamCommand and ExecuteProcess now support empty command line arguments 
This closes #247

Signed-off-by: Matt Burgess <mattyb149@apache.org>
 2016-03-17 23:15:12 -04:00
Pierre Villard e7a254f78e NIFI-1332 Fixed HTTP-204 handling + unit test 
This closes #259.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-03-17 23:03:51 -04:00
Aldrin Piri b44b177039 NIFI-1605 Adjust documentation and resources to reflect nifi.provenance.repository.rollover.time default 
This closes #263

Signed-off-by: Matt Burgess <mattyb149@apache.org>
 2016-03-17 22:14:24 -04:00
Oleg Zhurakousky 138c71de82 NIFI-1464 This closes #287. removed e.printStackTrace() from onTrigger in StandarProcessScheduler 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-17 17:18:56 -04:00
Bryan Bende 8773ec3d3e NIFI-1488 Refactoring HBase Kerberos support 
- Storing UGI so we can support multiple HBaseClientServices with different configs
- Creating nifi-hadoop-utils to hold utility code shared between HDFS and HBase processors
- Incorporating KerberosProperties into existing hadoop processors

This closes #281
 2016-03-17 17:12:57 -04:00
Matt Burgess 94cf1beeba NIFI-901: Fixed unit test to verify IP address of Cassandra contact point This closes #286 
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
 2016-03-17 13:53:07 -04:00
Andy LoPresto 76f2d5702f NIFI-1274 Added Kerberos authentication mechanism. 
NIFI-1274 Cleaned up TODO statements. (+3 squashed commits)
Squashed commits:
[fd101cd] Removed logic to check for presence of services to determine if token support is enabled when username/password authentication is enabled (Kerberos also requires tokens).
[c2ce29f] Reverted import changes to RulesResource.java.
[c269d72] Added Kerberos authentication mechanism.

Moved Kerberos service wiring from XML to Java to handle scenario where admin has not configured Kerberos (previously threw NullPointerException in FileSystemResource constructor). (+15 squashed commits)
Squashed commits:
[09fc694] Added Kerberos documentation to Admin Guide.
[ecfb864] Cleaned up unused logic.
[157efb3] Added logic to determine if client certificates are required for REST API (login, anonymous, and Kerberos service authentication all disabled).
Cleaned up KerberosService by moving logic to NiFiProperties.
[5438619] Added documentation for Kerberos login-identity-providers.xml.
[3332d9f] Added NiFi properties for Kerberos SSO.
[b14a557] Fixed canvas call to only attempt Kerberos login if JWT not present in local storage.
Added logic to handle ticket validation failure in AccessResource.
Changed wiring of Kerberos service beans to XML in nifi-web-security-context.xml for consistency.
[c31ae3d] Kerberos SPNEGO works without additional filter (new entry endpoint accepts Kerberos ticket in Authorization header and returns JWT so the rest of the application functions the same as LDAP).
[98460e7] Added check to only instantiate beans when Kerberos enabled to allow access control integration tests to pass.
[6ed0724] Renamed Kerberos discovery method to be explicit about service vs. credential login.
[ed67d2e] Removed temporary solution for Rules Resource access via Kerberos ticket.
[c8b2b01] Added temporary solution for Rules Resource access via Kerberos ticket.
[81ca80f] NIFI-1274 Added KerberosAuthenticationFilter to conduct SPNEGO authentication with local (client) Kerberos ticket.
Added properties and accessors for service principal and keytab location for NiFi app server.
Added KAF to NiFiWebApiSecurityConfiguration.
Added AlternateKerberosUserDetailsService to provide user lookup without dependency on extension bundle (nifi-kerberos-iaa-provider).
Added dependencies on spring-security-kerberos-core and -web modules to pom.xml.
[0605ba8] Added working configuration files to test/resources in kerberos module to document necessary config. This version requires the user to enter their Kerberos username (without realm) and password into the NiFi login screen and will authenticate them against the running KDC.
Also includes a sample keystore and root CA public key for configuring a secure instance.
[49236c8] Added kerberos module dependencies to nifi/pom.xml and nifi-assembly/pom.xml.
Added default properties to login-identity-providers.xml.
[928c52b] Added nifi-kerberos-iaa-providers-bundle module to nifi/pom.xml.
Added skeleton of Kerberos authenticator using Spring Security Kerberos plugin.
This closes #284

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
 2016-03-17 13:10:09 -04:00
Bryan Bende b188b0abd6 NIFI-1420 Fixing bug where a FlowFile should route to failure when PutSplunk can't createa connection, defaulting PutSplunk to TCP 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-16 17:43:28 -04:00
Mark Payne a7b97419e5 NIFI-1626: Throw an Exception proactively if too much state is attempting to be stored via ZooKeeperStateProvider 
NIFI-1626: Updated State Management section of Developer Guide

Signed-off-by: jpercivall <joepercivall@yahoo.com>
 2016-03-16 16:12:00 -04:00
Matt Burgess 98395de74f NIFI-1575: Add QueryDatabaseTable processor 2016-03-16 15:55:57 -04:00
Oleg Zhurakousky 148b4497b4 NIFI-1629 This closes #282. downgraded Kafka back to 0.8 - added context.yield to PutKafka - added lifecycle hooks to defend from Kafka deadlocks 
NIFI-1629 changd thread pool implementation in Get/PutKafka

Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-16 15:38:46 -04:00
Mark Payne 68cfc8c612 NIFI-1632: Fixed NPE that occurs if a capturing group is optional and not matched; updated docs to illustrate how this is handled 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-16 15:19:17 -04:00
Matt Burgess 0f61079300 NIFI-901: Add QueryCassandra and PutCassandraQL processors 
This closes #237

Signed-off-by: jpercivall <joepercivall@yahoo.com>
 2016-03-16 14:09:15 -04:00
Oleg Zhurakousky e83429a08c NIFI-1631 This closes #283. increased timeouts on test 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-16 11:06:21 -04:00
mans2singh 19bc5ba999 NIFI-1495 Adding support for AWS Kinesis Firehose 
This closes #213

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-03-16 09:56:08 -04:00
joewitt 8ed64c9083 NIFI-1420 added missing properties to block javadocs/sources generation on nar 2016-03-15 21:43:06 -04:00
joewitt 2b0a0c0597 NIFI-614 added necessary pom element to skip javadocs/sources for nar pom itself in jms proc nar. 2016-03-15 21:40:08 -04:00
joewitt d78c7faf14 NIFI-614 added necessary pom element to skip javadocs/sources for nar pom itself. 2016-03-15 21:35:50 -04:00
Oleg Zhurakousky 812da19cad NIFI-614 Added initial support for new style JMS 
NIFI-614 finalized JMSConnectionFactoryProvider ControllerService

NIFI-614 finalized implementation of both Processors and ControllerService

NIFI-614 added initial documentation

NIFI-614 addressed PR comment with unused import and squashed

NIFI-614 added @OnDisabled method

NIFI-614 changed POMs to 0.6

NIFI-614 removed local .gitignore

NIFI-614 added support for parsing Tibco URL

NIFI-614 removed setting of jms message id

NIFI-614 addressed PR comments, fixed tests

NIFI-614 addressed latest PR comments

NIFI-614 second round of PR comments addressed

NIFI-614 3rd round of PR comments addressed

NIFI-614 finalizing on PR comments

NIFI-614 more PR comments

This closes #222

Signed-off-by: jpercivall <joepercivall@yahoo.com>
 2016-03-15 15:48:03 -04:00
Oleg Zhurakousky cfeebfe7c7 NIFI-1464 fixed OnScheduled invocation to pass ProcessContext 
NIFI-1464 removed unused imports

Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-15 12:54:13 -04:00
Mark Payne a4a7d53769 NIFI-1627: Do not call context.yield for funnels and local ports; use batch size of 100 instead of 10 for local ports and funnels 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-15 12:54:03 -04:00
trkurc f22db218e3 NIFI-1481 fixed a couple bugs in bash script for cygwin and when JAVA_HOME not set 
Reviewed by Aldrin Piri (aldrin@apache.org). This closes #277
 2016-03-14 22:40:25 -04:00
puspendu.banerjee@gmail.com 76ce3f7520 NIFI-1481 support for dumping environment using nifi.sh env 
Reviewed and amended (Amendments reviewed by Aldrin Piri (aldrin@apache.org)) by Tony Kurc (tkurc@apache.org). This closes #218
 2016-03-14 19:24:21 -04:00
Oleg Zhurakousky c7df94e00f NIFI-1464 life-cycle refactoring part-2 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-14 17:13:55 -04:00
Mark Payne 56f79e1e85 NIFI-1588: Reworked how ListHDFS store state so that only 2 timestamps are stored and no paths 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-14 15:59:24 -04:00
Mark Payne 8c488d7e8e NIFI-1587: Always poll state from State Manager when running ListHDFS instead of relying on local state over the cluster state 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-13 14:36:36 -04:00
Mark Payne 5a8b2cf7f1 NIFI-1606: Run the onComponentRemoved logic of state providers in a background thread 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-13 14:08:24 -04:00
Mark Payne ce068ffc6c NIFI-1612: Do not ignore Remote Process Group's timeout configuration 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-13 13:40:45 -04:00
joewitt 730d999d4b NIFI-627 minor formating adjustments and removed exception based handling logic. This closes #268. This closes #274. 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-13 13:28:48 -04:00
Mike Moser 0cb4047add NIFI-627 removed flowfile penalization which could skew behavior when processor's Time Duration was less than Penalty Duration, improved over throttle penalization NIFI-990 corrected failure path NIFI-1329 refactored using FlowFileFilter to avoid repeatedly returning flowfiles to the input queue, producing misleading stats and excessive Tasks/Time used 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-13 13:28:43 -04:00
Mark Payne 7400b6f7c5 NIFI-1622: Ensure that the Nar Context Class Loader is used when calling Processor lifecycle methods 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-03-12 20:06:23 -05:00
Mark Payne a85c119533 Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/nifi 2016-03-11 12:55:23 -05:00
Oleg Zhurakousky 59fac58c96 NIFI-1464 ensured that OnUnscheduled is treated the same as OnScheduled 
NIFI-1464 polished javadocs, error messages and docs
 2016-03-11 12:54:50 -05:00
Oleg Zhurakousky 1c22f3f012 NIFI-1464 refactored the latest commit 2016-03-11 12:54:50 -05:00
Oleg Zhurakousky 48af0bfbc5 NIFI-1464 addressed latest PR comments 
NIFI-1464 polishing
 2016-03-11 12:54:50 -05:00
Oleg Zhurakousky f53f45def3 NIFI-1464 addressed PR comments from @apiri and @markap14 2016-03-11 12:54:50 -05:00
Oleg Zhurakousky 0c5b1c27f2 NIFI-1464, Refactored Processor's life-cycle operation sequence 
* Simplified and cleaned StandardProcessScheduler.start/stopProcessor methods
* Added stop/start operations to ProcessorNode.
* Removed unnecessary synchronization blocks related to ScheduledState in favor of enforcing order and idempotency via CAS operations. Those synchronization blocks were causing intermittent deadlocks whenever @OnScheduled blocks indefinitely.
* Added support for stopping the service when @OnScheduled operation hangs.
* Fixed the order of life-cycle operation invocation ensuring that each operation can *only* be invoked at the appropriate time
* Removed unnecessary locks from StandardProcessNode since Atomic variables are used.
* Removed calls to @OnStopped from ContinuallyRunningProcessTask while ensuring that procesor's full shut down in implementation of StandardProcessorNode.stop() method.
* Removed dead code
* Added comprehensive tests suite that covers 95% of Processor's life-cycle operations within the scope of FlowController, StandardProcesssScheduler and StandardProcessNode
* Improved and added javadocs on covered operations with detailed explanations.
 2016-03-11 12:54:50 -05:00
Matt Burgess e1cf37fb89 NIFI-1619: Fix Elasticsearch processor bug when flow file missing ID attribute 
This closes #269

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-03-11 12:01:28 -05:00
Bryan Bende 1373bf6725 NIFI-899 Rewrite of ListenUDP to use new listener framework, includes the following changes: 
- Adding Network Interface property to AbstractListenEventProcessor and ListenSyslog
- Adding sending host and sending port to DatagramChannelDispatcher
- Creation of common base class AbstractListenEventBatchingProcessor
- Refactor of ListenUDP, ListenTCP, and ListenRELP to all extend from AbstractListenEventBatchingProcessor
- Changing DatagramChannelDispatcher, socket handlers, and RELP handler to use offer() when queueing instead of put(), and log an error if the offer failed

This closes #266
 2016-03-11 09:07:05 -05:00
Pierre Villard 8f0116544a NIFI-1491 Deprecated existing method and added new one throwing eception 
Reviewed and amended (to add javadoc @deprecation link and to carry forward other deprecated methods) by Tony Kurc (tkurc@apache.org). This closes #227
 2016-03-10 22:15:13 -05:00
Michal Klempa 784f2a2c20 NIFI-1518 InferAvroSchema note has an option to set CSV delimiter 
Reviewed (and amended based on acknowledgement in PR review) by Tony Kurc (tkurc@apache.org). This closes #235.
 2016-03-10 21:36:31 -05:00
Mark Payne d3367a7dc3 nNFI-1618: Updated unit test to use 30,000 FlowFiles to test dropping of swapped out FlowFiles intead of using 210,000. Thiss unit test creates a lot of objects and when reducing the number of FlowFiles generated to 30K instead of 210K, all appears to work okay 2016-03-10 16:01:29 -05:00
Pasqualino Ferrentino 73e168e954 NIFI-1003 A relationship can be auto-terminable. In this case the processor will auto-terminate the relationship and allow the user to run it even if he does not connect those relationships and does not terminate them. 
This closes #217

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-03-10 14:56:46 -05:00
Aldrin Piri da234abd76 NIFI-1488 Adjusting unused imports and adding license exclusions to RAT plugin for build. 2016-03-10 08:58:12 -05:00
Mark Payne 2839a2f215 NIFI-15: Address issue where incomplete swap files can result in continually attempting to swap in data without ever being successful 
Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-03-09 23:46:53 -05:00
Jeff Lord 7f15626af5 NIFI-1488: Add HBase Kerberos Support with UGI 
Added `kerberos-principal` and `kerberos-keytab` properties
to the HBase service.
 2016-03-09 15:51:30 -05:00
Mark Payne a2164136db Merge branch 'NIFI-1573' of https://github.com/rpmiskin/nifi into NIFI-1573 2016-03-09 09:40:37 -05:00
Mark Payne 0f9b55afb6 Merge branch 'nifi-1509' of https://github.com/mans2singh/nifi into NIFI-1509 2016-03-09 08:19:45 -05:00
mans2singh 1211996d35 updated attributes using hashmap rather than individually 2016-03-08 21:12:49 -08:00
Mark Payne 9874d35b6b NIFI-1600: Ensure that if we move a RemoteProcessGroup, it also updates the Parent Process Group of the ports 2016-03-08 12:09:45 -05:00
Oleg Zhurakousky 998ac1c830 NIFI-1534 removed SupportsBatching annotation from PutKafka 
Signed-off-by: Matt Burgess <mattyb149@gmail.com>

This closes #249
 2016-03-08 10:03:27 -05:00
Bryan Bende 6f5fb59479 NIFI-1420 Adding Splunk bundle containing PutSplunk, and GetSplunk, and adding a ListenTCP processor to standard processors. Refactored internal code from PutSyslog to create a generic AbstractPutEventProcessor which PutSplunk extends from. 
This closes #233
 2016-03-07 18:21:17 -05:00
Oleg Zhurakousky 4ce7b679e1 NIFI-1595 fixed ReflectionUtils to honor bridge methods 
Refactored and simplified ReflectionUtils while at it
Added ReflectionUtilsTest

This closes #260.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
With adjustments to formatting and whitespace.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-03-07 15:27:35 -05:00
Matt Burgess a7e6820fd9 NIFI-1596: ExecuteSQL Error Creating Schema 2016-03-07 10:42:21 -05:00
Bryan Bende 19e53962ca NIFI-1579 Performance improvements for ListenSyslog which include removing an unnecessary yield and exposing a configurable size for the internal queue used by the processor, changing ListenSyslog to use a 20ms poll and use a long poll when batching, also including same improvements for ListenRELP 2016-03-04 09:17:45 -05:00
Mark Payne 6776060ac8 NIFI-1577: Close any streams that are left open for 'append' when the session is checkpointed, rather than waiting for it to be committed 2016-02-29 16:02:26 -05:00
Richard Miskin 738236a41d NIFI-1573 Allow programmatic access to a Processor's name 2016-02-27 15:18:22 +00:00
Aldrin Piri 58e50ae39d NIFI-1511 Incorporating Groovy unit tests as part of the main build and providing explicit compile scope on the Groovy dependency for the scripting bundle. This closes #220 
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
 2016-02-26 10:25:45 -05:00
Mark Payne 28c2a3e5a6 NIFI-1256: Added a 'Satisfies Expression' option for Matching Strategy of RouteText 
Signed-off-by: jpercivall <joepercivall@yahoo.com>
 2016-02-25 15:59:57 -05:00
Tony Kurc c7e24c7569 NIFI-1513: fixed some easy to fix errors 
Addressing checkstyle issue.

This closes #221

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
 2016-02-25 15:21:40 -05:00
Jeremy Dyer 32f476aaa7 NIFI-1357 Add Snappy compression to "CompressContent" Processor 
This closes #164.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-02-25 12:47:01 -05:00
Matt Gilman 0d13de0cf3 NIFI-1539: - Comparing octet stream content type by using starts with and ignores case. 2016-02-25 10:13:07 -05:00
Sönke Liebau fc92441981 NIFI-1539 - Add normalization of content type for content viewing 
Add code to ContentViewerController to strip content type of any trailing parameters and lowercase the type and subtype.

Added function to ViewableContent to enable retrieving the original value of the content type if needed.

This closes #242

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
 2016-02-25 10:12:45 -05:00
Mark Payne 6af108c0ca NIFI-1557: Ensure that Reporting Tasks & controller services are ordered by ID before fingerprinting 2016-02-23 18:02:58 -05:00
mans2singh 6138b0332e corrected contrib failure 2016-02-22 20:12:24 -08:00
mans2singh 50da696cda updated capability documentation based on comments 2016-02-22 19:56:12 -08:00
James Wing 24a77755de NIFI-1548 Fixing Controller Service Usage Button. This closes #245 
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
 2016-02-22 21:09:01 -05:00
Joe Skora 5ee83574d8 NIFI-1545: Adjust integration tests ITDeleteS3Object, ITFetchS3Object, and ITPutS3Object to account for AWS new proxy properties. Correct typo in AbstractS3Processor.createClient() log message. 
Reviewed by Tony Kurc (tkurc@apache.org). This closes #243
 2016-02-22 19:16:55 -05:00
Oleg Zhurakousky a87230358d NIFI-1536 This closes #241. ensured that UserGroupInformation is re-initialized on each call to 'AbstractHadoopProcessor.resetHDFSResources(..)' 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-02-22 17:23:53 -05:00
Matt Gilman e7676ffae5 NIFI-1546: - Ensuring that the data reference query is not null before attempt to extract a cluster node id from it. 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-02-22 12:39:37 -05:00
Mark Payne 122cba0ee7 NIFI-1527: Fixed issue that resulted in resource claims' claimant count getting incremented twice on restart 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-02-22 12:36:08 -05:00
Joe Skora 4c5263220c NIFI-1542: Update how PutS3Object handles a 403 AccessDenied response to a S3 ListMultipartUploads request. 
* Change log message from error to warning and added note about S3 permissions.
 * Advance the ageoff check time by the check interval as if the request succeeded to prevent re-checking on every upload when permission does not exist.

Reviewed by Tony Kurc (tkurc@apache.org) (with minor amendments for whitespace and error message consistency). This closes #240
 2016-02-21 18:11:23 -05:00
Bryan Bende 9576f41603 NIFI-1529 Forcing hadoop-libraries-nar to version 4.2.5 of httpclient and httpcore since that is what hadoop-client 2.6.2 normally brings in - Removing dependencies from the nar pom that should be transitively brought in by hadoop-client - Updating NOTICE files in hadoop-libraries-nar and nifi-assembly according to dependency changes 
This closes #236
 2016-02-19 16:35:49 -05:00
Matt Gilman a8edab2e79 NIFI-1497: - Introducing a one time use password service for use in query parameters when accessing UI extensions and downloading resources. - Using one time use tokens when accessing ui extensions and downloading resources. - Ensuring appropriate roles when accessing component details through the web context for custom UIs. - Addressing typo in class name. - Ensuring appropriate roles when accessing content through the content access. - Code clean up. - Refactoring some basic scripts for accessing JWT tokens so UI extensions can reuse common functionality. 
Signed-off-by: Bryan Bende <bbende@apache.org>
 2016-02-19 10:54:53 -05:00
joewitt 86ab4428f0 NIFI-1520 by default skip javadoc and source jar generation in nars and wars 
Reviewed by Tony Kurc (tkurc@apache.org). This closes #234
 2016-02-18 18:45:26 -05:00
Mark Payne 3bb18b9653 NIFI-1527: Ensure that we increment Claimant Counts for content claims that are referenced by Swapped-Out FlowFiles on restart of nifi 2016-02-18 16:25:57 -05:00
mans2singh bdc1ef1f8c corrected discription 2016-02-17 21:36:53 -08:00
puspendu.banerjee@gmail.com 1c73d9090a NIFI-1501 : Test Monitor Activity has spurious failures. 
Added Timeout for the test. So, in case something goes South, threshold will never be more that that Timeout period.

Reviewed with amendments for whitespace by Tony Kurc (tkurc@apache.org). This closes #225
 2016-02-17 21:55:10 -05:00
Pierre Villard 502aebc66a NIFI-1063 Fixed exception logging 
Fixed logging of an exception in PutHDFS

Reviewed by Tony Kurc (tkurc@apache.org). This closes #226
 2016-02-17 20:59:31 -05:00
Michal Klempa c2998a71ec NIFI-1514 Fixed ExecuteStreamCommand when expressions in command arguments are used, contents are not split by command separator 
This closes #223.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-02-17 17:48:23 -05:00
Tony Kurc 6c2ba997a0 Merge branch 'NIFI-1379-RC3' 2016-02-16 20:13:41 -05:00
mans2singh b921bf602e updated comment 2016-02-16 13:33:48 -08:00
mans2singh ad39b1c41f renamed class and removed unused imports 2016-02-16 13:25:28 -08:00
mans2singh 36f1caad33 Adding support for using an HTTP proxy with AWS processors. 
This closes #209

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-02-16 14:51:06 -05:00
Pierre Villard ef0018cf66 NIFI-1282 This closes #228. Fixed error message when attempting to start a disabled port 
Fixed error message when attempting to start a disabled port

Signed-off-by: joewitt <joewitt@apache.org>
 2016-02-16 09:38:56 -05:00
Tony Kurc ad73b5c9d9 NIFI-1379-RC3 prepare for next development iteration 2016-02-12 17:28:10 -05:00
Tony Kurc 8309dba80b NIFI-1379-RC3 prepare release nifi-0.5.0-RC3 2016-02-12 17:27:59 -05:00
Tony Kurc cae5b109c0 NIFI-1507: Update bouncycastle license reference 2016-02-12 16:58:26 -05:00
mans2singh 528c4bc591 nifi-1509 catch recoverable service exceptions explicity 2016-02-12 09:22:41 -08:00
mans2singh cd05bedda8 Nifi-1509 - commented out integration test 2016-02-11 21:14:28 -08:00
mans2singh 180a90d12b first commit for aws lambda 
corrected attributes of flow file

added shutdown + provenence calls

minor formatting and unused imports correction

removed unused property

updated to populate exception attributes in flow file

updated write attributes
 2016-02-11 20:59:54 -08:00
joewitt f44eb643dd NIFI-1461 added run.as filtered property for RPM and removed extraneous properties 2016-02-10 21:21:19 -05:00
Matt Gilman 778229eb52 NIFI-1499: - Ensuring the universal keystroke capture is available to UI extensions. 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-02-10 19:15:48 -05:00
Aldrin Piri 1a512cd1e6 NIFI-1484 Making use of timestamps at various points of execution to provide listing of all but the latest files which are held until a subsequent execution. 
Correcting nifi-amqp-nar bundle's pom description.

This closes #212.
 2016-02-10 10:25:47 -05:00
Matt Gilman f4487dd5f6 NIFI-1492: - Limiting the amount of state entries returned to a client. - Code clean up. 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-02-09 20:48:56 -05:00
Oleg Zhurakousky 8a05f6880a NIFI-1493 This closes #211. removed Executor from InvokeScriptedProcessor 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-02-09 20:31:57 -05:00
Aldrin Piri 1c03fc7871 Removing unused import to remedy checkstyle error on previous documentation update. 2016-02-09 16:13:16 -05:00
Aldrin Piri a33289393d Correcting documentation on SnippetResource#createSnippet to reflect that it is creating a snippet. 2016-02-09 15:33:46 -05:00
ledor473 eb6f9f0fec NIFI-1423 Allow to penalize FlowFiles that are routed to No Retry relationship 
This closes #183

Signed-off-by: jpercivall <joepercivall@yahoo.com>
 2016-02-08 11:20:08 -05:00
Aldrin Piri 2673370cba NIFI-1483 Correcting logic in terms of when local persistence files are removed during the migration process. 
Reviewed by Tony Kurc (tkurc@apache.org). This closes #206
 2016-02-07 17:40:12 -05:00
joewitt ea5818c398 NIFI-1486 added commons io and lang to nifi-amqp-nar NOTICE 
Reviewed by Tony Kurc (tkurc@apace.org). This closes #208
 2016-02-07 17:27:53 -05:00
Tony Kurc 5be83166ee NIFI-1485: Updated 'header' section of NiFi NOTICE files 2016-02-07 13:52:00 -05:00
Tony Kurc 303f8eabf1 NIFI-1379: Move to 0.5.0-SNAPSHOT, add tkurc code signing key to keys 2016-02-06 08:49:48 -05:00
Andy LoPresto b407379670 NIFI-1257 Resolved legacy compatibility issue with NiFi legacy KDF salt length dependent on cipher block size. 
Replaced screenshot for NiFiLegacy salt encoding.
Added description of legacy salt length determination in admin guide.
Added logic for NiFiLegacyCipherProvider to generate and validate salts of the length determined by the cipher block size.
Changed EncryptContent to default to Bcrypt KDF.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-02-05 23:38:58 -05:00
Matt Gilman 7314af6177 NIFI-259: - Addressing issues that arose from a merge conflict. 
Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-02-05 17:47:56 -05:00
Aldrin Piri 4df6512126 Merge branch 'NIFI-259' 2016-02-05 14:09:29 -05:00
Aldrin Piri 16f1852453 NIFI-259 Updating logger message for AbstractListProcessor to reference State Manager in onTrigger instead of the distributed cache. 2016-02-05 12:55:06 -05:00
Andy LoPresto 498b5023ce NIFI-1257 NIFI-1259 
Added utility method to return the maximum acceptable password length for PBE ciphers on JVM with limited strength crypto because BC implementation is undocumented (based on empirical evidence).
Updated EncryptionMethod definitions to accurately reflect need for unlimited strength crypto according to algorithm key length.
Added processor logic to invoke keyed cipher.
Added EncryptContent processor property for raw hex key (always visible until NIFI-1121).
Added validations for KDF (keyed and PBE) and hex key.
Added utility method to return list of valid key lengths for algorithm.
Added description to allowable values for KDF and encryption method in EncryptContent processor.
Added IV read/write to KeyedCipherProvider and changed from interface to abstract class.
Added salt read/write logic to NifiLegacy and OpenSSL cipher providers.
Changed RandomIVPBECipherProvider from interface to abstract class.
Updated strong KDF implementations.
Renamed CipherFactory to CipherProviderFactory.
Added unit test for registered KDF resolution from factory.
Updated default iteration count for PBKDF2 cipher provider.
Implemented Scrypt cipher provider.
Added salt translator from mcrypt format to Java format.
Added unit tests for salt formatting and validation.
Added surefire block to groovy unit test profile to enforce 3072 MB heap for Scrypt test.
Added local Java implementation of Scrypt KDF (and underlying PBKDF2 KDF) from Will Glozer.
Defined interface for KeyedCipherProvider.
Implemented AES implementation for KeyedCipherProvider.
Added Ruby script to test/resources for external compatibility check.
Added key length check to PBKDF2 cipher provider.
Changed default PRF to SHA-512.
Added salt and key length check to PBKDF2 cipher provider.
Added utility method to check key length validity for cipher families.
Added Bcrypt implementation.
Implemented PBKDF2 cipher provider.
Added default constructor with strong choices for PBKDF2 cipher provider.
Implemented NiFiLegacyCipherProvider and added unit tests.
Added key length parameter to PBKDF2 cipher provider.
Added PRF resolution to PBKDF2 cipher provider.
Added RandomIVPBECipherProvider to allow for non-deterministic IVs.
Added new keyed encryption methods and added boolean field for compatibility with new KDFs.
Added CipherFactory.
Improved Javadoc in NiFi legacy cipher provider and OpenSSL cipher provider.
Added KeyedCipherProvider interface.
Added OpenSSL PKCS#5 v1.5 EVP_BytesToKey cipher provider and unit test.

This closes #201.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-02-04 19:40:55 -05:00
jpercivall 29fb9c9393 NIFI-259 Corrected some logic in AbstractListProcessor regarding being elected primary node and improved readability in GetHBase a smidge 2016-02-04 16:07:18 -05:00
Matt Gilman 6f4c3db186 NIFI-259: 
- Fixing the line height to ensure the multiline ellipsis is consistent across browsers.
 2016-02-04 08:59:26 -05:00
puspendu.banerjee@gmail.com 0690aee452 NIFI-1460 - Performance improvement. test Timeout Mitigation. less IO, less dependency on /dev/(u)random. 
- Incorporated comments for UUID format conformance & optimized.
          - polished to satisfy this test-case. we don't need incrementAndGet
Reviewed by Tony Kurc (tkurc@apache.org). This closes #202
 2016-02-03 21:50:24 -05:00
Joe Skora 769b044e30 NIFI-1107: Re-integrate Multipart Upload changes into PutS3Object. 1. add Multipart upload logic to allow resuming an upload after process/instance restart, 2. add local state management to track the part uploaded for a flowfile, 3. add configurable AWS S3 state management to abort orphaned uploads, and 4. adapt to IT test naming. 
Pull request updates: * Cleanup property description to reflect processor details, not code specifics. * Only resume an upload from local state if uploadID still exists in S3, otherwise delete local state and start a new upload. * Check that local state from was actually created before trying to delete it.

Re-integrate Multipart Upload changes into PutS3Object.
1. add Multipart upload logic to allow resuming an upload after process/instance restart,
2. add local state management to track the part uploaded for a flowfile,
3. add configurable AWS S3 state management to abort orphaned uploads, and
4. adapt to IT test naming.

Fixed test issues
 * added mock client for testing local state without matching S3 state,
 * updated regex for multipart etag pattern.

Reviewed by Tony Kurc (tkurc@apache.org). This closes #192
 2016-02-03 20:54:32 -05:00
jpercivall 88d4d2ce5f NIFI-259 Corrected GetHttp state managment and added a new unit test 2016-02-03 20:04:39 -05:00
Matt Gilman 55b77fe03b NIFI-1223: - Removing conf directory from unit test. 
Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-02-03 16:05:38 -05:00
Matt Burgess 478226451c NIFI-210: Add support for multiple module paths for scripting 2016-02-03 10:41:47 -05:00
Matt Gilman 7711106d62 NIFI-259: - Adding additional documentation for embedded ZK. 
Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-02-03 10:13:19 -05:00
Aldrin Piri 8a668fd344 NIFI-259: Fixed bug that caused StateProvider.replace to return true if the value had never been set, instead of false. Fixed typos in administration-guide 
Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-02-02 23:03:31 -05:00
joewitt 2d6fec5fa9 NIFI-865 mild cleanup for LICENSE NOTICE 2016-02-02 21:13:28 -05:00
Oleg Zhurakousky dc8b62c3a6 NIFI-865 Added processors to Get and Put to/from AMQP-based messaging systems 
This closes #200

Signed-off-by: Bryan Bende <bbende@apache.org>
 2016-02-02 20:57:10 -05:00
Matt Burgess 75af3a2ebe NIFI-1275: Added FetchElasticsearch processor and support for secure clusters 
This closes #180.

Signed-off-by: jpercivall <joepercivall@yahoo.com>
 2016-02-02 18:21:31 -05:00
scarpacci 0c137bc22d Added path home property and unit test to elasticsearch processor in support of the node client 2016-02-02 17:26:40 -05:00
Matt Burgess 943d0a6e53 Adding to PutElasticsearch - validators, unit tests, etc. 2016-02-02 17:26:40 -05:00
scarpacci e6cfcf40d0 Initial commit for the elasticsearch bundle to Nifi 2016-02-02 17:26:39 -05:00
Matt Gilman 8cf34c3ea5 NIFI-1457: 
- Update heartbeat bean when primary state changes.
 2016-02-02 13:16:10 -05:00
Matt Burgess 77ab5d3684 NIFI-210: This closes #185. Add ExecuteScript and InvokeScriptedProcessor 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-02-01 21:09:06 -05:00
Bryan Bende 1e56de9521 NIFI-1417 Exposing several connection settings on the Solr processors to help deal with timeouts and adding an @OnStopped method to close the SolrClient which didn't appear to be happening before 
This closes #194
 2016-02-01 18:42:00 -05:00
Mark Payne 35d2b921ea NIFI-259: Distinguish between unable to communicate with ZK and 'bad version' when performing 'replace' method of ZooKeeperStateProvider 
Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-02-01 16:29:12 -05:00
Aldrin Piri a931e72787 NIFI-259: Expose the ZK exception code in the event of a KeeperException within the ZooKeeperStateProvider. 2016-02-01 16:29:12 -05:00
Aldrin Piri 447e401912 NIFI-259: Extending the StateProvider interface to provide a getSupportedScopes method and implemented this based on the capabilities of each of its implementations. Used supported scope to evaluated configurations at startup and prevent issues when trying to make use of state 2016-02-01 16:29:12 -05:00
Aldrin Piri 257eca9c46 NIFI-259 Removing StateProviderScope and refactoring to use Scope in its place. 2016-02-01 16:29:11 -05:00
Aldrin Piri f2c366cf9c NIFI-259: Correcting path to the ZooKeeperStateProvider. 2016-02-01 15:26:27 -05:00
Matt Gilman 72c8467b9f NIFI-259: 
- Addressing typo in documentation.
- Minor tweaks to admin guide.
- Adding support to stand up a ZooKeeperServer when a quorum peer is not distributed (ie supporting both embedded standalone and cluster).

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-02-01 14:07:38 -05:00
Aldrin Piri f47168213b NIFI-259: Adjusting validation logic for Connect String to catch errors at startup. 2016-02-01 14:07:38 -05:00
Mark Payne ef80549d63 Merge branch 'NIFI-1093' of https://github.com/olegz/nifi into NIFI-1093 2016-02-01 11:51:27 -05:00
Daniel Cave 171b9c4e94 NIFI-1093 - Added settings for processing JSON documents when validating against database columns to fail, log a warning, or ignore unmatched columns. Added tests for new settings. 
Signed-off-by: Mark Payne <markap14@hotmail.com>
 2016-02-01 11:45:15 -05:00
Aldrin Piri 6902812678 NIFI-259: Adjusting property description of the connect string to be more specific concerning socket tuple format and the associated default port if one is not specified. Providing minor cleanup and removing unused import. 2016-02-01 10:38:45 -05:00
Matt Gilman dbe8ff3f44 NIFI-1426: 
- Ensuring we aren't preventing default when focused in a textarea.
 2016-02-01 08:28:18 -05:00
joewitt 1bf7a585b2 NIFI-1456 This closes #198. reduced tests to 10MB instead of 100MB datasets and resolved IO issues which impact test stability 2016-01-31 21:56:53 -05:00
Richard Miskin 8ef337b65e NIFI-1434 Prevent array index exception in PutEmail 
Signed-off-by: jpercivall <joepercivall@yahoo.com>
 2016-01-30 10:56:31 -05:00
Oleg Zhurakousky 0126800201 NIFI-1093 added support for handling non-required columns 
Mostly based on contribution from Daniel Cave
 2016-01-29 13:41:27 -05:00
Oleg Zhurakousky 807eb8de6d NIFI-1447 modified SimpleProcessLogger Modified SimpleProcessLogger to ensure that whenever is called it prints full stack trace. 
Signed-off-by: jpercivall <joepercivall@yahoo.com>
 2016-01-29 12:39:27 -05:00
jpercivall be27c07c7f NIFI-1382 Created FetchDistributedMapCache processor 2016-01-29 12:12:23 -05:00
Ryan Blue 73c0637c25 NIFI-1193: Add support for storing data in Hive tables. 
Thanks to Ricky for helping with the test dependencies.
 2016-01-29 10:19:29 -05:00
Mark Payne a0224f1b4c NIFI-1431: Added documentation about how ETag and LastModified dates are used by the Processor 
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
 2016-01-28 16:02:05 -05:00
jpercivall ee30b27cc8 NIFI-1396 Correcting PostHttp chunked encoding functionality 2016-01-28 14:50:22 -05:00
Matt Gilman b3990ecdcf NIFI-1426: - Introducing a universal capture for key events to ensure a consistent behavior throughout the application. - Allowing backspace to remove components from the canvas. - Introducing a more consistent behavior around the escape button. 
Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-01-28 12:23:02 -06:00
Oleg Zhurakousky f70f7e3447 NIFI-1333 fixed FlowController shutdown deadlock. put read lock back. This closes #148 
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
 2016-01-28 08:31:30 -05:00
Matt Burgess aa6bc5d505 NIFI-1409: Fix missing transfer on error in ExecuteSql 
Signed-off-by: jpercivall <joepercivall@yahoo.com>
 2016-01-27 18:23:32 -05:00
Aldrin Piri c5a174ae6d Revert "NIFI-1193: Add support for storing data in Hive tables." 
This reverts commit 9dfef8f432.
 2016-01-27 10:03:41 -05:00
Andy LoPresto 93aac8cff3 NIFI-1365 
Added Groovy support for unit tests to pom with skeleton test.
Added Groovy unit tests for OCSPCertificateValidator.
Implemented positive & negative unit tests with cache injection for valid/revoked OCSP certificate.
Modified pom.xml to support Groovy unit tests with custom variable.

mvn clean test -Dgroovy=test

Added local cache injection into Groovy tests for OCSP certificate validation (see NIFI-1324 and NIFI-1364).
Set Java version to 1.7 for Groovy test src/target.
Moved Groovy unit test profile from nifi-web-security to root pom.
Added null check for algorithm argument in PGPUtil.
Changed buffer length check from ">= 0" to "> -1" because it was confusing other developers.
Resolved contrib-check line length issues.
Fixed contrib-check issues in OpenPGPKeyBasedEncryptorTest.
This closes #163

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
 2016-01-26 17:37:59 -05:00
Mark Payne 0d45f21f4b Merge branch 'NIFI-1200' of https://github.com/olegz/nifi into NIFI-1200 2016-01-26 16:38:47 -05:00
Oleg Zhurakousky 0395b90315 NIFI-1200 fixed CPU saturation in FileSystemRepository 
added minimal interval
added WARN message if set interval is lower

NIFI-1200 fixed unused imports
 2016-01-26 16:37:19 -05:00
Ryan Blue 9dfef8f432 NIFI-1193: Add support for storing data in Hive tables. 2016-01-26 13:52:03 -05:00
Mark Payne 0e92607466 Merge branch 'NIFI-1412' of https://github.com/mattyb149/nifi into NIFI-1412 2016-01-26 12:30:19 -05:00
Mark Payne 1c6abcbd55 Merge branch 'NIFI-1421' of https://github.com/rpmiskin/nifi into NIFI-1421 2016-01-26 09:20:33 -05:00
Mark Payne 68975dc2d7 NIFI-1356: Fixed typo in unit test that caused it to look for Shapes_Header.csv.avro instead of Shapes_header.csv.avro (i.e., case of h in Header) 2016-01-26 09:04:13 -05:00
Mark Payne 5ded6b223b Merge branch 'NIFI-1356' of https://github.com/jdye64/nifi into NIFI-1356 2016-01-26 08:51:34 -05:00
Richard Miskin c16c8db459 NIFI-1421 Make namespace ordering deterministic in SplitXML 2016-01-26 05:35:07 +00:00
Bryan Bende b7f7e6ed80 NIFI-1436 This closes #189. Combining stop() and close() into a single method to simplify, and adding checks on stopped flag in the run method of SocketChannelDispatcher and DatagramChannelDispatcher to ensure the run() method exits as soon as possible upon close() being called 
NIFI-1436 Adding synchronization on keys set in close() method  based on Selector JavaDoc

Signed-off-by: joewitt <joewitt@apache.org>
 2016-01-25 22:50:25 -05:00
Mark Payne 925138b6c4 NIFI-1404: Fixed NPE that resulted on startup if there was a Provenance file that had no events 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-01-25 21:55:48 -05:00
Jeremy Dyer 7008a3054e NIFI-1356 2016-01-25 18:28:38 -05:00
joewitt beb92492be NIFI-1421 This closes #184. Added content assertion to test and removed extra space 
Signed-off-by: Mark Payne <markap14@hotmail.com>
 2016-01-25 16:04:47 -05:00
Mark Payne 4a5fb37b33 Merge branch 'NIFI-1421' of https://github.com/rpmiskin/nifi into NIFI-1421 2016-01-25 15:38:08 -05:00
Mark Payne 329e1fe2f4 NIFI-259: Construct QuorumPeer object instead of ZooKeeperServer object because we want to join part of a cluster when using embedded ZK Server 2016-01-25 14:33:22 -05:00
Matt Gilman d71266502c NIFI-1435: - Addressing typo when initializing and showing the reporting task property table which prevented dynamic properties to be created. 2016-01-25 10:42:37 -05:00
Jeremy Dyer dc2e8ce512 NIFI-1356 2016-01-25 10:05:18 -05:00
Jeremy Dyer 8966643d48 NIFI-1156 2016-01-25 10:05:18 -05:00
Matt Burgess 91e7ba84e0 NIFI-1412: Fix error during Avro conversion where no fields are present 2016-01-25 08:35:14 -05:00
Aldrin Piri 8d37af07b9 NIFI-1381 Removing the hardcoded jms:// prefix and instead deferring to the URI specified by the processor properties of PutJMS 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-01-24 23:10:47 -05:00
jpercivall d42f1d4add NIFI-1388 Corrected the property descriptor for the GetMongo query property 
Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-01-24 20:36:04 -05:00
jpercivall 0db2dc9fc3 NIFI-1405 Adding option to invokeHttp to control using chunked encoding for http PUT and POST calls 
Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-01-24 15:52:05 -05:00
Mark Payne 05dabe034c NIFI-1433: Once we roll over journal files, don't attempt to roll them over again 
Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-01-24 15:29:47 -05:00
Richard Miskin dfed7e727d NIFI-1421 Added missing copyright notice to new test input 2016-01-23 08:52:54 +00:00
Mark Payne 93e2c52195 NIFI-1415: Do not enfoce that a password or Private Key Passphrase is set 
Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-01-22 20:46:14 -05:00
Matt Gilman a7d3f8d75f NIFI-1428: - Adding a button to link from a flowfile in a queue listing to a provenance search for that flowfile. 
Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-01-22 18:26:52 -05:00
Mark Payne 92e6961b50 NIFI-1416: If FetchSFTP's Remote File has a directory name in it, do not include that as part of the 'filename' attribute but instead add a 'path' attribute 
Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-01-22 18:15:19 -05:00
Matt Gilman b25db650fd NIFI-1400: - Addressing sort issues with the controller service and reporting task tables. Specifically addressing bulletins, type, and state. 
Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-01-22 16:41:14 -05:00
Bryan Bende 1089f0a956 NIFI-1273 Adding ListenRELP processor which includes refactoring code that was previously part of ListenSyslog into a reusable framework for implementing listener processors, as well as back-end code for handling the RELP protocol. Addressing some feedback from review and fixing logging statements Addressing review feedback, added AsyncChannelDispatcher and made ChannelHandlerFactory use generics for the dispatcher This closes #179 
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
 2016-01-22 16:28:58 -05:00
Matt Gilman 2f5f7b8302 NIFI-1429: - Only attempting to verify disabling of a referenced controller service when it's active. 
Signed-off-by: Mark Payne <markap14@hotmail.com>
 2016-01-22 16:22:40 -05:00
Matt Gilman 8392b46597 NIFI-1397: - Only resetting the node availability when closing the new controller service or reporting task dialog. 
Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-01-22 13:20:58 -05:00
Oleg Zhurakousky 3db6fffa68 NIFI-1378: fixed JMS URI validation simplified JmsFactory check for SSL and scheme-less URIs ensured URI validation is handled by ActiveMqConnectionFactory ensured informative error messages are shown to the userm removed unneeded test, cleaned up unused imports 
Reviewed by Tony Kurc (tkurc@apache.org). This closes #167
 2016-01-21 22:29:50 -05:00
Richard Miskin 68a9375f3e NIFI-1421 Update SplitXML to support namespace declarations 2016-01-21 22:19:15 +00:00
Matt Gilman fcf837b5af NIFI-259: 
- Restoring the call to isListFlowFilesEndpoint when deciding if a response is interrupted.
 2016-01-21 16:35:59 -05:00
Mark Payne 3e13996512 Merge branch 'master' into NIFI-259 2016-01-21 15:02:00 -05:00
Mark Payne b07e13a1d8 NIFI-259: Bug fixes 2016-01-21 13:44:44 -05:00
Matt Gilman 53322c99ac NIFI-1383: - Ensuring appropriate access and state prior to attempting an action with hot keys. - Fixing contrib-check issue. 2016-01-21 12:39:45 -05:00
Matt Gilman 561f5b740a NIFI-1383: - Ensuring that nodes are not kicked out of the cluster when failing to successfully process a mutable request (like copy/paste). - Showing a more descriptive error message when possible. - Ensuring we don't try to instantiate an incomplete flow snippet. 2016-01-21 12:39:28 -05:00
Mark Payne 8f9c0b9ca3 Merge branch 'NIFI-259' of https://git-wip-us.apache.org/repos/asf/nifi into NIFI-259 2016-01-21 12:00:44 -05:00
Mark Payne 09357297e2 NIFI-259: Ensured that thread pools were being shutdown properly 2016-01-21 12:00:40 -05:00
Matt Gilman c18f75c597 NIFI-259: 
- Ensuring component state is sorted accordingly when loaded in the table.
 2016-01-21 11:25:17 -05:00
Mark Payne f6ec437bc7 Merge branch 'master' into NIFI-259 2016-01-21 09:32:02 -05:00
Matt Gilman 0d7edcb3ac NIFI-108: 
- Removing sort from UI.
- Addressing issues with listing and flowfile retrieval when clustered.
- Making the context menu item available when source and destination are still running.
- Adding a refresh button to the queue listing table.
- Fixing the flowfile summary sorting in the cluster manager.
- Adding a message when the source or destination of a connection is actively running.
- Updating the documentation regarding queue interaction.
- Updating the error message when a flowfile is no longer in the active queue.
- Updated queue listing to allow listing to be done while source and destination are running but not sort or have ability to search
- Added heartbeat when we finish clearing queue
- Addressing comments from review.
 2016-01-21 08:13:32 -05:00
Jeremy Dyer 7d73ae77f8 NIFI-1358 Enable expression language support for AWS processor SECRET_KEY and ACCESS_KEY properties 
Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-01-21 00:28:42 -08:00
Aldrin Piri bc75dd3c17 NIFI-1406 Collecting the number of bytes sent regardless if one or multiple FlowFiles are sent. 
Reviewed by Tony Kurc (tkurc@apache.org). This closes #176
 2016-01-20 21:08:15 -05:00
Mark Payne 348831b20c Merge branch 'NIFI-1414' 2016-01-20 17:31:34 -05:00
smarthi e1588d4136 [NIFI-1411] - TestJdbcTypesDerby: java.sql.SQLSyntaxErrorException: TYPE 'DATETIME' does not exist 2016-01-20 16:51:58 -05:00
Aldrin Piri 9ebcc9e4fa NIFI-1393 Providing the ability to send using gzip Content-Encoding in PostHTTP if the endpoint server supports it regardless if the processor is configured to send as a FlowFile This closes #175 
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
 2016-01-20 16:46:57 -05:00
Mark Payne 2c9fb676cd NIFI-1156: Updated version from 0.4.0-SNAPSHOT to 0.4.2-SNAPSHOT; addressed some error handling issues; updated licensing information 2016-01-20 15:27:58 -05:00
Mark Payne d696391f76 NIFI-259: Update ZooKeeperStateServer so that the thread starting it doesn't block indefinitely, as this was causing NiFi to not shut down properly; applied patch for NIFI-1415 2016-01-20 11:52:41 -05:00
Mark Payne 593f1288d8 NIFI-259: Bug fixes, unit tests, documentation updates 2016-01-20 10:16:14 -05:00
Mark Payne 7ef8af9eea NIFI-259: Applied patch for NIFI-1414 2016-01-20 09:14:04 -05:00
Mark Payne 1845f6df8d NIFI-1414: Ensure that the DirectoryStream is closed after streaming contents of a directory 2016-01-20 09:12:14 -05:00
Mark Payne 6af2d4a342 NIFI-259: Fixed NullPointerException 2016-01-19 14:01:45 -05:00
Jeremy Dyer 8f688d4921 NIFI-1360 Add expression language support for "MergeContent" "Correlation Attribute Field" property 
Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-01-19 10:46:37 -08:00
mans2singh 704c333b0d NIFI-1325 migrating AWS processors to use AWSCredentialsProvider controller service and role based access and allow default credentials to assume role. 
Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-01-19 10:17:25 -08:00
Mark Payne d6a21537c1 NIFI-259: Do not require the State File anymore, since state management is being used now 2016-01-19 11:03:31 -05:00
smarthi 9e9182aa63 - Unit test creates resources in HOME directory This closes #174 
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
 2016-01-19 09:44:03 -05:00
Aldrin Piri f53aaed122 NIFI-569 Introducing the maven-failsafe-plugin to the root pom.xml. Refactoring AWS integration level tests to make use of the IT* prefix and removing @Ignore from these classes. This closes #173 
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
 2016-01-19 09:09:24 -05:00
Mark Payne 16dcf4595c Merge branch 'master' into NIFI-259 2016-01-18 08:48:30 -05:00
Mark Payne 95a3115f9b NIFI-259: Bug Fix 2016-01-17 20:32:04 -05:00
Mark Payne eba25ecaca NIFI-259, NIFI-1339: Added OnConfigurationRestored annotation, always invoke onPropertyModified even on restart when properties are changed from defaults, as was done previously 2016-01-17 14:40:27 -05:00
Jeremy Dyer ecb81ec113 NIFI-1362 Set mime.type attribute on response FlowFile based on InvokeHTTP response Content-Type 
Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-01-15 16:40:36 -05:00
Matt Gilman 5b62ff0fc3 NIFI-259: 
- Only providing the View State menu item for DFMs.
 2016-01-14 15:36:57 -05:00
Matt Gilman 4236125f2b NIFI-259: 
- Fixing Consumes for clear state endpoints.
 2016-01-14 15:35:53 -05:00
Matt Gilman ae6f615365 NIFI-259: 
- Code clean up.
- Adding component state actions to the controller service and reporting task tables.
 2016-01-14 13:09:39 -05:00
Matt Gilman 65dfcd06a3 NIFI-259: 
- Fixing contrib check issues.
 2016-01-14 11:16:35 -05:00
Mark Payne e943c41a47 NIFI-259: Minor bug fixes 2016-01-14 09:26:26 -05:00
Oleg Zhurakousky 92062f9beb NIFI-1317: removed duplicate 'name' instance variable 
Reviewed by Tony Kurc (tkurc@apache.org). This closes #169
 2016-01-14 00:43:09 -05:00
Matt Gilman 1a7e6c735d NIFI-259: 
- Exsuring the component state css file is included in aggregated builds.
 2016-01-13 17:20:11 -05:00
Matt Gilman f0d8f73f26 NIFI-259: 
- Addressing some minor layout issues with the view state dialog.
- Ensuring appropriate locking when attempting to clear state.
 2016-01-13 16:57:59 -05:00
Jeremy Dyer ee7400ef53 NIFI-1156 2016-01-13 15:59:47 -05:00
Mark Payne c65829f0cb Merge branch 'NIFI-259' of https://git-wip-us.apache.org/repos/asf/nifi into NIFI-259 2016-01-13 15:12:44 -05:00
Matt Gilman 3f4bd919a9 NIFI-259: 
- Merging responses when clustered to populate node details.
- Fixed bug when clearing processor state when clustered.
- Cleared the table after successfully clearing state.
 2016-01-13 15:12:17 -05:00
Mark Payne 0cd6f80f36 NIFI-259: Bug fixes 2016-01-13 15:11:53 -05:00
Mark Payne 06f525b674 Merge branch 'NIFI-259' of https://git-wip-us.apache.org/repos/asf/nifi into NIFI-259 2016-01-13 13:57:57 -05:00
Mark Payne 0151b1eed1 NIFI-259: Fixed checkstyle violations 2016-01-13 13:57:47 -05:00
Matt Gilman d05314c54b NIFI-259: 
- Initial implementation of viewing and clearing state for a processor.
 2016-01-13 13:35:24 -05:00
Mark Payne d39067ede6 NIFI-259: Update GetHBase to use new State Management feature; updated docs; bug fixes 2016-01-13 13:30:05 -05:00
Mark Payne bbce596d74 NIFI-259: Updated GetHBase to use new State Management; bug fixes; updated docs 2016-01-13 12:47:08 -05:00
Andy LoPresto c757c4d6ea NIFI-1385: Resolved test for missing file which was failing on Windows due to differing error message. 
Reviewed by Tony Kurc (tkurc@apache.org)
 2016-01-12 19:16:29 -05:00
Oleg Zhurakousky 133838a93f NIFI-1233 upgraded to Kafka 0.9.0.0 
Signed-off-by: jpercivall <joepercivall@yahoo.com>
 2016-01-12 18:21:38 -05:00
Mark Payne 774c29a4da NIFI-259: Added Stateful annotation as described on ticket 2016-01-12 15:28:35 -05:00
Mark Payne b4bc105ea9 Merge branch 'nifi-1283' of https://github.com/jvwing/nifi 2016-01-12 10:49:04 -05:00
Matt Gilman bbd35a0258 NIFI-259: 
- Adding endpoints for getting and clearing component state.
 2016-01-12 10:43:30 -05:00
Mark Payne f6b34cbbc5 NIFI-259: Removed bad unit test 2016-01-12 10:40:00 -05:00
Mark Payne e559c68766 NIFI-259: Began updating processors to use new state management 2016-01-12 10:21:47 -05:00
Matt Gilman 44ffddd8b9 NIFI-1324: 
- Addressing lengthy lines contrib-check issue.
- This closes #162
 2016-01-12 09:42:31 -05:00
Andy LoPresto ffbfffce6d NIFI-1324: 
Changed Maven dependencies for BouncyCastle bcprov and bcpg from jdk16:1.46 to jdk15on:1.53 (kept nifi-web-security on jdk16:1.46 because jdk15on:1.53 splits OCSP logic into new module bcpkix).
Added individual unit tests for PGP public keyring validation.
Passes all legacy unit tests.
Added TODOs for customizable brick encryption and refactoring shared code.
Cleaned up magic numbers to constants.
Added unit tests for OpenPGPPasswordBasedEncryptor (internal consistency and legacy file decrypt).
Began refactoring shared encrypt code from OpenPGP* implementations.
Extracted encrypt utility method from OpenPGPPasswordBasedEncryptor to PGPUtil class.
Added test resources (signed and unsigned key-encrypted files).
Added unit tests for OpenPGPKeyBasedEncryptor (internal consistency and external file decrypt).
Changed BC dependency for nifi-web-security to bcprov-jdk15on:1.53 and bcpkix-jdk15on:1.53.
Updated OCSPValidator to use new BC logic for OCSP validation. This code compiles but should be fully audited, as the legacy OCSP validation was not completely implemented.
Added skeleton of OCSP validator unit tests with successful keypair and certificate generation and signing code.
Added further unit tests for issued certificates.
Annotated unimplemented unit tests with note about Groovy integration.
Refactored Jersey call in OCSPCertificateValidator to internal method.
Added toString() to NiFi local OcspRequest.
Implemented positive & negative unit tests with cache injection for valid/revoked OCSP certificate.
Resolved contrib-check issues.
Removed ignored code in unit test.

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
 2016-01-12 09:22:51 -05:00
jpercivall 6b54753dbb NIFI-1316 adding option to DetectDuplicate to not cache the entry identifier 
Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-01-11 22:59:24 -05:00
Mark Payne 7a3e3efce1 NIFI-259: Updated documentation, added digest username/password 2016-01-11 16:43:46 -05:00
James Wing c526656a22 NIFI-1283 Fixing ControllerStatusReportingTask loggers to use fully-qualified class name 2016-01-11 13:39:15 -08:00
Mark Payne d2a969e3d6 NIFI-259: Initial implementation of State Management feature 2016-01-11 16:38:52 -05:00
Mark Payne 57dadb7286 NIFI-259: Initial implementation of State Management feature 2016-01-11 08:28:32 -05:00
joewitt 0c68e2c3a8 NIFI-1378 added validator to ensure the JMS URI has a scheme set 
Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-01-10 16:07:56 -05:00
Aldrin Piri 4f7a4e83b1 NIFI-1311 Avoid using sudo for nifi.sh commands when run.as is the same as the user invoking the script. 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-01-10 15:22:45 -05:00
Matt Gilman fdef994ead NIFI-1346: - Always showing input and output content for a provenance event. 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-01-10 15:08:09 -05:00
joewitt fddb4cd52a NIFI-1347 Updated to use explicit version reference 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-01-10 14:56:08 -05:00
Richard Miskin 8a8ae5508f NIFI-1313 Allow deletion of ProcessGroups that contain nested ProcessGroups 
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
 2016-01-08 16:22:55 -05:00
Aldrin Piri 317b2f4a4a NIFI-1361 Exposing Content-Type as a processor property for both InvokeHTTP and PostHTTP. This makes use of the mime.type attribute via the EL expression ${mime.type}, making more apparent to the user how this header is derived and allowing the explicit setting of a value. 2016-01-08 10:43:14 -05:00
Matt Gilman 6e9175459f NIFI-1363: - Using the appropriate namespace to call the function to detect if the content viewer is configured. 
Signed-off-by: jpercivall <joepercivall@yahoo.com>
 2016-01-07 17:15:44 -05:00
jpercivall 6c46f22b62 NIFI-1327 NIFI-1353 Updating version of OkHttp and OkHttp-Digest 2016-01-07 15:51:49 -05:00
Thad Guidry 5273109ab1 Removed duplicated dependency for nifi-mock in nifi-couchbase-processors 
pom.xml causing a warning during build.

This closes #161

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-01-07 13:51:52 -05:00
Aldrin Piri 47453d2c47 NIFI-1145 Providing delays in tests where resources are being initialized to allow heavily taxed environments sufficient time. 2016-01-07 12:05:23 -05:00
Aldrin Piri aca114926b NIFI-1145 
Increasing timeout values for TestStandardProcessScheduler#validateEnabledDisableMultiThread
Increasing timeout for testConcurrencyWithEnablingReferencingServicesGraph as 10s is not sufficient for overly taxed environments.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-01-07 12:05:23 -05:00
Mark Payne fba006c8e3 NIFI-1145: Extend timeout of unit test from 20 seconds to 120 seconds 
Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-01-07 12:05:23 -05:00
Aldrin Piri 3d487a5381 NIFI-1354 Using the newly generated keystore and truststore throughout the code base and updating an associated test to utilize the specified properties. 2016-01-06 18:16:37 -05:00
Andy LoPresto 8f0e7a1ac5 NIFI-1354 
Generated new self-signed RSA-2048 key for FQDN localhost in keystore and exported to truststore.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2016-01-06 18:16:37 -05:00
Mark Payne b330fd1601 NIFI-108: Built out 'skeleton' of the requests so that web tier can be written against it 
NIFI-108: Implementing ability to list FlowFiles in a queue

NIFI-108:
- Starting to add support for endpoints that will listing flowfiles in a queue.

NIFI-108: Added merging of response for listing of flowfiles in cluster manager

NIFI-108:
- Starting to add support for endpoints that will listing flowfiles in a queue.

NIFI-108:
- Starting to add support for endpoints that will listing flowfiles in a queue.

NIFI-108:
- Adding checkstyle issues.

NIFI-108: Add clusterNodeId to FlowFileSummaryDTO

NIFI-108: Added unit tests; added verifyCanList method to queue; fixed bugs

NIFI-108:
- Adding compilation error for IOException from getFlowFile().
- Code clean up.
- Javadocs.

NIFI-108:
- Verifying two phase commit for queue listing.
- Fixing checkstyle.
- Ensuring drop and listing requests are merged when created when clustered.

NIFI-108:
- Adding initial listing capabilities.
- Passing through the sort column and direction.

NIFI-108:
- Removing Delete FlowFile button.
- Ensuring sort flags are being passed correctly.
- Setting column widths.
- Also including the cluster node address in the flowfile summaries.

NIFI-108:
- Including queue size statistics in listing request.
- Showing connection name.

NIFI-108:
- Including queue size statistics in listing request.
- Ensuring verifyCanList runs when appropriate.

NIFI-108:
- Adding initial support for viewing flowfile details dialog.
- Adding initial support for click to content.

NIFI-108:
- Allowing the flowfile details dialog to be draggable.

NIFI-108:
- Only showing the flowfile listing table when the listing is successful and the listing is not empty.

NIFI-108:
- Reseting the queue stats when closing the listing table.

NIFI-108: Implemented sorting when performing listing of FlowFiles

NIFI-108: Fixed bug that caused the listFlowFiles operation to wait on a readLock before returning and performing work asynchronously; fixed bug in Write-Ahead FlowFile Repository that caused ContentClaims to be queued up for destruction instead of ResourceClaims - this caused millions of ContentClaims to be queued up instead of a single ResourceClaim in some tests

NIFI-108:
- Ensured the column sort indicator is reset when a new listing is opened.
- Removing unused import.

NIFI-108:
- Addressed issues found during the review.
 2016-01-06 16:20:23 -05:00
Joe Skora de2dd93f26 NIFI-1300 Corrects inconsistent error behavior in PutJMS and added PutJMS unit testing 
Reviewed by Tony Kurc (tkurc@apache.org) This closes #145
 2016-01-04 00:26:21 -05:00
gresockj 34bd2061f7 NIFI-1291: Adding BinFiles in nifi-processor-utilities, deprecating version in nifi-standard-processors. This closes #155. 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-01-03 23:05:28 -05:00
Aldrin Piri c696cb09b6 NIFI-1212 Treating text/csv files as plain text so they may also be displayed. This closes #154. 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-01-03 16:12:06 -05:00
ricky 236a2b72b6 NIFI-1335: Allow UnpackContent to Handle application/x-tar mime.type 
- Add additinal `mime.type` for tar archives (application/x-tar)
- Update `UnpackContent` tests to verify auto-detect format property
 2016-01-03 11:42:18 -05:00
joewitt a915534e23 Merge branch 'master' into NIFI-1343 2016-01-01 14:09:23 -05:00
joewitt e16a9a0c68 NIFI-1343 added unit tests to verify behavior 2016-01-01 14:08:46 -05:00
Mark Payne 4ab6e3cb3f NIFI-1219: Moved the kafka-tmp directory that gets created to the target directory; added missing license info to beginning of EmbeddedKafkaProducerHelper.java 2016-01-01 13:48:22 -05:00
Oleg Zhurakousky b043d04ecf added embedded Kafka server and tests 2016-01-01 13:48:22 -05:00
Mark Payne 02b325b07e NIFI-1343: Make comparisons case sensitive if configured to do so. This closes #153. 
Signed-off-by: joewitt <joewitt@apache.org>
 2016-01-01 13:34:07 -05:00
Mark Payne 8e031c987b NIFI-1164: Fixed contrib-check issues, log message 2015-12-31 15:37:00 -05:00
Mark Payne 71544cd22b Merge branch 'NIFI-1164' of https://github.com/olegz/nifi into NIFI-1164 2015-12-31 15:22:09 -05:00
James Harrison 03730adb3f NIFI-1338 - Clarifying documentation for InvokeHTTP 
Signed-off-by: joewitt <joewitt@apache.org>
 2015-12-31 13:48:00 -05:00
Oleg Zhurakousky 602c4a9639 NIFI-1164 addressed latest PR comments 2015-12-28 16:26:00 -05:00
ricky c45060f703 NIFI-987 Added Riemann (PutRiemann) Reporting 
- Introduced nifi-riemann-bundle for future Riemann backed monitoring

- Added initial PutRiemann processor for writing events to Riemann
  using the Riemann batch client.

	- Values for events are provided using the NiFi expression language
		e.g. Metric -> ${latency.milliseconds:divide(1000)}
 2015-12-28 15:45:11 -05:00
Oleg Zhurakousky 0d09054d9f NIFI-1164 addressed PR comment 
Added isActive check to the StandardControllerServiceNode:280 to ensure that
the IF statement can only have a chance to succeed if service is active. The service
will be indiscriminately deactivated as soon as disable(..) operation is invoked. This itself will
eliminate the race condition discovered by Mark

NIFI-1164 addressed PR comments
fixed the race condition described by Mark during disable call

NIFI-1164 polished javadoc
 2015-12-28 15:16:32 -05:00
Oleg Zhurakousky 909c0decd6 NIFI-1164 Fixed race condition and refactored 
Changed ControllerServiceNode by adding enable(..), disable(..) and isActive() operations. See javadocs for more details in both ControllerServiceNode and StandardControllerServiceNode

Refactored service enable/disable logic in StandardProcessScheduler and StandardControllerServiceNode . Below are some of the notes:
- No need for resetting class loader since its going to derive from the class loader of the service. In other words any classes that aren’t loaded and will be loaded within the scope of the already loaded service will be loaded by the class lower of that service
- No need to control 'scheduleState.isScheduled()’ since the logic has changed to use CAS operation on state update and the service state change is now atomic.
- Removed Thread.sleep(..) and while(true) loop in favor of rescheduling re-tries achieving better thread utilization since the thread that would normally block in Thread.sleep(..) is now reused.
- Added tests and validated that the race condition no longer happening

Added additional logic that allows the initiation of the service disabling while it is in ENABLING state. See javadoc of StandardProcessScheduler.enable/disable for more details.

NIFI-1164 polishing
 2015-12-28 09:22:23 -05:00
Oleg Zhurakousky ebcefaac23 NIFI-1289 reverted new method of NiFiProperties in favor of the localized reflection call in test to refresh properties. 
Reviewed and Amended (added comments) by Tony Kurc (tkurc@apache.org). This closes #150
 2015-12-26 09:58:08 -05:00
Oleg Zhurakousky 2845e93812 NIFI-1289 added support for refreshing properties - Added _getNewInstance()_ operation to NiFiProperties to ensure there is a way to refresh/reload NiFi properties - Fixed javadocs 
Signed-off-by: Bryan Bende <bbende@apache.org>

This closes #142
 2015-12-23 14:30:08 -05:00
Bryan Bende 3189a13da1 NIFI-1328 Explicitly setting the parameters based on type in PutSQL, falling back to setObject() 
This closes #146
 2015-12-23 13:08:56 -05:00
Bryan Bende 8d46041a13 NIFI-1319 Updating JdbcCommon to check meta.isSigned(i) to determine if Avro schema should use a long or int 
Signed-off-by: Bryan Bende <bbende@apache.org>
 2015-12-23 10:03:30 -05:00
joewitt f4ac8d75c5 NIFI-1312-RC1 prepare for next development iteration 2015-12-19 00:41:04 -05:00
joewitt d624ea4866 NIFI-1312-RC1 prepare release nifi-0.4.1-RC1 2015-12-19 00:40:53 -05:00
joewitt fb51483777 Revert "NIFI-1218 upgraded Kafka to 0.9.0.0 client API Tested and validated that it is still compatible with 0.8.* Kafka brokers" 
This reverts commit 37635232c7.
 2015-12-19 00:07:34 -05:00
joewitt 51821e01b6 Revert "NIFI-1218 addressed PR comments" 
This reverts commit 7c87968260.
 2015-12-19 00:06:35 -05:00
Matt Gilman 05395eeca5 NIFI-1294: - Updating exception handling to account for a possible issue in an dependency. 
Signed-off-by: joewitt <joewitt@apache.org>
 2015-12-18 21:33:07 -05:00
Mark Payne 0a58ab47b9 NIFI-1310: Do not yield context when encountering IOException and fixed bug that caused the penalization period for a node not to get set 
Signed-off-by: joewitt <joewitt@apache.org>
 2015-12-18 17:35:46 -05:00
Matt Gilman 43c7ecd221 NIFI-1298: - Addressing too many significant digits in the processor load average and [non]heap utilization. 
Signed-off-by: joewitt <joewitt@apache.org>
 2015-12-18 17:02:40 -05:00
Matt Gilman 0436383f49 NIFI-1286: - Removing provenance option from the Label context menu 
Signed-off-by: joewitt <joewitt@apache.org>
 2015-12-18 17:02:33 -05:00
Matt Gilman 608287f9fe NIFI-1309: 
- Addressing issues around remote process groups automatically issuing new account requests.
- Ensuring authorization issues are updated with status refresh.
 2015-12-18 15:57:38 -05:00
Mark Payne bd4f31a4c2 NIFI-1279: Take the FlowFile's 1-up id into account for the newest and oldest flowfile prioritizers 
Signed-off-by: joewitt <joewitt@apache.org>
 2015-12-18 12:08:31 -05:00
Mark Payne 1869b44682 NIFI-1285: Used a BufferedInputStream when swapping in data 
Signed-off-by: joewitt <joewitt@apache.org>
 2015-12-18 12:08:23 -05:00
Mark Payne 9ca0f95d06 NIFI-1272: Added unit test to verify behavior 2015-12-18 11:51:42 -05:00
ianwww fa4c5314ca nifi-1272 Fixing a bug in StandardProcessSession.get(int) which returns incorrect number of flowfiles 
Signed-off-by: Mark Payne <markap14@hotmail.com>
 2015-12-18 11:50:39 -05:00
Mike Moser f239be289e NIFI-1302 find dynamic libraries with standard names 
Signed-off-by: joewitt <joewitt@apache.org>
 2015-12-18 09:12:32 -05:00
Oleg Zhurakousky 7c87968260 NIFI-1218 addressed PR comments 
Signed-off-by: jpercivall <joepercivall@yahoo.com>
 2015-12-16 14:25:28 -05:00
Oleg Zhurakousky 37635232c7 NIFI-1218 upgraded Kafka to 0.9.0.0 client API Tested and validated that it is still compatible with 0.8.* Kafka brokers 
Signed-off-by: jpercivall <joepercivall@yahoo.com>
 2015-12-16 14:25:28 -05:00
Matt Gilman b19ff7cf37 NIFI-1215: 
- Only showing the run duration setting when applicable.
- Showing the user a warning that a source processor with a non 0 run duration could lose data when NiFi is restarted.
 2015-12-16 10:53:15 -05:00
Matt Gilman 51b8ecd01b NIFI-1185: 
- Using banners from the NCM rather than replicating to a node.
 2015-12-15 12:51:14 -05:00
Matt Gilman c75b5cfcea NIFI-1119: - Addressing race condition that caused the revision to be checked before the flow was loaded. 
Signed-off-by: jpercivall <joepercivall@yahoo.com>
 2015-12-15 10:05:44 -05:00
Matt Gilman 17be1c2d9f NIFI-1206: - Only enabling the enable/disable toolbar icon when appropriate. 
Signed-off-by: Joseph Percivall <joepercivall@yahoo.com>
 2015-12-14 18:09:11 -05:00