Apache
/
nifi
mirror of https://github.com/apache/nifi.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
3,738 Commits 144 Branches 197 Tags 399 MiB
Commit Graph

1003 Commits

Author SHA1 Message Date
Mark Payne 37d6b7350e NIFI-1155: Refactored StandardFlowFileQueue to update member variables more intelligently, using CAS operations instead of locks. This reduces code complexities because other optimizations that previously existed are no longer needed 
Signed-off-by: joewitt <joewitt@apache.org>
 2015-11-13 00:06:12 -05:00
Mark Payne 3ed0949c55 NIFI-1155: Ensure that when poll(FlowFileFilter, Set) is called, we properly update the indicator for whether or not queue is full 
Signed-off-by: joewitt <joewitt@apache.org>
 2015-11-13 00:06:12 -05:00
Mark Payne 854d203982 Merge branch 'NIFI-1143' of https://github.com/olegz/nifi into NIFI-1143 2015-11-11 14:14:54 -05:00
Oleg Zhurakousky 5baafa156a NIFI-1143 Fixed race condition which caused intermittent failures 
Fixed the order of service state check in PropertyDescriptor
Encapsulated the check into private method for readability
Modified and documented test to validate correct behavior.
For more details please see comment in https://issues.apache.org/jira/browse/NIFI-1143
 2015-11-11 14:06:08 -05:00
Oleg Zhurakousky 76690a8ee9 NIFI-1061 fixed deadlock caused by DBCPConnectionPool.onConfigured() 
Current implementation of DBCPConnectionPool was attempting to test if connection could be obtained via dataSource.getConnection().
Such call is naturally a blocking call and the duration of the block is dependent on driver implementation. Some drivers (e.g., Phoenix - https://phoenix.apache.org/installation.html)
attempts numerous retries before failing creating a deadlock when attempt was made to disable DBCPConnectionPool which was still being enabled.

This fix removes the connection test from DBCPConnectionPool.onConfigured() operation returning successfully upon creation of DataSource.
For more details see comments in https://issues.apache.org/jira/browse/NIFI-1061
 2015-11-11 08:33:51 -05:00
Mark Payne e6644b3747 Merge branch 'NIFI-1141' 2015-11-10 17:17:55 -05:00
Mark Payne 72008035b1 Merge branch 'NIFI-1000' 2015-11-10 15:02:02 -05:00
Mark Payne 11768cc388 NIFI-1141: Fixed the order of the arguments to the setCurrentContentClaim method 2015-11-10 11:35:14 -05:00
Mike Moser b75af5b344 Merge branch 'NIFI-1085' 2015-11-09 16:13:12 -05:00
Mark Payne 73c1671975 NIFI-1132: Limited number of Lineage Identifiers held to 100 and marked the getLineageIdentifiers() method as deprecated 2015-11-09 12:09:56 -05:00
joewitt 99629646fe NIFI-1122 moved to 0.4.0-SNAPSHOT 2015-11-06 23:41:15 -05:00
Mark Payne 0d6e81b54f Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/nifi 2015-11-06 09:16:12 -05:00
Bryan Bende ee87f33ce7 Fixing check-style violation in StandardFlowFileQueue 2015-11-06 09:10:10 -05:00
Mark Payne 1a388f957e NIFI-1111: We were subtracting values in the wrong order so that we always would 'yield' for a negative amount of time, which equated to not yielding at all. 2015-11-06 09:03:38 -05:00
Mark Payne 98f5a1ab77 NIFI-1110: Fixed bug that caused queue size to become negative when FlowFiles are expired 
Signed-off-by: Mark Payne <markap14@hotmail.com>
 2015-11-05 12:12:02 -05:00
Andrew Grande f1bd3d471d NIFI-1112 Fixed APIOperation Annotation on ReportingTaskResource. 
Reviewed by Tony Kurc (tkurc@apache.org)
 2015-11-04 18:31:34 -05:00
Mike Moser 6add372bc1 NIFI-1085 WebClusterManager fair locking and HeartbeatMonitoringTimerTask scheduling 2015-11-04 12:19:00 -05:00
Mark Payne 2b1d093120 NIFI-1105: Only trigger a processor that requires input to run if data is available for it process 2015-11-04 08:41:10 -05:00
Mark Payne dbf0c7893f NIFI-730: Completed merging of master 2015-11-02 14:31:28 -05:00
Mark Payne e4cebba3c7 Merge branch 'master' into NIFI-730 2015-11-02 14:02:16 -05:00
Mark Payne abf2c62884 NIFI-1051: Fixed checkstyle violations 2015-11-02 11:01:50 -05:00
Mark Payne 7a73867c46 Merge branch 'NIFI-1051' of https://github.com/olegz/nifi into NIFI-1051 2015-11-02 10:51:43 -05:00
Matt Gilman 5a04021dd7 NIFI-730: 
- Updating logic now that original is guaranteed to be non null.
- Always reporting 100% once the drop request has completed.
 2015-11-02 08:59:11 -05:00
Mark Payne f5727cfb0f NIFI-730: Ensure that we always populate queue counts when initiating a Drop FlowFile request 2015-11-02 08:36:50 -05:00
Mark Payne ad849c77df NIFI-1090: Fixed log message that was at info level but should have been debug level 2015-11-01 14:37:01 -05:00
Mark Payne 9c148da1c2 NIFI-869: Fixed checkstyle violations 2015-10-30 16:09:50 -04:00
Mark Payne ba72452f66 Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/nifi 2015-10-30 16:02:15 -04:00
Oleg Zhurakousky a3d43d23dc NIFI-869 Fixed SimpleProcessLogger to log correct messages Ensured that SimpleProcessLogger correctly interprets Throwable as discussed in JIRA Added tests 
Signed-off-by: Mark Payne <markap14@hotmail.com>
 2015-10-30 15:52:33 -04:00
Oleg Zhurakousky 90aea01350 NIFI-869 Fixed formatting issue Fixed formatting issue with printed error message which only apears when NiFi is cnfigured using Logback. Please see NIFI-869 for more details 
Signed-off-by: Mark Payne <markap14@hotmail.com>
 2015-10-30 15:52:33 -04:00
Oleg Zhurakousky 5c4042bd7c NIFI-1051 Allowed FileSystemRepository to skip un-readable entries. 
The exception was caused due to basic file permissions. This fix overrides
'visitFileFailed' method of SimpleFileVisitor to log WARN message and allow
FileSystemRepository to continue.
 2015-10-29 16:31:17 -04:00
Bryan Bende 5cc2b04b91 NIFI-986 Refactoring of action classes from nifi-user-actions to have interfaces in nifi-api, and adding getFlowChanges to EventAccess 
- Fixing empty java docs and adding sort by id asc to the history query
- Changing userDn to userIdentity in Action and FlowChangeAction
- Modifying NiFiAuditor to always save events locally, and implementing getFlowChanges for ClusteredEventAccess
 2015-10-29 16:28:36 -04:00
Mark Payne edf238e004 NIFI-730: Do not require a Read Lock in order to obtain backpressure configuration values for FlowFileQueue's 2015-10-28 10:27:58 -04:00
Joseph Percivall b885f955f4 NIFI-516 adding option to StandardProcessSession.read to close stream 
Signed-off-by: Mark Payne <markap14@hotmail.com>
 2015-10-26 20:23:13 -04:00
Mark Payne fc2aa2764c NIFI-10: Added FETCH and DOWNLOAD Provenance Events; updated FlowController to use DOWNLOAD event instead of SEND whenever a user downloads/views content via Provenance Event 2015-10-26 14:58:50 -04:00
Mark Payne 51f564024a NIFI-10: Added FETCH Provenance Event and updated processors to use this new event type 2015-10-26 14:58:50 -04:00
Mark Payne aec32a277c NIFI-1070: Added detailed debug-level logging about how FileSystemRepository is choosing to expire archived data 2015-10-26 14:36:03 -04:00
Matt Gilman f8c3377c84 NIFI-820: 
- Creating a context menu item for accessing provenance directly for a selected component.
 2015-10-26 10:16:01 -04:00
Mark Payne 0636f0e731 NIFI-810: Merged master into branch 2015-10-25 11:02:40 -04:00
Alex Moundalexis b809031195 NIFI-1046 shell clean up: +variable braces for consistency. Reviewed by Tony Kurc (tkurc@apache.org). This closes #106 2015-10-19 20:36:58 -04:00
Alex Moundalexis 9aa716b059 NIFI-1046 shell clean up: quoting prevents globbing & word splitting. 
Ignoring the unquoted array expansions on L188/190; suspect they ought to be quoted per [SC2068](https://github.com/koalaman/shellcheck/wiki/SC2068), but need to confirm desired arguments.

Reviewed by Tony Kurc (tkurc@apache.org)
 2015-10-19 20:36:07 -04:00
Alex Moundalexis 58d9b02517 NIFI-1046 shell clean up: remove backticks from nifi.sh. Reviewed by Tony Kurc (tkurc@apache.org) 2015-10-19 20:33:52 -04:00
Venkatesh Sellappa 9a8d763d8d NIFI-612 Remove FlowUnmarshaller. Builds, runs, could not find any latent references using reflection or other ways of loading classes without direct references. This closes #103. Signed off by Tony Kurc <tkurc@apache.org> 2015-10-17 09:43:10 -04:00
Matt Gilman 570202eb30 NIFI-730: 
- Fixing capitalization to remain consistent.
 2015-10-16 10:47:02 -04:00
Matt Gilman a872403831 NIFI-730: 
- Updating the styling of the drop request status dialog.
- Rendering any errors that may have occurred.
 2015-10-16 09:37:57 -04:00
Mark Payne 2b4999c018 NIFI-730: do not interrupt swap thread if drop flowfiles is canceled 2015-10-16 09:06:13 -04:00
Mark Payne b8dbd1018c NIFI-730: Updated details of Provenance Event for when a queue is emptied 2015-10-16 08:53:40 -04:00
Matt Gilman 39a050d2fd NIFI-730: 
- Adding emptying a queue when clustered.
 2015-10-14 17:47:06 -04:00
Mark Payne 09a3f6dadd NIFI-730: reordered states for drop flowfile request 2015-10-14 16:32:39 -04:00
Mark Payne 98a04eec74 NIFI-730: Set current queue size on drop flowfile request immediately when creating request 2015-10-14 13:48:59 -04:00
Matt Gilman cad0e7cf0f NIFI-730: 
- Adding progress bar for showing current status of drop request.
- Allowing the user to cancel the current drop request.
 2015-10-14 12:39:19 -04:00
Mark Payne a2ae99f899 NIFI-730: Make cancel request actually cancel 2015-10-14 11:11:48 -04:00
Mark Payne 5867193bc1 NIFI-730: Added additional testing logic to unit tests 2015-10-14 09:56:52 -04:00
Mark Payne 0af1acaafa NIFI-730: Return DropFlowFileStatus object when calling cancel 2015-10-14 09:46:21 -04:00
Mark Payne 77f7d7524c NIFI-730: bug fixes and code cleanup for swap manager and flowfile queue 2015-10-14 09:14:15 -04:00
Mark Payne afb76afcd0 NIFI-730: Added error messages if we fail to drop FlowFiles from queue 2015-10-13 15:57:18 -04:00
Mark Payne 72ff2a25d5 NIFI-730: Updated queue sizes appropriately during dropping of flowfiles 2015-10-13 15:46:56 -04:00
Matt Gilman 4b41aaab02 NIFI-730: 
- Fixing checkstyle violations.
- Wiring endpoints and framework model.
- Lowering the max delay while polling from 8 seconds to 4 seconds.
 2015-10-13 13:43:10 -04:00
Mark Payne af78354d84 NIFI-730: Added additional parameters to dropFlowFiles 2015-10-13 12:20:18 -04:00
Matt Gilman 09d6fe5cdb NIFI-730: 
- Fixing checkstyle violations.
- Renaming context menu item for emptying a queue.
 2015-10-13 10:12:58 -04:00
Mark Payne 9be37914dd Merge branch 'NIFI-730' of https://git-wip-us.apache.org/repos/asf/nifi into NIFI-730 2015-10-13 10:03:31 -04:00
Mark Payne 49a781df2d NIFI-730: Implemented swapping in and out on-demand by the FlowFileQueue rather than in a background thread 2015-10-13 10:03:03 -04:00
Matt Gilman e0ac7cde37 NIFI-730: 
- Starting to add support for deleting flow files from a queue by creating endpoints and starting to wire everything together.
- Adding context menu item for initiating the request to drop flow files.
 2015-10-12 10:00:54 -04:00
Mark Payne b8c51dc35d NIFI-730: Added methods for dropping queued flowfiles; refactored swap manager but have not yet started swapping flowfiles in or out from within the flowfile queue 2015-10-11 10:27:07 -04:00
Mike Moser f798127290 NIFI-1031 resolve reporting task error in flow.xml 
Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2015-10-09 10:53:23 -04:00
Matt Gilman 2583d7869a NIFI-442: 
NIFI-828:
- Always selecting the first item in the new component table.
- Enabling adding the selected component by typing Enter.
- Removing the 'filter by' in the new component dialogs and instead just searching every field.
 2015-10-07 19:11:55 -04:00
Mark Payne 8e2308b78d NIFI-810: Reworded validation errors pertaining ot upstream connections 2015-10-07 17:53:04 -04:00
Mark Payne ccfb57fe9f NIFI-810: Addressed several checkstyle violations 2015-10-07 17:48:51 -04:00
Mark Payne b974445ddd NIFI-810: rebased from master 2015-10-07 17:28:39 -04:00
Mark Payne 5ecdb1858e NIFI-810: Updated the wording on validation errors due to upstream connections 2015-10-07 17:26:14 -04:00
Matt Gilman 13edcfda2e NIFI-810: 
- Adding basic support for preventing connection when appropriate.
- Updating validation when [dis]connecting processors.
 2015-10-07 17:26:14 -04:00
Mark Payne 4afd8f88f8 NIFI-810: Created RequiresInput annotation and ensure that processors are invalid if connections do not agree 2015-10-07 17:26:14 -04:00
Matt Gilman 2215bc848b NIFI-810: 
- Adding basic support for preventing connection when appropriate.
- Updating validation when [dis]connecting processors.
 2015-09-25 17:46:58 -04:00
Mark Payne 034ee6de6b NIFI-810: Created RequiresInput annotation and ensure that processors are invalid if connections do not agree 2015-09-25 11:39:58 -04:00
Bryan Bende 600f91a262 NIFI-932 Adding hasIncomingConnection() and hasConnection(Relationship) to ProcessContext and updating ExecuteSQL 2015-09-18 17:05:31 -04:00
Matt Gilman ded74ec94c NIFI-876 prepare for next development iteration 2015-09-14 21:48:11 -04:00
Matt Gilman 2ec735e350 NIFI-876 prepare release nifi-0.3.0-RC1 2015-09-14 21:48:00 -04:00
Mark Payne 3d4ce34529 NIFI-938: If ResourceClaim is removed while a process has access to its stream, don't delete the claim 2015-09-14 11:47:02 -04:00
Mark Payne 992e841027 NIFI-948: Provide capability to register notifications for nifi lifecycle events 2015-09-13 19:55:12 -04:00
Mark Payne 3780159d01 NIFI-939: Fixed bug in equals and hashCode methods of StandardContentClaim; added unit test to TestStandardProcessSession in order to verify behavior of append() method 2015-09-11 12:50:39 -04:00
Joseph Percivall a83ed34f91 Commit for NIFI-836 to fix broken javadoc @link elements 
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
 2015-09-10 12:05:47 -04:00
Matt Gilman d1206fe074 NIFI-926: 
- Ensuring that nodes are not disconnected when the user attempts to reset a counter that does not exist on that node. This can happen when/if counters are adjusted conditionally.
 2015-09-07 08:41:47 -04:00
Mark Payne 2bb7853001 Merge branch 'NIFI-905' 2015-08-31 11:14:17 -04:00
Mark Payne 3159cec782 NIFI-905: Ensure that when archive threshold is hit, archived data is destroyed and if no archived data exists that Processors aren't blocked from updating content repo 2015-08-28 14:15:35 -04:00
Mark Payne 5de37f63d9 NIFI-902: Ensure that if we get an IOException during rollover of WAL, we are able to recover 2015-08-28 10:04:58 -04:00
Mark Payne fbec28bad9 NIFI-893: Ensure that if session.commit fails, previously 'checkpointed' sessions are rolled back 2015-08-27 15:17:55 -04:00
Bryan Bende 310347fd66 Merge branch 'NIFI-842' 2015-08-24 18:58:56 -04:00
ricky 7fb6e884a7 NIFI-866: Add Kerberos Support for Hadoop 
- Add krb5.conf to nifi.properties

	nifi.kerberos.krb5.file | path to krb5.conf

- Connections to secure Hadoop clusters will be determined by their config,
  that is, hadoop.security.authentication should be set to kerberos.

- Added two optional arguments to AbstractHadoopProcessor (principal and keytab),
  these are only required if the cluster you're connecting to is secured. Both of
  these options require the krb5.conf to be present in nifi.properties.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2015-08-24 15:36:22 -04:00
Bryan Bende 3bfe830afc NIFI-842 Updating org.apache.nifi.NiFi so it sends a started signal to bootstrap which can then be used to make retsart decisions 2015-08-24 12:00:21 -04:00
Mark Payne d351f3ed1b NIFI-882: When a ProcessGroup is removed from the graph, ensure that the removeXYZ method is called for each component in that Process Group so that resources are appropriately cleaned up 2015-08-21 15:55:04 -04:00
Mark Payne 1ddd736558 NIFI-854: Ensure that controller services are included in template if any processor (or other service) in the template references it, regardless of how many levels deep the component's ProcessGroup is 2015-08-21 14:46:20 -04:00
Mark Payne 15a8699dc4 NIFI-744: Addressed feedback from review, mostly adding documentation to a few points in the code 2015-08-21 11:08:34 -04:00
Mark Payne 68d94cc01b NIFI-744: Refactored ContentClaim into ContentClaim and ResourceClaim and allowed resource claim to be reused across sessions 2015-08-21 11:08:34 -04:00
Mark Payne 8ebf1f03c2 Merge branch 'nifi-868' of https://github.com/pukkaone/nifi into NIFI-868 2015-08-19 12:56:26 -04:00
Mark Payne 19f7db6986 NIFI-793: Added multi-threading to the indexing in the Persistent Provenance Repository 2015-08-19 12:24:25 -04:00
Mark Payne f171756a88 NIFI-862: Ensure that if we are unable to stop a controller service, we can still stop processors referencing it and the service itself 2015-08-19 10:12:57 -04:00
Mark Payne e409b6c5c1 NIFI-839: When disabling controller services, do so one-at-a-time in the process scheduler, so that we can ensure that all are disable-able and then disable them atomically 2015-08-19 08:55:15 -04:00
Chin Huang 8f98f80938 NIFI-868 Configure advertised host name of Remote Process Group Input Port 
Before this change, the host given out to clients to connect to a Remote
Process Group Input Port is the host where the NiFi instance runs.
However, sometimes the binding host is different from the host that
clients connect to. For example, when a NiFi instance runs inside a
Docker container, a client on a separate machine must connect to the
Docker host which forwards the connection to the container.

Add a configuration property to specify the host name to give out to
clients to connect to a Remote Process Group Input Port. If the property
is not configured, then give out the name of host running the NiFi
instance.
 2015-08-18 21:35:28 -07:00
joewitt 8e5347156f NIFI-841 fixed contrib check issues 
Signed-off-by: Mark Payne <markap14@hotmail.com>
 2015-08-18 20:06:42 -04:00
joewitt c3731703f3 NIFI-841 correct patch paths and fixed formatting issues 
Signed-off-by: Mark Payne <markap14@hotmail.com>
 2015-08-18 20:06:42 -04:00
Mark Payne 35439db347 NIFI-841: Ensure that IOExceptions on session commit are handled properly 
Signed-off-by: joewitt <joewitt@apache.org>
Signed-off-by: Mark Payne <markap14@hotmail.com>
 2015-08-18 20:06:41 -04:00
Matt Gilman ffa7ceb2b1 NIFI-861: 
- Addressing issue with blank processor names when attempting to generate ellipsis on the canvas.
 2015-08-17 15:23:36 -04:00
joewitt 6066557e66 NIFI-827 added headless entry 
Signed-off-by: joewitt <joewitt@apache.org>
 2015-08-17 09:16:34 -04:00
joewitt aa99884782 NIFI-850 removed nifi parent, updated nifi pom, moved all nifi subdirs up one level, fixed readme. 2015-08-15 13:12:22 -04:00