Commit Graph

1207 Commits

Author SHA1 Message Date
Matt Gilman ae8045d99b
NIFI-2505: - Updating the user identity field in the ActionDTO.
This closes #800.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-09 13:07:50 -04:00
Matt Gilman aae6325603
NIFI-2237: - Reorganizing the refactored REST endpoints. - Tweaking message text.
This closes #788.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-09 12:04:31 -04:00
Oleg Zhurakousky 5e5c3f7c3b NIFI-2504 This closes #810. fixed template export for RPG
- ensured that RPG and connections to it are treated as special case where instance ID is preserved
2016-08-08 16:54:27 -04:00
Mark Payne ed14bf22e7 NIFI-2493 This closes #798. Do not fingerprint Remote Ports' running state. When synchronizing remote flow with local flow, start/stop remote group ports as appropriate based on the inherited flow 2016-08-08 16:54:27 -04:00
Mark Payne 8752d11f18 NIFI-2495 This closes #808. Ensure that we always close Index Searchers when we're finished with them 2016-08-08 16:54:27 -04:00
Andre F de Miranda d6a2409d71 NIFI-1965 - Implement QueryDNS processor
This closes #496.
2016-08-08 22:27:40 +02:00
Andre F de Miranda 493919922e NIFI-1965 - Rename nifi-geo-bundle to nifi-enrich-bundle 2016-08-08 22:27:22 +02:00
Oleg Zhurakousky 9b647cd538 NIFI-1148 added IMAP/POP3 support added initial set of processors to support consumption of Email via IMAP/IMAPS and POP3 protocols
Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-1148 addressed PR comments from @trixpan

NIFI-1148 addressing PR comments

NIFI-1148 addressed PR comments

This closes #710
2016-08-08 11:57:28 -04:00
Mark Payne c1c052af71 NIFI-2406: Ensure that hearbeat monitor continues to run while instance is running. This way if a node sends heartbeat to this node as elected coordinator changes, we notify the node accordingly. Handle Exceptions more gracefully in leader election code. Tweaked some handling of how nodes reconnect to the cluster to ensure more stability with cluster
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #729
2016-08-08 09:15:10 -04:00
Joe Skora d094130a26 NIFI-2237:
Extend template handling to read only properties.
* Add read only property handling to type.hbs template.
* Add flag to ProcessorDTO.getRelationships to reflect read only nature of the property.
* Remove explicit "read only" message from ConnectionDTO.getAvailableRelationships to avoid duplicate text.
* This closes #806
2016-08-08 09:06:47 -04:00
Kirk Tarou e2f649cfea NIFI-2470 This closes #802.
Fixed WritesAttributes section to include all attributes with the
correct attribute names and descriptions
2016-08-07 23:58:09 -04:00
Oleg Zhurakousky 29f53c07f5 NIFI-2509 This closes #805. fixed NPE condition in KafkaConsumer 2016-08-07 23:48:57 -04:00
Joe Skora 903b1fe465 NIFI-2494 Remove extraneous text that shows on banner line. This closes #793 2016-08-05 16:03:26 -04:00
Scott Aslan f2f070c5be
NIFI-2468: Improved UI consistency.
- Addressing comments from PR. Moving new service button below the tab pane. (+2 squashed commits)
Squashed commits:
[f746d09] NIFI-2468:
- Addressing numerous inconsistencies throughout the UI (button placement above tables, timestamp styles, etc)
[7d1fb09] addressing some inconsistencies

This closes #794.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-08-05 12:32:06 -07:00
Mark Payne 7ffa30d21b NIFI-2423: Make use of the SSLContextService to provide SSL information
Signed-off-by: Oleg Zhurakousky <oleg@suitcase.io>
2016-08-05 14:14:38 -04:00
Oleg Zhurakousky c39a127ec8 NIFI-2322, NIFI-2423, NIFI-2412 Kafka improvements
- Fixed KafkaConsumer's connection block when broker is not available
- Fixed Serializer/Deserializer configs in both Consume/Publish Kafka
- Added sensitive properties for SSL ket/trust stores

NIFI-2322 fixed tests
2016-08-05 14:14:38 -04:00
Mark Payne 8d34223265 NIFI-2472: Normalize coordinates for all components in a snippet, not just processors
This closes #786

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-08-04 19:46:39 -04:00
Matt Gilman c0a253568e NIFI-2486: - Authorizing individual bulletins being returned through the bulletin board.
This closes #792

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-08-04 18:41:18 -04:00
Matt Gilman 04147ac22a
NIFI-2458: - Fixing issue with timezone when the initial provenance result was cancelled.
This closes #773.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-04 17:09:21 -04:00
Mark Payne 0e0166cec5
NIFI-2483: Removed 'passthrough' relationship
This closes #789.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-04 16:12:21 -04:00
Mark Payne 7ba10a6dea
NIFI-2484: Ensure that if a Processor throws an Exception from an @OnScheduled method that @OnStopped methods get called
This closes #791.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-04 15:34:10 -04:00
Bryan Bende aca2a92d6b
Fixing contrib-check in GetHDFSSequenceFileTest 2016-08-04 15:08:08 -04:00
Mike Moser b7b1dc2fe6 NIFI-2471 fix Hadoop configuration resources when talking to multiple Hadoop clusters
This closes #779.
2016-08-04 14:51:03 -04:00
Mark Payne bc6b22389b NIFI-2407: Fixed NPE 2016-08-04 14:50:30 -04:00
Jeff Storck 1ea49ad8cf NIFI-2407 Implements EL support on some properties of processors/services
- Added EL support to "Directory" property of ListHDFS and GetHDFS processors
 - Added EL support to "Destination" property for ConsumeJMS and PublishJMS processors
 - Added EL support to "MQ ConnectionFactory Implementation", "MQ client library path", "Broker URI" properties for the JMS Connection Factory Provider
 - Added EL support to "Database Connection URL", "Database Driver Class Name", "DB Driver jar url", "DB username", and "DB password" properties for the DBCP Connection Pool
 - Removed NotificationServiceManagerSpec.groovy, previous PR from Joe Witt covers the needed testing for Variable Registry/Value Lookup at this time
This closes #778.
2016-08-04 14:23:29 -04:00
Bryan Rosander 0d730c5cc1
NIFI-2435 - Wrapping reader call in doAs if necessary
This closes #780.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-04 14:01:05 -04:00
Matt Gilman 321a2398ba
NIFI-2467: - Resetting cached search term when closing the dialog to ensure the same search time works upon re-opening.
NIFI-2467: - Fixing issue preventing the selection of an autocomplete result.

This closes #784.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-04 11:22:01 -04:00
Matt Burgess 3943d72e95
NIFI-1868: Incorporate PutHiveStreaming review comments
This closes #706.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-04 10:05:56 -04:00
Matt Burgess 59659232c7
NIFI-1868: Downgrade to Hive 1.2.1 and remove ConvertAvroToORC
Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-04 10:05:45 -04:00
Matt Burgess c2019b9339
NIFI-1868: Add PutHiveStreaming processor
Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-04 10:05:44 -04:00
Mark Payne cda4310ad8 NIFI-2035: Verify existence of source and destination when creating a connection
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #690
2016-08-04 09:11:25 -04:00
Koji Kawamura 7a7165152e NIFI-2478 Fixed Zk leaders path to include root. 2016-08-04 08:19:04 -04:00
Scott Aslan ed5c44ba53 [NIFI-2053] update open/close palettes. This closes #775 2016-08-04 07:59:13 -04:00
Bryan Rosander fa4c6ab03c
NIFI-2193 - Added functionality to automate certificate generation, keystore and truststore generation, and nifi.properties keystore and truststore password population. Follow-on changes will be made under NIFI-2476.
This closes #695.

Signed-off-by: Andy LoPresto <alopresto@apache.org>

Defaulting to same keyStore, key password (+18 squashed commits)
Squashed commits:
[9d01ba0] NIFI-2193 - Fixing typo
[55440bc] NIFI-2193 - Standalone can run as long as there are no conflicting files/folders
[0ca34ed] NIFI-2193 - Fixing some filename, absolute path issues
[9d4f65b] NIFI-2193 - Incorporating feedback
[f7550b4] NIFI-2193 - Cleaning up imports
[59a7637] NIFI-2193 - Updating umask to allow owner to execute
[cf824e7] NIFI-2193 - Moving DN arg to CA service specific parent class
[921ee13] NIFI-2193 - Making keystore getInstance more consistent
[a283c4b] NIFI-2193 - Updating sample config files in assembly to reflect new structure
[8d3a21d] NIFI-2193 - Making TlsHelper static, adding option to use same password for Key, KeyStore
[b13d247] NIFI-2193 - Addressing PR feedback
[46ef8ed] NIFI-2193 - Removing commons-logging, log4j from notice
[d4cf41a] NIFI-2193 - Adding option to specify output file for CA certificate when using cli client
[b74bf25] NIFI-2193 - Removing Bouncy Castle from notice
[6e34f9a] NIFI-2193 - Adding CLI client for easier generation of client certificates
[2924fca] NIFI-2193 - nifi-toolkit-ssl -> nifi-toolkit-tls, removing unused constants
[886167e] NIFI-2193 - Adding slf4j to avoid runtime issue
[082de46] NIFI-2193 - Command line SSL config utility as well as certificate authority client/server
2016-08-03 21:34:11 -07:00
Mark Payne f0401e4774 NIFI-2419: Ensure that if a node is disconnected that we unregister for 'cluster coordinator' and 'primary node' roles by updating FlowController to know that it is disconnected. Also removed dead code that was needed in the master-worker clustering paradigm but not for zero-master-clustering
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #739
2016-08-03 23:38:50 -04:00
Matt Gilman c26398eaba NIFI-2402: - Removing client side check component move eligibility and instead relaying on verification server side. Cannot check client side as the current user may not have permissions to inspect required fields.
This closes #750

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-08-03 19:08:25 -04:00
Mark Payne 83a23f90d4
NIFI-2466: Added option to provide separate key password to StandardSSLContextService.
Fixed NPE (+2 squashed commits)
Squashed commits:
[c5d521a] NIFI-2466: Added unit test to verify changes; fixed validation
[aa4d418] NIFI-2446: Add option to specify key password when different than keystore password

This closes #776.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-08-03 15:56:12 -07:00
Matt Gilman 1511887a68 NIFI-2301: - Ensure all component specific policies are removed when the component is removed. - Allowing snippets to be created if the user has read or write access as we don't know what the intended snippet usage. When used the snippet is still authorized accordingly. - Ensuring actions involving Process Groups correctly authorize encapsulated components. - Not requiring read permissions when showing the delete button for Controller Services and Reporting Tasks.
This closes #757

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-08-03 18:06:30 -04:00
Matt Gilman 9338f102cb NIFI-2237:
- Updating Rest Endpoint documentation specifically regarding access policies.
- Ensuring the resource listing is accurate.
- Removing unnecessary code.
2016-08-03 16:18:30 -04:00
Matt Gilman fca59ff9d0 NIFI-2438:
- If a node reports forbidden, using an appropriate response failure exception.
This closes #760.
2016-08-03 15:01:04 -04:00
Bryan Bende 698cde69ba NIFI-2453 Making FileAuthorizer perform initial seeding when users and groups are already present
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #772
2016-08-03 11:59:03 -04:00
Mark Payne 16348b071d NIFI-2452: This closes #771. Ensure that we keep track of how many references we have to each lucene searcher and only close the underlying index reader if there are no references to the searcher. Also updated to prefer newer provenance events over older provenance events, and calculate FlowFile lineage based on an event id instead of a FlowFile UUID, as it's much more efficient 2016-08-03 08:05:26 -07:00
Mark Payne e9b87dd734 NIFI-2452: Ensure that we do not close Index Readers that are still in use 2016-08-03 07:54:10 -07:00
Mark Payne bc5237593e NIFI-2291: Correct the Content URI for 1.0.0 REST API; added cluster node identifier & whether or not clustered to ReportingContext so that the Reporting Task could make use of it
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #752
2016-08-03 10:35:11 -04:00
Oleg Zhurakousky 54549891e3 NIFI-2444 NIFI-2445 fixed PublishKafka
- fixed the logging issue NIFI-2444 by ensuring the ProcessLog is added to KafkaPublisher
- fixed KafkaPublisher's isAllAcked operation to ensure that it properly reports that the flow file has failed.
- added additional test
This closes #758.
2016-08-02 20:30:34 -04:00
Matt Gilman 52d97f966d NIFI-2443: - Addressing issue where reload group deferred was being resolved prematurely.
This closes #756

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-08-02 17:06:25 -04:00
jpercivall 2a6580992b NIFI-2455 Removing 'Queue Buffering Max Time' as a required property of PutKafka. This closes #765 2016-08-02 17:01:58 -04:00
Matt Gilman ecca76099a NIFI-2427: - Only showing read/write data policies for applicable components. Disabling for labels, controller services, reporting tasks, and templates.
This closes #748

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-08-02 16:25:02 -04:00
Bryan Bende a1bb94c08a NIFI-1869 Cloning controller services when referenced by reporting tasks and upgrading from 0.x to 1.0.0. This closes #767 2016-08-02 16:09:54 -04:00
Oleg Zhurakousky df11e1d2c0 NIFI-2425 fixed ControllerService referebcing in DTOs/Templates. This closes #770 2016-08-02 13:58:19 -04:00
joewitt c10d11d378 NIFI-2339 made exception statements more vague and generally limited to identifiers only to avoid any authorization issues. This closes #764 2016-08-02 12:21:43 -04:00
joewitt 6bcc415eb8 NIFI-2457 removed old state file mechanism and fixed state reset logic
This closes #768.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-02 12:11:43 -04:00
joewitt e23b235617 NIFI-2028 This closes #714. This closes #751. fixed typo in classname 2016-08-02 09:13:41 -04:00
Koji Kawamura aae2d27879 NIFI-2259: HTTP Site-to-Site can't handle DEST_FULL
HTTP Site-to-Site can't handle TRANSACTION_FINISHED_BUT_DESTINATION_FULL
scenario as expected.

That happens if the remote NiFi's input port destination relationship
becomes full during Site-to-Site client sends data. The data which has
already sent to the remote NiFi has to be committed successfully.
However, the remote NiFi returns 503 as a response of commit HTTP
request. Because it does check port availability.

The port availability check shouldn't be called at commit request, since
the session at source NiFi has already been committed. The remote NiFi
should commit its session as well, and return
TRANSACTION_FINISHED_BUT_DESTINATION_FULL response.

This fix makes a remote NiFi to keep the handshaken properties when it holds
transaction to be committed. Then if a transaction already has
handshaken properties, then use it, instead of doing a handshake process
again.
2016-08-02 09:08:00 -04:00
Koji Kawamura 809f042353 NIFI-2028: Fixed Site-to-Site Transit URI
Fixed Site-to-Site Transit URI for HTTP to be consistent with RAW socket.

- Removed url from CommunicationsSession since it's redundant as we have
  Peer.url, too. The value was not used from anywhere other than HTTP
Site-to-Site.
- Added createTransitUri method in Communicant interface, so that
  implementation can customize transitUri while providing consistent
interface.
2016-08-02 09:08:00 -04:00
Mark Payne 09840027a3 NIFI-2304: If node address is not set, default to localhost, as the admin guide says it should
This closes #688

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2016-08-02 12:09:34 +09:00
Koji Kawamura 9a2f8c598e NIFI-2441 This closes #766. View state fails with a JS error
Removed permission check causing "Cannot read property 'canRead' of
undefined". A given user won't have record level permission
difference for component state. It's not required here.
2016-08-01 22:54:37 -04:00
joewitt 05a99a93cb NIFI-2208 This closes #754. refactored as per comments on JIRA. Reduced API expsosure and tightened lifecycle management. 2016-08-01 14:17:26 -04:00
Scott Aslan 7f45251bbd [NIFI-2426] Update visibility of New service/task button in Controller Settings
- Ensuring button is hidden when the user doesn't have permissions to modify the controller.
- This closes #759
2016-08-01 13:32:42 -04:00
Scott Aslan 5d63ba6a19 NIFI-2303 This closes #749. Update lineage styles 2016-08-01 13:30:58 -04:00
Pierre Villard 8c011d91e1 NIFI-2060 - fixed unit tests to be compatible with 2.7.x
This closes #547
2016-08-01 12:46:51 -04:00
Matt Gilman 372a316deb NIFI-2401:
- Returning the enabled state of play and stop buttons in the palette. Allowing the user to always press if they have permissions to modify the selection.
 - Only attempting to schedule components that are not running, not disabled, and valid.

This closes #745

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-08-01 11:39:55 -04:00
Bryan Bende bfdf94d522 Removing extraneous authorizations.xml from nifi-resources/src/main/resources/conf, created automatically in secure mode 2016-08-01 11:07:26 -04:00
Oleg Zhurakousky 1bf10944ea NIFI-2366 - Fixed ID generation semantics in clustered environment
- added SnippetUtilsTest
- renamed TypeOneUUIDGenerator to ComponentIdGenerator

- changed lsb part of ComponentIdGenerator back to long
- Fixed 'isCopy' condition for clustered environments

This closes #718.
2016-07-31 15:24:02 -04:00
Yolanda M. Davis 8412d2662a NIFI-2208 - initial commit Custom Property Expression Language support with Variable Registry, includes bug fix for NIFI-2057
This closes #529

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-29 17:10:20 -04:00
Matt Burgess b213ed95e0 NIFI-2422: Fix SelectHiveQL handling of Number types
This closes #744.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-07-29 15:38:32 -04:00
Matt Gilman a6d259ee01 NIFI-2245:
- If a single response exists using it, instead of attempting to merge anything.

This closes #743.
2016-07-29 15:29:06 -04:00
Scott Aslan 0c8fc25ba4 NIFI-2430 This closes #737. Update some tooltip text and context menu text/icon 2016-07-29 12:54:31 -04:00
Scott Aslan a8224edb8d [NIFI-2428] update modal glasspane to always close, even if the dialog we closed by user in close handler 2016-07-29 12:12:00 -04:00
Scott Aslan f5dfa580db [NIFI-2348] When processor is selected, can now copy id from palette 2016-07-29 12:12:00 -04:00
Scott Aslan da5e468489 [NIFI-2371] fix cluster page padding 2016-07-29 12:12:00 -04:00
Matt Gilman a73c8bba30 NIFI-2302: - Showing a message on the history page when in a cluster to indicate that only the actions from the current node are displayed.
This closes #740

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-29 11:27:09 -04:00
Mark Payne 70d70732b5 NIFI-2388: Addressed a spot that assumes that Process Group is always set on StandardControllerServiceNode 2016-07-29 11:02:52 -04:00
Mark Payne bc0b5fe154 NIFI-2431: Before registering for the Cluster Coordinator role, check if another node already has this role. If so, do not register for this role until after the node has joined the cluster and inherited the flow. 2016-07-29 09:42:38 -04:00
Mark Payne cfc8a9613c NIFI-2395 This closes #734. Ensure that if we fail to index provenance events we do not prevent the repo from continuing to merge journals 2016-07-29 09:33:47 -04:00
Mark Payne cddbe7d41f NIFI-2388 This closes #728. Use null for group id if no group specified instead of throwing NPE 2016-07-29 09:14:50 -04:00
Mark Payne 3a153d3a48 NIFI-2337 This closes #731. Turn off logging for org.apache.curator.framework.recipes.leader.LeaderSelector since it logs only a single error, and that error is a but in Curator 2016-07-28 22:29:08 -04:00
Matt Gilman da238b16ef NIFI-2122: - Merging responses for the current user and the flow configuration. - Returning whether NiFi is configured with a policy based authorizer in the flow configuration. - Only showing the users and policy icons when configured with a policy based authorizer. - Failing faster when invoking the users or policies endpoint when not configured with a configurable authorizer.
This closes #736

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-28 16:51:40 -04:00
Bryan Bende 5e4ba04589 NIFI-2390 Separating of users and groups from authorizations.xml into separate file. This closes #735 2016-07-28 16:41:52 -04:00
Matt Gilman 01adb050f9 NIFI-2421: - Only attempting to clone policies when NiFI supports a configurable authorizer.
This closes #738

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-28 13:01:20 -04:00
Matt Gilman 09b124714e NIFI-2347: - Ensuring component specific policies are retained when using copy/paste. - This includes the policies for the component, data of the component, data transfers of the component, and policies of the component.
This closes #730

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-28 09:43:40 -04:00
Scott Aslan 25cadf5db1 [NIFI-2396] Update Bulletin Board shell header and filter control position. This closes #732 2016-07-28 09:09:25 -04:00
Bryan Bende dedd4fcda1 NIFI-2403
- Ensuring uniqueness on user identities and group names
- Ensure errors when updating a group are displayed.
- This closes #724
2016-07-28 08:45:47 -04:00
Matt Gilman ae344806c0 NIFI-2224: - Ensuring the template form is reset when the upload template dialog is closed. NIFI-2175: - Not submitting the template form is a template isn't choosen. NIFI-2176: - Ensuring a template is specified during creation. NIFI-2223: - Ensuring templates with the same name cannot be added. NIFI-2296: - Updating the tooltip for the upload template browse button.
- Cleaning up un-used parameters to REST endpoints.

This closes #725

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-27 17:38:12 -04:00
Scott Aslan f3e49fefa0 [NIFI-2367] Overlapping links repositioned on error/login pages
[NIFI-2025] update birdseye after dragging/dropping element on the canvas
[NIFI-2367] update width of content viewer combo
[NIFI-2355] update table sorting based on auth efforts
[NIFI-2027] update EL editors
[NIFI-2387] update bulletin alert backgroud color to actually change the color of the icon not the div background
[NIFI-2141] Hide bulletin icon/background on processors unless a bulletin exists
[NIFI-2400] close any open combos contained within a shell when closing the shell
[NIFI-2404] remove extra scrollbar from #node-events
[NIFI-2027] account for min widths of EL editors
[NIFI-2025] update birdseye after changing color
[NIFI-2027] Update EL editor checkbox text
[NIFI-2027] update checkbox text
[NIFI-2355] update table sorting
[NIFI-2141] Hide bulletin icon/background on processors unless a bulletin exists
[NIFI-2027] update EL editors
[NIFI-2367] Overlapping links repositioned on error/login pages
This closes #715
2016-07-27 16:29:25 -04:00
joewitt 15d1e1bbf8 NIFI-2405 corrected string equality checks
This closes #723.
2016-07-26 23:04:38 +02:00
Bryan Bende c3b4872b55 NIFI-2389 Refactoring identity mapping and applying it to FileAuthorizer for initial admin, cluster nodes, and legacy authorized users. This closes #719 2016-07-26 15:24:50 -04:00
Mark Payne 52bc23f5db NIFI-2316, NIFI-2318: Ensure that we do not save the flow before initializing the Run Status of components. Clarify the Node Event messages
This closes #678

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-26 14:24:32 -04:00
Matt Gilman 69586d8bd0 NIFI-2346:
- Introducing data resource for authorizing provenance events and queue listing.
- Authorizing entire proxy chain for data resource and data transfer resource.
NIFI-2338:
- Ensuring that replay authorization only happens once.

- Allowing users with access to policies for a component to be able to access all policies for that component.
-- Includes the component, data, data transfers, and policies.
- Fixing drop request completion to update the correct queued field.
- Fixing access control check for listing and emptying queues.
- Reseting selected policy when re-opening the policy management page.
- Fixing button/link visibility for available actions in policy management page.
- Fixing policy issues with policy removal when the underlying component is deleted.
- Updating file authorizer seeding to grant data access to node's in the cluster.

This closes #720.
2016-07-26 14:15:36 -04:00
Bryan Bende c27763a12f NIFI-1733 Initial commit for nifi-ranger-bundle
- Adding Authorizer implementation for Ranger
- Adding build profile and assembly that controls the inclusion of Ranger in the final assembly
- Add properties to specify ranger admin identity and a flag to indicate if ranger is using kerberos, plugin is updated to perform a UGI login if ranger is using kerberos
- Adding LICENSE and NOTICE for Ranger NAR, and some other licensing clean up
- Adding tests for kerberos properties, adding test for RangerBasePluginWithPolicies, cleaning up code to use Java 8 features

This closes #574
2016-07-26 10:15:22 -04:00
Oleg Zhurakousky 2ec43076fa NIFI-2344 This closes #711. fixed StandardControllerServiceNode.getRequiredControllerServices 2016-07-26 00:48:24 -04:00
Bryan Bende 67876f707b NIFI-2399 Correcting comparison of maxEventId against lastEventId in SiteToSiteProvenanceReportingTask 2016-07-25 23:37:08 -04:00
Mark Payne 7779af69b4 NIFI-2292: Funnel all cluster node status changes through the cluster coordinator instead of having each node broadcast changes to the whole cluster. This gives us the ability to increment the updateId consistently without race conditions.
This closes #717

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-25 23:17:37 -04:00
Bryan Bende 3e9867d5da NIFI-1950 Updating FileAuthorizer to convert access controls from input and output ports during legacy conversion. This closes #702. 2016-07-25 12:37:26 -04:00
Mark Payne b082858595 Revert "NIFI-2292: Funnel all cluster node status changes through the cluster coordinator instead of having each node broadcast changes to the whole cluster. This gives us the ability to increment the updateId consistently without race conditions."
This reverts commit 35ff0975bf.
2016-07-25 12:35:43 -04:00
Mark Payne 35ff0975bf NIFI-2292: Funnel all cluster node status changes through the cluster coordinator instead of having each node broadcast changes to the whole cluster. This gives us the ability to increment the updateId consistently without race conditions. 2016-07-25 12:14:13 -04:00
Scott Aslan 26ffc6d7ec [NIFI-2050] update dialog advanced buttons
[NIFI-2076] fix Add inline controller service dialog description width
[NIFI-2143] update to combo width
[NIFI-2136] [NIFI-2139] [NIFI-2134] update provenance details dialog
[NIFI-2368] add transparent glasspane when in a shell
[NIFI-2134] update flow-file details padding and .unset font-weight
[NIFI-2349] [NIFI-2351] update policies shell styles
[NIFI-2350] update empty queue dialog styles
[NIFI-2367] update links on message pane
This closes #703
2016-07-25 08:24:23 -04:00
Mark Payne 4e08ea6525 NIFI-2376 This closes #713. Ensure that we don't decrement claimant count more than once when append() throws an Exception 2016-07-24 17:00:21 -04:00
Mark Payne 6932a53ec9 NIFI-2360: Leave ZooKeeper running when a node is disconnected. Do not allow the last node in the cluster to be disconnected. Change ClusterProtocoLHeartbeater to use RetryNTime retry strategy instead of RetryForever because web requests could block on this
This closes #705

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-22 23:57:00 -04:00
joewitt 393a3925dd NIFI-2253 flexibly cleaning zookeeper connect string
This closes #704

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-22 18:24:52 -04:00
Joey Frazee 4179ce6644 NIFI-2090 This closes #669. Added options for segment names, parse fields in ExtractHL7Attributes 2016-07-22 10:11:03 -04:00
Matt Gilman 4a4d60e6af NIFI-2307: - Enforcing connection permissions based on the source and destination comonent. - Removing connection specific access policies. NIFI-2265: - Filtering out sensitive details in component status and status history when appropriate. NIFI-1800: - Adding parent process group id to the Controller Services table. NIFI-2077: - Removing some old un-used icons following the UI refresh. NIFI-2242: - Requiring write permissions for all components in a selection. NIFI-2080: - Updating style of the name in the selection context to handle scroll bars and use available width. NIFI-2331: - Addressing issue when removing a user/group which was causing the tenant policy to be removed. NIFI-2335: - Ensuring the flow is saved after starting/stopping a process group. NIFI-2235: - Ensuring we use consistent conditions between the context menu and the operate palette.
- Allowing users with read only access to the tenants page.
- Fixing current user integration test.
- Ensuring schedule methods are locked appropriately.
- Addressing comments from PR.

This closes #698

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-21 23:52:01 -04:00
Oleg Zhurakousky 0dbba811f3 NIFI-2298 This closes #687 added Kafka consume attributes to a FlowFile 2016-07-21 22:56:24 -04:00
jpercivall 4e224c283f NIFI-1899 reworking bi-directional Email processing in ListenSMTP
This closes #483
2016-07-21 10:39:21 -04:00
Andre F de Miranda 4f672832c0 NIFI-1899 - Introduce ExtractEmailAttachments and ExtractEmailHeaders processors - Introduce ListenSMTP (allows NiFi to receive data via email) - Addresses @ijokarumawak and @@JPercivall PR comments
Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-21 10:26:30 -04:00
Scott Aslan f352ea10b7 [NIFI-2107] [NIFI-2143] Update content-viewer, error, and login pages...Fix Provenance filter combo text. This closes #684 2016-07-21 08:22:22 -04:00
Mark Payne 5eba2b763e NIFI-2334: Delegate requests for cluster info to the cluster coordinator
This closes #697

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-20 21:38:53 -04:00
Yolanda M. Davis 35c0e1436e NIFI-2310 - upgraded jolt to 0.0.21 to resolve special character issue
This closes #674

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2016-07-21 10:35:42 +09:00
Mark Payne 370d88eeeb NIFI-2333: Consider reporting tasks and controller services when determining whether or not flow is empty
This closes #693

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-20 20:06:05 -04:00
Mark Payne 10b91ffe6c NIFI-2332: Ensure that bulletin node address is set when clustered but not overwritten if already set
This closes #692

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-20 19:44:32 -04:00
Mark Payne 7f2bda29d9 NIFI-2289: Ensure that we include Node ID's of all nodes when finding cluster coordinator, even if the node is currently still connecting or has not yet joined the cluster, which ccan be the case if all nodes in the cluster are restarting at the same time.
This closes #681

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-20 16:50:25 -04:00
Mark Payne 39a4d84a62 NIFI-2330: Ensure that we set ProcessGroupDTO's contents after populating sets of components, as the contents dto now copies the sets
This closes #686

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-20 16:05:42 -04:00
joewitt 2a8be95480 NIFI-2326 This closes #685. fixed test breaking static class causing ordering issues. Addressed potential issue in NarClassLoaders for multi-init scenarios - now idempotent for a given config 2016-07-20 15:13:37 -04:00
mans2singh 259f5bba47 NIFI-1972 - Ignite processor
This closes #502.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
2016-07-20 18:39:19 +02:00
Mark Payne 2a7f135f1c NIFI-2319: Ensure that when we set cluster node id's and node addresses, that we do so only if they are not already populated
This closes #680

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-20 11:39:57 -04:00
Mark Payne 108c815988 NIFI-2324: Log number of requests per URI if we have too many outstanding requests to replicate
This closes #683

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-20 11:22:24 -04:00
Scott Aslan 8f23afde50 [NIFI-2257] [NIFI-2294] refresh updateattribute processor advanced shell and fix connection config setting tab scrolling issue
[NIFI-2270] Update splash screen
This closes #646
2016-07-20 06:59:30 -04:00
Matt Burgess f1ba240326 NIFI-2156: Add ListDatabaseTables processor
This closes #642

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-19 13:00:43 -04:00
Bryan Bende 8e2663c54d NIFI-2309 Correcting AmbariReportingTask so it reports total task duration as seconds and as nano-seconds
This closes #672

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-19 11:35:09 -04:00
Scott Aslan deb2819bfb [NIFI-2105] [NIFI-2106] fix status history dialog issues. This closes #673 2016-07-19 07:33:34 -04:00
Mark Payne 41c0f19e48 NIFI-2305: Do not run processors that are marked as Primary Node Only if disconnected from cluster
This closes #667

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-18 15:54:14 -04:00
Oleg Zhurakousky f4d2919955 NIFI-826 (part deux)
- fixed clustering issues discovered after NIFI-826 was applied
2016-07-18 15:49:24 -04:00
Matt Gilman aa91032cde NIFI-2272:
- Ensuring the appropriate visibilty of the action in the policy management page.
NIFI-2273:
- Ensuring we load the policy or inform the user of the appropriate permissions of the effective policy.
NIFI-2239:
- Providing help tooltips for the policies in the management page.
NIFI-2283:
- Adding auditing for access policies, users, and groups.
NIFI-2263:
- Not replicating history requests throughout the cluster.
NIFI-2096:
- Fixing upload template file input in Firefox.
NIFI-2301:
- Removing relevant policies after component deletion.
2016-07-18 15:05:54 -04:00
Mark Payne 5c8636edf4 NIFI-2289: Directly ask ZooKeeper which node is cluster coordinator and add watches on the ZNode rather than relying on Node Status Updates over the cluster protocol because cluster protocol may get the events out-of-order
This closes #665.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-07-18 10:13:42 -04:00
Matt Burgess 01cae23745 NIFI-2157: Add GenerateTableFetch processor
This closes #645

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-17 17:54:42 -04:00
Matt Burgess 1c5975da86 NIFI-2174: Fix handling of time/date types in QueryCassandra
This closes #661

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-16 13:31:21 -04:00
Oleg Zhurakousky 52a961873b NIFI-826 This closes #617. Added deterministic template support 2016-07-15 20:41:18 -04:00
Mark Payne 4d4c525d9c NIFI-2281: Allow connections to be moved away from running local input ports and funnels
This closes #662

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-15 16:36:19 -04:00
Mark Payne c81dc1959a NIFI-1992:
- Updated site-to-site client and server to support clustered nifi instances
NIFI-2274:
- Ensuring we use the correct URI when updating a connection.

This closes #530
2016-07-15 16:13:59 -04:00
Joe Gresock b67c9b6f76 NIFI-2250: Enabling EL on GetSQS Queue URL
Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-15 12:59:37 -04:00
Koji Kawamura e89ee11199 NIFI-619: Make MonitorActivity more cluster friendly
With this commit, MonitorActivity can be configured as:

- Monitor activity per node individually
- Monitor cluster wide activity
  - Send notification flow-file from all of nodes
  - or only from a primary node

Changes:
- Added 'Monitoring Scope' property
- Added 'Reporting Node' property
- Falls back from cluster scope to node scope if necessary
- Persist the latest activity as Cluster state
- Examine cluster state on each node if necessary
- Only update the cluster state if current timestamp is later than existing
  timestamp stored in Zookeeper

This closes #575

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-15 12:47:37 -04:00
Matt Gilman 5cd5a4ce78 NIFI-2260:
- Addressing issue searching provenance on a specific node.
- Fixing issues viewing content and replaying events.

This closes #658.
2016-07-15 12:14:24 -04:00
Koji Kawamura 53326c7f9b NIFI-2267: A way for Processor to know node type
- Added NodeTypeProvider to expose flowController's isClustered and
  isPrimaryNode so that processor can know if it's running on a cluster
  and if it's a primary node.
- Added mechanism to simulate clustered or not, and primary or not, for testing

This closes #657

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-15 11:38:50 -04:00
Bryan Bende afc038d2c0 NIFI-2194 This closes #659. Caching metrics in AmbariReportingTask so each iteration sends last iteration's metrics 2016-07-15 11:25:22 -04:00
jpercivall 54574e3889 NIFI-2173 Fixing typo in LDAP referral strategy
This closes #629
2016-07-15 09:55:37 -04:00
Aldrin Piri d1129706e2 NIFI-1896 This closes #650. Refactored nifi-api into nifi-framework-api and other locations. The nifi-api is specific to that which is needed for intended extension points. 2016-07-14 18:24:48 -04:00
Yolanda M. Davis 048ba5366c NIFI-2020 - initial commit for custom transformation support
NIFI-2020 - updates to use lambdas/stream wherever possible and fix potential nullpointer issue.

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #564
2016-07-14 18:04:14 -04:00
Bryan Rosander 1da18a3f40 NIFI-2222 - Fixing SslContextFactory Properties set by keyStorePasswd and keyPasswd
This closes #632

Signed-off-by: James Wing <jvwing@gmail.com>
2016-07-14 14:07:24 -07:00
Andy LoPresto 4e3580f58e
NIFI-1688 Resolved issue in PostHTTP -- now uses SSLContextService's protocol setting. Modified construction of SSLSocketFactory to avoid hardcoding supported protocol. Added debug information for supported cipher suites.
This closes #624.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-07-14 16:43:48 -04:00
Puspendu Banerjee 6d2168049a fixes NIFI-1989
This closes #514

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-14 15:59:45 -04:00
James Wing a7c5b88a47
NIFI-2062 AWS SDK Update:
* Updated the AWS SDK dependency version to 1.11.8
* Removed GetDynamoDBTest references to JsonObject class dropped from AWS SDK
* Fixed integration test ITPutSNS.testPublish() - exception from not evaluating expressions
* Ignored integration test ITPutS3Object.testS3MultipartAgeoff() - did not work before or after SDK update

This closes #562.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-07-14 15:27:26 -04:00
Mark Payne f6136b69e5 NIFI-1413: Rebased against master and updated templates endpoint merge to use template entities instead of template dto's
This closes #649
2016-07-14 15:19:47 -04:00
Mark Payne 329192a7d3 NIFI-1413: Ensure that if a node's templates don't match the clusters that we take the following actions: -Local templates remain but aren't shown in the cluster's templates. -Any templates from the cluster that don't exist on the node are added to the node. -Any conflicting template definitions are replaced by those in the cluster
This closes #596
2016-07-14 15:01:46 -04:00
Scott Aslan 02784b5fca [NIFI-2227] fix race condition for showing the view button on the flow file details dialog
[NIFI-2109] update check boxes
[NIFI-2226] reset controller services properties table size on dialog open
This closes #653
2016-07-14 14:50:14 -04:00
Mark Payne 2540573d77 NIFI-2087: Do not update the lucene index to delete expired events. Instead, just wait for the entire index to expire and remove the files for that index. This is much more efficient.
This closes #652

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-14 14:49:30 -04:00
Mark Payne b83d9bde1a Fixed checkstyle problem 2016-07-14 14:01:28 -04:00
Matt Gilman 4f26072444 NIFI-2261
- Addressed issue enabling/disabling controller services where the wrong URI was referenced.
- Addressed with the update revisions in the controller service references.
- Addressed issue with showing the disconnected from cluster dialog on page load.
- Addressed issue with URI when adding a dynamic property.

This closes #654.
2016-07-14 13:59:26 -04:00
Matt Gilman 3373e18158 NIFI-1947 NIFI-2082:
- Making it more obviously when a nodes cluster state changes.
- Showing which node is the cluster coordinator.

This closes #651
2016-07-14 11:31:48 -04:00
Koji Kawamura 30889995cb NIFI-2145: Auto flow.xml archive
- Added following properties:
  - nifi.flow.configuration.archive.enabled
  - nifi.flow.configuration.archive.max.time
  - nifi.flow.configuration.archive.max.storage
- Removed manual archive operation:
  - Removed 'Back-up flow' link from UI since it's not needed any longer
  - Removed corresponding REST API controller/archive and its
    implementations
- Added FlowConfigurationArchiveManager to enclose archive related code
- Updated related docs
2016-07-14 10:35:16 -04:00
joewitt f987b21609 NIFI-1157 searched for and resolved all remaining references to deprecated items that were clearly addressable. 2016-07-14 09:32:35 -04:00