Commit Graph

1100 Commits

Author SHA1 Message Date
Oleg Zhurakousky 9b647cd538 NIFI-1148 added IMAP/POP3 support added initial set of processors to support consumption of Email via IMAP/IMAPS and POP3 protocols
Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-1148 addressed PR comments from @trixpan

NIFI-1148 addressing PR comments

NIFI-1148 addressed PR comments

This closes #710
2016-08-08 11:57:28 -04:00
Mark Payne c1c052af71 NIFI-2406: Ensure that hearbeat monitor continues to run while instance is running. This way if a node sends heartbeat to this node as elected coordinator changes, we notify the node accordingly. Handle Exceptions more gracefully in leader election code. Tweaked some handling of how nodes reconnect to the cluster to ensure more stability with cluster
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #729
2016-08-08 09:15:10 -04:00
Joe Skora d094130a26 NIFI-2237:
Extend template handling to read only properties.
* Add read only property handling to type.hbs template.
* Add flag to ProcessorDTO.getRelationships to reflect read only nature of the property.
* Remove explicit "read only" message from ConnectionDTO.getAvailableRelationships to avoid duplicate text.
* This closes #806
2016-08-08 09:06:47 -04:00
Kirk Tarou e2f649cfea NIFI-2470 This closes #802.
Fixed WritesAttributes section to include all attributes with the
correct attribute names and descriptions
2016-08-07 23:58:09 -04:00
Oleg Zhurakousky 29f53c07f5 NIFI-2509 This closes #805. fixed NPE condition in KafkaConsumer 2016-08-07 23:48:57 -04:00
Joe Skora 903b1fe465 NIFI-2494 Remove extraneous text that shows on banner line. This closes #793 2016-08-05 16:03:26 -04:00
Scott Aslan f2f070c5be
NIFI-2468: Improved UI consistency.
- Addressing comments from PR. Moving new service button below the tab pane. (+2 squashed commits)
Squashed commits:
[f746d09] NIFI-2468:
- Addressing numerous inconsistencies throughout the UI (button placement above tables, timestamp styles, etc)
[7d1fb09] addressing some inconsistencies

This closes #794.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-08-05 12:32:06 -07:00
Mark Payne 7ffa30d21b NIFI-2423: Make use of the SSLContextService to provide SSL information
Signed-off-by: Oleg Zhurakousky <oleg@suitcase.io>
2016-08-05 14:14:38 -04:00
Oleg Zhurakousky c39a127ec8 NIFI-2322, NIFI-2423, NIFI-2412 Kafka improvements
- Fixed KafkaConsumer's connection block when broker is not available
- Fixed Serializer/Deserializer configs in both Consume/Publish Kafka
- Added sensitive properties for SSL ket/trust stores

NIFI-2322 fixed tests
2016-08-05 14:14:38 -04:00
Mark Payne 8d34223265 NIFI-2472: Normalize coordinates for all components in a snippet, not just processors
This closes #786

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-08-04 19:46:39 -04:00
Matt Gilman c0a253568e NIFI-2486: - Authorizing individual bulletins being returned through the bulletin board.
This closes #792

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-08-04 18:41:18 -04:00
Matt Gilman 04147ac22a
NIFI-2458: - Fixing issue with timezone when the initial provenance result was cancelled.
This closes #773.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-04 17:09:21 -04:00
Mark Payne 0e0166cec5
NIFI-2483: Removed 'passthrough' relationship
This closes #789.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-04 16:12:21 -04:00
Mark Payne 7ba10a6dea
NIFI-2484: Ensure that if a Processor throws an Exception from an @OnScheduled method that @OnStopped methods get called
This closes #791.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-04 15:34:10 -04:00
Bryan Bende aca2a92d6b
Fixing contrib-check in GetHDFSSequenceFileTest 2016-08-04 15:08:08 -04:00
Mike Moser b7b1dc2fe6 NIFI-2471 fix Hadoop configuration resources when talking to multiple Hadoop clusters
This closes #779.
2016-08-04 14:51:03 -04:00
Mark Payne bc6b22389b NIFI-2407: Fixed NPE 2016-08-04 14:50:30 -04:00
Jeff Storck 1ea49ad8cf NIFI-2407 Implements EL support on some properties of processors/services
- Added EL support to "Directory" property of ListHDFS and GetHDFS processors
 - Added EL support to "Destination" property for ConsumeJMS and PublishJMS processors
 - Added EL support to "MQ ConnectionFactory Implementation", "MQ client library path", "Broker URI" properties for the JMS Connection Factory Provider
 - Added EL support to "Database Connection URL", "Database Driver Class Name", "DB Driver jar url", "DB username", and "DB password" properties for the DBCP Connection Pool
 - Removed NotificationServiceManagerSpec.groovy, previous PR from Joe Witt covers the needed testing for Variable Registry/Value Lookup at this time
This closes #778.
2016-08-04 14:23:29 -04:00
Bryan Rosander 0d730c5cc1
NIFI-2435 - Wrapping reader call in doAs if necessary
This closes #780.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-04 14:01:05 -04:00
Matt Gilman 321a2398ba
NIFI-2467: - Resetting cached search term when closing the dialog to ensure the same search time works upon re-opening.
NIFI-2467: - Fixing issue preventing the selection of an autocomplete result.

This closes #784.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-04 11:22:01 -04:00
Matt Burgess 3943d72e95
NIFI-1868: Incorporate PutHiveStreaming review comments
This closes #706.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-04 10:05:56 -04:00
Matt Burgess 59659232c7
NIFI-1868: Downgrade to Hive 1.2.1 and remove ConvertAvroToORC
Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-04 10:05:45 -04:00
Matt Burgess c2019b9339
NIFI-1868: Add PutHiveStreaming processor
Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-04 10:05:44 -04:00
Mark Payne cda4310ad8 NIFI-2035: Verify existence of source and destination when creating a connection
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #690
2016-08-04 09:11:25 -04:00
Koji Kawamura 7a7165152e NIFI-2478 Fixed Zk leaders path to include root. 2016-08-04 08:19:04 -04:00
Scott Aslan ed5c44ba53 [NIFI-2053] update open/close palettes. This closes #775 2016-08-04 07:59:13 -04:00
Bryan Rosander fa4c6ab03c
NIFI-2193 - Added functionality to automate certificate generation, keystore and truststore generation, and nifi.properties keystore and truststore password population. Follow-on changes will be made under NIFI-2476.
This closes #695.

Signed-off-by: Andy LoPresto <alopresto@apache.org>

Defaulting to same keyStore, key password (+18 squashed commits)
Squashed commits:
[9d01ba0] NIFI-2193 - Fixing typo
[55440bc] NIFI-2193 - Standalone can run as long as there are no conflicting files/folders
[0ca34ed] NIFI-2193 - Fixing some filename, absolute path issues
[9d4f65b] NIFI-2193 - Incorporating feedback
[f7550b4] NIFI-2193 - Cleaning up imports
[59a7637] NIFI-2193 - Updating umask to allow owner to execute
[cf824e7] NIFI-2193 - Moving DN arg to CA service specific parent class
[921ee13] NIFI-2193 - Making keystore getInstance more consistent
[a283c4b] NIFI-2193 - Updating sample config files in assembly to reflect new structure
[8d3a21d] NIFI-2193 - Making TlsHelper static, adding option to use same password for Key, KeyStore
[b13d247] NIFI-2193 - Addressing PR feedback
[46ef8ed] NIFI-2193 - Removing commons-logging, log4j from notice
[d4cf41a] NIFI-2193 - Adding option to specify output file for CA certificate when using cli client
[b74bf25] NIFI-2193 - Removing Bouncy Castle from notice
[6e34f9a] NIFI-2193 - Adding CLI client for easier generation of client certificates
[2924fca] NIFI-2193 - nifi-toolkit-ssl -> nifi-toolkit-tls, removing unused constants
[886167e] NIFI-2193 - Adding slf4j to avoid runtime issue
[082de46] NIFI-2193 - Command line SSL config utility as well as certificate authority client/server
2016-08-03 21:34:11 -07:00
Mark Payne f0401e4774 NIFI-2419: Ensure that if a node is disconnected that we unregister for 'cluster coordinator' and 'primary node' roles by updating FlowController to know that it is disconnected. Also removed dead code that was needed in the master-worker clustering paradigm but not for zero-master-clustering
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #739
2016-08-03 23:38:50 -04:00
Matt Gilman c26398eaba NIFI-2402: - Removing client side check component move eligibility and instead relaying on verification server side. Cannot check client side as the current user may not have permissions to inspect required fields.
This closes #750

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-08-03 19:08:25 -04:00
Mark Payne 83a23f90d4
NIFI-2466: Added option to provide separate key password to StandardSSLContextService.
Fixed NPE (+2 squashed commits)
Squashed commits:
[c5d521a] NIFI-2466: Added unit test to verify changes; fixed validation
[aa4d418] NIFI-2446: Add option to specify key password when different than keystore password

This closes #776.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-08-03 15:56:12 -07:00
Matt Gilman 1511887a68 NIFI-2301: - Ensure all component specific policies are removed when the component is removed. - Allowing snippets to be created if the user has read or write access as we don't know what the intended snippet usage. When used the snippet is still authorized accordingly. - Ensuring actions involving Process Groups correctly authorize encapsulated components. - Not requiring read permissions when showing the delete button for Controller Services and Reporting Tasks.
This closes #757

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-08-03 18:06:30 -04:00
Matt Gilman 9338f102cb NIFI-2237:
- Updating Rest Endpoint documentation specifically regarding access policies.
- Ensuring the resource listing is accurate.
- Removing unnecessary code.
2016-08-03 16:18:30 -04:00
Matt Gilman fca59ff9d0 NIFI-2438:
- If a node reports forbidden, using an appropriate response failure exception.
This closes #760.
2016-08-03 15:01:04 -04:00
Bryan Bende 698cde69ba NIFI-2453 Making FileAuthorizer perform initial seeding when users and groups are already present
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #772
2016-08-03 11:59:03 -04:00
Mark Payne 16348b071d NIFI-2452: This closes #771. Ensure that we keep track of how many references we have to each lucene searcher and only close the underlying index reader if there are no references to the searcher. Also updated to prefer newer provenance events over older provenance events, and calculate FlowFile lineage based on an event id instead of a FlowFile UUID, as it's much more efficient 2016-08-03 08:05:26 -07:00
Mark Payne e9b87dd734 NIFI-2452: Ensure that we do not close Index Readers that are still in use 2016-08-03 07:54:10 -07:00
Mark Payne bc5237593e NIFI-2291: Correct the Content URI for 1.0.0 REST API; added cluster node identifier & whether or not clustered to ReportingContext so that the Reporting Task could make use of it
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #752
2016-08-03 10:35:11 -04:00
Oleg Zhurakousky 54549891e3 NIFI-2444 NIFI-2445 fixed PublishKafka
- fixed the logging issue NIFI-2444 by ensuring the ProcessLog is added to KafkaPublisher
- fixed KafkaPublisher's isAllAcked operation to ensure that it properly reports that the flow file has failed.
- added additional test
This closes #758.
2016-08-02 20:30:34 -04:00
Matt Gilman 52d97f966d NIFI-2443: - Addressing issue where reload group deferred was being resolved prematurely.
This closes #756

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-08-02 17:06:25 -04:00
jpercivall 2a6580992b NIFI-2455 Removing 'Queue Buffering Max Time' as a required property of PutKafka. This closes #765 2016-08-02 17:01:58 -04:00
Matt Gilman ecca76099a NIFI-2427: - Only showing read/write data policies for applicable components. Disabling for labels, controller services, reporting tasks, and templates.
This closes #748

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-08-02 16:25:02 -04:00
Bryan Bende a1bb94c08a NIFI-1869 Cloning controller services when referenced by reporting tasks and upgrading from 0.x to 1.0.0. This closes #767 2016-08-02 16:09:54 -04:00
Oleg Zhurakousky df11e1d2c0 NIFI-2425 fixed ControllerService referebcing in DTOs/Templates. This closes #770 2016-08-02 13:58:19 -04:00
joewitt c10d11d378 NIFI-2339 made exception statements more vague and generally limited to identifiers only to avoid any authorization issues. This closes #764 2016-08-02 12:21:43 -04:00
joewitt 6bcc415eb8 NIFI-2457 removed old state file mechanism and fixed state reset logic
This closes #768.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-02 12:11:43 -04:00
joewitt e23b235617 NIFI-2028 This closes #714. This closes #751. fixed typo in classname 2016-08-02 09:13:41 -04:00
Koji Kawamura aae2d27879 NIFI-2259: HTTP Site-to-Site can't handle DEST_FULL
HTTP Site-to-Site can't handle TRANSACTION_FINISHED_BUT_DESTINATION_FULL
scenario as expected.

That happens if the remote NiFi's input port destination relationship
becomes full during Site-to-Site client sends data. The data which has
already sent to the remote NiFi has to be committed successfully.
However, the remote NiFi returns 503 as a response of commit HTTP
request. Because it does check port availability.

The port availability check shouldn't be called at commit request, since
the session at source NiFi has already been committed. The remote NiFi
should commit its session as well, and return
TRANSACTION_FINISHED_BUT_DESTINATION_FULL response.

This fix makes a remote NiFi to keep the handshaken properties when it holds
transaction to be committed. Then if a transaction already has
handshaken properties, then use it, instead of doing a handshake process
again.
2016-08-02 09:08:00 -04:00
Koji Kawamura 809f042353 NIFI-2028: Fixed Site-to-Site Transit URI
Fixed Site-to-Site Transit URI for HTTP to be consistent with RAW socket.

- Removed url from CommunicationsSession since it's redundant as we have
  Peer.url, too. The value was not used from anywhere other than HTTP
Site-to-Site.
- Added createTransitUri method in Communicant interface, so that
  implementation can customize transitUri while providing consistent
interface.
2016-08-02 09:08:00 -04:00
Mark Payne 09840027a3 NIFI-2304: If node address is not set, default to localhost, as the admin guide says it should
This closes #688

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2016-08-02 12:09:34 +09:00
Koji Kawamura 9a2f8c598e NIFI-2441 This closes #766. View state fails with a JS error
Removed permission check causing "Cannot read property 'canRead' of
undefined". A given user won't have record level permission
difference for component state. It's not required here.
2016-08-01 22:54:37 -04:00