Commit Graph

235 Commits

Author SHA1 Message Date
Mark Payne fff0148a0e NIFI-3129: When adding controller services to a snippet, ensure that we don't add the service multiple times, even when it's referenced by child process groups. This closes #1284 2016-11-30 13:07:51 -05:00
joewitt 92f17a995b NIFI-3100-rc2 prepare for next development iteration 2016-11-25 23:49:27 -05:00
joewitt 5536f690a8 NIFI-3100-rc2 prepare release nifi-1.1.0-RC2 2016-11-25 23:49:13 -05:00
Koji Kawamura a1ab5e844b NIFI-2729: This closes #1270. testSendSuccessWithProxy timeout in Travis
- Changed AtomicBoolean to CountDownLatch to avoid sleeping thread in
  some test cases
- Specified less number of threads for Jetty and LittleProxy than
  default to lower resource usage
- Added try catch for the specific gateway timeout case (504) so that
  test can pass even it happens while it fails with other errors
2016-11-25 19:41:15 -05:00
joewitt fb9cbccc38 NIFI-2954 This closes #1244. Moved StandardPropertyValidator to nifi-utils, documented scope/purpose of a few util libs, removed deps from nifi-utils. 2016-11-21 16:30:42 -05:00
Mark Payne 2ee66de1a6 NIFI-3066: Create unit test to reproduce bug that results in IllegalArgumentException: Cannot migrate FlowFiles from a Process Session to itself
NIFI-3066: Ensure that when a Bin is created, it is always created with its own new session

This closes #1245
2016-11-21 11:34:14 -05:00
Mark Payne 1be0871473 NIFI-2854: Refactor repositories and swap files to use schema-based serialization so that nifi can be rolled back to a previous version after an upgrade.
NIFI-2854: Incorporated PR review feedback

NIFI-2854: Implemented feedback from PR Review

NIFI-2854: Ensure that all resources are closed on CompressableRecordReader.close() even if an IOException is thrown when closing one of them

This closes #1202
2016-11-18 14:53:13 -05:00
jpercivall c4be800688 NIFI-2950 Adding support for whole number hex values and a fromRadix function
NIFI-2950 Fixing typo

This closes #1161
2016-11-17 07:27:51 -05:00
Bryan Rosander e5eda63705
NIFI-2943 - Toolkit uses JKS type over PKCS12 when creating truststore because non-Bouncy Castle providers cannot read certificates from PKCS12 truststore.
Peer review feedback (+2 squashed commits)
Squashed commits:
[0102c8e] NIFI-2943 - Peer review feedback
[9bcd495] NIFI-2943 - pkcs12 keystore improvements

1. loading pkcs12 keystores with bouncy castle everywhere
2. tls-toolkit client using jks truststore when keystore type is specified differently
3. tests

This closes #1165.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-11-16 17:13:10 -08:00
Koji Kawamura 13ea909122
NIFI-2851: Fixed CheckStyle error.
This closes #1215.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-11-14 00:44:16 -08:00
Mark Payne ad92474593 NIFI-2851: Added additional unit test to ensure correctness of demarcation when demarcator falls between buffered data
This closes #1116.
2016-11-11 15:45:40 -05:00
Oleg Zhurakousky 41f519e84c NIFI-2851 initial commit of perf improvements on SplitText
- introduced org.apache.nifi.stream.io.util.TextLineDemarcator
- refactored SplitText to use org.apache.nifi.stream.io.util.TextLineDemarcator
- updated SplitText's capability discription to provide more clarity around splits with headers.
2016-11-11 15:45:40 -05:00
Mark Payne c441a8696d NIFI-2850 This closes #1115. Added a migrate() method to ProcessSession and refactored BinFiles and MergeContent to use it 2016-11-09 16:25:03 -05:00
Bryan Bende d1d053725b NIFI-2909 Adding per-instance class loading capability through @RequiresInstanceClassLoading annotation
NIFI-1712 Applying per-instance class loading to HBaseClientService to allow specifying Phoenix Client JAR

-Refactoring the ClassLoading so that every processor, controller service, and reporting task gets an InstanceClassLoader with a parent of the NAR ClassLoader, and only components with @RequiresInstanceClassLoading will make a copy of the NAR ClassLoader resources, and addressing some review feedback

This closes #1156
2016-11-08 13:21:27 -05:00
Oleg Zhurakousky 5fd4a55791 NIFI-2778 added abilty to interrupt Lucene search
polishing

This closes #1138
2016-11-07 16:28:16 -05:00
James Wing a486fefb1e NIFI-2115 Detailed Version Info in About Box
* Java version and vendor
* OS name and version
* Release Tag
* Build revision (commit SHA), branch, and timestamp
* Handles formal releases, ad-hoc builds, and non-release source builds
* Standalone UI presence in About dialog, Summary -> System Diagnostics
* Cluster UI as Versions tab in Cluster dialog
* Reduce About Dialog Content
* Fix Missing Property Display Bugs
* Marking the build time as type string.
* This closes #583
2016-11-03 11:16:20 -04:00
jpercivall 1d74b5d3ce NIFI-2791 Adding 'math' expression language function
This closes #1157.
2016-10-25 19:45:38 +02:00
jpercivall e4a3e09643 NIFI-1662 Adding proper UI regex support for decimals in EL
This closes #1018
2016-10-24 10:31:16 -04:00
Matt Burgess 557e0b9f27 NIFI-1662: Added support for decimal literal in Expression Language 2016-10-24 10:20:21 -04:00
jpercivall 94ab999026 NIFI-1662 adding Expression Language decimal support 2016-10-24 10:20:21 -04:00
Andre F de Miranda 9b08f23b24
NIFI-1531 - Remove bogus .gitignore files from sub directories
This closes #1154.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-10-23 10:29:09 -04:00
Pierre Villard 6e7793305d NIFI-2731 MergeContent default max number of flow files and max number of bins should be smaller 2016-10-13 15:13:47 -04:00
Koji Kawamura c470fae065 NIFI-2863: S2S to allow cluster URL more leniently. This closes #1122
- Consolidated the target cluster URL resolving logic into
  SiteToSiteRestApiClient's as a common method
- Changed to more descriptive error message
- Added more unit test cases
2016-10-13 09:50:50 -04:00
Koji Kawamura 540ef63efa NIFI-2855: Site-to-Site with port forwarding.
- Added following properties:
  - nifi.web.http.port.forwarding
  - nifi.web.https.port.forwarding

This closes #1100.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2016-10-07 15:00:37 +09:00
Mark Payne 9304df4de0
NIFI-2865: Fixed bug in StreamDemarcator that is exposed when the final bit of data in a stream is smaller than the previous and the previous demarcation ended on a buffer length boundary
This closes #1110.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-10-06 15:51:37 -04:00
Mark Payne b9cb6b1b47
NIFI-2865: Refactored PublishKafka and PublishKafka_0_10 to allow batching of FlowFiles within a single publish and to let messages timeout if not acknowledged
Signed-off-by: Bryan Bende <bbende@apache.org>
2016-10-06 15:51:16 -04:00
Bryan Bende a4ed622152
Revert "NIFI-2865: Refactored PublishKafka and PublishKafka_0_10 to allow batching of FlowFiles within a single publish and to let messages timeout if not acknowledged"
This reverts commit 92cca96d49.
2016-10-06 15:19:00 -04:00
Mark Payne 92cca96d49
NIFI-2865: Refactored PublishKafka and PublishKafka_0_10 to allow batching of FlowFiles within a single publish and to let messages timeout if not acknowledged
This closes #1097.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-10-06 10:05:51 -04:00
Joe N 6f1af31ff2 NIFI-2852 base64 expression language functions
Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-10-05 17:33:07 -04:00
Andre F de Miranda 3b408f5601 NIFI-2816 - Clean typos across the code - Part 2. This closes #1085 2016-10-05 13:07:57 -04:00
Joe N 34e5a5321a NIFI-2826 Adding enqueue time to GetAzureEventHub processor
Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-10-03 10:41:33 -04:00
Andre F de Miranda 446cd44702 NIFI-2816 - Clean typos across the code
This closes #1057.
2016-09-26 17:47:31 +02:00
Koji Kawamura ae251c1a6f NIFI-2718: Show HTTP S2S Auth error on bulletin
This commit fixes following two issues, that happens when a Root Group Port
policy for S2S data transfer is removed at a remote NiFi, after a client NiFi has
connected to that port:

1. At client side, Remote Process Group should show that authorization
is failing on its bulletin, but the Exception is caught and
ignored. Nothing is shown on the UI with HTTP transport protocol.
RAW S2S shows error on RPG bulletin. This commit fixes HTTP S2S to
behave the same.

2. At server side, corresponding input-port or output-port should show
that it is accessed by an unauthorized client on its bulletin, but it's
not shown with HTTP transport protocol.
RAW S2S shows warning messages for this. This commit fixes HTTP S2S to
behave the same.

In order to fix the 2nd issue above, request authorization at
DataTransferResource is changed from using DataTransferAuthorizable
directly, to call RootGroupPort.checkUserAuthorization().

Because the blettin is tied to the Port instance and it's
difficult to produce blettin message from this resource.

Since RootGroupPort.checkUserAuthorization uses
DataTransferAuthorizable inside, the check logic stays the same as
before.

Adding a RootGroupPortAuthorizable to provide access to necessary components for performing the authorization.

This closes #996
2016-09-08 13:43:38 -04:00
Mark Payne a7e76cc00a NIFI-1966: When cluster is started up, do not assume that Cluster Coordinator has the golden copy of the flow but instead wait for some period of time or until the required number of nodes have connected, and then choose which flow is correct. This closes #977 2016-09-06 16:31:37 -04:00
Joseph Percivall 1fe18a1567 NIFI-2676-rc1 prepare for next development iteration 2016-08-26 11:40:58 -04:00
Joseph Percivall 74d5224783 NIFI-2676-rc1 prepare release nifi-1.0.0-RC1 2016-08-26 11:40:44 -04:00
Matt Burgess 0745990c2d NIFI-2604: Added validators and logic for multiple URLs/files/folders for DB driver location
This closes #912

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-08-25 23:35:41 -04:00
Bryan Bende 957c120343 NIFI-2664 Moving System.setProperty for krb5.conf to NiFi startup, and removing conflicting property from KerberosProvider config
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #946
2016-08-25 17:26:37 -04:00
Mark Payne f908ae3c3b NIFI-2669: This closes #949. Ensure that if Exception is thrown during Transaction initialization that the underlying client is closed/cleaned up. Also ensure that we generate bulletins when logging error/warn level log messages 2016-08-25 16:39:00 -04:00
Matt Gilman a6133d4ce3 NIFI-2635:
- Fixing contrib check issues.
- Clean up pom.
- Addressing issue where reporting task property descriptor using wrong scope.

NIFI-2635:
- Fixing issue with revisions when creating users and user groups.
- Forwarding requests to the coordinator instead of replicating.
- Tweaking verbage in dialog for removing users and groups.

This closes #943
2016-08-25 13:08:35 -04:00
Mark Payne 8536ad65f4 NIFI-2651: Ensure that when we disable transmission on an RPG that we interrupt any transactions in progress for http-based site-to-site
This closes #937.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2016-08-25 15:33:10 +09:00
Matt Gilman c2bfc4ef24 NIFI-2635: - Re-using the original request during the second phase of the two phase commit. - Forwarding requests to the coordinator when received by a node.
This closes #933

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-08-24 22:42:10 -04:00
Koji Kawamura 671301193b NIFI-2525: Fix Proxy auth issue with async send.
Without this fix, NiFi fails to send data via HTTP Site-to-Site through
Proxy which requires authentication due to AsynchronousCloseException.
It happens when async client replays producing contents in order to re-send the
request with auth credential for the proxy server, however the
connection is already closed.
This fix makes NiFi to send actual data only at the second round of requests, so that flow-file
contents can be sent without reading it twice.

Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #915
2016-08-24 20:39:17 -04:00
Koji Kawamura a3586e04d9 NIFI-2459: Site-to-Site bootstrap node failure
Refresh remote peer statuses even if the bootstrap node goes down.

Migrate existing code which handles the situation from
EndpointConnectionPool to PeerSelector, so that both RAW and HTTP
transport protocol has the same capability.

This closes #927.
2016-08-24 17:28:10 -04:00
Andy LoPresto c638191a47
NIFI-1831 Added internal logic and command-line tool to allow AES-encrypted sensitive configuration values in nifi.properties.
This closes #834.
2016-08-23 20:34:31 -07:00
Bryan Rosander 23350543ff
NIFI-2621 - Generating unique serial numbers for certificates
This closes #909.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-08-23 01:37:25 -07:00
Matt Burgess 7123a1a276 NIFI-2619: Added unit test showing bugs, Added logic to ClassLoaderUtils to trim module paths and accept URLs
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #907
2016-08-22 15:29:03 -04:00
Koji Kawamura a919844461 NIFI-2567: Site-to-Site to send large data via HTTPS
- It couldn't send data larger than about 7KB due to the mis-use of
  httpasyncclient library
- Updated httpasyncclient from 4.1.1 to 4.1.2
- Let httpasyncclient framework to call produceContent multiple times as
  it gets ready to send more data via SSL session
- Added HTTPS test cases to TestHttpClient, which failed without this
  fix
2016-08-19 14:24:53 -04:00
joewitt a5261914fb NIFI-2574 merging latest kerb changes to adjust for NiFiProperties 2016-08-17 09:06:18 -07:00
Matt Gilman 3f7216ab84 NIFI-2561: - Decoupling kerberos service and spnego principles and keytabs.
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #873
2016-08-17 08:14:05 -04:00
joewitt 7d7401add4 NIFI-2574 Changed NiFiProperties to avoid static initializer and updated all references to it. 2016-08-17 00:10:07 -07:00
Bryan Rosander fa5da543e6
NIFI-2526 - DN order, multiple standalone runs, client certificates
- Logic for sorting DN, reversing X500Names before using them to generate certificate
- Logging reordered dn
- Accounting for limited crypto pkcs12, allowing password specification for client certificates
- Updating tests to work with or without jce unlimited
- Loading keystore for test in try-with

This closes #824.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-15 10:21:34 -04:00
Bryan Bende fd0dd51ff5 NIFI-2553 Fixing handling of Paths in HDFS processors
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #843
2016-08-15 08:59:42 -04:00
joewitt d3b96dcac1 NIFI-2519 This closes #856. aligned threading model with subethastmp 2016-08-14 13:32:00 -04:00
Oleg Zhurakousky 48fa76ecff NIFI-2519 Fixed and refactored ListenSMTP processor
- Removed message queueing which could result in data loss
- Fixed life-cycle issues that coudl put processor in an unstable state
- Fixed PropertyDescriptor translation for Time units and Byte sizes
- Fixed broken tests
- Added additional tests

NIFI-2519 added default for SMTP_MAXIMUM_CONNECTIONS

NIFI-2519 addressed PR comments, polishing
- fixed intermittent deadlock on processor stop and added test for it
- the attributes that can not be extracted from the message but available via MessageContext are written into the outgoing FlowFile
- other minor fixes

NIFI-2519 addressed lates PR comments

NIFI-2519 added better messaging when server closes the connection

NIFI-2519 some polishing and additional tests to validate deadlocks

NIFI-2519 address latest PR comments
fixed deadlock condition for when the consumer is stopped while server is distributing messages
fixed MAX message size issue ensuring it is validated
set max connections to SMTPServer
polished pom
added L&N

NIFI-2519 PR comments
- fixed LICENSE
- Added usage of LimitingInputStream
- simplified SmtpConsumer by removing hasMessage operation
2016-08-14 13:31:46 -04:00
Devin Fisher 098a35c915 Fixed CharSequenceTranslatorEvaluator to meet style standard and added license text block
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #833
2016-08-12 11:31:50 -04:00
devin fisher 219e0e96fb Added more test for escape and unescape functions
Signed-off-by: Matt Burgess <mattyb149@apache.org>
2016-08-12 11:30:06 -04:00
devin fisher 8f74241a10 added functions for escaping text to the expression language
Signed-off-by: Matt Burgess <mattyb149@apache.org>
2016-08-12 11:30:06 -04:00
Devin Fisher 6f85440ebd fix a copy past error
Signed-off-by: Matt Burgess <mattyb149@apache.org>
2016-08-12 11:30:06 -04:00
Devin Fisher ebd11b1d8f EscapeJson function added to expression-language
Made use of org.apache.commons.lang3.StringEscapeUtils to do that
actual processing

Signed-off-by: Matt Burgess <mattyb149@apache.org>
2016-08-12 11:30:06 -04:00
Mark Payne 76a4a2c48b
NIFI-2544: Created integration tests for clustering and addressed a few minor bugs that were found in doing so
This closes #832.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-11 10:43:38 -04:00
Mark Payne 42df02f014 NIFI-2406 This closes #820. Addressed regression introduced in NIFI-2406 where the cluster does not recognize a new Cluster Coordinator when the coordinator is shutdown 2016-08-09 15:19:49 -04:00
Bryan Rosander fa4c6ab03c
NIFI-2193 - Added functionality to automate certificate generation, keystore and truststore generation, and nifi.properties keystore and truststore password population. Follow-on changes will be made under NIFI-2476.
This closes #695.

Signed-off-by: Andy LoPresto <alopresto@apache.org>

Defaulting to same keyStore, key password (+18 squashed commits)
Squashed commits:
[9d01ba0] NIFI-2193 - Fixing typo
[55440bc] NIFI-2193 - Standalone can run as long as there are no conflicting files/folders
[0ca34ed] NIFI-2193 - Fixing some filename, absolute path issues
[9d4f65b] NIFI-2193 - Incorporating feedback
[f7550b4] NIFI-2193 - Cleaning up imports
[59a7637] NIFI-2193 - Updating umask to allow owner to execute
[cf824e7] NIFI-2193 - Moving DN arg to CA service specific parent class
[921ee13] NIFI-2193 - Making keystore getInstance more consistent
[a283c4b] NIFI-2193 - Updating sample config files in assembly to reflect new structure
[8d3a21d] NIFI-2193 - Making TlsHelper static, adding option to use same password for Key, KeyStore
[b13d247] NIFI-2193 - Addressing PR feedback
[46ef8ed] NIFI-2193 - Removing commons-logging, log4j from notice
[d4cf41a] NIFI-2193 - Adding option to specify output file for CA certificate when using cli client
[b74bf25] NIFI-2193 - Removing Bouncy Castle from notice
[6e34f9a] NIFI-2193 - Adding CLI client for easier generation of client certificates
[2924fca] NIFI-2193 - nifi-toolkit-ssl -> nifi-toolkit-tls, removing unused constants
[886167e] NIFI-2193 - Adding slf4j to avoid runtime issue
[082de46] NIFI-2193 - Command line SSL config utility as well as certificate authority client/server
2016-08-03 21:34:11 -07:00
Mark Payne 83a23f90d4
NIFI-2466: Added option to provide separate key password to StandardSSLContextService.
Fixed NPE (+2 squashed commits)
Squashed commits:
[c5d521a] NIFI-2466: Added unit test to verify changes; fixed validation
[aa4d418] NIFI-2446: Add option to specify key password when different than keystore password

This closes #776.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-08-03 15:56:12 -07:00
Koji Kawamura 809f042353 NIFI-2028: Fixed Site-to-Site Transit URI
Fixed Site-to-Site Transit URI for HTTP to be consistent with RAW socket.

- Removed url from CommunicationsSession since it's redundant as we have
  Peer.url, too. The value was not used from anywhere other than HTTP
Site-to-Site.
- Added createTransitUri method in Communicant interface, so that
  implementation can customize transitUri while providing consistent
interface.
2016-08-02 09:08:00 -04:00
joewitt 05a99a93cb NIFI-2208 This closes #754. refactored as per comments on JIRA. Reduced API expsosure and tightened lifecycle management. 2016-08-01 14:17:26 -04:00
Oleg Zhurakousky 1bf10944ea NIFI-2366 - Fixed ID generation semantics in clustered environment
- added SnippetUtilsTest
- renamed TypeOneUUIDGenerator to ComponentIdGenerator

- changed lsb part of ComponentIdGenerator back to long
- Fixed 'isCopy' condition for clustered environments

This closes #718.
2016-07-31 15:24:02 -04:00
Yolanda M. Davis 8412d2662a NIFI-2208 - initial commit Custom Property Expression Language support with Variable Registry, includes bug fix for NIFI-2057
This closes #529

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-29 17:10:20 -04:00
Koji Kawamura b396867847 NIFI-2386 This closes #716. Site-to-Site fails without port no
It fails if a given URL doesn't have port in it.
This fixes its behavior with default http 80 and https 443 port.
2016-07-28 22:23:39 -04:00
Oleg Zhurakousky 6c7c3c0822 NIFI-826 (part 3)
- fixed ID generation routine that was causing miss-identification of the components
2016-07-19 15:52:35 -04:00
Oleg Zhurakousky 52a961873b NIFI-826 This closes #617. Added deterministic template support 2016-07-15 20:41:18 -04:00
Mark Payne c81dc1959a NIFI-1992:
- Updated site-to-site client and server to support clustered nifi instances
NIFI-2274:
- Ensuring we use the correct URI when updating a connection.

This closes #530
2016-07-15 16:13:59 -04:00
Aldrin Piri d1129706e2 NIFI-1896 This closes #650. Refactored nifi-api into nifi-framework-api and other locations. The nifi-api is specific to that which is needed for intended extension points. 2016-07-14 18:24:48 -04:00
Yolanda M. Davis 048ba5366c NIFI-2020 - initial commit for custom transformation support
NIFI-2020 - updates to use lambdas/stream wherever possible and fix potential nullpointer issue.

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #564
2016-07-14 18:04:14 -04:00
Koji Kawamura 30889995cb NIFI-2145: Auto flow.xml archive
- Added following properties:
  - nifi.flow.configuration.archive.enabled
  - nifi.flow.configuration.archive.max.time
  - nifi.flow.configuration.archive.max.storage
- Removed manual archive operation:
  - Removed 'Back-up flow' link from UI since it's not needed any longer
  - Removed corresponding REST API controller/archive and its
    implementations
- Added FlowConfigurationArchiveManager to enclose archive related code
- Updated related docs
2016-07-14 10:35:16 -04:00
joewitt f987b21609 NIFI-1157 searched for and resolved all remaining references to deprecated items that were clearly addressable. 2016-07-14 09:32:35 -04:00
joewitt 961be21a38 NIFI-1157 resolved deprecated nifi-api items and ripple effects 2016-07-14 09:32:34 -04:00
Matt Gilman e0c96794fa NIFI-2095:
- Adding a page for managing users and groups.
- Adding a page for managing access policies.
- Renaming accessPolicy in entity to permissions to avoid confusion with the accessPolicy model.
- Adding an Authorizable for access policies.
- Refactoring access policies endpoints.
NIFI-2022:
- Implementing site to site authorizations.
2016-07-12 15:45:13 -04:00
Bryan Bende ba763b95e8 NIFI-2003 Creating abstract authentication provider and incorporating into existing providers
NIFI-2201 Add support for seeding cluster nodes in authorizations.xml
- Passing client address along in user context on authorization requests
- This closes #628
2016-07-12 11:20:29 -04:00
Andy LoPresto 4b9df7d1e2 NIFI-2186 Refactored CertificateUtils to separate logic for DN extraction from server/client sockets. Added logic to detect server/client mode encapsulated in exposed method.
Added unit tests for DN extraction.
Corrected typo in Javadoc.
Switched server/client socket logic for certificate extraction -- when the local socket is in client/server mode, the peer is necessarily the inverse.
Fixed unit tests.
Moved lazy-loading authentication access out of isDebugEnabled() control branch.
This closes #622
2016-07-11 23:15:28 -04:00
Matt Gilman ce5330330a NIFI-1781:
- Updating UI according to permissions through out the application.
- Shuffling provenance events, template, and cluster search REST APIs according to resources being authorized.
- Moving template upload controls.
- Removing username where appropriate.
- Addressing issues when authorizing flow configuration actions.
- Code clean up.
2016-07-01 15:10:27 -04:00
Mark Payne ae9e2fdf0b NIFI-2123: Add authorization of provenance events; refactor core classes so that Authorizable is located within nifi-api. This closes #592 2016-06-30 07:57:17 -04:00
jpercivall cd846c8d62 NIFI-1834 adding fixes to PutTCP
This closes #543
2016-06-21 12:05:39 -04:00
Matt Brown a7c912ac18 PutTCP Processor created.
Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-06-21 12:05:13 -04:00
Will Song 9546bef86e
NIFI-1958 Added "wks" as an acceptable parameter and added unit test (+6 squashed commits)
Squashed commits:
[16dd4ba] NIFI-1958 fixed logic on incoming time units. Removed feature tests to convert to weeks because it will not be implemented.
[1b22e58] NIFI-1958 added logic to getTimeDuration to handle weeks as a string value to parse but did not add TimeUnit yet. 3/5 feature test pass.
[7136544] NIFI-1958 Moved tests to correct module.
[7d95653] NIFI-1958 Added feature tests for negative values.
[ffc3941] NIFI-1958 Added second feature test.
[7d16bbe] NIFI-1958 Added new feature test and regression test for week conversion.

This closes #544.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-06-17 17:24:19 -07:00
Joshi 548561ed4c NIFI-1955: Deprecate IntegerHolder, LongHolder, BooleanHolder
This closes #537
2016-06-17 12:32:53 -04:00
Matt Gilman fbd299e885 NIFI-1901: - Building component based access control tests for Connections, Funnels, Labels, Input Ports, Output Ports, Processors, and Process Groups. - Restoring Access Token Endpoint tests.
This closes #518

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-06-17 10:16:09 -04:00
Andy LoPresto 8b0c5a4911 NIFI-1981 Resolved issue where cluster communication without client certificates would fail even if needClientAuth set to false.
Fixed IDE setting for import wildcarding on Groovy files. (+4 squashed commits)
Squashed commits:
[4c3b174] NIFI-1981 Lowered logging level of client auth setting on cluster connection receive.
[b50f473] NIFI-1981 Finished logic to suppress exception on missing client certificates when clientAuth is set to WANT.
Added unit tests for CertificateUtil methods.
[ace35a2] NIFI-1981 Added test scope dependency on BouncyCastle and BC PKIX modules for CertificateUtils tests.
[2c463d1] NIFI-1981 Added ClientAuth enum and CertificateUtil methods to extract this setting from an SSLSocket.
Added logic to compare X509Certificate DNs regardless of RDN element order.
Added logic to suppress peer certificate exceptions when client authentication is not required.
Removed duplicate dependency in pom.xml.
2016-06-14 11:18:31 -04:00
Matt Gilman 75bb4bfaa2 NIFI-1781:
- Listening for window resize events more selectively.
- Fixing malformed request when configuring remote process group ports.
- Fixing malformed request when starting/stopping a selected process group.
- Fixing default value for authorizers.xml.
- This closes #524
2016-06-13 16:55:01 -04:00
Koji Kawamura c120c4982d NIFI-1857: HTTPS Site-to-Site
- Enable HTTP(S) for Site-to-Site communication
- Support HTTP Proxy in the middle of local and remote NiFi
- Support BASIC and DIGEST auth with Proxy Server
- Provide 2-phase style commit same as existing socket version
- [WIP] Test with the latest cluster env (without NCM) hasn't tested yet

- Fixed Buffer handling issues at asyc http client POST
- Fixed JS error when applying Remote Process Group Port setting from UI
- Use compression setting from UI
- Removed already finished TODO comments

- Added additional buffer draining code after receiving EOF
- Added inspection and assert code to make sure Site-to-Site client has
  written data fully to output
stream
- Changed default nifi.remote.input.secure from true to false

This closes #497.
2016-06-09 15:09:57 -04:00
Bryan Bende 37488d2789 Adding RAT exclusion for test resource in nifi-commons/nifi-expression-language 2016-06-07 17:55:07 -04:00
Mark Payne 6de738fd04 NIFI-1660: Fix formatting; update to NOTICE files of nars that bring in EL. Closed Reader that was opened in TestQuery. 2016-06-07 15:34:37 -04:00
Chris McDermott abad7d805e NIFI-1660 - Enhance the expression language with jsonPath function 2016-06-07 15:34:36 -04:00
Mark Payne df0e4e7960 NIFI-1897:
- Refactoring to allow requests to be replicated from a node to other nodes
- Renaming cluster node connection/read timeout properties.
- Renaming NCM DN to Cluster Coordinator DN.
- Fixing default values in properties.
- Starting to fix Spring context to load correctly in standalone mode.
- Using the cluster protocol to handle connection failures instead of heartbeats.
- Ensured replicate call is returned from ControllerResource.
- Ensure the appropriate classloader when serializing templates.
- Handling when the flow contents are null.
- This closes #488
2016-06-03 15:29:16 -04:00
Andy LoPresto 8127314975
NIFI-1919 Added replaceFirst() expression language method which accepts literal or pattern for replacement.
Reverted whitespace changes. (+8 squashed commits)
Squashed commits:
[329755c] NIFI-1919 Reverted import re-organization from IDE.
[cf73c2f] NIFI-1919 Updated expression language guide.
[d9a1455] NIFI-1919 Reverted changes to ReplaceEvaluator.
Added ReplaceFirstEvaluator.
Added replace first logic to Query buildFunctionEvaluator.
Added unit tests.
[e2eb880] NIFI-1919 Added replaceFirst to AttributeExpression lexer and parser grammar definitions.
[11fe913] NIFI-1919 Ignored demonstrative test for replaceAll as it behaves as expected.
[af97be1] NIFI-1919 Changed ReplaceEvaluator to use String#replaceFirst which interprets regex instead of compiling as literal.
Demonstrative unit test now passes but two existing unit tests fail. I am not sure these tests are correct.
[f24f17b] NIFI-1919 Added working unit test to illustrate fix.
[8a0d43b] NIFI-1919 Added Groovy unit test to demonstrate issue.
Added DelegatingMetaClass code to record it (test not complete).

This closes #474.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-06-01 16:39:58 -07:00
Bryan Bende 5df67c5dc2 NIFI-1907 Moving lazy init of SSLContext to StandardSiteToSiteClientConfig rather than the builder
This closes #457.
2016-05-24 09:51:18 -04:00
Andrew Grande d42ea467ca Reviewed, typos.
This closes #460
2016-05-22 10:58:48 -04:00
Mark Payne 4b74e4de74 NIFI-1745: Refactor how revisions are handled at NCM/Distributed to Node. This closes #454 2016-05-20 14:04:24 -04:00
Pierre Villard 372ffb8aa0 NIFI-1811 Removed ProcessorLog and updated dependent interfaces
This closes #403.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-05-19 13:08:09 -04:00
Oleg Zhurakousky 2d03489ec5 NIFI-1296, NIFI-1680, NIFI-1764, NIFI-1837, NIFI-1827, NIFI-1699 implemented new Kafka processors that leverage Kafka 0.9 API
- Improved StreamScanner for better performance
- Renamed StreamScanner to StreamDemarcator as suggested by Joe
- Added failure handling logic to ensure both processors can be reset to their initial state (as if they were just started)
- Provided comprehensive test suite to validate various aspects of both Publish and Consume from Kafka
- Added relevant javadocs
- Added initial additionalDetails docs
- Addressed NPE reported by NIFI-1764
- Life-cycle refactoring for the existing PutKafka to ensure producer restart after errors
- Incorporated code changes contributed by Ralph Perko (see NIFI-1837)
- Addressed partition issue in RoundRobinPartitioner discussed in NIFI-1827
- Updated PropertyDescriptor descriptions to reflect their purpose

NIFI-1296 added @Ignore on some Kafka tests to improve test time

NIFI-1296 reworked tests to avoid dependency on embedded Kafka

NIFI-1296 fixed spelling error

NIFI-1296 fixed trailing whitespaces in non-java files

This closes #366
2016-05-19 07:33:03 -04:00