Commit Graph

187 Commits

Author SHA1 Message Date
Mark Payne 8f8b8cdf46 NIFI-3071: Deprecated InputStreams & OutputStream sin org.apache.nifi.stream.io package in favor of using their Java counterparts 2016-12-05 09:29:19 -05:00
Koji Kawamura 7c5bd876bd NIFI-3026: Support multiple remote target URLs
- Added urls in addition to the existing url, to support multiple target
  URLs
- Backward compatibility is provided by returning the first url if
  multipe urls are specified, but component accessing the url doesn't
support multiple urls
- UI is not fully updated yet. Following UI components are planned to be updated
  by different commits
  - Search component: only the first URL is searchable and shown
  - Component status: RPG status shows only the first URL
  - Component action history: only the first URL is searchable and shown
  - Updated Search component to use URLs.

This closes #1208.
2016-12-02 14:01:39 -05:00
Mark Payne fff0148a0e NIFI-3129: When adding controller services to a snippet, ensure that we don't add the service multiple times, even when it's referenced by child process groups. This closes #1284 2016-11-30 13:07:51 -05:00
joewitt 92f17a995b NIFI-3100-rc2 prepare for next development iteration 2016-11-25 23:49:27 -05:00
joewitt 5536f690a8 NIFI-3100-rc2 prepare release nifi-1.1.0-RC2 2016-11-25 23:49:13 -05:00
Koji Kawamura a1ab5e844b NIFI-2729: This closes #1270. testSendSuccessWithProxy timeout in Travis
- Changed AtomicBoolean to CountDownLatch to avoid sleeping thread in
  some test cases
- Specified less number of threads for Jetty and LittleProxy than
  default to lower resource usage
- Added try catch for the specific gateway timeout case (504) so that
  test can pass even it happens while it fails with other errors
2016-11-25 19:41:15 -05:00
joewitt fb9cbccc38 NIFI-2954 This closes #1244. Moved StandardPropertyValidator to nifi-utils, documented scope/purpose of a few util libs, removed deps from nifi-utils. 2016-11-21 16:30:42 -05:00
Mark Payne 2ee66de1a6 NIFI-3066: Create unit test to reproduce bug that results in IllegalArgumentException: Cannot migrate FlowFiles from a Process Session to itself
NIFI-3066: Ensure that when a Bin is created, it is always created with its own new session

This closes #1245
2016-11-21 11:34:14 -05:00
Mark Payne 1be0871473 NIFI-2854: Refactor repositories and swap files to use schema-based serialization so that nifi can be rolled back to a previous version after an upgrade.
NIFI-2854: Incorporated PR review feedback

NIFI-2854: Implemented feedback from PR Review

NIFI-2854: Ensure that all resources are closed on CompressableRecordReader.close() even if an IOException is thrown when closing one of them

This closes #1202
2016-11-18 14:53:13 -05:00
jpercivall c4be800688 NIFI-2950 Adding support for whole number hex values and a fromRadix function
NIFI-2950 Fixing typo

This closes #1161
2016-11-17 07:27:51 -05:00
Bryan Rosander e5eda63705
NIFI-2943 - Toolkit uses JKS type over PKCS12 when creating truststore because non-Bouncy Castle providers cannot read certificates from PKCS12 truststore.
Peer review feedback (+2 squashed commits)
Squashed commits:
[0102c8e] NIFI-2943 - Peer review feedback
[9bcd495] NIFI-2943 - pkcs12 keystore improvements

1. loading pkcs12 keystores with bouncy castle everywhere
2. tls-toolkit client using jks truststore when keystore type is specified differently
3. tests

This closes #1165.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-11-16 17:13:10 -08:00
Koji Kawamura 13ea909122
NIFI-2851: Fixed CheckStyle error.
This closes #1215.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-11-14 00:44:16 -08:00
Mark Payne ad92474593 NIFI-2851: Added additional unit test to ensure correctness of demarcation when demarcator falls between buffered data
This closes #1116.
2016-11-11 15:45:40 -05:00
Oleg Zhurakousky 41f519e84c NIFI-2851 initial commit of perf improvements on SplitText
- introduced org.apache.nifi.stream.io.util.TextLineDemarcator
- refactored SplitText to use org.apache.nifi.stream.io.util.TextLineDemarcator
- updated SplitText's capability discription to provide more clarity around splits with headers.
2016-11-11 15:45:40 -05:00
Mark Payne c441a8696d NIFI-2850 This closes #1115. Added a migrate() method to ProcessSession and refactored BinFiles and MergeContent to use it 2016-11-09 16:25:03 -05:00
Bryan Bende d1d053725b NIFI-2909 Adding per-instance class loading capability through @RequiresInstanceClassLoading annotation
NIFI-1712 Applying per-instance class loading to HBaseClientService to allow specifying Phoenix Client JAR

-Refactoring the ClassLoading so that every processor, controller service, and reporting task gets an InstanceClassLoader with a parent of the NAR ClassLoader, and only components with @RequiresInstanceClassLoading will make a copy of the NAR ClassLoader resources, and addressing some review feedback

This closes #1156
2016-11-08 13:21:27 -05:00
Oleg Zhurakousky 5fd4a55791 NIFI-2778 added abilty to interrupt Lucene search
polishing

This closes #1138
2016-11-07 16:28:16 -05:00
James Wing a486fefb1e NIFI-2115 Detailed Version Info in About Box
* Java version and vendor
* OS name and version
* Release Tag
* Build revision (commit SHA), branch, and timestamp
* Handles formal releases, ad-hoc builds, and non-release source builds
* Standalone UI presence in About dialog, Summary -> System Diagnostics
* Cluster UI as Versions tab in Cluster dialog
* Reduce About Dialog Content
* Fix Missing Property Display Bugs
* Marking the build time as type string.
* This closes #583
2016-11-03 11:16:20 -04:00
jpercivall 1d74b5d3ce NIFI-2791 Adding 'math' expression language function
This closes #1157.
2016-10-25 19:45:38 +02:00
jpercivall e4a3e09643 NIFI-1662 Adding proper UI regex support for decimals in EL
This closes #1018
2016-10-24 10:31:16 -04:00
Matt Burgess 557e0b9f27 NIFI-1662: Added support for decimal literal in Expression Language 2016-10-24 10:20:21 -04:00
jpercivall 94ab999026 NIFI-1662 adding Expression Language decimal support 2016-10-24 10:20:21 -04:00
Andre F de Miranda 9b08f23b24
NIFI-1531 - Remove bogus .gitignore files from sub directories
This closes #1154.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-10-23 10:29:09 -04:00
Pierre Villard 6e7793305d NIFI-2731 MergeContent default max number of flow files and max number of bins should be smaller 2016-10-13 15:13:47 -04:00
Koji Kawamura c470fae065 NIFI-2863: S2S to allow cluster URL more leniently. This closes #1122
- Consolidated the target cluster URL resolving logic into
  SiteToSiteRestApiClient's as a common method
- Changed to more descriptive error message
- Added more unit test cases
2016-10-13 09:50:50 -04:00
Koji Kawamura 540ef63efa NIFI-2855: Site-to-Site with port forwarding.
- Added following properties:
  - nifi.web.http.port.forwarding
  - nifi.web.https.port.forwarding

This closes #1100.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2016-10-07 15:00:37 +09:00
Mark Payne 9304df4de0
NIFI-2865: Fixed bug in StreamDemarcator that is exposed when the final bit of data in a stream is smaller than the previous and the previous demarcation ended on a buffer length boundary
This closes #1110.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-10-06 15:51:37 -04:00
Mark Payne b9cb6b1b47
NIFI-2865: Refactored PublishKafka and PublishKafka_0_10 to allow batching of FlowFiles within a single publish and to let messages timeout if not acknowledged
Signed-off-by: Bryan Bende <bbende@apache.org>
2016-10-06 15:51:16 -04:00
Bryan Bende a4ed622152
Revert "NIFI-2865: Refactored PublishKafka and PublishKafka_0_10 to allow batching of FlowFiles within a single publish and to let messages timeout if not acknowledged"
This reverts commit 92cca96d49.
2016-10-06 15:19:00 -04:00
Mark Payne 92cca96d49
NIFI-2865: Refactored PublishKafka and PublishKafka_0_10 to allow batching of FlowFiles within a single publish and to let messages timeout if not acknowledged
This closes #1097.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-10-06 10:05:51 -04:00
Joe N 6f1af31ff2 NIFI-2852 base64 expression language functions
Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-10-05 17:33:07 -04:00
Andre F de Miranda 3b408f5601 NIFI-2816 - Clean typos across the code - Part 2. This closes #1085 2016-10-05 13:07:57 -04:00
Joe N 34e5a5321a NIFI-2826 Adding enqueue time to GetAzureEventHub processor
Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-10-03 10:41:33 -04:00
Andre F de Miranda 446cd44702 NIFI-2816 - Clean typos across the code
This closes #1057.
2016-09-26 17:47:31 +02:00
Koji Kawamura ae251c1a6f NIFI-2718: Show HTTP S2S Auth error on bulletin
This commit fixes following two issues, that happens when a Root Group Port
policy for S2S data transfer is removed at a remote NiFi, after a client NiFi has
connected to that port:

1. At client side, Remote Process Group should show that authorization
is failing on its bulletin, but the Exception is caught and
ignored. Nothing is shown on the UI with HTTP transport protocol.
RAW S2S shows error on RPG bulletin. This commit fixes HTTP S2S to
behave the same.

2. At server side, corresponding input-port or output-port should show
that it is accessed by an unauthorized client on its bulletin, but it's
not shown with HTTP transport protocol.
RAW S2S shows warning messages for this. This commit fixes HTTP S2S to
behave the same.

In order to fix the 2nd issue above, request authorization at
DataTransferResource is changed from using DataTransferAuthorizable
directly, to call RootGroupPort.checkUserAuthorization().

Because the blettin is tied to the Port instance and it's
difficult to produce blettin message from this resource.

Since RootGroupPort.checkUserAuthorization uses
DataTransferAuthorizable inside, the check logic stays the same as
before.

Adding a RootGroupPortAuthorizable to provide access to necessary components for performing the authorization.

This closes #996
2016-09-08 13:43:38 -04:00
Mark Payne a7e76cc00a NIFI-1966: When cluster is started up, do not assume that Cluster Coordinator has the golden copy of the flow but instead wait for some period of time or until the required number of nodes have connected, and then choose which flow is correct. This closes #977 2016-09-06 16:31:37 -04:00
Joseph Percivall 1fe18a1567 NIFI-2676-rc1 prepare for next development iteration 2016-08-26 11:40:58 -04:00
Joseph Percivall 74d5224783 NIFI-2676-rc1 prepare release nifi-1.0.0-RC1 2016-08-26 11:40:44 -04:00
Matt Burgess 0745990c2d NIFI-2604: Added validators and logic for multiple URLs/files/folders for DB driver location
This closes #912

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-08-25 23:35:41 -04:00
Bryan Bende 957c120343 NIFI-2664 Moving System.setProperty for krb5.conf to NiFi startup, and removing conflicting property from KerberosProvider config
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #946
2016-08-25 17:26:37 -04:00
Mark Payne f908ae3c3b NIFI-2669: This closes #949. Ensure that if Exception is thrown during Transaction initialization that the underlying client is closed/cleaned up. Also ensure that we generate bulletins when logging error/warn level log messages 2016-08-25 16:39:00 -04:00
Matt Gilman a6133d4ce3 NIFI-2635:
- Fixing contrib check issues.
- Clean up pom.
- Addressing issue where reporting task property descriptor using wrong scope.

NIFI-2635:
- Fixing issue with revisions when creating users and user groups.
- Forwarding requests to the coordinator instead of replicating.
- Tweaking verbage in dialog for removing users and groups.

This closes #943
2016-08-25 13:08:35 -04:00
Mark Payne 8536ad65f4 NIFI-2651: Ensure that when we disable transmission on an RPG that we interrupt any transactions in progress for http-based site-to-site
This closes #937.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2016-08-25 15:33:10 +09:00
Matt Gilman c2bfc4ef24 NIFI-2635: - Re-using the original request during the second phase of the two phase commit. - Forwarding requests to the coordinator when received by a node.
This closes #933

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-08-24 22:42:10 -04:00
Koji Kawamura 671301193b NIFI-2525: Fix Proxy auth issue with async send.
Without this fix, NiFi fails to send data via HTTP Site-to-Site through
Proxy which requires authentication due to AsynchronousCloseException.
It happens when async client replays producing contents in order to re-send the
request with auth credential for the proxy server, however the
connection is already closed.
This fix makes NiFi to send actual data only at the second round of requests, so that flow-file
contents can be sent without reading it twice.

Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #915
2016-08-24 20:39:17 -04:00
Koji Kawamura a3586e04d9 NIFI-2459: Site-to-Site bootstrap node failure
Refresh remote peer statuses even if the bootstrap node goes down.

Migrate existing code which handles the situation from
EndpointConnectionPool to PeerSelector, so that both RAW and HTTP
transport protocol has the same capability.

This closes #927.
2016-08-24 17:28:10 -04:00
Andy LoPresto c638191a47
NIFI-1831 Added internal logic and command-line tool to allow AES-encrypted sensitive configuration values in nifi.properties.
This closes #834.
2016-08-23 20:34:31 -07:00
Bryan Rosander 23350543ff
NIFI-2621 - Generating unique serial numbers for certificates
This closes #909.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-08-23 01:37:25 -07:00
Matt Burgess 7123a1a276 NIFI-2619: Added unit test showing bugs, Added logic to ClassLoaderUtils to trim module paths and accept URLs
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #907
2016-08-22 15:29:03 -04:00
Koji Kawamura a919844461 NIFI-2567: Site-to-Site to send large data via HTTPS
- It couldn't send data larger than about 7KB due to the mis-use of
  httpasyncclient library
- Updated httpasyncclient from 4.1.1 to 4.1.2
- Let httpasyncclient framework to call produceContent multiple times as
  it gets ready to send more data via SSL session
- Added HTTPS test cases to TestHttpClient, which failed without this
  fix
2016-08-19 14:24:53 -04:00