Commit Graph

1642 Commits

Author SHA1 Message Date
Bryan Rosander 1b4729e448
NIFI-3240 - AttributesToJson performance improvements
This closes #1352.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-01-03 10:20:16 -05:00
Pierre Villard be6bcf20ad
NIFI-2944 Check remote input hostname at startup.
Added unit test cases.

This closes #1379.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-01-02 20:25:03 -08:00
Scott Aslan 67cbef5df3
NIFI-2917: disable canvas refresh after ajax error and allow page refresh
This closes #1351.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-01-02 16:32:05 -08:00
Davy De Waele 55f4716f3d NIFI-3266 Added EL support for basePath and port in ListenHTTP
This closes #1373.
2016-12-30 16:35:46 +01:00
Matt Gilman 0d14db72f5 NIFI-3247:
- Removing unnecessary authorization check during second phase of connection creation.
- Ensuring that the remote group port returns the correct resource type though not super critical since it is not possible to create policies for remote ports.

This closes #1353.
2016-12-30 16:05:06 +01:00
jpercivall 8acbe9aa3f
NIFI-1582 Fixing how failures with state are handled and improving flowfile handling
This closes #1371

Signed-off-by: Bryan Rosander <brosander@apache.org>
2016-12-29 15:49:52 -05:00
Pierre Villard 4986b83b83 NIFI-3164 Added NAR loading duration
This closes #1366.

Signed-off-by: James Wing <jvwing@gmail.com>
2016-12-29 10:29:03 -08:00
Pierre Villard 5c095f3a18 NIFI-3164 Added message at startup to log NAR loading
This closes #1365.

Signed-off-by: James Wing <jvwing@gmail.com>
2016-12-28 14:10:30 -08:00
Bryan Rosander 35e8bedcc8 NIFI-3249 - UpdateAttribute performance improvements
This closes #1356

Signed-off-by: jpercivall <JPercivall@apache.org>
2016-12-28 14:54:05 -05:00
Pierre Villard 16898668c2 NIFI-3228 added s3.contenttype in PutS3Object
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #1362.
2016-12-28 10:54:21 -08:00
Joe Gresock 2ea8b4122b NIFI-3127: Fixing NPE for empty flow files in SplitText
This closes #1358.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2016-12-28 11:21:44 +09:00
Matt Gilman 2f3017638f
NIFI-3239:
- Indicating components are not restricted when appropriate.
- Updating unit tests.

This closes #1350.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-12-27 13:06:07 -05:00
Bryan Bende 978f483ba8
NIFI-3250 Fixing logic in StandardFlowFileQueue when migrating flow files to the active queue
This closes #3250.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-12-27 12:52:01 -05:00
Joe Gresock af8ed8b7de
NIFI-3253: Correcting documentation in Notify processor
This closes #1359.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-12-27 12:19:08 -05:00
Matt Gilman 6c5d4084d5 NIFI-3207: Fixed Template upload import URI
- Fixing the importUri when replicating a Template upload request. Previously was leveraging getResourceUri which considers proxy headers meant for client responses.

This closes #1334.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2016-12-27 15:44:25 +09:00
Bryan Bende 6ca25e10ef NIFI-3220 Add support for basic auth to Solr processors
This closes #1338.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2016-12-26 15:41:24 +09:00
Scott Aslan 5ea17d30c5 [NIFI-96] add align horizontal and align vertical capability to components on the canvas. This closes #1354 2016-12-22 16:47:34 -05:00
Matt Gilman e65aad8fe6 NIFI-2994: - Setting the default position for remote process groups.
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #1303
2016-12-22 14:39:56 -05:00
Matt Gilman acc9e0b45c NIFI-2967:
- Disabling the Add button in the new Processor, Controller Service, and Reporting Task dialog when no components match the filter.
This closes #1340
2016-12-22 11:52:20 -05:00
Koji Kawamura 908e7d3131
NIFI-2585: Add attributes to track s2s host and port
- Removed host and port field from Peer since the same information is
  available in PeerDescription
- Refactored variable names in SocketRemoteSiteListener to improve readability
- Changed how SocketRemoteSiteListener constructs PeerDescription
  instance. It used to use hard-coded 'localhost' as hostname, and
  getPort() which returns server's port. Since the peer is a remote peer,
  i.e the client, it should be client hostname and port.
- Added hostname resolution at DataTransferResource to make s2s.host
  value consistent with RAW transport. Without this, RAW uses hostname
  while HTTP uses IP address. It will be hard to be used from downstream flows.
- Replaced heavy use of mockito which was difficult to maintain, with
  nifi-mock
- Added SiteToSiteAttributes and more assertions in unit tests

This closes #1342.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-12-22 11:00:00 -05:00
Bryan Bende f7d761a28a
NIFI-2585 Moving attributes into loop in AbstractFlowFileServerProtocol, and also updating StandardRemoteGroupPort to apply the same attributes when doing a pull-based site-to-site.
Signed-off-by: Bryan Bende <bbende@apache.org>
2016-12-22 10:59:56 -05:00
Randy Gelhausen 28e5d85493
NIFI-2585: Add attributes to track where a flow file came from when receiving over site-to-site
Signed-off-by: Bryan Bende <bbende@apache.org>
2016-12-22 10:59:55 -05:00
Bryan Rosander 44c9ea0a66 NIFI-3236 - SplitJson performance improvements
This closes #1347.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2016-12-22 15:22:53 +09:00
Bryan Rosander 4285157675
NIFI-3235 - EvaluateJsonPath performance improvements
This closes #1346.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-12-21 16:29:23 -05:00
Matt Gilman 721c9ee7f0 NIFI-3167: - Ensuring the DELETE button is visible for cluster nodes. - Removing unecessary wrapper element and non breaking spaces.
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #1319
2016-12-21 12:02:44 -05:00
Matt Burgess 3b91635398 NIFI-3194: Fixed error handling in PutElasticsearchHttp
Thise closes #1327

Signed-off-by: jpercivall <JPercivall@apache.org>
2016-12-19 14:51:41 -05:00
Scott Aslan aef17f9a8b [NIFI-3170] remove action details if user does not have read perms, also update action details styles to match other dialogs. This closes #1322 2016-12-19 10:07:15 -05:00
Joe Gresock 34627f78b7
NIFI-190: Initial commit of Wait and Notify processors
This closes #1329.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-12-16 10:29:16 -05:00
Wesley-Lawrence 9a5986b0f6 NIFI-3175 Fix a bug in the ValidateCSV processor to handle empty CSV columns
This closes #1311.
2016-12-15 22:43:15 +01:00
Mark Payne ab2d07aa5a
NIFI-3186: Fixed NPE that was overlooked
This closes #1317.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-12-15 12:56:23 -05:00
Mark Payne 5776c4b1f9 NIFI-3173: When a template is created with multiple components in different groups all referencing the same controller service, ensure that controller service is added to the template at a high enough level that all components needing hte service can access it.
- Ensure that controller services are added to child process groups when creating snippet

- Addressed issue related to modifying higher-level process groups' controller services in snippet after having already visited the process group

This closes #1318

Signed-off-by: jpercivall <JPercivall@apache.org>
2016-12-14 14:10:59 -05:00
Byunghwa Yun 0f462a7c49 NIFI-3029: QueryDatabaseTable supports max fragments property
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1213
2016-12-14 10:51:27 -05:00
jpercivall 266bd80bde NIFI-3192 Adding a check if the FlowFile is null to UpdateAttribute
This closes #1325
2016-12-13 11:30:24 -05:00
Scott Aslan 868795cdaa [NIFI-3159] Add ellipsis to long PG name in access policies shell. This closes #1310 2016-12-13 09:41:31 -05:00
Scott Aslan 45e4514a40 [NIFI-3160] reload canvas when updating PG name of current PG. This closes #1315 2016-12-12 11:45:10 -05:00
Scott Aslan 2c84b44f54 [NIFI-3154] display ellipsis when text is sufficiently long in connection details and connection configuration dialogs. This closes #1305 2016-12-08 12:52:01 -05:00
Koji Kawamura d15bd0c00c NIFI-3165: Update RPG revision when RPGPort is updated. This closes #1308 2016-12-08 08:34:07 -05:00
Bryan Rosander 9ce0af22af NIFI-3138 - Adding urls, transport, proxy settings to rpg portion of flow config xsd
This closes #1304.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2016-12-08 14:37:07 +09:00
Scott Aslan 6b82e25c74 [NIFI-3106] updating hover text for remove counter icon. This closes #1299 2016-12-07 16:42:19 -05:00
jpercivall e36b37692c NIFI-1582 added state to UpdateAttribute as well as updated a few parts that hadn't be touched in years (referenced the 'FlowFileMetadataEnhancer' processor'. Also added a 'NUMBER_VALIDATOR' to StandardValidators
NIFI-1582 removing the option to use cluster state

NIFI-1582 addressing Oleg's comments

NIFI-1582 No longer forcing numbers as the init value and adding getStateValue() to EL instead of using 'ATTRIBUTE_state'

NIFI-1582 Removing init state value

NIFI-1582 Adding documentation for the changes to Init State value

This closes #319
2016-12-06 16:52:02 -05:00
Bryan Rosander 97d2d30423
NIFI-3131 - Qualifying nifi-env in nifi bat scripts, removing popd from toolkit bat scripts
This closes #1293

Signed-off-by: Bryan Rosander <brosander@apache.org>
2016-12-05 23:03:41 -05:00
Bryan Rosander d1f2492dec
NIFI-3132 - Using cygpath and quoting paths correctly in sh scripts when run in cygwin
This closes #1292

Signed-off-by: Bryan Rosander <brosander@apache.org>
2016-12-05 22:16:24 -05:00
Matt Gilman 2d6bba080f NIFI-3133: - Ensuring that Remote Group Ports are always authorized according to their Remote Process Group.
NIFI-3133:
- Using getSourceAuthorizable() when accessing flow files and content.

NIFI-3133:
- Decouple local and remote connectable's to avoid ambiguity with self referencing RPGs.

NIFI-3133:
- Addressing comments from the PR.

NIFI-3133:
- Fixed check verifying source/destination when creating a connection.

NIFI-3133:
- Only showing the go to link when the source component is not a remote port.

NIFI-3133:
- Removing unnecessary checking of remote group port authorization since it's handled by the parent RPG.

NIFI-3133:
- Fixing issue showing the connection details dialog when the source component is a RPG.

NIFI-3133:
- Ensuring the local connectable was found.

This closes #1297

Signed-off-by: jpercivall <JPercivall@apache.org>
2016-12-05 19:45:45 -05:00
Mark Payne fc0d336f36
NIFI-3152: Ensure that we always close the IndexWriter when appropriate in SimpleIndexManager, even if an IOException is thrown when trying to commit the IndexWriter
This closes #1300.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-12-05 16:07:54 -05:00
jpercivall 7633fe35c1 NIFI-3149 Adding proper literal support for number operators ('-' and '+') to ANTLR parser and UI highlighting
This closes #1298
2016-12-05 13:34:54 -05:00
Byunghwa Yun 2b6d6c25d8 NIFI-2866 The Initial Max Value of QueryDatabaseTable won't be case-sensitive
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1212
2016-12-05 12:50:43 -05:00
Mark Payne 8f8b8cdf46 NIFI-3071: Deprecated InputStreams & OutputStream sin org.apache.nifi.stream.io package in favor of using their Java counterparts 2016-12-05 09:29:19 -05:00
Matt Gilman c5ef076786
NIFI-2325:
- Adding support for LDAPS.

This closes #1275.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-12-02 12:41:03 -08:00
Matt Burgess 316cae16d3 NIFI-3140: Restored optional type handling in FetchElasticsearchHttp
This closes #1288

Signed-off-by: jpercivall <JPercivall@apache.org>
2016-12-02 14:10:52 -05:00
Koji Kawamura 7c5bd876bd NIFI-3026: Support multiple remote target URLs
- Added urls in addition to the existing url, to support multiple target
  URLs
- Backward compatibility is provided by returning the first url if
  multipe urls are specified, but component accessing the url doesn't
support multiple urls
- UI is not fully updated yet. Following UI components are planned to be updated
  by different commits
  - Search component: only the first URL is searchable and shown
  - Component status: RPG status shows only the first URL
  - Component action history: only the first URL is searchable and shown
  - Updated Search component to use URLs.

This closes #1208.
2016-12-02 14:01:39 -05:00
Matt Gilman d8d29811f5
NIFI-3135: - Authorizing restricted components on snippet usage.
- Updating REST API docs accordingly.
- Adding some integration tests to exercise the additional restricted component checks.

This closes #1287.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-12-02 12:55:23 -05:00
Matt Burgess 69b23adf1b NIFI-3095: Add EL support to Elasticsearch 2.x (and HTTP) processors
This closes #1276

Signed-off-by: jpercivall <JPercivall@apache.org>
2016-12-02 11:16:23 -05:00
Koji Kawamura 8da38acf31 NIFI-3141: Fixed TailFile ArrayIndexOutOfBounds.
- Added unit test cases to simulate NiFi version update which fails without this fix.
- Added state object migration code, add file.0. prefix to state keys,
  and add length from stored position.

This closes #1289
2016-12-02 07:53:04 -05:00
Bryan Bende 0297352350
NIFI-2017 Fixing failing test PermissionBasedStatusMergerSpec.groovy introduced in previous commit 2016-12-01 14:02:05 -05:00
Mark Payne 02f02b245a NIFI-2017: Fixed typo in nf-port.js that was referencing the wrong variable name to determine whether or not a port is transmitting
Fixed merging logic for root group status. Only consider a port transmitting when there is an active request, not when there is a 'queued' request
This closes #1279
2016-11-30 14:51:28 -05:00
Matt Gilman 1f40f298c0
NIFI-3130: - Loading the temporary instance from the correct type of component.
This closes #1285.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-11-30 14:13:43 -05:00
Mark Payne fff0148a0e NIFI-3129: When adding controller services to a snippet, ensure that we don't add the service multiple times, even when it's referenced by child process groups. This closes #1284 2016-11-30 13:07:51 -05:00
Matt Burgess ba513447d7
NIFI-3076: Fixed handling of 'medium' unsigned integers in JdbcCommon
This closes #1254.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-11-30 13:05:38 -05:00
Matt Gilman bc223fa197
NIFI-3117: - Checking referenced services when performing an action with a snippet that requires it.
- Updating the SnippetManager to automatically expire unused snippets.
- Making the checking of referenced services consistent across component removal.
- Adding checking of referenced services for all nested components in a snippet.
- Updating the REST API docs to clarify when a referenced service is authorized.
- Conditionally authorizing transitive service references.

This closes #1277.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-11-30 12:50:25 -05:00
jpercivall e3c7611347
NIFI-3112 Revert "NIFI-2199 - allows nifi.sh restart through ssh"
This reverts commit 88d125137e.

This closes #1282.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-11-29 17:32:47 -08:00
Mark Payne 9d6ca6e1e6
NIFI-2975: Only start expiring FlowFiles after FlowFile Repository has been restored
This closes #1274.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-11-29 09:54:15 -05:00
joewitt 92f17a995b NIFI-3100-rc2 prepare for next development iteration 2016-11-25 23:49:27 -05:00
joewitt 5536f690a8 NIFI-3100-rc2 prepare release nifi-1.1.0-RC2 2016-11-25 23:49:13 -05:00
Joey Frazee 617b62ac7a NIFI-3104 This closes #1273. Removed tests derived from HAPI test data 2016-11-25 19:39:36 -05:00
Koji Kawamura f4f60c3610 NIFI-3103: This closes #1272. Exclude localhost.cert from RAT check. 2016-11-25 19:33:48 -05:00
Andy LoPresto 2c3714536f
NIFI-3024 Added key migration for sensitive processor properties contained in flow.xml.gz. (nifi.sensitive.props.key)
This closes #1261.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-11-23 13:26:18 -08:00
Mark Payne 7ff14f7191 NIFI-3091: Ensure that we set the appropriate size on FlowFiles when modifying them
This closes #1267

Signed-off-by: jpercivall <JPercivall@apache.org>
2016-11-23 12:07:44 -05:00
Mark Payne 91ff810dba NIFI-3088: Ensure that on recovery of FlowFile Repository, if we find a FlowFile that maps to an unknown queue that we log a warning that the queue is missing and drop the FlowFile, rather than throwing an NPE
This closes #1266

Signed-off-by: jpercivall <JPercivall@apache.org>
2016-11-23 11:27:14 -05:00
joewitt 22143e386d NIFI-2991 This closes #1265. was allowed to restore twitter4j json.org usage until April. Resolution for that will be in NIFI-3089 2016-11-23 10:21:41 -05:00
Matt Burgess 7fc7494b21 NIFI-3087: This closes #1263. Added unit tests to PutElasticsearch(Http) to illustrate issue 2016-11-23 08:22:34 -05:00
Matt Burgess d3dbac50a8 NIFI-3087: Fixed issue with partial failure responses in PutElasticsearch(Http) 2016-11-23 08:11:49 -05:00
Jeff Storck 066accc274 NIFI-3086 Updated TailFile.java to deal with windows paths that were breaking the regex parsing Fixed documentation misspelling in TailFile.java Fixed assertions in several tests in TestTailFile that were dependent on the received order of flowfiles, this was differing between Windows and MacOS/Linux
This closes #1264

Signed-off-by: jpercivall <JPercivall@apache.org>
2016-11-23 00:01:12 -05:00
Joseph Percivall 50667ed222 NIFI-3084 Fixing TestPersistentProvenanceRepository and closing a stream leak in RecordReaders.
This closes #1260.
2016-11-22 17:31:47 -05:00
Oleg Zhurakousky 913ae91614 Added posix guard to ignore tests on windows
This closes #1259

Signed-off-by: jpercivall <JPercivall@apache.org>
2016-11-22 17:11:25 -05:00
Bryan Bende 78fed3ed56
NIFI-3068 Adding @RequiresInstanceClassLoading to PutHDFS to ensure the security configuration of the Hadoop client is fully isolated between instances of PutHDFS 2016-11-22 16:41:44 -05:00
Koji Kawamura 36796b5440 NIFI-3078: Test InvokeScriptedProcessor with JS.
This closes #1255.
2016-11-22 21:42:36 +01:00
Matt Burgess fa5fed9bb5 NIFI-3082: Fixed status code handling in PutElasticsearchHttp
This closes #1258.
2016-11-22 21:32:03 +01:00
Kathik Narayanan 9e884f6120 NiFI-3064 Convert to Avro fails when a MySQL query returns two columns with the same name even with different aliases
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1251
2016-11-22 11:49:25 -05:00
Mark Payne aa99cc6822
NIFI-3079: Ensure that in all cases we increment session's bytesRead count when finished reading from Content Repo
This closes #3079.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-11-22 11:26:22 -05:00
Scott Aslan ead9205458 [NIFI-3075] remove fixed width from combo options and add max-width css style to display ellipsis when options text is too long. This closes #1253 2016-11-21 17:27:03 -05:00
Mike Moser 15af764dd8 NIFI-2996
- validate processors only when they are in STOPPED state
- report validation errors via REST API on processors/services/tasks/ports only when they are in the STOPPED state
- This closes #1192
2016-11-21 17:02:25 -05:00
joewitt fb9cbccc38 NIFI-2954 This closes #1244. Moved StandardPropertyValidator to nifi-utils, documented scope/purpose of a few util libs, removed deps from nifi-utils. 2016-11-21 16:30:42 -05:00
Matt Burgess dba9d5924a NIFI-3011: Added Elasticsearch5 processors
This closes #1233

Signed-off-by: jpercivall <JPercivall@apache.org>
2016-11-21 15:58:40 -05:00
Matt Gilman 7f5eabd603
NIFI-3050: Implemented access control logic for restricted components.
- Addressing comments from PR.
- Adding restricted tags to relevant components.
- Showing a restricted icon overlay on the processor node on the canvas. (+1 squashed commit)
Squashed commits:
[f487682] NIFI-3050:
- Introducing a Restricted annotation for components that require elevated privileges to use.
- Updating the new Processor, Controller Service, and Reporting Task dialogs to include these details and prevent unauthorized selection.
- Including the Restricted description in the generated component documentation.
- Updating processor access control integration test to verify restricted component creation.
- Updating the developer, user, and admin guide to include the restricted component policy.

This closes #1247.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-11-21 12:07:48 -08:00
Matt Burgess eb5abf809d
NIFI-2927: Added Validation Query to HiveConnectionPool
This closes #1252.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-11-21 14:21:22 -05:00
Peter Wicks d65f46f7a5 NIFI-2750
Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-2750

Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-2750 Tweaking Property Retrieval

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1001
2016-11-21 12:58:17 -05:00
Mark Payne 2ee66de1a6 NIFI-3066: Create unit test to reproduce bug that results in IllegalArgumentException: Cannot migrate FlowFiles from a Process Session to itself
NIFI-3066: Ensure that when a Bin is created, it is always created with its own new session

This closes #1245
2016-11-21 11:34:14 -05:00
Joey Frazee e731f09573 NIFI-3061: Bumped azure-eventhubs version to 0.9.0
This closes #1250.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2016-11-21 10:30:28 +09:00
Matt Gilman c8830742ee NIFI-3020:
- Introducing a strategy for identifying users.
- Fixing issue with the referral strategy error message.
- Adding code to shutdown the application when the authorizer or login identity provider are not initialized successfully.

NIFI-3020:
- Updating the admin guide to document the identity strategy.

NIFI-3020:
- Ensuring the request replicator attempts to shutdown regardless of whether the flow service properly terminates.

This closes #1236
2016-11-19 08:38:41 -05:00
Mark Payne 1be0871473 NIFI-2854: Refactor repositories and swap files to use schema-based serialization so that nifi can be rolled back to a previous version after an upgrade.
NIFI-2854: Incorporated PR review feedback

NIFI-2854: Implemented feedback from PR Review

NIFI-2854: Ensure that all resources are closed on CompressableRecordReader.close() even if an IOException is thrown when closing one of them

This closes #1202
2016-11-18 14:53:13 -05:00
James Wing 7206318ecf NIFI-3033 GenerateFlowFile Dynamic Properties
This closes #1222.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2016-11-18 11:06:20 +09:00
Bryan Rosander e5eda63705
NIFI-2943 - Toolkit uses JKS type over PKCS12 when creating truststore because non-Bouncy Castle providers cannot read certificates from PKCS12 truststore.
Peer review feedback (+2 squashed commits)
Squashed commits:
[0102c8e] NIFI-2943 - Peer review feedback
[9bcd495] NIFI-2943 - pkcs12 keystore improvements

1. loading pkcs12 keystores with bouncy castle everywhere
2. tls-toolkit client using jks truststore when keystore type is specified differently
3. tests

This closes #1165.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-11-16 17:13:10 -08:00
Scott Aslan 878db82375 [NIFI-2949] update remote process group port styles. This closes #1226 2016-11-16 17:08:06 -05:00
Oleg Zhurakousky f1eca0a09a NIFI-1002 added L&N entries for Bouncy Castle
This closes #1235
2016-11-16 15:30:47 -05:00
Koji Kawamura 31ea50bce9 NIFI-1002: Added WebSocket support.
- Added LICENSE and NOTICE files.
- Fixed UnitTest errors.

This closes #1231
2016-11-16 12:17:37 -05:00
Bryan Bende 2f9ec03242 NIFI-2909 Fix logic in AbstractConfiguredComponent setProperties() for setting classpath resources in the InstanceClassLoader
This closes #1232
2016-11-16 11:34:52 -05:00
Matt Gilman 49afacc3ab NIFI-3022:
- Returning the appropriate authorizable when accessing provenance events for a manual DROP event by emptying a queue.
- Populating the component details of a provenance event when the source is a connection.
2016-11-16 10:13:18 -05:00
joewitt 45a5f5295c NIFI-2374 This closes #712. updated to latest tika versions where possible, updated doc, commented why cannot update media nar 2016-11-15 23:18:55 -05:00
Andre F de Miranda 3e571f9f1f NIFI-2374 - Adjust documentation wording to clarify IdentifyMimeType
is a non exhaustive list of mime type values
2016-11-15 22:42:21 -05:00
Koji Kawamura 26a5881d21 NIFI-1002: Added WebSocket support.
NIFI-1002: Added WebSocket support.

- Reflecting review comments
- Added displayName to peroperty descriptors

NIFI-1002: Added WebSocket support.

This closes #1184

- Reflecting review comments:
  - Removed unnecessary use of NarCloseable.withComponentNarLoader.
  - Removed intermediate on memory queue to make it simpler and more
    robust. Received messages in WebSocket layer now will be transferred
to downstream relationships directly.
2016-11-15 19:34:59 -05:00
joewitt b026f0bebe NIFI-2991 removed json.org references as appropriate for build. Twitter proc no longer deployed but reference kept in notice just in case someone uses it
This closes #1230

Signed-off-by: jpercivall <JPercivall@apache.org>
2016-11-15 18:09:27 -05:00
joewitt 6696529c99 Revert "NIFI-1893 Add processor for validating JSON"
This reverts commit f11682202b.

Signed-off-by: jpercivall <JPercivall@apache.org>
2016-11-15 18:09:14 -05:00
joewitt 1c9077334f NIFI-2991 removed usage of json lib and components still using it
Signed-off-by: jpercivall <JPercivall@apache.org>
2016-11-15 18:09:09 -05:00
Mark Payne 721964b7d8 NIFI-3036: When we replay a FlowFile, ensure that we are using the 'golden copy' of the associated Resource Claim, if the claim is still writable. Ensure that StandardResourceClaimManager retains the 'golden copy' of a Resource Claim until it is no longer writable and has a claim count of 0
This closes #1223

Signed-off-by: jpercivall <JPercivall@apache.org>
2016-11-15 16:51:34 -05:00
Scott Aslan 81357d4456 [NIFI-3042] Update class name for disabled combo option. This closes #1228 2016-11-15 16:43:27 -05:00
Mark Payne 9f3bf121f9 NIFI-3040: Fixed bug where we were generating a RepositoryRecord with an 'UPDATE' type instead of a 'CREATE' time for Replay of FlowFiles. This caused FlowFile to have no attributes when restored from the FlowFile Repository.
This closes #1224
2016-11-15 16:37:51 -05:00
jpercivall 2614b83543 NIFI-3043 Fixing Kafka unit test failures 2016-11-15 16:31:10 -05:00
Bryan Bende fe59b3415c NIFI-3003 Upgrading hadoop.version to 2.7.3 and fixing TDE issue with PutHDFS, ensuring clean up of instance class loaders, and adding classpath resource property to all HDFS processors
NIFI-3003 Addressing review feedback

NIFI-3003 added minor notice updates

This closes #1219
2016-11-15 16:23:27 -05:00
Pierre Villard 45bc3e054d NIFI-2751 - Pull batch in round-robin manner - fix for / by 0 2016-11-15 16:19:00 -05:00
Scott Aslan 45bf8430fc [NIFI-2844] Update CSS styles for Cluster Summary Dialog icons and refresh button. This closes #1104 2016-11-15 11:55:42 -05:00
Scott Aslan 8f928e5d6e NIFI-2926: add policy action item to the user table and add user centric policy dialog with user policy listing table compolete with GoTo action.
This closes #1188.
2016-11-15 09:58:58 -05:00
Andy LoPresto 59fea1cb4e NIFI-2654 Enabled encryption coverage for login-identity-providers.xml.
Squashed commits:
[5dd22a9] NIFI-2654 Updated administration guide with login-identity-providers.xml flags.

Exposed master key retrieval code in NiFiPropertiesLoader.
Added logic to decrypt login identity providers XML configuration.
Updated login-identity-providers.xsd to include encryption scheme attribute.
Added unit tests. (+18 squashed commits)
Squashed commits:
[57c815f] NIFI-2654 Resolved issue where empty LIP property elements could not be encrypted.
Added unit test and resource.
[27d7309] NIFI-2654 Wired in serialization logic to write logic for LIP.
Added comprehensive unit test for LIP & NFP in same test.
[b450eb2] NIFI-2654 Finalized logic for preserving comments in LIP parsing.
[5aa6c9c] NIFI-2654 Added logic for maintaining XML formatting (comments and whitespace) for LIP.
Added unit tests (w/o encryption works; w/ does not).
[b53461f] NIFI-2654 Added unit test for full tool invocation migrating a login-identity-providers.xml file and updating file and bootstrap.conf with key.
[2d9686c] NIFI-2654 Updated tool description and various logging statements.
Added unit test for full tool invocation encrypting a login-identity-providers.xml file and updating file and bootstrap.conf with key.
[8c67cb2] NIFI-2654 Added logic to encrypt LIP XML content.
Added unit tests.
[8682d19] NIFI-2654 Added logic to handle "empty" (commented) LIP files.
Added unit tests.
[077230e] NIFI-2654 Fixed logic to decrypt multiline and multiple-per-line XML elements.
Added unit tests and resources.
[d5bb8da] NIFI-2654 Ignored unit test for unreadable conf directory because directory was causing Maven build issues.
Removed test resources.
[7e50506] NIFI-2654 Fixed AESSensitivePropertyProvider bug handling cipher text with whitespace.
Added unit test.
[b69a661] NIFI-2654 Fixed AESSensitivePropertyProviderFactoryTest to reflect absence of key causes errors.
[6f821b9] NIFI-2654 Added standard password to arbitrary encryption test for use in test resources.
[d289ffa] NIFI-2654 Added LIP XML decryption.
Added unit tests.
[a482245] NIFI-2654 Added LIP test resources.
[7204df4] NIFI-2654 Changed logic to only perform properties encryption when file path is provided.
[729e1df] NIFI-2654 Removed population of default file locations for bootstrap.conf, nifi.properties, and login-identity-providers.xml as not all files may be desired.
Added/updated unit tests.
[7dba5ef] NIFI-2654 Started LIP work (arguments & parsing).
Added unit tests.

Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #1216
2016-11-14 22:39:06 -05:00
joewitt 8ad3125168 NIFI-2380 This closes #817. corrected licensing and notice and remove conflicting mail deps 2016-11-14 21:04:33 -05:00
Andre F de Miranda a347ebf87b NIFI-2380 - Introduce ExtractTNEFAttachment - A processor to extract
attachments embedded within a Transport Neutral Encapsulation Format (TNEF) attachment (i.e. winmail.dat)
2016-11-14 21:04:12 -05:00
Andy LoPresto 4d1bcc808f
NIFI-3032 Resolved issue where multiple invocations of NiFiPropertiesLoader.withKey() used cached key.
Added unit tests and resources.

NIFI-3032 Fixed bug in AESSensitivePropertyProvider#getIdentifierKey where the result was always the max available key size, not the size of the current key.
Added unit test.

This closes #1220

Signed-off-by: Bryan Rosander <brosander@apache.org>
2016-11-14 17:50:32 -05:00
Matt Gilman ae4f27a627 NIFI-3023:
- Ensuring there are no authorization issues prior to considering the remote process group's transmitting flag.
This closes #1205
2016-11-14 16:23:36 -05:00
Matt Gilman 2c91a1a33d NIFI-3023:
- Verifying permissions prior to checking Remote Process Group transmission status.
2016-11-14 16:03:47 -05:00
Mathias Tiberghien 65b7b377e3 NIFI-1526: DefaultSchedule annotation should be use on Custom Processor to set the default scheduling strategy, scheduling period or max number of concurrent task for each instance of the processor
DefaultSettings annotation should be use on Custom Processor to set the default penalty period, the yield duration or the bulletin log level for each instance of the processor
2016-11-14 12:04:21 -05:00
Oleg Zhurakousky 41f519e84c NIFI-2851 initial commit of perf improvements on SplitText
- introduced org.apache.nifi.stream.io.util.TextLineDemarcator
- refactored SplitText to use org.apache.nifi.stream.io.util.TextLineDemarcator
- updated SplitText's capability discription to provide more clarity around splits with headers.
2016-11-11 15:45:40 -05:00
Mark Payne b73ba7f8d4 NIFI-2999: When Cluster Coordinator changes, purge any old heartbeats so that we don't disconnect a node due to very old heartbeats
This closes #1210
2016-11-11 15:13:49 -05:00
joewitt b9ef0fb847 NIFI-2818 This closes #1059. aligned read method with read/write method 2016-11-11 14:52:13 -05:00
Andre F de Miranda f32bdf7be9 NIFI-2818 - Minimise fs permission required by NiFi
1 - Replace a r/w test over $NIFI_HOME/lib that is performed
            but never utilised by a RO test
            2 - Rename ensureDirectoryExistAndCanAccess to
            ensureDirectoryExistAndCanReadAndWrite and deprecate the
            former
2016-11-11 14:49:33 -05:00
Scott Aslan 52cde9ad13 [NIFI-2898] restore ellipsis for processor type, controller servies type, and reporting task type descriptions. This closes #1191 2016-11-11 14:46:20 -05:00
Byunghwa Yun 91f0170416 NIFI-3027: ExecuteProcess supports the expression language 2016-11-11 14:41:35 -05:00
Pierre Villard 06f191ca10
NIFI-2912 Allow custom text in GenerateFlowFile
This closes #1206.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-11-11 10:18:43 -08:00
Matt Gilman 9f417a84b9 NIFI-3002:
- Removing legacy code when retrieving site to site details when the target instance is this local NiFi.
- Ensures the UI accurately reflects available remote ports.
NIFI-2603:
- Fixing color of remote output port status in connection label.

This closes #1189.
2016-11-11 10:06:41 -05:00
Scott Aslan 4957b628fb [NIFI-3021] remove white-space:no-wrap on view configuration properties tables textareas
This closes #1204.
2016-11-10 22:12:08 +01:00
Matt Gilman f83863ebae NIFI-3009:
- Fixing NaN error when backpressure is not configured.
This closes #1200
2016-11-10 11:39:08 -05:00
Scott Aslan b426de1793 [NIFI-3012] update regex for the summary tables input, output, transferred, received, and sent column parse function. This closes #1197 2016-11-10 09:25:08 -05:00
Matt Burgess 435d9cf353 NIFI-2991 This closes #1194. Exclude org.json dependency from Hive bundle 2016-11-10 08:33:43 -05:00
João Henrique Ferreira de Freitas 1b5a28d130 NIFI-3013 GetSNMP fix stop and start null pointer exception
This closes #1195.
2016-11-10 13:30:16 +01:00
Mark Payne c441a8696d NIFI-2850 This closes #1115. Added a migrate() method to ProcessSession and refactored BinFiles and MergeContent to use it 2016-11-09 16:25:03 -05:00
Jeff Storck cc2fbcaac4 NIFI-3008 Enabled expression language on BASE_DIRECTORY and FILENAME properties in TailFile
This closes #1190
2016-11-09 12:11:11 -05:00
joewitt 3e892c55ec NIFI-2971 This closes #1172. addressed RPM bundled deps and handling exception for bootstrap cl construction 2016-11-09 11:08:43 -05:00
Matt Gilman bff89f17b3 NIFI-401: - Minor tweaks to PR #1117. - Ensuring existing configuraiton is retained and shown until the user explicits changes it. - Retaining, but disabling, deprecated options.
This closes #1185
This closes #1117
This closes #512

Signed-off-by: jpercivall <JPercivall@apache.org>
2016-11-08 17:06:43 -05:00
Brian Eugley 7eca2037bd NIFI-401
Signed-off-by: jpercivall <JPercivall@apache.org>
2016-11-08 17:06:36 -05:00
patricker 4b4e099f2e NIFI-2764
Signed-off-by: Matt Burgess <mattyb149@apache.org>

NIFI-2764: Updated call to convertToAvroStream in TestJdbcCommon

This closes #1009
2016-11-08 14:45:49 -05:00
Matt Burgess 63bda32a8b NIFI-2998: Add validator to Avro Record Name in InferAvroSchema
This closes #1187
2016-11-08 14:14:40 -05:00
Bryan Bende 78020825e9 NIFI-2841 Refactoring logic in SplitAvro RecordSplitter to avoid making two calls in a row to reader.hasNext()
This closes #1088
2016-11-08 13:50:42 -05:00
Bryan Bende d1d053725b NIFI-2909 Adding per-instance class loading capability through @RequiresInstanceClassLoading annotation
NIFI-1712 Applying per-instance class loading to HBaseClientService to allow specifying Phoenix Client JAR

-Refactoring the ClassLoading so that every processor, controller service, and reporting task gets an InstanceClassLoader with a parent of the NAR ClassLoader, and only components with @RequiresInstanceClassLoading will make a copy of the NAR ClassLoader resources, and addressing some review feedback

This closes #1156
2016-11-08 13:21:27 -05:00
Matt Gilman 2f0d9a34f1 NIFI-2982:
- Updating the labels in the Add Users/Groups to the Policy dialog..

This closes #1178.
2016-11-08 16:55:27 +01:00
Matt Gilman 9d3fe3f03e NIFI-2989:
- Updating the content of the delete policy dialog to make it a little more clear that the permissions for the component will revert back to the inherit policy.

This closes #1183.
2016-11-08 14:58:11 +01:00
Matt Gilman aa8ba0bd1c NIFI-2965:
- Ensuring the selection is cleared when going to the Process Group where the selected component policy is defined.
- Ensure that the selection context is updated when going to a component or group.

This closes #1174.
2016-11-08 14:49:12 +01:00
Oleg Zhurakousky 5fd4a55791 NIFI-2778 added abilty to interrupt Lucene search
polishing

This closes #1138
2016-11-07 16:28:16 -05:00
Pierre Villard 6aefc0b910 NIFI-2751 NIFI-2848 Get batch of flow files in a round-robin manner
This closes #1111
2016-11-07 14:01:18 -05:00
Oleg Zhurakousky c13cfa6ea6 NIFI-1322 fixed breaking changes introduced in previous commit
This closes #1181
2016-11-04 08:23:10 -04:00
Pierre Villard a7d06412f8 NIFI-1322 - PutHDFS - allow file append resolution 2016-11-04 08:23:10 -04:00
James Wing 50010fb340 NIFI-2948 Fix DynamoDB Cached Credentials
This closes #1182
2016-11-04 07:04:25 -04:00
Matt Burgess 0841f14c18
NIFI-2928: Fixed checkstyle violation in FetchFile
This closes #1180

Signed-off-by: Bryan Rosander <brosander@apache.org>
2016-11-03 14:46:10 -04:00
Oleg Zhurakousky ed035f85e3 NIFI-2928 added support to validate accessibility of the file system
NIFI-2928 polishing

This closes #1160
2016-11-03 12:25:31 -04:00