Commit Graph

5198 Commits

Author SHA1 Message Date
Mike Thomsen 3e7816da43
NIFI-5902 Added GeoEnrichIPRecord.
NIFI-5902 Updated GeoEnrichIPRecord with more error handling.

NIFI-5902 Made changes requested in a review.

NIFI-5902 Updated EL support on database path.

NIFI-5902 Made updates based on code review.

NIFI-5902 Made two changes from a code review.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3231
2019-06-12 15:25:42 -04:00
Mark Payne 8bed249f37
NIFI-6353: If LoadBalanceSession determines remote node's queue is full, mark session complete and penalize the Partition, rather than just penalizing the session. This allows the client to move on to another session, which avoids an issue where backpressure on one queue can prevent data from being transferred between nodes in another queue
This closes #3529.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-06-12 11:32:41 -04:00
Mark Payne 2465f67dfe
NIFI-6339: Only call @OnConfigurationRestored method on components when they are created if FlowController has been initialized
This closes #3530.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-06-12 10:56:04 -04:00
Shawn Weeks e913f5706f
NIFI-6243 Add Support for AtomicDistributedCache to the HBase 1.x and 2.x Map Cache Services
This closes #3462.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2019-06-12 15:31:39 +09:00
Koji Kawamura 9158498719
Renew my key.
Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2019-06-12 15:08:49 +09:00
Matthew Burgess 05f3cadee8 NIFI-6370: Allow multiple SQL statements in PutDatabaseRecord
This closes #3528.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2019-06-12 10:32:11 +09:00
Mark Payne f15332ff87 NIFI-6182: Updated dependency on Lucene to Lucene 8.0.0. Updated code necessary to use the new API. Updated WriteAheadProvenanceRepository so that upon startup if provenance indexes are written using the old Lucene format, , they are considered 'defunct' and the events are re-indexed in a background thread into a new index that uses the Lucene 8 format and the old index is then removed. Added Provenance Repository that consists of about 30 events and added integration test to ensure that the repo can be started up/initialized when pointing to a prov repo that was written using the old lucene format. 2019-06-11 09:45:46 -04:00
Andres Garagiola 1975101292 NIFI-6361: Add fix to PutFile processor
When PutFile uses 'replace' conflict resolution and max files, there is an issue when the folder has X files, and the limit is also X. The processor fails instead of replacing it, leaving X files. This commit fixes that issue.

This closes #3524.

Signed-off-by: Andres Garagiola  <andresgaragiola@gmail.com>

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2019-06-10 15:45:19 +09:00
Evan Reynolds 99350b761d NIFI-6349 Fix to MergeRecords when handling fragments over multiple iterations
Fixed fragment count attribute check

This closes #3517.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2019-06-10 14:33:11 +09:00
Andrew Lim 0b8d05aabd NIFI-6319 Update User Guide for RPG/S2S chnages for multiple URLs, batch settings and remote input/output ports
NIFI-6319 Improved Remote Process Group Ports screenshot

This closes #3503.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2019-06-10 14:18:55 +09:00
Matthew Burgess 99d6ed244c
NIFI-6352: Add ability to report all component metrics to Prometheus
This closes #3519.

Signed-off-by: Kevin Doran <kdoran@apache.org>
2019-06-07 15:30:38 -04:00
Ferenc Szabó aefdc9377c NIFI-6218 Support setting transactional.id in PublishKafka/PublishKafkaRecord
Added new property to the processors. If it is empty, then the behaviour is the same as before.
Adde unit test to check if KafkaProducer receives the new property.

Testing Done:
Unit tests.
Connecting to kafka and verifying transactionalId-s in kafka server log.

fixing review findings

This closes #3439.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2019-06-07 17:55:52 +09:00
Peter Wicks 2d3fce81c1
NIFI-6350 Fixed misleading flow controller startup error message.
This closes #3516.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2019-06-05 19:25:13 -07:00
Matthew Burgess 1d18735076
NIFI-6348: Added Custom ORDER BY Column property to GenerateTableFetch
NIFI-6348: Fixed doc

This closes #3515.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-06-05 10:00:12 -04:00
Mark Payne 60b5c13ce9
NIFI-6341: Addressed bug that can result in a deadlock if Thread 1 adds a Controller Service to a Process Group (which obtains group's write lock and AbstractComponent's Lock) while at the same time Thread 2 attempts to peform validation on the service (which obtains Controller Service's write lock, then AbstractComponent's Lock)
This closes #3510.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-06-04 09:55:26 -04:00
Matthew Burgess 08d2e69878
Revert "NIFI-6244: Provide default ORDER BY clause in MSSQL adapters"
This reverts commit 7fc49c2cb8.

This closes #3512
2019-06-04 09:02:03 -04:00
Bryan Bende e7f4af6801 NIFI-6315 Ensuring remote ports get tracked correctly when saving/retrieving versioned flows
NIFI-6315 Renaming public ports during import/change-version when name alreadye exists, and ignorning name change from flow diffs

NIFI-6315 Ensuring public ports get renamed during copy/paste

NIFI-6315 Set max concurrent tasks on Port based on value from VersionedPort

This closes #3488.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2019-06-03 11:25:04 +09:00
Mike Thomsen 407add7847 NIFI-5916 Added an option to enable empty flowfiles to be sent if there are no results from a query.
NIFI-5916 Fixed potential NPE.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3315
2019-06-01 11:27:53 -04:00
Andy LoPresto ed19f61682
NIFI-6331 Removed nifi-testharness module.
This closes #3506.
2019-05-31 13:01:46 -07:00
Andy LoPresto a9d1dd7b08 NIFI-6337 Resolved groovy unit test execution problems in nifi-properties-loader and fixed failing unit tests.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3508
2019-05-31 08:44:17 -04:00
Joe Witt a951a8ec61
NIFI-6329 trust update did not help and oraclejdk is not working on xenial - removing it
This closes #3505.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2019-05-30 18:05:40 -07:00
slyouts 097b9c3075 NIFI-6022 Added try/catch to protect against secondary exception in a primary exception catch clause
NIFI-6022 mods per PR review

NIFI-6022 mods per PR review

This closes #3476.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2019-05-31 08:51:36 +09:00
Joe Witt de27ab94b0
NIFI-6329 added dist section to specify trusty 2019-05-30 09:47:00 -04:00
Satwik 8560eb995a NIFI-5435 Added Prometheus reporting task.
NIFI-5435 Fixed Rebase Issue
NIFI-5435 changes in capability description of reporting task
NIFI-5435 Changes in capability description of reporting task
NIFI-5435 Changes to build Property Descriptors in static constructor
NIFI-5435 Indentation changes
NIFI-5435 Test code changes
NIFI-5435 Minor code changes
NIFI-5435 Changes related to client authentication
NIFI-5435 Prometheus metrics changes
NIFI-5435 Added process group name and id to metrics

This closes #3257

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2019-05-30 08:08:07 -04:00
Matt Gilman 999cfdcac9
NIFI-6302:
- Updating integration tests to verify pruned results.

This closes #3487.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2019-05-29 14:45:00 -07:00
Andy LoPresto e6c843f465
NIFI-6323 Changed URLs for repositories, project description, and mailing lists to use HTTPS.
NIFI-6323 Changed URLs for splunk.artifactoryonline.com to use HTTPS (certificate validity warning in browsers, but command-line connection using openssl s_client is successful).
NIFI-6323 Changed URLs for XMLNS schema locations to use HTTPS (the XMLNS and schema identifier remain http:// because they are not designed to be resolvable).
NIFI-6323 Fixed Maven XML schema descriptor URLs.

This closes #3497
2019-05-29 14:36:40 -04:00
Mike Thomsen 6a06cd3094 NIFI-6304 added trim, toLowerCase and toUpperCase to record path operations.
NIFI-6304 Updated code based on code review.

NIFI-6304 Updated documentation.

NIFI-6304 Refactored to make it simpler

NIFI-6304 Reverted Concat to its last state.

This closes #3478.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2019-05-28 21:06:20 +09:00
Koji Kawamura 99b20ac2d1 NIFI-6303 CSVWriter should write schema defined fields first, then extra ones
This closes #3484

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2019-05-26 16:52:12 -04:00
Andy LoPresto 6be47de87e NIFI-6296 Improved PR template messaging around commit process. 2019-05-24 16:18:08 -04:00
Peter Turcsanyi 2bfffd35b2 NIFI-6317: HandleHttpRequest timeout handling issues
- set cleanup thread scheduling to 5 seconds in order to prevent stale request staying in the cache for a long time
- disable timeout handling in HandleHttpRequest in order to prevent a new flowfile being generated at request expiration
- use HttpServletResponse.SC* status codes everywhere in HandleHttpRequest, get rid of importing javax.ws.rs
- add some more error logging to make bug investigations easier
- add a short message to 503 error responses to make bug investigations easier

This closes #3490.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2019-05-24 14:43:50 -04:00
Rob Fellows 0a31c38790
NIFI-6253 - Remove invalid/unnecessary argument in call to nfCanvasUtil.reload
This closes #3491
2019-05-24 11:02:09 -04:00
Rob Fellows 410854f53e
NIFI-6316 - Upgrade jQuery to the latest version (3.4.1).
This closes #3489
2019-05-24 09:52:02 -04:00
Bryan Bende b09b84226c
NIFI-6311 Upgrade to nifi-registry-client 0.4.0
This closes #3485.
2019-05-23 12:48:33 -04:00
thenatog fe68d43e1d
NIFI-6280 - Broke out the matching for /access/knox/** and /access/oidc/** to allow the Jetty security filters to be applied in the /access/oidc/logout and /access/knox/logout cases.
NIFI-6280 - Updated terminology in JwtAuthenticationFilter to authentication instead of authorization. Added stricter token parsing using an explicit regex pattern. Added tests.
NIFI-6280 - Updated terminology from Authorization to Authentication.
NIFI-6280 - Updated the access logout method to use getNiFiUserIdentity(). Updated javascript logout method to handle errors.
NIFI-6280 - Fixing checkstyle issues.
NIFI-6280 - Added some javadoc comments and logging. Renamed some variables for clarity. Fixed handling of exception when JWT does not match expected format.
NIFI-6280 - Cleaned up checkstyle, increased log severity level for logout action, and cleaned up Groovy syntax in test.

This closes #3482.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2019-05-22 17:05:06 -07:00
Koji Kawamura 8a50cb10b2 NIFI-2933 Remote input/output ports at any PG
Specify remote access at port creation.
Incorporated comments, and finished refactoring.
Renamed RootGroupPort to PublicPort.
Fix error message for creating a connection from a child PG having only PublicPorts.
Enhanced ProcessGroup instances rendered in the parent ProcessGroup
Loosen Port move check, allow moving public port between PG.
Show 'Remote NiFi Instance' info on Connection dialogs
Make labels narrative.
'Within Remote Group'.
Fixed DTO (de)serialization.
Return null only if all values are null.

This closes #3351.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2019-05-22 15:44:16 -04:00
Matt Gilman a97766d62f NIFI-6302:
- Ensuring Process Group contents are pruned when appropriate.

This closes #3477.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2019-05-22 14:12:50 -04:00
Koji Kawamura c470608daf NIFI-6035 Fix check-style issue 2019-05-22 16:07:17 +09:00
Aaron Leon 81ddd02ca8 NIFI-6100 Use setBytes in JdbcCommon for binary types
This closes #3410

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2019-05-21 14:22:56 -04:00
Mark Payne 8245bc3f80 NIFI-5922: Ensure that we import any default variable values on flow import 2019-05-21 11:19:17 -04:00
samhjelmfelt 650c6aa820 Small fix after rebase 2019-05-21 11:19:17 -04:00
Mark Payne 60b8fcac1b NIFI-5922: Bug fixes; initialize, setup, and enable controller services; code cleanup 2019-05-21 11:19:17 -04:00
samhjelmfelt 146689b36f NIFI-5922: Renaming from NiFi-Fn to NiFi-Stateless
Fixed docker image and moved it into the nifi-docker project

Fixed Docker container, YARN runtime, and OpenWhisk runtime
2019-05-21 11:19:17 -04:00
Mark Payne cbafd8f834 NIFI-5922: Addressed checkstyle violations; added to README Updates to nifi-fn proposal:
- Separated into nifi-fn-core, nifi-fn-bootstrap, nifi-fn-nar, nifi-fn-assembly; fully unpacks nars and runs flows
- Rebased against master and updated to version 1.10.0-SNAPSHOT
- Removed dependency on nifi-framework-core
- Added LICENSE/NOTICE files
2019-05-21 11:19:17 -04:00
Sam Hjelmfelt 417b3955d7 NIFI-5922: First Commit for NiFi-Stateless 2019-05-21 11:19:17 -04:00
archon 4d18eaa481 NIFI-6035: 1. Add formatWithTimeZone() and toDateWithTimeZone(); 2. Their test code and docs.
This closes #3481

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2019-05-21 07:33:45 -04:00
Otto Fowler 4b509aa5a5
NIFI-3221 This closes #3396. Add a new property for setting the argument passing strategy, either the existing parameter, or by adding new dynamic parameters, along with implementation and tests
This allows for passing arguments with quotes.

Signed-off-by: Joe Witt <joewitt@apache.org>
2019-05-15 15:03:07 -04:00
Michael Karpel d1fd1f5092 Support for flowfile attribute in TABLE_NAME
This closes #3472

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2019-05-15 12:31:11 -04:00
Mark Payne f08c2ee43f
NIFI-6285: Addressed issue that resulted in swapped data not being swapped back in if load balancing strategy changed while data was swapped out; added integration tests for swapping. In testing, also encountered an issue with data being swapped out while swap files were being recovered causing the queue size to be wrong and causing errors about not being able to swap data in, because it attempted to swap the data in twice.
This closes #3473.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-05-14 15:31:15 -04:00
Alan Jackoway 595835f6ee NIFI-6281 Rename ISO8061_INSTANT_VALIDATOR to ISO8601_INSTANT_VALIDATOR
ISO8061_INSTANT_VALIDATOR was misnamed - the correct standard is ISO 8601.
Also updated the error messages to have spaces in them, which is how the ISO site
displays them.

This closes #3465

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2019-05-14 12:45:49 -04:00
Bryan Bende 7825e40d15
NIFI-6262 - Upgrade to nifi-nar-maven-plugin 1.3.1
This closes #3463.
2019-05-14 11:00:21 -04:00