Commit Graph

3849 Commits

Author SHA1 Message Date
Mark Payne 05b5dd1488 NIFI-4409: Ensure that when record schema is inherited, the schema from teh ResultSet is used instead of the schema from the RecordReader because the schema from the RecordReader mmay not match the actual data
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2169
2017-09-22 12:41:17 -04:00
Pierre Villard 50d018566d
NIFI-4403 - add group name to bulletins model. This closes #2167 2017-09-22 11:53:58 -04:00
cam 0c0c33411d NIFI-4384 - Enhance PutKudu processor to support batch insert
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2160.
2017-09-22 10:56:15 +02:00
Bryan Bende 329dbe3a64 NIFI-4405 Adding charset property to GenerateFlowFile
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2168.
2017-09-21 17:10:47 +02:00
Bryan Bende a813ae113e NIFI-4391 Ensuring channel is closed when unable to connect in SocketChannelSender
NIFI-4391 Adding debug logging of client port upon connection

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2159.
2017-09-21 16:30:10 +02:00
Koji Kawamura 7b07eb0577 NIFI-4228: Fix PutDatabaseRecord to ignore unmatched fields
- Unmatched fields were ignored, but the number of prepared statement
  place holders were not correct.
- Added unit test code for generateUpdate.
- Added unit test code with "Ignore Unmatched Columns".

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2165.
2017-09-21 16:11:07 +02:00
btwood 90ed08ec33 NIFI-4326 Fix NullPointerException and strict addressing
This uses parseHeader() instead of getFrom() and getRecipients() in order to avoid strict addressing.
It also checks for null to solve a null pointer exception.
By contract, this processor should grab information "if available". Which means it should not fail if the info is unavailable.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2111.
2017-09-21 15:21:22 +02:00
Sébastien Bouchex Bellomié 2ee21b9255 NIFI-4394 Fixed CPU issue - gRPC
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2163.
2017-09-19 17:37:21 +02:00
Bryan Bende 66479464be NIFI-4387 Adding record tag to Parquet processors
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2156.
2017-09-16 14:41:07 +02:00
Matt Gilman eac47e90cb NIFI-4280:
- Adding support for the user to configure variables in the UI.
- Updating the endpoints for changing variables as necessary.
This closes #2135.
2017-09-14 11:12:54 -04:00
m-hogue 91383264d8 NIFI-3510: Added logging for when controller services are enabled/disabled
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2151.
2017-09-14 16:46:18 +02:00
Mark Payne e01d59a462 NIFI-4379: Event-Driven threads are constantly active, polling a queue to see if there is any work to do. Instead of getting number of active threads from the ScheduledExecutor for these threads, updated code to use an AtomicInteger that is incremented when there's work to do and decremented once complete
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2153.
2017-09-14 15:39:32 +02:00
Koji Kawamura 1e70e24267 NIFI-4352: Add CLOB and NCLOB support to PutSQL and ExecuteSQL
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2145.
2017-09-14 11:02:48 +02:00
Mark Payne e52e9acc59 NIFI-4377: Added a fieldName() function to RecordPath and addressed an issue that caused //* to not work
NIFI-4377: Updated RecordPath Guide to include the new fieldName() function and improved unit test for the function

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2147.
2017-09-14 09:40:36 +02:00
yuri1969 1f1269c817 NIFI-3369 - Disable "Add" button on new...
...connection dialog if no relationships selected

Disabled confirmation button for both Create and Configure Connection
dialogs when no relationship was selected.

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #2152
2017-09-13 13:04:24 -04:00
Mark Payne 0e50279467 NIFI-4322: If there is a failure checkpointing FlowFile Repository, ensure that we continue trying
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2146.
2017-09-13 12:14:13 +02:00
Mark Payne b452f8c251 NIFI-4378: Allow suppression of null values in JSON
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2148.
2017-09-12 21:41:59 +02:00
Koji Kawamura 7423b41e3c NIFI-4376: Remove expired S2S request from internal queue. This closes #2144.
Before this fix, expired requests stay in the queue and blocks newly
received requests to be offerred.
2017-09-12 15:31:10 -04:00
Mark Payne d698f227df NIFI-4290: Ensure that the InFlightMessageTracker is instantiated when getTracker() is called
This closes #2083.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-09-12 10:30:44 +09:00
yuri1969 cb7bc93e28
NIFI-4347 - Extend documentation with double-click shortcuts info. This closes #2126 2017-09-11 14:21:37 -04:00
Pierre Villard 5957f6ee95
NIFI-4341 - add provenance repository storage usage in UI
- addressing review comments
- This closes #2119
2017-09-11 13:16:53 -04:00
Gino Lisignoli 897b8ab601 NIFI-4374: Added buffer size to FTP connections
This closes #2141.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-09-11 23:43:56 +09:00
combine 1aeb517c27
NIFI-4370 View Details of NiFi Data Provenance throws NPE when cluster mode. This closes #2137 2017-09-11 09:46:32 -04:00
Joe Percivall 9367c28064 NIFI-2162 Updating OkHttp to 3.8.1 and OkHttp-Digest to 1.13 and refactoring InvokeHttp to adjust for changes
This closes #2004

Signed-off-by: Tony Kurc <tkurc@apache.org>
2017-09-10 22:50:37 -04:00
Wesley-Lawrence e203358bf3 NIFI-4258 Corrected a bug with the 'Informix Unload Escape Disabled' CSV format. This closes #2090. 2017-09-08 16:17:38 -04:00
Pierre Villard c9789adf21 NIFI-4351 - Fix for SiteToSiteProvenanceReportingTask when removing event type or component ID filters
This closes #2127.
2017-09-08 14:14:42 -04:00
Mark Payne 9ebf2cfaf1 NIFI-4306: Allow root field to be updated to a child field by adding property with name / and value /child, for instance
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2103.
2017-09-08 18:50:49 +02:00
Pierre Villard 6559604456 NIFI-4081 - Added raw message option in GrokReader
This closes #1921.
2017-09-08 12:44:37 -04:00
Koji Kawamura 1f67cbf628 NIFI-4004: Use RecordReaderFactory without FlowFile.
- Removed FlowFile from RecordReaderFactory, RecordSetWriterFactory and SchemaAccessStrategy.
- Renamed variable 'allowableValue' to 'strategy' to represent its meaning better.
- Removed creation of temporal FlowFile to resolve Record Schema from ConsumerLease.

- Removed unnecessary 'InputStream content' argument from
  RecordSetWriterFactory.getSchema method.

This closes #1877.
2017-09-08 12:37:40 -04:00
yuri1969 20d23e836e
NIFI-3941 - Clarify tab name for controller-level controller services dialog
* Changed the tab title since sharing the name makes things
less clear for newcomers.
* Suggested info sentence is omitted.

This closes #2124.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-09-07 18:29:13 -07:00
m-hogue 03e51ee8ac
NIFI-4335: Changed SSLContextService implementations to RestrictedSSLContextService for all Listen* processors
This closes #2131.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-09-07 17:34:00 -07:00
Pierre Villard c10ff574c4 NIFI-4257 - add custom WHERE clause in database fetch processors
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2050
2017-09-07 14:41:26 -04:00
patricker ae30c7f350 NIFI-3484: GenerateTableFetch Should Allow for Right Boundary
fix checkstyle issue, and added unit test showing data duplication issue, removed property

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2091
2017-09-07 11:05:38 -04:00
Mike Thomsen 527ce0b4ef NIFI-4269 Added the ability to serialize Mongo documents to a clean JSON view instead of just extended JSON.
NIFI-4269 incorporated changes from the code review.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2063
2017-09-06 15:56:12 -04:00
Pierre Villard 9ac88d210a NIFI-4342 - Add EL support to PutHiveStreaming
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2120
2017-09-06 13:33:18 -04:00
Pierre Villard 458c987fe3 NIFI-4355 - query execution time as attribute of ExecuteSQL
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2129
2017-09-06 13:29:19 -04:00
Andy LoPresto 0536c3edf1
NIFI-4353 Implemented new JAXB logic.
Added unit test and template test resource.
Added RAT exclusion.
This closes #2128
2017-09-06 13:13:24 -04:00
Pierre Villard c3754c3921 NIFI-4320 - Escape quotes for JSON output in QueryCassandra
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2114
2017-09-06 13:05:18 -04:00
Mark Payne bfd6c0aef7 NIFI-4116: Allow fields of Record returned from Lookup Service to be placed into record in the input, instead of requiring that the 'wrapper record' returned from Lookup be included
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #2110.
2017-09-05 21:29:16 -07:00
Matthew Burgess 20a6374bf7 NIFI-4334: Added Apache Commons Net to scripting NAR
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2117.
2017-08-31 09:22:22 +02:00
m-hogue d28e61c5dd
NIFI-2528 Added RestrictedSSLContextService interface with implementation. Changed ListenHTTP to allow only new StandardRestrictedSSLContextService.
This closes #1986.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-08-30 20:17:01 -07:00
Pierre Villard e62417ea6b NIFI-1923 - AttributesToJson regex property
added EL support

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2099
2017-08-30 14:01:27 -04:00
Pierre Villard a53a37f9ca NIFI-4319 - Fixed ArrayIndexOutOfBoundsException in QueryCassandra
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2112
2017-08-30 13:52:05 -04:00
Matt Gilman 3de0b8edff NIFI-4224:
- Ensuring the parent permissions are used when building the variable registry for a process group ancestry. This closes #2116.
2017-08-30 11:54:31 -04:00
Scott Aslan e2b8be53cb
[NIFI-4327] Parameterize node and npm in poms. This closes #2115 2017-08-28 15:10:24 -04:00
Koji Kawamura e68ff153e8
NIFI-3332: ListXXX to not miss files with the latest processed timestamp
Before this fix, it's possible that ListXXX processors can miss files those have the same timestamp as the one which was the latest processed timestamp at the previous cycle. Since it only used timestamps, it was not possible to determine whether a file is already processed or not.

However, storing every single processed identifier as we used to will not perform well.
Instead, this commit makes ListXXX to store only identifiers those have the latest timestamp at a cycle to minimize the amount of state data to store.

NIFI-3332: ListXXX to not miss files with the latest processed timestamp

- Fixed TestAbstractListProcessor to use appropriate time precision.
  Without this fix, arbitrary test can fail if generated timestamp does
  not have the desired time unit value, e.g. generated '10:51:00' where
  second precision is tested.
- Fixed TestFTP.basicFileList to use millisecond time precision explicitly
  because FakeFtpServer's time precision is in minutes.
- Changed junit dependency scope to 'provided' as it is needed by
  ListProcessorTestWatcher which is shared among different modules.

This closes #1975.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-08-28 11:31:04 -04:00
Koji Kawamura 28ee70222b
NIFI-4069: Make ListXXX work with timestamp precision in seconds or minutes
- Refactored variable names to better represents what those are meant for.
- Added deterministic logic which detects target filesystem timestamp precision and adjust lag time based on it.
- Changed from using System.nanoTime() to System.currentTimeMillis in test because Java File API reports timestamp in milliseconds at the best granularity. Also, System.nanoTime should not be used in mix with epoch milliseconds because it uses arbitrary origin and measured differently.
- Changed TestListFile to use more longer interval between file timestamps those are used by testFilterAge to provide more consistent test result because sleep time can be longer with filesystems whose timestamp in seconds precision.
- Added logging at TestListFile.
- Added TestWatcher to dump state in case assertion fails for further investigation.
- Added Timestamp Precision property so that user can set if auto-detect is not enough
- Adjust timestamps for ages test

This closes #1915.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-08-28 11:31:03 -04:00
Matt Gilman a4e729c7a7 NIFI-4324:
- Ensuring that sub context menus are removed when hiding to ensure they are correctly (re)created during mouseenter events.

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #2109
2017-08-25 16:20:32 -04:00
cam e3da44fb62 [NiFi-3973] Add PutKudu processor to ingest data to Kudu 2017-08-25 10:53:31 -04:00
Yolanda M. Davis 0eda71a9a6 NIFI-4310 - added changes to support detection of reporting tasks and controller services during isEmpty flow check. Added testing scenarios. This closes #2107. 2017-08-25 10:15:55 -04:00