Commit Graph

2487 Commits

Author SHA1 Message Date
Bryan Bende 6eab91923e NIFI-4418 Adding ListenUDPRecord processor. This closes #2173. 2017-09-25 13:19:23 -04:00
Koji Kawamura c6f4421c8e NIFI-2829: Fixed PutSQL time unit test.
The unit test for DATE type used GMT timezone, that causes an assertion error in timezones such as EST (-5).
We need to use local timezone instead of GMT, as Derby and PutSQL uses local timezone.

The unit test failed before as follows:
- Unit test code, passed: '2002-02-02 GMT'
- PutSQL code convertedi it to local: '2002-02-01 EST', and stored as '2002-02-01' in Derby database without timezone info
- Unit test code SELECT the inserted value, passed a GMT calender, then got epoch timestamp, which was '2002-01-31'

Support negative long value for timezones ahead of UTC.

- For timezones such as '+0800', it's possible that a local time e.g. '02:03:04' can be a negative epoch value. This commit changes LONG_PATTERN so that it can accept nevative values.
- Changed time values in unit tests to verify negative epoch values, and avoid using the same digits among different time unit for better readability.

This closes #2082
2017-09-23 17:41:42 -04:00
Mark Payne 3fb704c58f NIFI-4201: This closes #2024. Implementation of processors for interacting with Kafka 0.11
Signed-off-by: joewitt <joewitt@apache.org>
2017-09-22 22:08:19 -04:00
Deon Huang a29348f2a4 NIFI-4395 - GenerateTableFetch can't fetch column type by state after instance reboot
NIFI-4395: Updated unit test for GenerateTableFetch
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2166
2017-09-22 14:34:00 -04:00
Andy LoPresto 9e2c7be7d3
NIFI-4353
- Added XmlUtils class.
- Added unit test.
- Added XXE test resource.
- Refactored JAXB unmarshalling globally to prevent XXE attacks.
- Refactored duplicated/legacy code.
- Cleaned up commented code.
- Switched from FileInputStream back to StreamSource in AuthorizerFactoryBean.
- This closes #2134
2017-09-22 14:31:38 -04:00
Mike Thomsen 9a8e6b2eb1 NIFI-4345 Added a MongoDB controller service and a lookup service.
NIFI-4345: Changed Lookup Key to Lookup Value Field
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2123
2017-09-22 13:33:07 -04:00
Mark Payne 05b5dd1488 NIFI-4409: Ensure that when record schema is inherited, the schema from teh ResultSet is used instead of the schema from the RecordReader because the schema from the RecordReader mmay not match the actual data
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2169
2017-09-22 12:41:17 -04:00
Pierre Villard 50d018566d
NIFI-4403 - add group name to bulletins model. This closes #2167 2017-09-22 11:53:58 -04:00
cam 0c0c33411d NIFI-4384 - Enhance PutKudu processor to support batch insert
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2160.
2017-09-22 10:56:15 +02:00
Bryan Bende 329dbe3a64 NIFI-4405 Adding charset property to GenerateFlowFile
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2168.
2017-09-21 17:10:47 +02:00
Bryan Bende a813ae113e NIFI-4391 Ensuring channel is closed when unable to connect in SocketChannelSender
NIFI-4391 Adding debug logging of client port upon connection

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2159.
2017-09-21 16:30:10 +02:00
Koji Kawamura 7b07eb0577 NIFI-4228: Fix PutDatabaseRecord to ignore unmatched fields
- Unmatched fields were ignored, but the number of prepared statement
  place holders were not correct.
- Added unit test code for generateUpdate.
- Added unit test code with "Ignore Unmatched Columns".

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2165.
2017-09-21 16:11:07 +02:00
btwood 90ed08ec33 NIFI-4326 Fix NullPointerException and strict addressing
This uses parseHeader() instead of getFrom() and getRecipients() in order to avoid strict addressing.
It also checks for null to solve a null pointer exception.
By contract, this processor should grab information "if available". Which means it should not fail if the info is unavailable.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2111.
2017-09-21 15:21:22 +02:00
Sébastien Bouchex Bellomié 2ee21b9255 NIFI-4394 Fixed CPU issue - gRPC
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2163.
2017-09-19 17:37:21 +02:00
Bryan Bende 66479464be NIFI-4387 Adding record tag to Parquet processors
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2156.
2017-09-16 14:41:07 +02:00
Matt Gilman eac47e90cb NIFI-4280:
- Adding support for the user to configure variables in the UI.
- Updating the endpoints for changing variables as necessary.
This closes #2135.
2017-09-14 11:12:54 -04:00
m-hogue 91383264d8 NIFI-3510: Added logging for when controller services are enabled/disabled
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2151.
2017-09-14 16:46:18 +02:00
Mark Payne e01d59a462 NIFI-4379: Event-Driven threads are constantly active, polling a queue to see if there is any work to do. Instead of getting number of active threads from the ScheduledExecutor for these threads, updated code to use an AtomicInteger that is incremented when there's work to do and decremented once complete
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2153.
2017-09-14 15:39:32 +02:00
Koji Kawamura 1e70e24267 NIFI-4352: Add CLOB and NCLOB support to PutSQL and ExecuteSQL
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2145.
2017-09-14 11:02:48 +02:00
yuri1969 1f1269c817 NIFI-3369 - Disable "Add" button on new...
...connection dialog if no relationships selected

Disabled confirmation button for both Create and Configure Connection
dialogs when no relationship was selected.

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #2152
2017-09-13 13:04:24 -04:00
Mark Payne 0e50279467 NIFI-4322: If there is a failure checkpointing FlowFile Repository, ensure that we continue trying
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2146.
2017-09-13 12:14:13 +02:00
Mark Payne b452f8c251 NIFI-4378: Allow suppression of null values in JSON
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2148.
2017-09-12 21:41:59 +02:00
Koji Kawamura 7423b41e3c NIFI-4376: Remove expired S2S request from internal queue. This closes #2144.
Before this fix, expired requests stay in the queue and blocks newly
received requests to be offerred.
2017-09-12 15:31:10 -04:00
Mark Payne d698f227df NIFI-4290: Ensure that the InFlightMessageTracker is instantiated when getTracker() is called
This closes #2083.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-09-12 10:30:44 +09:00
Pierre Villard 5957f6ee95
NIFI-4341 - add provenance repository storage usage in UI
- addressing review comments
- This closes #2119
2017-09-11 13:16:53 -04:00
Gino Lisignoli 897b8ab601 NIFI-4374: Added buffer size to FTP connections
This closes #2141.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-09-11 23:43:56 +09:00
combine 1aeb517c27
NIFI-4370 View Details of NiFi Data Provenance throws NPE when cluster mode. This closes #2137 2017-09-11 09:46:32 -04:00
Joe Percivall 9367c28064 NIFI-2162 Updating OkHttp to 3.8.1 and OkHttp-Digest to 1.13 and refactoring InvokeHttp to adjust for changes
This closes #2004

Signed-off-by: Tony Kurc <tkurc@apache.org>
2017-09-10 22:50:37 -04:00
Wesley-Lawrence e203358bf3 NIFI-4258 Corrected a bug with the 'Informix Unload Escape Disabled' CSV format. This closes #2090. 2017-09-08 16:17:38 -04:00
Pierre Villard c9789adf21 NIFI-4351 - Fix for SiteToSiteProvenanceReportingTask when removing event type or component ID filters
This closes #2127.
2017-09-08 14:14:42 -04:00
Mark Payne 9ebf2cfaf1 NIFI-4306: Allow root field to be updated to a child field by adding property with name / and value /child, for instance
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2103.
2017-09-08 18:50:49 +02:00
Pierre Villard 6559604456 NIFI-4081 - Added raw message option in GrokReader
This closes #1921.
2017-09-08 12:44:37 -04:00
Koji Kawamura 1f67cbf628 NIFI-4004: Use RecordReaderFactory without FlowFile.
- Removed FlowFile from RecordReaderFactory, RecordSetWriterFactory and SchemaAccessStrategy.
- Renamed variable 'allowableValue' to 'strategy' to represent its meaning better.
- Removed creation of temporal FlowFile to resolve Record Schema from ConsumerLease.

- Removed unnecessary 'InputStream content' argument from
  RecordSetWriterFactory.getSchema method.

This closes #1877.
2017-09-08 12:37:40 -04:00
yuri1969 20d23e836e
NIFI-3941 - Clarify tab name for controller-level controller services dialog
* Changed the tab title since sharing the name makes things
less clear for newcomers.
* Suggested info sentence is omitted.

This closes #2124.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-09-07 18:29:13 -07:00
m-hogue 03e51ee8ac
NIFI-4335: Changed SSLContextService implementations to RestrictedSSLContextService for all Listen* processors
This closes #2131.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-09-07 17:34:00 -07:00
Pierre Villard c10ff574c4 NIFI-4257 - add custom WHERE clause in database fetch processors
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2050
2017-09-07 14:41:26 -04:00
patricker ae30c7f350 NIFI-3484: GenerateTableFetch Should Allow for Right Boundary
fix checkstyle issue, and added unit test showing data duplication issue, removed property

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2091
2017-09-07 11:05:38 -04:00
Mike Thomsen 527ce0b4ef NIFI-4269 Added the ability to serialize Mongo documents to a clean JSON view instead of just extended JSON.
NIFI-4269 incorporated changes from the code review.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2063
2017-09-06 15:56:12 -04:00
Pierre Villard 9ac88d210a NIFI-4342 - Add EL support to PutHiveStreaming
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2120
2017-09-06 13:33:18 -04:00
Pierre Villard 458c987fe3 NIFI-4355 - query execution time as attribute of ExecuteSQL
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2129
2017-09-06 13:29:19 -04:00
Andy LoPresto 0536c3edf1
NIFI-4353 Implemented new JAXB logic.
Added unit test and template test resource.
Added RAT exclusion.
This closes #2128
2017-09-06 13:13:24 -04:00
Pierre Villard c3754c3921 NIFI-4320 - Escape quotes for JSON output in QueryCassandra
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2114
2017-09-06 13:05:18 -04:00
Mark Payne bfd6c0aef7 NIFI-4116: Allow fields of Record returned from Lookup Service to be placed into record in the input, instead of requiring that the 'wrapper record' returned from Lookup be included
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #2110.
2017-09-05 21:29:16 -07:00
Matthew Burgess 20a6374bf7 NIFI-4334: Added Apache Commons Net to scripting NAR
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2117.
2017-08-31 09:22:22 +02:00
m-hogue d28e61c5dd
NIFI-2528 Added RestrictedSSLContextService interface with implementation. Changed ListenHTTP to allow only new StandardRestrictedSSLContextService.
This closes #1986.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-08-30 20:17:01 -07:00
Pierre Villard e62417ea6b NIFI-1923 - AttributesToJson regex property
added EL support

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2099
2017-08-30 14:01:27 -04:00
Pierre Villard a53a37f9ca NIFI-4319 - Fixed ArrayIndexOutOfBoundsException in QueryCassandra
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2112
2017-08-30 13:52:05 -04:00
Matt Gilman 3de0b8edff NIFI-4224:
- Ensuring the parent permissions are used when building the variable registry for a process group ancestry. This closes #2116.
2017-08-30 11:54:31 -04:00
Scott Aslan e2b8be53cb
[NIFI-4327] Parameterize node and npm in poms. This closes #2115 2017-08-28 15:10:24 -04:00
Koji Kawamura e68ff153e8
NIFI-3332: ListXXX to not miss files with the latest processed timestamp
Before this fix, it's possible that ListXXX processors can miss files those have the same timestamp as the one which was the latest processed timestamp at the previous cycle. Since it only used timestamps, it was not possible to determine whether a file is already processed or not.

However, storing every single processed identifier as we used to will not perform well.
Instead, this commit makes ListXXX to store only identifiers those have the latest timestamp at a cycle to minimize the amount of state data to store.

NIFI-3332: ListXXX to not miss files with the latest processed timestamp

- Fixed TestAbstractListProcessor to use appropriate time precision.
  Without this fix, arbitrary test can fail if generated timestamp does
  not have the desired time unit value, e.g. generated '10:51:00' where
  second precision is tested.
- Fixed TestFTP.basicFileList to use millisecond time precision explicitly
  because FakeFtpServer's time precision is in minutes.
- Changed junit dependency scope to 'provided' as it is needed by
  ListProcessorTestWatcher which is shared among different modules.

This closes #1975.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-08-28 11:31:04 -04:00
Koji Kawamura 28ee70222b
NIFI-4069: Make ListXXX work with timestamp precision in seconds or minutes
- Refactored variable names to better represents what those are meant for.
- Added deterministic logic which detects target filesystem timestamp precision and adjust lag time based on it.
- Changed from using System.nanoTime() to System.currentTimeMillis in test because Java File API reports timestamp in milliseconds at the best granularity. Also, System.nanoTime should not be used in mix with epoch milliseconds because it uses arbitrary origin and measured differently.
- Changed TestListFile to use more longer interval between file timestamps those are used by testFilterAge to provide more consistent test result because sleep time can be longer with filesystems whose timestamp in seconds precision.
- Added logging at TestListFile.
- Added TestWatcher to dump state in case assertion fails for further investigation.
- Added Timestamp Precision property so that user can set if auto-detect is not enough
- Adjust timestamps for ages test

This closes #1915.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-08-28 11:31:03 -04:00
Matt Gilman a4e729c7a7 NIFI-4324:
- Ensuring that sub context menus are removed when hiding to ensure they are correctly (re)created during mouseenter events.

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #2109
2017-08-25 16:20:32 -04:00
cam e3da44fb62 [NiFi-3973] Add PutKudu processor to ingest data to Kudu 2017-08-25 10:53:31 -04:00
Yolanda M. Davis 0eda71a9a6 NIFI-4310 - added changes to support detection of reporting tasks and controller services during isEmpty flow check. Added testing scenarios. This closes #2107. 2017-08-25 10:15:55 -04:00
yuri1969 6c426f7a12 NIFI-2715 - Match style to 'detail-value'
Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #2100
2017-08-24 18:39:12 -04:00
yuri1969 a9ee2aa5e4 NIFI-2715 - Keep 'Content Claim' header visible 2017-08-24 18:38:29 -04:00
yuri1969 a7ea178cdc NIFI-2715 - When performing a Queue Listing, if...
...FlowFile has no content, UI does not indicate this fact

For FlowFiles with no content:
* Added a simple title replacing the Content Claim's one.
* Defaulted File Size to 0 bytes using common formatting.
2017-08-24 18:38:29 -04:00
Bryan Bende cf57639396 NIFI-4311 Allowing umask to get set properly before initializing the FileSystem
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2106.
2017-08-22 22:40:26 +02:00
yuri1969 05700a2008 NIFI-3781 - Change tooltip for disabled state
Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #2102
2017-08-21 12:56:19 -04:00
yuri1969 a3401c3b95 NIFI-3781 - Controller Service Config Dialog...
..'View Details' button feels confusing and inconsistent with Processors

* Merged 'View Details' and 'Edit' buttons to a single 'Configure'/'View Details' one.
* Fixed confusing 'View Details' modal's title.
2017-08-21 12:56:12 -04:00
mans2singh 8cb501443b NIFI-4250 - Elasticsearch 5 delete processor
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #2045
2017-08-21 08:49:57 -04:00
Pierre Villard 41984bed13 NIFI-1835 - added text/xml to content viewer
Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #2098
2017-08-18 12:48:49 -04:00
Bryan Bende 60d4672195 NIFI-4304 Extracting HWX Schema Registry client version to a property and bumping to latest 0.3.0 release
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2096
2017-08-17 16:07:03 -04:00
Matt Burgess 6b5015e39b
NIFI-4218: Dynamic properties as query parameters in ESHttp processors
This closes #2049.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-08-17 14:29:05 -04:00
Mark Payne 69a08e78c2
NIFI-4295:
- When determining which controller services to return for a component, ensure that we don't show services that belong to 'child groups'
- Fixed a logic bug that determined which process group to use for obtaining controller services
- This closes #2087
2017-08-17 12:47:46 -04:00
Mark Payne 5cd8e93beb
NIFI-4224:
- Initial implementation of Process Group level Variable Registry
- Updated to incorporate PR Feedback
- Changed log message because slf4j-simple apparently has a memory leak; passing a String instead of passing in the Controller Service object as an argument addresses this.
- This closes #2051
2017-08-17 10:42:29 -04:00
Pierre Villard c1b99d584d NIFI-4028 - fix cache update when Wait releases flow files
NIFI-4028: Refactored Wait processor.

- Consolidated implementation for the cases of releasableFlowCount is 1 or more, in order to reduce complexity and behavior differences
- Added 'consumed' counter when total counter is used to release incoming FlowFiles
- Fixed method name typo, releaseCandidates

This closes #2055.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-08-17 08:30:07 +09:00
Pierre Villard 9f23ec360a NIFI-3931 - Added EL to properties in SFTP transfer
The properties are already evaluated against EL. Also added an
integration test that is failing without the change.

This closes #1968.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-08-17 08:20:30 +09:00
Arun Manivannan 6df112e4b1 NIFI-4062 Provide an option to disable DTD validation for EvaluateXPath and EvaluateXQuery
This closes #2093.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-08-17 08:07:37 +09:00
yuri1969 02c05bc203 NIFI-3780 - Should be able to specify Service...
...name when I create a new Controller Service within a Processor/Service configuration dialog

Changed CS name default's behavior according to PR feedback.

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #2081
2017-08-16 11:56:23 -04:00
yuri1969 79300de7a9 NIFI-3780 - Should be able to specify Service...
...name when I create a new Controller Service within a Processor/Service configuration dialog

I'm not sure how to integrate/style the text box. Suggestions are welcome.
2017-08-16 10:31:10 -04:00
Mark Payne abca9d1464
NIFI-4208: Fixed bug in SchemaRepositoryRecordSerde that would return null from deserializeEdit if there was no data; the interface documents that null cannot be returned from this method, as it is called only when data is expected to exist. As a result, if there is no data, we should throw EOFException instead, and the write-ahead log will handle that appropriately.
This closes #2086.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-08-15 12:10:43 -04:00
Pierre Villard a1706d12f5 NIFI-3281 - fix for (S)FTP processors when using EL against FFs
NIFI-3281 - Review - handle completePendingCommand return and added a unit test for ListFTP

NIFI-3281 - Review - Added flow file for EL evaluation in other methods and added unit test for NIFI-3590

This closes #1974.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-08-15 11:24:48 +09:00
Matt Burgess 50f22162b0
NIFI-4286: Fix NPE in GenerateTableFetch when Partition Size = 0
This closes #2078.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-08-14 15:28:07 -04:00
Pierre Villard 8aa4450084 NIFI-779 - fix for ReportingContext.getProperty(PropertyDescriptor)
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #2033
2017-08-14 13:41:10 -04:00
Mark Payne 451f9cf124 NIFI-4142: This closes #2015. Refactored Record Reader/Writer to allow for reading/writing "raw records". Implemented ValidateRecord. Updated Record Reader to take two parameters for nextRecord: (boolean coerceTypes) and (boolean dropUnknownFields)
Signed-off-by: joewitt <joewitt@apache.org>
2017-08-11 22:01:46 -07:00
Mark Payne 84935d4f78 NIFI-4023, NIFI-4077 This closes #2075. Addressed issue where repository was aging off the wrong index. When it should age off Index 1, it was removing Index 2. As a result, the earliest index is never aged off, and the newest index could potentially be aged off before it is ready to be. Also addressed issue where a query that attempts to read an event that has aged off throws FileNotFoundException (NIFI-4077) instead of skipping over the event. The JavaDocs indicate that the EventIterator should skip records that it cannot find, but SelectiveRecordReaderEventIterator throw FileNotFoundException instead
Signed-off-by: joewitt <joewitt@apache.org>
2017-08-11 22:01:46 -07:00
Andy LoPresto ae940d8624 NIFI-4237 Added working test for StringEncryptor decryption of sensitive flow values in FlowFromDOMFactory.
NIFI-4237 Cleaned up unused alternate approaches.

NIFI-4237 Added failing unit test for better error message.

NIFI-4237 Added logic to capture unhelpful encryption exception and provide context in message. All tests pass.

This closes #2077
2017-08-11 23:57:15 -04:00
Bryan Bende 28d5a70ec0
NIFI-4288 Ignoring tests that intermittently fail in TestPutUDP and TestPutTCP
This closes #2076.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-08-11 17:27:55 -07:00
Mike Moser d6ebc53076 NIFI-3736 modify default nifi.content.claim.max.appendable.size in nifi.properties to 1 MB 2017-08-11 15:53:36 -04:00
Bryan Bende f7da7e67f4 NIFI-4275 Adding support for specifying the timestamp on PutHBase processors
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2070.
2017-08-11 18:04:13 +02:00
Wayne Steel a6e8f0afe3 NIFI-4177 This closes #2002. MergeContent - Tar - Save modification timestamp like Tar does
Signed-off-by: joewitt <joewitt@apache.org>
2017-08-10 19:35:46 -07:00
Pierre Villard 69f263c4d1
NIFI-2167 - include disabled state for processors when copy/paste/templates
This closes #2057.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-08-10 18:41:24 -07:00
Mark Payne 96f5407ed9 NIFI-3329 This closes #2053. Removed check for latest version of flowfile when migrating flowfile from 1 session to another because we now longer are requiring latest version of flowfile be passed to session anywhere else. It was intended to be removed from StandardProcessSession.migrate when it was removed from StandardProcessSession.validateRecordState but it was overlooked. 2017-08-10 18:13:28 -07:00
Matt Gilman 20d6596df0 NIFI-3232:
- Adding support for cascading sub context menus.

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #2072
2017-08-10 17:01:25 -04:00
Pierre Villard 18c82eb6af
NIFI-4277 Fixed exception logging in StandardLogRepository
This closes #2068.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-08-10 11:54:32 -07:00
Matt Gilman 528b82634f
NIFI-4210:
- Introducing support for OpenId Connect.
- Updating REST API and UI to support the authorization code flow.
- Adding/fixing documentation.
- Implementing time constant equality checks where appropriate.
- Corrected error handling during startup and throughout the OIDC login sequence.
- Redacting the token values from the user log.
- Defaulting to RS256 when not preferred algorithm is specified.
- Marking the OIDC endpoints as non-guaranteed in to allow for minor adjustments if/when additional SSO techniques are introduced.

This closes #2047.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-08-10 11:15:35 -07:00
yuri1969 505e93065e NIFI-1027 - Treat label and node of a provenance graph event as one unit
The JIRA issue asks for treating both node and its label as one unit.
Described cursor issues seems to be fixed already.

However, there is an annoying dead space between a node and label
preventing displaying context menu, etc. Due to SVG group's nature
there has been added an opaque joint to remove the dead space.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2059.
2017-08-10 10:54:47 +02:00
Andy LoPresto 5150dff70b NIFI-4274 Removed EL evaluation logic from custom file validator.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2071.
2017-08-10 10:05:16 +02:00
Pierre Villard 6ff8321cf7
NIFI-4278 - add error message to invalid FFs in ValidateXml
This closes #2069.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-08-09 21:20:08 -07:00
Matt Burgess 9c4fdd4ef3 NIFI-4192: Add more Avro/metadata merge options to MergeContent
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2067.
2017-08-09 12:55:12 +02:00
Mike Thomsen 67819e5019
NIFI-4124 Added org.apache.nifi.mongo.PutMongoRecord.
Added changes based on code review. Changed:
* Put record reader instantiation inside of try-with.
* Put a batch size for the insert List.
* Ensured that session.transfer() to the success relationship will always happen.
Removed an unused import to fix the style check.

This closes #1945.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-08-07 22:10:27 -07:00
Bryan Bende 0029f025f8 NIFI-4152 Initial commit of ListenTCPRecord 2017-08-07 22:44:11 +02:00
yuri1969 e44f436bd8 NIFI-4271 - Revert double-click behavior back...
...to delete a bend point

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #2064
2017-08-07 15:58:48 -04:00
zhengsg ee3b58368d NIFI-4267 Change the class name for log in FileRemovalAction
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2061.
2017-08-07 21:42:46 +02:00
Wil Selwood f7f64a572c NIFI-4268: Add fetch tag to FetchFile processor.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2062.
2017-08-07 21:38:55 +02:00
Andy LoPresto 675d989003
NIFI-4139
- Moved key provider interface and implementations from nifi-data-provenance-utils module to nifi-security-utils module.
- Refactored duplicate byte[] concatenation methods from utility classes and removed deprecation warnings from CipherUtility.
- Created KeyProviderFactory to encapsulate key provider instantiation logic.
- Added logic to handle legacy package configuration values for key providers.
- Added unit tests.
- Added resource files for un/limited strength cryptography scenarios.
- Added ASL to test resources.
- Moved legacy FQCN handling logic to CryptUtils.
- Added unit tests to ensure application startup logic handles legacy FQCNs.
- Moved master key extraction/provision out of FBKP.
- Removed nifi-security-utils dependency on nifi-properties-loader module.
- Added unit tests.
2017-08-07 15:21:58 -04:00
Pierre Villard 8b54c2604c NIFI-4082 - Added EL on GetMongo properties
NIFI-4082 - Added EL on DB, URI and Collection

NIFI-4082 - Added UT for EL evaluation (URI, DB, Collection) and fixed ex. message for document validator.

This closes #1969
2017-08-07 13:40:27 -04:00
yuri1969 329b1caf82 NIFI-4266 - Allow double-click to display Event
...details

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #2058
2017-08-07 08:44:13 -04:00
Bryan Bende f8f1cc8d0d
NIFI-4024 Ensuring InputStream gets closed and cleaning up complex field handling
This closes #1961.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-08-04 14:54:05 -04:00
Mike Thomsen 496a32e12c
NIFI-4024 Added org.apache.nifi.hbase.PutHBaseRecord
Signed-off-by: Bryan Bende <bbende@apache.org>
2017-08-04 14:54:01 -04:00
Yolanda M. Davis afd4f9e034
NIFI-4022 - Initial update for SASL support for cluster management in Zookeeper
NIFI-4022 - adding sasl documentation update and update to test

This closes #2046.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-08-04 14:19:35 -04:00
Mark Payne 969bbe654c NIFI-4051: Ensure that we use the appropriate Lexical syntax for SQL query validation as we do when running
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2054.
2017-08-04 16:31:56 +02:00
Wesley-Lawrence 40cde0466a NIFI-4215 NiFi can now parse an Avro schema of a record that references an already defined record, including itself.
This closes #2052.
2017-08-03 15:13:07 -04:00
Matt Burgess 2d33582147 NIFI-4253: Add Oracle 12+ database adapter
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2048.
2017-08-03 09:59:29 +02:00
James Wing 2502b79bae NIFI-4215 Revert Complex Avro Schema Changes
This reverts commit cf49a58ee7.
2017-08-01 21:03:04 -07:00
yuri1969 bcf60aa556 Add double-click shortcut to RPG
Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #2009
2017-07-31 10:58:49 -04:00
yuri1969 ef9cb5be23 NIFI-1580 - Allow double-click to display config
* Added double-click shortcut opening config/details dialog to processors,
connections, ports and labels.
* Created a base for further default action selection, disabling, etc.
* Omitted default action configuration UI - that might be a separate JIRA ticket.
2017-07-31 10:36:37 -04:00
Wesley-Lawrence cf49a58ee7 NIFI-4215 Allow Complex Avro Schema Parsing
NiFi can now parse an Avro schema of a record that references an already defined record, including itself.

Signed-off-by: James Wing <jvwing@gmail.com>

This closes #2034.
2017-07-30 16:31:39 -07:00
mans2singh 0bb1411532 NIFI-4212 - RethinkDB Delete Processor
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #2030.
2017-07-29 14:49:22 -07:00
Leah Anderson 70878fe6d6 NIFI-4052 Add DynamicProperty annotation for LookupRecord
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2032.
2017-07-28 17:12:06 +02:00
Mark Payne 1d6b486b63 NIFI-4232: Ensure that we handle conversions to Avro Arrays properly. Also, if unable to convert a value to the expected object, include in the log message the (fully qualified) name of the field that is problematic
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2040.
2017-07-27 08:57:25 +02:00
Matt Burgess dc4006f423 NIFI-3335: Add initial.maxvalue support to GenerateTableFetch
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2039.
2017-07-27 08:43:00 +02:00
Pierre Villard 759f81bc1b
NIFI-4221 - Print app startup in human-readable time
This closes #2038.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-07-25 15:25:55 -07:00
Matt Burgess 9a5d4ff6b3 NIFI-4217 - Retain original DDL in CaptureChangeMySQL
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2036.
2017-07-25 17:54:11 +02:00
Matt Gilman 743c6b9c17 NIFI-4032: - Introducing the ManagedRangerAuthorizer. - Introducing the AuthorizationAuditor. - Updating authorization requests to utilize Authorizable where ever possible so allow for a singular place to audit resource not found as denied when the parent authorizable is null (no more inheritance). - Updating unit tests as appropriate. - Addressing issues with broken web-api integration tests.
NIFI-4032: - Generating the appropriate fingerprint for the ManagedRangerAuthorizer based on whether the UserGroupProvider is configurable. - Adding unit tests.

Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #2019
2017-07-24 15:51:27 -04:00
Pierre Villard 10692256d6 NIFI-917 Improve Rendering of Component Docs
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #2025.
2017-07-23 11:30:04 -07:00
Leah Anderson 3371e915cc NIFI-4087 This closes #2026. Fix to allow exclusion of filename from tika criteria. 2017-07-21 16:56:56 -04:00
Mark Payne 695e8aa98f
NIFI-106:
- Expose processors' counters in Stats History
- Only include counters in Processors' Status History if user has read access to corresponding Processor
- Addressed review feedback. Found and addressed bug where a counter is not present in all of the aggregate snaphot values for status history, resulting in the UI not rendering the chart properly
- This closes #1872
2017-07-21 14:18:11 -04:00
m-hogue c54b2ad81c NIFI-3736: change to honor nifi.content.claim.max.appendable.size and nifi.content.claim.max.flow.files properties. Added 100 MB cap for NiFiProperties.MAX_APPENDABLE_CLAIM_SIZE
This closes #2010.
2017-07-21 09:32:45 -04:00
Davide d334532b16 NIFI-4184: Enabled EL on PutHDFS Remote Group and Owner
I needed to put some attributes on REMOTE_GROUP and REMOTE_OWNER, in order to achieve it i put expressionLanguageSupported(true) on the PropertyDescriptor of REMOTE_GROUP and REMOTE_OWNER

This closes #2007.

Signed-off-by: Davide <davidde85@hotmail.it>
Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-07-21 16:31:46 +09:00
Andre F de Miranda b0be99036d NIFI-55 - Ensures ListenHTTP logs the source of an expired hold
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1623.
2017-07-20 23:14:32 +02:00
Koji Kawamura b4e0a6e206 NIFI-4205: Avoid duplicated data from TailFile
Before this fix, it is possible that TailFile to produce duplicated data
if an already tailed file has newer timestamp and fewer or the same
amout of data.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2021.
2017-07-20 12:14:05 +02:00
James Wing 1f330b4fe4 NIFI-11 Fixing checkstyle issue - usused import 2017-07-19 22:46:15 -07:00
mans2singh ea1fe1a9c0 NIFI-4188 New RethinkDB Get processor
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #2012.
2017-07-19 20:56:47 -07:00
Andre F de Miranda 451a88df43
NIFI-11
- Capture Exception to prevent failed evaluations from yielding processor
- Further capture evaluation exceptions as per PR feedback
- Adjust jUnit to new exception behavior
- This closes #1644
2017-07-19 11:50:48 -04:00
Bryan Bende f87d2a2f57 NIFI-4157 Improvements to PutTCP
Signed-off-by: Bryan Bende <bbende@apache.org>
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1989.
2017-07-19 12:25:42 +02:00
joewitt 59754500d5 NIFI-2829 ignoring unit test that appears broken. Commented on JIRA about the problem observed 2017-07-18 22:23:10 -04:00
Andre F de Miranda a307ea3a00 NIFI-4098 This closes #2013. Adjust ParseCEF to re-use bean validator in order to improve performance 2017-07-18 22:02:39 -04:00
Matt Burgess 844dbe4edb NIFI-4156: Fixed fragment.count in SplitText to equal emitted flow files
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2014.
2017-07-18 11:31:16 +02:00
deonhuang a6e94de0bb NIFI-2829 - Add Date and Time Format Support for PutSQL
Fix unit test for Date and Time type time zone problem
Enhance Time type to record milliseconds

This closes #1983.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-07-18 14:16:10 +09:00
patricker 03bff7c9fc NIFI-2829 Date and Time Format Support for PutSQL
This closes #1524.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-07-18 14:11:37 +09:00
Mark Payne a9529adab1 NIFI-4093: Fixed bug where we used the wrong Enum to determine how to process an update to the FlowFile Repo
Signed-off-by: Matt Burgess <mattyb149@apache.org>
2017-07-13 10:51:02 -04:00
Koji Kawamura 5c48655e69 NIFI-4112: Fix ConvertExcelToCSV to handle empty cells.
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1973
2017-07-13 10:40:01 -04:00
Koji Kawamura 8acee02393 NIFI-1613
- Truncate text data types only.
- Added conversion from a boolean to number.
2017-07-12 17:02:55 -04:00
Koji Kawamura 3844a821f1 NIFI-1613: ConvertJSONToSQL truncates numeric value wrongly.
- Changed boolean value conversion to use Boolean.valueOf.
- Updated comments in source code to reflect current situation more clearly.
- Updated tests those have been added since the original commits were made.
2017-07-12 17:02:55 -04:00
Toivo Adams 3b2e43b75c NIFI-1613 Initial version, try to improve conversion for different SQL types. New test and refactored existing test to reuse DBCP service.
nifi-1613 Adding numeric and Date/time types conversion and test.
2017-07-12 17:02:55 -04:00
Mark Payne b603cb955d NIFI-4060: Initial implementation of MergeRecord
NIFI-4060: Addressed threading issue with RecordBin being updated after it is completed; fixed issue that caused mime.type attribute not to be written properly if all incoming flowfiles already have a different value for that attribute

NIFI-4060: Bug fixes; improved documentation; added a lot of debug information; updated StandardProcessSession to produce more accurate logs in case of a session being committed/rolled back with open input/output streams
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1958
2017-07-12 16:36:48 -04:00
Matt Gilman eefad29167 NIFI-4127:
- Introducing composite ConfigurableUserGroupProvider and UserGroupProvider.
- Adding appropriate unit tests.
- Updating object model to support per resource (user/group/policy) configuration.
- Updating UI to support per resource (user/group/policy) configuration.
- Adding necessary documentation.
- Updating documentation to clarify integrity checks.
- Providing an example of configuring a composite implementation.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1978.
2017-07-11 18:13:04 +02:00
Joe Skora 325fe53fa1 NIFI-3677 - ExtractMediaMetadata should close TikaInputStream
* Added finally block to close TikaInputStream.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1993.
2017-07-11 18:04:50 +02:00
Jeff Storck 58fc5f24b8
NIFI-1586 Removed check for distributed ZK quorum before starting the DatadirCleanupMananger to enable autopurge during standalone ZK server usage. This closes #2001 2017-07-11 10:04:08 -04:00
mans2singh 0fd51b4d12 NIFI-4145 Added writes attributes annotation
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #1998.
2017-07-10 21:25:26 -07:00
m-hogue 78fbb8f2ee
NIFI-3939: Reviewed and corrected all incorrect nifi-web-api resource response types. This closes #1999 2017-07-10 14:40:13 -04:00
Mark Payne 87e062ff55
NIFI-4167: StandardResourceClaimManager should not synchronize on a ResourceClaim in order to determine the claim count. This closes #1996 2017-07-10 14:33:07 -04:00
Tony Kurc 2dc45a4dd7 NIFI-4154 Fixing line endings in .java,.html files
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #1982.
2017-07-09 12:13:15 -07:00
mans2singh affc88e599 NIFI-4118 RethinkDB bundle with put processor
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #1942.
2017-07-09 11:10:23 -07:00
Mark Payne 9e296830ab
NIFI-4151: Ensure that we properly call invalidateValidationContext() when properties change; ensure that in the controller service provider we don't replace a controller service with a new node if the ID's match, as we won't be able to actually add the new one to the flow. This closes #1995 2017-07-07 16:41:32 -04:00
Yolanda M. Davis 6df97bbc88
NIFI-4135 - added hadoop-client and enhanced Authorizers entity to support classpath for resources entry
NIFI-4135 - classpath under class

This closes #1956.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-07-07 10:08:48 -04:00
kwydler e89512e744 NIFI-4161: Adding expression evaluation to AWS PROXY_HOST and PROXY_HOST_PORT property usage
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1990.
2017-07-07 13:28:33 +02:00
Koji Kawamura e84f9a2416 NIFI-4160: SFTPTransfer connection timeout for opening channel.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1991.
2017-07-07 12:31:50 +02:00
Koji Kawamura 50c364a793 NIFI-4162: PutSQL batch update error message should include the cause
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1992.
2017-07-07 12:25:10 +02:00
Pierre Villard 45f82dc855 NIFI-4111 - NiFi shutdown
Fixed threads shutdown so that NiFi can shutdown gracefully

NIFI-4111 - Review - Handling SocketRemoteSiteListener (RAW S2S)

This closes #1963.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-07-07 14:23:18 +09:00
Mark Payne 3906d4e1d2 NIFI-1763: Initial implementation of ConfluentSchemaRegistry.
NIFI-1763: Fixed bug where the Confluent Schema Registry Schema Access Writer was not being created

Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #1938
2017-07-06 15:52:07 -04:00
Pierre Villard a3b72f1bb7
NIFI-4143 - externalize MAX_CONCURRENT_REQUESTS. This closes #1962 2017-07-06 15:38:55 -04:00
Mark Payne ba56774fa1
NIFI-4151: Updated UpdateAttribute to only create JAXB Context once; Minor performance tweaks to standard validators and StatusMerge.prettyPrint; updated AbstractConfiguredComponent to not create a new ValidationContext each time that validate is called but only when needed; updated FlowController, StandardControllerServiceProvider, and StandardProcessGroup so that component lookups can be performed using a ConcurrentMap at FlowController level instead of having to perform a depth-first search through all ProcessGroups when calling findProcessor(), findProcessGroup(), findXYZ()
This closes #1979
2017-07-06 13:43:56 -04:00
Mark Payne cff81c0cd2
NIFI-4153: Use a LinkedBlockingQueue instead of a SynchronousQueue for Request Replicator's thread pool so that requests will queue when all threads are active, instead of throwing an Exception. This closes #1980 2017-07-06 10:11:48 -04:00
m-hogue e6b166a3a2 NIFI-552: added regex properties for include and ignore filters in LogAttribute
This closes #1981

Signed-off-by: Tony Kurc <tkurc@apache.org>
2017-07-05 21:21:50 -04:00
m-hogue 47eece5798 NIFI-3193: added ability to authenticate using cert common names
This closes #1971.

Signed-off-by: Tony Kurc <tkurc@apache.org>
Also reviewed by Pierre Villard <pierre.villard.fr@gmail.com>
2017-07-05 15:28:54 -04:00
Pierre Villard 7843b885ee NIFI-4144 - added min/max age to ListHDFS processor
This closes #1966.

Signed-off-by: Tony Kurc <tkurc@apache.org>
2017-07-04 21:18:43 -04:00
Pierre Villard 8b5342dec0 NIFI-3974 - Added charset in LogAttribute
This closes #1967

Signed-off-by: Tony Kurc <tkurc@apache.org>
2017-07-02 12:03:07 -04:00
Pierre Villard 902b6e205d NIFI-4131 - GFT warning message when badly setting relationships
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1954
2017-06-30 15:12:10 -04:00
Mark Payne 7f4cfd51ea NIFI-3897: If swapping data into queue and an unexpected exception/error is thrown, do not lose track of the swap file
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1960.
2017-06-30 13:53:05 +02:00
Mark Payne cdc154f7c8 NIFI-4046: If we are unable to parse out any records from a Kafka Mesaage with ConsumeKafkaRecord, then we should route all of the bytes received to 'parse.failure'
NIFI-4046: Addressed issue of Record Writer failing with ConsumeKafkaRecord

This closes #1906.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-06-30 17:56:14 +09:00
m-hogue 58a623dfa2 NIFI-4037 added InvokeGRPC processor, with proto service IDL
NIFI-4038 added ListenGRPC processor

This closes #1947

Signed-off-by: Tony Kurc <tkurc@apache.org>
2017-06-29 22:01:22 -04:00
yuri1969 3bf1d12706
NIFI-4125 Added secure transform feature and configuration to TransformXML processor to mitigate XXE file system leaks.
This closes #1946.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-06-29 10:40:33 -07:00
Mike Thomsen 5172797448 NIFI-4122 Added the ability to combine multiple Mongo result documents into a single output JSON array.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1948.
2017-06-29 12:43:52 +02:00
Sreeram Garlapati 82ef671953 NIFI-4007 - Update azure EventHubs client to latest version
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1880.
2017-06-29 09:30:39 +02:00
Michael Andre Pearce bdfd710692 NIFI-3983 - Support ability to make JMS 2.0 durable subscriptions on Topic
Add logic in Consumer adding support for all topic consumer combinations, non-durable, durable, shared, durable-shared.
Add new optional config option to supply subscription name.
Add new optional config option to supply clientId.

This closes #1863.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-06-29 10:23:15 +09:00
Matt Burgess 9bfa7469cb NIFI-4137: Add support for UTF-8 in GetFTP and PutFTP
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #1957
2017-06-28 16:41:14 -04:00
Matt Gilman c99c036c20 NIFI-4045:
- Addressing issues causing the eventId to not be relayed when submitting a lineage request under certain conditions.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1903.
2017-06-27 21:48:55 +02:00
Andre F de Miranda 1e7eceee84 NIFI-4126 - Add SupportBatching, SideEffectFree and EventDriven annotations to ExtractGrok
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1950.
2017-06-27 20:56:56 +02:00
yuri1969 202eb5ccbe
NIFI-4119 - Improve UX of canvas label configuration by providing immediate focus to value field
This closes #1940.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-06-27 10:38:35 -07:00
kwydler c37aed4de2 NIFI-4128 Added "All" MessageAttributeNames to GetSQS processor ReceiveMessageRequest
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #1949.
2017-06-26 22:25:16 -07:00
Scott Aslan 24e2981012 [NIFI-4117] update connection config prioritizers to be contained within their parent element
This closes #1944.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
2017-06-26 20:25:50 +02:00
Steve Champagne 68b42c9e54 NIFI-4055: Add a compression option to AvroRecordSetWriter
This closes #1909.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
2017-06-26 19:16:31 +02:00
Koji Kawamura 91ed96f8c4 NIFI-4040: Continue if SFTP failed getting user home directory
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1898.
2017-06-26 19:09:22 +02:00
Koji Kawamura 69613f29c9
NIFI-4050: Fixed NPE at AbstractConfiguredComponent.validate
This closes #1905.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-06-23 13:58:22 -04:00
Pierre Villard 47165afc0c
NIFI-4073 - fix duplicated stack trace
This closes #1916.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-06-23 11:20:07 -04:00
Mark Payne 8878d732dd NIFI-4103: Return an empty Optional if an IP address cannot be found
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1933
2017-06-23 11:14:57 -04:00
Andy LoPresto ff65b42e88 NIFI-4110 Added Thread.sleep() to ensure provenance event indexing occurs before retrieval and unit test passes.
Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #1935
2017-06-23 08:48:34 -04:00
patricker 7c27d4c588 NIFI-3992: Added Original Headers to Include/Exclude
Some headers can cause problems with message parsing, specifically the 'Content-Type' header.
If an email contains attachments, ConsumeEWS may generate emails where the attachments cannot be extracted.

This closes #1867.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-06-23 17:38:19 +09:00
Andy LoPresto 253ea2e73b NIFI-4095 Changed minimum capture group count in ExtractText from 1 to 0.
Added unit test and removed obsolete test.
Added custom validation to enforce capture group if "include capture group 0" is false.
2017-06-22 23:31:35 +02:00
Koji Kawamura 5c755c006b
NIFI-4017: Emit provenance event from Notify.
This closes #1890.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-06-22 16:49:31 -04:00
Pierre Villard f54e146561
NIFI-4085 Add exception in log message when (S)FTP listing fails
This closes #1929.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-06-22 16:06:18 -04:00
Niels Zeilemaker 5b82b735ab
NIFI-4106 Increase log level of user authentication rejection to warn.
This closes #1934.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-06-22 14:03:25 -04:00
Jeff Storck 668a64cd56 NIFI-1586 Implements autopurging of transaction log and snapshot files
When NiFi is clustered, and autopurge.purgeInterval is greater than 1, the DatadirCleanupManager will be started in order to automatically purge transaction log and snapshot files based on the autopurge settings in zookeeper.properties

This closes #1928.
2017-06-22 09:30:38 -04:00
Jeff Storck c99100c934
NIFI-4010 Enables EL on Fetch/List/PutSFTP and List/Fetch/Put/DeleteHDFS processor properties
FetchSFTP/ListSFTP/PutSFTP: Private Key Path
ListHDFS/FetchHDFS/PutHDFS/DeleteHDFS: Hadoop Configuration Resources, Kerberos Principal, Kerberos Keytab, Kerberos Relogin Period

This closes #1148
This closes #1930.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-06-21 17:14:49 -04:00
Maurizio Colleluori 59a32948ea
NIFI-2923 Added evaluation of attribute expressions for Kerberos principal and keytab
Signed-off-by: Bryan Bende <bbende@apache.org>
2017-06-21 17:14:28 -04:00
Maurizio Colleluori 86fa1bba4f
NIFI-2923 Add expression language support to Kerberos parameters used by processors
Signed-off-by: Bryan Bende <bbende@apache.org>
2017-06-21 17:14:27 -04:00
Mark Payne 73e601bc55 NIFI-4102: If first line read does not match Grok expression, skip line
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1932
2017-06-21 15:10:22 -04:00
Joe Percivall 77e49b749e NIFI-4075 Updating 'addRemoteProcessGroup' in StandardFlowSerializer to properly use ScheduledStateLookup. This closes #1922. 2017-06-21 14:20:22 -04:00
Bryan Bende aabd4a25d2 NIFI-4043 Initial commit of nifi-redis-bundle
NIFI-4061 Initial version of RedisStateProvider
- Adding PropertyContext and updating existing contexts to extend it
- Added embedded Redis for unit testing
- Added wrapped StateProvider with NAR ClassLoader in StandardStateManagerProvider
- Updating state-management.xml with config for RedisStateProvider
- Renaming tests that use RedisServer to be IT tests so they don't run all the time

This closes #1918.
2017-06-21 12:47:19 -04:00
Matt Gilman 6bc6f955c0 NIFI-4059:
- Introducing the LdapUserGroupProvider.
- Updating documentation accordingly.
- Moving the IdentityMapping utilities so they were accessible.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1923.
2017-06-19 19:25:33 +02:00
Scott Aslan ad6af1d941 NIFI-3994 Add Create Template to the context menu
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #1883.
2017-06-17 16:46:31 -07:00
Wesley-Lawrence 5a7e6c6ac1 NIFI-4064 Make sure that Funnels with queued incoming FlowFiles, but no outgoing connections yield rather than continually check to run.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1914.
2017-06-17 15:50:40 +02:00
Chris Herrera 2800df30d3 NIFI-2692
- Removing problematic timeout for SMTP Listen
- Converting anonymous method to lambda
- Adding debug and error logging

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1924.
2017-06-17 15:22:33 +02:00
Matt Gilman 490e1da5db NIFI-4019:
- Adding support for X-Forwarded-* headers.
- Unrelated code clean up.
- Addressing additional proxying issues.
- Addressing mistyped landing page.
- Handled trailing slashes in proxy headers for context path
2017-06-15 14:17:35 -04:00
Mark Payne c0ad5bcc7b NIFI-1921: Fixed bug in how average lineage duration was calculated for cluster stat
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1912
2017-06-13 12:29:03 -04:00
Bryan Bende 2b47ade0c4
NIFI-4049 Fixing unit test that doesn't run on OSX and caused Travis failure 2017-06-12 16:36:23 -04:00
Koji Kawamura de1b84e2aa
NIFI-4049: Refactor AtomicDistributedMapCacheClient
To be used with cache engines that does not have revision number.

NIFI-4049: Refactor AtomicDistributedMapCacheClient

Removed old methods completely.

This closes #1904.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-06-12 10:34:48 -04:00
Koji Kawamura 7761903352 NIFI-4041: Add file.size attribute to ListFTP and ListSFTP
This closes #1899.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-06-12 10:53:09 +09:00
Mark Payne e7dcb6f6c5 NIFI-3921: Allow Record Writers to inherit schema from Record
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1902
2017-06-09 16:13:25 -04:00
Matt Gilman 4ed7511bee
NIFI-3653: - Introducing UserGroup and Policy provider interfaces.
- Introducing FileUserGroupProvider and FileAccessPolicyProvider.
- Refactoring FileAuthorizer to utilize the file based implementations.
- Introducing the StandardManagedAuthorizer.
- Decorating the configured ManagedAuthorizer to ensure integrity checks are still performed.
- Loading user groups if possible to use during access decisions.
- Merging responses for requests for AccessPolicies, Users, and UserGroups.
- Adding unit tests as appropriate.
- Adding methods to the User, Group, and AccessPolicy builder that more easily supports generating UUIDs.
- Fixing typo when seeding policies during startup.
- Fixing type in documentation and error messages.

This closes #1897.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-06-09 13:54:10 -04:00
Mark Payne d8dc2bb00b NIFI-4047: Addressed issue with inconsistent Lexical policy between used cached schemas and non-cached schemas
This closes #1907

Signed-off-by: Joe Percivall <JPercivall@apache.org>
2017-06-09 12:05:11 -04:00
Matt Gilman cc741d2be6
NIFI-3997:
- Bumping to next minor version.
2017-06-08 15:22:51 -04:00
Matt Gilman 1bf0a1a849
Merge branch 'NIFI-3997-RC1' 2017-06-08 14:30:10 -04:00
Bryan Bende b0c9428776 NIFI-4030 Populating default values on GenericRecord from Avro schema if not present in RecordSchema
This closes #1896.
2017-06-07 13:59:40 -04:00
Steve Champagne 45e035686f
NIFI-4029: Allow null Avro default values in HortonworksSchemaRegistry
This closes #1894.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-06-07 12:03:53 -04:00
Matt Gilman c86190c513 NIFI-4027: - Fixing the positioning of the tooltips in the component toolbar.
This closes #1893

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
2017-06-06 17:35:43 -04:00
Joe Gresock 3ef0fa48da NIFI-4014: Correcting Event Type text in S2S provenance reporting task
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1887.
2017-06-06 12:06:04 +02:00
Koji Kawamura e1880785d0
NIFI-4021: Fix StandardFlowSynchronizerSpec to pass independently.
This closes #1892.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-06-05 19:25:07 -07:00
Matt Gilman 6ee12e9b47
NIFI-3997-RC1prepare for next development iteration 2017-06-05 11:07:43 -04:00
Matt Gilman ddb73612bd
NIFI-3997-RC1prepare release nifi-1.3.0-RC1 2017-06-05 11:07:28 -04:00
Andre F de Miranda 082b99640e NIFI-4001 - Bump ParCEFone version
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1875.
2017-06-03 12:11:38 +02:00
Andre F de Miranda 4d78052dd4 NIFI-1709 - Introduce logic to probe Linux version using /etc/os-release to nifi.sh
Add explicit paths to support SLES 11 SP4 / OpenSUSE init.d layout

This closes #1794
2017-06-03 08:12:52 +10:00
Mark Payne 32314d70fd NIFI-4009: Added support for several key functions in RecordPath
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1881
2017-06-02 14:06:05 -04:00
Matt Burgess 0bddcfe730
NIFI-4002: Add PutElasticsearchHttpRecord processor
This closes #1878

Signed-off-by: Bryan Rosander <brosander@apache.org>
2017-06-02 13:01:13 -04:00
Mark Payne 37be0b9820 NIFI-3990: This closes #1870. Avoided creating garbage when we can avoid it in the JSON and csv readers and json writer
Signed-off-by: joewitt <joewitt@apache.org>
2017-06-02 10:01:40 -04:00
Mark Payne 067e9dfeb0
NIFI-4003: Expose configuration option for cache size and duration NIFI-4003: Addressed remaining spots where client does not cache information
This closes #1879.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-06-01 15:31:35 -04:00
Mark Payne 2595d816c4 NIFI-3986: This closes #1868. When we encounted EOFException from sequential record reader event iterator, just treat it as not having any more events
Signed-off-by: joewitt <joewitt@apache.org>
2017-06-01 15:08:12 -04:00
Mark Payne 7035694e37
NIFI-3995: Updated Hwx Encoded Schema Ref Writer to write 13 bytes for header instead of 14; added unit test to verify
This closes #1876.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-06-01 10:29:58 -04:00
Mark Payne a0b2311ff6 NIFI-3995: This closes #1873. No longer use the 14th byte in the header for hwx content-encoded schema reference
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-31 13:41:25 -04:00
Mark Payne 2b435cdfc6 NIFI-3985: This closes #1864. Added 'Starting Position' property to SiteToSiteReportingTask; also added additionalDetails.html that explains the schema and updated the reporting task to stop publishing when the user clicks 'stops' instead of running indefinitely until the reporting task has caught up
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-26 21:05:45 -05:00
Yolanda M. Davis dd50745a9f
NIFI-3984 - upgraded version of schema-registry client to 0.2.1
This closes #1866.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-26 17:59:53 -04:00
Pierre Villard 80dfe0257f ExtractGrok - Include exception message when compile fails
This closes: #1845

Signed-off-by: Andre F de Miranda <trixpan@users.noreply.github.com>
2017-05-27 00:56:25 +10:00
Bryan Bende e85081ed20 NIFI-3979 Documenting how ListHDFS maintains state and performs listings
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1860.
2017-05-26 10:40:22 +02:00
Mark Payne 5cb928131c
NIFI-3981: When serializing flow to cluster, use the Scheduled State of ports as they are configured to be, not the current state, since the current state may change as soon as the FlowController has finished initializing
This closes #1861.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-25 16:45:20 -04:00
Koji Kawamura 13b59b5621 NIFI-3958: Decimal logical type with undefined precision and scale.
- Oracle NUMBER can return 0 precision and -127 or 0 scale with variable scale NUMBER such as ROWNUM or function result
- Added 'Default Decimal Precision' and 'Default Decimal Scale' property to ExecuteSQL and QueryDatabaseTable to apply default precision and scale if those are unknown
- Coerce BigDecimal scale to field schema logical type, so that BigDecimals having different scale can be written

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1851
2017-05-25 14:37:17 -04:00
Mark Payne 36911957dc NIFI-3972: This closes #1855. When enabling Controller Services on startup, wait until service is enabled (or until 30 seconds elapses, whichever comes first) before returning. This avoids having Service A depend on Service B and then attempting to start Service A before Service B is fully enabled
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-25 13:32:02 -04:00
Mark Payne eb25c8547a NIFI-3972: Ensure that we wait until service state becomes enabled before triggering completable future that says that it's enabled 2017-05-25 13:31:53 -04:00
Joseph Niemiec 44fdc0e4ef NIFI-3640 uri eventhub changes
This closes #1617
2017-05-25 13:14:51 -04:00
Koji Kawamura 4d0667380a NIFI-3404: Improved UX of LookupAttributes.
- Added dependency notice.
- Added EL evaluation at SimpleKeyValueLookupService.
- Updated documentation.
- Updated CommonsConfigurationLookupService to throw LookupFailureException if it fails to get configuration so that error messages can be displayed at each processor bulletin.
- Added calling getConfiguration at OnEnabled of CommonsConfigurationLookupService, so that the service will stay in Enabling state if there is any issue.

Signed-off-by: Joey Frazee <jfrazee@apache.org>
2017-05-25 10:05:32 -05:00
Joey Frazee 46e2420d74 NIFI-3404 Added LookupAttribute processor and lookup controller services
Signed-off-by: Joey Frazee <jfrazee@apache.org>
2017-05-25 10:05:32 -05:00
Koji Kawamura 23cbc3b346 NIFI-3976: This closes #1857. Fix ConsumePOP3 and ConsumeIMAP to delete messages.
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-25 10:59:53 -04:00
Pierre Villard dc5e032368 NIFI-3791 - added back pressure data into S2SStatusReportingTask
This closes #1745.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-05-25 18:20:11 +09:00
Andre F de Miranda c07850aec3 NIFI-2702 - Deprecates nifi-standard-bundle/*JMS and document the recommended alternative
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1643.
2017-05-25 08:44:39 +02:00
Pierre Villard e05005584d NIFI-3809 - Added HTTP mode and HTTP proxy for S2S Reporting Tasks
This closes #1754.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-05-25 15:31:49 +09:00
Pierre Villard b6eb0ac0fb NIFI-3859 - Provide filtering options in S2SProvenanceReportingTask
This closes #1777.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-05-25 14:10:50 +09:00
Mark Payne eaefec6d81 NIFI-3971: This closes #1854. Fixed bug in calculating content size that was transferred when cloning a relationship
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-24 22:14:56 -04:00
Mark Payne 5aa3baca79 NIFI-3568: This closes #1577. Use a cached thread pool in order to allow ThreadPoolRequestReplicator to scale up the number of threads to some configurable max
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-24 21:21:45 -04:00
Scott Aslan 86728bac7e
[NIFI-3925] for the jquery.combo plugin: update the calculation of the width of the combo box options overlay and also updated the plugins styles to now leverage the css calc() to determine the exact width of .combotext and .combo-option-text elements. This closes #1848 2017-05-24 16:15:42 -04:00
Matt Gilman f97b3fe455
NIFI-3963: - Ensuring the RemoteGroupPort yields when the details cannot be refreshed from any of the configured remote instances.
This closes #1853.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-24 15:58:22 -04:00
Bryan Bende ae3db82303
NIFI-3644 Fixing the result handler in HBase_1_1_2_ClientMapCacheService to use the offsets for the value bytes
This closes #1645.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-24 14:59:28 -04:00
baolsen 152f002abf
Completed initial development of HBase_1_1_2_ClientMapCacheService.java which is compatible with DetectDuplicate (and other processors)
Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-24 14:59:23 -04:00
Mike Moser 232380dbfd NIFI-1452 on timer-driven yield, use the greater of yield duration or run schedule
This closes #1832.
2017-05-24 14:24:29 -04:00
Mark Payne b12cf8a6d2 NIFI-3671: This closes #1852. Ensure that we use the existing ResourceClaim (if it exists) when swapping data in, instead of creating a new one. Otherwise, if the ResourceClaim is still writable, then we may archive the data and then write to it, which can cause a NullPointerException in FileSystemRepository
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-24 14:21:18 -04:00
Pierre Villard fb7d6d1150 NIFI-3799 Add Input Requirement Info to Docs
Added input requirement info for processor documentation

This closes #1756.

Signed-off-by: James Wing <jvwing@gmail.com>
2017-05-24 10:45:45 -07:00
Mark Payne 08b66b5b6a NIFI-3969: Prevent merging flowfiles prematurely when all bins fill but some are already full and can be processed
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1850.
2017-05-24 19:36:18 +02:00
Pierre Villard bb96b0f464 NIFI-3816 - Added correct mime.type to S2S bulletin and provenance reporting tasks
This closes #1758.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-05-25 00:33:50 +09:00
Scott Aslan 7a46238baf
[NIFI-3943] align combo option item toolips to hover closer to the icon. This closes #1849 2017-05-24 11:14:48 -04:00
Pierre Villard ba49b8427c
NIFI-3191 - HDFS Processors Should Allow Choosing LZO Compression
This closes #1802.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-24 11:04:00 -04:00
Pierre Villard d63fa8a822 NIFI-3878 This closes #1790. AmbariReportingTask - Default run schedule set to 1 minute
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-24 00:09:17 -04:00
Andy LoPresto 6d07ebf50c NIFI-3954 This closes #1841. Upgraded version of httpclient library to handle TLS issues.
Added explicit dependency on httpcore for @ThreadSafe annotation.

Signed-off-by: joewitt <joewitt@apache.org>
2017-05-23 23:42:28 -04:00
Mark Payne 2c751a8e5b NIFI-3962: This closes #1846. Updated ConsumerLease to better handle batching of messages into a single FlowFile in the same way that it is handled for demarcated data 2017-05-23 23:20:13 -04:00
Matt Burgess fb925fc182 NIFI-3867: Fixed issue with getConnectionURL in HiveConnectionPool using Expression Language
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1847.
2017-05-23 21:28:59 +02:00
James Wing 9238fdb493 NIFI-3903 Unit Tests for AWS Processors
* Added non-integration unit tests for Lambda, S3, SNS, and SQS processors
* Moved non-integration tests out of integration test files
* Moved SQS integration tests to IT* files for clarification

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1814.
2017-05-23 19:25:55 +02:00
Joe Skora 0a7d149656 NIFI-3932 Symbols in API comments cause problems with Swagger Codegen
* Update ApiModelProperty descriptions to avoid problematic characters.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1821.
2017-05-23 19:04:54 +02:00
Matt Gilman 3ec0aa947f NIFI-3955:
- Filtering out the EventTime searchable field for all implementation of ProvenanceRepository. Prefer user query using the startDate/endDate fields.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1844.
2017-05-23 18:48:29 +02:00
James Wing b64232117f NIFI-3707 Sort Attributes in FlowFile Details UI
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1843.
2017-05-23 18:06:18 +02:00
Bryan Bende a8de27e69b NIFI-3732 Adding connect with timeout to StandardCommsSession and SSLCommsSession to avoid blocking
This closes #1842.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-05-23 15:43:27 +09:00
Matt Gilman ded396f0ef NIFI-3933:
- When monitoring heartbeats use the connected nodes as the basis for the check. This addresses the case when a node is terminated and no corresponding heartbeats exist.

This closes #1838.
2017-05-22 16:51:13 -04:00
Matt Gilman d33c4c72d4 NIFI-3933:
- When monitoring heartbeats use the connected nodes as the basis for the check. This addresses the case when a node is terminated and no corresponding heartbeats exist.
2017-05-22 16:50:30 -04:00
Mark Payne a1b07b1e9c
NIFI-3949: Updated Grok Reader to allow for sub-patterns to be used when determining the schema
This closes #1839.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-22 16:30:47 -04:00
Mark Payne 6937a6cf64 NIFI-3953: This closes #1837. Allow multiple schemas on same kafka topic/partition for ConsumeKafkaRecord_0_10
Also, updated record writers to ensure that they write the schema as appropriate if not using a RecordSet. Updated ConsumeKafkaRecord to allow for multiple schemas to be on same topic and partition

Signed-off-by: joewitt <joewitt@apache.org>
2017-05-22 14:37:53 -04:00
Mark Payne 6d16fdf170
NIFI-3952: Updated UpdateRecord to pass field-related variables to the Expression Language
This closes #1836.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-22 14:32:45 -04:00
Mark Payne 7f8987471d
NIFI-3946: Update LookupService to take a Map instead of a String for the input
NIFI-3946: Fixed issues where null values were returned instead of empty optionals

This closes #1833.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-22 11:52:41 -04:00
Bryan Bende 71cd497fef NIFI-3942 Added retry logic if a lookup fails due to InvalidDatabaseException which occurs if the underlying file was modified before we could refresh the reader
This closes #1831.
2017-05-22 09:12:51 -04:00
Bryan Bende f35e0ecdd0 NIFI-3942 Making IPLookupService reload the database file on the fly when detecting the file has changed 2017-05-22 09:06:42 -04:00
Mark Payne c49933f03d NIFI-3948: This closes #1834. Added flush() method to RecordWriter and call it when writing a single record to OutputStream for PublishKafkaRecord. Also removed no-longer-used class WriteAvroResult
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-19 23:05:04 -04:00
Tim Reardon 58ce52d5d6 NIFI-3854 This closes #1773. Expand expression language support for Kafka processors
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-19 15:51:23 -04:00
Matt Burgess 9294a26139
NIFI-3938: Added ScriptedLookupService, some refactor for reusable scripting classes
This closes #1828.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-05-19 11:20:38 -07:00
Bryan Bende d4f0c1d048 NIFI-3945 This closes #1829. Adding documentaion about security protocols to Kafka 0.10 processors
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-19 13:59:27 -04:00
Mark Payne ae9953db64 NIFI-3857: This closes #1825. Added PartitionRecord processor
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-19 02:08:52 -04:00
Mark Payne 9bd0246a96 NIFI-3863: Initial implementation of Lookup Services. Implemented LookupRecord processors. This required some refactoring of RecordSetWriter interface, so refactored that interface and all implementations and references of it 2017-05-19 01:02:41 -04:00
joewitt 3b98abbf41
NIFI-3940 upgraded to latest jruby which resolves an apparent lgpl bunding issue
This closes #1827.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-05-18 21:37:25 -07:00
joewitt 762293c01c NIFI-3935 made the old persistent prov test more forgiving of timing issues and extended lucene test timeouts for slower environments. These should be integration tests really.
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1823
2017-05-18 14:42:46 -04:00
joewitt be67f3552b
NIFI-3853 fixing tests to properly encapsulate their environments
This closes #1822.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-18 13:36:02 -04:00
Koji Kawamura 36e7bd6164 NIFI-3894: This closes #1820. Fixed close and consume order with compression.
Before this fix, 'NullPointerException: Inflater has been closed' can be thrown as the Inflater is closed before input stream is consumed.

Also, calling close from AbstractTransaction.receive is removed, because the DataPacket is exposed as its return value and this class will not be able to know when to close the stream.

Signed-off-by: joewitt <joewitt@apache.org>
2017-05-17 22:57:33 -04:00
Jeff Storck 8e1c79eaaf
NIFI-1963 Allows a node reconnecting to the cluster to inherit non-fingerprinted processor settings
Forces a node reconnecting to a cluster to serialize the updated flow to disk
Added most processor settings to the flow fingerprint (excluding name, style, comment, position, and schedule state)
Updated some test data for FingerprintFactoryTest to test for new fields added to the flow fingerprint
Updated StandardProcessorNode to allow processor comments and name to be settable while a processor is running
Updated StandardFlowSynchronizer to inherit non-fingerprinted processor settings (name, style, comment, and position) when flow is already synchronized
This closes #1812
2017-05-17 17:35:33 -04:00
Andre F de Miranda 289dde098e
NIFI-3896 - Makes DeprecationNotice more intuitive. This closes #1799
Update developers guide on how to deprecate a component
2017-05-17 16:37:11 -04:00
Mark Payne 5cf4bf61b3
NIFI-3917: Use a default value of 'false' for CSV Reader's Skip Header Line property. This closes #1819 2017-05-17 13:31:53 -04:00
Koji Kawamura 3fd7d3b272 NIFI-3873: Improve Avro logical types readability
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1788
2017-05-17 13:08:00 -04:00
Scott Aslan 2dde480029
[NIFI-3906] Introduce additional validation when handling user-supplied input to ensure appropriate values and output encoding within SlickGrid tables. This closes #1818 2017-05-17 13:04:31 -04:00
Mark Payne 9198bee038
NIFI-3923: Only start processors after repositories have been initialized. This closes #1817 2017-05-17 11:12:59 -04:00
Koji Kawamura 40a9cd4f2e NIFI-3919: Let AvroTypeUtil try every possible type
Before this fix, AvroTypeUtil can throw an Exception before trying every possible data types defined within a union field.

This closes #1816.
2017-05-17 10:53:27 -04:00
Koji Kawamura f019d509f3 NIFI-3918: Added Choice mapping to JsonTreeRowRecordReader. 2017-05-17 09:49:11 -04:00
Koji Kawamura 33dc3e36fb NIFI-3920: Remove unnecessary code from AvroTypeUtil
- Removed remaining duplicate lines of code left by NIFI-3861 refactoring.
- Added test case that writes Avro record having union field.

This closes #1813.
2017-05-17 09:45:21 -04:00
Matt Gilman c50d516076 NIFI-3848:
- Allowing the revision check to be overridden when the state of a nodes connection to the cluster changes.

This closes #1811
2017-05-16 17:09:48 -04:00
Matt Gilman 6b71b4cbb8 NIFI-3868: - Ensuring we do not attempt to group bulletins that lack permissions. - Only group bulletins when all nodes report the same message. - Retain the most recent bulletin.
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1801
2017-05-16 16:54:30 -04:00
Mark Payne ce1bc42ac5 NIFI-3912: This closes #1809. Fixed NPE that would result in validation failure for FreeFormTextRecordSetWriter
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-16 16:17:04 -04:00
Mark Payne 2f3ba57dd6
NIFI-3914: Do not validate that remote group ports can start transmitting if there is no connection
This closes #1810.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2017-05-16 16:10:00 -04:00
Bryan Bende 3a0004a665
NIFI-3904 Adding logic to only reload when incoming bundle is different and to use additional URLs from spec component
NIFI-3908 Changing UI to submit filterType for CSs and filter for processors and reporting tasks
2017-05-16 16:08:28 -04:00
Mark Payne 494a0e8928
NIFI-3900: Ensure that when we serialize a flow to send to Cluster Coordinator that we include the Scheduled State for processors as they are intended to be, not as they are currently because during startup they may not have been started yet.
NIFI-3900: Rebased against master and updated new unit test to use new method signature for FlowSerializer.serialize

This closes #1804.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-16 16:04:09 -04:00
Matt Gilman ac8e57259f
NIFI-3907 Added response headers.
Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-05-16 11:55:36 -07:00
Mark Payne fb94990e60 NIFI-3909: This closes #1806. If we have a FlowFile with 0 records, ensure that PublishKafkaRecord_0_10 handles the flowfile properly
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-16 14:42:42 -04:00
Scott Aslan f5f6cab646
NIFI-3906 Added formatting to tables.
Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-05-16 11:17:29 -07:00
Koji Kawamura 77a676bf92 NIFI-3894: Call Inflater/Deflater.end to free up memory
This closes #1796.
2017-05-16 13:49:31 -04:00
Mark Payne 4fdea680ec NIFI-3905: This closes #1805. When a Provenance Query is submitted to WriteAheadProvenanceRepository, purge any obsolete queries from the internal state before rejecting the query due to 'too many outstanding queries'
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-16 13:48:16 -04:00
Aldrin Piri 58cf15a912 NIFI-2199 NIFI-3112 Pipe stdout to /dev/null and allow stderr to propagate through.
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #1786
2017-05-16 13:31:15 -04:00
Mark Payne 3f4b276b71 NIFI-3910: Fixed issue where CSVRecordReader throws an Exception if a field is missing instead of using a null value
This closes #1807.
2017-05-16 13:16:46 -04:00
Matt Gilman 824712bffe
NIFI-3901:
- Addressing response code of POST /provenance/lineage.
- Ensuring cluster node details are set accordingly.

This closes #1803.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2017-05-16 12:36:27 -04:00
Mark Payne 603f713a40 NIFI-3860: Relax the constraint that the ProcessSession must always be given the most up-to-date version of a FlowFile
This closes #1778.
2017-05-16 08:51:41 -04:00
Matt Gilman 6ffb78d404
NIFI-3853:
- Filtering out certain control characters and unpaired Unicode surrogate codepoints prior to saving the flow.xml.

This closes #1784.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-05-15 19:02:03 -07:00
Peter G. Horvath bf15502e19 NIFI-3763 Add new processor to log user defined messages built with NiFi Expression Language
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1737
2017-05-15 14:18:57 -04:00
Koji Kawamura 1811ba5681 NIFI-2624: Avro logical types for ExecuteSQL and QueryDatabaseTable
- Added Logical type support for DECIMAL/NUMBER, DATE, TIME and TIMESTAMP column types.
- Added Logical type 'decimal' to AvroReader so that Avro records with logical types written by ExecuteSQL and QueryDatabaseTable can be consumed by AvroReader.
- Added JdbcCommon.AvroConversionOptions to consolidate conversion options.
- Added 'Use Avro Logical Types' property to ExecuteSQL and QueryDatabaseTable to toggle whether to use Logical types.
- Added 'mime.type' FlowFile attribute as 'application/avro-binary' so that output FlowFiles can be displayed by content viewer.

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1798
2017-05-15 14:15:23 -04:00
Tim Reardon 20a1fc24d7 NIFI-3885 DynamoDB Processor EL Support
Add EL support to remaining Dynamo processor properties

Signed-off-by: James Wing <jvwing@gmail.com>

This closes #1793.
2017-05-15 10:13:53 -07:00
Tim Reardon 196ca237e6 NIFI-3855 Support PutSlack attachments as dynamic properties
NIFI-3855 Refactor deprecated API

NIFI-3855 Log separate error messages in PutSlack

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1774
2017-05-15 09:08:43 -04:00
Andre F de Miranda d092551211
NIFI-3761
- adjust testFullyDocumentedProcessor to correctly eval to false when needed
- Introduce the ability to deprecate a component
- Allow documentation to inform user about deprecation of a component
- This closes #1718

NIFI-391 - Add set/getDeprecationReson to DocumentedDTO and use it within DtoFactory'
2017-05-15 08:43:32 -04:00
Matt Burgess 3353865ce9 NIFI-3867: Add Expression Language support to HiveConnectionPool properties
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1783.
2017-05-15 13:46:29 +02:00
Koji Kawamura bc68eb754f NIFI-3895: Fixed AvroReader to use Schema Access Strategy default value properly.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1797.
2017-05-15 13:27:26 +02:00
Scott Aslan 09f6a7040c
[NIFI-3830] force browsers to use URLSearchParams polyfill do to bugs and inconsistent browser implementations. This closes #1785 2017-05-13 07:34:33 -04:00
Steve Champagne 382eef2183 NIFI-3879: Allow null Avro default values
- Avro uses their own class for null values, so a standard check against
  null isn't picking them up.

This closes #1792.
2017-05-12 14:58:43 -04:00
Tim Reardon e9848f4276 NIFI-3881 Fix PutHiveStreaming EL evaluation
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1791
2017-05-12 14:06:32 -04:00
Tim Reardon afd2b04afd NIFI-3852 Add expression language support to Cassandra processors
Remove unused import

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1770
2017-05-12 13:52:07 -04:00
Mark Payne b1901d5fe0 NIFI-3838: Initial implementation of RecordPath and UpdateRecord processor
NIFI-3838: Updated version from 1.2.0-SNAPSHOT to 1.3.0-SNAPSHOT; removed unneeded value from AttributeExpression.ResultType enum

NIFI-3838: Addressed PR Review feedback

NIFI-3838: Allow for schemas to be merged together for a record; refactored RecordSetWriterFactory so that there is a method to obtain the schema and then the writer is created with that schema. Added additional unit tests

NIFI-3838: Addressed problems with documentation based on PR Review

NIFI-3838: Fixed checkstyle violation

NIFI-3838: Addressed issue of comparing different types of Number objects

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1772
2017-05-12 12:36:52 -04:00
Scott Aslan 1c58e78ce9 [NIFI-3876] trim whitespace processor names before applying ellipsis
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1789.
2017-05-12 16:01:43 +02:00
Matt Gilman dd0306cce7 NIFI-3837: - Being more specific regarding the escaping of back references in evaluated expressions.
Signed-off-by: Matt Burgess <mattyb149@apache.org>

fixed typo in comment

This closes #1781
2017-05-11 22:16:46 -04:00
Koji Kawamura 0b0ac196ea NIFI-3861: AvroTypeUtil used different constant.
Previous fix #1779 refactored the way to check Logical type to use string constants.
One of those refactoring used wrong constant mistakenly in normalizeValue method.

Fortunately, this defect is harmless since even though normalizeValue did not convert int to Time, DataTypeUtils.convertType does the same conversion.

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1782
2017-05-11 21:31:07 -04:00
Steve Champagne 6e4db6b11a NIFI-3871: Convert Avro map values
This closes #1787.

- When converting from a raw value to an Avro object, convert the values
  of any Avro map types so that they can be complex types like other
  records.
2017-05-11 20:25:19 -04:00
Matt Gilman cf497c9cf9
NIFI-3865: - Obtaining the appropriate input stream after accessing content. The stream was being consumed into a buffer and was not longer available.
This closes #1780.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-10 15:51:00 -04:00
Tim Reardon dc44e30698 NIFI-3856 PutCloudWatchMetric Stats, Dimensions
PutCloudWatchMetric support for StatisticSets and Dimensions

Signed-off-by: James Wing <jvwing@gmail.com>

This closes #1775.
2017-05-10 12:07:04 -07:00
Tim Reardon 361a58c531 NIFI-3851 Support EL in AWS Endpoint Override URL
Add expression language support to AWS Endpoint Override URL property

Signed-off-by: James Wing <jvwing@gmail.com>

This closes #1769.
2017-05-10 10:37:39 -07:00
Koji Kawamura 72de1cbdef NIFI-3861: Fixed AvroReader nullable logical types issue
- AvroReader did not convert logical types if those are defined with union
- Consolidated createSchema method in AvroSchemaRegistry and AvroTypeUtil as both has identical implementation and mai
ntaining both would be error-prone

This closes #1779.
2017-05-10 10:08:22 -04:00
Koji Kawamura 744ecc3d83 NIFI-3832: Fixed AvroRecordSetWriter validation NPE.
This closes #1768.
2017-05-09 14:11:43 -04:00
Bryan Bende 3af53419af
NIFI-3770-RC2 prepare for next development iteration 2017-05-05 20:50:28 -04:00
Bryan Bende 3a605af8e0
NIFI-3770-RC2 prepare release nifi-1.2.0-RC2 2017-05-05 20:50:14 -04:00
joewitt f4f174b899
NIFI-3826 added proper NOTICE entries for apache calcite and maxmind source
This closes #1766.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-05 20:00:22 -04:00
joewitt 829c02b058 NIFI-3825 fix L&N for jolt, json, and source deps
Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #1765
2017-05-05 18:43:04 -04:00
Jeremy Dyer 5f65b2561a NIFI-3600 Improve logging and relationship routing for failures in DeleteHDFS
realized that the session should be cloned here because its inside a for loop and the original flow file would be transferred but not be the latest flow file if an error occurred in the for loop

@trixpan at a high level what do you think about this approach?

NIFI-3600: Added unit test

NIFI-3600: Removed the hdfs.error.code attribute
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1595
2017-05-05 15:25:08 -04:00
Matt Gilman f0dfcc180d NIFI-3808:
- Making the selector for the enable/disable controller service state more specific. Previously name was too general and was affect other parts of the application.

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #1759
2017-05-05 15:19:28 -04:00
joewitt ee8dc7640f
NIFI-3820 added calcite to assembly notice and updated all copyright years for compress, avro, and jetty
This closes #1764.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-05 14:45:40 -04:00
Mark Payne 12249e5a08
NIFI-2268: Instead of allowing HandleHttpRequest to run constantly calling BlockingQueue.poll() as fast as possible, call BlockingQueue.poll(2, TimeUnit.MILLISECONDS) to avoid overutilization of CPU when there is no work to be done
This closes #1762.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-05 13:58:10 -04:00
Matt Gilman e6be5d3276
NIFI-3801:
- Making the version text next to the component name more subtle.

This closes #1760.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-05-05 13:50:21 -04:00
Koji Kawamura af6f63691c
NIFI-3818: PutHiveStreaming throws IllegalStateException
Changed from async append to sync as it breaks 'recursionSet' check in StandardProcessSession by updating it from multiple threads, resulting IllegalStateException to happen.

This closes #1761.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-05 13:25:59 -04:00
Scott Aslan 85405dae15
[NIFI-3813] fix broken PG config dialog when user has read only perms on the PG. This closes #1757 2017-05-05 12:37:56 -04:00
Bryan Bende d9410d6404 NIFI-3795 Moved no-git profile to nifi-nar-bundles
Removing buildnumber plugin and build-info-no-git profile from nifi-assembly
and adding build-info-no-git profile to nifi-nar-bundles pom

Signed-off-by: James Wing <jvwing@gmail.com>
2017-05-04 10:09:25 -07:00
Andy LoPresto 4f40eca16c
NIFI-3788 Switched Amazon HTTP client instantiation from using null HostnameVerifier (which defaulted to Strict, which cannot handle wildcard certificate hostnames) to DefaultHostnameVerifier, which is fine.
I still want to add unit tests and integration tests, but I ran a flow which had previously caused the reproducible exception and this worked fine (flow showed objects were put in S3, no exceptions, and I verified through AWS Web Console that new objects were present).

This closes #1753.

Signed-off-by: Bryan Rosander <brosander@apache.org>
2017-05-04 12:09:52 -04:00
Matt Gilman f3745065b0
NIFI-3800:
- Cleaning up the headers when replicating requests.

This closes #1752.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-05-04 12:05:24 -04:00
Andre F de Miranda 970ddf8f7e
NIFI-3794 - Expose the control of ListenRELP's CLIENT_AUTH property to DFM
This closes #1746.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-04 10:50:22 -04:00
Scott Aslan 98fb58975f [NIFI-3021] add white-space: pre to the #label-value element
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1749.
2017-05-04 07:45:08 +02:00
Bryan Bende 1e75f33677
NIFI-3789 Removing unnecessary intermittent test failure as described in JIRA
This closes #1743.
2017-05-03 14:07:18 -04:00
Mark Payne 9b177fbcba
NIFI-3787: Addressed NPE and ensure that if validation fails due to RuntimeException, that it gets logged. Also clarified documentation for Json Reader services
This closes #1742.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-03 13:13:43 -04:00
Matt Burgess b7c15c360b NIFI-3718: Fixed timezone issue in TestAvroReaderWithEmbeddedSchema
This closes #1738
2017-05-03 11:53:40 -04:00
Bryan Bende e723a2c5c4
NIFI-3776 Correcting links in documentation for SeeAlso and properties referencing controller services. This closes #1739 2017-05-03 11:15:48 -04:00
Matt Gilman ce233bdbbb
NIFI-3782: - Ensuring referenced Controller Services are not included in copy/paste requests when the Processor is located in a sub group.
This closes #1740.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-03 11:04:08 -04:00
Andre F de Miranda 54d47c7f74 NIFI-3726 - Introduces CompareFuzzyHash processor
- Abstract FuzzyhashContent to reduce a bit of code
            duplication

NIFI-3726 - Attempt to address peer review comments

NIFI-3726 - Addresses additional PR comments

NIFI-3726 - Fix checkstyle violations

NIFI-3726 - Fix issues with POMs

NIFI-3726 - Fixes the logging level to error in case of failure due to message too short

NIFI-3726 - Capture ArrayIndexOutOfBoundsException when consuming a hash source

            that matches but lacks a separator AND filename or matches but lacks
            a filename(i.e. ends with trailing separator)

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1692
2017-05-03 09:18:17 -04:00
Mark Payne daa277b067 NIFI-3775: This closes #1735. Ensure that a '0' byte is written out to Hortonworks Encoded Schema Reference header to indicate that GenericRecord is being used
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-02 17:43:26 -04:00
Jeff Storck 26d90fbccf
NIFI-1833 - Addressed issues from PR review.
NIFI-1833 Moved AbstractListProcessor.java, EntityListing.java, and ListableEntity.java from nifi-standard-processors into nifi-processor-utils
Moved TestAbstractListProcessor.java into nifi-processor-utils
Set nifi-azure-nar's nar dependency back to nifi-standard-services-api-nar
Fixed failing integration tests (ITFetchAzureBlobStorage.java, ITListAzureBlobStorage.java, and ITPutAzureStorageBlob.java) and refactored them to be able to run in parallel

NIFI-1833 Moved security notice info in the additional details documentation into the descriptions of the specific attributes for which those notices are intended
Added displayName usage to properties
Updated exception handling in FetchAzureBlobStorage.java and PutAzureBlobStorage.java to cause flowfiles with Output/InputStreamCallback failures to be routed to the processor's failure relationship
Cleaned up dependencies in pom

NIFI-1833 Removed unnecessary calls to map on Optional in the onTrigger exception handling of FetchAzureBlobStorage.java and PutAzureBlobStorage.java

NIFI-1833 Updates due to nifi-processor-utils being moved under nifi-nar-bundles

This closes #1719.

Signed-off-by: Bryan Rosander <brosander@apache.org>
2017-05-02 14:39:46 -04:00
Andrew Grande f30c8169ab
NIFI-1833 - Addressed issues from PR review.
Addressed dependency issues from the review.
Addressed a checkstyle issue.
Review: reworded the descriptions.
Review: implemented the reset condition logic.
Review: dropped static qualifier from method signatures, not required really
Review: removed sys.out, inlined a single method to get access to the ProcessContext.getName()
Switched to HTTPS as per MSFT recommendation. Some DRY. Dropped cruft.
Addressing review suggestions from 4/5
Review: documentation improvements
Review: documentation improvements

This closes #1636.

Signed-off-by: Bryan Rosander <brosander@apache.org>
2017-05-02 14:39:37 -04:00
Simon Elliston Ball 3488a169ca
NIFI-1833 - Azure Storage processors
Signed-off-by: Bryan Rosander <brosander@apache.org>
2017-05-02 14:39:16 -04:00