Commit Graph

4147 Commits

Author SHA1 Message Date
Koji Kawamura 4d0667380a NIFI-3404: Improved UX of LookupAttributes.
- Added dependency notice.
- Added EL evaluation at SimpleKeyValueLookupService.
- Updated documentation.
- Updated CommonsConfigurationLookupService to throw LookupFailureException if it fails to get configuration so that error messages can be displayed at each processor bulletin.
- Added calling getConfiguration at OnEnabled of CommonsConfigurationLookupService, so that the service will stay in Enabling state if there is any issue.

Signed-off-by: Joey Frazee <jfrazee@apache.org>
2017-05-25 10:05:32 -05:00
Joey Frazee 46e2420d74 NIFI-3404 Added LookupAttribute processor and lookup controller services
Signed-off-by: Joey Frazee <jfrazee@apache.org>
2017-05-25 10:05:32 -05:00
Koji Kawamura 23cbc3b346 NIFI-3976: This closes #1857. Fix ConsumePOP3 and ConsumeIMAP to delete messages.
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-25 10:59:53 -04:00
Pierre Villard dc5e032368 NIFI-3791 - added back pressure data into S2SStatusReportingTask
This closes #1745.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-05-25 18:20:11 +09:00
Andre F de Miranda c07850aec3 NIFI-2702 - Deprecates nifi-standard-bundle/*JMS and document the recommended alternative
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1643.
2017-05-25 08:44:39 +02:00
Pierre Villard e05005584d NIFI-3809 - Added HTTP mode and HTTP proxy for S2S Reporting Tasks
This closes #1754.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-05-25 15:31:49 +09:00
Pierre Villard b6eb0ac0fb NIFI-3859 - Provide filtering options in S2SProvenanceReportingTask
This closes #1777.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-05-25 14:10:50 +09:00
Mark Payne eaefec6d81 NIFI-3971: This closes #1854. Fixed bug in calculating content size that was transferred when cloning a relationship
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-24 22:14:56 -04:00
Mark Payne 5aa3baca79 NIFI-3568: This closes #1577. Use a cached thread pool in order to allow ThreadPoolRequestReplicator to scale up the number of threads to some configurable max
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-24 21:21:45 -04:00
Scott Aslan 86728bac7e
[NIFI-3925] for the jquery.combo plugin: update the calculation of the width of the combo box options overlay and also updated the plugins styles to now leverage the css calc() to determine the exact width of .combotext and .combo-option-text elements. This closes #1848 2017-05-24 16:15:42 -04:00
Matt Gilman f97b3fe455
NIFI-3963: - Ensuring the RemoteGroupPort yields when the details cannot be refreshed from any of the configured remote instances.
This closes #1853.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-24 15:58:22 -04:00
Bryan Bende ae3db82303
NIFI-3644 Fixing the result handler in HBase_1_1_2_ClientMapCacheService to use the offsets for the value bytes
This closes #1645.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-24 14:59:28 -04:00
baolsen 152f002abf
Completed initial development of HBase_1_1_2_ClientMapCacheService.java which is compatible with DetectDuplicate (and other processors)
Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-24 14:59:23 -04:00
Mike Moser 232380dbfd NIFI-1452 on timer-driven yield, use the greater of yield duration or run schedule
This closes #1832.
2017-05-24 14:24:29 -04:00
Mark Payne b12cf8a6d2 NIFI-3671: This closes #1852. Ensure that we use the existing ResourceClaim (if it exists) when swapping data in, instead of creating a new one. Otherwise, if the ResourceClaim is still writable, then we may archive the data and then write to it, which can cause a NullPointerException in FileSystemRepository
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-24 14:21:18 -04:00
Pierre Villard fb7d6d1150 NIFI-3799 Add Input Requirement Info to Docs
Added input requirement info for processor documentation

This closes #1756.

Signed-off-by: James Wing <jvwing@gmail.com>
2017-05-24 10:45:45 -07:00
Mark Payne 08b66b5b6a NIFI-3969: Prevent merging flowfiles prematurely when all bins fill but some are already full and can be processed
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1850.
2017-05-24 19:36:18 +02:00
Pierre Villard bb96b0f464 NIFI-3816 - Added correct mime.type to S2S bulletin and provenance reporting tasks
This closes #1758.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-05-25 00:33:50 +09:00
Scott Aslan 7a46238baf
[NIFI-3943] align combo option item toolips to hover closer to the icon. This closes #1849 2017-05-24 11:14:48 -04:00
Pierre Villard ba49b8427c
NIFI-3191 - HDFS Processors Should Allow Choosing LZO Compression
This closes #1802.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-24 11:04:00 -04:00
Pierre Villard d63fa8a822 NIFI-3878 This closes #1790. AmbariReportingTask - Default run schedule set to 1 minute
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-24 00:09:17 -04:00
Andy LoPresto 6d07ebf50c NIFI-3954 This closes #1841. Upgraded version of httpclient library to handle TLS issues.
Added explicit dependency on httpcore for @ThreadSafe annotation.

Signed-off-by: joewitt <joewitt@apache.org>
2017-05-23 23:42:28 -04:00
Mark Payne 2c751a8e5b NIFI-3962: This closes #1846. Updated ConsumerLease to better handle batching of messages into a single FlowFile in the same way that it is handled for demarcated data 2017-05-23 23:20:13 -04:00
Matt Burgess fb925fc182 NIFI-3867: Fixed issue with getConnectionURL in HiveConnectionPool using Expression Language
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1847.
2017-05-23 21:28:59 +02:00
James Wing 9238fdb493 NIFI-3903 Unit Tests for AWS Processors
* Added non-integration unit tests for Lambda, S3, SNS, and SQS processors
* Moved non-integration tests out of integration test files
* Moved SQS integration tests to IT* files for clarification

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1814.
2017-05-23 19:25:55 +02:00
Joe Skora 0a7d149656 NIFI-3932 Symbols in API comments cause problems with Swagger Codegen
* Update ApiModelProperty descriptions to avoid problematic characters.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1821.
2017-05-23 19:04:54 +02:00
Matt Gilman 3ec0aa947f NIFI-3955:
- Filtering out the EventTime searchable field for all implementation of ProvenanceRepository. Prefer user query using the startDate/endDate fields.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1844.
2017-05-23 18:48:29 +02:00
James Wing b64232117f NIFI-3707 Sort Attributes in FlowFile Details UI
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1843.
2017-05-23 18:06:18 +02:00
Andrew Lim 49270f8a8c NIFI-3944 Update docs and screenshots for Component Versioning
Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #1840
2017-05-23 11:06:59 -04:00
Bryan Bende a8de27e69b NIFI-3732 Adding connect with timeout to StandardCommsSession and SSLCommsSession to avoid blocking
This closes #1842.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-05-23 15:43:27 +09:00
Matt Gilman ded396f0ef NIFI-3933:
- When monitoring heartbeats use the connected nodes as the basis for the check. This addresses the case when a node is terminated and no corresponding heartbeats exist.

This closes #1838.
2017-05-22 16:51:13 -04:00
Matt Gilman d33c4c72d4 NIFI-3933:
- When monitoring heartbeats use the connected nodes as the basis for the check. This addresses the case when a node is terminated and no corresponding heartbeats exist.
2017-05-22 16:50:30 -04:00
Mark Payne a1b07b1e9c
NIFI-3949: Updated Grok Reader to allow for sub-patterns to be used when determining the schema
This closes #1839.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-22 16:30:47 -04:00
Mark Payne 6937a6cf64 NIFI-3953: This closes #1837. Allow multiple schemas on same kafka topic/partition for ConsumeKafkaRecord_0_10
Also, updated record writers to ensure that they write the schema as appropriate if not using a RecordSet. Updated ConsumeKafkaRecord to allow for multiple schemas to be on same topic and partition

Signed-off-by: joewitt <joewitt@apache.org>
2017-05-22 14:37:53 -04:00
Mark Payne 6d16fdf170
NIFI-3952: Updated UpdateRecord to pass field-related variables to the Expression Language
This closes #1836.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-22 14:32:45 -04:00
Mark Payne 1a3c525dd6
NIFI-3951: Fixed bug that calculated the index incorrectly when filtering for ArrayIndexPath
This closes #1835.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-22 12:39:32 -04:00
Mark Payne 7f8987471d
NIFI-3946: Update LookupService to take a Map instead of a String for the input
NIFI-3946: Fixed issues where null values were returned instead of empty optionals

This closes #1833.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-22 11:52:41 -04:00
Bryan Bende 71cd497fef NIFI-3942 Added retry logic if a lookup fails due to InvalidDatabaseException which occurs if the underlying file was modified before we could refresh the reader
This closes #1831.
2017-05-22 09:12:51 -04:00
Bryan Bende f35e0ecdd0 NIFI-3942 Making IPLookupService reload the database file on the fly when detecting the file has changed 2017-05-22 09:06:42 -04:00
Mark Payne c49933f03d NIFI-3948: This closes #1834. Added flush() method to RecordWriter and call it when writing a single record to OutputStream for PublishKafkaRecord. Also removed no-longer-used class WriteAvroResult
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-19 23:05:04 -04:00
Tim Reardon 58ce52d5d6 NIFI-3854 This closes #1773. Expand expression language support for Kafka processors
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-19 15:51:23 -04:00
Matt Burgess 9294a26139
NIFI-3938: Added ScriptedLookupService, some refactor for reusable scripting classes
This closes #1828.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-05-19 11:20:38 -07:00
Bryan Bende d4f0c1d048 NIFI-3945 This closes #1829. Adding documentaion about security protocols to Kafka 0.10 processors
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-19 13:59:27 -04:00
Mark Payne ae9953db64 NIFI-3857: This closes #1825. Added PartitionRecord processor
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-19 02:08:52 -04:00
Mark Payne 9bd0246a96 NIFI-3863: Initial implementation of Lookup Services. Implemented LookupRecord processors. This required some refactoring of RecordSetWriter interface, so refactored that interface and all implementations and references of it 2017-05-19 01:02:41 -04:00
joewitt 3b98abbf41
NIFI-3940 upgraded to latest jruby which resolves an apparent lgpl bunding issue
This closes #1827.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-05-18 21:37:25 -07:00
Andrew Lim 28eadaa683
NIFI-3911 Improve Controller Services and Reporting Task Documentation
This closes #1826.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-05-18 19:20:39 -07:00
joewitt 762293c01c NIFI-3935 made the old persistent prov test more forgiving of timing issues and extended lucene test timeouts for slower environments. These should be integration tests really.
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1823
2017-05-18 14:42:46 -04:00
joewitt be67f3552b
NIFI-3853 fixing tests to properly encapsulate their environments
This closes #1822.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-18 13:36:02 -04:00
Koji Kawamura 36e7bd6164 NIFI-3894: This closes #1820. Fixed close and consume order with compression.
Before this fix, 'NullPointerException: Inflater has been closed' can be thrown as the Inflater is closed before input stream is consumed.

Also, calling close from AbstractTransaction.receive is removed, because the DataPacket is exposed as its return value and this class will not be able to know when to close the stream.

Signed-off-by: joewitt <joewitt@apache.org>
2017-05-17 22:57:33 -04:00