5c755c006b
NIFI-4017: Emit provenance event from Notify.
...
This closes #1890 .
Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-06-22 16:49:31 -04:00
f54e146561
NIFI-4085 Add exception in log message when (S)FTP listing fails
...
This closes #1929 .
Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-06-22 16:06:18 -04:00
5b82b735ab
NIFI-4106 Increase log level of user authentication rejection to warn.
...
This closes #1934 .
Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-06-22 14:03:25 -04:00
668a64cd56
NIFI-1586 Implements autopurging of transaction log and snapshot files
...
When NiFi is clustered, and autopurge.purgeInterval is greater than 1, the DatadirCleanupManager will be started in order to automatically purge transaction log and snapshot files based on the autopurge settings in zookeeper.properties
This closes #1928 .
2017-06-22 09:30:38 -04:00
c99100c934
NIFI-4010 Enables EL on Fetch/List/PutSFTP and List/Fetch/Put/DeleteHDFS processor properties
...
FetchSFTP/ListSFTP/PutSFTP: Private Key Path
ListHDFS/FetchHDFS/PutHDFS/DeleteHDFS: Hadoop Configuration Resources, Kerberos Principal, Kerberos Keytab, Kerberos Relogin Period
This closes #1148
This closes #1930 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2017-06-21 17:14:49 -04:00
59a32948ea
NIFI-2923 Added evaluation of attribute expressions for Kerberos principal and keytab
...
Signed-off-by: Bryan Bende <bbende@apache.org>
2017-06-21 17:14:28 -04:00
86fa1bba4f
NIFI-2923 Add expression language support to Kerberos parameters used by processors
...
Signed-off-by: Bryan Bende <bbende@apache.org>
2017-06-21 17:14:27 -04:00
73e601bc55
NIFI-4102: If first line read does not match Grok expression, skip line
...
Signed-off-by: Matt Burgess <mattyb149@apache.org>
This closes #1932
2017-06-21 15:10:22 -04:00
77e49b749e
NIFI-4075 Updating 'addRemoteProcessGroup' in StandardFlowSerializer to properly use ScheduledStateLookup. This closes #1922 .
2017-06-21 14:20:22 -04:00
aabd4a25d2
NIFI-4043 Initial commit of nifi-redis-bundle
...
NIFI-4061 Initial version of RedisStateProvider
- Adding PropertyContext and updating existing contexts to extend it
- Added embedded Redis for unit testing
- Added wrapped StateProvider with NAR ClassLoader in StandardStateManagerProvider
- Updating state-management.xml with config for RedisStateProvider
- Renaming tests that use RedisServer to be IT tests so they don't run all the time
This closes #1918 .
2017-06-21 12:47:19 -04:00
6bc6f955c0
NIFI-4059:
...
- Introducing the LdapUserGroupProvider.
- Updating documentation accordingly.
- Moving the IdentityMapping utilities so they were accessible.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #1923 .
2017-06-19 19:25:33 +02:00
ad6af1d941
NIFI-3994 Add Create Template to the context menu
...
Signed-off-by: James Wing <jvwing@gmail.com>
This closes #1883 .
2017-06-17 16:46:31 -07:00
5a7e6c6ac1
NIFI-4064 Make sure that Funnels with queued incoming FlowFiles, but no outgoing connections yield rather than continually check to run.
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #1914 .
2017-06-17 15:50:40 +02:00
2800df30d3
NIFI-2692
...
- Removing problematic timeout for SMTP Listen
- Converting anonymous method to lambda
- Adding debug and error logging
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #1924 .
2017-06-17 15:22:33 +02:00
490e1da5db
NIFI-4019:
...
- Adding support for X-Forwarded-* headers.
- Unrelated code clean up.
- Addressing additional proxying issues.
- Addressing mistyped landing page.
- Handled trailing slashes in proxy headers for context path
2017-06-15 14:17:35 -04:00
c0ad5bcc7b
NIFI-1921: Fixed bug in how average lineage duration was calculated for cluster stat
...
Signed-off-by: Matt Burgess <mattyb149@apache.org>
This closes #1912
2017-06-13 12:29:03 -04:00
2b47ade0c4
NIFI-4049 Fixing unit test that doesn't run on OSX and caused Travis failure
2017-06-12 16:36:23 -04:00
de1b84e2aa
NIFI-4049: Refactor AtomicDistributedMapCacheClient
...
To be used with cache engines that does not have revision number.
NIFI-4049: Refactor AtomicDistributedMapCacheClient
Removed old methods completely.
This closes #1904 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2017-06-12 10:34:48 -04:00
7761903352
NIFI-4041: Add file.size attribute to ListFTP and ListSFTP
...
This closes #1899 .
Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-06-12 10:53:09 +09:00
e7dcb6f6c5
NIFI-3921: Allow Record Writers to inherit schema from Record
...
Signed-off-by: Matt Burgess <mattyb149@apache.org>
This closes #1902
2017-06-09 16:13:25 -04:00
4ed7511bee
NIFI-3653: - Introducing UserGroup and Policy provider interfaces.
...
- Introducing FileUserGroupProvider and FileAccessPolicyProvider.
- Refactoring FileAuthorizer to utilize the file based implementations.
- Introducing the StandardManagedAuthorizer.
- Decorating the configured ManagedAuthorizer to ensure integrity checks are still performed.
- Loading user groups if possible to use during access decisions.
- Merging responses for requests for AccessPolicies, Users, and UserGroups.
- Adding unit tests as appropriate.
- Adding methods to the User, Group, and AccessPolicy builder that more easily supports generating UUIDs.
- Fixing typo when seeding policies during startup.
- Fixing type in documentation and error messages.
This closes #1897 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2017-06-09 13:54:10 -04:00
d8dc2bb00b
NIFI-4047: Addressed issue with inconsistent Lexical policy between used cached schemas and non-cached schemas
...
This closes #1907
Signed-off-by: Joe Percivall <JPercivall@apache.org>
2017-06-09 12:05:11 -04:00
cc741d2be6
NIFI-3997:
...
- Bumping to next minor version.
2017-06-08 15:22:51 -04:00
1bf0a1a849
Merge branch 'NIFI-3997-RC1'
2017-06-08 14:30:10 -04:00
b0c9428776
NIFI-4030 Populating default values on GenericRecord from Avro schema if not present in RecordSchema
...
This closes #1896 .
2017-06-07 13:59:40 -04:00
45e035686f
NIFI-4029: Allow null Avro default values in HortonworksSchemaRegistry
...
This closes #1894 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2017-06-07 12:03:53 -04:00
c86190c513
NIFI-4027: - Fixing the positioning of the tooltips in the component toolbar.
...
This closes #1893
Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
2017-06-06 17:35:43 -04:00
3ef0fa48da
NIFI-4014: Correcting Event Type text in S2S provenance reporting task
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #1887 .
2017-06-06 12:06:04 +02:00
e1880785d0
NIFI-4021: Fix StandardFlowSynchronizerSpec to pass independently.
...
This closes #1892 .
Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-06-05 19:25:07 -07:00
6ee12e9b47
NIFI-3997-RC1prepare for next development iteration
2017-06-05 11:07:43 -04:00
ddb73612bd
NIFI-3997-RC1prepare release nifi-1.3.0-RC1
2017-06-05 11:07:28 -04:00
082b99640e
NIFI-4001 - Bump ParCEFone version
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #1875 .
2017-06-03 12:11:38 +02:00
4d78052dd4
NIFI-1709 - Introduce logic to probe Linux version using /etc/os-release to nifi.sh
...
Add explicit paths to support SLES 11 SP4 / OpenSUSE init.d layout
This closes #1794
2017-06-03 08:12:52 +10:00
32314d70fd
NIFI-4009: Added support for several key functions in RecordPath
...
Signed-off-by: Matt Burgess <mattyb149@apache.org>
This closes #1881
2017-06-02 14:06:05 -04:00
0bddcfe730
NIFI-4002: Add PutElasticsearchHttpRecord processor
...
This closes #1878
Signed-off-by: Bryan Rosander <brosander@apache.org>
2017-06-02 13:01:13 -04:00
37be0b9820
NIFI-3990: This closes #1870 . Avoided creating garbage when we can avoid it in the JSON and csv readers and json writer
...
Signed-off-by: joewitt <joewitt@apache.org>
2017-06-02 10:01:40 -04:00
067e9dfeb0
NIFI-4003: Expose configuration option for cache size and duration NIFI-4003: Addressed remaining spots where client does not cache information
...
This closes #1879 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2017-06-01 15:31:35 -04:00
2595d816c4
NIFI-3986: This closes #1868 . When we encounted EOFException from sequential record reader event iterator, just treat it as not having any more events
...
Signed-off-by: joewitt <joewitt@apache.org>
2017-06-01 15:08:12 -04:00
7035694e37
NIFI-3995: Updated Hwx Encoded Schema Ref Writer to write 13 bytes for header instead of 14; added unit test to verify
...
This closes #1876 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2017-06-01 10:29:58 -04:00
a0b2311ff6
NIFI-3995: This closes #1873 . No longer use the 14th byte in the header for hwx content-encoded schema reference
...
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-31 13:41:25 -04:00
2b435cdfc6
NIFI-3985: This closes #1864 . Added 'Starting Position' property to SiteToSiteReportingTask; also added additionalDetails.html that explains the schema and updated the reporting task to stop publishing when the user clicks 'stops' instead of running indefinitely until the reporting task has caught up
...
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-26 21:05:45 -05:00
dd50745a9f
NIFI-3984 - upgraded version of schema-registry client to 0.2.1
...
This closes #1866 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-26 17:59:53 -04:00
80dfe0257f
ExtractGrok - Include exception message when compile fails
...
This closes : #1845
Signed-off-by: Andre F de Miranda <trixpan@users.noreply.github.com>
2017-05-27 00:56:25 +10:00
e85081ed20
NIFI-3979 Documenting how ListHDFS maintains state and performs listings
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #1860 .
2017-05-26 10:40:22 +02:00
5cb928131c
NIFI-3981: When serializing flow to cluster, use the Scheduled State of ports as they are configured to be, not the current state, since the current state may change as soon as the FlowController has finished initializing
...
This closes #1861 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-25 16:45:20 -04:00
13b59b5621
NIFI-3958: Decimal logical type with undefined precision and scale.
...
- Oracle NUMBER can return 0 precision and -127 or 0 scale with variable scale NUMBER such as ROWNUM or function result
- Added 'Default Decimal Precision' and 'Default Decimal Scale' property to ExecuteSQL and QueryDatabaseTable to apply default precision and scale if those are unknown
- Coerce BigDecimal scale to field schema logical type, so that BigDecimals having different scale can be written
Signed-off-by: Matt Burgess <mattyb149@apache.org>
This closes #1851
2017-05-25 14:37:17 -04:00
36911957dc
NIFI-3972: This closes #1855 . When enabling Controller Services on startup, wait until service is enabled (or until 30 seconds elapses, whichever comes first) before returning. This avoids having Service A depend on Service B and then attempting to start Service A before Service B is fully enabled
...
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-25 13:32:02 -04:00
eb25c8547a
NIFI-3972: Ensure that we wait until service state becomes enabled before triggering completable future that says that it's enabled
2017-05-25 13:31:53 -04:00
44fdc0e4ef
NIFI-3640 uri eventhub changes
...
This closes #1617
2017-05-25 13:14:51 -04:00
4d0667380a
NIFI-3404: Improved UX of LookupAttributes.
...
- Added dependency notice.
- Added EL evaluation at SimpleKeyValueLookupService.
- Updated documentation.
- Updated CommonsConfigurationLookupService to throw LookupFailureException if it fails to get configuration so that error messages can be displayed at each processor bulletin.
- Added calling getConfiguration at OnEnabled of CommonsConfigurationLookupService, so that the service will stay in Enabling state if there is any issue.
Signed-off-by: Joey Frazee <jfrazee@apache.org>
2017-05-25 10:05:32 -05:00
46e2420d74
NIFI-3404 Added LookupAttribute processor and lookup controller services
...
Signed-off-by: Joey Frazee <jfrazee@apache.org>
2017-05-25 10:05:32 -05:00
23cbc3b346
NIFI-3976: This closes #1857 . Fix ConsumePOP3 and ConsumeIMAP to delete messages.
...
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-25 10:59:53 -04:00
dc5e032368
NIFI-3791 - added back pressure data into S2SStatusReportingTask
...
This closes #1745 .
Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-05-25 18:20:11 +09:00
c07850aec3
NIFI-2702 - Deprecates nifi-standard-bundle/*JMS and document the recommended alternative
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #1643 .
2017-05-25 08:44:39 +02:00
e05005584d
NIFI-3809 - Added HTTP mode and HTTP proxy for S2S Reporting Tasks
...
This closes #1754 .
Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-05-25 15:31:49 +09:00
b6eb0ac0fb
NIFI-3859 - Provide filtering options in S2SProvenanceReportingTask
...
This closes #1777 .
Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-05-25 14:10:50 +09:00
eaefec6d81
NIFI-3971: This closes #1854 . Fixed bug in calculating content size that was transferred when cloning a relationship
...
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-24 22:14:56 -04:00
5aa3baca79
NIFI-3568: This closes #1577 . Use a cached thread pool in order to allow ThreadPoolRequestReplicator to scale up the number of threads to some configurable max
...
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-24 21:21:45 -04:00
86728bac7e
[NIFI-3925] for the jquery.combo plugin: update the calculation of the width of the combo box options overlay and also updated the plugins styles to now leverage the css calc() to determine the exact width of .combotext and .combo-option-text elements. This closes #1848
2017-05-24 16:15:42 -04:00
f97b3fe455
NIFI-3963: - Ensuring the RemoteGroupPort yields when the details cannot be refreshed from any of the configured remote instances.
...
This closes #1853 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-24 15:58:22 -04:00
ae3db82303
NIFI-3644 Fixing the result handler in HBase_1_1_2_ClientMapCacheService to use the offsets for the value bytes
...
This closes #1645 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-24 14:59:28 -04:00
152f002abf
Completed initial development of HBase_1_1_2_ClientMapCacheService.java which is compatible with DetectDuplicate (and other processors)
...
Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-24 14:59:23 -04:00
232380dbfd
NIFI-1452 on timer-driven yield, use the greater of yield duration or run schedule
...
This closes #1832 .
2017-05-24 14:24:29 -04:00
b12cf8a6d2
NIFI-3671: This closes #1852 . Ensure that we use the existing ResourceClaim (if it exists) when swapping data in, instead of creating a new one. Otherwise, if the ResourceClaim is still writable, then we may archive the data and then write to it, which can cause a NullPointerException in FileSystemRepository
...
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-24 14:21:18 -04:00
fb7d6d1150
NIFI-3799 Add Input Requirement Info to Docs
...
Added input requirement info for processor documentation
This closes #1756 .
Signed-off-by: James Wing <jvwing@gmail.com>
2017-05-24 10:45:45 -07:00
08b66b5b6a
NIFI-3969: Prevent merging flowfiles prematurely when all bins fill but some are already full and can be processed
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #1850 .
2017-05-24 19:36:18 +02:00
bb96b0f464
NIFI-3816 - Added correct mime.type to S2S bulletin and provenance reporting tasks
...
This closes #1758 .
Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-05-25 00:33:50 +09:00
7a46238baf
[NIFI-3943] align combo option item toolips to hover closer to the icon. This closes #1849
2017-05-24 11:14:48 -04:00
ba49b8427c
NIFI-3191 - HDFS Processors Should Allow Choosing LZO Compression
...
This closes #1802 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-24 11:04:00 -04:00
d63fa8a822
NIFI-3878 This closes #1790 . AmbariReportingTask - Default run schedule set to 1 minute
...
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-24 00:09:17 -04:00
6d07ebf50c
NIFI-3954 This closes #1841 . Upgraded version of httpclient library to handle TLS issues.
...
Added explicit dependency on httpcore for @ThreadSafe annotation.
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-23 23:42:28 -04:00
2c751a8e5b
NIFI-3962: This closes #1846 . Updated ConsumerLease to better handle batching of messages into a single FlowFile in the same way that it is handled for demarcated data
2017-05-23 23:20:13 -04:00
fb925fc182
NIFI-3867: Fixed issue with getConnectionURL in HiveConnectionPool using Expression Language
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #1847 .
2017-05-23 21:28:59 +02:00
9238fdb493
NIFI-3903 Unit Tests for AWS Processors
...
* Added non-integration unit tests for Lambda, S3, SNS, and SQS processors
* Moved non-integration tests out of integration test files
* Moved SQS integration tests to IT* files for clarification
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #1814 .
2017-05-23 19:25:55 +02:00
0a7d149656
NIFI-3932 Symbols in API comments cause problems with Swagger Codegen
...
* Update ApiModelProperty descriptions to avoid problematic characters.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #1821 .
2017-05-23 19:04:54 +02:00
3ec0aa947f
NIFI-3955:
...
- Filtering out the EventTime searchable field for all implementation of ProvenanceRepository. Prefer user query using the startDate/endDate fields.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #1844 .
2017-05-23 18:48:29 +02:00
b64232117f
NIFI-3707 Sort Attributes in FlowFile Details UI
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #1843 .
2017-05-23 18:06:18 +02:00
a8de27e69b
NIFI-3732 Adding connect with timeout to StandardCommsSession and SSLCommsSession to avoid blocking
...
This closes #1842 .
Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-05-23 15:43:27 +09:00
ded396f0ef
NIFI-3933:
...
- When monitoring heartbeats use the connected nodes as the basis for the check. This addresses the case when a node is terminated and no corresponding heartbeats exist.
This closes #1838 .
2017-05-22 16:51:13 -04:00
d33c4c72d4
NIFI-3933:
...
- When monitoring heartbeats use the connected nodes as the basis for the check. This addresses the case when a node is terminated and no corresponding heartbeats exist.
2017-05-22 16:50:30 -04:00
a1b07b1e9c
NIFI-3949: Updated Grok Reader to allow for sub-patterns to be used when determining the schema
...
This closes #1839 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-22 16:30:47 -04:00
6937a6cf64
NIFI-3953: This closes #1837 . Allow multiple schemas on same kafka topic/partition for ConsumeKafkaRecord_0_10
...
Also, updated record writers to ensure that they write the schema as appropriate if not using a RecordSet. Updated ConsumeKafkaRecord to allow for multiple schemas to be on same topic and partition
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-22 14:37:53 -04:00
6d16fdf170
NIFI-3952: Updated UpdateRecord to pass field-related variables to the Expression Language
...
This closes #1836 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-22 14:32:45 -04:00
7f8987471d
NIFI-3946: Update LookupService to take a Map instead of a String for the input
...
NIFI-3946: Fixed issues where null values were returned instead of empty optionals
This closes #1833 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-22 11:52:41 -04:00
71cd497fef
NIFI-3942 Added retry logic if a lookup fails due to InvalidDatabaseException which occurs if the underlying file was modified before we could refresh the reader
...
This closes #1831 .
2017-05-22 09:12:51 -04:00
f35e0ecdd0
NIFI-3942 Making IPLookupService reload the database file on the fly when detecting the file has changed
2017-05-22 09:06:42 -04:00
c49933f03d
NIFI-3948: This closes #1834 . Added flush() method to RecordWriter and call it when writing a single record to OutputStream for PublishKafkaRecord. Also removed no-longer-used class WriteAvroResult
...
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-19 23:05:04 -04:00
58ce52d5d6
NIFI-3854 This closes #1773 . Expand expression language support for Kafka processors
...
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-19 15:51:23 -04:00
9294a26139
NIFI-3938: Added ScriptedLookupService, some refactor for reusable scripting classes
...
This closes #1828 .
Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-05-19 11:20:38 -07:00
d4f0c1d048
NIFI-3945 This closes #1829 . Adding documentaion about security protocols to Kafka 0.10 processors
...
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-19 13:59:27 -04:00
ae9953db64
NIFI-3857: This closes #1825 . Added PartitionRecord processor
...
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-19 02:08:52 -04:00
9bd0246a96
NIFI-3863: Initial implementation of Lookup Services. Implemented LookupRecord processors. This required some refactoring of RecordSetWriter interface, so refactored that interface and all implementations and references of it
2017-05-19 01:02:41 -04:00
3b98abbf41
NIFI-3940 upgraded to latest jruby which resolves an apparent lgpl bunding issue
...
This closes #1827 .
Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-05-18 21:37:25 -07:00
762293c01c
NIFI-3935 made the old persistent prov test more forgiving of timing issues and extended lucene test timeouts for slower environments. These should be integration tests really.
...
Signed-off-by: Matt Burgess <mattyb149@apache.org>
This closes #1823
2017-05-18 14:42:46 -04:00
be67f3552b
NIFI-3853 fixing tests to properly encapsulate their environments
...
This closes #1822 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-18 13:36:02 -04:00
36e7bd6164
NIFI-3894: This closes #1820 . Fixed close and consume order with compression.
...
Before this fix, 'NullPointerException: Inflater has been closed' can be thrown as the Inflater is closed before input stream is consumed.
Also, calling close from AbstractTransaction.receive is removed, because the DataPacket is exposed as its return value and this class will not be able to know when to close the stream.
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-17 22:57:33 -04:00
8e1c79eaaf
NIFI-1963 Allows a node reconnecting to the cluster to inherit non-fingerprinted processor settings
...
Forces a node reconnecting to a cluster to serialize the updated flow to disk
Added most processor settings to the flow fingerprint (excluding name, style, comment, position, and schedule state)
Updated some test data for FingerprintFactoryTest to test for new fields added to the flow fingerprint
Updated StandardProcessorNode to allow processor comments and name to be settable while a processor is running
Updated StandardFlowSynchronizer to inherit non-fingerprinted processor settings (name, style, comment, and position) when flow is already synchronized
This closes #1812
2017-05-17 17:35:33 -04:00
289dde098e
NIFI-3896 - Makes DeprecationNotice more intuitive. This closes #1799
...
Update developers guide on how to deprecate a component
2017-05-17 16:37:11 -04:00
5cf4bf61b3
NIFI-3917: Use a default value of 'false' for CSV Reader's Skip Header Line property. This closes #1819
2017-05-17 13:31:53 -04:00
3fd7d3b272
NIFI-3873: Improve Avro logical types readability
...
Signed-off-by: Matt Burgess <mattyb149@apache.org>
This closes #1788
2017-05-17 13:08:00 -04:00
2dde480029
[NIFI-3906] Introduce additional validation when handling user-supplied input to ensure appropriate values and output encoding within SlickGrid tables. This closes #1818
2017-05-17 13:04:31 -04:00
9198bee038
NIFI-3923: Only start processors after repositories have been initialized. This closes #1817
2017-05-17 11:12:59 -04:00
40a9cd4f2e
NIFI-3919: Let AvroTypeUtil try every possible type
...
Before this fix, AvroTypeUtil can throw an Exception before trying every possible data types defined within a union field.
This closes #1816 .
2017-05-17 10:53:27 -04:00
f019d509f3
NIFI-3918: Added Choice mapping to JsonTreeRowRecordReader.
2017-05-17 09:49:11 -04:00
33dc3e36fb
NIFI-3920: Remove unnecessary code from AvroTypeUtil
...
- Removed remaining duplicate lines of code left by NIFI-3861 refactoring.
- Added test case that writes Avro record having union field.
This closes #1813 .
2017-05-17 09:45:21 -04:00
c50d516076
NIFI-3848:
...
- Allowing the revision check to be overridden when the state of a nodes connection to the cluster changes.
This closes #1811
2017-05-16 17:09:48 -04:00
6b71b4cbb8
NIFI-3868: - Ensuring we do not attempt to group bulletins that lack permissions. - Only group bulletins when all nodes report the same message. - Retain the most recent bulletin.
...
Signed-off-by: Matt Burgess <mattyb149@apache.org>
This closes #1801
2017-05-16 16:54:30 -04:00
ce1bc42ac5
NIFI-3912: This closes #1809 . Fixed NPE that would result in validation failure for FreeFormTextRecordSetWriter
...
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-16 16:17:04 -04:00
2f3ba57dd6
NIFI-3914: Do not validate that remote group ports can start transmitting if there is no connection
...
This closes #1810 .
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2017-05-16 16:10:00 -04:00
3a0004a665
NIFI-3904 Adding logic to only reload when incoming bundle is different and to use additional URLs from spec component
...
NIFI-3908 Changing UI to submit filterType for CSs and filter for processors and reporting tasks
2017-05-16 16:08:28 -04:00
494a0e8928
NIFI-3900: Ensure that when we serialize a flow to send to Cluster Coordinator that we include the Scheduled State for processors as they are intended to be, not as they are currently because during startup they may not have been started yet.
...
NIFI-3900: Rebased against master and updated new unit test to use new method signature for FlowSerializer.serialize
This closes #1804 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-16 16:04:09 -04:00
ac8e57259f
NIFI-3907 Added response headers.
...
Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-05-16 11:55:36 -07:00
fb94990e60
NIFI-3909: This closes #1806 . If we have a FlowFile with 0 records, ensure that PublishKafkaRecord_0_10 handles the flowfile properly
...
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-16 14:42:42 -04:00
f5f6cab646
NIFI-3906 Added formatting to tables.
...
Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-05-16 11:17:29 -07:00
77a676bf92
NIFI-3894: Call Inflater/Deflater.end to free up memory
...
This closes #1796 .
2017-05-16 13:49:31 -04:00
4fdea680ec
NIFI-3905: This closes #1805 . When a Provenance Query is submitted to WriteAheadProvenanceRepository, purge any obsolete queries from the internal state before rejecting the query due to 'too many outstanding queries'
...
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-16 13:48:16 -04:00
58cf15a912
NIFI-2199 NIFI-3112 Pipe stdout to /dev/null and allow stderr to propagate through.
...
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>
This closes #1786
2017-05-16 13:31:15 -04:00
3f4b276b71
NIFI-3910: Fixed issue where CSVRecordReader throws an Exception if a field is missing instead of using a null value
...
This closes #1807 .
2017-05-16 13:16:46 -04:00
824712bffe
NIFI-3901:
...
- Addressing response code of POST /provenance/lineage.
- Ensuring cluster node details are set accordingly.
This closes #1803 .
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2017-05-16 12:36:27 -04:00
603f713a40
NIFI-3860: Relax the constraint that the ProcessSession must always be given the most up-to-date version of a FlowFile
...
This closes #1778 .
2017-05-16 08:51:41 -04:00
6ffb78d404
NIFI-3853:
...
- Filtering out certain control characters and unpaired Unicode surrogate codepoints prior to saving the flow.xml.
This closes #1784 .
Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-05-15 19:02:03 -07:00
bf15502e19
NIFI-3763 Add new processor to log user defined messages built with NiFi Expression Language
...
Signed-off-by: Matt Burgess <mattyb149@apache.org>
This closes #1737
2017-05-15 14:18:57 -04:00
1811ba5681
NIFI-2624: Avro logical types for ExecuteSQL and QueryDatabaseTable
...
- Added Logical type support for DECIMAL/NUMBER, DATE, TIME and TIMESTAMP column types.
- Added Logical type 'decimal' to AvroReader so that Avro records with logical types written by ExecuteSQL and QueryDatabaseTable can be consumed by AvroReader.
- Added JdbcCommon.AvroConversionOptions to consolidate conversion options.
- Added 'Use Avro Logical Types' property to ExecuteSQL and QueryDatabaseTable to toggle whether to use Logical types.
- Added 'mime.type' FlowFile attribute as 'application/avro-binary' so that output FlowFiles can be displayed by content viewer.
Signed-off-by: Matt Burgess <mattyb149@apache.org>
This closes #1798
2017-05-15 14:15:23 -04:00
20a1fc24d7
NIFI-3885 DynamoDB Processor EL Support
...
Add EL support to remaining Dynamo processor properties
Signed-off-by: James Wing <jvwing@gmail.com>
This closes #1793 .
2017-05-15 10:13:53 -07:00
196ca237e6
NIFI-3855 Support PutSlack attachments as dynamic properties
...
NIFI-3855 Refactor deprecated API
NIFI-3855 Log separate error messages in PutSlack
Signed-off-by: Matt Burgess <mattyb149@apache.org>
This closes #1774
2017-05-15 09:08:43 -04:00
d092551211
NIFI-3761
...
- adjust testFullyDocumentedProcessor to correctly eval to false when needed
- Introduce the ability to deprecate a component
- Allow documentation to inform user about deprecation of a component
- This closes #1718
NIFI-391 - Add set/getDeprecationReson to DocumentedDTO and use it within DtoFactory'
2017-05-15 08:43:32 -04:00
3353865ce9
NIFI-3867: Add Expression Language support to HiveConnectionPool properties
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #1783 .
2017-05-15 13:46:29 +02:00
bc68eb754f
NIFI-3895: Fixed AvroReader to use Schema Access Strategy default value properly.
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #1797 .
2017-05-15 13:27:26 +02:00
09f6a7040c
[NIFI-3830] force browsers to use URLSearchParams polyfill do to bugs and inconsistent browser implementations. This closes #1785
2017-05-13 07:34:33 -04:00
382eef2183
NIFI-3879: Allow null Avro default values
...
- Avro uses their own class for null values, so a standard check against
null isn't picking them up.
This closes #1792 .
2017-05-12 14:58:43 -04:00
e9848f4276
NIFI-3881 Fix PutHiveStreaming EL evaluation
...
Signed-off-by: Matt Burgess <mattyb149@apache.org>
This closes #1791
2017-05-12 14:06:32 -04:00
afd2b04afd
NIFI-3852 Add expression language support to Cassandra processors
...
Remove unused import
Signed-off-by: Matt Burgess <mattyb149@apache.org>
This closes #1770
2017-05-12 13:52:07 -04:00
b1901d5fe0
NIFI-3838: Initial implementation of RecordPath and UpdateRecord processor
...
NIFI-3838: Updated version from 1.2.0-SNAPSHOT to 1.3.0-SNAPSHOT; removed unneeded value from AttributeExpression.ResultType enum
NIFI-3838: Addressed PR Review feedback
NIFI-3838: Allow for schemas to be merged together for a record; refactored RecordSetWriterFactory so that there is a method to obtain the schema and then the writer is created with that schema. Added additional unit tests
NIFI-3838: Addressed problems with documentation based on PR Review
NIFI-3838: Fixed checkstyle violation
NIFI-3838: Addressed issue of comparing different types of Number objects
Signed-off-by: Matt Burgess <mattyb149@apache.org>
This closes #1772
2017-05-12 12:36:52 -04:00
1c58e78ce9
[NIFI-3876] trim whitespace processor names before applying ellipsis
...
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes #1789 .
2017-05-12 16:01:43 +02:00
dd0306cce7
NIFI-3837: - Being more specific regarding the escaping of back references in evaluated expressions.
...
Signed-off-by: Matt Burgess <mattyb149@apache.org>
fixed typo in comment
This closes #1781
2017-05-11 22:16:46 -04:00
0b0ac196ea
NIFI-3861: AvroTypeUtil used different constant.
...
Previous fix #1779 refactored the way to check Logical type to use string constants.
One of those refactoring used wrong constant mistakenly in normalizeValue method.
Fortunately, this defect is harmless since even though normalizeValue did not convert int to Time, DataTypeUtils.convertType does the same conversion.
Signed-off-by: Matt Burgess <mattyb149@apache.org>
This closes #1782
2017-05-11 21:31:07 -04:00
6e4db6b11a
NIFI-3871: Convert Avro map values
...
This closes #1787 .
- When converting from a raw value to an Avro object, convert the values
of any Avro map types so that they can be complex types like other
records.
2017-05-11 20:25:19 -04:00
cf497c9cf9
NIFI-3865: - Obtaining the appropriate input stream after accessing content. The stream was being consumed into a buffer and was not longer available.
...
This closes #1780 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-10 15:51:00 -04:00
dc44e30698
NIFI-3856 PutCloudWatchMetric Stats, Dimensions
...
PutCloudWatchMetric support for StatisticSets and Dimensions
Signed-off-by: James Wing <jvwing@gmail.com>
This closes #1775 .
2017-05-10 12:07:04 -07:00
361a58c531
NIFI-3851 Support EL in AWS Endpoint Override URL
...
Add expression language support to AWS Endpoint Override URL property
Signed-off-by: James Wing <jvwing@gmail.com>
This closes #1769 .
2017-05-10 10:37:39 -07:00
72de1cbdef
NIFI-3861: Fixed AvroReader nullable logical types issue
...
- AvroReader did not convert logical types if those are defined with union
- Consolidated createSchema method in AvroSchemaRegistry and AvroTypeUtil as both has identical implementation and mai
ntaining both would be error-prone
This closes #1779 .
2017-05-10 10:08:22 -04:00
744ecc3d83
NIFI-3832: Fixed AvroRecordSetWriter validation NPE.
...
This closes #1768 .
2017-05-09 14:11:43 -04:00
3af53419af
NIFI-3770-RC2 prepare for next development iteration
2017-05-05 20:50:28 -04:00
3a605af8e0
NIFI-3770-RC2 prepare release nifi-1.2.0-RC2
2017-05-05 20:50:14 -04:00
f4f174b899
NIFI-3826 added proper NOTICE entries for apache calcite and maxmind source
...
This closes #1766 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-05 20:00:22 -04:00
829c02b058
NIFI-3825 fix L&N for jolt, json, and source deps
...
Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
This closes #1765
2017-05-05 18:43:04 -04:00
5f65b2561a
NIFI-3600 Improve logging and relationship routing for failures in DeleteHDFS
...
realized that the session should be cloned here because its inside a for loop and the original flow file would be transferred but not be the latest flow file if an error occurred in the for loop
@trixpan at a high level what do you think about this approach?
NIFI-3600: Added unit test
NIFI-3600: Removed the hdfs.error.code attribute
Signed-off-by: Matt Burgess <mattyb149@apache.org>
This closes #1595
2017-05-05 15:25:08 -04:00
f0dfcc180d
NIFI-3808:
...
- Making the selector for the enable/disable controller service state more specific. Previously name was too general and was affect other parts of the application.
Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
This closes #1759
2017-05-05 15:19:28 -04:00
ee8dc7640f
NIFI-3820 added calcite to assembly notice and updated all copyright years for compress, avro, and jetty
...
This closes #1764 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-05 14:45:40 -04:00
12249e5a08
NIFI-2268: Instead of allowing HandleHttpRequest to run constantly calling BlockingQueue.poll() as fast as possible, call BlockingQueue.poll(2, TimeUnit.MILLISECONDS) to avoid overutilization of CPU when there is no work to be done
...
This closes #1762 .
Signed-off-by: Bryan Bende <bbende@apache.org>
2017-05-05 13:58:10 -04:00
Koji Kawamura
Pierre Villard
Niels Zeilemaker
Jeff Storck
Maurizio Colleluori
Mark Payne
Joe Percivall
Bryan Bende
Matt Gilman
Scott Aslan
Wesley-Lawrence
Chris Herrera
Steve Champagne
Joe Gresock
Andre F de Miranda
Matt Burgess
Yolanda M. Davis
Joseph Niemiec
Joey Frazee
baolsen
Mike Moser
Andy LoPresto
James Wing
Joe Skora
Tim Reardon
joewitt
Aldrin Piri
Peter G. Horvath
Jeremy Dyer