Commit Graph

163 Commits

Author SHA1 Message Date
Mark Payne 7c1ce17223
NIFI-4756: Updated PublishKafkaRecord processors to include attributes generated from schema write strategy into the message headers when appropriate
This closes #2396.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-01-16 16:31:34 -05:00
joewitt 41ce788812 NIFI-4751 changed to next minor release version snapshot 2018-01-12 15:15:32 -05:00
joewitt 36405e888c NIFI-4751-RC1 prepare for next development iteration 2018-01-08 23:39:49 -07:00
joewitt 46d30c7e92 NIFI-4751-RC1 prepare release nifi-1.5.0-RC1 2018-01-08 23:39:32 -07:00
joewitt 6ee6b5e57c NIFI-4734 refactored couchbase bundle, jms bundle, removed jms test-lib usage, fixed broken SeeAlso and service links causing docs warns. Corrected L&N in each bundle. This closes #2369.
Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-01-04 09:16:17 -05:00
Jeff Storck 42a1ee011b NIFI-4323 This closes #2360. Wrapped Get/ListHDFS hadoop operations in ugi.doAs calls
NIFI-3472 NIFI-4350 Removed explicit relogin code from HDFS/Hive/HBase components and updated SecurityUtils.loginKerberos to use UGI.loginUserFromKeytab. This brings those components in line with daemon-process-style usage, made possible by NiFi's InstanceClassloader isolation.  Relogin (on ticket expiry/connection failure) can now be properly handled by hadoop-client code implicitly.
NIFI-3472 Added default value (true) for javax.security.auth.useSubjectCredsOnly to bootstrap.conf
NIFI-3472 Added javadoc explaining the removal of explicit relogin threads and usage of UGI.loginUserFromKeytab
Readded Relogin Period property to AbstractHadoopProcessor, and updated its documentation to indicate that it is now a deprecated property
Additional cleanup of code that referenced relogin periods
Marked KerberosTicketRenewer is deprecated

NIFI-3472 Cleaned up imports in TestPutHiveStreaming
2018-01-03 11:31:47 -05:00
Mark Payne c91d99884a NIFI-4717: Several minor bug fixes and performance improvements around record-oriented processors
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2359
2017-12-29 10:43:21 -05:00
Mike Moser c59a967623 NIFI-4504, NIFI-4505 added removeAndGet, removeByPatternAndGet, and keySet methods to MapCache API
cleaned up some warnings on deprecated nifi.stream.io classes

This closes #2284.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-12-29 00:23:09 +09:00
Matthew Burgess 14d2291db8 NIFI-4496: Added JacksonCSVRecordReader to allow choice of CSV parser. This closes #2245. 2017-12-22 08:56:29 -05:00
Mark Payne 406db4867a NIFI-4702: When we check the next line for matches in Grok Reader, store the Map that is returned so that we don't have to re-evaluate the regexes the next time that nextRecord() is called
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2349
2017-12-18 09:41:05 -05:00
Mike Thomsen d7347a2dc3 NIFI-4644 Fixed LookupService API to allow for more than String/String lookup pairs. This closes #2304. 2017-12-13 16:04:23 -05:00
Mike Thomsen bfe92b9000 NIFI-3970 added changes from a code review.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2334
2017-12-11 19:16:34 -05:00
Matthew Burgess 63840377dd NIFI-3970: Add CSVRecordLookupService 2017-12-11 19:14:27 -05:00
Mark Payne f772f2f093 NIFI-4671: This closes #2328. Ensure that Avro Schemas that are created properly denote fields as being nullable iff the schemas says they are, for non-top-level fields
Signed-off-by: joewitt <joewitt@apache.org>
2017-12-11 11:46:15 -05:00
Mark Payne c138987bb4 NIFI-4656, NIFI-4680: This closes #2330. Fix error handling in consume/publish kafka processors. Address issue with HortonworksSchemaRegistry throwing RuntimeException when it should be IOException. Fixed bug in ConsumeerLease/ConsumKafkaRecord that caused it to report too many records received
Signed-off-by: joewitt <joewitt@apache.org>
2017-12-08 16:01:14 -05:00
Matthew Burgess 16a23f5a0f NIFI-4589: Allow multiple keys in FetchDistributedMapCache, add subMap operation to API
Signed-off-by: Mike Moser <mosermw@apache.org>

This closes #2260.
2017-11-21 20:13:35 +00:00
Matt Gilman 6b75921725 NIFI-4617:
- Ensuring a charset is specified when convert a string into bytes.
- Ensuring the aggregate snapshot is non null before attempting to promote the runStatus.

This closes #2278.
2017-11-20 09:39:23 -05:00
Matt Gilman 2da72f1524
NIFI-4593: - Ensuring the necessary jackson dependencies are bundled.
This closes #2263.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-11-16 15:44:37 -05:00
Koji Kawamura 116c846342 NIFI-4543: Improve HBase processors provenance transit URL
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2237.
2017-11-02 18:24:40 +01:00
patricker e3482cc772 NIFI-4534 Choose Character Set for CSV Record Read/Write streams
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2229.
2017-10-27 10:34:15 +02:00
Mark Payne 95e86c1f51 NIFI-4495: Expose the option for supressing null values on the JSON Writer
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2218.
2017-10-19 10:20:40 +02:00
patricker fd00df3d2f NIFI-4465 ConvertExcelToCSV Data Formatting and Delimiters
This closes #2194.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-10-17 14:56:49 +09:00
Matt Gilman 6baea8ccff
NIFI-4444:
- Upgrading to Jersey 2.x.
- Updating NOTICE files where necessary.
- Fixing checkstyle issues.

This closes #2206.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-10-12 10:27:02 -07:00
Bryan Bende 5930c0c212
NIFI-4346 Modifying HBase_1_1_2_LookupService to use HBase_1_1_2_ClientService, instead of extend it
This closes #2125.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-10-07 15:36:49 -04:00
Mike Thomsen eb97a68110
NIFI-4346 Created a LookupService that uses HBase as its back end.
Signed-off-by: Bryan Bende <bbende@apache.org>
2017-10-07 15:36:44 -04:00
joewitt 1d5df4a5ec NIFI-4466 fixed incorrect license header back to ASF and addressed some whitespace formatting in a couple of those poms. This closes #2197. 2017-10-06 11:15:59 -04:00
Mark Owens 701597775f NIFI-1547: DistributedMapCacheServer: Ambiguous error on
misconfiguration

Updated PersistentMapCache constructor in PersistentMapCache.java to
catch an OverlappingFileLockException and present a more useful error
message before propagating the exception forward. The log message alerts
user to possible duplicated persistencePath in call to
PersistentMapCache.

Created a test method to verify the exception is thrown as expected.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2192
2017-10-05 16:44:43 -04:00
Andy LoPresto d4168f5ff1
NIFI-4297
- Upgraded immediately actionable dependency versions from Meterian report.
- Upgraded jackson-core test dependencies for HBase and Elasticsearch modules.
- Only 3 instances of jackson-core < 2.8.6 (Google Cloud Platform and Spark Receiver modules).
- Upgraded version of poi dependency in nifi-email-processors to 3.16.
- Resolving dependency issues after rebasing against 1.5.0-SNAPSHOT.
- Removed jackson-databind from <dependencyManagement> block in nifi/pom.xml and added explicit reference to ${jackson.version} in all referenced artifacts.
- Removed jackson-mapper-asl from <dependencyManagement> block in nifi/pom.xml and added explicit reference to ${jackson.old.version} in all referenced artifacts.
- Removed Jasypt from <dependencyManagement> and added explicit version in test dependency for legacy compatibility.
- This closes #2084
2017-10-05 15:23:52 -04:00
Jeff Storck a57911d3db NIFI-4412-RC2 prepare for next development iteration 2017-09-28 13:45:36 -04:00
Jeff Storck e6508ba7d3 NIFI-4412-RC2 prepare release nifi-1.4.0-RC2 2017-09-28 13:45:21 -04:00
Arun Manivannan 3b1b326b13 NIFI-4416 CSVRecordReader does not accept escaped character as delimiter
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2172
2017-09-25 22:32:15 -04:00
Matthew Burgess 1ca4fa3a83
NIFI-4345: Removed lookup-services dependency from MongoDB Services NAR, cleaned up deps
- Added nifi-lookup-service-api to nifi-nar-bundles POM

This closes #2174.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-09-25 14:52:33 -04:00
Mike Thomsen 9a8e6b2eb1 NIFI-4345 Added a MongoDB controller service and a lookup service.
NIFI-4345: Changed Lookup Key to Lookup Value Field
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2123
2017-09-22 13:33:07 -04:00
Mark Payne b452f8c251 NIFI-4378: Allow suppression of null values in JSON
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2148.
2017-09-12 21:41:59 +02:00
Wesley-Lawrence e203358bf3 NIFI-4258 Corrected a bug with the 'Informix Unload Escape Disabled' CSV format. This closes #2090. 2017-09-08 16:17:38 -04:00
Pierre Villard 6559604456 NIFI-4081 - Added raw message option in GrokReader
This closes #1921.
2017-09-08 12:44:37 -04:00
Koji Kawamura 1f67cbf628 NIFI-4004: Use RecordReaderFactory without FlowFile.
- Removed FlowFile from RecordReaderFactory, RecordSetWriterFactory and SchemaAccessStrategy.
- Renamed variable 'allowableValue' to 'strategy' to represent its meaning better.
- Removed creation of temporal FlowFile to resolve Record Schema from ConsumerLease.

- Removed unnecessary 'InputStream content' argument from
  RecordSetWriterFactory.getSchema method.

This closes #1877.
2017-09-08 12:37:40 -04:00
m-hogue 03e51ee8ac
NIFI-4335: Changed SSLContextService implementations to RestrictedSSLContextService for all Listen* processors
This closes #2131.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-09-07 17:34:00 -07:00
m-hogue d28e61c5dd
NIFI-2528 Added RestrictedSSLContextService interface with implementation. Changed ListenHTTP to allow only new StandardRestrictedSSLContextService.
This closes #1986.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-08-30 20:17:01 -07:00
Bryan Bende 60d4672195 NIFI-4304 Extracting HWX Schema Registry client version to a property and bumping to latest 0.3.0 release
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2096
2017-08-17 16:07:03 -04:00
Mark Payne 451f9cf124 NIFI-4142: This closes #2015. Refactored Record Reader/Writer to allow for reading/writing "raw records". Implemented ValidateRecord. Updated Record Reader to take two parameters for nextRecord: (boolean coerceTypes) and (boolean dropUnknownFields)
Signed-off-by: joewitt <joewitt@apache.org>
2017-08-11 22:01:46 -07:00
Bryan Bende f7da7e67f4 NIFI-4275 Adding support for specifying the timestamp on PutHBase processors
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2070.
2017-08-11 18:04:13 +02:00
Andy LoPresto 5150dff70b NIFI-4274 Removed EL evaluation logic from custom file validator.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2071.
2017-08-10 10:05:16 +02:00
Mike Thomsen 496a32e12c
NIFI-4024 Added org.apache.nifi.hbase.PutHBaseRecord
Signed-off-by: Bryan Bende <bbende@apache.org>
2017-08-04 14:54:01 -04:00
Pierre Villard 45f82dc855 NIFI-4111 - NiFi shutdown
Fixed threads shutdown so that NiFi can shutdown gracefully

NIFI-4111 - Review - Handling SocketRemoteSiteListener (RAW S2S)

This closes #1963.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-07-07 14:23:18 +09:00
Mark Payne 3906d4e1d2 NIFI-1763: Initial implementation of ConfluentSchemaRegistry.
NIFI-1763: Fixed bug where the Confluent Schema Registry Schema Access Writer was not being created

Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #1938
2017-07-06 15:52:07 -04:00
Steve Champagne 68b42c9e54 NIFI-4055: Add a compression option to AvroRecordSetWriter
This closes #1909.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
2017-06-26 19:16:31 +02:00
Mark Payne 8878d732dd NIFI-4103: Return an empty Optional if an IP address cannot be found
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1933
2017-06-23 11:14:57 -04:00
Maurizio Colleluori 59a32948ea
NIFI-2923 Added evaluation of attribute expressions for Kerberos principal and keytab
Signed-off-by: Bryan Bende <bbende@apache.org>
2017-06-21 17:14:28 -04:00
Mark Payne 73e601bc55 NIFI-4102: If first line read does not match Grok expression, skip line
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1932
2017-06-21 15:10:22 -04:00