Commit Graph

4060 Commits

Author SHA1 Message Date
Matt Gilman d47bbd12ce
NIFI-4413: - Using the incoming request to determine the appropriate scheme for the data reference URI. Necessary for cases where the NiFi instance is behind a proxy running a different scheme.
This closes #2179.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-09-27 09:51:06 -04:00
m-hogue b7b6d9082c
NIFI-2529: Added support for SSLContextService protocols in HandleHttpRequest
Forced HandleHTTPRequest to use RestrictedSSLContextService and removed extraneous SSL algorithm checks
Throw RuntimeException if the chosen SSL protocol isn't supported by HandleHttpRequest

This closes #1985.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-09-26 18:48:19 -07:00
Andy LoPresto 2c1f5b49e4 NIFI-3116 This closes #2108. Added initial regression test for StringEncryptor to ensure continued functionality during removal of Jasypt.
Added external compatibility regression test for StringEncryptor to ensure continued functionality during removal of Jasypt.
Documents custom salt lengths and iteration counts for each encryption method.
Added (ignored) failing tests for keyed encryption (Jasypt does not support keyed encryption).
Changed StringEncryptor to non-final class and added protected default constructor.
Added failing test for initialization status.
Added utility methods in CipherUtility.
Moved PBE cipher providers (and tests) from nifi-standard-processors to nifi-security-utils module.
Implemented PBE and keyed encryption/decryption logic.
Moved Scrypt unit test back into scrypt package.
Resolved test failures in limited strength cryptographic environment.
Implemented keyed encryption/decryption and enabled unit tests.
Removed Jasypt dependency from production scope (kept in test scope for backward compatibility tests).

Signed-off-by: joewitt <joewitt@apache.org>
2017-09-26 10:55:29 -04:00
Arun Manivannan 3b1b326b13 NIFI-4416 CSVRecordReader does not accept escaped character as delimiter
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2172
2017-09-25 22:32:15 -04:00
Matthew Burgess 1ca4fa3a83
NIFI-4345: Removed lookup-services dependency from MongoDB Services NAR, cleaned up deps
- Added nifi-lookup-service-api to nifi-nar-bundles POM

This closes #2174.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-09-25 14:52:33 -04:00
Bryan Bende 6eab91923e NIFI-4418 Adding ListenUDPRecord processor. This closes #2173. 2017-09-25 13:19:23 -04:00
Koji Kawamura c6f4421c8e NIFI-2829: Fixed PutSQL time unit test.
The unit test for DATE type used GMT timezone, that causes an assertion error in timezones such as EST (-5).
We need to use local timezone instead of GMT, as Derby and PutSQL uses local timezone.

The unit test failed before as follows:
- Unit test code, passed: '2002-02-02 GMT'
- PutSQL code convertedi it to local: '2002-02-01 EST', and stored as '2002-02-01' in Derby database without timezone info
- Unit test code SELECT the inserted value, passed a GMT calender, then got epoch timestamp, which was '2002-01-31'

Support negative long value for timezones ahead of UTC.

- For timezones such as '+0800', it's possible that a local time e.g. '02:03:04' can be a negative epoch value. This commit changes LONG_PATTERN so that it can accept nevative values.
- Changed time values in unit tests to verify negative epoch values, and avoid using the same digits among different time unit for better readability.

This closes #2082
2017-09-23 17:41:42 -04:00
Mark Payne 3fb704c58f NIFI-4201: This closes #2024. Implementation of processors for interacting with Kafka 0.11
Signed-off-by: joewitt <joewitt@apache.org>
2017-09-22 22:08:19 -04:00
Deon Huang a29348f2a4 NIFI-4395 - GenerateTableFetch can't fetch column type by state after instance reboot
NIFI-4395: Updated unit test for GenerateTableFetch
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2166
2017-09-22 14:34:00 -04:00
Andy LoPresto 9e2c7be7d3
NIFI-4353
- Added XmlUtils class.
- Added unit test.
- Added XXE test resource.
- Refactored JAXB unmarshalling globally to prevent XXE attacks.
- Refactored duplicated/legacy code.
- Cleaned up commented code.
- Switched from FileInputStream back to StreamSource in AuthorizerFactoryBean.
- This closes #2134
2017-09-22 14:31:38 -04:00
Mike Thomsen 9a8e6b2eb1 NIFI-4345 Added a MongoDB controller service and a lookup service.
NIFI-4345: Changed Lookup Key to Lookup Value Field
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2123
2017-09-22 13:33:07 -04:00
Mark Payne 05b5dd1488 NIFI-4409: Ensure that when record schema is inherited, the schema from teh ResultSet is used instead of the schema from the RecordReader because the schema from the RecordReader mmay not match the actual data
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2169
2017-09-22 12:41:17 -04:00
Pierre Villard 50d018566d
NIFI-4403 - add group name to bulletins model. This closes #2167 2017-09-22 11:53:58 -04:00
cam 0c0c33411d NIFI-4384 - Enhance PutKudu processor to support batch insert
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2160.
2017-09-22 10:56:15 +02:00
Bryan Bende 329dbe3a64 NIFI-4405 Adding charset property to GenerateFlowFile
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2168.
2017-09-21 17:10:47 +02:00
Bryan Bende a813ae113e NIFI-4391 Ensuring channel is closed when unable to connect in SocketChannelSender
NIFI-4391 Adding debug logging of client port upon connection

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2159.
2017-09-21 16:30:10 +02:00
Koji Kawamura 7b07eb0577 NIFI-4228: Fix PutDatabaseRecord to ignore unmatched fields
- Unmatched fields were ignored, but the number of prepared statement
  place holders were not correct.
- Added unit test code for generateUpdate.
- Added unit test code with "Ignore Unmatched Columns".

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2165.
2017-09-21 16:11:07 +02:00
btwood 90ed08ec33 NIFI-4326 Fix NullPointerException and strict addressing
This uses parseHeader() instead of getFrom() and getRecipients() in order to avoid strict addressing.
It also checks for null to solve a null pointer exception.
By contract, this processor should grab information "if available". Which means it should not fail if the info is unavailable.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2111.
2017-09-21 15:21:22 +02:00
Sébastien Bouchex Bellomié 2ee21b9255 NIFI-4394 Fixed CPU issue - gRPC
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2163.
2017-09-19 17:37:21 +02:00
Bryan Bende 66479464be NIFI-4387 Adding record tag to Parquet processors
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2156.
2017-09-16 14:41:07 +02:00
Matt Gilman eac47e90cb NIFI-4280:
- Adding support for the user to configure variables in the UI.
- Updating the endpoints for changing variables as necessary.
This closes #2135.
2017-09-14 11:12:54 -04:00
m-hogue 91383264d8 NIFI-3510: Added logging for when controller services are enabled/disabled
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2151.
2017-09-14 16:46:18 +02:00
Mark Payne e01d59a462 NIFI-4379: Event-Driven threads are constantly active, polling a queue to see if there is any work to do. Instead of getting number of active threads from the ScheduledExecutor for these threads, updated code to use an AtomicInteger that is incremented when there's work to do and decremented once complete
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2153.
2017-09-14 15:39:32 +02:00
Koji Kawamura 1e70e24267 NIFI-4352: Add CLOB and NCLOB support to PutSQL and ExecuteSQL
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2145.
2017-09-14 11:02:48 +02:00
Mark Payne e52e9acc59 NIFI-4377: Added a fieldName() function to RecordPath and addressed an issue that caused //* to not work
NIFI-4377: Updated RecordPath Guide to include the new fieldName() function and improved unit test for the function

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2147.
2017-09-14 09:40:36 +02:00
yuri1969 1f1269c817 NIFI-3369 - Disable "Add" button on new...
...connection dialog if no relationships selected

Disabled confirmation button for both Create and Configure Connection
dialogs when no relationship was selected.

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #2152
2017-09-13 13:04:24 -04:00
Mark Payne 0e50279467 NIFI-4322: If there is a failure checkpointing FlowFile Repository, ensure that we continue trying
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2146.
2017-09-13 12:14:13 +02:00
Mark Payne b452f8c251 NIFI-4378: Allow suppression of null values in JSON
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2148.
2017-09-12 21:41:59 +02:00
Koji Kawamura 7423b41e3c NIFI-4376: Remove expired S2S request from internal queue. This closes #2144.
Before this fix, expired requests stay in the queue and blocks newly
received requests to be offerred.
2017-09-12 15:31:10 -04:00
Mark Payne d698f227df NIFI-4290: Ensure that the InFlightMessageTracker is instantiated when getTracker() is called
This closes #2083.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-09-12 10:30:44 +09:00
yuri1969 cb7bc93e28
NIFI-4347 - Extend documentation with double-click shortcuts info. This closes #2126 2017-09-11 14:21:37 -04:00
Pierre Villard 5957f6ee95
NIFI-4341 - add provenance repository storage usage in UI
- addressing review comments
- This closes #2119
2017-09-11 13:16:53 -04:00
Gino Lisignoli 897b8ab601 NIFI-4374: Added buffer size to FTP connections
This closes #2141.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-09-11 23:43:56 +09:00
combine 1aeb517c27
NIFI-4370 View Details of NiFi Data Provenance throws NPE when cluster mode. This closes #2137 2017-09-11 09:46:32 -04:00
Joe Percivall 9367c28064 NIFI-2162 Updating OkHttp to 3.8.1 and OkHttp-Digest to 1.13 and refactoring InvokeHttp to adjust for changes
This closes #2004

Signed-off-by: Tony Kurc <tkurc@apache.org>
2017-09-10 22:50:37 -04:00
Wesley-Lawrence e203358bf3 NIFI-4258 Corrected a bug with the 'Informix Unload Escape Disabled' CSV format. This closes #2090. 2017-09-08 16:17:38 -04:00
Pierre Villard c9789adf21 NIFI-4351 - Fix for SiteToSiteProvenanceReportingTask when removing event type or component ID filters
This closes #2127.
2017-09-08 14:14:42 -04:00
Mark Payne 9ebf2cfaf1 NIFI-4306: Allow root field to be updated to a child field by adding property with name / and value /child, for instance
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2103.
2017-09-08 18:50:49 +02:00
Pierre Villard 6559604456 NIFI-4081 - Added raw message option in GrokReader
This closes #1921.
2017-09-08 12:44:37 -04:00
Koji Kawamura 1f67cbf628 NIFI-4004: Use RecordReaderFactory without FlowFile.
- Removed FlowFile from RecordReaderFactory, RecordSetWriterFactory and SchemaAccessStrategy.
- Renamed variable 'allowableValue' to 'strategy' to represent its meaning better.
- Removed creation of temporal FlowFile to resolve Record Schema from ConsumerLease.

- Removed unnecessary 'InputStream content' argument from
  RecordSetWriterFactory.getSchema method.

This closes #1877.
2017-09-08 12:37:40 -04:00
yuri1969 20d23e836e
NIFI-3941 - Clarify tab name for controller-level controller services dialog
* Changed the tab title since sharing the name makes things
less clear for newcomers.
* Suggested info sentence is omitted.

This closes #2124.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-09-07 18:29:13 -07:00
m-hogue 03e51ee8ac
NIFI-4335: Changed SSLContextService implementations to RestrictedSSLContextService for all Listen* processors
This closes #2131.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-09-07 17:34:00 -07:00
Pierre Villard c10ff574c4 NIFI-4257 - add custom WHERE clause in database fetch processors
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2050
2017-09-07 14:41:26 -04:00
patricker ae30c7f350 NIFI-3484: GenerateTableFetch Should Allow for Right Boundary
fix checkstyle issue, and added unit test showing data duplication issue, removed property

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2091
2017-09-07 11:05:38 -04:00
Mike Thomsen 527ce0b4ef NIFI-4269 Added the ability to serialize Mongo documents to a clean JSON view instead of just extended JSON.
NIFI-4269 incorporated changes from the code review.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2063
2017-09-06 15:56:12 -04:00
Pierre Villard 9ac88d210a NIFI-4342 - Add EL support to PutHiveStreaming
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2120
2017-09-06 13:33:18 -04:00
Pierre Villard 458c987fe3 NIFI-4355 - query execution time as attribute of ExecuteSQL
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2129
2017-09-06 13:29:19 -04:00
Andy LoPresto 0536c3edf1
NIFI-4353 Implemented new JAXB logic.
Added unit test and template test resource.
Added RAT exclusion.
This closes #2128
2017-09-06 13:13:24 -04:00
Pierre Villard c3754c3921 NIFI-4320 - Escape quotes for JSON output in QueryCassandra
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2114
2017-09-06 13:05:18 -04:00
Mark Payne bfd6c0aef7 NIFI-4116: Allow fields of Record returned from Lookup Service to be placed into record in the input, instead of requiring that the 'wrapper record' returned from Lookup be included
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #2110.
2017-09-05 21:29:16 -07:00