Commit Graph

4180 Commits

Author SHA1 Message Date
Pierre Villard ea2c91f9d0 NIFI-4372 Wait processor - recommend prioritizer in documentation. This closes #2139. 2017-10-04 16:02:52 -04:00
Giovanni Lanzani f1bd866005
NIFI-4443 Increase StoreInKiteDataset flexibility
* The configuration property CONF_XML_FILE now support Expression
  Language and reuse a Hadoop validator;
* The ADDITIONAL_CLASSPATH_RESOURCES property has been added, so that
  things such as writing to Azure Blob Storage should become possible.

This closes #2186.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-10-04 15:49:51 -04:00
Noe 0a47a3bde5
NIFI-3349 retry stale connections
This closes #2149.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-10-04 12:47:28 -04:00
Mark Payne 8741b6f6a5
NIFI-4461: When reading from socket channel use blocking mode instead of sleeping; when writing, use a far smaller sleep duration
This closes #2193.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-10-04 11:45:35 -04:00
Mark Payne e773fa5513 NIFI-4328: Fix to pom.xml for swagger maven plugin. This closes #2191. 2017-10-03 13:23:58 -04:00
Matt Gilman 697c382d9b
NIFI-4328:
- Upgrading swagger and the corresponding maven plugin to address issues discovered in recursive domain models.
2017-10-03 09:50:11 -04:00
Koji Kawamura 58e4fb576e NIFI-4008: Allow 0 or more records within a message. This closes #1891. 2017-10-02 15:40:29 -04:00
Jeff Storck 36b16c9cdd NIFI-4412 Updating Docker Hub image version information to accommodate next release. 2017-10-02 14:20:40 -04:00
Jeff Storck 2694adcca9 Merge branch 'NIFI-4412-RC2' 2017-10-02 13:58:54 -04:00
Mark Payne b3be2459e4 NIFI-4330: Fixed checkstyle violations (tabs instead of spaces). This closes #2185. 2017-09-29 10:33:56 -04:00
gardellajuanpablo 2d5b8c7267 NIFI-4330 ConsumeKafka* throw NullPointerException if Kafka message has a null value
It is possible null values to be stored in Kafka topics. Fixed handle this scenario.
Notice without this fix, the consumer is unable to consume more messages (at least
without removing messages from the queue).
2017-09-29 10:57:53 -03:00
Mark Owens feaf44b623
NIFI-2184 JettyServer should confirm "docs" path exists before using it in .createDocsWebApp().
Refactored the createDocsWebApp method of the JettyServer.java class. Previously NiFi would fail to
start up and instead throw an IllegalStatException if the 'docs' directory did not exist in the
installation directory. With the update, if the 'docs' directory is missing, an attempt to create
the missing directory will be made and if successful will enable NIFI to startup successfully, barring
any other startup errors of course. The side effect of this change is that the help documentation
under the 'General' heading of the help page will be missing.

Three small helper methods were extracted from the original method.  Each related to a section of the
original code that could throw an exception. In each case if an exception is now thrown a more helpful
log message will be output and the process will be closed via the startUpfailure method rather than
throwing an exception.

This closes #2164.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-09-28 14:49:29 -07:00
Jeff Storck a57911d3db NIFI-4412-RC2 prepare for next development iteration 2017-09-28 13:45:36 -04:00
Jeff Storck e6508ba7d3 NIFI-4412-RC2 prepare release nifi-1.4.0-RC2 2017-09-28 13:45:21 -04:00
Pierre Villard bdab3cda0a NIFI-4340 - fix record path evaluation when array is [ null ]
This closes #2122.
2017-09-28 13:39:55 -04:00
Jeff Storck cd213db0e2 NIFI-4412 Added GPG key for Jeff Storck 2017-09-28 12:20:40 -04:00
Jeff Storck 502adc6e8a NIFI-4412 Removed duplicate dependency management definition for nifi-mongodb-client-service-api. 2017-09-28 11:08:47 -04:00
Koji Kawamura 9ff7d15603 NIFI-4363 Rename JAVA_ARGS to JAVA_OPTS 2017-09-28 08:31:13 +09:00
Jeff Storck d27103100d NIFI-4363 Replaced hardcoded heap size options with JAVA_ARGS environment variable.
This closes #2136.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-09-28 08:30:47 +09:00
Matt Gilman 6c798d18ef NIFI-4382:
- Adding support for KnoxSSO.
- Updated the docs for nifi.security.user.knox.audiences.
- The KnoxSSO cookie is removed prior to request replication.

This closes #2177
2017-09-27 16:22:18 -04:00
Matt Gilman d47bbd12ce
NIFI-4413: - Using the incoming request to determine the appropriate scheme for the data reference URI. Necessary for cases where the NiFi instance is behind a proxy running a different scheme.
This closes #2179.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-09-27 09:51:06 -04:00
m-hogue b7b6d9082c
NIFI-2529: Added support for SSLContextService protocols in HandleHttpRequest
Forced HandleHTTPRequest to use RestrictedSSLContextService and removed extraneous SSL algorithm checks
Throw RuntimeException if the chosen SSL protocol isn't supported by HandleHttpRequest

This closes #1985.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-09-26 18:48:19 -07:00
Andy LoPresto 2c1f5b49e4 NIFI-3116 This closes #2108. Added initial regression test for StringEncryptor to ensure continued functionality during removal of Jasypt.
Added external compatibility regression test for StringEncryptor to ensure continued functionality during removal of Jasypt.
Documents custom salt lengths and iteration counts for each encryption method.
Added (ignored) failing tests for keyed encryption (Jasypt does not support keyed encryption).
Changed StringEncryptor to non-final class and added protected default constructor.
Added failing test for initialization status.
Added utility methods in CipherUtility.
Moved PBE cipher providers (and tests) from nifi-standard-processors to nifi-security-utils module.
Implemented PBE and keyed encryption/decryption logic.
Moved Scrypt unit test back into scrypt package.
Resolved test failures in limited strength cryptographic environment.
Implemented keyed encryption/decryption and enabled unit tests.
Removed Jasypt dependency from production scope (kept in test scope for backward compatibility tests).

Signed-off-by: joewitt <joewitt@apache.org>
2017-09-26 10:55:29 -04:00
Arun Manivannan 3b1b326b13 NIFI-4416 CSVRecordReader does not accept escaped character as delimiter
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2172
2017-09-25 22:32:15 -04:00
Matthew Burgess 1ca4fa3a83
NIFI-4345: Removed lookup-services dependency from MongoDB Services NAR, cleaned up deps
- Added nifi-lookup-service-api to nifi-nar-bundles POM

This closes #2174.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-09-25 14:52:33 -04:00
Bryan Bende 6eab91923e NIFI-4418 Adding ListenUDPRecord processor. This closes #2173. 2017-09-25 13:19:23 -04:00
Koji Kawamura c6f4421c8e NIFI-2829: Fixed PutSQL time unit test.
The unit test for DATE type used GMT timezone, that causes an assertion error in timezones such as EST (-5).
We need to use local timezone instead of GMT, as Derby and PutSQL uses local timezone.

The unit test failed before as follows:
- Unit test code, passed: '2002-02-02 GMT'
- PutSQL code convertedi it to local: '2002-02-01 EST', and stored as '2002-02-01' in Derby database without timezone info
- Unit test code SELECT the inserted value, passed a GMT calender, then got epoch timestamp, which was '2002-01-31'

Support negative long value for timezones ahead of UTC.

- For timezones such as '+0800', it's possible that a local time e.g. '02:03:04' can be a negative epoch value. This commit changes LONG_PATTERN so that it can accept nevative values.
- Changed time values in unit tests to verify negative epoch values, and avoid using the same digits among different time unit for better readability.

This closes #2082
2017-09-23 17:41:42 -04:00
Mark Payne 3fb704c58f NIFI-4201: This closes #2024. Implementation of processors for interacting with Kafka 0.11
Signed-off-by: joewitt <joewitt@apache.org>
2017-09-22 22:08:19 -04:00
Deon Huang a29348f2a4 NIFI-4395 - GenerateTableFetch can't fetch column type by state after instance reboot
NIFI-4395: Updated unit test for GenerateTableFetch
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2166
2017-09-22 14:34:00 -04:00
Andy LoPresto 9e2c7be7d3
NIFI-4353
- Added XmlUtils class.
- Added unit test.
- Added XXE test resource.
- Refactored JAXB unmarshalling globally to prevent XXE attacks.
- Refactored duplicated/legacy code.
- Cleaned up commented code.
- Switched from FileInputStream back to StreamSource in AuthorizerFactoryBean.
- This closes #2134
2017-09-22 14:31:38 -04:00
Mike Thomsen 9a8e6b2eb1 NIFI-4345 Added a MongoDB controller service and a lookup service.
NIFI-4345: Changed Lookup Key to Lookup Value Field
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2123
2017-09-22 13:33:07 -04:00
Mark Payne 05b5dd1488 NIFI-4409: Ensure that when record schema is inherited, the schema from teh ResultSet is used instead of the schema from the RecordReader because the schema from the RecordReader mmay not match the actual data
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2169
2017-09-22 12:41:17 -04:00
Pierre Villard 50d018566d
NIFI-4403 - add group name to bulletins model. This closes #2167 2017-09-22 11:53:58 -04:00
cam 0c0c33411d NIFI-4384 - Enhance PutKudu processor to support batch insert
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2160.
2017-09-22 10:56:15 +02:00
Bryan Bende 329dbe3a64 NIFI-4405 Adding charset property to GenerateFlowFile
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2168.
2017-09-21 17:10:47 +02:00
Bryan Bende a813ae113e NIFI-4391 Ensuring channel is closed when unable to connect in SocketChannelSender
NIFI-4391 Adding debug logging of client port upon connection

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2159.
2017-09-21 16:30:10 +02:00
Koji Kawamura 7b07eb0577 NIFI-4228: Fix PutDatabaseRecord to ignore unmatched fields
- Unmatched fields were ignored, but the number of prepared statement
  place holders were not correct.
- Added unit test code for generateUpdate.
- Added unit test code with "Ignore Unmatched Columns".

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2165.
2017-09-21 16:11:07 +02:00
btwood 90ed08ec33 NIFI-4326 Fix NullPointerException and strict addressing
This uses parseHeader() instead of getFrom() and getRecipients() in order to avoid strict addressing.
It also checks for null to solve a null pointer exception.
By contract, this processor should grab information "if available". Which means it should not fail if the info is unavailable.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2111.
2017-09-21 15:21:22 +02:00
Sébastien Bouchex Bellomié 2ee21b9255 NIFI-4394 Fixed CPU issue - gRPC
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2163.
2017-09-19 17:37:21 +02:00
Bryan Bende 66479464be NIFI-4387 Adding record tag to Parquet processors
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2156.
2017-09-16 14:41:07 +02:00
Matt Gilman eac47e90cb NIFI-4280:
- Adding support for the user to configure variables in the UI.
- Updating the endpoints for changing variables as necessary.
This closes #2135.
2017-09-14 11:12:54 -04:00
m-hogue 91383264d8 NIFI-3510: Added logging for when controller services are enabled/disabled
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2151.
2017-09-14 16:46:18 +02:00
Mark Payne e01d59a462 NIFI-4379: Event-Driven threads are constantly active, polling a queue to see if there is any work to do. Instead of getting number of active threads from the ScheduledExecutor for these threads, updated code to use an AtomicInteger that is incremented when there's work to do and decremented once complete
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2153.
2017-09-14 15:39:32 +02:00
Koji Kawamura 1e70e24267 NIFI-4352: Add CLOB and NCLOB support to PutSQL and ExecuteSQL
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2145.
2017-09-14 11:02:48 +02:00
Mark Payne e52e9acc59 NIFI-4377: Added a fieldName() function to RecordPath and addressed an issue that caused //* to not work
NIFI-4377: Updated RecordPath Guide to include the new fieldName() function and improved unit test for the function

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2147.
2017-09-14 09:40:36 +02:00
yuri1969 1f1269c817 NIFI-3369 - Disable "Add" button on new...
...connection dialog if no relationships selected

Disabled confirmation button for both Create and Configure Connection
dialogs when no relationship was selected.

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #2152
2017-09-13 13:04:24 -04:00
Mark Payne 0e50279467 NIFI-4322: If there is a failure checkpointing FlowFile Repository, ensure that we continue trying
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2146.
2017-09-13 12:14:13 +02:00
Mark Payne b452f8c251 NIFI-4378: Allow suppression of null values in JSON
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2148.
2017-09-12 21:41:59 +02:00
Koji Kawamura 7423b41e3c NIFI-4376: Remove expired S2S request from internal queue. This closes #2144.
Before this fix, expired requests stay in the queue and blocks newly
received requests to be offerred.
2017-09-12 15:31:10 -04:00
Mark Payne d698f227df NIFI-4290: Ensure that the InFlightMessageTracker is instantiated when getTracker() is called
This closes #2083.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-09-12 10:30:44 +09:00