803 Commits

Author SHA1 Message Date
Kotaro Terada
0c9120a73f NIFI-5777: Update the log-level property of LogMessage to support EL
This closes #3122.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2018-11-06 16:20:40 +09:00
Matthew Burgess
d8d220ccb8
NIFI-5601: Add fragment.* attributes to GenerateTableFetch
Signed-off-by: Peter Wicks <patricker@gmail.com>

This closes #3074
2018-11-01 15:14:29 -06:00
Koji Kawamura
f0a7dda842 NIFI-5776: ListSFTP should log SSH_FXP_STATUS id value
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3120.
2018-11-01 13:01:40 +01:00
Kotaro Terada
2a035cc390 NIFI-1505: Add processor description for LogAttribute
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3121.
2018-11-01 12:56:01 +01:00
Gardella Juan Pablo
1f2cf4bc61 NIFI-5761 ReplaceText processor can stop processing data if it evaluates invalid expressions
NIFI-5761 Code review.
Remove startsWidth to check an exception. Added the dependency as provided.

NIFI-5761 Code review.
Remove provided.

NIFI-5761 Code review.
Improve logging.

This closes #3112.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2018-10-31 12:01:16 +09:00
Jeff Storck
4f14e517db Merge branch 'NIFI-5720-RC3' as part of the NiFi 1.8.0 release process 2018-10-26 13:56:23 -04:00
juliansniff
fdbcf34281 NIFI-5747 fix NPE when redirecting from HTTPS to HTTP for InvokeHTTP
Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2018-10-26 17:08:17 +09:00
Bryan Bende
10479a5a2a NIFI-5740 Ensuring permissions are restored after test completes in TestFetchFile
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3104
2018-10-23 15:26:33 -04:00
rednikotin
c8928ce350 NIFI-5727: Added replace unnecessary row count with -1 stub value when paging is used
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3094
2018-10-23 14:46:14 -04:00
Jeff Storck
c0182294ed NIFI-5720-RC3 prepare for next development iteration 2018-10-22 22:16:43 -04:00
Jeff Storck
98aabf2c50 NIFI-5720-RC3 prepare release nifi-1.8.0-RC3 2018-10-22 22:16:23 -04:00
Bryan Bende
f3f7cdbab9 NIFI-5719 Ensuring FetchFile routes to failure if the move completion strategy can't be completed
This closes #3088.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2018-10-18 14:37:55 +09:00
Mark Payne
d6422e2d2c NIFI-5711: Replaced JavaDocs that appear also to be copied/pasted from BufferedReader, ran IDE code formatter to get consistent whitespace/formatting
This closes #3085.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-10-17 10:57:57 -04:00
patricker
95e4294673 NIFI-5711 NLKBufferedReader appears extend and copy portions of the JDK BufferedReader 2018-10-17 10:57:53 -04:00
Mark Payne
32db43b306 NIFI-5686: Updated StandardProcessScheduler so that if it fails to schedule a Reporting Task, it re-schedules the @OnScheduled task instead of looping and calling Thread.sleep. As it was, the single-threaded Process Scheduler was, when calling ProcessScheduler.unschedule(), the unschedule task was not executing because the schedule task was using the only thread. But switching the logic to schedule the task for later and return, instead of calling Thread.sleep and looping, we are able to avoid blocking the one thread in the thread pool. Also, performed some trivial code cleanup and updated erroneous links in Java-docs.
NIFI-5686: Fixed unit test in TestSocketLoadBalancedFlowFileQueue; renamed TestProcessorLifecycle to ProcessorLifecycleIT as it is testing integration between many components and largely focuses on high numbers of concurrent tasks to see if it can trigger any threading bugs that may get introduced

NIFI-5686: Extended unit test timeouts
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3062
2018-10-15 15:21:05 -04:00
Matt Gilman
f55204cb69
NIFI-4806:
- Bumping to tika 1.19.1.

This closes #3069
2018-10-15 13:52:49 -04:00
Endre Zoltan Kovacs
d5f071b2fe NIFI-3469 multipart request support added to HandleHttpRequest
- introducing a in-memory-file-size-threashold, above which the incoming file is written to local disk
- using java.io.tmpdir for such file writes
- enhancing documentation
- documenting how to avoid premature HTTP response
- fix and UT for unsuccessful request registration

This closes #2991.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-10-15 13:14:31 -04:00
Brad Hards
e30a21cfc5
[NIFI-5697] Trivial description fix for GenerateFlowFile processor
This closes #3072.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2018-10-13 05:53:30 -04:00
Endre Zoltan Kovacs
5aa4263588 NIFI-1490: better field naming / displayname and description mix up fix
This closes #2994.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-10-11 15:41:38 -04:00
Endre Zoltan Kovacs
c81a135161 NIFI-1490: multipart/form-data support for ListenHTTP processor
- introducing a in-memory-file-size-threashold, above which the incoming file is written to local disk
- using java.io.tmpdir for such file writes
- enhancing documentation
2018-10-11 15:41:19 -04:00
Mark Payne
c425bd2880 NIFI-5533: Be more efficient with heap utilization
- Updated FlowFile Repo / Write Ahead Log so that any update that writes more than 1 MB of data is written to a file inside the FlowFile Repo rather than being buffered in memory
 - Update SplitText so that it does not hold FlowFiles that are not the latest version in heap. Doing them from being garbage collected, so while the Process Session is holding the latest version of the FlowFile, SplitText is holding an older version, and this results in two copies of the same FlowFile object

NIFI-5533: Checkpoint

NIFI-5533: Bug Fixes

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2974
2018-10-09 09:18:02 -04:00
Pierre Villard
768bcfb509 NIFI-5635 - Description PutEmail properties with multiple senders/recipients
This closes #3031

Signed-off-by: Mike Moser <mosermw@apache.org>
2018-10-07 17:21:01 -04:00
Matthew Burgess
c6572f042b NIFI-4517: Added ExecuteSQLRecord and QueryDatabaseTableRecord processors
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2945.
2018-10-06 10:53:11 +02:00
Andy LoPresto
f65286be83
NIFI-5622 Updated test resource keystores and truststores with SubjectAlternativeNames to be compliant with RFC 6125.
Refactored some test code to be clearer.
Renamed some resources to be consistent across modules.
Changed passwords to meet new minimum length requirements.

This closes #3018
2018-10-04 09:50:09 -04:00
joewitt
8e233ca2ef
NIFI-4806 updated tika and a ton of other deps as found by dependency versions plugin
This closes #3028
2018-10-04 09:32:04 -04:00
Colin Dean
0dd382370b NIFI-5612: Support JDBC drivers that return Long for unsigned ints
Refactors tests in order to share code repeated in tests and to enable
some parameterized testing.

MySQL Connector/J 5.1.x in conjunction with MySQL 5.0.x will return
a Long for ResultSet#getObject when the SQL type is an unsigned integer.
This change prevents that error from occurring while implementing a more
informational exception describing what the failing object's POJO type
is in addition to its string value.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3032
2018-09-28 13:46:04 -04:00
Nick Lewis
db645ec475 NIFI-3344 Added property to JoltTransformJSON allowing the user to specify pretty print, defaults to false
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2987.
2018-09-25 23:07:24 +02:00
Pierre Villard
78c4e223fc NIFI-5591 - Added avro compression format to ExecuteSQL
This closes #3023

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-09-25 09:25:13 -04:00
Mark Payne
2a964681ec NIFI-5514: Fixed bugs in MergeRecord around minimum thresholds not being honored and validation not being performed to ensure that minimum threshold is smaller than max threshold (would previously allow min record = 100, max records = 2 as a valid configuration)
NIFI-5514: Do not rely on ProcessSession.getQueueSize() to return a queue size of 0 objects because if the processor is holding onto data, the queue size won't be 0.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2954.
2018-09-24 22:36:02 +02:00
Otto Fowler
7e627f2fed
NIFI-5566 Added unit test for PR feedback.
Signed-off-by: Andy LoPresto <alopresto@apache.org>
2018-09-17 17:12:53 -07:00
Andy LoPresto
cd6877404b
NIFI-5147, NIFI-5566 Added CryptographicHashAttribute and CryptographicHashContent processors.
Deprecated HashContent processor.
Added documentation to HashAttribute processor.
Added shared HashService and HashAlgorithm enum.
Added unit tests.
Added #clearProperties() to TestRunner, StandardProcessorTestRunner, and MockProcessContext.
Updated processor manifest.
Updated Javadoc.
Added documentation about deprecated/renamed components.
Added logic handling for UTF-16 encoding where UTF-16BE is overridden so the BOM is not inserted prior to hashing.

This closes #2983.

Co-authored-by: Otto Fowler <ottobackwards@gmail.com>
Signed-off-by: Kevin Doran <kdoran@apache.org>
2018-09-17 17:03:57 -07:00
Andy LoPresto
7e8b7752ca NIFI-5569 Added keywords to Route* and ScanAttribute processors to improve discoverability.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2984.
2018-09-11 18:09:48 +02:00
Mark Payne
0274bd4ff3 Revert "NIFI-4272 support multiple captures when EL is present in replacement value"
This reverts commit f7f809c3d3632eea5234b31740984b73de322464.

NIFI-5474, NIFI-4272: When using Regex Replace with ReplaceText, and there are capturing groups, ensure that we populate the 'additionalVariables' map for each match of the regex. This allows Expression Language to reference the back-references properly even when there are multiple matches

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2951
2018-09-11 11:07:33 -04:00
Lars Francke
1b73578c48 NIFI-5408 Add documentation for NIFI-4279
This closes #2878

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-08-31 19:09:58 -04:00
Andy LoPresto
8f37b5ee10
NIFI-4426 Removed exclude from nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/pom.xml. 2018-08-30 13:05:09 -07:00
Matthew Burgess
8a8a8cb072 NIFI-5553: Fixed issue with rollback on result set errors
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2967.
2018-08-29 13:22:10 +02:00
Mark Payne
6ceee25859 NIFI-5551: If session.get() returns null, then return fron onTrigger instead of assuming that the FlowFile is non-null
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2964.
2018-08-27 16:52:07 +02:00
Mike Moser
8c0705cb6b NIFI-5275 PostHTTP SocketConfig setup, fixed connection pool when using HTTPS, setup idle connection checker, setup request retry handler, improved some exception handling and logging, and NIFI-1336 fix as well
This closes #2796.
Signed-off-by: Brandon Devries <devriesb@apache.org>
2018-08-24 10:07:11 -04:00
patricker
17ea29244e NIFI-5381 SFTP Unit Tests
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2846.
2018-08-10 20:17:28 +02:00
Mark Payne
d1ab17580f NIFI-5465: Set the Idle Timeout on jetty connectors to the same as the Request Timeout. 2018-08-06 10:33:24 -04:00
Matthew Burgess
9742dd2fac NIFI-5471: Push table aliasing to DatabaseAdapter impls
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2926.
2018-07-31 10:14:27 +02:00
Matthew Burgess
692943f016 NIFI-5454: Added EL support and copy.index attribute to DuplicateFlowFile
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2917.
2018-07-26 14:13:14 +02:00
Koji Kawamura
8b9d446118 NIFI-54096: Consolidating new model into existing List processors.
Adding followings:

- Use separate DistributedMapCache for tracking entities to avoid
conflict with existing code
- Added more validation
- Delete listed entities from cache if reset is needed
- Support Local scope
- Added Initial Listing Target

This closes #2876.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-07-17 13:59:57 -04:00
Otto Fowler
b10220439c
NIFI-5337 Syslog 5424 Record Reader and nifi-syslog-utils
- Create nifi-syslog-utils to move syslog parsing functionalty to a central location shared by the processors and serialization/record system.
- Refactor Processors to use these utils
- Update 5424 syslog classes using simple-syslog-5424 to pick up new changes to support this work, as well as keep dependencies/types from bleeding out to the
processors or readers
- Refactor Syslog5424Event and Parser
- Create Syslog5424RecordReader
- per review, handle blank message differently from eof
- name schema per review

This closes #2816.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-07-16 11:50:55 -04:00
Marco Gaido
31bb89514c NIFI-5426: Use NIO.2 API for ListFile 2018-07-13 13:54:27 -04:00
Andy LoPresto
ffbff42421 NIFI-5415 Renamed ListenSyslogGroovyTest to ITListenSyslogGroovy.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2883
2018-07-13 09:57:50 -04:00
patricker
0dd4a91a67 NIFI-5312 QueryDatabaseTable updates state when an SQLException is thrown
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2868
2018-07-10 19:03:14 -04:00
patricker
e13602f752 NIFI-4279 PutDatabaseRecord and ConvertJSONToSQL stream has already been closed error on Oracle
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2874
2018-07-10 15:30:07 -04:00
patricker
624bbab8f0 NIFI-5382 ConvertRecord Add Option To Drop 0 Record Files
NIFI-5382 ConvertRecord Add Option To Drop 0 Record Files

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2853
2018-07-10 12:56:53 -04:00
Andy LoPresto
f60585a9b6
NIFI-5376 Removed deprecation warnings.
Updated Javadoc for SiteToSiteClient#createTransaction() and HttpClient implementation.
Reverted exception listing in method contract for SiteToSiteClient#createTransaction and HttpClient tion of same.
Reverted import ordering in TestSiteToSiteClient.
Reverted exception listing in TestGetHDFSFileInfo, TestListHDFS, and StandardHttpFlowFileServerProtocol.
Restored @SuppressWarnings annotation and removed unnecessary "public static" keywords from inner classes in SiteToSiteClient.

This closes #2841.

Signed-off-by: Joe Witt <joewitt@apache.org>
2018-07-09 20:45:34 -07:00