Apache/nifi
1
0
Fork 0
mirror of https://github.com/apache/nifi.git synced 2025-02-08 19:14:57 +00:00
Code Issues Packages Projects Releases Wiki Activity
4,891 Commits 147 Branches 203 Tags
Commit Graph

830 Commits

Author SHA1 Message Date
Otto Fowler
1330b92cfa NIFI-5892 Wait timestamp lingers, potentially messing up downstream wait-notify pairs 
Clear the wait timestamp when transferring to failur or success

replace explicit attribute clear with function call, refactor and integrate into existing tests per review

This closes #3233.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
 2018-12-28 10:30:02 +09:00
SavtechSolutions
f9f386b0f0 NIFI-5871 ignore UUID attribute when copying flow file attributes (#3203) 
NIFI-5871 ignore UUID attribute when copying flow file attributes

Signed-off-by: Peter Wicks <patricker@gmail.com>
 2018-12-20 10:05:25 -07:00
Pierre Villard
590fa2063c NIFI-4892 - ValidateCSV: no doublequote escaping in invalid output 
NIFI-4892 - ValidateCSV: no doublequote escaping in invalid output

NIFI-5907 - unit test

This closes #2481.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
 2018-12-19 11:20:55 +09:00
Matthew Burgess
7548d7c859 NIFI-3988: Add fragment attributes to SplitRecord 
This closes #3217.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
 2018-12-18 15:07:43 +09:00
Koji Kawamura
0efddf47d5 NIFI-4579: Fix ValidateRecord type coercing 
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2794
 2018-12-13 16:10:41 -05:00
Pierre Villard
a6f91a1975 NIFI-5872 - Added compression option to JsonRecordSetWriter 
This closes #3208.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
 2018-12-10 16:25:59 +09:00
Mark Payne
72ea93a657
NIFI-5868: Added instrumentation around ListFile such that all disk accesses are timed and any unusually long listing times or disk access operations can be logged. Additionally, information is logged at a debug level including significant amounts of troubleshooting information when configured to do so 
This closes #3202.

Signed-off-by: Bryan Bende <bbende@apache.org>
 2018-12-05 13:01:14 -05:00
Matthew Burgess
8a12307d1a
NIFI-5855: Remove unnecessary ORDER BY clause in GenerateTableFetch when Partition Size is zero 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3191.
 2018-11-30 10:54:23 +01:00
Matthew Burgess
68a49cfad0 NIFI-5845: Add support for OTHER and SQLXML JDBC types to SQL/Hive processors 
NIFI-5845: Incorporated review comments

This closes #3184.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
 2018-11-29 09:50:21 +09:00
zenfenan
e34d653ba1
NIFI-5812: Marked Database processors as PrimaryNodeOnly 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3167.
 2018-11-22 09:33:01 +01:00
joewitt
2a4745c077
NIFI-5836 This closes #3181. changed the brittle timing based tests to integration tests 2018-11-21 13:11:06 -05:00
Kumara M S Hemantha
71499f774d
NIFI-5560 This closes #3000. Added Follow SYMLINK support for listFTP & listSFTP and getFTP & getSFTP processors 
Signed-off-by: joewitt <joewitt@apache.org>
 2018-11-21 10:53:38 -05:00
yjhyjhyjh0
3c7012ffda
NIFI-5744: Put exception message to attribute while ExecuteSQL fail 
This closes #3107.

Signed-off-by: Peter Wicks <patricker@gmail.com>
 2018-11-20 09:00:46 -07:00
Colin Dean
54402a1ecd NIFI-5828: Documents behavior of ExecuteSQL attrs when Max Rows Per Flow File is enabled 
This closes #3177.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
 2018-11-19 14:18:31 +09:00
Matthew Burgess
0207d0813e
NIFI-5604: Added property to allow empty FlowFile when no SQL generated by GenerateTableFetch 
co-authored by: Peter Wicks <patricker@gmail.com>
Signed-off-by: Peter Wicks <patricker@gmail.com>

This closes #3075.
 2018-11-15 13:37:43 -07:00
yjhyjhyjh0
75906226a6
NIFI-5780 Add pre and post statements to ExecuteSQL and ExecuteSQLRecord 
Signed-off-by: Peter Wicks <patricker@gmail.com>

This closes #3156.
 2018-11-15 13:18:31 -07:00
vadimar
d319a3ef2f NIFI-5788: Introduce batch size limit in PutDatabaseRecord processor 
NIFI-5788: Introduce batch size limit in PutDatabaseRecord processor

        Renamed 'batch size' to 'Maximum Batch Size'.
        Changed default value of max_batch_size to zero (INFINITE)
        Fixed parameter validation.
        Added unit tests

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3128
 2018-11-15 10:31:34 -05:00
Matthew Burgess
13011ac6d6 NIFI-5652: Fixed LogMessage when logging level is disabled 
This closes #3170.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
 2018-11-15 15:14:57 +09:00
Pierre Villard
4112af013d NIFI-4130 Add lookup controller service in TransformXML to define XSLT from the UI 
addressed review comments

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #1953
 2018-11-14 09:45:20 -05:00
Mark Payne
08189596d2 NIFI-5809: If QueryRecord has a single-column projection and that results in a null value, do not confuse that with a null value being returned from the Record Reader 
This closes #3163.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
 2018-11-12 14:39:27 +09:00
Mark Payne
564ad0cd71
NIFI-5718: Implemented LineDemarcator and removed NLKBufferedReader in order to improve performance 2018-11-09 14:26:20 -07:00
Arek Burdach
765df67817 NIFI-5757 Using Caffeine instead of slow synchronization on LinkedHashMap for caches - mainly avro schema caches 
This closes #3111.

Signed-off-by: Mark Payne <markap14@hotmail.com>
 2018-11-09 14:50:24 -05:00
Vish Uma
63f55d05b4 NIFI-5724 make database connection autocommit configurable 
making the database session autocommit value a configurable property

adding custom validation to PutSQL processor so as to disallow 'supports transaction' and 'rollback on failure' to be true when the autocommit value has been set to true

fixing some style issues to conform to standards

This closes #3113.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
 2018-11-09 17:07:27 +09:00
Mark Payne
49ba3643c7 Increase timeouts in all HandleHttpRequest unit tests from 3 seconds to 20 seconds to avoid frequent unit test failures 2018-11-08 16:20:28 -05:00
Mark Payne
50c0d1ed5d Increase timeouts in unit tests to avoid frequent failures 2018-11-08 16:17:50 -05:00
Koji Kawamura
78a1cb7c5e NIFI-5802: Add QueryRecord nullable field support 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3158.
 2018-11-08 09:50:45 +01:00
Mark Payne
3ba0c0ca82 NIFI-5798: Fixed bug in FlattenJson that was escaping text as Java instead of escaping as JSON 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3138.
 2018-11-08 09:27:30 +01:00
Kotaro Terada
0c9120a73f NIFI-5777: Update the log-level property of LogMessage to support EL 
This closes #3122.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
 2018-11-06 16:20:40 +09:00
Matthew Burgess
d8d220ccb8
NIFI-5601: Add fragment.* attributes to GenerateTableFetch 
Signed-off-by: Peter Wicks <patricker@gmail.com>

This closes #3074
 2018-11-01 15:14:29 -06:00
Koji Kawamura
f0a7dda842 NIFI-5776: ListSFTP should log SSH_FXP_STATUS id value 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3120.
 2018-11-01 13:01:40 +01:00
Kotaro Terada
2a035cc390 NIFI-1505: Add processor description for LogAttribute 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3121.
 2018-11-01 12:56:01 +01:00
Gardella Juan Pablo
1f2cf4bc61 NIFI-5761 ReplaceText processor can stop processing data if it evaluates invalid expressions 
NIFI-5761 Code review.
Remove startsWidth to check an exception. Added the dependency as provided.

NIFI-5761 Code review.
Remove provided.

NIFI-5761 Code review.
Improve logging.

This closes #3112.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
 2018-10-31 12:01:16 +09:00
Jeff Storck
4f14e517db Merge branch 'NIFI-5720-RC3' as part of the NiFi 1.8.0 release process 2018-10-26 13:56:23 -04:00
juliansniff
fdbcf34281 NIFI-5747 fix NPE when redirecting from HTTPS to HTTP for InvokeHTTP 
Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
 2018-10-26 17:08:17 +09:00
Bryan Bende
10479a5a2a NIFI-5740 Ensuring permissions are restored after test completes in TestFetchFile 
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3104
 2018-10-23 15:26:33 -04:00
rednikotin
c8928ce350 NIFI-5727: Added replace unnecessary row count with -1 stub value when paging is used 
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3094
 2018-10-23 14:46:14 -04:00
Jeff Storck
c0182294ed NIFI-5720-RC3 prepare for next development iteration 2018-10-22 22:16:43 -04:00
Jeff Storck
98aabf2c50 NIFI-5720-RC3 prepare release nifi-1.8.0-RC3 2018-10-22 22:16:23 -04:00
Bryan Bende
f3f7cdbab9 NIFI-5719 Ensuring FetchFile routes to failure if the move completion strategy can't be completed 
This closes #3088.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
 2018-10-18 14:37:55 +09:00
Mark Payne
d6422e2d2c NIFI-5711: Replaced JavaDocs that appear also to be copied/pasted from BufferedReader, ran IDE code formatter to get consistent whitespace/formatting 
This closes #3085.

Signed-off-by: Mark Payne <markap14@hotmail.com>
 2018-10-17 10:57:57 -04:00
patricker
95e4294673 NIFI-5711 NLKBufferedReader appears extend and copy portions of the JDK BufferedReader 2018-10-17 10:57:53 -04:00
Mark Payne
32db43b306 NIFI-5686: Updated StandardProcessScheduler so that if it fails to schedule a Reporting Task, it re-schedules the @OnScheduled task instead of looping and calling Thread.sleep. As it was, the single-threaded Process Scheduler was, when calling ProcessScheduler.unschedule(), the unschedule task was not executing because the schedule task was using the only thread. But switching the logic to schedule the task for later and return, instead of calling Thread.sleep and looping, we are able to avoid blocking the one thread in the thread pool. Also, performed some trivial code cleanup and updated erroneous links in Java-docs. 
NIFI-5686: Fixed unit test in TestSocketLoadBalancedFlowFileQueue; renamed TestProcessorLifecycle to ProcessorLifecycleIT as it is testing integration between many components and largely focuses on high numbers of concurrent tasks to see if it can trigger any threading bugs that may get introduced

NIFI-5686: Extended unit test timeouts
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3062
 2018-10-15 15:21:05 -04:00
Matt Gilman
f55204cb69
NIFI-4806: 
- Bumping to tika 1.19.1.

This closes #3069
 2018-10-15 13:52:49 -04:00
Endre Zoltan Kovacs
d5f071b2fe NIFI-3469 multipart request support added to HandleHttpRequest 
- introducing a in-memory-file-size-threashold, above which the incoming file is written to local disk
- using java.io.tmpdir for such file writes
- enhancing documentation
- documenting how to avoid premature HTTP response
- fix and UT for unsuccessful request registration

This closes #2991.

Signed-off-by: Mark Payne <markap14@hotmail.com>
 2018-10-15 13:14:31 -04:00
Brad Hards
e30a21cfc5
[NIFI-5697] Trivial description fix for GenerateFlowFile processor 
This closes #3072.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
 2018-10-13 05:53:30 -04:00
Endre Zoltan Kovacs
5aa4263588 NIFI-1490: better field naming / displayname and description mix up fix 
This closes #2994.

Signed-off-by: Mark Payne <markap14@hotmail.com>
 2018-10-11 15:41:38 -04:00
Endre Zoltan Kovacs
c81a135161 NIFI-1490: multipart/form-data support for ListenHTTP processor 
- introducing a in-memory-file-size-threashold, above which the incoming file is written to local disk
- using java.io.tmpdir for such file writes
- enhancing documentation
 2018-10-11 15:41:19 -04:00
Mark Payne
c425bd2880 NIFI-5533: Be more efficient with heap utilization 
- Updated FlowFile Repo / Write Ahead Log so that any update that writes more than 1 MB of data is written to a file inside the FlowFile Repo rather than being buffered in memory
 - Update SplitText so that it does not hold FlowFiles that are not the latest version in heap. Doing them from being garbage collected, so while the Process Session is holding the latest version of the FlowFile, SplitText is holding an older version, and this results in two copies of the same FlowFile object

NIFI-5533: Checkpoint

NIFI-5533: Bug Fixes

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2974
 2018-10-09 09:18:02 -04:00
Pierre Villard
768bcfb509 NIFI-5635 - Description PutEmail properties with multiple senders/recipients 
This closes #3031

Signed-off-by: Mike Moser <mosermw@apache.org>
 2018-10-07 17:21:01 -04:00
Matthew Burgess
c6572f042b NIFI-4517: Added ExecuteSQLRecord and QueryDatabaseTableRecord processors 
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2945.
 2018-10-06 10:53:11 +02:00