Commit Graph

4768 Commits

Author SHA1 Message Date
Mark Payne 5872eb3c4a NIFI-5331: When checkpointing SequentialAccessWriteAheadLog, if the journal is not healthy, ensure that we roll it over and ensure that if an Exception is thrown when attempting to fsync() or close() the journal, we continue creating a new one.
This closes #2952.
Signed-off-by: Brandon Devries <devriesb@apache.org>
2018-10-04 15:25:53 -04:00
Koji Kawamura 8f4d13eeac
NIFI-5581: Fix replicate request timeout
This closes #3044

- Revert 87cf474e54 to enable connection
pooling
- Changes the expected HTTP status code for the 1st request of a
two-phase commit transaction from 150 (NiFi custom) to 202 Accepted
- Corrected RevisionManager Javadoc about revision varidation protocol
2018-10-04 10:25:42 -04:00
Andy LoPresto f65286be83
NIFI-5622 Updated test resource keystores and truststores with SubjectAlternativeNames to be compliant with RFC 6125.
Refactored some test code to be clearer.
Renamed some resources to be consistent across modules.
Changed passwords to meet new minimum length requirements.

This closes #3018
2018-10-04 09:50:09 -04:00
joewitt 8e233ca2ef
NIFI-4806 updated tika and a ton of other deps as found by dependency versions plugin
This closes #3028
2018-10-04 09:32:04 -04:00
pepov de685a7a74
NIFI-5656 Handly empty "Node Group" property in FileAccessPolicyProvider consistently, add some logs to help with debugging, add test for the invalid group name and for the empty case.
This closes #3043.

Signed-off-by: Kevin Doran <kdoran@apache.org>
2018-10-03 11:12:19 -04:00
Brandon Devries b4c8e0179b Merge branch 'pr2931' 2018-10-02 11:08:43 -04:00
Mike Moser 76a9f98d7f NIFI-3531 Catch and rethrow generic Exception to handle RuntimeExceptions, and allow test to pass
This closes #2931.
Signed-off-by: Brandon Devries <devriesb@apache.org>
2018-10-02 11:06:51 -04:00
Brandon Devries 895323f3c2 Merge branch 'pr2949' 2018-10-02 09:40:36 -04:00
Mike Moser 4f538f1ecb NIFI-3672 updated PublishJMS message property docs
This closes #2949

Signed-off-by: Brandon Devries <devriesb@apache.org>
2018-10-02 09:39:17 -04:00
Matthew Burgess 813cc1f6a2 NIFI-5650: Added Xerces to scripting bundle for Jython 2.7.1
This closes #3042

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-10-01 10:57:58 -04:00
Mike Thomsen b1478cdb19 NIFI-5051 Created ElasticSearch lookup service.
NIFI-5051 Fixed checkstyle issue.

NIFI-5051 Converted ES lookup service to use a SchemaRegistry.

NIFI-5051 Cleaned up POM and added a simple unit test that uses a mock client service.

NIFI-5051 Added change; waiting for feedback.

NIFI-5051 Changed query setup based on code review. Changed tests to Groovy to make them easier to read with all of the inline JSON.

NIFI-5051 fixed a checkstyle issue.

NIFI-5051 Rebased to cleanup merge issues

NIFI-5051 Added changes from a code review.

NIFI-5051 Fixed a checkstyle issue.

NIFI-5051 Added coverage generator for tests.

Rebased.

NIFI-5051 Updated service and switched it over to JsonInferenceSchemaRegistryService.

NIFI-5051 Removed dead code.

NIFI-5051 Fixed checkstyle errors.

NIFI-5051 Refactored query builder.

NIFI-5051 Added placeholder gitignore to force test compile.

NIFI-5051 Added note explaining why the .gitignore file was needed.

NIFI-5051 Made constructor public.

NIFI-5051 Fixed path issue in client service integration tests.

NIFI-5051 Added additional mapping capabilities to let users massage the result set into the fields they want.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2615
2018-10-01 10:48:45 -04:00
Andy LoPresto 748cf74562
NIFI-5628 Added content length check to OkHttpReplicationClient.
Added unit tests.

This closes #3035
2018-09-28 14:34:03 -04:00
Colin Dean 0dd382370b NIFI-5612: Support JDBC drivers that return Long for unsigned ints
Refactors tests in order to share code repeated in tests and to enable
some parameterized testing.

MySQL Connector/J 5.1.x in conjunction with MySQL 5.0.x will return
a Long for ResultSet#getObject when the SQL type is an unsigned integer.
This change prevents that error from occurring while implementing a more
informational exception describing what the failing object's POJO type
is in addition to its string value.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3032
2018-09-28 13:46:04 -04:00
Jeff Storck e24388aa7f NIFI-5557 Added test in PutHDFSTest for IOException with a nested GSSException
Resolved most of the code warnings in PutHDFSTest

This closes #2971.
2018-09-27 16:24:31 -04:00
Endre Zoltan Kovacs 0f55cbfb9f NIFI-5557: handling expired ticket by rollback and penalization 2018-09-27 16:24:31 -04:00
Mark Payne 2e1005e884 NIFI-5640: Improved efficiency of Avro Reader and some methods of AvroTypeUtil. Also switched ServiceStateTransition to using read/write locks instead of synchronized blocks because profiling showed that significant time was spent in determining state of a Controller Service when attempting to use it. Switching to a ReadLock should provide better performance there.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3036
2018-09-27 15:38:47 -04:00
Mark Payne ad4c886fbf
NIFI-5634: When merging RPG entities, ensure that we only send back the ports that are common to all nodes - even if that means sending back no ports
This closes #3030
2018-09-27 10:11:15 -04:00
Pierre Villard ca70dbbb36 This closes #1661. 2018-09-26 13:05:22 +02:00
Pierre Villard db023f368b This closes #1142. 2018-09-25 23:47:51 +02:00
Nick Lewis db645ec475 NIFI-3344 Added property to JoltTransformJSON allowing the user to specify pretty print, defaults to false
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2987.
2018-09-25 23:07:24 +02:00
Pierre Villard 5d3558a79d NIFI-5633 - allDelineatedValues can throw ArrayIndexOutOfBoundsException
This closes #3029.
2018-09-25 14:07:00 -04:00
Pierre Villard 78c4e223fc NIFI-5591 - Added avro compression format to ExecuteSQL
This closes #3023

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-09-25 09:25:13 -04:00
Mark Payne 030129c7ce NIFI-5618: Avoid NPE when viewing Provenance Event details on a disconnected node
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3027.
2018-09-25 12:28:58 +02:00
Mark Payne 4b4c9e14cb NIFI-5630: Ensure that we include counters in Status History when present
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3026.
2018-09-25 12:24:17 +02:00
Pierre Villard c4d3b5e94f NIFI-5588 - Fix max wait time in DBCP Connection Pool
This closes #3022

Signed-off-by: zenfenan <zenfenan@apache.org>
2018-09-25 12:19:54 +05:30
Mark Payne 2a964681ec NIFI-5514: Fixed bugs in MergeRecord around minimum thresholds not being honored and validation not being performed to ensure that minimum threshold is smaller than max threshold (would previously allow min record = 100, max records = 2 as a valid configuration)
NIFI-5514: Do not rely on ProcessSession.getQueueSize() to return a queue size of 0 objects because if the processor is holding onto data, the queue size won't be 0.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2954.
2018-09-24 22:36:02 +02:00
Andrew Lim f5e9ea680a NIFI-5578 Correct errors in EL Guide including Date Manipulation and Escaping EL examples
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3017.
2018-09-21 11:03:36 +02:00
Andrew I. Christianson be2ed060a2
NIFI-5542 Added support for node groups to FileAccessPolicyProvider
This closes #2970.

Signed-off-by: Kevin Doran <kdoran@apache.org>
2018-09-20 10:27:28 -04:00
Matthew Burgess 2ee0af9663 NIFI-5614: Update commons-dbcp to commons-dbcp2 for DBCPConnectionPool
This closes #3014

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-09-20 08:17:52 -04:00
Mark Payne 73a7c8c6b4 NIFI-5613: Upgraded version of geoip2 library to version 2.12.0; fixed unit tests
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3013.
2018-09-20 13:55:13 +02:00
Kemix Koo 8cf0662e5b NIFI-5615: set the avro-binary MIME type for converting processors
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3016.
2018-09-20 12:07:43 +02:00
Pierre Villard 854ee3021c NIFI-5599 - Bump Kafka versions
This closes #3006

Signed-off-by: zenfenan <sivaprasanna246@gmail.com>
2018-09-20 14:33:23 +05:30
Mark Payne 3dd548e807 NIFI-5609: Fixed NullPointer when attempting to view status history for a component that has not yet run
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3012
2018-09-19 16:56:42 -04:00
Koji Kawamura f570cb980d
NIFI-375: Added operation policy
The operation policy allows that a user to operate components even if they does not have direct READ/WRITE
permission of the component.

Following operations are controlled by the new operate policy:
- Start/stop/enable/disable Processors, ControllerServices,
ReportingTasks, Input/OuputPorts
- Enable/disable transmission of RemoteInput/OutputPorts and
RemoteProcessGroups
- Terminate Processor threads

Refactored what API exposes

The previous commit let API exposes few fields in DTO. But we should
avoid returning partial DTO as it complicates authorization logic.

Instead, this commit adds StatusDTO for ReportingTaskEntity and
ControllerServiceEntity, so that it can be returned regardless of having
READ permission. Component DTO can only be returned with a READ
permission.

Refactor RPG same as ControllerService.

WIP incorporating review comments.

Incorporated review comments

- Cleaned up merger classes
- Recreate DTO instance at each function during two phase commmit

Restrict enabling ControllerService without read permission

Revert the last commit.

Fix review comments.

- Renamed confusing static method names and its parameters
- Removed unnecessary permission checks from UI condition

Fixed delete action display condition.

Fixed NPE at Summary.

Apply operation policy to activateControllerServices.

Removed OperationPermissible from ComponentEntity.

This closes #2990
2018-09-19 15:28:41 -04:00
“PrashanthVenkatesan” 9161b1787a NIFI-5327 Adding Netflowv5 protocol parser
NIFI-5327 Formated Code to adhere NiFi checkstyle
NIFI-5327 Incorporated code review comments
NIFI-5327 Update additionalDetails docs
NIFI-5327 Format IPV4 fields & rename nar
NIFI-5327 Update dependency jar version
NIFI-5327 Updated Code review comments
NIFI-5327 Updated Code review comments
NIFI-5327 Patch for code checkstyle violations
NIFI-5327 commit for retriggering build

This closes #2820

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-09-19 13:15:41 -04:00
Mark Payne ad80f5f064 NIFI-5598: Allow JMS Processors to lookup Connection Factory via JNDI
NIFI-5598: Expose JNDI Principal & Credentails as explicit properties
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3005
2018-09-19 12:56:58 -04:00
Mark Payne 0da4f50ea0 NIFI-5518: Added processors for integrating with Apache Kafka 2.0
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2948.
2018-09-19 15:50:08 +02:00
Mike Thomsen 1dea8faa06 NIFI-5239 Made a client service an optional source of connection pooling in Mongo processors.
This closes #2896

Signed-off-by: zenfenan <zenfenan@apache.org>
2018-09-19 17:04:25 +05:30
zenfenan c56a7e9ba5 NIFI-5510: Introducing PutCassandraRecord processor
NIFI-5510: Fixes for PR review comments

This closes #2992

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-09-18 08:10:14 -04:00
Otto Fowler 7e627f2fed
NIFI-5566 Added unit test for PR feedback.
Signed-off-by: Andy LoPresto <alopresto@apache.org>
2018-09-17 17:12:53 -07:00
Andy LoPresto cd6877404b
NIFI-5147, NIFI-5566 Added CryptographicHashAttribute and CryptographicHashContent processors.
Deprecated HashContent processor.
Added documentation to HashAttribute processor.
Added shared HashService and HashAlgorithm enum.
Added unit tests.
Added #clearProperties() to TestRunner, StandardProcessorTestRunner, and MockProcessContext.
Updated processor manifest.
Updated Javadoc.
Added documentation about deprecated/renamed components.
Added logic handling for UTF-16 encoding where UTF-16BE is overridden so the BOM is not inserted prior to hashing.

This closes #2983.

Co-authored-by: Otto Fowler <ottobackwards@gmail.com>
Signed-off-by: Kevin Doran <kdoran@apache.org>
2018-09-17 17:03:57 -07:00
Mohit Garg 07ad132a2b NIFI-5596 Upgraded splunk-sdk-java version
NIFI-5596 Updated L&N

L&N commit added by Mike Thomsen

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-09-17 12:52:17 -04:00
Mark Payne c8fc1327ee NIFI-5592: If an Exception is thrown by RecordReader.read() from ConsumeKafkaRecord, route Record to parse.failure relationship
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3001.
2018-09-15 14:49:24 +02:00
Matthew Burgess 5a84d650c3 NIFI-5449: Added Base64 Encode/Decode functions to RecordPath
NIFI-5449: Incorporated review comments

This closes #2920

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-09-14 14:04:25 -04:00
uday 57ae9b65a0 NIFI-1184 Mock FileSystem for PutHDFSTest
This closes #2892

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-09-14 13:30:03 -04:00
Matthew Burgess 4a25402c1a NIFI-3425: Provide ability to cache CQL statements
This closes #2986.

Co-authored-by: Michael A Giroux <magirou@rd6ul-92373g.infosec.tycho.ncsc.mil>
Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-09-14 11:37:18 -04:00
Bryan Bende 14729be837 NIFI-5577 Fixing update method in NaiveRevisionManager so that failed updates don't change the revision
This closes #2995.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-09-14 11:33:53 -04:00
Mark Payne 647152ab16
NIFI-5597: Restored FreeFormTextRecordSetWriter to the META-INF services file
This closes #3004.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-09-14 09:52:39 -04:00
Vijeta e0f7f244b8 Clarifying upsert query documentation for PutMongo processor.
This closes #2999

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-09-12 17:46:43 -04:00
Andy LoPresto 7e8b7752ca NIFI-5569 Added keywords to Route* and ScanAttribute processors to improve discoverability.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2984.
2018-09-11 18:09:48 +02:00