Commit Graph

4252 Commits

Author SHA1 Message Date
Pierre Villard 11a7d5c09f NIFI-4955 - Preserve columns ordering with CSV and ValidateRecord
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2528
2018-03-13 17:09:07 -04:00
Pierre Villard 95e543d4cc NIFI-4967 - CSVRecordReader does not read header with specific formats
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2536
2018-03-13 16:54:46 -04:00
Pierre Villard c75604a564 NIFI-4966 - JacksonCSVRecordReader - NPE with some CSV formats
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2535
2018-03-13 16:49:47 -04:00
Ed 29c2e18832 NIFI-4968: fix printing indefinite log errors
After first failure in customValidate, stop printing logs until any property is changed

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2537
2018-03-13 16:40:49 -04:00
Bryan Bende 9b098f3235
NIFI-4864 - Additional improvements to additonal resource loading
This closes #2470.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-03-13 13:23:28 -04:00
zenfenan fad152f389
NIFI-4864 Fixing Additional Resources property pointing at a directory won't find new JARs
NIFI-4864: Code refactor and improved additionalUrlsFingerprint implementation
Signed-off-by: Bryan Bende <bbende@apache.org>
2018-03-13 13:23:16 -04:00
Bryan Bende f1d7518547
NIFI-4969 Fixing error when importing versioned flow with a processor that uses event driven scheduling. This closes #2539 2018-03-13 13:15:24 -04:00
Mark Payne d14229e440 NIFI-4774: Allow user to choose which write-ahead log implementation should be used by the WriteAheadFlowFileRepository
Removed TODO comment

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2487
2018-03-13 13:03:39 -04:00
Ed c2616e6fe7
NIFI-4833 Add ScanHBase Processor
- New processor for scanning HBase records based on verious params like range of rowkeys, range of timestamps. Supports result limit and reverse scan.
- Adds Atlas Support for ScanHBase processor
- Fixed not recent version of FF
- Formatting and Style changes
- Single line to multiline if-then statements
- Removed HTML formatting that is not used for doc generation
- Fixed issue with limitRows
- Fixed issue with filter expression
- Refactored "processRows"
- Fixed possible NPE for bulkSize var
- Changed provenance to "receive" to indicate new data from external source.
- Updated min/max timestamp custom validation
- JSON array support
- Removed in-memory caching for records. Now records are being written directly to FF
- Removed unfinished flowfile from session, transfered original to "failure". Test cases update as well

This closes #2478.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-03-13 10:07:07 -04:00
m-hogue 6471f66acd NIFI-3402 - Added etag support to InvokeHTTP
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2150.
2018-03-13 10:00:11 +01:00
Deon Huang 7aabb99bcc NIFI-4800 Expose the flattenMode as property in FlattenJSON processor
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2530.
2018-03-13 09:28:22 +01:00
joewitt d78d95ad6f NIFI-4885 fixing checkstyle issue 2018-03-12 22:22:59 -04:00
Pierre Villard 9158c19123 NIFI-4958 - This closes #2529. Fix Travis job status + atlas profile
Signed-off-by: joewitt <joewitt@apache.org>
2018-03-12 16:33:20 -04:00
Ed 373cf090a4 NIFI-4953 - FetchHBaseRow - update log level for "not found" to DEBUG instead of ERROR
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2527.
2018-03-12 20:21:40 +01:00
Matt Gilman b1217f529b NIFI-4885:
- Introducing more granular restricted component access policies.

This closes #2515.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-03-12 14:19:42 -04:00
Matthew Burgess d4632bdd5d
NIFI-4944: Guard against race condition in Snappy for PutHiveStreaming
NIFI-4944: Removed unnecessary synchronized block, added more comments
This closes #2519
2018-03-12 13:22:40 -04:00
Matt Gilman 32a8db2253
NIFI-4936:
- Moving definition of ranger version into root pom.

This closes #2532.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-03-12 10:39:54 -04:00
Jorge Machado 06cb354c5e NIFI-4960: fix object setting. This closes #2531.
Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-03-12 09:44:23 -04:00
Joe Skora 49c7af03c1 NIFI-3039 Provenance Repository - Fix PurgeOldEvent and Rollover Size Limits
* Added low water mark for purgeOldEvents() to more quickly purge after a large spike in events.
* Adjusted rollover high water mark to avoid overrunning "nifi.provenance.repository.max.storage.size".
* Adjusted looping logic in mergeJournals() to use ".firstKey()" instead of ".entrySet().iterator().next()" to avoid unnecessary object creation.

Signed-off-by: Mike Moser <mosermw@apache.org>
This closes #1240
2018-03-09 21:49:13 +00:00
joewitt c71409fb5d
NIFI-4936 trying to quiet down the mvn output a bit so we dont exceed the travis-ci 4MB max
NIFI-4936 updated dependency handling pushing down delcarations where they belong
This closes #2512
2018-03-09 16:34:53 -05:00
Matt Gilman 8f66454455
NIFI-4945:
- Upgrading spring security version.

This closes #2524.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2018-03-09 11:50:26 -08:00
Andy LoPresto e9d585a785
NIFI-4870 Upgraded activemq-client and activemq-broker versions to 5.15.3.
- Updated copyright dates in NOTICE files.
- This closes #2469.
2018-03-09 13:59:13 -05:00
Mike Thomsen 3cdd3f60f6 NIFI-4949 Converted nifi-mongodb-services' unit tests into integration tests so that the @Ignore annotation doesn't have to be removed to make them run.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2525.
2018-03-09 09:36:15 +01:00
Kevin Doran 79fa4ad46f
NIFI-4835 Corrects Swagger spec response types in FlowResource 2018-03-08 14:30:31 -05:00
Kevin Doran fdca7d0a01
NIFI-4859 Corrects Swagger Spec VersionedFlowState allowableValues 2018-03-08 14:30:30 -05:00
Pierre Villard 1f8af1bde3 NIFI-4948 - MongoDB Lookup Service throws an exception if there is no match
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2522
2018-03-08 13:28:41 -05:00
Mike Thomsen dae2b73d94 NIFI-4929 Converted the majority of MongoDB unit tests to integration tests so they can be reliably run with 'mvn -Pintegration-tests integration-test'
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2508.
2018-03-08 19:23:44 +01:00
Andy LoPresto bff5b7ab71 NIFI-4928 Updated BouncyCastle dependencies to version 1.59.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2520.
2018-03-08 09:02:57 +01:00
Mark Payne dfd5339690 This closes #2480. 2018-03-06 15:13:48 -05:00
gardellajuanpablo 2e4ddbf3b2 NIFI-4893 Cannot convert Avro schemas to Record schemas with default arrays
Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-03-06 14:09:39 -05:00
Takanobu Asanuma bfecf0f06c
NIFI-4855:
- Fix the layout of NiFi API document
- update the sub-title to be more detailed
- This closes #2503
2018-03-06 10:30:18 -05:00
Matt Gilman 91f40febeb NIFI-4925:
- Addressing memory leak from lingering authorization results that did not represent actual access attempts. This closes #2511.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-03-05 15:37:31 -05:00
Jeff Storck e916594b69 NIFI-4872 Added annotation for specifying scenarios in which components can cause high usage of system resources.
- Initial set of components marked with the HighResourceUsageScenario annotation.
- Added customized descriptions to SystemResourceConsideration annotations for MergeContent, SplitContent, SplitJson, SplitText, and SplitXml.

This closes #2475.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-03-05 15:32:23 -05:00
Pierre Villard c5c508114a NIFI-4922 - Add badges to the README file
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #2505.
2018-03-03 16:58:21 -08:00
Matthew Burgess dd58a376c9 NIFI-4773 - Fixed column type map initialization in QueryDatabaseTable
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2504.
2018-03-02 17:32:14 +01:00
Bryan Bende 179e967b47 NIFI-4920 Skipping sensitive properties when updating component properties from versioned component. This closes #2505.
Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-03-02 11:15:39 -05:00
Mark Payne 74bb341abc NIFI-2630: Changed name of queue in unit test to be unique in order to avoid getting messages from another test if the other tests fails to properly shutdown the connection. This closes #2458. 2018-03-02 09:37:00 -05:00
Mike Moser 42e6fa42a3 NIFI-2630 Allow PublishJMS to send TextMessages
- Added configurable character set encoding for JMS TextMessages
- Improved PublishJMS/ConsumeJMS documentation
- Validate character set in property validator instead of OnScheduled
2018-03-02 09:11:58 -05:00
Pierre Villard c58b025186 NIFI-4916 - ConvertExcelToCSVProcessor inherit parent attributes. This closes #2500.
Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-03-02 08:47:32 -05:00
Mark Payne b4a4cc564e Fixed failing unit tests: Changed the queues used to unique names so that one test won't interfere with another; also changed JMSPublisherConsumerTest to JMSPublisherConsumerIT since it is an integration test between the publisher and consumer with ActiveMQ as the broker 2018-03-02 08:44:25 -05:00
Mark Payne eb844d8c6f NIFI-4901: This ticket was marked invalid by the creator. This closes #2486. 2018-02-28 14:22:54 -05:00
Derek Straka 44bc2d41d7 NIFI-4880: Add the ability to map record based on the aliases. This closes #2474
Signed-off-by: Derek Straka <derek@asterius.io>
Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-02-28 14:19:53 -05:00
Bryan Bende 5041bea773 NIFI-4839 Improving back-ref support so that ReferenceResolver is passed the option being resolved
- Adding ResolvedReference to encapsulate the results of resolving a back-reference.
- Update README.md
- Added OkResult for delete commands
- Added sync-flow-versions and transfer-flow-version to registry commands
- Returning appropriate status code when exiting standalone mode
- Adding security section to README

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2477.
2018-02-28 17:24:24 +01:00
Andrew Grande 2fd24b78e6 NIFI-4839 - The "Disabled" column had incorrect size and skewed the header 2018-02-28 17:24:15 +01:00
Bryan Bende 1911635a3a NIFI-4839 - Switching standalone mode to default to simple output
- Added pg-status command and improved output of pg-list
- Setting up back-refs for pg-list and using table layout for pg-get-vars and pg-get-version
- Only print usage on errors related to missing/incorrect options
2018-02-28 17:24:05 +01:00
Andrew Grande d1027879eb NIFI-4839 - Fixed handling of a connection object position - it doesn't have one and just returns null (calculated by the UI dynamically) 2018-02-28 17:23:57 +01:00
Bryan Bende b68eebd429 NIFI-4839 - Added abbreviation in simple output for name, description, and comments
- Refactored so that commands produce a result which can then be written or used
- Added support for back-referencing results, initially prototyped by Andrew Grande
- Fixed dynamic table layout when writing simple results
- Added a new command group called 'demo' with a new 'quick-import' command
- Fixes/improvements after previous refactoring
- Created a reusable TableWriter and updating a few result classes to use it
2018-02-28 17:23:44 +01:00
Andrew Grande cc3c1b1714 NIFI-4839 - Implemented nice dynamic table output for all list-XXX commands (in simple mode)
- Better output formatting for 'registry list-buckets'
- Implemented dynamic table formatting for 'registry list-XXX' commands
- Implemented dynamic table formatting for 'nifi list-registry-clients' command
- Implemented dynamic table formatting for 'nifi list-registry-clients' command
- Better handling of non-null, but empty descriptions and commit messages
2018-02-28 17:23:35 +01:00
Bryan Bende 69367ff0bf NIFI-4839 - Updating README and cleaning up descriptions and comments
- Making registryClientId optional and auto selecting when only one is available
- Added delete-bucket command
- Added delete-flow command for registry
2018-02-28 17:23:27 +01:00
Andrew Grande fe71c18ec5 NIFI-4839 - Support both public URLs and local files as inputs for import actions.
- The handling of empty canvas got lost in the merge, causing errors with a new NiFi instance.
- Broaden support for input, now supportes both local files _and_ any public URL with a schema recognized by Java runtime.
2018-02-28 17:23:18 +01:00