forgot to penalize flowfiles sent to failure
NIFI-5274 avoid rollback on StackOverflowError in ReplaceText
add comment
removed penalize on failure, doesn't really apply here
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#2767
- Updated doc for write attributes
- Removing redundant files
- Updates based on review
Changes:
- changed validators for regex params (3)
- removed TriggerWhenEmpty and TriggerSerially.
- removed unused LISTING_LAG_NANOS
- removed unimplemented customValidate
- removed unsused error handling for building/updating request params
- added MIME_TYPE for content
- suppressed output to original relationship when scheduled
- added full path to original/new FF when not found
- added warning messages for huge attribute values.
This closes#2639.
Signed-off-by: Bryan Bende <bbende@apache.org>
SelectHiveQL support only single SELECT statement.
This change adds support for pre- and post- select statements.
It will be useful for configuration queries, i.e. "set tez.queue.name=default", and others.
renamed selectQuery to hqlStatement for better readability
style check correction in comment
removed meaningless examples for post-queries
execute query without result set
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#2695
NIFI-5214 Added support for the new ProxyConfigurationService
NIFI-5214 Integration tests added.
NIFI-5214 Added missing pom.xml and a change from a code review.
NIFI-5214 Added another tag based on code review.
NIFI-5214 Added user-defined header support.
NIFI-5214 Added Basic Auth support.
NIFI-5214 Moved documentation.
NIFI-5214 Fixed checkstyle issues; added changes requested in a review.
NIFI-5214 Added changes requested in a code review.
NIFI-5214 Added verb test and @DynamicProperties
NIFI-5214 Added templated URL support to RestLookupService.
NIFI-5214 Fixed documentation based on code review changes.
NIFI-5214 Changed RestLookupService to use a property descriptor.
NIFI-5214 Updated documentation.
NIFI-5214 Made changes requested in code review.
NIFI-5214 Renamed nifi-standard-web-utils to reflect that it is for tests.
NIFI-5214: Refactor RestLookupService.
1. Added 'Base URL' property to address environment specific part of URL.
2. Removed 'Record Path Property Name' property, because the name of
a resulted record field of a record path can be obtained by field name.
3. Lower cased HTTP method name should be used throughout.
4. Added mimeType require check when body is specified.
5. Added debug log to print HTTP response code.
6. Prepare for NIFI-5287.
7. Fixed that mime.type being used regardless of whether body is
specified or not, caused NullPointerException when 'mime.type' is not
specified when it is not required.
8. Updated documentation.
NIFI-5214 Fixed AWS processors that broke w/ change to TestServer location.
NIFI-5214 Added changes requested in a code review.
Refactored the way to evaluate EL for URL property
- Use PropertyValue instead of PreparedQuery to utilize Variable
Registry.
- Removed BASE_URL because Variable Registry can be used at URL
NIFI-5214 Rebased to use new LookupService method.
This closes#2723.
Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
- Minor adjustments following PR.
- Avoiding additional find operation when authorizing components when populating component details.
- Requiring access to provenance events when downloading content or submitting a replay as they may provide events details.
- Updating the REST API docs detailing the required permissions.
- Updating the wording in the documentation regarding the provenance and data policies.
- Removed the event attributes from the authorization calls that were verifying access to provenance events.
- Only checking content availability when the user is authorized for the components data.
- Addressing typo in JavaDoc.
This closes#2703
NIFI-5287 Removed unneeded property descriptor.
NIFI-5287 Added additional changes from a code review.
This closes#2777.
Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
- Ensuring the proxy headers are considered when redirecting the user following a OIDC or Knox login exchange.
This closes#2763.
Signed-off-by: Bryan Bende <bbende@apache.org>
per review, simplify header building and remove new map
Per review:
- removed apache headers from 3rd party files per https://www.apache.org/legal/src-headers.html#3party
- referenced commit/repo in notice file
- created rat configuration to account for files without headers
Signed-off-by: joewitt <joewitt@apache.org>
NIFI-5231 Minor fixes from code review.
NIFI-5231 Added documentation and made record_count a constant.
NIFI-5231 Added changes requested in a code review.
NIFI-5231 Added changes based on code review.
This closes#2737.
Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
NIFI-5288 Made changes based on code review.
NIFI-5288 Theoretically supports nested arrays.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#2778
- Disabling the resize of SlickGrid when there is an active edit occuring. This will prevent the accidental closure of the current edit.
- Triggering a resize when the active editor closes.
This cloese #2766
Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
commit 9c31e45d3f
Author: Joe Trite <joetrite@gmail.com>
Date: Mon Mar 13 07:55:19 2017 -0400
NIFI-1705 Adding AttributesToCSV processor
commit 5e9afa9ccf
Merge: 3177eb18574cbfc4b6
Author: Joe Trite <joetrite@gmail.com>
Date: Sat Mar 4 08:12:39 2017 -0500
Merge remote-tracking branch 'origin/master'
# Conflicts:
# nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ScanAttribute.java
commit 3177eb1852
Author: Joe Trite <joetrite@gmail.com>
Date: Sat Mar 4 08:09:06 2017 -0500
NIFI-3497 Converted properties to use displayName, changed validator on demarcator property and created a method for duplicate code.
commit 74cbfc4b69
Merge: a974f78e0f8cad0f8c
Author: Joe Trite <joetrite@gmail.com>
Date: Sat Mar 4 07:47:46 2017 -0500
Merge branch 'master' into master
commit a974f78e03
Author: Joe Trite <joetrite@gmail.com>
Date: Sat Mar 4 07:43:02 2017 -0500
NIFI-3497 Converted properties to use displayName, changed validator on demarcator property and created a method for duplicate code.
commit 1bfaef8e87
Merge: 65ed46de989ec68d14
Author: Joe Trite <joetrite@gmail.com>
Date: Fri Mar 3 08:01:59 2017 -0500
Merge branch 'master' of https://github.com/joetrite/nifi
commit 65ed46de9a
Author: Joe Trite <joetrite@gmail.com>
Date: Fri Feb 24 18:09:36 2017 -0500
NIFI-3497 - fixing Pcontrib issues
commit c5d52cf6f0
Author: Joe Trite <joetrite@gmail.com>
Date: Thu Feb 23 10:19:01 2017 -0500
NIFI-3497 - excluding test files
Adding new test data files to exclude list.
commit b195934113
Author: Joe Trite <joetrite@gmail.com>
Date: Wed Feb 22 16:48:10 2017 -0500
NIFI-3497 - New dictionary files for test
Adding new dictionary files to support metadata dictionary option.
commit e296268f39
Author: Joe Trite <joetrite@gmail.com>
Date: Wed Feb 22 16:46:13 2017 -0500
NIFI-3497 test cases for metadata updates
Adding test cases to support metadata option update.
commit de7e348e62
Author: Joe Trite <joetrite@gmail.com>
Date: Wed Feb 22 16:36:08 2017 -0500
NIFI-3497 - Added metadata option
Added optional to post additional metadata as new attributed if a match is found in the dictionary.
commit 89ec68d14b
Author: Joe Trite <joetrite@gmail.com>
Date: Fri Feb 24 18:09:36 2017 -0500
NIFI-3497 - fixing Pcontrib issues
commit d71426037b
Author: Joe Trite <joetrite@gmail.com>
Date: Thu Feb 23 10:19:01 2017 -0500
NIFI-3497 - excluding test files
Adding new test data files to exclude list.
commit a7a7b6ace8
Author: Joe Trite <joetrite@gmail.com>
Date: Wed Feb 22 16:48:10 2017 -0500
NIFI-3497 - New dictionary files for test
Adding new dictionary files to support metadata dictionary option.
commit 8eb54a5019
Author: Joe Trite <joetrite@gmail.com>
Date: Wed Feb 22 16:46:13 2017 -0500
NIFI-3497 test cases for metadata updates
Adding test cases to support metadata option update.
commit f52e1f2a06
Author: Joe Trite <joetrite@gmail.com>
Date: Wed Feb 22 16:36:08 2017 -0500
NIFI-3497 - Added metadata option
Added optional to post additional metadata as new attributed if a match is found in the dictionary.
NIFI-1705 Adding AttributesToCSV processor.
NIFI-1705 Adding AttributesToCSV processor.
NIFI-1705 Adding AttributesToCSV processor.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#2711
- Preventing duplicate bulletins that are registried under different contexts (e.g. Controller and Component).
This closes#2771.
Signed-off-by: Mark Payne <markap14@hotmail.com>
NIFI-5241: Extended timeout that Jetty uses before closing an active HTTP connection. Because the UI refreshes every 30 seconds by default, and the Jetty connection pool times out every 30 seconds by default, we very frequently saw new HTTP connections being created for the UI refreshes. This resulted in 4 new connections and 4 SSL handshakes occurring every 30 seconds. By extending the timeout, we now see those connections being reused and SSL Handshakes no longer occurring frequently
NIFI-5241: Set Jetty idle timeout to double the amount of time for browser to refresh
NIFI-5241: Fixed synchronization issue with EventSumValue
This closes#2752
NIFI-5041: fixes http client version issue
Change-Id: I1b87ec4752ff6e1603025883a72113919aba5dd4
NIFI-5041: fixes Kerberos configuration
Change-Id: I868fdf3ea7cfd28cf415164e420f23bf3f6eefeb
NIFI-5041: adds new NOTICE entries
NIFI-5041: yields processor if no session is available, fixes error handling in session manager thread, fixes error returned in KerberosKeytabSPNegoScheme on authentication failure
Change-Id: I443e063ae21c446980087e5464a4b70373d730f6
NIFI-5041: makes the session manager thread exceptions visible to the users
Change-Id: I33fde5df6933cec2a87a4d82e681d4464f21b459
NIFI-5041: adds special SessionManagerException to identify error occurred on session manager thread
Change-Id: I25a52c025376a0cd238f14bda533d6f5f3e5fb4a
This closes#2630
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
- Created resuable KeytabUser and KeytabConfiguration in nifi-security-utils
- Refactored Solr processors to use a KeytabControllerService and no longer rely on JAAS system property
- Wrapped all calls in SolrProcessor onTrigger in a doAs when kerberos is enabled
- Added IT tests against MiniKDC
- This closes#2674
- Fixing missing message when a node is disconnected from a cluster.
- Updating endpoints to accept a flag to allow for changes to be made to a disconnected node.
- Updating custom UIs to acknowledge disconnected nodes prior to performing modifications.
- Avoid triggering async validation for each update to component when instantiating a template (such as copy/paste or templates). Added debug logging to indicate when and why we are triggering validation; removed unit test that made poor assumptions about the inner workings of the FlowSynchronizer that resulted in failures when we make calls into processors that the unit test doesn't know about"
This closes#2731.
- PR Fix - 'Execution' dropdown will now be shown in all cases
- Annotated ListGCSBucket with PrimaryNodeOnly
This closes#2509.
Signed-off-by: Mark Payne <markap14@hotmail.com>
Added license to YandexTranslate NOTICE file
Updated to use StringUtils.isBlank for detecting sourceLanguage field being blank and languages to file to align with new logic
Signed-off-by: Mark Payne <markap14@hotmail.com>
NIFI-4987: PR Review Fixes - Reverted getAndPutIfAbsent and added TTL setting with a different approach
NIFI-4987: PR Review Fixes - Added TTL to putIfAbsent()
This closes#2726.
Signed-off-by: Bryan Bende <bbende@apache.org>
NIFI-5052 fixed checkstyle issue.
NIFI-5052 Added changes requested in a code review.
NIFI-5052 Fixed a typo.
NIFI-5052 Added changes requested in a code review.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#2616
- Added ProxyConfigurationService to manage centralized proxy
configurations
- Adopt ProxyConfigurationService at FTP and HTTP processors
NIFI-4175 - Add HTTP proxy support to *SFTP processors
This closes#2018.
Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
NIFI-4199: Add ProxyConfigurationService to SFTP processors
- Fixed check style issue
- Use the same proxy related PropertyDescriptors from FTPTransfer and
SFTPTransfer
- Dropped FlowFile EL evaluation support to make it align with other
processors spec, Now it supports VARIABLE_REGISTRY
- Added ProxyConfigurationService to SFTP processors
- Added SOCKS proxy support to SFTP processors
NIFI-4199: Added ProxyConfigurationService to ElasticsearchHttp processors
- ElasticsearchHttp processors now support SOCKS proxy, too
- Added proxy support to PutElasticsearchHttpRecord
- Moved more common property descriptors to
AbstractElasticsearchHttpProcessor and just return static unmodifiable
property descriptor list at each implementation processors
NIFI-4196 - Expose AWS proxy authentication settings
NIFI-4196 - Fix jUnit errors
This closes#2016.
Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
NIFI-4199: Add ProxyConfigService to AWS processors
- Applied ProxyConfigService to S3 processors
- Added proxy support to following processors:
- PutKinesisFirehose, PutKinesisStream
- PutDynamoDB, DeleteDynamoDB, GetDynamoDB
- PutKinesisStream
- All AWS processors support HTTP proxy now
NIFI-4199: Add ProxyConfigService to Azure processors
NIFI-4199: More explicit validation and docs for Proxy spec
- Each processor has different supporting Proxy specs
- Show supported spec to ProxyConfigurationService property doc
- Validate not only Proxy type, but also with Authentication
NIFI-4199: Incorporated review comments
- Fixed TestListS3 property descriptor check
- Separate name and displayName
This closes#2016
This closes#2018
This closes#2704
Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
NIFI-5204: If processor joins cluster and inherits 'disabled' state but is still stopping, ensure that the state becomes disabled when the processor finishes stopping
This closes#2713
The bootstrap process (RunNiFi) detects Java 9 and adds "--add-modules=java.xml.bind" to the command to start NiFi
Updated OSUtils to detect Java 9 and reflectively invoke the Process.pid() method to get the PID of the NiFi process
Added java debug variable to nifi.sh to allow debugging of the bootstrap process (RunNiFi)
This closes#2708
Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
NIFI-950: Still seeing some slow response times when instantiating a large template in cluster mode so making some minor tweaks based on the results of CPU profiling
NIFI-5112: Refactored FlowSerializer so that it creates the desired intermediate data model that can be serialized, separate from serializing. This allows us to hold the FlowController's Read Lock only while creating the data model, not while actually serializing the data. Configured Jersey Client in ThreadPoolRequestReplicator not to look for features using the Service Loader for every request. Updated Template object to hold a DOM Node that represents the template contents instead of having to serialize the DTO, then parse the serialized form as a DOM object each time that it needs to be serialized.
NIFI-5112: Change ThreadPoolRequestReplicator to use OkHttp client instead of Jersey Client
NIFI-5111: Ensure that if a node is no longer cluster coordinator, that it clears any stale heartbeats.
NIFI-5110: Notify StandardProcessScheduler when a component is removed so that it will clean up any resource related to component lifecycle.
NIFI-950: Avoid gathering the Status objects for entire flow when we don't need them; removed unnecessary code
NIFI-950: Bug fixes
NIFI-950: Bug fix; added validation status to ProcessorDTO, ControllerServiceDTO, ReportingTaskDTO; updated DebugFlow to allow for pause time to be set in the customValidate method for testing functionality
NIFI-950: Addressing test failures
NIFI-950: Bug fixes
NIFI-950: Addressing review feedback
NIFI-950: Fixed validation logic in mock framework
This closes#2693
NIFI-5145 Fixed a mistake with evaluateAttributeExpressions found in a code review.
NIFI-5145 Removed a property descriptor from GetHBase that was a duplication of the one in VisibilityFetchSupport.
NIFI-5145 Added comments in the code that were requested in the review.
NIFI-5145 Fixed check style issue.
NIFI-5145 Fixed a few improperly scoped items in ListFileTransfer that impacted the FTP processor(s).
NIFI-5145 Changed which override gets called based on code review feedback.
NIFI-5145: Reverted changes to ListFileTransfer re ExpressionLanguageScope
NIFI-5145: Removed TODO from MockPropertyValue javadoc
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#2672
* Avoid exception by providing id in DeleteMessageBatchRequestEntry
* Include receipt handle property descriptor for user configuration
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#1888.
- Removing unnecessary logic in the zoom handler since the zoom event is no longer triggered during onClick.
This closes#2692
Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
Support for the new option to keep empty captures. I did not add to the GrokReader because I am not sure of the effect on
the schema extraction.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#2698.
- Separated Hook message de-duplication logic from NiFiAtlasHook to
NotificationSender
- NiFiAtlasHook used to send individual CREATE_ENTITY messages for each entities,
this commit changed it to bundle all new entities into a single
CREATE_ENTITY to preserve entity creation order, so that new DataSet
entities can be referred from new nifi_flow_path entities
- Added more unit tests
This closes#2542
Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
NIFI-4637 Removed integration test and updated the hbase client from 1.1.2 to 1.1.13 which is the final version for 1.1.X
NIFI-4637 Fixed EL support issue w/ tests.
NIFI-4637 Added more documentation to DeleteHBaseCells.
NIFI-4637 changed PutHBaseCell/JSON to use dynamic properties instead of a 'default visibility string.'
NIFI-4637 Added changes requested in a code review.
NIFI-4637 Moved pickVisibilityString to a utility class to make testing easier.
NIFI-4637 Added additionalDetails.html for PutHBaseRecord.
NIFI-4637 Added additional documentation and testing.
NIFI-4637 Added documentation for DeleteHBaseCells.
NIFI-4637 Added pickVisibilityLabel support to PutHBaseRecord and updated documentation to reflect that.
NIFI-4637 Reverted version bump to hbase client.
This closes#2518.
Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
- Updating how the nf-reporting-task module is injected to the nf-controller-service module.
This closes#2688
Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
- Added logic to check for simultaneous configuration of HTTP and HTTPS connectors in JettyServer.
- Added test logging resources. Added unit tests.
- Refactored shared functionality to generic method which accepts lambdas.
Fixed unit test with logging side effects.
- Added note about exclusive HTTP/HTTPS behavior to Admin Guide. Fixed typos.
This closes#2683.
Signed-off-by: Kevin Doran <kdoran@apache.org>
- Forcing FileSystem statistics thread to be interrupted when HDFS processors are stopped
- Stop creating temp components during import from registry, use bundle info instead
This closes#2668.
Signed-off-by: Mark Payne <markap14@hotmail.com>
Fixed bug in unit test
Addressed review feedback/addressed issue where if a group is moved inside another group, the descendant processors of the moved group did not have their service references updated properly. Also addressed an issue where if a service is defined in Group A, then Group B lives within Group A and has a processor that references a service at the level of Group A, we allowed user to move Group B outside of Group A (but wouldn't allow the processor to be moved out of scope by itself).
This closes#2678
QueryDatabaseTable and GenerateTableFetch processors were not able to
use max value state as expected, if max value column was wrapped with
escape characters. Due to a mis-match between computed state keys
and actual keys used in the managed state. State keys computed by
getStateKey method included escape characters while actual stored keys
did not. Resulted querying the same dataset again and again.
This commit added unwrapIdentifier method to DatabaseAdapter class to
remove database specific escape characters for identifiers such as table
and column names, so that max value state keys are populated correctly
even if identifiers are wrapped with escape characters.
This commit also added new DatabaseAdapter for MySQL, to handle MySQL
specific identifier escape with back-ticks.
This closes#2424
Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
Support creating target parent directories even if directory listing is disabled.
fixed typo in doc
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#2642
- Adding UI controls for terminating hung threads.
- Showing current number of terminated threads.
- Fixing issue when replicating terminate threads request throughout the cluster.
This closes#2607.
Signed-off-by: Mark Payne <markap14@hotmail.com>
- Allowing the enable/disable buttons to be active under the same conditions as the start/stop buttons.
This closes#2633.
Signed-off-by: Mark Payne <markap14@hotmail.com>
Adding Test Cases for PutSolrRecord Processor
Adding PutSolrRecord Processor in the list of Processors
Resolving checkstyle errors
Resolving checkstyle errors in test classes
Adding License information and additional information about the processor
1. Implementing Batch Indexing 2. Changes for nested records 3. Removing MockRecordParser
Fixing bugs with nested records
Updating version of dependencies
Setting Expression Language Scope
This closes#2561.
Signed-off-by: Bryan Bende <bbende@apache.org>
- Added dedicated conditions for Funnels
- Fixed stale Javadoc
- Stopped caching hasNonLoopConnection variable
- Grouped some conditions to isSourceComponent variable for better
readability
This closes#2634.
Signed-off-by: Mark Payne <markap14@hotmail.com>
Log debug message when ParseException is thrown.
Log warning message if other unknown Exception is thrown.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#2645
Existing web error index page suggests user may have meant /nifi/ and crafts a contextPath link, may as well redirect them automatically. Non-human (i.e. curl/wget) can still retrieve the original source without being redirected.
Signed-off-by: joewitt <joewitt@apache.org>
- Only include Maximum Value columns in the type map.
- Squashed commits in the previous PR
- Rebased against the latest master
- Added stop method to GenerateTableFetch so that it refreshes the
column type map when it gets restarted
- Fixed whitespacing around if/for statement
- Updated expressionLanguageSupported value since it is not auto-merged
correctly
This closes#2618.
Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
- Fixing process group audit advice.
- Setting spring security user in background threads.
- Removing unnecessary overloaded methods.
This closes#2626.
Signed-off-by: Mark Payne <markap14@hotmail.com>
Fixed dependency issue by providing a local JSON reader
Rebased + fixed conflict + updated versions in pom + EL scope
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#2575
- take into account input requirement for documentation rendering
- Renamed variable registry scope and added comments
- Doc + change in mock framework to check scope + update of components + UI
- Processing properties and property descriptors in Controller Service referencing components unconditionally.
This closes#2602.
Signed-off-by: Mark Payne <markap14@hotmail.com>
- Ensuring appropriate response in checkAuthorization when user is null.
- Ensuring the user reference is passed down when applying variable changes.
This closes#2598.
Signed-off-by: Mark Payne <markap14@hotmail.com>
The bug is not reproducible in Jython (it handles modules with every script reload). But Groovy loads JARs and dirs with classes only on setup.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#2584
NIFI-4325 Cleaned up how ElasticSearch client service builds SSLContext, added query attribute to flowfiles and other changes requested in a code review.
This closes#2113.
Signed-off-by: Joe Percivall <JPercivall@apache.org>
- Fixing required permission for PutParquet.
NIFI-5008:
- Ensuring all restricted components are tagged as such.
This closes#2583.
Signed-off-by: Bryan Bende <bbende@apache.org>
Refactor the two existing CSV classes to inherit from an abstract base class that parses the data
Add unit tests for calls to nextRecord that do not coerce types
This closes#2473.
Signed-off-by: Derek Straka <derek@asterius.io>
Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
If failed, use first method and throw error if not working.
Array.length can never be less then 0
This closes#2499.
Signed-off-by: Mark Payne <markap14@hotmail.com>
The feature allows users to convert from non-integral types to the correct underlying type. The
original behavior is maintained; however, now simple conversions take place automatically for some
logical types (date, time, and timestamp).
This closes#2526.
Signed-off-by: Derek Straka <derek@asterius.io>
Signed-off-by: Mark Payne <markap14@hotmail.com>
- Fixing RPG port merging.
- Adding unit tests.
- Removing unecessary sorting that wasn't maintained while clustered.
This closes#2551.
Signed-off-by: Mark Payne <markap14@hotmail.com>
address review comments
NIFI-4809 - Added Record Writer property
added unit tests and additional details doc
review comments
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#2430
Implemented review feedback. Refactored data model to make the API cleaner and delineate more along the lines of what permissions are required in order to see which details
Implementing review feedback
Removed sensitive information from the diagnostics reports
Fixed bug in merging logic for GCDiagnosticsSnapshots
This closes#2468
SelectHiveQL should emit FETCH instead of CONTENT_MODIFIED when it has
incoming connections.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
This closes#2543.
After first failure in customValidate, stop printing logs until any property is changed
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#2537
- New processor for scanning HBase records based on verious params like range of rowkeys, range of timestamps. Supports result limit and reverse scan.
- Adds Atlas Support for ScanHBase processor
- Fixed not recent version of FF
- Formatting and Style changes
- Single line to multiline if-then statements
- Removed HTML formatting that is not used for doc generation
- Fixed issue with limitRows
- Fixed issue with filter expression
- Refactored "processRows"
- Fixed possible NPE for bulkSize var
- Changed provenance to "receive" to indicate new data from external source.
- Updated min/max timestamp custom validation
- JSON array support
- Removed in-memory caching for records. Now records are being written directly to FF
- Removed unfinished flowfile from session, transfered original to "failure". Test cases update as well
This closes#2478.
Signed-off-by: Bryan Bende <bbende@apache.org>
* Added low water mark for purgeOldEvents() to more quickly purge after a large spike in events.
* Adjusted rollover high water mark to avoid overrunning "nifi.provenance.repository.max.storage.size".
* Adjusted looping logic in mergeJournals() to use ".firstKey()" instead of ".entrySet().iterator().next()" to avoid unnecessary object creation.
Signed-off-by: Mike Moser <mosermw@apache.org>
This closes#1240
- Addressing memory leak from lingering authorization results that did not represent actual access attempts. This closes#2511.
Signed-off-by: Mark Payne <markap14@hotmail.com>
- Initial set of components marked with the HighResourceUsageScenario annotation.
- Added customized descriptions to SystemResourceConsideration annotations for MergeContent, SplitContent, SplitJson, SplitText, and SplitXml.
This closes#2475.
Signed-off-by: Mark Payne <markap14@hotmail.com>
- Added configurable character set encoding for JMS TextMessages
- Improved PublishJMS/ConsumeJMS documentation
- Validate character set in property validator instead of OnScheduled
...Search results
* Separated the search functionality.
* Added a unit test.
* Added the PG info to UI (a mere draft).
* Introduce the nearest versioned group
* Removed the top level group results in favour of the nearest versioned group.
* This closes#2364
NIFI-4772: Introduced a new thread-pool with 2 threads that will be used for monitoring lifecycle task. This means that if all threads in the timer-driven thead pool are blocked by processors that don't complete their @OnScheduled methods, we have a separate thread pool that at least gives us a chance of interrupting those threads
NIFI-4772: Remove unused import
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#2403
HBase_1_1_2_ClientService performance dropped due to accessing HBase
admin upon every FlowFile transfer. Fixed that by getting master address
only once when the service is enabled and a connection is established.
This closes#2464
- Everywhere that we ignore adding remote ports we should ignore removing remote ports as well in flow diffs
This closes#2462.
Signed-off-by: Bryan Bende <bbende@apache.org>
- When Jetty initializes fails, performing a shutdown sequence to ensure all allocated resources are released.
This closes#2455.
Signed-off-by: Bryan Bende <bbende@apache.org>
Before this fix, MergeContent only processed the first bin even if there
were multiple bins.
There were two unit tests marked with Ignore those had been
failing because of this.
This closes#2444.
Signed-off-by: Mark Payne <markap14@hotmail.com>
- Applied the same scale adjustment not only to BigDecimal inputs, but
also to Double values.
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#2450
Adds stricter checks in AuthorizerFactoryBean for unique ids within
a given type of provider and requires unique providers in composite
and composite-configurable user group providers. Failed validation
checks cause startup to fail. Adds test cases for these new rules.
This closes#2419.
Signed-off-by: Bryan Bende <bbende@apache.org>
Added HostHeaderHandler regression unit tests before adding new functionality.
Added logic for parsing nifi.web.proxy.host property.
Added default property in nifi.properties and pom.xml.
Added logic for IPv6 and custom default hostnames.
Improved error messaging.
Added HostHeaderHandler unit tests.
Disabled HostHeaderSanitizationCustomizer in HTTP mode.
Fixed HTML escaping in error message.
Improved error message.
Added failing unit test for parsing custom hostnames.
Fixed custom hostname parsing.
Fixed unit tests.
Added TODO for IPv6 custom hostname parsing and unit test.
Added IPv6 custom hostname parsing and unit tests.
Fixed checkstyle issues.
Removed empty element in host list when no value defined.
Improved error message formatting.
Added unit tests.
Removed HostHeaderSanitizationCustomizer.
Removed InvalidPropertiesFormatException from NiFiProperties.
Removed InvalidPropertiesFormatException from HostHeaderHandler.
This closes#2415
using per-record flowfiles
NIFI-4428: Added initial L&N, fixed dependency hierarchy
NIFI-4428: Exposed additional ZK properties
NIFI-4428: More work to reduce dependencies in Druid artifacts
NIFI-4428: Removed inaccessible attribute
NIFI-4428: Fixed typo in property description
NIFI-4428: Updates to Druid NAR L&N
NIFI-4428: Updated LICENSE to refer to ICU License for icu4j
This commit add back the existing capability for those Processors to use
incoming FlowFile attributes to compute account name and account key,
which had been removed by NIFI-4004.
Also, the same capability is added for SAS token.
This closes#2400.
Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
- Only showing the import link in the new Process Group dialog when initiating from the toolbar.
This closes#2395.
Signed-off-by: Mark Payne <markap14@hotmail.com>
Adds support for NiFI Registry config files to the encrypt-config tool
in NiFi Toolkit.
Also adds decryption capability to encrypt-config tool.
This closes#2376.
Signed-off-by: Andy LoPresto <alopresto@apache.org>
NIFI-4436: Fixed a bug that caused a flow not to successfully change version if a connection is added to an existing component and that component is running at time of version change
NIFI-4436: Fixed bug with ordering of controller services being enabled and disabled
NIFI-4436: Fixed bug that prevented local input and output ports from being stopped and started as needed
NIFI-4436: Fixed bugs around referencing controller services that are at a higher level than the versioned flow
NIFI-4436: Ensure that we clear components from FlowController's cache when removed and that they are added to cache when created.
NIFI-4436: Fixed error message coming back if component is invalid when trying to be restarted/re-enabled
NIFI-4436: Addressed issue with children of a removed process group not being considered 'affected components' and as a result not being stopped/disabled/restarted/re-enabled
This closes#2219.
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
- Minor tweak to when the max height for the options list is calculated per PR comments.
- Resolving logic issue in two phase commit when updating variable registry.
- Fixing variable visibility.
- Updating buckets permissions based on new model.
- Adding check to ensure that flow name is non null before checking the length.
- Adding versioned flow state to the Process Group tab in the Summary table.
- Fixing issue with navigating to Controller Services from the local changes dialog.
- Code clean up.
- Improved error handling.
- Minor UX improvements.
- Always showing Process Group state to complement the aggregation counts.
- Adding the Process Group state to the top status bar.
- Code clean up.
- Improved error handling.
- Minor UX improvements.
- Adding message to indicate that variables do not support sensitive values.
- Preventing a user from changing the flow version to the current version.
- Only presenting buckets a user has appropriate permissions to.
- Adding basic auditing to the version control actions.
- Updating front end to use version control state/status.
- Fixing copy/paste issue during revert local changes.
- Code clean up in the breadcrumbs.
- Update VersionsResource authorization and two phase commit object usage.
- Addressing miscellaneous minor UX issues.
- Updating comments UX for all components.
- Updating the styling of PG and RPG to be more consistent.
- Adding the icons for nested versioned process groups.
- Calculating the number/states of nested versioned process groups.
- Fixing default border radius.
- Code clean up.
- Ensuring component visibility is updated after updating/reverting.
- Fixing sort on component name in local changes dialog
NIFI-4526:
- Added front end controls for updating RPG target URL.
- Clearing bucket/flow/versions when changing the selected registry/bucket.
- Using the versioned flow to get the group name when importing.
- Adding menu items for viewing local changes.
- Showing local changes during revert request.
- Adding the version number to the start version control, commit, and change version dialog.
- Showing a loading item in the combo's while querying for the registries, buckets, and flows.
- Adding tooltips to display version control information on the canvas.
- Adding progress bar dialogs for changing version and reverting local changes.
- Updating canvas and breadcrumb according to the version control state.
- Updating to use registry name, bucket name, and flow name where appropriate.
- Added the import dialog for importing a versioned flow into a new process group.
- Added the change version dialog for upgrading/downgrading a versioned flow.
- Adding support to save a version of a flow based on a selected Process Group.
- Adding support for revert changes back to the most recent version.
- Adding support to disconnect from version control.
- Moving the version control information out of the entity objects and into the dto's.
- Fixing checkstyle issues.
NIFI-4502:
- Updating the UI to allow for the user to register registry clients.
- Updating the version control menu item names.
- Initial checkpoint: able ot start version control and detect changes, in standalone mode, still 'crude' implementation
- Checkpoint: Can place flow under version control and can determine if modified
- Checkpoint: Change version working in some cases. Does not work if processor removed because COMPONENT_REMOVED type has ComponentA whose ID is the VersionedComponentID but we are trying to call ProcessorDAO.get() with this ID
- Checkpoint: Able to change flow from Version 1 to Version 2 and back. Not yet tested with controller services. Have not tried changing/removing connections. Not cluster-friendly yet. All inline, not in background. Have not taken into account ports, funnels, remote ports, etc. Have not tested with Labels yet
- Checkpoint after implementing ClusterReplicationComponentLifecycle instead of JerseyClientComponentLifecycle
- Checkpoint: Updated to allow starting version control and updating version in clustered mode
- Checkpoint: Updated versioning endpoint so that when version of a flow is updated, the bundle information is populated and the snapshot is replicated to the cluster.
- Checkpoint: Implemented endpoint for reverting to previously sync'ed version of a flow and updated version control endpoint so that Process Group can be pushed as a new version to existing flow instead of only creating a new flow
- Checkpoint: Updated so that if a Process Group is under Version Control and it has a child Process Group, which is also under Version Control, we can handle that gracefully. Not yet tested because it depends on updates to the nifi-registry module, which can't be compiled due to maven dependency conflicts
Signed-off-by: Matthew Burgess <mattyb149@apache.org>
This closes#2307
Replaced star imports, removed unused import
Added explanation for invalid Expression
Bryan Bende
Mike Moser
Ed
Pierre Villard
Dustin Rodrigues
Matthew Burgess
Ed B
Dongkyu Hwangbo
Koji Kawamura
zenfenan
Mike Thomsen
Mark Payne
Matt Gilman
Mark Bean
Marco Gaido
amitt90
thenatog
Otto Fowler
Junegunn Choi
Aldrin Piri
Joe Trite
Peter Toth
Nicolas Sanglard
Seiji Sogabe
Scott Aslan
veteranbv
Frederik Petersen
Max Viazovskyi
Andrew Psaltis
Jeff Storck
James Wing
Arun Manivannan
JohannesDaniel
Fokko Driesprong
Gardella Juan Pablo
himanshu
patricker
Andy LoPresto
Michal Misiewicz
sbouchex
joewitt
abhinavrohatgi30
Alex Moundalexis
Joe Percivall
mans2singh
Vi On
Andrew Lim
Arun A K
Derek Straka
Internet
Julian Gimbel
Jorge Machado
Deon Huang
m-hogue
Joe Skora
Kevin Doran
Takanobu Asanuma
yuri1969
jlwt90
dorian.bugeja
Peter G. Horvath
Joey Frazee
Senthilannaswamy
vvaks
Patrice Freydiere
Andy LoPresto
Kevin Doran
Andre F de Miranda