Commit Graph

722 Commits

Author SHA1 Message Date
Yolanda M. Davis 8e1452a3f3
NIFI-6649 - added separate query interval configuration for observation queries
NIFI-6649 - documentation update

NIFI-6649 - add debug logging for score and prediction information

NIFI-6649 - fix to ensure counts return minimum value of 0 if not infinite or NaN

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3719
2019-09-12 12:13:19 -04:00
mans2singh 9ec6314687
NIFI-6642 - JsonPath support for adding array element
NIFI-6642 - Updated json file used in tests

Clarified function doc

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3711
2019-09-11 11:07:59 -04:00
Andy I. Christianson 8a8b9c1d08
NIFI-6510 - Analytics framework (#3681)
* NIFI-6510 Implement initial analytic engine

* NIFI-6510 Implemented basic linear regression model for queue counts

* NIFI-6510 Initial analytics REST endpoint and supporting objects

* NIFI-6510 Connect the dots for StatusAnalytics -> API

* NIFI-6510 Added poc engine with prediction model caching

(cherry picked from commit e013b91)

DFA-9 - updated logging and corrected logic for checking if not in backpressure

(cherry picked from commit a1f8e70)

* NIFI-6510 Updated objects and interfaces to reflect 4 prediction metrics

(cherry picked from commit 050e0fc)

(cherry picked from commit 9fd365f)

* NIFI-6510 adjustments for interface updates, added call to StandardEventAccess, updated interface to use connection id

(cherry picked from commit 14854ff)

DFA-9 - reduced snapshot interval to 1 minute

(cherry picked from commit 36abb0a)

* NIFI-6510 Split StatusAnalytics interface into Engine and per-Connection versions

* NIFI-6510 Remove redundant connection prediction interfaces as we can just use ConnectionStatusAnalytics directly

* NIFI-6510 Revert "DFA-9 Remove redundant connection prediction interfaces as we can just use ConnectionStatusAnalytics directly"

This reverts commit 5b9fead1471059098c0e98343fb337070f1c75c1.

* NIFI-6510 Added prediction fields for use by UI, still need to be populated

* NIFI-6510 Analytics Framework Introduction (#10)

* DFA-9 - Initial refactor for Status Analytics - created additional interfaces for models, refactored callers to use StatusAnalytics objects with connection context. Implemented SimpleRegression model.

DFA-9 - added logging

* DFA-9 - relocated query window to CSA from model, adding the prediction percentages and time interval

* DFA-9 - checkstyle fixes

* NIFI-6510 Add prediction percent values and predicted interval seconds

(cherry picked from commit e60015d)

* NIFI-6510 Changes to inject flowManager instead of flow controller, also changes to properly reflect when predictions can be made vs not.

(cherry picked from commit 6fae058)

* NIFI-6510 Added tests for engine

(cherry picked from commit 6d7a13b)

* NIFI-6150 Added tests for connection status analytics class, corrected variable names

(cherry picked from commit 58c7c81)

* NIFI-6150 Make checkstyle happy

(cherry picked from commit b6e35ac)

* NIFI-6150 Fixed NaN check and refactored time prediction. Switched to use non caching engine for testing

* NIFI-6510 Fixed checkstyle issue in TestConnectionStatusAnalytics

* NIFI-6510 Adjusted interval and incorporated R-squared check

Updates to support multiple variables for features, clearing cached regression model based on r-squared values

Added ordinary least squares model, which truly uses multivariable regression. Refactor of interfaces to include more general interface for variate models (that include scoring support).

Ratcheck fixes

Added test for SimpleRegression. Minor fix for OLS model

fixed test errors

fixed checkstyle errors

(cherry picked from commit fab411b)

* NIFI-6510 Added property to nifi.properties - Prediction Interval for connection status analytics (#11)

* NIFI-6566 - Refactor to decouple model instance from status analytics object. Also allow configurable model from nifi.properties

NIFI-6566 - changes to allow scoring configurations for model in nifi.properties

NIFI-6566 - added default implementation value to NiFiProperties

NIFI-6566 - correction to default variable name in NiFiProperties, removed unnecessary init method from ConnectionStatusAnalytics

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3663

* NIFI-6585 - Refactored tests to use mocked models and extract functions.  Added check in ConnectionStatusAnalytics to confirm expected model by type

* NIFI-6586 - documentation and comments

This closes NIFI-6586

Signed-off-by: Andrew I. Christianson <andy@andyic.org>

* NIFI-6568 - Surface time-to-back-pressure and initial predictions in the UI
* Add multi-line tooltips with detail for connection queue back pressure graphics.
* Add estimated time to back pressure to connections summary table.
* Add back pressure prediction ticks.
* add moment.js to format predicted time to back pressure
* tweak summary table headings to match data displayed. re-order connection summary columns

* NIFI-6568 - Properly sort the min estimated time to back pressure in the connection summary table. Also added a js doc comment.

* NIFI-6510 - add an enable/disable property for analytics

* NIFI-6510 - documentation updates for enable/disable property

* NIFI-6510 - UI: handle the scenario where backpressure predictions are disabled (#3685)

* NIFI-6510 - admin guide updates to further describe model functionality

* NIFI-6510 - code quality fixes (if statement and constructor)

* NIFI-6510 - log warnings when properties could not be retrieved. fixed incorrect property retrieval for score threshold

* NIFI-6510 Extract out predictions into their own DTO

* NIFI-6510 Optimize imports

* NIFI-6510 Fix formatting

* NIFI-6510 Optimize imports

* NIFI-6510 Optimize imports

* NIFI-6510 - Notice updates for Commons math and Caffeine

* NIFI-6510 - UI updates to account for minor API changes for back pressure predictions (#3697)

* NIFI-6510 - Fix issue displaying estimated time to back pressure in connection summary table when only one of the predictions is known.

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3705

* NIFI-6510 Rip out useless members

* NIFI-6510 - dto updates to check for -1 value

* NIFI-6510 - checkstyle fix

* NIFI-6510 - rolled back last change and applied minNonNegative method

* NIFI-6510 Rip out useless members
2019-09-09 15:37:11 +00:00
mans2singh 5df6b0edbb
NIFI-6546 - Add JsonPath set value support
NIFI-6546 - Addressed review comments (double semi-colon, javadoc, error message, null init)
NIFI-6546 - Refactored test and added log error for JsonPath exception based on review
NIFI-6546 - Refactored tests based on review comments
NIFI-6546 - Removed redundant phone check and added constant for empty path
NIFI-6546 - Added brackets based on review comments

This closes #3646

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2019-08-29 18:13:09 -04:00
Alessandro D'Armiento bb9758be2c
NIFI-6500 (Original commit messages left to preserve authorship credit for multiple contributors)
Expression Language now supports padding functions:
- padLeft:
  -  attr:padLeft(Int n, Char c) will prepend to the `attr` attributes the `c` character until the size `n` is reached
  -  attr:padLeft(Int n) will prepend to the `attr` attributes the `'_'` character until the size `n` is reached
- padRight:
  -  attr:padRight(Int n, Char c) will append to the `attr` attributes the `c` character until the size `n` is reached
  -  attr:padRight(Int n) will append to the `attr` attributes the `'_'` character until the size `n` is reached

- In both cases, the padding function returns the `attr` `String` as is if its length is already equal of higher than the desired size `n`
- Returns null if `attr` does not exist or in case desiredLenght is higher than Integer.MAX_INT

Further test cases:
- Returns null if the input string is null
- Returns a string full of padding if the input string is empty
Supports PaddingString instead of PaddingCharacter
Apply suggestions from code review
Applying style suggestions
Co-Authored-By: Marco Gaido <marcogaido91@gmail.com>
style fixes
style fixes
Padding returns input string instead of null in case desired length is missing, is negative, or is overflowing
Better tests
doc update
less verbose parser notation
Doc and style fixes
Fixed `StringEvaluator.evaluate()` after rebase
Applying nitpicking suggestion
Co-Authored-By: Marco Gaido <marcogaido91@gmail.com>
Fixed `PaddingEvaluator` constructor issue
Removed unused import

This closes #3615

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2019-08-23 22:34:39 -04:00
Alessandro D'Armiento 81136bf550
NIFI-6502 RecordPath padding functions:
- PadLeft(label, desiredLength, paddingChar) prepends the paddingChar (or the default value '_' if a paddingChar is not provided) to the label string until desiredLength is reached.
- PadRight(label, desiredLength, paddingChar) appends the paddingChar (or the default value '_' if a paddingChar is not provided) to the label string until desiredLength is reached.
Added Apache license disclaimers
checkstyle fixes
Replaced functional interface with abstract method
Fixes and Further test cases:
- Returns null if the input string is null
- Returns a string full of padding if the input string is empty
wip support padding string
In order to be consistent with the feature introduced in #3615, RecordPath padLeft and padRight supports String padding.
Since nifi-record-path doesn't have the Apache Commons StringUtils dependency, the padding methods have been added to the available NiFi commons StringUtils class.
NIFI-6502 Updated top level NOTICE file to include citation for code borrowed from commons-lang3.
borrowed pad methods from lang3 StringUtils
Replaced `PadLeft` and `PadRight` record path functions with borrowed Apache Lang `StringUtils` padding methods and updated `nifi-assembly/NOTICE` accordingly

This closes #3613

Signed-off-by: Mike Thomsen <mthomsen@apache.org>
2019-08-19 17:26:19 -04:00
Matt Gilman 326bc91405
NIFI-5839:
- Introducing case insensitive group membership.
- Ensuring user identity and group name mapping is applied when the user identity or group name is inferred through group membership decisions.

This closes #3657.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-08-19 17:04:05 -04:00
Brandon Devries 5f696482b0 NIFI-4775: Updating documentation and updating rocksdb version
NIFI-4775: Rolling back fix for NIFI-6416

NIFI-4775: Documentation improvements

+1 from markobean

This closes #3648

Signed-off-by: Brandon <devriesb@apache.org>
2019-08-15 15:28:48 +00:00
mans2singh 2491c51d34
NIFI-6525 - Support JsonPath delete expressions
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3632.
2019-08-07 09:42:20 +02:00
Troy Melhase e973cacb2f
NIFI-5973 Adds ShellUserGroupProvider.
NIFI-5973 More comments and better defaults for the shell provider.
NIFI-5973 Fixed bug where user was being retrieved by identifier when identity was provided.
NIFI-5973 Fixed a formatting string in the OS X shell commands.
Updated testing conditions to run IT in OS X environment.
Changed unit test to provide identity rather than identifier.

This closes #3537.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2019-06-20 16:22:42 -07:00
Andrew Lim 0b8d05aabd NIFI-6319 Update User Guide for RPG/S2S chnages for multiple URLs, batch settings and remote input/output ports
NIFI-6319 Improved Remote Process Group Ports screenshot

This closes #3503.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2019-06-10 14:18:55 +09:00
Andy LoPresto e6c843f465
NIFI-6323 Changed URLs for repositories, project description, and mailing lists to use HTTPS.
NIFI-6323 Changed URLs for splunk.artifactoryonline.com to use HTTPS (certificate validity warning in browsers, but command-line connection using openssl s_client is successful).
NIFI-6323 Changed URLs for XMLNS schema locations to use HTTPS (the XMLNS and schema identifier remain http:// because they are not designed to be resolvable).
NIFI-6323 Fixed Maven XML schema descriptor URLs.

This closes #3497
2019-05-29 14:36:40 -04:00
Mike Thomsen 6a06cd3094 NIFI-6304 added trim, toLowerCase and toUpperCase to record path operations.
NIFI-6304 Updated code based on code review.

NIFI-6304 Updated documentation.

NIFI-6304 Refactored to make it simpler

NIFI-6304 Reverted Concat to its last state.

This closes #3478.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2019-05-28 21:06:20 +09:00
Koji Kawamura 8a50cb10b2 NIFI-2933 Remote input/output ports at any PG
Specify remote access at port creation.
Incorporated comments, and finished refactoring.
Renamed RootGroupPort to PublicPort.
Fix error message for creating a connection from a child PG having only PublicPorts.
Enhanced ProcessGroup instances rendered in the parent ProcessGroup
Loosen Port move check, allow moving public port between PG.
Show 'Remote NiFi Instance' info on Connection dialogs
Make labels narrative.
'Within Remote Group'.
Fixed DTO (de)serialization.
Return null only if all values are null.

This closes #3351.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2019-05-22 15:44:16 -04:00
archon 4d18eaa481 NIFI-6035: 1. Add formatWithTimeZone() and toDateWithTimeZone(); 2. Their test code and docs.
This closes #3481

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2019-05-21 07:33:45 -04:00
Troy Melhase 12e210277b
NIFI-4247 Support ranges in `tls-toolkit` SAN cli option.
This closes #3466.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2019-05-13 18:50:27 -07:00
Wil Selwood 1705ee3b66 NIFI-6274 Remove two reference markers from jsonPath docs
There are two superscript references in the JsonPath documentation that don't seem to mean any thing. They are not clickable and there are no other similar reference markers in the document.

This closes #2907

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2019-05-07 20:26:25 -04:00
Mark Owens 12a7456f74 NIFI-6267: Corrects error in expression language guide.
This closes #3464

Corrects small typo's in ReplaceAll entry of Expression Language Guide.
Removes an extraneous instance of 'Arguments:' and adds a missing 'T' to
the properly spell the word 'The'.

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2019-05-07 11:41:50 -04:00
Endre Zoltan Kovacs 9194726b35 NIFI-6257: removing duplicate table row from administration guide
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3461
2019-05-03 10:05:59 -04:00
simonl 0650521eb2
NIFI-6171 always send email scope for OIDC
NIFI-6171 re-added lookupEmail() as fallback
NIFI-6171 additional OIDC scopes via nifi.properties
NIFI-6171 alternative user identification (instead of email)  via nifi.properties
NIFI-6171 changed lookupEmail() so that any configured claim can be fetched fro the UserInfo endpoint

This closes #3398
This closes #2346
2019-04-26 16:40:37 -04:00
Andrew Lim 3ce5d935a5 NIFI-6176 Improve description for nifi.cluster.load.balance.max.thread.count property in Admin Guide 2019-04-15 11:03:24 -04:00
Lars Francke bfcc0ebd03
NIFI-6208: Fixes typos in Developers Guide
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3431.
2019-04-13 03:29:15 +02:00
Peter Turcsanyi c8eff590ef
NIFI-6142: Fixing some typos in user/dev guides
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3384.
2019-03-24 12:29:46 +01:00
Jan Hentschel 60b00a1e38 NIFI-2738 Removed references to nifi.cluster.request.replication.claim.timeout
This closes #2860

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2019-02-21 06:34:27 -05:00
joewitt 25cc7b4a1e
NIFI-6029 merging nifi 1.9.0 release into master 2019-02-19 22:55:49 -05:00
Denes Arvay 76e92c8682 NIFI-6052 Update NOTICE files to reflect 2019
This closes #3319

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2019-02-19 18:32:15 -05:00
joewitt 0e204f3576
NIFI-6029-RC2 prepare for next development iteration 2019-02-16 21:50:35 -05:00
joewitt 45bb53d2aa
NIFI-6029-RC2 prepare release nifi-1.9.0-RC2 2019-02-16 21:50:15 -05:00
Aleksandr Salatich 9e0687ab6b
NIFI-6018 Marking GetHTTP and PostHTTP deprecated. Adding info about deprecation to processors descriptions. Fix small mistake in developer-guide.
Suppressed deprecation warnings in TestGetHTTP, TestPostHTTP, TestGetHTTPGroovy, and TestPostHTTPGroovy. The annotation suppresses some warnings, but until Java 9, explicit FQCN imports are needed to avoid all warnings.

This closes #3312.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2019-02-15 14:09:26 -08:00
Pierre Villard 8a3d7a453a NIFI-5867 - add thread() EL function to get thread name
moved thread name evaluation in evaluate method

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #3198
2019-01-30 10:14:35 -05:00
Andy LoPresto c5c2b308ed
NIFI-5926 Added certificate commands to toolkit guide.
Applied formatting and organizational changes from PR review.

This closes #3242.

Signed-off-by: Andrew Lim <andrewlim.apache@gmail.com>
2019-01-04 09:38:58 -08:00
Peter Wicks 8ccf890a55
NIFI-5913 - Standardize Definition of UUID in Documentation 2018-12-20 10:11:07 -07:00
Jeff Storck cc47a8c0e1
NIFI-5748 Improved Proxy Header Support
- Fixed proxy header support to use X-Forwarded-Host instead of X-ForwardedServer
- Added support for the context path header used by Traefik when proxying a service (X-Forwarded-Prefix)
- Added tests to ApplicationResourceTest for X-Forwarded-Context and X-Forwarded-Prefix
- Updated administration doc to include X-Forwarded-Prefix
- Added NIFI_WEB_PROXY_CONTEXT_PATH env var to dockerhub and dockermaven start.sh scripts
- Added documentation for NIFI_WEB_PROXY_CONTEXT_PATH to dockerhub README.md
- Updated ApplicationResource to handle a port specified in X-ProxyPort and X-Forwarded-Port headers

This closes #3129.

Signed-off-by: Kevin Doran <kdoran@apache.org>
2018-12-17 17:32:06 -05:00
Wil Selwood 1a937b6511 NIFI-5875 Improve docs around the PriorityAttributePrioritizer
Clear up the documentation around the PriorityAttributePrioritizer to make it clearer that if the attributes are both number the largest wins but if they are not numbers they sort the other way around.

I had to go and look at the code to work out what was going on after reading the existing documentation.

Simplify PriorityAttributePrioritzer docs

Simplify the main section of documentation for PriorityAttributePrioritzer and then add a notes section with more details about the ordering.

Fix example.

Put back comment setting the priority attribute

Further improvements to the Priority ordering

This closes #3205.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2018-12-11 10:43:25 +09:00
Andrew Lim 5cf58b42e0 NIFI-5784 Edit Admin Guide to remove duplicate content that is in new Toolkit Guide
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3155.
2018-11-08 09:32:51 +01:00
Andrew Lim 59e102ad47 NIFI-5767 Added NiFi Toolkit Guide to docs
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3124.
2018-11-06 11:23:54 +01:00
Andrew Lim d0688d0785 NIFI-5677 Added note to clarify why modifying/creating variables not local changes in versioned flows
Update user-guide.adoc

This closes #3125.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2018-11-06 15:17:32 +09:00
Andrew Lim fdb1fd1a64 NIFI-5766 Make formatting in User Guide consistent with Admin Guide
This closes #3115

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
2018-10-30 12:04:29 -04:00
Jeff Storck c0182294ed NIFI-5720-RC3 prepare for next development iteration 2018-10-22 22:16:43 -04:00
Jeff Storck 98aabf2c50 NIFI-5720-RC3 prepare release nifi-1.8.0-RC3 2018-10-22 22:16:23 -04:00
Matt Gilman 02261311b3 NIFI-5737:
- Removing needClientAuth property since cluster comms now requires two way ssl. Jetty client auth settings are based on configured features.
- Removing dead code.
- Updating documentation.
- Removing references to needClientAuth property in all test resources.
- Removing overloaded util method with strict parameter.

This closes #3102.
2018-10-22 20:02:47 -04:00
Andrew Lim 5ec85299e7 NIFI-5653 Added default NiFi and Embedded Zookeeper port tables to Admin Guide
This closes #3089.
2018-10-17 15:29:11 -04:00
Andrew Lim 4039c21d67 NIFI-5701 Add documentation for Load Balancing connections to User Guide and add additional properties to Admin Guide
This closes #3080.
2018-10-16 21:00:38 -04:00
Andrew Lim fd5eccc593 NIFI-5659 Add documentation for Offloading Nodes functionality and new Node related CLI commands
This closes #3056
2018-10-15 18:02:14 -04:00
Mark Payne 619f1ffe8f NIFI-5516: Implement Load-Balanced Connections
Refactoring StandardFlowFileQueue to have an AbstractFlowFileQueue
Refactored more into AbstractFlowFileQueue
Added documentation, cleaned up code some
Refactored FlowFileQueue so that there is SwappablePriorityQueue
Several unit tests written
Added REST API Endpoint to allow PUT to update connection to use load balancing or not. When enabling load balancing, though, I saw the queue size go from 9 to 18. Then was only able to process 9 FlowFiles.
Bug fixes
Code refactoring
Added integration tests, bug fixes
Refactored clients to use NIO
Bug fixes. Appears to finally be working with NIO Client!!!!!
NIFI-5516: Refactored some code from NioAsyncLoadBalanceClient to LoadBalanceSession
Bug fixes and allowed load balancing socket connections to be reused
Implemented ability to compress Nothing, Attributes, or Content + Attributes when performing load-balancing
Added flag to ConnectionDTO to indicate Load Balance Status
Updated Diagnostics DTO for connections
Store state about cluster topology in NodeClusterCoordinator so that the state is known upon restart
Code cleanup
Fixed checkstyle and unit tests
NIFI-5516: Updating logic for Cluster Node Firewall so that the node's identity comes from its certificate, not from whatever it says it is.
NIFI-5516: FIxed missing License headers
NIFI-5516: Some minor code cleanup
NIFI-5516: Adddressed review feedback; Bug fixes; some code cleanup. Changed dependency on nifi-registry from SNAPSHOT to official 0.3.0 release
NIFI-5516: Take backpressure configuration into account
NIFI-5516: Fixed ConnectionDiagnosticsSnapshot to include node identifier
NIFI-5516: Addressed review feedback

This closes #2947
2018-10-04 16:11:05 -04:00
Andrew Lim f5e9ea680a NIFI-5578 Correct errors in EL Guide including Date Manipulation and Escaping EL examples
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3017.
2018-09-21 11:03:36 +02:00
Andrew I. Christianson be2ed060a2
NIFI-5542 Added support for node groups to FileAccessPolicyProvider
This closes #2970.

Signed-off-by: Kevin Doran <kdoran@apache.org>
2018-09-20 10:27:28 -04:00
Koji Kawamura f570cb980d
NIFI-375: Added operation policy
The operation policy allows that a user to operate components even if they does not have direct READ/WRITE
permission of the component.

Following operations are controlled by the new operate policy:
- Start/stop/enable/disable Processors, ControllerServices,
ReportingTasks, Input/OuputPorts
- Enable/disable transmission of RemoteInput/OutputPorts and
RemoteProcessGroups
- Terminate Processor threads

Refactored what API exposes

The previous commit let API exposes few fields in DTO. But we should
avoid returning partial DTO as it complicates authorization logic.

Instead, this commit adds StatusDTO for ReportingTaskEntity and
ControllerServiceEntity, so that it can be returned regardless of having
READ permission. Component DTO can only be returned with a READ
permission.

Refactor RPG same as ControllerService.

WIP incorporating review comments.

Incorporated review comments

- Cleaned up merger classes
- Recreate DTO instance at each function during two phase commmit

Restrict enabling ControllerService without read permission

Revert the last commit.

Fix review comments.

- Renamed confusing static method names and its parameters
- Removed unnecessary permission checks from UI condition

Fixed delete action display condition.

Fixed NPE at Summary.

Apply operation policy to activateControllerServices.

Removed OperationPermissible from ComponentEntity.

This closes #2990
2018-09-19 15:28:41 -04:00
Matthew Burgess 5a84d650c3 NIFI-5449: Added Base64 Encode/Decode functions to RecordPath
NIFI-5449: Incorporated review comments

This closes #2920

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-09-14 14:04:25 -04:00
Nick Lewis 3efcd45e99 NIFI-5565 Added reference to confluence documentation from the developers guide
This closes #2978

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-09-01 07:32:32 -04:00
Mark Payne aac2c6a60e
NIFI-5482: Made WriteAheadProvenanceRepository the default implementation
This closes #2960.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2018-08-23 16:49:55 -07:00
Andrew Lim 86beca0c58 NIFI-5469 Additional italics and code formatting corrections for consistency
This closes #2925

Signed-off-by: zenfenan <zenfenan@apache.org>
2018-08-13 15:03:41 +05:30
Andy LoPresto 57baae9ae2
NIFI-5476 Added logic to check CA certificate signature against additional certificates.
Moved utility code to TlsHelper.
Added unit tests.
Added command-line parsing for additional CA certificate path.
Added documentation on using the TLS Toolkit to generate and sign certificates using an externally-signed CA.
Updated toolkit external CA documentation to be inline with additional context from NIFI-5473.
Cleaned up toolkit documentation.
Improved error message by changing to absolute path.
Added Javadoc to and removed unthrown exception declarations from TlsHelper#verifyCertificateSignature().
Cleaned up unit tests with utility method.
Fixed checkstyle error.
Support conversion of a PKCS#8 formatted private key automatically to avoid forcing the user to do that. Also add some log messages for debugging when the parser fails to parse the appropriate object
Incorporated Peter's contribution for PKCS #8 to PKCS #1 conversion.
Added documentation and refactored methods.
Refactored unit test.
Added RAT exclusion for test resource.

This closes #2935.

Co-authored-by: pepov <peterwilcsinszky@gmail.com>

Signed-off-by: Matt Gilman <mcgilman@apache.org>
2018-08-07 12:07:35 -07:00
Andy LoPresto 5e6c43f83e
NIFI-5473 Added section on using external signed CA for TLS Toolkit.
Increased TOC level to 3 for ASCIIDOC generated documentation.

This closes #2927.

Signed-off-by: Andrew Lim <andrewlim.apache@gmail.com>
2018-08-02 20:55:39 -07:00
郑书国10209198 9a23d5bc60
NIFI-5447 Modify the description of 'nifi.security.needClientAuth',add default value describe
Added space and formatting to default value for needClientAuth in Admin Guide.

This closes #2912.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2018-07-23 18:14:51 -07:00
Mark Payne 72a5e6bfba NIFI-4407: Updated Expression Language Guide to provide details about how escaping $ is accomplished, and when the $ character should and should not be escaped. Fixed bug in the Query compiler that mistakenly would blindly replace 484 with $ even when the 484 did not precede an Expression. Added additional test cases.
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2898.
2018-07-18 10:52:13 +02:00
Otto Fowler 3746ae258d
NIFI-5436 Add ability to set the processor name in the ProcessContext when using the TestRunner.
This closes #2902.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2018-07-17 14:42:37 -07:00
Andy LoPresto bef91008bb
NIFI-5399 Added wildcard certificate documentation to Admin Guide.
Clarified CN vs. SAN entries.

This closes #2870.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>
2018-07-10 11:15:07 -07:00
Andrew Lim 0721d9a0ee NIFI-5330 Fixed broken image in Variables in Versioned Flows section of User Guide
Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2810
2018-06-25 15:18:17 -04:00
Andy LoPresto d42a1e8bf4
NIFI-5323-RC1 prepare for next development iteration 2018-06-19 20:02:21 -07:00
Andy LoPresto 99bcd1f88d
NIFI-5323-RC1 prepare release nifi-1.7.0-RC1 2018-06-19 20:02:01 -07:00
Matt Gilman fe31a06fdc
NIFI-4907:
- Minor adjustments following PR.
- Avoiding additional find operation when authorizing components when populating component details.
- Requiring access to provenance events when downloading content or submitting a replay as they may provide events details.
- Updating the REST API docs detailing the required permissions.
- Updating the wording in the documentation regarding the provenance and data policies.
- Removed the event attributes from the authorization calls that were verifying access to provenance events.
- Only checking content availability when the user is authorized for the components data.
- Addressing typo in JavaDoc.

This closes #2703
2018-06-14 12:21:47 -04:00
Mark Bean e27798797a
NIFI-4907: add 'view provenance' component policy
whitespace removed for checkstyle
2018-06-14 10:59:17 -04:00
zenfenan 0973c2d8d1 NIFI-543 Added annotation to restrict processor to run only on the primary node
- PR Fix - 'Execution' dropdown will now be shown in all cases
- Annotated ListGCSBucket with PrimaryNodeOnly

This closes #2509.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-05-25 11:50:29 -04:00
Andrew Lim 470d85042d NIFI-4585 Add nifi.cluster.node.max.concurrent.requests to Cluster Node Properties section of Admin Guide
This closes #2719

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
2018-05-22 12:03:37 -04:00
Andrew Lim 8aaa51af31 NIFI-5205 Updated default value for nifi.content.claim.max.appendable.size in Admin Guide
This closes #2716

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
2018-05-17 15:09:37 -04:00
Andrew Lim 775cf42560 NIFI-2966 Add policy resources to Access Policies tables in Admin Guide
This closes #2710

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-05-16 16:18:52 -04:00
Andy LoPresto 7a4990e7fe
NIFI-5146 Only support HTTP or HTTPS operation for NiFi API/UI
- Added logic to check for simultaneous configuration of HTTP and HTTPS connectors in JettyServer.
- Added test logging resources. Added unit tests.
- Refactored shared functionality to generic method which accepts lambdas.
  Fixed unit test with logging side effects.
- Added note about exclusive HTTP/HTTPS behavior to Admin Guide. Fixed typos.

This closes #2683.

Signed-off-by: Kevin Doran <kdoran@apache.org>
2018-05-09 12:14:11 -04:00
Matt Gilman 5cfa29e48f
NIFI-5135:
- Adding support for applying transforms to user identities and group names.

This closes #2673.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-05-07 11:17:26 -04:00
Andrew Lim ce0855e988 NIFI-5074 Added section Variables in Versioned Flows to User Guide
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2627.
2018-04-11 18:53:07 +02:00
Andrew Lim eb3391f1d2 NIFI-5063 Added screenshots and supporting text for Primary Node processors
This closes #2624

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
2018-04-10 13:34:18 -04:00
Andrew Lim b6c052066a NIFI-4941 Updated nifi.sensitive.props.additional.keys description to refer to nifi.properties
This closes #2620

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
2018-04-10 09:29:39 -04:00
Matthew Burgess b29304df79 NIFI-4857: Support String<->byte[] conversion for records
This closes #2570.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-04-09 14:53:21 -04:00
Andrew Lim 5f16f48a27
NIFI-5042 Added section Restricted Components in Versioned Flows and edited related section in Adding Components to the Canvas
This closes #2610.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-04-09 09:53:22 -04:00
joewitt 59f625d3c2 Merge branch 'NIFI-4995-RC3' as part of NiFi 1.6.0 release process 2018-04-06 16:48:08 -07:00
Pierre Villard 4c787799ff NIFI-4149 - Indicate if EL is evaluated against FFs or not
- take into account input requirement for documentation rendering
- Renamed variable registry scope and added comments
- Doc + change in mock framework to check scope + update of components + UI
2018-04-06 12:40:54 -04:00
Mike Moser dc9b4cb516
NIFI-3599 Allow back pressure object count and data size to be configurable in nifi.properties. This closes #2497 2018-04-05 13:50:16 -04:00
Koji Kawamura 1913b1e2a8
NIFI-4932: Enable S2S work behind a Reverse Proxy
Adding S2S endpoint Reverse Proxy mapping capability.
Added license header to SVG files.
Incorporated review comments.
Use regex to check property key processing.
Catch AttributeExpressionLanguageParsingException.
This closes #2510
2018-04-03 15:40:28 -04:00
joewitt d511fe3e4b NIFI-4995-RC3 prepare for next development iteration 2018-04-03 08:28:34 -07:00
joewitt f8466cb16d NIFI-4995-RC3 prepare release nifi-1.6.0-RC3 2018-04-03 08:28:15 -07:00
joewitt 7b5bf265a6 NIFI-4995 updating copyright year on all notices 2018-03-26 21:54:10 -04:00
jugi92 78daaf40f8 NIFI-4991 This closes #2564.
nifi.database.directory should be moved out of default location <root-level-nifi-dir>/database_repository to ease the update process.

Signed-off-by: joewitt <joewitt@apache.org>
2018-03-19 11:41:56 -04:00
Mark Payne d14229e440 NIFI-4774: Allow user to choose which write-ahead log implementation should be used by the WriteAheadFlowFileRepository
Removed TODO comment

Signed-off-by: Matthew Burgess <mattyb149@apache.org>

This closes #2487
2018-03-13 13:03:39 -04:00
Matt Gilman b1217f529b NIFI-4885:
- Introducing more granular restricted component access policies.

This closes #2515.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-03-12 14:19:42 -04:00
Willie Engelbrecht 1ec7bcd286 NIFI-4910 Fixing slight spelling mistake in error message, needs a space
Signed-off-by: James Wing <jvwing@gmail.com>

This closes #2492.
2018-02-26 21:01:32 -08:00
Leah Anderson a3edd34db0 NIFI-4840 Fix formatting for expression language guide for toRadix and fromRadix
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2449.
2018-02-13 09:33:53 +01:00
Andrew Lim 59970344fe NIFI-4791 Made external links open in new windows and fixed some broken links
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2413.
2018-01-20 15:15:48 +01:00
Matt Gilman 4b3b2ebf24
NIFI-4766:
- Documenting nifi.web.proxy.host property.
- Updating proxy confiugration.
NIFI-4618:
- Documenting nifi.web.proxy.context.path property.
- Addressing PR feedback.
- This closes #2410
2018-01-19 11:03:55 -05:00
joewitt 41ce788812 NIFI-4751 changed to next minor release version snapshot 2018-01-12 15:15:32 -05:00
joewitt 42edfa75b7 Merge branch 'NIFI-4751-RC1' 2018-01-12 15:00:32 -05:00
Marco Gaido 39a484b631
NIFI-4764: Add tooltips to status bar icons
add tooltip to process groups
cleanup
This closes #2394
2018-01-11 12:25:33 -05:00
joewitt 36405e888c NIFI-4751-RC1 prepare for next development iteration 2018-01-08 23:39:49 -07:00
joewitt 46d30c7e92 NIFI-4751-RC1 prepare release nifi-1.5.0-RC1 2018-01-08 23:39:32 -07:00
Andrew Lim acdaeebac7
NIFI-4679 Added new content in User Guide related to Registry integration and replaced screenshots and edited text for recent UI changes. This closes #2382 2018-01-08 15:07:27 -05:00
Bryan Bende ff7283d65d NIFI-4506 Adding toDate and format functions to record path. This closes #2221.
Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-01-03 11:01:33 -05:00
Kevin Doran 482f371958
NIFI-4701 Add authorizers.xml support to toolkit.
Adds authorizers.xml to the files understood by the encrypt-config
tool in the NiFi Toolkit. If enabled, then the sensitive properties
for LdapUserGroupProvider in authorizers.xml will be encrypted.
Also fixes a bug wherein encrypt-config replaces multiple XML nodes
in login-indentity-providers.xml when LdapProvider is not the first
provider listed in the file.
Enable properties in authorizers.xml to be encrypted by the master key.

This closes #2350.

Signed-off-by: Andy LoPresto <alopresto.apache@gmail.com>
2017-12-31 17:41:04 -05:00
joewitt 5b2e2afc17 NIFI-4678 also fixing a few asciidoc warnings 2017-12-14 17:09:12 -05:00
edwarzjl 68016ddbe8 NIFI-4678 This closes #2321. change all the old fasioned headings in docs to modern ones
Signed-off-by: joewitt <joewitt@apache.org>
2017-12-14 17:08:13 -05:00
Kevin Doran 2608351113
NIFI-4667 Fix LDAP Sync Interval
Corrects time unit conversion for the Sync Interval config property
for LdapUserGroupProvider in authorizers.xml.

Also enforces a minimum value of 10 secs for the Sync Interval to help
catch unintentional misconfigurations, for example users upgrading
from previous versions, where tiny Sync Interval values could be set
as a workaround for NIFI-4667.

This closes #2341
2017-12-14 15:01:52 -05:00
Matt Gilman 439e13a8d5
NIFI-4567:
- Adding new properties to allow the referenced attribute of a user/group to be configurable when detecting group membership.
- Expanding on documentation regarding the new properties.

This closes #2274.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-11-20 10:09:53 -05:00
Matthew Burgess 53de8c93ba NIFI-4603 - Externalize Max HTTP Header Size to nifi.properties
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2271.
2017-11-15 10:12:39 +01:00
Mark Payne af3a578711
NIFI-4598: When we retrieve the 'controller' from a remote NiFi instance in order to determine which ports are available, cache those results for up to some configurable amount of time (default 30 secs) so that we don't constantly issue HTTP Requests to the remote nifi
This closes #2270.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-11-13 14:46:38 -05:00
Andrew Lim 0e3d83c3b8
NIFI-4484 Update screenshots in User Guide for Reporting Task Controller Services tab. This closes #2209 2017-10-13 14:31:10 -04:00
Andrew Lim b2c68d2b29 NIFI-4462 Update User and Admin Guide with Variables window UI documentation and screenshots
This closes #2202

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
2017-10-09 15:21:42 -04:00
Jeff Storck a57911d3db NIFI-4412-RC2 prepare for next development iteration 2017-09-28 13:45:36 -04:00
Jeff Storck e6508ba7d3 NIFI-4412-RC2 prepare release nifi-1.4.0-RC2 2017-09-28 13:45:21 -04:00
Matt Gilman 6c798d18ef NIFI-4382:
- Adding support for KnoxSSO.
- Updated the docs for nifi.security.user.knox.audiences.
- The KnoxSSO cookie is removed prior to request replication.

This closes #2177
2017-09-27 16:22:18 -04:00
Mark Payne e52e9acc59 NIFI-4377: Added a fieldName() function to RecordPath and addressed an issue that caused //* to not work
NIFI-4377: Updated RecordPath Guide to include the new fieldName() function and improved unit test for the function

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #2147.
2017-09-14 09:40:36 +02:00
yuri1969 cb7bc93e28
NIFI-4347 - Extend documentation with double-click shortcuts info. This closes #2126 2017-09-11 14:21:37 -04:00
Yolanda M. Davis acee2627ca NIFI-4255 - Added flag to allow migration of existing (source) acls to destination, update to documentation
Added build-helper-maven-plugin to nifi-toolkit-zookeeper-migrator pom to build groovy test code
Moved spock spec from src/test/java to src/test/groovy
Minor code formatting updates to ZooKeeperMigrator.java

This closes #2065
2017-08-23 15:38:44 -04:00
Pierre Villard 0bcb19771f NIFI-1670 - typos/details in EL doc
Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #2097
2017-08-18 09:08:46 -04:00
Andrew Lim bac175a00f NIFI-4276 Add Write Ahead Provenance section to User Guide
Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #2074
2017-08-13 10:01:36 -04:00
Matt Gilman ec0c8278b4
NIFI-4206:
- Updating admin guide to include instructions for running NiFi behind a proxy.
- Including a brief example proxy configuration for NiFi specific properties.

This closes #2023.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-08-10 17:08:59 -07:00
Pierre Villard c03f4e731b
NIFI-968 - add @OnPrimaryNodeStateChange to dev guide
This closes #2027.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-08-10 12:21:04 -07:00
Matt Gilman 528b82634f
NIFI-4210:
- Introducing support for OpenId Connect.
- Updating REST API and UI to support the authorization code flow.
- Adding/fixing documentation.
- Implementing time constant equality checks where appropriate.
- Corrected error handling during startup and throughout the OIDC login sequence.
- Redacting the token values from the user log.
- Defaulting to RS256 when not preferred algorithm is specified.
- Marking the OIDC endpoints as non-guaranteed in to allow for minor adjustments if/when additional SSO techniques are introduced.

This closes #2047.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-08-10 11:15:35 -07:00
Andy LoPresto 675d989003
NIFI-4139
- Moved key provider interface and implementations from nifi-data-provenance-utils module to nifi-security-utils module.
- Refactored duplicate byte[] concatenation methods from utility classes and removed deprecation warnings from CipherUtility.
- Created KeyProviderFactory to encapsulate key provider instantiation logic.
- Added logic to handle legacy package configuration values for key providers.
- Added unit tests.
- Added resource files for un/limited strength cryptography scenarios.
- Added ASL to test resources.
- Moved legacy FQCN handling logic to CryptUtils.
- Added unit tests to ensure application startup logic handles legacy FQCNs.
- Moved master key extraction/provision out of FBKP.
- Removed nifi-security-utils dependency on nifi-properties-loader module.
- Added unit tests.
2017-08-07 15:21:58 -04:00
Yolanda M. Davis afd4f9e034
NIFI-4022 - Initial update for SASL support for cluster management in Zookeeper
NIFI-4022 - adding sasl documentation update and update to test

This closes #2046.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-08-04 14:19:35 -04:00
Matt Gilman eefad29167 NIFI-4127:
- Introducing composite ConfigurableUserGroupProvider and UserGroupProvider.
- Adding appropriate unit tests.
- Updating object model to support per resource (user/group/policy) configuration.
- Updating UI to support per resource (user/group/policy) configuration.
- Adding necessary documentation.
- Updating documentation to clarify integrity checks.
- Providing an example of configuring a composite implementation.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1978.
2017-07-11 18:13:04 +02:00
Andrew Lim 0c27a0888d
NIFI-3880 Added complete TLS Toolkit options to Admin Guide.
This closes #1965.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-06-30 14:45:35 -07:00
Andrew Lim 3089d9dce0 NIFI-4138 Add Component Alignment to User Guide
Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #1959
2017-06-29 11:54:56 -04:00
Pierre Villard ba3372a1dc NIFI-4079 Add concat in Record Path Guide. This closes #1920. 2017-06-21 14:56:07 -04:00
Matt Gilman 6bc6f955c0 NIFI-4059:
- Introducing the LdapUserGroupProvider.
- Updating documentation accordingly.
- Moving the IdentityMapping utilities so they were accessible.

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #1923.
2017-06-19 19:25:33 +02:00
Yolanda M. Davis 8ef4fddddd
NIFI-3696 - ConfigMigration and FileManager tools
This closes #1889.

Signed-off-by: Bryan Rosander <brosander@apache.org>
2017-06-12 09:50:02 -04:00
Matt Gilman 4ed7511bee
NIFI-3653: - Introducing UserGroup and Policy provider interfaces.
- Introducing FileUserGroupProvider and FileAccessPolicyProvider.
- Refactoring FileAuthorizer to utilize the file based implementations.
- Introducing the StandardManagedAuthorizer.
- Decorating the configured ManagedAuthorizer to ensure integrity checks are still performed.
- Loading user groups if possible to use during access decisions.
- Merging responses for requests for AccessPolicies, Users, and UserGroups.
- Adding unit tests as appropriate.
- Adding methods to the User, Group, and AccessPolicy builder that more easily supports generating UUIDs.
- Fixing typo when seeding policies during startup.
- Fixing type in documentation and error messages.

This closes #1897.

Signed-off-by: Bryan Bende <bbende@apache.org>
2017-06-09 13:54:10 -04:00
Andrew Lim f447fc73fa NIFI-4048 Added nifi.cluster.node.protocol.max.threads to Cluster Node Properties section of Admin Guide
This closes #1908

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
2017-06-09 12:59:22 -04:00
Matt Gilman cc741d2be6
NIFI-3997:
- Bumping to next minor version.
2017-06-08 15:22:51 -04:00
Matt Gilman 6ee12e9b47
NIFI-3997-RC1prepare for next development iteration 2017-06-05 11:07:43 -04:00
Matt Gilman ddb73612bd
NIFI-3997-RC1prepare release nifi-1.3.0-RC1 2017-06-05 11:07:28 -04:00
Mark Payne 32314d70fd NIFI-4009: Added support for several key functions in RecordPath
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1881
2017-06-02 14:06:05 -04:00
Andrew Lim de6a98618a NIFI-479 Add UI Extension Section to Developer Guide
Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #1882
2017-06-02 12:05:20 -04:00
Andrew Lim 3966f5ce86 NIFI-3959 Corrected Anatomy of Processor and Process group screenshots and text
Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #1862
2017-05-26 08:51:25 -04:00
Mark Payne 5aa3baca79 NIFI-3568: This closes #1577. Use a cached thread pool in order to allow ThreadPoolRequestReplicator to scale up the number of threads to some configurable max
Signed-off-by: joewitt <joewitt@apache.org>
2017-05-24 21:21:45 -04:00
Andrew Lim 49270f8a8c NIFI-3944 Update docs and screenshots for Component Versioning
Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #1840
2017-05-23 11:06:59 -04:00
Andrew Lim 28eadaa683
NIFI-3911 Improve Controller Services and Reporting Task Documentation
This closes #1826.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-05-18 19:20:39 -07:00
Andre F de Miranda 289dde098e
NIFI-3896 - Makes DeprecationNotice more intuitive. This closes #1799
Update developers guide on how to deprecate a component
2017-05-17 16:37:11 -04:00
Matt Gilman 6ffb78d404
NIFI-3853:
- Filtering out certain control characters and unpaired Unicode surrogate codepoints prior to saving the flow.xml.

This closes #1784.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-05-15 19:02:03 -07:00
Mark Payne b1901d5fe0 NIFI-3838: Initial implementation of RecordPath and UpdateRecord processor
NIFI-3838: Updated version from 1.2.0-SNAPSHOT to 1.3.0-SNAPSHOT; removed unneeded value from AttributeExpression.ResultType enum

NIFI-3838: Addressed PR Review feedback

NIFI-3838: Allow for schemas to be merged together for a record; refactored RecordSetWriterFactory so that there is a method to obtain the schema and then the writer is created with that schema. Added additional unit tests

NIFI-3838: Addressed problems with documentation based on PR Review

NIFI-3838: Fixed checkstyle violation

NIFI-3838: Addressed issue of comparing different types of Number objects

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #1772
2017-05-12 12:36:52 -04:00
Andrew Lim b6bdc4a0a8 NIFI-3850 Corrected errors in Admin Guide and made property formatting consistent
Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #1771
2017-05-09 15:06:23 -04:00
Bryan Bende 3af53419af
NIFI-3770-RC2 prepare for next development iteration 2017-05-05 20:50:28 -04:00
Bryan Bende 3a605af8e0
NIFI-3770-RC2 prepare release nifi-1.2.0-RC2 2017-05-05 20:50:14 -04:00
Sarah Olson 960ef9142d Documentation for admin toolkit
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #1736
2017-05-03 09:01:59 -04:00
Sarah Olson 16f37763f6
NIFI-3773 Added documentation and screenshots for Component Versioning.
This closes #1734.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-05-02 17:37:18 -04:00
Andrew Lim 580d65dfde
NIFI-3701 Documentation improvements for 1.x.
This closes # 1733.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-05-02 16:41:47 -04:00
Andre F de Miranda 3bfe323a5d NIFI-3769 - Document detail value of 'nifi.provenance.repository.warm.cache.frequency' 2017-05-02 08:44:01 -04:00
Andy LoPresto 946f4a1a28
NIFI-3721 Added documentation for Encrypted Provenance Repositories to Admin Guide and User Guide.
Added screenshot of encrypted provenance repository contents on disk.
Added note about clearing existing provenance repository when switching to encrypted implementation (see PR 1686 @ https://github.com/apache/nifi/pull/1686#issuecomment-298432578).

This closes #1713.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-05-01 17:19:25 -04:00
Andrew Lim 11b935a27b NIFI-3748 Add component linking functionality to documentation
Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #1724
2017-05-01 15:37:13 -04:00
Andrew Lim 0a7b9467e9 NIFI-3684 Make docs more explicit about anonymous access to a secured instance
This closes #1722

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
2017-05-01 13:13:54 -04:00
Andrew Lim 6512306fa4 NIFI-3752 Fixed broken links in User Guide
Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #1710
2017-04-27 15:42:44 -04:00
Bryan Rosander ee4b88620a NIFI-3737 - Reverting problematic pngs
Signed-off-by: Scott Aslan <scottyaslan@gmail.com>

This closes #1693
2017-04-25 12:47:48 -04:00
Bryan Rosander 879fe3efd6 NIFI-3737 - Ran optipng on all png files 2017-04-25 12:28:49 -04:00
Joe Percivall 5419891fe7 NIFI-3633 This closes #1610. Adding HttpNotificationService.
Signed-off-by: joewitt <joewitt@apache.org>
2017-04-11 00:37:12 -04:00
Mike Moser 2589df80ab NIFI-3304 removed unneeded Java 8 codecache info from Admin Guide
This closes #1621

Signed-off-by: Andre F de Miranda <trixpan@users.noreply.github.com>
2017-04-11 02:05:20 +10:00
Bryan Bende 556f309df0
NIFI-3520 Refactoring instance class loading
- Fixing FlowController to use appropriate class loader when instantiating processor
- Updating ExtensionManager to leverage new flag in MANIFEST from NAR plugin
- Adding ReloadComponent interface and refactoring instance class loading to use it
- Fixing FetchHDFS issue with TDE by using ugi.doAs
- Refactoring nifi-nar-utils so that ExtensionManager only lives in nifi-framework
- Caching temp components found during service loader in ExtensionManager
- Updating authorizables, docs, and fingerprinting to use the cached components
- Introducing a flag on @RequiresInstanceClassLoading to indicate if ancestor resources should be cloned
- Updating developer guide regarding cloneAncestorResources flag
- This closes #1635
2017-04-06 13:51:08 -04:00
Bryan Bende d90cf846b9 NIFI-3380 Bumping NAR plugin to 1.2.0-SNAPSHOT development to leverage changes from master, adding buildnumber-maven-plugin to nifi-nar-bundles to properly set build info in MANIFEST of NARs
- Refactoring NarDetails to include all info from MANIFEST
- Adding the concept of a Bundle and refactoring NarClassLoaders to pass Bundles to ExtensionManager
- Adding logic to fail start-up when multiple NARs with same coordinates exist, moving Bundle classes to framework API
- Refactoring bundle API to classes and creating BundleCoordinate
- Updating FlowController to use BundleCoordinate

- Updating the UI and DTO model to support showing bundle details that loaded an extension type.
- Adding bundle details for processor canvas node, processor dialogs, controller service dialogs, and reporting task dialogs.
- Updating the formating of the bundle coordinates.
- Addressing text overflow in the configuration/details dialog.
- Fixing self referencing functions.
- Updating extension UI mapping to incorporate bundle coordinates.
- Discovering custom UIs through the supplied bundles.
- Adding verification methods for creating extensions through the rest api.
- Only returning extensions that are common amongst all nodes.
- Rendering the ghost processors using a dotted border.
- Adding bundle details to the flow.xml.
- Loading NiFi build and version details from the framework NAR.
- Removing properties for build and version details.
- Wiring together front end and back end changes.
- Including bundle coordinates in the component data model.
- Wiring together component data model and flow.xml.
- Addressing issue when resolve unvesioned dependent NARs.

Updating unit tests to pass based on framework changes
- Fixing logging of extension types during start up

- Allowing the application to start if there is a compatible bundle found. - Reporting missing bundle when the a compatible bundle is not found. - Fixing table height in new component dialogs.

Fixing chechstyle error and increasing test timeout for TestStandardControllerServiceProvider
- Adding ability to change processor type at runtime
- Adding backend code to change type for controller services

- Cleaning up instance classloaders for temp components.
- Creating a dialog for changing the version of a component.
- Updating the formatting of the component type and bundle throughout.
- Updating the new component dialogs to support selecting source group.
- Cleaning up new component dialogs.
- Cleaning up documentation in the cluster node endpoint.

Adding missing include in nifi-web-ui pom compressor plugin
- Refactoring so ConfigurableComponent provides getLogger() and so the nodes provide the ConfigurableComponent
- Creating LoggableComponent to pass around the component, logger, and coordinate with in the framework

- Finishing clean up following rebase.

Calling lifecycle methods for add and remove when changing versions of a component
- Introducing verifyCanUpdateBundle(coordinate) to ConfiguredComponent, and adding unit tests

- Ensuring documentation is available for all components. Including those of the same type that are loaded from different bundles.

Adding lookup from ClassLoader to Bundle, adding fix for instance class loading to include all parent NARs, and adding additional unit tests for FlowController
- Adding validation to ensure referenced controller services implement the required API
- Fixing template instantiation to look up compatible bundle

- Requiring services/reporting tasks to be disabled/stopped.
- Only supporting a change version option when the item has multiple versions available.
- Limiting the possible new controller services to the applicable API version.
- Showing the implemented API versions for Controller Services.
- Updating the property descriptor tooltip to indicate the required service requirements.
- Introducing version based sorting in the new component dialog, change version dialog, and new controller service dialog.
- Addressing remainder of the issues from recent rebase.

Ensuring bundles have been added to the flow before proposing a flow, and incorporating bundle information into flow fingerprinting
- Refactoring the way missing bundles work to retain the desired bundle if available
- Fixing logger.isDebugEnabled to be logger.isTraceEnabled

- Auditing when user changes the bundle. - Ensuring bundle details are present in templates.

Moving standard prioritizers to framework NAR and refactoring ExtensionManager logic to handle cases where an extension is in a JAR directly in the lib directory

- Ensuring all nodes attempt to instantiate the same template instance when the available bundles may differ. - Fixing the auditing of copy/paste and template instantiation. - Running addtional verification methods when running standalone.

Refactoring controller service invocation handler to allow updating the node used by the invocation handler
- Ensuring the bundles in a proposed flow are compatible with the current instance when the current instance has no flow is going to accept the proposed flow
- Merging whether multiple versions of the component are available
- Setting NAR plugin back to current released version
- Cleaning up DocGenerator to not process multiple times

Addressing incorrect usage of nf.Common. - Using formatType in the new component type dialogs.

Improving error messages when looking for bundles

Addressing comments from PR. - Fixing references to global nf namespace. - Fixing injection of nfProcessGroupConfiguration in nfComponentVersion. - Fixing web api integration tests.

Not rendering unversioned in help documentation. - Ensuring the isExtentionMissing flag is correct after changing the component type.

Adding synchronization in node classes to ensure changing component can't occur when component is running, introducing MissingBundleException for better reporting when a node can't join cluster due to a missing bundle, and bumping NAR plugin to released version 1.2.0

Adding concept of missing components to fingerprinting to ensure nodes agree on missing components when joining a cluster

NIFI-3380: NIFI-3520: - Fixing hive nar dependency. - Marking DBCPService as provided. - Skipping services that require instance classloading and are cobundled with their service API. - Skipping components that require instance classloading and reference service APIs that are cobundled. - Addressing UI issues in the new component dialogs when re-opening with a filter applied.

Fixing checkstyles issue and adding back assume checks to distributed cache server test

Ensuring new component types are sorted correctly when shown initially.

This closes #1585.
2017-03-24 11:06:44 -04:00
Mike Moser 2dd61125d1 NIFI-2481 improved content repo archive description in Admin Guide
This closes: #1603

Signed-off-by: Andre F de Miranda <trixpan@users.noreply.github.com>
2017-03-24 22:49:01 +11:00
Wil Selwood eb37f300c7 NIFI-3634 Fix minor User Guide Typos
Signed-off-by: James Wing <jvwing@gmail.com>
2017-03-22 16:48:09 -07:00
Matt Gilman 9d4239be1e NIFI-3245:
- Adding notes to the controller service tables to better define their availability.
- Clarifying support of Safari.

This closes #1593

Signed-off-by: Scott Aslan <scottyaslan@gmail.com>
2017-03-13 13:48:11 -04:00
Matt Gilman 16bde02ed0
NIFI-3541: - Allowing the user to specify the network interface to send/receive data for a Remote Process Group.
This closes #1550.

Signed-off-by: Mark Payne <markap14@hotmail.com>
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2017-03-06 10:38:15 -05:00
Mark Payne 96ed405d70 NIFI-3356: Initial implementation of writeahead provenance repository
- The idea behind NIFI-3356 was to improve the efficiency and throughput of the Provenance Repository, as it is often the bottleneck. While testing the newly designed repository,
  a handful of other, fairly minor, changes were made to improve efficiency as well, as these came to light when testing the new repository:

- Use a BufferedOutputStream within StandardProcessSession (via a ClaimCache abstraction) in order to avoid continually writing to FileOutputStream when writing many small FlowFiles
- Updated threading model of MinimalLockingWriteAheadLog - now performs serialization outside of lock and writes to a 'synchronized' OutputStream
- Change minimum scheduling period for components from 30 microseconds to 1 nanosecond. ScheduledExecutor is very inconsistent with timing of task scheduling. With the bored.yield.duration
  now present, this value doesn't need to be set to 30 microseconds. This was originally done to avoid processors that had no work from dominating the CPU. However, now that we will yield
  when processors have no work, this results in slowing down processors that are able to perform work.
- Allow nifi.properties to specify multiple directories for FlowFile Repository
- If backpressure is engaged while running a batch of sessions, then stop batch processing earlier. This helps FlowFiles to move through the system much more smoothly instead of the
  herky-jerky queuing that we previously saw at very high rates of FlowFiles.
- Added NiFi PID to log message when starting nifi. This was simply an update to the log message that provides helpful information.

NIFI-3356: Fixed bug in ContentClaimWriteCache that resulted in data corruption and fixed bug in RepositoryConfiguration that threw exception if cache warm duration was set to empty string

NIFI-3356: Fixed NPE

NIFI-3356: Added debug-level performance monitoring

NIFI-3356: Updates to unit tests that failed after rebasing against master

NIFI-3356: Incorporated PR review feedback

NIFI-3356: Fixed bug where we would delete index directories that are still in use; also added additional debug logging and a simple util class that can be used to textualize provenance event files - useful in debugging

This closes #1493
2017-02-22 12:40:06 -05:00
Andrew Lim 855743f933
NIFI-3493 Corrected reference in Admin Guide to http not https for nifi.web.https.network.interface property
This closes #1515.

Signed-off-by: Bryan Rosander <brosander@apache.org>
2017-02-16 13:04:16 -05:00
Andrew Lim 702a0dcf17 NIFI-3480 Fix incorrect Admin Guide documentation regarding anonymous access
This closes #1512
2017-02-16 09:45:41 -05:00
Koji Kawamura 1eb98aefee NIFI-3373: Add nifi.flow.configuration.archive.max.count
- Add 'nifi.flow.configuration.archive.max.count' in nifi.properties
- Change default archive limit so that it uses archive max time(30 days)
  and storage (500MB) if no limitation is specified
- Simplified logic to delete old archives

This closes #1460.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-02-16 14:19:39 +09:00
Aldrin Piri 76fcf5def1 NIFI-3385 - This closes #1440. Updating copyright statements to reflect the new year. 2017-02-16 00:10:02 -05:00
Jeff Storck 8b90343715
NIFI-3355 Allows NiFi to bind to specific network interfaces, with separate interface lists for HTTP and HTTPS.
This closes #1508.

Signed-off-by: Bryan Rosander <brosander@apache.org>
2017-02-15 18:39:26 -05:00
Jeff Storck af2861f105
NIFI-3300 Implemented usage of ZooKeeper chroot capability in the connect string
Updated ZooKeeper connect string parsing tests
Updated admin doc for ZooKeeper Migrator migration of nifi root nodes, updated source and destination ZK check by servers in the connection string instead of the entire connection string
Added check between source and destination ZooKeeper paths to allow data to be written to the same ZooKeeper with a different path
Added test for writing to the same ZooKeeper with a different path
Added type parameter to server list in for ZooKeeperEndpointConfig

This closes #1456.

Signed-off-by: Bryan Rosander <brosander@apache.org>
2017-02-01 17:02:17 -05:00
Pierre Villard c15111d985 NIFI-3390 Added support for multiple LDAP servers. This closes #1441 2017-01-26 08:59:18 -05:00
Andrew Lim a1ecea3600
NIFI-3392 Enhanced documentation for provenance event type definitions.
This closes #1445.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-01-25 19:04:37 -08:00
Sarah Olson e0e6f0bacb
NIFI-3391 Update CRON Scheduling information in the Configuring a Processor section of the User Guide
This closes #1442.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-01-24 18:18:14 -08:00
Pierre Villard 3bb876eb83 NIFI-2908 Added TimeZone to toDate() and format() EL
This closes #1381.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2017-01-17 10:36:47 +09:00
Bryan Bende 47d7157412
NIFI-3299 Added example of sensitive property key migration to the admin guide.
This closes #1404.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2017-01-06 17:55:27 -08:00
jpercivall 7c95634203 NIFI-3286 Updating occurances of 'Scope.NODE' in Developer doc to 'Scope.LOCAL'
This closes #1397.
2017-01-05 18:48:35 +01:00
Sarah Olson 00149603c8 NIFI-3234: Updated two broken links in the Encryption Configuration sectionof the Admin Guide
This closes #1345

Signed-off-by: jpercivall <JPercivall@apache.org>
2016-12-20 17:41:08 -05:00
Mark Rebuck 445c075963
NIFI-3233 Fix minor typo in developer-guide.adoc
This closes #1343.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-12-20 11:18:06 -05:00
Matt Burgess 21ed556691 NIFI-3206: Add ifElse function to Expression Language
This closes #1333

Signed-off-by: jpercivall <JPercivall@apache.org>
2016-12-16 15:13:01 -05:00
Andrew Lim 32e1603bd6
NIFI-3090 Added new flow election cluster properties to Admin Guide property tables
This closes #1313.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-12-12 09:30:38 -05:00
Andrew Lim d701cd8364
NIFI-3178 Corrected missing Operate Palette button images in User Admin Guide
This closes #1314.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-12-12 09:25:15 -05:00
jpercivall e36b37692c NIFI-1582 added state to UpdateAttribute as well as updated a few parts that hadn't be touched in years (referenced the 'FlowFileMetadataEnhancer' processor'. Also added a 'NUMBER_VALIDATOR' to StandardValidators
NIFI-1582 removing the option to use cluster state

NIFI-1582 addressing Oleg's comments

NIFI-1582 No longer forcing numbers as the init value and adding getStateValue() to EL instead of using 'ATTRIBUTE_state'

NIFI-1582 Removing init state value

NIFI-1582 Adding documentation for the changes to Init State value

This closes #319
2016-12-06 16:52:02 -05:00
Andrew Lim bfe91db857
NIFI-3143 User Guide corrections needed for screenshots and text
This closes #1291.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-12-02 12:49:14 -08:00
Matt Gilman c5ef076786
NIFI-2325:
- Adding support for LDAPS.

This closes #1275.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-12-02 12:41:03 -08:00
Sarah Olson afe742700c
NIFI-3056 Updated User Guide screenshot and text with latest UI color changes
This closes #1269.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-12-01 22:52:42 -08:00
Andrew Lim 893f87a389 NIFI-3137 Add documentation for User Policies window
This closes #1286
2016-12-01 15:43:20 -05:00
joewitt 92f17a995b NIFI-3100-rc2 prepare for next development iteration 2016-11-25 23:49:27 -05:00
joewitt 5536f690a8 NIFI-3100-rc2 prepare release nifi-1.1.0-RC2 2016-11-25 23:49:13 -05:00
Andy LoPresto 2c3714536f
NIFI-3024 Added key migration for sensitive processor properties contained in flow.xml.gz. (nifi.sensitive.props.key)
This closes #1261.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-11-23 13:26:18 -08:00
Andrew Lim d5387679d1 NIFI-3041 Removed references to Windows start-nifi.bat and stop-nifi.bat in Admin Guide
This closes #1257.
2016-11-22 21:55:10 +01:00
Matt Gilman 7f5eabd603
NIFI-3050: Implemented access control logic for restricted components.
- Addressing comments from PR.
- Adding restricted tags to relevant components.
- Showing a restricted icon overlay on the processor node on the canvas. (+1 squashed commit)
Squashed commits:
[f487682] NIFI-3050:
- Introducing a Restricted annotation for components that require elevated privileges to use.
- Updating the new Processor, Controller Service, and Reporting Task dialogs to include these details and prevent unauthorized selection.
- Including the Restricted description in the generated component documentation.
- Updating processor access control integration test to verify restricted component creation.
- Updating the developer, user, and admin guide to include the restricted component policy.

This closes #1247.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-11-21 12:07:48 -08:00
Andrew Lim 7e7739819a
NIFI-3058 Add documentation for ZooKeeper Migrator to Admin Guide
NIFI-3058 ZooKeeper Migration edits and general spelling corrections

This closes #1246.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-11-21 13:37:57 -05:00
Matt Gilman c8830742ee NIFI-3020:
- Introducing a strategy for identifying users.
- Fixing issue with the referral strategy error message.
- Adding code to shutdown the application when the authorizer or login identity provider are not initialized successfully.

NIFI-3020:
- Updating the admin guide to document the identity strategy.

NIFI-3020:
- Ensuring the request replicator attempts to shutdown regardless of whether the flow service properly terminates.

This closes #1236
2016-11-19 08:38:41 -05:00
Andrew Lim 5a25884f52 NIFI-3052 Update screenshots in Admin Guide and Getting Starting guide for additional colors added to UI
This closes #1239
2016-11-18 14:05:17 -05:00
jpercivall c4be800688 NIFI-2950 Adding support for whole number hex values and a fromRadix function
NIFI-2950 Fixing typo

This closes #1161
2016-11-17 07:27:51 -05:00
Bryan Rosander e5eda63705
NIFI-2943 - Toolkit uses JKS type over PKCS12 when creating truststore because non-Bouncy Castle providers cannot read certificates from PKCS12 truststore.
Peer review feedback (+2 squashed commits)
Squashed commits:
[0102c8e] NIFI-2943 - Peer review feedback
[9bcd495] NIFI-2943 - pkcs12 keystore improvements

1. loading pkcs12 keystores with bouncy castle everywhere
2. tls-toolkit client using jks truststore when keystore type is specified differently
3. tests

This closes #1165.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-11-16 17:13:10 -08:00
Andrew Lim 2c135c44f2 NIFI-2985 Update User Guide for Backpressure Visual indicator
This closes #1234.
2016-11-16 20:58:10 +01:00
Andrew Lim 76b859c4ef NIFI-2953 Update Multi-tenant authorization doc for changes to policy management UI. This closes #1225 2016-11-15 16:10:46 -05:00
Andy LoPresto 59fea1cb4e NIFI-2654 Enabled encryption coverage for login-identity-providers.xml.
Squashed commits:
[5dd22a9] NIFI-2654 Updated administration guide with login-identity-providers.xml flags.

Exposed master key retrieval code in NiFiPropertiesLoader.
Added logic to decrypt login identity providers XML configuration.
Updated login-identity-providers.xsd to include encryption scheme attribute.
Added unit tests. (+18 squashed commits)
Squashed commits:
[57c815f] NIFI-2654 Resolved issue where empty LIP property elements could not be encrypted.
Added unit test and resource.
[27d7309] NIFI-2654 Wired in serialization logic to write logic for LIP.
Added comprehensive unit test for LIP & NFP in same test.
[b450eb2] NIFI-2654 Finalized logic for preserving comments in LIP parsing.
[5aa6c9c] NIFI-2654 Added logic for maintaining XML formatting (comments and whitespace) for LIP.
Added unit tests (w/o encryption works; w/ does not).
[b53461f] NIFI-2654 Added unit test for full tool invocation migrating a login-identity-providers.xml file and updating file and bootstrap.conf with key.
[2d9686c] NIFI-2654 Updated tool description and various logging statements.
Added unit test for full tool invocation encrypting a login-identity-providers.xml file and updating file and bootstrap.conf with key.
[8c67cb2] NIFI-2654 Added logic to encrypt LIP XML content.
Added unit tests.
[8682d19] NIFI-2654 Added logic to handle "empty" (commented) LIP files.
Added unit tests.
[077230e] NIFI-2654 Fixed logic to decrypt multiline and multiple-per-line XML elements.
Added unit tests and resources.
[d5bb8da] NIFI-2654 Ignored unit test for unreadable conf directory because directory was causing Maven build issues.
Removed test resources.
[7e50506] NIFI-2654 Fixed AESSensitivePropertyProvider bug handling cipher text with whitespace.
Added unit test.
[b69a661] NIFI-2654 Fixed AESSensitivePropertyProviderFactoryTest to reflect absence of key causes errors.
[6f821b9] NIFI-2654 Added standard password to arbitrary encryption test for use in test resources.
[d289ffa] NIFI-2654 Added LIP XML decryption.
Added unit tests.
[a482245] NIFI-2654 Added LIP test resources.
[7204df4] NIFI-2654 Changed logic to only perform properties encryption when file path is provided.
[729e1df] NIFI-2654 Removed population of default file locations for bootstrap.conf, nifi.properties, and login-identity-providers.xml as not all files may be desired.
Added/updated unit tests.
[7dba5ef] NIFI-2654 Started LIP work (arguments & parsing).
Added unit tests.

Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #1216
2016-11-14 22:39:06 -05:00
Matt Gilman bff89f17b3 NIFI-401: - Minor tweaks to PR #1117. - Ensuring existing configuraiton is retained and shown until the user explicits changes it. - Retaining, but disabling, deprecated options.
This closes #1185
This closes #1117
This closes #512

Signed-off-by: jpercivall <JPercivall@apache.org>
2016-11-08 17:06:43 -05:00
Bryan Bende d1d053725b NIFI-2909 Adding per-instance class loading capability through @RequiresInstanceClassLoading annotation
NIFI-1712 Applying per-instance class loading to HBaseClientService to allow specifying Phoenix Client JAR

-Refactoring the ClassLoading so that every processor, controller service, and reporting task gets an InstanceClassLoader with a parent of the NAR ClassLoader, and only components with @RequiresInstanceClassLoading will make a copy of the NAR ClassLoader resources, and addressing some review feedback

This closes #1156
2016-11-08 13:21:27 -05:00
Andy LoPresto 89eb2ce28c
NIFI-2652 Provided key migration capability for encrypted config tool.
Added test resources with 128-bit encryption for environments without unlimited strength cryptographic jurisdiction policies installed. All tests pass in both 128- and 256-bit environments. (+8 squashed commits)
Squashed commits:
[55f127c] NIFI-2652 Updated Admin Guide with instructions for encrypted config key migration.
[05abf0e] NIFI-2652 Added unit tests for negative cases for migration argument parsing.
Cleaned up TODOs and comments.
[9b73b22] NIFI-2652 Removed SCrypt mock from one unit test that didn't need it. Test pollution is removed and all tests pass.
[d17ea77] NIFI-2652 Removed SCrypt mock from one redundant unit test. One offender remains ignored.
[0924ce0] NIFI-2652 Removed SCrypt mock from one unit test that did not need it. Two offenders remain ignored.
[cb5f850] NIFI-2652 Expanded unit test for combinations into individual tests due to System.exit() only be capturable once per test.
Three tests which mock Scrypt for speed are temporarily ignored to perform test pollution identification.
[c9cc5dc] NIFI-2652 Added logic and unit test for all combinations of original key/password and new key/password.
[19713ec] NIFI-2652 Implemented first pass of key migration logic and provided single comprehensive unit test.

This closes #1186.
2016-11-07 15:33:10 -08:00
Andrew Lim b7c6392183 NIFI-2960 Remove unused/outdated doc images. This closes #1169 2016-11-03 13:54:04 -04:00
Andrew Lim 60da897e10 NIFI-2915 Updated reference to correct View Details image; Replaced RPG secure/not secure images; Changed template Note style to match rest of documentation. This closes #1168 2016-11-03 12:24:09 -04:00
jpercivall 1d74b5d3ce NIFI-2791 Adding 'math' expression language function
This closes #1157.
2016-10-25 19:45:38 +02:00
jpercivall e4a3e09643 NIFI-1662 Adding proper UI regex support for decimals in EL
This closes #1018
2016-10-24 10:31:16 -04:00
jpercivall 94ab999026 NIFI-1662 adding Expression Language decimal support 2016-10-24 10:20:21 -04:00
Andrew Lim 979b4d8ab9
NIFI-2801 Edited Kafka processor documentation to explicitly state which Kafka versions supported by each processor
This closes #1119.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-10-12 10:35:45 -07:00
Koji Kawamura 540ef63efa NIFI-2855: Site-to-Site with port forwarding.
- Added following properties:
  - nifi.web.http.port.forwarding
  - nifi.web.https.port.forwarding

This closes #1100.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2016-10-07 15:00:37 +09:00
Andrew Lim 53f7a21663 NIFI-2691 Replaced references to kerberos/spegno principle with principal in nifi.properties and admin guide
This closes: #1105

Signed-off-by: Andre F de Miranda <trixpan@users.noreply.github.com>
2016-10-07 01:22:49 +11:00
Joe N 6f1af31ff2 NIFI-2852 base64 expression language functions
Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-10-05 17:33:07 -04:00
Lee Laim 4dbdfea5dc
minor typo in expression-language-guide.doc
delimiter in example was asciidoc syntax

This closes #1098.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-10-04 17:27:44 -07:00
Andrew Lim 3ac3b8dc91 NIFI-2639 Update NiFi In Depth Doc: made text edits and replaced screenshots with new UI
This closes #1067

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-09-27 11:41:48 -04:00
Andre F de Miranda 446cd44702 NIFI-2816 - Clean typos across the code
This closes #1057.
2016-09-26 17:47:31 +02:00
Andre F de Miranda aa933a1941 NIFI-2755 - Fixes minor typo in Developers Guide
This closes #1003
2016-09-15 10:12:04 -04:00
Mark Payne a7e76cc00a NIFI-1966: When cluster is started up, do not assume that Cluster Coordinator has the golden copy of the flow but instead wait for some period of time or until the required number of nodes have connected, and then choose which flow is correct. This closes #977 2016-09-06 16:31:37 -04:00
Joseph Percivall 1fe18a1567 NIFI-2676-rc1 prepare for next development iteration 2016-08-26 11:40:58 -04:00
Joseph Percivall 74d5224783 NIFI-2676-rc1 prepare release nifi-1.0.0-RC1 2016-08-26 11:40:44 -04:00
Andrew Lim 7bc255b9b1
NIFI-2648 Edit formatting of TLS toolkit section of Admin guide for improved readability and consistency
This closes #934.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-08-24 14:37:35 -07:00
Andrew Lim 487f05a2b1
NIFI-2451 Updated Admin Guide with encrypted config documentation.
Changed "the user" to "you" in new sections for consistency. (+3 squashed commits)
Squashed commits:
[aadbbd1] NIFI-2451 Incorporated Drew's final comments to Admin Guide.
[b9a04ea] NIFI-2451 Added new documentation regarding encrypt-config tool after changes in NIFI-1831.
[5c07e68] NIFI-2451 Update Admin guide for encrypt-config command utility, new nifi.sensitive.props.additional.keys property, and removal of Java 7 reference for JCE Unlimited Strength Jurisdiction Policy

This closes #926.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-08-24 13:08:38 -07:00
Sarah Olson 7a4fed189c NIFI-2638: Updated iconProcessor.png with image from new UI.
This closes #924

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-08-23 18:07:13 -04:00
James Wing 7884d09948 NIFI-2536 Update User Guide Template section. This closes #852 2016-08-22 15:07:54 -04:00
Sarah Olson 4847ed28a1 NIFI-2410: This closes #885. updates to the Controller Services information in the User Guide. 2016-08-18 13:48:00 -04:00
Bryan Rosander 5b58e5a251
NIFI-2477 - Updating admin guide to account for NIFI-2476 NIFI-2477 - Adding client dn to doc, examples
This closes #875.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-17 16:36:40 -04:00
Andrew Lim 9a16ca72d7
NIFI-2477 Document TLS generation tool in Admin and Developer Guides
Signed-off-by: Bryan Bende <bbende@apache.org>
2016-08-17 16:36:25 -04:00
Sarah Olson 74661e6623 NIFI-2556 This closes #868. Updated User Guide with informaiton about custom properties 2016-08-17 13:24:49 -07:00
Matt Gilman 3f7216ab84 NIFI-2561: - Decoupling kerberos service and spnego principles and keytabs.
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #873
2016-08-17 08:14:05 -04:00
James Wing 4ec34da99b NIFI-2563 This closes #853. Fixing User Guide Label Icon 2016-08-14 14:42:12 -04:00
Andrew Lim d32a7ba761 NIFI-2449 This closes #845. Update Admin and Getting Started guides for custom properties 2016-08-14 14:38:01 -04:00
Devin Fisher 28b5dbec3a Changed the heading for doc to encode instead of escape
Signed-off-by: Matt Burgess <mattyb149@apache.org>
2016-08-12 11:30:06 -04:00
Devin Fisher c385651209 Added documentation for the new expression language functions for escapes
Signed-off-by: Matt Burgess <mattyb149@apache.org>
2016-08-12 11:30:06 -04:00
Andrew Lim 916292994c NIFI-2541 This closes #831. Update Multi-tenant authorization doc for UI change to Policy page 2016-08-10 15:47:59 -04:00
joewitt f7d2cd69d4 NIFI-2499 edited image paths in asciidoc 2016-08-09 16:37:20 -04:00
Sarah Olson 623d56c0d5 NIFI-2499 This closes #825. Updated User Guide screen shots and text to reflect the new UI. Added two new screenshots as well. 2016-08-09 16:37:20 -04:00
joewitt 120d2100a3 NIFI-2506 fixed build warns on administration-guide.doc 2016-08-08 10:04:51 -04:00
Andrew Lim 3fc67df0b4 NIFI-2506 This closes #799. Update screenshot in README.md for new 1.0 UI 2016-08-08 10:04:00 -04:00
Andrew Lim 026a8d53d2 NIFI-2502 This closes #797. Update Multi-tenant Authorization section in Admin Guide for addition of data policies 2016-08-08 09:53:56 -04:00
Andrew Lim 7a5560c33f NIFI-2420 This closes #763. This closes #762. Updated Admin Guide with Multi-tenant Auth section and related system properties 2016-08-01 22:25:08 -04:00
Rob Moran 731050352d NIFI-2450 Update Getting Started guide, Data Provenance section graphics 2016-08-01 21:55:52 -04:00
Andrew Lim 4d17504a0f NIFI-2434 This closes #747. Correct outdated info in Admin Guide: remove Java 8 refs, update browser versions, new weak crypto screenshot, new cluster screenshot, edit Site-to-Siteproperty descriptions, move cluster node properties to proper section 2016-08-01 11:38:59 -04:00
Andrew Lim 986f951ae8 NIFI-2321 This closes #707. Update Getting Started doc for 1.0 UI changes: made text edits/improvements and added new screenshots 2016-07-28 22:00:04 -04:00
Rob Moran 7e2740160a NIFI-2361 This closes #708. Update cluster information and add new graphic 2016-07-22 14:15:00 -04:00
Rob Moran 99b4af7820 NIFI-2211 This closes #700. This closes #691. Updating NiFi architecture graphics for overview doc 2016-07-21 12:16:21 -04:00
Andrew Lim 0e8e19a479 NIFI-2320 Update existing Getting Starting images with new UI images 2016-07-21 10:57:06 -04:00
Mark Payne 0ce352d203 NIFI-2145: Fixed some typos/misspellings 2016-07-14 11:09:42 -04:00
Koji Kawamura 30889995cb NIFI-2145: Auto flow.xml archive
- Added following properties:
  - nifi.flow.configuration.archive.enabled
  - nifi.flow.configuration.archive.max.time
  - nifi.flow.configuration.archive.max.storage
- Removed manual archive operation:
  - Removed 'Back-up flow' link from UI since it's not needed any longer
  - Removed corresponding REST API controller/archive and its
    implementations
- Added FlowConfigurationArchiveManager to enclose archive related code
- Updated related docs
2016-07-14 10:35:16 -04:00
joewitt cd1bccef8c NIFI-2258 resolved formatting issues causing build warnings
This closes #648
2016-07-14 10:10:35 -04:00
Haimo Liu 59ad51af6d NIFI-2209 This closes #623. Update overview.adoc
Updated NIFI overview documentation
2016-07-08 16:17:43 -07:00
Haimo Liu d61b80758c Add images for zero-master clustering
Add architecture images for zero-master clustering
2016-07-08 16:06:04 -07:00
Koji Kawamura c120c4982d NIFI-1857: HTTPS Site-to-Site
- Enable HTTP(S) for Site-to-Site communication
- Support HTTP Proxy in the middle of local and remote NiFi
- Support BASIC and DIGEST auth with Proxy Server
- Provide 2-phase style commit same as existing socket version
- [WIP] Test with the latest cluster env (without NCM) hasn't tested yet

- Fixed Buffer handling issues at asyc http client POST
- Fixed JS error when applying Remote Process Group Port setting from UI
- Use compression setting from UI
- Removed already finished TODO comments

- Added additional buffer draining code after receiving EOF
- Added inspection and assert code to make sure Site-to-Site client has
  written data fully to output
stream
- Changed default nifi.remote.input.secure from true to false

This closes #497.
2016-06-09 15:09:57 -04:00
Chris McDermott abad7d805e NIFI-1660 - Enhance the expression language with jsonPath function 2016-06-07 15:34:36 -04:00
Mark Payne cd011731ab NIFI-1960: Update admin guide regarding documentation for clustering
NIFI-1960: Updates to guide as follow-up from PR review
2016-06-03 15:23:39 -04:00
Andy LoPresto 8127314975
NIFI-1919 Added replaceFirst() expression language method which accepts literal or pattern for replacement.
Reverted whitespace changes. (+8 squashed commits)
Squashed commits:
[329755c] NIFI-1919 Reverted import re-organization from IDE.
[cf73c2f] NIFI-1919 Updated expression language guide.
[d9a1455] NIFI-1919 Reverted changes to ReplaceEvaluator.
Added ReplaceFirstEvaluator.
Added replace first logic to Query buildFunctionEvaluator.
Added unit tests.
[e2eb880] NIFI-1919 Added replaceFirst to AttributeExpression lexer and parser grammar definitions.
[11fe913] NIFI-1919 Ignored demonstrative test for replaceAll as it behaves as expected.
[af97be1] NIFI-1919 Changed ReplaceEvaluator to use String#replaceFirst which interprets regex instead of compiling as literal.
Demonstrative unit test now passes but two existing unit tests fail. I am not sure these tests are correct.
[f24f17b] NIFI-1919 Added working unit test to illustrate fix.
[8a0d43b] NIFI-1919 Added Groovy unit test to demonstrate issue.
Added DelegatingMetaClass code to record it (test not complete).

This closes #474.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-06-01 16:39:58 -07:00
Pierre Villard 372ffb8aa0 NIFI-1811 Removed ProcessorLog and updated dependent interfaces
This closes #403.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-05-19 13:08:09 -04:00
James Wing 347b281b2d NIFI-1887 Updating default timeout in Admin Guide
This closes #447.

Signed-off-by: James Wing <jvwing@gmail.com>
2016-05-16 11:57:33 -07:00
Pierre Villard b7aa381ab4 NIFI-1826 Updated documentation
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #396
2016-05-11 10:02:11 -04:00
Andrew Lim dc4f983c7a
NIFI-1862 User Guide corrections/improvements
Made multiple edits to the User Guide documentation for correcting errors (spelling/grammatical) and improving readability.

This closes #427.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-05-09 10:31:33 -07:00
Yolanda M. Davis defb6f5b61 NIFI-361 - Create Processors to mutate JSON data
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #354
2016-05-03 14:38:11 -04:00
jpercivall 0557095613 NIFI-1028 initial commit of NiFi In Depth documentation
NIFI-1028 addressing review comments
This closes #339
2016-05-03 14:01:45 -04:00
Matt Burgess 106b0fa0fc NIFI-981: Added SelectHiveQL and PutHiveQL processors
This closes #410.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-05-03 13:51:38 -04:00
Aldrin Piri eded0de154 NIFI-1807 Adding information on volatile content repository configuration settings. 2016-04-23 14:53:51 -04:00
Randy Gelhausen b4309e86bd NIFI-1807 Update administration-guide.adoc adding flowfile, provenance, and content repository volatile implementation documentation.
This closes #378.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-04-23 14:53:27 -04:00
Chris McDermott 32398693d5 NIFI-1661 add random() function to expression language.
Signed-off-by: Joe Skora <jskora@apache.org>
2016-04-21 14:05:00 -04:00
Matt Gilman 153f63ef43 NIFI-1551:
- Removing the AuthorityProvider.
- Refactoring REST API in preparation for introduction of the Authorizer.
- Updating UI accordingly.
- Removing unneeded properties from nifi.properties.
- Addressing comments from PR.
- This closes #359.
2016-04-15 16:03:00 -04:00
Andrew Lim acfc01213c
Update getting-started.adoc
Corrected button, menu item and icon inconsistencies/errors.  Fixed bulleted list formatting error in "Working with Templates" section.

This closes #329.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-04-12 17:08:15 -07:00
andrewmlim e66315c4db Update getting-started.adoc
Corrected spelling/grammatical errors

This closes #327.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-04-05 10:47:52 -07:00
Matt Gilman 2de7f3f884 Updating versions to 1.0.0-SNAPSHOT. 2016-04-04 11:36:20 -04:00
Aldrin Piri e977729b56 NIFI-1634-rc2 prepare for next development iteration 2016-03-23 18:56:34 -04:00
Aldrin Piri 0b9bd20d31 NIFI-1634-rc2 prepare release nifi-0.6.0-RC2 2016-03-23 18:56:22 -04:00
Aldrin Piri b44b177039 NIFI-1605 Adjust documentation and resources to reflect nifi.provenance.repository.rollover.time default
This closes #263

Signed-off-by: Matt Burgess <mattyb149@apache.org>
2016-03-17 22:14:24 -04:00
Andy LoPresto 76f2d5702f NIFI-1274 Added Kerberos authentication mechanism.
NIFI-1274 Cleaned up TODO statements. (+3 squashed commits)
Squashed commits:
[fd101cd] Removed logic to check for presence of services to determine if token support is enabled when username/password authentication is enabled (Kerberos also requires tokens).
[c2ce29f] Reverted import changes to RulesResource.java.
[c269d72] Added Kerberos authentication mechanism.

Moved Kerberos service wiring from XML to Java to handle scenario where admin has not configured Kerberos (previously threw NullPointerException in FileSystemResource constructor). (+15 squashed commits)
Squashed commits:
[09fc694] Added Kerberos documentation to Admin Guide.
[ecfb864] Cleaned up unused logic.
[157efb3] Added logic to determine if client certificates are required for REST API (login, anonymous, and Kerberos service authentication all disabled).
Cleaned up KerberosService by moving logic to NiFiProperties.
[5438619] Added documentation for Kerberos login-identity-providers.xml.
[3332d9f] Added NiFi properties for Kerberos SSO.
[b14a557] Fixed canvas call to only attempt Kerberos login if JWT not present in local storage.
Added logic to handle ticket validation failure in AccessResource.
Changed wiring of Kerberos service beans to XML in nifi-web-security-context.xml for consistency.
[c31ae3d] Kerberos SPNEGO works without additional filter (new entry endpoint accepts Kerberos ticket in Authorization header and returns JWT so the rest of the application functions the same as LDAP).
[98460e7] Added check to only instantiate beans when Kerberos enabled to allow access control integration tests to pass.
[6ed0724] Renamed Kerberos discovery method to be explicit about service vs. credential login.
[ed67d2e] Removed temporary solution for Rules Resource access via Kerberos ticket.
[c8b2b01] Added temporary solution for Rules Resource access via Kerberos ticket.
[81ca80f] NIFI-1274 Added KerberosAuthenticationFilter to conduct SPNEGO authentication with local (client) Kerberos ticket.
Added properties and accessors for service principal and keytab location for NiFi app server.
Added KAF to NiFiWebApiSecurityConfiguration.
Added AlternateKerberosUserDetailsService to provide user lookup without dependency on extension bundle (nifi-kerberos-iaa-provider).
Added dependencies on spring-security-kerberos-core and -web modules to pom.xml.
[0605ba8] Added working configuration files to test/resources in kerberos module to document necessary config. This version requires the user to enter their Kerberos username (without realm) and password into the NiFi login screen and will authenticate them against the running KDC.
Also includes a sample keystore and root CA public key for configuring a secure instance.
[49236c8] Added kerberos module dependencies to nifi/pom.xml and nifi-assembly/pom.xml.
Added default properties to login-identity-providers.xml.
[928c52b] Added nifi-kerberos-iaa-providers-bundle module to nifi/pom.xml.
Added skeleton of Kerberos authenticator using Spring Security Kerberos plugin.
This closes #284

Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-03-17 13:10:09 -04:00
Mark Payne a7b97419e5 NIFI-1626: Throw an Exception proactively if too much state is attempting to be stored via ZooKeeperStateProvider
NIFI-1626: Updated State Management section of Developer Guide

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-03-16 16:12:00 -04:00
Oleg Zhurakousky 8e7e2916fa NIFI-1464 fixed admin guide 2016-03-11 12:54:50 -05:00
Oleg Zhurakousky 59fac58c96 NIFI-1464 ensured that OnUnscheduled is treated the same as OnScheduled
NIFI-1464 polished javadocs, error messages and docs
2016-03-11 12:54:50 -05:00
Oleg Zhurakousky 0c5b1c27f2 NIFI-1464, Refactored Processor's life-cycle operation sequence
* Simplified and cleaned StandardProcessScheduler.start/stopProcessor methods
* Added stop/start operations to ProcessorNode.
* Removed unnecessary synchronization blocks related to ScheduledState in favor of enforcing order and idempotency via CAS operations. Those synchronization blocks were causing intermittent deadlocks whenever @OnScheduled blocks indefinitely.
* Added support for stopping the service when @OnScheduled operation hangs.
* Fixed the order of life-cycle operation invocation ensuring that each operation can *only* be invoked at the appropriate time
* Removed unnecessary locks from StandardProcessNode since Atomic variables are used.
* Removed calls to @OnStopped from ContinuallyRunningProcessTask while ensuring that procesor's full shut down in implementation of StandardProcessorNode.stop() method.
* Removed dead code
* Added comprehensive tests suite that covers 95% of Processor's life-cycle operations within the scope of FlowController, StandardProcesssScheduler and StandardProcessNode
* Improved and added javadocs on covered operations with detailed explanations.
2016-03-11 12:54:50 -05:00
Richard Miskin 99c7fe3b44 NIFI-963 Update admin guide to cover configuration of multiple lib directories
This closes #250.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-27 10:01:38 -05:00
Matt Gilman bb2d2c3674 Clarifying embedded ZooKeeper on NCM is not possible. 2016-02-22 16:34:43 -05:00
Matt Gilman 8cff13e749 Minor changes to the state management section of the admin guide. 2016-02-22 15:21:56 -05:00
Tony Kurc 6c2ba997a0 Merge branch 'NIFI-1379-RC3' 2016-02-16 20:13:41 -05:00
Randy Gelhausen e4d0ec7e60 NIFI-1508: NiFi Site to Site doc tweak
This closes #215.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-16 15:17:28 -05:00
Tony Kurc ad73b5c9d9 NIFI-1379-RC3 prepare for next development iteration 2016-02-12 17:28:10 -05:00
Tony Kurc 8309dba80b NIFI-1379-RC3 prepare release nifi-0.5.0-RC3 2016-02-12 17:27:59 -05:00
Lars Francke d01449ee72 NIFI-1496: State Manager documentation link wrong. This closes #214
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2016-02-10 08:46:42 -05:00
Tony Kurc 5be83166ee NIFI-1485: Updated 'header' section of NiFi NOTICE files 2016-02-07 13:52:00 -05:00
Tony Kurc 303f8eabf1 NIFI-1379: Move to 0.5.0-SNAPSHOT, add tkurc code signing key to keys 2016-02-06 08:49:48 -05:00
Andy LoPresto b407379670 NIFI-1257 Resolved legacy compatibility issue with NiFi legacy KDF salt length dependent on cipher block size.
Replaced screenshot for NiFiLegacy salt encoding.
Added description of legacy salt length determination in admin guide.
Added logic for NiFiLegacyCipherProvider to generate and validate salts of the length determined by the cipher block size.
Changed EncryptContent to default to Bcrypt KDF.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-05 23:38:58 -05:00
Lars Francke 0d72969053 NIFI-1482 - DeveloperGuide :: Controller Service is using wrong method name
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-05 23:35:10 -05:00
Aldrin Piri 4df6512126 Merge branch 'NIFI-259' 2016-02-05 14:09:29 -05:00
Andy LoPresto 498b5023ce NIFI-1257 NIFI-1259
Added utility method to return the maximum acceptable password length for PBE ciphers on JVM with limited strength crypto because BC implementation is undocumented (based on empirical evidence).
Updated EncryptionMethod definitions to accurately reflect need for unlimited strength crypto according to algorithm key length.
Added processor logic to invoke keyed cipher.
Added EncryptContent processor property for raw hex key (always visible until NIFI-1121).
Added validations for KDF (keyed and PBE) and hex key.
Added utility method to return list of valid key lengths for algorithm.
Added description to allowable values for KDF and encryption method in EncryptContent processor.
Added IV read/write to KeyedCipherProvider and changed from interface to abstract class.
Added salt read/write logic to NifiLegacy and OpenSSL cipher providers.
Changed RandomIVPBECipherProvider from interface to abstract class.
Updated strong KDF implementations.
Renamed CipherFactory to CipherProviderFactory.
Added unit test for registered KDF resolution from factory.
Updated default iteration count for PBKDF2 cipher provider.
Implemented Scrypt cipher provider.
Added salt translator from mcrypt format to Java format.
Added unit tests for salt formatting and validation.
Added surefire block to groovy unit test profile to enforce 3072 MB heap for Scrypt test.
Added local Java implementation of Scrypt KDF (and underlying PBKDF2 KDF) from Will Glozer.
Defined interface for KeyedCipherProvider.
Implemented AES implementation for KeyedCipherProvider.
Added Ruby script to test/resources for external compatibility check.
Added key length check to PBKDF2 cipher provider.
Changed default PRF to SHA-512.
Added salt and key length check to PBKDF2 cipher provider.
Added utility method to check key length validity for cipher families.
Added Bcrypt implementation.
Implemented PBKDF2 cipher provider.
Added default constructor with strong choices for PBKDF2 cipher provider.
Implemented NiFiLegacyCipherProvider and added unit tests.
Added key length parameter to PBKDF2 cipher provider.
Added PRF resolution to PBKDF2 cipher provider.
Added RandomIVPBECipherProvider to allow for non-deterministic IVs.
Added new keyed encryption methods and added boolean field for compatibility with new KDFs.
Added CipherFactory.
Improved Javadoc in NiFi legacy cipher provider and OpenSSL cipher provider.
Added KeyedCipherProvider interface.
Added OpenSSL PKCS#5 v1.5 EVP_BytesToKey cipher provider and unit test.

This closes #201.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-04 19:40:55 -05:00
Matt Gilman adfa5dc0eb NIFI-259: - Moving the state management documentation to after the Clustering configuration.
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-03 10:13:23 -05:00
Matt Gilman 7711106d62 NIFI-259: - Adding additional documentation for embedded ZK.
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-03 10:13:19 -05:00
Aldrin Piri 8a668fd344 NIFI-259: Fixed bug that caused StateProvider.replace to return true if the value had never been set, instead of false. Fixed typos in administration-guide
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-02 23:03:31 -05:00
Matt Gilman 72c8467b9f NIFI-259:
- Addressing typo in documentation.
- Minor tweaks to admin guide.
- Adding support to stand up a ZooKeeperServer when a quorum peer is not distributed (ie supporting both embedded standalone and cluster).

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-02-01 14:07:38 -05:00
Aldrin Piri 6902812678 NIFI-259: Adjusting property description of the connect string to be more specific concerning socket tuple format and the associated default port if one is not specified. Providing minor cleanup and removing unused import. 2016-02-01 10:38:45 -05:00
Matt Gilman 4e50263581 NIFI-1445:
- Updating supported browsers sections to accurately reflect the capabilities the UI employs and the dependencies the UI uses.
2016-01-28 15:04:48 -05:00
Mark Payne 8a9a44c102 NIFI-1078: Fixed typo in docs and added additional examples 2016-01-28 09:35:05 -05:00
jpercivall 23364f554c NIFI-1078 Fixing the 'now' EL documentation
Signed-off-by: Mark Payne <markap14@hotmail.com>
2016-01-28 08:56:31 -05:00
Mark Payne 4d88aaedc5 NIFI-1258: Added a new function named getDelimitedField to the Expression Language and put together a guide that walks through how to add a new function
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-01-21 22:09:25 -08:00
Mark Payne f6ec437bc7 Merge branch 'master' into NIFI-259 2016-01-21 09:32:02 -05:00
Matt Gilman 0d7edcb3ac NIFI-108:
- Removing sort from UI.
- Addressing issues with listing and flowfile retrieval when clustered.
- Making the context menu item available when source and destination are still running.
- Adding a refresh button to the queue listing table.
- Fixing the flowfile summary sorting in the cluster manager.
- Adding a message when the source or destination of a connection is actively running.
- Updating the documentation regarding queue interaction.
- Updating the error message when a flowfile is no longer in the active queue.
- Updated queue listing to allow listing to be done while source and destination are running but not sort or have ability to search
- Added heartbeat when we finish clearing queue
- Addressing comments from review.
2016-01-21 08:13:32 -05:00
Mark Payne 593f1288d8 NIFI-259: Bug fixes, unit tests, documentation updates 2016-01-20 10:16:14 -05:00
Mark Payne 0cd6f80f36 NIFI-259: Bug fixes 2016-01-13 15:11:53 -05:00
Mark Payne 0151b1eed1 NIFI-259: Fixed checkstyle violations 2016-01-13 13:57:47 -05:00
Mark Payne bbce596d74 NIFI-259: Updated GetHBase to use new State Management; bug fixes; updated docs 2016-01-13 12:47:08 -05:00
Mark Payne 774c29a4da NIFI-259: Added Stateful annotation as described on ticket 2016-01-12 15:28:35 -05:00
Mark Payne 7a3e3efce1 NIFI-259: Updated documentation, added digest username/password 2016-01-11 16:43:46 -05:00
Mark Payne d2a969e3d6 NIFI-259: Initial implementation of State Management feature 2016-01-11 16:38:52 -05:00
Mark Payne 57dadb7286 NIFI-259: Initial implementation of State Management feature 2016-01-11 08:28:32 -05:00
Jenn Barnabee ad4019f9aa NIFI-1235: Added section on Bending Connections in the Connecting Components section of the User Guide.
Signed-off-by: joewitt <joewitt@apache.org>
2016-01-10 14:52:10 -05:00
joewitt 684f48ff92 NIFI-1374 updated admin guide to provide advice for permgen and codecache
Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-01-10 14:48:20 -05:00
Jon Anderson 035562bb33 NIFI 1007 Updated Admin guide for stats configuration
Signed-off-by: joewitt <joewitt@apache.org>
2016-01-09 22:18:39 -05:00
Mark Payne b330fd1601 NIFI-108: Built out 'skeleton' of the requests so that web tier can be written against it
NIFI-108: Implementing ability to list FlowFiles in a queue

NIFI-108:
- Starting to add support for endpoints that will listing flowfiles in a queue.

NIFI-108: Added merging of response for listing of flowfiles in cluster manager

NIFI-108:
- Starting to add support for endpoints that will listing flowfiles in a queue.

NIFI-108:
- Starting to add support for endpoints that will listing flowfiles in a queue.

NIFI-108:
- Adding checkstyle issues.

NIFI-108: Add clusterNodeId to FlowFileSummaryDTO

NIFI-108: Added unit tests; added verifyCanList method to queue; fixed bugs

NIFI-108:
- Adding compilation error for IOException from getFlowFile().
- Code clean up.
- Javadocs.

NIFI-108:
- Verifying two phase commit for queue listing.
- Fixing checkstyle.
- Ensuring drop and listing requests are merged when created when clustered.

NIFI-108:
- Adding initial listing capabilities.
- Passing through the sort column and direction.

NIFI-108:
- Removing Delete FlowFile button.
- Ensuring sort flags are being passed correctly.
- Setting column widths.
- Also including the cluster node address in the flowfile summaries.

NIFI-108:
- Including queue size statistics in listing request.
- Showing connection name.

NIFI-108:
- Including queue size statistics in listing request.
- Ensuring verifyCanList runs when appropriate.

NIFI-108:
- Adding initial support for viewing flowfile details dialog.
- Adding initial support for click to content.

NIFI-108:
- Allowing the flowfile details dialog to be draggable.

NIFI-108:
- Only showing the flowfile listing table when the listing is successful and the listing is not empty.

NIFI-108:
- Reseting the queue stats when closing the listing table.

NIFI-108: Implemented sorting when performing listing of FlowFiles

NIFI-108: Fixed bug that caused the listFlowFiles operation to wait on a readLock before returning and performing work asynchronously; fixed bug in Write-Ahead FlowFile Repository that caused ContentClaims to be queued up for destruction instead of ResourceClaims - this caused millions of ContentClaims to be queued up instead of a single ResourceClaim in some tests

NIFI-108:
- Ensured the column sort indicator is reset when a new listing is opened.
- Removing unused import.

NIFI-108:
- Addressed issues found during the review.
2016-01-06 16:20:23 -05:00
Lemoda Laim 9bb77163cf NIFI-1334 fixed minor typos in expression-lang-guide documentation
This closes #152

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2015-12-29 12:20:21 -05:00
joewitt f4ac8d75c5 NIFI-1312-RC1 prepare for next development iteration 2015-12-19 00:41:04 -05:00
joewitt d624ea4866 NIFI-1312-RC1 prepare release nifi-0.4.1-RC1 2015-12-19 00:40:53 -05:00
Matt Burgess 04e9606770 NIFI-1290 Document the need for an explicit Validator for a PropertyDescriptor
Signed-off-by: joewitt <joewitt@apache.org>
2015-12-17 09:24:28 -08:00
joewitt d755e43ec8 NIFI-1122_nifi-0.4.0-RC2prepare for next development iteration 2015-12-08 13:00:10 -05:00
joewitt b66c029090 NIFI-1122_nifi-0.4.0-RC2prepare release nifi-0.4.0-RC2 2015-12-08 12:59:59 -05:00
joewitt 0f3a620154 NIFI-1250 removed erroneous ContentType reference and added docs for attributes to index
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2015-12-04 12:58:02 -05:00
Matt Gilman fbaf1c36de NIFI-1138: - Fixing typo in expression language documentation.
Signed-off-by: joewitt <joewitt@apache.org>
2015-12-01 16:06:48 -05:00
Matt Gilman 6bce858e4a NIFI-655:
- Updated user guide with screenshots of login process.
- Tweaked wording in admin guide.
- Triggering login on enter press in login form.
2015-11-25 16:42:22 -05:00
Matt Gilman c073253366 NIFI-655:
- Update admin guide with documentation for username/password authentication.
- Setting default anonymous roles to none.
- Making account status messages to users more clear.
- Deleting user keys when an admin revokes/deletes an account.
- Updating authentication filter to error back whenever authentication fails.
2015-11-25 14:17:23 -05:00
Jenn Barnabee b74617ba6a NIFI-1210: Fixes Copy and Paste descriptions and changes several references to canvas instead of graph to maintain consistency. Also capitalizes Site-to-Site in a couple of places for consistency.
Reviewed by Tony Kurc (tkurc@apache.org)
2015-11-22 08:29:14 -05:00
Jenn Barnabee 327342916e NIFI-1210: Makes various changes to the User Guide for the 0.4.0 version release.
Reviewed and amended (see ticket for details) by Tony Kurc (tkurc@apache.org)
2015-11-21 22:57:19 -05:00
Mark Payne d16392c618 NIFI-1114: Added documentation for nifi.kerberos.krb5.file property 2015-11-17 08:45:28 -05:00
Jenn Barnabee f98e335cde NIFI-1178: Updates to Administration Guide - System Properties section.
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-11-17 08:16:45 -05:00
joewitt 99629646fe NIFI-1122 moved to 0.4.0-SNAPSHOT 2015-11-06 23:41:15 -05:00
joewitt 92099ede2f NIFI-1058 2015-11-06 23:20:03 -05:00
Mark Payne 4c0cf7d72b NIFI-973: Created a Getting Started Guide
Signed-off-by: Mark Payne <markap14@hotmail.com>
2015-10-27 12:01:17 -04:00
Mark Payne 0636f0e731 NIFI-810: Merged master into branch 2015-10-25 11:02:40 -04:00
Aldrin Piri f6d3427955 NIFI-1024 Correcting references to nifi.cluster.manager.protocol.port 2015-10-09 14:22:00 -04:00
Mark Payne 4afd8f88f8 NIFI-810: Created RequiresInput annotation and ensure that processors are invalid if connections do not agree 2015-10-07 17:26:14 -04:00
Matt Gilman ded74ec94c NIFI-876 prepare for next development iteration 2015-09-14 21:48:11 -04:00
Matt Gilman 2ec735e350 NIFI-876 prepare release nifi-0.3.0-RC1 2015-09-14 21:48:00 -04:00
Mark Payne 992e841027 NIFI-948: Provide capability to register notifications for nifi lifecycle events 2015-09-13 19:55:12 -04:00
Joseph Percivall 9dae76b9c4 Initial commit of NIFI-867
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2015-09-09 15:09:33 -04:00
Joseph Percivall 59b7a71c8a NIFI-907: addressing comments to fix periods and link from expression lanaguage doc to developer doc
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2015-09-09 13:45:56 -04:00
Joseph Percivall 47e9a3cdee NIFI-907: Addressing comment on adding the value of the attribute within the enum
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2015-09-09 13:45:56 -04:00
Joseph Percivall 690e674a0d NIFI-907: Initial commit of core attributes in doc
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2015-09-09 13:45:56 -04:00
Matt Gilman 6f8eb9f5e3 NIFI-876:
- Updating the version of the nifi-nar-maven-plugin.
- Updating the minimum version of maven.
2015-08-23 12:32:52 -04:00
Mark Payne 4f92193159 NIFI-787: Ensure that Mock Framework passes a ProcessContext to methods using the @OnStopped annotation (if the method takes a ProcessContext object as an argument); updated developer guide to reflect that the object is allowed as an argument
Signed-off-by: joewitt <joewitt@apache.org>
2015-08-19 20:11:01 -04:00
Mark Payne 8ebf1f03c2 Merge branch 'nifi-868' of https://github.com/pukkaone/nifi into NIFI-868 2015-08-19 12:56:26 -04:00
Mark Payne 19f7db6986 NIFI-793: Added multi-threading to the indexing in the Persistent Provenance Repository 2015-08-19 12:24:25 -04:00
Chin Huang 8f98f80938 NIFI-868 Configure advertised host name of Remote Process Group Input Port
Before this change, the host given out to clients to connect to a Remote
Process Group Input Port is the host where the NiFi instance runs.
However, sometimes the binding host is different from the host that
clients connect to. For example, when a NiFi instance runs inside a
Docker container, a client on a separate machine must connect to the
Docker host which forwards the connection to the container.

Add a configuration property to specify the host name to give out to
clients to connect to a Remote Process Group Input Port. If the property
is not configured, then give out the name of host running the NiFi
instance.
2015-08-18 21:35:28 -07:00
Aldrin Piri c0dad8d1d8 NIFI-848 Extending the Administrator Guide to include guidance on providing multiple provenance and content repositories. 2015-08-17 10:46:13 -04:00
joewitt aa99884782 NIFI-850 removed nifi parent, updated nifi pom, moved all nifi subdirs up one level, fixed readme. 2015-08-15 13:12:22 -04:00
joewitt 300952a984 Reworked overall directory structure to make releasing nifi vs maven plugis easier 2015-01-15 21:11:07 -05:00
Jenn Barnabee a6f0948af1 NIFI-260 Replaced four screenshots so they have new logo
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2015-01-14 12:46:53 -05:00
Jenn Barnabee 78c069fb5b NIFI-162 Added a little to the admin guide
Signed-off-by: joewitt <joewitt@apache.org>
2015-01-12 07:56:32 -05:00
joewitt d996060dbf NIFI-162 added some content to the administrator guide 2015-01-09 13:33:34 -05:00
Jenn Barnabee d08cda1b6a NIFI-234 Updated User Guide to include Example Flow, description of prioritizers, updating connections, navigating within a flow (upstream and downstream), and touched on other management tools like Flow History, Flow Settings, and User Management, but said that is covered in Admin Guide.
Signed-off-by: Matt Gilman <matt.c.gilman@gmail.com>
2015-01-08 14:34:49 -05:00
Mark Payne bcc271046e NIFI-218: Fixed typo that caused markup not to render properly for any and all matching attributes functions 2015-01-07 14:38:56 -05:00
Mark Payne e631c493d4 NIFI-218: Added isEmpty and replaceEmpty functions to doc 2015-01-07 14:05:05 -05:00
joewitt 4d11ff7830 NIFI-150 small mod to fix references title 2015-01-06 21:54:29 -05:00
Jenn Barnabee c27f7b0574 NIFI-162 In the User Guide, I added an overview (stolen from the web page), added the Data Provenance section, and copy-edited all the existing sections.
Signed-off-by: joewitt <joewitt@apache.org>
2015-01-06 21:31:04 -05:00
Matt Gilman 23dcdeb61b NIFI-216:
- Ignoring irrelevant whitespace in EL editor.
- Adding IDE style function details for each function in the auto complete.
2015-01-06 11:41:13 -05:00
Mark Payne 2d0b163961 NIFI-221: included controller service needed in nar-bundles pom 2015-01-06 10:16:00 -05:00
Jenn Barnabee 7e70fd53e9 NIFI-162 I copy-edited the Overview
Signed-off-by: joewitt <joewitt@apache.org>
2015-01-05 17:10:23 -05:00
Mark Payne 090ad38679 NIFI-218: Finished documenting the functions; changed formatting so that classnames can be applied to the functions, arguments, description, return type, and subject Types. This allows the values to be pulled out into the UI. 2015-01-05 16:37:17 -05:00
joewitt 9e35c7da9f NIFI-222 moved away from project.version and instead use explicit version declaration. Release plugin should take care of it anyway 2015-01-03 00:58:58 -05:00
Mark Payne 68b7ad78cc NIFI-218: Started Expression Language guide 2014-12-31 16:18:38 -05:00
joewitt ad90fbf24f NIFI-162 more improvements to overview 2014-12-31 12:05:00 -05:00
joewitt d6c02a5640 NIFI-162 added a couple new sections to the overview to address architecture of nifi and performance 2014-12-31 02:39:01 -05:00
Mark Payne 43991777da NIFI-150: Added information about Templates 2014-12-30 15:44:16 -05:00
Mark Payne 6f1c12f5ef Updated user guide 2014-12-29 16:17:48 -05:00
joewitt 87b07384a5 NIFI-162 filled in a basic overview. Definitely still needs work though. Too wordy. Too specific. 2014-12-28 15:23:45 -05:00
joewitt b6f2dd280c NIFI-162 all doc components prepped and integrated to build process. Now just to fill in content 2014-12-26 23:06:31 -05:00
joewitt 21182ac260 NIFI-150 updated to ensure docs get built and end up in the assembly under docs. 2014-12-19 16:34:25 -05:00
joewitt d850768765 NIFI-150 fixed error with inserting license header 2014-12-19 15:32:55 -05:00
joewitt 4031bd5495 NIFI-150 fixed error with inserting license header - still need to figure out the license header addition 2014-12-19 14:57:13 -05:00
Mark Payne 9c38f358b8 NIFI-150: Removed old formatted document and updated asciidoc version of user guide 2014-12-19 14:11:43 -05:00
Mark Payne 62f3354a12 NIFI-150: Finished 'Anatomy of a Processor' section 2014-12-19 14:00:49 -05:00
joewitt 782b2c1882 NIFI-150 updated replace text plugin to avoid double addition of license header 2014-12-19 13:22:39 -05:00
Mark Payne d8911e75e6 NIFI-150: Added additional documentation 2014-12-19 12:28:14 -05:00
Mark Payne c213895cc2 NIFI-150: Added Apache License to the output HTML 2014-12-19 10:01:09 -05:00
Mark Payne 42566cd6aa NIFI-150: Added documentation 2014-12-18 17:25:22 -05:00
Mark Payne a23c7f2e2b NIFI-150: Added documentation in asciidoc format 2014-12-18 17:10:42 -05:00