Commit Graph

88 Commits

Author SHA1 Message Date
Pierre Villard c15111d985 NIFI-3390 Added support for multiple LDAP servers. This closes #1441 2017-01-26 08:59:18 -05:00
jpercivall 273e69f2cb NIFI-3274 Adding WriteAheadLog configuration options to WriteAheadLogLocalStateProvider
This closes #1386.
2017-01-05 19:59:58 +01:00
Bryan Rosander 97d2d30423
NIFI-3131 - Qualifying nifi-env in nifi bat scripts, removing popd from toolkit bat scripts
This closes #1293

Signed-off-by: Bryan Rosander <brosander@apache.org>
2016-12-05 23:03:41 -05:00
Bryan Rosander d1f2492dec
NIFI-3132 - Using cygpath and quoting paths correctly in sh scripts when run in cygwin
This closes #1292

Signed-off-by: Bryan Rosander <brosander@apache.org>
2016-12-05 22:16:24 -05:00
Matt Gilman c5ef076786
NIFI-2325:
- Adding support for LDAPS.

This closes #1275.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-12-02 12:41:03 -08:00
jpercivall e3c7611347
NIFI-3112 Revert "NIFI-2199 - allows nifi.sh restart through ssh"
This reverts commit 88d125137e.

This closes #1282.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-11-29 17:32:47 -08:00
joewitt 92f17a995b NIFI-3100-rc2 prepare for next development iteration 2016-11-25 23:49:27 -05:00
joewitt 5536f690a8 NIFI-3100-rc2 prepare release nifi-1.1.0-RC2 2016-11-25 23:49:13 -05:00
Matt Gilman c8830742ee NIFI-3020:
- Introducing a strategy for identifying users.
- Fixing issue with the referral strategy error message.
- Adding code to shutdown the application when the authorizer or login identity provider are not initialized successfully.

NIFI-3020:
- Updating the admin guide to document the identity strategy.

NIFI-3020:
- Ensuring the request replicator attempts to shutdown regardless of whether the flow service properly terminates.

This closes #1236
2016-11-19 08:38:41 -05:00
James Wing a486fefb1e NIFI-2115 Detailed Version Info in About Box
* Java version and vendor
* OS name and version
* Release Tag
* Build revision (commit SHA), branch, and timestamp
* Handles formal releases, ad-hoc builds, and non-release source builds
* Standalone UI presence in About dialog, Summary -> System Diagnostics
* Cluster UI as Versions tab in Cluster dialog
* Reduce About Dialog Content
* Fix Missing Property Display Bugs
* Marking the build time as type string.
* This closes #583
2016-11-03 11:16:20 -04:00
Michal Klempa 368ea7a2d2 NIFI-1069 Removing the exit directive
This closes #1159

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-10-25 12:44:19 -04:00
Michal Klempa 99d3c39748 NIFI-1069 - improve init script exit codes so that results are LSB compliant
This closes: #1093

Signed-off-by: Andre F de Miranda <trixpan@users.noreply.github.com>
2016-10-20 23:12:23 +11:00
Andre F de Miranda 3914141c45
NIFI-1710 - Resolve path name to nifi.sh script
This closes #1137.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-10-18 16:43:03 -07:00
Michal Klempa 88d125137e NIFI-2199 - allows nifi.sh restart through ssh
Closes: #1092

Signed-off-by: Andre F de Miranda <trixpan@users.noreply.github.com>
2016-10-15 00:48:04 +11:00
Andrew Lim 53f7a21663 NIFI-2691 Replaced references to kerberos/spegno principle with principal in nifi.properties and admin guide
This closes: #1105

Signed-off-by: Andre F de Miranda <trixpan@users.noreply.github.com>
2016-10-07 01:22:49 +11:00
Mark Payne a7e76cc00a NIFI-1966: When cluster is started up, do not assume that Cluster Coordinator has the golden copy of the flow but instead wait for some period of time or until the required number of nodes have connected, and then choose which flow is correct. This closes #977 2016-09-06 16:31:37 -04:00
Joseph Percivall 1fe18a1567 NIFI-2676-rc1 prepare for next development iteration 2016-08-26 11:40:58 -04:00
Joseph Percivall 74d5224783 NIFI-2676-rc1 prepare release nifi-1.0.0-RC1 2016-08-26 11:40:44 -04:00
Bryan Bende 957c120343 NIFI-2664 Moving System.setProperty for krb5.conf to NiFi startup, and removing conflicting property from KerberosProvider config
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #946
2016-08-25 17:26:37 -04:00
Andy LoPresto 6331dda8d3 NIFI-2643 Changed null check to !isBlank() for key in RunNiFi.
NIFI-2643 Updated comment in bootstrap.conf template.

This closes #932
2016-08-24 16:20:10 -04:00
Mark Payne c2ae7a6d7c NIFI-2605: - Fixing a regression bug where nodes would potentially be elected leader for Cluster Coordinator role when they do not have the correct flow
-  Ensure that we log which node is the cluster coordinator on startup instead of just indicating that there is one. If we later determine that there is none, ensure that we register for the role

This closes #900

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-08-24 12:30:48 -04:00
Andy LoPresto c638191a47
NIFI-1831 Added internal logic and command-line tool to allow AES-encrypted sensitive configuration values in nifi.properties.
This closes #834.
2016-08-23 20:34:31 -07:00
Matt Gilman 3f7216ab84 NIFI-2561: - Decoupling kerberos service and spnego principles and keytabs.
Signed-off-by: Yolanda M. Davis <ymdavis@apache.org>

This closes #873
2016-08-17 08:14:05 -04:00
Mark Payne 42df02f014 NIFI-2406 This closes #820. Addressed regression introduced in NIFI-2406 where the cluster does not recognize a new Cluster Coordinator when the coordinator is shutdown 2016-08-09 15:19:49 -04:00
Bryan Rosander fa4c6ab03c
NIFI-2193 - Added functionality to automate certificate generation, keystore and truststore generation, and nifi.properties keystore and truststore password population. Follow-on changes will be made under NIFI-2476.
This closes #695.

Signed-off-by: Andy LoPresto <alopresto@apache.org>

Defaulting to same keyStore, key password (+18 squashed commits)
Squashed commits:
[9d01ba0] NIFI-2193 - Fixing typo
[55440bc] NIFI-2193 - Standalone can run as long as there are no conflicting files/folders
[0ca34ed] NIFI-2193 - Fixing some filename, absolute path issues
[9d4f65b] NIFI-2193 - Incorporating feedback
[f7550b4] NIFI-2193 - Cleaning up imports
[59a7637] NIFI-2193 - Updating umask to allow owner to execute
[cf824e7] NIFI-2193 - Moving DN arg to CA service specific parent class
[921ee13] NIFI-2193 - Making keystore getInstance more consistent
[a283c4b] NIFI-2193 - Updating sample config files in assembly to reflect new structure
[8d3a21d] NIFI-2193 - Making TlsHelper static, adding option to use same password for Key, KeyStore
[b13d247] NIFI-2193 - Addressing PR feedback
[46ef8ed] NIFI-2193 - Removing commons-logging, log4j from notice
[d4cf41a] NIFI-2193 - Adding option to specify output file for CA certificate when using cli client
[b74bf25] NIFI-2193 - Removing Bouncy Castle from notice
[6e34f9a] NIFI-2193 - Adding CLI client for easier generation of client certificates
[2924fca] NIFI-2193 - nifi-toolkit-ssl -> nifi-toolkit-tls, removing unused constants
[886167e] NIFI-2193 - Adding slf4j to avoid runtime issue
[082de46] NIFI-2193 - Command line SSL config utility as well as certificate authority client/server
2016-08-03 21:34:11 -07:00
Mark Payne 16348b071d NIFI-2452: This closes #771. Ensure that we keep track of how many references we have to each lucene searcher and only close the underlying index reader if there are no references to the searcher. Also updated to prefer newer provenance events over older provenance events, and calculate FlowFile lineage based on an event id instead of a FlowFile UUID, as it's much more efficient 2016-08-03 08:05:26 -07:00
Bryan Bende bfdf94d522 Removing extraneous authorizations.xml from nifi-resources/src/main/resources/conf, created automatically in secure mode 2016-08-01 11:07:26 -04:00
Yolanda M. Davis 8412d2662a NIFI-2208 - initial commit Custom Property Expression Language support with Variable Registry, includes bug fix for NIFI-2057
This closes #529

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-07-29 17:10:20 -04:00
Mark Payne 3a153d3a48 NIFI-2337 This closes #731. Turn off logging for org.apache.curator.framework.recipes.leader.LeaderSelector since it logs only a single error, and that error is a but in Curator 2016-07-28 22:29:08 -04:00
Bryan Bende 5e4ba04589 NIFI-2390 Separating of users and groups from authorizations.xml into separate file. This closes #735 2016-07-28 16:41:52 -04:00
Bryan Bende c3b4872b55 NIFI-2389 Refactoring identity mapping and applying it to FileAuthorizer for initial admin, cluster nodes, and legacy authorized users. This closes #719 2016-07-26 15:24:50 -04:00
Bryan Bende 3e9867d5da NIFI-1950 Updating FileAuthorizer to convert access controls from input and output ports during legacy conversion. This closes #702. 2016-07-25 12:37:26 -04:00
Koji Kawamura 30889995cb NIFI-2145: Auto flow.xml archive
- Added following properties:
  - nifi.flow.configuration.archive.enabled
  - nifi.flow.configuration.archive.max.time
  - nifi.flow.configuration.archive.max.storage
- Removed manual archive operation:
  - Removed 'Back-up flow' link from UI since it's not needed any longer
  - Removed corresponding REST API controller/archive and its
    implementations
- Added FlowConfigurationArchiveManager to enclose archive related code
- Updated related docs
2016-07-14 10:35:16 -04:00
Bryan Bende ba763b95e8 NIFI-2003 Creating abstract authentication provider and incorporating into existing providers
NIFI-2201 Add support for seeding cluster nodes in authorizations.xml
- Passing client address along in user context on authorization requests
- This closes #628
2016-07-12 11:20:29 -04:00
Aldrin Piri baed85fa3d NIFI-2063 Adjusting handling of service install
by using script that delegates to installed nifi.sh at NIFI_HOME. Prohibit installation of service for Cygwin and Darwin environments that are guaranteed to not support it.

This closes #561

Signed-off-by: James Wing <jvwing@gmail.com>
2016-06-22 21:12:35 -07:00
ijokarumawak b1b7e35bed NIFI-1843 Default log level for LogAttribute.
- added a logger entry for LogAttribute to specify INFO as log level
- replaced some tab characters to whitespace (existing checkstyle error)

This closes #414.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-06-19 02:21:50 -04:00
Bryan Bende 679ad93f57 NIFI-1804 Adding ability for FileAuthorizer to automatically convert existing authorized-users.xml to new model
- Removing Resources class from file authorizer and updating ResourceType enum
- Updating ResourceFactory to be in sync with ResourceType enum and adding additional required permissions to the auto-conversion
- Adding root process group to the seeding of the initial admin
- Improvement so that users that are already part of a read-write policy, won't end up in a read policy for the same resource
- Removing rootGroupId from authorization context and auto-detecting it from the flow provided through nifi.properties
- This closes #507
2016-06-17 16:33:00 -04:00
Koji Kawamura c120c4982d NIFI-1857: HTTPS Site-to-Site
- Enable HTTP(S) for Site-to-Site communication
- Support HTTP Proxy in the middle of local and remote NiFi
- Support BASIC and DIGEST auth with Proxy Server
- Provide 2-phase style commit same as existing socket version
- [WIP] Test with the latest cluster env (without NCM) hasn't tested yet

- Fixed Buffer handling issues at asyc http client POST
- Fixed JS error when applying Remote Process Group Port setting from UI
- Use compression setting from UI
- Removed already finished TODO comments

- Added additional buffer draining code after receiving EOF
- Added inspection and assert code to make sure Site-to-Site client has
  written data fully to output
stream
- Changed default nifi.remote.input.secure from true to false

This closes #497.
2016-06-09 15:09:57 -04:00
Bryan Bende 8d8a9cba79 NIFI-1916 Updating FileAuthorizer to extend AbstractPolicyBasedAuthorizer and adding intial loading of data users, groups, and policies
- Implementing CRUD operations and unit tests for Users
- Implementing CRUD operations and unit tests for Groups
- Implementing CRUD operations and unit tests for AccessPolicies
- Adding support for seeding with an initial admin user
- Fixing delete for user and group so it removes references from policies
- Adding example to authorizations.xml
- Adding back the old users schema in preparation for auto-converting to the new format, and providing the AuthorizationConfigurationContext with access to the root process group id
- Refactoring some of the FileAuthorizer to ensure thread safety
- Adding /groups to policies created for initial admin
- This closes #473
2016-06-03 17:26:22 -04:00
Mark Payne df0e4e7960 NIFI-1897:
- Refactoring to allow requests to be replicated from a node to other nodes
- Renaming cluster node connection/read timeout properties.
- Renaming NCM DN to Cluster Coordinator DN.
- Fixing default values in properties.
- Starting to fix Spring context to load correctly in standalone mode.
- Using the cluster protocol to handle connection failures instead of heartbeats.
- Ensured replicate call is returned from ControllerResource.
- Ensure the appropriate classloader when serializing templates.
- Handling when the flow contents are null.
- This closes #488
2016-06-03 15:29:16 -04:00
Mark Payne 4b74e4de74 NIFI-1745: Refactor how revisions are handled at NCM/Distributed to Node. This closes #454 2016-05-20 14:04:24 -04:00
Yolanda M. Davis 1370eefdd7
NIFI-1812 initial commit for nifi-env script, includes updates to linux/windows scripts, logback changes, RunNifi/Shutdownhook changes
This closes #386.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-05-10 16:37:30 -07:00
Mark Payne 04c41c0654 NIFI-1678:
- Started refactoring heartbeating mechanism, using a new package: org.apache.nifi.cluster.coordination
- Added configuration options for ZooKeeper username & password for heartbeat management. Also addressed issue where nodes that were previously disconnected were asked to disconnect upon restart
- Ensure that ACL is set properly when creating heartbeat node. Removed unused ControllerStartupFailureMessage.java
- Changed ZooKeeper ACL's so that container nodes that would not be sensitive are wide open and removed the usage of username & password when communicating with ZooKeeper. This was done specifically because username/password combination is considered a 'testing' feature that should not be used in production and is not supported by Apache Curator
- Refactored CuratorHeartbeatMonitor into an abstract heartbeat monitor that is responsible for processing heartbeats and CuratorHeartbeatMonitor that is responsible for retrieving heartbeat information
- Refactored so that heartbeats are sent to Cluster Coordinator directly instead of to ZooKeeper. ZooKeeper is used to know which node is the cluster coordinator but heartbeats to the Cluster Coordinator provide additional information about the nodes.
- Started refactoring heartbeating mechanism, using a new package: org.apache.nifi.cluster.coordination
- Added configuration options for ZooKeeper username & password for heartbeat management. Also addressed issue where nodes that were previously disconnected were asked to disconnect upon restart
- Changed ZooKeeper ACL's so that container nodes that would not be sensitive are wide open and removed the usage of username & password when communicating with ZooKeeper. This was done specifically because username/password combination is considered a 'testing' feature that should not be used in production and is not supported by Apache Curator

NIFI-1727:
- Refactored logic for merging HTTP Requests that are federated across cluster

NIFI-1745:
- Refactoring how HTTP Requests are replicated to nodes
- Bug fixes and continuing to work on replication refactoring. Still need to handle cluster locking and revisions
- Begin work on RevisionManager
- Resolved some issues that resulted from rebase
- Fixed URIs to align with new URI's that will be used in 1.0.0
- This closes #413
2016-05-06 15:23:12 -04:00
Aldrin Piri 6938c3db54 NIFI-1817 Respecting when run.as is commented out by ensuring the chosen line starts with run.as optionally preceded by whitespace
This closes #383.
2016-04-27 15:22:19 -04:00
Mark Payne fb7b3fe4b8 NIFI-1678: Started refactoring heartbeating mechanism, using a new package: org.apache.nifi.cluster.coordination
Added configuration options for ZooKeeper username & password for heartbeat management. Also addressed issue where nodes that were previously disconnected were asked to disconnect upon restart

Ensure that ACL is set properly when creating heartbeat node. Removed unused ControllerStartupFailureMessage.java

Changed ZooKeeper ACL's so that container nodes that would not be sensitive are wide open and removed the usage of username & password when communicating with ZooKeeper. This was done specifically because username/password combination is considered a 'testing' feature that should not be used in production and is not supported by Apache Curator

Refactored CuratorHeartbeatMonitor into an abstract heartbeat monitor that is responsible for processing heartbeats and CuratorHeartbeatMonitor that is responsible for retrieving heartbeat information

Refactored so that heartbeats are sent to Cluster Coordinator directly instead of to ZooKeeper. ZooKeeper is used to know which node is the cluster coordinator but heartbeats to the Cluster Coordinator provide additional information about the nodes.

Code cleanup and incorporate comments from peer review

This closes #323
2016-04-22 15:01:04 -04:00
Matt Gilman 8c09a5c8d2 NIFI-1783: - Addressing mistake in exmample XML. - Ensuring the configured Authorizer is loaded correctly.
This closes #363.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-04-19 09:53:26 -04:00
Matt Gilman 153f63ef43 NIFI-1551:
- Removing the AuthorityProvider.
- Refactoring REST API in preparation for introduction of the Authorizer.
- Updating UI accordingly.
- Removing unneeded properties from nifi.properties.
- Addressing comments from PR.
- This closes #359.
2016-04-15 16:03:00 -04:00
Matt Gilman 5de40ccec3 NIFI-1553:
- Implementing a file based authorizer.
- Providing an example authorizations files.
- Address comments from PR.
- This closes #330
2016-04-07 16:28:42 -04:00
Matt Gilman 3f4ac3156c Revert "NIFI-1551:"
This reverts commit c4d06f203d. Accidently merged wrong branch.
2016-04-07 16:22:35 -04:00
Matt Gilman c4d06f203d NIFI-1551:
- Starting to remove the AuthorityProvider.
- This closes #330
2016-04-07 16:18:36 -04:00