Commit Graph

2658 Commits

Author SHA1 Message Date
Matt Burgess d6391652e0 NIFI-1663: Add ConvertAvroToORC processor
- Code review changes
 - This closes #477.
2016-06-29 12:18:27 -04:00
Bryan Bende f43f47694c NIFI-2138 Making AccessPolicy have a single RequestAction. This closes #590 2016-06-28 16:32:27 -04:00
Jeff Storck 41f3253445 NIFI-1952 Update to revision-locking for users and groups
Adding user and group summary objects (TenantEntity)
Fixed ComponentEntity JSON mapping issues when the id field is null
Removing unecessary revision checking.
Fixing error message when checking user, group, and policy revision.
This closes #589
2016-06-28 15:38:01 -04:00
Bryan Bende 8c837ba1ea NIFI-2127 Adding support for managing the user-group relationship from the Group side. This closes #588 2016-06-28 14:25:38 -04:00
Scott Aslan bb0ea78741 [NIFI-2027] Responsive Tables. This closes #577 2016-06-28 08:16:50 -04:00
Joe Gresock e3d13ecf50 NIFI-920: Adding InvalidPathException handling to UnpackContent
Signed-off-by: Bryan Bende <bbende@apache.org>
2016-06-27 13:49:49 -04:00
jpercivall ff341271a9 NIFI-2118 Addresssing License/Notice issues in nifi-assembly
This closes #586
2016-06-26 22:48:32 -04:00
Jeff Storck 64719b6f9b NIFI-1952 Updated StandardPolicyBasedAuthorizerDAO to throw ResourceNotFoundExceptions when user/group/policy not found
Added spec for StandardPolicyBasedAuthorizerDAO
Added exception mapper for AuthorizationAccessException, added mapper to nifi-web-api-context.xml
Added rest endpoints to get all users and user groups
Merged UsersResource and UserGroupsResource into TenantsResource
This closes #582
2016-06-26 22:23:25 -04:00
Bryan Bende b911c9dbdf NIFI-1916 Improvements to FileAuthorizer to not parse flow when unncessary and to recreate missing authorizations.xml. This closes #581 2016-06-25 17:10:59 -04:00
Matt Gilman 82268afb0d NIFI-1554: - Updating cluster detection to support access through the flow resource. - Using the correct resource when authorizing a root group port during site to site.
This closes #579.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-06-24 16:59:06 -04:00
Mark Payne 8da571d6d6 NIFI-2089: Ensure streams are closed before attempting to remove files
This closes #573

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-06-23 18:43:20 -04:00
Matt Gilman f0811ca45a NIFI-1554:
- Addressing access controls for the Controller resource.
- Addressing access controls for RAW site to site clients.
- Addressing access controls for downloading content (from provenance and queue).
- Addressing access controls for accessing queues.
- Addressing access controls for cluster endpoints.
- Addressing access controls for counter endpoints.
- Removing redundant authorization calls.
NIFI-2044:
- Requiring revision when creating components.
- Requiring component creation over POST requests.
NIFI-1901
- Continuing to restore access control tests.
- Converting access control tests to itegration tests.
- Restoring contrib check to travis build.
- This closes #567
2016-06-23 17:09:54 -04:00
jpercivall f0662a24ef NIFI-2092 Fixing unit tests which try to clean up directories before stopping the FlowController that has locks opened.
This closes #568
2016-06-23 16:40:00 -04:00
Mark Payne fd5327e1b9 NIFI-2059: Ensure that we properly pass along proxied entities in HTTP headers when secure and ensure that we don't keep creating new Root Group ID's once we've created one, even after restart. This closes #572. 2016-06-23 16:32:25 -04:00
Bryan Bende 4f2643f668 NIFI-2061 Added methods to AbstractPolicyBasedAuthorizer to generate and inherit a fingerprint
- Updated StandardFlowSynchronizer to compare authorization fingerprints
- This closes #566
2016-06-23 14:36:04 -04:00
Aldrin Piri baed85fa3d NIFI-2063 Adjusting handling of service install
by using script that delegates to installed nifi.sh at NIFI_HOME. Prohibit installation of service for Cygwin and Darwin environments that are guaranteed to not support it.

This closes #561

Signed-off-by: James Wing <jvwing@gmail.com>
2016-06-22 21:12:35 -07:00
Bryan Rosander 521ce63cc3 NIFI-2045 - Removing mqtt message from queue after session.comit()
This closes #548

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-06-22 16:45:40 -04:00
Joe Skora be83f401fa Fix pom version error pushing to 1.0.0 in nifi-media-bundle/nifi-media-nar/pom.xml.
(originally on commit 7e7d79fc6a)
2016-06-22 15:13:58 -04:00
Joe Skora 7e7d79fc6a NIFI-615 - Create a processor to extract WAV file characteristics.
* Create new ExtractMediaMetadata processor using Apache Tika Detector and Parser.
* Refactored nifi-image-bundle, nifi-image-nar, and nifi-image-processors to nifi-media-bundle, nifi-media-nar, and nifi-media-processors to reflect broader media related purpose.
* Preserved existing ExtractImageMetadata and ResizeImage processors as well as existing ImageViewerController components to prevent impact on existing uses.
* Resolved collision between ExtractImage and ExtractMedia processors due to common dependency on Noakes' Metadata Extractor project.
  - Updated bundle's Tika dependency from 1.7 to 1.8 and Drew Noakes' Metadata Extractor from 2.7.2 to 2.8.0.
  - Adjusted ExtractImageMetadata tests for enhanced attribute names in new Noakes' Metadata Extractor version.
* Fix assembly POM to remove duplicate reference to site-to-site nar and change nifi-image-nar reference to nifi-media-nar.
* Note the potential attribute changes on upgrade due to underlying libraries.

This closes #556.
2016-06-22 15:06:52 -04:00
Mark Payne 0e085bdddd NIFI-2065: When a provenance query matches the max number of results requested, stop querying lucene for improved performance
This closes #558
2016-06-22 13:27:31 -04:00
Mark Payne 36ab8474dd NIFI-1900: Verify that connection's destination is not running when trying to change destination
This closes #550
2016-06-22 12:39:53 -04:00
Jeff Storck f47be77b6a NIFI-1952 Create REST endpoints for user/group/policy management
created REST Resources for users, groups, and access policies
added Authorizables for users, groups, and access policies
added methods to DtoFactory and EntityFactory to create objects for users, groups, and access policies
extracted anonymous AuthorizableLookup impl in StandardNiFiServiceFacade.java to a protected class to make the lookup call mockable in tests
added methods to manage users/groups/access policies to StandardNiFiServiceFacade
added StandardNiFiServiceFacadeSpec to unit-test management of users/groups/access policies
added implementations for UserDAO, GroupDAO, AccessPolicyDAO.
added spring config for user/group/policy resources and daos
Updated user/group/policy creation via REST resources, no longer requires the use of the revision manager
updated StandardNiFiServiceFacadeSpec based on user/group/policy creation changes
condensed user/group/policy DAOs to a single DAO (StandardPolicyBasedAuthorizerDAO)
fixed spring config of user/group/policy REST resources
Updated to return ComponentEntity objects instead of just their IDs
mid-progress on updating tests
updated code and tests to return component entities from REST endpoints for users, groups, policies
This closes #526
2016-06-22 10:12:41 -04:00
Scott Aslan 32facaedb4 [NIFI-1879] Update dialog content scrollable styles. This closes #560 2016-06-22 08:11:12 -04:00
Matt Gilman 27ff5f9a87 NIFI-2075:
- Fixing issue rendering event rows while visible in the shell.
- Fixing issue go to/from the event table and lineage graph.
- Fixing visibility of the event table header, search, and filter controls.
- This closes #559
2016-06-22 07:47:03 -04:00
Oleg Zhurakousky 1eeef0e1c0 NIFI-2067 ignored intermittently failing MemoryTest left comment with the explanation as to why it was not removed
This closes #555

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-06-21 22:18:13 -04:00
Scott Aslan 8ed66e5df1 [NIFI-1879] Update containment for status history dialog. This closes #557 2016-06-21 21:49:36 -04:00
Bryan Rosander 61a2054b38 NIFI-2046 - Adding reference to @TriggerSerially in Processor javadoc
This closes #549

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-06-21 17:31:09 -04:00
Matt Gilman 9e54a3d260 NIFI-1946: - Updating palette styling.
- Providing selection context.
- Updating add connection icon.
NIFI-2058: - Updating defaults for connection backpressure.
NIFI-2014: - Updating birds eye colors.
NIFI-2016: - Updating default label color.
NIFI-2013: - Updating how we color processors.
NIFI-2043: - Using correct group id when creating controller services inline.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-06-21 17:11:21 -04:00
Scott Aslan 8b27ed905d [NIFI-1879] Responsive dialogs and dialog UX refresh. This closes #523 2016-06-21 14:48:14 -04:00
Pierre Villard 6f3abdbcbd NIFI-2066 dynamic port allocation in SNMP processors unit tests
This closes #552
2016-06-21 14:46:21 -04:00
Pierre Villard ef192cc859 NIFI-1751 Added proxy authentication in InvokeHttp processor
This closes #398.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-06-21 14:09:02 -04:00
jpercivall c360b57a1e NIFI-1834 fix nifi-ssl-context-service-api verision 2016-06-21 13:44:45 -04:00
Andy LoPresto a6e96953f6 Removed duplicate dependency declaration from nifi-standard-processors/pom.xml.
This closes #551.
2016-06-21 19:25:01 +02:00
jpercivall cd846c8d62 NIFI-1834 adding fixes to PutTCP
This closes #543
2016-06-21 12:05:39 -04:00
Matt Brown a7c912ac18 PutTCP Processor created.
Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-06-21 12:05:13 -04:00
Mark Payne ce8a0de368 NIFI-1994: Fixed issue with Controller Service Fully Qualified Class Names and ensure that services are added to the process groups as appropriate when instantiating templates
NIFI-1882: Ensuring Controller Services are copied as part of a ProcessGroupDTO. This closes #517
2016-06-20 22:24:25 -04:00
Oleg Zhurakousky d64fe416be
NIFI-2032 put back bulletinRepo reporting logic
NIFI-2032 polishing (+2 squashed commits)
Squashed commits:
[4549443] NIFI-2032 fixed 'enableControllerServices' logic
added getRequiredControllerServices() operation to ControllerServiceNode
[3d95c8e] NIFI-2032 Removed legacy references to bulletin repository in StandardControllerServiceProvider. (+1 squashed commit)
Squashed commits:
[6402eb9] NIFI-2032 fixed 'enableControllerServices' logic
added getRequiredControllerServices() operation to ControllerServiceNode

This closes #541.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-06-20 18:56:41 -07:00
Yolanda M. Davis cb3aa8f5c9 NIFI-1850 - Initial Commit for JSON-to-JSON Schema Converter Editor (merge from 0.7.0 - refactor for masterless cluster). This closes #511 2016-06-20 20:30:25 -04:00
Ryan Persaud 274dc0902e NIFI-1909 Adding ability to process schemaless Avro records to ConvertAvrotoJson.
- Made suggested changes and removed unused imports found by checkstyle
 - This closes #459.
2016-06-20 13:48:03 -04:00
Bryan Bende 8593bd771f NIFI-1895 Adding a property to PutHBaseJSON to allow specifying how to store the values
This closes #542.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-06-20 13:19:20 -04:00
rtempleton 0d2a9dc7e5 NIFI-1895 PutHBaseJSON processor treats all values as Strings
The operator will now inspect the node value to determine type and convert as such.
Numeric integral - Long (assumes widest type)
Numeric not integral - Double (assumes widest type)
Logical - Boolean
everything else (including current Complex Type logic) - String

Values that represent the row key continue to be implictly treated as Strings by the processor

Removed depenency on HBase utility Bytes class from the PutHBaseJSON processor.
Convenience methods to encode to byte array are now wrapped by the appropriate HBaseClientService instance.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-06-20 13:19:13 -04:00
ricky f5060a6d63 NIFI-1568: Add Filter Capability to UnpackContent
Adds a "File Filter" property to the `UnpackContent` processor
to allow users to specify which files are eligible for extraction.
By default, all files will be extracted.

Signed-off-by: Matt Burgess <mattyb149@apache.org>

Refactor how Unpacker is initialized

Re-uses unpackers to avoid creating them each time a
flowfile is received. Moved the package formats into
an enum for clarity.

Signed-off-by: Matt Burgess <mattyb149@apache.org>

Fix packaging format enum warning

The `AUTO_DETECT_FORMAT` enum for PackagingFormat
is not valid for initilization. When this value
is set, then we use the mime.type to determine
which packaging format to use.

We never pass `AUTO_DETECT_FORMAT` to the
`initUnpacker` method.

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #248
2016-06-20 10:44:38 -04:00
Pierre Villard 86bba1b202 NIFI-1537 Added SNMP processors
This closes #257
2016-06-19 14:58:41 -04:00
ijokarumawak b1b7e35bed NIFI-1843 Default log level for LogAttribute.
- added a logger entry for LogAttribute to specify INFO as log level
- replaced some tab characters to whitespace (existing checkstyle error)

This closes #414.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-06-19 02:21:50 -04:00
Jeremy Dyer 406d8a1991 NIFI-1805
Expose BackPressureObjectThreshold and BackPressureDataSizeThreshold to
ConnectionStatus

This closes #377.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-06-18 23:05:32 -04:00
Pierre Villard e6250d18cf NIFI-1620 Allow empty Content-Type in InvokeHTTP processor
This closes #272.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-06-18 12:47:07 -04:00
Will Song 9546bef86e
NIFI-1958 Added "wks" as an acceptable parameter and added unit test (+6 squashed commits)
Squashed commits:
[16dd4ba] NIFI-1958 fixed logic on incoming time units. Removed feature tests to convert to weeks because it will not be implemented.
[1b22e58] NIFI-1958 added logic to getTimeDuration to handle weeks as a string value to parse but did not add TimeUnit yet. 3/5 feature test pass.
[7136544] NIFI-1958 Moved tests to correct module.
[7d95653] NIFI-1958 Added feature tests for negative values.
[ffc3941] NIFI-1958 Added second feature test.
[7d16bbe] NIFI-1958 Added new feature test and regression test for week conversion.

This closes #544.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-06-17 17:24:19 -07:00
Bryan Bende 679ad93f57 NIFI-1804 Adding ability for FileAuthorizer to automatically convert existing authorized-users.xml to new model
- Removing Resources class from file authorizer and updating ResourceType enum
- Updating ResourceFactory to be in sync with ResourceType enum and adding additional required permissions to the auto-conversion
- Adding root process group to the seeding of the initial admin
- Improvement so that users that are already part of a read-write policy, won't end up in a read policy for the same resource
- Removing rootGroupId from authorization context and auto-detecting it from the flow provided through nifi.properties
- This closes #507
2016-06-17 16:33:00 -04:00
Adam Lamar db9a79f79d NIFI-1578: Create PutSlack processor
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #256
2016-06-17 15:32:17 -04:00
Mark Payne c955ec1689 NIFI-2033: Allow Controller Services to be scoped at Controller level instead of just group level. This closes #540 2016-06-17 13:26:30 -04:00