Commit Graph

2340 Commits

Author SHA1 Message Date
Joe Skora 7e7d79fc6a NIFI-615 - Create a processor to extract WAV file characteristics.
* Create new ExtractMediaMetadata processor using Apache Tika Detector and Parser.
* Refactored nifi-image-bundle, nifi-image-nar, and nifi-image-processors to nifi-media-bundle, nifi-media-nar, and nifi-media-processors to reflect broader media related purpose.
* Preserved existing ExtractImageMetadata and ResizeImage processors as well as existing ImageViewerController components to prevent impact on existing uses.
* Resolved collision between ExtractImage and ExtractMedia processors due to common dependency on Noakes' Metadata Extractor project.
  - Updated bundle's Tika dependency from 1.7 to 1.8 and Drew Noakes' Metadata Extractor from 2.7.2 to 2.8.0.
  - Adjusted ExtractImageMetadata tests for enhanced attribute names in new Noakes' Metadata Extractor version.
* Fix assembly POM to remove duplicate reference to site-to-site nar and change nifi-image-nar reference to nifi-media-nar.
* Note the potential attribute changes on upgrade due to underlying libraries.

This closes #556.
2016-06-22 15:06:52 -04:00
Mark Payne 0e085bdddd NIFI-2065: When a provenance query matches the max number of results requested, stop querying lucene for improved performance
This closes #558
2016-06-22 13:27:31 -04:00
Mark Payne 36ab8474dd NIFI-1900: Verify that connection's destination is not running when trying to change destination
This closes #550
2016-06-22 12:39:53 -04:00
Jeff Storck f47be77b6a NIFI-1952 Create REST endpoints for user/group/policy management
created REST Resources for users, groups, and access policies
added Authorizables for users, groups, and access policies
added methods to DtoFactory and EntityFactory to create objects for users, groups, and access policies
extracted anonymous AuthorizableLookup impl in StandardNiFiServiceFacade.java to a protected class to make the lookup call mockable in tests
added methods to manage users/groups/access policies to StandardNiFiServiceFacade
added StandardNiFiServiceFacadeSpec to unit-test management of users/groups/access policies
added implementations for UserDAO, GroupDAO, AccessPolicyDAO.
added spring config for user/group/policy resources and daos
Updated user/group/policy creation via REST resources, no longer requires the use of the revision manager
updated StandardNiFiServiceFacadeSpec based on user/group/policy creation changes
condensed user/group/policy DAOs to a single DAO (StandardPolicyBasedAuthorizerDAO)
fixed spring config of user/group/policy REST resources
Updated to return ComponentEntity objects instead of just their IDs
mid-progress on updating tests
updated code and tests to return component entities from REST endpoints for users, groups, policies
This closes #526
2016-06-22 10:12:41 -04:00
Scott Aslan 32facaedb4 [NIFI-1879] Update dialog content scrollable styles. This closes #560 2016-06-22 08:11:12 -04:00
Matt Gilman 27ff5f9a87 NIFI-2075:
- Fixing issue rendering event rows while visible in the shell.
- Fixing issue go to/from the event table and lineage graph.
- Fixing visibility of the event table header, search, and filter controls.
- This closes #559
2016-06-22 07:47:03 -04:00
Oleg Zhurakousky 1eeef0e1c0 NIFI-2067 ignored intermittently failing MemoryTest left comment with the explanation as to why it was not removed
This closes #555

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-06-21 22:18:13 -04:00
Scott Aslan 8ed66e5df1 [NIFI-1879] Update containment for status history dialog. This closes #557 2016-06-21 21:49:36 -04:00
Bryan Rosander 61a2054b38 NIFI-2046 - Adding reference to @TriggerSerially in Processor javadoc
This closes #549

Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-06-21 17:31:09 -04:00
Matt Gilman 9e54a3d260 NIFI-1946: - Updating palette styling.
- Providing selection context.
- Updating add connection icon.
NIFI-2058: - Updating defaults for connection backpressure.
NIFI-2014: - Updating birds eye colors.
NIFI-2016: - Updating default label color.
NIFI-2013: - Updating how we color processors.
NIFI-2043: - Using correct group id when creating controller services inline.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-06-21 17:11:21 -04:00
Scott Aslan 8b27ed905d [NIFI-1879] Responsive dialogs and dialog UX refresh. This closes #523 2016-06-21 14:48:14 -04:00
Pierre Villard 6f3abdbcbd NIFI-2066 dynamic port allocation in SNMP processors unit tests
This closes #552
2016-06-21 14:46:21 -04:00
Pierre Villard ef192cc859 NIFI-1751 Added proxy authentication in InvokeHttp processor
This closes #398.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-06-21 14:09:02 -04:00
jpercivall c360b57a1e NIFI-1834 fix nifi-ssl-context-service-api verision 2016-06-21 13:44:45 -04:00
Andy LoPresto a6e96953f6 Removed duplicate dependency declaration from nifi-standard-processors/pom.xml.
This closes #551.
2016-06-21 19:25:01 +02:00
jpercivall cd846c8d62 NIFI-1834 adding fixes to PutTCP
This closes #543
2016-06-21 12:05:39 -04:00
Matt Brown a7c912ac18 PutTCP Processor created.
Signed-off-by: jpercivall <joepercivall@yahoo.com>
2016-06-21 12:05:13 -04:00
Mark Payne ce8a0de368 NIFI-1994: Fixed issue with Controller Service Fully Qualified Class Names and ensure that services are added to the process groups as appropriate when instantiating templates
NIFI-1882: Ensuring Controller Services are copied as part of a ProcessGroupDTO. This closes #517
2016-06-20 22:24:25 -04:00
Oleg Zhurakousky d64fe416be
NIFI-2032 put back bulletinRepo reporting logic
NIFI-2032 polishing (+2 squashed commits)
Squashed commits:
[4549443] NIFI-2032 fixed 'enableControllerServices' logic
added getRequiredControllerServices() operation to ControllerServiceNode
[3d95c8e] NIFI-2032 Removed legacy references to bulletin repository in StandardControllerServiceProvider. (+1 squashed commit)
Squashed commits:
[6402eb9] NIFI-2032 fixed 'enableControllerServices' logic
added getRequiredControllerServices() operation to ControllerServiceNode

This closes #541.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-06-20 18:56:41 -07:00
Yolanda M. Davis cb3aa8f5c9 NIFI-1850 - Initial Commit for JSON-to-JSON Schema Converter Editor (merge from 0.7.0 - refactor for masterless cluster). This closes #511 2016-06-20 20:30:25 -04:00
Ryan Persaud 274dc0902e NIFI-1909 Adding ability to process schemaless Avro records to ConvertAvrotoJson.
- Made suggested changes and removed unused imports found by checkstyle
 - This closes #459.
2016-06-20 13:48:03 -04:00
Bryan Bende 8593bd771f NIFI-1895 Adding a property to PutHBaseJSON to allow specifying how to store the values
This closes #542.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-06-20 13:19:20 -04:00
rtempleton 0d2a9dc7e5 NIFI-1895 PutHBaseJSON processor treats all values as Strings
The operator will now inspect the node value to determine type and convert as such.
Numeric integral - Long (assumes widest type)
Numeric not integral - Double (assumes widest type)
Logical - Boolean
everything else (including current Complex Type logic) - String

Values that represent the row key continue to be implictly treated as Strings by the processor

Removed depenency on HBase utility Bytes class from the PutHBaseJSON processor.
Convenience methods to encode to byte array are now wrapped by the appropriate HBaseClientService instance.

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-06-20 13:19:13 -04:00
ricky f5060a6d63 NIFI-1568: Add Filter Capability to UnpackContent
Adds a "File Filter" property to the `UnpackContent` processor
to allow users to specify which files are eligible for extraction.
By default, all files will be extracted.

Signed-off-by: Matt Burgess <mattyb149@apache.org>

Refactor how Unpacker is initialized

Re-uses unpackers to avoid creating them each time a
flowfile is received. Moved the package formats into
an enum for clarity.

Signed-off-by: Matt Burgess <mattyb149@apache.org>

Fix packaging format enum warning

The `AUTO_DETECT_FORMAT` enum for PackagingFormat
is not valid for initilization. When this value
is set, then we use the mime.type to determine
which packaging format to use.

We never pass `AUTO_DETECT_FORMAT` to the
`initUnpacker` method.

Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #248
2016-06-20 10:44:38 -04:00
Pierre Villard 86bba1b202 NIFI-1537 Added SNMP processors
This closes #257
2016-06-19 14:58:41 -04:00
ijokarumawak b1b7e35bed NIFI-1843 Default log level for LogAttribute.
- added a logger entry for LogAttribute to specify INFO as log level
- replaced some tab characters to whitespace (existing checkstyle error)

This closes #414.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-06-19 02:21:50 -04:00
Jeremy Dyer 406d8a1991 NIFI-1805
Expose BackPressureObjectThreshold and BackPressureDataSizeThreshold to
ConnectionStatus

This closes #377.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-06-18 23:05:32 -04:00
Pierre Villard e6250d18cf NIFI-1620 Allow empty Content-Type in InvokeHTTP processor
This closes #272.

Signed-off-by: Aldrin Piri <aldrin@apache.org>
2016-06-18 12:47:07 -04:00
Will Song 9546bef86e
NIFI-1958 Added "wks" as an acceptable parameter and added unit test (+6 squashed commits)
Squashed commits:
[16dd4ba] NIFI-1958 fixed logic on incoming time units. Removed feature tests to convert to weeks because it will not be implemented.
[1b22e58] NIFI-1958 added logic to getTimeDuration to handle weeks as a string value to parse but did not add TimeUnit yet. 3/5 feature test pass.
[7136544] NIFI-1958 Moved tests to correct module.
[7d95653] NIFI-1958 Added feature tests for negative values.
[ffc3941] NIFI-1958 Added second feature test.
[7d16bbe] NIFI-1958 Added new feature test and regression test for week conversion.

This closes #544.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-06-17 17:24:19 -07:00
Bryan Bende 679ad93f57 NIFI-1804 Adding ability for FileAuthorizer to automatically convert existing authorized-users.xml to new model
- Removing Resources class from file authorizer and updating ResourceType enum
- Updating ResourceFactory to be in sync with ResourceType enum and adding additional required permissions to the auto-conversion
- Adding root process group to the seeding of the initial admin
- Improvement so that users that are already part of a read-write policy, won't end up in a read policy for the same resource
- Removing rootGroupId from authorization context and auto-detecting it from the flow provided through nifi.properties
- This closes #507
2016-06-17 16:33:00 -04:00
Adam Lamar db9a79f79d NIFI-1578: Create PutSlack processor
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #256
2016-06-17 15:32:17 -04:00
Mark Payne c955ec1689 NIFI-2033: Allow Controller Services to be scoped at Controller level instead of just group level. This closes #540 2016-06-17 13:26:30 -04:00
Matt Gilman 2c69c25323 NIFI-2007:
- Restoring bulletin functionality.
- Ensuring appropriate merging of bulletins in clustered responses.

- Updated the phrasing for cluster-related bulletins

This closes #531.
2016-06-17 13:20:20 -04:00
Matt Gilman 0b437e09a7 NIFI-2007:
- Restoring bulletin functionality.
- Ensuring appropriate merging of bulletins in clustered responses.
2016-06-17 13:20:20 -04:00
Joshi 548561ed4c NIFI-1955: Deprecate IntegerHolder, LongHolder, BooleanHolder
This closes #537
2016-06-17 12:32:53 -04:00
Joshi 541bbc0932 NIFI-1549: Fix JMSConsumer exception
This closes #539
2016-06-17 12:16:36 -04:00
João Henrique Ferreira de Freitas e27c2556db NIFI-1594: Add support to expression language to define index operation
The index operation should be one of "index", "update", " upsert ".

This lets the operation could be defined by incoming flowfile.

Signed-off-by: João Henrique Ferreira de Freitas <joaohf@gmail.com>
Signed-off-by: Matt Burgess <mattyb149@apache.org>

This closes #255
2016-06-17 11:22:47 -04:00
Matt Gilman fbd299e885 NIFI-1901: - Building component based access control tests for Connections, Funnels, Labels, Input Ports, Output Ports, Processors, and Process Groups. - Restoring Access Token Endpoint tests.
This closes #518

Signed-off-by: Bryan Bende <bbende@apache.org>
2016-06-17 10:16:09 -04:00
James Wing c2b26eb311 Merging NIFI-1941 Child Group Contents in Exported Templates
Signed-off-by: James Wing <jvwing@gmail.com>
This closes #532
2016-06-16 22:05:59 -07:00
Matt Gilman edce66d34f NIFI-1941:
- Ensuring descendent components are scheduled/unscheduled according to their respective ProcessGroups.
2016-06-16 20:17:16 -04:00
Matt Gilman 3156220a57 NIFI-1941:
- Ensuring child group contents are included in exported templates.
2016-06-16 20:00:39 -04:00
Oleg Zhurakousky 8e4a4532df
NIFI-1690 Changed MonitorMemory to use allowable values for pool names
- removed dead code from MonitorMemory
- added MonitorMemoryTest
- minor refactoring in MonitorMemory
- initial fix for NIFI-1731 (WARN/INFO logging) that was required by MonitorMemoryTest

NIFI-1690 polishing

NIFI-1690 address PR comments, removed default value for MEMORY_POOL_PROPERTY

NIFI-1690 addressed latest PR comments

NIFI-1690 fixed breaking changes

Fixed checkstyle issue in StandardProcessScheduler. (+2 squashed commits)
Squashed commits:
[03829c4] Fixed checkstyle issues.
[cb20fe6]

This closes #328.

Signed-off-by: Andy LoPresto <alopresto@apache.org>
2016-06-16 15:17:13 -07:00
JJ 2ce785766d NIFI-1037 Created a processor for HDFS' inotify events. This processor gets notifications for a select path.
This closes #493.
2016-06-17 00:02:28 +02:00
Joe Skora 4723f8e24c NIFI-1829 - Create new DebugFlow processor.
Signed-off-by: Mike Moser <mosermw@apache.org>
This closes #458
2016-06-16 17:23:19 -04:00
Matt Gilman 6e74c10f49 NIFI-2041:
- Fixing issue when attempting to refresh Controller Services when scoped by Process Group.
- This closes #535
2016-06-16 16:51:35 -04:00
Matt Gilman b3de8f76fe Addressing contrib-check issues. 2016-06-16 09:55:10 -04:00
Mark Payne b753a82d7e NIFI-2000: Ensure that if we override setters in ApplicationResource that we call the super class's setter as well
NIFI-2000: Updated to avoid multiple instance variables and instead just provide a getter in ApplicationResource

This closes #522
2016-06-15 13:00:37 -04:00
Pierre Villard 849c43b162 Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/nifi 2016-06-15 18:55:18 +02:00
Mark Payne 7facf95cc5 NIFI-1997: Use the 'autoResumeState' property defined in nifi.properties on each node instead of inheriting the property from the Cluster Coordinator
This closes #520
2016-06-15 12:55:34 -04:00
Oleg Zhurakousky 83d5cb365d NIFI-2009 removed 'annotationData' and 'validationContextFactory' instance variables from StandardProcessNode
NIFI-2009 fixed import ordering

This closes #527.
2016-06-15 18:54:25 +02:00