Commit Graph

108 Commits

Author SHA1 Message Date
Mike Thomsen 0e61dbc9a0
NIFI-7373
Added new methods to DistributedMapCacheClient for bulk get and put. Updated HBase 1.1.2 clients.
Added HBase 2 support.
Added Redis support.
2020-07-23 07:38:59 -04:00
Dustin Rodrigues f0e8daf45d
NIFI-7317 - make .java files non-executable
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #4181.
2020-04-06 12:19:46 +02:00
Joe Witt f694e6464f NIFI-7187 adding missing version strings from accumulo bundle pom
- Removed Cat X JSON.org dep inclusion which seems to not be necessary
- Updated a ton of easier/safer looking deps
- Updated tika due to CVE

This closes #4086

Signed-off-by: Mark Payne <markap14@hotmail.com>
2020-03-20 10:07:56 -04:00
jstorck ab1658130d
NIFI-7024: Added Kerberos Password support to HBase_1_1_2_ClientService and HBase_2_ClientService
This closes #4103.
2020-03-02 11:43:13 -05:00
jstorck 614136ce51
NIFI-7018: Initial commit of processors extending AbstractHadoopProcessor supporting kerberos passwords
AbstractHadoopProcessor will always authenticate the principal with a KerberosUser implementation and a UGI will be acquired from the Subject associated with the KerberosUser implementation
AbstractHadoopProcessor's getUserGroupInformation method will now attempt to check the TGT and relogin if a KerberosUser impelmentation is available, otherwise it will return the UGI referenced in the HdfsResource instance
Updated AbstractHadoopProcessor's customValidate method to consider the provided password and updated validation failure explanations when a KerberosCredentialsService is specified together with a principal, password, or keytab
Added toString method override to AbstractKerberosUser
Updated Hive/HBase components to be compatible with the KerberosProperties.validatePrincipalWithKeytabOrPassword method
Fixed null ComponentLog in GetHDFSSequenceFileTest

Added package-protected accessor method (getAllowExplicitKeytabEnvironmentVariable) to AbstractHadoopProcessor for determining if the environment variable "NIFI_ALLOW_EXPLICIT_KEYTAB" has been set
AbstractHadoopProcessor will now only fail validation when the NIFI_ALLOW_EXPLICIT_KEYTAB environment variable is set to false if a keytab is provided to allow the user to specify a principal and password
Added AbstractHadoopProcessorSpec to verify validation of principal/keytab/password/kerberos credential service combinations

This closes #4095.
2020-02-28 10:10:19 -05:00
Joe Witt 3de77ebacc
NIFI-7021-RC3 prepare for next development iteration 2020-01-19 14:14:40 -05:00
Joe Witt 633408bce7
NIFI-7021-RC3 prepare release nifi-1.11.0-RC3 2020-01-19 14:14:38 -05:00
Joe Witt 23c8234586
NIFI-7031 updating copyright year on NOTICES 2020-01-15 16:10:31 -05:00
Joe Witt f8c3d877cf
NIFI-6733 updating to next release version for master branch 2019-11-04 13:31:39 -05:00
Joe Witt 418179f5b2
NIFI-6733-RC3 prepare for next development iteration 2019-10-28 15:13:13 -07:00
Joe Witt b217ae20ad
NIFI-6733-RC3 prepare release nifi-1.10.0-RC3 2019-10-28 15:12:57 -07:00
Mike 02d3b7e92b NIFI-6656 Added a default visibility expression configuration item to… (#3723)
* NIFI-6656 Added a default visibility expression configuration item to the HBase map cache client services.

* NIFI-6656 Fixed validation bug.

* NIFI-6656 Added stylecheck fix.

* NIFI-6656 Deleted shared interface and moved property descriptor into both services.

* NIFI-6656 Removed dependency from hbase api.
2019-10-03 12:53:53 -04:00
Jeff Storck 33187f8058 NIFI-5176 NiFI builds on Java 11
Added "jigsaw" profile to multiple modules, which auto-activates when building with Java 11 and adds several dependencies: jaxb, activation, and annotation-api.
Updated SslContextFactory to return tuple of socket factory with trust manager for issue with okhttp api changes in java 9+
Updated TestGet/PostHTTPGroovy to use default SSL context to get default cipher suites
Updated StandardTemplateDAOSpec.groovy, was using a 37-character UUID, shortened to 36 characters.
Multiple tests updated to specifically use TLSv1.2, since two-way TLSv1.3 for some component tests fail during the Java 11 build.  Needs more investigation.
Replaced GString with String concatenation for map key's value in ScriptedReportingTaskGroovyTest to avoid runtime error of casting GString to String when retrieved from a map that is storing the GString
Removed nifi-toolkit-api plugin configuration of maven-compiler-plugin, it is inherited
Resolved deprecation errors during Groovy compile for bouncycastle, Extension should be used in place of X509Extension
Fixed JNAOverridingJUnitRunner's creation of the classpath for the custom classloader created to be able to mock jna classes
Removed import of IOUtils (from the wrong package) from InferenceSchemaStrategy
Updated maven-compiler-plugin version to 3.8.1
TLS (default in Java 11 is TLSv1.3) working for Site-to-Site client tests after upgrading JDK installation to JDK 11.0.3, httpclient5 5.0-beta4, and httpasyncclient 4.1.4
HttpNotificationService updated to find and use the first found X509TrustManager rather than casting directly from the array of TrustManagers returned from SslContextFactory
  Removed unnecessary throws declaration from getSslSocketFactory method
Replaced DnsContextFactory.class.getName() with a string to avoid having to export/open the jdk.naming.dns module
Updated TestGetIgniteCache and TestPutIgniteCache.java to skip tests on Java 11+ (via Assume), and noted the check should be removed once a version of Ignite is released that supports Java 11
Updated SpringContextProcessor to create proper ClassLoader and uses URLClassloader.getResource() instead of URLClassloader.findResource() in isConfigResolvable.
  Due to the application classloader no longer being a URLClassLoader in Java 9+, the URLClassLoader created in isConfigResolvable no longer has explicit access to the parent's resources as URLs.
  URLClassLoader.getResource() searches the parent classloaders, and must be used instead of URLClassLoader.findResource() which only searches the URLs in the URLClassLoader and does not search the parent classloaders.

NIFI-5176 Moved exclusion of jdk.tools to the jigsaw profile in the POMs of nifi-hbase_1_1_2-client-service and nifi-hbase_2-client-service

NIFI-5176 Updated site-to-site-client's POM to use properties to establish the correct httpclient dependency for when building with Java 11

This closes #3404.
2019-08-13 18:41:39 -04:00
Bryan Bende f2db1539a8
NIFI-6432 Updating get, containsKey, and remove in HBase_1_1_2_ClientMapCacheService and HBase_2_ClientMapCacheService so that they use the column family and qualifier specified in the service
This closes #3581.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2019-07-16 13:43:26 +09:00
Shawn Weeks e913f5706f
NIFI-6243 Add Support for AtomicDistributedCache to the HBase 1.x and 2.x Map Cache Services
This closes #3462.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2019-06-12 15:31:39 +09:00
Andy LoPresto e6c843f465
NIFI-6323 Changed URLs for repositories, project description, and mailing lists to use HTTPS.
NIFI-6323 Changed URLs for splunk.artifactoryonline.com to use HTTPS (certificate validity warning in browsers, but command-line connection using openssl s_client is successful).
NIFI-6323 Changed URLs for XMLNS schema locations to use HTTPS (the XMLNS and schema identifier remain http:// because they are not designed to be resolvable).
NIFI-6323 Fixed Maven XML schema descriptor URLs.

This closes #3497
2019-05-29 14:36:40 -04:00
Lars Francke 49e74b29db NIFI-6198 Infinite recursion in HBase*ClientService
This closes #3419.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2019-04-09 09:29:32 +09:00
Bryan Bende 25cb29e109
NIFI-6177 Refactor HBaseListLookupService tests to remove use of TestRecordLookupProcessor
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3400.
2019-04-06 14:31:50 +02:00
Sandish Kumar 82e2c97782
NIFI-5956 Option BlockCache HBaseScanProcessor
This closes #3295.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-03-14 12:22:41 -04:00
Mike Thomsen b0a93b473b
NIFI-5980 Added HBase_1_1_2_ListLookupService.
NIFI-5980 Added HBase_2_ListLookupService.

This closes #3278.

Signed-off-by: Bryan Bende <bbende@apache.org>
2019-02-25 16:42:11 -05:00
joewitt 25cc7b4a1e
NIFI-6029 merging nifi 1.9.0 release into master 2019-02-19 22:55:49 -05:00
Denes Arvay 76e92c8682 NIFI-6052 Update NOTICE files to reflect 2019
This closes #3319

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2019-02-19 18:32:15 -05:00
joewitt 0e204f3576
NIFI-6029-RC2 prepare for next development iteration 2019-02-16 21:50:35 -05:00
joewitt 45bb53d2aa
NIFI-6029-RC2 prepare release nifi-1.9.0-RC2 2019-02-16 21:50:15 -05:00
Mark Payne 1ac5b93144
NIFI-5859: Added XML-based documentation writer that can be used to document a component.
- Found several instances of nifi-framework-api's ProviderException being thrown from processors. Changed those to IllegalStateException, as ProviderException is not an appropriate Exception in those cases, and extensions should not depend on nifi-framework-api.
- Performed some cleanup, moving Property Descriptors from Controller Service API's/specs into the implementations. Adding to the Service API results in bringing in nifi-utils to the nifi-standard-services-api-nar, which is a bad practice. The 'main' service api nar should not have a dependency on a util class.

NIFI-5859: Added javadocs. Fixed pom.xml that was left pointing to snapshot version of nar maven plugin

NIFI-5859: Addressing review feedback: adding component type, multiple additional details into separate file(s)

This closes #3192.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-12-10 11:16:01 -05:00
Bryan Bende 60064a9f68 NIFI-5884 Bumping hbase-client version from 1.1.2 to 1.1.13
This closes #3211.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2018-12-10 11:00:08 +09:00
Bryan Bende f1e03b5ed5
NIFI-5877 Protecting against null values when obtaining master address
Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #3206.
2018-12-06 22:03:23 +01:00
Jeff Storck c0182294ed NIFI-5720-RC3 prepare for next development iteration 2018-10-22 22:16:43 -04:00
Jeff Storck 98aabf2c50 NIFI-5720-RC3 prepare release nifi-1.8.0-RC3 2018-10-22 22:16:23 -04:00
thenatog 5eb5e96b16
NIFI-5665 - Changed netty versions to more closely match the original netty dependency version.
NIFI-5665 - Fixed version for nifi-spark-bundle.
NIFI-5665 - Fixing copy and paste error.

This closes #3067
2018-10-12 15:56:52 -04:00
joewitt 8e233ca2ef
NIFI-4806 updated tika and a ton of other deps as found by dependency versions plugin
This closes #3028
2018-10-04 09:32:04 -04:00
Jan Hentschel c2657ee043 NIFI-5423 Removed duplicated dependencies in sub-modules
This closes #2887

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-08-15 11:01:51 -04:00
Mark Bean b279624398 NIFI-5368 controller services validated prior to enabling; referenced controller services must be enabled for referencing component to be valid (mock framework)
This closes #2873.

Signed-off-by: Mark Payne <markap14@hotmail.com>
2018-07-10 11:23:40 -04:00
Andy LoPresto d42a1e8bf4
NIFI-5323-RC1 prepare for next development iteration 2018-06-19 20:02:21 -07:00
Andy LoPresto 99bcd1f88d
NIFI-5323-RC1 prepare release nifi-1.7.0-RC1 2018-06-19 20:02:01 -07:00
Pierre Villard 64cd34016b NIFI-5233 - Add EL support with Variable Registry scope for HBase client service
This closes #2738

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-05-25 10:37:45 -04:00
Koji Kawamura f7035f0497 NIFI-4637: Fix ExtensionManager warning on VisibilityLabelService
This closes #2729

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-05-23 07:34:49 -04:00
Mike Thomsen 0b851910f3 NIFI-4637 Added support for visibility labels to the HBase processors.
NIFI-4637 Removed integration test and updated the hbase client from 1.1.2 to 1.1.13 which is the final version for 1.1.X

NIFI-4637 Fixed EL support issue w/ tests.

NIFI-4637 Added more documentation to DeleteHBaseCells.

NIFI-4637 changed PutHBaseCell/JSON to use dynamic properties instead of a 'default visibility string.'

NIFI-4637 Added changes requested in a code review.

NIFI-4637 Moved pickVisibilityString to a utility class to make testing easier.

NIFI-4637 Added additionalDetails.html for PutHBaseRecord.

NIFI-4637 Added additional documentation and testing.

NIFI-4637 Added documentation for DeleteHBaseCells.

NIFI-4637 Added pickVisibilityLabel support to PutHBaseRecord and updated documentation to reflect that.

NIFI-4637 Reverted version bump to hbase client.

This closes #2518.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2018-05-11 00:55:40 +00:00
zenfenan 1dbfcb9445 NIFI-5062: Removed hbase-client dependecy from hbase bundle
This closes #2636

Signed-off-by: Mike Thomsen <mikerthomsen@gmail.com>
2018-04-20 08:50:25 -04:00
joewitt 59f625d3c2 Merge branch 'NIFI-4995-RC3' as part of NiFi 1.6.0 release process 2018-04-06 16:48:08 -07:00
Pierre Villard 4c787799ff NIFI-4149 - Indicate if EL is evaluated against FFs or not
- take into account input requirement for documentation rendering
- Renamed variable registry scope and added comments
- Doc + change in mock framework to check scope + update of components + UI
2018-04-06 12:40:54 -04:00
joewitt d511fe3e4b NIFI-4995-RC3 prepare for next development iteration 2018-04-03 08:28:34 -07:00
joewitt f8466cb16d NIFI-4995-RC3 prepare release nifi-1.6.0-RC3 2018-04-03 08:28:15 -07:00
joewitt 7b5bf265a6 NIFI-4995 updating copyright year on all notices 2018-03-26 21:54:10 -04:00
Mark Payne 0b0aebe148
NIFI-4917: Externalize Keytab and Principal configuration from Processors to a Controller Service. This gives us the ability to allow users to interact with those Keytabs/Principals to which they've been given access without allowing them access to all Keytabs and Principals
- Addressed review feedback; rebased against master

This closes #2552.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-03-21 14:23:41 -04:00
Ed c2616e6fe7
NIFI-4833 Add ScanHBase Processor
- New processor for scanning HBase records based on verious params like range of rowkeys, range of timestamps. Supports result limit and reverse scan.
- Adds Atlas Support for ScanHBase processor
- Fixed not recent version of FF
- Formatting and Style changes
- Single line to multiline if-then statements
- Removed HTML formatting that is not used for doc generation
- Fixed issue with limitRows
- Fixed issue with filter expression
- Refactored "processRows"
- Fixed possible NPE for bulkSize var
- Changed provenance to "receive" to indicate new data from external source.
- Updated min/max timestamp custom validation
- JSON array support
- Removed in-memory caching for records. Now records are being written directly to FF
- Removed unfinished flowfile from session, transfered original to "failure". Test cases update as well

This closes #2478.

Signed-off-by: Bryan Bende <bbende@apache.org>
2018-03-13 10:07:07 -04:00
joewitt c71409fb5d
NIFI-4936 trying to quiet down the mvn output a bit so we dont exceed the travis-ci 4MB max
NIFI-4936 updated dependency handling pushing down delcarations where they belong
This closes #2512
2018-03-09 16:34:53 -05:00
Mike Thomsen 143d7e6829 NIFI-3538 Added DeleteHBaseRow
This closes #2294.

Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
2018-02-14 10:04:25 +09:00
Koji Kawamura b4a9f52a4e
NIFI-4866: Fixed HBase performance issue
HBase_1_1_2_ClientService performance dropped due to accessing HBase
admin upon every FlowFile transfer. Fixed that by getting master address
only once when the service is enabled and a connection is established.

This closes #2464
2018-02-11 16:14:52 -05:00
joewitt 41ce788812 NIFI-4751 changed to next minor release version snapshot 2018-01-12 15:15:32 -05:00