Apache
/
nifi
mirror of https://github.com/apache/nifi.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Initial code contribution

This commit is contained in:
joewitt 2014-12-08 15:22:14 -05:00
commit 4d998c12c9
2352 changed files with 350029 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
.gitignore vendored Normal file
View File

@ -0,0 +1,9 @@
target
.project
.settings
nbactions.xml
# Intellij
.idea/
*.iml
*.iws

15
DISCLAIMER Normal file
View File

@ -0,0 +1,15 @@
Apache NiFi is an effort undergoing incubation at the Apache Software
Foundation (ASF), sponsored by the Apache Incubator PMC.
Incubation is required of all newly accepted projects until a further review
indicates that the infrastructure, communications, and decision making process
have stabilized in a manner consistent with other successful ASF projects.
While incubation status is not necessarily a reflection of the completeness
or stability of the code, it does indicate that the project has yet to be
fully endorsed by the ASF.
For more information about the incubation status of the Apache NiFi project
you can go to the following page:
http://nifi.incubator.apache.org/

202
LICENSE Normal file
View File

@ -0,0 +1,202 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

7
NOTICE Normal file
View File

@ -0,0 +1,7 @@
Apache NiFi
Copyright 2014 The Apache Software Foundation
This product includes software developed at
The Apache Software Foundation (http://www.apache.org/).
Please see LICENSE for additional copyright and licensing information.

65
README.md Normal file
View File

@ -0,0 +1,65 @@
# Apache NiFi
Apache NiFi is a dataflow system based on the concepts of flow-based programming. It is currently apart of the Apache Incubator.
## Table of Contents
- [Features](#features)
- [Getting Started](#getting-started)
- [Getting Help](#getting-help)
- [Requirements](#requirements)
- [License](#license)
- [Disclaimer](#disclaimer)
## Features
Apache NiFi supports powerful and scalable directed graphs of data routing, transformation, and system mediation logic. Some of the high-level capabilities and objectives of Apache NiFi include:
- Web-based user interface for seamless experience between design, control, feedback, and monitoring of data flows
- Highly configurable along several dimensions of quality of service such as loss tolerant versus guaranteed delivery, low latency versus high throughput, and priority based queuing
- Fine-grained data provenance for all data received, forked, joined, cloned, modified, sent, and ultimately dropped as data reaches its configured end-state
- Component-based extension model along well defined interfaces enabling rapid development and effective testing
## Getting Started
Execute <nifi install dir>/bin/nifi.sh
## Getting Help
If you have questions, you can reach out to our mailing list: dev@nifi.incubator.apache.org
([archive](http://mail-archives.apache.org/mod_mbox/incubator-nifi-dev)).
We're also often available in IRC: #nifi on
[irc.freenode.net](http://webchat.freenode.net/?channels=#nifi).
## Requirements
* JDK 1.7 or higher
## License
Except as otherwise noted this software is licensed under the
[Apache License, Version 2.0](http://www.apache.org/licenses/LICENSE-2.0.html)
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
## Disclaimer
Apache NiFi is an effort undergoing incubation at the Apache Software
Foundation (ASF), sponsored by the Apache Incubator PMC.
Incubation is required of all newly accepted projects until a further review
indicates that the infrastructure, communications, and decision making process
have stabilized in a manner consistent with other successful ASF projects.
While incubation status is not necessarily a reflection of the completeness
or stability of the code, it does indicate that the project has yet to be
fully endorsed by the ASF.

485
assemblies/nifi/pom.xml Normal file
View File

@ -0,0 +1,485 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-parent</artifactId>
<version>0.0.1-SNAPSHOT</version>
</parent>
<artifactId>nifi</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>pom</packaging>
<name>NiFi Release</name>
<description>This is the assembly for the Main Standard Release of NiFi</description>
<build>
<plugins>
<plugin>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<id>unpack-shared-resources</id>
<goals>
<goal>unpack-dependencies</goal>
</goals>
<phase>generate-resources</phase>
<configuration>
<outputDirectory>${project.build.directory}/generated-resources</outputDirectory>
<includeArtifactIds>nifi-resources</includeArtifactIds>
<includeGroupIds>org.apache.nifi</includeGroupIds>
<excludeTransitive>false</excludeTransitive>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<configuration>
<descriptors>
<descriptor>src/main/assembly/dependencies.xml</descriptor>
</descriptors>
</configuration>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.3.2</version>
<executions>
<execution>
<goals>
<goal>exec</goal>
</goals>
</execution>
</executions>
<configuration>
<executable>${project.build.directory}/${project.artifactId}-${project.version}-${nifi.assembly.id}.dir/${nifi.executable}</executable>
</configuration>
</plugin>
</plugins>
<resources>
<resource>
<directory>${basedir}/src/main/resources</directory>
</resource>
<resource>
<directory>${project.build.directory}/generated-resources</directory>
<filtering>true</filtering>
</resource>
</resources>
</build>
<dependencies>
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>jcl-over-slf4j</artifactId>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>jul-to-slf4j</artifactId>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>log4j-over-slf4j</artifactId>
<version>${org.slf4j.version}</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-api</artifactId>
<version>0.0.1-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-runtime</artifactId>
<version>${framework.version}</version>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-resources</artifactId>
<version>${framework.version}</version>
<classifier>resources</classifier>
<scope>provided</scope> <!-- Provided - we don't want the zip in the libs -->
<type>zip</type>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-framework-nar</artifactId>
<version>${framework.version}</version>
<type>nar</type>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>volatile-provenance-repository-nar</artifactId>
<version>0.0.1-SNAPSHOT</version>
<type>nar</type>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>persistent-provenance-repository-nar</artifactId>
<version>0.0.1-SNAPSHOT</version>
<type>nar</type>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>standard-services-api-nar</artifactId>
<version>0.0.1-SNAPSHOT</version>
<type>nar</type>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>ssl-context-service-nar</artifactId>
<version>0.0.1-SNAPSHOT</version>
<type>nar</type>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>distributed-cache-services-nar</artifactId>
<version>0.0.1-SNAPSHOT</version>
<type>nar</type>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-standard-nar</artifactId>
<version>0.0.1-SNAPSHOT</version>
<type>nar</type>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-jetty-bundle</artifactId>
<version>0.0.1-SNAPSHOT</version>
<type>nar</type>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>update-attribute-nar</artifactId>
<version>0.0.1-SNAPSHOT</version>
<type>nar</type>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>monitor-threshold-nar</artifactId>
<version>0.0.1-SNAPSHOT</version>
<type>nar</type>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>hadoop-libraries-nar</artifactId>
<version>0.0.1-SNAPSHOT</version>
<type>nar</type>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>hadoop-nar</artifactId>
<version>0.0.1-SNAPSHOT</version>
<type>nar</type>
</dependency>
<!-- <dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>execute-script-nar</artifactId>
<version>0.0.1-SNAPSHOT</version>
<type>nar</type>
</dependency>-->
</dependencies>
<properties>
<framework.version>0.0.1-SNAPSHOT</framework.version>
<!--Wrapper Properties-->
<nifi.wrapper.jvm.heap.initial.mb>256</nifi.wrapper.jvm.heap.initial.mb>
<nifi.wrapper.jvm.heap.max.mb>512</nifi.wrapper.jvm.heap.max.mb>
<nifi.initial.permgen.size.mb>128</nifi.initial.permgen.size.mb>
<nifi.max.permgen.size.mb>128</nifi.max.permgen.size.mb>
<nifi.wrapper.logfile.maxsize>10m</nifi.wrapper.logfile.maxsize>
<nifi.wrapper.logfile.maxfiles>10</nifi.wrapper.logfile.maxfiles>
<!-- nifi.properties: core properties -->
<nifi.flowcontroller.autoResumeState>true</nifi.flowcontroller.autoResumeState>
<nifi.flowcontroller.graceful.shutdown.period>10 sec</nifi.flowcontroller.graceful.shutdown.period>
<nifi.flowservice.writedelay.interval>500 ms</nifi.flowservice.writedelay.interval>
<nifi.administrative.yield.duration>30 sec</nifi.administrative.yield.duration>
<nifi.flow.configuration.file>./conf/flow.xml.gz</nifi.flow.configuration.file>
<nifi.flow.configuration.archive.dir>./conf/archive/</nifi.flow.configuration.archive.dir>
<nifi.reporting.task.configuration.file>./conf/reporting-tasks.xml</nifi.reporting.task.configuration.file>
<nifi.controller.service.configuration.file>./conf/controller-services.xml</nifi.controller.service.configuration.file>
<nifi.authority.provider.configuration.file>./conf/authority-providers.xml</nifi.authority.provider.configuration.file>
<nifi.templates.directory>./conf/templates</nifi.templates.directory>
<nifi.database.directory>./database_repository</nifi.database.directory>
<nifi.flowfile.repository.implementation>org.apache.nifi.controller.repository.WriteAheadFlowFileRepository</nifi.flowfile.repository.implementation>
<nifi.flowfile.repository.directory>./flowfile_repository</nifi.flowfile.repository.directory>
<nifi.flowfile.repository.partitions>256</nifi.flowfile.repository.partitions>
<nifi.flowfile.repository.checkpoint.interval>2 mins</nifi.flowfile.repository.checkpoint.interval>
<nifi.flowfile.repository.always.sync>false</nifi.flowfile.repository.always.sync>
<nifi.swap.manager.implementation>org.apache.nifi.controller.FileSystemSwapManager</nifi.swap.manager.implementation>
<nifi.queue.swap.threshold>20000</nifi.queue.swap.threshold>
<nifi.swap.storage.directory>./flowfile_repository/swap</nifi.swap.storage.directory>
<nifi.swap.in.period>5 sec</nifi.swap.in.period>
<nifi.swap.in.threads>1</nifi.swap.in.threads>
<nifi.swap.out.period>5 sec</nifi.swap.out.period>
<nifi.swap.out.threads>4</nifi.swap.out.threads>
<nifi.content.repository.implementation>org.apache.nifi.controller.repository.FileSystemRepository</nifi.content.repository.implementation>
<nifi.content.claim.max.appendable.size>10 MB</nifi.content.claim.max.appendable.size>
<nifi.content.claim.max.flow.files>100</nifi.content.claim.max.flow.files>
<nifi.content.repository.directory.default>./content_repository</nifi.content.repository.directory.default>
<nifi.content.repository.archive.max.retention.period />
<nifi.content.repository.archive.max.usage.percentage />
<nifi.content.repository.archive.enabled>false</nifi.content.repository.archive.enabled>
<nifi.content.repository.always.sync>false</nifi.content.repository.always.sync>
<nifi.content.viewer.url />
<nifi.restore.directory />
<nifi.ui.banner.text></nifi.ui.banner.text>
<nifi.ui.autorefresh.interval>30 sec</nifi.ui.autorefresh.interval>
<nifi.nar.library.directory>./lib</nifi.nar.library.directory>
<nifi.nar.working.directory>./work/nar/</nifi.nar.working.directory>
<nifi.documentation.working.directory>./work/docs/components</nifi.documentation.working.directory>
<nifi.sensitive.props.algorithm>PBEWITHMD5AND256BITAES-CBC-OPENSSL</nifi.sensitive.props.algorithm>
<nifi.sensitive.props.provider>BC</nifi.sensitive.props.provider>
<nifi.h2.url.append>;LOCK_TIMEOUT=25000;WRITE_DELAY=0;AUTO_SERVER=FALSE</nifi.h2.url.append>
<nifi.remote.input.socket.port>9990</nifi.remote.input.socket.port>
<!-- persistent provenance repository properties -->
<nifi.provenance.repository.implementation>org.apache.nifi.provenance.PersistentProvenanceRepository</nifi.provenance.repository.implementation>
<nifi.provenance.repository.directory.default>./provenance_repository</nifi.provenance.repository.directory.default>
<nifi.provenance.repository.max.storage.time>24 hours</nifi.provenance.repository.max.storage.time>
<nifi.provenance.repository.max.storage.size>1 GB</nifi.provenance.repository.max.storage.size>
<nifi.provenance.repository.rollover.time>5 mins</nifi.provenance.repository.rollover.time>
<nifi.provenance.repository.rollover.size>100 MB</nifi.provenance.repository.rollover.size>
<nifi.provenance.repository.query.threads>2</nifi.provenance.repository.query.threads>
<nifi.provenance.repository.compress.on.rollover>true</nifi.provenance.repository.compress.on.rollover>
<nifi.provenance.repository.indexed.fields>EventType, FlowFileUUID, Filename, ProcessorID</nifi.provenance.repository.indexed.fields>
<nifi.provenance.repository.indexed.attributes />
<nifi.provenance.repository.index.shard.size>500 MB</nifi.provenance.repository.index.shard.size>
<nifi.provenance.repository.always.sync>false</nifi.provenance.repository.always.sync>
<nifi.provenance.repository.journal.count>16</nifi.provenance.repository.journal.count>
<!-- volatile provenance repository properties -->
<nifi.provenance.repository.buffer.size>100000</nifi.provenance.repository.buffer.size>
<!-- Component status repository properties -->
<nifi.components.status.repository.implementation>org.apache.nifi.controller.status.history.VolatileComponentStatusRepository</nifi.components.status.repository.implementation>
<nifi.components.status.repository.buffer.size>288</nifi.components.status.repository.buffer.size>
<nifi.components.status.snapshot.frequency>5 mins</nifi.components.status.snapshot.frequency>
<!-- nifi.properties: web properties -->
<nifi.web.war.directory>./lib</nifi.web.war.directory>
<nifi.web.http.host />
<nifi.web.http.port>8080</nifi.web.http.port>
<nifi.web.https.host />
<nifi.web.https.port />
<nifi.jetty.work.dir>./work/jetty</nifi.jetty.work.dir>
<!-- nifi.properties: security properties -->
<nifi.security.keystore />
<nifi.security.keystoreType />
<nifi.security.keystorePasswd />
<nifi.security.keyPasswd />
<nifi.security.truststore />
<nifi.security.truststoreType />
<nifi.security.truststorePasswd />
<nifi.security.needClientAuth />
<nifi.security.authorizedUsers.file>./conf/authorized-users.xml</nifi.security.authorizedUsers.file>
<nifi.security.user.credential.cache.duration>24 hours</nifi.security.user.credential.cache.duration>
<nifi.security.user.authority.provider>file-provider</nifi.security.user.authority.provider>
<nifi.security.x509.principal.extractor />
<nifi.security.support.new.account.requests />
<nifi.security.ocsp.responder.url />
<nifi.security.ocsp.responder.certificate />
<!-- nifi.properties: cluster common properties (cluster manager and nodes must have same values) -->
<nifi.cluster.protocol.heartbeat.interval>5 sec</nifi.cluster.protocol.heartbeat.interval>
<nifi.cluster.protocol.is.secure>false</nifi.cluster.protocol.is.secure>
<nifi.cluster.protocol.socket.timeout>30 sec</nifi.cluster.protocol.socket.timeout>
<nifi.cluster.protocol.connection.handshake.timeout>45 sec</nifi.cluster.protocol.connection.handshake.timeout>
<nifi.cluster.protocol.use.multicast>false</nifi.cluster.protocol.use.multicast>
<nifi.cluster.protocol.multicast.address />
<nifi.cluster.protocol.multicast.port />
<nifi.cluster.protocol.multicast.service.broadcast.delay>500 ms</nifi.cluster.protocol.multicast.service.broadcast.delay>
<nifi.cluster.protocol.multicast.service.locator.attempts>3</nifi.cluster.protocol.multicast.service.locator.attempts>
<nifi.cluster.protocol.multicast.service.locator.attempts.delay>1 sec</nifi.cluster.protocol.multicast.service.locator.attempts.delay>
<!-- nifi.properties: cluster node properties (only configure for cluster nodes) -->
<nifi.cluster.is.node>false</nifi.cluster.is.node>
<nifi.cluster.node.address />
<nifi.cluster.node.protocol.port />
<nifi.cluster.node.protocol.threads>2</nifi.cluster.node.protocol.threads>
<nifi.cluster.node.unicast.manager.address />
<nifi.cluster.node.unicast.manager.protocol.port />
<!-- nifi.properties: cluster manager properties (only configure for cluster manager) -->
<nifi.cluster.is.manager>false</nifi.cluster.is.manager>
<nifi.cluster.manager.address />
<nifi.cluster.manager.protocol.port />
<nifi.cluster.manager.node.firewall.file />
<nifi.cluster.manager.node.event.history.size>10</nifi.cluster.manager.node.event.history.size>
<nifi.cluster.manager.node.api.connection.timeout>30 sec</nifi.cluster.manager.node.api.connection.timeout>
<nifi.cluster.manager.node.api.read.timeout>30 sec</nifi.cluster.manager.node.api.read.timeout>
<nifi.cluster.manager.node.api.request.threads>10</nifi.cluster.manager.node.api.request.threads>
<nifi.cluster.manager.flow.retrieval.delay>5 sec</nifi.cluster.manager.flow.retrieval.delay>
<nifi.cluster.manager.protocol.threads>10</nifi.cluster.manager.protocol.threads>
<nifi.cluster.manager.safemode.duration>0 sec</nifi.cluster.manager.safemode.duration>
<!--
Properties to execute nifi using Maven. It is assumed that the assembly has a
format of type 'dir' so that the executable can be referenced in a defined location.
-->
<nifi.assembly.id>bin</nifi.assembly.id>
<nifi.executable>${project.artifactId}-${project.version}/bin/${project.artifactId}</nifi.executable>
</properties>
<profiles>
<profile>
<id>standard-rpm</id>
<activation>
<activeByDefault>false</activeByDefault>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>rpm-maven-plugin</artifactId>
<version>2.0.1</version>
<executions>
<execution>
<goals>
<goal>rpm</goal>
</goals>
</execution>
</executions>
<configuration>
<name>${project.artifactId}</name>
<projversion>${project.version}</projversion>
<summary>NiFi data flow framework</summary>
<description>
Nifi is a simple event processing framework largely
based on the Flow-Based Programming concepts.
</description>
<copyright>NONE</copyright>
<url>http://nifi.apache.org</url>
<group>Utilities</group>
<requires>
<require>jdk</require>
</requires>
<prefix>/opt/${project.artifactId}</prefix>
<defineStatements>
<defineStatement>_use_internal_dependency_generator 0</defineStatement>
</defineStatements>
<defaultDirmode>750</defaultDirmode>
<defaultFilemode>640</defaultFilemode>
<defaultUsername>dataflow</defaultUsername>
<defaultGroupname>dataflow</defaultGroupname>
<mappings>
<mapping>
<directory>/opt/${project.artifactId}/${project.artifactId}-${project.version}</directory>
</mapping>
<mapping>
<directory>/opt/${project.artifactId}/${project.artifactId}-${project.version}/bin</directory>
<filemode>750</filemode>
<sources>
<source>
<location>${project.build.directory}/generated-resources/bin/nifi</location>
<destination>${project.artifactId}</destination>
<filter>true</filter>
</source>
<source>
<location>${project.build.directory}/generated-resources/bin/wrapper-linux-x86-32</location>
</source>
<source>
<location>${project.build.directory}/generated-resources/bin/wrapper-linux-x86-64</location>
</source>
</sources>
</mapping>
<mapping>
<directory>/opt/${project.artifactId}/${project.artifactId}-${project.version}/conf</directory>
<configuration>true</configuration>
<sources>
<source>
<location>${project.build.directory}/generated-resources/conf</location>
<excludes>
<exclude>nifi.properties</exclude>
</excludes>
<filter>true</filter>
</source>
<source>
<location>${project.build.directory}/generated-resources/conf/nifi.properties</location>
<destination>${project.artifactId}.properties</destination>
<filter>true</filter>
</source>
</sources>
</mapping>
<mapping>
<directory>/opt/${project.artifactId}/${project.artifactId}-${project.version}/lib</directory>
<dependency />
<sources>
<source>
<location>${project.build.directory}/generated-resources/lib</location>
<excludes>
<exclude>**/*-aix*</exclude>
<exclude>**/*-freebsd*</exclude>
<exclude>**/*-hpux*</exclude>
<exclude>**/*-linux-ia*</exclude>
<exclude>**/*-linux-ppc*</exclude>
<exclude>**/*-macosx*</exclude>
<exclude>**/*-solaris*</exclude>
<exclude>**/*-windows*</exclude>
</excludes>
</source>
</sources>
</mapping>
<mapping>
<directory>/opt/${project.artifactId}/${project.artifactId}-${project.version}/docs</directory>
<sources>
<source>
<location>${project.build.directory}/generated-resources/docs</location>
</source>
</sources>
</mapping>
<mapping>
<directory>/opt/${project.artifactId}/${project.artifactId}-${project.version}/logs</directory>
<sources>
<source>
<location>${project.build.directory}/generated-resources/logs</location>
</source>
</sources>
</mapping>
</mappings>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>
</project>

123
assemblies/nifi/src/main/assembly/dependencies.xml Normal file
View File

@ -0,0 +1,123 @@
<?xml version="1.0"?>
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<assembly>
<id>bin</id>
<formats>
<format>dir</format>
<format>tar.gz</format>
</formats>
<includeBaseDirectory>true</includeBaseDirectory>
<baseDirectory>${project.artifactId}-${project.version}</baseDirectory>
<dependencySets>
<dependencySet>
<scope>runtime</scope>
<outputDirectory>lib</outputDirectory>
<useTransitiveFiltering>true</useTransitiveFiltering>
</dependencySet>
</dependencySets>
<fileSets>
<fileSet>
<directory>${project.build.directory}/generated-resources/bin</directory>
<outputDirectory>bin</outputDirectory>
<directoryMode>0750</directoryMode>
<fileMode>0750</fileMode>
<filtered>false</filtered>
</fileSet>
<fileSet>
<directory>${project.build.directory}/generated-resources/conf</directory>
<outputDirectory>conf</outputDirectory>
<excludes>
<exclude>nifi.properties</exclude>
</excludes>
<directoryMode>0750</directoryMode>
<fileMode>0640</fileMode>
<filtered>true</filtered>
</fileSet>
<fileSet>
<directory>${project.build.directory}/generated-resources/lib</directory>
<outputDirectory>lib</outputDirectory>
<directoryMode>0750</directoryMode>
<fileMode>0640</fileMode>
</fileSet>
<fileSet>
<directory>${project.build.directory}/generated-resources/logs</directory>
<outputDirectory>logs</outputDirectory>
<directoryMode>0750</directoryMode>
<fileMode>0640</fileMode>
</fileSet>
<fileSet>
<directory>${project.build.directory}/generated-resources/docs</directory>
<outputDirectory>docs</outputDirectory>
<directoryMode>0750</directoryMode>
<fileMode>0640</fileMode>
<filtered>false</filtered>
</fileSet>
</fileSets>
<files>
<file>
<source>${project.build.directory}/generated-resources/conf/nifi.properties</source>
<outputDirectory>conf</outputDirectory>
<destName>${project.artifactId}.properties</destName>
<fileMode>0640</fileMode>
<filtered>true</filtered>
</file>
<file>
<source>${project.build.directory}/generated-resources/bin/nifi.sh</source>
<outputDirectory>bin</outputDirectory>
<destName>nifi.sh</destName>
<fileMode>0750</fileMode>
<filtered>true</filtered>
</file>
<file>
<source>${project.build.directory}/generated-resources/docs/README.md</source>
<outputDirectory>docs</outputDirectory>
<destName>README.md</destName>
<fileMode>0640</fileMode>
<filtered>true</filtered>
</file>
<file>
<source>${project.build.directory}/generated-resources/DISCLAIMER</source>
<outputDirectory>./</outputDirectory>
<destName>DISCLAIMER</destName>
<fileMode>0640</fileMode>
<filtered>true</filtered>
</file>
<file>
<source>${project.build.directory}/generated-resources/DISCLAIMER</source>
<outputDirectory>./</outputDirectory>
<destName>DISCLAIMER</destName>
<fileMode>0640</fileMode>
<filtered>true</filtered>
</file>
<file>
<source>${project.build.directory}/generated-resources/LICENSE</source>
<outputDirectory>./</outputDirectory>
<destName>LICENSE</destName>
<fileMode>0640</fileMode>
<filtered>true</filtered>
</file>
<file>
<source>${project.build.directory}/generated-resources/NOTICE</source>
<outputDirectory>./</outputDirectory>
<destName>NOTICE</destName>
<fileMode>0640</fileMode>
<filtered>true</filtered>
</file>
</files>
</assembly>

32
commons/core-flowfile-attributes/pom.xml Normal file
View File

@ -0,0 +1,32 @@
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-parent</artifactId>
<version>0.0.1-SNAPSHOT</version>
</parent>
<artifactId>nifi-core-flowfile-attributes</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
<name>NiFi: Core FlowFile Attributes</name>
<dependencies>
</dependencies>
</project>

72
commons/core-flowfile-attributes/src/main/java/org/apache/nifi/flowfile/attributes/CoreAttributes.java Normal file
View File

@ -0,0 +1,72 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.flowfile.attributes;
public enum CoreAttributes implements FlowFileAttributeKey {
/**
* The flowfile's path indicates the relative directory to which a FlowFile belongs and does not
* contain the filename
*/
PATH("path"),
/**
* The flowfile's absolute path indicates the absolute directory to which a FlowFile belongs and does not
* contain the filename
*/
ABSOLUTE_PATH("absolute.path"),
/**
* The filename of the FlowFile. The filename should not contain any directory structure.
*/
FILENAME("filename"),
/**
* A unique UUID assigned to this FlowFile
*/
UUID("uuid"),
/**
* A numeric value indicating the FlowFile priority
*/
PRIORITY("priority"),
/**
* The MIME Type of this FlowFile
*/
MIME_TYPE("mime.type"),
/**
* Specifies the reason that a FlowFile is being discarded
*/
DISCARD_REASON("discard.reason"),
/**
* Indicates an identifier other than the FlowFile's UUID that is known to refer to this FlowFile.
*/
ALTERNATE_IDENTIFIER("alternate.identifier");
private final String key;
private CoreAttributes(final String key) {
this.key = key;
}
@Override
public String key() {
return key;
}
}

21
commons/core-flowfile-attributes/src/main/java/org/apache/nifi/flowfile/attributes/FlowFileAttributeKey.java Normal file
View File

@ -0,0 +1,21 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.flowfile.attributes;
public interface FlowFileAttributeKey {
String key();
}

2
commons/data-provenance-utils/.gitignore vendored Executable file
View File

@ -0,0 +1,2 @@
/target
/target

42
commons/data-provenance-utils/pom.xml Normal file
View File

@ -0,0 +1,42 @@
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-parent</artifactId>
<version>0.0.1-SNAPSHOT</version>
</parent>
<artifactId>data-provenance-utils</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
<name>data-provenance-utils</name>
<dependencies>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-api</artifactId>
<version>[0.0.1-SNAPSHOT,1.0.0-SNAPSHOT)</version>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-core-flowfile-attributes</artifactId>
<version>0.0.1-SNAPSHOT</version>
</dependency>
</dependencies>
</project>

87
commons/data-provenance-utils/src/main/java/org/apache/nifi/provenance/AsyncLineageSubmission.java Normal file
View File

@ -0,0 +1,87 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.provenance;
import java.util.Collection;
import java.util.Date;
import java.util.UUID;
import org.apache.nifi.provenance.lineage.ComputeLineageSubmission;
import org.apache.nifi.provenance.lineage.LineageComputationType;
/**
*
*/
public class AsyncLineageSubmission implements ComputeLineageSubmission {
private final String lineageIdentifier = UUID.randomUUID().toString();
private final Date submissionTime = new Date();
private final LineageComputationType computationType;
private final Long eventId;
private final Collection<String> lineageFlowFileUuids;
private volatile boolean canceled = false;
private final StandardLineageResult result;
public AsyncLineageSubmission(final LineageComputationType computationType, final Long eventId, final Collection<String> lineageFlowFileUuids, final int numSteps) {
this.computationType = computationType;
this.eventId = eventId;
this.lineageFlowFileUuids = lineageFlowFileUuids;
this.result = new StandardLineageResult(numSteps, lineageFlowFileUuids);
}
@Override
public StandardLineageResult getResult() {
return result;
}
@Override
public Date getSubmissionTime() {
return submissionTime;
}
@Override
public String getLineageIdentifier() {
return lineageIdentifier;
}
@Override
public void cancel() {
this.canceled = true;
}
@Override
public boolean isCanceled() {
return canceled;
}
@Override
public LineageComputationType getLineageComputationType() {
return computationType;
}
@Override
public Long getExpandedEventId() {
return eventId;
}
@Override
public Collection<String> getLineageFlowFileUuids() {
return lineageFlowFileUuids;
}
}

81
commons/data-provenance-utils/src/main/java/org/apache/nifi/provenance/AsyncQuerySubmission.java Normal file
View File

@ -0,0 +1,81 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.provenance;
import java.util.Date;
import java.util.concurrent.TimeUnit;
import org.apache.nifi.provenance.search.Query;
import org.apache.nifi.provenance.search.QuerySubmission;
/**
*
*/
public class AsyncQuerySubmission implements QuerySubmission {
public static final int TTL = (int) TimeUnit.MILLISECONDS.convert(60, TimeUnit.SECONDS);
private final Date submissionTime = new Date();
private final Query query;
private volatile boolean canceled = false;
private final StandardQueryResult queryResult;
/**
* Constructs an AsyncQuerySubmission with the given query and the given
* number of steps, indicating how many results must be added to this
* AsyncQuerySubmission before it is considered finished
*
* @param query
* @param numSteps
*/
public AsyncQuerySubmission(final Query query, final int numSteps) {
this.query = query;
queryResult = new StandardQueryResult(query, numSteps);
}
@Override
public Date getSubmissionTime() {
return submissionTime;
}
@Override
public String getQueryIdentifier() {
return query.getIdentifier();
}
@Override
public void cancel() {
this.canceled = true;
queryResult.cancel();
}
@Override
public boolean isCanceled() {
return canceled;
}
@Override
public Query getQuery() {
return query;
}
@Override
public StandardQueryResult getResult() {
return queryResult;
}
}

95
commons/data-provenance-utils/src/main/java/org/apache/nifi/provenance/NamedSearchableField.java Normal file
View File

@ -0,0 +1,95 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.provenance;
import org.apache.nifi.provenance.search.SearchableField;
import org.apache.nifi.provenance.search.SearchableFieldType;
import static java.util.Objects.requireNonNull;
/**
*
*/
public class NamedSearchableField implements SearchableField {
private final String identifier;
private final String searchableName;
private final SearchableFieldType fieldType;
private final String friendlyName;
private final boolean attribute;
NamedSearchableField(final String identifier, final String searchableName, final String friendlyName, final boolean attribute) {
this(identifier, searchableName, friendlyName, attribute, SearchableFieldType.STRING);
}
NamedSearchableField(final String identifier, final String searchableName, final String friendlyName, final boolean attribute, final SearchableFieldType fieldType) {
this.identifier = requireNonNull(identifier);
this.searchableName = requireNonNull(searchableName);
this.friendlyName = requireNonNull(friendlyName);
this.attribute = requireNonNull(attribute);
this.fieldType = requireNonNull(fieldType);
}
@Override
public String getIdentifier() {
return identifier;
}
@Override
public String getSearchableFieldName() {
return searchableName;
}
@Override
public String getFriendlyName() {
return friendlyName;
}
@Override
public boolean isAttribute() {
return attribute;
}
@Override
public SearchableFieldType getFieldType() {
return fieldType;
}
@Override
public String toString() {
return friendlyName;
}
@Override
public int hashCode() {
return 298347 + searchableName.hashCode() + (attribute ? 1 : 0);
}
@Override
public boolean equals(final Object obj) {
if (obj == null) {
return false;
}
if (!(obj instanceof SearchableField)) {
return false;
}
final SearchableField other = (SearchableField) obj;
return (this.searchableName.equals(other.getSearchableFieldName()) && attribute == other.isAttribute());
}
}

53
commons/data-provenance-utils/src/main/java/org/apache/nifi/provenance/SearchableFieldParser.java Normal file
View File

@ -0,0 +1,53 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.provenance;
import java.util.ArrayList;
import java.util.List;
import org.apache.nifi.provenance.search.SearchableField;
public class SearchableFieldParser {
public static List<SearchableField> extractSearchableFields(final String indexedFieldString, final boolean predefinedField) {
final List<SearchableField> searchableFields = new ArrayList<>();
if (indexedFieldString != null) {
final String[] split = indexedFieldString.split(",");
for (String fieldName : split) {
fieldName = fieldName.trim();
if (fieldName.isEmpty()) {
continue;
}
final SearchableField searchableField;
if (predefinedField) {
searchableField = SearchableFields.getSearchableField(fieldName);
} else {
searchableField = SearchableFields.newSearchableAttribute(fieldName);
}
if (searchableField == null) {
throw new RuntimeException("Invalid Configuration: Provenance Repository configured to Index field '" + fieldName + "', but this is not a valid field");
}
searchableFields.add(searchableField);
}
}
return searchableFields;
}
}

84
commons/data-provenance-utils/src/main/java/org/apache/nifi/provenance/SearchableFields.java Normal file
View File

@ -0,0 +1,84 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.provenance;
import org.apache.nifi.provenance.search.SearchableField;
import org.apache.nifi.provenance.search.SearchableFieldType;
import java.util.Collection;
import java.util.Collections;
import java.util.HashMap;
import java.util.Map;
/**
*
*/
public class SearchableFields {
public static final SearchableField Identifier = new NamedSearchableField("Identifier", "identifier", "Identifier", false);
public static final SearchableField EventTime = new NamedSearchableField("EventTime", "time", "Event Time", false, SearchableFieldType.DATE);
public static final SearchableField FlowFileUUID = new NamedSearchableField("FlowFileUUID", "uuid", "FlowFile UUID", false);
public static final SearchableField Filename = new NamedSearchableField("Filename", "filename", "Filename", false);
public static final SearchableField EventType = new NamedSearchableField("EventType", "eventType", "Event Type", false);
public static final SearchableField TransitURI = new NamedSearchableField("TransitURI", "transitUri", "Transit URI", false);
public static final SearchableField ComponentID = new NamedSearchableField("ProcessorID", "processorId", "Component ID", false);
public static final SearchableField AlternateIdentifierURI = new NamedSearchableField("AlternateIdentifierURI", "alternateIdentifierUri", "Alternate Identifier URI", false);
public static final SearchableField FileSize = new NamedSearchableField("FileSize", "fileSize", "File Size", false, SearchableFieldType.DATA_SIZE);
public static final SearchableField Details = new NamedSearchableField("Details", "details", "Details", false, SearchableFieldType.STRING);
public static final SearchableField Relationship = new NamedSearchableField("Relationship", "relationship", "Relationship", false, SearchableFieldType.STRING);
public static final SearchableField LineageStartDate = new NamedSearchableField("LineageStartDate", "lineageStartDate", "Lineage Start Date", false, SearchableFieldType.DATE);
public static final SearchableField LineageIdentifier = new NamedSearchableField("LineageIdentifiers", "lineageIdentifier", "Lineage Identifier", false, SearchableFieldType.STRING);
public static final SearchableField ContentClaimSection = new NamedSearchableField("ContentClaimSection", "contentClaimSection", "Content Claim Section", false, SearchableFieldType.STRING);
public static final SearchableField ContentClaimContainer = new NamedSearchableField("ContentClaimContainer", "contentClaimContainer", "Content Claim Container", false, SearchableFieldType.STRING);
public static final SearchableField ContentClaimIdentifier = new NamedSearchableField("ContentClaimIdentifier", "contentClaimIdentifier", "Content Claim Identifier", false, SearchableFieldType.STRING);
public static final SearchableField ContentClaimOffset = new NamedSearchableField("ContentClaimOffset", "contentClaimOffset", "Content Claim Offset", false, SearchableFieldType.LONG);
public static final SearchableField SourceQueueIdentifier = new NamedSearchableField("SourceQueueIdentifier", "sourceQueueIdentifier", "Source Queue Identifier", false, SearchableFieldType.STRING);
private static final Map<String, SearchableField> standardFields;
static {
final SearchableField[] searchableFields = new SearchableField[]{
EventTime, FlowFileUUID, Filename, EventType, TransitURI,
ComponentID, AlternateIdentifierURI, FileSize, Relationship, Details,
LineageStartDate, LineageIdentifier, ContentClaimSection, ContentClaimContainer, ContentClaimIdentifier,
ContentClaimOffset, SourceQueueIdentifier};
final Map<String, SearchableField> fields = new HashMap<>();
for (final SearchableField field : searchableFields) {
fields.put(field.getIdentifier(), field);
}
standardFields = Collections.unmodifiableMap(fields);
}
private SearchableFields() {
}
public static Collection<SearchableField> getStandardFields() {
return standardFields.values();
}
public static SearchableField getSearchableField(final String fieldIdentifier) {
return standardFields.get(fieldIdentifier);
}
public static SearchableField newSearchableAttribute(final String attributeName) {
return new NamedSearchableField(attributeName, attributeName, attributeName, true);
}
}

324
commons/data-provenance-utils/src/main/java/org/apache/nifi/provenance/StandardLineageResult.java Normal file
View File

@ -0,0 +1,324 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.provenance;
import java.util.ArrayList;
import java.util.Collection;
import java.util.Collections;
import java.util.Comparator;
import java.util.Date;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReadWriteLock;
import java.util.concurrent.locks.ReentrantReadWriteLock;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.apache.nifi.provenance.lineage.ComputeLineageResult;
import org.apache.nifi.provenance.lineage.EdgeNode;
import org.apache.nifi.provenance.lineage.EventNode;
import org.apache.nifi.provenance.lineage.FlowFileNode;
import org.apache.nifi.provenance.lineage.LineageEdge;
import org.apache.nifi.provenance.lineage.LineageNode;
/**
*
*/
public class StandardLineageResult implements ComputeLineageResult {
public static final int TTL = (int) TimeUnit.MILLISECONDS.convert(30, TimeUnit.MINUTES);
private static final Logger logger = LoggerFactory.getLogger(StandardLineageResult.class);
private final Collection<String> flowFileUuids;
private final Collection<ProvenanceEventRecord> relevantRecords = new ArrayList<>();
private final Set<LineageNode> nodes = new HashSet<>();
private final Set<LineageEdge> edges = new HashSet<>();
private final int numSteps;
private final long creationNanos;
private long computationNanos;
private final ReadWriteLock rwLock = new ReentrantReadWriteLock();
private final Lock readLock = rwLock.readLock();
private final Lock writeLock = rwLock.writeLock();
private Date expirationDate = null;
private String error = null;
private int numCompletedSteps = 0;
private volatile boolean canceled = false;
public StandardLineageResult(final int numSteps, final Collection<String> flowFileUuids) {
this.numSteps = numSteps;
this.creationNanos = System.nanoTime();
this.flowFileUuids = flowFileUuids;
updateExpiration();
}
@Override
public List<LineageNode> getNodes() {
readLock.lock();
try {
return new ArrayList<>(nodes);
} finally {
readLock.unlock();
}
}
@Override
public List<LineageEdge> getEdges() {
readLock.lock();
try {
return new ArrayList<>(edges);
} finally {
readLock.unlock();
}
}
public int getNumberOfEdges() {
readLock.lock();
try {
return edges.size();
} finally {
readLock.unlock();
}
}
public int getNumberOfNodes() {
readLock.lock();
try {
return nodes.size();
} finally {
readLock.unlock();
}
}
public long getComputationTime(final TimeUnit timeUnit) {
readLock.lock();
try {
return timeUnit.convert(computationNanos, TimeUnit.NANOSECONDS);
} finally {
readLock.unlock();
}
}
@Override
public Date getExpiration() {
readLock.lock();
try {
return expirationDate;
} finally {
readLock.unlock();
}
}
@Override
public String getError() {
readLock.lock();
try {
return error;
} finally {
readLock.unlock();
}
}
@Override
public int getPercentComplete() {
readLock.lock();
try {
return (numSteps < 1) ? 100 : (int) (((float) numCompletedSteps / (float) numSteps) * 100.0F);
} finally {
readLock.unlock();
}
}
@Override
public boolean isFinished() {
readLock.lock();
try {
return numCompletedSteps >= numSteps || canceled;
} finally {
readLock.unlock();
}
}
public void setError(final String error) {
writeLock.lock();
try {
this.error = error;
numCompletedSteps++;
updateExpiration();
if (numCompletedSteps >= numSteps) {
computationNanos = System.nanoTime() - creationNanos;
}
} finally {
writeLock.unlock();
}
}
public void update(final Collection<ProvenanceEventRecord> records) {
writeLock.lock();
try {
relevantRecords.addAll(records);
numCompletedSteps++;
updateExpiration();
if (numCompletedSteps >= numSteps && error == null) {
computeLineage();
computationNanos = System.nanoTime() - creationNanos;
}
} finally {
writeLock.unlock();
}
}
/**
* Computes the lineage from the relevant Provenance Event Records. This
* method must be called with the write lock held and is only going to be
* useful after all of the records have been successfully obtained
*/
private void computeLineage() {
final long startNanos = System.nanoTime();
nodes.clear();
edges.clear();
Map<String, LineageNode> lastEventMap = new HashMap<>(); // maps FlowFile UUID to last event for that FlowFile
final List<ProvenanceEventRecord> sortedRecords = new ArrayList<>(relevantRecords);
Collections.sort(sortedRecords, new Comparator<ProvenanceEventRecord>() {
@Override
public int compare(final ProvenanceEventRecord o1, final ProvenanceEventRecord o2) {
// Sort on Event Time, then Event ID.
final int eventTimeComparison = Long.compare(o1.getEventTime(), o2.getEventTime());
if (eventTimeComparison == 0) {
return Long.compare(o1.getEventId(), o2.getEventId());
} else {
return eventTimeComparison;
}
}
});
// convert the StandardProvenanceRecord objects into Lineage nodes (FlowFileNode, EventNodes).
for (final ProvenanceEventRecord record : sortedRecords) {
final LineageNode lineageNode = new EventNode(record);
final boolean added = nodes.add(lineageNode);
if (!added) {
logger.debug("Did not add {} because it already exists in the 'nodes' set", lineageNode);
}
// Create an edge that connects this node to the previous node for the same FlowFile UUID.
final LineageNode lastNode = lastEventMap.get(record.getFlowFileUuid());
if (lastNode != null) {
// We calculate the Edge UUID based on whether or not this event is a SPAWN.
// If this event is a SPAWN, then we want to use the previous node's UUID because a
// SPAWN Event's UUID is not necessarily what we want, since a SPAWN Event's UUID pertains to
// only one of (potentially) many UUIDs associated with the event. Otherwise, we know that
// the UUID of this record is appropriate, so we just use it.
final String edgeUuid;
switch (record.getEventType()) {
case JOIN:
case CLONE:
case REPLAY:
edgeUuid = lastNode.getFlowFileUuid();
break;
default:
edgeUuid = record.getFlowFileUuid();
break;
}
edges.add(new EdgeNode(edgeUuid, lastNode, lineageNode));
}
lastEventMap.put(record.getFlowFileUuid(), lineageNode);
switch (record.getEventType()) {
case FORK:
case JOIN:
case REPLAY:
case CLONE: {
// For events that create FlowFile nodes, we need to create the FlowFile Nodes and associated Edges, as appropriate
for (final String childUuid : record.getChildUuids()) {
if (flowFileUuids.contains(childUuid)) {
final FlowFileNode childNode = new FlowFileNode(childUuid, record.getEventTime());
final boolean isNewFlowFile = nodes.add(childNode);
if (!isNewFlowFile) {
final String msg = "Unable to generate Lineage Graph because multiple events were registered claiming to have generated the same FlowFile (UUID = " + childNode.getFlowFileUuid() + ")";
logger.error(msg);
setError(msg);
return;
}
edges.add(new EdgeNode(childNode.getFlowFileUuid(), lineageNode, childNode));
lastEventMap.put(childUuid, childNode);
}
}
for (final String parentUuid : record.getParentUuids()) {
LineageNode lastNodeForParent = lastEventMap.get(parentUuid);
if (lastNodeForParent != null && !lastNodeForParent.equals(lineageNode)) {
edges.add(new EdgeNode(parentUuid, lastNodeForParent, lineageNode));
}
lastEventMap.put(parentUuid, lineageNode);
}
}
break;
case RECEIVE:
case CREATE: {
// for a receive event, we want to create a FlowFile Node that represents the FlowFile received
// and create an edge from the Receive Event to the FlowFile Node
final LineageNode flowFileNode = new FlowFileNode(record.getFlowFileUuid(), record.getEventTime());
final boolean isNewFlowFile = nodes.add(flowFileNode);
if (!isNewFlowFile) {
final String msg = "Found cycle in graph. This indicates that multiple events were registered claiming to have generated the same FlowFile (UUID = " + flowFileNode.getFlowFileUuid() + ")";
setError(msg);
logger.error(msg);
return;
}
edges.add(new EdgeNode(record.getFlowFileUuid(), lineageNode, flowFileNode));
lastEventMap.put(record.getFlowFileUuid(), flowFileNode);
}
break;
default:
break;
}
}
final long nanos = System.nanoTime() - startNanos;
logger.debug("Finished building lineage with {} nodes and {} edges in {} millis", nodes.size(), edges.size(), TimeUnit.NANOSECONDS.toMillis(nanos));
}
void cancel() {
this.canceled = true;
}
/**
* Must be called with write lock!
*/
private void updateExpiration() {
expirationDate = new Date(System.currentTimeMillis() + TTL);
}
}

752
commons/data-provenance-utils/src/main/java/org/apache/nifi/provenance/StandardProvenanceEventRecord.java Normal file
View File

@ -0,0 +1,752 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.provenance;
import java.util.ArrayList;
import java.util.Collections;
import java.util.Date;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
import org.apache.nifi.flowfile.FlowFile;
import org.apache.nifi.flowfile.attributes.CoreAttributes;
import org.apache.nifi.processor.Relationship;
/**
* Holder for provenance relevant information
* <p/>
* @author none
*/
public final class StandardProvenanceEventRecord implements ProvenanceEventRecord {
private final long eventTime;
private final long entryDate;
private final ProvenanceEventType eventType;
private final long lineageStartDate;
private final Set<String> lineageIdentifiers;
private final String componentId;
private final String componentType;
private final String transitUri;
private final String sourceSystemFlowFileIdentifier;
private final String uuid;
private final List<String> parentUuids;
private final List<String> childrenUuids;
private final String alternateIdentifierUri;
private final String details;
private final String relationship;
private final long storageByteOffset;
private final String storageFilename;
private final long eventDuration;
private final String contentClaimSection;
private final String contentClaimContainer;
private final String contentClaimIdentifier;
private final Long contentClaimOffset;
private final long contentSize;
private final String previousClaimSection;
private final String previousClaimContainer;
private final String previousClaimIdentifier;
private final Long previousClaimOffset;
private final Long previousSize;
private final String sourceQueueIdentifier;
private final Map<String, String> previousAttributes;
private final Map<String, String> updatedAttributes;
private volatile long eventId;
private StandardProvenanceEventRecord(final Builder builder) {
this.eventTime = builder.eventTime;
this.entryDate = builder.entryDate;
this.eventType = builder.eventType;
this.componentId = builder.componentId;
this.componentType = builder.componentType;
this.transitUri = builder.transitUri;
this.sourceSystemFlowFileIdentifier = builder.sourceSystemFlowFileIdentifier;
this.uuid = builder.uuid;
this.parentUuids = builder.parentUuids;
this.childrenUuids = builder.childrenUuids;
this.alternateIdentifierUri = builder.alternateIdentifierUri;
this.details = builder.details;
this.relationship = builder.relationship;
this.storageByteOffset = builder.storageByteOffset;
this.storageFilename = builder.storageFilename;
this.eventDuration = builder.eventDuration;
this.lineageStartDate = builder.lineageStartDate;
this.lineageIdentifiers = Collections.unmodifiableSet(builder.lineageIdentifiers);
previousClaimSection = builder.previousClaimSection;
previousClaimContainer = builder.previousClaimContainer;
previousClaimIdentifier = builder.previousClaimIdentifier;
previousClaimOffset = builder.previousClaimOffset;
previousSize = builder.previousSize;
contentClaimSection = builder.contentClaimSection;
contentClaimContainer = builder.contentClaimContainer;
contentClaimIdentifier = builder.contentClaimIdentifier;
contentClaimOffset = builder.contentClaimOffset;
contentSize = builder.contentSize;
previousAttributes = builder.previousAttributes == null ? Collections.<String, String>emptyMap() : Collections.unmodifiableMap(builder.previousAttributes);
updatedAttributes = builder.updatedAttributes == null ? Collections.<String, String>emptyMap() : Collections.unmodifiableMap(builder.updatedAttributes);
sourceQueueIdentifier = builder.sourceQueueIdentifier;
}
public String getStorageFilename() {
return storageFilename;
}
public long getStorageByteOffset() {
return storageByteOffset;
}
void setEventId(final long eventId) {
this.eventId = eventId;
}
@Override
public long getEventId() {
return eventId;
}
@Override
public long getEventTime() {
return eventTime;
}
@Override
public Set<String> getLineageIdentifiers() {
return lineageIdentifiers;
}
@Override
public long getLineageStartDate() {
return lineageStartDate;
}
@Override
public long getFileSize() {
return contentSize;
}
@Override
public Long getPreviousFileSize() {
return previousSize;
}
@Override
public ProvenanceEventType getEventType() {
return eventType;
}
@Override
public Map<String, String> getAttributes() {
final Map<String, String> allAttrs = new HashMap<>(previousAttributes.size() + updatedAttributes.size());
allAttrs.putAll(previousAttributes);
for (final Map.Entry<String, String> entry : updatedAttributes.entrySet()) {
if (entry.getValue() != null) {
allAttrs.put(entry.getKey(), entry.getValue());
}
}
return allAttrs;
}
@Override
public String getComponentId() {
return componentId;
}
@Override
public String getComponentType() {
return componentType;
}
@Override
public String getTransitUri() {
return transitUri;
}
@Override
public String getSourceSystemFlowFileIdentifier() {
return sourceSystemFlowFileIdentifier;
}
@Override
public String getFlowFileUuid() {
return uuid;
}
@Override
public List<String> getParentUuids() {
return parentUuids == null ? Collections.<String>emptyList() : parentUuids;
}
@Override
public List<String> getChildUuids() {
return childrenUuids == null ? Collections.<String>emptyList() : childrenUuids;
}
@Override
public String getAlternateIdentifierUri() {
return alternateIdentifierUri;
}
@Override
public long getEventDuration() {
return eventDuration;
}
@Override
public String getDetails() {
return details;
}
@Override
public String getRelationship() {
return relationship;
}
@Override
public long getFlowFileEntryDate() {
return entryDate;
}
@Override
public String getContentClaimSection() {
return contentClaimSection;
}
@Override
public String getContentClaimContainer() {
return contentClaimContainer;
}
@Override
public String getContentClaimIdentifier() {
return contentClaimIdentifier;
}
@Override
public Long getContentClaimOffset() {
return contentClaimOffset;
}
@Override
public String getSourceQueueIdentifier() {
return sourceQueueIdentifier;
}
@Override
public Map<String, String> getPreviousAttributes() {
return previousAttributes;
}
@Override
public String getPreviousContentClaimContainer() {
return previousClaimContainer;
}
@Override
public String getPreviousContentClaimIdentifier() {
return previousClaimIdentifier;
}
@Override
public Long getPreviousContentClaimOffset() {
return previousClaimOffset;
}
@Override
public String getPreviousContentClaimSection() {
return previousClaimSection;
}
@Override
public Map<String, String> getUpdatedAttributes() {
return updatedAttributes;
}
@Override
public int hashCode() {
final int eventTypeCode;
if (eventType == ProvenanceEventType.CLONE || eventType == ProvenanceEventType.JOIN || eventType == ProvenanceEventType.FORK) {
eventTypeCode = 1472;
} else if (eventType == ProvenanceEventType.REPLAY) {
eventTypeCode = 21479 + (int) (0x7FFFFFFF & eventTime); // use lower bits of event time.
} else {
eventTypeCode = 4812 + eventType.hashCode() + 4 * uuid.hashCode();
}
return -37423 + 3 * componentId.hashCode() + (transitUri == null ? 0 : 41 * transitUri.hashCode())
+ (relationship == null ? 0 : 47 * relationship.hashCode()) + 44 * eventTypeCode;
}
@Override
public boolean equals(final Object obj) {
if (obj == null) {
return false;
}
if (obj == this) {
return true;
}
if (!(obj instanceof StandardProvenanceEventRecord)) {
return false;
}
final StandardProvenanceEventRecord other = (StandardProvenanceEventRecord) obj;
// If event ID's are populated and not equal, return false. If they have not yet been populated, do not
// use them in the comparison.
if (eventId > 0L && other.getEventId() > 0L && eventId != other.getEventId()) {
return false;
}
if (eventType != other.eventType) {
return false;
}
if (!componentId.equals(other.componentId)) {
return false;
}
if (different(parentUuids, other.parentUuids)) {
return false;
}
if (different(childrenUuids, other.childrenUuids)) {
return false;
}
// SPAWN had issues indicating which should be the event's FlowFileUUID in the case that there is 1 parent and 1 child.
if (!uuid.equals(other.uuid)) {
return false;
}
if (different(transitUri, other.transitUri)) {
return false;
}
if (different(relationship, other.relationship)) {
return false;
}
return !(eventType == ProvenanceEventType.REPLAY && eventTime != other.getEventTime());
}
private boolean different(final Object a, final Object b) {
if (a == null && b == null) {
return false;
}
if (a == null || b == null) {
return true;
}
return !a.equals(b);
}
private boolean different(final List<String> a, final List<String> b) {
if (a == null && b == null) {
return false;
}
if (a == null && b != null) {
return true;
}
if (a != null && b == null) {
return true;
}
if (a.size() != b.size()) {
return true;
}
final List<String> sortedA = new ArrayList<>(a);
final List<String> sortedB = new ArrayList<>(b);
Collections.sort(sortedA);
Collections.sort(sortedB);
for (int i = 0; i < sortedA.size(); i++) {
if (!sortedA.get(i).equals(sortedB.get(i))) {
return true;
}
}
return false;
}
@Override
public String toString() {
return "ProvenanceEventRecord ["
+ "eventId=" + eventId
+ ", eventType=" + eventType
+ ", eventTime=" + new Date(eventTime)
+ ", uuid=" + uuid
+ ", fileSize=" + contentSize
+ ", componentId=" + componentId
+ ", transitUri=" + transitUri
+ ", sourceSystemFlowFileIdentifier=" + sourceSystemFlowFileIdentifier
+ ", parentUuids=" + parentUuids
+ ", alternateIdentifierUri=" + alternateIdentifierUri + "]";
}
public static class Builder implements ProvenanceEventBuilder {
private long eventTime = System.currentTimeMillis();
private long entryDate;
private long lineageStartDate;
private Set<String> lineageIdentifiers = new HashSet<>();
private ProvenanceEventType eventType = null;
private String componentId = null;
private String componentType = null;
private String sourceSystemFlowFileIdentifier = null;
private String transitUri = null;
private String uuid = null;
private List<String> parentUuids = null;
private List<String> childrenUuids = null;
private String contentType = null;
private String alternateIdentifierUri = null;
private String details = null;
private String relationship = null;
private long storageByteOffset = -1L;
private long eventDuration = -1L;
private String storageFilename;
private String contentClaimSection;
private String contentClaimContainer;
private String contentClaimIdentifier;
private Long contentClaimOffset;
private Long contentSize;
private String previousClaimSection;
private String previousClaimContainer;
private String previousClaimIdentifier;
private Long previousClaimOffset;
private Long previousSize;
private String sourceQueueIdentifier;
private Map<String, String> previousAttributes;
private Map<String, String> updatedAttributes;
@Override
public Builder fromEvent(final ProvenanceEventRecord event) {
eventTime = event.getEventTime();
entryDate = event.getFlowFileEntryDate();
lineageStartDate = event.getLineageStartDate();
lineageIdentifiers = event.getLineageIdentifiers();
eventType = event.getEventType();
componentId = event.getComponentId();
componentType = event.getComponentType();
transitUri = event.getTransitUri();
sourceSystemFlowFileIdentifier = event.getSourceSystemFlowFileIdentifier();
uuid = event.getFlowFileUuid();
parentUuids = event.getParentUuids();
childrenUuids = event.getChildUuids();
alternateIdentifierUri = event.getAlternateIdentifierUri();
eventDuration = event.getEventDuration();
previousAttributes = event.getPreviousAttributes();
updatedAttributes = event.getUpdatedAttributes();
details = event.getDetails();
relationship = event.getRelationship();
contentClaimSection = event.getContentClaimSection();
contentClaimContainer = event.getContentClaimContainer();
contentClaimIdentifier = event.getContentClaimIdentifier();
contentClaimOffset = event.getContentClaimOffset();
contentSize = event.getFileSize();
previousClaimSection = event.getPreviousContentClaimSection();
previousClaimContainer = event.getPreviousContentClaimContainer();
previousClaimIdentifier = event.getPreviousContentClaimIdentifier();
previousClaimOffset = event.getPreviousContentClaimOffset();
previousSize = event.getPreviousFileSize();
sourceQueueIdentifier = event.getSourceQueueIdentifier();
if (event instanceof StandardProvenanceEventRecord) {
final StandardProvenanceEventRecord standardProvEvent = (StandardProvenanceEventRecord) event;
storageByteOffset = standardProvEvent.storageByteOffset;
storageFilename = standardProvEvent.storageFilename;
}
return this;
}
@Override
public Builder setFlowFileEntryDate(final long entryDate) {
this.entryDate = entryDate;
return this;
}
@Override
public Builder setLineageIdentifiers(final Set<String> lineageIdentifiers) {
this.lineageIdentifiers = lineageIdentifiers;
return this;
}
@Override
public Builder setAttributes(final Map<String, String> previousAttributes, final Map<String, String> updatedAttributes) {
this.previousAttributes = previousAttributes;
this.updatedAttributes = updatedAttributes;
return this;
}
@Override
public Builder setFlowFileUUID(final String uuid) {
this.uuid = uuid;
return this;
}
public Builder setStorageLocation(final String filename, final long offset) {
this.storageFilename = filename;
this.storageByteOffset = offset;
return this;
}
@Override
public Builder setEventTime(long eventTime) {
this.eventTime = eventTime;
return this;
}
@Override
public Builder setEventDuration(final long millis) {
this.eventDuration = millis;
return this;
}
@Override
public Builder setLineageStartDate(final long startDate) {
this.lineageStartDate = startDate;
return this;
}
public Builder addLineageIdentifier(final String lineageIdentifier) {
this.lineageIdentifiers.add(lineageIdentifier);
return this;
}
@Override
public Builder setEventType(ProvenanceEventType eventType) {
this.eventType = eventType;
return this;
}
@Override
public Builder setComponentId(String componentId) {
this.componentId = componentId;
return this;
}
@Override
public Builder setComponentType(String componentType) {
this.componentType = componentType;
return this;
}
@Override
public Builder setSourceSystemFlowFileIdentifier(String sourceSystemFlowFileIdentifier) {
this.sourceSystemFlowFileIdentifier = sourceSystemFlowFileIdentifier;
return this;
}
@Override
public Builder setTransitUri(String transitUri) {
this.transitUri = transitUri;
return this;
}
@Override
public Builder addParentFlowFile(final FlowFile parentFlowFile) {
if (this.parentUuids == null) {
this.parentUuids = new ArrayList<>();
}
this.parentUuids.add(parentFlowFile.getAttribute(CoreAttributes.UUID.key()));
return this;
}
public Builder addParentUuid(final String uuid) {
if (this.parentUuids == null) {
this.parentUuids = new ArrayList<>();
}
this.parentUuids.add(uuid);
return this;
}
@Override
public Builder removeParentFlowFile(final FlowFile parentFlowFile) {
if (this.parentUuids == null) {
return this;
}
parentUuids.remove(parentFlowFile.getAttribute(CoreAttributes.UUID.key()));
return this;
}
@Override
public Builder addChildFlowFile(final FlowFile childFlowFile) {
if (this.childrenUuids == null) {
this.childrenUuids = new ArrayList<>();
}
this.childrenUuids.add(childFlowFile.getAttribute(CoreAttributes.UUID.key()));
return this;
}
public Builder addChildUuid(final String uuid) {
if (this.childrenUuids == null) {
this.childrenUuids = new ArrayList<>();
}
this.childrenUuids.add(uuid);
return this;
}
@Override
public Builder removeChildFlowFile(final FlowFile childFlowFile) {
if (this.childrenUuids == null) {
return this;
}
childrenUuids.remove(childFlowFile.getAttribute(CoreAttributes.UUID.key()));
return this;
}
public Builder setContentType(String contentType) {
this.contentType = contentType;
return this;
}
@Override
public Builder setAlternateIdentifierUri(String alternateIdentifierUri) {
this.alternateIdentifierUri = alternateIdentifierUri;
return this;
}
@Override
public Builder setDetails(String details) {
this.details = details;
return this;
}
@Override
public Builder setRelationship(Relationship relationship) {
this.relationship = relationship.getName();
return this;
}
public Builder setRelationship(final String relationship) {
this.relationship = relationship;
return this;
}
@Override
public ProvenanceEventBuilder fromFlowFile(final FlowFile flowFile) {
setFlowFileEntryDate(flowFile.getEntryDate());
setLineageIdentifiers(flowFile.getLineageIdentifiers());
setLineageStartDate(flowFile.getLineageStartDate());
setAttributes(Collections.<String, String>emptyMap(), flowFile.getAttributes());
uuid = flowFile.getAttribute(CoreAttributes.UUID.key());
this.contentSize = flowFile.getSize();
return this;
}
@Override
public Builder setPreviousContentClaim(final String container, final String section, final String identifier, final Long offset, final long size) {
previousClaimSection = section;
previousClaimContainer = container;
previousClaimIdentifier = identifier;
previousClaimOffset = offset;
previousSize = size;
return this;
}
@Override
public Builder setCurrentContentClaim(final String container, final String section, final String identifier, final Long offset, final long size) {
contentClaimSection = section;
contentClaimContainer = container;
contentClaimIdentifier = identifier;
contentClaimOffset = offset;
contentSize = size;
return this;
}
@Override
public Builder setSourceQueueIdentifier(final String identifier) {
sourceQueueIdentifier = identifier;
return this;
}
private void assertSet(final Object value, final String name) {
if (value == null) {
throw new IllegalStateException("Cannot create Provenance Event Record because " + name + " is not set");
}
}
public ProvenanceEventType getEventType() {
return eventType;
}
public List<String> getChildUuids() {
return Collections.unmodifiableList(childrenUuids);
}
public List<String> getParentUuids() {
return Collections.unmodifiableList(parentUuids);
}
@Override
public StandardProvenanceEventRecord build() {
assertSet(eventType, "Event Type");
assertSet(componentId, "Component ID");
assertSet(componentType, "Component Type");
assertSet(uuid, "FlowFile UUID");
assertSet(contentSize, "FlowFile Size");
switch (eventType) {
case ADDINFO:
if (alternateIdentifierUri == null) {
throw new IllegalStateException("Cannot create Provenance Event Record of type " + eventType + " because no alternate identifiers have been set");
}
break;
case RECEIVE:
case SEND:
assertSet(transitUri, "Transit URI");
break;
case ROUTE:
assertSet(relationship, "Relationship");
break;
case CLONE:
case FORK:
case JOIN:
if ((parentUuids == null || parentUuids.isEmpty()) && (childrenUuids == null || childrenUuids.isEmpty())) {
throw new IllegalStateException("Cannot create Provenance Event Record of type " + eventType + " because no Parent UUIDs or Children UUIDs have been set");
}
break;
default:
break;
}
return new StandardProvenanceEventRecord(this);
}
}
}

168
commons/data-provenance-utils/src/main/java/org/apache/nifi/provenance/StandardQueryResult.java Normal file
View File

@ -0,0 +1,168 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.provenance;
import java.util.ArrayList;
import java.util.Collection;
import java.util.Date;
import java.util.List;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReadWriteLock;
import java.util.concurrent.locks.ReentrantReadWriteLock;
import org.apache.nifi.provenance.search.Query;
import org.apache.nifi.provenance.search.QueryResult;
public class StandardQueryResult implements QueryResult {
public static final int TTL = (int) TimeUnit.MILLISECONDS.convert(30, TimeUnit.MINUTES);
private final Query query;
private final long creationNanos;
private final int numSteps;
private final ReadWriteLock rwLock = new ReentrantReadWriteLock();
private final Lock readLock = rwLock.readLock();
private final Lock writeLock = rwLock.writeLock();
// guarded by writeLock
private final List<ProvenanceEventRecord> matchingRecords = new ArrayList<>();
private long totalHitCount;
private int numCompletedSteps = 0;
private Date expirationDate;
private String error;
private long queryTime;
private volatile boolean canceled = false;
public StandardQueryResult(final Query query, final int numSteps) {
this.query = query;
this.numSteps = numSteps;
this.creationNanos = System.nanoTime();
updateExpiration();
}
@Override
public List<ProvenanceEventRecord> getMatchingEvents() {
readLock.lock();
try {
if (matchingRecords.size() <= query.getMaxResults()) {
return new ArrayList<>(matchingRecords);
}
final List<ProvenanceEventRecord> copy = new ArrayList<>(query.getMaxResults());
for (int i = 0; i < query.getMaxResults(); i++) {
copy.add(matchingRecords.get(i));
}
return copy;
} finally {
readLock.unlock();
}
}
@Override
public long getTotalHitCount() {
readLock.lock();
try {
return totalHitCount;
} finally {
readLock.unlock();
}
}
@Override
public long getQueryTime() {
return queryTime;
}
@Override
public Date getExpiration() {
return expirationDate;
}
@Override
public String getError() {
return error;
}
@Override
public int getPercentComplete() {
readLock.lock();
try {
return (numSteps < 1) ? 100 : (int) (((float) numCompletedSteps / (float) numSteps) * 100.0F);
} finally {
readLock.unlock();
}
}
@Override
public boolean isFinished() {
readLock.lock();
try {
return numCompletedSteps >= numSteps || canceled;
} finally {
readLock.unlock();
}
}
void cancel() {
this.canceled = true;
}
public void setError(final String error) {
writeLock.lock();
try {
this.error = error;
numCompletedSteps++;
updateExpiration();
if (numCompletedSteps >= numSteps) {
final long searchNanos = System.nanoTime() - creationNanos;
queryTime = TimeUnit.MILLISECONDS.convert(searchNanos, TimeUnit.NANOSECONDS);
}
} finally {
writeLock.unlock();
}
}
public void update(final Collection<ProvenanceEventRecord> matchingRecords, final long totalHits) {
writeLock.lock();
try {
this.matchingRecords.addAll(matchingRecords);
this.totalHitCount += totalHits;
numCompletedSteps++;
updateExpiration();
if (numCompletedSteps >= numSteps) {
final long searchNanos = System.nanoTime() - creationNanos;
queryTime = TimeUnit.MILLISECONDS.convert(searchNanos, TimeUnit.NANOSECONDS);
}
} finally {
writeLock.unlock();
}
}
/**
* Must be called with write lock!
*/
private void updateExpiration() {
expirationDate = new Date(System.currentTimeMillis() + TTL);
}
}

74
commons/data-provenance-utils/src/main/java/org/apache/nifi/provenance/lineage/EdgeNode.java Normal file
View File

@ -0,0 +1,74 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.provenance.lineage;
import static java.util.Objects.requireNonNull;
public class EdgeNode implements LineageEdge {
private final String uuid;
private final LineageNode source;
private final LineageNode destination;
public EdgeNode(final String uuid, final LineageNode source, final LineageNode destination) {
this.uuid = uuid;
this.source = requireNonNull(source);
this.destination = requireNonNull(destination);
}
@Override
public String getUuid() {
return uuid;
}
@Override
public LineageNode getSource() {
return source;
}
@Override
public LineageNode getDestination() {
return destination;
}
@Override
public int hashCode() {
return 43298293 + source.hashCode() + destination.hashCode();
}
@Override
public boolean equals(final Object obj) {
if (this == obj) {
return true;
}
if (obj == null) {
return false;
}
if (!(obj instanceof EdgeNode)) {
return false;
}
final EdgeNode other = (EdgeNode) obj;
return (source.equals(other.source) && destination.equals(other.destination));
}
@Override
public String toString() {
return "Edge[Source=" + source + ", Destination=" + destination + "]";
}
}

109
commons/data-provenance-utils/src/main/java/org/apache/nifi/provenance/lineage/EventNode.java Normal file
View File

@ -0,0 +1,109 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.provenance.lineage;
import java.util.List;
import org.apache.nifi.flowfile.attributes.CoreAttributes;
import org.apache.nifi.provenance.ProvenanceEventRecord;
import org.apache.nifi.provenance.ProvenanceEventType;
public class EventNode implements ProvenanceEventLineageNode {
private final ProvenanceEventRecord record;
private String clusterNodeIdentifier = null;
public EventNode(final ProvenanceEventRecord event) {
this.record = event;
}
@Override
public String getIdentifier() {
return String.valueOf(getEventIdentifier());
}
@Override
public String getClusterNodeIdentifier() {
return clusterNodeIdentifier;
}
public void setClusterNodeIdentifier(final String nodeIdentifier) {
this.clusterNodeIdentifier = nodeIdentifier;
}
@Override
public LineageNodeType getNodeType() {
return LineageNodeType.PROVENANCE_EVENT_NODE;
}
@Override
public ProvenanceEventType getEventType() {
return record.getEventType();
}
@Override
public long getTimestamp() {
return record.getEventTime();
}
@Override
public long getEventIdentifier() {
return record.getEventId();
}
@Override
public String getFlowFileUuid() {
return record.getAttributes().get(CoreAttributes.UUID.key());
}
@Override
public List<String> getParentUuids() {
return record.getParentUuids();
}
@Override
public List<String> getChildUuids() {
return record.getChildUuids();
}
@Override
public int hashCode() {
return 2938472 + record.hashCode();
}
@Override
public boolean equals(final Object obj) {
if (obj == null) {
return false;
}
if (this == obj) {
return true;
}
if (!(obj instanceof EventNode)) {
return false;
}
final EventNode other = (EventNode) obj;
return record.equals(other.record);
}
@Override
public String toString() {
return "Event[ID=" + record.getEventId() + ", Type=" + record.getEventType() + ", UUID=" + record.getFlowFileUuid() + ", Component=" + record.getComponentId() + "]";
}
}

76
commons/data-provenance-utils/src/main/java/org/apache/nifi/provenance/lineage/FlowFileLineage.java Normal file
View File

@ -0,0 +1,76 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.provenance.lineage;
import java.util.ArrayList;
import java.util.Collection;
import java.util.List;
import static java.util.Objects.requireNonNull;
public class FlowFileLineage implements Lineage {
private final List<LineageNode> nodes;
private final List<LineageEdge> edges;
public FlowFileLineage(final Collection<LineageNode> nodes, final Collection<LineageEdge> edges) {
this.nodes = new ArrayList<>(requireNonNull(nodes));
this.edges = new ArrayList<>(requireNonNull(edges));
}
@Override
public List<LineageNode> getNodes() {
return nodes;
}
@Override
public List<LineageEdge> getEdges() {
return edges;
}
@Override
public int hashCode() {
int sum = 923;
for (final LineageNode node : nodes) {
sum += node.hashCode();
}
for (final LineageEdge edge : edges) {
sum += edge.hashCode();
}
return sum;
}
@Override
public boolean equals(final Object obj) {
if (obj == null) {
return false;
}
if (obj == this) {
return true;
}
if (!(obj instanceof FlowFileLineage)) {
return false;
}
final FlowFileLineage other = (FlowFileLineage) obj;
return nodes.equals(other.nodes) && edges.equals(other.edges);
}
}

83
commons/data-provenance-utils/src/main/java/org/apache/nifi/provenance/lineage/FlowFileNode.java Normal file
View File

@ -0,0 +1,83 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.provenance.lineage;
import static java.util.Objects.requireNonNull;
public class FlowFileNode implements LineageNode {
private final String flowFileUuid;
private final long creationTime;
private String clusterNodeIdentifier;
public FlowFileNode(final String flowFileUuid, final long flowFileCreationTime) {
this.flowFileUuid = requireNonNull(flowFileUuid);
this.creationTime = flowFileCreationTime;
}
@Override
public String getIdentifier() {
return flowFileUuid;
}
@Override
public long getTimestamp() {
return creationTime;
}
@Override
public String getClusterNodeIdentifier() {
return clusterNodeIdentifier;
}
@Override
public LineageNodeType getNodeType() {
return LineageNodeType.FLOWFILE_NODE;
}
@Override
public String getFlowFileUuid() {
return flowFileUuid;
}
@Override
public int hashCode() {
return 23498723 + flowFileUuid.hashCode();
}
@Override
public boolean equals(final Object obj) {
if (obj == null) {
return false;
}
if (obj == this) {
return true;
}
if (!(obj instanceof FlowFileNode)) {
return false;
}
final FlowFileNode other = (FlowFileNode) obj;
return flowFileUuid.equals(other.flowFileUuid);
}
@Override
public String toString() {
return "FlowFile[UUID=" + flowFileUuid + "]";
}
}

43
commons/flowfile-packager/pom.xml Normal file
View File

@ -0,0 +1,43 @@
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-parent</artifactId>
<version>0.0.1-SNAPSHOT</version>
</parent>
<artifactId>flowfile-packager</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
<name>FlowFile Packager</name>
<dependencies>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-compress</artifactId>
<version>1.9</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>3.3.2</version>
</dependency>
</dependencies>
</project>

28
commons/flowfile-packager/src/main/java/org/apache/nifi/util/FlowFilePackager.java Normal file
View File

@ -0,0 +1,28 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.util;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.util.Map;
public interface FlowFilePackager {
void packageFlowFile(InputStream in, OutputStream out, Map<String, String> attributes, long fileSize) throws IOException;
}

104
commons/flowfile-packager/src/main/java/org/apache/nifi/util/FlowFilePackagerV1.java Normal file
View File

@ -0,0 +1,104 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.util;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.nio.charset.StandardCharsets;
import java.util.Map;
import org.apache.commons.compress.archivers.tar.TarArchiveEntry;
import org.apache.commons.compress.archivers.tar.TarArchiveOutputStream;
import org.apache.commons.lang3.StringEscapeUtils;
public class FlowFilePackagerV1 implements FlowFilePackager {
public static final String FILENAME_ATTRIBUTES = "flowfile.attributes";
public static final String FILENAME_CONTENT = "flowfile.content";
public static final int DEFAULT_TAR_PERMISSIONS = 0644;
private final int tarPermissions;
public FlowFilePackagerV1() {
this(DEFAULT_TAR_PERMISSIONS);
}
public FlowFilePackagerV1(final int tarPermissions) {
this.tarPermissions = tarPermissions;
}
@Override
public void packageFlowFile(final InputStream in, final OutputStream out, final Map<String, String> attributes, final long fileSize) throws IOException {
try (final TarArchiveOutputStream tout = new TarArchiveOutputStream(out)) {
writeAttributesEntry(attributes, tout);
writeContentEntry(tout, in, fileSize);
tout.finish();
tout.flush();
tout.close();
}
}
private void writeAttributesEntry(final Map<String, String> attributes, final TarArchiveOutputStream tout) throws IOException {
final StringBuilder sb = new StringBuilder();
sb.append("<?xml version=\"1.0\" encoding=\"UTF-8\"?><!DOCTYPE properties\n SYSTEM \"http://java.sun.com/dtd/properties.dtd\">\n");
sb.append("<properties>");
for (final Map.Entry<String, String> entry : attributes.entrySet()) {
final String escapedKey = StringEscapeUtils.escapeXml(entry.getKey());
final String escapedValue = StringEscapeUtils.escapeXml(entry.getValue());
sb.append("\n <entry key=\"").append(escapedKey).append("\">").append(escapedValue).append("</entry>");
}
sb.append("</properties>");
final byte[] metaBytes = sb.toString().getBytes(StandardCharsets.UTF_8);
final TarArchiveEntry attribEntry = new TarArchiveEntry(FILENAME_ATTRIBUTES);
attribEntry.setMode(tarPermissions);
attribEntry.setSize(metaBytes.length);
tout.putArchiveEntry(attribEntry);
tout.write(metaBytes);
tout.closeArchiveEntry();
}
private void writeContentEntry(final TarArchiveOutputStream tarOut, final InputStream inStream, final long fileSize) throws IOException {
final TarArchiveEntry entry = new TarArchiveEntry(FILENAME_CONTENT);
entry.setMode(tarPermissions);
entry.setSize(fileSize);
tarOut.putArchiveEntry(entry);
final byte[] buffer = new byte[512 << 10];//512KB
int bytesRead = 0;
while ((bytesRead = inStream.read(buffer)) != -1) { //still more data to read
if (bytesRead > 0) {
tarOut.write(buffer, 0, bytesRead);
}
}
copy(inStream, tarOut);
tarOut.closeArchiveEntry();
}
public static long copy(final InputStream source, final OutputStream destination) throws IOException {
final byte[] buffer = new byte[8192];
int len;
long totalCount = 0L;
while ((len = source.read(buffer)) > 0) {
destination.write(buffer, 0, len);
totalCount += len;
}
return totalCount;
}
}

146
commons/flowfile-packager/src/main/java/org/apache/nifi/util/FlowFilePackagerV2.java Normal file
View File

@ -0,0 +1,146 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.util;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.util.Map;
/**
* <p>
* Packages a FlowFile, including both its content and its attributes into a
* single file that is stream-friendly. The encoding scheme is as such:
* </p>
*
* <pre>
* Length Field : indicates the number of Flow File Attributes in the stream
* 1 to N times (N=number of Flow File Attributes):
* String Field : Flow File Attribute key name
* String Field : Flow File Attribute value
* Long : 8 bytes indicating the length of the Flow File content
* Content : The next M bytes are the content of the Flow File.
* </pre>
*
* <pre>
* Encoding of String Field is as follows:
* Length Field : indicates the length of the String
* 1 to N bytes (N=String length, determined by previous field, as described above) : The UTF-8 encoded string value.
* </pre>
*
* <pre>
* Encoding of Length Field is as follows:
* First 2 bytes: Indicate length. If both bytes = 255, this is a special value indicating that the length is
* greater than or equal to 65536 bytes; therefore, the next 4 bytes will indicate the actual length.
* </pre>
*
* <p>
* Note: All byte-order encoding is Network Byte Order (Most Significant Byte
* first)
* </p>
*
* <p>
* The following example shows the bytes expected if we were to encode a
* FlowFile containing the following attributes where the content is the text
* "Hello World!":
*
* <br><br>
* Attributes:
* <pre>
* +-------+-------+
* | Key + Value |
* + --------------+
* | A | a |
* + --------------+
* | B | b |
* + --------------+
* </pre> Content:<br>
* Hello World!
* <br><br>
* Packaged Byte Encoding (In Hexadecimal Form):
* <p>
*
* <pre>
* 00 02 00 01 41 00 01 61
* 00 01 42 00 01 62 00 00
* 00 00 00 00 00 0C 48 65
* 6C 6C 6F 20 57 6F 72 6C
* 64 21
* </pre>
*/
public class FlowFilePackagerV2 implements FlowFilePackager {
private static final int MAX_VALUE_2_BYTES = 65535;
private final byte[] writeBuffer = new byte[8];
@Override
public void packageFlowFile(final InputStream in, final OutputStream out, final Map<String, String> attributes, final long fileSize) throws IOException {
writeFieldLength(out, attributes.size()); //write out the number of attributes
for (final Map.Entry<String, String> entry : attributes.entrySet()) { //write out each attribute key/value pair
writeString(entry.getKey(), out);
writeString(entry.getValue(), out);
}
writeLong(out, fileSize);//write out length of data
copy(in, out);//write out the actual flow file payload
}
private void copy(final InputStream in, final OutputStream out) throws IOException {
final byte[] buffer = new byte[65536];
int len;
while ((len = in.read(buffer)) > 0) {
out.write(buffer, 0, len);
}
}
private void writeString(final String val, final OutputStream out) throws IOException {
final byte[] bytes = val.getBytes("UTF-8");
writeFieldLength(out, bytes.length);
out.write(bytes);
}
private void writeFieldLength(final OutputStream out, final int numBytes) throws IOException {
// If the value is less than the max value that can be fit into 2 bytes, just use the
// actual value. Otherwise, we will set the first 2 bytes to 255/255 and then use the next
// 4 bytes to indicate the real length.
if (numBytes < MAX_VALUE_2_BYTES) {
writeBuffer[0] = (byte) (numBytes >>> 8);
writeBuffer[1] = (byte) (numBytes);
out.write(writeBuffer, 0, 2);
} else {
writeBuffer[0] = (byte) 0xff;
writeBuffer[1] = (byte) 0xff;
writeBuffer[2] = (byte) (numBytes >>> 24);
writeBuffer[3] = (byte) (numBytes >>> 16);
writeBuffer[4] = (byte) (numBytes >>> 8);
writeBuffer[5] = (byte) (numBytes);
out.write(writeBuffer, 0, 6);
}
}
private void writeLong(final OutputStream out, final long val) throws IOException {
writeBuffer[0] = (byte) (val >>> 56);
writeBuffer[1] = (byte) (val >>> 48);
writeBuffer[2] = (byte) (val >>> 40);
writeBuffer[3] = (byte) (val >>> 32);
writeBuffer[4] = (byte) (val >>> 24);
writeBuffer[5] = (byte) (val >>> 16);
writeBuffer[6] = (byte) (val >>> 8);
writeBuffer[7] = (byte) (val);
out.write(writeBuffer, 0, 8);
}
}

93
commons/flowfile-packager/src/main/java/org/apache/nifi/util/FlowFilePackagerV3.java Normal file
View File

@ -0,0 +1,93 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.util;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.util.Map;
public class FlowFilePackagerV3 implements FlowFilePackager {
public static final byte[] MAGIC_HEADER = {'N', 'i', 'F', 'i', 'F', 'F', '3'};
private static final int MAX_VALUE_2_BYTES = 65535;
private final byte[] writeBuffer = new byte[8];
@Override
public void packageFlowFile(final InputStream in, final OutputStream out, final Map<String, String> attributes, final long fileSize) throws IOException {
out.write(MAGIC_HEADER);
if (attributes == null) {
writeFieldLength(out, 0);
} else {
writeFieldLength(out, attributes.size()); //write out the number of attributes
for (final Map.Entry<String, String> entry : attributes.entrySet()) { //write out each attribute key/value pair
writeString(entry.getKey(), out);
writeString(entry.getValue(), out);
}
}
writeLong(out, fileSize);//write out length of data
copy(in, out);//write out the actual flow file payload
}
private void copy(final InputStream in, final OutputStream out) throws IOException {
final byte[] buffer = new byte[65536];
int len;
while ((len = in.read(buffer)) > 0) {
out.write(buffer, 0, len);
}
}
private void writeString(final String val, final OutputStream out) throws IOException {
final byte[] bytes = val.getBytes("UTF-8");
writeFieldLength(out, bytes.length);
out.write(bytes);
}
private void writeFieldLength(final OutputStream out, final int numBytes) throws IOException {
// If the value is less than the max value that can be fit into 2 bytes, just use the
// actual value. Otherwise, we will set the first 2 bytes to 255/255 and then use the next
// 4 bytes to indicate the real length.
if (numBytes < MAX_VALUE_2_BYTES) {
writeBuffer[0] = (byte) (numBytes >>> 8);
writeBuffer[1] = (byte) (numBytes);
out.write(writeBuffer, 0, 2);
} else {
writeBuffer[0] = (byte) 0xff;
writeBuffer[1] = (byte) 0xff;
writeBuffer[2] = (byte) (numBytes >>> 24);
writeBuffer[3] = (byte) (numBytes >>> 16);
writeBuffer[4] = (byte) (numBytes >>> 8);
writeBuffer[5] = (byte) (numBytes);
out.write(writeBuffer, 0, 6);
}
}
private void writeLong(final OutputStream out, final long val) throws IOException {
writeBuffer[0] = (byte) (val >>> 56);
writeBuffer[1] = (byte) (val >>> 48);
writeBuffer[2] = (byte) (val >>> 40);
writeBuffer[3] = (byte) (val >>> 32);
writeBuffer[4] = (byte) (val >>> 24);
writeBuffer[5] = (byte) (val >>> 16);
writeBuffer[6] = (byte) (val >>> 8);
writeBuffer[7] = (byte) (val);
out.write(writeBuffer, 0, 8);
}
}

30
commons/flowfile-packager/src/main/java/org/apache/nifi/util/FlowFileUnpackager.java Normal file
View File

@ -0,0 +1,30 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.util;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.util.Map;
public interface FlowFileUnpackager {
Map<String, String> unpackageFlowFile(InputStream in, OutputStream out) throws IOException;
boolean hasMoreData() throws IOException;
}

155
commons/flowfile-packager/src/main/java/org/apache/nifi/util/FlowFileUnpackagerV1.java Normal file
View File

@ -0,0 +1,155 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.util;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.util.HashMap;
import java.util.Map;
import java.util.Map.Entry;
import java.util.Properties;
import org.apache.commons.compress.archivers.tar.TarArchiveEntry;
import org.apache.commons.compress.archivers.tar.TarArchiveInputStream;
public class FlowFileUnpackagerV1 implements FlowFileUnpackager {
private int flowFilesRead = 0;
@Override
public Map<String, String> unpackageFlowFile(final InputStream in, final OutputStream out) throws IOException {
flowFilesRead++;
final TarArchiveInputStream tarIn = new TarArchiveInputStream(in);
final TarArchiveEntry attribEntry = tarIn.getNextTarEntry();
if (attribEntry == null) {
return null;
}
final Map<String, String> attributes;
if (attribEntry.getName().equals(FlowFilePackagerV1.FILENAME_ATTRIBUTES)) {
attributes = getAttributes(tarIn);
} else {
throw new IOException("Expected two tar entries: "
+ FlowFilePackagerV1.FILENAME_CONTENT + " and "
+ FlowFilePackagerV1.FILENAME_ATTRIBUTES);
}
final TarArchiveEntry contentEntry = tarIn.getNextTarEntry();
if (contentEntry != null && contentEntry.getName().equals(FlowFilePackagerV1.FILENAME_CONTENT)) {
final byte[] buffer = new byte[512 << 10];//512KB
int bytesRead = 0;
while ((bytesRead = tarIn.read(buffer)) != -1) { //still more data to read
if (bytesRead > 0) {
out.write(buffer, 0, bytesRead);
}
}
out.flush();
} else {
throw new IOException("Expected two tar entries: "
+ FlowFilePackagerV1.FILENAME_CONTENT + " and "
+ FlowFilePackagerV1.FILENAME_ATTRIBUTES);
}
return attributes;
}
protected Map<String, String> getAttributes(final TarArchiveInputStream stream) throws IOException {
final Properties props = new Properties();
props.loadFromXML(new NonCloseableInputStream(stream));
final Map<String, String> result = new HashMap<>();
for (final Entry<Object, Object> entry : props.entrySet()) {
final Object keyObject = entry.getKey();
final Object valueObject = entry.getValue();
if (!(keyObject instanceof String)) {
throw new IOException("Flow file attributes object contains key of type "
+ keyObject.getClass().getCanonicalName()
+ " but expected java.lang.String");
} else if (!(keyObject instanceof String)) {
throw new IOException("Flow file attributes object contains value of type "
+ keyObject.getClass().getCanonicalName()
+ " but expected java.lang.String");
}
final String key = (String) keyObject;
final String value = (String) valueObject;
result.put(key, value);
}
return result;
}
@Override
public boolean hasMoreData() throws IOException {
return flowFilesRead == 0;
}
public static final class NonCloseableInputStream extends InputStream {
final InputStream stream;
public NonCloseableInputStream(final InputStream stream) {
this.stream = stream;
}
@Override
public void close() {
}
@Override
public int read() throws IOException {
return stream.read();
}
@Override
public int available() throws IOException {
return stream.available();
}
@Override
public synchronized void mark(int readlimit) {
stream.mark(readlimit);
}
@Override
public synchronized void reset() throws IOException {
stream.reset();
}
@Override
public boolean markSupported() {
return stream.markSupported();
}
@Override
public long skip(long n) throws IOException {
return stream.skip(n);
}
@Override
public int read(byte b[], int off, int len) throws IOException {
return stream.read(b, off, len);
}
@Override
public int read(byte b[]) throws IOException {
return stream.read(b);
}
}
}

143
commons/flowfile-packager/src/main/java/org/apache/nifi/util/FlowFileUnpackagerV2.java Normal file
View File

@ -0,0 +1,143 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.util;
import java.io.EOFException;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.util.HashMap;
import java.util.Map;
public class FlowFileUnpackagerV2 implements FlowFileUnpackager {
private final byte readBuffer[] = new byte[8192];
private Map<String, String> nextAttributes = null;
private boolean haveReadSomething = false;
@Override
public boolean hasMoreData() throws IOException {
return nextAttributes != null || !haveReadSomething;
}
protected Map<String, String> readAttributes(final InputStream in) throws IOException {
final Map<String, String> attributes = new HashMap<>();
final Integer numAttributes = readFieldLength(in); //read number of attributes
if (numAttributes == null) {
return null;
}
if (numAttributes == 0) {
throw new IOException("flow files cannot have zero attributes");
}
for (int i = 0; i < numAttributes; i++) { //read each attribute key/value pair
final String key = readString(in);
final String value = readString(in);
attributes.put(key, value);
}
return attributes;
}
@Override
public Map<String, String> unpackageFlowFile(final InputStream in, final OutputStream out) throws IOException {
final Map<String, String> attributes;
if (nextAttributes != null) {
attributes = nextAttributes;
} else {
attributes = readAttributes(in);
}
final long expectedNumBytes = readLong(in); // read length of payload
copy(in, out, expectedNumBytes); // read payload
nextAttributes = readAttributes(in);
haveReadSomething = true;
return attributes;
}
protected String readString(final InputStream in) throws IOException {
final Integer numBytes = readFieldLength(in);
if (numBytes == null) {
throw new EOFException();
}
final byte[] bytes = new byte[numBytes];
fillBuffer(in, bytes, numBytes);
return new String(bytes, "UTF-8");
}
private void fillBuffer(final InputStream in, final byte[] buffer, final int length) throws IOException {
int bytesRead;
int totalBytesRead = 0;
while ((bytesRead = in.read(buffer, totalBytesRead, length - totalBytesRead)) > 0) {
totalBytesRead += bytesRead;
}
if (totalBytesRead != length) {
throw new EOFException();
}
}
protected long copy(final InputStream in, final OutputStream out, final long numBytes) throws IOException {
int bytesRead;
long totalBytesRead = 0L;
while ((bytesRead = in.read(readBuffer, 0, (int) Math.min(readBuffer.length, numBytes - totalBytesRead))) > 0) {
out.write(readBuffer, 0, bytesRead);
totalBytesRead += bytesRead;
}
if (totalBytesRead < numBytes) {
throw new EOFException("Expected " + numBytes + " but received " + totalBytesRead);
}
return totalBytesRead;
}
protected long readLong(final InputStream in) throws IOException {
fillBuffer(in, readBuffer, 8);
return (((long) readBuffer[0] << 56)
+ ((long) (readBuffer[1] & 255) << 48)
+ ((long) (readBuffer[2] & 255) << 40)
+ ((long) (readBuffer[3] & 255) << 32)
+ ((long) (readBuffer[4] & 255) << 24)
+ ((readBuffer[5] & 255) << 16)
+ ((readBuffer[6] & 255) << 8)
+ ((readBuffer[7] & 255)));
}
private Integer readFieldLength(final InputStream in) throws IOException {
final int firstValue = in.read();
final int secondValue = in.read();
if (firstValue < 0) {
return null;
}
if (secondValue < 0) {
throw new EOFException();
}
if (firstValue == 0xff && secondValue == 0xff) {
int ch1 = in.read();
int ch2 = in.read();
int ch3 = in.read();
int ch4 = in.read();
if ((ch1 | ch2 | ch3 | ch4) < 0) {
throw new EOFException();
}
return ((ch1 << 24) + (ch2 << 16) + (ch3 << 8) + (ch4));
} else {
return ((firstValue << 8) + (secondValue));
}
}
}

161
commons/flowfile-packager/src/main/java/org/apache/nifi/util/FlowFileUnpackagerV3.java Normal file
View File

@ -0,0 +1,161 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.util;
import java.io.EOFException;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.util.Arrays;
import java.util.HashMap;
import java.util.Map;
public class FlowFileUnpackagerV3 implements FlowFileUnpackager {
private byte[] nextHeader = null;
private boolean haveReadSomething = false;
private final byte readBuffer[] = new byte[8192];
@Override
public boolean hasMoreData() throws IOException {
return nextHeader != null || !haveReadSomething;
}
private byte[] readHeader(final InputStream in) throws IOException {
final byte[] header = new byte[FlowFilePackagerV3.MAGIC_HEADER.length];
for (int i = 0; i < header.length; i++) {
final int next = in.read();
if (next < 0) {
if (i == 0) {
return null;
}
throw new IOException("Not in FlowFile-v3 format");
}
header[i] = (byte) (next & 0xFF);
}
return header;
}
@Override
public Map<String, String> unpackageFlowFile(final InputStream in, final OutputStream out) throws IOException {
final byte[] header = (nextHeader == null) ? readHeader(in) : nextHeader;
if (!Arrays.equals(header, FlowFilePackagerV3.MAGIC_HEADER)) {
throw new IOException("Not in FlowFile-v3 format");
}
final Map<String, String> attributes = readAttributes(in);
final long expectedNumBytes = readLong(in); // read length of payload
copy(in, out, expectedNumBytes); // read payload
nextHeader = readHeader(in);
haveReadSomething = true;
return attributes;
}
protected Map<String, String> readAttributes(final InputStream in) throws IOException {
final Map<String, String> attributes = new HashMap<>();
final Integer numAttributes = readFieldLength(in); //read number of attributes
if (numAttributes == null) {
return null;
}
if (numAttributes == 0) {
throw new IOException("flow files cannot have zero attributes");
}
for (int i = 0; i < numAttributes; i++) { //read each attribute key/value pair
final String key = readString(in);
final String value = readString(in);
attributes.put(key, value);
}
return attributes;
}
protected String readString(final InputStream in) throws IOException {
final Integer numBytes = readFieldLength(in);
if (numBytes == null) {
throw new EOFException();
}
final byte[] bytes = new byte[numBytes];
fillBuffer(in, bytes, numBytes);
return new String(bytes, "UTF-8");
}
private void fillBuffer(final InputStream in, final byte[] buffer, final int length) throws IOException {
int bytesRead;
int totalBytesRead = 0;
while ((bytesRead = in.read(buffer, totalBytesRead, length - totalBytesRead)) > 0) {
totalBytesRead += bytesRead;
}
if (totalBytesRead != length) {
throw new EOFException();
}
}
protected long copy(final InputStream in, final OutputStream out, final long numBytes) throws IOException {
int bytesRead;
long totalBytesRead = 0L;
while ((bytesRead = in.read(readBuffer, 0, (int) Math.min(readBuffer.length, numBytes - totalBytesRead))) > 0) {
out.write(readBuffer, 0, bytesRead);
totalBytesRead += bytesRead;
}
if (totalBytesRead < numBytes) {
throw new EOFException("Expected " + numBytes + " but received " + totalBytesRead);
}
return totalBytesRead;
}
protected long readLong(final InputStream in) throws IOException {
fillBuffer(in, readBuffer, 8);
return (((long) readBuffer[0] << 56)
+ ((long) (readBuffer[1] & 255) << 48)
+ ((long) (readBuffer[2] & 255) << 40)
+ ((long) (readBuffer[3] & 255) << 32)
+ ((long) (readBuffer[4] & 255) << 24)
+ ((readBuffer[5] & 255) << 16)
+ ((readBuffer[6] & 255) << 8)
+ ((readBuffer[7] & 255)));
}
private Integer readFieldLength(final InputStream in) throws IOException {
final int firstValue = in.read();
final int secondValue = in.read();
if (firstValue < 0) {
return null;
}
if (secondValue < 0) {
throw new EOFException();
}
if (firstValue == 0xff && secondValue == 0xff) {
int ch1 = in.read();
int ch2 = in.read();
int ch3 = in.read();
int ch4 = in.read();
if ((ch1 | ch2 | ch3 | ch4) < 0) {
throw new EOFException();
}
return ((ch1 << 24) + (ch2 << 16) + (ch3 << 8) + (ch4));
} else {
return ((firstValue << 8) + (secondValue));
}
}
}

119
commons/flowfile-packager/src/main/java/org/apache/nifi/util/Unpackage.java Normal file
View File

@ -0,0 +1,119 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.util;
import java.io.BufferedInputStream;
import java.io.BufferedOutputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.util.Map;
import java.util.UUID;
public class Unpackage {
private static void printUsage() {
System.out.println("java " + Unpackage.class.getCanonicalName() + " <version> <input file 1> [<input file 2> <input file 3> ... <input file N>]");
System.out.println("<version> : The version of the FlowFile Package format. Valid values are 1, 2, 3");
System.out.println("<input file X> : The FlowFile package to unpack");
System.out.println();
}
public static void main(final String[] args) throws IOException {
if (args.length < 2) {
printUsage();
return;
}
final String version = args[0];
int inputFileCount = 0;
int outputFileCount = 0;
for (int i = 1; i < args.length; i++) {
final String filename = args[i];
final File inFile = new File(filename);
if (inFile.isDirectory()) {
System.out.println("WARNING: input file " + inFile + " is a directory; skipping");
continue;
}
if (!inFile.exists() || !inFile.canRead()) {
System.out.println("ERROR: unable to read file " + inFile);
continue;
}
final File outputDir = new File(inFile.getAbsolutePath() + ".unpacked");
if (!outputDir.exists() && !outputDir.mkdir()) {
System.out.println("ERROR: Unable to create directory " + outputDir);
continue;
}
final File tempFile = new File(outputDir, ".temp." + UUID.randomUUID().toString() + ".unpackage");
inputFileCount++;
try (final FileInputStream fis = new FileInputStream(inFile);
final BufferedInputStream bufferedIn = new BufferedInputStream(fis)) {
final FlowFileUnpackager unpackager = createUnpackager(version);
while (unpackager.hasMoreData()) {
outputFileCount++;
final Map<String, String> attributes;
try (final FileOutputStream fos = new FileOutputStream(tempFile);
final BufferedOutputStream bufferedOut = new BufferedOutputStream(fos)) {
attributes = unpackager.unpackageFlowFile(bufferedIn, bufferedOut);
}
String outputFilename = attributes.get("filename");
if (outputFilename == null) {
outputFilename = attributes.get("nf.file.name");
}
final File outputFile = new File(outputDir, outputFilename);
tempFile.renameTo(outputFile);
final File attributeFilename = new File(outputDir, outputFilename + ".attributes");
try (final FileOutputStream fos = new FileOutputStream(attributeFilename);
final BufferedOutputStream bufferedOut = new BufferedOutputStream(fos)) {
for (final Map.Entry<String, String> entry : attributes.entrySet()) {
bufferedOut.write((entry.getKey() + "=" + entry.getValue() + "\n").getBytes("UTF-8"));
}
}
}
}
}
System.out.println("Unpacked " + inputFileCount + " packages into " + outputFileCount + " files");
}
public static FlowFileUnpackager createUnpackager(final String version) {
switch (version) {
case "1":
return new FlowFileUnpackagerV1();
case "2":
return new FlowFileUnpackagerV2();
case "3":
return new FlowFileUnpackagerV3();
default:
System.out.println("ERROR: Invalid version: " + version + "; must be 1, 2, or 3");
return null;
}
}
}

56
commons/flowfile-packager/src/test/java/org/apache/nifi/util/TestPackageUnpackageV3.java Normal file
View File

@ -0,0 +1,56 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.util;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.util.Arrays;
import java.util.HashMap;
import java.util.Map;
import org.junit.Test;
public class TestPackageUnpackageV3 {
@Test
public void test() throws IOException {
final FlowFilePackager packager = new FlowFilePackagerV3();
final FlowFileUnpackager unpackager = new FlowFileUnpackagerV3();
final byte[] data = "Hello, World!".getBytes("UTF-8");
final Map<String, String> map = new HashMap<>();
map.put("abc", "cba");
final ByteArrayOutputStream baos = new ByteArrayOutputStream();
final ByteArrayInputStream in = new ByteArrayInputStream(data);
packager.packageFlowFile(in, baos, map, data.length);
final byte[] encoded = baos.toByteArray();
final ByteArrayInputStream encodedIn = new ByteArrayInputStream(encoded);
final ByteArrayOutputStream decodedOut = new ByteArrayOutputStream();
final Map<String, String> unpackagedAttributes = unpackager.unpackageFlowFile(encodedIn, decodedOut);
final byte[] decoded = decodedOut.toByteArray();
assertEquals(map, unpackagedAttributes);
assertTrue(Arrays.equals(data, decoded));
}
}

30
commons/naive-search-ring-buffer/pom.xml Normal file
View File

@ -0,0 +1,30 @@
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-parent</artifactId>
<version>0.0.1-SNAPSHOT</version>
</parent>
<artifactId>naive-search-ring-buffer</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
<name>NiFi Ring Buffer</name>
</project>

135
commons/naive-search-ring-buffer/src/main/java/org/apache/nifi/util/NaiveSearchRingBuffer.java Normal file
View File

@ -0,0 +1,135 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.util;
import java.util.Arrays;
/**
* <p>
* A RingBuffer that can be used to scan byte sequences for subsequences.
* </p>
*
* <p>
* This class implements an efficient naive search algorithm, which allows the
* user of the library to identify byte sequences in a stream on-the-fly so that
* the stream can be segmented without having to buffer the data.
* </p>
*
* <p>
* The intended usage paradigm is:
* <code>
* <pre>
* final byte[] searchSequence = ...;
* final CircularBuffer buffer = new CircularBuffer(searchSequence);
* while ((int nextByte = in.read()) > 0) {
* if ( buffer.addAndCompare(nextByte) ) {
* // This byte is the last byte in the given sequence
* } else {
* // This byte does not complete the given sequence
* }
* }
* </pre>
* </code>
* </p>
*/
public class NaiveSearchRingBuffer {
private final byte[] lookingFor;
private final int[] buffer;
private int insertionPointer = 0;
private int bufferSize = 0;
public NaiveSearchRingBuffer(final byte[] lookingFor) {
this.lookingFor = lookingFor;
this.buffer = new int[lookingFor.length];
Arrays.fill(buffer, -1);
}
/**
* Returns the contents of the internal buffer, which represents the last X
* bytes added to the buffer, where X is the minimum of the number of bytes
* added to the buffer or the length of the byte sequence for which we are
* looking
*
* @return
*/
public byte[] getBufferContents() {
final int contentLength = Math.min(lookingFor.length, bufferSize);
final byte[] contents = new byte[contentLength];
for (int i = 0; i < contentLength; i++) {
final byte nextByte = (byte) buffer[(insertionPointer + i) % lookingFor.length];
contents[i] = nextByte;
}
return contents;
}
/**
* Returns the oldest byte in the buffer
*
* @return
*/
public int getOldestByte() {
return buffer[insertionPointer];
}
/**
* Returns <code>true</code> if the number of bytes that have been added to
* the buffer is at least equal to the length of the byte sequence for which
* we are searching
*
* @return
*/
public boolean isFilled() {
return bufferSize >= buffer.length;
}
/**
* Clears the internal buffer so that a new search may begin
*/
public void clear() {
Arrays.fill(buffer, -1);
insertionPointer = 0;
bufferSize = 0;
}
/**
* Add the given byte to the buffer and notify whether or not the byte
* completes the desired byte sequence.
*
* @param data
* @return <code>true</code> if this byte completes the byte sequence,
* <code>false</code> otherwise.
*/
public boolean addAndCompare(final byte data) {
buffer[insertionPointer] = data;
insertionPointer = (insertionPointer + 1) % lookingFor.length;
bufferSize++;
if (bufferSize < lookingFor.length) {
return false;
}
for (int i = 0; i < lookingFor.length; i++) {
final byte compare = (byte) buffer[(insertionPointer + i) % lookingFor.length];
if (compare != lookingFor[i]) {
return false;
}
}
return true;
}
}

72
commons/naive-search-ring-buffer/src/test/java/org/apache/nifi/util/TestNaiveSearchRingBuffer.java Normal file
View File

@ -0,0 +1,72 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.util;
import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertTrue;
import org.junit.Test;
public class TestNaiveSearchRingBuffer {
@Test
public void testAddAndCompare() {
final byte[] pattern = new byte[]{
'\r', '0', 38, 48
};
final byte[] search = new byte[]{
'\r', '0', 38, 58, 58, 83, 78, '\r', '0', 38, 48, 83, 92, 78, 4, 38
};
final NaiveSearchRingBuffer circ = new NaiveSearchRingBuffer(pattern);
int counter = -1;
for (final byte b : search) {
counter++;
final boolean matched = circ.addAndCompare(b);
if (counter == 10) {
assertTrue(matched);
} else {
assertFalse(matched);
}
}
}
@Test
public void testGetOldestByte() {
final byte[] pattern = new byte[]{
'\r', '0', 38, 48
};
final byte[] search = new byte[]{
'\r', '0', 38, 58, 58, 83, 78, (byte) 223, (byte) 227, (byte) 250, '\r', '0', 38, 48, 83, 92, 78, 4, 38
};
final NaiveSearchRingBuffer circ = new NaiveSearchRingBuffer(pattern);
int counter = -1;
for (final byte b : search) {
counter++;
final boolean matched = circ.addAndCompare(b);
if (counter == 13) {
assertTrue(matched);
} else {
assertFalse(matched);
}
}
}
}

59
commons/nifi-expression-language/pom.xml Normal file
View File

@ -0,0 +1,59 @@
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-parent</artifactId>
<version>0.0.1-SNAPSHOT</version>
</parent>
<artifactId>nifi-expression-language</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
<name>NiFi Expression Language</name>
<build>
<plugins>
<plugin>
<groupId>org.antlr</groupId>
<artifactId>antlr3-maven-plugin</artifactId>
<version>3.5.2</version>
<executions>
<execution>
<goals>
<goal>antlr</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>org.antlr</groupId>
<artifactId>antlr-runtime</artifactId>
<version>3.5.2</version>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-api</artifactId>
<version>[0.0.1-SNAPSHOT, 1.0.0-SNAPSHOT)</version>
</dependency>
</dependencies>
</project>

207
commons/nifi-expression-language/src/main/antlr3/org/apache/nifi/attribute/expression/language/antlr/AttributeExpressionLexer.g Normal file
View File

@ -0,0 +1,207 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
lexer grammar AttributeExpressionLexer;
@header {
package org.apache.nifi.attribute.expression.language.antlr;
import org.apache.nifi.attribute.expression.language.exception.AttributeExpressionLanguageParsingException;
}
@rulecatch {
catch(final Exception e) {
throw new AttributeExpressionLanguageParsingException(e);
}
}
@members {
public void displayRecognitionError(String[] tokenNames, RecognitionException e) {
final StringBuilder sb = new StringBuilder();
if ( e.token == null ) {
sb.append("Unrecognized token ");
} else {
sb.append("Unexpected token '").append(e.token.getText()).append("' ");
}
sb.append("at line ").append(e.line);
if ( e.approximateLineInfo ) {
sb.append(" (approximately)");
}
sb.append(", column ").append(e.charPositionInLine);
sb.append(". Query: ").append(e.input.toString());
throw new AttributeExpressionLanguageParsingException(sb.toString());
}
public void recover(RecognitionException e) {
final StringBuilder sb = new StringBuilder();
if ( e.token == null ) {
sb.append("Unrecognized token ");
} else {
sb.append("Unexpected token '").append(e.token.getText()).append("' ");
}
sb.append("at line ").append(e.line);
if ( e.approximateLineInfo ) {
sb.append(" (approximately)");
}
sb.append(", column ").append(e.charPositionInLine);
sb.append(". Query: ").append(e.input.toString());
throw new AttributeExpressionLanguageParsingException(sb.toString());
}
}
// PUNCTUATION & SPECIAL CHARACTERS
WHITESPACE : (' '|'\t'|'\n'|'\r')+ { $channel = HIDDEN; };
COMMENT : '#' ( ~('\n') )* '\n' { $channel = HIDDEN; };
DOLLAR : '$';
LPAREN : '(';
RPAREN : ')';
LBRACE : '{';
RBRACE : '}';
COLON : ':';
COMMA : ',';
DOT : '.';
SEMICOLON : ';';
NUMBER : ('0'..'9')+;
TRUE : 'true';
FALSE : 'false';
//
// FUNCTION NAMES
//
// ATTRIBUTE KEY SELECTION FUNCTIONS
ANY_ATTRIBUTE : 'anyAttribute';
ANY_MATCHING_ATTRIBUTE : 'anyMatchingAttribute';
ALL_ATTRIBUTES : 'allAttributes';
ALL_MATCHING_ATTRIBUTES : 'allMatchingAttributes';
ANY_DELINEATED_VALUE : 'anyDelineatedValue';
ALL_DELINEATED_VALUES : 'allDelineatedValues';
// NO-SUBJECT FUNCTIONS
NEXT_INT : 'nextInt';
IP : 'ip';
UUID : 'UUID';
HOSTNAME : 'hostname'; // requires boolean arg: prefer FQDN
NOW : 'now';
// 0 arg functions
TO_UPPER : 'toUpper';
TO_LOWER : 'toLower';
TO_STRING : 'toString';
LENGTH : 'length';
TRIM : 'trim';
IS_NULL : 'isNull';
NOT_NULL : 'notNull';
TO_NUMBER : 'toNumber';
URL_ENCODE : 'urlEncode';
URL_DECODE : 'urlDecode';
NOT : 'not';
// 1 arg functions
SUBSTRING_AFTER : 'substringAfter';
SUBSTRING_BEFORE : 'substringBefore';
SUBSTRING_AFTER_LAST : 'substringAfterLast';
SUBSTRING_BEFORE_LAST : 'substringBeforeLast';
STARTS_WITH : 'startsWith';
ENDS_WITH : 'endsWith';
CONTAINS : 'contains';
PREPEND : 'prepend';
APPEND : 'append';
INDEX_OF : 'indexOf';
LAST_INDEX_OF : 'lastIndexOf';
REPLACE_NULL : 'replaceNull';
FIND : 'find'; // regex
MATCHES : 'matches'; // regex
EQUALS : 'equals';
EQUALS_IGNORE_CASE : 'equalsIgnoreCase';
GREATER_THAN : 'gt';
LESS_THAN : 'lt';
GREATER_THAN_OR_EQUAL : 'ge';
LESS_THAN_OR_EQUAL : 'le';
FORMAT : 'format'; // takes string date format; uses SimpleDateFormat
TO_DATE : 'toDate'; // takes string date format; converts the subject to a Long based on the date format
MOD : 'mod';
PLUS : 'plus';
MINUS : 'minus';
MULTIPLY : 'multiply';
DIVIDE : 'divide';
TO_RADIX : 'toRadix';
OR : 'or';
AND : 'and';
// 2 arg functions
SUBSTRING : 'substring';
REPLACE : 'replace';
REPLACE_ALL : 'replaceAll';
// STRINGS
STRING_LITERAL
@init{StringBuilder lBuf = new StringBuilder();}
:
(
'"'
(
escaped=ESC {lBuf.append(getText());} |
normal = ~( '"' | '\\' | '\n' | '\r' | '\t' ) { lBuf.appendCodePoint(normal);}
)*
'"'
)
{
setText(lBuf.toString());
}
|
(
'\''
(
escaped=ESC {lBuf.append(getText());} |
normal = ~( '\'' | '\\' | '\n' | '\r' | '\t' ) { lBuf.appendCodePoint(normal);}
)*
'\''
)
{
setText(lBuf.toString());
}
;
fragment
ESC
: '\\'
(
'"' { setText("\""); }
| '\'' { setText("\'"); }
| 'r' { setText("\r"); }
| 'n' { setText("\n"); }
| 't' { setText("\t"); }
| '\\' { setText("\\\\"); }
| nextChar = ~('"' | '\'' | 'r' | 'n' | 't' | '\\')
{
StringBuilder lBuf = new StringBuilder(); lBuf.append("\\\\").appendCodePoint(nextChar); setText(lBuf.toString());
}
)
;
ATTRIBUTE_NAME : (
~('$' | '{' | '}' | '(' | ')' | '[' | ']' | ',' | ':' | ';' | '/' | '*' | '\'' | ' ' | '\t' | '\r' | '\n' | '0'..'9')
~('$' | '{' | '}' | '(' | ')' | '[' | ']' | ',' | ':' | ';' | '/' | '*' | '\'' | ' ' | '\t' | '\r' | '\n')*
);

139
commons/nifi-expression-language/src/main/antlr3/org/apache/nifi/attribute/expression/language/antlr/AttributeExpressionParser.g Normal file
View File

@ -0,0 +1,139 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
parser grammar AttributeExpressionParser;
options {
output=AST;
tokenVocab=AttributeExpressionLexer;
}
tokens {
QUERY;
ATTRIBUTE_REFERENCE;
ATTR_NAME;
FUNCTION_CALL;
EXPRESSION;
MULTI_ATTRIBUTE_REFERENCE;
QUOTED_ATTR_NAME;
}
@header {
package org.apache.nifi.attribute.expression.language.antlr;
import org.apache.nifi.attribute.expression.language.exception.AttributeExpressionLanguageParsingException;
}
@members {
public void displayRecognitionError(String[] tokenNames, RecognitionException e) {
final StringBuilder sb = new StringBuilder();
if ( e.token == null ) {
sb.append("Unrecognized token ");
} else {
sb.append("Unexpected token '").append(e.token.getText()).append("' ");
}
sb.append("at line ").append(e.line);
if ( e.approximateLineInfo ) {
sb.append(" (approximately)");
}
sb.append(", column ").append(e.charPositionInLine);
sb.append(". Query: ").append(e.input.toString());
throw new AttributeExpressionLanguageParsingException(sb.toString());
}
public void recover(final RecognitionException e) {
final StringBuilder sb = new StringBuilder();
if ( e.token == null ) {
sb.append("Unrecognized token ");
} else {
sb.append("Unexpected token '").append(e.token.getText()).append("' ");
}
sb.append("at line ").append(e.line);
if ( e.approximateLineInfo ) {
sb.append(" (approximately)");
}
sb.append(", column ").append(e.charPositionInLine);
sb.append(". Query: ").append(e.input.toString());
throw new AttributeExpressionLanguageParsingException(sb.toString());
}
}
// functions that return Strings
zeroArgString : (TO_UPPER | TO_LOWER | TRIM | TO_STRING | URL_ENCODE | URL_DECODE) LPAREN! RPAREN!;
oneArgString : ((SUBSTRING_BEFORE | SUBSTRING_BEFORE_LAST | SUBSTRING_AFTER | SUBSTRING_AFTER_LAST | REPLACE_NULL |
PREPEND | APPEND | FORMAT | STARTS_WITH | ENDS_WITH | CONTAINS) LPAREN! anyArg RPAREN!) |
(TO_RADIX LPAREN! anyArg (COMMA! anyArg)? RPAREN!);
twoArgString : ((REPLACE | REPLACE_ALL) LPAREN! anyArg COMMA! anyArg RPAREN!) |
(SUBSTRING LPAREN! anyArg (COMMA! anyArg)? RPAREN!);
// functions that return Booleans
zeroArgBool : (IS_NULL | NOT_NULL | NOT) LPAREN! RPAREN!;
oneArgBool : ((FIND | MATCHES | EQUALS_IGNORE_CASE) LPAREN! anyArg RPAREN!) |
(GREATER_THAN | LESS_THAN | GREATER_THAN_OR_EQUAL | LESS_THAN_OR_EQUAL) LPAREN! anyArg RPAREN! |
(EQUALS) LPAREN! anyArg RPAREN! |
(AND | OR) LPAREN! anyArg RPAREN!;
// functions that return Numbers
zeroArgNum : (LENGTH | TO_NUMBER) LPAREN! RPAREN!;
oneArgNum : ((INDEX_OF | LAST_INDEX_OF) LPAREN! anyArg RPAREN!) |
(TO_DATE LPAREN! anyArg? RPAREN!) |
((MOD | PLUS | MINUS | MULTIPLY | DIVIDE) LPAREN! anyArg RPAREN!);
stringFunctionRef : zeroArgString | oneArgString | twoArgString;
booleanFunctionRef : zeroArgBool | oneArgBool;
numberFunctionRef : zeroArgNum | oneArgNum;
anyArg : NUMBER | numberFunctionRef | STRING_LITERAL | zeroArgString | oneArgString | twoArgString | booleanLiteral | zeroArgBool | oneArgBool | expression;
stringArg : STRING_LITERAL | zeroArgString | oneArgString | twoArgString | expression;
functionRef : stringFunctionRef | booleanFunctionRef | numberFunctionRef;
// Attribute Reference
subject : attrName | expression;
attrName : singleAttrName | multiAttrName;
singleAttrRef : ATTRIBUTE_NAME | STRING_LITERAL;
singleAttrName : singleAttrRef ->
^(ATTR_NAME singleAttrRef);
multiAttrFunction : ANY_ATTRIBUTE | ANY_MATCHING_ATTRIBUTE | ALL_ATTRIBUTES | ALL_MATCHING_ATTRIBUTES | ANY_DELINEATED_VALUE | ALL_DELINEATED_VALUES;
multiAttrName : multiAttrFunction LPAREN stringArg (COMMA stringArg)* RPAREN ->
^(MULTI_ATTRIBUTE_REFERENCE multiAttrFunction stringArg*);
attributeRef : subject ->
^(ATTRIBUTE_REFERENCE subject);
functionCall : functionRef ->
^(FUNCTION_CALL functionRef);
booleanLiteral : TRUE | FALSE;
zeroArgStandaloneFunction : (IP | UUID | NOW | NEXT_INT | HOSTNAME) LPAREN! RPAREN!;
oneArgStandaloneFunction : HOSTNAME^ LPAREN! booleanLiteral RPAREN!;
standaloneFunction : zeroArgStandaloneFunction | oneArgStandaloneFunction;
attributeRefOrFunctionCall : (attributeRef | standaloneFunction);
expression : DOLLAR LBRACE attributeRefOrFunctionCall (COLON functionCall)* RBRACE ->
^(EXPRESSION attributeRefOrFunctionCall functionCall*);
query : expression EOF ->
^(QUERY expression);

72
commons/nifi-expression-language/src/main/antlr3/org/apache/nifi/attribute/expression/language/antlr/output/AttributeExpressionLexer.tokens Executable file
View File

@ -0,0 +1,72 @@
ALL_ATTRIBUTES=4
ALL_DELINEATED_VALUES=5
ALL_MATCHING_ATTRIBUTES=6
AND=7
ANY_ATTRIBUTE=8
ANY_DELINEATED_VALUE=9
ANY_MATCHING_ATTRIBUTE=10
APPEND=11
ATTRIBUTE_NAME=12
CEIL=13
COLON=14
COMMA=15
CONTAINS=16
DIVIDE=17
DOLLAR=18
DOT=19
ENDS_WITH=20
EQUALS=21
EQUALS_IGNORE_CASE=22
FALSE=23
FIND=24
FLOOR=25
FORMAT=26
GREATER_THAN=27
GREATER_THAN_OR_EQUAL=28
HOSTNAME=29
INDEX_OF=30
IP=31
IS_NULL=32
LAST_INDEX_OF=33
LBRACE=34
LENGTH=35
LESS_THAN=36
LESS_THAN_OR_EQUAL=37
LPAREN=38
MATCHES=39
MINUS=40
MOD=41
MULTIPLY=42
NEXT_INT=43
NOT=44
NOT_NULL=45
NOW=46
NUMBER=47
OR=48
PLUS=49
PREPEND=50
RBRACE=51
REPLACE=52
REPLACE_ALL=53
REPLACE_NULL=54
RPAREN=55
SEMICOLON=56
STARTS_WITH=57
STRING_LITERAL=58
SUBSTRING=59
SUBSTRING_AFTER=60
SUBSTRING_AFTER_LAST=61
SUBSTRING_BEFORE=62
SUBSTRING_BEFORE_LAST=63
TO_DATE=64
TO_LOWER=65
TO_NUMBER=66
TO_RADIX=67
TO_STRING=68
TO_UPPER=69
TRIM=70
TRUE=71
URL_DECODE=72
URL_ENCODE=73
UUID=74
WHITESPACE=75

62
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/EmptyPreparedQuery.java Normal file
View File

@ -0,0 +1,62 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language;
import java.util.Map;
import org.apache.nifi.expression.AttributeValueDecorator;
import org.apache.nifi.flowfile.FlowFile;
import org.apache.nifi.processor.exception.ProcessException;
public class EmptyPreparedQuery implements PreparedQuery {
private final String value;
EmptyPreparedQuery(final String value) {
this.value = value;
}
@Override
public String evaluateExpressions(final FlowFile flowFile, final AttributeValueDecorator decorator) throws ProcessException {
return value;
}
@Override
public String evaluateExpressions() throws ProcessException {
return value;
}
@Override
public String evaluateExpressions(final AttributeValueDecorator decorator) throws ProcessException {
return value;
}
@Override
public String evaluateExpressions(final FlowFile flowFile) throws ProcessException {
return value;
}
@Override
public String evaluateExpressions(Map<String, String> attributes) throws ProcessException {
return value;
}
@Override
public String evaluateExpressions(Map<String, String> attributes, AttributeValueDecorator decorator) throws ProcessException {
return value;
}
}

39
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/PreparedQuery.java Normal file
View File

@ -0,0 +1,39 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language;
import java.util.Map;
import org.apache.nifi.expression.AttributeValueDecorator;
import org.apache.nifi.flowfile.FlowFile;
import org.apache.nifi.processor.exception.ProcessException;
public interface PreparedQuery {
String evaluateExpressions(FlowFile flowFile, AttributeValueDecorator decorator) throws ProcessException;
String evaluateExpressions() throws ProcessException;
String evaluateExpressions(AttributeValueDecorator decorator) throws ProcessException;
String evaluateExpressions(FlowFile flowFile) throws ProcessException;
String evaluateExpressions(Map<String, String> attributes) throws ProcessException;
String evaluateExpressions(Map<String, String> attributes, AttributeValueDecorator decorator) throws ProcessException;
}

1183
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/Query.java Normal file
View File

File diff suppressed because it is too large Load Diff

65
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/StandardAttributeExpression.java Normal file
View File

@ -0,0 +1,65 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language;
import org.apache.nifi.expression.AttributeExpression;
import org.apache.nifi.expression.AttributeValueDecorator;
import org.apache.nifi.flowfile.FlowFile;
import org.apache.nifi.processor.exception.ProcessException;
public class StandardAttributeExpression implements AttributeExpression {
private final Query query;
public StandardAttributeExpression(final Query query) {
this.query = query;
}
@Override
public ResultType getResultType() {
return query.getResultType();
}
@Override
public String evaluate() throws ProcessException {
return evaluate((AttributeValueDecorator) null);
}
@Override
public String evaluate(final AttributeValueDecorator decorator) throws ProcessException {
return evaluate(null, decorator);
}
@Override
public String evaluate(final FlowFile flowFile) throws ProcessException {
return evaluate(flowFile, null);
}
@Override
public String evaluate(final FlowFile flowFile, final AttributeValueDecorator decorator) throws ProcessException {
final Object evaluationResult = query.evaluate(flowFile).getValue();
if (evaluationResult == null) {
return "";
}
String result = evaluationResult.toString();
if (decorator != null) {
result = decorator.decorate(result);
}
return Query.unescape(result);
}
}

58
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/StandardExpressionLanguageCompiler.java Normal file
View File

@ -0,0 +1,58 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language;
import org.apache.nifi.attribute.expression.language.exception.AttributeExpressionLanguageParsingException;
import org.apache.nifi.expression.AttributeExpression;
import org.apache.nifi.expression.ExpressionLanguageCompiler;
import org.apache.nifi.expression.AttributeExpression.ResultType;
public class StandardExpressionLanguageCompiler implements ExpressionLanguageCompiler {
@Override
public AttributeExpression compile(final String expression) throws IllegalArgumentException {
try {
return new StandardAttributeExpression(Query.compile(expression));
} catch (final AttributeExpressionLanguageParsingException e) {
throw new IllegalArgumentException(e.getMessage());
}
}
@Override
public boolean isValidExpression(final String expression) {
return Query.isValidExpression(expression);
}
@Override
public String validateExpression(final String expression, final boolean allowSurroundingCharacters) {
try {
Query.validateExpression(expression, allowSurroundingCharacters);
return null;
} catch (final AttributeExpressionLanguageParsingException aelpe) {
return aelpe.getMessage();
}
}
@Override
public ResultType getResultType(final String expression) throws IllegalArgumentException {
try {
return Query.getResultType(expression);
} catch (final AttributeExpressionLanguageParsingException e) {
throw new IllegalArgumentException(e);
}
}
}

83
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/StandardPreparedQuery.java Normal file
View File

@ -0,0 +1,83 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import org.apache.nifi.expression.AttributeValueDecorator;
import org.apache.nifi.flowfile.FlowFile;
import org.apache.nifi.processor.exception.ProcessException;
import org.antlr.runtime.tree.Tree;
public class StandardPreparedQuery implements PreparedQuery {
private final List<String> queryStrings;
private final Map<String, Tree> trees;
public StandardPreparedQuery(final List<String> queryStrings, final Map<String, Tree> trees) {
this.queryStrings = new ArrayList<>(queryStrings);
this.trees = new HashMap<>(trees);
}
@Override
public String evaluateExpressions(Map<String, String> attributes) throws ProcessException {
return evaluateExpressions(attributes, null);
}
@Override
public String evaluateExpressions(final Map<String, String> attributes, final AttributeValueDecorator decorator) throws ProcessException {
final StringBuilder sb = new StringBuilder();
for (final String val : queryStrings) {
final Tree tree = trees.get(val);
if (tree == null) {
sb.append(val);
} else {
final String evaluated = Query.evaluateExpression(tree, val, attributes, decorator);
if (evaluated != null) {
sb.append(evaluated);
}
}
}
return sb.toString();
}
@Override
public String evaluateExpressions(final FlowFile flowFile, final AttributeValueDecorator decorator) throws ProcessException {
final Map<String, String> expressionMap = Query.createExpressionMap(flowFile);
return evaluateExpressions(expressionMap, decorator);
}
@Override
public String evaluateExpressions() throws ProcessException {
return evaluateExpressions((FlowFile) null, null);
}
@Override
public String evaluateExpressions(final AttributeValueDecorator decorator) throws ProcessException {
return evaluateExpressions((FlowFile) null, decorator);
}
@Override
public String evaluateExpressions(final FlowFile flowFile) throws ProcessException {
return evaluateExpressions(flowFile, null);
}
}

32
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/BooleanEvaluator.java Normal file
View File

@ -0,0 +1,32 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation;
import org.apache.nifi.expression.AttributeExpression.ResultType;
public abstract class BooleanEvaluator implements Evaluator<Boolean> {
@Override
public ResultType getResultType() {
return ResultType.BOOLEAN;
}
@Override
public int getEvaluationsRemaining() {
return 0;
}
}

43
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/BooleanQueryResult.java Normal file
View File

@ -0,0 +1,43 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation;
import org.apache.nifi.expression.AttributeExpression.ResultType;
public class BooleanQueryResult implements QueryResult<Boolean> {
private final Boolean value;
public BooleanQueryResult(final Boolean value) {
this.value = value;
}
@Override
public Boolean getValue() {
return value;
}
@Override
public ResultType getResultType() {
return ResultType.BOOLEAN;
}
@Override
public String toString() {
return String.valueOf(getValue());
}
}

34
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/DateEvaluator.java Normal file
View File

@ -0,0 +1,34 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation;
import java.util.Date;
import org.apache.nifi.expression.AttributeExpression.ResultType;
public abstract class DateEvaluator implements Evaluator<Date> {
@Override
public ResultType getResultType() {
return ResultType.DATE;
}
@Override
public int getEvaluationsRemaining() {
return 0;
}
}

45
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/DateQueryResult.java Normal file
View File

@ -0,0 +1,45 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation;
import java.util.Date;
import org.apache.nifi.expression.AttributeExpression.ResultType;
public class DateQueryResult implements QueryResult<Date> {
private final Date date;
public DateQueryResult(final Date date) {
this.date = date;
}
@Override
public Date getValue() {
return date;
}
@Override
public ResultType getResultType() {
return ResultType.DATE;
}
@Override
public String toString() {
return String.valueOf(getValue());
}
}

32
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/Evaluator.java Normal file
View File

@ -0,0 +1,32 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation;
import java.util.Map;
import org.apache.nifi.expression.AttributeExpression.ResultType;
public interface Evaluator<T> {
QueryResult<T> evaluate(Map<String, String> attributes);
ResultType getResultType();
int getEvaluationsRemaining();
Evaluator<?> getSubjectEvaluator();
}

33
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/NumberEvaluator.java Normal file
View File

@ -0,0 +1,33 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation;
import org.apache.nifi.expression.AttributeExpression.ResultType;
public abstract class NumberEvaluator implements Evaluator<Long> {
@Override
public ResultType getResultType() {
return ResultType.NUMBER;
}
@Override
public int getEvaluationsRemaining() {
return 0;
}
}

43
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/NumberQueryResult.java Normal file
View File

@ -0,0 +1,43 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation;
import org.apache.nifi.expression.AttributeExpression.ResultType;
public class NumberQueryResult implements QueryResult<Long> {
private final Long value;
public NumberQueryResult(final Long value) {
this.value = value;
}
@Override
public Long getValue() {
return value;
}
@Override
public ResultType getResultType() {
return ResultType.NUMBER;
}
@Override
public String toString() {
return String.valueOf(getValue());
}
}

26
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/QueryResult.java Normal file
View File

@ -0,0 +1,26 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation;
import org.apache.nifi.expression.AttributeExpression.ResultType;
public interface QueryResult<T> {
T getValue();
ResultType getResultType();
}

32
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/StringEvaluator.java Normal file
View File

@ -0,0 +1,32 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation;
import org.apache.nifi.expression.AttributeExpression.ResultType;
public abstract class StringEvaluator implements Evaluator<String> {
@Override
public ResultType getResultType() {
return ResultType.STRING;
}
@Override
public int getEvaluationsRemaining() {
return 0;
}
}

43
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/StringQueryResult.java Normal file
View File

@ -0,0 +1,43 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation;
import org.apache.nifi.expression.AttributeExpression.ResultType;
public class StringQueryResult implements QueryResult<String> {
private final String value;
public StringQueryResult(final String value) {
this.value = value;
}
@Override
public String getValue() {
return value;
}
@Override
public ResultType getResultType() {
return ResultType.STRING;
}
@Override
public String toString() {
return String.valueOf(getValue());
}
}

50
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/cast/BooleanCastEvaluator.java Normal file
View File

@ -0,0 +1,50 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation.cast;
import java.util.Map;
import org.apache.nifi.attribute.expression.language.evaluation.BooleanEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.BooleanQueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.StringEvaluator;
public class BooleanCastEvaluator extends BooleanEvaluator {
private final StringEvaluator subjectEvaluator;
public BooleanCastEvaluator(final StringEvaluator subjectEvaluator) {
this.subjectEvaluator = subjectEvaluator;
}
@Override
public QueryResult<Boolean> evaluate(final Map<String, String> attributes) {
final QueryResult<String> result = subjectEvaluator.evaluate(attributes);
if (result.getValue() == null) {
return new BooleanQueryResult(null);
}
return new BooleanQueryResult(Boolean.valueOf(result.getValue().trim()));
}
@Override
public Evaluator<?> getSubjectEvaluator() {
return subjectEvaluator;
}
}

115
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/cast/DateCastEvaluator.java Normal file
View File

@ -0,0 +1,115 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation.cast;
import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.Map;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import org.apache.nifi.attribute.expression.language.evaluation.DateEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.DateQueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.StringQueryResult;
import org.apache.nifi.attribute.expression.language.exception.AttributeExpressionLanguageException;
import org.apache.nifi.attribute.expression.language.exception.AttributeExpressionLanguageParsingException;
import org.apache.nifi.expression.AttributeExpression.ResultType;
public class DateCastEvaluator extends DateEvaluator {
public static final String DATE_TO_STRING_FORMAT = "EEE MMM dd HH:mm:ss zzz yyyy";
public static final Pattern DATE_TO_STRING_PATTERN = Pattern.compile("(?:[a-zA-Z]{3} ){2}\\d{2} \\d{2}\\:\\d{2}\\:\\d{2} (?:.*?) \\d{4}");
public static final String ALTERNATE_FORMAT_WITHOUT_MILLIS = "yyyy/MM/dd HH:mm:ss";
public static final String ALTERNATE_FORMAT_WITH_MILLIS = "yyyy/MM/dd HH:mm:ss.SSS";
public static final Pattern ALTERNATE_PATTERN = Pattern.compile("\\d{4}/\\d{2}/\\d{2} \\d{2}\\:\\d{2}\\:\\d{2}(\\.\\d{3})?");
public static final Pattern NUMBER_PATTERN = Pattern.compile("\\d+");
private final Evaluator<?> subjectEvaluator;
public DateCastEvaluator(final Evaluator<?> subjectEvaluator) {
if (subjectEvaluator.getResultType() == ResultType.BOOLEAN) {
throw new AttributeExpressionLanguageParsingException("Cannot implicitly convert Data Type " + subjectEvaluator.getResultType() + " to " + ResultType.DATE);
}
this.subjectEvaluator = subjectEvaluator;
}
@Override
public QueryResult<Date> evaluate(final Map<String, String> attributes) {
final QueryResult<?> result = subjectEvaluator.evaluate(attributes);
if (result.getValue() == null) {
return new DateQueryResult(null);
}
switch (result.getResultType()) {
case DATE:
return (DateQueryResult) result;
case STRING:
final String value = ((StringQueryResult) result).getValue().trim();
if (DATE_TO_STRING_PATTERN.matcher(value).matches()) {
final SimpleDateFormat sdf = new SimpleDateFormat(DATE_TO_STRING_FORMAT);
try {
final Date date = sdf.parse(value);
return new DateQueryResult(date);
} catch (final ParseException pe) {
throw new AttributeExpressionLanguageException("Could not parse input as date", pe);
}
} else if (NUMBER_PATTERN.matcher(value).matches()) {
return new DateQueryResult(new Date(Long.valueOf(value)));
} else {
final Matcher altMatcher = ALTERNATE_PATTERN.matcher(value);
if (altMatcher.matches()) {
final String millisValue = altMatcher.group(1);
final String format;
if (millisValue == null) {
format = ALTERNATE_FORMAT_WITHOUT_MILLIS;
} else {
format = ALTERNATE_FORMAT_WITH_MILLIS;
}
final SimpleDateFormat sdf = new SimpleDateFormat(format);
try {
final Date date = sdf.parse(value);
return new DateQueryResult(date);
} catch (final ParseException pe) {
throw new AttributeExpressionLanguageException("Could not parse input as date", pe);
}
} else {
throw new AttributeExpressionLanguageException("Could not implicitly convert input to DATE: " + value);
}
}
case NUMBER:
return new DateQueryResult(new Date((Long) result.getValue()));
default:
return new DateQueryResult(null);
}
}
@Override
public Evaluator<?> getSubjectEvaluator() {
return subjectEvaluator;
}
}

72
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/cast/NumberCastEvaluator.java Normal file
View File

@ -0,0 +1,72 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation.cast;
import java.util.Map;
import java.util.regex.Pattern;
import org.apache.nifi.attribute.expression.language.evaluation.DateQueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
import org.apache.nifi.attribute.expression.language.evaluation.NumberEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.NumberQueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.StringQueryResult;
import org.apache.nifi.attribute.expression.language.exception.AttributeExpressionLanguageParsingException;
import org.apache.nifi.expression.AttributeExpression.ResultType;
public class NumberCastEvaluator extends NumberEvaluator {
private final Evaluator<?> subjectEvaluator;
private static final Pattern NUMBER_PATTERN = Pattern.compile("-?\\d+");
public NumberCastEvaluator(final Evaluator<?> subjectEvaluator) {
if (subjectEvaluator.getResultType() == ResultType.BOOLEAN) {
throw new AttributeExpressionLanguageParsingException("Cannot implicitly convert Data Type " + subjectEvaluator.getResultType() + " to " + ResultType.NUMBER);
}
this.subjectEvaluator = subjectEvaluator;
}
@Override
public QueryResult<Long> evaluate(final Map<String, String> attributes) {
final QueryResult<?> result = subjectEvaluator.evaluate(attributes);
if (result.getValue() == null) {
return new NumberQueryResult(null);
}
switch (result.getResultType()) {
case NUMBER:
return (NumberQueryResult) result;
case STRING:
final String trimmed = ((StringQueryResult) result).getValue().trim();
if (NUMBER_PATTERN.matcher(trimmed).matches()) {
return new NumberQueryResult(Long.valueOf(trimmed));
} else {
return new NumberQueryResult(null);
}
case DATE:
return new NumberQueryResult(((DateQueryResult) result).getValue().getTime());
default:
return new NumberQueryResult(null);
}
}
@Override
public Evaluator<?> getSubjectEvaluator() {
return subjectEvaluator;
}
}

49
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/cast/StringCastEvaluator.java Normal file
View File

@ -0,0 +1,49 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation.cast;
import java.util.Map;
import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.StringEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.StringQueryResult;
public class StringCastEvaluator extends StringEvaluator {
private final Evaluator<?> subjectEvaluator;
public StringCastEvaluator(final Evaluator<?> subjectEvaluator) {
this.subjectEvaluator = subjectEvaluator;
}
@Override
public QueryResult<String> evaluate(final Map<String, String> attributes) {
final QueryResult<?> result = subjectEvaluator.evaluate(attributes);
if (result.getValue() == null) {
return new StringQueryResult(null);
}
return new StringQueryResult(String.valueOf(result.getValue()));
}
@Override
public Evaluator<?> getSubjectEvaluator() {
return subjectEvaluator;
}
}

60
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/AndEvaluator.java Normal file
View File

@ -0,0 +1,60 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation.functions;
import java.util.Map;
import org.apache.nifi.attribute.expression.language.evaluation.BooleanEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.BooleanQueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
public class AndEvaluator extends BooleanEvaluator {
private final BooleanEvaluator subjectEvaluator;
private final BooleanEvaluator rhsEvaluator;
public AndEvaluator(final BooleanEvaluator subjectEvaluator, final BooleanEvaluator rhsEvaluator) {
this.subjectEvaluator = subjectEvaluator;
this.rhsEvaluator = rhsEvaluator;
}
@Override
public QueryResult<Boolean> evaluate(final Map<String, String> attributes) {
final QueryResult<Boolean> subjectValue = subjectEvaluator.evaluate(attributes);
if (subjectValue == null) {
return new BooleanQueryResult(null);
}
if (Boolean.FALSE.equals(subjectValue.getValue())) {
return new BooleanQueryResult(false);
}
final QueryResult<Boolean> rhsValue = rhsEvaluator.evaluate(attributes);
if (rhsValue == null) {
return new BooleanQueryResult(false);
}
return new BooleanQueryResult(rhsValue.getValue());
}
@Override
public Evaluator<?> getSubjectEvaluator() {
return subjectEvaluator;
}
}

50
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/AppendEvaluator.java Normal file
View File

@ -0,0 +1,50 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation.functions;
import java.util.Map;
import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.StringEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.StringQueryResult;
public class AppendEvaluator extends StringEvaluator {
private final StringEvaluator subject;
private final StringEvaluator appendEvaluator;
public AppendEvaluator(final StringEvaluator subject, final StringEvaluator append) {
this.subject = subject;
this.appendEvaluator = append;
}
@Override
public QueryResult<String> evaluate(final Map<String, String> attributes) {
final String subjectValue = subject.evaluate(attributes).getValue();
final String appendValue = appendEvaluator.evaluate(attributes).getValue();
final String result = (subjectValue == null ? "" : subjectValue)
+ (appendValue == null ? "" : appendValue);
return new StringQueryResult(result);
}
@Override
public Evaluator<?> getSubjectEvaluator() {
return subject;
}
}

45
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/AttributeEvaluator.java Normal file
View File

@ -0,0 +1,45 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation.functions;
import java.util.Map;
import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.StringEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.StringQueryResult;
public class AttributeEvaluator extends StringEvaluator {
private final StringEvaluator nameEvaluator;
public AttributeEvaluator(final StringEvaluator nameEvaluator) {
this.nameEvaluator = nameEvaluator;
}
@Override
public QueryResult<String> evaluate(final Map<String, String> attributes) {
final String nameValue = nameEvaluator.evaluate(attributes).getValue();
final String attributeValue = attributes.get(nameValue);
return new StringQueryResult(attributeValue);
}
@Override
public Evaluator<?> getSubjectEvaluator() {
return null;
}
}

53
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/ContainsEvaluator.java Normal file
View File

@ -0,0 +1,53 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation.functions;
import java.util.Map;
import org.apache.nifi.attribute.expression.language.evaluation.BooleanEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.BooleanQueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.StringEvaluator;
public class ContainsEvaluator extends BooleanEvaluator {
private final StringEvaluator subject;
private final StringEvaluator search;
public ContainsEvaluator(final StringEvaluator subject, final StringEvaluator search) {
this.subject = subject;
this.search = search;
}
@Override
public QueryResult<Boolean> evaluate(final Map<String, String> attributes) {
final String subjectValue = subject.evaluate(attributes).getValue();
if (subjectValue == null) {
return new BooleanQueryResult(false);
}
final String searchString = search.evaluate(attributes).getValue();
return new BooleanQueryResult(searchString == null ? false : subjectValue.contains(searchString));
}
@Override
public Evaluator<?> getSubjectEvaluator() {
return subject;
}
}

50
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/DateToNumberEvaluator.java Normal file
View File

@ -0,0 +1,50 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation.functions;
import java.util.Date;
import java.util.Map;
import org.apache.nifi.attribute.expression.language.evaluation.DateEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
import org.apache.nifi.attribute.expression.language.evaluation.NumberEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.NumberQueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
public class DateToNumberEvaluator extends NumberEvaluator {
private final DateEvaluator subjectEvaluator;
public DateToNumberEvaluator(final DateEvaluator subjectEvaluator) {
this.subjectEvaluator = subjectEvaluator;
}
@Override
public QueryResult<Long> evaluate(final Map<String, String> attributes) {
final QueryResult<Date> subjectResult = subjectEvaluator.evaluate(attributes);
if (subjectResult.getValue() == null) {
return new NumberQueryResult(null);
}
return new NumberQueryResult(subjectResult.getValue().getTime());
}
@Override
public Evaluator<?> getSubjectEvaluator() {
return subjectEvaluator;
}
}

57
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/DivideEvaluator.java Normal file
View File

@ -0,0 +1,57 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation.functions;
import java.util.Map;
import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
import org.apache.nifi.attribute.expression.language.evaluation.NumberEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.NumberQueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
public class DivideEvaluator extends NumberEvaluator {
private final NumberEvaluator subject;
private final NumberEvaluator divideValue;
public DivideEvaluator(final NumberEvaluator subject, final NumberEvaluator divideValue) {
this.subject = subject;
this.divideValue = divideValue;
}
@Override
public QueryResult<Long> evaluate(final Map<String, String> attributes) {
final Long subjectValue = subject.evaluate(attributes).getValue();
if (subjectValue == null) {
return new NumberQueryResult(null);
}
final Long divide = divideValue.evaluate(attributes).getValue();
if (divide == null) {
return new NumberQueryResult(null);
}
final long result = subjectValue / divide;
return new NumberQueryResult(result);
}
@Override
public Evaluator<?> getSubjectEvaluator() {
return subject;
}
}

53
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/EndsWithEvaluator.java Normal file
View File

@ -0,0 +1,53 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation.functions;
import java.util.Map;
import org.apache.nifi.attribute.expression.language.evaluation.BooleanEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.BooleanQueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.StringEvaluator;
public class EndsWithEvaluator extends BooleanEvaluator {
private final StringEvaluator subject;
private final StringEvaluator search;
public EndsWithEvaluator(final StringEvaluator subject, final StringEvaluator search) {
this.subject = subject;
this.search = search;
}
@Override
public QueryResult<Boolean> evaluate(final Map<String, String> attributes) {
final String subjectValue = subject.evaluate(attributes).getValue();
if (subjectValue == null) {
return new BooleanQueryResult(false);
}
final String searchString = search.evaluate(attributes).getValue();
return new BooleanQueryResult(searchString == null ? false : subjectValue.endsWith(searchString));
}
@Override
public Evaluator<?> getSubjectEvaluator() {
return subject;
}
}

89
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/EqualsEvaluator.java Normal file
View File

@ -0,0 +1,89 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation.functions;
import java.util.Date;
import java.util.Map;
import org.apache.nifi.attribute.expression.language.evaluation.BooleanEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.BooleanQueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
import org.apache.nifi.expression.AttributeExpression.ResultType;
public class EqualsEvaluator extends BooleanEvaluator {
private final Evaluator<?> subject;
private final Evaluator<?> compareTo;
public EqualsEvaluator(final Evaluator<?> subject, final Evaluator<?> compareTo) {
this.subject = subject;
this.compareTo = compareTo;
}
@Override
public QueryResult<Boolean> evaluate(final Map<String, String> attributes) {
final Object a = subject.evaluate(attributes).getValue();
if (a == null) {
return new BooleanQueryResult(false);
}
final Object b = compareTo.evaluate(attributes).getValue();
if (b == null) {
return new BooleanQueryResult(false);
}
if (subject.getResultType() == compareTo.getResultType()) {
return new BooleanQueryResult(a.equals(b));
}
final String normalizedSubjectValue = normalizeValue(subject.getResultType(), a);
if (normalizedSubjectValue == null) {
return new BooleanQueryResult(false);
}
final String normalizedCompareToValue = normalizeValue(compareTo.getResultType(), b);
if (normalizedCompareToValue == null) {
return new BooleanQueryResult(false);
}
return new BooleanQueryResult(normalizedSubjectValue.equals(normalizedCompareToValue));
}
private String normalizeValue(final ResultType type, final Object value) {
if (value == null) {
return null;
}
switch (type) {
case STRING:
return (String) value;
case DATE:
return String.valueOf(((Date) value).getTime());
case BOOLEAN:
case NUMBER:
default:
return String.valueOf(value);
}
}
@Override
public Evaluator<?> getSubjectEvaluator() {
return subject;
}
}

60
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/EqualsIgnoreCaseEvaluator.java Normal file
View File

@ -0,0 +1,60 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation.functions;
import java.util.Map;
import org.apache.nifi.attribute.expression.language.evaluation.BooleanEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.BooleanQueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
public class EqualsIgnoreCaseEvaluator extends BooleanEvaluator {
private final Evaluator<?> subject;
private final Evaluator<?> compareTo;
public EqualsIgnoreCaseEvaluator(final Evaluator<?> subject, final Evaluator<?> compareTo) {
this.subject = subject;
this.compareTo = compareTo;
}
@Override
public QueryResult<Boolean> evaluate(final Map<String, String> attributes) {
final Object a = subject.evaluate(attributes).getValue();
if (a == null) {
return new BooleanQueryResult(false);
}
final Object b = compareTo.evaluate(attributes).getValue();
if (b == null) {
return new BooleanQueryResult(false);
}
if (a instanceof String && b instanceof String) {
return new BooleanQueryResult(((String) a).equalsIgnoreCase((String) b));
}
return new BooleanQueryResult(a.equals(b));
}
@Override
public Evaluator<?> getSubjectEvaluator() {
return subject;
}
}

72
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/FindEvaluator.java Normal file
View File

@ -0,0 +1,72 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation.functions;
import java.util.Map;
import java.util.regex.Pattern;
import org.apache.nifi.attribute.expression.language.evaluation.BooleanEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.BooleanQueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.StringEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.literals.StringLiteralEvaluator;
public class FindEvaluator extends BooleanEvaluator {
private final StringEvaluator subject;
private final StringEvaluator search;
private final Pattern compiledPattern;
public FindEvaluator(final StringEvaluator subject, final StringEvaluator search) {
this.subject = subject;
this.search = search;
// if the search string is a literal, we don't need to evaluate it each time; we can just
// pre-compile it. Otherwise, it must be compiled every time.
if (search instanceof StringLiteralEvaluator) {
this.compiledPattern = Pattern.compile(search.evaluate(null).getValue());
} else {
this.compiledPattern = null;
}
}
@Override
public QueryResult<Boolean> evaluate(final Map<String, String> attributes) {
final String subjectValue = subject.evaluate(attributes).getValue();
if (subjectValue == null) {
return new BooleanQueryResult(false);
}
final Pattern pattern;
if (compiledPattern == null) {
pattern = Pattern.compile(search.evaluate(attributes).getValue());
} else {
pattern = compiledPattern;
}
final boolean found = pattern.matcher(subjectValue).find();
return new BooleanQueryResult(found);
}
@Override
public Evaluator<?> getSubjectEvaluator() {
return subject;
}
}

60
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/FormatEvaluator.java Normal file
View File

@ -0,0 +1,60 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation.functions;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.Map;
import org.apache.nifi.attribute.expression.language.evaluation.DateEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.StringEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.StringQueryResult;
public class FormatEvaluator extends StringEvaluator {
private final DateEvaluator subject;
private final StringEvaluator format;
public FormatEvaluator(final DateEvaluator subject, final StringEvaluator format) {
this.subject = subject;
this.format = format;
}
@Override
public QueryResult<String> evaluate(final Map<String, String> attributes) {
final Date subjectValue = subject.evaluate(attributes).getValue();
if (subjectValue == null) {
return new StringQueryResult(null);
}
final QueryResult<String> formatResult = format.evaluate(attributes);
final String format = formatResult.getValue();
if (format == null) {
return null;
}
return new StringQueryResult(new SimpleDateFormat(format).format(subjectValue));
}
@Override
public Evaluator<?> getSubjectEvaluator() {
return subject;
}
}

60
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/GreaterThanEvaluator.java Normal file
View File

@ -0,0 +1,60 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation.functions;
import java.util.Map;
import org.apache.nifi.attribute.expression.language.evaluation.BooleanEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.BooleanQueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
import org.apache.nifi.attribute.expression.language.evaluation.NumberEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
public class GreaterThanEvaluator extends BooleanEvaluator {
private final NumberEvaluator subject;
private final NumberEvaluator comparison;
public GreaterThanEvaluator(final NumberEvaluator subject, final NumberEvaluator comparison) {
this.subject = subject;
this.comparison = comparison;
}
@Override
public QueryResult<Boolean> evaluate(final Map<String, String> attributes) {
final Long subjectValue = subject.evaluate(attributes).getValue();
if (subjectValue == null) {
return new BooleanQueryResult(false);
}
final Long comparisonValue = comparison.evaluate(attributes).getValue();
if (comparisonValue == null) {
return new BooleanQueryResult(false);
}
return new BooleanQueryResult(subjectValue > comparisonValue);
}
;
@Override
public Evaluator<?> getSubjectEvaluator() {
return subject;
}
}

60
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/GreaterThanOrEqualEvaluator.java Normal file
View File

@ -0,0 +1,60 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation.functions;
import java.util.Map;
import org.apache.nifi.attribute.expression.language.evaluation.BooleanEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.BooleanQueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
import org.apache.nifi.attribute.expression.language.evaluation.NumberEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
public class GreaterThanOrEqualEvaluator extends BooleanEvaluator {
private final NumberEvaluator subject;
private final NumberEvaluator comparison;
public GreaterThanOrEqualEvaluator(final NumberEvaluator subject, final NumberEvaluator comparison) {
this.subject = subject;
this.comparison = comparison;
}
@Override
public QueryResult<Boolean> evaluate(final Map<String, String> attributes) {
final Long subjectValue = subject.evaluate(attributes).getValue();
if (subjectValue == null) {
return new BooleanQueryResult(false);
}
final Long comparisonValue = comparison.evaluate(attributes).getValue();
if (comparisonValue == null) {
return new BooleanQueryResult(false);
}
return new BooleanQueryResult(subjectValue >= comparisonValue);
}
;
@Override
public Evaluator<?> getSubjectEvaluator() {
return subject;
}
}

58
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/HostnameEvaluator.java Normal file
View File

@ -0,0 +1,58 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation.functions;
import java.net.InetAddress;
import java.net.UnknownHostException;
import java.util.Map;
import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.StringEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.StringQueryResult;
public class HostnameEvaluator extends StringEvaluator {
private final StringQueryResult hostname;
public HostnameEvaluator(final boolean preferFQDN) throws UnknownHostException {
String address;
if (preferFQDN) {
try {
address = InetAddress.getLocalHost().getCanonicalHostName();
} catch (final Exception e) {
address = InetAddress.getLocalHost().getHostName();
}
} else {
address = InetAddress.getLocalHost().getHostName();
}
hostname = new StringQueryResult(address);
}
@Override
public QueryResult<String> evaluate(final Map<String, String> attributes) {
return hostname;
}
@Override
public Evaluator<?> getSubjectEvaluator() {
return null;
}
}

46
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/IPEvaluator.java Normal file
View File

@ -0,0 +1,46 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation.functions;
import java.net.InetAddress;
import java.net.UnknownHostException;
import java.util.Map;
import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.StringEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.StringQueryResult;
public class IPEvaluator extends StringEvaluator {
private final StringQueryResult ipAddress;
public IPEvaluator() throws UnknownHostException {
ipAddress = new StringQueryResult(InetAddress.getLocalHost().getHostAddress());
}
@Override
public QueryResult<String> evaluate(final Map<String, String> attributes) {
return ipAddress;
}
@Override
public Evaluator<?> getSubjectEvaluator() {
return null;
}
}

53
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/IndexOfEvaluator.java Normal file
View File

@ -0,0 +1,53 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation.functions;
import java.util.Map;
import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
import org.apache.nifi.attribute.expression.language.evaluation.NumberEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.NumberQueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.StringEvaluator;
public class IndexOfEvaluator extends NumberEvaluator {
private final StringEvaluator subject;
private final StringEvaluator indexEvaluator;
public IndexOfEvaluator(final StringEvaluator subject, final StringEvaluator indexEvaluator) {
this.subject = subject;
this.indexEvaluator = indexEvaluator;
}
@Override
public QueryResult<Long> evaluate(final Map<String, String> attributes) {
final String subjectValue = subject.evaluate(attributes).getValue();
if (subjectValue == null) {
return new NumberQueryResult(-1L);
}
final String indexEvalValue = indexEvaluator.evaluate(attributes).getValue();
return new NumberQueryResult((long) subjectValue.indexOf(indexEvalValue));
}
@Override
public Evaluator<?> getSubjectEvaluator() {
return subject;
}
}

45
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/IsNullEvaluator.java Normal file
View File

@ -0,0 +1,45 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation.functions;
import java.util.Map;
import org.apache.nifi.attribute.expression.language.evaluation.BooleanEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.BooleanQueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
public class IsNullEvaluator extends BooleanEvaluator {
private final Evaluator<?> subject;
public IsNullEvaluator(final Evaluator<?> subject) {
this.subject = subject;
}
@Override
public QueryResult<Boolean> evaluate(final Map<String, String> attributes) {
final Object subjectValue = subject.evaluate(attributes).getValue();
return new BooleanQueryResult(subjectValue == null);
}
@Override
public Evaluator<?> getSubjectEvaluator() {
return subject;
}
}

53
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/LastIndexOfEvaluator.java Normal file
View File

@ -0,0 +1,53 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation.functions;
import java.util.Map;
import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
import org.apache.nifi.attribute.expression.language.evaluation.NumberEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.NumberQueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.StringEvaluator;
public class LastIndexOfEvaluator extends NumberEvaluator {
private final StringEvaluator subject;
private final StringEvaluator indexEvaluator;
public LastIndexOfEvaluator(final StringEvaluator subject, final StringEvaluator indexEvaluator) {
this.subject = subject;
this.indexEvaluator = indexEvaluator;
}
@Override
public QueryResult<Long> evaluate(final Map<String, String> attributes) {
final String subjectValue = subject.evaluate(attributes).getValue();
if (subjectValue == null) {
return new NumberQueryResult(-1L);
}
final String indexEvalValue = indexEvaluator.evaluate(attributes).getValue();
return new NumberQueryResult((long) subjectValue.lastIndexOf(indexEvalValue));
}
@Override
public Evaluator<?> getSubjectEvaluator() {
return subject;
}
}

46
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/LengthEvaluator.java Normal file
View File

@ -0,0 +1,46 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation.functions;
import java.util.Map;
import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
import org.apache.nifi.attribute.expression.language.evaluation.NumberEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.NumberQueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.StringEvaluator;
public class LengthEvaluator extends NumberEvaluator {
private final StringEvaluator subject;
public LengthEvaluator(final StringEvaluator subject) {
this.subject = subject;
}
@Override
public QueryResult<Long> evaluate(final Map<String, String> attributes) {
final String subjectValue = subject.evaluate(attributes).getValue();
return new NumberQueryResult((long) ((subjectValue == null) ? 0 : subjectValue.length()));
}
@Override
public Evaluator<?> getSubjectEvaluator() {
return subject;
}
}

60
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/LessThanEvaluator.java Normal file
View File

@ -0,0 +1,60 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation.functions;
import java.util.Map;
import org.apache.nifi.attribute.expression.language.evaluation.BooleanEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.BooleanQueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
import org.apache.nifi.attribute.expression.language.evaluation.NumberEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
public class LessThanEvaluator extends BooleanEvaluator {
private final NumberEvaluator subject;
private final NumberEvaluator comparison;
public LessThanEvaluator(final NumberEvaluator subject, final NumberEvaluator comparison) {
this.subject = subject;
this.comparison = comparison;
}
@Override
public QueryResult<Boolean> evaluate(final Map<String, String> attributes) {
final Long subjectValue = subject.evaluate(attributes).getValue();
if (subjectValue == null) {
return new BooleanQueryResult(false);
}
final Long comparisonValue = comparison.evaluate(attributes).getValue();
if (comparisonValue == null) {
return new BooleanQueryResult(false);
}
return new BooleanQueryResult(subjectValue < comparisonValue);
}
;
@Override
public Evaluator<?> getSubjectEvaluator() {
return subject;
}
}

60
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/LessThanOrEqualEvaluator.java Normal file
View File

@ -0,0 +1,60 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation.functions;
import java.util.Map;
import org.apache.nifi.attribute.expression.language.evaluation.BooleanEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.BooleanQueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
import org.apache.nifi.attribute.expression.language.evaluation.NumberEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
public class LessThanOrEqualEvaluator extends BooleanEvaluator {
private final NumberEvaluator subject;
private final NumberEvaluator comparison;
public LessThanOrEqualEvaluator(final NumberEvaluator subject, final NumberEvaluator comparison) {
this.subject = subject;
this.comparison = comparison;
}
@Override
public QueryResult<Boolean> evaluate(final Map<String, String> attributes) {
final Long subjectValue = subject.evaluate(attributes).getValue();
if (subjectValue == null) {
return new BooleanQueryResult(false);
}
final Long comparisonValue = comparison.evaluate(attributes).getValue();
if (comparisonValue == null) {
return new BooleanQueryResult(false);
}
return new BooleanQueryResult(subjectValue <= comparisonValue);
}
;
@Override
public Evaluator<?> getSubjectEvaluator() {
return subject;
}
}

71
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/MatchesEvaluator.java Normal file
View File

@ -0,0 +1,71 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation.functions;
import java.util.Map;
import java.util.regex.Pattern;
import org.apache.nifi.attribute.expression.language.evaluation.BooleanEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.BooleanQueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.StringEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.literals.StringLiteralEvaluator;
public class MatchesEvaluator extends BooleanEvaluator {
private final StringEvaluator subject;
private final StringEvaluator search;
private final Pattern compiledPattern;
public MatchesEvaluator(final StringEvaluator subject, final StringEvaluator search) {
this.subject = subject;
this.search = search;
// if the search string is a literal, we don't need to evaluate it each time; we can just
// pre-compile it. Otherwise, it must be compiled every time.
if (search instanceof StringLiteralEvaluator) {
this.compiledPattern = Pattern.compile(search.evaluate(null).getValue());
} else {
this.compiledPattern = null;
}
}
@Override
public QueryResult<Boolean> evaluate(final Map<String, String> attributes) {
final String subjectValue = subject.evaluate(attributes).getValue();
if (subjectValue == null) {
return new BooleanQueryResult(false);
}
final Pattern pattern;
if (compiledPattern == null) {
pattern = Pattern.compile(search.evaluate(attributes).getValue());
} else {
pattern = compiledPattern;
}
final boolean matches = pattern.matcher(subjectValue).matches();
return new BooleanQueryResult(matches);
}
@Override
public Evaluator<?> getSubjectEvaluator() {
return subject;
}
}

57
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/MinusEvaluator.java Normal file
View File

@ -0,0 +1,57 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation.functions;
import java.util.Map;
import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
import org.apache.nifi.attribute.expression.language.evaluation.NumberEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.NumberQueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
public class MinusEvaluator extends NumberEvaluator {
private final NumberEvaluator subject;
private final NumberEvaluator minusValue;
public MinusEvaluator(final NumberEvaluator subject, final NumberEvaluator minusValue) {
this.subject = subject;
this.minusValue = minusValue;
}
@Override
public QueryResult<Long> evaluate(final Map<String, String> attributes) {
final Long subjectValue = subject.evaluate(attributes).getValue();
if (subjectValue == null) {
return new NumberQueryResult(null);
}
final Long minus = minusValue.evaluate(attributes).getValue();
if (minus == null) {
return new NumberQueryResult(null);
}
final long result = subjectValue - minus;
return new NumberQueryResult(result);
}
@Override
public Evaluator<?> getSubjectEvaluator() {
return subject;
}
}

57
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/ModEvaluator.java Normal file
View File

@ -0,0 +1,57 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation.functions;
import java.util.Map;
import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
import org.apache.nifi.attribute.expression.language.evaluation.NumberEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.NumberQueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
public class ModEvaluator extends NumberEvaluator {
private final NumberEvaluator subject;
private final NumberEvaluator modValue;
public ModEvaluator(final NumberEvaluator subject, final NumberEvaluator modValue) {
this.subject = subject;
this.modValue = modValue;
}
@Override
public QueryResult<Long> evaluate(final Map<String, String> attributes) {
final Long subjectValue = subject.evaluate(attributes).getValue();
if (subjectValue == null) {
return new NumberQueryResult(null);
}
final Long mod = modValue.evaluate(attributes).getValue();
if (mod == null) {
return new NumberQueryResult(null);
}
final long result = subjectValue % mod;
return new NumberQueryResult(result);
}
@Override
public Evaluator<?> getSubjectEvaluator() {
return subject;
}
}

57
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/MultiplyEvaluator.java Normal file
View File

@ -0,0 +1,57 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation.functions;
import java.util.Map;
import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
import org.apache.nifi.attribute.expression.language.evaluation.NumberEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.NumberQueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
public class MultiplyEvaluator extends NumberEvaluator {
private final NumberEvaluator subject;
private final NumberEvaluator multiplyValue;
public MultiplyEvaluator(final NumberEvaluator subject, final NumberEvaluator multiplyValue) {
this.subject = subject;
this.multiplyValue = multiplyValue;
}
@Override
public QueryResult<Long> evaluate(final Map<String, String> attributes) {
final Long subjectValue = subject.evaluate(attributes).getValue();
if (subjectValue == null) {
return new NumberQueryResult(null);
}
final Long multiply = multiplyValue.evaluate(attributes).getValue();
if (multiply == null) {
return new NumberQueryResult(null);
}
final long result = subjectValue * multiply;
return new NumberQueryResult(result);
}
@Override
public Evaluator<?> getSubjectEvaluator() {
return subject;
}
}

49
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/NotEvaluator.java Normal file
View File

@ -0,0 +1,49 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation.functions;
import java.util.Map;
import org.apache.nifi.attribute.expression.language.evaluation.BooleanEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.BooleanQueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
public class NotEvaluator extends BooleanEvaluator {
private final BooleanEvaluator subjectEvaluator;
public NotEvaluator(final BooleanEvaluator subjectEvaluator) {
this.subjectEvaluator = subjectEvaluator;
}
@Override
public QueryResult<Boolean> evaluate(final Map<String, String> attributes) {
final QueryResult<Boolean> subjectValue = subjectEvaluator.evaluate(attributes);
if (subjectValue == null) {
return new BooleanQueryResult(null);
}
return new BooleanQueryResult(!subjectValue.getValue());
}
@Override
public Evaluator<?> getSubjectEvaluator() {
return subjectEvaluator;
}
}

45
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/NotNullEvaluator.java Normal file
View File

@ -0,0 +1,45 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation.functions;
import java.util.Map;
import org.apache.nifi.attribute.expression.language.evaluation.BooleanEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.BooleanQueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
public class NotNullEvaluator extends BooleanEvaluator {
private final Evaluator<?> subject;
public NotNullEvaluator(final Evaluator<?> subject) {
this.subject = subject;
}
@Override
public QueryResult<Boolean> evaluate(final Map<String, String> attributes) {
final Object subjectValue = subject.evaluate(attributes).getValue();
return new BooleanQueryResult(subjectValue != null);
}
@Override
public Evaluator<?> getSubjectEvaluator() {
return subject;
}
}

39
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/NowEvaluator.java Normal file
View File

@ -0,0 +1,39 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation.functions;
import java.util.Date;
import java.util.Map;
import org.apache.nifi.attribute.expression.language.evaluation.DateEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.DateQueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
public class NowEvaluator extends DateEvaluator {
@Override
public QueryResult<Date> evaluate(final Map<String, String> attributes) {
return new DateQueryResult(new Date());
}
@Override
public Evaluator<?> getSubjectEvaluator() {
return null;
}
}

52
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/NumberToDateEvaluator.java Normal file
View File

@ -0,0 +1,52 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation.functions;
import java.util.Date;
import java.util.Map;
import org.apache.nifi.attribute.expression.language.evaluation.DateEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.DateQueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
import org.apache.nifi.attribute.expression.language.evaluation.NumberEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
public class NumberToDateEvaluator extends DateEvaluator {
private final NumberEvaluator subject;
public NumberToDateEvaluator(final NumberEvaluator subject) {
this.subject = subject;
}
@Override
public QueryResult<Date> evaluate(final Map<String, String> attributes) {
final QueryResult<Long> result = subject.evaluate(attributes);
final Long value = result.getValue();
if (value == null) {
return null;
}
return new DateQueryResult(new Date(value));
}
@Override
public Evaluator<?> getSubjectEvaluator() {
return subject;
}
}

41
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/OneUpSequenceEvaluator.java Normal file
View File

@ -0,0 +1,41 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation.functions;
import java.util.Map;
import java.util.concurrent.atomic.AtomicLong;
import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
import org.apache.nifi.attribute.expression.language.evaluation.NumberEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.NumberQueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
public class OneUpSequenceEvaluator extends NumberEvaluator {
private static final AtomicLong value = new AtomicLong(0L);
@Override
public QueryResult<Long> evaluate(final Map<String, String> attributes) {
return new NumberQueryResult(value.getAndIncrement());
}
@Override
public Evaluator<?> getSubjectEvaluator() {
return null;
}
}

60
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/OrEvaluator.java Normal file
View File

@ -0,0 +1,60 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation.functions;
import java.util.Map;
import org.apache.nifi.attribute.expression.language.evaluation.BooleanEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.BooleanQueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
public class OrEvaluator extends BooleanEvaluator {
private final BooleanEvaluator subjectEvaluator;
private final BooleanEvaluator rhsEvaluator;
public OrEvaluator(final BooleanEvaluator subjectEvaluator, final BooleanEvaluator rhsEvaluator) {
this.subjectEvaluator = subjectEvaluator;
this.rhsEvaluator = rhsEvaluator;
}
@Override
public QueryResult<Boolean> evaluate(final Map<String, String> attributes) {
final QueryResult<Boolean> subjectValue = subjectEvaluator.evaluate(attributes);
if (subjectValue == null) {
return new BooleanQueryResult(null);
}
if (Boolean.TRUE.equals(subjectValue.getValue())) {
return new BooleanQueryResult(true);
}
final QueryResult<Boolean> rhsValue = rhsEvaluator.evaluate(attributes);
if (rhsValue == null) {
return new BooleanQueryResult(false);
}
return new BooleanQueryResult(rhsValue.getValue());
}
@Override
public Evaluator<?> getSubjectEvaluator() {
return subjectEvaluator;
}
}

57
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/PlusEvaluator.java Normal file
View File

@ -0,0 +1,57 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation.functions;
import java.util.Map;
import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
import org.apache.nifi.attribute.expression.language.evaluation.NumberEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.NumberQueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
public class PlusEvaluator extends NumberEvaluator {
private final NumberEvaluator subject;
private final NumberEvaluator plusValue;
public PlusEvaluator(final NumberEvaluator subject, final NumberEvaluator plusValue) {
this.subject = subject;
this.plusValue = plusValue;
}
@Override
public QueryResult<Long> evaluate(final Map<String, String> attributes) {
final Long subjectValue = subject.evaluate(attributes).getValue();
if (subjectValue == null) {
return new NumberQueryResult(null);
}
final Long plus = plusValue.evaluate(attributes).getValue();
if (plus == null) {
return new NumberQueryResult(null);
}
final long result = subjectValue + plus;
return new NumberQueryResult(result);
}
@Override
public Evaluator<?> getSubjectEvaluator() {
return subject;
}
}

50
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/PrependEvaluator.java Normal file
View File

@ -0,0 +1,50 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation.functions;
import java.util.Map;
import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.StringEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.StringQueryResult;
public class PrependEvaluator extends StringEvaluator {
private final StringEvaluator subject;
private final StringEvaluator prependEvaluator;
public PrependEvaluator(final StringEvaluator subject, final StringEvaluator prepend) {
this.subject = subject;
this.prependEvaluator = prepend;
}
@Override
public QueryResult<String> evaluate(final Map<String, String> attributes) {
final String subjectValue = subject.evaluate(attributes).getValue();
final String prependValue = prependEvaluator.evaluate(attributes).getValue();
final String result = (prependValue == null ? "" : prependValue) + (subjectValue == null ? "" : subjectValue);
return new StringQueryResult(result);
}
@Override
public Evaluator<?> getSubjectEvaluator() {
return subject;
}
}

55
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/ReplaceAllEvaluator.java Normal file
View File

@ -0,0 +1,55 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation.functions;
import java.util.Map;
import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.StringEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.StringQueryResult;
public class ReplaceAllEvaluator extends StringEvaluator {
private final StringEvaluator subject;
private final StringEvaluator search;
private final StringEvaluator replacement;
public ReplaceAllEvaluator(final StringEvaluator subject, final StringEvaluator search, final StringEvaluator replacement) {
this.subject = subject;
this.search = search;
this.replacement = replacement;
}
@Override
public QueryResult<String> evaluate(final Map<String, String> attributes) {
final String subjectValue = subject.evaluate(attributes).getValue();
if (subjectValue == null) {
return new StringQueryResult(null);
}
final String searchValue = search.evaluate(attributes).getValue();
final String replacementValue = replacement.evaluate(attributes).getValue();
return new StringQueryResult(subjectValue.replaceAll(searchValue, replacementValue));
}
@Override
public Evaluator<?> getSubjectEvaluator() {
return subject;
}
}

55
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/ReplaceEvaluator.java Normal file
View File

@ -0,0 +1,55 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation.functions;
import java.util.Map;
import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.StringEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.StringQueryResult;
public class ReplaceEvaluator extends StringEvaluator {
private final StringEvaluator subject;
private final StringEvaluator search;
private final StringEvaluator replacement;
public ReplaceEvaluator(final StringEvaluator subject, final StringEvaluator search, final StringEvaluator replacement) {
this.subject = subject;
this.search = search;
this.replacement = replacement;
}
@Override
public QueryResult<String> evaluate(final Map<String, String> attributes) {
final String subjectValue = subject.evaluate(attributes).getValue();
if (subjectValue == null) {
return new StringQueryResult(null);
}
final String searchValue = search.evaluate(attributes).getValue();
final String replacementValue = replacement.evaluate(attributes).getValue();
return new StringQueryResult(subjectValue.replace(searchValue, replacementValue));
}
@Override
public Evaluator<?> getSubjectEvaluator() {
return subject;
}
}

47
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/ReplaceNullEvaluator.java Normal file
View File

@ -0,0 +1,47 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation.functions;
import java.util.Map;
import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.StringEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.StringQueryResult;
public class ReplaceNullEvaluator extends StringEvaluator {
private final StringEvaluator subject;
private final StringEvaluator resultEvaluator;
public ReplaceNullEvaluator(final StringEvaluator subject, final StringEvaluator resultEvaluator) {
this.subject = subject;
this.resultEvaluator = resultEvaluator;
}
@Override
public QueryResult<String> evaluate(final Map<String, String> attributes) {
final String subjectValue = subject.evaluate(attributes).getValue();
return new StringQueryResult(subjectValue == null ? resultEvaluator.evaluate(attributes).getValue() : subjectValue);
}
@Override
public Evaluator<?> getSubjectEvaluator() {
return subject;
}
}

53
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/StartsWithEvaluator.java Normal file
View File

@ -0,0 +1,53 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation.functions;
import java.util.Map;
import org.apache.nifi.attribute.expression.language.evaluation.BooleanEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.BooleanQueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.StringEvaluator;
public class StartsWithEvaluator extends BooleanEvaluator {
private final StringEvaluator subject;
private final StringEvaluator search;
public StartsWithEvaluator(final StringEvaluator subject, final StringEvaluator search) {
this.subject = subject;
this.search = search;
}
@Override
public QueryResult<Boolean> evaluate(final Map<String, String> attributes) {
final String subjectValue = subject.evaluate(attributes).getValue();
if (subjectValue == null) {
return new BooleanQueryResult(false);
}
final String searchString = search.evaluate(attributes).getValue();
return new BooleanQueryResult(searchString == null ? false : subjectValue.startsWith(searchString));
}
@Override
public Evaluator<?> getSubjectEvaluator() {
return subject;
}
}

64
commons/nifi-expression-language/src/main/java/org/apache/nifi/attribute/expression/language/evaluation/functions/StringToDateEvaluator.java Normal file
View File

@ -0,0 +1,64 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.attribute.expression.language.evaluation.functions;
import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.Map;
import org.apache.nifi.attribute.expression.language.evaluation.DateEvaluator;
import org.apache.nifi.attribute.expression.language.evaluation.DateQueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.Evaluator;
import org.apache.nifi.attribute.expression.language.evaluation.QueryResult;
import org.apache.nifi.attribute.expression.language.evaluation.StringEvaluator;
import org.apache.nifi.attribute.expression.language.exception.IllegalAttributeException;
public class StringToDateEvaluator extends DateEvaluator {
private final StringEvaluator subject;
private final StringEvaluator format;
public StringToDateEvaluator(final StringEvaluator subject, final StringEvaluator format) {
this.subject = subject;
this.format = format;
}
@Override
public QueryResult<Date> evaluate(final Map<String, String> attributes) {
final String subjectValue = subject.evaluate(attributes).getValue();
final String formatValue = format.evaluate(attributes).getValue();
if (subjectValue == null || formatValue == null) {
return new DateQueryResult(null);
}
try {
return new DateQueryResult(new SimpleDateFormat(formatValue).parse(subjectValue));
} catch (final ParseException e) {
throw new IllegalAttributeException("Cannot parse attribute value as a date; date format: "
+ formatValue + "; attribute value: " + subjectValue);
} catch (final IllegalArgumentException e) {
throw new IllegalAttributeException("Invalid date format: " + formatValue);
}
}
@Override
public Evaluator<?> getSubjectEvaluator() {
return subject;
}
}

Some files were not shown because too many files have changed in this diff Show More