honeymoose
/
OpenSearch
mirror of https://github.com/honeymoose/OpenSearch.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Better handling of sensitive data in registered watches and watcher settings 

A watch may contain sensitive data that typically you don't want to expose in plain text. Exposing means:
 - storing it as plain text in the `.watches` and `.watch_history` indices
 - storing it in memory in plain text (can be access via mem dump)
 - returning it to the user via API in plain text

Examples of such sensitive data:
 - The `password` for the email service (can be configured on the watch itself)
 - The `password` for http input when using basic auth
 - The `passowrd` for webhook action when using basic auth

A new `SecretService` (you heard it right... secret service) was added to handel the secrets across the board. When a watch is first added to watcher, this service converts all the sensitive data to secrets. From that moment on, all sensitive data associated with the watch (whether in stored in the index or in memory) is hidden behind the secret. This service is also used to "reveal" the original sensitive data on-demand when needed (for example, when the email is sent, it is sent with the original text).

There are two implementations for the `SecretService`. The default one is "plain text" where the created secrets don't really hide anything. The second implementation is based on Shield. If Shield is installed and enabled, the `ShieldSecretService` is used which uses shield's crypto service to potentially encrypt the sensitive data (only potentially because Shield's system key must be defined for encryption to take effect, without the system key, the crypto service will not encrypt and instead return the sensitive data in plain text)

Note, even when Shield is installed, the encryption of sensitive data will only be applied if the `watcher.shield.encrypt_sensitive_data` setting is set to `true`. By default it is set to `false`.

The `get watch` and `execute watch` APIs were updated to filter out sensitive data (using special "hide secrets" parameter).

When shield is integrated, we use shield's settings filter to filter out sensitive settings from the REST nodes info API (when shield is not installed or enabled, we don't do this filtering).

For this change several other refactoring needed to take place
 - The http auth codebase was refactored to be more modular. Just like with other modular constructs in watcher, we separated `HttpAuth` from `ApplicableHttpAuth` where the former is the configuration construct and tha latter is the applicable ("executable") construct.
 - Changed `WatchStore#put` to accept a watch (instead of the watch source). That's more natural way of looking at a store. Also, a `Watch` can now create and return itself as `ByteReference`. In addition, we now don't directly store the watch source as it was sent by the user, instead, we first parse it to a watch (important step to both validate the source and convert all sensitive data to secrets) and then serialize the watch back to `ByteReference`. This way we're sure that only the secrets are stored and not the original sensitive data.
 - All `ToXContent` implementation were updated to properly propagate the `Params`

Docs were added to the Shield Integration chapter

Original commit: elastic/x-pack-elasticsearch@4490fb0ab8
This commit is contained in:
uboness 2015-04-24 11:17:59 +02:00
parent 735369b5f4
commit 280732a120
87 changed files with 1738 additions and 368 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
all-signatures.txt
View File

@ -1,3 +1,3 @@
@defaultMessage Convert to URI
java.net.URL#getPath()
java.net.URL#getFile()
java.net.URL#getFile()

3
core-signatures.txt
View File

@ -67,4 +67,7 @@ org.elasticsearch.common.joda.time.DateTime#<init>(int, int, int, int, int, int)
org.elasticsearch.common.joda.time.DateTime#<init>(int, int, int, int, int, int, int)
org.elasticsearch.common.joda.time.DateTime#now()
@defaultMessage params is not passed down to the serialized xcontent
org.elasticsearch.common.xcontent.XContentBuilder#field(java.lang.String, org.elasticsearch.common.xcontent.ToXContent)

4
src/main/java/org/elasticsearch/watcher/WatcherModule.java
View File

@ -24,6 +24,7 @@ import org.elasticsearch.watcher.support.TemplateUtils;
import org.elasticsearch.watcher.support.clock.ClockModule;
import org.elasticsearch.watcher.support.http.HttpClientModule;
import org.elasticsearch.watcher.support.init.InitializingModule;
import org.elasticsearch.watcher.support.secret.SecretModule;
import org.elasticsearch.watcher.support.template.TemplateModule;
import org.elasticsearch.watcher.transform.TransformModule;
import org.elasticsearch.watcher.transport.WatcherTransportModule;
@ -58,7 +59,8 @@ public class WatcherModule extends AbstractModule implements SpawnModules {
new ActionModule(),
new HistoryModule(),
new ExecutionModule(),
new WatcherShieldModule(settings));
new WatcherShieldModule(settings),
new SecretModule(settings));
}
@Override

16
src/main/java/org/elasticsearch/watcher/WatcherService.java
View File

@ -26,17 +26,19 @@ import java.util.concurrent.atomic.AtomicReference;
public class WatcherService extends AbstractComponent {
private final TriggerService triggerService;
private final Watch.Parser watchParser;
private final WatchStore watchStore;
private final WatchLockService watchLockService;
private final ExecutionService executionService;
private final AtomicReference<State> state = new AtomicReference<>(State.STOPPED);
@Inject
public WatcherService(Settings settings, TriggerService triggerService, WatchStore watchStore, ExecutionService executionService,
WatchLockService watchLockService) {
public WatcherService(Settings settings, TriggerService triggerService, WatchStore watchStore, Watch.Parser watchParser,
ExecutionService executionService, WatchLockService watchLockService) {
super(settings);
this.triggerService = triggerService;
this.watchStore = watchStore;
this.watchParser = watchParser;
this.watchLockService = watchLockService;
this.executionService = executionService;
}
@ -89,18 +91,18 @@ public class WatcherService extends AbstractComponent {
}
}
public IndexResponse putWatch(String name, BytesReference watchSource) {
public IndexResponse putWatch(String id, BytesReference watchSource) {
ensureStarted();
WatchLockService.Lock lock = watchLockService.acquire(name);
WatchLockService.Lock lock = watchLockService.acquire(id);
try {
WatchStore.WatchPut result = watchStore.put(name, watchSource);
Watch watch = watchParser.parseWithSecrets(id, false, watchSource);
WatchStore.WatchPut result = watchStore.put(watch);
if (result.previous() == null || !result.previous().trigger().equals(result.current().trigger())) {
triggerService.add(result.current());
}
return result.indexResponse();
} catch (Exception e) {
logger.warn("failed to put watch [{}]", e, name);
throw new WatcherException("failed to put [" + watchSource.toUtf8() + "]", e);
throw new WatcherException("failed to put watch [{}]", e, id);
} finally {
lock.release();
}

8
src/main/java/org/elasticsearch/watcher/actions/ActionWrapper.java
View File

@ -86,10 +86,10 @@ public class ActionWrapper implements ToXContent {
builder.startObject();
if (transform != null) {
builder.startObject(Transform.Field.TRANSFORM.getPreferredName())
.field(transform.type(), transform)
.field(transform.type(), transform, params)
.endObject();
}
builder.field(action.type(), action);
builder.field(action.type(), action, params);
return builder.endObject();
}
@ -176,10 +176,10 @@ public class ActionWrapper implements ToXContent {
builder.startObject();
if (transform != null) {
builder.startObject(Transform.Field.TRANSFORM_RESULT.getPreferredName())
.field(transform.type(), transform)
.field(transform.type(), transform, params)
.endObject();
}
builder.field(action.type(), action);
builder.field(action.type(), action, params);
return builder.endObject();
}

4
src/main/java/org/elasticsearch/watcher/actions/ExecutableActions.java
View File

@ -37,7 +37,7 @@ public class ExecutableActions implements Iterable<ActionWrapper>, ToXContent {
public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
builder.startObject();
for (ActionWrapper action : actions) {
builder.field(action.id(), action);
builder.field(action.id(), action, params);
}
return builder.endObject();
}
@ -99,7 +99,7 @@ public class ExecutableActions implements Iterable<ActionWrapper>, ToXContent {
public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
builder.startObject();
for (ActionWrapper.Result result : results.values()) {
builder.field(result.id(), result);
builder.field(result.id(), result, params);
}
return builder.endObject();
}

20
src/main/java/org/elasticsearch/watcher/actions/email/EmailAction.java
View File

@ -14,6 +14,9 @@ import org.elasticsearch.watcher.actions.email.service.Authentication;
import org.elasticsearch.watcher.actions.email.service.Email;
import org.elasticsearch.watcher.actions.email.service.EmailTemplate;
import org.elasticsearch.watcher.actions.email.service.Profile;
import org.elasticsearch.watcher.support.secret.Secret;
import org.elasticsearch.watcher.support.secret.SensitiveXContentParser;
import org.elasticsearch.watcher.support.xcontent.WatcherParams;
import java.io.IOException;
import java.util.Locale;
@ -96,7 +99,9 @@ public class EmailAction implements Action {
}
if (auth != null) {
builder.field(Field.USER.getPreferredName(), auth.user());
builder.field(Field.PASSWORD.getPreferredName(), new String(auth.password()));
if (!WatcherParams.hideSecrets(params)) {
builder.field(Field.PASSWORD.getPreferredName(), auth.password(), params);
}
}
if (profile != null) {
builder.field(Field.PROFILE.getPreferredName(), profile.name().toLowerCase(Locale.ROOT));
@ -112,7 +117,7 @@ public class EmailAction implements Action {
EmailTemplate.Parser emailParser = new EmailTemplate.Parser();
String account = null;
String user = null;
String password = null;
Secret password = null;
Profile profile = Profile.STANDARD;
Boolean attachPayload = null;
@ -128,7 +133,7 @@ public class EmailAction implements Action {
} else if (Field.USER.match(currentFieldName)) {
user = parser.text();
} else if (Field.PASSWORD.match(currentFieldName)) {
password = parser.text();
password = SensitiveXContentParser.secretOrNull(parser);
} else if (Field.PROFILE.match(currentFieldName)) {
profile = Profile.resolve(parser.text());
} else {
@ -148,8 +153,7 @@ public class EmailAction implements Action {
Authentication auth = null;
if (user != null) {
char[] passwd = password != null ? password.toCharArray() : null;
auth = new Authentication(user, passwd);
auth = new Authentication(user, password);
}
return new EmailAction(emailParser.parsedTemplate(), account, auth, profile, attachPayload);
@ -187,7 +191,7 @@ public class EmailAction implements Action {
@Override
public XContentBuilder xContentBody(XContentBuilder builder, Params params) throws IOException {
return builder.field(Field.ACCOUNT.getPreferredName(), account)
.field(Field.EMAIL.getPreferredName(), email);
.field(Field.EMAIL.getPreferredName(), email, params);
}
}
@ -225,7 +229,7 @@ public class EmailAction implements Action {
@Override
protected XContentBuilder xContentBody(XContentBuilder builder, Params params) throws IOException {
return builder.field(Field.SIMULATED_EMAIL.getPreferredName(), email);
return builder.field(Field.SIMULATED_EMAIL.getPreferredName(), email, params);
}
}
@ -314,7 +318,7 @@ public class EmailAction implements Action {
}
public Builder setAuthentication(String username, char[] password) {
this.auth = new Authentication(username, password);
this.auth = new Authentication(username, new Secret(password));
return this;
}

21
src/main/java/org/elasticsearch/watcher/actions/email/service/Account.java
View File

@ -8,6 +8,7 @@ package org.elasticsearch.watcher.actions.email.service;
import org.elasticsearch.common.logging.ESLogger;
import org.elasticsearch.common.settings.ImmutableSettings;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.watcher.support.secret.SecretService;
import javax.activation.CommandMap;
import javax.activation.MailcapCommandMap;
@ -37,12 +38,14 @@ public class Account {
CommandMap.setDefaultCommandMap(mailcap);
}
private final ESLogger logger;
private final Config config;
private final SecretService secretService;
private final ESLogger logger;
private final Session session;
Account(Config config, ESLogger logger) {
Account(Config config, SecretService secretService, ESLogger logger) {
this.config = config;
this.secretService = secretService;
this.logger = logger;
session = config.createSession();
}
@ -61,6 +64,7 @@ public class Account {
}
Transport transport = session.getTransport(SMTP_PROTOCOL);
String user = auth != null ? auth.user() : null;
if (user == null) {
user = config.smtp.user;
@ -68,14 +72,19 @@ public class Account {
user = InternetAddress.getLocalAddress(session).getAddress();
}
}
char[] password = auth != null ? auth.password() : null;
if (password == null) {
password = config.smtp.password;
String password = null;
if (auth != null && auth.password() != null) {
password = new String(auth.password().text(secretService));
} else if (config.smtp.password != null) {
password = new String(config.smtp.password);
}
if (profile == null) {
profile = config.profile;
}
transport.connect(config.smtp.host, config.smtp.port, user, password != null ? new String(password) : null);
transport.connect(config.smtp.host, config.smtp.port, user, password);
try {
MimeMessage message = profile.toMimeMessage(email, session);

5
src/main/java/org/elasticsearch/watcher/actions/email/service/Accounts.java
View File

@ -7,6 +7,7 @@ package org.elasticsearch.watcher.actions.email.service;
import org.elasticsearch.common.logging.ESLogger;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.watcher.support.secret.SecretService;
import java.util.HashMap;
import java.util.Map;
@ -19,12 +20,12 @@ public class Accounts {
private final String defaultAccountName;
private final Map<String, Account> accounts;
public Accounts(Settings settings, ESLogger logger) {
public Accounts(Settings settings, SecretService secretService, ESLogger logger) {
Settings accountsSettings = settings.getAsSettings("account");
accounts = new HashMap<>();
for (String name : accountsSettings.names()) {
Account.Config config = new Account.Config(name, accountsSettings.getAsSettings(name));
Account account = new Account(config, logger);
Account account = new Account(config, secretService, logger);
accounts.put(name, account);
}

11
src/main/java/org/elasticsearch/watcher/actions/email/service/Authentication.java
View File

@ -5,7 +5,8 @@
*/
package org.elasticsearch.watcher.actions.email.service;
import java.util.Arrays;
import org.elasticsearch.watcher.support.secret.Secret;
import java.util.Objects;
/**
@ -14,9 +15,9 @@ import java.util.Objects;
public class Authentication {
private final String user;
private final char[] password;
private final Secret password;
public Authentication(String user, char[] password) {
public Authentication(String user, Secret password) {
this.user = user;
this.password = password;
}
@ -25,7 +26,7 @@ public class Authentication {
return user;
}
public char[] password() {
public Secret password() {
return password;
}
@ -35,7 +36,7 @@ public class Authentication {
if (o == null || getClass() != o.getClass()) return false;
Authentication that = (Authentication) o;
return Objects.equals(user, that.user) &&
Arrays.equals(password, that.password);
Objects.equals(password, that.password);
}
@Override

14
src/main/java/org/elasticsearch/watcher/actions/email/service/Email.java
View File

@ -118,23 +118,23 @@ public class Email implements ToXContent {
builder.startObject();
builder.field(Field.ID.getPreferredName(), id);
if (from != null) {
builder.field(Field.FROM.getPreferredName(), from);
builder.field(Field.FROM.getPreferredName(), from, params);
}
if (replyTo != null) {
builder.field(Field.REPLY_TO.getPreferredName(), (ToXContent) replyTo);
builder.field(Field.REPLY_TO.getPreferredName(), replyTo, params);
}
if (priority != null) {
builder.field(Field.PRIORITY.getPreferredName(), priority);
builder.field(Field.PRIORITY.getPreferredName(), priority, params);
}
builder.field(Field.SENT_DATE.getPreferredName(), sentDate);
if (to != null) {
builder.field(Field.TO.getPreferredName(), (ToXContent) to);
builder.field(Field.TO.getPreferredName(), to, params);
}
if (cc != null) {
builder.field(Field.CC.getPreferredName(), (ToXContent) cc);
builder.field(Field.CC.getPreferredName(), cc, params);
}
if (bcc != null) {
builder.field(Field.BCC.getPreferredName(), (ToXContent) bcc);
builder.field(Field.BCC.getPreferredName(), bcc, params);
}
builder.field(Field.SUBJECT.getPreferredName(), subject);
builder.field(Field.TEXT_BODY.getPreferredName(), textBody);
@ -492,7 +492,7 @@ public class Email implements ToXContent {
public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
builder.startArray();
for (Address address : addresses) {
builder.value(address);
address.toXContent(builder, params);
}
return builder.endArray();
}

34
src/main/java/org/elasticsearch/watcher/actions/email/service/EmailTemplate.java
View File

@ -155,31 +155,47 @@ public class EmailTemplate implements ToXContent {
public XContentBuilder xContentBody(XContentBuilder builder, Params params) throws IOException {
if (from != null) {
builder.field(Email.Field.FROM.getPreferredName(), from);
builder.field(Email.Field.FROM.getPreferredName(), from, params);
}
if (replyTo != null) {
builder.field(Email.Field.REPLY_TO.getPreferredName(), (Object[]) replyTo);
builder.startArray(Email.Field.REPLY_TO.getPreferredName());
for (Template template : replyTo) {
template.toXContent(builder, params);
}
builder.endArray();
}
if (priority != null) {
builder.field(Email.Field.PRIORITY.getPreferredName(), priority);
builder.field(Email.Field.PRIORITY.getPreferredName(), priority, params);
}
if (to != null) {
builder.field(Email.Field.TO.getPreferredName(), (Object[]) to);
builder.startArray(Email.Field.TO.getPreferredName());
for (Template template : to) {
template.toXContent(builder, params);
}
builder.endArray();
}
if (cc != null) {
builder.field(Email.Field.CC.getPreferredName(), (Object[]) cc);
builder.startArray(Email.Field.CC.getPreferredName());
for (Template template : cc) {
template.toXContent(builder, params);
}
builder.endArray();
}
if (bcc != null) {
builder.field(Email.Field.BCC.getPreferredName(), (Object[]) bcc);
builder.startArray(Email.Field.BCC.getPreferredName());
for (Template template : bcc) {
template.toXContent(builder, params);
}
builder.endArray();
}
if (subject != null) {
builder.field(Email.Field.SUBJECT.getPreferredName(), subject);
builder.field(Email.Field.SUBJECT.getPreferredName(), subject, params);
}
if (textBody != null) {
builder.field(Email.Field.TEXT_BODY.getPreferredName(), textBody);
builder.field(Email.Field.TEXT_BODY.getPreferredName(), textBody, params);
}
if (htmlBody != null) {
builder.field(Email.Field.HTML_BODY.getPreferredName(), htmlBody);
builder.field(Email.Field.HTML_BODY.getPreferredName(), htmlBody, params);
}
return builder;
}

10
src/main/java/org/elasticsearch/watcher/actions/email/service/InternalEmailService.java
View File

@ -12,6 +12,8 @@ import org.elasticsearch.common.logging.ESLogger;
import org.elasticsearch.common.settings.ImmutableSettings;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.node.settings.NodeSettingsService;
import org.elasticsearch.watcher.shield.WatcherSettingsFilter;
import org.elasticsearch.watcher.support.secret.SecretService;
import javax.mail.MessagingException;
@ -20,17 +22,21 @@ import javax.mail.MessagingException;
*/
public class InternalEmailService extends AbstractLifecycleComponent<InternalEmailService> implements EmailService {
private final SecretService secretService;
private volatile Accounts accounts;
@Inject
public InternalEmailService(Settings settings, NodeSettingsService nodeSettingsService) {
public InternalEmailService(Settings settings, SecretService secretService, NodeSettingsService nodeSettingsService, WatcherSettingsFilter settingsFilter) {
super(settings);
this.secretService = secretService;
nodeSettingsService.addListener(new NodeSettingsService.Listener() {
@Override
public void onRefreshSettings(Settings settings) {
reset(settings);
}
});
settingsFilter.filterOut("watcher.actions.email.service.account.*.smtp.password");
}
@Override
@ -79,7 +85,7 @@ public class InternalEmailService extends AbstractLifecycleComponent<InternalEma
}
protected Accounts createAccounts(Settings settings, ESLogger logger) {
return new Accounts(settings, logger);
return new Accounts(settings, secretService, logger);
}
}

4
src/main/java/org/elasticsearch/watcher/actions/index/IndexAction.java
View File

@ -126,7 +126,7 @@ public class IndexAction implements Action {
@Override
protected XContentBuilder xContentBody(XContentBuilder builder, Params params) throws IOException {
if (response != null) {
builder.field(Field.RESPONSE.getPreferredName(), response());
builder.field(Field.RESPONSE.getPreferredName(), response, params);
}
return builder;
}
@ -181,7 +181,7 @@ public class IndexAction implements Action {
return builder.startObject(Field.SIMULATED_REQUEST.getPreferredName())
.field(Field.INDEX.getPreferredName(), index)
.field(Field.DOC_TYPE.getPreferredName(), docType)
.field(Field.SOURCE.getPreferredName(), source)
.field(Field.SOURCE.getPreferredName(), source, params)
.endObject();
}
}

4
src/main/java/org/elasticsearch/watcher/actions/logging/LoggingAction.java
View File

@ -63,8 +63,8 @@ public class LoggingAction implements Action {
if (category != null) {
builder.field(Field.CATEGORY.getPreferredName(), category);
}
builder.field(Field.LEVEL.getPreferredName(), level);
builder.field(Field.TEXT.getPreferredName(), text);
builder.field(Field.LEVEL.getPreferredName(), level, params);
builder.field(Field.TEXT.getPreferredName(), text, params);
return builder.endObject();
}

6
src/main/java/org/elasticsearch/watcher/actions/webhook/WebhookAction.java
View File

@ -98,8 +98,8 @@ public class WebhookAction implements Action {
@Override
protected XContentBuilder xContentBody(XContentBuilder builder, Params params) throws IOException {
return builder.field(Field.REQUEST.getPreferredName(), request)
.field(Field.RESPONSE.getPreferredName(), response);
return builder.field(Field.REQUEST.getPreferredName(), request, params)
.field(Field.RESPONSE.getPreferredName(), response, params);
}
}
@ -137,7 +137,7 @@ public class WebhookAction implements Action {
@Override
protected XContentBuilder xContentBody(XContentBuilder builder, Params params) throws IOException {
return builder.field(Field.SIMULATED_REQUEST.getPreferredName(), request);
return builder.field(Field.SIMULATED_REQUEST.getPreferredName(), request, params);
}
}

14
src/main/java/org/elasticsearch/watcher/client/WatchSourceBuilder.java
View File

@ -118,20 +118,20 @@ public class WatchSourceBuilder implements ToXContent {
throw new BuilderException("failed to build watch source. no trigger defined");
}
builder.startObject(Watch.Parser.TRIGGER_FIELD.getPreferredName())
.field(trigger.type(), trigger)
.field(trigger.type(), trigger, params)
.endObject();
builder.startObject(Watch.Parser.INPUT_FIELD.getPreferredName())
.field(input.type(), input)
.field(input.type(), input, params)
.endObject();
builder.startObject(Watch.Parser.CONDITION_FIELD.getPreferredName())
.field(condition.type(), condition)
.field(condition.type(), condition, params)
.endObject();
if (transform != null) {
builder.startObject(Watch.Parser.TRANSFORM_FIELD.getPreferredName())
.field(transform.type(), transform)
.field(transform.type(), transform, params)
.endObject();
}
@ -141,7 +141,7 @@ public class WatchSourceBuilder implements ToXContent {
builder.startObject(Watch.Parser.ACTIONS_FIELD.getPreferredName());
for (Map.Entry<String, TransformedAction> entry : actions.entrySet()) {
builder.field(entry.getKey(), entry.getValue());
builder.field(entry.getKey(), entry.getValue(), params);
}
builder.endObject();
@ -183,10 +183,10 @@ public class WatchSourceBuilder implements ToXContent {
builder.startObject();
if (transform != null) {
builder.startObject(Transform.Field.TRANSFORM.getPreferredName())
.field(transform.type(), transform)
.field(transform.type(), transform, params)
.endObject();
}
builder.field(action.type(), action);
builder.field(action.type(), action, params);
return builder.endObject();
}
}

2
src/main/java/org/elasticsearch/watcher/condition/Condition.java
View File

@ -18,7 +18,7 @@ public interface Condition extends ToXContent {
abstract class Result implements ToXContent {
private final String type;
private final boolean met;
protected final boolean met;
public Result(String type, boolean met) {
this.type = type;

2
src/main/java/org/elasticsearch/watcher/condition/script/ScriptCondition.java
View File

@ -82,7 +82,7 @@ public class ScriptCondition implements Condition {
@Override
public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
return builder.startObject()
.field(Field.MET.getPreferredName(), met())
.field(Field.MET.getPreferredName(), met)
.endObject();
}

8
src/main/java/org/elasticsearch/watcher/history/WatchRecord.java
View File

@ -133,9 +133,11 @@ public class WatchRecord implements ToXContent {
builder.startObject();
builder.field(Parser.WATCH_ID_FIELD.getPreferredName(), name);
builder.startObject(Parser.TRIGGER_EVENT_FIELD.getPreferredName())
.field(triggerEvent.type(), triggerEvent)
.field(triggerEvent.type(), triggerEvent, params)
.endObject();
builder.startObject(Watch.Parser.CONDITION_FIELD.getPreferredName())
.field(condition.type(), condition, params)
.endObject();
builder.startObject(Watch.Parser.CONDITION_FIELD.getPreferredName()).field(condition.type(), condition, params).endObject();
builder.field(Parser.STATE_FIELD.getPreferredName(), state.id());
if (message != null) {
@ -146,7 +148,7 @@ public class WatchRecord implements ToXContent {
}
if (execution != null) {
builder.field(Parser.WATCH_EXECUTION_FIELD.getPreferredName(), execution);
builder.field(Parser.WATCH_EXECUTION_FIELD.getPreferredName(), execution, params);
}
builder.endObject();

2
src/main/java/org/elasticsearch/watcher/input/ExecutableInput.java
View File

@ -32,7 +32,7 @@ public abstract class ExecutableInput<I extends Input, R extends Input.Result> i
return input.type();
}
I input() {
public I input() {
return input;
}

2
src/main/java/org/elasticsearch/watcher/input/Input.java
View File

@ -41,7 +41,7 @@ public interface Input extends ToXContent {
@Override
public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
builder.startObject()
.field(Field.PAYLOAD.getPreferredName(), payload);
.field(Field.PAYLOAD.getPreferredName(), payload, params);
toXContentBody(builder, params);
return builder.endObject();
}

4
src/main/java/org/elasticsearch/watcher/input/http/HttpInput.java
View File

@ -51,7 +51,7 @@ public class HttpInput implements Input {
@Override
public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
builder.startObject();
builder.field(Field.REQUEST.getPreferredName(), request);
builder.field(Field.REQUEST.getPreferredName(), request, params);
if (extractKeys != null) {
builder.field(Field.EXTRACT.getPreferredName(), extractKeys);
}
@ -124,7 +124,7 @@ public class HttpInput implements Input {
@Override
protected XContentBuilder toXContentBody(XContentBuilder builder, Params params) throws IOException {
return builder.field(Field.SENT_REQUEST.getPreferredName(), sentRequest)
return builder.field(Field.SENT_REQUEST.getPreferredName(), sentRequest, params)
.field(Field.HTTP_STATUS.getPreferredName(), httpStatus);
}

2
src/main/java/org/elasticsearch/watcher/input/simple/SimpleInput.java
View File

@ -51,7 +51,7 @@ public class SimpleInput implements Input {
@Override
public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
return builder.value(payload);
return payload.toXContent(builder, params);
}
public static SimpleInput parse(String watchId, XContentParser parser) throws IOException {

3
src/main/java/org/elasticsearch/watcher/rest/action/RestAckWatchAction.java
View File

@ -6,6 +6,7 @@
package org.elasticsearch.watcher.rest.action;
import org.elasticsearch.watcher.rest.WatcherRestHandler;
import org.elasticsearch.watcher.support.xcontent.WatcherParams;
import org.elasticsearch.watcher.watch.Watch;
import org.elasticsearch.watcher.client.WatcherClient;
import org.elasticsearch.watcher.transport.actions.ack.AckWatchRequest;
@ -36,7 +37,7 @@ public class RestAckWatchAction extends WatcherRestHandler {
@Override
public RestResponse buildResponse(AckWatchResponse response, XContentBuilder builder) throws Exception {
return new BytesRestResponse(RestStatus.OK, builder.startObject()
.field(Watch.Parser.STATUS_FIELD.getPreferredName(), response.getStatus())
.field(Watch.Parser.STATUS_FIELD.getPreferredName(), response.getStatus(), WatcherParams.HIDE_SECRETS)
.endObject());
}

12
src/main/java/org/elasticsearch/watcher/shield/ShieldIntegration.java
View File

@ -10,6 +10,7 @@ import org.elasticsearch.common.inject.Inject;
import org.elasticsearch.common.inject.Injector;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.shield.ShieldPlugin;
import org.elasticsearch.shield.ShieldSettingsFilter;
import org.elasticsearch.shield.ShieldVersion;
import org.elasticsearch.shield.authc.AuthenticationService;
import org.elasticsearch.transport.TransportMessage;
@ -26,6 +27,7 @@ public class ShieldIntegration {
private final boolean enabled;
private final Object authcService;
private final Object userHolder;
private final Object settingsFilter;
@Inject
public ShieldIntegration(Settings settings, Injector injector) {
@ -33,6 +35,8 @@ public class ShieldIntegration {
enabled = installed && ShieldPlugin.shieldEnabled(settings);
authcService = enabled ? injector.getInstance(AuthenticationService.class) : null;
userHolder = enabled ? injector.getInstance(WatcherUserHolder.class) : null;
settingsFilter = enabled ? injector.getInstance(ShieldSettingsFilter.class) : null;
}
public boolean installed() {
@ -49,6 +53,12 @@ public class ShieldIntegration {
}
}
public void filterOutSettings(String... patterns) {
if (settingsFilter != null) {
((ShieldSettingsFilter) settingsFilter).filterOut(patterns);
}
}
static boolean installed(Settings settings) {
try {
Class clazz = settings.getClassLoader().loadClass("org.elasticsearch.shield.ShieldPlugin");
@ -71,7 +81,7 @@ public class ShieldIntegration {
}
}
static boolean enabled(Settings settings) {
public static boolean enabled(Settings settings) {
return installed(settings) && ShieldPlugin.shieldEnabled(settings);
}

38
src/main/java/org/elasticsearch/watcher/shield/ShieldSecretService.java Normal file
View File

@ -0,0 +1,38 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.watcher.shield;
import org.elasticsearch.common.component.AbstractComponent;
import org.elasticsearch.common.inject.Inject;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.shield.crypto.CryptoService;
import org.elasticsearch.watcher.support.secret.SecretService;
/**
*
*/
public class ShieldSecretService extends AbstractComponent implements SecretService {
private final CryptoService cryptoService;
private final boolean encryptSensitiveData;
@Inject
public ShieldSecretService(Settings settings, CryptoService cryptoService) {
super(settings);
this.encryptSensitiveData = componentSettings.getAsBoolean("encrypt_sensitive_data", false);
this.cryptoService = cryptoService;
}
@Override
public char[] encrypt(char[] text) {
return encryptSensitiveData ? cryptoService.encrypt(text) : text;
}
@Override
public char[] decrypt(char[] text) {
return encryptSensitiveData ? cryptoService.decrypt(text) : text;
}
}

43
src/main/java/org/elasticsearch/watcher/shield/WatcherSettingsFilter.java Normal file
View File

@ -0,0 +1,43 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.watcher.shield;
import org.elasticsearch.common.inject.Inject;
/**
*
*/
public interface WatcherSettingsFilter {
void filterOut(String... patterns);
class Noop implements WatcherSettingsFilter {
public static Noop INSTANCE = new Noop();
private Noop() {
}
@Override
public void filterOut(String... patterns) {
}
}
class Shield implements WatcherSettingsFilter {
private final ShieldIntegration shieldIntegration;
@Inject
public Shield(ShieldIntegration shieldIntegration) {
this.shieldIntegration = shieldIntegration;
}
@Override
public void filterOut(String... patterns) {
shieldIntegration.filterOutSettings(patterns);
}
}
}

6
src/main/java/org/elasticsearch/watcher/shield/WatcherShieldModule.java
View File

@ -61,5 +61,11 @@ public class WatcherShieldModule extends AbstractModule implements PreProcessMod
protected void configure() {
bind(ShieldIntegration.class).asEagerSingleton();
bind(WatcherUserHolder.class).toProvider(Providers.of(userHolder));
if (enabled) {
bind(WatcherSettingsFilter.Shield.class).asEagerSingleton();
bind(WatcherSettingsFilter.class).to(WatcherSettingsFilter.Shield.class);
} else {
bind(WatcherSettingsFilter.class).toInstance(WatcherSettingsFilter.Noop.INSTANCE);
}
}
}

23
src/main/java/org/elasticsearch/watcher/support/http/HttpClient.java
View File

@ -12,6 +12,8 @@ import org.elasticsearch.common.component.AbstractComponent;
import org.elasticsearch.common.inject.Inject;
import org.elasticsearch.common.io.Streams;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.watcher.support.http.auth.ApplicableHttpAuth;
import org.elasticsearch.watcher.support.http.auth.HttpAuthRegistry;
import javax.net.ssl.*;
import java.io.IOException;
@ -42,10 +44,12 @@ public class HttpClient extends AbstractComponent {
private static final String SETTINGS_SSL_SHIELD_TRUSTSTORE_ALGORITHM = SETTINGS_SSL_SHIELD_PREFIX + "truststore.algorithm";
final SSLSocketFactory sslSocketFactory;
final HttpAuthRegistry httpAuthRegistry;
@Inject
public HttpClient(Settings settings) {
public HttpClient(Settings settings, HttpAuthRegistry httpAuthRegistry) {
super(settings);
this.httpAuthRegistry = httpAuthRegistry;
if (!settings.getByPrefix(SETTINGS_SSL_PREFIX).getAsMap().isEmpty() ||
!settings.getByPrefix(SETTINGS_SSL_SHIELD_PREFIX).getAsMap().isEmpty()) {
sslSocketFactory = createSSLSocketFactory(settings);
@ -93,8 +97,9 @@ public class HttpClient extends AbstractComponent {
}
}
if (request.auth() != null) {
logger.debug("applying auth headers");
request.auth().update(urlConnection);
logger.trace("applying auth headers");
ApplicableHttpAuth applicableAuth = httpAuthRegistry.createApplicable(request.auth);
applicableAuth.apply(urlConnection);
}
urlConnection.setUseCaches(false);
urlConnection.setRequestProperty("Accept-Charset", Charsets.UTF_8.name());
@ -110,11 +115,11 @@ public class HttpClient extends AbstractComponent {
byte[] body = Streams.copyToByteArray(urlConnection.getInputStream());
HttpResponse response = new HttpResponse(urlConnection.getResponseCode(), body);
logger.debug("http status code: {}", response.status());
logger.debug("http status code [{}]", response.status());
return response;
}
/** SSL Initialization * */
/** SSL Initialization **/
private SSLSocketFactory createSSLSocketFactory(Settings settings) {
SSLContext sslContext;
// Initialize sslContext
@ -132,10 +137,10 @@ public class HttpClient extends AbstractComponent {
trustStoreAlgorithm = TrustManagerFactory.getDefaultAlgorithm();
}
logger.debug("SSL: using trustStore[{}], trustAlgorithm[{}]", trustStore, trustStoreAlgorithm);
logger.debug("using trustStore [{}] and trustAlgorithm [{}]", trustStore, trustStoreAlgorithm);
Path path = Paths.get(trustStore);
if (Files.notExists(path)) {
throw new ElasticsearchIllegalStateException("Truststore at path [" + trustStore + "] does not exist");
throw new ElasticsearchIllegalStateException("could not find truststore [" + trustStore + "]");
}
KeyManager[] keyManagers;
@ -156,13 +161,13 @@ public class HttpClient extends AbstractComponent {
// Retrieve the trust managers from the factory
trustManagers = trustFactory.getTrustManagers();
} catch (Exception e) {
throw new RuntimeException("Failed to initialize a TrustManagerFactory", e);
throw new RuntimeException("http client failed to initialize a TrustManagerFactory", e);
}
sslContext = SSLContext.getInstance(sslContextProtocol);
sslContext.init(keyManagers, trustManagers, new SecureRandom());
} catch (Exception e) {
throw new RuntimeException("[http.client] failed to initialize the SSLContext", e);
throw new RuntimeException("http client failed to initialize the SSLContext", e);
}
return sslContext.getSocketFactory();
}

11
src/main/java/org/elasticsearch/watcher/support/http/HttpMethod.java
View File

@ -6,12 +6,15 @@
package org.elasticsearch.watcher.support.http;
import org.elasticsearch.ElasticsearchIllegalArgumentException;
import org.elasticsearch.common.xcontent.ToXContent;
import org.elasticsearch.common.xcontent.XContentBuilder;
import java.io.IOException;
import java.util.Locale;
/**
*/
public enum HttpMethod {
public enum HttpMethod implements ToXContent {
HEAD("HEAD"),
GET("GET"),
@ -46,4 +49,10 @@ public enum HttpMethod {
throw new ElasticsearchIllegalArgumentException("unsupported http method [" + value + "]");
}
}
@Override
public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
return builder.value(name().toLowerCase(Locale.ROOT));
}
}

11
src/main/java/org/elasticsearch/watcher/support/http/HttpRequest.java
View File

@ -14,6 +14,7 @@ import org.elasticsearch.common.xcontent.XContentBuilder;
import org.elasticsearch.common.xcontent.XContentParser;
import org.elasticsearch.watcher.WatcherException;
import org.elasticsearch.watcher.support.WatcherUtils;
import org.elasticsearch.watcher.support.http.auth.ApplicableHttpAuth;
import org.elasticsearch.watcher.support.http.auth.HttpAuth;
import org.elasticsearch.watcher.support.http.auth.HttpAuthRegistry;
@ -32,7 +33,9 @@ public class HttpRequest implements ToXContent {
final @Nullable HttpAuth auth;
final @Nullable String body;
public HttpRequest(String host, int port, @Nullable Scheme scheme, @Nullable HttpMethod method, @Nullable String path, @Nullable ImmutableMap<String, String> params, @Nullable ImmutableMap<String, String> headers, @Nullable HttpAuth auth, @Nullable String body) {
public HttpRequest(String host, int port, @Nullable Scheme scheme, @Nullable HttpMethod method, @Nullable String path,
@Nullable ImmutableMap<String, String> params, @Nullable ImmutableMap<String, String> headers,
@Nullable HttpAuth auth, @Nullable String body) {
this.host = host;
this.port = port;
this.scheme = scheme != null ? scheme : Scheme.HTTP;
@ -89,8 +92,8 @@ public class HttpRequest implements ToXContent {
builder.startObject();
builder.field(Field.HOST.getPreferredName(), host);
builder.field(Field.PORT.getPreferredName(), port);
builder.field(Field.SCHEME.getPreferredName(), scheme);
builder.field(Field.METHOD.getPreferredName(), method);
builder.field(Field.SCHEME.getPreferredName(), scheme, params);
builder.field(Field.METHOD.getPreferredName(), method, params);
if (path != null) {
builder.field(Field.PATH.getPreferredName(), path);
}
@ -101,7 +104,7 @@ public class HttpRequest implements ToXContent {
builder.field(Field.HEADERS.getPreferredName(), headers);
}
if (auth != null) {
builder.field(Field.AUTH.getPreferredName(), auth);
builder.field(Field.AUTH.getPreferredName(), auth, params);
}
if (body != null) {
builder.field(Field.BODY.getPreferredName(), body);

24
src/main/java/org/elasticsearch/watcher/support/http/HttpRequestTemplate.java
View File

@ -121,24 +121,34 @@ public class HttpRequestTemplate implements ToXContent {
public XContentBuilder toXContent(XContentBuilder builder, ToXContent.Params params) throws IOException {
builder.startObject();
builder.field(Parser.SCHEME_FIELD.getPreferredName(), scheme);
builder.field(Parser.SCHEME_FIELD.getPreferredName(), scheme, params);
builder.field(Parser.HOST_FIELD.getPreferredName(), host);
builder.field(Parser.PORT_FIELD.getPreferredName(), port);
builder.field(Parser.METHOD_FIELD.getPreferredName(), method);
builder.field(Parser.METHOD_FIELD.getPreferredName(), method, params);
if (path != null) {
builder.field(Parser.PATH_FIELD.getPreferredName(), path);
builder.field(Parser.PATH_FIELD.getPreferredName(), path, params);
}
if (this.params != null) {
builder.field(Parser.PARAMS_FIELD.getPreferredName(), this.params);
builder.startObject(Parser.PARAMS_FIELD.getPreferredName());
for (Map.Entry<String, Template> entry : this.params.entrySet()) {
builder.field(entry.getKey(), entry.getValue(), params);
}
builder.endObject();
}
if (headers != null) {
builder.field(Parser.HEADERS_FIELD.getPreferredName(), headers);
builder.startObject(Parser.HEADERS_FIELD.getPreferredName());
for (Map.Entry<String, Template> entry : headers.entrySet()) {
builder.field(entry.getKey(), entry.getValue(), params);
}
builder.endObject();
}
if (auth != null) {
builder.field(Parser.AUTH_FIELD.getPreferredName(), auth);
builder.startObject(Parser.AUTH_FIELD.getPreferredName())
.field(auth.type(), auth, params)
.endObject();
}
if (body != null) {
builder.field(Parser.BODY_FIELD.getPreferredName(), body);
builder.field(Parser.BODY_FIELD.getPreferredName(), body, params);
}
return builder.endObject();
}

11
src/main/java/org/elasticsearch/watcher/support/http/Scheme.java
View File

@ -6,10 +6,13 @@
package org.elasticsearch.watcher.support.http;
import org.elasticsearch.ElasticsearchIllegalArgumentException;
import org.elasticsearch.common.xcontent.ToXContent;
import org.elasticsearch.common.xcontent.XContentBuilder;
import java.io.IOException;
import java.util.Locale;
public enum Scheme {
public enum Scheme implements ToXContent {
HTTP("http"),
HTTPS("https");
@ -35,4 +38,10 @@ public enum Scheme {
throw new ElasticsearchIllegalArgumentException("unsupported http scheme [" + value + "]");
}
}
@Override
public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
return builder.value(name().toLowerCase(Locale.ROOT));
}
}

47
src/main/java/org/elasticsearch/watcher/support/http/auth/ApplicableHttpAuth.java Normal file
View File

@ -0,0 +1,47 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.watcher.support.http.auth;
import org.elasticsearch.common.xcontent.ToXContent;
import org.elasticsearch.common.xcontent.XContentBuilder;
import java.io.IOException;
import java.net.HttpURLConnection;
public abstract class ApplicableHttpAuth<Auth extends HttpAuth> implements ToXContent {
private final Auth auth;
public ApplicableHttpAuth(Auth auth) {
this.auth = auth;
}
public final String type() {
return auth.type();
}
public abstract void apply(HttpURLConnection connection);
@Override
public final XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
return auth.toXContent(builder, params);
}
@Override
public boolean equals(Object o) {
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
ApplicableHttpAuth<?> that = (ApplicableHttpAuth<?>) o;
return auth.equals(that.auth);
}
@Override
public int hashCode() {
return auth.hashCode();
}
}

12
src/main/java/org/elasticsearch/watcher/support/http/auth/AuthModule.java
View File

@ -7,6 +7,9 @@ package org.elasticsearch.watcher.support.http.auth;
import org.elasticsearch.common.inject.AbstractModule;
import org.elasticsearch.common.inject.multibindings.MapBinder;
import org.elasticsearch.watcher.support.http.auth.basic.ApplicableBasicAuth;
import org.elasticsearch.watcher.support.http.auth.basic.BasicAuth;
import org.elasticsearch.watcher.support.http.auth.basic.BasicAuthFactory;
/**
*/
@ -14,9 +17,12 @@ public class AuthModule extends AbstractModule {
@Override
protected void configure() {
MapBinder<String, HttpAuth.Parser> parsersBinder = MapBinder.newMapBinder(binder(), String.class, HttpAuth.Parser.class);
bind(BasicAuth.Parser.class).asEagerSingleton();
parsersBinder.addBinding(BasicAuth.TYPE).to(BasicAuth.Parser.class);
MapBinder<String, HttpAuthFactory> parsersBinder = MapBinder.newMapBinder(binder(), String.class, HttpAuthFactory.class);
bind(BasicAuthFactory.class).asEagerSingleton();
parsersBinder.addBinding(BasicAuth.TYPE).to(BasicAuthFactory.class);
bind(HttpAuthRegistry.class).asEagerSingleton();
}
}

112
src/main/java/org/elasticsearch/watcher/support/http/auth/BasicAuth.java
View File

@ -1,112 +0,0 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.watcher.support.http.auth;
import org.elasticsearch.ElasticsearchParseException;
import org.elasticsearch.common.Base64;
import org.elasticsearch.common.ParseField;
import org.elasticsearch.common.base.Charsets;
import org.elasticsearch.common.xcontent.XContentBuilder;
import org.elasticsearch.common.xcontent.XContentParser;
import java.io.IOException;
import java.net.HttpURLConnection;
/**
*/
public class BasicAuth extends HttpAuth {
public static final String TYPE = "basic";
private final String username;
private final String password;
private final String basicAuth;
public BasicAuth(String username, String password) {
this.username = username;
this.password = password;
basicAuth = "Basic " + Base64.encodeBytes((username + ":" + password).getBytes(Charsets.UTF_8));
}
public String type() {
return TYPE;
}
@Override
public XContentBuilder innerToXContent(XContentBuilder builder, Params params) throws IOException {
builder.startObject();
builder.field(Parser.USERNAME_FIELD.getPreferredName(), username);
builder.field(Parser.PASSWORD_FIELD.getPreferredName(), password);
return builder.endObject();
}
public void update(HttpURLConnection connection) {
connection.setRequestProperty("Authorization", basicAuth);
}
@Override
public boolean equals(Object o) {
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
BasicAuth basicAuth = (BasicAuth) o;
if (!password.equals(basicAuth.password)) return false;
if (!username.equals(basicAuth.username)) return false;
return true;
}
@Override
public int hashCode() {
int result = username.hashCode();
result = 31 * result + password.hashCode();
return result;
}
public static class Parser implements HttpAuth.Parser<BasicAuth> {
static final ParseField USERNAME_FIELD = new ParseField("username");
static final ParseField PASSWORD_FIELD = new ParseField("password");
public String type() {
return TYPE;
}
public BasicAuth parse(XContentParser parser) throws IOException {
String username = null;
String password = null;
String fieldName = null;
XContentParser.Token token;
while ((token = parser.nextToken()) != XContentParser.Token.END_OBJECT) {
if (token == XContentParser.Token.FIELD_NAME) {
fieldName = parser.currentName();
} else if (token == XContentParser.Token.VALUE_STRING) {
if (USERNAME_FIELD.getPreferredName().equals(fieldName)) {
username = parser.text();
} else if (PASSWORD_FIELD.getPreferredName().equals(fieldName)) {
password = parser.text();
} else {
throw new ElasticsearchParseException("unsupported field [" + fieldName + "]");
}
} else {
throw new ElasticsearchParseException("unsupported token [" + token + "]");
}
}
if (username == null) {
throw new HttpAuthException("username is a required option");
}
if (password == null) {
throw new HttpAuthException("password is a required option");
}
return new BasicAuth(username, password);
}
}
}

32
src/main/java/org/elasticsearch/watcher/support/http/auth/HttpAuth.java
View File

@ -6,34 +6,12 @@
package org.elasticsearch.watcher.support.http.auth;
import org.elasticsearch.common.xcontent.ToXContent;
import org.elasticsearch.common.xcontent.XContentBuilder;
import org.elasticsearch.common.xcontent.XContentParser;
import java.io.IOException;
import java.net.HttpURLConnection;
/**
*
*/
public interface HttpAuth extends ToXContent {
public abstract class HttpAuth implements ToXContent {
public abstract String type();
public abstract void update(HttpURLConnection connection);
@Override
public final XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
builder.startObject();
builder.field(type());
builder = innerToXContent(builder, params);
return builder.endObject();
}
public abstract XContentBuilder innerToXContent(XContentBuilder builder, Params params) throws IOException;
public static interface Parser<Auth extends HttpAuth> {
String type();
Auth parse(XContentParser parser) throws IOException;
}
String type();
}

8
src/main/java/org/elasticsearch/watcher/support/http/auth/HttpAuthException.java
View File

@ -11,11 +11,11 @@ import org.elasticsearch.watcher.WatcherException;
*/
public class HttpAuthException extends WatcherException {
public HttpAuthException(String msg) {
super(msg);
public HttpAuthException(String msg, Object... args) {
super(msg, args);
}
public HttpAuthException(String msg, Throwable cause) {
super(msg, cause);
public HttpAuthException(String msg, Throwable cause, Object... args) {
super(msg, cause, args);
}
}

28
src/main/java/org/elasticsearch/watcher/support/http/auth/HttpAuthFactory.java Normal file
View File

@ -0,0 +1,28 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.watcher.support.http.auth;
import org.elasticsearch.common.xcontent.XContentParser;
import java.io.IOException;
/**
*
*/
public abstract class HttpAuthFactory<Auth extends HttpAuth, AAuth extends ApplicableHttpAuth<Auth>> {
public abstract String type();
public abstract Auth parse(XContentParser parser) throws IOException;
public abstract AAuth createApplicable(Auth auth);
public AAuth parseApplicable(XContentParser parser) throws IOException {
Auth auth = parse(parser);
return createApplicable(auth);
}
}

20
src/main/java/org/elasticsearch/watcher/support/http/auth/HttpAuthRegistry.java
View File

@ -17,11 +17,11 @@ import java.util.Map;
*/
public class HttpAuthRegistry {
private final ImmutableMap<String, HttpAuth.Parser> parsers;
private final ImmutableMap<String, HttpAuthFactory> factories;
@Inject
public HttpAuthRegistry(Map<String, HttpAuth.Parser> parsers) {
this.parsers = ImmutableMap.copyOf(parsers);
public HttpAuthRegistry(Map<String, HttpAuthFactory> factories) {
this.factories = ImmutableMap.copyOf(factories);
}
public HttpAuth parse(XContentParser parser) throws IOException {
@ -32,14 +32,22 @@ public class HttpAuthRegistry {
if (token == XContentParser.Token.FIELD_NAME) {
type = parser.currentName();
} else if (token == XContentParser.Token.START_OBJECT && type != null) {
HttpAuth.Parser inputParser = parsers.get(type);
if (inputParser == null) {
HttpAuthFactory factory = factories.get(type);
if (factory == null) {
throw new HttpAuthException("unknown http auth type [" + type + "]");
}
auth = inputParser.parse(parser);
auth = factory.parse(parser);
}
}
return auth;
}
public <A extends HttpAuth, AA extends ApplicableHttpAuth<A>> AA createApplicable(A auth) {
HttpAuthFactory factory = factories.get(auth.type());
if (factory == null) {
throw new HttpAuthException("unknown http auth type [{}]", auth.type());
}
return (AA) factory.createApplicable(auth);
}
}

34
src/main/java/org/elasticsearch/watcher/support/http/auth/basic/ApplicableBasicAuth.java Normal file
View File

@ -0,0 +1,34 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.watcher.support.http.auth.basic;
import org.elasticsearch.common.Base64;
import org.elasticsearch.common.base.Charsets;
import org.elasticsearch.watcher.support.http.auth.ApplicableHttpAuth;
import org.elasticsearch.watcher.support.secret.SecretService;
import java.net.HttpURLConnection;
/**
*/
public class ApplicableBasicAuth extends ApplicableHttpAuth<BasicAuth> {
private final String basicAuth;
public ApplicableBasicAuth(BasicAuth auth, SecretService service) {
super(auth);
basicAuth = headerValue(auth.username, auth.password.text(service));
}
public static String headerValue(String username, char[] password) {
return "Basic " + Base64.encodeBytes((username + ":" + new String(password)).getBytes(Charsets.UTF_8));
}
public void apply(HttpURLConnection connection) {
connection.setRequestProperty("Authorization", basicAuth);
}
}

112
src/main/java/org/elasticsearch/watcher/support/http/auth/basic/BasicAuth.java Normal file
View File

@ -0,0 +1,112 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.watcher.support.http.auth.basic;
import org.elasticsearch.ElasticsearchParseException;
import org.elasticsearch.common.ParseField;
import org.elasticsearch.common.xcontent.XContentBuilder;
import org.elasticsearch.common.xcontent.XContentParser;
import org.elasticsearch.watcher.support.http.auth.HttpAuth;
import org.elasticsearch.watcher.support.http.auth.HttpAuthException;
import org.elasticsearch.watcher.support.secret.Secret;
import org.elasticsearch.watcher.support.secret.SensitiveXContentParser;
import org.elasticsearch.watcher.support.xcontent.WatcherParams;
import java.io.IOException;
/**
*
*/
public class BasicAuth implements HttpAuth {
public static final String TYPE = "basic";
final String username;
final Secret password;
public BasicAuth(String username, char[] password) {
this(username, new Secret(password));
}
public BasicAuth(String username, Secret password) {
this.username = username;
this.password = password;
}
@Override
public String type() {
return TYPE;
}
public String getUsername() {
return username;
}
@Override
public boolean equals(Object o) {
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
BasicAuth basicAuth = (BasicAuth) o;
if (!username.equals(basicAuth.username)) return false;
return password.equals(basicAuth.password);
}
@Override
public int hashCode() {
int result = username.hashCode();
result = 31 * result + password.hashCode();
return result;
}
@Override
public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
builder.startObject();
builder.field(Field.USERNAME.getPreferredName(), username);
if (!WatcherParams.hideSecrets(params)) {
builder.field(Field.PASSWORD.getPreferredName(), password, params);
}
return builder.endObject();
}
public static BasicAuth parse(XContentParser parser) throws IOException {
String username = null;
Secret password = null;
String fieldName = null;
XContentParser.Token token;
while ((token = parser.nextToken()) != XContentParser.Token.END_OBJECT) {
if (token == XContentParser.Token.FIELD_NAME) {
fieldName = parser.currentName();
} else if (token == XContentParser.Token.VALUE_STRING) {
if (Field.USERNAME.getPreferredName().equals(fieldName)) {
username = parser.text();
} else if (Field.PASSWORD.getPreferredName().equals(fieldName)) {
password = SensitiveXContentParser.secret(parser);
} else {
throw new ElasticsearchParseException("unsupported field [" + fieldName + "]");
}
} else {
throw new ElasticsearchParseException("unsupported token [" + token + "]");
}
}
if (username == null) {
throw new HttpAuthException("username is a required option");
}
if (password == null) {
throw new HttpAuthException("password is a required option");
}
return new BasicAuth(username, password);
}
interface Field {
ParseField USERNAME = new ParseField("username");
ParseField PASSWORD = new ParseField("password");
}
}

39
src/main/java/org/elasticsearch/watcher/support/http/auth/basic/BasicAuthFactory.java Normal file
View File

@ -0,0 +1,39 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.watcher.support.http.auth.basic;
import org.elasticsearch.common.inject.Inject;
import org.elasticsearch.common.xcontent.XContentParser;
import org.elasticsearch.watcher.support.http.auth.HttpAuthFactory;
import org.elasticsearch.watcher.support.secret.SecretService;
import java.io.IOException;
/**
*
*/
public class BasicAuthFactory extends HttpAuthFactory<BasicAuth, ApplicableBasicAuth> {
private final SecretService secretService;
@Inject
public BasicAuthFactory(SecretService secretService) {
this.secretService = secretService;
}
public String type() {
return BasicAuth.TYPE;
}
public BasicAuth parse(XContentParser parser) throws IOException {
return BasicAuth.parse(parser);
}
@Override
public ApplicableBasicAuth createApplicable(BasicAuth auth) {
return new ApplicableBasicAuth(auth, secretService);
}
}

49
src/main/java/org/elasticsearch/watcher/support/secret/Secret.java Normal file
View File

@ -0,0 +1,49 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.watcher.support.secret;
import org.elasticsearch.common.xcontent.ToXContent;
import org.elasticsearch.common.xcontent.XContentBuilder;
import java.io.IOException;
import java.util.Arrays;
/**
*
*/
public class Secret implements ToXContent {
protected final char[] text;
public Secret(char[] text) {
this.text = text;
}
public char[] text(SecretService service) {
return service.decrypt(text);
}
@Override
public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
return builder.value(new String(text));
}
@Override
public boolean equals(Object o) {
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
Secret secret = (Secret) o;
return Arrays.equals(text, secret.text);
}
@Override
public int hashCode() {
return Arrays.hashCode(text);
}
}

34
src/main/java/org/elasticsearch/watcher/support/secret/SecretModule.java Normal file
View File

@ -0,0 +1,34 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.watcher.support.secret;
import org.elasticsearch.common.inject.AbstractModule;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.watcher.shield.ShieldIntegration;
import org.elasticsearch.watcher.shield.ShieldSecretService;
/**
*
*/
public class SecretModule extends AbstractModule {
private final boolean shieldEnabled;
public SecretModule(Settings settings) {
shieldEnabled = ShieldIntegration.enabled(settings);
}
@Override
protected void configure() {
if (shieldEnabled) {
bind(ShieldSecretService.class).asEagerSingleton();
bind(SecretService.class).to(ShieldSecretService.class);
} else {
bind(SecretService.PlainText.class).asEagerSingleton();
bind(SecretService.class).to(SecretService.PlainText.class);
}
}
}

29
src/main/java/org/elasticsearch/watcher/support/secret/SecretService.java Normal file
View File

@ -0,0 +1,29 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.watcher.support.secret;
/**
*
*/
public interface SecretService {
char[] encrypt(char[] text);
char[] decrypt(char[] text);
class PlainText implements SecretService {
@Override
public char[] encrypt(char[] text) {
return text;
}
@Override
public char[] decrypt(char[] text) {
return text;
}
}
}

240
src/main/java/org/elasticsearch/watcher/support/secret/SensitiveXContentParser.java Normal file
View File

@ -0,0 +1,240 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.watcher.support.secret;
import org.apache.lucene.util.BytesRef;
import org.elasticsearch.ElasticsearchException;
import org.elasticsearch.common.xcontent.XContentParser;
import org.elasticsearch.common.xcontent.XContentType;
import java.io.IOException;
import java.util.Map;
/**
*
*/
public class SensitiveXContentParser implements XContentParser {
public static Secret secret(XContentParser parser) throws IOException {
char[] chars = parser.text().toCharArray();
if (parser instanceof SensitiveXContentParser) {
chars = ((SensitiveXContentParser) parser).secretService.encrypt(chars);
return new Secret(chars);
}
return new Secret(chars);
}
public static Secret secretOrNull(XContentParser parser) throws IOException {
String text = parser.textOrNull();
if (text == null) {
return null;
}
char[] chars = parser.text().toCharArray();
if (parser instanceof SensitiveXContentParser) {
chars = ((SensitiveXContentParser) parser).secretService.encrypt(text.toCharArray());
return new Secret(chars);
}
return new Secret(chars);
}
private final XContentParser parser;
private final SecretService secretService;
public SensitiveXContentParser(XContentParser parser, SecretService secretService) {
this.parser = parser;
this.secretService = secretService;
}
@Override
public XContentType contentType() {
return parser.contentType();
}
@Override
public Token nextToken() throws IOException {
return parser.nextToken();
}
@Override
public void skipChildren() throws IOException {
parser.skipChildren();
}
@Override
public Token currentToken() {
return parser.currentToken();
}
@Override
public String currentName() throws IOException {
return parser.currentName();
}
@Override
public Map<String, Object> map() throws IOException {
return parser.map();
}
@Override
public Map<String, Object> mapOrdered() throws IOException {
return parser.mapOrdered();
}
@Override
public Map<String, Object> mapAndClose() throws IOException {
return parser.mapAndClose();
}
@Override
public Map<String, Object> mapOrderedAndClose() throws IOException {
return parser.mapOrderedAndClose();
}
@Override
public String text() throws IOException {
return parser.text();
}
@Override
public String textOrNull() throws IOException {
return parser.textOrNull();
}
@Override
public BytesRef utf8BytesOrNull() throws IOException {
return parser.utf8BytesOrNull();
}
@Override
public BytesRef utf8Bytes() throws IOException {
return parser.utf8Bytes();
}
@Override @Deprecated
public BytesRef bytesOrNull() throws IOException {
return parser.bytesOrNull();
}
@Override @Deprecated
public BytesRef bytes() throws IOException {
return parser.bytes();
}
@Override
public Object objectText() throws IOException {
return parser.objectText();
}
@Override
public Object objectBytes() throws IOException {
return parser.objectBytes();
}
@Override
public boolean hasTextCharacters() {
return parser.hasTextCharacters();
}
@Override
public char[] textCharacters() throws IOException {
return parser.textCharacters();
}
@Override
public int textLength() throws IOException {
return parser.textLength();
}
@Override
public int textOffset() throws IOException {
return parser.textOffset();
}
@Override
public Number numberValue() throws IOException {
return parser.numberValue();
}
@Override
public NumberType numberType() throws IOException {
return parser.numberType();
}
@Override
public boolean estimatedNumberType() {
return parser.estimatedNumberType();
}
@Override
public short shortValue(boolean coerce) throws IOException {
return parser.shortValue(coerce);
}
@Override
public int intValue(boolean coerce) throws IOException {
return parser.intValue(coerce);
}
@Override
public long longValue(boolean coerce) throws IOException {
return parser.longValue(coerce);
}
@Override
public float floatValue(boolean coerce) throws IOException {
return parser.floatValue(coerce);
}
@Override
public double doubleValue(boolean coerce) throws IOException {
return parser.doubleValue(coerce);
}
@Override
public short shortValue() throws IOException {
return parser.shortValue();
}
@Override
public int intValue() throws IOException {
return parser.intValue();
}
@Override
public long longValue() throws IOException {
return parser.longValue();
}
@Override
public float floatValue() throws IOException {
return parser.floatValue();
}
@Override
public double doubleValue() throws IOException {
return parser.doubleValue();
}
@Override
public boolean isBooleanValue() throws IOException {
return parser.isBooleanValue();
}
@Override
public boolean booleanValue() throws IOException {
return parser.booleanValue();
}
@Override
public byte[] binaryValue() throws IOException {
return parser.binaryValue();
}
@Override
public void close() throws ElasticsearchException {
parser.close();
}
}

80
src/main/java/org/elasticsearch/watcher/support/xcontent/WatcherParams.java Normal file
View File

@ -0,0 +1,80 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.watcher.support.xcontent;
import org.elasticsearch.common.collect.ImmutableMap;
import org.elasticsearch.common.xcontent.ToXContent;
/**
*
*/
public class WatcherParams extends ToXContent.DelegatingMapParams {
public static final WatcherParams HIDE_SECRETS = WatcherParams.builder().hideSecrets(true).build();
static final String HIDE_SECRETS_KEY = "hide_secrets";
static final String COLLAPSE_ARRAYS_KEY = "collapse_arrays";
private ImmutableMap<String, String> params;
private WatcherParams(ImmutableMap<String, String> params, ToXContent.Params delegate) {
super(params, delegate);
}
public boolean hideSecrets() {
return paramAsBoolean(HIDE_SECRETS_KEY, false);
}
public boolean collapseArrays() {
return paramAsBoolean(COLLAPSE_ARRAYS_KEY, false);
}
public static WatcherParams wrap(ToXContent.Params params) {
return params instanceof WatcherParams ?
(WatcherParams) params :
new WatcherParams(ImmutableMap.<String, String>of(), params);
}
public static boolean hideSecrets(ToXContent.Params params) {
return wrap(params).hideSecrets();
}
public static boolean collapseArrays(ToXContent.Params params) {
return wrap(params).collapseArrays();
}
public static Builder builder() {
return builder(ToXContent.EMPTY_PARAMS);
}
public static Builder builder(ToXContent.Params delegate) {
return new Builder(delegate);
}
public static class Builder {
private final ToXContent.Params delegate;
private final ImmutableMap.Builder<String, String> params = ImmutableMap.builder();
private Builder(ToXContent.Params delegate) {
this.delegate = delegate;
}
public Builder hideSecrets(boolean hideSecrets) {
params.put(HIDE_SECRETS_KEY, String.valueOf(hideSecrets));
return this;
}
public Builder collapseArrays(boolean collapseArrays) {
params.put(COLLAPSE_ARRAYS_KEY, String.valueOf(collapseArrays));
return this;
}
public WatcherParams build() {
return new WatcherParams(params.build(), delegate);
}
}
}

2
src/main/java/org/elasticsearch/watcher/transform/Transform.java
View File

@ -40,7 +40,7 @@ public interface Transform extends ToXContent {
@Override
public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
builder.startObject();
builder.field(Field.PAYLOAD.getPreferredName(), payload);
builder.field(Field.PAYLOAD.getPreferredName(), payload, params);
xContentBody(builder, params);
return builder.endObject();
}

4
src/main/java/org/elasticsearch/watcher/transform/chain/ChainTransform.java
View File

@ -57,7 +57,7 @@ public class ChainTransform implements Transform {
builder.startArray();
for (Transform transform : transforms) {
builder.startObject()
.field(transform.type(), transform)
.field(transform.type(), transform, params)
.endObject();
}
return builder.endArray();
@ -111,7 +111,7 @@ public class ChainTransform implements Transform {
builder.startArray(Field.RESULTS.getPreferredName());
for (Transform.Result result : results) {
builder.startObject()
.field(result.type(), result)
.field(result.type(), result, params)
.endObject();
}
return builder.endArray();

2
src/main/java/org/elasticsearch/watcher/transform/script/ScriptTransform.java
View File

@ -76,7 +76,7 @@ public class ScriptTransform implements Transform {
@Override
protected XContentBuilder xContentBody(XContentBuilder builder, Params params) throws IOException {
return null;
return builder;
}
public static Result parse(String watchId, XContentParser parser) throws IOException {

3
src/main/java/org/elasticsearch/watcher/transport/actions/execute/ExecuteWatchRequest.java
View File

@ -10,7 +10,6 @@ import org.elasticsearch.action.ValidateActions;
import org.elasticsearch.action.support.master.MasterNodeOperationRequest;
import org.elasticsearch.common.io.stream.StreamInput;
import org.elasticsearch.common.io.stream.StreamOutput;
import org.elasticsearch.watcher.watch.WatchStore;
import java.io.IOException;
import java.util.HashSet;
@ -199,6 +198,6 @@ public class ExecuteWatchRequest extends MasterNodeOperationRequest<ExecuteWatch
@Override
public String toString() {
return "execute {[" + WatchStore.INDEX + "][" + id + "]}";
return "execute[" + id + "]";
}
}

7
src/main/java/org/elasticsearch/watcher/transport/actions/execute/TransportExecuteWatchAction.java
View File

@ -28,6 +28,7 @@ import org.elasticsearch.watcher.history.WatchRecord;
import org.elasticsearch.watcher.input.simple.SimpleInput;
import org.elasticsearch.watcher.license.LicenseService;
import org.elasticsearch.watcher.support.clock.Clock;
import org.elasticsearch.watcher.support.xcontent.WatcherParams;
import org.elasticsearch.watcher.throttle.Throttler;
import org.elasticsearch.watcher.transport.actions.WatcherTransportAction;
import org.elasticsearch.watcher.trigger.manual.ManualTriggerEvent;
@ -84,9 +85,7 @@ public class TransportExecuteWatchAction extends WatcherTransportAction<ExecuteW
if (request.isSimulateAllActions()) {
ctxBuilder.simulateAllActions();
} else {
for (String actionIdToSimulate : request.getSimulatedActionIds()){
ctxBuilder.simulateActions(actionIdToSimulate);
}
ctxBuilder.simulateActions(request.getSimulatedActionIds().toArray(new String[request.getSimulatedActionIds().size()]));
}
if (request.getTriggerData() != null) {
ctxBuilder.triggerEvent(new ManualTriggerEvent(watch.id(), executionTime, request.getTriggerData()));
@ -104,7 +103,7 @@ public class TransportExecuteWatchAction extends WatcherTransportAction<ExecuteW
WatchRecord record = executionService.execute(ctxBuilder.build());
XContentBuilder builder = XContentFactory.jsonBuilder();
record.toXContent(builder, ToXContent.EMPTY_PARAMS);
record.toXContent(builder, WatcherParams.builder().hideSecrets(true).build());
ExecuteWatchResponse response = new ExecuteWatchResponse(builder.bytes());
listener.onResponse(response);
} catch (Exception e) {

5
src/main/java/org/elasticsearch/watcher/transport/actions/get/TransportGetWatchAction.java
View File

@ -19,10 +19,11 @@ import org.elasticsearch.common.xcontent.XContentBuilder;
import org.elasticsearch.common.xcontent.json.JsonXContent;
import org.elasticsearch.threadpool.ThreadPool;
import org.elasticsearch.transport.TransportService;
import org.elasticsearch.watcher.WatcherService;
import org.elasticsearch.watcher.license.LicenseService;
import org.elasticsearch.watcher.support.xcontent.WatcherParams;
import org.elasticsearch.watcher.transport.actions.WatcherTransportAction;
import org.elasticsearch.watcher.watch.Watch;
import org.elasticsearch.watcher.WatcherService;
import org.elasticsearch.watcher.watch.WatchStore;
import java.io.IOException;
@ -67,7 +68,7 @@ public class TransportGetWatchAction extends WatcherTransportAction<GetWatchRequ
BytesReference watchSource = null;
try (XContentBuilder builder = JsonXContent.contentBuilder()) {
builder.value(watch);
watch.toXContent(builder, WatcherParams.builder().hideSecrets(true).build());
watchSource = builder.bytes();
} catch (IOException e) {
listener.onFailure(e);

8
src/main/java/org/elasticsearch/watcher/trigger/schedule/DailySchedule.java
View File

@ -50,9 +50,13 @@ public class DailySchedule extends CronnableSchedule {
public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
builder.startObject();
if (params.paramAsBoolean("normalize", false) && times.length == 1) {
builder.field(Parser.AT_FIELD.getPreferredName(), times[0]);
builder.field(Parser.AT_FIELD.getPreferredName(), times[0], params);
} else {
builder.field(Parser.AT_FIELD.getPreferredName(), (Object[]) times);
builder.startArray(Parser.AT_FIELD.getPreferredName());
for (DayTimes dayTimes : times) {
dayTimes.toXContent(builder, params);
}
builder.endArray();
}
return builder.endObject();
}

10
src/main/java/org/elasticsearch/watcher/trigger/schedule/MonthlySchedule.java
View File

@ -5,9 +5,9 @@
*/
package org.elasticsearch.watcher.trigger.schedule;
import org.elasticsearch.watcher.WatcherSettingsException;
import org.elasticsearch.common.xcontent.XContentBuilder;
import org.elasticsearch.common.xcontent.XContentParser;
import org.elasticsearch.watcher.WatcherSettingsException;
import org.elasticsearch.watcher.trigger.schedule.support.MonthTimes;
import java.io.IOException;
@ -48,9 +48,13 @@ public class MonthlySchedule extends CronnableSchedule {
@Override
public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
if (params.paramAsBoolean("normalize", false) && times.length == 1) {
return builder.value(times[0]);
return times[0].toXContent(builder, params);
}
return builder.value(times);
builder.startArray();
for (MonthTimes monthTimes : times) {
monthTimes.toXContent(builder, params);
}
return builder.endArray();
}
public static Builder builder() {

2
src/main/java/org/elasticsearch/watcher/trigger/schedule/ScheduleRegistry.java
View File

@ -5,10 +5,10 @@
*/
package org.elasticsearch.watcher.trigger.schedule;
import org.elasticsearch.watcher.WatcherSettingsException;
import org.elasticsearch.common.collect.ImmutableMap;
import org.elasticsearch.common.inject.Inject;
import org.elasticsearch.common.xcontent.XContentParser;
import org.elasticsearch.watcher.WatcherSettingsException;
import java.io.IOException;
import java.util.Map;

2
src/main/java/org/elasticsearch/watcher/trigger/schedule/ScheduleTrigger.java
View File

@ -51,7 +51,7 @@ public class ScheduleTrigger implements Trigger {
@Override
public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
return builder.startObject().field(schedule.type(), schedule).endObject();
return builder.startObject().field(schedule.type(), schedule, params).endObject();
}
public static Builder builder(Schedule schedule) {

8
src/main/java/org/elasticsearch/watcher/trigger/schedule/WeeklySchedule.java
View File

@ -48,9 +48,13 @@ public class WeeklySchedule extends CronnableSchedule {
@Override
public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
if (params.paramAsBoolean("normalize", false) && times.length == 1) {
return builder.value(times[0]);
return times[0].toXContent(builder, params);
}
return builder.value(times);
builder.startArray();
for (WeekTimes weekTimes : times) {
weekTimes.toXContent(builder, params);
}
return builder.endArray();
}
public static Builder builder() {

8
src/main/java/org/elasticsearch/watcher/trigger/schedule/YearlySchedule.java
View File

@ -48,9 +48,13 @@ public class YearlySchedule extends CronnableSchedule {
@Override
public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
if (params.paramAsBoolean("normalize", false) && times.length == 1) {
return builder.value(times[0]);
return times[0].toXContent(builder, params);
}
return builder.value(times);
builder.startArray();
for (YearTimes yearTimes : times) {
yearTimes.toXContent(builder, params);
}
return builder.endArray();
}
public static Builder builder() {

12
src/main/java/org/elasticsearch/watcher/trigger/schedule/support/MonthTimes.java
View File

@ -103,10 +103,14 @@ public class MonthTimes implements Times {
@Override
public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
return builder.startObject()
.field(DAY_FIELD.getPreferredName(), days)
.field(TIME_FIELD.getPreferredName(), (Object[]) times)
.endObject();
builder.startObject();
builder.field(DAY_FIELD.getPreferredName(), days);
builder.startArray(TIME_FIELD.getPreferredName());
for (DayTimes dayTimes : times) {
dayTimes.toXContent(builder, params);
}
builder.endArray();
return builder.endObject();
}
public static Builder builder() {

10
src/main/java/org/elasticsearch/watcher/trigger/schedule/support/Times.java
View File

@ -13,10 +13,10 @@ import org.elasticsearch.common.xcontent.ToXContent;
*/
public interface Times extends ToXContent {
public static final ParseField MONTH_FIELD = new ParseField("in", "month");
public static final ParseField DAY_FIELD = new ParseField("on", "day");
public static final ParseField TIME_FIELD = new ParseField("at", "time");
public static final ParseField HOUR_FIELD = new ParseField("hour");
public static final ParseField MINUTE_FIELD = new ParseField("minute");
ParseField MONTH_FIELD = new ParseField("in", "month");
ParseField DAY_FIELD = new ParseField("on", "day");
ParseField TIME_FIELD = new ParseField("at", "time");
ParseField HOUR_FIELD = new ParseField("hour");
ParseField MINUTE_FIELD = new ParseField("minute");
}

12
src/main/java/org/elasticsearch/watcher/trigger/schedule/support/WeekTimes.java
View File

@ -85,10 +85,14 @@ public class WeekTimes implements Times {
@Override
public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
return builder.startObject()
.field(DAY_FIELD.getPreferredName(), days)
.field(TIME_FIELD.getPreferredName(), (Object[]) times)
.endObject();
builder.startObject();
builder.field(DAY_FIELD.getPreferredName(), days);
builder.startArray(TIME_FIELD.getPreferredName());
for (DayTimes dayTimes : times) {
dayTimes.toXContent(builder, params);
}
builder.endArray();
return builder.endObject();
}
public static Builder builder() {

14
src/main/java/org/elasticsearch/watcher/trigger/schedule/support/YearTimes.java
View File

@ -108,11 +108,15 @@ public class YearTimes implements Times {
@Override
public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
return builder.startObject()
.field(MONTH_FIELD.getPreferredName(), months)
.field(DAY_FIELD.getPreferredName(), days)
.field(TIME_FIELD.getPreferredName(), (Object[]) times)
.endObject();
builder.startObject();
builder.field(MONTH_FIELD.getPreferredName(), months);
builder.field(DAY_FIELD.getPreferredName(), days);
builder.startArray(TIME_FIELD.getPreferredName());
for (DayTimes dayTimes : times) {
dayTimes.toXContent(builder, params);
}
builder.endArray();
return builder.endObject();
}
public static Builder builder() {

67
src/main/java/org/elasticsearch/watcher/watch/Watch.java
View File

@ -32,6 +32,8 @@ import org.elasticsearch.watcher.input.InputRegistry;
import org.elasticsearch.watcher.input.none.ExecutableNoneInput;
import org.elasticsearch.watcher.license.LicenseService;
import org.elasticsearch.watcher.support.clock.Clock;
import org.elasticsearch.watcher.support.secret.SensitiveXContentParser;
import org.elasticsearch.watcher.support.secret.SecretService;
import org.elasticsearch.watcher.throttle.Throttler;
import org.elasticsearch.watcher.throttle.WatchThrottler;
import org.elasticsearch.watcher.transform.ExecutableTransform;
@ -47,6 +49,7 @@ import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicLong;
import static org.elasticsearch.common.joda.time.DateTimeZone.UTC;
import static org.elasticsearch.common.xcontent.XContentFactory.jsonBuilder;
import static org.elasticsearch.watcher.support.WatcherDateUtils.*;
public class Watch implements TriggerEngine.Job, ToXContent {
@ -140,7 +143,6 @@ public class Watch implements TriggerEngine.Job, ToXContent {
return nonceCounter.getAndIncrement();
}
@Override
public boolean equals(Object o) {
if (this == o) return true;
@ -158,24 +160,34 @@ public class Watch implements TriggerEngine.Job, ToXContent {
@Override
public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
builder.startObject();
builder.field(Parser.TRIGGER_FIELD.getPreferredName()).startObject().field(trigger.type(), trigger).endObject();
builder.field(Parser.INPUT_FIELD.getPreferredName()).startObject().field(input.type(), input).endObject();
builder.field(Parser.CONDITION_FIELD.getPreferredName()).startObject().field(condition.type(), condition).endObject();
builder.field(Parser.TRIGGER_FIELD.getPreferredName()).startObject().field(trigger.type(), trigger, params).endObject();
builder.field(Parser.INPUT_FIELD.getPreferredName()).startObject().field(input.type(), input, params).endObject();
builder.field(Parser.CONDITION_FIELD.getPreferredName()).startObject().field(condition.type(), condition, params).endObject();
if (transform != null) {
builder.field(Parser.TRANSFORM_FIELD.getPreferredName()).startObject().field(transform.type(), transform).endObject();
builder.field(Parser.TRANSFORM_FIELD.getPreferredName()).startObject().field(transform.type(), transform, params).endObject();
}
if (throttlePeriod != null) {
builder.field(Parser.THROTTLE_PERIOD_FIELD.getPreferredName(), throttlePeriod.getMillis());
}
builder.field(Parser.ACTIONS_FIELD.getPreferredName(), (ToXContent) actions);
builder.field(Parser.ACTIONS_FIELD.getPreferredName(), actions, params);
if (metadata != null) {
builder.field(Parser.META_FIELD.getPreferredName(), metadata);
}
builder.field(Parser.STATUS_FIELD.getPreferredName(), status);
builder.field(Parser.STATUS_FIELD.getPreferredName(), status, params);
builder.endObject();
return builder;
}
public BytesReference getAsBytes() {
// we don't want to cache this and instead rebuild it every time on demand. The watch is in
// memory and we don't need this redundancy
try {
return toXContent(jsonBuilder(), ToXContent.EMPTY_PARAMS).bytes();
} catch (IOException ioe) {
throw new WatcherException("could not serialize watch [{}]", ioe, id);
}
}
public static class Parser extends AbstractComponent {
public static final ParseField TRIGGER_FIELD = new ParseField("trigger");
@ -194,6 +206,7 @@ public class Watch implements TriggerEngine.Job, ToXContent {
private final ActionRegistry actionRegistry;
private final InputRegistry inputRegistry;
private final Clock clock;
private final SecretService secretService;
private final ExecutableInput defaultInput;
private final ExecutableCondition defaultCondition;
@ -202,7 +215,7 @@ public class Watch implements TriggerEngine.Job, ToXContent {
@Inject
public Parser(Settings settings, LicenseService licenseService, ConditionRegistry conditionRegistry, TriggerService triggerService,
TransformRegistry transformRegistry, ActionRegistry actionRegistry,
InputRegistry inputRegistry, Clock clock) {
InputRegistry inputRegistry, Clock clock, SecretService secretService) {
super(settings);
this.licenseService = licenseService;
@ -212,6 +225,7 @@ public class Watch implements TriggerEngine.Job, ToXContent {
this.actionRegistry = actionRegistry;
this.inputRegistry = inputRegistry;
this.clock = clock;
this.secretService = secretService;
this.defaultInput = new ExecutableNoneInput(logger);
this.defaultCondition = new ExecutableAlwaysCondition(logger);
@ -219,13 +233,44 @@ public class Watch implements TriggerEngine.Job, ToXContent {
}
public Watch parse(String name, boolean includeStatus, BytesReference source) {
return parse(name, includeStatus, false, source);
}
/**
* Parses the watch represented by the given source. When parsing, any sensitive data that the
* source might contain (e.g. passwords) will be converted to {@link org.elasticsearch.watcher.support.secret.Secret secrets}
* Such that the returned watch will potentially hide this sensitive data behind a "secret". A secret
* is an abstraction around sensitive data (text). There can be different implementations of how the
* secret holds the data, depending on the wired up {@link SecretService}. When shield is installed, a
* {@link org.elasticsearch.watcher.shield.ShieldSecretService} is used, that potentially encrypts the data
* using Shield's configured system key.
*
* This method is only called once - when the user adds a new watch. From that moment on, all representations
* of the watch in the system will be use secrets for sensitive data.
*
* @see org.elasticsearch.watcher.WatcherService#putWatch(String, BytesReference)
*/
public Watch parseWithSecrets(String id, boolean includeStatus, BytesReference source) {
return parse(id, includeStatus, true, source);
}
private Watch parse(String id, boolean includeStatus, boolean withSecrets, BytesReference source) {
if (logger.isTraceEnabled()) {
logger.trace("parsing watch [{}] ", source.toUtf8());
}
try (XContentParser parser = XContentHelper.createParser(source)) {
return parse(name, includeStatus, parser);
XContentParser parser = null;
try {
parser = XContentHelper.createParser(source);
if (withSecrets) {
parser = new SensitiveXContentParser(parser, secretService);
}
return parse(id, includeStatus, parser);
} catch (IOException ioe) {
throw new WatcherException("could not parse watch [" + name + "]", ioe);
throw new WatcherException("could not parse watch [{}]", ioe, id);
} finally {
if (parser != null) {
parser.close();
}
}
}

14
src/main/java/org/elasticsearch/watcher/watch/WatchExecution.java
View File

@ -73,10 +73,14 @@ public class WatchExecution implements ToXContent {
public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
builder.startObject();
if (inputResult != null) {
builder.startObject(Parser.INPUT_RESULT_FIELD.getPreferredName()).field(inputResult.type(), inputResult).endObject();
builder.startObject(Parser.INPUT_RESULT_FIELD.getPreferredName())
.field(inputResult.type(), inputResult, params)
.endObject();
}
if (conditionResult != null) {
builder.startObject(Parser.CONDITION_RESULT_FIELD.getPreferredName()).field(conditionResult.type(), conditionResult).endObject();
builder.startObject(Parser.CONDITION_RESULT_FIELD.getPreferredName())
.field(conditionResult.type(), conditionResult, params)
.endObject();
}
if (throttleResult != null && throttleResult.throttle()) {
builder.field(Parser.THROTTLED.getPreferredName(), throttleResult.throttle());
@ -85,11 +89,13 @@ public class WatchExecution implements ToXContent {
}
}
if (transformResult != null) {
builder.startObject(Transform.Field.TRANSFORM_RESULT.getPreferredName()).field(transformResult.type(), transformResult).endObject();
builder.startObject(Transform.Field.TRANSFORM_RESULT.getPreferredName())
.field(transformResult.type(), transformResult, params)
.endObject();
}
builder.startObject(Parser.ACTIONS_RESULTS.getPreferredName());
for (ActionWrapper.Result actionResult : actionsResults) {
builder.field(actionResult.id(), actionResult);
builder.field(actionResult.id(), actionResult, params);
}
builder.endObject();
builder.endObject();

7
src/main/java/org/elasticsearch/watcher/watch/WatchStore.java
View File

@ -124,13 +124,12 @@ public class WatchStore extends AbstractComponent {
* Creates an watch with the specified name and source. If an watch with the specified name already exists it will
* get overwritten.
*/
public WatchPut put(String name, BytesReference source) {
public WatchPut put(Watch watch) {
ensureStarted();
Watch watch = watchParser.parse(name, false, source);
IndexRequest indexRequest = createIndexRequest(name, source);
IndexRequest indexRequest = createIndexRequest(watch.id(), watch.getAsBytes());
IndexResponse response = client.index(indexRequest);
watch.status().version(response.getVersion());
Watch previous = watches.put(name, watch);
Watch previous = watches.put(watch.id(), watch);
return new WatchPut(previous, watch, response);
}

28
src/test/java/org/elasticsearch/watcher/actions/email/EmailActionTests.java
View File

@ -20,8 +20,10 @@ import org.elasticsearch.test.ElasticsearchTestCase;
import org.elasticsearch.watcher.actions.email.service.*;
import org.elasticsearch.watcher.execution.WatchExecutionContext;
import org.elasticsearch.watcher.execution.Wid;
import org.elasticsearch.watcher.support.secret.Secret;
import org.elasticsearch.watcher.support.template.Template;
import org.elasticsearch.watcher.support.template.TemplateEngine;
import org.elasticsearch.watcher.support.xcontent.WatcherParams;
import org.elasticsearch.watcher.watch.Payload;
import org.junit.Test;
@ -74,7 +76,7 @@ public class EmailActionTests extends ElasticsearchTestCase {
}
EmailTemplate email = emailBuilder.build();
Authentication auth = new Authentication("user", "passwd".toCharArray());
Authentication auth = new Authentication("user", new Secret("passwd".toCharArray()));
Profile profile = randomFrom(Profile.values());
boolean attachPayload = randomBoolean();
@ -220,7 +222,7 @@ public class EmailActionTests extends ElasticsearchTestCase {
assertThat(executable.action().getAttachData(), is(attachData));
assertThat(executable.action().getAuth(), notNullValue());
assertThat(executable.action().getAuth().user(), is("_user"));
assertThat(executable.action().getAuth().password(), is("_passwd".toCharArray()));
assertThat(executable.action().getAuth().password(), is(new Secret("_passwd".toCharArray())));
assertThat(executable.action().getEmail().priority(), is(new Template(priority.name())));
if (to != null) {
assertThat(executable.action().getEmail().to(), arrayContainingInAnyOrder(addressesToTemplates(to)));
@ -273,7 +275,7 @@ public class EmailActionTests extends ElasticsearchTestCase {
emailTemplate.replyTo(randomBoolean() ? "reply@domain" : "reply1@domain,reply2@domain");
}
EmailTemplate email = emailTemplate.build();
Authentication auth = randomBoolean() ? null : new Authentication("_user", "_passwd".toCharArray());
Authentication auth = randomBoolean() ? null : new Authentication("_user", new Secret("_passwd".toCharArray()));
Profile profile = randomFrom(Profile.values());
String account = randomAsciiOfLength(6);
boolean attachPayload = randomBoolean();
@ -281,14 +283,30 @@ public class EmailActionTests extends ElasticsearchTestCase {
EmailAction action = new EmailAction(email, account, auth, profile, attachPayload);
ExecutableEmailAction executable = new ExecutableEmailAction(action, logger, service, engine);
boolean hideSecrets = randomBoolean();
ToXContent.Params params = WatcherParams.builder().hideSecrets(hideSecrets).build();
XContentBuilder builder = jsonBuilder();
executable.toXContent(builder, Attachment.XContent.EMPTY_PARAMS);
executable.toXContent(builder, params);
BytesReference bytes = builder.bytes();
XContentParser parser = JsonXContent.jsonXContent.createParser(bytes);
parser.nextToken();
ExecutableEmailAction parsed = new EmailActionFactory(ImmutableSettings.EMPTY, service, engine)
.parseExecutable(randomAsciiOfLength(4), randomAsciiOfLength(10), parser);
assertThat(parsed, equalTo(executable));
if (!hideSecrets) {
assertThat(parsed, equalTo(executable));
} else {
assertThat(parsed.action().getAccount(), is(executable.action().getAccount()));
assertThat(parsed.action().getEmail(), is(executable.action().getEmail()));
assertThat(parsed.action().getAttachData(), is(executable.action().getAttachData()));
if (auth != null) {
assertThat(parsed.action().getAuth().user(), is(executable.action().getAuth().user()));
assertThat(parsed.action().getAuth().password(), nullValue());
assertThat(executable.action().getAuth().password(), notNullValue());
}
}
}
@Test(expected = EmailActionException.class) @Repeat(iterations = 100)

10
src/test/java/org/elasticsearch/watcher/actions/email/service/AccountTests.java
View File

@ -9,6 +9,8 @@ import org.elasticsearch.common.settings.ImmutableSettings;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.test.ElasticsearchTestCase;
import org.elasticsearch.watcher.actions.email.service.support.EmailServer;
import org.elasticsearch.watcher.support.secret.Secret;
import org.elasticsearch.watcher.support.secret.SecretService;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
@ -157,7 +159,7 @@ public class AccountTests extends ElasticsearchTestCase {
.put("smtp.port", server.port())
.put("smtp.user", USERNAME)
.put("smtp.password", PASSWORD)
.build()), logger);
.build()), new SecretService.PlainText(), logger);
Email email = Email.builder()
.id("_id")
@ -197,7 +199,7 @@ public class AccountTests extends ElasticsearchTestCase {
.put("smtp.port", server.port())
.put("smtp.user", USERNAME)
.put("smtp.password", PASSWORD)
.build()), logger);
.build()), new SecretService.PlainText(), logger);
Email email = Email.builder()
.id("_id")
@ -240,7 +242,7 @@ public class AccountTests extends ElasticsearchTestCase {
Account account = new Account(new Account.Config("default", ImmutableSettings.builder()
.put("smtp.host", "localhost")
.put("smtp.port", server.port())
.build()), logger);
.build()), new SecretService.PlainText(), logger);
Email email = Email.builder()
.id("_id")
@ -258,7 +260,7 @@ public class AccountTests extends ElasticsearchTestCase {
}
});
account.send(email, new Authentication(USERNAME, PASSWORD.toCharArray()), Profile.STANDARD);
account.send(email, new Authentication(USERNAME, new Secret(PASSWORD.toCharArray())), Profile.STANDARD);
if (!latch.await(5, TimeUnit.SECONDS)) {
fail("waiting for email too long");

15
src/test/java/org/elasticsearch/watcher/actions/email/service/AccountsTests.java
View File

@ -7,6 +7,7 @@ package org.elasticsearch.watcher.actions.email.service;
import org.elasticsearch.common.settings.ImmutableSettings;
import org.elasticsearch.test.ElasticsearchTestCase;
import org.elasticsearch.watcher.support.secret.SecretService;
import org.junit.Test;
import static org.hamcrest.Matchers.equalTo;
@ -25,7 +26,7 @@ public class AccountsTests extends ElasticsearchTestCase {
.put("default_account", "account1");
addAccountSettings("account1", builder);
Accounts accounts = new Accounts(builder.build(), logger);
Accounts accounts = new Accounts(builder.build(), new SecretService.PlainText(), logger);
Account account = accounts.account("account1");
assertThat(account, notNullValue());
assertThat(account.name(), equalTo("account1"));
@ -39,7 +40,7 @@ public class AccountsTests extends ElasticsearchTestCase {
ImmutableSettings.Builder builder = ImmutableSettings.builder();
addAccountSettings("account1", builder);
Accounts accounts = new Accounts(builder.build(), logger);
Accounts accounts = new Accounts(builder.build(), new SecretService.PlainText(), logger);
Account account = accounts.account("account1");
assertThat(account, notNullValue());
assertThat(account.name(), equalTo("account1"));
@ -55,7 +56,7 @@ public class AccountsTests extends ElasticsearchTestCase {
addAccountSettings("account1", builder);
addAccountSettings("account2", builder);
Accounts accounts = new Accounts(builder.build(), logger);
Accounts accounts = new Accounts(builder.build(), new SecretService.PlainText(), logger);
Account account = accounts.account("account1");
assertThat(account, notNullValue());
assertThat(account.name(), equalTo("account1"));
@ -74,7 +75,7 @@ public class AccountsTests extends ElasticsearchTestCase {
addAccountSettings("account1", builder);
addAccountSettings("account2", builder);
Accounts accounts = new Accounts(builder.build(), logger);
Accounts accounts = new Accounts(builder.build(), new SecretService.PlainText(), logger);
Account account = accounts.account("account1");
assertThat(account, notNullValue());
assertThat(account.name(), equalTo("account1"));
@ -92,13 +93,13 @@ public class AccountsTests extends ElasticsearchTestCase {
.put("default_account", "unknown");
addAccountSettings("account1", builder);
addAccountSettings("account2", builder);
new Accounts(builder.build(), logger);
new Accounts(builder.build(), new SecretService.PlainText(), logger);
}
@Test
public void testNoAccount() throws Exception {
ImmutableSettings.Builder builder = ImmutableSettings.builder();
Accounts accounts = new Accounts(builder.build(), logger);
Accounts accounts = new Accounts(builder.build(), new SecretService.PlainText(), logger);
Account account = accounts.account(null);
assertThat(account, nullValue());
}
@ -107,7 +108,7 @@ public class AccountsTests extends ElasticsearchTestCase {
public void testNoAccount_WithDefaultAccount() throws Exception {
ImmutableSettings.Builder builder = ImmutableSettings.builder()
.put("default_account", "unknown");
new Accounts(builder.build(), logger);
new Accounts(builder.build(), new SecretService.PlainText(), logger);
}
private void addAccountSettings(String name, ImmutableSettings.Builder builder) {

7
src/test/java/org/elasticsearch/watcher/actions/email/service/InternalEmailServiceTests.java
View File

@ -10,6 +10,9 @@ import org.elasticsearch.common.settings.ImmutableSettings;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.node.settings.NodeSettingsService;
import org.elasticsearch.test.ElasticsearchTestCase;
import org.elasticsearch.watcher.shield.WatcherSettingsFilter;
import org.elasticsearch.watcher.support.secret.Secret;
import org.elasticsearch.watcher.support.secret.SecretService;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
@ -28,7 +31,7 @@ public class InternalEmailServiceTests extends ElasticsearchTestCase {
@Before
public void init() throws Exception {
accounts = mock(Accounts.class);
service = new InternalEmailService(ImmutableSettings.EMPTY, new NodeSettingsService(ImmutableSettings.EMPTY)) {
service = new InternalEmailService(ImmutableSettings.EMPTY, new SecretService.PlainText(), new NodeSettingsService(ImmutableSettings.EMPTY), WatcherSettingsFilter.Noop.INSTANCE) {
@Override
protected Accounts createAccounts(Settings settings, ESLogger logger) {
return accounts;
@ -49,7 +52,7 @@ public class InternalEmailServiceTests extends ElasticsearchTestCase {
when(accounts.account("account1")).thenReturn(account);
Email email = mock(Email.class);
Authentication auth = new Authentication("user", "passwd".toCharArray());
Authentication auth = new Authentication("user", new Secret("passwd".toCharArray()));
Profile profile = randomFrom(Profile.values());
when(account.send(email, auth, profile)).thenReturn(email);
EmailService.EmailSent sent = service.send(email, auth, profile, "account1");

5
src/test/java/org/elasticsearch/watcher/actions/email/service/ManualPublicSmtpServersTests.java
View File

@ -12,6 +12,8 @@ import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.common.xcontent.ToXContent;
import org.elasticsearch.common.xcontent.XContentBuilder;
import org.elasticsearch.node.settings.NodeSettingsService;
import org.elasticsearch.watcher.shield.WatcherSettingsFilter;
import org.elasticsearch.watcher.support.secret.SecretService;
import org.junit.Ignore;
import java.io.IOException;
@ -37,7 +39,6 @@ public class ManualPublicSmtpServersTests {
.put("watcher.actions.email.service.account.gmail.smtp.password", new String(terminal.readSecret("password: ")))
.put("watcher.actions.email.service.account.gmail.email_defaults.to", terminal.readText("to: "))
);
}
}
@ -131,7 +132,7 @@ public class ManualPublicSmtpServersTests {
static InternalEmailService startEmailService(Settings.Builder builder) {
Settings settings = builder.build();
InternalEmailService service = new InternalEmailService(settings, new NodeSettingsService(settings));
InternalEmailService service = new InternalEmailService(settings, new SecretService.PlainText(), new NodeSettingsService(settings), WatcherSettingsFilter.Noop.INSTANCE);
service.start();
return service;
}

4
src/test/java/org/elasticsearch/watcher/actions/email/service/support/EmailServer.java
View File

@ -126,11 +126,11 @@ public class EmailServer {
public static interface Listener {
public interface Listener {
void on(MimeMessage message) throws Exception;
public static class Handle {
class Handle {
private final List<Listener> listeners;
private final Listener listener;

4
src/test/java/org/elasticsearch/watcher/actions/index/IndexActionTests.java
View File

@ -27,6 +27,7 @@ import org.elasticsearch.watcher.execution.TriggeredExecutionContext;
import org.elasticsearch.watcher.execution.WatchExecutionContext;
import org.elasticsearch.watcher.execution.Wid;
import org.elasticsearch.watcher.support.http.HttpClient;
import org.elasticsearch.watcher.support.http.auth.HttpAuthRegistry;
import org.elasticsearch.watcher.support.init.proxy.ClientProxy;
import org.elasticsearch.watcher.support.init.proxy.ScriptServiceProxy;
import org.elasticsearch.watcher.test.WatcherTestUtils;
@ -43,6 +44,7 @@ import static org.elasticsearch.common.xcontent.XContentFactory.jsonBuilder;
import static org.elasticsearch.index.query.QueryBuilders.matchAllQuery;
import static org.elasticsearch.search.builder.SearchSourceBuilder.searchSource;
import static org.hamcrest.Matchers.*;
import static org.mockito.Mockito.mock;
/**
*/
@ -57,7 +59,7 @@ public class IndexActionTests extends ElasticsearchIntegrationTest {
Watch watch = WatcherTestUtils.createTestWatch("testAlert",
ClientProxy.of(client()),
ScriptServiceProxy.of(internalCluster().getInstance(ScriptService.class)),
new HttpClient(ImmutableSettings.EMPTY),
new HttpClient(ImmutableSettings.EMPTY, mock(HttpAuthRegistry.class)),
new EmailService() {
@Override
public EmailService.EmailSent send(Email email, Authentication auth, Profile profile) {

9
src/test/java/org/elasticsearch/watcher/actions/webhook/WebhookActionTests.java
View File

@ -29,11 +29,12 @@ import org.elasticsearch.watcher.execution.TriggeredExecutionContext;
import org.elasticsearch.watcher.execution.WatchExecutionContext;
import org.elasticsearch.watcher.execution.Wid;
import org.elasticsearch.watcher.support.http.*;
import org.elasticsearch.watcher.support.http.auth.BasicAuth;
import org.elasticsearch.watcher.support.http.auth.HttpAuth;
import org.elasticsearch.watcher.support.http.auth.HttpAuthFactory;
import org.elasticsearch.watcher.support.http.auth.HttpAuthRegistry;
import org.elasticsearch.watcher.support.http.auth.basic.BasicAuthFactory;
import org.elasticsearch.watcher.support.init.proxy.ClientProxy;
import org.elasticsearch.watcher.support.init.proxy.ScriptServiceProxy;
import org.elasticsearch.watcher.support.secret.SecretService;
import org.elasticsearch.watcher.support.template.Template;
import org.elasticsearch.watcher.support.template.TemplateEngine;
import org.elasticsearch.watcher.support.template.xmustache.XMustacheScriptEngineService;
@ -74,6 +75,7 @@ public class WebhookActionTests extends ElasticsearchTestCase {
private ThreadPool tp = null;
private ScriptServiceProxy scriptService;
private SecretService secretService;
private TemplateEngine templateEngine;
private HttpAuthRegistry authRegistry;
private Template testBody;
@ -92,9 +94,10 @@ public class WebhookActionTests extends ElasticsearchTestCase {
engineServiceSet.add(mustacheScriptEngineService);
scriptService = ScriptServiceProxy.of(new ScriptService(settings, new Environment(), engineServiceSet, new ResourceWatcherService(settings, tp), new NodeSettingsService(settings)));
templateEngine = new XMustacheTemplateEngine(settings, scriptService);
secretService = mock(SecretService.class);
testBody = new Template(TEST_BODY_STRING );
testPath = new Template(TEST_PATH_STRING);
authRegistry = new HttpAuthRegistry(ImmutableMap.of("basic", (HttpAuth.Parser) new BasicAuth.Parser()));
authRegistry = new HttpAuthRegistry(ImmutableMap.of("basic", (HttpAuthFactory) new BasicAuthFactory(secretService)));
}
@After

16
src/test/java/org/elasticsearch/watcher/input/http/HttpInputTests.java
View File

@ -5,7 +5,9 @@
*/
package org.elasticsearch.watcher.input.http;
import com.carrotsearch.randomizedtesting.annotations.Repeat;
import org.elasticsearch.ElasticsearchIllegalArgumentException;
import org.elasticsearch.common.bytes.BytesReference;
import org.elasticsearch.common.collect.ImmutableMap;
import org.elasticsearch.common.collect.MapBuilder;
import org.elasticsearch.common.joda.time.DateTime;
@ -25,9 +27,12 @@ import org.elasticsearch.watcher.input.simple.SimpleInput;
import org.elasticsearch.watcher.license.LicenseService;
import org.elasticsearch.watcher.support.clock.ClockMock;
import org.elasticsearch.watcher.support.http.*;
import org.elasticsearch.watcher.support.http.auth.BasicAuth;
import org.elasticsearch.watcher.support.http.auth.HttpAuth;
import org.elasticsearch.watcher.support.http.auth.HttpAuthFactory;
import org.elasticsearch.watcher.support.http.auth.HttpAuthRegistry;
import org.elasticsearch.watcher.support.http.auth.basic.BasicAuth;
import org.elasticsearch.watcher.support.http.auth.basic.BasicAuthFactory;
import org.elasticsearch.watcher.support.secret.SecretService;
import org.elasticsearch.watcher.support.template.Template;
import org.elasticsearch.watcher.support.template.TemplateEngine;
import org.elasticsearch.watcher.trigger.schedule.IntervalSchedule;
@ -55,13 +60,15 @@ public class HttpInputTests extends ElasticsearchTestCase {
private HttpClient httpClient;
private HttpInputFactory httpParser;
private SecretService secretService;
private TemplateEngine templateEngine;
@Before
public void init() throws Exception {
httpClient = mock(HttpClient.class);
templateEngine = mock(TemplateEngine.class);
HttpAuthRegistry registry = new HttpAuthRegistry(ImmutableMap.<String, HttpAuth.Parser>of("basic", new BasicAuth.Parser()));
secretService = mock(SecretService.class);
HttpAuthRegistry registry = new HttpAuthRegistry(ImmutableMap.<String, HttpAuthFactory>of("basic", new BasicAuthFactory(secretService)));
httpParser = new HttpInputFactory(ImmutableSettings.EMPTY, httpClient, templateEngine, new HttpRequest.Parser(registry), new HttpRequestTemplate.Parser(registry));
}
@ -110,7 +117,7 @@ public class HttpInputTests extends ElasticsearchTestCase {
String body = randomBoolean() ? randomAsciiOfLength(3) : null;
Map<String, Template> params = randomBoolean() ? new MapBuilder<String, Template>().put("a", new Template("b")).map() : null;
Map<String, Template> headers = randomBoolean() ? new MapBuilder<String, Template>().put("c", new Template("d")).map() : null;
HttpAuth auth = randomBoolean() ? new BasicAuth("username", "password") : null;
HttpAuth auth = randomBoolean() ? new BasicAuth("username", "password".toCharArray()) : null;
HttpRequestTemplate.Builder requestBuilder = HttpRequestTemplate.builder(host, port)
.scheme(scheme)
.method(httpMethod)
@ -125,7 +132,8 @@ public class HttpInputTests extends ElasticsearchTestCase {
requestBuilder.putHeaders(headers);
}
XContentParser parser = XContentHelper.createParser(jsonBuilder().value(InputBuilders.httpInput(requestBuilder).build()).bytes());
BytesReference source = jsonBuilder().value(InputBuilders.httpInput(requestBuilder).build()).bytes();
XContentParser parser = XContentHelper.createParser(source);
parser.nextToken();
HttpInput result = httpParser.parseInput("_id", parser);

21
src/test/java/org/elasticsearch/watcher/support/http/HttpClientTest.java
View File

@ -11,9 +11,14 @@ import com.squareup.okhttp.mockwebserver.MockWebServer;
import com.squareup.okhttp.mockwebserver.RecordedRequest;
import org.elasticsearch.ElasticsearchException;
import org.elasticsearch.common.base.Charsets;
import org.elasticsearch.common.collect.ImmutableMap;
import org.elasticsearch.common.settings.ImmutableSettings;
import org.elasticsearch.test.ElasticsearchTestCase;
import org.elasticsearch.watcher.support.http.auth.BasicAuth;
import org.elasticsearch.watcher.support.http.auth.HttpAuthFactory;
import org.elasticsearch.watcher.support.http.auth.HttpAuthRegistry;
import org.elasticsearch.watcher.support.http.auth.basic.BasicAuth;
import org.elasticsearch.watcher.support.http.auth.basic.BasicAuthFactory;
import org.elasticsearch.watcher.support.secret.SecretService;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
@ -30,16 +35,20 @@ public class HttpClientTest extends ElasticsearchTestCase {
private MockWebServer webServer;
private HttpClient httpClient;
private HttpAuthRegistry authRegistry;
private SecretService secretService;
private int webPort = 9200;
private int webPort;
@Before
public void init() throws Exception {
for (; webPort < 9300; webPort++) {
secretService = new SecretService.PlainText();
authRegistry = new HttpAuthRegistry(ImmutableMap.<String, HttpAuthFactory>of(BasicAuth.TYPE, new BasicAuthFactory(secretService)));
for (webPort = 9200; webPort < 9300; webPort++) {
try {
webServer = new MockWebServer();
webServer.start(webPort);
httpClient = new HttpClient(ImmutableSettings.EMPTY);
httpClient = new HttpClient(ImmutableSettings.EMPTY, authRegistry);
return;
} catch (BindException be) {
logger.warn("port [{}] was already in use trying next port", webPort);
@ -95,7 +104,7 @@ public class HttpClientTest extends ElasticsearchTestCase {
HttpRequest.Builder request = HttpRequest.builder("localhost", webPort)
.method(HttpMethod.POST)
.path("/test")
.auth(new BasicAuth("user", "pass"))
.auth(new BasicAuth("user", "pass".toCharArray()))
.body("body");
HttpResponse response = httpClient.execute(request.build());
assertThat(response.status(), equalTo(200));
@ -111,7 +120,7 @@ public class HttpClientTest extends ElasticsearchTestCase {
ImmutableSettings.builder()
.put(HttpClient.SETTINGS_SSL_TRUSTSTORE, resource.toString())
.put(HttpClient.SETTINGS_SSL_TRUSTSTORE_PASSWORD, "testnode")
.build());
.build(), authRegistry);
webServer.useHttps(httpClient.sslSocketFactory, false);
webServer.enqueue(new MockResponse().setResponseCode(200).setBody("body"));

22
src/test/java/org/elasticsearch/watcher/test/AbstractWatcherIntegrationTests.java
View File

@ -32,6 +32,7 @@ import org.elasticsearch.plugins.Plugin;
import org.elasticsearch.search.builder.SearchSourceBuilder;
import org.elasticsearch.shield.ShieldPlugin;
import org.elasticsearch.shield.authc.esusers.ESUsersRealm;
import org.elasticsearch.shield.crypto.InternalCryptoService;
import org.elasticsearch.test.ElasticsearchIntegrationTest;
import org.elasticsearch.test.ElasticsearchIntegrationTest.ClusterScope;
import org.elasticsearch.test.InternalTestCluster;
@ -641,14 +642,18 @@ public abstract class AbstractWatcherIntegrationTests extends ElasticsearchInteg
/** Shield related settings */
static class ShieldSettings {
public static class ShieldSettings {
static boolean auditLogsEnabled = SystemPropertyUtil.getBoolean("tests.audit_logs", false);
public static final String TEST_USERNAME = "test";
public static final String TEST_PASSWORD = "changeme";
static boolean auditLogsEnabled = SystemPropertyUtil.getBoolean("tests.audit_logs", true);
static byte[] systemKey = generateKey(); // must be the same for all nodes
public static final String IP_FILTER = "allow: all\n";
public static final String USERS =
"test:{plain}changeme\n" +
TEST_USERNAME + ":{plain}" + TEST_PASSWORD + "\n" +
"admin:{plain}changeme\n" +
"monitor:{plain}changeme";
@ -685,10 +690,20 @@ public abstract class AbstractWatcherIntegrationTests extends ElasticsearchInteg
.put("shield.authc.realms.esusers.files.users_roles", writeFile(folder, "users_roles", USER_ROLES))
.put("shield.authz.store.files.roles", writeFile(folder, "roles.yml", ROLES))
.put("shield.transport.n2n.ip_filter.file", writeFile(folder, "ip_filter.yml", IP_FILTER))
.put("shield.system_key.file", writeFile(folder, "system_key.yml", systemKey))
.put("shield.authc.sign_user_header", false)
.put("shield.audit.enabled", auditLogsEnabled)
.build();
}
static byte[] generateKey() {
try {
return InternalCryptoService.generateKey();
} catch (Exception e) {
throw new RuntimeException(e);
}
}
static File createFolder(File parent, String name) {
File createdFolder = new File(parent, name);
//the directory might exist e.g. if the global cluster gets restarted, then we recreate the directory as well
@ -718,5 +733,4 @@ public abstract class AbstractWatcherIntegrationTests extends ElasticsearchInteg
}
}
}

3
src/test/java/org/elasticsearch/watcher/test/WatcherTestUtils.java
View File

@ -41,6 +41,7 @@ import org.elasticsearch.watcher.support.http.HttpMethod;
import org.elasticsearch.watcher.support.http.HttpRequestTemplate;
import org.elasticsearch.watcher.support.init.proxy.ClientProxy;
import org.elasticsearch.watcher.support.init.proxy.ScriptServiceProxy;
import org.elasticsearch.watcher.support.secret.Secret;
import org.elasticsearch.watcher.support.template.xmustache.XMustacheTemplateEngine;
import org.elasticsearch.watcher.support.template.Template;
import org.elasticsearch.watcher.support.template.TemplateEngine;
@ -159,7 +160,7 @@ public final class WatcherTestUtils {
TemplateEngine templateEngine = new XMustacheTemplateEngine(ImmutableSettings.EMPTY, scriptService);
Authentication auth = new Authentication("testname", "testpassword".toCharArray());
Authentication auth = new Authentication("testname", new Secret("testpassword".toCharArray()));
EmailAction action = new EmailAction(email, "testaccount", auth, Profile.STANDARD, false);
ExecutableEmailAction executale = new ExecutableEmailAction(action, logger, emailService, templateEngine);

4
src/test/java/org/elasticsearch/watcher/test/integration/EmailActionIntegrationTests.java
View File

@ -84,8 +84,8 @@ public class EmailActionIntegrationTests extends AbstractWatcherIntegrationTests
.trigger(schedule(interval(5, IntervalSchedule.Interval.Unit.SECONDS)))
.input(searchInput(searchRequest))
.condition(scriptCondition("ctx.payload.hits.total > 0"))
.addAction("_index", emailAction(EmailTemplate.builder().from("_from").to("_to")
.subject("{{ctx.payload.hits.hits.0._source.field}}"))))
.addAction("_email", emailAction(EmailTemplate.builder().from("_from").to("_to")
.subject("{{ctx.payload.hits.hits.0._source.field}}")).setAuthentication(USERNAME, PASSWORD.toCharArray())))
.get();
if (timeWarped()) {

148
src/test/java/org/elasticsearch/watcher/test/integration/EmailSecretsIntegrationTests.java Normal file
View File

@ -0,0 +1,148 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.watcher.test.integration;
import org.elasticsearch.action.get.GetResponse;
import org.elasticsearch.common.settings.ImmutableSettings;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.common.xcontent.support.XContentMapValues;
import org.elasticsearch.watcher.actions.email.service.EmailTemplate;
import org.elasticsearch.watcher.actions.email.service.support.EmailServer;
import org.elasticsearch.watcher.client.WatcherClient;
import org.elasticsearch.watcher.shield.ShieldSecretService;
import org.elasticsearch.watcher.support.secret.SecretService;
import org.elasticsearch.watcher.test.AbstractWatcherIntegrationTests;
import org.elasticsearch.watcher.transport.actions.execute.ExecuteWatchResponse;
import org.elasticsearch.watcher.transport.actions.get.GetWatchResponse;
import org.elasticsearch.watcher.watch.WatchStore;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import javax.mail.internet.MimeMessage;
import java.util.Map;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.TimeUnit;
import static org.elasticsearch.watcher.actions.ActionBuilders.emailAction;
import static org.elasticsearch.watcher.client.WatchSourceBuilders.watchBuilder;
import static org.elasticsearch.watcher.condition.ConditionBuilders.alwaysCondition;
import static org.elasticsearch.watcher.input.InputBuilders.simpleInput;
import static org.elasticsearch.watcher.trigger.TriggerBuilders.schedule;
import static org.elasticsearch.watcher.trigger.schedule.Schedules.cron;
import static org.hamcrest.Matchers.*;
/**
*
*/
public class EmailSecretsIntegrationTests extends AbstractWatcherIntegrationTests {
static final String USERNAME = "_user";
static final String PASSWORD = "_passwd";
private EmailServer server;
private Boolean encryptSensitiveData;
@After
public void cleanup() throws Exception {
server.stop();
}
@Override
protected Settings nodeSettings(int nodeOrdinal) {
if(server == null) {
//Need to construct the Email Server here as this happens before init()
server = EmailServer.localhost("2500-2600", USERNAME, PASSWORD, logger);
}
if (encryptSensitiveData == null) {
encryptSensitiveData = shieldEnabled() && randomBoolean();
}
return ImmutableSettings.builder()
.put(super.nodeSettings(nodeOrdinal))
.put("watcher.actions.email.service.account.test.smtp.auth", true)
.put("watcher.actions.email.service.account.test.smtp.port", server.port())
.put("watcher.actions.email.service.account.test.smtp.host", "localhost")
.put("watcher.shield.encrypt_sensitive_data", encryptSensitiveData)
.build();
}
@Test
public void testEmail() throws Exception {
WatcherClient watcherClient = watcherClient();
watcherClient.preparePutWatch("_id")
.setSource(watchBuilder()
.trigger(schedule(cron("0 0 0 1 * ? 2020")))
.input(simpleInput())
.condition(alwaysCondition())
.addAction("_email", emailAction(
EmailTemplate.builder()
.from("_from")
.to("_to")
.subject("_subject"))
.setAuthentication(USERNAME, PASSWORD.toCharArray())))
.get();
// verifying the email password is stored encrypted in the index
GetResponse response = client().prepareGet(WatchStore.INDEX, WatchStore.DOC_TYPE, "_id").get();
assertThat(response, notNullValue());
assertThat(response.getId(), is("_id"));
Map<String, Object> source = response.getSource();
Object value = XContentMapValues.extractValue("actions._email.email.password", source);
assertThat(value, notNullValue());
if (shieldEnabled() && encryptSensitiveData) {
assertThat(value, not(is((Object) PASSWORD)));
SecretService secretService = getInstanceFromMaster(SecretService.class);
assertThat(secretService, instanceOf(ShieldSecretService.class));
assertThat(new String(secretService.decrypt(((String) value).toCharArray())), is(PASSWORD));
} else {
assertThat(value, is((Object) PASSWORD));
SecretService secretService = getInstanceFromMaster(SecretService.class);
if (shieldEnabled()) {
assertThat(secretService, instanceOf(ShieldSecretService.class));
} else {
assertThat(secretService, instanceOf(SecretService.PlainText.class));
}
assertThat(new String(secretService.decrypt(((String) value).toCharArray())), is(PASSWORD));
}
// verifying the password is not returned by the GET watch API
GetWatchResponse watchResponse = watcherClient.prepareGetWatch("_id").get();
assertThat(watchResponse, notNullValue());
assertThat(watchResponse.getId(), is("_id"));
source = watchResponse.getSourceAsMap();
value = XContentMapValues.extractValue("actions._email.email.password", source);
assertThat(value, nullValue());
// now we restart, to make sure the watches and their secrets are reloaded from the index properly
assertThat(watcherClient.prepareWatchService().restart().get().isAcknowledged(), is(true));
ensureWatcherStarted();
// now lets execute the watch manually
final CountDownLatch latch = new CountDownLatch(1);
server.addListener(new EmailServer.Listener() {
@Override
public void on(MimeMessage message) throws Exception {
assertThat(message.getSubject(), is("_subject"));
latch.countDown();
}
});
ExecuteWatchResponse executeResponse = watcherClient.prepareExecuteWatch("_id")
.setRecordExecution(false)
.setIgnoreThrottle(true)
.get();
assertThat(executeResponse, notNullValue());
source = executeResponse.getWatchRecordAsMap();
value = XContentMapValues.extractValue("watch_execution.actions_results._email.email.success", source);
assertThat(value, notNullValue());
assertThat(value, is((Object) Boolean.TRUE));
if (!latch.await(5, TimeUnit.SECONDS)) {
fail("waiting too long for the email to be sent");
}
}
}

9
src/test/java/org/elasticsearch/watcher/test/integration/HttpInputIntegrationTest.java
View File

@ -15,7 +15,8 @@ import org.elasticsearch.test.junit.annotations.TestLogging;
import org.elasticsearch.watcher.client.WatcherClient;
import org.elasticsearch.watcher.history.HistoryStore;
import org.elasticsearch.watcher.support.http.HttpRequestTemplate;
import org.elasticsearch.watcher.support.http.auth.BasicAuth;
import org.elasticsearch.watcher.support.http.auth.basic.ApplicableBasicAuth;
import org.elasticsearch.watcher.support.http.auth.basic.BasicAuth;
import org.elasticsearch.watcher.support.template.Template;
import org.elasticsearch.watcher.test.AbstractWatcherIntegrationTests;
import org.elasticsearch.watcher.trigger.schedule.IntervalSchedule;
@ -59,7 +60,7 @@ public class HttpInputIntegrationTest extends AbstractWatcherIntegrationTests {
.input(httpInput(HttpRequestTemplate.builder(address.getHostName(), address.getPort())
.path("/index/_search")
.body(jsonBuilder().startObject().field("size", 1).endObject())
.auth(shieldEnabled() ? new BasicAuth("test", "changeme") : null)))
.auth(shieldEnabled() ? new BasicAuth("test", "changeme".toCharArray()) : null)))
.condition(scriptCondition("ctx.payload.hits.total == 1"))
.addAction("_id", loggingAction("watch [{{ctx.watch_id}}] matched")))
.get();
@ -80,7 +81,7 @@ public class HttpInputIntegrationTest extends AbstractWatcherIntegrationTests {
.trigger(schedule(interval("1s")))
.input(httpInput(HttpRequestTemplate.builder(address.getHostName(), address.getPort())
.path("/_cluster/stats")
.auth(shieldEnabled() ? new BasicAuth("test", "changeme") : null)))
.auth(shieldEnabled() ? new BasicAuth("test", "changeme".toCharArray()) : null)))
.condition(scriptCondition("ctx.payload.nodes.count.total > 1"))
.addAction("_id", loggingAction("watch [{{ctx.watch_id}}] matched")))
.get();
@ -109,7 +110,7 @@ public class HttpInputIntegrationTest extends AbstractWatcherIntegrationTests {
.path(new Template("/idx/_search"))
.body(body);
if (shieldEnabled()) {
requestBuilder.auth(new BasicAuth("test", "changeme"));
requestBuilder.auth(new BasicAuth("test", "changeme".toCharArray()));
}
watcherClient.preparePutWatch("_name1")

229
src/test/java/org/elasticsearch/watcher/test/integration/HttpSecretsIntegrationTests.java Normal file
View File

@ -0,0 +1,229 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.watcher.test.integration;
import com.squareup.okhttp.mockwebserver.MockResponse;
import com.squareup.okhttp.mockwebserver.MockWebServer;
import com.squareup.okhttp.mockwebserver.RecordedRequest;
import org.elasticsearch.action.get.GetResponse;
import org.elasticsearch.common.settings.ImmutableSettings;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.common.xcontent.support.XContentMapValues;
import org.elasticsearch.watcher.WatcherException;
import org.elasticsearch.watcher.client.WatcherClient;
import org.elasticsearch.watcher.shield.ShieldSecretService;
import org.elasticsearch.watcher.support.http.HttpRequestTemplate;
import org.elasticsearch.watcher.support.http.auth.basic.ApplicableBasicAuth;
import org.elasticsearch.watcher.support.http.auth.basic.BasicAuth;
import org.elasticsearch.watcher.support.secret.SecretService;
import org.elasticsearch.watcher.test.AbstractWatcherIntegrationTests;
import org.elasticsearch.watcher.transport.actions.execute.ExecuteWatchResponse;
import org.elasticsearch.watcher.transport.actions.get.GetWatchResponse;
import org.elasticsearch.watcher.watch.WatchStore;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import java.net.BindException;
import java.util.Map;
import static org.elasticsearch.common.xcontent.XContentFactory.jsonBuilder;
import static org.elasticsearch.watcher.actions.ActionBuilders.loggingAction;
import static org.elasticsearch.watcher.actions.ActionBuilders.webhookAction;
import static org.elasticsearch.watcher.client.WatchSourceBuilders.watchBuilder;
import static org.elasticsearch.watcher.condition.ConditionBuilders.alwaysCondition;
import static org.elasticsearch.watcher.input.InputBuilders.httpInput;
import static org.elasticsearch.watcher.input.InputBuilders.simpleInput;
import static org.elasticsearch.watcher.trigger.TriggerBuilders.schedule;
import static org.elasticsearch.watcher.trigger.schedule.Schedules.cron;
import static org.hamcrest.Matchers.*;
/**
*
*/
public class HttpSecretsIntegrationTests extends AbstractWatcherIntegrationTests {
static final String USERNAME = "_user";
static final String PASSWORD = "_passwd";
private MockWebServer webServer;
private static Boolean encryptSensitiveData;
@Before
public void init() throws Exception {
for (int webPort = 9200; webPort < 9300; webPort++) {
try {
webServer = new MockWebServer();
webServer.start(webPort);
return;
} catch (BindException be) {
logger.warn("port [{}] was already in use trying next port", webPort);
}
}
throw new WatcherException("unable to find open port between 9200 and 9300");
}
@After
public void cleanup() throws Exception {
webServer.shutdown();
}
@Override
protected Settings nodeSettings(int nodeOrdinal) {
if (encryptSensitiveData == null) {
encryptSensitiveData = shieldEnabled() && randomBoolean();
}
return ImmutableSettings.builder()
.put(super.nodeSettings(nodeOrdinal))
.put("watcher.shield.encrypt_sensitive_data", encryptSensitiveData)
.build();
}
@Test
public void testHttpInput() throws Exception {
WatcherClient watcherClient = watcherClient();
watcherClient.preparePutWatch("_id")
.setSource(watchBuilder()
.trigger(schedule(cron("0 0 0 1 * ? 2020")))
.input(httpInput(HttpRequestTemplate.builder(webServer.getHostName(), webServer.getPort())
.path("/")
.auth(new BasicAuth(USERNAME, PASSWORD.toCharArray()))))
.condition(alwaysCondition())
.addAction("_logging", loggingAction("executed")))
.get();
// verifying the basic auth password is stored encrypted in the index when shield
// is enabled, and when it's not enabled, it's stored in plain text
GetResponse response = client().prepareGet(WatchStore.INDEX, WatchStore.DOC_TYPE, "_id").get();
assertThat(response, notNullValue());
assertThat(response.getId(), is("_id"));
Map<String, Object> source = response.getSource();
Object value = XContentMapValues.extractValue("input.http.request.auth.basic.password", source);
assertThat(value, notNullValue());
if (shieldEnabled() && encryptSensitiveData) {
assertThat(value, not(is((Object) PASSWORD)));
SecretService secretService = getInstanceFromMaster(SecretService.class);
assertThat(secretService, instanceOf(ShieldSecretService.class));
assertThat(new String(secretService.decrypt(((String) value).toCharArray())), is(PASSWORD));
} else {
assertThat(value, is((Object) PASSWORD));
SecretService secretService = getInstanceFromMaster(SecretService.class);
if (shieldEnabled()) {
assertThat(secretService, instanceOf(ShieldSecretService.class));
} else {
assertThat(secretService, instanceOf(SecretService.PlainText.class));
}
assertThat(new String(secretService.decrypt(((String) value).toCharArray())), is(PASSWORD));
}
// verifying the password is not returned by the GET watch API
GetWatchResponse watchResponse = watcherClient.prepareGetWatch("_id").get();
assertThat(watchResponse, notNullValue());
assertThat(watchResponse.getId(), is("_id"));
source = watchResponse.getSourceAsMap();
value = XContentMapValues.extractValue("input.http.request.auth.basic", source);
assertThat(value, notNullValue()); // making sure we have the basic auth
value = XContentMapValues.extractValue("input.http.request.auth.basic.password", source);
assertThat(value, nullValue()); // and yet we don't have the password
// now we restart, to make sure the watches and their secrets are reloaded from the index properly
assertThat(watcherClient.prepareWatchService().restart().get().isAcknowledged(), is(true));
ensureWatcherStarted();
// now lets execute the watch manually
webServer.enqueue(new MockResponse().setResponseCode(200).setBody(jsonBuilder().startObject().field("key", "value").endObject().bytes().toUtf8()));
ExecuteWatchResponse executeResponse = watcherClient.prepareExecuteWatch("_id")
.setRecordExecution(false)
.setIgnoreThrottle(true)
.get();
assertThat(executeResponse, notNullValue());
source = executeResponse.getWatchRecordAsMap();
value = XContentMapValues.extractValue("watch_execution.input_result.http.http_status", source);
assertThat(value, notNullValue());
assertThat(value, is((Object) 200));
RecordedRequest request = webServer.takeRequest();
assertThat(request.getHeader("Authorization"), equalTo(ApplicableBasicAuth.headerValue(USERNAME, PASSWORD.toCharArray())));
}
@Test
public void testWebhookAction() throws Exception {
WatcherClient watcherClient = watcherClient();
watcherClient.preparePutWatch("_id")
.setSource(watchBuilder()
.trigger(schedule(cron("0 0 0 1 * ? 2020")))
.input(simpleInput())
.condition(alwaysCondition())
.addAction("_webhook", webhookAction(HttpRequestTemplate.builder(webServer.getHostName(), webServer.getPort())
.path("/")
.auth(new BasicAuth(USERNAME, PASSWORD.toCharArray())))))
.get();
// verifying the basic auth password is stored encrypted in the index when shield
// is enabled, when it's not enabled, the the passowrd should be stored in plain text
GetResponse response = client().prepareGet(WatchStore.INDEX, WatchStore.DOC_TYPE, "_id").get();
assertThat(response, notNullValue());
assertThat(response.getId(), is("_id"));
Map<String, Object> source = response.getSource();
Object value = XContentMapValues.extractValue("actions._webhook.webhook.auth.basic.password", source);
assertThat(value, notNullValue());
if (shieldEnabled() && encryptSensitiveData) {
assertThat(value, not(is((Object) PASSWORD)));
SecretService secretService = getInstanceFromMaster(SecretService.class);
assertThat(secretService, instanceOf(ShieldSecretService.class));
assertThat(new String(secretService.decrypt(((String) value).toCharArray())), is(PASSWORD));
} else {
assertThat(value, is((Object) PASSWORD));
SecretService secretService = getInstanceFromMaster(SecretService.class);
if (shieldEnabled()) {
assertThat(secretService, instanceOf(ShieldSecretService.class));
} else {
assertThat(secretService, instanceOf(SecretService.PlainText.class));
}
assertThat(new String(secretService.decrypt(((String) value).toCharArray())), is(PASSWORD));
}
// verifying the password is not returned by the GET watch API
GetWatchResponse watchResponse = watcherClient.prepareGetWatch("_id").get();
assertThat(watchResponse, notNullValue());
assertThat(watchResponse.getId(), is("_id"));
source = watchResponse.getSourceAsMap();
value = XContentMapValues.extractValue("actions._webhook.webhook.auth.basic", source);
assertThat(value, notNullValue()); // making sure we have the basic auth
value = XContentMapValues.extractValue("actions._webhook.webhook.auth.basic.password", source);
assertThat(value, nullValue()); // and yet we don't have the password
// now we restart, to make sure the watches and their secrets are reloaded from the index properly
assertThat(watcherClient.prepareWatchService().restart().get().isAcknowledged(), is(true));
ensureWatcherStarted();
// now lets execute the watch manually
webServer.enqueue(new MockResponse().setResponseCode(200).setBody(jsonBuilder().startObject().field("key", "value").endObject().bytes().toUtf8()));
ExecuteWatchResponse executeResponse = watcherClient.prepareExecuteWatch("_id")
.setRecordExecution(false)
.setIgnoreThrottle(true)
.get();
assertThat(executeResponse, notNullValue());
source = executeResponse.getWatchRecordAsMap();
value = XContentMapValues.extractValue("watch_execution.actions_results._webhook.webhook.response.status", source);
assertThat(value, notNullValue());
assertThat(value, is((Object) 200));
value = XContentMapValues.extractValue("watch_execution.actions_results._webhook.webhook.request.auth.username", source);
assertThat(value, notNullValue()); // the auth username exists
value = XContentMapValues.extractValue("watch_execution.actions_results._webhook.webhook.request.auth.password", source);
assertThat(value, nullValue()); // but the auth password was filtered out
RecordedRequest request = webServer.takeRequest();
assertThat(request.getHeader("Authorization"), equalTo(ApplicableBasicAuth.headerValue(USERNAME, PASSWORD.toCharArray())));
}
}

93
src/test/java/org/elasticsearch/watcher/test/integration/WatcherSettingsFilterTests.java Normal file
View File

@ -0,0 +1,93 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.watcher.test.integration;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.elasticsearch.common.settings.ImmutableSettings;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.common.xcontent.json.JsonXContent;
import org.elasticsearch.common.xcontent.support.XContentMapValues;
import org.elasticsearch.http.HttpServerTransport;
import org.elasticsearch.node.internal.InternalNode;
import org.elasticsearch.shield.authc.support.SecuredString;
import org.elasticsearch.test.rest.client.http.HttpRequestBuilder;
import org.elasticsearch.test.rest.client.http.HttpResponse;
import org.elasticsearch.watcher.test.AbstractWatcherIntegrationTests;
import org.junit.After;
import org.junit.Test;
import java.io.IOException;
import java.util.Map;
import static org.elasticsearch.shield.authc.support.UsernamePasswordToken.BASIC_AUTH_HEADER;
import static org.elasticsearch.shield.authc.support.UsernamePasswordToken.basicAuthHeaderValue;
import static org.elasticsearch.watcher.test.AbstractWatcherIntegrationTests.ShieldSettings.TEST_PASSWORD;
import static org.elasticsearch.watcher.test.AbstractWatcherIntegrationTests.ShieldSettings.TEST_USERNAME;
import static org.hamcrest.CoreMatchers.nullValue;
import static org.hamcrest.Matchers.is;
/**
*
*/
public class WatcherSettingsFilterTests extends AbstractWatcherIntegrationTests {
private CloseableHttpClient httpClient = HttpClients.createDefault();
@After
public void cleanup() throws IOException {
httpClient.close();
}
@Override
protected Settings nodeSettings(int nodeOrdinal) {
return ImmutableSettings.builder()
.put(super.nodeSettings(nodeOrdinal))
.put(InternalNode.HTTP_ENABLED, true)
.put("watcher.actions.email.service.account._email.smtp.host", "host.domain")
.put("watcher.actions.email.service.account._email.smtp.port", 587)
.put("watcher.actions.email.service.account._email.smtp.user", "_user")
.put("watcher.actions.email.service.account._email.smtp.password", "_passwd")
.build();
}
@Test
public void testGetSettings_SmtpPassword() throws Exception {
String body = executeRequest("GET", "/_nodes/settings", null, null).getBody();
Map<String, Object> response = JsonXContent.jsonXContent.createParser(body).map();
Map<String, Object> nodes = (Map<String, Object>) response.get("nodes");
for (Object node : nodes.values()) {
Map<String, Object> settings = (Map<String, Object>) ((Map<String, Object>) node).get("settings");
assertThat(XContentMapValues.extractValue("watcher.actions.email.service.account._email.smtp.user", settings), is((Object) "_user"));
if (shieldEnabled()) {
assertThat(XContentMapValues.extractValue("watcher.actions.email.service.account._email.smtp.password", settings), nullValue());
} else {
assertThat(XContentMapValues.extractValue("watcher.actions.email.service.account._email.smtp.password", settings), is((Object) "_passwd"));
}
}
}
protected HttpResponse executeRequest(String method, String path, String body, Map<String, String> params) throws IOException {
HttpServerTransport httpServerTransport = getInstanceFromMaster(HttpServerTransport.class);
HttpRequestBuilder requestBuilder = new HttpRequestBuilder(httpClient)
.httpTransport(httpServerTransport)
.method(method)
.path(path);
if (params != null) {
for (Map.Entry<String, String> entry : params.entrySet()) {
requestBuilder.addParam(entry.getKey(), entry.getValue());
}
}
if (body != null) {
requestBuilder.body(body);
}
if (shieldEnabled()) {
requestBuilder.addHeader(BASIC_AUTH_HEADER, basicAuthHeaderValue(TEST_USERNAME, new SecuredString(TEST_PASSWORD.toCharArray())));
}
return requestBuilder.execute();
}
}

10
src/test/java/org/elasticsearch/watcher/watch/WatchServiceTests.java
View File

@ -34,6 +34,7 @@ public class WatchServiceTests extends ElasticsearchTestCase {
private TriggerService triggerService;
private WatchStore watchStore;
private Watch.Parser watchParser;
private WatcherService watcherService;
private ExecutionService executionService;
private WatchLockService watchLockService;
@ -42,9 +43,10 @@ public class WatchServiceTests extends ElasticsearchTestCase {
public void init() throws Exception {
triggerService = mock(TriggerService.class);
watchStore = mock(WatchStore.class);
watchParser = mock(Watch.Parser.class);
executionService = mock(ExecutionService.class);
watchLockService = mock(WatchLockService.class);
watcherService = new WatcherService(ImmutableSettings.EMPTY, triggerService, watchStore, executionService, watchLockService);
watcherService = new WatcherService(ImmutableSettings.EMPTY, triggerService, watchStore, watchParser, executionService, watchLockService);
Field field = WatcherService.class.getDeclaredField("state");
field.setAccessible(true);
AtomicReference<WatcherService.State> state = (AtomicReference<WatcherService.State>) field.get(watcherService);
@ -61,7 +63,8 @@ public class WatchServiceTests extends ElasticsearchTestCase {
WatchLockService.Lock lock = mock(WatchLockService.Lock.class);
when(watchLockService.acquire(any(String.class))).thenReturn(lock);
when(watchStore.put(any(String.class), any(BytesReference.class))).thenReturn(watchPut);
when(watchParser.parseWithSecrets(any(String.class), eq(false), any(BytesReference.class))).thenReturn(watch);
when(watchStore.put(watch)).thenReturn(watchPut);
IndexResponse response = watcherService.putWatch("_name", new BytesArray("{}"));
assertThat(response, sameInstance(indexResponse));
@ -84,7 +87,8 @@ public class WatchServiceTests extends ElasticsearchTestCase {
WatchLockService.Lock lock = mock(WatchLockService.Lock.class);
when(watchLockService.acquire(any(String.class))).thenReturn(lock);
when(watchStore.put(any(String.class), any(BytesReference.class))).thenReturn(watchPut);
when(watchParser.parseWithSecrets(any(String.class), eq(false), any(BytesReference.class))).thenReturn(watch);
when(watchStore.put(watch)).thenReturn(watchPut);
IndexResponse response = watcherService.putWatch("_name", new BytesArray("{}"));
assertThat(response, sameInstance(indexResponse));

12
src/test/java/org/elasticsearch/watcher/watch/WatchTests.java
View File

@ -60,11 +60,12 @@ import org.elasticsearch.watcher.support.http.HttpClient;
import org.elasticsearch.watcher.support.http.HttpMethod;
import org.elasticsearch.watcher.support.http.HttpRequest;
import org.elasticsearch.watcher.support.http.HttpRequestTemplate;
import org.elasticsearch.watcher.support.http.auth.BasicAuth;
import org.elasticsearch.watcher.support.http.auth.HttpAuth;
import org.elasticsearch.watcher.support.http.auth.HttpAuthFactory;
import org.elasticsearch.watcher.support.http.auth.HttpAuthRegistry;
import org.elasticsearch.watcher.support.http.auth.basic.BasicAuthFactory;
import org.elasticsearch.watcher.support.init.proxy.ClientProxy;
import org.elasticsearch.watcher.support.init.proxy.ScriptServiceProxy;
import org.elasticsearch.watcher.support.secret.SecretService;
import org.elasticsearch.watcher.support.template.Template;
import org.elasticsearch.watcher.support.template.TemplateEngine;
import org.elasticsearch.watcher.test.WatcherTestUtils;
@ -104,6 +105,7 @@ public class WatchTests extends ElasticsearchTestCase {
private EmailService emailService;
private TemplateEngine templateEngine;
private HttpAuthRegistry authRegistry;
private SecretService secretService;
private ESLogger logger;
private Settings settings = ImmutableSettings.EMPTY;
@ -114,7 +116,8 @@ public class WatchTests extends ElasticsearchTestCase {
httpClient = mock(HttpClient.class);
emailService = mock(EmailService.class);
templateEngine = mock(TemplateEngine.class);
authRegistry = new HttpAuthRegistry(ImmutableMap.of("basic", (HttpAuth.Parser) new BasicAuth.Parser()));
secretService = mock(SecretService.class);
authRegistry = new HttpAuthRegistry(ImmutableMap.of("basic", (HttpAuthFactory) new BasicAuthFactory(secretService)));
logger = Loggers.getLogger(WatchTests.class);
}
@ -128,6 +131,7 @@ public class WatchTests extends ElasticsearchTestCase {
ScheduleRegistry scheduleRegistry = registry(schedule);
TriggerEngine triggerEngine = new ParseOnlyScheduleTriggerEngine(ImmutableSettings.EMPTY, scheduleRegistry);
TriggerService triggerService = new TriggerService(ImmutableSettings.EMPTY, ImmutableSet.of(triggerEngine));
SecretService secretService = new SecretService.PlainText();
ExecutableInput input = randomInput();
InputRegistry inputRegistry = registry(input);
@ -150,7 +154,7 @@ public class WatchTests extends ElasticsearchTestCase {
BytesReference bytes = XContentFactory.jsonBuilder().value(watch).bytes();
logger.info(bytes.toUtf8());
Watch.Parser watchParser = new Watch.Parser(settings, mock(LicenseService.class), conditionRegistry, triggerService, transformRegistry, actionRegistry, inputRegistry, SystemClock.INSTANCE);
Watch.Parser watchParser = new Watch.Parser(settings, mock(LicenseService.class), conditionRegistry, triggerService, transformRegistry, actionRegistry, inputRegistry, SystemClock.INSTANCE, secretService);
boolean includeStatus = randomBoolean();
Watch parsedWatch = watchParser.parse("_name", includeStatus, bytes);