Simplify Plugin Manager for official plugins

Plugin Manager can now use another simplified form when a user wants to install an official plugin hosted at elasticsearch download service.

The form we use is:

```sh
bin/plugin install pluginname
```

As plugins share now the same version as elasticsearch, we can automatically guess what is the exact current version of the plugin manager script.

Also, download service will now use `/org.elasticsearch.plugins/pluginName/pluginName-version.zip` URL path to download a plugin.

If the older form is provided (`user/plugin/version` or `user/plugin`), we will still use:

 * elasticsearch download service at `/user/plugin/plugin-version.zip`
 * maven central with groupIp=user, artifactId=plugin and version=version
 * github with user=user, repoName=plugin and tag=version
 * github with user=user, repoName=plugin and branch=master if no version is set

Note that community plugin providers can use other download services by using `--url` option.

If you try to use the new form with a non core elasticsearch plugin, the plugin manager will reject
it and will give you all known core plugins.

```
Usage:
    -u, --url     [plugin location]   : Set exact URL to download the plugin from
    -i, --install [plugin name]       : Downloads and installs listed plugins [*]
    -t, --timeout [duration]          : Timeout setting: 30s, 1m, 1h... (infinite by default)
    -r, --remove  [plugin name]       : Removes listed plugins
    -l, --list                        : List installed plugins
    -v, --verbose                     : Prints verbose messages
    -s, --silent                      : Run in silent mode
    -h, --help                        : Prints this help message

 [*] Plugin name could be:
     elasticsearch-plugin-name    for Elasticsearch 2.0 Core plugin (download from download.elastic.co)
     elasticsearch/plugin/version for elasticsearch commercial plugins (download from download.elastic.co)
     groupId/artifactId/version   for community plugins (download from maven central or oss sonatype)
     username/repository          for site plugins (download from github master)

Elasticsearch Core plugins:
 - elasticsearch-analysis-icu
 - elasticsearch-analysis-kuromoji
 - elasticsearch-analysis-phonetic
 - elasticsearch-analysis-smartcn
 - elasticsearch-analysis-stempel
 - elasticsearch-cloud-aws
 - elasticsearch-cloud-azure
 - elasticsearch-cloud-gce
 - elasticsearch-delete-by-query
 - elasticsearch-lang-javascript
 - elasticsearch-lang-python
```

core/src/main/java/org/elasticsearch/plugins/PluginManager.java

@@ -73,7 +73,7 @@ public class PluginManager {
     // By default timeout is 0 which means no timeout
     public static final TimeValue DEFAULT_TIMEOUT = TimeValue.timeValueMillis(0);
 
-    private static final ImmutableSet<Object> BLACKLIST = ImmutableSet.builder()
+    private static final ImmutableSet<String> BLACKLIST = ImmutableSet.<String>builder()
             .add("elasticsearch",
                     "elasticsearch.bat",
                     "elasticsearch.in.sh",
@@ -81,6 +81,21 @@ public class PluginManager {
                     "plugin.bat",
                     "service.bat").build();
 
+    private static final ImmutableSet<String> OFFICIAL_PLUGINS = ImmutableSet.<String>builder()
+            .add(
+                    "elasticsearch-analysis-icu",
+                    "elasticsearch-analysis-kuromoji",
+                    "elasticsearch-analysis-phonetic",
+                    "elasticsearch-analysis-smartcn",
+                    "elasticsearch-analysis-stempel",
+                    "elasticsearch-cloud-aws",
+                    "elasticsearch-cloud-azure",
+                    "elasticsearch-cloud-gce",
+                    "elasticsearch-delete-by-query",
+                    "elasticsearch-lang-javascript",
+                    "elasticsearch-lang-python"
+            ).build();
+
     private final Environment environment;
     private String url;
     private OutputMode outputMode;
@@ -133,6 +148,10 @@ public class PluginManager {
                 // ignore
                 log("Failed: " + ExceptionsHelper.detailedMessage(e));
             }
+        } else {
+            if (PluginHandle.isOfficialPlugin(pluginHandle.repo, pluginHandle.user, pluginHandle.version)) {
+                checkForOfficialPlugins(pluginHandle.name);
+            }
         }
 
         if (!downloaded) {
@@ -384,6 +403,15 @@ public class PluginManager {
         }
     }
 
+    protected static void checkForOfficialPlugins(String name) {
+        // We make sure that users can use only new short naming for official plugins only
+        if (!OFFICIAL_PLUGINS.contains(name)) {
+            throw new IllegalArgumentException(name +
+                    " is not an official plugin so you should install it using elasticsearch/" +
+                    name + "/latest naming form.");
+        }
+    }
+
     public Path[] getListInstalledPlugins() throws IOException {
         try (DirectoryStream<Path> stream = Files.newDirectoryStream(environment.pluginsFile())) {
             return Iterators.toArray(stream.iterator(), Path.class);
@@ -597,9 +625,15 @@ public class PluginManager {
         SysOut.println("    -h, --help                        : Prints this help message");
         SysOut.newline();
         SysOut.println(" [*] Plugin name could be:");
-        SysOut.println("     elasticsearch/plugin/version for official elasticsearch plugins (download from download.elasticsearch.org)");
+        SysOut.println("     elasticsearch-plugin-name    for Elasticsearch 2.0 Core plugin (download from download.elastic.co)");
+        SysOut.println("     elasticsearch/plugin/version for elasticsearch commercial plugins (download from download.elastic.co)");
         SysOut.println("     groupId/artifactId/version   for community plugins (download from maven central or oss sonatype)");
         SysOut.println("     username/repository          for site plugins (download from github master)");
+        SysOut.newline();
+        SysOut.println("Elasticsearch Core plugins:");
+        for (String o : OFFICIAL_PLUGINS) {
+            SysOut.println(" - " + o);
+        }
 
         if (message != null) {
             SysOut.newline();
@@ -652,17 +686,26 @@ public class PluginManager {
         List<URL> urls() {
             List<URL> urls = new ArrayList<>();
             if (version != null) {
-                // Elasticsearch download service
+                // Elasticsearch new download service uses groupId org.elasticsearch.plugins from 2.0.0
-                addUrl(urls, "http://download.elasticsearch.org/" + user + "/" + repo + "/" + repo + "-" + version + ".zip");
+                if (user == null) {
+                    // TODO Update to https
+                    addUrl(urls, String.format(Locale.ROOT, "http://download.elastic.co/org.elasticsearch.plugins/%1$s/%1$s-%2$s.zip", repo, version));
+                } else {
+                    // Elasticsearch old download service
+                    // TODO Update to https
+                    addUrl(urls, String.format(Locale.ROOT, "http://download.elastic.co/%1$s/%2$s/%2$s-%3$s.zip", user, repo, version));
                     // Maven central repository
-                addUrl(urls, "http://search.maven.org/remotecontent?filepath=" + user.replace('.', '/') + "/" + repo + "/" + version + "/" + repo + "-" + version + ".zip");
+                    addUrl(urls, String.format(Locale.ROOT, "http://search.maven.org/remotecontent?filepath=%1$s/%2$s/%3$s/%2$s-%3$s.zip", user.replace('.', '/'), repo, version));
                     // Sonatype repository
-                addUrl(urls, "https://oss.sonatype.org/service/local/repositories/releases/content/" + user.replace('.', '/') + "/" + repo + "/" + version + "/" + repo + "-" + version + ".zip");
+                    addUrl(urls, String.format(Locale.ROOT, "https://oss.sonatype.org/service/local/repositories/releases/content/%1$s/%2$s/%3$s/%2$s-%3$s.zip", user.replace('.', '/'), repo, version));
                     // Github repository
-                addUrl(urls, "https://github.com/" + user + "/" + repo + "/archive/" + version + ".zip");
+                    addUrl(urls, String.format(Locale.ROOT, "https://github.com/%1$s/%2$s/archive/%3$s.zip", user, repo, version));
                 }
+            }
+            if (user != null) {
                 // Github repository for master branch (assume site)
-            addUrl(urls, "https://github.com/" + user + "/" + repo + "/archive/master.zip");
+                addUrl(urls, String.format(Locale.ROOT, "https://github.com/%1$s/%2$s/archive/master.zip", user, repo));
+            }
             return urls;
         }
 
@@ -708,6 +751,10 @@ public class PluginManager {
                 }
             }
 
+            if (isOfficialPlugin(repo, user, version)) {
+                return new PluginHandle(repo, Version.CURRENT.number(), null, repo);
+            }
+
             if (repo.startsWith("elasticsearch-")) {
                 // remove elasticsearch- prefix
                 String endname = repo.substring("elasticsearch-".length());
@@ -722,6 +769,10 @@ public class PluginManager {
 
             return new PluginHandle(repo, version, user, repo);
         }
+
+        static boolean isOfficialPlugin(String repo, String user, String version) {
+            return version == null && user == null && !Strings.isNullOrEmpty(repo);
+        }
     }
 
 }

core/src/test/java/org/elasticsearch/plugins/PluginManagerTests.java

@@ -54,9 +54,7 @@ import static org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertDire
 import static org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertFileExists;
 import static org.hamcrest.CoreMatchers.is;
 import static org.hamcrest.CoreMatchers.not;
-import static org.hamcrest.Matchers.arrayWithSize;
+import static org.hamcrest.Matchers.*;
-import static org.hamcrest.Matchers.equalTo;
-import static org.hamcrest.Matchers.notNullValue;
 
 @ClusterScope(scope = Scope.TEST, numDataNodes = 0, transportClientRatio = 0.0)
 @LuceneTestCase.SuppressFileSystems("*") // TODO: clean up this test to allow extra files
@@ -513,6 +511,27 @@ public class PluginManagerTests extends ElasticsearchIntegrationTest {
         }
     }
 
+    @Test
+    public void testOfficialPluginName_ThrowsException() throws IOException {
+        PluginManager.checkForOfficialPlugins("elasticsearch-analysis-icu");
+        PluginManager.checkForOfficialPlugins("elasticsearch-analysis-kuromoji");
+        PluginManager.checkForOfficialPlugins("elasticsearch-analysis-phonetic");
+        PluginManager.checkForOfficialPlugins("elasticsearch-analysis-smartcn");
+        PluginManager.checkForOfficialPlugins("elasticsearch-analysis-stempel");
+        PluginManager.checkForOfficialPlugins("elasticsearch-cloud-aws");
+        PluginManager.checkForOfficialPlugins("elasticsearch-cloud-azure");
+        PluginManager.checkForOfficialPlugins("elasticsearch-cloud-gce");
+        PluginManager.checkForOfficialPlugins("elasticsearch-delete-by-query");
+        PluginManager.checkForOfficialPlugins("elasticsearch-lang-javascript");
+        PluginManager.checkForOfficialPlugins("elasticsearch-lang-python");
+
+        try {
+            PluginManager.checkForOfficialPlugins("elasticsearch-mapper-attachment");
+            fail("elasticsearch-mapper-attachment should not be allowed");
+        } catch (IllegalArgumentException e) {
+            // We expect that error
+        }
+    }
 
     /**
      * Retrieve a URL string that represents the resource with the given {@code resourceName}.

core/src/test/java/org/elasticsearch/plugins/PluginManagerUnitTests.java

@@ -20,16 +20,18 @@
 package org.elasticsearch.plugins;
 
 import com.google.common.io.Files;
-
+import org.elasticsearch.Version;
 import org.elasticsearch.common.settings.Settings;
 import org.elasticsearch.env.Environment;
 import org.elasticsearch.test.ElasticsearchTestCase;
 import org.junit.Test;
 
 import java.io.IOException;
+import java.net.URL;
 import java.nio.file.Path;
 
 import static org.elasticsearch.common.settings.Settings.settingsBuilder;
+import static org.hamcrest.Matchers.hasSize;
 import static org.hamcrest.Matchers.is;
 
 /**
@@ -55,4 +57,14 @@ public class PluginManagerUnitTests extends ElasticsearchTestCase {
 
         assertThat(configDirPath, is(expectedDirPath));
     }
+
+    @Test
+    public void testSimplifiedNaming() throws IOException {
+        String pluginName = randomAsciiOfLength(10);
+        PluginManager.PluginHandle handle = PluginManager.PluginHandle.parse(pluginName);
+        assertThat(handle.urls(), hasSize(1));
+        URL expected = new URL("http", "download.elastic.co", "/org.elasticsearch.plugins/" + pluginName + "/" +
+            pluginName + "-" + Version.CURRENT.number() + ".zip");
+        assertThat(handle.urls().get(0), is(expected));
+    }
 }

docs/reference/migration/migrate_2_0.asciidoc

@@ -731,3 +731,42 @@ to prevent clashes with the watcher plugin
 === Percolator stats
 
 Changed the `percolate.getTime` stat (total time spent on percolating) to `percolate.time` state.
+
+=== Plugin Manager for official plugins
+
+Some of the elasticsearch official plugins have been moved to elasticsearch repository and will be released at the
+same time as elasticsearch itself, using the same version number.
+
+In that case, the plugin manager can now use a simpler form to identify an official plugin. Instead of:
+
+[source,sh]
+---------------
+bin/plugin install elasticsearch/plugin_name/version
+---------------
+
+You can use:
+
+[source,sh]
+---------------
+bin/plugin install plugin_name
+---------------
+
+The plugin manager will recognize this form and will be able to download the right version for your elasticsearch
+version.
+
+For older versions of elasticsearch, you still have to use the older form.
+
+For the record, official plugins which can use this new simplified form are:
+
+* elasticsearch-analysis-icu
+* elasticsearch-analysis-kuromoji
+* elasticsearch-analysis-phonetic
+* elasticsearch-analysis-smartcn
+* elasticsearch-analysis-stempel
+* elasticsearch-cloud-aws
+* elasticsearch-cloud-azure
+* elasticsearch-cloud-gce
+* elasticsearch-delete-by-query
+* elasticsearch-lang-javascript
+* elasticsearch-lang-python
+

docs/reference/modules/plugins.asciidoc

@@ -14,19 +14,26 @@ and more.
 ==== Installing plugins
 
 Installing plugins can either be done manually by placing them under the
-`plugins` directory, or using the `plugin` script. Several plugins can
+`plugins` directory, or using the `plugin` script.
-be found under the https://github.com/elasticsearch[elasticsearch]
-organization in GitHub, starting with `elasticsearch-`.
 
 Installing plugins typically take the following form:
 
+[source,shell]
+-----------------------------------
+bin/plugin --install plugin_name
+-----------------------------------
+
+The plugin will be automatically downloaded in this case from `download.elastic.co` download service using the
+same version as your elasticsearch version.
+
+For older version of elasticsearch (prior to 2.0.0) or community plugins, you would use the following form:
+
 [source,shell]
 -----------------------------------
 bin/plugin --install <org>/<user/component>/<version>
 -----------------------------------
 
-The plugins will be
+The plugins will be automatically downloaded in this case from `download.elastic.co` (for older plugins),
-automatically downloaded in this case from `download.elastic.co`,
 and in case they don't exist there, from maven (central and sonatype).
 
 Note that when the plugin is located in maven central or sonatype