mirror of https://github.com/apache/druid.git
Upgrade various build and doc links to https. (#7722)
* Upgrade various build and doc links to https. Where it wasn't possible to upgrade build-time dependencies to https, I kept http in place but used hardcoded checksums or GPG keys to ensure that artifacts fetched over http are verified properly. * Switch to https://apache.org.
This commit is contained in:
Gian Merlino
Fangjin Yang
parent
dd7dace70a
commit
b6941551ae
|
|
@ -21,7 +21,7 @@ ARG DRUID_RELEASE=druid/druid:0.14.0
|
|||
FROM $DRUID_RELEASE
|
||||
|
||||
COPY sha256sums.txt /tmp
|
||||
RUN wget -O /opt/druid/extensions/mysql-metadata-storage/mysql-connector-java-5.1.38.jar http://central.maven.org/maven2/mysql/mysql-connector-java/5.1.38/mysql-connector-java-5.1.38.jar \
|
||||
RUN wget -O /opt/druid/extensions/mysql-metadata-storage/mysql-connector-java-5.1.38.jar https://repo1.maven.org/maven2/mysql/mysql-connector-java/5.1.38/mysql-connector-java-5.1.38.jar \
|
||||
&& sed -e '/^#/d' /tmp/sha256sums.txt > /tmp/sha256sums-stripped.txt \
|
||||
&& sha256sum -c /tmp/sha256sums-stripped.txt \
|
||||
&& rm -f /opt/druid/lib/mysql-connector-java-5.1.38.jar \
|
||||
|
|
|
|||
|
|
@ -24,7 +24,7 @@ title: "DataSketches extension"
|
|||
|
||||
# DataSketches extension
|
||||
|
||||
Apache Druid (incubating) aggregators based on [datasketches](http://datasketches.github.io/) library. Sketches are data structures implementing approximate streaming mergeable algorithms. Sketches can be ingested from the outside of Druid or built from raw data at ingestion time. Sketches can be stored in Druid segments as additive metrics.
|
||||
Apache Druid (incubating) aggregators based on [datasketches](https://datasketches.github.io/) library. Sketches are data structures implementing approximate streaming mergeable algorithms. Sketches can be ingested from the outside of Druid or built from raw data at ingestion time. Sketches can be stored in Druid segments as additive metrics.
|
||||
|
||||
To use the datasketches aggregators, make sure you [include](../../operations/including-extensions.html) the extension in your config file:
|
||||
|
||||
|
|
|
|||
|
|
@ -24,7 +24,7 @@ title: "DataSketches HLL Sketch module"
|
|||
|
||||
# DataSketches HLL Sketch module
|
||||
|
||||
This module provides Apache Druid (incubating) aggregators for distinct counting based on HLL sketch from [datasketches](http://datasketches.github.io/) library. At ingestion time, this aggregator creates the HLL sketch objects to be stored in Druid segments. At query time, sketches are read and merged together. In the end, by default, you receive the estimate of the number of distinct values presented to the sketch. Also, you can use post aggregator to produce a union of sketch columns in the same row.
|
||||
This module provides Apache Druid (incubating) aggregators for distinct counting based on HLL sketch from [datasketches](https://datasketches.github.io/) library. At ingestion time, this aggregator creates the HLL sketch objects to be stored in Druid segments. At query time, sketches are read and merged together. In the end, by default, you receive the estimate of the number of distinct values presented to the sketch. Also, you can use post aggregator to produce a union of sketch columns in the same row.
|
||||
You can use the HLL sketch aggregator on columns of any identifiers. It will return estimated cardinality of the column.
|
||||
|
||||
To use this aggregator, make sure you [include](../../operations/including-extensions.html) the extension in your config file:
|
||||
|
|
|
|||
|
|
@ -24,7 +24,7 @@ title: "DataSketches Quantiles Sketch module"
|
|||
|
||||
# DataSketches Quantiles Sketch module
|
||||
|
||||
This module provides Apache Druid (incubating) aggregators based on numeric quantiles DoublesSketch from [datasketches](http://datasketches.github.io/) library. Quantiles sketch is a mergeable streaming algorithm to estimate the distribution of values, and approximately answer queries about the rank of a value, probability mass function of the distribution (PMF) or histogram, cummulative distribution function (CDF), and quantiles (median, min, max, 95th percentile and such). See [Quantiles Sketch Overview](https://datasketches.github.io/docs/Quantiles/QuantilesOverview.html).
|
||||
This module provides Apache Druid (incubating) aggregators based on numeric quantiles DoublesSketch from [datasketches](https://datasketches.github.io/) library. Quantiles sketch is a mergeable streaming algorithm to estimate the distribution of values, and approximately answer queries about the rank of a value, probability mass function of the distribution (PMF) or histogram, cummulative distribution function (CDF), and quantiles (median, min, max, 95th percentile and such). See [Quantiles Sketch Overview](https://datasketches.github.io/docs/Quantiles/QuantilesOverview.html).
|
||||
|
||||
There are three major modes of operation:
|
||||
|
||||
|
|
|
|||
|
|
@ -24,7 +24,7 @@ title: "DataSketches Theta Sketch module"
|
|||
|
||||
# DataSketches Theta Sketch module
|
||||
|
||||
This module provides Apache Druid (incubating) aggregators based on Theta sketch from [datasketches](http://datasketches.github.io/) library. Note that sketch algorithms are approximate; see details in the "Accuracy" section of the datasketches doc.
|
||||
This module provides Apache Druid (incubating) aggregators based on Theta sketch from [datasketches](https://datasketches.github.io/) library. Note that sketch algorithms are approximate; see details in the "Accuracy" section of the datasketches doc.
|
||||
At ingestion time, this aggregator creates the Theta sketch objects which get stored in Druid segments. Logically speaking, a Theta sketch object can be thought of as a Set data structure. At query time, sketches are read and aggregated (set unioned) together. In the end, by default, you receive the estimate of the number of unique entries in the sketch object. Also, you can use post aggregators to do union, intersection or difference on sketch columns in the same row.
|
||||
Note that you can use `thetaSketch` aggregator on columns which were not ingested using the same. It will return estimated cardinality of the column. It is recommended to use it at ingestion time as well to make querying faster.
|
||||
|
||||
|
|
|
|||
|
|
@ -24,7 +24,7 @@ title: "DataSketches Tuple Sketch module"
|
|||
|
||||
# DataSketches Tuple Sketch module
|
||||
|
||||
This module provides Apache Druid (incubating) aggregators based on Tuple sketch from [datasketches](http://datasketches.github.io/) library. ArrayOfDoublesSketch sketches extend the functionality of the count-distinct Theta sketches by adding arrays of double values associated with unique keys.
|
||||
This module provides Apache Druid (incubating) aggregators based on Tuple sketch from [datasketches](https://datasketches.github.io/) library. ArrayOfDoublesSketch sketches extend the functionality of the count-distinct Theta sketches by adding arrays of double values associated with unique keys.
|
||||
|
||||
To use this aggregator, make sure you [include](../../operations/including-extensions.html) the extension in your config file:
|
||||
|
||||
|
|
|
|||
|
|
@ -45,7 +45,7 @@ Core extensions are maintained by Druid committers.
|
|||
|druid-basic-security|Support for Basic HTTP authentication and role-based access control.|[link](../development/extensions-core/druid-basic-security.html)|
|
||||
|druid-bloom-filter|Support for providing Bloom filters in druid queries.|[link](../development/extensions-core/bloom-filter.html)|
|
||||
|druid-caffeine-cache|A local cache implementation backed by Caffeine.|[link](../configuration/index.html#cache-configuration)|
|
||||
|druid-datasketches|Support for approximate counts and set operations with [DataSketches](http://datasketches.github.io/).|[link](../development/extensions-core/datasketches-extension.html)|
|
||||
|druid-datasketches|Support for approximate counts and set operations with [DataSketches](https://datasketches.github.io/).|[link](../development/extensions-core/datasketches-extension.html)|
|
||||
|druid-hdfs-storage|HDFS deep storage.|[link](../development/extensions-core/hdfs.html)|
|
||||
|druid-histogram|Approximate histograms and quantiles aggregator. Deprecated, please use the [DataSketches quantiles aggregator](../development/extensions-core/datasketches-quantiles.html) from the `druid-datasketches` extension instead.|[link](../development/extensions-core/approximate-histograms.html)|
|
||||
|druid-kafka-eight|Kafka ingest firehose (high level consumer) for realtime nodes(deprecated).|[link](../development/extensions-core/kafka-eight-firehose.html)|
|
||||
|
|
|
|||
|
|
@ -271,7 +271,7 @@ JavaScript-based functionality is disabled by default. Please refer to the Druid
|
|||
|
||||
#### DataSketches Theta Sketch
|
||||
|
||||
The [DataSketches Theta Sketch](../development/extensions-core/datasketches-theta.html) extension-provided aggregator gives distinct count estimates with support for set union, intersection, and difference post-aggregators, using Theta sketches from the [datasketches](http://datasketches.github.io/) library.
|
||||
The [DataSketches Theta Sketch](../development/extensions-core/datasketches-theta.html) extension-provided aggregator gives distinct count estimates with support for set union, intersection, and difference post-aggregators, using Theta sketches from the [datasketches](https://datasketches.github.io/) library.
|
||||
|
||||
#### DataSketches HLL Sketch
|
||||
|
||||
|
|
@ -304,7 +304,7 @@ Note the DataSketches Theta and HLL aggregators currently only support single-co
|
|||
|
||||
#### DataSketches Quantiles Sketch
|
||||
|
||||
The [DataSketches Quantiles Sketch](../development/extensions-core/datasketches-quantiles.html) extension-provided aggregator provides quantile estimates and histogram approximations using the numeric quantiles DoublesSketch from the [datasketches](http://datasketches.github.io/) library.
|
||||
The [DataSketches Quantiles Sketch](../development/extensions-core/datasketches-quantiles.html) extension-provided aggregator provides quantile estimates and histogram approximations using the numeric quantiles DoublesSketch from the [datasketches](https://datasketches.github.io/) library.
|
||||
|
||||
We recommend this aggregator in general for quantiles/histogram use cases, as it provides formal error bounds and has distribution-independent accuracy.
|
||||
|
||||
|
|
|
|||
|
|
@ -41,9 +41,8 @@ RUN ssh-keygen -q -N "" -t rsa -f /root/.ssh/id_rsa
|
|||
RUN cp /root/.ssh/id_rsa.pub /root/.ssh/authorized_keys
|
||||
|
||||
# zulu java 8
|
||||
RUN rpm --import http://repos.azulsystems.com/RPM-GPG-KEY-azulsystems
|
||||
RUN rpm --rebuilddb
|
||||
RUN sudo curl -o /etc/yum.repos.d/zulu.repo http://repos.azulsystems.com/rhel/zulu.repo
|
||||
COPY setup-zulu-repo.sh /root/setup-zulu-repo.sh
|
||||
RUN /root/setup-zulu-repo.sh
|
||||
RUN yum install -y zulu-8
|
||||
|
||||
ENV JAVA_HOME /usr/lib/jvm/zulu-8
|
||||
|
|
|
|||
|
|
@ -0,0 +1,67 @@
|
|||
#!/bin/bash
|
||||
|
||||
# Licensed to the Apache Software Foundation (ASF) under one or more
|
||||
# contributor license agreements. See the NOTICE file distributed with
|
||||
# this work for additional information regarding copyright ownership.
|
||||
# The ASF licenses this file to You under the Apache License, Version 2.0
|
||||
# (the "License"); you may not use this file except in compliance with
|
||||
# the License. You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
#
|
||||
# Script to set up the Azul Zulu JDK yum repository.
|
||||
#
|
||||
|
||||
# Hardcode GPG key so we don't have to fetch it over http.
|
||||
cat <<'EOT' > /root/RPM-GPG-KEY-azulsystems
|
||||
-----BEGIN PGP PUBLIC KEY BLOCK-----
|
||||
Version: GnuPG v1
|
||||
|
||||
mQINBFNgFa8BEADTL/REB10M+TfiZOtFHqL5LHKkzTMn/O2r5iIqXGhi6iwZazFs
|
||||
9S5g1eU7WMen5Xp9AREs+OvaHx91onPZ7ZiP7VpZ6ZdwWrnVk1Y/HfI59tWxmNYW
|
||||
DmKYBGMj4EUpFPSE9EnFj7dm1WdlCvpognCwZQl9D3BseGqN7OLHfwqqmOlbYN9h
|
||||
HYkT+CaqOoWDIGMB3UkBlMr0GuujEP8N1gxg7EOcSCsZH5aKtXubdUlVSphfAAwD
|
||||
z4MviB39J22sPBnKmaOT3TUTO5vGeKtC9BAvtgA82jY2TtCEjetnfK/qtzj/6j2N
|
||||
xVUbHQydwNQVRU92A7334YvCbn3xUUNI0WOscdmfpgCU0Z9Gb2IqDb9cMjgUi8F6
|
||||
MG/QY9/CZjX62XrHRPm3aXsCJOVh/PO1sl2A/rvv8AkpJKYyhm6T8OBFptCsA3V4
|
||||
Oic7ZyYhqV0u2r4NON+1MoUeuuoeY2tIrbRxe3ffVOxPzrESzSbc8LC2tYaP+wGd
|
||||
W0f57/CoDkUzlvpReCUI1Bv5zP4/jhC63Rh6lffvSf2tQLwOsf5ivPhUtwUfOQjg
|
||||
v9P8Wc8K7XZpSOMnDZuDe9wuvB/DiH/P5yiTs2RGsbDdRh5iPfwbtf2+IX6h2lNZ
|
||||
XiDKt9Gc26uzeJRx/c7+sLunxq6DLIYvrsEipVI9frHIHV6fFTmqMJY6SwARAQAB
|
||||
tEdBenVsIFN5c3RlbXMsIEluYy4gKFBhY2thZ2Ugc2lnbmluZyBrZXkuKSA8cGtp
|
||||
LXNpZ25pbmdAYXp1bHN5c3RlbXMuY29tPokCOAQTAQIAIgUCU2AVrwIbAwYLCQgH
|
||||
AwIGFQgCCQoLBBYCAwECHgECF4AACgkQsZmDYSGb2cnJ8xAAz1V1PJnfOyaRIP2N
|
||||
Ho2uRwGdPsA4eFMXb4Z08eGjDMD3b9WW3D0XnCLbJpaZ6klz0W0s2tcYSneTBaSs
|
||||
RAqxgJgBZ5ZMXtrrHld/5qFoBbStLZLefmcPhnfvamwHDCTLUex8NIAI1u3e9Rhb
|
||||
5fbH+gpuYpwHX7hz0FOfpn1sxR03UyxU+ey4AdKe9LG3TJVnB0WcgxpobpbqweLH
|
||||
yzcEQCNoFV3r1rlE13Y0aE31/9apoEwiYvqAzEmE38TukDLl/Qg8rkR1t0/lok2P
|
||||
G6pWqdN7pmoUovBTvDi5YOthcjZcdOTXXn2Yw4RZVF9uhRsVfku1Eg25SnOje3uY
|
||||
smtQLME4eESbePdjyV/okCIle66uHZse+7gNyNmWpf01hM+VmAySIAyKa0Ku8AXZ
|
||||
MydEcJTebrNfW9uMLsBx3Ts7z/CBfRng6F8louJGlZtlSwddTkZVcb26T20xeo0a
|
||||
ZvdFXM2djTi/a5nbBoZQL85AEeV7HaphFLdPrgmMtS8sSZUEVvdaxp7WJsVuF9cO
|
||||
Nxsvx40OYTvfco0W41Lm8/sEuQ7YueEVpZxiv5kX56GTU9vXaOOi+8Z7Ee2w6Adz
|
||||
4hrGZkzztggs4tM9geNYnd0XCdZ/ICAskKJABg7biDD1PhEBrqCIqSE3U497vibQ
|
||||
Mpkkl/Zpp0BirhGWNyTg8K4JrsQ=
|
||||
=d320
|
||||
-----END PGP PUBLIC KEY BLOCK-----
|
||||
EOT
|
||||
|
||||
rpm --import /root/RPM-GPG-KEY-azulsystems
|
||||
rpm --rebuilddb
|
||||
|
||||
# Do not include "gpgkey" in the repo definition -- we've already imported it, above.
|
||||
cat <<'EOT' > /etc/yum.repos.d/zulu.repo
|
||||
[zulu]
|
||||
name=zulu-$releasever - Azul Systems Inc., Zulu packages for $basearch
|
||||
baseurl=http://repos.azulsystems.com/rhel/$releasever/$basearch
|
||||
enabled=1
|
||||
gpgcheck=1
|
||||
protect=1
|
||||
EOT
|
||||
|
|
@ -84,7 +84,7 @@
|
|||
<repository>
|
||||
<id>hortonworks</id>
|
||||
<name>hortonworks</name>
|
||||
<url>http://repo.hortonworks.com/content/repositories/releases</url>
|
||||
<url>https://repo.hortonworks.com/content/repositories/releases</url>
|
||||
</repository>
|
||||
</repositories>
|
||||
</project>
|
||||
|
|
|
|||
|
|
@ -44,7 +44,7 @@
|
|||
<repositories>
|
||||
<repository>
|
||||
<id>confluent</id>
|
||||
<url>http://packages.confluent.io/maven/</url>
|
||||
<url>https://packages.confluent.io/maven/</url>
|
||||
</repository>
|
||||
</repositories>
|
||||
|
||||
|
|
|
|||
|
|
@ -17,7 +17,7 @@
|
|||
~ under the License.
|
||||
-->
|
||||
|
||||
This module provides druid aggregators based on http://datasketches.github.io/ .
|
||||
This module provides Druid aggregators based on https://datasketches.github.io/.
|
||||
|
||||
Credits: This module is a result of feedback and work done by following people.
|
||||
|
||||
|
|
|
|||
|
|
@ -25,7 +25,7 @@
|
|||
<groupId>org.apache.druid.extensions</groupId>
|
||||
<artifactId>druid-datasketches</artifactId>
|
||||
<name>druid-datasketches</name>
|
||||
<description>Druid Aggregators based on datasketches lib http://datasketches.github.io/</description>
|
||||
<description>Druid Aggregators based on datasketches lib https://datasketches.github.io/</description>
|
||||
|
||||
<parent>
|
||||
<groupId>org.apache.druid</groupId>
|
||||
|
|
|
|||
|
|
@ -34,7 +34,7 @@
|
|||
<licenses>
|
||||
<license>
|
||||
<name>Apache License, Version 2.0</name>
|
||||
<url>http://www.apache.org/licenses/LICENSE-2.0</url>
|
||||
<url>https://www.apache.org/licenses/LICENSE-2.0</url>
|
||||
</license>
|
||||
</licenses>
|
||||
|
||||
|
|
|
|||
|
|
@ -34,14 +34,18 @@ apt-get install -y mysql-server
|
|||
apt-get install -y supervisor
|
||||
|
||||
# Zookeeper
|
||||
wget -q -O - http://www.us.apache.org/dist/zookeeper/zookeeper-3.4.13/zookeeper-3.4.13.tar.gz | tar -xzf - -C /usr/local \
|
||||
&& cp /usr/local/zookeeper-3.4.13/conf/zoo_sample.cfg /usr/local/zookeeper-3.4.13/conf/zoo.cfg \
|
||||
&& ln -s /usr/local/zookeeper-3.4.13 /usr/local/zookeeper
|
||||
wget -q -O /tmp/zookeeper-3.4.14.tar.gz "https://apache.org/dist/zookeeper/zookeeper-3.4.14/zookeeper-3.4.14.tar.gz"
|
||||
tar -xzf /tmp/zookeeper-3.4.14.tar.gz -C /usr/local
|
||||
cp /usr/local/zookeeper-3.4.14/conf/zoo_sample.cfg /usr/local/zookeeper-3.4.14/conf/zoo.cfg
|
||||
ln -s /usr/local/zookeeper-3.4.14 /usr/local/zookeeper
|
||||
rm /tmp/zookeeper-3.4.14.tar.gz
|
||||
|
||||
# Kafka
|
||||
# Match the version to the Kafka client used by KafkaSupervisor
|
||||
wget -q -O - http://www.us.apache.org/dist/kafka/2.1.0/kafka_2.12-2.1.0.tgz | tar -xzf - -C /usr/local \
|
||||
&& ln -s /usr/local/kafka_2.12-2.1.0 /usr/local/kafka
|
||||
wget -q -O /tmp/kafka_2.12-2.1.1.tgz "https://apache.org/dist/kafka/2.1.1/kafka_2.12-2.1.1.tgz"
|
||||
tar -xzf /tmp/kafka_2.12-2.1.1.tgz -C /usr/local
|
||||
ln -s /usr/local/kafka_2.12-2.1.1 /usr/local/kafka
|
||||
rm /tmp/kafka_2.12-2.1.1.tgz
|
||||
|
||||
# Druid system user
|
||||
adduser --system --group --no-create-home druid \
|
||||
|
|
|
|||
6
pom.xml
6
pom.xml
|
|
@ -40,14 +40,14 @@
|
|||
<licenses>
|
||||
<license>
|
||||
<name>Apache License, Version 2.0</name>
|
||||
<url>http://www.apache.org/licenses/LICENSE-2.0</url>
|
||||
<url>https://www.apache.org/licenses/LICENSE-2.0</url>
|
||||
</license>
|
||||
</licenses>
|
||||
|
||||
<developers>
|
||||
<developer>
|
||||
<name>Druid Committers</name>
|
||||
<url>http://druid.io/community/index.html#committers</url>
|
||||
<name>Apache Druid Committers</name>
|
||||
<url>https://druid.apache.org/community/#committers</url>
|
||||
</developer>
|
||||
</developers>
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue