4b3bd8bd63
Migrating java-util from Metamarkets. ( #3585 )
...
* Migrating java-util from Metamarkets.
* checkstyle and updated license on java-util files.
* Removed unused imports from whole project.
* cherry pick metamx/java-util@826021f .
* Copyright changes on java-util pom, address review comments.
2016-10-21 14:57:07 -07:00
6f21778364
Support finding segments in AWS S3. ( #3399 )
...
* support finding segments from a AWS S3 storage.
* add more Uts
* address comments and add a document for the feature.
* update docs indentation
* update docs indentation
* address comments.
1. add a Ut for json ser/deser for the config object.
2. more informant error message in a Ut.
* address comments.
1. use @Min to validate the configuration object
2. change updateDescriptor to a string as it does not take an argument otherwise
* fix a Ut failure - delete a Ut for testing default max length.
2016-10-10 17:27:09 -07:00
49c0fe0e8b
Show candidate hosts for the given query ( #2282 )
...
* Show candidate hosts for the given query
* Added test cases & minor changes to address comments
* Changed path-param to query-pram for intervals/numCandidates
2016-09-22 08:32:38 -07:00
39351fb8d2
Mask properties from logging ( #3332 )
...
* Mask properties from logging
* mask "password" by default
2016-08-08 21:36:10 +05:30
47894c4eff
add comment for default hadoop coordinates ( #3257 )
...
1) Modify CliHadoopIndexer to share constant from `TaskConfig.DEFAULT_DEFAULT_HADOOP_COORDINATES`
2) add comment to pom.xml as discussed in
https://github.com/druid-io/druid/pull/3044
fix name
2016-07-18 15:23:11 -07:00
6cd1f5375b
Better harmonized dimensions for query metrics. ( #3245 )
...
All query metrics now start with toolChest.makeMetricBuilder, and all of
*those* now start with DruidMetrics.makePartialQueryTimeMetric. Also, "id"
moved to common code, since all query metrics added it anyway.
In particular this will add query-type specific dimensions like "threshold"
and "numDimensions" to servlet-originated metrics like query/time.
2016-07-14 11:55:51 -07:00
ea03906fcf
Configurable compressRunOnSerialization for Roaring bitmaps. ( #3228 )
...
Defaults to true, which is a change in behavior (this used to be false and unconfigurable).
2016-07-08 10:24:19 +05:30
b8a4f4ea7b
DumpSegment: Add --dump bitmaps option. ( #3221 )
...
Also make --dump metadata respect --column.
2016-07-06 12:42:50 -07:00
d2636d1a64
[pull-deps] If --clean flag is not set, skip creating root extension directories if they already exist. ( #3130 )
2016-07-01 11:18:57 -05:00
485e381387
remove datasource from hadoop output path ( #3196 )
...
fixes #2083 , follow-up to #1702
2016-06-29 08:53:45 -07:00
45f553fc28
Replace the deprecated usage of NoneShardSpec ( #3166 )
2016-06-25 10:27:25 -07:00
da660bb592
DumpSegment tool. ( #3182 )
...
Fixes #2723 .
2016-06-23 14:37:50 -07:00
e7ac2e8ba2
Set a non-zero exit code when error incommand parsing ( #3137 )
2016-06-13 20:38:37 -07:00
3b3e772748
Add --no-default-remote-repositories flag to pull-deps. ( #3120 )
2016-06-13 17:01:18 +05:30
15ccf451f9
Move QueryGranularity static fields to QueryGranularities ( #2980 )
...
* Move QueryGranularity static fields to QueryGranularityUtil
* Fixes #2979
* Add test showing #2979
* change name to QueryGranularities
2016-05-17 16:23:48 -07:00
eaaad01de7
[QTL] Datasource as lookupTier ( #2955 )
...
* Datasource as lookup tier
* Adds an option to let indexing service tasks pull their lookup tier from the datasource they are working for.
* Fix bad docs for lookups lookupTier
* Add Datasource name holder
* Move task and datasource to be pulled from Task file
* Make LookupModule pull from bound dataSource
* Fix test
* Fix code style on imports
* Fix formatting
* Make naming better
* Address code comments about naming
2016-05-17 15:44:42 -07:00
b489f63698
Supervisor for KafkaIndexTask ( #2656 )
...
* supervisor for kafka indexing tasks
* cr changes
2016-05-04 23:13:13 -07:00
c29cb7d711
add pending task based resource management strategy ( #2086 )
2016-04-27 10:40:53 -07:00
c868d374c0
Add missing modules for validating tasks (Fix for #2618 )
2016-03-30 14:39:20 +09:00
0872448ff0
make Coordinator IndexingService helpers pluggable
...
Fixes #2682
IndexingService helpers are added according to the settings in runtime.properties.
Rather than having all the config.isXXX checks there, it makes sense to have a pluggable
approach for allowing the dynamic configuration to bring in implementations for helpers
without having to have hard-coded sets of available helpers. Plus, it will also make it possible for extensions to plug helpers in.
With https://github.com/druid-io/druid-api/pull/76 , we could conditionally bind a helper to Coordinator's runlist.
The condition is driven by the value set in the runtime.properties.
2016-03-25 11:48:54 -05:00
7b93195dc6
do not inject properties directly in module
2016-03-22 14:30:10 -05:00
5da9a280b6
Query Time Lookup - Dynamic Configuration
2016-03-18 09:45:05 -07:00
a52c6d3bee
Fix some google related imports
2016-03-17 11:03:29 -07:00
cf342d8d3c
Merge pull request #2517 from b-slim/adding_lookup_snapshot_utility
...
[QTL][Lookup] lookup module with the snapshot utility
2016-03-17 11:39:47 -05:00
0c86b29ef0
lookup module with the snapshot utility
2016-03-17 09:20:41 -05:00
32399bc711
Merge pull request #2588 from metamx/abstract-zkworker
...
Use ImmutableWorkerInfo instead of ZKWorker
2016-03-15 10:27:21 -07:00
5ec5ac92c6
Merge pull request #2382 from himanshug/broker_segment_tier_selection
...
at broker, if configured, only add segments from specific tiers to the timeline
2016-03-14 16:53:06 -07:00
9cceff2274
Use ImmutableWorkerInfo instead of ZKWorker
...
review comments
add test for equals and hashcode
2016-03-14 11:17:15 -07:00
187569e702
DataSource metadata.
...
Geared towards supporting transactional inserts of new segments. This involves an
interface "DataSourceMetadata" that allows combining of partially specified metadata
(useful for partitioned ingestion).
DataSource metadata is stored in a new "dataSource" table.
2016-03-10 17:41:50 -08:00
ba1185963b
Fix a bunch of dependencies
...
* Eliminate exclusion groups from pull-deps
* Only consider dependency nodes in pull-deps if they are not in the following scopes
* provided
* test
* system
* Fix a bunch of `<scope>provided</scope>` missing tags
* Better exclusions for a couple of problematic libs
2016-03-10 10:18:08 -08:00
7557eb2800
CliPeon: Fix deadlock on startup by eagerly creating ExecutorLifecycle, ChatHandlerResource.
...
See stack traces here, from current master: https://gist.github.com/gianm/bd9a66c826995f97fc8f
1. The thread "qtp925672150-62" holds the lock on InternalInjectorCreator.class,
used by Scopes.SINGLETON, and wants the lock on "handlers" in Lifecycle.addMaybeStartHandler
called by DiscoveryModule.getServiceAnnouncer.
2. The main thread holds the lock on "handlers" in Lifecycle.addMaybeStartHandler, which it
took because it's trying to add the ExecutorLifecycle to the lifecycle. main is trying
to get the InternalInjectorCreator.class lock because it's running ExecutorLifecycle.start,
which does some Jackson deserialization, and Jackson needs that lock in order to inject
stuff into the Task it's deserializing.
This patch eagerly instantiates ChatHandlerResource (which I believe is what's trying to
create the ServiceAnnouncer in the qtp925672150-62 jetty thread) and the ExecutorLifecycle.
2016-03-02 10:53:42 -08:00
fb7eae34ed
Merge pull request #2249 from metamx/workerExpanded
...
Use Worker instead of ZkWorker whenever possible
2016-02-24 13:23:22 +05:30
ac13a5942a
Use Worker instead of ZkWorker whenver possible
...
* Moves last run task state information to Worker
* Makes WorkerTaskRunner a TaskRunner which has interfaces to help with getting information about a Worker
2016-02-23 15:02:03 -08:00
23c993c9e7
Add druid.indexer.server.maxChatRequests for QoS; deprecate separate ports.
...
- Add druid.indexer.server.maxChatRequests, which sets up a QoSFilter on the main Jetty server.
- Deprecate druid.indexer.runner.separateIngestionEndpoint
- Deprecate druid.indexer.server.chathandler.*
2016-02-19 13:36:09 -08:00
e0c049c0b0
Make startup properties logging optional.
...
Off by default, but enabled in the example config files. See also #2452 .
2016-02-12 14:12:16 -08:00
368988d187
Merge pull request #2291 from druid-io/lookupManager
...
Promoting LookupExtractor state and LookupExtractorFactory to be a first class druid state object.
2016-02-11 16:07:27 -06:00
4e119b7a24
Adding lookup ref manager and lookup dimension spec impl
2016-02-11 12:11:51 -06:00
40ade32a1f
Fix dependencies.
...
* Don't put druid****selfcontained.jar at the end of the hadoop isolated classpath
* Add `<scope>provided</scope>` to prevent repeated dependency inclusion in the extension directories
2016-02-11 07:30:14 -08:00
2bde8b1d68
Make hadoop classpath isolation more explicit
...
* Fixes #2428
2016-02-10 12:09:17 -08:00
d1cb17d3f7
at broker - only add segments from specific tiers to the timeline
2016-02-09 22:33:22 -06:00
1aa363cea7
new quickstart
2016-02-04 09:37:38 -08:00
ffbe923fb5
remove convertProperties
...
fixes https://github.com/druid-io/druid/issues/2335
2016-02-02 23:43:50 +05:30
eddc18ecf2
Add basic tests to some CliXXX classes
2016-01-20 16:19:15 -08:00
ac6c90e657
Merge pull request #1953 from metamx/taskRunnerResourceManagement
...
Move resource managemnt to be the responsibility of the TaskRunner
2016-01-20 22:27:47 +05:30
fe841fd961
Merge pull request #2118 from guobingkun/fix_segment_loading
...
Fix loading segment for historical
2016-01-19 14:25:48 -06:00
c4ad50f92c
Fix loading segment for historical
...
Historical will drop a segment that shouldn't be dropped in the following scenario:
Historical node tried to load segmentA, but failed with SegmentLoadingException,
then ZkCoordinator called removeSegment(segmentA, blah) to schedule a runnable that would drop segmentA by deleting its files. Now, before that runnable executed, another LOAD request was sent to this historical, this time historical actually succeeded on loading segmentA and announced it. But later on, the scheduled drop-of-segment runnable started executing and removed the segment files, while historical is still announcing segmentA.
2016-01-19 10:29:49 -06:00
e4d15f3729
Merge pull request #2133 from navis/log4j-shudown-callback
...
callbacks registered to Log4jShutdown is not executed when stop is called
2016-01-15 21:28:46 +09:00
976d4c965b
Move resource managemnt to be the responsibility of the TaskRunner
2016-01-13 10:38:22 -08:00
8d1686039a
Merge pull request #1830 from metamx/peonDaemonChecker
...
Check for non-daemon threads when CliPeon exits
2016-01-11 10:48:52 -08:00
443ce2db9d
callbacks registered to Log4jShutdown is not executed when stop is called
2016-01-05 08:37:54 +09:00
43e7523474
Merge pull request #2189 from pjain1/fix_realtime_example
...
add ChatHandlerServerModule to realtime example
2016-01-04 07:57:54 -08:00
807cd6fa83
add ChatHandlerServerModule to realtime example
2015-12-22 05:40:54 -06:00
89b477970f
DataSegmentFinder tool
...
`insert-segment-to-db` is a tool that can insert segments into Druid metadata storage. It is intended to be used
to update the segment table in metadata storage after people manually migrate segments from one place to another.
It can also be used to insert missing segment into Druid, or even recover metadata storage by telling it where the
segments are stored.
Note: This tool expects users to have Druid cluster running in a "safe" mode, where there are no active tasks to interfere
the segments being inserted. Users can optionally bring down the cluster to make 100% sure nothing is interfering.
2015-12-21 00:02:04 -06:00
d95adb5749
Merge pull request #1949 from metamx/overlord-console-redirect
...
redirect to overlord console by default
2015-12-10 15:11:00 -08:00
dcd15735cd
Merge pull request #2015 from metamx/handoff-notifier-coordinator
...
Remove ServerView from RealtimeIndexTasks and use coordinator http endpoint for handoff information
2015-12-08 14:06:39 -08:00
9491e8de3b
Remove ServerView from RealtimeIndexTasks and use coordinator http endpoint for handoffs
...
- fixes #1970
- extracted out segment handoff callbacks in SegmentHandoffNotifier
which is responsible for tracking segment handoffs and doing callbacks
when handoff is complete.
- Coordinator now maintains a view of segments in the cluster, this
will affect the jam heap requirements for the overlord for large
clusters.
realtime index task and nodes now use HTTP end points exposed by the
coordinator to get serverView
review comment
fix realtime node guide injection
review comments
make test not rely on scheduled exec
fix compilation
fix import
review comment
introduce immutableSegmentLoadInfo
fix son reading
remove unnecessary logging
2015-12-09 01:54:09 +05:30
cacd8f7a27
Merge pull request #2028 from navis/parse-spec-validator
...
fix for #1939 , Druid ParseSpec validator
2015-12-07 11:53:01 -08:00
62ba9ade37
unifying license header in all java files
2015-12-05 22:16:23 -06:00
4a70d8e775
ParseSpecValidator
2015-12-04 11:08:02 +09:00
21c84b5ff7
Merge pull request #1896 from gianm/allocate-segment
...
SegmentAllocateAction (fixes #1515 )
2015-11-18 21:05:46 -08:00
8fcf2403e3
Merge pull request #1943 from metamx/realtime-caching
...
Enable caching on intermediate realtime persists
2015-11-17 15:06:43 -08:00
dbe201aeed
Merge pull request #1929 from pjain1/jetty_threads
...
separate ingestion and query thread pool
2015-11-17 12:14:25 -08:00
6c498b7d4a
separate ingestion and query thread pool
2015-11-17 13:42:41 -06:00
d7eb2f717e
enable query caching on intermediate realtime persists
2015-11-17 10:58:00 -08:00
e4e5f0375b
SegmentAllocateAction ( fixes #1515 )
...
This is a feature meant to allow realtime tasks to work without being told upfront
what shardSpec they should use (so we can potentially publish a variable number
of segments per interval).
The idea is that there is a "pendingSegments" table in the metadata store that
tracks allocated segments. Each one has a segment id (the same segment id we know
and love) and is also part of a sequence.
The sequences are an idea from @cheddar that offers a way of doing replication.
If there are N tasks reading exactly the same data with exactly the same logic
(think Kafka tasks reading a fixed range of offsets) then you can place them
in the same sequence, and they will generate the same sequence of segments.
2015-11-11 16:54:35 -08:00
fa6142e217
cleanup and remove unused imports
2015-11-11 12:25:21 -08:00
b519fcf99b
redirect to overlord console by default
2015-11-11 09:55:18 -08:00
a57cbfd2c3
Merge pull request #1387 from metamx/enableShutdownLogging
...
Add special handler to allow logger messages during shutdown
2015-11-09 17:20:09 -08:00
8f231fd3e3
cleanup druid codebase
2015-11-04 13:59:53 -08:00
8defe29270
Merge pull request #1901 from guobingkun/fix_typo_and_rename
...
Fix metadata typo and rename default extension directory
2015-11-03 14:02:11 -08:00
3b3d88bd03
Merge pull request #1857 from noddi/feature/intervals-route
...
Queryable intervals
2015-11-03 13:19:04 -08:00
962f65cc76
fix metadata typo and rename default extension directory
2015-11-03 14:50:42 -06:00
84f7d8d264
making static final variables in HadoopDruidIndexerConfig upper case
2015-11-02 23:24:26 -06:00
657a5ac346
fix pull-deps remoteRepository option
2015-10-30 11:32:56 -05:00
6de51e022d
Coordinator - add intervals route.
2015-10-30 11:51:42 +01:00
194f632b00
Check for non-daemon threads when CliPeon exits
2015-10-29 11:56:38 -07:00
7a2ceef690
Add special handler to allow logger messages during shutdown
...
* Adds a special PropertyChecker interface which is ONLY for setting string properties at the very start of psvm
2015-10-27 14:33:36 -07:00
4914925d65
New extension loading mechanism
...
1) Remove maven client from downloading extensions at runtime.
2) Provide a way to load Druid extensions and hadoop dependencies through file system.
3) Refactor pull-deps so that it can download extensions into extension directories.
4) Add documents on how to use this new extension loading mechanism.
5) Change the way how Druid tarball is generated. Now all the extensions + hadoop-client 2.3.0
are packaged within the Druid tarball.
2015-10-21 14:22:36 -05:00
5db4c5a089
move cache bindings into a dedicated module
2015-10-02 00:29:05 -04:00
940e1aa3eb
Replace funky imports with standard ones.
...
1) Lots of Guava imports were not coming from the actual Guava
2) junit.framework.Assert should be org.junit.Assert
2015-08-28 18:02:05 -07:00
9a0c15c52c
update airline to 0.7
2015-08-25 16:07:00 -07:00
2093187c91
rework tarball distribution:
...
- move assembly out of druid-services into a 'distribution' module
- create separate 'extensions-distribution' module and assembly to
package extensions and their dependencies into a local maven
repository
- include this extensions maven repository in the binaries tarball
2015-08-18 18:32:33 -07:00
15fa43dd43
changing DatasourcePathSpec, to get segment list, so that hadoop indexer uses overlord action to get list of segments and passes when running as an overlord task. and, uses metadata store directly when running as standalone hadoop indexer
...
also, serialized list of segments is passed to DatasourcePathSpec so that hadoop classloader issues do not creep up
2015-08-16 14:07:35 -05:00
db19d2d547
Revert "Update to guice 4.0"
2015-08-14 09:26:07 -07:00
08f50fcc37
explicitly call lifecycle stop
2015-08-14 17:35:17 +05:30
7e61216287
Update to guice 4.0
...
- Mark a lot of `@Provides` methods as final since guice 4.0 disallows overriding them
2015-08-10 13:57:18 -07:00
9dd18de1a5
Merge pull request #1596 from metamx/console-artifact
...
make console a maven dependency instead of filedump
2015-08-04 15:28:45 -07:00
07ee1acca2
make console a maven dependency instead of filedump
2015-08-04 09:49:34 -07:00
ba59f8afc4
Merge pull request #1550 from himanshug/optionally_log_all_requests
...
print *all* HTTP requests to log if configured
2015-07-28 11:19:36 -07:00
90b475975e
Log all the requests to debug level in logger io.druid.jetty.RequestLog
2015-07-28 12:56:04 -05:00
4801de62a2
make "announce" the chathandler default in realtime node,
...
remove doc references to chathandler type "announce" since it is the default now,
2015-07-27 12:14:28 -05:00
e21195f987
Merge pull request #1469 from guobingkun/table_config
...
Inconsistent property names for "druid.metadata.storage.tables.xxx"
2015-07-17 07:43:19 -07:00
2b87ad307d
Make CliPeon run shutdown hooks properly
...
* Also make jetty threads daemon
2015-07-14 15:15:26 -07:00
4a0ae7d8d5
Fix inconsistent druid property names for "druid.metadata.storage.tables.xxx" between document and code
2015-06-29 10:12:30 -05:00
4a3a7dd1ad
read hadoop-indexer configuration file from HDFS
2015-06-24 14:08:53 -07:00
056cab93ed
Add Hadoop Converter Job and task
...
* Fixes https://github.com/druid-io/druid/issues/1363
* Add extra utils in JobHelper based on PR feedback
2015-06-09 14:47:38 -07:00
aa10626b61
adding druid.db.tables.xxx renames and druid.db.connectory.*validation* from ConvertProperties
2015-05-03 08:41:15 -05:00
8722f4cc18
UTs for ConvertProperties
2015-05-03 08:40:17 -05:00
40121bc800
Merge pull request #1324 from metamx/fixMasterPropertiesTest
...
Partial revert of 467fcfae03
2015-04-29 22:15:21 -07:00
de16eb34af
Partial revert of 467fcfae03
2015-04-29 17:20:51 -07:00
bfdb5dcb5d
Merge pull request #1317 from metamx/overlord-endpoints
...
proxy /druid/indexer requests on the coordinator to the indexing overlord
2015-04-29 16:07:42 -07:00
e055a7e869
Add options to the properties converter to update from 0.6.x to 0.7.x
...
* Removed 0.5 --> 0.6 converter
2015-04-29 15:46:16 -07:00
6e84caf3d5
Proxy Requests from coordinator to overlord
...
Add unit test and support https scheme
review comments
2015-04-29 20:58:26 +05:30
b131a4fd3b
adding router numMaxThread configuration for ProxyServlet HttpClient executor
2015-03-23 21:58:01 -05:00
c847b3b861
Merge pull request #1218 from himanshug/task_storage_config_fix
...
add correct guice init of TaskStorageConfig to CliPeon
2015-03-17 13:08:43 -07:00
4a8f5dfa0f
add correct guice init of TaskStorageConfig to CliPeon
2015-03-17 14:03:54 -05:00
3ec4a30ab5
initial commit
...
review comments
more refactoring and cleaning of redundant code
add UT + docs + more refactoring
fixes + review comments
more cleanup
end points to fetch history
review comments
remove unnecessary changes
review comments rename header name
review comments + add test for MetadataRulesManager
review comments docs
2015-03-12 22:50:29 +05:30
956361dbbd
Refactoring: Moved BaseJettyServerInitializer.java to JettyServerInitUtils.java and changed it from abstract class to a utils class containing static methods
...
And, moved jetty stuff to its own package io.druid.server.initialization.jetty
2015-03-05 19:33:27 -06:00
a8648d8f3d
feature to add servlet filters in a druid node via extension modules
2015-03-03 22:35:02 -06:00
17743b94f3
Cleanup router servlet + tests for gzip proxying
...
- Uses method overrides instead of modified Jetty code, now that
ProxyServlet provides enough method hooks for proper overrides.
This means we may also benefit from any Jetty ProxyServlet fixes
- Adds test for async proxy servlet to make sure gzip encoding is
properly proxied.
- Router now proxies POST requests for requests that are not Druid
queries, by only treating /druid/v2/* endpoints as queries.
2015-03-02 11:36:15 -08:00
467fcfae03
UTs for services
2015-02-25 15:45:59 -08:00
42eba986ce
Towards consistent null handling
...
This commit also includes
1) the addition of a context parameter on timeseries queries that allows it to ignore empty buckets instead of generating results for them
2) A cleanup of an unused method on an interface
2015-02-02 12:53:07 -08:00
d05032b98a
towards a community led druid
2015-01-31 20:57:36 -08:00
2c2771b90e
Make dynamic worker selection actually work
2015-01-27 14:17:42 -08:00
852e863425
Merge pull request #981 from druid-io/strictModuleTyping
...
Use Module instead of generic Object in Guice related items
2015-01-05 12:43:20 -08:00
65286a24e0
Change zk configs to use Jackson injection instead of Skife
...
* Also added generic config testing class JsonConfigTesterBase
2014-12-29 10:36:12 -08:00
7c8d4a7433
Use Module instead of generic Object in Guice related items
2014-12-19 10:54:06 -08:00
c0af7b2a5d
fix broken ut
2014-12-15 18:29:15 -08:00
d1fcd4e92e
Add log4j in services
2014-12-15 17:39:19 -08:00
4f0c0c5ae2
i hate guice
2014-12-15 16:04:19 -08:00
ac407fb6ba
clean up defaults
2014-12-15 15:05:02 -08:00
a520835972
Merge pull request #957 from metamx/fix-gzip-encoding
...
fix gzip compression
2014-12-15 13:20:36 -08:00
56dfe5d883
Merge branch 'master' of github.com:metamx/druid
2014-12-15 09:42:08 -08:00
88858ea1c3
make default handler service announcing
2014-12-15 09:42:03 -08:00
4b5282d224
add loadstatus endpoint for serverView status
2014-12-12 13:18:31 -08:00
0de56efe5c
fix gzip compression
...
- fixes compression not working for POST requests
- router now forwards raw bytes instead of decompressing
- cleanded up router servlet initialization
- add test for gzip on get and post methods
- use Jersey annotation when possible in QueryResource
2014-12-10 17:30:17 -08:00
5f3f4e0a60
prevent /status from redirect, fixes #953
2014-12-10 11:32:22 -08:00
332e4a4d1c
Merge pull request #902 from metamx/cache-config
...
Commonalize the cache config and change default to disable cache
2014-12-04 11:30:24 -07:00
d388a8fe89
Replace google-http-client imports with real guava imports.
2014-12-03 10:52:57 -08:00
d23fd1e1ab
make host+port more explicit
...
- document the behavior for node host/port initialization
- throw exception if settings make no sense
- fixes announcement for nodes without host/port defaults
- makes code clearer as to when host vs. host+port are used
2014-11-26 22:03:25 -08:00
3ff569ef2d
Merge pull request #879 from metamx/rtr-with-pref
...
Rewrite autoscaling and enable easier configuration of worker selection and autoscaling behaviour
2014-11-24 17:54:28 -07:00
083a012aa2
Commonalize the cache config and change default to disable cache
2014-11-24 14:54:11 -08:00
28c609b187
fix some brokenness
2014-11-24 13:31:56 -08:00
1aaea9a0d7
address code review
2014-11-24 10:52:30 -08:00
9da66291e1
change naming to common config
2014-11-21 15:36:42 -08:00
590d31799e
Merge pull request #876 from metamx/remove-backwards-compatible
...
Remove backwards compatible
2014-11-19 14:33:14 -07:00
64719b15e0
rewrite autoscaling with tests
2014-11-18 15:41:06 -08:00
c91310914b
fix a few naming things
2014-11-17 16:05:18 -08:00
32600e10bb
address code review
2014-11-17 15:55:22 -08:00
1af6b337f2
optionally choose what worker to send tasks to
2014-11-17 14:50:56 -08:00
d914afe1cd
make defaultVersion configurable for non-jar testing
2014-11-17 13:54:32 -08:00
0c2d06475d
merge from master
2014-11-17 19:19:18 +05:30
10b7ca9fa9
Merge pull request #849 from metamx/default-coordinator-service-name
...
Default service names
2014-11-12 15:06:02 -08:00
3a9def7fb1
add broker default and define constants
2014-11-12 14:53:53 -08:00
59f093882d
fix Derby instantiation
2014-11-10 16:55:53 -08:00
98d05e63bf
make derby instance only on coordinator
2014-11-10 15:34:49 -08:00
f956fdad2e
fix injection once more
2014-11-07 14:26:03 -08:00
0498df25df
override metadata storage injection in CliHadoopIndexer
2014-11-07 13:44:56 -08:00
70f49538dd
i swear injection is fixed now
2014-11-07 12:04:09 -08:00
aeb194ad12
fix injection for real
2014-11-07 11:43:53 -08:00
2336e6c167
Merge pull request #758 from metamx/jisoo-metadata
...
make metadata storage pluggable
2014-11-07 11:30:11 -07:00
d8bab70647
provide proper defaults for indexing service name
2014-11-06 18:16:01 -08:00
4b6438537e
fix missing service name/port bindings for peons
2014-11-06 11:57:08 -08:00
3676ec2d89
fix missing service name/port bindings for peons
2014-11-03 21:11:49 -08:00
f56d026d7d
fix realtime module + cleaner metadata abstraction
...
- better metadata abstractions
- more db->metadata renaming
2014-11-03 16:40:23 -08:00
9c06db021f
rename db->metadata postgres->postgresql
2014-10-31 10:30:27 -07:00
df0af894f4
fix factory binding + remove unused imports
2014-10-30 22:35:37 -07:00
fb4d41cedb
make the injection gods happy
2014-10-30 21:16:36 -07:00
377151beda
better abstraction for metadatastorage
2014-10-30 18:23:35 -07:00
b7edda2b33
add back code that got lost in translation
2014-10-29 12:54:04 -07:00
aa754b86e8
build success!
2014-10-24 11:28:42 -07:00
41ebdd2bae
Merge branch 'druid-0.7.x' into fix-test
...
Conflicts:
processing/pom.xml
processing/src/main/java/io/druid/segment/IndexMerger.java
processing/src/main/java/io/druid/segment/MetricHolder.java
processing/src/main/java/io/druid/segment/data/CompressedObjectStrategy.java
processing/src/test/java/io/druid/query/groupby/GroupByQueryRunnerTest.java
processing/src/test/java/io/druid/segment/SchemalessTestFull.java
server/src/test/java/io/druid/client/CachingClusteredClientTest.java
services/src/main/java/io/druid/cli/CliHistorical.java
2014-10-24 11:07:52 -07:00
bef74104d9
merge with 0.7.x and resolve any conflicts
2014-10-23 17:24:06 -07:00
37979282fe
enabled ansi-quote in mysql; insert statement should now work
2014-10-21 00:09:19 -07:00
7d5c5f2083
fixed createTable; fixed miscellaneous stuff; added DerbyMetadataRuleManagerProvider
2014-10-17 00:10:36 -07:00
035b4c1af4
more cleanup
2014-10-14 00:57:15 +05:30
454acd3f5a
remove backwards compatible code
...
1) remove backwards compatible and deprecated code
2) make hashed partitions spec default
2014-10-13 19:30:44 +05:30
66277ddf2b
deleted dbSegmentPublisher
2014-10-10 12:41:35 -07:00
d4217f1588
Merge pull request #777 from metamx/initialized-endpoint
...
Historical endpoints accessible while loading + new loadstatus endpoint
2014-10-06 17:35:58 -06:00
81641d7887
http accessible while loading + status endpoint
2014-10-06 13:32:27 -07:00
c7b4d5b7b4
Merge branch 'master' into druid-0.7.x
...
Conflicts:
processing/src/test/java/io/druid/segment/filter/SpatialFilterTest.java
2014-10-02 18:12:10 -07:00
0e50852985
fixed MetadataTaskStorage and handler
2014-09-30 14:09:23 -07:00
4a09678739
make the selection strategy in rtr extendable
2014-09-29 14:24:02 -07:00
74565c9371
cleaned up the code
2014-09-27 13:10:01 -07:00
6a641621b2
finished merging into druid-0.7.x; derby not working (to be fixed)
2014-09-26 14:24:53 -07:00
43cc6283d3
trying to revert files that have overwritten changes
2014-09-26 12:38:04 -07:00
4244accab5
Merge branch 'druid-0.7.x' of github.com:metamx/druid into druid-0.7.x
2014-09-26 11:39:50 -07:00
eaf0a48b92
Merge branch 'master' into druid-0.7.x
...
Conflicts:
cassandra-storage/pom.xml
common/pom.xml
examples/pom.xml
hdfs-storage/pom.xml
histogram/pom.xml
indexing-hadoop/pom.xml
indexing-service/pom.xml
kafka-eight/pom.xml
kafka-seven/pom.xml
pom.xml
processing/pom.xml
processing/src/main/java/io/druid/guice/PropertiesModule.java
rabbitmq/pom.xml
s3-extensions/pom.xml
server/pom.xml
services/pom.xml
2014-09-26 11:39:24 -07:00
3bf39cc9f8
attempted to fix merge-conflicts
2014-09-24 15:55:42 -07:00
273205f217
initial attempt for abstraction; druid cluster works with Derby as a default
2014-09-19 17:39:59 -07:00
f1e3518f37
initialize segment view before announcing broker
2014-09-16 00:07:31 -07:00
0209b7e440
fix bridge port
2014-09-15 15:57:08 -07:00
469ccbbe5e
Merge branch 'master' into druid-0.7.x
...
Conflicts:
cassandra-storage/pom.xml
common/pom.xml
examples/pom.xml
hdfs-storage/pom.xml
histogram/pom.xml
indexing-hadoop/pom.xml
indexing-service/pom.xml
kafka-eight/pom.xml
kafka-seven/pom.xml
pom.xml
processing/pom.xml
processing/src/main/java/io/druid/query/FinalizeResultsQueryRunner.java
processing/src/main/java/io/druid/query/UnionQueryRunner.java
processing/src/main/java/io/druid/query/groupby/GroupByQueryRunnerFactory.java
processing/src/main/java/io/druid/query/topn/TopNQueryEngine.java
processing/src/main/java/io/druid/query/topn/TopNQueryRunnerFactory.java
rabbitmq/pom.xml
s3-extensions/pom.xml
server/pom.xml
server/src/test/java/io/druid/server/initialization/JettyTest.java
services/pom.xml
2014-09-11 16:20:50 -07:00
508e982190
Merge remote-tracking branch 'origin/master' into druid-0.7.x
...
Conflicts:
cassandra-storage/pom.xml
common/pom.xml
examples/config/historical/runtime.properties
examples/config/overlord/runtime.properties
examples/config/realtime/runtime.properties
examples/pom.xml
hdfs-storage/pom.xml
histogram/pom.xml
indexing-hadoop/pom.xml
indexing-service/pom.xml
kafka-eight/pom.xml
kafka-seven/pom.xml
pom.xml
processing/pom.xml
rabbitmq/pom.xml
s3-extensions/pom.xml
server/pom.xml
server/src/main/java/io/druid/server/ClientQuerySegmentWalker.java
services/pom.xml
2014-08-30 22:42:36 -07:00
58ab759fc6
remove unused imports
2014-08-29 14:03:47 -07:00
5888275206
Repeat less repeatedly.
2014-08-26 13:39:09 -07:00
890ba5a4ca
Better usage messages for CliHadoopIndexer.
2014-08-26 13:29:11 -07:00
953c855410
CliHadoopIndexer: Modify maven arguments to be consistent with pull-deps.
2014-08-26 12:04:59 -07:00
1fd30ab588
default service/host/port for all nodes
2014-08-15 17:14:05 -07:00
17a3e65dc4
add missing tables
2014-08-15 12:50:30 -07:00
6f0e768d82
add tools comand to create metadata storage tables
2014-08-14 17:13:55 -07:00
b740987381
Merge pull request #616 from metamx/jisoo
...
add retry feature on broker side; modify QueryRunner inferface and tests
2014-08-11 15:36:16 -06:00
d082718ed2
fix assembly file
2014-08-11 14:28:44 -07:00
08c7aabb34
fix deadlock condition that can exist in the coordinator
2014-08-07 15:55:25 -07:00
6f58cd1156
fix merge conflcts; change test files
2014-08-04 17:45:56 -07:00
7803ca07e7
fix broken chat handler
2014-07-29 17:58:21 -07:00
89149b5fcb
prepare for next release
2014-07-29 17:38:19 -07:00
05a09d2550
allow overriding coordinator static resource
2014-07-29 15:13:57 -07:00
2ad1bd3f44
add back metrics
2014-07-29 10:22:55 -07:00
088c2386dc
rewrite router async logic
2014-07-28 16:32:15 -07:00
2d4f1889e8
completely async request proxying + jetty update
2014-07-25 16:22:43 -07:00
bb5c5b3ce7
fix config with broker select and add docs
2014-07-25 10:50:41 -07:00
fec788f33f
Merge branch 'master' into smarter-router
...
Conflicts:
services/src/main/java/io/druid/cli/RouterJettyServerInitializer.java
2014-07-22 17:06:45 -07:00
e76561e400
use async gzip filter for async servlet
2014-07-22 16:55:43 -07:00
00f086665c
default async timeout to server idle timeout
2014-07-22 16:55:21 -07:00
f847a46ccf
fix typo
2014-07-22 15:53:50 -07:00
8f2fc595ee
allow router to override selection based on config
2014-07-22 10:12:02 -07:00
bf71d079f3
fix #636
2014-07-16 15:20:20 -07:00
534acc9eab
fix mm
2014-07-15 22:50:38 -07:00
935b642b2c
fix overlord binding problems
2014-07-15 16:13:11 -07:00
c4397bd65a
fixed merge conflicts
2014-07-07 11:26:11 -07:00
4c40e71e54
address cr
2014-06-19 14:48:46 -07:00
a870fe5cbe
inject column config
2014-06-19 14:47:57 -07:00
00856f0fec
add retry feature on broker side; modify QueryRunner inferface and tests
2014-06-18 16:55:33 -07:00
0bc1915067
Merge pull request #578 from metamx/new-guava
...
Update guava, java-util, and druid-api
2014-06-18 14:23:32 -06:00
5227b43050
fix more test breakage
2014-06-17 10:35:01 -07:00
a63cda3281
Merge branch 'master' into new-guava
...
Conflicts:
server/src/main/java/io/druid/server/QueryResource.java
2014-06-13 10:08:10 -07:00
c506d014bf
fix link to docs
2014-06-04 12:55:48 -07:00
bb4105ed1a
fix broken standalone hadoop ingestion
2014-06-04 09:23:46 -07:00
c97768b27d
Merge branch 'master' of github.com:metamx/druid into new-guava
2014-06-04 09:12:59 -07:00
d8338fc51d
fix backwards compatibility
2014-06-04 16:40:06 +05:30
77ec4df797
update guava, java-util, and druid-api
2014-06-03 13:43:38 -07:00
49a8dc9e3e
fix backwards incompatiable change with broker.balancer config
2014-05-28 13:27:43 -07:00
b6fb4245aa
Merge branch 'master' into new-schema
...
Conflicts:
indexing-hadoop/src/main/java/io/druid/indexer/HadoopDriverConfig.java
indexing-hadoop/src/main/java/io/druid/indexer/HadoopDruidIndexerConfig.java
indexing-hadoop/src/main/java/io/druid/indexer/HadoopDruidIndexerConfigBuilder.java
pom.xml
server/src/main/java/io/druid/segment/realtime/RealtimeManager.java
server/src/main/java/io/druid/segment/realtime/firehose/EventReceiverFirehoseFactory.java
2014-05-06 14:32:51 -07:00
544c0f08f6
Merge pull request #459 from metamx/move-firehose
...
Move the event receiver firehose such that standalone realtime nodes can use it
2014-05-05 17:09:43 -06:00
e90eb4b83a
add custom tier selector
2014-04-28 15:31:15 -07:00
9de170540b
fix bug with selection in cli
2014-04-28 14:00:53 -07:00
3c3daf452e
Smarter server selection at the router level
2014-04-28 13:58:10 -07:00
171d20d52d
Merge branch 'move-firehose' of github.com:metamx/druid into move-firehose
2014-04-25 14:13:19 -07:00
eef034ca7e
Merge branch 'master' of github.com:metamx/druid into move-firehose
2014-04-25 14:13:08 -07:00
76e0a48527
Merge branch 'master' into new-schema
...
Conflicts:
indexing-hadoop/src/main/java/io/druid/indexer/DbUpdaterJob.java
indexing-hadoop/src/test/java/io/druid/indexer/HadoopDruidIndexerConfigTest.java
indexing-service/src/main/java/io/druid/indexing/common/task/HadoopIndexTask.java
server/src/main/java/io/druid/segment/realtime/plumber/RealtimePlumber.java
server/src/main/java/io/druid/segment/realtime/plumber/RealtimePlumberSchool.java
2014-04-25 14:03:28 -07:00
8d650ae131
PullDependencies changes.
...
- Pull default Hadoop version by default.
- Allow additional coordinates on the command line.
2014-04-22 14:33:53 -07:00
c00fb1d08e
Merge pull request #445 from metamx/hadoop-version-update
...
Hadoop version update
2014-04-17 19:16:44 -06:00
a87213e15f
point code docs to latest instead of a version
2014-04-16 10:50:39 -07:00
51e2da9d8d
use a separate annotation for the router, less confusing
2014-04-16 10:08:58 -07:00
24a0268036
cleanup code in router and add forwarding for get requests
2014-04-16 09:36:18 -07:00
508cf6a4f7
Merge branch 'master' of github.com:metamx/druid
2014-04-15 14:23:58 -07:00
9b06b6aa58
misc fixes for router
2014-04-15 14:23:54 -07:00
077dd7c589
enable cache monitor on computes by default
2014-04-14 16:11:12 -07:00
88c592ccaa
add more meaningful request logging
2014-04-10 10:37:25 -07:00
ea8b2dffd5
Add chatHandlerResource
2014-04-09 14:25:18 +05:30
7db756f819
prepare for next release
2014-04-08 16:25:59 -07:00
a4df23f092
prepare for next release
2014-04-08 11:42:59 -07:00
c6f4b344cb
add bindings for chat handler provider
2014-04-08 11:56:23 +05:30
4ef2bbbfc1
prepare for next release
2014-04-07 14:37:02 -07:00
2e9579a146
prepare for next release
2014-04-07 12:43:30 -07:00
1843316db6
commonalize event receiver firehose
2014-04-03 20:46:36 -07:00
5d8d506fe8
fix commented out filters
2014-04-03 17:06:28 -07:00
d9062d91fa
fix endpoint bugs and more docs
2014-04-03 17:01:33 -07:00
b0bf08554a
fix broken build
2014-04-02 22:22:42 -07:00
Akash Dwivedi
jaehong choi
Navis Ryu
Nishant
Gian Merlino
Bingkun Guo
Xavier Léauté
Hyukjin Kwon
Charles Allen
David Lim
Bingkun Guo
Parag Jain
Slim
Slim Bouguerra
Jonathan Wei
Himanshu Gupta
fjy
Xavier Léauté
Bartosz Ługowski
sahner
Davide Anastasia
Himanshu Gupta
Eric Tschetter
jisookim0513
jisookim0513
jisookim0513
fjy
jisookim0513
jisookim0513