84cc32ba32
Add more logging for when ingestion may be throttled
2015-01-08 14:51:46 -08:00
c0af7b2a5d
fix broken ut
2014-12-15 18:29:15 -08:00
1a1b0e6f23
merge from master and review comments
2014-12-09 13:16:45 +05:30
a79389a9e5
update joda-time and fix min/max instant
2014-12-02 17:27:22 -08:00
cbffe3c648
merge from master and resolve conflicts
2014-11-17 18:07:08 +05:30
d19b895fb9
add tests for empty sequences
2014-11-13 14:49:13 -08:00
788a4a846b
add encoding
2014-11-11 17:31:08 -08:00
a89b539b4f
Merge pull request #823 from metamx/roaring
...
Configurable bitmap indexes: roaring and concise
2014-11-11 17:26:38 -08:00
1cc162727b
address code review
2014-11-11 14:05:37 -08:00
59f093882d
fix Derby instantiation
2014-11-10 16:55:53 -08:00
8cbd21af90
fix defaults
2014-11-10 15:40:57 -08:00
98d05e63bf
make derby instance only on coordinator
2014-11-10 15:34:49 -08:00
8bebb24fd5
Union Queries move merge to historical
2014-11-07 18:05:16 +05:30
5bda4ee1dd
global task entry type
2014-11-06 17:08:20 -08:00
ac753ed0f0
always use validation query
2014-11-06 17:08:01 -08:00
10aa8a2d56
add explicit type names
2014-11-06 17:07:24 -08:00
350bb09605
refactor sql storage to abstract task storage
2014-11-05 17:19:37 -08:00
253c0a70b3
simplify interface + more tests
2014-11-04 15:03:13 -08:00
696c9ba944
fix formatting
2014-10-31 13:32:18 -07:00
a6016d70fc
added deleted classes back
2014-10-31 13:10:09 -07:00
9c06db021f
rename db->metadata postgres->postgresql
2014-10-31 10:30:27 -07:00
fb4d41cedb
make the injection gods happy
2014-10-30 21:16:36 -07:00
377151beda
better abstraction for metadatastorage
2014-10-30 18:23:35 -07:00
37979282fe
enabled ansi-quote in mysql; insert statement should now work
2014-10-21 00:09:19 -07:00
7d5c5f2083
fixed createTable; fixed miscellaneous stuff; added DerbyMetadataRuleManagerProvider
2014-10-17 00:10:36 -07:00
c7b4d5b7b4
Merge branch 'master' into druid-0.7.x
...
Conflicts:
processing/src/test/java/io/druid/segment/filter/SpatialFilterTest.java
2014-10-02 18:12:10 -07:00
0e50852985
fixed MetadataTaskStorage and handler
2014-09-30 14:09:23 -07:00
9d7b5d9b0f
fixed javadoc; fixed pom files; deleted unnecessary class
2014-09-30 13:47:35 -07:00
91eecfe3e7
fix #730 - memory check throws exception if fails
...
fix exception handling, do not catch ProvisionException
2014-09-30 23:03:40 +05:30
74565c9371
cleaned up the code
2014-09-27 13:10:01 -07:00
aa887edb73
added two seperate modules for mysql and postgres
2014-09-27 13:08:53 -07:00
6a641621b2
finished merging into druid-0.7.x; derby not working (to be fixed)
2014-09-26 14:24:53 -07:00
43cc6283d3
trying to revert files that have overwritten changes
2014-09-26 12:38:04 -07:00
eaf0a48b92
Merge branch 'master' into druid-0.7.x
...
Conflicts:
cassandra-storage/pom.xml
common/pom.xml
examples/pom.xml
hdfs-storage/pom.xml
histogram/pom.xml
indexing-hadoop/pom.xml
indexing-service/pom.xml
kafka-eight/pom.xml
kafka-seven/pom.xml
pom.xml
processing/pom.xml
processing/src/main/java/io/druid/guice/PropertiesModule.java
rabbitmq/pom.xml
s3-extensions/pom.xml
server/pom.xml
services/pom.xml
2014-09-26 11:39:24 -07:00
3bf39cc9f8
attempted to fix merge-conflicts
2014-09-24 15:55:42 -07:00
273205f217
initial attempt for abstraction; druid cluster works with Derby as a default
2014-09-19 17:39:59 -07:00
cfa92e8217
fix incorrect nullable annotations
2014-09-15 12:13:52 -07:00
469ccbbe5e
Merge branch 'master' into druid-0.7.x
...
Conflicts:
cassandra-storage/pom.xml
common/pom.xml
examples/pom.xml
hdfs-storage/pom.xml
histogram/pom.xml
indexing-hadoop/pom.xml
indexing-service/pom.xml
kafka-eight/pom.xml
kafka-seven/pom.xml
pom.xml
processing/pom.xml
processing/src/main/java/io/druid/query/FinalizeResultsQueryRunner.java
processing/src/main/java/io/druid/query/UnionQueryRunner.java
processing/src/main/java/io/druid/query/groupby/GroupByQueryRunnerFactory.java
processing/src/main/java/io/druid/query/topn/TopNQueryEngine.java
processing/src/main/java/io/druid/query/topn/TopNQueryRunnerFactory.java
rabbitmq/pom.xml
s3-extensions/pom.xml
server/pom.xml
server/src/test/java/io/druid/server/initialization/JettyTest.java
services/pom.xml
2014-09-11 16:20:50 -07:00
88de45c695
ensure jackson injection is available on startup
2014-08-29 15:13:52 -07:00
58ab759fc6
remove unused imports
2014-08-29 14:03:47 -07:00
ac05836833
make Java 8 javadoc happy
2014-08-29 13:58:50 -07:00
4c40e71e54
address cr
2014-06-19 14:48:46 -07:00
77ec4df797
update guava, java-util, and druid-api
2014-06-03 13:43:38 -07:00
4de7d4e127
make timeline findOvershadowed a set
2014-05-19 20:24:47 -07:00
1b6e816c28
Merge pull request #544 from metamx/java-8-fix-tests
...
Make tests pass with Java 8
2014-05-19 18:11:52 -06:00
3a17a60d4c
more comprehensive merge sequence tests
2014-05-15 16:27:13 -07:00
b51f0f7837
update java-util and add more merge sequence tests
2014-05-15 16:19:12 -07:00
43853a1d9e
Merge pull request #545 from metamx/union-query-fix
...
Union query fix
2014-05-15 17:17:46 -06:00
ddc36c89cc
union query fix and a broken test
2014-05-15 23:10:40 +05:30
8fb76138bd
tests must not assume findOvershadowed ordering
2014-05-14 22:48:10 -07:00
62c55eaf26
fix broken CombiningSequence.toYielder behavior
2014-05-14 17:08:15 -07:00
e0ff2aa0d6
make isPostgreSQL check URI instead of metadata
2014-03-27 12:13:10 -07:00
fa34f2ac59
more consistent placeholder references
2014-03-26 16:41:27 -07:00
e84128ebbe
remove database specific code from ConfigManager
2014-03-26 14:29:37 -07:00
aec6457a4d
fix config manager
2014-03-26 09:42:33 -07:00
70db460f97
Blocking Executors and maxPendingPersists, oh my!
...
- Execs.newBlockingSingleThreaded can now accept capacity = 0.
- Changed default maxPendingPersists from 2 to 0.
- Fixed serde of maxPendingPersists in RealtimeIndexTasks.
2014-03-05 10:55:12 -08:00
e40725d5f3
Added support for PostgreSQL on overlord nodes
2014-02-26 01:24:37 +01:00
a835db2a3c
Added support for task storage in PostgreSQL
2014-02-26 01:24:37 +01:00
d089f05127
Improved support for PostgreSQL in config management
2014-02-17 18:11:15 +00:00
8fac0b0b7a
Fixed automatic tables creation for PostgreSQL
2014-02-17 18:11:15 +00:00
53c5124b99
DbConnector: Indexes on tasks table
2014-02-05 15:59:14 -08:00
5c043803c6
review comments
2014-01-20 13:40:36 +05:30
874cdb1b5b
review comments
2014-01-17 12:51:28 +05:30
3db33c3268
review comments
2014-01-17 12:51:10 +05:30
6f5e1afbff
add blocking mechanism after 2 pending persist batches
2014-01-17 03:01:04 +05:30
d49554185a
fix broken default table names
2013-11-07 17:38:00 -08:00
560981acbf
a ton of fixes to docs
2013-10-10 15:05:01 -07:00
c47fe202c7
Fix HadoopDruidIndexer to work with the new way of things
...
There are multiple and sundry changes in here.
First, "HadoopDruidIndexer" has been split into two pieces, (1) CliHadoop which pulls the hadoop version and builds up the right classpath with the proper hadoop version to run the indexer and (2) CliInternalHadoopIndexer which actually runs the indexer.
In order to work around a bunch of jets3t version conflicts with Hadoop and Druid, I needed to extract the S3 deep storage stuff into its own module. I then also moved the HDFS stuff into its own module so that I could eliminate the dependency on Hadoop for druid-server.
In doing these changes, I wanted to make the extensions buildable with only the druid-api jar, so a few other things had to move out of Druid and into druid-api. They are all API-level things, however, so they really belong in druid-api instead.
Lastly, I removed the druid-realtime module and put it all in druid-server.
2013-10-09 15:15:44 -05:00
bc8db7daa5
1) make chat handler resource work again
...
2) add more default configs
3) make examples work again
2013-10-02 14:22:39 -07:00
87259321b6
port hadoop druid indexer to new guice framework
2013-09-26 11:04:42 -07:00
5712b29c8c
Fix issues with bindings and handling extensions
...
The way the Guice bindings were setup previously, each process only had bindings
for the things it cared about. This became problematic when adding extension modules
that bound everything that they could possibly need expecting that the processes would
only instantiate what they actually do need. Guice tries to fail-fast and verifies that all
bindings exist before it does anything, which is a problem because the extension bind
some objects that don't necessarily have all of their dependencies bound in all processes.
The fix for this is to build a single Injector with all bindings in it and let each of the
processes only load the things that they care about. This also requires the use of
Module overrides and other such interesting things, which are node done.
In doing the fix, I also swapped out the way that the DataSegmentPusher/Puller stuff is bound, as well as made the Cassandra stuff fail if its settings are not provided. This all of a sudden made all of the things require Cassandra's settings, so I migrated the Cassandra deep storage stuff into its own module.
In doing these changes, I also discovered that some properties weren't properly converting for the ConvertProperties command (specifically, the properties related to data segment loading and pushing), so I fixed that.
2013-09-20 17:45:01 -05:00
64f3821b4e
throw exception for annotated members
2013-09-16 17:36:31 -07:00
3e4a4f5566
1) Adjustments to allow for the addition of new Runnables via modules. Modules should implement CliCommandCreator and add their stuff there.
2013-09-11 17:41:20 -05:00
dadd9bc5ce
1) Rename MetricSelector(Factory) to ColumnSelector(Factory)
...
2) Hard coded plugin as a first pass, won't actually work for anyone else.
2013-09-04 11:19:46 -05:00
3c39f90c89
1) Move Firehose interface and dependencies to druid-api
...
2) Move DataSegment* interfaces and dependencies to druid-api
2013-08-31 16:43:28 -05:00
5ab671050e
No more com.metamx.druid, it is now all io.druid!
2013-08-30 19:42:12 -05:00
bd0756e360
More stuff moved, things still compiling and tests still passing. Yay!
2013-08-30 18:58:35 -05:00
56e2b956d0
OMG!!! A lot of stuff has been moved. Modules have been created and destroyed, but everything is compiling and unit tests are passing, OMFG this is awesome.!
2013-08-30 18:21:04 -05:00
ca2c56d827
1) Remove PostAggregator and put it in druid-api
...
2) Remove io.druid files from client
2013-08-29 16:50:09 -05:00
cb90ed05b0
Revert the previous commit. After going down this path, I realized that extracting things enough to allow Queries to be extended without depending on Druid proper was going to lead down a very nasty path. So, I've decided against that. Extending queries will require a tight dependency on Druid proper.
2013-08-29 16:45:03 -05:00
2a46086e20
1) Didn't remove the io.druid files from client. Remove those and make sure things compile
...
2) Switch DefaultObjectMapper to CommonObjectMapper
3) Create new DefaultObjectMapper in client that has Query stuff registered on it by default
2013-08-29 15:25:36 -05:00
1f37e962f6
1) Delete the files that have moved to druid-api and get them through the druid-api dependency. Still need to fix Searchable on StorageAdapter, but other than that, everything should compile.
2013-08-28 16:18:40 -05:00
bfe96739a3
1) Move various "api" classes to io.druid packages and make sure things compile and stuff -- continued! Come classes didn't get committed?
2013-08-28 15:52:55 -05:00
9c30ced5ea
1) Move various "api" classes to io.druid packages and make sure things compile and stuff
2013-08-28 15:51:02 -05:00
5fa944dd26
Merge branch 'master' into guice
...
Conflicts:
client/src/main/java/com/metamx/druid/coordination/BatchDataSegmentAnnouncer.java
client/src/main/java/com/metamx/druid/curator/announcement/Announcer.java
client/src/main/java/com/metamx/druid/query/filter/SelectorDimFilter.java
client/src/main/java/com/metamx/druid/query/search/SearchQueryQueryToolChest.java
indexing-service/src/main/java/com/metamx/druid/indexing/common/tasklogs/S3TaskLogs.java
indexing-service/src/main/java/com/metamx/druid/indexing/coordinator/ForkingTaskRunner.java
indexing-service/src/main/java/com/metamx/druid/indexing/coordinator/RemoteTaskRunner.java
indexing-service/src/main/java/com/metamx/druid/indexing/worker/WorkerCuratorCoordinator.java
indexing-service/src/test/java/com/metamx/druid/indexing/coordinator/RemoteTaskRunnerTest.java
pom.xml
server/src/main/java/com/metamx/druid/http/MasterMain.java
server/src/main/java/com/metamx/druid/http/MasterServletModule.java
server/src/main/java/com/metamx/druid/master/DruidMasterConfig.java
server/src/test/java/com/metamx/druid/master/DruidMasterTest.java
server/src/test/java/com/metamx/druid/query/group/GroupByQueryRunnerTest.java
2013-08-27 14:27:32 -05:00
269997dc94
1) ExecutorNode is working, except for the running of the task. Need to adjust it to be able to run a task and then everything will be wonderful
2013-08-26 18:08:41 -05:00
55dbda2046
1) Worker appears to be running! It's also now known as the MiddleManager
2013-08-23 17:59:48 -05:00
b897c2cb22
1) IndexCoordinator appears to work as the CliOverlord now, yay!
2013-08-23 14:11:34 -05:00
9b9a2a9f53
Merge branch 'master' into num-shard-spec
...
Conflicts:
common/src/main/java/com/metamx/druid/partition/IntegerPartitionChunk.java
2013-08-21 15:01:46 -07:00
54ad854471
Merge branch 'master' into rt-ut
2013-08-21 13:15:02 -07:00
88661b26a0
bug fix for RTR removing workers race condition and partition chunks not being sorted by chunk number
2013-08-21 11:14:54 -07:00
9c5fcc0984
Add missing license headers
2013-08-20 17:48:15 -07:00
8388cbf69d
NumberedShardSpec
...
Each shard has a number, and there are a fixed number of shards that will
form a complete set.
2013-08-20 16:45:12 -07:00
9c145f5ce3
basic unit tets for realtime functinality
2013-08-19 15:00:10 -07:00
5f292143e0
disable merging segments with linear shard spec
2013-08-06 16:17:49 -07:00
d66af7625c
1) Eliminate the need for SingleSegmentLoader
...
2) Setup the configuration glue for ServerInventoryViews and DataSegmentAnnouncer
3) Make processes run and work again!
2013-08-02 17:05:01 -07:00
9e78bb38f5
Merge branch 'master' into guice
...
Conflicts:
client/src/main/java/com/metamx/druid/QueryableNode.java
client/src/main/java/com/metamx/druid/client/ServerInventoryView.java
client/src/main/java/com/metamx/druid/coordination/SingleDataSegmentAnnouncer.java
client/src/main/java/com/metamx/druid/initialization/CuratorDiscoveryConfig.java
client/src/main/java/com/metamx/druid/query/MetricsEmittingExecutorService.java
indexing-hadoop/src/test/java/com/metamx/druid/indexer/HadoopDruidIndexerConfigTest.java
indexing-service/src/main/java/com/metamx/druid/indexing/common/TaskToolbox.java
indexing-service/src/main/java/com/metamx/druid/indexing/coordinator/http/IndexerCoordinatorNode.java
indexing-service/src/main/java/com/metamx/druid/indexing/worker/executor/ExecutorNode.java
indexing-service/src/main/java/com/metamx/druid/indexing/worker/http/WorkerNode.java
pom.xml
server/src/main/java/com/metamx/druid/coordination/ServerManager.java
server/src/main/java/com/metamx/druid/coordination/ZkCoordinator.java
server/src/main/java/com/metamx/druid/db/DatabaseRuleManager.java
server/src/main/java/com/metamx/druid/db/DatabaseSegmentManager.java
server/src/main/java/com/metamx/druid/http/ComputeNode.java
server/src/main/java/com/metamx/druid/http/MasterMain.java
server/src/main/java/com/metamx/druid/loading/SegmentLoaderConfig.java
server/src/main/java/com/metamx/druid/loading/SingleSegmentLoader.java
server/src/main/java/com/metamx/druid/master/DruidMaster.java
2013-08-01 16:42:47 -07:00
019bb5d453
1) Another whole bunch of changes to annotate things and create Modules and bind stuff. But OMFG, the compute node actually appears to be working!
...
2) The compute node works with Guice
3) The compute node fires up with Guice and appears to work
4) Did I mention that the compute node, now called historical node, fires up with Guice and appears to work?
2013-08-01 15:28:08 -07:00
f4fc8b2f6e
1) A whole boondoggle of changes to Guicify the instantiation of things for the Historical nodes
...
2) Add GuiceInjectableValues that supports @JacksonInject resolution via Guice
3) Refactor RequestLogger stuff and setup polymorphic deserialization method of creating objects
2013-07-31 18:36:47 -07:00
9de2e5b096
1) Adjust JsonConfigurator to be able to attach the proper property path if it is renamed via a @JsonProperty annotation
...
2) JsonConfigurator now requires that fields be annotated with @JsonProperty. It will fail fast if they aren't.
2013-07-31 14:55:34 -07:00
fjy
nishantmonu51
Xavier Léauté
Charles Allen
jisookim0513
jisookim0513
jisookim0513
fjy
Gian Merlino
Tugdual Saunier
cheddar