Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
14,283 Commits 111 Branches 503 Tags 721 MiB
Commit Graph

150 Commits

Author SHA1 Message Date
Nishant 351d570684 Improve startup script - create PID and LOG dir if they do not exist (#3808) 2017-01-02 09:20:22 -08:00
Nishant 93c34d3c3f Ability to add hadoop config directory via environment variable (#3781) 2016-12-16 11:19:15 -08:00
kaijianding f1dee037d6 fix 'No Such File' error when execute script out of druid installation directory (#3517) 2016-11-01 09:57:09 -07:00
Nishant 94b3c74cdc Druid launch script improvements (#3175) 
* Add status command to launch scripts

* make druid init script to pick up config directories from environment variables

make druid init script to pick up config directories from environment
variables
 2016-06-24 15:02:34 -07:00
Charles Allen 2863adef60 Add jconsole.sh example script for connecting (#2947) 
* Can change logging at run-time
 2016-05-16 15:37:15 -07:00
Walton Seymour 13a2b265a5 Adds init scripts (#2939) 2016-05-10 09:00:13 -07:00
fjy 1aa363cea7 new quickstart 2016-02-04 09:37:38 -08:00
Bingkun Guo 4914925d65 New extension loading mechanism 
1) Remove maven client from downloading extensions at runtime.
2) Provide a way to load Druid extensions and hadoop dependencies through file system.
3) Refactor pull-deps so that it can download extensions into extension directories.
4) Add documents on how to use this new extension loading mechanism.
5) Change the way how Druid tarball is generated. Now all the extensions + hadoop-client 2.3.0
are packaged within the Druid tarball.
 2015-10-21 14:22:36 -05:00
David Lim f42f6247ee Modified the Twitter firehose to process more properties 
Add dimensions such as screen name, retweet and verified booleans,
source, location, and originator information to support additional
analytics.
 2015-09-25 00:21:15 -06:00
David Lim 13e808368f Disable metadata publishing attempt in example script 
Example config no-ops DataSegmentPusher and DataSegmentAnnouncer but
still binds SegmentPublisher to MetadataSegmentPublisher which was
throwing exceptions when it tried to publish and no DB was configured.
Setting this binds NoopSegmentPublisher instead.
 2015-09-16 15:57:04 -07:00
Gian Merlino 5ca440ff15 run_druid_server.sh 2015-08-18 18:32:33 -07:00
Gian Merlino 601cd82987 Use local repository instead of maven central: 
- Use local repo in run_example_server.
- Add extensions-repo to common.runtime.properties for the tarball.
 2015-08-18 18:32:33 -07:00
fjy 0074138bb9 fix versions in tutorials 2015-07-15 16:18:16 -07:00
Charles Allen 27e25fe513 fix IRCFirehoseFactory json name/nick mismatch 
* Now it is nick as per wikipedia example
 2015-05-01 12:35:44 -07:00
fjy 4575c5378c Rework the druid docs 2015-04-21 10:11:11 -07:00
Fangjin Yang 40cb702079 Merge pull request #1242 from andrewserff/master 
twitter_runtime.spec example does not work
 2015-03-26 10:03:06 -07:00
nishantmonu51 99bf430844 fix examples and doc 2015-03-26 19:01:10 +05:30
Andrew Serff 518b42c900 Updated the twitter_runtime.spec so that it works and includes geospatial indexs 2015-03-25 16:21:43 -06:00
fjy 6a47c1530c update versions to prepare for rc release 2015-03-19 11:39:38 -07:00
James Estes 9cb5d9bd00 Various improvements to the tutorial and building experience. 
- Improve geo ip download stability
- Retry default geo ip download if corrupt
- Clarify docs on building & where to get artifact
 2015-02-25 18:49:16 -07:00
fjy 46bd614d63 fix broken links 2015-02-24 20:14:17 -08:00
James Estes 562de6c621 Update docs and examples for log4j2 usage. 
- Put configs early in classpath in examples so log4j2.xml will get picked up properly
- Add an example log4j2.xml file.
- Update Logging doc.
 2015-02-19 11:40:56 -07:00
Xavier Léauté 8e33e58422 rename Druid binary assembly 2015-02-18 17:42:55 -08:00
Xavier Léauté 19e11ae601 Use default ports in examples 2015-02-18 11:46:27 -08:00
Xavier Léauté cff218ee20 update example + tutorial for kafka 0.8 2015-02-17 23:58:51 -08:00
fjy 3fab045b2d Go through and fix mistakes in tutorials and docs 2015-02-17 15:21:16 -08:00
fjy 13de51494e update versions in tutorials and docs 2015-02-12 16:11:31 -08:00
fjy 67ecb07ccd update version in docs 2015-02-03 14:26:42 -08:00
fjy 7540f2f562 Add more docs for production clusters 2015-01-21 14:21:27 -08:00
Charles Allen 687c82daa8 Added more Twitter fields to TwitterSpritzerFirehoseFactory 
* Now with GEOGRAPHY support!
 2014-12-12 15:27:00 -08:00
fjy 28b72a69ad redocumenting ingestion 2014-12-08 16:15:46 -08:00
fjy fe0a56cad6 fix newline 2014-11-24 14:10:07 -08:00
fjy 28c609b187 fix some brokenness 2014-11-24 13:31:56 -08:00
fjy 9da66291e1 change naming to common config 2014-11-21 15:36:42 -08:00
fjy 8ee4d12562 Refactor structure for examples and extensions 2014-11-21 14:45:24 -08:00
fjy f3970bb1d8 Merge branch 'master' into new-init 
Conflicts:
	examples/config/historical/runtime.properties
	examples/config/overlord/runtime.properties
	examples/config/realtime/runtime.properties
 2014-08-08 12:44:29 -07:00
fjy 7be93a770a make all firehoses work with tasks, add a lot more documentation about configuration 2014-05-28 16:33:59 -07:00
fjy b6fb4245aa Merge branch 'master' into new-schema 
Conflicts:
	indexing-hadoop/src/main/java/io/druid/indexer/HadoopDriverConfig.java
	indexing-hadoop/src/main/java/io/druid/indexer/HadoopDruidIndexerConfig.java
	indexing-hadoop/src/main/java/io/druid/indexer/HadoopDruidIndexerConfigBuilder.java
	pom.xml
	server/src/main/java/io/druid/segment/realtime/RealtimeManager.java
	server/src/main/java/io/druid/segment/realtime/firehose/EventReceiverFirehoseFactory.java
 2014-05-06 14:32:51 -07:00
nishantmonu51 477e01daf7 review comments 2014-05-02 15:21:06 +05:30
fjy 2adcf07f5f Merge branch 'master' into new-schema 
Conflicts:
	indexing-hadoop/src/main/java/io/druid/indexer/DetermineHashedPartitionsJob.java
	indexing-service/src/main/java/io/druid/indexing/common/task/RealtimeIndexTask.java
	indexing-service/src/test/java/io/druid/indexing/common/task/TaskSerdeTest.java
	processing/src/test/java/io/druid/segment/TestIndex.java
	server/src/main/java/io/druid/segment/realtime/RealtimeManager.java
	server/src/main/java/io/druid/segment/realtime/plumber/RealtimePlumberSchool.java
 2014-03-17 10:59:31 -07:00
fjy b4f1591260 clean up examples, finish paper 2014-03-13 18:52:08 -07:00
fjy d089e65682 fix build.sh 2014-03-06 13:06:09 -08:00
fjy 422e776a0e fix bug in build script 2014-03-06 12:51:24 -08:00
fjy 20cac8c506 not compiling yet but close 2014-02-19 15:54:27 -08:00
fjy 3b17c4c03c a whole bunch of docs and fixes 2014-01-13 18:01:56 -08:00
fjy c1b69df9ec remove unused run_ec2 script 2014-01-10 16:28:00 -08:00
Stefán Freyr Stefánsson f644c8ea66 Ugly quick'n'dirty way of getting lyra to work. 2013-11-19 10:18:00 +00:00
jamesbw 414615c43b added services jar to examples script classpath 
Signed-off-by: jamesbw <github+jamesbw@squareup.com>
 2013-11-12 15:41:44 -08:00
fjy aeb411a3a3 fix according to code review and fix broken examples 2013-11-07 15:42:48 -08:00
fjy 5bca36247f fix more docs 2013-10-18 15:37:43 -07:00
fjy 560981acbf a ton of fixes to docs 2013-10-10 15:05:01 -07:00
fjy a9a723bd11 clean up poms, add a new loading your own data tutorial, add new validation, clean up logs 2013-10-09 15:42:39 -07:00
Russell Jurney 806bc0f58c Changed webstream example to second indexGranularity. 2013-10-09 12:23:00 -07:00
Russell Jurney 4e30a14438 Reverted minute/second change. 2013-10-08 13:27:39 -07:00
Russell Jurney caa3c663f5 Changed wikipedia example to second indexGranularity. 2013-10-08 13:05:38 -07:00
fjy cdbbf445fd remove ec2 startup files; they are obsoleted by whirr 2013-10-06 14:22:44 -07:00
fjy ac330f72bb first set of changes to standarize the naming convention we use in druid 2013-10-03 16:36:48 -07:00
fjy bc8db7daa5 1) make chat handler resource work again 
2) add more default configs
3) make examples work again
 2013-10-02 14:22:39 -07:00
cheddar 6bfe2310bb Merge branch 'master' into guice 
Conflicts:
	client/pom.xml
	client/src/main/java/com/metamx/druid/query/segment/LegacySegmentSpec.java
	indexing-common/pom.xml
	indexing-service/src/main/java/com/metamx/druid/indexing/common/TaskToolbox.java
	indexing-service/src/main/java/com/metamx/druid/indexing/common/TaskToolboxFactory.java
	indexing-service/src/main/java/com/metamx/druid/indexing/common/task/RealtimeIndexTask.java
	indexing-service/src/main/java/com/metamx/druid/indexing/worker/executor/ExecutorNode.java
	indexing-service/src/test/java/com/metamx/druid/indexing/coordinator/RemoteTaskRunnerTest.java
	indexing-service/src/test/java/com/metamx/druid/indexing/coordinator/TaskLifecycleTest.java
	indexing-service/src/test/java/com/metamx/druid/indexing/coordinator/TaskQueueTest.java
	realtime/src/main/java/com/metamx/druid/realtime/RealtimeNode.java
	realtime/src/main/java/com/metamx/druid/realtime/firehose/IrcFirehoseFactory.java
	realtime/src/main/java/com/metamx/druid/realtime/firehose/WikipediaIrcDecoder.java
	realtime/src/main/java/com/metamx/druid/realtime/plumber/RealtimePlumberSchool.java
	server/src/main/java/com/metamx/druid/BaseServerNode.java
	server/src/main/java/com/metamx/druid/http/ComputeNode.java
	server/src/main/java/com/metamx/druid/index/v1/IncrementalIndexStorageAdapter.java
	server/src/main/java/com/metamx/druid/query/group/GroupByQueryEngine.java
 2013-09-18 16:46:54 -05:00
fjy 898f937f3c also update wikipedia example query.body 2013-09-04 14:20:25 -07:00
fjy 9f391c2bdd update example wiki realtime spec to match publically avaiable segments 2013-09-04 13:59:23 -07:00
Russell Jurney a11eae4144 Edited assembly.xml to exclude the selfcontained jars, and edited path of run_example_server to use the lib/ directory instead of selfcontained jars, thus reducing the size of the assembly tarball from 150MB to 50MB. 2013-09-04 12:53:38 -07:00
Russell Jurney 3e9d2d497d Changed port number for realtime server from 8080 to 8083 to make the exaxmples work with the new realtime config. 2013-08-29 15:14:09 -07:00
cheddar 5fa944dd26 Merge branch 'master' into guice 
Conflicts:
	client/src/main/java/com/metamx/druid/coordination/BatchDataSegmentAnnouncer.java
	client/src/main/java/com/metamx/druid/curator/announcement/Announcer.java
	client/src/main/java/com/metamx/druid/query/filter/SelectorDimFilter.java
	client/src/main/java/com/metamx/druid/query/search/SearchQueryQueryToolChest.java
	indexing-service/src/main/java/com/metamx/druid/indexing/common/tasklogs/S3TaskLogs.java
	indexing-service/src/main/java/com/metamx/druid/indexing/coordinator/ForkingTaskRunner.java
	indexing-service/src/main/java/com/metamx/druid/indexing/coordinator/RemoteTaskRunner.java
	indexing-service/src/main/java/com/metamx/druid/indexing/worker/WorkerCuratorCoordinator.java
	indexing-service/src/test/java/com/metamx/druid/indexing/coordinator/RemoteTaskRunnerTest.java
	pom.xml
	server/src/main/java/com/metamx/druid/http/MasterMain.java
	server/src/main/java/com/metamx/druid/http/MasterServletModule.java
	server/src/main/java/com/metamx/druid/master/DruidMasterConfig.java
	server/src/test/java/com/metamx/druid/master/DruidMasterTest.java
	server/src/test/java/com/metamx/druid/query/group/GroupByQueryRunnerTest.java
 2013-08-27 14:27:32 -05:00
Russell Jurney 1669e46338 Fixing bug where twitter example does not run. 2013-08-22 19:48:21 -07:00
Xavier Léauté ef12439df9 add wikipedia realtime example 2013-08-21 17:56:31 -07:00
fjy 85ee8bb267 port realtime to guice 2013-08-13 17:08:45 -07:00
cheddar 9f71d42314 1) Broker works with Guice 
2) Extract ServerViewModule to handle various ServerView thingies
3) Extract QueryToolChestModule and QueryRunnerFactoryModule to reuse code for configuration of Query stuff
4) Extract QueryJettyServerInitializer to reuse between Historical and Broker nodes
5) Remove ClientMain, BrokerMain and BrokerNode
 2013-08-06 12:01:52 -07:00
cheddar d64bc2f762 1) Eliminate Old mains() that aren't needed anymore. 2013-08-05 13:33:43 -07:00
cheddar 5f01c633bd Merge pull request #192 from activitystream/rabbitmq 
Adding a RabbitMQFirehoseFactory implementation
 2013-07-19 16:44:44 -07:00
Stefán Freyr Stefánsson af97b44a6f merging pull request from cheddar - jacksonifying the RabbitMQ connection and config. 2013-07-19 11:16:50 +00:00
Dhruv Parthasarathy f114e2f425 query file fixed 2013-07-18 13:37:33 -07:00
Stefán Freyr Stefánsson 217d539e46 Adding a newline at the end of the file. 2013-07-16 15:18:03 +00:00
Stefán Freyr Stefánsson 22ec7c53fc Code cleanup and formatting, documentation and moving all configuration into the consumerProps. 2013-07-16 15:08:39 +00:00
Stefán Freyr Stefánsson d2f8e52752 Merge remote-tracking branch 'upstream/master' into rabbitmq 2013-07-16 13:08:10 +00:00
Stefán Freyr Stefánsson a43ccf587f Fixed some errors in the realtime.spec and query.body files and cleaned up the format to make it more readable as an example. 2013-07-15 18:38:21 +00:00
Stefán Freyr Stefánsson 13328a6b36 Implemented a better query.body for the test I'm using. Added a shutdown listener to the connection and channel to get a notification when they are closed. Question remains whether the connection should be re-established when that happens. 2013-07-15 09:50:58 +00:00
Stefán Freyr Stefánsson 4d998e4c27 Making more things configurable through the realtime.spec file. 2013-07-13 16:49:39 +00:00
Stefán Freyr Stefánsson fc74333a28 adding an example dir to test the rabbitmq implementation 2013-07-13 16:14:10 +00:00
cheddar 09b1130810 Merge pull request #186 from rjurney/master 
Made upload.sh executable
 2013-07-11 13:06:58 -07:00
Russell Jurney a33aa57ca9 Made webstream example work. 2013-07-10 17:04:51 -07:00
Russell Jurney a39148a0af Fixed hanging ssh bug FINALLY 2013-07-09 18:41:06 -07:00
Russell Jurney 2df7463311 Removed dupe numThreads value from configs, fiddling with run_ec2.sh/env.sh 2013-07-09 14:44:03 -07:00
Russell Jurney 570fb1cf19 Working stuff, doing pull request. CTRL-C problem remains. 2013-07-08 17:30:37 -07:00
Russell Jurney 2403f1eceb Seperated env/run files. 2013-07-08 13:42:50 -07:00
Russell Jurney 056bd9cfe6 Created log directory, and echo ssh instructions. 2013-07-05 18:03:37 -07:00
Russell Jurney 1e46c6f4a2 Removed debug on ssh 2013-07-05 16:51:01 -07:00
Russell Jurney 9e615bad7f Works, as long as you hit ctrl-C when it says so. 2013-07-05 13:40:45 -07:00
Russell Jurney 5f0f9b29ec Working on fixing hang problem with mysql install. 2013-07-04 14:11:48 -07:00
Russell Jurney ca81b81f1b Moved mysql setup to bottom 2013-07-03 19:53:06 -07:00
Russell Jurney 86ecdc4c26 Removed credentials from ec2 script, ready for publishing. 2013-07-02 17:17:36 -07:00
Russell Jurney 8a5f4960a9 Working ec2 bootstrap script for ec2 environment 2013-07-02 12:46:14 -07:00
Russell Jurney c1f6c0ab4a In progress ec2 script. 2013-07-01 19:34:43 -07:00
Russell Jurney 8a1512ae4b Initial script to boot and configure an ubuntu 12.04 server for running the four druid node types. 2013-07-01 17:55:01 -07:00
Russell Jurney 0bd4e8b016 Fixed interactive twitter example to ask for consumer/access keys and secrets upon run. 2013-06-13 18:20:28 -07:00
cheddar c4a3a9b2fc 1) Fix scripts to run the examples from a fresh build 2013-06-10 11:53:57 -07:00
Steve Harris cfe9b7f177 Added a few comments and made sure it still works for non-kits 2013-05-04 21:02:31 -07:00
Steve Harris c0e38981ff Forgot to edit the source file so config was messed up 2013-05-04 20:50:53 -07:00
Steve Harris 5a9bf98b6c Some changes to automatically build a kit with samples. 2013-05-04 20:04:16 -07:00
cheddar e1a2c43baf 1) Adjust the example code to all be in one repository. 
2) Adjust the run scripts to be able to switch between different examples.
 2013-05-01 12:51:26 -05:00