Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
druid/examples/quickstart/tutorial/rollup-index.json

48 lines
1.2 KiB
JSON
Raw Normal View History

New quickstart and tutorials (#6126) * New quickstart and tutorials * PR comments * Fix tranquility
2018-08-09 16:37:52 -04:00
{
Tutorials use new ingestion spec where possible (#9155) * Tutorials use new ingestion spec where possible There are 2 main changes * Use task type index_parallel instead of index * Remove the use of parser + firehose in favor of inputFormat + inputSource index_parallel is the preferred method starting in 0.17. Setting the job to index_parallel with the default maxNumConcurrentSubTasks(1) is the equivalent of an index task Instead of using a parserSpec, dimensionSpec and timestampSpec have been promoted to the dataSchema. The format is described in the ioConfig as the inputFormat. There are a few cases where the new format is not supported * Hadoop must use firehoses instead of the inputSource and inputFormat * There is no equivalent of a combining firehose as an inputSource * A Combining firehose does not support index_parallel * fix typo
2020-01-15 17:08:29 -05:00
"type" : "index_parallel",
New quickstart and tutorials (#6126) * New quickstart and tutorials * PR comments * Fix tranquility
2018-08-09 16:37:52 -04:00
"spec" : {
"dataSchema" : {
"dataSource" : "rollup-tutorial",
Tutorials use new ingestion spec where possible (#9155) * Tutorials use new ingestion spec where possible There are 2 main changes * Use task type index_parallel instead of index * Remove the use of parser + firehose in favor of inputFormat + inputSource index_parallel is the preferred method starting in 0.17. Setting the job to index_parallel with the default maxNumConcurrentSubTasks(1) is the equivalent of an index task Instead of using a parserSpec, dimensionSpec and timestampSpec have been promoted to the dataSchema. The format is described in the ioConfig as the inputFormat. There are a few cases where the new format is not supported * Hadoop must use firehoses instead of the inputSource and inputFormat * There is no equivalent of a combining firehose as an inputSource * A Combining firehose does not support index_parallel * fix typo
2020-01-15 17:08:29 -05:00
"timestampSpec": {
"column": "timestamp",
"format": "iso"
},
"dimensionsSpec" : {
"dimensions" : [
"srcIP",
"dstIP"
]
New quickstart and tutorials (#6126) * New quickstart and tutorials * PR comments * Fix tranquility
2018-08-09 16:37:52 -04:00
},
"metricsSpec" : [
{ "type" : "count", "name" : "count" },
{ "type" : "longSum", "name" : "packets", "fieldName" : "packets" },
{ "type" : "longSum", "name" : "bytes", "fieldName" : "bytes" }
],
"granularitySpec" : {
"type" : "uniform",
"segmentGranularity" : "week",
"queryGranularity" : "minute",
"intervals" : ["2018-01-01/2018-01-03"],
"rollup" : true
}
},
"ioConfig" : {
Tutorials use new ingestion spec where possible (#9155) * Tutorials use new ingestion spec where possible There are 2 main changes * Use task type index_parallel instead of index * Remove the use of parser + firehose in favor of inputFormat + inputSource index_parallel is the preferred method starting in 0.17. Setting the job to index_parallel with the default maxNumConcurrentSubTasks(1) is the equivalent of an index task Instead of using a parserSpec, dimensionSpec and timestampSpec have been promoted to the dataSchema. The format is described in the ioConfig as the inputFormat. There are a few cases where the new format is not supported * Hadoop must use firehoses instead of the inputSource and inputFormat * There is no equivalent of a combining firehose as an inputSource * A Combining firehose does not support index_parallel * fix typo
2020-01-15 17:08:29 -05:00
"type" : "index_parallel",
"inputSource" : {
New quickstart and tutorials (#6126) * New quickstart and tutorials * PR comments * Fix tranquility
2018-08-09 16:37:52 -04:00
"type" : "local",
"baseDir" : "quickstart/tutorial",
"filter" : "rollup-data.json"
},
Tutorials use new ingestion spec where possible (#9155) * Tutorials use new ingestion spec where possible There are 2 main changes * Use task type index_parallel instead of index * Remove the use of parser + firehose in favor of inputFormat + inputSource index_parallel is the preferred method starting in 0.17. Setting the job to index_parallel with the default maxNumConcurrentSubTasks(1) is the equivalent of an index task Instead of using a parserSpec, dimensionSpec and timestampSpec have been promoted to the dataSchema. The format is described in the ioConfig as the inputFormat. There are a few cases where the new format is not supported * Hadoop must use firehoses instead of the inputSource and inputFormat * There is no equivalent of a combining firehose as an inputSource * A Combining firehose does not support index_parallel * fix typo
2020-01-15 17:08:29 -05:00
"inputFormat" : {
"type" : "json"
},
New quickstart and tutorials (#6126) * New quickstart and tutorials * PR comments * Fix tranquility
2018-08-09 16:37:52 -04:00
"appendToExisting" : false
},
"tuningConfig" : {
Tutorials use new ingestion spec where possible (#9155) * Tutorials use new ingestion spec where possible There are 2 main changes * Use task type index_parallel instead of index * Remove the use of parser + firehose in favor of inputFormat + inputSource index_parallel is the preferred method starting in 0.17. Setting the job to index_parallel with the default maxNumConcurrentSubTasks(1) is the equivalent of an index task Instead of using a parserSpec, dimensionSpec and timestampSpec have been promoted to the dataSchema. The format is described in the ioConfig as the inputFormat. There are a few cases where the new format is not supported * Hadoop must use firehoses instead of the inputSource and inputFormat * There is no equivalent of a combining firehose as an inputSource * A Combining firehose does not support index_parallel * fix typo
2020-01-15 17:08:29 -05:00
"type" : "index_parallel",
Add support maxRowsPerSegment for auto compaction (#6780) * Add support maxRowsPerSegment for auto compaction * fix build * fix build * fix teamcity * add test * fix test * address comment
2019-01-10 12:50:14 -05:00
"maxRowsPerSegment" : 5000000,
Deprecate NoneShardSpec and drop support for automatic segment merge (#6883) * Deprecate noneShardSpec * clean up noneShardSpec constructor * revert unnecessary change * Deprecate mergeTask * add more doc * remove convert from indexMerger * Remove mergeTask * remove HadoopDruidConverterConfig * fix build * fix build * fix teamcity * fix teamcity * fix ServerModule * fix compilation * fix compilation
2019-03-16 02:29:25 -04:00
"maxRowsInMemory" : 25000
New quickstart and tutorials (#6126) * New quickstart and tutorials * PR comments * Fix tranquility
2018-08-09 16:37:52 -04:00
}
}
}