Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
druid/examples/quickstart/tutorial/deletion-index.json

61 lines
1.4 KiB
JSON
Raw Normal View History

New quickstart and tutorials (#6126) * New quickstart and tutorials * PR comments * Fix tranquility
2018-08-09 16:37:52 -04:00
{
Tutorials use new ingestion spec where possible (#9155) * Tutorials use new ingestion spec where possible There are 2 main changes * Use task type index_parallel instead of index * Remove the use of parser + firehose in favor of inputFormat + inputSource index_parallel is the preferred method starting in 0.17. Setting the job to index_parallel with the default maxNumConcurrentSubTasks(1) is the equivalent of an index task Instead of using a parserSpec, dimensionSpec and timestampSpec have been promoted to the dataSchema. The format is described in the ioConfig as the inputFormat. There are a few cases where the new format is not supported * Hadoop must use firehoses instead of the inputSource and inputFormat * There is no equivalent of a combining firehose as an inputSource * A Combining firehose does not support index_parallel * fix typo
2020-01-15 17:08:29 -05:00
"type" : "index_parallel",
New quickstart and tutorials (#6126) * New quickstart and tutorials * PR comments * Fix tranquility
2018-08-09 16:37:52 -04:00
"spec" : {
"dataSchema" : {
"dataSource" : "deletion-tutorial",
Tutorials use new ingestion spec where possible (#9155) * Tutorials use new ingestion spec where possible There are 2 main changes * Use task type index_parallel instead of index * Remove the use of parser + firehose in favor of inputFormat + inputSource index_parallel is the preferred method starting in 0.17. Setting the job to index_parallel with the default maxNumConcurrentSubTasks(1) is the equivalent of an index task Instead of using a parserSpec, dimensionSpec and timestampSpec have been promoted to the dataSchema. The format is described in the ioConfig as the inputFormat. There are a few cases where the new format is not supported * Hadoop must use firehoses instead of the inputSource and inputFormat * There is no equivalent of a combining firehose as an inputSource * A Combining firehose does not support index_parallel * fix typo
2020-01-15 17:08:29 -05:00
"timestampSpec": {
"column": "time",
"format": "iso"
},
"dimensionsSpec" : {
"dimensions" : [
"channel",
"cityName",
"comment",
"countryIsoCode",
"countryName",
"isAnonymous",
"isMinor",
"isNew",
"isRobot",
"isUnpatrolled",
"metroCode",
"namespace",
"page",
"regionIsoCode",
"regionName",
"user",
{ "name": "added", "type": "long" },
{ "name": "deleted", "type": "long" },
{ "name": "delta", "type": "long" }
]
New quickstart and tutorials (#6126) * New quickstart and tutorials * PR comments * Fix tranquility
2018-08-09 16:37:52 -04:00
},
"metricsSpec" : [],
"granularitySpec" : {
"type" : "uniform",
"segmentGranularity" : "hour",
"queryGranularity" : "none",
"intervals" : ["2015-09-12/2015-09-13"],
"rollup" : false
}
},
"ioConfig" : {
Tutorials use new ingestion spec where possible (#9155) * Tutorials use new ingestion spec where possible There are 2 main changes * Use task type index_parallel instead of index * Remove the use of parser + firehose in favor of inputFormat + inputSource index_parallel is the preferred method starting in 0.17. Setting the job to index_parallel with the default maxNumConcurrentSubTasks(1) is the equivalent of an index task Instead of using a parserSpec, dimensionSpec and timestampSpec have been promoted to the dataSchema. The format is described in the ioConfig as the inputFormat. There are a few cases where the new format is not supported * Hadoop must use firehoses instead of the inputSource and inputFormat * There is no equivalent of a combining firehose as an inputSource * A Combining firehose does not support index_parallel * fix typo
2020-01-15 17:08:29 -05:00
"type" : "index_parallel",
"inputSource" : {
New quickstart and tutorials (#6126) * New quickstart and tutorials * PR comments * Fix tranquility
2018-08-09 16:37:52 -04:00
"type" : "local",
fixup docs to download from Apache mirror, fixup tarball name and path, change references from quickstart/* to quickstart/tutorial/* (#6570)
2018-11-02 00:47:29 -04:00
"baseDir" : "quickstart/tutorial/",
New quickstart and tutorials (#6126) * New quickstart and tutorials * PR comments * Fix tranquility
2018-08-09 16:37:52 -04:00
"filter" : "wikiticker-2015-09-12-sampled.json.gz"
},
Tutorials use new ingestion spec where possible (#9155) * Tutorials use new ingestion spec where possible There are 2 main changes * Use task type index_parallel instead of index * Remove the use of parser + firehose in favor of inputFormat + inputSource index_parallel is the preferred method starting in 0.17. Setting the job to index_parallel with the default maxNumConcurrentSubTasks(1) is the equivalent of an index task Instead of using a parserSpec, dimensionSpec and timestampSpec have been promoted to the dataSchema. The format is described in the ioConfig as the inputFormat. There are a few cases where the new format is not supported * Hadoop must use firehoses instead of the inputSource and inputFormat * There is no equivalent of a combining firehose as an inputSource * A Combining firehose does not support index_parallel * fix typo
2020-01-15 17:08:29 -05:00
"inputFormat" : {
"type" : "json"
},
New quickstart and tutorials (#6126) * New quickstart and tutorials * PR comments * Fix tranquility
2018-08-09 16:37:52 -04:00
"appendToExisting" : false
},
"tuningConfig" : {
Tutorials use new ingestion spec where possible (#9155) * Tutorials use new ingestion spec where possible There are 2 main changes * Use task type index_parallel instead of index * Remove the use of parser + firehose in favor of inputFormat + inputSource index_parallel is the preferred method starting in 0.17. Setting the job to index_parallel with the default maxNumConcurrentSubTasks(1) is the equivalent of an index task Instead of using a parserSpec, dimensionSpec and timestampSpec have been promoted to the dataSchema. The format is described in the ioConfig as the inputFormat. There are a few cases where the new format is not supported * Hadoop must use firehoses instead of the inputSource and inputFormat * There is no equivalent of a combining firehose as an inputSource * A Combining firehose does not support index_parallel * fix typo
2020-01-15 17:08:29 -05:00
"type" : "index_parallel",
Add support maxRowsPerSegment for auto compaction (#6780) * Add support maxRowsPerSegment for auto compaction * fix build * fix build * fix teamcity * add test * fix test * address comment
2019-01-10 12:50:14 -05:00
"maxRowsPerSegment" : 5000000,
Deprecate NoneShardSpec and drop support for automatic segment merge (#6883) * Deprecate noneShardSpec * clean up noneShardSpec constructor * revert unnecessary change * Deprecate mergeTask * add more doc * remove convert from indexMerger * Remove mergeTask * remove HadoopDruidConverterConfig * fix build * fix build * fix teamcity * fix teamcity * fix ServerModule * fix compilation * fix compilation
2019-03-16 02:29:25 -04:00
"maxRowsInMemory" : 25000
New quickstart and tutorials (#6126) * New quickstart and tutorials * PR comments * Fix tranquility
2018-08-09 16:37:52 -04:00
}
}
}