From a47c0d25797794043f1ca985ce5f71da7775ad83 Mon Sep 17 00:00:00 2001 From: Gian Merlino Date: Fri, 23 Apr 2021 20:06:08 -0700 Subject: [PATCH] Clarify meaning of "root-level fields" in the documentation. (#11143) --- docs/ingestion/index.md | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/docs/ingestion/index.md b/docs/ingestion/index.md index ccc9f0b29ac..a8bae6be027 100644 --- a/docs/ingestion/index.md +++ b/docs/ingestion/index.md @@ -489,12 +489,13 @@ Normal interpretation occurs when either `dimensions` or `spatialDimensions` is Schemaless interpretation occurs when both `dimensions` and `spatialDimensions` are empty or null. In this case, the set of dimensions is determined in the following way: -1. First, start from the set of all input fields from the [`inputFormat`](./data-formats.md) (or the [`flattenSpec`](./data-formats.md#flattenspec), if one is being used). -2. Any field listed in `dimensionExclusions` is excluded. -3. The field listed as `column` in the [`timestampSpec`](#timestampspec) is excluded. -4. Any field used as an input to an aggregator from the [metricsSpec](#metricsspec) is excluded. -5. Any field with the same name as an aggregator from the [metricsSpec](#metricsspec) is excluded. -6. All other fields are ingested as `string` typed dimensions with the [default settings](#dimension-objects). +1. First, start from the set of all root-level fields from the input record, as determined by the [`inputFormat`](./data-formats.md). "Root-level" includes all fields at the top level of a data structure, but does not included fields nested within maps or lists. To extract these, you must use a [`flattenSpec`](./data-formats.md#flattenspec). All fields of non-nested data formats, such as CSV and delimited text, are considered root-level. +2. If a [`flattenSpec`](./data-formats.md#flattenspec) is being used, the set of root-level fields includes any fields generated by the flattenSpec. The useFieldDiscovery parameter determines whether the original root-level fields will be retained or discarded. +3. Any field listed in `dimensionExclusions` is excluded. +4. The field listed as `column` in the [`timestampSpec`](#timestampspec) is excluded. +5. Any field used as an input to an aggregator from the [metricsSpec](#metricsspec) is excluded. +6. Any field with the same name as an aggregator from the [metricsSpec](#metricsspec) is excluded. +7. All other fields are ingested as `string` typed dimensions with the [default settings](#dimension-objects). > Note: Fields generated by a [`transformSpec`](#transformspec) are not currently considered candidates for > schemaless dimension interpretation.