[Bug]Kinesis-data-format IT can not work (#11071)

* start schema-resgity and replace json template

* add docs

Co-authored-by: yuezhang <yuezhang@freewheel.tv>

integration-tests/README.md

@@ -258,7 +258,7 @@ of the integration test run discussed above. Running these tests requires the us
 their own Cloud. 
 
 Currently, the integration test supports Amazon Kinesis, Google Cloud Storage, Amazon S3, and Microsoft Azure.
-These can be run by providing "kinesis-index", "gcs-deep-storage", "s3-deep-storage", or "azure-deep-storage" 
+These can be run by providing "kinesis-index", "kinesis-data-format", "gcs-deep-storage", "s3-deep-storage", or "azure-deep-storage" 
 to -Dgroups for Amazon Kinesis, Google Cloud Storage, Amazon S3, and Microsoft Azure respectively. Note that only
 one group should be run per mvn command.
 

integration-tests/script/docker_compose_args.sh

@@ -68,8 +68,14 @@ getComposeArgs()
       # default
       echo "-f ${DOCKERDIR}/docker-compose.yml"
     fi
+  else
+    if [ "$DRUID_INTEGRATION_TEST_GROUP" = "kinesis-data-format" ]
+    then
+      # default + with override config + schema registry container
+      echo "-f ${DOCKERDIR}/docker-compose.override-env.yml -f ${DOCKERDIR}/docker-compose.schema-registry.yml"
     else
     # with override config
       echo "-f ${DOCKERDIR}/docker-compose.override-env.yml"
     fi
+  fi
 }

integration-tests/src/test/java/org/apache/druid/tests/indexer/AbstractKinesisIndexingServiceTest.java

@@ -117,6 +117,11 @@ public abstract class AbstractKinesisIndexingServiceTest extends AbstractStreamI
             "%%STREAM_PROPERTIES_KEY%%",
             "endpoint"
         );
+        spec = StringUtils.replace(
+                spec,
+                "%%SCHEMA_REGISTRY_HOST%%",
+                StringUtils.format("http://%s", config.getSchemaRegistryInternalHost())
+        );
         return StringUtils.replace(
             spec,
             "%%STREAM_PROPERTIES_VALUE%%",