+ When the record's schema is not provided but inferred based on the data itself, providing a value for the "Field Name for Content" property
+ is especially important. (For detailed information on schema inference, see the "Schema Inference" section below.)
+ Let's focus on cases where an XML element (called <field_with_attribute> in the examples) has an XML attribute and some content and no sub-elements.
+ For the examples below, let's assume that a ConvertRecord processor is used, and it uses an XMLReader controller service and an XMLRecordSetWriter
+ controller service. The settings for XMLReader are provided separately for each example. The settings for XMLRecordSetWriter are common
+ for all the examples below. This way an XML to XML conversion is executed and comparing the input data with the output highlights
+ the schema inference behavior. The same behavior can be observed if a different Writer controller service is used.
+ XMLRecordSetWriter was chosen for these examples so that the input and the output are easily comparable.
+ The settings of the common XMLRecordSetWriter are the following:
+
| Property Name | +Property Value | +
|---|---|
| Schema Access Strategy | +Inherit Record Schema |
+
| Suppress Null Values | +Never Suppress |
+
+ XMLReader settings: +
+ +| Property Name | +Property Value | +
|---|---|
| Schema Access Strategy | +Infer Schema |
+
| Expect Records as Array | +false |
+
| Field Name for Content | +not set | +
+ Input: +
+ +
+
+ <record>
+ <field_with_attribute attr="attr_content">
+ content of field
+ </field_with_attribute>
+ </record>
+
+
+ As mentioned above, the element called "field_with_attribute" has an attribute and some content but no sub-element.
+ ++ Output: +
+ +
+
+ <record>
+ <field_with_attribute>
+ <attr>attr_content</attr>
+ <value></value>
+ </field_with_attribute>
+ </record>
+
+
+
+ In the XMLReader's settings, no value is set for the "Field Name for Content" property. In such cases the schema inference logic
+ adds a field named "value" to the schema. However, since "Field Name for Content" is not set, the data processing logic is instructed
+ not to consider the original content of the parent XML tags (<field_with_attribute> the content of which is "content of field"
+ in the example). So a new field named "value" appears in the schema but no value is assigned to it from the data, thus the field is empty.
+ The XML attribute (named "attr") is processed, a field named "attr" is added to the schema and the attribute's value ("attr_content") is assigned to it.
+ In a case like this, the parent field's original content is lost and a new field named "value" appears in the schema with no data assigned to it.
+ This is to make sure that no data is overwritten in the record if it already contains a field named "value". More on that case in Example 3 and Example 4.
+
+ In this example, the XMLReader's "Field Name for Content" property is filled with the value "original_content". The input data is the same as + in the previous example. +
+ ++ XMLReader settings: +
+ +| Property Name | +Property Value | +
|---|---|
| Schema Access Strategy | +Infer Schema |
+
| Expect Records as Array | +false |
+
| Field Name for Content | +original_content |
+
+ Input: +
+ +
+
+ <record>
+ <field_with_attribute attr="attr_content">
+ content of field
+ </field_with_attribute>
+ </record>
+
+
+ + Output: +
+ +
+
+ <record>
+ <field_with_attribute>
+ <attr>attr_content</attr>
+ <original_content>content of field</original_content>
+ </field_with_attribute>
+ </record>
+
+
+
+ The XMLReader's "Field Name for Content" property contains the value "original_content" (the concrete value is not important, what is important
+ is that a value is provided and it does not clash with the name of any sub-element in <field_with_attribute>).
+ This explicitly tells the XMLReader controller service to create a field named "original_content" and make the original content of
+ the parent XML tag the value of the field named "original_content". Adding the XML attributed named "attr" works just like in the first example.
+ Since the <field_with_attribute> element had no child-element with the name "original_content", no data is lost.
+
+ In this example, XMLReader's "Field Name for Content" property is left empty. In the input data, the <field_with_attribute> element
+ has some content and a sub-element named <value>.
+
+ XMLReader settings: +
+ +| Property Name | +Property Value | +
|---|---|
| Schema Access Strategy | +Infer Schema |
+
| Expect Records as Array | +false |
+
| Field Name for Content | +not set | +
+ Input: +
+ +
+
+ <record>
+ <field_with_attribute attr="attr_content">
+ content of field<value>123</value>
+ </field_with_attribute>
+ </record>
+
+
+ + Output: +
+ +
+
+ <record>
+ <field_with_attribute>
+ <attr>attr_content</attr>
+ <value>123</value>
+ </field_with_attribute>
+ </record>
+
+
+
+ The "Field Name for Content" property is not set, and the XML element has a sub-element named "value". The name of the sub-element clashes with the
+ default field name added to the schema by the Schema Inference logic (see Example 1). As seen in the output data, the input XML attribute's value
+ is added to the record just like in the previous examples. The value of the <value> element is retained, but the content of the
+ <field_with_attribute> that was outside of the sub-element, is lost.
+
+ In this example, XMLReader's "Field Name for Content" property is given the value "value". In the input data, the <field_with_attribute> element
+ has some content and a sub-element named <value>. The name of the sub-element clashes with the value of the "Field Name for Content" property.
+
+ XMLReader settings: +
+ +| Property Name | +Property Value | +
|---|---|
| Schema Access Strategy | +Infer Schema |
+
| Expect Records as Array | +false |
+
| Field Name for Content | +value |
+
+ Input: +
+ +
+
+ <record>
+ <field_with_attribute attr="attr_content">
+ content of field<value>123</value>
+ </field_with_attribute>
+ </record>
+
+
+ + Output: +
+ +
+
+ <record>
+ <field_with_attribute>
+ <attr>attr_content</attr>
+ <value>content of field</value>
+ </field_with_attribute>
+ </record>
+
+
+
+ The "Field Name for Content" property's value is "value", and the XML element has a sub-element named "value". The name of the sub-element clashes with the
+ value of the "Field Name for Content" property. The value of the <value> element is replaced by the content of the
+ <field_with_attribute> element, and the original content of the <value> element is lost.
+
+ To avoid losing any data, the XMLReader's "Field Name for Content" property needs to be given a value that does not clash with any sub-element's name + in the input data. In this example the input data is the same as in the previous one, but the "Field Name for Content" property's value is "original_content", + a value that does not clash with any sub-element name. No data is lost in this case. +
+ ++ XMLReader settings: +
+ +| Property Name | +Property Value | +
|---|---|
| Schema Access Strategy | +Infer Schema |
+
| Expect Records as Array | +false |
+
| Field Name for Content | +original_content |
+
+ Input: +
+ +
+
+ <record>
+ <field_with_attribute attr="attr_content">
+ content of field<value>123</value>
+ </field_with_attribute>
+ </record>
+
+
+ + Output: +
+ +
+
+ <record>
+ <field_with_attribute>
+ <attr>attr_content</attr>
+ <value>123</value>
+ <original_content>content of field</original_content>
+ </field_with_attribute>
+ </record>
+
+
+
+ It can be seen in the output data, that the attribute has been added to the <field_with_attribute> element as a sub-element,
+ the <value> retained its value, and the original content of the <field_with_attribute> element has been added as a sub-element
+ named "original_content". This is because a value was chosen for the "Field Name for Content" property that does not clash with any of
+ the existing sub-elements of the input XML element (<field_with_attribute>). No data is lost.
+
diff --git a/nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/test/java/org/apache/nifi/xml/TestInferXmlSchema.java b/nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/test/java/org/apache/nifi/xml/TestInferXmlSchema.java
index b5bdd76025..2a4cd14f10 100644
--- a/nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/test/java/org/apache/nifi/xml/TestInferXmlSchema.java
+++ b/nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/test/java/org/apache/nifi/xml/TestInferXmlSchema.java
@@ -93,7 +93,8 @@ public class TestInferXmlSchema {
@Test
public void testStringFieldWithAttributes() throws IOException {
- final RecordSchema schema = inferSchema("src/test/resources/xml/TextNodeWithAttribute.xml", true);
+ final String contentFieldName = "contentfield";
+ final RecordSchema schema = inferSchema("src/test/resources/xml/TextNodeWithAttribute.xml", contentFieldName, true);
assertEquals(3, schema.getFieldCount());
@@ -106,12 +107,16 @@ public class TestInferXmlSchema {
final RecordSchema childSchema = ((RecordDataType) softwareDataType).getChildSchema();
assertSame(RecordFieldType.BOOLEAN, childSchema.getDataType("favorite").get().getFieldType());
- assertSame(RecordFieldType.STRING, childSchema.getDataType("value").get().getFieldType());
+ assertSame(RecordFieldType.STRING, childSchema.getDataType(contentFieldName).get().getFieldType());
}
private RecordSchema inferSchema(final String filename, final boolean ignoreWrapper) throws IOException {
+ return inferSchema(filename, "contentfield", ignoreWrapper);
+ }
+
+ private RecordSchema inferSchema(final String filename, final String contentFieldName, final boolean ignoreWrapper) throws IOException {
final File file = new File(filename);
- final RecordSourceFactory