USRealEstate
/
web-api-commander
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Issue #73 - Initial ResourceInfo Model Generation

This commit is contained in:
Joshua Darnell 2021-04-27 16:17:46 -07:00
parent 19f72345b1
commit 0ac742e7d1
5 changed files with 328 additions and 51 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

96
README.md
View File

@ -58,48 +58,51 @@ $ java -jar path/to/web-api-commander.jar
Doing so displays the following information: Doing so displays the following information:
``` ```
usage: java -jar web-api-commander usage: java -jar web-api-commander
--bearerToken <b> Bearer token to be used with the --bearerToken <b> Bearer token to be used with the
request. request.
--clientId <d> Client Id to be used with the request. --clientId <d> Client Id to be used with the request.
--clientSecret <s> --clientSecret <s>
--contentType <t> Results format: JSON (default), --contentType <t> Results format: JSON (default),
JSON_NO_METADATA, JSON_FULL_METADATA, JSON_NO_METADATA, JSON_FULL_METADATA,
XML. XML.
--entityName <n> The name of the entity to fetch, e.g. --entityName <n> The name of the entity to fetch, e.g.
Property. Property.
--generateDDAcceptanceTests Generates acceptance tests in the --generateDDAcceptanceTests Generates acceptance tests in the
current directory. current directory.
--generateMetadataReport Generates metadata report from given --generateMetadataReport Generates metadata report from given
<inputFile>. <inputFile>.
--generateQueries Resolves queries in a given RESOScript --generateQueries Resolves queries in a given RESOScript
<inputFile> and displays them in <inputFile> and displays them in
standard out. standard out.
--generateReferenceDDL Generates reference DDL to create a --generateReferenceDDL Generates reference DDL to create a
RESO-compliant SQL database. Pass RESO-compliant SQL database. Pass
--useKeyNumeric to generate the DB using --useKeyNumeric to generate the DB
numeric keys. using numeric keys.
--generateReferenceEDMX Generates reference metadata in EDMX --generateReferenceEDMX Generates reference metadata in EDMX
format. format.
--getMetadata Fetches metadata from <serviceRoot> --generateResourceInfoModels Generates Java Models for the Web API
using <bearerToken> and saves results in Reference Server in the current
<outputFile>. directory.
--help print help --getMetadata Fetches metadata from <serviceRoot>
--inputFile <i> Path to input file. using <bearerToken> and saves results
--outputFile <o> Path to output file. in <outputFile>.
--runRESOScript Runs commands in RESOScript file given --help print help
as <inputFile>. --inputFile <i> Path to input file.
--saveGetRequest Performs GET from <requestURI> using the --outputFile <o> Path to output file.
given <bearerToken> and saves output to --runRESOScript Runs commands in RESOScript file given
<outputFile>. as <inputFile>.
--serviceRoot <s> Service root URL on the host. --saveGetRequest Performs GET from <requestURI> using
--uri <u> URI for raw request. Use 'single quotes' the given <bearerToken> and saves
to enclose. output to <outputFile>.
--useEdmEnabledClient present if an EdmEnabledClient should be --serviceRoot <s> Service root URL on the host.
used. --uri <u> URI for raw request. Use 'single
--useKeyNumeric present if numeric keys are to be used quotes' to enclose.
for database DDL generation. --useEdmEnabledClient present if an EdmEnabledClient should
--validateMetadata Validates previously-fetched metadata in be used.
the <inputFile> path. --useKeyNumeric present if numeric keys are to be used
for database DDL generation.
--validateMetadata Validates previously-fetched metadata
in the <inputFile> path.
``` ```
When using commands, if required arguments aren't provided, relevant feedback will be displayed in the terminal. When using commands, if required arguments aren't provided, relevant feedback will be displayed in the terminal.
@ -227,6 +230,17 @@ New Cucumber BDD acceptance tests will be generated and placed in a timestamped
To update the current tests, copy the newly generated ones into the [Data Dictionary BDD `.features` directory](src/main/java/org/reso/certification/features/data-dictionary/v1-7-0), run the `./gradlew build` task, and if everything works as expected, commit the newly generated tests. To update the current tests, copy the newly generated ones into the [Data Dictionary BDD `.features` directory](src/main/java/org/reso/certification/features/data-dictionary/v1-7-0), run the `./gradlew build` task, and if everything works as expected, commit the newly generated tests.
## Generating RESO Web API Reference Server Data Models
The RESO Commander can be used to generate data models for the Web API Reference server from the currently approved [Data Dictionary Spreadsheet](src/main/resources/RESODataDictionary-1.7.xlsx).
The Commander project's copy of the sheet needs to be updated with a copy of the [DD Google Sheet](https://docs.google.com/spreadsheets/d/1SZ0b6T4_lz6ti6qB2Je7NSz_9iNOaV_v9dbfhPwWgXA/edit?usp=sharing) prior to generating reference metadata.
```
$ java -jar path/to/web-api-commander.jar --generateResourceInfoModels
```
New ResourceInfo Models for the Web API Reference Server will be generated and placed in a timestamped directory relative to your current path.
## Generating RESO Data Dictionary Reference Metadata ## Generating RESO Data Dictionary Reference Metadata
In addition to generating DD acceptance tests, the RESO Commander can generate reference metadata based on the current reference [Data Dictionary Spreadsheet](src/main/resources/RESODataDictionary-1.7.xlsx). In addition to generating DD acceptance tests, the RESO Commander can generate reference metadata based on the current reference [Data Dictionary Spreadsheet](src/main/resources/RESODataDictionary-1.7.xlsx).

2
src/main/java/org/reso/certification/codegen/BDDProcessor.java
View File

@ -122,7 +122,7 @@ public class BDDProcessor extends WorksheetProcessor {
return tags; return tags;
} }
private static String padLeft(String s, int n) { public static String padLeft(String s, int n) {
String[] padding = new String[n]; String[] padding = new String[n];
Arrays.fill(padding, " "); Arrays.fill(padding, " ");
return String.join("", padding) + s; return String.join("", padding) + s;

8
src/main/java/org/reso/certification/codegen/DDLProcessor.java
View File

@ -149,7 +149,7 @@ public class DDLProcessor extends WorksheetProcessor {
.append("\n\n") .append("\n\n")
.append("CREATE TABLE IF NOT EXISTS ") .append("CREATE TABLE IF NOT EXISTS ")
//exception for ouid so it doesn't become o_u_i_d //exception for ouid so it doesn't become o_u_i_d
.append(CaseFormat.UPPER_CAMEL.to(CaseFormat.LOWER_UNDERSCORE, resourceName).replace("o_u_i_d", "ouid")) .append(buildDbTableName(resourceName))
.append(" ( ") .append(" ( ")
.append(templateContent).append(",\n") .append(templateContent).append(",\n")
.append(PADDING).append(PADDING).append(buildPrimaryKeyMarkup(resourceName)).append("\n") .append(PADDING).append(PADDING).append(buildPrimaryKeyMarkup(resourceName)).append("\n")
@ -164,6 +164,12 @@ public class DDLProcessor extends WorksheetProcessor {
LOG.info(this::buildInsertLookupsStatement); LOG.info(this::buildInsertLookupsStatement);
} }
public static String buildDbTableName(String resourceName) {
return CaseFormat.UPPER_CAMEL.to(CaseFormat.LOWER_UNDERSCORE, resourceName).replace("o_u_i_d", "ouid");
}
private static String buildCreateLookupStatement(boolean useKeyNumeric) { private static String buildCreateLookupStatement(boolean useKeyNumeric) {
return return
"\n\n/**\n" + "\n\n/**\n" +

249
src/main/java/org/reso/certification/codegen/ResourceInfoProcessor.java Normal file
View File

@ -0,0 +1,249 @@
package org.reso.certification.codegen;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.apache.poi.ss.usermodel.Sheet;
import org.reso.commander.common.Utils;
import org.reso.models.ReferenceStandardField;
import static org.reso.certification.codegen.DDLProcessor.buildDbTableName;
import static org.reso.certification.containers.WebAPITestContainer.EMPTY_STRING;
public class ResourceInfoProcessor extends WorksheetProcessor {
final static String
ANNOTATION_TERM_DISPLAY_NAME = "RESO.OData.Metadata.StandardName",
ANNOTATION_TERM_DESCRIPTION = "Core.Description",
ANNOTATION_TERM_URL = "RESO.DDWikiUrl";
private static final Logger LOG = LogManager.getLogger(ResourceInfoProcessor.class);
private static final String
FILE_EXTENSION = ".java";
public void processResourceSheet(Sheet sheet) {
super.processResourceSheet(sheet);
markup.append(ResourceInfoTemplates.buildClassInfo(sheet.getSheetName(), null));
}
@Override
void processNumber(ReferenceStandardField row) {
markup.append(ResourceInfoTemplates.buildNumberMarkup(row));
}
@Override
void processStringListSingle(ReferenceStandardField row) {
markup.append(ResourceInfoTemplates.buildStringListSingleMarkup(row));
}
@Override
void processString(ReferenceStandardField row) {
markup.append(ResourceInfoTemplates.buildStringMarkup(row));
}
@Override
void processBoolean(ReferenceStandardField row) {
markup.append(ResourceInfoTemplates.buildBooleanMarkup(row));
}
@Override
void processStringListMulti(ReferenceStandardField row) {
markup.append(ResourceInfoTemplates.buildStringListMultiMarkup(row));
}
@Override
void processDate(ReferenceStandardField row) {
markup.append(ResourceInfoTemplates.buildDateMarkup(row));
}
@Override
void processTimestamp(ReferenceStandardField row) {
markup.append(ResourceInfoTemplates.buildTimestampMarkup(row));
}
@Override
void processCollection(ReferenceStandardField row) {
LOG.debug("Collection Type is not supported!");
}
@Override
void generateOutput() {
LOG.info("Using reference worksheet: " + REFERENCE_WORKSHEET);
LOG.info("Generating ResourceInfo .java files for the following resources: " + resourceTemplates.keySet().toString());
resourceTemplates.forEach((resourceName, content) -> {
//put in local directory rather than relative to where the input file is
Utils.createFile(getDirectoryName(), resourceName + "Definition" + FILE_EXTENSION, content);
});
}
@Override
String getDirectoryName() {
return startTimestamp + "-ResourceInfoModels";
}
@Override
public void afterResourceSheetProcessed(Sheet sheet) {
assert sheet != null && sheet.getSheetName() != null;
String resourceName = sheet.getSheetName();
String templateContent =
markup.toString() + "\n" +
" return " + resourceName + "Definition.fieldList;\n" +
" }\n" +
"}";
resourceTemplates.put(resourceName, templateContent);
resetMarkupBuffer();
}
public static final class ResourceInfoTemplates {
/**
* Contains various templates used for test generation
* TODO: add a formatter rather than using inline spaces
*/
public static String buildClassInfo(String resourceName, String generatedTimestamp) {
if (resourceName == null) return null;
if (generatedTimestamp == null) generatedTimestamp = Utils.getIsoTimestamp();
final String definitionName = resourceName + "Definition";
return "package org.reso.service.data.definition;\n" + "\n" +
"import org.apache.olingo.commons.api.edm.EdmPrimitiveTypeKind;\n" +
"import org.reso.service.data.meta.FieldInfo;\n" +
"import org.reso.service.data.meta.ResourceInfo;\n" + "\n" +
"import java.util.ArrayList;\n" + "\n" +
"// This class was autogenerated on: " + generatedTimestamp + "\n" +
"public class " + definitionName + " extends ResourceInfo {\n" +
" private static ArrayList<FieldInfo> fieldList = null;\n" + "\n" +
" public " + definitionName + "() {" + "\n" +
" this.tableName = " + buildDbTableName(resourceName) + ";\n" +
" this.resourcesName = " + resourceName + ";\n" +
" this.resourceName = " + resourceName + ";\n" +
" }\n" + "\n" +
" public ArrayList<FieldInfo> getFieldList() {\n" +
" return " + definitionName + ".getStaticFieldList();\n" +
" }\n" + "\n" +
" public static ArrayList<FieldInfo> getStaticFieldList() {\n" +
" if (null != " + definitionName + ".fieldList) {\n" +
" return " + definitionName + ".fieldList;\n" +
" }\n" + "\n" +
" ArrayList<FieldInfo> list = new ArrayList<FieldInfo>();\n" +
" " + definitionName + ".fieldList = list;\n" +
" FieldInfo fieldInfo = null;\n";
}
public static String buildBooleanMarkup(ReferenceStandardField field) {
if (field == null) return EMPTY_STRING;
//TODO: refactor into one method that takes a type name and returns the appropriate content
return "\n" +
" fieldInfo = new FieldInfo(\"" + field.getStandardName() + "\", EdmPrimitiveTypeKind.Boolean.getFullQualifiedName());\n" +
" fieldInfo.addAnnotation(\"" + field.getDisplayName() + "\", \"" + ANNOTATION_TERM_DISPLAY_NAME + "\");\n" +
" fieldInfo.addAnnotation(\"" + field.getDefinition() + "\", \"" + ANNOTATION_TERM_DESCRIPTION + "\");\n" +
" fieldInfo.addAnnotation(\"" + field.getWikiPageUrl() + "\", \"" + ANNOTATION_TERM_URL + "\");\n" +
" list.add(fieldInfo);" +
"\n";
}
public static String buildDateMarkup(ReferenceStandardField field) {
if (field == null) return EMPTY_STRING;
return "\n" +
" fieldInfo = new FieldInfo(\"" + field.getStandardName() + "\", EdmPrimitiveTypeKind.Date.getFullQualifiedName());\n" +
" fieldInfo.addAnnotation(\"" + field.getDisplayName() + "\", \"" + ANNOTATION_TERM_DISPLAY_NAME + "\");\n" +
" fieldInfo.addAnnotation(\"" + field.getDefinition() + "\", \"" + ANNOTATION_TERM_DESCRIPTION + "\");\n" +
" fieldInfo.addAnnotation(\"" + field.getWikiPageUrl() + "\", \"" + ANNOTATION_TERM_URL + "\");\n" +
" list.add(fieldInfo);" +
"\n";
}
/**
* Provides special routing for Data Dictionary numeric types, which may be Integer or Decimal
*
* @param field the numeric field to build type markup for
* @return a string containing specific markup for the given field
*/
public static String buildNumberMarkup(ReferenceStandardField field) {
if (field == null) return EMPTY_STRING;
if (field.getSuggestedMaxPrecision() != null) return buildDecimalMarkup(field);
else return buildIntegerMarkup(field);
}
public static String buildDecimalMarkup(ReferenceStandardField field) {
if (field == null) return EMPTY_STRING;
return "\n" +
" fieldInfo = new FieldInfo(\"" + field.getStandardName() + "\", EdmPrimitiveTypeKind.Decimal.getFullQualifiedName());\n" +
" fieldInfo.addAnnotation(\"" + field.getDisplayName() + "\", \"" + ANNOTATION_TERM_DISPLAY_NAME + "\");\n" +
" fieldInfo.addAnnotation(\"" + field.getDefinition() + "\", \"" + ANNOTATION_TERM_DESCRIPTION + "\");\n" +
" fieldInfo.addAnnotation(\"" + field.getWikiPageUrl() + "\", \"" + ANNOTATION_TERM_URL + "\");\n" +
" list.add(fieldInfo);" +
"\n";
//TODO: Length is actually scale for Decimal fields by the DD! :/
//TODO: Add setScale property to Decimal types in FieldInfo
//TODO: Precision is actually Scale for Decimal fields by the DD! :/
//TODO: Add setPrecision property to Decimal types in FieldInfo
}
public static String buildIntegerMarkup(ReferenceStandardField field) {
if (field == null) return EMPTY_STRING;
return "\n" +
" fieldInfo = new FieldInfo(\"" + field.getStandardName() + "\", EdmPrimitiveTypeKind.Int64.getFullQualifiedName());\n" +
" fieldInfo.addAnnotation(\"" + field.getDisplayName() + "\", \"" + ANNOTATION_TERM_DISPLAY_NAME + "\");\n" +
" fieldInfo.addAnnotation(\"" + field.getDefinition() + "\", \"" + ANNOTATION_TERM_DESCRIPTION + "\");\n" +
" fieldInfo.addAnnotation(\"" + field.getWikiPageUrl() + "\", \"" + ANNOTATION_TERM_URL + "\");\n" +
" list.add(fieldInfo);" +
"\n";
}
private static String buildStandardEnumerationMarkup(String lookupName) {
//TODO: add code to build Lookups
return "\n /* TODO: buildStandardEnumerationMarkup */\n";
}
public static String buildStringListMultiMarkup(ReferenceStandardField field) {
if (field == null) return EMPTY_STRING;
//TODO: add multi lookup handler
return "\n /* TODO: buildStringListMultiMarkup */\n";
}
public static String buildStringListSingleMarkup(ReferenceStandardField field) {
if (field == null) return EMPTY_STRING;
//TODO: add single lookup handler
return "\n /* TODO: buildStringListSingleMarkup */\n";
}
public static String buildStringMarkup(ReferenceStandardField field) {
if (field == null) return EMPTY_STRING;
String content = "\n" +
" fieldInfo = new FieldInfo(\"" + field.getStandardName() + "\", EdmPrimitiveTypeKind.String.getFullQualifiedName());\n" +
" fieldInfo.addAnnotation(\"" + field.getDisplayName() + "\", \"" + ANNOTATION_TERM_DISPLAY_NAME + "\");\n" +
" fieldInfo.addAnnotation(\"" + field.getDefinition() + "\", \"" + ANNOTATION_TERM_DESCRIPTION + "\");\n" +
" fieldInfo.addAnnotation(\"" + field.getWikiPageUrl() + "\", \"" + ANNOTATION_TERM_URL + "\");\n";
if (field.getSuggestedMaxLength() != null) {
content +=
" fieldInfo.setMaxLength(" + field.getSuggestedMaxLength() + ");\n";
}
content +=
" list.add(fieldInfo);" + "\n";
return content;
}
public static String buildTimestampMarkup(ReferenceStandardField field) {
if (field == null) return EMPTY_STRING;
return "\n" +
" fieldInfo = new FieldInfo(\"" + field.getStandardName() + "\", EdmPrimitiveTypeKind.DateTime.getFullQualifiedName());\n" +
" fieldInfo.addAnnotation(\"" + field.getDisplayName() + "\", \"" + ANNOTATION_TERM_DISPLAY_NAME + "\");\n" +
" fieldInfo.addAnnotation(\"" + field.getDefinition() + "\", \"" + ANNOTATION_TERM_DESCRIPTION + "\");\n" +
" fieldInfo.addAnnotation(\"" + field.getWikiPageUrl() + "\", \"" + ANNOTATION_TERM_URL + "\");\n" +
" list.add(fieldInfo);" +
"\n";
}
}
}

24
src/main/java/org/reso/commander/App.java
View File

@ -4,10 +4,7 @@ import org.apache.commons.cli.*;
import org.apache.logging.log4j.LogManager; import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger; import org.apache.logging.log4j.Logger;
import org.apache.olingo.commons.api.format.ContentType; import org.apache.olingo.commons.api.format.ContentType;
import org.reso.certification.codegen.BDDProcessor; import org.reso.certification.codegen.*;
import org.reso.certification.codegen.DDLProcessor;
import org.reso.certification.codegen.DataDictionaryCodeGenerator;
import org.reso.certification.codegen.EDMXProcessor;
import org.reso.models.ClientSettings; import org.reso.models.ClientSettings;
import org.reso.models.ODataTransportWrapper; import org.reso.models.ODataTransportWrapper;
import org.reso.models.Request; import org.reso.models.Request;
@ -222,6 +219,14 @@ public class App {
} catch (Exception ex) { } catch (Exception ex) {
LOG.error(getDefaultErrorMessage(ex)); LOG.error(getDefaultErrorMessage(ex));
} }
} else if (cmd.hasOption(APP_OPTIONS.ACTIONS.GENERATE_RESOURCE_INFO_MODELS)) {
APP_OPTIONS.validateAction(cmd, APP_OPTIONS.ACTIONS.GENERATE_RESOURCE_INFO_MODELS);
try {
DataDictionaryCodeGenerator generator = new DataDictionaryCodeGenerator(new ResourceInfoProcessor());
generator.processWorksheets();
} catch (Exception ex) {
LOG.error(getDefaultErrorMessage(ex));
}
} else if (cmd.hasOption(APP_OPTIONS.ACTIONS.GENERATE_REFERENCE_EDMX)) { } else if (cmd.hasOption(APP_OPTIONS.ACTIONS.GENERATE_REFERENCE_EDMX)) {
APP_OPTIONS.validateAction(cmd, APP_OPTIONS.ACTIONS.GENERATE_REFERENCE_EDMX); APP_OPTIONS.validateAction(cmd, APP_OPTIONS.ACTIONS.GENERATE_REFERENCE_EDMX);
try { try {
@ -373,10 +378,10 @@ public class App {
private static class APP_OPTIONS { private static class APP_OPTIONS {
//parameter names //parameter names
static final String SERVICE_ROOT = "serviceRoot"; static final String SERVICE_ROOT = "serviceRoot";
static final String BEARER_TOKEN = "bearerToken"; static final String BEARER_TOKEN = "bearerToken";
static final String CLIENT_ID = "clientId"; static final String CLIENT_ID = "clientId";
static final String CLIENT_SECRET = "clientSecret"; static final String CLIENT_SECRET = "clientSecret";
static final String INPUT_FILE = "inputFile"; static final String INPUT_FILE = "inputFile";
static final String OUTPUT_FILE = "outputFile"; static final String OUTPUT_FILE = "outputFile";
static final String URI = "uri"; static final String URI = "uri";
@ -517,6 +522,8 @@ public class App {
.desc("Runs commands in RESOScript file given as <inputFile>.").build()) .desc("Runs commands in RESOScript file given as <inputFile>.").build())
.addOption(Option.builder().argName("t").longOpt(ACTIONS.GENERATE_DD_ACCEPTANCE_TESTS) .addOption(Option.builder().argName("t").longOpt(ACTIONS.GENERATE_DD_ACCEPTANCE_TESTS)
.desc("Generates acceptance tests in the current directory.").build()) .desc("Generates acceptance tests in the current directory.").build())
.addOption(Option.builder().argName("i").longOpt(ACTIONS.GENERATE_RESOURCE_INFO_MODELS)
.desc("Generates Java Models for the Web API Reference Server in the current directory.").build())
.addOption(Option.builder().argName("r").longOpt(ACTIONS.GENERATE_REFERENCE_EDMX) .addOption(Option.builder().argName("r").longOpt(ACTIONS.GENERATE_REFERENCE_EDMX)
.desc("Generates reference metadata in EDMX format.").build()) .desc("Generates reference metadata in EDMX format.").build())
.addOption(Option.builder().argName("k").longOpt(ACTIONS.GENERATE_REFERENCE_DDL) .addOption(Option.builder().argName("k").longOpt(ACTIONS.GENERATE_REFERENCE_DDL)
@ -559,6 +566,7 @@ public class App {
public static final String VALIDATE_METADATA = "validateMetadata"; public static final String VALIDATE_METADATA = "validateMetadata";
public static final String SAVE_GET_REQUEST = "saveGetRequest"; public static final String SAVE_GET_REQUEST = "saveGetRequest";
public static final String GENERATE_METADATA_REPORT = "generateMetadataReport"; public static final String GENERATE_METADATA_REPORT = "generateMetadataReport";
public static final String GENERATE_RESOURCE_INFO_MODELS = "generateResourceInfoModels";
} }
} }
} }