Commit Graph

175 Commits

Author SHA1 Message Date
Joshua Darnell 958771d71c
126 revert cucumberio dependencies to 72x (#127)
* #126: Reverted dependencies

* #126: Build field map when metadata loaded from a file
2022-07-18 20:40:54 -07:00
Robert Gottesman 4a5f0333be
Create RESODataDictionary-1.7-flat.xlsx (#123)
Updated DD Wiki 1.7 flat spreadsheet
2022-07-11 22:40:30 -07:00
Robert Gottesman 9a5e75d04a
Update RESODataDictionary-2.0.xlsx (#122)
completed issue 121
2022-07-11 09:34:38 -07:00
Joshua Darnell fb384dfb49
#94 Added Testing Rules for the Lookup Resource (#108)
* Updated odata-openapi commit

* Update odata-openapi submodule

* Issue #94: Initial addition of Lookup resoruce tests

* Issue #94: added Cucumber BDD for Lookup Resource and stubs for gluecode

* Issue #94: Added Lookup Replication Tests

* Updated .gitignore

* Issue #94: Retrieve data using fetch api

* Issue #94: Added timestamp query replication strategy

* Issue #94: Added timestamp query replication strategy for  asc

* Issue #94: Add relevant annotation and metadata cache tests

* Issue #94: Added required annotation and lookup name testing in the lookup resource for standard fields

* Issue #94: WARN when empty LookupName items are found

* Issue #94: update to use TopAndSkip replication strategy instead

* Issue #94: Changed fail to warn when lookup name not present. Changed DataAvailability to use the in-container cache.

* Issue #94: separated the required annotation check from lookup name checks in the lookup data

* Issue #94: Lookup Resource Cleanup

* Issue #105: Add fast-fail behavior to all XML Metadata validation steps

* Added better page handling and logging for sampling

* Issue #94: move annotation reader to common utils

* Issue #94: added JSON Serializer

* Issue #94: Lookups JSON serializer

* Issue #94: Refactored utils into ODataUtils, separated functions

* Issue #94: Refactored ODataUtils to use Optional, and added comments

* Issue #94: further conversion to optional

* Issue #94: created new json serializers

* #94: Intermediate commit

* #94: updated sampling strategy

* #107: Added support for OriginatingSystemName and OriginatingSystemID

* Updated Docker instructions in the README

* #106: updated docker README

* Issue #94: annotated lookup field serializer

* Issue #94: adding parallel stream logic

* #94: merge main
2022-07-08 07:10:14 -07:00
Robert Gottesman ede2c1f405
Update RESODataDictionary-2.0.xlsx (#120)
Update contains new data generated by wikigen process
2022-07-05 16:26:54 -07:00
Joshua Darnell 55d823579b
#112: Added DD 2.0 Sheet. TODO: organize into folders (#118) 2022-06-30 10:53:15 -07:00
Joshua Darnell 68c2f1fbb7
#116 Improved Directory Handling and Upgraded Dependencies (#117)
* Issue #114: Add better Windows support for tags

* Issue #114: removing additional -Dtags arg

* Issue #116: added prepareEnvironment pre-build task

* Issue #116: upgraded dependencies
2022-06-24 07:35:06 -07:00
Joshua Darnell b5b274f48e
114 improve tags support in buildgradle (#115)
* Issue #114: Add better Windows support for tags

* Issue #114: removing additional -Dtags arg
2022-06-24 06:32:52 -07:00
Joshua Darnell 0c2c11e5e1
Update _config.yml 2022-06-06 22:55:51 -07:00
Joshua Darnell ce6d36dd7b
Create CNAME 2022-06-06 22:43:19 -07:00
Joshua Darnell c1d7f2ce64
Issue #101: bump log4j version (#103)
* Issue #101: added new version of odata-openapi submodule
2021-12-30 17:59:46 -08:00
Joshua Darnell 4185abe338
Issue #101: Update Log4J (#102) 2021-12-30 15:16:21 -08:00
Joshua Darnell e33ff98a61
Issue #96: Migrate Node Utils to Separate Repo (#98)
* Issue #96: Migrate Node Utils to Separate Repo
* Updated gradle dependencies, retested builds
2021-12-20 16:46:19 -08:00
Joshua Darnell 9181ef192d
Support for Field and Lookup Resources (#95)
* Issue #85: bump Web API Core version to 2.0.0
* Added strict flag=true by default for metadata and availability checks. JSON Schema WIP.
* Issue #88: added batch test runner as well as improved Web API Messages
* Issue #91: added field availability metrics
* Issue #91: availability and field count rollups
* Issue #91: calculate lookup value availabilities
* Issue #91: lookup rollups
* Issue #91: added lookup availability
* Issue #91: resource availability calculations
* Issue #91: adding comments and addressing null values
* Issue #91: cleanup lookup iterations
* Issue #88: added XMLMetadataToJSONSchemaSerializer
* Issue #93: Added Field and Lookup resource
* Issue #93: Updated README
2021-12-12 21:22:42 -08:00
Joshua Darnell 6ff3562792 Set theme jekyll-theme-slate 2021-07-22 05:14:22 -07:00
Joshua Darnell 6162c360e2 Set theme jekyll-theme-cayman 2021-07-22 05:11:10 -07:00
Joshua Darnell 856c7bc037
Issue #82: refactored availability report to use frequencies and track lookups at a granular level (#83) 2021-07-22 05:05:34 -07:00
Joshua Darnell 5cf84f3564 Set theme jekyll-theme-modernist 2021-07-19 21:06:41 -07:00
Joshua Darnell 856e326773 Set theme jekyll-theme-slate 2021-07-19 21:04:52 -07:00
Joshua Darnell c2e0ef5e4c Set theme jekyll-theme-hacker 2021-07-19 21:04:35 -07:00
Joshua Darnell c6457d5329
Updated configurations and removed Cucumber Reports (#81)
- Removed all Cucumber Reporting steps and code, updated README to reflect the removal.
- Moved all output files to `build/certification` within the project directory and put them under results, reports, and cucumberJson (for downstream Cucumber Reporting plugins, such as that in Jenkins).
- Addded GradleDocker config file so containers may be built that specifically run Docker tasks.
- Added GitHub Workflow to add a GradleDocker image to the GitHub Docker registry on commit to main.
- Reorganized and updated README.
- removed additional --strict flags in Cucumber JVM (since they're deprecated).
- Fixed issue with synonym checker not running in when `-Dstrict=true` is passed.
- Separated payloads and IDX backing glue code.
2021-07-19 07:47:12 -07:00
Joshua Darnell 8c68e140c9
Payloads Sampling Tool (#75)
* Sampling with SHA 256 scoring
* Optional multi-threaded fetching and use of timezone offsets
* Data availability reports (availability-report.json in the /build directory)
* Disabled logging, changed payload sampling logic, encoded files for payload fields only, and added standard cookie spec headers to fix warnings
* Cleaned up logging and build.gradle
* Added line to quiet velocity logs
2021-07-13 16:25:03 -07:00
Joshua Darnell d6a2ce3649
Update README.md
Added updated info about OpenJDK.
2021-06-02 15:34:47 -07:00
Joshua Darnell 19f72345b1
Make Report Generator Friendlier with Jenkins and Build Processes (#72)
* Added file generation to DD Metadata tests
* Updated most libraries and build.gradle accordingly
* Fixed annotation processor so it read Term instead of Type
2021-04-13 16:57:10 -07:00
Joshua Darnell b59ab5b4ec
issue-60-null-timestamp-handling (#70)
Going to merge since this is a null-check issue. Has passed testing.
2021-04-05 17:13:41 -07:00
Joshua Darnell e5549c84d4
Update gradle.yml (#71) 2021-04-05 17:05:00 -07:00
michaelpede ef10501763
Update DDLProcessor.java (#69) 2021-04-01 18:55:34 -07:00
Joshua Darnell 4ea56248f0
Add jenkins dd gradle task (#66)
* Added minimal DD build task for Jenkins environments
* Added Web API Core Jenkins Job
* Updated Olingo client and Guice
* Updated build.gradle
* Issue #55 - added =true test and reorganized tests, as well as generated new reference metadata and other artifacts
* Issue #52 - add better error messages for fetch-by-key test
* Fixed unreported issue with generateDDAcceptanceTests not working from the command line
* Improved metadata reports and updated reference metadata
* Improvements to Metadata and Metadata Report Generation. Added JSON schema for report format
* Issue #61 - added DDL generation to the RESO Commander for Reference Server
* Issue #61 - updated template .sql DDL files in /src/main/resources/ddl
* Issue #61 - added Lookup resource Inserts to DDL
* Issue #61 - fixed formatting issues
* Issue #51 - Improved Jenkins Builds in the Commander
* Fix failing tests with the mock Add/Edit spec
* Clean up tags for tests
* Issue #51 - improved Jenkin behavior, and added the ability to pass a set of reference queries
* Issue #51 - cleaned up constants
* Issue #51 - improved Jenkins Builds. Added Scenario.Write messages for DD 1.7 tests.
* Issue #51 - added better support for error handling in the testing scenarios, as well as fast failure on DD testing
2021-03-31 19:24:27 -07:00
Joshua Darnell fc556b39c0
Added Images for DD IntelliJ Walkthrough (#54) 2021-02-12 21:34:05 -08:00
Joshua Darnell 0aa39b0c6d
Issue 47 allow string lookups (#49)
Issue #47 - added tests for Collection based any and all tests
2021-01-08 04:09:19 -08:00
Joshua Darnell 30c346cd2e
Issue #47 - allow Edm.String and Collection(Edm.String) for Single and Multi lookups (#48)
Issue #47 - allow Edm.String and Collection(Edm.String) for single and multi lookups
2021-01-07 23:02:56 -08:00
Joshua Darnell 26dedd0031
Issue #45 - fix handling of all() so that result must contain no items or exactly contain the asserted value (#46) 2021-01-07 10:12:10 -08:00
Joshua Darnell bf1c55c818
Issue 34 Web API Core Tests (#43)
* Added DD Spreadsheet, Command Line option, and parser
* Initial check-in of DD generator and generated .feature files for 1.7 resources
* Added file naming and header info to the generated files, regenerated Gluecode
* Changed generator templates
* Added support for additional DD resources and updated generator
* Added new Commander build
* Changed templates
* Refactored generator and changed DD templates
* Refactored WorksheetProcessor into base class
* Added EDMX Generator
* Added dependency injection of test containers, changed from Java8 Lambda test expressions to functions
* Added skip logic and resource tags to generators
* Generated new tests and improved skipped test messaging
* Corrected Test Template
* Finished first round of DD tests: field presence
* Expanded field map to include all resources rather than just the one included in the Web API test
* DD test generator intermediate check-in
* EDMX Generator build enumerations generator
* EDMX generator can generate valid EDMX files. See: /src/main/resources
* Re-generated EDMX and BDD Tests from sanitized spreadsheet
* Issue #30 - Convert Web API Server tests to Core, and remove Date Part comparisons
* Updated DD BDD test format
* Wrapped up test templates and generated new BDD acceptance tests
* Cleaned up EDMX generation
* Wrapped up first round of EDMX generation using Key (but not KeyNumeric). Metadata validation passes on generated file
* Updated EDMX generator with Single and Multiple Enumerations
* Updated reference XSLX, generated new EDMX and BDD Tests
* Added updated BDD generators with improved tags and addedcomments with descriptions in the generated EDMX
* Added generation of comments for both Standard Fields and Standard Enumerations, and also added a StandardEnumeration class and builder
* Updated reference EDMX and BDD test generators to use Collections of enumerations rather than multi-select enum types. Added newly generated EDMX and BDD tests
* Added the ability to generate Standard Relationships into NavigationProperty definitions
* Added updated reference EDMX document
* Updated Standard Relationships and re-generated EDMX and Open API definitions
* Removed Partner Property Definitions and extra info logging
* Issue #34 - Rename Test IDs
* Issue #34 - Added query generator and additional RESOScript queries
* Issue #34 - Added remaining Core test queries to RESOScript files and cleaned up test IDs
* Added Data Dictionary tests for Collections, TODO: README and non-collection based enumerations
* Issue #37 - added support for multiple enumerations that use IsFlags=true
* Issue #37 - added better reporting
* Issue #37 - Allow Edm.Double in addition to Edm.Decimal as per Cert group 10/1/2020 call
* Issue #37 - added Gradle task to generate minimal 'pretty' reports.
* Issue #37 - added updated .JAR and updated build.gradle
* Issue #37 - Added additional error information to test results
* Issue #37 - Added BDD generation for Lookups and stubs for testing. Prepended 'sample' to the RESOScript templates
* Issue #37 - committing new BDD tests
* Issue #37 - Added enumeration MAY and MUST support
* Issue #37 - Added field-level synonyms checking and fixed invalid synonyms in DD sheet
* Issue #37 - added test to check that closed enumerations only have standard values
* Issue #37 - added better logging, including more info about standard vs. non-standard enumerations
* Issue #37 - further improvement of OData type error messages
* Issue #37 - Added fuzzy matching of enumerations using Levenshtein distance
* Issue #37 - Improved Edit Distance behavior, added IgnoredItem and ignored.json, and updated Vha, Usda, and Fha with VHA, USDA, and FHA
* Issue #37 - Added better error messages, --strict=true for strict testing mode, and metadata-report.json to output results to aggregation server
* Issue #37 - Added auto-generating output directory as well as a generateCertificationReports_DD_1_7 task that takes -DpathToJsonResults from the cert task and creates a pretty report
* Issue #37 - Added better error message formatting and handling, and help messages for gradle tasks
* Issue #37 - updated README and fixed some of the existing Web API Commander JAR tasks, as well as improved the all-in-one generateCertificationReport task
* Issue #37 - Cleaned up some items in the README and message output
* Update README.md
* Issue #34 - fixed metadata-request name in RESOScript
* Issue #34 - Added logging and container caching to Web API acceptance tests, and updated sample payloads
* Issue #34 - Cleaned up DD merge, Added Collections flag, updated README
* Issue #34 - Added ODataTransportWrapper and odata-openapi as a submodule
* Issue #37 - removed Data Dictionary lookup tests and added annotations to reference metadata
* Issue #37 - added new DD 1.7 Open API 3 spec
2021-01-06 00:09:01 -08:00
Joshua Darnell e6e1af4b5d
Issue 37 Add Data Dictionary testing to the RESO Commmander (#41)
* Added DD Spreadsheet, Command Line option, and parser
* Initial check-in of DD generator and generated .feature files for 1.7 resources
* Added file naming and header info to the generated files, regenerated Gluecode
* Changed generator templates
* Added support for additional DD resources and updated generator
* Added new Commander build
* Changed templates
* Refactored generator and changed DD templates
* Refactored WorksheetProcessor into base class
* Added EDMX Generator
* Added better EDMX generation
* Intermediate check-in for EDMX processor
* Added dependency injection of test containers, changed from Java8 Lambda test expressions to functions
* Added skip logic and resource tags to generators
* Generated new tests and improved skipped test messaging
* Corrected Test Template
* Finished first round of DD tests: field presence
* Expanded field map to include all resources rather than just the one included in the Web API test
* DD test generator intermediate check-in
* EDMX Generator build enumerations generator
* EDMX generator can generate valid EDMX files. See: /src/main/resources
* Re-generated EDMX and BDD Tests from sanitized spreadsheet
* Issue #30 - Convert Web API Server tests to Core, and remove Date Part comparisons
* Updated DD BDD test format
* Wrapped up test templates and generated new BDD acceptance tests
* Cleaned up EDMX generation
* Wrapped up first round of EDMX generation using Key (but not KeyNumeric). Metadata validation passes on generated file
* Updated EDMX generator with Single and Multiple Enumerations
* Updated reference XSLX, generated new EDMX and BDD Tests
* Added updated BDD generators with improved tags and addedcomments with descriptions in the generated EDMX
* Added generation of comments for both Standard Fields and Standard Enumerations, and also added a StandardEnumeration class and builder
* Updated reference EDMX and BDD test generators to use Collections of enumerations rather than multi-select enum types. Added newly generated EDMX and BDD tests
* Added the ability to generate Standard Relationships into NavigationProperty definitions
* Added updated reference EDMX document
* Updated Standard Relationships and re-generated EDMX and Open API definitions
* Removed Partner Property Definitions and extra info logging
* Added Data Dictionary tests for Collections, TODO: README and non-collection based enumerations
* Issue #37 - added support for multiple enumerations that use IsFlags=true
* Issue #37 - added better reporting
* Issue #37 - Allow Edm.Double in addition to Edm.Decimal as per Cert group 10/1/2020 call
* Issue #37 - added Gradle task to generate minimal 'pretty' reports.
* Issue #37 - added updated .JAR and updated build.gradle
* Issue #37 - Added additional error information to test results
* Issue #37 - Added BDD generation for Lookups and stubs for testing. Prepended 'sample' to the RESOScript templates
* Issue #37 - committing new BDD tests
* Issue #37 - Added enumeration MAY and MUST support
* Issue #37 - Added field-level synonyms checking and fixed invalid synonyms in DD sheet
* Issue #37 - added test to check that closed enumerations only have standard values
* Issue #37 - added better logging, including more info about standard vs. non-standard enumerations
* Issue #37 - further improvement of OData type error messages
* Issue #37 - Added fuzzy matching of enumerations using Levenshtein distance
* Issue #37 - Improved Edit Distance behavior, added IgnoredItem and ignored.json, and updated Vha, Usda, and Fha with VHA, USDA, and FHA
* Issue #37 - Added better error messages, --strict=true for strict testing mode, and metadata-report.json to output results to aggregation server
* Issue #37 - Added auto-generating output directory as well as a generateCertificationReports_DD_1_7 task that takes -DpathToJsonResults from the cert task and creates a pretty report
* Issue #37 - Added better error message formatting and handling, and help messages for gradle tasks
* Issue #37 - updated README and fixed some of the existing Web API Commander JAR tasks, as well as improved the all-in-one generateCertificationReport task
* Issue #37 - Cleaned up some items in the README and message output
* Update README.md
2021-01-05 23:47:04 -08:00
Joshua Darnell 017348cf1b
Create codeql-analysis.yml 2020-10-14 10:32:36 -07:00
Joshua Darnell 31d64ad132 Update issue templates 2020-09-29 23:34:10 -07:00
Joshua Darnell 7428d7142c
Issue #30 - Convert Web API Server tests to Core, and remove Date Par… (#31)
Issue #30 - Convert Web API Server tests to Core, and remove Date Part comparisons
2020-06-16 10:19:48 -07:00
Joshua Darnell a2843cad82
Issue #25 - Increase Fractional Seconds Precision (#28)
* Added improved fractionalsecond Commander acceptance tests, need to backfill into Web API server cert
* Added Date tests
* Refactored fractionalseconds tests to accept greater precision, and refactored Web API Server tests to use new payload comparison helpers. Also added more tests.
* Changed private access of Timestamp compare functions to public so they could be used from other test functions
* Updated if/then statetments to use switch() instead
* Generated new Commander Jar
2020-05-07 17:44:15 -07:00
Joshua Darnell fa117b19fd
Fixed incomplete client settings tags in resoscript templates (#22)
Fixes for platinum, gold, and mock platinum RESOScripts
2020-05-01 09:19:07 -07:00
Joshua Darnell cd3fd1ab84
Set encoding to UTF-8 and Fix Comparison Tests (#27)
* Fixed issue with UTF-8 encodings not being picked up from the Windows console
* Additional fixes to comparison tests
* Added tests for Integer comparisons
* Added String comparison tests
* Added timestamp comparison tests
* Renamed test files to match conventions
* Added Fractional Date Part tests
* Generated automated acceptance tests for remainder of compare() operations in TestUtils
* Checked in new Commander build
2020-04-27 20:23:49 -07:00
dgmyrek 59c409d37d
Merge pull request #20 from RESOStandards/issue-19-fail-all-if-metadata-fails
Issue #19 fail all if metadata fails
2020-04-16 13:48:02 -05:00
Joshua Darnell dfb1566b0e Fixed additional issue where XML Metadata needed to be fetched for individual test debugging 2020-04-13 04:45:51 -07:00
Joshua Darnell 66097735a4 Fixed small bug in validate test workflow 2020-04-13 03:28:58 -07:00
Joshua Darnell 3c96e39a7b Removed stray line from workflow YAML 2020-04-13 03:02:31 -07:00
Joshua Darnell ac0254d337 GitHub workflow - removed post-build artifact and added gradle caching 2020-04-13 02:59:15 -07:00
Joshua Darnell 1a8346cc1f Fixed YAML syntax error (v2) 2020-04-13 02:48:03 -07:00
Joshua Darnell d069c8b30c Fixed YAML syntax error 2020-04-13 02:41:59 -07:00
Joshua Darnell ce6eeb0cf0 Added DataSystem good/bad container tests, removed rest of Edm fetch logic, added build artifacts to github workflows to build commander lib, and improved state variables in the Web API Test Container 2020-04-13 02:40:22 -07:00
Joshua Darnell 5274dd8557 Cleaned up metadata checking and removed extra Edm metadata fetch, favoring deserialization from the XML Response payload instead 2020-04-12 15:19:27 -07:00
Joshua Darnell 4da68de5c3 Added additional WebAPIContainer tests, formatted code, started to clean up test/testContainer logic 2020-04-12 13:37:18 -07:00