iSharkFly-Open
/
VistA-fmts
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

some load and triples insertion tests for Fileman file design F2N

This commit is contained in:
george 2011-11-12 23:18:44 +00:00
parent 828a100869
commit c543ee1a1f
3 changed files with 336 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

103
docs/F2N-benchmarks.txt Normal file
View File

@ -0,0 +1,103 @@
GTM>D IMPORT^C0XMAIN("smart-rdf-in/collins-frank.rdf")
STARTED: 3111104.130509
READING IN: smart-rdf-in/collins-frank.rdf
200 LINES READ
ADDED: _:G072744409 _S:795646155 fmts:rdfSource _TXT_INCOMING_RDF_FILE_/home/geo
rge/fmts/trunk/samples/smart-rdf-in/collins-frank_rdf_585185215
128 XML NODES PARSED
INSERTING 99 TRIPLES
ENDED AT: 3111104.13051
ELAPSED TIME: 1 SECONDS
APPROXIMATELY 99 TRIPLES PER SECOND
GTM>D IMPORT^C0XMAIN("smart-rdf-in/reed-richard.rdf")
STARTED: 3111104.130606
READING IN: smart-rdf-in/reed-richard.rdf
722 LINES READ
ADDED: _:G758268243 _S:177410100 fmts:rdfSource _TXT_INCOMING_RDF_FILE_/home/geo
rge/fmts/trunk/samples/smart-rdf-in/reed-richard_rdf_213828749
462 XML NODES PARSED
INSERTING 339 TRIPLES
ENDED AT: 3111104.13061
ELAPSED TIME: 4 SECONDS
APPROXIMATELY 84 TRIPLES PER SECOND
GTM>D IMPORT^C0XMAIN("smart-rdf-in/cole-susan.rdf")
STARTED: 3111104.130628
READING IN: smart-rdf-in/cole-susan.rdf
3428 LINES READ
ADDED: _:G271187746 _S:899679576 fmts:rdfSource _TXT_INCOMING_RDF_FILE_/home/geo
rge/fmts/trunk/samples/smart-rdf-in/cole-susan_rdf_538236597
2101 XML NODES PARSED
SKIPPING NODE: 9
SKIPPING NODE: 26
SKIPPING NODE: 29
SKIPPING NODE: 32
SKIPPING NODE: 35
INSERTING 1425 TRIPLES
ENDED AT: 3111104.130645
ELAPSED TIME: 17 SECONDS
APPROXIMATELY 83 TRIPLES PER SECOND
GTM>D IMPORT^C0XMAIN("smart-rdf-in/ford-shirley.rdf")
STARTED: 3111104.130703
READING IN: smart-rdf-in/ford-shirley.rdf
8922 LINES READ
ADDED: _:G740421472 _S:922849860 fmts:rdfSource _TXT_INCOMING_RDF_FILE_/home/geo
rge/fmts/trunk/samples/smart-rdf-in/ford-shirley_rdf_809878775
5470 XML NODES PARSED
ERROR, NO OBJECT FOUND FOR NODE: 4217
ERROR, NO OBJECT FOUND FOR NODE: 4226
ERROR, NO OBJECT FOUND FOR NODE: 4232
ERROR, NO OBJECT FOUND FOR NODE: 4258
ERROR, NO OBJECT FOUND FOR NODE: 4267
ERROR, NO OBJECT FOUND FOR NODE: 4273
INSERTING 3745 TRIPLES
ENDED AT: 3111104.130756
ELAPSED TIME: 53 SECONDS
APPROXIMATELY 70 TRIPLES PER SECOND
GTM>D IMPORT^C0XMAIN("smart-rdf-in/gracia-paul.rdf")
STARTED: 3111104.130817
READING IN: smart-rdf-in/gracia-paul.rdf
10698 LINES READ
ADDED: _:G289354757 _S:026395070 fmts:rdfSource _TXT_INCOMING_RDF_FILE_/home/geo
rge/fmts/trunk/samples/smart-rdf-in/gracia-paul_rdf_297957633
6571 XML NODES PARSED
ERROR, NO OBJECT FOUND FOR NODE: 2751
ERROR, NO OBJECT FOUND FOR NODE: 2760
ERROR, NO OBJECT FOUND FOR NODE: 2766
ERROR, NO OBJECT FOUND FOR NODE: 6329
INSERTING 4512 TRIPLES
ENDED AT: 3111104.130928
ELAPSED TIME: 71 SECONDS
APPROXIMATELY 63 TRIPLES PER SECOND
GTM>D IMPORT^C0XF2N("qds/qds.rdf")
STARTED: 3111104.152012
READING IN: qds/qds.rdf
73528 LINES READ
ADDED: _:G951670203 _S:805831840 fmts:rdfSource _TXT_INCOMING_RDF_FILE_/home/ge
o
rge/fmts/trunk/samples/qds/qds_rdf_394416761
71530 XML NODES PARSED
PARSE COMPLETE AT 3111104.152023
ELAPSED TIME: 8 SECONDS
APPROXIMATELY 8941 NODES PER SECOND
TRIPLES COMPLETE AT 3111104.153436
ELAPSED TIME: 853 SECONDS
APPROXIMATELY 81 TRIPLES PER SECOND <------ this calculation is wrong, because "batches" of 10000 nodes at a time were processed mixed in with the dom node->triple processing
INSERTING 69537 TRIPLES
INSERTION COMPLETE AT 3111104.152023
ELAPSED TIME: 127 SECONDS
APPROXIMATELY 547 NODES PER SECOND <------ I only wish that this were true, but alas it is based on the time to process the last "batch" divided by the total number of triples
ENDED AT: 3111104.153643
ELAPSED TIME: 991 SECONDS
APPROXIMATELY 70 TRIPLES PER SECOND <----- this "overall" rate is the one to watch...

95
docs/F2N-gpl-benchmark.txt Normal file
View File

@ -0,0 +1,95 @@
*************************
benchmark of F2N qds load on gpl.mdc-crew.net with GTM version 5.4-001_x8664 and MM turned off
****************************
GTM>D WGET^C0XF2N("https://trac.opensourcevista.net/svn/qrda/qds/qds.rdf")
STARTED: 3111104.140909
DOWNLOADING: https://trac.opensourcevista.net/svn/qrda/qds/qds.rdf
73944 LINES READ
DOWNLOAD COMPLETE AT 3111104.140936
ELAPSED TIME: 27 SECONDS
APPROXIMATELY 2738 LINES PER SEC
ADDED: _:G388521152 _S:326561669 fmts:rdfSource _TXT_INCOMING_RDF_FILE_https://trac.opensourcevista.net/svn/qrda/qds/qds.rdf_617430774
71530 XML NODES PARSED
PARSE COMPLETE AT 3111104.14102
ELAPSED TIME: 34 SECONDS
APPROXIMATELY 2103 NODES PER SECOND
TRIPLES COMPLETE AT 3111104.143302
ELAPSED TIME: 1362 SECONDS
APPROXIMATELY 51 TRIPLES PER SECOND <--- bogus
INSERTING 69537 TRIPLES
INSERTION COMPLETE AT 3111104.14102
ELAPSED TIME: 248 SECONDS
APPROXIMATELY 280 NODES PER SECOND <--- bogus
ENDED AT: 3111104.14371
ELAPSED TIME: 1681 SECONDS
APPROXIMATELY 41 TRIPLES PER SECOND < --- correct overall rate
*************************
benchmark of F2N qds load on gpl.mdc-crew.net with GTM version 5.4-001_x8664 and MM turned on
****************************
GTM>D WGET^C0XF2N("https://trac.opensourcevista.net/svn/qrda/qds/qds.rdf")
STARTED: 3111104.14521
DOWNLOADING: https://trac.opensourcevista.net/svn/qrda/qds/qds.rdf
73944 LINES READ
DOWNLOAD COMPLETE AT 3111104.145231
ELAPSED TIME: 21 SECONDS
APPROXIMATELY 3521 LINES PER SEC
ADDED: _:G356052990 _S:119674668 fmts:rdfSource _TXT_INCOMING_RDF_FILE_https://t
rac.opensourcevista.net/svn/qrda/qds/qds.rdf_073921057
71530 XML NODES PARSED
PARSE COMPLETE AT 3111104.1453
ELAPSED TIME: 21 SECONDS
APPROXIMATELY 3406 NODES PER SECOND
TRIPLES COMPLETE AT 3111104.151902
ELAPSED TIME: 1562 SECONDS
APPROXIMATELY 44 TRIPLES PER SECOND <-- bogus
INSERTING 69537 TRIPLES
INSERTION COMPLETE AT 3111104.1453
ELAPSED TIME: 223 SECONDS
APPROXIMATELY 311 NODES PER SECOND <-- bogus
ENDED AT: 3111104.152245
ELAPSED TIME: 1835 SECONDS
APPROXIMATELY 37 TRIPLES PER SECOND <-- correct overall rate
*************************
benchmark of F2N qds load on gpl.mdc-crew.net with GTM version 5.4-002B_x8664 and MM turned on
****************************
GTM>D WGET^C0XF2N("https://trac.opensourcevista.net/svn/fmts/trunk/samples/qds/q
ds.rdf")
STARTED: 3111104.170957
DOWNLOADING: https://trac.opensourcevista.net/svn/fmts/trunk/samples/qds/qds.rdf
73944 LINES READ
DOWNLOAD COMPLETE AT 3111104.171013
ELAPSED TIME: 16 SECONDS
APPROXIMATELY 4621 LINES PER SEC
ADDED: _:G310877834 _S:868017911 fmts:rdfSource _TXT_INCOMING_RDF_FILE_https://t
rac.opensourcevista.net/svn/fmts/trunk/samples/qds/qds.rdf_284111764
71530 XML NODES PARSED
PARSE COMPLETE AT 3111104.171039
ELAPSED TIME: 17 SECONDS
APPROXIMATELY 4207 NODES PER SECOND
TRIPLES COMPLETE AT 3111104.173038
ELAPSED TIME: 1199 SECONDS
APPROXIMATELY 57 TRIPLES PER SECOND
INSERTING 69537 TRIPLES
INSERTION COMPLETE AT 3111104.171039
ELAPSED TIME: 192 SECONDS
APPROXIMATELY 362 NODES PER SECOND
ENDED AT: 3111104.17335
APPROXIMATELY 362 NODES PER SECOND
ENDED AT: 3111104.17335
ELAPSED TIME: 1433 SECONDS
APPROXIMATELY 48 TRIPLES PER SECOND <== this is the interesting number, up from 37 tps

138
docs/F2N-raven-benchmarks.txt Normal file
View File

@ -0,0 +1,138 @@
GTM>D IMPORT^C0XMAIN("smart-rdf-in/collins-frank.rdf")
STARTED: 3111104.130509
READING IN: smart-rdf-in/collins-frank.rdf
200 LINES READ
ADDED: _:G072744409 _S:795646155 fmts:rdfSource _TXT_INCOMING_RDF_FILE_/home/geo
rge/fmts/trunk/samples/smart-rdf-in/collins-frank_rdf_585185215
128 XML NODES PARSED
INSERTING 99 TRIPLES
ENDED AT: 3111104.13051
ELAPSED TIME: 1 SECONDS
APPROXIMATELY 99 TRIPLES PER SECOND
GTM>D IMPORT^C0XMAIN("smart-rdf-in/reed-richard.rdf")
STARTED: 3111104.130606
READING IN: smart-rdf-in/reed-richard.rdf
722 LINES READ
ADDED: _:G758268243 _S:177410100 fmts:rdfSource _TXT_INCOMING_RDF_FILE_/home/geo
rge/fmts/trunk/samples/smart-rdf-in/reed-richard_rdf_213828749
462 XML NODES PARSED
INSERTING 339 TRIPLES
ENDED AT: 3111104.13061
ELAPSED TIME: 4 SECONDS
APPROXIMATELY 84 TRIPLES PER SECOND
GTM>D IMPORT^C0XMAIN("smart-rdf-in/cole-susan.rdf")
STARTED: 3111104.130628
READING IN: smart-rdf-in/cole-susan.rdf
3428 LINES READ
ADDED: _:G271187746 _S:899679576 fmts:rdfSource _TXT_INCOMING_RDF_FILE_/home/geo
rge/fmts/trunk/samples/smart-rdf-in/cole-susan_rdf_538236597
2101 XML NODES PARSED
SKIPPING NODE: 9
SKIPPING NODE: 26
SKIPPING NODE: 29
SKIPPING NODE: 32
SKIPPING NODE: 35
INSERTING 1425 TRIPLES
ENDED AT: 3111104.130645
ELAPSED TIME: 17 SECONDS
APPROXIMATELY 83 TRIPLES PER SECOND
GTM>D IMPORT^C0XMAIN("smart-rdf-in/ford-shirley.rdf")
STARTED: 3111104.130703
READING IN: smart-rdf-in/ford-shirley.rdf
8922 LINES READ
ADDED: _:G740421472 _S:922849860 fmts:rdfSource _TXT_INCOMING_RDF_FILE_/home/geo
rge/fmts/trunk/samples/smart-rdf-in/ford-shirley_rdf_809878775
5470 XML NODES PARSED
ERROR, NO OBJECT FOUND FOR NODE: 4217
ERROR, NO OBJECT FOUND FOR NODE: 4226
ERROR, NO OBJECT FOUND FOR NODE: 4232
ERROR, NO OBJECT FOUND FOR NODE: 4258
ERROR, NO OBJECT FOUND FOR NODE: 4267
ERROR, NO OBJECT FOUND FOR NODE: 4273
INSERTING 3745 TRIPLES
ENDED AT: 3111104.130756
ELAPSED TIME: 53 SECONDS
APPROXIMATELY 70 TRIPLES PER SECOND
GTM>D IMPORT^C0XMAIN("smart-rdf-in/gracia-paul.rdf")
STARTED: 3111104.130817
READING IN: smart-rdf-in/gracia-paul.rdf
10698 LINES READ
ADDED: _:G289354757 _S:026395070 fmts:rdfSource _TXT_INCOMING_RDF_FILE_/home/geo
rge/fmts/trunk/samples/smart-rdf-in/gracia-paul_rdf_297957633
6571 XML NODES PARSED
ERROR, NO OBJECT FOUND FOR NODE: 2751
ERROR, NO OBJECT FOUND FOR NODE: 2760
ERROR, NO OBJECT FOUND FOR NODE: 2766
ERROR, NO OBJECT FOUND FOR NODE: 6329
INSERTING 4512 TRIPLES
ENDED AT: 3111104.130928
ELAPSED TIME: 71 SECONDS
APPROXIMATELY 63 TRIPLES PER SECOND
*************************
benchmark of F2N qds load on raven with GT.M V5.4-002B Linux x86_64 and MM turned off
****************************
GTM>D IMPORT^C0XF2N("qds/qds.rdf")
STARTED: 3111104.152012
READING IN: qds/qds.rdf
73528 LINES READ
ADDED: _:G951670203 _S:805831840 fmts:rdfSource _TXT_INCOMING_RDF_FILE_/home/ge
o
rge/fmts/trunk/samples/qds/qds_rdf_394416761
71530 XML NODES PARSED
PARSE COMPLETE AT 3111104.152023
ELAPSED TIME: 8 SECONDS
APPROXIMATELY 8941 NODES PER SECOND
TRIPLES COMPLETE AT 3111104.153436
ELAPSED TIME: 853 SECONDS
APPROXIMATELY 81 TRIPLES PER SECOND <------ this calculation is wrong, because "batches" of 10000 nodes at a time were processed mixed in with the dom node->triple processing
INSERTING 69537 TRIPLES
INSERTION COMPLETE AT 3111104.152023
ELAPSED TIME: 127 SECONDS
APPROXIMATELY 547 NODES PER SECOND <------ I only wish that this were true, but alas it is based on the time to process the last "batch" divided by the total number of triples
ENDED AT: 3111104.153643
ELAPSED TIME: 991 SECONDS
APPROXIMATELY 70 TRIPLES PER SECOND <----- this "overall" rate is the one to watch...
*************************
benchmark of F2N qds load on raven with GT.M V5.4-002B Linux x86_64 and MM turned on
****************************
GTM>D IMPORT^C0XF2N("qds/qds.rdf")
STARTED: 3111104.175524
READING IN: qds/qds.rdf
73528 LINES READ
ADDED: _:G802962321 _S:387624835 fmts:rdfSource _TXT_INCOMING_RDF_FILE_/home/geo
rge/fmts/trunk/samples/qds/qds_rdf_405517303
71530 XML NODES PARSED
PARSE COMPLETE AT 3111104.175535
ELAPSED TIME: 8 SECONDS
APPROXIMATELY 8941 NODES PER SECOND
TRIPLES COMPLETE AT 3111104.180951
ELAPSED TIME: 856 SECONDS
APPROXIMATELY 81 TRIPLES PER SECOND
INSERTING 69537 TRIPLES
INSERTION COMPLETE AT 3111104.175535
ELAPSED TIME: 130 SECONDS
APPROXIMATELY 534 NODES PER SECOND
ENDED AT: 3111104.181201
ELAPSED TIME: 997 SECONDS
APPROXIMATELY 69 TRIPLES PER SECOND