From c543ee1a1f5bc76974cfe600558261fadc452096 Mon Sep 17 00:00:00 2001 From: george Date: Sat, 12 Nov 2011 23:18:44 +0000 Subject: [PATCH] some load and triples insertion tests for Fileman file design F2N --- docs/F2N-benchmarks.txt | 103 +++++++++++++++++++++++++ docs/F2N-gpl-benchmark.txt | 95 +++++++++++++++++++++++ docs/F2N-raven-benchmarks.txt | 138 ++++++++++++++++++++++++++++++++++ 3 files changed, 336 insertions(+) create mode 100644 docs/F2N-benchmarks.txt create mode 100644 docs/F2N-gpl-benchmark.txt create mode 100644 docs/F2N-raven-benchmarks.txt diff --git a/docs/F2N-benchmarks.txt b/docs/F2N-benchmarks.txt new file mode 100644 index 0000000..a0f4c36 --- /dev/null +++ b/docs/F2N-benchmarks.txt @@ -0,0 +1,103 @@ +GTM>D IMPORT^C0XMAIN("smart-rdf-in/collins-frank.rdf") + +STARTED: 3111104.130509 +READING IN: smart-rdf-in/collins-frank.rdf +200 LINES READ +ADDED: _:G072744409 _S:795646155 fmts:rdfSource _TXT_INCOMING_RDF_FILE_/home/geo +rge/fmts/trunk/samples/smart-rdf-in/collins-frank_rdf_585185215 +128 XML NODES PARSED +INSERTING 99 TRIPLES + ENDED AT: 3111104.13051 + ELAPSED TIME: 1 SECONDS + APPROXIMATELY 99 TRIPLES PER SECOND + +GTM>D IMPORT^C0XMAIN("smart-rdf-in/reed-richard.rdf") + +STARTED: 3111104.130606 +READING IN: smart-rdf-in/reed-richard.rdf +722 LINES READ +ADDED: _:G758268243 _S:177410100 fmts:rdfSource _TXT_INCOMING_RDF_FILE_/home/geo +rge/fmts/trunk/samples/smart-rdf-in/reed-richard_rdf_213828749 +462 XML NODES PARSED +INSERTING 339 TRIPLES + ENDED AT: 3111104.13061 + ELAPSED TIME: 4 SECONDS + APPROXIMATELY 84 TRIPLES PER SECOND + +GTM>D IMPORT^C0XMAIN("smart-rdf-in/cole-susan.rdf") + +STARTED: 3111104.130628 +READING IN: smart-rdf-in/cole-susan.rdf +3428 LINES READ +ADDED: _:G271187746 _S:899679576 fmts:rdfSource _TXT_INCOMING_RDF_FILE_/home/geo +rge/fmts/trunk/samples/smart-rdf-in/cole-susan_rdf_538236597 +2101 XML NODES PARSED +SKIPPING NODE: 9 +SKIPPING NODE: 26 +SKIPPING NODE: 29 +SKIPPING NODE: 32 +SKIPPING NODE: 35 +INSERTING 1425 TRIPLES + ENDED AT: 3111104.130645 + ELAPSED TIME: 17 SECONDS + APPROXIMATELY 83 TRIPLES PER SECOND + +GTM>D IMPORT^C0XMAIN("smart-rdf-in/ford-shirley.rdf") + +STARTED: 3111104.130703 +READING IN: smart-rdf-in/ford-shirley.rdf +8922 LINES READ +ADDED: _:G740421472 _S:922849860 fmts:rdfSource _TXT_INCOMING_RDF_FILE_/home/geo +rge/fmts/trunk/samples/smart-rdf-in/ford-shirley_rdf_809878775 +5470 XML NODES PARSED +ERROR, NO OBJECT FOUND FOR NODE: 4217 +ERROR, NO OBJECT FOUND FOR NODE: 4226 +ERROR, NO OBJECT FOUND FOR NODE: 4232 +ERROR, NO OBJECT FOUND FOR NODE: 4258 +ERROR, NO OBJECT FOUND FOR NODE: 4267 +ERROR, NO OBJECT FOUND FOR NODE: 4273 +INSERTING 3745 TRIPLES + ENDED AT: 3111104.130756 + ELAPSED TIME: 53 SECONDS + APPROXIMATELY 70 TRIPLES PER SECOND + +GTM>D IMPORT^C0XMAIN("smart-rdf-in/gracia-paul.rdf") + +STARTED: 3111104.130817 +READING IN: smart-rdf-in/gracia-paul.rdf +10698 LINES READ +ADDED: _:G289354757 _S:026395070 fmts:rdfSource _TXT_INCOMING_RDF_FILE_/home/geo +rge/fmts/trunk/samples/smart-rdf-in/gracia-paul_rdf_297957633 +6571 XML NODES PARSED +ERROR, NO OBJECT FOUND FOR NODE: 2751 +ERROR, NO OBJECT FOUND FOR NODE: 2760 +ERROR, NO OBJECT FOUND FOR NODE: 2766 +ERROR, NO OBJECT FOUND FOR NODE: 6329 +INSERTING 4512 TRIPLES + ENDED AT: 3111104.130928 + ELAPSED TIME: 71 SECONDS + APPROXIMATELY 63 TRIPLES PER SECOND + +GTM>D IMPORT^C0XF2N("qds/qds.rdf") + +STARTED: 3111104.152012 +READING IN: qds/qds.rdf +73528 LINES READ +ADDED: _:G951670203 _S:805831840 fmts:rdfSource _TXT_INCOMING_RDF_FILE_/home/ge +o +rge/fmts/trunk/samples/qds/qds_rdf_394416761 +71530 XML NODES PARSED +PARSE COMPLETE AT 3111104.152023 + ELAPSED TIME: 8 SECONDS + APPROXIMATELY 8941 NODES PER SECOND +TRIPLES COMPLETE AT 3111104.153436 + ELAPSED TIME: 853 SECONDS + APPROXIMATELY 81 TRIPLES PER SECOND <------ this calculation is wrong, because "batches" of 10000 nodes at a time were processed mixed in with the dom node->triple processing +INSERTING 69537 TRIPLES +INSERTION COMPLETE AT 3111104.152023 + ELAPSED TIME: 127 SECONDS + APPROXIMATELY 547 NODES PER SECOND <------ I only wish that this were true, but alas it is based on the time to process the last "batch" divided by the total number of triples + ENDED AT: 3111104.153643 + ELAPSED TIME: 991 SECONDS + APPROXIMATELY 70 TRIPLES PER SECOND <----- this "overall" rate is the one to watch... + diff --git a/docs/F2N-gpl-benchmark.txt b/docs/F2N-gpl-benchmark.txt new file mode 100644 index 0000000..8c3f013 --- /dev/null +++ b/docs/F2N-gpl-benchmark.txt @@ -0,0 +1,95 @@ +************************* + +benchmark of F2N qds load on gpl.mdc-crew.net with GTM version 5.4-001_x8664 and MM turned off + +**************************** + + +GTM>D WGET^C0XF2N("https://trac.opensourcevista.net/svn/qrda/qds/qds.rdf") + +STARTED: 3111104.140909 +DOWNLOADING: https://trac.opensourcevista.net/svn/qrda/qds/qds.rdf +73944 LINES READ +DOWNLOAD COMPLETE AT 3111104.140936 + ELAPSED TIME: 27 SECONDS + APPROXIMATELY 2738 LINES PER SEC +ADDED: _:G388521152 _S:326561669 fmts:rdfSource _TXT_INCOMING_RDF_FILE_https://trac.opensourcevista.net/svn/qrda/qds/qds.rdf_617430774 +71530 XML NODES PARSED +PARSE COMPLETE AT 3111104.14102 + ELAPSED TIME: 34 SECONDS + APPROXIMATELY 2103 NODES PER SECOND +TRIPLES COMPLETE AT 3111104.143302 + ELAPSED TIME: 1362 SECONDS + APPROXIMATELY 51 TRIPLES PER SECOND <--- bogus +INSERTING 69537 TRIPLES +INSERTION COMPLETE AT 3111104.14102 + ELAPSED TIME: 248 SECONDS + APPROXIMATELY 280 NODES PER SECOND <--- bogus + ENDED AT: 3111104.14371 + ELAPSED TIME: 1681 SECONDS + APPROXIMATELY 41 TRIPLES PER SECOND < --- correct overall rate + +************************* + +benchmark of F2N qds load on gpl.mdc-crew.net with GTM version 5.4-001_x8664 and MM turned on + +**************************** + + +GTM>D WGET^C0XF2N("https://trac.opensourcevista.net/svn/qrda/qds/qds.rdf") + +STARTED: 3111104.14521 +DOWNLOADING: https://trac.opensourcevista.net/svn/qrda/qds/qds.rdf +73944 LINES READ +DOWNLOAD COMPLETE AT 3111104.145231 + ELAPSED TIME: 21 SECONDS + APPROXIMATELY 3521 LINES PER SEC +ADDED: _:G356052990 _S:119674668 fmts:rdfSource _TXT_INCOMING_RDF_FILE_https://t +rac.opensourcevista.net/svn/qrda/qds/qds.rdf_073921057 +71530 XML NODES PARSED +PARSE COMPLETE AT 3111104.1453 + ELAPSED TIME: 21 SECONDS + APPROXIMATELY 3406 NODES PER SECOND +TRIPLES COMPLETE AT 3111104.151902 + ELAPSED TIME: 1562 SECONDS + APPROXIMATELY 44 TRIPLES PER SECOND <-- bogus +INSERTING 69537 TRIPLES +INSERTION COMPLETE AT 3111104.1453 + ELAPSED TIME: 223 SECONDS + APPROXIMATELY 311 NODES PER SECOND <-- bogus + ENDED AT: 3111104.152245 + ELAPSED TIME: 1835 SECONDS + APPROXIMATELY 37 TRIPLES PER SECOND <-- correct overall rate + +************************* + +benchmark of F2N qds load on gpl.mdc-crew.net with GTM version 5.4-002B_x8664 and MM turned on + +**************************** +GTM>D WGET^C0XF2N("https://trac.opensourcevista.net/svn/fmts/trunk/samples/qds/q +ds.rdf") + +STARTED: 3111104.170957 +DOWNLOADING: https://trac.opensourcevista.net/svn/fmts/trunk/samples/qds/qds.rdf +73944 LINES READ +DOWNLOAD COMPLETE AT 3111104.171013 + ELAPSED TIME: 16 SECONDS + APPROXIMATELY 4621 LINES PER SEC +ADDED: _:G310877834 _S:868017911 fmts:rdfSource _TXT_INCOMING_RDF_FILE_https://t +rac.opensourcevista.net/svn/fmts/trunk/samples/qds/qds.rdf_284111764 +71530 XML NODES PARSED +PARSE COMPLETE AT 3111104.171039 + ELAPSED TIME: 17 SECONDS + APPROXIMATELY 4207 NODES PER SECOND +TRIPLES COMPLETE AT 3111104.173038 + ELAPSED TIME: 1199 SECONDS + APPROXIMATELY 57 TRIPLES PER SECOND +INSERTING 69537 TRIPLES +INSERTION COMPLETE AT 3111104.171039 + ELAPSED TIME: 192 SECONDS + APPROXIMATELY 362 NODES PER SECOND + ENDED AT: 3111104.17335 + APPROXIMATELY 362 NODES PER SECOND + ENDED AT: 3111104.17335 + ELAPSED TIME: 1433 SECONDS + APPROXIMATELY 48 TRIPLES PER SECOND <== this is the interesting number, up from 37 tps \ No newline at end of file diff --git a/docs/F2N-raven-benchmarks.txt b/docs/F2N-raven-benchmarks.txt new file mode 100644 index 0000000..365e7e0 --- /dev/null +++ b/docs/F2N-raven-benchmarks.txt @@ -0,0 +1,138 @@ +GTM>D IMPORT^C0XMAIN("smart-rdf-in/collins-frank.rdf") + +STARTED: 3111104.130509 +READING IN: smart-rdf-in/collins-frank.rdf +200 LINES READ +ADDED: _:G072744409 _S:795646155 fmts:rdfSource _TXT_INCOMING_RDF_FILE_/home/geo +rge/fmts/trunk/samples/smart-rdf-in/collins-frank_rdf_585185215 +128 XML NODES PARSED +INSERTING 99 TRIPLES + ENDED AT: 3111104.13051 + ELAPSED TIME: 1 SECONDS + APPROXIMATELY 99 TRIPLES PER SECOND + +GTM>D IMPORT^C0XMAIN("smart-rdf-in/reed-richard.rdf") + +STARTED: 3111104.130606 +READING IN: smart-rdf-in/reed-richard.rdf +722 LINES READ +ADDED: _:G758268243 _S:177410100 fmts:rdfSource _TXT_INCOMING_RDF_FILE_/home/geo +rge/fmts/trunk/samples/smart-rdf-in/reed-richard_rdf_213828749 +462 XML NODES PARSED +INSERTING 339 TRIPLES + ENDED AT: 3111104.13061 + ELAPSED TIME: 4 SECONDS + APPROXIMATELY 84 TRIPLES PER SECOND + +GTM>D IMPORT^C0XMAIN("smart-rdf-in/cole-susan.rdf") + +STARTED: 3111104.130628 +READING IN: smart-rdf-in/cole-susan.rdf +3428 LINES READ +ADDED: _:G271187746 _S:899679576 fmts:rdfSource _TXT_INCOMING_RDF_FILE_/home/geo +rge/fmts/trunk/samples/smart-rdf-in/cole-susan_rdf_538236597 +2101 XML NODES PARSED +SKIPPING NODE: 9 +SKIPPING NODE: 26 +SKIPPING NODE: 29 +SKIPPING NODE: 32 +SKIPPING NODE: 35 +INSERTING 1425 TRIPLES + ENDED AT: 3111104.130645 + ELAPSED TIME: 17 SECONDS + APPROXIMATELY 83 TRIPLES PER SECOND + +GTM>D IMPORT^C0XMAIN("smart-rdf-in/ford-shirley.rdf") + +STARTED: 3111104.130703 +READING IN: smart-rdf-in/ford-shirley.rdf +8922 LINES READ +ADDED: _:G740421472 _S:922849860 fmts:rdfSource _TXT_INCOMING_RDF_FILE_/home/geo +rge/fmts/trunk/samples/smart-rdf-in/ford-shirley_rdf_809878775 +5470 XML NODES PARSED +ERROR, NO OBJECT FOUND FOR NODE: 4217 +ERROR, NO OBJECT FOUND FOR NODE: 4226 +ERROR, NO OBJECT FOUND FOR NODE: 4232 +ERROR, NO OBJECT FOUND FOR NODE: 4258 +ERROR, NO OBJECT FOUND FOR NODE: 4267 +ERROR, NO OBJECT FOUND FOR NODE: 4273 +INSERTING 3745 TRIPLES + ENDED AT: 3111104.130756 + ELAPSED TIME: 53 SECONDS + APPROXIMATELY 70 TRIPLES PER SECOND + +GTM>D IMPORT^C0XMAIN("smart-rdf-in/gracia-paul.rdf") + +STARTED: 3111104.130817 +READING IN: smart-rdf-in/gracia-paul.rdf +10698 LINES READ +ADDED: _:G289354757 _S:026395070 fmts:rdfSource _TXT_INCOMING_RDF_FILE_/home/geo +rge/fmts/trunk/samples/smart-rdf-in/gracia-paul_rdf_297957633 +6571 XML NODES PARSED +ERROR, NO OBJECT FOUND FOR NODE: 2751 +ERROR, NO OBJECT FOUND FOR NODE: 2760 +ERROR, NO OBJECT FOUND FOR NODE: 2766 +ERROR, NO OBJECT FOUND FOR NODE: 6329 +INSERTING 4512 TRIPLES + ENDED AT: 3111104.130928 + ELAPSED TIME: 71 SECONDS + APPROXIMATELY 63 TRIPLES PER SECOND + +************************* + +benchmark of F2N qds load on raven with GT.M V5.4-002B Linux x86_64 and MM turned off + +**************************** + + +GTM>D IMPORT^C0XF2N("qds/qds.rdf") + +STARTED: 3111104.152012 +READING IN: qds/qds.rdf +73528 LINES READ +ADDED: _:G951670203 _S:805831840 fmts:rdfSource _TXT_INCOMING_RDF_FILE_/home/ge +o +rge/fmts/trunk/samples/qds/qds_rdf_394416761 +71530 XML NODES PARSED +PARSE COMPLETE AT 3111104.152023 + ELAPSED TIME: 8 SECONDS + APPROXIMATELY 8941 NODES PER SECOND +TRIPLES COMPLETE AT 3111104.153436 + ELAPSED TIME: 853 SECONDS + APPROXIMATELY 81 TRIPLES PER SECOND <------ this calculation is wrong, because "batches" of 10000 nodes at a time were processed mixed in with the dom node->triple processing +INSERTING 69537 TRIPLES +INSERTION COMPLETE AT 3111104.152023 + ELAPSED TIME: 127 SECONDS + APPROXIMATELY 547 NODES PER SECOND <------ I only wish that this were true, but alas it is based on the time to process the last "batch" divided by the total number of triples + ENDED AT: 3111104.153643 + ELAPSED TIME: 991 SECONDS + APPROXIMATELY 70 TRIPLES PER SECOND <----- this "overall" rate is the one to watch... + +************************* + +benchmark of F2N qds load on raven with GT.M V5.4-002B Linux x86_64 and MM turned on + +**************************** + + +GTM>D IMPORT^C0XF2N("qds/qds.rdf") + +STARTED: 3111104.175524 +READING IN: qds/qds.rdf +73528 LINES READ +ADDED: _:G802962321 _S:387624835 fmts:rdfSource _TXT_INCOMING_RDF_FILE_/home/geo +rge/fmts/trunk/samples/qds/qds_rdf_405517303 +71530 XML NODES PARSED +PARSE COMPLETE AT 3111104.175535 + ELAPSED TIME: 8 SECONDS + APPROXIMATELY 8941 NODES PER SECOND +TRIPLES COMPLETE AT 3111104.180951 + ELAPSED TIME: 856 SECONDS + APPROXIMATELY 81 TRIPLES PER SECOND +INSERTING 69537 TRIPLES +INSERTION COMPLETE AT 3111104.175535 + ELAPSED TIME: 130 SECONDS + APPROXIMATELY 534 NODES PER SECOND + ENDED AT: 3111104.181201 + ELAPSED TIME: 997 SECONDS + APPROXIMATELY 69 TRIPLES PER SECOND