Test results of IOF Core

This page contains the results of various tests conducted on IOF Core. These tests include evalutation of consequences of axioms and rules in IOF Core, validations through competency questions, checking of data models, and benchmarking of the performance of various tools for IOF Core.

Index

Allen’s Relations

IOF Core includes 7 interval relations, proposed by Allen (meets, before, overlaps, starts, finishes, occursDuring, and occursSimultaneouslyWith) and their inverses (except occursSimulataneouslyWith) applicable between ‘bfo:temporal interval' and ‘bfo:process’ instances. Along with these relations, a set of SWRL rules for inferring these relations among ‘bfo:temporal interval' and ‘bfo:process’ instances based on 'bfo:precedes’ relations among ‘bfo:temporal instances' and a set of property chains for composition of various Allen’s relations are also available separately from IOF Core.

Test Case # Allen-1 (Functional testing using competency questions)

Purpose	Functional testing of Allen’s relations using competency questions
Scope	Temporal relations among various processes and their influence on the participants of these processes.
Conducted by	Arkopaul Sarkar
IOF Core version	IOF Core spring release candidate available at https://github.com/iofoundry/ontology/tree/core_allen_interval_algebra
Date of testing	15 Aug 2023
User story	In a factory producing metal parts, various machining processes, such as turning and milling, lapping, grinding along with laser marking, quality check and packing takes place. Workers are assigned to corresponding machine to carry out these operations for work orders on various metal parts. The company wants to monitor the assignment of workers along with machine allocations.
Data	Production data on 10 work orders. Important columns are: Activity: process occurred Resource: machine used Start timestamp: begining time of the process Complete timestamp: end time of the process Span: the duration of the process Work Order: # of work order Part Desc. : The type of the part produced Worker ID: Identifier of the worker assigned to the machine for the process Note: The data is a part of and customised from the original data retrieved from http://fluxicon.com/disco/ presented by Dafna Levy (dafnal@nool.co.il)
Test criteria	The following competency questions should be available for query and the answers to the query should match the corresponding answers.
Pre-process steps	The data is loaded to GraphDB (https://www.ontotext.com/products/graphdb ) modeled using IOF Core (spring release candidate) and SWRL rules (see Pre-process steps of Test Case # Allen-2).
System configuration	Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz 2.59 GHz with 32.0 GB RAM
Methodology	Each CQs are tested with the following SPARQL queries as mentioned in the following table using https://www.ontotext.com/products/graphdb workbench on the KG below.
Result	All tests are passed as shown in the following table.

CQ Identifier	SPARQL Query	Result
core-s1-1		Passed
core-s1-2		Passed
core-s1-3		Passed
core-s1-4		Passed
core-s1-5		Passed
core-s1-6		Passed
core-s1-7		Passed
core-s1-8		Passed
core-s1-9		Passed
core-s1-10		Passed

Test Case # Allen-2 (Inferences by SWRL rules and inverse axioms)

Purpose	Testing the consequence of SWRL rules for asserting Allen’s relations
Scope	Temporal relations among time instances, intervals, and processes.
Conducted by	Arkopaul Sarkar
IOF Core version	IOF Core spring release candidate available at https://github.com/iofoundry/ontology/tree/core_allen_interval_algebra
Date of testing	18 Aug 2023
User story	In a factory producing metal parts, various machining processes, such as turning and milling, lapping, grinding along with laser marking, quality check and packing takes place. Workers are assigned to corresponding machine to carry out these operations for work orders on various metal parts. The company wants to monitor the assignment of workers along with machine allocations.
Data	Production data on 10 work orders. Important columns are: Activity: process occurred Resource: machine used Start timestamp: begining time of the process Complete timestamp: end time of the process Span: the duration of the process Work Order: # of work order Part Desc. : The type of the part produced Worker ID: Identifier of the worker assigned to the machine for the process Note: The data is a part of and customised from the original data retrieved from http://fluxicon.com/disco/ presented by Dafna Levy (dafnal@nool.co.il)
Test criteria	The SWRL rules on Allen’s relation should be able to infer the desired consequence when executed.
Pre-process steps	The dates are transformed to xsd:DateTime format by using OntoRefine (https://www.ontotext.com/products/ontotext-refine/). The data is loaded to GraphDB (https://www.ontotext.com/products/graphdb ) modeled using IOF Core (spring release candidate) by Insert query in . bfo:precedes and occursSimultaneouslyWith between instances of ‘bfo: temporal instance’ using the insert queries in . KG generated by step 3:
System configuration	Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz 2.59 GHz with 32.0 GB RAM
Methodology	The KG generated by Step 3 of pre-process is opened in Protege (https://protege.stanford.edu/ v 5.5.0). Core ontology release candidate and SWRL rules from http://spec.industrialontologies.org/rules/core/TemporalRelations02 are imported. Reasoning is completed using Hermit reasoner (http://www.hermit-reasoner.com/ v 1.4.3.456) in 140099 ms. As Rules 8-12, 14, and 15 requires other rules to succeed, only the consequences of these rules are evaluated. For each rule, at least one example of consequence is searched in the inferred ontology. The rule is considered successful if the rule is found in one of the explanation for the consequence and the inference is based on correct assertion.
Result	All tests are passed (see in the table below)

#	Targer	Explanation	Result
1	Rule S8		Passed
2	Rule S9		Passed
3	Rule S10		Passed
4	Rule S11	P41 LaserMarking	Passed
5	Rule S12		Passed
6	Rule S14		Passed
7	Rule S15		Passed

Gain of Role and Loss of Role

“Gain of Role” (GOR) and “Loss of Role” (LOR) are two new process types are included in the IOF to provide ability to model when a bearer starts bearing the role and when it loses the role as well as the duration for which the bearer bears the role.

Test Case # role-1 (Functional testing using competency questions)

Purpose	Functional testing of GOR and LOR using competency questions
Scope	Classification of GOR and LOR and query based on temporal durations of Roles borne by different bearers and other contemporary events
Conducted by	Arkopaul Sarkar
IOF Core version	IOF Core spring release candidate available at https://github.com/iofoundry/ontology/tree/core_allen_interval_algebra
Date of testing	18 Aug 2023
User story	Three operators are employed at a control room that remains operational for 24 hours. Operators work in three shifts of around 8 hours each having around 10 minutes for change-over between shifts. The following entries were recorded in the logbook on 28^th May 2016. As per policy, the shift starts for an operator as soon as she arrives in the control room and ends when she leaves the room.
Data	[2016-05-28T00:03:00Z] Operator 1 arrived for change-over. [2016-05-28T00:10:00Z] Operator 3 left the control. [2016-05-28T03:24:00Z] Incident 1 is observed. [2016-05-28T03:26:00Z] Incident 1 is stopped. [2016-05-28T07:58:00Z] Operator 2 arrived for change-over. [2016-05-28T08:07:00Z] Operator 1 left the control. [2016-05-28T16:01:00Z] Operator 3 arrived for change-over. [2016-05-28T16:05:00Z] Incident 2 is observed. [2016-05-28T16:08:00Z] Operator 2 left the control. [2016-05-28T23:43:00Z] Incident 3 is observed.
Test criteria	The following competency questions should be available for query and the answers to the query should match the corresponding answers.
Pre-process steps	The data is manually modeled in Protege (https://protege.stanford.edu/ v 5.5.0) using IOF Core (spring release candidate). Reasoning is completed using Hermit reasoner (http://www.hermit-reasoner.com/ v 1.4.3.456). The inferred KG is loaded into https://www.ontotext.com/products/graphdb directly as the following RDF.
System configuration	Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz 2.59 GHz with 32.0 GB RAM
Methodology	Each CQs are tested with SPARQL queries given in the following table using https://www.ontotext.com/products/graphdb workbench on the KG below.
Result	All tests are passed as shown in the following table.

CQ Identifier	SPARQL Query	Result
core-s1-1		Passed
core-s1-2		Passed
core-s1-3		Passed
core-s1-4		Passed
core-s1-5		Passed
core-s1-6	test-core-s1-4 (change filter by incident3)	Passed

Mapping with OWL-Time

IOF-Core includes provision to assert calendar date and clock time to the instances of bfo:Temporal Instant by using two sub-classes of ValueExpression: TemporalInstantValueExpression and TemporalIntervalValueExpression and a data property hasDateTimeValue which can link a xsd:DateTime to TemporalInstantValueExpression. However, both TemporalInstantValueExpression and TemporalIntervalValueExpression classes are mapped to https://www.w3.org/TR/owl-time/ classes to allow users to utilize the capability of https://www.w3.org/TR/owl-time/ for asserting calendar and clock time as well as durations in various formats, granularity and systems.

Test Case # time-1 (Data models)

Purpose	Test the ability to consistently use https://www.w3.org/TR/owl-time/ time description class and properties in conjunction with IOF-Core constructs
Scope	Calendar and clock
Conducted by	Arkopaul Sarkar
IOF Core version	IOF Core spring release candidate available at https://github.com/iofoundry/ontology/tree/core_allen_interval_algebra
Date of testing	18 Aug 2023
User story	Three operators are employed at a control room that remains operational for 24 hours. Operators work in three shifts of around 8 hours each having around 10 minutes for change-over between shifts. The following entries were recorded in the logbook on 28^th May 2016. As per policy, the shift starts for an operator as soon as she arrives in the control room and ends when she leaves the room.
Data	[2016-05-28T00:03:00Z] Operator 1 arrived for change-over. [2016-05-28T00:10:00Z] Operator 3 left the control. [2016-05-28T03:24:00Z] Incident 1 is observed. [2016-05-28T03:26:00Z] Incident 1 is stopped. [2016-05-28T07:58:00Z] Operator 2 arrived for change-over. [2016-05-28T08:07:00Z] Operator 1 left the control. [2016-05-28T16:01:00Z] Operator 3 arrived for change-over. [2016-05-28T16:05:00Z] Incident 2 is observed. [2016-05-28T16:08:00Z] Operator 2 left the control. [2016-05-28T23:43:00Z] Incident 3 is observed.
Test criteria	The following competency questions should be available for query and the answers to the query should match the corresponding answers.
Pre-process steps	The data is manually modeled in Protege (https://protege.stanford.edu/ v 5.5.0) using IOF Core (spring release candidate). Reasoning is completed using Hermit reasoner (http://www.hermit-reasoner.com/ v 1.4.3.456). The inferred KG is loaded into https://www.ontotext.com/products/graphdb directly as the following RDF.
System configuration	Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz 2.59 GHz with 32.0 GB RAM
Methodology	Each CQs are tested with SPARQL queries given in the following table using https://www.ontotext.com/products/graphdb workbench on the KG below.
Result	All tests are passed as shown in the following table.