Evaluation of IOF Core 202401 release

Owned by Arkopaul Sarkar
Last updated: 2024-02-01
10 min read

This page contains the results of various tests conducted on IOF Core including evaluations of consequences of axioms and rules in IOF Core, validations through competency questions, checking of data models, and benchmarking of the performance of various tools for IOF Core.

Table of contents

 

Allen’s Relations

IOF Core includes 7 interval relations, proposed by Allen (meets, before, overlaps, starts, finishes, occursDuring, and occursSimultaneouslyWith) and their inverses (except occursSimulataneouslyWith) applicable between ‘bfo:temporal interval' and ‘bfo:process’ instances. Along with these relations, a set of SWRL rules for inferring these relations among ‘bfo:temporal interval' and ‘bfo:process’ instances based on 'bfo:precedes’ relations among ‘bfo:temporal instances' and a set of property chains for the compositions of various Allen’s relations are also available separately from IOF Core.

Test Case # Allen-1 (Functional testing using competency questions)

Purpose

Functional testing of Allen’s relations using competency questions

Scope

Temporal relations among various processes and their influence on the participants of these processes.

Conducted by

@Arkopaul Sarkar

IOF Core version

IOF Core spring release candidate available at https://github.com/iofoundry/ontology/tree/core_allen_interval_algebra

Date of testing

Aug 15, 2023

User story

In a factory producing metal parts, various machining processes, such as turning and milling, lapping, grinding along with laser marking, quality check and packing takes place. Workers are assigned to corresponding machine to carry out these operations for work orders on various metal parts. The company wants to monitor the assignment of workers along with machine allocations.

Data

Production data on 10 work orders. Important columns are:

  • Activity: Type of process

  • Resource: machine used

  • Start timestamp: beginning time of the process

  • Complete timestamp: end time of the process

  • Span: the duration of the process

  • Work Order: # of work order

  • Part Desc. : The type of the part produced

  • Worker ID: Identifier of the worker assigned to the machine for the process

Note: The data is a part of and customised from the original data retrieved from http://fluxicon.com/disco/ presented by Dafna Levy (dafnal@nool.co.il)

Test criteria

The following competency questions should be available for query and the answers to the query should match the corresponding answers.

Pre-process steps

The data is loaded to GraphDB (https://www.ontotext.com/products/graphdb ) modeled using IOF Core (spring release candidate) and SWRL rules (see Pre-process steps of Test Case # Allen-2).

System configuration

Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz 2.59 GHz with 32.0 GB RAM

Methodology

Each CQs are tested with the following SPARQL queries as mentioned in the following table using https://www.ontotext.com/products/graphdb workbench on the KG below.

Result

All tests are passed as shown in the following table.

CQ Identifier

SPARQL Query

Result

CQ Identifier

SPARQL Query

Result

core-s1-1

Passed

core-s1-2

Passed

core-s1-3

Passed

core-s1-4

Passed

core-s1-5

Passed

core-s1-6

Passed

core-s1-7

core-s1-8

core-s1-9

core-s1-10

Test Case # Allen-2 (Inferences by SWRL rules and inverse axioms)

Purpose

Testing the consequence of SWRL rules for asserting Allen’s relations

Scope

Temporal relations among time instances, intervals, and processes.

Conducted by

@Arkopaul Sarkar

IOF Core version

IOF Core spring release candidate available at https://github.com/iofoundry/ontology/tree/core_allen_interval_algebra

Date of testing

Aug 18, 2023

User story

In a factory producing metal parts, various machining processes, such as turning and milling, lapping, grinding along with laser marking, quality check and packing takes place. Workers are assigned to corresponding machine to carry out these operations for work orders on various metal parts. The company wants to monitor the assignment of workers along with machine allocations.

Data

Production data on 10 work orders. Important columns are:

  • Activity: process occurred

  • Resource: machine used

  • Start timestamp: begining time of the process

  • Complete timestamp: end time of the process

  • Span: the duration of the process

  • Work Order: # of work order

  • Part Desc. : The type of the part produced

  • Worker ID: Identifier of the worker assigned to the machine for the process

Note: The data is a part of and customised from the original data retrieved from http://fluxicon.com/disco/ presented by Dafna Levy (dafnal@nool.co.il)

Test criteria

  1. The SWRL rules on Allen’s relation should be able to infer the desired consequence when executed.

Pre-process steps

  1. The dates are transformed to xsd:DateTime format by using OntoRefine (https://www.ontotext.com/products/ontotext-refine/).

  2. The data is loaded to GraphDB (https://www.ontotext.com/products/graphdb ) modeled using IOF Core (spring release candidate) by Insert query in .

  3. bfo:precedes and occursSimultaneouslyWith between instances of ‘bfo: temporal instance’ using the insert queries in .

KG generated by step 3: e

System configuration

Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz 2.59 GHz with 32.0 GB RAM

Methodology

  1. The KG generated by Step 3 of pre-process is opened in Protege (https://protege.stanford.edu/ v 5.5.0).

  2. Core ontology release candidate and SWRL rules from http://spec.industrialontologies.org/rules/core/TemporalRelations02 are imported.

  3. Reasoning is completed using Hermit reasoner (http://www.hermit-reasoner.com/ v 1.4.3.456) in 140099 ms.

  4. As Rules 8-12, 14, and 15 requires other rules to succeed, only the consequences of these rules are evaluated.

  5. For each rule, at least one example of consequence is searched in the inferred ontology.

  6. The rule is considered successful if the rule is found in one of the explanation for the consequence and the inference is based on correct assertion.

Result

All tests are passed (see in the table below)

#

Targer

Explanation

Result

#

Targer

Explanation

Result

1

Rule S8

2

Rule S9

 

3

Rule S10

4

Rule S11

P41 LaserMarking

5

Rule S12

6

Rule S14

 

7

Rule S15

 

Gain of Role and Loss of Role

“Gain of Role” (GOR) and “Loss of Role” (LOR) are two new process types included in the IOF to provide the ability to model when a bearer starts bearing the role and when it loses the role as well as the duration for which the bearer bears the role.

Test Case # role-1 (Functional testing using competency questions)

Purpose

Functional testing of GOR and LOR using competency questions

Scope

Classification of GOR and LOR and query based on temporal durations of Roles borne by different bearers and other contemporary events

Conducted by

@Arkopaul Sarkar

IOF Core version

IOF Core spring release candidate available at https://github.com/iofoundry/ontology/tree/core_allen_interval_algebra

Date of testing

Aug 18, 2023

User story

Three operators are employed at a control room that remains operational for 24 hours. Operators work in three shifts of around 8 hours each having around 10 minutes for change-over between shifts. The following entries were recorded in the logbook on 28th May 2016. As per policy, the shift starts for an operator as soon as she arrives in the control room and ends when she leaves the room.

Data

[2016-05-28T00:03:00Z] Operator 1 arrived for change-over. 

[2016-05-28T00:10:00Z] Operator 3 left the control.

[2016-05-28T03:24:00Z] Incident 1 is observed.

[2016-05-28T03:26:00Z] Incident 1 is stopped.

[2016-05-28T07:58:00Z] Operator 2 arrived for change-over.

[2016-05-28T08:07:00Z] Operator 1 left the control.

[2016-05-28T16:01:00Z] Operator 3 arrived for change-over.

[2016-05-28T16:05:00Z] Incident 2 is observed.

[2016-05-28T16:08:00Z] Operator 2 left the control.

[2016-05-28T23:43:00Z] Incident 3 is observed.

Test criteria

The following competency questions should be available for query and the answers to the query should match the corresponding answers.

Pre-process steps

  1. The data is manually modeled in Protege (https://protege.stanford.edu/ v 5.5.0) using IOF Core (spring release candidate).

  2. Reasoning is completed using Hermit reasoner (http://www.hermit-reasoner.com/ v 1.4.3.456).

  3. The inferred KG is loaded into https://www.ontotext.com/products/graphdb directly as the following RDF.

System configuration

Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz 2.59 GHz with 32.0 GB RAM

Methodology

Each CQs are tested with SPARQL queries given in the following table using https://www.ontotext.com/products/graphdb workbench on the KG below.

Result

All tests are passed as shown in the following table.

CQ Identifier

SPARQL Query

Result

CQ Identifier

SPARQL Query

Result

core-s1-1

core-s1-2

core-s1-3

core-s1-4

core-s1-5

core-s1-6

test-core-s1-4 (change filter by incident3)

Mapping with OWL-Time

IOF-Core includes a provision to assert calendar date and clock time to the instances of bfo:Temporal Instant by using two sub-classes of ValueExpression: TemporalInstantValueExpression and TemporalIntervalValueExpression and a data property hasDateTimeValue which can link a xsd:DateTime to TemporalInstantValueExpression. However, both TemporalInstantValueExpression and TemporalIntervalValueExpression classes are mapped to https://www.w3.org/TR/owl-time/ classes to allow users to utilize the capability of https://www.w3.org/TR/owl-time/ for asserting calendar and clock time as well as durations in various formats, granularity and systems.

Test Case # time-1 (Validation of data modelled using both IOF-Core and OWL-Time)

Purpose

Test the ability to consistently use https://www.w3.org/TR/owl-time/ time description class and properties in conjunction with IOF-Core constructs

Scope

Calendar and clock

Conducted by

@Arkopaul Sarkar

IOF Core version

IOF Core spring release candidate available at https://github.com/iofoundry/ontology/tree/core_allen_interval_algebra

Date of testing

Aug 20, 2023

User story

Test scenarios are derived from the examples given in https://www.w3.org/TR/owl-time/#examples.

Data

Test data are derived from the examples given in https://www.w3.org/TR/owl-time/#examples.

Examples covered:

  1. DateTimeDescription vs dateTime (https://www.w3.org/TR/owl-time/#dtd-vs-dt )

  2. Use of temporal reference systems (https://www.w3.org/TR/owl-time/#different-TRS)

  3. Temporal precision (https://www.w3.org/TR/owl-time/#temporal-precision )

  4. A Use Case for Scheduling (https://www.w3.org/TR/owl-time/#scheduling )

Test criteria

  1. Examples in OWL-Time should be reproduced without loss of data.

  2. The data model is consistent.

Pre-process steps

  1. The data is manually modeled in Protege (https://protege.stanford.edu/ v 5.5.0) using IOF Core (spring release candidate) and https://www.w3.org/TR/2022/CRD-owl-time-20221115/ (Source file available at https://raw.githubusercontent.com/w3c/sdw/gh-pages/time/rdf/time.ttl).

  2. The mapping file is imported from https://spec.industrialontologies.org/ontology/core/commonstocoremapping/MappingOWLTimeToIOF .

  3. Reasoning is completed using Hermit reasoner (http://www.hermit-reasoner.com/ v 1.4.3.456).

System configuration

Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz 2.59 GHz with 32.0 GB RAM

Methodology

The assertions in the following file are compared with the assertions given in https://www.w3.org/TR/owl-time/#examples in Turtle format as well as reasoned for checking consistency.

Result

All tests are passed as shown in the following table. The ontology was processed without issues by Hermit reasoner in 5651 ms.

#

Example modeled

Model using OWL-Time

Model using IOF Core and OWL-Time

Result

#

Example modeled

Model using OWL-Time

Model using IOF Core and OWL-Time

Result

1

DateTimeDescription vs dateTime

ex:meetingStart a :Instant ; :inDateTime ex:meetingStartDescription ; :inXSDDateTimeStamp 2017-04-12T10:30:00+10:00 . ex:meetingStartDescription a :DateTimeDescription ; :unitType :unitMinute ; :minute 30 ; :hour 10 ; :day "---12"^^xsd:gDay ; :dayOfWeek :Wednesday ; :dayOfYear 102 ; :week 15 ; :month "--04"^^xsd:gMonth ; :monthOfYear greg:April ; :timeZone <https://www.timeanddate.com/time/zones/aest> ; :year "2017"^^xsd:gYear .
:meetingStart rdf:type owl:NamedIndividual , obo:BFO_0000203 ; Core:hasValueExpressionAtAllTimes :meetingStartDescription . :meetingStartDescription rdf:type owl:NamedIndividual , time:DateTimeDescription ; intervals:monthOfYear intervals:April ; time:dayOfWeek time:Wednesday ; time:timeZone <https://www.timeanddate.com/time/zones/aest> ; time:unitType time:unitMinute ; time:day "---12"^^xsd:gDay ; time:dayOfYear 102 ; time:hour 10 ; time:minute 30 ; time:month "--04"^^xsd:gMonth ; time:week 15 ; time:year "2017"^^xsd:gYear ; Core:hasDateTimeValue "2017-04-12T10:30:00+10:00"^^xsd:dateTime .

2

Use of temporal reference systems

ex:AbbyBirthday a :Instant ; :inDateTime ex:AbbyBirthdayHebrew ; :inTimePosition ex:AbbyBirthdayUnix ; rdfs:label "Abby's birthdate"^^xsd:string ; :inDateTime ex:AbbyBirthdayGregorian ; :inXSDDateTimeStamp "2001-05-23T08:20:00+08:00"^^xsd:dateTimeStamp ; . ex:AbbyBirthdayGregorian a :DateTimeDescription ; :day "---23"^^xsd:gDay ; :dayOfWeek :Wednesday ; :dayOfYear "143"^^xsd:nonNegativeInteger ; :hour "8"^^xsd:nonNegativeInteger ; :minute "20"^^xsd:nonNegativeInteger ; :month "--05"^^xsd:gMonth ; :monthOfYear greg:May ; :timeZone <https://www.timeanddate.com/time/zones/awst> ; :unitType :unitMinute ; :year "2001"^^xsd:gYear ; . ex:AbbyBirthdayUnix a :TimePosition ; :hasTRS <http://dbpedia.org/resource/Unix_time> ; :numericPosition 990577200 ; rdfs:label "Abby's birthdate in Unix time"^^xsd:string ; . ex:UnixTime rdfs:subClassOf time:TimePosition ; rdfs:subClassOf [ rdf:type owl:Restriction ; owl:hasValue <http://dbpedia.org/resource/Unix_time> ; owl:onProperty time:hasTRS ; ] ; .

3

Temporal precision

4

A Use Case for Scheduling

 

https://oagi.org/pages/connect-membership