/
'Create DT' Process

'Create DT' Process

Owned by Oh, Hakju (IntlCtr)
Last updated: 2021-12-02 by Serm Kulvatunyou
11 min read

  1. Click ‘Create DT’ Button.

  2. Choose ‘Base DT’

    1. There are two types of Base DT to be chosen.

      1. CDT<serm> This is fine.

        Create or Update CDT is out of scope.

        </serm>

      2. BDT

        1. Default

        2. Unqualified

        3. Qualified: In this case, new DT must have a qualifier. <serm>I’m thinking we should do away with these 3 types designation. They are primarily for the purpose of describing the OAGIS import. The Default BDT in particular shouldn’t be needed anymore. A BDT being qualified or unqualified would depend on whether it has qualifiers or not; we don’t need that designation/classification.

          Note that per CCTS, the user can specify multiple qualifiers. I think we will just use a single text field and rely on the user to format multi-qualifier correctly? E.g., “Two Point_ Decimal_ “ as in “Two Point_ Decimal_ Measure. Type”. Implementing a separate table may make searching too slow?

          So a BDT can be based on a CDT or a BDT.

          </serm>

  3. Created an initial DT with the same data_type_term of the Based DT and the user cannot directly change the data_type_num value.

    <serm> This is fine even with changes in import and OAGIS serialization rules I introduced later below. </serm>

    1. Available Actions

      1. Edit version_num: Currently, all DT’s version_num are 1.0 <serm> this column needs to be replaced by the revision number same as for all other types of CCs.</serm>

      2. Change Base DT: We may allow this function in Rev. 1, but should notice like 'You may lose all contents you made`. <serm>11/30/2021: No base DT change offered right now.</serm>

      3. Edit den?: For example, the Default BDT ‘Code_1E7368. Type’ is derived from Code CDT and there is no column to put such six alphanumeric codes in dt table. Since it is not a qualifier, we may need to add new column or revisit DT naming rules.

<serm> I took time to go thru CCTS, CDT catalog, and the UN/CEFACT XML NDR 3.0 specifications. There is no way we can be compliant to all these three specs. So I decided to comply with CCTS only (or the most). Then CDT catalog and the NDR. OAGIS DTs broke NDR rules already. I think we can say that we didn’t break CDT catalog specs because it only talks about CDTs, and not BDTs. The only iffy issue is the spec gave a list of valid representation terms. I needed to play with OAGIS BDT representation term to keep them compliant with CCTS and still have enough strings to generate type names that match current OAGIS. We can interpret that the list of valid representation terms in CDT Catalog spec is only for CDT. And CCTS spec didn’t talk about representation term for BDT at all. It only says that BDT data type term must be the same as its based CDT.

Some additional note for reference just in case. “Representation Terms of all CDTs are the same as Data Type Term. The CEFACT NDR does not talk about Representation Term for BDT either. CCTS says that DT term can be qualified whereas Representation Term is never qualified. They both supposed to reflect value domain. CDTs shall have unique data type term. CDT define a list of Representation Terms which are exactly the same as DT Terms. Representation Terms are also used in the SC. I kinda think that CCTS meant for representation term to be CDT Primitives.”

ISO 15000-5 says that DEN of a CCT is [Representation Term]. Type. However, the table listing approved CCT uses the word Object Class in place of Representation Term, so this is confusing.

I just found that the six number suffix comes from the UN/CEFACT XML Schema NDR.

It represents unique identifier. It sounds like according to that NDR all BDTs are supposed to have that 6-digit identifier, but again some of OAGIS BDTs don’t have it.

So we should make a new column for that six digit code and use it in the serialization when exists.

Below are import and serialization rules that will effect the way BDTs are currently imported.

Rules:

How to populate Six digit ID, DT Term, Qualifier, and Representation Term columns.

A. For Unqualified and Qualified BDT. These rules have to be done in the specified order

  1. Six digit ID = empty string

  2. DT Term is as usual. It uses the DT Term of the CDT it is based on.

  3. Qualifier = space-separated schema type name, substract 'Type’ from the end, and then substract DT Term from the end. [for unqualified BDT, this should always result in empty string (except the CodeContentType and IDContentType special cases]. If the based type has a qualifier, determine whether another “_” is needed in the qualifier by taking the qualifier and substract the qualifier of the based BDT from the end. If there is anything left, the “_” should be inserted behind/in-between. Note that I could see right now that if the based type has a qualifier, only the additional qualifier is kept in the Qualifier column of the derived type. We need to change for the derived type to have the whole qualifier, e.g., “Preferred_ Open” for the whole thing to work.

  4. Representation Term = space-separated schema type name, subtract ‘Type’ from the end, substract qualifier from the front

B. For default BDT

  1. Six digit ID = the six character at the end.

  2. DT Term is as usual. It uses the DT Term of the CDT it is based on.

  3. Qualifier = empty string.

  4. Representation Term = Same as DT Term

For all rules below, truncate any duplicate consecutive words.

For serialized type name rules, remove spaces, Identifer => ID,

  1. DEN = DT Term + “. Type”, when there is no qualifier

  2. DEN = Qualifier(s) + “_ “ + DT Term + “. Type” when there is qualifier

  3. OAGIS Serialized Type name = Qualifier + Representation Term + “Type” when there is no six digit ID.

  4. OAGIS Serialized Type name = Qualifier + Representation Term + “Type” + [“_”+Six Digit ID] when there is a six digit ID.

Validation cases

  1. CodeType_1E7368 BDT: DT Term = “Code”; Six Digit ID = “1E7368”; Qualifier=””; Representation Term = “Code”; DEN = “Code. Type”; Serialized Name = “CodeType_1E7368”; Based on Code CDT.

  2. TextType_0VCBZ5 BDT: DT Term = “Text”; Six Digit ID = “0VCBZ5”; Qualifier=””; Representation Term = “Text”; DEN = “Text. Type”; Serialized Name = “TextType_1E7368”; Based on Text CDT.

  3. NormalizedStringType BDT: DT Term = “Text”; Six Digit ID = “”; Qualifier=”Normalized String”; Representation Term = “”; DEN = “Normalized String_ Text. Type”; Serialized Name = “NormalizedStringType”; Based on TextType_0VCBZ5.

  4. ExpressionType BDT: DT Term = “Text”; Six Digit ID = “”; Qualifier=”Expression”; Representation Term = “”; DEN = “Expression_ Text. Type”; Serialized Name = “ExpressionType”; based on TextType_0F0ZX1 BDT

  5. ActionExpressionType BDT: DT Term = “Text”; Six Digit ID = “”; Qualifier=”Action Expression”; Representation Term = “”; DEN = “Action Expression_ Text. Type”; Serialized Name = “ActionExpressionType”; based on TextType_0F0ZX1 BDT.

  6. IDContentType BDT: DT Term = “Identifier”; Six Digit ID = “”; Qualifier=”Identifier Content”; Representation Term = “”; DEN = “Identifier Content_ Identifier. Type”; Serialized Name = “IDContentType”; based on IDType_B3F14E.

  7. TextType BDT: DT Term = “Text”; Six Digit ID = “”; Qualifier=””; Representation Term = “Text”; DEN = “Text. Type”; Serialized Name = “TextType”; Based on Text CDT.

  8. TimeType_100DCA BDT: DT Term = “Time”; Six Digit ID = “100DCA”; Qualifier=””;Representation Term = “Time”; DEN = “Time. Type”; Serialized Name = “TimeType_100DCA”; Based on Text CDT.

  9. DayOfWeekHourMinuteUTCType BDT: DT Term = “Time”; Six Digit ID = “”; Qualifier=”Day Of Week Hour Minute UTC”; Representation Term = “”; DEN = “Day Of Week Hour Minute UTC_ Time. Type”; Serialized Name = “DayOfWeekHourMinuteUTCType”; Based on TimeType_100DCA.

  10. OpenValueType BDT: DT Term = “Value”; Six Digit ID = “”; Qualifier=”Open”; Representation Term = “Value”; DEN = “Open_ Value. Type”; Serialized Name = “OpenValueType”; Based on ValueType.

  11. PreferredOpenNameType BDT: DT Term = “Name”; Six Digit ID = “”; Qualifier=”Preferred_ Open” ; Representation Term = “Name”; DEN = “Preferred_ Open_ Value. Type”; Serialized Name = “PreferredOpenNameType”; Based on OpenNameType

  12. CodeContentType BDT: DT Term = “Code”; Six Digit ID = “”; Qualifier=”Code Content”; Representation Term = “”; DEN = “Code Content_ Code. Type”; Serialized Name = “CodeContentType”; based on CodeType_1E7368.

  13. CurrencyCodeContentType BDT: DT Term = “Code”; Six Digit ID = “”; Qualifier=”Currency_ Code Content”; Representation Term = “”; DEN = “Currency_ Code Content_ Code. Type”; Serialized Name = “CurrencyCodeContentType”; Based on CodeContentType.

  14. CurrencyCodeType BDT: DT Term = “Code”; Six Digit ID = “”; Qualifier=”Currency”; Representation Term = “Code”; DEN = “Currency_ Code. Type”; Serialized Name = “CurrencyCodeType”; Based on CurrencyCodeContentType.

  15. ProcessConfirmationCodeContentType BDT: DT Term = “Code”; Six Digit ID = “”; Qualifier=”Process Confirmation_ Code Content”; Representation Term = “”; DEN = “Process Confirmation_ Code Content_ Code. Type”; Serialized Name = “ProcessConfirmationCodeContentType ”; Based on CodeContentType.

</serm>

  1. Add new supplementary component: Some BDTs in OAGIS have custom supplementary components that don’t exist in CCS catalog. Furthermore, we allowed hidden CCS supplementary components using maximum cardinality = 0.

    1. We may need to show hidden supplementary components on DT tree.

    2. Cardinality rules followed the same logic as same as ASCC/BCC

    3. Edit primitive typeto be used in the expression.

    4. Facets?

<serm>

Yes - the ability to add SC - There is a question to whether added SC needs to be retroactively added to all based BDTs and CDT as well. The current DB has this done even though the old import rule didn’t suggest that. This may be because some old import rules imply the need without explicitly stating it. Hakju think it is because of the BDT_SC_PRI_RESTRI table. Serm will double check.

Yes - show hidden SC. It would be good to render anything with zero max cardinality with strikethrough font.

Cardinality rules - Max cardinality has to be max out at 1. SC cannot have more than 1 max cardinality.

Yes - Primitive Restriction can be one of the XBTs or a code list or agency id list.

One issue might be, the current implementation copies SCs into the derived BDTs - this is to support cardinality restriction. This means that when a CDT or BDT has been revised, all derived BDTs (direct and indirect) have to be revised. This would have been the case anyway, if BDTs were to exist only on the BIE realm, i.e., BDTs would have to be uplifted.

</serm>

Maybe we say that there is no revision to BDT. Only that new one can be created and the BCCP can be revised to use the new one. We may allow BDT deprecation and replacement marking. This will simplify the application logic. But it could potentially mean that the user has to recreate a chain of BDT derivation manually. This may be ok b/c maybe the whole chain is not needed.

<serm>I think this is good. We should prototype this.</serm>

Is there any restriction to changing BDT in a BCCP, i.e., do we restrict to compatible data type term?

<serm>Just like association deletion, we shouldn’t put this restriction. OTOH, we should aim to provide a backward compatibility validation during the release process. But we can certainly warn that changing BDT with an incompatible data type term may result in backward incompatibility. </serm>

There is a question to about changing CC name.

<serm> Similar to above we should allow CC name change but warn about potential backward incompatibility. And again provide a backward compatibility validation during the release process. related to #1040</serm>

Why the Boolean CDT primitive was mapped to xbt_BooleanType in the design doc?

<serm>This was the case b/c Boolean CDT Primitive has True or False as possible values. xsd:boolean can have true/false or 1/0, but xbt_BooelanType limits the values to true/false. And Score now only allows xbt_BooleanType only.</serm>

 

Facets - Can we consider this out of scope for now?

<serm>I would make this out of scope for now. First, I think hardly anybody needs to use this. Also, it is harder to make this syntax independent. And It is not clear how this would interact with the content in the XBT table. Maybe Additional facets are implemented on the BIE side. I need to think more about this.

What we need at the minimum is for the user to see the content in the XBT table. I guess somehow on the BDT detail page, be able to click on the primitive to go to the its detail where the Name, Schema_Definition, the columns representing maps to other type systems are displayed.

One question I have is, can we merge the BuiltinType into Schema_Definition columns? The rows that are Null in the Schema_Definition column can take the value from the BuiltInType column. And where both columns have values, the values in BuiltInType column seem not needed?

</serm>

Got to look at the UI.

<serm> I generally like the fact that BDTs are listed in the CC View/Edit page. But there may be reasons to have a separate page for BDT management as well. For example, we might want to display default primitive to better differentiate different BDTs that have the same DEN. If clicking on the row shows some BDT details that includes primitives, that would help.

In the value domain area of the UI, I prefer for it to look like this.

Note that when the Value Domain Type is Code List or Agency ID List, the CDT Primitive is always Token. And the user can select on a Code List or the Agency ID List in the Value Domain column.

</serm>

Value domain - allow more than one code lists, agency ID list?

<serm> Let’s go ahead and allow this. </serm>

 

Example

 

 

 

DT management is mean management the CCTS 23 CDT and OAGIS defined DTs. => DT

  • BBIE/BBIE_SC contains BDT(bbie.bdt_pri_restri_id, bbie.code_list_id, bbie.agency_id_list_id).

 

  1. Add six_digit_id and representation_term to DT table.

    1. data_type_term: allow CCTS 23 CDTs. can not be edit.

    2. qualifier: "_ " separated string
      (1) 12 DTs are changed from unqualified to qualified. (e.g. NormalizedStringType)
      (2) the case that qualifier is not stacking.
      Code Content_ Code. Type
      Currency Code Content_ Code. Type
      Currency_ Code. Type
      - "Currency_ Code. Type" should behave "Code Content_ {qualifier}" but it has "Currency" for qualifier. is the qualifier can be editable by developer?
      - "Code Content" added when it import from OAGI schema. b/c it has a code list as restriction.
      Is any endfix rule needed? like this case?

    3. six_digit_id:
      (1) six_digit_id should be Unique?
      (2) After remove the six_digit_id from den, den is no longer unique. is it okay?
      (3) Should we exclude six_digit_id and qualifier? there's no coexistence case.

    4. representation_term is always the same with data_type_term or null. if we don't need to change representation_term then we can change it to boolean type.

  2. Create DT.

    1. Copy data_type_term, representation_term from base.
      (1) If presentation_term is null, shall we follow null or replace it with data_type_term?

    2. qualifier: {base_qualifier}? {Default_qualifier_value} e.g. "Process Category Qualifier"

    3. six_digit_id is not copied from base.

    4. copy restriction values(BDT_PRI_RESTRI) from base.

    5. copy DT_SC from base.

    6. copy DT_SC restriction values(BDT_SC_PRI_RESTRI) from base.

  3. Revise DT.

    1. same as CCs, insert revised records to DT and DT_SC table.

    2. insert new DT restriction values(BDT_PRI_RESTRI) records.

    3. insert new DT_SC restriction values(BDT_SC_PRI_RESTRI) records.

  4. Cancel DT.
    remove all revised records.

  5. create Supplementery Component

    1. Can DT_SC be created regardless of DT?
      (1) CCTS CDT should not be revised/editied/deleted.

    2. DT_SC.representation_term can not be changed.
      (1) DT_SC Creation:
      - DT_SC.representation_term is limited to 23 data_type_terms of CCTS CDT.
      - Add default restriction values of selected DT's default restriction values
      - Any primitive/codelist/agnecyIdList can be added to DT? regardless of owner DT type.

  6. Add Value Domain

    1. Primitive
      (1) Multiple choices are available, and XBT expression can be selected for each selected primary.
      - Should there be restrictions between the primitives?
      Numeric and string types must be prevented from cross-selecting each other.
      (2) Default XBT expressions of selected primitives.
      - Should there be restrictions between the primitive and XBT expressions?

    2. Code list / agency id can only be selected within the same release as dt.
      (1) Should the end-user code list be selectable?

https://oagi.org/pages/connect-membership