ISO8601 Duration - Representation of decimal seconds

b Barry

on June 17, 2013

I have a question around the SDTM rule SD1011 for time duration. Some of the records in the MK domain are being flagged "Invalid ISO 8601 value" for the MKDUR variable (i.e. values: PT360.0S, PT2.4S, PT2.3S, PT3.7S). The source data is muscle testing (MK domain) and it does collect very accurate measurements to the 10th of a second.

The ISO8601 document says this about decimal representation:

If necessary for a particular application, the lowest order components may have a decimal fraction. The

decimal fraction shall be divided from the integer part by the decimal sign specified in ISO 31-0, i.e. the

comma [,] or full stop [.]. Of these, the comma is the preferred sign. The decimal fraction shall at least

have one digit, the maximum number of digits in the decimal component needs to be agreed by the

partners in information interchange. If the magnitude of the number is less than unity, the decimal sign

shall be preceded by a zero (see ISO 31-0).

With an example later given as:

Time interval by duration and context — duration with only hour component with one or more digits in the

integral part and one or more digits in the decimal part

Basic format : PTnn,nnH

I had a look at the 3.1.3 configuration file online, and it looks like the test for ISO8601 compliance is done using a regular expression:

(-?P((([0-9]+(\.[0-9]+)?Y)?([0-9]+(\.[0-9]+)?M)?([0-9]+(\.[0-9]+)?D)?(T([0-9]+(\.[0-9]+)?H)?([0-9]+(\.[0-9]+)?M)?([0-9]+S)?)?)|[0-9]+(\.[0-9]+)?W))|((((([0-9][0-9][0-9][0-9])((-(([0][1-9])|([1][0-2])))((-(([0][1-9])|([1-2][0-9])|([3][0-1])))(T((([0-1][0-9])|([2][0-3]))((:([0-5][0-9]))(((:([0-5][0-9]))((\.[0-9]+)?))?)?)?((((\+|-)(([0-1][0-9])|([2][0-3])):[0-5][0-9])|(Z)))?))?)?)?))/((([0-9][0-9][0-9][0-9])((-(([0][1-9])|([1][0-2])))((-(([0][1-9])|([1-2][0-9])|([3][0-1])))(T((([0-1][0-9])|([2][0-3]))((:([0-5][0-9]))(((:([0-5][0-9]))((\.[0-9]+)?))?)?)?((((\+|-)(([0-1][0-9])|([2][0-3])):[0-5][0-9])|(Z)))?))?)?)?)))|(((([0-9][0-9][0-9][0-9])((-(([0][1-9])|([1][0-2])))((-(([0][1-9])|([1-2][0-9])|([3][0-1])))(T((([0-1][0-9])|([2][0-3]))((:([0-5][0-9]))(((:([0-5][0-9]))((\.[0-9]+)?))?)?)?((((\+|-)(([0-1][0-9])|([2][0-3])):[0-5][0-9])|(Z)))?))?)?)?))/(((\+|-)?P(((([0-9]([0-9]+)?)Y)?(([0-9]([0-9]+)?)M)?(([0-9]([0-9]+)?)D)?)(T((([0-9]([0-9]+)?)H)?(([0-9]([0-9]+)?)M)?(([0-9]([0-9]+)?)((\.[0-9]+)?)S)?)?)?|((([0-9]([0-9]+)?)W))))))|((((\+|-)?P(((([0-9]([0-9]+)?)Y)?(([0-9]([0-9]+)?)M)?(([0-9]([0-9]+)?)D)?)(T((([0-9]([0-9]+)?)H)?(([0-9]([0-9]+)?)M)?(([0-9]([0-9]+)?)((\.[0-9]+)?)S)?)?)?|((([0-9]([0-9]+)?)W)))))/((([0-9][0-9][0-9][0-9])((-(([0][1-9])|([1][0-2])))((-(([0][1-9])|([1-2][0-9])|([3][0-1])))(T((([0-1][0-9])|([2][0-3]))((:([0-5][0-9]))(((:([0-5][0-9]))((\.[0-9]+)?))?)?)?((((\+|-)(([0-1][0-9])|([2][0-3])):[0-5][0-9])|(Z)))?))?)?)?))))

I’m no regular expression expert, but this looks like a configuration issue: although decimal years, months, days, hours and minute are acceptable (using decimal point, not a comma), it appears that the allowance for decimal seconds is missing from the standard representation of duration (i.e. PnYnMnDTnHnMnS) - for seconds to be allowed, I think the highlighted part ought to be ([0-9]+(\.[0-9]+)?S)?

(It should also be noted that this test apparently won’t report an error when a decimal is incorrectly used for a component that is not the lowest order component of the duration - e.g. PT0.5H30S)

Forums: SDTM

s Sergiy

on June 18, 2013

Hi Barry, We did not include

Hi Barry,

We did not include fractions of seconds into duration, because SDTM IG specifies a use of two-digits only for seconds.

My understanding is that CDISC SDTM standard utilized only some features of ISO8601 format.

Nevertheless I do not see how a fraction of seconds can be damaging for SDTM standard compliance. The only potential risk is an architecture of the FDA Janus data warehouse in a case if that system has any special relevant limitations.

Meanwhile we will modify OpenCDISC specifications to accept fractions of seconds.

Thank you for your suggestion!
Regards,

Sergiy

k Karen

on March 13, 2017

Nine decimal digits

Hi - I have a colleague with a study with MODUR durations in nanoseconds. Here's a sample value: PT0.404328791S. It's true, valid data. Your software is flagging it as an error, SD1011. Would you please tweak the (already very impressive) regular expression to allow nine digits?

Thank you, Karen

j Jozef

on March 14, 2017

ISO 8601 duration

I don't think it is a good idea to do this test using a regular expression. There are much better ways:

- using XQuery:
$duration := 'PT1H2M34.123456789S'
where not($duration castable as xs:duration)
(: give an error message :)
...

- using Java:
java.time.Duration d;
try {
    d = java.time.duration.parse("PT1H2M34.123456789S")
} catch (Exception e) {
    // invalid duration - throw an error
    ...
}