TS 29.571: regex pattern allows all non-empty strings and will not apply restrictions as intended

[davidjwbbc]

Description

Several string types in TS 29.571 use a regular expression pattern to restrict the allowed strings and some of these contain a catch all "^(…|.+)$" style pattern. However the inclusion of "^.+$" as one of the options will override any other rules in the regex.

This happens on the following defined types defined in TS 29.571:

• VarUeId
• VarUeIdRm
• Gpsi
• GpsiRm
• Pei
• PeiRm
• Supi
• SupiRm
• SupiOrSuci

Suggested solution

The ".+" regex option should be preceded with an assertion operator to exclude the prefixes of other parts of the same regex option. For example for the VarUeId an assertion excluding strings starting with "imsi-", "nai-", "msisdn-", "extid-", "gci-" and "gli-" should be included in with the ".+" making that part "(?!(?:imsi|nai|msisdn|extid|gci|gli)-).+" and the whole regex would be

^(imsi-[0-9]{5,15}|nai-.+|msisdn-[0-9]{5,15}|extid-[^@]+@[^@]+|gci-.+|gli-.+|(?!(?:imsi|nai|msisdn|extid|gci|gli)-).+)$

The "(?!…)" operator acts a non-consuming negative assertion meaning that the next part of the string must not match the ellipsis part and this will exclude patterns already matches by previous pattern options. The part matched by the contents of "(?!…)" are not counted towards the final pattern match and are just an assertion about the string contents to match.

Additional context

This issue also applies to some types in the following specifications too:

• TS 29.503
• TS 29.504
• TS 29.505
• TS 29.518
• TS 29.562

[rjb1000]

TS 29.571 is under CT4 control, so 5G-MAG will need to identify a member with a CT4 delegate in order to propose this change.

[rjb1000]

The editor of TS 29.571 is Peter Schmitt of Huawei.

@haudiobe will send an e-mail to his CT4 colleague Hanna.

[haudiobe]

Spoke internally, here are some questions and comments:

1. I am not very proficient with regular expression pattern matching so just to make sure I understand the issue: is the problem that, for instance for VarUeId, the inclusion of ^(…|.+)$ at the end of the definition results in any string matching the pattern (even strings which do not start with any of the specified prefixes like imsi, nai, etc)?
2. the description of the issue mentions other specs affected in addition to TS 29.571 (such as TS 29.503) but is it because those other specs re-use types defined in TS 29.571? Or are other types defined in these specs.

Further note: Peter Schmitt is now the CT Chair and does not attend CT4 very much, so we would probably need to contact another one of the Huawei delegates in CT4 if we do not - we correct this ourselves.

[davidjwbbc]

Answers for @haudiobe:

1. So the issue is that the prescribed starts, e.g. "imsi-", "nai-", etc. that have fixed formats can also be matched by the ".+" option in the regex, negating their restrictions on the allowed strings. This means, for example, "imsi-complete-garbage!" would be accepted even though it doesn't match the "imsi-[0-9]{5,15}" rule, and reading the text of the specification indicates that this should not be the case. The ".+" is there for future expansion, but should be restricted to formats not already defined. This is why I suggested adding the negative assertion operator to the ".+" to exclude the other strings already defined.
2. I mentioned the other specifications because they also define types which include a ".+" option in their regex restrictions.

[rjb1000]

Peter Schmitt is now the CT Chair and does not attend CT4 very much, so we would probably need to contact another one of the Huawei delegates in CT4 if we do not - we correct this ourselves.

Do you know if any progress on this issue was made at the CT meetings last week, @haudiobe?

[rjb1000]

This one seems to have fallen down a black hole. Was anything ever done, @haudiobe?

[haudiobe]

This was the response from Peter Schmitt

"Dear Hannah;

If I Understood correctly the intention is to remove the .+
The .+ was added to make the definition extendable and to avoid that any extension would mean backward incompatible change.

Need to check if there are other ways to make the regular expression extendable.

Best regards
Peter "

I sent another message today.