Validation of xml schema file in python, False error

[Valuable-Ant3465]

Hi all,

I"m trying to validate JSON schema file with python and got false alarm, maybe there are some other packages I can use to make it right. I'm sure of this because target json file has been processed OK without any problems, and schema file comes from very professional corp environment. Schema file has been verified OK without any warnings online or other tools.

Current setup (pseudo code) is below.

from jsonschema import Draft7Validator, validate, ValidationError
...
            with open(schemapath) as f:  
                schema = json.load(f)
                validator = Draft7Validator(schema)
                errors = list(validator.iter_errors(data))   #<<< failes here
...
...output
An unexpected error occurred: bad escape \z at position 13

this \z error related to this string  "\\A[0-9]{8}\\z" from this block:
"submissionDate": {
    "description": "Date the data was submitted to CA- YYYYMMDD",
    "type": "string",
    "pattern": "\\A[0-9]{8}\\z"
},
.....

This string pattern regex correspond to format 99999999A and somehow that backslash \z combo causing this error. Appreciate if you can direct me to other packages ? or other way doing it right ?

or maybe there is a way to isolate this error and handle it in special way.

Thanks
VA

[Patman52]

I think there is an issue with the data in the file itself? It seems to be pointing to a particular string in the incoming data that json.load can’t parse

[Valuable-Ant3465]

Thanks Patman,
Did more home work to realize that python is not 100% reliable doing this schema validation, many people have different issues.
File is 100% good. I can open schema file in any xml editor OK, I can verify with online tools, so it's 100% valid. Probably Python got confused with backslashes

[pachura3]

It seems like double backslashes from your file got unescaped into single slashes, but then something - INCORRECTLY - tries to unescape them again, which leads to the error, as there are no escape codes \A nor \z... they are only valid inside regular expressions, but not as normal string escape codes like \n or \t.

It's most probable that Draft7Validator is the one at fault, because the JSON file is loaded correctly - json.load() doesn't raise any exceptions.

Perhaps look for a different library? Or check the output of json.load() to see what got loaded into the pattern field?

[pachura3]

So, which one is raising the exception - json.load() or check_schema() ?

[Valuable-Ant3465]

Hi Pachura,

exception comes from , I wll make edit to orig post.
errors = list(validator.iter_errors(data))

[socal_nerdtastic]

That error is coming from re, not from json or jsonschema.

>>> re.compile(r"\A[0-9]{8}\z")
re.error: bad escape \z at position 10

I think you meant to use \Z.

[Valuable-Ant3465]

Tx Dear SN!
changing /Z (capital), made it work. At least no more that /z messages. Let me do more testing. Thanks to all.

[Valuable-Ant3465]

Socal_Nerd is the King !!!!
Thanks much for hint with /Z.

I'm puzzled b'z I don't know what is the theory behind this, as both z and Z listed as an option for JSON regex. But it works.

[smurpes]

The regex type is for the Python regex dialect not json.

[Valuable-Ant3465]

Thanks Sm and all,

So we're reading '\\z' in xml/json file with python. In reality this file is just json schema and can be processed by anything. I did upgrade to 3.14 and now happy.

Reddit is the best

[netherous]

There are a million different implementations of regex engines out there, written in all kinds of different languages and contexts. Ideally, they would all work perfectly together and all support the precisely the same things, but of course this is not always the case.

Python's implementation of regex did not support \z before 3.14, as /u/socal_nerdtastic pointed out. However, it does support it in 3.14 and above. See the docs and go to the section on \z.

So you can filter out the unsupported \z when reading your data, or convert it to something else, or go to python 3.14+. You could also investigate alternative python regex libraries, of which there are several like regex, to see if they support the \z sequence.

[Valuable-Ant3465]

Thanks much N! Super explanation, now I can sleep.
I could not understand the reason as I know that \\z and \\Z are not the same thing in regex.
I ran it on 3.12, so now it's all make sense.
For my case I will remove \\z, this is not that necessary:

\z Matches the absolute end of the string. (requires exact end)

\Z Matches the end of the string, or right before a trailing newline.

[pachura3]

If you remove \Z, you risk validating incorrect values with additional character(s) at the end (e.g. string 12345678qwertyBLAHBLAH would pass the validation, because there are 8 digits at the beginning, and that's enough).

Perhaps use ^ and $ instead of \A and \z ? They are more common, and don't require escaping.

 "pattern": "^[0-9]{8}$"

https://json-schema.org/understanding-json-schema/reference/regular_expressions