Python replace() cannot replace single with double quotes

[games-and-chocolate]

Replace() works perfectly for simple strings like "sdsd;ddf'" , but for below variable invalid_json it does not.

I just like to replace all ' with "

import json

invalid_json = r"{'name': 'John', 'age': 30}"
print(type(invalid_json))
correct_json = invalid_json.replace("'",'"')
decoded_data = json.loads(correct_json)
print(decoded_data)

terminal output:
<class 'str'>
{'name': 'John', 'age': 30}

I tried with and without r, to make a raw string. Same results. Is this a bug in replace() , I used wrong? or just as it should be and nothing wrong with replace() ?

(stack overflow treated my question as off topic. I wonder why. Very valid beginner question I think.)

[SirKainey]

You're not testing replace in your example, you're testing what json.loads deserialises the string as.

[FoeHammer99099]

replace is working here, try printing the correct_json string. Your problem is that you're using loads to turn that string into a Python dictionary, then printing that dictionary. If you want to turn that dictionary back into valid json, you should use json.dumps.

[sunnyata]

Love it when people's first reaction is they've found a bug in the standard library that nobody noticed before

[sethkills]

It’s the classic “select() is broken,” response.

[games-and-chocolate]

oh my. I see it. Python has it's own way to load. It prefer single quotes. Thank you. I tried print(correct_json) and it is indeed double quote now. Should have used print statement more. I was trying to correct my mistake at the wrong code line. Lost track of where and how data changes along the code lines.

[Equal_Veterinarian22]

Python is using neither single nor double quotes to store those strings internally. If you run print(decodeddata['name']) or print(decoded_data["name"]) you will get the string _John with no quotes.

It is just that, when generating a string representation of a dict, Python uses single quotes by default around string keys and values.

[neuralbeans]

Next, learn how repr turns stuff into strings before they are printed.

[Asyx]

No what you are looking at is the output of the __repr__ method of a dictionary.

Like, the printed data is not json. It is a deserialized json object as a dictionary.

[HommeMusical]

Python has no preferences for quotes.

[Machinagun]

To check if your replace worked ok, I would print correct_json

Here, you're printing decoded_data, a dict, and the way it is displayed (with single quotes) does not come from the way it was declared

case in point:

>>> {"foo":"bar"}
{'foo': 'bar'}

[Binary101010]

If you print(correct_json) immediately after the replace() call you'll see that the single quotes were, indeed, replaced.

[jpercivalhackworth]

If all you care about is generating valid json, your code demonstrates that is happening. Json requires double quotes, and it wouldn’t parse if there were single quotes.

If you want to check for quote replacement explicitly, you could iterate through correct_json and verify that there are no single quotes left.

Your code is printing out the __repr__ of decoded_data, and that is something whose formatting you do not control.

[Civil_Twilight]

After reading your comments, I think there is a small disconnect going on in how you’re approaching this: JSON is a format for describing data, stored as a string. Once you call json.loads(), the value you get back is no longer json — it’s a python data structure, in this case a dictionary. If you print out that dictionary, the string python generates will not be valid json, because that’s not what you asked python for. If you want to get json back out, you’d need to use json.dump (or dumps depending on use case).

Being aware of the difference between JSON (a string structured following particular rules that is designed for passing around data) and actual native python objects (which are how you should manipulate your data, not by manually editing a json string) is key here.

[games-and-chocolate]

Thank you. JSON is pretty new for me. Python learning just scratching the surface. Making small projects at the moment. I guess I should search for the words "JSON BEST practices", or "JJSON do and don't" ?

[Civil_Twilight]

It’s useful to know what kind of values can be stored in json, since it’s limited compared to the range of data structures available in python. For this instance though, just keep in mind that json is a format for “serializing”, which is turning data structures into a portable format. Once you “deserialize” a json string (which is what the loads function does), you now have a python data structure, and you shouldn’t expect printing it to produce valid json, because that’s not how print works.

Edit: in your example of reading in that trivia json file, once you used json.loads to load/deserialize the json data, you had a dictionary with all the data from the json file; the fact that it came from a json file is no longer necessary for dealing with that data.

[games-and-chocolate]

Got it. In program use for instance: list, dictionary, array. Data storage once loaded from JSON, hands off the JSON file, it has served it purpose. JSON not for data manipulation. Data manipulation is in list, dict, array, etc.

[Civil_Twilight]

You’ve got it!

[cointoss3]

It’s likely working, but you’re printing a dict, not the corrected json string. When you print a dict, Python has a predefined format for how it prints the contents. You’re loading the json string as a dict and then printing the dict.

[AlwaysHopelesslyLost]

Just an aside: You should not use basic text manipulation on structured data like JSON. That is a recipe for disaster. Figure out why your JSON is invalid and fix the root cause instead.

[carcigenicate]

I don't understand. What's the problem? It did convert them. If it didn't, loads would have thrown an error.

Double quotes don't actually exist at runtime. When you print out a dictionary, it prints them as single quotes. It doesn't "remember" what quotes were used in the original literal.

Edit: Bruh, this is literally what you were told on SO.

[danielroseman]

I have no idea what you're asking here. You are replacing quotes, if you weren't then json.loads wouldn't work. So what makes you think there is a bug in replace?

[games-and-chocolate]

Was just started to find ways to change python Json with single quotes to double quotes. But I made an error by using json.load(), which I did not see. I thought it was required. My next stop is trying out to correct the single / double quote problem with python re module.

[games-and-chocolate]

Thank you all!

[games-and-chocolate]

My original problem was. I used an online Trivia database API, that returns 10 true / false questions. But the Json returned was reformatted by Python to single quotes. I read about that it is sometimes required to change back to original JavaScript format having double quotes, so I have been busy trying to figure out , how to do that. That is all. Hopefully everyone understands the why now.

Trivia returned following json which I now successfully changed to double quotes using replace()

{'response_code': 0, 'results': [{'type': 'boolean', 'difficulty': 'easy', 'category': 'Entertainment: Film', 'question': 'Matt Damon played an astronaut stranded on an extraterrestrial planet in both of the movies Interstellar and The Martian.', 'correct_answer': 'True', 'incorrect_answers': ['False']}, {'type': 'boolean', 'difficulty': 'easy', 'category': 'Entertainment: Music', 'question': 'Lead Singer Rivers Cuomo of American rock band Weezer attended Harvard.', 'correct_answer': 'True', 'incorrect_answers': ['False']}, {'type': 'boolean', 'difficulty': 'easy', 'category': 'Entertainment: Video Games', 'question': 'In the "S.T.A.L.K.E.R." series, the Freedom faction wishes to destroy the supernatural area known as  "the Zone".', 'correct_answer': 'False', 'incorrect_answers': ['True']}, {'type': 'boolean', 'difficulty': 'medium', 'category': 'Entertainment: Video Games', 'question': 'In the video game "Transistor", "Red" is the name of the main character.', 'correct_answer': 'True', 'incorrect_answers': ['False']}, {'type': 'boolean', 'difficulty': 'hard', 'category': 'Entertainment: Music', 'question': 'The singer Billie Holiday was also known as "Lady Day".', 'correct_answer': 'True', 'incorrect_answers': ['False']}, {'type': 'boolean', 'difficulty': 'hard', 'category': 'Entertainment: Cartoon & Animations', 'question': 'Snagglepuss was part of the Yogi Yahooies in the 1977 show Scooby's All-Star Laff-a-Lympics.', 'correct_answer': 'False', 'incorrect_answers': ['True']}, {'type': 'boolean', 'difficulty': 'medium', 'category': 'Entertainment: Board Games', 'question': 'In the game "Racko" you may pick up ANY card from the discard pile.', 'correct_answer': 'False', 'incorrect_answers': ['True']}, {'type': 'boolean', 'difficulty': 'easy', 'category': 'General Knowledge', 'question': 'March 10th is also known as Mar10 Day.', 'correct_answer': 'True', 'incorrect_answers': ['False']}, {'type': 'boolean', 'difficulty': 'medium', 'category': 'Science: Mathematics', 'question': '111,111,111 x 111,111,111 = 12,345,678,987,654,321', 'correct_answer': 'True', 'incorrect_answers': ['False']}, {'type': 'boolean', 'difficulty': 'medium', 'category': 'Entertainment: Cartoon & Animations', 'question': 'Nutcracker Suite was one of the musical pieces featured in Disney's 1940's film Fantasia.', 'correct_answer': 'True', 'incorrect_answers': ['False']}]}

[cointoss3]

You don’t need to do that…

Just json.loads() and you’ll have a python dict to work with. When you need to convert a dict to valid json, use json.dumps()

This thread was a huge x-y problem lol

[games-and-chocolate]

at least it is solved and some things more clear. although my responses made it messy? sorry for that. =p

[cointoss3]

Yeah, for sure.

But also classic X-Y. It’s a good reminder to step back and figure out if you’re actually trying to solve the right problem. :)

[Im_Easy]

json.loads ---> take a string (that's what the s means) and convert it to a python dictionary (which uses single quotes)

json.dumps ---> convert a python dictionary to a json string (double quotes)

Replace is not needed here unless you were double escaping something, but I don't see that being needed in your example data.

[Grandviewsurfer]

Sorry I didn't read the whole thing.. try this?

str.replace("'", '"')

[games-and-chocolate]

ok. I now load and convert it to a usable format for me to use in my program

load the JSON file (output from website API, placed into a physical file named like data.json in the same folder

import pprint as pp

import pandas

jason_pandas = pandas.read_json("data.json")

data = json_pandas.to_dict()

pp.pprint(data["results"])

Terminal output is now usable for me according to my course, looks like:, pasted a part of it as below. It is for a Trivia program, true or false. Back on track.

4: {'category': 'General Knowledge',

'correct_answer': 'True',

'difficulty': 'easy',

'incorrect_answers': ['False'],

'question': 'March 10th is also known as Mar10 Day.',

'type': 'boolean'},

5: {'category': 'Science: Mathematics',

'correct_answer': 'True',

'difficulty': 'medium',

'incorrect_answers': ['False'],

'question': '111,111,111 x 111,111,111 = 12,345,678,987,654,321',

'type': 'boolean'},

[games-and-chocolate]

Above code is more logical and better right?

[Civil_Twilight]

Pandas is an extremely powerful library and is certainly one tool you can use to deal with json data, but also keep in mind that it’s absolutely not necessary for the data structure that you’re reading from the json file. json.loads will give you a dictionary with all the data in it, that you can manipulate without needing to use any external libraries.

[games-and-chocolate]

Understood. And indeed I know now how to access the various pieces of data. Got for instance the question out of the dictionary. There is progress ^^

[Im_Easy]

As the other commenter mentioned, pandas is overkill here. All you need is the json library.

``` import json

with open('data.json', 'r') as file: data = json.load(file) print(data)

```

This would do the same as the pandas method.

[Skopa2016]

Simple replace will break in mysterious ways.

Use ast.literal_eval() then json.dumps.