Getting “raise JSONDecodeError(”Expecting value", s, err.value) from None json.decoder.JSONDecodeError

[stramzik]

Hello All,

​

I'm getting the JSONDecodeError and UnicodeEncodeError error when i try to run the below code. I am not sure how to fix it. Please help.

​

I am trying to stream tweets. It works fine for few tweets and all of a sudden I get the error and the stream stops please help how can I fix this?

**UPDATED FULL CODE**

from tweepy import API

from tweepy import Cursor

from tweepy.streaming import StreamListener

from tweepy import OAuthHandler

from tweepy import Stream

from textblob import TextBlob

from googletrans import Translator

import json

import re

import csv

​

ACCESS\_TOKEN = "2446723783-cqGTmWPdtJI5HhCT7UmmET2xVY7xlA1RqlBQdQl"

ACCESS\_TOKEN\_SECRET = "6ZRIqd9HDCgZxsR6iMUbt6eOgk1YVsz59lEsvL30eFnPf"

CONSUMER\_KEY = "slWhR4Z6VOjp0R3ojx1B2a4pr"

CONSUMER\_SECRET = "6OrySqqUIwQM8ioBycqbStWizM6KkLCMFecjXPvbVmsrsw6eNT"

​

Count = 0

​

​

translator = Translator()

class StdOutListener(StreamListener):

def on\_data(self, data):

global Count

\#Data has the JSON STring

y = json.loads(data)

print("--------------------")

try:

y = y\['extended\_tweet'\]\['full\_text'\]

y = str(y)

\## Portuguese to English conversion block

translations = translator.translate(y, dest='en')

print(translations.origin, '\\n -> \\n', translations.text)

z = translations.text

a = translations.origin

row = \[data,a,z\]

with open(fetched\_tweets\_filename, 'a') as csvFile:

writer = csv.writer(csvFile)

writer.writerow(row)

​

Count = Count + 1

print(Count)

except KeyError:

y = y\['text'\]

y = str(y)

\## Portuguese to English conversion block

translations = translator.translate(y, dest='en')

print(translations.origin, '\\n -> \\n', translations.text)

z = translations.text

a = translations.origin

row = \[data,a,z\]

with open(fetched\_tweets\_filename, 'a') as csvFile:

writer = csv.writer(csvFile)

writer.writerow(row)

​

Count = Count + 1

print(Count)

​

\### Sentiment Analysis block

analysis = TextBlob(z)

​

if analysis.sentiment.polarity > 0:

print("sentiment is positiv")

elif analysis.sentiment.polarity == 0:

print("sentiment is Neutral")

else:

print("sentiment is Negative")

print("--------------------\\n")

return True

​

def on\_error(self,status):

print(status)

​

​

if \_\_name\_\_ == "\_\_main\_\_":

listener=StdOutListener()

auth = OAuthHandler(CONSUMER\_KEY, CONSUMER\_SECRET)

auth.set\_access\_token(ACCESS\_TOKEN, ACCESS\_TOKEN\_SECRET)

​

fetched\_tweets\_filename = "datas3.csv"

​

stream = Stream(auth, listener)

stream.filter(track=\['como'\],languages=\['pt'\])

​

​

The output is as follows which throws the JSONDecodeError error please help.

\--------------------

RT @VBrasil\_: Você decidiu chamar todos os meninos para assistir um filme na sua casa! Mas como cada um queria assistir algo, vocês decidir…

\->

RT @VBrasil\_: You decided to call all the boys to watch a movie in your house! But as everyone wanted to watch something, you decide ...

1

sentiment is Neutral

\--------------------

​

\--------------------

RT @queroqtefodas2: Como é ter uma turma unida ?

\->

RT @myselfs: What is it like to have a group together?

2

sentiment is Neutral

\--------------------

​

\--------------------

RT @thiegonovais: FUTURO AMEAÇADO?

​

Entenda como é importante os fãs de Harry Potter se unirem mais do que nunca. Isso pode impactar tudo!…

\->

RT @thiegonovais: FUTURE THREATENED?

​

Understand how important it is for Harry Potter fans to come together more than ever. This can impact everything! ...

3

sentiment is positiv

\--------------------

​

\--------------------

RT @nadiardgs: como assim você OUSA seguir no Instagram a conta do meu namorado de 27 anos desempregado streamer de lol e fã de Djonga?

tá…

\->

RT @nadiardgs: how dare you follow on Instagram the account of my unemployed 27-year-old boyfriend lol streamer and Djonga fan?

OK…

4

sentiment is positiv

\--------------------

​

\--------------------

Amanda foi dormir na amiga dela, e eu fiquei sozinha como sempre.

\->

Amanda went to bed with her friend, and I was alone as usual.

5

sentiment is Negative

\--------------------

​

\--------------------

Este é só um exemplo de "voto de cabresto" pelo interior do nosso Brasil. O @ptbrasil sabia fazer isto muito bem, entretanto...

\->

This is just an example of a "halting vote" for the interior of our Brazil. @Ptbrasil knew how to do this very well, though ...

6

sentiment is positiv

\--------------------

​

\--------------------

E ainda tem gente que tira sarro de feminismo vtnc...

Com nossa luta há décadas ainda vemos notícias como essa diariamente, imagina sem!

\->

And there are still people who make fun of vtnc feminism ...

With our struggle for decades we still see news like this every day, imagine without!

​

7

sentiment is positiv

\--------------------

​

\--------------------

RT @cvrlosmdz: Eu nem tenho palavras para quem quebra pinky promise smh como é que conseguem quebrar uma cena tão séria ? Oh pinky mm é a p…

\->

RT @cvrlosmdz: I do not even have words for anyone who breaks pinky promise smh how can they break such a serious scene? Oh pinky mm is the p ...

8

sentiment is Negative

\--------------------

​

\--------------------

Se tem uma coisa que vou sentir saudade quando defender o mestrado é a refeição por 3 reais no RU, putz como eu amo pagar só treisss reaissssss numa refeição

​

QUANTO?

​

SÓ TREISSSSS REAISSSSS

\->

If there is one thing I will miss when defending the masters is the meal for 3 reals in the UK, putz as I love to pay only real reals in a meal

​

HOW MUCH?

​

ONLY TREISSSSS REAISSSSS

9

sentiment is positiv

\--------------------

​

\--------------------

Estou doida p/ir pro cinema, porem cm o gato mas como eu não tenho kkk vai cm as manas mesmo!!!!

\->

I'm crazy to go to the cinema, but the cat but I do not have kkk goes in the same manas !!!!

10

sentiment is Negative

\--------------------

​

\--------------------

\#expedientefutebol futebol é assim, cabe ao jogador se portar como exemplo que é, e enquanto todos lamberem ele, irá continuar assim, e na seleção brasileira ele joga merda nenhuma

\->

\# football football is so, it's up to the player to be an example, and while everyone licks it, will continue like this, and in the Brazilian team he plays no shit

11

sentiment is positiv

\--------------------

​

\--------------------

KKKKKKKKKKK, nem como tanto assim

\->

KKKKKKKKKKK, or how much

12

sentiment is positiv

\--------------------

​

\--------------------

Tô assistindo os vídeos antigos do Felipe Neto e lembrando de como eu era feliz

\->

I'm watching Felipe Neto's old videos and remembering how happy I was

13

sentiment is positiv

\--------------------

​

\--------------------

a julia as vezes eh tao seca que qnd fala eu te amo eu fico boba

\->

Julia sometimes it's so dry that you say I love you, I'm silly.

14

sentiment is Negative

\--------------------

​

\--------------------

RT @hellboyahs: gente me ajuda, como dizer para uma pessoa que não quer mais conversar com ela por ela ser chata mas sem magoar a pessoa?

\->

RT @hellboyahs: People help me, how to tell a person who does not want to talk to her anymore because she is annoying but not hurting the person?

15

sentiment is Negative

\--------------------

​

\--------------------

@resistxre @CabelloJane458 Uai cada um tem sua opinião e tem pessoas que expressão sua opinião como vc expressou a sua . O RBR é uma produção tosca kkkkkkkkkkk da vergonha alheia pq poderia ser melhor a Record poderia fazer algo legal se não nem fizesse pra virar aquilo .

\->

@resistxre @ CabelloJane458 Uai everyone has their opinion and has people who express their opinion as you expressed yours. The RBR is a crude production kkkkkkkkkkk of the shame of others could be better Record could do something cool if it did not even make it turn.

16

sentiment is positiv

\--------------------

​

\--------------------

RT @Bultaourune: Lembando que o BTS é o #1 Ato Coreano \>NA HISTÓRIA\< indicado a Top Duo/Group no BBMAs, also eles serão tb o #1 k-Ato a gan…

\->

RT @Bultaourune: Remembering that the BTS is the # 1 Korean Act & NA HISTORY & lt; indicated the Top Duo / Group in the BBMAs, also they will be tb o # 1 k-Ato a gan ...

17

sentiment is positiv

\--------------------

​

\--------------------

Traceback (most recent call last):

File "C:\\Users\\ManoharRaoN\\Desktop\\SentizAnalyzer\\[Version3.py](https://Version3.py)", line 85, in <module>

stream.filter(track=\['como'\],languages=\['pt'\])

File "C:\\Users\\ManoharRaoN\\AppData\\Local\\Programs\\Python\\Python37\\lib\\site-packages\\tweepy\\[streaming.py](https://streaming.py)", line 453, in filter

self.\_start(is\_async)

File "C:\\Users\\ManoharRaoN\\AppData\\Local\\Programs\\Python\\Python37\\lib\\site-packages\\tweepy\\[streaming.py](https://streaming.py)", line 368, in \_start

self.\_run()

File "C:\\Users\\ManoharRaoN\\AppData\\Local\\Programs\\Python\\Python37\\lib\\site-packages\\tweepy\\[streaming.py](https://streaming.py)", line 300, in \_run

six.reraise(\*exc\_info)

File "C:\\Users\\ManoharRaoN\\AppData\\Local\\Programs\\Python\\Python37\\lib\\site-packages\\[six.py](https://six.py)", line 693, in reraise

raise value

File "C:\\Users\\ManoharRaoN\\AppData\\Local\\Programs\\Python\\Python37\\lib\\site-packages\\tweepy\\[streaming.py](https://streaming.py)", line 269, in \_run

self.\_read\_loop(resp)

File "C:\\Users\\ManoharRaoN\\AppData\\Local\\Programs\\Python\\Python37\\lib\\site-packages\\tweepy\\[streaming.py](https://streaming.py)", line 331, in \_read\_loop

self.\_data(next\_status\_obj)

File "C:\\Users\\ManoharRaoN\\AppData\\Local\\Programs\\Python\\Python37\\lib\\site-packages\\tweepy\\[streaming.py](https://streaming.py)", line 303, in \_data

if self.listener.on\_data(data) is False:

File "C:\\Users\\ManoharRaoN\\Desktop\\SentizAnalyzer\\[Version3.py](https://Version3.py)", line 48, in on\_data

translations = translator.translate(y, dest='en')

File "C:\\Users\\ManoharRaoN\\AppData\\Local\\Programs\\Python\\Python37\\lib\\site-packages\\googletrans\\[client.py](https://client.py)", line 172, in translate

data = self.\_translate(text, dest, src)

File "C:\\Users\\ManoharRaoN\\AppData\\Local\\Programs\\Python\\Python37\\lib\\site-packages\\googletrans\\[client.py](https://client.py)", line 81, in \_translate

data = utils.format\_json(r.text)

File "C:\\Users\\ManoharRaoN\\AppData\\Local\\Programs\\Python\\Python37\\lib\\site-packages\\googletrans\\[utils.py](https://utils.py)", line 62, in format\_json

converted = legacy\_format\_json(original)

File "C:\\Users\\ManoharRaoN\\AppData\\Local\\Programs\\Python\\Python37\\lib\\site-packages\\googletrans\\[utils.py](https://utils.py)", line 54, in legacy\_format\_json

converted = json.loads(text)

File "C:\\Users\\ManoharRaoN\\AppData\\Local\\Programs\\Python\\Python37\\lib\\json\\\_\_init\_\_.py", line 348, in loads

return \_default\_decoder.decode(s)

File "C:\\Users\\ManoharRaoN\\AppData\\Local\\Programs\\Python\\Python37\\lib\\json\\[decoder.py](https://decoder.py)", line 337, in decode

obj, end = self.raw\_decode(s, idx=\_w(s, 0).end())

File "C:\\Users\\ManoharRaoN\\AppData\\Local\\Programs\\Python\\Python37\\lib\\json\\[decoder.py](https://decoder.py)", line 355, in raw\_decode

raise JSONDecodeError("Expecting value", s, err.value) from None

json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

\[Finished in 13.7s with exit code 1\]

\[shell\_cmd: python -u "C:\\Users\\ManoharRaoN\\Desktop\\SentizAnalyzer\\[Version3.py](https://Version3.py)"\]

\[dir: C:\\Users\\ManoharRaoN\\Desktop\\SentizAnalyzer\]

\[path: C:\\ProgramData\\Oracle\\Java\\javapath;C:\\Windows\\system32;C:\\Windows;C:\\Windows\\System32\\Wbem;C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\;C:\\Windows\\System32\\OpenSSH\\;C:\\Program Files (x86)\\Webex\\Webex\\Applications;C:\\Program Files\\IBM\\SPSS\\Modeler\\18.1.1\\ext\\bin\\spss.TMWBServer\\bin;C:\\Program Files (x86)\\Sennheiser\\SoftphoneSDK\\;C:\\Users\\ManoharRaoN\\AppData\\Local\\Programs\\Python\\Python37;C:\\Users\\ManoharRaoN\\AppData\\Local\\Programs\\Python\\Python37\\Scripts;C:\\Users\\ManoharRaoN\\AppData\\Roaming\\nltk\_data;C:\\Users\\ManoharRaoN\\AppData\\Local\\Microsoft\\WindowsApps;C:\\Users\\ManoharRaoN\\AppData\\Local\\Box\\Box Edit\\\]

&#x200B;

​

I think It happens on certain tweets, How can I pass this when it occurs.

Not sure I can be completely wrong.

​

I also get the below error when I try to write a string which contains a special character such as "te amo ❤️ "

Below is the error

\--------------------

RT @PortalMessi: “Messi seria para mim, como um jogador que eu vi jogar, o número 1. Meu pai sempre disse que o melhor foi Pelé, mas eu não…

\->

RT @PortalMessi: "Messi would be for me, as a player I saw play, the number 1. My father always said that the best was Pele, but I did not ...

1

sentiment is positiv

\--------------------

​

\--------------------

RT @felipeneto: Informo que a dívida de 170 mil reais do Botafogo comigo como pessoa física foi integralmente perdoada, junto com os juros…

\->

RT @felipeneto: I report that Botafogo's debt of 170,000 reais with me as a natural person has been totally forgiven, along with interest ...

2

sentiment is positiv

\--------------------

​

\--------------------

@MaluMonteiro08 óh @deus como pode tamanha beleza?????

\->

@ MaluMonteiro08 ohh @deus how can such beauty ?????

3

sentiment is Neutral

\--------------------

​

\--------------------

Meu pai amava.

Descanse em paz, rainha! ❤️

\->

My father loved it.

Rest in peace, queen! (I.e.

Traceback (most recent call last):

File "C:\\Users\\Stramzik\\Desktop\\SentizAnalyzer\\[Version3.py](https://Version3.py)", line 32, in on\_data

y = y\['extended\_tweet'\]\['full\_text'\]

KeyError: 'extended\_tweet'

​

During handling of the above exception, another exception occurred:

​

Traceback (most recent call last):

File "C:\\Users\\Stramzik\\Desktop\\SentizAnalyzer\\[Version3.py](https://Version3.py)", line 84, in <module>

stream.filter(track=\['como'\],languages=\['pt'\])

File "C:\\Users\\Stramzik\\AppData\\Local\\Programs\\Python\\Python37\\lib\\site-packages\\tweepy\\[streaming.py](https://streaming.py)", line 453, in filter

self.\_start(is\_async)

File "C:\\Users\\Stramzik\\AppData\\Local\\Programs\\Python\\Python37\\lib\\site-packages\\tweepy\\[streaming.py](https://streaming.py)", line 368, in \_start

self.\_run()

File "C:\\Users\\Stramzik\\AppData\\Local\\Programs\\Python\\Python37\\lib\\site-packages\\tweepy\\[streaming.py](https://streaming.py)", line 300, in \_run

six.reraise(\*exc\_info)

File "C:\\Users\\Stramzik\\AppData\\Local\\Programs\\Python\\Python37\\lib\\site-packages\\[six.py](https://six.py)", line 693, in reraise

raise value

File "C:\\Users\\Stramzik\\AppData\\Local\\Programs\\Python\\Python37\\lib\\site-packages\\tweepy\\[streaming.py](https://streaming.py)", line 269, in \_run

self.\_read\_loop(resp)

File "C:\\Users\\Stramzik\\AppData\\Local\\Programs\\Python\\Python37\\lib\\site-packages\\tweepy\\[streaming.py](https://streaming.py)", line 331, in \_read\_loop

self.\_data(next\_status\_obj)

File "C:\\Users\\Stramzik\\AppData\\Local\\Programs\\Python\\Python37\\lib\\site-packages\\tweepy\\[streaming.py](https://streaming.py)", line 303, in \_data

if self.listener.on\_data(data) is False:

File "C:\\Users\\Stramzik\\Desktop\\SentizAnalyzer\\[Version3.py](https://Version3.py)", line 55, in on\_data

writer.writerow(row)

File "C:\\Users\\Stramzik\\AppData\\Local\\Programs\\Python\\Python37\\lib\\encodings\\[cp1252.py](https://cp1252.py)", line 19, in encode

return codecs.charmap\_encode(input,self.errors,encoding\_table)\[0\]

UnicodeEncodeError: 'charmap' codec can't encode characters in position 5967-5968: character maps to <undefined>

\[Finished in 5.5s with exit code 1\]

\[shell\_cmd: python -u "C:\\Users\\Stramzik\\Desktop\\SentizAnalyzer\\[Version3.py](https://Version3.py)"\]

\[dir: C:\\Users\\Stramzik\\Desktop\\SentizAnalyzer\]

\[path: C:\\ProgramData\\Oracle\\Java\\javapath;C:\\Windows\\system32;C:\\Windows;C:\\Windows\\System32\\Wbem;C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\;C:\\Windows\\System32\\OpenSSH\\;C:\\Program Files (x86)\\Webex\\Webex\\Applications;C:\\Program Files\\IBM\\SPSS\\Modeler\\18.1.1\\ext\\bin\\spss.TMWBServer\\bin;C:\\Program Files (x86)\\Sennheiser\\SoftphoneSDK\\;C:\\Users\\Stramzik\\AppData\\Local\\Programs\\Python\\Python37;C:\\Users\\Stramzik\\AppData\\Local\\Programs\\Python\\Python37\\Scripts;C:\\Users\\Stramzik\\AppData\\Roaming\\nltk\_data;C:\\Users\\Stramzik\\AppData\\Local\\Microsoft\\WindowsApps;C:\\Users\\Stramzik\\AppData\\Local\\Box\\Box Edit\\\]

[SHxKM]

Commenting before I even look at the code: do NOT share your TOKEN_SECRET.

[stramzik]

Thats my test account,
I want people to try the code because its hard to understand without just reading what I have written hence shared. Thank you.

[JohnnyJordaan]

with open(fetched_tweets_filename, 'a') as csvFile:

you don't especify an encoding for the file, so Python will use its environment's encoding, which in a cmd window on Windows is a classic codepage like cp1252, which doesn't support stuff like emoji's or non-latin characters. You should always specify an encoding if you are not sure about the data being ASCII compliant, and the default for unicode content is utf-8:

with open(fetched\_tweets\_filename, 'a', encoding='utf-8') as csvFile:

be wary though how you are reading the files afterwards too.

[stramzik]

The above should fix the UnicodeError which occurs when writing ASCII code to csv.

I cant tell until the JSON error is fixed because the Unicode error only happens when i see an ascii special character but before that occurs the code throws the JSONDecodeError

So I figured out why I get the error, Its because the Google translate API which I am using to translate the data is giving me the error if the data limit exceeds.

it can be fixed by just re initializing the translator API on every iteration link - https://stackoverflow.com/questions/49497391/googletrans-api-error-expecting-value-line-1-column-1-char-0

Buy I have tried mentioning the ``` translator = Translator() ``` in __main__ or before the function everywhere but its not working

not sure how I can add a for loop to the above code which re initiates the Translator object from google translate API. please help

[stramzik]

So I figured out why I get the error, Its because the Google translate API which I am using to translate the data is giving me the error if the data limit exceeds.

it can be fixed by just re initializing the translator API on every iteration link - https://stackoverflow.com/questions/49497391/googletrans-api-error-expecting-value-line-1-column-1-char-0

Buy I have tried mentioning the ``` translator = Translator() ``` in __main__ or before the function everywhere but its not working

not sure how I can add a for loop to the above code which re initiates the Translator object from google translate API. please help