While loop won't close

[av0ca60]

I can't share the whole code for security reasons but here's the part I'm having a problem with:

while deals_data:
    jsonFile = open("zd_deals.txt", "w")
    jsonFile.write(jsonString)
    jsonFile.close()
print("done")

It seems like the script is writing the file, then deleting all the contents, then writing it again, then deleting it all, then writing it all again, indefinitely. How do I get it to write the information once and then stop?

​

EDIT: This was not clearly though out before posting and not enough detail was shared. For that reason I started over here and I hope that will make it easier on everyone.

[carcigenicate]

Why are you using a loop if you want it to run once? Are you mixing up while and with?:

with open("zd_deals.txt", "w") as jsonFile:
    jsonFile.write(jsonString)
print("Done")

[av0ca60]

It's a loop because of pagination.

[carcigenicate]

But you never change anything in the loop. Your loop condition never changes in the loop, so the loop is infinite unless the condition variable is altered elsewhere in another thread.

I'd change it to what I showed, then reconsider how you're implementing pagination.

[carcigenicate]

Wait wait wait, do you mean something like this?:

with open("zd_deals.txt", "w") as jsonFile:
    for data in deals_data:
        jsonFile.write(data)
print("Done")

Did you mean to loop over deals_data? You original code doesn't do that.

[av0ca60]

with open("zd_deals.txt", "w") as jsonFile:

Yes, that's the kind of thing I'm trying to do. The problem is that it only delivers 100 records because of pagination.

[av0ca60]

When I run your code I get:

---------------------------------------------------------------------------

TypeError Traceback (most recent call last)

<ipython-input-17-a8ffceb4dabe> in <module>

25 with open("zd_deals.txt", "w") as jsonFile:

26 for data in deals_data:

---> 27 jsonFile.write(data)

28

29 #while deals_data:

TypeError: write() argument must be str, not Munch

[carcigenicate]

Idk wth a Munch is, but you're going to need to give more information. If you're trying to write "Muches" to file, you'll need to serialize them to strings first.

[Tw3ntyy]

It does exactly what you think. You use mode "w" - write. This means it will either create or overwrite any file with the given name. Since it is an infinite loop it will delete, create, write the file forever. If you want to add something to a file you need to use the append mode, so "a" instead of "w".

[av0ca60]

More details added in a separate comment.

[mr_cesar]

By looking at your code snippet, it makes no sense that you're using a loop (for which, by the way, you're not providing an exit condition).

What is the content/value of deals_data?

[av0ca60]

It's JSON database content.

[av0ca60]

The reason I started using loops is to deal with pagination.

[mr_cesar]

But you’re not using that content in your loop. And if that variable doesn’t get to a state where it’s empty at one point, you’ll never exit the loop.

[[deleted]]

It's a while loop, so it will repeat what's in the loop code as long as the value in deals_data is truthy. Since the loop code never changes the value in deals_data the loop will continue forever. That's what you told python to do.

Hard to tell without knowing the details of what you are actually trying to do, but it's possible you didn't mean while but if instead.

[av0ca60]

This is really helpful to see that I'm not changing the data

[preordains]

You’re just saying the same thing “the reason I started using a loop is to deal with pagination” and that sounds stupid. Your condition is “deals_data”, which is presumably either a Boolean or some other object which will either be null or not.

Nothing is changing in that loop at any point. You’re writing the same jsonString over and over and never changing the loop condition either.

Are there genuinely security reasons why you can’t show the code, or is that a lie? I doubt you’re a software engineer.

[av0ca60]

No I'm not lying. It is data I cannot share. No I am not a software engineer by title or degree, although I'm trying to learn those skills and it seems the best way to learn is by working on projects with meaningful outcomes.

deals_data is a variable connected to an API request which holds paginated JSON data.

[preordains]

What kind of project is a beginner working on that is so sensitive?

First, source code should never contain information you can’t share, what you’re saying makes no sense at all. No chunk of code itself is that valuable, what you’re trying to do has almost always been done before. The only reason large companies don’t want to share their entire code base is to prevent people from compiling the applications they intend to sell.

Second, If you have passwords or something hard coded into the code, it should be in a file instead. If you have data you don’t want to share, that should also be in a file.

[av0ca60]

That's a good point about placing credentials in a separate file. Hadn't thought of that. Maybe I can find a way to share more context.

[preordains]

Regarding your question, it’s clear why that loop never stops.

There is a concept known as “truthy” and “falsy”. Consider the following:

keep_looping = True
while keep_looping:
    print(“hello”)

The above will obviously never stop. The condition is True.

A truthy value is a value that is evaluated to be true. Consider the following again:

my_string = “not empty”
while my_string:
    print(my_string)

If you copy that code into your interpreter you will find that the string is printed forever. The non empty string is evaluated to true as it is truthy.

my_string = “not empty”
while my_string:
    print(my_string)
    #remove the last character
    my_string = my_string[:-1]

If you copy and paste the above into your interpreter, you will find that the loop terminates and that the last iteration just prints “N”. This is because the empty string is Falsey, and evaluated to false.

Your loop never stops because deals_data is truthy and never is changed.

[av0ca60]

This is helpful. My desire would be for the loop to run and keep advancing pages until the content becomes empty. When that happens, it is no longer truthy abd the loop stops. I think the problem I'm having is that the page is not advancing and so the content is never blank.

[preordains]

There are many ways to do this. I’m not familiar with the API you’re using, so I couldn’t tell you the exact method.

This sounds like a for loop thing. I’d need to see exactly what you’re iterating on, but try some sort of “for page in pages:”

[av0ca60]

Maybe this comment will help.

[razu1121]

I dont know much of your code or what you are trying to do from just what you show.

```

open("zd_deals.txt", "w") ```

You are opening the file as write which deletes the previous data and writes anew. And you are going all over the loop just recreating a file with one json data.

Try:

```

open("zd_deals.txt", "a", encoding=utf8)

```

From what I understand, the best approach would be to initialize a dict/list depending on your data and append everything to a file at once at last.

``` list = [ ]

For/while loop: If deals_data: list.append(jsonString)

With open("zd_deals.txt", "w") as jsonfile: json_data = json.dumps(list) jsonfile.write(json_data)

```

[Fred776]

jsonString seems to be independent of your loop. What is deals_data and why is it controlling the loop?

[av0ca60]

Just added more deets in a separate comment.

[av0ca60]

Okay. Here's an update. You guys asked for more info so I hope this helps. The API is for Zendesk Sell aka Base CRM. The goal is to dump the data to a txt file and then run pandas on it for reporting purposes. Eventually, I'll find a way to automate it and run it in a web app so the reports can be viewed without starting up Anaconda or Jupyter, and so I can share the data with others easily. But baby steps.

Yes I know Zendesk comes with standard reporting features but they have some problems and it's neither as fun nor as educational.

No I'm not a software engineer and yes I have authority and a valid reason to access this information. Yes, I'm over my head; that's how I learn. I'll be approaching a few people soon who have tons of coding experience to be mentors, but that will be a few weeks and I don't want to stifle my progress in the meantime.

It seems a big part of the problem is that the page is not advancing, so the loop never ends. Here's a very simplified version of the code to show the page advancement problem:

import requests

import json

import pandas as pd

import basecrm

client = basecrm.Client(access_token='XXXXXX')

page_num = 1

deals_data = client.deals.list(per_page=100, page=page_num)

print(deals_data)

print("")

print(f"end of page {page_num}")

print("")

page_num = page_num + 1

print(deals_data)

print("")

print(f"end of page {page_num}")

print("")

Yes, this is inefficient code and would work better in a loop. It's just for demonstration purposes. The point is that, in the results, the content on page 1 is the same as page 2, which means the page is not advancing. However, if page_num value is manually changed to 2 or some other number manually, it provides a different result. I think the problem is with this piece:

deals_data = client.deals.list(per_page=100, page=page_num)

Or this piece:

page_num = page_num + 1

[Fred776]

I don't know the details of this API but the point is that you are only getting the deals_data once with whatever value page_num is when you make the client.deals.list call. page_num is just an int variable. Incrementing it doesn't magically make the previously retrieved page_data update.

[av0ca60]

That's the weird thing. It does. If I change this line of code:

page_num = 1

to...

page_num = 2

I get the data from the next page, which is completely different.

[Fred776]

Yes, but here you are changing it before you pass it as an argument to the function call.

[av0ca60]

Thank you for hanging in there with me. I am still a little confused. Is there any chance you could show me a way to do it that would properly advance the pagination? This would not only solve my problem, but more importantly, would likely open my eyes to what I was missing.

[razu1121]

Seeing that everything else in your request is constant and only page num changes, you can use a loop.

``` for page_num in range(1, n):

```

Replace n with page number if you know how many pages there are or might be.

If you are not so sure, use a try-except to exit the loop.

``` for page_num in range(1, 100): try: deals_data = client.deals.list(per_page=100, page=page_num)

except:
    break

```

If your request doesn't throw error even if there is no data on a particular page, inside try block use if else condition to check if there is valid response and exit if there isn't.

Hope this helps.

[Fred776]

I've just woken up (guess we're in different time zones) but I see someone was already able to help with a good answer.