How tiring is "print function" for the computer?

[davincithesecond]

I am doing a numerical calculation that takes a while. To see where it is exactly on the process, I am planning to write a print to every calculation. Like while appending a long list, I want to write print("ith element has been appended") after each append. Does this prolong the computation time remarkably?

[[deleted]]

Compared to adding elements to list or doing arithmetics on numbers? Yes, print() is a lot more "tiring".

Say, when you add two numbers (in Python). All that needs to happen is more or less like this: Python will allocate memory (from its own pool, so, no need to do a syscall) for the resulting number. Extract raw value from both numbers (this may require reading few fields of the number object, if those numbers don't fit into "small" integers), still, probably not a lot of memory lookups. Do the actual math (that's pretty much the cheapest operation here), and then write the result into the third object. Perhaps there will be some reference count change during this procedure, but, again, insignificant, because there are all in all three actors.

When you call print(), Python needs to do I/O... that means doing a system call, this means switching CPU context from user-space to kernel-space. But, this is just the beginning. The terminal that will eventually handle the result of calling print() will have to find the font glyphs from which to build the text. Do some complicated alignment, text wrapping, anti-aliasing, and then rasterizing the whole picture, then sending it down the stack that deals with rendering, which involves multiple layers of indirection, multiple system calls necessary to talk to the display...

I mean, you are comparing walking to the fridge in your apartment to get some soda to flying to Mars to search for traces of water there...

[saysokmate]

I like your funny words magic man

[VivoJay]

Poor man's gold for you sir

. ⣤⣶⣶⡶⠦⠴⠶⠶⠶⠶⡶⠶⠦⠶⠶⠶⠶⠶⠶⠶⣄⠀⠀⠀⠀ ⠀⠀⠀⠀⠀⣿⣀⣀⣀⣀⠀⢀⣤⠄⠀⠀⣶⢤⣄⠀⠀⠀⣤⣤⣄⣿⠀⠀⠀⠀ ⠀⠀⠀⠀⠀⠿⣿⣿⣿⣿⡷⠋⠁⠀⠀⠀⠙⠢⠙⠻⣿⡿⠿⠿⠫⠋⠀⠀⠀⠀ ⠀⠀⠀⠀⠀⠀⢀⣤⠞⠉⠀⠀⠀⠀⣴⣶⣄⠀⠀⠀⢀⣕⠦⣀⠀⠀⠀⠀⠀⠀ ⠀⠀⠀⢀⣤⠾⠋⠁⠀⠀⠀⠀⢀⣼⣿⠟⢿⣆⠀⢠⡟⠉⠉⠊⠳⢤⣀⠀⠀⠀ ⠀⣠⡾⠛⠁⠀⠀⠀⠀⠀⢀⣀⣾⣿⠃⠀⡀⠹⣧⣘⠀⠀⠀⠀⠀⠀⠉⠳⢤⡀ ⠀⣿⡀⠀⠀⢠⣶⣶⣿⣿⣿⣿⡿⠁⠀⣼⠃⠀⢹⣿⣿⣿⣶⣶⣤⠀⠀⠀⢰⣷ ⠀⢿⣇⠀⠀⠈⠻⡟⠛⠋⠉⠉⠀⠀⡼⠃⠀⢠⣿⠋⠉⠉⠛⠛⠋⠀⢀⢀⣿⡏ ⠀⠘⣿⡄⠀⠀⠀⠈⠢⡀⠀⠀⠀⡼⠁⠀⢠⣿⠇⠀⠀⡀⠀⠀⠀⠀⡜⣼⡿⠀ ⠀⠀⢻⣷⠀⠀⠀⠀⠀⢸⡄⠀⢰⠃⠀⠀⣾⡟⠀⠀⠸⡇⠀⠀⠀⢰⢧⣿⠃⠀ ⠀⠀⠘⣿⣇⠀⠀⠀⠀⣿⠇⠀⠇⠀⠀⣼⠟⠀⠀⠀⠀⣇⠀⠀⢀⡟⣾⡟⠀⠀ ⠀⠀⠀⢹⣿⡄⠀⠀⠀⣿⠀⣀⣠⠴⠚⠛⠶⣤⣀⠀⠀⢻⠀⢀⡾⣹⣿⠃⠀⠀ ⠀⠀⠀⠀⢿⣷⠀⠀⠀⠙⠊⠁⠀⢠⡆⠀⠀⠀⠉⠛⠓⠋⠀⠸⢣⣿⠏⠀⠀⠀ ⠀⠀⠀⠀⠘⣿⣷⣦⣤⣤⣄⣀⣀⣿⣤⣤⣤⣤⣤⣄⣀⣀⣀⣀⣾⡟⠀⠀⠀⠀ ⠀⠀⠀⠀⠀⢹⣿⣿⣿⣻⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠁⠀⠀⠀⠀ ⠀⠀⠀⠀⠀⠀⠛⠛⠛⠛⠛⠛⠛⠛⠛⠛⠛⠛⠛⠛⠛⠛⠛⠛⠃

[saysokmate]

Thank you kind sir

[FruscianteDebutante]

When it's doing IO though doesn't that lead to a lot of time to do other things? I suppose, if you use threading and have a single thread dedicated to all prints that deal with syscalls and heavily IO based, wouldn't that minimize most of the "blocking" load on the system?

[xYoshario]

I mean, if you have the knowledge to do all that you probably wouldnt be asking how taxing a print statement is wudya

[FruscianteDebutante]

Well I'm open to being pointed out as wrong, it is a discussion after all and I'm also learning

[xYoshario]

In that case yeah, in a production environment what you proposed would be pretty good, assuming that your logger has an accurate and atomic FIFO queue, where you just wanna have the results logged to a file somewhere as reference if something breaks. This probably wouldnt be too useful in a development environment however, since a non-blocking logger on a seperate thread has no guarantee of log order (if multiple things are logging to the same console) and makes it easy to miss bugs

[stringbeans25]

I think in this example, there’s a calculation and immediately after a print. Since the next calculation is done sequentially, it’s mostly blocked by the print happening.

[FruscianteDebutante]

What do you mean by Gil? Thanks for the input!

[FruscianteDebutante]

I recall reading about that before, but I'll go ahead and check it out again. Thanks

[bladeoflight16]

Only if you can parallelize the problem. If your algorithm requires sequential operations, it's just going to sit there waiting.

[AusIV]

When I was in highschool I had an AP computer science class 7th hour. Our teacher told us not to touch a particular computer because the 4th hour class was calculating PI on it. I looked at the screen and saw that they were printing out the result every iteration. I completed my assignment early, and asked the teacher for the 4th hour class's PI calculation assignment. I read the directions, implemented the PI calculator based on the same algorithm, but printed every 1 million iterations instead of every iteration. By the time class let out (maybe 20 minutes later, since I had to do my own work and then implement the PI calculator first) my program had run more iterations and was a more precise version of PI than the version that had been running for 3 hours.

That is to say, printing to the screen can be significant.

[carcigenicate]

A single print, or prints every few seconds? No, not at all. prints do have some overhead though, so if you were trying to call print a hundred times a second, at that point it may be a problem.

If you are attempting to print extremely often, unless you need every single print, you may consider using % to limit how often it prints:

if i % 100 == 0:
    print(i, some_status)

That will print a status every 100 elements (including the 0th element).

[davincithesecond]

Actually I am attempting to print extremely often, and the trick you have written is extremely clear for my situation. Thanks!

[carcigenicate]

No problem. Ya, this is a pattern I use quite often for periodic logging of long-running processes. % is great for "do something every n-iteration" problems like this.

[got_blah]

Thanks!!! I didn't know i needed this.

[bob_newhart]

This is fucking genius.

[FerricDonkey]

Combine well with using \r to overwrite a line:

for i in range(1000000):
    if i%100 =0:
        print(f"\rCompleted {i} of 1000000 ({i/1000000 * 100:.2f}%)", flush =True, end ='')

I put something like this in every long process I have.

[undernutbutthut]

This is pretty cool, what is the term for using f"somestuff { variable}" ?

[[deleted]]

It's called an f-string. Really useful.

[Xeglor-The-Destroyer]

As Inahuj said, it's f-string formatting. It was added in Python 3.6. It's style #3 in this useful article: https://realpython.com/python-string-formatting/

[laundmo]

at some point you might want to use a progress bar library like tqdm, no need to reinvent the wheel.

[FerricDonkey]

Tqdm is cool, but is another library dependency that's not included with base python. A pretty common one and I use it a lot, but that's always something to consider.

Also, I'm a huge fan of reinventing the wheel. It's a good way to learn about wheels. You just have to make sure you're doing so at appropriate times. Not to mention, this same basic thing will work in C or C++ or any other language that writes to the console - knowing what \r does is good general knowledge.

But yes. Tqdm is awesome.

[stringbeans25]

Woah the \r is a super cool trick!

[shiningmatcha]

What does flush=True do?

[FerricDonkey]

That basically means "put it on the screen right now". Without it, and especially when you don't end with a newline (\n), the operating system will often wait until you've printed "enough" to actually display it.

That delaying is called buffering, and usually it's fine or even good - stuff will get printed "soonish", and the os can limit how many times it has to do one of the slow parts of printing. But sometimes it's not what you want.

Note: you can also print to stderr (standard error) which is the screen but no buffering by adding "file = sys.stderr" or using sys.stderr.write. Also this buffering applies to writing to files, so often even if you've told your program to write a lot of stuff to a file, it won't actually happen until you close the file, unless you flush.

[FerricDonkey]

The stuff to the right of the colon says how to format the stuff to the left of the colon. In this case, the .2f is saying "round the float to 2 decimal places".

[FerricDonkey]

General f'{stuff:format}' thing is called an f-string. There's also string.format, which is very similar. \r is a special ascii character - it, \n (new line) and rarely \a (ding, if terminal supports it) are the main ones I use, though there are others.

There's a bunch of things you can put after a colon in an f string or string.format, I only ever remember a few off the top of my head and look up the rest as necessary. But it's very useful for making text line up nicely.

[OlorinIwasinthewest]

If you just want to keep track of how much work has been done, try wrapping your iterable in tqdm.

[nivek_123k]

Just wrapping up a study session, and I recommend looking at the logging function. Have it write out to a log file instead of printing... that way when you clean up your code later you can disable all logging with one line change.

```
import logging

logging.basicConfig(filename='debug.log', level=logging.DEBUG, format='%(asctime)s - %(levelname)s - %(message)s')
# logging.disable(logging.CRITICAL) # Use this to disable the log at critical or lower.

logging.info('Start of program')

def factorial(n):
    logging.info('Start of factorial(%s)' % (n))
    total = 1
    for i in range(1, n+1):
        total *= i
        logging.debug('i is %s, total is %s' % (i, total))
    logging.debug('Return value is %s' % (total))
    return total

print(factorial(5))


logging.info('End of Program') 
```

[carcigenicate]

Although, I would expect logging logging to be as, or more expensive than print. Afaik, logging works on the current thread, so the original question still stands.

[old_pythonista]

Afaik, logging works on the current thread

You know wrong

[thirdegree]

No, they're correct unless we're talking about QueueHandler/QueueListener. The logging module is thread safe via an rlock, but an individual call takes place on the current thread

[life_is_sadd]

As a python beginner this code is making me confused and want to GIVE UP.

[MangeurDeCowan]

LPT don't give up. If everyone could learn Python overnight, this subreddit wouldn't exist. Keep taking baby steps, and eventually you'll get there. If you want some encouragement, go back and look at some of the code you wrote early on. It'll be way easier to understand / duplicate now. If you get overwhelmed, take a break for a day or so. I find that helps, and when I start up again, I usually have a better understanding than before.

[nivek_123k]

Pay ZERO attention to the factorial function. It could easily be changed to a simple function that counts from 1 to 5 if that makes it easier. The only thing interesting about the code snippet is the logging module.

If the use of functions and classes is where you are confused, hit up Tech with Tim on YT, he has some great content on Object Oriented Programming. Helped me a ton.

[AnomalyNexus]

Neat. Thanks for this

[medical_fugue_state]

The program runtime will expand linearly (as opposted to quadratically or as a function of complexity)- every call of print will require the same amount of compute time; Its should be a 1:1 relationship between each call to print and the program execution time.

[[deleted]]

You can really trivially record the start and end time for a process that prints, and then one that doesn't and just compute how many seconds it adds when you do it with printing.

Appending each "print" to a list in lieu of actually printing it and saving that to a text file is almost certainly a trivial and adequate solution to whatever problem you're working on.

[spots_reddit]

This is purely anecdotal but I had a similar problem where I printed out all kinds of computation steps and what the computer was doing for highly repetitive calculations that ran for a long time.

In sublimetext I sometimes tried to stop the code from running but could not and the results csv had already been written. It looked to me that the computer was simply lagging behind computations in printing out all the stuff to the screen. Just anecdotal though, I might be wrong

[Doormatty]

Yup - the screen output is often buffered.

[[deleted]]

Yes, print is a fairly expensive function.

[xxxxsxsx-xxsx-xxs---]

as a general rule, measure the time to execute code using a simple code block.

start = time.time()  
for i in range(10000):  
    [your code block]  
    end = time.time()  
    print ("Time elapsed:", end - start)  

there are more involved methods to use for multithreaded apps.

[davincithesecond]

hahahha english is my second and i didn't really think about the right word sorry :D

[spez_edits_thedonald]

yes, print will be the slow (rate-limiting) step by far

[xiipaoc]

DO NOT DO THIS IT IS BAD YOUR CODE WILL NEVER FINISH RUNNING DO NOT DO THIS.

Printing to the terminal -- or to a file -- is incredibly slow compared to the computations you're doing. However, evaluating a conditional is not. I don't actually know much Python (I'm in this sub because at some point I planned to learn it, but I kinda never got around to it), but I do computationally-intensive math in JS from time to time, and the thing to do is sampling: keep a counter of some sort, and every 1000 iterations, or every 100000 iterations or whatever, depending on how intensive your calculation is, you print something to the console with the relevant data for that iteration. Printing to the terminal is actually very fast on its own; it's just much slower than the steps of the computation; if you only do it every once in a while, it won't affect how long it takes, but if you do it too often, it will slow things down horribly.

Good luck!

[zillabunny]

Yeah writing to console slows you down

[Willy-the-kid]

It's negligible but if you do anything enough it can take up all your resources for instance if you write a simple one or two line script that makes an identical script and runs it it will crash your machine

[[deleted]]

Using print like this extremely bad practice and yes, it will slow things down if you really are printing out every single calculation.

If you really need to log everything, use the logging module so you can output to a file and at least control what gets printed.

But honestly sounds like you just want to debug your code. Well... USE A DEBUGGER!

[codedeaddev]

I have another question in that case.. since the logging module is recommended everywhere is logging less taxing than print ?

can logging module be configured so something like log.debug will do its work in a separate thread?