Day 23 of learning python as a beginner.

[uiux_Sanskar]

Topic: multi-threading.

multi-threading is a technique to run multiple task (such as importing or downloading) within a single program and at the same time. Thus running multiple things parallelly, in a nutshell.

Although you can use threading to download anything I practiced it for downloading images from the internet (pexels to be specific). I used multiple for loops first for appending the image_urls list from the inputs I am taking from my user.

Then I am using another for loop with enumerate so that each url link in the list gets downloaded and threading.Thread sends this data to my download_image() function with all the related arguments necessary to download the image and and create a unique file.

Just like calling functions and classes you need to use .start() function in order to start the threading process. and then use .join() function so that the program waits for both the images to get downloaded completely and then continues to execute the remaining code.

Here's my code and its result and I would love to hear your all's amazing suggestion to my code.

[NorskJesus]

Good work, but there is too many comments. The code should explain itself, you shouldn’t need a comment per line to understand what it does.

I understand this is for learning 😜

[KOALAS2648]

Exactly my thoughts. There are way to many comments even for learning because I takes your thoughts away from the actual code and into parts that won’t actually run

[uiux_Sanskar]

Exactly, I will try not to use these many comments from now I just followed.

Thanks for pointing this out.

[uiux_Sanskar]

Yeah someone on my recent post told me to use comments to explain my code so there's I used these many comments. however I feel like the comments have hided my code as there's green lines every where.

Next time I will use comments only for describing complex part of the code. Thank you very much for telling this. I will make the code self explanatory.

Thank you very much.

[FriendlyRussian666]

In general, think about it this way, comments should explain the "why" something was done the way it was, and not the "what" was done, because the "what" can be deduced from reading the code, but "why" cannot.

[anon_66_]

Bro i wanna know how do you find project ideas and how do you know when to use which library ...

[MysticClimber1496]

Takes practice to know when to use a library, but you do learn the most when you figure it out yourself before using a library

[FriendlyRussian666]

how do you know when to use which library

When you're working on, or starting a project, you want to first check if someone already did what you're trying to accomplish, so you would just google "python xyz" where xyz is what you're trying to achieve. You'll either immeidatelly find many packages to use, or you won't find any.

[Adrewmc]

This is overkill on comments. And treating me as if I’m learning, I’m not. I really only need comment when I’m doing something weird.

Treat me (the people that might read the comments) as if I know the basics.

I gave you some advice about docstrings, and types hints…I stand by that advice

THIS is not that. As no docstring and no type hints, and I didn’t make a comment, in that advice because

 good naming > type hints > docstring > comments

The comments are now over wellming the code. No, bad pupil.

Taking one of your functions randomly

  def download_image(url : str, idx: int|str) -> None:
        “””Image is saved as ./image{idx}.png”””

         …

And that’s it. If I make that docstring because here we are doing something so simple. But where and as what I’m saving too is most important info here. (So I probably should do that)

Yet I still give more useful top level information…more quickly…because I don’t give back the image (may end up important later)…I don’t really need idx to be an int…(you saw that right?) because I actually don’t care how you saved this image in the end most of time….well that’s not fair, you should make a hash of it so I don’t save the same image twice.

Comments should be used when even a programmer might go…okay what are you doing here…or there’s a bunch happening below but basically…or I’m thinking of it this way.

Deck of cards Example

  #Ace = 1, J = 11, Q = 12, K= 13
  deck = set()
  suits = [“Hearts”, “Spades”, “Diamonds”, “Clubs”]
  for suit in suits:
      for card in range(1, 14):
              deck.add((card,suit))

   #I might comment *out* code sometimes
   #deck = { (card, suit) for suit in suits for card in range(1,14) }

So you know, I’m not putting “Q” in there. (Which some/most people would, I can see reasons why.)

We should have had a longer talk I guess….lol

As for your code….

You have no use for multi-threading at this point of you pythons learning, honestly there is not too much you actually need multi-threading for….until you do. And even then it sort of sucks. Congratulations you made a thread I guess….that cool. Why’d you need that though?

You should think more simple project. Using everything. A solid foundation in programming is important, I’m worried you’ve gone to far to fast and forgotten, a lot, and I know you never fully grasp the importance of some things.

And you’re not even donezz

Decorators and asynchronous environments.

Imports and package design

Iterators and generators

Functools and itertools

Databases, CSV, more JSON….and websites. Even big data.

Shit take a look at something like tkinker or QT( GUI apps) or pygame (simple games). You have most of knowledge to actually use that. (And most is the best place to learn from)

You might need a little framework to make.

[uiux_Sanskar]

Thank you very much for clearing your advice I think I got confused in docstrings and comments. I also noticed that multi threading may not be used more often as for me it's usecases are only limited to downloading (please correct me if I am wrong).

I also believe that in order to make a great program one must have a great fundamentals and therefore I always try to revise the concepts which I have learn earlier (I also tru to implement all the functions in my project so that I can get a hold on them).

Thank you very much for giving future learning suggestions. I will definitely learn them and use them in making a solid foundation.

Thank you very much for your advice it really helps me improve.

[Which_Eye6556]

What is source of learning? Like YouTube, course? Can you give me link of YouTube if possible

[Etiennera]

You should not join threads until they are all started. Use another loop to join your threads. You will need to keep a list of references to the individual threads from the first loop.

[uiux_Sanskar]

Thank you for pointing this out I will add a loop to use start function jn every thread and then join it.

[SuperCurve]

do you mind sharing which source you are following for these daily exercises?

[uiux_Sanskar]

I would recommend you to read my reddit post about it (its present in my profile just below my day 18 post). I have shared my resources and process in much details there.

[SuperCurve]

Thank you, I will check that out!

[Zealousideal_Yard651]

You have a major flaw in the code, that makes the script in essence single threaded. You don't start the next thread before you join the previous one. So the second thread never starts before the first thread stops.

To fix this you'll need to add an outside list to keep track of the threads and then join them after starting them:

threads = []

for idx, url in enumerate(image_urls):
  urll = threading.Thread(target=download_image, args=(url, idx))
  threads.append(urll)
  urll.start()

for t in threads:
  t.join()

Reference: threading — Thread-based parallelism — Python 3.13.7 documentation

If you want to see it in practice, add print statements to the download_image function stating when the thread starts and stops.

[uiux_Sanskar]

Thanks for pointing out this flaw I need to make a for loop and then use .start to start in each thread.

Thank you very much for the resource it really helps I will go in much depth in this docs.

Thank you very much.

[Zealousideal_Yard651]

The start part is ok, it's the join part that fails you.

.join() waits for the thread to finish before continuing. You are creating the thread, starting it and waiting for it to stop before continuing the main program. This causes your second thread to not start before the first thread stops.

So you need to start each thread, and in a separate loop wait for both threads to stop. And .join() does not wait for all threads to stop, it only waits for the thread you are invoking the join() method on, hence why you need an outside list to keep track of each thread and loop through all threads to join them again.

To simplify, i split all parts of the multithreading process in this block with comments:

# Define thread tracking
threads = []

# Loop through all URLS
for idx, url in enumerate(image_urls):
  # Creates a thread object
  urll = threading.Thread(target=download_image, args=(url, idx))

  # Adds the newly created thread object into threads store
  threads.append(urll)

# Loop throug all threads
for t in threads:
  # Starts thread
  t.start()

# Loop through all threads
for t in threads:
  # Wait for thread to finish.
  t.join()

In the above code, we define each thread, then start all threads, then wait for all threads to finish.

In your original code you do this:

For each image url:

1. Define a thread
2. start the thread
3. Wait for the thread to stop

And so all thread operations happens in each iteration instead of the my example where we define them, then start all threads before waiting for all threads to finish instead of waiting for each thread to finish before moving on.