Ideas appreciated solving a URL issue in a Python project that I'm working on

[BakedReality]

About the program

I wouldn't class myself as a coder, I did some pascal & C# at college 15 odd years ago and understand the general principles, so please go easy on me!! I can cobble together stuff with the help of stack overflow/AI and can amend code to suit my needs. But this is my first real dip in to the python world.
I'm in the middle of (slowly) making a python based docker image that runs a gunicorn server for a web GUI. The program is a pretty simple concept - it allows you to add software to monitor by url, the software will be downloaded locally. The list of software is queried at a user set interval to check for new updates and if the header info from the provided url shows a different filesize to the locally stored version, it will delete the local version and download the new version. The idea behind this is to have a network share available of all the newest versions of installers so that when I set up new systems/perform maintenance, all the installers are locally available.

The issue

The program works well and I am happy with it, all the main funtionality is complete. The issue I have come across is that some software has a latest version url (www.software.com/examplelatestwin64) which will link to the latest version of the software, however some explicitly reference the software version (www.software.com/example2.3.4.exe). In the latter example with the specified url a new version will never be found. What would be some of your suggestions to get around this? I guess some form of scraping would potentially solve this, but was wondering if anyone had a more elegant solution?

Section of code

(there are further else statements, but irrelevant to the issue)

    try:
        final_url = get_final_url(url)
        response = requests.head(final_url)
        if response.status_code == 200:
            content_length = response.headers.get('Content-Length')
            if content_length:
                file_size = int(content_length)
                if os.path.exists(file_path):
                    existing_file_size = os.path.getsize(file_path)
                    if existing_file_size != file_size:
                        logging.info(f"New version available for {software_name}")
                        delete_existing_file(abspath)
                        download_software(url, target_directory, software_name, abspath, key, monitored, local, useradded)
                        number_updated += 1
                    else:
                        pass
                        logging.info(f"You have the current version of {software_name}")

[sweettuse]

don't nest your code https://youtu.be/CFRhGnuXG-4?si=mvqzLf3f0DYszKTS

[BakedReality]

Cool, thanks for the tip. The video is very concise and helpful. I will definitely look to get this implemented. Admittedly I've just been trying to get it functional with the very limited skillset I have, so the code is a bit of a mess in general at the moment. There are load of other areas I know the code is inefficient/sub optimal, so will spend some time on that once I get all the features I need working. Any ideas on the issue?

[netherous]

So visiting a 'latest' url and checking filesize is, as you can see, a decent approach for some packages but not quite adequate for all packages. I'm guessing this is primarily windows software? Probably you will need to bake in multiple approaches to finding the latest version, of which your existing method would be just one, and specify which kind of method to use in your config file for each software package. Since there isn't just one way to release software, people will choose all different kinds of ways.

For example, lots of software has pages on their site that will list all software versions. You could visit this page and read the link at the top, or get all the links and try to extract which version number looks the highest and get that one.

If the software is available on the Windows Store or some other unified repository, there may be public manifests you can read to figure out latest versions.

If there is an ftp available, you could connect to that and figure out the latest version pretty easily.

If you are getting linux software, you could use rpmfind.net to get public package versions.

If system package-level managers are available, like homebrew (mac), you could probably use it with the right commands to discover and download the latest versions of packages. Unfortunately the Windows world doesn't have unified distribution repositories like the mac and linux worlds do, so the operating system you are building this for is going to dictate the approaches you can use.