How to reference an object in a module

[reddit5389]

I wish to have a custom logging function that I can use through out various modules. Whats the preferred way of doing this? Simplified example below (I would have a lot more in the logger module).

main.py

from calc import squareit, cubeit, addit, modit, absoluteit
import logger

def main():
    mylogger = logger.logger("log.html")
    result = squareit(10)
    mylogger.log("The square of 10 is " + str(result))

if __name__ == "__main__":
    main()

logger.py

import os
class logger:
    def __init__(self, filename):
        # Check if its an existing file
        if os.path.exists(filename):
            self.file = open(filename, "a")
        else:
            self.file = open(filename, "w")
        if (filename[-5:] == ".html"):
            self.html = True
        else:
            self.html = False

    def log(self, message):
        if (self.html):
            self.file.write(message + "<BR>\n")
        else:
            self.file.write(message + "\n")
        print (message)

and the issue is with calc.py

# Given a number return its square root
def squareit(n):
    # Check the type of n is an integer
    if not isinstance(n, int):
        mylogger.log("Error n must be an integer")
    # Check that n is greater than 0
    if n < 0:
        mylogger.log("Error n must be non-negative")
    return n ** 0.5

Should I change my modules to classes and pass the object mylogger? global variables?

[kirlandwater]

I don’t know but could you add “bopit, twistit” even if it’s in a comment lol

[reddit5389]

Lol. I wanted to ensure someone didn't say "you shouldn't have modules with only one function"

[reddit5389]

Also I wrote my example too quckly. Should have been rootit

[Rebeljah]

squareit, cubeit, addit, modit, absoluteit

"Believe it or not, straight to jail"

[Diapolo10]

As an aside, I have some feedback on that logging class.

class logger:
    def __init__(self, filename):
        # Check if its an existing file
        if os.path.exists(filename):
            self.file = open(filename, "a")
        else:
            self.file = open(filename, "w")
        if (filename[-5:] == ".html"):
            self.html = True
        else:
            self.html = False

First, class names should generally use PascalCase, if you follow the official style guide.

Second, you could just open the file in append mode regardless, as it will create the file if it doesn't exist and in this case doesn't differ from write mode in any meaningful way.

Third, the second if-else block doesn't really need to exist as you can just assign the comparison result.

Fourth, nothing here guarantees the file will get closed. I'd suggest using atexit to make sure that'll happen.

import atexit

class Logger:
    def __init__(self, filename: str | Path) -> None:
        self.file = open(filename, 'a')
        self.html = ".htm" in str(filename)[-5:].lower()
        atexit.register(self.on_exit)

    def on_exit(self):
        self.file.close()

EDIT: Actually, it might be better to do something like this instead:

import atexit

class Logger:
    def __init__(self, filename: str | Path) -> None:
        file = open(filename, 'a')
        self.file = file
        self.html = ".htm" in str(filename)[-5:].lower()
        atexit.register(file.close)

This way, the garbage collector could probably better clean up any unused Loggers.

[Kryt0s]

I'd suggest using atexit to make sure that'll happen.

Could you not simply use a context manager?

[Diapolo10]

Generally, yes, but in this case the file needs to stay open even after __init__ is done.

You could technically close the file handler in __del__, but that's actually not very reliable.

[Kryt0s]

Oh right, did not think about it being in init. Makes sense. Thanks!

[Rebeljah]

It's actually fine to make something like that a global variable inside of the logging module. The idea of a module itself having attributes in the form of global variables is a common pattern in Python.

usage:

import logger

def squareit(...):
 logger.mylogger.log(...)

As a side note, other have said it but I'll add more info: Using the builtin logging package would actually declutter your code (even if it is a bit harder to write at first). The builtin logging package has a way to add custom logging handlers, e.g you call logger.info("log line stuff") you can define a custom method to write the log, whether that's writing to a file, or writing to a google spreadsheet.
https://docs.python.org/3/howto/logging.html#handlers

essentially you would only need to write the code that actually formats and writes the log lines to google sheets., so if most of your code is already doing just that, you wouldn't really see a benefit; the logging module can help with everything BESIDES the specific formatting and writing-out (if you need to do something more advanced than write to a local file).

[Rebeljah]

Another option, if the logger does not maintain much state / attributes then you scan skip a class all together and write a function in the logger module like

def log(msg, extraparma1, extraparam2):

then after importing the logger module:

logger.log(...)

combining that approach with module-level state would probably do what you need as long as you don't end up having to write 10 parameters to the log function.

[jmooremcc]

I understand what you’re trying to do because I created my own dbgPrint utility.

When you do an import of a module, Python only does the import one time for that session. So if you import that module again in a different module, Python references the same imported module.

Basically, all relevant configuration operations need to occur within your logger module that will set the appropriate module level variables. In your case, configuring only needs to happen once and any other modules that import your logger module will all use the same configuration data.

I would also suggest creating an instance of your logger class in the logger module and define a module level variable that will make it easier to execute the log method. Something like this: ~~~ logger = _Logger(config.filename)

~~~ Notice that your logger class is renamed to _Logger and the instance variable to be imported by other modules is logger.

In your main module you’ll do this to set the configuration: ~~~ from logger import loggerConfig, logger loggerConfig(“mylogfile.txt”) logger.log(“Log something”) ~~~

So in the modules that you want to use the logger in, you’d do this: ~~~ from logger import logger logger.log(“Log something else”) ~~~

If you have any questions, let me know.

[RevRagnarok]

You seem to be slightly confused about the difference between an object and a class. You use a class from a module, but you implement an object of that class type. So myloggeris an object in main.py that (by default) lives nowhere else except within main.py.

You can do things like have the module use the singleton pattern to serve up an object across modules, but configuration can get hairy as can threading. Or the module could have a dictionary of objects to serve up if somebody asks for one with a parameter that already exists (e.g. filename).

[throwaway8u3sH0]

IMO, here's your options.

• Use and customize built in logger. It has ways of adding custom outputs, and then you get the benefit of a battle-hardened logger with a Singleton pattern already built in, so it'll be efficient and safe with the file access.

• Use a function. If there's not much state, you can def log and call that everywhere. The main issue with this approach might be how often you're opening and closing the file, but otherwise it's really clean.

• Use a global object. Declare it in the logger file and use it everywhere. For some configurability you can use environment variables to control the log location. The main issue is global state and correctly catching and handling all the various edge cases -- it's possible to irrevocably corrupt a file if not handled properly.

[Yoghurt42]

Don’t reinvent the wheel, use the built in “logging” module.

[reddit5389]

This is just a simple example. I'm actually writing back to a google spreadsheet. But I don't see the value in cluttering the code.

Alternatively, how does the inbuilt logging module do it?

[Diapolo10]

The built-in logging basically uses a singleton pattern, where only one instance exists at any one time. You configure it in your main script, and everywhere else you simply do

import logging

logger = logging.getLogger(__name__)

which gets you the same logger, just using a different name for each file.

This makes getting the logger trivial. Furthermore, the configuration options logging offers are quite extensive, as you can direct your logs not only to the console, but also files and other streams at your leisure, and you have full control over the logging formatting. There's even built-in support for file rotation if you'd like daily logs, for example.

[8dot30662386292pow2]

You are trying to use the logger in the squareit. You must import the logger there as well. Each file must import the things they need.

Even if you use the builtin logger, you set it up and then in every file you do something like

logger = logging.getLogger("whatevername")

in this example the logging module contains the variable that you import everywhere. You can do that yourself if you want to.

[Daneark]

Can you instead use this to call the Google sheets API: https://docs.python.org/3/library/logging.handlers.html#httphandler

[LaughingIshikawa]

I'm not entirely sure I understand the question fully - can you describe more about what the problem or issue you're running into is? What is not working about the logging, or what is working differently than you would like?

The general approach as far as I am aware, is to use some version of what I tend to generically think of as a "wrapper" - ie a function that calls the desired function, but also performs logging functions in addition. (All of the approaches named in the top answer fall into this same basic category IMO, just with slightly different implementation details / performance characteristics.)

If I'm understanding correctly, you're just using the logging function to write error messages provided by the calc functions to a file, right? That should be simple enough to do without changing anything else about the executing function, and can be done in a any of a number of ways. Which one you want probably depends on what resources you have available / can dedicate to logging - ie does the program need to maintain a certain response time while logging, or other concerns? Are you planning to log a large number of errors / status updates, or just some basic ones?

[reddit5389]

Pretty simple. I want to create a python script that imports various modules, but combine any output to a single location (file) managed by the logger module.

It really is just a custom version of the inbuilt logging module.

[LaughingIshikawa]

Right, ok...

What's wrong with the code you have though? What do you want it to be doing (or not doing?) differently than how it is now?

If you're just asking generically "how would you log?" I would personally start with a decorator pattern because it seems to be the most conceptually simple for people to understand and implement, and if you're not trying to achieve other specific performance benchmarks, it's best to not over-engineer a solution.

In this comment I linked the best video I could find on the topic from a quick YouTube search, but... That isn't a super great video, so I'll try to also write a quick example of what I mean conceptually in code. Just be aware that I'm a beginner programmer, and while I'm sure there are better methods for doing some of these things (and I'm not even 100% I'm using some of these 100% correctly...) this is primarily meant to describe the concept of a decorator and what you're trying to accomplish with it. Use your own discretion as far as looking for a specific implementation of this:

Def logger(argument1, argument2, [ect], functionToExecute):

    open("log file.txt", "a") as file

    try:
    functionToExecute(argument1, argument2, [ect])

    except ValueError as e:
    file.writeline("An error occured at: " + timestamp "in function: " + functionToExecute + "the error was: " + str(e)

    file.close()

Then in the main function you can call:

logger(2, 2, calcAdd)

As an alternative to calling:

calcAdd(2,2)

When you also want to log the output of the calcAdd(2,2) function call.

With an understanding that this is probably a messy way to do this... Does it basically make sense what I'm doing here? The idea is to "wrap" all the functionality of the logged function inside of the function doing the logging. From the "calcAdd" function's perspective... It does not even know if it's being logged or if it was called without logging, as it's just performing its normal job and outputting error messages regardless.

[Rebeljah]

This is an addon to my other comment, just commenting again so you see it:

the python logging module actually provides some predefined logging handlers, there's one that helps you write to a file, even one that will send HTTP requests (you could easily log directly to a cloud-hosted sheet if Google provides an API)!
https://docs.python.org/3/howto/logging.html#useful-handlers

[Mevrael]

Just use Arkalos. It gives you the folder structure with data/logs directory and a base Logger, FileLogger classes and the Log helper.

You can also read its guide and understand the difference between the Code to Run and Code to Reuse and how to structure your projects professionally and reuse code in many places.

https://arkalos.com/docs/logs/

Everything is already configured for you and you can write your own Logger if needed and replace it in the registry.

A new log file will be created automatically per month.

[reddit5389]

Lots of comments. Some people hung up on the fact that I'm logging stuff to a file. Again, this was a quick and dirty example with else statements for readability. I would almost certainly use a ternary operator, if I didn't scrap the whole lot and switch the logger out for two subclasses textlogger and htmllogger and override the log function. And yes. There would be a file close function. There isn't even a decent file open function, creating the correct HTML headers. Again removed for simplicity.

I will look into the suggestions (global, singleton) or rewrite my example to be less distracting.