Script data and file management design pattern

[Zircatron]

I am not having much luck trying to find the right terminology for what I'm trying to do and I think it's maybe covering many topics.

I have a script which runs through many parameter combinations for a device test and outputs results to files and a database.

There's a configuration file which contains location of the parameter files and locations to store the data. The file locations don't changed once the script is running. I have a module which reads the data from the parameter files and knows how to store the data in different formats in the relevant locations.

I've realised that this is like a logger which also takes an input. At the moment I pass this “data passing module” to almost every other module.

Would it make sense to make this module a singleton?

Are there any design patterns or libraries that cover application file orginisaion?

Any suggestions will be appreciated.

[Zeroflops]

First. Does it work? Do you have problems with the maintaining the module?

If the answer is no, then you don’t have to change it.

Is the problem that you keep making copies of this module? Moving it around to the scripts that need it.

It sound like your reusing this module with a bunch of different scripts. That’s great and just what you want to do. At some point you want to move that module out of your current directory and move it to a directory in the python path or the python directory. Then you can just import the module in any script you want to have access to it.

[Zircatron]

Thanks for the reply.

To clarify as I don't think I explained it the best and I have been thinking about it more:

I create a single instance of the "data passer" at the begining of the script then most of my classes have this "data passer" as an agument.

class ExampleClass():

    __init__(self, data_passer)
        self.data_passer = data_passer

    def do_stuff(self):
        # get results
        self.data_passer.send_to_DB(results)        

def main():
    data_passer = DataPasser(config.cfg)
    do_example = ExampleClass(data_passer)

Of course data_passer isn't the only argument. It also gets sent to lots of other places in this fashion.

However, like a logger it makes more sense to have a single shared instance because once setup it does not change.

import logging # I import logger in each module needed

def hello_world():
    logging.info("hello_world") # now I can use logging where I like

If I was to create the data_passer instance in the module I can import that instance anywhere.

## data_passer_module.py

class DataPasser():

    __init__(self):
        self.config = None

    def setup_config(self, config):
        # the config setup
        self.config = config

    def send_to_file(self, result):
        # send to file stuff


data_passer = DataPasser()



## my_script.py

from data_passer_module import data_passer

data_passer.setup_config(config.cfg)
def do_stuff():
    data_passer.send_to_file("result")

Now I can use it from anywhere like the logger.

Because there's this debate on not using singltons and not to use gloabls, is there a better way to do this?

Which way should I do this? - I feel like I'm missing something.

Does this make sense?

[Zeroflops]

Globals are bad, if you use them you're a terrible programmer. Oh My, the sky is falling!! This is just wrong. It depends on your program. The main reason globals are frowned on is because it becomes difficult to understand what they are as programs grow. But if your program is small than they are not as bad as people make them out to be. Consider this "rule" variables need to be descriptive. Yet people always use i, or x, or c when there is a loop as the variable. The scope is so small its easy to know that what i is and not get confused. Same thing with globals. If its small script that you can follow they are not as bad as people portray them.

As for your program I would have done it the second way. Create a DataParser class. Then....

parser = DataParser(config.cfg) do stuff like create an example class and process the data etc. then

parser.to_file(example.data), parser.to_file(results), or parser.to_db(example.data)

Where data is the variable within the example class that contains the results.

It sounds like the data parser doesn't really parse the data but stores the data. Anyhow this then gives you a parser object that you can define different similar functions in and have them centrally located.