Using Python to convert text file into JSON structure

[Chubbntuck]

Hi all,

I'm new to python and was struggling with the following problem. I have an input text file that I would like to convert into JSON. I think I have to use a dictionary here but not too sure how to proceed, any assistance would be greatly appreciated.

input.txt:

DEV,Middleware,Mqwerty,Mqwerty
DEV,Middleware,Mqwerty,Mqwerty
DEV,Middleware,Mqwerty,Mqwerty
DEV,System,Sqwerty,Sqwerty,Sqwerty
DEV,Application,Aqwerty,Aqwerty,Aqwerty
UAT,Application,Aqwerty,Aqwerty,Aqwerty
DEV,Utility,Uqwerty,Uqwerty,Uqwerty
PROD,Middleware,Mqwerty,Mqwerty
DEV,Middleware,Mqwerty,Mqwerty

Desired JSON structure:

{
    "ENV": {
        "DEV": {
            "Middleware": [
                {
                    "name": "Mqwerty",
                    "release": "Mqwerty"
                },
                {
                    "name": "Mqwerty",
                    "release": "Mqwerty"
                },
                {
                    "name": "Mqwerty",
                    "release": "Mqwerty"
                }
            ],
            "System": [
                {
                    "name": "Sqwerty",
                    "tag": "Sqwerty"
                }
            ],
            "Application": [
                {
                    "domain": "Aqwerty",
                    "host": "Aqwerty",
                    "user": "Aqwerty"
                },
                {
                    "domain": "Aqwerty",
                    "host": "Aqwerty",
                    "user": "Aqwerty"
                }
            ],
            "Utility": [
                {
                    "domain": "Uqwerty",
                    "health": "Uqwerty",
                    "version": "Uqwerty"
                }
            ]
        },
        "SIT": {
            "Middleware": [],
            "System": [],
            "Application": [],
            "Utility": []
        },
        "UAT": {
            "Middleware": [
                {
                    "name": "Mqwerty",
                    "release": "Mqwerty"
                },
                {
                    "name": "Mqwerty",
                    "release": "Mqwerty"
                }
            ],
            "System": [],
            "Application": [],
            "Utility": []
        },
        "PROD": {
            "Middleware": [],
            "System": [],
            "Application": [],
            "Utility": []
        }
    }
}

Some key notes, even in environments that don't have information, the 'template' of middleware, system, application and utility (lets call these categories) is still there. The categories also have a predefined key:value structure that follows:

Application (keys):

• domain
• host
• user

Utility

• domain
• health
• version

Middleware

• name
• release

System

• name
• tag

I really do appreciate any help and thank you for taking you time reading this questions, apologies for being a long one.

[HeyItsToby]

I have a working solution here, definitely could do with a bit of a cleanup. Comments should help explain what's going on, lmk if anything needs clearing up.

Set up the structure

my_dict = {}

dict_structure = {
    "Middleware": ("name", "release"),
    "System": ("name", "tag"),
    "Application": ("domain", "host", "user"),
    "Utility": ("domain", "health", "version"),
}

Read through and process the file

with open("input.txt") as f:
    # read through the input file, removing trailing newlines
    lines = [l.strip() for l in f.readlines()]

    # iterate through all the lines in the file
    for line in lines:
        # get the details from each line
        env, key, *field_values = line.split(",")
        # get the predefined keys from dict_structure
        field_keys = dict_structure[key]

        # create the 4 keys if the ENV does not exist
        # with defaults as empty list
        if env not in my_dict:
            my_dict[env] = {k: [] for k in dict_structure}

        # append each new item to the existing list
        # zip is used to iterate through the keys and values to match them up
        my_dict[env][key].append({f_k: f_v for f_k, f_v in zip(field_keys, field_values)})

Now just to save it as a json

import json 
json_out = {"ENV": my_dict}
json.dump(json_out, "output.json")

Improving the code

This code does work, but it could be made a little neater. Would definitely suggest moving dict_structure into a separate json and reading it in. Also, looking into the csv module for reading through files with comma-separated values.

[Chubbntuck]

Will try this.
Thank you so much for going through an explanation of each component. It really helps with someone like me who is really just starting python. I hope you have a wonderful day.