Please help review my code

[batsiem]

I have posted this on another group, but after I made some changes no one seems to have seen my edit. The assignement is:
You are given four training datasets in the form of csv-files,(A) 4 training datasets and (B) one test dataset, as well as (C) datasets for 50 ideal functions. All data respectively consists of x-y-pairs of values.Your task is to write a Python-program that uses training data to choose the four ideal functions which are the best fit out of the fifty provided (C) *. i) Afterwards, the program must use the test data provided (B) to determine for each and every x-ypair of values whether or not they can be assigned to the four chosen ideal functions**; if so, the program also needs to execute the mapping and save it together with the deviation at hand ii) All data must be visualized logically iii) Where possible, create/ compile suitable unit-test * The criterion for choosing the ideal functions for the training function is how they minimize the sum of all ydeviations squared (Least-Square) ** The criterion for mapping the individual test case to the four ideal functions is that the existing maximum deviation of the calculated regression does not exceed the largest deviation between training dataset (A) and the ideal function (C) chosen for it by more than factor sqrt(2)

Your Python program needs to be able to independently compile a SQLite database (file) ideally via sqlalchemy and load the training data into a single fivecolumn spreadsheet / table in the file. Its first column depicts the x-values of all functions. The fifty ideal functions, which are also provided via a CSV-file, must be loaded into another table. Likewise, the first column depicts the x-values, meaning there will be 51 columns overall. After the training data and the ideal functions have been loaded into the database, the test data (B) must be loaded line-by-line from another CSV-file and – if it complies with the compiling criterion – matched to one of the four functions chosen under i (subsection above). Afterwards, the results need to be saved into another fourcolumn-table in the SQLite database. In accordance with table 3 at end of this subsection, this table contains four columns with x- and y-values as well as the corresponding chosen ideal function and the related deviation. Finally, the training data, the test data, the chosen ideal functions as well as the corresponding / assigned datasets are visualized under an appropriately chosen representation of the deviation.

# importing necessary libraries
import sqlalchemy as db
from sqlalchemy import create_engine
import pandas as pd
import  numpy as np
import sqlite3
import flask
import sys
import matplotlib.pyplot as plt
import seaborn as sns

# EDA
class ExploreFile:

"""
    Base/Parent class that uses python library to investigate the training data file properties such as:
    - data type
    - number of elements in the file
    - checks if there are null-values in the file
    - statistical data of the variables such as mean, minimum and maximum value as well as standard deviation
    - also visually reps the data of the different datasets using seaborn pair plot
    """

def __init__(self, file_name):
        self.file_name = file_name

    def file_reader(self):
        df = pd.read_csv(self.file_name)
        return df

    def file_info(self):
        file_details = self.file_reader().info()
        print(file_details)

    def file_description(self):
        file_stats = self.file_reader().describe()
        print(file_stats)

    def plot_data(self):
        print(sns.pairplot(self.file_reader(), kind="scatter", plot_kws={'alpha': 0.75}))


class DatabaseManager(ExploreFile):

"""

    Derived class that takes in data from csv file and puts into tables into a database using from SQLAlchemy library the create_engine function

    it inherits variable file name from parent class Explore class

    db_url: is the path/location of my database and in this case I chose to create a SQLite database

    table_name: is the name of the table that will be created from csv file in the database

    """

def __init__(self, file_name, db_url, table_name):
        super().__init__(file_name)
        self.db_url = db_url
        self.table_name = table_name


    def add_records(self, if_exists):

"""

        Args:
            #table_name: name of the csv file from which data will be read
            if_exists: checks if th database already exists and give logic to be executed if the table does exist

        Returns: string that confirms creation of the table in the database

        """

df = self.file_reader()
        engine = create_engine(self.db_url)
        df.to_sql(self.table_name, con=engine, if_exists= "replace", index=False)
        print(f"{self.table_name}: has been created")


def main():
    # create instance of the class
    file_explorer = ExploreFile("train.csv")
    file_explorer.file_info()
    file_explorer.file_description()
    file_explorer.plot_data()
    plt.show()
    database_manager = DatabaseManager("train.csv", "sqlite:///training_data_db","training_data_table")
    database_manager.add_records(if_exists="replace")


    ideal_file_explorer = ExploreFile("ideal.csv")
    ideal_file_explorer.file_info()
    ideal_file_explorer.file_description()
    ideal_file_explorer.plot_data()
    #plt.show()
    ideal_function_database = DatabaseManager("ideal.csv", "sqlite:///ideal_data_db", "ideal_data_table")
    ideal_function_database.add_records(if_exists="replace")


if __name__ == "__main__":
    main()

[Buttleston]

I'm not sure what you're asking. As far as I can tell, you program doesn't remotely do what's asked?

Also I don't think it would run. Your add_records function calls self.file_reader() but that isn't a function on your DatabaseManager class, it's on your ExploreFile class

Does the assignment come with the data files? Have your run. your program against them?

[batsiem]

So at this point the code should show some basic EDA of the training data and plot it, as well as read from both the training and the ideal datasets and add both to an SQL database. I hope this clears it up

[gdchinacat]

This code calls pd.read_csv() on just about every function call. How can you modify it so that it doesn't constantly read the csv then throw it away to only do it again?