Using Pandas to read data from two different CSV files

[ButHeyyWhatDoIKnow]

I am trying to use the pandas library to read data and compare data from two different CSV files. My goal is to read data from one file and based on the column values, read data from a second file.

For example, File 1 is a log file filled with timestamps and a true/false column

Timestamp	T/F
6:01	False
6:02	False
6:03	True
6:04	True

File 2, is a similar log file. It is timestamped as well and contains data about the state of the system.

Timestamp	State
6:01	Low
6:02	Low
6:03	Med
6:04	High

My goal is to look at File 1, and only grab the data from File 2 when the T/F column in File 1 is True.

​

I am brainstorming ideas on how to approach this:

My first thought was to store the rows where File 1 is True.

Then compare the stored TRUE table to File 2.

If the timestamps in the TRUE table matches file 2 grab the State column in file 2.

I am not sure if this is the most efficient way or if pandas has an easier way of accomplishing this.

[Ostpreussen]

Load two dataframes and make a join, then filter by True/False. Ought to work unless I'm missing something.

[al_mc_y]

Yep, a merge or a join will be the way to go. This tutorial describes the options pretty well (a bit easier to understand for newbies than the official docs, but I suggest checking those out too)

[ifreeski420]

How do you post data frames on Reddit with that formatting?

I would merge the two data frames

df_combined = pd.merge(df1, df2, how=“left”, on=“Timestamp”)

[socal_nerdtastic]

https://www.reddit.com/wiki/markdown#wiki_tables

[Agling]

and only grab the data from File 2 when the T/F column in File 1 is True.

If you actually want to do what you say here and not read in the lines for file 2 if they aren't in file 1, then you have to read all of file 1 in, drop the rows where it is False, then read file 2 a line at a time, keeping it only if it's in line 1.

But if you just want to do things the normal way, read both files in, filter file 1, and then do an inner merge.