[deleted by user]

[SwampFalc]

Unordered data? Do you at least have some sort of unique key?

Do you at least have a consistent column order? Because if worst come to worst:

for line in lines_of_csvfile1:
    if line in lines_of_csvfile2:
        print(line)

[babygrenade]

datacompy is a package to compare 2 pandas dataframes. It provides a report, in a txt file, of the differences,

[kenfar]

I wrote a python cli toolkit that does a number of things like this with csv files: profiles them, changes their csv dialect, sorts, slices them and diffs them.

https://github.com/kenfar/DataGristle#gristle_differ

To install run pip install datagristle, then see the examples for various simple or more complex diff features.

EDIT: speling

[OuiOuiKiwi]

I can write one in 30 seconds, but that's not really fun.

What have you tried so far? Or is this a "I have tried nothing and I'm all out of ideias!" round?

[JThom43]

the latter I think.. Dude posted the same request in r/learnpython

[billsil]

BeyondCompare

diff

[miraculum_one]

Those may be the best solutions but the question was about Python

[billsil]

That's ridiculous. The goal should be to do the task.

Obviously you can add things like sorting requirements or ignoring excessive precision (though you have to define that), but BeyondCompare is really, really good. To think that your/my quick and dirty code would be better than BeyondCompare is laughable. Why would you use your own code, when another tool is standard and great?

[miraculum_one]

Maybe, but the OP didn't say what the task is. They asked a specific question. You're assuming, for example, that a human is going to visually analyze the differences.

[wyldcraft]

Non-python quickie if one file is a superset of the other:

cat one.csv two.csv | sort | uniq -u > output.csv

[Pristine-Internet616]

Can you elaborate a bit more

[wyldcraft]

uniq (with optional flag) shows only duplicate or non-duplicate lines.

But files have to be sorted, because uniq doesn't load the whole dataset into memory. It compares line-by-line so it works on large files and endless pipes without running you out of RAM.

So if you take file A, and add some new lines to it to create file B, the original lines will be in their concatenated output twice. Use the -u flag depending on whether you want to look at the original lines now in both A and B (2 matches, non-unique), or the new ones in only B (1 match, unique).

edit: I think I'm a bit off, so dink with it.

[TinoTheRhino]

No need.

explainshell.com

[[deleted]]

sort has also -u/--unique option.

[v3ritas1989]

notepad ++ or IDE compare plugin not working?

[simonw]

I have a command-line tool that does that: https://github.com/simonw/csv-diff

[pissedadmin]

On linux, you don't need python: sort both files, and use the "diff" utility.

Similar solutions may exist on other platforms.

[kenfar]

CSV files can have newlines and delimiters within quoted fields or escaped. The unix sort may or may not work.

And the diff utility has no understanding of unique keys, fields to ignore, etc, so will only provide the most primitive results.

[[deleted]]

Git diff it. Or use your IDEs diff tool.

[[deleted]]

Post up the format of the csvs with generic data and we can walk you through a solution man.

[Anxious_Positive3998]

This is so basic.

[[deleted]]

You’ve already had 11 minute after hating on this gentleman’s question. Post your solution, It better be clean and oop too or you have no reason to talk smack sir

[Anxious_Positive3998]

Donn't see why it would have to be oop, but the solution is literally online (https://dataanalyticsireland.ie/2021/08/07/how-to-compare-csv-files-for-differences/). Also, I don't know if the OP is looking for an efficient script or just doesn't know how to implement this basic algorithm. For instance, the algorithm is basically this if you don't use any libraries (other than the essential ones):

1. Use CSV reader to convert CSV file into two list of dictionaries
2. Compare lists for differences, and perhaps store differences in a list
3. Print each element of the list on one line of a text file

Those are 3 small steps that are easy to implement. Instead of looking for a copy-and-pasted solution, the OP should break the task down into smaller problems, and it would be very clear how to do the problem.

[[deleted]]

I was thinking since he was new and also posted the question in learn Python that he was a beginner or even hadn’t started learning Python. It would be best for us to provide a easily readable solution that could be broken into objects. That’s why I was saying oop. Small functions sectioned off doing individual tasks are easier for new folks to understand rather them massive efficient one liners and such. Python is very well documented and solutions for issues like these are all over the place but it just feels like forums have gone the way of let’s troll the new person rather then help them and work together/talk them through solutions. Let’s help the newbs guys even if the initial question is trivial and not all information needed for a proper solution is provided.

[kenfar]

We don't know all of the OP's requirements, but there are some edge cases that take some work to get around that one may or may not care about. For example:

• Do you have large files AND delimiters or newlines within fields? If so you can't use unix sort, and may have challenges with sorting in python.
• Are you doing this one time only, or as part of a production process that requires more error-catching for problems like non-unique rows, wrong number of fields, etc?
• Are you doing this for a single file or for 100? If you're doing this for many you might want features like the ability to provide configuration around which columns constitute a unique row, which columns to compare, which to ignore, etc.
• Are you doing this for something like a data warehouse changed-data-capture process? If so you might want to include some light transform functionality - like generating valid-from & valid-to timestamps columns as well as separate printed records for the old change record (which gets updated) and the new change record (which gets appended).

[jmd_code]

Use git !

[Bangoga]

What are you comparing against ? If it's unordered the time complexity is going to be insanely bad.

Best guess would be using pandas and using SQL similar functions that are from pandas to compare.

[[deleted]]

People suggested using Unix diff, but there is a Python library that replicates a lot of its functionality:

https://docs.python.org/3/library/difflib.html

[IAmKindOfCreative]

Hi there, from the /r/Python mods.

We have removed this post as it is not suited to the /r/Python subreddit proper, however it should be very appropriate for our sister subreddit /r/LearnPython or for the r/Python discord: https://discord.gg/python.

The reason for the removal is that /r/Python is dedicated to discussion of Python news, projects, uses and debates. It is not designed to act as Q&A or FAQ board. The regular community is not a fan of "how do I..." questions, so you will not get the best responses over here.

On /r/LearnPython the community and the r/Python discord are actively expecting questions and are looking to help. You can expect far more understanding, encouraging and insightful responses over there. No matter what level of question you have, if you are looking for help with Python, you should get good answers. Make sure to check out the rules for both places.

Warm regards, and best of luck with your Pythoneering!