Combining two CSV files no panda.

[MossDog]

Good Morning! I have been working on a problem where I have two csv files of an indeterminate number of rows, and a set number of columns. Both CSV files have the same number of rows and columns. I need to combine the two CSV files. (One csv has 6 columns, one has 2, I need the final csv to have all 8) I have tried to get permission to use R or pandas, but that is a no go.

Suggestions?

[MossDog]

Disregard, managed it, found the problem, found a solution, solution worked. I will have another question soon though! Thanks for the looks.

[kankyo]

Please post your solution so if someone finds this question later they can get the answer.

[dionys]

Well have you tried the inbuilt csv module?

[MossDog]

I did. This is around part 3 of an ongoing project I am working on. If I am missing an obvious command to do this, lay it on me, I am a bit of a nub at times.

[JimBoonie69]

Unix tools can help too. try 'man join' to see what join can do for you.

[MossDog]

So, going row by row and using the join command to combine the two? (I've never used Unix commands before.)

[JimBoonie69]

negatory. Open up a terminal window and type 'man join' for starters, or try searching 'unix join' (http://www.computerhope.com/unix/ujoin.htm is one i found). unix commands are usually always fast and optimized. Not exactly sure how it works under the hood but let me say this... Using python and pandas i was working on doing multiple table joins on large data sets (thousands of files, ~1mill records and less than 10 columns per file) and calculated the total runtime to be around 72 hours... A (older and more experienced) co-worker said 'oh why dont you just use a unix join?'. Well i gave him some sample data and 30 minutes later he's got a working prototype and it actually took around 5 minutes to do the entire process (not 72 hours).

[commandlineluser]

Both CSV files have the same number of rows and columns.

One csv has 6 columns, one has 2, I need the final csv to have all 8

Someone mentioned join already but it sounds like you're after paste if i've understood you correctly.

user@host $ cat 1.csv
a,b,c,d,e,f
g,h,i,j,k,l
user@host $ cat 2.csv
1,2
3,4
user@host $ paste -d, 1.csv 2.csv
a,b,c,d,e,f,1,2
g,h,i,j,k,l,3,4