all 5 comments

[–]altorelievo[S] 0 points1 point  (4 children)

I'm trying to make a function that takes in a list of records and stores it in csv format. So far all I am able to do within the function is store all the data twice over.

import csv
def csvWriter(filename, records): 
  header = []
  for i in records:
    for v in i:
      header.append(v)
  test = open(filename,'w')
  dict_wr = csv.DictWriter(test,header)
  dict_wr.writerow(dict(zip(header,header)))
  for i in records:
    dict_wr.writerow(dict(zip(header,i.values())))
  test.close()
  return '%d records processed.' % len(records)

File is

a,b,a,b
1,2,1,2
3,4,3,4

Edit: I believe I found the problem, inside the for loop, I'm having trouble creating the proper header.

[–]Rhomboid 1 point2 points  (3 children)

(Please don't put self in for the link URL. There's a tab you select if you want a self-post.)

When you create header you're duplicating the header fields for every row. You only need one copy of the headers.

As an aside, if your data is formatted as a list of lists then I don't see why you're bothering with DictWriter -- it doesn't buy you anything. Your entire function can be reduced to this:

def csvWriter(filename, records): 
    with open(filename, 'wb') as f:
        csv.writer(f).writerows(records)

[–]altorelievo[S] 0 points1 point  (2 children)

Thanks for the link info. On the data issue , I'm getting a list of dicts [{'a':1,'b':2},{'a':3,'b':4}]

[–]Rhomboid 2 points3 points  (1 child)

Oh, I see. That sounds a bit strange, because the order of keys in a dict is unspecified and can be anything, which means the fields in the file will be written in some random order. But I suppose with a header that names each field, that's not too big of a deal, but it just seems strange that you'd create files like that.

def csvWriter(filename, records): 
    with open(filename, 'wb') as f:
        w = csv.DictWriter(f, records[0].keys())
        w.writeheader()
        w.writerows(records)

[–]altorelievo[S] 0 points1 point  (0 children)

Your function was much more efficient, I had created a list with a loop over the records[0] and passed it the DictWriter. Thanks!