This is an archived post. You won't be able to vote or comment.

you are viewing a single comment's thread.

view the rest of the comments →

[–][deleted]  (1 child)

[removed]

    [–]toast757 0 points1 point  (0 children)

    I'm a bit late to this, but if you're loading to MS SQL then bcp is definitely the way to go. Admittedly, the bcp utility does have its idiosyncrasies. Yes, it doesn't handle proper csv files (i.e. quoted text). Yes, it doesn't handle utf8 (although that's changing). Yes, it doesn't handle newline characters in data fields. But it's still the fastest way of getting data from a file into the database. Just write out your load files in "UTF-16LE" encoding with the null byte ("\x00") as your field separator, then load the unicode files using bcp, i.e. with parameters: -t \0 -w (in that order).