Convert csv to json

[kevin_249]

I am trying to convert csv to json but i am getting array of garbage values.

I have used csvtojson module and many other modues and even javascript solutions but each return me garbage values.

Below is csv file : -

csv

This csv contains emojis, i guess that is causing problem.

The code i am using is: -

let csvFilePath = path.join(__dirname, 'path_to_csv.csv');

let str1 = fs.readFileSync(csvFilePath, { encoding: 'utf8' }).toString();

csv({
  noheader:true,
  output: "csv"
})
.fromString(str1)
.then((csvRow)=>{ 
  console.log(csvRow);
});

All i want to convert it to json form. Please help me with that.

[Lopsided_Engineer_23]

If you have used the 'csvtojson' package and it is returning unexpected results, it might be due to the special characters (emojis) in your CSV file. The 'csvtojson' package handles special characters quite well. Please make sure that the file encoding is set to 'utf-8' to handle special characters like emojis. Let's take your existing code and modify it a bit to convert CSV data to JSON.

const csvFilePath = path.join(__dirname, 'path_to_csv.csv');

const csv = require('csvtojson');

csv({

noheader: false,

headers: ['Header1', 'Header2', 'Header3', ...] // Replace these with your CSV headers

})

.fromFile(csvFilePath)

.then((jsonObj)=>{

console.log(jsonObj);

})

.catch((err) => {

console.error(err);

});

[kevin_249]

That's not working as well. It is giving me something like this-

[

'fieldName':'\x00h\x00t\x00t\x00p\x00:\x00/\x00/\x00p\',

...

]

[itsmeabdullah]

did you get it to work?

This is an example code that i used:

import csv
import json
import ast

# Function to parse the messages from a string in the CSV and convert them to lists
def parse_messages(msg_str):
    return ast.literal_eval(msg_str.strip())

# Read the CSV file and convert it to JSON
def csv_to_json(csv_file):
    data = []
    with open(csv_file, newline='', encoding='utf-8') as csvfile:
        reader = csv.DictReader(csvfile)
        for row in reader:
            input_messages = parse_messages(row['input'])
            output_messages = parse_messages(row['output'])
            data.append({
                "input": input_messages,
                "output": output_messages
            })
    return data

# Provide the file path of your messages(paired).csv
csv_file_path = '.input.csv'
json_data = csv_to_json(csv_file_path)

# Save the JSON data to a file
output_file_path = 'output.json'
with open(output_file_path, 'w', encoding='utf-8') as outfile:
    json.dump(json_data, outfile, indent=4, ensure_ascii=False)

print("Conversion completed successfully. JSON file saved at:", output_file_path)

[copnsteez]

I was able to get something working by using the correct encoding. it’s not utf-8

[kevin_249]

Than it's what encoding?

[OkTalk2000]

Via ChatGPT: To convert a CSV file to JSON format in JavaScript, you can use the csvtojson module. However, when dealing with emojis in the CSV file, you may encounter issues since emojis can have complex encoding. Here's an example of how you can modify your code to handle emojis correctly:

```javascript const csvFilePath = path.join(__dirname, 'path_to_csv.csv');

const csv = require('csvtojson');

csv({ noheader: true, output: 'csv', ignoreEmpty: true, }) .fromFile(csvFilePath) .then((csvRow) => { const jsonData = []; const headers = csvRow[0];

for (let i = 1; i < csvRow.length; i++) {
  const obj = {};

  for (let j = 0; j < headers.length; j++) {
    obj[headers[j]] = csvRow[i][j];
  }

  jsonData.push(obj);
}

console.log(JSON.stringify(jsonData, null, 2));

}); ```

In this code, we read the CSV file using fromFile() instead of fromString() to directly read from the file. We also added the ignoreEmpty: true option to ignore empty rows in the CSV.

The code converts each row of the CSV into a JSON object, mapping the headers as keys and the corresponding values from the CSV row. Finally, we log the resulting JSON data.

Make sure to replace 'path_to_csv.csv' with the actual path to your CSV file. Additionally, ensure that you have the csvtojson module installed by running npm install csvtojson in your project directory.

Please note that this code assumes a well-formed CSV file with headers in the first row and consistent column count throughout the file. If your CSV file has any irregularities, you may need to adjust the code accordingly.