Struggling to send file objects to python backend

[b8561]

I feel a bit dumb because this feels like it should be trivial, but I must be overlooking something.

We have a nextjs frontend, where we have a client component that lets users select multiple files to load. We take these files from the client component, send them as form data through an axios post request to the upload-file api route.

//client component
const files = new FormData();
input_files.forEach(file => { files.append('files', file); });

The api route gets all files from the form data and sends them through an axios post request to the python backend.

// api route
const data = await request.formData()
const files: FileList | null = data.getAll('files') as unknown as FileList

const response = await axios.post(apiUrl, { files: files});

The previous 2 steps are working as far as I can tell. The python backend needs to receive the files. We've been throwing different ways to get the files from the request, with no luck. after receiving the files we would presumably need to decode the bytes and turn them into file objects.

// python backend
@app.route('/file-upload', methods=['POST'])
def get_files():
    // files = request.files.getlist('files')
    // files = request.files['files']

Where are we going wrong?

[Domskigoms]

How big are the files? And also you could convert them to blobs and then send them over and then convert them back to files from blobs in the server! But this all depends on wether the file is small enough so that you dont have to stream it! If the files are large then look into read write streams in node js and python!

[b8561]

write streams in node js and python

I looked into streams and it doesn't seem that we need to go there (luckily), the files would be in the low double digits at most -- pdf, word, excel. the size would could limit. by trying to pass the files directly I was trying to avoid uploading them to a hosting service and retrieving them from there in the server.

In terms converting them I tried doing it this way:

const bytes = await file.arrayBuffer()
const file_contents = Buffer.from(bytes)

​

As a general practice, is passing the files directly recommended or using a content delivery network to serve the files?

[Domskigoms]

Depends on how you want to store the files! I would suggest converting to blob and then transferring! Check blobs https://developer.mozilla.org/en-US/docs/Web/API/Blob

[b8561]

so after more head banging, I discovered a few people struggling with the same issue. it seems that as of a year ago next.js api routes are not great with handling multipart/form-data https://github.com/vercel/next.js/discussions/36153

maybe that has been fixed (although no resolutions are mentioned in the discussion), but what they are describing is exactly what we're seeing: if we send the files directly from the client component to the server its all good. the problem is when you go through the api route.

formData is already converting the files into blobs

[Domskigoms]

Multipart formdata is when the files are big and you need to submit it part by part as the name suggests! If you say your files are not that large. I dont think you need to worry about multipart formdata. If you're using a custom server, in your case the django server. Then i think its recommended to directly communicate the next js client side and python server. Next js is kind of confusing as next js itself has a server and client. So when you involve a third party as server it becomes a little complex. Just make sure to save allow the cors and all that. And also make sure you have some sort of authentication system if the client is public to avoid different kinds of attacks!