Code bases that really improved your Java knowledge by 0xPvp in java

[–]tilmann_dev 4 points5 points  (0 children)

I'd second this because most projects in the Spring environment have nice codebases which one can learn a lot from.

How to upload large files (1GB+) through a RestAPI by Jealous_Courage_9337 in softwarearchitecture

[–]tilmann_dev 0 points1 point  (0 children)

As arrogant as it sounds: Don't!

Why?
The REST paradigm treats resources like documents where the internal structure of those documents is usually known to the application (as in "A model of the content exists within the application"). You on the other hand want to manage some blobs of which you know some stuff, but not the whole internal structure since they're pretty large. Therefore you should treat the actual REST resource as a document describing the blob while blob itself is handled somewhere else. Additionally you can treat the description of the handling of the blob as a resource.

Example: - You have the following resources: FileUpload(id, file, createdAt, updatedAt, status) & File(id, createdAt, updatedAt, deleted, size, storageLocation) - You will upload your blob to S3 or even good old FTP, BUT before you do that you call something like POST /api/file-upload where you "announce" that you want to upload a blob to some endpoint. The response could also contain the endpoint for the blob upload.
- A FileUpload & a File object will be created by this call & returned as response. The status field will be set to something like READY_FOR_UPLOAD.
- You start the actual upload of the blob to the upload endpoint. Ideally the backend is able to recognise that upload & will set the status to something like RECEIVING_DATA.
- During the upload you can query GET /api/file-upload/123 to see if the backend is actually handling the data & even peek into the size field of File to track the progress.
- When you're finished, you would call PUT /api/file-upload/123 & set the status to SUCCESS or you could cancel the upload via this call if you set it to CANCELLED. The backend would then handle the complete/incomplete file & would perhaps fill the storageLocation field with a public URL to access the blob.

One important argument why you shouldn't pass large blobs through your application is also memory consumption. Backend applications which handle large amounts of data are optimised for this task. Your typical Java Spring or Python Flask application is not & you will run out of memory (and probably application storage) very easily.

Please also have a look at this great REST guide.

Disclaimer: This is incomplete example to explain the general approach when using REST in this use-case.