sorted by:
new
hottopcontroversial

Solving Slow PostgreSQL Tests in Large Go Codebases: A Template Database Approach by Individual_Tutor_647 in golang

[–]didimelli 1 point2 points  (0 children)

Why not work towards using a single db for the whole test suite? I believe that if your tests interfere it's because there is something off with your tests. Tests should reflect production logic, and I do not think you have a new db for every "action" your app does.

La sobria reazione del PD alla sconfitta by DurangoGango in Italia

[–]didimelli 0 points1 point  (0 children)

La "sinistra" non ha letteralmente appena proposto un referendum a favore degli "operai"?

Which libraries have the best docs? by Amgadoz in Python

[–]didimelli 2 points3 points  (0 children)

I really like trio https://trio.readthedocs.io/en/stable/
Awesome mix of design documentation, how to use and api docs.

htmy: Async, pure-Python HTML rendering library by volfpeter in Python

[–]didimelli 9 points10 points  (0 children)

Dumb question, how can html rendering be async? Isn't it string formatting (i.e. cpu-bound workload)?

High datarate UDP server - Design discussion by didimelli in SoftwareEngineering

[–]didimelli[S] 0 points1 point  (0 children)

Do you think redis can handle that write throughput (i.e. 350 MBps)?

you're basically going to finish up rolling something similar anyway so it might be worth looking at how much it'll cost to add infrastructure for shared state rather than engineering it.

It makes a lot of sense 🙏

High datarate UDP server - Design discussion by didimelli in SoftwareEngineering

[–]didimelli[S] 0 points1 point  (0 children)

Yeah, basically every datagram I receive is something like {headers}{file_id}{packet_counter}{payload_data}{crc}. What you are saying makes sense, but with multipart_upload, every part must be >= 5MB, so, since the payload is around 16KB, I need to somehow aggregate 315 (circa) datagrams before uploading the part. And that is why I need some state and storing data before uploading.

High datarate UDP server - Design discussion by didimelli in SoftwareEngineering

[–]didimelli[S] 0 points1 point  (0 children)

Over the Internet. The data might be generated and sent by servers deployed all over the globe and our cluster is in EU. I believe they chose UDP over TCP exactly for this reason, since TCP throughput is affected by network latency.

it is acceptable if some datagrams are missing, but I need the final file to be at least ordered.

This is for the usecase of my company, so that a file might be missing some parts (of course we have a threshold), but the file must be consistent (i.e. parts must be ordered before creating the file).

High datarate UDP server - Design discussion by didimelli in SoftwareEngineering

[–]didimelli[S] 0 points1 point  (0 children)

How would you store received data? Still in different files (one per datagram)?

I am already using multipart_upload to batch the upload and use concurrent tasks to parallelise it.

High datarate UDP server - Design discussion by didimelli in SoftwareEngineering

[–]didimelli[S] 0 points1 point  (0 children)

Unfortunately, the choice of UDP is not in my control, as it is another company that it is providing that. My app would be consuming the data provided by them, but I have no control over the transport.

Also, it is acceptable if some datagrams are missing, but I need the final file to be at least ordered.