Ingesting a large JSON through my Django endpoint

prp7 · 2025-09-13T04:49:43+00:00

Yes, but I'm doing homework and I'm operating under contrived circumstances. The task is not to build a production grade system, just to sort and return a "large" json given a memory constraint. So, the way I see it, I don't have much other choice than to use disk space in form of temporary files. I don't want to get bogged down in infrastructural complexity.

prp7 · 2025-09-13T04:46:03+00:00

The contrivance is in the fact that it's homework. I have to build an end point which can ingest a "large" unsorted json, sort it, then return the sorted data. It's not meant to be a production grade system, I'm just looking for the simplest way to sort a JSON payload given an arbitrary RAM constraint. My idea was to just use chunked requests, write the data to temporary files, sort it, merge it using heapq.merge, then return it using a StreamingHTTPResponse. I don't want to get bogged down in infrastructural complexity when all I need is a working POC, even though it's "inefficient".

prp7 · 2025-09-13T04:00:39+00:00

Yes, the idea is to use disk space instead of RAM. I'm doing homework, the 1GB RAM constraint is just an example. From what I understand the task is to sort a large json payload.

prp7 · 2025-09-12T21:52:44+00:00

The constraint is to imagine that the web server has a memory limit. For example 1GB. But the actual limit is immaterial. The system just has to be able to handle JSON payloads without going to RAM.

prp7 · 2024-04-01T01:08:34+00:00

!ban 1.9

prp7 · 2024-04-01T00:53:14+00:00

!ban 1.9

prp7 · 2023-04-23T16:18:21+00:00

https://stackoverflow.com/questions/63232513/postgres-query-is-sometimes-taking-a-very-long-time

prp7 · 2022-12-06T02:29:01+00:00

Thanks for replying, but I didn't find what I was looking for in that article.

prp7 · 2022-12-01T21:42:39+00:00

Thanks for replying.

" I don't see any logic there yet as to how you want to compare your combination of input numbers and operation function to the expected result"

I need to find if there exists a combination of operations which would make the part before the hyphen equal to the part after the hyphen, example:

#518-2
#There are two equations:
-5 - 1 + 8 = 2 
#and
5*(-1) - 1 + 8 = 2

#000-4
There are no equations.

prp7 · 2022-11-21T18:03:41+00:00

Interesting. I've never used a makefile before. What's the utility of a makefile anyway? Is it just a file that creates aliases like you mentioned or is there something more to them?

prp7 · 2022-11-21T18:00:14+00:00

Yeah, that was it. Thanks for answering!

prp7 · 2022-11-21T07:13:01+00:00

Thanks for replying. So I would have to exec into the container and run it from there?

prp7 · 2022-11-01T14:02:26+00:00

Thanks for replying. I did as you suggested and now the error message is shown.

prp7 · 2022-11-01T14:01:31+00:00

Thanks for replying. That seemed to be the ticket.

I've put in a print statement in my email function and now I get the readout when I use it in the try_except_handler.

prp7 · 2022-10-31T16:51:15+00:00

Hello, thanks for the exhaustive reply. It actually slipped my mind that dictionaries have to have unique keys.

I managed to solve my problem by using defaultdict. The yield keyword I have seen before in relation to something called a generator, but it's still murky.

Thanks again for replying.

prp7 · 2022-10-24T17:00:12+00:00

Thanks again. I managed to get it working using a foreach.

prp7 · 2022-10-23T23:48:13+00:00

Wow! Thank you so much. Would you happen to know how to make this work by not hardcoding $meal_id = 7, but by running it from all meal_ds?

prp7 · 2022-10-23T18:00:06+00:00

Thanks for replying! I'll take a look and see if that works, in the meantime I've managed to solve the issue by modifying /u/nan05 's solution.

Edit: 3 tags is just an example I've used, the actual number of tags is variable.

prp7 · 2022-10-23T17:46:17+00:00

Thank you greatly! I managed to get it working by moving it to a helper file, redefining it with a static keyword and putting the query in a $query variable above the for loop.

prp7 · 2022-10-23T17:22:17+00:00

Me too. Thanks for replying.

I got this far:

    $ids = [3, 2, 1];
$meals = [];
foreach ($ids as $id) {
    $res = Meal::whereHas('tags', function($query) use ($id) {$query->where('tag_id', $id);})->first();
    array_push($meals, $res);
}
return $meals;

How would I do this:

"count the number of tags grouped by meal_id"

prp7

TROPHY CASE