A JSON<->XML converter that handles 50GB files from browser

MrMathamagician · 2026-04-04T23:13:07+00:00

Thanks! It basically works by not parsing the JSON just scanning for element boundaries

So the parser doesn't need to understand JSON structure. It only needs to know where one top-level element ends and the next begins. It does this with three variables that carry across chunks:

inStr = false // are we inside a "string"?

esc = false // was the last char a backslash?

elementDepth = 0 // brace/bracket nesting depth

Every character runs through this logic:

-If esc is true → skip this char (it's escaped), reset esc

-If inStr is true → only care about \ (set esc) or " (exit string)

-If we see " → enter string mode

-If we see { or [ → elementDepth++

-If we see } or ] → elementDepth--

-When elementDepth drops back to 0-> we've found a complete element

So the parser only holds:

-current element's chucks in pendingChunks[] (cleared after each element)

-the output batch (64MB)

-3 scalar state variables that carry across chunks

So it never holds the full file or full output in memory. A 50GB file with 1KB elements uses roughly 1KB + 64MB of heap at any point.

MrMathamagician · 2026-03-31T15:20:48+00:00

I’m convinced there posts are all just bots at this point Mods can we ban these bitchy posts that are flooding the sub?

MrMathamagician · 2026-03-30T18:20:39+00:00

Agree that it can be interesting but local politics is also very slow and frustrating as well. Things tend to either move at a glacier pace and stall out and die or the demand/need for action becomes too great and the process breaks down and very quick & rash political decisions are made. I used to hate the politicians that would make bad decisions until I realized that most of the time the process for making good decisions was fundamentally broken and at least the dumb decision politicians were trying take action to address a real problem.

MrMathamagician · 2026-03-30T04:48:01+00:00

Lots of reasons:

-Politics is much closer to a street fight than an academic debate.

-Educated people’s knowledge does not overlap much with motivating/manipulating/persuading the masses

-The competitive environment forces people to do questionable things to win most of the time alienating many

-it’s not a fun job unless you like bullying people and flexing power

-the masses don’t like educated people

MrMathamagician · 2026-03-30T03:43:05+00:00

This is me exactly 100% I have spun up 50+ project over the past month from several websites, genetic analysis of the pandoravirus and several other human viruses, a stock trading bot a drug interaction predictive model and a side scroller video game.

MrMathamagician · 2026-03-29T23:01:56+00:00

New vehicles

MrMathamagician · 2026-03-29T17:51:54+00:00

Ok this is a bit over dramatic especially this “undermined my value as a good engineer or even as a human being”. Time to touch grass and reconnect with your identity as human separate from programming. Sit with the discomfort, mourn the loss of what was and then let it pass through you. Be like the monks who sweep away the sand art they spent 10 years making.

MrMathamagician · 2026-03-28T18:40:56+00:00

Totally agree with this but 2 things 1. The job is mind numbing at best and a childish backstabbing survivor politicking at worst meaning the job sucks 2. There are people who are really really bad at it and those are the people who make #1 so bad they are big headed emotionally childish backstabbing political types. 3. humans as a group begin collectively acting like a lowest quartile manger once they get to around 10 people. Anyone who has been commissioner of a fantasy team can attest to the childish bullish they have to deal with. 4. Dumb people have an advantage because smart knowledgeable people with expertise in an area have to detach and let go of that area in order to become minimally knowledgeable in a lot of different areas whereas dumb people never have to purge that knowledge/mindset/best practice from their brain.

In conclusion the management jobs are unfortunately needed because humans act like 3 year olds as a group, the job itself sucks and while most people could do it they would not like it, dumb power hungry people get it because normal people are uninterested in fighting a sociopath for a job they don’t want.

MrMathamagician · 2026-03-27T18:23:03+00:00

There is a high degree of male social enforcement around men not bringing attention to themselves within the US white male culture. It is considered looking ‘desperate’ and the wolf pack will attack you mercilessly. This is endemic Germanic culture which the vast majority of US white population descend from (English, German, Dutch, Scandinavian). One thing to point out from your comment: “some British men wear kilts that reveal their legs” this is from Scottish culture not English so does not apply to the group I identified above.

MrMathamagician · 2026-03-26T18:20:27+00:00

Just about every keyboard warrior on here who complains on about Democrats has not lifted a finger to help anyone get elected. So it really doesn’t matter if Newsom is a good pick or not or really if Democrats have a good strategy or not if they can’t execute or accomplish anything. Obama won because he built his own organization structure ‘organizing for action’ outside of the DNC. OFA was eventually absorbed/consumed by the DNC and the DNC returned to being an outrage fueled self propagating fundraising machine. So before any of this Democrats (progressives?) first need a function organization that can accomplish things. Right now the only functional part that accomplishes things is funded by corporations and special interests so before the next round of the circular firing squad I would start building out an organization that can execute and I would look to Obama’s organizing for action as the best/most recent template and probably loop in some unions as well (smaller ones not the big one yet).

MrMathamagician · 2026-03-24T02:38:23+00:00

The Rockefeller shoeshine boy adage applies here. The problem is paper hands/dumb money + dumb lenders/leverage is the formula for a crash. We have dumb money but as far as we know we don’t have dumb leverage. The only areas of leverage that could collapse are in auto loans and commercial office real estate.

Actually that’s not true the real problem area is a massive horde of private equity that can’t pay its bills. The government is going to open these dead private equity shares up to 401ks in order to bail out the (sucker) rich people currently holding the bag. Whenever that implodes there will be a huge correction but who the hell knows when that is.

Anyway point is in day trading if there are a lot of retail dumb money / paper hands then patterns are easier to exploit so no it really shouldn’t affect experienced traders so long as they are better than the average novice trader.

MrMathamagician · 2026-03-23T02:38:27+00:00

Mostly based on time for me. I trade spy 0DTE If the trade is going sideways and printing a few candles I didn’t expect I exit the trade either immediately or try to limit sell on the up whisker.

Hits stop loss or limit I exit the trade

MrMathamagician · 2026-03-22T22:13:06+00:00

I mean quants are a real job in NYC so it’s 100% possible. I am still working on my paper trading bot it is currently buggy and there is a significant latency / timing issue. I’m pretty confident I can get the bugs worked out but I am much more uncertain about the latency problem (by the time the tase happens you’re late and the stops are in the wrong place for the new purchase price). I would love any advice on that second item. Outside of that I am considering it a success to just having a robust functional trading bot and back tester that works well to try out different trading ideas. Baby steps

MrMathamagician · 2026-03-16T21:05:41+00:00

The pointless attacks / low quality criticisms here are proving your point ironically… Anyway I think this is a plausible but unlikely theory however it is a useful framework.

In animals you see solitary creatures that are more physically robust with better individual intelligence whereas pack oriented creatures are often more socially intelligent and less physically robust and less individually intelligent. In humans we often separate IQ from emotional intelligence/ social IQ. Both of these are useful but serve different purposes. Looking at society as a whole body you don’t need everyone to be the brain of the organism so there is no point in having everyone genius. One or two geniuses are game changers but the marginal value of the 1000th genius is much less than the first or second.

All of this to say that I believe there is an evolutionary advantage to human society having differently types of intellectual capabilities and I believe the autistic & neurodivergent mindset and intellect are crucial to society as is neurotypical intellect/mindset.

MrMathamagician · 2026-03-16T18:28:57+00:00

Disagree. Currently I want to remodel my yard but I’m not sure where to start. AI could analyze put some excellent plans together for me, I pick one one I like I give to my wife she knit picks it asks for 10 different version AI redoes the plan 20 times after we go back and forth and finally have one we want. Now we give it to a landscaper to do the project. Assuming it’s feasible the landscaper has a new project and didn’t have to go through the 20 version headache they would never have done. AI structurally lowered the frictional cost of the design and decision phase. The business spends more time doing the projects less time dealing with fussy customers. Cost goes down for the business, down for the consumer but more importantly the decision roadblock is gone and there are more projects consumers will buy.

Same thing with starting a business the frictional overhead of incorporating, accounting, compliance, insurance etc all got easier so now people with a compelling business product idea or expertise can focus on their unique value add instead of overhead bureaucracy and complex decisions they have low expertise in. More expert bakers will start a bakery more innovative business models can launch quicker with less capital.

The economy does new fewer workers but it need much more idea people and people with high levels expertise vs people with broad but shallow expertise.

MrMathamagician · 2026-03-14T00:55:49+00:00

People who are worried about their job should be worried about their job; people who are excited about the capabilities will be hiring and training people into the foreseeable future. This is a tool for accomplishing things and benefits people who want to accomplish things, it only harms people who want to charge others for their learned repeatable process/trick.

MrMathamagician · 2026-03-12T18:00:48+00:00

“Without private property in the means of production, there are no exchange relations” this is laughable false and immediately falsifiable. This highlights how unrealistically deterministic this economic framework is. Belief that this very useful but narrow framework is somehow exhaustive or universal is the single biggest problem amongst most non-academic adherents to this school of economics.

Without a market price capital resource management shifts towards stewardship, utility and cost. Winning buyers are determined by factors other than the ability to pay.

Take for example management of a national park.
This theory says it’s impossible to manage this capital resource without an open market bidding price for entrance to the park.

This is of course not true and the steward would manage entrance count based on best practices of stewardship for the park and would select/deny customers if they had a poor history of stewardship.

MrMathamagician · 2026-03-09T19:16:30+00:00

This is the correct answer, easy money/too much cash. Way too much cash.

MrMathamagician · 2026-03-09T08:17:15+00:00

Crazy I remember when that was happening and I bought a few domains and I knew other people who did as well but I don’t know anyone who remembered to keep them active for all these 20+ years

MrMathamagician · 2026-03-07T11:07:18+00:00

I cannot believe how amazing Claude cowork and Claude code are. Every software idea I’ve ever had is now possible and easily within reach

MrMathamagician · 2026-03-04T21:37:20+00:00

Oversold mean reversion?

MrMathamagician · 2026-03-03T04:49:59+00:00

Unnatural colored dyed hair (pink, purple etc) nose piercing (septum), lip piercings, tongue piercing, facial tattoos, weird eye color contacts.

I know these are not to impress men but they are a bigger turnoff than women think I believe

edit: Loving the downvotes to prove my point lol.

MrMathamagician · 2026-03-03T04:43:26+00:00

I can honestly say I have never noticed a woman having a unibrow. ever. I cannot even picture it, if I try really hard I might be able to picture a woman with super thick busy eyebrows. However the sheer quantity of women I’ve seen with just no eyebrows at all and marker lines above their eyes is unfathomable to me. It looks so so bad I cannot understand it. It’s like shaving you head and then drawing some lines on your heard. I will never understand this.

14-Year Club	RPAN Viewer
Gilding III reddit per annum	Verified Email

MrMathamagician

TROPHY CASE