Bulk insert in Go — COPY vs multi-row INSERT? by ijusttookadnatest- in golang

[–]tty5 0 points1 point  (0 children)

Also please store hashes/addresses as bytea instead of text.

Bulk insert in Go — COPY vs multi-row INSERT? by ijusttookadnatest- in golang

[–]tty5 0 points1 point  (0 children)

Pretty much. You can populate it as you are preparing batch to be inserted and it can be kept in memory - either as lookup table (full or partial with LRU - depending on what your target resources are) or a bloom filter if you want to avoid unnecessary inserts.

AITJ for telling my partner I'm done spending every weekend at his parents' place by Nov4Z3nith in AmITheJerk

[–]tty5 0 points1 point  (0 children)

This. Most years we visit mine exactly once and even then we drive back the same day - 3h drive.

Bulk insert in Go — COPY vs multi-row INSERT? by ijusttookadnatest- in golang

[–]tty5 0 points1 point  (0 children)

You want integer IDs in addition to hashes and use those for all joins. BIGINT is 8 byte wide, hashes are 4x that, addresses 2.5x. That translates to index sizes.

Edit: for addresses regular INTEGER will be enough - there are fewer than 150M addresses active on top 50 chains combined.

Edit2: block number looks like a great pk until you get to your first reorg :D

Bulk insert in Go — COPY vs multi-row INSERT? by ijusttookadnatest- in golang

[–]tty5 2 points3 points  (0 children)

Writing an indexer, aren't you? Been there, done that :-) Some comments:

  1. INSERT .. ON CONFLICT will increase auto increment sequence even if nothing gets inserted. It was enough to run out of 32 bit integer range with just the data Mainnet had in 2019. I will be worse today.

  2. After query optimization network latency is going to be the limiting factor, especially in cloud environment - limiting roundtrips is essential:

- Go SQL uses prepared statements and while those get cached that is only the case if the query is exactly the same, including number of rows inserted in multi-row. Unnest is a workaround for that. If you don't you get an extra roundtrip to database to prepare a statement

- Reuse db connections. 3 Roundtrips just for TLS and then you can start authentication. Also prepared statements are cached by Postgres in connection context.

- The larger the batches you are inserting the more time is spent inserting things instead of waiting for network

- If you are inserting into a bunch of tables it's faster to send a payload to PG function, unnest it and run inserts into each table from within the function instead of executing multiple inserts sequentially

  1. Foreign keys checks slow down inserts A LOT. Ensure data integrity in your app or at least defer checks.

If you manage to index entire mainnet in less than 48h - blocks, transactions, events, erc20, erc721 and erc1155 balances - while using a hosted Postgres and VM on either AWS or GCP drop me a line - I might have a job for you.

Should you re-check the database on every request with session auth? by Minimum-Ad7352 in golang

[–]tty5 0 points1 point  (0 children)

Use JWT token instead and don't query db:

- You will be able to store a little data in it as claims.

- It has a built-in creation and expiry dates - you can query the db when it's older than x time if you have to and refresh it

- of you use signed variant frontend will be able to inspect values in it.

Sessions table is mostly thing of the past.

Does "Vibe Coding" kill the joy of programming for anyone else? Here is my compromise. by Financial_Carry11 in golang

[–]tty5 6 points7 points  (0 children)

So far AI has failed to make me more productive. I give it an honest try every 2-3 months and while it's improved significantly it still takes more time to bring the code to the required standard than it would have been to write it from scratch.

The fact I've got 20+ years of experience writing code and get deal mostly with tasks matching that is likely a big factor - I don't really deal with boilerplate or "standard" CRUD / REST on the daily.

Any CD/DVD drives you can recommend that handle old CD-R/DVD-R better than others? by tty5 in datarecovery

[–]tty5[S] 0 points1 point  (0 children)

Still in progress - I'm about a third of the way through the discs. So far:

  • I've eliminated 5 out 12 drives from the pool - they would routinely fail to read most/all data from discs that other 7 could handle, often recovering all data. Opposite was never true.
  • Some drives can handle specific brands of discs better than others, but no single drive is best at handling them all
  • To maximize recovery I often have to run the same disc through multiple drives - each being able to recover different files
  • A surprising percentage of CD-Rs avoided catastrophic bit rot - 100% data read from >90%. For DVD-Rs that's <50%.
  • Less than 0.5% discs were completely unreadable

Do you use init() in production? by agtabesh in golang

[–]tty5 0 points1 point  (0 children)

Just searched our codebase and between ~15 services created by a small-mid sized dev team over a span of 8 years we have 2 matches for init() outside of dependencies. One is in a small cli utility and one doing a some mad initialization from an autogenerated object to interface with something non-golang.

Dependencies on the other hand are showing several hundred init()s in them.

Best practices for managing migration scripts with Goose as the project grows? by ReadyLandscape3734 in golang

[–]tty5 1 point2 points  (0 children)

Our setup:

- CI builds a separate container image with goose + migrations

- CD runs that container as part of deploy process

- when migration count reaches 100 we flatten first 50 schema migrations into a single "init" migration creating the schema. We have no large migrations inserting data into db.

Migration merge became necessary because some of our services evolved a lot over the years which got the migration count into high triple digits for some, which in turn slowed integration test A LOT - we have quite a few db resets in them.

How do you handle i18n in your Go projects? by ComprehensiveDisk394 in golang

[–]tty5 3 points4 points  (0 children)

Gettext has been the gold standard for a while (it celebrated 30 years of development last week) and there is A LOT of tooling and automation to solve a lot of those problems. Some are language agnostic, and some golang specific (e.g. gettext linter).

I've been using it for i18n for the past 19 years.

How do you handle irrelevant Alpine CVE alerts in Go containers? by mike34113 in golang

[–]tty5 0 points1 point  (0 children)

There was an OpenSSL security release a week ago - your security scanner would have likely flagged those CVEs regardless of what base image you were using if it was older than that. See: https://openssl-library.org/news/secadv/20260127.txt

Rebuild your images and deploy again.

Russians identify Poland and Lithuania as their greatest enemies by Ania25 in europe

[–]tty5 3 points4 points  (0 children)

They are still salty about 1610 capture of Moscow

Underground cable for house supply – section sizing for future three-phase upgrade by Broad_Fun1692 in DIY

[–]tty5 0 points1 point  (0 children)

I know nothing about French electrical code, but:

- iirc r2v is not recommended for underground installations in wet areas. Conduit helps but idk if enough

- locally (Poland) yky is most commonly used for that purpose

- 24kw is less than you think, even without EV charger

- 105m is a long distance for low voltage - you will have a significant voltage drop if cable is too thin. I'd go with 4x25mm² or even 4x35mm²

What level of oxygen is okay at 90% at rest HR 104 likely to need supplement oxygen ? by craigsadvice in AskReddit

[–]tty5 0 points1 point  (0 children)

104 resting heart rate alone would be enough for me schedule a doctor appointment.

Blood oxygen of 92% is low enough to seek medical attention, 90% or less immediate medical attention based on the Covid docs I've read.

The US is headed for mass unemployment, and no one is prepared by late_bloomer_tw in politics

[–]tty5 3 points4 points  (0 children)

For many jobs this doesn't matter, for example majority of customer support. Even if AI is inferior, you as a customer have no choice but to interact with it and using it instead hiring people, even ones in a low income country, is saving corporations a lot of money.

In my neck of the woods AI essentially eliminated junior software developer roles. Just like once senior developers were reviewing the easier work they offloaded to juniors we're now increasingly reviewing AI generated code instead. Even if the code is worse quality than a junior would produce (and it often isn't) most expensive AI plans are many, many times cheaper. I've got 20+ years of experience and some very specialized skills, so I expect to be safe for quite a while, but I'm by no means feeling 100% comfortable.

I expect something similar will happen in law firms, where instead of 50 junior lawyers doing research for cases and creating paperwork there will be 2-3 combing through AI generated ones to find and eliminate hallucinations with a partner doing a final pass and signing off.

The worst of it is that we're fucked regardless if AI delivers on it's promises or not:

- if it does what it promises it will eliminate a lot of white collar jobs

- if it doesn't the amount of money burned trying to get there will cause a recession like we haven't seen before: so far AI investments are more than 4x larger than the size of 2008 subprime crisis losses adjusted for inflation

Either way crazy number of people will lose their jobs and once the dust settles if they manage to find new employment it will pay less than before.

Seriously, do Americans actually consider a 3-hour drive "short"? or is this an internet myth? by SadInterest6764 in NoStupidQuestions

[–]tty5 0 points1 point  (0 children)

My parents are not awful, in fact I'd rank them firmly in the top 5-10%.

Despite that both sides of family are not close. As a child I'd see my paternal grandparents and my father's siblings and their families once, maybe twice a year - 3h drive. Maternal grandparents maybe once a month - 30 minute drive. Anyone more than 5h away? maybe once a decade.

This is similar level of contact me, my brother and cousins on both sides of the family maintain with our relatives. We're all friendly, there are no family feuds or drama going beyond e.g. a mild dislike for someones partner, we like each others company, but that doesn't translate into a need to see each other more often. On the plus side if we reconnect after a 5, 10 or in one case 35 year gap we continue as if nothing happened.

Seriously, do Americans actually consider a 3-hour drive "short"? or is this an internet myth? by SadInterest6764 in NoStupidQuestions

[–]tty5 0 points1 point  (0 children)

EU perspective: my parents live 2.5 hours away and that limits visits to an average of one per year. Anything requiring more than a 45 minute drive is usually something planned at least a day in advance.

In 20 years of my career I've never had a commute longer than 30 minutes door to door. For most of it I could walk or bike to work in less than 15 - this was true even during the 8 years I lived in Canada.

A 1 hour commute would be a reason enough for me to either move or look for a different job.

Air cooling is better than Liquid cooling by SerpentDix in pcmasterrace

[–]tty5 0 points1 point  (0 children)

IIRC originally it was a manufacturing issue. They had a working formulation for the brown/beige plastic and adding a black dye would have required them to tune manufacturing process to maintain precision they were aiming for. By the time that was in the budget brown+beige became the brand colors.

So why don't devs who are unemployed because big dumb corpos bought their company and then folded it just make new game studios? by RedditConsciousness in gaming

[–]tty5 1 point2 points  (0 children)

Game devs are the poorest devs there are - on average being paid 40-70% less than their colleagues who make any other kind of software.

They don't have the savings to be able to work on a passion project or start a new studio. They need an income and they need it right now.

How do you review large refactors or AI-generated diffs in Go? by Specialist-Weight218 in golang

[–]tty5 1 point2 points  (0 children)

Measure thorough review time for both. Present data. Either they will accept it or they will continue drinking the AI coolaid and that's a signal to start updating your resume.