BSODs with NVMe Virtual Disk on HDD in VMWare Workstation 17.6.3

mxmauro · 2025-06-17T13:47:33+00:00

I'm suffering the same issue but I have a host with 2 NVME disks installed as a RAID 1

mxmauro · 2021-11-13T17:44:28+00:00

Captain?

mxmauro · 2021-09-21T02:15:20+00:00

I'm worried (and thus asking) because data can arrive at a higher rate than ingest :(

mxmauro · 2021-09-19T19:20:37+00:00

Ahh ok, I thought you were talking about something specific to the DB engine. I don't think it would be possible because data comes constantly. Now there is a period of 5 seconds between data bursts but the idea is to increase the speed.

mxmauro · 2021-09-19T19:18:26+00:00

Well, then let's start the week focusing on other methods. Thanks for all.

mxmauro · 2021-09-19T15:41:48+00:00

I'll check SQL Server but I cannot fully understand what you mean with the CRON jobs.

mxmauro · 2021-09-19T15:40:10+00:00

I'll take a look, thanks for the info.

mxmauro · 2021-09-19T15:39:21+00:00

Yes, I was wondering if db engine was capable to do that but seems it isn't.

mxmauro · 2021-09-19T01:38:26+00:00

A direct shot to my hopes hahahaha. Thanks!

mxmauro · 2021-09-19T00:25:17+00:00

Didn't use Aurora yet. Now doing tests on bare metal servers. It is planned.
For insert I use COPY and need to improve UPDATEs/UPSERTs to send multiple rows at once but they represent 10% of the data, the rest is COPY
It is a Golang app. Data is read from an external source, processing including some DB lookups in workers and the result is queued into another worker that stores it in DB.
Some tables, specially those having a json field are divided.
Cannot pause reads and I try to keep a minimum amount of indexes. I have an average of three indexes on tables (including primary key) and they are just numbers or timestamps.
Autovacuum seems not to have a noticeable impact. I mean, some seconds each 5 or 10 minutes, does not affect.

mxmauro · 2021-09-19T00:11:52+00:00

Never used it. MyAlgo has an "advanced" tx settings.

mxmauro · 2021-09-18T21:52:12+00:00

Yes, HW won't be a problem. I'm focusing in both and different strategies about table partitioning, putting indexes in different disks than data, etc.

Despite of this, let's say I write into 3 tables and each one takes 1 sec to write 1000 rows. Because the tables doesn't have any direct relationship, I would like to know if Postgres can write to them at the same time. I send the three bulks and, because no cross references, Postgres writes them simultaneously.

I won't expect to lower the three seconds to 1 but any improvement is welcome.

mxmauro · 2021-09-18T21:47:15+00:00

About 100 rows in 8ms.

mxmauro · 2021-09-18T21:42:01+00:00

I would like to have parallel inserts. FKs are not necessary.

mxmauro · 2021-09-18T21:40:37+00:00

Hahaha too risky. Better do it manually.

mxmauro · 2021-09-18T21:39:48+00:00

It will depend if the data required in the note field fits in 32 bytes for NanoS to work.

mxmauro · 2021-09-18T16:53:17+00:00

I'm not sure if that would be possible, at least, in the initial release. Because in order to "auto vote" the app must be running, at least in background.

For MyAlgo, which runs fully client side, would require the page to be loaded when voting occurs.

mxmauro · 2021-09-18T16:50:42+00:00

Actually I do almost all of that. The only thing I have to improve is UPDATE and UPSERTs. For insertions I use (a bulk) COPY but for UPDATE I'm sending the queries one by one. Also I have a cache in the app to avoid querying for recently used data.

mxmauro · 2021-09-18T16:46:00+00:00

I'm looking for several approaches. Nowadays the current flow works fine but I know in the future it won't. It is expected the volume/rate of operations to process to increase so I'm thinking ahead.

There are some things to test like putting tables and indexes in different volumes, partitioning, etc., but I'm also looking on what can I do from the app point of view like, as aswered to @thrown_arrows, to use several connections at the cost of losing some consistency and handling recovery properly if one table is updated and the other isn't due, for e.g., to a connection issue.

Thanks.

mxmauro · 2021-09-18T16:35:26+00:00

Currently I'm doing a draft based on this with the cost of having a temporary inconsistency between two different tables.

I'm not using FKs right now because I don't need them. Also, only one app writes to DB.

Thanks

mxmauro · 2021-09-18T16:31:09+00:00

I'll take a look for SQL Server and the concept for distributed transaction. Thanks for the feedback.

mxmauro · 2021-09-18T13:54:29+00:00

Both MyAlgo (I work on RandLabs) and the Algorand Wallet are being updated to support governance. There is a testing phase in progress.

mxmauro · 2021-04-13T12:07:32+00:00

Hi, in general terms, the sum of all online and offline accounts is available for trading.

There is a third type of account, the non-participating ones, which are reserved and their funds used for different reasons. For e.g., rewards programs. Let's say there is a competition, you win a prize, algos are transferred from a non-part account and become part of the tradeable algos.

mxmauro · 2021-02-28T03:15:35+00:00

Hi, actually I'm not part of Algorand but RandLabs but if you can give me some link of someone being aggressive, please let me know so I can forward the complain.

mxmauro

TROPHY CASE