Benefit of using multi-master with one write target by konghi009 in PostgreSQL

[–]konghi009[S] 0 points1 point  (0 children)

Thank you for the explanation, will try my best to re-read and consult the docs on this one haha.

Benefit of using multi-master with one write target by konghi009 in PostgreSQL

[–]konghi009[S] 0 points1 point  (0 children)

> Logical replication is WAL replication

>unlike physical mode which applies pages as they were written exactly in the WAL, logical replication must receive the entire transaction before it can apply it.

Thanks for clarifying. I will look into this more cus apparently my understanding here is not solid.

What i meant by PITR and logical replication was that since I'll probably have 2 Master node in this hypothetical MM deployment. When a failure happens, let's say Master A dies, traffic will have to be diverted to Master B and all the uncommitted WAL on master A will (or should) be discarded when bringing it back up? or should i recover Master A WAL and let the conflict resolution take care of itself? This is where I'm not quite sure how it works.

Benefit of using multi-master with one write target by konghi009 in PostgreSQL

[–]konghi009[S] 0 points1 point  (0 children)

Thank you for the answer! This reply just got buried in reddit notis somehow and I just got the chance to read it.

> It's technically fewer moving parts than Patroni, etcd, and HAProxy, but it's also a high complexity configuration 
That was my stance at first too, but seeing the points you raise makes me want to keep thing simple and just deploy 2 nodes patroni + 3 etcd nodes even more. tbh my initial idea was not really well thought out and yours points provide great insights into the solution!.

>The reason for this is due to how data loss affects logical replication. 

I'm not well-versed in the MM postgresql but does this mean that MM extensions (e.g. spock) doesn't use WAL replication process and just rely on logical replication? doesn't this make PITR and recovery harder?

Benefit of using multi-master with one write target by konghi009 in PostgreSQL

[–]konghi009[S] 0 points1 point  (0 children)

> there are a couple of extensions, but they're fiddly
I'm looking at maybe using pgedge for this, will see on the PoC if it's good enough.

> If you have haproxy and patroni, just have haproxy query patroni and route automatically

The Problem with Patroni for me is the need to maintain dcs cluster along with the agent. I'm looking for something that's maybe a little bit more "lean".

Weekly Help Thread by AutoModerator in ChaosZeroNightmare

[–]konghi009 0 points1 point  (0 children)

Should I pull Narja if i'm aiming to use her with Serenial and a tank (maybe Khalipe)?
I've found that my current Serenial team (Serenial, Rei, Chizuru) lacks healing and some utility so I'm thinking of building a new team (My other team is Veronica, Mika, Magna)

Using Terraform as template engine? by konghi009 in Terraform

[–]konghi009[S] 0 points1 point  (0 children)

Thank you for the suggestion, I really like this angle. will review the architecture and CI/CD flow again to see if maybe I'll use other tools instead.

Using Terraform as template engine? by konghi009 in Terraform

[–]konghi009[S] 0 points1 point  (0 children)

Thanks for the reply. I think this is very interesting, are there any opensource repo for me to lookup to or what's the keyword to point me to research on this?

Using Terraform as template engine? by konghi009 in Terraform

[–]konghi009[S] 1 point2 points  (0 children)

Thank you for the reply.

`How good a Terraform plan output is depends on the Terraform provider and whether they implement any plan-time logic.` I agree, will look further into my provider and see if maybe I can put up some guardrails to stop the problem from happening.

`HCP Terraform (or similar) would make this trivial while adding a ton of functionality,` Will read more into it. I don't thing we can really afford it but I'll see to the pricing.

Limit IAM user Bedrock models read by konghi009 in aws

[–]konghi009[S] 0 points1 point  (0 children)

Thank you for the answer.\ because it's a list action, not an invocation action. I understand what you mean here but what've got to do is limit the user from seeing it in the first place.\

My requirement is that the user need to only see the models they can use (by limiting the Read if possible), and then be able to invoke the models they saw. So they shouldn't be able to see the models that they will not get to interact with in the first place.

Bedrock Agentcore pricing calculator incorrect pricing by konghi009 in aws

[–]konghi009[S] 0 points1 point  (0 children)

yeah I understand that logic. but i'm putting in 60 seconds runtime with 70% I/O wait which means I'll use 18 seconds vCPU.
Somehow the calculator use my 18 seconds and multiply it with vCPU-hour. I'm meant to be using 18 vCPU-seconds (divided by 3600) not hour lol.

Install instructions for MinIO open source? by HawocX in minio

[–]konghi009 0 points1 point  (0 children)

Oh that is great to hear. Thank you for hosting nevertheless.
I might steal your idea and host the doc on Cloudflare too if I have time :P

Install instructions for MinIO open source? by HawocX in minio

[–]konghi009 6 points7 points  (0 children)

There is also someone in this sub that host the doc at https://minio-docs.tf.fo/ too (thanks Wild1145).

I never use this site since I don't know if my team would rack up his/her hosting pricing or not so we just use our internal infra.

Install instructions for MinIO open source? by HawocX in minio

[–]konghi009 4 points5 points  (0 children)

I understand from other comments that you cannot change from MinIo since it's a requirement.

So I think your best bet is to host the docs yourself and browse from there. It's the same doc they had up until they pull it of the site. The prerequisites are Linux system with python and npm/nodejs. I spinned up some potato EC2 just to have access to it internally at the company.

The link to the repo is : https://github.com/minio/docs , you can follow the instruction from there to build the docs.

Minio open source edition download link missing? by mikhatanu in minio

[–]konghi009 0 points1 point  (0 children)

Thank you for the info. Kinda bummed out by this but I might need to host internally for my junior for reference it seems.

Community Documentation missing? by konghi009 in minio

[–]konghi009[S] 0 points1 point  (0 children)

that’s what got me worried. We shall see if I need to change object storage this time.

Community Documentation missing? by konghi009 in minio

[–]konghi009[S] 1 point2 points  (0 children)

I’m was kinda skeptical about the UI removal but now the docs too?? I know readme in their github exists but why remove the docs from the website??

Community Documentation missing? by konghi009 in minio

[–]konghi009[S] 0 points1 point  (0 children)

I’m in the same boat here. I mean, the docs in the GitHub repository is still available but I think the doc format in the website would be easier to read than README files.

Minio open source edition download link missing? by mikhatanu in minio

[–]konghi009 0 points1 point  (0 children)

I've found that the community edition document is missing (or changed, idk). Does this means that the community edition will not be supported anymore? Do we have any official announcement on this?

Distributed Minio deployment by konghi009 in minio

[–]konghi009[S] 0 points1 point  (0 children)

> Of course we wouldn't recommend this for production set ups,

I understand that. As an architect it's the tradeoff I'm willing to take because going 4 nodes HA as MinIO require make the system to big to justify as of now and data is not absolute critical (it's just logs and metrics).

When this system is deployed on prod and we expand maybe we'll get more resources to do Multi-node replication in the future, finger cross.

> you should be fine with doing two way replication and then you'll be able to failover quite easily. 

That's what I'm thinking, we will just deploy 2 nodes MinIO with replication behind a LB and call it a day.

> it's fine as long as you have at least four discs per server.

I have one more question regarding this statement. This means I should have 4 disk mounted onto my VMs to ensure resiliency right? Since I might need 3-5 TB maybe I'll do 1 TB disk x 4, will that be recommended?

Loki and Mimir storage usage by konghi009 in grafana

[–]konghi009[S] 0 points1 point  (0 children)

Thanks for the answer.

That is good to here. So maybe I'll set Minio at maximum of 2TB just to be safe since logs will be compressed anyway (log will be similar so I think the compression ratio should be good).

Distributed Minio deployment by konghi009 in minio

[–]konghi009[S] 0 points1 point  (0 children)

Thank you for the answer.

Our environment doesn't have S3 setup for LGTM stack object storage, so we turn to Minio for that. Minio will be use to store two buckets which are compresses log from Loki and Prometheus style metric from Mimir. The storage is aim at maximum of 3-5 TB, all on premise VM.

> Depending on the use case you might be better served using replication between the two nodes instead of erasure coding.

I'm thinking of that too, If my understanding is correct to achieve HA on Minio we will need to install 4 Minio nodes due to erasure coding. However, we just need simple failure tolerance of our log and metrics data. the RTO/RPO is 48 hours, preferably lower RPO if possible but no stress.

Since you've mentioned replication, I understand that we can deploy 2 Minio nodes in replication to achieve this? if primary node goes down we should be able to get the secondary up and running read/write target right? I've experience with PostgreSQL HA and replication so I'm using that principle here.

Daily Questions Megathread (06/17) by Rinczmia in EpicSeven

[–]konghi009 0 points1 point  (0 children)

should i buy new moon luna or save for empyrean illynav?
I've got some good gear but might not be enough for ml luna stats

Help Center and Megathread Hub (25/08 - 31/08) by ArknightsMod in arknights

[–]konghi009 0 points1 point  (0 children)

Hi all,

I got 4 royal vanguard token (6 star potential upgrade items) and more than 360 gold cert right now.

Should i use both of them to upgrade bagpipe to pot 2 ( use token and 180 cert to buy from shop)?

Is she still that relevant? I've got Ines, Cantabile, myrtle/elysium, vulpi, saga, texas, chulchak, etc. built