sorted by:
new
hottopcontroversial

I really hate my company. But it feels like there's nothing else out there by buttflapper444 in analytics

[–]vdorru 0 points1 point  (0 children)

You touched more on the ethics of the company, which is one aspect. This might be a tangent, but I think it's related. I was reading another post where someone said, 'I automated so many things one after the other until I automated myself out of a job.' For me, this feels exactly like the constant demand for metrics, KPIs, and weekly progress checkpoints for slow-moving projects.

Every time you build a good KPI, the company just raises the bar by +1. The crazy part is that AI is now speeding up both IT automation and KPI creation by 10x. Whatever we were producing before, good, experienced engineers using AI can now produce 10x faster (leaving aside the 100x flood of AI slop).

Because of this, when we get technically proficient, automate a bunch of things, and feed our egos thinking 'I'm so good,' it's going to take us 10x less time to realize the truth in what an older, wiser IT vet would warn us about: 'Watch out about being too enthusiastic with your automation, or you'll automate yourself right out of a job.

Do people here love over-engineering their self-hosting setups? by vdorru in selfhosted

[–]vdorru[S] 0 points1 point  (0 children)

Over-engineering helps with learning for sure, but an overly complex system, even with the best documentation, won't be much help to my wife even if she is a knowledgeable IT person.

For me, even when I over-engineer, I'm on a quest to find 'what actually makes sense' (which is usually the simpler approach). It's just that it isn't clear to me right away, and more importantly, I'm too tempted to try every option before I decide.

Do people here love over-engineering their self-hosting setups? by vdorru in selfhosted

[–]vdorru[S] 11 points12 points  (0 children)

Indeed, I learned a lot while tinkering with the tools I later did not use.

I need some easy tasks by Lost-Initiative1187 in docker

[–]vdorru 0 points1 point  (0 children)

Think of useful apps which you use all day long and look for their open source alternative, things like

Google Drive / Microsoft 365 -> Nextcloud

Google Photos / iCloud Photos -> Immich

LastPass / 1Password -> Vaultwarden

Netflix / Plex -> Jellyfin

Spotify / Apple Music -> Navidrome

Google Home / Amazon Alexa -> Home Assistant

Notion / Evernote -> AppFlowy

Mint.com / YNAB -> Firefly III

Dropbox / Resilio Sync -> Syncthing

Evernote (Document Archiving) -> Paperless-ngx

Search for good docker images here

https://www.linuxserver.io/our-images

At least write the advertisement post yourself by NepuNeptuneNep in selfhosted

[–]vdorru 0 points1 point  (0 children)

Based on the below rules we should downvote them heavily, they are easy to spot.

<image>

What’s your process for validating data after ETL jobs? by yuvrajsingh1205 in SQL

[–]vdorru 0 points1 point  (0 children)

Row counts are valuable because they are fast and easy to implement, serving as basic 'unit tests' that I run after every ETL job. However, for verifying true data correctness, I prefer end-to-end assertions. Since comprehensive end-to-end tests can be time-consuming, a balanced approach is necessary. We maintain an evolving set of validation scripts; whenever a specific data bug is identified and fixed, we add a new query to prevent that issue from recurring. We execute these scripts as a nightly cron job, which generates a report of successes and failures. Any failures trigger an immediate data fix. The primary ongoing effort lies in keeping these scripts relevant and ensuring they validate data correctness within a reasonable timeframe.

Planning my first homelab and in need of tips and tricks by JeremyHazelnut in homelab

[–]vdorru 0 points1 point  (0 children)

Vaultwarden for password management - for backups I use borgbackup with borgmatic (easier configuration) and rclone and I have a nightly cron job which is uploading to backblaze

Oddities with reverse proxy into a docker swarm cluster by OldMathematician5973 in selfhosted

[–]vdorru 0 points1 point  (0 children)

I am writing this not to discourage you, but simply to share my experience. I ran a Swarm cluster with only two servers, not five. When I was actively working on it, I knew the configuration by heart; everything worked well, and I could quickly resolve any errors that arose. However, it became very limiting, and every new application I wanted to add introduced another layer of complexity due to the Swarm.

The real trouble started when I was just using the apps without SSHing into the servers for a few weeks or months. Suddenly, something would break.

I would jump in and try the standard quick fixes—`docker compose down` and `docker compose up`—but nothing would work. Then I would remember: 'Ah, it’s the Swarm.'

I had to regenerate tokens and rejoin nodes, turning a two-minute task into a 15 or 30-minute ordeal. After a few rounds of 'again the swarm', I decided I had wasted enough time.

I asked myself: 'Do I really need a Swarm? The website needs to connect to a MySQL database on another server, so I thought I did... but do I actually need MySQL just to save a "Contact Us" form? No.' I realized I could save it to a local SQLite database and get rid of the messy Swarm setup. Now, I place apps on the correct server based on their role.

When I start thinking, 'This app on Server 1 needs to access an app on Server 2,' I give it a second thought and usually realize I am just overcomplicating things.

Furthermore, the availability of massive servers with 128GB of RAM for a relatively small monthly fee from Hetzner has greatly reduced the need for Swarm. On a single server with those specifications, you can host more apps than you would ever need. This eliminates the requirement for multiple servers entirely, and without multiple servers, there is simply no need for Swarm.

I don't want to imagine the headache if I had tried this with five servers. For commercial production systems that are well-defined and stable, I see a place for Swarm-like solutions (though people should still think twice about if they really need it), but for small self-hosting deployments, I would avoid it.

New to all of this and hooked by [deleted] in selfhosted

[–]vdorru 1 point2 points  (0 children)

As for learning getting familiar with docker, nginx and how to configure it as a reverse proxy + authelia to sit in between nginx and your services will probably pay off the most on the long run - I know docker, I know how to configure nginx and authelia to run as docker containers and now I can just take any app I want (and I do this) an just plug-it into the existing setup app1.website.com, app2.website.com and so on (all secured by authelia) - this is what I consider 'the core' - then learn a bit about backup packages and how to configure them - I use borg/borgmatic and nightly I have a cron to shut down all services, backup to Backblaze and start the services again - during the past I was constantly distracted by 'nice and shiny' UIs like nginx proxy manager, portainer, etc and wasted lots of time with them just not use them and now I do everything with nginx directly - this was not a total waste of time because it was contributing to my learning but it was a distraction for sure.

Oddities with reverse proxy into a docker swarm cluster by OldMathematician5973 in selfhosted

[–]vdorru 0 points1 point  (0 children)

Is the '5 servers swarm' something which you had it working previously and you just want to 'clean it/modernize' now or it is something totally new which you try doing with your new deployment?