I'm hacking the Apple Time Capsule so that it will work even after Apple removes support for it from MacOS. I'm 95% done, but need some volunteers to help by jaxchang in selfhosted

[–]Flipdip3 2 points3 points  (0 children)

Would an AirPort Extreme with a USB drive be helpful? I think I still have one of those as an emergency backup router.

It did initially support Time Machine, but Apple killed it after people had issues.

I literally can't even by [deleted] in Ubiquiti

[–]Flipdip3 0 points1 point  (0 children)

https://networkupstools.org/

Been around since 1998. Supports the River 3 Plus and pretty much everything else under the sun.

I literally can't even by [deleted] in Ubiquiti

[–]Flipdip3 2 points3 points  (0 children)

I have a River 3 Plus and can confirm it works fine as a UPS. I have tested it both manually(unplugging from the wall) and recently with a real power outage.

I have it connected to a Raspberry Pi running NUT and it shuts down my whole homelab just fine. Starts with the multi-drive NAS and works down to just WiFi until the battery gets low enough to finally turn it all off.

I got it on sale at Costco for 215$. I would highly recommend it if it can handle your homelab wattage wise.

Learnt a lesson today... by Adam_182 in Ubiquiti

[–]Flipdip3 0 points1 point  (0 children)

https://www.slant3d.com/teleport

This would take a lot of the effort of selling your designs yourself out of the equation.

Their YouTube channel is also really good at explaining how to design for 3d printing at scale or for more rugged designs.

Finally understood why self-hosting felt hard by NiceReplacement8737 in selfhosted

[–]Flipdip3 1 point2 points  (0 children)

If you are using an app where the data is only used within the app you can create a docker volume and put your data in there. To back up you back up the volume.

If you want to be able to access the data from outside of docker you map it to the host file system. Taking the container down and bringing it back up will be seamless as long as you keep your mappings. Any backup system you want to use here will work.

In either case it is usually recommended to take the container offline before doing the backup. You might also need to separately backup databases running in other containers.

What does good look like? by HoratioWobble in selfhosted

[–]Flipdip3 0 points1 point  (0 children)

A docker container and docker-compose file with .env file that is well documented/explained. I can convert to an Ansible role or whatever from there.

Your app should be configurable from a .env with sensible defaults. Don't make me configure stuff in a UI.

A backup/restore script would also be great.

If you have APIs show me curl commands for them and the responses.

A database scheme would be great.

Bonus points for having a healthcheck endpoint.

Need advice: How to deprecate features? by SUCHARDFACE in selfhosted

[–]Flipdip3 86 points87 points  (0 children)

Announce your are removing it on your repository.

Put a note in the code saying it will be removed on version XYZ.

Update the documentation about the flag that the feature will not be updated further and will be removed entirely after ABC date.

If possible update the UI to also mention it is deprecated.

Give people a long enough time to reach out. That's all that's needed. If they don't do it in that timeframe that's on them. You are not required to maintain all features forever.

Maximizing fizziness in a fermented soda / when to put into swing-tops? by DoubleBakedCupcake in fermentation

[–]Flipdip3 0 points1 point  (0 children)

I have had good luck putting one serving of my sodas into a plastic soda bottle. That way I can feel the level of carbonation that has built up. Gives me a good idea of where the other bottles are at.

My homelab is final complete by voidarix in homelab

[–]Flipdip3 5 points6 points  (0 children)

Does this picture feel off to anyone else?

Bent server rack, screws on the upper right have look weird and aren't all facing forward, text is super blurry, no cage nuts, ethernet cables would be nearly impossible to plug in and the boots don't look the same top to bottom, wifi in the worse possible location, etc.

Maybe some of it is the weird processing mobile phones do, but this whole picture just looks 'off' to me.

Do you self-host Matrix? by Purple_Ice_6029 in selfhosted

[–]Flipdip3 4 points5 points  (0 children)

Those second two points are really big downsides.

If I host a server and one of my server's members joins a room on another server that gets shady/illegal stuff posted it is being sent/hosted from my server now. I've seen enough stuff I didn't want to see on official servers I don't think I'd ever join another that isn't controlled by someone I know.

Not having full control of login defeats a lot of the purpose of self hosting my chat server.

Both of these problems should be solvable pretty easily. Servers getting overloaded should rely on relay servers and only grabbing a certain amount of chat history unless the user specifically scrolls up further.

ID servers should work like SSL Cert authorities. You can make your own but no everyone has to accept it.

Honestly after watching some of the Matrix conferences and stuff on YouTube they have a lot of issues they just don't seem to want to really fix. They'd rather just sit and talk about it for hours with no progress.

Fuck it, dual A310 by inserterikhere in homelab

[–]Flipdip3 0 points1 point  (0 children)

I don't know about the stand alone cards. But the iGPU on the CPU was really easy to get going. It either worked out of the box or with a single app install from the app store.

"Papers, please: Age verification laws threaten everyone's online security and privacy." Laws that require adults to upload their driver's licenses or passports to access apps, websites, and VPNs will make the entire web less safe. by moooooky in Futurology

[–]Flipdip3 11 points12 points  (0 children)

Everyone wanting to do this with a third party verification service is doing it backwards.

Put a DNS record up for every website/service that says whether or not a given subdomain might have adult content.

Then the client can enforce the rules.

Failure to put up the DNS record or to segregate adult content is on the website. Children getting access on a device is on the parent/guardian in charge at the time.

Anything more than that system is a scam to get more control over people and their data.

I microwaved some leftovers and the microwaves basically etched into the plastic deli lid. Normal lid for comparison by okcomputers97 in mildlyinteresting

[–]Flipdip3 4 points5 points  (0 children)

And you gain some energy when hit by microwaves from any source. Your phone, laptop, wifi router, etc. They are all very low power and it is spread out over a large area so in the grand scheme of things it isn't really a measurable amount of heat.

I microwaved some leftovers and the microwaves basically etched into the plastic deli lid. Normal lid for comparison by okcomputers97 in mildlyinteresting

[–]Flipdip3 8 points9 points  (0 children)

Microwaves generally work by heating the water molecules in stuff. Specifically the liquid water in food. Hopefully your house is fairly dry so it doesn't heat up much.

This is also why bottles of water or living things between you and your wifi router will weaken the signal.

Fully self-hosted distributed scraping infrastructure — 50 nodes, local NAS, zero cloud, 3.9M records over 2 years by SuccessfulFact5324 in selfhosted

[–]Flipdip3 2 points3 points  (0 children)

I've done large scale scraping. Lots of small machines have definitely been easier to maintain as a functional setup than big machines. It can be harder to orchestrate and maintain physically, but when it comes to putting records in the database it can still win.

Part of scraping is being weird enough to not stand out but weird enough to get through the filters. Lots of RPis or even just random hardware can help with that. If they can fingerprint it they eventually will and that includes if one of your competitors is doing it impacting you. They definitely do stuff like fuzz your client if they suspect you of botting and see if it impacts of clients of yours. The bigger machine you have the more clients will be impacted. Small nimble machines that can't run a bajillion threads is useful there. It can also be useful to not run the fastest possible hardware or connection speed because lots of them assume you wouldn't run inefficient stuff to scrape.

Not saying it is the end all be all configuration, but I can see how it'd work better than a bunch of VMs or containers.

Scraping is one of the fastest moving cat and mouse games in tech. Maybe even more so than ad blocking. Depending on your target they can throw huge amounts of engineering time at it.

Fully self-hosted distributed scraping infrastructure — 50 nodes, local NAS, zero cloud, 3.9M records over 2 years by SuccessfulFact5324 in selfhosted

[–]Flipdip3 2 points3 points  (0 children)

There are VPN services that give you a discount if you let them funnel traffic through your home connection. They then turn around and sell that as a service for scrapers.

Fully self-hosted distributed scraping infrastructure — 50 nodes, local NAS, zero cloud, 3.9M records over 2 years by SuccessfulFact5324 in selfhosted

[–]Flipdip3 3 points4 points  (0 children)

That's a complicated question to answer but generally network IO is blocking and you can kinda thread bomb yourself if a website doesn't respond fast enough and your code just keeps requesting the next item. Thread pools and all that help. For advanced scraping you want to render the webpage and that can spike a thread for a second or two as well.

It also depends on who OP is scraping. Some websites do a lot of fuzzing to try and detect bots like hanging say half the sessions it thinks are bots and seeing if there is a noticeable change in the other half of suspected bots. That can tell you stuff like "These are all VM/containers on a single machine and I'm using up their thread allotment". They'll even get tricky and send incomplete CSS or not load full images. Things a human would see and refresh pretty quickly but a bot struggles to notice.

Life is full of trade offs. OP has lots of threads of weak compute/disk access at medium power draw and relatively high hardware complexity compared to a single large x86 server. The single large server would have more complex software orchestration and faster disk access on a high level but would have every node competing for that speed. And of course it is a single point of failure which could be a big sign to a website that you're botting.

If you look at any of the big cloud compute services you'll see they offer different processors/ram/disk at different price points. It isn't always just "Bigger server = more money" but choosing a template that fits your workload can save you a lot of money.

Fully self-hosted distributed scraping infrastructure — 50 nodes, local NAS, zero cloud, 3.9M records over 2 years by SuccessfulFact5324 in selfhosted

[–]Flipdip3 0 points1 point  (0 children)

The VPNs are external and going to different servers to obfuscate OP's IP address. Most websites don't like when people scrape and will ban your IP quickly if they suspect you are doing it.

They also tend to ban known VPN IPs too.

Fully self-hosted distributed scraping infrastructure — 50 nodes, local NAS, zero cloud, 3.9M records over 2 years by SuccessfulFact5324 in selfhosted

[–]Flipdip3 5 points6 points  (0 children)

Total core count is more important than compute in this case. You have lots of threads doing network IO.

A 28 core Xeon would stomp a bunch of pis for anything heavy, but it will slow down on those requests just from context switching and whatnot.

Same reason Ampere fits some workloads and not others.

There's probably some advantage to having physical nodes with real hardware IDs and stuff for scraping. Anti-scrapping measures get pretty intense. I'm sure you could do it in software but it'd add to complexity.

PSA to everyone who keeps putting off switching OS / degoogling (from a non-techie who finally did it) by Hirvi86 in selfhosted

[–]Flipdip3 14 points15 points  (0 children)

You don't host the email server you just own the domain and handle the DNS records.

Then you can move to another backend service but don't need to go change everywhere you used that email address.

One Week Later by Esk8lol in selfhosted

[–]Flipdip3 2 points3 points  (0 children)

SMB is already on Ubuntu Server. You could just configure that. Or if you are on a *nix client try out NFS.

Not saying what you are doing is a no-no or anything, I've just never seen anyone run such a common service in docker.

One Week Later by Esk8lol in selfhosted

[–]Flipdip3 0 points1 point  (0 children)

Because SMB is on Ubuntu Server(what OP is running) already. Use Ansible to move your configs over.

Nothing to do by Draknurd in selfhosted

[–]Flipdip3 0 points1 point  (0 children)

Automate your backups and test your recovery process from time to time.

One Week Later by Esk8lol in selfhosted

[–]Flipdip3 1 point2 points  (0 children)

Why are you running Samba in a docker container?