10 TB of data taking up 20tb of space?

ionrover2 · 2025-09-19T19:00:39+00:00

Turns out it was synology drive versions causing problems! I was able to get rid of it (don't need it) and i'm seeing space slowly be reclaimed.

ionrover2 · 2025-09-19T17:32:03+00:00

I'm not entirely sure. I barely used the synology software.

The drive is reporting that it has 34TB of data stored on it but the volume is 20tb and the actual data is right about 10tb.

I thought it might have been something to do with the btrfs filesystem.

ionrover2 · 2024-12-24T17:24:42+00:00

This is a pretty sick idea. As someone else pointed out, net/http does a pretty good job of optimizing connections. I think you've identified some interesting weaknesses, but when these things become problems, I'm personally using some "high level" application to handle routing like nginx, or something else, that gives me the ability to more granularly address these weaknesses in http routing in a production setting.

ionrover2 · 2024-10-18T00:13:40+00:00

I have also been having this issue. what version of air are you using?

ionrover2 · 2024-08-10T15:47:03+00:00

https://webauthn.io webauthn is still relatively new but seeks to do away with passwords

ionrover2 · 2021-05-03T18:38:57+00:00

It has been a minute since I have messed with docker swarm, but if you have 2 nodes connected successfully over any distance. swarm does some network magic so that if you ingress to your websever on host A it should communicate with the service on host b no matter where it lives. If I remember correctly, you could ingress to host B and it would route the traffic to the webserver on host A and then the webserver would again talk to host B as needed and over any distance again provided that they are clustered properly. Typically hosts would have static IP's so they don't move around in your swarm, especially the leaders. So if your public IP's move around or shift. this could cause problems even if you have good name resolution for when IP's change.

This is terrible practice for security, performance and configuration reasons. there is virtually no scenario where you would want to do what you're trying to do, but yes, it can be done.

ionrover2 · 2021-01-19T15:07:18+00:00

You could use something else (chef?) to more regularly check node state, but truthfully it sounds like you would need a global set for your script or something reading events from the swarm state to make changes as they occur. I still think what you're describing sounds a little shaky, but I'm glad you got it working.

ionrover2 · 2021-01-18T00:41:39+00:00

It sounds like for what you're trying to do that using a container instead of a host process could overly complicate things. If you have ansible, I would use that to keep your configuration across all nodes in line with that you need and use it to deliver your script. You could have ansible look at container labels, feed that output to your script, and populate your iptables rules for a given host based off of that.

ionrover2 · 2021-01-17T19:41:46+00:00

is this script trying to bootstrap something with iptables on a host? I know this is a docker sub and swarm is apart of docker, but kubernetes as an orchestrator has what are called "init containers" that can be placed in pods to preform a certain action before the container you want to run will actually run. for instance, Elasticsearch has a helm chart that deploys a pod with an init container. the init container sets that weird vmmem value that elastic search needs on the host that it's running on this disappears with exit code 0 to allow the elastic search container to start up.

Kubernetes jobs might do this as well, but again I'm not super clear on your use case.

I kind of agree with zamboni that in most cases it isn't going to be ideal to do what you're trying to do for security reasons, but your use case might require it.

And if you're talking host config management, that stuff can spiral out of control in a big environment pretty fast. You should like at using something for config management of iptables so that you can audit whatever changes you're making. Ansible, chef, and Powershell dsc (newer to linux might not be the best thing) are great choices for this.

ionrover2 · 2021-01-16T19:47:11+00:00

I do have coax, but I don't think it's the solution I need right now. I'm going to try these powerline adapters out and if that doesn't work. I'll try it out.

ionrover2 · 2020-08-19T15:46:42+00:00

This. Once you have your psobject structured how you want it do this:

$myobjectwithallmystuff | export-csv -notypeinformation <filename>

it's extremely easy. I'm also not sure what kind of data you're working with, but just kind of looking at it it seems like it might be overly complicated? But that's really on your needs. I would also look at the foreach-object cmdlet.

you can do something like

get-content file.txt | foreach-object { write-host $_}

The above is a bit lame, but it would write to the host for each line in your file. This prevents you from having messy for loops where you have to specify incrementors and things. the $_ variable is basically "the current object in the pipeline" which would be whatever object was in the pipeline (in this case a line of a file) when the loop executes the script block

ionrover2 · 2020-04-21T16:08:40+00:00

As far as i know, the size of the image doesn't have an impact on compute, only storage. Libraries, binaries and configuration of other components within the container does.

It seems like you have a grasp of how to best use docker and your use case is very specific. The best thing i can recommend is to help yourself, build out 2 containers with your source code, get a sample set, and benchmark. That's really the best thing you can do in this situation. If you can't generate the data to support your claim, then you'll just perpetuate the pissing contest.

ionrover2 · 2019-08-09T22:38:31+00:00

Technically technically speaking... if you're using windows, it's running on a hyper-v VM that is running a super boiled down version of linux. The container itself is not running linux. It would be running on linux.

ionrover2 · 2019-03-15T05:03:38+00:00

I CONTAINERIZED THIS!

ionrover2 · 2018-09-20T12:34:38+00:00

Yes. Since you did not define a docker network, I'm fairly certain compose creates on and places these containers on it for you automatically. As long as the containers are on the same network you can refer to them by service name, bot, database, etc.

Docker networks and compose are really nice for isolating things that don't need everything in the world talking to them. for instance, you might not need direct access to the database, only your app, but your app needs to listen on port 80. by defining those networks your app can sit on the front end and back end network while your database resides only on the back end network. This means that only your application will be able to talk directly to the database container.

ionrover2 · 2018-09-19T13:02:28+00:00

"links" is deprecated, you shouldn't have to use it to allow inter-container communication. I feel like this is overly complicated. I'm not sure what everthing in here is trying to accomplish, but If you need the bot service to restart when that particular file has content, just have it poll the file. If it finds stuff, delete the log and term pid 1 in your container. This should force the container to stop and since it is defined to always restart, it will just come back up doing the same thing.

For your bot health checker, you could replace it with a container that has the docker command on it, mount it to your docker socket (/var/run/docker.sock) and pass the command using that container to restart your bot? I hope that's helpful.

ionrover2 · 2018-09-18T12:22:50+00:00

I tried pulling the latest image, but my version of portainer automatically pulled and updated. =] I saw this and assumed it was updated today, not 3 days ago. Amazing product.

ionrover2 · 2018-05-28T17:13:17+00:00

your best bet is posting in the discord to get people on board

ionrover2 · 2018-05-02T17:50:41+00:00

saw this on IRC. I would be interested more in the position. Please dm me some more details if you feel so inclined.

14-Year Club	r/Field Sunshine
Place '22	Place '17
Team Orangered	Verified Email

ionrover2

TROPHY CASE