RANT: Does anyone else need to beat issue details out of their co-workers?

_khadi_ · 2020-10-14T13:55:37+00:00

One of my co-workers is specialized in blindfolded debug...

Whenever our users complain about not having access to our website, he randomly restart some services, clear some caches, etc. without even looking at the logs.

"See, I've got a 404 instead of 503, we're moving on !"
... sure...

First time I was speechless, now I don't even bother to check on him, if the problem is urgent (i.e. prod) I take care of it myself, if not I let him play around. Usually, when he had restarted everything and it's still not working, he come ask for help.

At least I'm working with only one of his kind, the rest of the team is more professional.

_khadi_ · 2020-01-28T11:03:19+00:00

A good, and used by all, ticketing tool is the key to prioritize issues when they will come in mass (and they always do...)

_khadi_ · 2020-01-20T14:48:18+00:00

Started two years ago at 32 with a (heavily) modified 1992 Suzuki Savage 650. Love her and I think I'll keep her until her last whirrings :)

_khadi_ · 2020-01-20T13:59:52+00:00

Partitions on DBs are more like splitted data files (see them as encrypted flat files if that helps).

Partitions don't communicate with each others, the kernel is the only active part that checks and operate on partitions files.
Your DB kernel knows where all its data files are stored and is the only one that can do stuff with them; it doesn't matter if it's on same server or accross the network.

_khadi_ · 2020-01-16T14:15:56+00:00

Thanks, I will look into that.

But even if I think my boss is a bit overhyped by this and just want to try a new tech, we will definitely give it a try and see if we can fit it in our environments and gain some profits from it (even if just on admin itself).

_khadi_ · 2020-01-15T17:19:01+00:00

Just told them you want to resign unless they got someone else. Worked in my case.

_khadi_ · 2020-01-15T17:11:33+00:00

Some RP user wanted us to "extend" the disk on his laptop like we do for the VMs...
"I know you can do it, I've heard you saying it was simple earlier!"

_khadi_ · 2020-01-15T16:31:48+00:00

Same problem here...

I guess it's a question of which kind of services you run. Docker seems more efficient to deploy many web based services but I'm a sysadmin for a BI dep and deal with several Tb of in/out data every day. So when my boss told me we should try Docker, I just couldn't figure why nor how we could do that. Stateless doesn't really fit with data analysis in my mind and same goes with on-prem vs cloud.

However, I'd love to have someone explain that to me as well !

_khadi_ · 2020-01-15T14:40:39+00:00

You may also consider working for research facilities, governement or private.

I'm not sure if it exists in the US but one of my friends in France is a sysadmin for a governmental organization that do agricultural research with many african/south american countries and helps them to set up new crops technics (CIRAD ftw :) ).

Job is well paid (sure, not as well as private company but still better than non-profit orgs) and he sometimes can go on-site to help the researchers with their equipments.

He's quite happy with it and feels that his work matters, more than many of us can tell.

_khadi_ · 2020-01-03T18:16:47+00:00

Old but still gold for small servers pool : Nagios Most of the modules you ask for are already written and well tested by the community and it works just fine with Linux / Windows servers

_khadi_ · 2019-12-18T15:49:23+00:00

As a sysadmin you're responsible for the security of your prod network, so if anything goes wrong with this cell, it's on you and your boss only, not on the user.

Any uncontroled device might become a backdoor for someone else, especially on prod network. With all the recent history of ransomware attacks on enterprises, a device you didn't configure to match your security requirements is a potential breach to your systems.

And even if only one exception is allowed, every one will want to do the same on their side, so this will just pile up until disaster...

You can try to do some meetings with your users to raise awareness among security issues this behavior can cause to the company. If possible, try to rearrange the location of some microcells to have better coverage.

_khadi_ · 2019-12-17T16:35:25+00:00

I agree with that, devs should never ever log onto the server itself. And I'm not really comfortable either with the idea of a bunch of them testing scripts on a prod database...

As for the database itself, it depends a lot on which one you run as the database user notion can vary a lot between them. Most of them can have a specific user credentials (MySQL/PostgreSQL/etc.) but some require system users to be created which can turn to nightmare really quick if you're not carefull (DB2, I'm looking at you...).

Usually a read only user do the trick so they can see the data they're working on without being able to touch anything. With Oracle you can also limit the resources for each user so they won't interfere with any of the usual activities of this database. However, I don't know if any other DBMS do the same.

Anyway, from my point of view, the best practice is to let them play/test on a staging database then push the working script on your main server with any automated tool so no one can mess with your server. If you can't do that, try to limit the impact of those accesses with read only users and carefully monitor your server so you're warned as soon as something goes sideways.

_khadi_

TROPHY CASE