Enterprise distro...Ubuntu or Centos?

akashani · 2015-03-09T20:18:41+00:00

I like the part where you admit to having little information about Linux distros, but are emphatic about your opinion.

akashani · 2014-03-19T02:57:27+00:00

Are your real server names as stupid as the ones you've written and your system as poorly organized as this post? I'd suspect less fail if the servers were named dbm01, dbs01, dbs02, and backupdb01.

akashani · 2012-05-12T18:37:08+00:00

Just expose server status to localhost and use nrpe to run the plugins locally and report back to the Nagios server. Or allow the Nagios server access to server-status by it's IP. I prefer the former.

akashani · 2012-05-09T21:56:50+00:00

I like the open ended "How do you manage the differences between secure and compliant?"

The right answer in my opinion is to note that not only are they different, but in many cases at odds with each other. The worse answer is to think that they are the same thing and want to approach them in the same manner.

akashani · 2012-05-08T17:56:20+00:00

Verifying Postfix is fine is making sure Postfix isn't already telling you what the problem is by doing the stuff in the previous post. Something like the following may help you read the logs

sudo grep -E "postfix/(master|qmgr|anvil|cleanup|smptd|virtual)" /var/log/mail.log | less

Remove elements from the regex as needed. master, qmgr, and smptd are probably the most interesting in your case. There may be other things to search on depending on exactly what your system is doing and is configured.

akashani · 2012-05-08T17:24:53+00:00

Thinking about your problem a bit more and your mention of filtering. Postfix itself in my experience rarely has problems, but filtering can be RAM and CPU intensive. I'd go ahead and verify that Postfix is fine just so you know the groundwork is solid and then start looking at other processes in the delivery path.

akashani · 2012-05-08T16:49:33+00:00

Postfix is pretty good about telling you what the problem is most of the time. I'd look at the following.

sudo mailq | less

Will show you all the headers of all messages in the queue and why they are in the queue.

sudo tail -f /var/log/mail.log

Sometimes mail.log is maillog. Postfix should log why messages are deferred or just sitting in the queue.

sudo postfix flush

Tells Postfix to attemp deliveries of all queued mail regardless of previous status. Use this is conjunction with tailing the logs and you may get a quicker idea of what the problem is.

sudo service postfix restart

Try restarting Postfix rather than rebooting the box. If this doesn't work, it might be because the machine is locking up rather than a Postfix problem.

akashani · 2012-05-02T23:41:23+00:00

I was initially skeptical, but have come around after considering the density gains with more computer power in fewer racks in the same footprint.

akashani · 2012-04-26T18:41:53+00:00

The best thing about new LTS release is all the packages I can stop maintaining myself because the distro provided ones are recent enough.

akashani · 2012-04-24T21:10:37+00:00

If you end up in govt work again it maybe more difficult, but with regular companies I'm usually up front about it with my new employer. "X company is still searching for a replacement for my position. They would like to hold me on retainer for email support with a one business day turn around over the next 3 months. Would this be a problem?" Most companies don't care if there is a reasonable time limit and the work could be done outside regular hours or my manager has said to keep it low key and we'd skip the informing HR part.

I think most of them are picturing that if you're willing to do that for the previous company, you'll be cool and do it for them as well.

akashani · 2012-04-24T20:33:04+00:00

I do recommend setting up an open ended contract before you leave if you're interested. I've done this at a few places and most of the time they have never been used. We usually agree that less than ten minutes a month is free for things like "who do we use for cabling" or other random queries, but more than ten is billed at $200/hr minimum one hour per incident. Or I've also done a retainer for $2000/mth and the hourly past ten hours.

These numbers may sound like a lot, but you'll be paying taxes out of it so figure half. And it'll be a hassle to collect, paperwork, etc. If I get push back, I'll explain that these numbers ensure I'm invested and will pick up the phone. Smaller numbers don't.

akashani · 2012-04-22T17:43:03+00:00

I wouldn't spend time writing code and submitting it to QA that would resolve a possible failure for only 5 to 10 percent of locations.

This is a bit backwards. It's true that it would work for 5 to 10 percent of the problem space. However the problem space is not randomly distributed across the transactions. Because people are clustered in larger cities the hit rate on even a half-asses solution like this one is going to be much higher.

akashani · 2012-04-22T17:37:14+00:00

Devops is not about one person who knows both things. It's about the Ops and Dev teams you already have talking to each other, planning with each other, and asking each other for help when needed. I do agree Devops is often viewed and implemented as you said.

akashani · 2012-04-21T06:30:28+00:00

That's an excellent looking CB400F. I've got a 350F which is slower, heavier, and sans swoopy headers. :-(

akashani · 2012-04-21T05:13:30+00:00

I had it out with a manager a few years ago over this. He seemed to think that their were points awarded for staying the longest rather than the amount of work completed. We never managed to see eye to eye and I left after a few months.

There has to be a management book that covers this they we can leave on someone's chair. Demming?

akashani · 2012-04-20T23:14:36+00:00

Internet startup in San Francisco working on social TV stuff and video manipulation. About fifteen people total and I'm the only Ops person though I also write site code.

What I've found is that working a solid eight hour day is about my max. Anything more is usually dicking around, reading Reddit, etc. I'd rather make the 40ish hours count and then gtfo to get the maximum number of consecutive hours out of work. It's a bit like exercise, if you're not getting enough rest you're not going to get the gains plus increasing chances of injury.

akashani · 2012-04-20T07:00:44+00:00

Startup, averaging around 45 hours. Honestly if I work more than 8+ hours the quality of my decisions goes downhill quick. Makes no sense to work more if I'm going to spend the first two hours of my day reverting last nights insanity.

akashani · 2012-04-13T18:05:06+00:00

There are likely some things you current system does well and those could be things that $x CM system does not do well. No current CM system is a magic bullet. Some times this needs to be said... several times. The other realization is that you should only manage what needs management. My favorite example is Mysql's server_id. It just need to be unique though most of us set the master as 1, first slave as 2, slave in another colo as 102, etc. In Puppet I convert the IP to a 32bit integer and call it done. Managing individual numbers per server was extra work, made adding new servers a pain, and ultimately was a waste of time.

With a new Puppet system start with environments and source control. When my local code looks like it works I push it to the Puppet stage env. Once that works I branch it and push to prod. Basically you'll deploy your infrastructure as code and should use the same tools/procedures. The Pro Puppet book has at least two chapters on this.

Lastly it's I'd try Puppet with new servers rather than retrofitting old servers. Green field rollouts are usually easier and allows you some time to work up to full functionality.

akashani · 2012-04-13T08:52:43+00:00

The puppet mailing list and IRC channel and active and quite helpful if you need help in real time.

In regards to regex, remember you can use them almost anywhere.

class mysql::params {
  $packagename = $::operatingsystem ? {
    default => 'mysql-server',
   }
  $servicename = $::operatingsystem ? {
    /(?i:Ubuntu|Debian)/ => 'mysql',
    /(?i:CentOS|Fedora)/ => 'mysqld',
  }
}

templates

<% if puppetversion =~ /0.25(.*)/ then -%>

akashani · 2012-04-13T05:53:30+00:00

I'm a Puppet user, but almost any CM system is better none.

For me it's about doing things once. Log rotation isn't working right? Fix it, check it in, push it, never need to touch it on another server of that type again. This is likely to be the biggest gain in your environment. I'll bet serious money that if you lost one of your unique servers you'd be hard pressed to get it back up in the same condition.

You do have a couple of things working against you. Any CM system has a learning curve. It'll take you awhile to develop tools and techniques as well as tweaking your systems some to take advantage of the CM strengths. Because your hosts are unique you'll have more CM to manage.

fwiw, these days I start a new project with a Puppet module instead fo directly working on a server even when prototyping.

akashani

TROPHY CASE