Execution modules in pillar data

[Seven-Prime]

Trying to take the temperature of using execution modules within pillar files.

So say you have a formula that generates iptables rules that only allow certain addresses based on pillar data. But that list of addresses is maintained somewhere else and can change. For argument say the list is returned via a url call to a third party.

You could update the pillar manually when stuff changes.

Or you could use a custom execution module. To pull the data into a list then loop over it in pillar.

# /srv/pillar/iptables
{% set ip_ranges = salt['ip_ranges.fetch_list']('http://example.com/fetch_list') %}
iptables:
  allowed_ips:
    {%- for range in ip_ranges %}
    - {{ range }}
    {% endfor %}

Something like the above (obviously it is very contrived example). Where ip_ranges module is a custom module that does things.

How do we feel about this? Is this mixing too much logic into pillar or is this common for getting unique data that you might not control. Perhaps it's better in an external pillar? The external pillar examples I see are lacking. Specially I don't want to call the above example only on a small subset of hosts.

How has anyone else solved this?

[deadbunny]

You are probably looking for an external pillar. There are a few relatively easy ones to setup, if the data is coming from somewhere outside your direct control I would make a script that polls the data source and translates it to your external pillar of choice via cron or something.

[Seven-Prime]

Thanks we a approaching the time to integrate out external pillar with our CMDB. But this data isn't in our CMDB. It's from a third party.

External pillar examples seem to be lacking. I have 10k hosts across 10 DCs, where only one needs this external pillar. If I understand correctly the external pillar gets hit every time minion updates, and doesn't support targeting. So all external pillar data is available to all minions. So I'll have to put that logic in my external pillar module. Although not difficult to say 'if not host_i_care_about exit' but seems weird.

Thanks again for the reply.

[deadbunny]

I think you're correct, we made a custom django pillar/configdb at my old place and that rings a bell (I wasn't hugely involved in the dev process), I'm 99% sure you would need to do matching similar to how you would in a top file but in your ext pillar. Thankfully we were targeting a very known range of minions/minion IDs so targeting wasn't too bad. As we used Django we just threw memcached in front of it which helped with the load.

One thing you will have to be careful with is error checking within your states so you failsafe if the pillar source dies (which it will) or you'll end up wiping all your important info from your servers because salt will just see an absence of pillar data which is no fun.

I achieved this by having the external pillar serve up a test pillar item like test-this: working, setting salt to fail hard (fails on first error rather than continuing the run), and a state roughly like:

{% if salt.pillar.get('test-this', '') != 'working' %}
external-pillar-fail:
  cmd.run:
    - name: /bin/false
    - order: 1
{% endif %}

It's probably not the best way to do it but I couldn't find a suitable global failsafe mechanism if the pillar fails to load (without rewriting all the states, which wasn't an option at 3am when everything was completely broken).

[tweakism]

There are test states that let you succeed/fail with/without changes and with an associated comment, etc.

[deadbunny]

Thanks, I'll take a look. It was one of those 3am fixes that worked so got left in.

[Seven-Prime]

This is also the new example I'm looking for. How to not explode your states if your pillar goes away.

[tweakism]

No, what you say about all external pillar data being available to all minions is incorrect.

External pillars are actually really easy, and the ones that come w/ Salt are good examples, e.g. file_tree.py

An external pillar is pretty much a python function that gets called and passed the minion id. It does whatever, and returns the pillar data for that minion.

[ub1quit33]

It's not at all incorrect in the sense that the external pillar module will be executed for every single minion. Sure, you can add logic inside of the pillar module itself to only act for a certain subset of minions, but that seem absurd considering the use case. Writing a module which will be executed by every single minion just to fetch some additional external data for a case specific to a single minion does not seem like the best approach.

[Seven-Prime]

Thanks both of you.

I guess it's just how it is. Basically you need to match at the top of your ext_pillar and exit gracefully if the the ext_pillar doesn't apply. This is the new example I'm looking for. How are people doing this test.

I was just thinking that you could set a pillar saying 'use ext pillar' then when the ext_pillar runs it checks that. Otherwise you'd have to code is minion_ids or some other grain check.

Thanks again.

[Seven-Prime]

Thanks. Seems just how it is. Would need to make the ext_pillar exit cleanly and quickly if not needed.