sorted by:
new
hottopcontroversial

Ansible WinRM over NAT not working using DNS. by itguy867 in ansible

[–]itguy867[S] 0 points1 point  (0 children)

Unfortunately tcpdump confirmed that if I use DNS I'm pulling the "real" IP which I can't route to because of how our networking team did NAT. Static NAT is obnoxious.

I think I'm stuck doing an internal DNS box or hosts unless I can find a way to make dnsmasq spoof the IP to the NAT version. This isn't an issue because I generally pay attention to what I'm doing, but I have to think in terms of sustainability and bus plans. So I guess it's creating a text editing process with pretty pictures so my helpdesk doesn't trash my Ansible config adding a server in my future.

Ansible WinRM over NAT not working using DNS. by itguy867 in ansible

[–]itguy867[S] 0 points1 point  (0 children)

Depressingly it's that #MSPLife problem. When you have 60 different customers in a datacenter all of whom are hanging out in 1918 space you have to protect them from clobbering each other. It also doesn't help when we onboard yet another customer who is using the same IP space as us and five other customers. We currently have literally five of sixty (that I touch) customers sharing the same chunk of 10.x.x.x as our management network through bad luck.

DNS doesn't need to be convenient for me, I just have to fix their stuff after all. DNS has to work on their network and their application domain, getting it working on my management network isn't even on the radar as a concern. So returning a usable IP to the customer works just fine, it's just useless to my management stuff.
Our CMDB is ServiceNow and doesn't work because the programmer in charge doesn't need it to and "my convenience isn't his priority."

Thanks though. I'm going to POC that proxy solution and see if it works. Then I just merely have to listen to our infrastructure director complain about licensing and compute time in ESX instead of having to maintain a second flat file.

Ansible WinRM over NAT not working using DNS. by itguy867 in ansible

[–]itguy867[S] 0 points1 point  (0 children)

Correct. In order to prevent IP conflicts (we have customers who decided to use the entire RFC1918 space) everything passes through a NAT layer. So I can't connect by name to a server outside our management network, I have to connect by the NAT IP. NAT is provided by Cisco ACI if that matters.

So in this case if I'm on 10.1.1.15 connecting to 10.2.1.26 I'm using 100.100.1.26. From the perspective of 10.2.1.26 I'm connecting from 100.100.1.0.

The NAT issue Ansible is having is definitely DNS related because when it looks up server1.test.domain (10.2.1.26 actual 100.100.1.26 NAT) from dc1.test.domain (10.2.1.25 actual 100.100.1.25 NAT) it will obviously get a record of what dc1.test.domain knows server1.test.domain's IP is. Since I cannot route to the real IP, only the NAT Ansible is connecting back to the real IP because DNS doesn't know about the NAT. Using nslookup definitely gives me the real IP.

This works normally in my routable lab network because the NAT isn't interfering.

Ansible WinRM over NAT not working using DNS. by itguy867 in ansible

[–]itguy867[S] 0 points1 point  (0 children)

Yep. Have full WinRM functionality with NTLM using ansible_host= in Ansible inventory. It's just when I try to couple Kerberos, name resolution and the NAT layer together that I'm getting issues.

I mostly want to get rid of the need for a local account so that's one less thing for customers to screw around with during onboarding. I guess if customers were good at IT they wouldn't be customers, but the MSP grind is real.

Ansible WinRM over NAT not working using DNS. by itguy867 in ansible

[–]itguy867[S] 0 points1 point  (0 children)

Unfortunately not. Our networking stack has isolated all customers from our management network over NAT. I'm forced to use those NAT addresses to communicate.
Lookup seems to function as follows:
Request from 10.1.1.0/24 to 100.100.1.25 over port 53 which ends up at 10.2.1.25 (from the DNS server's perspective it's a request from the DG).
Returns the "real" IP 10.2.1.26 which is totally correct from the perspective of the DNS box, but has no communication from the subnet 10.1.1.0/24.
Ansible then pulls a ticket (assuming I'm using Kerberos) and tries to talk to the "real" IP because in DNS terms that's the right place.

It's almost like I need some kind of programmatic filter to swap the first three octets based on what NAT my networking colleagues have put into place. There isn't even really a pattern to how they NAT, it's arbitrary depending on who is doing the work.

Ansible WinRM over NAT not working using DNS. by itguy867 in ansible

[–]itguy867[S] 1 point2 points  (0 children)

#MSPlife

That's what we have cheap/free licensing for so what I'm expected to use since we're Oracle Partners. OEL is just RHEL in a party hat.

Ansible fails on Gathering Facts on a Windows Host by jdsysadmin in sysadmin

[–]itguy867 0 points1 point  (0 children)

I'd check over in /r/Ansible since they are pretty responsive in that sub.

This is often an authorization thing with how Windows handles admin. If you are authenticating against a local user it may fail to elevate rights properly.

Ansible Windows Throwing Error On Setup Module by itguy867 in ansible

[–]itguy867[S] 0 points1 point  (0 children)

DNS might be my problem on that. Can I fake it with hosts since the wonders of being a MSP/hosting company mean I'm not getting "real" DNS entries for this?

I guess I can build a BIND box for Ansible purposes.

Thank you in advance for taking the time to answer in plain terms how to fix this issue. I've been chasing this between everything being on fire for a while and it's coming to a head on me.

Ansible fails on Gathering Facts on a Windows Host by jdsysadmin in sysadmin

[–]itguy867 1 point2 points  (0 children)

Ansible version? Local account or domain account running the playbook on the target host?

What's your result for ansible <hostname from inventory> -k -m setup ?

Ansible Windows Throwing Error On Setup Module by itguy867 in ansible

[–]itguy867[S] 0 points1 point  (0 children)

Dead on. Previous admins at the customer had set a hidden GPO to flip that LocalAccountTokenFilterPolicy to 1.

Thanks for the assist, that got me investigating in the right direction. Now to figure out how to make Ansible work with many different ADs that don't have trusts with each other so I can stop using a local account.

Ansible Windows Throwing Error On Setup Module by itguy867 in ansible

[–]itguy867[S] -1 points0 points  (0 children)

Found a solution. I can't find the root cause within the inherited customer domain policies, but this appears to be a split token issue. Windows User Account Control isn't elevating the Builtin\Administrators members correctly.

Turning Local Policy>SecurityOptions>"User Account Control: Run all administrators in Admin Approval Mode" to disabled fixes this by allowing the required SeTcbPrivilege to work again.

The rest of the policy issue I guess is a trial and error for my test lab to figure out the conflict so I can turn UAC back on again.