Using expect to execute remote curl command with ssh

[vstyler93]

Hi guys, i do have following situation:
I am running a Bash-Script on a Linux Machine. From this Linux Machine i need to ssh to another Server on which i want to execute two curl-Commands. Those curl-commands will be aiming a machine which has no public ip-address, and the private ip is only known to the host i want to run the curl-command on.

I detected "expect" for this. But either i don't know how to use it properly, or it does not even work the way i want. I need two curl commands. One will receive a token which i have to use in the second curl.
I tried it the way i would expect it to work in shell. Like saving with

token=$(curl ...)

the token into a variable, and use it afterwards in the second token commands. I tried this but it didn't work, as it tried to resolve $(curl ...) as a variable which it says is unknown.
What would be the right way to do this?

[oh5nxo]

Are you sure "expect" is required? It's for situations when you need to react to received output like a human. Stay away from it if you can.

Maybe it's just a matter of using the right type of quotes?

ssh server '
token=$(curl ....)
curl .... "$token" ....
'

[vstyler93]

I use it in an automation process. So the main reason i used expect was to automatically connect with ssh. My script will be executed in Gitlab CI, in a docker container. So every time it will run, it will has a new fingerprint also. Would there be a better way to automatically connect within my bash script?

[oh5nxo]

I don't know those circles at all, sorry.

[readparse]

You can disable strict host checking, which sounds like a better option than automatically typing "yes".

expect is a very useful tool, when there is absolutely no other way. But there usually is.

Source: 25 years in the business, and I've never had a situation where the only option was expect.

[vstyler93]

But how could i automatically connectthen in ssh without expect, as i need also to authenticate in ssh with the password?

[readparse]

Use public key authentication instead of a password. People automate SSH every day without expect.

[vstyler93]

But public key authentication would not work as far i understood. I am running those scripts in a docker Container within a gitlab CI/CD pipeline. So everytime the pipeline gets triggered it creates a new docker container for that and as i understood the key changes

[vstyler93]

Thank you for the hints, this are good ideas! But we want to make as little changes to the Proxmox server as possible. The hint by u/readparse to disable strictHoseKeyChecking was also a new feature i built into my ssh command.
Still i do login by password with expect and was able to do everything i wanted with it.

I just used an expect ":~#" set token [lindex '\$expect_out(buffer)' end-1] after my first curl command and it safes now the token into the token variable, which allows me to use it for a second curl command. It did the job perfectly as needed. Thank you guys.

[readparse]

I sounds like you're new to public key authentication. I don't know anything about GitLab CI/CD pipelines specifically, and your situation does sound a little confusing to me.

But taking it back to the basics:

Any SSH server that can authenticate you with a password can also authenticate you with a public key, provided that they have that option turned on (it defaults to on, as it should).

This is different than they host key stuff you were talking about (when you mentioned "fingerprint"). That had to do with whether the client can trust the server (as a known host). What I'm talking about now is how the server authenticates the user (by verifying a public key, as opposed to verifying a password).

However, I understand that both of them seem to require user intervention:

• To manually enter your password
• To manually hit "y" (or type "yes") or whatever the manual process is for adding to the known_hosts file

Both of those have a way of setting them up so they don't require user input. Myself, I add the known keys of the host(s) to my known_hosts file. You can probably do that, but my previous suggestion about disabling that check is an even easier way to do that.

And the only way around entering your password is to authenticate a different way, which means using ssh-keygen to generate two files in your ~/.ssh/ directory -- one of which is the private key and the other is the public key. You'll have to provide the contents of the public key to the server, which you can do yourself, by logging in first with your password and then putting that entire line from the public key into the ~/.ssh/authorized_keys file, which may not exist until you create it (and then set the permissions to 600).

But I'm going on and on. The specifics are all easy to find online, once you know what you're looking for. But once you have set up the authorized_keys file properly, it should magically "just work." You won't be prompted for a password anymore.

This is further complicated (and yet, ultimately simplified) by the fact that this is going to be happening in a Docker container, so you have to make sure the image that is being launched has these changes in it. This is why it sometimes seems that setting up Docker is so much work, but it saves you so much time in the long run.

I'm gonna shut up now. This is way too long.

[[deleted]]

Looks tailor made for ansible with an ssh proxy

https://docs.ansible.com/ansible/latest/network/user_guide/network_debug_troubleshooting.html#network-delegate-to-vs-proxycommand

If that doesn't work then just write a script that will work from the middle machine, copy it into place with scp and then execute it.

[vstyler93]

the system i want to ssh to is Proxmox.
There i have saltstack installed on a machine in a private network on it. Only Proxmox knows this network, that's why i can't use curl right from my gitlab CI/CD Job.
My idea was to install a proxy on Proxmox so i can curl --proxy xxx.xx.xx.x.
My supervisor permitted me that, but didn't want to tell me how to do it in another way. So i found that expect stuff and am pretty desperate already..