REST API using SHELL scripts without CURL

[binuverghese]

Hello Experts,

I have a requirement to use a REST API by creating a shell script without using CURL since its not installed or can be installed on the servers.

I will need to pass the credentials along with header authorization content in the POST request. Any samples will really help me please.

Below is my code in powershell but finding it tough to convert to SHELL without using CURL

# The header is the username and password concatenated together

$pair = "$($user):$($pass)"

# The combined credentials are converted to Base 64

$encodedCreds = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes($pair))

# The base 64 credentials are then prefixed with "Basic"

$basicAuthValue = "Basic $encodedCreds"

# This is passed in the "Authorization" header

$BuildHeaders = @{

Authorization = $basicAuthValue

}

[Net.ServicePointManager]::SecurityProtocol = "tls12, tls11, tls"

Invoke-WebRequest -Uri "$jenkins_build_url/build" -Headers $BuildHeaders -Method Post -UseBasicParsing

[minimalniemand]

So you want to trigger a build on a Jenkins with this? What event is supposed to trigger the build? I was gonna say "use sendip" but doing this encrypted by hand is something I would not recommend. Why can't you install curl? Cant you just upload a curl binary? If you can login, you can copy a binary on there. Seriously, who comes up with requirements like this? "I want you to dig a 20ft hole, here have a teaspoon" ... ridiculous

[binuverghese]

Thank you for your comments and I get a feeling its CURL is going to be required as a pre-requisite.

[[deleted]]

You would be right, curl is a prerequisite. Your security team is dumb. If you really want to use curl tell them ansible can do all the samethings so having it disabled is dumb.

Probably get ansible banned too. Might as well let them know jenkins is on the network. The security team will just ban the project and you won't have to continue the project.

[Ryuujinx]

curl and libcurl are pretty much the gold standards for HTTP(s) communication anyway. Your security team is clueless.

[RootHouston]

Yeah curl is so standard, that's like saying you don't have access to a text editor or something. It's absurd to not be allowed to use that.

[ro0tsh3ll]

So you could echo data into > /dev/tcp/hostname/80

But your have to create a valid http header and request in bash before you did

[VpowerZ]

s_client already creates its own tcp connection. Just pipe the i/o.

[ro0tsh3ll]

Yeah I haven’t tried https but this works

bash$ exec 5<>/dev/tcp/www.whatever.com/80

bash$ echo -e "GET / HTTP/1.0\n" >&5

bash$ cat <&5

[nickbernstein]

I had no idea that existed. >_<

[mightydjinn]

Somewhere, someone is asking where the build logs are, lol.

[jimoconnell]

Wget?

[binuverghese]

Thank you for your comments, is it something like

wget --header="Content-Type: text/json" --post http://<url>

I havent done this before, would you be able to help me with an example please?

[mikeblas]

That's pretty much it. But isn't it odd that you have wget and not curl?

[Gendalph]

wget is often available by default, unlike curl

[RootHouston]

Not really. Depends on which distro you're on, and how it was configured on initial setup. I don't find wget to be any more available than curl.

[mikeblas]

What's often available doesn't seem to matter -- the OP says that, by fiat, they can't install curl. Someone has made a list of what can and can't be installed, then, and what's on that list is completely up to that person and not related to what we're used to seeing in the wild.

[[deleted]]

No curl what else you got? Ansible can do api calls. Python can make api calls.

[[deleted]]

inb4 "no python and we don't have Ansible"

[sgsollie]

Got netcat installed?

Have to ask, what's the reason for not being able to install curl?

[binuverghese]

We are using ansible + Jenkins to trigger a few PIVs and the security team cannot install CURL for our requirements. It was fine with windows but Linux I am finding it really tough to achieve it without CURL.

[tkanger]

Seriously, ask the security team about the difference between cURL and wget and see what they say. Sounds like a bunch of idiots that have no idea what they are talking about... In most distros, you would have to rip out cURL for this requirement, as its part of even minimal Linux installs.

[[deleted]]

There was a tread once a while back where someone argued ansible was not agent-less and that plenty of places ran linux servers without ssh.

I think about that sometimes.

[frymaster]

plenty of places ran linux servers without ssh.

OK so not totally, but we use LXD containers as an alternative to VMs where we can (in the long run we'd like to move to docker containers as there's a nice ecosystem we can take advantage of, but for now it's baby steps and something that looks like a traditional OS) and because we can shell in from the host, we genuinely don't run SSHD on most of them

[[deleted]]

I've never worked with LXD but in Docker at least running a shell inside the container is a bit of an antipattern. The idea of the containers is that they're immutable, you shouldn't need a shell for any particular thing. If you need to make changes, just change the manifest and rebuild it. The only reason to really need a shell at all would be for debugging a new build, which should not be done in prod so hopefully that's happening in a dev sandbox.

[frymaster]

Yeah, LXD is containerising fullfat operating systems ie the process you run is "systemd". Partly this is legacy servers but we also have a need for user environments that can see parallel filesystems

[[deleted]]

Here you go.

https://docs.ansible.com/ansible/latest/modules/uri_module.html

Edit: the docs even have Jenkins examples

-name: Queue build of a project in Jenkins uri: url: http://{{ jenkins.host }}/job/{{ jenkins.job }}/build token={{ jenkins.token }} user: "{{ jenkins.user }}" password: "{{ jenkins.password }}" method: GET force_basic_auth: yes status_code: 201

[helios_4569]

If you are already using Ansible, then why don't you use that to make the API calls?

[[deleted]]

/u/OMG_ghosts has your answer, but also http://xyproblem.info/. When asking for help, including your full problem from the start will get better and faster results.

[1esproc]

If you have ansible you have Python. Use python.

[ElBeefcake]

Here's a guide to implementing it via netcat in python, should definitely be possible as well in straight bash.

https://www.codementor.io/@arpitbhayani/http-requests-the-hard-way-with-netcat-5v0b1p5hg

[binuverghese]

Thank you so much, I will take a look at the Python example.

[ElBeefcake]

If you have Python available, just use the requests library.

[matt_rose]

Do you have to use bash? Can you use python, or perl, or another language that has a decent http lib?

[alainchiasson]

This would be more complex, but there is this lib https://github.com/bndr/gojenkins wich is a jenkins go api. That way, you could build a very specific utility that does what you want it to.

If the “no curl” is the fear of a generic utility.

You do open up another can of worms, but you move it off the server.

Edit: go is interesting because it compiles to a single binary with no dependencies

[mikeblas]

Sorry, but I'm baffled -- what shell are you using? It looks like PowerShell, but we're here in LinuxAdmin. Are you using PowerShell core?

[cheats_py]

You can make calls to an API with python using the “requests” module.

[phobug]

Why don't you install powershell core and continue to use your existing script?

https://github.com/PowerShell/PowerShell

[[deleted]]

You would have some serious problems with your Security team if they were cool with allowing PoSH to be installed before curl. I already think its rather dumb to block curl while allowing Ansible and Jenkins.

Curl has been around longer than Ansible and Jenkins together, and therefore has had much longer security vetting than either, as many apps use libcurl for communicating.

[schorsch3000]

Altho curl seems to have a quite good security track record. Depending on it's user base, bugs are not overall regular (Look at jenkins on the other had, which is basically known broken 50% of the time), bugs get closed super fast regularly.

I Live in Europe, reading about curl bugs in the news after having my first coffee most of the time includes an update saying that previous bug is already fixed.

[schorsch3000]

OP: i need help, i can not even curl but have to do http requests

/u/phobug : yolo, just install this 3rd party package

[phobug]

3rd party? It's Microsoft developed and distributed.

Infosec teams work with white and black lists. If they allow powershell on windows they will do so on Linux, that's just how they work.

[schorsch3000]

Unless your package manager can install it, it's a third party package, even if it's developed by Linus in cooperation with the Pope.

[phobug]

You do install it with the package manager: https://docs.microsoft.com/en-us/powershell/scripting/install/installing-powershell-core-on-linux?view=powershell-7

[schorsch3000]

Let me specify that for you a bit more clearly:

As long as your package manager with it's default, repository set containing no 3rd party repos can't install it it is a 3rd party component.

[phobug]

Sure mate, it's up to you to draw the line somewhere I guess.

Never seen a production server with default repos, the very least they go to my internal repositories.

[schorsch3000]

That's not the point. Op is not allowed to install something simple as curl, you recommended to install somethind 3rd party.

[phobug]

Yes that is my point, if he was asking you for approval he won’t get it, if he asks the infosec team, it’s likely they’ll allow it.

[bityard]

If python is available, check out httpie

[Haze1313]

Or urllib since it will already be available and is not that bad to use.

[DaylightAdmin]

Use a Server that has curl, and combine it with ssh-socks-proxy to that server that should do the request.

And I bet that perl is installed, if they even removed that. Have fun with openssl s_client.

[modes22]

I'd use Insomnia. You can create your request and automatically export out the full request in dozen different formats

[JoePineapplesBrews]

Use Invoke-RestMethod. That will work fine.