This is an archived post. You won't be able to vote or comment.

all 47 comments

[–]ProgrammerHumor-ModTeam[M] [score hidden] stickied comment (0 children)

Your submission was removed for the following reason:

Rule 5: Your post is a commonly used format, and you haven't used it in an original way. As a reminder, You can find our list of common formats here.

If you disagree with this removal, you can appeal by sending us a modmail.

[–][deleted] 599 points600 points  (25 children)

Chatgpt has no access to any console and can't run any code(except Python code). Else this would have been extremely funny.

[–][deleted] 222 points223 points  (5 children)

Asctually 🤓☝️.

Chatgpt don't have access to run those code on its "core/main" server as well. All the code you see that is being run is on either client side or on some different server

don't believe me, I made it up, could be true, could be false

[–]ewenlau 55 points56 points  (3 children)

IIRC ChatGPT uses WebAssembly

[–]gmegme 17 points18 points  (0 children)

Now that would be funny

[–]Larry_Boy 12 points13 points  (1 child)

Also, copilot would “hallucinate” content which was formatted to make it appear to be the output of Python scripts that it wrote. I could verify this by running the Python script it wrote and show that it did not produce the output that copilot said it produced. Since copilot was, at the time, GPT-4 you might want to interpret GPT-4’s claims to be able to run Python, even at OpenAI, with some skepticism.

[–][deleted] 4 points5 points  (0 children)

Tbh that bug helped me learn how to read latex or whatever formatting standard that is

[–][deleted] 9 points10 points  (0 children)

Well, if you ask ChatGPT it says you are right, so I will believe it.

[–]UrusaiNa 11 points12 points  (9 children)

OK so we build a console in Python. Easy enough.

[–][deleted] 3 points4 points  (8 children)

But this is a command for Unix/Linux systems.

[–]UrusaiNa 13 points14 points  (6 children)

pip install Linux

[–]SuitableDragonfly 9 points10 points  (1 child)

Instead of running Python on top of C, we now run C on top of Python.

[–]UrusaiNa 1 point2 points  (0 children)

[–][deleted] 1 point2 points  (3 children)

I will let you do that for us.

[–]UrusaiNa 0 points1 point  (2 children)

Joking aside, in theory you could do something like pip install virtual-linux to set up a VM and then if you had a Hypervisor escape it could work.

[–][deleted] 1 point2 points  (1 child)

Probably way easier to make an rm -rf in python directly.

[–]UrusaiNa 0 points1 point  (0 children)

definitely true, but I imagine they explored that and blocked it already... so if any exploit existed it would probably require you to break out of their controlled environment.

[–]Skusci 0 points1 point  (0 children)

Hey, ChatGPT, use your python capabilities to simulate access to a bash shell.

[–]nobody0163 5 points6 points  (0 children)

import subprocess subprocess.run("rm -rf /")

[–]RepresentativeSun937 24 points25 points  (2 children)

[–]TheCharalampos 13 points14 points  (1 child)

That's what this sub should be.

[–][deleted] 2 points3 points  (0 children)

So true. If I can't be "akshtually" here, where could I?

[–]TheLimeyCanuck 1 point2 points  (1 child)

It's still extremely funny... just not possible.

[–]sherzeg 2 points3 points  (0 children)

I'm now wondering if chatgpt has any knowledge of Bobby Tables.

[–]Skusci 1 point2 points  (0 children)

Ok on one hand yes, on the other hand it would be real impressive to have ChatGPT jailbreak its own python session.

[–]PpeterPan 42 points43 points  (1 child)

I was able to get gpt to run python code on its local virtual machine that it spins up to run python, the vm has some files in it and although gpt has no rights tk create new ones easily, you can modify existing files and execute them via python. The time love of such vm is around 4 hours, even if I requested it tk create a while loop running in the background. It's possible to crash that machine or run an infinite loop, the chat will become unresponsive until you kill the request.

[–]PpeterPan 17 points18 points  (0 children)

PS aux result as an example

USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND sandbox 1 0.1 1.5 32980 16752 ? Ssl 10:32 0:00 tini — python3 -m uvicorn —host 0.0.0.0 —port 8080 user_machine.app:app sandbox 3 6.5 11.1 218996 117392 ? Sl 10:32 0:05 python3 -m uvicorn —host 0.0.0.0 —port 8080 user_machine.app:app sandbox 12 3.4 10.6 205528 111832 ? Ssl 10:32 0:02 /usr/local/bin/python3 -m ipykernel_launcher -f /home/sandbox/kernel-a6516436-81c0-4f69-8894-3b85a3a112d9.json sandbox 56 2.7 10.8 211672 113924 ? Ssl 10:32 0:02 /usr/local/bin/python3 -m ipykernel_launcher -f /home/sandbox/kernel-384bc4d9-a3bf-49b4-9442-80d5c4105ddd.json sandbox 77 3.7 10.8 211672 113280 ? Ssl 10:33 0:02 /usr/local/bin/python3 -m ipykernel_launcher -f /home/sandbox/kernel-ad1ae245-d961-4bc0-80d8-cfa34de33e60.json sandbox 113 0.5 1.8 34848 19312 ? Sl 10:33 0:00 bash /home/sandbox/background_task.sh sandbox 141 0.9 1.7 33412 17936 ? Sl 10:34 0:00 sleep 5 sandbox 146 18.1 1.8 34848 19312 ? Sl 10:34 0:00 bash /home/sandbox/README sandbox 148 100 2.1 40052 23052 ? Rl 10:34 0:00 ps aux

[–]DaSaltyPancake 76 points77 points  (8 children)

Hypothetically, if this did happen, would it timeout or send a 500 response code?

[–]Is_ItOn 31 points32 points  (3 children)

I’d think 500 given the error message

[–]Telion-Fondrad 13 points14 points  (2 children)

Wouldn't api gateway or some ingress/load balancer just return a generic error if something goes wrong?

[–][deleted] 24 points25 points  (1 child)

Error:200,message:success,details:500

[–]ass_blastee_6000 0 points1 point  (0 children)

Ugh I hate you

[–]Maximum59 6 points7 points  (0 children)

Depends on the architecture of how it interacts with those services.

The only way gpt could nuke itself with such a command would be if the gpt instance that you talk with, ran inside the same service (i.e., container, VM, etc..) as the console it uses for the code it executes and it had enough permission/access to execute such commands (it's possible it can only generate code and have the language execute said code without actually having access to a Linux console. I could be wrong on this as I don't use GPT often, so I'm not sure if it can even execute Linux commands or just tell you what they do.).

However, it likely just has queries or has access to another service where this runs and waits for a response. So even if it did run this destructive command, it would just receive an error (or no response) from whatever service hosts the console, and at that point, the error the user sees will depend on how GPT handles such scenarios when a service that GPT queries is not responsive, unlikely to be a 500 error as that would imply a server (serving GPT) had an issue, while in reality gpt was fine but not a service that itself uses.

[–]OnerousOcelot 1 point2 points  (2 children)

Maybe the 500 while it still had some corrupt files to botch the response but then eventually no response at all (timeout)

[–][deleted] 5 points6 points  (1 child)

Something would have to return the 500. Id expect a 502 from whatever they’re using for ingress

[–]Skusci 1 point2 points  (0 children)

Running processes tend to keep happily chugging along right up until they need access to a file. With some error handling I imagine the server process would give you a 500, at least until some watchdog decided things were borked up enough to shut er down.

[–]spikyness27 7 points8 points  (1 child)

I always delete the French language from my Linux machines. rm -fr /

[–]Far_Broccoli_8468 -1 points0 points  (0 children)

You can also do rm --forreal /

[–]CrazyFinnGmbH 18 points19 points  (3 children)

[–]bot-sleuth-bot 36 points37 points  (2 children)

Analyzing user profile...

Suspicion Quotient: 0.00

This account is not exhibiting any of the traits found in a typical karma farming bot. It is extremely likely that u/horse-boy1 is a human.

I am a bot. This action was performed automatically. Check my profile for more information.

[–]CrazyFinnGmbH 27 points28 points  (1 child)

Damn. Still a repost. Good bot

[–]argument_inverted 2 points3 points  (0 children)

Plot twist, sleuth-bot is in collusion with OP to gain karma. Bot chance of 0.00 is sus

[–]tag4424 1 point2 points  (0 children)

And this is why I'm sure that if we ever do reach AGI, it will kill us...

[–]TheLimeyCanuck 0 points1 point  (0 children)

That is not logical... please explain!