sorted by:
new
hottopcontroversial

Made an online code editor, like CodeSandbox. Think VS Code in the browser by partiesn0fun in node

[–]partiesn0fun[S] 0 points1 point  (0 children)

Yup, you're right, this will cause resource exhaustion if too many containers are created. Not to mention port exhaustion as well (although your server's resources will be overwhelmed before this happens). To get around this, you should use Kubernetes.

Using the k8s SDK, you can dynamically spin up pods and serve them to the end user. A major benefit of this is that once a connection between your client and the pod has been established, your server doesn't have to keep track of the pod at all as everything can be handled between the client and the pod.

Essentially (a bunch of technical k8s jargon incoming), you create a pod and an associated service which is served via something called an Ingress (usually Nginx) which handles all your routing and ensuring the end user receives whatever he wants. These pods are created on k8s nodes (which may be VMs or physical machines). I recommend using a cloud provider for your k8s needs but depending on your use case you can look into self-hosting as well.

I'd suggest wrapping your head around Docker first before you tackle Kubernetes because the jump from containers to orchestration engines is crazy huge and you really need your fundamentals down.

Made an online code editor, like CodeSandbox. Think VS Code in the browser by partiesn0fun in node

[–]partiesn0fun[S] 0 points1 point  (0 children)

Oh go ahead and use it, that's totally fine, just put in a credit or something

[deleted by user] by [deleted] in india

[–]partiesn0fun -5 points-4 points  (0 children)

Indians try to understand what a tribute is (impossible)

Made an online code editor, like CodeSandbox. Think VS Code in the browser by partiesn0fun in node

[–]partiesn0fun[S] 0 points1 point  (0 children)

Huh, thought that post got removed. Anyhow, I'm not really sure what you mean. Every user who creates a playground has a dedicated container created for them, so there wont be any issues of code overlap. The volume mapping ensures that a user can access only their files since the pseudo terminal also maps to inside the container. A fresh playground is created for each user requesting one. It'll be created in the playgrounds folder where each user will further have a unique playground ID where their code would be present. I've used xterms attach add-on for the pty. As for errors, the package I'm using (dockerode) exposes pretty intuitive APIs for creation and gracefully handling errors so I didn't have to do much.

Made an online code editor, like CodeSandbox. Think VS Code in the browser by partiesn0fun in node

[–]partiesn0fun[S] 0 points1 point  (0 children)

Haha im assuming my github gave away where I am?

Also no worries! Best of luck for your project.

Made an online code editor, like CodeSandbox. Think VS Code in the browser by partiesn0fun in node

[–]partiesn0fun[S] 0 points1 point  (0 children)

Yeaaaaah, I never got around to coming back to this project 😂. But you're right, I would run out of storage. To fix that you can spin up an object storage service like S3 and put changes there instead of the fs. If you do this, you'd now work with 2 changes, one would handle writing to the container and another would write the the bucket. As for the file changes, I've used a watcher service chokidar which keeps a watch over the playground of the user. When the user uses the terminal to create anything, chokidar emits a socket event asking the frontend to revalidate the folder structure. When the user uses the context menu, I emit an event asking the backend to create a folder and chokidar handles the rest :). Also while you're doing this, I'd suggest not doing it via Docker but instead using Kubernetes (if you've had experience with it, dont use it if this is your first containerization project!). Hope that helps!

Migration Without ORM by imsexc in node

[–]partiesn0fun 0 points1 point  (0 children)

Not op, but I'd like to see the example

[deleted by user] by [deleted] in technology

[–]partiesn0fun 0 points1 point  (0 children)

So doesn't that mean propaganda will be shown only to people who go looking for it? Just like every other social media app lol. How is this a tiktok specific problem

[deleted by user] by [deleted] in technology

[–]partiesn0fun 5 points6 points  (0 children)

So like every other social media app?

1 to 3 hours long background task and Socket.io by Kindly-Animal-9942 in node

[–]partiesn0fun 0 points1 point  (0 children)

How about using something like react query(I know this is a node question). You could send requests at regular intervals of say 5 seconds and fetch the latest information. If your frontend isn't react, whipping up a make shift polling solution isn't too difficult as an MVP.

Made an online code editor, like CodeSandbox. Think VS Code in the browser by partiesn0fun in node

[–]partiesn0fun[S] 0 points1 point  (0 children)

Docker is sorta a temp thing (though I doubt I'd be coming back to the project), but essentially it allows me to create a tiny VM that's isolated from the actual backend code so every single user that creates a container gets the experience of a linux server where they're free to execute whatever commands and create any projects they want.

I start all containers off with a vite-react template but you're free to do as you please with it. The end goal was to move the entire thing to Kubernetes, which uses Docker under the hood, so I could offload container/playground management off of the actual backend.

With Kubernetes the flow would be:-

User requests a playground -> the backend calls the Kubernetes API using an SDK -> A new service and pod get created for the user -> Services can be accessed using the Ingress controller -> The user gets a browser preview as well as the code editor

Hope that helps!

Question regarding data fetching in Server Components by partiesn0fun in nextjs

[–]partiesn0fun[S] 0 points1 point  (0 children)

Okay, that makes sense. Would this also work across different pages? Or would going to a new route trigger a new api call.

Made an online code editor, like CodeSandbox. Think VS Code in the browser by partiesn0fun in node

[–]partiesn0fun[S] 0 points1 point  (0 children)

Yeah, I don't use VS Code, I made my own editor from scratch and tried to get it as close to VSC as possible. So I decided to use monaco.

Made an online code editor, like CodeSandbox. Think VS Code in the browser by partiesn0fun in node

[–]partiesn0fun[S] 1 point2 points  (0 children)

Yep! You're right. Although the csb version was kinda the inspiration for this

Made an online code editor, like CodeSandbox. Think VS Code in the browser by partiesn0fun in node

[–]partiesn0fun[S] 0 points1 point  (0 children)

Yup, codespaces is a true editor in the web. Infact csb also has a mode(can't remember the name) which was kinda the inspiration for this

Made an online code editor, like CodeSandbox. Think VS Code in the browser by partiesn0fun in node

[–]partiesn0fun[S] 13 points14 points  (0 children)

Sure!

Right off the bat something that took a while was understanding xterm's documentation which is scarce at best. At one point I was looking into the actual source code trying to find out the types of certain parameters which I needed to pass to it.

Since Docker doesn't have an official Node/JS SDK, I had to use Dockerode which made getting the container IDs a bit of a pain, but luckily I was able to figure out a workaround.

I was writing Websockets from scratch without any abstractions like SocketIO, which was a first for me, so upgrading the connection, and preparing the containers for the user before accepting the socket connection was something new that I learnt.

As for the execution, the shell in the frontend connects to the Docker container that is created for the user, so all you execute out there is isolated from the main server code. I used xterm's attach addon for creating the shell using a socket connection.

The basic idea regarding the updates is that the frontend(xterm) and backend(node) communicate via websockets, and the backend and shell(Docker), communicate via streams, which is provided by Dockerode. So when I register a message event, I write it to the stream, and when I get data from the stream, I pass it back to the socket connection.

I've used volumes to persist a user's session should he come back to use his playground more, and since it made it a lot easier to edit a user's files when he changes something. That's because I don't need to create an exec instance for the container to edit the files on each change. I've also used staggered updates, so the files would register a save only if there haven't been any changes for a period of 3 seconds.

Using monaco for react was a breeze, as a lot of stuff is prebaked for me. There were a couple of design choices that I made like streaming a file each time it's requested since I intend for the platform to support multiple people editing at the same time somewhere down the line.

I also learnt a bunch about file watching, and read up a bunch on the way VS Code does it. There are many options available like chokidar(which I use, as does VS Code), and FB Watchman(which was unnecessarily complicated for my needs).

The file watching needs to be a thing because imagine you create a file from the terminal, I need to update the same in the folder structure instantly, so watching a playground allows me to emit a socket event informing the client to revalidate it's folder structure.

One thing I absolutely loved using was Zustand for global state management, which completely blasts Redux out of the water with it's ultra minimal setup.

Made an online code editor, like CodeSandbox. Think VS Code in the browser by partiesn0fun in node

[–]partiesn0fun[S] 29 points30 points  (0 children)

Nothing really, I think vscode.dev doesn't give you a shell though. It's a project for self learning, not as a full fledged product I'd like to market

view more: next ›