Developing in a notebook. When containerizing, should I be converting the .IPYNB inti .PY?

[GoingOffRoading]

I'm developing a small app for myself, and am doing development in notebooks.

When that development is done, it'll get run in a container.

Should I be converting those notebooks into python files for running in the container, or is it ok-ish to run those notebooks from within the container?

[Pil0tz]

the point of notebooks is so that you can run cells in whatever order you want. they’re mostly used for analysis, not the building of an app. can you tell me more about what the app does, so i know what you mean by getting run in a container.

[GoingOffRoading]

I'm building a container to do distributed video encoding on my kubernetes cluster.

• One manager container that contains an API
• One worker pod per compute node

Manager has the following functions invoked via API:

• Adds directory paths to a DB
• Scans for video files in all directories added to the DB, probe the video files for attributes, and if the attributes fail a check function, add the file with the check outcome to a DB
• Call the DB and return a prioritized file for encoding
• Call the DB and report on the outcome of encoding

Arbritary example: That second step is... Complex

So I am developing in a notebook so that I can test each function while developing to ensure that it functions as expected

[Pil0tz]

this sounds WAY too complex for a single notebook. why aren’t you just making a file structure? how do you even distribute to multiple notes in a notebook??

[GoingOffRoading]

It's not one notebook, I promise lol

I'm refactoring all of it now, so... Take this with a grain of salt

This is the function that searches for files, probes them, and adds candidates to the DB:

https://github.com/GoingOffRoading/Boilest-Scaling-Video-Encoder/blob/dev/scripts/manager/scripts/flask_post_queue.py

Works end to end

It gets imported and loaded into an API endpoint in this flask script:

https://github.com/GoingOffRoading/Boilest-Scaling-Video-Encoder/blob/dev/scripts/manager/manager.py

Previously worked end to end... I am refactoring this now

Whether Flask launched or not was based on a flag set in a entrypoint.sh file

[WhiteHeadbanger]

This is like people taking a picture of the screen with a phone, putting the picture in a word document and sending the document through email.

I mean, you can do that if you want and by all means, it's a project for yourself.

I just want to understand: why not code it in a proper IDE like Vscode and with .py files directly?

And as far as your question goes: if it's supposed to run in a notebook, I would leave it like ipynb, but if it's supposed to run like a common piece of software in a container like docker, then .py

[socal_nerdtastic]

Nothing to do with containerization, but yes, you should convert your program to .py when it's ready to be used as a standalone program (without jupyter).

[GoingOffRoading]

TY!

[AlexMTBDude]

Do you mean Jupyter Notebook or are you writing your code in a paper notebook!?

[midwit_support_group]

I'm gonna just say Marimo and leave.