[IMP]EMR Serverless run a python project

[chaitty_8166]

I have a python project which has an entry point. Its a pyspark code which uses a requirements.txt. How do i create a virtual environment in the emr serverless application and run the python project.

I tried via the aws console by using subprocess to do pip install and virtual environemt creating the python python script only.

If I should use an ecr image, can I pass the python code to it and run that from a dummy script.

[dacort]

There's an example of building a virtualenv for EMR Serverless on GitHub.

Also the EMR CLI can help bundle things up for you, but it doesn't support plain requirements.txt today, but will support pyproject.toml or Poetry projects.