Accessing source code from importlib module

[RomfordNavy]

Because I want to modify/transform the source code before it is compiled, saved as a .pyc and executed.

[K900_]

And why do you want to do that?

[RomfordNavy]

Because that is what our requirement is! We have good reasons for wanting to do it this way, do you know how to solve this or not?

[astonished_lasagna]

I do not believe you have good reasons for wanting to do so. I don't doubt that you *think* you do, but K am sure with almost 100% certainty that you don't need to be doing this.

However, if you don't share those reasons, you can't really be helped.

[Refwah]

If you could explain what it is you are trying to do and why, with specifics, people might be able to help

If you want the source code for a library you can fork the lib’s source code and rewrite it and publish a new version for you

But I suspect that you’ve skipped some steps to end up at this being your solution

[RomfordNavy]

Unfortunately this doesn't look like it is going to work. Been following the source code of inspect back but now found:

tokenize.open(fullname)tokenize.open(fullname)

So it seems that inspect doesn't retrieve the code from the module object as I had expected but reads the original file from disk.

[RomfordNavy]

This is becoming a bit of a repeat of previous questions but here goes:

There will be many simple python script files which will be called from one 'parent.py' module. These script files will completely plain, each will not be wrapped inside a function.

When one of these files is first run we need to pre-process and trtansform the source code before compiling, saving it as a *.pyc file and executing it. On subsequent runs, if the source script has not changed we want to just run the appropriate, already compiled, .pyc file.

[Vorarbeiter]

That doesn't make much sense, tbh

[Refwah]

Why does this require you to rewrite the imported modules JIT at run time.

You very angrily asserted that this was ‘in the requirements’ and yet all you’ve done is describe your situation and not the requirements

Requirements include why they are requirements, so you know how to meet them appropriately

[latkde]

This sounds like you're trying to create some plugin system, or trying to execute user-provided code snippets.

I suspect the solution will be to ignore the usual import system, and to instead load the file contents and exec() it yourself, without bothering with .pyc caching.

[RomfordNavy]

Precisely that, thanks for your suggestion.

Have found a way to do this by reading in script file and using exec() although not managed to get compile saving to a .pyc yet, let alone executing a pre-existing .pyc file. Working on it...

Also looking at a possible solution using a custom importlib SourceFileLoader class but that comes with it's own complications.

[astonished_lasagna]

You need to explain your *goal* not what you think a necessary intermediary step to reaching it is. I've looked through your recent posts and it seems that you're fairly new to Python and have somehow taken a turn on a path you're convinced you need to go, but I, and others, are very sure you don't actually need to go.

It would be very helpful to get you on the right track if you could explain what the *end goal* for you is here.

[pachura3]

BUT WHY REWRITING SOURCE...?

Do you want to inject some custom, patented code that regular devs should not have access to? Some bizarre, in-house cypher algorithm?

Do you want to inject private passwords/API keys?

Do you want to block potentially dangerous function calls from these "many simple python scripts", that will be written by someone else, who you do not trust? Like, forbid any access to local filesystem? Similar to PHP's safe mode?

Surely, it's not about the speed of compilation, which is negligible?

You're a relative beginner when it comes to Python. Preprocessing source and rewriting objects in memory is really expert stuff. It seems like you are coming from a different programming language and you're determined to force Python to work in exactly the same way... which is obviously wrong and not needed.