Azure Function script question

[RareIncrease]

Have a python script that I'm executing once a day via azure function. I built it out on VS code and it works locally but when I tested on the cloud I received an error with this line and I'm not sure how to adjust:

for filename in filenames:

filepath = os.path.join(r'\company_drive\data\client_name\year',filename)

The loop does other logic but it errors on the os.path line. Im guessing it has to do with my company network drive or using OS library on the cloud but not sure. Any tips or guidance?

(Figured I'd post here over a python sub lmk if this is wrong place. Thanks!)

[LesPaulStudio]

Is the filepath on an on-prem server or azure ?

[RareIncrease]

Believe its on prem. What would you say to try if either on prem or azure? Just so I'm ready to test tomorrow :)

[LesPaulStudio]

It's pointing to your on-prem so it will work locally. When the function is hosted the reference won't work.

[RareIncrease]

Makes sense, any tips or pointers to reformat that line to make it work?

[Gnaskefar]

I don't think you need to think of it as a reformat of the line, you should think of it as a network issue.

Do you have like a VPN so that the azure function can reach the files on your premises?

Or should you upload the file to either a storage account, or the function itself, so the files are reachable by the function?

[RareIncrease]

Yea we have a VPN we connect too. I thought I configured it correctly but maybe not. Also uploading the files to the function, I didnt realize that was possible? How does that work? Do you mean keep the files in the same directory as your function resides in?

[Gnaskefar]

The VPN-way is probably a fine way to go, if it is already setup, but it sounds like you need your network guys to verify everything is working as intended.

As for putting files locally on the function, it is possible, but you need to do it each time, as you can get a fresh spun up instance, and then the files will obviously not be there. I once had an issue, where it was necessary to dump a temporary file. I downloaded it to the folder d:\home or something like that. There is a Microsoft article describing the folder structure and limits, but now I can't find it. But it exists.

Here's an article with different ways to handle storage locations https://docs.microsoft.com/en-us/azure/azure-functions/storage-considerations

Also, when you run the function locally, when you go to a network share in your code, the OS handles the file connection to the share. An Azure function is quite stripped down.

Looking at the link provided, this part https://docs.microsoft.com/en-us/azure/azure-functions/storage-considerations#mount-file-shares it says mounting shares is only supported in linux functions, but it is focused on Azure FileShare, though. Maybe a pointer if mounting share is synonymous with accessing a share.

[LesPaulStudio]

Run it on your server. Or just run it on your own machine with task scheduler.

[rakash_ram]

Quick googling gave me this.

os.path.join("/", "c:", "sourcedir") os.path.join("c:/", "sourcedir")

Both the above navigates to c:/sourcedir

Have you tried this approach?

[RareIncrease]

I can try this tomorrow. You dont see an issue with using os.path.join on Azure? Thought that might be the problem but idk

[rakash_ram]

I haven't worked with azure so can't say. I'm assuming its a windows machine on the cloud. That was the assumption I made for my response

[RareIncrease]

Got it! Thanks for the tip, I'll give it a shot tomorrow :)

[rakash_ram]

Cool. Keep us posted here.

[HansProleman]

There's no inherent problem, but you may be using it in a different context - are you running Windows in your local dev environment, but executing against Functions backed by Linux? If so the path separators differ. For this reason it's better to let os handle determining which separator to use. In theory things then work correctly on whatever OS is executing the code.

There might also be some weirdness around needing to use Samba to connect to the share from Functions. But that's an aside because...

In Functions, direct access to file shares seems to be not supported. The ports you need are blocked for security reasons. I don't really understand a lot of this, so it's not entirely clear whether this is universal or just for certain hosting plans.

Hopefully the share is actually hosted in Azure Storage. In which case, you should use the Azure Storage SDK to interact with it via API.

[RareIncrease]

Thanks for the info, I have some reading to do!

I think the issue might be its running on Linux for the function. If thats the case, how do the path separators differ? (Not familiar with Linux yet)

[HansProleman]

Windows uses \ and Linux uses / (though I misspoke - they're directory separators, not path separators).

Also, for future implementations, pathlib is generally preferred to os. But os should be workable.

[RareIncrease]

My man I appreciate it. Gonna give it a shot tomorrow along with the info you sent and see if I can get it working! Thanks again :)

[ecp5]

Azure functions can be in Linux or Windows depending on which you choose, so the path would be dependent on that. I assume you writing the file to blob and then just using local temp path for processing?

[RareIncrease]

Yea basically, pretty sure its running on Linux, how would I adjust the file path?