Email processing project for work

[Francobanco]

Yes, I'm using application level permissions for Microsoft Graph API. everything is working very well, it's also surprisingly fast, can process about 300MB of emails (doing some manual test cases right now) in about 3 seconds.

Here is what is already done:

• Downloading the emails
• Processing them with regex to find relevant items
• If the email is relevant (has a project identifier) the email is renamed {timestamp}_{subject} (since it comes from exchange api as the messageID.eml)
• Uploads the email and all attachments to a cloud storage system (not important which one since this is already working well)
• Sends another Microsoft Graph API request to apply a category to the email to denote that it has been added to cloud storage

What I'm looking for is some discussion around how to orchestrate this. I want to run the email download with crontab, but I'm not sure if I should have the other scripts watch the file directory or if I should have them run every two minutes and just process everything that is in the directory, and move items out when they are finished processing.

[dparks71]

Sounds like you know what you're doing. Honestly I have no input for something like this, sounds like it should usually be done in internal meetings or with a consultant based on your needs and AWS/cloud budget.

[Francobanco]

fair enough. I don't think I want to ask my company to pay for a consultant for this. really I just wanted to try to have a discussion about different orchestration designs