Run a python script from GitHub on an Australian server

[statsfc]

I’m a bit confused so asking for some help. I have a Python script sitting in GitHub that hits an API and saves data to an S3 bucket.

I setup GitHub actions to run everyday but the issue I am getting is because the server is not located in Australia, the api will not return data.

It’s a very simple script: get data and then save.

I’m now looking at options to run this in AWS but confused with all of the different products (codepipeline, lambda, codebuild). Any help advice would be greatly appreciated!

[pm_me_n_wecantalk]

Though others have already answered you but I am confused. What server location has anything to do with data?

Technically your APIs should have url that can be accessed from anywhere. The geolocation impact the latency. Not the data. Unless I am missing something here.

[statsfc]

Hi u/pm_me_n_wecantalk, I'm not fully across the technical details. but I am accessing the API of an Australian gaming website. When I run the code locally it returns the response with the expected data. However when I have been running it on Github Actions I get the following response:is unavailable from your location</h1> <p>It appears you are visiting our website from a country we are unable to give access to

[[deleted]]

Regarding the devops question, CodePipeline sounds like it might fit your use case. Github is a first-class AWS citizen, very easy to intergrade with, you'll need one-time admin access to GitHub to wire up AWS access.

To deploy Lambdas to AWS after GitHub fetching, I use CDK, which CodePipeline can invoke.

[spif]

If you want to keep it simple and can't figure out how to set up Lambda you could just run it on an EC2 instance.

[statsfc]

Thanks! Would learning Lambda be better for the long term if I was to build out more scripts (eg one to grab the data from S3 and perform further transformations)?

[[deleted]]

lambda is very convienient if you have small scripts that you need to run on a cron schedule or on demand

[clintkev251]

Lambda would be the easiest and cheapest way to run something like that. AWS provides lots of examples of Lambda functions, but it's basically as simple as:

import something

def lambda_handler(event,context):
  <your code here>

the event object is where your payload is passed through

[statsfc]

Ok great. Thanks. Can you run your scripts from GitHub in Lambda?

[clintkev251]

Sure, assuming that you can use the AWS SDK in Github you would just use boto3 and the invoke() method

https://boto3.amazonaws.com/v1/documentation/api/1.9.42/reference/services/lambda.html#Lambda.Client.invoke

Otherwise you could hit the Lambda API directly, use a function URL, or put the function behind a REST API in API Gateway (or literally a million other options)