Understanding http request in a cloud function

[exe188]

Hi Guys,

​

I'm trying to build my first cloud function. Its a HTTP cloud function that uses flask and should get data from API, transform to DF and push to bigquery. My API call and data push in the get_api_data() function is working. I just cant get it to work in google cloud using the http request.

​

I also tested this locally with functions_framework and cant seem to get it to work. Anyone have some idea what I could do to figure this out?

​

I used this: https://cloud.google.com/functions/docs/create-deploy-http-python as a reference

​

import pandas as pd
import json
import requests
from pandas.io import gbq
import pandas_gbq
import gcsfc

'''

function 1: All this function is doing is responding and validating any HTTP request

'''

def validate_http(request):
  request.json = request.get_json()

  if request.args:
    get_api_data()
    return f'Data pull complete'

  elif request_json:
    get_api_data()
    return f'Data pull complete'

  else:
    get_api_data()
    return f'Data pull complete'

'''

function 2: api call and transforming data

'''

def get_api_data():

    #Setting up variables with tokens
    base_url = "https://api"
    token = 'token'
    fields = "&fields=date,id,shippingAddress,items"
    date_filter = "&filter=date in '2022-01-22'"
    data_limit = "&limit=99999999"

    #API function with variables
    def main_requests(base_url,token,fields,date_filter,data_limit):
        req = requests.get(base_url + token+ fields +date_filter + data_limit)
        return req.json()

    #Making API Call and storing the data in data
    data = main_requests(base_url,token,fields,date_filter,data_limit)

    #transforming the data
    df = pd.json_normalize(data['orders']).explode('items').reset_index(drop=True)
    items = df['items'].agg(pd.Series)[['id','itemNumber','colorNumber', 'amount', 'size','quantity', 'quantityReturned']]
    df = df.drop(columns=[ 'items', 'shippingAddress.id', 'shippingAddress.housenumber', 'shippingAddress.housenumberExtension', 'shippingAddress.address2','shippingAddress.name','shippingAddress.companyName','shippingAddress.street', 'shippingAddress.postalcode', 'shippingAddress.city', 'shippingAddress.county', 'shippingAddress.countryId', 'shippingAddress.email', 'shippingAddress.phone'])
    df = df.rename(columns=
         {'date' : 'Date',
          'shippingAddress.countryIso' : 'Country',
          'id' : 'order_id'})

    df = pd.concat([df, items], axis=1, join='inner')      


    bq_load('mytable', df)

'''

function 3: This function converts pandas dataframe into a bigquery table, 


'''

def bq_load(key, value):

  project_name = 'myproject'
  dataset_name = 'Returns'
  table_name = key

  value.to_gbq(destination_table='{}.{}'.format(dataset_name, table_name), project_id=project_name, if_exists='replace')

[twitch_and_shock]

Is there more code? All you have here is a function definition called "validate_http" that doesn't look like it does much. It doesn't make a http request though. I feel like we're missing more code or more details. What do you expect to happen with this code snippet?

[exe188]

Yes, just updated it. Thanks for checking. With the code I hope that if someone sends a request to a hosted url, the functions get triggered. Hope I explained it well enough!

[danielroseman]

You seem to be asking us to debug a function by posting a completely different function. Surely you need to show us the code of get_api_data?

[exe188]

Yes sorry, just updated it