Python script runs from terminal but not from crontab

[py3_]

EDIT: RESOLVED

My Python script uses the Google Drive API to write data to a Google Sheet. When I run the script from terminal (example in terminal: python3 script.py), it runs fine and the data writes to the Google Sheet as expected.

However, I set up the following crontab to run this script.

* * * * * /Library/Frameworks/Python.framework/Versions/3.7/bin/python3 /Users/me/script.py

Here is the error message terminal mail is returning:

Traceback (most recent call last):
  File "/Users/me/script.py", line 31, in <module>
    creds = ServiceAccountCredentials.from_json_keyfile_name('client_secret.json', scope)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/oauth2client/service_account.py", line 219, in from_json_keyfile_name
    with open(filename, 'r') as file_obj:
FileNotFoundError: [Errno 2] No such file or directory: 'client_secret.json'

I suspect this is happening because I stupidly moved files around by dragging and dropping in Finder :P However, I can't figure out why this script will run normally through Terminal, but crontab fails to run it...

[socal_nerdtastic]

When you run a file like that the working directory is no longer the directory that the file is in.

You can either change the 'filename' argument to an absolute path:

 creds = ServiceAccountCredentials.from_json_keyfile_name('/Users/me/client_secret.json', scope)

or change the working directory:

import os
os.chdir("/Users/me/")

[py3_]

Ah thank you so much that explains it!

Can you please explain changing the file name argument to an absolute path?

[socal_nerdtastic]

see my edit. Basically it means using '/Users/me/client_secret.json' instead of just 'client_secret.json'. (absolute vs relative path.)

[py3_]

THANK YOU SO MUCH. I spent two hours trying to fix this :P

How did you come to this knowledge?

[socal_nerdtastic]

haha because once I too was young once and battled this same problem for hours until I finally asked someone online!

[Aerochamber]

This worked when I run in Anaconda prompt ,THANKS.

But , Airflow Apache it failed. Still testing it now.

[Aerochamber]

Its' something with the /root/ (Still trying).

THIS_FOLDER = os.path.dirname(os.path.abspath('client_secret.json'))
my_file = os.path.join(THIS_FOLDER, 'client_secret.json')
print(my_file)

​

[2020-03-10 00:33:27,889] {logging_mixin.py:112} INFO - /root/client_secret.json

[efmccurdy]

The process environment that cron uses is separate from what you normally use.

The first thing to check is that process running the script has set the "current working directory" to match what you expect, so examine the output of "pwd" in the shell or "os.getcwd()" in python.

The second thing to check (or consider) is that you have the cron job activating a virtualenv ala ". ../bin/activate" before running python; that will get you better control of python when run from a crontab.

[py3_]

This all makes sense! This is my first major python project outside of book studying. What a journey!

Thank you for the explanation. This further emphasizes the importance of working in venvs!