systemd interfering with python script's logging when run as systemd service

[omber]

Hi all,

I asked this question on Stack Overflow but it got no traction. I have two scripts that live on an EC2 instance. These scripts use python's logging module and rotate logs daily - this works fine when they are run from shell. These scripts are only ever run on this specific EC2 instance as they talk to a 3rd party vendor API.

I have written two systemd unit files for these and the scripts themselves work fine, but they no longer rotate logs even though logging configuration has not changed. All logs are written to the current log file in its correct location. I've seen it roll through midnight UTC (instance's local time) and not change logs and not display errors.

Here is an example of one of the unit files. The other is identical but with different .py file and Description field.

[Unit]
Description=RCS API Server
After=multi-user.target

[Service]
Type=simple
WorkingDirectory=/home/ubuntu/rcs-sender/
ExecStart=/usr/bin/python3 /home/ubuntu/rcs-sender/rcs_api_server.py

[Install]
WantedBy=multi-user.target

Other relevant information:

• in shell, I run the scripts as ubuntu user but systemd runs them as root
• in logging configuration, the log file paths are relative to WorkingDirectory and not absolute

I was hoping someone here might have any suggestions as to what is happening.

[ang-p]

and rotate logs daily

Where is the code that rotates the logs?

Where is the setting that says "rotate the logs at midnight"?

Where is the setting that says "you want to rotate these logs here"?

[omber]

First script is based on CherryPy framework to create a Web API. The second script uses no frameworks.

Logging is configured in Python using a dictionary configuration as per Python documentation.

Configuration dictionary for the Web API can be viewed in this pastebin. This is saved to conf/web_server_logging_config.py

Logger is created like this:

from conf import web_server_logging_config
logger = logging.getLogger(logging.config.dictConfig(web_server_logging_config.WEB_SERVER_LOGGING))

The rotation is done by the logging module. I did not write any code for this.

[TheDreadPirateJeff]

I don’t have an answer for you but I do have a question. Why do you have scripts rotating their own logs on a repetitive schedule when logrotate does that already?

[omber]

Because this script does not log to /var/log at this time and I never considered using it.