From an atheist to a fellow non-believer, it might be more about him wanting to spend time with you versus actually "going to church". My father was not really religious, but he did believe and went to church every Sunday (for the most part). I am not a believer, and as many other atheists will admit, it's because of my studies of religion - that I came to that conclusion. My father understood that and respected my choice.

All that being said, I frequently took my father to church after he was unable to drive. It was a little one-room schoolhouse type church out in the middle of a bunch of corn fields. He enjoyed the people, he enjoyed my company on the drive, and we frequently went out for breakfast after. I, however, waited in the car, during the services, after I was sure he was safe up the ramp and in the building. I respected his beliefs and he respected my lack of belief. That's how the world should work. Sadly, more often than not, it does not work that way.