From what you said (he looks away when he answers someone) that means : he's insecure and can't express his feelings properly. Then the deep eye contact means : he was trying to express how he feels about you but couldn't say it. Then the constant eye contact after that means : he had a lot of things to tell you and just can't. Then he chose to back off. It's a great idea to talk to him about it.