You should not judge others by their religion. The person can be any religion (ateism is also a belief system) and as long as you dont get effected by him, you should treat him as a human no matter what his belief is especially if that person is so good to you and you feel like he is your father.
He may critisize your religious beliefs as long as you let him to. If you guys are that much close to each other you can ask him to stop and he should respect that as you respect his belief (of no-belief).
Every person can comment on an ayat from Kuran. It doesnt matter what people think. You make your own decisions and and you will see the Kuran as you want to see it. Dont get stuck too much about what people think and say about Kuran. Get your own lesson out of it and try to make yourself a better person with its teachings.
Let me answer your "big" question.
Does Islam approve of such relation with a non-Muslim who is also critical about Islam?
Not really. Our prophet have made trades with Christians and Jews. He also made good friends with few of them. Its not like you are going to marry this guy or you go to his ateist temple and pray with him nor practice his beliefs. You have a strong friendship (or you can call it Father-Son relationship) and you can always keep it that way.
My dear brother, I just read the last part of your message and to be honest i couldn`t find in Kuran which says anything good about non-believers. I wouldnt take Kuran word by word but its up to you to destroy this precious relationship because of your own view of Kuran.
I would personally follow below ayat.
"