I think you did a good thing Simon. If it had been some insane killer, he would've probably asked to come home with you, not try to walk away. From what i know (not much) you should be safe. According to what you say, he seems like a decent fellow. Good job!