Yes I would definitely give his stuff all back to him. The more you keep it around and see it the harder it will be to get over it. Plus even if he wouldn't care if you had it some of his brothers might. Just box it all up and take it over to him. If he has Greek stuff of yours (other than stuff like formal favors, etc) I would ask for that back also, for the same reason.
Look at it this way, even if some other chick does end up wearing his stuff, YOU wore it first.