Casual Fling Guy: do you think its OK to go back to him after 3 years?

It sounds like you are just looking for something to do, some down time between relationships would be desirable to assess what you want out of the relationship. In the end it is just a hook up but it could set a pattern for future relationships. You seem to like older guys, go for something new, and try someone your age, if you haven't already. That might be exciting. To repeat you could be setting up a dysfunctional pattern to dating.