Love Thy Neighbor

There is this guy who lives next door to us and he and Tim have kind of become buddies over the 5 years we have lived here. Mike is in his 40s, and lives with his mom. Despite the fact that he is kind of a deadbeat, he is not really living there because he can't hack it at life. No, he takes care of his mom and the house he grew up in, a fairly noble undertaking, right? Kind of.
Like I said he is sort of a deadbeat. He doesn't hold a steady job, and just kind of works as a handy man here and there when he feels like it. I have heard from a neighbor who he did some work for that he is kind of lazy, and didn't do a very good job, but still he is a nice enough guy.
Before we had a snow blower he would always do the strip of sidewalk between our houses after a snow storm. He lends us tools when we need them, and always chats with Tim over the fence. For some reason this summer their "relationship" has really blossomed and now Tim and Mike spend at least one night a week drinking beer and chain smoking. Is it horrible to say that I wish it would stop?
As I said, Mike is a nice enough guy, but he is astoundingly racist. I find it beyond painful to talk to him sometimes. I just can't listen to him complain about the "Pollocks down the street", or the "colored family" around the corner, and those are the just the basic slurs he makes. Once he gets comfortable with you (as he has with us), you get all the offensive gems about the terrorists, and any other minority. I learned early on that it was not worth it to argue with him on any level. He is one of those people who will never see the error in his judgment, so I don't bother and quietly do a happy dance that he has yet to procreate and pass on his ignorance.
My biggest problem is not only the racism, it is also the fact that he over stays his welcome. He stopped by our party Friday and ended up staying until 3am, making fun of the fact that we had a generator powering our house while the rest of the neighborhood "suffered". He came back on Saturday night to drink more of our beer and continue to kind of pick on Tim about our generator. He didn't stay as late this time, but he did drop plenty of racial slurs. I finally excused myself and went and washed my mildewy towels and did the dishes since our power had come back on. He tried to come over again last night, but I was kind of bitchy and I think he got the hint.
How do you strike a balance between anonymity (did I mention that his mom is home all day and totally keeps track of our comings and goings, and he has started to as well) and community? How do I maintain a neighborly relationship with this guy and not go crazy?