I think the guy that drives the green NST AE86 (*Willard?) wears regular skater shoes. If its good enough for him, its gotta be good for me.
In fact, I prefer to drive wearing my beat-up VANS and Chucks (Chuck Taylor, that is). However, I am not a racer, so i wouldn't know proper racing shoes.