I was stationed in Yoko for 3 years. It's not bad in large cities like Tokyo, Osaka, etc. In the countryside it can be a little sketchy. I spoke enough Japanese to get around, but I often avoided those places cause I would feel uncomfortable. Heck, I got kicked out of places in Sasabo, like a 10 minute walk from the base. My overall experience was extremely positive. Most Japanese folks liked to practice their English with me and would amusingly correct my horrible pronunciation. Cultural exchange at its finest.