Well, let me say, I also never had a bad experience while visiting. Japanese people are very polite and reserved. I think it's hard to pick up on anything, especially in any daytime setting when everyone is trying their best to seem professional.
I'm sure speaking fluent Japanese would really help. Also, having a group of people you can get to know over time. You'll notice that all of my negative interactions were with strangers. I went there entirely alone, and worked alone while there, so I didn't really have a choice but to go out in the world and try to interact with people.
I think I could probably have a similar experience in the same context. I'm sure there are plenty of Japanese people who would be happy to be my friend, and heck, many of the Japanese people I met were very nice.
It's just that some were not so nice, and I think a general lack of comfort with someone who is different becomes obvious if you keep an eye out for it.
I'm glad you and your daughter had good experiences in the country though! I hope to hear more of the same. It's very reassuring for me.