...that really irks me about complaining minorities in general is the way they fail to see how their actions are disproportionately evaluated against the "standards" set by the majority and the stereotypes held by that majority.
A black guy in the U.S. who commits a robbery is doing more damage to the "image" of blacks as a "race" than a white guy who does the same thing...because he lends creedence to previously-held majority beliefs, however irrational or wrong those may be. The converse of this is true: a white guy dragging a black man to death behind a pickup truck just reinforces the image many blacks have of whites. These attitudes aren't universal, and don't necessarily carry over intact from one locality to the next.
The greatest challenge facing a member of a minority is recognizing when he absolutely needs to be on his best behavior; recognizing when he is in enlightened company that will permit him to be himself without consequences, or when he is in the company of more provincial, less worldly people who can be relied upon to misinterpret this fellow's behavior as representative of a much larger group of people. Being deft and playing off the prejudices of these people can improve the lot of the minority, sometimes, but realistically the best I think we can hope for is not making things worse. Incremental improvement is the natural state of human relations that aren't tweaked constantly, so the longer gaijin are in Japan and don't cause trouble, the better things will get, generally. But the next reefer dealing English teacher or murdering, illegal Chinese makes it that much harder for the next English teacher or legal Chinese immigrant.
Whether or not it is right or "fair" of people to react with prejudice, it is a fact of life...the way of the world. Long-term, it's probably not a bad thing at all...it serves the wildebeast pretty well to react with alarm when a lion is in its midst, and that instinct is within each of us. Damaging the reputation of far more people than just yourself is something we should each avoid, but coming to grips with and addressing the idiocy of those who came before us and made our journey harder is sometimes our fate. A failure to react to this responsibility appropriately, whether you wanted to assume it or not, can breathes new life into old, tired prejudices.
In the end, the greatest mistake you can make is to assume that you, personally, can change the mind of any large number people. Ultimately, people think what they want.