The key is to state it in a way that isn't racing.. For instance:
A ricer came up next to me and downshifted. I of course DID NOT downshift to 3rd and DID NOT blow right by him. The Ricer DID NOT look like an idiot since we DID NOT race.
Easy as 1 2 3