I think unless the drag is compared to a non modified z we can't really determine how bad or how little a difference drag makes. All I know that I lean towards it making more of an impact then not.
I remember watching a myth busters experiment about leaving your truck bed hatch open or closed while driving to increase MPG. It was found that you will lose about 3-5 mpgs if you leave the hatch down instead of up. All because the turbulence thats created in the bed of the truck while the hatch is down. So you can see how much harder the truck was working just to keep the same speed.
Again this argument only pertains to certain exhaust designs as previously mentioned.