Test drive the Nismo 6MT, I probably would have went that route, but they didn't have those the first week the Z came out. I bought my touring the weekend they hit the dealers, was waiting for 3 months :-)
You probably will feel a little cramped in the cabin coming from a big car like the Challenger, but you will get over that soon enough. I came from a Mustang, and I'm really happy with the way my Z handles and drives with just a few basic mods. springs, sways, spacers, exhaust