As you come off the gas on shifts there is a loud discharge, "woosh" ...sound like pressure releasing...is this pretty loud woosh normal for this Mazda Speed MX 5?
It could be normal, depends on what the noise level is, but I'm suspicious of a loose air intake system clamp allowing excess turbo boosted air to escape when you close the throttle when the turbo's spinning fast from just having been under load.
Here's a directory link for you if you want an opinion from someone who can listen to it:
http://repairpal.com/mazda-repair-in-scottsdale-arizona
RepairPal Certified shops can diagnose and repair your 2004 Mazda Miata. They provide transparent pricing with a minimum 12-month or 12,000-mile warranty on repairs and services.