my understanding is that officially it depends on what the dealer said. If he/she requested the cards then its a house mistake and you all suck it up. If it was player only then you all deserve a round penalty for being stupid enough to listen to a player.
Had this happen a few times, in cash the player was joking around, and no real harm was done. I've seen this in a tournament, and the player was believed to be angle shooting, can't remember the exact details, but I believe he was given a few hands away from the table and warned it would be a more serious penalty if it happened again.
The had is not dead unless the dealer says so, play to the whistle, the dealers whistle.