I'm not sure attachments can be delete. At least I don't have a button for it. I've marked them as private now.