On my phone, which is a large, when in the mail app your press Menu>Account Settings>Incoming Settings and scroll down and it says 'Delete email from server' and gives you a drop down choice of Never or When I delete from Inbox.
The set up is slightly different on the mini but should be along the lines of the above - try settings>Sony Ericsson>Email.
