It's because when you open Exchange, it sends a request to the Gmail/Yahoo servers (where your mail is
really stored) for an update of any actions that have occurred since its last request (such as new e-mails/sent e-mails/drafts/etc). It includes activity from pretty much every mail folder.
So when you send an e-mail from Gmail/Yahoo itself, their servers save that sent e-mail. And then when you open Exchange, it syncs with Gmail/Yahoo servers - which is storing your inbox items,
as well as your sent items - thus reflecting the outbound mail you wish to remove from Exchange.
To change this though...
If you have Outlook 2007 or previous:
Go to Tools > Rules and Alerts. From there select the E-mail Rules tab and uncheck "Move Sent Items from Web Mail".
If you have Outlook 2010:
Go to File > Manage Rules & Alerts. Do the same as above.
Hopefully that'll take care of it for you.