One of the things that's most nifty about PowerShell is the easy COM access it gives you, although sometimes it's easy to forget.  Here are a few examples that may help give some ideas.  Yes, you can automate Outlook from tons of other languages as well, I'm just trying to get across that it's pretty easy in PowerShell :)  Also, you can automate lots of other apps as well, especially other office apps like Excel.

Some constants to keep in mind.

CONST olAppointmentItem = 1
CONST olFolderDeletedItems = 3
CONST olFolderOutbox = 4
CONST olFolderSentMail = 5
CONST olFolderInbox = 6
CONST olFolderCalendar = 9
CONST olFolderContacts = 10
CONST olFolderJournal = 11
CONST olFolderNotes = 12
CONST olFolderTasks = 13
CONST olFolderDrafts = 16

Emptying your Deleted Items folder

$deletedItems = $outlook.Session.GetDefaultFolder(3) # == olFolderDeletedItems
$deletedItems.Items | %{ $_.delete() }

Create an appointment for tomorrow

$calendar = $outlook.Session.GetDefaultFolder(9) # == olFolderCalendar
$appt = $calendar.Items.Add(1) # == olAppointmentItem
$appt.Start = [datetime]::now + [timespan]::fromdays(1)
$appt.Subject = "something to do tomorrow"
$appt.Location = "my office"
$appt.Save()

Look at the subjects of the first 5 emails you sent

$sentMail = $outlook.Session.GetDefaultFolder(5) # == olFolderSentMail
$sentMail.Items | select -first 5 TaskSubject

Adding a contact

$contacts = $outlook.Session.GetDefaultFolder(10) # == olFolderContacts
$newcontact = $contacts.Items.Add()
$newcontact.FullName = "Windows PowerShell"
$newcontact.JobTitle = "Doing your dirty work"
$newcontact.CompanyName = "Microsoft Corporation"
$newcontact.Save()

When you're all done with whatever you're using, ideally you should be nice and clean up these references (unless you want to keep Outlook open, of course), with commands like:

rm variable:newcontact
rm variable:contacts
rm variable:outlook