Kathy Schrock, one of the all time pioneer educators on the web and in using technology in education, has some great pointers on her blog. I like her list of the 7 Ps. I would modify one thing. She recommends 512 to 1024 MB of main memory and I would recommend going with the 1024 MB. While 1024 MB is not cheap memory remains the cheapest thing you can add to a computer to speed it up. So if you can afford it go for it.
Two other things I recommend as options: If your student really wants a large screen consider buying a monitor that they can keep in their room. That way you can get the best of both worlds – a large screen and portability. In fact a laptop running Windows XP Professional will let one use both the built in monitor and the external monitor at the same time. Two screens really does add to productivity. I’ve seen that in my own work. The other thing I recommend is a large capacity external hard drive. These connect using the USB port and are fast, reliable backup devices. Trust me you want to have your student backup their data regularly. As a side benefit to them there will be more room for the gigs of music and video they will want to download (hopefully legally).
Also regarding software, if your student is going to study computer science make sure the department they are going to is a member of the MSDN Academic Alliance. That way you will know that there is a lot of valuable software they can get for free. There is a special high school MSDN AA program as well BTW. So if you have a computer science loving student staying in high school their teacher may appreciate knowing about that.