I'm no expert, I'd actually classify myself similar to you... Here is my idea of a cheap, nice machine for overclocking...
Get a cheap case nothing fancy. I would use an old case i have lying around, but if you don't have one just buy a cheap one, preferrably without a power supply because those psu's are shit in a cheap case.
Buy a nice power supply. I recommend a low end 350watt power supply, but 300watts is the bare minimum. My 350watt psu has worked great. It is very nicely put together, and you can tell it is a solid piece of equipment. Mine cost $50 about 6 months ago. If you wanna spend $25 on it, just get some no name (BUT NOT GENERIC!!!) 350watt or more power supply. I've seen 450watt no name brand power supplies for $25, so it shouldn't be a problem.
If you want a cheap cpu that is easy to overclock, go with a Athlon Tbird. If you buy a 1.33GHz tbird, i think i heard someone say you could get it up to 1.53GHz+ with some decent cooling. I believe they run about $50. The idea of getting a Tbird is, unlike the Athlon Xp's, the tbirds don't have a locked multiplier meaning you can overclock it safer, without having to raise the FSB as high. buying a decent heatsink and fan is a must... you don't have to get the $50 ones, but read some reviews on one... Generally the ones that cool the best for under $20 are the ones with 7,000rpm fans. i got my dynatron for $15... accordign to hardware doctor, it goes up to 8,000rpm. MAKE SURE YOU USE THERMAL PASTE, although i'm sure you already knew that.
As motherboards go, buy a cheaper brand that has proven to by reliable. There is no need to go get the $150 ASUS board for this, and 266FSB is all you will need for the most part... 333FSB wouldn't be in a "cheap system". The board would be more expensive, and so would the ram. You might try Shuttle or ECS. You probably won't have to spend more than $50 on the board. Get a motherboard with DDR. Try to get a mo/bo that has a lot of onboard stuff, like audio, network card, or even video. You can always upgrade later and you don't need a 128MB Geofrce 4 Ti4600 to get it running... think cheap.
If you get a 266FSB motherboard, PC2100 should be good enough, although PC2700 is ideal. I wouldn't get more than 256.
I wouldn't buy a real ncie video card for it, i'd use an old one lying around like a voodoo3, until you know the system works. If you have to buy a video card, you migth try a Geforce 2, unless you want to invest in something up to date.
I would get a 20-40GB 7200rpm hard drive. in my opinion, 5400rpm is too out of date to buy anymore. If you have a slave drive in your current machine that you don't ened, you could use that.
Just get a floppy or a CD-ROM so that you can install an OS. If you're going to buy a CD-ROM, you might consider a DVD-ROM, because they are just as cheap now.
I can't think of anything else you would need. You might wanna even get some parts used! Getting a used cpu and motherboard together is a find, so keep your eyes out on forums and ebay. Everything you don't want to buy used, look for the most expensive part you need at
http://www.pricewatch.com and looks to see if a site that looks good has all the other parts you need. It's ideal to order all of the parts from one site for shipping reasons.
You might want to look at this thread with similar question asked by me
http://www.pcabusers.com/forums/showthr ... adid=21599
About that thing that is reviewed... It's some sort of Bus that puts soem ports from the back of your computer up front for convinence. It has stuff like USB, firewire, and a couple of switches to turn things like fans on and off. I think it's a waste of money, i wouldn't say it's worth buying one