Okay firstly learn soldering. And get all your basic electronics stuff from radioshack. Like soldering iron, solder, etc. To get more specific stuff like leds, psp parts, etc. You can get leds at ebay and psp parts from
dealextreme.com. Some easy mods could include a usb charge mod. It's easy, but it requires soldering to the mobo so if you're careless or you have bad hand-eye coordination, then I wouldn't suggest it. Honestly, if you're careless or have bad hand-eye coordination, you shouldn't mod in general. So yeah I'd probably start off with the usb charge mod. The tutorial can be found in our tutorials section.
good luck! Also, remember to use the search button. Many questions can be answered through that