Welcome to Tom's Hardware Forums!
Start out with a digital camera like this one - Nikon D3100 Digital SLR Camera with 18-55mm NIKKOR VR Lens. You can see the details at
http://www.bhphotovideo.com/c/product/730210-REG/Nikon_25472_D3100_Digital_SLR_Camera.html
B&H Photo is a great online store to buy electronics; very similar to Newegg, and I am a B&H customer for over 25 years and recommend them. Local stores such as Best Buy, Walmart, Sam's Cluc, Costco, are just as good but maybe priced a little higher.
After you start out with the Nikon D3100 and get familiar with Manual settings as well as Automatic settings, look into a "Macro lens". A Macro lens is what you really need for close-up photography of items like jewelry, stamps...