Most people think that to draw the best artwork, you need the most expensive tools. This is untrue. I get most of my paper, pens, pencils, and other supplies not from an expensive craft store but from office supply stores (such as Office Depot or Office Max; Staples is especially good). What follows is a list of supplies, typical costs, and some tips on where to go or what to do to get the best price.

You don't have to be too picky when it comes to normal graphite pencils, although some brands of graphite are less likely to smear than others. Mechanical pencils are cheap, precise, and easily attainable; I highly recommend them. If you don't care for them very much, then a normal box of school pencils will work just as well as any brand of art pencil.
There's also non photo blue pencils. You can buy these individually at art stores, or in boxes. You save money if you buy them in bulk. They're harder to erase than graphite, but easier not to photocopy. There ARE brands of erasable non photo blue pencils out there; you just have to search for them.
Everything listed here is easily available at Office Deopt, Office Max, or Staples. You can also find these at most art stores, but they'll probably cost more.
PRICE LIST
Mechanical Pencils - $0.50-$1.50 each
Plain lead pencils - $0.10 - $0.50 each (A box will normally cost $2 - $4 dollars)
"Art" pencils - $0.50 - $1.00 each ($4.00 - $7.00 for a set)
Non Photo blue pencils - $0.50 - $1.20 each (A box of 12 runs $3.50 - $5.50. They're $4.69 where I am.)
The eraser I recommend the most are the plastic ones. They look like little white blocks. These will erase very well, won't crack, and last quite a while. Many different companies make them, but they're esentially all the same. Once again, the fanciest and most expensive isn't always the best.
There are also gum erasers, rubber erasers, and electric erasers. Gum erasers look like grey globs of tough clay. They're pretty good, but they make your hands smell strange, aren't as portable as other erasers, and most of the people who use them have irresistible urges to shape them into bunny rabbits or rubber balls. Rubber erasers are the pink, white, and green things that come on the end of pencils, or large blocks of the same. These are the cheapest, but don't erase as well, aren't as durable as other erasers, and they leave shavings everywhere. (They also tend to leave pink and green stains on the paper at the worst possible times). Electric erasers are large mechanical things that will erase most anything. It's basically a long plastic eraser attached to a spinning motor. Draftsmen with LOTS of pencil lines to erase use these.
PRICE LIST
Plastic erasers - $.50 -$1.50
Gum Erasers - $0.50 - #2.50
Rubber Erasers - $0.50 - $1.00
Electric Erasers - $25.00 -$75.00 (No kidding!)
There's dozens of pens out there you can choose, of many different brands, types and styles. I'll try and go over the pens that I listed in the inking section.
First, there's technical pens. These are pens used by draftsmen. Usually you can only find these in art stores, but office supply stores have begun to carry the disposable tech-pens. These aren't of as good quality as the normal tech pens, nor are they (supposed to be) refillable, but they ARE a great deal cheaper. (In truth, you CAN reopen a disposable tech pen and channel ink into it, but this process is messy, and the wrong type of ink can clog up the pen.) Pigma Pens are a good substitute for Tech pens, and are MUCH cheaper. You can purchase pigmas at any office supply store. An initial set of dip pens will cost you a little bit of money, but if you take GOOD care of them, they'll last you forever. Ballpoints are the cheapest but they don't photocopy well.
Brushes come in many different types, sizes, and prices, from the bargain bin plastic bristle brush to the rare llama hair watercolor brush. Then there's the wide range of brush pens as well. Most red sables are all you need in the way of actual brushes, but sadly you can't buy red sables at most office supply stores. You'll probably have to go to an actual art store for this one.
PRICE LIST
Technical Pens - $6 - $15 (A set will run you about $60)
Disposable Tech pens - $5 - $7 (A set will cost about $45)
Pigma Pens - $2 - $3 each (Set of four is around $10)
Dip Pens - $3 - $5 for nib and holder (Basic drawing set is about $9 -$15)
Ballpoints - $0.50 each approx.
Brushes - $1 - $25 (Avg $2 for a usable one)
Brushpens - $2 -$3

There's a wide range of things you can color with, from markers to watercolors to pencils.
Prismacolor makes what is probably the best selection of colored pencils and markers. You can find small sets at your office supply store, and large sets at the art store. You'll probably be better off purchasing your watercolors from an art supply store, although there's no one brand that's best. Personally, I use Angora water colors. The set has a wide range of colors, and they last. (My set has lasted me three years so far and will probably last a couple more.)
PRICE LIST
Markers - $1.50 - $3.00 each
Set of Markers - (small) $20 -$40, (LARGE) $40 - $200
Colored Pencils - $1 -$2 each
Colored Pencil set - (small) $20 -$40, (LARGE) $40 -$100
Watercolor set - $10 - $40
Depending on the medium you're going to be using, you can use many different types of paper, from normal typing paper to sketchbook paper to cardstock to bristol and watercolor paper. It's a good idea to use thick paper, mostly because it's less prone to rip when you erase on it.
A packet of normal "thin" paper is pretty inexpensive, but it's FAR from durable. Sketchbook paper is pretty good for most media, but then you have to carefully cut the drawing out of the sketchbook if you plan to sell it. Bristol and watercolor paper are both durable. (Note: if you're actually working in watercolors, don't use watercolor paper.)
There's a brand of cardstock paper known as the "Wasau Papers" that you can get at your local office supply place that is fairly inexpensive and durable. This paper comes in size 8-1/2" x 11" packets of 250 sheets, textured ("bristol," as the packet says), or smooth ("vellum"). A packet of this stuff will last you several months.
PRICE LIST
Photocopy paper - $2 - $5 for 500 sheets
Sketchbooks - $5 -$10 each
Bristol - $3 - $7 (price varies on size and quantity. A pad of 20 sheets costs me about $3.50)
Watercolor paper - $5 -$15 per pad
Wasau papers - $6 - $8 for a packet of 250 sheets
PHOTOCOPIES
If you plan to show your artwork around, or sell it at conventions, you'll have to make copies. You don't have to go to a professional printer to get this done. You can get nice inexpensive prints at your local copy shop.
My first word of advice is to *NOT* go to Kinko's. They're overpriced and many times you don't get what you pay for. Office Depot usually has very good quality photocopy machines and the prices are about half that of Kinko's.
Color copies tend to run a bit on the expensive side. Your best bet is to see if there's a Staples near you. If not, shop around. There's many high-quality, low-priced printers out there.
PRICE LIST
Black and white copy - $0.03 - $0.10 per side
Cardstock B&W - $0.07 - $0.14 per side
Color Copies - $0.75 - $1.75 per side