What are you looking to do? i.e., do you want to race some day, or do you just like riding for fitness reasons?
Heart rate monitors are interesting tools; you can pick up basic units fairly cheaply these days. I personally use one in spinning classes at the gym just to verify that I'm not cheating myself. There are a variety of books out there that give information on how to effectively use one.
The "new rage" thing is to ditch heart-rate monitors and go with power meters, which tell you exactly how much power you're cranking out for each pedal stroke. Highly cool gadgets that are unfortunately still pretty expensive. If you're just riding for fitness though, a power meter is probably serious overkill.
Cyclometers are nice just so that you know how far you've gone. There are various books and internet resources that list various rides, complete with directions. They generally list turns in terms of distance, so it obviously helps to know how far you've gone.
My favorite cyclometer feature though is cadence -- i.e., how fast you're turning the pedals over. Most people find a cadence of high-80s, low-90s is ideal. You can always count yourself (i.e., count how many times your knee comes up in 6 seconds and multiply by 10), and over time you'll learn to "feel" what your cadence is, but until then -- it's nice to have a visual reminder that you're "bogging down".
All that said -- it is nice to know how fast you're going, especially when you're in a full tuck bombing down some hill.
As for knowledge -- learn how to change a flat. You will get one.
It's easy to learn... Easiest way to learn is to have somebody show you. Head down to your local shop and have them show you how it's done, or some shops offer clinics in basic bike maintenance, which are usefull to go to.