Well, first is to figure out how you're getting there. You've got a few options:
1. Breck - Grand Junction via I-70 - south over Red Mtn Pass to Durango.
2. Breck - Buena Vista - Alamosa - Durango
3. Breck - Salida - Gunnison - Lake City - Durango
The first is probably the most obvious and boring, we'll ignore that.
The second is probably the fastest. It's a pretty drive and you'll go through the Sangre de Cristos. Pagosa Springs is on the way and if you time it right, you can have a late afternoon / early evening soak there. However, overall I don't think it's very exciting of a drive.
The third option has the best chances for a quick day hike and some cool little towns along the way. Now, I'm not sure how long you were planning on spending driving between those locations, but here's my recommedation for going there.. this gets you way off the beaten path. It's possible to complete this route in a day, but it would be a long day. You might want to consider staying in Silverton overnight. We'll assume you want to do this in a day. Also, we're going to consider you have 4WD or at least a car with decent clearance. (You really don't need 4WD for this in good conditions and it's almost possible to do all this with a normal passenger car.)
Start early in the morning in Breck. Go to Daylight Donuts and get fresh donuts around 7am or so. Don't linger in town, you've got some driving to do.
Head up and over Hoosier Pass, take a second to admire Mt Lincoln and Mt Bross (both 14'ers) as you cross to the other side of Hoosier Pass. Blow through Alma and Fairplay. Take a right at Fairplay and head toward Buena Vista. Take a left at 287 and head to Salida. Turn right at the next intersection, head up and over Monarch Pass toward Gunnison. Just beyond Gunnison turn left on Hwy 149 and head to Lake City. Now - if the weather is bad for some reason, you can either continue on to Montrose or go to Lake City.
This is where things get interesting.
At Lake City, reevaluate the weather again. If it's crap, continue on to Creede and get to Durango that way. It's good, then head through town. Now, it may be close to lunchtime and you'll either want to get food here at one of the cool little places or stop at the grocery store for something. Lake City is a neat place and definitely worth a quick stop. Just beyond Lake City is Lake San Cristobal. Turn up that road - it may say "Cinnamon Pass" - this is where you're headed. Note: there is a similar road going to a similar location leaving from Lake City called "Engineer Pass". You DO NOT want to get on that road.
Okay, so head up and over Cinnamon Pass. I think it'll take you 4 - 5 hours. Stop along the way and take a short hike if you want. This is gorgeous terrain in the San Juans and some of the best you can see from a car. Unless something has changed recently, the road shouldn't be that bad. If you bring your skis, you could even make some turns if you're willing to hike for 20 or 30 minutes.
On the other side of Cinnamon Pass you'll find one of the largest, semi-restored ghost towns in Colorado - Animas Forks. It's definitely worth spending a bit of time walking around.
From there, it's a short drive on to Silverton. Spend some time there, possibly even all night. It's a quaint little mountain town that isn't completely ruined (yet.) You could even drive up to the ski area about 10 minutes from town, but there's not much to see there.
If you don't decide to spend the night, continue south from Silverton to Durango.
That's my $.02.