Evaluate it with a boot in the binding, not by hand. Most brakes splay wider before pivoting down, but you may not be able to get the correct action when doing it by hand (ie, they may begin to rotate before splaying). I've seen people bend their brakes too wide because of this. So, stick a boot in, then release the binding. If the brakes rub, they will need to be bent wider.
In general, if there is 3-4mm between the metal arm and the sidewall when the brake is down, it should be fine.
EDIT: Not to dispute Mike, but I don't think the brake needs to clear the edge when it's in the up position. Most brakes are meant to tuck in about 5-8 mm in the stowed position, and it's OK for them to be inside of the edge. When the boot first releases they will splay out, and that's when it's important for them to clear the sidewalls. So that's why I suggest clicking a boot in and then releasing it to evaluate the fit.