The most obvious questions needed to know before answering are:
1. what type of binding
2. are you talking about a race plate or the little plastic lifters
there will be two schools of thought on this.
1. there are the folks who live by mounting bindings flat on a fat ski. allows the ski to flex more in variable conditions.
2. there are the folks who prefer the lifter for the potentially increased edge hold it will lend the ski on packed terrain.
if you are merely talking about the plastic lifters that came with your bindings (assumming these are either Rossi Axial or Look PX binders) the amount of lift is going to be so minimal (I want to say it's between 1 and 3mm) that you can go either way and probably won't notice the difference one way or another.
If you are in fact talking about race plates, I would say No.