Baby Step 6- The Dilemna

How do you decide how much extra per month to put toward your mortgage if you are on baby step 6?  We are having a hard time being pulled in different directions.  We want to pay off the mortgage asap, but we also want to give some away, have some fun, go on a vacation or two, and do some upgrades on our home.  The sooner we pay the house off the sooner we can give in a big way and really help others.  How do you find the balance or do you? Are you putting every extra penny toward your mortgage with focused intensity? Or are you enjoying life a little and putting a certain amount per month and not a penny more?