6 Reasons to Not Pay Off Your Mortgage Early

Buying and subsequently paying off a home is likely the largest asset transaction any of us take on. It’s an obligation so large that many of us will die before ever doing it. But is it worth it? Should you pay off your house? Should you focus on getting rid of your mortgage entirely?

In this article, I’ll give you several intriguing reasons why it may not be the best idea to pay off your mortgage early. What may surprise you the most is that not paying off your home can be a financially sound move that is grounded in math.

Penalties

Yes, you can be penalized for paying off your mortgage early.

If such a penalty does exist for your specific mortgage it can impart quite a blow. Such fees can be up to 2 percent of the overall loan balance if you pay off the loan within the first year or two of repayment and drop to 1 percent in the third year.

It should be noted that this type of penalty is not common, although it was likely not something you were worried about when you initially purchased the home. You were probably more concerned with qualifying for the loan and moving into the house itself. You probably weren’t keeping tabs on making advanced payments towards the principal.

Luckily, the industry has recently changed for the better. There was a time prior to the financial crisis, before the Dodd-Frank Act, that penalties could be as high as 5%.

Again, these types of penalties are uncommon… but you do need to review your origination paperwork just to be sure.

Low Interest Rates

It seems like ever since 2008 every financial advisor and talking head on financial TV has been touting that we are at, ‘historically low interest rates.’ Now that well over 13 years have gone by, we are at even lower rates… I would venture to guess that when another 13 years go by, we will probably be close to where we are now or even lower yet.

But that said, if you have a reasonable credit score, then your mortgage interest rate should at least be below 4% if not substantially lower (if it’s not you should consider refinancing). With that in mind, if your loan is accruing interest at a 4% APR then paying off your mortgage early is analogous to making a 4% return on your money.

4% (or less) isn’t very much. In fact, at least for 2021, 4% is less than the rate of inflation.

This means that the bank loaned you say $100,000 to buy a home. After you pay them back, they will only be able to buy about $98,000 worth of goods with that money (5.8% inflation for 2021 subtracting the 4% APR).

Let that sink in. You borrowed money, then paid it back and it was worth less. Hopefully your income keeps up with inflation, if it does then it was technically easier to pay back the debt later than it was when you initially took out the loan.

Now 5% inflation is high if not very high, but this observation can be generalized into a rule of thumb: over time, your mortgage payment becomes more and more insignificant.

Of course, again, this assumes your wages keep up with inflation. As the dollar is slowly devalued over time, your mortgage payment stays the same. As such, it becomes a smaller and smaller part of your budgeting equation. Paying it off early has less of an impact on your life and your budget the longer you have the mortgage.

Better Returns Elsewhere

Of all the potential reasons you may consider to not pay off your mortgage early this reason might be the most influential… both on your decision making and your net worth.

Essentially, the better returns elsewhere argument comes down to where you can make $1 work the hardest. When you earn money, you are given the ability to then put that money to work. If you put $1 towards paying off your mortgage, you will essentially be earning whatever interest rate on that $1 that you would have paid in interest.

For example, if your mortgage interest rate was 3%, then you can say that your $1 earned 3%.

But what if you can earn more elsewhere. What if it makes sense to hold onto that $1 because you can make 4%, 5% or maybe even say… 12% somewhere else? That’s the power of investing your $1 in the Stock Market. In fact, the 30 year Combined Annual Growth Rate (CAGR) of the broader Stock Market is about 12%.

When you choose not to pay more down on your mortgage than is required, and instead choose to invest your $1 over and over in the stock market you can benefit from compound interest. Over time you will have more and more money working for you at a higher rate.

Many wealthy investors will try and refinance their homes year after year to unlock more principal… they know the more money they can ‘take out of their house’ and put to work in other investments the better off financially they will be.

Finding better returns than what you pay in terms of interest on your mortgage is fairly easy. Mortgages are typically backed by the underlying house. In other words, if you don’t pay your mortgage the bank gets your house… so it’s a relatively low risk bet for a bank thus they charge a lower interest rate.

If your time horizon to invest that $1 is longer than 7 or 8 years, the risk of you losing money in the broader stock market is low. Thus, you stand a reasonable chance to earn a ‘premium’ on that $1 before needing or wanting to use it elsewhere (such as during retirement).

To wrap this section up… paying down your mortgage more than you absolutely must is not a good idea if you have a better way to spend that dollar. Spending that dollar could mean investing it. Additionally, it may even make sense to refinance your mortgage to a longer term so that you can untie even more dollars from your house and put it to work elsewhere.

Lack of Flexibility

Another consideration when it comes to paying down a mortgage early is liquidity. Paying additional money towards your mortgage is easy. Taking that same dollar back out is hard. If you have limited job security or a poorly funded emergency fund you may find that there is more value having access to liquidity vs a smaller home debt balance.

This thought process becomes even more clear when you consider a specific yet very realistic extreme: default.

What if you lost your job? What if you are in an accident? What if a loved one or spouse requires significant medical treatment above and beyond what insurance will pay?

The result from these scenarios may end up meaning that you are unable to pay your mortgage payment. Very often the result is bankruptcy and the loss of the house, all principal accrued, and possibly even more above and beyond to the bank.

What if instead of paying down your mortgage you set aside that money into a robust emergency fund? Perhaps you could have a higher threshold before the financial tragedy strikes. Perhaps instead of having 2 or 3 months of funds saved you could have 6 or 8 months of safety?

Not paying down your mortgage early may mean the ability to keep your house during a personal financial crisis. It may mean the difference between life as you know it and being forced onto the street. Putting it a different way: if you decide to pay down your mortgage make sure you can weather extreme but very possible financial situations.

Sit down with your spouse, your family, or your financial advisor and try to hammer out what the worst-case scenario would look like. Make sure you have 6, 8 or even 12 months of cash set aside somewhere safe… like an FDIC insured savings account.

This is not money that would be invested in the Stock Market (as mentioned in the reason above) unless you are thinking about a money market account. This is money that would need to be accessible and stable even if markets are crashing.

Any of the personal crises listed in this section could easily happen, and are probably even more likely to happen, during a major economic crisis. It’s more likely you will lose your job during an extrinsic crisis… its probably also more likely you are to suffer a heart attack due to the stress as well.

Taxes

Another thing to consider when paying down your mortgage is the inability to take advantage of the mortgage interest tax deduction. This deduction is spelled out in detail by the IRS here. The mortgage interest deduction allows you to deduct all the interest paid on a mortgage balance of up to $750,000.

Thus, on a $750,000 (the extreme case) 30-year loan at 3% interest you would be able to deduct a bit over $20,000 from your taxes per year. Note: this would go down each year as the balance of your mortgage goes down. Depending on your tax bracket, that could several thousand dollars.

But isn’t not paying interest at all better than paying interest and getting a deduction?

The answer is yes and no. It all ties into the point above about how to make a dollar work the best given its environment. If your mortgage was at 3% and your marginal tax rate is say 20% then you would be effectively paying 3% – (20% of 3%) = 2.4% interest on your mortgage.

Thus, in this scenario, the ability to find a place where a dollar can make more money than 2.4% is even easier than finding a place to make more than 3%. Considering the mortgage deduction makes the calculus to not pay down your mortgage even easier.

Debt Anxiety

Let me take a second and reverse course. There is one legitimate reason to pay down your mortgage that I would like to address: debt anxiety.

If having debt, whether it be credit card debt or a mortgage against your house, causes you anxiety then you should consider paying it off. There is no point in allowing your personal finances to cause you stress when it is not necessary… just make sure you have a solid emergency fund.

Wishing away debt anxiety is not an option for everyone. For some folks its something that they worry about all the time. Not having that anxiety could mean a much better quality of life. Quality of life will always supersede the mathematically right answer when it comes to paying off your house.

After all what’s the point of having even more money if it makes you anxious?

Summary

Should you pay off your mortgage early? Well, if you have an interest rate close to anything around 3% or 4% then the answer is a definite no. Chances are your money would be better off in a longer-term investment (assuming you have most of a 30-year mortgage left to pay off) or in a liquid savings account if your emergency account is currently underfunded.

At the end of the day, however, your happiness and quality of life come before the absolute best financial outcome. The decision to pay down or not down your mortgage will probably be one of the least important decisions you need to make despite the overall home purchase being the largest and most consequential (strange, huh?).

Other decisions like funding your retirement, establishing an emergency fund, or setting some money aside for your children(s)’s college will likely be much more impactful decision.

Financial Warning and Disclosure

I write this blog for fun. Please don’t consider what I have written here as financial advice. You need to do your own research.

Additionally, you should consider hiring someone for professional advice… someone who has your best interest at heart. This person should have a fiduciary responsibility on your behalf… someone like a Financial Advisor or an Accountant. I don’t claim to be either one of these… even on the weekends.

All that said, I hope you enjoyed the article… throw your comments down below! I love hearing from readers. Some of the topics can be quite unique for many individuals and hearing about everyone’s own experiences can help expand how we think about things.