The impact on your wallet of the Standard & Poor’s downgrade of the nation’s credit rating is similar to what would happen if your own credit score declined: The cost of borrowing money is likely to go up, the Washington Post explained in the aftermath of S&P’s decision.

While consumers who have fixed interest rate mortgages will be immune to any changes in borrowing costs, home buyers shopping for a loan or those with mortgages that fluctuate may see a rise in rates later on, some analysts say.

Mark Vitner, senior economist at Wells Fargo Securities, told the Associated Press that he doesn’t expect the downgrade to drive up interest rates instantly since the economy is still weak and borrowers aren’t competing for money and driving rates higher. However, he expects in three to five years, loan demand will be much higher and then the downgraded credit rating might cause rates to rise.

Analysts are still waiting to see if the other rating agencies, Moody’s and Fitch, follows S&P’s lead in its downgrade of the U.S. credit rating. If so, the aftermath could be much worse, analysts say.

The debt deal reached by Congress last week was expected to save the U.S. from any credit rating downgrade. However, S&P said lawmakers fell short in its deal. Congress deal called for $2 trillion in U.S. deficit reduction over the next 10 years; S&P had called for $4 trillion.