Did they break the contract? How? If there are defects left to be corrected, do not pay the final retention until they correct them. Notfiy them in writing via certified mail that corrections are needed.
If you allowed them to install the windows 7-1/2 months late, you agreed to continue the contract, late or not. So yes, you have to pay them. Given that you have physically taken the windows in your property, they can file a lien on your property if you do not pay.
Since you have issues, I’d suggest a partial payment to them, advising that the remainder of the payment will be sent when all problems are taken care of.
Partial payment is at least necessary. Ask for or check over your detailed bill. You must pay them for any materials received in good condition. The broken windows and labor are the disputed items.