The affair was not just on Dean, it was almost premeditated on Rory's part. She'd been giving jealous looks to him and Lindsey the whole time he was helping fix up the Inn. She knew he was married and didn't stop blame is 60/40 on Dean as far as I'm concerned.