I second some of the above. Good that you saw it coming at the last second, and most importantly you guys did what so many others don't. You stayed with the boat. About 13-14 yrs ago, there was a situation in Vineyard sound with some classmates of mine. They tried to swim for it. 2 of the 3 didn't make it.
So many folks try to "fix" that sort of situation by taking action. I think most will agree that you did the right thing, and stayed with the boat. It is just too hard to see a person swimming in those sort of conditions.
Glad to hear that you are both ok.