I think we had 30 some-odd weddings, marriages.

Some people both on the show or another.

I think some 40 some-odd children were born, he tells Cagle.

Image

Credit: Greg Gayne/FOX

The get-togethers were amazing.

Watchfull episodes of the Jess Cagle Interviewnow at People/Entertainment Weekly internet.

Available on connected and mobile devices or just download the PEN app on your iOS or Android.

Image

The one memory Sutherland distinctly recalls came at the very end of filming.

A long-time handheld camera operator was picking up final coverage.

Thats the thing that I really remember was how much those people meant to me, he adds.

And I probably didnt let them know enough how much they did, but they did.

It was the thing that I miss the most.

We all spent eight years trying to finish this thing and we finally did.

I was lost for about a year.

It was like the worst breakup I have ever had.

It was very bizarre.

Designated Survivorairs Wednesdays at 10 p.m. ET/9 p.m. CT on ABC.

Watch the full Sutherland interview on PEN.

Go toPEOPLE.com/PEN, or download the free app on your Smart TV, mobile and web devices.