Marmion was modeled after the antebellum mansion Waverly. Eudora and her college friends used to walk there. On the walk was a railroad trestle which inspired its appearance as a focal point in the book.
These photos and description of Waverly Mansion are interesting.
http://www.exploresouthernhistory.com/waverly.html