The public library (not to be confused with the Peabody Institute Library in Peabody or the Peabody Institute in Baltimore) was originally built in the Gothic style at its present location in 1869 with funds provided by local philanthropist George Peabody. However, the building was destroyed by fire in 1890.

The large Georgian Revival style structure was then built in 1892 for the sum of $34,218. The basement was remodeled in 1963 for use as a Children's Room and stacks and then in 1980, a new floor was added to replace the upstairs auditorium and 12,000 feet were added in the basement to create a new Children's Room and to house the new Archival Center.

The library maintains departments for adult services, archives, a children's room, information services and young adults. The George B. Gordon Meeting Room is available for reservation, which can be done online. Outside, the grounds contain Peabody Park to the rear of the library. You can also find the Peabody Institute on Facebook.