Since March 1977, the Edgar Allan Poe/Sullivan's Island Library has been uniquely housed in renovated Battery Gadsden, a former Spanish-American War four-gun battery. Previously, the library had been located in the Sullivan's Island Elementary School building and then in the Township Building. At the opening ceremonies in the newly renovated quarters on March 11, 1977, County Council Chairman Lonnie Hamilton, III noted the unusual use of a former military facility by quoting from Isaiah 2:4, "They shall beat their swords into ploughshares."

The library is named for author Edgar Allan Poe, who was stationed on Sullivan's Island as a private in the United States Army in 1827 and 1828, and who used the island setting as the background for his famous story, "The Gold Bug."

Battery Gadsden was named for Colonel James Gadsden (1788-1858) of South Carolina, who had served as an engineer officer during the War of 1812 and later as inspector general. During the Seminole Wars, Gadsden served as quartermaster general of the Florida Volunteers. The Battery was placed on the National Register of Historic Places in July 1974.

Battery Gadsden was designed to mount four six-inch guns on pedestal mounts with fixed cartridges. Battery Gadsden was officially transferred to the Artillery Sept. 10, 1906. The four guns were removed for service in France in 1917, and the carriages were scrapped in 1920. The walls of the battery are two feet thick, a solid protection for the 15,000 books in the 2,000 sq. ft. library.