The house was one of ten houses of refuge commissioned by the U.S. Treasury Department for the United States Life-Saving Service as havens for shipwrecked sailors and travelers along the barren east coast of Florida.
The House of Refuge offered shelter to the survivors of the Georges Valentine shipwreck in 1904. Captain William E. Rea was the Keeper of the House of Refuge at the time and aided the seven survivors.

On October 17, during the same storm the Spanish ship Cosme Calzado wrecked three miles north of the
Georges Valentine, but fifteen of the sixteen men survived. The surviving crew joined the survivors of the Georges Valentine at the House of Refuge. The men later returned home via Jacksonville, Florida except for one. Edward Sarkenglov remained and became a local fisherman. Captain Rea and his wife lived in the House of Refuge until May 1907.
The facility also served as a lookout for enemy submarines in World War II. It was saved by the Historical Society of Martin County in 1955, and is listed on the National Register of Historic Places.
Today the House of
Refuge is itself a survivor; it is the only one of
the original ten houses of refuge to remain on the
Florida Coast.
With the permission of the Martin County Historical Society, Florida Ghost Team investigated this historic site in 2004. There were a couple of events during the investigation that rose suspicion of some paranormal activity.