He married Elizabeth Georgina Frances Elliot, daughter of Admiral Sir George Elliot and Hersey Susan Sydney Wauchope, on 28 February 1865 and had four children.
Helen Alice Carnegie (died 1908), who married barrister Sir Francis Lacey in 1890.[2]
Lt. Ian Ludovic Andrew Carnegie (1881–1909), who married Anna Barbara Atkinson, youngest daughter of Col. Bradford Atkinson, in 1905. After his death, she married Capt. Reginald Dalrymple.[2]
He rebuilt Longwood House, Hampshire, his main home. The architect was George Devey. He died on 9 September 1891 at age 47.[2]