Charles Allan Gavan Duffy was the son of Charles Gavan Duffy, clerk of the Senate in the Australian Federal Parliament, then held in Melbourne, and his wife Ella. His grandfather, Sir Charles Gavan Duffy, had been a leading Irish nationalist in the 1840's, and was elected to the House of Commons; on emigrating in 1855 he became a leading figure in Victorian politics and society, including holding the office of Premier.

Young Charles was a medical student when he enlisted on 18 February 1918, and aged nineteen years old. He was sent to Maribyrnong for training and, on 26 March, 1918, taken on strength with the 35 Reinforcements to the Field Artillery as a gunner. The Reinforcements sailed for France from Sydney on 30 April, on board SS Port Darwin, disembarking at Southampton on 16 July. From Southampton, they marched into Heytesbury to the Australian Service Corps Training Depot.

Here Charles Gavan Duffy became ill with bronchitis and was sent to hospital at Sutton Very, where he remained from 18-29 September, 1918. He was returned to Heytesbury, but was charged with being AWOL on 19 January, 1919, having absented himself from camp from 7-17 January. On 1 January , 1919, he is recorded as being with the Gunner Artillery Detail, but as they were redundant some three months after the war had ended, he was immediately moved over to the 19 Australian Depot of Supply, posted overseas and taken on strength with the 10 Depot of Supply at Cologne. He returned to England on 2 June, 1919, disembarking a day later and marching in to the Australian Service Corps Depot. He was arrested for drunkenness at Westminster on 19 June, returned to Australia on 22 July and was discharged from service on 25 September, 1919.

In 1924, he was living at Trinity College, University of Melbourne, graduating as a Bachelor of Medicine. In 1926, he was awarded his Bachelor of Surgery degree and in 1931, his Doctor of Medicine. He was registered as a Member of the Royal College of Physicians, London, in 1934.

Charles Allan Gavan Duffy fought again in World War 11, and finally retired from the military in 1953, with the rank of Lieutenant Colonel, late of the Royal Army Medical Corps. He married Margaret Marianne Lucy Bowers, born in Staines, Middlesex in 1914. They had three sons: Charles Frank Gavan Duffy (b. 12 September, 1942); Desmond Gavan Duffy (b. 16 May, 1947, died January 1954); John Gavan Duffy (b.3 September,1955). Charles Gavan Duffy died in December, 1979, aged 80.