The house was built for the Austrian born, landscape painter, Eugene von Guerard in 1862. It was initially described in the rate books as having four rooms but von Guerard had an additional three rooms built in 1866. James Dickson was the builder. One of the additional rooms was built as the artist's studio. It can be seen from the street - a single room perched on top of the back section of the house with a window on each side. Von Guerard lived there until 1882 when he returned to Europe. The next owner was R.S. Smythe, entrepreneur and manager of musicians, who leased the house to Miss Christian, R.A.M., a singer, who advertised she would resume tuition in March of 1885 at her residence.
The house was advertised for auction on 4 March 1925, and again on 8 February 1939, this time on behalf of the estate of John H. Daley, when it was rented at £104 yearly. The purchasers were Lord and Lady Casey for whom it served as a much loved town-house until Lady Casey's death in 1983. It is still owned by her estate and is used as a temporary residence for visiting artists.
It is uncertain who named the house 'Little Parndon' but the name was in use by 1937. The original Little Parndon was a village in Essex, now incorporated in the town of Harlow.

*Prior to 1890 the address was 12 Gipps Street.