Charles Smith, the first owner of the house, owned a timber yard and saw mill in Albert Street, East Melbourne, in the 1850s-70s. He never lived in the house. He was the uncle of Helen Macpherson Smith, philanthropist. See Catalogue Reference below.

From the time of its building until 1869 it was the home of Mrs. Ainslie's School for Young Ladies which then moved around the corner to 179 Gipps Street and became known as Ormiston College.

Then known as St. Helens the house remained in the hands of the original owners, the Smith family, until 1882 when it became the property of Mrs. Olivia Gertrude Keenan. Mrs. Keenan appears to have run it as a lodging house or private hotel, and legend has it that it was a popular home away from home for cast members of J.C. Williamson's theatre troupes.

In 1888 four rooms were added. This may refer to what appears to be a separate cottage annexed to the house's northern wall.  On the old MMBW maps this cottage appears as continuous with the house.

Mrs. Keenan's daughter, also Olivia Gertrude, married Richard Byrne, a manufacture's agent and importer, and in 1894 the couple moved in and the house reverted to a family home. Richard died in 1908 but Mrs. Byrne and her children continued to live there until 1914. Her eldest son, Herbert Richard Byrne, was a major in the army and distinguished himself during World War I, receiving a DSO and being twice mentioned in despatches. It may have been he that gave rise to the second legend that during both world wars the house provided accommodation for army officers attending nearby training grounds.

After the Byrnes' departure the house again became a lodging house and, later, apartments known as Kelvin Mansions. In 1951 the property was re-named Magnolia Court after the impressive magnolia tree that once graced the front garden.

In 1956 it was acquired by the Presbyterian Church and the motel annexe was added. In 1985 it was sold and the new owners refurbished it and upgraded the hotel.