When Charles La Trobe, lieutenant-governor of Victoria, left Melbourne to return to England in 1854 he left his friend, Dr James Palmer, to sub-divide and sell his Jolimont estate.  Palmer bought the Jolimont Square section himself and subdivided the rest of the land around the periphery of estate for sale to individual buyers.  The section in the middle of the estate which was occupied by La Trobe’s cottage and its smaller neighbour, soon to become known as Jolimont House and The Hermitage respectively, remained intact.  This was the area bounded by Agnes Street, Palmer Street, Charles Street and Sophie Lane. Initially the two houses were rented out but in 1865 the land was offered for sale and in 1866 was bought by John Lupton, a wealthy Riverina grazier. 
In 1872 Lupton notified the Melbourne City Council that he intended to build a house in Palmer Street.  It appears in the 1874 Rate Books as a five-roomed house ‘off Jolimont Street’.  In 1874 Lupton died.  His widow, Mary Anne (nee Waite), initially retained ownership of Jolimont House and rented it out while the rest of the land was sub-divided into four lots and sold in 1877.  Three lots were given a Palmer Street address, two were vacant land and the third contained The Hermitage.  The fourth lot was described as ‘A brick cottage, containing six rooms, erected on land having [a frontage of] 19ft. 7in. to Charles street by depth of 125ft. 6in along a right-of-way.’ This has to be Lupton’s 1872 Palmer Street house. 
Further information from the Notice of Intention to Build tells us that Halliday and Walker were the builders. Subsequently four tender notices appeared in the newspapers advertising for bricklayers and painters.  The last two were in the name of A Craik.  Both William Halliday, contractor, and Abraham Craik, carpenter, had addresses in Cardigan Street, Carlton, so possibly they worked in conjunction. 
William Henry Wills, or Wells, a civil servant, was the next owner.  He sold to Richard Kissane, sergeant of police, about 1883-84.  Kissane lived in the house until about 1895 when he moved out and it became tenanted.
In 1911 the house was sold for £520 to William Ernest Lloyd, carpenter and, later, labourer, who moved in with his wife, Adele Alyce (nee Morton), and their two small children. In 1913 tenders were invited from bricklayers for a ‘small job’.  This could refer to the separate room or studio at the rear of the property which does not appear on the MMBW map of 1895 and therefore has to have been built more recently. In 1926 an application was made to erect a verandah.  The house now took on quite a different character from its original appearance.  
The verandah has a central arch between two square openings which sit above masonry balustrades. Four masonry pillars separate the openings. Above the arch the house’s name, Home, is written in stylised lettering.  The end verandah wall is filled in around a small stained-glass window depicting an idyllic rural scene: a house sits among rolling hills, over which the sun is just rising.  The hills lead down to the sea, and above the blue sky is streaked with protective bands of clouds.  A post and rail fence in the foreground provides a sense of security and safety to this peaceful scene.
The dream was short lived. The following year the Lloyd’s twenty-year old son, William (known as Jack), was killed when the motor-cycle he was riding skidded and hit a verandah post in Bridge Road, Richmond.  In 1928 their son-in-law, Charles Douglas Redfearn, a watchmaker who had married their daughter, Alice Adele, in 1918, died suddenly, apparently by suicide. Alice returned to live with her parents at 19 Charles Street. She continued to live there after their deaths and until her own death in 1968.  
The house was sold in 1968 to Kathleen Richman. Kathleen Richman has entered East Melbourne folklore as the ‘pram lady’.  Neighbours did not know her name then, or anything else about her but she was frequently seen on the streets of East Melbourne and Richmond fossicking in the rubbish bins.  Her old wicker pram was filled with bottles, and more bags of bottles were tied to its exterior, all on their way to the brewery for sale and re-use.  A changing flotilla of dogs accompanied her.   She only stopped when the Council introduced a glass re-cycling scheme and bottles were crushed rather than re-used.
Born in 1905 as Kathleen Romelli she grew up in the Wimmera where her father worked as a bullock driver. She became a tailoress.  After moving to Melbourne she met and married James Henry Richman, formerly a seaman and waterside worker, but by then a labourer and truck-driver.  After inheriting some money after the death of her parents in the 1950s she invested in a property on the corner of Jolimont Road and Jolimont Street which comprised two houses (102 Jolimont Road and 96 Jolimont Street).  Later, after separating from her husband, she moved in to one of these houses.  
She rented out 19 Charles Street as rooms until her death in 1987 when the house was again put on the market. It was sold for $264,000. It was one of four valuable parcels of real estate she owned at her death, along with cash accounts, term deposits and mortgage loans.  Her estate was valued for probate at $1,636,374.73 but was in fact worth far more. 
In 1989 the new owner applied for a building permit to make alterations.  This would appear to be the infill addition which can be seen from Sophie Lane which runs beside the house. More recent history of the house is not available.  It has been vacant for many years and is in a sad state of neglect.