A musician rather than a painter by profession, Brown was almost fifty when the art critic Frank Rutter spotted and admired his watercolours. He introduced Brown to Lucien Pissarro, who guided his development in oil painting and impressionism; they became good friends and made several painting trips together to Devon with fellow artist James Manson. The friendship with Pissarro brought Brown into close contact with the Camden Town Group.

Like Pissarro, Brown applied the pigment in small touches, emphasising the grey, mauve, blue and green colours of the spectrum and two of his paintings were included in a Post-Impressionist and Futurist Exhibition held in London in 1913. Although he showed a few of his pictures at the New English Art Club up to 1919, Brown rarely exhibited his work publicly. Island House in Chiswick was the family home of his second wife, Helen Milne, whom he married in 1927.