Gustav Holst came to JAGS as Music Master early in his career, before becoming widely known as a composer. He was introduced by his friend Ralph Vaughan Williams, and was on the staff from 1904 to 1920.

He came on one day a week - Friday - to play the hymn for prayers and teach singing to the school, which numbered as many as 290. There was no Music School at this time, and these singing lessons took place in what is now known as the Holst Hall. He taught each class in turn, and the school choir.

A former pupil commented, "He was a born teacher, and could have drawn music from a stone". His entry in the Staff Register records his salary as £50 pa, rising to £78 in 1918 - not a large sum, even then. At the same time, he was teaching singing at St Paul's Girls' School (from 1905) and at Wycombe Abbey (1912-17). The register also shows the 'Von' crossed out when anti-Germanic feeling prior to the 1914-18 war led him to drop this from his name.

In 1905 the school performed Tableaux from Tennyson's Princess, for which Holst composed settings of the poems. The songs were sung from manuscript (some of which are in the school archives) and were published in 1907 as Songs from The Princess, Opus 20A, with the dedication 'to the girls of the James Allen's Girls' School.'

His Golden Goose, a ballet with chorus, also received its first performance in the gardens here. Holst was responsible too for establishing a School Orchestra at JAGS, and gave lessons on the instruments of the orchestra in addition to singing - entries in the account books for petty cash include items such as 'transport of Mr Von Holst's cello.' While teaching here, he was also Musical Director of Morley College, and in 1916 initiated the Thaxted Whitsuntide Festival for the Morley students.

In 1920 he brought the Festival to Dulwich, to the Old College Chapel and garden. The following year JAGS girls were invited to take part in a pageant of Purcell's Diocletian performed in the gardens of Bute House, Hammersmith.

In 1918, Holst took a year's leave of absence. Unfit for military service (due to poor eyesight and neuritis) he undertook the role of Musical Organiser for troops of the Army of the Black Sea, working in Salonika and Constantinople. From here, he sent postcards to some of his former pupils.

He left the school in 1921, but remained in touch and was remembered with great affection by his pupils. The magazine for 1925-6 records: "We have pleasure in announcing that the School has received gifts of music from Mr Holst and Miss Day (the musician Nora Day, a former pupil of his, who taught at JAGS 1918 -1920) … All will appreciate Mr Holst's kindly thought in presenting us with signed copies of his Savitri, The Perfect Fool, The Planets and other works, and we are very proud to have them." These works form the core of the Holst archive, slightly damaged, unfortunately, when an incendiary bomb hit the school in September 1940.