The Celastraceae is a large and rather diverse family distributed in the tropical and temperate zones of the world. Most of its species have been subject to considerable nomenclatural confusion and name changes. Since the species was described by Carl Peter Thunberg (1794) as Ilex crocea, it has been treated under several species names within Elaeodendron or Cassine. The common names have also been a source of confusion.

Thunberg encountered the species near the Grootvadersbosch near Swellendam in October 1774. He described the species as follows: 'Yellow wood (Geelhout, Ilex crocea) is of a yellow colour, almost like box, of a close texture, and handsome. It is used for planks and beams in the construction of houses, for tables, doors, cupboards, window frames, and butter churns'. His description of the wood is entirely correct but it is probably not as widely used as the yellowwood derived from the two species of Podocarpus also present in the Grootvadersbosch (with the possible exception of butter churns that have been reported as being made from wood of Elaeodendron croceum). This forest was an important source of timber for Cape Town at the time but Thunberg evidently confused the uses of this yellow wood with that of Podocarpus species.

The common names recorded by Thunberg are reflected in his name: croceum means 'saffron yellow'. This colour is present as bright yellow pigment in the bark of many species of the family Celastraceae, including Elaeodendron croceum. Ecklon & Zeyher (1834) gave a literal translation of saffron wood when they proposed the synonymous generic name Crocoxylon. Other names that are synonyms for Elaeodendron croceum include: Crocoxylon excelsum; Cassine crocea, Elaeodendron capense and until recently Cassine papillosa.

The name of the genus Elaeodendron is derived from the Mauritius tree, Bois d'Olive (or olive tree), after the resemblance of the fruit to the true olive was noted by the first scientist collecting this tree.