To add the ability to format code in Umbraco's rich text editor I followed these notes from Umbraco, tinyMceConfig - Rich Text Editor Configuration.

Umbraco uses tiny for it's rich text editor which includes a code renderer but it is not enabled by default.

Tiny in turn uses a syntax highlighting library, Prism.

When getting the Prism files from their website I also got the line numbers plugin so that I can add 'line-numbers' to the class attribute of a pre tag so that line numbers are displayed next to my code.