To add the ability to format code in Umbraco's rich text editor I followed these notes from Umbraco, tinyMceConfig - Rich Text Editor Configuration.
Umbraco uses tiny for it's rich text editor which includes a code renderer but it is not enabled by default.
Tiny in turn uses a syntax highlighting library, Prism.
When getting the Prism files from their website I also got the line numbers plugin so that I can add 'line-numbers' to the class attribute of a pre tag so that line numbers are displayed next to my code.