When I wrote my first ML book in 2014 the chapter on Neural Networks came with a fairly big warning. Only use neural nets when all other options such as decision trees, supprt vector machines, k-means etc had all been tried first.

I would do talks and make the audience bet on three models and their accuracy. It gave them skin in the game, it made them think, and they usually got it wrong with the small amount of data that was on hand.

Nice post.