Higher-order theories of gravity: diagnosis, extraction and reformulation via non-metric extra degrees of freedom-a review