Competition between spin liquids and valence-bond order in the frustrated spin-1/2 Heisenberg model on the honeycomb lattice