We report the discovery of extended X-ray emission around the powerful high-redshift quasar GB 1508+5714 at z=4.3, revealed in a long Chandra ACIS observation. The emission feature is 3-4 arcsec away from the quasar core, which corresponds to a projected distance of about 25 kpc. The X-ray spectrum is best fitted with a power law of photon index 1.92+-0.35 (90 per cent c.l.). The X-ray flux and luminosity reach 9.2x10^-15 ergs (0.5-8 keV) and 1.6x10^45 erg s^-1 (2.7-42.4 keV rest frame, Omega_Lambda=0.73, Omega_m=0.27, H_0=71 km s^-1 Mpc^-1), which is about 2 percent of the total X-ray emission of the quasar. We interpret the X-ray emission as inverse Compton scattering of Cosmic Microwave Background photons. The scattering relativistic electron population could either be a quasi-static diffuse cloud fed by the jet, or an outer extension of the jet with a high bulk Lorentz factor. We argue that the lack of an obvious detection of radio emission from the extended component could be a consequence of Compton losses on the electron population, or of a low magnetic field. Extended X-ray emission produced by inverse Compton scattering may be common around high redshift radio galaxies and quasars, demonstrating that significant power is injected into their surrounding by powerful jets.