The primary concern of this thesis is to understand the formation and properties of the first galaxies, as well as the influence of the first stars in terms of radiative, mechanical and chemical feedback. In particular, we elucidate the role of turbulence, ionizing radiation by massive Population III stars, mechanical feedback by highly energetic supernovae, and chemical enrichment. In light of the next generation of ground- and space based telescopes, we derive their observational signature in terms of recombination radiation, bremsstrahlung and 21 cm emission. We find that the cumulative 21 cm signal of the first starbursts might be observable by JWST. These probes are essential to test the theoretical framework of the first stars and galaxies and shed some light on this elusive period of cosmic history.