Hilbert Space embeddings of distributions are a versatile tool for describing distributions. This talk discusses how they can be used to correct covariate shift (that is, when the distribution of the training set differs from that of the test set) and how they can be used to perform estimation when only summary information is available instead of full label information. In both cases we obtain rates of convergence that match settings with full label information and correct distributions. Moreover, similar techniques can be used for density estimation. The talk concludes with an outlook to how Hilbert Space methods can be used for message passing in graphical models.