Musings on Statistics
Historically, the concept of uncertainty has been around the notion of how frequently one observes an event of interest, but it has since expanded to account and quantify for a lot of other, more intuitive forms of unknowability. Perhaps these could be explained mechanistically, i.e. considering the uncertainties associated with complex events as the aggregate of the component-wise uncertainties that make up the event.
This way of thinking is completely unnecessary though, as the axioms of probability can be assumed in many scenarios, no matter what the fundamental meaning of a probability is.
A model is perhaps just an idealisation or a description that is a collection of random variables which link up to each other using a distributional assumption. We can do a lot of cool stuff using models though, e.g. describing the plausibility of things.
Another interesting thing about statistical reasoning is that a lot of ideas in the philosophy of science (e.g. falsification, “strength” of induction, not being able to study hypotheses individually, the inability to separate evidence from theory, simplicity of hypotheses, etc.) have corresponding statistical parallels.
I’ll try to give examples of efficient gaussian process computation here, like the vec trick (Kronecker product trick), efficient toeliptz and circulant matrix computations, RTS smoothing and Kalman filtering using state space representations, and so on.
Very untidy first working draft of the idea mentioned on the efficient computation page. Here, I fit a spectral mixture to some audio data to build a “generative model” for audio. I’ll implement efficient sampling later, and I’ll replace the arbitrary way this is trained with an LSTM-RNN to go straight from text/spectrograms to waveforms.
First of my experiments on audio modelling using gaussian processes. Here, I construct a GP that, when sampled, plays middle c the way a grand piano would.
… using Stan & HMC
A Minimal Working Example