/------------------------------------------------------------------\ | | | Variational Bayesian State-Space models | | (aka VB Linear Dynamical Systems) | | | | Copyright Matthew J. Beal 22/05/01 version 3.0 (11/08/03) | | http://www.cs.toronto.edu/~beal/software.html | | | | README file for version 3.0 (11/08/03) | | | | Please notify me, and include an acknowledgement/citation in any | | publications/research that use this code. Thank you. | | | \------------------------------------------------------------------/ o Also in this tar file are the matlab .m functions:- trigamma,fixedpointsolver,klgamma, lndet, Fcalc, getmarginals, backwardpass, forwardpass, digamma, the script sinus, and miscellaneous other .m files. o This version v3.0 of the code is bug-free as far as I can tell. o Works with multiple, variable length, multi-dimensional sequences. o Support: none provided just yet, aside from this extended help file for vssminpn.m (see bottom of this README), and the demo script sinus.m. o Please make sure you check chapter 5 of my thesis before emailing me with requests for help. o I am very interested in collaborating with people who have in mind exceedingly interesting and important applications of this model! o For reference material you are directed to: Ghahramani, Z. and Beal, M.J. (2001) Propagation Algorithms for Variational Bayesian Learning. In Adv. Neur. Inf. Proc. Sys. 13. MIT Press, Cambridge, MA, 2001. available from http://www.cs.toronto.edu/~beal/papers.html and chapter 2 and especially chapter 5 of my thesis: Beal, M.J. (2003) Variational Algorithms for Approximate Bayesian Inference. PhD. Thesis, Gatsby Computational Neuroscience Unit, UCL. available from http://www.cs.toronto.edu/~beal/papers.html o Your feedback is always welcome! Good luck! -Matthew Beal 12/08/03 email: beal [at] cs.toronto.edu o Known bugs: none so far at v3.0. ------------------------------------------------------------------ function net = vssminpn(Yn,inpn,k,its,dispopt,cFbool,hypit,net); ------------------------------------------------------------------ o type 'help vssminpn' at the prompt. An expanded help follows below:- REQUIRED arguments:- Yn is a (1 by N) cell array of observation sequences, with each entry in the array a (p by Tn) matrix of observations, where p is the dimensionality of the data, and Tn is the length of the n'th sequence. Thus the sequences are allowed to be different lengths. inpn has the same form as Yn, with each cell being a (pinp by Tn) input sequence, where pinp is the dimensionality of the driving inputs to the state-space model. You are allowed to specify just a single input sequence, and it is replicated as many times as needed for each of the N training sequences; then inpn is a (1 by 1) cell array, with the entry being a matrix which must have second dimension at least max_n(Tn). OPTIONAL arguments: k is the dimension of the hidden state-space. The default k is p, the dimensionality of the observation sequence. its is the number of iterations of VBEM required, default 100. dispopt is the degree of visual output verbosity during learning, there are 4 degrees: 0-nothing; 1-info+progress; 2-printout of F (if cFbool set); 3-graphical progress (if cFbool set). cFbool is an option whether or not to calculate the lower bound after each iteration. It is nice to calculate it, to be sure that the lower bound is monotonically increasing (the function will produce an error message if F does not increase at every iteration). However, in all the data and scenarios that I have tested in this version (v3.0), F increases monotonically, as it should. Since calculating F may be costly, and we do not need it to learn the model, you can set cFbool to zero, which is the default. Dispopt 2 and 3 are (mostly) redundant if cFbool is false. hypit is an option to specify the iterations at which different hyperparameter optimisations begin, and is a three-component row vector, defaulting to [10 20 30]. These are, respectively, the iterations where algorithm begins dynamics-related optimisations (i.e. alpha and beta hyperparameters), output-related optimisations (i.e. gamma and delta hyperparameters), and the remaining hyperparameters (pa,pb for the noise, and X0m_p and X0ci_p for the hyperparameters of the initial state prior distribution). net is a pre-learnt network, which can be specified if one wants to continue training where the last run left off, or even if you want to train on a new set of observation and input sequences, but starting from the posterior over parameters and hyperparameters settings obtained on the original data set. Note that by specifying this, any specified k is overwritten by that specified in the net, and also the net must be expected to see the same dimensionalities of each of input and observations sequences. OUTPUT structure 'net' The network structure returned is very rich in information, for example, to display the:- o final hyperparam alpha, type net.hyp.alpha o history of the bound F, type net.hist.F (last column!) o cov. of the inputs-to-state matrix's rows, type net.param.SigB ** o posterior mean of the inputs-to-state matrix, type net.exp.B o the expected natural parameters of the model, type net.exp et cetera. Explore around the structure - many good things. ** I would STRONGLY advise you to read chapter 5 of my thesis, which ** explains the code using almost identical symbols and naming ** conventions, and provides algorithms to show you exactly what the code ** is doing. For example, net.param.SigC is *not* the covariance of the C ** matrix rows, as one might naively think --- it is related though. ------------------------------------------------------------------