Paying in Attention

Search

SearchSearch

Fully Visible Sigmoid Belief Network

Nov 14, 20241 min read

BayesianMachineLearningMachineLearningStatistics The Fully Visible Sigmoid Belief Network (FVSBN) is an autoregressive model where a collection of binary random variables have a joint distribution determined by the following parameterized conditional distributions:

Misplaced &\mathbb{P}(X_1=1; \theta_1) &= \theta_1 \space\space\space \mathbb{P}(X_1=0; \theta_1) = 1-\theta_1 \\ \mathbb{P}(X_2 \space | \space X_1; \theta_2) &= \sigma(\theta_{2,0} + \theta_{2,1} X_1)\\ &\cdots \\ \mathbb{P}(X_N \space | \space X_1, \cdots, X_{N-1}; \theta_N) &= \sigma(\theta_{N,0} + \sum_{k=1}^{N-1} \theta_{N,k} X_k) \end{aligned}$$ where $\sigma$ is the sigmoid/logit function. One can then sample from this distribution sequentially $$\begin{aligned} \tilde{X}_1 &\sim \mathbb{P}(X_1) \\ \tilde{X}_2 &\sim \mathbb{P}(X_2 \space | \space X_1 = \tilde{X}_1) \\ &\cdots \\ \tilde{X}_m &\sim \mathbb{P}(X_{m} \space | \space X_1 = \tilde{X}_1, \cdots, X_{m-1} = \tilde{X}_{m-1}) \end{aligned}$$

Graph View

Backlinks