Moonwalk

Try it

A Moonwalk program describes a generative model that produces a target vector, given parameters and a covariate matrix. Each element of the target vector is assumed to be independent, ie, depends only on the parameters and its corresponding row in the covariate matrix.

These programs are designed to be used as the kernel of a gradient-based optimizer or sampler such as Hamiltonian Monte Carlo.

Example

Here is an example Moonwalk program, to give you the flavor.

data radon_data {
 y <- c(1,2,3)
 county <- c(4,5,6)
 x <- c(7,8,9)
}

model radon targets radon_data.y {
 y[N] in (y_hat[N] +- sigma_y)
  y_hat[N] = a[county[N]] + x[N] * b
  a[_] in (mu_a +- sigma_a)
  mu_a in (0 +- 10)
  b in (0 +- 1)
}

(Compare with the varying intercept radon model in the Stan documentation).

Program Structure

Programs operate on data. Currently, the only way to import data is by declaring it inline, in a data block, eg,

data radon_data {

The block must contain one or more numeric vector literals in R's data dump syntax.

The programs themselves much declare a model name and a target vector by referencing a data block, eg

model radon targets radon_data.y {

The program that follows inside the curly braces is a sequence of variable definitions and bounds that describe how the target vector is produced. Programs proceed "backwards" in the sense that variables are always used before they are defined, and their definitions are always optional.

The idea here is that programs are "complete" right from the first line, in that they are always describing some model to produce the target vector. As they get longer, that model simply gets more sophisticated. This is well suited to interactive, exploratory use with immediate feedback.

Comments

Anything from a "#" to the end of a line is treated as a comment.

Definitions

A definition looks like

variable_name = expression

The variable_name on the left must refer to a variable that has appeared earlier in the program (for example, the target variable).

The expression can introduce new variables just by naming them, and can include standard arithmetic operators, and the log and exp functions.

For example,

a = log(b) + 3

Bounds

Bounds look like

variable_name in (range)

The range can take a number of different forms:

• Continuous ranges use ~, like (-1 ~ 1)
• Discrete ranges use ..., like (0 ... N)
• The bounds can be made "soft" by including a range marker on the outside, like (0 ... x ...). This is understood to mean a discrete variable that is unbounded above, but where x is a typical high value.
• As another example, (~ 5 ~ 10 ~) is a variable that is fully unbounded, but most likely lies between 5 and 10.
• Continuous unbounded ranges that center on a mean or typical value can also be expressed as (a +- b).

As in some of these examples, range specifications may reference (or introduce) new variables.

For any given variable, you can provide either a definition, or bounds, or neither (but not both).

Even when variables have not been explicitly bounded, they often receive implicit bounds from how they are used. For example, using log(x) in a program will implicitly bound x to be positive.

Vectors and Multi-Dimensional Arrays

When you introduce a new variable, you can index it using [] notation, eg

a = b[c]

This implies that:

• c has discrete and finite bounds
• b is a vector with one element for every possible value of c. Put another way, c's range is b's domain.

Later references to b would be required to use a named index on the left hand side of any definition or bounds, eg

b[J] = x[J] * r

Note that we are using that named index to index into another variable, which propagates the implied vector size; that is, x, too, must logically have one element for every possible value of c.

Index names can be anything you choose, but must start with a capital letter or an underscore.

Multi-dimensional arrays can be defined simply by repeating the [] notation, eg, b[c][d].

Targets and Covariates

The target variable can only be bounded, not defined. It is always taken to be a vector whose size is simply the size of the dataset, so that bounds statement will always contain a single index name, eg:

y[N] in (y_hat[N] +- sigma_y)

Columns from the covariate matrix are necessarily vectors of the same size as the target, and will be indexed relative to it (like x[N] in the example program.) You don't need to mark them in any other way, and indeed the same variable can be treated as either a covariate or a parameter depending on which data columns you provide when running the program.

Defaults for underspecified variables

If a variable isn't defined or bounded, and its bounds cannot be otherwise inferred, it will be given default bounds.

• If the variable is discrete, it will be bounded to (0...1) (note: not yet supported)
• If the variable is continuous, it will be an unbounded real.
• In either case, if the variable is a large vector or array, it may be regularized to sit around an auxiliary mean.

Extending models

You can declare a new model that extends an earlier one, like this:

model radon2 extends radon {

The new model will target the same variable as the model it is extending, and will automatically include all of the same definitions, but it can add more, further refining the model.

Extending a model does not affect the original model, and you can have multiple child models that extend a single parent in different ways.