As part of the probability class I’m taking, I have the option to create a poster to present at the end of the semester. Naturally, the topic must incorporate some aspect of probability, and as I researched it more I became fascinated with the applciations of probability to statistical mechanics. Topics in statistical mechanics also seem to lend themselves well to being simulated computationally, which is another area of interest. All in all, for my topic I ended up choosing one of the most common introductory examples of statistical physics: the Ising Model.
Physics
The Ising Model attributes magnetism in a material to individual magnetic moments on a lattice, each of which can either be spin-up (+1) or spin-down (-1). We can model the overall energy of the material as
$$ E = -\frac{J}{2} \sum_{i,j\in \Lambda} s_{i,j} (s_{i+1,j}+s_{i-1,j}+s_{i,j+1}+s_{i+1,j-1}) $$
We will be working with square lattices only of length $L$, so in the energy formula $\Lambda=L^2$ is the “volume” of the lattice, and each $s_{i,j}$ is a magnetic moment at $i^{th}, j^{th}$ position. Additionally, $J$ is the interaction strength of the spins (which could be a different value for each pair of moments), but for simplicity’s sake we will set it to 1 for all pairs of moments. Furthermore, the energy formula technically has the extra term $\sum_{i}h_i s_i$, where $h_i$ represents a magnetic field at each moment. Again, for the sake of simplicity, we will set this term to zero for every moment. All in all, at the heart of the formula is the fact that the energy of the lattice is the sum of every moment mulitplied by the sum its nearest neighbours. The one-half term handles overcounting that occurs since this equation looks at all possible combinations, not just the unique ones that we care about.
A configuration of the lattice is an assignment of +1 or -1 to every moment of the lattice. Thus, for an $L$x$L$ lattice there are $2^{L^2}$ possible configurations. We represent a configuration as $c$ and the energy corresponding to $c$ as $E(c)$. We’re interested in observing phase transitions in the material using magnetization as an observable. This next part will feel like a lot of hand-waving (because there is on my part), but we can calculate an observable using the formula
$$ \langle M \rangle = \frac{1}{Z(\beta)} \sum_{c}O(c)exp[-\beta E(c)] $$
$\beta = \frac{1}{k_B T}$, where $k_B$ is Boltzmann’s constant and $T$ is the temperature. Furthermore, in the preceding equation $Z(\beta) = \sum{c} exp[-\beta E(c)]$ is the canonical partition function. I myself am still figuring out why this equation is useful and how it is derived, but from what I understand it acts as a normalizing constant in the equation for the observable. The important aspect to note is that the equation for the observable sums across all possible configurations of the lattice, which we’ve already noted is $2^{L^2}$, a prohibitively large number for anything but the smallest of lattices. Therefore, we need a way to generate a smaller number of configurations that will give us a good approximation.
Simulation
Monte Carlo simulations is essentially just a name given to any algorithm that simulates a process via random sampling. Examples including modeling investment portfolio returns or the spread of disease. You can even approximate integrals using a Monte Carlo method:
$$ \int_{A}f(x)dx \approx \frac{1}{N} \sum_{i=1}^{N} f(x_i) $$
First, we need to briefly describe what a Markov chain is: it’s basically a description of possible events in which the probability of the next event is only dependent on the current state. In other words, it is a memoryless process. In terms of simulating the Ising model, P(c \rightarrow c’) is the probability that we move from the current configuration $c$ to some configuration $c’$; note how it only depends on where we currently are in the process. If we do a memoryless random walk through the possible configurations, the Markov chain has the special property that the probability of being in configuration $c$ is \pi (c), the stationary distribution. Thus, we’ll construct a Markov chain with the stationary distribution $\pi(c) = exp(-\beta E(c))$.
The Metropolis-Hastings Algorithm provides a way to accomplish just this. It’s a three step process:
- Start a configuration $c$
- Propose a move to a new configuration $c’$ with probability $P(c \rightarrow c’)$
- Accept the move to $c’$ with probability $\textbf{min}(1, \frac{\pi (c’)}{\pi (c)} \frac{P(c’ \rightarrow c)}{P(c \rightarrow c’)})$
In the Metropolis-Hastings Algorithm we we propose moves that are equiprobable both ways, $P(c \rightarrow c’) = P(c’ \rightarrow c)$. We accomplish this by changing one single spin of the existing configuration. I also find it helpful to think about the probability of moving to the new configuration in terms of energy. Because the transition probabilities are the same they simply reduce to 1, and we’ve seen already that $\pi(c’) = exp(-\beta E(c’))$ and $\pi(c) = exp(-\beta E(c))$. Thus, $\frac{\pi (c’)}{\pi (c)} =\frac{exp(-\beta E(c’))}{exp(-\beta E(c))} = exp(-\beta \delta E)$. If $E(c’) < E(c) \rightarrow exp(-\beta \delta E) > 1$. What this tells us is that if the proposed configuration is of lower energy than the current configuration, we unconditioanlly move to it, which makes physical sense as systems always try to move into states of lower energy in the absence of energy sources. If $\frac{\pi (c’)}{\pi (c)} < 0$ we accept the move with probability $exp(- \beta \delta E)$, so that we’re sampling from the probability distribution of our configurations.
Here, each $x_i$ is a uniformly distributed randomly chosen sample in the domain $A$, and as $N \rightarrow \infty$ the approximation converges to the actual solution. So the problem we’re trying to solve is how to best choose random configurations of the lattice that will give us a good approximation of the material’s behavior. Since each configuration of the lattice appears with probability $P(c) = exp(-\beta E(c))$ what we’d like to have is a way to generate configurations with such a probability.
Implementation
So what does it look like to computationally perform the Metropolis-Hastings algorithm? Well, I’ve written a mediocre C++ program to explore this question. Because the code is fairly complex (at least for me), the full code can be found here, but I’ll just break down the interesting parts.
for (double b = betaLower; b <= betaUpper; b+= betaStep)
{
magnetArray = {};
myGen.setInitialConfig();
for(size_t n = 0; n < N; ++n)
{
magnet = myGen.getCurrMag();
magnetArray.push_back(magnet);
for (size_t l = 0; l < volume; ++l)
{
rand1 = engine() % volume;
rand2 = static_cast<double>(engine()) / RAND_MAX;
myGen.updateLattice(b, rand1, rand2);
}
}
}
In this code we’re looping through a collection of $\beta$ values, effectively testing the lattice’s behavior at different temperatures. Before we begin sampling new configurations, we set our lattice to a cold start (every spin is either +1 or -1) using the line myGen.setInitialConfig(); where myGen is an instance of the class Ising that contains methods for simulating and measuring the Ising model. We then enter the configuration loop, where for each $\beta$ value we sample 10,000 configurations, which is a number large enough for our generated configurations to be largely noncorrelated. At the beginning of each pass through the loop we also store the magnetization of the previous configuration:
$$ M(c) = \frac{1}{L^2} \sum_{i,j \in \Lambda} s_{i,j} $$
The method updateLattice() is then called:
void Ising::updateLattice(double currentBeta, int rand1, double rand2)
{
int oppositeSpin = -1 * latticeSpin[rand1];
int neighbourSum = latticeSpin[neighbourArray[rand1][0]] + latticeSpin[neighbourArray[rand1][1]]
+ latticeSpin[neighbourArray[rand1][2]] + latticeSpin[neighbourArray[rand1][3]];
int deltaEnergy = latticeSpin[rand1] * neighbourSum * 2;
if (exp(-1 * deltaEnergy * currentBeta) > rand2)
{
latticeSpin[rand1] = oppositeSpin;
currMag += 2 * oppositeSpin;
}
}
We start by flipping a random spin of our lattice. We then need to calculate the energy difference between our current configuration and the proposed configuration. You could sum through all the magnetizations of both the original and proposed lattices and take their difference, but that would be extraordinarily time intensive. Instead, we remember that each new configuration changes the spin of only one moment, meaning the change is only local to that moment and its nearest neighbours. Thus we are able to derive the expression $\delta E = \frac{1}{2} s_{i,j} (s_{i+1,j} + s_{i-1,j} + s_{i,j+1} + s_{i,j-1})$, assuming the simplifications we made at the beginning. We then check if our acceptance conditions are met, and if it is we accept the move. Notice also that because the magnetization changes by only one spin we are able to keep a running tally of the magnetization of the material, instead of having to recompute the sum of the entire lattice every new configuration.
It’s also worth noting how the lattice is stored. Instead of creating an $LxL$ array, we can store it in a one-dimensional array by using a super-index, where every $i^{th}, j^{th}$ moment is stored at the $I = i + Lj$ index. The lattice also implements periodic boundary conditions, meaning moments at the edges are treated as neighbours to moments at opposite edge. To make accessing neighbours easier, we implement the array neighbourArray, which is of dimensions $L^2 x 4$. The row indeces correspond to the super-index we’re at on our lattice, and the four “column” indeces correspond to the superindeces of the four nearest neighbours. That’s implemented at the beginning of the algorithm with this method:
void Ising::findNeighbours()
{
for (int superI = 0; superI < L*L; superI++)
{
if (superI - 1 >= 0) neighbourArray[superI][0] = superI - 1;
else neighbourArray[superI][0] = superI + (L-1);
if (superI + L < L*L) neighbourArray[superI][1] = superI + L;
else neighbourArray[superI][1] = superI - (L*(L-1));
if (superI + 1 < L*L) neighbourArray[superI][2] = superI + 1;
else neighbourArray[superI][2] = superI - (L-1);
if (superI - L >= 0) neighbourArray[superI][3] = superI - L;
else neighbourArray[superI][3] = superI + L*(L-1);
}
}
Finally, after each pass through our configurations loop we output the magnetization of the lattice inside a file. Thus, for each $\beta$ value we will have a file with $N$ magnetizations corresponding to each of our $N$ proposed configurations.
Measurements and Results
An important preliminary to mention is that before we begin making measurements based on the magnetizations we recorded, we wait a certain number of steps so that our magnetization values settle around their equilibrium. If we don’t do this, our measurements will be off.
We are interested in taking four particular measurements of our model:
- Mean magnetization: $m = \frac{1}{N} \sum_{n=1}^{N} M_n$ Basically we’re just taking the averages of the magnetizations produced by the Metropolis-Hastings algorithm at a certain temperature.
- Susceptibility: $\chi = \mu_2 V$. This is a measure of the variance amongst our magnetization values.
- Skewness: $B_3 = \frac{\mu_3}{(\mu_2)^\frac{3}{2}}$. At my level of understanding this measurement doesn’t mean much, but it essentially gives information about the symmetry of the distribution of the magnetizations.
- Kurtosis: $B_4 = \frac{\mu_4}{(\mu_2)^2}$. Again, at my level of understanding I’m not able to gain much insight out of this measurement yet. It was mostly included for thoroughness.
In the measurement definitions each $\mu_k = \frac{1}{N} \sum_{n=1}^{N} (M_n - m)^k$ Measuring these in the code is fairly trivial, but what isn’t trivial is the results:
We see that there is a sudden transition occuring in $m$ close to $\beta = 0.45$ and a discontinuity in $\chi$ at around the same point. Even in the skewness and kurtosis plots we can see irregular behavior at this point. From the definition of the mean. Basically what we’re is seeing is that for $\beta < ~0.45$ there is a roughly equal number of moments that are spin-up and spin-down. However, as we pass through $\beta = ~0.45$ the spin-up moments become dominant, until they near instantaneously are the primary spin amongst the lattice. Remember that we start the lattice at each $\beta$ in a cold start, which in the case of this particular graph was a cold start at spin-up, or +1. Also remember that $\beta$ is inversely proportional to temperature. Thus, what we’re seeing is that at high temperatures the moments equally distributed between spin-up and spin-down, but as temperature decreases we hit a phase transition where the moments hardly change from where they were set in the cold start. This also makes physical sense: moments can’t just change spins randomly because that requires energy. So as the temperature decreases, representing less energy in the form of heat available to the lattice, the moments primarily stay in their initial states.
I’ve made a gif of the lattice’s simulated behavior around the phase transition. Notice how there are more spin-up moments on average. It’s also interesting to note how moments of a certain spin tend to coagulate around each other.
Note and Acknowledgement
I should note that in I’ve been implicitly using unitless quantities during this discussion, where our unitless variables are $\hat{\beta} = J \beta$ and $\hat{E} = \frac{E}{J}$. But as we simplified $J$ to $1$, this detail wasn’t especially important. Also, I should acknowledge some resources I used to help me with the implementation of the code and my own understanding of simulating the Ising model. First is an assignment from Goethe University in Frankfurt, Germany, and the other being a online resource for a computational physics class at the University of Illinois Urbana Champaign, each of which has been respectively linked.
]]>
