My first visit ever to Times Square was in November 2008. I’d just attended the Web 2.0 Summit in San Francisco and was making my way back to South Africa. Like everyone seeing the circus of light in Times Square for the first time, I was super impressed. I paid specific notice to all the tech companies and took photos of their billboards — MySpace, Yahoo! Wow, to have that kind of marketing budget. Jerry Yang and Chris DeWolfe were both on stage at Web 2.0 and very confident about the future. Even though the US economy was in serious trouble, companies like these would surely weather the storm.

I was the CTO of Mxit at the time and we were the biggest mobile instant messenger in the world. Our user-base was limited largely to users in South Africa and Indonesia. It wasn’t gaining traction in the US at all — Americans barely used SMS — and folks at Web 2.0 nodded politely and didn’t understand mobile instant messaging much. That we had chat bots and monetized through in-app purchases of virtual currency no one understood. It seemed mobile IM was only ever going to be a niche thing for emerging markets with high SMS costs.

My photos of MySpace and Yahoo billboards, Times Square, New York 2008

Ok, so strictly speaking mad cow is a bovine disease not affecting the Equus genus of horses, donkeys and zebras — which would supposedly include unicorns. Still, the craziness of it describes how MySpace wiped out half a billion dollars of NewsCorp investment, pocket change compared to the subsequent mad cow value destruction at Yahoo. More fantastically WhatsApp (and WeChat and Line specifically), are just smartphone evolutions of a mobile business model we pioneered at Mxit. Their combined value is likely in the order of $100 billion. Mxit on the other hand — dead. The company shut down in late 2015. Wouldn’t it have been valuable to be able to make at least some type of prediction of where these tech companies, and mobile social networks in particular, were headed?

I wanted to solve this problem for myself so I enrolled at MIT as a Sloan Fellow where I spent a year focusing on digital strategy and the powerful modeling framework called “system dynamics”. System dynamics modeling is widely used by scenario planners in capital intensive long-term industries like oil-and-gas and mining to simulate complex problems, and for making forecasts in complex industries. Prof John Sterman at MIT, the leading authority on system dynamics, is famous for making climate change predictions with it.

Futhermore, the diffusion of products into society has been well studied, and mostly follows a predictable pattern. The research has its basis in the study of epidemiology and technology adoption follows the same basic laws as viral infections. Like Ebola, fads like Pokemon Go are very intense but over relatively quickly. The more sustainable products like Facebook act more like HIV, with users remaining “infected” for a long time. (They’re both terrible diseases and on a purely biological level one has to appreciate how effective they unfortunately are.)

Using system dynamics techniques one can model these diffusion dynamics and make useful forecasts of adoption and sustainability. The downside is that these techniques take a little time to master. Additionally the modeling software is not very user-friendly. The combined effect is that system dynamics itself hasn’t been widely adopted. But I’m going to explain the basics here and share some tools and further resources so you can do it yourself. (Bear with me for 5 minutes  — you might save yourself a couple of million!)

“Pieter created a growth model for our business in January of 2014, and as we approach the end of 2014, the model is on target. Impressively the model also predicted certain dramatic outcomes for some of our competitors, and those predictions have held true.” — Brett Loubser, WeChat Africa

The basics of System Dynamics

System dynamics is a methodology and mathematical modeling technique for framing, understanding, and discussing complex issues and problems. Originally developed at MIT’s Sloan School in the 1950s to help corporate managers improve their understanding of industrial processes, system dynamics is currently being used throughout the public and private sector for policy analysis and design.

Sytem dynamics employs two basic concepts: (1) a Stock or Reservoir (symbolized by a rectangle), representing an accumulation of anything (users/capital/goods), and (2) a Flow (symbolized by lines) representing movement of anything (user churn/capital expenses).

Together stocks and flows form cause and effect loops that show how problems develop over time and what the likely consequences of proposed solutions are.

So lets say you have users on a social networking plaform, but they’re churning out because they’re loosing interest in your platform. If you think of a bucket leaking water, the active users would be the water in the bucket (the stock), and the leaks would be users/month churning out (the flow).

In a system dynamics model the bucket and its combined leaks would look like the diagram below, with the water in the bucket shown as a block (stock), and the leaks as an arrow (the flow) dissipating into the little cloud of nothingness. The two triangles is a symbol for a “valve” regulating a flow — think of it as a faucet.

Now you’ve probably experienced that a full bucket leaks at a much higher rate than one that is nearly empty. So the rate of the leak is dependent on the remaining amount of water in the bucket. As the water empties, the leak weakens. This is the “dynamic” nature of system dynamics and in a complex system with many such interconnected “buckets” it leads to non-linear effects which are impossible to anticipate unless you model them.

The total leakage would therefore be a function of two things — how much water remains in the bucket, and the size (and number) of holes. These two variables that influence the rate of leak are shown by the blue arrows.

With that you have the basics of a system dynamics model as created in a software package called Vensim, and if you started with 1 liter and a small leak, a simulation of the remaining water in the bucket would look something like this:

Now lets look at viral product dynamics.

The Bass diffusion model

Developed by Frank Bass in 1969, the Bass diffusion model is a differential equation that describes the process of how new products get adopted in a population. It’s widely used in new products’ sales forecasting and technology forecasting. Bass states that there are two ways to convert consumers to use a product — through advertising, and through word-of-mouth.

A typical strategy is to launch with an advertising campaign to make consumers aware of the product and then hopefully positive word-of-mouth carries it from there. Advertising primes the pumps but word-of-mouth has to take over and ends up having a much bigger effect. (Think of how this went for the iPhone for example — Steve’s launch, lots of press, and then everyone raving about it. It was your techie friend proudly showing you the iPhone he queued for that finally sold you on it). Nothing new there, but with Bass it’s possible to quantify advertising effectiveness and word-of-mouth’s impact using only three variables.

Bass’ model assumes that once you’re a user, you stay one. Most mobile applications however have very high churn, and the average app looses 90% of its daily active users within the first 30 days. So next we have to make some approximation of churn.

Little’s Law and average engagement time

Ultimately there are only two things that matter for any business: how fast customers are arriving, and how long they stay. Bass showed us how fast they arrive. Little’s Law gives us a basic approximation of how long they’ll stay. John Little was an MIT professor in operations research and worked on queueing theory. Queueing theory tackles problems within the context of the following flow in a store:

Arrival –> Service –> Departure

Little’s Law then states that the long-term average number of customers in a stable system L is equal to the long-term average effective arrival rate, λ, multiplied by the average time a customer spends in the system, W; or expressed algebraically:

L = λW

If you think of a Starbucks shop you can determine how much time, W, you’re going to spend in the queue, by taking the average number of people in the queue, L and dividing by the number of new arrivals in a given period of time, λ. Assume we notice that there are on average 12 customers in the queue and at the counter. With an arrival rate of 100 new people per hour, customers must be spending 7 minutes (12/100 =0.12 hour) on average getting their coffee.

What we’re looking for as a very basic approximation of churn on a mobile social network is the average amount of time users spend being active on the platform. You can approximate that with Little’s Law by taking the average monthly active users (MAU) and dividing by the average monthly new registered users (NRUs). So for a fictional platform shown below, you’ll get 2.5 months on average.

NOTE: This is a basic approximation. See NOTE 8 for a more in-depth discussion of retention modeling.

Putting it all together

Now think of a basic model of product adoption as three buckets on top of each other, each “leaking” down into the next one. You’ll have potential adopters — lets say everyone with a smartphone in the US — being converted to active users through advertising and word-of-mouth, and then after some time churning out, still having the app on their phone but no longer using it.

So now you need to know only four variables— how many potential adopters you have (M), a measure of advertising effectiveness (P), a word-of-mouth coefficient (Q), and the average time of use (T).

Potential adopters (M) you get through market studies. We already approximated average time of use (T) with Little’s Law. There’s some average values for advertising effectiveness (P) and word-of-mouth (Q) in the literature, but you could also find them through running regressions of actual data.

For a fairly successful mobile social network I’ve found these values to be around P=0.05, Q=0.35. Given that for most mobile apps and social games the bulk of users churn out after 2–3 months, make the average time a generous 2.5 months. If you turn the diagram above on its side you get the system dynamics model below. (You can download the model from my website and the Vensim tool from Ventana Systems and play with it yourself. See NOTES)

If you take an app like Pokemon Go it clearly has some extreme virality due to the novelty of augmented reality, helped by the fact that it was released at the perfect time — at the height of summer holidays in the US when millions of teens are looking for something to do. Now with roughly 90MM smartphone owners in the US for ages 13–35, let’s assume 70MM are susceptible to try Pokemon Go. With all the press around it, the advertising exposure is probably tripple normal at least, so make that P=0.15. Virality is some of the best ever, so let’s make that six times the average (Q=~2.0). Cross-checking the first weeks’ downloads to the model, that seems to hold.

What you get is massive downloads that will quickly burn through the entire addressable market. Most likely by the end of summer the downloads will plummet in the US. Once everyone has downloaded the app, that’s it. The falacy of so much current reporting around Pokemon Go is that the success of the app is being measured by downloads — huge mistake.

Casual gaming traditionally has fairly bad retention, with users quickly loosing interest after a couple of weeks. This is combined by the fact that these apps compete for attention and time. Right now there’s quite a lot of that available given that it’s school holidays, so we’ll make the generous assumption of 2.5 months of average use. This all changes dramatically after Labour Day when school start again so we reduce it down to one month from that point forward. When you then look at projected monthly active users (MAU) in the US, the following will happen.

Given all the hype, everyone is going to try this out, pushing up downloads and supposed active users (remember, everyone trying it out once is going to be counted as an active user initially). Combined with the effect of summer holidays coming to an end things will turn after Labor Day and look quite different by the end of October. You’re going to see a huge surge, followed by a massive drop towards the end of Fall. Pokemon Go will have to make an impossible amount of money in just a few months to justify adding $9B to Nintendo’s valuation — mad cow strikes again!

But the market loves a good story more than reality

If you walked around Times Square recently you could see overwhelmingly large yellow billboards. Just as I stood in awe at MySpace and Yahoo’s displays in 2008, millions of people would find it inconceivable that a company like SnapChat could be out of business within five years.

It seems that despite the huge valuations, very little analysis actually goes into the underlying dynamics of “unicorns” like SnapChat. Not that that’s unique — you only have to see “The Big Short” to know that very little analysis went into sub-prime mortgages too and look where that got us!

Even basic models highlight the fundamental health of many of these platforms. SXSW in 2015 saw a mad scramble of investors trying to get in on the Meerkat action — dead within months. The craze over Pokemon Go and the 50% rise in Nintendo’s stock price smacks again of a lack of analysis. But why let facts stand in the way of a good story. Silicon Valley loves hype and Wall Street loves short-term thinking. As in “The Big Short”, more eccentric Michael Burrys can only make the world a saner place.

The US Supreme Court’s Justice Stewart famously said of his test for obscenity “I know it when I see it”. For investors and VCs it’s perhaps time to rely less on gut and be a little more dilligent when it comes to spotting unicorns. Applying some of the MIT-style quantitative thinking may just save you a billion.

***

Notes and Resources:

1. Thank you to Arthur Goldstuck, Toby Shapshak and Bruce Love who reviewed the draft and provided valuable feedback.
2. This was originally posted to my blog at www.pieternel.net
3. John Sterman wrote the definitive book on the subject: Business Dynamics. Viral models are discussed in detail in chapter 9.
4. My basic model can be downloaded from my blog. You’ll also find the Pokemon specific model there. You’ll also need Ventana’s free download of Vensim 6.4 to view and run the simulations.
5. I highly recommend MIT Executive Education’s course on Understanding and Solving Complex Business Problems, normally presented by John Sterman himself. You’ll be extensively exposed to system dynamics and modelling with Vensim during this course.
6. A course on system dynamics is available on MIT Open Courseware
7. You can read more about Little’s Law and its impact on startups here.
8. NOTE: this part of the model is a very basic approximation. Retention is the biggest challenge for most mobile startups. This side of the model can be expanded by modelling standard retention curves using D1, D7, D30 retention numbers. However my purpose is to give a basic demonstration so as not to overcomplicate things. An average time of two months is decent for something like casual mobile gaming and probably what one should expect for something like Pokemon Go. Also, the notion of “average” is a dangerous concept in social platforms where almost all behavior follows power law distributions rather than normal (Guassian) ones.
9. Further model expansions include: a) allowing for the size of potential adopters to grow over time b) new adopters are much more viral than long-term users — split the active userbase into cohorts and assign diffirent levels of virality to them, c) allow churned out users to be reactivated into active users.
10. A note on the Bass model’s word-of-mouth coefficients. The coefficient Q consists in the literature of two variables “c” and “i” that are multiplied to give Q. Ie: Q = c*i. “c” is the contact rate — the rate at which newly “infected” users come into contact with “potential users” in a given period of time. “i” is the probably of infecting the user after contact. These two variables are seperated out in the Vensim model. Their product will give you the “Q” value discussed above.
11. Finally, read everything by Nassim Taleb 🙂