## Definition of Markov's Model The Markov model is a mathematical model used in [[product framework]] to forecast the probability of users transitioning from one state to another within a product or service. It is based on probability theory and assumes that the transition probability depends only on the current state and is independent of previous states. The Markov model is used to determine the effectiveness of different marketing and [[product strategy]] as well as to forecast future sales. ## How Markov's Model Works Here's an example of how the Markov model can be used in product management: Suppose a company has a mobile app that offers a free trial period and then requires a paid subscription to access premium features. The company wants to understand how many users are likely to convert from the free trial to the paid subscription, so they use the Markov model to analyze user behavior. They track the user journey through the app and define the following states: - State 1: User signs up for the free trial - State 2: User uses the app during the free trial - State 3: User cancels the free trial - State 4: User subscribes to the paid version They also collect data on the transition probabilities between the states, based on historical user behavior. > For example, they might find that 50% of users who sign up for the free trial move to state 2, while 30% cancel the trial and 20% subscribe to the paid version. Using the Markov model, they can calculate the probability of a user moving from one state to another over time, and use that information to forecast the number of users who are likely to convert to paid subscribers. This can help them optimize their marketing and product strategies to increase conversion rates and revenue. ## Types of Markov Models ### Discrete-Time Markov Chains This is the simplest and most commonly used type of Markov model. It models a system that changes states at discrete points in time, and the transition probabilities are constant over time. ### Continuous-Time Markov Chains This type of Markov model is used to model systems that change states continuously over time, such as a chemical reaction. The transition probabilities can vary over time. ### Hidden Markov Models This type of Markov model is used to model systems where the states cannot be directly observed, but can only be inferred from the observations. It is often used in speech recognition and natural language processing. ### Markov Decision Processes This type of Markov model is used to model decision-making processes, where the outcome of a decision depends on the current state and the actions taken. It is often used in artificial intelligence and robotics. Each type of Markov model has its own unique characteristics and applications, and the choice of model depends on the specific problem being addressed. ### Semi-Markov Models This is a generalization of the Markov model where the time spent in each state is not necessarily exponentially distributed. Semi-Markov models are often used in reliability engineering and queueing theory. ### Markov Chain Monte Carlo Methods This is a class of algorithms that use Markov chains to obtain samples from a probability distribution. Markov Chain Monte Carlo methods are widely used in Bayesian statistics and machine learning. ### Hidden Semi-Markov Models This is a combination of the Hidden Markov Model and the Semi-Markov Model. Hidden Semi-Markov Models are used to model systems where the states cannot be observed directly and the time spent in each state is not necessarily exponentially distributed. ### Markov Chain Network Models This is a type of Markov model that is used to model complex systems with multiple interacting components. Markov Chain Network Models are often used in epidemiology and ecology. ## Markov's Model Cases ### Dualingvo app Markov's Model Case The Growth Model is a series of metrics we developed to jump-start our growth strategy with data. It is a [Markov Model](https://en.wikipedia.org/wiki/Markov_model?ref=blog.duolingo.com) that breaks down topline metrics (like DAU) into smaller user segments that are still meaningful to our business. To do this, we classify all Duolingo learners (past or present) into an activity state each day, and monitor rates of transition between states. These transition probabilities are monitored as retention rates (e.g., NURR or New User Retention Rate), “deactivation” rates (e.g., Monthly Active User, or MAU, loss rate), and “activation” rates (e.g., reactivation rate). ![[Pasted image 20230410103820.png]] The model above classifies users into 7 mutually-exclusive user states: - **New users**: learners who are experiencing Duolingo for the first time ever - **Current users:** learners active today, who were also active in the past week - **Reactivated users:** learners active today, who were also active in the past month (but not the past week) - **Resurrected users:** learners active today, who were last active >30 days ago - **At-risk Weekly Active Users:** learners who have been active within the past week, but not today - **At-risk Monthly Active Users**: learners who were active within the past month, but not the past week - **Dormant Users:** learners who have been inactive for at least 30 days ## Markov's Model Releated Articles - https://www.productschool.com/blog/product-management-2/markov-model-in-product-management/ - https://en.wikipedia.org/wiki/Markov_model