A Markov decision model for optimising economic production lot size under stochastic demand

Traditional approaches towards determining the economic production lot (EPL) size in manufacturing applications assume deterministic demand, often at a constant rate. In this paper, an optimisation model is developed for determining the EPL size that minimises production and inventory costs of a periodic review production-inventory system under stochastic demand. Adopting such a Markov decision process approach, the states of a Markov chain represent possible states of demand. The decision of whether or not to produce additional inventory units is made using dynamic programming. This approach demonstrates the existence of an optimal state-dependent EPL size, and produces an optimal lot sizing policy, as well as the corresponding total production and inventory costs.


Introduction
Manufacturing industries continually face the challenge of optimising production and inventory levels.This is a considerable challenge when the demand for manufactured items follows a stochastic trend.Two major problems are usually encountered: (i) determining the most desirable period during which to produce additional units of the item in question and (ii) determining the economic production lot (EPL) size corresponding to the best policy in a periodic review production-inventory system when demand is uncertain.
According to Goyal [1], the EPL size may be computed each time a product is scheduled for production when all products are produced on a single machine.The distribution of lot size can be computed when the demand for each product is a stochastic variable with a known distribution.This distribution is independent for non-overlapping time periods and identical for equal time periods.
According to Khouja [2], the EPL size may be determined under conditions of increasing demand, shortages and partial backlogging.In this case the unit production cost is a PK Mubiru function of the production rate and the quality of the production process deteriorates with increased production rate.[3] presented a lot sizing model for a single vendor and single buyer with stochastic demand.They relaxed the assumption of deterministic demand and assumed the lead time to vary linearly with the lot size.The lot size comprises of a lot sizedependent run time and constant delay times, such as moving, waiting and set-up times.Optimality guidelines on the lot sizing policy, total production and inventory costs are, however, not suggested.Kampf and Kochel [4] examined the stochastic lot sizing problem where cost of waiting and lost demand is taken into consideration.The size of the optimal economic lot size is, however, excluded in the formulation of their model.

Daya and Hariga
In this paper, a production-inventory system is considered whose goal is to optimise the EPL size, the lot sizing policy and the total costs associated with production and holding inventory.At the beginning of each period, a major decision has to be made, namely whether to produce additional units of the stocked item or to cancel production and utilise the available units in stock.The paper is organised as follows.After describing the mathematical model in §2, consideration is given in §3 to the process of estimating the model parameters.The model is solved in §4 and applied to a special case study in §5.Some final remarks follow in §6.

Model development
The demand during each time period over a fixed planning horizon is classified as either favourable (denoted by state F) or unfavourable (denoted by state U), and the demand during any such period is assumed to depend on the demand of the preceding period.The transition probabilities over the planning horizon from one demand state to another may be described by means of a Markov chain.Suppose one is interested in determining an optimal course of action, namely to produce additional stock units (a decision denoted by S = 1) or not to produce additional units (a decision denoted by S = 0) during each time period over the planning horizon, where S is a binary decision variable.Optimality is defined such that the lowest expected total production and inventory costs are accumulated at the end of a total of N consecutive time periods spanning the planning horizon under consideration.In this paper, a two-period (N = 2) planning horizon is considered.

Model variables and parameters
Varying demand is modelled by means of a Markov chain with transition matrix , where the entry Q S ij in row i and column j of the transition matrix denotes the probability of a transition in demand from state i ∈ {F, U} to state j ∈ {U, F} under a given lot sizing policy S ∈ {0, 1}.The number of customers observed in the system and the number of units demanded during such a transition is captured by the customer matrix and demand matrix , respectively.Furthermore, denote the number of units in stock and the total (production, inventory and storage) cost during such a transition by the stock matrix and the cost matrix , respectively.Also, denote the expected total future cost and the already accumulated total cost at the end of time period 1 when the demand is in state i ∈ {F, U} for a given lot sizing policy S ∈ {0, 1} by respectively w S i and a S i , and let w S = [w S F , w S U ] T and a S = [a S F , a S U ] T , where "T" denotes matrix transposition.

Finite-period dynamic programming formulation
Recalling that demand can either be in state F or in state U, the problem of finding an EPL size may be expressed as a finite period dynamic programming model.Let c n (i) denote the optimal expected total production and inventory costs accumulated during periods n, n + 1, . . ., N given that the state of the system at the beginning of period n is i ∈ {F, U}.The recursive equation relating c n and c n+1 is together with the final conditions c N +1 (F) = c N +1 (U) = 0.This recursive relationship may be justified by noting that the cumulative total production and inventory costs T S ij + c n+1 (j) resulting from reaching state j ∈ {F, U} at the start of period n + 1 from state i ∈ {F, U} at the start of period n occurs with probability Q S ij .Clearly, where "T" denotes matrix transposition, and hence the dynamic programming recursive equations result, where (4) represents the Markov chain stable state.
3 Computing Q S , T S and p S The demand transition probability from state i ∈ {F, U} to state j ∈ {F, U}, given lot sizing policy S ∈ {0, 1} may be taken as the number of customers observed with demand initially in state i and later with demand changing to state j, divided by the sum of customers over all the states.That is, When demand outweighs on-hand inventory, the cost matrix T S may be computed by means of the relation where c p denotes the unit production cost, c h denotes the unit holding cost and c s denotes the unit storage cost.Therefore, for all i, j ∈ {F, U} and S ∈ {0, 1}.
A justification for expression (6) is that D S ij − Y S ij units must be produced in order to meet the excess demand.Otherwise production is cancelled when the demand is less than or equal to the on-hand inventory.
The EPL size when demand is initially in state i ∈ {F, U}, given lot sizing policy S ∈ {0, 1}, is The following conditions must, however, hold: , and p S i = 0 when D S ij ≤ Y S ij , 2. S = 1 when c p > 0, and S = 0 when c p = 0, 3. c s > 0 when shortages are allowed, and c s = 0 when shortages are not allowed.

Computing an optimal strategy
The optimal EPL sizing strategy is found in this section, for each time period separately.

Optimisation strategy during period 1
When the demand is favourable (i.e. in state F), the optimal lot sizing policy during period 1 is The associated total production and inventory costs and EPL size are then respectively.Similarly, when the demand is unfavourable (i.e. in state U), the optimal lot sizing policy during period 1 is In this case the associated total production and inventory costs and EPL size are

Optimisation strategy during period 2
Using the dynamic programming recursive equation ( 1), and recalling that a S i denotes the already accumulated total cost at the end of period 1 as a result of decisions made during that period, it follows that Therefore, when demand is favourable (i.e. in state F) the optimal lot sizing policy during period 2 is while the associated total production and inventory costs and EPL size are respectively.Similarly, when the demand is unfavourable (i.e. in state U), the optimal lot sizing policy during period 2 is In this case the associated total production and inventory costs and EPL size are respectively.

Case study
In order to demonstrate use of the model in §2-4, a real-case application from Nice House of Plastics, a manufacturer of plastic utensils in Uganda, is presented in this section.Plastic jerry cans are among the products manufactured by this company, and the demand for these cans fluctuates from month to month.The company wants to avoid over-producing when demand is low or under-producing when demand is high, and hence seeks decision support in terms of an optimal lot sizing policy, the associated production-inventory costs and specifically a recommendation as to the EPL size of plastic jerry cans over a two-week period.

Data collection
A sample of 30 customers was used.Past data revealed the following demand pattern and stock levels of jerry cans during the first week of the month when demand was favourable (F) or unfavourable (U): If additional jerry cans are produced during week 1, then the customer matrix, the demand matrix and the stock matrix are given by respectively.In either case the unit production cost (c p ) is $2.00, the unit holding cost per week (c h ) is $0.50, and the unit shortage cost per week (c s ) is $1.00.

Computation of model parameters
Using ( 5) and ( 6), the state transition matrix and the cost matrix for week 1 are

The optimal strategy and EPL size
Since 7.04 < 11.25, it follows that S = 1 is an optimal lot sizing policy for week 1 with associated total production and inventory costs of $7.04 and an EPL size of 40 − 37 = 3 units if demand is favourable.Since 49.95 < 61.43, it follows that S = 0 is an optimal lot sizing policy for week 1 with associated total production and inventory costs of $49.95 and an EPL size of 0 units if demand is unfavourable.
If demand is favourable, then the accumulated production and inventory costs at the end of week 1 are a 1 F = 7.035 + (0.67)(7.035) + (0.33)(49.95) = $28.23, a 0 F = 11.25 + (0.50)(7.035) + (0.50)(49.95) = $39.74.Since 28.23 < 39.74, it follows that S = 1 is an optimal lot sizing policy for week 2 with associated accumulated production and inventory costs of $28.23 and an EPL size of 40 − 37 = 3 units in the case of favourable demand.However, if demand is unfavourable, then the accumulated production and inventory costs at the end of week 1 are Since 85.74 < 104.09, it follows that S = 0 is an optimal lot sizing policy for week 2 with associated accumulated production and inventory costs of $85.74 and an EPL size of 0 units in the case of unfavourable demand.
When shortages are not allowed, the values of S, c n (i) and p S i may be computed for i ∈ {F, U} in a similar fashion after substituting c s = 0 into the matrix function T S = (c p + c h + c s )(D S − Y S ).

Conclusion
A production-inventory model with stochastic demand was presented in this paper.The model determines an optimal lot sizing policy, production-inventory costs and the EPL size of a given product with stochastic demand.The decision of whether or not to produce additional stock units is modelled as a multi-period decision problem using dynamic programming over a finite planning horizon.The working of the model was demonstrated by means of a real case study.
if additional jerry cans are not produced during week 1, these matrices are