Department of Mathematical Sciences, Unit Catalogue 2009/10 |
MA20036: Stochastic processes |
 Credits:
| 6 |
| Level:
| Intermediate |
| Period: | Semester 2 |
| Assessment:
| EX 100% |
| Supplementary Assessment: | MA20036 Mandatory extra work (where allowed by programme regulations) |
| Requisites:
| Before taking this unit you must take MA10003 and take MA10005 and take MA20034 |
| Description: | Aims & Learning Objectives: Aims: To present a formal description of Markov chains and Markov processes, their qualitative properties and ergodic theory. To apply results in modelling real life phenomena, such as biological processes, queuing systems, renewal problems and machine repair problems. Objectives: On completing the course, students should be able to * Classify the states of a Markov chain, find hitting probabilities, expected hitting times and invariant distributions * Calculate waiting time distributions, transition probabilities and limiting behaviour of various Markov processes. Content: Markov chains with discrete states in discrete time: Examples, including random walks. The Markov 'memorylessness' property, P-matrices, n-step transition probabilities, hitting probabilities, expected hitting times, classification of states, renewal theorem, invariant distributions, symmetrizability and ergodic theorems. Markov processes with discrete states in continuous time: Examples, including Poisson processes, birth & death processes and various types of Markovian queues. Q-matrices, resolvents, waiting time distributions, equilibrium distributions and ergodicity. |