Verification and validation of simulation models

[EddieK]

How can you validate the simulation model after you’ve verified it?

[ewilliams]

Validation requires ongoing and in-depth interaction and communication with the client engineers and managers; it is a much more generally interactive process than verification. If the system being modeled already exists, clearly model results should be compared with current system reality. If the system being modeled does not yet exist, compare model results with a similar system if one is available. Model output should also be checked against expert opinion whenever possible. A particularly valuable validation method in many situations is the Turing Test (named after Alan Turing). In this test, the client is shown model output and output metrics from the existing system -- but not told which is which. If the client cannot tell them apart, the model passes the Turing test. If the client can, the model fails the Turing test, and the modeler next asks "How could you tell?" Example reply: My department always has a big backup behind the milling machine, and the model doesn't." The modeler now has concrete information and clues with which to find and correct the modeling error(s).

[jclark]

Is there a systematic approach that can be used for verification purposes? If so, what does it entail?

[ewilliams]

Good verification techniques are: desk-checking of code and model logic; walkthroughs with colleagues, putting ONE entity into the model and stepping through its progress, temporarily removing ALL randomness from the model and examining time-lapses against spreadsheet calculation, watching the animation closely, noting cases where average queue length approximately equals half of maximum queue length (the queue may be growing without logical bound), and noting whether some parts (e.g., "blocks") of the model are never reached by entities. The reference "Handbook of Simulation" (editor Jerry Banks) contains a comprehensive and somewhat theoretical chapter by Osman Balci on model verification (and validation) techniques.