I’ve mentioned in the past the value I’m seeing in a design book (Measurement, Design and Analysis, by Pedhazur & Schmelkin) I’m reading for a course. The authors reference a book by H. Zeisel (Say it with figures) who “pointed out that, according to an old Chinese statistical joke, the rate of mortality among people who are visited by a doctor is much higher than among those who are not visited by a doctor.”  Reflect for a moment on this joke. Once you get it, think about how this applies to the world of GIPS(R) verifications, when a non-random approach is used.

If the verifier selects only a certain group of composites to review (e.g., “marketed”), might it be quite likely that they will conform with the standards, especially if the firm being verified knows that there’s a greater likelihood of only these being checked?

These non-random verifications can be likened to what Pedhazur & Schmelkin refer to as “quasi-experimental designs,” “that suffer, to a greater or lesser extent, from serious shortcomings and pitfalls…[and] that utmost circumspection be exercised in the interpretation of the results, and in conclusions…based on them.”

Perhaps I’m beginning to sound like a broken record (whatever a “record” is), but by continuing to periodically bring this subject up I am hopeful that the GIPS Verification Subcommittee will take action to come out in opposition to such practices as they are fraught with problems.