Performance Perspectives Blog

Reflections on an old Chinese statistical joke

by | Dec 1, 2009

I’ve mentioned in the past the value I’m seeing in a design book (Measurement, Design and Analysis, by Pedhazur & Schmelkin) I’m reading for a course. The authors reference a book by H. Zeisel (Say it with figures) who “pointed out that, according to an old Chinese statistical joke, the rate of mortality among people who are visited by a doctor is much higher than among those who are not visited by a doctor.”  Reflect for a moment on this joke. Once you get it, think about how this applies to the world of GIPS(R) verifications, when a non-random approach is used.

If the verifier selects only a certain group of composites to review (e.g., “marketed”), might it be quite likely that they will conform with the standards, especially if the firm being verified knows that there’s a greater likelihood of only these being checked?

These non-random verifications can be likened to what Pedhazur & Schmelkin refer to as “quasi-experimental designs,” “that suffer, to a greater or lesser extent, from serious shortcomings and pitfalls…[and] that utmost circumspection be exercised in the interpretation of the results, and in conclusions…based on them.”

Perhaps I’m beginning to sound like a broken record (whatever a “record” is), but by continuing to periodically bring this subject up I am hopeful that the GIPS Verification Subcommittee will take action to come out in opposition to such practices as they are fraught with problems.

Free Subscription!

The Journal of Performance Measurement

The Performance Measurement Resource.

Click to Subscribe