The Responsible Designer

Exactly?!


https://imgs.xkcd.com/comics/2_2x.png
Exactitude is one form of software model rigor.

I like the way the word exactitude sounds. But it's really just a synonym for exactness.

When your software performs calculations based on physical laws or mathematical formulas exactitude isn't arbitrary. Exactitude is also highly desirable when performing financial transactions or calibrating an oscilloscope (I’ve modeled both).

But what about other situations? How exact does your software need to be and at what cost? The amount of exactitude needed isn’t always clear or obvious. Is it really needed, or is merely something that’s desired (if it is easy enough to do)?

One form of exactitude we are often asked to model in our software are processes to determine definitive answers to non-trivial questions. Such as, “Are you really who you say you are?” or, “Are there sufficient resources?” or, “Will the prescribed treatment be effective?”

There rarely are definitive answers to difficult questions like these. To gain traction on understanding how much exactitude is needed, consider adding more precision around the question your model is trying to answer (and how confident you need to be with likely answers). Instead of seeking certitude, strive to model processes that will provide answers that are “good enough” for your purposes.

Let’s take a naive cut on authentication to illustrate something that is difficult to ascertain with certainty. Well, how might we go about determining whether some individual or some system is “genuine” or “real” (or in most cases, genuine enough) to access some functionality of our system?

There are several synonyms for the verb authenticate, with slightly different meanings:

Each of these synonyms implies different modeling approaches to determine some degree of authenticity. Consider these examples:

The user is asked to login by entering their username and password. What is the software trying to do? It is trying to confirm that the information provided is known to the system by verifying the name and password entered against a (usually encrypted) digital record.

The user is asked to answer a secret question. The software then attempts to corroborate their identity by matching their answer to one they gave earlier (Just one more check that someone is more likely to be who they claim to be if they can remember their initial answer).

The user is asked to correctly identify a captcha. The software is simply attempting to substantiate that they are not a bot. Bots are getting smarter, and captcha mechanisms are getting more challenging/annoying as a consequence. (Another primary use of captcha information is to supply a consistent set of training data so AI models can be tuned to more accurately recognize patterns, but that is a topic for another day.)

At various times our software is likely to perform multiple forms of authentication. Confirming, corroborating, substantiating, verifying, or validating are all different ways to attest to the authenticity of some one or some thing. Definitive answers will strengthen claims of authenticity. But they won't guarantee authenticity.

If our software needs more rigorous authentication mechanisms, we can always turn to more comprehensive solutions—such as those provided by vendors of facial recognition or biometric systems. For example, Intel with its RealSense ID product takes a much more rigorous approach. It claims that people will be authenticated quickly with less that 1 in a million false positives and a 99.76 per cent true acceptance rate. I worry about that 0.24 per cent who fail to be identified. Out of 10,000 people trying to enter a building, 24 will be excluded. But I guess that’s as good as they can do. But who says that's good enough?

The point is, we can be more or less rigorous about authenticating users (or any other difficult question), depending on our system’s actual needs. Pinning down real needs can be hard. So, sometimes we need to create models in order to expose the limits of proposed rigorous solutions. If we do, we'll be more informed when we decide how much effort to expend in order to obtain reasonably rigorous answers.