Towards a fair, thriving Web in the age of AI

In this issue: BON book and site came back to life;  specification drivers; metamorphic testing; keynote on verification and AI

BON site and book

Developed by Kim Waldén and Jean-Marc Nerson, BON, the Business Object Notation, is a simple and powerful formalism and method for describing system architectures, based on ideas of abstraction and Design by Contract closely related to those of Eiffel. Think UML done simpler and better, with only a small set of notations, which users can learn in just a few minutes, to focus on the essentials: the architecture of their systems.

EiffelStudio supports BON diagrams natively, both as output (producing a diagram from Eiffel classes) and as input (generating Eiffel classes from user's graphical BON input.

The approach is described in Waldén's and Nerson's outstanding book, Object-Oriented Software Architecture (Prentice Hall). It was also document in a site that Kim Waldén was maintaining, containing in particular the entire book in PDF.

Kim unfortunately passed away, far too young, and the site became unavailable for a while. We have been able to retrieve it and the pages, including the book, and to make them available (with the kind permission of Kim Waldén's wife Gabrielle and his close collaborator Paul Cohen) on the eiffel.org site, at

bon-method.eiffel.org

The page's design is not fancy but the information is more relevant than ever. Many people had asked what had happened to the BON site and we are happy that it is is now available again.

.

A useful and neglected idea

Object-oriented programming with Design by Contract (in other words, the application to programming of Abstract Data Types) makes it possible to express the formal properties of objects of a certain kind within the description of the corresponding class. In this application it is peerless.

Things are not as naturally easy when we want to talk about several objects. For example, every respectable textbook includes as an example the properties, for a stack s and element x, that

top (push (s, x)) = x

pop (push (s, x)) = s

(if you push an element, the top of the stack is that element, and popping yields the original stack, two properties which do not apply, for example, to queues or priority queues). It is not possible within the OO framework, however, to express that if you pop a stack and push (on the resulting stack) its original top, you get back the stack that you had in the first place. This impossibility comes in part from the multi-object nature of this property.

In a PhD thesis at the University of Toulouse, Alexander Naumchev addressed the problem with an ingenious notion: specification drivers. Like a test driver, a specification driver “talks about” some objects by exercising their operations, but with the aim of specifying their properties formally rather than executing them.

We wrote a paper on it:

Alexander Naumchev and Bertrand Meyer: Complete Contracts Through Specification Drivers, in TASE 2016, 10th International Symposium on Theoretical Aspects of Software Engineering, Shanghai, 17-19 July 2016, IEEE Computer Society, 2016, pages 160-167.

Available at se.ethz.ch/~meyer/publications/methodology/drivers_TASE.pdf

The paper is not idea since it is a bit too academic and more generally the idea has not widely caught on as far as I know, but I believe it deserves to be better known as it extends the OO paradigm to cover in principle any kind of formal specification.

A specification driver is simply a routine, in a class produced expressly for specification purposes, which manipulates the “Classes Under Specification” (by analogy with “Implementation Under Test”), creating instances and calling operations on them, interspersed with formal properties in the form of contract elements (the usual suspects: preconditions, postconditions, loop invariants and variants, class invariants). Actually a specification driver can serve as a test driver (but not the reverse in general).

There are many possible applications. I will only mention one, in the next section.

Metamorphic testing

Unlike specification drivers, it would be unfair to characterize metamorphic testing as a neglected idea; it has attracted considerable attention and led to lots of paper in academia, but seems to remain only sparsely used in industry.

Metamorphic testing follows from the observation that it may be possible to test systems even when you do not have a precise specification of individual operation results: what you test instead is the change in these results when the inputs change. It may be impossible to specify the expected effect of an operation, but possible to specify the effect of changing its arguments.

An example is the testing of a Web search. We cannot easily specify the effect of a search for, say, “Paris population”. But we can specify how the results should change if we add a word (for example “2026” or “adults”): the new results should be a subset of the original. Or if we remove a word. Or if we replace a word by another.

Note how well metamorphic testing lends itself to specification drivers.  As with the contrasting stack-operation examples, the difference with traditional testing can be viewed as the difference between:

• Examining properties from within (specification or testing of operations on an object, against their expected properties).
• Examining properties from their outside (considering several tests and expressing properties of their combination, in the same way we were considering several stacks).

This paradigm seems particularly appropriate for a crucial problem of today's IT scene: how to test AI applications. As in the search case, but on a much larger scale and with much more worrying consequences, there is generally no exact specification of what an AI tool should return. But we can specify properties about these AI mechanisms. I believe metamorphic testing (supported by specification drivers) is poised to become a key tool in that field.

Forthcoming keynote talk

This Thursday, October 26, I am giving the keynote at the annual “Swiss Testing Day”. The event as a whole, largely directed to practitioners, looks attractive, and I like the overall theme, “Defining Quality in a Dangerous Decade” (great title!).

My talk is entitled “Software Verification in the Age of AI”. Here is the event's page.

Cover photo: Coat of Boris Godunov in the production of Mussorgsky's opera, warehouse of the Théâtre du Capitole, Toulouse, France.