“Object Success” now available

A full, free online version of Object Success
(1995)

success_cover

 

I am continuing the process of releasing some of my earlier books. Already available: Introduction to the Theory of Programming Languages (see here) and Object-Oriented Software Construction, 2nd edition (see here). The latest addition is Object Success, a book that introduced object technology to managers and more generally emphasized the management and organizational consequences of OO ideas.

The text (3.3 MB) is available here for download.

Copyright notice: The text is not in the public domain. It is copyrighted material (© Bertrand Meyer, 1995, 2023), made available free of charge on the Web for the convenience of readers, with the permission of the original publisher (Prentice Hall, now Pearson Education, Inc.). You are not permitted to copy it or redistribute it. Please refer others to the present version at bertrandmeyer.com/success.

(Please do not bookmark or share the above download link as it may change, but use the present page: https:/bertrandmeyer.com/success.) The text is republished identically, with minor reformatting and addition of some color. (There is only one actual change, a mention of the evolution of hardware resources, on page 136, plus a reference to a later book added to a bibliography section on page 103.) This electronic version is fully hyperlinked: clicking entries in the table of contents and index, and any element in dark red such as the page number above, will take you to the corresponding place in the text.

The book is a presentation of object technology for managers and a discussion of management issues of modern projects. While it is almost three decades old and inevitably contains some observations that will sound naïve  by today’s standards, I feel  it retains some of its value. Note in particular:

  • The introduction of a number of principles that went radically against conventional software engineering wisdom and were later included in agile methods. See Agile! The Good, the Hype and the Ugly, Springer, 2014, book page at agile.ethz.ch.
  • As an important example, the emphasis on the primacy of code. Numerous occurrences of the argument throughout the text. (Also, warnings about over-emphasizing analysis, design and other products, although unlike “lean development” the text definitely does not consider them to be “waste”. See the “bubbles and arrows of outrageous fortune”, page 80.)
  • In the same vein, the emphasis on incremental development.
  • Yet another agile-before-agile principle: Less-Is-More principle (in “CRISIS REMEDY”, page 133).
  • An analysis of the role of managers (chapters 7 to 9) which remains largely applicable, and I believe more realistic than the agile literature’s reductionist view of managers.
  • A systematic analysis of what “prototyping” means for software (chapter 4), distinguishing between desirable and less good forms.
  • Advice on how to salvage projects undergoing difficulties or crises (chapters 7 and 9).
  • A concise exposition of OO concepts (chapter 1 and appendix).
  • A systematic discussion of software lifecycle models (chapter 3), including the “cluster model”. See new developments on this topic in my recent “Handbook of Requirements and Business Analysis”, Springer, 2022, book page at bertrandmeyer.com/requirements.
  • More generally, important principles from which managers (and developers) can benefit today just as much as at the time of publication.

The download link again (3.3 MB): here it is.

The mathematics of the seven messengers

In my previous article I referred to the short story The Seven Messengers by Dino Buzzati, of which I have written a translation. Here is a quantitative analysis. I will also refer the reader to a very nice article published in 2009 on this topic: The Seven Messengers and the “Buzzati Sequence” by Giorgio D’Abramo from the National Institute of Astrophysics in Rome. It is available here on arXiv. I discovered ita few years ago after working out my own “sequence” and had a short and pleasant correspondence with Dr. D’Abramo. You can compare our respective derivations, which I think are equivalent. Here is mine.

Although Buzzati gives absolute values (40 leagues per day), all that matters is the ratio m between the messengers’ and caravan’s speeds (m > 1). The relevant measures of time are:

  • The messenger-day, which take as unit of time.
  • The caravan-day, which is m times a messenger-day.

If as unit of distance we take ground covered in one day by a messenger, then time is equal to distance.

So if Tn is the time when a messenger rejoins the caravan after his n-th trip back home, we have

Tn + Tn+1  = m (Tn+1 – Tn)                  [1]

Justification of [1]:  both sides measure the time from when the messenger leaves (for the n-th time) to when he next rejoins the caravan. Note that the messenger goes back for his n+1-st trip on the very day he completes the n-th one.  On the left we have the time/distance  covered by the messenger (Tn to go home, plus Tn+1 to catch up). On the right, Tn+1 – Tn is the time/distance covered by the caravan in caravan units, which we multiply by m to get messenger-days.

The equality can be rewritten

Tn+1 = (m + 1) / (m – 1) Tn

yielding a geometric progression

 Tn = Kn T0                  [2]

where T0 is when the messenger leaves for his first trip, and the constant K is (m + 1) / (m – 1).

The Prince, who is as bad at horses as he (unlike Buzzati) is at math, had initially expected m = 2. Then K is 3 / 1, that is to say, 3. In that case the progression [2] would have been Tn = 3n T0. Even then, he would have found the result disappointing: while the first messenger returns the first time after three days, the third messenger, for example, returns the fifth time after about almost 1000 days (35 is 243, to be multiplied by 4), i.e. close to a year, and the last messenger returns for the sixth time after 16 years ( 36 × 8 /365).

The way things actually happen in the the story, the Prince determines after a while that m = 3/2 (the messengers go faster by half than the caravan), so K is 5. (In the text: Soon enough, I realized that it sufficed to multiply by five the number of days passed so far to know when the messenger would be back with us.) The unit travel times (Kn) of messengers are as follows, giving return times if multiplied by two for the first messenger (since he first leaves on the second day), three for the second messenger) and so on:

 

(1)          5 days: as stated in the story, the first return is after 10 days for Alexander, 15 for Bartholomew, 20 for Cameron…

(2)          25 days: Alexander returns for the second time after almost one month.

(3)          4 months

(4)          Close to two years (20 months)

(5)          8 years and a half

(6)          43 years

(7)          214 years

(8)          Millennium

Buzzati was a journalist by trade; I do not know what mathematical education he had, but find his ingenuity and mastery impressive.

(By the way, there might be a good programming exercise here, with a graphical interface showing the caravan and the messengers going about their (opposite) business, and controls to vary the parameters and see what happens.)

Another point on which the Prince is delusional is his suspicion that he would have fared better by selecting more than 7 messengers, a number he now finds “ridiculously low”. It would have cost him more money but not helped him much, since the number of messengers only affects the initial value in the geometric progression: T0 in [2]. What truly matters is the exponential multiplier Kn, where the constant K  — defined as (m + 1) / (m – 1) — is always greater than 1, inexorably making the Tn values take off to dazzling heights by the law of compound interest  (the delight of investors and curse of borrowers).

Obviously, as m goes to infinity that constant K = (m + 1) / (m – 1)  approaches its limit 1. Concretely, what messenger speed would it take for the Prince’s scheme to work to his satisfaction? The story indicates that the caravan covers 40 leagues a day; that is about 160 kilometers (see here). Ambitious but feasible (8 hours a day excluding the inevitable stops, horses on trot); in any case, I would trust Buzzati, not just because people in the 1930s had a much more direct informal understanding of horse-based travel than we do, but mostly because of his own incredible attention to details. So they are going at about 20 kilometers per hour. Now assume that for the messengers, instead of horses that only go 50% faster than the caravan, he has secured a small fleet of Cessna-style individual planes. They might fly at 180 km/h. That’s m = 9, nine times faster. Hence now K = 10 / 8 = 1.25. So we only lose 25% on each return trip; planes or no planes, the law of compound interest takes its revenge on the prince all the same, only a bit later.

The Seven Messengers

A number of years ago I discovered the short stories of the Italian writer Dino Buzzati (most famous for his novel translated as The Desert of the Tartars). They have a unique haunting quality, for which the only equivalent which I can summon is Mahler’s Des Knaben Wunderhorn or perhaps the last variation of the Tema Con Variazoni in Mozart’s Gran Partita. I was particularly fascinated by the first one, I Sette Messaggeri (The Seven Messengers) in a collection entitled La Boutique del Mistero (The Mystery Boutique, Mondadori, first published in 1968 although I have a later softcover edition). I resolved to translate it. I completed the translation only now. It starts like this:

Day after day, having set out to explore my father’s realm, I am moving further away from the city, and the dispatches that reach me become ever more infrequent.

I began the journey not long after my thirtieth birthday and more than eight years have since passed; to be exact, eight years, six months and fifteen days of unceasing travel. I believed, when I departed, that within a few weeks I would easily have reached the confines of the kingdom, but instead I have continued to encounter new people and new lands, and, everywhere, men who spoke my own language and claimed to be my subjects.

At times I think that my geographer’s compass has gone awry and that while always believing to be heading south we may in reality have gone into circles, stepping back into our tracks without increasing the distance from the capital city; such might be the reason why we have not yet reached the outer frontier.

More often, though, I am tormented by a suspicion that the frontier may not exist, that the realm spreads out without any limit whatsoever, and that no matter how far I advance I will never arrive at its end. I set off on my journey when I was already past thirty years old, too late perhaps. My friends, and even my family, were mocking my project as a pointless sacrifice of the best years of my life. In truth, few of my faithful followers consented to leave with me. Insouciant as I was – so much more than now! – I was anxious to maintain communication, during the journey, with those dear to me, and among the knights in my escort I chose the seven best ones to serve as my messengers.

I believed, without having given it more thought, that seven would be more than enough. With the passing of time I have realized that this number was, to the contrary, ridiculously low; this even though none among them has fallen ill, or run into bandits, or exhausted his mounts. All seven have served with a tenacity and a devotion that I will find it hard ever to recompense.

To distinguish more easily between them, I assigned them names with initial letters in alphabetical order: Alexander, Bartholomew, Cameron, Dominic, Emilian, Frederic, Gregory.

Not being used to straying so far away from my home, I dispatched the first, Alexander, at the end of the second evening of our journey, when we had already traveled some eighty leagues. The next evening, to ensure the continuity of communications, I sent out the second one, then […]

That is only the beginning. The full text appears here but it is password-protected. Here is why: in 2010 I managed to locate the right holders and wrote to them asking for permission to publish an English translation and put it on the web. I received a polite, negative answer. So I gave up. Browsing around more recently, though, I found two freely available translations on the Web. (I also found the original Italian text here, although with a few differences from the published version.) All for the better, you would say, except that one of the translations is in my opinion awful and the other not that much better. Buzzati is a stylist in the tradition of Flaubert, in whose texts you quickly notice (especially when translating) that every word is exactly the right one, the only possible one, at the only possible place in the only possible sentence. You cannot translate a Buzzati story as you would an article in today’s paper. You have at least to try to respect the music of the text. So I completed my own attempt after all, but I still don’t want to violate anyone’s copyright. (Perhaps I am being silly.) In any case, though, I can certainly publish a fair-use extract as above and use the text for myself and my colleagues and friends. So if you want access just ask me.

One unique feature of the Seven Messengers is that it is a geek’s delight: it is actually based on a mathematical series. I wrote an analysis of the underlying math, but to avoid spoiling your pleasure if you want to look at it by yourself first I put it in a separate entry of this blog. Click here only if you do want the spoiler.

Macron and Borne: profiles in courage

The French president, Emmanuel Macron, and prime minister, Elizabeth Borne, are showing incredible political courage in promoting an indispensable reform of the pension system. The international press (with the exception of one recent reasonable Washington Post editorial) has largely taken the side of the strikers, explaining sententiously that the proper answer would be to tax companies more (as to the efficiency of that approach, here is an old but still valid example, from a left-wing paper). The unions have vowed, in the words of one of their leaders, to “bring the country to its knees” and seem intent on reaching this goal literally. (It may be useful  to point out that unions in France are not what the term suggests. In other countries a union represents the workers at a company or administration. In France every organization has several unions, usually 4 or 5, competing for, typically, a small minority of the workers, but with a role enshrined in the constitution. They are really state-supported political organizations, of various political hues, several of them openly hostile to employers and to capitalism. Interesting approach.)

The reform of the pension system was part of Macron’s electoral program and has been amended repeatedly to take into account the special characteristics of manual or otherwise difficult worth. Months of attempted negotiations took place with those union representatives who were willing to talk. The extreme left and extreme right were united to defeat the reform and at the last minute, after innumerable debates in Parliament which had resulted in a majority-backed solution, intimated enough moderate-right deputies to force the government to use a special constitutional mechanism (“article 49-3”) to ram it through. Who knows how many disruptions of basic services the country will have to endure in the coming months as saboteurs of various kinds try to make good on their promise to prevent the country from functioning. The attitude of the international bien-pensant press, who fans the flames (as they did with the Gilets Jaunes protests 5 years ago),  while castigating the January 6 Washington rioters, who are of the same ilk, is unconscionable.

The entire political class knows that a reform is indispensable, and has been delayed far too long, out of the cowardice of previous governments. Macron’s and Borne’s goal is simple: to preserve France’s pension system (the very system that the opponents deceitfully accuse them of destroying), based on solidarity between generations, workers paying for retirees, as opposed to a capitalization-based system with its dependence on the ups and downs of the stock market. Thanks in particular to a generous health service, people live ever longer; the new plan makes them work a couple of years more to help ensure the sustainability of the approach. Macron is in his second, non-renewable term and has decided that he would not leave office without having carried out this part of his duty. Borne, an outstanding manager with a distinguished record, has taken the risk of sacrificing her political career by bringing the reform through. (In the Fifth Republic’s mixed presidential system, the conventional wisdom is that the prime minister is the president’s “fuse”, an expendable resource for implementing difficult tasks. Cynical and tough, but a direct consequence of the constitution designed by De Gaulle and his deputy Debré 60 years ago.)

In the meantime, Macron and Borne are showing Europe and the world what true dedication and leadership mean.

Le courage de Macron

(An English variant will appear tomorrow.)

La presse nationale et internationale est déchaînée contre Borne et Macron. Les extrémistes et factieux de tous bords jurent de “mettre le pays par terre” (comment, au passage, peut-on accepter ce genre de langage de la part d’un responsable “syndical”?).

Toute la classe politique sait bien sûr que la réforme est indispensable. Elle est le seul moyen de protéger le système français de retraites par répartition. Elle tient compte de la pénibilité des travaux. Elle remet la France au niveau des pays voisins. Elle est le bon sens même. Elle suit des années de tergiversation de la part des gouvernements précédents effarouchés, et des mois de consultation avec les “partenaires sociaux”, si l’on peut parler de concertation pour une tentative de dialogue avec des gens qui ne cherchent que le tintamarre politique.

Quel courage, quelle détermination chez le président et la première ministre, qui au milieu des insultes sacrifient leur intérêt personnel au bien public. Les émeutiers — dans la tradition des ligues des années trente, des gilets jaunes, des voyous du 6 janvier 2021 à Washington — essayent de les faire reculer par la force, mais la raison et le droit triompheront.

The legacy of Barry Boehm

August of last year brought the sad news of Barry Boehm’s passing away on August 20. If software engineering deserves at all to be called engineering today, it is in no small part thanks to him.

“Engineer” is what Boehm was, even though his doctorate and other degrees were all in mathematics. He looked the part (you might almost expect him to carry a slide rule in his shirt pocket, until you realized that as a software engineer he did not need one) and more importantly he exuded the seriousness, dedication, precision, respect for numbers, no-nonsense attitude and practical mindset of outstanding engineers. He was employed as an engineer or engineering manager in the first part of his career, most notably at TRW, a large aerospace company (later purchased by Northrop Grumman), turning to academia (USC) afterwards, but even as a professor he retained that fundamental engineering ethos.

 

boehm_tichy_basili

 

LASER Summer School, Elba Island (Italy), September 2010
From left: Walter Tichy, Barry Boehm, Vic Basili (photograph by Bertrand Meyer)

Boehm’s passion was to turn the study of software away from intuition and over to empirical enquiry, rooted in systematic objective studies of actual projects. He was not the only one advocating empirical methods (others from the late seventies on included Basili, Zelkowitz, Tichy, Gilb, Rombach, McConnell…) but he had an enormous asset: access to mines of significant data—not student experiments, as most researchers were using!—from numerous projects at TRW. (Basili and Zelkowitz had similar sources at NASA.) He patiently collected huge amounts of project information, analyzed them systematically, and started publishing paper after paper about what works for software development; not what we wish would work, but what actually does on the basis of project results.

Then in 1981 came his magnum opus, Software Engineering Economics (Prentice Hall), still useful reading today (many people inquired over the years about projects for a second edition, but I guess he felt it was not warranted). Full of facts and figures, the book also popularized the Cocomo model for cost prediction, still in use nowadays in a revised version developed at USC (Cocomo II, 1995, directly usable through a simple Web interface at softwarecost.org/tools/COCOMO/

Cocomo provides a way to estimate both the cost and the duration of a project from the estimated number of lines of code (alternatively, in Cocomo II, from the estimated number of function points), and some auxiliary parameters to account for each project’s specifics. Boehm derived the formula by fitting from thousands of projects.

When people first encounter the idea of Cocomo (even in a less-rudimentary form than the simplified one I just gave), their first reaction is often negative: how can one use a single formula to derive an estimate for any project? Isn’t the very concept ludicrous anyway since by definition we do not know the number of lines of code (or even of function points) before we have developed the project? With lines of code, how do we distinguish between different languages? There are answers to all of these questions (the formula is ponderated by a whole set of criteria capturing project specifics, lines of code calibrated by programming language level do correlate better than most other measures with actual development effort, a good project manager will know in advance the order of magnitude of the code size etc.). Cocomo II is not a panacea and only gives a rough order of magnitude, but remains one of the best available estimation tools.

Software Engineering Economics and the discussion of Cocomo also introduced important laws of software engineering, not folk wisdom as was too often (and sometimes remains) prevalent, but firm results. I covered one in an article in this blog some time ago, calling it the “Shortest Possible Schedule Theorem”: if a serious estimation method, for example Cocomo, has determined an optimal cost and time for a project, you can reduce the time by devoting more resources to the project, but only down to a certain limit, which is about 75% of the original. In other words, you can throw money at a project to make things happen faster, but the highest time reduction you will ever be able to gain is by a quarter. Such a result, confirmed by many studies (by Boehm and many others after him), is typical of the kind of strong empirical work that Boehm favored.

The CMM and CMMI models  of technical management are examples of important developments that clearly reflect Boehm’s influence. I am not aware that he played any direct role (the leader was Watts Humphrey, about whom I wrote a few years ago), but the models’ constant emphasis on measurement, feedback and assessment are in line with the principles  so persuasively argued in his articles and books.

Another of his famous contributions is the Spiral model of the software lifecycle. His early work and Software Engineering Economics had made Boehm a celebrity in the field, one of its titans in fact, but also gave him the reputation, deserved or not, of representing what may be called big software engineering, typified by the TRW projects from which he drew his initial results: large projects with large budgets, armies of programmers of variable levels of competence, strong quality requirements (often because of the mission- and life-critical nature of the projects) leading to heavy quality assurance processes, active regulatory bodies, and a general waterfall-like structure (analyze, then specify, then design, then implement, then verify). Starting in the eighties other kinds of software engineering blossomed, pioneered by the personal computer revolution and Unix, and often typified by projects, large or small but with high added value, carried out iteratively by highly innovative teams and sometimes by just one brilliant programmer. The spiral model is a clear move towards flexible modes of software development. I must say I was never a great fan (for reasons not appropriate for discussion here) of taking the Spiral literally, but the model was highly influential and made Boehm a star again for a whole new generation of programmers in the nineties. It also had a major effect on agile methods, whose notion of  “sprint ” can be traced directly the spiral. It is a rare distinction to have influenced both the CMM and agile camps of software engineering with all their differences.

This effort not to remain wrongly identified with the old-style massive-project software culture, together with his natural openness to new ideas and his intellectual curiosity, led Boehm to take an early interest in agile methods; he was obviously intrigued by the iconoclasm of the first agile publications and eager to understand how they could be combined with timeless laws of software engineering. The result of this enquiry was his 2004 book (with Richard Turner) Balancing Agility and Discipline: A Guide for the Perplexed, which must have been the first non-hagiographic presentation (still measured, may be a bit too respectful out of a fear of being considered old-guard) of agile approaches.

Barry Boehm was an icon of the software engineering movement, with the unique position of having been in essence present at creation (from the predecessor conference of ICSE in 1975) and accompanying, as an active participant, the stupendous growth and change of the field over half a century.

 

boehm_shanghai

Barry Boehm at a dinner at ICSE 2006, Shanghai (photograph by Bertrand Meyer)

I was privileged to meet Barry very early, as we were preparing a summer school in 1978 on Programming Methodology where the other star was Tony Hoare. It was not clear how the mix of such different personalities, the statistics-oriented UCLA-graduate American engineer and the logic-driven classically-trained (at Oxford) British professor would turn out.

Boehm could be impatient with cryptic academic pursuits; one exercise in Software Engineering Economics (I know only a few other cases of sarcasm finding its refuge in exercises from textbooks) presents a problem in software project management and asks for an answer in multiple-choice form. All the proposed choices are sensible management decisions, except for one which goes something like this: “Remember that Bob Floyd [Turing-Awarded pioneer of algorithms and formal verification] published in Communications of the ACM vol. X no. Y pages 658-670 that scheduling of the kind required can be performed in O (n3 log log n) instead of O (n3 log n) as previously known; take advantage of this result to spend 6 months writing an undecipherable algorithm, then discover that customers do not care a bit about the speed.” (Approximate paraphrase from memory [1].)

He could indeed be quite scathing of what he viewed as purely academic pursuits removed from the reality of practical projects. Anyone who attended ICSE 1979 a few months later in Munich will remember the clash between him and Dijkstra; the organizers had probably engineered it (if I can use that term), having assigned them the topics  “Software Engineering As It Is” and “Software Engineering as It Should Be”, but it certainly was spectacular. There had been other such displays of the divide before. Would we experience something of the kind at the summer school?

No clash happened; rather, the reverse, a meeting of minds. The two sets of lectures (such summer schools lasted three weeks at that time!) complemented each other marvelously, participants were delighted, and the two lecturers also got along very well. They were, I think, the only native English speakers in that group, they turned out to have many things in common (such as spouses who were also brilliant software engineers on their own), and I believe they remained in contact for many years. (I wish I had a photo from that school—if anyone reading this has one, please contact me!)

Barry was indeed a friendly, approachable, open person, aware of his contributions but deeply modest.

Few people leave a profound personal mark on a field. A significant part of software engineering as it is today is a direct consequence of Barry’s foresight.

 

Note

[1] The full text of the exercise will appear shortly as a separate article on this blog.

 

Recycled A version of this article appeared previously in the Communications of the ACM blog.

Logical beats sequential

Often,  “we do this and then we do that” is just a lazy way of stating “to do that, we must have achieved this.” The second form is more general than the first, since there may be many things you can “do” to achieve a certain condition.

The extra generality is welcome for software requirements, which should describe essential properties without over-specifying, in particular without prescribing a specific ordering of operations  when it is only one possible sequence among several, thereby restricting the flexibility of designers and implementers.

This matter of logical versus sequential constraints is at the heart of the distinction between scenario-based techniques — use cases, user stories… — and object-oriented requirements. This article analyzes the distinction. It is largely extracted from my recent textbook, the Handbook of Requirements and Business Analysis [1], which contains a more extensive discussion.

1. Scenarios versus OO

Scenario techniques, most significantly use cases and user stories, have become dominant in requirements. They obviously fill a need and are intuitive to many people. As a general requirement technique, however, they lack abstraction. Assessed against object-oriented requirements techniques, they suffer from the same limitations as procedural (pre-OO)  techniques against their OO competitors in the area of design and programming. The same arguments that make object technology subsume non-OO approaches in those areas transpose to requirements.

Scenario techniques describe system properties in terms of a particular sequence of interactions with the system. A staple example of a use case is ordering a product through an e-commerce site, going through a number of steps. In contrast, an OO specification presents a certain number of abstractions and operations on them, chracterized by their logical properties. This description may sound vague, so we move right away to examples.

2. Oh no, not stacks again

Yes, stacks. This example is rather computer-sciency so it is not meant to convince anyone but just to explain the ideas. (An example more similar to what we deal with in the requirements of industry projects is coming next.)

A stack is a LIFO (Last-In, First-Out) structure. You insert and remove elements at the same end.

 

Think of a stack of plates, where you can deposit one plate at a time, at the top, and retrieve one plate at a time, also at the top. We may call the two operations put and remove. Both are commands (often known under the alternative names push and pop). We will also use an integer query count giving the number of elements.

Assume we wanted to specify the behavior of a stack through use cases. Possible use cases (all starting with an empty stack) are:

/1/

put
put ; put
put ; put ; put       
— etc.: any number of successive put (our stacks are not bounded)

put ; remove
put ; put ; remove
put ; put ; remove ; remove
put ; put ; remove ; remove ; put ; remove

We should also find a way to specify that the system does not support such use cases as

/2/

remove ; put

or even just

/3/

remove

We could keep writing such use cases forever — some expressing normal sequences of operations, others describing erroneous cases — without capturing the fundamental rule that at any stage, the number of put so far has to be no less than the number of remove.

A simple way to capture this basic requirement is through logical constraints, also known as contracts, relying on assertions: preconditions which state the conditions under which an operation is permitted, and postconditions which describe properties of its outcome. In the example we can state that:

  • put has no precondition, and the postcondition

          count = old count + 1

using the old notation to refer to the value of an expression before the operation (here, the postcondition states that put increases count by one).

  • remove has the precondition

count > 0

and the postcondition

count = old count – 1

since it is not possible to remove an element from an empty stack. More generally the LIFO discipline implies that we cannot remove more than we have put.(Such illegal usage sequences are sometimes called “misuse cases.”)

(There are other properties, but the ones just given suffice for this discussion.)

The specification states what can be done with stacks (and what cannot) at a sufficiently high level of abstraction to capture all possible use cases. It enables us to keep track of the value of count in the successive steps of a use case; it tells us for example that all the use cases under /1/ above observe the constraints: with count starting at 0, taking into account the postconditions of put and remove, the precondition of every operation will be satisfied prior to all of its calls. For /2/ and /3/ that is not the case, so we know that these use cases are incorrect.

Although this example covers a data structure, not  requirements in the general sense, it illustrates how logical constraints are more general than scenarios:

  • Use cases, user stories and other  forms of scenario only describe specific instances of behavior.
  • An OO model with contracts yields a more abstract specification, to which individual scenarios can be shown to conform, or not.

3. Avoiding premature ordering decisions

As the stack example illustrates, object-oriented specifications stay away from premature time-order decisions by focusing on object types (classes) and their operations (queries and commands), without making an early commitment to the order of executing these operations.

In the book, I use in several places a use-case example from one of the best books about use cases (along with Ivar Jacobson’s original one of course): Alistair Cockburn’s Writing Effective Use Cases (Pearson Education, 2001). A simplified form of the example is:

1. A reporting party who is aware of the event registers a loss to the insurance company.

2. A clerk receives and assigns claim to a claims agent.

3. The assigned claims adjuster:

3.1 Conducts an investigation.
3.2 Evaluates damages.
3.3 Sets reserves.
3.4 Negotiates the claim.
3.5 Resolves the claim and closes it.

(A reserve in the insurance business is an amount that an insurer, when receiving a claim, sets aside as to cover the financial liability that may result from the claim.)

As a specification, this scenario is trying to express useful things; for example, you must set reserves before starting to negotiate the claim. But it expresses them in the form of a strict sequence of operations, a temporal constraint which does not cover the wide range of legitimate scenarios. As in the stack example, describing a few such scenarios is helpful as part of requirements elicitation, but to specify the resulting requirements it is more effective to state the logical constraints.

Here is a sketch (in Eiffel) of how a class INSURANCE_CLAIM could specify them in the form of contracts. Note the use of require to introduce a precondition and ensure for postconditions.

class INSURANCE_CLAIM feature

        — Boolean queries (all with default value False):
    is_investigated, is_evaluated, is_reserved,is_agreed,is_imposed, is_resolved:

BOOLEAN

    investigate
                — Conduct investigation on validity of claim. Set is_investigated.
        deferred
        ensure
            is_investigated
        end

    evaluate
                — Assess monetary amount of damages.
        require
            is_investigated
        deferred
        ensure
            is_evaluated
            — Note: is_investigated still holds (see the invariant at the end of the class text).
        end

    set_reserve
                — Assess monetary amount of damages. Set is_reserved.
        require
            is_investigated
            — Note: we do not require is_evaluated.
        deferred
        ensure
            is_reserved
        end
 

    negotiate
                — Assess monetary amount of damages. Set is_agreed only if negotiation
                — leads to an agreement with the claim originator.
        require
                   is_reserved
is_evaluated   
                   

        deferred
        ensure
            is_reserved
            — See the invariant for is_evaluated and is_investigated.
        end

    impose (amount: INTEGER)
                — Determine amount of claim if negotiation fails. Set is_imposed.
        require
            not is_agreed
            is_reserved
        deferred
        ensure
            is_imposed
        end

    resolve
                — Finalize handling of claim. Set is_resolved.
        require
            is_agreed or is_imposed
        deferred
        ensure
            is_resolved
        end

invariant                    — “⇒” is logical implication.

is_evaluated is_investigated
is_reserved 
is_evaluated
is_resolved
is_agreed or is_imposed
is_agreed
is_evaluated
is_imposed
is_evaluated
is_imposed
not is_agreed

                          — Hence, by laws of logic, is_agreed not is_imposed

end

Notice the interplay between the preconditions, postconditions and class invariant, and the various boolean-valued queries they involve (is_investigated, is_evaluated, is_reserved…). You can specify a strict order of operations o1, o2 …, as in a use case, by having a sequence of assertions pi such that operation oi has the contract clauses require pi and ensure pi+1; but assertions also enable you to specify a much broader range of allowable orderings as all acceptable.
The class specification as given is only a first cut and leaves many aspects untouched. It will be important in practice, for example, to include a query payment describing the amount to be paid for the claim; then impose has the postcondition payment = amount, and negotiate sets a certain amount for payment.
Even in this simplified form, the specification includes a few concepts that the original use case left unspecified, in particular the notion of imposing a payment (through the command impose) if negotiation fails. Using a logical style typically uncovers such important questions and provides a framework for answering them, helping to achieve one of the principal goals of requirements engineering.

4. Logical constraints are more general than sequential orderings

The specific sequence of actions described in the original use case (“main success scenario”) is compatible with the logical constraints: you can check that in the sequence

investigate
evaluate
set_reserve
negotiate
resolve

the postcondition of each step implies the precondition of the next one (the first has no precondition). In other words, the temporal specification satisfies the logical one. But you can also see that prescribing this order is a case of overspecification: other orderings also satisfy the logical specification. It may be possible for example — subject to confirmation by Subject-Matter Experts — to change the order of evaluate and set_reserve, or to perform these two operations in parallel.

The specification does cover the fundamental sequencing constraints; for example, the pre- and postcondition combinations imply that investigation must come before evaluation and resolution must be preceded by either negotiation or imposition. But they avoid the non-essential constraints which, in the use case, were only an artifact of the sequential style of specification, not a true feature of the problem.

The logical style is also more conducive to conducting a fruitful dialogue with domain experts and stakeholders:

  • With a focus on use cases, the typical question from a requirements engineer (business analyst) is “do you do A before doing B?” Often the answer will be contorted, as in “usually yes, but only if C, oh and sometimes we might start with B if D holds, or we might work on A and B in parallel…“, leading to vagueness and to more complicated requirements specifications.
  • With logic-based specifications, the two fundamental question types are: “what conditions do you need before doing B?” and “does doing A ensure condition C?”. They force stakeholders to assess their own practices and specify precisely the relations between operations of interest.

5. What use for scenarios?

Use-cases and more generally scenarios, while more restrictive than logical specifications, remain important as complements to specifications. They serve as both input and output to more abstract requirements specifications (such as OO specifications with contracts):

  • As input to requirements: initially at least, stakeholders and Subject-Matter Experts often find it intuitive to describe typical system interactions, and their own activities, in the form of scenarios. Collecting such scenarios is an invaluable requirements elicitation technique. The requirements engineer must remember that any such scenario is just one example walk through the system, and must abstract from these examples to derive general logical rules.
  • As output from requirements: from an OO specification with its contracts, the requirements engineers can produce valid use cases. “Valid” means that the operation at every step satisfies the applicable precondition, as a consequence of the previous steps’ postconditions and of the class invariant. The requirements engineers can then submit these use cases to the SMEs and through them to stakeholders to confirm that they make sense, update the logical conditions if they do not (to rule out bad use cases), and check the results they are expected to produce.

6. Where do scenarios fit?

While many teams will prefer to write scenarios (for the purposes just described) in natural language, it is possible to go one step further and, in an object-oriented approach to requirements, gather scenarios in classes. But that point exceeds the scope of the present sketch. We will limit ourselves here to the core observation: logical constraints subsume sequential specifications; you can deduce the ltter from the former, but not the other way around; and focusing on abstract logical specifications leads to a better understanding of the requirements.

Reference

Bertrand Meyer: Handbook of Requirements and Business Analysis, Springer, 2022. See the book page with sample chapters and further material here.

Recycled(This article was first published on the Communications of the ACM blog.)

New paper: optimization of test cases generated from failed proofs

Li Huang (PhD student at SIT) will be presenting at an ISSRE workshop the paper Improving Counterexample Quality from Failed Program Verification, written with Manuel Oriol and me. One can find the text on arXiv here. (I will update this reference with the official publication link when I have it.)

The result being presented is part of a more general effort at combining proofs and tests (with other papers in the pipeline). The idea of treating proofs and tests as complementary rather than competing methods of software verification is an old pursuit of mine (which among other consequences resulted in the creation with Yuri Gurevich of the Tests and Proofs conference, which I see is continuing to run). A particular observation is that failure means a different thing for proofs and tests.

A failed test provides interesting information (in fact it is a successful proof — of incorrectness). A successful proof is, of course, also interesting (in principle it should be end of the story), whereas a successful test tells us very little. But in the practice of program proving the common occurrence is failure to prove a program element correct. You are typically left with no clue as to the source of the failure. In the AutoProof verification system for Eiffel, we are able to rely on the underlying technology (Boogie and Z3) to extract a counterexample which gives concrete evidence: as with a failed test, a programmer can in general quickly understand what is wrong.

In other words, the useless negative result of the bottom-left entry of the above picture can produce a useful result:

Pasted

The general approach is the subject of another article but this one focuses on producing tests that are actually significant for the programmer. If you get very large values, you will not immediately be able to relate to them. Hence the need for a process of minimization, described in the article. The results on our examples are encouraging, making it possible to evidence the bug on very small integer values.

Reference

Li Huang, Bertrand Meyer and Manuel Oriol: Improving Counterexample Quality from Failed Program Verification, 6th International Workshop on Software Faults, October 2022. Preprint available on arXiv here. The program workshop is available here; the presentation is on Monday, 31 October, 15:55 CET (7:55 AM Los Angeles, 10:55 New York).

 

New book: the Requirements Handbook

cover

I am happy to announce the publication of the Handbook of Requirements and Business Analysis (Springer, 2022).

It is the result of many years of thinking about requirements and how to do them right, taking advantage of modern principles of software engineering. While programming, languages, design techniques, process models and other software engineering disciplines have progressed considerably, requirements engineering remains the sick cousin. With this book I am trying to help close the gap.

pegsThe Handbook introduces a comprehensive view of requirements including four elements or PEGS: Project, Environment, Goals and System. One of its principal contributions is the definition of a standard plan for requirements documents, consisting of the four corresponding books and replacing the obsolete IEEE 1998 structure.

The text covers both classical requirements techniques and novel topics such as object-oriented requirements and the use of formal methods.

The successive chapters address: fundamental concepts and definitions; requirements principles; the Standard Plan for requirements; how to write good requirements; how to gather requirements; scenario techniques (use cases, user stories); object-oriented requirements; how to take advantage of formal methods; abstract data types; and the place of requirements in the software lifecycle.

The Handbook is suitable both as a practical guide for industry and as a textbook, with over 50 exercises and supplementary material available from the book’s site.

You can find here a book page with the preface and sample chapters.

To purchase the book, see the book page at Springer and the book page at Amazon US.

Winter will be warm

It is easy to engage in generalities; it is risky to make firm predictions. In the first case there is no reckoning; in the second one the actual events can prove you wrong for everyone to see.

I am taking the risk. Here is my prediction: Putin’s energy blackmail (Western Europe will freeze this winter!) will fail. We’ll have some trouble but by and large we’ll be OK.

The basic reason is simple: great idea (from the blackmailer’s viewpoint), terrible execution. (Do we see a pattern there?) If you are going to freeze Europe by cutting off gas, you keep the suspense until the last minute and shut off the valves in October, leaving your targets no time to react.

Instead they did it all wrong! They started making noises in the Spring and cutting off supplies in August. The result: people listened. Governments and technocrats got to work, with some time to get organized. A company such as EDF in France is sometimes criticized as too big and monolithic, but they know their business, which is to provide energy, and are pretty good at it. I would bet that they and their counterparts in the electricity and gas industries all over the continent are working day and night to find alternative sources.

In addition, no day passes without some announcement of new energy-saving measures. Some may seem like for show only but the accumulated result will be significant. Recently everyone (for example the usually better inspired Guardian) was mocking Macron’s prime minister Borne and her ministers for showing up to work in padded jeans and sweaters to save on heating, but that kind of message can be influential. (Almost a half-century ago Jimmy Carter was telling Americans that instead of turning the temperature to 19 degrees C in summer and 21 in winter they should do the reverse. He too was derided. But he was right and that kind of advice will finally come to pass. One of the few positive outcomes of the current tragedy.)

So yes, you succeeded in making yourself a big nuisance. And no, it won’t destroy us. It will make us stronger — also warmer.