Interview with Michael Jackson

Michael Jackson was born in Birmingham in 1936. His parents moved in 1938 to a house in London, which was his home until he married in 1961. During the blitz, when many children were evacuated from London, his mother took Michael and his older sister, Joan, to Scotland, where they had family. They returned to London, but in 1944, during the “second Blitz”, they left to live for some months in Cheltenham.

During the time the family spent travelling, apart from one term at a school in Edinburgh, Michael’s education was provided by local tutors that his mother found for him and his sister. The first school Michael attended in London was Henrietta Barnett’s school, which, although it was a girl’s school, took boys up to the age of eight.  He then went to the Hall School in Hampstead, before progressing to Harrow as a boarder. He says of his time at Harrow: “I enjoyed it greatly.  At the end of every term I wanted to go home for the holidays, and at the end of every holiday I wanted to go back to school for the coming term.” While concentrating on classics, Michael also studied maths under Christopher Strachey who inspired his love of programming; it was in one of Christopher’s classes that Michael wrote his first program.

After Harrow, Michael went to Oxford to read Classics: Latin and Greek language and literature, Classical Greek and Roman history, and Classical Greek and modern philosophy.

Michael finished his degree at Oxford and having requested a deferment of his National Service he went to Cambridge to study maths; a subject which he had enjoyed at school.

In 1961, having gained his maths degree from Cambridge, Michael took his first job as a trainee consultant for Maxwell Stamp, an economic and industrial consultancy.  He explains: “They had understood that there were some new-fangled things called computers.  They knew that I was interested in them.  Not that I had any particular knowledge about them.  They must have thought: “What more can you want?  An ignoramus who’s interested in computers.  That’s what we need.”

The consultancy sent Michael on a programming course for the Bull Gamma 300, an electromechanical computer made by a French company.  He says: “This machine had the same kind of hardware technology as tabulating machines, including a plugboard for programming.  It was a large and wonderful assemblage of rotary switches and relays and all that kind of stuff.”

After this experience he was sent to Philip Hill Higginson, a bank who were transferring a share registration system from an IBM punch card system (card sorters, collators, gang punches and Model 407 tabulators) to an IBM 1401.  Michael describes the experience: “Like learning Christopher’s programming (at school), I thought this is the water, and I think I’m a duck.  It was wonderful.”

The 1401 was a very simple character machine and the programming language was the machine language, although there were mnemonics for some of the instructions.  It was equipped with mixed-radix arithmetic, allowing calculations in sterling currency. It had a core store of 4000 6-bit characters.

Michael explains: “The 1401 was so simple and the store was so small. The biggest 1401 was 16K of 6-bit characters, and this one was 4K, so when you printed out a core dump, you could see the whole thing.  ……. It was small enough and, in a slightly quirky way, well enough designed to be very easy to program.  And you couldn’t make too much of a mess in a machine that size.”

After his time at the bank, Michael was next sent to the NAAFI where he worked on converting the NAAFI payroll from punched cards to run on a Honeywell 400. This was a 48-bit word machine, much bigger than the 1401, with lots of tape drives. He describes it as quite quirky, with a teletype operator’s console from which the operating system of the Honeywell 400 allowed you to inspect and change what was in core storage, and even to inspect and modify tape files. He explains: “I wrote the programs. I was given a free run of the machine at night, but I had to work nights in order to do this. I soon became aware that programming wasn’t as easy as I had thought earlier.  So, in 1963, I introduced assertions into my programs, with run-time checks on the assertions. If a check failed, the console typed out ‘chaos’ and the storage address at which the program had stopped.” Initially, when the payroll was run, Michael sat in the observation room, ready with his listings to deal with any  “chaos” stoppages. Over a few months the incidents of chaos reduced and eventually seemed to have been entirely eliminated. He adds: “That was when I realised that programming wasn’t as easy as it had seemed at first.” Something that Michael summed up in a quotation at a later stage in his life as: “I was a careful designer.  I realised that program design was hard and a program of any size was very likely to be erroneous.”

 By this time, despite any stress, Michael knew that he loved the machines. He says: “It’s problem-solving, but it’s also the joy of your train set.”

In 1964 Michael moved to the newly launched John Hoskyns and Company which was set up by John Hoskyns, and John Pearce. The business grew rapidly thanks to John Hoskyns’s ability to raise investment money, later including a ten per cent share purchase by America Express.

Michael recruited Barry Dwyer to Hoskyns at that time and the two frequently worked together. Michael explains: “He (Barry) was better educated in computing than I was. I didn’t really have any computer education at all. I had a little experience and various intuitions, some of which were useful, and most of which probably weren’t. Barry could add some actual knowledge and wisdom to it. We started something we called segmented-level programming, which was really just top-down programming with flowcharts and modules.  We gave a seminar about it in New York, and did various things like this. Gradually things got a bit more intelligible, and some of the early ideas of Jackson Structured Programming (JSP) emerged: many must certainly be attributed to Barry, there’s no doubt about that.”

However, after the initial success, the economic climate took a downturn in the late months of 1970. Michael says: “it all turned sour” for Hoskyns, and Michael found his work “reorganised out of existence”.

However, having made some money from his Hoskyns shares, Michael decided to set up Michael Jackson Systems Limited in 1971, complete with an office on the side of his house in Pinner.

He hired Brian Boulter from Hoskyns to join, him telling him: “I’m not sure what we’re going to do, but you can do it for a year, and I’ll pay you this much.  And maybe at the end of the year I’ll have to say, ‘I’m sorry Brian, but, the game’s up, I can’t keep you on,’ or maybe I’ll say ‘we’ll carry on.’”  Brian stayed twelve years, until his tragically early death in 1982.

Michael’s original idea for a software product was an automatic testing system for COBOL programs. It did not prove commercially successful, and the company started providing programming courses which Michael describes as acquiring “a certain notoriety”. The courses were based on Michael’s design method derived from his experience of structuring file processing programs to make them easier to develop and maintain.

Michael and Brian, and more people who joined the company, established a network of licensees across Europe. Many companies ordered in-house courses and some sent their trainers to the “train the trainers” course. The method would eventually be named Jackson Structured Programming (JSP) by the Swedish licensees. As Michael explains: “the name JSP wasn’t our idea, but it was commercially helpful because it dsitinguished our method from its compeitors. Technically, the method became well enough defined for us to write a COBOL pre-compiler for JSP programs. And that’s what we did, and it became the company’s main software offering.”

Michael says of JSP: “I like to think that it was really good for the problems that were commonly discussed in courses on modular programming, program design, and so on, and for many if not most of the data processing problems that people dealt with in practice.”  He goes on to explain: “Systems were mostly based on tape drives, because they had big files, and you couldn’t put big files on disks.  Disks were hopelessly expensive, so data processing applications relied on sequential tape files having obvious and natural regular data structures. The insights embodied in JSP, I think, were genuinely very useful.  Evidence of the virtues of the approach, as I used to claim and still would, was that by the later stages of the course, people who had been listening and paying attention would all produce the same answers to the same problems.”

In 1975, encouraged by Tony Hoare who was a series editor of the Academic Press at the time, Michael wrote his first book ‘Principles of Program Design’, describing the JSP method in detail. Michael had known Tony Hoare at Oxford; in 1973 he became a member of the IFIPS WG2.3 working group on programming methodology, after describing the method to the members, including Tony Hoare, Edsger Dijkstra and Cliff Jones. JSP was adopted by the Government as their standard programming method following some courses for the Civil Service College in Norwich and London.

With JSP proving to be very successful, Jackson Structured Development (JSD) was born. It was a system development method for both individual programs and entire systems. It was most readily applicable to information systems but could be extended to the development of real-time embedded systems. Michael explains: “JSD was an attempt to introduce the notions that I have very briefly sketched here into the development of data processing systems.  To be honest, it wasn’t successful. There was a book about it, but many potential users and customers were unconvinced, and probably they were quite right.  So, although it had some good insights, it rather limped along.”

In the mid-eighties, Michael sold the company to the Swedish licensees who insisted he remained with the company for a further five years. It was during this time that his interest started shift to broader software engineering concerns and he began to develop his problem frame approach.  He explains: “I started proposing simple intuitions to myself, like, software development is just a matter of creating descriptions.  So the question is, what are you describing, how do you describe it, how do the descriptions come together to make the system, and so on.  And I thought a lot about that.”

Around 1989, Michael started working as a part-time researcher with Pamela Zave at AT&T Research, (Bell Telephone Laboratories as it was then). Pamela was interested in JSP and in using it for telephone switching software. Michael had first met Pamela in the 1970s when he first visited Bell to deliver a course.  He explains: “We talked about the problems of telephone switching, because Bell Labs had been programming electronic switching for the domestic telephone network since 1968. Later Michael Jackson Systems Limited sent John Cameron to give a JSP course at Bell Labs. By the time I became involved in work with Pamela, it was, I think, Number 5 ESS, so it had been through many modifications and changes.  But, not to put too fine a point on it, it was a gigantic mess.  The basic difficulty was that the calling features—call forwarding, call blocking, call waiting, and dozens of others—interacted in surprising and sometimes very inconvenient ways. It was a serious and daunting challenge of software structuring.”

He continues: “These features were really difficult to get right individually, and in combination they got the software into a mess. The development teams had difficult organisational problems. They had to distribute new versions of the software regularly to all their customers, and they also had to be working on a new system while they were still installing and supporting the previous system. They had arrived at the situation which Microsoft called ‘infinite bugs’: when you find a bug, you fix it and then you have two new bugs instead.  Various plans and attempts were made to redesign the software architecture; but the problem seemed intractable.”

Pamela and Michael spent all their time discussing the problem. “And then, one day,” he says, “there was a wonderful revelation.” That revelation lead to Michael and Pamela inventing Distributed Feature Composition (DFC). He explains the inspiration like this: “One of the standard subscriber services was 1-800 dialling: you can dial “1-800-AIRWAYS”, and you get through to British Airways.  So how does this work?  Well, the way it works is that, when you dial 1-800 something, your local switch sends the call to an ‘800 server’: that’s a physical machine in the network dedicated to handling 800 numbers. That machine looks up the dialled code and forwards the call to British Airways.  This was the key to Distributed Feature Composition.  It treats the features as if each one had its own hardware, its own computer.  Of course, you’re doing it all in software in the implementation. You can replicate the features, put them together in a kind of pipe-and-filter structure, and govern their interactions by the way they communicate and so on, so forth. It made a big impact on the problem of feature interaction in phone systems.  Just in time for voice over IP to undermine the value of the domestic network.”

While this was going on, Michael continued to work on the problem frames approach, something he describes as not just an interest but an obsession even today. He says: “I kept thinking about it, and in 1995 Software Requirements & Specifications was published. This contained a lot of pieces about software, sometimes light-hearted, motivated by the ideas of problem frames, and prefaced by a frank admission: the book is called A Lexicon, a collection of articles about different things, because I was not yet in a position to make a coherent approach out of the whole thing.  And that’s where it was in 1995.”

In 2001 Michael was invited by Bashar Nuseibeh to become a Visiting Research Professor at the Open University. Here he continued to work on his problem frames ideas. A major cooperation was with Jon Hall and his wife Lucia Rapanotti, who wrote some papers on problem frames and organised an international conference. Michael adds: “More recently they’ve extended their scope to problem-oriented engineering more generally. The basic ideas there can be traced back to their work on problem frames.”

On the subject of the problems and reliability of software engineering today and what has been learned Michael says: “I think, one way of thinking of it is this. Over-simplifying greatly, there are two kinds of software engineering.  There’s engineering of software, and there’s engineering by software.  Engineering of software is about designing and transforming programs whose purpose is to compute results inside the computer. Engineering by software is about embedded systems and cyber-physical systems, where the software is being used to govern behaviour in the physical world.  And, that means that your product is actually not the software, nor is it the execution of the software.  It’s the behaviour that results in the physical world.  The structure and content of that behaviour is what you’re really producing.  But in order to produce the behaviour, you must have physical machines executing the software. The structure of the problem, that is, of the system behaviour, is probably not the appropriate structure for the software to be deployed on the hardware that’s available.  So some serious engineering of software is needed to transform from the behaviour structure to the structure of the deployed software. That understanding seems to be neglected in many places where you might expect it to be present.”

Michael gives two examples of how failures in engineering by software have led to loss of life; one in medicine and one in the US military. He then goes on to discuss a quote from Tony Hoare. In a paper called “How did software get to be so reliable without proof?” in 1995, Hoare wrote ‘It has turned out that the world does not suffer significantly from the type of problem our research was designed to solve.’

Michael comments: “Well, I think there’s a deep point in the background to that. Some people have thought that the problem with software was correctness.  That’s to say, correct programs would satisfy their formal specifications, regarded purely as computer programs.  So if you could do program proving and correctness proofs, and show that the program was correct in this narrow sense, you would get rid of errors of that kind. I think what he was saying in this quotation is that what many mathematicians thought mattered most, which was this kind of program correctness, is actually not the crucial aspect of the problem.  And I think that’s absolutely right.  The problem is that, the programs that matter that we’re talking about are programs that interact with the world.  And, this is engineering by software. What you are producing as a software engineer is the behaviour in the world, and that’s where the main effort has to be. This is a recognition dear to my heart.”

Interviewed by: Mark Jones on the 7th November 2018 at BCS London

Transcribed by: Susan Hutton

Abstracted by: Helen Carter