Interview with Professor Harold Thimbleby

Professor Harold Thimbleby was born in Harborough Magna, near Rugby in Warwickshire, in 1955.

His mother was a pianist and then became a fulltime mother to Harold and his sister. His father was an architect. The family moved to Rugby where his father had built up his own practice. Harold says: “He designed all sorts of things from people’s extensions to factories, to whole rows of shops and so on.  I remember working in his office a lot.  My grandfather was an electrical engineer and I spent most Saturdays at my granddad’s who had a workshop. He got me excited with electronics, engineering, valves, long sparks, and stuff like that.”

Harold, his sister and parents shared a large house with his maternal grandparents. Having built an oscilloscope and other electronics equipment in the family’s main room, he was eventually given a dedicated room in the house to turn into a lab. He says: “I’ve realised recently that that is what changed my life.  I spent all my time in that room, building things, blowing things up, testing things, and doing sums to design circuits. I had a lot of fun.”

At school, Harold sat A level physics and came top in the country. He attributes this success to his experience of doing electronics and physics at home. He adds: “I’d been doing it for well over the “ten years to become an expert” by the time I got to the national exams.”

At the age of ten, Harold was invited by his uncle, a lecturer at the local Polytechnic in Rugby, to visit and take a look at their Elliot 803 computer, which filled a whole room. When the Polytechnic upgraded to an ICL 1900, Harold regularly cycled over and dropped off cards for the 1900 to run programs.  He says: “I had a lot of fun. I tried everything.  ALGOL 60 is where I started, then I learnt Fortran, BASIC, Snobol, ML/1, LISP, and more.  I discovered if I ran BASIC programs I could get more time on this mainframe than if I ran ALGOL 60 programs and that transformed what I could do.  When I came across ML/1, which is a macro-processing language by Peter Brown (who was then at the University of Canterbury, Kent), I started using that as, being a macro language, it could change programs, so you could program about programming.

Then my school got a teletype and a modem and linked up to the Modular One computer at the University of Warwick.  I started programming in BCPL. I got in touch with people like Martin Richards who’d designed BCPL. I then did an A level in computing in my spare time, it was fun, but somehow I didn’t think of it as a real subject, like I thought physics was a real subject.”

After completing his A levels, Harold read physics at Queen Elizabeth College in the University of London. He says: “It seemed like the obvious thing to do.”

It was at university that Harold found his passion for computers. He explains: “Although I’d played with the computers at the local college in Rugby, I didn’t think of computers as a serious subject so I did physics. The first year physic laboratory had a PDP-7 in it, and I started using it. PDP-7s are minicomputers, though this one was the size of a few filing cabinets. I had the whole thing to myself. I’ve never looked back, and I’ve been a computer scientist ever since.”

During term-time Harold ran the College’s access to London University’s mainframes which included CDC 7600 and 6600 computers.  In one holiday, Harold worked as the chief programmer at the Royal College of Art which had an Elliot 905 with a vector graphics radar display, and a fixed head drive of 100 megabytes. He says of the experience: “It seemed like a big disk at the time (the single disk was in a box about 2m high), it was great fun. Artists would come in and draw fun vector graphic things and there was a light pen, so it was interactive graphics, whereas most of the work I’d done previously was card decks and paper tape, so interactive programming was good fun.”

Having completed his physics degree, in 1977 Harold went on to complete a Masters degree in computer science at Queen Mary College. The lab there was the first place outside of America to get a Unix. Harold adds: “The lab where I did my PhD had got Unix and that was great fun.  With George Coulouris, the laboratory’s research theme was “the office of the future,” how are we going to design computers to run the office of the future?  They were all excited about window displays, colour displays, mice, and things like that, and stuff would change everything.  We were way ahead of the game.  However, as a physicist, in the back of my mind, I was thinking, what evidence have you got? What are the theories that make you believe reliably that it is going to be the office of the future?  I can see it was hugely engaging, but there was no argument, no reasoning, that I could see that this was actually going to change offices reliably, other than by excitement.  So I did my PhD in what we now call human-computer interaction, about the theories of computing that made sense about why these ideas would work for interaction.”

Peter Landin was one of the professors at Queen Mary College. He opened my eyes to formal methods. Looking back, Harold says, “I had seen programming as something you can just do if you are good at it, as I was. It didn’t require much thinking about to stretch your mind, like physics obviously did (or did for me). Physics has Maxwell’s Equations, general relativity, Hamiltonians, and much much more that pushes you to the limits; computer science it seemed had nothing hard in comparison even though I was well read in it. People like John McCarthy (inventor of LISP and AI) just “talked” about programs. I know, sure, there is lots of maths about basic computing, but I didn’t see how it scaled up to connect to the job of programming systems that did anything useful. But Peter showed me you could reason theoretically about real programming, and so computer science became to me a real, serious subject, as much a serious science as physics obviously was.”

Harold completed his PhD in 1981 at Queen Mary College, the point at which human computer interaction (HCI) was starting to get off the ground. Harold explains that at the start of his PhD, HCI (not that it was called that at the start!) was seen as computer science and by the end of his PhD it had shifted to being seen as psychology supported by computer science. He adds: “Then, a couple of years later, the Alvey project got off the ground and people started to say HCI is psychology. So the field shifted quite a lot over that period.”

After completing his PhD, Harold spent a year lecturing at Queen Mary College. He was then offered two jobs; one at Barts Hospital which wanted him to set up their digital systems, and bring in Unix, the other was a lectureship in computer science at the University of York. With two young children, Harold and his wife decided York would be a better place for the young family to grow up, so he joined York University’s computer science faculty.

 Harold says of the department: “University of York computer science at that point was famous for its work in Ada. It had some VAX-780s, some PDP-11s and other mainframes, and I ended up being in charge of some of the Unix systems.  So I was in the core of the computing infrastructure of the department, but my interests was user-interface design — many people called it man-machine interaction (MMI) then, or human-computer interaction (HCI), as it was becoming known.”

At York, Harold met psychologists, Andrew Monk and Nick Hammond, who already knew about him. Harold says: “We started having weekly meetings talking about the problems of HCI. Together with Colin Runciman and Michael Harrison, who were close colleagues of mine in the computer science department, we realised, this was a thing we could do jointly.”

 While there were other individuals across the UK looking at HCI, Harold believes that he and his colleagues were one of the first groups to bring together a multi-discipline view on the subject.

 With the team at York, Harold received grants from the Alvey Project and was able to work on a number of projects. He explains one of them, saying: “We called it the Five Man Project, because it involved Nick Hammond, Andrew Monk, Michael Harrison, Colin Runciman and me; five men on this multidisciplinary project.  I would like to think it was based on a paper I wrote in my PhD about generative design principles. There was a large amount of literature on design rules and guidelines, as computers got off the ground and started exposing problems. For example, Sid Smith at Mitre had an enormous book of thousands of things like the delete key must delete characters, and you must have undo. These were ad hoc rules. We realised if you build a theory for them, then if you formalise these principles you can start looking at whether the rules work together or not.  One of the things we discovered, for example, is that ‘undo’ is really quite an interesting principle.  You think it’s kind of obvious, if you make a mistake you should be able to undo it.  When you use formal methods you start considering exactly what happens if you undo the undo, what does that mean? It turns out you’ve got to make some decisions about what it means, but if you just say your computer system must have undo and you haven’t thought through these decisions, you’re going to end up with bugs, or unexpected properties that will trip up users.”

Harold stayed at York for six years and was then offered the Chair of Computer Science at Stirling University.  He accepted it on the basis that he would get to work with Peter Henderson, not realising that it was actually Peter’s job he was applying for! Peter had moved to Southampton.

In 1994, Harold moved to Middlesex University to help set up the Computer Science Department, alongside Norman Revell who had been at City University.  Harold says: “They appointed us both and said run the department between yourselves.  Norman wanted to take charge of all of the teaching and administrative side, and I took charge of all of the research side. There wasn’t much of a department of computer science when we took over; in a few years we had grown from six to twenty, and then it grew very quickly as we got it going.”

Speaking about the development of HCI alongside the development of the computer industry with the advent of mobile phones, the internet, the world wide web etc, Harold says: “Industry and universities are fairly separate things.  They’ve got different goals and attitudes. For instance, Gary Marsden, Matt Jones, and I did some very good work on designing mobile phones.  We got several patents because we had some great ideas.  However, nobody in industry was interested. We talked to Motorola and Nokia and so on, and they agreed that what we were doing was better, but they want to make money, they want to sell things, and our thinking wasn’t how they thought.  It’s a shame.  Until Apple came out with the iPhone, a lot of the mobile phones we had were very sub-optimal and confusing. The group of us wrote lots of papers analysing these things in different ways — you can find lots of ways of improving them.”

In 2001, when John Long retired from leading UCL’s Ergonomics Unit, Harold applied. He says: “I put in a bid saying HCI is psychology and computer science, so let’s create a new UCL Interaction Centre, UCLIC, as I called it, to bring psychology and computer science together to look at these issues, design of mobile phones and anything that’s interactive and has got a computer in it.  They bit the bullet and appointed me.”

After accepting but before starting, the two heads of the psychology and computer science department both changed, and the new ones were much less interested in working together. As a result, Harold found himself spending a lot of time dealing with the internal politics which finally prompted him to leave. He says: “I got fed up politicking rather than doing science and interaction and so on, it seemed like a waste of my life. I started looking around for other jobs.  Matt Jones spotted a role at Swansea University and encouraged me to apply for it.

In 2005, having decided to move on from UCL, Harold was appointed Chair at Swansea University. Harold says: “Matt Jones joined me and we set up the FIT Lab, the Future Interaction Technology Lab.”

In 2007 Harold published his book Press On.  It was published by MIT, and winner of a few prizes including the Computer and Information Sciences Award from the Association of American Publishers.

 He says of it: “The book is all about how to design interactive systems, to program them so they do what you want them to do, and so you can analyse them and do all sorts of things and to optimise them.  If you want to reduce the number of keystrokes to do tasks, start with the engineering then you can do studies to see whether people like it, which is not so much an engineering question but interesting psychology.”

In 2021, Harold published ‘Fix IT: See and Solve the Problems of Digital Healthcare’.  This book, too, won an award: the British Medical Association’s best book in General Medicine.

He explains that he originally became interested in digital healthcare after visiting one of his Masters students in hospital who was recovering from a serious car accident. Harold says: “I was just shocked at the computer-based equipment he was connected to.  There was one infusion pump that had a Post-it note on it that said ‘Don’t press this button’.  It was crazy, what if that Post-it note falls off, why can’t the machine do what it’s supposed to do without people having to write arrows and instructions on Post-it notes?”

With a grant, Harold bought the same infusion pump and started examining how it worked and discovered over 30 problems with it. He says: “When I found this long list of what I considered to be bugs in the infusion pump, I decided to be humble about it and realised that maybe I don’t understand what infusion pumps are supposed to do clinically. I talked to the manufacturers and I talked to some anaesthetists and they got excited, because anaesthetists love technology. For a week, I scrubbed up and I went undercover, I worked as a porter, so I pushed patients into the operating room. I did exactly what I was told and so it was perfectly safe. I just sat in the operating room and watched and saw how they used all this equipment for real.

“In all the six big operations I observed something went wrong with some bits of the computing technology.  Sometimes it was pretty straightforward, but in one operation, the ventilator crashed.  The anaesthetist sighed, got up and rebooted the ventilator. Once it’s rebooted, it doesn’t know there’s a patient on it or what the total volume of their lungs is and all the rest of it, so the anaesthetist has to enter all this data and gets the patient breathing again. When everything had calmed down, I asked if he would report it and he said “the patient is alright”, so no he wouldn’t report it. So, that’s an example of one of the problems we had. There is a cultural problem that these things happen all the time and the professionals, like the anaesthetist, just sort them out and the patient comes out at the other end and is alright. But let’s say two things had gone wrong at the same time, the anaesthetist maybe wouldn’t have been able to cope, so it’s a dangerous problem.

“I got a small grant to study infusion pumps, and by this time I’d moved to Swansea at this stage, so I worked with UCL (Ann Blandford) and Queen Mary College (Paul Curzon), and we set up a large programme grant to study digital healthcare, to see if we can make a bigger difference.” See http://www.chi-med.ac.uk for more details.

Asked whose responsibility it is to record these kinds of issues, Harold says it should be something that manufacturers monitor, and they could do it easily if the machines are connected to the internet.

 He goes on to say there are lots of issues in healthcare HCI research, including gaining ethical clearance and permissions. He says: “Lots of people stay out of research in healthcare in hospitals because of the ethical hurdles. At the other end of it people think computers are wonderful, so why do you need to do research anyway, as the solution to healthcare’s problems is new computers. Whereas if you ask me, the solution to healthcare’s problem is better computers and it’ll take a bit of work to find out how to design better computer systems, but who wants to do that hard work when everybody else just wants to buy new things?  So, there’s a big cultural thing to realise this is a problem and that it’s a soluble problem, but the solution isn’t just new exciting stuff, it’s thinking about it, studying it, and thinking about what you really want to do.

“One of the common problems healthcare has got is it is really complicated, so people say let’s computerise it but then you discover that you’ve computerised a mess, so you’re just making the mess faster! You need to go in there with some plan to improve what’s going on, and then you’ve got to convince the doctors and nurses that this improved thing is better, and only then is it worth computerising.  Lots of people have built computer systems for hospitals and they don’t talk to each other, which is called the interoperability problem. It’s a lot of work to sort it out and lots of people are in denial about the underlying engineering problems because they see it as a training problem and as our computers are all old, we need to get new, up to date ones. But first we have to ask them what have you learnt about making computers safer that makes it worth getting an up to date one? And the answer is, nobody knows.  Well, in that case, maybe wait until we’ve done the research.” That sounds harsh, but “evidence based medicine” is how everything else works in healthcare; one has to ask, why not in digital healthcare too?

Asked about the potential and risks of AI in healthcare, Harold says: “A potentiality is it’s got people to recognise our biases a lot better, which is interesting … These AI systems learnt from data that was made from what humans were doing; the biases were there already, but the AI has made it visible, so that’s a big improvement — we now realise it’s a problem, so we need to solve it and the AI’s making that problem rigorous so that it can be solved, so these risks of AI have got people thinking, and that’s good.

“But, there are problems in diagnosis, for example, recognising breast cancer from X-rays. There’s a whole range of interesting things where some clinicians feel it’s competing with their authority. Sometimes the AI says something and people just take it for granted the AI’s right, but it may not be.  So it changes things.  It also changes how we blame. We’re used to humans making mistakes, so it’s no surprise when doctors misdiagnose things, but when a piece of AI misdiagnoses something, it raises some really interesting problems, for example, should the clinician have made the right diagnosis and ignored the AI, but then what’s the point of having the AI if it can’t do the right things and it’s undermining the clinician? Who’s liable when something goes wrong?  The big difference is the doctor who diagnoses is just one individual, whereas the company that made this thing (the AI) has got thousands of employees, and it’s got lots of lawyers, so, most likely, it’s going to make sure it doesn’t get sued.  So there are interesting cultural and legal issues that come up, particularly when things go wrong.”

Asked about his proudest achievements, Harold highlights acting as an expert witness to help courts understand computer systems. He says: “That’s a perfect job for a professor, because you’ve got a real world problem — people might go to prison or get fined, and you’ve got to get to the bottom of this problem that’s really important. And in a case with computers I’m using everything I know about computers to see what has the court missed, what have they misunderstood. Through the expert witnessing I’ve done, I’ve kept people out of prison, because the prosecution ended up going to court over some issue actually caused by a computer, but the hospital misunderstood it and blamed a nurse or nurses. Those nurses would have been sent to prison because the computer evidence appears to say the nurses made a mistake so it’s their fault; professional negligence. In court I’ve been able to say, actually, yes, the computer says this but the computer was mismanaged, or it lost data, or whatever, and you’ve misunderstood the big picture of what’s going on.”

Harold has also been involved with computerised visa applications through expert witnessing.

He continues:  “There’s a very interesting point of law in Britain called the presumption of computer evidence being correct.  It’s called a presumption because it’s not saying the computer evidence is correct, it’s saying we’ll presume it’s correct because if you question it, you could question anything and waste court time.  If you get a speeding ticket, and you start saying did the speed camera work, you end up going down a bottomless pit that’s actually got nothing to do with your breaking the speed limit.  Of course the speed camera worked (almost always!). So that’s why the presumption in law is there.

“However, people get prosecuted over things that are actually caused by computer errors. Getting to the bottom of those computer errors is really, really, really difficult, because under the presumption the prosecution doesn’t have to reveal the information about the computers. … The most famous case is perhaps when over 700 people were prosecuted for embezzling money from the Post Office; it is called the Horizon scandal. Horizon was the computer system, it had bugs and it reported losses of money, which hadn’t actually happened. … It was an absolute disaster and it’s the biggest miscarriage of justice in Britain ever.  Because I’ve been working with legal people about this presumption and the fact that the presumption s completely out of touch with the reality of how computers work, I’ve done some work with the barristers working with the Horizon postmasters and that has resulted in some papers we’ve written together that have gone to the Department of Justice to try and fix the presumption of computer evidence being correct. It’s clearly inappropriate in the modern world.”

After being told that his father had died in hospital due to a mistake by the anaesthetist, Harold spent a long time trying to get to the full details. It was an experience that led him, with his wife’s expertise, to create a two minute digital story. He explains: “I work in human error and work in medical devices, so I completely understand how this mistake happened. I got some outline information and that seemed fine, but then I asked for more information and I never got the details I wanted. I then made a complaint and that’s when you then go down a rabbit warren.  … I finally got hold of the Datix report which is the official thing when something happens. People write up who the patient is and what was wrong and what they did and so on. The Datix report on my dad’s death was all lies, and that is when I decided to do something about it and I escalated it”

Having escalated the case, Harold says everything got very complicated, he continues:  “That’s when my wife said, let’s make a digital story. Talking to lawyers you end up with a really long complicated thing and every sentence is justified and it gets logged, and it gets boring — and misses the point.  My wife’s technique is that everybody’s got two minutes, so how are you going to get it down to a two-minute story?”

Harold together with his wife created a two minute story using photos with a voice-over by Harold. He continues: “The pictures were of dad alive being a father and grandfather and everything else, and my voice-over was the two-minute key things, explaining that a mistake was made that anybody could have made but it was reported (on Datix) in a way to conceal the mistake. My story said if you want to improve processes you’ve got to sort that out. I talked about things that could be done, and I ended it with a couple of textual words on the screen saying, we want to make healthcare safer, that’s the starting point, not covering up problems.

Harold also posted the DVD of the story to the Chief Executive who shared it with the board and then shared a list of things that the team intend to change as a result of the message.  “That little two-minute video is now used in training new people starting at the hospital.”

Looking to the future, Harold says: “I think most people would say things like implants, prosthetics and AI are some of the exciting things. Harold also highlights the complicated nature of healthcare and managing the many individual different systems that do not necessarily work together to the patient’s benefit, for example, patient records, he says: “Sorting all that out isn’t just getting patient records to work, it’s changing the culture in the hospitals to rationalise it and re-engineer it and that’s really difficult. It’s going to be especially difficult because most people think computers just work, and computers just solve problems.  They don’t realise these problems are complicated.  If you just computerise the NHS or healthcare, you end up with the mess going faster.  What you want to do is computerise it so that you understand and make the processes better, and that’s going to be really difficult but ultimately much more useful.”

He points to the fact that we all like to do things in our own professional ways, adding: “We’re all proud about all the work we put in to becoming experts and then people who study how we work, and how we work with other colleagues is becoming important because of everyone’s interaction and communication through computers. We are uncovering huge cultural problems that are going to have to be sorted out.

“So, in the next ten years, I think there are going to be ‘earthquakes’ in culture and that’s going to be the interesting area, not the technologies.  Technologies are going to expose all these problems.  AI has already made some biases obvious, but the biases were there already, we just didn’t know. Computers, not just AI, are doing the same all over healthcare (and everything else) – highlighting the interoperability problems. Obviously you can have bad computer systems that are not interoperable, but almost all of the interoperable problems are exposing problems that were there in healthcare before computers even got in.”

Throughout his career, Harold has been able to visit various international universities. Harold shares some of his experiences, saying: “I spent about a year at the ETH in Zurich during my PhD at Queen Mary.   Nik Wirth, who’d developed Modular Two and Pascal, was developing a new piece of hardware called Lilith. I worked on building the text editor for Lilith, which was going to be basically a Modular Two reincarnation of ded, a brilliant easy-to-use editor Richard Bornat and I had worked on at Queen Mary College.  The lab I was working in at Zurich was involved with the hardware and the serious programming language issues and implementing compilers and things. So they were pretty disdainful of user-interface problems, but they liked the editor.  Later, I spent three months at the University of Cape Town with one of my PhD students, Gary Marsden, who had ended up as a professor in Cape Town.”

“I spent quite a bit of time in New Zealand.  In fact I’ve lectured at every university in New Zealand, which was fun. I had some sabbaticals in the University of Calgary in Canada — I chose University of Calgary as that was where Brian Gaines had ended up (he had been one of my PhD examiners), and where I met Ian Witten, who went on  to the University of Waikato in New Zealand.”

From his own experience, Harold advises people to ensure they nurture and build a community of like minded people. He says: “Network. Easy first steps are things like going to seminars in other universities. There are lots of things you can do: imagine you are famous, what would you be doing? Well, one thing is, you’d be going to seminars and talking with everyone. So why not start going to seminars now? Build a community of people who are aligned with what you’re trying to do, or you align with what they’re trying to do and you can help them. You don’t ever have to work on your own.” And you make friends who will stay with your for life. I have fantastic friends all over the world.

Interviewed by Elisabetta Mori

Transcribed by Susan Nicholls

Abstracted by Lynda Feeley