Are human-centered design and human-systems integration the same thing?

Keeping IT Brief introduction:  From technology and acquisition to citizen services and mission execution, a culture shift is underway with federal government leaders increasingly seeking out solutions that just disrupt the way business has traditionally been done. Here we connect with those government and industry leaders driving this change and delivering the real results in support of their agency mission. This is Keeping IT Brief.

Susan Sharer: Welcome to Keeping IT Brief. We're going to interview Mary Quinn, a Leidos Technical Fellow and Human Systems Integrator (HSI). Mary, many of us have a basic understanding of human factors and how the best designs or the systems of products people use most easily and effectively have incorporated human factors into their design. Are human-centered design and HSI the same thing?

Mary Quinn: Well, Susan, that's a really great question. And they're terms that people confuse a lot and they are similar but they're not exactly the same. A good way to think about this is to think about the services and the products that we build. And we think about them as having three main parts, usually there's a hardware and software and then there's people.  Sometimes there's no hardware, sometimes there's no software but there's always people. And Human System Integration is just the tools and techniques that we need to understand the people part of that solution. And this includes a lot of people, it is not just the people who operate the system but people who maintain it, who support it, sustain it, live with it, pay for it, and, in some cases, even have to survive it. So we need to know in advance the intended and unintended consequences of our systems to the people part of the service or the product.

So HSI considers the human in every stage of the system development process. This means that we can make the most of the human capabilities and take into account human or environmental limitations that we need to be aware of when we're building the products and services. It's about designing a system that focuses on the human. So, there's about seven main disciplines to Human Systems Integration and Human Factors Engineering, which is one of the ones you mentioned, is really one of those key disciplines. So it focuses mainly on the physical and psychological issues of designing products and services for people. Other HSI areas, though, look at the number of people that are needed, the particular skills that people need to use the product or the service, what training they need, and sometimes the impact of the system on their workload and the changes to their working conditions that are brought on by the product or service. And sometimes we need to think about how to keep people safe while they're using these products.

You also mentioned human-centered design, and human-centered design is a specific methodology that we use a lot. And lately we've been noticing this terminology cropping up a lot in our communications from our customers. So we're pretty psyched that folks are beginning to notice and ask for our methodologies by name.

So briefly human-centered design has three main components. The first thing that we do is we have to understand the people, the people who will use our product or service, we need to know what their mission is, what their needs are, what their challenges are, what their specialized skills are, all that kind of stuff.

And once we understand that, the second part is we take it to designers and we help designers design the products and services that meet those needs within the mission, within the challenges. And we build prototypes and we pilot test the prototypes with operational end users and other stakeholders multiple times during the design stage so that we don't get too far down the wrong path before we can make corrections. And finally, once the systems are deployed, we test the impact on the humans. And more times than not, we recommend tweaks that can make the system better like changing the content or format of the training. And usually these tweaks are pretty minor and that is if we did the first two steps right. So this way people see their voice in the solution and they're more likely to use it without challenges. So that's a pretty long answer with lots of terminology, but HSI is a really broad field.

Susan Sharer: So Mary, is there an ideal scenario when HSI would be a difference-maker?

Mary Quinn: No. And I like to tell you Susan, I think every scenario that includes a human is the perfect scenario for HSI, but I can certainly talk about one that I think is a really good example. A few years ago, I was asked if I could help gather data during a major Army training event where a new technology that had been developed was going to be tested live for the first time. And this particular software had been under development for a couple of years and heaven only knows how many billions of dollars were spent developing the system. And I'll never forget how excited the engineers were when the test started. Finally, this cutting-edge technology that they worked on for so long, was so proud of, was about to be deployed and there was a pretty interesting group of observers. There were general officers from the military and there were experts from industry.

All these folks standing around when the test started. And so when the test started what happened was the Army analyst who was sitting in front of the screen and was supposed to use the software, all of a sudden he shook his head, he put his hands on the table and pushed his chair back, he threw up his arms and he shouted, "Where are my dots? I can't do my job without my dots." Whoops! What he was referring to was the format that he had learned to use to determine what sort of the vehicle was moving based on speed, the terrain, stuff like that. The user interface was nothing like what he was trained on and he didn't understand the new look and he certainly didn't trust the information in the format the engineers had design. HSI would have ensured that the engineers and end users had conversations long before that day.

Needless to say, there was a lot of very expensive rework involved in fixing the “missing dots problem”. All this could have been avoided had HSI professionals been involved from the beginning. This one is all three of those components of the human-centered design process I mentioned earlier, they didn't know the end user, they didn't develop design prototypes, they didn't pilot test. Well, they did pilot test but they used retired experts who really weren't knowledgeable about the latest technology. And you know if an analyst makes mistakes yes, someone could be seriously hurt. So they weren't willing to trust a product that didn't allow them to use their subject matter expertise. This is a huge issue and should have been addressed very early on during the requirements and design phase of that project.

Susan Sharer: That's a fantastic example. So when you describe it that way it makes me think about how you design systems but how does that apply to designing human-centered solutions for example, health-based solutions?

Mary Quinn: Oh, that's really a great question, Susan. I've got a good example that I can share with you. This example is the Leidos counseling center for excellence and it supports the military, family, and nonmedical counseling project that's called MFLC which provides counseling services in military locations around the world. You can imagine the military population has some very unique needs. So part of Leidos team is doing is to develop a better understanding of what the counselor's needs and wants are in order to facilitate the best result for they're very unique patients. And key part of this is what we call counselor for life model. As soon as the counselor's hired, we establish a baseline measurement of their skills and we develop a career plan. Then we continue to provide them opportunities to build the specialized skills they need to grow, maintain proficiency, and progress along their career path.

HSI helps tease out all those unique challenges and strengths of military family members, including challenges related to multiple deployments of the military members, families having to relocate frequently, and all those other stresses related to serving in the military. The counseling center of excellence identifies the best ideas and practices that add value to the influx and how to deliver the information and training so that the counselors can put it to practical use. Essentially, customizing training to fit the needs of the counselor given the needs of the communities that they're in. The center of excellence is also part of a continuous feedback loop that collects data and allows us to study the impact of their counseling and the counseling trends from site to site. So we're constantly reassessing our effectiveness and changing to make sure we're maintaining their effectiveness.

Susan Sharer: Excellent. So Mary, how do you make sure it works for the people? What types of methods do you use to validate success?

Mary Quinn: Well, as you might imagine there's lots of ways to test and validate our solutions but they vary based on the particulars, characteristics of the project, and the people that are involved in the systems. There are lots of tools, some are standardized, some are even automatic, and some we develop to fit the particular situation using behavior science techniques. Basically most systems are interested in understanding the impact of the system on the product in terms of changes and effectiveness, efficiency, and user satisfaction. But some systems require very different types of measures like safety, workload changes, measures of situation awareness, calibration of trust, and even survivability. This requires methods like direct and indirect observation of behavior by people who are trained in behavior science, who collect data through things like keystrokes mouse movements, eye tracking, even hesitations.  All this can be used are in pilot testing to provide feedback.

We also involve operational end users as much as can. We use very structured settings and scenarios as true to the user's environment as possible and we observe interactions between the user and the system prototypes. Of course, we also use stuff like surveys to gather opinions and some innovative ideas. The types and number of tests that we use are pretty much customized depending on what the system is designed to do and the characteristics of the people that are involved.

Susan Sharer: Awesome, that's fantastic information. So Mary, how does the person become an HSI expert? And what is your background?

Mary Quinn: As you can imagine from all the domains that make up HSI there's lots of approaches to become an HSI professional but the commonality is that it's critical to have some formal training in human behavior. You can't just say, "Hey, I'm a human and I know how humans think so I can play the role of an HSI expert." It is a bit more complex than that. But you asked me about my path, I'm not a human factors engineer, I'm a behavioral scientist. Oddly enough, I started out as a special ed(ucation) teacher. I worked with children and youth with serious emotional and behavior disorders and eventually with a lot of twists and turns became an expert in field of psychology known as Applied Behavior Analysis. I initially used this to design interventions and test effectiveness with individuals and small groups of institutionalized incarcerated children and youth. I didn't even know at the time that I was doing HSI but I was.

Since then, I've done lots of different things.  For instance, I helped build models to predict malicious behavior of organizations, investigated how to use virtual games to assess and mitigate cognitive bias and intelligence analysis and lots of other very interesting things. Right now, my focus is on understanding how humans and autonomy can best team up. My particular interest is trust, how we help people calibrate the trust they have in their autonomous partners? We don't want to over or under trust a system that we use to make our lives easier. Kind of like the first time I ignored Waze’s recommendation to get off I-95 at the next exit.  I distrusted the advice and I spent the next hour of my life stuck behind an accident. Now, when Waze says get off, I get off. It's all fascinating stuff really.  Its probably the thing I like the best about HSI. It really opens up many doors and you never have a chance to get bored doing this job.

Susan Sharer: I love it, Mary. So you have any final thoughts that you'd like to share with our listeners in terms of helping them decide when to add HSI into their design?

Mary Quinn: To get right down to it, everything we do is for someone. In some way every single service or product that we built has some kind of impact on people.  Everything. And to really be effective teams have to include some specialized knowledge about how best to integrate the human into the system and not the other way around, we don't build systems and hope humans change to use them.  We need to build the system that we can integrate into what humans currently do. Sometimes I think we get so enamored with technology that we forget that technology, at least for the time being, is meant to serve people. If it weren't for the people, we wouldn't need the services or the products in the first place. So, ultimately, it's about respecting people and caring about their needs, their challenges, their privacy, and then building those services and products that they can use and trust.

Susan Sharer: I love it. So it sounds like if you want to make sure a system or product optimizes human capabilities you need to make sure to include HSI in the design process across the board.

Mary Quinn: Totally agree.