When you think of an institution as storied and grand as the Metropolitan Museum of Art, terms such as “agile” and “DIY” might not be the first to come to mind. But at the Met, staff from different departments are working together to employ rapid, low-cost research methods to better understand the needs of museum visitors.

For this post, I interviewed two staff members from the Met, Liz Filardi and Karen Plemons, about how they are using rapid research methods to inform the development and design of apps, websites, and digital games. The methods and approaches they’re employing can be applied not only in large institutions like the Met, but in small museums as well.

Liz Filardi (left) and Karen Plemons (right)

Liz is in the Digital Media Department, and manages mobile projects in all stages of development. Karen is in the Education Department, and oversees educational research and evaluation efforts. Liz and Karen are their own “guerrilla team,” doing user research and usability evaluation on a shoestring. The methods they use include card sorting, think aloud user testing with clickable prototypes, interviews, surveys, and visitor observations.

Dana Mitroff Silvers (DMS): How did you start working together?

Karen Plemons (KP): Well, I kind of accosted a colleague (in the Digital Media department) in the elevator one evening! I said, “You know, I have some really good data on our younger visitors. And I know you’re trying to capture younger visitors. I can be a resource to you.” I explained that I can help support the work they’re doing in the Digital Media team.

Liz Filardi (LF): And then when I heard about Karen and her work, I started to think about the importance of user research in the projects we’re doing in Digital Media. There is a big emphasis in the Digital Media department on understanding users. We even have a new position in our group, a Digital Analyst.

Staff reviewing visitor data in an internal workshop. Image courtesy Metropolitan Museum of Art.

Now, when I’m kicking off a new project, or we have to make a design decision and we know that there are a lot of stakeholders with vested interests, I’ll approach Karen and ask, “How would you go about this?” Then, Karen develops the testing instrument, and we both carry out the user research. Collaborating with Karen has been one of my best resources to better understand audiences and visitors.

KP: And now I’m working on three projects with the Digital Media department. There has been a real shift in how Education and Digital Media collaborate and work together. We’re really making an effort to break down the silos between departments. And the ironic thing is we work on the same floor. It’s not even like we’re in other parts of the building. There’s another team in Digital Media that is a 15-minute walk away, but we are on the same floor! So it’s not difficult for me to be in an elevator with someone and say, “Hey, I can help you with that project!”

Visitors performing a card sort activity.
Image courtesy Metropolitan Museum of Art.

DMS: Can you give me an example of how you’re using user research on a digital project at the Met?

KP: We used card sorting for the content in the Met App. A card sort is a great way to find out how users group things together and what content is important to them.

In the app, we have exhibition information, event listings, permanent collection objects, and our Twitter feed. To help determine the mix of content that was most useful, we did card sorts with museum visitors. It helped us understand how visitors made connections, and what was most important to them.

After we had an idea of the potential content in the app, we created laminated cards, and gave visitors tasks such as, “Prioritize the five most helpful or interesting things you see here” and “Prioritize the five least helpful or interesting things you see here.”  We would then have participants talk through why they made those choices.

Twitter content inside the Met App.

And we noticed a trend: that visitors were grouping the cards related to selfies and social media in the “least helpful” category, saying they didn’t want to use social media in the museum. Yet, we also noticed that the Twitter cards were testing really well because people were drawn to the images in combination with short, colloquial text (e.g., Tweets about artists’ birthdays). Visitors did not seem to notice or mind that the source of this content was social media.

It’s possible that if we had just done a survey, we may have concluded that users didn’t want social media in the app and left it out entirely. But through the card sort, we were able to understand nuances around different types of social content.

DMS: What does a typical project look like for you?

LF: I might call Karen on a Monday. Then we’ll meet on a Tuesday and put together our instrument and testing plan. We’ll think about what audiences we want to target, and how can we engage with that audience based on what programs are happening at the Museum. We’ll do testing on a Thursday or a Friday. On the following Monday, we will synthesize the results, and put together the design recommendations.

KP: User research and testing doesn’t have to be super expensive and lengthy. And you don’t have to have the biggest sample size. The nice thing about working inside a museum is that we can go downstairs and engage with visitors in the galleries—literally, go downstairs and do some research and testing. In an ideal world, we’ll also find time to go outside the museum too. For example, we’ve done usability testing at the Chelsea Market at lunch.

DMS: What does “design thinking” mean to you?

LF: As it relates to user research, it’s about putting yourself in someone else’s shoes, understanding their needs, and being willing to go from there. Designing from this place often means you have to set aside your preconceived ideas. It often also means making stronger decisions more quickly.

KP: We often talk about constraints as fueling creativity. When working on projects, we often have to be within a box (as determined by funding, or time, or other limitations). But we can be very creative within this box. Which is where we can bring in and leverage what we know about particular users and audiences, and craft the best solutions for them.  I tend to think of design thinking as a tension between constraint and creativity, where research becomes key. Through constant agile research and testing, we are always learning and integrating our findings.

Follow Karen on Twitter and Liz on Twitter.

Also, check out the slides from their workshop at the 2015 Museums and the Web conference, An Introduction to Agile User Research and Testing.