How usability testing can improve Toronto’s shelter intake system

Earlier this year, Code for Canada worked with the City of Toronto’s Shelter, Support and Housing Administration (SSHA) team to conduct usability testing on an updated version of the intake form for their Shelter Management Information System (SMIS), a centralized information management system that is used throughout Toronto's emergency shelter system.
 

Why usability testing matters

“We were looking at making a bunch of changes to SMIS and it’s important to get meaningful feedback,” said Roshni Patel, Business Analyst with SSHA. “User testing helped us make sure that what we are developing is actually working for our shelter support staff and people experiencing homelessness.”

Shelter staff use SMIS to intake clients who arrive at shelters in need of support services. In addition to gathering contact information and support needs, SSHA has expanded the capabilities of SMIS by making the data collection process more comprehensive. This helps shelter staff and case managers better understand clients’ needs so that they can provide the best support possible, and further informs the strategic directions of the Division

“We have to find the balance between the frontline user experience and the broader strategic vision of what the system needs to do,” adds Stephanie Malcher, Manager of Strategic Programs. “But in order to reach that balance, we need the voices of everyone — policy experts, front-line staff, and people with lived experience of homelessness.”

Designing the test

Enhancements to SMIS have been fairly limited since the system was introduced in 2009. In recent years, SSHA has demonstrated a commitment to making improvements to the way SMIS supports staff to do their job, and supports people to move from homelessness to permanent housing. This includes changes based on extensive user research performed by a team of Code for Canada fellows in 2019.

The recent usability sessions included intake staff from a select number of shelter sites that piloted the latest round of enhancements at the end of 2020. The updated intake form was considerably longer than previous versions and included additional sections, such as questions on demographic data, mental health, and support needs, among others. The updated form also included prompting questions and explanatory text for each section to assist staff with the intake process.

Code for Canada's user research team held seven hour-long testing sessions with staff from three shelter sites serving the following sectors: 2SLGBTQI youth; mixed gender; and men only. Staff from SSHA, including IT, sat in on some of the sessions. Due to physical distancing requirements as a result of the pandemic, all sessions were conducted virtually.

“It was a great opportunity for IT staff to feel connected with their work. They don’t often get the opportunity to hear directly from the people using the product,” added Stephanie. “Our team is always trying to make sure we’re connecting people with the whole process. Having the folks who are writing the code hear from users firsthand about how their work can lead to someone moving from a shelter to a home is really impactful.”

What we learned

Overall, shelter intake staff prefer the updated version of the intake form compared to previous versions due to the amount of detailed information collected and the user-friendly interface. However, one of the largest challenges they faced is that the intake is much longer than previous versions of the form; the added length of intake can cause clients to feel frustrated or tired, making it more difficult for staff to do their jobs.

“How people were using it didn’t actually align perfectly to what we had thought,” added Roshni. While SSHA expected shelter staff to use the form as a guide and focus on the most relevant questions, the testing sessions revealed that staff were completing the entire intake form.

The inclusion of prompting questions and explanatory text have been valuable in helping participants obtain personal information from their clients. In particular, this feature has been very helpful for staff who may be newer to the intake process.

While there were no glaring usability issues, a few shelter staff testers did note frustrations around the location of the save button and lack of confirmation after saving.

“It’s not something we would have thought about but we heard from users it was quite frustrating to scroll all the way up and have no confirmation all their work has been saved,” added Stephanie. “Learning all these small changes make such a big difference was surprising and also a good reminder of ‘we have to think about all that.’”

What’s next?

SSHA has already begun making changes that were recommended in the report, including adding drop down options, changing the location of the save button, and improving the UX design.

SSHA will continue to conduct user research and release as part of their development roadmap. The transformation will roll out in interim releases. Each release will be preceded with, and informed by, user research work.

“We're embedding user research in our plans so every change we make is informed not only by policy or funding directions and technical fixes but also informed by the user’s perspective,” said Stephanie.

Interested in conducting inclusive usability testing for your products? We connect technology creators with diverse and underrepresented communities to help you better serve your users needs. Learn more about how Code for Canada's inclusive user research service can help.