Background
What language motivates people to apply for a new career in the energy industry? That’s what we set out to answer when a major energy company was re-launching their "careers" website and wanted to validate that the content and direction of the page were appropriate before investing in code. We conducted a small-scale qualitative study with target audience members over a three-week period.
By the end of it, we had validated that the content was resonating with the people they wanted to reach, but that the reading level was too high for the audience and the calls to action were not clear enough. Because of the study design, we could pinpoint the exact passages which were difficult to read and could give concrete suggestions for improvement based on language that participants naturally used.
Our Process
We started with a collaborative discovery session to uncover assumptions the content team had made and to establish goals for the research effort.
We interviewed stakeholders to understand the current problem space and to understand their goals for the project.
Audience Analysis
- Young people with little to no job experience (HS-aged or recent graduates)
- Experienced workers looking to switch to the energy industry
Career mentors and counselors - Secondary Audience
The content team wanted to learn more about how their primary audience went about finding careers – where they searched, who influenced them, what language they used. They also wanted to test sample content they had written to see if it aligned with what their audience needed.
Like our design process, our research process is collaborative. As we talked through the needs of each audience group and the project goals, it was clear that we should focus on understanding the career search patterns of potential employees first and consider the needs of career counselors second.
Recruitment for this project was straightforward; we worked from the company’s updated contact list of community partners and people interested in their programs. We knew that if this group of people provided negative feedback, it would be a clear indication that the content should be changed.
- We used open-ended questions so participants could talk about their career journey in their own words. Starting with this segment let us get an unfiltered look at the type of language people used when discussing what they looked for in a career. After the open-ended questions, we had participants give numeric ratings for specific aspects of careers - comparing those numeric ratings with what participants said earlier in the interview added a degree of reliability to the responses.
- We presented sample content using Google Slides. To avoid accidentally influencing people's responses, we did not show the content until after the exploratory questions.
Every research study has to deal with bias. In this case, using language that was contained in the content could create “anchor bias,” a phenomenon where people rally around a word that they wouldn’t ordinarily use to describe something. It could also create “confirming bias,” where people say what they think you want to hear. Because we wanted to know what language people use when talking about careers, it was extremely important that their answers were their own.
Findings
Once our participants were lined up, it was time to see what people had to say! While we conducted interviews with both audience segments, we only tested content with potential job seekers (the primary audience). Here’s what we learned:
Relatability
The content highly resonated with participants. Participants highlighted the same key phrases, which also aligned with their stated values during the interview portion. As they highlighted, many participants stated, “Yes! This is true!” and “I like that one too”. Only one participant used the red highlighter to indicate that they felt a phrase “was not that relevant” in one of the three tested passages. Overall, this technique demonstrated that participants related to the content and allowed us to pinpoint the key phrases that made it relatable – valuable information for content creators.
Task
Instruction: "Highlight what you like most in green, and what you'd like to change or remove in red"
Result
Only one participant used the red highlighter to indicate that they felt a phrase “was not that relevant” in one of the three tested passages.
Overall I like this, it’s nice and matches with what I use. I like being able to create new or add to an existing agreement.”
- Sharon, SVP Coaches Console
”
“
“Putting all this (content permissions) within the agreement is amazing - we’re getting it all done in one spot! I like it.”
- Kate, Primary User of Coaches Console
We found 3 of the 5 passages to be over the content team’s desired 8th grade reading level. Using the results from the “Fill in the Blank” section, we highlighted the most difficult passages and recommended editing those passages for clarity and simplicity.
Task
Instruction: "When you get a blank, say what word you think should go there."
Result
We found 3 of the 5 passages to be over the content team’s desired 8th grade reading level.
Of the three CTAs, only one aligned with participants’ expectations. Participants confused the second and third CTA for each other, which suggested that the language was too similar and needed to be rewritten. We recommended using verbs for each heading and replacing the word “find” in the third CTA with the word “match” based on people’s assumptions for the word “find”.
Task
Instruction: "Tell us what you would expect to see if you click the button in each section"
Result
Of the three CTAs, only one aligned with participants’ expectations.
Final Takeaway
With the research in hand, the content team was able to make important revisions to the language on the site and improve their overall conversion.
Research doesn’t have to be time-consuming and labor-intensive to be effective. It’s possible to learn valuable information using simple tools on a limited timeframe as long as your research method matches your study objective and you approach the challenge with flexibility.
