ICTworks

You’re in for a treat! This is the story of my two-year design journey to build X-kit Achieve! Mobile from my view as project lead, written with Nicola du Toit, the UX Designer.

On this journey we learned much about how to develop a mobile application, and get it adopted by South African youth. I hope you enjoy reading about our key design choices, lessons and findings, and find them helpful in your work.

X-kit Background

X-kit Achieve! is an established education brand in South Africa. The series consists of curriculum-aligned study guides and exam practise books, in both print and ebook formats. Given the rise in mobile uptake in the country – by 2015, 38% of the population were mobile internet subscribers – our challenge was to design a mobile version that complemented the existing book series.

The mobile version needed to be suitable for users aged 13 to 18 and the language and tone appropriate for learners studying in their second or third language. Our personas included learners who were struggling academically and needed help (and were motivated enough to seek it), and achievers who wanted to be stretched in their schoolwork.

The minimum viable product (MVP) consisted of a mobile website with two views: a basic WAP site for low-end devices like the Nokia 5130 (still popular at that time) and a richer HTML site for phones with more current web browsers (Opera, Android, iOS, Windows Phone, etc.) Both versions could load on tablet and PC browsers, of course. If the mobile site generated interest, we would develop an Android version.

X-kit Initial Features

Then, and now, learners are able to login, select their grade and view the subjects available for that grade. Each subject has 10 topics, one of which is free. If a user buys a grade/subject combination, she can do multiple choice quizzes of 10 questions each, in three scaffolded levels of cognitive difficulty (Practise your basics; Apply your skills; and Master your topic).

Because the quiz questions are drawn from a large pool of content, the learner can practise a certain topic as many times as she wants to with new questions. The quizzes have optional hints and are automatically graded for immediate feedback. Learners can view fully worked solutions to the questions they’ve just answered. There is also theory content available for each topic.

Even in the Minimum Viable Product we tried to “design for delight”. Learners can track their progress and also their overall ranking on assorted leaderboards. They can level up (if they get 80% of more on a level) and earn badges as they progress. The leaderboards and badges add a gamification aspect to their learning journey.

Learners can share their quiz results on Facebook or Twitter (although this functionality hasn’t been used much). They can also customise their profile information and choose from a set of avatars.

We decided to not let learners upload their own profile photos (our legal team saw alarm bells!), but the feedback from focus groups was that learners liked the collection of ready-made avatars (because “there was something for everyone”). We also provided a simple but personalised message on every login, which the learners loved.

Content Length

The core unit of learning in X-kit is a quiz question, consisting of the question itself and either multiple choice or free text response options. Text and images (e.g. graphs or diagrams) can be used in the questions and the answers.

In the later grades of high school the questions can become complex and long. But on a mobile site, how long is too long? We conducted lean user testing with 12 learners at one high school in Cape Town. Admittedly, this was not a big sample, but given limited time and budget, it was good enough.

It turns out that the ideal question length was between 100 and 130 words. If questions were longer, it was better to break them into multi-part questions, to minimise unnecessary scrolling and cognitive load for the user.

Direct to Customer

X-kit Achieve is what is called “supplementary” educational material, meaning the series is not on the official government textbook list. Not being “core” material, it was mainly bought by individuals. We initially continued this focus by targeting private users.

X-kit mobile is a paid-for product, based on a try-before-you-buy model. The first level in each content area is free, the user can then purchase more content with airtime. But what if the user doesn’t have airtime?

We chose to separate users from buyers. When purchasing a topic the learner needs to enter the mobile number of the person responsible for payment, e.g. a parent or teacher.
The responsible person receives an SMS and needs to confirm payment before the learner can access the topic purchased. Later we allowed payment with credit cards and via bank transfer.

It soon became apparent that the platform could just as easily have value for schools. A new institutional model for the project thus aimed to scale the mobile version.

Institutional Model and Teachers

Core to the institutional model was adding teachers as a new user group. The key was to support teachers in their daily activities, not increase their workload. This meant taking a less-is-more approach. We decided on the following features:

• Teachers can assign ready-made quizzes as homework for their entire class.
• Dashboard view to track assignments – how many learners have completed the assignment and scores to see what they struggled with.
• Ability to drill down to individual learner progress per topic/subject.
• Learners receive reminders to complete homework.
• Schools and classes can have their own leaderboards (something learners asked for).

It was critical that all of these features be accessible from a mobile phone – for teachers as well as learners.

In field studies at our pilot school in Johannesburg, we found the following:

• The teachers needed constant support: initial training, ongoing training, courtesy calls and reminders, in-person visits. Many of us know this to be generally true in ICT4D implementations.
• Teachers wanted to create and assign their own custom quizzes, not just assign the ready-made ones in X-kit. They wanted more autonomy as users.

Transition from mobile to app

In 2015, based on the initial success of the X-kit mobile site, it was decided to build a complementary Android app. At the time Android devices were becoming increasingly common in the South African market, and it was a good opportunity for the team to get to grips with app design and development.

The app design process started with a detailed review of the existing mobile site. The app needed to look and feel related to the existing site, but we also had an opportunity to provide a clean, fresh and appealing interface that conformed to Google’s Material Design guidelines.

Additionally, the app needed to cater for offline usage, with intermittent syncing of data. Although the mobile website was optimised for minimal data usage, users still complained about the costs of data and access. Offline mode and syncing meant that users could practice on-the-go, and sync their progress and results when connected to wifi at home or at school, for example. (Since then Google has added a resource for offline app design.)

We iterated through several versions of low- to high-fidelity wireframes, to establish the user journey and overall user experience of the app. For each iteration we tried to do some form of usability testing (one-on-one observations) or user research (e.g. interviews and focus groups), to get input from learners and teachers at every step of the design process.

We started out with hand-drawn paper wireframes. We then used LucidChart for mockups and mapping out the user journey, and UXPin to design an interactive prototype. The UI design work happened in a few iterations using Adobe InDesign.

Facilitating individual usability tests and focus groups with teenage learners turned out to be challenging – learners were often reticent (being more comfortable in a group of their peers) and had to be put at ease before they felt able to provide feedback with confidence. However, their input at several points in the design process gave us invaluable insight and feedback into how they felt about the new app.

Once the app was live we facilitated onboarding and training with high school learners at a “No Fee” school in a low-income area in the Eastern Cape. In collaboration with a very enthusiastic and innovative Mathematics teacher at the school (working against all the odds to help his learners get good Maths marks), we got Grade 11 and 12 learners signed up and using X-kit (either the mobile website or the Android app, depending on their device).

We encountered numerous challenges in this space, the greatest one – yet again – being internet access. While most learners at least had a low-end mobile device, they did not have data. We provided a 3G router for learners to connect to but the network coverage in the area was surprisingly unreliable so we battled to get all the learners online and registered for X-kit.

This school had very little in the way of learning resources like textbooks, posters, computers, projectors in classrooms and so on. Seeing these learners’ daily school environment certainly highlighted the need for supplementary, digital educational material – like X-kit Achieve Mobile – that can be delivered directly to learners and support them in their studies.

However, there are still potentially significant barriers to accessing e-learning materials outside of main city centres in South Africa. Hopefully these barriers will be reduced as reliable mobile network coverage increases and mobile data costs are driven down over time.

Reporting: Less is More

When it came to spec-ing and developing reports there were two schools of thought. One, from the data scientist at Pearson was to collect every data point because later we may think of reports to run. Two, from our third-party developer: initially collect and report only on the basics. Later, when you actually know what you want to know, develop those reports (and cut your losses with uncollected data).

As we’ve said before, too much data can overwhelm, not reveal insights. Our developer was right: after building the app we only really used about 20% of all the reports and dashboards that we had developed, and spent more time on creating the new actually-needed reports.

In retrospect, what would have helped was to define the x number of core questions that needed to be answered and only report for those. For example:

• How many users have signed up?
• Which subject and topics are most popular? (To direct where our content development resources could best be used.)
• How many users take at least one free quiz? (55% initially. This led to the next design hypothesis for validation: reducing the number of clicks from homepage to quiz start to 3 would increase the 55% to 80%.)
• For each subject, how many questions are answered correctly? (To provide guidance to the content team on whether their questions were too easy or too hard, and how helpful the hints were.)
• As a paid-for product, what was the conversion rate from free to paying user?

Final Thoughts

Working in an agile, iterative manner with an interdisciplinary team of developers, designers and content specialists was extremely productive and interesting. While this kind of collaboration is the norm in software development environments, it wasn’t common in the publishing industry at the time.

In a traditionally print-based company, digital was fairly new to many teams and throughout the process we did internal training and change management. For example, Nicola presented a lunch-and-learn session on the design journey from mobile site to app. The change management on the inside was almost as important as the outward facing end result.