Nov 16
< all postsiMate - Building a Mobile App for Mood Tracking
Reading Time: 11 mins
This isn't a tutorial but rather a showcase of the cool parts of a project
Introduction
In my second year of university, I had a module called Team Software Engineering. Where the task was to, over the course of the year, implement a substantial piece of software based on a given topic. Our title was “iMate”.
“iMate” is a concept which strives to be a mental health assistant for students and young people. It aims to help people track their mood and focus on mental health awareness.
It uses a tailored set of questionnaires to determine the user's mood (more on determining mood later). Then, it provides a set of tasks that, depending on the mood, can be used to improve or maintain the user's mood if it is already positive.
Features
- The user’s mood is tracked through a journal, which shows them how their mood has changed over time throughout the week.
- Provides the user with breathing exercises to ease anxiety.
- A Profile and Account system to allow users to log in and save and customise their experience.
- Provides the user with resources to support their mental health.
The Software Architecture
For this project, my team and I used C# because the group was more comfortable with that. We opted to build a mobile app because of its portability for a wide range of users, given that almost all (98%) UK adults aged 16-24 now have smartphones (Baker, 2024).
To build a Mobile app atop the DotNet framework, we used a multi-platform application UI framework called Maui (Microsoft, 2022) for the frontend, and the backend API, we used ASP.NET along with a PostgreSQL database.
The software architecture reflects the project’s time constraints, spanning two semesters; therefore, parts aren’t fully fleshed out. I’ll discuss this later.
Determining Mood
Computationally, determining something so complex as someone's mood is a challenging feat. A selection of different models attempts to capture the complex nature of human emotions. One such Model is the Circumplex model (Yarwood, n.d.). The Circumplex model, which measures Arousal and Valence, was developed by James Russell. Arousal looks at the activation energy level, and valence pertains to how positive or negative the emotion is (Murphy, 2024).
The difficulty in using such a model for an app is gaining access to the heart rate measures of the user (though a potential solution is using FitBit data, which is in turn limited by the need for a Fitbit).
A more simplistic yet powerful model is the PAD model, which measures features in three numeric dimensions and is more straightforward due to its existing usage in the mental health industry.
PAD measures mood based on three scales: the Pleasure-Displeasure scale (how pleasant one feels in a situation), Arousal-Nonarousal (how energised or soporific one feels) and Dominance-Submissiveness (whether or not the user feels in control or not).
The configurations for these are experimentally determined; for instance, boredom or fatigue can be described as one that is low on pleasure, arousal, and dominance (Murphy, 2024b).
Implementation
In order to implement this, we used a questionnaire based on the NHS Core-10 set of questions (Core-10, 2015). From these, we extrapolated a score based on a decision tree-style algorithm. The difference here is that we didn’t implement a full decision tree algorithm using entropy but rather implemented a tree manually, which would be traversed based on the questions the user answers; we continue this traversal until we reach a leaf node and, hence, a mood category.
private static Node root = new DecisionTreeClassifier(999, "Root")
.WithChild(new DecisionTreeClassifier(0, "Negative")
.WithChild(new DecisionTreeClassifier(0, "Intensity")
.WithChild((new DecisionTreeClassifier(1, "Angry/Disgusted")).Build())
.Build()
)
.WithChild(new DecisionTreeClassifier(1, "Passivity")
.WithChild((new DecisionTreeClassifier(0, "Anxious/Stressed")).Build())
.WithChild((new DecisionTreeClassifier(1, "Lonely/Sad/Depressed")).Build())
.Build()
).Build())
.WithChild(new DecisionTreeClassifier(1, "Positive")
.WithChild((new DecisionTreeClassifier(1, "Active")
.WithChild(new DecisionTreeClassifier(0, "Happy/Excited").Build())
.WithChild(new DecisionTreeClassifier(1,"Loved/Grateful").Build())
).Build())
.WithChild((new DecisionTreeClassifier(0, "Passive")
.WithChild(new DecisionTreeClassifier(0, "Relaxed/Bored/Sleepy").Build())
).Build()
).Build()
).Build();
A leaf node assigned a certain score to the mood for all 3 dimensions of pad. Since these scores aren’t set in stone we used a k-Nearest Neighbours (kNN) algorithm to decide which most closely aligns with the given moods.
I implemented a simple kNN classifier for such a purpose, which is outlined below:
using PADCoordinateVector = (float, float, float);
public class EmotionClassifier
{
private readonly Dictionary<PADCoordinateVector, string> _padRanges;
public EmotionClassifier(List<PadRanges> ranges)
{
this._padRanges = new Dictionary<PADCoordinateVector, string>();
// Construct a dictionary from the data from the database
foreach (var moodValue in ranges)
{
PADCoordinateVector vec =
new PADCoordinateVector(moodValue.valuePleasure,
moodValue.valueArousal, moodValue.valueDominance);
this._padRanges[vec] = moodValue.mood;
}
}
private static float Euclidean_Distance(PADCoordinateVector pointA,
PADCoordinateVector pointB)
{
// compute the Euclidean distance between points
float xSquare = (float)Math.Pow((pointA.Item1 - pointB.Item1), 2);
float ySquare = (float)Math.Pow((pointA.Item2 - pointB.Item2), 2);
float zSquare = (float)Math.Pow((pointA.Item3 - pointB.Item3), 2);
return float.Sqrt(xSquare + ySquare + zSquare);
}
private string KNN(PADCoordinateVector point)
{
// store the distances to each point
Dictionary<string, float> distances = new Dictionary<string, float>();
foreach (PADCoordinateVector coord in this._padRanges.Keys)
{
// compute the distance to the current point
float dist = Euclidean_Distance(point, coord);
distances.Add(this._padRanges[coord], dist);
}
// Sort the dictionary
var sortedDict = distances.OrderBy(distance => distance.Value);
// return the key of the smallest distance
return sortedDict.First().Key;
}
public string ClassifyEmotionByPAD(int PleasureDispleasure,
int ArousalNonArousal, int DominantSubmissive)
{
// normalise values placing range from 0-1
float PD_n = PleasureDispleasure / 10f;
float AN_n = ArousalNonArousal / 10f;
float DS_n = DominantSubmissive / 10f;
PADCoordinateVector coords = new PADCoordinateVector(PD_n, AN_n, DS_n);
// get the mood
string mood = KNN(coords);
return mood;
}
}
The above is a relatively standard kNN implementation minus any real training; it constructs a vector from data which is passed in via the API and classifies it based on its 3D Euclidian distance, which is computed as follows:
Or, more generally, in a D-Dimensional space:
The classifier places a new point onto the following 3D space:
App Design
The app looks like the image below; we aimed to make it simple and to the point while also keeping it engaging. We picked colours that were easy on the eyes but fell within the correct contrast guidelines based on the Web Content Accessibility Guidelines (WCAG 2).
We aimed to give users four easy-access options on the main page to help them get started and lead them in the right direction.
During development, we found that “Breathe” and “Meditate” were very similar in principle, so they were switched to “Breathe” along with “resources”.
The final product can be seen in the video below:
How we built it?
Frontend
The UI of the app was implemented using Maui. This MVVM (michaelstonis, 2022) framework uses a Model, View, ViewModel pattern. A model defines the data required by the page. For example, a user profile page may need fields such as a name and username.
class PersonalInfo
{
public PersonalInfo(string username, string fullName, int age, string gender)
{
Username = username;
FullName = fullName;
Age = age;
Gender = gender;
}
public string Username { get; set; }
public string FullName{ get; set; }
public int Age { get; set; }
public string Gender { get; set; }
}
A ViewModel is the link between the View and the Model; it provides functions to access and modify the data. We can use this to make API calls to fetch user data from the backend.
For instance, the truncated example below fetches the user information via a httpService and updates the class properties. Note here that the properties are marked as Observable, meaning the UI will reflect any changes if those values change.
Note: Commands are functions run by interactive elements from the View.
namespace iMate.ViewModels
{
partial class PersonalInfoViewModel : ViewModelBase
{
[ObservableProperty]
private string _username;
public ICommand updateProfileCommand { get; set; }
public PersonalInfoViewModel(IHttpService httpService) : base(httpService)
{
fetchProfileData();
updateProfileCommand = new Command(UpdateProfile);
}
private void fetchProfileData()
{
string token = await SecureStorage.Default.GetAsync("auth_token");
User Profile = await HttpService.FetchProfile(token);
...
}
public async void UpdateProfile()
{
var profileData = new ProfileDataModel {...}
string content = JsonSerializer.Serialize(profileData);
HttpService.UpdateProfile(content);
...
}
}
}
Finally, a View or, in our case, a page on the app can be a combination of a C# file and an XML file or just a C# file. We opted to use XML because it felt much cleaner and more organised. I won’t include an XML in this article because they are quite long and make for bad examples. However, if you are interested, the code link will be provided at the end.
A Page needs to take any classes you are passing by Dependency Injection (DI), in my case, the HttpService.
public partial class PersonalInfoPage : ContentPage
{
private PersonalInfoViewModel _viewModel;
public PersonalInfoPage(IHttpService httpService)
{
InitializeComponent();
_viewModel = new PersonalInfoViewModel(httpService);
BindingContext = _viewModel;
}
private void ChangePictureTapped(object sender, EventArgs e)
{
Navigation.PushAsync(new ProfilePhotoSelector());
}
}
I found this method of DI super useful because it let me just pass in the code I needed to make API requests into each class that needs it, and reuse all the code!
And moving on to the backend…
Backend
The backend API used ASP.NET and, therefore, the MVCS (Wikipedia Contributors, 2023) pattern. MVCS is based on the MVC pattern and consists of a model representing the data, a service providing the database interface, and a controller handling the API's routes. Since this was a REST API, the views part of MVCS had no purpose.
The Model part defines the database model, essentially a definition of an SQL table. A mood model could look as follows:
public class MoodEntry
{
[Key]
[DatabaseGenerated(DatabaseGeneratedOption.Identity)]
public int Id { get; set; }
[Required, ForeignKey(nameof(userID))]
public int userID { get; set; }
[DataType(DataType.DateTime)]
public DateTime date { get; set; }
public int? moodID { get; set; }
public MoodEntry()
{
// default constructor
}
public MoodEntry(int userID, DateTime date, int moodID)
{
this.userID = userID;
this.date = date;
this.moodID = moodID;
}
}
A service is the link to the dataset itself; we use this to abstract away any functions that make SQL queries. In ASP, we can use LINQ or the provided functions to access the database.
public async Task<IEnumerable<MoodEntry>> GetJournalSummary(string token)
{
int id = await GetUserID(token);
var now = DateTime.Now;
// Calculate the start of the current week (Sunday)
var startOfWeek = now.StartOfWeek(DayOfWeek.Sunday);
// Calculate the end of the current week (Saturday) - Add 6 days to get Saturday
var endOfWeek = startOfWeek.AddDays(6);
var query = await (
from Entry in ctx.MoodEntry
where Entry.date >= startOfWeek && Entry.date <= endOfWeek && Entry.userID == id
select Entry).ToListAsync();
return query;
}
Controllers defined the actual API routes of the API. For instance, a GET request to compute the mood based on some values can look like this:
[HttpGet]
[Route("api/v1/[controller]/")]
public async Task<string> calculateMood(int Pleasure, int Arousal, int Dominance)
{
// /api/v1/Mood?Pleasure=2&Arousal=0&Dominance=5
// get the ranges from the database
List<PadRanges> padRanges = (await _service.GetPADDictionary()).ToList();
EmotionClassifier classifier = new EmotionClassifier(padRanges);
string mood = classifier.ClassifyEmotionByPAD(Pleasure, Arousal, Dominance);
return mood;
}
Notice that the [controller] comes from the class name. Here, we show the MoodController. Therefore, the route ends up being /api/v1/Mood.
This was a basic overview; there is a lot more going on in the background than I made visible in this post, but I urge you to check the repo.
What I learned?
This project taught me a lot about various topics, including project management, Object Oriented Programming and Design Patterns.
I found it primarily interesting because it wasn’t an easy problem to solve or one that has really already been solved (at least in the open-source realm). So it required some crafty solutions to come up with what I would classify as a very basic application. It applied many different aspects of my degree, such as Machine Learning, UX Development and OOP.
Project Management Methodologies
We managed this over the span of around seven months using the Agile methodology; it taught me a lot about Agile and its shortcomings. Namely, the idea of scope creep led to the project taking a lot longer than we anticipated due to changing requirements. Another downside was the focus on short-term deliverables; since we needed to prototype every sprint, the software quality suffered due to a lack of QA.
Software Design Patterns
Having spent the entire year writing C#, I learned a thing or two about C# design patterns. I've never read the Design Patterns "bible" (Wikipedia Contributors, 2018), but I watched a talk or two from Uncle Bob.
Doing this project gave those ideas a rhyme and a reason but also allowed the downsides to resurface. Object-oriented programming gets in the way a lot. Granted, you get something that is nicely organised at the end, but you also end up with something that has a lot more complexity than is needed. Maybe OOP was the right choice here, but I can't help but think… was all that complexity necessary.
Limitations
Authentication
iMate is by far not perfect; I mean, look at my bootleg solution to authentication. I manually created a JWT token:
using static BCrypt.Net.BCrypt;
using System.Security.Cryptography;
public static string JWT(string username, string password)
{
string currentDateTime = DateTime.Now.ToString("ddMMyyyyHHmm");
string plainText = username + password + currentDateTime;
// Encode the string as a byte array using UTF-8 encoding
byte[] plainTextBytes = Encoding.UTF8.GetBytes(plainText);
var inputHash = SHA256.HashData(plainTextBytes);
return Convert.ToHexString(inputHash);
}
And then stored it with any logged-in user in the database:
public class AuthTokens
{
[Key, Required] public string token { get; set; }
[Required, ForeignKey(nameof(userID))] public int userID { get; set; }
public AuthTokens() { }
public AuthTokens(int _userID, string _token)
{
this.userID = _userID;
this.token = _token;
}
}
This was then used as a lookup table for a logged in user to get their profile. This isn’t at all a good solution; one could even call it the worst possible thing I could have done, but hey, at least the passwords were hashed.
A proper implementation would use an expiration date, refresh tokens, and claims.
Mood Calculation
Computing mood is challenging, primarily based on some questions. Human emotions are complex, and we can't change that; no matter the complexity of implementations, it will always be a generalisation.
To improve this, we would have to use better ML models or at least use more data for training. Other metrics like pupil dilation, heart rate, blood pressure, and body language could also be looked into. This could be an incredible Computer Vision and NLP project in the future.
What would I do differently?
The primary thing I would change is that I would use something other than Maui. To cut it some slack, we did use it in Beta; it felt more hindering than anything else. Maybe it was because we’ve never used it before this or had to uptake OOP guidelines. We’ll never know, but I would definitely opt for something more cross-platform and less complex.
A note on bias here: productivity in any tool improves the more you use it, but it doesn’t really make me want to go back and use it again.
In terms of the app itself, some other cool additions would be:
- A ChatBot you could talk to in order to help you figure out your mood.
- Gamification features like pets.
- A shop to customise your pet using the points earned from doing tasks.
References
Code is Available here: https://github.com/iMate-TSE/
Baker, N. (2024). UK Mobile Phone Statistics 2024. online Uswitch. Available at: https://www.uswitch.com/mobiles/studies/mobile-statistics/.
CE (2015). CORE-10 information. online Clinical Outcomes in Routine Evaluation (and CST). Available at: https://www.coresystemtrust.org.uk/home/instruments/core-10-information/.
michaelstonis (2022). Model-View-ViewModel. online learn.microsoft.com. Available at: https://learn.microsoft.com/en-us/dotnet/architecture/maui/mvvm.
Microsoft (2022). .NET Multi-platform App UI. online Microsoft. Available at: https://dotnet.microsoft.com/en-us/apps/maui.
Murphy, T.F. (2024a). Circumplex Model of Arousal and Valence. online Psychology Fanatic. Available at: https://psychologyfanatic.com/circumplex-model-of-arousal-and-valence/
Murphy, T.F. (2024b). Pleasure-Arousal-Dominance Model. online Psychology Fanatic. Available at: https://psychologyfanatic.com/pleasure-arousal-dominance-theory/.
Wikipedia Contributors (2018). Design Patterns. online Wikipedia. Available at: https://en.wikipedia.org/wiki/Design_Patterns.
Wikipedia Contributors (2023). Service layer pattern. online Wikipedia. Available at: https://en.wikipedia.org/wiki/Service_layer_pattern.
Yarwood, M. (n.d.). Circumplex Models. online psu.pb.unizin.org. Available at: https://psu.pb.unizin.org/psych425/chapter/circumplex-models/.