Extended reality for remote consulting: use case from a breast care unit
Sharing the success of a recent publication on the Royal College of Surgeons Bulletin on a remote consultation pilot using the Microsoft Hololens 2 device
The reason for this application experience is to try and reduce the anxiety that adolescents feel when coming into the hospital, going into the unknown is anxiety triggering for nearly anyone but for an adolescent with anxiety it can feel like life or death, which is why this orientation is needed before they visit the hospital or when just coming into A&E. The added features that are implemented will also be used to help deescalate the adolescent if they are overly anxious and help parents understand what their child is going through.
The application has an easy-to-use main menu, that will allow you to navigate through the different sections of the application. To orientate the user there are 360 images of the ward and A&E within the hospital, this will allow the user to click to move through the areas and see what the rooms look like, there is also a map of the area that can be displayed allowing the user to have a full understanding of where they are within the space at all times. There will also be two more sections that will be dedicated to information such as the leaflets handed out at the hospital and CAMHS, this will be for the parents and adolescents so they can go through the information and terminology together with special dedicated parent tabs that have all the hospital information they may need such as the car park charges and where the cafés are located on site, as this is information that could make a visit less stressful for both parent and adolescent.
Breakout rooms were added to give the user a chance to have a break from the experience if it is getting too much for them. The user will have a variety of calming experiences to choose from such as colouring, mosaic tile creations and breathing exercises. These experiences are available from the main menu and in the quick menu tab at the bottom of the device allowing for quick access to help the user to deescalate their feelings.
“In the UK, a little over 1 in 10 of us will be living with an anxiety disorder at any one time – that’s over 8 million people. But everyone’s experience of anxiety disorders is different.” (Health, 2022) There many forms of anxiety, generalised anxiety disorder also known as GAD is the most common form of anxiety, but there are others such as panic disorder, social anxiety disorder, phobias, and agoraphobia. these are just a few but there are many variants of anxiety that can present in anyone.
The NHS have specific guidance for whether children are showing symptoms of anxiety. When small children have anxiety, they don’t understand how they are feeling or how to show how they are feeling this. Some parents notice their children becoming irritable, tearful, or clingy, have difficulty sleeping or are waking in the night, they may start wetting the bed, have bad dreams. (Anxiety in children, 2022). However, parents with older children may notice that their adolescent lacks confidence to try new things or seems unable to face simple, everyday challenges such as seeing friends, going out in public or going to school, they find it hard to concentrate, have problems with sleeping or eating, have angry outbursts, could have a lot of negative thoughts, or keep thinking that bad things are going to happen.
Anxiety effects children differently to adults, where adults can be affected by their anxiety with everyday activities such as with their work, finances, and friends. Children have different issues to overcome such as school, friends and playing which can affect their growth and mental development. During the COVID-19 pandemic there was an incline in children’s mental health the UK parliament investigated research done after this pandemic on this topic and found, although limited research had been done on the mental health of children during the pandemic, most studies that had been undertaken found on average children’s mental health had worsened during the COVID-19 pandemic (Children’s Mental Health and the COVID-19 Pandemic, 2021). Therefore, a focus on children’s mental health needs to be looked at and helped as the pandemic had a poor effect on children in many ways.
This application is being built in Unity, which is a game engine toolkit. This is not traditionally used to build applications as it is considered to have too much overhead. Using the UI toolkit will help with making the application which can support future development if the NHS decide to include AR, VR or interactive media, as Unity can support these features without any additional libraries or plugins. With the need for the application being widely available, Unity is the ideal platform to build the application in as it means it can be pushed to multiple devices without changes to the core application code.
When designing an application, it is very important to think of the user throughout the process knowing their needs, wants and interests. I know the need is to reduce anxiety before coming into the hospital, I know the user will want to have all the information and get through the hospital process as quickly and efficiently as possible and with such a wide age range, the users interests can vary but their main goal is still the same to get through the hospital experience as quickly, as possible as people with high anxiety will want to go back to their comfort zone quickly.
Therefore, the focus for the UX of the application was to make it as simple to use as possible as the user will already be going through a wide range of emotions when they first encounter the application. Using the NHS icons and matching them as closely to the relevant titles so when you look at the icon it portrays what you should see when you click on that area, for example the accident and emergency walkthrough having an ambulance as its icon as ambulances are mostly related to A&E. As this application is for children I did want to make the UX more fun and give the application a more childish feel, but as stated by Hodent in her book ‘What UX is Really about?’, she states about the problem being what is fun for UX so you should consider the “engage-ability” more, so we may not agree on what fun means but we know we are engaged when having fun (Hodent, 2021) making the application more engaging became the way forward, while still allowing for the information to be the key element of the application.
The application is very text heavy, so the typography needed to be a priority ensuring all pages had the same line spaces and padding either side of the text this is known as a grid system ensuring the ease of navigation for the viewer (Samara, 2017), this has been done for the text to ensure the reader can easily read the information that is being provided, the grid system was also used for the navigation as it meant the same padding was kept between each icon making it easier for the reader to navigate around the application.
The breakout rooms are designed to help reduce anxiety. I had the opportunity to speak to a play therapist, Beth Chapmen. During this meeting we spoke about the different things she does during a session with the adolescents to get ideas of what could be translated into a digital format, during the conversation she gave me lots of helpful tips and insights into different therapy methods she provides. One that really stuck out to me was a mask activity where the kids would be asked to decorate the outside of a mask with how they feel on the outside and what they show people, then on the inside of the mask they decorate it to how they feel on the inside. I thought this was a really interesting and expressive task for the adolescents and with more time I felt it could make a very interesting breakout room.
The main play therapies that stuck out and could be turned into a digital form where colouring sheets having simple patterns like mandalas meant the kids could easily focus on the shapes and different colours. Beth Chapmen mentioned to not be afraid of giving a variety of colours, as choice in colour is important for the adolescents this also led me into looking how colours effect emotion as it may be interesting to look at if an adolescent uses mostly red for example, does this mean they are angry or mostly blue does this make them sad. What I learnt during my findings was that colours can mean a wide range of things, for example, creative bloq shows how colours and web design work together and they show yellow as happy, friendly, warming (Cao, 2021). Whilst in a paper by Fugate and Franco where they investigate which colour represents anger, when looking at figure 2 it shows yellow as being many different emotions happy, jealous, sad, surprise, calm, contempt, envy, fear (Fugate and Franco, 2019). These are two different views on the colour yellow and the emotion it can provoke showing that colour can mean anything depending on who you are, which is why it was important to add such a wide variety of colour to give the adolescents options to express themselves.
The next game was mosaic tiles, this game allows the adolescents to make patterns. This was supposed to mimic stain glass windows, in an in-person environment the play therapist would ask them to cut out shapes from different images or coloured paper and allow them to stick them down however they feel. This activity allows them to focus on something other than how they are feeling and is used as a distraction, this felt appropriate for the application.
Mindful breathing was mentioned a lot during the talks as it is the number one and most effective form benefiting your mental health as it is a body mind exercise that when done will help you focus on something other than your emotions (Ma et al., 2017). This was a simple breathing exercise with rings pulsing in and out and encouraging you to click on the rings at the correct time of breathing.
These games where designed by me but were outsourced for specialist development. I worked on my skills writing briefs, I wrote how each game should work as I was going to finish the design, I oversaw all stages of development which improved my management skills and used the industrial placement funding to pay the developer.
The original posters for the NHS that are full of QR codes did work well for my fact checking of information to be included in the information and CAMHs section of the application, as it was full of the relevant information that was asked to be included. It worked well as I did not need to keep asking for information to be checked till the very end as I worked through each QR code and extracted the relevant and appropriate information.
After extracting all the information I could from the poster, I then compiled it all into a word document which I then shared with the NHS paediatric team giving them the opportunity to add or edit any information as they want. This process was longer then I hoped as it took a while for the team to come back to me with any answers, but once done the section was well populated with informative text for parents and adolescents.
Once the decision was made for an application, the NHS Paediatric team and I had lots of discussions on what to prioritise and how the application should look. We all had lots of thoughts and most the comments from the NHS staff where ‘I know nothing about technology but is this possible’. After getting the team over their own technology anxieties I felt the process was a lot more fluid and they were willing to share their bigger ideas with me, which helped us narrow the application functions down to some main points, that we wanted a guided walkthrough, we wanted key information that you get on leaflets to be included and we wanted games to help give the adolescents a break.
From this I was lucky enough to go in front of two ex-CAMHs patient and get their opinions on the application. We started with me giving a PowerPoint on what we wanted the app to do and achieve, I had a questionnaire made to record their responses but sadly this wasn’t used at the time, I manged to get some typed notes from what was said by the two CAMHs patient, overall the feedback was very positive and showed that this application would be helpful. Another point they brought forward was how their phones are confiscated on the ward so the application may not be helpful for if you are going in for a mental health reason, this then sparked a conversation on if this was correct to do in the situation so overall this test group was helpful for both me and NHS staff.
Moving on from this test group, I made a wireframe using Adobe XD, where I showed how the app would look and function once this was approved the application development could fully begin. During the application development it wasn’t possible to have any more meetings with the ex CAMHs patients as the NHS team were struggling to get appointments made or deadlines were missed due to a surge in workload for them. I tried to ensure this didn’t affect the application too much using the i-DAT team to check all functionality and design of the application as I went along, I also asked family and friends to test the app to see how users would react to the application and to check its ease of use.
A forum was created to get opinions of the wider public, I made this form in word so it could be done in paper form and in google forums so there was an online presence it also meant I could get the information back quickly to try see what the public in the A&E and ward waiting rooms would like to see and what they would like to know so all the relevant information and characters could be included. There are very little responses to this questionnaire, I sent a few emails to the relevant NHS staff to see if this could be pushed more but sadly nothing to major happened. Due to the contract with the NHS, I could do no further outside testing as it was stated I couldn’t widely share the development of the application and I had to get permission to show anyone from the public.
More user testing was needed throughout this project but overall, the immersion and information providing of the application is at a high standard for a Beta application. Once more user testing has been achieved and any final adjustments are made, the application will then be ready to roll out within the hospital. So far, the application has had some success as it has been shortlisted for the HETT Unexpected Innovation Awards 2022. This is an award that celebrates new digital and health innovations.
Further future developments for the application would be adding more areas such as MRI, X-ray, and other areas the NHS want in the application. Linking pre-approved games to the breakout area would add to the number of activities available to the adolescents, it could also be a paid partnership that would allow for more funding for the application as well as providing the NHS approved applications more users for example health for kids which is promoted by the NHS has a game to help children with their worries. The game lantern allows children to write down their worries and attach it to a lantern that is then released giving the children a feeling of letting their worries go whilst reminding the children it’s okay to talk to someone about how you are feeling, simple games like this could make a big impact on the application and the adolescents.
Cao, J., 2021. 12 colours and the emotions they evoke. [online] Creative Bloq. Available at: <https://www.creativebloq.com/web-design/12-colours-and-emotions-they-evoke-61515112> [Accessed 15 August 2022].
Fugate, J. and Franco, C., 2019. What Color Is Your Anger? Assessing Color-Emotion Pairings in English Speakers. Frontiers in Psychology, 10.
Health, M., 2022. What is anxiety? – Mental Health UK. [online] Mental Health UK. Available at: <https://mentalhealth-uk.org/help-and-information/conditions/anxiety-disorders/what-is-anxiety/#> [Accessed 01 August 2022].
Hodent, C., 2021. What UX Is Really About. p.46.
Ma, X., Yue, Z., Gong, Z., Zhang, H., Duan, N., Shi, Y., Wei, G. and Li, Y., 2017. The Effect of Diaphragmatic Breathing on Attention, Negative Affect and Stress in Healthy Adults. Frontiers in Psychology, 8.
nhs.uk. 2022. Anxiety in children. [online] Available at: <https://www.nhs.uk/mental-health/children-and-young-adults/advice-for-parents/anxiety-in-children/> [Accessed 01 August 2022].
Post UK Parliament. 2021. Children’s Mental Health and the COVID-19 Pandemic. [online] Available at: <https://post.parliament.uk/research-briefings/post-pn-0653/> [Accessed 3 August 2022].
Samara, T., 2017. Making and breaking the grid (revised and updated). Rockport Publishers, p.20.
Sharing the success of a recent publication on the Royal College of Surgeons Bulletin on a remote consultation pilot using the Microsoft Hololens 2 device
Implementing new patient-facing technology in the NHS is not for the faint hearted! It is essential to gather a team around you, which includes patients
https://www.youtube.com/watch?v=bEwxuXprezI A wheelchair that uses Eye tracking to aid people with motor neurone disease in order to give them the freedom of mobility for longer, the
The rheumatology questionnaire is a mixed-reality application developed in-house for the paediatric ward for reducing anxiety in adolescents. Patients often come in and are asked
Watch the Digital Futures Team take part in InSites workshop 2024! https://immersive.tsdft.uk/wp-content/uploads/2024/05/insites_workshop_2024-1080p.mp4 Share: Twitter LinkedIn On Key Related Posts