Experiential Design / Task 1

24/09/2024 - 21/10/24 / Week 1 - Week 4
Iman binti Kamarudin / 0364014
Experiential Design / Bachelors of Design (Hons) in Creative Media

Task 1: Trending Experiences

---

---

LECTURE

Week 1

For our first class we went over the MIB and looked at some past projects from previous students. Afterwards we conducted an activity and were briefed on our first exercise/homework.

AR [Augmented Reality]

Enhances the physical world by combining real and virtual elements. Extends visuals

MR [Mixed Reality]

Can interact with virtual objects. Enhances the physics of the real world.

VR [Virtual Reality]

Computer generated world. User feels immersed in the virtual world, extending experience.

AR can be marker based ('activated' by scanning a certain object) or markerless (more to do with the physical space/area)

Week 2

Today we went over user mapping and journey maps. Afterwards, we divided ourselves into groups and made our own journey maps based on a specific place. We had to identify the gain points, pain points and solutions. We also had to try to figure out how AR could be applied in some of the solutions.

Fig 1.1 - Terminology Chart

Empathy Map

Tool used to dissect what we know about a user. It uses user knowledge in order to;

- Create a shared understanding

- Aid in decision making

Week 3

In week 3, Mr. Razif conducted a tutorial in Unity. Afterwards, we were informed that we had to conduct research on AR experiences and then come up with ideas and mockeups/sketches for the app we wanted to make.

Week 4

During week 4, we had a tutorial on how to make a markerless AR experience. We were also taught about how to run the build on the actual devices we made it for (eg. iPhone).

---

INSTRUCTIONS

EXERCISE 1

For our first activity we went onto Google and searched up 'dinosaur', 'cat' and 'dog'. Then we clicked on the '3D' button. Afterwards, you had to scan the ground and a 3D model of the animal will appear as if its there in the room through your phone.

Fig 2.1 - Dinosaur 3D AR

Fig 2.2 - Cat 3D AR

EXERCISE 2

For our second exercise we have to create a mockup/prototype of an AR experience based on a place such as the kitchen, gym or shopping mall.

Some ideas I came up with for each place;

Gym: Tutorials on how to use the machines in the gym. Scan the machine and an animated person will appear and go through all the steps.

Kitchen: Tracker for cooking. Lets you know what seasonings you already put in your dish and how long to cook your ingredients for and when to add things in. Basically a step by step cooking guide.

Shopping Mall: Guide and directory to stores in the mall. 

I chose to go with the kitchen idea since I struggle with cooking meals so I feel like I would know what could be helpful for users in the experience. I decided to create 4 mockup prototype screens to show the key features of the AR experience. To make it seem more believable as an AR app, I took my own photos with my phone.

The first feature is a page for recipes for you to select. 

Fig 3.1 - Recipe Page Mockup

Once you select your recipe, you can go through a step by step guide with animations to show you how to cook the meal. There are 'previous' and 'next' buttons at the bottom of the screen so you can go back and forth to understand the steps easily.

Fig 3.2 - Animated Guide Mockup

The third mockup shows the user in the middle of making the meal and displays what seasonings are yet to be put in, and which ones have already been added to the pan. It also tells the user what ingredient is missing on the cutting board.

Fig 3.3 - Meal in Progress Mockup

The fourth feature is a helpful guide to show you how to dice your onion if you have no idea how to. It shows a dashed line on the onion itself to let you know where to cut.

Fig 3.4 - AR Onion Dicing Guide Mockup

EXERCISE 3

In class we had to create a journey map based on a place. My group decided to base it on Genting Highlands Outdoor Theme Park and we made the map on Canva.

We first decided what areas the user would interact with first and then wrote the pain and gain points for each area.

Fig 4.1 - Genting Highlands Theme Park Journey Map pt.1

Fig 4.2 - Genting Highlands Theme Park Journey Map pt.2

EXERCISE 4

Download Unity and Vuforia. Follow and create an AR experience exercise.

The tutorial taught us how to create a marker-based AR experience. We went over how to create license key and upload an image target. We also learned how to make a video play when an image is scanned.

For my experience, I used a poster from Sabrina Carpenter's tour as the image target and when its scanned, it played a video from her entrance on her tour.

 

Fig 5.1 - AR Setup

Fig 5.2 - Image Target Scanned

EXERCISE 5

Create a main menu and credits scene and link them so you can switch between scenes when the buttons are pressed on the screen.

In class we learned how to create different scenes and display text/buttons. We were also taught how to change the colours of the buttons when hovered over and when pressed.

Fig 6.1 - Main Menu Scene

Fig 6.2 - Credits Scene

Fig 6.3 - Hover Effect on 'Start' Button

Then, we wrote a script so that it switches between scenes when the buttons are pressed.

Fig 6.4 - Script

Fig 6.5 - 'Start' Button On Click Action

EXERCISE 6

During class we had a tutorial on creating a markerless AR experience. We added a plane finder to detect the ground and a ground plane for our object to spawn on. 

Fig 7.1 - Markerless AR Cube

After, we were taught about the settings to be able to build and run the experience on our phone/device. Since I use a PC and an iPhone, I had to download VMWare and iOS Sonoma to be able to use XCode.

TASK 1

In order to get a better knowledge on what apps to make, I have to research what is currently in the AR market. When I searched up popular AR apps, a lot of them were games/for entertainment such as Pokemon Go. Other popular apps are about furniture placement/planning like Ikea Place

When I thought about examples of AR experiences that are currently popular, social media filters came to mind. On TikTok and Instagram, people often use filters when recording videos or taking pictures. Some of the filters even serve as games and there have been multiple cases where they went viral. This is placed under the entertainment genre.

What Is In The Current AR Market?

1. Navigation

Apps like "Google Maps AR" can help direct people to their destinations by overlaying arrows and distance markers on the screen. Some stores or malls have also used navigation AR to help people find stores or facilities within the building easier.

Fig 8.1 - Google Maps AR

2. E-commerce

"Shopify AR" and "YouCam Makeup" allow you to 'try on' and view products virtually without the financial commitment. Users can see how products look in their house, to scale, or see what shade of makeup products fits their face.

Fig 8.2 - Shopify AR

Fig 8.3 - YouCam Makeup

3. Education

Apps like "Google Expeditions" (now discontinued) can help educational institutions with immersive learning sessions. By placing students in different places through virtual means, it provides them a safe way to explore and learn. It also saves the institutions the costs of museum and bus tickets to take the children there physically. 

Fig 8.4 - Marker-based AR Expedition

"Night Sky 11" shows users the night sky and positions of satellites in space, which are hardly visible in places with high light pollution. It can also map out constellations and planets. This app provides educational materials to users and allows them to see the stars above in real time, wherever they are.

Fig 8.5 - Night Sky 11 Screenshot

4. Entertainment

Pokemon Go is a popular example of an AR game. You are able to see pokemon as if they exist in your physical world through your phone. You can explore your area to find pokemons in your local places and interact with them.

Fig 8.6 - Pokemon Go AR

Instagram and TikTok filters are also used for entertainment. People make short minigames and it creates an interactive, fun experience for users. 

Fig 8.7 - Instagram Filter Games

Fig 8.8 - TikTok Filter Game

After the research phase, I though about what achievable experiences I could create that I also cared about. I came up with an idea for the club that I'm in as I know that there is a problem that an AR app could solve in terms of navigation. I also though about making an education AR experience as a lot of past students have done that, so I decided to make it art related as it's my favourite thing to do to pass time. Lastly, I came up with an idea related to music and discovering artists.

After listing down the ideas, I wrote the problem statements as well as the aims and goals for the app to see what features I should focus on.

Then, I came up with mockups to support my ideas. I made simple graphics using Procreate and only did the main page that users would interact with.

TAYLORS MUSIC CLUB MOCKUPS

Fig 9.1 - Navigation AR Mockup

Fig 9.2 - Marker Based AR Mockup

Fig 9.3 - Scanned Marker AR Mockup

ART TUTORIAL APP

Fig 10.1 - Marker Based AR Mockup

Fig 10.2 - AR Animated Tutorial Mockup

MUSIC ARTIST DISCOVERY APP

Fig 11.1 - Marker Based Poster AR Mockup 1

Fig 11.2 - Scanned Marker AR 1

Fig 11.3 - Marker Based Poster AR Mockup 2

Fig 11.4 - Scanned Marker AR 2

FINAL

Fig 12.1 - App Proposals

---

REFLECTION

My experience with all the exercises were pretty positive. Even though it was a bit hard to catch up during class, I feel like when something went wrong, I learned more about what step I missed and why it was vital to having the experience work. It was also pretty fun getting to do the exercises (like exercise 4) because I actually got to try out the experience for myself and see the results.

I observed that there is so many ways AR could be included in our daily life to make things easier or just provide us entertainment and I didn't know that some apps/features were considered as Augmented Reality, like instagram filters for example. I also learnt the difference between AR and VR and what they each provide towards your experience.

I found that it was hard coming up with app ideas that didn't seem too complicated or too simple. While there is so many ways that you can provide education through an AR experience, I didn't really want to go that direction as I didn't know what I was interested in/passionate about enough to want to teach other people. On the other hand, I wasn't sure that I had the capabilities to model or animate things for a storybook based app or a game.