But which one is the best you are probably wondering? Allow me to explain a bit of my vocabulary first. 

There are 4 kinds of messages you can get on Dive Chat :
- Dives - Channel based messages from organizations
- DMs or Direct Messages - messages with 1 recipient and 1 sender
- Group Chats - like DMs but with several people instead of one person d
- Event chats - these are temporary chats that are generated when you say you are attending an event
‍
The first solution applies a bubble-like decision system. It focuses on removing the bottom navigation bar and lets people find chats based on their 7 most viewed messaging systems. The second option is familiar for iOS mobile users as it is a carbon copy of iMessages but with a distinction on the top for group chats, group messages, and user event chats.

This version lets people see, from a systematic view, all their messages for chats, channel-based messages and event based messages.

The third one is a composition that was designed to allow users to view up to 8 chats at once and it groups direct messages and group messages together ( as seen on one’s phone normally ) Now each has their merits and weaknesses so it is best we test them on some users!

‍Composition Testing
‍Test Details
- 30 individuals were given a test to find basic information about groups on a prototype.
- They were asked to find which group had 4 unread messages.
- This test was designed to see which could be navigated the fastest without prior knowledge of the app or its structure

Each version had 10 testers and I averaged the time it took each person to complete the tasks as you see to the side.

Results

Explain
‍Based on the chart you see above, people were able to understand the Tile Mode design better than any other design as they were able to complete tasks 3 times faster in this mode than the current iMessage mode. 
‍The Winner and the Reason
‍The tile mode won for a few reasons. A handful of them may be contributed to …
- Unique grid system
- Decreases the amount of clicks to find a group
- Allows you to view up to 12 groups at once ( more than the original 6 groups )
- Fastest speed meant strangers could learn and navigate the product easier

Based on this design system, I chose to redesign the entire app to focus on the minimalistic style. So it was naturally the mode I chose to hinge my entire design system for Dive X around. An issue with the current Dive Chat was it was developed at a 48 Hour hackathon and, since then, the app has been about designing what the Engineering Team and C-Suite team thought “captured their vision” rather than asking the users what they wanted out of the system.

So this new information architecture was one that was inspired by the user input. Due to limitations on time and my limited abilities as an app developer, I had to choose 1 color to design this tool in – light mode or dark mode.

Testing & Results
‍Between a control & variable testing group, the results were very significant. As you can see in the graph to the side, most people found this new version of Dive much easier to understand and navigate. The variable having Dive X's and the control having the original version. After contacting 60 people, 2 groups of 30, people were 5 times more likely to pitch dive to their student orgs if they were using the Dive X version of the app instead of the original version of the app.

‍With more time…
‍I would have tested all 18 concepts. I simply chose the 3 I felt were most intuitive to understand. If it wasn’t for the fact I was very tight on time with a 21 day deadline and 4 other tests to conduct, I would have included more time for testing. I also would have run these concepts by the C-Suite to ask them their thoughts. I only had time to get their opinion on one of the 18 mockups.

So, now that we know public groups are, pretty much, non-existent on Dive Chat, how do we give people something to do when most groups are unsearchable without a specific code? The solution I chose to go with were Global Communities!

Something interesting about Dive Chat is our very large ambassador program and how it is made up of college students, each with their own career goals and majors and aspirations. My proposal for solving this problem was to create a virtual community room ( similar to SubReddits or Linkedin Groups ) that allows people with certain career goals across the country to communicate with each other and network. 

In essence, we are basically creating groups that are catered around a specific professional theme like “Resume Critiques” that users can join at the start of the onboarding process.

‍The Winner and the Reason
‍Now we knew how we needed to shorten the onboarding process so we could make room for a community finding feature. All that remains is simply deciding which screens should be removed from the onboarding process and move to in-app experiences. Below you can see a graph of the results of how long it took people to complete certain onboarding tasks on average thanks to Google Analytics. Clearly, the profile prompt and description were taking up much more time than was needed for an onboarding process so I removed that in the Dive X program and shifted that input of information to an in-app exercise the next time someone opens the app after joining their first group.

When the new onboarding process you see was completed, I did a test to verify the success with an AB control variable group test again. The design was successful but here were the reasons why I found it successful.
·       I went with this design since it is fast, simple, and clean
·       It Gathered repeated data like student status & university by requiring a university email. (with a student email, no need to verify phone number)
·       23 out of 24 prefer this flow to the original flow
·       I promoted finding other Dives ( public groups ) in the onboarding process
·       Verification emails were sent before a user can use the full application
·       About me and Profile Prompts were moved to in-app experiences
·       No excessive animations or filler screens
·       Due to time restrictions, no password verification for this MVP

The results of the AB test are below.
‍

Each participant was asked to rate their experience on a scale of 1 to 10 for the “complexity” they found in the onboarding process. I then grouped answers together ( i.e. 1 and 2s == very simple to understand and 9s and 10s = very complex )  to identify a more obvious pattern.  As you can see in this graph, most people found the new onboarding process very simple and quick to finish.

This part, specifically, could get a case study in its own right so feel free to let me know if you would like me to present a pitch deck with more details!  

‍With More Time…
‍I would make the groups more unique for each individual based on their field of study. I.e. If someone wants to work in tech, I would make a Tech Group Dive for them to join. This way it would add value to the consumers as it will help them see Dive as a nation-wide networking tool that is has the potential to be.

Something that our competitors ( like Groupme, Slack, Discord ) tend to lack. I would also love to create a more overall “full” experience for finding groups with a proper search engine, but with all my restrictions, this was a good “proof of concept” to validate if there is a need to allocate resources to this function.

However, this would have required not only more testing, but a few people to be a part of these “sample” groups.  I’d imagine it would take at least 7 days to build these groups and build decent conversations at minimum, so it was ideal for time reasons to not test too many ideas during this 21 day sprint. 

Due to time restrictions, I chose to not conduct design validation for this feature because I wanted to allocate extra time for developing and designing the web application as I would be coding it from scratch in addition to designing it.

With more time, I would have done AB testing as I did with the first 2 solutions. However, since the first 2 solutions performed better when the number of clicks decreased, I made the assumption decreasing clicks here would improve performance.

As you can see, only the redundant, non-group chat specific questions were removed and thus we simplified the process so it takes less clicks
‍
With more time…  
‍Naturally, a huge limitation on this Core Problem was the solution didn’t have much testing. There were ways I could have tested it like by asking users to complete certain tasks and ask them to recall information. However, due to coding time, I was already a day over schedule for this project due to coding difficulties setting up and retaining firebase data.

And of the 5 core problems the next one is the most demanded problem and this one was the most “vague” and thus was likely not able to be solved in the time restrictions I was given. That being said, I feel this was an essential part for Dive X as it introduces the new design system for the app and it paved the groundwork for other designers to explore this UI and iterate on it.   But in terms of facts, by decreasing the number of steps it takes to complete a task, we did make the solution “less complex”.
‍

‍Testing & Results With more time…  As the product design lead of the web design team for Dive, I had a lot of insights I had acquired from past experiments and I inputted them into this design. So, unlike the other design changes, I knew exactly what to expect when building this part of the program out. It was mostly a “technical” design problem if nothing else. Something I would love to have though would be a fellow designer or engineer to help with this feature. Due to the physical size of the canvas associated with designing web tools, I ran out of time much quicker than I expected.

With 4 more days, I would have been able to test the various interaction patterns to make a better decision on choosing the layout of the web version. Fortunately, users were happy with any web version so this problem was successfully solved.  

Due to NDA reasons, the results will be available during an interview. However, I can share that people who used the Web Version of Dive were 3 times more likely to use our referral system to invite organizations to join Dive Chat.

To learn more, please visit the Dive Chat Monetization Case Study. But in short, this is how I approached validation.

I had users from the prior experiments for Solution 1, Solution 2, and Solution 4 all opt into testing the new features for value proposition. I chose them because they were all adept to using the new Dive Chat X product from the prior tests.

I did a control and variable tests again, but this time, I got people's opinions based on using the Stage and Discovery. While I need to save the data and testing for a portfolio case study presentation, I can tell you the result was that Discovery increased the user satisfaction of the product significantly while improving retention in the testing group.

‍Composition Testing

‍The testing for this feature was simple – during the time I was working on the “Complex” Core Problem and the Scalable Core Problem, I had made 2 versions of Dive for a pair sample groups to try using in an AB test. I simply asked them to use the app and when I asked them a few days later they would tell me if they were likely to recommend the app.  

‍The Winner and the Reason
‍People who had the Delayed Messages, Discovery, and Stage were much more likely to refer to the app because 7/8 were willing to refer to it when it had those features compared to the control group that only had 3/8 referrals.

‍With more time…  
‍While I know the group I contacted for this procedure was smaller than the others, it was mostly due to the fact I was running out of time. Based on those results, I decided to move ahead and finished the design with the new features added on.

With more time, I would test this on more people. Something else was I tested this on individual small organizations that all attended the Same University ( Texas A&M & NYU ) due to time restrictions. In reality, I should have tested this in several schools to get an accurate representation. However, this amount of validation was enough for me to proceed in my evaluation of Dive X and Dive Original.

One of the other main takeaways I had was how technically complicated I found implementing a video conference feature for the app. I had never done it before and it was already difficult ( and costly ) to just get the video to be stored, but streaming the video live in high quality was a difficult task indeed. I would have loved help or more time to figure out how to make the feature more effective.

Lastly, in hindsight, this feature should have been the first one I tried to evaluate. I chose to start with Unorganized Flow because that would minimize repeated effort for my work in the future. But earnestly, the business would have benefited most from evaluating new features compared to a redesign of the UI.

But which one is the best you are probably wondering? Allow me to explain a bit of my vocabulary first. 

There are 4 kinds of messages you can get on Dive Chat :
- Dives - Channel based messages from organizations
- DMs or Direct Messages - messages with 1 recipient and 1 sender
- Group Chats - like DMs but with several people instead of one person d
- Event chats - these are temporary chats that are generated when you say you are attending an event
‍
The first solution applies a bubble-like decision system. It focuses on removing the bottom navigation bar and lets people find chats based on their 7 most viewed messaging systems. The second option is familiar for iOS mobile users as it is a carbon copy of iMessages but with a distinction on the top for group chats, group messages, and user event chats.

This version lets people see, from a systematic view, all their messages for chats, channel-based messages and event based messages.

The third one is a composition that was designed to allow users to view up to 8 chats at once and it groups direct messages and group messages together ( as seen on one’s phone normally ) Now each has their merits and weaknesses so it is best we test them on some users!

‍Composition Testing
‍Test Details
- 30 individuals were given a test to find basic information about groups on a prototype.
- They were asked to find which group had 4 unread messages.
- This test was designed to see which could be navigated the fastest without prior knowledge of the app or its structure

Each version had 10 testers and I averaged the time it took each person to complete the tasks as you see to the side.

Results

Explain
‍Based on the chart you see above, people were able to understand the Tile Mode design better than any other design as they were able to complete tasks 3 times faster in this mode than the current iMessage mode. 
‍The Winner and the Reason
‍The tile mode won for a few reasons. A handful of them may be contributed to …
- Unique grid system
- Decreases the amount of clicks to find a group
- Allows you to view up to 12 groups at once ( more than the original 6 groups )
- Fastest speed meant strangers could learn and navigate the product easier

Based on this design system, I chose to redesign the entire app to focus on the minimalistic style. So it was naturally the mode I chose to hinge my entire design system for Dive X around. An issue with the current Dive Chat was it was developed at a 48 Hour hackathon and, since then, the app has been about designing what the Engineering Team and C-Suite team thought “captured their vision” rather than asking the users what they wanted out of the system.

So this new information architecture was one that was inspired by the user input. Due to limitations on time and my limited abilities as an app developer, I had to choose 1 color to design this tool in – light mode or dark mode.

Testing & Results
‍Between a control & variable testing group, the results were very significant. As you can see in the graph to the side, most people found this new version of Dive much easier to understand and navigate. The variable having Dive X's and the control having the original version. After contacting 60 people, 2 groups of 30, people were 5 times more likely to pitch dive to their student orgs if they were using the Dive X version of the app instead of the original version of the app.

‍With more time…
‍I would have tested all 18 concepts. I simply chose the 3 I felt were most intuitive to understand. If it wasn’t for the fact I was very tight on time with a 21 day deadline and 4 other tests to conduct, I would have included more time for testing. I also would have run these concepts by the C-Suite to ask them their thoughts. I only had time to get their opinion on one of the 18 mockups.

So, now that we know public groups are, pretty much, non-existent on Dive Chat, how do we give people something to do when most groups are unsearchable without a specific code? The solution I chose to go with were Global Communities!

Something interesting about Dive Chat is our very large ambassador program and how it is made up of college students, each with their own career goals and majors and aspirations. My proposal for solving this problem was to create a virtual community room ( similar to SubReddits or Linkedin Groups ) that allows people with certain career goals across the country to communicate with each other and network. 

In essence, we are basically creating groups that are catered around a specific professional theme like “Resume Critiques” that users can join at the start of the onboarding process.

‍The Winner and the Reason
‍Now we knew how we needed to shorten the onboarding process so we could make room for a community finding feature. All that remains is simply deciding which screens should be removed from the onboarding process and move to in-app experiences. Below you can see a graph of the results of how long it took people to complete certain onboarding tasks on average thanks to Google Analytics. Clearly, the profile prompt and description were taking up much more time than was needed for an onboarding process so I removed that in the Dive X program and shifted that input of information to an in-app exercise the next time someone opens the app after joining their first group.

When the new onboarding process you see was completed, I did a test to verify the success with an AB control variable group test again. The design was successful but here were the reasons why I found it successful.
·       I went with this design since it is fast, simple, and clean
·       It Gathered repeated data like student status & university by requiring a university email. (with a student email, no need to verify phone number)
·       23 out of 24 prefer this flow to the original flow
·       I promoted finding other Dives ( public groups ) in the onboarding process
·       Verification emails were sent before a user can use the full application
·       About me and Profile Prompts were moved to in-app experiences
·       No excessive animations or filler screens
·       Due to time restrictions, no password verification for this MVP

The results of the AB test are below.
‍

Each participant was asked to rate their experience on a scale of 1 to 10 for the “complexity” they found in the onboarding process. I then grouped answers together ( i.e. 1 and 2s == very simple to understand and 9s and 10s = very complex )  to identify a more obvious pattern.  As you can see in this graph, most people found the new onboarding process very simple and quick to finish.

This part, specifically, could get a case study in its own right so feel free to let me know if you would like me to present a pitch deck with more details!  

‍With More Time…
‍I would make the groups more unique for each individual based on their field of study. I.e. If someone wants to work in tech, I would make a Tech Group Dive for them to join. This way it would add value to the consumers as it will help them see Dive as a nation-wide networking tool that is has the potential to be.

Something that our competitors ( like Groupme, Slack, Discord ) tend to lack. I would also love to create a more overall “full” experience for finding groups with a proper search engine, but with all my restrictions, this was a good “proof of concept” to validate if there is a need to allocate resources to this function.

However, this would have required not only more testing, but a few people to be a part of these “sample” groups.  I’d imagine it would take at least 7 days to build these groups and build decent conversations at minimum, so it was ideal for time reasons to not test too many ideas during this 21 day sprint. 

Due to time restrictions, I chose to not conduct design validation for this feature because I wanted to allocate extra time for developing and designing the web application as I would be coding it from scratch in addition to designing it.

With more time, I would have done AB testing as I did with the first 2 solutions. However, since the first 2 solutions performed better when the number of clicks decreased, I made the assumption decreasing clicks here would improve performance.

As you can see, only the redundant, non-group chat specific questions were removed and thus we simplified the process so it takes less clicks
‍
With more time…  
‍Naturally, a huge limitation on this Core Problem was the solution didn’t have much testing. There were ways I could have tested it like by asking users to complete certain tasks and ask them to recall information. However, due to coding time, I was already a day over schedule for this project due to coding difficulties setting up and retaining firebase data.

And of the 5 core problems the next one is the most demanded problem and this one was the most “vague” and thus was likely not able to be solved in the time restrictions I was given. That being said, I feel this was an essential part for Dive X as it introduces the new design system for the app and it paved the groundwork for other designers to explore this UI and iterate on it.   But in terms of facts, by decreasing the number of steps it takes to complete a task, we did make the solution “less complex”.
‍

‍Testing & Results With more time…  As the product design lead of the web design team for Dive, I had a lot of insights I had acquired from past experiments and I inputted them into this design. So, unlike the other design changes, I knew exactly what to expect when building this part of the program out. It was mostly a “technical” design problem if nothing else. Something I would love to have though would be a fellow designer or engineer to help with this feature. Due to the physical size of the canvas associated with designing web tools, I ran out of time much quicker than I expected.

With 4 more days, I would have been able to test the various interaction patterns to make a better decision on choosing the layout of the web version. Fortunately, users were happy with any web version so this problem was successfully solved.  

Due to NDA reasons, the results will be available during an interview. However, I can share that people who used the Web Version of Dive were 3 times more likely to use our referral system to invite organizations to join Dive Chat.

To learn more, please visit the Dive Chat Monetization Case Study. But in short, this is how I approached validation.

I had users from the prior experiments for Solution 1, Solution 2, and Solution 4 all opt into testing the new features for value proposition. I chose them because they were all adept to using the new Dive Chat X product from the prior tests.

I did a control and variable tests again, but this time, I got people's opinions based on using the Stage and Discovery. While I need to save the data and testing for a portfolio case study presentation, I can tell you the result was that Discovery increased the user satisfaction of the product significantly while improving retention in the testing group.

‍Composition Testing

‍The testing for this feature was simple – during the time I was working on the “Complex” Core Problem and the Scalable Core Problem, I had made 2 versions of Dive for a pair sample groups to try using in an AB test. I simply asked them to use the app and when I asked them a few days later they would tell me if they were likely to recommend the app.  

‍The Winner and the Reason
‍People who had the Delayed Messages, Discovery, and Stage were much more likely to refer to the app because 7/8 were willing to refer to it when it had those features compared to the control group that only had 3/8 referrals.

‍With more time…  
‍While I know the group I contacted for this procedure was smaller than the others, it was mostly due to the fact I was running out of time. Based on those results, I decided to move ahead and finished the design with the new features added on.

With more time, I would test this on more people. Something else was I tested this on individual small organizations that all attended the Same University ( Texas A&M & NYU ) due to time restrictions. In reality, I should have tested this in several schools to get an accurate representation. However, this amount of validation was enough for me to proceed in my evaluation of Dive X and Dive Original.

One of the other main takeaways I had was how technically complicated I found implementing a video conference feature for the app. I had never done it before and it was already difficult ( and costly ) to just get the video to be stored, but streaming the video live in high quality was a difficult task indeed. I would have loved help or more time to figure out how to make the feature more effective.

Lastly, in hindsight, this feature should have been the first one I tried to evaluate. I chose to start with Unorganized Flow because that would minimize repeated effort for my work in the future. But earnestly, the business would have benefited most from evaluating new features compared to a redesign of the UI.