BlackGap Tech Mentor

Let me check you in. (Overview)

Want me to send you some information?

Download the PDF

Exploratory Research

Where this is coming from. (Background)

I found out there was a high demand for diversity in computing careers

  • % Black US population: 13%

  • % Black bachelor’s degree in…

    • CS: 3%

    • Computer engineering 5%

    • IT: 7%

  • % Black work at…

    • Silicon Valley: 3%

    • Technical positions: 3%

    • Tech company: 7%

  • %Black computing faculty…

    • Tenure: 2%

    • Non-tenure 3%

With more research I learned this is influenced by challenges with URM students’ computing identity

  • Performance (resilience & self-efficacy)

  • Willingness to participate (interest & belonging)

  • Near-peer mentoring can help mitigate

So my colleagues and I developed BlackGap Tech Mentor

  • Proposed intelligent virtual mentoring system (VMS) tailored to undergraduate underrepresented minority computing students

  • Explored needs and preferences, developed information database, and prototyped virtual mentoring system

So let’s talk. (Focus group)

I developed a focus group protocol informed by the Mentor Effectiveness Scale (Berk, 2005) and other base-level design fundamentals. Twenty Black computing undergrads and grad students ages 22-54yo were recruited.

I conducted and audio recorded one of three semi-structured 1 1/2 hour focus groups.

Guess what they said. (Focus group results)

I performed a qualitative inductive-deductive thematic analysis on the transcribed audio recording using the Mentor Effectiveness Scale as preselected codes.

They suggested uses for a virtual mentoring system.

Use when…

  • Uncomfortable asking people around you

  • In need for technical assistance

  • In need to grasp concepts better

  • In need for general information


It can be the inappropriate tool if

  • you’re looking for detailed information


Works best if it is:

  • Able to develop rapport and trust

  • Able to develop a personal relationship

  • Intelligent

User Research

I have to ask some questions. (Survey)

I developed a user survey incorporating…

  • User interest/likelihood of attending grad school

  • Factors that persuade a user to (or not to) pursue grad school and professoriate.

  • 3 questions a user would ask a mentor in pursuing CS careers, grad school & professoriate

71 Black computing undergrads were recruited

  • 94% black, 5% multiracial, 1% Native American

  • 18-25yo

  • 2.4-4.0 GPAs

I analyzed with descriptive statistics

  • Levels of interests in grad school

  • Likelihood to pursue grad school

I performed a qualitative inductive-deductive thematic analysis on the open-ended survey data for factors that persuade users to attend grad school. I synthesized. computing grad school program FAQ topics to use as preselected codes. Mentor questions were used for information database and to explore question syntax.

Let me tell you, they were… (Survey results)

portvmsT01.PNG

Prototype Testing

Building the relationship. (Testing the VMS)

Colleagues developed a virtual mentoring system (VMS).

VMS was built by computing colleagues using natural language understanding engine (Google Dialgueflow. Avatar (for embodied conversational agent) programmed by colleague using SitePal and JavaScript. NLU also connected to SMS (via Twilio) and Twitter (Twitter Developer Platform).

The VMS existed on three platforms:

  • SMS text messaging

  • Twitter direct messaging

  • Web-based embodied conversational agent (ECA)

Logic of an ECA VMS

Prototype of Twitter VMS

Prototype of ECA VMS

I developed a user survey in which 20 Black CS students participated using all three platforms (ECA, SMS, Twitter). The survey included

  • System Usability Scale (SUS)

  • Mentor Effectiveness Scale (MES)

  • Open-ended questions based on Interactive Design Foundation UX measures

  • Open-ended recommendations and potential audiences

Participants were provided with the SMS phone number, Twitter handle, and website url to interact with the virtual mentor. They were required to send at least 10 questions about graduate school.

Relationship check. (Results)

I analyzed with descriptive statistics

  • MES

  • SUS

  • Recommendations

I used a simple regression to compared

  • ECA vs Twitter & SMS

  • Twitter vs SMS

I performed a qualitative inductive-deductive thematic analysis on the open-ended survey data on Interactive Design Foundation UX. I synthesized computing grad school program FAQ topics to use as preselected codes

Believe to be Accessible

Likelihood to Interact

Likelihood to Recommendation

Thematic analysis from VMS use

Let me show you (VMS app prototype)

I created an intermediate-fidelity app prototype using Figma taking design considerations from these use cases.

 

Chat feature

Search feature

Profile & Settings feature

Delivery

Talk to you later. (Conclusion)

I led writing and co-wrote for 3 conference manuscripts

  • I also presented it at HCII’19 in Orlando, FL in July 2019

  • Cummings, R. T. Dennis, S., Mack, N. A., Nias, J., & Gosha, K. (2019). Developing a question corpus for a conversational agent designed to prepare interested Black undergraduates for the professoriate in STEM. Proceedings of RESPECT’19: the 2019 Research on Equity and Sustained Participation in Engineering, Computing, and Technology conference, Minneapolis, MN: Curran Associates, Inc.

  • Mack, N. A., Huff, E. W., Cummings, R. T., & Gosha, K. (2019). Exploring the needs and preferences of underrepresented minority students for an intelligent virtual mentoring system. Proceedings of HCII’19: the International Conference on Human-Computer Interaction. Orlando, FL: Springer. (poster).

  • Huff, E. W., Mack, N. A., Cummings, R. T., Womack, K., Gosha, K., & Gilbert, J. E. (2019). Evaluating the usability of pervasive conversational user interfaces for virtual mentoring. . Proceedings of HCII’19: the International Conference on Human-Computer Interaction. Orlando, FL: Springer.

This information was also distributed to the administrators of IAAMCS and was associated with 2 grants.