00:00:02.580

Wilma: Hi everyone, and welcome back to Optimistic Design - a speaker series where we take a practical positive look at the future of design, ethical innovation, and technology.

I’m your host, Wilma Lam, Associate Director of Strategy here at Substantial. Today I’m excited to catch up with Andy Leong.

He is a product design lead at Karat, a company founded in 2014 with a mission to unlock opportunity for both hiring companies and candidates via re-engineering the interview process itself.

As a designer and he helps the Karat team build a more human centered technical interview at scale because path into design first featured stints in music and videography before becoming a product designer. Hi Andy, welcome, and thank you for joining me.

00:00:46.050

Andy Leong: Glad to be here.

00:00:47.610

Wilma: Great, and just to get us started I love to hear you know how you got into design to begin?

00:00:55.080

Andy Leong: yeah how am I think you know growing up, I just always love creating things and especially like working on on team projects and things like that grew up playing in bands making movies, with friends shooting photography like all those sorts of; like high school creative project stuff yeah and then my first job out of college was at this ED tech company in La.

I was helping like teach some classes, as well as designing like curriculum and other materials, I was sort of transitioned into doing design there more full time. Both you know, like marketing materials, as well as our website and some e-commerce stuff I really just gained a lot of experience. Like helping run and build a small business it wasn't like a high growth startup or anything like that just kind of a more.

You know, small mom and pop type of operation, but my boss was really great just giving me a lot of space to learn, and you know build things that I thought would be valuable and so after a while there I started to get hungry for something that was more fast paced yeah the more typical like high growth sort of startup experience.

And so I applied to this product design fellowship program in San Francisco called White Space which was this amazing experience that I was paired with a startup but then got to participate in workshops and critiques and other things with the other fellows in the program over the first like three months there so that's actually what introduced me to Karat where I am today.

But had a great time, just like in that program learning, not only from you know the direct sort of instruction input that I got but really learning from the other eight folks in the in the program and like helping with the things that they were working on at their startups and just having this sort of like many, you know design school experience and a lot of ways and yeah but I care ever since.

00:03:09.300

Wilma: Awesome, I mean, it sounds like you've had a number and really forming in design learning experiences as you think of cost kind of like your past experience, I became a designer you are today and who do you think has mostly influenced and inspired you in your work now?

00:03:26.490

Andy Leong: yeah you know, I was thinking about this and I kind of just kept going back to my dad who's not a designer by any stretch, but is really just this like cereal hobbyist like growing out there is always some new thing that he would just get really into and it wasn't like.

You know, he had to I don't know go to some special school for this thing you're just kind of like he would figure out how to do it on its own and just kind of learn new skills, all the time, so whether it was like repairing cars or like learning a new musical instrument or how to golf or, like all sorts of different things like I think he was just a great model of like that, like lifelong learner you know.

Yeah, and I think growing up, I sort of implicitly latch on to that so a lot of my life lot of my career has just been, you know just figuring stuff out trying to: Do the do the work do the research to figure out like well what is like the best way to do this, but not being too intimidated by like you know I don't have this like formal background or training and this thing but kind of just being able to throw myself into it yeah.

00:04:44.610

Wilma: Well it's amazing that you had that learning mindset like models for you from such a young age, your dad sounds like early on, for us, has been great. So, at the start of this, I gave a little bit of insight into you know Karat mission, but could you talk a little bit with like in more detail about the mission specifically and how your role as a designer fits into Karats larger mission.

00:05:09.600

Andy Leong: Yeah, um see I guess like just as a starting point, like the problem space that we're in I think… You know, we believe that the job interview is this has potential to be this like amazing gateway to Opportunity right for both job seekers and for companies that are hiring but in their typical form, I think there's a lot of things that are just fundamentally broken about.

A job interview right, like any anyone who's interviewed for a job can probably attest to having some negative experiences there there's lots of opportunity for bias to seep into the process it's stressful for the camera and for the interviewer. And can lead to a lot of missed opportunities, a lot of teams, or you know bad hires I guess team is made up of just sort of Sub optimal TEAM members. And so that's just like you know issues kind of at the individual.

00:06:08.520

Andy Leong: Like interview level, but when you look at the number of candidates a company typically has to interview to fill a role, a lot of the companies we work with have very high bar so it's anywhere from like 50 to 100 candidates that they see before they fill a single role.

You multiply that by the number of open roles that a company and you start to just see like the scale of that problem of the inefficiencies in the interview process, so at Karat, we think we can start to address that problem.

Basically, by being really good at conducting interviews and then partnering with companies to interview candidates on their behalf. So that's kind of like a high level what what Karat is trying to do. As a designer as a product designer, in my career I’m embedded on a small team with a PM and a few engineers and essentially my role is to you know, listen to the needs of our users and translate them into features and capabilities in our service.

So it goes kind of like a an end to end like full stack designer type of role lots of functional you know requirement gathering spec writing and testing type work in addition to the more you know, like production level UI/UX design craft type work, yeah.

00:07:39.390

Wilma: Now that's really helpful just to give a lay of the land of the organization and how you work within it. So, from my understanding from some previous conversations with you the focus of Karat, can you interview product right now is on something called a technical interview which I believe programmers go through I’m not super familiar with this space. So it'd be great if you could talk a little bit about you know what attended interview is like why Karat and your team has been focused on these particular kinds of interviews and how they're conducted.

00:08:13.710

Andy Leong: yeah I mean, I was in the same boat, when I first started I hadn't experienced a technical interview, at least to my knowledge, before but yeah you're right on essentially it is a pretty common step for a you know software engineering or software engineering related type of role something where you have to know how to code and. Basically, like they are a you're presented with some sort of challenge like you know here's a given situation, some some starter code, maybe have like this is an input, you need to parse or something basic operation and then you need to write a program that does what you know the property is asking for, and it's sort of seen as just like a gut check like can this person actually code do they have the technical chops necessary to you know be successful in this type of role.

00:09:22.170

Wilma: And what yeah so based on this kind of understanding, you talk a lot little bit about like what are the kind of problems currently with that and cleaner views like how long did they take and what specifically is Karat trying to solve for?

00:09:38.580

Andy Leong: Problems with the technical interview. I think like, one of the things that is really common with with technical interviews is that they're not representative of what you would actually do in a day to day job.

As a software engineer it's maybe this like toy problem or a thing, where maybe the solution is actually really simple, but you just have to have really specific like domain knowledge to know what that like trick question is and generally like there's a lot of, not not necessarily something that this isn't something that's unique to technical interviews, but to interviewing in general that.

The person conducting the interview, maybe hasn't put in a lot of prep into how they should do that or what types of questions they should ask, or what types of things, should they be looking for in a candidate. And so that's one thing that I think we've just spent a lot of time trying to nail down your, how can we create and ask questions that are actually predictive of on the job performance that are fair that, like any candidate with those skills, should be able to you know give an adequate answer to this and then how can we sort of make sure that the person conducting the interview is doing it consistently or that they're doing it consistently with the way that we think they should be done so that, regardless of which interviewer you happen to get on a given day you're going to be getting the same experience as a candidate.

00:11:31.740

Wilma: yeah, no that's super helpful and definitely makes no sense, having been an interviewer before it's definitely something you wonder about it's like oh my consistent enough from interview to interview did I prep as well for this candidate is for the last line and see introduce a little bit of some of the aspects around Karats platform, could you actually walk through like what is it like to go through the interview process with Karat?

00:12:00.780

Andy Leong: So in typical hiring process at any company there's going to be several stages. To that process we kind of slot right into the middle of that, we're really focused on assessing for technical ability right so just answering the question can this person code will they be successful in a job, where they're required to code at this company and that process is usually you know there's an initial like you apply as a candidate, maybe you submit a resume than a recruiter or someone on the recruiting team reviews your resume seems like to get a basic level of this person.

You know, going to fit in with like our requirements here there's probably a recruiter phone call and then, if that goes well, they get sent to a Karat interview after the Karat interview if they've kind of passed their bar, then they would move on to like an onsite loop, which is basically you know, a series of interviews more behavioral interviews with people that actually work at the company and then from there, you know get it made an offer, and hopefully get hired.

So the stage that we're focused on right is like this middle kind of technical assessment and so. After the recruiter screen the candidate is sent an invite to schedule their interview on our platform and we actually pair the combat with one of our we call our interviewers interview engineers.

00:13:35.430

Andy Leong: Because they are software engineers by trade but are really specialized in the art and craft of interviewing so we pair them with an interview engineer who actually conduct the interview, we give the candidate some like you know prep materials resources and then the basic. It is this alive like one hour long video chat with a collaborative code editor editor tool.

The format kind of depends on the role we do a lot of customization depending on you know the company that is doing the hiring and what their needs are. But generally, there's some like introduction time a discussion of like a past project, maybe some knowledge questions around like a specific.

You know domain that they need to be competent at, but the the bulk of the time is spent on a programming exercise where kind of like what I described earlier where they're asked to you know program a solution to a problem. After that one hour ends our interview engineer writes up a bunch of notes about what happened we actually do a quality check process for every interview, so we make sure that. It was done according to our guidelines and is consistent with all the other interviews that we conduct. And then we deliver a recap of everything to the company that's you know, doing the hiring and we give them like a lot of the just the raw results of what the candidate, but we also provide a recommendation of you know what we think they should do in terms of next steps.

So then, from there yeah the hiring company kind of takes it that's the the loop that we're focused on.

00:15:29.550

Wilma: Yeah no that's really helpful and I think if I recall correctly, that Karat platform, it also allows interviewer or interviewee these two to select like whatever time works best. For them, and so potentially to do a redo could you talk a little bit about like those features and like what prompted them?

00:15:49.830

Andy Leong: yeah well, I think, like one thing that we've. Just looked at, since the beginning is is understanding that there's, you know there's a power dynamic typically at play with any like hiring experience right that it's sort of like the company doing the hiring.

Traditionally, at least carries a lot of, like their share of sort of the power in that relationship, increasingly, especially in tech, and especially in engineering roles that's shifted a little bit as it's gotten so competitive for good engineering talent.

But I think we wanted to make it so that it's not just there's a one-sided experience, where the candidate has to jump through all these hoops and isn't really offered. You know any like constellations along the way. And so, one of the things that were able to do is offer basically 24 seven scheduling so our network of interview engineers is a global network, people are doing this typically in like a freelance.

Engagement model so maybe they have a normal nine to five job, maybe they have some like development projects that they do on the side, but then I’m interviewing on the care platform is sort of this side business for them, so instead of being limited to conducting your interview as a candidate.

During like business-like typical work hours having to make up like a doctor's appointment, or something so that you can get out of showing up to work to make it to your interview we actually have, I think, like over half of our interviews are conducted on like nights and weekends so it's like one thing that we do to sort of just be more convenient for Canada and then another key thing that you mentioned, is our review policy and so. Because it's so easy for us to spin up an interview, we have the scale and capacity to do that.

We can really easily offer the candidate a chance to retake their interview if they felt like it wasn't representative of their best performance. A lot of times you know people just have a bad day maybe there's something really distracting but they feel like I committed to this time I got to show up and do this interview.

Maybe it was just like the question we asked like hit up this one gap of knowledge that they have but it wasn't actually representative of you know their true ability. And so we offer candidates, the chance to before you know there's been a decision from the hiring company that they can say you know what I actually like think I can show up better than I did that first time and and retake the interview with a different question and different interviewer. And I go from there we've actually had a number of Canada to their first interview yeah like they wouldn't have made it through to the next round and they improve on their second one and go on to get hired so it's cool to have names and stories of candidates who are actually in jobs because of this policy that otherwise you know wouldn't have happened.

00:19:10.380

Wilma: yeah, no that's like some incredible change, not only to see but actually not have metrics around. And yeah I’m thinking around also Karat’s core mission of driving systemic change in the tech industry, I can see how it would like what we've just talked about it's it's changing like the mindsets. Basically, think about systemic change in the tech industry when it comes to hiring I mean, could you like expand upon like what that means?

00:19:38.610

Andy Leong: yeah I think there's like probably two big areas where that comes into play for us like. I mean you know it's no secret tech has a lot of issues with diversity right that as much as it's this new and innovative industry, I think. You know, we have a belief that's not really living up to its potential, because the teams building the future that are creating the technology that you know we're using.

In all the different aspects of our lives that those teams aren't always representative of you know, the wider population and there's, you know just gaps and things that aren't being problem spaces that aren't being addressed or that are being addressed it adequately right, and so I think that's one thing that if if we can stay true to our mission of delivering a more fair interview more predictive more enjoyable for candidates that we might actually be able to counterbalance some of some of those issues and then I think the second thing is just that hiring in general, especially in tech for high growth companies that need to hire really fast it's just the interview process is a huge bottleneck to growth, like I think we've done surveys with engineering leaders who say like if you could have you know I forgot what the product is, but like an extra you know X amount of cash or an extra like 50 engineers like which would you prefer and everyone always picks the engineers, they want the headcount because that's the biggest like limiting factor on them achieving what their missions are so I think if we can address that inefficiency, then we're like just such a key piece in in the story of all these companies that were hiring or that we're helping higher, as well as you know, the stories of these candidates who are now able to get into the jobs that they want and get access to those opportunities.

00:21:51.420

Wilma: Now that's great and super helpful and definitely need it I’ve been really important to the industry a large. I'm curious about you know, earlier we talked a little bit about opportunities for candidates to be able to you know do interviews on nights and weekends, or the need for the reading policy.

You know, these must have come out of like early on understanding of unique needs both for interviewers and interviewees so kind of reflecting back on those early days of Karat I'd love to understand, you know, as a designer and as part of the product team, how did you include like the point of view and experiences of interviewees interviewers and firing partners in the design and product development process.

00:22:37.080

Andy Leong: yeah see I when I joined Karat it was a very, very small shop they're like more or less 10 of us on on the full-time team and so not only was our sort of core team small but we only had you know, a very small number of actual, like customers were paying us to do this, we had a very small number of candidates that were interviewing and a small number of interview engineers conducting the interview, so it was really easy to talk to three, four or five people and feel like okay that's a pretty representative of a collection of voices.

We really were built like slack is such an integral part of our company culture, and we have a lot of our different stakeholders, especially the interview engineering Community they're just in our main slack like kind of partitioned off, and you know, in terms of permissions and stuff but it's really easy to just access this big group of stakeholders that we serve and get give them updates on things to coordinate schedules to get feedback on like early ideas and things like that.

And so that was like I think an easy thing that like we could just go out and talk to the people directly there, there really weren't very many layers in between us and in our users and then even internally, I think every team was super connected when we're that small right like we can all the whole company can fit in a small room around one table and so we're just very aware of the different problems that each person each team was encountering and admit it. I know just easy to figure out Okay, this is like how we could address that like we're working on this, one thing that should like get out the problem you're facing over here and just moving you know really fast in the early days.

00:24:49.260

Wilma: So I’m curious, you know as Karat has scaled and grown obviously a lot more users, a lot more hiring partners, I imagine a lot more interview engineers. So today, how do you kind of make sure that the voice of all the stakeholders is still brought in, so that ongoing design process that you have?

00:25:10.260

Andy Leong: yeah, so I think you know some of that old ways is definitely still happening slack is still a big part of our culture, I'd say maybe say it's more so the case with internal stakeholders that that's still like a really strong, like sort of communication channel. But at the same time, I think it's it's harder to get a sense of the broader picture when that's like your only tool right so fortunately as the company has grown as our user base has grown.

You know our sample size of data has grown a lot too so there's just a lot more, you have more usable like metrics more opportunities. To feature flag, things are testing so like a small percentage of users before rolling out to everyone, I just gathering data and feedback in more like sort of quantifiable ways I guess there's definitely an opportunity, I think, as we've grown to to branch out into other like communication channels, and so I think today it's sort of figuring out like our challenges, how do we balance, like the two of those and make sure that we're not overly reliant on one or the other, but we can kind of get the best of each style of feedback.

00:26:35.130

Andy Leong: yeah.

00:26:35.970

Wilma: That makes a lot of sense I’m just like our conversation here, I feel like you've been able to reflect. A lot on your experiences at Karat and what it's like to be a designer any company that's experiencing high growth and went from you know, a small team to a larger team now.

So I’m wondering if you had to give advice to like new UI/UX designers that are interested in joining an early stage startup what you're saying?

00:27:02.790

Andy Leong: yeah well, I guess, like do you mean advice? For like how to get the job or like how to be successful, once you're there?

00:27:11.580

Wilma: Any how to be successful.

Once yeah.

00:27:15.300

Andy Leong: um I mean I think like you know what I’ll caveat, this was a this was my experience I don't know if this is like a universally true thing by any means, but my first couple of years, I think I was not an expert in what I was doing right, I was not an expert in technical interviews or managing this like three way marketplace or anything like that, but I just jumped in and tried to provide value like wherever I felt like I could provide value, it's really hard to go into an early stage startup with the mindset that you're going to be really specialized on this one thing like if you're a designer you know I really love motion design.

00:27:58.290

Andy Leong: Maybe a startup isn't going to be a great like an early stage startup you might not be able to just do that right, you need to be able to wear a lot of different hats and fill in where there's need so that was like you know what I try to do, especially in the early days is, you know, especially as, like the first designer it's like yeah there's a lot of product work to do, but there's also we didn't have a designer for marketing or for internal culture stuff or you know I had a background in photography so I ended up shooting a lot of like promotional photos or just documenting like team events and things right.

00:28:39.720

Andy Leong: And, and I think the other thing I would sort of encourages just to really invest in your team as well that, like, you know the people that you're working with are going to make or break. The experience right there there's not like oh there's this other part of the company that we can like rely on or whatever it's like the people that are there are it and I think it's important to like make sure that people are you're being taken care of that, those relationships are healthy and ultimately, I think, like you're having fun together, I think the early days I served for sure, should be, should be fun, it should be enjoyable like you should be able to push each other in good ways towards like what you're working towards but also Just enjoying the ride, you know yeah.

00:29:34.890

Wilma: Now I think that's wonderful advice, especially about like the people and the culture really just makes your day to day experience. So, it's been you know five years since you joined Karat, maybe over that time and we will touch on a little bit earlier, the tech industry conversation hiring has changed like quite a bit. As you think about the future of hiring intact, you know what is top of mine, now and in particular I’m always curious to hear what are you optimistic about?

00:30:05.610

Andy Leong: um yeah, I mean I think one thing you know that's obviously been like an issue and been there for a while, is just the conversation around diversity equity inclusion in tech, and I think hiring and obviously it's just one part of you know that conversation and how that's left out, but it's a critical part right like getting people in the door. Right now, just like is isn't happening at the level that it needs to and so, I think part of that you know a lot of the model for what companies are looking for.

In a candidate is still kind of based on what the skills are of the people who have already been there, which, if we're looking at like historically is not the most diverse or inclusive group you're only going to perpetuate that in like assessing for those things in the hiring process, so I think something that I’m hopeful, for, as I look towards the future, they feel I feel like people are starting to listen to those voices who've been saying for a long time, like hey you're missing out on like a lot when you're only looking at people that went to these schools, who look like us and you know, worked at these companies or whatever.

That there's yeah just like so much to be gained in terms of innovation in terms of I don't know just like team dynamics and when you're getting out of the business of building like a monoculture. So at a high level I think I’m you know, looking forward to more of that in the future, I think in terms of like what care of sort of focus on specifically I’m excited for the next, like generation of assessments that's maybe better at looking for things that you might not have thought were important before but can actually have like a material positive impact on your team right, so one thing we're starting to look at is like, how can we identify potential in a candidate, so not just like oh this candidate is at this bar like currently but like is this person like on track to be at this level if they receive like X, Y or Z.

And then, like related Lee I guess getting better at equipping hiring managers and those making hiring decisions to play make better decisions right so it's one thing to like be able to assess for those things and to present. Your fuller picture of what a candidate skill sets include, but I think it's the other side of the coin that's like arguably just as important as reeducating hiring managers and people making those hiring decisions to sort of value those criteria those competencies in the same way that they do the things that they've historically value yeah.

00:33:22.620

Wilma: Now that's a really thoughtful answer and I appreciate that you talked about it, both at the industry level, and you know what Karat doing now specifically.

Yeah I really enjoyed this conversation I think it's especially kind of inspiring like why you're talking about there's sort of a mindset shift to not, how can we evaluate people and, like take them out of the interview process, but how can actually assess them as a whole person and try to actually you know find as many ways as possible to allow them to continue with the interview process to try to like say yes to candidate versus saying now it seems like such a big mindset shift for the industry, but a lot more, I think, human.

So, Andy I want to say thank you so much for joining me and for anyone that's watching out there if you'd like to learn more about Andy and his team, and what they're doing, you can please go to K-A-R-A-T dot com and thank you so much for tuning in. If you'd like to follow along and hear more about Optimistic Design and what we're up to here, you can head to substantial dot com backslash optimistic design and join us next month, as we continue to take a future focused look at design, ethical innovation, and technology.

I’m Wilma Lam, I look forward to seeing you again, and thank you Andy.

00:34:45.660

Andy Leong: This is great. Thank you.