Zeta’s on a Mission to Make Payments Invisible
Mar 31, 2021
By 2022, invisible payments are expected to reach a whopping $78 billion in annual transactions.
Part of this is due to the boom of digital payments during the pandemic. And Zeta—a company rethinking payments from core to the edge—is benefiting from this boom.
We invited Ramki Gadipatti, Co-Founder & CTO of Zeta to chat about the fascinating innovations coming from fintech and why Zeta is on a mission to make payments invisible.
Read on for some takeaways from the interview or listen to it here:
1. Why Zeta plans to make payments invisible
Payments are typically an elaborate process, involving lots of steps and verifications—making it impossible for the user to not feel its presence.
Zeta’s aim is to shrink this payment experience so the user won’t feel like it exists and use technology to speed up and minimize the authentications and validations involved.
Today, they’re spreading the phenomenon of invisible payments through players in the banking and financial industries. They help these organizations realize their ambitions, like reaching audiences their products haven’t been able to reach until now.
Financial institutes need systems to run their banking products. These systems are usually referred to as core banking products. All the physical branches of these institutions are connected in these products—including customers’ transactions, balances, and fraud detection. These systems are quite outdated and in desperate need of an upgrade.
If a bank were to conceive a product, they need to think about the integration and manifestation of that single product across 30 systems. This process itself could take anywhere between 8 and 18 months and sometimes the product might not even see the light of day!
As technology pervades every aspect of consumers’ lives and continues to make everything easier and quicker, consumers obviously expect their banks and other financial institutions they depend on to level up as well. The pace of the systems needs to be matched with the pace of the customer.
Zeta is on a mission to solve these challenges.
2. How Zeta is innovating fintech
Tachyon, a product of Zeta, is reducing financial institutes’ time-to-market from several years to a few days of support. They’re providing an integrated platform of payments and core banking services as a SAAS to give banks a faster time-to-market and make them future-ready. With this offering, banks don’t have to worry about buying and integrating 10 different systems to offer new products to their customers anymore.
Banks used to look at distribution as purely geographical—having a presence in a certain number of locations. Wherever they’re located is where their business takes place. With the rise of digital financial services, the user base is no longer restricted to physical locations. Platforms like Whatsapp, Swiggy, and Twitter are the distribution channels for the banks of the future and Fusion enables financial institutions to reach these channels and participate in the fintech revolution.
3. Diving into the tech stack
Zeta’s tech stack enables them to achieve maximum security and performance at the foundation.
At the bottom-most layer, Zeta has its complete operations and computing infrastructure platform called Olympus.
Their entire posture on the cloud is completely opaque to the cloud service provider because they’re dealing with extremely sensitive data. They have a zero-trust architecture at the foundation that deals with a lot of Kubernetes, OpenWhisk, Spark, Presto, and more.
Some languages they use include Java and Golang, but the engineers that deal with this layer don’t give the languages as much importance as they do to the raw solutions and algorithms to the complex questions. Some examples of questions include: How do I manage several data centers coming together? How do I respond to an issue in production swiftly?
On the top of the stack, they have a product framework layer where they deal with, for example, the scalability of banks’ accounting systems. Handling 30,000 TPS (transactions per second) is very different and much more complex than serving 30,000 web requests per second.
They do a lot of component engineering, library engineering, and so on. There are engineers working in Vue.js and React Native for the front end and they also have their own native frameworks for delivering UI.
With the problems the Zeta engineering team faces and the platforms they work on, the teams are always having conversations based around architecture and design, understanding the complex business space they serve, and debating the viability of a solution in the short and long term.
How HackerRank Helped Manulife Save $200k+ By Bringing Developers Careers to Life
Dec 01, 2020
When people with a passion for technology get together, they create things that change the world.
That’s the framework for hiring engineers at Manulife Financial Corporation. We sat down (virtually, of course) with Naveed Zahid, Director, Engineering Transformation and Liana Calleri, Talent Acquisition Consultant to discuss how Manulife leverages HackerRank to provide unlimited opportunities for their engineers to develop and succeed throughout their careers.
Listen to the full interview hosted by Aadil Bandukwala, Director of Marketing at HackerRank, or read below for the highlights.
1. How do you provide unlimited opportunities for your employees to develop and succeed?
Liana Calleri
We really are on a digital transformation journey and a big foundation of that is the talent within technology.
About two years ago, we created an IT career framework at Manulife to attract, develop, and retain IT talent. It really aligned with what a lot of other tech companies were doing and continued to follow as well. This framework continues to provide dual career tracks for managers and subject matter experts. It’s enabling us to have a dynamic work environment and provide a win-win for the employees and the company.
We’re also using a ton of different technologies, including JavaScript, React, Node.js, GraphQL, JMeter, New Relic, and Azure to name a few.
Culture matters here. And we are sure to tie this into as an opportunity to develop and succeed. Especially in these times, we are staying connected and engaged as much as we can.
2. What does it really take to bring developers' careers to life?
Naveed Zahid
At Manulife, we focus on having a number of key initiatives that narrow in on that career development of our engineers—which should be front and center for being competitive and innovative in this industry.
Our IT career framework allows our engineers to have very clear expectations of not only what the role entails, but more importantly, how do they get promoted in our organization. So that's the foundation we have established at Manulife to help embrace that idea.
But it goes beyond that. Manulife invests heavily in the learning and growth of our employees with Manulife University. This two-week program offers various different streams including software engineering, quality engineering, reliability engineering, performance engineering, platform engineering and security engineering programs.
But what sets Manulife University apart is that it’s both practical and hands on. We have something called a “proof of technology” where after they've completed their learning phase, they have the opportunity to apply the technologies they were just taught. This is where technologies like React, GraphQL, AKS, and New Relic really come into play. Not only do we teach them, but they get to actually use it. This framework has been highly successful at Manulife.
To us, building that engineering community and culture at Manulife really is important. We conduct internal hackathons that we run quarterly, and strive to celebrate our engineers so we can ensure they’re collectively working together to build great things and innovate.
3. Can you share a couple key metrics that matter the most as an engineering leader?
Naveed Zahid
Recruiting for engineers is so competitive right now. And finding the right individual is even harder.
One of the holy grail metrics that we look at from a recruiting standpoint is trying to minimize the overall time required within the hiring process for both the interviewer and candidate. This keeps our engineers focused on their work and allows us to stay constantly competitive with other tech companies that are trying to attract top talent individuals. The golden standard for us is less than two weeks from that initial contact to that offered letter.
Additionally, we are trying to find opportunities for automating the onboarding experience for our engineers—so the overall experience is exceptional for our future employees. For us, it’s all about how to minimize time-to-attract talent as much as possible. We also try to capture those metrics and opportunities to improve our recruiting process overall.
Liana Calleri
I want to point out, this is not just about talent acquisition. This is implementing company-wide diversity and inclusion efforts as well. Over the next two years, Manulife and John Hancock are investing more than $3.5 million to promote this diversity, equity, and inclusion in the workplace and in the communities we serve. So it’s a really dynamic time to be a part of something like this.
The goals are threefold:
Increase the representation of diverse talent at all levels in the organization.
Create greater inclusion across the company through our enhanced training.
Support organizations helping POC communities.
We also have initiatives like Women in Tech and mentorship programs that all tie back to retention and enhancing our engineering program, so our engineers can thrive in their careers.
4. How does HackerRank fit into your hiring process?
Liana Calleri
I would say one of the most important aspects of how we hire here at Manulife is to minimize bias in our hiring practices. HackerRank allows us to remove bias and focus on the technical competence of the individual.
Using the unbiased scoring system, we can achieve this because candidates can complete tests at home and on their own schedule. This has been a great success for us as a hiring company.
5. How are you measuring the success of the HackerRank platform?
Naveed Zahid
The goal was to leverage the Developer Skills Platform to ensure a solid hiring strategy for our engineering teams. During the Plan phase of the Developer Skills Platform, we started looking at our overall roles and levels at Manulife, leveraging that IT career framework.
HackerRank has enabled us to start curating tests that we are sending out as part of our at-home assessment and leveraging it for our Codepair* entries.
This simplified scoring mechanism has made it so much easier for our recruiters and hiring managers to identify that top talent with a high level of confidence and eliminate any bias.
For the Interview phase, we leverage Codepair* as part of our engineering hiring panel. This allowed us to interact with our candidates as naturally as possible to assess their skills and see how they collaborate with the engineers in our organization. This is really important at Manulife because we have situations where our engineers have to pair with other engineers.
We love how we’re able to capture feedback from multiple interviewers at the same time, which is really instrumental at minimizing that bias during that Codepair* interview.
We're really excited about applying the Rank feature in how we're identifying talent and selecting candidates. I know that functionality was just recently introduced, but we are really excited to start embedding that as part of our process as well.
To date, we have completed about 230 Codepair* interviews. And while those numbers are great, we wanted to look at it from a return on investment perspective.
At Manulife, we’ve calculated a savings of about $215,000 since 2019 because HackerRank allows us to feel confident that we're only bringing candidates that we feel have the skills that we're looking for, as well as the fit to that Codepair* interview. So, when you start taking a look at the overall return on investment, we do feel that HackerRank has been widely successful at Manulife.
How Blackstone Scales Developer Hiring to Create a Positive Economic Impact & Long-term Value
Oct 21, 2020
A talented developer is valuable on their own. But an even more valuable developer is one that works well with others.
That’s how John Stecher, Chief Technology Officer at Blackstone, determines a qualified candidate. “You want a team of people that can actually work together to produce more than the sum of the parts,” says John. “We look for folks that can collaborate well and help each other out.”
Our own APAC Marketing Director, Aadil Bandukwala, spoke with John to learn more about how Blackstone weaves innovation into their tech hiring program and how HackerRank has successfully up-leveled the quality of candidates coming through the door.
Read the Q&A below for high-level insights or listen to the full podcast here.
What is your vision for Blackstone Technology and Innovations?
I want Blackstone to be viewed as a firm that has world-class technologists when compared to other leading companies in financial services. It’s extremely important to me that folks have an inherent curiosity about financial services, especially as we continue to ramp up hiring over the next few years.
Our technology team plays a key role in accelerating Blackstone to be one of the top investment firms in the world. We do this by building and deploying tools and systems that empower our professionals to be as productive as possible. So the more productive we make our team, the more hours we can dedicate to bringing value to our customers.
Time is the one element in life you never get back. And to me, that’s what a lot of technology is. Technology enables people to devote more time to do better work.
Another key pillar in my overall vision from a technology perspective is our Cybersecurity practice. Protecting our clients’ information is absolutely paramount. As we continue to bolster that team, my hope is to implement security into everything we do.
The third pillar is to become a lot more data-centric. As a private equity firm, Blackstone does business with a number of firms that generate very interesting data that can actually help us improve the performance of those companies—as well as help us source new investments down the road.
The fourth element is continuing to become a better team. Teamwork is the key to doing your best work. You need to work hard, you need to have hard conversations, but ultimately, you need to have each other’s back throughout the whole thing. It’s important to build a team that has more of a military mindset of working towards the same goal together. And the more you work for each other, the more likely you are to adopt that mentality.
What are your core values? How are those values weaved into Blackstone Technology and Innovations?
We are forward-thinking problem solvers who can take projects from idea to implementation. I think that’s where we shine: taking ideas from customers and clients of the firm as well as portfolio companies, distilling them down, and building what matters to them. That is really one of the core things we do here. You have to have forward-thinking people, people that understand the problem space, and want to work in it to execute.
We have a very open culture that empowers our colleagues to grow, share ideas, and make a significant impact on the firm. From day one, analysts can come in and have an impact.
Although our team is not small by any standards, the impact somebody can have at Blackstone Technology and Innovations on day one versus at a classic technology company is critical. And I believe that from day one, everyone should contribute to the success of the firm.
Our values involve openness, excellence, and teamwork. We specialize in identifying diverse business challenges across the firm and building the right solutions to drive Blackstone’s success. Sometimes it is building our own solution, other times it requires bringing in external solutions.
I believe good engineers can balance prioritizing a client’s needs and being a team player. When you work with so many diverse professionals across the organization, you really have to understand them in order to work together, know what you’re good at, and what you’re not good at.
What are the skills and qualities that are a requirement to be hired at Blackstone?
We recruit from a broad range of schools, companies, and countries—including all the top engineering schools in the US. But what we really want to bring in is more than skills. One of the best things you can bring in the door is the diversity of viewpoints. So when we’re hiring, we aim to bring in lateral talent and newly hired talent that really has a broad diversity of viewpoints across the board, because that truly helps you build the best solution.
We want to bring together a broad perspective on how to solve problems. So, problem solvers that have diverse backgrounds are really what we look for upfront.
Beyond that, we look for people that really value working together as a team. There is the whole concept out there around acquiring the “10x developer.” In my personal experience with the 10x developer, they actually have a net negative multiplying effect on the teams around them. Because they struggle to work with others well. Now there is, of course, the unicorn 10x developer out there that does it really well.
But for the most part, you want a team of people that can actually work together to produce more than the sum of the parts. And so, we look for folks that can collaborate well and help each other out. And we look for folks that can communicate and understand complex conversations and concepts that exist in the financial services world.
Beyond that, we look for candidates who are inquisitive, client-centric, dedicated, transparent, and innovative. But if you really take a step back, what I look for is somebody that doesn’t just stop at good enough, it’s somebody that kind of takes the next step.
What I’ve discovered is candidates will either just write the code, or they’ll actually take the time to comment. It shows they want to take this further and be better than just average.
Another aspect of an interview that I pay a great deal of attention to is how many questions candidates ask me. The interviews should never just be one-way conversations, and the quality of questions you get back typically dictates the thought process that somebody has. And I’ve found throughout my career that the folks who are inquisitive from an interview perspective typically come across better at the end of the day.
We use HackerRank’s Developer Skills Platform to make sure we’re getting folks in the door who are top-notch from the very beginning. From the core technology side of the fence, it’s a broad technology team. From managing data centers, and core infrastructure, which is where you get into low-level TCP/IP stacks, and all sorts of firewall and network bandwidth issues, which is a very different kind of mindset than a developer has. Then up the stack, we have quite a large swath of Python developers, where we look for folks that are just top-notch. They’re involved in data analytics, data transformation, ETL processes, down into the C sharp and dotnet world, and then deep into the JavaScript world as well.
What have you been able to achieve with HackerRank?
About a year ago, we began leveraging the HackerRank challenge test as the first step of our formal interview process. The questions and results are discussed in the first round of interviews—which makes the first round interview a lot more conversational because there’s a topic. And so, it gives us a catalyst to have a good conversation. That’s important because it gets you both started on the right foot. And also ensures the time spent together is optimized as much as possible.
Then the results with the HackerRank challenge are reviewed at the end of the interview process to evaluate a candidate’s coding ability. So we like to use HackerRank as a bookend: it starts the process and ends the process.
Overall, HackerRank has helped us streamline our recruiting process significantly. In fact, in the time that I have been here, I have seen HackerRank actually up-level the quality of candidates that we are bringing in the door. So as we go forward, and we expect to hire a good deal more developers, I see it continuing to grow in importance and being a real linchpin of the process that we have.
Cloud Computing Expert Kesha Williams on Hiring, Mentoring, & Creating Community in Tech
Aug 11, 2020
Kesha Williams is an award-winning software engineer, machine learning practitioner, and AWS training architect at A Cloud Guru. An Amazon-recognized pioneer in machine learning and an expert Java developer, Kesha is a technical instructor in Java, cloud, DevOps, and machine learning. She’s taught and mentored thousands of developers across the world through her work, and through her social and professional networking platform, Colors of STEM.
She’s also a member of the HackerRank Skills Advisory Council, a panel of tech industry experts dedicated to defining an industry-standardized library of technical skills as a resource for those in tech.
HackerRank Senior Director of Product Management, Dan Somrack, connected with Kesha remotely to learn more about her journey from Java developer, to engineering manager, to cloud computing instructor. Along the way, we got Kesha’s advice on hiring cloud engineers—and her take on the importance of mentoring and community in tech.
Listen to the full interview below, or read on for our biggest takeaways.
A crash course in key cloud computing skills: what they are, and how to find folks that have them
While the notion of cloud computing has existed since the 1960s, cloud computing as we know it today is a much more recent advancement. Popular cloud computing providers like Amazon Web Services (AWS), for example, only emerged in 2006.
Since then, it’s revolutionized the world of tech. “When you’re using cloud computing, you as a developer, or as an organization—you’re no longer responsible for maintaining the hardware,” Kesha explains. “You actually leave that to the cloud providers.” It allows companies to ship products to market faster, and cheaper.
But hiring cloud engineers to build and maintain those cloud environments can be tricky. “Some of the requirements are different from your average software engineering job,” Kesha says.
The key skills of a cloud engineer skill set
When it comes to seeking out cloud engineer candidates, she says the key is to look for someone that understands both cloud architecture and development.
“You can’t just be great at architecture, and you can’t just be great at development. You have to blend those two skills into this cloud engineering role,” Kesha explains. When evaluating a cloud engineer, you need to ensure they have a balance of both skills.
"I've worked with architects that really understand the theory of how cloud services work...but practically, when it comes to implementation, things don't always fit together as neatly as you expect them to,” Kesha recalls. "On the flip side, I've seen very skilled developers...but they don't really understand the pros and cons of the different cloud services, and how they fit together." The key is to find candidates who can comfortably and practically exercise both skill sets.
Kesha’s advice for identifying strong cloud engineer candidates
But cloud engineer skills aren’t necessarily easy to assess in a simple 1:1 phone call. To begin narrowing down top candidates, she says, you can start with looking for certifications. But they’re not the be all and the end all to finding the right candidate.
“If they have the certification, and the job experience, and they’ve mastered one of the cloud providers—like Amazon Web Services (AWS)—it’s a good base,” Kesha says. For AWS, some of the certifications she looks for are the AWS Solutions Architect certification, and The Cloud Practitioner certification for candidates newer to the field.
“Make sure that they understand the different compute services, the storage services, the security services, and networking, and how they all fit together,” Kesha says. Looking at their side projects, too, can help garner a more complete understanding of their experience. As they’re evaluating candidates, she says, hiring managers can also use the HackerRank Skills Directory as a guide to leveling candidates. It defines a wide variety of key cloud computing skills, along with key competencies expected at the basic, intermediate, and advanced levels for each.
Upleveling cloud computing skills as a developer
And for developers looking to uplevel their skills in cloud computing, or prepare for a cloud engineer interview? Kesha says the Skills Directory can help them level themselves and determine areas to skill up. “For developers, it helps identify the areas that you need to know, or maybe areas where you’re strong, and areas where you’re weak and may need to skill up before an interview,” Kesha explains.
“Think of it like a learning path, really, with an outline to follow” Kesha says. “It shows you, for the basic competency [in a skill], these are the things that you need to know, or for intermediate, or for advanced.”
For those getting started in cloud, Kesha suggests starting with a foundational skill. “Always have a foundational skill that can apply to the cloud, but that you can fall back on,” Kesha explains. “For me, that foundational skill is Java.” From there, you build layers of expertise. “Once you have that foundational skill, then master one of the cloud service providers like AWS. And then, once you’ve mastered their basic services, pick a specialty,” she suggests. Whether it’s the internet of things (IoT), machine learning, or cybersecurity, taking this layered approach will help build up your experience. “It just really helps you stay marketable,” Kesha says.
The role of mentoring in tech: tips for getting started & creating community
In addition to being a cloud instructor and speaker, Kesha also dedicates much of her time to mentoring. She’s a mentor at Technovation, the New York Academy of Sciences, and her own organization, Colors of STEM. In a lot of ways, her motivation to mentor others stems from her own history in tech.
Getting started in mentoring
“When I was coming up in tech, I didn't really have a lot of female role models, or role models that looked like me. And being in the industry for 25 years—over those years, there were actually several times where I considered leaving tech because of how alone and isolated I felt,” Kesha explains. “And I just didn't want others to feel that way, and so I wanted to be for others what I never had.”
For those that want to get started in mentoring, Kesha suggests using a framework with mentees that helps relate the tech back to day-to-day problems. “I try to show them how it can apply to real life. That’s the first step,” she says. “And then I try to demystify the technology, and just show them that it’s not as complicated as people make it out to be. And I do that through real world examples.” For example, in a cloud computing mentorship course she taught for Women Who Code, she highlighted a soda theft detection model and an emotion detection model she created at home with AWS DeepLens.
The importance of finding community in tech
Mentoring is just one way that Kesha helps others to build community in tech. To her, it’s been an influential force in her career. “There have been many times where I’m the only female, or I’m the only African American [in the room]. And what’s worked to keep me in IT was realizing that there’s nothing else I enjoy doing,” Kesha explains. “There’s nothing else I want to do with my life. And so I had to find other people that were just like me.” For Kesha, that came in the form of building community.
“I joined Women Who Code many years ago, and that, for me, was a saving grace,” Kesha recalls. “I would attend events, and I would just look around the room, and I would see people that were just like me. And having that community is really what kept me in IT.”
From 0 to 800+ Remote Interviews: Lessons from ServiceNow’s Two-Week Transition to Remote Hiring
Jul 10, 2020
To respond to COVID-19, ServiceNow had to take their entire hiring process online—which meant a transition to 100% remote interviews. In this interview, NancyDeLeon, Director of Global Talent Acquisition, explains how they managed it.
At its core, ServiceNow’s purpose is simple but impactful: “to make the world of work, work better for people.” By crafting digital workflows for IT, employees, and customer support, ServiceNow is known for streamlining complex workflows—and helping companies improve productivity in the process. It’s how they’ve been able to help drive widespread digital transformation at their 6,200+ global customers.
But when the COVID-19 pandemic took hold, the meaning of “work” took on a new definition. As safety necessitated a shift to work-from-home, ServiceNow’s 75 global offices were suddenly emptied. While employees began to adapt to this new form of work, companies around the world began to freeze or slow hiring. Hiring at ServiceNow, however, continued at its original pace: 30% growth year over year.
To continue to fuel company growth, ServiceNow shifted to 100% remote interviews. To learn how they did it, HackerRank CEO and Co-founder, Vivek Ravisankar, connected with Nancy DeLeon, Director of Talent Acquisition at ServiceNow. In their remote interview, they talk through how Nancy and her team led the Americas engineering organization from 0 to 800+ remote technical interviews in a matter of weeks, and took their entire internship program online.
You can listen to Vivek and Nancy’s full interview on HackerRank Radio, or read through the highlights below.
Three lessons learned from a two-week transition to remote interviews
Like many companies, the onsite interview was a core component of ServiceNow’s technical hiring process. But that changed quickly once employees started working from home. “Changing from onsite to a remote interview environment—it may not seem like such a tough thing to accomplish,” Nancy says. “However, going to 100% virtual interviews in less than a week in a high volume environment... that's going to be quite challenging.”
Granting employees access to remote interviewing tools wasn’t nearly enough. It necessitated a complete revamp of their hiring approach. “We had the daunting task of training a global engineering team, our recruiting team, and our talent acquisition operations team in a very short period of time,” Nancy recalls. “And we said, you know what? With HackerRank and ServiceNow: challenge accepted. And the results were fantastic.”
Lesson #1: Focus on training and documentation first
Transitioning the entire engineering organization from onsites to remote interviews was no easy task. The entire hiring organization—from recruiting, to engineering, to operations—had to relearn how to facilitate and run technical interviews. To ease the learning curve, Nancy’s team started by distributing training for each stakeholder organization. But documentation was the main tool they used to keep hiring on track.
“So, internally, we created a knowledge base,” Nancy says. “This allowed us to have a reference point where our internal teams knew where to go, and how to get things scheduled.” Having a central point of reference, she says, was a core driver for helping the team transition. Interviewers’ comfort with this new process was key for facilitating a positive candidate experience.
“The interviewers were able to find the questions right away. They were able to implement the questions—and the best part was the whiteboarding aspect, which made it very easy,” she explains. “On the candidate side, they felt that they were able to really showcase and highlight their skillset.”
Lesson #2: Tap into the power of existing tools and partnerships
According to Nancy, it’s ServiceNow’s existing talent tech stack that helped them pivot quickly. By leveraging HackerRank their team was already familiar with, they were able to scale from 0 to 800+ remote interviews in just a few weeks. “We were able to successfully implement about 800+ CodePair* interviews—and that's not including the CodeScreen* test that we actually rolled out [beforehand],” Nancy explains.
While that shift was driven by her team’s planning and coordination, Nancy says it couldn’t be done without leveraging their existing tools and partnerships. “Part of this success was because of the strong partnership with HackerRank. The team helped us with a seamless process and that resulted in our team being able to meet the high demand for the remote interviews.”
To Nancy, that’s what made the transition for interviewers feasible. “They really felt the tool was very easy and comprehensive and with the job aids, it was very intuitive,” Nancy explains. “So on the interviewer side, it was very seamless.”
Lesson #3: Make room for candidate feedback at scale
Facilitating feedback on the new process was key to ensuring a smooth transition. That boiled down to gathering internal feedback from interviewers as well as candidates being interviewed. “We had to capture success metrics, and of course overall satisfaction,” Nancy says.
It’s those metrics around satisfaction and experience that helped Nancy’s team monitor candidate sentiment. By monitoring candidate sentiment, they were able to keep tabs on what was (and what wasn’t) working.
“When I was measuring candidate experience with HackerRank CodeScreen*, it was actually one of the surprise results that I got. I was actually expecting much more of a negative result since there wasn't a human being on the other side,” Nancy explains. “But in fact strangely enough, most candidates said, ‘I really feel it was unbiased.’” That early feedback gave her team confidence to double down on their process.
The new future of work: ServiceNow’s take on distributed work and the road to recovery
At this point, the future of work is, in a lot of ways, uncertain. And though hiring continues for ServiceNow, they’re still in the process of determining what work post-COVID will look like. Nancy thinks it’s unlikely that work will return to the pre-COVID “normal”—but she does think there will be more room for face-to-face interaction once recovery starts.
The post-COVID workplace: an opportunity for flexible working?
Like most workplaces, remote work has forced ServiceNow to shift to an entirely remote hiring process. And given its success, she thinks it may be around to stay.
“We are going to continue with remote interviewing. We've successfully proven that we can hire and work virtually while maintaining our productivity,” Nancy says. “So, I foresee that this is going to continue as an option for our candidates and our employees.”
But on the other hand, Nancy doesn’t feel physical meetings will disappear entirely. “I also recognize that we're social beings and we're naturally going to need to be physically connected in some way,” Nancy explains. “So, I also see that we are going to still have some onsite interviewing, and working physically in the office will also return. But I see that as possibly an option—versus mandated—as a future part of our workforce.”
For ServiceNow, she sees the flexibility between physical and remote interviews as a talent branding need, too. “To be competitive as a company, we're going to have to consider how we’re able to provide those types of options, and how we’re going to be enabled to do both.”
Advice for candidates on the hunt for work
As the world sets eyes on the road to recovery, Nancy has some words of advice for candidates that have unexpectedly re-entered the job market due to layoffs and furloughs. “My best advice for candidates who are looking for their next opportunity is, number one, please take care of yourself and your family,” Nancy says. “Losing a job is like losing a little piece of us—so we are all grieving. Take the time to re-center yourself.”
She advises taking a strategic approach to searching for a new role. And part of that strategy is about taking the job search in stride. “Don't sit in front of your computer eight hours or a full day looking for a job,” Nancy suggests. “Instead schedule that time. Send out your applications, and then for the rest of the day, do something for yourself. Invest in your development and think about the future you want to be in.”
*Disclaimer: This blog post contains messaging around the "CodePair" and "CodeScreen" features which are now called "Interview" within the HackerRank product as of 10/06/20.
Making Remote Work: How Bloomberg Adapted to Virtual Interviews
Apr 29, 2020
Seemingly overnight, remote work—and by extension, virtual interviews—have become the new normal. For those hiring, the transition forced interview processes to adapt to a remote format in a matter of weeks. For most, it’s been a hectic, challenging transformation (to say the least).
But luckily, the transition to virtual interviews was something that Bloomberg’s engineering organization had inadvertently prepared for. They’ve been refining their engineering hiring process for years, focusing on creating a standardized, consistent hiring process that maintains a static hiring bar for the entire organization. It’s a huge part of what helped them make the transition to virtual interviews in a matter of weeks.
Since 2014, Kristen Arena has been helping to shape Bloomberg’s hiring process across both campus and experienced hire recruiting. She currently leads a team in campus recruiting, and is the lead recruiter for their CTO office.
HackerRank CEO and Co-Founder Vivek Ravisankar sat down with Kristen to learn how Bloomberg managed to adapt to virtual interviews in the face of stay-at-home orders. You can listen to their full audio interview below, or keep reading for our key takeaways.
How Bloomberg’s existing interview framework helped ease the transition
Even before stay-at-home orders were implemented, Bloomberg’s team had relied on a standardized interview process to hire for its engineering team. And according to Kristen, it’s those standardizations that helped ease the transition to virtual interviews.
Kristen says that though the process has some nuances across different roles, the interview process for most Bloomberg engineering roles follows a similar format. It generally consists of a phone interview followed by a few rounds of in person technical and HR interviews, either in our office or on campus.
Leveraging their existing foundation
“We have a very solid foundation, and a structure in place,” Kristen explains. Bloomberg used that existing structure to translate it to an online process. “We were able to say: ‘Alright, how do we keep it intact? How do we do it remotely?’”
The goal was to take the existing interview process—from tech phone interview to in person HR interview—and replicate it in a virtual setting without sacrificing the personal touch of an in-person experience. Though the first step of the interview process was already conducted remotely, the other steps were traditionally conducted on-site. So they had to be translated to a remote experience.
On top of that, they also had to adapt their internal procedures to maintain the same evaluation process: from pre-interview briefs, to interviewer prep, to group debriefs.
Replicating the on-site interview process virtually
Bloomberg’s transformation to a virtual interview process happened over a matter of weeks. But despite the tight timeline, they spared no amount of effort in ensuring the new, virtual process was thorough and candidate-centric.
Adapting candidate prep and increasing recruiter touch points
One change that Bloomberg had to account for: bolstering candidate prep. “We have always prided ourselves on really making sure that our candidates feel comfortable with the interview process,” Kristen explains. It’s why they’d already invested so much in their robust library of candidate prep resources before the transition to remote work. “Our approach, given the situation, was to just adopt that same mentality, but heighten it.”
To go the extra mile to get candidates comfortable with this new format, the Bloomberg team implemented a few primary changes:
Adding a virtual interview prep call: Each candidate gets a personal call from a member of the recruiting team to walk through the logistics of the interview. They share the schedule, what to expect in each round, explanations of when they’ll be able to take breaks, how much time they’ll have between rounds, expected next steps, and more.
Testing the interview setup one-on-one: The team also helps candidates head off technical difficulties by testing conferencing and interview tools with the candidate ahead of time. “From a technology perspective, we want to make sure they’re ready to go—so testing out the conferencing tool that we use, making sure they’re comfortable with HackerRank, and that they’re aware of the diagram feature within HackerRank we use for design questions,” Kristen says.
Simulating the in-office experience: Even from afar, Kristen’s team also goes above and beyond to help candidates experience the day-to-day of working at Bloomberg. For example, in lieu of an office tour, they share videos of the office with content about their company culture. To go a step further, they even give candidates a peek into the product by offering regular demos. “If you're a candidate and you have an interview coming up, you can drop into a video conference at some point during the week for a demo,” Kristen says. “It’s a little snapshot of Bloomberg, and a demo of our product—just to learn a little bit more about us before your actual interview.”
Together, these new measures help put candidates at ease—and in turn, enable them to perform their best in the interview. “In the context of the remote interview, things can get really lost. So we want to make sure that they feel comfortable,” Kristen explains.
Rethinking interviewer prep and feedback
Changing to a virtual interview process also necessitated a change in internal prep—especially when it comes to interviewers. “Our goal is always to make sure that the candidate is comfortable and has a great experience. And that really comes directly from the interviewers,” Kristen notes. “When we transitioned to remote interviewing, we were well aware that this poses a whole new set of challenges for interviewers as well as candidates.”Luckily, their interviewers already had a strong base to build on, since all interviewers at Bloomberg go through a training process before interviewing candidates. “They’re trained on how to assess various competencies, how to ask technical questions, and how to ensure each candidate will be a culture add to Bloomberg,” Kristen explains.
The structure gave them a head start on transitioning to virtual interviews—but they still had to brace for the change. To prep, they helped interviewers replicate the on-site, off-site by focusing on a few key asks:
Conducting virtual pre-briefs: Before the interview happens, the recruitment team brings the interviewers together to discuss the day-of game plan, just like they would for an in-person interview. They discuss what each round will cover, who’s covering it, and answer any questions—for every single interview, both for campus and senior candidates. “We use that same mentality going throughout the interview. So, we’ll use our virtual conference room to jump in, to share feedback, and to go into the next round,” Kristen says.
Sharing virtual interview best practices: To help the interviewers adjust to this new medium, the recruitment team shares their own tips for virtual interviews. For example, they asks that they create a neutral environment without distractions, and do their best to replicate eye contact by looking at the camera when speaking. Without body language to pick up on, some candidates can be even more nervous than usual; she asks interviewers to go out of their way to make candidates comfortable so it doesn’t impact the interview.
Maintaining group debriefs: “We’re very consensus-driven,” Kristen says. “So when we debrief, we want to hear everyone’s feedback.” Just like they would for an in-person interview, the recruitment team invites anyone the candidate has met or spoken with to a group debrief. Having an open conversation—albeit from afar—ensures that nothing is lost in translation from interviewer notes or assessments.
On the future of (remote) work and hiring
The interviewing process may have changed, but according to Kristen, Bloomberg’s hiring plans haven’t changed. “We’re hiring as usual,” she reports. Their internships and work has gone 100% virtual for the time being, but hiring hasn’t slowed. “Candidates are still looking for roles, and we’re still looking for candidates. So we’ve just been trying our best to make sure that our process aligns with the current situation, and that candidates feel supported.”To Bloomberg’s recruitment team, that means adopting a candidate-centric approach: one where empathy dictates the interviewing process. “I think it's that relationship-building piece, and really letting the candidate guide the process: listen to them, listen to what their challenges are from a remote perspective, and be empathetic as well. I think that human aspect is critical,” Kristen says.
The flexible future of interviewing
Will we revert to old hiring processes and practices post-pandemic? Kristen is cautiously optimistic. “I do think that, hopefully, someday, we will return to in-person interviewing,” she says. “I think that will always be an element, because it’s valued and important.”
But on the flip side, she says, she could potentially see a world where candidates could have more options to choose virtual interviews—but nothing is set in stone yet.
For Bloomberg, the process changes forced by stay-at-home orders have inadvertently tested their adaptability, Kristen notes. “I think [these changes] just opened another door—another way of being able to recruit in a way that maybe we didn’t think was possible or useful before. But it is. And we’re doing it.”
Tips for Remote Hiring: How Atlassian is Making Virtual Interviews a Reality
Apr 03, 2020
Remote hiring is a process that companies across all sectors are now incorporating into their hiring strategy. To help set your team up for remote hiring success, HackerRank’s Co-founder and CEO, Vivek Ravisankar, spoke with Atlassian’s Andy Mountney about remote hiring best practices.
When it comes to remote interviewing, Atlassian is ahead of the curve—in fact, it’s something they’ve been doing since their founding. Originally started in Australia, Atlassian began hiring remotely as a means to bring international talent to work at their Sydney HQ. Pursuing international talent meant some candidates weren’t able to attend a traditional onsite—so remote interviews became a must-have. They’ve been refining and perfecting their remote hiring process ever since.
As the Talent Acquisition Site Lead for Atlassian’s Sydney location, Andy Mountney has been helping his technical teams conduct remote interviews since 2018. To hear his advice on creating a smooth remote hiring process, read the key takeaways below.
Define, document, and communicate a consistent remote hiring process
Changes in policies and procedures will inevitably happen when making the shift to remote hiring. To minimize room for dysfunction, Andy advises that teams spend time defining and documenting what the new remote hiring process will look like. Then communicate the process to internal team members and candidates.
Start by defining & documenting your process
Explicitly defining the new workflow prevents confusion and empowers interviewers to deliver a positive remote candidate experience. It also proactively answers questions from interviewers, and enables teams to adjust quickly if a teammate—or the entire company—suddenly has to work remotely.
“I think one of the real strengths we had when making this pivot wasn't just the fact that we've already conducted so many remote interviews. It was the documentation that we had to support it,” says Andy. “With it, we could lift very quickly at scale and build consistency into that change...so I’d certainly encourage people to document it and think about it.”
Communicate clearly, and often
What’s next after defining and documenting your new remote hiring process? Communicate it to your candidates. Many candidates have never gone through an entirely remote hiring process; it’s the recruiter’s responsibility to set expectations. Andy says it’s also crucial that recruiters communicate the intended outcome of the process, should there be a mutual fit.
“It's important to make sure that the candidate understands [if] there could be an offer coming out of this process still if it's successful,” says Andy. “You want your recruiters to be [asking]: Will you be ready if we make an offer? How can we help you make that decision?”
Especially when taking your hiring process remote, it’s helpful to amp up your candidate communications. Atlassian has done this by increasing the number of digital touch points they have with their candidates throughout the hiring process—just one of the ways they’ve adapted their in-person hiring process to a remote-friendly one.
Don’t skimp on candidate or interviewer prep
It’s natural to feel more relaxed when working from home. But it’s important for candidates to treat the remote interview the same way they would treat a traditional onsite. Andy encourages candidates to dedicate plenty of time to prepping; from his perspective, it’s the best way they can set themselves up for success.
Preparing to interview from home
“Typically, if you were going to an interview as a candidate, you're going to prepare for going to a physical place. And you're going to think about, perhaps, your journey there,” says Andy. “If you're interviewing remotely as a candidate, it’s still important to take that step and just think about the environment you’re going to deliver your interview in.”
At Atlassian, Andy says it’s standard to send candidates a guide with interview prep tips and suggestions. He likes to include things like:
Instruction on how to access the video interview tool they’ll be using, and any other tools or software they’ll need for the interview
A reminder to choose a clean and comfortable interview space
The names and LinkedIn profiles of the people on the interview panel
Adopt a 10-minute rule
Andy says it’s equally important that the interview panel prepares for each interview.
“From the interviewer's perspective, it's important to make sure that interviewers are set up to switch into that interview mindset—just as they would be in the workplace.”
To enable that, Atlassian likes to set aside an extra 10 minutes before the interview for both the interviewer and the candidate to get ready. During these 10 minutes, the interviewer and the candidates have the time to set up, calm any nerves, and get ready for their discussion.
“We block that time out in the interviewer’s calendar so that they have that 10 minutes to get their brain into the right place to deliver the interview, get their laptops set up, and get their tools set up for the interview,” Andy explains. “I think that really helps calm someone down and get them back into that right place to deliver.”
When in doubt, practice empathy
For most candidates, remote interviewing is a new experience. And to make that transition smooth, empathy is a must-have. “I think for remote interviews, it’s just about thinking ‘How can I help facilitate something which is possibly a first-time experience for the person on the other end?,’” Andy stressed. And that starts with the interviewer.
Focus on putting the candidate at ease
“As an interviewer, you have as much responsibility as the candidate does to make it a great experience,” Andy says. To accurately evaluate fit, it’s key that interviewers give candidates a welcoming, comfortable atmosphere to showcase their skills.
“For the interviewer, [it’s important] to make the interviewing experience as comfortable as possible, and to ensure that the [candidate] is only being evaluated on the skills and attributes that they should be.” That means keeping focus on the candidate’s skills—not their video background, or accidental interruptions from their family.
Evaluate on skills, not minor mishaps
Say, for example, the candidate is having trouble with their video tool in the first 5-10 minutes of the interview. While some interviewers might see this as a red flag, Andy stresses that this isn’t a chance to judge the candidate. Instead, he feels strongly that interviewers should keep the evaluation to the interview itself.
“[It’s a] big responsibility of the interviewer to help calm that person, settle them, and help them reset [in the event of an issue],” Andy argues. “It’s not an opportunity to judge.” A little bit of empathy for the candidate can go a long way, and can help you stay focused on what matters: their skills.
Use the flexibility of remote hiring to your advantage
Since remote interviewing can be done from home—versus an office—it’s inherently more flexible. Andy sees that as an opportunity to build an even stronger relationship with the candidate.
“I think one of the advantages of the remote [interview] is that you can build in breaks very comfortably...whether it’s hard stops over the course of 24 hours, or a longer period of time,” Andy notes. With remote hiring, companies have the chance to break the interview process up into smaller, more manageable pieces.
Take, for example, breaks between interviews. “Typically, [you’d be] trying to fill that space if somebody was onsite,” Andy explains. To keep the candidate engaged, you might have to orchestrate activities like office tours, team lunches, or coffee breaks.
But by cutting them out, you give candidates more room to reset, or prepare for their next round. “Now, you’re actually giving that time back to candidates.” And that helps candidates get in the right headspace to deliver a great interview.
The future of remote hiring at Atlassian
So how will this shift to remote hiring impact Atlassian’s hiring process in the long run? “We always want to learn through experience—and inevitably, we’re going to learn through this experience.” Andy believes. “We’re really fascinated to see the impact of a remote-only interviewing model.”
Atlassian regularly surveys candidates on their interview experience: the highs, the lows, and everything in between. They plan to keep a close eye on that data to learn from this transition as it unfolds.
“Will it enhance or deliver specific changes that we’ve identified today? Absolutely not,” Andy admits. “But when we get back in the office, it’ll be a good opportunity to retro, reflect, and understand what the opportunities are: what we could’ve done better, and what [the process] will look like going forward.”
Want to listen to the conversation live? Catch Vivek and Andy’s interview on HackerRank Radio
Beyond Technical Acumen: Kaggle’s CEO on the Key Elements of a Data Scientist Skill Set
Nov 27, 2019
What are the core components of a strong data scientist skill set? And how should you evaluate a data scientist candidate once you find them? HackerRank CEO and Co-founder, Vivek Ravisankar, sat down with Kaggle CEO and Co-founder, Anthony Goldbloom to explore the ins and outs of this growing role.
With over 3.8MM users, Kaggle is the world’s largest data science and machine learning community. It’s home to 25,000+ public datasets, nearly 300,000 public notebooks, and a library of data science micro-courses. Through their data science competitions, they encourage their community to tackle real-world machine learning problems across industries.
As CEO and Co-founder of Kaggle, and as a data scientist himself, Anthony Goldbloom is one of the foremost experts on the exploding field of data science.
Its rapid rise to popularity has led to confusion amongst hiring teams. What skills does a data scientist have? What do they work on? And what’s the difference between data scientists and machine learning engineers?
To understand through the eyes of an expert, we sat down with Anthony to learn how data science as we know it came to be—and what skill set defines a “data scientist” today. Here’s what we learned:
2012 and 2018 were defining years for the field
According to Anthony, 2012 was an annus mirabilis (or, a “miracle year”) for deep learning. With the introduction of neural networks, machine learning and artificial intelligence (AI) took off. That lead to a landslide of advances in natural language processing, speech, and computer vision. It’s why we’ve seen such a boom in applications of computer vision across cases like self-driving vehicles, radiology and security cameras.
Thanks in part to those advances, neural networks and gradient-boosting now play a major part in defining the day-to-day of a data scientist. “Those are the two things I think data scientists today are spending most of their time on,” Anthony said.
Hear the extended audio interview on our podcast, HackerRank Radio:
But advances in 2018 may change that focus moving forward, according to Anthony. “You could argue that 2018 was the annus mirabilis for natural language processing.” With new advances in natural language process, we’re bound to see an increase in use cases.
“Just as autonomous cars, and radiology, and some security use cases have been unlocked by computer vision—what sort of use cases might we see around natural language processing?” Though only time will tell, teams may see those use cases impact hiring needs moving forward.
The role "data scientist" has many meanings
Within data science, there are a number of distinct roles, from data analysts, to data engineers, data scientists, and more. But the difference between them isn’t always clear amongst employers. “I’m not surprised that companies are confused—because it is fairly confusing.”
When it comes to distinguishing data scientists from data analysts, Anthony’s criteria is simple. “I classify a data scientist as someone who writes code in order to produce inputs.” A data analyst or business analyst, on the other hand, leans primarily on tools like Tableau or Excel. A data scientist’s role can also span a wider set of responsibilities. “[A data scientist] could be doing anything from writing pivot tables all the way through to training machine learning models.”
But even the term “data scientist” has its own set of nuances. To Anthony, “data scientist” is an umbrella term that can be used to describe a variety skill sets and focal areas. In his eyes, there are 2 primary subcategories: type I data scientists, and type II data scientists.
Type I data scientists
Type I data scientists focus on building algorithms that will go into production. The algorithms that power Facebook's newsfeed and Netflix's content recommendations are good examples.
As a part of their work, most type I data scientists spend their days training a machine learning algorithm (often, either a neural network or a gradient-boosting machine). This is the category where Anthony places Machine Learning Engineers and AI Engineers.
Type II data scientists
The work of type II data scientists, on the other hand, generally isn’t destined for production. Instead, this type of data science focuses on deriving and analyzing insights that a business can ultimately productionize. “It’s insights that you can productionize,” Anthony says.
A type II data science may use similar tools to a type data scientist, like machine learning algorithms. But they may also be working on something as simple as a pivot table.
A strong data scientist skill set isn't just about technical know-how
No matter the subtype of data scientist, one thing is clear: technical acumen alone doesn’t an effective data scientist. Instead, a well-rounded data scientist has a combination of both hard and soft skills, including (but not limited to) technical skills, business acumen, creativity, and communication. So hiring teams need to look for much more than technical expertise.
Underlining the importance of business savvy
Anthony has seen this firsthand through competitions on Kaggle. “People will build an amazing algorithm on [a] problem,” Anthony explained. “But if the evaluation metric is wrong, or the target variable isn’t a variable that’s useful to predict, then the algorithm is completely useless.”
Understanding the algorithm’s ultimate application is the difference between a functional, but ineffective algorithm, and one that solves a real problem. Even the most technically sound algorithm—if designed to produce a meaningless output—won’t yield a desirable result. “Building an effective algorithm requires you to be strong technically, but it also requires you to have good business context.”
To produce good work, a great data scientist needs to know how their work adds value to the company, and ultimately, how it will be used.
How Anthony's team evaluates Kaggle profiles (hint: it's not about competition scores)
So, how does Anthony’s team find data scientists with this unique skill set? For starters, they turn to Kaggle. “Our community’s actually a very good signal.”
But reading a Kaggle profile isn’t as intuitive as you might think. “We grade people on 3 criteria: their competition performance, the kernels [and notebooks] they share—how many upvotes they get—and their contribution to discussion [in the forums].” Typically, a candidate that’s done well in any 1 of those 3 areas will get an interview.
That said, it doesn’t mean that all criteria are created equal. “Of the 3 criteria...I care about most about discussion,” Anthony said. Why? Because it’s an indicator of both technical and soft skills. “You only get upvotes if you are technically insightful and you’re a clear communicator.” But in competitions, for example, you can be technically strong with poor communication skills—and a strong data scientist skill set requires both.
Where the most successful data scientists come from
Data science is still a burgeoning field—so candidates that come to data science from diverse backgrounds are something of a norm. With the right mix of technical expertise, curiosity, storytelling, and cleverness, people from virtually any field can become a data scientist.
In fact, an analysis on graduates of the Insight Data Science Fellows Program—a training fellowship designed to help PhD graduates transition into data science—showed that successful fellows stemmed from fields ranging from Physics to Neuroscience to the Social Sciences.
Of roughly 700 Insight Data Science fellows analyzed, graduates came from a variety of academic backgrounds ranging from Physics to Social Sciences (via Scott Crole)
Given the variety of backgrounds they come from, resumes and verbal interviews aren’t always the most effective way to evaluate the skills data scientists have to offer. When it comes to hiring, Anthony has had the most success evaluating data scientists through miniature projects.
“[We’d] give them a project that we cared about, or that would look like a project they’d tackle internally,” he said. For Anthony, it gives a more nuanced look into their skill set as a data scientist. “We learned more from that than anything else, frankly.”
The project-based approach didn’t just spotlight their technical skills. By asking candidates why they made the decisions they did, it also gave the panel an opportunity to explore soft skills like business savvy, storytelling, and communication. “And I think there’s probably just about no substitute for that—because a good data scientist can come from a very wide variety of backgrounds.”
Evaluating data scientist skill sets
Data science is an evolving field—and like any new field, it’s going to align on the nomenclature, the use cases, and the skill sets that define it. But by diving into the history of the field—and the way it’s applied today—you can better understand how to hire them.
Hiring data scientists at your organization? Read more about HackerRank Projects for Data Science, or read what we learned about data science hiring from our survey of 70,000+ developers and technical professionals:
Connecting Global Tech Ecosystems: Andela Shines a Light on Africa’s Developer Talent
Aug 12, 2019
This is the 14th episode of HackerRank Radio, a podcast for engineering leaders interested in solving developers’ toughest problems today: Hiring the right developers. Hosted by Vivek Ravisankar (CEO & Co-founder, HackerRank). You can subscribe to us on iTunes and Google Play.
One of our favorite discussion topics at HackerRank is the widely debated talent shortage. Is there a real lack of tech talent in the market? Or are we missing qualified candidates when we source? The latter can certainly be true, especially if you limit your talent pool to your backyard. But casting a wider net isn’t always a go-to strategy for most companies.
Andela is on a mission to change that. The company specializes in engineering-as-a-service and enables organizations to accelerate their product roadmap. How? By building globally distributed engineering teams with Africa’s top software developers. It may not be the first place you think of when sourcing developer talent, but the continent is home to an up and coming, fast-growing tech scene.
In our latest episode of HackerRank Radio, our CEO and Cofounder Vivek Ravisankar spoke with Jeremy Johnson, CEO and Cofounder of Andela. The two dive into everything from the tech talent shortage, the experimental advantage startups have, and the industry shift to building distributed teams.
Listen to the episode or skim through the transcript below.
Vivek: Welcome to HackerRank Radio episode 14. Today’s guest is at the forefront of putting Africa’s developer talent or the map. Andela helps companies build distributed engineering teams with Africa’s top software developers. I have the pleasure of speaking with their CEO and Cofounder Jeremy Johnson. Welcome, Jeremy.
Jeremy: Thanks Vivek. It’s great to be here. Thank you for having me.
Vivek: Okay. Are you excited to be at this podcast?
Jeremy: Yes, I’m excited. Of course. We’re big HackerRank fans in general.
Vivek: That is good. That is, by the way, for all the listeners that was not planted that was totally all Jeremy. I gave a quick intro about Adela but just like a couple of lines, maybe would love to get an overview of about you, about Andela. If you can get started.
Jeremy: Yeah, of course. So, you know, my background has been in the technology space essentially since college, I after winning a business plan competition dropped out of Princeton to start a company designed to try to facilitate the college admissions process for students from underrepresented high schools around the US. And that came out of growing up in Trenton, New Jersey, and being sort of pulled between two worlds between Trenton and Princeton growing up. I’ve stayed in and around that space for most of my life in that sort of education technology vertical. I co-founded a company called 2U after that, which we took public in 2014.
I was invited at the beginning of 2014 in January out to Nairobi to give a talk for the MasterCard Foundation. That kicked off a series of conversations, in large part, around why there was so little business interaction between the tech ecosystems that were growing rapidly across the continent of Africa and the more established and visible ones in the US, like, you know, Silicon Valley and you know by that point New York as well. And, you know, rapidly expanding across the, across the country.
And so, as that progressed and we started to dig in we realized something that we thought was pretty profound, but also really simple, and that’s that aptitude and cognitive ability are completely evenly distributed around the world. You’ve probably heard the phrase talent is evenly distributed, opportunity is not. It’s one that we’ve spent a lot of time trying to help the world see.
When you think about what happens in the tech world over the past. you know, five to 10 years you’ve seen a pretty significant shift towards comfort with not needing everyone to be in the same location so distributed teams. Initially, it was remote and then distributed and if you believe that brilliance is evenly distributed, and you believe that you can quantify the output of engineers and therefore are comfortable with distributed teams, all of a sudden Africa becomes the most obvious thing in the world.
It’s the youngest fastest-growing continent on the planet. You have a very large English-speaking population and you have a generation of young people that understand the impact technology can have and the smartest, most driven people wanting to be software developers. And that’s a very different phenomenon and set of circumstances than you’re going to find anywhere else in the world and it’s created I think a lot of the foundation for Andela’s success, you know, fast forward five years.
Vivek: Yeah that’s a fascinating story. And it’s amazing that the reason why we started HackerRank was based on a very similar thesis, which is what we call ‘skill over pedigree.’ What matters is are you able to do the job versus which college did you go to, or which company you worked at before. I think what matters is do you have the skills to do the job or not. So it’s based on a very similar thesis.
Walk us through your early days.
The birth of Andela
Jeremy: There were two separate streams that came together. One was that that conference and I was invited there by Christina Sass or co-founder and longtime president and I also had a friend and mentee who had gotten to know over the couple years before this, who was a young Nigerian entrepreneur, who reached out to me in the middle of my 2U days to ask for feedback on his prior startup, which was essentially trying to build a MOOC platform, a Massive Open Online Course platform, that was Africa focused. And You know, I met him for the first time in Europe was like, all right, this guy’s awesome. Well, I’m happy to help them whatever I can. And so we became friends. And when I got back from that trip to Nairobi, he happened to be coming through New York and had decided that the company wasn’t going to evolve the way that he was hoping it would. And so wanted to figure out what is going to do next and asked for advice and we sat down and I pitched him on the notion of thinking a little bit differently about how you connect extraordinary talent and potential talent into the global tech ecosystem, leveraging a very like high-quality version of education technology. He and his co-founders, the previous company were like, that sounds super interesting. Yeah, we’ll give it a shot.
And so, you know, the four of them, me, Christina decided to kick off a pilot and there’s six of us. And so it’s, it was a fascinating founding group because you one large number of co-founders, but also three from Africa and three from North America, and I think that was actually a critical part of really why Andela was able to find its footing on both continents early on. You know, two people from Nigeria one from Cameroon two from the United States and one from Canada, so it was a diverse cast of characters.
We decided to pilot an experiment in Lagos in mid-2014 where we would see basically what would happen if you were to offer to pay people to become software developers or better software developers. And with no website, we ended up getting, through Twitter alone, 700 applicants for four spots. And we thought it was a fluke. Like what’s going on here? So we decided to run it again and this time we added in basic aptitude test and this time around, still no website, looking for 20 people we end up with 2400 applicants.
But what was even more interesting was the testing service we were working with at the time called us a couple of days in and they were like, what are you guys doing? It was almost frantic, and we said, what are you talking about? And they said, we’ve 1) Never had so many people apply for a job and 2) of those applicants 42 are in the top 2% aptitude and problem-solving of any people on the planet.
Vivek: Wow.
Jeremy: And we looked at each other and realized that this was not a fluke. And this is something we had to do that had to exist in the world. And, you know, from there, now five years in we’ve had over 150,000 applicants to Andela. We’ve accepted 0.7% and we are Africa’s most elite engineering organization. We’re a company now of a little over 1400 developers spread across six countries in Africa. And, you know, continuing to grow very rapidly.
Vivek: Yeah. Wow, that’s a fascinating story. It’s amazing. I’ve always been amazed by how really great companies start off almost as I was an experiment. ‘Oh, we’ll just try and see where it goes.’ And then it sort of like balloons because you get like so much demand inbound and then you say, ‘oh, wow, like we’ve actually created something real.’ This is very, very similar. For us, like, you know, Hari and I, my co-founder, we went to the same college and this was really like a side project that we got started. Until like companies started to use us a lot more and we decided, ‘okay, you know what we really need to quit and do this full time.’ It’s pretty amazing to see your story arc very, very similar to that.
Jeremy: Yeah, that’s the birth of many great things and I think it actually speaks to one of the key advantages that startups have is that you’re just able to test things in ways that you wouldn’t be able to in a much larger, more established organization or they’re just really structurally difficult. The bigger, you get the harder it is to experiment. And startups are able to just in some ways just kind of see how the world responds to things in a much more agile way. And as a result of it. You got the chance to test things that would otherwise sound crazy to a number of smart people and sometimes you’re right.
Hiring candidates with non-traditional backgrounds
Vivek: Yeah, absolutely. So as you continue to grow from the early hundreds of applicants to now you have like over 1400 people in staff, so you have like hundreds of thousands of applicants. I’m assuming a lot or maybe a majority of them don’t have prior computer science background or coding skills.
Jeremy: It’s a bit more varied at this point. We don’t require prior experience. We don’t require a college or high school degree, but there’s a massive amount of self-selection. I think one of the things that the traditional tech world gets, you know, fundamentally wrong, and there are a few of these, but about Africa, is just how much existing talent there already is. In every major urban area across the continent, you have significant and growing tech ecosystems that are yes young, but they feel like New York in 2008, you know it is still very exciting and bustling and they’re cool things happening. And I feel like the world generally misses that and so there is a lot of talent that doesn’t have a way to show the world just how good they are because the world doesn’t have a good way of trying to look for and evaluate them.
And so a lot of what Andela ends up doing in terms of breaking down the barriers between like opportunity and brilliance, like the barriers that separate them, is relatively straightforward. Things like how do you attract incredible talent, how do you then assess really effectively? How do you then help nurture that talent and then provide the kind of support and oversight that a global team needs to be able to scale effectively? Because building and managing distributed teams is hard. You know, we make those things a lot simpler for both companies and engineers.
So when a company is thinking about how am I going to scale? How am I going to advance this product roadmap anywhere near as fast as I need to? We just become the most effective option possible.
Vivek: Yeah no absolutely, totally get it. But when you’re getting these applicants who obviously don’t fit into the traditional mold of ‘I did my computer science degree’ or ‘I went and worked at this company’ you’re really betting a lot on their potential to grow and develop and learn which if you get it right. It’s going to be like a giant upside like a 10X upside. But if you factor it wrong, it’s going to be a disaster or a pretty bad outcome. What have you learned over the course of like five or six years on the ability to spot potential when you don’t have the relevant experience and candidates?
Jeremy: You know, it’s been an evolutionary journey. We obviously knew early on things like aptitude and problem-solving ability were going to be important. We knew that communication would matter but we certainly didn’t know how much, but we also learned a lot along the way.
An example of something that I was really surprised by, although in retrospect makes complete sense, was looking early on, at how much people demonstrated that they just found the notion of solving problems with technology interesting. And so, not how good they were at it at the moment, but how interesting they found it. And what we realized was that over even a relatively short span of time the people that found coding interesting, even if they weren’t quite as like as high like output from an aptitude standpoint, they were, long term, going to end up being the best developers. So does intelligence matter? Of course. But whether you are in the top 2% or top 1% turned out to matter a lot less. Whether you are really interested in software development and solving problems with technology–that matters immensely. And it’s going to be wildly deterministic in terms of your overall likelihood of success.
Another thing that we found that I also was really surprised by but in retrospect makes total sense: someone’s ability to take feedback directly corresponds to their ability to learn. So if you immediately respond to feedback with how are you going to argue against it then it, it almost doesn’t matter how smart you are. And in fact, there can be a little bit of an inverse correlation, because the smarter you are, the more you’re able to argue against feedback effectively as opposed to listening to it. And the people that actually grow the fastest are the ones that get critical feedback and think to themselves, how could that possibly be true and if it is true what can I learn from it? If you meet people like that, they are the ones that are going to be able to leverage a learning experience to dramatically alter their career path and life and it’s just awesome to see.
Vivek: Yeah that is that is definitely pretty, pretty cool. So do you do incorporate this part of this point of understanding if they will respond to feedback and others in your interview process? Have you added this to the rubric?
Jeremy: We do in all sorts of ways. We now have over 60 different assessment criteria that we’re looking at, and measuring each in multiple ways. And so it started off as a very simple, simple process and has become significantly more complex over time.
Overcoming inherent bias
Vivek: Interesting. Interesting. Now let’s maybe, maybe we can move to the other side, which is the companies. Can you talk to me about your early your first few customers? How did you convince them to say, hey, we have an amazing pool of software developers who we have vetted, we have screened. They’re working and now you need to outsource your projects, there’s going to be NDA that’s going to be like, you know, all sorts of things—confidential code. Like this probably, like a lot of challenges you might have faced until you crossed the threshold of ‘Okay enough companies use us and its pretty common.’ Can you talk to me about your early days on the company side? How did you convince them?
Jeremy: Yeah. The short version is it was really tough. In the early days. It wasn’t just convincing them that the system or concept could work. We also had to help them overcome a huge degree of bias that they didn’t even realize they had. And so, you know, for most people, I believe are inherently good. Most people want to be a force for positivity and positive change in the world. The challenge is, most people also believe what they’ve seen. Like they believe something after they’ve seen it. And that means that if they haven’t worked with a developer from Kenya before, they don’t yet innately know that there are amazing developers in Kenya and that is not them trying to be offensive, it’s not them trying to willfully deny it–it’s just a human challenge.
And so I’ve talked to senior leaders at Microsoft, Facebook, Google, who have brought their leadership teams over to Africa, and in the past couple years, almost all of them have for the first time, brought the leadership teams there because it’s such a rapidly growing important continent for the world. Each of them has said that after that trip the entire conversation changed. And it wasn’t that the facts on the ground changed, it wasn’t the data change–it was that we are all people and we believe what we’ve seen. And once they saw just how exciting those ecosystems were it changed their perception and therefore what they were able to imagine about reality.
We’ve had to do that a lot over the years–to change that perception and in the beginning, the first companies were the ones that were generally the smallest and hungriest. So they were the ones who understood the inherent argument that brilliance evenly distributed and they were looking for the best talent they could possibly find. They recognize that that top 2% talent in the US locally was getting like basically just gobbled up by the Facebook’s and Google’s of the world and that they weren’t even close to being able to compete from a comp standpoint. They just felt trapped. And so they were the first ones to say, “well, this sounds crazy, but you know, I’ll give it a try and see what happens. My friend is working with you and he says they’re really effective to get experience. So I trust him. And let’s see what happens.” And so, you saw these pockets forming around the country of CTOs who started working with us and then telling their friends, hey, actually, it’s kind of great. I was skeptical at first, but this has been a way better experience than I anticipated and I think that’s also, you know, to your comment earlier about like the common beginning of great things like I think this is another one of those sort of universal truths. Is that a few people realize something, and they start to share it, and all of a sudden it’s spread socially because it’s a competitive advantage for those folks and they’re able to, you know, they’re able to leverage it in order to move faster.
Vivek: Yeah no absolutely no it is. It’s amazing. I got reminded of a quote from the CTO of Looker, who was actually one of our customers, that once they started using HackerRank “I finally see the candidate I wanted to see, you know, it’s similar. Which is like once you experience something which you had like a prior bias to and you see that working. I mean, like, at some level, humans are all biased. Your bias shifts towards “Oh, this actually is going to work. This is going to work pretty well.”
Jeremy: Completely. Our brains play tricks on us. So it’s why it’s so important to take a step back and really again take feedback and, you know, ask hard questions because we see the world through the lens of our own experiences and we only experience a very small chunk through the actual world.
The future of work: Industry shift to distributed teams
Vivek: Yeah no absolutely in terms of, you know, we talked a little bit about how you started off as like a company started building teams, everyone should be located at the same place, and now people started to move towards, ‘Okay I’m fine with the concept of a remote engineering team’ and now you’re starting to see it distributed, which is fine, regardless of where you actually work–which place, which country.
The most extreme example that I’ve come across on this aspect was GitLab. I was talking to Sid, the founder and CEO of GitLab, they are about 400 to 500 people now and he’s planning to grow to about 1000 and they don’t have an office. You know, it’s completely distributed, and everything is on Zoom. I don’t know if you’ve seen the GitLab ‘About Us’ page like everything is so well documented. And it’s amazing to see how you can actually build a 1000-person company without an office. Do you think this is kind of an extreme version? Or do you think like this is what it will be in the future where like you know you fast forward, maybe like 10-15 years if you don’t have a remote or distributed presence, you’re going to be odd, as opposed to right now, if you have like a remote or distributed team, you’re probably the minority. What do you think is going to happen?
Jeremy: Yeah, I think this is one of those ones that feels really complex until you look at the trends and understand what’s driving them and then all of a sudden it becomes super clear.
We’ve gone through a couple of bumps, I’d say, in the move towards comfort with distributed, and you know there have been some high-profile companies like Yahoo and IBM, that sort of tested it and pulled back. I see that in the same way that I see sort of the adoption of almost anything that is new, and retrospect looks obvious. We are moving towards a world where We’re going to be able to have this conversation, but have it feel through you know either VR or AR like we’re in the same room and that world is not 50 years away. It’s not even 20 years away, it’s probably 10. At that point, the notion of everyone having to physically transport themselves to the same location becomes sort of insane.
And so I don’t think it’s going to like, over the next 10 years happen really rapidly. I think it’s gonna look an awful lot like ‘Well how’d you go broke. Mr. Gatsby?’ ‘Well, very slowly and then very quickly.’ Like it’s going to go very slowly, every year it’ll be a couple of percent more companies that are moving towards distributed, and then it’s going to just completely like explode and become just obvious and it’s going to seem as if the world has just fundamentally changed when in actuality, it was a slow move over time. With companies like, you know, yes. GitLab, but also Envision also Automattic, the parent company of WordPress, that are, four or five 600 person companies, bigger Automattic now, that are entirely distributed and have been for a number of years, and people will look back and they won’t seem so crazy anymore. They will seem like they were just ahead of the time ahead of the curve. And it’s, it’s hard to catch up to that if you can make it work it’s a superpower.
Vivek: Yeah, no, I, I think it’s a question of timeline versus if it’s going to happen.
Jeremy: Exactly right.
Vivek: Yeah, in order for that to happen. Do you think it’s a mindset change that is the biggest blocker, or do you think there are also infrastructure elements that need to get better? For example, right now, if you have like Slack Zoom. Is that good enough or like you do you think know there needs to be like more technological advancements that make you feel like this person is sitting next to you or working what which of these two things is probably a bigger challenge to overcome that?
Jeremy: I think it’s going to be a combination. But I think the slow growth that we’re going to continue seeing is going to be driven by things like slack and zoom.
But also, honestly, I think HackerRank plays a material role there, as well, as the more you can quantify what someone’s able to do the less you need to look over their shoulder. And so, you know, the better JIRA gets, the less you care about where your engineers are located. And so I think you’re going to see slow, steady growth until there’s an infrastructural shift where you have something that is like the Zoom equivalent for the, you know, for video conferencing, but instead for either VR or AR that just makes it feel easy that makes it feel obvious that makes it feel like of course, it would make sense for companies to do this. And I think the first early movers are going to be companies that have offices all over the place. And they’re going to become much more productive and then from there, it’s going to move into well ‘Why do we need all these offices in the first place?’ And you’re going to see large companies doing you know, doing the kinds of analyses that Stripe did when they announced that their newest engineering hub was going to be distributed where they say actually not only if we do this could we have access to a broader talent pool. But we also can, you know, cut 10% of our costs out of the equation and pay people more so now we can really, really attract better talent and as a result of that have a competitive advantage when it comes to scaling.
Vivek: Yeah, no, that was a that was like a very good move by Stripe. They’ve been continuing to open offices across the boat and you know stripe is a very engineering friendly company to see them endorse. This is definitely good.
Building trust-based human relationships
What about the other side of distributed engineering teams, which is, you know, I’ve found that as much as it’s more powerful–you have access to talent across the world. You know, one of the core competencies of the ability to work together is bonding that you have. Working with teams does require like in-person.
So we have an office in Bangalore, in London, for example, and one of the things that I recommend every exec in our office to do is to go and meet them in person, not Zoom. It’s amazing to see how relationships completely transform when you meet somebody in person versus talking over Zoom. What’s your thought on that? Like if you get fully distributed, do you think you would miss out on that or what are the cons?
Jeremy: Yeah, it’s a great question like, first and foremost, like, you’re right. There are material cons. I don’t mean to paint a totally rosy picture here. I think there are there are real downsides to not bringing people together in physical proximity on a regular basis. And I think you’re definitely speaking to one of them. And that’s the speed at which you build trust is actually much slower if you don’t meet in person. Because if you meet in person, the way that we’re hardwired as people is to like start, and again, this gets into the challenge of bias, but it’s to start making assumptions about like I can relate to this person. I can feel close to them. I can trust them. And the more that you can trust someone, the more you can actually work effectively like trust is actually a meaningful variable in the productivity of teams–super meaningful.
And once you realize that the answer, I think, though, is not that you, therefore, need to have everyone coming into the same office every day, but rather what companies like Automattic and I’m guessing GitLab does as well, but I don’t know for sure, is you bring people together physically every once in a while.
And so for most of the companies that have done distributed really well. They will have at least an annual, but oftentimes more frequent, get together in person. And they say, look, we’re saving all this money on offices, we’re going to spend money to get together and go on a really interesting off-site. And you get everyone together and really bond and the ones that do it really well yes we’ll focus a bit on getting work done, but they mostly focus on building trust-based human relationships. And the more that you can do that, the more you can then not have to come into the office and still get great work done because you’ve built that foundation of trust.
And so when we start working with a new partner company, we work with a couple hundred companies around the world. But, you know, the majority 80 to 90% are in the US. And that means that, and most of them are, as you can imagine, not fully distributed. And so when we start working with a new partner company we actually subsidize the cost of the Andela engineers traveling to the headquarters of the company, whether that’s you know New York or San Francisco or Austin or Salt Lake or, you know, London or Cape Town.
And we do that because that in-person time for a couple of weeks during onboarding is so useful for building trust and rapport and it meaningfully improves the overall effectiveness of the team.
Vivek: Yeah. No, that’s so you did it. You do this across all companies whenever they have like projects?
Jeremy: Yeah. That’s exactly right. I mean, we don’t really do. It’s less project-based when our engineers work with companies, it’s typically you know you’re typically working with them for a year or two. And so it’s a longer-term kind of relationship and therefore that trust is even more important.
Got it. No, I fully agree that’s, I mean like at the end of the day, every decision is going to be a trade-off on what you would like to do more than others. Nothing is like perfect hundred percent, but it’s good to see how you’re actually building trust by bringing people on site. There’s another company in my YC batch. It’s called Zapier. I think you might have probably heard about them.
Jeremy: I know them well.
Vivek: Yeah, so they’re also fully distributed and we stopped and he does an offsite definitely once a year, maybe like twice a year, he brings the entire company together and I was asking how do you fund this, like they’re the bootstrapped, they’re growing. He had a really funny answer. So he said, like whatever you’re spending on office space. I’m just putting it towards the off-site.
So yeah, definitely the trust factor or like building a team component is super important.
What’s next for Andela?
Vivek: All right, so we’re probably towards the end of the podcast. You know, you’ve been continuing to grow pretty well. You’ve also raised funding from pretty strong investors. What is next for Andela, like when you look maybe over the next two or three years. I know everybody asks what do you want to do in like 10 years and it’s super, super hard for me to answer that. But maybe like in the next two or three years?
Jeremy: That’s a wise question. I think that you know we overestimate what we can do in a year, as people, but we dramatically underestimate what’s possible and 10 it’s just so difficult to predict what’s going to happen with compounding interest that far out. But two or three is reasonable. You know, the core problem that Andela is trying to solve is the fact that it is fundamentally true that brilliance is evenly distributed around the world and opportunity isn’t. And so that isn’t just a challenge for people looking for opportunity. It’s also a challenge for companies looking to hire. So I see us as being very much, sort of, fellow travelers actually with HackerRank. Working to break down the barriers that prevent brilliance and opportunity from finding one another.
And real impact requires real scale. So I anticipate that you’re going to see us over the next couple years, not just continuing to grow, you know, the sort of overall like at scale and impact of like how we are operating and where we’re operating, but also to we’re more and more building out the internal tools and really data behind how do you make distributed teams work effectively. And a lot of folks when they’re coming to us they’re yes, looking for talent and yes, understand why we are in somewhat of a unique position in terms of what we’re able to offer, but they’re also hearing oh, and it just seems to work when you guys do it. people working with you just feel like you know what this is just this is the thing I can, I can manage.
In some ways similar to, you know, you hear people talking about Zoom and they say, it just seems to work. And so we want to lean into that as much as we possibly can, you know, we believe that’s what our customers, our partners are asking for and we believe that’s the best way for us to break down the barriers that prevent brilliance and opportunity from connecting.
Vivek: Yeah absolutely. Maybe, maybe you should use Andela’s developers to build a tech infrastructure that you’re dreaming of.
Yeah. Awesome. Well, thank you, Jeremy, for taking time and congrats on all the success super inspiring mission. It’s very inspiring to see what you’ve done. What you’ve grown and you’re completely changing the landscape of how companies think about talent building teams.
So congratulations on all your growth. And thank you for joining our podcast.
Jeremy: Of course it was such a pleasure Vivek. It’s great to great to catch up as always and look forward to talking soon.
How Atlassian Approaches Diversity and Inclusion with Balance & Belonging
Jul 02, 2019
This is the 13th episode of HackerRank Radio, featuring HackerRank’s VP of Customer Success, Gaurav Verma, and his interviews with customers from the HackerRank main () meetup series.
According to the 2018 Atlassian State of Diversity & Inclusion in U.S. Tech survey, 80% of respondents believe that diversity and inclusion are important. But over the years, there has been a 50% decrease in individual participation.
Aubrey Blanche, the Global Head of Diversity and Belonging at Atlassian, believes the decline of D&I efforts and enthusiasm is because companies are taking the wrong approach.
In this interview, Aubrey and Gaurav Verma, HackerRank’s VP of Customer Success, discusses why diversity and inclusion should be replaced with balance and belonging. Aubrey also shares concrete and practical ways companies can create balanced teams, establish an atmosphere where everyone feels empowered, and eliminate bias from the hiring process. Listen to the episode below, or scroll below to skim the transcript.
Gaurav Varma: Welcome to HackerRank Radio. I'm Gaurav Varma, SVP of Customer Success here at HackerRank. This week we're bringing you a conversation I had with Aubrey Blanche global head of diversity and belonging at Atlassian at our flagship event HackerRank Main Palo Alto. Aubrey and I talked about why organizations should stop thinking about diversity and inclusion, and prioritize balance and belonging. Aubrey also shares why resumés, degree requirements and the concept of a cultural fit, promote biased hiring.
Aubrey, thank you for being here. It would be great if you could introduce yourself. Tell us a little bit more about you.
Aubrey Blanche: I'm Aubrey. I'm so excited to be here. So I'm the global head of diversity and belonging at Atlassian and I'm gonna ask all of you for a favor actually, we'll talk about it, but I'm trying to change my job title and I need your help. My job is really to help Atlassian hire the right people and then make sure that they can thrive once they're there. The other way I describe my job is crushing the hierarchy with capitalism. I'm really lucky. I'm so excited to be here in particular because actually HackerRank was what got me into diversity and inclusion work.
Gaurav: That’s awesome.
Aubrey: So back when I was at Palantir, I basically convinced someone to let me run an experiment on engineering recruiting to be like, “It's busted, you're not hiring enough women” and they were like, “No, no, no we're a meritocracy,” and I did an experiment and I was like, “Haha, No.” And so, we ended up overhauling the process and HankerRank actually was really key to work with us to make it so that we removed a lot of bias from their process.
Gaurav: That's great. And so in every single one of us, there's some form of unconscious bias. I think I'm notorious. I do it all the time.
Aubrey: It's really like how many.
Gaurav: How many, right?
Aubrey: And to what degree, right?
Eliminating Bias From The Hiring Process
Gaurav: It starts from the resumé. I see the email address in our ATS when it gets sent to me and I'm trying to decipher like, my own email address. You can infer a fair bit from it, right? Male, Indian origin, I don't think you'll get the hair part of it, but everything else you'll get. And then one tends to then let me go google this individual. You go on LinkedIn there's a picture. It creeps in and regardless of how hard you try to make a conscious effort not to, it does. And how do you protect yourself against that?
Aubrey: I think what you do is you have to do really smart systems design. So the fact is, you know that you could throw resumés on the floor and pick up half of them and interview those candidates and you'd have the same success in job performance as if you scanned resumés. Literally, the number one thing about resumés that predicts whether someone gets an onsite is typos on it, so like, resumés are useless. I genuinely think so.
But I think that's it, you have to design your processes to limit the potential expression of your bias. Because we're human, our brains physiologically can't not be biased. We've learned a lot of rules about people. So if you can get rid of resumés awesome, if you can use tools, genuinely, like HackerRank, to do any kind of skills testing as opposed to deciding something else. Or if you're not at the point where your organization can get rid of resumés, we have to do a little more stakeholder management, thinking about ways to read them that are more structured. Don't look for experience working at Google. Why? Because they've had historically highly discriminatory hiring processes. Right? I don't want to call out just Google, like, the entire industry. Google just has a lot of employees.
And so, if you're looking for signs that someone has had a lot of money, go hire someone from Stanford, go hire someone who went to Google or Facebook. I did go to Stanford. There are a lot of smart people there but there are a lot of people that are just rich there too. And I think that's important to know, so when you're looking at a resumé especially for a university student, does that student have a 2.8 GPA but did they work an extra job because they had to support family at home? Because students of color are significantly more likely to have to be like that. And when you have a 3.7 resumé cut off, all you're actually selecting for is family wealth and whiteness, for the most part, because of the way those things move together in the society.
So I think we can make those intentional choices and I think it goes back to a larger point which is, I'm not actually a recruiter, so to do this work well you have to have an incredible partnership with the recruiting team. I'm really lucky that we have an awesome one, but we've worked really hard. Where possible, get rid of degree requirements. Why? Because you can code without one or you can code with a degree in philosophy. I code and I have a degree in journalism. Make sure you're taking to account people's whole life story. I'd rather hire the student with a 2.5 GPA who had two extra jobs, than an internship at a fancy company. Why? They're gonna get shit done. They know how to solve problems.So that's it. I think you can also make sure that you're killing the concept of culture fit. All that is, is an intractable morass of unconscious bias.
At Atlassian, we talk about values alignment and that isn't just some like rebranding term. We actually went out, did focus groups with our employees, identified the specific behaviors that are associated with each of those values, and we now run a highly structured interview that looks for those types of behaviors in their past work. You have to have been at Atlassian for at least a year, especially nominated and go through extra training to even do that interview. If there's one thing you can do, because all culture fit means is “you're just like me”, and when we have an industry that's imbalanced in the way that it is, that just repeats those things. And so, it's not that interviewing for people who are going to work well in your environment is bad. That's actually really great, but you want to be very intentional and specific about what things you're looking for versus what you're not. Otherwise, it just becomes who road crew with the interview at Princeton.
That little quip actually comes from a study that showed that in finance if you play the same sport as your interviewer, you are four times more likely to get another interview. And something like crew or whatever, there's very strong socioeconomic components to what types of sports you tend to play.
Gaurav: Yes. I think mine does bring some diversity to the whole panel.
Aubrey: There’s a question over here. Sorry, there are no rules, right?
Gaurav: Yes. Absolutely.
Participant: Would you be discriminating against a hardworking child from a rich family, who happens to be in Stanford?
Aubrey: No, not if you have a conscious set of criteria. But why are you going to give the Stanford kid an extra look over a kid who went to Arizona State? Yes, so I’m not saying don't hire people from Stanford. I'm just saying don't give them an extra advantage that the people from other schools don't get.
Participant: You can have a very hardworking child who just happens to be at Stanford.
Aubrey: Absolutely. And so, if you're doing the skills test for that then they would get the same assessment. But I know at companies I've seen where they literally have separate interview processes. If you went to a core school like Stanford versus MIT and so that's what I'm saying is, the fact is, the students at Stanford regardless of their wealth or how hard they've worked, I can tell you I do not come from a family with 10 million dollars in the bank, and so yes what I'm saying is don't give the kids from Stanford extra benefits over students who went to other schools because that's actually what the practice is in the current state – not at every company and not in the same way. But yes, the idea is you should have the same skills tests for everyone and then you hire whoever meets the bar. Fairness. It is also legal. It's like legal, ethical and right. But what we know is the processes as they're generally practiced now are highly inequitable and so what we're doing is correcting for that.
Balance & Belonging Over Diversity and Inclusion
Gaurav: You mentioned this earlier about title, you don't like your title. So you're not a big fan of the term diversity. Tell us a little bit more about that.
Aubrey: Not a fan is like a really light way to describe that. I didn't have any benchmarking data on attitudes and behaviors towards diversity and inclusion in tech. I was like, “Are people excited or are they engaged?”, no one had this data. So I went out and collected it in an international survey and our team we actually decided to release all that. Because I thought if it is useful to us, it'll be useful to you. And what we found last year, I put some secret trick questions in the survey, in one of the questions I asked last year was, “Which of these groups are part of diversity?” Here's the fun part, I put all the groups: Men, women, non-binary people, white people, black people, Asian people – whatever. So, the word “diversity” is overwhelmingly associated only with white women and black Americans. Even in Australia, Australians were more likely to say that African-Americans were diverse than Indigenous Australians. That's weird. And so, my argument is that the word diversity is actually highly problematic, not aligned to the goals that we have and getting in the way of creating equity for white women and black Americans in the first place.
So what we should be talking about is building balanced teams. First of all, who is going to die on the hill of building an imbalanced team, like, absolutely none of your engineer managers right? It doesn’t make sense. But the reason it also helps is because it allows a greater space for other types of identities. So we can hold, for example, intersectionality – inside balance, so we can actually think about women of color who face significantly more barriers to advancement than white women do. We can also have space for that straight white cisgender man who grew up in a trailer park. Who historically has heard that, “Oh, he's just privileged,” when in fact a lot of the barriers that he faced are similar to some of his peers who come from other low-income backgrounds, who may or may not be from other marginalized groups. And the third reason I think it's actually really helpful is because I have found that internally at Atlassian, switching to this language has helped us have more direct conversations about race than I've ever seen. So, the fact is like white fragility is real and the word diversity is so tied up with blackness specifically that before you even start to have a productive conversation, people emotionally shut down.
So balance, the word, literally just people have less hangups about it, and so we're able to actually get into the question of, “Oh, hey we've all seen the hiring manager who's like, ‘I really care about diversity’” and you're like, “You have hired six white women which is great, but what about everybody else?” Not to nag on white women. We note that they also deserve more opportunity than they're often given.
And so, I think then you go to the question of, okay great, your team is balanced in terms of gender. In what other ways do we need to work on balance? And then we get to a constructive conversation about this category and what are we going to do about it. And like I said, for us, it has been completely transformative because people start bringing in aspects of their identity –
Autism, parenting status, military status, struggling with addiction and mental illness, death of a spouse – these are things that people talk about and write about internally at work and even at our customer summits, and I think it's because we've created space for them. So, my boss won't let me change my job title to head of balance and belonging until I make balance happen like “fetch,” so I'm hoping that you will help me.
I only ever use the word diverse when it is very clear that I am also including people from majority groups.
White people are a part of diversity, otherwise we're contributing to the idea of people of color and women and anyone who doesn't fit that majority being the other and we shouldn't accept that there is one default for people. We're all the default.
So that's it, I'm a secret Latina. Right. You didn't know because of my name. I'm like the people who the LinkedIn parses miss. But I think that's it, is when you say balance, it goes back to this. We all have different perspectives that we bring to the team, but those are fundamentally informed by our identities and life experiences. You can't just say “diversity of thought” that comes from diversity of life experience.
It's the reason that algorithms nowadays don't work for people of color because the people in the room building don't even know to ask the questions, not because they're evil, not because they're stupid but because their life experience has not given them that knowledge. Giving them an hour training is not going to solve it. You just need to teach them how to work well with people from different backgrounds.
How To Build Balanced Teams
Gaurav: That's super profound. You've done a lot of research and one of the reports that you had state of diversity which we’ll now call “State of balance” report.
Aubrey: I’m trying, yes. This year, we’re trying to make it happen.
Gaurav: That’s awesome. You found that 80 percent of people think D&I, which we're now going to call balance is important, but that doesn't necessarily translate to action. This is a big one. Especially here in the Valley, the report found that the number of formal D&I has shrunk since 2017. Why do you think that is and how do we solve for it?
Aubrey: Yes. So, we sort of quantified this existence of diversity fatigue and I think that's what's driving it.
So we found that there was a 10 percentage point drop in companies who had a formal D&I program, but like a 10% jump in companies that said they cared about D&I. And the thing I offer to you is, I would prefer if your company is not putting actual time and money into it, just say that you don't care. No one's going to judge that, but what you're doing is you're creating a space where candidates can't find a place where they're gonna be safe and where they're going to thrive. And so, I think that's what's happening is at the corporate level— it's 2013 was sort of when this next big wave, Tracey Chow asked, “Where are the numbers ?”, and you've seen a lot of companies pour a lot of money into branding and into PR but not into the structural change. And so, I think recruiters are really the front line of creating that change, which is, why are we doing a team fit interview? Why aren't we looking for people who add something to the team? Why aren't we selecting for skills instead of pedigree? Those little choices create more fairness because I think also the way that— and I'll put this on me and my peers, the way that we've talked about this work has been really wrong.
So, we talk about it a lot at the company level, it's like, “Oh, we have 30% this” but your average middle manager doesn't know what the hell they can do to impact that number, right? Or you're like, “Empower women” and they're like, “Do I go buy Dove deodorant? What am I supposed to be doing?” And I joke but it's true. Instead of being like, “You should value diversity!”, I say, “Well could you institute a no-interruptions rule in your meeting?”, why? Well, because it turns out that women, people of color and people from East Asian countries are significantly more likely to be interrupted while speaking in meetings. So, you're gonna get the good ideas on the table. Now, does that feel like a diversity thing? No. It’s also just a good way to have an effective meeting. But I think that's it.
You have to get really tactical, because folks don't know what to do to impact the whole culture which is because they can't. They can impact the culture of their teams. And we need to talk about balance, building balanced teams. Because your average hiring manager, knows how to do something about their team.
And if every person made their team a little bit more balanced, a little bit more having a culture of belonging, the entire company culture would shift, but it's not because you're asking people to act there. And I think that's it, give people tactics. Things like, I'm doing a talk with our security team today who's growing a lot and they said, “Well what can we do to make sure we go in a more balanced way?” And they go “Here is all of the things our recruiting team is doing from a structural and a process level.” I was like, “What about referrals?” And they’re like, “Well, our referrals aren't very balanced.” And I was like, “Well, so you five…” and it happened to be a group of men who were sitting to my right, and I said, “How many of you have been to women in cybersecurity meet ups?” And they were like, “Uh, none of us.” I was like, “Well, why not? You want to go hire women. The women in cybersecurity go to those meet-ups, go meet them and then give them jobs.” And they were like, “Oh yeah, we can totally do that.” And again, it was not a tactic that ever occurred.
So I would say there is that is, give people one job and something to do because often they will try to do your job. They're like, “Can I help you source?” And you're like, “No, we have awesome sourcers. What I need is for you to go build a reputation as someone who's going to support people.” Because the candidate's gonna come in and say, “Well, what have you done to help build a team where I'm going to thrive.” And the hiring manager is going to be like, “Oh, we have a diversity program.” No one's going to believe that hiring manager.
To me, a great answer for a candidate and I can tell you because we've got to hire this one, I talked to her after she joined and she was like, “You know, the hiring manager on the team, I'm the first woman on my team and I talked to the hiring manager about it and I said, ‘What are you doing to make the team more inclusive?” And he said, ‘Well, I've been working with recruiting to find more women, but we've also been trying to have no interruptions rules in the meetings, and this is something I feel really awkward talking about but I'm trying to be more proactive.’”
That was his long answer. He did this one tiny thing and “I feel awkward” and she was like, “I'll go work for this person.”
Gaurav: Wow.
Aubrey: So I think that's what it is, is don't make these big pledges about, like, “We care about diversity” everyone says that. It's about how can you do a small thing that's totally within your power and you should also feel awesome about yourself about that creates, a little bit of change. And then when that becomes your habit and you don't even notice, pick a new thing and do that thing.
Gaurav: That's great. I always felt like—and Vivek and I have talked about this and what I started to is, there’s only one question I ask them and it’s, “Core values, do you have posters about it everywhere” and he goes, “No”, and I go, “I'm gonna come join this company.” Because we tend to make a lot of noise about these things, put posters up and it's on our website and everything, but do we really mean it? Do we really embody these? It is really important.
Aubrey: Yes. I think that's what it's about, it’s like break it down into chunks. You don't have to move the whole mountain. None of us can do it alone. This is a gnarly structural issue that we all put little bits into creating it. And so, it's going to take us all just slowly moving and undoing that. But to me that's more optimistic. You absolutely can do something.
Gaurav: That's great. This is really great.
Aubrey: We're just more uncomfortable when we're around people not like ourselves and so I think there might be some of that driving it too.
Gaurav: That’s awesome. So Aubrey I can have you here all day, thank you so much.
Aubrey: Thank you for having me.
Gaurav: Thank you for tuning in. If you have any questions, tweet us at HackerRank.
Our TOPPODCAST Picks 
Stay Connected
![Banner reading "[Data Sheet] Hackerrank Overview"](https://blog.hackerrank.com/wp-content/uploads/INBLOG_Zeta_Blog.png)
To respond to COVID-19, ServiceNow had to take their entire hiring process online—which meant a transition to 100% remote interviews. In this interview, NancyDeLeon, Director of Global Talent Acquisition, explains how they managed it. 
Granting employees access to remote interviewing tools wasn’t nearly enough. It necessitated a complete revamp of their hiring approach. “We had the daunting task of training a global engineering team, our recruiting team, and our talent acquisition operations team in a very short period of time,” Nancy recalls. “And we said, you know what? With HackerRank and ServiceNow: challenge accepted. And the results were fantastic.”![Pull quote from Nancy DeLeon from ServiceNow that reads: “I was actually expecting much more of a negative result since there wasn't a human being on the other side. But in fact, strangely enough, most candidates said, ‘I really feel [the assessment] was unbiased.’”](https://blog.hackerrank.com/wp-content/uploads/servicenow_nancydeleon_assessments.png)
It’s those metrics around ![Quote from Nancy DeLeon of ServiceNow that reads: “To be competitive as a company, we're going to have to consider how we’re able to provide those types of options, and how we’re going to be enabled to do both [physical and virtual work].”](https://blog.hackerrank.com/wp-content/uploads/servicenow_nancydeleon_futureofwork.png)
For ServiceNow, she sees the flexibility between physical and remote interviews as a talent branding need, too. “To be competitive as a company, we're going to have to consider how we’re able to provide those types of options, and how we’re going to be enabled to do both.”
Seemingly overnight, remote work—and by extension, virtual interviews—have become the new normal. For those hiring, the transition forced interview processes to adapt to 
To go the extra mile to get candidates comfortable with this new format, the Bloomberg team implemented a few primary changes:
Luckily, their interviewers already had a strong base to build on, since all interviewers at Bloomberg go through a training process before interviewing candidates. “They’re trained on how to assess various competencies, how to ask technical questions, and how to ensure each candidate will be a culture add to Bloomberg,” Kristen explains.![Pull quote from Kristen Arena that reads: "“I think it's that relationship-building piece, and really letting the candidate guide the process [that’s key]: listen to them, listen to what their challenges are from a remote perspective, and be empathetic as well. I think that human aspect is critical.”](https://blog.hackerrank.com/wp-content/uploads/kristen-arena-bloomberg-human-aspect-quote.png)
To Bloomberg’s recruitment team, that means adopting a 
