Talk Title: Lessons from Duolingo: From Startup to IPO

Effective talk: Some information on running a startup

Year of Talk: 2022

Video Link: Lessons from Duolingo: From Startup to IPO Disclaimers:

This can be different for each startup, and it's important to be able to adapt.

The CEO of Duolingo already had previous startups and had experience getting money and running a team.

People:

• Many startups won't have a lot of money for enough devs. So the team did their best to search for the best possible people and keep them on the team.

• Early on, it's extremely useful to hire people with more general skills. Allowing you to more easily fill many gaps just enough to get the job done.

• When the company starts off, it's hard to get someone who can fit into every role. So it can be useful to find people who culturally fit good enough and then improve it as time goes on.

• Hiring people who are diverse from the start helps give opinions on future candidates better and helps fuel creativity.

• Use internships as a hiring funnel. Let them better test out people with less knowledge, and if they meet certain criteria, they can get hired.

• Having interns also gives them a chance to learn how the company works without fully committing to the company full-time.

• When looking to hire someone, if it's planned to take 6 months to hire someone, then determine if it's worth training someone internally during those 6 months instead.

• Keeping good/high-end developers in the company helps reduce hiring costs as they already know how everything works and don't need various months of training to a suitable level.

• If employees stay, it helps cement the culture that the team wants to have.

Business & Strategy:

• An early strategy was to make an app to teach languages. At the time, a lot of people were only learning through classes, so they believed that this was a way to disrupt the market.

• The stages for Duolingo were User Growth, Monetization, Learning Efficacy, and Philanthropy.

• A/B Testing helped better understand what users liked the most. Allowing the team to move quickly.

Step 1 (User Growth):

• The goal at the start was to get as many users as possible instead of worrying about monetization.

• With more users, it made it easier to test as a lot of people could give their feedback on things. And being able to get the best items out of the door super quickly could bring even more users.

• Periodic “All hands on deck” to help with redesigning the app and logo to keep it feeling fresh and new.

Step 2 (Monetization):

• With a lot of users, the team could start monetizing different parts of the app to get more money.

• Started by having ads and then transitioned to the subscription model.

• In-game currencies and In-App Purchases were included in Duolingo.

• Adding new verticals like Duolingo English Test (DET), which works like the TOEFL.

• Made sure to stay aligned with the core mission of the company and worked on making sure the monetization did the same.

• By doing so, it lets the team stay on course and be a mission-driven brand while still making enough money to continue growing.

• At the same time, the team had strong user loyalty, which helped reduce the annoyance people had when adding monetization.

Step 3 (Learning Efficacy):

• When the business was sustainable, it allowed the company to invest in longer-term projects.

• Adding metrics, curriculum overviews, and user retention.

  • Decision-making allowed the team to be accepting of items that might be harder to test at the start.

• Determining what might limit the company from growing in the next 3 years. What the team could do to fix it.

Step 4 (Innovative & Philanthropy):

• Spent time working on in-house tech and tools.

• Includes a new analytics toolkit, speech recognizer, and other items.

• Try to use easily accessible options at the start of the company, and as the teams outgrow the options, then look into making your internal tools.

• Working on new verticals like certificates or teaching new subjects like math.

• It was difficult to stop the team from trying to do everything at once, but they had to force themselves not to risk spending too much money at once.

• They worked on the items with the best impact and return on investment.

Prioritization:

• Make sure that the team understands that each employee is working to improve the experience of just a few hundred to a few thousand users. Make sure to spend your time well.

Few Learnings:

• Take user feedback with a grain of salt. There are a lot of things that could have affected their review.

• Trying to use data to better understand what a larger group of users is feeling.

• Remember that a lot of the angry people will tend to be the most vocal of the user base. Try to find ways to get info on everyone else.

• Try not to over-scale a feature until it's been tested and vetted properly. This could lead to a product that's unfinished or could still be improved on.

• When asked a yes or no question, saying “Not yet” seemed to help the team a lot. 

• Especially as there would be features that the team was working on, but they didn't want to announce them yet.

Organizational Struggles:

Early days:

• The team had a flatter structure with people working on specific areas while others spent time on other stuff.

• Everyone would report to the CEO directly instead of managers.

• By having direct reporting to the CEO directly it helps reduce the amount of office politics that might normally occur.

• Didn't get official managers until 2.5 years into the company's existence.

Role:	First hires:
Design/Engineering	Pre-Launch (2011 - 2013)
Marketing	Early 2013
Product Manager	Early 2014
Engineering Manager	Early 2015
Learning Specialist	Mid 2015
Talent Acquisition/HR	Late 2016
Data Scientist	Mid 2017
Program Manager	Late 2019
Technical Program Manager	Early 2020

Layer 1 (Teams):

• Created teams based on metrics, features, or a problem area.

• Each team had to be able to complete its tasks independently from other teams.

• This helped reduce cross-contamination of priorities when working on tasks.

• Some of the first teams were for broad areas like Monetization or User Growth.

• As time passed and the team grew, they fragmented into smaller teams to better specialize in specific areas.

• A big factor for their growth was finding the best leaders for the new teams.

• They would rather have a senior person lead multiple teams instead of forcing a more junior person to lead a team.

• Having a junior person put into lead roles forces them to spend too much time doing lead work instead of learning and getting better at their main jobs.

Layer 2 (Areas):

• By the time the company was looking into adding a new layer, 33 unique teams were working on different areas.

• Grouping common teams into a common area allowed them to pool certain resources together and optimize for what common things were being worked on.

• Areas like: Growth, Monetization, Platform, Product Quality, etc.

• Some teams don't have good enough ways to combine into larger areas like marketing and finance.

Making & Defining Teams:

• It's important to name teams correctly to help everyone understand what everyone is working on.

• Helps when someone needs to reach out for information on an area, they can more quickly understand which team might best be able to help them.

• The organization usually started being very grandiose, but by the end of it, the teams were streamlined to the minimum viable team/plan.

• Smaller reorganizations every three to six months, larger reorganizations once a year, and adding a new layer every three to four years.

• Make sure to understand that it's nearly impossible to have a perfect organizational structure. It's a dynamic thing that will need changing every so often.

Processes:

OKRs (Objectives and Key Results):

• A set of objectives or key results that the team wants to achieve.

• Helps set goals for the teams and know the direction they want to target.

• Three to four higher-level objectives are evaluated every quarter.

• Once graded, every goal helps tell the teams how well they were able to plan and execute their goals. Not usually how good the team is/was.

• This helps better understand unforeseen problems that could have come up during production.

• Helps find communication difficulties in the team.

OKR Lessons:

• Make sure to keep consistent language when making goals. Helps everyone understand what's expected.

• Set up a system to control scope creep from occurring.

• When collaborating between teams, make sure that goals are verbatim between both as to help improve understanding of expectations.

• The OKRs helped guide teams towards a common goal/mission and kept them on track.

Automated Experiment Reports:

• An early tool the team set up was automated testing, which would run overnight and return a report for the team in the morning.

• This gave quick feedback on tests and their effects.

• It also returns metrics that the team believed were important when trying to understand their users.

• By making it automatic, it allowed more people/teams to access it.

Relatable Growing Pains:

Engineering pains:

• Many times, there were ownership voids or long backlogs of bugs/features that the teams couldn't work on.

• Having a group specifically in charge of fixing bugs that contains multiple senior leaders.

• Setting up an automated bot that scraped for duplicate logs also helped streamline the process and reduce the time spent going over logs that don't need fixing.

• Setting up all-hands-on-deck sessions to help rework legacy code that was causing problems and needed improvements quickly.

Communication Pains:

• When working on a feature, team members would need to find multiple other team members to sign off on their changes.

• Solved by creating a master spreadsheet that contained every part of the project and who needed to be contacted.

• Created a framework for collaboration to improve the initial feature development by including everyone who might need to spend time working on the feature.

• The team created an onboarding set of documentation that would help people more easily join the company and start working quickly.

Emotional Pains:

• Handling Low performers and people leaving.

• The company gave managers handbooks on how to identify low performers and how to best handle them.

• It also gave everyone standards and where to aim for their skills.

• Giving transparency on why projects were closing or people were being moved to other teams/areas.

• Helped calm their nerves and become more understanding.

• Make it clear that if something fails, it won't impact their careers.