Hi! I’m @fortkle, Tech Lead and Engineering Lead for the Avvy division at AnotherBall. Alongside product development, I also conduct first-round interviews for server-side engineers.

Recently, I made a small change to how I ask questions in those interviews. It was not a major redesign of the interview process. I simply split what had previously been a single design question into several stages.

Looking back at why I made this change, I realized that it reflects what we are trying to assess when we talk about an engineer’s “development capabilities.” In this post, I’ll introduce one of the questions we actually use and explain the thinking behind it.

Trying a Bit of Product Development During the Interview

In our first-round interviews for server-side engineers, we set aside time to discuss product development scenarios that could realistically arise in Avvy. I take the role of a Product Manager (PdM), bring up an idea whose details have not yet been worked out, and ask the candidate to think as a server-side engineer at AnotherBall.

During the interview, I tell them:

From here, I’ll speak to you in the role of an Avvy PdM.
Please think as an AnotherBall engineer and ask me anything you need to know.

I answer their questions in the role of the PdM. Based on those answers, the candidate digs deeper, organizes the underlying needs and assumptions, works out what we should build, and then moves on to technical design. For a short time during the interview, we try to work through product development together as we would at AnotherBall.

For example, we might use scenarios like these:

• Show the gift ranking and each user’s current position in real time during an event
  • In Avvy, users can watch a streamer’s live stream and send gifts while the stream is live.
  • Avvy regularly holds events in which users compete in rankings based on points earned through support such as gifts.
• Create an event that only users who meet certain conditions can join
  • For example, the conditions might include “started their first stream during a certain period” or “earned at least a certain number of coins.”
• Build a mechanism that prevents League demotion on days when a streamer cannot go live
  • Avvy has a system called the League, which is similar to a ranking system for streamers. A streamer’s League goes up or down based on their daily streaming results.

We are not asking candidates to guess Avvy’s current implementation or the architecture we use. We want to understand what they ask when given an ambiguous request, how they interpret the PdM’s answers, and how they update their initial ideas through the conversation.

In this post, I’ll use the first scenario—showing a gift ranking and the user’s current position in real time during an event—as an example.

“When a Gift Is Sent, I Want the Ranking to Update in Real Time”

For this example, assume that users compete in a ranking based on gifts sent during the event. In the role of the PdM, I make the following request:

When a gift is sent, I want the ranking to update in real time.

The detailed specifications have not been decided yet. If you were a server-side engineer at AnotherBall, how would you proceed?

We Were Trying to Assess Too Many Things with One Question

Previously, after presenting this request, I would simply ask, “How would you proceed?” or “How would you design it?” Actual product development can also begin with an unstructured request raised in a casual conversation, so at first glance, this seemed like a realistic question.

However, with the previous approach, we had not clearly separated the information that the interviewer should provide upfront from the information we expected the candidate to uncover through questions. We were also trying to assess several things from a single response: the ability to understand and organize the underlying need, technical design ability, and the ability to consider failure scenarios.

As a result, when something was missing from a candidate’s answer, it was difficult to tell whether they had failed to ask for necessary context, whether the interviewer had not provided enough information, or whether there was an issue with the technical design itself.

Without separating these concerns, we were effectively asking candidates to solve an under-specified problem however they liked.

For example, suppose someone hears the request and immediately starts discussing a design using Redis or WebSockets. Does that mean they are not good at clarifying the underlying need? Or did they simply assume that the requirements had already been decided because the interviewer had presented it as a design question?

On the other hand, asking a large number of detailed questions is not necessarily better. If the PdM’s answers do not change the candidate’s next questions or their design, that is also different from the kind of conversation we have in actual product development.

Looking at the question again, we were trying to assess three broad capabilities:

1. Can the candidate identify the underlying need and the necessary conditions from an ambiguous request?
2. Can they produce a technical design based on clarified conditions?
3. Can they design for recovery when failures such as missing or duplicate processing occur?

We therefore decided that it would be better to assess these in separate stages rather than trying to see everything through one large question.

So We Split the Question into Three Stages

We now use the same scenario but divide the discussion into three main stages.

1. What Would You Ask the PdM First?

We do not immediately ask for a technical design.

The PdM says, “When a gift is sent, I want the ranking to update in real time.”
What would you ask the PdM first? Please feel free to ask me anything while I act as the PdM.

At this stage, we look at whether the candidate tries to turn the still-ambiguous phrase “real-time ranking” into concrete conditions needed for the design. For example, the conversation might touch on questions like these:

• Why do we want the ranking to update in real time?
• Who should see it, and on which screen?
• How much delay would still count as “real time”?
• Are temporary inconsistencies in the displayed ranking acceptable?
• Does the final ranking need the same level of immediacy?

However, we do not grade candidates using a checklist of whether they asked every one of these questions. We also look at whether they can dig deeper based on the PdM’s answers and update the hypothesis they initially had.

2. We Give Everyone the Same Conditions and Ask Them to Design the System

After discussing the underlying need and assumptions, the interviewer gives every candidate the same set of conditions. For example:

• The ranking may be delayed by around 10–30 seconds.
• The live-stream screen displays the current user’s position.
• The event ranking page displays the top 100 users.
• Gift traffic may increase significantly near the end of the event.
• Temporary inconsistencies in the displayed ranking are acceptable.
• The final ranking must be accurate because it is used to determine rewards.

We then ask:

Given these conditions, how would you design the server-side system?

We give all candidates the same conditions because we want to assess their ability to understand the need separately from their technical design ability. If candidates design the system using only the information they personally managed to uncover, differences in how much information they obtained would also change the difficulty of the design problem itself.

In the first stage, we observe the conversation with the PdM. In the second stage, we provide the same conditions and discuss how the candidate would design a system around them. We are not evaluating whether they can name Redis or a particular cloud service.

Where and how should the confirmed gift transaction be recorded? How should the ranking used for display be generated? Should the live display and final ranking calculation be separated? We ask why they chose a particular design under the given conditions.

3. Finally, We Introduce One Failure Scenario

After discussing the normal flow, we add a single failure scenario:

The gift transaction succeeds, but updating the ranking fails.
What should the design take into account to prepare for this situation?

Here, we look at whether the candidate separates the gift record from the ranking displayed on screen. Can the failed operation be retried? Can the same operation run multiple times without applying the result twice? Can the ranking be recalculated from the authoritative record?

There is much more I would like to write about this stage, including the separation of authoritative and derived data, retries, and idempotency. To keep this article focused, I’ll leave those topics for another time.

In a real system, every operation does not always complete as expected. Being able to preserve data through a failure and recover afterward is also an important part of a server-side engineer’s development capabilities.

Why Do We Start by Asking, “What Would You Ask the PdM First?”

Of these three stages, what I particularly want to discuss in this article is why we begin by asking, “What would you ask the PdM first?” Being able to produce a technical design from completed requirements is, of course, important. However, in actual product development, requirements such as “updates must appear within ten seconds,” “temporary inconsistencies are acceptable,” and “the final ranking must be accurate” are not always neatly defined at the beginning.

The starting point is often something like:

I want to update the ranking in real time.

This is still a request whose background and purpose have not been fully articulated. From there, the PdM and engineer talk, understand what they are trying to achieve, and turn it into concrete technical conditions. We consider that process to be part of an engineer’s development capabilities.

Requirements Are Made Concrete Through Conversation

I previously wrote a post titled “Understanding the Need, Not Just the Requirements.” As I discussed there, I distinguish between a “need” and a “requirement” as follows:

• Need: The state or value we want to create for users or the business
• Requirement: A concrete condition the system must satisfy to meet that need

The request, “When a gift is sent, I want the ranking to update in real time,” may look like a fairly concrete requirement. In particular, the phrase “real time” makes it easy to see the problem as a purely technical one: how should we implement a real-time ranking?

However, at this point, we do not know how many seconds “real time” means or why the ranking needs to update so quickly. What the PdM actually wants to achieve might be something like this:

We want users to feel the excitement of the event by seeing their gift immediately affect their position in the ranking.

Only after understanding this need can we discuss whether the experience fails unless the update appears within one second, whether a delay of ten or thirty seconds is acceptable, whether temporary inconsistencies are acceptable, and how accurate the final ranking needs to be.

In other words, the phrase “real-time ranking” already contains part of a proposed solution to the underlying need.

Is That Requirement Really Necessary?

One idea that stayed with me after reading The Agile Samurai, a popular introductory book on agile development, was that calling something a “requirement” can cause it to be treated as indispensable and non-negotiable from the outset.

For example, if we accept “real time” as a fixed requirement, the conversation immediately becomes about how to build a real-time system. On the other hand, if we go back and ask why real time is needed, we can identify which conditions are truly necessary to meet the underlying need.

The conversation may reveal that an update within one second really is important. If so, we should treat that as a requirement and design accordingly. But if a delay of ten to thirty seconds still delivers the intended experience, we gain more options for aggregation and client updates. We may be able to provide a sufficient experience while reducing load and cost.

Understanding the underlying need is not only about empathizing with users. Knowing what must be protected and what can be relaxed gives us more technical options and helps us make better trade-offs.

Between a Need and a Requirement, There Is Conversation

The flow in this example can be summarized as follows:

The important point is not that the engineer hears the need and then defines the requirements alone. The PdM understands the user experience and business outcome they want to create. The engineer understands how technical difficulty, system load, and cost change depending on the conditions. By bringing that knowledge into the conversation, they can define feasible requirements that still meet the underlying need.

One way we build this kind of shared understanding in Avvy development is through user story mapping. When considering a new feature or experience, we lay out how users move through Avvy and discuss which experiences matter most and how much we should build in the current scope, with the PdM, designer, and engineers participating together.

One thing I learned from Jeff Patton’s book User Story Mapping is that creating shared understanding through conversations around stories is more important than simply writing a polished specification and handing it off. The completed map is valuable, but personally, I find even greater value in the conversations that happen while creating it and the shared understanding the team develops through that process.

The interaction in an interview cannot reproduce actual product development exactly. Even so, rather than having the PdM hand over completed requirements, starting from an ambiguous request lets us briefly experience a process that matters in our day-to-day development.

Looking back, this conversation was exactly what we wanted to see by beginning the interview question with, “What would you ask the PdM first?”

Conclusion

When people talk about an engineer’s development capabilities, they often think of programming, databases, infrastructure, and architecture. These are all important, of course. In the question introduced here, after clarifying the need and assumptions, we also discuss the technical design and how the system should behave when something fails.

At the same time, actual product development does not always begin with completed requirements. It also involves starting from an ambiguous request, talking with PdMs and designers, understanding the value we want to deliver to users, and making what we should build concrete while communicating technical constraints and options. We consider that process to be part of the development capabilities we want to assess in interviews.

This does not mean that engineers should make every decision in place of the PdM. It means bringing the PdM’s understanding of users and the business together with the engineer’s technical knowledge through conversation.

You are absolutely welcome to read this article before interviewing with us. The scenario changes from interview to interview, and what we want to understand is not whether you already know the correct answer, but how you change your questions and design based on the PdM’s responses. If you think through these ideas beforehand, I believe we can have an even deeper conversation.

Of course, this format has its own issues. It favors people who can put their thoughts into words on the spot, and it may be tough for those who think best when given time to reflect. We plan to keep improving the format — for example, by letting candidates share their thinking with diagrams and charts rather than words alone.

For a short time during the interview, we try developing a product together as if we were working at AnotherBall. We will continue using actual interviews to make this a better opportunity for both sides to understand each other.

We’re Hiring!

AnotherBall is looking for engineers who can understand the underlying product need, communicate across roles, and work with the team to turn ideas into reality. If you are interested in product development at Avvy, please take a look at our open positions!

AnotherBall Careers

References

• Jonathan Rasmusson, The Agile Samurai: How Agile Masters Deliver Great Software
• Jeff Patton, User Story Mapping: Discover the Whole Story, Build the Right Product