Automation Developer Technical Assessment: Portfolio Review (The Three Tests)

Quick Answer

The Three Tests That Reveal the Failure Path

The assessment succeeds when the three tests probe the failure path, not the happy path, because a developer who lists the tools without demonstrating the retries, the dead-letter queues, and the run logs is a developer who has built workflows that break silently, and the portfolio that shows only the happy path hides the failure path the production system lives on. Run these tests in order, because each test reveals a different layer of the failure engineering.

• 1. The retry-and-dead-letter-queue build asks the developer to build a workflow that fails gracefully, with a retry policy and a dead-letter queue that captures the events the retries cannot save. A developer who builds the happy path and skips the failure path is a developer who ships workflows that break silently, because the dead-letter queue is the control that turns a silent failure into a reviewable one.
• 2. The run-log review asks the developer to instrument the workflow with run logs that record every execution, because a workflow without run logs is a workflow you cannot debug, and the developer who does not instrument the workflow is the developer who cannot tell you why it broke.
• 3. The manual-review-path design asks the developer to route the unhandled edge to a human, because the workflow that handles every case automatically is a workflow that has not met the case it cannot handle, and the manual-review path is the control that keeps the human in the loop the automation cannot replace.
• The portfolio trap a portfolio of working workflows proves the developer can build the happy path and says nothing about the failure path, because the portfolio is the demo the developer chose, and the failure path is the production the developer did not show.

A technical assessment for an automation developer is a hands-on test of the failure handling, not a resume review, because a developer who lists the tools without demonstrating the retries, the dead-letter queues, and the run logs is a developer who has built workflows that break silently, and the portfolio that shows only the happy path hides the failure path the production system lives on. The assessment runs on three tests: a retry-and-dead-letter-queue build that proves the developer engineers for failure, a run-log review that proves the developer instruments the workflow, and a manual-review-path design that proves the developer routes the unhandled edge to a human. The assessment fails when buyers hire on the portfolio alone, because a portfolio of working workflows proves the developer can build the happy path and says nothing about the failure path the business runs on. Most owners resolve this by choosing to hire an n8n workflow developer, or you want to hire a Voice AI automation architect once the scope is clear. The test you run first decides whether you hire a builder or a liability.

Key Takeaways

• A technical assessment for an automation developer is a hands-on test of the failure handling, not a resume review.
• A portfolio that shows only the happy path hides the failure path the production system lives on.
• The three tests are a retry-and-dead-letter-queue build, a run-log review, and a manual-review-path design.
• A developer who builds the happy path and skips the failure path ships workflows that break silently.
• The test you run first decides whether you hire a builder or a liability.

What an Automation Developer Technical Assessment Is

An automation developer technical assessment is a hands-on test that probes the failure path, not the happy path, because a developer who lists the tools without demonstrating the retries, the dead-letter queues, and the run logs is a developer who has built workflows that break silently, and the portfolio that shows only the happy path hides the failure path the production system lives on. The assessment is distinct from a portfolio review, which shows the demo the developer chose, because the assessment shows the layer the developer did not choose, and the layer the developer did not choose is the layer the business runs on. An assessed developer proves the failure path; a portfolio-reviewed developer proves the happy path, and the difference is the test the buyer runs before the hire. the vetting standards behind elite technical RevOps talent.

Why the Tests Come Before the Hire?

The mistake that drives the wrong hire is reviewing the portfolio and skipping the hands-on test. A portfolio of working workflows proves the developer can build the happy path and says nothing about the failure path the production system lives on, because the portfolio is the demo the developer chose, and the developer chooses the path that works, not the path that fails. The result is a hire who builds workflows that break silently, because the failure path the portfolio hid is the failure path the production system exposes, and the buyer who hired on the portfolio alone hired a builder, not an engineer. The tests are the work that matches the hire to the failure path before the hire ships the workflow, because a developer who cannot engineer for failure ships workflows that fail silently.

The distinction matters because the tests decide whether the buyer hires a builder or a liability. A developer who passes the three tests engineers for the failure path and ships workflows that fail loudly; a developer who passes the portfolio review alone builds the happy path and ships workflows that break silently, and the difference is the test the buyer runs before the hire. The tests are the work that matches the assessment to the production before the production arrives, because a developer who ships the happy path without the failure path breaks the business quietly.

The Three Tests of the Assessment

The assessment succeeds when the three tests probe the failure path. First, the retry-and-dead-letter-queue build: ask the developer to build a workflow that fails gracefully, with a retry policy that re-attempts failed operations and a dead-letter queue that captures the events the retries cannot save. A developer who builds the happy path and skips the failure path is a developer who ships workflows that break silently, because the dead-letter queue is the control that turns a silent failure into a reviewable one. Second, the run-log review: ask the developer to instrument the workflow with run logs that record every execution, because a workflow without run logs is a workflow you cannot debug, and the developer who does not instrument the workflow is the developer who cannot tell you why it broke.

Third, the manual-review-path design: ask the developer to route the unhandled edge to a human, because the workflow that handles every case automatically is a workflow that has not met the case it cannot handle, and the manual-review path is the control that keeps the human in the loop the automation cannot replace. The three tests together reveal the failure path the portfolio hides, because the retry-and-dead-letter-queue build proves the recovery, the run-log review proves the traceability, and the manual-review path proves the human-in-the-loop edge. The tests you run before the hire are the failure path you have after the workflow ships.

The Portfolio Trap That Hides the Failure Path

The fastest way to expose the failure path is to escape the portfolio trap. A portfolio of working workflows proves the developer can build the happy path and says nothing about the failure path, because the portfolio is the demo the developer chose, and the developer chooses the path that works, not the path that fails. The portfolio trap is the mistake of hiring on the demo the developer curated, and the developer who curates the happy path is the developer who hides the failure path the production system lives on. The three tests are the controls that probe the failure path the portfolio hides, because the failure path is the layer the business runs on, not the layer the demo shows.

The portfolio trap works because the happy path feels like competence. A portfolio of working workflows looks like a developer who can build, and the build is real for the happy path and absent for the failure path, because the failure path the portfolio hides is the failure path the production system exposes. The tests that probe the failure path are the tests that expose the competence the portfolio asserts, because the developer who engineers for failure is the developer the production system needs, and the developer who builds the happy path alone is the developer the production system breaks. The portfolio trap is the mistake that lets the failure path run undetected, and the three tests are the escape.

How to Structure the Engagement

Structure the engagement with the three tests as the gate before the hire, because a developer who cannot pass the failure-path tests is a developer who ships workflows that break silently. Run the retry-and-dead-letter-queue build, the run-log review, and the manual-review-path design as the assessment, not the portfolio, and weight the failure path heavier than the happy path, because the failure path is the layer the business runs on. A hire gated by the three tests turns the assessment into an asset; a hire gated by the portfolio alone turns it into a liability.

Build the off-ramp into the scope from day one. The engagement should end with you owning a documented, monitored automation stack that fails loudly without the developer, not with a dependency on their continued presence. Require the retry policies, the dead-letter queues, the run logs, and the manual-review paths to be handed over. A developer who passes the three tests leaves you with a system that fails safely; a developer who passes the portfolio alone leaves you with a system that breaks the business quietly.

Frequently Asked Questions

What is a technical assessment for an automation developer? A hands-on test of the failure handling, not a resume review. Why does a portfolio review fail? A portfolio shows the happy path the developer chose and hides the failure path the production system lives on. What is a retry-and-dead-letter-queue build? A test that asks the developer to build a workflow that fails gracefully. What is a run-log review? A test that asks the developer to instrument the workflow with execution logs. What is a manual-review-path design? A test that asks the developer to route the unhandled edge to a human.