Why our n8n polling loop worked once, then lost its job ID

Cover: AI-generated editorial illustration.

Our Pinterest draft workflow started a background job, waited, then checked its status. The first status request worked. The next one returned a 404 because the request no longer contained the job identifier.

The fix was to read that identifier from the node that started the job, rather than assuming every response would repeat it. This is a small example of a common integration problem: the data coming out of a node is not necessarily the data that went in.

The workflow and the symptom#

The relevant path was:

Start draft generation → Wait → Check draft job → Job finished?
                           ↑                         |
                           └──────── no ─────────────┘

The start endpoint returned a job ID. The status endpoint returned a log, completion flag, and exit code. These simplified payloads illustrate the difference:

{ "job_id": "demo-job-001" }

{ "log": ["Generating drafts"], "done": false, "code": null }

The second object has no job_id. After the HTTP Request node returned it, that response became the item passed through the false branch and the next wait.

Our original polling URL ended with:

/api/job?id={{ $json.job_id }}

That expression used the current item’s JSON. On the second loop, its job_id was missing. The server received an empty ID and reported that the job did not exist.

Read the stable identifier from its source#

We changed the URL expression to:

/api/job?id={{ $('Start draft generation').first().json.job_id }}

The named node runs once at the start of this workflow. Its first output holds the ID for this one job. The polling response can now change without losing the reference.

This expression is specific to a single-job workflow. If a workflow starts several jobs, taking the first output for every item can associate requests with the wrong job. Use the correct item linking or explicitly carry the identifier with each job’s state instead.

The expression field also needs the base URL for your actual service. Our service is private and is not a public API provided by n8n or Pinterest.

What we verified#

On October 24, the corrected manual execution:

• Completed successfully.
• Ran the status-check node twice.
• Reached the final Draft summary node.
• Reported six pending drafts, with an existing image for each.
• Performed no Pinterest publishing action.

The completed run therefore exercised the return through the loop that failed previously. We also made the final summary read the successful job branch, so it would not accidentally show the first incomplete status response.

A sanitized verification summary records these observations. It is a summary of our saved execution, not a portable test suite or a promise that another API behaves identically.

How to investigate this in your own workflow#

1. Open the failed execution and select the HTTP Request node.
2. Compare its actual request URL with the URL you intended.
3. Inspect the item arriving on each loop, especially any identifier used in the URL.
4. Check which fields the API promises to return; do not assume it echoes the request.
5. Fix the reference and test a job that needs more than one poll.
6. Inspect the final output, rather than relying only on a green execution status.

Also set a reasonable polling interval and execution limit. This particular successful run does not establish recovery behavior for expired jobs, restarts, repeated submissions, or provider outages. Those need separate tests before relying on a workflow for client work.

A second-poll test you can repeat#

Use deliberately different response shapes when testing this pattern. If every mock response repeats the ID, the mock hides the bug. The smallest useful sequence is a start response with an ID, one incomplete status response without it, and one completed status response without it.

Stage	Current item	ID for the next request
Start	job_id: demo-job-001	Available in the current item
First status check	done: false, code: null	Missing from the current item; still in Start
Second status check	done: true, code: 0	Read from Start again

Download the free polling practice pack. It contains a local JavaScript simulation, synthetic JSON cases, and a test worksheet. Run node polling-demo.mjs after unzipping. It needs Node.js, no packages, credentials, or network access. The simulation demonstrates the state-loss bug and checks successful completion, a failed job, and a poll limit. It is a learning companion, not an importable n8n workflow or a test of your provider.

In n8n, keep the existing workflow inactive while reproducing the problem. Use a test endpoint or synthetic responses and a manual trigger. Open the second run of the polling node in the execution details, rather than inspecting only its first run. Compare the evaluated URL, the incoming item, and the response. Keep the job identifier visible in your test notes, with private IDs redacted before sharing.

Completion needs two decisions#

“Finished” and “successful” are different conditions. In our service, done: true means the process stopped; code: 0 means it stopped successfully. A nonzero exit code must go to the failure branch, not the normal inventory summary. Your provider may use status strings such as failed or completed instead: match its documented contract.

Decide the maximum wait before turning on a schedule. For example, a test budget of ten polls at a fifteen-second interval is about 150 seconds of waiting, plus request time. This is an illustrative budget, not a recommended limit for every API. Record both a poll limit and an elapsed-time deadline if jobs can take unpredictable amounts of time. A timeout should identify the job and explain the next recovery step; it should not quietly appear as success.

A transport failure also differs from an unfinished job. A request that times out does not establish that the background job failed. Blindly starting the whole workflow again could create a second job. Check the provider’s status and duplicate-request behavior before adding automatic retries. The successful CiteLab run above did not verify those failure cases; the practice pack simulates them only to help you design your own checks.

A useful handover record#

Keep a short record with the workflow version, test date, input fixture, expected result, actual result, and the person responsible for a failed execution. Include where to find logs and how to resume or stop a job. This makes the workflow easier to maintain when the person who built it is unavailable.

Before relying on it for client work, test at least these cases:

• A job that completes immediately and one requiring several polls.
• A finished job with a failure exit code.
• An ID that is missing, unknown, or expired.
• A job that never finishes within the agreed wait budget.
• A request failure after the job has already started.
• Two jobs in one execution, if your workflow supports them.

For that last case, do not copy our .first() expression unchanged. It deliberately picks one start item. Multi-job workflows need a reliable association between each job and its status requests. n8n’s item linking documentation explains the concept; verify the mapping with two distinct IDs in your own execution.

Want a second pair of eyes?#

The free walkthrough and practice pack are enough to investigate this specific bug yourself. For a manual review of one small n8n workflow, see the $99 Workflow Reliability Review. It includes prioritized findings, suggested changes, and a test plan. Scope and delivery date are confirmed before payment; implementation and ongoing monitoring are separate.

References and next steps#

n8n documents current-node input data and reading another node’s output. Its Wait node documentation explains the available pause and resume modes.

Use the workflow readiness checklist to record your own tests. For the bigger picture, read how our Pinterest draft workflow is organized.