A trading bot working successfully on demo shows that a particular sequence completed in the chosen environment. It does not establish the price or quantity at which an order would have filled in a real market. To investigate a difference, compare events and conditions rather than only the two accounts’ final profit.

The following protocol is for examining available logs and planning a check. It does not require opening a live account or placing trades for an experiment. If live execution data is unavailable, leave that conclusion unverified while retaining useful evidence about connectivity and software behaviour.

Identify what the demo actually simulates

The labels demo, paper and testnet do not specify one execution model. Ask the environment provider which data it receives, how it assigns fill prices, whether it supports the required orders and how it accounts for fees. An environment can test message exchange effectively while treating liquidity and order queues differently.

For example, IBKR describes its paper account as a simulator without real execution or clearing. Its documented limitations include top-of-book simulated fills and differences for certain order types. These are IBKR-specific limitations, not a description of every demo account.

Also distinguish real-time observation from a historical backtest with an out-of-sample period. They answer different questions: replaying history does not replace observing a current connection, while a functioning connection does not establish strategy quality.

Freeze the comparison conditions

Create a test record before analysing results. Do not silently change it halfway through. If a parameter changes, record when and begin a separate observation segment. Otherwise, a difference caused by a new setting may be incorrectly attributed to the environment.

• Software version, settings and configuration identifier.

• Venue, environment, account type and position-accounting model.

• Exact instrument name, contract specification and quantity units.

• Observation window, time zone and clock-synchronisation status.

• Data source, price type and available information depth.

• Execution rules, fees and other applicable charges.

Two instruments are not equivalent merely because their symbols look similar. Different specifications or quote sources can change a signal before any order is submitted. For an API connection, also check key permissions and environment ownership, keeping secrets out of the test record.

Match the event chain

For each case, retain the times of data receipt, signal creation, request submission, venue response and execution. Use order identifiers within each environment. Define the matching rule between environments in advance: for example, the same instrument, side and originating signal when the input data is genuinely shared.

Do not select pairs afterwards because their profits look similar. If the signals differ, record that as a separate finding. Unmatched cases belong in the report too: a missing request or skipped signal can matter more than the average price of successfully completed trades.

The QuantConnect reconciliation guide identifies data and execution modelling as sources of differences between historical simulation and live results. The practical approach here is to find the first stage that diverges and then examine its effect on later events.

A worked partial-fill example

Suppose a hypothetical purchase is for 100 units. Demo records the entire quantity at 25. An available live log records 60 units at 25 and the remaining 40 at 25.05. The second record’s average price is (60 × 25 + 40 × 25.05) / 100 = 25.02.

For equal quantity, the purchase cost differs by 2 currency units: 2,502 versus 2,500 before fees. This is an arithmetic illustration, not a forecast of a typical difference. Had the remaining 40 units not filled, treating both records as the same completed position would have been incorrect.

The next question is why the records differ. Did submission times, available quotes and simulation rules match? Did the second portion fill later? An average price alone cannot establish that the program was slow or that the broker caused a problem.

Classify the differences before calculating performance

1. Data: different quotes, a missing update, timestamp differences or contract specifications.

2. Decision: a different signal, setting, filter or existing position state.

3. Request: different quantity, order type, parameters or an acceptance failure.

4. Execution: partial fills, waiting, cancellation or a different price.

5. Accounting: fees, other charges and open results included under different rules.

The difference between two machine clocks is not automatically a measured network delay. Likewise, a result based on actual execution prices should not have an estimated slippage amount subtracted again. The execution and cost audit guide provides the detailed record structure.

Write a conclusion the evidence supports

State the period, number of matched and unmatched events, confirmed differences and questions without sufficient evidence. List exactly what was checked: reading the account, constructing a request, handling a response or obtaining actual execution. “The connection works” is useful evidence, but it is different from “the trading result is reproduced”.

For NH PRO preparation, use the connection and operating manual. Demo/live comparison concerns trading environments; it does not describe software access terms or imply that every product provides a demo version.