An independent Jet accuracy test should be small enough for a crew to repeat and rich enough to expose the workflow’s weak points. It needs a written hypothesis, a representative test field, control that is separate from checkpoints, a frozen capture and processing recipe, and a report that shows spatial residuals rather than a single flattering number.
Artec currently publishes several Jet specifications: plus or minus 15 millimeters mapping accuracy in general environments, plus or minus 10 millimeters indoors or underground, plus or minus 5 millimeters change-detection capability, and a stated SLAM drift figure of plus or minus 0.03 percent. These figures are manufacturer definitions under stated contexts. The crew’s test should evaluate its own end-to-end result for the deliverable and site conditions it expects to encounter.
Write the question in one sentence
A useful hypothesis looks like this: “Using the documented walking route, control design, software version, and export settings, the delivered point cloud will meet the project’s horizontal and vertical acceptance criteria across this representative indoor-outdoor area.” The sentence should name the coordinate frame, feature or surface being tested, statistic, threshold, and reporting convention.
Avoid testing a vague claim such as general accuracy. Point-to-plane distance on a wall, coordinate residual at a target, hard-surface elevation, pipe diameter, and repeat-cloud change are different measurements. Select the one that matches the planned work.
Build a representative test field
The test field should include easy and difficult conditions: structured interior geometry, a repetitive corridor, an open area, a vertical transition, a loop closure, reflective or dark surfaces where relevant, and the feature types used in production. Keep it accessible enough for repeat runs and independent surveying.
| Component | Purpose | Rule |
|---|---|---|
| Project control | Place or constrain the dataset | May be used in adjustment |
| Independent checkpoints | Test delivered coordinates | Withheld from adjustment |
| Reference surfaces | Test local shape or alignment | Surveyed or characterized independently |
| Known dimensions | Test feature extraction | Measured with suitable independent method |
| Repeat run | Test reproducibility | Same recipe, separately acquired |
Distribute checkpoints across distance, direction, elevation, surface type, and capture segment. Do not cluster them next to control. The current USGS Lidar Base Specification, 2025 rev. A, is aimed at national elevation data, yet it offers a sound principle: final product accuracy should be evaluated with surveyed checkpoints and documented methods. The project’s governing standard determines exact counts and statistics.
Survey the reference independently
Use an independent method with uncertainty suitable for the threshold being tested. Record instrument, calibration status, setup, observation procedure, environmental conditions, network adjustment, coordinate reference system, vertical datum, units, and estimated checkpoint uncertainty. If the reference method’s uncertainty is large relative to the acceptance threshold, the test cannot resolve the claim well.
Targets should have defined geometry and an extraction procedure. A checkerboard center, sphere center, plane, nail, or surveyed surface point produces different observations. Document how the same feature will be located in the point cloud and in the reference survey.
Freeze the capture protocol
The protocol names device identity, firmware or software, configuration, deployment mode, battery and storage arrangement, start area, route, direction, pace band, stops, loop closures, control observations, environmental limits, and field log. Run at least two independent repetitions, preferably with a remobilization or another trained operator if operator repeatability matters.
Do not tune the second run after seeing the first result unless the test is explicitly developmental. If a protocol change is necessary, version it and restart the comparison. Otherwise the team learns how to pass its test rather than how repeatable the workflow is.
Freeze the processing protocol
Preserve raw data and specify software version, trajectory workflow, loop handling, control use and weighting, coordinate transformation, filtering, clipping, resolution, and export format. A person following the record should be able to reproduce the accepted output. Store each run separately and avoid manual edits that cannot be described.
Process the runs before revealing checkpoint coordinates to the operator where practical. That blind step reduces unconscious selection or adjustment around the answers.
Calculate and map residuals
For coordinate checkpoints, compute delivered minus reference values in each axis and the required horizontal or vertical summaries. Include signs. Plot vectors or colored residuals on a plan and by elevation. Report maximums, outliers, and the project-required statistic. Incorporate reference survey uncertainty when the governing standard requires it.
For surfaces, define the neighborhood, fitting method, slope treatment, and rejection rules before calculation. For known dimensions, record point selection and fitting. For repeat clouds, keep registration and comparison rules from hiding disagreement through a best-fit alignment that would not exist in production.
Look for failure shape
- A steady offset suggests reference, datum, target extraction, or transformation trouble.
- A rotation suggests control geometry or registration issues.
- Growth with route length can point to trajectory behavior or weak closures.
- A level-specific shift may reveal a transition problem.
- Surface-dependent errors can arise from incidence, reflectance, sampling, or reference method.
These are diagnostic leads, not automatic conclusions. Check the raw record and rerun a targeted test before assigning cause.
Report pass, fail, and operating envelope
The report includes the hypothesis, site diagram, conditions, equipment, reference survey, control and checkpoint distinction, capture and processing protocols, results, maps, exclusions, deviations, and raw evidence. State whether the workflow passed the predefined criterion. Then define the tested operating envelope: route length, geometry, deployment, surfaces, speed, environment, and software.
A passing test supports the documented workflow inside the tested envelope. It does not certify every future site or every possible processing choice.
Repeat the test after material hardware, software, mount, control, or process changes and on a sensible calibration schedule. The payoff is fantastic: crews stop arguing about brochure numbers and start improving a procedure they can measure.