Skip to content

Add support for significant figures (sig_figs) parameter - #273

Open
m-messer wants to merge 1 commit into
mainfrom
feature/sig_fig
Open

Add support for significant figures (sig_figs) parameter#273
m-messer wants to merge 1 commit into
mainfrom
feature/sig_fig

Conversation

@m-messer

@m-messer m-messer commented Aug 7, 2026

Copy link
Copy Markdown
Member

Problem

This was a requested feature (#270). At the moment, the evaluation function can only check a numeric answer using absolute or relative tolerance. Some questions need something more specific: they need to check that a student's answer is written to a required number of significant figures, not just that the value is close enough.

For example, if the correct answer is 92 and the question asks for 4 significant figures, 92.00 should be accepted but 92 should not, even though they represent the same number. Tolerance alone cannot tell these apart, since it only checks how close the value is, not how precisely it was written.

Changes

  • app/utility/expression_utilities.py: added four new functions.
    • round_to_sig_figs: rounds a number to a given number of significant figures.
    • split_numeric_string: checks that a response is written as a plain number (e.g. "92.00"), and splits it into its whole-number part, decimal part, and whether a decimal point was written.
    • count_sig_figs: counts the significant figures in those parts, following the standard rules (leading zeros do not count, trailing zeros after a decimal point do).
    • sig_figs_match: combines the three functions above to decide whether a response is both numerically correct and written to the right number of significant figures. To compare the rounded response and rounded answer, it uses Python's math.ulp function as the allowed error, instead of comparing with ==. Rounding a number to a fixed number of significant figures does not always produce an exact result in floating point arithmetic, so two values that should be treated as equal can end up differing by a tiny amount. math.ulp(x) gives the smallest possible gap between x and the next representable floating point number, so using it as the allowed error accepts only that unavoidable floating point rounding error, without being loose enough to accept a genuinely different value. This is the same idea as numpy.spacing, which is what the equivalent Is-Similar feature uses, but math.ulp is part of the standard library, so no new dependency (numpy) needed to be added to this project.
  • app/evaluation.py: added the sig_figs parameter, with significant_figures accepted as an alternative name. Added checks that raise an error if sig_figs is used together with atol/rtol, or if sig_figs is not a positive whole number.
  • app/context/symbolic.py: in check_equality, added a check at the very start of the function. If sig_figs is set and the comparison is a direct response = answer (not some other custom comparison), the function now uses sig_figs_match instead of its normal equality check. This has to happen before the normal check, not after, because 92.00 and 92 would otherwise already be treated as equal (they are the same number) before the significant figures were ever counted.
  • app/context/physical_quantity.py: made the equivalent change inside quantity_match, which is the function that decides whether a response matches the answer for physical quantities. When sig_figs is set, it replaces the normal value check with sig_figs_match, using the response's value as originally written (before unit conversion) so the significant figures can be counted correctly. The unit comparison itself is unchanged. Also updated the existing logic that estimates a tolerance from the number of significant figures in the answer, so it is skipped whenever sig_figs is set, keeping the two features independent.
  • app/docs/user.md: added a section explaining the new sig_figs/significant_figures parameter, with an example and the significant figure counting rules.
  • app/docs/dev.md: added a short technical note for developers on how the feature fits into the existing code.
  • app/tests/expression_utilities_test.py: added tests for the four new functions.
  • app/tests/symbolic_evaluation_test.py and app/tests/physical_quantity_evaluation_test.py: added tests that call the full evaluation function with sig_figs set, covering correct answers, wrong values, wrong precision, non-numeric answers, and the new error cases.

Checklist

  • Tests added/updated
  • Docs updated (app/docs.md) if user-facing behaviour changed

Closes #270

@peterbjohnson peterbjohnson left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Questions:

  • units: are there tests for physical quantities with different units, e.g. 1 mile vs 1609 m, combined with sig_figs? How does it work?

  • symbols: can symbols be used with sig_figs? E.g. x**2?

  • relaxed: we previously discussed two versions of sig_figs. One is the scientific version, implemented here, the other is the more relaxed sense of 'correct within x sig_fig'. Are we offering both, or only the first? What's our reasoning?

  • clashes: is this the first time we've had incompatible parameters? Any thoughts on user experience, given that we provide atol/rtol by default?

I've also made a minor inline comment

Comment thread app/context/symbolic.py
# must still fail, so this can't be gated behind "ordinary equality already returned False".
lhs_string = criterion.children[0].content_string().strip()
rhs_string = criterion.children[1].content_string().strip()
if {lhs_string, rhs_string} == {"response", "answer"}:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this different to the test in the physical_quantity context line 277, which permitted the opposite variant too (both of response==answer and answer==response)?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature: Support Sig Fig

2 participants