Repository navigation
Code extraction error for the previous substeps for specific questions 13, 62 and 76 #49
Description
The entire code is not being used for the previous code which causes a cascading issue in the next steps.
if self.previous_llm_code[prev_step] is None:
if (
(prob_id == "13" and prev_step == 5) or
(prob_id == "62" and prev_step == 0) or
(prob_id == "76" and prev_step == 2)
):
prev_file_path = Path(
"../data",
f"{prob_id}.{prev_step+1}.txt"
)
else:
prev_file_path = Path(
self.output_dir,
self._get_background_dir(),
f"{prob_id}.{prev_step + 1}.py"
)
if prev_file_path.is_file():
prev_file_content = prev_file_path.read_text(encoding='utf-8')
func_name = extract_function_name(
prob_data["sub_steps"][prev_step]["function_header"]
)
function_code = get_function_from_code(
prev_file_content, func_name
)
self.previous_llm_code[prev_step] = function_code
else:
raise Exception(f'Generating problem {prob_id} step {num_steps} ahead of step {prev_step + 1}.')
A simple fix to this is to not do any extraction from the .txt files and use the entire code as it is:
if self.previous_llm_code[prev_step] is None:
if (
(prob_id == "13" and prev_step == 5) or
(prob_id == "62" and prev_step == 0) or
(prob_id == "76" and prev_step == 2)
):
prev_file_path = Path(
"../data",
f"{prob_id}.{prev_step+1}.txt"
)
else:
prev_file_path = Path(
self.output_dir,
self._get_background_dir(),
f"{prob_id}.{prev_step + 1}.py"
)
if prev_file_path.is_file():
prev_file_content = prev_file_path.read_text(encoding='utf-8')
if (
(prob_id == "13" and prev_step == 5) or
(prob_id == "62" and prev_step == 0) or
(prob_id == "76" and prev_step == 2)
):
# Use the entire .txt content for these special cases
self.previous_llm_code[prev_step] = prev_file_content
else:
# Extract the function code for all other cases
func_name = extract_function_name(
prob_data["sub_steps"][prev_step]["function_header"]
)
function_code = get_function_from_code(
prev_file_content, func_name
)
self.previous_llm_code[prev_step] = function_code
else:
raise Exception(f'Generating problem {prob_id} step {num_steps} ahead of step {prev_step + 1}.')
Activity
prann-ab commentedon Aug 7, 2026
#61 should cover the 13 and 62 halves of this.
Root cause is in
extract_function_name: it matchesdef ...(beforeclass ...(, and a class-basedfunction_headerincludes its own first method, soclass Maxwell:/class EnlargedBlock:resolve to__init__.get_function_from_codethen returns just that method and the class is dropped from the code accumulated into every later sub-step — which is why the dependent steps fail regardless of what the model generates.Two smaller issues in the same function, both fixed there: the class pattern requires a base-class paren so
class Foo:never matches, and the patterns are unanchored so they can match docstring prose (some headers document a parameter asenv: class Block, which yieldsenvas the "function name").11 of 341 sub-step headers change, all genuine class headers; 330 are identical.
76.3 is a plain function and already extracted correctly (987/1020 chars, unchanged by the patch) — if you are still seeing failures on that one it is likely a separate cause, and I would be glad to look if you have a repro.