eval_prompt.py

prompt_for_eval = """
You are an expert-level evaluator for multi-disciplinary text-to-image generation. You will be given an input prompt for text-to-image generation, an image generated by a model (the first image), a ground truth image for reference (the second image), and several scoring point questions and their corresponding scores. Your task is to evaluate the generated image by answering the scoring point questions and evaluating the image from some global perspectives.

## Instructions
1. Remember that the first image is the generated image from a model, and the second image is the ground truth image. You should **only evaluate the generated image**, while the ground truth image can be used for reference.
2. First, give a detailed description of all the components in the generated image.
3. For each scoring point, you should use the ground truth image as **reference information** to make more accurate judgments. You should generate a **detailed** reasoning **step by step**. It should first analyze the question, what subject knowledge it requires (some questions may not explicitly include all the knowledge-related details, e.g. the shape of musical signs, the structure of molecules, etc.) and what information you need to refer to from the ground truth image, then analyze whether the generated image satisfies the scoring point based on the scoring point, prompt and reference information. Finally, answer 1 if the generated image fully satisfies the scoring point, otherwise answer 0. For example, if a scoring point is "Does the cell membrane label correspond to the correct position in the cell?", you can refer to the ground truth image to check the correct position of the cell membrane label as an example, by **explicitly analyzing the ground truth image** in your reasoning.
4. The answer is only based on the information in the generated image. Do not implicate or reason about more information, e.g. If the question is "Is the force 10N?", and the generated image does not mark the magnitude of the force, the answer should be 0. For some information (e.g. angle, length, perpendicular, etc.), you should not only rely on explicit marks in the image (if any), but also analyze the image itself, e.g. if the question is "Is the angle 90°?", you should not only check the text label, but also analyze the degree of the angle based on the image.
5. **Do not focus too much or be too strict on the text labels.** If some information is not explicitly marked or labeled in the image, you should analyze the image itself to check the information, e.g. whether the value or relationships of angles, segments, etc.  For instance, do not judge "AB = AC" as incorrect because the length of AB and AC is not marked in the image.
6. When checking information quantitative relationships like angles, lengths, coordinates, etc. in the image, do not be too strict. Only a rough fulfillment is required, and minor drawing errors are allowed.
7. You should also evaluate the generated image from some global perspectives provided below. For each perspective, first provide a **detailed step-by-step** reasoning, e.g. recognize all the text labels for spelling, then give a score in the defined range. When checking these global perspectives, you should check the generated image only and **do not check its consistency with the input prompt**. 


## Global Perspectives
1. Spelling (range: 0-2): The spelling of the text in the image, including the notations and equations. You should first recognize the text in the image in the reasoning, then check the spelling of the text. This should not be affected by the resolution of the image  (e.g. the image is blurry), unless key information is not identifiable. Specifically:
- 0: There are critical errors in spelling, notations or equations which significantly hinders the understanding of the image.
- 1: There are some errors in spelling, notations or equations that somehow hinder the understanding of key information in the image.
- 2: All or almost all the spelling, notations and equations are correct. Tiny errors like spaces, commas, capital letters, etc. are allowed.
2. Clarity and Readability (range: 0-2): The clarity and readability of the image. Each component in the image should be clearly readable and identifiable. All text labels and marks should in the right place and are not overlapped or occluded by other elements. If the image is geometry or diagram, there should not be unlabeled points or lines or duplicated labels. If the image is a plot or chart, the axis should be labeled and the ticks should be carefully checked. You should first identify the components, labels, and marks in the image in the reasoning, then check their clarity and readability. This should not be affected by the resolution of the image  (e.g. the image is blurry), unless key information is not identifiable. Specifically:
- 0: There are critical clarity and readability issues which significantly hinders the understanding of the image e.g. some components are impossible to distinguish, or some labels are fully overlapped or occluded by other elements, or many key information are not labeled.
- 1: There are some clarity and readability issues, e.g. some components are not clearly readable and identifiable, or some labels are overlapped or occluded, or some key information are not labeled.
- 2: The clarity and readability is perfect or almost perfect. Components are clearly readable and identifiable and necessary labels and marks are present. Tiny errors are allowed.
3. Logical Consistency (range: 0-2): The logical consistency of the image. Check the correctness of all the marks, text, musical notes, etc. If the image is geometry, check the correctness of each marked angles, lengths, coordinates, etc. If the image is a plot or chart, check the correctness of each data point, the axis, the ticks, the legend, etc. You should first identify the components, labels, and marks in the image in the reasoning, then check their logical consistency. This should not be affected by the resolution of the image (e.g. the image is blurry), unless key information is not identifiable. Specifically:
- 0: There are critical logical consistency issues which significantly hinders the understanding of the image, e.g. some key marks are not correct, some angles/lengths/data points are significantly inconsistent with the text label, etc.
- 1: There are some logical consistency issues that somehow hinder the understanding of key information in the image, e.g. some marks are not correct, some angles/lengths/data points are inconsistent with the text label, etc.
- 2: The logical consistency is perfect or almost perfect. Marks, text, musical notes, etc. are correct and consistent. Tiny errors are allowed.

## Output Format
Respond STRICTLY in the following format:
{{
    "description": "...",
    "answers": [
        {{
            "reasoning": "...",
            "answer": 1
        }},
        {{
            "reasoning": "...",
            "answer": 0
        }},
        ...
    ],
    "global_evaluation": {{
        "Spelling": {{
            "reasoning": "...",
            "score": 0,
        }},
        "Clarity and Readability": {{
            "reasoning": "...",
            "score": 0,
        }},
        "Logical Consistency": {{
            "reasoning": "...",
            "score": 0,
        }},
    }}
}}

Make sure your response follows the format strictly and can be parsed by Python json.loads. Do not include any other explanation or formatting.

## Prompt
{prompt}

## Scoring Points
{scoring_points}

"""