SystemMessage ---------------------------------------- SYSTEM PROMPT FOR BRAIN MODEL: === GLOBAL INSTRUCTIONS === - Environment: macOS. Current time is 2026-03-12 03:26:31.794937. - You will receive task you need to complete and a JSON input from previous step which contains the short memory of previous actions and your overall plan. - If the task message includes a "Selected skills" section, use those skill instructions as primary guidance when choosing the next goal. - You will also receive 1-2 images, if you receive 2 images, the first one is the screenshot before last action, the second one is the screenshot you need to analyze for this step. - You need to analyze the current state based on the input you received, then you need give a step_evaluate to evaluate whether the previous step is success, and determine the next goal for the actor model to execute. - You can only ask the actor model to use the apps that are already installed in the computer, The available apps in this macbook is: Home, Google Chrome, OmniPlayerStore, Messages, Stocks, OpenVPN Connect, App Store, Image Capture, Music, Effect House, TV, Notes, Clash Verge, GarageBand, zoom.us, Books, Image Playground, Safari, 百度网盘HD, Idle Police Tycoon, WeChat, 翼狐客户端, Mission Control, 抖音, QQLive, Calculator, TextEdit, IINA, Stickies, 小宇宙, 斗鱼直播, Pages, Weather, 腾讯元宝, Clock, Dictionary, Tips, Contacts, Marmoset Viewer, Automator, 语雀, Games, Apps, FaceTime, Shortcuts, ClashX, 爱奇艺, Journal, Maps, Keynote, iMovie, Police Station, VtTalkStudent, News, Freeform, VoiceMemos, Photo Booth, 优酷, System Settings, wpsoffice, FindMy, Photos, Reminders, Podcasts, Phone, The Grand Mafia, QuickTime Player, Numbers, The Unarchiver, Jail Manager Simulator, TencentMeeting, Maxon, 东方财富 2, iPhone Mirroring, Calendar, QQMusic, Siri, Chess, Time Machine, Mail, Passwords, Preview, Knock, Font Book, Spotify - If you need full contents from specific recorded files, output a read_files request instead of analysis/current_state. You will receive the full file contents and then respond with the normal schema. YOU MUST **STRICTLY** FOLLOW THE JSON OUTPUT FORMAT BELOW—DO **NOT** ADD ANYTHING ELSE. It must be valid JSON, so be careful with quotes and commas. - Always adhere strictly to JSON output format: { "analysis": { "analysis": "Detailed analysis of how the current state matches the expected state", "sop_check": "Identify which step of the Selected Skill applies to this moment. Quote the step exactly. Write 'None' if no skill." }, "current_state": { "step_evaluate": "Success/Failed (based on step completion and your analysis)", "ask_human": "Describe what you want user to do or No (No if nothing to ask for confirmation. If something is unclear, ask the user for confirmation, like ask the user to login, or confirm preference.)", "next_goal": "Generate an actionable, procedural goal based on the current state (screenshots/memory) and any Selected Skills. If a skill applies, adapt its step(s) to the current screen and include concrete action details." }} OR (for read files only): { "read_files": { "files": ["file_a.txt", "file_b.txt"] } } === ROLE-SPECIFIC DIRECTIVES === - Role: Brain Model for MacOS 15+ Agent. Determine the state and next goal based on the plan. Evaluate the actor's action effectiveness based on the input image and memory. For most actions to be evaluated as **“Success,”** the screenshot should show the expected result—for example, the address bar should read `"youtube.com"` if the agent pressed Enter to go to youtube.com. - **Responsibilities** 1. Analysis and evaluate the previous goal. 2. Determine the next goal for the actor model to execute. 3. Check the provided image/data carefully to validate step success. 4. Mark **step_evaluate** as `"Success"` if the step is complete or correctly in progress; otherwise `"Failed"`. 5. If a page/app is still loading, or it is too early to judge failure, mark `"Success"`—but if the situation persists for more than five steps, mark that step `"Failed"`. 6. If a step fails, **CHECK THE IMAGE** to confirm failure and provide an alternative goal. - Example: The agent pressed Enter to go to youtube.com, but the image shows a Bilibili page → mark `"Failed"` and give the instruction that how to go to the correct webpage. - If the loading bar is clearly still progressing, mark `"Success"`. 7. If something is unclear (e.g., login required, preferences), ask the user for confirmation in **ask_human**; otherwise, mark `"No"`. 8. In the case of chatting with someone, you should ask the actor record the message history when the screenshot changes. 9. YOU MUST WRITE THE DETAIL TEXT YOU WANT THE ACTOR TO INPUT OR EXECUTE IN THE NEXT GOAL, DO NOT JUST WRITE "INPUT MESSAGE" OR "CLICK SEND BUTTON", YOU NEED TO WRITE DOWN THE MESSAGE DETAILS. UNLESS THE Necessary information remembered CONTAINS THAT MESSAGE OR INFO. 10. You should do the analyzation (including the user analyzation in the screenshot) in the analysis field. 11. When you ask the actor to scroll down and you want to store the information in the screenshot, you need to write down in the next goal that you want the actor to record_info (with a short `file_name`), then scroll down. 12. When you want to scroll for a whole page or half page, you need to specify it in the next goal. 13. If you find the information in the screenshot will help the later execution of the task, you need to write down in the next goal that you want the actor to record_info (with a short `file_name`), and what info to record. === ACTION-SPECIFIC REMINDERS === - **Text Input:** Verify the insertion point is correct. - **Scrolling:** Confirm that scrolling completed. - **Clicking:** Based on the two images, determine if the click led to the expected result. --- *Now await the Actor's input and respond strictly in the format specified above.* ============================================================ HumanMessage ---------------------------------------- The overall user's task is: 继续在当前东方财富2条件单新建页面：填写测试参数但绝对不要提交。填写内容：代码510300，方向买入，数量100，有效期当日；触发价=当前价下调0.30%，委托价=触发价上调0.10%。禁止点击任何包含提交/确认/下单/买入/卖出字样的按钮。填写完成后停止并输出 READY_FOR_CONFIRMATION。 The step by step plan is: Iteration: 1/1 Plan: Fill in the conditional order creation form in the East Money application with test parameters for stock code 510300. Set the direction to buy, quantity to 100, and validity to same day. Calculate the trigger price as 0.30% below the current price, and the order price as 0.10% above the trigger price. Do not click any submit, confirm, order, buy, or sell buttons. After completing all fields, output the text 'READY_FOR_CONFIRMATION'. ============================================================ AIMessage ---------------------------------------- ============================================================ ToolMessage ---------------------------------------- macOS automation session started ============================================================ HumanMessage ---------------------------------------- [Text Content] CURRENT APPLICATION STATE: Previous step is 10. Recorded info files (filenames only): record.txt Previous Actions Short History: Recent steps: Step 1 | Eval: success | Goal: Click on the stock code input field (请输入名称/代码) at the top of the form and type '510300' to search for the stock. Step 2 | Eval: success | Goal: Type '510300' in the search field at the top of the stock selection dialog to search for the stock. Step 3 | Eval: failed | Goal: Type '510300' in the stock code search field at the top of the dialog to search for the stock. Step 4 | Eval: success | Goal: Click on the search input field at the top of the stock selection dialog (the field with placeholder text '请输入证券代码/名称') to ensure it's focused, then type '510300' to search for the stock. Step 5 | Eval: success | Goal: Click on the search result '沪深300ETF 华泰柏瑞 510300' to select this stock and proceed to the conditional order form. Step 6 | Eval: success | Goal: Scroll down in the conditional order form to see all available fields including quantity, validity period, and any other required fields. Record the current price (4.711) for calculating trigger and order prices. Step 7 | Eval: success | Goal: Scroll up in the conditional order form to access the trigger price field (监控价) and other upper fields that need to be filled. Step 8 | Eval: success | Goal: Click on the 监控价 (trigger price) input field in the 触发条件 section and enter the calculated trigger price value of 4.697 (current price 4.711 minus 0.30%). Step 9 | Eval: success | Goal: Clear the current value in the 监控价 (trigger price) field and type '4.697' to set the trigger price as current price minus 0.30%. Pending steps: Step 10 | Eval: pending | Goal: Scroll down in the conditional order form to access the quantity field (委托数量) and validity period options. Then click on the quantity input field and enter '100'. [Image URL] data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAABegAAAPWCAYAAACbZnh6AAAMSmlDQ1BJQ0MgUHJvZmlsZQAAeJyVVw... [Image URL] data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAABegAAAPWCAYAAACbZnh6AAAMSmlDQ1BJQ0MgUHJvZmlsZQAAeJyVVw... ============================================================ RESPONSE {'analysis': {'analysis': 'The form has scrolled down successfully, revealing the quantity field (委托数量) and other options. I can see the trigger price is correctly set to 4.697. Now I need to fill in the quantity field with 100, then set the order price to 4.702 (trigger price + 0.10%), and find the validity period option.', 'sop_check': 'None'}, 'current_state': {'step_evaluate': 'Success', 'ask_human': 'No', 'next_goal': "Click on the quantity input field (委托数量) in the 委托方式 section and type '100' to set the order quantity."}} ============================================================