SystemMessage ---------------------------------------- SYSTEM PROMPT FOR ACTION MODEL: === GLOBAL INSTRUCTIONS === - Environment: macOS. Current time is 2026-03-12 03:26:31.798224. - You will receive the goal you need to achieve, and execute appropriate actions based on the goal you received. - You can only open the apps that are already installed in the computer, The available apps in this macbook is: Home, Google Chrome, OmniPlayerStore, Messages, Stocks, OpenVPN Connect, App Store, Image Capture, Music, Effect House, TV, Notes, Clash Verge, GarageBand, zoom.us, Books, Image Playground, Safari, 百度网盘HD, Idle Police Tycoon, WeChat, 翼狐客户端, Mission Control, 抖音, QQLive, Calculator, TextEdit, IINA, Stickies, 小宇宙, 斗鱼直播, Pages, Weather, 腾讯元宝, Clock, Dictionary, Tips, Contacts, Marmoset Viewer, Automator, 语雀, Games, Apps, FaceTime, Shortcuts, ClashX, 爱奇艺, Journal, Maps, Keynote, iMovie, Police Station, VtTalkStudent, News, Freeform, VoiceMemos, Photo Booth, 优酷, System Settings, wpsoffice, FindMy, Photos, Reminders, Podcasts, Phone, The Grand Mafia, QuickTime Player, Numbers, The Unarchiver, Jail Manager Simulator, TencentMeeting, Maxon, 东方财富 2, iPhone Mirroring, Calendar, QQMusic, Siri, Chess, Time Machine, Mail, Passwords, Preview, Knock, Font Book, Spotify - All the coordinates are normalized to 0-1000. You MUST output normalized positions. - The maximum number of actions you can output in one step is 5. - Always adhere strictly to JSON output format: { "action": [List of all actions to be executed this step], } WHEN OUTPUTTING MULTIPLE ACTIONS AS A LIST, EACH ACTION MUST BE AN OBJECT. === ROLE-SPECIFIC DIRECTIVES === - Role: Action Model for MacOS 15+ Agent. Execute actions based on goal. - Responsibilities: 1. Follow the next_goal precisely using available actions: Complete task: {done: {}} Type: {input_text: {'text': {'type': 'string'}}} Open a mac app: {open_app: {'app_name': {'type': 'string'}}} Run an AppleScript: {run_apple_script: {'script': {'type': 'string'}}} Single Hotkey: {Hotkey: {'key': {'type': 'string'}}} Press Multiple Hotkey: {multi_Hotkey: {'key1': {'type': 'string'}, 'key2': {'type': 'string'}, 'key3': {'anyOf': [{'type': 'string'}, {'type': 'null'}], 'default': None}}} RightSingle click at specific pixel: {RightSingle: {'position': {'description': 'Coordinates (normalised) [x,y]', 'items': {'type': 'number'}, 'type': 'array'}}} Left click at specific pixel: {Click: {'position': {'description': 'Coordinates (normalised) [x,y]', 'items': {'type': 'number'}, 'type': 'array'}}} Drag an object from one pixel to another: {Drag: {'position1': {'description': 'Starting Coordinates (normalised) [x,y]', 'items': {'type': 'number'}, 'type': 'array'}, 'position2': {'description': 'Ending Coordinates (normalised) [x,y]', 'items': {'type': 'number'}, 'type': 'array'}}} Move mouse to specific pixel: {move_mouse: {'position': {'description': 'Coordinates (normalised) [x,y]', 'items': {'type': 'number'}, 'type': 'array'}}} Scroll up: {scroll_up: {'position': {'description': 'Coordinates (normalised) [x,y] to execute scroll', 'items': {'type': 'number'}, 'type': 'array'}, 'dx': {'anyOf': [{'type': 'integer'}, {'type': 'null'}], 'description': 'Amount to scroll left, between 0 and 25'}, 'dy': {'anyOf': [{'type': 'integer'}, {'type': 'null'}], 'description': 'Amount to scroll up, between 0 and 25. 25 is one page up'}}} Scroll down: {scroll_down: {'position': {'description': 'Coordinates (normalised) [x,y] to execute scroll', 'items': {'type': 'number'}, 'type': 'array'}, 'dx': {'anyOf': [{'type': 'integer'}, {'type': 'null'}], 'description': 'Amount to scroll left, between 0 and 25'}, 'dy': {'anyOf': [{'type': 'integer'}, {'type': 'null'}], 'description': 'Amount to scroll down, between 0 and 25. 25 is one page down'}}} Tell the short memory that you are recording information: {record_info: {'text': {'description': 'information you need to record', 'type': 'string'}, 'file_name': {'description': 'filename summary for the recorded info', 'type': 'string'}}} Wait: {wait: {}} 2. If the next goal involves the intention to store information, you must output the action "record_info" with both `text` and `file_name`. 3. When the next goal involves analyzing the user information, you must output a record_info action with a detailed analysis based on the screenshot, brain's analysis, and the stored information. The `file_name` should be a short summary ending in `.txt`. ============================================================ AIMessage ---------------------------------------- ============================================================ ToolMessage ---------------------------------------- macOS automation session started ============================================================ HumanMessage ---------------------------------------- [Text Content] CURRENT APPLICATION STATE: Analysis to the current screen is: {'analysis': 'The previous action successfully scrolled up, revealing the upper portion of the conditional order form. I can now see the stock code (510300) is already selected, current price is 4.711, and the trigger price field (监控价) is visible and empty. I need to fill in: 1) Trigger price = 4.711 × 0.997 = 4.697, 2) Order price = 4.697 × 1.001 = 4.702, 3) Quantity = 100, 4) Validity = 当日.', 'sop_check': 'None'}. Your goal to achieve in this step is: Click on the 监控价 (trigger price) input field in the 触发条件 section and enter the calculated trigger price value of 4.697 (current price 4.711 minus 0.30%). [Image URL] data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAABegAAAPWCAYAAACbZnh6AAAMSmlDQ1BJQ0MgUHJvZmlsZQAAeJyVVw... ============================================================ RESPONSE action=[ActionModel(Click={'position': [718, 274]}), ActionModel(input_text={'text': '4.697'})] ============================================================