SystemMessage ---------------------------------------- SYSTEM PROMPT FOR ACTION MODEL: === GLOBAL INSTRUCTIONS === - Environment: macOS. Current time is 2026-03-12 03:26:31.798224. - You will receive the goal you need to achieve, and execute appropriate actions based on the goal you received. - You can only open the apps that are already installed in the computer, The available apps in this macbook is: Home, Google Chrome, OmniPlayerStore, Messages, Stocks, OpenVPN Connect, App Store, Image Capture, Music, Effect House, TV, Notes, Clash Verge, GarageBand, zoom.us, Books, Image Playground, Safari, 百度网盘HD, Idle Police Tycoon, WeChat, 翼狐客户端, Mission Control, 抖音, QQLive, Calculator, TextEdit, IINA, Stickies, 小宇宙, 斗鱼直播, Pages, Weather, 腾讯元宝, Clock, Dictionary, Tips, Contacts, Marmoset Viewer, Automator, 语雀, Games, Apps, FaceTime, Shortcuts, ClashX, 爱奇艺, Journal, Maps, Keynote, iMovie, Police Station, VtTalkStudent, News, Freeform, VoiceMemos, Photo Booth, 优酷, System Settings, wpsoffice, FindMy, Photos, Reminders, Podcasts, Phone, The Grand Mafia, QuickTime Player, Numbers, The Unarchiver, Jail Manager Simulator, TencentMeeting, Maxon, 东方财富 2, iPhone Mirroring, Calendar, QQMusic, Siri, Chess, Time Machine, Mail, Passwords, Preview, Knock, Font Book, Spotify - All the coordinates are normalized to 0-1000. You MUST output normalized positions. - The maximum number of actions you can output in one step is 5. - Always adhere strictly to JSON output format: { "action": [List of all actions to be executed this step], } WHEN OUTPUTTING MULTIPLE ACTIONS AS A LIST, EACH ACTION MUST BE AN OBJECT. === ROLE-SPECIFIC DIRECTIVES === - Role: Action Model for MacOS 15+ Agent. Execute actions based on goal. - Responsibilities: 1. Follow the next_goal precisely using available actions: Complete task: {done: {}} Type: {input_text: {'text': {'type': 'string'}}} Open a mac app: {open_app: {'app_name': {'type': 'string'}}} Run an AppleScript: {run_apple_script: {'script': {'type': 'string'}}} Single Hotkey: {Hotkey: {'key': {'type': 'string'}}} Press Multiple Hotkey: {multi_Hotkey: {'key1': {'type': 'string'}, 'key2': {'type': 'string'}, 'key3': {'anyOf': [{'type': 'string'}, {'type': 'null'}], 'default': None}}} RightSingle click at specific pixel: {RightSingle: {'position': {'description': 'Coordinates (normalised) [x,y]', 'items': {'type': 'number'}, 'type': 'array'}}} Left click at specific pixel: {Click: {'position': {'description': 'Coordinates (normalised) [x,y]', 'items': {'type': 'number'}, 'type': 'array'}}} Drag an object from one pixel to another: {Drag: {'position1': {'description': 'Starting Coordinates (normalised) [x,y]', 'items': {'type': 'number'}, 'type': 'array'}, 'position2': {'description': 'Ending Coordinates (normalised) [x,y]', 'items': {'type': 'number'}, 'type': 'array'}}} Move mouse to specific pixel: {move_mouse: {'position': {'description': 'Coordinates (normalised) [x,y]', 'items': {'type': 'number'}, 'type': 'array'}}} Scroll up: {scroll_up: {'position': {'description': 'Coordinates (normalised) [x,y] to execute scroll', 'items': {'type': 'number'}, 'type': 'array'}, 'dx': {'anyOf': [{'type': 'integer'}, {'type': 'null'}], 'description': 'Amount to scroll left, between 0 and 25'}, 'dy': {'anyOf': [{'type': 'integer'}, {'type': 'null'}], 'description': 'Amount to scroll up, between 0 and 25. 25 is one page up'}}} Scroll down: {scroll_down: {'position': {'description': 'Coordinates (normalised) [x,y] to execute scroll', 'items': {'type': 'number'}, 'type': 'array'}, 'dx': {'anyOf': [{'type': 'integer'}, {'type': 'null'}], 'description': 'Amount to scroll left, between 0 and 25'}, 'dy': {'anyOf': [{'type': 'integer'}, {'type': 'null'}], 'description': 'Amount to scroll down, between 0 and 25. 25 is one page down'}}} Tell the short memory that you are recording information: {record_info: {'text': {'description': 'information you need to record', 'type': 'string'}, 'file_name': {'description': 'filename summary for the recorded info', 'type': 'string'}}} Wait: {wait: {}} 2. If the next goal involves the intention to store information, you must output the action "record_info" with both `text` and `file_name`. 3. When the next goal involves analyzing the user information, you must output a record_info action with a detailed analysis based on the screenshot, brain's analysis, and the stored information. The `file_name` should be a short summary ending in `.txt`. ============================================================ AIMessage ---------------------------------------- ============================================================ ToolMessage ---------------------------------------- macOS automation session started ============================================================ HumanMessage ---------------------------------------- [Text Content] CURRENT APPLICATION STATE: Analysis to the current screen is: {'analysis': 'The East Money (东方财富) application is open with a conditional order creation form (定价买入单 - Pricing Buy Order). The form is empty and ready for input. I need to fill in: stock code 510300, direction buy (already set as buy order), quantity 100, validity same day, trigger price = current price - 0.30%, and order price = trigger price + 0.10%. The first step is to enter the stock code.', 'sop_check': 'None'}. Your goal to achieve in this step is: Click on the stock code input field (请输入名称/代码) at the top of the form and type '510300' to search for the stock. [Image URL] data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAABegAAAPWCAYAAACbZnh6AAAMSmlDQ1BJQ0MgUHJvZmlsZQAAeJyVVw... ============================================================ RESPONSE action=[ActionModel(Click={'position': [720, 144]}), ActionModel(input_text={'text': '510300'})] ============================================================