CS34 HomeAgent: A Sandbox for Simulating Smart

Project number: CS34

Project Source: School of Computer Science

Project Title: HomeAgent: A Sandbox for Simulating Smart Home Interactionand Automation Using Large Language Models

Project Description and Scope: This project involves upgrading the HomeAgentsandbox framework, designed for simulating smart home environments andinteractions with large language models (LLMs). HomeAgent currentlyintegrates family agents and a HomeBot to emulate realistic human behavioursand smart home automation. The goal of this project is to enhance the existingframework with new features and improvements.Project Scope:

• Upgrade to GPT-4o:o Objective: Update the underlying model of HomeAgent to GPT-4o.o Details: Integrate GPT-4o into the HomeAgent framework,replacing the current model to leverage the advanced reasoning,tool use, and human interaction capabilities of GPT-4o.

• Add Family Agent Functionality:o Objective: Enhance the sandbox environment to supportadditional family agents.o Details: Modify the simulation environment to allow the inclusionof more family agents, enabling a more comprehensive simulationof household interactions and dynamics.

• Extend Simulation Timeline:o Objective: Extend the simulation capabilities to cover a full week.o Details: Update the simulation timeline to allow scenarios andinteractions to unfold over a week, providing a more detailed andextended observation of agent behaviours and smart homeautomation processes.

• Improve User Interface:o Objective: Enhancedai 写CS34  HomeAgent: A Sandbox for Simulating Smart   the user interface to display environment dataand agent actions more effectively.o Details: Redesign the user interface to provide real-time updateson the smart home environment, including temperature,humidity, and IoT device statuses, as well as detailed logs of agentactions and interactions.Expected outcomes/deliverables: Enhanced realism and complexity insmarthome simulations. Improved interaction capabilities with more family agents.Comprehensive data collection and analysis over extended simulation periods.User-friendly interface providing clear and detailed insights into the smarthome environment and agent behaviours

Specific required knowledge, skills, and/or technology: Proficiency in Python

and familiarity with the PyGame framework. Experience with deep learning,particularly with large language models such as GPT-4. UI/UX design skills for

enhancing user interfaces. (Good to have but not essential).

Fields that this project may involve: Software Development; ArtificialIntelligence; NLP.

Resources provided by the client: Provide working project code as the startingpoint. Provide detailed documentation as the current project description.