This is the first in the RobotVRM series of applications that aim to work autonomously.
It automatically recognizes and starts talking if you project an object on the camera, and can also use Arduino to connect to peripherals via Bluetooth (only the HC-02 is supported as a connection point).
The appearance of the character can be changed with VRM, and the LLM selection and prompts can be changed to change its behavior.
Voice can also be set up with an API key or by setting up a home server.
There is also the ability to communicate via chat.
******* Important *******
To use this app, you will need an API key such as Anthropic or OpenAI. (Anthropic is recommended).
The API fee is not included in this app.
* Please note that the API fee may increase if you run the app for a long time.
* Gemini is not supported.
If you want to playback audio, you will need a separate environment for audio generation, such as a Style-Bert-VITS2 server, etc.
You can easily generate voice by using ElevenLabss API key, but the API fee is not included in this application.
******* Important *******
This app is based on the open source, Nike-chans aituber-kit based on pixivs ChatVRM (around the time of MIT license, before 2024/9/25).
Nikechan Twitter account
https://x.com/tegnike
aituber-kit (at the time of MIT license, before 2024/9/25)
https://github.com/tegnike/aituber-kit/tree/5c1be3dae7e38871839f37857e550b8f7387f718
ChatVRM
https://github.com/pixiv/ChatVRM
The operating environment for the above videos is as follows
LLM: Claude 3 Haiku
Voice: ElevenLabs/Jesica