已发布项目

muse-gadget-for-esp-mosaico-518923ca

Revision 1

First release (0.1.0). Meta's Muse on the ESP-Mosaico board: talk with the AI key, captions on the screen, replies spoken in Chinese by the board. Enter your own SDK token on the board (Settings > MUSE > SDK token) before pairing. Supports the camera module and the Interaction module (presence greeting, lights, infrared).

Muse Gadget for ESP-Mosaico

Meta's Muse, the voice companion, on the ESP-Mosaico board: talk to it with the AI key, read its replies on the screen and hear them from the speaker. Built from the Muse Gadget SDK (Apache-2.0) as an ESP-Mosaico application, so it installs from here like any other and keeps Vibe Mode for updates.

What you need

  • An ESP-Mosaico board. Its screen, microphone, speaker, AI key and BOOT key are all it uses.
  • A Muse account and the Muse app on your phone, with an SDK token from gadgets.muse.ai (Account > SDK tokens). Every gadget needs one to pair.
  • 2.4 GHz Wi-Fi. The board has no 5 GHz.
  • Optional: the camera module in the left slot, for Muse's camera.capture; the Interaction module in either slot, for the presence greeting, lights and infrared.

First start

  1. Install this application and let the board restart into it.
  2. Press BOOT for the menu, open Settings, then the MUSE page, and tap SDK token. Type your mgst_… token (48 characters) and confirm. The row shows Set. The token stays on the board through restarts and pairing resets.
  3. In the Muse app: Settings > Devices > Developer mode, then Add Device and pick MuseGadget-XXXXXX.
  4. When the board asks, short-press the AI key within 60 s to confirm.
  5. Give your Wi-Fi name and password in the app. The board joins and goes online.

Over Bluetooth, the sdk.token=mgst_… command sets the token too, and sdk.token= (empty) forgets it.

Using it

Key What it does
AI key, short press Talk to Muse; press again to stop listening. Confirms a pairing when one is pending
AI key, held 5 s Forgets the pairing and Wi-Fi, then restarts
AI key, held while powering on Vibe Mode, for installing and updating applications
BOOT The menu: settings, screen off

Replies in Chinese are spoken by the board's own voice (esp-sr, Xiaole) with captions following the speech; replies in other languages are shown as text. Markdown is left out of the captions.

With the Interaction module plugged in, someone walking up after a quiet minute turns the screen on and lights the module for 6 s; after ten quiet minutes Muse also greets them for the time of day.

What Muse can do with the board

Muse calls these itself when they fit what you ask:

Command What it does
voice.say Says something unprompted, in the board's voice
voice.configure Volume and speaker on/off
display.show_text Shows a page of text
display.configure Brightness, auto-sleep time, screen on/off
display.draw_url, display.show_animation Shows an image or animation (upstream Muse)
camera.capture Takes a picture with the camera module
presence.read Whether someone is there, from the Interaction module
lights.set The module's six LEDs, one colour, optionally for a number of seconds
ir.send_nec One NEC infrared code from the module

Known limits

  • The board reaches Muse's servers directly. Where they are blocked, it needs a network that forwards its traffic; this image has no forwarding settings of its own.
  • English and other non-Chinese replies are shown, not spoken: the board's offline voice reads Chinese only.
  • Now and then ESP-Iris over USB stops answering while Muse keeps running; unplugging and replugging the cable brings it back.

Source and licences