Build a desk robot that looks at you

Robots feel alive when they react to us. In this capstone you build a small desk robot whose head is a servo and whose eyes are the ultrasonic sensor. It looks around until it finds the nearest object (you!), then keeps turning to face it as you move. Come very close and it gets a fright and squeaks; walk away and it gets bored and looks around again. A web page shows its face, with eyes that follow the head, and lets you send it to sleep.

What you'll learn

• How to find the nearest object with a scan
• How to follow a moving object with small corrections
• How to give a device moods with a state machine
• How a web page can show a live, animated face

What you need

• ESP32 board, expansion board and USB cable (ESP32 IoT Starter Kit)
• Ultrasonic sensor HC-SR04 (ESP32 IoT Starter Kit)
• Servo motor (ESP32 IoT Starter Kit)
• 3 resistors of the same value, anything from 1 kΩ to 10 kΩ (ESP32 IoT Starter Kit)
• Passive buzzer module (45-in-1 Sensor Kit)
• A 2.4 GHz Wi-Fi network
• A small cardboard box as the head
• Breadboard and jumper wires (ESP32 IoT Starter Kit)

Make it real: cut two holes in a small box for the sensor's round "eyes", push the sensor in from the inside and tape the box onto the servo horn. Connect the sensor with four female-to-male jumper wires, loose enough for the head to turn.

Protect the ESP32

The HC-SR04 answers with 5 V on its Echo pin, which is too much for the ESP32. As in the radar capstone, three equal resistors lower it to about 3.3 V: one from Echo to the ESP32 pin, and two in a row from the ESP32 pin to GND.

Wiring

Sensor modules from different makers can label and order their pins differently. Always follow the labels printed on your module.

The moods

The sketch

Install the ESP32Servo library (by Kevin Harrington).

Replace YOUR_WIFI_NAME and YOUR_WIFI_PASSWORD with your own network details (2.4 GHz only). Open the Serial Monitor at 115200 baud to see the address, then open it in a browser on a phone or computer on the same Wi-Fi.

#include <WiFi.h>
#include <WebServer.h>
#include <ESP32Servo.h>

const char* WIFI_NAME = "YOUR_WIFI_NAME";
const char* WIFI_PASSWORD = "YOUR_WIFI_PASSWORD";

const int SERVO_PIN = 13;
const int TRIG_PIN = 26;
const int ECHO_PIN = 27;
const int BUZZER_PIN = 23;
const int SEE_CM = 80;          // things further away are ignored
const int TOO_CLOSE_CM = 12;

const byte LOOKING = 0, WATCHING = 1, SURPRISED = 2, SLEEPING = 3;
const char* const MOODS[] = {"looking", "watching", "surprised", "sleeping"};

WebServer server(80);
Servo head;
byte mood = LOOKING;
int angle = 90, scanStep = 4;
int lastCm = 0;
unsigned long moodStart = 0, lastSeen = 0, lastMove = 0;

const char PAGE[] = R"rawliteral(
<!DOCTYPE html><html><head><meta charset='utf-8'>
<meta name='viewport' content='width=device-width, initial-scale=1'><title>Desk robot</title>
<style>
body{font-family:sans-serif;text-align:center;background:#12285A;color:#fff;margin:0;padding:20px}
.face{width:260px;height:180px;margin:20px auto;border-radius:30px;background:#1F5FA8;display:flex;justify-content:center;align-items:center;gap:30px}
.eye{width:70px;height:70px;border-radius:50%;background:#fff;position:relative;overflow:hidden;transition:height .3s}
.pupil{width:30px;height:30px;border-radius:50%;background:#12285A;position:absolute;top:20px;left:20px;transition:left .2s}
button{font-size:18px;padding:8px 18px;margin:6px;border-radius:20px;border:0}
</style></head><body>
<h1>Desk robot</h1>
<div class='face'><div class='eye'><div class='pupil'></div></div><div class='eye'><div class='pupil'></div></div></div>
<p style='font-size:24px' id='mood'>-</p><p id='info'>-</p>
<button onclick="fetch('/sleep')">Sleep</button><button onclick="fetch('/wake')">Wake</button>
<script>
async function update() {
  const d = await (await fetch('/data')).json();
  document.getElementById('mood').textContent = d.mood;
  document.getElementById('info').textContent = d.cm ? 'Nearest: ' + d.cm + ' cm at ' + d.angle + '°' : 'Nothing in sight';
  document.querySelectorAll('.pupil').forEach(function (p) { p.style.left = (20 + (90 - d.angle) / 90 * 18) + 'px'; });
  document.querySelectorAll('.eye').forEach(function (e) {
    e.style.height = d.mood === 'sleeping' ? '8px' : (d.mood === 'surprised' ? '90px' : '70px');
  });
}
setInterval(update, 300);
update();
</script></body></html>
)rawliteral";

int measureCm() {
  digitalWrite(TRIG_PIN, LOW);
  delayMicroseconds(2);
  digitalWrite(TRIG_PIN, HIGH);
  delayMicroseconds(10);
  digitalWrite(TRIG_PIN, LOW);
  unsigned long echoTime = pulseIn(ECHO_PIN, HIGH, 8000);   // up to ~1.3 m
  int cm = echoTime / 58;
  return (echoTime == 0 || cm > SEE_CM) ? 0 : cm;           // 0 = nothing
}

void look(int newAngle) {
  angle = constrain(newAngle, 20, 160);
  head.write(angle);
}

void enter(byte newMood) {
  mood = newMood;
  moodStart = millis();
  if (newMood == SURPRISED) {                 // squeak!
    for (int f = 1500; f <= 3000; f += 300) {
      tone(BUZZER_PIN, f);
      delay(25);
    }
    noTone(BUZZER_PIN);
    look(angle + (angle < 90 ? 25 : -25));    // jerk back
  }
  if (newMood == SLEEPING) look(90);
}

void think() {
  if (mood == SLEEPING) return;
  if (mood == SURPRISED) {
    if (millis() - moodStart > 1000) enter(WATCHING);
    return;
  }

  if (mood == LOOKING) {
    look(angle + scanStep);                   // scan from side to side
    if (angle <= 20 || angle >= 160) scanStep = -scanStep;
    lastCm = measureCm();
    if (lastCm > 0) {
      lastSeen = millis();
      enter(WATCHING);
    }
    return;
  }

  // WATCHING: measure a little to the left, straight ahead and a little to the right
  int here = measureCm();
  look(angle - 6);
  delay(25);                                  // give the servo time to turn
  int left = measureCm();
  look(angle + 12);
  delay(25);
  int right = measureCm();
  look(angle - 6);

  int best = here, move = 0;                  // 0 counts as "nothing there"
  if (left > 0 && (best == 0 || left < best)) { best = left; move = -4; }
  if (right > 0 && (best == 0 || right < best)) { best = right; move = 4; }
  look(angle + move);                         // turn towards the closest

  lastCm = best;
  if (best > 0) lastSeen = millis();
  if (best > 0 && best < TOO_CLOSE_CM) enter(SURPRISED);
  else if (millis() - lastSeen > 4000) enter(LOOKING);
}

void setup() {
  Serial.begin(115200);
  pinMode(TRIG_PIN, OUTPUT);
  pinMode(ECHO_PIN, INPUT);
  head.setPeriodHertz(50);
  head.attach(SERVO_PIN, 500, 2400);
  look(90);

  WiFi.begin(WIFI_NAME, WIFI_PASSWORD);
  while (WiFi.status() != WL_CONNECTED) delay(500);
  Serial.print("Open http://");
  Serial.println(WiFi.localIP());

  server.on("/", []() { server.send(200, "text/html", PAGE); });
  server.on("/data", []() {
    String json = "{\"mood\":\"" + String(MOODS[mood]) + "\",\"angle\":" + String(angle) + ",\"cm\":" + String(lastCm) + "}";
    server.send(200, "application/json", json);
  });
  server.on("/sleep", []() { enter(SLEEPING); server.send(200, "text/plain", "ok"); });
  server.on("/wake", []() { enter(LOOKING); server.send(200, "text/plain", "ok"); });
  server.begin();
}

void loop() {
  server.handleClient();
  if (millis() - lastMove >= 60) {            // think about 15 times per second
    lastMove = millis();
    think();
  }
}

How it works

The robot has four moods, and think() runs the code for the current one about 15 times per second. In Looking mode it turns the head a few degrees each time and measures. As soon as anything is closer than 80 cm, it switches to Watching.

Watching uses a simple but effective trick: measure straight ahead, a little to the left and a little to the right, then turn a few degrees towards whichever is closest. Repeated many times per second, those small corrections keep the head pointed at you as you move. If nothing has been seen for four seconds, the robot gives up and looks around again. Get closer than 12 cm and it plays a rising squeak with tone() and jerks its head back: Surprised.

The web page polls /data about three times per second and moves the pupils left and right with the head's angle. The eyes get taller when the robot is surprised and become slits when it sleeps, all with a few lines of CSS.

Try this next

• Add a "happy" mood when you stay at a friendly distance for 10 seconds, with a short melody.
• Let it fall asleep by itself after five minutes of looking without finding anyone.
• Add a second servo to make it nod.

Troubleshooting

• It never connects to Wi-Fi: check the network name and password (both are case-sensitive) and make sure the network is 2.4 GHz.
• The page doesn't open: the phone or computer must be on the same Wi-Fi as the ESP32, not on mobile data.
• It never finds anything: check Trig on GPIO 26 and the divider on GPIO 27. Point it at a flat object at about 40 cm.
• It turns away from you instead of towards you: your servo turns the other way. Swap -4 and 4 in think().
• The head shakes nervously: echoes from the table. Raise the robot a few centimetres, or reduce the 6 and 12 degree checks to 4 and 8.
• The ESP32 restarts when the servo moves: servos draw bursts of current. Use a good USB port or a powered hub, and check that the servo's red wire is on 5V.