Fly an airplane game on your phone by tilting a sensor

Flight simulators use a joystick that you tilt, and drones are steered by tilting too. In this capstone you build both halves of a flying game. The controller is the MPU-6050 motion sensor, which you hold like a flight stick: bank left and right, pull back to climb, push forward to dive. The ESP32 reads its accelerometer and gyroscope 100 times a second, combines them into smooth angles and streams them over Wi-Fi. The game runs in the browser on your phone or computer: golden rings rush towards you out of the horizon, and you fly through them for points, while the horizon tilts as your plane banks. Every ring you hit beeps on the breadboard.

What you'll learn

• How to combine a gyroscope and an accelerometer with a complementary filter on the ESP32
• How to stream motion data to a browser game over a WebSocket at 40 messages a second
• How a simple perspective trick makes a 2D canvas look 3D
• How a game sends events back to the hardware

What you need

• ESP32 board, expansion board and USB cable (ESP32 IoT Starter Kit)
• GY-521 MPU-6050 module (45-in-1 Sensor Kit)
• 1 push button (ESP32 IoT Starter Kit)
• Passive buzzer (ESP32 IoT Starter Kit)
• Long jumper wires, and a ruler or small box to hold the sensor like a flight stick
• A phone or computer on the same Wi-Fi
• Breadboard and jumper wires (ESP32 IoT Starter Kit)

Wiring

Tape the sensor flat onto the end of a ruler or a small box, with the button where your thumb rests. That is your flight stick.

The sketch

Replace YOUR_WIFI_NAME and YOUR_WIFI_PASSWORD with your own network details (2.4 GHz only).

In the Arduino IDE, open Library Manager and install WebSockets by Markus Sattler (in the Library Manager, search for "WebSockets" and pick the one by Markus Sattler).

#include <WiFi.h>
#include <WebServer.h>
#include <ESPmDNS.h>
#include <WebSocketsServer.h>
#include <Wire.h>

const char* WIFI_NAME = "YOUR_WIFI_NAME";
const char* WIFI_PASSWORD = "YOUR_WIFI_PASSWORD";
const int MPU = 0x68, BUTTON_PIN = 32, BUZZER_PIN = 33;
const float GYRO_SHARE = 0.96;

WebServer server(80);
WebSocketsServer sockets(81);
float pitch = 0, roll = 0, pitchZero = 0, rollZero = 0, biasX = 0, biasY = 0;
unsigned long lastMicros, lastSend = 0;
bool lastButton = HIGH;

const char PAGE[] PROGMEM = R"rawliteral(<!doctype html><html><head><meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1"><title>Sky rings</title>
<style>body{margin:0;background:#0B1020;color:#fff;font-family:Arial,sans-serif;text-align:center}
canvas{width:100%;max-width:960px;display:block;margin:0 auto}#info{padding:6px}</style></head><body>
<div id="info">Connecting...</div><canvas id="c" width="800" height="450"></canvas>
<script>
var c = document.getElementById('c'), g = c.getContext('2d'), info = document.getElementById('info');
var W = 800, H = 450, K = 330, sock, tilt = { p: 0, r: 0 }, btn = 0;
var plane = { x: 0, y: 0 }, rings = [], score, lives = 3, speed = 9, playing = false, best = 0;
function start() { plane = { x: 0, y: 0 }; rings = []; score = 0; lives = 3; speed = 9; playing = true; }
function send(m) { if (sock && sock.readyState === 1) sock.send(m); }
function connect() {
  sock = new WebSocket('ws://' + location.hostname + ':81/');
  sock.onopen = function () { info.textContent = 'Hold the stick level and press the button to take off'; };
  sock.onclose = function () { info.textContent = 'Reconnecting...'; setTimeout(connect, 1000); };
  sock.onmessage = function (e) {
    var v = e.data.split(',').map(Number);
    tilt.p = v[0]; tilt.r = v[1];
    if (v[2] && !btn && !playing) start();
    btn = v[2];
  };
}
connect();
var last = performance.now(), spawn = 0;
function frame(now) {
  var dt = Math.min(0.05, (now - last) / 1000); last = now;
  var bank = Math.max(-40, Math.min(40, tilt.r)), climb = Math.max(-40, Math.min(40, tilt.p));
  if (playing) {
    plane.x = Math.max(-1, Math.min(1, plane.x + bank / 40 * 1.4 * dt));
    plane.y = Math.max(-0.55, Math.min(0.55, plane.y - climb / 40 * 1.2 * dt));
    speed += dt * 0.15; spawn -= dt;
    if (spawn <= 0) { rings.push({ x: (Math.random() * 2 - 1) * 0.85, y: (Math.random() * 2 - 1) * 0.45, z: 12 }); spawn = 1.6; }
    rings.forEach(function (r) { r.z -= speed * dt; });
    rings = rings.filter(function (r) {
      if (r.z > 1) return true;
      if (Math.abs(r.x - plane.x) < 0.22 && Math.abs(r.y - plane.y) < 0.22) { score++; send('ring'); }
      else { lives--; send('miss'); if (lives <= 0) { playing = false; best = Math.max(best, score); send('over'); } }
      return false;
    });
  }
  // sky and ground, tilted with the plane's bank
  g.save(); g.translate(W / 2, H / 2); g.rotate(-bank * Math.PI / 180 * 0.6);
  g.fillStyle = '#7EB6FF'; g.fillRect(-W, -H * 1.5, W * 2, H * 1.5 + climb * 3);
  g.fillStyle = '#3C9E4A'; g.fillRect(-W, climb * 3, W * 2, H * 1.5); g.restore();
  // rings, far ones first
  rings.slice().sort(function (a, b) { return b.z - a.z; }).forEach(function (r) {
    var s = 1 / r.z, x = W / 2 + r.x * K * s, y = H / 2 + r.y * K * s, rad = 0.22 * K * s;
    g.strokeStyle = '#F2C94C'; g.lineWidth = Math.max(2, 14 * s); g.beginPath(); g.arc(x, y, rad, 0, Math.PI * 2); g.stroke();
  });
  // the plane
  g.save(); g.translate(W / 2 + plane.x * K, H / 2 + plane.y * K); g.rotate(bank * Math.PI / 180);
  g.fillStyle = '#DE544A'; g.fillRect(-60, -6, 120, 12); g.fillStyle = '#1B3A7B'; g.fillRect(-12, -22, 24, 44);
  g.fillStyle = '#DE544A'; g.fillRect(-22, 14, 44, 8); g.restore();
  g.fillStyle = '#fff'; g.font = 'bold 22px Arial'; g.textAlign = 'left'; g.fillText('Rings ' + (score || 0), 16, 32);
  g.textAlign = 'right'; g.fillText('Lives ' + (playing ? lives : 0) + '   Best ' + best, W - 16, 32);
  if (!playing) { g.textAlign = 'center'; g.font = 'bold 30px Arial'; g.fillText(score === undefined ? 'Press the button to take off' : 'Game over - press the button', W / 2, H / 2); }
  requestAnimationFrame(frame);
}
requestAnimationFrame(frame);
</script></body></html>)rawliteral";

void readSensor(float& ax, float& ay, float& az, float& gx, float& gy) {
  Wire.beginTransmission(MPU);
  Wire.write(0x3B);
  Wire.endTransmission(false);
  Wire.requestFrom(MPU, 14);
  int16_t v[7];
  for (int i = 0; i < 7; i++) {
    v[i] = Wire.read() << 8;
    v[i] |= Wire.read();
  }
  ax = v[0] / 16384.0; ay = v[1] / 16384.0; az = v[2] / 16384.0;
  gx = v[4] / 131.0; gy = v[5] / 131.0;
}

void calibrate() {                       // hold the stick still and level
  float sx = 0, sy = 0, sp = 0, sr = 0, ax, ay, az, gx, gy;
  for (int i = 0; i < 200; i++) {
    readSensor(ax, ay, az, gx, gy);
    sx += gx; sy += gy;
    sp += atan2(-ax, sqrt(ay * ay + az * az)) * 180 / PI;
    sr += atan2(ay, az) * 180 / PI;
    delay(3);
  }
  biasX = sx / 200; biasY = sy / 200;
  pitch = pitchZero = sp / 200;
  roll = rollZero = sr / 200;
}

void onSocket(uint8_t client, WStype_t type, uint8_t* payload, size_t length) {
  if (type != WStype_TEXT) return;
  String msg = String((char*)payload).substring(0, length);
  if (msg == "ring") tone(BUZZER_PIN, 1319, 80);
  else if (msg == "miss") tone(BUZZER_PIN, 196, 200);
  else if (msg == "over") tone(BUZZER_PIN, 131, 600);
}

void setup() {
  Serial.begin(115200);
  pinMode(BUTTON_PIN, INPUT_PULLUP);
  Wire.begin(21, 22);                    // SDA, SCL
  Wire.setClock(400000);
  Wire.beginTransmission(MPU);
  Wire.write(0x6B);                      // wake up
  Wire.write(0);
  Wire.endTransmission();
  delay(100);
  calibrate();
  WiFi.begin(WIFI_NAME, WIFI_PASSWORD);
  Serial.print("Connecting");
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  MDNS.begin("skyrings");                // http://skyrings.local
  server.on("/", []() { server.send_P(200, "text/html", PAGE); });
  server.begin();
  sockets.begin();
  sockets.onEvent(onSocket);
  Serial.print("\nOpen http://skyrings.local or http://");
  Serial.println(WiFi.localIP());
  lastMicros = micros();
}

void loop() {
  server.handleClient();
  sockets.loop();

  float ax, ay, az, gx, gy;
  readSensor(ax, ay, az, gx, gy);
  unsigned long now = micros();
  float dt = (now - lastMicros) / 1000000.0;
  lastMicros = now;
  float accPitch = atan2(-ax, sqrt(ay * ay + az * az)) * 180 / PI;
  float accRoll = atan2(ay, az) * 180 / PI;
  pitch = GYRO_SHARE * (pitch + (gy - biasY) * dt) + (1 - GYRO_SHARE) * accPitch;
  roll = GYRO_SHARE * (roll + (gx - biasX) * dt) + (1 - GYRO_SHARE) * accRoll;

  bool button = digitalRead(BUTTON_PIN);
  if (lastButton == HIGH && button == LOW) {   // each take-off also sets "level"
    pitchZero = pitch;
    rollZero = roll;
  }
  lastButton = button;

  if (millis() - lastSend >= 25) {       // 40 messages a second
    lastSend = millis();
    char msg[40];
    snprintf(msg, sizeof(msg), "%.1f,%.1f,%d", pitch - pitchZero, roll - rollZero, button == LOW);
    sockets.broadcastTXT(msg);
  }
  delay(5);
}

Fly!

1. Upload the sketch with the sensor lying still and level: it calibrates the gyroscope at start-up. Open the Serial Monitor at 115200 baud.
2. Open the address from the Serial Monitor on your phone or computer, on the same Wi-Fi: http://skyrings.local, or the number it shows, such as http://192.168.1.57.
3. Hold the stick level and press the button to take off. Each press also sets the current position as "level".
4. Tilt right to fly right and left to fly left. Tilt the front down to dive and up to climb. Fly through the golden rings; each ring you miss costs a life.
5. If your plane moves the wrong way, hold the sensor the other way round, or turn it a quarter turn on the ruler.

How it works

The angles come from the same complementary filter as in the tilt maze: the gyroscope gives smooth, fast changes, and the accelerometer slowly corrects the gyroscope's drift. On the ESP32 the loop runs about 150 times a second, and 40 times a second the latest angles go out over the WebSocket.

The game looks 3D, but it is drawn on a flat canvas with one trick: perspective. Every ring has a distance z. On screen, its position and size are divided by z: far rings (z = 12) are tiny and close to the middle, the horizon, and they grow and spread out as z shrinks. When a ring reaches z = 1, it is at the plane's distance, and the game checks whether the plane is inside it. The horizon rotates with the plane's bank and moves with the climb angle, which is enough to make it feel like flying.

Tilt controls a speed, not a position: banking 40 degrees moves the plane sideways at full speed, and holding level keeps it where it is. That's how real aircraft work, and it's why the button's "level" reset matters.

Try this next

• Add clouds that pass by, using the same perspective trick as the rings.
• Make some rings red: flying through them costs a life.
• Add a second player with another ESP32 and stick, and race each other's scores on the same page.
• Use the joystick instead of the motion sensor and compare: which controller feels more like flying?

Troubleshooting

• skyrings.local doesn't open: some phones don't understand .local names. Type the number from the Serial Monitor instead, with http:// in front.
• The plane drifts slowly to one side: the gyroscope was moved during calibration. Restart the ESP32 with the sensor lying still, then press the button to set level.
• The angles stay at 0: check SDA on GPIO 21 and SCL on GPIO 22, and VCC on 3V3.
• The game stutters: the Wi-Fi is weak. Move closer to the router, or use the ESP32 near the phone.
• It never connects to Wi-Fi: check the network name and password (both are case-sensitive) and make sure the network is 2.4 GHz.