Build a colour detector that learns colours from examples

Optional

This lesson has free STL files. Print the parts yourself with one of these 3D printers.

Build a colour detector that learns colours from examples – EEMENTOR STEM™ lesson

How does a machine learn what "red" is? Not from a rule written by a programmer, but from examples, the way you learned it as a toddler. In this lesson the UNO becomes a colour scanner. Its RGB LED shines red, then green, then blue on an object, and the light sensor measures how much of each comes back. That gives every object a fingerprint of three numbers. You teach it by putting an object under the hood and typing its colour name. After a few examples, it recognises new objects by finding the examples whose fingerprints are closest: that is the k-nearest-neighbour method, one of the oldest and simplest kinds of machine learning.

What you'll learn

Why a red object reflects mostly red light
How to turn an object into a fingerprint of numbers, called features
How the k-nearest-neighbour method classifies new things
Why more and better examples make a model more accurate

What you need

UNO R3 board and USB cable (UNO R3 Starter Kit)
RGB LED module (45-in-1 Sensor Kit)
3 resistors, 220 Ω (UNO R3 Starter Kit)
Photoresistor module (45-in-1 Sensor Kit)
A few coloured objects: paper, sticky notes, building bricks
The printed parts from the 3D printing box below (colour scanner hood), or the cardboard version described there
Breadboard and jumper wires (UNO R3 Starter Kit)

3D printing · free STL files

Print the colour scanner hood

A dark hood that holds the RGB LED and the light sensor side by side, pointing down at the object, and keeps the room light out. Objects slide in under it through a slot.

⬇ Scanner hood STL · 70 × 56 × 39 mm

Print settings and how to put it together

Print settings: Black or dark PLA, so no light leaks through, 0.2 mm layers, 3 walls, 20 % infill. Print it the way it lies in the file. No supports.

Push the RGB LED module and the photoresistor module into the two pockets in the roof, both pointing down.
Put the hood on the table. Slide a coloured object, like a sticky note, a LEGO brick or a sweet, under it through the slot.
Lead the wires out through the gap at the back.

No 3D printer? Tape the two modules next to each other inside a small black box with one open side.

Wiring

From	To
UNO pin 9	220 Ω resistor, then RGB module R
UNO pin 10	220 Ω resistor, then RGB module G
UNO pin 11	220 Ω resistor, then RGB module B
RGB module −	GND
Photoresistor module S	UNO pin A0
Photoresistor module + and −	5V and GND

Sensor modules from different makers can label and order their pins differently. Always follow the labels printed on your module.

The sketch

const int RED_PIN = 9, GREEN_PIN = 10, BLUE_PIN = 11, SENSOR_PIN = A0;
const int MAX_SAMPLES = 30, MAX_COLOURS = 6, K = 3;

float samples[MAX_SAMPLES][3];       // fingerprints you taught it
byte sampleColour[MAX_SAMPLES];      // which colour each one is
String colourNames[MAX_COLOURS];
int sampleCount = 0, colourCount = 0;

int lightLevel() {
  delay(60);                         // let the LED and sensor settle
  long sum = 0;
  for (int i = 0; i < 8; i++) sum += analogRead(SENSOR_PIN);
  return sum / 8;
}

void led(int r, int g, int b) {
  analogWrite(RED_PIN, r);
  analogWrite(GREEN_PIN, g);
  analogWrite(BLUE_PIN, b);
}

// The fingerprint: how much red, green and blue comes back, as shares of the total
void scan(float f[3]) {
  led(0, 0, 0);
  int dark = lightLevel();                       // room light, to subtract
  led(255, 0, 0); float r = abs(lightLevel() - dark);
  led(0, 255, 0); float g = abs(lightLevel() - dark);
  led(0, 0, 255); float b = abs(lightLevel() - dark);
  led(0, 0, 0);
  float total = r + g + b + 1;
  f[0] = r / total * 100;
  f[1] = g / total * 100;
  f[2] = b / total * 100;
}

float distance(float a[3], float b[3]) {
  float d = 0;
  for (int i = 0; i < 3; i++) d += (a[i] - b[i]) * (a[i] - b[i]);
  return sqrt(d);
}

// k-nearest neighbours: find the K closest examples and let them vote
int classify(float f[3], int& votes) {
  int nearest[K];
  float best[K];
  for (int k = 0; k < K; k++) { nearest[k] = -1; best[k] = 1e9; }
  for (int i = 0; i < sampleCount; i++) {
    float d = distance(f, samples[i]);
    for (int k = 0; k < K; k++) {
      if (d < best[k]) {                        // insert into the top-K list
        for (int m = K - 1; m > k; m--) { best[m] = best[m - 1]; nearest[m] = nearest[m - 1]; }
        best[k] = d;
        nearest[k] = i;
        break;
      }
    }
  }
  int tally[MAX_COLOURS] = {0};
  for (int k = 0; k < K; k++) if (nearest[k] >= 0) tally[sampleColour[nearest[k]]]++;
  int winner = 0;
  for (int c = 1; c < colourCount; c++) if (tally[c] > tally[winner]) winner = c;
  votes = tally[winner];
  return winner;
}

void teach(String name) {
  int c = -1;
  for (int i = 0; i < colourCount; i++) if (colourNames[i] == name) c = i;
  if (c < 0) {
    if (colourCount == MAX_COLOURS) { Serial.println("I know enough colours already."); return; }
    c = colourCount++;
    colourNames[c] = name;
  }
  if (sampleCount == MAX_SAMPLES) { Serial.println("My memory is full. Type forget to start again."); return; }
  scan(samples[sampleCount]);
  sampleColour[sampleCount] = c;
  sampleCount++;
  Serial.print("Learned an example of ");
  Serial.print(name);
  Serial.print(". I know ");
  Serial.print(sampleCount);
  Serial.println(" examples now.");
}

void setup() {
  Serial.begin(9600);
  pinMode(RED_PIN, OUTPUT);
  pinMode(GREEN_PIN, OUTPUT);
  pinMode(BLUE_PIN, OUTPUT);
  Serial.println("Put an object under the hood, type its colour (like red) and press Enter.");
}

void loop() {
  if (Serial.available()) {
    String text = Serial.readStringUntil('\n');
    text.trim();
    text.toLowerCase();
    if (text == "forget") {
      sampleCount = colourCount = 0;
      Serial.println("Forgotten everything.");
    } else if (text.length() > 0) {
      teach(text);
    }
  }
  static unsigned long lastGuess = 0;
  if (sampleCount > 0 && millis() - lastGuess > 3000) {   // guess every 3 seconds
    lastGuess = millis();
    float f[3];
    scan(f);
    int votes;
    int c = classify(f, votes);
    Serial.print("I see R ");
    Serial.print(f[0], 0);
    Serial.print(" G ");
    Serial.print(f[1], 0);
    Serial.print(" B ");
    Serial.print(f[2], 0);
    Serial.print("  ->  ");
    Serial.print(colourNames[c]);
    Serial.print(" (");
    Serial.print(votes);
    Serial.print(" of ");
    Serial.print(min(K, sampleCount));
    Serial.println(" neighbours agree)");
  }
}

Teach it colours

Open the Serial Monitor at 9600 baud, line ending Newline.
Put a red object under the hood, type red and press Enter. Move it a little and teach it again: three examples per colour work well.
Teach two or three more colours, like green, yellow and blue.
Now put a new object under the hood: every three seconds the UNO says which colour it thinks it is, and how many neighbours agree. Type forget to start again.

How it works

A red object looks red because it reflects red light and swallows most of the green and blue. So when the LED shines red, a lot comes back; with green or blue, much less. The sketch measures all three, subtracts the room light, and divides each by the total. Those three shares are the object's features. Dividing by the total means a red object counts as red whether it is close to the sensor or far away.

To classify a new object, the sketch measures its distance to every example it knows, as if the three numbers were a position in a room, and picks the K = 3 nearest. They vote, and the colour with the most votes wins. There is no training step at all: the examples are the model. That makes k-nearest-neighbour easy to understand, but it gets slower the more examples it has to compare.

Two colours that reflect light in a similar way, like orange and red, are hard to tell apart. More examples, and examples from the edges of each colour, help. That is true for every machine learning model.

Try this next

Light the RGB LED in the colour it recognised, so you can see its answer.
Use it to sort sweets: add a servo that tips each one into the right cup.
Try K = 1 and K = 5. Which gets more right?

Troubleshooting

Every colour looks the same: the hood lets in too much light. Put it on a dark surface and check that the LED and the sensor point at the object.
It mixes up two colours: teach more examples of both, and check that the shares in "I see R G B" really differ.
"My memory is full": the UNO has only 2 KB of memory. Type forget and teach fewer examples.
A printed part is too tight or too loose: printers differ by a few tenths of a millimetre. Sand or file a tight hole, and fix a loose part with a drop of hot glue or a strip of tape.