KangarooBot/Spring/2021

ESP32Cam QR Code Detection & Edge Detection

Author/s: Andre Troncoso

Table of Contents

Introduction

The ESP32-cam is a microcontroller that has a OV2640 camera that is built-in. It has 4 MB of PSRAM which is great for a small computer vision application. Essentially, the KangarooBot utilized the ESP32-cam for scanning of QR codes and the detection of the maze side walls. Additionally, the error of the robot between the left and right wall is calculated by the ESP32-cam and sent to the robot using a UART protocol set at 9600 baud rate and 8N1 serial format. This error is then translated to the control system of the motors where the appropriate speeds are applied to deter the robot from collision.

QR Code Detection

Figure 1 – Card Prototype

#ifndef CARD_H_
#define CARD_H_
#include
namespace Card{
    static std::string KEY_STR = “key”;
    static std::string EXIT_STR = “exit”;
    static std::string NO_CARD_STR = “no card”;
    enum card_t{
        KEY,
        EXIT,
        NO_CARD
    };
    card_t stringToCardEnum(const std::string& str){
        if(str.compare(KEY_STR) == 0){
            return KEY;
        }else if(str.compare(EXIT_STR) == 0){
            return EXIT;
        }else{
            return NO_CARD;
        }
    }
    std::string& cardEnumToString(const card_t card){
        switch(card){
            case KEY:
                return KEY_STR;
            case EXIT:
                return EXIT_STR;
            default:
                return NO_CARD_STR;
        }
    }
    static void init();
    static void update();
    static card_t scanCard();
}
#endif
Description: The figure shows the prototype code for a card. Notice that card is defined as an enumerated class which is typical representation for various objects. This enumeration is mapped with std::strings that can be converted interchangeably. It is recommended that enumerated class instances can be mapped with std::string for debugging purposes with a console.
Figure 2 – Transmission of QR Code Data using UART from the ESP32 end
static void onQrCodeTask(void* pv_parameters){
  while(true){
    struct QRCodeData qr_code_data;
    if(READER.receiveQrCode(&qr_code_data, SCAN_DELAY)){
      bool is_qr_code_data_valid = (bool)qr_code_data.valid;
      if(is_qr_code_data_valid){
        const char* char_ptr = (const char*)qr_code_data.payload;
        std::string str = std::string(char_ptr);
        Card::card_t card = Card::stringToCardEnum(str);
        Serial.println(char_ptr);
        if(card == Card::KEY){
          digitalWrite(CARD_LED_PIN, HIGH);
        }
        Serial1.write((uint8_t)card);
      }
    }
    vTaskDelay(V_TASK_DELAY);
  }
}
Description: The figure shows a function that is called during ISR routine from the perspective ESP32 microcontroller. The payload is UTF-8 bit string that is decoded using Alvaro Viebrantz ESP32QRCodeReader library. The payload is mapped with an enumerated value which is sent through the serial 1 port that is open on pins 12 and 13.

Edge Detection

Figure 1 – Vision Frame

Description: This figure refers to the gray scale frame indicated by the instructor.

Figure 2 –  Error Calculation

Error Calculation Code

static int8_t pixelError(uint8_t* const buffer, const int width, const int height){
  const int a = width / 2;
  int i;
  int j;
  int left_hand_pixels = 0;
  int right_hand_pixels = 0;
  int8_t pixel_error;
  uint8_t gray_scale_val;
  for(i = 0; i < a; i++){
    for(j = 0; j < height; j++){
      gray_scale_val = buffer[i + width * j];
      if(gray_scale_val < UPPER_BOUND_GRAY_SCALE && gray_scale_val > LOWER_BOUND_GRAY_SCALE){
        left_hand_pixels = i;
      }
    }
  }
  for(i = a; i < width; i++){
    for(j = 0; j < height; j++){
      gray_scale_val = buffer[i + width * j];
      if(gray_scale_val < UPPER_BOUND_GRAY_SCALE && gray_scale_val > LOWER_BOUND_GRAY_SCALE){
        right_hand_pixels = width – i;
      }
    }
  }
  pixel_error = (int8_t)(right_hand_pixels – left_hand_pixels);
  return pixel_error;
}

Description: The figure shows the side wall error calculation using the difference of two horizontal pixel distances. The error is later fed into a P-controller for motor control.

Motor Control

static void Navigation::pathFollow(){
    Navigation::PIXEL_ERROR = Serial1.read();
    Navigation::Y = (double)PIXEL_ERROR / 127 * K;
    if(Navigation::Y > 0){
        Navigation::turnLeft();
    }else if(Navigation::Y < 0){
        Navigation::turnRight();
    }else{
        Navigation::forward();
    }
}
Description: If Y happens to negative then it will turn right for Y milliseconds other wise turn left for Y milliseconds.

Conclusion

The most difficult task for this part of the project is testing/calibrating the camera sensor. Moreover, finding the correct grayscale threshold values that match with the green hue walls of the maze was challenging. The IDE that made the integration of several different API possible is VSCode with the PlatformIO extension.

References/Resources

  1. ESP32QRCodeReader: https://github.com/alvarowolfx
  2. Introduction to Edge Detection | What is Edge Detection: https://www.mygreatlearning.com/blog/introduction-to-edge-detection/
  3. Professor Hill Sketch of an Error Calculation
  4. Proportional-only Control: https://control.com/textbook/closed-loop-control/proportional-only-control/