Point the Seeing Wand, push the button, and it will tell you what it sees. This fun project can be built over a few hours and has a practical side for the vision impaired. Yes, "there is an app for that," but the form factor is much cooler and the cost less than a smartphone as it is based on the $10 Raspberry Pi Zero W.

This project came about as Hobbes wanted to teach his kids about #innovation. They had already done their own fair share through STEM clubs, FIRST Lego League, and just playing with cardboard and Legos at home. At 12 and 14, they even presented at the 2018 IEEE STEM conference held at Princeton University about their FLL projects and experience (yup, proud dad!). The Seeing Wand was intended to push the boundaries of what they thought could be accomplished with relatively few resources in a short timeframe through #integration.

The Seeing Wand works by taking a picture of where it is pointed at when a button is pushed, sending the image to Microsoft's Cognitive Services to get a description, then speaking the description using the espeak text-to-speech (TTS) software. The camera used is Raspberry Pi's Camera Module v2, and the speaker is Pimoroni's SpeakerPhat. These are wired and packaged inside a tube with two momentary push buttons and a power source. For the example shown, we used a scrap piece of PVC and a 2200mAh power cube device charger with a 5V/1A output that was a conference tchotchke.

If you are new to the Raspberry Pi, install the Raspbian operating system using NOOBS and configure WiFi.1 Make sure the Pi is working, then shut it down. It is best to do a shutdown than removing power from the Pi to avoid corrupting the SD card.

Next, assemble the SpeakerPhat, carefully following Pimoroni's instructions.2 Make sure to get a good solder on the speaker contacts as otherwise audio will be weak or non-existant.

And in Issue 72 of MagPi on Artificial Intelligence:
MagPi cover
Featured in Issue 71 of MagPi – the official Raspberry Pi magazine:
MagPi cover

If soldering the speaker wires was more challenging than you like, you can avoid having to solder anything else by using the Pimoroni GPIO Hammer Headers. This kit includes male and female headers, and an installation jig. Follow the fitting guide for the headers using care not to bend the pins.3 We used the male header on the Pi and the female header on the SpeakerPhat. Instead of connecting the headers directly, though, we used male/female jumper wires to make the connection between them so as to more easily wire up the buttons as well.

Connect the following pins between the Pi and SpeakerPhat: 2 (5V), 12 (BCM18), 17 (3V3), 35 (BCM19), 39 (Ground), 40 (BCM21). Although the SpeakerPhat pinout 4 also lists pins 3 (BCM2/SDA) and 5 (BCM3/SCL), we did not use these as they are only for powering the LEDs on the SpeakerPhat, and we will use pin 5 for one of the buttons.

The first momentary-touch push button is connected to pins 5 (BCM3) and 6 (Ground), and the second button to pins 13 (BCM27) and 14 (Ground). When pins 5 and 6 are connected together, the Pi automatically powers on, so we will use these for our on/off button. The buttons may be wired directly or through a breadboard.

The camera connects to the Pi using a ribbon cable. There are both short and long cables available. We used a short cable because of the way we fit everything into the tube as discussed later.

At this point, everything should be wired together and it may be best to setup the Seeing Wand software and test everything out before fitting it into the tube. Connect the Pi to a power source and it will start booting up. Once boot up, open up a terminal window and install the software.

The SpeakerPhat software is installed with:

For the security conscious, you may want to download and review the file before sending it to the shell. Once installed, you will find a Pimoroni directory with some test scripts for making sure the speaker works.

The espeak TTS software is installed with:

The camera software should already be in the base Raspbian. If not, install the desired version:

The Seeing Wand software is available on GitHub.5 Create a sub-directory in your home directory, download the files into it, and set the .py and .sh files to be executable:

Once downloaded, run the setup script to configure the Pi to autostart the button triggers:

In order to use Microsoft Cognitive Services, you will need to get an API key.6 This is freely available for a limited number of requests. Once you have a key, edit the file SeeingWand.py in the SeeingWand directory and enter it where you see {MS_API_KEY}. Although not perfect, the Microsoft Cognitive Service Vision API worked remarkably well, and when it didn't, we had a good laugh at what it described seeing.

Now, point, push and listen. With everything working, shutdown the Pi by pressing the on/off button. Once the Pi light goes out, unplug the Pi and begin assembling the wand.

As we took the solderless approach, a 6" length of 2" PVC was required. The camera conveniently fit into a closet rod supporting end cap, which has a screw hole that the lens fit up against, with a piece of 1" rigid insulation cut to tightly hold the camera in place inside the cap. A PVC end cap with a hole drilled through for the lens may work just as well. The power cube came with a short USB cable, however we had to cut out some of the plastic around the cable connector on the Pi end to allow it to bend inside the tube; if you do this, use extra care not to cut into the wires. All the parts were gently put into the tube, with the camera at one end and the power cube at the other with its USB connectors facing out so to allow easy (un)plugging of the Pi and charging cable. The wires to the buttons on the breadboard ran through the bottom (power cube) end. The camera cap, power cube, and breadboard with buttons, were all held in place with a single rubber band.

That's it – Have fun!

References

  1. https://www.raspberrypi.org/learning/software-guide/
  2. https://learn.pimoroni.com/tutorial/sandyj/assembling-speaker-phat
  3. https://learn.pimoroni.com/tutorial/sandyj/fitting-hammer-headers
  4. https://pinout.xyz/pinout/speaker_phat
  5. https://github.com/hobbez/SeeingWand
  6. https://azure.microsoft.com/en-us/services/cognitive-services/computer-vision/

Parts List

Pinouts

Photos