Twitter Reflecting Christmas Tree Lights Controller

Posted by barnoid Sun, 22 Dec 2013 14:36:00 GMT

Twitter Reflecting Christmas Tree Lights Controller
A project to make a string of Christmas tree lights that reflect the current state of Twitter’s opinion about Christmas. More here:
ModelCanon EOS 5D Mark II
Exposure0.167 sec (1/6)
Exposure ProgramProgram AE
ISO Speed200
Date and Time (Original)2013:12:21 21:30:55
Exposure Bias-2/3 EV
FlashOff, Did not fire
Focal Length60 mm
Metering ModeCenter-weighted average
Lens TypeCanon EF 24-105mm f/4L IS
Camera Temperature35 C

Christmas Twitter Tree Lights 2

Posted by barnoid Sun, 22 Dec 2013 16:05:00 GMT

I recently bought a string of 50 individually addressable RGB LEDs from Embedded Adventures and so had to decide what to do with them. I decided it would be festive if they could reflect Twitter's current feeling about Christmas.

So I started Googling sentiment analysis and quickly discovered that Stanford University's Natural Language Processing Group has released the source and data for their sentiment analyser. When fed a sentence it outputs one of "Very negative", "Negative", "Neutral", "Positive" or "Very positive". It's pretty good but it's trained on movie reviews so it's not quite so accurate on tweets about Christmas. For example "Christmas time is defo the best time of the year !!" is classed as very negative.

The sentiment analysis is fairly slow. Much slower than the rate that the tweets come in, so I'm using beanstalkd for queueing. The tweets come in from the Twitter streaming API (filtering for "christmas" and "xmas"), get cleaned up, @usernames removed, hashes removed from hashtags, URLs deleted, then put in a queue. My desktop computer is the fastest in the house so it runs four instances of the sentiment analysis, pulling tweets out of the queue, processing them, pre-pending one of N, n, -, p or P and sticking the result in a second queue.

Raspberry Pi controlling the lights.
Raspberry Pi controlling the lights.

The lights themselves are controlled by a Raspberry Pi. They are connected to the SPI pins via a level shifter from Sparkfun. The pins on the Pi all work at 3.3V while the lights need 5V hence the shifter is needed. I'm using a fairly recent Raspian. To enable the SPI device /dev/spidev0.0 I removed the "blacklist spi_bcm2708" from /etc/modprobe.d/raspi-blacklist.conf. Then I just need to write a byte each for red, green and blue for each LED to the device file.

So the code on the Pi reads from the second queue, and shifts an LED on to the chain with a colour based on the initial sentiment letter: red for negative, green for positive, white for neutral, bright cyan for very positive and bright orange for very negative. The sentiment analysis processing can't keep up with the rate of incoming tweets so I've had it skip every other tweet.

The result is quite a pretty light display, though it's hard to say whether or not it's successful in reflecting the opinion on Twitter of Christmas.

The Code

This is the code that streams tweets from Twitter, cleans them up and puts them in a queue. It uses the tweetstream and beaneater gems.


require 'rubygems'
require 'tweetstream'
require 'beaneater'

TweetStream.configure do |config|
        config.consumer_key       = "SECRET"
        config.consumer_secret    = "SECRET"
        config.oauth_token        = "SECRET"
        config.oauth_token_secret = "SECRET"
        config.auth_method        = :oauth

beanstalk =['localhost:11300'])
tube = beanstalk.tubes["tweets-in"]

count = 0'xmas', 'christmas') do |status|
        tweet = status.text
        tweet.gsub!(/@.+?($|\s)/, "")
        tweet.gsub!(/^RT /, "")
        tweet.gsub!(/https?:.+?($|\s)/, "")
        tweet.gsub!(/[^\w\d \(\),\.\?\!\-\'\:\$\&\;\+]/, "")
        tweet.gsub!(/&/, "and")
        tweet.gsub!(/&.+?;/, "")
        if count % 2 == 0 then
        count += 1


This is the code that pulls tweets out of one queue, feeds them to the sentiment analyser and puts them in another queue with the result.


require 'rubygems'
require 'beaneater'

beanstalk =[''])
tube_in = beanstalk.tubes["tweets-in"]
tube_out = beanstalk.tubes["tweets-out"]

IO.popen("./run_sentiment", "r+") do |sent_io|
        # Wait until the sentiment analyser has printed its startup text
        while not sent_io.readline.match(/EOF/)

        while true do
                tweet_job = tube_in.reserve
                tweet = tweet_job.body
                sent_io.puts tweet
                sentiment = sent_io.readline.chomp
                case sentiment
                when "  Very positive"
                        sentiment = "P"
                when "  Positive"
                        sentiment = "p"
                when "  Neutral"
                        sentiment = "-"
                when "  Negative"
                        sentiment = "n"
                when "  Very negative"
                        sentiment = "N"
                puts "#{sentiment} #{tweet}"
                tube_out.put("#{sentiment} #{tweet}")


This starts the sentiment analyser telling it to accept input on STDIN. This is referenced as run_sentiment above.


cd stanford-corenlp-full-2013-11-12
java -cp "*" -mx5g edu.stanford.nlp.sentiment.SentimentPipeline -stdin 2>&1

This code runs on the Pi, controlling the lights.


require 'beaneater'

class Led

        attr_accessor :r, :g, :b

        def initialize(r = 0, g = 0, b = 0)
                @r = r
                @g = g
                @b = b

        def out
                return "#{@r.chr}#{@g.chr}#{@b.chr}"

        def to_s
                return "%02X%02X%02X" % [ @r, @g, @b ]


LED_V_POS =,128,255)
LED_POS =,64,0)
LED_NEG =,0,0)
LED_V_NEG =,128,0)

beanstalk =[''])
tube = beanstalk.tubes['tweets-out']

leds = []
50.times do
end"/dev/spidev0.0", "w") do |spi|

        while true do

                job = tube.reserve
                s = job.body[0]
                case s
                when 'P'
                        leds.unshift LED_V_POS
                when 'p'
                        leds.unshift LED_POS
                when '-'
                        leds.unshift LED_NEUTRAL
                when 'n'
                        leds.unshift LED_NEG
                when 'N'
                        leds.unshift LED_V_NEG
                #p leds

                50.times do |k|
                        spi.write leds[k].out



Twitter Reflecting Christmas Tree Lights

Posted by barnoid Sat, 21 Dec 2013 21:46:20 GMT

A project to make a string of Christmas tree lights that reflect the current state of Twitter’s opinion about Christmas. More here:

TwitBeeb 3

Posted by barnoid Thu, 22 Oct 2015 11:39:00 GMT

TwitBeeb on test at Bristol Hackspace, photo by John Honniball.

TwitBeeb is a BBC B microcomputer (vintage 1981) from which you can Tweet.

The Beeb itself was my main computer until 1994, I taught myself to program on it. A few years ago I pulled a rather scuffed up BBC out of a skip at Sussex University and took it home. Inside it had an add-on ROM board with several programs in ROM chips including a serial terminal emulator called Termulator. This allows it to connect to other computers via its serial port and for everything typed on the BBC's keyboard to be sent to the other computer which sends responses back to be displayed on the BBC's screen. It acts as what used to be called a "dumb terminal", just handling the input and output while a more powerful computer does the actual work. This was quite a common way of doing things back when computers were room-sized. Several terminals would connect to a large shared computer, possibly over phone lines using modems.

My first tests with the Beeb terminal involved connecting it to a Linux server using a cable I made from this recipe. It was a bit tricky to find 5 pin domino DIN plugs and note the warning about marking the top because you can plug them in either way up. I ran a text-based web browser called Lynx on the server via the BBC terminal and pointed it at Twitter. Termulator was not very good at handling the full Linux terminal and browser, it tended to crash and you can't use the arrow keys which makes using Lynx awkward. I did manage to send one tweet.

More recently, I got my hands on a Raspberry Pi. While considering what to do with it I remembered my BBC terminal experiments. Given that the Pi is cast from the same mould as the BBC, in terms of their educational intentions, they seem like an obvious pairing. They share a common heritage, in that the Raspberry Pi's ARM processor was designed by some of the same team who worked on the BBC Micro. I also enjoy the fact that in this case the terminal is physically far bigger than the vastly more powerful computer it connects to.

Some spec comparisons:

BBC Micro (Model B)Raspberry Pi (Model B)
Release Date1st December 198129th February 2012
CPU6502 2 MHz (8 bit)ARM1156 700 MHz (32 bit)
RAM32 KB256 MB (262144 KB)
StorageNone in this instance, except the 16KB ROMs containing the OS and terminal software.2 GB SD card

Twitter BBC early experiment.
The first experiment with the BBC B and Twitter via Lynx.

So due to Termulator's problems handling a full text-mode browser and because that would have been too complicated for use on the stand at an event like a Makerfaire I wrote my own specialised kind of Twitter client. The BBC's OS has a set of special bytes that you can send to control things like text colour and cursor position, I used these to make it pretty. It reminded me of the kind of programs that I used to write when I was 11, it was kind of odd to be doing that stuff in Ruby rather than BASIC.

Here is the code, on Github. It searches for a hashtag and displays some results on the screen below the title. At the bottom of the screen it shows a prompt very much like the BASIC one, you type your tweet there and press Return to send it. The hashtag is appended for you. The code stops you typing too many characters.

For TwitBeeb's first public outing at Derby Mini Makerfaire I had the Pi talking to the Internet using a USB Wifi dongle. I set up the network by connecting to the Pi's serial console from my laptop using a USB to serial converter and a Sparkfun level shifter. When the network was working I connected to the Pi with ssh and moved the USB to serial converter to the Pi and connected the serial end to the BBC (minus the level shifter). I then started Termulator on the Beeb, put it in BBC VDU mode and started the twitbeeb script on the Pi. I used the command line to redirect standard in and out for the script to the serial port device, having first set it to run at 4800 baud and several other parameters using stty.

It was quite popular at the Faire, mostly as a spectacle though. People were reluctant to come up with a witty tweet on the spot. It tweets from the @twitbeeb account, you can see people's tweets there. Most confusion came when there was a need to find the "delete" key, and also "@" hidden in plain view on a key of its own.



Since its public debut at the first Derby Mini Maker Faire in 2012, TwitBeeb has been on show several times. I'm about to show it again at the fourth Derby Mini Maker Faire.

Here are some of its outings:

TwitBeeb at Brighton Mini Maker Faire 2012
TwitBeeb at Brighton Mini Maker Faire 2012, photo by Hollie Lubbock.

Bristol Raspberry Jam - August 2012

I set it up in the demo room with a bunch of other people's projects, then went off to watch some talks. About 20 minutes later someone came over to tell me that my project was on fire. It turns out the capacitor in the BBC's power supply had failed, as they will do after 30 years. Those capacitors are partly made of paper so they make a lot of smoke when they fail. It didn't get used much that day.

Brighton Mini Maker Faire - September 2012

Having replaced all the power supply capacitors, I showed Twitbeeb on the Bristol Hackspace stand at Brighton Mini Maker Faire 2012.

Bristol Mini Maker Faire - March 2013

The next outing was the first Bristol Mini Maker Faire, with a few tweaks to the code to make it compatible with Ruby 1.9.

This happened:

TwitBeeb and Eben Upton at Bristol Mini Maker Faire 2013
TwitBeeb and Eben Upton at Bristol Mini Maker Faire 2013.

Eben and Liz Upton of Raspberry Pi fame were the special guests at the Faire. I had a brief chat with Eben and he seemed to like TwitBeeb.

Bristol Hackspace for BV Open Studios - June 2013

Between the last show and this one Twitter decided to close down their old API. So I had to scramble to basically rewrite my API code the day before this event.

Derby Mini Maker Faire - October 2014

I implemented some new features for this outing. When idle it now automatically reloads the tweets (previously someone had to press Escape to trigger this) and it cycles between tweets, a screen of self description and the Derby Mini Maker Faire logo.

Bristol Mini Maker Faire - August 2015

On the Bristol Hackspace stand again at the second Bristol Mini Maker Faire. Swindon Hackspace had a lovely Teletype typing out tweets nearby, so people were able able to use TwitBeeb to send it messages.

Oh really?!?

There's more!

David James Lennon has created a TwitBeeb twin using my code from Github! He will be showing it this Saturday (24th October 2015) at the RISC OS London show, which happens to be the same day the original TwitBeeb will be at Derby Mini Maker Faire.

His TwitBeeb tweets to @burrBeep. I've added a new tweet search page to the rotation that displays the result of a search for that Twitter handle. Hopefully we'll be able to get some inter-show chats going.

I've added a few other nice features, including word wrapping for the tweet list. When I implemented word wrapping for Ode, my terrible poetry generator that I wrote on my BBC (the same one) when I was about 13, it consisted of about 120 horrible lines of BASIC. This time it's basically just one regular expression. What a difference 20odd years experience and a computer science degree can make!

Taking TwitBeeb to shows has become easier over time as I've gradually improved its reliability. Nowadays it's pretty much at the ideal point where once it's connected to the network it can be left to its own devices. It's nice to chat to people about it still though. What happens often at a Maker Faire is that a parent of about my age with small children in tow will spot TwitBeeb and say "Look kids! This is the computer I used at school!". The children will look nonplussed and may attempt to poke and swipe things on the screen before mashing the keyboard. Meanwhile the parent realises that it's actually showing real live twitter and asks how it works.

I expect I'll continue to take TwitBeeb to events, as long as the Twitter API keeps working at least. I'd like to take it to the big Maker Faire in Newcastle next year, and maybe to EMF Camp 2016 as well, if I can keep it dry.

BBC B Tweeting Machine

Posted by barnoid Tue, 23 Aug 2011 20:42:05 GMT

BBC B Tweeting Machine
My BBC B running serial terminal software called Termulator, connected to my Linux desktop which is running the text only web browser Lynx to use Twitter. The terminal software is fairly temperamental, certain codes seem to make everything freeze up. Also, it’s hard to find 5 pin DIN plugs with the right pin configuration these days.
ModelCanon EOS 5D Mark II
Exposure0.025 sec (1/40)
Exposure ProgramProgram AE
ISO Speed500
Date and Time (Original)2011:08:23 17:59:52
Exposure Bias-2/3 EV
FlashOff, Did not fire
Focal Length35 mm
Metering ModeEvaluative
Lens TypeCanon EF 24-105mm f/4L IS
Camera Temperature18 C