Ambient Poematron

Ambient Poematron is a small box with a three-colour e-paper display that shows a newly generated poem every five minutes. If you like the current poem you can unplug it and it will stick around.

The hardware consists of a Raspberry Pi Zero with a Waveshare black-white-red 2.13 inch e-paper display hat in a case designed for that combo made by Pi Hut.

The poems are generated using Conceptnet. I briefly describe Conceptnet in my post about Not Your Bot. I’d tried a few experiments using Conceptnet to make similes and metaphors with a plan to improve the poetry of Poetoid Lyricam. But Lyricam has a few tricky requirements: the poetry generating must be fast, it’s already doing time consuming AI stuff and the wait from pressing the button to poem printout is critical for not boring people; it must be self-contained, Lyricam has no internet connection so can’t call the Conceptnet API; and it must be memory efficient, Lyricam runs on a Pi 3B+ for power efficiency so it is limited to 1GB and the AI stuff needs a lot of that.

ConceptNet Conversion

You can download all of Conceptnet’s edges in the form of a huge (34 million line) TSV file. To allow me to quickly look up all the terms related to a particular word I wrote a program to loop over each line of the file, filtering them for edges where both ends are in English, creating files for each term that starts an edge, then adding lines to the files with each relationship and edge ending term.

For example these are some lines that can be found in the file for “cat”:

Antonym dog
AtLocation best_chair_by_fire
AtLocation many_peoples_homes
CapableOf eat_mice
CapableOf fancy_certain_foods_over_others
CapableOf kill_bird
CapableOf meow
Desires eat
HasA four_legs
HasA sharp_claws
HasProperty difficult_to_wash_in_bath
IsA mammal
IsA much_loved_pet
NotDesires chased_by_dog
RelatedTo person
RelatedTo monkey
RelatedTo trickery
RelatedTo domesticated_animal
RelatedTo retroreflective
UsedFor companion_to_humans

This resulted in 925,110 files totalling 11GB, which I grouped in directories by first letter. But why not just use some kind of database for this? While this limits the ways I can query the data, a database engine is likely to consume a big chunk of RAM that I can’t afford, but I can use a nice big SD card. The operating system is well optimised for dealing with large numbers of files.

I can now generate sort of similes for words by opening that word’s file, selecting a random RelatedTo entry, opening that entry’s file and selecting something from there. A HasProperty is usually good.

For example, starting with “cat” as above. Selecting “monkey” and opening the “monkey” file we find the HasProperty “hanging_off_tree”. Resulting in a simile for “cat”: “as hanging off tree as a monkey”.

They’re not going to be great similes but as with all my poetry generating, I invoke poetic licence.

I made Ambient Poematron as a present for my friend Libby’s birthday, so the poems start from a small list of things she likes. These are stored in a file one per line along with an emoji representing them. To generate a poem a random line is picked from the file and the word is sent to one of two programs that generate different poetic forms.

Poem Form 1

This one is an affectionate tribute to the selected thing. It always starts with “O {thing}!“, followed by a simile generated as described above.

Then two lines based on entries in the {thing} file, using these templates:

• “You’re such {IsA}“
• “You’re no {Antonym}“
• “I saw you in the {AtLocation}“
• “I know you can {CapableOf}“
• “You can {CapableOf} so well”
• “Your {HasA} is so [fine / lovely / exquisite]“
• “I would give you the {Desires} you so desire”
• “I would never make you {NotDesires}“
• “You make me want to {CausesDesire}“
• “You are so often {ReceivesAction}“
• “I admire your quality of {HasProperty}“

Finally a line that starts either “Puts me in mind of” or “You make me think of” followed by three RelatedTo items from the {thing} file.

An example:

O cat!

As good to ship things in as a box

I admire your quality of difficult to wash in bath

I know you can sleep on windowsill

Puts me in mind of lion and plate and another

Poem Form 2

This one depends more on colourful metaphors. Starting with the selected thing it follows RelatedTo and IsA links through the network a maximum of three steps. So first it opens the {thing} file, gathers its contents and selects a random RelatedTo or IsA entry, then it opens that file and repeats the process, keeping a cache of the items it’s visited and the entries for them. It then builds a list of all the IsA items it has in the cache, for later convenience.

To generate the poem it starts with “{thing}:” then the list of visited items each preceded by “like”. Comparing each to the previous in the chain.

Some examples:

Cat: like lion; like stylized; like realistic.

Cat: like ring; like shape; like description.

Cat: like surface; like aspect; like aspect ratio.

That’s followed by two lines that ask whether the original thing is either of two things from the IsA list. Sometimes they obviously relate to the original thing, sometimes to one of the other list entries. Often you can’t tell what they relate to.

Finally the poem ends with one of “I don’t know”, “I can’t say”, “Who knows?”, “Nobody knows”, or “Who can say?”.

An example:

Cat: like two; like after; like before.

A cat is a felis.

Or a cat is a pawn?

Nobody knows

Another:

Cat: like water; like density; like function.

A cat is an opaque gem.

Or a cat is a common name for h2o?

Who knows?

E-Paper Displaying

Now I just need to get the poem displayed on the e-paper, should be straightforward. No, this was by far the hardest bit.

The e-paper expects two 1-bit bitmaps the size of its resolution, one for the black part and one for the red part. I wanted to make it show the text in black aligned left, overlaid on the emoji in red aligned to the right. To do this I used Imagemagick’s convert and composite commands. Imagemagick is a very powerful graphics library that can do most of the kinds of thing Photoshop and the like can do, you just have to work out how to invoke it properly.

I ended up doing this in several stages using temporary files. First I produce the white background and emoji in red:

convert -size 212x104 -font Symbola -gravity SouthEast +antialias +dither -fill red -rotate 180 caption:"$emoj" conceptnet-poem-red.png

Symbola is a font with emoji simple one colour form. +antialias and +dither turn off any font finesse that will complicate things later. -rotate 180 because the display is mounted upside-down. "$emoj" is the emoji associated with the word that was used to generate the poem.

Then I make the text image:

convert -size 212x104 -font Lato-Medium -gravity West +antialias +dither -rotate 180 -alpha on -fill black -background none -channel black -threshold 70% caption:@- conceptnet-poem-black.png

-alpha on and -background none make the background transparent for later compositing. Despite telling it to turn off antialiasing I think it still happened, so I threshold the black channel to make it one colour. caption:@- tells it to take the piped in poem text and lay it out sensibly, altering the size to fit.

Originally I just composited these two together, but the red on the e-paper is quite dark so it sometimes made parts of the text hard to read. So I make a copy of the text pixels inverted to white and smudged out:

convert conceptnet-poem-black.png -negate -channel A -morphology Dilate Octagon conceptnet-poem-white.png

Then I stick the black text on top of the white smudged text to make black text with a white border:

composite conceptnet-poem-black.png conceptnet-poem-white.png conceptnet-poem-black.png

Then that onto the white background with the red emoji. Outputting bmp now because I don’t need transparency or other png features any more:

composite conceptnet-poem-black.png conceptnet-poem-red.png conceptnet-poem.bmp

Next I replace the red parts of that image with white leaving only the black text and convert to 1-bit monochrome:

convert -fill white -opaque red conceptnet-poem.bmp -monochrome conceptnet-poem-black.bmp

Then similarly replace the black with white leaving only the red and convert to monochrome:

convert -fill white -opaque black conceptnet-poem.bmp -monochrome conceptnet-poem-red.bmp

Finally these two files can be sent to the e-paper display. For that I use one of the python examples for the Waveshare e-paper with everything stripped out except the bit that displays the bitmap files.

Finishing Touches

Finally, I had this all run from a cron job every five minutes and at boot. I enabled overlay filesystem in raspi-config so that it never changes the data on the SD card. This means the power can be unplugged without risking corrupting the filesystem.

For testing I lived with it on my desk for a while, checking it didn’t generate anything too horrible. Very occasionally it makes a poem that’s too long so the text is illegible, but it’s not often enough to justify going back into battle with Imagemagick.

Conclusion

The gift was well received I think and it was successful in helping me realise some poetic avenues I’d been meaning to pursue into Conceptnet.

Apart from looking at reusing some of these techniques in Lyricam, I’d like to make some more Poematrons, maybe with bigger e-paper displays, maybe an audio version built in to a radio, maybe a Twitter bot.