Creating an Animation Studio with a Raspberry Pi and Python

When I was younger I used to love the character Morph. For those of you too young to remember, he was an animated plasticine model who appeared alongside Tony Hart.

Image courtesy of Giles Farrington via Flickr

He was created by Aardman Animations who went on to create other animated films such as Wallace and Gromit and Chicken Run. I often thought that it would have been great fun to create my own plasticine models and capture them on film. There was one small problem... making plasticine models was one thing, filming them and turning them into a film was something else!!

After finishing my blog which turned a Raspberry Pi and camera into a time-lapse camera I started to think about this. It's not a great leap to go from making time-lapse movies to animation movies. I also thought while I am at it, why not wrap it up with a nice user interface?

What a great idea for a blog post I thought ...

... so here is the Python code for your very own Raspberry Pi animation studio! All you need is a Raspberry Pi, a camera for your Pi and the contents of this blog.

This is written in Python 2.7 and not Python 3. To get started you will need to use the IDLE icon on your Raspberry Pi rather than the IDLE 3 icon. For those of you new to the Python world, don't spend too much time worrying if you should using Python 2 or 3. Its easy to change from one to another. I am using Python 2.7 for this program as we will need access to some libraries, which are not available yet in Python 3.

Before we dive into the code lets think about what we want to achieve.

• We probably want a Graphical User Interface (GUI) to make things look nice.
• We definitely want to be able to take pictures, and save them to add into our film.
• We all make mistakes, so we want to delete a frame if we are not happy with it.
• We probably want to be able to scroll through our frames to see what they look like.
• We want to turn our images into a film, and we may want to adjust some inputs to tweak that process.

Sounds as though we have the basic structure of our program defined.

Before we embark on this, if you have not already done so please read my blog post explaining how to get started with Tkinter (For the GUI) and my time-lapse blog. I will borrow heavily from both of these blogs, as if you have some code written which works... why would you write it again from scratch? In fact I keep all my code I have written and am always referring back to it, which I think is good practice, and one you should adopt!

One final word before you get started, I would recommend you save your work very often, just in case!

Lets get started!

The first thing we need to do is to install a few libraries which we need to run this program. We will be using a tool called avconv. This can be installed by typing the following into a command line.

sudo apt-get install -y libav-tools

You will also need to install python-imaging-tk

sudo apt-get install -y python-imaging-tk

First of all here it the complete program. Have a read through it and see which parts you understand. If you have followed my previous blogs it should all be quite straight forward. We will then break it down into more detail in a minute.

from Tkinter import *
from ttk import *
import os
import glob
import Image, ImageTk

## Functions to select frames
def firstFrame():
    if countFrames()>0:
        frameNumber.set(1) #If there are any frames
    else:
        frameNumber.set(0) #Set to 0 if there are none
    updateGUI()
    return None

def lastFrame():
    frameNumber.set(countFrames())
    updateGUI()
    return None

def nextFrame():
    if frameNumber.get() < (countFrames()):
        frameNumber.set(frameNumber.get()+1)
    updateGUI()
    return None

def prevFrame():
    if frameNumber.get() > 1:
        frameNumber.set(frameNumber.get()-1)
    updateGUI()
    return None

def takeFrame():
    frameNumber.set(countFrames()+1)   
    os.system("raspistill -o image%s.jpg"%(setZero()))
    updateGUI()
    return None

##Deletes the last stored frame
def deleteFrame():    
    frameNumber.set(countFrames())
    if countFrames() > 0:
        os.remove('image%s.jpg'%setZero())
        frameNumber.set(countFrames())
        updateGUI()
    return None

##Returns the number of jpg files in the directory.
def countFrames():
    openFiles = glob.glob('image*.jpg')
    return len(openFiles)

## sets the right number of zeros for the frameNumber
def setZero():
    return str(frameNumber.get()).zfill(7)

##Updates the image in the GUI and refreshes
def updateGUI():
        
    if countFrames()>0: # Checks to see if there are any existing frames
        fileName = "image%s.jpg"%setZero()
        inFile = Image.open(fileName)        
        ## Resize image to suit GUI
        ## Was 2592 x 1944 - Divide both of these by 6
        newSizeFile = inFile.resize((400,300))
        
        displayFile =  ImageTk.PhotoImage(newSizeFile)#Convert to Tkinter compatible
        Label(mainframe, image=displayFile).grid(column=1, row = 1, sticky = 'WE',columnspan = 5)
        root.update()#Refreshes
                
    else:
        
        frameImage = PhotoImage(file = "blankScreen.gif")
        Label(mainframe, image=frameImage).grid(column=1, row = 1, sticky = 'WE',columnspan = 5)
        root.update()

## Creates the video file.
def createFilm():
    setfpsIn = fpsIn.get()
    setfpsOut = fpsOut.get()
    os.system("avconv -r %s -i image%s.jpg -r %s -vcodec libx264 -crf 20 -g 15 -vf crop=2592:1458,scale=1280:720 timelapse.mp4"%(setfpsIn,'%7d',setfpsOut))
    return None


##Setup of main GUI
root = Tk()
root.title ('Animation Studio')
root.geometry("410x450")

mainframe = Frame(root, padding = '3 3 12 12')
mainframe.grid(column=0, row = 0, sticky=(N,W,E,S))
mainframe.columnconfigure(0, weight=1)
mainframe.rowconfigure(0,weight=1)

##Variables
frameNumber = IntVar()
fpsIn = IntVar()
fpsOut = IntVar()

##Set Variables
frameNumber.set(countFrames())
fpsIn.set(10)
fpsOut.set(24)

##Frame Select
Button(mainframe, text="<<", command = firstFrame).grid(column = 1, row = 2, sticky = 'WE')
Button(mainframe, text="<", command = prevFrame).grid(column = 2, row = 2, sticky = 'WE')
Button(mainframe, text=">", command = nextFrame).grid(column = 4, row = 2, sticky = 'WE')
Button(mainframe, text=">>", command = lastFrame).grid(column = 5, row = 2, sticky = 'WE')
Entry(mainframe, textvariable = frameNumber, width = 7).grid(column = 3, row = 2, sticky = 'WE')


##Take Frame
Button(mainframe, text="Take Frame", command = takeFrame).grid(column = 1, row = 3, sticky = "WE", columnspan = 5)

##Delete Frame
Button(mainframe, text="Delete Last Frame", command = deleteFrame).grid(column = 1, row = 4, sticky = "WE", columnspan = 5)

##Set FPS in
Label(mainframe, text='Set FPS In').grid(column=1, row = 5, sticky = W)
Entry(mainframe, textvariable = fpsIn, width = 7).grid(column = 2, row = 5, sticky = 'WE')

##Set FPS out
Label(mainframe, text='Set FPS Out').grid(column=4, row = 5, sticky = W)
Entry(mainframe, textvariable = fpsOut, width = 7).grid(column = 5, row = 5, sticky = 'WE')

##Create Film
Button(mainframe, text="Create Film", command = createFilm).grid(column = 1, row = 7, sticky = "WE", columnspan = 5)

updateGUI()

root.mainloop()

To start with there are a few libraries which you will need to access. These are always imported first into your program, as you cannot use them until they have been imported.

from Tkinter import *
from ttk import *
import os
import glob
import Image, ImageTk

Next we have all the function which the program calls. Before we write these, let us jump down the code a little and set out our GUI. Once the GUI is set up we can write the required functions later.

##Setup of main GUI
root = Tk()
root.title ('Animation Studio')
root.geometry("410x450")

These lines of text set up the layout of the GUI. They help us define the name and the size of the window. The values stated as 410x450 are the width and height of our GUI. You can change these if you would like to, but I think these are good values.

Now we define a frame within our GUI.

mainframe = Frame(root, padding = '3 3 12 12')
mainframe.grid(column=0, row = 0, sticky=(N,W,E,S))
mainframe.columnconfigure(0, weight=1)
mainframe.rowconfigure(0,weight=1)

Within our window we need to have one frame. Frames can split the window into different areas. These lines of code define how this frame fits within our window.

As you can see we now define a few variables.

##Variables
frameNumber = IntVar()
fpsIn = IntVar()
fpsOut = IntVar()

I know from my time-lapse program that I might want to change a few parameters when I am creating my film, for example I might want to change how many of my frames I want to display every second (fpsIn), and I might want to change the frame rate the video is created at (fpsOut).

The other variable I will need is to know which of the frames we are currently looking at on the GUI. We will call this variable frameNumber.

Once I have created the variables, I want to assign them values.

##Set Variables
frameNumber.set(countFrames())
fpsIn.set(10)
fpsOut.set(24)

I set the value of fpsIn to be 10, and fpsOut to be 24. The software will allow you to change them later, but these are good default values for you to start with.

Setting the frameNumber looks a little different. I am not setting it with a number, but a function. What is that all about? Well I think when you load the program it would be nice to set the variable frameNumber to the last frame you have taken. You can then carry on from where you left off. Therefore I will write a function called countFrames() which returns the number of frames I have taken. We will get onto writing the function for this later.

Lets now look at laying out the actual GUI.

Firstly we want to be able to scroll through our images. I think it would be nice to scroll through them one at a time and also jump to the first and the last frame. We have created a variable called frameNumber which holds the information about the current frame we are displaying. We should also display that number in our GUI, so we know what we are looking at.

My proposal is for the buttons to use greater than or less than symbols like this < or this > to move forward or back one frame. We will also use two of them together to jump to the first or last frame like so << >>. We will also want to display a number showing which of our images we are looking at.

So our buttons should look this this

<<   <   frame number   >   >>

The code to create these 4 buttons and text entry field is as follows:

##Frame Select
Button(mainframe, text="<<", command = firstFrame).grid(column = 1, row = 2, sticky = 'WE')
Button(mainframe, text="<", command = prevFrame).grid(column = 2, row = 2, sticky = 'WE')
Button(mainframe, text=">", command = nextFrame).grid(column = 4, row = 2, sticky = 'WE')
Button(mainframe, text=">>", command = lastFrame).grid(column = 5, row = 2, sticky = 'WE')
Entry(mainframe, textvariable = frameNumber, width = 7).grid(column = 3, row = 2, sticky = 'WE')

Here we have created 4 buttons and a text entry field. Each of the four buttons calls a different function to allow us to jump to the first/previous/next/last frame, depending upon which button we click.
The last line is an entry field, which we are using to display the value stored in our variable frameNumber.
As we are using the grid layout system, each of these items is positioned using the row and column values. You can see they are all on the same row, but in different columns.

Below these buttons we want a button which allows us to take a frame or picture.

##Take Frame
Button(mainframe, text="Take Frame", command = takeFrame).grid(column = 1, row = 3, sticky = "WE", columnspan = 5)

This is a button we will want to click on to take a new frame. The button is laid out as the buttons above, but has a different label "Take Frame" and calls a different function called takeFrame when clicked. The other difference is we have a columnspan value in there. This is telling our program do not constrain the button to one column. Otherwise it would be the same width as the << button, but allow it to span 5 columns. This should make the button the same size as the size of the <<   <   frame number   >   >>   buttons combined.

We will now create a button to allow us to delete a frame.

##Delete Frame
Button(mainframe, text="Delete Last Frame", command = deleteFrame).grid(column = 1, row = 4, sticky = "WE", columnspan = 5)

This follows the same format as the Take Frame button. Obviously it has a different title and calls a different function.

Now for the values of fpsIn and fpsOut we create a text entry field to define them, and a label to explain the text entry field.

##Set FPS in
Label(mainframe, text='Set FPS In').grid(column=1, row = 5, sticky = W)
Entry(mainframe, textvariable = fpsIn, width = 7).grid(column = 2, row = 5, sticky = 'WE')

##Set FPS out
Label(mainframe, text='Set FPS Out').grid(column=4, row = 5, sticky = W)
Entry(mainframe, textvariable = fpsOut, width = 7).grid(column = 5, row = 5, sticky = 'WE')

Our final button is the button to create the film. This is what we will click on when all the images are taken and we want the film to be made.

##Create Film
Button(mainframe, text="Create Film", command = createFilm).grid(column = 1, row = 7, sticky = "WE", columnspan = 5)

Again this is similar to the other buttons we have created.

The next thing we need is to show the image we have taken. To show images in Tkinter we need to put them into what is called a label. However the other thing we want to do is to update the label each time we change the image. This happens quite often in our program, therefore to save us writing the same code over and over again, we will put the label command into a function which allows us to update it when we need to. We shall create the call to the function now, but we will come back to writing the function in a minute.

updateGUI()

Although this function displays the image at the top of the GUI, as this function refreshes the Tkinter screen it is important that this function is called after all the other widgets, such as buttons / labels, created in the GUI.

root.mainloop()

Finally we finish this section with a root.mainloop() which is important to get our Tkinter program running.

OK we should now have defined out GUI. However we skipped over any functions it called, so we need to go back and write those now.

The functions in my program appear in an order I think they make sense to be in. However they are not in order of what was written first. I will not go through them from top to bottom, but will jump around a little. As some functions call other functions I think the order I am explaining them in will start to make some sense.

First of all we have talked about a function called countFrames(). This is a function that will count the number of frames and return the total number back to us. We will use this function throughout the program, and have already called it while setting the value in our variable frameNumber.

Lets look at writing a function called countFrames()

##Returns the number of jpg files in the directory.
def countFrames():
    openFiles = glob.glob('image*.jpg')
    return len(openFiles)

The first line defines the name of the function and the fact it takes no inputs.

In the timelapse blog we saved our files with the file-name in the format imageXXXXXXX.jpg. Where XXXXXXX was an incremental number. We will keep the naming convention the same in this program.


The next line uses a module called glob, which we imported at the top of our program. This line creates a list called openFiles. Then it stores all the file-names which start with "image" and end with ".jpg" into this list. Remember a * denotes a wildcard, so the program finds all options which fit with the * substituted for real text. This means whatever number our file has in it, this line will add it into the list.

Finally we can return the length of the list, which is counting how many items are in the list. This will be the number of frames we have.

We could have written this function on one line, however I think think using two lines makes it more readable.

This function we have just written returns a number. This is very convenient, as when we are doing anything with the frames, such as scrolling through them it makes sense to refer to each frame by a number. A number is also very easy for Python to increase or decrease. However for each frame we also save a picture file. We spoke earlier that this file name would be in the format imageXXXXXXX.jpg. This ensures that the file names are all the same size and stay in the correct order within our folder. To turn our frame number into this format we will have to add some leading zeroes onto it, and turn it into a string (text).

As an example image 1 would become image0000001.jpg, image 100 becomes image0000100.jpg etc. Seven digits in total should be more than enough for your film.

We will call a function which will find the number of the frame we are in, and return the number, as a string, with the right amount of zeros in front of it.

This will help us determine our file-name.

## sets the right number of zeros for the frameNumber
def setZero():
    return str(frameNumber.get()).zfill(7)

The first line defines the function name, as the brackets are empty there are no parameters being passed into it.
The second line does 4 things.

• It gets the current frame number - frameNumber.get()
• It converts the number into a string - str(frameNumber.get())
• It adds some leading zeros so there are 7 digits. It does this with the zfill command - str(frameNumber.get()).zfill(7)
• It returns this string back to where it was called - return str(frameNumber.get()).zfill(7)

Very easy. The key to this is using the .zfill function to add in the leading zeros.

We talked previously about displaying our current image in Tkinter. Tkinter uses a Label to display images. Although we defined all other parts of the GUI earlier we said as we would have to refresh this label quite often, so it made sense to write it in a function, so we can simply call the function every time we want to refresh the image.

Let us write that function now.

##Updates the image in the GUI and refreshes
def updateGUI():
        
    if countFrames()>0: # Checks to see if there are any existing frames
        fileName = "image%s.jpg"%setZero()
        inFile = Image.open(fileName)        
        ## Resize image to suit GUI
        ## Was 2592 x 1944 - Divide both of these by 6
        newSizeFile = inFile.resize((400,300))
        
        displayFile =  ImageTk.PhotoImage(newSizeFile)#Convert to Tkinter compatible
        Label(mainframe, image=displayFile).grid(column=1, row = 1, sticky = 'WE',columnspan = 5)
        root.update()#Refreshes
                
    else:
        
        frameImage = PhotoImage(file = "blankScreen.gif")
        Label(mainframe, image=frameImage).grid(column=1, row = 1, sticky = 'WE',columnspan = 5)
        root.update()

## Creates the video file.
def createFilm():
    setfpsIn = fpsIn.get()
    setfpsOut = fpsOut.get()
    os.system("avconv -r %s -i image%s.jpg -r %s -vcodec libx264 -crf 20 -g 15 -vf crop=2592:1458,scale=1280:720 timelapse.mp4"%(setfpsIn,'%7d',setfpsOut))
    return None

This is quite a large function, so I will repeat each line below so you know which line I am referring to to help save any confusion!

The first line as always defines the name of the function. Nothing is being passed into this function.

def updateGUI():

As we are putting one of our images into the GUI we need to first of all do a check. What if there are no images already? Not having an image there will ruin our layout a little. Therefore we need a contingency for when we have not taken any images.

Lets first deal with the case when there are some images. We can use our countFrames function to determine if any images have been taken. If this reveals a number greater then 0 there are some images. That countFrames() function is becoming quite useful!

    if countFrames()>0: # Checks to see if there are any existing frames

Now we want to open up the file which relates to the frame we are currently viewing. Previously we showed we can transform the number of the current frame into a string with some leading zeros already attached by running our function setZero() well we do that in the next line.

        fileName = "image%s.jpg"%setZero()

This creates a variable called fileName. It populates it with the string "image%s.jpg", but the clever part of this line is it replaces the %s after image with whatever is returned from our setZero function. %s is an easy way to get different information into strings.

Now we have our file name we can use Image.open to open that file.

        inFile = Image.open(fileName) 

 I know that the images the camera takes are 2592 x 1944 pixels. These are quite large. To make it easier to view on the screen I will resize these. By dividing 2592 by 6 and 1944 by 6 I get 400 x 300. Apart from being nice numbers and about the right size it ensures the aspect ration is kept the same i.e. the images are not stretched either horizontally or vertically.

        newSizeFile = inFile.resize((400,300))

 Tkinter cannot display .jpg images. So we have to convert our image to allow Tkinter to display it. We do this by the following command.

        displayFile =  ImageTk.PhotoImage(newSizeFile)

 Finally we get around to creating the label which we use to display files in Tkinter.

        Label(mainframe, image=displayFile).grid(column=1, row = 1, sticky = 'WE',columnspan = 5)

 We would normally have placed this line with all the other Tkinter buttons and labels. However as we want to decide when we want to update the label it was better off as a function. If we were not modifying the image we may have left it in the usual place, with all the other Tkinter widgets.

 Finally we refresh the window

        root.update()#Refreshes

 So that covers the times when we have already taken frames. We now need to look into the times when there are no existing frames. This is likely when we are using the project for the first time and do not have existing .jpg images.

If we did not load an image the positioning of our GUI would all be thrown out. Therefore if there are no images we will load one which we have pre-created.

 We need to start with an else statement.

    else:

 Now we load our image, which this time is a .gif image. We don't need to do any conversion on this as Tkinter handles .gif images quite nicely. We load the file and store it in a variable called frameImage

        frameImage = PhotoImage(file = "blankScreen.gif")

 Now as before we create a Label to display this blank image, and then refresh the screen.

        Label(mainframe, image=frameImage).grid(column=1, row = 1, sticky = 'WE',columnspan = 5)
        root.update()

And that finishes our updateGUI function. It is important that you have the blankScreen.gif image in the same folder as your Animation Studio folder, otherwise the software will not work.

 We should start now looking at the buttons for scrolling through the images.

## Functions to select frames
def firstFrame():
    if countFrames()>0:
        frameNumber.set(1) #If there are any frames
    else:
        frameNumber.set(0) #Set to 0 if there are none
    updateGUI()
    return None

The idea of this function is to go to the first frame or picture we have taken. As we have already created a variable called frameNumber, we should be able to set that to the right value, and then update our GUI to suit. Should be simple... but we have to cover all eventualities. What if we have not taken a picture? We need to ensure our program deals with that.

Using an if ... else statement should be just the trick.

If we have taken frames previously the number of frames will be greater than 0. Therefore frame 1 will be the first frame, so we should set the frame we want to look at as 1. If we have not taken any frames, then the first will be frame 0. We will set the variable frameNumber to suit.

After naming our function the first line makes use of our frameCount function we have written earlier.

def firstFrame():
    if countFrames()>0:
        frameNumber.set(1) #If there are any frames

This says if the value from countFrames is greater than 0, which should be the case if any frames have been taken, then we can set the frame number to 1. This should be the first frame we have taken.

If the else statement comes into play

    else:
        frameNumber.set(0) #Set to 0 if there are none

Then there are no frames which have been taken. We should set the frame number to 0.

Now we make use of our great updateGUI() function which refreshes the screen to reflect the frame number we are currently on.

    updateGUI()
    

Finally we return None.

    
    return None

The next function helps us navigate to the last frame we have taken.

def lastFrame():
    frameNumber.set(countFrames())
    updateGUI()
    return None

Firstly we define the function name.

def lastFrame():

Now we can set the frameNumber to the value returned from the function countFrames(). This should return the number of the last frame.

    frameNumber.set(countFrames())

We just need to update our GUI to reflect this change.

    updateGUI()
    return None

Finally we return None.

The next function is the next frame function. We need to think about this one a little more. If we are at the last frame and try to move along, our program will likely crash. Therefore we should only allow our program to move to the next frame if the frame we are currently on is less than the total number of frames.

def nextFrame():
    if frameNumber.get() < (countFrames()):
        frameNumber.set(frameNumber.get()+1)
    updateGUI()
    return None

So after defining the function name.

def nextFrame():

We can check to see if the current frame is not more than the total frames we have taken.

    if frameNumber.get() < (countFrames()):

If we have not gone over the total number of frames, then we can increase our current frame number by one. We do this by getting the current frame number, adding one to it and then setting this as our frameNumber. This is all done in a single line.

        frameNumber.set(frameNumber.get()+1)

Again we update the GUI and return None

    updateGUI()
    return None

The previous frame function is similar to the next frame function.

def prevFrame():
    if frameNumber.get() > 1:
        frameNumber.set(frameNumber.get()-1)
    updateGUI()
    return None

However this time we check to ensure our current frame is greater than 1. We should only allow the user to reduce the frame count if this is the case.

We do that check by using frameNumber.get()

    if frameNumber.get() > 1:

We then use the same method we used to increase our frame number by 1 by reducing it by one.

        frameNumber.set(frameNumber.get()-1)

Finally we again updateGUI() and then return None.

    updateGUI()
    return None

The next thing we need to do is create the function which allows us to take frames.

def takeFrame():
    frameNumber.set(countFrames()+1)   
    os.system("raspistill -o image%s.jpg"%(setZero()))
    updateGUI()
    return None

No matter which frame we are looking at we don't want to overwrite a frame. We want to save the frame we take as the number AFTER the total number of frames, which is a number not currently used.

Therefore the first thing we do is to set the variable frameNumber to one after the total number of frames.

    frameNumber.set(countFrames()+1) 

We take the image in a very similar way we did in the timelapse program we created.

    os.system("raspistill -o image%s.jpg"%(setZero()))

The only difference is we use the setZero function to convert the frame number to text to be inserted in the file name.

Finally we update the GUI

    updateGUI()

and return None

    return None

The next function which we need to write is to delete frames. We could program it to delete any frame but there are pitfalls to this, as we would have to replace these deleted frames, either taking a new frame or by reducing the number on all the remaining frames after the one we have deleted. This could get complicated. An easier option would be to allow us to delete the last frame. For the purpose of this project that's what we shall do.

##Deletes the last stored frame
def deleteFrame():    
    frameNumber.set(countFrames())
    if countFrames() > 0:
        os.remove('image%s.jpg'%setZero())
        frameNumber.set(countFrames())
        updateGUI()
    return None

First we will define the function name.

def deleteFrame():   

We will set the frame to the last frame, by calling the countFrames function.

    frameNumber.set(countFrames())

Next we check there are some frames. We don't want to try to delete any frames which are not there.

    if countFrames() > 0:

We will use the os.remove function to remove the frame we are currently on, which is of course the last frame, we have taken.

        os.remove('image%s.jpg'%setZero())

As we were looking at the last frame, which we just deleted, we should recount the frames and set the current frame to the last frame. This will avoid the program trying to update to a frame which no longer exists! Always likely to cause the software to crash!

        frameNumber.set(countFrames())

Finally update the GUI then return None

        updateGUI()
    return None

The final function we have to write is the function to create the film we have been taking.

## Creates the video file.
def createFilm():
    setfpsIn = fpsIn.get()
    setfpsOut = fpsOut.get()
    os.system("avconv -r %s -i image%s.jpg -r %s -vcodec libx264 -crf 20 -g 15 -vf crop=2592:1458,scale=1280:720 timelapse.mp4"%(setfpsIn,'%7d',setfpsOut))
    return None

The first two lines ensures we get the values from our variables for fpsIn and fpsOut.

    setfpsIn = fpsIn.get()
    setfpsOut = fpsOut.get()

Now we create the video using the avconv tool.

    os.system("avconv -r %s -i image%s.jpg -r %s -vcodec libx264 -crf 20 -g 15 -vf crop=2592:1458,scale=1280:720 timelapse.mp4"%(setfpsIn,'%7d',setfpsOut))

There is a little more about this on my timelapse blog. This takes all the images you have created and turns them into a video.

Finally we return None

    return None

And that should finalise the program.

All that is required is to now run the program by pressing F5. As a reminder remember to have the blankScreen.gif image in the same location as your program, as this is where your program will look for it.

blankScreen.gif

If you would like to download the source code, you can do so by clicking on the link below.

Animation Studio Source Code

When you run the program you should see a user interface like this.

I hope you have enjoyed this blog post, I certainly had fun writing it! I would very much like to see some of your animation movies you create with this!

Creating a Time-Lapse Camera with the Raspberry Pi and Python

After the success using openCV with the Raspberry Pi camera to determine the positions of circles in an image, I thought it would be nice to explore more uses of the camera. My inspiration came from a chat with a consultancy I visited. They were in the middle of a large expansion of their site, and were having a lot of building work carried out. To document the building work they had set up a Raspberry Pi with a camera to capture the work and to create a time-lapse video of it.

How much fun does that sound?

My mind was made up - I decided to write a time-lapse video program using Python.

I knew taking the images would be quite simple, but the conversion into video would be more tricky.

My first thought was to make use of the openCV libraries again to turn the images into video. However while I believe this is possible, I really struggled to find the solution. Documentation for openCV is very much geared to C++ and not Python.

My second attempt was to use ffmpeg. Some googling had shown this was a nice solution for the conversion. However after installing and running it I got a message saying

More googling told me I should install ffmpeg from binaries, which I am more than comfortable to do, but it does add complexity into my blog post...

Wait a minute! What was that last sentence on the error message? "Please use avconv instead"

More googling required!

It turns out that avconv does exactly what I need it to do. It converts a pile of images into a video. There are a lot of examples on the web explaining what settings you should use with avconv. However while the avconv website has a lot of information on there I found the best explanation came from the excellent Raspberry Pi Spy website, whose post was also explaining how to create a time-lapse video. Its worth having a look at his page, as he explains how to take images and create video using only the command line.

http://www.raspberrypi-spy.co.uk/2013/05/creating-timelapse-videos-with-the-raspberry-pi-camera/

Right so how do we write a Python program to create a time-lapse video?

The first thing you need to do is to install libav-tools.

Type the following into a command line.

sudo apt-get install -y libav-tools

Here is the full program, have a read through it and see if you can figure out what each line is doing. I will then explain each line in more detail.

import os
import time

FRAMES = 1000
FPS_IN = 10
FPS_OUT = 24
TIMEBETWEEN = 6
FILMLENGTH = float(FRAMES / FPS_IN)


frameCount = 0
while frameCount < FRAMES:
    imageNumber = str(frameCount).zfill(7)
    os.system("raspistill -o image%s.jpg"%(imageNumber))
    frameCount += 1
    time.sleep(TIMEBETWEEN - 6) #Takes roughly 6 seconds to take a picture

os.system("avconv -r %s -i image%s.jpg -r %s -vcodec libx264 -crf 20 -g 15 -vf crop=2592:1458,scale=1280:720 timelapse.mp4"%(FPS_IN,'%7d',FPS_OUT))

To begin with we need to import two libraries, os which will allow us to interact with the command line, and time to enable us to set the time between frames.

import os
import time

Now we will set some global variables. The nice thing about global variables is that you can change a variable that appears all through your program by just changing it in one location. Global variables also make your code more readable, as words explain what the variable is better than a number appearing thorughout your code. It also makes for code which is easier to modify as you know that all your global variables are specified at the top of your code.

We will set 5 global variables.

FRAMES = 1000
FPS_IN = 10
FPS_OUT = 24
TIMEBETWEEN = 6
FILMLENGTH = float(FRAMES / FPS_IN)

FRAMES sets the number of frames or images you will take and add to your video.
FPS_IN sets the number of the Frames Per Second (FPS) that go into the video. So if you want 10 of your frames to be used per second put the value to 10.
FPS_OUT sets the Frames Per Second of the video created. i.e. creates a video running at 24 Frames Per Second. If FPS_IN is less that FPS_OUT some FPS_IN frames will be used several times to bring the number up to FPS_OUT. Setting this value to 24 is a good value for digital video.
TIMEBETWEEN states the time (in seconds) between the frames that you are shooting with your camera. The Raspberry Pi camera takes roughly 6 seconds to take an image, so 6 seconds is the shortest time between shots.
FILMLENGTH works out how long your film will be in seconds. If you want to know this value, then you get your program to print it using the following line.

print FILMLENGTH

I am not going to print this out, but I do use it as a reminder of how long the film I am making will be.

Now we have set up all our variables we can get down to business. We want to take images every so often and save them as files. We will want to keep track of how many images we have taken so we know when to stop. So lets create a variable to do that.

frameCount = 0

The next thing we will do is enter a WHILE loop. A while loop keeps going WHILE something is true

while frameCount < FRAMES:

So while our number in frameCount is less than ( < ) the number we have stored in FRAMES we will run through the next 4 lines of code.

We will create a name for the pictures we want to save.

    imageNumber = str(frameCount).zfill(7)

The reason we want to do this is we want the files to be stored with incremental numbers. This line is quite clever (I think!)

It says we will create a variable called imageNumber. In that variable we will store a string of the value in frameCount. Remember a string is classed as text and not a number. Then using the .zfill(7) command we will ensure that the string has 7 digits by filling all preceding digits with a zero if there are not enough numbers. Some examples are:

'1' becomes '0000001'
'123456' becomes '0123456'

It's not a tool you use very often, but if you want something to be a certain amount of characters it's very useful!

Now we have the name of the image file we are going to create, lets take the image! We are going to use the line you can type into the shell command to take the image.

    os.system("raspistill -o image%s.jpg"%(imageNumber))

What does this line mean? Well the line for taking pictures with the Raspberry Pi camera and storing it as image.jpg is

raspistill -o image.jpg

but this needs to be typed into the command line. Well os.system allows you to access the command line.

You will notice there is a %s in there with %(imageNumber) after the text.

This says, take whatever is in the value imageNumber and put it in place of the %s. So if imageNumber was 0000001 our file would be called image0000001.jpg.

This is a great technique of easily modifying what is in a string.

As we want our number to increase each time we go through the while loop let us now increase frameCount.

    frameCount += 1

With imageNumber being made up of frameCount and some leading zeros, each time we run through the while loop we will get a different number as frameCount is increased each time.

Finally we want to be able to vary the time between taking each frame. This allows us to take our images a set distance apart. It takes roughly 6 seconds to take a picture with the Raspberry Pi camera.

    time.sleep(TIMEBETWEEN - 6) #Takes roughly 6 seconds to take a picture

Therefore the minimum time between each frame is 6 seconds. If we want the camera to wait 10 seconds per image, as it takes 6 seconds to take a picture we only want to sleep between frames for 4 seconds. Therefore we tell the program to sleep for a period of TIMEBETWEEN - 6.

This brings us to the last line of the code. As I mentioned earlier I found the Raspberry Pi Spy website to have the best details on how to use avconv. They suggest typing the following line into the command line to create video from the images. They also explain why.

avconv -r 10 -i image%4d.jpg -r 10 -vcodec libx264 -crf 20 -g 15 -vf crop=2592:1458,scale=1280:720 timelapse.mp4

I have modified this line slightly to make it a little more suitable for our program. I want to be able to set some of the variables using our global variables. Therefore I have changed the line slightly to this.

os.system("avconv -r %s -i image%s.jpg -r %s -vcodec libx264 -crf 20 -g 15 -vf crop=2592:1458,scale=1280:720 timelapse.mp4"%(FPS_IN,'%7d',FPS_OUT))

We already know why we use the os.system command, as this allows us access to the command line. I have also added in a few %s commands to switch in our global variables. This is using the same technique we used on the line where we took the picture. The difference is there are three variables we are switching in.

Let us look at he code inside the os.system brackets.

Most of the code is the same as from the Raspberry Pi Spy webpage, but there are a few differences.

• -r %s this sets the frame rate for the number of our frames we want to use in the video per second. The %s calls the first item in the list %(FPS_IN,'%7d',FPS_OUT), which is FPS_IN, one of our global variables. 
• -i image%s.jpg determines the name of the images we want to load into our video. Again we call something within our list  %(FPS_IN,'%7d',FPS_OUT), this time the second item. What does %7d do? Remember this is being run in the command line, so it's not a python command. The %7d iterates through 7 digit numbers. This is neat as we created our image files with 7 digit numbers. So it is iterating through the images we created. 
• -r %s as for the first -r %s in this line this sets the number of FPS. However we call the FPS_OUT variable and insert this and not the FPS_IN variable. This will create a video of so many Frames Per Second depending on the value of FPS_OUT. If you are unsure 24 is a good default number to use.

All that is left now is to run your program. One word of warning is that avconv is not quick on the Raspberry Pi. 1000 Frames took 3.5 hours to turn into a video. However it's not a huge problem, you just leave it running over night!

I hope you found this tutorial interesting and I look forward to seeing some of your time-lapse videos!

Python - Getting started with OpenCV

A couple of years ago a colleague of mine created a program to ensure that an item I had designed was calibrated properly. The program used a webcam to check a bracket was in the right position and reported back a pass or a fail. I did not know much about the workings of the program, other than it used something called a "Hough Transform". As a non-programmer at the time, I was impressed. I thought this was a very cool program.

A few weeks ago I needed to do something similar. I wanted to check that I could repeatedly position an item in the same location. My colleague has long left the company, so I thought it might be a good opportunity to see if I could write a similar program in Python. It was also an opportunity to finally put my Raspberry Pi camera module to good use.

If you don't have a Raspberry Pi camera, don't worry, you can test this out on pictures taken with a normal camera. I will also supply you some images later on for you to use. One of these I took with a camera phone, so you could do the same if you want to practice.

My starting point for this program was I knew that my colleague had used a Hough Transform, and that there were some good vision libraries available called openCV. I also knew these were available on Python. My program uses a Hough Circle Transform as opposed to a Hough Line Transform.

The objective I was trying to achieve was to be able to check the position of an item and to determine its offset in x, y and any rotation (theta) of the object.

I knew I was able to add some fiducial to the item, so opted for two circles. To make my circles a universal size, rather than drawing around coins to make my circles, I used the inside of a CD as a template. However I will show you how to tweak your program for other circles later!

The first thing you need to do it to install the openCv libraries onto your Raspberry Pi.

sudo apt-get install libopencv-dev python-opencv

To begin with I struggled to find information to get me started, and there seemed to be a lot of confusing information scattered about the web. What I did find out which makes things a little easier to understand is that openCV has released two types of Python interface called cv and cv2. If you are googling for further information it's worth keeping this in mind. We are going to use cv2 in this tutorial.

First of all here is my code and then we will analyse it line by line.

import os
import cv2
import math

##Resize with resize command
def resizeImage(img):
    dst = cv2.resize(img,None, fx=0.25, fy=0.25, interpolation = cv2.INTER_LINEAR)
    return dst

##Take image with Raspberry Pi camera
os.system("raspistill -o image.jpg")

##Load image
img = cv2.imread("/home/pi/Desktop/image.jpg") 
grey = cv2.imread("/home/pi/Desktop/image.jpg",0) #0 for grayscale

##Run Threshold on image to make it black and white
ret, thresh = cv2.threshold(grey,50,255,cv2.THRESH_BINARY)

##Use houghcircles to determine centre of circle
circles = cv2.HoughCircles(thresh,cv2.cv.CV_HOUGH_GRADIENT,1,75,param1=50,param2=13,minRadius=0,maxRadius=175)
for i in circles[0,:]:
    #draw the outer circle
    cv2.circle(img,(i[0],i[1]),i[2],(0,255,0),2)
    #draw the centre of the circle
    cv2.circle(img,(i[0],i[1]),2,(0,0,255),3)

##Determine co-ordinates for centre of circle
x1 = circles[0][0][0]
y1 = circles[0][0][1]
x2 = circles[0][1][0]
y2 = circles[0][1][1]
##Angle betwen two circles
theta = math.degrees(math.atan((y2-y1)/(x2-x1)))

##print information
print "x1 = ",x1
print "y1 = ",y1
print "x2 = ",x2
print "y2 = ",y2
print theta
print circles

##Resize image
img = resizeImage(img)
thresh = resizeImage(thresh)
##Show Images 
cv2.imshow("thresh",thresh)
cv2.imshow("img",img)

cv2.waitKey(0)

First we import 3 modules - os, cv2 and math.

import os
import cv2
import math

Now we create a function to resize images. Although we do our analysis on the full image, we will make the images smaller before we display them on the screen.

##Resize with resize command
def resizeImage(img):
    dst = cv2.resize(img,None, fx=0.25, fy=0.25, interpolation = cv2.INTER_LINEAR)
    return dst

There is a more in-depth explanation of the resize function on the geometric image transformations help page in the openCV documentation.

http://docs.opencv.org/modules/imgproc/doc/geometric_transformations.html

However the important things to note are:

• img is the image we want to resize.
• fx=0.25 and fy=0.25 are the factors that x and y are multiplied by. 0.25 makes the image 1/4 size.

Next we take an image using the Raspberry Pi camera.

##Take image with Raspberry Pi camera
os.system("raspistill -o image.jpg")

os.system allows us to input a command into the command line. We know from the camera documentation that "raspistill -o image.jpg" will take an image with the camera and store it as image.jpg.

Now we load the image twice into our program. The first time as a colour image, the second as a grey-scale image.

##Load image
img = cv2.imread("/home/pi/Desktop/image.jpg") 
grey = cv2.imread("/home/pi/Desktop/image.jpg",0) #0 for grayscale

We then use cv2.threshold to turn our image into black and white.

##Run Threshold on image to make it black and white
ret, thresh = cv2.threshold(grey,50,255,cv2.THRESH_BINARY)

There is more information about the threshold function on the transformations help page on the openCV documentation.

http://docs.opencv.org/modules/imgproc/doc/miscellaneous_transformations.html

The important parts of the code are:

• grey - the image we are converting.
• 50 - is the threshold value. This is between 0 and 255. 0 is white and 255 is black. You may need to modify this value depending on your lighting conditions.
• 255 - If a pixel is above the threshold value, in our case 50, we will make it 255 (black). Else is it 0 (white)

On this black and white image we now run hough circles.

This line in the code carries out the HoughCircle transform. This is the most important line in the code, and the one you are most likely to have to modify to suit your image.

##Use houghcircles to determine centre of circle
circles = cv2.HoughCircles(thresh,cv2.cv.CV_HOUGH_GRADIENT,1,75,param1=50,param2=13,minRadius=0,maxRadius=175)

• thresh - refers to the fact we are carrying out the hough transform on the black and white image.
• 75 - refers to the minimum distance allowed between circles. If you are getting too many circles close together you may want to increase this and vice versa.
• Param1 = 50. This is one of the parameters which determines where the circles are, you can play around with this figure if needs be.
• Param2 = 13. This is the more important of the parameters. If you are getting too many circles then increase this number, and vice versa. Small changes to this make large differences!
• minRadius - the smallest radius allowed for a circle.
• MaxRadius - the largest radius allowed for a circle.

HoughCircle returns a x and y co-ordinate for each circle. It also returns a radius. We are storing these values in the variable called circles.

Again there is more information available about HoughCircle on the function detection page of the openCV documentation under HoughCircles.

http://docs.opencv.org/modules/imgproc/doc/feature_detection.html

We now go through each of these circles in order. For each of the circles we draw its circumference and its centre onto the coloured image we stored in the variable img.

for i in circles[0,:]:
    #draw the outer circle
    cv2.circle(img,(i[0],i[1]),i[2],(0,255,0),2)
    #draw the centre of the circle
    cv2.circle(img,(i[0],i[1]),2,(0,0,255),3)

More information on the circle command is seen on the OpenCV docs page which covers drawing functions.

http://docs.opencv.org/modules/core/doc/drawing_functions.html

Now as I stated at the start of this blog there was an actual reason for writing this program. I wanted to be able to log the x and y co-ordinates of the circles to see how they varied when putting in different items. I also wanted to work out the angle between the circles, and see how that varied.

The next lines of code separate out the x and y co-ordinates. They also work out the angle between them using simple trigonometry.

##Determine co-ordinates for centre of circle
x1 = circles[0][0][0]
y1 = circles[0][0][1]
x2 = circles[0][1][0]
y2 = circles[0][1][1]
##Angle betwen two circles
theta = math.degrees(math.atan((y2-y1)/(x2-x1)))

Finally to display the images nicely on the Raspberry Pi I pass each image through the function to resize them...

##Resize image
img = resizeImage(img)
thresh = resizeImage(thresh)

...then I display the images.

##Show Images 
cv2.imshow("thresh",thresh)
cv2.imshow("img",img)

There is no need to display the images if you don't want to. However I think its good to see what the thresh image and the final image looks like. These are good to help any fault finding. 

You also need the next line for the code to work.

cv2.waitKey(0)

Here is a link to the code, so you can download it rather than type it.

HoughCircles.py

If you don't have a Raspberry Pi Camera there are a few images below which you can use to test your code on. Just change the code to comment out the line about taking the image with the camera, and change the name of the files you are opening to suit.

The first one is taken using my Raspberry Pi camera.

Hough_image1.jpg

The second is from my phone.

Hough_image2.jpg

If you have any problems with the code, try it on these two images first, as I know these work. I think most problems you will have will be to do with lighting. There was a fair amount of playing around with certain parameters, particularly param2 to get this to work.

Keep an eye on my blog, as there could well be some more openCV programs at some point!