Showing posts with label Linux. Show all posts
Showing posts with label Linux. Show all posts

Monday, 1 February 2016

Turn Your Raspberry Pi into a Twitterbot

Ok. So this was a post I had decided I was not going to write. The reason for this is I did not want to help people create something negative, such as a spam generator.

Then I had a quick look at the Wikipedia page about Twitterbots. I realised there are some really fantastic ideas out there for Twitterbots, which can have a positive impact.

So I decided to go ahead and write the blog post, and to trust people to use it the right way. :-)

Besides, one of the the Twitterbots I have written is testing the Infinite Monkey Theorem to simulate a monkey trying to type the complete works of Shakespeare. The monkey automatically tweets his progress (@TheMonkeyBard).

So who am I to judge? :-)



In this blog post I am going to show you how I wrote another Twitterbot, which sends out a tweet, from a list of pre-written tweets, at a certain time. For me I use it to automatically post my blogs to twitter at certain times of the day. To disguise the fact I am doing this, I have added a slight element of randomness in there, so it doesn't look too much like a Twitterbot. More on that later.

To start with here is the code in its entirety. Have a read through it and see if it makes sense to you, we will then run through this line by line.

from __future__ import print_function #Allows python3 use of print()
import random
import string
import tweepy
import os

FILEPATH = '/home/pi/Desktop/TwitterBot' 

# Consumer keys and access tokens, used for OAuth 
consumer_key =  'YourConsumerKey' 
consumer_secret = 'YourConsumerSecret'
access_token = 'YourAccessToken' 
access_token_secret =  'YourAccessTokenSecret'

# OAuth process, using the keys and tokens
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)

# Creation of the actual interface, using authentication
api = tweepy.API(auth)


## This function sends the tweets. It will check the length of them first of all.      
def sendTweet(tweetText):
    randomNumber = random.randrange(4) #number 0 - (randrange-1) use 4
    #print (randomNumber) #Use for debugging
    if randomNumber == 0:
        api.update_status(status=tweetText) 
        #print (tweetText) #Use for debugging
        return None
    else:
        return None 
        
## This function will print one of the random Tweets
def randomTweet():
    try:
        tweetFile = open(os.path.join(FILEPATH,'Tweets.txt'),'r')
        tweetList = tweetFile.readlines()
        tweetFile.close()
        randomChoice = random.randrange(len(tweetList))
        #print (tweetList[randomChoice]) #For debugging only
        sendTweet(tweetList[randomChoice])
        return None
    except IOError:
        return None        
 
randomTweet()

Now in order to get started with this you will need to do two things.

1. Install Tweepy.
2. Create an app on Twitter

I am not going to explain either of these processes, as I learned how to do this from another blog by the wonderful Raspi.tv. They have done a far better job than I could ever do on this, so please go and read this blog on setting up Tweepy, and then come back to me. :-)

The link you need to follow is here:

http://raspi.tv/2013/how-to-create-a-twitter-app-on-the-raspberry-pi-with-python-tweepy-part-1#app

Right, you are back? Good!

So the first lines of code call in some libraries including Tweepy, which should now be installed on your Raspberry Pi.

from __future__ import print_function #Allows python3 use of print()
import random
import string
import tweepy
import os

The first line allows us to use the Python 3 version of print, which is something I am trying to do a little more these days. The next 4 lines all call libraries we need to use.

The next line caused me the most problems. When you run your program using cron (more on this later), your program needs to know exactly where to go looking for any files you call. Normally I would be in the current directory when I run this program, so this line would not matter. When you run your program using cron it is starting from a different directory, so this line is necessary.

If you don't know what cron is don't worry, it's not difficult. I will explain all later.

FILEPATH = '/home/pi/Desktop/TwitterBot' 

The path to my directory is /home/pi/Desktop/TwitterBot, but you will have to modify this line to suit your requirements.

So how do you find out the location of your program? Well, you can navigate to it in the command line, and then type:

pwd

This will return the directory of where your files are. In my case I store them on the desktop in a folder called TwitterBot. We will use this information later in our program.

So you read the blog post by Raspi.tv? You should therefore have been able to set up an app in twitter, and be able to fill out the information required in these lines of code.

# Consumer keys and access tokens, used for OAuth 
consumer_key =  'YourConsumerKey' 
consumer_secret = 'YourConsumerSecret'
access_token = 'YourAccessToken' 
access_token_secret =  'YourAccessTokenSecret'

# OAuth process, using the keys and tokens
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)

# Creation of the actual interface, using authentication
api = tweepy.API(auth)

Ensure you replace the text in the first 4 lines with your consumer key, consumer secret, access token and access token secret. For your reference, if you just want a reminder of where to find all the information, you need to go to the twitter apps website.

https://apps.twitter.com/

That's all the difficult twitter stuff out of the way. There are just two functions we need to write. One chooses a random tweet, and one tweets it.

Let us look at the function which sends the tweets.

## This function sends the tweets. It will check the length of them first of all.      
def sendTweet(tweetText):
    randomNumber = random.randrange(4) #number 0 - (randrange-1) use 4
    #print (randomNumber) #Use for debugging
    if randomNumber == 0:
        api.update_status(status=tweetText) 
        #print (tweetText) #Use for debugging
        return None
    else:
        return None 

The first line

def sendTweet(tweetText):

creates the function, but also expects some text, stored in a variable called tweetText, to be passed into it. It is the text in tweetText that we will be tweeting.

So this function does two things. It sends the tweet, which is what you would expect, but not every time... I have added an element of randomness into this so it only tweets once in every four attempts. Why did I do this? Well I wanted to post tweets at four times during the day - Midnight, 6am, lunchtime and 6pm, using GMT.  But I don't want to send a tweet every time. I think one tweet a day, on average, is enough to advertise my blog post. So if I use cron to potentially post four times a day, but reduce the chance of posting to one time in four, on average I will post once a day, but at one of the four pre-determined times. Hope that makes sense!

I did this to make it look a little less like a Twitterbot, which would normally post the same time every day. It just mixes it up a little and means there is a chance for a reader from other time zones to see one of my posts.

So that's what the first line does, it picks a random number from 0 to 3, and stores it in the variable randomNumber. You can increase or decrease the frequency by playing around with the number in random.randrange().

    randomNumber = random.randrange(4)

I then use an if statement to check if that number is == 0, which in our program will be once in every four attempts on average.

    if randomNumber == 0:

If it is we then send a tweet made up of the text we passed to our function, which is stored in tweetText. This makes use of the Twitter API to send the tweet.

        api.update_status(status=tweetText) 

We then return from the function.

        return None

If randomNumber is not equal to zero, the program will just return without doing anything.

    else:
        return None

All very simple so far.

The second function we need to write is randomTweet(). This chooses one of the random tweets to send to the sendTweet() function. The tweets it will be choosing from are stored in a text file called Tweets.txt.

## This function will print one of the random Tweets
def randomTweet():
    try:
        tweetFile = open(os.path.join(FILEPATH,'Tweets.txt'),'r')
        tweetList = tweetFile.readlines()
        tweetFile.close()
        randomChoice = random.randrange(len(tweetList))
        #print (tweetList[randomChoice]) #For debugging only
        sendTweet(tweetList[randomChoice])
        return None
    except IOError:
        return None  


We start off by naming the function. As you will see from the empty brackets this function is not expecting anything to be passed into it.

def randomTweet():

We are going to use Try and Except in this function. I do this in case there is an error loading up the file as I don't want the program to crash if it cannot find the file it is trying to open.

    try:
        tweetFile = open(os.path.join(FILEPATH,'Tweets.txt'),'r')
        tweetList = tweetFile.readlines()
        tweetFile.close()

So we Try the following.

We open the file Tweets.txt and store the contents in the variable tweetFile. But as I mentioned earlier when we come to run cron, we need to give the full path to this file. So we use this command

(os.path.join(FILEPATH,'Tweets.txt')

to join Tweets.txt onto the filepath stored in FILEPATH. The reason I have stored the filepath in FILEPATH, is if I want to change it in the future, I only have to modify it in one place. That place is at the top of the program, so easy to find!

We then store the contents of tweetFile into tweetList using readlines. This separates the text into a list which stores each line as a different item in the list.

     tweetList = tweetFile.readlines()

Finally we close the open text file.

     tweetFile.close()

We then analyse the length of our list, and pick a random number from 0 to the length of the list. The result is stored in randomChoice.

        randomChoice = random.randrange(len(tweetList))

And then we send the item which is in that random location in the list to the function we wrote earlier called sendTweet.

        sendTweet(tweetList[randomChoice])

Then we return None

        return None

Finally if there was an error in our try part of the function we catch it and just return from the function doing nothing.

    except IOError:
        return None  

Ok so far so good? Anything missing?

Well if we don't call a function in our program then nothing will run. So we just call the function randomTweet() to kick things off the ground.

randomTweet()

And that is that, the programming is complete.  There really is nothing tricky in all of this.

But if you want this program to tweet we are not quite finished. There are two things remaining. You need to create a text file called Tweets.txt and store this in the same folder as your program. This text file should have each tweet you want to send on a different line in the text file, as shown below.

Tweet 1
Tweet 2
Tweet 3
Tweet 4
Tweet 5

Finally while we have our python program completed, and our Tweets.txt file created, we need some method of telling our Raspberry Pi to run it when we want it to be run.

This is where the wonderful tool called cron comes into its own.

In the past I have written a blog post explaining how cron works. Definitely worth a read. But if you just want to get on and get your Twitterbot underway then I will explain what you need to do.

I said earlier that I would like my Twitterbot to (potentially) tweet 4 times a day. I say potentially as there is randomness in the the code to ensure it only tweets one in four times. To tell the Raspberry Pi to try to tweet at four times in the day I will have to add four lines of code to my crontab file.

On your Raspberry Pi you need to open a terminal window and type

Crontab -e

This will open up the file you need to modify, probably using a text editor called nano.

You will need to enter the following text at the bottom of this:

00 00 * * * /usr/bin/python /home/pi/Desktop/TwitterBot/TwitterBot.py 
00 06 * * * /usr/bin/python /home/pi/Desktop/TwitterBot/TwitterBot.py 
00 12 * * * /usr/bin/python /home/pi/Desktop/TwitterBot/TwitterBot.py 
00 18 * * * /usr/bin/python /home/pi/Desktop/TwitterBot/TwitterBot.py

A brief explanation of each of these lines is as follows:
  • For each of the lines the first 00 refers to the minutes past the hour we want our program to run. 
  • 00,06,12,18 refer to the hour, (24 hour format) at which we want the program to run. 
  • * * * refer to day of Month, Month and Day of week respectively. We want to ignore all these, so use a * for each.  
  • The /usr/bin/python explains you want to launch your program with python. 
  • /home/pi/Desktop/TwitterBot/TwitterBot.py gives the full path to the program you want to run. 

Press ctrl-x and follow the instructions to save and exit.

And that is that. You have a fully functioning Twitterbot which will try to tweet 4 times a day, but on average will only manage it once :-)

I hope this has given you the basics of turning your Raspberry Pi into a Twitterbot. Let me know what Twitterbots you create!

Tuesday, 10 February 2015

Check downloads using SHA-1 and MD5

When you download a large file, such as a distribution of Linux for the Raspberry Pi, you are provided with a checksum to check the download is all correct. Has anyone ever checked their download? Would you know how to? I have to admit I rarely check. However if things did go wrong it would be very easy to overlook the fact the download went wrong. As you will see in this blog it is also possible to use this tool to check that SD cards have been written to correctly.

There are two types of checksum which are commonly used SHA-1 and MD5. I will explain how to use both of these.

First lets look how to do this on an Apple Mac.

To determine the MD5 open the terminal and type either

openssl md5 filename

or for SHA-1

openssl sha1 filename

for the MD5 you don't need to type openssl.

On a Linux system type the following into a terminal.

md5sum filename

and for SHA-1 you need to type

sha1sum filename

depending which checksum you are wanting to use.

Don't forget, if your filename is in a different folder, you will have to include the location along with the filename as below

location/of/folder/filename

Did you know that you are able to check that the image has burned correctly onto say an SD Card by checking the card once you have burned it?

For example on an Apple Mac to check a disk type the following

sudo openssl sha1 /dev/rdisk2

where /dev/rdisk2 is the path to the disk you want to check. Note you will probably need to use the sudo command to ensure you have the correct permissions. This is handy if you want to check that all your disks have burned correctly.


Monday, 9 June 2014

Scheduling Python Programs using Cron

Cron has become one of my favourite utilities in Linux. It allows you to schedule tasks to automatically happen at certain times. It is one of those tools that is so simple and so powerful, but all too often overlooked, or not known about.



I first stumbled across Cron when I decided to set up and administer a wiki server for the company I work for. There is a lot of data on the wiki and it is important that it gets backed up regularly.

Initially I would take a dump of the MySQL database, and then copy the files over on a daily basis by hand.

I then wrote a Python script which carried out these tasks for me. I just had to remember to run the script on a daily basis.

Finally I automated this using Cron.
  • Now every morning (Monday - Friday) at 1:00 am the company wiki is automatically backed up to two locations.
  • Secondly on the 1st of every Month at 2:00 am the wiki is automatically backed up to two different locations.

All this is done without me ever having to worry if it has been done. Cron looks after it all for me.

What is more is it is very simple to use.

This tutorial will help you get to grips with Cron and have your Raspberry Pi or any other Linux computer working away for you in the background.

Firstly Cron does require the time on your Linux computer to be correct, otherwise tasks will not happen when you expect them to! Not that such a basic mistake would ever catch me out while writing this tutorial...

So if you are using a Raspberry Pi, it might be worth setting the time at this point.

type

sudo raspi-config

into the command line.

Then choose

Internationalisation Options > Change Timezone > Europe > London



for those of you in the UK. Obviously everyone else choose a location to suit your local timezone!

For the purposes of this blog post we will assume you are wanting to run a Python program called test.py saved on your Desktop. If you want an example Python program to run then save the following as a Python program called test.py. When run this will create a new folder called test on your Desktop.

import os

os.chdir("/home/pi/Desktop")
os.makedirs("test")

Now again in your command line we will edit a file called crontab, this is short for Cron Tables, a place where you store all your actions.

To edit crontab type the following into the command line.

sudo crontab -e

This will bring up a text file which looks as follows.


The majority of the text in this file briefly explains Cron and how to use it. The first thing you will notice is that all lines start with a #. This means these lines are ignored by Cron, they are for your reference only.

While there are some basic instructions included in this file the most useful line is the bottom line.

# m h dom mon dow command

These are all explained in the file to mean the following.
  • m - minute
  • h - hour
  • dom - day of month
  • mon - month
  • dow - day of week
  • command - This is the command you want to run.

Lets start with the command, as that's the easy bit.

If you want to automatically run a Python program the command is made up of two parts.

On the Raspberry Pi, if you are using Python 2.7, the first part of this will be as follows.

/usr/bin/python

or for Python 3

/usr/bin/python3

The second part is a link to the Python program. For a program called test.py saved on your desktop you would need to have

/home/pi/Desktop/test.py

These two parts are separated by a space to make up the command.

/usr/bin/python /home/pi/Desktop/test.py

These other parameters (m h dom mon dow) are what you will use to specify when your task will run. For each of them you can specify a number or if you want to use all possible values a * is used to signify this.
  • The minutes can be numbers from 0 - 59 and indicates the minute in the hour you want the task to run.
  • The hours can be from 0 - 23 and indicates the hour you want the task to run.
  • Day of the month, 1-31, indicates the date within the month in which you would like to run the task.
  • Month, 1-12, indicates in which month you would like to run the task.
  • Day of the Week, 0-6, indicates on which day you would like to run the task. 0 is Sunday, 1 is Monday etc. 
The secret to setting this correctly is to use a combination of these to define the schedule.

If you wanted to run your python program every hour, then you would only have to supply a value for the minutes, specifying at what point in the hour you wanted your program to run. For example to run your program every 35 minutes past the hour you would enter the following into your crontab.

35 * * * * /usr/bin/python /home/pi/Desktop/test.py

The hours and minutes are easy to work out. If you want to run something everyday at 4:35 you would simply create a line in your crontab which would say.

35 4 * * * /usr/bin/python /home/pi/Desktop/test.py

Making this run on the 12th of the month only you would need to put the figure 12 into the day of the month field also.

35 4 12 * * /usr/bin/python /home/pi/Desktop/test.py

Taking this further you could specify that you only want your program to run at 4:35 on the 12th March.

35 4 12 3 * /usr/bin/python /home/pi/Desktop/test.py

Alternatively you may want to to run at 4:35 every Monday, so would need to use the Day of the week field. The numbers from 0 - 6 represent the days of the week. 0 is a Sunday, 1 is Monday etc.

35 4 * * 1 /usr/bin/python /home/pi/Desktop/test.py

We have put the Day of the Month and the Month back to being a * for this to happen, as we don't want to limit when our program to run on the 12th March only. Leaving these values in would only run the program on the 12th March at 4:35 if that day actually happened to be a Monday! The * implies it can run any day or month.

Easy huh?

There are a few other special characters which help you define when your program should run.

A comma can be used to allow you to add more values into a field. For instance

35 4 * * 1,4 /usr/bin/python /home/pi/Desktop/test.py

The 1,4 indicates the program will run on Monday and Thursday.

Using a - allows you to choose a series of values.

35 4 * * 1-4 /usr/bin/python /home/pi/Desktop/test.py

The 1-4 indicates the program will run Monday - Thursday. ie. Monday, Tuesday, Wednesday and Thursday.

Finally a / denotes increments of ranges.

/15 * * * * /usr/bin/python /home/pi/Desktop/test.py

This will run the program every 15 minutes.

One final useful command is

@reboot /usr/bin/python /home/pi/Desktop/test.py

Will run the program when you reboot your Raspberry Pi or Computer.

From these few examples you have seen that Cron gives you a huge amount of flexibility on when you schedule tasks.

I hope you have found this blog post useful and that it will help you get started automating your programs.