The Data

I decided to monitor all geolocalized tweets with my home city (Caracas) as origin over a period of five days, two days before the elections, the election day, and two days after it.

To fetch the data, I used the Twython library since I've worked with it before. Therefore, it should be easy to setup a quick script to fetch the data and save it in a text file.

Once the API and OAUTH keys are correctly following Twython's documentation, the next step is to fetch the data originating in certain location, passing it as a parameter to the streaming API. Twitter uses a set of bounding boxes to track the location of a tweet and only geolocated Tweets falling within the requested bounding boxes will be included.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

import json, os
from twython import TwythonStreamer

caracas_location = "-67.017225,10.419554,-66.778716,10.52006"

class MyStreamer(TwythonStreamer):
    def on_success(self, data):
        if data['coordinates'] is not None:
            with open('tweets.txt', 'a') as f:
                f.write(json.dumps(data))
                f.write('\n')

    def on_error(self, status_code, data):
        with open('errors.txt', 'a') as f:
            f.write('error: {0}: {1}'.format(status_code, data))



stream = MyStreamer(os.environ['APP_KEY'], os.environ['APP_SECRET'],
                        os.environ['OAUTH_TOKEN'], os.environ['OAUTH_TOKEN_SECRET'])

stream.statuses.filter(locations=caracas_location)

Since I just wanted to fetch geolocalized tweets, I checked for the existance of the coordinates entity in the …

Jpeg support in PIL and pillow.

1
2
3
4

$ sudo apt-get install libjpeg libjpeg-dev libfreetype6 libfreetype6-dev zlib1g-dev
$ sudo ln -s /usr/lib/`uname -i`-linux-gnu/libfreetype.so /usr/lib/
$ sudo ln -s /usr/lib/`uname -i`-linux-gnu/libjpeg.so /usr/lib/
$ sudo ln -s /usr/lib/`uname -i`-linux-gnu/libz.so /usr/lib/

Installing lxml in Python (Debian based).

If you're getting the "fatal error: libxml/xmlversion.h: No such file or directory" error, just install the following development files:

1

$ sudo apt-get install python-dev libxml2-dev libxslt1-dev

Problems with 'matplotlib' and freetype.

If you're having problems installing matplotlib in a Python virtualenv, and are getting the 'freetype missing' error, you sould install the development files for freetype, and (in most cases) rebuild the python dependencies for matplotlib.

1
2

$ sudo apt-get  -u install libfreetype6-dev
$ sudo apt-get build-dep python-matplotlib

After that, you can just use pip normally to install matplotlib

1

$ pip install matplotlib

PS. I know of cases where you have to update python-virtualenv and python-pip after you use build-dep. Just apt-get upgrade your installation. Today I learned about the 'pydoc' command from Python.

Problems with locale

In some machines I've found problems when setting locales from Python. First check the results of running

1

$ locale

In my case is:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

LANG=es_VE.UTF-8
LANGUAGE=es_VE:es
LC_CTYPE="es_VE.UTF-8"
LC_NUMERIC="es_VE.UTF-8"
LC_TIME="es_VE.UTF-8"
LC_COLLATE="es_VE.UTF-8"
LC_MONETARY="es_VE.UTF-8"
LC_MESSAGES="es_VE.UTF-8"
LC_PAPER="es_VE.UTF-8"
LC_NAME="es_VE.UTF-8"
LC_ADDRESS="es_VE.UTF-8"
LC_TELEPHONE="es_VE.UTF-8"
LC_MEASUREMENT="es_VE.UTF-8"
LC_IDENTIFICATION="es_VE.UTF-8"
LC_ALL=

Then use:

1
2
3
4
5
6

import locale

try:
    locale.setlocale(locale …