Luis Alberto Santana

Python Tips

2020-07-01T15:00:00-04:00

A collection of small Python scripts and tips.

This post is based on a Twitter thread I started in April 2020 and works as a centralized way to read all the tips in an easier format than Twitter's 280 characters.

Both will be updated frequently.

List Flatten without explicit loops

Using Itertools' chain

import itertools

test = [[-1, -2], [30, 40], [25, 35]]
list(itertools.chain.from_iterable(test))

>>  [-1, -2, 30, 40, 25, 35]

Using map and strings (Suggested by @Romestantc )

test =  [[-1, -2], [30, 40], [25, 35]]
map(int, ''.join(c for c in test.__str__() if c not in '[]').split(',') )

>> [-1, -2, 30, 40, 25, 35]

Not pretty in my opinion ;)

Count individual items of any iterable

from collections import Counter
count = Counter(['a', 'b', 'c', 'a', 'a', 'b', 'd'])

print(count)
>> Counter({'a': 3, 'b': 2, 'c': 1, 'd': 1})

count['a']
>> 3

Repeat a series of values from any iterable

import itertools as it

data = it.cycle([1, 2])

for i in range(10):
    print(next(data))

>> 1
2
1
2
1
2
1
2
1
2

Name slices to reuse them

# slice(start, end, step)

STEPTWO = slice(None, None, 2)
integer_list = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

integer_list[STEPTWO]

>> [0, 2, 4, 6, 8]

This is the same as:

1	integer_list[::2]

Reverse any "indexable" collection that supports slices

# slice(None, None, -1) or [::-1]

integer_list = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
integer_list[::-1]

>> [9, 8, 7, 6, 5, 4, 3, 2, 1, 0]

Use array to keep homogeneous type of objects in your lists

from array import array

integer_list = array('i', [1, 2, 3])

integer_list = array('i', [1, 2, "a"])
>> TypeError: an integer is required (got type str)

The constructor of array includes a type code parameter and an optional initial list.

Swap dictionary key-values using zip

Using zip:

data = { 'a': 1, 'b': 2, 'c': 3, 'd': 4 }

data.items()
>> dict_items([('a', 1), ('b', 2), ('c', 3), ('d', 4)])

z = zip(data.values(), data.keys())

dict(z)
>>  { 1: 'a', 2: 'b', 3: 'c', 4: 'd' }

Using dictionary comprehensions:

data = { 'a': 1, 'b': 2, 'c': 3, 'd': 4 }

{ v:k for k, v in data.items() }
>>  { 1: 'a', 2: 'b', 3: 'c', 4: 'd' }

If there are repeated values, the result will be overwritten, so be careful.

Namedtuples as lightweight, inmutable, record-like objects

from collections import namedtuple

Person = namedtuple("Person", "name age gender")
bob = Person("Bob", 30, "male")

bob.age
>> 30

bob[1]
>> 30

Something similar can be achieved using Dataclasses (for Python 3.8+)

Built-ins set and frozenset for unordered collections of hashable objects

A = set([1, 2, 3])
B = set([3, 4, 5])

A.union(B)
>> {1, 2, 3, 4, 5}

A.intersection(B)
>> {3}

A.symmetric_difference(B)
>> {1, 2, 4, 5}

frozenset has the same interface but returns an inmutable set object.

Itertools' dropwhile and takewhile to filter from an iterator

from itertools import  dropwhile

numbers = [-2, -1, 0, 1, 2]

f = lambda x: x < 1

list ( dropwhile (f, numbers) )
>>  [1, 2]

takewhile takes the items if True. The difference with built-in function filter is that the iteration stops whenever the test-function is false.

numbers = [-2, -1, 0, 1, 2, -3, -4]
list(dropwhile(f, numbers))

>> [1, 2, -3, -4]

As @jgomo3 Observes, one way to think about those functions is that they simply divide a sequence in halves.

takewhile gives you the first part. dropwhile the last part.

Traspose a matrix with List Comprehensions

M = [[1,2,3],
     [4,5,6],
     [7,8,9]]

MT = [[row[i] for row in M] for i in range(len(M))]

print(MT)
>> [[1, 4, 7], [2, 5, 8],  [3, 6, 9]]

Also, NumPy provides methods to for easier matrix manipulation.

Datetime to UTC

To create timezone aware datetimes in Python:

from datetime import datetime, timezone, timedelta

tz = timezone(timedelta(hours=-6))  # UTC-6
local = datetime(2020, 4, 16, 13, 40, 0, 0, tzinfo=tz)

local.isoformat()
>> '2020-04-16T13:40:00-06:00'

from datetime import datetime, timezone

# assuming `local` is a datetime object

print(local.isoformat() )
>> '2020-04-16T13:40:00-06:00'

local_in_utc = local.astimezone(timezone.utc)

print(local_in_utc.isoformat())
>> '2020-04-16T19:40:00+00:00'

You can always use pytz to handle timezones.

from datetime import datetime
from pytz import timezone

dt = datetime(2020, 4, 16, 13, 40, 0, tzinfo=pytz.utc)

print(dt.isoformat())
>> '2020-04-16T13:40:00+00:00'

functools.singledispatch to achieve parametric polymorphism in Python

from functools import singledispatch


@singledispatch
def process(num=None):
    raise NotImplementedError("Implement process function.")

@process.register(int)
def sub_process(num):
    # processing interger
    return f"Integer {num} has been processed successfully!"

@process.register(float)
def sub_process(num):
    # processing float
    return f"Float {num} has been processed successfully!"

# use the function
print(process(12.0))
print(process(1))

Recruiting Tips for Recruiters

2020-05-13T19:30:00-04:00

Since I started my professional life, several years ago, I've dealt with technical recruiters, that's great, right? I'm able to improve my quality of life, grow in a career path, and pursue interesting projects.

Unfortunately, my day to day experience with recruiters is not always great, usually, it takes a lot of time to filter interesting offers from less interesting ones. Based on my experience, I'll try and give a few hints on what makes the recruiting/sourcing process more enjoyable for engineers, engineering managers, product managers, and UX specialists, and hopefully with better results for recruiters.

Instead of ranting about a specific situation on Twitter, I thought it might be useful to describe what "good recruiting" looks like for me.

Let's start:

Do your homework: most engineers and engineering managers I know maintain some kind of blog, Github profile, or LinkedIn profile. Some engineers give talks at local Meetups. They have invested a lot of time publicly sharing some of their work. It is nice to deal with recruiters that have taken their time reviewing that public information if recruiting is a numbers game, you'll drastically increase your chance of screening the right candidates if you do a little research beforehand. And... please, please, please, get the name right in your first contact you wouldn't believe me how many recruiters call me by another name...
Be expressive with the job description: No, I'm not interested in a position I know nothing about, giving out my number to "know more", is not practical to anyone. Tell as much as you can about the position, the team, the kind of projects. The more information a possible candidate knows about the position the more interested (or not) they can be about continue the selection process. You want a software engineer to work on the frontend with React in a team of three? Say so. Do you want to fill a management position? Tell about the project, the team, the stakeholders. We all value our time, the more information you can give in your first contacts the better for everyone.
Mention the company you're recruiting for: This might sound silly, but unless it is obvious who your employer is, it is important to mention the company that you're recruiting for. People usually want to research the company online. If you "don't reveal" the company you're recruiting for, it looks shady and for most of us that's an automated "no follow".

2.1 What about the phone screen? The phone screen comes in after the candidate is interested in "knowing more" about the position or project, not before. I have received tens of messages from recruiters that demand my phone number before they give me any information about the position they want to fill. If you want to do a quick phone screen, mention it as part of the general information about the position you give in your first contact.
Understand the position you're trying to fill: It is a common mistake to ask candidates for +5 years experience in technologies that have existed only for two years or to confuse Java with Javascript. You must understand the position so you can ask the right questions and don't seem clueless to candidates. The more you understand the position, the more questions you can answer... which takes me to the next point.
Be present: You must accompany the candidates through the process, just finding a possible candidate is only the first step. Be there to answer the questions the candidate could have about the process, explain the selection process so there are no doubts, and give useful feedback if the candidate is not selected for any reason. Remember, you are the face of the company you work for, a good recruiting experience leaves the candidate excited to work with you. Just don't disappear.
Be excited about the project: Some recruiters don't show any excitement or interest in the position they're recruiting for... I'm sorry, but if it isn't important to you, why should I be interested?
What's the compensation package/salary range: Let's face it, money is important, people want to know if it is worth it to do a selection process and to invest time in interviews. If you're aware that your compensation package is lower than others, I'm sure the company would have a way to balance that. Don't be obscure with salary ranges.

We could resume all these tips with a simple one Respect everybody's time.

Is there any other tip you think is important? Let me know in the comments!

Thanks Chris for taking your time proofreading this article.

[Debian] Changing Docker's default /var/lib/docker directory

2019-09-10T18:40:00-04:00

By default, Docker storage its files in the /var/lib/docker directory in the file system. If, like me, you're running out of disk space in the default file system, you can quickly change the default directory. This are the steps I followed in Debian 10, it should work with any distro that uses systemd

As superuser, modify the systemd docker startup script. Edit the file /lib/systemd/system/docker.service replacing the ExecStart command:

1	$ sudo vim /var/systemd/system/docker.service

Change the line:

1	ExecStart=/usr/bin/docker daemon -H fd://

1	ExecStart=/usr/bin/docker daemon -g /new/path -H fd://

then stop the service, and be sure that the daemon is completely stopped:

1 2	$ sudo systemctl stop docker $ ps -aux \| grep -i docker

The only output should be the one from grep. Then you can reload the system and -optionally- synchronize your current docker data to the new directory

1
2
3

$ sudo systemctl daemon-reload
$ sudo mkdir /new/path
$ sudo rsync -aqxP /var/lib/docker /new/path

Your docker installation should be running in the new data directory:

1	$ ps -aux \| grep -i docker

That's it.

A Quick Guide to Install Spotify in Debian 9.0

2017-07-05T09:30:00-04:00

A quick guide of the steps I took to install Spotify version 1.0 in Debian 9.0. First of all, the Spotify team doesn't (officially) support the Linux version, so even if they claim to be compatible with de lastest version of Ubuntu's LTS, it tends to fall behind with the dependencies.

With a fresh install of Debian, you'll need dirmngr installed before adding the Spotify repository to your apt list:

1	$ sudo apt install dirmngr

After dirmngr installation, you'll need to add the Spotify keys and repositories (just check the instructions in the Spotify for Linux website):

1
2

$ sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys BBEBDCB318AD50EC6865090613B00F1FD2C19886
$ echo deb http://repository.spotify.com stable non-free | sudo tee /etc/apt/sources.list.d/spotify.list

If you follow the instructions at Spotify's website, the installation should be ready with the update and install commands from apt:

1	$ sudo apt update && sudo apt install spotify-client

But is a common case that not all the dependencies are up to date between Spotify and Debian, so you'll need to add them "by hand".

The following packages have unmet dependencies:
 spotify-client : Depends: libssl1.0.0 but it is not installable
                                  Recommends: libavcodec54 but it is not installable or
                                                          libavcodec-extra-54 but it is not installable
                                  Recommends: libavformat54 but it is not installable
E: Unable to correct problems, you have held broken packages.

First you'll have to download the exact same version of the package that Spotify needs, in my case it was libssl1.0.0, I downloaded it from: https://packages.debian.org/jessie/libssl1.0.0.

Then just install it with apt and try re-installing Spotify:

1
2
3

$ wget http://security.debian.org/debian-security/pool/updates/main/o/openssl/libssl1.0.0_1.0.1t-1+deb8u6_amd64.deb
$ sudo apt install ./libssl1.0.0_1.0.1t-1+deb8u6_amd64.deb  # This can be replaced with sudo dpkg -i ./libssl1.0.0_1.0.1t-1+deb8u6_amd64.deb
$ sudo apt install spotify-client

You should be able to use Spotify in your Debian 9.0 following these steps!

(If you have a premium account, you can use Spotify inside of third party clients such as Clementine in KDE)

Sources:

Guía rápida de instalación de Spotify en Debian 9.0

2017-07-05T09:30:00-04:00

Una guía rápida de los pasos que tomé para instalar la versión 1.0 de Spotify en Debian 9.0. Primero que nada, la versión para Linus de Spotify no está soportada oficialmente, entonces, a pesar de que Spotifiy la anuncia como compatible con la última versión LTS de Ubuntu, tiende a quedar atrasada en cuanto a sus dependencias.

En una instalación limpia de Debian, será necesario tener instalado dirmngr antes de poder agregar los repositorios de Soptify a tus repositorios apt:

1	$ sudo apt install dirmngr

Luego de la instalación de dirmngr, es necesario agregar las claves y repositorios de Spotify (siguiendo los pasos de la página de instalación de Spotify para Linux):

1
2

$ sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys BBEBDCB318AD50EC6865090613B00F1FD2C19886
$ echo deb http://repository.spotify.com stable non-free | sudo tee /etc/apt/sources.list.d/spotify.list

Si sigues las instrucciones en el website de Spotify, la instalación debería terminar con los comandos de update e install de apt:

1	$ sudo apt update && sudo apt install spotify-client

Pero es común que no todas las dependencias estén al día entre Spotify y Debian, entonces habrá que agregarlas "a mano".

The following packages have unmet dependencies:
 spotify-client : Depends: libssl1.0.0 but it is not installable
                                  Recommends: libavcodec54 but it is not installable or
                                                          libavcodec-extra-54 but it is not installable
                                  Recommends: libavformat54 but it is not installable
E: Unable to correct problems, you have held broken packages.

En primer lugar, deberás descargar la versión exacta del paquete que Spotify necesita, en mi caso era necesario libssl1.0.0, el cual descargué desde: https://packages.debian.org/jessie/libssl1.0.0.

Luego, sólo tendrás que instalarlo con apt e intentar instalar Spotity:

1
2
3

$ wget http://security.debian.org/debian-security/pool/updates/main/o/openssl/libssl1.0.0_1.0.1t-1+deb8u6_amd64.deb
$ sudo apt install ./libssl1.0.0_1.0.1t-1+deb8u6_amd64.deb  # This can be replaced with sudo dpkg -i ./libssl1.0.0_1.0.1t-1+deb8u6_amd64.deb
$ sudo apt install spotify-client

Deberías poder usar Spotify en tu Debian 9.0 siguiendo estos pasos!

(Si tienes una cuenta Premium, puedes usar Spotify en aplicaciones de terceros como Clementine en KDE)

Fuentes:

A Farewell to a Great Company

2016-11-08T09:30:00-04:00

This month marks my final month as a CTO/Lead at Posma Group. It's been six years of ups and downs, working in varied and interesting projects, but most of all, working with amazing people, both coworkers and clients. In six years, we went from just the 3 cofounders to a team of 20+ between two offices, working in lots of different projects.

Thanks to all the people who helped me grow and learn along the way. It was a great experience in so many ways.

I wish all the luck to Jorge and Oscar in their aim to keep growing the company and the great team of Posma Group. I'm sure we'll work together in some project, sometime in the future!

Now, let's see what comes next!

Visualizing Tweets During Election Weekend

2015-12-10T09:30:00-05:30

On Sunday (Dec. 06) parliamentary elections were held in Venzuela. I took the opportunity to fetch some data from Twitter's public stream and use graphics to show what was happening in "social media" during that day. To achieve this I relied in the Python programming language.

The Data

I decided to monitor all geolocalized tweets with my home city (Caracas) as origin over a period of five days, two days before the elections, the election day, and two days after it.

To fetch the data, I used the Twython library since I've worked with it before. Therefore, it should be easy to setup a quick script to fetch the data and save it in a text file.

Once the API and OAUTH keys are correctly following Twython's documentation, the next step is to fetch the data originating in certain location, passing it as a parameter to the streaming API. Twitter uses a set of bounding boxes to track the location of a tweet and only geolocated Tweets falling within the requested bounding boxes will be included.

import json, os
from twython import TwythonStreamer

caracas_location = "-67.017225,10.419554,-66.778716,10.52006"

class MyStreamer(TwythonStreamer):
    def on_success(self, data):
        if data['coordinates'] is not None:
            with open('tweets.txt', 'a') as f:
                f.write(json.dumps(data))
                f.write('\n')

    def on_error(self, status_code, data):
        with open('errors.txt', 'a') as f:
            f.write('error: {0}: {1}'.format(status_code, data))



stream = MyStreamer(os.environ['APP_KEY'], os.environ['APP_SECRET'],
                        os.environ['OAUTH_TOKEN'], os.environ['OAUTH_TOKEN_SECRET'])

stream.statuses.filter(locations=caracas_location)

Since I just wanted to fetch geolocalized tweets, I checked for the existance of the coordinates entity in the Tweet object and only save the tweets with geolocation data (lat/long). After running the script for a few days I have around of ~50,000 tweets to play with.

Visualizing the Data

Where were the tweets located?

The first that I wanted to do with the data was to see what was the origin of the tweets during that period. I decided to use a heatmap on top of a geographical map of the city. To do it, I used the development version of the Folium library, which implement mapping in Python on top of Leafleat.js and its plugins.

Click here to see an interactive full version of the heatmap (Warning big file!)

The code used to generate the heatmap, once I've read the text file into a list named data is as simple as:

Creating a map centered in Caracas (10.497084, -66.8854171).
Add each data point to the map using the add_children method.
Writing the html file.

import folium
from folium import plugins

# Create a heatmap with the data.
heatmap_map = folium.Map(location=[10.497084, -66.8854171], zoom_start=12)
heatmap_map.add_children(plugins.HeatMap([[t[1], t[0]] for t in [tweet["coordinates"]["coordinates"] for tweet in data]]))
heatmap_map.save("heatmap.html")

NOTE: Twitter coordinates are formated as geoJSON that is: longitude first, and then latitude. The code swaps them to add (lat, long) points to the map.

In the heatmap, we can clearly see the parks and protected areas in Caracas with zero activity. The same way, there is a lot less activiy in Caricuao, than in the rest of the city. That would be an expected result in the parks and protected areas, but since Caricuao is a large neighborhood the results are clearly interesting.

How does activity look?

So, there's a heatmap of the location of the tweets, but how can we represent the activity over the five days?

I thought that a calendar type chart, such as the one that shows the contributions activity on Github profiles, was an interesting way to represent the data visually. Grouping the tweets by day and hours in cells, and increasing the intensity of the cell depending on the total amount of tweets during that moment.

To achieve this, I decided to use matplotlib as the plotting library, insted of using a Javascript based library such as D3.js.

import matplotlib
import matplotlib.pyplot as plt
from matplotlib.ticker import FuncFormatter, MaxNLocator

import numpy as np

#
# Load the dataset into a nested list with the `Date` and the `hour` data
#

fig, ax = plt.subplots(figsize=(10, 4))
image = np.array(data_array)
cax = ax.imshow(image, cmap=plt.cm.magma, interpolation='nearest')

#
# Add some parameters for the ticks
#

# Add colorbar/legend
cbar = fig.colorbar(cax, orientation='horizontal')

plt.subplots_adjust(wspace=5, hspace=5)
plt.savefig("tweet_frequency_heatmap.png")

With some visual adjustments such as adding labels to the ticks, I've got the graphic I wanted.

NOTE: The lack of tweets during the morning of December 4, is due the absence of data in that period.

As expected, after midnight, when most people are sleeping, the amount of tweets is very close to zero in normal days (Dec 04, Dec 05, Dec 08). On the election day, the graph shows above "normal" activity, and we can see the exact moment when the results of the election were announced on national TV. Between 12 a.m and 2 a.m. of Dec. 07 the amount of tweets in Caracas duplicated.

More visualizations coming soon!

Tools

This little project was used to research and learn the basics of the tooling available to represent data visually (graphs, charts, maps, etc.) some of the tools researched were:

You can check the Python wiki for a comprehensive list of libraries and resources for plotting data.

Quick tip: Adding Variables to env in a Virtualenv (for development purpose)

2015-07-16T13:40:00-05:30

If you're working with third party APIs, you might find code like YOUR_SECRET_KEY="some secret api key" in your source code, this is a bad practices for a lot of reasons (security, source code sharing, etc). Instead, the recommended way to manage this kind of situation is to add the value as a enviroment variable, and read it in your code with something like this:

1 2	import os os.environ.get('YOUR_SECRET_KEY')

So, how do you avoid to add the variable to the enviroment each time you do some coding? If you're working with virtualenv you simply add it in the env/bin/activate script:

1 2	YOUR_SECRET_KEY="some secret api key" export YOUR_SECRET_KEY

Debian (development tips)

2014-12-13T11:58:00-05:30

These are some extra steps that I've found necessary when starting development in a recently-installed Debian machine.

Jpeg support in PIL and pillow.

$ sudo apt-get install libjpeg libjpeg-dev libfreetype6 libfreetype6-dev zlib1g-dev
$ sudo ln -s /usr/lib/`uname -i`-linux-gnu/libfreetype.so /usr/lib/
$ sudo ln -s /usr/lib/`uname -i`-linux-gnu/libjpeg.so /usr/lib/
$ sudo ln -s /usr/lib/`uname -i`-linux-gnu/libz.so /usr/lib/

Installing lxml in Python (Debian based).

If you're getting the "fatal error: libxml/xmlversion.h: No such file or directory" error, just install the following development files:

1	$ sudo apt-get install python-dev libxml2-dev libxslt1-dev

Problems with 'matplotlib' and freetype.

If you're having problems installing matplotlib in a Python virtualenv, and are getting the 'freetype missing' error, you sould install the development files for freetype, and (in most cases) rebuild the python dependencies for matplotlib.

1 2	$ sudo apt-get -u install libfreetype6-dev $ sudo apt-get build-dep python-matplotlib

After that, you can just use pip normally to install matplotlib

1	$ pip install matplotlib

PS. I know of cases where you have to update python-virtualenv and python-pip after you use build-dep. Just apt-get upgrade your installation. Today I learned about the 'pydoc' command from Python.

Problems with locale

In some machines I've found problems when setting locales from Python. First check the results of running

$ locale

In my case is:

LANG=es_VE.UTF-8
LANGUAGE=es_VE:es
LC_CTYPE="es_VE.UTF-8"
LC_NUMERIC="es_VE.UTF-8"
LC_TIME="es_VE.UTF-8"
LC_COLLATE="es_VE.UTF-8"
LC_MONETARY="es_VE.UTF-8"
LC_MESSAGES="es_VE.UTF-8"
LC_PAPER="es_VE.UTF-8"
LC_NAME="es_VE.UTF-8"
LC_ADDRESS="es_VE.UTF-8"
LC_TELEPHONE="es_VE.UTF-8"
LC_MEASUREMENT="es_VE.UTF-8"
LC_IDENTIFICATION="es_VE.UTF-8"
LC_ALL=

Then use:

import locale

try:
    locale.setlocale(locale.LC_ALL, 'es_VE.utf-8')
except locale.Error:
    raise

Change es_VE.UTF-8 for yours.

Creating postgresql roles

This one is not just for Debian based systems, but for PostgreSQL, giving privileges to a specific user in your Database Server is really simple:

CREATE USER user_name WITH PASSWORD 'userpasswd';

CREATE DATABASE new_database;

GRANT ALL PRIVILEGES ON DATABASE new_database to user_name;

Local dependencies for scipy

Before you're able to build scipy using pip install, you'll need to have some dependencies installed.

1	$ sudo apt-get -u install libatlas-base-dev libatlas-dev liblapack-dev libblas-dev

You'll also need a Fortran compiler installed.

1	$ sudo apt-get -u install gfortran

A Simple Django Project Layout

2012-03-12T10:00:00-05:30

UPDATE: This article was originally written when Django version was 1.4. Since this article was published I made changes based on the new features of Django. You can see some of them in this repository: https://github.com/jackboot7/django-project-template.

This is short entry on how I organize my Django projects, in a way that will be easy to deploy to a production server. This is not meant to be a full tutorial on Django, but just to document the way I layout my Django applications. I consider this layout a work in progress, as each day I learn something new that I can integrate into my project's layout. Anyway, I hope someone else can find this entry useful.

For most of my small/pet/side projects, I use a simple server schema, working only with a local development machine and a production server, I know that is a good practice to use a staging server with the same setup as the production server, but for small projects I found this unnecessary.

Project Layout

First, let me show you the layout of most of my projects:

| - project_domain_name/
| | - .gitignore (.hgignore )
| | - requirements.pip
| | - env/                       # Virtual enviroment.
| | - var/
| | - server_config/             # Server configuration files.
| | | - nginx.conf
| | | - uwsgi.ini
| | |
| | - server_logs/               # Server logs.
| | | - errors.log
| | | - access.log
| | |
| | - project_dir/               # Django project. Most of the time I name it 'webapp'
| | | - apps/                    # Application apps.
| | | - templates/
| | | - static/
| | | - manage.py
| | | - urls.py
| | | - settings.py
| | | - local_settings.py

Ignored files

Some of this files and dirs are meant to be ignored by the control version system.

*local_settings.py*, for every local development machine, there should be a local settings file, this is where you put all the local development settings such as database config, or additional apps such as south or django-extensions.

I use *env/* as a directory to install the virtual enviroment, in case you're using virtualenvwrapper you won't need to have this in your project directory.

*var/* is a place to put everything related to the project in runtime, such as sqlite database files, or generated files from compass or coffeescript. Just like the local settings, this directory its supposed to be different for every local development machine.

Settings and Local Settings

As you can see, I use two separate settings files, that way I can keep my production settings on my version control repository, and do daily work with the local settings of my development machine.

At the end of the settings.py file I import the local_settings module, in which I overwrite or add any specific setting that differ from the production one.

try:
    from local_settings import *
except ImportError:
        pass

This way the deployment process is easier from the application's setup point of view.

Paths

The paths on my settings' file are defined using the os module. Even if this isn't the standard way to define paths, I find this useful as it makes easier to work with a project in different machines (local and production, more than one developer on the same project).

import os

SITE_ROOT = os.path.dirname(os.path.realpath(__file__))
PROJECT_DIR = os.path.split(SITE_ROOT)[0]
VAR_ROOT = os.path.join(PROJECT_DIR, 'var')

if not os.path.exists(VAR_ROOT):
    os.mkdir(VAR_ROOT)

If we add this on the top of the setting file, we can later use these names to define standard variables such as *TEMPLATE_DIRS* or *STATICFILES_DIR*, like this:

1 2	STATICFILES_DIRS = (os.path.join(SITE_ROOT, 'static'),) TEMPLATE_DIRS = (os.path.join(SITE_ROOT, 'templates'),)

And so on with every absolute path that I have to define in my settings file.

This is a very simple layout for Django projects, but I've found that as simple as it is, it's still very powerful. In my github you can find a sample of this layout, feel free to fork it and improve it!

I always find interesting how other people have different standards and use different techniques to setup their own projects, that's why I really like to recommend Django Sites, which is a repository of django-based sites along with source code.

How do you layout your Django applications? Do you have any recommendation?

Python Audio Tools

2012-02-15T11:04:00-05:30

Some days ago, I started to work on a web application that requires to encode audio wave files (.wav) into three other formats, being: MP3, Flac and M4A. That encoding will run in its own Celery task.

One thing I knew for certain is that I wouldn't be running commands such as lame or flac directly.

Since the project is being written in Python, I though it would be nice to use a library to do all the encoding directly in Python code, without calling any external program..

If you're going to write an audio related application in Python, I recommend a visit to the audio section in the Python wiki, where you can find all the audio related modules that are available for Python.

I decided to use the API from Python Audio Tools, which is a very complete library to do audio manipulation. In the backend, it uses lame, flac and other commands to do the enconding of the files, but I won't have to run those commands directly.

So far, I'm very happy with the results I've got using audiotools, since it deals with metadata for the audio files (ID3v2 tagging, et al.) and it can even embed artwork into the files (if the format support such metadata).

I wrote a small Python script to show a basic use of this library, if you want, you can check it out on my githhub repo. I'll be adding a little more functionality to it as soon as I got some free time.

(Local) Online Python Documentation

2010-10-15T11:38:00-05:30

Today I learned about the pydoc command from Python. Even if you're offline you can access the standard library documentation running from a local copy using the command mentioned above.

Simply introduce this command in your terminal:

1	$ pydoc -p 8000

Then you can go to http://locahost:8000. This way you can browse the documentation for all your installed Python modules in the same way you'd use the 'help' command from the Python CLI.

[1] http://docs.python.org/library/pydoc.html

[2] http://pydoc.org/

Documentación de Python en línea (Local)

2010-10-15T11:38:00-05:30

Hoy aprendí sobre el comando pydoc de Python. Incluso estando fuera de línea, puedes tener acceso a toda la documentación de la biblioteca estándar de Python a partir de una copia local.

Para usarlo, simplemente debes ejecutar el comando en tu terminal:

1	$ pydoc -p 8000

Entonces podrás ir a http://locahost:8000 y navegar la documentación de todos los módulos instalados localmente del mismo modo que lo harías usando la función help del intérprete de Python.

[1] http://docs.python.org/library/pydoc.html

[2] http://pydoc.org/