The Glenbot

All, any, and reduce in python

Posted on August 24th, 2010 –  0 comments  –  python 

Not too long ago, at work, a discussion was conjured over the uses of all, any, reduce in python. These are standard built-in python functions that can be a time saver in certain situations. All of these are similar in the sense that they:

1. Return an object (everything in python is an object)
2. Can evaluate an iterable based on operators like OR/AND
3. Can be used to produce cleaner, more readable code
4. May be efficient when large amounts of data need to be evaluated

Let's look at the following code:

a = True 
b = False 
c = True 
d = True  

# long winded approach 
if a and b and c and d:     
    print 'All true'  

# using all 
if all([a,b,c,d]):     
    print 'All true'  

# using reduce 
from operator import and_ 
if reduce(and_,[a,b,c,d]):     
    print 'All true'

Your results for each of the snippets above should be None because b = False. Here is the same example using the OR expression:

a = True 
b = False 
c = True 
d = True  

# long winded approach 
if a or b or c or d:     
    print 'One was true'  

# using any 
if any([a,b,c,d]):     
    print 'One was true'  

# using reduce 
from operator import and_ 
if reduce(and_,[a,b,c,d]):     
    print 'One was true' 

The result in all the cases will print 'One was true'.

Reduce gives you a little more flexibility. It can be used to add/subtract numbers in an iterable, evaluate with AND/OR, or a more practicle use could be to combine filter criteria in a Django query. For example:

from operator import or_
from django.db.models import Q
from models import Poll

kwargs = [{'question':'test1'},{'question':'test2'}]
query_objects = []

for kwarg in kwargs:
    query_objects += [Q(**kwarg)]

if query_objects:
    query = reduce(operator.or_,query_objects)
    polls = Poll.objects.filter(query)   

Another use case for (any) could be to check if a certain parameter exists in a query string like so:

query_string = '&a=1&b=2&c=3'

if any(map(query_string.find,['a','b'])):
    print 'It exists!'

In most cases "Readability counts". If using any, all, map, and reduce causes your code to be overly complex when it should be simple avoid using them. Use them out of necessity.

Loading mentions Retweet

Iphone on T-mobile vs. Iphone on AT&T

Posted on July 30th, 2010 –  0 comments  –  iphone 

I have been using an iPhone 2G (first generation) on T-Mobile for quite a while now. In the beginning it made sense. Cheaper plan, better service, and I could still enjoy the iPhone. Now that Apple is no longer updating the iPhone 2G i'm going to have to make a choice: Get a 3G or 3GS and jailbreak it or make the move to AT&T for the iPhone 4. 

Here is a small side by side comparison of the pros and cons of using the iPhone on the somewhat sister companies:

As you can see by the table, AT&T gets the little perks like better 3G*, visual voicemail, and picture messaging. T-Mobile gets the service and price. Remember that call reception depends on your city/county. 

Either way, I went with AT&T for a few reasons:

1. I didn't have to jailbreak my phone anymore (sometimes this got ugly)
2. With rollover minutes and the A-list I could get a comparable price/plan
3. Visual voicemail, picture messaging
4. My whole family is on AT&T
5. All of T-Mobile android based phones sucked.
6. I had no idea when/if T-mobile was actually going to get the iPhone (still rumors)
7. I love the iPhone and can't live without it ;)

The plan I went with on AT&T is:

• 900 Minutes with rollover and A-list
• 1500 text messages
• 2GB data plan (which by the way is plenty if you use WiFi)

I have had 2 weeks time to test the phone and I am experiencing just as many dropped calls on AT&T as I was on T-Mobile. The only thing that changed are the dead zones.

T-mobile was a great company to me for the last 5 years and I wish I could have stayed with them, I just wanted the high tech phone without the hassle (KISS). 

Hope this helps in the decision making process. 

* I am basing the better 3G on people whom I personally know that owned T-Mobile android phones with 3G and experienced many outages inside of buildings. 

Loading mentions Retweet

HTML image tag "onerror" attribute

Posted on July 17th, 2010 –  0 comments  –  html  tutorial 

Sometimes a certain feature or coding syntax will catch you off guard. It will bring you back to Earth letting you know that you really don't know everything about web development. For me, that was the onerror attribute of the HTML image tag. 

This tag is particularly useful. Let's say you are screen scraping HTML content on a search listing to generate an image preview, now you can remove the image element if it doesn't exist. You know, everyone hates that annoying spacing that occurs when the image tag exists but the image doesn't.

Here is a simple demo:

<!DOCTYPE HTML>
<html lang="en-US">
<head>
        <title>Image Tag onerror Attribute</title>
        <meta charset="UTF-8">
</head>
<body>
        <img src="http://theglenbot.com/a-broken-image.jpg" onerror="this.parentNode.removeChild(this);" />
</body>
</html>

I'm not really sure if this is standard HTML but it seems to work in IE7+, Chrome 4+, and FF 3.0+. Also seems like a logical solution.

Loading mentions Retweet

Creating a custom Http403 exception in Django

Posted on June 23rd, 2010 –  0 comments  –  django  python 

Django comes packed with lots of wonderful shortcuts for rendering the correct response to a user. Shortcuts like get_object_or_404 which display 404 pages use the simple Http404 exception that comes with the http library. An example view:

from django.shortcuts import get_object_or_404

def my_view(request):
    my_object = get_object_or_404(MyModel, pk=1)

In the background Django raises the Http404 exception which throws a message letting you know there wasn't a matching query or slug. You may also raise the Http404 exception ala carte in your view like this:

from django.http import Http404

def my_view(request):
    raise Http404

But what about the times when you need to let a user know that they do not have permissions to be there? For example, they may try to export data that doesn't belong to them and we want to return a nice page with a message. In the background the http request should return a status 403 while the front end returns the message. 

We could technically return the HttpResponseForbidden object in the view but that would require passing a template parameter and context to get the nice custom look we want. We can automate this. Instead of returning HttpResponseForbidden let's create our own Http403 exception similar to the Http404 exception that automatically uses a 403.html template. 

First, you want to create a middleware class that will handle the Http403 exception like so:

from django.template import RequestContext 
from django.conf import settings 
from django.core.exceptions import PermissionDenied  

class Http403(Exception):    
    pass  

def render_to_403(*args, **kwargs):     
    """     
        Returns a HttpResponseForbidden whose content is filled with the result of calling     
        django.template.loader.render_to_string() with the passed arguments.     
    """     
    if not isinstance(args,list):         
        args = []         
        args.append('403.html')              

    httpresponse_kwargs = {'mimetype': kwargs.pop('mimetype', None)}          
    response = HttpResponseForbidden(loader.render_to_string(*args, **kwargs), **httpresponse_kwargs)              

    return response  

class Http403Middleware(object):     
    def process_exception(self,request,exception):         
        if isinstance(exception,Http403):             
            if settings.DEBUG:                 
                raise PermissionDenied             
            return render_to_403(context_instance=RequestContext(request))      

The middleware class Http403Middleware catches the Http403 exception in views and raises PermissionDenied (for debug=True) or returns render_to_403 (for debug=False). The middleware class must be placed in your Django settings under MIDDLEWARE_CLASSES.

Let's assume that the code above is placed in an application called "base" and the file it resides in is called middleware.py. Then MIDDLEWARE_CLASSES would look like:

MIDDLEWARE_CLASSES = (
    'django.middleware.common.CommonMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',

    # The HTTP 403 exception
    'base.middleware.Http403Middleware',
)

Now you can create the 403.html template and place it in your templates directory provided in TEMPLATE_DIRS. That could look something like:

<!DOCTYPE HTML>
<html lang="en-US">
<head>
        <meta charset="UTF-8">
        <title>The force is not with you today.</title>
</head>
<body>
        <h1>The force is not with you today</h1>
    <p>You cannot jedi mind trick your way into viewing this page, try again later.</p>
</body>
</html>

The last step is to raise the Http403 exception in your views when someone is denied access. Http403 exists in middleware.py according to the examples above but to better suite Django standards lets place it in a file called http.py. Here is an example of the view:

from django.shortcuts import render_to_response
from base.http import Http403

def index(request):
    if not request.user.has_perm('app_label.permission'):
        raise Http403
    return render_to_response('app/view.html',{})

Another note: the render_to_403 function should be put in a file called shortcuts.py to stick to Django standards but you can put it where you want.

You can pretty much do this with any response code but 404 and 403 are the most common ones that require a front facing message. Happy Coding!

Loading mentions Retweet

PyParseURI - JavaScript's ParseURI Python Port

Posted on April 13th, 2010 –  0 comments  –  python  tools 

I use JavaScript heavily at work in hope to create a nice interface/experience. One little utility that I have used for different projects is Steven Levithan's ParseURI. It's easy and it gets the tedious URL parsing job done. Try the JavaScript demo out. 

One lazy Sunday afternoon I decided to port it over to Python. The functionality and datatype the class returns remains true to the original but now you can use it in your Python based projects.

You can find PyParseURI over at GitHub

Or you can you easy_install:

easy_install pyparseuri

Here is an example of how to use it:

Usage:
from parse_uri import ParseUri
p = ParseUri()
parsed_uri = p.parse('http://www.example.com')
print parsed_uri.authority 
'www.example.com'
print parsed_uri.protocol
'http'

And so on....

Sample Output:
{	
	'protocol': 'http', 
	'authority': 'www.example.com', 
	'anchor': None, 
	'relative': '', 
	'source': 'http://www.example.com', 
	'host': 'www.example.com', 
	'userInfo': None, 
	'file': '', 
	'queryKey': {}, 
	'directory': None, 
	'query': None, 
	'path': '',
	'password': None, 
	'port': None, 
	'user': None
}

Loading mentions Retweet

Multiple Virtual Hosts with Apache in Windows on your PC

Posted on February 11th, 2010 –  4 comments  –  apache  tutorial  windows 

When you work in an environment that requires you to switch back and forth from Drupal, to Wordpress, to Django, it can become difficult to manage the file and URL structure in Apache. A URL like localhost/wordpress/thesite/blog-post-here just sucks. Running applications from the root of the URL helps to ease the transfer from development to production.

Let's see if we can sort this out.

Setting up the folder structure

Assuming Apache is installed and running, let's start with organizing the document root. The default install of Apache creates the document root under program files. 

C:\Program Files\Apache Software Foundation\Apache2.2\htdocs

If needed, substitute C:\Program Files\Apache Software Foundation\Apache2.2\htdocs with your document root setting. Sticking with the default root, create subfolders for each application. For this tutorial we will create the folders Drupal, Wordpress, and Django.

The structure should look like this:

C:\Program Files\Apache Software Foundation\Apache2.2\htdocs\drupal
C:\Program Files\Apache Software Foundation\Apache2.2\htdocs\wordpress
C:\Program Files\Apache Software Foundation\Apache2.2\htdocs\django

For later testing, create a file labeled  index.html, insert the text It Works!and place it within each subfolder.

The next step would be to setup virtual hosts in the Apache configuration file. Unfamiliar with virtual hosts? Apache documentation provides tutorials.

The default Apache configuration file is located here:

C:\Program Files\Apache Software Foundation\Apache2.2\conf\httpd.conf

Virtual Hosts

The first part to setting up virtual hosts is to provide Apache with the NameVirtualHost directive. This is required if you want to use name based virtual hosts. Place this directive at the very bottom of the configuration file (away from other directives to avoid issues).

NameVirtualHost *

Directly after the NameVirtualHost directive we can start setting up virtual hosts. We need three for Drupal, Wordpress and Django. 

<VirtualHost *>
    ServerName wordpress

    # set the document root
    DocumentRoot "C:/Program Files/Apache Software Foundation/Apache2.2/htdocs/wordpress"

    # set the directory settings
    <Directory "C:/Program Files/Apache Software Foundation/Apache2.2/htdocs/wordpress">
        Options All Includes Indexes
        Order allow,deny
        Allow from all
    </Directory>
</VirtualHost>

# virtual host for drupal site
<VirtualHost *>
    ServerName drupal

    # set the document root
    DocumentRoot "C:/Program Files/Apache Software Foundation/Apache2.2/htdocs/drupal"

    # set the directory settings
    <Directory "C:/Program Files/Apache Software Foundation/Apache2.2/htdocs/drupal">
        Options All Includes Indexes
        Order allow,deny
        Allow from all
    </Directory>
</VirtualHost>

# virtual host for django site
<VirtualHost *>
    ServerName django
    # set the document root
    DocumentRoot "C:/Program Files/Apache Software Foundation/Apache2.2/htdocs/django"

    # set the directory settings
    <Directory "C:/Program Files/Apache Software Foundation/Apache2.2/htdocs/django">
        Options All Includes Indexes
        Order allow,deny
        Allow from all
    </Directory>
</VirtualHost>

Virtual hosts will allow us to use Drupal, Wordpress, and Django sites from http://drupal/, http://wordpress/, http://django/ respectively. Note that all the directories within the configuration file are expressed with a frontslash and not a backslash. Running more than one site under each URL is just a matter of creating more subfolders. 

C:\Program Files\Apache Software Foundation\Apache2.2\htdocs\wordpress\site1
C:\Program Files\Apache Software Foundation\Apache2.2\htdocs\wordpress\site2
C:\Program Files\Apache Software Foundation\Apache2.2\htdocs\wordpress\site3

The HOSTS file

The last step is to map the value of the ServerName directive from the virtual hosts example above to your computers loopback ip address in the windows hosts file. 127.0.0.1 is generally the loopback ip.

Hosts can be found here:

C:\Windows\System32\drivers\etc\hosts

The content of the hosts file should look like:

# Copyright (c) 1993-2009 Microsoft Corp.
#
# This is a sample HOSTS file used by Microsoft TCP/IP for Windows.
#
# This file contains the mappings of IP addresses to host names. Each
# entry should be kept on an individual line. The IP address should
# be placed in the first column followed by the corresponding host name.
# The IP address and the host name should be separated by at least one
# space.
#
# Additionally, comments (such as these) may be inserted on individual
# lines or following the machine name denoted by a '#' symbol.
#
# For example:
#
#      102.54.94.97     rhino.acme.com          # source server
#       38.25.63.10     x.acme.com              # x client host

# localhost name resolution is handled within DNS itself.
#	127.0.0.1       localhost
#	::1             localhost

Add the following lines to the bottom of the file.

127.0.0.1    wordpress
127.0.0.1    drupal
127.0.0.1    django

Save the file, reload Apache, and test the URLs in your browser. If everything works right you should see It works! rendered in the browser window. If there is an issue check your paths in the Apache configuration file. Make sure they all use forward slashes.

With each application under its own virtual host you can also you customize they way they run and keep the settings isolated within each one. For instance, Django requires mod_python and Wordpress requires the htaccess file to be enabled but those settings do not need to cross pollinate. The possibilities are endless. 

Happy Coding!

Loading mentions Retweet

My Google Reader Development Bundle

Posted on February 4th, 2010 –  1 comment  –  rss 

Education, knowledge sharing, and creation are all parts of my daily diet. I enjoy reading about what other developers say and so should you! Exercising the knowledge sharing, here is my development RSS bundle. 

http://www.google.com/reader/bundle/user/04345256806204706511/bundle/Development

Some of my favorites

High Scalability - http://highscalability.com/

Reliably Broken - http://reliablybroken.com/b/

B-list - http://www.b-list.org/

Signal vs. Noise - http://37signals.com/svn/posts

Armin Ronacher - http://lucumr.pocoo.org/

Please feel free to leave comments about your favorite development feeds.

Loading mentions Retweet

Django Template Syntax Highlighting For Dreamweaver

Posted on January 14th, 2010 –  0 comments  –  django  python 

My development environment consists of Eclipse, Dreamweaver, and Putty. 

Loading mentions Retweet

Django Model Field to MySQL Datatype

Posted on December 16th, 2009 –  0 comments  –  django  mysql  python 

Creating models and syncing them with the database in Django is a pretty easy process. Remembering the Django model fields and what MySQL data types they use in the database is not. 

I have created a spreadsheet that relates a Django models field to its MySQL datatype.

Django Model Field to MySQL Datatype Spreadsheet

Loading mentions Retweet

Tweetstone Project

Posted on November 19th, 2009 –  0 comments

This morning I released Tweetstone. It's a way to calculate when you will reach any number of tweets. Let's say your goal is 1000 tweets. Just enter in your name and the number of tweets and it will give you an estimated date of completion based on your recent frequency of tweeting. 

The twitter frequency is derived from the Twequency API and the rest is calculated from the Twitter API  with some simple math. The design was done by my good friend Illustrationdan. 

It was a fun little side project and brought back some cowboy programming memories. :)

Try it out!  

Loading mentions Retweet