Tag: python

Python Bunch object or How to make dictionaries act like objects

Programming

Python dictionaries are really powerful, but sometimes you need an object that is a dictionary but it’s also an object which keys are methods of the same object.
Now think if you can have a class called Bunch that makes something like this possible:

bunch = Bunch(a = 7, b = 3, c = 'hello')
In [5]: bunch.a
Out[5]: 7
In [6]: bunch.b
Out[6]: 3
In [7]: bunch.c
Out[7]: 'hello'

But it’s also more powerful than this:

In [9]: d = {'a.b.c':1, 'a.c':2}
In [10]: bunch = Bunch(**d)
In [11]: bunch.a
bunch.a      bunch.a.b.c  bunch.a.c
In [13]: print bunch.a.b.c
1
In [14]: print bunch.a.c
2
In [15]: print bunch['a.b.c']
1

You can also use the dot notation to describe an hierarchy of objects and the Bunch object will take care of it

It’s not a dream, I’ve just implemented this python Bunch Object and this is the code:

class C(object): pass

def rec_getattr(obj, attr):
    """
    Get object's attribute. May use dot notation.
    """
    if '.' not in attr:
        return getattr(obj, attr)
    else:
        L = attr.split('.')
        return rec_getattr(getattr(obj, L[0]), '.'.join(L[1:]))

def rec_setattr(obj, attr, value):
    """
    Set object's attribute. May use dot notation.
    """
    if '.' not in attr:
        setattr(obj, attr, value)
    else:
        L = attr.split('.')
        if not hasattr(obj, L[0]):
            setattr(obj, L[0], C())
        rec_setattr(getattr(obj, L[0]), '.'.join(L[1:]), value)

class Bunch(dict):
    def __init__(self,**kw):
        dict.__init__(self,kw)
        self.__dict__.update(kw)
        for k,v in kw.iteritems():
            rec_setattr(self, k, v) 

Comments, suggestions, patches are welcome 🙂

How to install pymatlab

HowToubuntu

For a university project i had to deal with a web application written in python and some computations that are matlab based. Of course my webapp was in python so I had to find a library to interact with python. I’ve found pymatlab to be a good library for this goal.

Installing pymatlab was a bit tricky. I had to install matlab on Linux, then install the python-numpy package and then install pymatlab with this sets of commands


sudo -s

export LIBRARY_PATH=/usr/local/MATLAB/R2011a/bin/glnxa64/

export PATH=$PATH:/usr/local/MATLAB/R2011a/bin/
 export C_INCLUDE_PATH=/usr/local/MATLAB/R2011a/extern/include/
 apt-get install csh
 pip install pymatlab

Then I was able to run matlab inside my python programs.

Why IronPython is shit…

Coding

I’m not a C# fan, I’m more a python fan though. Anyway. At work I had to make C# be able to call python functions. So I started googling. I found some nice posts from:

Run python from C# and vice versa

But thanks to .Net 4.0 It seems that a lot of troubles have been resolved introducing the new dynamic type in C#. With that type I can finally do some work in python then export any class to my C# world.

There are still some issues in using any python module: Ironpython is not compatible with the normal python interpreter, or CPython. You can use Ironclad to make Ironpython run C CPython modules but they don’t support Python 2.7 yet and the last update from their release is from  07/2010.

Install IronPython

After installing IronPath you have to modify windows PATH system variable such that you can execute ipy in a shell and get a IronPython console. To do so you have to add the following two lines at the end of your PATH variable on Windows:


;c:Program Files (x86)IronPython 2.7;c:Program Files (x86)IronPython 2.7Sciprts

Install Setuptools

Tried to install distribute. Of course it didn’t work. It crashed. I’ve successfully managed to setup setuptools but their support is broken and i’m unable to install simple python eggs due to multiple error.   These are the steps that I’ve followed

c:UsersvinzDesktop>ipy distribute_setup.py
Downloading http://pypi.python.org/packages/source/d/distribute/distribute-0.6.2
7.tar.gz
Extracting in c:usersvinzappdatalocaltemptmpanru2x
Now working in c:usersvinzappdatalocaltemptmpanru2xdistribute-0.6.27
Installing Distribute

Process is terminated due to StackOverflowException.
Something went wrong during the installation.
See the error message above.

Use setuptools instead http://peak.telecommunity.com/dist/ez_setup.py

Open a priviledged shell http://www.howtogeek.com/howto/windows-vista/run-a-command-as-administrator-from-the-windows-vista-run-box/


ipy ez_setup.py

Since there is no support to zlib in ironpython you cannot install any egg anyway even if you installed setuptools.

Install an egg

I’ve managed to install setuptools, however installing a simple package fails because .pth files are not managed correctly. So the best way of installing eggs is just to take their source, maybe unpacking an egg since it’s just a zip file, and copy directly on the IronPython site-packages directory. I’ll still describe the process I did just in case some reader sees that I’ve done a mistake.

To install an egg, just unzip it and run:


ipy -X:Frames setup.py install --user

The –user will install the package for user only. To install the package system-wide you have to install from an administrator shell.

The best thing to do to install an egg is just to unpack it, it’s a zip file, then copy the directory of your library in C:Program Files (x86)IronPython 2.7Libsite-packages . That directory will be included in sys.path of every ipy interpreter.

First problems

Just run a first example which uses ElementTree and i got:


NotImplementedError: iterparse is not supported on IronPython. (CP #31923)

Surfing a bit the web I found that It’s a bug from Dec 2011 http://ironpython.codeplex.com/workitem/31579

So I’ve downloaded and installed elementTree, It will install on normal Python’s site-packages, so you have to move the folder unver IronPython’s site-packages. After doing so, when you need to import etree just do:


import platform
if platform.python_implementation() == 'IronPython':
from elementtree import ElementTree
else:
import xml.etree.ElementTree

This code will import the just installed elementtree library or the system library one depending if you are on normal Python interpreter (CPython) or IronPython.

Conclusions

From this experience I’ve learnt many things:

  • IronPython is not a full python interpreter: it lacks many basic support like the one for setuptool and distribute
  • IronPython is very bugged, even the standard library is bugger. Thus import as external module everything you need.
  • IronPython cannot use the vast majority of python modules because to support .NET jit it breaks CPython compatibility

Thus. Keep away from it! It’s just crap. Microsoft’s dream of .Net simply doesn’t work for python. Yeah, you can access Python functions and class from C# but you miss the goal that makes people use python: a very huge amount of libraries and packages that are easy to install and integrate. Sorry Microsoft, it’s yet another #Fail

How to integrate mongoengine in pyramid

Programming

For a project in the university i had to build a webapp, since it was a edu-based project and i wanted to try something which diverges from the classic Apache+django+mysql approach and i built a Cherokee+pyramid+mongodb applciation. In this article ill explain how to use mongoengine, a state-of-the-art django like ORM for mongodb in your pyramid application.

First of all i think that pyramid has the best and liter interface i’ve ever seen. This will let u integrate mongodb with very small work. If u ever used mongoengine, u just need to call:


from mongoengine import connect

connect('database')

So we need to call it in our application. In pyramid all the application is initialized in the __init__.py file which has this content:

from pyramid.config import Configurator
from myproject.resources import Root

def main(global_config, **settings):
    """ This function returns a Pyramid WSGI application.
    """
    config = Configurator(root_factory=Root, settings=settings)
    config.add_view('myproject.views.my_view',
                    context='myproject.resources.Root',
                    renderer='myproject:templates/mytemplate.pt')
    config.add_static_view('static', 'myproject:static')
    return config.make_wsgi_app()

You have to change it to be like:

from pyramid.config import Configurator
from myproject.resources import Root

def main(global_config, **settings):
    """ This function returns a Pyramid WSGI application.
    """
    config = Configurator(root_factory=Root, settings=settings)
    config.add_view('myproject.views.my_view',
                    context='myproject.resources.Root',
                    renderer='myproject:templates/mytemplate.pt')
    config.add_static_view('static', 'myproject:static')
    #this will make mongoengine working
    connect(settings['db_name'])
    #this will make all the @view_config in the view working
    config.scan()
    return config.make_wsgi_app()

Of course u have to define your db_name in production.ini or development.ini or both in a way which looks like this:

[app:milo_app]
use = egg:milo_app
reload_templates = true
debug_authorization = false
debug_notfound = false
debug_routematch = false
debug_templates = true
default_locale_name = en
db_name = milo

That’s it, u can now use mongoengine in the resources.py and in any other file u want to create like this:

from mongoengine import *
class User(Document):
	email = StringField(required=True)
	first_name = StringField()
	last_name = StringField()
	password = StringField()
	cwid = IntField()

class Comment(EmbeddedDocument):
	autor = ReferenceField(User)
	content = StringField()

class Genre(EmbeddedDocument):
	name = StringField()

class Movie(Document):
	__name__ = 'Movie'
	__parent__ = Root

	#movie is identified by title and year
	title = StringField(required=True)
	date = DateTimeField()
	description = StringField()
	trailer = URLField()
	poster = StringField()
	image = StringField()
	genre = ListField(StringField())
	comments = ListField(EmbeddedDocumentField(Comment))

	def __str__(self):
		return 'Movie(%s, %s, %s, %s, %s)' % (self.title, self.date, self.poster, self.image, self.trailer)

then in any view u can do:

Movie.objects()
User.objects()

To retrieve all the movies and all the users (in my case)
I hope this helps!

How to install psycopg2 under virtualenv

Programmingubuntu

If you have tried to install psycopg2 (Postgresql support for python) which is used by the popular sqlalchemy framework under virtualenv you will end up with something like this

(python-keygrabber-env)goshawk@cacserver:~/python-keygrabber-env/aranciulla/keygrabber$ pip install psycopg2
Downloading/unpacking psycopg2
  Running setup.py egg_info for package psycopg2
    
    Error: pg_config executable not found.
    
    Please add the directory containing pg_config to the PATH
    or specify the full executable path with the option:
    
        python setup.py build_ext --pg-config /path/to/pg_config build ...
    
    or with the pg_config option in 'setup.cfg'.
    Complete output from command python setup.py egg_info:
    running egg_info

writing pip-egg-info/psycopg2.egg-info/PKG-INFO

writing top-level names to pip-egg-info/psycopg2.egg-info/top_level.txt

writing dependency_links to pip-egg-info/psycopg2.egg-info/dependency_links.txt

warning: manifest_maker: standard file '-c' not found



Error: pg_config executable not found.



Please add the directory containing pg_config to the PATH

or specify the full executable path with the option:



    python setup.py build_ext --pg-config /path/to/pg_config build ...



or with the pg_config option in 'setup.cfg'.

----------------------------------------
Command python setup.py egg_info failed with error code 1
Storing complete log in /home/goshawk/.pip/pip.log
(python-keygrabber-env)goshawk@cacserver:~/python-keygrabber-env/aranciulla/keygrabber$ 

Don’t worry! you need to just install two packages on your ubuntu machine: libpq-dev python-dev. To do so just type in a terminal:

sudo apt-get install libpq-dev python-dev

and thus

(python-keygrabber-env)goshawk@cacserver:~/python-keygrabber-env/aranciulla/keygrabber$ pip install psycopg2
Downloading/unpacking psycopg2
  Running setup.py egg_info for package psycopg2
    
    no previously-included directories found matching 'doc/src/_build'
    warning: no files found matching 'NEWS-2.0'
    warning: no files found matching 'NEWS-2.3'
Installing collected packages: psycopg2
  Running setup.py install for psycopg2
    building 'psycopg2._psycopg' extension
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4-beta2 (dt dec pq3 ext)" -DPG_VERSION_HEX=0x080407 -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.7 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/psycopgmodule.c -o build/temp.linux-x86_64-2.7/psycopg/psycopgmodule.o -Wdeclaration-after-statement
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4-beta2 (dt dec pq3 ext)" -DPG_VERSION_HEX=0x080407 -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.7 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/green.c -o build/temp.linux-x86_64-2.7/psycopg/green.o -Wdeclaration-after-statement
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4-beta2 (dt dec pq3 ext)" -DPG_VERSION_HEX=0x080407 -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.7 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/pqpath.c -o build/temp.linux-x86_64-2.7/psycopg/pqpath.o -Wdeclaration-after-statement
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4-beta2 (dt dec pq3 ext)" -DPG_VERSION_HEX=0x080407 -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.7 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/utils.c -o build/temp.linux-x86_64-2.7/psycopg/utils.o -Wdeclaration-after-statement
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4-beta2 (dt dec pq3 ext)" -DPG_VERSION_HEX=0x080407 -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.7 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/bytes_format.c -o build/temp.linux-x86_64-2.7/psycopg/bytes_format.o -Wdeclaration-after-statement
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4-beta2 (dt dec pq3 ext)" -DPG_VERSION_HEX=0x080407 -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.7 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/connection_int.c -o build/temp.linux-x86_64-2.7/psycopg/connection_int.o -Wdeclaration-after-statement
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4-beta2 (dt dec pq3 ext)" -DPG_VERSION_HEX=0x080407 -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.7 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/connection_type.c -o build/temp.linux-x86_64-2.7/psycopg/connection_type.o -Wdeclaration-after-statement
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4-beta2 (dt dec pq3 ext)" -DPG_VERSION_HEX=0x080407 -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.7 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/cursor_int.c -o build/temp.linux-x86_64-2.7/psycopg/cursor_int.o -Wdeclaration-after-statement
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4-beta2 (dt dec pq3 ext)" -DPG_VERSION_HEX=0x080407 -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.7 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/cursor_type.c -o build/temp.linux-x86_64-2.7/psycopg/cursor_type.o -Wdeclaration-after-statement
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4-beta2 (dt dec pq3 ext)" -DPG_VERSION_HEX=0x080407 -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.7 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/lobject_int.c -o build/temp.linux-x86_64-2.7/psycopg/lobject_int.o -Wdeclaration-after-statement
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4-beta2 (dt dec pq3 ext)" -DPG_VERSION_HEX=0x080407 -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.7 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/lobject_type.c -o build/temp.linux-x86_64-2.7/psycopg/lobject_type.o -Wdeclaration-after-statement
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4-beta2 (dt dec pq3 ext)" -DPG_VERSION_HEX=0x080407 -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.7 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/notify_type.c -o build/temp.linux-x86_64-2.7/psycopg/notify_type.o -Wdeclaration-after-statement
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4-beta2 (dt dec pq3 ext)" -DPG_VERSION_HEX=0x080407 -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.7 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/xid_type.c -o build/temp.linux-x86_64-2.7/psycopg/xid_type.o -Wdeclaration-after-statement
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4-beta2 (dt dec pq3 ext)" -DPG_VERSION_HEX=0x080407 -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.7 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/adapter_asis.c -o build/temp.linux-x86_64-2.7/psycopg/adapter_asis.o -Wdeclaration-after-statement
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4-beta2 (dt dec pq3 ext)" -DPG_VERSION_HEX=0x080407 -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.7 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/adapter_binary.c -o build/temp.linux-x86_64-2.7/psycopg/adapter_binary.o -Wdeclaration-after-statement
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4-beta2 (dt dec pq3 ext)" -DPG_VERSION_HEX=0x080407 -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.7 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/adapter_datetime.c -o build/temp.linux-x86_64-2.7/psycopg/adapter_datetime.o -Wdeclaration-after-statement
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4-beta2 (dt dec pq3 ext)" -DPG_VERSION_HEX=0x080407 -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.7 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/adapter_list.c -o build/temp.linux-x86_64-2.7/psycopg/adapter_list.o -Wdeclaration-after-statement
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4-beta2 (dt dec pq3 ext)" -DPG_VERSION_HEX=0x080407 -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.7 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/adapter_pboolean.c -o build/temp.linux-x86_64-2.7/psycopg/adapter_pboolean.o -Wdeclaration-after-statement
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4-beta2 (dt dec pq3 ext)" -DPG_VERSION_HEX=0x080407 -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.7 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/adapter_pdecimal.c -o build/temp.linux-x86_64-2.7/psycopg/adapter_pdecimal.o -Wdeclaration-after-statement
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4-beta2 (dt dec pq3 ext)" -DPG_VERSION_HEX=0x080407 -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.7 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/adapter_pfloat.c -o build/temp.linux-x86_64-2.7/psycopg/adapter_pfloat.o -Wdeclaration-after-statement
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4-beta2 (dt dec pq3 ext)" -DPG_VERSION_HEX=0x080407 -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.7 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/adapter_qstring.c -o build/temp.linux-x86_64-2.7/psycopg/adapter_qstring.o -Wdeclaration-after-statement
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4-beta2 (dt dec pq3 ext)" -DPG_VERSION_HEX=0x080407 -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.7 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/microprotocols.c -o build/temp.linux-x86_64-2.7/psycopg/microprotocols.o -Wdeclaration-after-statement
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4-beta2 (dt dec pq3 ext)" -DPG_VERSION_HEX=0x080407 -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.7 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/microprotocols_proto.c -o build/temp.linux-x86_64-2.7/psycopg/microprotocols_proto.o -Wdeclaration-after-statement
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4-beta2 (dt dec pq3 ext)" -DPG_VERSION_HEX=0x080407 -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.7 -I. -I/usr/include/postgresql -I/usr/include/postgresql/8.4/server -c psycopg/typecast.c -o build/temp.linux-x86_64-2.7/psycopg/typecast.o -Wdeclaration-after-statement
    gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions build/temp.linux-x86_64-2.7/psycopg/psycopgmodule.o build/temp.linux-x86_64-2.7/psycopg/green.o build/temp.linux-x86_64-2.7/psycopg/pqpath.o build/temp.linux-x86_64-2.7/psycopg/utils.o build/temp.linux-x86_64-2.7/psycopg/bytes_format.o build/temp.linux-x86_64-2.7/psycopg/connection_int.o build/temp.linux-x86_64-2.7/psycopg/connection_type.o build/temp.linux-x86_64-2.7/psycopg/cursor_int.o build/temp.linux-x86_64-2.7/psycopg/cursor_type.o build/temp.linux-x86_64-2.7/psycopg/lobject_int.o build/temp.linux-x86_64-2.7/psycopg/lobject_type.o build/temp.linux-x86_64-2.7/psycopg/notify_type.o build/temp.linux-x86_64-2.7/psycopg/xid_type.o build/temp.linux-x86_64-2.7/psycopg/adapter_asis.o build/temp.linux-x86_64-2.7/psycopg/adapter_binary.o build/temp.linux-x86_64-2.7/psycopg/adapter_datetime.o build/temp.linux-x86_64-2.7/psycopg/adapter_list.o build/temp.linux-x86_64-2.7/psycopg/adapter_pboolean.o build/temp.linux-x86_64-2.7/psycopg/adapter_pdecimal.o build/temp.linux-x86_64-2.7/psycopg/adapter_pfloat.o build/temp.linux-x86_64-2.7/psycopg/adapter_qstring.o build/temp.linux-x86_64-2.7/psycopg/microprotocols.o build/temp.linux-x86_64-2.7/psycopg/microprotocols_proto.o build/temp.linux-x86_64-2.7/psycopg/typecast.o -lpq -o build/lib.linux-x86_64-2.7/psycopg2/_psycopg.so
    
    no previously-included directories found matching 'doc/src/_build'
    warning: no files found matching 'NEWS-2.0'
    warning: no files found matching 'NEWS-2.3'
Successfully installed psycopg2
Cleaning up...
(python-keygrabber-env)goshawk@cacserver:~/python-keygrabber-env/aranciulla/keygrabber$ 

That’s all! 😀 Hope it was useful

Smart serializzation with django (aka making django working with extjs)

Coding

It’s been a while that i didn’t write anything. Well i’ve been too busy with work&studies. But today i’ve developed a nice and painless way to resolve relationships between models in django such that i can serialize the relationship using one field only getting the most significant one. This approach is very useful in ajax/javascript (extjs actually scenarios).
Let’s assume that we have an application in django and i want it to send data to a extjs client application (yep extjs based javascript is a program by itself IMHO). The problem in this scenario is that we have to emit django models as json representations such that extjs can read them. I’ve been googling for a while and i found that the used approach is to get the dictionary-like representation of the instance and send it. But this approach has many drawbacks: you end up having a lot of field_id fields which are exactly the relations and a lot of fields starting with _ which are private python memebers.

My solution was to add a .flat() method such that when called it serializes as python dict the instance, expanding relationships using the most significant fields. Who is the one which sets that significant fields ? It’s you, straight in the model definition. In this way the configuration is distributed within models and you obtain exactly what you wanna have for that relationships. You can also get more than one field for each relation and just join them using a separator. But let’s look at the code:

class CommonObj(object):
    def __repr__(self):
        return str(self.__dict__)

    def __str__(self):
        return self.__repr__()

    def flat(self, mapping={}, func=lambda x:x, separator=' '):
        '''
            Emit a dictionary representing the class. It's able to expand the relationship between classes
            using the special __config__ class you can define in your models. When representation will occour
            it will expand the relations and will return a single field value which is the result of the join
            using the fields_separator of the main_fields list defined in __config__ for each related class.
            If the __config__ is missed it will try to use the name of the relation as the value for the emitted
            dictionary.

            mapping defines how to map the results. it's a dictionary composed by {existent_key:rep_key ...} which
            changes all the existent methods of the object (the existent_key) with the rep_key
        '''

        def get_wanted_rep(key):
            if key in mapping:
                return mapping[key]
            return key

        res = dict()
        iter = dict(self.__dict__)
        for k,v in iter.iteritems():
            if k[0] == '_':
                continue
            elif k[-3:] == '_id' and k != 'id':
                ist = getattr(self, k[:-3])
                if ist:
                    try:
                        attrs = ist.__class__.__config__.main_fields
                    except AttributeError:
                        attrs = [k[:-3]]

                    fields = list()
                    for attr in attrs:
                        fields.append(func(unicode(getattr(ist, attr))))

                    try:
                        sep = ist.__class__.__config__.fields_separator
                        if sep:
                            separator = sep
                    except AttributeError:
                        pass

                    res[get_wanted_rep(k[:-3])] = separator.join(fields)
            else:
                try:
                    res[get_wanted_rep(k)] = func(v)
                except AttributeError:
                    #needed to print numbers
                    res[get_wanted_rep(k)] = v
        return res

class Persona(CommonObj, models.Model):
    nome = models.CharField(max_length=30)
    cognome = models.CharField(max_length=30)
    secondo_cognome = models.CharField(max_length=30, blank=True, null=True)
    indirizzo = models.CharField(max_length=100, blank=True, null=True)
    citta = models.CharField(max_length=15, blank=True, null=True)
    provincia = models.CharField(max_length=30, blank=True, null=True)
    cap = models.IntegerField(blank=True, null=True)
    telefono = models.CharField(max_length=15, blank=True, null=True)
    cellulare = models.CharField(max_length=10, blank=True, null=True)
    email = models.EmailField(blank=True, null=True)
    cliente = models.BooleanField()
    problemi_udito = models.BooleanField()
    porta_apparecchio = models.BooleanField()
    venuto = models.BooleanField()
    spontaneo = models.BooleanField()
    perdita_media = models.CharField(max_length=5, blank=True, null=True, choices=perdita_media_choices)
    tipo_apparecchio = models.CharField(max_length=30, blank=True, null=True)
    lato_apparecchio = models.IntegerField(blank=True, null=True, choices=lato_choices)
    pila = models.IntegerField(blank=True, null=True, choices=pila_choices)
    note = models.TextField(blank=True, null=True)
    preventivo = models.TextField(blank=True, null=True)

We have defined a model which inherits from the CommonObj class which adds the .flat() method. It’s a smart and configurable serializer that expands relations trying to guess the value (it gets the value which has the same name of the relation) it’s able to perform operations thanks to the lambda function parameter to the returned values. Here is an example of this powerful serializer.

goshawk@earth:~/Projects/cacerp/cacerp$ python manage.py shell
Python 2.6.6 (r266:84292, Sep 15 2010, 16:22:56)
Type "copyright", "credits" or "license" for more information.

IPython 0.10 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object'. ?object also works, ?? prints more.

In [1]: from  callcenter.models import Persona

In [2]: p = Persona.objects.get(pk=1)

In [4]: p.flat()
Out[4]:
{'cap': 20151L,
 'cellulare': u'',
 'citta': u'MILANO',
 'cliente': False,
 'cognome': u'XXXXX',
 'email': u'',
 'id': 1L,
 'indirizzo': u'VIA MARIO BORSA 24',
 'lato_apparecchio': None,
 'nome': u'XXXXXXX',
 'note': u'AS ',
 'perdita_media': None,
 'pila': None,
 'porta_apparecchio': False,
 'preventivo': u'',
 'problemi_udito': False,
 'provincia': u'MI',
 'secondo_cognome': u'',
 'spontaneo': False,
 'telefono': u'XXXXXXX',
 'tipo_apparecchio': u'',
 'venuto': False}

It’s possible do define a mapping between the model names and the wanted result. Let’s assume we want CAP instead of cap as result and we want city instead of citta. It’s very easy to accomplish it:

In [9]: p.flat(mapping={'cap':'CAP', 'citta':'city'})
Out[9]:
{'CAP': 20151L,
 'cellulare': u'',
 'city': u'MILANO',
 'cliente': False,
 'cognome': u'XXXXX',
 'email': u'',
 'id': 1L,
 'indirizzo': u'VIA MARIO BORSA 24',
 'lato_apparecchio': None,
 'nome': u'XXXXXX',
 'note': u'AS ',
 'perdita_media': None,
 'pila': None,
 'porta_apparecchio': False,
 'preventivo': u'',
 'problemi_udito': False,
 'provincia': u'MI',
 'secondo_cognome': u'',
 'spontaneo': False,
 'telefono': u'XXXXXX',
 'tipo_apparecchio': u'',
 'venuto': False}

As you can see it’s changed as our defined mapping. Another powerful feature is to do an action on the values thanks to the func function parameter. Here is what we can do:

In [30]: p.flat(func=lambda x: x.capitalize())
Out[30]:
{'cap': 20151L,
 'cellulare': u'',
 'citta': u'Milano',
 'cliente': False,
 'cognome': u'Xxxxxx',
 'email': u'',
 'id': 1L,
 'indirizzo': u'Via mario borsa 24',
 'lato_apparecchio': None,
 'nome': u'Xxxxx',
 'note': u'As ',
 'perdita_media': None,
 'pila': None,
 'porta_apparecchio': False,
 'preventivo': u'',
 'problemi_udito': False,
 'provincia': u'Mi',
 'secondo_cognome': u'',
 'spontaneo': False,
 'telefono': u'xxxxxxxx',
 'tipo_apparecchio': u'',
 'venuto': False}

As you can verify, the values have been capitalized (first character with maiusc on and with maiusc off all the others).
Hope you like it and helps!

%d bloggers like this: