Amit Mendapara

KalaPy 0.4.2 Released...

2010-12-11T13:48:00.001+05:30

Hello everyone,

So finally, I am back on KalaPy and announcing the release of KalaPy 0.4.2 .This is a bugfix release and doesn’t provide any new feature.

Bugs fixings:

Fixed database session store.
Imporved database engine loading.
Improved error handling.
Fixed session middleware (attribute error).
Fixed package.add_rule (register view func only if given).
Connect signals only if database is configured.

Links:

Regards

Amazing BSNL 3G!

2010-11-08T16:22:00.004+05:30

Update: Support for these devices are now officially available in many recently released Linux distros.

Update: Support for these devices have been fully integrated. Use this PPA (ubuntu maverick only) to install/update the requied packages.

Recently, I bought a 3G datacard from BSNL after a promising demo given by a BSNL representative. I was causious because of my previous experience with their EVDO service which never satisfied the promised speed (upto 3.1 mbps). This one (the 3G service) is really good even though it's still under initial stage of deployment. Most of the time, I am getting about 2mbps download speed and 1.5mbps upload speed. This is one of the most affordable Internet service offer in India with this speed (unlimited usage for just 1500 INR).

Here I am sharing my experience because, the device I bought (Teracom LW273 7.2mbps) doesn't work out of the box with Ubuntu. Of course, they provide a Linux application but it's only 32bit package and doesn't work well under 64bit version of Ubuntu. Also, I prefer the official NetworkManager to connect to the Internet instead of such a crappy closed source application.

Here is how I managed to use the device with NetworkManager:

First install usb_modeswith package if it is not installed.

$ sudo apt-get install usb_modeswitch

Using lsusb I found that the device has three configurations. The default one is to mount the device as a cddrive so that you can install the provided application and next two configurations are to mount the device as modem. So first I tried switching the device mode from cdrom to modem with following command:

$ sudo usb_modeswitch -v 230d -p 0001 -u 3

Here 230d is vendor id and 0001 is the product id of the device and as we are switching to modem mode with configuration number 3 is given. After issuing this command the device was switched to modem mode and then I was able to configure the NetworkManager with APN as bsnlnet to connect to the internet.

In order to make this process automatic we need to create a udev rule and a device specific configuration for the usb_modeswith. Here's how I did this:

1. Create udev rule.

Open the /lib/udev/rules.d/40-usb_modeswitch.rules and search for a line with:

LABEL="modeswitch_rules_begin"

Right after that line, add a new line with following contents:

ATTRS{idVendor}=="230d", ATTRS{idProduct}=="0001", RUN+="usb_modeswitch '%b/%k'"

2. Create usb_modeswitch configuration

Create a new configuration /etc/usb_modeswitch.d/230d:0001 and put the following contents in that file.

########################################################

# BSNL Teracom



DefaultVendor= 0x230d

DefaultProduct=0x0001



Configuration=3

That's it. Just remove the device and replug it. Now you should see the device is being mounted as modem instead of cdrom.

Have a fun with BSNL 3g!

Oh yes, here is my latest speedtest result:

Python virtualenv with easy!

2010-09-18T20:52:00.004+05:30

Hi all,

This time not related to KalaPy but something that will help you dealing withPython virtualenv easily. See virtualenv docs for more information.

I have created a shell script to manage virtualenv.The pyenv script can be used to create new virtualenv, activate it, removing existing virtualenv etc. Let meexplain how to use it:

Create new virtualenv named ‘my’:

$ pyenv -c my

The virtualenv will be created in $HOME/.pyenv/my directory.

Activate a virtualenv named ‘my’:

$ . pyenv -a my

Note that, you should use ‘.’ as source in order to make changes in current shell environment.

De-activate current virtualenv:

$ . pyenv -d

Delete an existing virtualenv:

$ pyenv -D my

Create a new virtualenv with alternative python interpreter:

$ pyenv -c py25 -P /usr/bin/python25
$ pyenv -c apy -P /opt/ActivePython-2.6/bin/python

The script will create virtualenv with distribute package for better easy_install support. It also provides pip command to manage python packages.

Have fun...

KalaPy 0.4.1 Released...

2010-08-13T11:46:00.000+05:30

Hello everyone,

KalaPy 0.4.1 has been released.This is a bugfix release and doesn’t provide any new feature.

Bugs fixings:

Added docs on deployement options.
Fixed terminal output issue on win32.
Fixed package name resolution issue.
Fixed MySQL engine (table with InnoDB engine).
Fixed logging (missing reference).

Links:

Regards

KalaPy 0.4 Released...

2010-08-06T11:07:00.000+05:30

Hello everyone,

I am pleased to announce the release of KalaPy 0.4.As said previously,this release comes with lots new features and improvements...

New features:

Implemented addon package feature.
Added wiki_extended example package.
Added settings.STATIC_LINKS to override static dirs & links
Integrated ipython support (shell command).
Translate Jinja2 templates with easy (see docs).
Implemented logging support.

Refactoring:

The web module is splited into several smaller modules.
Application class is no longer Package subclass
ModelCache and Loader have been merged into an object Pool
Reimplemented context locals
Improved admin commands (help, colorize output, error handling)
Lots of new tests
Improved documentation

Bugs fixings:

Fixed early sending of signal request-finished.
Improved signals api.
Fixed GAE issue (sys.path reset, while using gae runserver).
Fixed a regression (db.String validation).

Links:

Regards

KalaPy 0.4 is coming!

2010-08-03T17:22:00.001+05:30

Hello again,

After almost a month, KalaPy is ready for next release 0.4 within next fewdays. There are lots of improvements but the most important feature in this releaseis the packaging system. A package can be an application package (e.g. wiki) oran addon package that extends an existing package (e.g. wiki_extended).

Now every package has it’s own settings.py module to provide somepackage specific configuration settings.
A package can be an addon package. In that case, the resources provided bythe package would be served as resources of the extending package.
An example addon package named wiki_extended has been created in includedexample project.

Here you can see the screenshots of the original wiki application package andafter activating wiki_extended package.

The original one:

The extended one:

You can see, with the addon package you can override css/javascript as well astemplates of any package. The view functions and models provided by the packagewould also be considered as resources of the original package.

Besides that, following features have been implemented:

Allows to override static links with the settings.STATIC_LINKSconfiguration, and links to /favicon.ico and /robots.txt have beenprovided this way.
Logging support with rotating log file handler.

There are also some internal changes and bug fixings. A full changes log will beprovided at the release time.

Links:

Regards

KalaPy 0.3 released...

2010-07-09T19:57:00.001+05:30

Hello Everyone,

I am pleased to announce the release of KalaPy 0.3. This version comes with lots of improvements and support for MySQL and Google App Engine.

Changes:

http://github.com/cristatus/kalapy/raw/master/CHANGES

Links:

Regards

MySQL support in KalaPy!

2010-07-02T13:01:00.001+05:30

Hello Everyone,

MySQL support has been implemented. So now KalaPy supports PostgreSQL, MySQL, SQLite3 and GAE backends.

Besides that the GAE backend support is improved and now fully functional with one exception; it doesn't support transaction. Though, transaction is partially supported if only a single model is involved and there is no unique field defined in the model using db.run_in_transation function.

Another change in DAL API is the way of defining validators and unique contraints. Previourly, we have some helper functions to do that but now those functions have been removed and uses standard descriptive way to define validators and unique contraints.

class Hello(db.Model):
    name = db.String()
    a = db.String()
    b = db.String()

    __unique__ = [name, (a, b)]

    def validate_name(self, value):
        if value is not None and len(value) < 5:
            raise db.ValidationError('name too short')

The __unique__ attribute lists single or combined unique constraints. Single field unique constraint can also be defined with unique=True parameter while defining a field.

A method name with validate_ as prefix and field name as the rest will be attached as validator to that field.

I am in the process of releasing v0.3 and expecting it during next week. Now, I am focusing on improving documentation. I welcome you to help me improving the documentation.

Regards

Google AppEngine support in KalaPy!

2010-06-25T15:30:00.001+05:30

Hello Everyone,

I have just implemented Google AppEngine support in KalaPy. You can see a KalaPy example running on AppEngine.

As I mentioned earlier in my previous post,the DAL API has been changed to deal with GAE related issues. Let me make some corrections regarding these changes.

The NOT IN operator has been removed (this can be done with multiple != filters). The OR operation should beperformed with Q like this:

from kalapy.db import Query, Q
from models import User

q1 = User.all().filter(Q('name ==', 'some')|Q('name ==', 'someone'))

q2 = User.all().filter('name ==', 'some') \
               .filter('lang ==', 'fr_FR')

The q1 is equivalent to:

SELECT * FROM "hello_user" WHERE "name" = 'some' OR "name" = 'someone'

And the q2 is equivalent to:

SELECT * FROM "hello_user" WHERE "name" = 'some' AND "lang" = 'fr_FR'

There are still some issues with GAE backend implementation, especially:

Referential integrity is not implemented yet
Unique constraint is not implemented yet
OneToOne field will not work as expected
Decimal field is not supported at the moment
Transaction is not supported

The issues will be resolved before releasing KalaPy 0.3 which I am planingto release during the first week of the coming July.

Beside that, I have also implemented an admin command to deal with GAE relatedtasks like create app.yaml, launching dev appserver, updating the applicationon appengine etc.

$ cd /path/to/your-kalapy-project
$ ./admin.py gae --help
admin.py gae <action> [options] [args]

Perform google appengine specific tasks.

Use the 'appcfg.py' and 'dev_appserver.py' (included in appengine sdk)
for other appengine specific tasks.

options:

  -a --address hostname for the appserver
  -p --port    port number for the appserver
  -i --install install libs (extra libs as arguments)
  -v --verbose enable verbose output
  -h --help    display help and exit

available actions:

  prepare    prepare this project for google appengine.
  rollback   launch 'appcfg.py rollback'
  runserver  launch 'dev_appserver.py runserver'
  update     launch 'appcfg.py update'

You can install required python modules with ./admin.py gae prepare -i [module1 [module1 [...]]command. For example, with KalaPy example application (change settings.DATABASE_ENGINE = "gae"):

$ cd /path/to/kalapy/example
$ ./admin.py gae prepare -i docutils roman
$ ./admin.py gae runserver

Get the latest sources from the github and help me improving the project.Visit the official project website for latest updates.

Regards

Proposed API Change (DAL)

2010-06-20T16:00:00.001+05:30

Hi All,

The Query interface of the DAL API is going to be changed in next version to make it compatible with Google App Engine as GAE doesn't support OR operator. However,the IN operator can provide limited OR support.

Also the Query.filter would be changed to follow GAE's Query.filtersignature. The OR and NOT IN operators are not supported in GAE, but the DAL API would implement support for these operators.

So, the following syntax:

Query(User).filter('name = :name and age >= :age', name='some', age=18)

will be replace with:

Query(User).filter('name =', 'some').filter('age >=', 18)

Support of OR operator will implemented as || operator, like:

from kalapy.db import Query, Q

Query(User).filter(Q('name =', 'some') || Q('age >=', 18))

The operator NOT IN would be expanded to (only GAE):

Query(User).filter('name not in', ['some', 'thing', 'else'])

# to

Query(User).filter('name !=', 'some') \
           .filter('name !=', 'thing') \
           .filter('name !=', 'else')

Currently, PostgreSQL and SQLite3 backend engines are implemented. Developmentis in progress to add support for Google App Engine and OpenERP.

Stay Tuned!

KalaPy - A Web Framework

2010-06-15T20:56:00.001+05:30

Hi all,

The Rapido project has been renamed to KalaPy. A new version 0.2 has been released with new package name and an official project website has been launched.

For those who don't know, KalaPy is a new web application framework for Python designed after modularity in mind, provides it's own Database Abstraction Layer, WSGI compatible web components based on Werkzeug, templating using Jinja2 and full internationalisation based on Babel and pytz.

Links:

Regards

Amit Mendapara

The Rapido project is renamed!

2010-06-15T11:04:00.000+05:30

Hi all,

So finally, I have renamed the Rapido project to KalaPy. Thanks you all for your interest and voting for new name. See the poll result here.

The name KalaPy is derived from Sanskrit term Kalapi, means a peacock, a beautiful bird. The name should be pronounced exactly like the original word Kalapi.

The project source repository is relocated to http://github.com/cristatus/kalapy. The PyPI package is located here http://pypi.python.org/pypi/KalaPy.

I am in the process of creating a dedicated website for the project, till then, follow the documentation at github.

Regards

Poll Result!

2010-06-14T10:44:00.001+05:30

Hi all,

Here is the result of the poll requesting you to suggest a new name for the Rapido project.

Rapido	1	7%
Keka	2	14%
Kalapi	1	7%
KalaPy	7	50%
Pavo	1	7%
Peacock	1	7%
Other	2	14%

The poll will be closed in a day. I am going to launch the official project website once we decide on the name. Thank you all for the interest.

Regards

Name the Rapido Project!

2010-06-07T21:49:00.004+05:30

Update: the poll is closed now, see the result bellow

Most of you might be aware about my new open source project which I started with a code name Rapido and released the preview version under the same name on github. But soon I realized that the term Rapido is already being used by some internet companies, so I decided to rename the project.

Here are few names being suggested. Interestingly when I was thinking about a name, I never get any other hints then one related to a bird peacock. I think, this is because of some unforgettable memories of my childhood when I was always mesmerized by this beautiful bird...

Feel free to suggest your own preferred name too!

Poll Result

Rapido 1 6%
Keka 2 13%
Kalapi 1 6%
KalaPy 8 50%
Pavo 1 6%
Peacock 2 13%
Other 2 13%
Regards

It's Rapido!

2010-06-03T23:22:00.002+05:30

Hello everyone,

So finally, it is released. The project was code named Rapido and released with the same name. You can grab the code from github. Please feel free to feedback me with your thoughts/suggestions about how to improve it and what you are expecting in future.

Here is the documentation http://packages.python.org/Rapido.

Regards

Glimpse of Web Components...

2010-05-17T18:26:00.016+05:30

Hello everyone,

UPDATE: I have decided to do something similar to Flask so later, if possible, we can base the web components on top of Flask.

After a careful study of several WSGI tools, I concluded to used Werkzeug as the base of the web components of the proposed framework and I already implemented initial set of web components so that I can start working on BigTable implementation.

Why Werkzeug? Well, it's fully WSGI compliant pure python implementation with no thirdparty dependencies and provides some really nice features like auto-reload, a powerful routing api and an amazing interactive debugger, and most importantly works with Google AppEngine too.

Visit Werkzeug for more information. Finally, Jinja2 would be the templating engine.

Okay, so let I show you how the main web request handlers, would look like.

@web.route('/find/<limit>')
@web.route('/find/<limit>/<offset>')
def find(model, limit, offset=0):
    """Search for resources within the given limits

    Following requests will be routed to this method

    GET /find/user/40/0
    GET /find/user/40/41
    GET /find/user/40/81
    GET /find/user/100

    """
    ...
    ...

@web.route('/get')
@web.route('/get/<key>')
def get(model, key=None):
    """Get a resource identified by given key.

    Following requests will be routed to this method

    GET /get/user
    GET /get/user/1

    """
    ...
    ...
    return web.render_template(
        'template.html', temp=late, var=ables)

@web.route('/save/<model>', methods=['POST'])
@web.route('/save/<model>/<key>', methods=['POST'])
def save(self, model, key=None):
    """Save the post/put data

    Following requests will be routed to this method

    POST /save/user
    POST /save/user/1

    The data can be accessed from self.response.form attribute
    """
    ...
    ...
    return web.redirect(
                web.url_for('/save', model=model, key=key))

@web.route('/delete', methods=['POST'])
def delete(model, key):
    """Delete a resource identified with given key

    Following requests will be routed to this method

    POST /delete/user/1
    """
    ...
    return web.redirect(web.url_for('/user', key=None))


@web.route('/')
def index():
    """Index page
    """
    ...

@web.route('/')
def default(args):
    """All other requests would be dispatched to this method
    """
    ...

The request is exported via the web namespace which is bound to current request context and can only accessed if request is started. There are few more but wait till I release the code...

That's it. Everything else will be handled by the framework including handling of static contents, template rendering, creating wsgi application etc.

Regards

Glimpse of DAL (Database Abstraction Layer) API...

2010-05-13T20:47:00.003+05:30

Hello everyone,

I have finished initial set of DAL api and implemented two backend engines for sqlite3 and PostgreSQL and both the engines have successfully passed the unit tests. Next, I am going to implement BigTable backend engine. It's going to be really interesting as I have seen a possibility to have backend engines for OpenERP and Tryton. I will provide more information later on this but first let I introduce how the API looks like:

Please go through my previous posts on this topic if you haven't read them yet.

class User(db.Model):
    name = db.String(size=100, required=True)
    lang = db.String(size=9, selection=[('en_EN', 'English'),
                                        ('fr_FR', 'French'),
                                        ('de_DE', 'German;)])
    dob = db.Date()
    email = db.String(size=100, required=True, unique=True)
    phone = db.String(size=20)

    @db.validate(email)
    def chk_email(self, value):
        if not re.match('...', value):
            raise db.ValidationError('Invalid email address')

As you know this is how a model class can be constructed. All database related API to be used by applications is exposed via db namespace. Here is the list of the exposed API:

db.Model
db.Query

db.Field
db.String
db.Text
db.Boolean
db.Integer
db.Float
db.Decimal
db.Date
db.Time
db.DateTime
db.Binary

db.ManyToOne
db.OneToOne
db.OneToMany
db.ManyToMany

db.FieldError
db.ValidationError
db.DatabaseError
db.IntegrityError

db.get_model
db.get_models
db.validate     # a decorator to assign a validator method to a field
db.unique       # a model class helper to define combined unique constraints
db.meta         # a model class decorator to change meta information like model name

>>> @db.meta(name='base.user')
>>> class User(db.Model):
>>>     ...

db.transaction.enter_transaction
db.transaction.leave_transaction
db.transaction.commit
db.transaction.rollback
db.transaction.run_in_transaction

Here is the Model class api:

# properties
Model.is_saved
Model.is_dirty

# methods
Model.__init__(**kw)
Model.save()
Model.delete()
Model.get(key)
Model.all()
Model.select(*names)
Model.fields()

Here, Model.all() returns an instance of Query, and Model.select(*names) returns list of tuple of the values for the matched records. For example:

User.select('name', 'lang').filter('lang == :lang', lang='fr_FR').order('-name').fetch(-1)

Will return a list of tuple (name_value, lang_value) of the matched records. It is similar to:

SELECT "name", "lang" FROM "base_user" WHERE "lang" = "fr_FR" ORDER BY "name" DESC

The Field class exposed following methods:

# methods
Field.__init__(...)
Field.validate(value)
Field.empty(value)
Field.default_value()
Field.python_to_database(value)
Field.database_to_python(value)

# properties
Field.name
Field.data_type
Field.default
Field.is_required
Field.is_unique

You can see the Field api is self explanatory and much similar to the most of the existing ORM library. Now let's see the Query class.

Query.__init__(model)
Query.filter(query, **params)
Query.order(spec)
Query.count()
Query.fetch(limit, offset=0)
Query.delete()
Query.update(**kw)

If you are familiar with Django or Google App Engine this is similar to the query object available in those frameworks and works almost similar. Let's see an example:

q = Query(User) # or q = User.all()
first_ten = q.order('-name').fetch(10)

q = q.filter('name in :names', names=['some', 'one', 'else'])
all_matched = q.fetch(-1)

q.update(lang='fr_FR')
q.delete()

The last two method calls resembles:

UPDATE "base_use" SET "lang" = "fr_FR" WHERE "name" IN ("some", "one", "else")

-- and

DELETE FROM "base_user" WHERE "name" IN ("some", "one", "else")

So these are the most common api exposed to the user to be used in applications

The API for backend engines are kept hidden from outside world in a subpackage db.engines which must be imported manually if one wishes to use that api. A backend engine should be implemented using an interface class defined as IDatabase and the engine should expose the implementation as Database. Besides that an engine should also expose engine specific DatabaseError and IntegriryError classes.

The IDatabase defines following interface:

IDatabase.__init__(...)
IDatabase.connect()
IDatabase.close()
IDatabase.commit()
IDatabase.rollback()
IDatabase.cursor()

IDatabase.get_data_type(field)

IDatabase.exists_table(name)
IDatabase.create_table(model)
IDatabase.alter_table(model, name=None)
IDatabase.drop_table(name)

IDatabase.update_records(*objs)
IDatabase.delete_records(*objs_or_keys)

IDatabase.select_from(query, params)
IDatabase.select_count(query, params)

You can see the interface is very simple but few more interface methods are expected in future to support enhanced query syntax.

Besides the DAL api I have also implemented test cases and few helper classes and functions to easy writing new test cases. The framework is also going to provide Django style command line admin tools to do common administrative tasks. See all theses things in the first alpha release expected be made public in the first week of the next month. Stay tuned...

Regards

More on DAL...

2010-05-04T15:52:00.006+05:30

Hello again,

I have just implemented few more interesting features to DAL (Database Abstraction Layer). The relational fields and the way to interact with them. The DAL now supports ManyToOne, OneToOne, OneToMany and ManyToMany relationships. Let me explain with few examples:

class User(db.Model):
    name = db.String(size=100, required=True)
    dob = db.Date()
    groups = db.ManyToMany('Group', reverse_name='members')

class Group(db.Model):
    name = db.String(size=100, required=True)
 
class Address(db.Model):
    street1 = db.String(size=100, required=True)
    street2 = db.String(size=100)
    city = db.String(size=100)
    zip = db.String(size=20)
    state = db.String(size=100)
    country = db.String(size=4, selection=get_country_names)

    user = db.ManyToOne(User)

Here the User has a many-to-many relationship with Group represented by groups field. Defining an M2M relationship using db.ManyToMany with reverse_name attribute, the target model will automatically get a ManyToMany field with that name pointing to the respective model. In this case Group will get an M2M field members pointing to User.

Similarly, Address model has one ManyToOne field named user pointing to model User. In result, the User model will automatically get a OneToMany field pointing to Address with name address_set. M2O field also excepts reverse_name attribute if you want to use another name.

Now, let's check few more cases

class Car(db.Model):
    name = db.String()
    model = db.String()
    engine = db.OneToOne('Engine')
    parts = db.OneToMany('Part')

class Engine(db.Model):
    name = db.String()

class Part(db.Model):
    name = db.String()

Here Car & Engine has one-to-one relationship represented by the field engine. A reverse lookup field car will be automatically created in Engine. Also, Car has one O2M field parts pointing to Part, so Part will automatically get an M2O field named car pointing to Car.

Now let's see how it can be used:

u = User(name="some")
u.address_set.add(Address(street1="st1"))
u.save()

Similarly, you can add records to M2M field. There is not need to save the records but later the parent instance will take care of saving all unsaved related instances.

g1 = Group(name='user')
g2 = Group(name='admin')

u.groups.add(g1, g2)
u.save()

Here g1, g2 are not saved and added to u.groups, these unsaved instances will be saved by u.save.

The related model instances are not directly accessible via field variables of type OneToMany and ManyToMany but one should use any of the provided methods to those fields. Like:

# get query object of all addresses and fetch first 10 records
u.address_set.all().fetch(10)

# remove an address
u.address_set.remove(a)

# clear all addresses
u.address_set.clear()

On the other hand, ManyToOne and OneToOne fields directly points to the related instance.

c = Car(name='Nano')
e = Engine(name='Micro')
c.engine = e

c.save()

print c.engine.name
print e.car.name

Notice that we haven't saved e but just c. Yes, the DAL API will take care of saving all the related objects if they are modified.

Besides that support for unique constraints have been improved, see:

from xyz import db
from xyz.db import has_unique

class Some(db.Model):
    a = db.String()
    b = db.String()
    c = db.String()
 
    has_unique(a, [b, c])

Here we have added two unique constraints to model Some. One for a and a combined unique constraint for b and c.

That's it for now, more details as the work progresses so keep watch on my blog.

Regards

The Database Abstraction Layer (DAL)

2010-04-30T12:35:00.007+05:30

Hello everyone,

This is the third and last article I promised in the answers to my friends who are curious to know about what I am doing right now. I am not going to repeat introductory text again and simply jumping on what the Database Abstraction Layer (DAL) after all and how it would work.

The framework is being implemented keeping scalability in mind as well as support for wide range of deployment options including clouds. Other goals are keep it simple, do it your self, keep the thirdparty dependencies minimal.

So the first obvious thing that comes in mind is the database support ranging from RDBMS to highly scalable NoSQL DBMS to OODBMS. We have our reasons to choose one or another DBMS, be it cost, performance or something else. We all know how difficult it is to change the options once we deploy the application. Yes there are ORM available that can solve this issue but most of them only supports RDBMS. I am not going to explain pros and cons of theses available ORM but will provide you some glimpse on how the proposed DAL would work and easy your way. It's up to you to evaluate it against existing solutions.

Let I first show you how the DAL API looks like:

from xyz import db

class User(db.Model):
    name = db.String(size=100, required=True)
    dob = db.Date(requied=True)
    email = db.String(size=100, required=True, unique=True)

    @db.validate('dob')
    def check_dob(self, value):         
        # if calculated age is less then 18 raise ValidationError
        pass

class Article(db.Model):
    title = db.String(size=100, required=True)
    publish_date = db.Date(default_now=True, required=True)
    text = db.Text(required=True)
    user = db.ManyToOne('User', required=True)

class Comment(db.Model):
    title = db.String(size=100, required=True)
    comment_date = db.Date(default_now=True, required=True)
    text = db.Text(required=True)
    article = db.ManyToOne('Article', required=True)
    user = db.ManyToOne('User', required=True)

    @db.validate('text')
    def check_comment(self, value):
        # check value for blog compliance
        pass

Sounds familiar, isn't it? The first import statement says `import from xyz`. Yes I still haven't decided on the name.

Next, you must define your schema as python classes derived from db.Model. The DAL is there to enforce the schema in the case of underlying backend DBMS doesn't support schema, like BigTable and some other schemaless DBMS while giving that task to the DBMS is it supports schema.

Database tables declared as subclasses of `Model` defines table properties as class members of type `Field` (db.String, db.Date, db.Text are all kind of db.Field). So if you want to publish an article with title, body and date, you would do it as shown above.

Some DBMS doesn't support constraints so validation is left as the responsibility of programmer. The DAL will take care of `required` and/or `unique` constraints as well as `referential integrity` if the DBMS doesn't support it.

Till now it just looks similar to `django` models but wait let I show you one more interesting feature of the proposed Model API, inheritance.

class A(db.Model):
    a = db.String()

class B(db.Model):
    b = db.String()

class C(A, B):
    c = db.String()

This is not supported. To keep the DAL simple, multiple inheritance (of Model) is not supported. This is allowed in `django` though. Let's see how inheritance works with DAL.

class A(db.Model):
    a = db.String()

    def do_something(self):
        pass

class B(A):
    b = db.String()

class C(B):
    c = db.String()

    def do_somethinf(self):
        super(C, self).do_something()
        # do something else

class D(A):
    d = db.String()

    def do_something(self):
        supper(D, self).do_something()
        # do something here

Here, B is derived from A, C from B and D from A but wait, this is not the case. The DAL is implemented in such a way that no matter which model you inherit from, it always inherits from the last defined derived class of the first defined parent. That is, the hierarchy would be:

A -> B -> C -> D instead of A -> B -> C and A -> D

Another interesting feature is, no matter which class you use to instantiate a model, you will have an object of the latest defined class in the hierarchy. Why this is so? Let I explain:

Suppose your application is using the above defined `User` model class like this:

user = User(**kwargs)
user.do_something()
user.save()

Where `kwargs` is a `dict` of form variables coming from an http post request.

Now if you think that `User` should have one more property `lang` but you don't want to change your running system by modifying the source code, you simply create a subclass of `User` and all the methods/members defined in that subclass will be available to the application.

class UserEx(User):
    lang = db.String(size=6, selection=[('en_EN', 'English'),
                                        ('fr_FR', 'French'), 
                                        ('de_DE', 'German')])

    def do_something(self):
        super(UserEx, self).do_something()
        ...

So now if the html form has `lang` field, the above code will work without any change and still saving `lang` value. You can also change the behavior of the base class by overriding methods. This will allow you to implement highly modular applications with the proposed DAL API.

This is somewhat similar to `openerp` osv objects. If you familiar with `openerp` then you should be aware of how easy it is to extend existing data model by simply creating and osv class with `_inherits`, but as far as I know, the behavior of `openerp` orm is not fully object oriented as shown here.

Another problem of supporting almost all kind of database is query language support. RDBMS supports SQL while NoSQL DMBS might have their own query language. The DAL doesn't support SQL at all but sill it provides a way to filter the records. Let's see:

query = User.all().filter('name = :name and dob <= :dob', 
                        name='some', dob='01-01-1992').order('-dob')
users = query.fetch(10)

for user in users:
    print user.name, user.dob

The above query is equivalant to:

SELECT * FROM "user"
WHERE "name" LIKE '%some%' AND "dob" <= '01-01-1992'
ORDER BY "dob" DESC LIMIT 10

The syntax is again similar to `google appengine` ORM with same restrictions. You can only perform search on single table, you can't use literal in query string but use named parameters which are bound to the provided parameters.

So this is how the DAL API would be, I will explain it with more information later. I am enabling anonymous comments for this post to see what you think about it. I will be happy to see your useful suggestions and ideas.

I will give more information on the DAL and the other components of the framework as it continues to reach to the first prototype stage. Stay tuned...

Regards

Next big thing!

2010-04-20T16:58:00.002+05:30

Hi,

This is the second post in the series of three articles I promised last time.

So let I inform you that I am working on a very interesting project and it will be released under an opensource license when first prototype is ready with a simple usecase.

So let first I tell you about how I come to the decision to start such a project before telling you anything more about it.

I had no other work when I left my previous job so I catch this opportunity to check several technologies like Google App Engine, django, Amazon EC2 and various schema less database solutions like BigTable and SimpleDB.

As I was previously working with the great openobject framework, I suddenly started thinking about the possibility of having an application framework that can be deployed on variety of platform but should be as powerful as openobject and django.

Obviously, openobject is designed specifically for PostgreSQL, an open source RDBMS, it is not easy to deploy openobject applications on such a highly scalable platforms like google apps or amazon ec2 effectively.

Also django data model is designed keeping RDMBS in mind. Though django project is working on a very interesting feature for their ORM, connection routing, where you can configure your django application to use replicas of your database as a writable master and several readonly slaves. This will allow you to scale your application datastore without much efforts and improve fault tolerance and responsiveness of the application.

As far as scalability is concerned, there is nothing wrong with RDBMS if you have good infrastructure and workforce to maintain it. But I am sure you can understand how much effort it requires to maintain such systems.

The proposed application framework will allow you to implement highly scalable web applications that can be deployed on variety of platforms with any kind of database system, be it an RDBMS or NoSQL databases like BigTable or SimpleDB and even with OODBMS like zodb.

The framework is being designed after django and openobject patterns. It is not a clone nor a hybrid of these frameworks but being developed from scratch. The core of the framework is it's DAL (data abstraction layer) API similar to django ORM in look and feel but works somewhat similar to the ORM of the openobject framework.

An application (called project in django context) of this proposed framework consists of one or more modules (called apps in django context). Each module defines models, logic and views and the framework will take care of exposing the resources to the web in the form of web services or HTML based web interface. And all this with very little efforts as you do with openobject.

The next post will be on the DAL API. Be in touch...

Regards

It's alive again...

2010-04-17T13:42:00.003+05:30

Hi,

After a long silence my blog is alive again. Actually, I am not much talkative person nor good at all with writing skills. Please ignore the mistakes I do and correct me if possible.

This is the first of the three posts I am planing to write within next few weeks about what I am currently working on. The first one is about the response to my friends who wondered when I left my previous job and asking me the reasons and what could be my next destination.

I was a main engineer and team leader of a small development team of the web client project, part of openobject, an open source application framework. I really enjoyed nearly three years working there on such a wonderful project. It was really hard to decide to leave especially at the crucial stage of the project but I had no other choice. To be honest, it's all personal reasons.

When I left, I was not clear about what I should do next, whether to join another company or going back to my previous freelancing style work. Still I am not clear, but during these few weeks I am doing some research and come to a decision to start something interesting, and this is what I am going to explain by the next two posts. So be in touch...

Again, I would like to thank all of my colleagues for the all well wishes and love.

Thank you all

Thank you!

2010-03-29T12:50:00.000+05:30

Hello friends,

I am very very thankful to you all for the grand farewell, your gratitude and love. The respect you have given is unforgettable and unbelievable...

Thank you all again...

Path to 5.0 - The View Editor!!!

2008-10-10T20:32:00.014+05:30

Welcome again,

So as promised, this is the second post to give you some glimpse of one the most exciting new feature of upcoming version 5.0 of the web client, the View Editor. Which lets you change the view of any resource easily.

Let's see the view designer in action...

You can see, how easy it is to customize objects and their views. The changes made with the view editor will be saved for all users.

Additionally, in this upcoming major release, you will see some RAD tools included with the web client like Object Designer and Workflow Designer. With these tools, you can customize the OpenERP without writing a single line of code (more on this later)...ter

Path to 5.0

2008-09-26T12:32:00.014+05:30

Hello Everyone,

It has been long since I have talked on any new features of the OpenERP Web Client (formerly eTiny). We are working hard to make the OpenERP one of the most advanced OpenSource ERP ever made, and as a result we are going to publish the next major version 5.0 of the OpenERP loaded with tones of new features within next two months. As you all know, various components of OpenERP is now hosted on the OpenSource development portal Launchpad.net as OpenObject framework to work closer with the community and partners.

Being an active member of the OpenERP community, some of you might have ideas on what's going on with the OpenERP. I will publish few blogs on new features to keep you informed. So this one is the first of them. Today, I will tell you about how to test the bleeding-edge version of OpenERP (the trunk version).

Here are few quick steps (Ubuntu Linux). Please go through the forum and the wiki for detailed help:

$ sudo apt-get install postgresql
$ sudo apt-get install python-xml
$ sudo apt-get install python-lxml
$ sudo apt-get install python-libxml2
$ sudo apt-get install python-libxslt1
$ sudo apt-get install python-psycopg2
$ sudo apt-get install python-imaging
$ sudo apt-get install python-pyparsing
$ sudo apt-get install python-reportlab
$ sudo apt-get install python-matplotlib
$ sudo apt-get install python-pyopenssl
$ sudo apt-get install python-pychart
$ sudo apt-get install python-pydot
$ sudo apt-get install python-tz
$ sudo apt-get install graphviz
$ sudo apt-get install ghostscript

$ sudo apt-get install python-gtk2
$ sudo apt-get install python-glade2
$ sudo apt-get install python-hippocanvas
$ sudo apt-get install python-egenix-mxdatetime

$ mkdir openerp-trunk
$ cd openerp-trunk
$ bzr branch lp:openobject-server server
$ bzr branch lp:openobject-addons addons
$ bzr branch lp:openobject-client client
$ bzr branch lp:openobject-client-web web

The web client requires TurboGears 1.0.3.2 or later, if your distribution provides packages for this version you can install TG with apt-get, but we recommend to use the latest stable version of TurboGears 1.0.x series. See http://docs.turbogears.org/1.0/InstallNix for more information.

$ sudo apt-get install python-setuptools
$ sudo easy_install TurboGears==1.0.7

Check whether TurboGears is installed properly or not:

$ tg-admin info

It will show you information about various components of TurboGears and it's dependencies.

Before you start OpenERP server, create a role in PostgreSQL without any password with the same name as the Linux user who is going to run the openerp-server:

$ sudo su postgres -c "createuser -d -S -R terp"

Now start the OpenERP server:

$ cd openerp-trunk/server/bin
$ sudo su terp -c "./openerp-server.py --addons-path=../../addons"

You will see a message saying `INFO:web-services:the server is running, waiting for connections...`

Now start the OpenERP web client:

$ cd openerp-trunk/web
$ ./start-openerp-web.py

You will see a message saying `INFO HTTP: Serving HTTP on http://0.0.0.0:8080/`

That's it!!!

You are ready to test the bleeding-edge version of OpenERP Server and the Web Client. As part of the community, we welcome inputs from you by means of bug reports, suggestions and translations to improve the OpenObject projects. Go though the Launchpad project site for more information...

Custom addons path (advice for developers)

2008-08-20T15:29:00.003+05:30

As said in my previous post, now it's possible to place addons outside the bin directory. To achieve this I have implemented few helper function in addons module to get the actual path of the module or other resources (files especially).

For example, if you want to locate your_addon/report/some.xslt irrespective of where the addons is located, you should use addons.get_module_resource function instead of hard coding the filepath.


import addons
myfile = addons.get_module_resource('your_addon', 'report/some.xslt')

One more important change in tools.file_open, though transparent is worth to check. Here is the docstring of the modified method...


def file_open(name, mode="r", subdir='addons', pathinfo=False):
    """Open a file from the Tiny ERP root, using a subdir folder.

    >>> file_open('hr/report/timesheer.xsl')
    >>> file_open('addons/hr/report/timesheet.xsl')
    >>> file_open('../../base/report/rml_template.xsl', subdir='addons/hr/report', pathinfo=True)

    @param name: name of the file
    @param mode: file open mode
    @param subdir: subdirectory
    @param pathinfo: if True returns tupple (fileobject, filepath)

    @return: fileobject if pathinfo is False else (fileobject, filepath)

Please change you modules accordingly to make them compatible with the new custom addons path...

Custom addons path...

2008-08-05T16:31:00.003+05:30

With the latest trunk, there is no need to put addons under the bin directory. Now you can put addons in separate directory. You can start OpenERP server like this:


$ ./tinyerp-server.py --addons-path=/path/to/addons

The bin/addons will be considered as default addons directory which can be overridden by the /path/to/addons. That is, if an addon exists in bin/addons as well as in /path/to/addons (custom path) the later will be given preference over the one found in bin/addons (default path).

See the Stephane Wirtel's blog on how to get the latest version of the server from the Launchpad.

What's going on?

2008-07-30T15:53:00.009+05:30

Hello everyone,

Since the release of the first stable version (v1.0) lots of new things have been added to the OpenERP Web Client (formerly eTiny). You might already have observed those if you have tried the development version of the OpenERP Web Client. There are lots to say about the new features, but the most noticeable are:

1. Buttons in ListGrid (similar to forms)

Now you can place a button in Tree view (ListGrid) similarly what you can do with Forms.

<tree>
   <field name="name" .../>
   ...
   <button name="cancel" .../>
</tree>

2. Localized datetime/number format

The long awaited localized datetime/number format has been implemented.

	en_EN	fr_FR	es_ES
Date	30-07-2008	30/07/2008	30/07/2008
Number	3,000.00	3 000,00	3.000,00

3. View Editor

A simple yet powerful view editor allows users to change/design views. See the view editor in action.

4. Object Designer

With Object Designer, one can create custom OpenERP objects or can change existing ones. See the object designer in action.

5. Workflow Editor

One can change/create workflows visually with this powerful tool. See the workflow editor in action...

6. Dashboards

Customizable dashboards per user. User can change the dashboard either using the view editor or arranging the dashlets with drag & drop...

7. A new `attrs` attribute similar to `states`

The new attrs attribute can be used to dynamically change other attributes of view components based on the values of other fields. It's somewhat similar to states attribute, but is clientside only.

<form string="Some Form">
   <notebook>
       <page string="Main">
     <field name="name" attrs="{'required': [('state','=','draft')],
                                        'readonly': [('state','=','open')]}"/>
           <field name="active" .../>
     <field name="state" ...>
           <group attrs="{'invisible': [('active', '=',0),
                                        ('state','=','draft')]}">
               ...
           </group>
       </page>
   </notebook>
</form>

The required and readonly attributes of the field name then depends on the value of fields active and state. Similarly, the visibility of the group depends on active field.

Visibility will only be supported on buttons, groups and pages to maintain form layout in OpenERP Web Client. Readonly/Required attributes will be supported for simple widgets only and not complex ones.

8. Flash Charts

Charts/Graphs have been replace with the beautiful flash charts using Open Flash Charts.

9. Speed Improvements

Lot's of improvements here.

Rendering speed of very big forms has been 4 times faster now.
Switch O2M using Ajax (no page reload)
Client side validation for required fields (no page reload)
Editable ListGrid (full Ajax)

And lots more...

As you all know, we are in process of releasing the next stable version of OpenERP (aka 4.4). We are going to change the version number of Web Client to reflect the corresponding server version. That is, the next version of the Web Client will be v4.4.

I am eager to see your suggestions/views/ideas to improve it's usability and eargonomy...

eTiny 1.0.1 released...

2008-06-26T12:52:00.003+05:30

We are happy to announce the first bug fix release (v1.0.1) of the stable version 1.0 of eTiny. The web client of OpenERP, the Enterprise Management Software.

OpenERP is one of the most advanced Open Source ERP, known for it's simplicity and adaptability. eTiny brings the most powerful ERP to the web mode, giving you more performance, flexibility, simplicity and even more accesibility.

ChangeLog:

Fixed navigation problem (now remembers the offset of O2M widgets)
Safari 3.0 issue (possible bug in Safari)

Safari submits selection fields even if name attribute is missing.

Better calendar view

Rendering issues (FF3)
Tangoish colors
Item sorting problem in month view

Fixed onchange method for many2one.
Fixed (#767), fails to load the translation catalog if installed as egg.
Fixed o2m view record problem (bug #757)
Fixed #696, 'View Log' on User creates error
fixed sidebar link translation problem
Fixed `Reference widget`, now it's an extended `M2O` (using same JS).
Form view parser now checks for duplicate fields.
Fixed `editable list view` in popup (unable to get values of parent fields)
Visibility of Manage button in configuration.
Fixed one2many (default_get_context)
Improved Cache system
Cache for i18n catalog to speedup the `gettext` operations.
Fixed Import/Export bugs (649, 650)
Fixed Calendar view (handle server errors)

Links:

For more information, visit http://openerp.com...

eTiny 1.0 (stable) released...

2008-02-25T11:34:00.003+05:30

We are glad to announce the release of the first stable version 1.0 of eTiny. The web client of Tiny ERP, the Enterprise Management Software.

Tiny ERP is one of the most advanced Open Source ERP, known for it's simplicity and adaptability. eTiny brings the most powerful ERP to the web mode, giving you more performance, flexibility, simplicity and even more accesibility.

Highlights:

High Performance
Easy deployment
Clean & Ergonomic
Ajax enabled
New Calendar View mode
Complete French translation
Open Source (GPL)

Browser Compatibility:

Firefox 1.5/2.0
Internet Explorer 6.0/7.0
Safari 3.0
Opera 9.0

Links:

For more information, visit http://tinyerp.com...

eTiny 1.0 Screenshots

2008-02-22T13:27:00.006+05:30

Login Screen

Dashboard

Main Menu

All Sales Orders

Sales Order Form

Cases Form

Project Tasks (Calendar View) - Month Mode

Project Tasks (Calendar View) - Week Mode

Project Tasks (Calendar View) - Day Mode

Validation

Editable ListGrid

Forms & eTiny

2008-01-05T14:11:00.001+05:30

If you have used eTiny, you might have observed that some views don't render well with eTiny while it just looks perfact with the traditional GTK+ client.

Today, I will describe how to design Tiny ERP form that renders perfact with eTiny. For detailed information on view specification, please explore the developers guild on the Wiki.

Here is an example form view:

<form col="4" string="Form Title">

    <field name="field1" .../>
    <field name="field2" .../>

    </newline>

    <separator string="Object State" colspan="4"/>

    <field name="field3" .../>
    <button name="open_button" states="draft,cancel .../>
    <button name="cancel_button" stats="draft,open" .../>

</form>

With GTK+ Client, the resulting form should look like:

But with the web client, you might see bad looking result like.

The problem is due the placement of open_button in column 3, which is actually used by previous row to place label for field2. As eTiny tries to allocate minimum possible width to labels and maximum possible to widgets, column 3 will get maximum possible value due to the placement of button widget.

To overcome with the issue, I suggest you to modify your view as:

<form col="4" string="Form Title">

    <field name="field1" .../>
    <field name="field2" .../>

    </newline>

    <separator string="Object State"/>
    
    <field name="field3" .../>

    <group col="2" colspan="2">
        <button name="open_button" states="draft,cancel .../>
        <button name="cancel_button" stats="draft,open" .../>
    </group>

</form>

Now your view will be rendered perfectly with both the clients...

GTK+

eTiny

In short, you should use group whenever you put buttons in your view. And don't put any widget with colspan=1 in the column which is supposed to be used by labels.

eTiny 1.0rc2 is released...

2007-12-27T16:17:00.000+05:30

We are happy to announce the next release candidate version of eTiny with lots of improvements and bug fixings...

New Features:

Image Widget (type=binary widget=image)
M2M for editable listview
Duplicate Record
O2M with pagination support (list mode)
FloatTime widget
ContextMenu (CTRL+MOUSE R) for all fields

Changes:

Sidebar Links (Reports/Wizard) now opens in new window
View Cache for production environment only
Improved Form Layout
and bug fixings...

Visit http://tinyerp.com for more information...

eTiny 1.0rc1 is released...

2007-11-06T03:39:00.000+05:30

We are glad to announce the first release candidate version of eTiny - Web Client of Tiny ERP, the Enterprise Management Software jointly developed by Tiny and Axelor.

Features:

High Performance
Light weight
Easy deployment
Clean & Ergonomic
Ajax enabled

Browser Compatibility:

Firefox 1.5/2.0
Internet Explorer 6.0/7.0
Safari 3.0
Opera 9.0

Performance:

Maintains cache of views to improve performance. We have achieved very high performance even on an old P4 system.

Calendar View:

eTiny supports new Calendar View, which let you manage your time based data easily.

Move/Resize calendar items with Drag & Drop.
Quickly add/edit/delete items.
You can view your data in day, week or month mode.
Filter data using search filter or legends...

Editable List View:

New editable list view allows you to quickly add/edit records without switching to form view. Additionally list views are now sortable, you can sort any column by clicking on the header area.

and much more...

Visit http://www.tinyerp.com for more information.