Django optimisation in production
Few days ago I realised that my django installation in production started producing too many memory errors – thanks to monit memory alerts were filling my mailbox without any serious service interruption. But hundreds email messages daily annoyed me enough to look deeper into trouble.
1) I switched on django debug true, noticed a large delay on SQL request. Also enabled in my.cnf slow-log part with number 5.
Solution one: Added db_index=True to the model.py to the fields used in where clauses. Indexes which I created with help of ./manage sqlindexes didn’t work, so I had to login into dbshell and perform EXPLAIN and CREATE INDEX manually.
2) Added distinct() to most of my queries – I didn’t bother to filter duplicate queries, because values() was doing it for me.
3) Added db.reset_queries() to the top of my view.py, because I deploy django via fcgi with nginx frontend, django was tracking queries for a “session” which is never ended.
4) I didn’t notice any significant performance improvement until I decided to create a cache of sql requests, put /manage.py createcachetable my_cache_table and then went to settings.py.
OMG:
- CACHE_BACKEND = "locmem:///?max_entries=3000"
which is the worst cache settings you can have for django in production. changing it to
- CACHE_BACKEND = "db://my_cache_table"
produced serious performance improvement. (Yes, I can put memcached or redis if I need it, but I don’t think I need it at the moment – my traffic is google bot.)
No tags for this post.