littlelogs

Keep a social journal of your work progress as you make and learn things.

#postgres

October 2015

josh
josh

Successfully copied a copy of the live #exist database from #mysql to #postgres using pg_loader. Woo! I still can't quite believe it happened as quickly and smoothly as it did (database is several gigs). I should even be able to switch the live database over without taking the site down. For now I'll just work with making sure postgres is fine on my dev server, and plan my next steps.

josh
josh

Today I'm starting to look at how I can bring the current #exist codebase in line with the functionality I wanted out of the planned exist-core v2, rather than rewriting and being susceptible to the Second System Effect. This involves:

  • Migrating from #mysql to to #postgres
  • Adding the new event stream table with #json-b metadata field
  • Potentially migrating to #python 3 (perhaps relying on 2to3?)
  • Pruning away a lot of accumulated cruft that wouldn't have appeared in v2
  • Potentially open-sourcing the result

Right now I'm on the first step. I'm looking at using pgloader to migrate data into postgres. In the past I've run into issues migrating to postgres (and so quickly gave up) but I'm hoping pgloader will be my shining light.

September 2015

josh
josh

Having issues with #littlelogs private groups and dynamic databases. Basically I can't dynamically route queries to databases that haven't been predefined in settings, because django-dynamic-router seems to have threading issues (at least in my setup) and the selected database doesn't stick. A pretty fruitless day of work. Going to look at django-tenant-schemas and weigh up switching to #postgres versus some other alternative (maybe generating new DB setting files and restarting django to include them). Still very leery of keeping everyone in the one database (or schema).

August 2015

josh
josh

More #postgres testing and wrangling today for my #exist experiment. Found some handy functions like date_trunc and generate_series that will be handy when doing things like finding the most popular hour for items, maybe even some k-means clustering.

February 2015