littlelogs

Keep a social journal of your work progress as you make and learn things.

josh
josh

Today I'm starting to look at how I can bring the current #exist codebase in line with the functionality I wanted out of the planned exist-core v2, rather than rewriting and being susceptible to the Second System Effect. This involves:

  • Migrating from #mysql to to #postgres
  • Adding the new event stream table with #json-b metadata field
  • Potentially migrating to #python 3 (perhaps relying on 2to3?)
  • Pruning away a lot of accumulated cruft that wouldn't have appeared in v2
  • Potentially open-sourcing the result

Right now I'm on the first step. I'm looking at using pgloader to migrate data into postgres. In the past I've run into issues migrating to postgres (and so quickly gave up) but I'm hoping pgloader will be my shining light.

mattimck
mattimck

Wow, this is super interesting! If I was back in Melbourne, I'd want to pick your brain over coffee on this decision! Is this the first mention of it being open, or did I miss something a month ago? Very exciting, regardless... Good luck! Can't wait to see what comes of it!

21 Oct 2015

josh
josh

Thanks! I guess you missed it, and I had trouble looking back for it too, but here's the initial announcement log with some reasons on why.

Unfortunately it made more sense when I was writing it from scratch, as it's really only the core (DB, API, and data processing stuff) that I want to open-source, keeping the Exist front-end closed as it currently stands. It will be hard to split up the current codebase along those lines. But I do care about making it open (scary as that is) so hopefully I can make it work.

I'd be happy to discuss it too, if/when you're back ;) I might write a blog post about some of my reasoning in the meantime.

21 Oct 2015

mattimck
mattimck

Yeah, definitely missed at. Anyway, awesome stuff... You two continue to impress me :) Look forward to the blog post at some point.

I'm imagine @ginatrapani would have some good experience on the balance this type of thing requires - although from memory I think she started in the opposite direction with #thinkup (ie. open source first).

21 Oct 2015

ginatrapani
ginatrapani

Yep, I've only done open source first (with proprietary layers later), which is easier than retrofitting existing code to open source. I have a lot of mixed feelings about my experience open-sourcing code--it's so valuable, and yet sooo much work and overhead. I salute your efforts!

21 Oct 2015

josh
josh

That's pretty much how I feel — it's important, but it's going to be so much work! Thanks. I hope I can pull it off.

21 Oct 2015