NoSQL, a DBAs Perspective

In order to summarize this post I'll just start with this. We will shortly have Redis deployed in our production environment. I can't really tell you how happy this makes me.

As a DBA, and more importantly an "Open Source DBA", I find many of the new NoSQL options intriguing.

MongoDB:
I've been following the MongoDB project for a while now, and while it's not my cup of tea it does seem very interesting. I've been a web developer for 12+ years and can see where some might think it's a perfect fit, but the traditional developer in me wants data structures that make more sense to programmers. Developers have been struggling with SQL for so long it seems odd and off-putting to replace it with JSON and another query language. It just doesn't seem like the way to go. ActiveRecord is very popular because it feels like regular OOP and fits the developer mindset.

Redis and Ohm:
Wow. It's almost like an epiphany. All my data structures including queues, hashes, arrays, even time series data(sorted lists) stored conveniently in a wicked fast central location. It's like the grail for programmer happiness. Developers don't have to learn anything new. Great libraries handle the interaction with the DB and make it feel as natural as possible to work with these data structures.

All I can say is watch this space. If you're reading this and haven't been following the Redis project. Go check it out. You'll be glad you did. I've been looking for an optimal mechanism for storing time series data for 10+ years now. I'm a huge telemetry geek and Redis with sorted sets is a game changer.

Hashes for storing variable length object data is sweet too. The newest Ohm libraries are using this new functionality and I'm sure the speed is amazing. Forking to save the data is a bit memory intensive for large sets, but it's a nearly optimal solution to the problem of persistence.

I could go on but I won't. You can probably tell I'm very interested in Redis and look forward to many projects using this novel new tech.

I suspect that SQL databases are here to stay, but options are always welcome. If NoSQL does end up taking over the world I'm one DBA who's happy to go back to dev and use these great new alternatives.

Slight delay

Well you may have noticed that the 15th came and went. There's been a slight delay in a datamapper release I've been waiting for. The current release uses objectspace features and those of you familiar with JRuby know that it adversely affects performance if you use them. The DM folks promise that the fix has already been applied and are rolling a release in the coming week(s?).

As soon as this happens I should be ready to roll some gem releases of my own.

I've been thinking of ripping out echoe in favor of just gemcutter anyway. Maybe I'll use the extra time to do that. If nothing else it would be nice to have both projects maintained in the same way.

Gonna try this blogging thing again

Now that I finally have something to offer. I'm going to try this blogging thing again.

I have two gems in the works. One is a framework called CommonThread for message oriented middleware in JRuby. The other is a JRuby wrapper for SNMP4J called SNMP4JR.

I just opened fresh new Lighthouse accounts for both and am ready to start working on public releases. I'm shooting for May 15, 2010 for a public gemcutter release of both.

For those of you who have tried/use phpMyTop or MyTop.rb I have updates coming for you all as well. I'll probably be moving them from sourceforge to github however. I see no reason to split my projects up. I'll shoot for June 1 for these. The fresh new versions are actively in use here at Manheim, but need some love for public consumption again.

The big development push is something I've been working on for years now. CommonThread is going on 6 years old now and has been significantly reworked this year for JRuby. Ruby never had good enough threading support to warrant a public release and JRuby wasn't mature enough(IMO) until recently. We now have a thread safe ORM in Ruby/JRuby called DataMapper. I've stripped out ActiveRecord in favor of it, however you can use any thread safe ORM you like. I tried Ohm with it for a bit, but had some threading issues with redis(that have been fixed now). If you need extra performance I strongly suggest it. I'll be adding queue support for Redis in the near future as well as JMS once I complete my JRuby conversion of the framework.

The goal is to use CommonThread and SNMP4JR . . as well as Jmx4r to build a fantastic database management/monitoring tool that surpasses MyTop.rb. My goal is a public release by the end of the year. So I'll be using the gems I'm releasing for a substantial project, much like the folks at 37Signals.

I'm going to blog along the way so watch this space.