Friday, August 2, 2013

Gearmand 1.1.9 Released

The basics:


  • Added gearman_task_is_finished()
  • Improved SSL support.
  • Exceptions are now supported.
  • gearmand excepts its root CA via the environmental variable GEARMAND_PORT. 
  • libgearman will now except GEARMAND_CA_CERTIFICATE, GEARMAN_CLIENT_PEM, and GEARMAN_CLIENT_KEY


http://launchpad.net/gearmand

From the feedback I am getting the SSL appears to be working.  If you are interested in learning about how the SSL works, you can find out more about it on the docs website.

I am curious to see how many people get tripped up over the fact that an SSL setup requires that you not use self signed certificates (I may support these later, but I wanted to get it right in the beginning).

Unless something major comes up the next version will be a release candidate and the version after will be the formal release :)

Tuesday, March 12, 2013

MySQL vs NoSQL vs Postgres vs SQL Server, Cage Match

Tonight I went to compare the usual suspects for a set of slides I was doing and thought I would share the results.  

I do not believe anyone should be surprised by the results. MySQL became less interesting once it was acquired, and the number of Postgres deployments is greater than what all of the NoSQL market combined adds up to. When looking at emerging technologies, press time given to a single software product has no relation to what its actual market share is (which isn't as obvious as one might think to everyone).

When I get asked what my current opinion about NoSQL vs SQL is, I say that I continue to see NoSQL as a spotlight highlighting the failures in the design of the current relational database software vendors(*). Which should not damn the concept of the relational database, but somehow it happens anyways.

The two biggest issues that stand out to me are that current leading relational databases never solved scale out very well, and online operations are too expensive.

Would a document model really matter if schemas could be altered online? You would still have the weak vs strongly typed arguments that can exist in schema/table design, but if the operation to add additional characteristics to a table (object) was online or automatic I believe a lot of the arguments you see for schema less design would go away.

The other giant failure was that the vendors didn't approach scale out as a strong requirement for future designs. Going back a few years you can see where the vendors would talk about shared nothing designs, but only in the scope of support handful of nodes in a cluster. A hundred or so at best, which is small potatoes for where some of the NoSQL vendors who focus on scale out shoot for.

Its funny to think about how much has been written on this topic that has focused on the SQL language, when no new language has emerged from the NoSQL market that has any size-able adoption. At this point most of the NoSQL vendors now talk about how they have, or are, adding a SQL interface to their products.

The innovation hasn't been in the language, but in the design of database engines themselves.

The other bit of humor in this, is that if you take apart how most of the major vendors have implemented their storage engines, you would find that in the inner design few are strongly typed, and few track relationships in the core of their design. Most of the major vendors could do more operational changes online if they were willing to put reasonable scoping on what can be altered. The Innodb row format is inflexible, but the addition of some version information and a catalog documenting the versions would fix that.

To sum this up, the need to preserve backwards compatibility is what has kept the relational database vendors from solving the needs that caused the NoSQL solutions to come to market. I don't see any of them tackling this anytime soon; the margins on their current products doesn't dictate the need for them to do this.

A final thought about the state of the current nosql databases.

One of the "hard things" that almost(?) none of the current NoSQL vendors has tackled is the JOIN, i.e. the fundamental feature which is required to do anything beyond simple key/pair or basic search. Join optimizers are hard to write(*).  Network aware join optimizers are particularly difficult to write mainly because direct attached storage, AKA a hard drive, is a lot more reliable on a second to second basis than the network.

*) I also get asked which one I would pick if I had to pick one. Currently I suggest Cassandra, though more often then not a table with a key and a blob would work as well.

*) One of the nice things about MySQL's design was that someone designing a new engine could focus on the datastore and punt on the rest (well, sort of,... the pluggable storage engine interface never matured to the point of really playing nicely with other people's designs).

Thursday, February 28, 2013

Pioneer 1021 / 1121, Airplay, You suck

After a year of owning both a Pioneer VSX-1021 and a VSX-1121, I would never ever buy Pioneer equipment again.

The Airplay software on the Pioneer crashes a lot. When it doesn't crash, it plays music out of sync with other Apple airplay devices. 

The HMG server? Who let that ship? Did they look up from their Atari 2600 for more then a few seconds before saying "Menus! What sort of crazy new concepts for an interface will you think of next? Ship it!"

Their support for their iPad and iPhone application has been non-existant.

On the other hand, the Pioneer sounds better then my 20 year old Onkyo amp (I think). 

Someday, someone, is going to build a modern amplifier right. 

I really doubt it will be Pioneer.

Friday, February 22, 2013

You can be right, or you decide to save everyone else a bunch of time...

This is one of my favorite valgrind finds at the moment (well, last few months):
|| ==881== Syscall param sendmsg(mmsg[0].msg_hdr) points to uninitialised byte(s)
|| ==881==    at 0x386BAF3D6B: sendmmsg (sendmmsg.c:36)
|| ==881==    by 0x386DE0B4D6: __libc_res_nsend (res_send.c:1132)
|| ==881==    by 0x386DE08BDF: __libc_res_nquery (res_query.c:226)
|| ==881==    by 0x386DE097B9: __libc_res_nsearch (res_query.c:582)
|| ==881==    by 0x5E8CAC7: _nss_dns_gethostbyname4_r (dns-host.c:309)
|| ==881==    by 0x386BAD9AF1: gaih_inet (getaddrinfo.c:845)
|| ==881==    by 0x386BADD750: getaddrinfo (getaddrinfo.c:2359)

If you really want to be right, and not cost a lot of other people time when you know you are right and valgrind is wrong. You could leave a snarky comment in the code where you create a fix to silence the valgrind error.

Or you could decide to not fix the warning, and cost a lot of other developers their time as they trip over you academically correct code which spews errors when run through valgrind.


Thursday, January 3, 2013

Child Processes

When we, some of my peers, talk about "child processes"...

20 years ago the discussion was around the question of "who all is running netrek, and can we have someone who is root kill their process?". This was a quality of life issue, a very real first world problem.

15 years ago it was about zombies. Software usage was such that we started noticing quality. Child processes that became zombies. Seriously, zombies.

10 years ago  it was all about tuning Apache such that it had the optimal number of "children" such that you would not go into swap because you miscalculated the amount of memory that would be used. We often had too many children.

5 years ago the debate around child processes vs threads became a real conversation due to the changes in hardware. 

Today? 

I find my peers talking about child processes and sleep time, where child processes are in the 1 week to 12 month range with a average sleep time that is anywhere between 2 and 8 hours.

*) Can you find reversed joke in the above statement?

Monday, December 3, 2012

Installing Open Stack


It took me 4 hours to go from bare metal to a small cluster of computers today.

The install was done on Fedora (Redhat), and included Horizon, Nova, Swift, Cinder, Glance, and Keystone. This was using Folsom.

I hit exactly one install bug that I fixed in under 10 minutes. I fed Pepper once. 

Not bad.

Tuesday, October 30, 2012

Burning Man, Steve Jobs, The Importance of "use" over "testing"

My only real contribution to Burning Man, beyond the price of my ticket this year, was to help Ignite BRC by carrying a monitor to Center Camp, and to point out that Steve Jobs had now left the building.

How did I know that Steve Jobs had left the building?

I did the see the news reports, the book, and had walked past the memorial that someone had built on a local street corner. This wasn't like the moment in grad school where I had to ask "O.J. did what?" when one of my students tuned on a radio in class to hear the jury report. 

But how did I really know that he was dead?

A week before Burning Man my laptop failed to work with a projector. This is how I knew Steve Jobs had departed this world.

I doubt Steve ever touched Mountain Lion.  My presentation was given from a brand new laptop, with a shiny from the factory setup of OSX. 
 
Out of the box my OSX setup did not work with the projector. 

That would have never happened if Steve had been actively using Mountain Lion. 

Can you imagine what would have happened to the developers if Steve would have been alive, let alone experienced, such a simple failure of technology? 

It would not have been pretty.

One week after my own experience with OSX not working with a projector I found myself in a different environment reliving project-fail. 

There is something to be said about knowing for certain that the reason that the laptop sitting in front of you is not able to run a presentation is because the OS is at fault.

I was standing in the desert, in a giant circus tent, with dust in the air, staring at a hodgepodge of cables that were connected to some ancient CRT. There is a lot of things that might be contributing to the problem.  

When I was asked, "why isn't this working?"

The most useful thing I could contribute was:

"Running the presentations off a mac isn't going to work; the laptop is running Mountain Lion. Go find a Windows laptop."

The presentation was moved off the Mac, and onto a PC. 

The PC immediately worked. 

I am wondering if anyone at Apple used Mountain Lion to give a presentation before it was released. It is hard to imagine that someone didn't run into this problem long before Mountain Lion was released (as far as I know Apple has not  fix the bug yet).

I am sure that there is a lot of testing happening each release.

Testing software is not the same as "using" software.

Apple is a company, someone, at least once a day, must be giving a presentation. 

How did they miss this?

Steve Jobs would have presented Mountain Lion to the world using Mountain Lion. 

If that presentation had blown up on him,...

Egads.