In the brave new world of MaybeSQL, we still need DBAs
Yesterday, Chuck Reeve’s tweeted an article from Daniel Lemire’s blog entitled Who will need database administrators in 2020?. The thesis is that with the advent of all these NoSQL technologies, the role of DBA will become unnecessary. I disagree with this for two reasons. First of all, NoSQL will not replace SQL. Secondly, your NoSQL data store probably needs a DBA, even if he has a different title.
Just a quick note, I’ve worked with enough SQL databases to make broad generalizations about them. The only NoSQL database I have experience with is MongoDB.
NoSQL will not completely replace RDBMSes
SQL databases are the primary practical implementation of the relational model. Most of the “trade school” explanations of the relational model and normal forms use SQL syntax as an example. Relational databases are great at storing data in an organized fashion. Through constraints you can enforce most business rules. Triggers will allow you to do the rest. Relational databases also usually have fine grained access control systems, and mechanisms for auditing changes. Finally, if you have to build a report from your data in a way you never did before or planned to, its usually nice to be able to start out with your data normalized.
Now there are a lot of things a NoSQL database like MongoDB does better than most RDBMSes. For example, MongoDB would be better suited for hosting a simple blog than MySQL. However, MongoDB has not been around all that long, and before MongoDB, relational databases did a good enough job. Now there are many things SQL is better at than MongoDB. For example, I would never use mongo for a complex inventory system. However, many technologists, like Daniel, have been focused on thing that NoSQL is good at, like blogs and simple ecommerce sites. These technologists recognize NoSQL as disruptive technology in the data management field. However, they make the mistake of assuming NoSQL will usurp the role of relational databases completely.
To put it another way, in the brave new MaybeSQL future, we will use SQL for some things and NoSQL for others. The things we will use SQL for, like complex inventory systems, will have complex schemas and need specialists to manage all that data. We already call those specialists DBAs.
Your NoSQL Database needs a DBA
Ok I lied. Your NoSQL database might not need a DBA, just like your relational database might not need one. In relational database shops without formal DBA positions, there are usually defacto DBAs, senior developers who’ve made it their business to manage the companies databases because management would not allocate a dedicated salary to that function. Currently, I am serving as a defacto DBA for some small databases.
Now I’ve also been playing with MongoDB a lot. I’ve contributed to mongo, spoke about mongo, and been to three mongo conferences. I’ve talked to a lot of people using mongo, and I’ve made a lot of observations. My primary observation is mongo tends to get used in startups. These startups don’t have dedicated DBAs. However, they do have well rounded senior developers that perform DBA and sysadmin functions. Many of these NoSQL programmers also know more about relational databases than I do, which is why they didn’t fight “the mongo way” tooth and nail before accepting it like I did. Now as is the nature of startups, most of the businesses these programmers work at will fail. However, a few will succeed and get big enough to have to hire technologists with more specialized roles. I expect to see a mongo specialists role thats part sysadmin and part programmer evolving at these companies. For companies that use a a combination of a relational databases and MongoDB, I expect a DBA to be hired, learn MongoDB, and take ownership of managing the data stored in that companies MongoDB instances.
Conclusion
NoSQL databases were designed for different problems than relational databases. Relational databases were not designed for things like blogs and massive sites lie facebook. They were used for this role because they were the best tool at the time for the job. MongoDB on the other hand was founded by a founder of doubleclick, who wanted to build a database that scaled the way a database for websites should scale. MongoDB is taking a piece of the pie from relational databases, but not all of it. Also, just like not all relational databases have a full time DBA to maintain them, not all NoSQL databases have a full time administrator. However, that does not mean that a role similar to DBA for NoSQL databases is unnecessary.
What three things got me here.
Paul Randal wrote a What three events brought you here article on his blog and asked some others in the SQL server community to do the same. I of course was not on that list. However, no one ever invited me to blog about anything before so why should I be shy now.
I have decided to not tell this story in terms of professional accomplishments. For me, my hobbies have always led my career.
The importance of changing ones mind
I do not consider myself an expert on Microsoft SQL server. I prefer Postgresql. However, I am a fan of referential integrity and seek to enforce it at the database level, regardless of the engine. I like my databases to reject invalid data like DMV clerks rejecting incorrectly filled out forms.
When I work with Microsoft SQL server I struggle to enforce referential integrity in the same manner as I do with Postgres. Recently, I wanted to restrict the values that may be stored in a column in a table. A mapping table was not the answer in this case. I thought the answer was to user T-SQL RULES. I implemented a few and prepared to add them to my standard practices. However, about a week later I discovered this nugget on the MSDN:
CREATE RULE will be removed in a future version of Microsoft SQL Server. Avoid using CREATE RULE in new development work, and plan to modify applications that currently use it. We recommend that you use check constraints instead. Check constraints are created by using the CHECK keyword of CREATE TABLE or ALTER TABLE. For more information, see CHECK Constraints.
All of a sudden I needed to unlearn what I had learned. I do this on a regular basis. I consider myself quite stubborn. And I quite value this stubbornness. I will argue my position to the death if I believe it is right. I pride myself on caring about the things no one else cares about, and being able to explain why you should care, if necessary. However, I can be persuaded to change my mind. It’s not so much that I bend or break, it’s more like I recrystallize. Sometimes the transition is instantaneous. Sometime my faith in a concept is broken and I spend a while looking for a replacement. This transition was rather instantaneous as I was presented with a clear message: “What you are using is depreciated and doing it this other way will work better anyway.”
So what is the lesson here? The lesson is simple. Be unwavering in your beliefs, until you have a good reason to change them. Then be unwavering in the new beliefs until they are demonstratively proven inferior.