Just A Programmer We're just programmers

27Apr/123

All Aboard #SQLAmtrak

Update: I have settled on the train.

I have three announcements in this post. First of all, I’d like to announce that I will be speaking at SQL Saturday 121 in Malvern, PA, just outside of one of my favorite cities, Philadelphia. Secondly, I’d like to announce that I will be taking Amtrak there from Newark Penn Station in Newark, NJ. Finally, I’m inviting you all to join me in a little adventure I’d like to call #SQLAmtrak!

Why #SQLAmtrak?

Because I love traveling by rail, and I’m sure I’m not the only one! I enjoy flying, and long drives. However, Amtrak is my favorite way to travel. In fact, just this Tuesday and Thursday I took Amtrak from Newark to Philadelphia to speak at PhillyNJ.NET and PhillyDB respectively. Previously I’ve also taken Amtrak to DC for MongoDC, and in December I took the entire route of the California Zephyr as part of a convoluted journey to MongoSV and the first MongoDB Masters conference. (BTW, I’ve also taken Amtrak to non-nerdy things with my wife. I swear sometimes I pretend to be normal!)

When you love something this much you want to share it with the world! I want to share this experience with members of #SQLFamily, most of whom I only know through the internet, and many I’ve not ever met in virtually.  I know if we all spend an hour and a half on a train together, it will be a great social experience. BTW if you need a business justification, call it a networking experience.

Ok, but how many people from Newark will be attending SQL Saturday 121?

The answer might be zero, because I actually live in Jersey City. Seriously though, this Amtrak journey makes sense for several groups of people.

  • If you live or work in New York City, the train stops at New York Penn Station before Newark
  • The train will also stop in Metro Park and Trenton, which are easily accessible from Central Jersey
  • If you work in the city of Philadelphia, you can join us for the last part of the journey from Philadelphia 30th Street station to Paoli.
  • Finally, if you are flying in, you can fly to EWR, you can take the New Jersey Transit one stop from Newark Airport to Newark Penn Station. Its quite likely that you can get a cheaper/faster/more direct flight from wherever you are coming to EWR as opposed to a Philadelphia airport.

What train are we taking?

Originally I was proposing that we take Amtrak Keystone Train 655. Which gets to Paoli at 8:40pm. However, I have book tickets for Keystone Train 651, which arrives at 6:02. The complete schedule for both trains is as follows:

Station Train 651 Departs Train 655 Departs Ticket Price
NY Penn 4:03 6:35 $50
Newark 4:20 6:52 $49
EWR * 3:49 6:40 $63
Trenton 4:55 7:28 $32
Conrwells Heights N/A 7:40
North Philly N/A 7:52
30th Street Phil 5:35 8:00 $6.50
Ardmore 5:48 8:15
Paoli 6:02 8:40

* For those flying in, its $5.50 for an Air train ticket to the NJ transit station and $8.50 to take NJ Transit from EWR to Newark Penn. You can buy those tickets on arrival.

Note that you need to buy a ticket in advance if you will get on the train before 30th Street. Also, ticket prices go up the longer you wait. Finally, there is no business class available on this train. Even if there was, not everyone would want to spring for it, and the whole point of this is for the gathering to be social, so we will take coach.

What if I get to Newark Penn early?

People will want to fly in early, and will likely want to have a light meal and or drinks before departing. For those of us who will be departing from Newark, I am proposing we go to Iberia Restaurant a short walk from Newark Penn in the Portuguese Ironbound section of Newark, voted best Sangria in the Ironbound by my wife and I. Weather permitting there is outdoor seating, and the tapas are great.

What do I have to do to take part?

If you plan on going, email my at justin@ the domain of this blog. Everyone is responsible for buying their own train tickets. I’m going to hold off until Sunday May 6th (one week from today) before buying mine in case I get some feedback that other people would prefer a different time. Iberia doesn’t require reservations, so you don’t have to commit to going to the restaurant.

What about the return trip?

I’m open to the idea of a coordinated return trip. However, I don’t expect a good consensus to happen for that.

How should I promote this?

Blog/tweet/etc about this. Tell your friends! Use the hashtag #SQLAmtrak.

9Dec/100

In the brave new world of MaybeSQL, we still need DBAs

Yesterday, Chuck Reeve’s tweeted an article from Daniel Lemire’s blog entitled Who will need database administrators in 2020?. The thesis is that with the advent of all these NoSQL technologies, the role of DBA will become unnecessary. I disagree with this for two reasons. First of all, NoSQL will not replace SQL. Secondly, your NoSQL data store probably needs a DBA, even if he has a different title.

Just a quick note, I’ve worked with enough SQL databases to make broad generalizations about them. The only NoSQL database I have experience with is MongoDB.

NoSQL will not completely replace RDBMSes

SQL databases are the primary practical implementation of the relational model. Most of the “trade school” explanations of the relational model and normal forms use SQL syntax as an example. Relational databases are great at storing data in an organized fashion. Through constraints you can enforce most business rules. Triggers will allow you to do the rest. Relational databases also usually have fine grained access control systems, and mechanisms for auditing changes. Finally, if you have to build a report from your data in a way you never did before or planned to, its usually nice to be able to start out with your data normalized.

Now there are a lot of things a NoSQL database like MongoDB does better than most RDBMSes. For example, MongoDB would be better suited for hosting a simple blog than MySQL. However, MongoDB has not been around all that long, and before MongoDB, relational databases did a good enough job. Now there are many things SQL is better at than MongoDB. For example, I would never use mongo for a complex inventory system. However, many technologists, like Daniel, have been focused on thing that NoSQL is good at, like blogs and simple ecommerce sites. These technologists recognize NoSQL as disruptive technology in the data management field. However, they make the mistake of assuming NoSQL will usurp the role of relational databases completely.

To put it another way, in the brave new MaybeSQL future, we will use SQL for some things and NoSQL for others. The things we will use SQL for, like complex inventory systems, will have complex schemas and need specialists to manage all that data. We already call those specialists DBAs.

Your NoSQL Database needs a DBA

Ok I lied. Your NoSQL database might not need a DBA, just like your relational database might not need one. In relational database shops without formal DBA positions,  there are usually defacto DBAs, senior developers who’ve made it their business to manage the companies databases because management would not allocate a dedicated salary to that function. Currently, I am serving as a defacto DBA for some small databases.

Now I’ve also been playing with MongoDB a lot. I’ve contributed to mongo, spoke about mongo, and been to three mongo conferences. I’ve talked to a lot of people using mongo, and I’ve made a lot of observations. My primary observation is mongo tends to get used in startups. These startups don’t have dedicated DBAs. However, they do have well rounded senior developers that perform DBA and sysadmin functions. Many of these NoSQL programmers also know more about relational databases than I do, which is why they didn’t fight “the mongo way” tooth and nail before accepting it like I did. Now as is the nature of startups, most of the businesses these programmers work at will fail. However, a few will succeed and get big enough to have to hire technologists with more specialized roles. I expect to see a mongo specialists role thats part sysadmin and part programmer evolving at these companies. For companies that use a a combination of a relational databases and MongoDB, I expect a DBA to be hired, learn MongoDB, and take ownership of managing the data stored in that companies MongoDB instances.

Conclusion

NoSQL databases were designed for different problems than relational databases. Relational databases were not designed for things like blogs and massive sites lie facebook. They were used for this role because they were the best tool at the time for the job. MongoDB on the other hand was founded by a founder of doubleclick, who wanted to build a database that scaled the way a database for websites should scale. MongoDB is taking a piece of the pie from relational databases, but not all of it. Also, just like not all relational databases have a full time DBA to maintain them, not all NoSQL databases have a full time administrator. However, that does not mean that a role similar to DBA for NoSQL databases is unnecessary.

Tagged as: , , No Comments
21Jan/100

What three things got me here.

Paul Randal wrote a What three events brought you here article on his blog and asked some others in the SQL server community to do the same. I of course was not on that list. However, no one ever invited me to blog about anything before so why should I be shy now.

I have decided to not tell this story in terms of professional accomplishments. For me, my hobbies have always led my career.

16Jan/080

The importance of changing ones mind

I do not consider myself an expert on Microsoft SQL server. I prefer Postgresql. However, I am a fan of referential integrity and seek to enforce it at the database level, regardless of the engine. I like my databases to reject invalid data like DMV clerks rejecting incorrectly filled out forms.

When I work with Microsoft SQL server I struggle to enforce referential integrity in the same manner as I do with Postgres. Recently, I wanted to restrict the values that may be stored in a column in a table. A mapping table was not the answer in this case. I thought the answer was to user T-SQL RULES. I implemented a few and prepared to add them to my standard practices. However, about a week later I discovered this nugget on the MSDN:

CREATE RULE will be removed in a future version of Microsoft SQL Server. Avoid using CREATE RULE in new development work, and plan to modify applications that currently use it. We recommend that you use check constraints instead. Check constraints are created by using the CHECK keyword of CREATE TABLE or ALTER TABLE. For more information, see CHECK Constraints.

All of a sudden I needed to unlearn what I had learned. I do this on a regular basis. I consider myself quite stubborn. And I quite value this stubbornness. I will argue my position to the death if I believe it is right. I pride myself on caring about the things no one else cares about, and being able to explain why you should care, if necessary. However, I can be persuaded to change my mind. It’s not so much that I bend or break, it’s more like I recrystallize. Sometimes the transition is instantaneous. Sometime my faith in a concept is broken and I spend a while looking for a replacement. This transition was rather instantaneous as I was presented with a clear message: “What you are using is depreciated and doing it this other way will work better anyway.”

So what is the lesson here? The lesson is simple. Be unwavering in your beliefs, until you have a good reason to change them. Then be unwavering in the new beliefs until they are demonstratively proven inferior.

Tagged as: No Comments