A message...

def christmas
   "Christmas"
end

def wish_a(holiday)
   puts "I wish you all a merry #{send(holiday)}!!"
end

wish_a(:christmas)

Is Smart UI what we need?

I just read a very perturbing section in Domain Driven Design called "Smart UI". This unthinkable, heretical pattern advocates that you put your business logic in the UI and that sometimes, it might even be worth it.

It was too late. I knew that the seeds of reason had started growing inside of me, like a vine on steroids. Soon, one of the thorns pierced my heart and I gave up.

According to Evans, Smart UI is useful for small projects or for teams that are not sophisticated enough to handle the complexity of Domain Driven Design. In that case, putting all the business logic in the UI will at least isolate it in the sense that all one has to do to modify it is to replace the UI. Of course, this means trouble in the long run. Things will get progressively harder and harder to change and you will end up with a monolithic beast that demands too much attention and sleeps on your side of the bed. But...

What if this is the best my team can do? Should I embark on a crusade to teach them Agility when I know deep inside that they don't really care? As far as they're concerned, they think things are just fine the way they are right now.

Why don't I think the same thing? Why am I on this quest for new methods and patterns?

I want to have fun programming like I used to in high school. This is a deep motivation of mine. Every company that I've worked for so far has somehow succeeded in making it more of a burden. I love the pureness of the craft, the manipulation of concepts, the massaging of data. On the other hand, I find that trying to make things fit is a bother. I love Quality things that just work.

This is who I am. I'm attracted by the beauty of mathematics, the usefulness of Ruby and the creativity and science of cooking food.

I'm put off by by the monotony of building data centric web applications that simply mirror the database schema with some extra validation and business rules sprinkled all over the place. I'm put off by anything that takes time and requires me to stop my creative flow, by things that create friction and that refuse to work even though it seems like you did everything right.

I also hate getting off track like I did just now... But it had to get off my chest. Sometimes I don't even know why I do it; why I sit at a desk and try to make code behave. This little rant helped me refocus on what makes me tick and on why I might want to go the Smart UI way with my team. In the end, it is the customer that counts. I might not become the equivalent of a Michelin three stars chef this way but at least I'll make some people happy... in the short term. The future, fortunately, will force us to change. By this time, I might gain enough influence and wisdom to help my team migrate towards a more... educated way of doing things.

Who is your Chief Engineer?

I really like Scott Bellware's post about a chief engineer to drive the development of software within your organization, which is based on the definition of the same role at Toyota (which, incidentally, is referenced quite often when talking about lean software development.) I also like the though that such chief engineers need to be grown and cultivated within an organization with a culture of change already in place. This is very similar to what Mary Poppendieck calls master developers in her book Lean Software Development. These people are responsible for representing the customer as domain experts and oversee the design and development of software since they know about the technology and still use it! 

In another post, Scott rewords the chief engineer's responsibility within a context of acceptance testing:

A Chief Engineer in a software product development organization can write acceptance tests without the burden of elaborate end-user testing tools. He can use the common tools of the trade. He understands the imperatives of using tests as documentation and uses usability-focused test authorship, and sets standards for authorship that his organization cultivates and follows.

If I'm reading between the lines correctly, it sounds like the chief engineer might not be writing code anymore, but can sure write tests to insure that the software is implemented in a way that its behaviors pass the tests that he wrote. I like that very much. I've read in so many places and truly believe that great managers and leaders still code. They have to have their hands wet and dirty. Writing acceptance tests, or BDD and other kinds of tests is a great way to do that while at the same time making sure that they are not in the way of the developers. Talk about having your feet in both the customer's world and in your own world at the same time! I don't think this concept exists any other way in software development.

Now, how do we get a chief engineer if we are a one man show like myself? I guess it is up to me to cultivate myself into one. But that can take years! I guess the only solution, in the meantime, is to surround myself with customers and use all the tools that we have so that I can understand them as best I can. BDD, DDD, acceptance tests, etc... Eventually, either I'll become a chief engineer in that domain or I'll end up cultivating another one, which will mean that the business that I haven't started growing yet will be making money (crossing my fingers!)

Thanks Scott for the great post!

SQL separation of concerns

After writing my post last night, I started thinking about ways to remove the bad smell coming out of my SQL code. The issue is that I wrote a stored procedure whose

My thoughts followed this logic:

Well, that's okay. I'll just put an obvious comment in there asking anybody not to change the result set. I'll also rename the stored procedure, ending it with "ForInsert" so that it is obvious what it's for.

Oh man, that is even worse! Now it's like I just sprayed some strawberry fragrance to cover that nasty bathroom smell... You know what I mean!

I guess I could put in some output parameters in my stored procedure instead and use those to insert the data I want. That way, as long as nobody removes a parameter, I'm fine.

Yes, that's better. Compared to my previous thought, this one removes the problem where another programmer would change the order of the fields returned, remove or even add fields in the returned result set. Adding field is the biggest issue here. I know I like to "supplement" stored procedures here and there by adding extra information that is returned to me. Which makes me think...

If I want a stored procedure to return extra data, I'll write another stored procedure that returns the result set of the first one with my extra stuff added to it. That way, I won't break the initial stored procedure.

Hey, I'm learning as I go, Ok? Cut me some slack. I'm going through the usual programming learning curve here. I'm fairly advanced when it comes to regular code like C# or Ruby, but mediocre with SQL. It feels like I just came out of the "Look 'ma! SQL code!" stage and started wondering about encapsulation and separation of concerns. Of course, that is much harder to achieve in SQL than it is in any good object oriented language.

I had one last thought, which I haven't acted upon yet. What about using views and filtering the data on the view instead of passing parameters to a stored procedure? Or what about using common table expression? What about temporary tables? I haven't had time to think those through just yet, but I will...

SQL code can smell too

I just wrote some bad SQL code tonight that I think I'll have to revisit tomorrow morning when I get back to work.

I think that using a stored procedure to populate an Insert statement is simply wrong. Should anyone modify that stored procedure or change the order in which the values are returned, my insert will fail. Can you smell that? Yeah, it smells bad.

I was simply trying to "encapsulate" things a little and make my code less cluttered. Maybe using a common table expression or temporary table would be a much better idea.

Losing your work

Windows Live Writer, which I use to write this blog, decided to stop allowing me to use the undo function just at the moment when I had accidentally deleted 3/4th of my post. Of course, I had saved my work both before and after the accident. I guess the "save often" philosophy works only when you can trust the undo button to do its work properly.

I hope you guys were not hoping for my next installment of my SQL training recap. I really don't feel like writing it all again and I'm getting a little tired of SQL code. Yeah, it gets old.

Could somebody write a text editor that keeps track of the history of your file? Kthxbye.

State of my Digital Self

When I created this blog last July, my goal was to brand myself online and to create an online persona that would represent me, my thoughts and my values with the goal of becoming visible and gaining the respect and recognition of my peers in the software development world.

I started blogging without caring about the whether I was right or not with the assumption that the process of learning that takes place while writing is more important to me than being right. I have, after all, the rest of my life to be right.

I opened a Twitter account with the intent of meeting new people and hoping that a few would follow me. What I discovered, however, is that Twitter is at its best when you use it to follow the right people. That is, following people you want to meet or that something that you don't, be it skills, money or fame. That is the magic of Twitter and that is how I got to go to the best conference I've ever been to and meet people who would have a lot to teach me, and still do continuously!

I'm also on LinkedIn. However, I'm not sure I have uncovered the key to that social networking site yet. I don't know how to make it work for me. It is a great place for people to examine your professional past and references, but that's about it so far. I'm sure I'm missing something, but what? I need to look into it some more.

Then there's Shelfari, a social site where you can advertise the books you've read, the ones you'd like to read and the ones you own. Seems interesting. I opened up an account just a couple days ago so we'll see where that takes me.

And last but not least: Facebook. I do not have an account on Facebook simply because I never understood it. I used to have an account on MySpace and I hated it. Should I do Facebook? Is it different? I guess I'll have to check it out. Today!

Advanced SQL training

I ranted a bit about SQL in general in my previous post, which was supposed to be a quick intro to the SQL training I'm putting myself through at home. Instead, I rambled on about developers being scared of SQL and bending over backwards in order to avoid it.

Anyway, my boss asked me last week to go to a SQL training. He correctly identified my lack of knowledge and understanding about the database tools we use at work and wanted to correct that. But instead of going to a training that just covers the basics in a breath-first manner, I bought two books: Inside SQL Server 2005: T-SQL Querying and Inside SQL Server 2005: T-SQL Programming. Oh my God... The amount and depth of knowledge in those books is staggering. I have four days to go from "just getting by" SQL developer to a "data can't hide from him" database guru. Of course, I told my boss that I'd be blogging about my discoveries in order to cement the knowledge in my head but now I find that I will not have enough of my four days of reclusion at home to learn everything I want to learn. I need to focus on what I need for work.

For this training, I dove deep into two books and started with the fundamentals...

Fundamentals

What steps SQL Server goes through to evaluate a query is a very useful thing to know and helps explain why certain things can or can't be done inside a query.

Take the following query for the Northwind database, which select the total number of orders made by all american employees where that number is greater than 100:

SELECT COUNT(Orders.OrderID) AS TotalOrderCount, Employees.EmployeeID

FROM Orders

    JOIN Employees ON Orders.EmployeeID = Employees.EmployeeID

WHERE Employees.Country = 'USA'

GROUP BY Employees.EmployeeID

HAVING COUNT(Orders.OrderID) > 100

ORDER BY TotalOrderCount

SQL Server starts by executing the FROM and JOIN statements by performing a cross-join between the Orders table and the Employees table and storing the resulting rows in a virtual table that we'll call VT1. Then the ON statement is executed and only the rows that match the condition in the query are kept in another virtual table: VT2. The WHERE statement is then examined and SQL Server gets all the rows in VT2 where Country = 'USA' and stores them in VT3. VT4 is created by grouping together the rows based on employeeID, meaning that we are keeping only one row per employeeID and that we have to somehow aggregate the rest of the data from now on.

The HAVING clause indicates that we only want to keep the rows where the count of orderIDs per employeeID (aggregating the data) is greater than 100. The result is stored in VT5.

Then, we select the employeeID and the count of orders (remember, the rest of the data after executing the GROUP BY statement needs to be aggregated somehow) and assign, optionally, column aliases, which you can now use in the query, into VT6. Finally, VT7 is created when we use the ORDER BY clause.

Interstingly enough, VT7 is not returned. Instead, a cursor is what the query gives us back because of the ORDER BY clause. Without it, a virtual table would indeed by returned. This is why you cannot use ORDER BY inside subqueries, which are expected to return a table. After an ORDER BY statement, only functions that take cursors as input, like TOP(n), can be used.

So let's go back over the order in which SQL Server executes the query (I'll include every possible SQL statement this time):

1- FROM .. JOIN
2- ON
3- WHERE
4- GROUP BY
5- WITH
6- HAVING
7- SELECT
8- DISTINCT
9- ORDER BY
10- TOP

Any alias that you define in a query, be it a column alias, a table alias or a subquery alias can only be used in subsequent steps in the list or SQL Server will not recognize them.

On a side note, it is important to know about the UNKNOWN value in SQL. Usually, TRUE or FALSE is return when comparing values. But since some values can be NULL, comparing such a value with another one results in UNKNOWN. When a filter is applied to a query (WHERE, HAVING, ON), unknown values are considered like FALSE. But inside a check constraint, like making sure the salary in a column is greater than zero, unknown values are considered like TRUE.

Hey, I like SQL!

SQL is one of my weak spots. I've known that for a while, since college actually. I never thought I'd need it, you know? Ha, add that to the list of things that seemed superfluous to me at the time! SQL? That's what DBAs are for, no? Math? That's what mathematicians are for, no? Parallel programming? I'm never going to work on more than one CPU at a time! Statistics? I'll just look up formulas online. Ah, being young...

Well my use of SQL has increased linearly since I got out of college and I always managed to learn just enough to get by. However, it has caught up with me. I work in a manufacture that creates pumps and valves for the oil fields. Tractability is important and we keep track of everything that is done to our parts during the manufacturing process. Add to that the HR system and our accounting software and we have massive amounts of data in the database. That data is there, just waiting to be used.

Not being great with databases in general is a flaw since I do not know how to use SQL as a tool. Here are a ton of nails, but I don't have a hammer, just a C# screwdriver (I do VB.Net, so I what I really have is a plastic screwdriver!)

Software developers and DBAs have been at war since they ever met. DBAs complain that software developers are nosy and don't know how to use a database properly and exist only to circumvent the systems they have put in place to safeguard against both data problems and intruders. Programmers, on the other hand, complain that DBAs take too long to take care of their requests and are just creating job security for themselves.

On top of that, software developers in general just don't like databases. It seems alien to us. We can't debug it, it gathers data all at the same time and we can't ever be sure that we even got the right data to begin with! Why not just grab everything, loop over it and grab what we need? After all, our data is never complex.

So we attempted to build object databases, which are far from being mainstream at the moment. We like them because we know Objects. But SQL just isn't going away. So we created ORM frameworks to help get rid of the repetitive CRUD operations in our models so we don't have to ever deal with SQL. Then, we declared a holy war on stored procedures, the only thing that can still possibly resist ORMs.

Stored procedures are not faster than code! They're just there so that the DBAs have something to do. There is nothing a stored procedure can do that code can't do!

To all of this I say: Bullshit! Programmers react to SQL the same way VB ASP.Net programmers react to functional languages. They don't want to touch them with a 10 foot pole simply because they don't understand them. All their arguments are a facade to hide the fact that they don't want to learn the new languages.

You don't think the same way when you program in Javascript or Lisp or Ruby versus VB.Net. In the same manner, using SQL forces you to think in SQL, an entirely different beast altogether.

I've really come around recently concerning SQL. I used to be like what I described above, but the more comfortable I became with the language, the more I started understanding it. I realized it is a tool in it's own right, used to solve a wide range of problems and that trying to resolve everything with my favorite language was not always the right thing to do.

If you deal with data and databases, learn SQL. Learn your database management system. There is really nothing else out there that can work this efficiently with relational data. Sure, use ORMs, since they are a tool that helps you keep a clean model. But do not be afraid to use SQL. Do not let the ORM do all the SQL thinking for you the same way you wouldn't let the politicians in Washington tell you what to think! ORMs can't do everything, after all. Sometimes, you just need the data without an associated object.