paul vidal - pragmatic big data nerd

Monthly Archives

2 Articles

Who decided that stored procedures should not be commented?

by paul 0 Comments
Who decided that stored procedures should not be commented?

I’ve been spending the past couple of weeks working on stored procedures. Glimpsing into my career so far, I realize how much stored procedures are the backbone of many organizations dealing with data. Stored procedures are something of a potpourri between magic behavior, bespoke black boxes, and the sedimentation of code layers accumulated over years of feature additions implemented by a battalion of sometimes well-intented PL/SQL programmers with tight deadlines. Furthermore, stored procedures, more than any other type of data manipulation, are what the actual live production systems rely upon. It is not uncommon for a piece of software to have hundreds of store procedures essential for it to work, and for good reason. Indeed, store procedures are extremely efficient. So much so that even unoptimized pieces of code harboring redundant test and an unreasonable amount of nested outer joins still run in a few milliseconds. Efficient they are. But you know what they are not? Commented. Seriously, the packages I worked with recently contain tens of thousands of lines of code but never contain more than 10 lines of comments, mostly containing something along the lines of “– 10/10/2014 added by Jay” or “– requirement R3045”. And as far as I can remember, relying solely on my flawed memory and anecdotal evidence, this is the case with the vast majority of stored procs. Therefore, after I spending some time curved into a ball crying, I asked myself: “why?”.

Common consensus about commenting code

Childishly, I first assumed that every piece of code should be commented, and the only reason for not commenting code would be laziness/lack of time/lack of understanding/hatred for whomever would read your code in the future. I was obviously misguided as one often is when assuming anything to be simple. Indeed, they are many times when commenting renders your code less readable, or is an excuse for bad coding. This article in particular, Common Excuses Used To Comment Code and What To Do About Them does an excellent job at highlighting when commenting is sub-optimal:

The code is not readable without comments. Or, when someone (possibly myself) revisits the code, the comments will make it clear as to what the code does. The code makes it clear what the code does. In almost all cases, you can choose better variable names and keep all code in a method at the same level of abstraction to make is easy to read without comments.

    We want to keep track of who changed what and when it was changed. Version control does this quite well (along with a ton of other benefits), and it only takes a few minutes to set up. Besides, does this ever work? (And how would you know?)
    I wanted to keep a commented-out section of code there in case I need it again. Again, version control systems will keep the code in a prior revision for you – just go back and find it if you ever need it again. Unless you’re commenting out the code temporarily to verify some behavior (or debug), I don’t buy into this either. If it stays commented out, just remove it.
    The code too complex to understand without comments. I used to think this case was a lot more common than it really is. But truthfully, it is extremely rare. Your code is probably just bad, and hard to understand. Re-write it so that’s no longer the case.
    Markers to easily find sections of code. I’ll admit that sometimes I still do this. But I’m not proud of it. What’s keeping us from making our files, classes, and functions more cohesive (and thus, likely to be smaller)? IDEs normally provide easy navigation to classes and methods, so there’s really no need to scan for comments to identify an area you want to work in. Just keep the logical sections of your code small and cohesive, and you won’t need these clutterful comments.
    Natural language is easier to read than code. But it’s not as precise. Besides, you’re a programmer, you ought not have trouble reading programs. If you do, it’s likely you haven’t made it simple enough, and what you really think is that the code is too complex to understand without comments.

Why this consensus does not apply to stored procedures

As much as these arguments make sense I don’t think they apply to store procedures:

    “you can choose better variable names and keep all code in a method at the same level of abstraction”: You can’t easily change table fields and names, nor can you cut a big nested SQL statement gracefully.
    “Version control does this quite well”: Version control is almost never implemented for stored procedures.
    “I wanted to keep a commented-out section of code there in case I need it again.”: OK, that’s just BS.
    “[complex code] is extremely rare.”: Nested SQL queries are inherently complex and MUCH less readable than traditional code.
    “Markers to easily find sections of code.”: I never saw a problem with that.
    “you ought not have trouble reading programs”: Except queries are the opposite of natural language. Please, please, please SQL developer, let me know why you are doing this particular join

To summarize, I still don’t understand why stored procedures are generally not commented, while it would seem they are the type of code that could benefit the most from comments. Maybe NoSQL will change this, but in the meantime, I will start this crusade, and make sure people explain their code, yo!

You should not try to be more productive

by paul 0 Comments
You should not try to be more productive
Coffee, a gift from the gods

You should not try to be more productive

Sometimes, my job requires long hours to meet impossible deadlines. I think that this is a situation with which many people can empathize. So much so that the self help industry is riddled with recipes to optimize your work habits, and become more productive. Leaving aside the overwhelming pseudoscientific nature of most of the self-help industry, productivity has become a market. Apps, website, gurus, all working to make you work better and more efficiently. Here what I think: trying to be productive is counter productive.

Where the productivity industry fails

If you start diving into productivity advice, you will start to have the feeling that productivity is like a cauldron. A cauldron that accumulates anyone’s latest advice concoction, no mater its origin. Want to be more productive? Eat Kale. Listen to music with a tempo accelerated by 5%. Take notes on paper because it sticks in your memory better. Do adult coloring. Play brain training apps that have no scientific backup what so ever. Always reach inbox zero or always make sure that you have at least 10000 emails unread. Anything goes. Anything will make you more productive. In hindsight, fantasizing about being more productive is a great way to procrastinate. To be perfectly transparent, I’m guilty of having read a lot of these articles in my days. OK, literally (in its proper sense) a few months ago. I could argue that I was just reading them for entertainment but I’m more honest than that. I get fooled often, which is why I have a lot to write about. Take a step back on these productivity advice, and I think you will reach the same conclusion as I do: most of them are a mix of unproven facts, common sense and non applicable pieces of advice that take away actual productive time.

What I do instead

But Paul, does that mean that we are doomed to be as efficient as we are today with no improvement route what so ever? Yes. Also, life has no meaning. But that’s besides the point. Of course, there is always room for improvement. So here is what I suggest to do instead of trying to be productive: 1. Prioritize. 2. Execute. That’s it. Prioritizing is a necessary evil but should not be taking too much of time. The best way to do it is to do it on small chunks, like an everyday to do list. Most of the time should be spent on executing however. Once I’m executing something, I never think twice about the priority. This is why I run every day without ever thinking of what I could do instead. It is on my list, I have to do it, so I do it. I do not think that going any deeper than this has any value, really. Just list what you need to do, and do the things. It’s important to note that sometimes, due to lack of time of ever unpredictable facts of life, execution fails. This just means that another prioritizing cycle needs to happen.

After thoughts

As many of my less technical posts, this article is obviously an opinion piece. The reason I started this blog in the first place is to share things I learned throughout my career without anyone spelling it out simply to me. It is not aimed to be definitive, since my opinions are prone to change, but instead is aimed at provoking conversations and trigger perhaps more thorough scrutiny and even scientific evaluation of the methods of productivity. It would be interesting to dive into this subject, but unfortunately for me, my tasks are prioritized, and this is not one of them, so…. back to work!