Reinventing Wheels (Just Occasionally)

stone-age wheelThere’s a pull in the software world towards re-use, and this is (mainly) a good thing: We use a third-party library, rather than re-inventing something that the library does. We pick a framework for larger features, services and layers that we need in our work, rather than building them from the ground up.

In general, this saves time and reduces the number of bugs we need to iron out. I still think it is usually the way to go.

However, I also think that we should re-invent wheels occasionally: Not use a library, re-build something from scratch… for the hell of it, so we understand the feature we would otherwise just have re-used, so we gain a better knowledge of the area the framework “just handles” for us. The additional work often leads to a better understanding and so, sometimes, I think it’s worthwhile.

In a previous job of mine, we used to ask interviewees “How would you design a web server?”. Our reason for this was that it demonstrated knowledge of basic HTTP, and an understanding of the kinds of issues you might encounter when using an existing web server. That “under the bonnet” knowledge is often useful, even if we never expected anyone to need to build the thing from scratch.

Recently, I’ve been redesigning a UI framework that I wrote a few months back. It was server-driven, so I’ve refactored it to be entirely browser-side including templating, having gained enough JavaScript experience to feel this was a worthwhile and achievable task. I realise there are several great UI frameworks out there (more than I own fingers, actually!), but I wanted to experience the issues involved in writing my own, to understand the difficulties in making it work cross-browser, to explore ways of generating views with templates, and therefore to be able to assess the other frameworks more effectively. I may face bugs with this framework, but I now feel better able to resolve them. In-fact, more able than I would be with the bugs that no-doubt still exist in those third-party frameworks (go on, tell me they’re bug-free and I’ll laugh 😉 ).

It appears that what I’ve ended up with is a very usable framework, which does some things the others don’t (or at least does them in a way that I prefer). I’ll be using it for my own app, even if it otherwise never sees the light of day as a framework in itself. So perhaps the additional unintended consequence of reinventing wheels is that, just occasionally, you end up with a better wheel in the end.

I suppose that progress never occurs unless people dare to reinvent wheels occasionally, and it requires the audacity to do it anyway, even when intuition tells us not to.

Regularly Changing Focus

micromacroSoftware engineering is all about abstraction: Focussing on concepts at a certain level of detail whilst hiding unnecessary concerns (the smaller picture) and, to a degree, how the work may be used (the bigger picture). This focus allows engineers to work on a few key concepts at a time and to build something that represents those concepts and behaviours well.

I see abstraction as a way of slicing the world so that we can represent it in software without becoming overwhelmed by either the bigger or smaller picture and all of the (temporarily) unnecessary complications therein. You simply cannot build software whilst you consider all layers of the onion, or all levels of the stack from bits & bytes right up to user interfaces, the internet and beyond. There is too much to consider at one time. We would never get anything built.

Even when working on one project, I find it best to alter my level of focus regularly. Particularly when it comes to considering the bigger picture, stepping back from more detailed work to reconsider it with a wider focus regularly brings new insights. Occasionally, the more detailed work may even be invalidated. And this is good, because we’re building something that needs to be relevant (right?). The world moves on, or gets misunderstood, at different levels and we need to periodically re-check how our work fits at each level.

Many engineers focus at one level of abstraction for a prolonged period, and usually a favourite level where their skills align well: UI design, RESTful APIs, concurrency and performance, UX, data storage, etc. Being able to step back, or dive deeper, and unite our work with that of others is a valuable skill. I’ve seen UI folks unwilling to talk to server-side developers, and vice versa. And Java web developers choosing not to talk to .Net experts, despite both technologies sharing the same bigger (and smaller) picture.

Seemingly disparate projects or technologies can share so much either in their detail, or their wider usage. Operating systems make use of some of the same underlying technologies and standards, but they also attempt to implement similar broad user stories and solve the same UX problems. Where they differ is all of the layers in between. The same goes for smartphones, with shared network technology (detailed level) and application behaviour (broad level), but also shared browser implementations now. Such seemingly different environments as iOS and Android unite both in detail and broader concepts, despite their many differences when viewed from other levels.

Alternating between a micro and macro level of focus, and all levels in between, helps us to see similarities with other work, points of integration, and opportunities for reuse. It helps us to spot patterns. I think it’s another part of keeping our work relevant and useful, rather than developing software in a vacuum.

What Are You Doing On The Side?

hands-on-keyboardIt used to be the case that you could land a software development job based on a list of technical skills. You either had enough of the ones required by the job spec to get the role, or you didn’t. Lately, side projects have become the “must have” accessory for candidates, and a list of promised skills is no-longer sufficient.

I must admit I agree with this change. I’ve long said that good engineers are ones who were coding since childhood, or who at least learn new tech skills in their spare time and don’t expect to be sent on courses. These people naturally have side projects. In-fact, they couldn’t imagine not having a side project.

And so our side projects are becoming as crucial as our CV or resumé. We should always be building something worth showing. That means something functional, useful and complete, not just proof that we applied a particular JavaScript library, or learned some CSS. These are side projects, not just demos, tinkering, playing or dipping into things idly.

I’m about to take another break from full-time employment, precisely so that I can spend time on a side project and throw myself at it with an intensity that I couldn’t achieve whilst in full-time work. The 2-3 hours per day that I manage around my day-job just aren’t enough. I’ve realised that the process of discovery will only happen if I work 8-12 hours a day on it. Only then will I know if it has wings and could become a business, or whether it will just be a great CV for me. I’m that convinced of the value of side projects, that I’m getting rid of the word “side” and making it my main project for a while.

So what are you doing on the side?

The Multiple Coding Styles of a Software Developer

bookshelvesIf software was like literature then, as software developers, we wouldn’t just be free to build a personal lifelong writing style and apply it to everything, as authors do. We’d need to be Dan Brown one day, Paulo Coelho the next and, dare I say it, Jilly Cooper the day after that. Unlike with authors, a personal style isn’t what makes a good developer, but rather the almost split personality of someone who can move between styles and write for different target audiences or varying uses as required.

It struck me this past week how many different styles of coding we adopt and switch between. Sometimes the differences are subtle, not immediately visible, and it is more a case of coding with certain considerations in mind. Other times, even the visual appearance or layout of the code we write will be different, depending upon the style required, the audience, and any other concerns we have in mind at the time.

Browser-side, JavaScript must be written in a very conservative manner, to work across the widest range of browsers, the developers of which seem to have interpreted the relevant “standards” in different ways (and some of whom just haven’t bothered). There isn’t the luxury of using the latest-and-greatest browser feature, unless we check for its availability at run-time and code an alternative behaviour for when it isn’t available, thus gracefully degrading (or perhaps backdating) the behaviour as required. This is somewhat like writing a book using complex vocabulary, but explaining every term for readers who don’t understand.

We can’t just include a new third-party library without considering the bandwidth required to download it and whether it works on all of those browsers. Our own JavaScript must be compact, or compact-able by a tool such as Google’s Closure compiler, such that it has a small download size and perhaps isn’t easy to debug by prying eyes. Competitors would always love to reverse engineer it and so, at run-time, our aim is actually unreadable (but reliable) code.

Our browser-side code needs to execute efficiently on the worst possible target device, so that it is still responsive for the user of that device, no-matter how much we wish that user would just upgrade. This means caching the results of calculations we may re-use, “lazily” calculating results at the last minute where they might not even be required, and considering the most efficient way of doing each of those calculations when they happen. For these reasons, browser-side code may visually seem overly complex, or we might lay it out and document it verbosely with our reasons for each decision we made. Perversely, efficient code may be lengthier than inefficient code (before using tools to compact or obfuscate it), whereas literature made for efficient reading would probably be short and concise.

When writing Server-side code, we don’t need to consider cross-environment issues so much, although code written in languages such as Java might be deployed on multiple operating systems. We can include third-party libraries we need, so long as their licence model is right for us, taking advantage of their features and perhaps shortening or simplifying our code. Performance considerations still apply, and if we are writing code to handle multiple users at once, we need to consider how best to use the available processors, memory and to minimise the number of threads we need (we often forget they aren’t free). We must write code that makes the application appear responsive to the number of users we intend to support normally, but still remain stable during our worst predicted spike in usage, such as when a news site mentions us or during an advertising campaign. The longer-term stability of the application becomes important, unless we intend to schedule regular restarts, and memory leaks that only manifest themselves over days, weeks or months become a real concern. Our code may be littered with accounts of why it is written in a particular way, to address those concerns.

Code written for offline use, perhaps overnight or in a batch job, doesn’t need to be so concerned about the resources it uses. We might sacrifice code efficiency for code clarity, particularly if the processing being performed involves complex business logic. Optimising that logic unnecessarily might mean it is harder to understand and develop further.

Code that performs safety-/business-critical activities, or even those at pivotal points in a single transaction such as shopping cart checkout processing, must be written to maximise the chances that it will work reliably, or to be recoverable in the event of a failure which we’d rather won’t, but know will, occur at some point — databases do unfortunately go down, connections fail, hard drives fill up, etc. We might write an audit log, generate unique transaction ids, or pre-check the availability of all resources that we require before committing to performing a crucial activity, to minimise the number of failures. Our code may seem lengthier and slightly simplistic or sequential, as we are writing it with reliability in mind.

Sometimes, we write code purely to demonstrate something or to experiment with a feature. This needs to be understandable by others, and probably won’t be too efficient. The more important thing is that someone can see what we did. This also often applies to code written as part of automated tests, as it should be clear what they are testing, and how. Again, efficiency can often go out the window here.

It is true that some developers spend more of their time writing code in one style than another, through working in particular industries or coding for certain uses. But I suspect that most of us need to move between all of these styles at some point in any given year. Sometimes we switch hourly, particularly when moving between writing production code and automated tests of that same code.

I think that this range of necessary styles is another aspect that makes software development so enjoyable. I doubt you could get stuck in a rut, with this constant switching. It’s like getting to write fiction, romance and a crime thriller on the same day…. well, almost.

Facebook, I think we should take a break from each other.

005 - cropIt’s not you, it’s me. Ok, maybe it’s you.

Lately, my relationship with Facebook hasn’t been working as well as I’d like it to.

I noticed a few telltale signs a month or so back, when I began getting increasingly annoyed both with my own use of it, and with the way other friends were using it. I found my News Feed full of… well, news, actually. Articles from that day’s news, with my friends’ comments on them. Mainly about gun laws in the US, or about various political thoughts, or the usual flood of crazies and celebrities saying this or that, which friends felt the need to highlight as unacceptable. All worthy commentary. But not what I signed up for.

This used to be social media. “Social”, meaning interaction with friends, news about their families, their relationships, their holidays, their pets, etc. I didn’t sign up for Town Crier Book, or Moan Book, or Conspiracy Theory Book, or Please Pay Attention To Me Book. Again, all valid ways of using the site, but not what I want to wade through every day. It doesn’t represent my friendships with these people, whom I genuinely care about. What’s worse, this kind of interaction now represents about 95% of the content on my News Feed. And so it began to annoy me and yet, because my friends were creating the content, I feels like I can’t really do much to stem the flow.

Facebook offers some laughably primitive ways of filtering content, but no-one really has faith that choosing to receive “Only Important Updates” will actually deliver news of births, deaths or weddings above the endless stream of lolcats, memes, moans and trivia. I’ve often told people not to announce important news on there without sending email too. And yet, I’ve seen marriages, illnesses, new jobs, lost jobs, impending trips… the list goes on, all announced on Facebook alone, with the assumption that all friends will see the announcement. And so, it seems that fear of missing out is the flip side of the facebook agony.

When it comes to my own part in this, I acknowledge that I also mis-use Facebook. I go through phases of feeling compelled to over-comment on my life, or of expecting something out of the site that I know I’m never going to get… and then getting annoyed with it. Years ago, there used to be real dialogue on there: If you posted something interesting, people would comment, show signs of enjoying it, or at least disagree. Either way, there was social activity and a sense of sharing. Lately, it feels like a bunch of people (me included) all talking at the internet, hoping someone is listening… but usually no-one is. It’s like we’re shouting at the moon, somehow believing that the shouting will make a difference.

So, I feel like I’ve been hit by the perfect Facebook storm: A deluge of unwanted info, fear of missing out on important announcements from people I care about, and the compulsion to over-contribute myself. It seems that our very involvement in Facebook leaves little room to tailor that involvement. You’re either on Facebook 110%, or you’re not on it at all. There is no middle ground.

And so, I’m doing what I’ve done with every relationship in my life that hasn’t seemed to be working; I’m creating some distance, with which I hope to be able to better judge the situation at some point in the future. I’m taking a break from Facebook. That’s right; no more cupcake pics, details of pub trips, work moans, etc, etc. Some of my Facebook friends are no-doubt laughing now, because I’ve done this before, and failed. But I genuinely feel it’s broken this time. It’s affecting my life and my friendships. Perversely, I need to risk losing those friendships by stepping away from Facebook, in order to improve them longer-term.

What practical steps am I taking? I’ve deleted the Facebook app on my mobile devices, set DNS on my PC and Macs to point elsewhere and removed any email notifications from the site. I’m basically saying to Facebook “Don’t call me, I’ll call you”.

How long am I going to take a break for? Honestly, I don’t know. I know that I need to get over any “cold turkey” aspect, and so it has to be at least a month to make it worthwhile. I strongly suspect that I will struggle with it for several weeks, and will need to resist the temptation to check for comments on my departure announcement. Beyond that, I’ll see what effect it has on my life. I need to see whether friends make an effort to contact me off-Facebook, which is how I’ll be contacting them. But I also suspect I’ll be back on it in some form within a month or three.

What am I doing to replace Facebook in my friendships? Old-fashioned communication, of the kinds we used to use back in the 90’s: email, phone and face-to-face. I’ll still be on Twitter too, as that serves a different purpose.

If you “Like” what I’m doing, let me know if you try it yourselves. What’s the worst that could happen? We might all rediscover the lives we had before Facebook 🙂

Build something, for someone. Please.

Wooden building blocks closeupUs software engineers build up an impressive list of skills during our careers: We become fluent in several programming languages, often switching between them on an hourly basis. We build websites in HTML, CSS and JavaScript, to be used on a diverse range of browsers and devices. We know our way around several operating systems, many frameworks, different templating languages, regular expressions and scripting languages. On a daily basis, we have to consider aspects as diverse as usability, scalability, compatibility, … the list goes on. It really is no wonder that many of us enjoy our careers and feel privileged to work in such a diverse field. We spend our days immersed in technology, and never stop learning. It’s safe to say that software engineering is far from boring!

But I feel that many software engineers get too wrapped up in all of this wonderful complexity, all of these issues and considerations. We (and I’m guilty here too) forget the most crucial aspect of what we’re doing. The one thing that matters above all else: Building something, for someone to use.

If I said that nothing else mattered without that final product delivery, I’m sure I’d be unpopular. But really, technology for technology’s sake offers us nothing. It’s only when technology meets with real life, and real people, that it has value. Of-course, if we learn something from a project, then that new knowledge alone is a benefit, but only if it leads to subsequent projects that actually see the light of day.

Particularly in our side projects — and it seems we all have them — I’ve seen engineers get overly wrapped up in choosing a UI framework, agonising over which language is most appropriate, arguing almost religiously over databases, fine-tuning the optimal format for transferring or storing data, finding the best OS to deploy on, debating whether it’s right or wrong to have JavaScript on the server, considering whether Java has a future, and worrying whether to use Amazon or another cloud provider instead. All of these aspects are important and considering them has real value, but only if we’re building something for someone to use. In themselves, in isolation, these considerations really don’t mean anything at all.

It’s like agonising over the typeface of a book that never gets published, or the choice of medium for a work of art that no-one gets to see. What does it matter if the project that never sees the light of day was coded in Java, Scala, Perl or C++? Will anyone mind of the website that never gets published is usable, compliant and built upon the best UI framework?

So the whole point of this post is to say, that if you’re a software engineer like I am,  immerse yourself in all of these aspects and enjoy the wonderful, deep, complex, diverse and exciting world of technology out there. But rather than remaining wrapped up in it… please have an end goal of building something with it, for someone to use 🙂