Why Time as a Solopreneur Might Make You a Better Team Player

forwardTeamwork: Good. Lone work: Bad.

Or at least that’s how the stereotype drummed into us through education and our careers seems to go. We learn to focus our CVs and resumés on our team contributions, and to downplay the things we achieved entirely alone.

But what if the things we achieved on our own somehow made us more effective the next time we joined a team? What if a balance of both experiences made for better team players?

It’s my experience that teamwork between people who have never known an occasional lone accomplishment can suffer from a few blind spots: Such team members may have an inaccurate view of their contribution and how to tune it, having only ever seen their work as part of a larger deliverable. There’s a danger of feeling like a full four-wheel contributor, whilst actually running on three wheels and being unaware how much your contribution leans on the work of others. People who have never known personal wins can see a team project as their only chance to demonstrate their abilities, and may be over-invested in personal outcomes rather than team outcomes, safe in the knowledge that they can shine elsewhere or next time.

So what do I mean by lone accomplishments? Lone accomplishments could be a product (software, hardware, anything), a piece of art, training materials, blog posts, or a book. Anything you actually launch, publish or deliver to an audience, for which the entire deliverable depends on you, and for which you have no backup or other input to assist. Anything for which there is no scapegoat once you deliver it; it is yours, good and bad.

I’ve done this twice, with software products of my own. They were far from a commercial success, but the lessons learned were worth more than the time I spent and the salary I forfeited in those months. Since then, I’ve gone back to teamwork in startups, and really appreciated the perspective that my lone achievements have given me.

So what do lone accomplishers bring to a team? How can lone work strengthen and define you?

  • People with lone projects under their belt tend to have a more accurate view of their own strengths and weaknesses, and an appreciation for the roles others perform in a company.. because they’ve probably filled them at some point, albeit probably rather clumsily. Most former solopreneurs will have noticed how they couldn’t possibly fill all those roles alone… and will appreciate that it is only possible to cover them well in a team where a variety of skillsets are present.
    For me, as a software engineer, I now have a great appreciation of the other roles involved in startup life: Marketing, PR, design, QA, Ops, etc. Prior to undertaking my own projects, I probably had a dismissive or inaccurate view of those roles.
  • People with lone projects under their belt already have personal “wins” in the bag, so they don’t feel the need to act as if the next team project is their own pet project or chance to shine. This makes conversations more about a team win, rather than a personal win, less religious, more exploratory, and more practical.
  • People with lone projects under their belt tend to appreciate the difficulties of building something entirely alone, and therefore the need for others. The mature lesson from this is a willingness to delegate… and truly delegate, giving people a long leash and adjusting it appropriately. Knowing I can’t do it alone, but might be able to achieve it with others, makes me value the contribution of others and the need to delegate effectively.

Why is lone work a rarity? Aside from the widespread celebration of teamwork, we probably shy away from lone work because we know it will truly define or break us; there is no middle ground, and so we fear it. Through it, we will either discover, once and for all, that we can lean into a challenge on our own, or that we don’t (yet) have what it takes. Unlike teamwork, there is no scapegoat, no fallback, no-one else to blame. This is a hard reality to face.

But I’m convinced that every day we put off an attempt at lone work, is a day we never really understand what we are capable of; both alone, and in terms of our contribution to a team.

I’m convinced that, particularly in industries that involve building something (software, hardware, arts), we should exercise our lone capabilities on a regular basis, and at least once a year. It’s an ongoing eye opener on who we are, what we’re capable of, and therefore what we each bring to a team.

Far from being a red flag when hiring, lone work — when properly learned from — could be the thing that truly makes a strong team player.

Why Early UX Should Scratch Deeper than the UI

uxThe early stages of software engineering projects aiming to expose anything more than a trivial User Interface (UI) quite sensibly begin with a heavy focus on the User Experience (UX), as this dictates a great deal of what the software needs to do. This is usually a stage of the project that produces many wireframes, mockups, and a sense of what the user — or different user “personas” — will be attempting to achieve with the software.

This UX-centric approach undeniably works really well, and tends to result in software that delivers a strong, usable user experience.

However, I’ve seen a UX-/UI-only focus in the early stages of some projects lead to blind spots and incorrect assumptions which can force costly rework later in the project. For this reason, I’d assert that early UX work should scratch deeper than just the UI, if only to validate assumptions and, in-particular, to confirm what’s actually possible in terms of the underlying data and services.

Server-Side Constraints

Too often, in UX-centric early project stages, the assertion is that the server-side data, APIs and services are much less relevant than the UI. This is often correct as it can distract from the goal of designing usable software, but there’s an underlying assumption that needs validating: That the server can deliver anything the UX/UI work dictates it needs.

This is often not the case… If the server-side data model is built on serious data modelling or clever algorithms, those services and that model are often constrained in terms of what they can provide. There may be heavy computational costs involved in exposing certain data in a timely manner, if it’s even possible.

As always, using specific examples of server-side data in UX work can really help to clarify what’s possible. Even just involving a few token Data / Server-Side engineers can be enough to verbally “sign off” that the server will be able to support what’s required.

This form of early full-stack validation can prevent costly rework later, and can even help to direct server-side work appropriately by providing added context about what the UI needs.

Server-Side Data Model and User Mental Model

An added benefit in considering the server-side data model during UX work is that the user’s own mental model of what they are trying to achieve when they use the software is so often a version of that server-side model.

Considering, albeit leanly, how we might model data on the server can help to clarify the ways in which a user might expect (via their own mental model) to visualise, interact with and modify that data. After all, this is effectively what they’re going to be doing by using the software; accessing and manipulating server-side data and services indirectly via a UI.

Even just tying together the correct terminology, so UX and server-side work are talking the same language, can prevent costly disconnects between different parts of the team.

Again, precise examples of server-side data, or a little involvement of the relevant engineers, can really help here.

Lean Approach to Scratching the Full Stack

But doesn’t involving server-side engineers and data scientists in UX discussions slow down those discussions? Doesn’t it negate the value of considering the software purely from the point of view of the user at first?

I’d say not, but only if it’s clear that the focus of this stage is still primarily UX, and that any deeper layers in the stack are to be validated only. Crucially, any details of how the eventual UI will interact with the server should be avoided, unless they too dictate what is possible in the UI. No-one needs to build the server at this stage (though that work might be proceeding in parallel), so long as they can talk in as much detail as required about what it, and its services, will look like.

 

I think this is another example of how, in software engineering, regularly considering the bigger picture, even when performing early focussed work, can avoid costly mistakes. In complex systems, the bigger and the smaller views are often related or constrained in unexpected ways.

Distil Complexity, Nail Theory, Build Software… Just Not All at Once

questionmark

Trying to pat your head whilst rubbing your stomach is the standard way to demonstrate that we, as humans, find it difficult to perform certain activities concurrently.

The software project equivalent is attempting to overlap activities that teams find it difficult to handle in combination. It’s just a reality that some things are hard enough on their own, and so we hamper our efforts if we attempt to combine them.

It’s been my experience that projects where there is at least an attempt to separate, as much as possible, the following activities are those that experience fewer hiccups. It may still be necessary to iterate through these activities regularly, and they are not waterfall-style one-offs, but being aware of the complexity of doing more than one of them at once can help to simplify things for teams and to ensure one flows into the other:

  • Distilling Domain Complexity – Every software product is built to deliver a solution in a unique problem domain. Often, there is much to clarify in this domain, and much complexity to unravel and distil. This can be something of a voyage of discovery, particularly if it takes a while to extract that knowledge from a domain expert and to figure out which aspects of it the project will require. As building products for a market can be like chasing a moving target, it is highly likely that this domain-related activity will be revisited many times during a project;
  • Nailing Academic Theory – Beyond well-worn Software Architecture and Computer Science principles, a project may rely upon deeper academic theory, such as Artificial Intelligence, Machine Learning, Big Data… or non-tech project-specific areas such as psychology, natural language, etc. Each of these alone are huge fields, full of theory. The problem is, you can’t build software with theory, and so we need to pin down practical forms of that theory that will be implemented in software. This may involve prototyping, comparative analysis of different approaches, and finally a choice between palatable options.
    There is a tendency to linger over these decisions, and an attractiveness in remaining in that land of possibilities. However, software engineering requires knowns, not possibilities, and the sooner theory is nailed down — for this project or iteration anyway, as we can always make different decisions next time — the better.
  • Building Software – Software engineering is concerned with building stable, scalable, understandable software to meet a need. There is sufficient difficulty in doing that in isolation, without overlapping with the previous activities. For large chunks of time, teams need to focus on building software, unencumbered by academic theory or domain ambiguities. It is almost an unwritten assumption on some projects that most of the difficulty lies in the previous two activities (problem domain and academic theory), and that building the resulting software solution will be comparatively easy or, at least, will be more of a known activity. However, the complexities and practicalities of building software require that teams focus on the activity where possible.

Although we may revisit each of these activities many times during a project, identifying and managing the boundaries between them helps to isolate them from one another. One great way to do that is by creating and maintaining great team documentation: Domain knowledge and concrete decisions based upon academic theory are perfect examples where documentation can form a neat output from one activity (albeit perhaps occasionally updated), to be consumed by another. Documentation needn’t be heavy and cumbersome and writing just enough, but before it is needed, should be encouraged.

I wonder, how many times that we’ve seen projects go awry could we identify examples of these activities bleeding into one another too much? And, where projects have gone comparatively smoothly, I’d be interested in hearing whether it was because these activities were understood to be complex in combination.

I guess it’s all part of improving the task of building complex software, with a human team.

Software Frameworks: Occasionally, Reject or Reinvent the Wheel

moonSoftware frameworks are everywhere, and rightly so: Spring, Grails, Angular, Knockout, Backbone… the list is endless.

In essence, a framework is an abstraction upon which we write our app-/product-specific code. We use them to enable reuse of well-worn functionality and, like all abstractions, to hide the implementation and underlying complexity.

Use of frameworks allows us to simplify and speed up development efforts. It’s rather a no-brainer, hence their proliferation.

But under what conditions would we choose not to use a framework or to write our own? Surely we’d just be reinventing the wheel? Perhaps, but just occasionally, maybe the wheel doesn’t fit or we need a better wheel.

Framework Fitness

To be used on a project, a framework must at least be:

  • Functional – It goes without saying that a framework must do what we need it to do; delivering the functionality we require. There must be a clear benefit in using it;
  • Configurable / Customisable – It must be possible configure or extend that functionality (via customisation or plugins), tailoring it to our specific needs, and perhaps even turn off portions of it if we don’t need them or if they are inappropriate;
  • Documented – There must be up-to-date documentation, to make the above possible. We mustn’t need to dig to discover how to use the framework;
  • Efficient – The framework must not introduce performance problems of its own. If there is a convenience in writing our app using a framework, that must not be paid for via a performance degradation we aren’t willing to accept.
    (Anyone who has seen the numerous levels in a Spring stack trace will know how intriguing it is to see just how much is going on behind the scenes, and will occasionally have wondered about the potential performance impact of it all);
  • Proven & Stable – Once we commit to a framework, we are essentially relying on it. The stability of our app / product is directly affected by the stability of the framework. If we want to deliver cast-iron stability to our end users, then we’d better use a framework that itself offers cast-iron stability! At the very least, a framework must have been proven to be robust in production, preferably by someone else, before we choose to launch a product with it;
  • Owned & Supported – If there are unforeseen problems, or bugs — and there are always bugs — it helps if the framework is supported either by a company or an active development community. It is no good waiting six months for a fix to a framework problem that can’t otherwise be worked around.

If a candidate framework is missing or weak in one or more of the above, we might need to look elsewhere or choose to develop the functionality we’re looking for ourselves, perhaps itself in the form of a framework for reuse elsewhere. The cost of ditching or switching frameworks if we don’t assess the suitability up-front can be significant.

Healthy Curiosity

Whether or not a particular frameworks suits our needs, I think there’s educational value in knowing at least a little about the technology that lies beneath that framework. Such knowledge of the foundations upon which it is based may help us to debug problems or, if appropriate, to invent a replacement for the framework ourselves.

Without knowledge of what the framework is doing, we won’t know whether it is doing any good. It is always worth having an idea of whether our own efforts would be better, if only to confirm that they wouldn’t be!

Healthy Skepticism

Blind use of frameworks because other similar projects are using them, or use of them on projects where we use a mere fragment or sliver of their functionality, should be discouraged. The risks probably outweigh the benefits. I think we always need to look at the upside of using a particular framework, specifically for this project.

 

In conclusion, frameworks are largely a huge benefit in software development, but only if we know why we’re using them, when we’re sure that they fit our purpose, and when we understand (at least at a high level) what they are doing beneath the hood.

Just occasionally, omission of the framework or use of own implementation may be a more sensible option. It’s worth remembering that.

Sipping from the Firehose: The next Challenge for Social Media?

firehoseDespite my often tempestuous relationship with it, I love social media. Everyone’s a publisher; individuals and businesses alike.

If you want a platform to say something, you’ve got it. With the click of a button, your content is out there. The potential reach is tantalising, and we all religiously know how many friends and followers we have. Publishing is certainly cathartic and feels productive.

But there’s a problem we’re overlooking: Social media is ineffective for finding and consuming content.

Even when it reaches our devices and screens, there is simply too much to consume or it is unstructured, often interwoven with irrelevant and sponsored content. It is well known that a tweet has a very short lifespan, after which it is almost entirely unlikely to be read, hence a plethora of tools to help us schedule tweets to increase their audience. Due to the way they are presented to us, Facebook posts last longer, but our news feeds have grown exponentially and we seem less willing to scroll through it all.

Worse still, we no-longer even see everything that’s published with us in mind as the audience, because algorithmic filtering is now common, particularly on Facebook.

What does this mean? No-one’s really listening. Well, certainly not the size of audience we expected. The tantalising reach of social media — which basically means your content reached a device or screen — is misleading at best. We aren’t focussing on genuine reach; meaningful consumption of that content.

Even when content reaches its intended audience… their attention span is often scattered, knackered, insufficient by that point to absorb it, because they are drowning in other content. A page view means nothing if it lasted 2 seconds, or they scrolled on by the summary and didn’t even click. A “like” or a “favourite” may not mean they actually read the content, which is surely what we really wanted?

Attempting to sip from a firehose probably isn’t the best way to consume content, and surely publishing on social media is intended as a two-sided affair? We’ve focussed so-far almost exclusively on publishing itself, because it’s by far the easiest angle to tackle. We’ve constructed the firehose.

Isn’t it about time we did something for the consumers of all that content? After all, we spent time crafting it, so we want it to find the right audience and have an impact on them when it reaches them?

Is making social media consumable again its next great challenge?

Software Teams: One Vital Missing Question?

questionmarkBuilding software as a team may appear to be mainly about writing and releasing code. But it is underpinned by something much more fundamental: An ability to define and share information and complexity within the team.

It’s about making sure the ways in which we each model that complexity in our head match those of other team members. Unfortunately this doesn’t just happen magically, despite our best hopes and intentions.

If we aren’t all in possession of the relevant information and a similar view of the complexity we’re faced with, we’re forever building products that no-one wants (disconnected requirements and use cases), wasting time repeating development effort (wrong design / architecture / decisions), or building things that just don’t work (badly integrated, installed or deployed).

I’ve found that encouraging everyone to ask themselves one deceptively simple question, regularly and about almost everything they do, may provide the missing perspective that’s vital in keeping everyone on the same page: “Who else might need to know this?”  Yes, rather simplistic and, of-course, you have to act on the answer, but I think just remembering to ask that question about everything is the key change required.

What kinds of complexity and information are we needing to share as a team? For even the simplest of software projects:

  • Domain knowledge – Every business / sector has its own concepts. People arrive in the team with different backgrounds and therefore with a different need to familiarise with that domain knowledge. Without this, a fair portion of the thousands of little domain decisions an engineer needs to make each day will be inappropriate or wrong;
  • Terminology – For the domain and product, we’d better all be using the same words, right?! I’ve seen projects, where key terminology wasn’t nailed down early, go horribly wrong;
  • Research & Decisions – What we looked into (tech, architecture, anything), the decisions we made based upon our findings and the things we’ve learned. Recording these prevents us from revisiting prior decisions or mistakes unnecessarily, but also allows us to choose to revisit them, pick them apart and change our minds;
  • Architecture & Design – What we think we’re building, and how we’re breaking that down into components, infrastructure, etc;
  • Plans & Tasks – How we’ll build it, the individual chunks of work, sprint-by-sprint planning, agile boards… plus bugs as they arise;
  • Milestones – Points by which we’ll measure our progress. These should be in terms of usable chunks of the deliverable product itself, not just “sprint 1”, “sprint 2”;
  • Progress – Where we actually are versus the plan, and what else has happened on the way (because the plan always changes);
  • Product specifics – Usage, configuration, installation, etc. These arise throughout the project, and need capturing somewhere so the product is installable, releasable, usable, etc;
  • Product – The actual deliverable; probably many different versions of it and details of what went into each of them.

When it comes to sharing this complexity and information, most teams work on the basis that people will always ask what they need to know, at the time they discover the need. This is on-demand sharing. In an increasingly agile world, perhaps it feels like the right approach.

I’d assert that anyone missing a piece of information will often (though not always) ask what they know they need to know, but that may be incomplete. Even if they do in-fact ask, it will probably be for a very restricted fragment of information they know they are missing. What they will be without is context, background, and the top-down structure that would make the fragments more easily digestible by them. Context would also allow them to retain the information and to see the links — the bigger picture — rather than building their own mental picture iteratively from scratch, bottom-up, and perhaps by trial and error.

I’d say we should sharing all of this information much more keenly, and certainly before we think someone else may need it.

But isn’t keen sharing wasteful? Certainly not as wasteful as all those missed connections, delayed understanding, repeated work and the constant need to chase clarity. I’m not advocating writing lengthy documents, just capturing information, making it discoverable and keeping it up-to-date. Once captured, it can always be further structured at a later date.

For some of the information we need to share, there are already great tools out there, particularly task management and agile planning. But what are the best ways to share other information and complexity, and what are the pitfalls?

  • Online docs / wiki (lowest impact, longest-term retention, most transferable) – This is by far the best way to share to allow people to seek out information long-term and efficiently, but it comes with a health warning and an obligation: Wiki contents should be regularly reorganised for easiest navigation and digestion. They should also be updated, or marked as obsolete or, better still, archived when no-longer relevant. If a new hire needs the wiki explaining to them, it isn’t a wiki, it’s a dumping ground! The general rule is to encourage people to leave the wiki in a better state than when they found it. Besides the code base itself, the wiki represents shared team knowledge in its most accessible form;
  • Email – Whenever you email something, the need for which may go beyond a few days or a week, ask yourself the question “How will someone who isn’t on the receiving end of that email get the info?”. To retain the email benefit of highlighting the info to people right now, you may wish to email a link to a Wiki page instead, and this avoids the next new hire being unable to find it. Never email anything that doesn’t need drawing to the team’s attention now. If it simply needs to be available somewhere (a logical “somewhere”, preferably), put it on the wiki. Email inboxes are not a good repository for information;
  • Verbally / Whiteboard (highest impact, shortest-term retention, not transferable) – For the highest impact, nothing beats verbal communication; a stand-up, huddle round someone’s desk or a whiteboard, a meeting or presentation. Much of what was said may be lost beyond the occasion itself though, and it certainly isn’t available to anyone who wasn’t in the room (unless you recorded and uploaded it). Anything shared verbally or on a whiteboard should also be available on the wiki, either as a presentation / image, or by being absorbed into existing content after the meeting. Even just chronologically capturing whiteboard contents can help, and the structure can be rearranged later. Ask yourself if you’ve had to explain the same concept more than a few times. If so, it probably deserves to be documented somewhere.

Building software is about managing and sharing complexity. It may sound rather remedial, but encouraging people to regularly ask themselves “Who else might need to know this?” can affect a powerful change in the way a team works together and shares information. The answer usually prompts keen sharing and, if we also encourage the most appropriate methods and levels of sharing, it means that others will have access to the information when they need it, more often than not. Far from being cumbersome or time-consuming, keen sharing greases the wheels that lead to good teamwork… and perhaps even to better software.

In 2015, Tech Will Continue To Be the Enabler, Not the Whole Story

10251565_830162623679537_14497031_n

“The Year of Big Data”

“The Year of the Cloud”

“The Year of the Internet of Things”

These are snippets of New Year headlines predicting the year ahead… but from previous years, some as far back as 2011, not just from this January.

These technologies have been around, in some form, for a number of years and are inevitably constantly evolving. As with all reasonably new technologies, we have a habit, in January, of hailing this to be the year that they will make their mark. I’d argue all of these have already made their mark in some way, and will continue to do so as the relevant tech improves. But the real story — the real mark — is when organisations commit to and make genuine use of these technologies in business and human contexts.

The real Big Data story isn’t Hadoop, or Apache Spark (though they are the tech enablers)… it’s when a business consumes and makes use of petabytes of incoming and historical customer data to make timely decisions that optimise and improve (and, of-course, monetise) the experience of each customer. Just warehousing large amounts of data and, theoretically, being able to perform distributed computations against it isn’t really a Big Data story; we need to make use of it, in ways that impact and have the buy-in of the wider business. Many businesses have begun doing this in recent years, and that  is the real story.

The real Cloud story isn’t Heroku or Amazon Web Services (AWS), or SaaS, Paas or anything else ending “aas” (though they will continue to be the tech enablers)… it was the point at which businesses could make real deployments of their apps & services, beyond physically renting rack space or buying their own hardware, and could scale those deployments up and down at will to suit their ever-changing needs. It was the point at which budding entrepreneurs could use it to do the same, and bootstrap a startup from almost nothing.

The real story of the Internet of Things (IoT) probably isn’t any of the tech-based articles we’ve read so-far (though those early devices are examples of tech enablers), but will probably be when we actually monitor and respond to our environmental impact, manage infrastructure, optimise energy usage and diagnose or treat medical patients… on a large scale, via the use of connected devices. Some of the devices we’ve seen so far are heading in this direction, but they seem to be more about proofs of concept and novelty, and less about genuine benefit at this stage. So perhaps these are still early days for IoT and we’ve yet to see it make its real mark or write its real story, which will involve way more than the devices themselves.

So whilst it’s great to hail this year (and previous years) as the one in which certain technologies will make their mark, it’s worth remembering that they are merely the tech enablers in a wider business & human story, which is where their true mark will be made. Or else, what is technology really for?

The Great Offline

yorkshire-landscapeWell, not entirely offline. I didn’t actually spend New Year in a cave. But I did decide to spend the last two weeks of 2014 away from most forms of social media: Facebook, Instagram, Twitter. Basically anything I felt I was over-using and could do with a break from.

It seemed a good time, from 18th December until 1st January, to remind myself what the world is like without those sites and that minute-by-minute urge to “contribute”. Aside from the usual seasonal socialising and some much-needed down time, I planned to focus on other projects and prepare — at a more leisurely pace — a few lengthier blog posts.

After the first two days of the self-imposed social media break went by, it became clear that I needed to widen the scope. The first sign was the recruiter who messaged me on LinkedIn but who then took three replies from me to convince that I really, really only wanted to talk to him in January. Then there were the emails from folks wanting to arrange work-related drinks in the final week before Xmas. So, whilst I still checked my email once a day, and promised myself I’d reply if anything urgent came up, I decided to leave the rest there in my inbox, unread other than their subject line, until 1st January.

The relief at dropping the social media frenzy for a while and letting email pile up was rather unexpected, palpable and amazing. I rediscovered a slightly-less-connected world and, strangely, one in which I felt more deeply connected to real things: Face-to-face encounters weren’t interrupted by Facebook’ing, phone calls had my full attention, meals cooked at home didn’t get uploaded pictorially to Instagram. They just got eaten. I know, how refreshingly retro!

There were also other benefits: This blog post, to my delight, is easier to write without the constant distraction of inbound social media. I also drafted a longer software-related post for LinkedIn next week, the flow of which seemed to have eluded me until then but which came easily during the break from the constant notifications. I got my head around a new product idea that I’d been failing to define beforehand, and which I now found the focus to think deeply and easily about.

By unplugging from the distractions, and yet still remaining online in other ways, I regained a degree of that longer-term focus required to cultivate ideas, respond at-length, react appropriately and digest the articles and news that I was still choosing to read. Regaining that focus, albeit briefly, gave me a real sense of what social media and always-on email is doing to us. It’s somewhat scary what we’ve lost without even noticing, and how short-term and reactionary we — and hence our lives — have become.

Of-course, now the two weeks are over, I’m back in the social media world and my inbox has been tamed again, I realise I must have missed things whilst I was away: The constant commentary on every news story. The social status bingo of everyone else’s Xmas and holiday pics. The reactions and counter-reactions to every domestic stress and strain of the season. Minute, by minute,… by online minute.

Yeah, on second thoughts… perhaps I didn’t miss that much really.

Happy New Year!

The Best-Laid Plans of Task-Based Engineering

018-linkedinWhen faced with the challenge of building software of any considerable size, we rather sensibly tend to begin by breaking down the complexity and fleshing out a design or architecture. Then we break down the work involved in building that design into tasks or user “stories”.

Sometimes we group stories into wider “epics”, themes or stages, but generally we estimate, schedule, and conduct the work of building software at a task level: Tasks are what we assign, place on agile boards, and discuss our progress in terms of.

(I’ll refer to everything as “tasks” from here on, but I also mean user stories, and much of what I’m referring to also applies to bug fixes and other ticketed items)

Tasks are abstractions of the underlying engineering work required, and a task-based view of engineering makes the process of building complex software more understandable, manageable and distributable as a team. But it also tends to ignore certain truths about software complexity, tasks themselves and the ways we actually work on them, both on our own and together in teams. In-particular, the tools we currently use to manage task-based engineering are rather restrictive and cumbersome.

So, given that it seems we need task-based engineering, what is it about the way we approach it, and current task-based tools, that can hamper our efforts?

Tasks Abstract Away Complexity

A task is simply an abstraction of the work involved in building a software feature, a use case, a component or maybe just of fixing a bug. As with all abstractions, it necessarily hides detail that, for all but the simplest of tasks, may be discovered later when we actually work on it, despite our best efforts at planning time.

Sometimes, as the true complexity of the work involved in a task becomes clear, we realise that we should restructure the task to break it into more manageable chunks, and that sub-tasks should ideally replace some (but perhaps not all) of the original parent task. This discovery of structure and complexity may occur several times, to an arbitrary depth, and perhaps only after part of the original task has already been completed. We may even discover aspects of the original design, and the tasks slated to build it, are wrong.

Rather than implying poor planning, this is simply a part of the discovery process and the complexity inherent in building software, particularly in an agile manner where we’re trying not to pre-think too much up-front. We need to accommodate that inevitable change, reflect it in the tasks we are working on and in those slated for the future.

Most current task management tools aren’t great at letting us change anything more than very small numbers of tasks and task-based planning. They certainly don’t help us to visualise the results we end up with. In a sense, they leave us with quite brittle and sizeable task-based plans. With some persistence though, it is usually possible to reflect the required changes, though this may require time that isn’t available and we risk the task-based view drifting away from the engineering reality.

What can also be particularly tough with current tools is arbitrarily defining sub-tasks to any depth, as we discover complexity and break down a piece of work further. We may need to have parent and child tasks representing slightly overlapping and part-completed functionality, whilst being able to see estimates for all of that work. I’m not sure that current tools allow much of this without significant discipline in our use of them. In this sense, they are too brittle to allow us to reflect the engineering reality beyond our (now outdated) initial plan.

Tasks Need Surrounding Context

Our planning process envisioned the whole (a delivery, a completed system, or a bunch of features for a sprint), then broke it down into tasks. Quite often, rather a bewilderingly large number of them. The tasks somehow lose our view of the whole, and it is hard to determine whether those tasks, collectively, still make up a meaningful delivery or omit anything crucial.

It seems like we need to retain some visualisation or record of the structure by which we arrived at the individual tasks, so we can work backwards from the tasks to the whole. Even when working on an individual task, we often need to get enough surrounding context to work on it, as each task rarely describes that in sufficient detail.

Being able to see where tasks fit into the bigger picture is particularly necessary when our plan inevitably changes over time, to avoid us getting lost in a changing sea of tasks, the intended outcome of which we can no-longer visualise clearly.

Current task management tools don’t allow us to retain the more complex structure of groups of tasks. We are limited to “epics”, tags, simple sub-tasks and basic categorisation. We need better ways of visualising how the tasks fit together to form the whole that we’re working towards, allowing us to spot omissions and to track progress. After all, the collective structure of the tasks will probably be as complex as the structure of the overall system we’re building.

Tasks Need Collaboration (Sometimes)

No-matter how much we break them down, often a task requires unexpected collaboration with someone else (together, or a hand off then a hand back), integration with their work or reaching an agreement. That micro-collaboration is hard to model, but crucial to a task’s completion, though we often assume (perhaps hope?) it is irrelevant.

Whether or not one person “owns” the task and the responsibility for its completion, several people may need to do work on it and must find time in their workload. This is particularly true with user stories, as we don’t tend to break them down into chunks any smaller than a story we can define in business / product terms.

Current task management tools suggest a single assignee per task, or multiple assignees but with no record of individual effort required or how the collaboration will occur. We actually need to reflect a more complex sense of ownership and collaboration, even if most of the work will be done by one person with fleeting collaboration at some point. Failing to capture this in a basic form can be a cause of tasks being unexpectedly “blocked” during sprints, or the logical next steps towards their completion being unclear.

Tasks Have Dependencies (Sometimes)

Sometimes, a task depends on other tasks being completed (or part-completed), or a bunch of tasks need to happen in a certain logical order. Often a task that is scheduled to be completed this week or this sprint is blocked by an implied dependency that won’t be resolved anytime soon. We must model dependencies, or risk impacting our progress if we discover them late, or if many tasks get blocked on a key resource that we should have planned around earlier.

Current task management tools can model dependencies clumsily at the task level, but aren’t good at helping us to visualise them. This can often be another cause of tasks being blocked during sprints. They certainly aren’t good at modelling dependencies between groups of tasks, and we tend to have to visualise those ourselves: e.g. All tasks in Stage 2 depend on the tasks in Stage 1.

A visual representation of dependencies at a task and group level, and ability to move them around, would give us a more efficient view on how best to approach those tasks in a sensible order.

Task Completion Often Isn’t Clean

Often a task is almost completed, with a small piece of work to be finished, perhaps when unblocked by another person or after a dependency or decision is resolved. The inability to easily mark the task as partially complete, or to define the remaining work (perhaps in a sub-task), hampers visibility of the true state of the task, particularly on agile boards.

Creating a new sub-task can be cumbersome, and it needs adding into the plan or sprint, in correct relation to the original task, so it isn’t forgotten or confusing when seen in isolation. Often the administrative effort leads to us just leaving the original task open until the remaining work is completed.

Current task management tools assume task completion is clean and atomic, leading to tasks remaining “in progress” when they are often 90% complete and the tool should allow us to easily reflect that. This often hampers visibility of remaining work, particularly close to the end of a sprint.

So What?

Clearly, I’m pointing out some broad problems with simplistic views of task-based engineering here and suggesting some very general solutions. Not all of these problems will apply to a project but, particularly with larger teams, I’ve seen many of them and their impact is real. Some teams tackle these problems well as part of the way they work, but others clearly struggle with the realities of task-based engineering.

A quick summary:

  • Tasks are valuable abstractions for understanding and managing engineering complexity, but the methods and tools we currently use to manage them are rather rigid. We need to be able to visualise and change large numbers of tasks easily, see their overall status and visualise how they combine to form a planned delivery or feature;
  • We need to be able to break tasks down to any level, describe what overlaps and what’s already complete, so we can model our fluid discovery of complexity but retain an ability to estimate and gauge the remaining effort involved. Again, current tools and methods are too brittle;
  • We need to be able to model and visualise task and task group dependencies, where they matter (they don’t always), or even just record that entire swathes of tasks must take place in a certain loose order. Current tools make this too cumbersome;
  • We need to model the necessary collaboration within individual tasks, both simple and more complex, so we can be aware of it. Current tools are too cumbersome to make this effective or even worth the effort, and agile boards are too simplistic.
  • Most crucially… All of this must remain light, easy to visualise and easy to change. Current approaches and tools are way too cumbersome, and often work against us.

As they say, “Your mileage may vary”. So I’d be really interested in hearing how this ties into your own views of task-based software engineering, whether you encounter any of these (or other) problems, and if you think the kind of broad changes I’m suggesting might improve things.

One thing is certain: Task-based engineering is here to stay!

The Stuff I’m Thankful For

mandmsAlthough we don’t celebrate Thanksgiving here in the UK, I like the way many other folks are using it as a chance to express what they’re thankful for. I suspect we don’t do that often enough.

So, in another break from tech posts, here goes…

Passion – I’m lucky enough to have found a career I love, from an early age, and to know 100% that this is still what I want to do. Although my precise job / role may change, I will always, at some level, be a software engineer. Not many folks find their passion this easily, and so I’m really thankful that I have.

Health – At 43, I’m surprisingly well and — although the people telling me this usually do so in dark bars! — I’m told I don’t look my age. <looks in nearest mirror> I still love exercise (not enough, but I’m working on it), walk wherever I can in this amazing city and, touch wood, nothing about my health currently stops me from enjoying that. I’m thankful I decided to ditch the drink 9+ months ago. It wasn’t causing me problems but, via some other inspiring people, I caught a glimpse of what life without it could be like, and ditching it has made me realise how much of an energy / health drain it was. So I’m thankful for that. And my liver is thankful too!

Friends – I firmly believe I’ve been one of the luckiest people ever, in terms of the friends I have. I don’t have as many as some folks, but the ones I have are people I can truly rely upon. Collectively, they know me from all angles, flaws and all. Between them, I have people I can be stupid with, chat seriously with, talk intellectually with, or just have a fun night with. It can sometimes be a lonely old world and so this, more than many other things, I’m thankful for.

Fun – I still appear to know how to have fun and I’m thankful I haven’t forgotten how to do that. Life can be too serious. I’m massively thankful that this is also still true without alcohol, as I know many can’t do it without a drink. I am still the guy who’s up and going at 3am… with a lime & soda.

Confidence – Lastly, I am finally comfortable in my own skin. I think that’s all I need to say about the last one 🙂