Perspective, Software Development

For the love of the User

Software is for the user. It is not for the Software Engineers who develop it. In the end, software will succeed or fail to meet user needs. The user is the arbiter of software’s fate. Oddly though, many software developers tend to resent their users. The users are prone to strange behaviors. Sometimes they can even come across as whinny children to jaded developers. But we must do away with this flawed way of thinking. We must act as humble stewards, gentle of heart, and eager to please.

Users are the life blood of a software product. Without them, the product will fail. As a result their needs are paramount, and must be address to the best of our abilities. If this is the case, then why are developers so often frustrated by their users? Remember we are fluent in the machine tongue. Generally speaking, users aren’t. Sure they can use the machines, to a limited degree. But they don’t understand them like we do.

Imagine you are in a foreign country. The only way to get your work done is to cajole a lumbering beast into action for you. Without understanding the beast’s language, even simple tasks could be infuriating. Users who are less familiar with software might feel the same. Only remember that we specialize software to particular tasks. As a result users need to learn, remember and use a variety of these ‘beasts’ to get their work done. Also remember, they are being evaluated by their ability to get work done, using your software.

And so scared, frustrated, and feeling impotent, they turn to us. They wonder why their actions did not work. They ask for strange features or work-flows. All these feeling arise because they don’t understand their tools. Sure we could ‘educate them’. But if the way to use a tool is less than obvious, or they only use it seldom, then you can expect them to forget. Not to mention, you have to convince them to take the time to get trained, rather than working. Even we don’t feel comfortable trading training time for working time. So why should we ask that of them?

Two paths remain to us. We can tell the user’s they are wrong and constantly bicker with them, trying to explain the proper way. Or we can choose to listen. The way we thought was obvious is not. They need more help, because the grammar of machines is difficult. I would call this path ‘Stewardship’. We have to think of the code as belonging to the users, not to us. In so doing, it becomes clear what choices we need to make. If the code is for the user, then their needs overrule ours. If they aren’t fluent, we must may the software more approachable.

We are like gardeners. The land we tend is not our own, but still we make it bloom with brilliant flowers. We cherish the blossoms, and suffer when they are trodden upon. But the garden is not for us. Imagine if the gardener chased off the owner with a spade when he ask for a new row of lilies. The gardener would be marched off and a new one brought in to replace him. This is not an exact analogy, since users pick their software. They might just avoid a certain gardener altogether.

If instead, we are gentle and approachable, we could better tend our gardens. If no one ever walks our garden paths, then we put to waste all the love and beauty to garden contains. Software without users, despite its brilliant design, and delicious complexity, is dead. If we want vibrant, living software we must serve our users. We cannot lord our understanding over them, but must instead steward the code for them. With gentle hearts, we can learn their needs, and make the garden they need. In the process we may discover an even greater beauty.

Advertisements
Standard
Perspective

ROI of Training

What kind of investments do you make? Do you favor immediate returns on investment? Or do you favor guaranteed returns? How long are you willing to wait? Time spent in training or practice is equal to making an investment. Different methods or focuses produce different results.

Much of the training available in the software industry focuses on new frameworks. With a myriad to choose from, there is no shortage. There are many introductory courses. All encourage picking up the tool and applying it to basic problems. Yet these frameworks are subject to change. Two years down the line the framework will change. Sometimes in two years it can become obsolete. In other cases, it becomes an industry standard.

All in all, these skills degrade. Some of the degradation comes from market changes. Lack of practice also contributes to the decay. How often do you truly use that obscure array access format in language so-and-so? Rarely, for most of us. Yet there is a class of training and investments that are less likely to degrade: People skills.

People skills are usually presented in management or leadership courses. They are an investment class of their own. The opportunities to practice people skills are vastly more numerous. As a result they do not suffer as much ‘lack-of-practice’ degradation. Furthermore, people skills remain in demand for many higher level positions. Want to be a consultant? You need People skills. Want to start your own company? You’re gonna need people skills. But these skills are difficult to acquire. In fact, these skills are in high demand precisely because they are difficult to acquire.

People skills are also applicable across industries, if you ever wanted to move. The skills of a software developer carry over any industry we develop for. Much the same way, the core skills of a manager translate well across industries. As career capital, they pay large dividends.

The best investment for anyone strong depends on what they want from life? I enjoy the challenge and rewards of programming. But I am interested in the role of management, and in its unique challenges. With an eye to the future, people skills appear to be the best investment. The skills suffer less degradation with time, and have remained in demand over the long haul. What do you value in your investments? Do you want to expert in technologies? Or do you want to diversify? Hopefully this perspective provides another lens for reflections.

Standard
Perspective

Software Engineers are actually ‘Creatives’

My Blog has moved! You can find my latest content at daniel.scheufler.io. Please continue reading here.

Have you seen the Project audio for Github? Or the plethora of esoteric languages like Piet or ><> (pronounced: fish)? Recently I had a ‘coding challenge’ for fun at work. The challenge was to print a poem in a language we didn’t already know. In that time I’ve picked up four such languages, which got me thinking. We, developers, enjoy some of this artsy kind of stuff. Sure it’s not your typical artwork, except for Piet. But it is very creative at heart. Then the light-bulb lit up, Software Developers are members of the ‘Creatives’ community.

To be clearer, ‘Creatives’ is the group of designers, and artists who contribute to projects in more media-centric ways. For example, the icons, the color scheme, or the marketing campaigns and slogans all fit the bill. Software Engineers are usually inclined to disdain the ‘Creatives’. This is because it is harder to measure what creatives actually do. But I suspect there are some lessons we can learn, if we would open our eyes.

Generally, one manages ‘Creatives’ in a particular way. The method enables the creative freedom. It also establishes the safe environment needed to ‘try something new’. This allows them to bring their brilliance to any given project. This environment makes sense for the product they deliver. If a creative team does not feel safe, very few ideas appear, leading to lack of success. Of course, such an environment is difficult to setup and very easy to tear down. One cruel word, or breach of trust and the system comes toppling down.

In contrast, many software development teams are more hierarchical in their style. Additionally, there tend to be more men than women. As a result, software teams tend to favor codified leadership. We like increased authority rather than community, at least in traditional corporations. Most design ‘discussions’ are arguments in fancy dress. And sometimes this can work well, since engineers usually like to debate. But we should consider adopting some strategies from the ‘creatives’ side.

A safe environment with respectful discussion rather than debate could be desirable. This environment would allow us to foster new and brilliant solutions. In order for this to work, software developers would have to realize they are ‘creatives’ too. We would have to change our ‘comfortable’ behavior to allow for a better team environment. This, of course, comes down to culture, but also to maturity. We must be mature enough to admit we might have missed something. And we must be mature enough to want the best ideas, rather than our own.

At a manager’s level, if Software engineers are creative, then we ought to manage them differently. Dictating the chapter and verse of a solution you want is unproductive. It limits the benefit you can gain from your brilliant engineers. Instead, we need to challenge them. Provide them a problem and your rough idea of a solution and then encourage improvement upon it. Don’t dictate your needs as you would to your digital assistant. Rather begin a discussion about the best way forward. Before committing to deadlines, allow for ideas to circulate and then commit. Who knows, maybe we can learn something from those crazy ‘creatives’ in the east wing after all?

Standard
Perspective

Software Developers are Translators

What is a Software Developer? I know they ‘develop software’. But that is like saying water is wet. What is the fundamental action we train software Developers for? Could you use it to distinguish to excellent from the mediocre? I’d say you can. The fundamental task for Software developers is translation. Software Developers are in essence, translators.

Software Developers are individuals who can speak both to man and to machine. At a low level, they literally translate human sentences and ideas into instructions for computers. Ever used a poor translator app? Then you know proper translation takes some finesse. You have to understand the culture of the language you are translating to. You need to understand the idioms and the proper grammatical structure. Otherwise your translation won’t turn out well. It might sound like terrible high-school writing. Worse, it might be a hollow mechanical echo of the original work.

The difference between a ‘good’ developer and a mediocre one hinges on the mastery of the language. The good developer knows the right idiom to convey the fine details of a phrase, while the mediocre developer might be able to eventually explain the instruction. The good developer can instruct with elegance and in some cases even a flourish. This even extends to translating behaviors. In technical language this is the UX. A good developer accounts for the expectations and the wants of those using it. He elegantly handles the use case. The trouble comes when trying to quantify these differences. They are differences in quality rather than quantity. But that is for a different discussion.

A good developer also recognizes the short-comings of the medium. Some things can translate when in their original form, others need to be recast. The target language/culture may not be able to sustain the desired content. Vocal inflection is useful for in-person discussion. They do not translate well into text message. Instead they need to be recast to maintain their emphasis. Put in plain language, computers can only do so much. A good developer recognizes the limitations of the system. He then communicates these limits to the humans he is translating for.

In a proper environment, the software developer would act as a representative of the machine to man. The good developer would help him to understand the needs and abilities of the machine. At the same time, the developer would act as a representative of man to machine. He would ensure that behaviors would meet expectations of man. These two tasks would form a cycle. The developer brings the needs of man to the machine. Then he returns with the limitations and requirements of the machine. Over several cycles, and accommodations, we find a successful system. In spirit this cycle is like Agile software development, though without the extra trappings.

Now if a Software developer is a type of Translator, should we change how we measure them? We can or ought to borrow from the way the Translation business operates. Or adopt their performance measurements? A good software developer acts as a fluent translator between the worlds of man and of machine. Through clever use of language, they translate the desires and needs of men to the machines. And then they bring back the requirements and limitations of the machine. In a proper cycle, both can prosper from this feedback. The good software developer is like a good translator. He is aware of the idioms of his target language as well as the culture. Thus he can make an effective, compelling translation of the original.

Standard
Software Development

If you give a Dev a board game…

From my first lecture on C, I have been tinkering with side projects. I’ve done projects purely for exploration and entertainment, like a text-based adventure games. More recently I’ve done utility projects like a script to correct QIF formatted text. Recently I took on a project of a larger scope.
 
A while back,I read an article about a simulation of Machikoro. It is a ‘city-building game’, with rules that are easy to translate to code. In particular, the idea of using the simulator to ‘evolve’ an optimal strategy for the game captivated me. This was applying Machine-learning to a board game. I figured ‘I could do that’, and got to work. I encountered many distractions and set-backs, including a new baby. But this month I am pleased to admit that I have hit a milestone.
 
To support the ‘evolution’ aspect, I had to be able to run thousands of simulations in a reasonable amount of time. And after a bit over a month of concerted effort, I made it. I took my code from being a collection of classes to a library and simulator able to run 1000 games in 15 seconds.
 
I started back in December with classes to represent the deck of cards, a strategy for play, and a player state. The first step after this was to create a basic AI* to act upon the player state, and a given strategy. Borrowing from the article I had found, I decided to make the strategy more static. The decision logic reduced to constant decisions like ‘always yes’, or ‘always the cheapest available’. Then the AI only needed to use the Strategy to answer queries from the Game.
*Note: I am capitalizing and italicizing Class names for ease of identification.
After the simplified AI was complete, I got to work on the Game, which would simulate a single game. I decided that I wanted to use fluent APIs to instantiate a Game. I spend a good chunk of time to get these write, but it helped to make the main routine clearer. While I developed the Game, I decided to abstract the mechanisms of the game. This allowed me to separate the calculations from the sequence in which they are applied. I extracted the Engine to handle things like calculating which AI if any has won, or how much money this AI gets with this dice roll. Meanwhile the Game can manage whose turn it is, and who rolls the dice.
 
Testing both the Game and the Engine were somewhat arduous, but it was time well spent. I caught numerous bugs, and infinite loops before I ever ran a full simulation. Thankfully the Deck, State, and AI were all similarly tested. But I do wish that I had adhered more tightly to TDD. Instead I was very eager to getting the core functionality working.
 
Once these pieces were in place, I initiated my GitFlow, branching Master, Dev, and a new Feature. After pushing version 1.0 to Git, I started work on a new Feature, multi-game simulation! And while I tinkered with a Simulator, I realized that my fluent APIs had a bug. So I went back to Dev, and produced a Hotfix, which was merged into Master. From there I re-based the Feature, and continued my work.
 
With the Simulator, I needed to initialize a Game, but also to be able to run it N times, without interference from the previous rounds. So I had a two-pronged approach, I would accumulate the results of each game, and I would allow a Game to be reset. Learning from my forebears, I was sure to include randomization of the first-player when I reset. This removed the skewing of First-move advantage from my results. With the core Game working and fluently initialized, I was able to simple inject it into a Simulator to run.
 
The original simulator was able to run 1000 games in around 80 seconds. This performance is alright, but my personal dev box has 8 cores and the Simulator was maxing out just one. So to improve performance , I began to look into Python multi-threading. I found two similar flavors of concurrent operations in Python.
I elected to try Tasks first, as it seemed similar to Microsoft’s Task Parallel Library. Sadly I was not quite right about that. The BatchSimulator’s performance was terrible. For some reason it never used multiple cores. The original time for the BatchSimulator was 150 seconds for 1000 games. While it is likely this was user error, it was enough to discourage me from pursuing Tasks further.
 
So I turned to concurrents. And with concurrents, I had much better luck. In this case I spawned some sub-processes. I created the Coordinator to provide each fork with its own copy of the given Game, and an assigned number of games to run. Then each fork created its own Simulator, and ran the given number of games. Once each Simulator completed, the Coordinator would accumulate the results. After all the forks completed, the coordinator calculates the final statistics. This provides an overall winner. To make this easier, I extracted the SimulationResults class. I then added public methods for merging and calculations. By leveraging sub-processes, and existing code, the Coordinator was able to run at least 1000 games in ~16 seconds. Now I say at least, because the Coordinator divides the games evenly among the sub-processes. So to ensure that at least 1000 games are run, it must round up on the division of games per sub-process. But having more data is never a bad thing.
 
I was able to push and close this Feature recently, and I am very pleased with the progress. I went from single game simulation to rather performant 1000 game simulation in a month. I now have something to show for my ideas and my work. This milestone leaves me at a good break point. I can either continue working on the simulator to pursue the machine-learning angle. Or I can change focus and return to this project later. At the moment, I don’t know what direction I will turn. But I wanted to take a step back and look at what I have accomplished, and share my ‘geeking out’ a bit.
 
If anyone is interested in the source, you can find it here.
Standard
Perspective

Amish Software

Last week, I spoke about ‘Resuscitating the dread word ‘Agile’ ‘. The core of that post was that Business does ‘Agile’ wrong. Business misunderstands what it is buying when it pursues the ‘Agile’ practices. Business thinks it is buying a software product. But what agile provides is usually strategic information about customer desires. These ideas continued to boil in my brain when another thought hit me. The software Industry is still very young. We’ve only been making software since we had computers. By contrast, we’ve been making furniture much, much longer. Like since we’ve had agriculture…

As a result, we haven’t had time to discover suitable manufacturing patterns. That being said, many businesses are seeking these patterns from other industries, for example furniture or automobile manufacturing. Software Businesses have been searching for ways to apply the old practices in an effort to get a handle on this new beast.

From what I’ve seen, read, or heard, there are a handful of predominant patterns. Today I want to talk about two archetypes that I have experienced. The first pattern follows my previous post. It is the Business ‘Agile’ way of development. We get short, controlled, well-measured, impotent steps. We know when we’ll finish, because we can measure the man-hours it will take to produce the rest of the pieces. Like knowing how many steps are left in the process on a factory line. Each man has a task he must do on the item, and then he hands it off to another person. Sure someone might be working on a wheel, and another on the engine. But he needs to finish polishing the mirror. Then later they can put all those things together and out rolls a new Model T.

For those who have experienced this style of work, it can be draining or down-right degrading to your spirit. You become the cog in a machine that produces standard measurable quality. There is little for you to be proud of, nor any part of the work that could be held aloft as the epitome of excellence. To put it bluntly, these practiced vulgarize the creative work of software engineers. It encourages management to think of the developer as expendable and replaceable. Note, I am not arguing for indispensability of the engineer. Instead I would argue for recognition of individual value and contribution.

In contrast, I have worked for a shop where developers behave more like craftsmen. The best allusion that comes to mind is a weathered carpenter sanding a table, or assembling a barn with time-tested methods. He rests easy knows that his creation will stand for generations. To be completely honest, I think of the Amish Barns I’ve seen up north. Specifically their construction techniques that required no nails.

The developers at this shop naturally wore many hats, from testing to deployment. But when you spoke to them, you could tell they were pleased with their work. Discussions were lively, and issues were rather easy to resolve, even across larger systems. As you can guess, this shop was much smaller in scale. It was also lead up by two seasoned Software Engineers.

So, the two styles are Factory-line, or Craftsmen. Speaking from experience, I by far prefer the craftsmen shop. The culture of the office was freer, but maintained a strong discipline. Not only that, the output of the office was a higher quality too. That is not to say that high-quality code did not emerge from the Factory-line. Merely that producing such quality was more difficult and less common in the factory-line shop.

With these two types in mind, I realized that the differenced weren’t merely in style of management, but ran much deeper. The first difference I found was in the language that each shop favored. The Craftsmen shop favored a recent edition of C++. The Factory-line shop favored C# for new development. The language differences were just the surface, but they are a good touch-point for the pattern of differences. One shop favored older, well-worn and stable libraries. The other favored newer, less stable and generally more finicky libraries. And when I realized that, I saw that the problem was also with us, the software developers.

Perhaps it is a reaction to Business ‘Agile’, or perhaps its just a part of the culture. We scrambled for the ‘new technology’ that will ‘solve all the old problems better, faster, easier’. Of course, after a time, experience teaches that this can never be the case. But on the whole, the software development community leans towards neophilia. With the rapid change in our tools, it appears we may also be drifting towards less proficiency with these tools.

And that is when it hit me. It’s not only business that does agile wrong. We do it too. In fact, we might even be pursuing software development the wrong way. In the rush to find the new, to get the business people off our back, we haven’t had time to look around for another option. At least, those of us who work for business majors haven’t. Perhaps companies founded by software developers can still get it. Or maybe it’s a size thing.

So as I see it we’ve two predominant patterns. First, we have the factory, with its pre-cut particle board. And its pieces precariously held together with glue and the prayers of the innocent. AS one alternative, we’ve the Amish carpenter, crafting his table, or drawers. Using well known and well-worn tools handed down to him by his father. The factory furniture might be ready sooner. It might be cheaper too. But the table so finely crafted, will stand the test of time. Such quality comes at a cost. The question before us is: Is quality worth the cost?

The question isn’t just about price. It is also about practice. What are we willing to give up to gain quality? Are we willing to work with older, more stable technology? I know that when I started in the work-force, I would usually sneer job postings that listed ancient or arcane languages on them. I am beginning to reconsider my stance. What do we want to be? Factory cogs, or Amish carpenters? Or perhaps is there another way?

Standard
Perspective

‘Code is read more often than it is written’

My Blog has moved! You can find my latest content at daniel.scheufler.io. Please continue reading here.

At first glance, this would seem an obvious statement. And it is in a way. When Python language creator Guido Van Rossum created Python, he did so with this thought in mind. As a result, the culture of Python is partly molded around “readability counts”.

The more I thought that statement, the more I realized the marvel it held. ‘Code is read more often than it is written’. If asked to choose between reading and writing, I would have said the same. And yet I realized now, that much of the code I have read, and some of the code I have written does not show this. I wondered, why did my behavior, and that of my peers, not match what I knew to be true? If we believed that code was read more often, they why is so much of our code so hard to read?

At the core, our behavior remains unchanged because this quote is only an observation. There is not imperative contained in it. Without the means of an imperative, the observation cannot turn into an action. Instead the reader would need to derive ‘Code ought to be easily read’ from ‘Code is read more often than it is written’. I trust most would be equal to the task, given a basic desire to optimize.

When I first discovered this, I did not pay it nearly enough attention. I went blithely on my way. Some time later, during the quiet of a vacation, the thought came storming back. I was left dumbfounded. How could I have not seen it earlier? I realize now, it was because I had not given my self enough time to think. With the lighter load during vacation, I was able to think, and so naturally the thought came.

This moment of serendipity also encouraged other considerations. Specifically, what other imperatives had I missed with casual observations? I quickly realized this is dark territory. It would be difficult to turn every observation into a possible imperative. Worse still, these observations might be biases, leading to bad imperatives. Or they might be too weak to lead to a meaningful imperative.

In all cases, the question remains, what have I missed? I believe, especially in software, that we are caught in a rush to develop, to implement, and to finish. As a result, we do not give ourselves time to ask, ‘Is this the best way?’ Business demands that we move with purpose, and that is a reasonable demand. But for the best results, we need time to consider if we go in a way that will deliver us to the goal we seek. I will continue to look for miss-able observations may turn out to change everything.

Addendum:

While drafting this article, two other examples of ‘observation leading to imperative’ appeared. The first was fictional, from Foundation and Earth by Isaac Asimov. In the book, the protagonist remarks with surprise at the neural interface to a computer. Instead of being an over-the-head affair, it was through the hands. His realization was that humans sense and interact with the world through their hands. I may revisit this in a later branch of this discussion on design.

The second example, sprouted from the first, specifically interaction and design. Recently the IoT movement has brought integration to our homes. In particular, the voice-interaction, such as Amazon’s Echo or Google Now. I observed that these devices extended a natural principle: ‘Humans use their voice to make their wishes known.’

Standard