Software Development, Work Projects

Going where no QA has gone before!

As a developer having QA you can rely on is great! They are welcome friends helping us cultivate our precious software. But there are dark places which even a QA cannot shine a light. When your software has no interface, what can a QA do, but wish you luck? But what if there was a way for QAs to interact with otherwise UI-less software? Enter Cucumber, a tool that allows QA to shine a light in dark places.

I rediscovered Cucumber, while researching test automation frameworks. Cucumber is a framework for Behavioral Driven Development. After experimenting for a time, I realized Cucumber opens a whole realm of possibilities. Cucumber encourages the expression of program actions in the human tongue. With a proper translation mechanism, Cucumber could act as a mediator between QA and the UI-less software. 

Cucumber translates the human tongue into functions through the Gherkin language. For example, a tester would define a test case like this: 

Scenario: Messages are saved until the consumer arrives
Given the queues are empty
And I publish a message to the queue with ‘SomeDetails’
When Alice subscribes to the queue
Then Alice should receive a message with ‘SomeDetails’

It is fairly easy to understand the behavior that is being described in this scenario. Cucumber ties the keywords Given, When, and Then to functions which execute the described action using a Regex Match string. This can include free-hand parameters such as ‘SomeDetails’. 

Properly designed, the Givens and Whens can be setup to be repeatable and re-compose-able. Doing so allows the QA to describe more complex scenarios with different combinations of the same simple behaviors. As a result, once the initial steps are available, a QA could test to their hearts content with little developer support.

Cucumber improves the documentation of a product. Test document expected behaviors in a common tongue. This makes them available to all parts of the company.

But great care must be taken to ensure that the compose-able parts function precisely as described and without side-effects. Imperfections in the design or the aforementioned side-effects will destroy test-validity and erode trust in the test cases written using Cucumber.

Cucumber was designed to improve TDD, enabling members of a team to describe the function of a program in a human tongue. This same feature creates a tool for empowering QA. Given careful planning and design, you can compose a terse but flexible set of instructions. These allow a QA to test projects they could never touch before! By blending the skills of a developer and a QA, we can reap the best of all our talents. All it takes is an investment to allow our friend in QA to come with us!

Advertisements
Standard
Perspective, Work Projects

To build a better performance review

It’s that time of year again, Performance review season! In reality, most managers did this last month or earlier. The employees usually find out a bit later. This passive relationship employees had with their performance reviews always bugged me. So this year I decided to do something about it.

I hadn’t made up my mind until the day of. I realized that I would miss out on valuable insight, since I had only started about six months earlier. For an hour before the meeting, I scoured the internet for resource, and I compiled a list of questions. I don’t have the links anymore, but I am certain with few Google searches you can find a good number of lists. A good place to start might be “performance review questions”.

The core three I found in some sources address the basics of a performance review: What went well? What should I not do anymore? What should I start doing? The strength of these three is that they are easy to remember, and the cover a lot of ground. So if you’re short of time, just remember to ask: What actions to Continue/ Stop/ Start? Nice and simple.

But, these questions also fall short, since they aren’t specific. Moreover they don’t always produce actionable answers. For a review ‘actionable’ is the most important attribute. You have to be able to act on it. So I sifted through a few more lists and found several very useful additions.

What actions can I take to deliver more value? – This questions can deliver in spades. It allows the manager to point to specific things that you can do. As a result you can point to them later, and ask for feedback. In my case, it provided the foundation for future questions, and started a useful conversation.

What are your goals for me next year? /What do you want my priorities to be next year? These questions are good to ask earlier on, as it frames the rest of the discussion on actions. Once you have agreed on the goals or priorities, it becomes easier to see actions that can be taken to meet them.

What new knowledge/skill do you think I may need to develop this year? To me this question feels like a natural extension of the ‘Start’ subset. Especially in software, new skills or knowledge are nearly always needed. In my case, since it followed the priorities question, it didn’t provide a great deal more information. Some of the goals already implied the answer to this question. For example, if the goal was ‘drafting a schema for Mongo to handle X kind of data’, I will need MongoDB basics. As well as any information I can find on MongoDB or non-relational schemas.

How do you think the business is going to change in the future?/What challenges do we face? This question provides perspective. And it can fill in missing answers for the previous question. Further, it may encourage your manager to think of you filling the role of meeting that challenge, which is always good! In my case, I got some perspective on the larger motions within the company that I wasn’t aware of.

What can I do to improve my rating in ___ next year? For a rating where your performance isn’t where you want it this is a good question. It encourages you and your manager to point to concrete items which you can improve as well as the expected improvements. In my case, there wasn’t enough information to offer a correction. This was due to the duration of my employment rather than to lack of interaction between myself and my manager.

What can I do to make you more successful? This is the single biggest impact question that I found! It allowed me to mention some areas where I thought I could help my manager, since he’s a busy fellow. It got him thinking. Almost immediately he thought of a task that he could hand-off. Not only would it help him, it would be beneficial for me to experience as well. Through this question I was able to set myself up as a true ally of my manager. Now all that is left is for me is to do my best on the new task, in short, to be faithful in the small things.

What career opportunity do you see for someone with my background? This question is more sensitive. Specifically, your manager may need to be careful with phrasing. They will want to avoid coming across as a promise of a position in the future. My manager and I were able to work around this, by changing the context of the question. Instead I asked for some of his insight. What might be useful places to expand my skills? What areas of software that I might enjoy based on our work experience together, and personality? From this new question, I received several valuable insights, and suggestions. For those who have been with their employers longer this may be a safer question. For those who are fresh, I would definitely suggest the alternative version.

Along these same lines, I avoided this next question: What could I do to give myself the best opportunity to move onto the next level in the organization? I am certain similar concerns would grip a manager with this question. I also feel it is more appropriate to ask about particular positions. It is definitely a question for someone intending to stay with an organization for the long-haul.

And last but not least, my favorite question: What is the most difficult thing about doing performance reviews? This question broke the tension in the meeting pretty fast. It also allowed my manager to voice some of his thoughts on the review process. It helped me to understand the rating better, and to be better prepared for next year’s review.

After reviewing my notes, I plan to keep questions 1, 2, 3, 5, 6, 9 (especially), and 12. I am reviewing the value of questions 4, 7, 8, and 10; and I plan to drop the 11th question. For convenience, I’ve included the numeric, ordered, list below.

  1.  What actions should I continue?
  2. What actions should I stop?
  3. What actions should I start?
  4. What actions can I take to deliver more value?
  5. What are your goals for me next year? /What do you want my priorities to be next year?
  6. What new knowledge/skill do you think I may need to develop this year?
  7. How do you think the business is going to change in the future?/What challenges do we face?
  8. What can I do to improve my rating in ___ next year?
  9. What can I do to make you more successful?
  10. What career opportunity do you see for someone with my background?

    • Alternatively: What might be useful places to expand my skills? What areas of software that I might enjoy based on our work experience together, and personality?
  11. What could I do to give myself the best opportunity to move onto the next level in the organization?
  12. What is the most difficult thing about doing performance reviews?

 

Standard
Perspective, Software Development, Work Projects

Where’d my UX go?

Disclaimer: I am not the happy looking chap in the photo.

I was working on a personal project recently when a realization dawned on me. User Experience Design,also known as UX design, and software design collide more frequently. And not only in the User Interface layer.

Before I get too far, when I talk about UX, I am referring to the experience the user has while attempting to use the device or object, or code. I think this image does an excellent job of describing good UX concisely.

Link: http://i.imgur.com/9LqhOl3.jpg

It’s pretty easy to tell what UX is like with a Graphic User interface, or a GUI. After all, this is the part everyone touches. If a website is snappy and the layout makes sense, that is good UX. If it is clear how to do the operation you want, without needing to consult the magic talking paperclip, then it is a good UX. But it seems that once you go below the GUI layer, the lessons on good UX vanish.

I was working on a Fluent Testing API for python when I realized it. In version 1, I had all the functionality for this API bound up in a single class. Sure, it limited the import tree, and made it easy for me to develop. For version 2, I decided to pull the functions into separate classes. And while I was writing out some example cases, I realized that this simple code change resulted in an augmented User Experience!

You see, by pulling the various functions into different classes, I allowed the IDE to create better prompts. The better prompts now guide a user of my API through the proper pattern of using my API. Since there were fewer functions to choose from, it is now clearer how to proceed. The user no longer has to consult a lot of documentation. This is a simple example, but it did get me thinking.


In fact, one week prior, I added a Facade to one of my library at work. The Facade simplified interactions with my  Library. Now other software engineers could more readily use my library’s functionality. I am surprised that I didn’t think of it at the time, but APIs are a Software Engineer’s UI layer. As a result, they should be subject to a UX review!

I mentioned earlier that I have noticed that, on the whole, UX degrades as you leave the GUI layer. Two factors are responsible, in my opinion. First, the majority of UX review and work goes into the GUI layer. And this focus makes sense. The vast majority of software interaction is through such a layer. As an aside, finding a UX guy who can talk about UX and about API design can be difficult. I usually have a heck of a time getting time with them to review a GUI design with them!

The Second factor is a lack of discipline. I am not throwing stones here, the first version of my Testing API is example of such a lack! I collected all the functionality in a single class because it was easier for me!  I wanted to get the functionality together and to reduce the import tree. In hindsight this is a silly reason. And yet, it was enough to change my behavior.

So now that I’ve seen the problem, what can I do? Well, I noticed the improvements made in the UX for version 2 by writing up some examples. That is to say, I used it. This is a good start, bu submitting it to user testing would be a better step. After all, as the design I was intimately familiar with the inner workings and the proper usage of the tool. But a fresh user wouldn’t be. And if there is anything I have learned developing software: the user never does exactly what you expect them to.

Besides more user testing, some cross-functional education might help. This recent epiphany put me in mind of a tech talk that I hadn’t finished. You can find the youtube video here. I am hoping that revisiting the principles from the talk will continue to improve my designs!

Standard
Perspective, Work Projects

So tell me, Why do we pay you?

 

 

So imagine this: you are sitting at your desk at work. When a mid-level manager strolls up and asks you the following: “Why are we paying you to do what you do?” Could you answer him? Would he understand the value of keeping you if you did?

Recently, I had a chance at work to answer this exact question. Thankfully, the situation was much more relaxed. You see a project that I am the technical lead of was awarded a Project Manager. To me this means that the business thinks that what I am doing is valuable. Valuable enough to pay someone to make sure that it is well staffed, organized and properly directed. I am pleased with the direction we are going.

But as one might expect, the Project Manager was not a software engineer. So naturally explaining the value of the project in software terms wouldn’t help. The make matters more complicated, the project is not customer facing. It is in fact a business solution, helping to tie many other services together.

Since this project had been without a Project Manager for a while, we had never completely translated the value of the project into business terms. As a result our first meeting with the PM experienced a little bit of a disconnect. We spoke about the value the software provided to other software applications. But our PM had some trouble translating that into meaningful terms for herself.

So following the meeting, I took it upon myself to attempt a business translation. I took what I knew about the project and its value and translated that roughly into business terms. I certainly did not do a perfect job. But I believe that some benefit may come from discussing what I have learned over the years and was able to apply to this project.

As I see it, Business Value comes down to just one thing: More Money. But there are two ways to achieve this end. Either you make them money, or your save them money. If you can translate your work into one of these two parts you can usually make a business case for it.

Our project was created to simplify the process of connecting many apps. The project would make it easier to maintain, and easier to extend the connections between application. It would also support new feature implementation. Moreover, since the new system is simpler, it reduces the likelihood of incorrect actions by our system. So how does one translate this into saving or earning more money?

If the project make it easier to maintain a system, then it reduces the man-hours spent on maintenance, right? If you save man-hours, then you are saving money right? If you want to go the extra mile, provide an estimate of how many man-hours it saves. Then multiply those hours by the average software engineer hourly rate. This provides a number of dollars that your project can save the business!

Since software projects often provide a benefit for many years; try to provide information about saving during a single year. So if I save 8 man-hours a month, I save the company 96 man-hours or 3600$ per year (assuming a salary of 75K$).

To take the example a step further, I can also factor in the cost of my work on the project. Let us say that it will take me 42 man-hours ( or roughly 1 week of work) to complete this feature. Then it will cost the company 1575$ to produce. The result is a net savings of 2025$!

Going back to my project, since it makes it easier to extend, the cost to implement it will be rather low. If you can quantify how much time it will take to implement your solution, you can provide a more accurate estimate to the business.

The project is designed to make it easier to add new features to our system. This means that it reduces development cost. Additionally, it means that the new feature can make it to the market faster. Do not under-estimate this! Faster Time-to-Market can provide your company a competitive edge.  The edge comes from either by being the first to provide a particular functionality, or by begin able to under-bid competitors. While it can be difficult to provide exact value numbers on this, you can highlight the man-hours saved in development. I find it easiest to understand by providing a comparison to the current process.

To provide a good estimate, I recommend using a current project. Comment on the time it took to implement the part of the feature related to the new project. Then compare it to the time it might have taken using the new project. This provides a tangible example of the value your project creates.

Finally, my project simplifies the logic of linking multiple services. This simplification reduces the risk of errors. To put a value on this, you need the time saved. Start with the current time spend debugging. Then estimate the how much your project would save, based the project’s implementation.  My project simplifies by pushing the logic into a more appropriate context. It also allows the using services to dictate the communications they wish to react to. As a result nearly all the current bugs could be eliminated. Not 100% of course, but a sizable chunk, perhaps 70-85%. Using this estimate, you can translate the man-hours saved into dollar value as before.

Translating from software value into Business value isn’t always easy. But it is doable. Further, it is quite approachable if you have the right mindset. How am I saving the company money, or how am I earning them more money? Once you have these answers, you can begin the translation into business terms.

To be sure, the estimates you provide when you first start will be a bit optimistic. But with time, practice and experience, they can become more realistic. I am certain that the estimates I provided to my PM were lacking in some respects. But something is better than nothing! And I got to learn a valuable lesson in translation!

If you have a project to translate, I’d be happy to discuss! If you have any pointers, I would appreciate your suggestions! Just send me a message! Good luck and Happy hunting!

Standard
Automation, Software Development, Work Projects

How to increase Team Velocity by 50% III

sport-1014015_960_720Last time, I discussed the development process and some of the end results of a automated test-generation system. I have mentioned from the beginning that it enabled my team to increase our velocity by 50%. Today, I will discuss how long it took for us to realized that increase. I will also talk about some further improvements that allowed us to reach that level.

As mentioned in the last post, we were able to achieve a 50% increase in our delivered story points per iteration. To be sure, this increase did not happen overnight. It took roughly 3 iterations before we learned how to use the system most effectively. It took an two more iteration before we reached our new plateau.

As we used the system we began to realize several weaknesses in it. The clearest of these was the systems rapid rate of decay. If we got even a little lazy, the system magnified that laziness. And we would then have to spend much more time just to fix it.  Sort of like cleaning one’s room. Some mess attracts more mess. But if you’d just put the laundry away you wouldn’t spend a couple hours extra on the weekends just to clear it away.

In a similar fashion we had to adopt better habits to keep our system pristine and operating. As a team we had to adopt better habits, one of which I mentioned before. We adopted the practice of having our requirements discussions with the Database open. We then kept it up to date with the conversation.

Now in theory, this fixation with cleanliness would only need to be maintained during active development of the data model. Once the data model development was completed, the Test generation system would not longer be as necessary. Presuming the system runs for the last time on a completely specified data model, and that all models correctly meet their criteria, the auto-generation system would be effectively retired.  While its final output would be kept for posterity. However, I was transferred to another team before such an even occurred, and so cannot speak from experience.

But before I left the team, I actually returned to school for my last semester. I then returned to work with the team again, this time as a full-hire. When I returned, the team had expanded on the auto-generated tests. They had added new types of tests and were beginning to have trouble maintaining my original t4 architecture. This was the first improvement that I made to the system when I returned.

My original design had become cluttered and bloated. This was due to intense aggregation of the test implementation and the generation-decision logic. So as any good programmer would, I created layers of abstraction. I created a hierarchy of t4 files. Since you can refer to functions created in other t4, I organized the test implementation logic in one file and the test generation logic in another for each test category. Some categories were particularly large and so I split their logic out into yet more.

At most, I believe the nesting was 3 deep. But by adding this abstraction, all further extension of the generation system were greatly eased. Additionally, while abstracting the tests, I discovered several generation errors, and corrected them. Further improving the test coverage of the system.

While the reorganization was taking place, one of my colleagues was making another improvement. At the time, the generation system produced something like 5000 tests. However, they used a network database. As a result, running all the tests would take 2 hours or so. My colleague created a script that was run before the test-suite executed. It would create a seed copy of the database on the SQL server running on the local machine. As a result, the execution time went from 2 hours to around 16 minutes! Again, this was not am improvement I made. But it did greatly increase our efficiency and so I feel it is imperative to mention it here.

After finishing the re-organization of the system, and improving our execution speed, I happened on an interesting idea. I realized that we could apply the same concept to test another aspect of our code. At the time, I was tasked with writing some tests that would confirm that our triggers were working as expected after a schema upgrade. I realized we could use a similar system to test the proper creation of Table, Keys, Triggers, and restrictions of the database itself.

Most of us agreed that testing this through the entity was cumbersome and unnecessary. The trigger executed after the entity was saved, and thus testing it would require a second read cycle, which is slow while using the entity. So instead we decided to use SQL queries directly.

It was at this stage that the idea struck me. I offered it to our team lead, and she again supported the idea. And this time, having learned several lessons from the last time, I was able to whip out a working system for the desired test in an iteration. Over the following iterations, I expanded the trigger test to several other tables. I added both structural and key verification tests, which eased many of our worries regarding the schema upgrade process. At this point,I was transferred to another team. I saw the successful extension of the system, so I am unable to comment on the value it added to the team in the long run.

I will leave off with just three points. First, If you are willing to put in a little extra effort, you can buy time for your team to pay down technical debt. This is done by investing in strong meaningful tests. These tests, if properly written, will pay dividends whenever the system is changed. And the system is always changing.

Secondly, All test systems require maintenance. A test is only as valuable as the code that it verifies, and if that code changes, the test may also need to change. When the business function a test covers is no longer valid, the test should be removed. It is like weeding a garden (if the peonies in the garden could turn into dandelions spontaneously). 

Finally, A quick excursions into a new way of doing things, can pay off in many ways. It can invigorate the team, especially if the system is time or labor saving! Everyone likes to work less! The new way can stimulate new ideas, as it did with the trigger tests. And of course if the time-savings do pan out, your team can achieve even more in the same period of time! I hope my discussion has provided some food for thought, and that perhaps you too will consider a little automation of your own! Feel free to PM me if you are curious about any system details that I did not mention.

Standard
Automation, Software Development, Work Projects

How to increase Team Velocity by 50% II

If you missed the first post in the series you can find it here!

Last time, I opened with the hook of increasing a team’s velocity by 50%. I introduced an automation project that would generate integration tests for us. Before that system, the testers, myself included, had trouble keeping pace with the rest of the team. Worse still, we found out later that some of the entities we release had bugs in them! But I had an idea. I assembled a rough outline and a demonstration for the Team Lead. Then after some discussion she gave it a green light. She also gave me one month to set up the necessary scaffolding, while she got the team ahead of schedule.

The core of this automation system was T4 templates. For those unfamiliar, they are a file generation framework created by Microsoft. By writing .NET code in the .tt file, one can control the contents of the generated text files. This includes generating C# code, and other file types.

We used these templates to generate partial test classes containing the predefined test cases. Not every entity would get the same kind of tests. For example, some entities would have doubles that could not be negative. Others might have a string that had to be populated. There were even different edge cases supported within the same data type. A database containing various flags would dictate what tests to generate.

To review, the database housed two kinds of tables: the Main table, and an entity specific table. The Main table controlled whether tests were generated, and linked to the entity tables. The entity tables housed information on the properties to test. AS well as  the boundary conditions and other requirements for testing.

One challenge I discovered while scaffolding was ensuring that Parent-Child relationships were honored. I couldn’t just assign a random ID to the parentID field. The program database would kick that out with a constraint. I discussed and brainstormed on this problem with the Senior tester. We finally decided to create a helper class that could act as a factory for the tested entities.

The factory would assign all the required fields of an entity with appropriate values. For the most part, these were randomly generated numbers, or strings. The Helper’s factory functions were called to create the entity-under-test’s Parents. Following this logic, the helper would create the entire entity tree.  This would work at any level of child, leaving our database in an appropriate state.

In database testing there are four basic level tests: Create, Read, Update, and Delete. To support these cases, one must control when to save an entity to the database. To help in these cases, we added alternative factory functions to the helper, selected by parameter flags.

Up to this point we wrote the helper functions manually. This became difficult to maintain, and so we automated it as well, again using T4 templates. But unlike the test generators, we could not honor the generate flags in the database. There were cases where an entity was not ready to test, but a child, or a parent was wishing to test with it. Instead, we opted to generate factory functions for every listed entity.

By the time I had finished this level of scaffolding it was time to bring the team on board the project. We delegated by a series of test cases. ‘Not equal null’ tests to this developer, ‘Less than the specified max length’ to another and so on.

The size of the system, in comparison to its scaffold, exploded during this time. I spend much less time coding the system, and much more helping and directing the other developers. I sought guidance from the Team Lead often.  I did so to ensure that I was not ruffling feather or otherwise harming my effectiveness as a leader.

With grace and patience, she guided me on better practices.  She offered ideas for how to help the developer understand. Many of her ideas made it into loose documentation that I sent to the developers for reference. But the developers weren’t the only ones who had to understand the system. I also had to find a way to communicate the value and usage to the Product Manager.

I was blessed with an understanding PM. She allowed me to walk her through the basics of the system, and what it meant. In the end, she decided that it was the perfect place for her to define the AC for any new or modified entities. Once she understood the structure of the tables, she happily filled in the requirements. Moreover, she was able to provide them in greater detail than we’d been able to achieve before.

Instead of having some loose requirements, we had detailed expectations. Or in other cases, we had a description of the desired end-state of a modified entity. This greatly reduced confusion altogether. And resulted in  far fewer follow-up meetings with the PM. This alone would have increased our team speed. This test Database provided our team, PM included, a common medium to communicate in. And on top of that, it provided enough details for all parties to understand!

Back to the practical use of the database, I crafted template SQL queries. These allowed the PM to add new entities, or change existing ones. And with her existing skill she easily found the information she wanted. These tools, including the database, allowed the team to accommodate the availability of the PM’s time. Some weeks she would be out with customers, while on others she was free most of the day for discussion. With the test Database, she could tell us what she wanted without having to be present for everyone of our meetings!

After a month of expeditious work by our team, we had the core of our automation system ready to use! The developers returned to new development. The testers moved to round out the automation, and to maintain it. Our first process change was adding another step to our storyboard. The developers would now generate the core integration tests for an entities and run them. If those tests didn’t pass, then they would fix their entity, before it every went into QA.

This extra step saved the testers a great deal of time, since the Developers would see the common bugs. And this reduced the back-and-forth between Development and QA immensly! With the extra time, the testers could focus on maintaining the system. We could also pursue exploratory testing!

One drawback in this system was that every time a developer wanted to run their entity tests, it had to change a flag in the database. This flag change affected everyone! Which lead to some confusion in the first week. My first iteration of improvement added an override list on each user’s box. This allowed a developer to test without modifying the database.

On the topic of maintenance, our automation system was great at handling standard cases. But it was somewhat ornery about special cases, and especially so for one-offs. We had to add a couple of tables to identify specific special relationships. This way we could test them specifically without interrupting the existing structure.

Further, we had to carefully manage access to this database to protect it against accidental corruption. Which meant we had to allow the developers to read, but not write to the database. Both of these requirements were non ideal. But in hindsight, we should have expected them, considering the tools used to create the system.

But the required maintenance did encourage the team to adopt a better development process. Instead of immediately going to work on new entities, we would start with a through review of the specifications. We adopted the habit of always having the test Database open during these meetings. And  we kept it up-to-date with the discussion. When we finished with the meeting, the database accurately reflected our expectations. The developer could immediately and confidently begin their work.

The automation system was beneficial for all. Though it did not completely free the testers from test maintenance. It did free up our time for exploratory testing. With the benefits to the developers in rapid feedback, they saved time. For the PM, it provided a fertile communication medium. And all together the team was able to achieve a 50% increase in our velocity for a given iteration. It was a good way to end an internship.

The next post, the last in this series,  I’ll cover what happened by the time I returned as a full-time developer. This will include exact quantification of the team’s new stable velocity. I will cover the improvements we made on the system, and even a scion system based on the same idea!

Standard
Automation, Software Development, Work Projects

How to increase Team Velocity by 50% I

 

How would you like to increase your team’s velocity by 50%? Well, I was able to accomplish such a feat as the capstone to my last internship. It took a lot of help from the team I was on, and a lot of support from the Team Lead. But we implemented a system that increased our velocity by 50% in roughly a month.

Our tale begins with the database access team, working with Entity Framework. I had just arrived. I was learning C#, Integration Test practices, and other new technologies as fast as I could.  I was shadowing the team’s current tester to learn the ropes, and contributing where I could. Even with two of testers, the developers often need to wait for the us to catch up, up to an iteration after them. Moreover, released entities are later discovered to have inadequate test coverage, or undesirable behaviors.

The idea hit me one day when I was literally copying and pasting tests, and then replacing class names. I admit this was a poor habit, but I was endeavoring to keep up with the team. What I realized was a great number of our tests were of the same type. This kind of property gets that battery of tests, that kind gets a different set. Thanks to recent lessons in T4 templates, I realized we needed to generate the correct battery of tests for the entities in a programmatic way.

I tried to compile a set of known test scenarios. Then I enumerated the entities we’d produced or modified in the last PSI that matched the set I wrote. These data sets provided a good coverage. But I needed a place to store that information for all the entities, and the templates had to be able to read it. I experimented a bit and found that, with some C# code, the templates were able to read the a database.

The database tables would represent a single entity. The rows would represent the properties. They could even store boundary conditions! Then a master table would control what entities would have tests generated. The master table could also present the relationships between entities.

But this was too big a task for me. I needed help, so I brought my idea and some of my findings to my team lead. She called a small meeting between myself, the other tester, and the architect. I explained my idea, and showed them some of what I’d done.

After a lot of discussion, the team lead decided that it was a worthwhile project. She then set about getting the team ahead of schedule. This was we could put a month into the new project, and not be behind. Meanwhile she had me setup the core of the system and a scaffold to make sure we could delegate work. Additionally we spent a lot of time polishing the format of the generated tests. By the end of that month we were ready to start making our team faster!

With a  moment of inspiration, and the support of the team lead, my plan to put in place a system to generate our tests was ready to begin. In my next post I will discuss more of the implementation details. The various discoveries that occurred while the team worked on the project will also come up. In the end, we easily achieved a 50% increase in our team velocity!

Standard