Product Manager versus Quality Engineering Lead

Our concept of agile has a Product Manager (PM) representing the customer. The PM is also active in defining the features needed per release train and in generally working close with the team to define stories and the iteration-by-iteration flow. This thinking embraces Steven Wheelwright’s (http://www.hbs.edu/faculty/swheelwright/index.html) model of the heavyweight product leader and seems necessary ensure we have a train payload that makes market sense. It does present a challenge when combined with the separate notion of a closely-coupled but independent software quality engineering (SQE) team. Each development team (4 to 10 people) generally has two or more engineers charged with the responsibility of ensuring all the features of the train, new and previously developed, have, and continue to have, high quality. This is usually done several different forms of testing with the SQE engineers laying down strategy and developing the tests.

A lead for each train represents the quality side for all product decisions and in specifying work for other SQE engineers attached to the team. These leads hold a fairly powerful position since they can hold up a release if not comfortable with the state of the quality as reflected by test results, defect load, and general readiness of several factors/artifacts. Operating within a strict development timeboxes and having a multi-year history of generally not missing a release (and doing up to 11 a year), the decision to hold up a train release, of course, would be a big decision. Using a sports metaphor, the train goes into a Quality Overtime and means the release takes a day-by-day slip until the lead is ready to sign off. During this time, all members of the team, including all development engineers, report to the lead to ensure the Overtime has the necessary resources. This is consistent with our thinking that all dev engineers report to quality during hardening that may occur before release.

While the heavyweight empowerment for the PM is necessary it can also be a point of conflict if the PM drifts a bit and starts to specify what the SQE engineers’ storycards and tasks. This generally arises from a PM being pushy in wanting to ensure that defects are found as soon as features are completed. We generally meet this situation by reminding the PM that the SQE team has a history of keeping up. So it becomes a bit of reset and enforcement in roles.

There is at least one other, and more complex, reason for this arising. The Lead SQE is generally focused on functional testing necessary to show/ensure feature quality. Other testing, though, may be driven by market needs apparent only to the PM. These may include security testing, stress testing, unique configuration testing apart from standard, and forms of conformance testing. While these are all doable, a PM popping these needs on a SQE team already driven to meet functional quality checks can result in a bit of a breakdown.

Some have observed that this situation is tough on the SQE lead and the gravity is for the SQE to sign-up for more than can be accomplished. A better situation, sometimes requiring management attention from the SQE side to ensure “table balancing” is for the PM to state the market needs and then to come to an agreement of testing and when it might occur: Day 1 before release, Day 2 immediately following release, or Day 3 on customer request. Done early in a train, this type of agreement, sometimes called negotiated quality, is the best situation since the SQE Lead can work to include the additional testing beyond the normal functional test work.

We recently had to work our way through a negotiated quality situation. My default assumption has always been that the SQE Lead is empowered to handle the negotiation from the quality side. Management in Moscow pointed out that that is not always the case as many engineers are good at engineering but not equipped to be good in arriving at a win-win solution. This gets even more so, when the PM is at company headquarters and the SQE Lead is offshore. It probably means that the SQE Site Lead, responsible for ensuring quality activities at the offshore site work well, must now participate in the heavy lifting of getting agreement to the needs. Done right, the SQE Lead for the respective project will participate and become better at negotiation.

An improvement beyond this is to provide training for everyone on negotiation with the emphasis of getting to win-win. I have done this at a previous company and it was helpful in empowering the Leads, giving them approaches and in reducing their being “whipped around” to the PM’s view, but it was not an overnight win. It was, thought, a starting point that did lead to success with lots of re-enforcing the roles by management. It’s important to keep in mind that software development, and especially with agile, is sometimes more about people than technology.

Standish Group research

I found an interesting Standish Group research which was done in 1995. It is about a set of causes why software projects succeed. They ranked all reasons based on score collected from many different vendors.

• Users involvement (score: 19)


• Top management support ( score: 16)


• Requirements accuracy (score: 13)
  

• Right planing (score: 11)


• Realistic expectations (score: 10)


• Iterative development (score: 10)


• Competent stuff (score: 8)


• Stuff involvement (score: 6)


• Clear goal definition (score:3)


• Purposeful and hardworking personal (score:3)

In spite of the age of this report I would agree with this conclusion.
This is good that Agile practices address several causes from this list. For instance: Requirements accuracy, Right planing and Iterative development. But the heaviest reasons from the list: Users involvement and Top management support should be addressed on another level of management.
I thing that nowadays people are more concentrated on bottom causes than firsts ones. One of the PingIdentity advantages is an ultimate close relationships between top management and engineering team including offshore. Probably it is a key factor of a company productivity. Speaking about User involvement seems to be an intensive growing area.

Elaboration and Stuff of the Agile Pipeline

Our experience with agile is that it yields a considerable speed advantage to the market over the traditional waterfall and RUP framework approaches. While just an intuition, the 30% to 40% boosts reported in the press could be very close to the gain we are seeing in our Denver office. Stretching agile to offshore, we also see a boost over a traditional approach with the estimates being 20% or so. I should note that our first offshore relationship, roughly a year, with a large provider in India, we did not see any gain and instead discovered high attrition and agile were a deadly combination leaving us with low velocity/feature output. The situation with Moscow is quite different with agile being very compatible to the team.

Starting with a definition: The agile team “core” is those engineers that take a storycard, task it out, do design, implementation, unit testing, integration, and demonstration over two-week (our choice) iteration. In our model, the core depends on the product manager (PM) to identity and elaborate product features and on the software quality engineering (SQE) members to do feature acceptance/testing. This all sounds good but once a world class core is ramped up the pacing problems across the dev pipeline, feature to accepted code, become focus points. The PM can fail to “feed” the core with enough feature detail to keep them going and the SQE guys can fail to keep up with the feature outlay. Today, I’m focusing on the front-end, the elaboration process.

A simple model for the front-end looks like this:

So this looks fairly straightforward but raises the question of who does the elaboration. Start with the view that the PM is charged with elaboration. This is really only one possibly with the elaboration taking several different paths and the picture looking like:

Let me emphasize that “elaboration” means “written description of requirements” This shows 4 possibilities with the PM taking the elaboration load, a sharing in some way, the dev engineer doing it all, or a null path. The top bubble often becomes the second bubble with the engineer asking questions but, of course, the answer may not be written down. The null approach occurs in at least two ways: 1. the feature is so simple that even my youngest cat could implement it if he knew Java versus Modula or 2. The engineer is well grounded in the product, having maybe built much of it from the ground up, understands the usage market well, and can go directly to design from a brief feature description. Before the SQE manager beats me up, it is important to note that if the null path is taken, it might be necessary for the implementation to be explained to the quality team or they might be left wondering what the feature means when implemented. The same words apply to the tech pubs team that must write about each feature for the user documentation.

Our experience is that a development train payload, composed of many features, may spread features out across all four of the paths. The null path is the fastest path to functionality but honest scoring must account for the time taken for verbal walkthrough/explanation to the SQE team on finishing up the implementation. It seems that as a team grows with product creation and grows in its understanding of the market needs, meaning many releases without much team churn, the null path takes a larger percentage than in the early days. Velocity, feature production for core members, does go up with this being one of several contributing factors. Definitely the manifesto words “Written software over comprehensive documentation” come to mind.

Now, with this simple model, we can look at what happens when we go global with the team split across between Denver and Moscow. The PM is in Denver close to the market. The dev team is offshore. The challenge: Don’t bottleneck with elaboration.

The cautious approach is for the PM to elaborate everything. They are close to the domestic market and definitely should understand it. A high aggregate velocity, total feature product across all core members, means more than one PM might be needed not to slow things down or the PM might have to be several weeks in front of the current iteration cranking out requirements grist for the core to chew on. That sounds very much like waterfall-like agile. (The oxymoron light just went on.) One alternative is to have another PM in Moscow with who shares the elaboration and question answering from the team. We’ve done that and it works. We have also done a variation where the solution is a daily verbal exchange to elaborate features. It means lots of hours on Skype and it also is somewhat like the null approach in that somewhere the SQE team must catch up/understand the functionality behind a feature. There is also the approach where the developer takes the SQE team through the functionality, they take notes, develop an attack plan and hit the features with tests. This might also mean the SQE team has someone whose job is to interview the developer and render the functionality to written requirements. Some places see this as the role of a requirements engineer.

Just thinking about what works and could work better, the front end comes to mind and I have not even touched on the all the craziness that happens in prioritizing the requirements and keeping that priority dynamic as the train flies along through the many iterations.