PPQC Blog

Is a Prediction Model Required for Maturity Level 4

2011-10-09T14:31:00.000-07:00

Is it necessary to have a prediction model for L4 and L5?

I collect data from different projects and put them in a 3 sigma band and take care of outliers. Then I compare the mean and standard deviation of current PBB with previous PBB. And I also check against the LSL and USL set by the management team the trend coming out of this PBB. In case the mean, SD, LCL and UCL decrease from the previous PBB, I update the management team and we check the projects on the basis of this.

What happens when I talk to someone in my circle of friends they talk about regression, simulation, Monte Carlo, etc. In my organization we need to show process performance against set targets, so why make life so complex with so many above mentioned methods?

From what I can glean from your question is that you have developed a Process Performance Baseline (PPB). But I have no idea why you have done this and what value you are getting from knowing this PPB. What use is it to your organization? How does it help you meet your Quality and Process Performance Objectives (QPPOs) and your business goals?

If you are not interested in fully implementing ML 4 or ML 5, then I suppose you don’t need anything else as long as you are deriving some sort of benefit from this PPB. However, for a complete High Maturity implementation, you are expected to do the proper statistical analysis of data distributions, probability statistics, process engineering, etc. to derive appropriate PPBs and PPMs. And to be a PPM instead of merely a forecast model and be of use for "what-if" analyses, the PPMs must contain controllable factors that have an impact on the outcome.

Also, if you want to implement ML 4 and/or ML 5, then you need to have established organizational QPPOs, a number of PPBs that support the QPPOs and can be used to evaluate the feasibility of achieving them, and a set of Process Performance Models (PPMs) that are derived from your historical data that are used in conjunction with the PPBs to predict each project’s ability to meet its QPPOs, as well as a number of other activities.

Therefore, if all that you have is one PPB and nothing else, you only have a partial implementation of OPP and still have a lot of work ahead of you before you can consider that you have implemented ML 4, let alone ML 5.

SCAMPI Document Review

2011-10-09T04:16:00.000-07:00

I have two questions or requests for clarification:

During the document review in SCAMPI B, wherever we do not have a doc, ppt, or xls as an artifact, we provide screenshots from tools where the planning and tracking are done. However, our Lead Appraiser (LA) is asking for access to tools for all the ATMs. But it is not possible to give access to certain tools as they are client specific and only the team working on the project gets access (that too after signing an NDA). How do we handle this situation? Even if the tool is internal, access is very much restricted based on role in the project. Can the LA really require access to tools for all ATMs, which is not allowed as per the policy of an organization? Is there a guideline on what process to follow in case access to certain tools or application is restricted?

Is there a material, which shows a linkage of all the PAs (representing interaction of PAs) and gives a holistic view of CMMI PAs when applied to an organization at Maturity L5. This is more from a training perspective.

Have you explained these restrictions to your Lead Appraiser? The Lead Appraiser should be flexible regarding access to restricted tools. As a Lead Appraiser, I would find the screen shots as acceptable evidence in your situation. And if I wanted or needed to view additional evidence, I would request that you provide a demonstration of the tool by an authorized user and have this person be directed by the ATMs or Lead Appraiser to view specific information. I have used screen shots as evidence on numerous appraisals in the past. The SCAMPI method allows for screen shots and tool demos for just such reasons as yours. If your Lead Appraiser is unwilling to abide by your restrictions on tool access, then I would strongly urge you to find a new Lead Appraiser and possibly report him or her to the SEI.

Look in Chapter 4 of the CMMI-DEV book and you will find a series of diagrams that show the linkages between the PAs at a very high level.

Why Would I Need a CMMI Expert?

2011-10-09T04:05:00.000-07:00

If my organization is already doing something and my results are in order, what is the value of me picking up something that I am doing and saying that this is what fulfils the requirement of configuration audits or for that matter any other practice. Why do I need a specific (CMMI) legal expert to do the hair splitting and argue that what I do indeed meets CMMI requirements and three other who say it does not? I would be keen to understand how my organization, or for that matter any other organization in the community, would be better off.

If you are able to identify that what you are doing satisfies the practice, then you are correct in that you don’t need a CMMI expert to tell you that your practice meets the CMMI. However, in my experience, since people look at the CMMI as a requirements specification, they have difficulty determining that some of their practices are in fact CMMI compliant and therefore take steps to implement additional, and perhaps non-value added, redundant practices to “pass” the appraisal. Therefore it is important to have a CMMI consultant and/or Lead Appraiser perform a gap analysis to determine if the organization has made the proper interpretations of the CMMI in their implementation.

On the surface the CMMI is a simple model, but the more you study it, you find additional layers of complexity that can lead to misunderstandings or extra non-value added practices.

I would maintain that if you were only using the CMMI for helping you identify areas for process improvement, were not interested in being appraised, and you had some internal process improvement specialists who are knowledgeable of the CMMI, ISO, etc., then you most likely would not need to use an external CMMI expert. However, if your goal is to be appraised to the CMMI, then it is vitally important to work with a CMMI consultant and/or Lead Appraiser.

One last point, you refer to a CMMI legal expert. That is a telling statement. In my experience, even when I encounter CMMI Lawyers in an organization, they lose sight of the purpose of the CMMI and process improvement. They are more interested in “what if” scenarios. Such as What if I do this or write this document, will that be CMMI compliant? The focus is more on explicitly covering all of the CMMI requirements rather than doing what is beneficial to the organization’s business practices. And if you find yourself or others in your organization splitting hairs over whether a practice meets or doesn’t meet the CMMI, you have probably lost sight of what you are trying to do from a process improvement perspective. You should be keeping things as simple as possible for your organization, and the hair splitting comes into play when your implementation may be too complex.

Hope this explanation helps.

Is This a Valid Performance Model?

2011-09-11T14:20:00.000-07:00

Is a reliability growth model considered to be a valid PPM in the CMMI?

Asking this question out of context with what you do in the organization does not have a lot of meaning. The correct answer is both yes and no. Please remember what High Maturity is all about. You begin with setting your business goals and objectives and use them to derive your Quality and Process Performance Objectives (QPPOs). These QPPOs in turn will lead you to the proper measures, process performance baselines (PPBs), and process performance models (PPMs) that your organization needs to quantitatively and statistically manage your work.So, if Reliability Growth is a critical process or sub-process and you have sufficient data to analyze to determine that you have a stable and capable process, then a reliability growth model might be considered a valid PPM.

But just selecting models without performing the analysis I just sketched out is incorrect and you will not be able to demonstrate that your organization is a High Maturity organization.

Thanks for the detail. I just happen to see in CMMI v1.3 High Maturity in which the "reliability model growth" which was given as example in OPP SP1.5 (CMMI v1.2) is deleted. Does it mean that reliability growth model will not be accepted in CMMI v1.3? Or the reliability growth model is not acceptable by the experts? Or is it only good if you use CMM v1.2 and not for CMMI v1.3?

As CMMI v1.3 is an improvement and the practices are carefully analyzed by the SEI and experts, is it advisible to use the reliability growth model given in CMMI v1.2? Or there is any chance that CMMI v1.3 will include the reliability growth model as an example?

Apparently there is some misunderstanding of my answer above. Whether the CMMI contains the reliability growth model as an example or not is irrelevant to whether or not it is a good model. Your organization has to mathematically analyze its data, business objectives, QPPOs, PPBs, and PPMs to determine if there is a need for using a reliability growth model. Do the following analysis:

Describe the reliability growth model in probabilistic terms.
Define the critical sub-processes (those that must be consistently and correctly followed every time) that can be managed using the reliability growth model.
Define how a project manager uses the reliability growth model in the context of his or her projects to predict performance, "what-if" analysis, and predict QPPO achievement.
Provide an equation or show by other means how the stable sub-processes that you have identified in your processes contribute to the reliability growth model.
List the other models that are used in conjunction with reliability growth model and why it has statistical relevance.

Once you have performed this analysis you will have enough information to answer this question yourself.

Audit Findings

2011-09-10T20:52:00.000-07:00

My personal experience shows that when audits are planned monthly or at milestones, it is very difficult to take any proactive quality measures. Let's say that SQA is conducting a review at the end of the design phase just before the milestone review, and during the audit they identify that a particular design option has been selected without applying DAR, then how can they close this type of reported non-compliance by having evidence that the project team is fixing the issue? What I have seen is that sometimes the project team considers the same non-compliance as an oversight like other types of mistakes and they close the non-compliance by labeling it as a lessons learned. Although as SQA I know that there might be a chance that this same issue can occur again in the future. But apart from presenting the findings to the milestone review meeting, we have nothing to do. And the SQA group does not have insight into most of the organization's processes where this type of event occurs so we can ensure every project is following the process per the plan. So please shed some light on this topic and suggest that what type of postmortem we can do as a reactive response and what type of proactive measure we can take?

It sounds like from your description that all that SQA does is flag a problem and then the project team declares what they are going to do and makes the final decision. In other words SQA has no control over the non-compliance after identifying the problem. This is an incorrect implementation of SQA. The SQA or PPQA people are the “eyes and ears” of senior management and if there is a disagreement between the Project Team and SQA about an audit finding, that must be escalated to Senior Management for resolution. The Project Team does not have the authority to declare that an audit finding has been correctly resolved. SQA has the responsibility and authority to decide if the non-compliance is being properly identified and worked. If SQA feels that the Project Team’s action to address the non-compliance is inadequate, then SQA should not accept the closure and insist that the Project Team take appropriate corrective actions. If SQA meets resistance, then SQA should escalate the issue to top management for resolution. Resolution may involve doing nothing, training or re-training the people following the process, modifying the process, or some combination.
Hope this explanation helps.

PI SP 3.1 Confirm Readiness of Product Components for Integration

2011-09-10T20:42:00.000-07:00

I need help with understanding this practice. Here is the situation: In our organization we have implemented MS Team System. This tool allows us to analyze the code from different perspectives. We have implemented peer reviews. The code reviews allow us to verify if the complies with the design specification. We have also implemented CM audits to check the identification of every configuration item. Consequently, I´m not certain we are fully aligned with this practice.

The purpose of PI SP 3.1 is to ensure that all of the components that you will be assembling are ready for assembly. For purely a software project, this practice is pretty easy and straightforward. At a minimum, you want to be certain that every module has been properly checked into your CM system, that every configuration unit has been unit tested, and that the external and internal interfaces have been examined to verify that they comply with the documented interface descriptions. It sounds like you might have most of these activities covered by MS Team System and your peer reviews. What I don’t see in your description is any activity associated with checking the interfaces against their descriptions. When you are integrating hardware and software, or have a large and complex software project with many different systems, this practice becomes more complicated.
Hope this short explanation helps.

I have one more question. Should we run unit tests for every configuration unit? Is it possible to implement actions other than unit testing to comply with this best practice? I think that the Static Code Analysis in VS Team System checks the interfaces betwen components. In the peer reviews of code we check the interfaces against their descriptions documented in design specifications.

You are actually focusing on the wrong topic. Instead you should be seeking answers to these types of questions.

What do your business goals and objectives tell you about the required quality level of products?
What is the reason for performing unit tests? Or what are you trying to achieve by unit testing the code?
Do your customer requirements and your business goals and objectives require a quality level that demands that you perform unit tests before creating a product build?
What are your requirements for each configuration item before creating a build?

Answers to these questions will provide the answers to your questions.

Basically, your configuration audits are there in order for you to determine if all of the configuration items are ready to be assembled. Perhaps the Static Code Analysis in VS Team System is satisfactory, perhaps it is not. That is for you to decide based on the quality requirements for your product.

Hope this explanation helps, but there is no clear answer to your question without being able to spend some time with you and your organization to perform an in-depth analysis of your processes and procedures.

Traceability in Pure Testing Projects

2011-09-10T20:35:00.000-07:00

I have a question about addressing requirements traceability for pure testing projects (Understanding requirements->Writing Manual Test cases->executing them). If the application is not developed by us, what information other than Module name, Requirement ID, description, and Manual Test Case ID needs to be mapped?

There is no definitive answer to your question. The actual answer is up to you and your organization to decide what is necessary for your traceability. What does each testing project need to know about traceability? If you can answer that question, then you have the answer to your question as well. What you have listed sounds reasonable, but only you can determine if it is complete or that you need to add other elements.

Review Activity for a Short Term Project

2011-02-06T04:34:00.000-08:00

Our organization will be going through CMMI Maturity Level 2 Appraisal in a couple of months. I have a PPQA question. As per the PPQA Process Area (PA), we require a review of the work products (content/template) and procedures required at Maturity Level2 during the project life cycle. We have one project that is 3 months long. There are many work products that will be produced during the project development life cycle.

Requirement documents such as SRS, Use cases, Bidirectionally traceability matrix document, change log, etc;
Plans for all the PAs, e.g. requirements management plan, project plan, configuration plan, etc;
Development artifacts, such as ERD, Code, UML diagrams, etc;
QC artifacts, such as test cases, test reports, etc.
Monitoring/controlling artifacts, such as Issue list, MoMs, Risks, etc.

How is it possible to review the work products for a 3 month project when we don't have a separate QA department and the stakeholders involved in development do the work product reviews one way or the other.

This same question holds true for reviewing procedures.

Of course, we review high priority documents, such as Project Plan, Use Cases, ERD, Application; but not all of them.

Can you help me understand what should be done for a short duration project, such that the PPQA PA requirements are met and we don't have to hire separate people just to fulfill the requirement?

The first thing that I would do is postpone your ML 2 SCAMPI A appraisal as apparently you have a major risk to achieving ML 2 since PPQA does not appear to be in place in your organization. And even if you could put PPQA in place for a 3 month project between now and your appraisals, that may still not be enough time to demonstrate institutionalization, meaning that you have a repeatable process. Essentially you will have one project using PPQA, which is one data point. And it is not possible to determine institutionalization from one data point. Your organization will be at serious risk of not achieving ML 2.

Industry average shows that PPQA is 3 – 5% of your organization. You haven’t told me how large your organization is. But if your organization is 25 people, than 1 person should be assigned to perform the PPQA practices.

I think that you are misunderstanding the differences between reviewing a work product and objectively evaluating a work product. It sounds like your project teams are already reviewing the work products. The role of PPQA is not to review the work products, but to audit the work products and processes to ensure that the work products follow the specific standards and are products according to your documented processes.

I highly recommend that you, or someone you select in your organization, take a training class on how to perform PPQA. I cannot adequately explain how to perform PPQA and answer your specific questions in this blog. The person you select for the training needs to be taught how to conduct a work product audit, how to conduct a process audit, how to plan PPQA audits, how to communicate audit results, and how to track audit non-compliances to resolution. If you don’t already have this capability in house, it will take some time to develop it internally. And I strongly advise against using an external consultant to provide this service. PPQA is for the benefit of your organization and management. It is essentially the eyes and ears of your senior management. And an external consultant may be motivated by other considerations than your best business interests if asked to provide PPQA services.

Why Isn't the SEI Implementing the CMMI for Itself?

2011-01-11T15:46:00.000-08:00

Why doesn't the SEI use its own model- CMMI for all its different product development and services? Even for SEI projects and program management it is crucial, and they have customers the world over. If the SEI goes for CMMI ML3 Appraisal it will be great for the user community and they can achieve their mission in a planned manner, right?

Do Lead Appraisers & SEI Partners feel that they can benefit if the SEI gets CMMI ML3 (defined Process)?

In such a case, who will appraise the SEI? (sorry for such a hypothetical Question)

As the SEI does not develop software, but delivers services, the CMMI-DEV doesn’t apply. That is why the SEI has not been previously appraised to the CMMI. However, the SEI is now implementing the CMMI-SVC for the services it delivers. This is a good thing and the SEI Partners are noticing some of the improvements. Obviously, by the SEI’s Conflict of Interest policy, a CMMI-SVC Lead Appraiser external to the SEI organization being appraised would have to lead the appraisal team.

Appraisals: Practice or Subpractice level?

2010-11-01T05:29:00.000-07:00

For successful SCAMPI appraisals, is there any reason to prepare process compliance at the sub-practice level? Would appraisers be looking for evidence at that level?

This is a question answered by taking the 3-day Introduction to CMMI class and also by your Lead Appraiser. There are three CMMI components: Required, Expected, and Informative. An appraisal only covers the Required (Goals) and Expected (Practices) components. Your Lead Appraiser should also be providing some training or guidance on how to build the PIIDs, which contain the objective evidence for an appraisal. And the whole appraisal team is involved in reviewing the PIIDs during the Readiness Review to determine if the evidence is proper for a SCAMPI appraisal.

If a Lead Appraiser or the appraisal team is appraising you to the sub-practice level, they have gone too far. The SCAMPI method is only concerned with appraising the organization to the Goals and Practices.

How Do We Select a High Maturity Consultant?

2010-09-13T14:11:00.000-07:00

My friend is a Quality Manager in a company who has reached CMMI Maturity Level 3. They now want to achieve Maturity Level 5. They started taking quotes from different companies. In the selection process they found that there are 3 or 4 major players in our country who have up to 3 High Maturity Lead Appraisers. Most of these companies have submitted proposals for consulting and appraisal in a single quote. Now my friend fears that:

Most of them already have at least 40 High Maturity clients and at least 30 Maturity Level 3 clients. Will they have the capacity do lead the appraisal on time for my friend's company ( considering 12 SCAMPI appraisals per year per LA) ?
Most of them deliver the consulting and training activities , which is 70 % of the contract value and sometimes they break the contracts and not deliver the SCAMPI, which is still highly profitable, since only 30 % value is lost, and no need to deal with High Maturity appraisal needs.

In order to address this issue, does the SEI publish a list of contracts for SEI Partner? Does the SEI have a specific committee or group to look in to the capacity management and availability management of their SEI partners, so that companies will not have such concerns?

All that the SEI does is maintain a list of SEI Partners and certified High Maturity Lead Appraisers. If there is indeed a problem as you have stated, then you or your friend should contact the SEI about the SEI Partner in question as this certainly sounds like unethical behavior.

Another issue is that an SEI-certified High Maturity Lead Appraiser cannot appraise the organization if he or she has provided the consulting to the organization, unless the SEI approves the potential Conflict of Interest.

The steps taken by SEI in this area are impressive. Also I understand that the control on appraisals/per year is established by SEI. Out of curiosity i would like to ask follow up questions.

I belive the critical part of this entire process is that consulting and apprisal services cannot be performed by the same Lead appriser. If there is a need of separate contract for SCAMPI A appraisal activities, and it cannot be included in a consulting contract, then it can have more credibility. Also like ISO where the certification agencies are audited annually (correct me if am wrong), will SEI do an onsite audit on SEI partners? Or do they have a databse of all the contracts established by SEI partners around the world (considering 800 to 1000 appraisals per year)? Because the user community trusts the SEI more than the SEI Partners ( for most of the users it may be the first time to contract with an SEI Partner and they might not be sure of the guidelines provided by the SEI or about the SEI Ethics commitee). All this can be prevented if the SEI takes a copy of all contracts established for SCAMPI A across the countries. What are your views on it? .

Note:The intent of the question is to increase the user communities' trust on SEI to increase, but not to reduce the credibility of SEI partners/Lead Appraisers.

The SEI does not have the time or resources to perform annual on-site audits of the SEI Partners. And as an SEI Partner, I would not welcome an on-site audit by the SEI. It would be additional expense for me.

What the SEI does provide that may help with your concerns is that they perform a QA audit of the results of every appraisal submitted by a Lead Appraiser. If the appraisal results do not meet the evaluation criteria, then a more in-depth audit occurs. What can then happen is that if the problems are serious enough, the Lead Appraiser can lose his or her CMMI credentials. This has happened to a number of Lead Appraisers since this policy was put in place.

In addition, each Lead Appraiser must be certified by the SEI, which provides another layer of credibility.

The SEI Partners provide the service and the certified-Lead Appraisers deliver the service. As a buyer of CMMI services, it is your responsibility to learn about the SEI policies regarding Ethics and Conflict of Interest, as well as the credibility of the different SEI Partners and Lead Appraisers. Otherwise, you get what you pay for. In other words, buyer beware!

And depending upon what country you are in, the SEI Partners are trusted as much or more than the SEI by the user community.

Achieving a Maturity Level Without a Consultant or Training

2010-09-12T04:45:00.000-07:00

I was wondering if it is possible to go it alone with CMMI Maturity Level 2. I have been told by many that attempting CMMI Maturity Level 2 without a consultant or highly trained staff would be somewhat challenging. What are your thoughts?

Also, having limited resources for process improvement being a strong possibility, do you have any recommendations for online sources that can help offset the costs?

Though it is entirely possible to implement the CMMI without hiring a CMMI consultant, that is a high risk approach. Though I don’t see how you could avoid taking the Introduction to CMMI training class. That, in my opinion, would be a huge mistake. There are some areas in the CMMI that are open to interpretation when trying to go it alone and you can end up doing things in the spirit of achieving Maturity Level 2 that have no business value to you. In addition, since your organization has to provide 4 to 8 appraisal team members, each appraisal team member must take the SEI’s Introduction to CMMI class.

These are all worthwhile expenditures of your process improvement budget. And when you compare these expenses to your internal costs for process improvement, these are usually negligible in comparison.

But if you are trying to do things as cheaply as possible, that begs the question, why are you even considering implementing the CMMI and trying to achieve Maturity Level 2? Basically you get what you pay for. Going the cheap route doesn’t demonstrate management commitment to process improvement and can result in wasted effort, wasted money, and an aborted process improvement initiative.

Bi-directional Traceability

2010-09-12T04:28:00.000-07:00

Our organization is in the process of preparing for a CMMI Maturity Level 2 SCAMPI A appraisal. We are concerned about our approach for bi-directional traceability REQM SP 1.4. We maintain traceability is follows:

1. High Level Requirements <--> Use Cases (Includes GUIs and Database Interactions) <--> Test Cases

2. Use Cases <--> Source Code

Note: One can trace from Test Cases to Source Code through the Use Cases and Vice Versa, but the traceability is not direct. The reason behind this is, test cases are generated from use cases and are tested against the application (black box testing). Source code does not have associated test cases.

Is this kind of traceability considered bi-directional and is satisfactory for Maturity Level 2?

What you describe is one of many ways to implement bi-directional traceability and meet the intent of the CMMI. If your method supports your business goals and objectives and there are no quality issues, your approach should be acceptable for a Maturity Level 2 appraisal.

It is interesting that you think tracing from Test Cases to Source Code via Use Cases may not be acceptable. Traceability is a multi-dimensional mapping that can have one-to-many and many-to-one relationships. As long as you trace from the top all the way to the bottom and vice versa, you should be fine no matter how many links there are in the chain, and the chain can have branches as well.

Please note that bi-directional traceability does not mean tracing one whole document to another whole document. What it means is that a given item in one document (a specific requirement for example) can trace to multiple items in another document, multiple items in one document can trace to one item in another document, one item can trace to one item, etc.

Some Appraisal Questions

2010-09-11T04:46:00.000-07:00

Why does the SEI ask for focus projects instead of all the projects done by the company?
Usually companies can select consultants and the Lead Appraiser (LA). Why is the SEI giving the right to chose the LA by the company or consultant?
Why do the appraisal results expire after 3 years?
Why doesn't the SEI have compliance appraisals every 6 months or 1 year similar to ISO?

The SCAMPI method is a sampling method to determine the degree of institutionalization of the processes on the projects. Therefore the use of focus projects. For organizations where there are only 1 or 2 projects, then all of the projects are usually included in the appraisal scope. But for organizations with many projects, it would be prohibitive to evaluate all of the projects. That is why it is the responsibility of the Lead Appraiser to select the focus projects, along with input from the organization. The principle here is that if the processes are truly institutionalized throughout the organization, then it doesn’t matter which projects are selected for the appraisal. Any set of selected projects should be representative of how all projects in the organization behave.
If I understand your statement, you are incorrect. The organization does select the CMMI consultant and Lead Appraiser. However, only SEI-certified Lead Appraisers are allowed to lead and report SCAMPI appraisal results. If a Lead Appraiser is NOT SEI-certified and he or she leads a SCAMPI appraisal, then the appraisal results are NOT valid.
The appraisal results expire after three years because in the past many organizations tended to backslide in their process maturity after having their appraisal. The three year period is long enough to address the findings from the SCAMPI A appraisal and prepare for a re-appraisal at the same or higher Maturity Level. If there is no expiration date, then there could be less motivation to continue with Process Improvement.
In addition, there is no such thing as a compliance appraisal at this time. There has been some discussion along these lines, but nothing has been settled. There is a fundamental difference between ISO audits and CMMI appraisals. ISO is a standard and the result of the audit is certification. CMMI is a set of guidelines for process improvement and the result is Maturity Level or Capability Level that is valid for three years.

Is Going Directly for a CMMI ML 5 Appraisal Allowed?

2010-09-11T04:33:00.000-07:00

Is a CMMI v1.2 ML 5 appraisal allowed in the following situation?

One of my company's divisions was successfully appraised to CMMI v1.1 ML 5 but the appraisal results expired in 2009. Now my company wants all three divisions, which are located in different cities, appraised to ML 5. Though skipping levels is not recommended, however, is it allowed to go for the appraisal?

There is nothing from the SEI that prevents a company from being appraised to whatever Capability Level or Maturity Level that it chooses. That being said, what does drive the CL or ML is the organization being able to collect, analyze, and correctly use data for statistical and quantitative analysis, as well as being able to institutionalize the behaviors and practices. You should hire an SEI-Certified High Maturity Lead Appraiser and have him or her perform a Class C or Class B appraisal to determine the risks with your current approach and implementation of ML 5. The outcome of this exercise will determine if it is feasible to achieve ML 5 at all three sites.

How Do I Become A Lead Appraiser?

2010-08-02T14:59:00.000-07:00

Would you please tell me step by step the procedure to become a SCAMPI Lead Appraiser (LA)? I am currently working with a Maturity Level 5, company in India with 5.5 years of relevant experience in Process and Quality Consulting. It would be really great if you could provide me with some references, emails, and sites as a roadmap.

Here are the steps and requirements for becoming a SCAMPI Lead Appraiser. This information is directly from the SEI’s web site.

To become an instructor or Lead Appraiser, you must successfully complete authentic SEI courses. The first prerequisite course for becoming either an instructor or Lead Appraiser is the Introduction to CMMI Version 1.2 course. This courses is available from SEI Partners (see SEI Partner Network Directory and Guide to Services) or from the SEI. The second prerequisite is the Intermediate Concepts of CMMI Version 1.2 course that is available only from the SEI.

CMMI Version 1.2 Instructor Training is the final course leading to candidacy as an Introduction to CMMI Instructor. After successfully completing this course, the candidate instructor must also be observed, by an authorized SEI Observer, teaching the SEI's Introduction to CMMI V1.2 course. Upon successful completion of the observation, the instructor is then authorized as an Introduction to CMMI Instructor.

SCAMPI Lead Appraiser Training is the final course leading to candidacy as a SCAMPI Lead Appraiser. After successfully completing this course, the candidate Lead Appraiser must be observed leading a SCAMPI A appraisal using a CMMI model and be approved by an authorized SEI Observer.

You can find more information on the SEI’s site by searching for SCAMPI Lead Appraiser. Of course you will have to either upgrade to v1.3 or take the v1.3 classes when the new versions are released later this year.

Query on CMMI for Development v1.3

2010-07-15T17:09:00.000-07:00

Our organization was appraised at CMMI ML3 in Oct, 2009 and we'll be going for ML5 in Q1 2011. Although we have been preparing ourselves for CMMI for Development v1.2, we are also aware that the CMMI for Development v1.3 will be out in January, 2011. We wish to get appraised for CMMI for Development v1.3. I have gone through some of the PPTs and PDFs on web but none of them give a clear insight into the changed expectations from the existing PAs or expectations from new PAs to be added.

It'll be a great help if anyone can provide me the draft version of v1.3 or any detailed document about the same.

One item that you may not be aware of is that the SEI would like to see at least 18 months between an ML 3 appraisal and a High Maturity appraisal. If you are planning for an appraisal in Q1 2011, that would be less than 18 months. Therefore you would have a very high probability of your appraisal results being audited by the SEI, which could take a long time before being accepted. I would encourage you to hire a High Maturity Lead Appraiser as soon as possible, if you haven’t already done so, and move your plans for your ML 5 appraisal at least 3 months or more into the future to ensure that you have enough data for performing the High Maturity practices and enough time for institutionalization of HM.

You will have to wait for the v1.3 release in November 2010. The problem with using drafts is that things can change before the release.

REQM and RD in the CMMI

2010-07-07T20:59:00.000-07:00

Why is REQM Management at Maturity Level 2 and Requirement Development at Maturity Level 3? We develop the requirements first and then manage them in the project.

There reason for the placement is due to the meaning of ML 2 vs. ML 3. ML 2 is all about stabilizing projects and gaining control over project estimates. Once the organization has achieved this, then it can begin to evaluate how to improve the engineering areas.

Since you need to have a baseline upon which to plan a project and the other ML 2 Process Areas, that is why REQM is the first Process Area in ML 2. The intent is to manage the collection of project requirements: good, bad, or indifferent. And use this collection to plan the project, etc. Then when you have achieved ML 2 and move to ML 3, then you can address how to improve the Requirements Elicitation to obtain better requirements.

Please keep in mind that the CMMI is a collection of guidelines and best practices for doing process improvement. The CMMI is not a roadmap for how to do software engineering.

Implementing CMMI Along With ISO 9001

2010-07-06T06:37:00.000-07:00

Suppose a company already has an ISO 9001 certification. Then they decide to achieve an appraisal at level 3. Suppose further that their plan is to add additional process assets according to their gap analysis. But then they discover that certain ISO 9001 work instructions or templates require changes to meet the requirements of CMMI level 3. If these assets are changed, would that necessarily invalidate the ISO 9001 certification? Under what conditions would the certification be invalidated, and what needs to be done under those circumstances? Is there a way to avoid this issue?

Without having any details as to your situation, I find it hard to believe that if you are ISO 9001 certified that the CMMI is causing you to make changes to your quality system that would invalidate your ISO 9001 certification. The ISO 9001 standard and the CMMI-DEV model are compatible. Are you working with an SEI-certified Lead Appraiser/consultant? If not, you may be making some decisions to change processes and process assets that are not necessary.

To properly address your concerns, you should have a Lead Appraiser conduct a gap analysis of your organization to determine what you currently have in place that is compatible with the CMMI and identify those gaps that need to be addressed in order for your organization to achieve Maturity Level 3. And any updates to existing processes and process assets should be compatible with both the ISO standard and the CMMI-DEV.

Project Planning SP 1.2 - Task Attribute: Effort or Size?

2010-07-02T16:41:00.000-07:00

Is it possible to establish estimates of work product and task attributes by means of task time estimates? Can the task effort be similar to the size of a task?

By going directly to task time estimates you have effectively skipped performing Project Planning Specific Practice 1.2. The intent of SP 1.2 is for you to perform some sort of basis of estimate for the project’s tasks and activities. This is a bottom-up approach. If you are not used to this approach, it can be a struggle at first to take a step backwards from task time estimates and really understand the underlying assumptions that people are making in their heads about the factors that are driving the task time estimates. Some very basic task attributes include estimating the number of pages in a document that is being produced or updated, the number of technical drawings being produced or updated for a hardware item, the number of new or modified interfaces, the number of new or modified screens , etc. Then based on your historical data from previous projects, it is possible for you to empirically determine a set of productivity factors that will convert these sizing parameters into effort and arrive at the task time estimates. The bottom line is effort of a task is not the same as the size of a task.

Do you think this practice would be Fully, Largely, Partially or Not Implemented? Would this be this a problem in a SCAMPI A appraisal? What do you think about that?

Taking this example out of context with everything else your organization is doing makes this a difficult question to answer. The appraisal team is the only group that would be qualified to make that judgment based on documented evidence and the interviews. However, as a Lead Appraiser, I would have to say that you have a problem that needs to be addressed before you conduct a SCAMPI. The SCAMPI rules state that if a Process Area is in the scope of the appraisal, then all of its Specific and Generic practices are applicable. And if you are not performing a practice, which may or may not be the case, then there could be issues in Project Planning that impact Goal Satisfaction and result in a Maturity Level 1 rating.

To provide you the best answer, you should be talking to your Lead Appraiser and have him or her give you the proper guidance on this issue. As a risk mitigation, I would recommend that you put a process in place to estimate sizing parameters that are then used to calculate effort. Your estimators are already doing this, but it sounds like they are doing it in their heads. You just have to break the process down into smaller steps to allow the sizing estimates to be captured first. There is benefit to doing this.

Appraisal Sponsor Roles and Responsibilities

2010-07-01T06:50:00.000-07:00

I am looking for information regarding the roles and responsibilities of the appraisal sponsor.

My organization is preparing for an external appraisal and I'd like to provide the sponsor with a summary of her role and what she will be expected to do for the appraisal such as review and sign the appraisal plan, attending the opening and findings briefings etc.

It sounds like you have a pretty good understanding already of the appraisal sponsor’s duties. It would be helpful to you to ask this question of your Lead Appraiser to see if he or she has any specific requests for the appraisal sponsor. As a Lead Appraiser, these are my expectations. The appraisal sponsor:

Provides the funding and senior management commitment for process improvement and the appraisal
Does not serve on the appraisal team
Meets at least once with the Lead Appraiser to discuss the appraisal
Provides the business objectives for the organization
Signs the Appraisal Input Document, Appraisal Plan, and Appraisal Disclosure Statement
Attends the Opening Briefing to reinforce why the appraisal is being conducted and the importance of everyone to support the appraisal team
Meet with the appraisal team before the Final Findings presentation to privately receive the results and prepare the proper message for the organization at the Final Findings Presentation
Attend the Final Findings Presentation and at the conclusion of the presentation thank the appraisal team for their efforts and thank the organization for their efforts regardless of the outcome. If the results were bad news, turn it into a positive statement of encouragement, etc.
Complete the feedback form in SAS for the appraisal results
Receive the appraisal results and maintain the appraisal record for three years
Do not publicly disclose the appraisal results until the SEI has completed its quality review and announced the results to the Lead Appraiser and appraisal sponsor

How to Start With the CMMI

2010-06-30T06:30:00.000-07:00

If a private software company wants to do get a certificate of quality for CMMI Level 2, what must this company do in order to obtain Level 2? And also what are the steps that must be followed from the start (initial) to achieve CMMI Level 2 (Managed)?

First let me state that there is no certification of any type for the CMMI. Individual Lead Appraisers will issue something looks like a certificate to the organization that was appraised indicating the SCAMPI A results. But this is not a certification. The only things that the SEI certifies are the CMMI instructors and Lead Appraisers.

Here are the necessary steps for achieving Maturity Level 2 for the CMMI-DEV. These steps are not necessarily sequential, some can occur concurrently.

Hire an SEI-certified Lead Appraiser to conduct a Gap Analysis of the company to determine the current process strengths and weaknesses and help the company construct a Process Improvement Plan (PIP).
Obtain executive management sponsorship for the process improvement effort.
Train the people responsible for the company’s processes and for addressing the action items in the PIP on the 3-day SEI Introduction to CMMI v1.2 class.
Address the issues from the Gap Analysis, document the necessary processes and procedures, and begin conducting the PPQA process and work product audits.
Allow time for the new and/or modified processes to get some use on various projects.
Conduct a SCAMPI B appraisal as a dress rehearsal for the SCAMPI A. Identify any issues and weaknesses that are potential risks to achieving Maturity Level 2.
Create a new PIP to address the SCAMPI B identified weaknesses and risks.
Address these issues
Conduct the Maturity Level 2 SCAMPI A appraisal.

Using the CMMI-SVC to Transform an Organization into a High-Functioning, Customer-Driven Profit Center

2010-04-14T01:07:00.001-07:00

For those of you who were not able to attend the 2010 North American SEPG Conference in Savannah, GA in March, here is a copy of my presentation on the CMMI-SVC.

As a company grows and matures from a startup entrepreneurial venture to a sustainable corporation, the departments and company services that begin as good ideas expand and evolve to support the company’s growing business. Many times these services simply develop without any strategic vision resulting in institutionalized behaviors that are incompatible with the company’s business goals and objectives. Consequently, the transition to a larger corporation becomes a challenge. A notable example is a company’s Engineering Services Department.

When people think of Engineering Services, the Customer Support or Help Desk team is what first comes to mind. However, other services such as Product Training, Field Services (product installation and troubleshooting), and Engineering Sales Support may be provided as well.

As a product development company begins selling product, the Customer Support function becomes one of its first service offerings whether or not it recognizes it as such. In addition, it is natural for the focus of the Customer Support function to be on pleasing their customer base, as many sales are contingent upon repeat business and word of mouth until the company and its product line become established in the marketplace. Nevertheless, without a clear idea of its charter and strategic direction to support business growth and identify new markets and service offerings, the Customer Support Specialists focus instead on supporting their customer base on non-company and non-product issues and questions that consume internal resources without any tangible benefit to the company. Once a company starts banging its head on the “glass ceiling” as it attempts to grow, the leadership may recognize that its current Engineering Services approach does not support its strategic business goals and objectives.

In these circumstances, the company is not necessarily interested in implementing the CMMI for Services (CMMI-SVC) and becoming appraised to either Maturity Level 2 or Maturity Level 3. However, by using the Continuous Representation, the CMMI-SVC can provide the needed guidance to help a company restructure and reorganize its Engineering Services approach in order to become a profit center or revenue generating function.

In this presentation, we will present a case study for OMNI Flow Computers, Inc., a company that specializes in the design, development, and manufacture of panel-mount multi-run, multi-tasking liquid and gas flow computers, and field-mount, hazardous area controllers/RTUs for liquid and gas custody transfer metering systems. The challenge facing OMNI was to develop its Engineering Services Department into a high-functioning, customer-driven profit center. OMNI’s Engineering Services Department consists of three groups: Customer Support, Training, and Engineering Field Services. Customer Support handles customer questions, concerns, and issues. The Training group provides training on the OMNI product line to its customers and users. Engineering Field Services provides on-site troubleshooting services on an as-needed basis.

As the Training and Engineering Field Services groups were recent additional capabilities, Customer Support presented the biggest obstacle to overcome. Noted management consultant Peter Drucker declared several years ago that Quality in a service or product is not what you put into it. It is what the client or customer gets out of it. Moreover, an obstacle to achieving this objective was one of the core challenges faced by the department: developing an appropriate customer focus and developing new service offerings. A major reason for these challenges is the nature of OMNI products. OMNI's customers integrate their products into custody transfer systems that involve a wide variety of large-scale hardware and electronic equipment from other manufacturers. OMNI’s customers usually develop and commission these systems for their clients and end users. Therefore, when calls come in to OMNI’s Customer Support group, the first challenge they had to overcome was determining if the customer’s issue was actually an OMNI product issue or the result of an external issue. The next challenge was to determine the root cause of the issue, so that the customer would receive a timely resolution of their issue.

Fortunately, the release of the CMMI-SVC came at the right time for OMNI. Of the seven new services Process Areas (PAs), many of the Specific Practices and associated informative material proved useful in guiding the transformation of OMNI’s Engineering Services Department. The Service Delivery PA provided excellent guidance for establishing and documenting Engineering Services’ existing service offerings, ensuring that each group was prepared to deliver the defined service offerings, and delivering the service offerings. The Incident Resolution and Prevention PA provided excellent guidance for identifying, documenting, tracking, reporting, and resolving customer complaints, issues, and other service interruptions. The Service Continuity PA helped focus the Engineering Services Department manager to identify and prioritize the department’s essential functions and necessary resources. The Strategic Service Management PA brought the needed focus to establish the Engineering Services strategic needs and plans for its standard services.

The OMNI Engineering Services Department’s journey is not yet over. They are still growing, maturing, and learning what it means to become a high-functioning customer-driven profit center. However, along the way they learned some valuable lessons. This presentation will discuss some of the pitfalls they encountered, what strategies worked and what did not work, as well as provide some practical advice to aid other organizations facing similar challenges.

If you make customers unhappy in the physical world, they might each tell six friends. If you make customers unhappy on the Internet, they can each tell 6000 friends. - Jeff Bezos

Customer service is not a department. It is an attitude. – Unknown

This presentation provides a case study of a computer manufacturer that used the CMMI for Services to help transform its Engineering Services department (Customer Support, Training, and Engineering Field Services groups) into a high-functioning, customer-driven profit center. Challenges, successful approaches, lessons learned, and practical advice to aid other organizations facing similar challenges will be presented.

Using the CMMI-SVC to Transform an Organization into a High-Functioning, Customer-Driven Profit Center

View more presentations from Henry Schneider.

Process Improvement vs. Process Maintenance

2010-04-13T23:55:00.000-07:00

An organization on its process improvement journey may include process maintenance activities under the heading of process improvement. Most organizations do that. As far as I know, CMMI does not talk of process maintenance activities. To me, a few examples of process maintenance activities are those required for metrics data collection, project monitoring, process definition, making plans, etc. On the other hand, process improvement will include activities like metrics data analysis, identifying significant variations from planned arrangement, process re-definition based on improvement suggestions and metrics analysis, revising the plans in line to changing requirements, and taking appropriate preventive and/or corrective actions.

Are there any benchmarks for healthy process maintenance vs. improvement activities for software organizations?

These two concepts in inextricably intertwined. I would find it hard to believe that an organization does not identify any process improvement suggestions by just doing process maintenance. Even if you are just maintaining your Maturity Level, you will still identify improvements to the status quo. And I contend that even for process maintenance you have to perform data analysis, identify variations, and identify appropriate corrective actions, otherwise how would you be able to determine that you are maintaining what you have already achieved? I do not understand the need to define these two items as separate activities. What would be the point of merely maintaining your processes?

Configuration Management - Change Request Number Traceability

2010-04-11T13:45:00.000-07:00

I would like your help with understanding the following subject:

A change request is processed in a change control tool. Each change request has a number. If a proposed change is accepted, a schedule is created for making the change.

When it is necessary to store the configuration items modified in the configuration system, we have the option of describing what we have done. We record the implemented changes and the reasons for the changes, however we don't track the change request number.

Will the lack of tracing the change request number create a weakness in the Configuration Management Process Area? Is just recording the historical information about the changes in the comments and using the change control tool sufficient?

How important to your projects and business is it to track the change request number? Have you encountered issues with change control on the projects because you have not maintained this information? Recording and tracking the change request number is a very easy thing to do and I am wondering why you haven’t done that.

What also concerns me is that you imply that recording a description of the change made to a baselined item may be optional. If this is true, then it is then possible for someone to decide not to record this information. Recording the information should be mandatory.

Not tracing the change request number and the possibility of not recording a description of the change sound like high risk items that can cause you difficulties in the future. As a risk mitigation, I would record all change numbers and require that all changes made are clearly described in the event that someone else in the future needs to understand the nature and reasons for the change.

And since you are using a tool, which I assume is an off-the-shelf product, you may in fact be tracing the change request number and not even realize it. But from an appraisal point of view, it sounds like you might have most of Configuration Management covered and the problem you are asking about might be viewed by the appraisal team as a weakness. But is it a Goal breaking weakness? This a question that must be answered by the appraisal team.

Configuration Management - SP 1.3-1 , SP 2.2.-1

2010-04-10T17:13:00.000-07:00

I would like your opinion in relation to situation below.

Once a change was been verbally approved, is there some problem in relation to CM SP 1.3 (Create or Release Baselines) - sub-practice 1 and CM SP 2.2 (Control configuration items) - sub-practice 2, if the record of the modification in the schedule and the change control tool are done in subsequent periods until a maximum limit of time of the defined period for monitoring of the project?

Example:

Verbal approval of the change - Monday

Beginning of the work - Tuesday, with storage of the configuration items in the configuration system (without baseline).

Register of the change in the change control tool and record of the activities in the project schedule - Friday

Collection of the progress and effort - Next Monday (weekly Monitoring, all monday)

In advance, thank you very much.

The first problem I see is the verbal approval. Verbal approvals are difficult to document or provide as evidence and over time can be forgotten or misremembered, not to downplay the risks associated by not documenting decisions.

Secondly, since you are asking about the sub-practices, that makes me wonder if you have taken the SEI’s Introduction to CMMI class. If you have taken the class, your instructor should have made clear to you the distinction between the required, expected, and informative components of the model. The sub-practices are informative components and are therefore provided only as information to help you understand the intent of the Specific and Generic Practices and Goals. To be clear, you are neither required nor expected to implement the sub-practices.

If the weekly schedule you described works for your projects, then you should be able to map it to the Configuration Management (CM) Specific Practices (SPs). However, the described weekly schedule sounds like it may have some gaps.

I strongly suggest that you have an SEI-certified Lead Appraiser conduct a Gap Analysis (Class C appraisal) of your organization, especially CM, to determine the gaps between your implementation and the CMMI.

Appraisal Scope Question for Agile Development

2010-03-31T06:48:00.000-07:00

I am preparing for an appraisal coming up in a few months and I have run into an interesting question that I would like to ask you about.

Some of the projects in the Organizational Unit (OU) follow an Agile method for development. It is really a modified combination of SCRUM and XP. One of the questions they asked me was whether they would have to provide evidence for all of the Process Areas (PAs) in the scope of the appraisal? I was perplexed by this question because it seemed rather obvious to me, so I replied "of course." But this question stuck in the back of my mind so I dug into the SCAMPI Method Definition Document (MDD) for more clarification.

From the MDD, Method Assumptions and Design Principles, page I-19 and I-20: "The extent to which an organizational unit has implemented appraisal reference model practices can be determined only by considering, in aggregate, the extent to which those practices are implemented within the organizational unit by project and support groups. This process, in turn, necessitates the consideration of objective evidence for each instantiation, for each model practice within the appraisal scope."

This seems fairly clear to me, we must examine objective evidence for all practices of all instantiations within the scope of the appraisal. However, under the process definition in part 2 of the MDD you can find less convincing information under the Parameters and Limits section of 1.1.3 Determine Appraisal Scope. "Sample projects and support groups selected to form the organizational scope (i.e., the combination of focus and non-focus projects and support functions) must represent all critical factors identified for the organizational unit to which the results will be attributed." So it seems that as long as the combination of all focus and non-focus projects represent all the critical factors it should be sufficient. I know, critical factors are different from the PAs in the model scope. :-)

Also, in Parameters and Limits it clearly states: "Focus projects must provide objective evidence for every PA within the model scope of the appraisal..." "Non-focus projects must provide objective evidence for one or more PAs within the model scope of the appraisal..." So it seems that if an Agile project was a non-focus project they would not have to provide evidence for all PAs. Do you think this is correct?

The MDD goes on to further say in the Parameters and Limits: "In appraisals where the reference model scope includes any project-related PA, the organizational scope must include at least one focus project. If the organizational unit includes more than 3 projects, then the organizational scope must include sufficient focus projects and non-focus projects to generate at least 3 instances of each practice in each project-related PA in the model scope of the appraisal."

So, conceivably, you could have only 1 focus project, and say maybe 6 non-focus projects with each only having a few PAs to "generate at least 3 instances" of the practices that need to be covered in the scope of the appraisal.

I know this may seem a bit extreme, but believe me, I have run into stranger scenarios than this one. lol :-)

Ideally, all projects would cover all PAs. But the role and function of non-focus projects really seems to muddy the waters in my opinion.

What are your thoughts?

There is an SEI Technical Note CMU/SEI-2008-TN-003 published in November 2008 titled CMMI® or Agile: Why Not Embrace Both! by Hillel Glazer, Jeff Dalton, David Anderson, Mike Konrad, and Sandy Shrum that contains a lot of pertinent information about Agile and the CMMI, as well as Hillel's recently published companion article in Crosstalk Love and Marriage: CMMI and Agile Need Each Other http://www.stsc.hill.af.mil/crosstalk/2010/01/1001Glazer.html

In my opinion, if Agile is one of many different development methods in use by the organization and the Agile projects do not cover all of the PAs in scope of the appraisal, then it is not in the spirit of the MDD and non-focus projects to mix and match a number of Agile projects just so there is evidence for all the PAs. The way that I look at non-focus projects is that there may be some projects in the appraisal scope that do not have evidence because the projects have not reached that part of the lifecycle yet, NOT that they are not performing the practice.

And, just because a project is using Agile in a Maturity Level (ML) 3 organization does not mean that they don’t have to perform all of the PAs. There should be evidence available though they may be using terminology that is not obviously mapping to the CMMI. Someone with deep CMMI and Agile knowledge should work with the organization to show them how what they are doing does comply with the CMMI.

High Maturity Reference Books

2010-03-25T15:38:00.000-07:00

I would like to read and understand more about High Maturity practices and I am also very keen to develop statistical skills so I will be able to construct a PPM. Can you please recommend some good reference books and/or links?

I would recommend the following books:

“Statistical Methods from the Viewpoint of Quality Control” by Walter A. Shewhart

“Understanding Variation: The Key to Managing Chaos” by Donald J. Wheeler

“Measuring the Software Process” by William A. Florac and Anita D. Carleton

“Building Continual Improvement” by Donald J. Wheeler and Sheila R. Poling

“Metrics and Models in Software Quality Engineering” by Stephen H. Kan

“Practical Software Measurement” by John McGarry, David Card, Cheryl Jones, Beth Layman, Elizabeth Clark, Joseph Dean, and Fred Hall

These are all excellent reference books to better your understanding of statistical and quantitative thinking.

Shortened CMMI Explanation

2010-03-21T14:31:00.000-07:00

I have just started CMMI in my organization. I am looking for the shortened explanation for each of the CMMI Process Areas. I then plan to preach CMMI in much simpler, easy to understand language which would be like bulleted points. I also intend to offer seminars in my organization with different departments to make them understand what exactly do they are required to do. If you can help me, I would be very thankful.

I applaud your initiative and your plans. The best advice that I can give you is to take the SEI’s 3-day Introduction to CMMI class. That class will provide you with the necessary information for you to craft a shortened version for your organization. As an alternative, you can read the CMMI. The introduction of each Process Area provides a short explanation of the intent of each Process Area. But if you do not have any background with the CMMI, hire a CMMI consultant and explain to him or her your needs and have them help you prepare the training material you desire. What you are asking for is not free. It takes time and effort to produce and is a consultant's intellectual property.

PPQA After Maturity Level 2

2010-03-19T13:57:00.000-07:00

I work in an IT organization that achieved CMMI Maturity Level 2 several years ago (we let the rating lapse) and I was wondering if you had some ideas on the following two questions:

1) What types of activities would PPQA engage in if the org had been Maturity Level 2 (I think they could have pursued Maturity Level 3 and been close)? Please also consider that the company is pursuing other types of improvement methods and models such as lean/6-sigma and ITIL.

2) What strategies should we pursue to show the worth of PPQA? Even in the good old CMM days and SQA one of the issues I had was that it was difficult to show the practical monetary worth of these support functions; one generally had to take it on faith that PPQA/SQA delivered some degree of worth to the company. Any thoughts?

The answer to question 1 is simple. Just read the PPQA Process Area and GP 2.9. The PPQA activities include performing both process and work product audits of the project and organization processes. For Maturity Level 2 that would mean auditing your REQM, PP, PMC, SAM, MA, PPQA, and CM processes.

The answer to question 2 is a bit more difficult. Basically you are asking, what is the cost of quality? One method you can use is to look at the total cost for the project and analyze it using Crosby’s Cost of Quality Model. The total costs break down into two categories: the Cost of Quality and the Cost of Performance.

The Cost of Performance includes such things as: generating plans, documentation, and developing requirements, design, code, and integration.

The Cost of Quality breaks down further into two categories: Cost of Conformance and Cost of Non-Conformance.

The Cost of Non-Conformance includes fixing defects, reworking documents, updating source code, re-reviews, re-tests, patches, engineering changes, CCBs, external failures and fines, Customer Support, and Help Desk.

The Cost of Conformance breaks down to two more categories: Cost of Appraisal and Cost of Prevention.

The Cost of Appraisal includes reviews, walkthroughs, testing (first time), independent V&V, and Audits.

The Cost of Prevention includes training, policies, procedures, tools, planning, quality improvement, data gathering and analysis, root cause analysis, and quality reporting.

The cost of PPQA is included in the Cost of Prevention.

When you consider these definitions and cost break down, the only category that will be affected by PPQA is the Cost of Non-Conformance. When PPQA audits the processes and work products, the audits will reveal non-conformances with people following the documented processes and procedures, which lead to re-work. By addressing these non-conformances, the goal is to reduce or effectively eliminate the rework and that is where you can demonstrate the value of PPQA.

Hope this helps.

Product Vision vs. Product Roadmap

2010-03-08T13:28:00.000-08:00

I have a question. Is the Product/ System Solution Vision and the Product/ System Solution roadmap one and the same thing? And which comes first, the Product Vision or the Product Roadmap?

Product Vision and Product Roadmap are not CMMI terms. So, in the context of the CMMI, simply use the standard dictionary definitions of vision and roadmap and you should have your answer.

The way I look at these two concepts is that vision refers to what you are supposed to achieve and roadmap is how you get there. Therefore vision would come first followed by the roadmap.

The bottom line is that you should define these terms for your organization if you think that they apply and are important.

Manage Corrective Actions of PMC

2010-03-07T16:08:00.000-08:00

During the analysis of a problem management tool I identified several actions that were open without a deadline and status record or with a deadline expired and without status record. Some actions were without a deadline because they depended on external agents to be resolved (customer, priority addressed by higher level) and others without updating the history and deadline because these didn't have modified status. The monitoring of these actions was performed through interviews. What is the impact of this condition on sub-practice 2.3-1 - Manage Corrective Action of PMC? Does this condition affect the reach of this practice? For me this condition characterizes a non-compliance. I would like to know your opinion.

The correct answer depends upon what you have written in your documented procedures for identifying issues and taking corrective actions. Do your processes and procedures allow you to to conduct verbal reviews of issues and take corrective actions? I do agree with you that you have identified some problems with your process and the proper completion of forms.

If you believe these are true non-conformances, but your documented processes allow this to happen, then it would be a good idea to modify your processes accordingly. Otherwise, you have identified a gap between the documented and practiced processes. These issues should have also been identified through the PPQC process and work product audits.

Also, since the sub-practices are an informative component, they provide additional material to help you understand the intent of the Specific Goals and Practices. There is no expectation that the sub-practices have to be implemented. Therefore, the situation you described really has no impact on PMC SP 2.3 Sub-practice 1.

CMMI for Modeling and Simulation Efforts

2010-02-26T12:04:00.000-08:00

I am interested in applying CMMI techniques to modeling and simulation efforts. I do not mean the modeling and simulation of the software engineering process (I have plenty of information, models etc. on that).

We do modeling and simulation work (e.g. growth of the biofuel industry, migration issues, watershed management) to inform policy makers. We have a grassroots 'best practices' movement and are looking for suggestions, ideas, etc. to improve our process. We do not use 'popular' tools and the niche software we do use is not conducive to configuration management tools etc. In addition, we rarely have a requirements document. We are essentially doing research.

Do you have any suggestions?

In my experience, a pure R&D shop has difficulty implementing the CMMI-DEV because of the free form nature of the environment and work. However, what makes more sense for you to investigate is the CMMI-SVC. It sounds like you are performing engineering or research and analysis services for policy makers.

As you may not have a requirements document for your niche software, you most surely have requirements for the research you are performing for your clients. If not, then you run the risk of having your results called into question.

Therefore, I think that you would be better served by the CMMI-SVC than the CMMI-DEV as it sounds like you may not be doing a lot of development, but instead providing research services.

Project Tracking Through Milestones

2010-02-24T15:51:00.000-08:00

We have a Project management process defined for covering Project Planning (PP) and Project Monitoring and Control (PMC). It includes a Work Breakdown Structure (WBS) (task effort size must be about 20 hours), estimation process, assigned resources, etc. We are using MS-Project for tracking the schedule, and each month there is an Excel report with a project status summary.

People think it is very heavy to track the all of the fine-grained tasks in MS-Project. They would prefer tracking the project through milestones in Excel.

I’ve heard about using the burn down chart in Scrum, and I know some organizations use agility within CMMI model.

It is possible to do that? Could we be more “agile”?

The short answer is yes. The CMMI does NOT prescribe any project management tool or level of tracking. These decisions are left up to the organization to make. Sounds like what you need is a process that fits your organization. If people are complaining that the tracking process is too cumbersome to use, then you should definitely examine other methods. Take a top down approach from the business goals and objectives. What project tracking information does a project manager need in order to determine if the business goals and objectives are being met? Once you answer this question, that will help you decide the proper level of project tracking and monitoring. If you are still having difficulty figuring out the best method, work with a CMMI consultant to help you define a process that will be the best fit for your organization.

How Do I Determine Estimates of the Size Attributes?

2010-02-23T09:38:00.000-08:00

CMMI-DEV Project Planning (PP) Specific Practice (SP) 1.2 states 'Establish and maintain estimates of the attributes of the work products and tasks.'

During appraisals it has been noted that most of the time we are relying on expert knowledge to determine effort and cost rather than estimating the sizing attributes. Just using expert knowledge makes it difficult to improve our effort estimation, hence having a dedicated set of attributes is essential in order for us to improve our estimations. We have evaluated some comprehensive methods (e.g. Function Points), but these are not considered as beneficial to our projects. We are now looking for an approach that allows us to improve our estimates with a simpler method which could then be refined based on our needs. Would you please give me advice how to tackle this challenge?

What PP SP 1.2 is looking for is the Basis of Estimate for the work product. If you are estimating a document for example, what aspects or attributes of the document can you estimate that when combined with a productivity factor will yield effort and cost? You determine these attributes by analyzing the historical data from your organization’s past projects. So based on the project’s requirements, you could estimate the number of pages that would be written for a document, the number of figures or drawings that need to be created or modified, etc. These items are then sizing parameters.

For source code, you need to analyze historical data from past projects to give an empirical estimate. As you say, function points are comprehensive. A simpler method may be to classify your software requirements in categories such as interface requirements, display requirements, processing requirements, reporting requirements, etc. and then using your historical data determine the correlations between these requirements categories and code size. Many organizations start with this simple kind of estimation/prediction model and continually refine it by incorporating data from each new project until a fairly accurate estimation model emerges. This estimation model is then very specific to the organization and type of software development it performs.

Hope these ideas help.

GP 2.9 in PPQA process area - what do I need?

2010-02-04T08:08:00.000-08:00

I'm responsible for coordinating CMMI ML2 implementation in my organization. For PPQA GP 2.9 "Objectively evaluate adherence of the process against its process description, standards, and procedures, and address noncompliance" who can be an internal auditor of my organization?

And about tools and forms for this evaluation: Could I elaborate a checklist for this evaluation? Do you have any examples of this?

An objective evaluation implies some independence from the people performing the process activities. That means the people who perform the PPQA activities do not evaluate their own work. Organizations perform GP 2.9 of PPQA in a variety of ways.

1. Someone else in the organization who is not performing the PPQA audits of REQM, PP, PMC, etc. audits the PPQA activities

2. If the company is large and has several divisions, someone who performs the PPQA activities in another division audits the PPQA activities in your division

3. An external auditor (e.g., ISO 9000) audits the PPQA activities

4. An external consultant audits the PPQA activities

5. If your company is a government supplier, then the government may be auditing the PPQA activities

6. Etc.

So in your case, you could use an internal auditor to audit the PPQA activities as long as that person is not auditing his or her own work.

As far as a checklist, that needs to be developed by the person who is auditing the PPQA activities, just like the PPQA audits develop the checklists for the other Process Areas. The checklists need to cover both a process audit and a work product audit, and it may be easier to have two checklists instead of one. The checklist needs to be based on your documented PPQA process and process assets, not the CMMI.

If you are having difficulty understanding how to write a checklist, then I strongly encourage you and your organization to have someone come and train you how to perform PPQA audits. It is very important to conduct these properly; otherwise you could face difficulties with your process improvement efforts.

SCAMPI A Appraisal Questions for FAR Groups

2009-12-06T03:26:00.000-08:00

Can you share the Questionnaires for a ML 3 SCAMPI A appraisal? Currently my organization is preparing for SCAMPI A and I want to trained the interviewees as this is their first experience an appraisal.

I am looking for some questions based on each Process Area, so the FAR Group Members will be prepared to answer them.

I have searched on Google but not able to find any information. :(

I am not surprised that you have not been able to find what you are looking for on the internet. A Lead Appraiser will not share his or her questions with you. Simply because the Lead Appraiser does not want the interviewees to be trained on what answers to provide, thereby biasing the results of any SCAMPI appraisal. If you and the organization are nervous about the appraisal, don’t be. There is no preparation for the interviews necessary. All that is expected by the Lead Appraiser is that any interviewee should be able to talk about HOW they perform their job duties. If people cannot do that, then the organization is not ready for an appraisal.

If you still feel uneasy, talk to your Lead Appraiser and have him or her conduct a Class B appraisal with interviews. A Class B or SCAMPI B appraisal can function as a dress rehearsal for a SCAMPI A without having the organization and the people worry about “flunking” or “passing” the appraisal. The interviewees will have the experience of being interviewed.

If you still want to perform an internal set of interviews, simply take the CMMI and step through the applicable practices with different groups of people (project managers, configuration managers, developers, testers, etc.) and ask them HOW they perform the Specific and Generic Practices. There is no set of standard questions. Each Lead Appraiser has their own style of questioning interviewees.

REQM SP 1.2 Question

2009-12-05T01:22:00.000-08:00

Can you please help me?

I am developing a study based on CMMI-DEV and I am having some difficulty with the interpretation of Specific Practice 1.2. My concern is how to apply SP 1.2 in practice. Who are the members of the project that must be committed to the requirements (business analysts, systems analysts, test analysts, developers, systems architects, project manager, and project leader)?

How should their commitment be documented? For example, for each new or changed requirement the members must sign a document that means that they are aware of the requirement and are committed to it.

Do you have a copy of the Addison Wesley published CMMI-DEV version 1.2? If so, there are some very helpful tips in the margins for REQM SP 1.2.

When you have a set of requirements for a project team, everyone on the team impacted by the requirements needs to share a common understanding of the requirements. In addition, since you typically do not have the full set of requirements available at project start, the project team needs to continually evaluate and reevaluate their ability to meet the requirements as the requirements evolve over the life of the project. One way of evaluating requirements is an impact assessment on the existing requirements, design, documents, test cases, current and downstream tasks and activities, cost, schedule, etc. for a new or changed requirement. By going through an impact analysis, that will provide a way for the project participants to communicate their ability to meet the requirements and associated commitments. The resulting impact analysis report can be one way of documenting the commitment. Other methods for documenting commitments include meeting minutes, document signatures, or email. However, just signing a document indicating that the participants are aware of a requirement really does not indicate that they are committed to the requirements and you might run into difficulties later on in the lifecycle.

Please keep in mind that commitments include both the resources involved (people, tools, and facilities) and the schedule for completion.

Covering SAM SP2.2 and SP2.3

2009-12-02T04:00:00.000-08:00

I want to ask you about CMMI Supplier Agreement Management (SAM) SP 2.2 Monitor Selected Supplier Processes and SP 2.3 Evaluate Selected Supplier Work Products. I have difficulty in understanding these practices and how to apply them in my company. We have a project that needs to buy new hardware and deliver it to customer.

How can I select and monitor supplier processes in this case?

Can you please explain that to me and give me an example of a supplier process that is applicable in our situation?

The best way to answer this question is to attend the SEI’s Introduction to CMMI 3-day class. During the SAM topic the instructor will explain the difference between these two practices for you. I would also suggest that you read the informative material in the SAM Process Area, for not just these two practices, but all of the Process Areas. The informative material is extremely rich in information, hints, tips, etc. and it provides guidance for understanding the intent of each practice and goal. I would also recommend that you not use the goal and practice titles for anything other than labels. The titles are shortened statements and do not always communicate the correct meaning of the goal or practice.

If you read the informative material for SAM SP 2.2 and SP 2.3 it will be immediately clear to you the differences. And if you still have questions regarding your specific circumstances, I suggest that you ask your SEI-certified Lead Appraiser. Without having a good understanding of your company and how it conducts business, asking specific implementation questions will most likely not provide the correct advice to you.

However, here is a very brief description of the differences between these two practices.

SAM SP 2.2 “Select, monitor, and analyze processes used by the supplier.” You as the acquirer of the supplier’s products or services must define in the supplier’s contract or agreement those specific supplier processes that are critical to your success. You have to state how and when you will be monitoring these critical processes. This practice is done as a risk mitigation to avoid surprises at the end of the contract when the supplier delivers the end product or service. Think of this practice as the acquirer performing PPQA process audits of the supplier’s processes.

SAM SP 2.3 “Select and evaluate work products from the supplier of custom-made products.” If your supplier is only delivering off-the-shelf product to you without any customization, this practice does not apply. However, if your supplier is providing custom-made or modifications to your specifications, then you need to decide which of these products are critical to your success and define those in the supplier’s contract or agreement. You have to state how and when you will be evaluating the products. Think of this practice as the acquirer performing PPQA work product audits of the supplier’s work products.

CMMI Practices for Documentation Teams

2009-12-01T05:09:00.000-08:00

Our organization is CMMI Maturity Level 3 Ver 1.2 certified. Our delivery teams are going to implement CMMI practices soon. I did see the processes of the organization and though all the roles appeared from project leader to developer to manager, except for documentation teams. In the same context, I am very curious to see if there are any set of practices to be followed for the documentation department in CMMI as all the processes at CMMI ML2, ML3 are specific to project management, engineering, support, organization areas.

Your question actually raises some other questions:

What are the roles and responsibilities of your documentation teams and documentation department?
If this group of people is responsible for the technical documentation (e.g. requirements, design, etc.) and the user documentation, how and why were they excluded from your ML 3 SCAMPI A?
How are the delivery teams different from the organization that achieved Maturity Level 3?

What is puzzling with your question is that REQM, RD, TS, PI, and CM cover the different aspects of writing and controlling the various documents associated with designing, developing, maintaining, operating, using, and deploying products. And then VER covers the inspection/review of the documents before placing them in the baseline and controlling them with CM.

It does appear from your description that your organization omitted the documentation people as a process role from your process documentation. In my opinion, at a minimum, your PPQA audits should have identified this omission long before your SCAMPI A appraisal. Then your Lead Appraiser should have identified this gap during the appraisal planning process before the SCAMPI A and should have taken steps to address the gap or postponed your appraisal until the documentation group was included in the scope. Since your documentation group was apparently not included in the scope of your appraisal, this oversight also calls into question your Lead Appraiser’s credentials and quite possibly the validity of your SCAMPI A results.

The bottom line, in my opinion, and based on only what you stated, your documentation group should have been included in the scope of your ML 3 SCAMPI A. Even if all they do is Document Configuration Control (which would be covered under CM) or Document Quality Assurance (which would be covered under PPQA).

The answers to my above questions could provide additional information that would change my opinion.

Finding a Lead Appraiser

2009-11-29T03:52:00.000-08:00

My company needs a very inexpensive Lead Appraiser – where can I find one?

I hate to say this, but you are using the wrong criteria for contracting a Lead Appraiser. When it comes to the CMMI and Lead Appraisers, you get what you pay for. And in point of fact, if you go with the lowest price Lead Appraiser that you can find, it is quite possible that this person is either very inexperienced as a Lead Appraiser or may not have the necessary experience or background to provide credible services. The risk you run is that the SEI may not accept the appraisal results in this situation. Then you have spent the money and may need to repeat everything, which can be a very expensive proposition.

Much better criteria to use when considering to hire a Lead Appraiser include:

number of years experience as a Lead Appraiser
number of appraisals led (SCAMPI A, B, and C)
experience in working with small organizations
ability to interpret the CMMI to appropriately fit your organization's needs
recommendations/testamonials from clients who have worked with the Lead Appraiser

You should interview the Lead Appraiser. Prepare some questions and scenarios that apply to your organization and learn how they would interpret the CMMI in your context and what they would recommend.

The basic question comes down to why do you want to be appraised? Your internal costs for implementing the CMMI will far outweigh any external costs associated with an SEI-certified Lead Appraiser. You should be implementing the CMMI because there is some business value associated with the CMMI. Lead Appraisers are experts in process improvement. As an analogy, if you are going into the hospital for major surgery, do you go to the lowest price surgeon you can find (possibly someone with questionable credentials) or do you go to a surgeon that has a reputation for quality and success?

Think of it this way. Selecting a Lead Appraiser is a very important decision. There is certainly cost factors as well as risks to consider, and possibly other considerations as well. Therefore, I suggest that you look at the Decision Analysis and Resolution (DAR) Process Area and use the practices described there to select your CMMI services provider.

Advice on Process Performance Models

2009-11-21T16:08:00.000-08:00

We have developed a Process Performance Model (PPM) for system testing defects. In our model we analyzed multiple factors (independent variables) that may influence the number of system test defects. However, the strange part is that the regression analysis has eliminated all of the variables except one, testing effort. Some of the other variables included things like number of test cases, tester's experience, and size (LOC).

Can a valid PPM only have one independent variable? The p-value is less than 0.05, and the R squared value is 0.8, so it seems that it would be reliable. But only one independent variable seems rather limited and weak to me, but I can't think of a justifiable reason why it is invalid considering we have analyzed data from several independent variables and only one was found to be statistically significant.

What do you think?

It does sound a bit odd. Perhaps you have overlooked some other X Factor that you should be evaluating. But it is conceivable to have a model with only one independent variable.

Basically your model looks like Y = mX +b. But given all of the PPMs that I have seen, they all contain multiple X factors.

Several things could be going on here.

1. The regression analysis was faulty

2. The list of X factors evaluated was incomplete

3. One or more of the X factors were a combination of factors that should have been separately evaluated

4. Not enough data were evaluated for the regression analysis to yield proper answers

5. The distribution of the data is not a normal distribution (Gaussian distribution aka Bell Shaped Curve) and analytic techniques for a normal distribution were mistakenly used

You probably should have a statistical expert independently examine your analysis to see if the resulting PPM is in fact a linear relationship between X and Y.

The other option is to use the PPM for several months of analysis and see how accurate it is. If the PPM yields accurate predictions, then it is most likely a good PPM.

One thing that can happen if you have not been careful with storing the raw data and/or not collecting at the proper level of detail is that factors can be inadvertently combined and thus cancel each other out thereby yielding a factor that on the surface looks reasonable, but may in fact not be. For example if one element is high and the other one it is combined with low, then the result will be somewhere in between. Think of two out of phase sine waves. Each one by itself looks like a sine wave, but the combination is something else entirely.

So as an example, suppose you are collecting defect data (requirements defects, input defects, design defects, coding defects, test case defects, etc.) and you have neglected to use these categories to count and distinguish the different types of defects. Then the defect count probably won’t correlate with anything because the individual categories are cancelling each other out. This is the kind of data behavior that caused many of the early HM adopters to throw away their historical data and start over from scratch. So it is very important to follow the Measurement and Analysis PA to properly define all of the base measures and to properly store them so when it comes time to perform the statistical analysis of the data, there is sufficient information stored to allow different types of analyses to be performed to isolate the specific information of interest.

Hope this explanation helps.

Integrated Project Management (IPM) - SP 2.2

2009-10-22T04:58:00.000-07:00

Could you please provide some examples of evidence for IPM Specific Practice 2.2 and some examples of a critical dependency?

Integrated Project Management (IPM) Specific Practice 2.2 states “Participate with relevant stakeholders to identify, negotiate, and track critical dependencies.” This practice is one of many compound practices in the CMMI. Therefore, to completely cover the practice, the organization would have to provide evidence that the project is participating with the relevant stakeholders. And that participation would be to identify critical dependencies, negotiate critical dependencies, and track critical dependencies. So the type of Direct Evidence that I would expect to see as a Lead Appraiser or Appraisal Team Member from the organization would be a list of critical dependencies, issues, and action items that were the result of the participation and for Indirect Evidence I would expect to see the meeting minutes or review/inspection reports that resulted in the list provided as Direct Evidence.

Now, if you read the Tips and Hints in the CMMI for IPM SP 2.2, these will provide insight into what a critical dependency is. Each project will have some constraints such as time, quality, budget, etc. that it must meet. By applying these constraints to the project schedule the Project Manager will be able to define the critical path through the project tasks and activities. The critical path contains all of the tasks and activities that have a direct impact on meeting the scheduled end date for the project. If a task on the critical path slips or runs over, then the end date of the project has a corresponding slip or overrun. Tasks that are not on the critical path can move about a bit before they impact the end date. The project tasks and activities on the critical path define the critical dependencies for the project. For example, if both Task A and Task B are on the critical path and Task B cannot begin before Task A completes, that is one critical dependency. Another example of a critical dependency is an external group is supplying a work product to the project and a Task cannot begin until the work product is received and accepted. This type of critical dependency may also be called a critical deliverable.

Hope this explanation helps.

Bi-directional Traceability Question

2009-10-19T04:53:00.000-07:00

Would you please explain to me the meaning of bi-directional traceability? If I am not wrong, Horizontal Traceability indicates mapping of a requirement across all the SDLC phases like Requirement->Design->Construction->Testing. But I do not get what is meant by Vertical Traceability. Would you please educate me on this?

Vertical traceability is top-down bottom-up traceability. Vertical traceability is the most common type of traceability. Basically it is tracing from the highest level requirements all the way down to the product component level (could be even to a specific line of code). And then starting at the lowest level and tracing all the way back up to the top. Horizontal traceability is tracing within a document or between peer documents. What you have listed as tracing from requirements to design to construction to test is actually vertical traceability. And when you come right down to it, it doesn’t really matter what type of traceability you label it. All that matters is that you can trace forward from a starting point to an end point and vice versa. The level and complexity of the traceability is determined by the criticality of the product and project. For a program like the Space Shuttle or Space Station, you would need very rigorous requirements traceability. Whereas a small application or a product would only need some simple traceability.

REQM - SP1.1 Query

2009-10-17T16:33:00.000-07:00

In REQM – SP1.1 calls for “Criteria for evaluation and acceptance.” Can you tell me what this expected “criteria” would be?

I suggest looking at Requirements Management (REQM) Process Area in your copy of the CMMI. On page 490 in REQM Sub-practice 2 there is a list of example evaluation and acceptance criteria.

REQM does not expect the organization to have evaluation and acceptance criteria. What you are expected to provide is evidence that support the required components (Specific Goal 1 in this case; evidence that demonstrates that your requirements are managed and inconsistencies with project plans and work products are identified) and evidence that supports the expected components (Specific Practice 1.1 in this case; evidence that demonstrates that you have developed an understanding with the requirements providers on the meaning of the requirements). If your Lead Appraiser is telling you that you have to provide these criteria, then he or she should be challenged as to why they are telling you that you need to provide this evidence.

The evaluation and acceptance criteria are one way of developing this understanding. Please bear in mind the section title is TYPICAL Work Products, not EXPECTED Work Products, or REQUIRED Work Products. The list of Typical Work Products is merely a list of example work products.

Selection of Suitable Lifecycle Processes

2009-10-16T18:37:00.000-07:00

Selection of a suitable lifecycle for the project/product is more of a technical issue. Then why is it covered in Integrated Project Management (IPM) rather than Technical Solution (TS)?

Selecting a suitable lifecycle is a function of the project requirements and the application domain. The selection is something that should be performed when planning the project activities, not while performing the actual software engineering activities. Selecting a lifecycle model and defining the project’s lifecycle phases are necessary for planning and estimating the project. That is why lifecycles appear in Project Planning (SP 1.3) and Integrated Project Management (sub-practice 1 of SP 1.1) . The selection of the lifecycle is a project management activity.

From another perspective, the purpose of Technical Solution is to design, develop, and implement solutions to requirements. And selecting an appropriate lifecycle model is an activity that needs to be done before you can design, develop, and implement a solution to the requirements.

Process Performance Model and QPM Inquiry

2009-10-16T18:07:00.000-07:00

We are a CMMI Maturity Level 5 organization and now we are preparing for our re-appraisal.
We have Process Performance Models (PPMs) with different X- prameters and good R^2, R (adjusted) & P(value).

We perform simulations to calculate the probability of achieving our organizational goal and the determination that the probability % is good enough to achieve the goal.

Now we are in our process of implementing the PPM. My question is: Shall we use the PPM to estimate the X- parameters in my projetct? Or what should I do with this model after that ?

Please advise.

I hate to say this, and I could be wrong, but reading your question it sounds like you are asking what do I need in order to be Maturity Level 5 (ML 5)? Is this equation sufficient? And by asking what do I do with the PPM, I may be misunderstanding you, but if you were really at ML 5 you wouldn’t be asking this question. I get the impression from your question that your organization doesn’t know why it is using a specific technique for the PPMs.

The best solution to your question is to ask your High Maturity Lead Appraiser for help and guidance as he or she should have enough understanding and knowledge of your organization to provide the answer.

But once again, the nature of your question gives me the uneasy feeling that your organization may not even be Maturity Level 4.

Applicability of SAM

2009-08-13T07:14:00.001-07:00

Does Supplier Agreement Management (SAM) apply to the supplier? Suppose you're an organization that is supplying to another organization , for example to develop a software package. This organization is not using suppliers. I assume that SAM is not applicable to this organization. Is my assumption correct?

Your assumption is correct. But allow me to restate your scenario to avoid any confusion by using the word organization mulitple times in one sentence. Organization A develops a software package and delivers it to Organization B. Organization A does not use any suppliers or outsource any work. In this case Organization B is the acquiring organization and Organization A is the supplier organization. Therefore, SAM would apply to Organization B, but not to Organization A.

Rationale for Maturity Level 5

2009-08-13T07:07:00.000-07:00

Our organization is currently at Maturity Level 3 and is now planning to go for Maturity Level 5. Our CEO is asking for the short term and long term benefits and challenges for implementing Maturity Level 5. How do we provide this information?

The question that needs to be asked is why does your organization want to achieve Maturity Level 5 (ML 5)? If the organization has already achieved Maturity Level 3 (ML 3), what is the motivation for achieveing ML 5? Most likely it is not mandated in a contract or by its customers. But achieving ML 5 is desirable to be competitive in the marketplace. Therefore, the reasons for achieving ML 5 should provide some indication of the short term and long term benefits.

The first step in becoming a High Maturity organization is defining your Quality and Process Performance Objectives (QPPOs) that are based on your business goals and objectives as well as your customer needs. The QPPOs should be stated in a form such that they specify a timeframe. And that information will provide some ideas of the short term and long term benefits.

However, the best advice that I can provide is to hire an SEI-certified High Maturity Lead Appraiser (HMLA) who will work with you to help explain the benefits of ML 5 to the CEO. The HMLA should have experience working with many different organizations at various Maturity Levels and be able to talk about the different challanges that have been faced by other organizations, as well as the challenges and risks within your organization.

At this point, it sounds like your organization is just beginning its journey to ML 5, so I would have to be convinced that your processes are stable enough to provide the data needed to quantify any short term and long term benefits. I think that the best you could do at this point is communicate this information in qualitative terms. Any quantitative information may not be accurate until you have implemented High Maturity.

The Right Model to Follow

2009-08-10T14:31:00.000-07:00

Our operations involve providing reporting solutions & implementing them based on tools like SAP BOBJ, MSTR, MS, ORCL primarily setting up datamarts & ETL processes and also providing technical support to customers as a line of business (Java/.net support for third party tools based on bugs/additional requirements given by customers at random)

Question 1: For such an organization , would it be right to follow CMMI for Development or CMMI for Services or both?
Question 2: Are appraisals made on CMMI V1.3 or are we still in CMMI V1.2. If still on 1.2, when are we likely to move to V1.3?
Question 3: If only a few projects or one division of an organization is appraised, will the appraisal rating stand for the company?

Question 1 – This question is difficult to answer based upon your description. Are you developing a product or delivering a service? I really cannot tell from your description, it could be either or both. If you are developing a product, then the CMMI-DEV might be applicable. If you are delivering a service, then the CMMI-SVC might be applicable. And you could blend the two. Another aspect to consider is are you planning to have a SCAMPI A appraisal? If so, then your Lead Appraiser will be able to help you decide which constellation is applicable to your organization. If you are not planning on being appraised, then I would suggest that you look at both constellations and use the Process Areas that apply best to your organization.
Question 2 – CMMI v1.2 is the current version. The SEI recently announced that v1.3 will be released in November 2010. And based on past experience, the SEI will most likely allow appraisals against v1.2 to be conducted up until October 31, 2011.
Question 3 – The short answer is no. The appraisal results ONLY apply to the organization that was appraised, not the entire company.

Examples of Applying CMMI or CMMI-SVC to Government Organizations

2009-08-10T14:27:00.000-07:00

I'm looking for cases where CMMI or CMMI-SVC have been applied (successfully or otherwise) within a government organization. I'm trying to identify lessons learned that would be useful to a US federal government department that has considered applying CMMI practices for process improvement.

There have been multiple US Government and Military organizations that have implemented and been appraised to the CMMI. Just look at the SEI's list of appraisal results http://sas.sei.cmu.edu/pars/pars.aspx and you will find these organizations. As far as the CMMI-SVC goes, this constellation was only released in late Feb 2009 and organizations are just beginning to use the model. So I seriously doubt that there are any examples of use available at this point in time. My guess is that the first time anyone will see examples of the CMMI-SVC being used will be at the 2010 SEPG Conference in Savannah, Georgia March 2010.

Auditor Directing a PPM

2009-08-10T14:19:00.000-07:00

We are trying for CMMI ML 5 Ver 1.2, and our auditor has asked us to come up with a PPM for predicting the outcome of CAR and OID. Can you help me on how to go about it?

When you say auditor, I assume that you mean your Lead Appraiser (LA) is asking you for a PPM for predicting the outcome of Causal Analysis and Resolution (CAR) and Organizational Innovation and Deployment (OID). Is your Lead Appraiser an SEI-certified High Maturity Lead Appraiser? Has your organization identified the need for a Process Performance Model (PPM) to predict the CAR and OID outcomes, or is this solely a request from your LA? Your LA is not the person to tell you which PPMs you need. Do you have Process and Product Quality Objectives (QPPOs) that require PPM(s) to predict CAR and OID outcomes? If the answer is no, then you don't need a PPM for CAR and OID.

What I find odd is that you do not mention a Process Performance Baseline (PPB) for CAR and OID. If you are going to define and develop a PPM, then you really need to develop the CAR and OID PPBs first before you can determine the PPMs. From your brief description, it sounds like your Lead Appraiser may have overstepped his boundaries in asking for the CAR and OID PPM.

Identifying Risks

2009-08-07T09:45:00.000-07:00

What is the difference between PP SP 2.2 Identify Project Risk and RSKM SP 2.1 Identify Risks?

What you are asking about is one of the basic differences between Maturity Level 2 (ML 2) and Maturity Level 3 (ML 3). Project Planning (PP) is a ML 2 Process Area (PA) and Risk Management (RSKM) is a ML 3 PA. At ML 2, the project only needs to be able to identify risks and that is what PP Specific Practice (SP) 2.2 addresses. At ML 3, RSKM builds upon the foundation of identifying and tracking risks put in place by PP and Project Monitoring and Control (PMC). RSKM SP 2.1 therefore builds upon PP SP 2.2 by adding more rigor for risk identification. Just read the informative material and sub-practices for both SPs and you will immediately see and understand the difference.

Project Specific Findings

2009-08-07T09:40:00.000-07:00

While reading the SCAMPI A Method Definition Document (MDD), I discovered that providing project specific findings is an option that can be requested by the project sponsor. I always believed that the Findings Presentation could only include findings at organizational unit level. So if the sponsor requests project specific findigs as part of the appraisal output, how are they communicated? Included in the final findings? If this is the case, what about the non-attribution of findings? Or via a separate document?

The SCAMPI Method does indeed allow for project specific findings if the Appraisal Sponsor requests them. If requested, then project specific findings could be communicated in the Final Findings Presentation along with the other information, or they could be communicated separately. Reporting will be negotiated and documented in the Appraisal Plan. Keep in mind that even if the Appraisal Sponsor doesn’t request project specific findings, the PIIDs will contain project specific observations, so if someone were interested in finding out about project specific information, all they would have to do would be to read the PIIDs. The organization is required to retain the PIIDs used for the appraisal for three years, the same length of time that the appraisal results are valid.

The non-attribution issue concerns identifying any individual, project, or group as the SOURCE of the information. So by reporting project specific findings, the Lead Appraiser and Appraisal Team are not violating the non-attribution rules unless they reveal the source of the finding(s). When I conduct an appraisal, the final Appraisal Team activity is to scrub the PIIDs of all attribution information (names, interview sessions, etc.) and deleting all previous versions so the organization only retains the scrubbed PIIDs.

Selecting Projects for a SCAMPI Appraisal

2009-08-05T10:13:00.000-07:00

One of my client organisations works on hardware and software design projects related to locomotive design for a manufacturer. They want to adopt the CMMI-DEV and scope their SCAMPI appraisals only for their software projects. Both, hardware and software design projects are undertaken by the same organisation, under the same management, same company name and at the same location. Is it be proper for the organization to exclude the hardware projects from their CMMI journey and appraisals? I feel it is not proper and it violates the principle of institutionalization of processes across the organization. Moreover, if the Lead Appraiser agrees to conduct an appraisal for software projects only, he or she will violate the principle of randomly selecting the projects for the appraisal.

The way I look at this situation depends upon on how the company is organized. If there are separate hardware and software development groups, departments, or divisions that deliver products to a program office (for example), then the software group could be appraised on its own. The same could be true for the hardware group. In fact, this situation occurs quite frequently in my experience here in the United States.

However, if the hardware and software groups are tightly intertwined in building and delivering a product (meaning you cannot separate the two), then I would say that both the hardware and software groups had to be appraised together.

The correct decision requires the Lead Appraiser to have a very good understanding of the organization and its business, as well as being a function of how the organization defines a project and what its process documentation states.

CMMI Documentation Request

2009-08-04T07:38:00.000-07:00

I have read your blog. Can you provide me some documented CMMI processes (Draft)? I would like to learn and implement the process. I have good experience using ISO 9001 for the past 10 years.

There is no such thing as a set of documented CMMI processes that can be provided for a company, though some unscrupulous consultants will try to sell you a set. Rather, there are industry accepted standards for documenting processes, such as ETVX. These process documentation standards are easily available by searching on the internet. The actual processes for REQM , PP, PMC, SAM, MA, PPQA, CM, etc. are a function of the organization’s business and its practices, though there is some commonality across companies and organizations. However, this commonality exists at the CMMI level. The CMMI is a set of guidelines for process improvement. The implementation of these guidelines will differ from organization to organization. The best advice that I can give you is to document all of your current business practices using an industry accepted process documentation standard, avoiding the temptation to improve the process. By simply documenting your existing processes, you will discover opportunities for improvement, but don’t make them at this point. Just document the process as it is currently practiced. Once you have all of your processes documented, then compare the results to the CMMI, add the missing practices, and address any improvement opportunities. Then you will have a set of processes that your employees will have ownership of and will also comply with the CMMI.

Estimating Cost vs. Establishing the Project's Budget

2009-08-04T07:15:00.000-07:00

What is the difference between Estimation of Project Cost (PP SP 1.4) and Establishing Project Budget (PP SP 2.1)?

Project Planning (PP) Specific Practice (SP) 1.4 addresses estimating the effort and cost for the project’s work products and tasks based on specified estimation rationale, models, and/or historical data. SP 2.1 takes this information for the individual work products and tasks, along with the major milestones identified in the project’s defined life cycle, any scheduling assumptions and constraints, and task predecessor/successor relationships to construct the project’s schedule and overall budget. Simply put, SP 1.4 concerns individual items in the project and SP 2.1 concerns the project as a whole.

Requirements Traceability, To What Level?

2009-08-04T07:09:00.000-07:00

It certainly makes sense to include relations between requirements and source code in the requirements traceability matrix. If requirements change, we've a direct view on the impact on source level. But can this be managed and maintained, even with a tool? Is it wortwhile to put a huge effort in maintaining relations between requirements and the source code? Isn't it sufficient to define these relations at the level of design elements and user acceptance test cases? In this case, we've a means to verify that each requirement is covered by design and test. I believe this strongly reduces the risk of uncovered requirements in the final delivery, which is one of the main purposes of requirements traceability.

The answer is, you do whatever is necessary to meet your business goals and objectives. It depends upon the criticality of the product or service you are delivering. If your product is highly complex and someone could lose their life if there was a missed requirement, then it is necessary to put a huge amount of effort into requirements traceability. Just consider the Space Shuttle program, the amount of requirements etc. The Space Shuttle software is as close to zero defects as you will ever find. And at the other end of the spectrum, you would be justified limiting the amount of effort for traceability.

PPQA or VER?

2009-08-03T12:48:00.000-07:00

I have a question. When people performa a review to assure than a coding standard is being used, is it considered a PPQA audit or a verification activity (VER)?

The correct answer is, it depends upon the nature of the review. If your documented software development process states that the coding standard is used to write code. Then a process audit of the software development process would be looking at the coding standard and determining if it was indeed being used by the developers. That would be a Process and Product Quality Assurance (PPQA) audit activity. If your documented verification process stated that a code peer review involves comparing the code to the coding standard, then that would be a Verification (VER) activity. And if your documented processes specified both of these conditions, then the answer to your question is both a PPQA audit activity and a VER activity. How you view the code review against the coding standard is therefore context dependant.

If you are asking this question because you are preparing your Direct and Indirect Evidence for your PIIDs and a SCAMPI A appraisal, then you will need to explain the context so the appraisal team will be able to correctly evaluate the evidence.

Software Sizing

2009-08-03T12:42:00.000-07:00

We are trying to achieve CMMI Maturity Level (ML) 3 in my company and we have decided to skip ML 2. So, now, one of our problems is related to software size estimation. We defined a proprietary method, based on Use Case Points and Function Points, but the practitioners are struggling with it. From your experience, what other methods have you seen or implemented in the companies with this same problem? Or, if a proprietary method was defined, what were the main aspects to take in account?

I would strongly urge you to forget the ML 3 Process Areas until you have mastered ML 2. There is a fundamental difference between how a ML 2 Project Manager approaches Project Planning (PP) and Project Monitoring and Control (PMC) vs. a ML 3 Project Manager. Estimation being one of the differences. Use Case Points and Functions Points are fairly sophisticated concepts and there are challenges with getting consistency in determining what each of these things are. I would recommend that you take a step back from the model and the projects and look at your historical project data. Use the actual effort, costs, etc. from previous projects to estimate a new project. Forget about Use Case Points and Function Points for now. Once you have mastered being able to use historical information to build an empirical estimation model, then it might make sense to add a layer of sophistication by considering Use Case Points or Function Points.

Another recommendation is let the Project Manager create the project estimates and then review them with the practitioners as a sanity check rather than ask the practitioners to create the estimates. Over time as the organization gains experience estimating projects etc., then it makes sense to involve the practitioners up front in the estimation process. You have to learn to crawl first with estimation before you can sprint with the big boys.

Application of the CMMI for Services

2009-07-31T08:24:00.000-07:00

I am in the process of preparing a "justification" presentation that talks about why the CMMI is important and how it can be applied. Though I have enough material and data to substantiate this, I don't have data that is contextual. I am looking for data, links, or any input that would point me to where the CMMI was applied to a Staff Augmentation organization or a comparison where I can see how each CMMI constellation (DEV, ACQ, and SVC) can be applied (Development Orgs, Support and Maintenance Org, Outsourcing Org, Staff Augmentation Org).

Though I could be wrong here, but I seriously doubt that the kind of information you seek exists at this point in time. Keep in mind that the CMMI for Services (CMMI-SVC) was only released in late February 2009 and the first appraisals to this constellation cannot occur at least for another 2 or 3 months. So there won’t be any anecdotal evidence or case studies available to provide the information you seek. I would imagine that the first opportunity to see any information of the type will be at the 2010 North American SEPG Conference. And since you are asking about applying the CMMI to a Staff Augmentation organization, the applicable CMMI constellation would the CMMI-SVC. There just isn’t a lot of information about its benefits right now, not enough time has elapsed since its release.

Is There a Difference Between Software and Firmware Development wrt the CMMI?

2009-07-31T08:19:00.000-07:00

Would you kindly let me know the difference between the development of software and firmware and what CMMI practices are useful for firmware development? Is there a need to have Configuration plan for firmware development projects to control the code version?

Though the location of where the software or firmware resides is different as well as the code, there is no difference in the development practices and activities that occur. The CMMI for Development (CMMI-DEV) applies equally well to any type of product development. All projects have requirements, the work needs to be planned and managed, the work products and deliverables need to be managed, designs have to be developed, etc. The specifics in the processes most likely will vary from hardware development to software development to firmware development, but all of the Process Areas (Engineering, Support, Project Management, and Process Management) apply.

Product Planning and Configuration

2009-07-30T08:20:00.000-07:00

It is common knowledge that Maturity Level 2 is project specific, and still I find at times Lead Appraisers asking funny questions during SCAMPI A appraisals. Quite recently, one of my friends told me that his Lead Appraiser is looking for planning at the product level, as well as Configuratuion Management at the product level. I was a bit amazed, thinking that Maturity Level 2 focuses on Project Planning, not product planning. What do you think about this situation? Have you been faced with this situation before? Is there a workaround for it?

From what you describe, it sounds like this Lead Appraiser could be misinterpreting the CMMI and possibly misleading the organization. The CMMI is quite clear that the Project Planning (PP) Process Area (PA) is for project planning purposes, not product planning.

"The purpose of PP is to establish and maintain plans that define project activities."

However, sometimes the difference between project and product can be blurred. By not knowing the context of the situation you described, the Lead Appraiser may have been trying a different approach to draw project planning information out in the interview sessions.

In one respect, it really doesn’t matter the line of questioning in a SCAMPI interview session. The Lead Appraiser could really ask about any topic. However, once he or she starts deviating from the CMMI, they are on shaky ground and could lose credibility. What does matter however, is the set of findings produced by the Lead Appraiser and the Appraisal Team. If there are findings associated with product planning that cannot be tied to the satisfaction of a CMMI Specific Goal or a Specific Practice, then these would be non-model findings and should have no impact on the resulting appraisal rating. However, if these non-model findings do impact the appraisal rating and the Lead Appraiser and Appraisal Team fail to demonstrate the linkage to Goal and Practice satisfaction/implementation, then the Lead Appraiser has not correctly performed his or her Lead Appraiser duties and the SEI should be informed about this issue so it can be investigated.

Why SAM is excluded ?

2009-07-30T08:09:00.000-07:00

Why is SAM removed from CMMI L3?

Supplier Agreement Management (SAM) is not excluded from the CMMI-DEV or CMMI-SVC. When you say L3, I assume you mean Maturity Level 3 and SAM is definitely NOT excluded from ML 3. If, however, the Lead Appraiser in working with the organization determines that SAM is not applicable to the work performed by the organization, SAM will be considered Not Applicable to the scope of the appraisal. And that could be at any Maturity Level. Please read previous my posts regarding SAM for more information. http://ppqc.blogspot.com/2009/04/excluding-supplier-agreement-management.html and http://ppqc.blogspot.com/2009/07/cmmi-novice-question.html

Criteria For Selecting and Qualifying an Instructor

2009-07-29T07:05:00.000-07:00

What are the important criteria that we should use to qualify and select internal and external instructors?

First of all you need to specify what kind of instructor you want to select and qualify, as well as the topic(s) you wish this person to teach. Then you would need to identify the core competencies for instructors in your organization. Coincidentally, I just created a job description for a Trainer position at my client based on some core competencies I found by searching the Internet. Here is a list of candidate competencies for your consideration:

Establishes and maintains instructor credibility

Demonstrates content expertise
States the learning objectives
Provides a model of professional and inter-personal behavior

Demonstrates effective communication skills

Uses appropriate verbal and non-verbal language
Uses frames of reference, language, and technical terms familiar to the students
Demonstrates that students have understood the message

Demonstrates effective presentation skills

Uses appropriate modulation of voice to keep students engaged and interested
Makes eye contact with individual students in the class
Uses appropriate gestures to communicate
Make effective use of moving around the room while facilitating discussions
Uses anecdotes, stories, analogies, real world examples, and humor effectively to illustrate key concepts
Uses relevant and short “war stories”
Encourages students to share their experiences

Demonstrates effective questioning skills

Asks students open ended questions
Appropriately directs questions to students
Uses active listening techniques to draw information from students
Paraphrases student questions to ensure that everyone understands

Responds appropriately to student’s need for clarification or feedback

Identifies students with clarification and feedback needs
Provides prompt, timely, and specific feedback focused on the performance, not the person
Welcomes questions from the students

Provides positive reinforcement and motivational incentives

Matches learning outcomes to student needs and goals
Determines if the student has mastered the material by the end of the class
Uses introductory activities to develop learning motivation
Makes the learning experience relevant and fun
Uses feedback and reinforcement during instruction

Uses instructional methods appropriately

Demonstrates proficiency with a variety of teaching methods
Limits the amount of lecture time
Uses demonstrations
Uses group and individual exercises
Facilitates group discussions
Uses instructional techniques that fit the situation
Encourages group dynamics associated with the media selected
Uses slides or other media as an outline for class discussion rather than lecture

Uses media effectively

Demonstrates proficiency with the training equipment (projector, laptop, hardware, etc.)
Troubleshoots problems associated with training equipment
Makes effective use of flip charts, presentation slides, hardware, etc.
Faces the audience when presenting and avoids reading the slides

Demonstrates understanding of the principles of adult learning

Makes the training topics relevant to the student’s needs
Makes the students partners in the learning experience

Need Some Clarification About Process Improvement

2009-07-29T06:58:00.000-07:00

What types of activities should be considered process improvement? If we modify the templates (any major changes), do we categorize these changes as process improvement?

Merely modifying a template may or may not constitute process improvement. Your template may have changed because of external reasons (your customer wants you to use a different template) that have nothing to do with process improvement. You should explain the rationale for modification, then you would have a stronger case to demonstrate process improvement.

Process improvement suggestions can come from any number of sources:

Appraisal findings
PPQA audit findings
Lessons learned
Employee suggestions

If you couple the source of the process improvement suggestion with the actual change, usually spelled out in the Process Improvement Plan, then you have the information you need.

Process Improvement Measures

2009-07-28T06:30:00.000-07:00

Do you have any suggestions for a size and quality measure for a Process Improvement organization?

There are other questions that need to be answered first in order to provide a proper response to your question. What does your Process Improvement process say? Examine your process to see what type of activities you have documented, where are the spots in the process you can collect data, and what data you can collect. You also have to consider your business goals and objectives for your Process Improvement organization. Then you will have some idea of the measures you can use for size and quality of your Process Improvement effort. Experts can offer you all different ideas, but if these ideas don’t support your goals and/or your process is written in such a way that it doesn’t support these measures, then you will have to consider something else.

Measurement and Analysis vs. Generic Practice 2.8

2009-07-28T06:22:00.000-07:00

Would you expect that measures being reported for Generic Practice (GP) 2.8 on all Maturity Level (ML) 2 and ML 3 Process Areas (PAs) would follow the complete Measurement and Analysis (MA) PA? That is these GP 2.8 measures would have written objectives, operational definitions, storage, collection and analysis procedures in line with MA Specific Goal (SG) 1 and SG 2. Or can the PAs be monitored (measured) and controlled without the full breath of the MA PA?

This is a good question. So let’s take a step backwards and look at the CMMI and Generic Practice – PA Relationships. The summary table in the Generic Practice section of the model clearly states that Project Monitoring and Control (PMC) can implement GP 2.8 for all project-related processes. And MA provides general guidance about measuring, analyzing, and recording information that can be used in establishing measures for monitoring actual performance of the process. Please note that this information is GUIDANCE and part of the INFORMATIVE material. Therefore, it is not required that the org use MA for GP 2.8. HOWEVER, from a practical point of view, why would MA be part of the model if there wasn’t a requirement and expectation that it would be implemented? Since it is a ML 2 PA and you are asking about ML 3, as a Lead Appraiser I would expect to see that MA was used for defining, collecting, analyzing, and reporting both the project and process measures. Without implementing MA for the process measures, the org would be receiving little to no benefit from GP 2.8. And as I have seen GP 2.8 implemented, sometimes the process measures are embedded in the project measures that have been defined using MA.

Requirements Traceability Matrix Question

2009-07-27T06:24:00.000-07:00

Is Requirements Traceability Matrix (RTM) a configurable item (CI) or not? Do we need to maintain its version history?

This is a question you really have to answer for yourself by first addressing some more basic questions. How critical is the RTM to the success of your project, organization, and business? What do you use the RTM for? Tracing requirements? Verifying and validating requirements? Regression testing? What value to you, the project, and the organization is the change history of the RTM? If you identify strong business needs for the RTM, then the answer to original question will become obvious.

From my perspective, the RTM is not a CI per se as it is a tool for managing your requirements and not necessarily a product component. But it is a very important, if not essential tool, and maintaining its change history could be necessary for project and organizational success.

Appraisal Team Member Qualification Requirements

2009-07-27T06:09:00.000-07:00

What are the requirements to be an Appraisal Team Member (ATM) in SCAMPI A appraisal?

I have heard many opinions on ATM qualifications, such as:

Not part of process definition
Not involved with Process Implementation
Not being an SEPG member.

I am unaware of any criteria set by the SEI on this topic.

The ATM qualifications are spelled out in the SCAMPI Method Definition Document (MDD) Section 1.3.2 SELECT TEAM MEMBERS. I have extracted the pertinent text here.

Parameters and Limits
The minimum acceptable team size for a SCAMPI A appraisal is four people (including the appraisal team leader).
All team members must have previously completed the SEI-licensed Introduction to CMMI course.
With regard to engineering field experience, the team (as a group) must have an average of at least 6 years of experience, and the team total must be at least 25 years of experience in each of the disciplines to be covered in the appraisal.
With regard to management experience, the team (as a group) must have a total of at least 10 years of experience, and at least one team member must have at least 6 years of experience as a manager.
The team must, in aggregate, have representative experience in the lifecycles being appraised.

Optional Practices
Although not required in the Parameters and Limits section above, the following are considered recommended best practices and should be employed whenever feasible:

Each member should have good written and oral communication skills, the ability to facilitate the free flow of communication, and the ability to perform as team players and negotiate consensus.
At least half of the team members should have participated in a previous process appraisal.
Team members should be perceived by the appraisal sponsor as credible.

Additional appraisal team member selection considerations include

Consider the personal characteristics of individual team members (e.g., communication preferences and personality types) and how these characteristics may affect the dynamics of the team.
Use one or more authorized SCAMPI Lead Appraisers as team members.

And there is one more ATM requirement that is documented in the MDD errata. The Appraisal Sponsor cannot be an Appraisal Team Member.

Importance of the Sub-Practices

2009-07-26T12:26:00.000-07:00

I am trying to explain to a organization that they need to look at the sub-practice areas and make sure that the artifacts for the SCAMPI A also answer the subpractice areas. I understand that the subpractice area are a detailed description that provides guidance.

Has the organization been trained on the SEI’s 3-day Intro to CMMI class? The CMMI instructor should have explained and emphasized the role of the informative material (e. g., sub-practices). And in the words of Rusty Young, this material is “informative” NOT “ignorative.” Another way to look at the informative material is if it has no value to the model, there is no point in including it. Then the model would only consist of goal and practice statements, which would only take about 10 pages to document. The sub-practices are provided to help the reader understand the intent of the practice and goal statements.

HOWEVER, in a SCAMPI A appraisal the appraisal team will only be evaluating the required (goals) and expected (practices) components of the model, NOT the informative material (sub-practices et. al.) So you would be mistaken if you required the organization to provide evidence (Direct and Indirect) for the sub-practices. The organization only provides evidence for the goals and practices in a SCAMPI A.

CMMI Novice Question

2009-07-26T12:15:00.000-07:00

I would like to know if all the Maturity Level 2 Process Areas must be completed for a Maturity Level 2 appraisal? There are some process areas which are not applicable to our organization.

For the CMMI for Development (CMMI-DEV) there is only one Process Area (PA) that can be designated Not Applicable (N/A), SAM. Therefore, at a minimum, REQM, PP, PMC, MA, PPQA, and CM are required for a Maturity Level 2 (ML 2) appraisal. And if the organization has outsourced some work, then SAM is also applicable.

I find it hard to believe that you can state that one or more of these PAs are not applicable to your organization. Every project has requirements to manage from the janitor to the President. Everyone works on a project. You just have to define what a project is. And then you manage the project. Everyone can define specific measures that can be used to manage the project. Everyone has some sort of configuration items or documents that have to be managed. And everyone needs some sort of objective evaluation of the process and project compliance. For a small organization, you may have combined one or more of these PAs under one person. But that does not mean these PAs are not applicable.

Now if you said that you had problems with the engineering PAs (RD, TS, PI, VER, and VAL), then I would suggest that the CMMI-DEV may not be the appropriate model constellation for your use and you should look at the CMMI for Services (CMMI-SVC) or CMMI for Acquisition (CMMI-ACQ).

Reducing the Time it Takes to Review a Document

2009-07-24T09:18:00.000-07:00

I have problem. Currently we are taking 12 minutes to summarize articles that are published in our newsletter. Can you suggest how can we reduce the processing time to 8 minutes?

Without having any insight at all into your process, I cannot provide you a solution. However, if you want to shorten the process time, here are the steps I would follow:

First answer the question why you want to shorten the process time. Is this desire driven by your business goals and objectives and/or your process goals and objectives?
If the answer is yes, then define and document your process.
Define, collect, and analyze appropriate process measures that can be used to understand your process time.
Analyze the process to determine which step or steps are the major drivers for the current process time of 12 minutes.
Pilot different process step changes that could result in shortening your process time to 8 minutes and still meet your business goals and objectives and your process goals and objectives.
Based on this analysis, change your process.

Cut Off Time for Updating Documents Prior to a SCAMPI

2009-07-24T09:12:00.000-07:00

What is the normal time-frame an organization is allowed to continue making changes to their documentation prior to a SCAMPI?

There is no hard and fast rule for this practice. You need to work this time frame out with your Lead Appraiser to see what he or she is comfortable with. But, think about what you are asking for a minute. Are you talking about process changes? Changes to artifacts? Or both?

If you are talking about process changes, then you need to consider the purpose of the SCAMPI. One of the jobs of the appraisal team is to determine the amount of institutionalization. In order to determine the degree of institutionalization (GGs and GPs), changes to the processes and procedures need to be minimized so there is sufficient time for institutionalization and to collect and present the proper Direct and Indirect Evidence. To be on the safe side and mitigate this risk, organizations may decide to have no process changes for six months before the SCAMPI.

If you are talking about the artifacts, then you need to keep in mind the definition of Focus and Non-Focus Projects and work with your Lead Appraiser to determine your evidence needs. In my experience, my clients have taken the risk mitigation approach of “freezing” the evidence about a month before the Readiness Review to build the PIIDs and then only allow changes after that point if there are weaknesses in the PIIDs that need to be addressed before the SCAMPI.

Please keep in mind that I am not advocating freezing the processes 6 months before an appraisal, it just has been my experience that as a risk mitigation some clients have held off making changes until after their appraisal. This behavior is typical for a first time SCAMPI A in a risk averse organization who wants to do everything possible to have a successful SCAMPI A. After all the CMMI is a set of process improvement guidelines, so I as a Lead Appraiser would expect to see evidence of continuous process improvement. But the org has to take an intelligent approach when rolling out new changes. The workforce gets frustrated with chasing a moving target if the processes and assets are frequently changing, i.e. major updates.

Responsibilities of the Configuration Control Board (CCB)

2009-05-20T14:48:00.000-07:00

Would you please tell me the usual responsiblilities for a Configuration Control Board (CCB)? Does the project need to get an approval from CCB before creating a new baseline?

Basically the CCB is responsible for baselining specific configuration items: documents, designs, requirements, products, product components, etc. Once a configuration item has been baselined, the CCB is then responsible for reviewing and approving Change Requests to a baselined configuration item. If the CCB approves a change to a configuration item, after the item and other associated changes have been updated, then there is a new baseline version.

Levying CMMI Requirements on Your Suppliers

2009-05-20T14:44:00.000-07:00

I would like to know whether or not an acquirer can specify a CMMI Maturity Level (ML) as a requirement in a call for tenders or invitation to public bidding. What the ML is required? What is the domain ? and for what kind of projects?

The answer is yes. The acquirer can specify any requirements they want the vendors to meet. In my experience, I have seen acquirers specifying that the bidders be either at ML 2 or ML 3. What this means though is that the acquirer has done its homework and appropriately determined the necessary Maturity Level for the vendor to support the acquirer’s business and quality goals and objectives. There should be a good match between the ML of the acquirer and the ML of the vendor, to work well it may be best if both organizations are at the same ML. Otherwise, there can be problems.

As the acquirer, you probably would find benefit from implementing the CMMI for Acquisition (CMMI-ACQ). The CMMI-ACQ provides a lot of guidance for tenders and contracts that meet the acquirer’s needs.

Can the Same Firm Provide CMMI Consulting & SCAMPI A Services?

2009-05-20T14:31:00.000-07:00

I have been hearing that the SEI has stringent rules stating that the consulting firm cannot perform a SCAMPI A appraisal as it results in conflict of interest and the appraisal may lose it's objectivity. Couold you please throw light on this issue and give me the correct information? It would be great if you can refer to any documents listing these rules e.g. SEI audit policies.

Yes, the same COMPANY and same LEAD APPRAISER can provide both CMMI consulting and SCAMPI A appraisal services to the same client. However, the Lead Appraiser and SEI Partner must notify the SEI if there is a Conflict of Interest. The situation to avoid is the Lead Appraiser appraising his or her own work in a SCAMPI A appraisal. The Code of Professional Conduct (CoPC) spells out the policy and the steps that must be taken if there is an indentified Conflict of Interest. http://www.sei.cmu.edu/partners/copc.html

The primary issue concerns the type of consulting provided by the Lead Appraiser. If the consulting consists of writing processes and process assets for the client, then there is a clear Conflict of Interest. But if the consulting consists of training on different ways to implement a Process Area, reviewing documents and identifying strengths and weaknesses, reviewing evidence for inclusion in the PIIDs, etc. then there is no Conflict of Interest.

So most Lead Appraisers will try their best to remain ethical and either just provide the hands-on consulting and have another Lead Appraiser lead the SCAMPI A appraisal OR lead the SCAMPI A appraisal and help the organization get prepared for the appraisal.

Procedures Specify HOW to Do Something, Not WHAT to Do

2009-04-22T11:02:00.000-07:00

A common problem I notice when reviewing an organization's processes and procedures is that the organization doesn't understand how to document a procedure. Rather than simply telling someone HOW to perform the process, the document states a set of requirements that must be met by the people performing the procedure, which then allows multiple ways of performing the procedure by any person attempting to use the document. There seems to be a general misunderstanding that commands or directives tell someone HOW to perform a process, when in actuality all that is being communicated is WHAT to do.

The CMMI Glossary defines process as activities that can be recognized as implementations of practices in a CMMI model. These activities can be mapped to one or more practices in CMMI process areas to allow a model to be useful for process improvement and process appraisal.

The Glossary also defines a process element as the fundamental unit of a process. A process can be defined in terms of sub-processes and/or process elements. A sub-process can be further decomposed into sub-processes and/or process elements; a process element cannot. Each process element covers a closely related set of activities (e.g. estimating element, peer review element). Process elements can be portrayed using templates to be completed, abstractions to be refined, or descriptions to be modified or used. A process element can be an activity or task.

Page 53 of the CMMI book provides the following description of Maturity Level 2:

At Maturity Level 2, the projects of the organization have ensured that processes are planned and executed in accordance with policy; the projects employ skilled people who have adequate resources to produce controlled outputs; involve relevant stakeholders; are monitored, controlled, and reviewed; and are evaluated for adherence to their process descriptions. The process discipline reflected by Maturity Level 2 helps to ensure that existing practices are retained during times of stress. When these practices are in place, projects are performed and managed according to their documented plans.

At Maturity Level 2, the status of work products and the delivery of services are visible to management at defined points (e.g., at major milestones and at the completion of major tasks). Commitments are established among relevant stakeholders and are revised as needed. Work products are appropriately controlled. The work products and services satisfy their specified process descriptions, standards, and procedures.

Page 54 of the CMMI book provides the following description of Maturity Level 3:

At Maturity Level 3, processes are well characterized and understood, and are described in standards, procedures, tools, and methods. The organization’s set of standard processes, which is the basis for Maturity Level 3, is established and improved over time. These standard processes are used to establish consistency across the organization. Projects establish their defined processes by tailoring the organization’s set of standard processes according to tailoring guidelines.

A critical distinction between Maturity Levels 2 and 3 is the scope of standards, process descriptions, and procedures. At Maturity Level 2, the standards, process descriptions, and procedures may be quite different in each specific instance of the process (e.g., on a particular project). At Maturity Level 3, the standards, process descriptions, and procedures for a project are tailored from the organization’s set of standard processes to suit a particular project or organizational unit and therefore are more consistent, except for differences allowed by the tailoring guidelines.

Another critical distinction is that at Maturity Level 3, processes are typically described more rigorously than at Maturity Level 2. A defined process clearly states the purpose, inputs, entry criteria, activities, roles, measures, verification steps, outputs, and exit criteria. At Maturity Level 3, processes are managed more proactively using an understanding of the interrelationships of the process activities and detailed measures of the process, its work products, and its services.

Page 295 in Organizational Process Definition provides some Tips that relate to processes and process assets:

Organizational process assets support a fundamental change in behavior. Projects no longer create their processes from scratch but instead use the best practices of the organization, thus improving quality and saving time and money.

Standard processes define the key activities performed in an organization. Some examples of standard processes include requirements elicitation, design, and testing; planning, estimating, monitoring, and control; and product delivery and support.

The objective is to decompose and define the process so that it can be performed consistently across projects but will allow enough flexibility to meet the unique requirements of each project.

And from the book Interpreting the CMMI by Margaret Kulpa and Kent Johnson, second edition on page 198 there is a section on Defining Procedures that supports the CMMI material quoted above. In the context of the CMMI, procedures are equivalent to sub-processes and process elements.

Procedures are step-by-step instructions on how to perform a task. To be repeatable, the steps need to be broken down to a level that anyone who needs to perform the task, with a general understanding of the work to be done, can perform the work adequately by following the instructions. Procedures are a subset of processes. The process is what to do; the procedures are how to do the steps of the process.

Procedures are step-by-step instructions of how your processes are performed. They include:

Sequence of activities
Deliverables
Controls
Inspections and reviews
Guidelines and standards used

In addition, in the CMMI book on pages 99 and 100, Bill Curtis, an author of the Software CMM, talks about policies, PPQA, and process improvement:

Polices that merely regurgitate goals from CMMI process areas represent a lost opportunity for executives to communicate their expectations for behavior in their organizations. Once policies are established, executives need visibility into compliance. Assurance groups have influence only to the extent that executives attend to their reports and address noncompliance. However, the greatest value of assurance groups, and this is subtle in Process and Product Quality Assurance (PPQA), is when they serve as mentors to project managers and technical staff on practices that support compliance. Consequently, assurance groups need to be staffed with competent developers and managers so that they are credible in transferring knowledge of best practices across the organization.

Process improvement must be conducted as a project. Executives must assign responsibility for managing the project, provide funding and resources, expect periodic status reports, and measure results. The person assigned to lead the improvement project must be a good role model for other project managers. Executives should ask frequent questions about project plans and the assumptions underlying them. The guidebooks, defined processes, measures, checklists, and other artifacts produced through process improvements are organizational assets. They should be treated as products, albeit for internal use, and be produced with the same discipline used in producing any other product.

Consequences and Summary
By not specifying how to perform the process steps it gives carte blanche to whoever is performing the process to do whatever they want to in order to satisfy the step. In addition, by not specifying how to perform the process steps, it makes it impossible to perform a PPQA process audit. The person performing the objective evaluation of the process (process audit) needs to know how the process is supposed to be performed in order to create a checklist so he or she can objectively evaluate whether or not the process is being followed the way it is supposed to be followed.

Essentially, by not supplying the information of how to perform a process, you do not have a repeatable process, which is at the core of Maturity Level 2. A repeatable process means that every time someone uses the same documented process description, he or she will perform it the same way. The only way that you can ensure this outcome is to document how the process is performed. At Maturity Level 2 it is permissible to have multiple documented processes for the same process. At Maturity Level 3 the organization is supposed to examine the multiple documented processes and look for exemplar practices to elevate to the organization level to become the standard process for the organization.

The challenge facing the process writers is to document the process such that it is neither too detailed nor too general to follow. If it is too detailed, then you have painted yourself into a corner and the process may be too detailed to correctly follow. If too general, then it basically allows you to do anything you want. Either way, there is no benefit to the organization.

This issue also points out a fundamental difference between ISO 9000 and the CMMI. At its core, ISO 9000 is a set of standards for a quality system. It requires the existence of processes good, bad, or indifferent. In contrast, the CMMI is a set of guidelines for process integration and product improvement. The whole focus of the CMMI is process improvement. And it becomes very difficult to do process improvement if you don’t spell out how a process is performed.

To illustrate the issue allow me to cite an example of incorrectly worded procedure step.

The Project Manager derives estimates for the size of the software work products, or changes to the size of the software work products.

There is more than one way to estimate size if you don’t specify how to perform this step. By not specifying how, you cripple your ability to analyze the process and identify process improvements.

If I were to ask the Project Manager how he or she peforms this step, I might find out that they do this by using the project requirements and historical data (notes and reports from past projects and lessons learned) from similar past projects to determine size factors (SLOCs, number of files, database size, number of screens) and applying appropriate scaling factors using engineering judgment. The Project Manager then reviews the size estimates with the Team Leads for reasonableness. This is an excellent explanation and practice.

By providing this level of detail, now there is enough information for PPQA to perform a process audit and also enough information to perform process improvement and determine if an estimation model can be built to improve the accuracy of project estimates.

Requirements Development/Management Question

2009-04-20T06:59:00.000-07:00

I was slightly confused after reading this statement in the Requirements Development (RD) Process Area (PA), Specific Practice (SP) 2.1 states "Establish and maintain product and product component requirements, which are based on the customer requirements"

The informative material goes on to say "The modification of requirements due to approved requirement changes is covered by the "maintain" function of this specific practice; whereas, the administration of requirement changes is covered by the Requirements Management process area."

I do not understand the difference between maintenance of requirements in RD and administration of requirement changes in REQM. How are they different?

The Requirements Management (REQM) Process Area (PA) is there to help an organization manage changes to an agreed to set of requirements (baseline). So, no one makes any changes to the requirements (add/modify/delete) without following the documented requirements change control process, which should include review and approval by the relevant stakeholders of the requirements.

And actually, REQM is a specific instance of Configuration Management (CM). But REQM is such an important and vital practice that it is its own Process Area. So it is entirely possible for an organization to have one Change Management process that it uses to manage changes to requirements, documents, plans, test scripts, code, etc.

In contrast the intent of Requirements Development (RD) is to actively elicit and further refine the requirements that are then agreed to and incorporated into the requirements baseline. Specific Practice 2.1 contains the phrase “establish and maintain,” which has a special meaning within the context of the CMMI. If you have taken the Introduction to CMMI class you would have learned what this phrase means. This phrase is also defined in the CMMI Glossary and it means formulate, document, and use. For SP 2.1, this means that the product and product component requirements are formulated (elicited, discussed, refined, reviewed, etc.), documented (written down in a requirements specification, requirements management tool, etc.), and used (develop the project plan, design, code, tests, etc.).

So the connection between REQM and RD is that proposed requirements changes are reviewed, analyzed, approved, and processed by REQM and the authorized and approved changes are propagated through the baselined requirements by RD.

I would not get too focused on trying to understand the separation between REQM and RD. You have identified one of the many areas in the CMMI where there is a tight coupling between PAs, if not overlap. What is important is that you

create a requirements baseline that does not change unless there is an approved change request,
investigate, impact, review, and disposition all proposed requirements changes with the relevant stakeholders (approved or rejected), and
only make approved changes to the baselined requirements and appropriately update the traceability

Applicability of the Informative Material

2009-04-15T17:21:00.000-07:00

I have been told that if an organization has to go for CMMI ML 5 the organization has to address all of the sub-practices even though they are informative material and if the company is only going for ML 3, then they can apply appropriate sub-practices. Is this true?

Please allow me to try to explain the model. There are at least two ways to look at the CMMI: 1) implementing the model and 2) appraising the organization against the model.

In addition, there are three CMMI components: Required, Expected, and Informative. These components only have meaning when you are talking about appraisals. The Required components are the Specific and Generic Goals, the Expected components are the Specific and Generic Practices, and everything else is an Informative Component.

When you are implementing the model, you should not be concerned about differentiating between the different types of components. From Chapter Two of the CMMI-ACQ (and this statement applies to ALL CMMI constellations): “All model components are important because the informative material helps you to understand the expected and required material. It is best to take these model components as a whole. If you understand all three types of material, you can then understand all the pieces and how they fit together to form a framework that can benefit your organization.”

When the organization is being appraised against the CMMI in a formal SCAMPI A appraisal, the organization will only be appraised against the Required and Expected components, regardless of the Maturity Level. However, the appraisal team may be evaluating the evidence and perhaps asking questions in the interview sessions at the sub-practice level just to gain a better understanding of how the organization is addressing each of the Required and Expected components. The organization will not be penalized if it is not performing one or more sub-practices. The appraisal team will be identifying and documenting weaknesses with the organization’s implementation of the goals and practices.

Because there has been a misunderstanding of what High Maturity means (ML 4 and ML 5), the SEI has been emphasizing that the proper implementation of the goals and practices for OPP, QPM, OID, and CAR means reading, understanding, and implementing the types of activities described in the Informative material. So, for a ML 4 or ML 5 SCAMPI, the organization will not be evaluated against the OPP. QPM, OID, and CAR sub-practices, but weaknesses will be noted at the goal and practice level if the organization has not properly implemented these Process Areas to meet the intent, which is gained by understanding the Informative material.

Quantify Functionality - RD Process Area

2009-04-14T11:41:00.000-07:00

Sub-practice 1 of Requirements Development SP 3.2 states " Analyze and quantify functionality required by end users". What does quantifying functionality mean? Does it refer to quality characteristics of functionality or does it refer to Function Point estimation?

What quantify means in this sub-practice is to spell out the required functionality requirements so everyone is in agreement, as well as state exactly how many functionality requirements there are. And since this is a sub-practice, this is an informative component of the model and it is supplied for clarification of the intent of the practice statement “Establish and maintain a definition of required functionality.” If you are implementing RD in your organization, you are not required to implement the sub-practices. You need to do what makes good business sense for your organization in order to determine the definition of the required functionality.

OSSP Question

2009-04-14T11:36:00.000-07:00

What does "organization’s set of standard processes" mean?

As I have indicated in previous blog entries, the first place you should always check for definition questions is the CMMI Glossary. You will find a comprehensive definition of the OSSP there. If after reading the definition you still have questions, then read the Organization Process Definition (OPD) Process Area which will put the OSSP into context. And if you still have unanswererd questions, please post them to this blog.

Project Classification Part Two

2009-04-14T11:25:00.000-07:00

This is a follow on discussion to http://ppqc.blogspot.com/2009/04/project-classification.html

I will try to make my point clear here. Let's assume for a moment that an organization has identified project types i.e development and testing. If such an organization has to go for CMMI ML 2, then what should be their approach to further segregate each project type for better process implementation? I would like to look at project duration (start date and end dates) , the SDLC (phases), number of people working on project or effort put on the project.

Is there any other technique or anything else I should look at to categorize each project type into small, medium and large? Because I do not find CMMI publishing any kind of guidelines for segregating projects into small, medium and large. So ideally as you said ..project size... Can we say that to look at project size we need to look at effort required to complete the project and duration of project? So I am looking for other parameters to give the organization better picture of the basis to arrive at small, medium and large projects.

Thank you so much for the clarification.

You seem to be making a distinction between project type and project classification. In my mind project type and project classification are two different terms that mean the same thing: category. As you have discovered, there isn’t any guidance for what is a project type or classification. This categorization is up to the organization to determine for its internal purposes. Project size could be a classification, but is probably not the best one to use. As I have suggested in other posts, application domain, platform, language, methodology, etc. are potentially better categories to use. And the primary reason for labeling projects with a category is to allow the organization to compare like against like. For example, there is not a whole lot of value in comparing a web development project to a mainframe project, or a web development project to a road construction project. You want to compare apples to apples and oranges to oranges, not a mixture.

But I think that the bigger issue here is defining what is a “project” for your organization. A project has a defined start and stop, budget, resources, tasks, schedule, identified deliverables, and typically operates according to a plan. And a project can be composed of other projects. Once you have decided on what a project is for your organization, then it becomes pretty obvious the project types or classifications that make sense for your organization.

Typically I do not find that organizations have separate projects for development and testing. Testing is part of the development lifecycle and typically not treated as a separate project. However, you may have a testing plan, testing resources, funds, and a testing schedule. So testing could be a sub-project of the main development project. Therefore, I would not normally expect that projects would be categorized as development or testing. However, I have seen projects categorized as development or maintenance projects. Once you have decided on the primary project categories for your organization, then it might make sense to look at size (small, medium, large) within a specific category to further refine your definitions and comparisons.

Requesting Processes and Assets

2009-04-10T14:15:00.000-07:00

I have read your CMMI blog. Can I have some draft CMMI process documentation? I would like to learn and implement the process.

There is no such thing as a set of documented CMMI processes that can be provided for a company, though some unscrupulous consultants will try to sell you a set. Rather, there are industry accepted standards for documenting processes, such as ETVX. These process documentation standards are easily available by searching on the internet. The actual processes for REQM , PP, PMC, SAM, MA, PPQA, CM, etc. are a function of the organization’s business and its practices, though there is some commonality across companies and organizations. However, this commonality exists at the CMMI level. The CMMI is a set of guidelines for process improvement. The implementation of these guidelines will differ from organization to organization. The best advice that I can give you is to document all of your current business practices using an industry accepted process documentation standard, avoiding the temptation to improve the process. By simply documenting your existing processes, you will discover opportunities for improvement, but don’t make them at this point. Just document the process as it is currently practiced. Once you have all of your processes documented, then compare the results to the CMMI, add the missing practices, and address any improvement opportunities. Then you will have a set of processes that your employees will have ownership of and will also comply with the CMMI.

And, if you are interested in learning more about the CMMI, take the SEI-licensed Introduction to CMMI class from an SEI-authorized instructor.

Project Classification

2009-04-10T14:11:00.000-07:00

I heard from a colleague that project classification needs to be done to be compliant to the Project planning processes. Which Project Planning Specific Practice (SP) refers to project classification?

I am not sure what you mean by project classification. Do you mean application domain? Or project size? Or something else entirely?

My best advice to you is not to listen to colleagues who may not be CMMI experts, but instead work with a CMMI consultant or SEI-certified Lead Appraiser. They will provide you with the best information and CMMI interpretations.

The CMMI does not state that you must classify projects for Project Planning (PP). That is why you are having difficulty determining the applicable Specific Practice.

Applicability of Requirements Development (RD)

2009-04-10T14:09:00.000-07:00

Is Requirements Development (RD) applicable for maintenance projects?

If the CMMI-DEV is applicable to your maintenance projects, then the short answer is YES!

Excluding Supplier Agreement Management

2009-04-10T14:04:00.000-07:00

Page 440 of CMMI-DEV 1.2 model clearly states that: "SAM process area does not directly address arrangements in which the supplier is integrated into the project team and uses the same processes and reports to the same management as the product developers (for example, integrated teams)."

This statement opens room for some Lead Appraisers to trigger the default button: "SAM is out." But the paragraph continues with the following statement: "Typically, these situations are handled by other processes or functions, possibly external to the project, though some of the specific practices of this process area may be useful in managing the formal agreement with such a supplier."

In my oppinion, not considering SAM may incur problems in the future because you may be postponing the elaboration of a mature way to handle suppliers and contracts and this will be necessary when the organization evolves to higher maturity levels. And, of course, it will be necessary to survive in a global IT world driven by strong and stronger supplier/acquirer relationships.

In my country, only 2 out of 16 organizations who published their Maturity Level 2 appraisals considered SAM in their scope. And, guess what? Many of these organizations use a high number of contractors in their development phases. So, what led them to exclude SAM?

It is important to understand the intent of SAM. If you are augmenting your staff by having a supplier provide people and these people then act, for all intents and purposes, as your employees, then SAM does not apply. Basically, they are following your processes and not managing any of the work on their own, you are managing the work. However, if you give the supplier a chunk of work that they can manage by themselves using their own processes, then SAM applies. So both the organization and the Lead Appraiser need to be aware that if the relationship changes with the supplier, then SAM may move from being N/A to in scope for an appraisal. And you raise a good point, whether or not the Lead Appraiser determines if SAM is in or out of scope, the organization should be aware of the necessary practices it should have in place to manage a supplier.

And if the organization wants to become more sophisticated in managing suppliers, they should be using the CMMI-ACQ.

PPM - Data Analysis

2009-04-10T13:57:00.000-07:00

We are a CMMI ver 1.1 level 5 assessed software company and we are in the process of ver 1.2 assessment. As part of this, we are developing the Process Performance Models. We would like to get input on the following points.

What kind of statistical analysis are required on the data that is collected for the PPM development?.
Do we need to perform Gage R&R test on the data that is collected?. As mentioned, since we are a software company and not a manufacturing company, i am not too sure about the data collection that needs to happen at multiple instances by the same person on the same tool and different person on the same tool.

For example, to check for Repeatability, if I am considering Schedule Variance of various feature development as a measure, there might not be any difference in the schedule variance that will be measured by different people, if the operational definition is clear for the metric.
Now to check for Reproducibility, if competency of the resource for code development is considered as a measure, this could be changing from period to period, as unlike in manufacturing industry where the activities are of repetitive nature. So the reproducibility will be minimal in this scenario.
If its mandatory to perform Gage R&R analysis on the data, can you throw some ideas on the different areas where this can be applied and how the analysis can be performed. Please share your thoughts on this.

It sounds like the Process Performance Models (PPMs) were overlooked for your organization when it was appraised to CMMI v1.1 3 years ago. I would like to make several points regarding PPMs.

The CMMI does not specify any required statistical analysis technique for PPM development. Based on your data, your QPPOs, PPBs, the organization has to decide the proper analytic techniques to use. There is a wide variety available for use.
What is the reason you are considering Repeatability and Reproducibility? Are you led to these items by your Quality and Process-Performance Objectives (QPPOs)?
There is no CMMI requirement to use Gage R&R.
It sounds to me that it would be a good idea for you and your organization to have someone facilitate a Measurement and Analysis workshop for you to properly identify your measures, PPBs, and PPMs.
You are asking some questions that cannot be properly answered on this blog unless we are working directly with your organization and have some knowledge of your business.

PPQA Audits

2009-04-10T13:50:00.000-07:00

Would you please distinguish the different types of audits 1) Projects, 2) Process and 3) products? Does PPQA audit the Project, Process, or Product? Or all the three? And from which area do we need to collect improvements, 1, 2, or 3? I'm confused, can you help?

You say that you are confused. I Let me try to provide an explanation for what I think you are asking about PPQA. The intent of PPQA is to act as the eyes and ears of senior management to ensure that the practitioners are following the documented processes to produce the work products. So PPQA performs two types of audits: process audits and work product audits. Now the processes being audited can be at the individual level, project level, or the organization level. And the processes being audited are not restricted to the CMMI Process Areas. The organization has to determine which processes to audit based on its business goals and objectives, so there may be processes audited in addition to the processes covered by the CMMI.

A process audit is conducted by first studying the documented process and then interviewing the practitioners to determine if they are following the process as documented.

Each process has one or more work products that are produced by following the process. These work products can be at the individual, project, or organizational level as well. The work products can be audited by sitting at a desk and reviewing the work product against the documented requirements for the work product. Is the work product produced correctly? Does it contain the proper level of information? Etc.

Both process and work product audits will identify non-compliances. By analyzing the non-compliance issues, PPQA should be able to identify the underlying causes for the issues and recommend one or more process improvement suggestions.

Estimation Rationale

2009-04-09T14:42:00.000-07:00

I guess this seems little silly, but I needed more clarity on it. According to PP 1.2, attributes need to be estimated. Can anyone give me examples of attributes for project? I guess, when we say small, medium , large type projects they form the characteristics of project and not the attibutes. For sure, once we are clear with attributes we can surely ensure that our effort estimates (PP 1.4) are based on estimation rationale. How will project attibutes map up to estimation rationale?

What PP SP 1.2 is addressing is the Basis of Estimate for the project tasks, deliverables, etc. Basically this practice is looking for the sizing attributes that you use to ultimately determine your effort estimates. For a document, an attribute you might estimate is the number of pages for the document that have to be added, modified, or deleted. For hardware, it might be the number of drawings you have to maintain. For software, it might be the number of lines of code, function points, etc. Just using small, medium, or large doesn’t provide a Basis for Estimate. The Basis of Estimate is then your estimation model that is derived from the historical data from prior projects going through this same process. When you first begin to use the Basis of Estimate approach you may not have very good data, or no data at all. So the best you can do is make a guess as to the estimate (SWAG or engineering judgment). But over time as you collect the project data from each project, you will be able to refine this approach and have much better estimates. Then you take these sizing parameters or attributes and apply a productivity factor, again based on your historical data, to arrive at the effort and cost estimates in SP 1.4.

MA and PPQA Questions

2009-04-08T12:03:00.000-07:00

I have the following two basic queries about CMMI ML 2:

While writing a Metrics and measurement process, should we address the organization level metrics data consolidation and review. As ML 2 is project specific, is it proper to also document the organization level data consolidation? Also can anyone tell me, the right site for definition of metrics like requirement stability index, schedule variance, effort variance etc.?
Similarly while documenting PPQA process, is it proper to start with defining an organization level PPQA plan? I am looking for boundaries where to limit writing processes compliant to ML 2. I know that G.P 2.1 to G.P 2.10 must be in place to achieve CMMI ML 2, but the organization specific plans/areas must not be mentioned/documented at CMMI ML 2.

You sound like you are focusing on CMMI compliance rather than on your business goals and objectives. One of the basic tenets of the model is your business objectives. That is where your focus belongs. And if done properly, you will have the side benefit of being CMMI compliant. So, to address your questions:

When documenting your Measurement and Analysis process, you should focus on those measures that are important to you. Remember, the first MA practice SP 1.1 states “Establish and maintain measurement objectives that are derived from identified information needs and objectives.” So whatever you have identified as information needs and objectives, that should be your MA focus. At ML 2, for many organizations that are just doing this for the first time, I recommend the org take baby steps and begin with a project focus. But you don’t have to be restricted to the project, an ML 2 org may have also identified some org level measures as well. Go to the Practical Software and Systems Measurement web site for the specific measurement information you need www.psmsc.com
There are NO CMMI-imposed restrictions on the limits of PPQA. Your organization must define its own limits for the processes you are going to audit. Since GP 2.9 applies to all Process Areas, at a minimum for ML 2, PPQA applies to all of the ML 2 Process Areas you have implemented in your organization. But, if there are other processes that are critical and/or important to the success of your business, then it makes perfect sense to have PPQA audit them as well. Again, do what is right for your business.

Integrated SCAMPI

2009-04-01T09:08:00.000-07:00

I am faced with a problem with determining which CMMI to use (CMMI-DEV, CMMI-SVC, or CMMI-ACQ) for an in-house IT Department that performs all three types of functions for the organization. I face the following questions:

Is there an integrated SCAMPI for all three models held together? Or is the scope is simply determined by adding different PAs from different models? In this case, against what model will the ratings be announced?
What about the cost paid to the SEI? Is it calculated differently for such a SCAMPI?
What about exclusions if all PAs from these three models that are not fully applicable? Is there a way other than pursuing Continuous Representation?
Can you recommend any research work done already on integrating the three model for designing and implementing the OSSP?

I will be greatful for your help and support...

As a first step you should hire an SEI-certified Lead Appraiser, preferably in all three constellations, to provide you the proper advice as to which constellation is appropriate for your organization.

I would only recommend that an organization use the CMMI-ACQ if their primary focus was acquiring products and services from vendors. The CMMI-DEV and CMMI-SVC both have the Supplier Agreement Management Process Area, so either constellation will work if acquisition is not the primary focus of the organization.

Here are my answers to your specific questions:

It is possible to conduct blended SCAMPI A appraisals that cover more than one constellation. But your Lead Appraiser will have to work with your organization and the SEI on how best to perform the blended appraisal and determine your Capability or Maturity Level ratings.
There are NO fees paid to the SEI by the organization for any appraisal. Any appraisal costs are negotiated between you and your Lead Appraiser.
The determination of the appraisal scope is performed jointly with your Lead Appraiser when planning the appraisal. The appraisal scope specifies the representation and the Process Areas being evaluated.
I am unaware of any reported results using blended constellations. Though Mike Phillips from the SEI has said that blended SCAMPIs are permissible. I suggest that you contact the SEI and ask for this kind of information, if it exists.

Definition of SOW

2009-04-01T09:04:00.000-07:00

I would like to know what is the content of the Statement of Work and how does this document meet requirements of the CMMI?

As a best practice with definition questions, the first place to look is the CMMI Glossary. In this case there is a definition of Statement of Work (SOW) in the Glossary. “A description of contracted work required to complete a project.” In other words, the SOW contains the requirements for what you have to delivery to the customer. And there are varying levels of detail from SOW to SOW depending on the size, criticality, and nature of the work. For large government contracts, the SOW may also contain the Work Breakdown Structure (WBS) for the program/project as well. At a minimum, the SOW will contain the customer’s requirements that the organization will further develop. So the SOW supports RD and may also support PP.

CMMI-SVC and Emergence of Open Organizations

2009-04-01T09:00:00.000-07:00

Emergence of Open Organization- when will it happen? LinkedIn + Google+Sourceforge+Newscale + Visa
I foresee Newscale/ Pinky/SEI coming up with a master service catalogue that could be integrated with Google where any Group member can have his own service catalog in his profile. The Group Service catalog could well be an Organization's service catalog. Probably any individual would say he has his office is in Yahoo!, Google etc and may add that he works for a particular social enterprise/many enterprises. I also feel that some governmental procedures or a global standard will evolve for the governance of Open companies. My question is does the CMMI-SVC address this?

What you are asking about is a virtual office environment and the tools to enable this environment. There are two practices in the CMMI that address work environments: OPD SP 1.6 Establish and maintain work environment standards and IPM SP 1.3 Establish and maintain the project’s work environment based on the organization’s work environment standards. Since both OPD and IPM are Core Process Areas (PAs), they are common to all CMMI constellations CMMI-DEV, CMMI-ACQ, and CMMI-SVC. So your virtual office concept can work with any of the three constellations.

Advice on a Gap Analysis

2009-04-01T08:57:00.000-07:00

For the first assessment, does the gap analysis have to be performed against practices or against work products?

There are no specified standard requirements for a Gap Analysis. The Gap Analysis is intended to identify major differences between a model standard and actual practice. A Gap Analysis may involve just reviewing documents, just interviewing people, or a combination of both. I like to evaluate a combination of documents and people for a Gap Analysis. This approach yields the best results, because in many cases the process as practiced is different from the process as documented.