A Test Managers Guide Back To The Basics

Pages 37
Views 3
of 37
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
ISTQB TEST MANAGER COURSE MATERIAL A TEST MANAGERS GUIDE BY LUCIAN DAN CANIA A LITTLE BIT ABOUT ME First of all there is not a lot to say, I have 10 years of software test management experience with a diversified background in Telecom and Finance IT. Currently I am offering Test Management consulting services across Europe for either project delivery or quality assurance purposes. The advantages that I bring to the table is a cross view from different perspectives as I also covered Delivery, Operations, Service Management and Release and Deployment Management that help me look better understand open points for a multitude of stakeholders. Diverse Experience Although my background is not in training and I am not a certified training consultant, I manged to get certified along my career as a Prince2 Practitioner, ITIL Intermediate for Service Management, Service Transition and Continual Service Improvement, but I am also certified as a AgilePM Practitioner and ISTQB Test Manager. In the past i worked on several transformation programs like ERP implementation, Retail, Billing, but also I worked cross Europe on major technical upgrades and a IFRS Global implementation. Index 1 Back to the basics 2 Reviews 3 Defect Management 4 Test Management 5 Test tools & automation 6 People skills 7 Improving the test process Exam Prep at the END This book is based on the ISTQB Advanced Syllabus version 2012 and it also references the ISTQB Foundation Syllabus version 2018. It uses terminology definitions from the ISTQB Glossary version 3.2. This book is intended for testing professionals which have at least 2 years of experience and are interested in taking the ISTQB Advanced Test Manager certification Exam. Reading this book does not guarantee a successful grade at the certification Exam and you should also study the ISTQB material provided for FREE at www.istqb.org https://cania-consulting.com/ Back to the basics In testing there are 4 Test Levels. They can be encountered when testing both a system and a system of systems (multiple, dispersed, independent systems in context as part of a larger, more complex system). Acceptance Test System Test Integration Test Component / Unit Test Component / Unit Test focuses on components that are separately testable. Integration Test focuses on interactions between components or systems. System Test focuses on the behavior and capabilities of a whole system or product, often considering the end-to-end tasks the system can perform and the non-functional behaviors. Acceptance Test focuses on the behavior and capabilities of a whole system or product: confidence, completion, fit for use & purpose. The best way to navigate across the Test Levels is to follow a predefined course, to constantly monitor it as you go along and apply course corrections whenever you see fit. Start with a Plan that you will Monitor by using Metrics in order to Control Test Planning Is the activity of establishing or updating a test plan which starts at the initiation of the test process and in line with the Test Strategy. Test planning applies for each test level and also includes the methods for monitoring for each. Test Plan is a document describing the scope, approach, resources and schedule of intended test activities. As a record of the test planning process, it also covers: test items features to be tested testing tasks who will do each task degree of tester independence test environment test design techniques Entry and Exit Criteria (with their rationale) risk assessment based on requirements contingency planning based on risk assessment integration of reactive test techniques in execution During test planning, the Test Manager defines the approach for each level: What is tested Goals & Objectives Test techniques & tools In order to have an effective planning, we need to consider the complex relationships between test phases, but also between development and test. Sine examples would be the requirement traceability matrix or informal transfer of information. Another factor for effective planning would be the proper listing of the testing scope with each feature associated with a design specification, environment, etc. Contact with all stakeholders has to be initiated at this stage, but also all external dependencies identified and service level agreements put in place. In order to properly measure the progress, evaluate the Entry and Exit criteria and to exercise control, we need to put in place metrics starting with Test Planning. In other words, the requirement traceability matrix is a document that maps and traces user requirement with test cases. The main purpose of Requirement Traceability Matrix is to see that all test cases are covered so that no functionality should miss while doing Software testing. Test Plan content example as per IEEE829 standard Test plan identifier Introduction Test items Features to be tested Features not to be tested Approach Item pass/fail criteria Suspension criteria and resumption requirements Test deliverables Testing tasks Environmental needs Responsibilities Staffing and training needs Schedule Risks and contingencies Approvals Metrics Are a measurement scale and the method used for measurement. It is important that the proper set of metrics is established as they are mainly used to measure the progress of testing. This will also enable testers to report results in a consistent way and with coherent tracking (Example: % of test coverage, % of test execution, etc.). Although they should be as automated as possible to allow immediate understandings of where we are, metrics should be defined based on specific objectives that can also be presented to stakeholders at various meetings, for various concerns. Project metrics Product metrics measure progress measure some attribute toward established of the product, such as project exit criteria. the extent to which it has been tested or the Process metrics defect density. measure the capability of the testing or People metrics measure the capability of development process, individuals or groups, such as such as the percentage the implementation of test of defects detected by cases within a given schedule. testing. Monitor & Control A testing schedule and monitoring framework need to be established to track progress versus plan. Due to this, all ongoing activities should have targets which are tracked via ongoing measurements. When I am stating all ongoing activities, I am also referring at test analysis, test design, test implementation and not only at test execution. It is important to be able to relate the information and status of the test work products in an understandable and relevant manner based on the audience of those reports. Not everyone is looking for the same level of details. The aim of test control is to compare actual progress versus the plan and implement corrective actions. Common examples Test Condition execution vs plan Test Case status 50 In Progress 40 9.2% Passed 27.5% 30 20 Failed 10 6.4% 0 No Run Mon Tue Wed Thu Fri 53.2% Test Analysis Is process of analyzing the test basis (all documents from which the requirements of a component or system can be inferred) and defining test objectives. Test Design Is the process of transforming general test objectives into tangible test conditions and test cases. Test Implementation Is the process of developing and prioritizing test procedures, creating test data and, optionally, preparing test harnesses and writing automated test scripts. Test Execution Is the process of running a test on the component or system under test, producing actual result. Test Analysis Is process of analyzing the test basis (all documents from which the requirements of a component or system can be inferred) and defining test objectives. Covers WHAT is to be tested in the form of test conditions and can start as soon as the basis for testing is established for each test level. It can be performed in parallel, integrated or iteratively with Test Design. Evaluates and reviews the test objectives and product risks, while it defines detailed measures and targets for success. deciding on the level of detail should consider The level of testing; level of detail and quality of the test basis. System/software complexity and development lifecycle used. Project and product risk Relationship between test basis, what is to be tested and how is to be tested Test management tool used The level of maturity of the test process and the skills and knowledge of the test analysts The level at which Test Design and other test work products are specified Availability of stakeholders for consultation Test Condition Is an item or event of a component or system that could be verified by one or more test cases (ex: function, transaction, feature, etc.). A test condition may or may not specify values or variables. It all depends on the context at that test level. Some might be generic like "Test Payment" and others may be specific like "Test Payment with VISA for 3 items and a cost over 100$". Don;t forget, if you go specific, then expect a higher number of test conditions. Check what you need at that stage and adapt. It may not be the same for Component Test as for System Test. advantages of detailed test conditions More flexibility in relating other test work products Better and more detailed monitoring and control Contributes to defect prevention by occurring early Relates testing work products to stakeholders in terms that they can understand Influences and directs other testing activities, but also other development activities Enables test design, implementation and execution to be optimized by more efficient coverage Basis for clearer horizontal traceability within a test level disadvantages of detailed test conditions Potentially time-consuming Maintainability can become difficult Level of formality needs to be defined and implemented across the team GO detailed when Lightweight test design documentation methods Little or no formal requirements or other development work products The project is large-scale, complex or high risk GO generic when Component (Unit) level testing Less complex projects where simple hierarchical relationships exist Acceptance testing where use cases can be utilized to help define tests Test Design Is an item or event of a component or system that could be verified by one or more test cases (ex: function, transaction, feature, etc.). Covers HOW something is to be tested by identifying test cases with step wise elaboration for the test conditions (from Test Analysis) or from the test basis using techniques identified in the test strategy or plan. This phase can start for a given Test Level once Test Conditions are identified and enough information is available to enable the production of Test Cases. Although it can be merged together with Test Analysis, for higher levels of testing it will remain a separate activity. It is likely that some tasks that normally occur during test implementation will be integrated into the test design process. Especially when using an iterative approach. The coverage of test conditions by either creating low- level and high-level test cases can be optimized by the creation of test data starting in Test Design. In other words, a test case is a set of input values, execution preconditions, expected results and execution post-conditions, developed for a particular objective or test condition, such as to exercise a particular program path or to verify compliance with a specific requirement. Test Implementation Is the process of developing and prioritizing test procedures, creating test data and, optionally, preparing test harnesses and writing automated test scripts. This is when tests are organized and prioritized and when test designs are implemented as test cases, test procedures and test data. It is of great importance to pick the right tests and run them in the right order. The importance of this even grows exponentially in risk-based strategies when we prioritize based on the likelihood of risk and problems. During this stage you should ensure: delivery of test environment delivery of test data constraints, risks and priorities are checked test team is ready for execution entry criteria is checked (explicit/implicit) Some organizations may follow the IEEE829 standard to define inputs and their associated expected results during testing. Other only have rigorous rules when they need to provide evidence of compliance for regulatory projects or for adherence to standards. In the most common cases, the test inputs are usually documented together with expected results, test steps and stored test data. Just like test conditions and test cases, even during test implementation we will face the decision to go into an extensive (detailed) stage or to have a light (generic) approach. This decision should be taken by your understanding of the development lifecycle and by the predictability of software features under test. For example, in agile or iterative lifecycles where code changes dramatically from iteration to iteration, the implementation work changes significantly between each stage. Please do not count off the extensive implementation preparation due to the above: Concrete test cases provide working examples of how the software behaves When tests are archived for long term and re-use in regression these details may become valuable Domain experts are likely to verify versus a concrete test rather than an abstract business rule Further weakness in software specification is identified Some defects can be found only in production-like test environments. These are often expensive to procure and difficult to configure and manage. Similar challenges are also faced for the use of production data or production like data which can even lead to daa privacy or other headackes. Test implementation is not all about manual testing, this is the stage where automation scripting takes place, the stage where automation versus manual prioritization and execution order is established. And I am not talking only about automation, even tool acquisition is done here, especially for test data generation required to prepare for load, volume or performance testing. Quick reminder before moving forward Test Case A set of preconditions, inputs, actions (where applicable), expected results and post conditions, developed based on test conditions. Test Script A sequence of instructions for the execution of a test. Test Suite groups of test scripts, as well as a test execution schedule. Test Charter An instruction of test goals and possible test ideas on how to test. Documentation of test activities in session-based exploratory testing. Test Execution Is the process of running a test on the component or system under test, producing actual result. Should finish before execution starts Tests are designed or at least defined Tools are in place for test management and defect management and test automation (if applicable) Standards for test logging and defect reporting are published Execution begins once The test object is delivered The Entry criteria for test execution is met During execution, a Test Managers role is to: Monitor progress according to the plan Initiate and carry out control actions to guide testing Ensure that test logs provide an adequate record of relevant details for tests and events During execution it is important to keep a traceability between test conditions, the test basis and the test objective and to have the appropriate level of test logging. Time should be reserved for experienced-based and defect-based test sessions driven by testers findings. Entry Criteria Set of generic and specific conditions for permitting a process to go forward with a defined task Purpose is to prevent a task from starting which would entail more effort compared to the effort needed to remove the failed entry criteria Exit Criteria Set of generic and specific conditions, agreed with stakeholders for permitting a process to complete Prevent a task from being considered completed when there are still outstanding tasks not finished Used to report progress against a plan and to know when to stop testing A Test Managers should: ensure that effective processes are in place to provide necessary information for evaluating entry & exit criteria make sure that the definition of the information requirements and methods for collection are part of test planning ensure that members of the test team are responsible for providing the information required in an accurate and timely manner The evaluation of exit criteria and reporting of results is a test management activity. We can measure properties of the test execution process Number of test conditions, cases, pass, failed, etc. test conditions test cases test procedures planned tests executed tests passed tests failed 0 10 20 30 40 50 Total defects, classified by severity, priority, status Critical Blocker Verification Open High 4.3% Low 4.5% 9.3% High 11.6% 10.6% 22.7% Low 15.9% 31.9% Rejected 11.6% Clarification Severity Priority 5.8% Status Medium Closed 53.2% Medium 58.1% 56.8% Change requests Quality risks New Postponed 16.7% 23.3% Deferred 21.7% Accepted 10% Acknowledge Mitigated 13% 65.2% Rejected 50% Planned versus actual 20 60 15 40 10 costs 20 execution 5 0 0 Week 1 Week 2 Week 3 Week 4 Week 5 Day 1 Day 2 Day 3 Day 4 Day 5 Test Closure consists of finalizing and archiving the testware and evaluating the test process, including preparation of a test evaluation report. Once test execution is deemed to be complete, the key outputs should be captured: Test completion check - ensuring that all test work is indeed concluded Test artifacts handover - delivering valuable work products to those who need them Lessons learned - performing or participating in retrospective meetings where important lessons Archiving results, logs, reports, and other documents These tasks are important (often missed) and should be explicitly included as part of the test plan. A Test Managers should: Look for opportunities to reuse test work products Keep in mind that retrospectives should apply to testing as well as to the entire project and indeed the wider organization. Problems tend to be systemic, not isolated Examples of tasks within a project Waterfall Requirements Design Implementation Verification Maintenance Deploy Testing Design Feedback Planning Analysis Agile V-Model Requirements Acceptance Test Level Test Plan Specification System Test System Design Integration Test Unit Design Unit Test Code Development Glossary Each of the terms specified below are defined as per the ISTQB® Glossary which is displayed online at: https://glossary.istqb.org/en/search/ software lifecycle The period of time that begins when a software product is conceived and ends when the software is no longer available for use. The software lifecycle typically includes a concept phase, requirements phase, design phase, implementation phase, test phase, installation and checkout phase, operation and maintenance phase, and sometimes, retirement phase. Note these phases may overlap or be performed iteratively. system of systems Multiple heterogeneous, distributed systems that are embedded in networks at multiple levels and in multiple interconnected domains, addressing large-scale inter- disciplinary common problems and purposes, usually without a common management structure. test basis The body of knowledge used as the basis for test analysis and design. test planning The activity of establishing or updating a test plan. test plan Documentation describing the test objectives to be achieved and the means and the schedule for achieving them, organized to coordinate testing activities. measurement The process of assigning a number or category to an entity to describe an attribute of that entity. metric A measurement scale and the method used for measurement. test anal
Related Documents
View more...
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks

We need your sign to support Project to invent "SMART AND CONTROLLABLE REFLECTIVE BALLOONS" to cover the Sun and Save Our Earth.

More details...

Sign Now!

We are very appreciated for your Prompt Action!