Bài giảng Software Testing and Quality Assurance: Standard for Software Test Documentatio

1Software Testing and Quality Assurance
Standard for Software Test Documentation
Dr. Dao Nam Anh
Faculty of Information Technology
University of Technology and Management
2Resources
► Pressman, Software Engineering, McGraw Hill (chapter 18 
& 19)
► Sommerville, Software Engineering, Addison-Wesley 
(chapter 22 & 23)
► Software Testing and QA Theory and Practics, Chapter 7, 
WILEY Publish
► Foundations Of Software Testing, Istqb Certification, 
Dorothy Graham, Erik Van Veenendaal, Isabel Evans, Rex 
Black
► Jovanović, Irena, Software Testing Methods and 
Techniques
► Lâm Quang Vũ, 
3Nội dung
►Qui trình
► Standard for Software Test Documentation
► Test plan
► Test-case specification
► Test-incident report (bug report)
► Test-summary report
► Test-Schedule
► Change in defect number 
► Chiến thuật kiểm thử
► Testing Technics 
4Quản lý quá trình kiểm thử
Qui trình
►plan
►execute
►evaluate
►document
►report
5Quản lý quá trình kiểm thử
Qui trình
6Quản lý quá trình kiểm thử
Standard for Software Test Documentation
Standard for Software Test Documentation [IEEE 
829]:
1. Test plan: the scope, approach, resources, and schedule 
of the testing activities.
2. Test-design specification: the refinements of the test 
approach, and the features to be tested by the design and 
its associated tests.
3. Test-case specification: a test case identified by a test-
design specification.
4. Test-procedure specification: the steps for executing a 
set of test cases or, more generally, the steps used to 
analyze a software item in order to evaluate a set of 
features.
7Quản lý quá trình kiểm thử
Standard for Software Test Documentation
5. Test-item transmittal report: the test items being
transmitted for testing, including the person responsible
for each item, its physical location, and its status.
6. Test log: a chronological record of relevant details
about the execution of tests.
7. Test-incident report (bug report): any event that
occurs during the testing process which requires
investigation.
8. Test-summary report: the results of the designated
activities, and evaluations based on these results.
8Quản lý quá trình kiểm thử
Test plan
1. Test-plan identifier: specifies the unique identifier 
assigned to the test plan.
2. Introduction: summarizes the software items and 
software features to be tested, provides references to the 
documents relevant for testing (overall project plan, quality 
assurance plan, configuration management plan, applicable 
standards).
3. Test items: identifies the items to be tested, including 
their version/revision level; provides references to the 
relevant item documentation (requirements specification, 
design specification, user’s guide, operations guide, 
installation guide, ); also identifies items which are 
specifically excluded from testing.
9Quản lý quá trình kiểm thử
Test plan
4. Features to be tested: identifies all software features and their 
combinations to be tested, identifies the testdesign specification 
associated with each feature and each combination of features.
5. Features not to be tested: identifies all features and significant 
combinations of features which will not be tested, and the reasons for 
this.
6. Approach: describes the overall approach to testing (the testing 
activities and techniques applied, the testing of non-functional 
requirements such as performance and security, the tools used in
testing); specifies completion criteria (for example, error frequency or 
code coverage); identifies significant constraints such as testing-
resource availability and strict deadlines; serves for estimating the 
testing efforts.
10
Quản lý quá trình kiểm thử
Test plan
7. Item pass/fail criteria: specifies the criteria to be used to 
determine whether each test item has passed or failed 
testing.
8. Suspension criteria and resumption: specifies the criteria 
used to suspend all or portion of the testing activity on the 
test items (at the end of working day, due to hardware 
failure or other external exception, ), specifies the testing 
activities which must be repeated when testing is resumed.
9. Test deliverables: identifies the deliverable documents, 
typically test-design specifications, test-case specifications, 
test-procedure specifications, test-item transmittal reports, 
test logs, test-incident reports, description of test-input 
data and test-output data, description of test tools.
11
Quản lý quá trình kiểm thử
Test plan
10. Testing tasks: identifies the set of tasks necessary to 
prepare and perform testing (description of the main 
phases in the testing process, design of verification 
mechanisms, plan for maintenance of the testing 
environment, ).
11. Environmental needs: specifies both the necessary and 
desired properties of the test environment (hardware, 
communications and systems software, software libraries, 
test support tools, level of security for the test facilities, 
drivers and stubs to be implemented, office or laboratory 
space, ).
12
Quản lý quá trình kiểm thử
Test plan
12. Responsibilities: identifies the groups of persons responsible for 
managing, designing, preparing, executing, witnessing, checking, and 
resolving the testing process; identifies the groups responsible for 
providing the test items (section 3) and the environmental needs
(section 11).
13. Staffing and training needs: specifies the number of testers by skill 
level, and identifies training options for providing necessary skills.
14. Schedule: includes test milestones (those defined in the overall 
project plan as well as those identified as internal ones in the testing 
process), estimates the time required to do each testing task, identifies 
the temporal dependencies between testing tasks, specifies the 
schedule over calendar time for each task and milestone.
13
Quản lý quá trình kiểm thử
Test plan
► 15. Risks and contingencies: identifies the high-
risk assumptions of the test plan (lack of skilled 
personnel, possible technical problems, ), 
specifies contingency plans for each risk 
(employment of additional testers, increase of 
night shift, exclusion of some tests of minor 
importance, ).
► 16. Approvals: specifies the persons who must 
approve this plan.
14
Quản lý quá trình kiểm thử
Test-case specification
1. Test-case-specification identifier: specifies the unique 
identifier assigned to this test-case specification.
2. Test items: identifies and briefly describes the items and 
features to be exercised by this test case, supplies 
references to the relevant item documentation 
(requirements specification, design specification, user’s 
guide, operations guide, installation guide, ).
3. Input specifications: specifies each input required to 
execute the test case (by value with tolerances or by 
name); identifies all appropriate databases, files, terminal 
messages, memory resident areas, and external values 
passed by the operating system; specifies all required 
relationships between inputs (for example, timing).
15
Quản lý quá trình kiểm thử
Test-case specification
► 4. Output specifications: specifies all of the outputs and features (for 
example, response time) required of the test items, provides the exact 
value (with tolerances where appropriate) for each required output or 
feature.
► 5. Environmental needs: specifies the hardware and software 
configuration needed to execute this test case, as well as other
requirements (such as specially trained operators or testers).
► 6. Special procedural requirements: describes any special constraints 
on the test procedures which execute this test case (special set-up, 
operator intervention, ).
► 7. Intercase dependencies: lists the identifiers of test cases which must 
be executed prior to this test case, describes the nature of the
dependencies.
16
Quản lý quá trình kiểm thử
Test-incident report (bug report)
►1. Bug-report identifier: specifies the unique 
identifier assigned to this report.
►2. Summary: summarizes the (bug) incident 
by identifying the test items involved (with 
version/revision level) and by referencing 
the relevant documents (testprocedure 
specification, test-case specification, test 
log).
17
Quản lý quá trình kiểm thử
Test-incident report (bug report)
3. Bug description: provides a description of the incident, so as to correct the bug, 
repeat the incident or analyze it off-line:
· Inputs.
· Expected results.
· Actual results.
· Date and time.
· Test-procedure step.
· Environment.
· Repeatability (whether repeated; whether occurring always, occasionally or just once).
· Testers.
· Other observers.
· Additional information that may help to isolate and correct the cause of the incident; 
for example, the sequence of operational steps or history of user-interface 
commands that lead to the (bug) incident.
4. Impact: Priority of solving the incident / correcting the bug (urgent, high, 
medium, low).
18
Quản lý quá trình kiểm thử
Test-summary report
1 Test-summary-report identifier: specifies the 
unique identifier assigned to this report.
2. Summary: summarizes the evaluation of the test 
items, identifies the items tested (including their 
version/revision level), indicates the environment 
in which the testing activities took place, supplies 
references to the documentation over the testing 
process (test plan, test-design specifications, test-
procedure specifications, test-item transmittal 
reports, test logs, test-incident reports, ).
19
Quản lý quá trình kiểm thử
Test-summary report
3. Variances: reports any variances/deviations of the test 
items from their design specifications, indicates any 
variances of the actual testing process from the test plan 
or test procedures, specifies the reason for each variance.
4. Comprehensiveness assessment: evaluates the 
comprehensiveness of the actual testing process against 
the criteria specified in the test plan, identifies features or 
feature combinations which were not sufficiently tested 
and explains the reasons for omission.
5. Summary of results: summarizes the success of testing 
(such as coverage), identifies all resolved and unresolved 
incidents.
20
Quản lý quá trình kiểm thử
Test-summary report
6. Evaluation: provides an overall evaluation of each 
test item including its limitations (based upon the 
test results and the item-level pass/fail criteria).
7. Summary of activities: summarizes the major 
testing activities and events, summarizes resource 
consumption (total staffing level, total person-
hours, total machine time, total elapsed time used 
for each of the major testing activities, ).
8. Approvals: specifies the persons who must 
approve this report (and the whole testing phase).
21
Quản lý quá trình kiểm thử
Test-Schedule
22
Quản lý quá trình kiểm thử
Test-Schedule
23
Quản lý quá trình kiểm thử
Change in defect number 
24
Quản lý quá trình kiểm thử
Chiến thuật kiểm thử
► Lũy tiến: test mỗi phần riêng lẻ. 
► Bottom-Up (lũy tiến): module cấp thấp nhất được 
test trước. 
► Top-Down (lũy tiến): module cấp cao nhất được 
test trước. 
► Big bang: tích hợp hệ thống, gặp nhiều bất lợi. 
► Tĩnh: mã nguồn được kiểm tra, không được thực 
thi (thanh tra, code review,). 
►Động: mã nguồn được thực thi, không cần kiểm 
tra (Blackbox & Whitebox). 
25
Quản lý quá trình kiểm thử
Testing Technics 
26
Q & A