Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CTAL-TA Traceability matrix table in LaTeX #61

Closed
danopolan opened this issue Jun 6, 2024 · 29 comments · Fixed by #90
Closed

CTAL-TA Traceability matrix table in LaTeX #61

danopolan opened this issue Jun 6, 2024 · 29 comments · Fixed by #90
Assignees

Comments

@danopolan
Copy link
Contributor

Despite the provided code in #51 comment I am not able to pull the traceability matrix for CTAL TA and I am not able to find an issue with the code.

Initially I have only tried the first part with BOs, but the pipeline failed and cannot find out the problem.

Later I have added all LOs, which makes the full table, but of course, with the same result.

Can you please:

  1. Find the issue(s) I made
  2. Suggest how to debug pipeline failures for LaTeX tables.
  3. Suggest how to debug LaTeX tables creation. Is it possible to use some online tooling? Or syntax checker? Overleaf didn't worked for me :(
  4. Add these hints to the example document.
@Witiko
Copy link
Contributor

Witiko commented Jun 6, 2024

Initially I have only tried the first part with BOs, but the pipeline failed and cannot find out the problem.

You have one extra alignment character (&) in the row with TA-BO6, please see https://github.com/istqborg/istqb-ctal-ta/commit/a7801f604d824a5a404e5a9185e5de0ecbd50d37#r142825658.

Later I have added all LOs, which makes the full table, but of course, with the same result.

Your full table from commit https://github.com/istqborg/istqb-ctal-ta/commit/18e3d5fd285231e73d9d29e0ec4f59a3342c6886 does not seem in line with the criteria from item 5 in #51 (comment). Namely, my code from #51 (comment) assumes that the table will fit on a single page and will only contain a single matrix. Both these assumptions seem broken by the full table, although it's difficult for me to see how the full table should look just by looking at the code from commit https://github.com/istqborg/istqb-ctal-ta/commit/18e3d5fd285231e73d9d29e0ec4f59a3342c6886. If you need a table that would span several pages, you may need to break the table into several smaller tables, each of which should fit on a single page and be written in a separate ```{=tex} block.

If all "traceability matrix" tables follow a unified format, perhaps it would be easier for you to write them in YAML or Markdown instead of LaTeX. This would make them easier to read, write, and automatically validate. Alternatively, I can produce the LaTeX tables for you if you provide me with screenshots or other examples of the expected output. However, this may slow you down depending on my availability; removing the need for writing tables in LaTeX should be significantly more time-efficient.

@Witiko
Copy link
Contributor

Witiko commented Jun 6, 2024

If all "traceability matrix" tables follow a unified format, perhaps it would be easier for you to write them in YAML or Markdown instead of LaTeX. This would make them easier to read, write, and automatically validate. Alternatively, I can produce the LaTeX tables for you if you provide me with screenshots or other examples of the expected output. However, this may slow you down depending on my availability; removing the need for writing tables in LaTeX should be significantly more time-efficient.

For example, I can encode the traceability matrix from #51 (comment) with the following simple YAML document:

matrix:
  1:  [Understand what testing is and why it is beneficial, 6]
  2:  [Understand fundamental concepts of software testing, 22]
  3:  [Identify the test approach and activities to be implemented depending on the context of testing, 6]
  4:  [Assess and improve the quality of documentation, 9]
  5:  [Increase the effectiveness and efficiency of testing, 20]
  6:  [Align the test process with the software development lifecycle, 6]
  7:  [Understand test management principles, 6]
  8:  [Write and communicate clear and understandable defect reports, 1]
  9:  [Understand the factors that influence the priorities and efforts related to testing, 7]
  10: [Works par of a cross-functional, 8]
  11: [Know risks and benefits related to test automation, 5]
  12: [Identify essential skills required for testing, 5]
  13: [Understand the impact of risk on testing, 4]
  14: [Effectively report on test progress and quality, 4]

Both "FL" and "Foundation Level" are already part of the metadata from the original document and don't need to be specified.

However, your full table from commit https://github.com/istqborg/istqb-ctal-ta/commit/18e3d5fd285231e73d9d29e0ec4f59a3342c6886 indicates that a traceability matrix can be more complex and would require a more general YAML format. If you could help me understand the anatomy of a traceability matrix, we should be able to find a better input format than LaTeX.

@danopolan
Copy link
Contributor Author

If there is a way how to create such a table easier than using LaTeX, that would be a much preferred.

The table is always a combination of LOs and BOs, and the table width and length depend on the number of BOs and LOs in the syllabus.

Attached, you can find the table for CTAL TA syllabus. All products follow this structure; just the content and respective sizes are different.

ISTQB_CTAL-TA_Appendix B table.docx

@Witiko
Copy link
Contributor

Witiko commented Jul 30, 2024

I designed a YAML schema for writing traceability matrices:

business-outcomes: map(str(), key=str())
learning-objectives: include('learning-objectives', required=False)
---
learning-objectives:
  sections: map(str(), key=any(str(), int()))
  objectives: map(include('objective'), key=str())
---
objective:
  description: str()
  k-level: str()
  business-outcome: str(required=False)
  business-outcomes: any(list(str()), str(), required=False)

For example, here is how we would write the traceability matrix from document ISTQB_CTAL-TA_Appendix B table.docx:

business-outcomes:
  BO-TA1: Support and perform appropriate testing based on the software development lifecycle followed
  BO-TA2: Apply the principles of risk-based testing
  BO-TA3: Select and apply appropriate test techniques to support the achievement of test objectives
  BO-TA4: Provide documentation at appropriate levels of detail and quality
  BO-TA5: Determine the appropriate types of functional testing to be performed
  BO-TA6: Contribute to non-functional testing
  BO-TA7: Contribute to the prevention of defects
  BO-TA8: Improve the efficiency of the test process with the use of tools
  BO-TA9: Specify the requirements for test environments and test data
learning-objectives:
  sections:
    1: The Test Analyst’s Tasks in the Test Process
    1.1: Testing in the Software Development Lifecycle
    1.2: Involvement in the test activities
    1.3: Tasks related to work products
    2: The Test Analyst's Tasks in Risk-Based Testing
    2.1: Risk Analysis
    2.2: Risk Control
    3: Test Analysis and Design
    3.1: Data-based Techniques
    3.2: Behavior-Based Techniques
    3.3: Rule-Based Techniques
    3.4: Experience-Based Test Techniques
    3.5: Applying the Most Appropriate Test Techniques
    4: Testing Software Quality Characteristics
    4.1: Functional testing
    4.2: Usability testing
    4.3: Flexibility testing
    4.4: Compatibility testing
    5: Software Defect Prevention
    5.1: Defect prevention practices
    5.2: Supporting phase containment
    5.3: Mitigating the recurrence of defects
  objectives:
    TA-1.1.1:
      description: Summarize the involvement of the Test Analyst in different SDLCs
      k-level: K2
      business-outcome: BO-TA1
    TA-1.2.1:
      description: Summarize the tasks performed by the TA as part of test analysis
      k-level: K2
      business-outcome: BO-TA1
    TA-1.2.2:
      description: Summarize the tasks performed by the TA as part of test design
      k-level: K2
      business-outcome: BO-TA1
    TA-1.2.3:
      description: Summarize the tasks performed by the TA as part of test implementation
      k-level: K2
      business-outcome: BO-TA1
    TA-1.2.4:
      description: Summarize the tasks performed by the TA as part of test execution
      k-level: K2
      business-outcome: BO-TA1
    TA-1.3.1:
      description: Differentiate between high-level and low-level test cases
      k-level: K2
      business-outcome: BO-TA4
    TA-1.3.2:
      description: Explain the quality criteria for test cases
      k-level: K2
      business-outcome: BO-TA4
    TA-1.3.3:
      description: Give examples of test environment requirements
      k-level: K2
      business-outcomes: [BO-TA4, BO-TA9]
    TA-1.3.4:
      description: Explain the test oracle problem and potential solutions
      k-level: K2
      business-outcomes: [BO-TA4, BO-TA9]
    TA-1.3.5:
      description: Give examples of test data requirements
      k-level: K2
      business-outcomes: [BO-TA4, BO-TA9]
    TA-1.3.6:
      description: Use keyword-driven testing to develop test scripts
      k-level: K2
      business-outcomes: [BO-TA1, BO-TA8]
    TA-1.3.7:
      description: Summarize the types of tools applied in managing the testware
      k-level: K2
      business-outcome: BO-TA8
    TA-2.1.1:
      description: Summarize the test analyst’s contribution to product risk analysis
      k-level: K2
      business-outcome: BO-TA2
    TA-2.2.2:
      description: Analyze the impact of changes to determine the scope of regression testing
      k-level: K4
      business-outcomes: [BO-TA2, BO-TA8]
    TA-3.1.1:
      description: Apply domain testing
      k-level: K3
      business-outcome: BO-TA3
    TA-3.1.2:
      description: Apply combinatorial testing
      k-level: K3
      business-outcome: BO-TA3
    TA-3.1.3:
      description: Summarize the benefits and limitaions of random testing
      k-level: K2
      business-outcome: BO-TA3
    TA-3.2.1:
      description: Explain CRUD testing
      k-level: K2
      business-outcome: BO-TA3
    TA-3.2.2:
      description: Apply state-based testing
      k-level: K3
      business-outcome: BO-TA3
    TA-3.2.3:
      description: Apply scenario-based testing
      k-level: K3
      business-outcome: BO-TA3
    TA-3.3.1:
      description: Apply decision table testing
      k-level: K3
      business-outcome: BO-TA3
    TA-3.3.2:
      description: Apply metamorphic testing
      k-level: K3
      business-outcome: BO-TA3
    TA-3.4.1:
      description: Prepare test charters for session-based testing
      k-level: K3
      business-outcome: BO-TA3
    TA-3.4.2:
      description: Prepare checklists supporting experience-based testing
      k-level: K3
      business-outcome: BO-TA3
    TA-3.4.3:
      description: Give examples for benefits and limitations of crowd testing
      k-level: K2
      business-outcome: BO-TA3
    TA-3.5.1:
      description: Select appropriate test techniques to mitigate product risks for a given situation
      k-level: K4
      business-outcomes: [BO-TA2, BO-TA3]
    TA-3.5.2:
      description: Explain the benefits of automating the test design
      k-level: K2
      business-outcome: BO-TA8
    TA-4.1.1:
      description: Differentiate between functional correctness, functional appropriateness and functional completeness testing
      k-level: K2
      business-outcome: BO-TA5
    TA-4.2.1:
      description: Explain how the test analyst contributes to usability testing
      k-level: K2
      business-outcome: BO-TA6
    TA-4.3.1:
      description: Explain how the test analyst contributes to adaptability and installability testing
      k-level: K2
      business-outcome: BO-TA6
    TA-4.4.1:
      description: Explain how the test analyst contributes to interoperability testing
      k-level: K2
      business-outcomes: [BO-TA3, BO-TA6]
    TA-5.1.1:
      description: Explain how the test analyst can contribute to defect prevention
      k-level: K2
      business-outcome: BO-TA7
    TA-5.2.1:
      description: Use a model of the test object to detect anomalies in a specification
      k-level: K3
      business-outcome: BO-TA7
    TA-5.2.2:
      description: Apply a review technique to a test basis document to find defects
      k-level: K3
      business-outcome: BO-TA7
    TA-5.3.1:
      description: Analyze test results to identify potential improvements to defect detection
      k-level: K4
      business-outcome: BO-TA7
    TA-5.3.2:
      description: Explain how defect classification supports root cause analysis
      k-level: K2
      business-outcome: BO-TA7

Here is how an author would include the traceability matrix from file matrix.yml in a document:

\markdownSetup {
  import = {
    istqb/syllabus = matrix,
  }
}
\markdownInput[snippet=matrix]{matrix.yml}

This would produce the entire appendix with the traceability matrix:

image
image
image
image
image

If the learning-objectives part of the YAML file is excluded, only the first page of the appendix would be produced with zeros on the diagonal. This allows authors to produce the YAML file incrementally.

@Witiko
Copy link
Contributor

Witiko commented Jul 30, 2024

@danopolan: Please, let me know if this is what you had in mind, so that I can implement it.

@Witiko
Copy link
Contributor

Witiko commented Jul 31, 2024

I designed a YAML schema for writing traceability matrices: [...]

Admittedly, there is some duplicity where the author writes learning objectives (LOs) twice: once at the beginnings of chapters and then again in the YAML file. If this is undesirable, we can collect the LOs from the chapter beginnings and only specify the business objectives (BOs) and the mapping between the LOs and BOs.

Here is how this might look like in a YAML schema:

business-outcomes: map(str(), key=str())
learning-objectives: map(list(str()), str(), key=str(), required=False)

Here is how we would write the traceability matrix from document ISTQB_CTAL-TA_Appendix B table.docx:

business-outcomes:
  BO-TA1: Support and perform appropriate testing based on the software development lifecycle followed
  BO-TA2: Apply the principles of risk-based testing
  BO-TA3: Select and apply appropriate test techniques to support the achievement of test objectives
  BO-TA4: Provide documentation at appropriate levels of detail and quality
  BO-TA5: Determine the appropriate types of functional testing to be performed
  BO-TA6: Contribute to non-functional testing
  BO-TA7: Contribute to the prevention of defects
  BO-TA8: Improve the efficiency of the test process with the use of tools
  BO-TA9: Specify the requirements for test environments and test data
learning-objectives:
  1.1.1: BO-TA1
  1.2.1: BO-TA1
  1.2.2: BO-TA1
  1.2.3: BO-TA1
  1.2.4: BO-TA1
  1.3.1: BO-TA4
  1.3.2: BO-TA4
  1.3.3: [BO-TA4, BO-TA9]
  1.3.4: [BO-TA4, BO-TA9]
  1.3.5: [BO-TA4, BO-TA9]
  1.3.6: [BO-TA1, BO-TA8]
  1.3.7: BO-TA8
  2.1.1: BO-TA2
  2.2.2: [BO-TA2, BO-TA8]
  3.1.1: BO-TA3
  3.1.2: BO-TA3
  3.1.3: BO-TA3
  3.2.1: BO-TA3
  3.2.2: BO-TA3
  3.2.3: BO-TA3
  3.3.1: BO-TA3
  3.3.2: BO-TA3
  3.4.1: BO-TA3
  3.4.2: BO-TA3
  3.4.3: BO-TA3
  3.5.1: [BO-TA2, BO-TA3]
  3.5.2: BO-TA8
  4.1.1: BO-TA5
  4.2.1: BO-TA6
  4.3.1: BO-TA6
  4.4.1: [BO-TA3, BO-TA6]
  5.1.1: BO-TA7
  5.2.1: BO-TA7
  5.2.2: BO-TA7
  5.3.1: BO-TA7
  5.3.2: BO-TA7

Here is how an author would include the traceability matrix from file matrix.yml in a document:

\markdownSetup {
  import = {
    istqb/syllabus = matrix,
  }
}
\markdownInput[snippet=matrix]{matrix.yml}

This would produce the entire appendix with the traceability matrix:

image
image
image
image
image

If the learning-objectives part of the YAML file is excluded, only the first page of the appendix would be produced with zeros on the diagonal. This allows authors to produce the YAML file incrementally.

@danopolan
Copy link
Contributor Author

The simplified solution without duplicating LOs is sleek and simple. So I would go with it.
LOs in chapter beginnings are the single source of truth, so it have to contain all LOs with K-levels. The traceability matrix just adds a different visualization.

Also, BOs are currently stored in chapter 0 (see TA syllabus) for all syllabi. We could get it from there easily as well, and avoid duplications.

@Witiko
Copy link
Contributor

Witiko commented Jul 31, 2024

LOs in chapter beginnings [...] have to contain all LOs with K-levels.

They contain both the LOs and their K-levels. Furthermore, they also contain the named sections and subsections that are used to structure the LOs and that are also included in the traceability matrix table.

BOs are currently stored in chapter 0 (see TA syllabus) for all syllabi.

I did not know this. We may need to add an explicit identifier for the business outcomes section such as {.business-outcomes}, so that we can easily locate the section with the table even in syllabi in other languages.

Since both LOs and BOs are already known, we may simplify the YAML schema even further to only contain the incidence matrix:

matrix: map(list(str()), str(), key=str())

Here is how we would write the traceability matrix from document ISTQB_CTAL-TA_Appendix B table.docx:

matrix:
  1.1.1: TA-BO1
  1.2.1: TA-BO1
  1.2.2: TA-BO1
  1.2.3: TA-BO1
  1.2.4: TA-BO1
  1.3.1: TA-BO4
  1.3.2: TA-BO4
  1.3.3: [TA-BO4, TA-BO9]
  1.3.4: [TA-BO4, TA-BO9]
  1.3.5: [TA-BO4, TA-BO9]
  1.3.6: [TA-BO1, TA-BO8]
  1.3.7: TA-BO8
  2.1.1: TA-BO2
  2.2.2: [TA-BO2, TA-BO8]
  3.1.1: TA-BO3
  3.1.2: TA-BO3
  3.1.3: TA-BO3
  3.2.1: TA-BO3
  3.2.2: TA-BO3
  3.2.3: TA-BO3
  3.3.1: TA-BO3
  3.3.2: TA-BO3
  3.4.1: TA-BO3
  3.4.2: TA-BO3
  3.4.3: TA-BO3
  3.5.1: [TA-BO2, TA-BO3]
  3.5.2: TA-BO8
  4.1.1: TA-BO5
  4.2.1: TA-BO6
  4.3.1: TA-BO6
  4.4.1: [TA-BO3, TA-BO6]
  5.1.1: TA-BO7
  5.2.1: TA-BO7
  5.2.2: TA-BO7
  5.3.1: TA-BO7
  5.3.2: TA-BO7

Here is how an author would include the traceability matrix from file matrix.yml in a document:

\markdownSetup {
  import = {
    istqb/syllabus = matrix,
  }
}
\markdownInput[snippet=matrix]{matrix.yml}

This would produce the entire appendix with the traceability matrix:

image
image
image
image
image

@danopolan
Copy link
Contributor Author

danopolan commented Jul 31, 2024

I did not know this. We may need to add an explicit identifier for the business outcomes section such as {.business-outcomes}, so that we can easily locate the section with the table even in syllabi in other languages.

I can do that for existing syllabi without any issues. It could bring simplification in setting up MD table for BOs.

Since both LOs and BOs are already known, we may simplify the YAML schema even further

That is great. Super simple input for complex tables, which otherwise took hours to set up even in Excel.

@Witiko
Copy link
Contributor

Witiko commented Jul 31, 2024

It could bring simplification in setting up MD table for BOs.

Definitely so. The authors could write the BOs as a list, like they do for LOs, and the template would cause them to be displayed as a table. For example, here is how the BOs in the TA syllabus might be written:

## Business Outcomes {.business-outcomes}

This section lists the Business Outcomes expected of a candidate who has achieved the Advanced Level Test Analyst certification.

An Advanced Level Test Analyst Certified Tester can:

1. Support and perform appropriate testing based on the software development lifecycle followed
2. Apply the principles of risk-based testing
3. Select and apply appropriate test techniques to support the achievement of test objectives
4. Provide documentation with appropriate levels of detail and quality
5. Determine the appropriate types of functional testing to be performed
6. Contribute to non-functional testing
7. Contribute to defect prevention
8. Improve the efficiency of the test process with the use of tools
9. Specify the requirements for test environments and test data

The prefix TA would be added from the metadata together with the text BO, so there is no need for the author to specify it.

Compare this with the current text:

## Business Outcomes

This section lists the Business Outcomes expected of a candidate who has achieved the Advanced Level Test Analyst certification.

An Advanced Level Test Analyst Certified Tester can…

| Code | Name |
| :--- | ---- |
| TA-BO1 | Support and perform appropriate testing based on the software development lifecycle followed |
| TA-BO2 | Apply the principles of risk-based testing |
| TA-BO3 | Select and apply appropriate test techniques to support the achievement of test objectives |
| TA-BO4 | Provide documentation with appropriate levels of detail and quality |
| TA-BO5 | Determine the appropriate types of functional testing to be performed |
| TA-BO6 | Contribute to non-functional testing |
| TA-BO7 | Contribute to defect prevention |
| TA-BO8 | Improve the efficiency of the test process with the use of tools |
| TA-BO9 | Specify the requirements for test environments and test data |

I can do that for existing syllabi without any issues.

Yes, you can add {.business-outcomes} to headings immediately. However, the potential conversion of the tables to lists seems best done together with the implementation of the traceability matrix tables. If I will be converting the tables to lists, adding {.business-outcomes} to the headings is almost no extra work for me.

@Witiko
Copy link
Contributor

Witiko commented Aug 2, 2024

In commit 3ae4b00 from PR #90, I added support for automatically collecting the Learning Objectives from the Markdown text and storing them into data structures during typesetting.

Here is an example code that recalls and formats the Learning Objectives from all chapters:

% Learning objectives
\istqbsection{Learning objectives}
\ExplSyntaxOn
\begin{itemize}
\seq_map_inline:Nn
  \g_istqb_learning_objective_chapter_numbers_seq
  {
    \item Chapter~#1:~{}
    \prop_item:Nn
      \g_istqb_learning_objective_chapter_names_prop
      { #1 }
    \begin{itemize}
    \prop_get:NnN
      \g_istqb_learning_objective_subchapter_numbers_prop
      { #1 }
      \l_tmpa_clist
    \clist_map_inline:Nn
      \l_tmpa_clist
      {
        \item Subchapter~#1.##1:~{}
        \prop_item:Nn
          \g_istqb_learning_objective_subchapter_names_prop
          { #1.##1 }
        \begin{itemize}
        \prop_get:NnN
          \g_istqb_learning_objective_numbers_prop
          { #1.##1 }
          \l_tmpb_clist
        \clist_map_inline:Nn
          \l_tmpb_clist
          {
            \item Learning~objective~#1.##1.####1:~{}
            \prop_item:Nn
              \g_istqb_learning_objective_names_prop
              { #1.##1.####1 }
            \begin{itemize}
            \item K-Level:~{}
            \prop_item:Nn
              \g_istqb_learning_objective_k_levels_prop
              { #1.##1.####1 }
            \end{itemize}
          }
        \end{itemize}
      }
    \end{itemize}
  }
\end{itemize}
\ExplSyntaxOff

Here is the output produced by the code when we append it to file example-document.tex:

image

The next steps are:

  • Add support for structured Business Outcomes in Markdown, so that we can automatically collect the Business Outcomes from the Markdown text and store them into data structures during typesetting.
  • Add support for ingesting Traceability Matrix YAML files and typesetting appendices with Traceability Matrix Tables based on Traceability Matrix YAML files and the Learning Objectives and Business Outcomes collected from the Markdown text.

@Witiko
Copy link
Contributor

Witiko commented Aug 11, 2024

In commit cf7f850 from PR #90, I added markup for Business Outcomes to the example document:

image

Then, in commit bb24022 from PR #90, I added code that collects and stores Business Outcomes in data structures.

Here is an example code that recalls and formats the Business Outcomes:

% Business outcomes
\istqbsection{Business outcomes}
\ExplSyntaxOn
\begin{itemize}
\seq_map_inline:Nn
  \g_istqb_business_outcomes_numbers_seq
  {
    \item[#1.]
    \prop_item:Nn
      \g_istqb_business_outcomes_names_prop
      { #1 }
  }
\end{itemize}
\ExplSyntaxOff

Here is the output produced by the code when we append it to file example-document.tex:

image

The next step is to add support for ingesting Traceability Matrix YAML files and typesetting appendices with Traceability Matrix Tables based on Traceability Matrix YAML files and the Learning Objectives and Business Outcomes collected from the Markdown text.

@Witiko
Copy link
Contributor

Witiko commented Aug 12, 2024

In commit ba30d4a from PR #90, I translated the headers of the Business Outcomes tables.

For example, see Section 0.4 from the document syllabus-de.tex in https://github.com/istqborg/istqb-ct-tas/pull/7:

## Geschäftsergebnisse {.business-outcomes}

In diesem Abschnitt werden die Geschäftsergebnisse aufgeführt, die von einem Kandidaten erwartet werden, der die Zertifizierung zum Test Automation Strategy Specialist erworben hat.

Ein Certified Tester Testautomatisierungsstrategie-Spezialist kann...

1. Faktoren von Software und Systemen verstehen, die den Erfolg der Testautomatisierung beeinflussen
1. Kosten und Risiken der Implementierung einer Testautomatisierungslösung identifizieren
1. Verstehen der Rollen und Verantwortlichkeiten von Personen, die an der Testautomatisierung beteiligt sind
1. Planen Sie die Integration der Testautomatisierung über Teststufen hinweg
1. Strategische Überlegungen zur Testautomatisierung in verschiedenen Softwareentwicklungslebenszyklus-Modellen identifizieren
1. Verstehen der Anwendbarkeit und Durchführbarkeit von Testautomatisierung
1. Planen von Testautomatisierungslösungen, die den organisatorischen Anforderungen entsprechen
1. Verstehen der Einsatzstrategien für die Testautomatisierung
1. Verstehen der Abhängigkeiten der Testautomatisierung innerhalb der Testumgebung
1. Verstehen der Kosten für die Einrichtung und Wartung von Testautomatisierung
1. Lernen, welche Metriken zur Testautomatisierung die Entscheidungsfindung unterstützen
1. Erkennen, wie Testautomatisierung einen Mehrwert für das Projekt und die Organisation bringt
1. Identifizieren Sie Anforderungen an die Testautomatisierung und die Testberichterstattung, um die Bedürfnisse der Stakeholder zu erfüllen
1. Definieren der Übergangsaktivitäten vom manuellen Testen zur Testautomatisierung
1. Definieren einer Testautomatisierungsstrategie, die sicherstellt, dass Projekte Ressourcen und Methoden gemeinsam nutzen, um eine konsistente Umsetzung innerhalb der Organisation zu gewährleisten

image

@Witiko
Copy link
Contributor

Witiko commented Aug 12, 2024

In commit e6e5e43, I added an example file example-document/traceability-matrix.yml:

traceability-matrix:
  1.1.1: 1
  1.1.2: 2
  1.1.3: [2, 3]
  1.2.1: [3, 4]
  1.2.2: [3, 4, 5]
  1.3.1: [4, 5, 6]

In commit df237cf from PR #90, I added snippet istqb/common/traceability-matrix that processes files traceability-matrix.yml.

Here is an example code that recalls and formats learning objectives, business outcomes, and the contents of the example file example-document/traceability-matrix.yml:

% Traceability Matrix
\markdownSetup {
  import = {
    istqb/syllabus = {
      traceability-matrix as matrix,
    }
  }
}
\begin{istqbappendices}
\markdownInput[snippet=matrix]{example-document/traceability-matrix.yml}
\istqbsection{Traceability matrix}
\istqbsubsection{Business objective co-occurences}
\ExplSyntaxOn
\begin{itemize}
\seq_map_inline:Nn
  \g_istqb_business_outcomes_numbers_seq
  {
    \item [ \g_istqb_prefix_tl - BO #1: ]
    \prop_get:NnNTF
      \g_istqb_business_outcomes_diagonal_prop
      { #1 }
      \l_tmpa_tl
      { \l_tmpa_tl{}~occurences }
      { No~occurences }
  }
\end{itemize}
\ExplSyntaxOff
\istqbsubsection{Learning objectives}
\ExplSyntaxOn
\begin{itemize}
\seq_map_inline:Nn
  \g_istqb_learning_objective_chapter_numbers_seq
  {
    \item Chapter~#1:~{}
    \prop_item:Nn
      \g_istqb_learning_objective_chapter_names_prop
      { #1 }
    \begin{itemize}
    \prop_get:NnN
      \g_istqb_learning_objective_subchapter_numbers_prop
      { #1 }
      \l_tmpa_clist
    \clist_map_inline:Nn
      \l_tmpa_clist
      {
        \item Subchapter~#1.##1:~{}
        \prop_item:Nn
          \g_istqb_learning_objective_subchapter_names_prop
          { #1.##1 }
        \begin{itemize}
        \prop_get:NnN
          \g_istqb_learning_objective_numbers_prop
          { #1.##1 }
          \l_tmpb_clist
        \clist_map_inline:Nn
          \l_tmpb_clist
          {
            \item Learning~objective~#1.##1.####1:~{}
            \prop_item:Nn
              \g_istqb_learning_objective_names_prop
              { #1.##1.####1 }
            \begin{itemize}
            \item K-Level:~{}
            \prop_item:Nn
              \g_istqb_learning_objective_k_levels_prop
              { #1.##1.####1 }
            \prop_get:NnNT
              \g_istqb_traceability_matrix_prop
              { #1.##1.####1 }
              \l_tmpa_clist
              {
                \item Related~business~outcomes:~{}
                \g_istqb_prefix_tl - BO
                \clist_use:Nn
                  \l_tmpa_clist
                  { ,~\g_istqb_prefix_tl - BO }
              }
            \end{itemize}
          }
        \end{itemize}
      }
    \end{itemize}
  }
\end{itemize}
\ExplSyntaxOff
\end{istqbappendices}

Here is the output produced by the code when we append it to file example-document.tex:

image

The next is to properly format the traceability matrix, as shown in #51 (comment) and #61 (comment).

@Witiko
Copy link
Contributor

Witiko commented Aug 13, 2024

In commits 6721b0a and e92b119 from PR #90, I updated the snippet istqb/common/traceability-matrix, so that it also typesets an appendix with the first page of the traceability matrix table, and I demonstrated the snippet on the example document:

image

The next step is to also typeset the second page of the traceability matrix table.

@Witiko
Copy link
Contributor

Witiko commented Aug 21, 2024

In commits 0e4e315 and c2ed15e from PR #90, I so that it typesets an appendix with the full traceability matrix table, and I demonstrated the snippet on the example document.

For example, here is an excerpt from the file example-document.tex:

\markdownSetup {
  import = {
    istqb/syllabus = {
      % [... other imports]
      traceability-matrix as matrix,
    },
  },
}

% [... the rest of the .tex file]

% Appendices
\begin{istqbappendices}

%% Traceability Matrix
\markdownInput[snippet=matrix]{example-document/traceability-matrix.yml}

\end{istqbappendices}

Here is the file example-document/traceability-matrix.yml:

traceability-matrix:
  1.1.1: 1
  1.1.2: 2
  1.1.3: [2, 3]
  1.2.1: [3, 4]
  1.2.2: [3, 4, 5]
  1.3.1: [4, 5, 6]

Here is the output produced:

image
image

@Witiko
Copy link
Contributor

Witiko commented Aug 21, 2024

@danopolan Besides your review of #90, please let me know if you'd like me to continue by adding traceability matrices to any existing syllabi. Having me do it allows me to iron out any kinks in the implementation from #90 that may only become apparent after typesetting an actual traceability matrix for actual learning objectives and business outcomes.

@danopolan
Copy link
Contributor Author

danopolan commented Aug 21, 2024

@Witiko this is great. We will or you could test traceability matrix rendering it in full scope for CTAL TA syllabus in beta branch. You are welcome to proceed with changes there.
But I would expect an update of the https://github.com/istqborg/istqb_product_template project as part of these changes.

I will implement it to other syllabi while we have TA implementation approved by @mhamburg as an Product Owner of TA syllabus.

@Witiko
Copy link
Contributor

Witiko commented Aug 21, 2024

[Y]ou could test traceability matrix rendering it in full scope for CTAL TA syllabus in beta branch.

I will be happy to later today. As I wrote in #61 (comment), I welcome the opportunity to try out the implementation on an actual syllabus before you do, so that I can fix the bugs before they impact the authors.

I assume the traceability matrix for the CTAL-TA syllabus is given by the document ISTQB_CTAL-TA_Appendix B table.docx that we discussed earlier in this ticket?

But I would expect an update of the https://github.com/istqborg/istqb_product_template project as part of these changes.

That's not a problem. An example traceability matrix is currently already part of the example document example-document.tex from repository istqb_product_base.

@danopolan
Copy link
Contributor Author

danopolan commented Aug 21, 2024

Traceability Matrix for TA v4.0 is attached.

ISTQB_CTAL-TA_Overview_v4.0_Alpha.docx

So YAML representation is:

traceability-matrix:
  1.1.1: 1
  1.2.1: 1
  1.2.2: 1
  1.2.3: 1
  1.2.4: 1
  1.3.1: 4
  1.3.2: 4
  1.3.3: [4,9]
  1.3.4: [4,9]
  1.3.5: [4,9]
  1.3.6: [1,8]
  1.3.7: 8
  2.1.1: 2
  2.2.2: [2,8]
  3.1.1: 3
  3.1.2: 3
  3.1.3: 3
  3.2.1: 3
  3.2.2: 3
  3.2.3: 3
  3.3.1: 3
  3.3.2: 3
  3.4.1: 3
  3.4.2: 3
  3.4.3: 3
  3.5.1: [2,3]
  3.5.2: 8
  4.1.1: 5
  4.2.1: 6
  4.3.1: 6
  4.3.1: [3,6]
  5.1.1: 7
  5.2.1: 7
  5.2.2: 7
  5.3.1: 7
  5.3.2: 7

@Witiko
Copy link
Contributor

Witiko commented Aug 21, 2024

In commit https://github.com/istqborg/istqb-ctal-ta/commit/27ffec3fda5fc2898f99bcd97da46d40d08d51a2, I added a traceability matrix to branch beta of the repository istqb-ctal-ta.

Here is the output:

image
image
image
image
image

@danopolan: Please, let me know if you would like anything changed.

@danopolan
Copy link
Contributor Author

Looks good to me. I have asked @mhamburg to check it out as well.

@mhamburg
Copy link

mhamburg commented Aug 22, 2024

The left column heading "Business Outcomes: Advanced Level" is incomplete, should contain the full syllabus name (Business Outcomes: Advanced Level Test Analyst).

@mhamburg
Copy link

The introductory text on lines 1749-1750 is not as in the Syllabus template v4.2.

@mhamburg
Copy link

Otherwise it looks good, thank you,

@Witiko
Copy link
Contributor

Witiko commented Aug 22, 2024

The left column heading "Business Outcomes: Advanced Level" is incomplete, should contain the full syllabus name (Business Outcomes: Advanced Level Test Analyst).

That's not a problem, I will add the full syllabus name.

The introductory text on lines 1749-1750 is not as in the Syllabus template v4.2.

I am sorry, I don't have access to the Syllabus template v4.2.

What would be the correct introductory text for the CTAL-TA syllabus according to the Syllabus template v4.2?

@danopolan
Copy link
Contributor Author

The text should be: "This section lists the traceability between the Business Outcomes and the Learning Objectives of <Module Name>."
Module name is Advanced Level Test Analyst

@Witiko
Copy link
Contributor

Witiko commented Aug 22, 2024

In commit 25bb5f7, I implemented the requested changes.

Here is how the traceability matrix of CTAL-TA looks after my changes:

image
image
image
image
image

Please, let me know if this is OK with you or if there are other changes that you would like.

@danopolan
Copy link
Contributor Author

This is fine. We can finally leave this task.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants