Incorporate AI/LLM Orientation into the Foundation Curriculum

[alxlo]

The Foundation curriculum contains no explicit treatment of AI or LLMs. Trainers are reporting consistent participant demand for orientation on how these tools and patterns connect to existing Foundation topics.

This topic cuts across the curriculum rather than sitting in one place. We expect trainers to interweave relevant content into existing learning goals - on quality characteristics, requirements, architectural patterns, and documentation - rather than adding a standalone section. What the curriculum needs to provide is the framing for that.

Trainer Feedback

Original (German):

Häufiges und auffälliges Teilnehmerfeedback: Auch im Foundation-Level bereits eine Orientierung zu iSAQB-Themen und AI-Touchpoints zu erhalten.

• Architekturmuster: probabilistisch vs. deterministisch (LLMs)
  • Qualitätsmerkmal: Erklärbarkeit (Explainability) (ist nicht im ISO Standard; hier gibt es Nachvollziehbarkeit, was sich eher auf Sicherheitsaspekte bezieht.)
  • Einordnung von Anforderungen unter dem Gesichtspunkt "Nicht-Determinismus"
• KI als Werkzeug
  • Codegenerierung & Analyse
  • Dokumentation oder Analyse von Legacy-Software
  • Bias in Modellen
    • Informationen aus KI-Entscheidungsfindung im Architekturkontext
    • Fairness-Metriken
• Knowledge Base
  • Darf ich für diese Funktion eine KI nutzen?
  • Human-in-the-loop
  • Bewertbarkeit von Informationen aus KI kann teilweise nur von Experten gemacht werden...
• Optimierung von Texten in Architekturdokumentation
  • Formulierungen einfacher verständlich
  • Formulierungen in der Fachsprache (LLM mit Glossar trainiert)
  • Formulierungen stakeholdergerecht (Zusammenfassungen mit einfachen Worten)

Translation (English):

Frequent and noticeable participant feedback: participants want orientation on iSAQB topics and AI touchpoints already at Foundation level.

• Architectural patterns: probabilistic vs. deterministic (LLMs)
  • Quality characteristic: Explainability (not in the ISO standard; the ISO standard has "traceability", which relates more to security aspects)
  • Classifying requirements from the perspective of "non-determinism"
• AI as a tool
  • Code generation & analysis
  • Documentation or analysis of legacy software
  • Bias in models
    • Information from AI decision-making in the architecture context
    • Fairness metrics
• Knowledge base
  • Am I allowed to use AI for this function?
  • Human-in-the-loop
  • Evaluating AI-generated information can in some cases only be done by domain experts...
• Optimising text in architecture documentation
  • Phrasing made more accessible
  • Phrasing in technical language (LLM trained on a glossary)
  • Phrasing appropriate for stakeholders (summaries in plain language)

Scope

Two follow up Advanced Level modules define what is out of scope here:

• SWARC4AI covers software architecture for AI systems. That territory stays there.
• AGENTA (upcoming) will cover how to use AI for software architecture work. The boundary between what belongs in Foundation and what belongs in AGENTA needs to be agreed before we add content.

The FL curriculum is already full. Any addition requires a corresponding reduction elsewhere, or an explicit decision to limit coverage to awareness level.

[alxlo]

I have deliberately left open whether this becomes a dedicated LG, is distributed across several existing LGs, or whether we introduce a catch-all LG along the lines of "How to deal with rapidly emerging technologies." The third option came up briefly in the T3WG: the idea being to create a frame that covers not just AI/LLMs but also absorbs future developments, so we do not have to reopen the curriculum with every new wave.

[n-nkm]

Somebody put me on this issues discussion list, so I will discuss. My stake with this is rather flimsy, but if an argument has to be made....

On the one hand AI shifts the way software is created. On the other hand it doesn't really change how software architecture is done (it does shift the opportunity costs massively which changes the discipline as a whole, but the way we evaluate things stay the same).

I believe anyone taking a training seriously needs to have an answer to how AI changes the software landscape so picking it up at a few stations is fair. I think it could serve to illustrate how an AI environment changes the viability of POCs or A/B testing as evaluation method for a decision or how maintainability could be achieved by either writing understandable code or just modularizing in a way where one building block can be recreated by an agent in little to no time (and so on and so on...). This could be put in contrast to environments where such technology is not available and how that changes decisionmaking. I think this would be valuable in understanding, seeing those parts shift might be a great specification by example.

I am not too worried about SWARC4AI as it deals in "usual" quality goals, so mentioning such examples is enough from time to time.

I think an LG of "Participants understand how current developments change the software architecture landscape and can describe the impact such changes have on their work (currently: AI)" would be an option, however it very much leaves a lot of freedom for trainers to implement it "how they see fit", while adding it to several LGs as "Needs to be discussed with AI as an example" will create more consistent mentions of the topic at controlled places and a more consistent foundation level overall. When AI either disappears or becomes the "default" we will have to remove them eventually though.

So I guess I prefer the second option and believe we should find 3-5 places where to add it.

From a council perspective it is annoying because we put AGENTA and SWARC4AI under restrictions to talk about external effects of AI usage, so picking up the topic in another curriculum poses the question on whether we expect the same and where to put it (should we just put a statement on the iSAQB website?) (summoning @mikesperber for this one)
Simply for "mentioning" AI having to do a full disclaimer seems unfair too.....

Edit: I am advocating to cut into 03-09 as it is on a software development level and only further illustrates what should already be explained in 03-08 patterns. I haven't given it a lot of thought though.

I also think not adding an LG would be a fair assessment.

[mikesperber]

From a council perspective it is annoying because we put AGENTA and SWARC4AI under restrictions to talk about external effects of AI usage, so picking up the topic in another curriculum poses the question on whether we expect the same and where to put it (should we just put a statement on the iSAQB website?) (summoning @mikesperber for this one) Simply for "mentioning" AI having to do a full disclaimer seems unfair too.....

The summonee abides. Indeed.

As the iSAQB presently has no place to put position statements (even though such statements exist, i.e. on digital sovereignty), it would need to be in the FL curriculum as well.

[alxlo]

Representing the stance of the FL WG we decided that:

• the topic WILL be INCLUDED in the new curriculum
• we have to identify which existing LGs are affected / have cross references here in order to decide whether this
  • A) ends up as distributed additions to exiting LGs,
  • B) ends up in a single place as either (B1) a new LG or (B2) a whole chapter, or
  • C) both of A and B

@mikesperber and @bertjan will evaluate this with help by @gernotstarke and @alxlo