Introduction to Responsible AI

Slides — Chapter 1

Fei Huang, UNSW Sydney

Table of contents

• Part 1: Why Responsible AI?
• Part 2: Six Principles of AI Ethics
• Part 3: The Ethical AI Lifecycle
• Break (10 min)
• Part 4: Regulatory Context
• Part 5: Course Roadmap

About the instructor

Dr. Fei Huang

Associate Professor, Risk and Actuarial Studies, UNSW Business School

feihuang@unsw.edu.au · feihuang.org

Her research and teaching focus on responsible AI, insurance, and data-driven decision-making that is accurate, interpretable, and equitable.

Course materials and assignments

• Lecture notes are published at responsible-ai.feihuang.org
• An assignment is issued after each lecture
• Many questions are open-ended, to accommodate the multidisciplinary nature of this class
• Expect roughly 2 hours to complete each assignment
• You will have 1 week to submit

Today’s roadmap

2-hour session, five parts, five discussion breaks

Time	Part
0:00 – 0:20	Why Responsible AI?
0:20 – 0:40	Six principles of AI ethics
0:40 – 1:05	The ethical AI lifecycle
1:05 – 1:15	Break
1:15 – 1:45	Regulatory context
1:45 – 2:00	Course roadmap

Full detail for every slide lives in Lecture-Introduction.qmd (the reading notes). These slides are the in-class version, condensed, with discussion prompts built in. Adjust timing live. The discussion breaks are the first thing to trim if running behind.

Learning objectives

By the end of this chapter, you should be able to:

Responsible AI: designing, deploying, and governing AI so that automated decisions are fair, explainable, and privacy-respecting, not just accurate.

• Explain why responsible AI matters for consequential automated decisions (decisions that meaningfully affect a person’s life)
• Describe six core AI ethics principles
• Outline a lifecycle view of ethical AI
• Identify key regulatory developments across jurisdictions
• Map fairness, explainability, and privacy to this course

Note

Insurance recurs as a worked example, but the principles apply to any consequential automated decision: hiring, lending, healthcare, criminal justice.

Part 1: Why Responsible AI?

The promise

• More accurate risk assessment and resource allocation
• Faster, more consistent screening of applications, claims, cases
• New data sources → finer-grained decisions
• Automation frees professionals for higher-value work

Machine learning and AI are transforming decision-making across industries: actuaries pricing insurance risk, underwriters assessing loans, HR teams screening applicants, and clinicians triaging patients are all working alongside automated systems now. This slide is the optimistic case. The next slide is the tension.

The problem

• Consequential decisions affecting access, pricing, and opportunity, all at scale
• Complexity obscures how decisions are made
• Data reflects historical patterns, including past discrimination
• Errors and biases scale automatically across large populations

Responsible AI is not about slowing innovation — it is about making innovation defensible.

Responsible AI is interdisciplinary

No single field is sufficient on its own:

• Domain expertise: what counts as a fair or reasonable outcome, in context
• Statistics and machine learning: the modelling and measurement tools
• Law: the binding obligations
• Economics: the trade-offs and incentives at stake
• Risk management: embedding it all into governance

It also needs collaboration across stakeholders: customer advocates, regulators, and industry practitioners, and across professions within a single firm: actuaries, data scientists, lawyers, risk officers, business leaders.

A failure is not always obvious

Illustrative example: hidden proxy discrimination

US insurers have used credit-based insurance scores since the early 1990s. The scores are genuinely predictive of claims cost.

A 2007 FTC study found the score is far from race-neutral. It placed over 25% of Black consumers in the lowest, most expensive score band, against 3% of white consumers (Kiviat 2019).

Removing race as an input wouldn’t have fixed this. Credit history already tracks it.

Guidance from Australia’s Human Rights Commission and the Actuaries Institute makes the same point directly: removing a protected attribute from a dataset is not sufficient, since other data may still act as a proxy for it (Australian Human Rights Commission and Actuaries Institute 2022). The regulatory aftermath: investigations in 17+ US states, five congressional hearings, and four states banning the practice outright. Every other state instead imposes constraints such as disclosure and hardship exemptions. Chapter 8 returns to this case in depth.

The same pattern recurs across sectors

Healthcare

A widely used US algorithm allocated care-management resources by predicted cost rather than predicted illness, systematically under-referring Black patients, who historically incurred lower costs for the same level of need (Obermeyer et al. 2019).

Hiring

Amazon scrapped an internal hiring tool after finding it penalised résumés containing the word “women’s,” a pattern learned from a decade of mostly male submissions (Dastin 2018).

Credit

Apple Card’s credit-limit algorithm drew a New York regulatory investigation after customers reported women receiving lower limits than their husbands despite similar finances, though the investigation found no fair-lending violation (New York State Department of Financial Services 2021).

China: e-commerce

Platforms charging loyal returning customers more than new customers for the same product, dàshùjù shāshú (“big data exploiting familiar customers”), became common enough to prompt an explicit ban in the 2022 Algorithm Recommendation Provisions (Cyberspace Administration of China 2022).

Questions that follow from all four: is this fair, by which definition? Can the org explain the outcome to a regulator? What about the privacy of the data that informed it? Who’s accountable? Fairness, explainability, and privacy get a quantitative toolkit in this course. Accountability is addressed too, mainly through governance practice in Chapter 8.

💬 Discuss

Think of an automated decision system you’ve encountered: a loan, a job application, an insurance quote, a hiring screen.

What data might it use as an unintentional proxy for a protected attribute (a characteristic like race, gender, or age that anti-discrimination law protects)?

Take a moment to think it through, then we’ll take a few answers.

Indicative answers: proxy attributes

Credit and lending

Postcode can proxy for race, a legacy of historical redlining. Employment gaps can proxy for gender, from career breaks tied to childcare.

Hiring

Graduation year can proxy for age. Specific résumé wording can proxy for gender, as with Amazon’s tool penalising “women’s.”

Insurance pricing

A credit-based score can proxy for race, as in this chapter’s own worked example. Vehicle engine size and mileage can proxy for gender and age, though each also carries genuine risk signal, which is what makes them harder to simply exclude.

“Proxy-ness” is a matter of degree, not a yes/no flag.

Push students past the label to the mechanism: name the specific correlated feature, and explain why it correlates (a historical pattern like redlining, a structural factor like caregiving responsibilities falling disproportionately on women, or a genuine but legally protected correlation like engine size and gender). A hiring example worth adding if it doesn’t come up: video-interview tone or facial-expression scoring as a proxy for disability or cultural background.

Part 2: Six Principles of AI Ethics

Six principles

1. Fairness and non-discrimination: no unjustified disparate outcomes
2. Transparency and explainability: interpretable to developers and affected parties
3. Accountability: clear who is responsible when AI causes harm
4. Privacy and data ethics: rights over collection, use, sharing, retention
5. Contestability: meaningful channels to challenge and appeal
6. Stability and robustness: reliable under drift, adversarial inputs, time

This course develops three of these six quantitatively: fairness, explainability, privacy. Reid Blackman calls this same trio AI’s “Big Three” ethical risks (Blackman 2022).

(Huang 2025). Walk through each briefly: (1) direct vs indirect discrimination: proxies count; (2) two levels: how the system works, and why this decision; (3) developer vs deploying org vs regulator vs certifying professional; (4) third-party data, telematics (e.g., GPS-tracked driving data), behavioural data raise new tensions; (5) increasingly embedded in regulation: GDPR (the EU’s General Data Protection Regulation) Art 22, Australian Privacy Act reforms; (6) a model fair and accurate at deployment may not stay that way two years later.

Why these three specifically: each already has a maturing toolkit of quantitative methods (the fairness criteria, explainability techniques, and privacy methods this course covers) with precise, well-established definitions, even though choosing among them is itself a domain-specific judgment call. Blackman singles out the same trio in Ethical Machines (2022) as AI’s “Big Three” ethical risks. Accountability, contestability, and stability still matter. This course treats them as governance and monitoring practices rather than as separate quantitative modules.

Three pillars

A similar cast of stakeholders recurs across all three pillars: the people affected by the decision, the regulators who oversee that domain, and the organisation deploying the system. What differs is the question each pillar asks.

Pillar	Core question
Fairness	Does the model treat people equitably?
Explainability	Can decisions be understood and justified?
Privacy	Are personal data used appropriately?

Same three questions, different named stakeholders, in other domains: applicants and hiring managers, borrowers and credit regulators, patients and health data custodians. Each pillar has a principled dimension and a quantitative dimension, and this course addresses both.

From principles to practice

How do we operationalise “fairness” or “explainability” in a way that is measurable, auditable, and defensible?

Principle	Covered in
Fairness	Ch 2–3
Explainability	Ch 4–5
Privacy	Ch 6–7
Accountability	Ch 2, 4, 8
Contestability	Ch 2, 4–5
Stability & robustness	Ch 8

💬 Discuss

Of the six principles, which do you think is hardest to turn into something measurable and auditable, and why?

Indicative answer: hardest to measure

Most commonly nominated: accountability and contestability

Accountability is an organisational-structure question (is responsibility clearly assigned?), not a property of the model, so it’s audited through process evidence rather than a statistical test. Contestability has no single metric either. It’s measured through process indicators like the share of contested decisions overturned.

Also defensible: stability and robustness

Drift and adversarial-robustness tests exist, but operationalising “reliable over time” needs ongoing monitoring infrastructure, not a one-off number.

Hard to measure doesn’t mean unimportant. It’s exactly why fairness, explainability, and privacy, which already have a mature quantitative toolkit, are this course’s three pillars.

Accountability, contestability, and stability are treated as governance and monitoring practices in this course, mainly through Chapter 8, rather than as separate quantitative modules. That framing is a deliberate scoping choice, not a claim that they matter less.

Part 3: The Ethical AI Lifecycle

AI does not fail at deployment

“The most frequent failure in data analysis is mistaking the type of question being considered.” — Leek and Peng (2015)

A common source of AI failure lies in problem formulation, not the model: the wrong question, the wrong outcome variable, the wrong sense of who’s affected.

Ethical risk lives at every stage of the lifecycle, not just at deployment.

A six-stage lifecycle view

The ethical AI lifecycle. Source: Huang (2025).

Stage 1 Problem definition: does the objective align with fairness, transparency, regulatory standards, is optimisation balanced between firm profitability and customer welfare, who owns AI use, oversight, and risk? Stage 2 Data collection: does the data invade privacy, do proxies like postcode or occupation encode sensitive attributes indirectly, is consent meaningful, does the data reflect historical bias? Stage 3 Exploratory data analysis: could input features have discriminatory effects directly or through correlation with protected groups, can each feature be justified to regulators and customers? Stage 4 Modelling: does the model balance accuracy with fairness, explainability, transparency, can outputs be assessed for disparities across groups? Stage 5 Evaluation: does performance hold up across subgroups, do results generalise under different scenarios, have trade-offs been communicated? Stage 6 Deployment: are models audited regularly after launch, can customers appeal or contest an outcome, is there a feedback loop into future iterations?

A practitioner’s role

Domain professionals are increasingly involved at every stage, not just development and validation, but governance, risk assessment, stakeholder communication. Responsible AI is a professional obligation.

Six types of question

Leek and Peng (2015): most analysis failures come from answering the wrong type of question, not answering the right one badly.

Type	Asks
Descriptive	What happened?
Exploratory	What patterns exist?
Inferential	What can we infer about a population from a sample?
Predictive	What is likely to happen?
Causal	Does X cause Y?
Mechanistic	How exactly does the system work?

Examples: descriptive, average claim size by policy type. Exploratory, do interview scores cluster by which recruiter conducted them. Inferential, is this sample’s average loan default rate representative of the full borrower population. Predictive, how many days is this patient expected to stay in hospital. Causal, does raising the excess (the deductible a policyholder pays before cover starts) reduce claim frequency. Mechanistic, how a drug interacts biologically to lower blood pressure. (Peng and Matsui 2015)

Descriptive vs exploratory: how would you tell these two apart?

Inferential vs predictive vs causal: how would you tell these three apart?

Causal vs mechanistic: how would you tell these two apart?

Mistaking a descriptive or exploratory pattern for a causal one is an ethical failure, not only a technical one. A pattern that reflects historical bias or a proxy for a protected attribute (postcode, gender) can get treated as if it justifies risk, laundering that bias into the model.

💬 Discuss

A consulting firm has two teams working for two different clients, each asked a predictive question.

Team A works for a major supermarket, forecasting stock demand. No individual is priced or screened based on the output.

Team B works for an auto insurer, predicting claims for individual policyholders, to set their premiums.

For each team, what should drive model class, feature selection, and evaluation criteria? Why might the same question type lead to different choices?

Take a moment to think it through, then we’ll take a few answers.

Discussion: same question, different stakes

Team A — retail demand forecasting

No individual is priced or screened, so accuracy is the dominant consideration. A complex, high-accuracy model is the right choice.

Team B — insurance pricing

Same question shape, applied to individuals, changes everything. Interpretability now matters as much as accuracy, feature selection needs scrutiny for proxies, and fairness metrics join the evaluation.

Both teams ask a predictive question. Only Team B is making a consequential decision about an identifiable individual, and that difference, not the question type alone, is what should drive model choice, feature selection, and evaluation criteria.

Break (10 min)

Part 4: Regulatory Context

A rapidly changing landscape

A jurisdiction-by-jurisdiction survey. Chapters 2, 4, and 6 develop the specific fairness, explainability, and privacy mechanisms in depth.

United States: sector-by-sector

No federal AI law. States and sector regulators lead:

• Colorado SB21-169 (2021) (Colorado General Assembly 2021): first US law on algorithmic discrimination in insurance
• NYC Local Law 144 (2021) (New York City Council 2021): independent bias audits for automated employment decision tools
• NY DFS (Department of Financial Services) Circular Letter No. 7 (2024) (New York State Department of Financial Services 2024): bias testing, governance, explainability to consumers
• CFPB and ECOA (2023) (Consumer Financial Protection Bureau 2023): adverse-action notice duty (the duty to tell a rejected applicant why) applies to AI-driven credit decisions

Colorado SB21-169 prohibits insurers from using external consumer data in ways that unfairly discriminate based on race, colour, national or ethnic origin, religion, sex, sexual orientation, disability, gender identity, or gender expression. NYC LL144 requires the audit each year, requires publishing the results, and requires notifying candidates a tool is in use, not just the audit itself. More sector rules: NAIC (National Association of Insurance Commissioners) Model Bulletin (2023) gives US state insurance regulators guidance on governance and third-party model oversight. Illinois’s AI Video Interview Act regulates AI-based hiring assessments specifically. Insurance and employment have moved fastest so far, but sector-by-sector regulation is spreading to lending, healthcare, and beyond.

European Union: horizontal regulation

EU AI Act (2024) (European Parliament and Council of the European Union 2024): risk-based, applies across sectors.

High-risk (Annex III) includes life and health insurance risk assessment and pricing, employment, credit, education, law enforcement:

• Conformity assessment (a formal check that a system meets legal requirements) before deployment
• Mandatory risk management + data governance
• Transparency, documentation, human oversight
• Post-market monitoring

Life and health insurance risk assessment and pricing is one Annex III category among several, and doesn’t extend to other insurance lines like the motor insurance examples used in this course. The Act treats it as structurally similar to employment and credit decisions, not a special case. GDPR predates the AI Act by six years and remains the more consequential instrument in practice for most organisations. The AI Act applies on top of GDPR’s existing Article 22 obligations on automated decision-making, not instead of them.

Australia: principles-based, multi-regulator

No standalone AI Act, existing sector laws apply, enforced in parallel by regulators such as ASIC (the Australian Securities and Investments Commission). The absence of a new AI law does not mean an absence of accountability.

Important

Obligations don’t transfer to a vendor. If a third-party model produces unfair outcomes, the regulator looks to the deploying organisation (Parra-Orlandoni and Carvão 2026).

Australia is not the EU

	EU AI Act	Australia (2026)
Legislation	Standalone AI-specific Act	No standalone AI Act (yet)
Coverage	Mandatory “high-risk” category	Existing laws apply to AI
Regulator	Single AI regulator	Multi-regulator (ASIC, APRA, OAIC…)
Enforcement	Prohibitions and fines	Standards-led, risk-based

No new AI law ≠ no accountability. The regulators are already watching.

Case Study: ASIC REP 798

A governance failure

A licensee deployed an AI credit-default model, no AI strategy, no policies, no risk rating. Review ten months in found “limited understanding” of the third-party platform, “incomplete model documentation,” “poor governance.” Described as a “black box.” The licensee kept using it for months more.

Source: ASIC Report 798 (Oct 2024) (Australian Securities and Investments Commission 2024)

At what point does continuing to use a system you can’t explain stop being an operational gap and become an ethical failure?

ASIC published 11 questions it expects licensees to answer about any AI system. REP 798 found only 12 of 23 firms reviewed had fairness policies, and half hadn’t updated risk frameworks for AI. Also cover: APRA (Australian Prudential Regulation Authority) CPS 230 (board-approved operational risk frameworks, in force July 2025); OAIC (Office of the Australian Information Commissioner)/Privacy Act reforms (Dec 2026, automated-decision disclosure); FAR (Financial Accountability Regime; personal civil penalties up to A$1.56M).

The trust gap

Australia has the lowest AI trust of any country surveyed

• 30% of Australians believe AI’s benefits outweigh its risks — the lowest of any country in the study
• 78% are concerned about negative outcomes from AI
• 30% think current safeguards and regulation are adequate

Source: KPMG & University of Melbourne, Trust, Attitudes and Use of AI: A Global Study 2025 (KPMG and University of Melbourne 2025)

This is the public REP 798 is written for. A governance failure like the one on the previous slide is exactly what erodes the 30% further.

Australia: what’s coming

Timing	Development
Oct 2025	AI6 guidance released (Ch 8)
Dec 2025	National AI Plan, mandatory guardrails shelved
Early 2026	AI Safety Institute launched
Dec 2026	Privacy Act ADM disclosure mandatory

China: binding, sector-specific

No comprehensive AI law yet, though the State Council’s 2026 Legislative Work Plan calls for “accelerating comprehensive legislation” (General Office of the State Council of the People’s Republic of China 2026). Sector rules cover most consequential use:

• NFRA (National Financial Regulatory Administration) Guidance on AI in Banking & Insurance (2026) (National Financial Regulatory Administration (China) 2026): applies directly to banking and insurance institutions
• PIPL (Personal Information Protection Law) Article 24 (2021) (National People’s Congress (China) 2021): transparency, fair results, right to explanation (the ability to ask why an automated decision was made)
• Algorithm Recommendation Provisions (2022) (Cyberspace Administration of China 2022): bans “big data price discrimination” (charging a customer a worse price using their own data) where it applies

NFRA guidance (Jin Fa [2026] No. 8) covers risk classification and high-risk application approval, human oversight of high-risk applications, transparency, avoiding algorithmic discrimination, data and cybersecurity protection, and procedural requirements for externally introduced GenAI models. PIPL Art 24 bans “unreasonable differential treatment… in transaction prices.” This is targeted, not a blanket ban on price differentiation. A more directly insurance-specific fair-pricing rule, Article 27 of the Measures for the Administration of Consumer Rights Protection by Banking and Insurance Institutions, is covered in Chapter 2 alongside the Algorithm Recommendation Provisions above. Generative AI Interim Measures (2023) require security assessment and registration before launch, plus content-labelling rules (2025) (Cyberspace Administration of China and Ministry of Industry and Information Technology and Ministry of Public Security and National Radio and Television Administration 2025), which mandate two forms of labelling: a visible label on the content itself (a watermark, tag, or icon) so viewers can tell it’s AI-generated, and an embedded label in the file’s metadata that persists even if the visible label is stripped, so platforms and regulators can still trace it after it’s shared or reposted.

Singapore: voluntary, principles-based

• Model AI Governance Framework (2019, updated 2020) (Personal Data Protection Commission Singapore and Infocomm Media Development Authority 2020): extended for GenAI (2024) and Agentic AI (2026)
• AI Verify (AI Verify Foundation 2023): open-source governance testing toolkit
• MAS (Monetary Authority of Singapore) FEAT (Fairness, Ethics, Accountability, Transparency) Principles (Monetary Authority of Singapore 2018) + Veritas Initiative (Monetary Authority of Singapore 2019): financial-sector fairness methodology

Note

FEAT/Veritas ask how to define, measure, justify, monitor fairness, the financial-sector counterpart to Chapter 2’s criteria, developed independently by a regulator.

AI Verify is stewarded by the AI Verify Foundation, launched June 2023 as a global open-source community, building on Singapore’s 2022 pilot. FEAT’s four dimensions: Fairness, no discriminatory outcomes, considered throughout the model’s lifecycle. Ethics, aligned with institutional values and societal expectations, human oversight retained over consequential decisions. Accountability, clear responsibility for AI outcomes, including at board and senior-management level. Transparency, internal documentation plus meaningful explanations to affected customers where appropriate. Veritas was developed across several phases from 2020 to 2023, with use cases in credit risk scoring and customer marketing for banks, and predictive underwriting and fraud detection for insurers, backed by an open-source toolkit that automates fairness-metric assessment.

Common themes across jurisdictions

1. Governance and accountability
2. Fairness testing and bias assessment
3. Explainability and transparency
4. Privacy and data protection

Themes 2–4 map directly onto this course’s three pillars. Governance/accountability is cross-cutting, addressed within each pillar’s methods (the audit protocol in Ch 2, documentation throughout) rather than as a fourth pillar. Single Act vs sectoral rules vs voluntary frameworks is a question of regulatory strategy. The underlying obligations converge.

💬 Discuss

If you were designing AI regulation from scratch, the EU’s single comprehensive law, or the US/Australia’s sector-by-sector approach?

What’s the trade-off?

Indicative answer: regulatory design trade-off

Comprehensive law (EU)

Consistent definitions and obligations across sectors and member states, less regulatory arbitrage, one compliance standard for multinational firms. Costs: slow to negotiate, risks being too generic for sector-specific nuance, harder to amend as the technology moves.

Sector-by-sector (US/Australia)

Faster to respond to a specific, visible harm, such as NYC’s bias-audit law following hiring-tool harms directly. Rules fit existing sector regulators. Costs: gaps and inconsistency across sectors and states, and a tendency to regulate fastest where harm is most visible, leaving newer applications under-regulated until something goes wrong.

The trade-off: consistency and predictability vs speed and tailoring. Australia’s mix of existing sector law plus emerging cross-cutting guidance is itself a middle path between the two extremes.

No single correct choice here. Push students to connect their answer back to compliance cost, the regulatory expertise required, and the pace of technological change, not just abstract principle.

Part 5: Course Roadmap

Structure: principles and practice

Ch	Title	Focus
2	Fairness Principles	Criteria, model designs
3	Fairness Practice	French motor insurance (R)
4	Explainability Principles	PFI, PDP, ALE, SHAP, LIME
5	Explainability Practice	XGBoost insurance model (Python)
6	Privacy Principles	k-Anonymity, DP, synthetic data
7	Privacy Practice	Insurance micro-data (R)
8	Trade-offs & Integration	Systemic risk, governance

Case Study: COMPAS

COMPAS recidivism scoring

ProPublica’s 2016 Machine Bias investigation (Angwin et al. 2016) found that among defendants who did not reoffend, Black defendants were flagged “high risk” at nearly twice the rate of white defendants. The vendor countered COMPAS was well-calibrated, same reoffense rate within each score band, across race.

Both sides had a point. A tool can be calibrated (sufficiency) and still violate error-rate parity (separation). Chapters 2–3 give you this vocabulary.

Case Study: UK A-level algorithm

Ofqual’s 2020 grading algorithm

COVID cancelled exams. An algorithm converted teacher predictions into final grades (Centre for Multilevel Modelling, University of Bristol 2020), downgrading ~40%, leaning on each school’s history, so state-school students were downgraded more than students at small, historically high-performing private schools.

No one could get “why this grade?” answered before results day. Public backlash, including the Prime Minister calling it a “mutant algorithm,” forced a reversal within days. Chapters 4–5 cover the tools that could have quantified this beforehand.

Case Study: Cambridge Analytica

Cambridge Analytica (2018)

Data on ~87M Facebook users, collected via a “personality quiz” under the guise of research, repurposed for political micro-targeting without consent. FTC (the U.S. Federal Trade Commission) fined Facebook $5B (Federal Trade Commission 2019).

Not a breach, but a repurposing. Data collected for one purpose, used for another. Chapters 6–7 cover this exact boundary.

A quantitative course

By the end, you should be able to:

• Fit and compare fairness-aware models and quantify the trade-off
• Interpret complex models with state-of-the-art explanation tools
• Assess and reduce re-identification risk
• Generate and evaluate synthetic data
• Communicate technical results to non-technical stakeholders

Principles chapters draw examples from insurance, lending, hiring, healthcare, criminal justice alike. Practice chapters apply methods to real insurance datasets as a consistent worked case study (R, Python, XGBoost, SHAP, sdcMicro, synthpop). The same methods adapt to other domains, though outcomes, fairness considerations, and legal context differ.

Connecting the pillars

Illustrative example: the telematics insurer

GPS + accelerometer data → distance, time of day, speeding, acceleration, braking.

• Fairness: proxies for age, gender, socioeconomic status?
• Explainability: can the insurer explain which factors drove the premium (the price paid for coverage)?
• Privacy: legal basis for continuous location data? Third-party sharing? Deletion rights?

Same three-way tension in workplace productivity-monitoring software and health wearables feeding insurance or employer wellness programs. A complete responsible AI framework must address all three pillars together.

💬 Discuss: wrap-up

Of the three case studies today (COMPAS, UK A-levels, Cambridge Analytica), which failure would be hardest to fix with a purely technical solution, and why?

Indicative answers: which failure is hardest to fix

COMPAS

Has a real technical dimension, the calibration-vs-error-rate-parity impossibility result from Chapters 2–3, but is ultimately a policy choice about which fairness criterion to prioritise when base rates differ. Better modelling alone can’t resolve that.

UK A-level algorithm

Arguably the most technically fixable: the failure was a lack of transparency and no tested appeal channel before results day, both addressable with Chapters 4–5 tools.

Cambridge Analytica

The hardest to fix technically. A consent and purpose-limitation failure, not a modelling error, so no technical safeguard fixes it. It needs legal and organisational controls instead.

Most real failures need technical tools combined with governance and policy choices. That’s exactly Chapter 8’s theme.

Recommended reading

• Huang (2025): ethical AI lifecycle framework
• Barocas and Selbst (2016): legal/technical treatment of disparate impact

Chapters 2–7 each have their own reading list.

Next class

Chapter 2: Fairness Principles

Bring one example (from any domain) of a decision you think might be unfair, and why.

AI Verify Foundation. 2023. “AI Verify Foundation.” https://aiverifyfoundation.sg/.

Angwin, Julia, Jeff Larson, Surya Mattu, and Lauren Kirchner. 2016. “Machine Bias.” ProPublica. https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing.

Australian Human Rights Commission and Actuaries Institute. 2022. Guidance Resource: Artificial Intelligence and Discrimination in Insurance Pricing and Underwriting. https://humanrights.gov.au/resource-hub/by-resource-type/publications/technology-and-human-rights/guides/guidance-resource-ai-and-discrimination-insurance.

Australian Securities and Investments Commission. 2024. Report 798: Beware the Gap — Governance Arrangements in the Face of AI Innovation. https://www.asic.gov.au/regulatory-resources/find-a-document/reports/rep-798-beware-the-gap-governance-arrangements-in-the-face-of-ai-innovation/.

Barocas, Solon, and Andrew D Selbst. 2016. “Big Data’s Disparate Impact.” California Law Review 104: 671–732.

Blackman, Reid. 2022. Ethical Machines: Your Concise Guide to Totally Unbiased, Transparent, and Respectful AI. Harvard Business Review Press.

Centre for Multilevel Modelling, University of Bristol. 2020. The 2020 GCSE and a-Level ’Exam Grades Fiasco’: A Secondary Data Analysis of Students’ Grades and Ofqual’s Algorithm. https://www.bristol.ac.uk/cmm/research/grade/.

Colorado General Assembly. 2021. Senate Bill 21-169: Protecting Consumers from Unfair Discrimination in Insurance Practices. Colorado Division of Insurance. https://doi.colorado.gov/for-consumers/sb21-169-protecting-consumers-from-unfair-discrimination-in-insurance-practices.

Consumer Financial Protection Bureau. 2023. “Consumer Financial Protection Circular 2023-03: Adverse Action Notification Requirements and Proper Use of the CFPB’s Sample Forms.” https://www.federalregister.gov/documents/2024/04/17/2024-08003/consumer-financial-protection-circular-2023-03-adverse-action-notification-requirements-and-proper.

Cyberspace Administration of China. 2022. Provisions on the Administration of Algorithmic Recommendation in Internet Information Services (互联网信息服务算法推荐管理规定). https://www.cac.gov.cn/2022-01/04/c_1642894606364259.htm.

Cyberspace Administration of China and Ministry of Industry and Information Technology and Ministry of Public Security and National Radio and Television Administration. 2025. Measures for the Labelling of AI-Generated and Synthesised Content (人工智能生成合成内容标识办法). https://www.cac.gov.cn/2025-03/14/c_1743654684782215.htm.

Dastin, Jeffrey. 2018. Amazon Scraps Secret AI Recruiting Tool That Showed Bias Against Women. Reuters. https://www.reuters.com/article/us-amazon-com-jobs-automation-insight/amazon-scraps-secret-ai-recruiting-tool-that-showed-bias-against-women-idUSKCN1MK08G.

European Parliament and Council of the European Union. 2024. Regulation (EU) 2024/1689 Laying down Harmonised Rules on Artificial Intelligence (Artificial Intelligence Act). https://eur-lex.europa.eu/eli/reg/2024/1689/oj/eng.

Federal Trade Commission. 2019. “FTC Imposes $5 Billion Penalty and Sweeping New Privacy Restrictions on Facebook.” https://www.ftc.gov/news-events/news/press-releases/2019/07/ftc-imposes-5-billion-penalty-sweeping-new-privacy-restrictions-facebook.

General Office of the State Council of the People’s Republic of China. 2026. State Council 2026 Legislative Work Plan (国务院2026年度立法工作计划). https://www.news.cn/20260511/64fb4f5178b042e6aa16bf55932f038e/c.html.

Huang, Fei. 2025. “Check Your AI: A Framework for Its Use in Actuarial Practice.” The Actuary. https://www.theactuary.com/features/2025/06/25/check-your-ai-framework-its-use-actuarial-practice.

Kiviat, Barbara. 2019. “The Moral Limits of Predictive Practices: The Case of Credit-Based Insurance Scores.” American Sociological Review 84 (6): 1134–58. https://doi.org/10.1177/0003122419884917.

KPMG, and University of Melbourne. 2025. Trust, Attitudes and Use of Artificial Intelligence: A Global Study 2025 — Australia Snapshot. https://kpmg.com/au/en/insights/artificial-intelligence-ai/trust-in-ai-global-insights-2025.html.

Leek, Jeffrey T, and Roger D Peng. 2015. “What Is the Question?” Science 347 (6228): 1314–15.

Monetary Authority of Singapore. 2018. “Principles to Promote Fairness, Ethics, Accountability and Transparency (FEAT) in the Use of Artificial Intelligence and Data Analytics in Singapore’s Financial Sector.” https://www.mas.gov.sg/publications/monographs-or-information-paper/2018/feat.

Monetary Authority of Singapore. 2019. “Veritas Initiative.” https://www.mas.gov.sg/schemes-and-initiatives/veritas.

National Financial Regulatory Administration (China). 2026. Guidance on the Safe Development and Application of Artificial Intelligence in the Banking and Insurance Sectors (关于银行业保险业人工智能安全开发应用的指导意见, Jin Fa [2026] No. 8). https://www.nfra.gov.cn/cn/view/pages/governmentDetail.html?docId=1261784&generaltype=1.

National People’s Congress (China). 2021. Personal Information Protection Law of the People’s Republic of China (中华人民共和国个人信息保护法). http://www.npc.gov.cn/npc/c2/c30834/202108/t20210820_313088.html.

New York City Council. 2021. Local Law 144 of 2021: Automated Employment Decision Tools. https://rules.cityofnewyork.us/rule/automated-employment-decision-tools-updated/.

New York State Department of Financial Services. 2021. Report on Apple Card Investigation. https://www.dfs.ny.gov/system/files/documents/2021/03/rpt_202103_apple_card_investigation.pdf.

New York State Department of Financial Services. 2024. “Insurance Circular Letter No. 7 (2024): Use of Artificial Intelligence Systems and External Consumer Data and Information Sources in Insurance Underwriting and Pricing.” https://www.dfs.ny.gov/industry-guidance/circular-letters/cl2024-07.

Obermeyer, Ziad, Brian Powers, Christine Vogeli, and Sendhil Mullainathan. 2019. “Dissecting Racial Bias in an Algorithm Used to Manage the Health of Populations.” Science 366 (6464): 447–53.

Parra-Orlandoni, M. Alejandra, and Paulo Carvão. 2026. “You Outsourced the AI — but You Still Own the Risk.” Harvard Business Review. https://hbr.org/2026/07/you-outsourced-the-ai-but-you-still-own-the-risk.

Peng, Roger D, and Elizabeth Matsui. 2015. The Art of Data Science: A Guide for Anyone Who Works with Data. Skybrude Consulting, LLC.

Personal Data Protection Commission Singapore and Infocomm Media Development Authority. 2020. “Model Artificial Intelligence Governance Framework (Second Edition).” https://www.pdpc.gov.sg/help-and-resources/2020/01/model-ai-governance-framework.