This chapter provides an outline of the regulations and laws that have been passed in various countries that relate to the adoption of ethical AI practices in organizations and companies developing large-scale AI solutions. The primary objective of this chapter is to enable you to understand the main principles of Responsible AI so that you are aware of the legal implications if you fail to abide by the regulations. You will also be made aware of the different levels of risk associated with AI applications and what it means to accept or ban AI systems on the basis of their potential risk. Furthermore, you will learn about the commonly followed and enforced initiatives, actions, and guidelines that remove bias against different minority groups in populations. In addition, this chapter also studies the problems and roadblocks faced by governments when designing fair ML solutions and provides recommended actions for building large-scale trustworthy AI solutions.

In this chapter, we will cover the following topics:

• Regulations and enforcements passed by individual nations
• Special regulations for children and minority groups
• Next steps for trustworthy AI

Regulating AI is still a nascent area. Many countries and organizations have come up with proposals for how to regulate and enforce laws and rules related to AI adoption in various industries. In this section, we will discuss the regulations put forward by different authorities to enforce the unbiased implementation of AI systems.

Regulations in the European Union

On October 20, 2020, the European Parliament of the European Union (EU) adopted three resolutions primarily aimed at the following:

• Developing a structured foundational system for the ethical aspects of artificial intelligence (AI), robotics, automation, and other transformational changes that impact the everyday lives of ordinary people
• Formulating an accountable civil authority that will judge the aforementioned impact and decide on punitive action
• Strategizing a technique to respond to the challenges posed by AI systems regarding intellectual property rights

The first resolution gave the EU the opportunity to highlight the essence of a human-centric and human-created AI approach. The second resolution prompted the EU to take the initiative to address the risks created by AI-based systems by introducing proportionate and flexible rules for different types of risks. These risks were classified with the following labels: unacceptable risk, high risk, limited risk, and minimal risk. Furthermore, there were different liability rules according to the severity of the risk.

However, the last resolution, which focuses on the intellectual property rights at stake in the development of AI technologies (https://ai-regulation.com/news-eps-resolutions-on-ethical-framework-civil-liability-and-intellectual-property-rights-for-ai/), still remains unaddressed.

These AI rules set forth in the draft regulation are harmonized (meaning that they apply throughout the EU) and will not have a direct effect on countries outside the EU, but they will have some extra-territorial impact. For instance, compliance with these rules by all countries making AI services available in the EU is mandatory, just as with the General Data Protection Regulation (GDPR). The EU’s AI regulation has an extra-territorial effect on the following:

• Providers releasing AI-enabled products, solutions, and services to the EU market (irrespective of where the services originate from and who the providers of the services are)
• Customers of AI-enabled systems who use AI systems within the EU
• Providers and consumers of AI systems geographically positioned outside the EU who are consuming services that reside within the EU

The EU has placed special emphasis on the governance of AI systems. It has also stressed the importance of regulating the algorithm-driven systems used by consumers. This can address issues related to privacy, especially where biometric recognition systems are concerned. In essence, the objectives of the EU regulations are as follows:

• Make sure that AI systems in the EU market are safe and respect existing laws on fundamental rights and wider EU values
• Make sure the law is clear to help investment and innovation in AI
• Make sure the laws that protect people’s rights and safety are enforced for AI systems
• Make sure that there is a single market for safe and trustworthy AI applications so that people can trust them

These regulations provide a new legal framework for AI in the EU. This framework establishes rules regarding the development, market placement, and use of AI systems that follow a proportionate, risk-based approach with specific restrictions to protect humans from harm. The rules aim for an open, connected community that can benefit from the rewards offered by AI technology while protecting citizens’ safety.

Propositions/acts passed by other countries

Similar to the EU, other attempts to regulate AI have been made. The regulations passed/proposed by the US, Australia, and the Institute of Electrical and Electronics Engineers (IEEE) are noteworthy in this regard.

AI regulation acts in the US

In the consumer finance context, to eliminate bias from AI algorithms, two main acts have been passed by the US government that have received attention: the Equal Credit Opportunity Act (ECOA) and the Fair Housing Act (https://www.brookings.edu/research/an-ai-fair-lending-policy-agenda-for-the-federal-financial-regulators/). The ECOA laid down rules that prohibit creditors from issuing, sanctioning, or approving credit transactions from lenders who discriminate against race, color, religion, national origin, sex, marital status, and age. Even allowing discrimination against individuals who may have received income from public or government authorities or have legal employment rights is treated as a violation of the act. Likewise, the Fair Housing Act has issued prohibitory orders against any discrimination when properties are being sold or rented or when assigning mortgages. Both of these acts aim to ban disparate treatment and disparate impact, where bias emerges intentionally or unintentionally.

Figure 3.1 – Graphs from the Zillow Housing Aspirations Report, showing housing discrimination observed in young adults and black communities

As demonstrated in Figure 3.1, this act seeks to eliminate discrimination against black, Hispanic, Asian, and other minority communities when searching for housing, where white people are likely to be favored by housing authorities and owners as potential renters over equally qualified minority groups.

The Federal Trade Commission (FTC) has put forward a proposition for truth, fairness, and equity in any AI-based service that an organization wants to promote and sell to its customers. The FTC memo made it clear that the FTC’s authority is to be used under Section 5 of the FTC act. This, along with the implementation of the Fair Credit Reporting Act (FCRA) and the ECOA, should curb the application of biased algorithms.

FTC chair Rebecca Slaughter has been a prominent voice for economic justice, raising concerns on issues related to algorithm-based bias. The FTC has further warned that companies could suffer severe punitive action, even prosecution, under the ECOA or the FCRA for biased and unfair predictions from AI systems. The most important guideline by the FTC is that organizations must stay transparent and accountable for the algorithms they develop and put into practice.

The need for an unbiased system and a defined risk management framework has been felt by other bodies, such as the US Department of Commerce (DoC). There was new momentum in the area of trustworthy AI with the passing of the National Defense Authorization Act in 2021. This led US Congress to have the National Institute of Standards and Technology (NIST) devise “a voluntary risk management framework for trustworthy AI systems.” This initiative triggered the development of the AI Risk Management Framework (AI RMF), drafting best practices for organizations to control risks emerging from AI systems. It also helped them to select the right trade-off between fairness and accuracy, privacy and accuracy, and privacy and fairness. The DoC has also established the National Artificial Intelligence Advisory Committee (NAIAC) as part of the National AI Initiative Act of 2020. The primary objective of this committee is to study the current state of AI in the US, evaluate the competency and state of the science around AI, and accordingly, offer recommendations to increase opportunities for historically underrepresented populations.

Figure 3.2 – Healthcare discrimination observed in black adults

Time and again, limiting societal benefits only to privileged groups has had a negative impact on society. As illustrated in Figure 3.2, restricting opportunities for minority populations has not only impeded extending social facilities to black adults but also resulted in denying them adequate healthcare facilities and the right to receive equal treatment facilities. The NAIAC has become more vigilant in curbing such discrimination to promote equality and justice. In the absence of action to promote fairness and equity, our society will continue to create biased real-world datasets that generate biased ML models. The absence of equity and justice in society leads to more discrimination against minority groups, which limits their opportunities for quality education, healthcare, career advancement, and more. The previous figure (taken from https://www.kff.org/report-section/kff-the-undefeated-survey-on-race-and-health-main-findings/) shows the percentage by which, in the US, the black community falls short of having proper access to quality services as compared to the white community.

To foster trade and technology between the EU and the US, the EU-US Trade and Technology Council (TTC) released an Inaugural Joint Statement to aid in the development of new transformational systems that use AI and promote universal human rights. Without this kind of action, biased systems will foster a loss of respect and egalitarianism for the common man.

Other trustworthy AI initiatives have been advocated by the United Nations Educational, Scientific and Cultural Organization (UNESCO), the Organization for Economic Co-operation and Development (OECD), and the Council of Europe. The OECD has been the greatest supporter of the EU government in analyzing and measuring the socioeconomic impacts of AI technologies and applications. In addition, it actively engages with policy-makers and regulators to increase opportunities for underrepresented sectors, classify AI systems, and evaluate risk, fairness, transparency, safety, and accountability to converge AI practices consistently across borders.

The White House Office of Science and Technology Policy (OSTP), established by Congress in 1976, promulgated 10 principles for consideration in the regulatory and non-regulatory approaches to the advancement and application of AI:

• Use best practices to build systems and frameworks that help to bolster public confidence, faith, and belief.
• Ensure that the right education and tools are in place with regard to AI standards and technology so that people are more likely to participate and exchange their views on the real-world experience of large-scale, productionized AI systems.
• Incorporate the best levels of scientific honesty, morality, and righteousness with regard to any kind of input, such as data that impacts AI model predictions.
• Develop transparent risk assessment and risk management methodologies across business units in a collaborative manner in order to learn from each other’s ethical issues and take proactive action.
• Predict the costs associated with the positives and negatives of the practical use of AI by production systems.
• Analyze performance metrics and adopt dynamic learning methodologies to learn about and make changes to system behavior and data.
• Evaluate and formulate the standards needed to promote fairness by removing any kind of bias and discrimination from AI systems.
• Use transparency tools to gain public trust and guarantee the design of safe and secure systems
• Establish regular checkpoints to ensure the confidentiality, integrity, and availability of unbiased AI data that can be used to build safe and fair systems
• Support collaboration, partnership, and inter-departmental teamwork to ensure the consistency and predictability of AI policies

The US has been very proactive in investigating loopholes in current AI solutions and has come up with an initial proposal to curb discrimination. The Blueprint for an AI Bill of Rights, published by the White House in October 2022, is a stepping-stone toward protecting individuals and communities from threats caused by AI-driven technologies.

President Biden wanted to remove inequity from all decision-making processes by ensuring the incorporation of fairness principles that respect the civil rights of Americans and grant equality of opportunity and radical justice in the US. With his support, the OSTP has stated five principles that respect civil rights, civil liberties, and privacy. It will defend freedom of speech and voting and forbid discriminatory practices. By protecting the public’s private data across public and private sectors, the guiding principles seek to provide a structured framework that can be applied to automated systems. In an attempt to provide equality of opportunity in the areas of education, housing, credit, employment, healthcare, financial services, safety, social services, non-deceptive information about goods and services, and government benefits, the five basic principles have been laid out as follows.

Safe and effective systems

The objective of this principle is to protect the public from unsafe or ineffective systems. There is to be an audit process to validate the risks assessed by domain experts and stakeholders from different communities. In addition, this principle lays down the best practices for risk mitigation and 24-hour monitoring for AI-based systems. By following rigorous, domain-specific standards, automated systems should be able to prevent unforeseen dangers and ensure public safety.

Algorithmic discrimination protections

The objective of this principle is to proactively prevent discrimination by algorithms. This will not only prevent unjustified differential treatment relating to differences in race, sex (including pregnancy, medical conditions, intersex status, and sexual orientation), gender identity, religion, age, national origin, disability, veteran status, genetic information, and other demographic and social statuses, but also ensure continuous measures to provide equitable treatment to everyone.

Data privacy

The objective of this principle is to ensure adequate built-in protection and safety standards during data collection, use, access, transfer, and deletion processes to avoid any violation of data privacy. This principle governs seeking appropriate permission (by designers and developers) from customers to fully respect users’ privacy choices.

Notice and explanation

The objective of this principle is to provide transparent, authentic, and up-to-date documentation of the overall system functionality, with adequate descriptions of the individual system components. In addition, such systems should notify the public of any change in system functionality after calibrating the level of risk caused by the changes.

Human alternatives and fallback

The objective of this principle is to ensure quick fallback to human alternatives when automated systems fail. In such cases, public safety is of paramount importance, so a fallback can guarantee equitable, accessible, and effective functionality without any harmful effects.

AI regulation acts in India

The Responsible AI Proposal given by India’s National Association of Software and Service Companies (NASSCOM) has introduced a concept of licensing (called Responsible AI Licenses (RAIL)) in order to protect developers’ AI source code against malicious use. Furthermore, the licenses provide a way to restrict the misuse of the code for inequity, bias, or societal harm. In addition to licensing, NASSCOM has stressed the communication of risks both internally and externally through dashboards, where different stakeholders (such as senior leadership, legal experts, data scientists, and DevOps) can collaborate on Responsible AI metrics and regulatory compliance. Transparency through visualization tools, dashboards, internal audits, and proactive communication to users about privacy risks is the top priority for NASSCOM to curb the ethical and societal risks of AI.

NASSCOM has also highlighted the following aspects of dataset verification and validation in the creation of unbiased models:

• Identify the principal elements, their importance, and the learning parameters in the model that govern the model’s outcomes.
• Assess the ability of the model to predict results correctly for all demographics through a fair, representative dataset, without skew in any of the features.
• Evaluate the target variable and any links with present data features that can have a positive or negative impact.
• Ensure legal compliance. Collect data with permission and organize timely reviews/audits as mentioned in GDPR. This includes having an agreement for processing data that states the possible uses of the data that is collected.
• Ensure bare-minimum data collection to avoid the unauthorized use of data by third parties, thereby minimizing privacy attacks.
• Adhere to special guidelines while collecting unique data. With regard to regulations such as the EU’s GDPR, unique or special datasets containing data such as political beliefs, sexual orientation, and religious beliefs should not result in unintended use without prior consent.
• Prevent the unauthorized use of data by tracking the data management life cycle and maintaining up-to-date information.

NASSCOM has come up with best practices for data collection, training research personnel, making evaluation guidelines for error estimation, and the peer review of datasets. It has emphasized the importance of mitigating the harm caused by discrimination during the preprocessing, in-processing, and postprocessing stages of the algorithmic pipeline. Local, global, attribute-based, and causal explanations (discussed in Chapter 9, Model Explainability) have been recommended to assess fairness and transparency. Recommendations put forward by EU regulations have been adopted by NASSCOM. NASSCOM also provides best practices for deploying production-grade applications, including the following:

• Use counterfactual analysis to understand the impact of a feature that may be surpassed or perturbed for a specific data point, due to the dominance of other features. This helps to analyze how features affect prediction results, such as a loan application being rejected by a model that would have accepted the application if the applicant’s income was 10,000 USD higher.
• Use a robust and secure deployment workflow to handle peak load and bursts without causing any downtime for production systems. A well-managed security pipeline should sustain an AI system in extreme environments.
• Use Continuous Integration and Continuous Deployment (CI/CD) pipelines to monitor and retrain models based on data drift. The objective is to establish feedback channels and put an escalation chain in place that ensures trustworthy and scalable deployment in practice.

AI regulation acts in Australia

Australia has also come up with principles and practices for ethical AI regulation, not only to build public trust in AI-enabled products and organizations but also to drive customer loyalty to AI-enabled services. Under Australian regulations, AI outcomes should be geared toward the following objectives:

• Human, societal, and environmental well-being: AI-powered decision systems and architectures should act for the benefit of individuals, society, and the environment. They should empower human beings, providing them with enough data to make informed decisions. Furthermore, overall societal and environmental well-being cannot be achieved by solely depending on machines. Hence, there should be support for proper review mechanisms, such as human-in-the-loop, human-on-the-loop, and human-in-command approaches. These defined processes involve the optimization of the entire ML process, with active feedback from human beings. Using human intervention methods, tasks such as annotation, active learning, transfer learning, and step-by-step optimization become easier as machines and humans work together effectively and collaboratively.
• Human-centered values: AI predictions should respect human rights, morals, diversity, and the autonomy of individuals. They should benefit all human beings, including future generations, and other living beings. Design choices and parameters should have sustainable and environmentally friendly techniques in place to carefully evaluate and certify that AI systems are considering all potential harmful effects.
• Fairness: AI-/ML-based systems should be designed to be inclusive and accessible for individuals with disabilities. The accessibility options should not unfairly discriminate against individuals, communities, or groups. This is a key objective to prevent unfair and biased systems that could lead to several negative effects, from disrespecting and disregarding vulnerable groups to aggravating prejudice and discrimination. As is evident from Figure 3.3 (sourced from https://sitn.hms.harvard.edu/flash/2020/racial-discrimination-in-face-recognition-technology/), four major face recognition algorithms demonstrated poor performance on darker-skinned women, with error rates shooting higher than 34% than for lighter-skinned men. The objective of introducing fairness into AI/ML algorithms is not only to remove bias in the recognition of skin tones or gender but also to foster diversity by ensuring accessibility for all, irrespective of attributes.

Figure 3.3 – Lower accuracy for darker female and male populations from face recognition technology applications

• Privacy protection and security: AI systems should have strong security policies built in to protect any individual’s privacy rights. There should be a strong emphasis on all kinds of data protection, from data ingestion to building ML models, to ensure the security of data. In addition to privacy and data protection, ample data governance mechanisms must also be ensured. Lineage and data governance have big roles to play as far as quality, integrity, and legitimate access to data are concerned.
• Reliability and safety: AI systems should not deviate from their stated goals and should reliably operate in accordance with their intended purpose and with sufficient resilience and security. They need to be safe, accurate, reliable, and reproducible. In addition, they should be designed with a fallback plan so that if they break the Service-Level Agreement (SLA) or exhibit a failure in their operational routine, they can be restored to a previous state (based on defined checkpoints). Such restore activities should also be reliable and safe to reduce and prevent unintentional harm.
• Transparency and explainability: AI systems should be transparent, responsible, and easily understood by people. It should be easy for people to recognize the cause of the outcomes of any model. Transparent data and AI models should be traceable so it is possible to deduce present and past outcomes, and causes should be explained to concerned stakeholders in simple language. This is one of the major requirements, not only for businesses but also for individuals, to make people aware of the system’s capabilities and limitations.
• Contestability: When the outcome of a predictive ML model/AI system significantly influences society by impacting an individual, community, group, or environment, there should be a timely auditing process. The auditing process provides a mechanism to analyze model results in terms of fairness. People should even be allowed to challenge the use or outcomes of an AI system.
• Accountability: The people responsible for the different phases of the ML model should be identifiable and accountable for the outcomes of the AI systems, and human oversight of such AI systems should be provided. The auditability and assessment of algorithms, data, and design processes play a key role therein, especially in critical applications.

The same ethical principles (including privacy and fairness) were put forward by NITI Aayog in India in 2021. Their proposal focuses on the safety and reliability of AI systems through continuous monitoring.

IEEE AI regulation guidelines

The IEEE’s Ethically Aligned Design: A Vision for Prioritizing Human Well-being with Autonomous and Intelligent Systems document, first released in 2019, lays out eight key aspects related to ethical issues of AI and mitigation strategies. These areas of focus, as depicted in Figure 3.4, include principles for creating a strong foundation for Ethically Aligned Design frameworks and platforms:

• Sustainable development
• Personal data rights and agency over digital identity
• Legal frameworks for accountability
• Policies for education and awareness

The system serves as a guideline to increase human trust in data-driven AI systems:

Figure 3.4 – The fundamental pillars of Responsible AI

We have learned about the fundamental building blocks of an ethical system. Now let’s discuss how collaboration among different organizations can enable us to leverage the power of AI/ML to address the needs of entire population segments.

AI-based predictive analytics and profiling have demonstrated many limitations pertaining to minority groups’ opportunities and development. To promote social welfare facilities (such as improving facilities used by children), there has been extensive statistical analysis of studies related to different cases in the past, where data has been sourced from different databases, including public welfare benefits data, medical records, and judicial information. Studies and detailed investigations have revealed that there have been wide variations/inconsistencies in model input data, the data recorded is not systematic, and validation criteria have been applied inconsistently.

Along with the draft regulation (from the EU Commission) and regulatory proposals set forth by individual countries, regulatory bodies have come forward to translate the proposals for the digital space. The respective bodies have been instrumental in proposing a new regulatory framework for machinery products and transformational digital techniques being applied in the fields of robotics, IoT, IoMT, blockchain, VR, military, autonomous vehicles, and others. The key goal is to enable trust between AI providers and users by addressing the risks posed by such systems.

Chatbots are one such practical example of real-world AI systems. In some cases, they have led to increased risk for children with developmental disabilities because chatbots don’t recognize the children’s disabilities, and hence do not recognize appeals for help. Neither can a chatbot provide adequate advice to children. One instance was recorded in 2018 and was shared with the public by the BBC (https://www.bbc.com/news/technology-46507900). The report demonstrates how two mental health chatbots failed to capture children’s reports of sexual abuse. Even though the chatbot had been designed with children in mind and was considered safe for children, its confused response posed additional challenges to young users.

This is not the only problem with chatbots. The privacy threats from chatbots include spoofing (impersonating someone else), tampering with data, data theft, and vulnerability to cyberattacks. Security and fairness are primary considerations of AI ethics, and when chatbots are found to enforce bias by responding to a reply based on the best-matching keywords, it has raised concern among ethicists. Ideally, chatbots should not respond with biased responses when the input does not match exactly with the words already in its store, and instead, it should learn and update its dictionary with the input. Hence, it has become the top priority among AI ethicists to educate social organizations as well as producers and companies about the side effects of AI solutions. Furthermore, child rights advocates have raised questions about data retention policies in chatbots and issues related to parental consent, because some chatbots rely on stored voice recordings to continuously learn and respond. Historical data has been found to reinforce systemic bias and introduce discrimination among children. Hence, it is essential for experts to validate individual profiles and stop individuals from being impacted by bias from AI system proxies.

Privacy advocates have become increasingly concerned with data privacy and have cautiously taken steps to warn about government mass surveillance activities, law enforcement, and examination activities, as well as other inquiry-based tools. Private information about individuals cannot be collected without consent, as this information can be utilized to identify, segment, investigate, and suppress communities.

Lack of knowledge of the collection process of biometric data, the people involved in the collection process, the storage methodology, and the application of techniques risks the trust that people have in AI. Some instances demonstrate the availability of limited data for minority groups, and these instances have raised questions related to fairness. Success rates for facial recognition detection software have been questioned by AI ethicists, especially for groups such as children and women of color. Biased training datasets have been known to promote social bias and lead to discrimination or further disadvantages for minority communities.

Promoting equality for minority groups

It is essential to stop bias from flowing into AI systems. Bias prevents a system from supporting alternative developmental trajectories for minority groups (even children). Trustworthy, equal-opportunity-oriented AI-enabled systems should be free of stereotypes and enable possibilities for every child, including girls and LGBT children. Some actions centered around the development and inclusion of children, ethnic minorities, and LGBT people would be the following:

• Prioritize fairness and non-discrimination among all minorities who are vulnerable, such as children, the black community, and LGBT people.
• Safeguard children’s data and privacy rights to ensure their safety.
• Increase transparency, explainability, and accountability for children so that the moral rights and dignity of children are preserved.
• Empower governments and businesses with knowledge of AI and children’s rights.
• Prepare minority groups by educating them so that they are aware of the harmful impacts of biased AI solutions when deployed at scale.
• Enable an AI-driven ecosystem to support and nurture individual talent to fulfill its full potential. This necessitates the elimination of any prejudicial bias against children, or certain groups of children, that directly or indirectly leads to biased treatment.

AI must be for everyone, including all children; also, privacy must be ensured in an AI world.

Educational initiatives

The initiatives that have been organized to educate institutions, educational agencies, and data professionals are the first step toward making people aware of the existence of AI systems and their risk levels (https://www.orrick.com/en/Insights/2021/07/AI-Tips-10-Steps-to-Future-Proof-Your-Artificial-Intelligence-Regulatory-Strategy).

One of the primary purposes of such educational initiatives is to educate AI practitioners on how to identify high, medium, and low risks and take appropriate action to evaluate an AI model’s predictive competence.

Identifying and evaluating the risk levels of systems

High-risk AI systems should be banned for public use, and their usage can be limited for research and future improvement. AI-enabled systems that should be banned include those that do the following:

• Manipulate human behavior, opinions, or decisions through their design (such as their user interfaces), architectural frameworks, and AI models, which can lead to detrimental decisions against individuals.
• Make predictions about an individual or a group based on protected attributes (such as gender and ethnicity).
• Execute indiscriminate surveillance on all individuals without any differentiation with respect to protected attributes. General surveillance can lead to the monitoring of, spying on, or tracking of individuals, without receiving user consent.

In addition to putting best practices in place, we should be able to identify high-risk systems. Along with general public education, we need to educate experienced data professionals who are building large-scale ML systems. Only then will we be able to prevent the default prediction of, say, a “captain” as a man. To do that, we need to remove probabilistic bias, as illustrated in the following figure for a word embedding model (https://developers.googleblog.com/2018/04/text-embedding-models-contain-bias.html):

Figure 3.5 – Higher probability of a captain being predicted as male

As you can see, the model predicts that the pronoun “he” is more likely to be appropriate for this scenario.

Understanding high-risk systems

Systems classified as high-risk enable the government and authorities to take measures such as adding security techniques and conducting audits to identify the risks. AI-enabled systems that are identified as high-risk are those that do the following:

• Perform remote biometric identification of people in publicly accessible spaces.
• Function as safety components in the operation of essential public infrastructure networks.
• Prioritize the dispatching of emergency services, such as firefighters and medical aid.
• Determine merit access and assign individuals to educational and vocational training institutions through admissions tests and other merit assessments.
• Evaluate performance related to recruitment, promotion, rewards, and the termination of any kind of contractual relationship.
• Evaluate the creditworthiness of people to certify their eligibility for benefits, grants, and services. This principle aims to avoid discrepancies in the creditworthiness of individuals by reducing, for example, the likelihood of higher mortgage rates being assigned to black and Hispanic people, as shown in Figure 3.6:

Figure 3.6 – A biased system resulting in some minorities paying higher mortgage amounts

• Access individual risks and use predictions to be proactive in governing the risk. The central objective is to determine the authenticity of the information provided by a person. Transparent, reliable systems should have proper validation schemes incorporated to prevent, investigate, detect, or prosecute a criminal offense.
• Function as prediction systems for crimes or events of social unrest to evaluate and assign resources for monitoring and surveillance as part of a detailed criminal investigation.
• Examine and enable the processing of asylum and visa applications and associated complaints to determine the eligibility of individuals to enter specific countries/territories, such as the EU.
• Aid legal systems, such as judges at court, except for ancillary tasks.
• Access safety metrics before deploying drones in a warzone. One current example of this, at the time of writing, is in the Ukraine-Russia war, where Russian forces have tested new “swarm” drones. It comes under the umbrella of military ethics to evaluate the accuracy of these automated weapons, which are capable of tracking and shooting down enemy aircraft.

Such high-risk systems mandate that the training and testing datasets used in building the ML models work using authentic and unbiased representations. This is to allow sufficient transparency to understand their outputs and necessitates the inclusion of proper human-interface tools. In addition, these systems require consistent performance and system monitoring throughout the life cycle, along with ensuring high accuracy, robustness, and security. Non-high-risk systems should also be subjected to national testing and piloting schemes.

EU regulations have stressed the importance of ethical risk assessment procedures for AI systems where misuse can be foreseen and risks arise due to limited knowledge of application, control failures, and hazards associated with robotics applications. Risk assessment frameworks should apply the same acceptable risk standards to AI-based robotic applications as they do to tasks performed by a human. Standards set forth by EU regulations encourage public and stakeholder participation in the development of robots; some of the key design considerations for AI models and robots are as follows:

• Forbid the design of AI systems and robots that would be responsible for societal harm. This ensures that AI equipment invented and manufactured by humans does not kill humans. Robots should be safe and designed to serve the purpose for which they are intended, without having any hidden motivations.
• Hold responsible the designers or manufacturers of AI-enabled robots.
• Incorporate privacy in every design that deals with data, AI, ML, and models as long as the system is available to the public.
• Allow the safe and ethical use of robots, so as to prohibit their unethical behavior through detailed accounting.
• Run educative training programs for AI robot designers, individual and organizational users, and industry and government bodies, as well as civil society groups.
• Autonomous robots should be categorized, based on their behavior, as either illegal, immoral, permissible, or supererogatory, allowing them to be withdrawn from service according to regulatory guidelines.

International AI initiatives and cooperative actions

International initiatives are undertaken to provide harmony and synchronization to educate people across different nations. The primary aim is to establish a universal standard to raise awareness of AI ethics and support research and technology collaboration. This enables organizations across different countries to join hands and contribute to establishing synergy in the initiatives.

Some of the international strategies on AI aim to resolve the differences between humans due to protected attributes. In an attempt to provide a consolidated framework for governments across nations, they strongly uphold a “common vision for the Future of AI. G7 Common Vision is one of those, where the leaders of the G7 (Canada, France, Germany, Italy, Japan, the United Kingdom and the United States) met in 2018 and committed to 12 principles for AI” (https://www.europarl.europa.eu/RegData/etudes/STUD/2020/634452/EPRS_STU(2020)634452_EN.pdf) by the following means:

• Support, lead, and educate human-centric AI through commercial adoption in a systematic way that allows technology to be implemented in an ethical and trustworthy manner.
• Allocate funds to encourage investment in R&D so that, along with the efforts of researchers, architects, and data professionals, it generates public interest in new technologies.

Educate, train, and reskill individuals for the workforce to help them adapt to emerging innovative technologies.

Support campaigns and research studies to investigate the sources of bias for underrepresented groups, such that sufficient data for women and marginalized individuals is taken into consideration before building models.

The WHO’s ethics and governance team for AI (https://news.un.org/en/story/2021/06/1094902) for health has reported that AI in healthcare is already being used among rich nations to aid the speed and accuracy of diagnosis and screening for diseases. The use of AI in clinical care and improved health research and drug development should involve mandatory safety and privacy mechanisms that respect the confidentiality of patient data. Furthermore, the WHO stresses the need to develop systems that comply with regulations, have robust quality control measures, and provide accurate predictions for all individuals irrespective of age, gender, ethnicity, and features protected by human rights codes.

The UN General Assembly resolution UN A/RES/74/299 and the AI for Road Safety initiative have aligned to highlight the role of innovative automotive and digital technologies in reducing the number of global deaths and injuries from road traffic accidents by more than 50% by 2030 (https://news.un.org/en/story/2021/10/1102522). The objective is to concentrate on road safety data, regulatory frameworks, and the design of safer vehicles and road infrastructure to yield a much easier post-crash response. The International Telecommunication Union has also been working with AI for Road Safety to support continuous monitoring and automated driving safety data protocols that are aligned with ethical and legal bodies.

We even see ads promoted by Facebook (such as one in 2019) that are biased against gender, race, or religion. Facebook has been active “in promoting job advertisements for roles in nursing or secretarial work [to women], whereas job ads for janitors and taxi drivers had been mostly shown to men” (https://research.aimultiple.com/ai-bias/). These examples demonstrate how important it is for organizations to educate strategic, data, and architectural teams to abide by laws and follow the right practices to build and release ethical systems.

Implications of law enforcement

The European Commission for the Efficiency of Justice (CEPEJ) of the Council of Europe has been playing a leading role in setting ethical principles pertaining to the use of AI in judicial systems to guide policy-makers, legislators, and justice professionals (https://www.coe.int/en/web/cepej/cepej-european-ethical-charter-on-the-use-of-artificial-intelligence-ai-in-judicial-systems-and-their-environment). The aim is to apply AI responsibly to improve efficiency and quality in a manner that complies with the fundamental rights guaranteed in the European Convention on Human Rights (ECHR) and the Council of Europe’s Convention on the Protection of Personal Data.

With regard to privacy standards put forward by the European Data Protection Board (EDPB), a violation of GDPR can result in a fine of 30 million euros or 6% of the organization’s global annual turnover (whichever is higher). The same penalty is expected in the case of a breach of an unacceptable-risk AI system or infringement of the data governance provisions for high-risk AI systems. The maximum fines can increase based on the degree of infringement, such as releasing a prohibited AI application on the market. Other kinds of infringements related to issuing incorrect information would result in fines of up to 10 million euros, or 2% of the global annual turnover. The fines may be as high as 20 million euros, or 4% of the total worldwide annual turnover, for acts of non-compliance.

One of the foremost examples where regulations and ethical AI laws can be applied is in the usage of unethical AI tools such as the biased recruiting tool that was revealed by Amazon’s ML specialists. Amazon realized in 2015 that the AI-based tool they had in place to aid in their hiring process was not rating candidates in a gender-neutral fashion for software developer and other technical positions. The tool eventually turned out to make biased decisions against women. On further investigation, the organization found out that ML models were trained to vet applicants that were solely based on the observed patterns that were submitted in the company database over a 10-year period. As the tool penalized resumes containing words such as “women’s,” the organization decided to scrap this hiring tool and disband the team working on the project.

In these sections, we have learned about the AI regulations formulated by the EU and the acts and initiatives being undertaken by the US, Australia, and other international bodies. Now, let’s look at how organizations, companies, and AI researchers should build trustworthy AI systems.

At the time of writing this book, more than 50 countries have adopted some form of AI regulation policy (https://oecd.ai/en/dashboards). Still, like AI itself, forming a robust policy around AI is a difficult task, made more difficult because of the evolving nature of AI itself. However, there are some steps put forward by AI researchers and policy-makers that can help provide trustworthy AI to consumers.

One of the main steps for companies is crafting organization-wide policies and procedures to create a compliance-by-design program. Such programs help to promote the transparency and explainability of systems and support innovation at the same time. Furthermore, they should establish a regular audit-and-review process to analyze usage regularly and document all such processes. A design-and-review process should be established. This would entail having a process built to respond to any questions from regulators seeking additional information. In addition to the principles stated in the preceding sections formulated by the EU or the regulatory bodies of respective countries, companies also need to engage in PR/external communication to raise awareness of their products and connect potential customers and partners to revenue-generating channels.

Organizations should continuously seek to improve the fairness, integrity, privacy, and accuracy of any AI system. Some of the recommended best practices laid down by Google include the following:

• Equip all AI-enabled systems with a human-centered design approach so that they focus on user experience and account for the diversity of users and use cases.
• Identify and monitor multiple governance criteria to evaluate training and set up proactive measures based on monitoring results. This helps to determine the usefulness of model metrics to the goals of the AI system.
• Examine raw data to assess the evaluation metrics, including accuracy, and judge the predictive capabilities of the system.
• Ascertain the limits of the dataset and model, propagate these limitations across departments, and follow the recommended practices to remove bias and ensure fairness in terms of the input data.
• Undertake rigorous, diverse, and regular integration and unit testing of AI systems.
• Put in place regular monitoring and update techniques for all AI systems after deployment to consider real-world performance and user feedback.

The increasing adaptation of AI and solutions for minorities, including children, has led ethicists and data professionals to realize the need for unbiased AI systems. The goal is to uphold children’s collective right to protection and increase the participation of children from all backgrounds. This holds good for any AI-based systems with which children interact, irrespective of whether or not the system was designed for children. Some of the principles that organizations should infuse in their AI system development value proposal and technical know-how are as follows:

• Include specially designed KPIs and metrics that can be integrated into AI-enabled systems to monitor and track children’s well-being. As children are influenced by AI systems during their interactions with those systems, the design of such systems should pay special attention to well-being frameworks and metrics. Furthermore, such systems should be tested on children and success parameters should be closely linked to children’s well-being and development.
• Formulate AI policies and strategies by evaluating how AI systems can benefit children. This will help to ascertain the overall benefits received by children, as existing policies and strategies already evaluate the risks associated when children interact with such systems.
• Adopt design styles that protect children’s rights by placing the child at the center of ethical AI policy and system design, development, and deployment. Protecting children’s rights helps children’s development and well-being by enforcing privacy by design, safety by design, and inclusion by design.
• Design AI systems such that children’s development opportunities and rights to health, education, clean air, water, and safety are ensured. For example, AI-based educational and recommendation systems developed for children should not display advertisements that are not beneficial to children’s education and intellectual growth. AI-based systems should increase environmental sustainability and not impact the environment negatively. Any negative impact, such as an increase in carbon footprint, would seriously impact the well-being of children and their ability to live on a sustainable and healthy planet. The training, deployment, and computational infrastructure of AI systems should be tracked using carbon emission metrics to combat climate change, devise mitigation strategies, and promote better ML modeling.
• Promote the explainability and transparency of AI systems by addressing children’s needs, such as the use of age-appropriate language to describe AI. This would add more meaning to AI systems’ explainability and make them transparent. Furthermore, the AI application should empower child users according to legal and policy frameworks, principles, and regulations. There should also be a space for redressal.
• Develop systems to facilitate necessary skills among children and teachers to prepare children for present and future developments in AI. Furthermore, funding and incentives for child-centered AI policies and strategies should aim for infrastructure development and bridge the digital divide in a way that supports the equitable sharing of the benefits of AI. The end goal is to build a solid foundation of child-centered AI that includes protection (do no harm), provision (do good), and participation (include all children).

Despite the identification of the next steps in successfully building next-generation ethical AI systems, some direct challenges to government adoption of AI have been cited by the World Economic Forum (https://www.weforum.org/agenda/2019/08/artificial-intelligence-government-public-sector/). The main roadblocks that were identified include the following:

• Effective use of data: There’s a lack of understanding of the value of data and it is rarely deployed in a scalable infrastructure. There is also a lack of implementation of data governance processes.
• Data and AI skills: Governments, and other government-funded organizations, face a shortage of funds to hire expert data professionals that many big private companies can afford. This gap in attracting talent affects the quality of the AI solutions produced.
• AI ecosystem: Companies operating in the AI market experience frequent shifts in the problem they are solving and how customers see the solutions. Start-ups pioneering AI solutions also experience a dearth of talent to scale ML models ethically for large projects.
• Legacy culture: Governments and government-funded organizations have difficulty in adopting transformative technology, mostly because employees are not educated on ethical principles and are reluctant to take risks.
• Procurement mechanisms: ML-based solutions and systems are often treated as intellectual property with terms and conditions, which may make it difficult for governments to customize existing solutions within the stipulated time.

Existing challenges and gaps

The frameworks proposed by different ethical associations and bodies have addressed the moral and ethical dilemmas identified and presented previously, but there are still a few gaps to address.

Figure 3.7 – How human psychology plays a role in determining profession, exhibiting bias toward women on the left and bias toward men on the right

Along with the OECD, the EU has stressed the importance of societal and environmental well-being (including sustainability and environmental friendliness) and formulated ethics guidelines on the principle of prevention of harm. However, it does not provide examples of how to achieve this.

Ethics bodies have also looked at the impacts on human psychology of using AI and how people interact with AI. This is demonstrated in Figure 3.7, which shows how our existing social structure tends to affect how we perceive different professions. Hence, it is evident that the EU needs to reposition social relationships within the context of ethics so that the data available in our society (demonstrating biased decisions concerning different protected attributes, as shown in the figure) does not end up training ML systems, consequently leading to circular bias. The x axis here represents how different professions tend to be associated with men, women, or both.

The EU draft on Responsible AI emphasizes the need for constant monitoring to evaluate AI interaction with humans and sets forth that in the event of such interactions, the system should be well tested against all types of social interactions through creative simulations. Such simulations can create different scenarios involving human and robot interactions. Even this specification is not sufficient to provide safety and protection in the long run, as human-robot relationships are complex and have an impact on the human psyche. Such complex interactions need to be considered in the creation of next-generation robotic autonomous AI systems.

As Figure 3.8 illustrates, there is still fear among pedestrians about self-driving autonomous vehicles. One of the primary elements of future autonomous vehicle systems is using real test data to increase the percentage of pedestrians who feel safe with self-driving autonomous cars. In addition, accountability is also an important factor, in case of any harm caused by these vehicles. Hence, quantifying safety and protection remains one of the main tasks when defining trustworthy AI systems in the future:

Figure 3.8 – Safety concerns among pedestrians about self-driving cars

Proposed solutions and improvement areas

The art of making an AI solution public comes with several challenges and questions. To address the questions, experiments were conducted by the French Council of State and other countries of the EU, as well as the US, to implement four different types of regulations. These regulations, described here, attempt to develop and certify authentic ethical AI solutions:

• Algorithmic Accountability Act-driven regulation
• Regulations proposed by government regulators, including the Food and Drug Administration (FDA) (for the healthcare industry), the National Highway Traffic Safety Administration (NHTSA) (transportation), and the Federal Trade Commission (FTC) (retail)
• Regulations enabled by considering labor laws and civil rights laws
• Regulations driven by the California Consumer Privacy Act

A survey has shown that after applying experimental regulations to AI solutions, the adoption of AI in business processes decreases by 16% due to a significant increase in the cost of implementing the AI strategy. It has been observed that, often, managers reduce the funds allocated for employee training to develop a wider AI strategy that can sustain AI regulations. The need at present is to set standards for certification, testing, auditing, and technology that validate model performance metrics in all use cases.

Organizations have come up with regulatory sandboxes that allow private sectors to run tests to evaluate how systems perform with real individuals and local regulatory constraints. As illustrated in the following figure, these sandboxes involve a few rounds of internal evaluation and stress testing with Explainable AI (XAI) to audit and correct the systems with appropriate feedback.

The figure illustrates the different components of a regulatory sandbox. This kind of sandbox allows us to determine the following:

• The eligibility criteria for a dataset to be considered, such as how diverse the dataset is in terms of attributes. The procedure of application is to be included in the regulatory sandbox testing.
• Actual participation of the individuals in the regulatory testing process.
• The exiting condition for the sandbox, depending on the models being evaluated for risk.
• The rights and obligations of participants.

An XAI assessment framework helps us to do the following:

• Understand and explain the internal characteristics to validate interpretability.
• Evaluate the effect of the external environment on the system.
• Assess the effect of the system on the external environment in safety-critical scenarios.
• Ensure the application of better governance, policy, and regulatory theories.

If a system poses life-threatening risks to health and safety and fundamental rights, it should be stopped right away to prevent any further damage during its development and testing efforts. These experimental regulations and regulatory sandboxes help with evidence-based lawmaking. The core structure comprises pre-established facts, assumptions, and a legislative framework, which are the building blocks of an authentic, mature, and scalable AI system that abides by regulatory compliance.

The following figure shows an AI system in a sandbox to evaluate the effectiveness of the overall AI regulatory framework for economic, social, and political impact.

Figure 3.9 – AI sandbox for risk assessment and testing for regulations and policies

Let’s wrap up this chapter.

In this chapter, we have learned about the regulations and laws put into practice by different ethical and governing bodies from varying nations. We now understand their basic principles and how they can aid in nurturing the growth of equitable systems, particularly those that fall under the high-risk category, such as models to do with justice, autonomous cars, healthcare, and systems that are expected to not only protect basic human rights but also promote the development of sustainable activities on earth. In addition, we stressed the importance of generating awareness about any possible misuse of AI by concentrating on overall well-being, effectiveness, transparency, accountability, competence, privacy, and fairness. We now understand the potential loss, or the damage done, in the case of a violation or failure and why it is important to abide by existing regulations or laws, as well as how this loss has a negative impact on society. This chapter also provided us with an insight into the gaps in existing regulations and laws and what further refinement is needed. These gaps leave us with a responsibility to be mindful of the design of such systems so that they serve as demonstrative examples for future generations’ systems.

In order to do that, let’s study how to build and evaluate large-scale big data and ML model pipelines in the next chapter.

• Key provisions of the Draft AI Regulation: https://www.allenovery.com/en-gb/global/news-and-insights/publications/key-provisions-of-the-draft-ai-regulation
• Policy guidance on AI for children: https://www.unicef.org/globalinsight/media/2356/file/UNICEF-Global-Insight-policy-guidance-AI-children-2.0-2021.pdf
• AI Policy and National Strategies: https://aiindex.stanford.edu/wp-content/uploads/2021/03/2021-AI-Index-Report-_Chapter-7.pdf
• European draft Regulation on artificial intelligence: Key questions answered: https://www.ey.com/en_ch/law/european-draft-regulation-on-artificial-intelligence-key-questions-answered
• Amazon scraps secret AI recruiting tool that showed bias against women: https://www.reuters.com/article/us-amazon-com-jobs-automation-insight-idUSKCN1MK08G
• AI, Machine Learning & Big Data Laws and Regulations 2022 | Australia: https://www.globallegalinsights.com/practice-areas/ai-machine-learning-and-big-data-laws-and-regulations/australia
• U.S. Artificial Intelligence Regulation Takes Shape: https://www.jdsupra.com/legalnews/u-s-artificial-intelligence-regulation-1161759/
• Draft Eu Regulation On Ai And Its Impact On Healthcare: https://www.kantify.com/insights/draft-eu-regulation-on-ai-and-its-impact-on-healthcare
• FTC warns it could crack down on biased AI: https://www.theverge.com/2021/4/20/22393873/ftc-ai-machine-learning-race-gender-bias-legal-violation
• Ranchordas, S. (2021). Experimental Regulations for AI: Sandboxes for Morals and Mores. University of Groningen Faculty of Law Research Paper, (7). https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3839744, https://www.nomos-elibrary.de/10.5771/2747-5174-2021-1-86.pdf
• An AI fair lending policy agenda for the federal financial regulators: https://www.brookings.edu/research/an-ai-fair-lending-policy-agenda-for-the-federal-financial-regulators/
• Blueprint For An Ai Bill Of Rights: https://www.whitehouse.gov/wp-content/uploads/2022/10/Blueprint-for-an-AI-Bill-of-Rights.pdf
• Responsible Ai Architect’s Guide: https://indiaai.gov.in/responsible-ai/pdf/architect-guide.pdf