Cheat sheet: AWS Certified AI Practitioner Certification | by Chanakya Kumar

Disclaimer:

This is not a comprehensive guide to the AWS Certified AI Practitioner Certification preparation and should be used primarily towards quick revision or recap.
Most the content is directly taken from AWS tutorials or AWS documentations.

Getting started:

Useful tips:

I strongly recommend having basic knowledge of all AWS services (even the non-AI ones). Pro tip — consider preparing for AWS Certified Cloud Practitioner before going for AWS Certified AI Practitioner.
A 30-minute exam extension is available upon request to non-native English speakers when taking an exam in English. Refer the “English as a second language” section under Requesting Accommodations.
Register for your exam using the code AWSRetake2025 between October 9, 2024 and February 15, 2025 to be eligible for a free exam retake.

Content:

AI > ML > DL > GenAI

Generative AI stack

AWS AI services

Amazon SageMaker — SageMaker is a fully managed machine learning service that you can use to build, train, and deploy your own custom models. [Infrastructure layer].
Amazon Bedrock — Amazon Bedrock is a fully managed service that makes FMs from Amazon and leading AI startups available through an API. With Amazon Bedrock, you can quickly get started, experiment with FMs, privately customize them with your own data, and seamlessly integrate and deploy FMs into AWS applications. If you’d prefer to experiment with building AI applications, you can get hands-on experience by using PartyRock, an Amazon Bedrock Playground.
Amazon Fraud Detector
Amazon Lex — use to build conversational interfaces, speech to text
Amazon Transcribe — automatic speech recognition, speech to text
Amazon Polly — text to speech
Amazon Personalize — for real time personalization and recommendations
Amazon Q — generative AI powered assistant
Amazon Rekognition — detect/analyze images and videos
Amazon Kendra — reimagines enterprise search for websites and applications so that individuals can readily find the content they are looking for. (Vector Database). Intelligent document search.
Amazon OpenSearch — Vector Database
Amazon Comprehend — NLP, summarization
Amazon Textract — extracting text from scanned documents, PDF, images
Amazon Translate — language translation
Amazon Augmented AI (Amazon A2I) — helps build the workflows required for human review of ML predictions.
AWS Glue — ETL service

Amazon SageMaker

Amazon SageMaker Data Wrangler is a low-code no-code (LCNC) tool. It provides an end-to-end solution to import, prepare, transform, featurize, and analyze data by using a web interface.
Amazon SageMaker Feature Store helps data scientists, machine learning engineers, and general practitioners to create, share, and manage features for ML development.
SageMaker Canvas use machine learning to generate predictions without needing to write any code — a Low Code No Code platform
Amazon SageMaker Experiments to experiment with multiple combinations of data, algorithms, and parameters, all while observing the impact of incremental changes on model accuracy.
SageMaker Model Monitor monitors model performance, detects quality issues and drift, alerts users. (Model drift is the degradation of an AI model’s performance over time due to changes in the data or the relationships between input and output variables.)
SageMaker Jumpstart provides pre-trained FMs. You can host the models in SageMaker with no additional training. However, this solution requires you to configure and monitor the production endpoint that hosts the ML model. (vs. Bedrock is fully managed).
Amazon SageMaker Ground Truth — RLHF (Reinforcement Learning with Human Feedback) — provides cost effective labour for reinforcement learning.

Inference Latency
SageMaker inference latency: Real time (1 min) < Serverless (few mins) < Asynchronous (hours) < Batch (days)

Management and Governance

AWS CloudTrail
Amazon CloudWatch
AWS Config
AWS Trusted Advisor — provides information on how to optimize account environments for cost and performance,
AWS Well-Architected Tool

CloudTrail helps you track and audit user activity, account activity, API activity for security and compliance (Who did what?), while CloudWatch is about monitoring and optimizing system performance and operational health.

Security, Identity, and Compliance

AWS Artifact — store documentation. on-demand access to security and compliance documents.
AWS Audit Manager
AWS Identity and Access Management (IAM)
Amazon Inspector — checks AWS resources for security exposures and vulnerabilities in configurations
AWS Key Management Service (AWS KMS) — encryption
Amazon Macie — You can use Macie to discover, classify, and protect sensitive data that is stored in Amazon S3. Macie is useful for data security. However, Macie primarily focuses on data at rest.
AWS Secrets Manager

Foundational models (FMs)
ML models trained on a broad spectrum of generalized and unlabeled data and capable of performing a wide variety of general tasks such as understanding language, generating text and images, and conversing in natural language. The size and general-purpose nature of FMs make them different from traditional ML models, which typically perform specific tasks, like analyzing text for sentiment, classifying images, and forecasting trends.

Prompt Engineering vs. Fine tuning vs. RAG
With these FMs as a foundation, you can further optimize their outputs with prompt engineering, fine-tuning, or RAG.

Process to tweaking the input prompt to direct the AI model towards desired inference.

Pre-training focuses on unsupervised learning from large-scale unlabeled data, capturing general language understanding.

Fine-tuning builds on pre-training and adapts the models to specific tasks using task-specific labeled data, enabling specialized performance. Fine-tuning is a supervised learning process that involves taking a pre-trained model and adding specific, smaller datasets. Adding these narrower datasets modifies the weights of the data to better align with the task.

There are two ways to fine-tune a model:
– Instruction fine-tuning uses examples of how the model should respond to a specific instruction. Prompt tuning is a type of instruction fine-tuning. Instruction-based fine-tuning uses labeled examples that are formatted as prompt-response pairs and that are phrased as instructions.
– Reinforcement learning from human feedback (RLHF) provides human feedback data, resulting in a model that is better aligned with human preferences. Reinforcement learning uses a trial-and-error process and a reward-based system.

Note: For prompt-based learning and RAG, you are not customizing the FM. However, when you fine-tune an FM, you are customizing the FM (modify the weights) and creating a private copy of the FM.

Retrieval Augmented Generation (RAG)

Retrieval Augmented Generation (RAG) is a technique used to augment a large language model (LLM) with external data, such as a company’s internal documents. This provides the model with the context it needs to produce accurate and useful output for your specific use case.

Prompt + [Context] -> AI model

Generative Adversarial Networks (GANs)
are a type of generative model that involves two neural networks competing against each other in a zero-sum game framework. The two networks are generator and discriminator.

Responsible AI
Responsible AI tools: (Transparency)

Amazon SageMaker Role Manager

Amazon SageMaker Model Cards

Amazon SageMaker Model Dashboard

AI Service Cards are a form of responsible AI documentation that provides a single place to find information on the intended use cases and limitations, responsible AI design choices, and deployment and performance optimization best practices for AWS AI services.

AI Service Cards, Amazon provides transparent documentation on Amazon services that help you build your AI services. With SageMaker Model Cards, you can catalog and provide documentation on models that you create or develop yourself.

Interpretability is the access into a system so that a human can interpret the model’s output. Observe the inner mechanics of the AI/ML method.
Explainability is how to take an ML model and explain the behavior in human terms. With complex models (for example, black boxes), you cannot fully understand how and why the inner mechanics impact the prediction. However, through model agnostic methods (for example, partial dependence plots, SHAP dependence plots, or surrogate models) you can discover meaning between input data attributions and model outputs. With that understanding, you can explain the nature and behavior of the AI/ML model.
Model controllability is measured by how much control you have over the model by changing the input data.

Security, Compliance, and Governance for AI Solutions

Jailbreaking attempts involve crafting carefully constructed prompts or input sequences that aim to bypass or exploit vulnerabilities in the AI system’s filtering mechanisms or constraints.
Hijacking and prompt injection refer to the technique of influencing the outputs of generative models by embedding specific instructions within the prompts themselves. The goal is to hijack the model’s behavior and make it produce outputs that align with the attacker’s intentions, such as generating misinformation or running malicious code.
Poisoning — corrupting training data
Exposure — exposing personal or protected information of/from training data
Prompt leaking — exposing context or sensitive information about the model
Hallucinations — The model generates inaccurate responses that are not consistent with the training data. These are called hallucinations.

Shared responsibility mode
AWS is responsible for protecting the infrastructure that runs all the services offered by Amazon Bedrock. You are responsible for managing and encrypting your data and applying correct user access controls to data and API calls made to Amazon Bedrock.

AWS’s responsibilities are the security of the cloud.

Customer responsibilities are security in the cloud.

Evaluation metrics

BLEU (Bilingual Evaluation Understudy): BLEU is a metric used to evaluate the quality of machine translations by comparing them to human reference translations. It quantifies the similarity between the machine-generated text and the reference text
ROUGE (Recall-Oriented Understudy for Gisting Evaluation): ROUGE is another metric used in the evaluation of text generation models. It assesses the overlap of n-grams, word sequences, and phrases between the generated text and reference text. ROUGE captures more nuanced content overlap than BLEU, making it suitable for tasks like text summarization and machine translation.
BERTScore: BERTScore is an evaluation metric that utilizes pre-trained contextual embeddings from models like BERT. It measures the similarity of word embeddings between the generated text and reference text. BERTScore has the advantage of considering semantic similarity, making it a valuable addition to traditional metrics.