Oracle Cloud Infrastructure 2023 AI Foundations Associate Practice Tests and PDFs

Tokens are the basic units of text representation in large language models. They can be words, subwords, characters, or symbols. Tokens are used to encode the input text into numerical vectors that can be processed by the model’s neural network. Tokens also determine the vocabulary size and the maximum sequence length of the model3. Reference: Oracle Cloud Infrastructure 2023 AI Foundations Associate |Oracle University

Large language models are trained on massive amounts of data to capture the complexity and diversity of natural language. Larger model sizes mean more parameters, which enable the model to learn more patterns and nuances from the data. Larger models also tend to generalize better to new tasks and domains. However, larger models also require more computational resources, data quality, and data size to train and deploy. Therefore, large language models handle the trade-off by prioritizing larger model sizes to achieve better performance, while using various techniques to optimize the training and inference efficiency4. Reference: Artificial Intelligence (AI) | Oracle

The attention mechanism in the Transformer architecture is a technique that allows the model to focus on the most relevant parts of the input and output sequences. It computes a weighted sum of the input or output embeddings, where the weights indicate how much each word contributes to the representation of the current word. The attention mechanism helps the model capture the long-range dependencies and the semantic relationships between words in a sequence12. Reference: The Transformer Attention Mechanism - MachineLearningMastery.com, Attention Mechanism in the Transformers Model - Baeldung

Model training is the process of finding the optimal values for the model parameters that minimize the error between the model predictions and the actual output. This is done by using a learning algorithm that iteratively updates the parameters based on the input features and the output1. Reference: Oracle Cloud Infrastructure Documentation

Machine learning is a branch of artificial intelligence that enables computers to learn from data and experience without being explicitly programmed. Machine learning algorithms can adapt to new data and situations and improve their performance over time2. Reference: Artificial Intelligence (AI) | Oracle