HR Ghosted You. ChatGPT Didn't. Here's Why.

Introduction

Think of all the moments when you were ghosted or left on read by someone; you're likely to have your own stories. When you ask a query or apply for a job, there is a high possibility that you will be ghosted, as if you don't even exist. On the other hand, we have ChatGPT, which answers your dumb queries with zero judgement and shares legit knowledge.

But how? Let's walk through it at a high level. Before diving into internals, let's understand: AI Application Development vs core ML Engineering.

Imagine this scenario. You are an engineer working under a product manager at a startup. Since he is a product manager, you expect him to have solid product sense rather than engineering knowledge.

You build a URL Shortener service as per the requirements, and he needs to demo your product in front of the potential customers. Now answer a simple question: Does he need to understand the core architecture and internals of the product? The answer is obvious, it is a NO! He simply needs to understand how the product works.

This analogy will help us understand the difference between an Application Developer who builds AI Products and an ML Engineer who engineers the core engine that powers the AI Products. Product Managers are similar to Application Developers who have minimal understanding of the product (engine) internals but knows to apply them for multiple use-cases. You (Engineer) are similar to an ML Engineer who builds a core engine, or what we call LLM (Large Language Model).

---

Large Language Model (LLM)

In the previous section, we introduced a new word, LLM (Large Language Model). In simple terms, LLM is the core engine that takes in an input and returns a suitable output. Large Language Models are pre-trained on large datasets. Based on the pre-trained data, new content is generated.

The first and most popular LLM that entered the market was GPT from OpenAI. GPT stands for Generative Pre-trained Transformer. As the name suggests, Transformer is a core engine that works on the self attention mechanism that takes input tokens and returns new output tokens (new generated content). Transformer is generative in nature, based on the pre-trained data. That's it!

LLMs can solve problems ranging from mathematics to computer science. This feels like magic, but under the hood, this is pure mathematics. Let's dive into this in the upcoming sections.

Common applications of LLMs in daily life are:

• Researching on a topic

• Generation of an essay

• Generating code snippets and so on...

---

ChatGPT

OpenAI built the ChatGPT application which allows the end users to consume GPT (Generative Pre-trained Transformer) model with zero understanding of the internals of the model.

We write a prompt and hit send. And we simply receive a response. If we observe the response carefully, it could be different each and every single time. That implies the response is not stored in a database or any sort of store, instead it generates new content and predicts the next word based on the input given. This is the simple request-response flow.

---

Models do not understand text

Let us consider an analogy:

You are given a 1000 pages book to read. What would be your first approach? The most common approach would be to breakdown the entire book into multiple chapters and consume each and every single chapter. It would be very hard to consume an entire book all at once (so break that into consumable pieces).

In GenAI world, these are called tokens. Tokens are the most fundamental units which is passed as an input into the model. Input is given in natural language and models do not understand them. A process called tokenisation takes place to break the text into multiple tokens.

Models takes in input tokens and predicts the next token (generation of context). If input tokens are text it would be extremely hard for the model to predict the next character / word. Instead if input token is a number, it would be comparatively easier for the model to mathematically predict the next token. This is why we need tokens as numbers. Let's dive deeper.

---

Tokenisation

Tokens are the most fundamental units which are passed as an input into the model. Tokens are numbers (token ids) which are derived from a predefined dictionary / vocabulary. In the vocabulary, the tokens can be a character / sub-word / word and assigned to a tokenId (number). Every LLM has a different vocabulary.

Based on vocabulary, the text is broken down into tokens and is mapped to a number. We end up with a sequence of numbers which will help the model to mathematically come up with the next possible token. Pure prediction! Pure mathematics!

The process of breaking down text (natural language) into tokens is called tokenisation. This the first step in the transformer world.

---

Transformers

Transformers are the core engine whose architecture is based on "Attention is all you need" and self attention mechanism. Transformers takes in tokens as an input and predicts the next possible token based on probability distribution. Since transformers consumes tokens as an input.

Step 1: Tokenisation

The conversion of natural language text into tokens.

Step 2: Embeddings

The tokens are predefined in the LLM vocabulary. But in real world, these tokens can have semantic meaning and relationships with other tokens. How can we establish these semantic relationships (meaning and logical relationships) in the model architecture. Answer : Vector Embeddings.

Embeddings are matrix representations which contains sequence of numbers, numerically representing a token. Tokens are placed at a certain coordinate point in a high-dimensional space representing its semantic meaning. The tokens with similar meaning are always closer, i.e has similar embeddings.

Lets take an example:

1. Man is King

2. Woman is Queen

Token "Man" is placed near token "Woman" and "King" is placed near "Queen". Man has a vector embedding [0.1, -0.2, 0.3] and woman would have a vector embedding of [0.2, -0.3, 0.4]. So similar embeddings mean they have some semantic relationship.

In short, mapping tokens to its real semantic meaning.

Step 3: Positional Encoding

Let us consider an example:

Statement 1: John loves Pizza
Statement 2: Pizza loves John

Two statements have same number of tokens (3) and all of them have the exact same vector embeddings. Meaning there exists a single semantic relationship in the coordinate system "inferring both the statements have similar meaning". But both the statements are NOT the same, their meaning differs based on the position of tokens.

Hence, extra positional data is added to the token embeddings to slightly differ the position and inferring a different meaning. This process is said to positional encoding.

Step 4: Self-Attention Mechanism and Feed Forward Layer

Let us consider an example:

Statement 1: Cricket Bat
Statement 2: Flying Bat

Both statements have a common token "Bat". They both have the same vector embedding and the same position as well. Still the meaning of "Bat" differs as it is used in different "contexts". How does LLM understand actual meaning of token "Bat"? Answer: Self attention mechanism.

Self attention mechanism allows the tokens to "talk" to each other, making the model context aware. The tokens in the sequence talks to each other and other tokens updates the embeddings of "Bat" based on its usage, ensuring "Bat" token has different meaning in different contexts.

Feed Forward Layer is the neural network layer which contains pre-defined weights (initially later weights change on training) and vector embeddings are passed into this layer, mathematically using matrix multiplications - new vectors are returned.

Step 5: Linear transform and Softmax

Linear transform converts the vectors into a sequence of numbers mathematically called logits. These logits are converted into a probability distribution by the softmax function. Softmax converts the logits into probabilities, based on which the model predicts the next token.

Temperature settings alter the logits before softmax is applied, allowing the model to control the output randomness. Higher the temperature, the more creative and random the predicted token selection becomes. Lower the temperature, the model chooses safer, higher-probability predicted tokens.

Phases

There exists 2 phases in transformers:

1. Training phase

   1. Transformer is trained in multiple steps. An input is given to the transformer, does all the steps above and returns a random gibberish output. Eg 2 + 2 in the training phase, transformer may return 90 as an output.

   2. The difference between the expected result and the model prediction is calculated, this is called cross entropy loss. Based on this loss, model back propagation is done where the weights in the neural network gets updated.

   3. In simpler terms, we say: "Hey, it is not 90, it less than that " => it predicts 10 => "Hey it is still lesser" => predicts 5 => "1 less" => predicts 4.

   4. Done. Model learns from its mistakes and progressive training.

2. Inference phase

   1. Similar to training phase, we give an input and expect an output.

   2. But here we do not perform model back propagation as we "use" model or inference model that is already trained.

   3. This is the step that application developers do. Training phase is done by ML engineers.

---

This sums up the high level architecture of Transformer! Done and dusted!

---

Final Thoughts

We discussed the working of GPT and high level architecture of transformers. Hope you enjoyed! I truly did writing this!