Intro to GenAI for Developers
Learn the fundamentals of Generative AI, GPT, and Transformers—explained for developers, researchers, and ML engineers with real-world examples
Topics to Cover in This Blog:
What is Generative AI and example of GEN-AI ?
What is a Transformer?
How do Transformers work behind the scenes?
Code example with ChatGPT
Conclusion
What is Generative AI?
Imagine a robot that can draw, write stories, do math, or talk like a human.
Generative AI means making new things (text, pictures, music, or even code).
Here is few example of Generative AI
GPT – Generative Pre-trained Transformer
BERT – Bidirectional Encoder Representations from Transformers
T5 – Text-to-Text Transfer Transformer
(keep eye 👁️ on Transformer)
What is a Transformer and How It work behind the seen ?
The Transformer is a special kind of brain for computers. It helps machines understand and generate language (like GPT does).
When you see Figure 1: The Transformer architecture, it usually has two big parts:
1. Encoder (the reader)
Think of the encoder as a student reading a book.
It takes the input sentence (like “Hello, how are you?”).
It breaks the sentence into tokens (small pieces, like Lego blocks).
Then it creates a hidden understanding of the sentence.
Decoder (the writer)
The decoder is like a storyteller.
It looks at the encoder’s understanding and tries to write an answer.
For example, if the input was “Hello,” the decoder might output “Namaste.”
The Secret Sauce: Attention Mechanism
Inside the Transformer, there’s something called “Attention”.
Attention helps the model focus on the important words in a sentence.
Example: In the sentence “The cat sat on the mat”,
if the question is “Who sat on the mat?”,
the model pays more attention to “cat” than the other words.
Figure: The Transformer model has two main parts – the Encoder (reader) and the Decoder (writer), connected by the Attention mechanism that helps it focus on the right words.
Understanding Tokenization, Embeddings & Positional Encoding (with Code Example)
Tokenization (Breaking Text into Pieces)
Before AI can work with text, it first breaks sentences into smaller parts called tokens.
Example
"Hello, my name is Saurabh"become
["hello", ",", "my", "name", "is", "Saurabh"]
Converting Tokens into Numbers (Token IDs)
Computers can’t understand words directly, so every token is mapped to a number (ID).
Example
"hello" → 7592 "my" → 2026 "name" → 2171
Word Embeddings (Vectors)
Numbers alone aren’t enough. We need embeddings — special lists of numbers (vectors) that capture the meaning of a word.
Example
"cat" → [0.12, -0.34, 1.56, …] "dog" → [0.14, -0.28, 1.49, …]
Positional Encoding (Order Matters)
AI also needs to know the order of words.
For example:“My name is Saurabh”
“Saurabh is my name”
Both have the same words, but different meanings.
This is where positional encoding comes in — it tells the model which word comes first, second, third, and so on.
Generative AI is a super-smart robot brain that can read, write, and create new things. By breaking text into tokens, turning them into embeddings, and understanding word order with Transformers, AI can generate meaningful language — and now you know how the magic happens!
