Transformer Part II – Code Implementation
https://huggingface.co/datasets/bird-of-paradise/transformer-from-scratch-tutorial/blob/main/Transformer_Implementation_Tutorial.ipynb https://github.com/hkproj/pytorch-transformer Views: 34
ComputingNotes
Electromagnetic Spectrum
References: What is Electromagnetic Spectrum? The electromagnetic (EM) spectrum is the entire range of all types of electromagnetic radiation. Radiation is energy that travels and spreads out as it goes, the visible light that comes from a lamp in your house and the radio waves that come from a radio station are two types of ... Read More
Wireless Channel
References: What is wireless channel? In wireless communication, a wireless channel refers to the physical medium, the air or space, through which electromagnetic signals travel from a transmitter to a receiver. Unlike a wired connection (like fiber optics or Ethernet cables) where the signal is contained within a physical “pipe,” a wireless channel is open ... Read More
Network Coding Type – Opportunistic
References: What is opportunistic network coding? In the world of wireless communication, the biggest challenge isn’t just sending data, it’s doing so without creating a digital traffic jam. While traditional routers simply act as relay stations, a more intelligent approach has emerged: Opportunistic Network Coding. Instead of treating every packet as a solo traveler, we ... Read More
A Guide to Network Coding
References: What is Network Coding? In traditional networking, routers act like simple relay stations i.e. they receive a packet and forward it toward its destination. However, this “forwarding-only” approach creates physical bottlenecks that limit how much data a network can handle. Network coding is a revolutionary paradigm that changes this by allowing routers to “mix” ... Read More
Data Structure and Algorithms – Array
Resources: Array Array is one of the most fundamental and widely used data structures. It serves as a building block for more complex structures like heaps, hash tables, and matrices. 1. What is an Array? An array is a collection of items stored at contiguous memory locations. The idea is to store multiple items of ... Read More
Calculus – Transcendental Functions
References: Transcendental Functions In mathematics, a transcendental function is a function that does not satisfy a polynomial equation with polynomial coefficients. To put it simply: it is any function that is not algebraic. While algebraic functions can be constructed using a finite number of elementary operations (addition, subtraction, multiplication, division, and taking roots), transcendental functions ... Read More
Transformer Part I – Theory, Original Paper etc.
Resources: Transformer Transformer is an encoder and decoder model. It uses a mechanism called Attention. Transformer model consists of encoder and decoder. Analogy Imagine you are at a loud party. You hear fragments of conversations. To understand a specific sentence, your brain does three things instantly: Transformer Architecture The Transformer architecture is an Encoder-Decoder structure. ... Read More
Transformer Part IV – Visualization
Views: 20
LM, N-Grams, RNNs, LLM, Fine Tuning, LoRA, QLoRA – Part III
LLMs: Fine-tuninghttps://developers.google.com/machine-learning/crash-course/llm/tuning Foundation Models A Foundation LLM (or base/pre-trained model) is a general-purpose model trained on vast amounts of data. It understands grammar and can perform creative tasks like writing poetry. However, to solve specific problems (like classification or regression), it often serves as a starting platform rather than a finished solution. Fine Tuning Fine-tuning ... Read More