Notes
LM, N-Grams, RNNs, LLM, Fine Tuning, LoRA, QLoRA – Part III
LLMs: Fine-tuninghttps://developers.google.com/machine-learning/crash-course/llm/tuning Foundation Models A Foundation LLM (or base/pre-trained model) is a general-purpose model trained on vast amounts of data. It understands grammar and can perform creative tasks like writing poetry. However, to solve specific problems (like classification or regression), it often serves as a starting platform rather than a finished solution. Fine Tuning Fine-tuning ... Read More
LM, N-Grams, RNNs, LLM, Fine Tuning, LoRA, QLoRA – Part II
Introduction to Large Language Modelshttps://developers.google.com/machine-learning/crash-course/llm LLMs: What is a Large Language Model? An LLM is a predictive technology that estimates the next “token” (word, character, or subword) in a sequence. They outperform older models (like N-grams) because they use vastly more parameters and can process significantly more context at once. Transformer It is most successful ... Read More
LM, N-Grams, RNNs, LLM, Fine Tuning, LoRA, QLoRA – Part I
Introduction to Large Language Modelshttps://developers.google.com/machine-learning/crash-course/llm What is Language Model? At its simplest, a language model is a statistical tool that predicts the next piece of text in a sequence. The N-gram Approach Early language models used “N-grams,” which are simply ordered sequences of words where N represents the number of words. Context Context refers to ... Read More
Quantum Error Correction Challenges – Quantum Computing
Books References: Exploration in Quantum Computing by Colin P. WilliamsQuantum Computation and Quantum Information by Nielsen et. al Quantum error correction (QEC) is widely considered the “holy grail” of quantum computing and also its most formidable bottleneck. While classical computers can easily correct errors by copying data (redundancy), quantum computers are forbidden from doing this ... Read More
Data Structure and Algorithms – An Overview
T. H. Cormen, Ed., Introduction to algorithms, 2nd. ed., 10th pr. Cambridge, Mass.: MIT Press [u.a.], 2007. Data Structure and Algorithsm, Sixth Edition, GoodRich et al., Wiley What is a data structure? https://www.ibm.com/think/topics/data-structure What are algorithms? An Algorithm is simply a specific, step-by-step set of instructions used to complete a task or solve a problem. ... Read More
Calculus – Application of Integrals (Area between Curves)
References: Integrals can be used to calculate many things such as area between curves, volumes and surfaces of solids, length of curves etc. Areas between Curves: In calculus, we define the exact area under a curve not by measuring it directly, but by slicing it into rectangles, adding them up, and then making the slices ... Read More
Quantum Error Correction – Quantum Computing
Books References: In an idea set up, we assume that logical qubits evolves unitarily following Schrodinger’s equation from the moment quantum comptuer is prepared, go through some computations and measured. However, during this process, real quantum system couples with its environment causing information leak out of logical state of the qubits in the quantum memory ... Read More
Random Linear Network Coding (RLNC)
Network Coding Approaches for Distributed Computation over Lossy Wireless Networks https://pmc.ncbi.nlm.nih.gov/articles/PMC10047255B. Fan, B. Tang, Z. Qu, and B. Ye, “Network Coding Approaches for Distributed Computation over Lossy Wireless Networks,” Entropy, vol. 25, no. 3, p. 428, Feb. 2023, doi: 10.3390/e25030428. Ho, T.; Médard, M.; Koetter, R.; Karger, D.R.; Effros, M.; Shi, J.; Leong, B. A ... Read More
Network Coding – Wireless Communication
https://www.nature.com/research-intelligence/nri-topic-summaries/network-coding-in-wireless-communication-systems-micro-22183https://www.microsoft.com/en-us/research/wp-content/uploads/2007/07/tr-2007-90.pdf Network Coding: Generalization of Routing Information delivery in today’s wireless networks is performed by routing which is intermediate nodes (routers) store and forward data. Now, network coding is the recent generalization of routing in which nodes can generate output data which is produced from encoding or applying certain functions on the previously received input ... Read More