In this video, we take a look at one of the most beautiful algorithms ever created: the Fast Fourier Transform (FFT). This is a tricky algorithm to understand so we take a look at it in a context that we are all familiar with: polynomial multiplication. You will see how the core ideas of the FFT can be "discovered" through asking the right questions. The key insights that are presented in this video is that polynomial multiplication can be improved significantly by multiplying polynomials in a special value representation. The challenge that presents itself is the problem of converting a polynomial from a standard coefficient representation to value representation. 

We see that the FFT is an incredibly efficient recursive algorithm that performs this task, and we also discover that a slightly tweaked FFT (Inverse FFT) can also solve the reverse problem of interpolation. If this video doesn't blow your mind, I don't know what will. 

0:00 Introduction
2:19 Polynomial Multiplication
3:36 Polynomial Representation
6:06 Value Representation Advantages
7:07 Polynomial Multiplication Flowchart
8:04 Polynomial Evaluation
13:49 Which Evaluation Points?
16:30 Why Nth Roots of Unity?
18:28 FFT Implementation
22:47 Interpolation and Inverse FFT
26:49 Recap

Also a subtle mistake that a commenter made me aware of -- at 26:40 instead of replacing w with (1/n * e^{-2 * pi i/ n}), the actual right way to do this is by taking the final output of the IFFT at the end of the recursion and dividing by n. 

So the full change is w = e^{-2 pi i / n} 
And then somewhere outside the scope of the IFFT function ifft_result = 1/n * IFFT(values)

The treatment of the FFT in this video is inspired by several well known references, mainly Introduction to Algorithms (Cormen et al.) and Algorithms (Papadimitriou et al.).

Support: https://www.patreon.com/reducible

This video wouldn't be possible without the open source manim library created by 3blue1brown: https://github.com/3b1b/manim

Here is link to the repository that contains the code used to generate the animations in this video: https://github.com/nipunramk/Reducible

Elegant proof that the matrix used in the proof that (d + 1) points uniquely define a degree d polynomial is invertible: https://math.stackexchange.com/questions/426932/why-are-vandermonde-matrices-invertible

Music:
Lift Motif by Kevin MacLeod is licensed under a Creative Commons Attribution license (https://creativecommons.org/licenses/...)
Source: http://incompetech.com/music/royalty-...
Artist: http://incompetech.com/

All other music by Aakash Gandhi

SVG Attributions:
Earth: Designed by Flat Icons from www.flaticon.com, CC BY 4.0 https://creativecommons.org/licenses/by/4.0, via Wikimedia Commons
GPS: Icons made by https://www.flaticon.com/authors/pause08 from https://www.flaticon.com/
Wireless Comms: Design inspired by https://svgsilh.com/image/297434.html

The Fast Fourier Transform (FFT): Most Ingenious Algorithm Ever?

Visit  https://brilliant.org/Reducible/ to get started learning STEM for free, and the first 200 people will get 20% off their annual premium subscription.

Chapters:
00:00 Introducing JPEG and RGB Representation
2:15 Lossy Compression
3:41 What information can we get rid of?
4:36 Introducing YCbCr
6:10 Chroma subsampling/downsampling
8:10 Images represented as signals
9:52 Introducing the Discrete Cosine Transform (DCT)
11:32 Sampling cosine waves
12:43 Playing around with the DCT
17:38 Mathematically defining the DCT
21:02 The Inverse DCT
22:45 The 2D DCT
23:49 Visualizing the 2D DCT
24:35 Introducing Energy Compaction
26:05 Brilliant Sponsorship
27:23 Building an image from the 2D DCT
28:20 Quantization
30:23 Run-length/Huffman Encoding within JPEG
32:56 How JPEG fits into the big picture of data compression

The JPEG algorithm is rather complex and in this video, we break down the core parts of the algorithm, specifically color spaces, YCbCr, chroma subsampling, the discrete cosine transform, quantization, and lossless encoding. The majority of the focus is on the mathematical and signal processing insights that lead to advancements in image compression and the big themes in compression as a whole that we can take away from it. 

Animations created jointly by Nipun Ramakrishnan and Jesús Rascón.

References/Additional Resources:
https://www.red.com/red-101/video-chroma-subsampling - great resource on different types of chroma subsampling

http://weitz.de/dct/ - play around with the DCT

https://www.cse.iitd.ac.in/~pkalra/col783-2017/DCT-History.pdf - paper referenced in video

http://www.ee.ic.ac.uk/hp/staff/dmb/courses/DSPDF/00300_Transforms.pdf - a more rigorous signal processing approach to the DCT

https://www.impulseadventure.com/photo/jpeg-huffman-coding.html - great landing point for learning more about how huffman codes work in the context of JPEG

https://www.youtube.com/watch?v=CPT4FSkFUgs&list=PLpsTn9TA_Q8VMDyOPrDKmSJYt1DLgDZU4&ab_channel=DanielHarding - a great playlist I recommend that dives deep into actually implementing a JPEG decoder

This video wouldn't be possible without the open source library manim created by 3blue1brown and maintained by Manim Community.
The Manim Community Developers. (2021). Manim – Mathematical Animation Framework (Version v0.11.0) [Computer software]. https://www.manim.community/

Here is link to the repository that contains the code used to generate the animations in this video: https://github.com/nipunramk/Reducible

All music in the video is from Aakash Gandhi

The Unreasonable Effectiveness of JPEG: A Signal Processing Approach

In this video, we take a look at one of the more challenging computer science concepts: Recursion. We introduce 5 simple steps to help you solve challenging recursive problems and show you 3 specific examples, each progressively more difficult than the last.

Support: https://www.patreon.com/reducible

This video wouldn't be possible without the open source manim library created by 3blue1brown: https://github.com/3b1b/manim

Here is link to the repository that contains the code used to generate the animations in this video: https://github.com/nipunramk/Reducible

Music: 
October by Kai Engel https://freemusicarchive.org/music/Kai_Engel/Chapter_Four__Fall/Kai_Engel_-_Chapter_Four_-_Fall_-_05_October
November by Kai Engel
https://freemusicarchive.org/music/Kai_Engel/Chapter_Four__Fall/Kai_Engel_-_Chapter_Four_-_Fall_-_08_November
Cobweb Morning by Kai Engel
https://freemusicarchive.org/music/Kai_Engel/Chapter_Four__Fall/Kai_Engel_-_Chapter_Four_-_Fall_-_04_Cobweb_Morning

5 Simple Steps for Solving Any Recursive Problem

In this video, we go over five steps that you can use as a framework to solve dynamic programming problems. You will see how these steps are applied to two specific dynamic programming problems: the longest increasing subsequence problem and optimal box stacking. The five steps in order are as follows:

1. Visualize examples
2. Find an appropriate subproblem
3. Find relationships among subproblems
4. Generalize the relationship
5. Implement by solving subproblems in order

After taking an in depth look at these problems, at the end of the video we will also have a discussion about common subproblems that you may encounter while solving dynamic programming problems.

Error correction: for the box problem, using dictionary solutions only works if we are given unique boxes -- using a list of subproblems would be a better way to solve it if we wanted to handle duplicate boxes (similar to how we did the longest increasing subsequence). 

Support: https://www.patreon.com/reducible

This video wouldn't be possible without the open source manim library created by 3blue1brown: https://github.com/3b1b/manim

Here is link to the repository that contains the code used to generate the animations in this video: https://github.com/nipunramk/Reducible

Music:
Prelude No. 2 by Chris Zabriskie is licensed under a Creative Commons Attribution license (https://creativecommons.org/licenses/by/4.0/)
Source: http://chriszabriskie.com/preludes/
Artist: http://chriszabriskie.com/

All other music by Aakash Gandhi

5 Simple Steps for Solving Dynamic Programming Problems

Visit  https://brilliant.org/Reducible/ to get started learning STEM for free, and the first 200 people will get 20% off their annual premium subscription.

Chapters:
0:00 Introduction
1:35 Exploiting redundancy
2:09 Huffman Codes
4:22 Run Length Encoding
5:23 Lempel-Ziv Schemes (LZSS)
13:50 Transforming the problem
14:36 Filtering
17:46 How PNG Picks Filters
18:58 Heuristics for Filtering
21:06 PNG Encoding/Decoding
23:04 The Compromise: PNG Is Slow
23:31 Quite OK Image (QOI) Format
28:26 Benchmark Results
29:27 Final Thoughts
30:36 Brilliant Sponsorship

Developed in the mid 1990's, the PNG encoding standard for images is now the most used image format on the web. One aspect that makes PNG quite attractive for users is image quality is perfectly preserved as a result of lossless compression. In this video, we dive into how exactly PNG is able to compress images so well, while also retaining all the information in the image. 

One thing we discover during this journey is how slow PNG is in practice. Towards the end, we discuss a relatively new image format called QOI that is somewhat comparable to PNG, but 20-50x faster. But what's perhaps the most remarkable aspect of QOI is how a simple set of rules lead to surprisingly good compression on a variety of images. 

Animations created jointly by Nipun Ramakrishnan and Jesús Rascón.

*Minor error in QOI section: QOI_DIFF_SMALL should have a tag of 01 not 10 (QOI_DIFF_MED has the tag 10.*

References:
PNG Specification: https://www.w3.org/TR/PNG/#9Filters and http://www.libpng.org/pub/png/spec/1.2/png-1.2.pdf

A nice interactive guide to LZSS: https://go-compression.github.io/algorithms/lzss/

A definitive guide to all the details we didn't have time to talk about with respect to PNG: http://www.libpng.org/pub/png/book/chapter08.html

A deep dive into all the details of DEFLATE: https://www.youtube.com/watch?v=oi2lMBBjQ8s&ab_channel=BillBird

Initial blog post introducing QOI: https://phoboslab.org/log/2021/11/qoi-fast-lossless-image-compression

Data on Image File Format Usage across Websites: https://w3techs.com/technologies/overview/image_format

QOI Specification: https://qoiformat.org/qoi-specification.pdf

QOI Source Code/Benchmark Results https://qoiformat.org/: 

This video wouldn't be possible without the open source library manim created by 3blue1brown and maintained by Manim Community.
The Manim Community Developers. (2022). Manim – Mathematical Animation Framework (Version v0.11.0) [Computer software]. https://www.manim.community/

Here is link to the repository that contains the code used to generate the animations in this video: https://github.com/nipunramk/Reducible

Music in this video comes from Jesús Rascón, Aaskash Gandhi, and Myuu.

Socials:
Patreon: https://www.patreon.com/reducible
Twitter: https://twitter.com/Reducible20

Big thanks to the community of Patreons that support this channel. Special thanks to the following Patreons:
Andreas
Adam Dřínek
Burt Humburg
Matt Q
Winston Durand
Andjela Arsic
Mutual Information
Richard Wells
Sebastian Gamboa
Zac Landis

How PNG Works: Compromising Speed for Quality

In this video, I introduce the field of graph theory. We first answer the important question of why someone should even care about studying graph theory through an application perspective. Afterwards, we introduce definitions and essential terminology in graph theory, followed by a discussion of the types of graphs you may encounter. We then define several ways to represent graphs as a data structure and finish off the video with a discussion of what types of interesting problems you can ask about graphs to help motivate the ideas in future videos. 

Typo correction: at 5:12 the vertex set V should be {0, 1, 2, 3, 4} instead of {0, 1, 2, 3, 4, 5} (there is no vertex 5). 

Big thanks to Dániel László Bertalan for making the closed captions for this video!

The sudoku example was inspired by this incredible reddit visualization: https://www.reddit.com/r/dataisbeautiful/comments/6ty4vf/visualizing_the_sudoku_connectivity_graph_more_in/

Support: https://www.patreon.com/reducible

This video wouldn't be possible without the open source manim library created by 3blue1brown: https://github.com/3b1b/manim

Here is link to the repository that contains the code used to generate the animations in this video: https://github.com/nipunramk/Reducible

Music: 
October by Kai Engel https://freemusicarchive.org/music/Ka...
November by Kai Engel
https://freemusicarchive.org/music/Ka...
Cobweb Morning by Kai Engel
https://freemusicarchive.org/music/Ka...

Introduction to Graph Theory: A Computer Science Perspective

In this video, we start with an interesting animation of blobby objects which we introduce as metaballs. There's a lot of surprisingly intricate ideas behind making these objects render on a screen. We'll see how folks in computer graphics attempted to solve this problem through a really elegant algorithm called marching squares. Marching squares is a really powerful algorithm that allows you to render any implicit function. But what's even more impressive in my opinion is the many clever shifts in perspective that allowed a vague problems such as this one to be transformed into a clear, well-defined, and solvable problem.

0:00 Introduction
3:29 Circles and Ellipses
4:57 Defining the Problem
6:00 A Guessing Game
8:29 Contours around Two Points
10:35 Sampling The Space
12:32 Breaking Down Cases
15:00 A Clever Optimization
17:20 How Marching Squares Works
18:59 Parallel Marching Squares
20:21 How Do Metaballs Work?
24:59 Marching Cubes
25:58 Some Parting Thoughts

References/Additional Resources:
http://jamie-wong.com/2014/08/19/metaballs-and-marching-squares/ - the initial inspiration for the framework of this video, great introduction to metaballs and how they can be rendered using marching squares.

http://www.geisswerks.com/ryan/BLOBS/blobs.html  - great resource on implementing metaballs and some of the physics inspirations behind implicit functions

Original Marching cubes paper: https://www.researchgate.net/publication/202232897_Marching_Cubes_A_High_Resolution_3D_Surface_Construction_Algorithm 

Sebastian Lague Video on Marching Cubes:
https://www.youtube.com/watch?v=M3iI2l0ltbE

Further reading on polynomial approximations of metaball implicit functions:
https://www.researchgate.net/publication/242914163_Data_Structure_for_Soft_Objects

Implementation Help for Marching Cubes
http://paulbourke.net/geometry/implicitsurf/ 

Excellent lecture by Casey Muratori about Marching Cubes: https://www.youtube.com/watch?v=SDS5gLSiLg0&t=978s

https://courses.lumenlearning.com/physics/chapter/19-4-equipotential-lines/ - some of the physics behind equipotential lines in electric fields

Support: https://www.patreon.com/reducible
Twitter: https://twitter.com/Reducible20

This video wouldn't be possible without the open source library manim created by 3blue1brown and maintained by Manim Community.
The Manim Community Developers. (2021). Manim – Mathematical Animation Framework (Version v0.11.0) [Computer software]. https://www.manim.community/

Here is link to the repository that contains the code used to generate the animations in this video: https://github.com/nipunramk/Reducible

All music in the video is from Aakash Gandhi

How Computers Draw Weird Shapes (Marching Squares)

Weekly system design newsletter: https://bit.ly/3tfAlYD

Checkout our bestselling System Design Interview books: 
Volume 1: https://amzn.to/3Ou7gkd
Volume 2: https://amzn.to/3HqGozy

Other things we made:
Digital version of System Design Interview books: https://bit.ly/3mlDSk9
Twitter: https://bit.ly/3HqEz5G
LinkedIn:  https://bit.ly/39h22JK

Animation tools: Illustrator and After Effects

ABOUT US: 
Covering topics and trends in large-scale system design, from the authors of the best-selling System Design Interview series.