23 | JRF•ML research•Ex-Data engineer • coding • cooking • maths
Bengaluru
Joined April 2025
- Tweets12.2K
- Following1.4K
- Followers3.7K
- Likes27.6K
Pinned Tweet
Math and ML → Part 1: The Mathematical Foundation of Regularization
Regularization is one of the most crucial techniques in machine learning for preventing overfitting. But have you ever wondered why it actually works from a mathematical perspective?
The Problem: When a model overfits, it essentially memorizes the training data instead of learning the underlying patterns. This leads to excellent performance on training data but poor generalization to unseen test data.
The Mathematical Root Cause: Let's dive into the math
In ordinary linear regression (first equation), we estimate parameters (θ*) by solving a system of equations.
Here's where things get tricky:
When your feature matrix exhibits high multicollinearity (features are highly correlated), certain columns become linearly dependent on others. Mathematically, this drives the determinant of the matrix toward zero, making it singular or leaning more towards a singular matrix.
The problems? The inverse becomes unstable, leading to:
>Unreliable parameter estimates
>Extreme coefficient values
>Poor model generalization
And here's the reality: Multicollinearity isn't some edge case; it's incredibly common in real world datasets.
The Mathematical Solution: Regularization, look at the second equation. Notice how the regularization parameter (α) is added to the diagonal of the matrix?
By adding αI to X^T X, we're mathematically guaranteeing that the matrix remains invertible, regardless of multicollinearity in the original data.
Even when X^T X is singular, (X^T X + αI) is always invertible for α > 0.
This ensures:
>Stable parameter estimation
>Controlled coefficient magnitudes
>Better generalization to new data
This is Part 1 of my "Math & ML" series, where I'll bridge the gap between mathematical theory and practical machine learning.
I want to show you the "why" behind the algorithms, not just the "how."
If you found this insightful and want more content connecting mathematics to ML intuition, let me know in the comments!
Retweets appreciated to help others discover the mathematical beauty in ML!
Manoj retweeted
Karnataka is the best state to go on a road trip. The sheer diversity is hard to beat.
From the lush green forests of Malnad to the rocky hills of North Karnataka to the sun kissed coastline of Mangalore - Udupi to the beautiful hills of Chikmagalur, Kodagu to the thick jungles of Bandipur, Nagarhole to the stunning Jog Falls.
From the magnificent Hoysala era temples of Belur, Halebid to glory of Hampi to the cave temples of Badami to the dazzling splendor of Mysuru Maharaja Palace.
On top of that very hospitable and friendly people, safe atmosphere and great food.
71 frames of our incredible Karnataka as a tribute to the 71st Kannada Rajyotsava!
71ನೇ ಕನ್ನಡ ರಾಜ್ಯೋತ್ಸವ!
ಒಂದು ರಾಜ್ಯ, ಹಲವು ಜಗತ್ತು!
#KarnatakaRajyotsava
Manoj retweeted
All Hindus are praying for this Muslim girl and demanding strong punishment for the boy. God forbid, if the situation were the other way around, many Muslims would be busy justifying the accused, calling him mentally ill, or coming up with some other excuse.
We have seen people even justify terror attacks and describe terrorists as “bhatke hue naujawan.”
There are many things Indian Muslims need to learn from Hindus.
Manoj retweeted
71 frames of our incredible Karnataka as a tribute to the 71st Kannada Rajyotsava!
71ನೇ ಕನ್ನಡ ರಾಜ್ಯೋತ್ಸವ!
ಒಂದು ರಾಜ್ಯ, ಹಲವು ಜಗತ್ತು!
#KarnatakaRajyotsava
Manoj retweeted
ಕನ್ನಡ ರಾಜ್ಯೋತ್ಸವದ ಸಂದರ್ಭದಲ್ಲಿ, ನಮ್ಮ ಕನ್ನಡಿಗರಿಗಾಗಿ ಕನ್ನಡದಲ್ಲೊಂದು ಹರಟೆ! ಕನ್ನಡಿಗನಾದ ನನಗೆ ತಾಂತ್ರಿಕ ವಿಷಯಗಳ ಬಗ್ಗೆ ಕನ್ನಡದಲ್ಲೇ ಮಾತನಾಡುವ ಅವಕಾಶ ಸಿಗುವುದು ಅಪರೂಪ.
A chit-chat in Kannada, for our Kannadigas! A rare opportunity for me to discuss technology in my mother tongue.
Link below.
people living in their cocoons without knowing the shyt reality of the job market and world
i want that life man
Manoj retweeted
To be held at @IITKgp , no registration fee, accomodation and TA is covered (if you get selected).
Register now at docs.google.com/forms/d/e/1F…
For more information do visit: ruralconnect2k26.swanlab.in/
Manoj retweeted
People visit France for a week and stuff their mouth with cotton to pronounce Bordeaux or Chamonix correctly to get validation from white skin folks. And then they live 10 years here and think it’s cool not to be able to pronounce Baiyyappanahalli or Hesaraghatta :/
Decided one thing that is I'm not doing PhD here
Now have to decide whether to continue this work or leave
My prof wants me to join phd under them
I am still confused whether to do PhD in india or abroad
When i was joining I told that i would do this only for 6-8 months after that I will leave
Now I worked on one objective and drafted 2 papers which are under review
I wanted experience and I got it how this side works and everything
So I was thinking to quit and maybe prep for gate last 2 months
Then look for better opportunities
Only downside is if anyone works on those papers when the review comes my name may get pushed to second author