Exercise 1 – Properties of Generalized Linear Models
Generalized Linear Models (GLMs) extend the exponential family to incorporate covariates A covariate, dear traveler of the Data Plains, is one of those mysterious creatures that quietly influences your outcome variable while pretending it’s just “passing through.” It’s the input — the X to your Y, the question to your answer, the weather to your umbrella sales. It doesn’t shout. It doesn’t demand attention. But without it, your precious prediction would be floating in statistical space like a lost sock in a cosmic dryer.
Think of covariates as the party guests who bring all the drama but never appear on the invitation. They shape the night, alter the mood, and leave you wondering why everyone’s crying over the guacamole.
You don’t have to understand them fully just yet — many a Neuronite spent entire epochs mistaking them for noise or furniture.
But heed this warning: random covariate shift is coming. It looms just offstage, wrapped in distribution drift and armed with a sinister smile. When it arrives, your model — so carefully trained, so sure of itself — will suddenly forget how to tie its own shoelaces.
So mark the word: covariate. Whisper it into your loss function. Write it in the margin of your notes. One day soon, you'll need it. Probably right after deployment., modeling the relationship between a response variable \( y \) and predictors \( \mathbf{x} \). A GLM consists of three components:
1. Random Component: The response \( y \) follows a distribution from the exponential family (e.g., Gaussian, Bernoulli, Poisson).
2. Systematic Component: A linear predictor \( \eta = \mathbf{w} \cdot \mathbf{x} \), where \( \mathbf{w} \) are weights and \( \mathbf{x} \) are covariates.
3. Link Function: A function \( g \) such that \( g(\mu) = \eta \), where \( \mu = E[y] \) is the expected value of the response.
Key properties of GLMs include:
- They use the exponential family to model various response distributions.
- The link function connects the linear predictor to the expected response.
- The log-likelihood is concave, allowing efficient maximum likelihood estimation (MLE) via iterative methods like Newton-Raphson.
Question:
Which of the following are true properties of GLMs? Select all that apply.
1. Random Component: The response \( y \) follows a distribution from the exponential family (e.g., Gaussian, Bernoulli, Poisson).
2. Systematic Component: A linear predictor \( \eta = \mathbf{w} \cdot \mathbf{x} \), where \( \mathbf{w} \) are weights and \( \mathbf{x} \) are covariates.
3. Link Function: A function \( g \) such that \( g(\mu) = \eta \), where \( \mu = E[y] \) is the expected value of the response.
Key properties of GLMs include:
- They use the exponential family to model various response distributions.
- The link function connects the linear predictor to the expected response.
- The log-likelihood is concave, allowing efficient maximum likelihood estimation (MLE) via iterative methods like Newton-Raphson.
Question:
Which of the following are true properties of GLMs? Select all that apply.