Modern FastLAD techniques aim to bring into the realm of seconds or minutes , comparable to OLS on the same hardware.
| Technique | Key Idea | Complexity (rough) | When It Shines | |-----------|----------|--------------------|----------------| | | Specialized LP simplex for L1 norms; exploits the structure of the LAD problem. | O(n p) per iteration; usually < 30 iterations. | Small‑to‑medium n (≤ 10⁵) with dense X. | | Iteratively Re‑Weighted Least Squares (IRLS) | Approximate |y‑Xβ| by a weighted L2 problem; weights = 1 / max(|r|, ε). | Each IRLS step O(np) (solve weighted LS). | Works well when a good initial β is available; moderate n. | | Coordinate Descent (CD) | Update one coefficient at a time using a 1‑D LAD sub‑problem (median of residuals). | O(n p) per full sweep; linear convergence in practice. | Very high‑dimensional, sparse X (e.g., text data). | | Alternating Direction Method of Multipliers (ADMM) | Split the absolute‑value term with an auxiliary variable; each sub‑problem is cheap (soft‑thresholding). | O(np) per ADMM iteration; can be parallelized. | Distributed/ GPU environments; n > 10⁶. | | Proximal Gradient / Stochastic Sub‑gradient | Use sub‑gradients of |r|; step size schedule + variance reduction. | O(p) per stochastic step; many passes over data. | Streaming data, online learning. | | Quantile‑Regression Formulation (τ = 0.5) | LAD = median quantile regression → solve via interior‑point methods tuned for τ = 0.5. | O(np √p) (state‑of‑the‑art interior‑point). | When you already need full quantile‑regression suite. | fastlad
: Add photos, "first conversation" screenshots, or even tiny QR codes that link to songs or videos. Modern FastLAD techniques aim to bring into the