Hey there, folks! Billy here, back with another tale from the trenches of applied data science. Today, we’re tackling the beast known as the mass point—a pesky cluster of values that throws off our beloved linear regression, especially after a log transformation. Spoiler alert: it’s like having…
This one summarizes what I've captured from Chapter 14 of "Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction" book by Guido W. Imbens and Donald B. Rubin. Few shortcuts I've used: T - shortcut for treatment groupC - shortcut for control groupCovered sections and their summary: 14.1…
This is the summary of "Recommendations as Treatments" (paper) by Joachims et al. Image created by Dall-e: "Humorous illustration of Inverse Propensity Weighting. It features a quirky professor explaining the concept to amused students in a classroom, with a seesaw representing the 'Treated' and 'Untreated' groups."The main proposal of…
Causal Inference cheatsheet based on Matheus Facure's book on Causal Inference. Created by ChatGPT. Overview of Causal Inference ConceptsConceptDescriptionExampleReferenceCausal InferenceDetermining the cause-and-effect relationship between variables.Assessing the impact of a new drug on patient recovery.LinkTreatment/InterventionThe variable or action being studied for its effect on an outcome.A new…
The Relationship Between Covariance and Dot ProductCovariance and the dot product are related concepts in mathematics and statistics, particularly in the context of vectors and random variables. Here's how they are connected: Dot ProductThe dot product (also known as the scalar product) of two vectors a and b in n-dimensional…
Proof Of Concept (POC) is one of the key phases in Machine Learning (ML)-related projects. Iteratively coming up with a better modeling approach, improving data quality, and finally, delivering a minimum viable solution for a grand problem is fascinating and you learn a lot.…
Most of my previous work consisted of various data analysis and ML-related tasks. As of recently, I have been working on tasks related to data engineering, so I have decided to learn more about it. I have stumbled upon Chris Riccomini's talk @QConSanFrancisco and have learned quite a few terminologies…
Recently, I have spent my evenings participating in the Kaggle's "Jane Street Market Prediction" competition. To preserve the know-how acquired from the competition, I have written this blog post. If you would like to learn more about the competition click here. Contents Anonymized data set know-how Cross-validation know-how Neural network…