Machine Learning Fundamentals, Part 2 with Shannon Wirtz
We continue our exploration of machine learning fundamentals with Shannon Wirtz, diving deeper into advanced model architectures, training techniques, and evaluation methods.
We start with ensemble learning - why combining multiple models often outperforms single models, and how techniques like Random Forest and XGBoost prevent overfitting through clever sampling strategies.
From there, we explore neural networks, understanding how they learn directly from raw data through sequences of linear and nonlinear transformations.
The conversation covers the evolution from convolutional neural networks (perfect for images) to recurrent neural networks (for sequences) to Transformers (the architecture behind modern LLMs).
We dive into how Transformers revolutionized natural language processing through parallelization and attention mechanisms, enabling the large language models we see today.
We then shift to the critical topic of model evaluation - exploring loss functions, gradient descent, learning rates, and the importance of proper train/validation/test splits.
Shannon explains why you need separate validation and test sets, how k-fold cross-validation works, and the various metrics used to assess model performance beyond simple accuracy.
Topics include:
* Ensemble learning: why combining models works (Random Forest, XGBoost)
* Neural networks: linear and nonlinear transformations, neurons, and layers
* Convolutional Neural Networks (CNNs): recognizing visual patterns and edges
* Transformers: the architecture behind modern LLMs, attention mechanisms, and why they’re so powerful
* Training and evaluation: loss functions, gradient descent, and learning rates
* Train/validation/test splits and why you need all three
* K-fold cross-validation: a more robust evaluation approach
* Performance metrics including precision, recall, F1 score, AUC, and the confusion matrix
* Model interpretation: white box vs black box models
* Interpretation techniques including partial dependence plots, SHAP values, and individual conditional expectations
* Learning resources: Andrew Ng’s courses, Kaggle, DataCamp, and hands-on projects
Shannon also shares his personal learning journey, from rote learning to practical hands-on experience, and discusses how he learns most effectively through immediate feedback and engaging projects.
Whether you’re looking to understand how modern AI systems work or seeking practical guidance on getting started with machine learning, this episode provides both theoretical depth and practical strategies for building and evaluating ML models.
This is Part 2 of a 2-part series. In
Part 1 [https://compiledconversations.com/14/]
, we explored the foundations of machine learning - including core concepts, terminology, different learning approaches, and fundamental model types.
Show Links
*
Shannon Wirtz on LinkedIn [https://www.linkedin.com/in/shannon-wirtz-a8387144/]
*
Ensemble Learning [https://en.wikipedia.org/wiki/Ensemble_learning]
*
Random Forest [https://en.wikipedia.org/wiki/Random_forest]
*
XGBoost [https://en.wikipedia.org/wiki/XGBoost]
*
Neural Networks [https://en.wikipedia.org/wiki/Neural_network]
*
Convolutional Neural Networks [https://en.wikipedia.org/wiki/Convolutional_neural_network]
*
DeepSeek-OCR [https://github.com/deepseek-ai/DeepSeek-OCR]
*
ImageNet [https://www.image-net.org/]
*
AlexNet [https://en.wikipedia.org/wiki/AlexNet]
*
Transformers [https://en.wikipedia.org/wiki/Transformer_%28machine_learning_model%29]
*
Attention Is All You Need [https://arxiv.org/abs/1706.03762]
*
Gradient Descent [https://en.wikipedia.org/wiki/Gradient_descent]
*
Cross-Validation [https://en.wikipedia.org/wiki/Cross-validation_%28statistics%29]
*
Precision and Recall [https://en.wikipedia.org/wiki/Precision_and_recall]
*
F1 Score [https://en.wikipedia.org/wiki/F-score]
*
AUC (Area Under Curve) [https://en.wikipedia.org/wiki/Receiver_operating_characteristic]
*
Confusion Matrix [https://en.wikipedia.org/wiki/Confusion_matrix]
*
SHAP Values [https://shap.readthedocs.io/]
*
Partial Dependence Plots [https://scikit-learn.org/stable/modules/partial_dependence.html]
*
Andrew Ng’s Machine Learning Course [https://www.coursera.org/specializations/machine-learning-introduction]
*
DeepLearning.AI [https://www.deeplearning.ai/]
*
Kaggle [https://www.kaggle.com/]
*
DataCamp [https://www.datacamp.com/]
*
R Programming Language [https://www.r-project.org/]
*
Why Machines Learn [https://anilananthaswamy.com/why-machines-learn]
*
Hands-On Machine Learning [https://www.oreilly.com/library/view/hands-on-machine-learning/9781492032632/]
*
Neural Networks and Deep Learning [http://neuralnetworksanddeeplearning.com/]
