Machine Learning Guide

Discussing Databricks with Ming Chang from Raybeam [https://www.raybeam.com/] (part of DEPT® [https://deptagency.com/])

Conversation with Dirk-Jan Kubeflow (vs cloud native solutions like SageMaker) Dirk-Jan Verdoorn [https://www.linkedin.com/in/dirk-jan-verdoorn/] - Data Scientist at Dept Agency Kubeflow [https://www.kubeflow.org/]. (From the website:) The Machine Learning Toolkit for Kubernetes. The Kubeflow project is dedicated to making deployments of machine learning (ML) workflows on Kubernetes simple, portable and scalable. Our goal is not to recreate other services, but to provide a straightforward way to deploy best-of-breed open-source systems for ML to diverse infrastructures. Anywhere you are running Kubernetes, you should be able to run Kubeflow. TensorFlow Extended (TFX) [https://www.tensorflow.org/tfx]. If using TensorFlow with Kubeflow, combine with TFX for maximum power. (From the website:) TensorFlow Extended (TFX) is an end-to-end platform for deploying production ML pipelines. When you're ready to move your models from research to production, use TFX to create and manage a production pipeline. Alternatives: * Airflow [https://airflow.apache.org/] * MLflow [https://mlflow.org/]

Chatting with co-workers about the role of DevOps in a machine learning engineer's life Expert coworkers at Dept * Matt Merrill [https://www.linkedin.com/in/theothermattm/] - Principal Software Developer * Jirawat Uttayaya [https://www.linkedin.com/in/jirawat/] - DevOps Lead * The Ship It Podcast [https://shipit.io/] (where Matt features often) Devops tools * Terraform [https://www.terraform.io/] * Ansible [https://www.ansible.com/] Pictures (funny and serious) * Which AWS container service should I use? [https://www.reddit.com/r/aws/comments/nrulld/the_recent_all_the_ways_to_run_containers_on_aws/] * A visual guide on troubleshooting Kubernetes deployments [https://learnk8s.io/troubleshooting-deployments] * Public Cloud Services Comparison [https://comparecloud.in/] * Killed by Google [https://killedbygoogle.com/] * aCloudGuru AWS curriculum [https://acloudguru.com/blog/engineering/which-aws-certification-should-i-take]

(Optional episode) just showcasing a cool application using machine learning Dept uses Descript for some of their podcasting. I'm using it like a maniac, I think they're surprised at how into it I am. Check out the transcript & see how it performed. * Descript [https://www.descript.com?lmref=4bjg1g] * The Ship It Podcast [https://open.spotify.com/show/2P2BVhs19aqHVg7jUPuTjL] How to ship software, from the front lines. We talk with software developers about their craft, developer tools, developer productivity and what makes software development awesome. Hosted by your friends at Rocket Insights. AKA shipit.io * Brandbeats Podcast by BASIC [https://open.spotify.com/show/6P6UxVuOtSmrWb28rd4Pc2?si=ea136c020a8f4819] An agency podcast with views on design, technology, art, and culture. Explore the new microsite at www.brandbeats.basicagency.com

Show notes: ocdevel.com/mlg/mla-17 [https://ocdevel.com/mlg/mla-17] Developing on AWS first (SageMaker or other) Consider developing against AWS as your local development environment, rather than only your cloud deployment environment. Solutions: 1. Stick to AWS Cloud IDEs (Lambda [https://aws.amazon.com/lambda/], SageMaker Studio [https://aws.amazon.com/sagemaker/studio/], Cloud9 [https://aws.amazon.com/cloud9/] 2. Connect to deployed infrastructure via Client VPN [https://docs.aws.amazon.com/vpn/latest/clientvpn-admin/what-is.html] * Terraform example [https://timeular.com/blog/creating-an-aws-client-vpn-with-terraform/] * YouTube tutorial [https://www.youtube.com/watch?v=uADf-JbN-ZE] * Creating the keys [https://docs.aws.amazon.com/vpn/latest/clientvpn-admin/client-authentication.html#mutual] 3. LocalStack [https://localstack.cloud/] Infrastructure as Code * Terraform [https://www.terraform.io/] * CDK [https://aws.amazon.com/cdk/] * Serverless [https://www.serverless.com/]