🔴TechBeats live : LLM Quantization "vLLM vs. Llama.cpp" | Ep07

𝗧𝗵𝗲 𝗜𝗻𝗴𝗹𝗼𝗿𝗶𝗼𝘂𝘀 𝗖𝗹𝗼𝘂𝗱 𝗧𝗿𝗮𝗽𝘀 ☁️| Ep 09

👨🏽‍🚀 Welcome to Episode 09 of "Tech Beats unplugged" 𝗜𝗻𝗴𝗹𝗼𝗿𝗶𝗼𝘂𝘀 𝗖𝗹𝗼𝘂𝗱 𝗧𝗿𝗮𝗽𝘀 ☁️" With Mark Boost [https://www.linkedin.com/in/markboost/] CEO at Civo [https://www.linkedin.com/company/civocloud/] Cloud . 𝗜𝘀 𝘁𝗵𝗲 𝗖𝗹𝗼𝘂𝗱 𝗯𝗿𝗼𝗸𝗲𝗻 ...𝗼𝗿 𝗷𝘂𝘀𝘁 𝘂𝗻𝗳𝗮𝗶𝗿? This time, Mark Boost [https://www.linkedin.com/in/markboost/] and Kosseila H. go deep into the infamous 𝗖𝗹𝗼𝘂𝗱 𝘁𝗿𝗮𝗽𝘀 and explore where control is lost - and how to regained it for customers. 🌟 We’re so excited to share our latest tech Beats show with you! Topics discussed: 1. (00:23) Introduction 2. (01:42) From Formula 3 to serial entrepreneur 3. (08:44) lessons from s𝗰𝗮𝗹𝗶𝗻𝗴 𝗺𝘂𝗹𝘁𝗶𝗽𝗹𝗲 𝗰𝗼𝗺𝗽𝗮𝗻𝗶𝗲𝘀 (pivoting, competition, timing) 4. (18:59) CIVO Elevator pitch 5. (21:50) Why the 𝗰𝗹𝗼𝘂𝗱, 𝗮𝘀 𝘄𝗲 𝗸𝗻𝗼𝘄 𝗶𝘁, 𝗶𝘀 𝗯𝗿𝗼𝗸𝗲𝗻 ? 6. (28:59) The infamous cloud traps: egress fees, exit tax 7. (37:50) startup FREE credit : what's the catch? 8. (44:02) Cloud regulation & fairness: blessing or curse? 9. (47:32) Is Regulation killing AI innovation in Europe ? 10. (51:31) David vs Goliath: independent providers vs the Big 3 hyperscalers "the MOAT" 11. (56:56) 𝗔𝗜 𝗲𝗻𝗲𝗿𝗴𝘆 𝗰𝗿𝗶𝘀𝗶𝘀: how do we power AI without torching the grid👨🏻‍🔧🏭? 12. (01:01) ESG vs Greenwashing? 13. (01:03) 𝗢𝘂𝘁𝗮𝗴𝗲 𝗰𝗵𝗮𝗼𝘀: CEO mistakes — and how to do it right (CrowdStrike case) 𝗦𝗵𝗼𝘄 𝗡𝗼𝘁𝗲𝘀 * Civo website : civo.com [civo.com] * Egress Fees: Big Tech Braces For Digital Market Act [https://www.adexchanger.com/daily-news-roundup/tuesday-23012024/] * Civo talk "The cloud is broken":  The Cloud is Broken  [https://www.youtube.com/watch?v=x1Oo3T9Mphw] * Digital Market Acts [https://ec.europa.eu/commission/presscorner/detail/en/ip_24_4761] * ⁠European Data Act⁠⁠ [https://digital-strategy.ec.europa.eu/en/policies/data-act] * Free credits:A Call for Fair Competition by Addressing the Impact of Free Cloud Credits [https://www.linkedin.com/pulse/call-fair-competition-simon-hansford-0ogbe/?trackingId=VDIuXnFTRmuIogz03rWqfg%3D%3D] * Green policies ESG: ESG reporting: Carbon in the cloud [https://www.cio.com/article/2517653/esg-reporting-carbon-in-the-cloud.html] * Data Center Co2 footprint calculator [https://8billiontrees.com/carbon-offsets-credits/carbon-ecological-footprint-calculators/carbon-footprint-of-data-centers/] 🎙𝗔𝗯𝗼𝘂𝘁 𝗠𝗮𝗿𝗸 𝗕𝗼𝗼𝘀𝘁:⁠⁠⁠⁠⁠⁠⁠⁠Website: ⁠⁠https://markboost.com/ [https://www.youtube.com/redirect?event=video_description&redir_token=QUFFLUhqbFFzZm9CbnM0ZWpjZ3hsZGd3MkkyTHh1U3U0UXxBQ3Jtc0ttS1k4SnA3S25lWXhvQTN0c2dPcW01V19pRlhPd2JJd2E5aXhVTmo5OWV5ZDJyT2xKUTB3RGJzRm5oTlRTSi0wMnl6RmRKRkN2ZlFvY2xUaHljSzRRLXliZ3lYakVJR0pyTEExa1BVUHhmaFkzRFNSUQ&q=https%3A%2F%2Fmarkboost.com%2F&v=ByjQhnLznO0]LinkedIn: ⁠⁠ ⁠markboost   [https://www.linkedin.com/in/markboost]X(Twitter): ⁠@markboost10 tips for a successful career [markboost.com/2024/08/22/10-tips-for-a-successful-career/ ]

𝗡𝘂𝘁𝗮𝗻𝗶𝘅 𝗶𝗻 𝘁𝗵𝗲 𝗔𝗴𝗲 𝗼𝗳 𝘁𝗵𝗲 𝗩𝗶𝗿𝘁𝘂𝗮𝗹𝗶𝘇𝗮𝘁𝗶𝗼𝗻 𝗦𝗵𝗮𝗸𝗲-𝗨𝗽 (w/ Michael webster) | Ep08

👨🏽‍🚀 Welcome to Episode 08 of "Tech Beats unplugged" 🌐🦘This time the episode cuts through the 𝗩𝗠𝘄𝗮𝗿𝗲 𝗕𝗿𝗼𝗮𝗱𝗰𝗼𝗺 chaos and breaks down where Nutanix really fits in today’s virtualized/multicloud world. With the help of no other than Nutanix Guru Michael Webster all the way from News Zeeland 🎧 We’re so excited to share our latest tech Beats show with you🧡! Please share away 🤗 We hope you'll enjoy it!!! Topics discussed: 1. (00:21) Introduction 2. (01:58) What's your story with Nutanix 3. (09:41) Hyper Converged Infrastructure. What's that? 4. (13:38) Understanding Nutanix. 5. (18:25) The VMware–Broadcom Chaos for customers 6. (26:00) Running VMware on Nutanix (and moving off) 7. (30:05) Nutanix hardware ... a Lego of Nodes 8. (36:20) Nutanix building blocks - What's Controller VM 9. (43:05) Nutanix 𝗸𝗶𝗹𝗹𝘀 𝘁𝗵𝗲 𝗹𝗲𝗴𝗮𝗰𝘆 𝗥𝗔𝗜𝗗 storage - how? 10. (52:04) Nutanix; from 𝗵𝗮𝗿𝗱𝘄𝗮𝗿𝗲 𝘁𝗼 𝘀𝗼𝗳𝘁𝘄𝗮𝗿𝗲 company 11. (57:00) What is Nutanix MOAT going forward ? Show Notes * VMware customers face up to 1,500% price increase News⁠ [https://www.networkworld.com/article/3994107/vmware-customers-in-europe-face-up-to-1500-price-increases-under-broadcom-ownership.html] * ⁠Nutanix How it works: ⁠⁠⁠Video [https://www.youtube.com/watch?v=IKdzcwMY950] * Dutch court forces Broadcom to support VMware migration after 85% price hike: ⁠Networkworld⁠ [https://www.networkworld.com/article/4015489/dutch-court-forces-broadcom-to-support-vmware-migration-after-85-price-hike-backlash.html] * Nutanix Bible (nutanixbible.com [https://nutanixbible.com/])⁠ [https://github.com/JamesWoolfenden/ghat] * ⁠AT&T sues broadcom over licensing: ⁠⁠⁠ [https://adnanthekhan.com/2024/12/21/cacheract-the-monster-in-your-build-cache/]⁠Forbes [https://www.forbes.com/sites/stevemcdowell/2024/09/06/why-atts-suing-broadcom-over-forced-vmware-license-changes/] * Nutanix AHV: ⁠Built-in Enterprise-Class Virtualization [https://youtu.be/80UiJp0i7K0]⁠ 🎙About Michael Webster: * ⁠⁠⁠⁠⁠⁠ [https://www.linkedin.com/in/erjosito]Website: ⁠longwhiteclouds.com⁠ [https://longwhiteclouds.com ] * LinkedIn: ⁠Steve Giguere⁠ [https://www.linkedin.com/in/stevegiguere/] * X(Twitter): @vcdxnz001 [https://x.com/vcdxnz001] Brought to You by Cloudthrill [https://cloudthrill.ca].

🔴TechBeats live : LLM Quantization "vLLM vs. Llama.cpp" | Ep07

👋🏼 Hey AI heads 🎙️ Join us for the very first Tech Beats Live 🔴, hosted by Kosseila—aka @CloudDude from @CloudThrill. 🎯 This chill & laid-back livestream will unpack LLM quantization 🔥: * ✅ WHY it matters * ✅ HOW it works * ✅ Enterprise (vLLM) vs Consumer (@Ollama) trade-offs * ✅ and WHERE it’s going next. We’ll be joined by two incredible guest stars to talk Enterprise vs Consumer Quantz 🗣️: 🔷 Eldar Kurtić – bringing the enterprise perspective with vLLM. 🔷 Colin Kealty – aka Bartowski, creator of the top-downloaded GGUF quantized LLMs on Hugging Face. 🫵🏼 Come learn and have some fun 😎. 𝐂𝐡𝐚𝐩𝐭𝐞𝐫𝐬: (00:00) Host Introduction (04:07) Eldar Intro (07:33) Bartowski Intro (13:04) What’s Quantization! (16:19) Why LLM Quantization Matters? (20:39) Training vs Inference – “The New Deal” (27:46) Biggest Misconception About Quantization (33:22) Enterprise Quantization in Production (vLLM) (48:48) Consumer LLMs & Quantization (Ollama, llama.cpp, GGUF) – “LLMs for the People” (01:06:45) BitNet 1-Bit Quantization from Microsoft (01:28:14) How Long It Takes to Quantize a Model (Llama-3 70B) – GGUF or lm-compressor (01:34:23) What Is I-Matrix & Why People Confuse It with IQ Quantization? (01:39:36) What’s LoRA & LoRA-Q? (01:42:36) What Is Sparsity? (01:47:42) What Is Distillation? (01:52:34) Extreme Quantization (Unsloth) of Big Models (DeepSeek) at 2-bits 70 % Size Cut (01:57:27) Will Future Models (Llama-5) Be Trained on FP4 Tensor Cores? (02:02:15) The Future of LLMs on Edge Devices (Google AI Edge) (02:08:00) How to Evaluate the Quality of a Quantized Model (02:26:09) Hugging Face’s Role in the World of LLM/Quantization (02:33:46) Hugging Face’s Role in the World of LLM/Quantization (02:36:41) LocalLlama Sub-Reddit Down (Moderator Goes Bananas) (02:40:11) Guests’ Hope for the Future of LLMs & AI in General 📖 Check out the quantization blog: https://bitly/LLMQuant [https://www.youtube.com/redirect?event=video_description&redir_token=QUFFLUhqbGZlNEtpUW1VZU9XUFhucG5Tbng2aUtabVNtQXxBQ3Jtc0tuYWNXVDBsaWpLaHVxRDhRZklqNjZhX0tTYWF6cFA2UC1VYU1OSW1Fc2N4SUF1WEJremZxZXc5U3hSMEZkLXhwUDV5Z3dCOFFXc2oyc1I1d1gtU2FuSVRhWE1jem12OGUxbnYzS2dITURQQVhaMHVwOA&q=https%3A%2F%2Fcloudthrill.ca%2Fllm-quantization-all-you-need-to-know&v=XTE0oS7b6fM] #AI #LLM #Quantization #TechBeatsLive #LocalLlama #vLLM #Ollama

Ep06: "GitHub Security horror stories " (with Steve Giguere)

👨🏽‍🚀 Welcome to Episode 06 of "Tech Beats unplugged" This time, we’re diving headfirst into 𝐭𝐡𝐞 𝐜𝐫𝐚𝐳𝐢𝐞𝐬𝐭 𝐆𝐢𝐭𝐇𝐮𝐛 𝐬𝐞𝐜𝐮𝐫𝐢𝐭𝐲 𝐬𝐭𝐨𝐫𝐢𝐞𝐬, and who better to join us than Steve Giguere, an industry veteran and security expert who’s seen it all. From supply chain security mayhem to GitHub Actions gone wrong, we uncover real-world security blunders, attack vectors, and best practices to keep your repos and workflows safe. 🌟 We’re so excited to share our latest tech Beats show with you🧡! Please share away 🤗 We hope you'll enjoy it!!! Topics discussed: 1. (00:00) Introduction 2. (03:53) Software Supply Chain Security acronyms (SAST, DAST, IAST, etc.) 3. (09:15) “A workflow is an application within your application” - What does that mean?! 4. (12:16) Public vs. Private Repos - Are private orgs still at risk? 5. (18:27) Self-hosted runners: Safe or security nightmare? 6. (21:16) GitHub Environment Variables - How critical are they? 7. (22:55) Secrets, masks, and how secure they really are 8. (28:05) Artifact vs. Caching: Which is safer? 9. (31:27) Craziest GitHub security screw-ups Steve has ever seen 🔥 10. (36:42) Common attack vectors in GitHub Actions 11. (44:19) Best security practices for GitHub Actions - Low-hanging fruit fixes 🍏 12. (50:22) Are public actions safe? Can they be scanned? 13. (53:52) xz backdoor fiasco - Lessons from the latest supply chain attack 14. (59:00) NVD’s slowdown - What’s at stake? Show Notes * CI/CD Goat (Deliberately vulnerable CI/CD environment): GitHub [https://github.com/cider-security-research/cicd-goat] * GitHub cache poisoning: Cacheract Attack [https://adnanthekhan.com/2024/12/21/cacheract-the-monster-in-your-build-cache/] | ScribeSecurity [https://scribesecurity.com/blog/github-cache-poisoning/] * Your GitHub Secrets in Plain Text: CloudThrill [https://cloudthrill.ca/your-github-secrets-aint-that-secret] * Ghat tool (Updating dependencies in GitHub Actions): GitHub [https://github.com/JamesWoolfenden/ghat] * OpenSSF Scorecard: Website [https://scorecard.dev/] * The GitHub Worm (Asi Greenholts): Palo Alto Blog [https://www.paloaltonetworks.com/blog/prisma-cloud/github-actions-worm-dependencies/] * OWASP Top 10 CI/CD Risks: OWASP [https://owasp.org/www-project-top-10-ci-cd-security-risks/] * Heartbleed OpenSSL Exploit: Wikipedia [https://en.wikipedia.org/wiki/Heartbleed] 🎙About Steve Giguere: * ⁠⁠⁠⁠ [https://www.linkedin.com/in/erjosito]Website: stevegiguere.com [https://stevegiguere.com] * LinkedIn: Steve Giguere [https://www.linkedin.com/in/stevegiguere/] * Book: Cloud Native Application Protection Platforms – O'Reilly [https://www.oreilly.com/library/view/cloud-native-application/9781098141691/] * Personal Blog: Codifyre [https://codifyre.com/] * Talk Lessons Learned from OSS and GitOps Journey: YouTube [https://youtu.be/xH_wzHwwQho] * OWASP Lisbon Talk: YouTube [https://youtu.be/-WxtnUHhrlc?feature=shared] * StayWiredIn YouTube Show: StayWiredIn [https://www.youtube.com/@staywiredin] * DevSecOps Podcast: Spotify [https://open.spotify.com/show/0XVk0AKg26yLTCMMwkIA7m]

Ep05: "Deploy Local LLMs 𝐢𝐧 the Cloud (𝟏𝟎𝟎% 𝐃𝐚𝐭𝐚 𝐏𝐫𝐢𝐯𝐚𝐜𝐲)"

👨🏽‍🚀 Welcome to Episode 05 of "Tech Beats unplugged" This time, we tried something completely crazy – we're letting the AI hosts take over! That's right 😎. We're flipping the script and giving the AI the mic to guide us through the fascinating world of local LLMs. but that's not all as this episode is actually inspired by my recent talk at Oracle Cloud World in Vegas. The topic? You guessed it: Local LLMs in the cloud. 🌟 We’re so excited to share our latest tech Beats show with you🧡! We hope you'll enjoy it!!! Topics discussed: 1. (00:00) Introduction 2. (01:00) Why OpenAI Might Not Be Your BFF? 3. (02:40) Local/Open LLMs to the Rescue! 4. (03:38) What's Quantization! 5. (04:30) Where to find these Open LLMs? 6. (05:02) Inference Engines (Ollama)! 7. (05:50) What's a modelfile! 8. (06:40) What about deploying local AI to the cloud?(OKE/managed kubernetes) 9. (07:30) From zero to cloud deployment Hero 10. (08:28) What's Next (LLM ethic benchmark) 11. (09:55) Outro. Show Notes * My local LLM GitRepo: Ollama_lab [https://github.com/brokedba/ollama_lab] * Helm leaderboard for model safety: Sandford Helm model leaderboard [https://crfm.stanford.edu/helm/classic/latest/#/leaderboard] * My talks in Oracle cloud world 2024: OCW2024LLM [https://bit.ly/OCW2024LLM]