Kodsnack

Fredrik talks to Matt Topol [https://github.com/zeroshade] about Arrow [https://arrow.apache.org/] and how the Arrow ecosystem is evolving. Arrow is an open source, columnar in-memory data format designed for efficient data processing and analytics - which means passing data between things without needing to transform it, and ideally even without needing to copy it. What makes the ecosystem grow, and why is it very cool to have Arrow on the GPU? What is the connection between Arrow, machine learning, and Hugging face? Matt emphasizes the value of open standards, even as they work with or within more closed systems they can help open things up, and help bring about more modular solutions so that developers can focus on doing their core area really well. This episode can be seen as a follow-up to episode 567 [https://kodsnack.se/567/], where Matt first joined to discuss everything Arrow. Recorded during Øredev 2024 [https://archive.oredev.org/2024/#/]. Thank you Cloudnet [http://www.cloudnet.se/] for sponsoring our VPS [http://en.wikipedia.org/wiki/Virtual_private_server]! Comments, questions or tips? We a re @kodsnack [https://www.twitter.com/kodsnack], @tobiashieta [https://www.twitter.com/tobiashieta], @oferlund [https://twitter.com/oferlund]and @bjoreman [https://www.twitter.com/bjoreman] on Twitter, have a page on Facebook [https://www.facebook.com/kodsnack] and can be emailed at info@kodsnack.se [info@kodsnack.se] if you want to write longer. We read everything we receive. If you enjoy Kodsnack we would love a review in iTunes [http://itunes.apple.com/se/podcast/kodsnack/id561631498?l=en]! You can also support the podcast by buying us a coffee (or two!) through Ko-fi [https://ko-fi.com/kodsnack]. Links * Matt [https://github.com/zeroshade] * Matt’s Øredev 2023 talks: State of the Apache Arrow ecosystem: How your project can leverage Arrow! [https://www.youtube.com/watch?v=CEdbHXBPXHk&list=PLOUKmSqExtAH0k42evc9j3fiqfgHu00Cf&index=68] and Leveraging Apache Arrow for ML workflows [https://www.youtube.com/watch?v=dSgvht0RMzk&list=PLOUKmSqExtAH0k42evc9j3fiqfgHu00Cf&index=79] * Previous episodes with Matt [https://kodsnack.se/people/matt-topol/] * Øredev 2024 [https://archive.oredev.org/2024/#/] * Matt’s Øredev 2024 talks - on Arrow ADBC [https://www.youtube.com/watch?v=k8fu67y7Lrs&list=PLOUKmSqExtAFpg3krEd6CXr3uIyUgP97b&index=79] and Composable and modular data systems [https://www.youtube.com/watch?v=JnCEn6tSdEI&list=PLOUKmSqExtAFpg3krEd6CXr3uIyUgP97b&index=73] * ADBC [https://arrow.apache.org/adbc/current/index.html] - Arrow database connectivity * Arrow [https://arrow.apache.org/] * Snowflake [https://www.snowflake.com/en/] * Snowflake drivers for ADBC [https://arrow.apache.org/adbc/main/driver/snowflake.html] * Bigquery [https://en.wikipedia.org/wiki/BigQuery] * The Bigquery driver [https://arrow.apache.org/adbc/16/r/adbcbigquery/index.html] * Microsoft Fabric [https://www.microsoft.com/en-us/microsoft-fabric] * Duckdb [https://en.wikipedia.org/wiki/DuckDB] * Postgres [https://en.wikipedia.org/wiki/PostgreSQL] * SQLite [https://en.wikipedia.org/wiki/SQLite] * Arrow flight [https://arrow.apache.org/docs/format/Flight.html] - RPC framework for services based on Arrow data * Arrow flight SQL [https://arrow.apache.org/docs/format/FlightSql.html] * Microsoft Power BI [https://en.wikipedia.org/wiki/Microsoft_Power_BI] * Velox [https://velox-lib.io/] * Apache datafusion [https://datafusion.apache.org/] * Query planning [https://en.wikipedia.org/wiki/Query_plan] * Substrait [https://substrait.io/] - query IR * Polaris [https://polaris.apache.org/] * Libcudf [https://docs.rapids.ai/api/libcudf/stable/] * Nvidia RAPIDS [https://developer.nvidia.com/rapids] * Pytorch [https://en.wikipedia.org/wiki/PyTorch] * Tensorflow [https://en.wikipedia.org/wiki/TensorFlow] * Arrow device interface [https://arrow.apache.org/docs/format/CDeviceDataInterface.html] * DLPack [https://github.com/dmlc/dlpack] - in-memory tensor structure * Tensors [https://en.wikipedia.org/wiki/Tensor_%28machine_learning%29] * Nanoarrow [https://arrow.apache.org/nanoarrow/latest/index.html] * Voltron data [https://voltrondata.com/] - where Matt used to work. He’s now at Columnar [https://columnar.tech/] * Theseus [https://voltrondata.com/how-it-works] GPU compute engine * The composable data management system manifesto [https://www.vldb.org/pvldb/vol16/p2679-pedreira.pdf] * Support us on Ko-fi! [https://ko-fi.com/kodsnack] * Matt’s book [https://www.packtpub.com/en-us/product/in-memory-analytics-with-apache-arrow-9781835461228] - In-memory analytics with Apache Arrow * Spark [https://spark.apache.org/docs/latest/index.html] * Spark connect [https://spark.apache.org/docs/latest/spark-connect-overview.html] * RPC [https://en.wikipedia.org/wiki/Remote_procedure_call] * UDFs [https://github.com/arrow-udf/arrow-udf] * Photon [https://www.databricks.com/product/photon] * Datafusion [https://datafusion.apache.org/] * Apache Cassandra [https://www.google.com/search?client=safari&rls=en&q=apache+cassandra&ie=UTF-8&oe=UTF-8] * ODBC [https://en.wikipedia.org/wiki/Open_Database_Connectivity] * JDBC [https://en.wikipedia.org/wiki/Java_Database_Connectivity] * R [https://en.wikipedia.org/wiki/R_%28programming_language%29] - programming language for statistical computing * Hugging face [https://huggingface.co/] * Ray [https://www.ray.io/] * Stringview [https://arrow.apache.org/docs/format/Columnar.html#variable-size-binary-view-layout] - “German-style strings” * Scaling up with R and Arrow [https://arrowrbook.com/] - the book on using Arrow with R Titles * It’s gotten a lot bigger * The bones of it are in the repo * (Powered by ADBC) * Individual compute components * Feed it substrate * Where the ecosystem is going * Arrow on the GPU * The data stays on the GPU * A forced copy * Leverage that device interface * Without forcing the copy * Shy of that last mile * Turtles all the way down * The guy who said yes * German-style strings

Fredrik snackar med Viktor Löfgren [https://www.marginalia.nu/] om Marginalia search [https://marginalia-search.com/] - sökmotorn Viktor bygger på heltid och som i praktiken ensam utvecklare. Hur får man ens idén? Är det så svårt som det verkar? Hur kunde Marginalia search bli ett heltidsjobb? Det började med “det här kan väl aldrig funka, men varför inte?” Viktor berättar hela historien, från hybris och Raspberry pi till heltidsjobb och en riktig server. Ett projekt som faktiskt är “web scale” på riktigt! Och det med enserver. Allt är Java! Varför, och hur funkar det? Är det kanske en liten nackdel att de flesta som jobbar med Java har jobb? Och borde vi inte undvika tydligt definierade projekt och mål lite oftare och optimistiskt ta oss an svårare saker? Viktor slår ett slag för att försöka lösa lite större och svårare problem, istället för att förenkla och begränsa allt från start. Det finns inga problem med sökmotorer som är supersvåra, men man behöver ha många verktyg i sin verktygslåda. Återanvändning av kod? Inte så mycket som man skulle kunna tro. Kopiera och kasta bort är en bra riktning. Planering, listor, och att låta hjärnan jobba klart med bakgrundsprocesser. Rankning av sidor, analyser, och utmaningen att kartlägga, förstå, och motverka olika typer av sökmotoroptimering. Ett stort tack till Cloudnet [https://www.cloudnet.se/] som sponsrar vår VPS [https://en.wikipedia.org/wiki/Virtual_private_server]! Har du kommentarer, frågor eller tips? Vi är @kodsnack [https://social.podsnack.se/@kodsnack], @thieta [https://6510.nu/@thieta], @krig [https://6510.nu/@krig], och @bjoreman [https://toot.cafe/@bjoreman] på Mastodon, har en sida på Facebook [https://www.facebook.com/] och epostas på info@kodsnack.se [info@kodsnack.se] om du vill skriva längre. Vi läser allt som skickas. Gillar du Kodsnack får du hemskt gärna recensera oss i iTunes [https://itunes.apple.com/se/podcast/kodsnack/id561631498?l=en]! Du kan också stödja podden genom att ge oss en kaffe (eller två!) på Ko-fi [https://ko-fi.com/kodsnack], eller handla något i vår butik [https://shop.spreadshirt.se/kodsnack/]. Länkar * Viktor [https://www.marginalia.nu/] * Marginalia search [https://marginalia-search.com/] - Viktors sökmotor * Viktor har också pratat sökmotor i Kompilator 99 [https://kompilator.se/99] * Raspberry pi [https://en.wikipedia.org/wiki/Raspberry_Pi] * Vektorsökning [https://en.wikipedia.org/wiki/Vector_database] * Marginalia search på Github [https://github.com/MarginaliaSearch/MarginaliaSearch] * NLnet [https://en.wikipedia.org/wiki/NLnet] * Gemini [https://en.wikipedia.org/wiki/Gemini_%28protocol%29], diskuterat i Kodsnack 565 [https://kodsnack.se/565/] * Smarta mål [https://en.wikipedia.org/wiki/SMART_criteria] * Conways lag [https://en.wikipedia.org/wiki/Conway%27s_law] * Stöd oss på Ko-fi! [https://ko-fi.com/kodsnack] * Page och Brins paper [http://infolab.stanford.edu/~backrub/google.html] som presenterar Google. Appendix A diskuterar problem med att blanda annonsering och sökmotorer * John Carmacks .plan-filer med att göra-listor [https://raw.githubusercontent.com/ESWAT/john-carmack-plan-archive/refs/heads/master/by_year/johnc_plan_1996.txt] * Snurran som hämtar innehållet på sidor [https://www.marginalia.nu/log/a_121_profiling_websites/] - efter att JS och popovers och annat har kört * Ladybird [https://ladybird.org/] * Search engines information retrieval in practice [https://ciir.cs.umass.edu/irbook/] - gratis e-bok Viktor tipsar om Titlar * Jag har byggt en internetsökmotor * I min hybris * En miljard dokument * Sökmotorförkunskaper * Åttiotalsteknik * En server av PC-hårdvara * Motsatsen till React- och Nodeutveckling * Boomer till hjärtat * Jag som står för förändringarna * Lösa problem som alla har * Klart att det inte går * Åka till månen innan smarta mål * Ett sätt att jobba som funkar om man jobbar själv * 100% bild av hela kodbasen * En väldigt stor verktygslåda * Handskruvat * Mina Pokemon * Vad är tillräckligt mycket?

Fredrik talks to Grace Jansen [https://www.linkedin.com/in/grace-jansen/?originalSubdomain=uk] about cloud tools, and bringing them to your local machine in a better way. Opentelemetry is a great tool, but it’s not the whole story for observability. Gathering the data is just the first step. In the second half, we leave telemetry and talk about realizing you have things to share and sharing them with other people. Find out what makes you tick, and share experiences around that. Grace also shares some concrete presentation-building tips at the end. Ask the question, and be more you! Recorded during Øredev 2024 [https://archive.oredev.org/2024/#/]. Thank you Cloudnet [http://www.cloudnet.se/] for sponsoring our VPS [http://en.wikipedia.org/wiki/Virtual_private_server]! Comments, questions or tips? We a re @kodsnack [https://www.twitter.com/kodsnack], @tobiashieta [https://www.twitter.com/tobiashieta], @oferlund [https://twitter.com/oferlund]and @bjoreman [https://www.twitter.com/bjoreman] on Twitter, have a page on Facebook [https://www.facebook.com/kodsnack] and can be emailed at info@kodsnack.se [info@kodsnack.se] if you want to write longer. We read everything we receive. If you enjoy Kodsnack we would love a review in iTunes [http://itunes.apple.com/se/podcast/kodsnack/id561631498?l=en]! You can also support the podcast by buying us a coffee (or two!) through Ko-fi [https://ko-fi.com/kodsnack]. Links * Grace [https://www.linkedin.com/in/grace-jansen/?originalSubdomain=uk] * Øredev 2024 [https://archive.oredev.org/2024/#/] * Grace’s Øredev 2024 presentations: Cloud-native dev tools: bringing the cloud back to earth [https://www.youtube.com/watch?v=IBrQYbsT2XI&list=PLOUKmSqExtAFpg3krEd6CXr3uIyUgP97b&index=67], and Becoming a cloud-native doctor [https://www.youtube.com/watch?v=1x8BwPESV3E&list=PLOUKmSqExtAFpg3krEd6CXr3uIyUgP97b&index=118] * Opentelemetry [https://opentelemetry.io/] * Distributed tracing [https://aws.amazon.com/what-is/distributed-tracing/] * Microprofile [https://microprofile.io/] - open source specification for distributed tracing * Jakarta [https://jakarta.ee/] - the artist previously known as Java EE * Reactive messaging [https://microprofile.io/specifications/reactive-messaging/] * Openapi [https://www.openapis.org/] * Telemetry [https://en.wikipedia.org/wiki/Telemetry#Software] * Openliberty [https://openliberty.io/] * Quarkus [https://quarkus.io/] * Payara [https://www.payara.fish/products/cloud-native-java-deployment-platform/] * Jboss [https://en.wikipedia.org/wiki/JBoss_%28company%29] * Prometheus [https://en.wikipedia.org/wiki/Prometheus_%28software%29] * Grafana [https://en.wikipedia.org/wiki/Grafana] * Kibana [https://en.wikipedia.org/wiki/Kibana] * Fluid [https://fluid-cloudnative.github.io/docs] * Jaeger [https://www.jaegertracing.io/] - tracing platform * Torill Kornfeldt [https://kodsnack.se/136/] talked about resurrecting mammoths [https://vimeo.com/144804778] at Øredev 2015 * Sven Jungmann - can we teach machines to smell [https://www.youtube.com/watch?v=5MvOgmAvhlU&list=PLOUKmSqExtAFpg3krEd6CXr3uIyUgP97b&index=94]? * Support us on Ko-fi! [https://ko-fi.com/kodsnack] * Ants and AI models [https://swish.ai/what-do-ants-and-ai-have-in-common/] * Holly Cummins [https://hollycummins.com/] * Less waste, more joy, and a lot more green: How Quarkus makes Java better [https://www.youtube.com/watch?v=mx4p2zytjvc&list=PLOUKmSqExtAFpg3krEd6CXr3uIyUgP97b&index=106] - Holly’s Øredev 2024 presentation Titles * After-lunch lull * So polyglot * Ready for microservices * (You need) Many minds * Now I have a pile * (Take) The best of nature * The path was being them * Something I bring to the table * Ask the question * A unique presentation

Fredrik snackar med Patrik Svensson [https://patriksvensson.se/about] om Opencli [https://opencli.org/] - Patriks nyskapade förslag till en standard för att beskriva kommandoradsapplikationers gränssnitt. Det borde inte vara en stor grej att publicera ett förslag till en spec för någonting. Faktum är att mycket fler borde göra det! Fler borde få hybris. Ett stort tack till Cloudnet [https://www.cloudnet.se/] som sponsrar vår VPS [https://en.wikipedia.org/wiki/Virtual_private_server]! Har du kommentarer, frågor eller tips? Vi är @kodsnack [https://social.podsnack.se/@kodsnack], @thieta [https://6510.nu/@thieta], @krig [https://6510.nu/@krig], och @bjoreman [https://toot.cafe/@bjoreman] på Mastodon, har en sida på Facebook [https://www.facebook.com/] och epostas på info@kodsnack.se [info@kodsnack.se] om du vill skriva längre. Vi läser allt som skickas. Gillar du Kodsnack får du hemskt gärna recensera oss i iTunes [https://itunes.apple.com/se/podcast/kodsnack/id561631498?l=en]! Du kan också stödja podden genom att ge oss en kaffe (eller två!) på Ko-fi [https://ko-fi.com/kodsnack], eller handla något i vår butik [https://shop.spreadshirt.se/kodsnack/]. Länkar * Patrik [https://patriksvensson.se/about] * Patriks röst hörs i podden Modermodemet [https://modermodemet.se/] * Spectre.console [https://spectreconsole.net/] * Cake [https://cakebuild.net/] * Opencli [https://opencli.org/] * Spectre.console.cli [https://spectreconsole.net/cli/] * Kathleen Dollard [https://github.com/KathleenDollard] * Openapi [https://en.wikipedia.org/wiki/OpenAPI_Specification] * Chet Husk [https://devblogs.microsoft.com/dotnet/author/chethusk/] - PM för .net-CLI på Microsoft * Typespec [https://typespec.io/] * getopts [https://en.wikipedia.org/wiki/Getopts] * Opencli på Github [https://github.com/spectreconsole/open-cli] * Man pages [https://en.wikipedia.org/wiki/Man_page] * Stöd oss på Ko-fi! [https://ko-fi.com/kodsnack] * Aritet [https://en.wikipedia.org/wiki/Arity] * Podcasting 2.0-specen [https://github.com/Podcastindex-org/podcast-namespace/discussions] * ID3-standarden [https://id3.org/id3v2.3.0] - för att lägga metadata i mp3-filer * Podcast chapters [https://chaptersapp.com/] * Daniel Stenberg [https://daniel.haxx.se/] och Curls kommandoradsflaggor [https://ec.haxx.se/cmdline/options/index.html] * Mitchell Hashimoto [https://github.com/mitchellh] - grundare av Hashicorp [https://en.wikipedia.org/wiki/HashiCorp] * Ghostty [https://ghostty.org/] Titlar * Om det fanns en spec * Inte för mänsklig konsumtion * Inte rädd för att göra bort mig * Handknacka en spec * Halvbakade tankar * Klia CLI * Det finns ju manpages * Inga problem att göra bort mig * En enorm, komplex best * Tiotusen företag i ett företag * Dialekter av CLI-applikationer * -build * Hela Jira-spektrat

Fredrik snackar med Lars Wikman [https://underjord.io/lars.html] om Lars senaste projekt: att arrangera konferenserna Goatmire Elixir [https://goatmire.com/] och Nervesconf EU [https://goatmire.com/]. Vad handlar konferenserna om? Hur kommer det sig att Lars arrangerar dem? Hur får man tag på sponsorer, talare, och allt annat som behövs? Och varför är det bra i sammanhanget att vara programmerare, och lite puckad? Vill du gå på konferenserna? (Klart du vill!) Använd koden getasnack [https://ti.to/goatmire/elixir/discount/getasnack] så får du rabatt! Och ja, vi förklarar också vad sjutton Goatmire egentligen betyder. Ett stort tack till Cloudnet [https://www.cloudnet.se/] som sponsrar vår VPS [https://en.wikipedia.org/wiki/Virtual_private_server]! Har du kommentarer, frågor eller tips? Vi är @kodsnack [https://social.podsnack.se/@kodsnack], @thieta [https://6510.nu/@thieta], @krig [https://6510.nu/@krig], och @bjoreman [https://toot.cafe/@bjoreman] på Mastodon, har en sida på Facebook [https://www.facebook.com/] och epostas på info@kodsnack.se [info@kodsnack.se] om du vill skriva längre. Vi läser allt som skickas. Gillar du Kodsnack får du hemskt gärna recensera oss i iTunes [https://itunes.apple.com/se/podcast/kodsnack/id561631498?l=en]! Du kan också stödja podden genom att ge oss en kaffe (eller två!) på Ko-fi [https://ko-fi.com/kodsnack], eller handla något i vår butik [https://shop.spreadshirt.se/kodsnack/]. Länkar * Lars [https://underjord.io/lars.html] * Tidigare avsnitt med Lars [https://kodsnack.se/people/lars-wikman/] * Goatmire Elixir [https://goatmire.com/] * Nervesconf EU [https://goatmire.com/] * Varberg [https://sv.wikipedia.org/wiki/Varberg] * Nerves [https://nerves-project.org/] * Elixir [https://elixir-lang.org/] * Nervesconf USA [https://nervesconf.us/] * Gig city Elixir [https://www.gigcityelixir.com/] * Lars nyhetsbrev [https://underjord.io/newsletter.html] * Kramfors [https://sv.wikipedia.org/wiki/Kramfors] * The art of gathering [https://www.priyaparker.com/book-art-of-gathering] - av Priya Parker * Myconf [https://myconf.io/] * Varbergs teater [https://varberg.se/kulturhuset-komedianten/varbergs-teater] * Videon från teatern [https://www.youtube.com/watch?v=54dZLnJFC5A] på Underjords Youtubekanal [https://www.youtube.com/c/underjord] * Varbergs sparbank [https://www.varbergssparbank.se/] * Lars har bloggat om arrangerandet [https://underjord.io/making-of-an-elixir-conference.html] * Helene Mattisson [https://www.linkedin.com/in/helenemattisson/] * Sessionize [https://sessionize.com/] * Papercall [https://www.papercall.io/] * Oban [https://oban.pro/] * Kivra [https://sv.wikipedia.org/wiki/Kivra_%28f%C3%B6retag%29] * Erlang solutions [https://www.erlang-solutions.com/] * Glesys [https://glesys.se/] * Ampere [https://amperecomputing.com/] - bygger ARM-servrar * Ampereone [https://amperecomputing.com/briefs/ampereone-family-product-brief] * Forza football [https://forzafootball.com/] * Entryfy [https://entryfy.com/en/] * Nerves på Open collective [https://opencollective.com/nerves-project] * Nervescloud [https://nervescloud.com/] * Hotell Gästis [https://www.hotellgastis.se/hotell-gastis-varberg/] * Leninbadet [https://www.hotellgastis.se/bad/] * Oswag [https://oswag.org/] * Tvååker [https://sv.wikipedia.org/wiki/Tv%C3%A5%C3%A5ker] * XLNT reklam [https://www.xlntreklamvarberg.se/kontakt/] - trycker tröjor, bland annat * Design is my hobby [https://uxplanet.org/the-history-and-meaning-of-graphic-design-is-my-passion-meme-5226bdf4b1] * Inkscape [https://en.wikipedia.org/wiki/Inkscape] * Open graph [https://ogp.me/] * Saša Jurić [https://www.theerlangelist.com/] * Elixir in action [https://www.manning.com/books/elixir-in-action] * The soul of Erlang & Elixir [https://www.youtube.com/watch?v=JvBT4XBdoUE] * Codebeam [https://codebeameurope.com/] * Elixirconf [https://elixirconf.com/] * Andrea Leopardi [https://goatmire.com/speaker/andrea-leopardi] * Elixir Umbrella [https://elixirschool.com/en/lessons/advanced/umbrella_projects] * Zach Daniel [https://www.zachdaniel.dev/] * Ash [https://www.ash-hq.org/] * Stöd oss på Ko-fi! [https://ko-fi.com/kodsnack] * Louise Blanc [https://www.goatmire.com/talk/from-object-oriented-to-functional-thinking-my-elixir-journey] - pratar om att se Elixir med nya ögon * Bruce Tate [https://grox.io/about] * Marc Lainez och hans bilbyggargäng [https://goatmire.com/talk/a-nerves-car] * HCA och Waterparkprojektet [https://www.youtube.com/watch?v=cVQUPvmmaxQ] * Varbergs kallbadhus [https://sv.wikipedia.org/wiki/Varbergs_kallbadhus] * Tito [https://ti.to/home] * Luma [https://lu.ma/] - Meetupkonkurrent * CSV [https://en.wikipedia.org/wiki/Comma-separated_values] * Alchemy conf [https://alchemyconf.com/] * FOSDEM [https://fosdem.org/2025/] * Goatmire international [https://global.goatmire.com/] * Buffer [https://buffer.com/] * Getakärr [https://sv.wikipedia.org/wiki/Getak%C3%A4rr] Titlar * Alltså inte Stockholm * Det är inte Stockholm * Designa ett bra event * Stället där jag gifte mig * Ingen fattar vad jag gör * Eventmänniska * Insyltade i Elixir * En linje där allt är spikat * När talarna känner sig omhändertagna * Vara programmerare, och lite puckad * Ett treakters monodrama * Ett överdåd av karaktär * Jag vill inte ha en changelog * Hennes praktikplats känner mig * BEAMen på BEAMigt sätt * Bunden till scenen * Min lilla Inkscapedans