- Conference - lists.ivorysql.org

New submission: Spat: Hijacking Shared Memory for a Redis-Like Experience in PostgreSQL

by HOW2025

Florents Tselai has submitted a new session! Spat: Hijacking Shared Memory for a Redis-Like Experience in PostgreSQL Databases traditionally use shared memory as a transient workspace for query execution, caching, and transaction management, hidden from direct user control. But what if we could repurpose it for user-facing, high-performance data structures and general caching? In this session, I’ll explore PostgreSQL’s shared memory APIs available to extension developers, including recent features like the DSM Registry. I’ll introduce Spat, a Redis-like in-memory data structure server embedded within PostgreSQL, where data is stored entirely in shared memory. Spat provides an extended key-value model in which keys are strings, and values can be strings, lists, sets, or hashes. This offers a fast, lightweight alternative for ephemeral storage inside PostgreSQL. This talk will explore the technical challenges and opportunities of using shared memory beyond its traditional scope, providing ideas for anyone interested in extending PostgreSQL. Open session [https://sessionize.com/app/organizer/session/19661/937793] These notifications can be turned off on Edit event [https://sessionize.com/app/organizer/event/edit/19661#_tab-advanced] page. Tip: You've used Classic (confer***(a)***orysql.org) to login. You're receiving this message because you had joined event 'HOW2025' on Sessionize. Sessionize.com, PO Box 6945, Unit 34564, London W1A 6US, UK

May 22, 2025

New submission: Doing More with JSONPath: Turning a Query Language into a Transformation Engine

by HOW2025

Florents Tselai has submitted a new session! Doing More with JSONPath: Turning a Query Language into a Transformation Engine PostgreSQL’s JSONPath engine is often seen as just a filtering tool — but its real potential lies much deeper. In this talk, we’ll explore how the JSONPath infrastructure can serve as a foundation for building powerful, domain-specific languages (DSLs) within Postgres. Drawing on my recent work to extend JSONPath with richer text processing functions like lower(), replace(), we’ll demonstrate how these additions evolve JSONPath from a query language into a lightweight transformation engine. You’ll learn:• How JSONPath is implemented in PostgreSQL and why it’s extensible• The design rationale behind adding composable text functions to the language• How these functions open the door to expressive, pipeline-style data transformations• Why JSONPath’s AST and evaluation model make it a great blueprint for custom DSLs If you’re interested in Postgres internals, building smarter queries, or embedding logic closer to the data — this is your invitation to rethink what JSONPath can do. Open session [https://sessionize.com/app/organizer/session/19661/937792] These notifications can be turned off on Edit event [https://sessionize.com/app/organizer/event/edit/19661#_tab-advanced] page. Tip: You've used Classic (confer***(a)***orysql.org) to login. You're receiving this message because you had joined event 'HOW2025' on Sessionize. Sessionize.com, PO Box 6945, Unit 34564, London W1A 6US, UK

May 22, 2025

New submission: A Minimalist’s Guide to Doing AI with Postgres

by HOW2025

Florents Tselai has submitted a new session! A Minimalist’s Guide to Doing AI with Postgres You don’t need a cluster of GPUs, a Python microservice zoo, or heavyweight orchestration to start doing AI inside your database. In this talk, we’ll explore a set of minimalist, practical recipes for integrating LLMs and AI tools directly into PostgreSQL — using nothing more than SQL, curl, a couple of minimal extensions, some Makefile glue, and a bit of creativity. The goal: AI workflows that are simple, inspectable, reproducible — and entirely inside your database. If you’re into small, elegant solutions instead of complex pipelines, this talk is for you. Open session [https://sessionize.com/app/organizer/session/19661/937788] These notifications can be turned off on Edit event [https://sessionize.com/app/organizer/event/edit/19661#_tab-advanced] page. Tip: You've used Classic (confer***(a)***orysql.org) to login. You're receiving this message because you had joined event 'HOW2025' on Sessionize. Sessionize.com, PO Box 6945, Unit 34564, London W1A 6US, UK

May 22, 2025

New submission: Data Engineering Stratagems with Postgres: From Ancient Strategic Thought to Postgres Extensions

by HOW2025

Florents Tselai has submitted a new session! Data Engineering Stratagems with Postgres: From Ancient Strategic Thought to Postgres Extensions This talk presents a collection of practical stratagems for data engineers working with PostgreSQL—rooted in ancient strategic thought but adapted for a world of Unix pipes, streaming data, and programmable databases. Each stratagem is a distilled principle drawn from years of building scalable and elegant data systems, and reflects a mindset that treats Postgres not just as a database, but as an operating system for data. We’ll begin with philosophical foundations—ideas from Sun Tzu, the Thirty-Six Stratagems, and military strategy—and show how these timeless patterns can inform modern data design. Then, we’ll walk through how to apply them across the stack: composing data flows with Unix pipelines, building resilient systems with Postgres functions and triggers, and extending Postgres itself with custom types, operators, and C extensions.Expect a mix of thought frameworks, real code, and hard-won lessons. This talk is for engineers who want to design like strategists, execute like hackers, and make Postgres a source of leverage, not just storage Open session [https://sessionize.com/app/organizer/session/19661/935588] These notifications can be turned off on Edit event [https://sessionize.com/app/organizer/event/edit/19661#_tab-advanced] page. Tip: You've used Classic (confer***(a)***orysql.org) to login. You're receiving this message because you had joined event 'HOW2025' on Sessionize. Sessionize.com, PO Box 6945, Unit 34564, London W1A 6US, UK

May 19, 2025

New submission: Who Needs a Warehouse When You've Got a Lakehouse?

by HOW2025

Nilanjan Chatterjee has submitted a new session! Who Needs a Warehouse When You've Got a Lakehouse? The data scene has seen a big change in recent years, moving from old-fashioned data warehouses to more easy and strong lakehouse architectures. This switch means not just a technical advancement but a basic rethinking of how groups keep, handle, and get value from their data assets.The traditional data warehouse worked well with structured data because there was a defined schema, consistency, and strong BI performance. The high costs and inflexibility of such warehouses meant that they were not suitable for handling unstructured data- much less a large volume of such unstructured data.This is what gave birth to the data lake: an inexpensive place to store enormous volumes of highly variegated datasets-with schema-on-read flexibility. It then often turned into the "data swamp" due to quality issues, analytical performance shortcomings, and governance issues.Then came the lakehouse architecture, a brilliant amalgamation that took unto it the strengths of both warehouse and lake. Open session [https://sessionize.com/app/organizer/session/19661/932740] These notifications can be turned off on Edit event [https://sessionize.com/app/organizer/event/edit/19661#_tab-advanced] page. Tip: You've used Classic (confer***(a)***orysql.org) to login. You're receiving this message because you had joined event 'HOW2025' on Sessionize. Sessionize.com, PO Box 6945, Unit 34564, London W1A 6US, UK

May 15, 2025

New submission: SQL Indexes - Boon or Bane?

by HOW2025

Nilanjan Chatterjee has submitted a new session! SQL Indexes - Boon or Bane? SQL indexes are a powerful tool for optimizing database performance, but their effectiveness depends on platform-specific strengths and trade-offs. In Azure SQL, automated features like index tuning and columnstore indexes streamline analytical workloads, while managed maintenance reduces fragmentation risks. For PostgreSQL, flexibility shines with specialized indexes (e.g., GIN for JSONB, BRIN for time-series) and partial/expression-based indexing, enabling tailored optimizations. Both platforms enforce data integrity via unique indexes, and read-heavy systems benefit significantly. However, Azure’s automation can lead to unintended index drops, and columnstore indexes require partitioning discipline. PostgreSQL demands manual upkeep (e.g., VACUUM for bloat) and risks suboptimal plans without proper composite index design. The downsides center on write overhead and cost. Azure SQL’s indexing increases DTU consumption and storage costs, especially in geo-replicated setups. PostgreSQL’s MVCC model causes index bloat, impacting distributed systems like Citus. Over-indexing in either system inflates storage: Azure’s tiered pricing penalizes excess, while PostgreSQL’s self-managed flexibility still demands cost-awareness. Ultimately, indexes are a boon when aligned with platform capabilities (e.g., Azure’s analytics focus, PostgreSQL’s data-type diversity) but a bane if applied generically without workload analysis and maintenance planning. Open session [https://sessionize.com/app/organizer/session/19661/932739] These notifications can be turned off on Edit event [https://sessionize.com/app/organizer/event/edit/19661#_tab-advanced] page. Tip: You've used Classic (confer***(a)***orysql.org) to login. You're receiving this message because you had joined event 'HOW2025' on Sessionize. Sessionize.com, PO Box 6945, Unit 34564, London W1A 6US, UK

May 15, 2025

New submission: An Introduction to Memory Contexts

by HOW2025

Chris Travers has submitted a new session! An Introduction to Memory Contexts When writing extensions in C or related languages, how PostgreSQL manages memory is important. This talk will go over how PostgreSQL manages memory, how we select memory management lifecycles, and how we can verify that our extensions are using this system safely. The talk includes key discussions of best practices. Open session [https://sessionize.com/app/organizer/session/19661/932615] These notifications can be turned off on Edit event [https://sessionize.com/app/organizer/event/edit/19661#_tab-advanced] page. Tip: You've used Classic (confer***(a)***orysql.org) to login. You're receiving this message because you had joined event 'HOW2025' on Sessionize. Sessionize.com, PO Box 6945, Unit 34564, London W1A 6US, UK

May 15, 2025

New submission: The Future of PostgreSQL in a changing world

by HOW2025

Chris Travers has submitted a new session! The Future of PostgreSQL in a changing world PostgreSQL, like open source software in general arose in the mid-1990's. This time corresponded during a time when there was a massive effort at building a grand interconnected, highly globalized world. Now, geopolitics is shifting and efforts are made at de-risking and de-globalization. The current trends of de-risking and de-globalization pose both risks and opportunities for projects like PostgreSQL. This talk explores the threats and opportunities for PostgreSQL and other open source projects in this changing world. Open session [https://sessionize.com/app/organizer/session/19661/932614] These notifications can be turned off on Edit event [https://sessionize.com/app/organizer/event/edit/19661#_tab-advanced] page. Tip: You've used Classic (confer***(a)***orysql.org) to login. You're receiving this message because you had joined event 'HOW2025' on Sessionize. Sessionize.com, PO Box 6945, Unit 34564, London W1A 6US, UK

May 15, 2025

New submission: Building Petabyte-Scale Postgres Clusters

by HOW2025

Chris Travers has submitted a new session! Building Petabyte-Scale Postgres Clusters At one of the previous places I have worked, we built a semi-structured data store that stored 10PB of log data for the support team's use. This was done with PostgreSQL and involved less hardware than ElasticSearch at a cost of some functionality. Today I am one of the contributors to an open source successor to that project. This talk will go through the design of the system on a high level, how it works, and how to architect large-scale distributed data platforms using PostgreSQL or other storage technologies. Open session [https://sessionize.com/app/organizer/session/19661/932611] These notifications can be turned off on Edit event [https://sessionize.com/app/organizer/event/edit/19661#_tab-advanced] page. Tip: You've used Classic (confer***(a)***orysql.org) to login. You're receiving this message because you had joined event 'HOW2025' on Sessionize. Sessionize.com, PO Box 6945, Unit 34564, London W1A 6US, UK

May 15, 2025

New submission: PostgreSQL 18 and Asynchronous I/O: An Architectural Leap Toward Modern Data Workloads

by HOW2025

Wagner Bianchi has submitted a new session! PostgreSQL 18 and Asynchronous I/O: An Architectural Leap Toward Modern Data Workloads PostgreSQL 18 introduces the first foundational integration of asynchronous I/O (async I/O) into the ecosystem, marking a historic shift in how PostgreSQL interacts with modern Linux storage subsystems. This paper explains the architectural changes behind adopting io_uring, the modern kernel asynchronous I/O framework introduced in Linux 5.1. We will explore how PostgreSQL now leverages memory-mapped shared ring buffers (Submission Queue and Completion Queue) to minimize syscall overhead, reduce latency, and unlock higher parallelism during disk operations. The paper will also cover how PostgreSQL’s shared buffer architecture complements asynchronous I/O, and will compare PostgreSQL’s conservative gradual adoption strategy to other RDBMS engines such as MySQL/InnoDB and Oracle Database. Practical examples, early benchmarking scenarios, and operational tuning recommendations for PostgreSQL 18 on XFS with io_uring will be presented. This hands-on, system-oriented guide will help engineers, DBAs, and cloud architects understand what to expect from asynchronous I/O in PostgreSQL and how to prepare for its maturation in PostgreSQL 19 and beyond. Open session [https://sessionize.com/app/organizer/session/19661/931230] These notifications can be turned off on Edit event [https://sessionize.com/app/organizer/event/edit/19661#_tab-advanced] page. Tip: You've used Classic (confer***(a)***orysql.org) to login. You're receiving this message because you had joined event 'HOW2025' on Sessionize. Sessionize.com, PO Box 6945, Unit 34564, London W1A 6US, UK

May 13, 2025