Is it even a database anymore?.. I'll show you how PostgreSQL ALONE can replace ALL your backend stack components.
0:00 - is it even a database anymore?
1:15 - the quickest history of PostgreSQL
2:34 - object-relational database
4:38 - table inheritance
6:03 - pg_trunk: a package manager for PostgreSQL extensions
7:31 - stored procedure that uses python's pandas
7:55 - your database is not a DUMB store
8:38 - PostgreSQL's FEATURE OF FEATURES
9:04 - pg_cron & hstore: replacing redis cache
10:53 - NOTIFY/LISTEN: replacing redis pub/sub
11:51 - pgmq: replacing message queue like SQS
13:10 - JSONB: replacing MongoDB
13:29 - NoSQL vendors mimic SQL
14:26 - Apache AGE & TimescaleDB: replacing graph and time series databases
15:09 - FTS: replacing ElasticSearch
16:20 - pgml: LLM and chatbots running in PostgreSQL
16:39 - Postgrest: no-code REST API for your DB tables
18:06 - pgtap: unit-testing for PostgreSQL
18:42 - pg_graphql: no-code GraphQL API for your DB tables
19:06 - Omnigres: PostgreSQL as a platform
19:25 - "boring technology" and "radical simplicity" with PostgreSQL
20:03 - trying to play pg_doom (Doom in PostgreSQL)
Code & links from the episode: https://github.com/vivus-ignis/the-art-of-the-terminal/tree/master/postgresql-is-not-a-database
ever since completing my research for this
episode in fact I have only one question is it even a database anymore stay with me and I'll
show you how posr s can replace all your back and stack components after more than 35 years of
active development and a new release coming every year posg scale today is the most popular and
according to its website most advanced open-source relational database if we look into the latest
stch overlow developer survey it's also the most loved database among
professional developers in
the list of companies using posel in production there are many household names every major cloud
provider has a managed poser scel database service but unlike all the competitors in the pretty
crowded database Niche there is no single company governing development of POS SQL there is
no King like lenus stals Matts or gidan rossom to approve technical decisions the official road map
page even claims we enjoy allowing developers to explore the topics of their choosi
ng sounds like
a product owner's nightmare scrum yes scrum scrum so what posq really is and how come it is so
successful database project with an elephant masket was not always considered cool just
look at these stack Overflow developer service for the past several years back in late '90s and
2000s every web developer knew what lamp acronym stood for Linux Apache MySQL PHP MySQL an open-
Source database from Sweden became a go-to storage solution for booming internet businesses it was
free
it was easy to set up and work with and it was fast Facebook used MySQL Twitter used MySQL
YouTube used MySQL poq project at the time was focused on painstaking implementation of more
features better compliance with SQL standards query optimizations indexing improvements crash
recovery the list goes on it was clear that its contributors followed a very different philosophy
instead of speed and E of use the key selling points of mysq they wanted to build the most
feature Rich and standards
compliant database in the world so how feature reach is feature Rich
poql the official website calls pogl an object relational database what does it even mean the
idea is to bridge the gap between object oriented concepts with object hierarchies and complex data
structures on the one side and relational database design where you have only tables rows and columns
on the other so how does this Fusion of Two Worlds work in poql first of all you are not limited by
native data types although eve
n here you can feel what feature reach means in addition to Common
numeric Boolean and character types poql supports right out of the box several ways to store dat
time a bunch of network address data types that allow for quick submitting calculations and for
example geometry types like polygon or Circle to name just a few of supported native types
but because as I said above posq is an object relational data base you can as easily Define
your own types and use them as if they were built in
to the system let's say I work on roleplaying
game and I want to store game characters in the database let me Define the character type this is how I can create a table and use my
new custom data type there here's the game [Music] character this is
how I can query my character stable okay let's go back to objectoriented
Concepts you've seen complex data structures type casting and the dot notation to access object
properties another Cornerstone objectoriented concept that is implemented in
posg scale is
inheritance table inheritance for my roleplay game example let's say I want now to have a
table for NPCs NPCs are special characters that always have a quest to offer to a player
I trust you have a good reason for interrupting me which means I can inherit my new NPC table
from the already existing characters table time to populate it with a couple of [Music] NPCs pay
attention Fields character id and character are not defined in the NPC's stable they are inherited
from the pa
rent table here are my two NPCs in the NPCs table now the interesting part all three
game characters two NPC players and one player character are now stored in the P table let's
not forget that objectoriented programming is code and data bundled together when it comes to
running code in the database posar scel has to offer more than any of its competitors just
look at the table of supported programming languages for stored procedures before I can
go straight to the code I need to enable the
language Handler and for that I need a package
manager yes there is a package manager for poql [Music] extensions [Music] [Music] [Music] is [Music] this is the code of the start
procedure I've inserted some test data for it [Music] already let's [Music] see with
objectoriented features of porra ql and the power of stored procedures your database is
not at dump storage anymore and you can actually set your backend application free from data logic
as this gentleman shows in his blog post th
ings like user management which is data logic can be
fully self-contained in a database with help of Po scale extensions you can hash passwords generate
Json web tokens QR codes anonymize personal data render HTML from templates and much much more
there are three different actively developed HTTP clients that work entirely within database context
the more I learn about poql the more I come to a conclusion that it has extension points everywhere
this extensibility is the feature of features
that makes po so Universal that it can in fact
replace your whole back end so now let me show you how one database system one software product
can replace a whole technological zoo of typical backend project shall we start with sketching it's
fairly easy to build a key value store on top of pql I would like to have expiring values like in
redis so let me install a PG Chrome extension for that [Music] [Music] is now to the [Music] [Music] database [Music]
0:10:29.200,1193:02:47.295
[Music] an
other popular red feature pops up can be
replaced by built-in notify listen commands [Music] [Music] as you may expect from a relational
database notifications can be wrapped in transactions and of course you
can create a trigger which will send a notification whenever say a new
game character is added to the character table since we started with messages how
about replacing your message que with posq PGM Q operates similarly to AWS managed
message Q in service sqs it ensures exactly once
delivery by temporarily
hiding recently accessed [Music] messages Engineers from prequel a company that facilitates
cross database synchronization have replaced a popular message broker software rabbit mq
that served their back and job que with just a POS skel table row level read WR
locks and ured that each job was picked by a single worker the goal was to make
their infrastructure simpler and easier to reason about we'll come back to this
idea later in this episode next candidate for rep
lacement in my list is mongodb to
begin with poel supports Json data type [Music] natively the query syntax may look
different from JavaScript TK [ __ ] syntax but hey all the cool nosql databases out there
after being around for at least some time are all adapted in some kind of an SQL like language
look at for example influx DB couch base so moving to the real standards compliant SQL is a good
thing there is something about this technology that makes it so resilient and future proof that
even no SQL vendors understand that they should at least mimic the structured query language
if you decide to move your Json documents to poql like for example Guardian did you will need
more disk space because there will be no document Keys compression like in [ __ ] but then again
you will get out of the box integration with everything else Sports BQ has to offer and of
course less infrastructure to maintain what about other types of nosql data databases you can turn
your post SQL into a
graph database with Apache AG project and use both SQL and a graph language
open Cipher to model store and analyze connected data for example you can map relationships in
social networks or track connections within power grids things like that for time series data
there is time scale DB an extension designed to efficiently handle timestamp data so that you can
say monitor iot device metrics or maybe analyze user activity logs for web services time scale
DB Works nicely with grafana by the
way if you have a bunch of text Data though often times you
need to search through it not with like statement in SQL which is pattern matching but in a way
a search engine does it with stop words where stemming and ranking of search results that is the
search should find results with similar words and sort them accordingly in a typical backend setup
this functionality is covered by elastic search elastic search would continuously get the data
to index from somewhere else maybe from a poql d
atabase and provide a standalone search service
but if you choose to integrate posg sql's full text search capabilities you can achieve similar
functionality directly within your database given that you wouldn't have to synchronize data
for search you will get an always upto-date index and as with all the examples above your
infrastructure will be simpler and therefore easier to maintain of course you should mind
the constraints but for many maybe even most projects out there running an ela
stic search
cluster for full text search is an Overkill when it comes to text of course you cannot get
around llms large language models imagine you can query an open source llm model imagine a
chatbot running entirely on the database side here you go you need a machine with a GPU and a
pgml extension okay so you ditch a message queue and a whole bunch of nosql databases replace them
with SP SQL you can now search through your data in near real time and run machine learning models
right wi
thin the database now what's left of your backend stack the API itself post gr is the web
server that turns your database into a rest API in a typical web framework to add a new entity you
would have to prepare database migrations Define a data model in some kind of an orm and then
configure API endpoints in many cases the code will just wrap around simple crud operations to
create update delete a record in a database with pogress instead that you just need to create
a database table that's
it pogress will take care of authentication and authorization using
post gq's own Ro based access controls with ro level granularity to go beyond simple crud as
you may have guessed it you got to use stored procedures within stored procedures you can
of course leverage object relational nature of fgra SC and all the extensions I've mentioned
before pops up messaging message cues key value object graph and time series data stores
as well as full text search and machine learning functionalit
y and yes you can unit test
St procedures just like you would do with your web [Music] framework [Music] If instead of rest you'd rather prefer graphql
you covered as well PG graphql extension creates a transparent mapping between your data in
poq tables and graphql entities so you can run graphql queries using a single fun function
call if you couple PG graphql with postr you can run graphql queries over HTTP by now you probably
have no doubt that posg is in fact a platform omn project too
k this idea to extremes going as
far as managing containers storing files in S3 and even providing payment support all within
poql regardless if you want to go all in with omn or just make your backend infrastructure a bit
simpler to maintain and reason about pogress qu is on your side it's not only as feature Rich as
you can imagine but also Enterprise grade reliable think about it maybe you don't need another shiny
document database or a search cluster instead adapting boring technology a
nd embracing radical
Simplicity might yield long-term benefits for your project besides that improving your ES skills
seems to be a very future proof investment and while you're getting used to the idea that posq
is not a database I'll play Doom right in my SQL client
Comments
In 2016, I worked on a project where almost all the business logic was in a database (Hi Oracle with PL/SQL) and it was a nightmare. There is only one upside to this approach: if you move all business logic (especially if you move cache, elastic search, etc. like in the video) into a database, you are probably safe from having to let go forever because no one would be able to understand how things work. And the phrase "Hey, join us, we have everything in the database" doesn't seem so attractive to developers. I believe it may be attractive for DBAs but moving all the stuff into the database you limit yourself in hiring.
Good video! But you have a green screen ^^ Use it, move yourself to the side, point at things. Might be hard in the beginning, but if you already go the length of adding a black background. Might as well have fun. :D
I started using postgres because it had views and mysql didn't
Thank you 🙏- I like the perspective to leave out network within the backend services 🤘
Where was the "Scrum..., Scrum....Scrum" clip from?
yes, but, what are the cons? too much CPU usage? too much Disk usage? hard to maintain and debug? or the cons are that now one person can do the job of an entire team?
This is one of the best PostgreSQL videos that I have ever seen on the web.
I liked the video, hated the features on the database. This feels wrong, it's like killing a mother with a baseball bat.
During my apprenticeship I was dealing with Oracle and DB/2 servers. That taught me the advantages of procedual languages like PL/SQL. But those servers weren't exactly affordable to the common hobbyist at that time. MySQL was fast but didn't have any sort of PL at that time. So I ended up starting to use PostgreSQL for my own projects, to get the features I had gotten used to and more. It wasn't the fastest but it was more grown up than other free SQLs.
Fantastic video, very interesting and insightful. Keep up the good work!
just a matter of time before someone writes the linux kernel in SQL
The dude went completely crazy at the end 😆😆😆😆😆 Nicely done video, good job and thank you! You deserve a million subscribers 😎
I really enjoyed this video. Gained a subscriber.
Great video. Thank you for your work.
Wow. Nothing more needs sayin'.
Doing data operations as close to the data sounds awful. To ensure reliability it needs to be done 5 microservices away AT LEAST!
I've heard of this piece of tech my entire coding career but never taken a moment to look at it or had anybody try to convince me it was the way to go or what I use it for... Getting all the accessibility you describe I not surprised there's a cult following... Quite cult, but cult nonetheless. I appreciate your video. I am very much vision impaired through half of my YouTube viewing (The eyes ain't what they used to be) and the only thing I could take out of this video was that it could do things all of the concrete demonstration/typing ASMR left me completely in the dark... literally. That's my only complaint about this video and I realize I'm probably a minor edge case, so if the ethereal music and the tapping of the keyboard is what people love about your stuff don't even consider this feedback... This is my first video encounter with you btw, I didn't know if your. Explaining what you're doing audibly while you're typing into the terminal would make the non-visual component of your video more complete for me in these cases I didn't have my vision enhancing tools available. Even if you just describe what you're doing. I thought I could see some text and some of these moments. It was difficult to understand what was happening from audio alone in those moments. That's just a minor critique. The video is really good to at least put it on my radar but it's capable of. Thanks for that!
WOW. But should we ?
Lol isn't this building up a single point of failure? I did enjoy this video, very informative.
Run Postgres on Postgres - full circle