PostgreSQL is an open-source database management system. It is a great way to learn to program, design and debug.
One of the main features of PostgreSQL is Extensions.
Extensions of PostgreSQL can be an as small extension as pg_fincore, used to understand the OS cache of your PostgreSQL pages to the large projects like PostGIS and Citus data.
The PostgreSQL extension is a piece of script or code which enhances the functionality of your PostgreSQL server.
In this post, we are going to understand…
Everything about PostgreSQL Extensions.
This post mainly focuses on behind the scenes of PostgreSQL extensions, we start with creating our own PostgreSQL extension and work on its internal aspects.
Before we begin, I highly recommend you to know the basics of
- How to create and compile a C Program on Linux
- What is shared library
- How to create and run a shared Library file
- How to create a simple function in PostgreSQL
here.
Table of Contents
What is a PostgreSQL Extension?
PostgreSQL Extension is a package of one or more SQL objects (such as functions, procedures, tables, and indexes).
The advantage of packaging these related objects into a single extension, instead of using one or more scattered scripts, is that PostgreSQL will manage the whole set of objects as a single package.
PostgreSQL has got few default extensions out of the box and a user can create custom extensions based on the needs.
How to create a PostgreSQL Extension
A PostgreSQL extension can be created in PostgreSQL by the command
create extension extension_name;
But to execute the above command in your database properly, you must have at least two important files under the share directory called /isntall_directory/share/extension path.
They are
- Control file.
- SQL script file.
Control file: The file format must be extension_name.control, which tells the basics about an extension to PostgreSQL
SQL script file: The file in the format extension–version.sql contains the functions that you would like to add.
If the extension is to be updated with the modified SQL commands, the version of the new file should be extension–old_version–new_version.sql
When an extension is installed, all the packaged objects are placed into a database schema. If the extension can be moved to another schema, the extension is said to be relocatable.
Let us create our own PostgreSQL extension for add_ten function.
Step 1. Create a SQL script file (add_ten–1.0.sql)
1 2 3 4 5 6 7 8 9 10 |
[root@prim shared]# cat add_ten--1.0.sql CREATE OR REPLACE FUNCTION add_ten(sum INTEGER) RETURNS INTEGER AS $$ DECLARE result INTEGER; BEGIN result=sum+10; RETURN result; END; $$ LANGUAGE plpgsql; |
add_ten—1.0.sql is just a normal function that we run in PostgreSQL.
Step 2. Create a control file (add_ten.control)
1 2 3 4 5 6 |
[root@prim shared]# cat add_ten.control # sum_fn extension comment = 'Add ten to given number' default_version = '1.0' module_pathname = '$libdir/add_ten' relocatable = false |
Here
comment: A descriptive comment about the extension, applied only at installation time.
default_version: The version to install as default when no specific version is required.
module_pathname: The value of this parameter will be substituted for each occurrence of MODULE_PATHNAME in the script file(s). If it is not set, no substitution is made. Typically, this is set to $libdir/shared_library_name and then MODULE_PATHNAME is used to CREATE FUNCTION commands for C-language functions so that the script files do not need to hard-wire the name of the shared library.
relocatable:Â A Boolean flag that indicates if the extension can be relocated.
Other metadata may include directory, default_version, comment, encoding, module_pathname, requires, superuser, relocatable, and schema.
Step 3. Create a Makefile to install the extension into your shared library.
1 2 3 4 5 6 7 8 |
[root@prim shared]# cat Makefile EXTENSION = add_ten DATA = add_ten--1.0.sql PG_CONFIG = pg_config PGXS := $(shell $(PG_CONFIG) --pgxs) include $(PGXS) [root@prim shared]# |
Now, upon the installation of Makefile, the installer will look for the <name–version>.sql file and <name>.control in the current directory and place the files in PostgreSQL shared library.
to install the extension, simply run make install
1 2 3 4 5 6 7 |
[root@prim shared]# make install /bin/mkdir -p '/usr/pgsql-10/share/extension' /bin/mkdir -p '/usr/pgsql-10/share/extension' /usr/bin/install -c -m 644 .//add_ten.control '/usr/pgsql-10/share/extension/' /usr/bin/install -c -m 644 .//add_ten--1.0.sql '/usr/pgsql-10/share/extension/' [root@prim shared]# [root@prim shared]# |
The installation will create two files in the shared library path.
1 2 3 4 5 6 7 8 |
[root@prim extension]# ls -lrt add_ten* -rw-r--r--. 1 root root 136 May 13 21:50 add_ten.control -rw-r--r--. 1 root root 157 May 13 21:50 add_ten--1.0.sql [root@prim extension]# [root@prim extension]# [root@prim extension]# pwd /usr/pgsql-10/share/extension [root@prim extension]# |
If the installation fails, make sure that your pg_config pointing to correct location.
1 2 3 |
[root@prim extension]# pg_config | grep SHAREDIR SHAREDIR = /usr/pgsql-10/share [root@prim extension]# |
or you can simply set path before invoking Makefile
export PATH=/usr/pgsql-10/bin:$PATH
Now, create and test extension in the database with create extension add_ten command
1 2 3 4 5 6 7 8 9 10 |
postgreshelp=# postgreshelp=# create extension add_ten; CREATE EXTENSION postgreshelp=# postgreshelp=# postgreshelp=# select add_ten(17); add_ten --------- 27 (1 row) |
Check the version of the extension with \dx command.
1 2 3 4 5 6 7 8 |
postgreshelp=# postgreshelp=# \dx List of installed extensions Name | Version | Schema | Description ---------+---------+------------+------------------------------ add_ten | 1.0 | public | Add ten to given number plpgsql | 1.0 | pg_catalog | PL/pgSQL procedural language (2 rows) |
Well, If I have the enhancements, How can I update the existing extension?
How can I update the existing PostgreSQL extension?
The above extension accepts a single value and adds 10 to that.
what if you want it to accept two values?
1 2 3 4 5 6 7 |
postgreshelp=# select add_ten(17,14); ERROR: function add_ten(integer, integer) does not exist LINE 1: select add_ten(17,14); ^ HINT: No function matches the given name and argument types. You might need to add explicit type casts. postgreshelp=# |
As you can see, the function is not accepting two values.
Now we update our existing extension to accept two values.
To update the existing extension, we need to create the two required files, control and SQL files again.
Step 1. Create a SQL script file (add_ten–1.0–1.1.sql)
1 2 3 4 5 6 7 8 9 10 |
[root@prim extension]# cat add_ten--1.0--1.1.sql CREATE OR REPLACE FUNCTION add_ten(sum INTEGER,sum2 INTEGER) RETURNS INTEGER AS $$ DECLARE result INTEGER; BEGIN result=sum+sum2; RETURN result; END; $$ LANGUAGE plpgsql; |
Step 2. Create a control file (add_ten.control)
1 2 3 4 5 6 |
[root@prim extension]# cat add_ten.control # sum_fn extension comment = 'Accept two values and add them' default_version = '1.1' module_pathname = '$libdir/add_ten' relocatable = false |
Step 3. Create a Makefile to install the extension into your shared library.
1 2 3 4 5 6 7 |
[root@prim shared]# cat Makefile EXTENSION = add_ten DATA = add_ten--1.0--1.1.sql PG_CONFIG = pg_config PGXS := $(shell $(PG_CONFIG) --pgxs) include $(PGXS) |
Step 4: run make installÂ
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
[root@prim shared]# make install /bin/mkdir -p '/usr/pgsql-10/share/extension' /bin/mkdir -p '/usr/pgsql-10/share/extension' /usr/bin/install -c -m 644 .//add_ten.control '/usr/pgsql-10/share/extension/' /usr/bin/install -c -m 644 .//add_ten--1.0--1.1.sql '/usr/pgsql-10/share/extension/' [root@prim shared]# [root@prim shared]# [root@prim shared]# cd /usr/pgsql-10/share/extension [root@prim extension]# [root@prim extension]# ls -lrt add_ten* -rw-r--r--. 1 root root 143 May 13 22:28 add_ten.control -rw-r--r--. 1 root root 172 May 13 22:28 add_ten--1.0--1.1.sql [root@prim extension]# |
You got the new set into library path.
Step 5: Update the existing PostgreSQL Extension
you can update the existing PostgreSQL extension with below command.
ALTER EXTENSION <NAME> UPDATE TO ‘<VERSION>’;
1 2 3 4 5 6 7 8 9 |
postgreshelp=# ALTER EXTENSION add_ten UPDATE TO '1.1'; ALTER EXTENSION postgreshelp=# postgreshelp=# postgreshelp=# select add_ten(17,14); add_ten --------- 31 (1 row) |
Check the settings
1 2 3 4 5 6 7 |
postgreshelp=# \dx List of installed extensions Name | Version | Schema | Description ---------+---------+------------+------------------------------ add_ten | 1.1 | public | Accept two values and add them plpgsql | 1.0 | pg_catalog | PL/pgSQL procedural language (2 rows) |
Now the update path for PostgreSQL extension should look like this
1 2 3 4 5 6 |
postgreshelp=# select * from pg_extension_update_paths('add_ten'); source | target | path --------+--------+---------- 1.0 | 1.1 | 1.0--1.1 1.1 | 1.0 | (2 rows) |
Well, we have understood that an PostgreSQLÂ extension can be configured if we have a control file and SQL file.
As the PostgreSQL is written in C, what if you want to have a function written in C, in your extension?
PostgreSQL Extension based on C function
To run your C code in your PostgreSQL extensions you need a portable build system which is provided by PGXN. We will learn more about PGXN later in this post.
PostgreSQL installation provides a build infrastructure for extensions, called PGXS, which uses a global variable USE_PGXS
 to include the global PGXS
makefiles so that simple PostgreSQL extension modules can be built simply against an already installed server. PGXSÂ is mainly intended for extensions that include C code, although it can be used for pure-SQL extensions too.
Now, we write the same function called add 10 to given value in C function and create extension based on that.
When you write your function in C, in addition to your control file and SQL file, you will have a dynamic library shared object file (.so) as well.
Step 1. Create a SQL script file (add_cfun–1.0.sql)
1 2 3 |
CREATE OR REPLACE FUNCTION add_cfun(arg INTEGER) RETURNS INTEGER AS 'MODULE_PATHNAME','add_cfun' LANGUAGE C STRICT; |
Step 2. Create a control file (add_cfun.control)
1 2 3 4 5 |
# demo extension comment = 'add cfun' default_version = '1.0' module_pathname = '$libdir/add_cfun' relocatable = true |
Step 3. Create a C program(add_cfun.c)
1 2 3 4 5 6 7 8 9 10 11 12 13 |
#include "/usr/pgsql-10/include/server/postgres.h" #include "/usr/pgsql-10/include/server/fmgr.h" #include "/usr/pgsql-10/include/server/utils/errcodes.h" PG_MODULE_MAGIC; PGDLLEXPORT Datum add_cfun(PG_FUNCTION_ARGS); PG_FUNCTION_INFO_V1(add_cfun); Datum add_cfun(PG_FUNCTION_ARGS) { int32 arg = PG_GETARG_INT32(0); PG_RETURN_INT32(arg + 10); } |
Step 4. Create a Makefile to install the extension into your shared library.
1 2 3 4 5 6 7 |
MODULES = add_cfun EXTENSION = add_cfun DATA = add_cfun--1.0.sql PG_CONFIG = pg_config PGXS := $(shell $(PG_CONFIG) --pgxs) include $(PGXS) |
At this point of time, we have 4 files.
1 2 3 4 5 6 7 8 |
[root@prim s6]# ls -lrt total 16 -rw-r--r--. 1 root root 117 May 13 23:07 add_cfun--1.0.sql -rw-r--r--. 1 root root 356 May 13 23:08 add_cfun.c -rw-r--r--. 1 root root 145 May 13 23:09 Makefile -rw-r--r--. 1 root root 116 May 13 23:13 add_ten.control [root@prim s6]# [root@prim s6]# |
Step 5. Now, install the PostgreSQL extension using make install
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
[root@prim s6]# make install gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -fPIC -I. -I./ -I/usr/pgsql-10/include/server -I/usr/pgsql-10/include/internal -I/usr/include -D_GNU_SOURCE -I/usr/include/libxml2 -I/usr/include -c -o add_cfun.o add_cfun.c gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack- protector --param=ssp-buffer-size=4 -m64 -mtune=generic -fPIC -L/usr/pgsql-10/lib -Wl, --as-needed -L/usr/lib64 -Wl,--as-needed -Wl,-rpath,'/usr/pgsql-10/lib',--enable-new-dtags -shared -o add_cfun.so add_cfun.o /bin/mkdir -p '/usr/pgsql-10/share/extension' /bin/mkdir -p '/usr/pgsql-10/share/extension' /bin/mkdir -p '/usr/pgsql-10/lib' /usr/bin/install -c -m 644 .//add_cfun.control '/usr/pgsql-10/share/extension/' /usr/bin/install -c -m 644 .//add_cfun--1.0.sql '/usr/pgsql-10/share/extension/' /usr/bin/install -c -m 755 add_cfun.so '/usr/pgsql-10/lib/' [root@prim s6]# [root@prim s6]# |
The above command installs the PostgreSQL extension and copies SQL file and Control file to a shared location and .so file shared library.
1 2 3 4 5 6 7 8 9 10 11 12 |
[root@prim s6]# [root@prim s6]# cd /usr/pgsql-10/share/extension/ [root@prim extension]# ls -lrt add_cfun* -rw-r--r--. 1 root root 118 May 13 23:14 add_cfun.control -rw-r--r--. 1 root root 117 May 13 23:14 add_cfun--1.0.sql [root@prim extension]# [root@prim extension]# [root@prim extension]# cd /usr/pgsql-10/lib/ [root@prim lib]# ls -lrt add_cfun* -rwxr-xr-x. 1 root root 10522 May 13 23:14 add_cfun.so [root@prim lib]# [root@prim lib]# |
Step 6 Create an extension in database and test
1 2 3 4 5 6 7 8 9 10 |
postgreshelp=# postgreshelp=# create extension add_cfun; CREATE EXTENSION postgreshelp=# postgreshelp=# postgreshelp=# select add_cfun(16); add_cfun ---------- 26 (1 row) |
Existing extensions so far in the database
1 2 3 4 5 6 7 8 |
postgreshelp=# \dx List of installed extensions Name | Version | Schema | Description ----------+---------+------------+------------------------------ add_cfun | 1.0 | public | add cfun add_ten | 1.1 | public | Add ten to given number plpgsql | 1.0 | pg_catalog | PL/pgSQL procedural language (3 rows) |
How to drop an extension?
postgreshelp=# drop extension add_ten;
DROP EXTENSION
postgreshelp=#
We are going to retrospect what we have learned so far on the next page