1 2 3 4 5 |
[postgres@stagdb pg_dynshmem]$ ls -lrt total 8 -rw------- 1 postgres postgres 6928 Feb 17 16:36 mmap.2106693104 [postgres@stagdb pg_dynshmem]$ pwd /u01/pgsql/10/pg_dynshmem |
There is a file in pg_dynshmem sub-directory under PostgreSQL data directory.
- But what is that file?
The answer is here,
But before we understand that, we will have to understand some basics. If you want to understand the things practically quickly jump into practicals section but I would highly recommend you to go thru a bit theory.
Table of Contents
PostgreSQL Dynamic Shared Memory Types
PostgreSQL supports the following dynamic shared memory types
- posix
- sysv
- windows
- mmap
A bit background:
What is a process and what is Inter-process communication(IPC)?
A process is a series of actions or steps to achieve something.
A process within a system may be independent or cooperative. If the process is independent it is not affected by the execution of other processes, on the other side if it is cooperative, it can be affected by other processes.
Inter process communication is a mechanism which allows processes to communicate with each other and synchronize their actions.
Linux Kernel provides different types of IPC mechanisms of which PostgreSQL uses System V or POSIX IPC or MMAP mechanisms.
PostgreSQL’s shared buffers uses one of the above mechanisms for caching its pages.
When data from one process needs to be shared with another process, The first process simply writes data into the shared memory segment.
As soon as it is written, the data becomes available to the second process.
Once a shared-memory object is created, processes with access to the object can use pointers to directly read and write into it.
Also read: Operating system kernel parameters and PostgreSQL
Why dynamic shared memory control segment?
As we discussed, there are four dynamic shared memory types your PostgreSQL supports but we limit our discussion to POSIX and MMAP as we are dealing with Linux Operating System and POSIX is newer version of SYS V.
Creating and managing the Dynamic Shared Memory is operating system task. Once it is created,
- dynamic shared memory segments need to be reference-counted, so that when the last mapping is removed, the segment automatically goes away
- If a backend is terminated uncleanly, the postmaster needs to remove all leftover segments during the crash-and-restart process, just as it needs to reinitialize the main shared memory segment.
- And if all processes are terminated uncleanly, the next postmaster startup needs to clean up any segments that still exist, again just as we already do for the main shared memory segment.
Neither POSIX nor MMAP keep track of these details.
- So we need to keep track ourselves of what we have outstanding.
- Ensure the mapping is there for back-ends and segments which are necessary
To solve the above challenges, PostgreSQL, in fact Robert Haas has worked on implementing a concept called “dynamic shared memory control segment“.
Dynamic shared memory control segment
The dynamic shared memory control segment is created at startup (or re initialization) time by the postmaster before any user process are created. It is used to store a list of the identities of all the other dynamic shared memory segments we have outstanding and the reference count of each.
The following things the dynamic shared memory control segment does
- If the postmaster goes through a crash-and-reset cycle, it scans the control segment and removes all the other segments mentioned there, and then recreates the control segment itself.
- If the postmaster is killed off (e.g. kill -9) and restarted, it locates the old control segment and proceeds similarly.
- If the whole operating system is rebooted, the old control segment won’t exist any more,except under the mmap-a-regular-file implementation, which handles cleanup by scanning the relevant directory rather than relying on the control segment.
In summary, Thus, there are four separate occasions on which we remove shared memory segments:
- resource owner cleanup,
- backend exit (for any session-lifespan mappings and anything else that slips through the cracks),
- postmaster exit (in case a child dies without cleaning itself up),
- and postmaster startup (in case the postmaster dies without cleaning up).
Let’s work on practicals
Where do my dynamic shared memory control segment stored when my dynamic_shared_memory_type is MMAP?
Ensure that your dynamic_shared_memory_type is mmap. if not set it in your postgresql.conf file or set it using
1 2 3 4 5 6 7 8 9 |
postgres=# alter system set dynamic_shared_memory_type='mmap'; ALTER SYSTEM postgres=# select name, setting from pg_settings where name like 'dynamic_shared_memory_type'; name | setting ----------------------------+--------- dynamic_shared_memory_type | mmap (1 row) |
Restart the PostgreSQL cluster
/usr/local/pgsql_10/bin/pg_ctl stop -D /u01/pgsql/10
Once the cluster is restarted, you can see the file mmap.345067392 which is your dynamic shared memory control segment
1 2 3 4 5 6 |
[postgres@stagdb pg_dynshmem]$ ls -lrt total 56 -rw------- 1 postgres postgres 50128 Feb 17 21:38 mmap.345067392 [postgres@stagdb pg_dynshmem]$ pwd /u01/pgsql/10/pg_dynshmem [postgres@stagdb pg_dynshmem]$ |
The alert log says,
2020-02-17 21:38:30.463 IST [4831] DEBUG: dynamic shared memory system will support 2088 segments
2020-02-17 21:38:30.463 IST [4831] DEBUG: created dynamic shared memory control segment 345067392 (50128 bytes)
Let me restart the cluster
/usr/local/pgsql_10/bin/pg_ctl restart -D /u01/pgsql/10
The alert log says,
2020-02-17 21:43:09.920 IST [4831] DEBUG: cleaning up dynamic shared memory control segment with ID 345067392
….
2020-02-17 21:43:10.129 IST [4953] DEBUG: dynamic shared memory system will support 2088 segments
2020-02-17 21:43:10.138 IST [4953] DEBUG: created dynamic shared memory control segment 1230606383 (50128 bytes)
Normal shutdown has cleaned the dynamic shared memory control segment and created a new one during the next startup.
[postgres@stagdb pg_dynshmem]$ ls -rlt mmap.1230606383
-rw——- 1 postgres postgres 50128 Feb 17 21:43 mmap.1230606383
When I removed the file and stopped my cluster, I got below error
2020-02-17 21:49:16.698 IST [4953] LOG: could not remove shared memory segment “pg_dynshmem/mmap.1230606383”: No such file or directory
So the file in under pg_dynshmem is called dynamic shared memory control segment.
Where do my dynamic shared memory control segment stored when my dynamic_shared_memory_type is POSIX?
If the dynamic shared memory segment type is POSIX then dynamic shared memory control segment is stored in /dev/shm
1 2 3 4 5 6 |
[postgres@stagdb 10]$ cd /dev/shm/ [postgres@stagdb shm]$ ls -rlt total 60 -rw------- 1 postgres postgres 50128 Feb 17 21:52 PostgreSQL.659433638 [postgres@stagdb shm]$ pwd /dev/shm]$ |
The use of the mmap option, which is not the default on any platform, is generally discouraged because the operating system may write modified pages back to disk repeatedly, increasing system I/O load;
however, it may be useful for debugging, when the pg_dynshmem directory is stored on a RAM disk, or when other shared memory facilities are not available.
The article here will help you to choose between POSIX and MMAP.
Dynamic Shared Memory Types – Benchmark
The following benchmark test clearly shows POSIX worked better than MMAP.
Tabular form of benchmark results
Clients | POSIX tps | MMAP tps |
2 | 529 | 227 |
4 | 860 | 569 |
6 | 2308 | 1493 |
8 | 37734 | 31418 |
10 | 66453 | 67267 |
Setup used to perform benchmark test
- RAM : 12 GB
- DISK : HDD
- CPU : 8
- SHARED BUFFERS : 128 MB
- TABLES: pgbench_accounts(260 MB), pgbench_branches (168 kB), pgbench_history (0 bytes), pgbench_tellers (744 kB)
- script used
1 2 3 4 5 6 7 |
#!/bin/sh for clients in 2 4 6 8 10 do THREADS=${clients} /usr/local/pgsql_10/bin/pgbench -h 127.0.0.1 -j ${THREADS} -c ${clients} -T 20 -S postgres > result_mmap${clients}.txt done |
Further reading?
Robert Haas has an awesome write-up
FP*********@ma**.com
” target=”_blank” rel=”noopener noreferrer”>here while he has released the patch.
Thank you for giving your valuable time to read the above information.
If you liked the work please like the page on Facebook
I hope the content served your purpose in reaching out the blog.
Suggestions for improvement of the blog are highly appreciable. Please contact us for any information/suggestion/feedback.
If you want to be updated with all our articles
please follow us on Facebook | Twitter
Please subscribe to our newsletter.