History

Jeff Moe 020fc5e4d1 moar postgres-xl		6 years ago
..
README.md	moar postgres-xl	6 years ago

README.md

Postgres-XL

Best view this file raw for now: https://code.forksand.com/forksand/forksand-it-manual/raw/branch/master/source/resources/apps/postgres-xl/README.md

https://www.postgres-xl.org

https://www.postgres-xl.org/documentation/install-short.html https://www.postgres-xl.org/documentation/install-procedure.html

Set up 9 servers

Data nodes: pgxl1 pgxl2 pgxl3 pgxl4 pgxl5

GTM: pgxl-gtm-1 pgxl-gtm-2

Coordinators: pgxl-coord-1 pgxl-coord-2 pgxl-coord-3

Open Up firewall.

Open up firewall ports for postgres and others. All TCP (?). Postgres: 5432 GTM: 20001-20003 Coordinator: 30001-30003, 31001-31003 Data Node: 40001-40003, 41001-41003

-A INPUT -p tcp --dest 10.53.1.0/24 --dport 5432 -j ACCEPT -A INPUT -p tcp --dest 10.53.1.0/24 --dport 20001 -j ACCEPT -A INPUT -p tcp --dest 10.53.1.0/24 --dport 20002 -j ACCEPT -A INPUT -p tcp --dest 10.53.1.0/24 --dport 20003 -j ACCEPT -A INPUT -p tcp --dest 10.53.1.0/24 --dport 30001 -j ACCEPT -A INPUT -p tcp --dest 10.53.1.0/24 --dport 30002 -j ACCEPT -A INPUT -p tcp --dest 10.53.1.0/24 --dport 30003 -j ACCEPT -A INPUT -p tcp --dest 10.53.1.0/24 --dport 31001 -j ACCEPT -A INPUT -p tcp --dest 10.53.1.0/24 --dport 31002 -j ACCEPT -A INPUT -p tcp --dest 10.53.1.0/24 --dport 31003 -j ACCEPT -A INPUT -p tcp --dest 10.53.1.0/24 --dport 40001 -j ACCEPT -A INPUT -p tcp --dest 10.53.1.0/24 --dport 40002 -j ACCEPT -A INPUT -p tcp --dest 10.53.1.0/24 --dport 40003 -j ACCEPT -A INPUT -p tcp --dest 10.53.1.0/24 --dport 41001 -j ACCEPT -A INPUT -p tcp --dest 10.53.1.0/24 --dport 41002 -j ACCEPT -A INPUT -p tcp --dest 10.53.1.0/24 --dport 41003 -j ACCEPT

Meh, open it wide:

-A INPUT -p tcp --dest 10.53.1.0/24 -j ACCEPT

Set up SSH between hosts

ssh-keygen -t ed25519 set up root ssh between all nodes

/etc/hosts

set up /etc/hosts with all cluster name/IPs.

Set up /etc/hosts with all cluster name/IPs for Postgres interface, with pg- prefix

Get source

Binary: wget https://www.postgres-xl.org/downloads/postgres-xl-10r1.tar.gz

Git (better): git clone https://git.postgresql.org/git/postgres-xl.git

Install Dependencies

apt install build-essential bison flex libreadline-dev zlib1g-dev pbzip2

Build

git checkout remotes/origin/XL_10_STABLE

mkdir build_dir cd build_dir ../configure --bindir=/usr/local/bin make -j8 sudo make install

System Startup

vim /etc/systemd/system/postgresql.service [Unit] Description=PostgreSQL database server Documentation=man:postgres(1)

[Service] Type=notify User=postgres ExecStart=/usr/local/bin/postgres -D /usr/local/pgsql/data ExecReload=/bin/kill -HUP $MAINPID KillMode=mixed KillSignal=SIGINT TimeoutSec=0

[Install] WantedBy=multi-user.target

Configure

adduser postgres

Run this as root: /sbin/ldconfig /usr/local/pgsql/lib

XXX no? Add this to ~/.bashrc LD_LIBRARY_PATH=/usr/local/pgsql/lib export LD_LIBRARY_PATH

Set up user postgres to ssh to all nodes on database ethernet interface.

chown postgres:postgres /usr/local/pgsql

https://www.postgres-xl.org/documentation/pgxc-ctl.html

Log into pgxl1, run: sudo su - postgres pgxc_ctl At PGXC promt run: prepare config empty

GTM master, GTM slave, GTM proxy, Coordinator master/slave and Datanode master/slave need its own port to listen to. Coordinators and Datanodes also need a pooler port to pool connections to Datanodes.

While adding a Coordinator master and a Datanode master, extra server configuration and extra pg_hba configuration parameters can be specified in a file.

add gtm master name host port dir add gtm slave name host port dir add gtm_proxy name host port dir add coordinator master name host port pooler dir extraServerConf extraPgHbaConf add coordinator slave name host port pooler dir archDir add datanode master name host port pooler dir waldir extraServerConf extraPgHbaConf add datanode slave name host port pooler dir waldir archDir

XLDIR=/usr/local/pgsql export XLDIR pgxc_ctl prepare config empty

=============================== vim pgxc_ctl/pgxc_ctl.conf Comment out: coordPgHbaEntries=(::1/128) datanodePgHbaEntries=(::1/128)

Enable/extend: coordPgHbaEntries=(127.0.0.1/32 10.53.1.0/24) datanodePgHbaEntries=(127.0.0.1/32 10.53.1.0/24)

XXX Set up slaves/proxies

pgxc_ctl add gtm master gtm_1 pg-pgxl-gtm-1 20001 "$XLDIR/gtm-1" pgxc_ctl add gtm master gtm_2 pg-pgxl-gtm-2 20002 "$XLDIR/gtm-2"

pgxc_ctl add gtm_proxy proxy_1 pg-pgxl-proxy-1 20010 "$XLDIR/proxy-1"

XXX pgxc_ctl add gtm slave gtm_2 pg-pgxl-gtm-2 20002 "$XLDIR/gtm-2" XXX pgxc_ctl add gtm_proxy gtm_2 pg-pgxl-gtm-2 20003 "$XLDIR/gtm-3"

pgxc_ctl add coordinator master coord_1 pg-pgxl-coord-1 30001 31001 "$XLDIR/coord-1" none none pgxc_ctl add coordinator master coord_2 pg-pgxl-coord-2 30001 31001 "$XLDIR/coord-2" none none pgxc_ctl add coordinator master coord_3 pg-pgxl-coord-3 30001 31001 "$XLDIR/coord-3" none none

pgxc_ctl add datanode master datanode_1 pg-pgxl1 40001 41001 "$XLDIR/data-1" none none none pgxc_ctl add datanode master datanode_2 pg-pgxl2 40001 41001 "$XLDIR/data-2" none none none pgxc_ctl add datanode master datanode_3 pg-pgxl3 40001 41001 "$XLDIR/data-3" none none none pgxc_ctl add datanode master datanode_4 pg-pgxl4 40001 41001 "$XLDIR/data-4" none none none pgxc_ctl add datanode master datanode_5 pg-pgxl5 40001 41001 "$XLDIR/data-5" none none none pgxc_ctl monitor all createdb -p 30001 test

Create database

ssh pgxl-coord-1 sudo su - postgres psql -p 30001 create database foo;

Check it replicated:

ssh pgxl1 sudo su - postgres psql -p 40001 \l

MISC FIXES/TUNING

in /usr/local/pgsql/[data|gtm|coord]/postgresql.conf Set max connections from 100 to 1000.

LOCATION: StreamConnection, /home/jebba/devel/postgres-xl/build_dir/../src/gtm/libpq/pqcomm.c:351 25497:3689674496:2018-11-27 13:55:47.067 MST -LOG: could not accept new connection: Too many open files

/etc/security/limits.d/pgxl.conf: XXX increase fileno (open files) limit

soft nofile 8192
hard nofile 8192

ERROR

Many: ERROR: Unique index of distributed table must contain the hash distribution column.

Many like: 2018-11-27 15:51:23.636 MST [1114] ERROR: canceling autovacuum task 2018-11-27 15:51:23.636 MST [1114] CONTEXT: automatic analyze of table "ao_odoo.public.stock_quant"

2018-11-27 15:52:36.257 MST [988] LOG: checkpoints are occurring too frequently (26 seconds apart) 2018-11-27 15:52:36.257 MST [988] HINT: Consider increasing the configuration parameter "max_wal_size".

2018-11-27 15:53:20.026 MST [3020] ERROR: Hash/Modulo distribution column does not refer to hash/modulo distribution column in referenced table. 2018-11-27 15:53:20.026 MST [3020] STATEMENT: ALTER TABLE ONLY public.account_tax_template ADD CONSTRAINT account_tax_template_company_id_fkey FOREIGN KEY (company_id) REFERENCES public.res_company(id) ON DELETE SET NULL;

After uploading the odoo database, it ended and hung on this forever. The datanode's postgres was hitting CPU. REFRESH MATERIALIZED VIEW