Module on PIGSTY

Module: PGSQL

Mon, 01 Jan 0001 00:00:00 +0000

The most advanced open-source relational database in the world!

With battery-included observability, reliability, and maintainability powered by Pigsty

Concept

Overview of PostgreSQL in Pigsty

Configuration

Describe the cluster you want

Identity: Parameters used for describing a PostgreSQL cluster
Primary: Define a single instance cluster
Replica: Define a basic HA cluster with one primary & one replica
Offline: Define a dedicated instance for OLAP/ETL/Interactive queries.
Sync Standby: Enable synchronous commit to ensure no data loss
Quorum Commit: Use quorum sync commit for an even higher consistency level
Standby Cluster: Clone an existing cluster and follow it
Delayed Cluster: Clone an existing cluster for emergency data recovery
Citus Cluster: Define a Citus distributed database cluster
Major Version: Define a PostgreSQL cluster with specific major version

Administration

Admin your existing clusters

Module: INFRA

Mon, 01 Jan 0001 00:00:00 +0000

Pigsty has a battery-included, production-ready INFRA module, to provide ultimate observability.

Configuration | Administration | Playbook | Dashboard | Parameter

Overview

Each Pigsty deployment requires a set of infrastructure components to work properly. which including:

Component	Port	Domain	Description
Nginx	80	`h.pigsty`	Web Service Portal (YUM/APT Repo)
AlertManager	9093	`a.pigsty`	Alert Aggregation and delivery
Prometheus	9090	`p.pigsty`	Monitoring Time Series Database
Grafana	3000	`g.pigsty`	Visualization Platform
Loki	3100	-	Logging Collection Server
PushGateway	9091	-	Collect One-Time Job Metrics
BlackboxExporter	9115	-	Blackbox Probing
Dnsmasq	53	-	DNS Server
Chronyd	123	-	NTP Time Server
PostgreSQL	5432	-	Pigsty CMDB & default database
Ansible	-	-	Run playbooks

Pigsty will set up these components for you on infra nodes. You can expose them to the outside world by configuring the infra_portal parameter.

Module: NODE

Mon, 01 Jan 0001 00:00:00 +0000

Configuration | Administration | Playbook | Dashboard | Parameter

Concept

Node is an abstraction of hardware resources, which can be bare metal, virtual machines, or even k8s pods.

There are different types of nodes in Pigsty:

Common nodes, nodes that managed by Pigsty
Admin node, the node where pigsty is installed and issue admin commands
Infra node, the node where the INFRA module is installed

The admin node is usually overlapped with the infra node, if there’s more than one infra node, the first one is often used as the default admin node, and the rest of the infra nodes can be used as backup admin nodes.

Module: ETCD

Mon, 01 Jan 0001 00:00:00 +0000

ETCD is a distributed, reliable key-value store for the most critical data of a distributed system

Configuration | Administration | Playbook | Dashboard | Parameter

Pigsty use etcd as DCS: Distributed configuration storage (or distributed consensus service). Which is critical to PostgreSQL High-Availability & Auto-Failover.

You have to install ETCD module before any PGSQL modules, since patroni & vip-manager will rely on etcd to work. Unless you are using an external etcd cluster.

You don’t need NODE module to install ETCD, but it requires a valid CA on your local files/pki/ca. Check ETCD Administration SOP for more details.

Module: MINIO

Mon, 01 Jan 0001 00:00:00 +0000

Min.IO: S3-Compatible Open-Source Multi-Cloud Object Storage

Configuration | Administration | Playbook | Dashboard | Parameter

MinIO is an S3-compatible object storage server. It’s designed to be scalable, secure, and easy to use. It has native multi-node multi-driver HA support and can store documents, pictures, videos, and backups.

Pigsty uses MinIO as an optional PostgreSQL backup storage repo, in addition to the default local posix FS repo. If the MinIO repo is used, the MINIO module should be installed before any PGSQL modules.

Module: REDIS

Mon, 01 Jan 0001 00:00:00 +0000

Concept

The entity model of Redis is almost the same as that of PostgreSQL, which also includes the concepts of Cluster and Instance. The Cluster here does not refer to the native Redis Cluster mode.

The core difference between the REDIS module and the PGSQL module is that Redis uses a single-node multi-instance deployment rather than the 1:1 deployment: multiple Redis instances are typically deployed on a physical/virtual machine node to utilize multi-core CPUs fully. Therefore, the ways to configure and administer Redis instances are slightly different from PGSQL.

Module: MONGO

Mon, 01 Jan 0001 00:00:00 +0000

Configuration | Administration | Playbook | Dashboard | Parameter

Overview

MongoDB was once a stunning technology, allowing developers to cast aside the “schema constraints” of relational databases and quickly build applications. However, over time, MongoDB abandoned its open-source nature, changing its license to SSPL, which made it unusable for many open-source projects and early commercial projects. Most MongoDB users actually do not need the advanced features provided by MongoDB, but they do need an easy-to-use open-source document database solution. To fill this gap, FerretDB was born.

Module: DOCKER

Mon, 01 Jan 0001 00:00:00 +0000

Deploy docker on Pigsty managed nodes: Configuration | Administration | Playbook | Dashboard | Parameter

Concept

Docker is a popular container service which provides standardize software deliver solution.

Configuration

Docker module is different from other modules, which does not require pre-configuration to install & enable.

Just run the docker.yml playbook on any Pigsty managed node.

But if you wish to add docker daemon as prometheus monitoring target, you have to specify the docker_enabled parameters on those nodes.

Software & Tools

Sun, 23 Jun 2024 00:00:00 +0000

PostgreSQL is the most popular database in the world, and countless software is built on PostgreSQL, around PostgreSQL, or serves PostgreSQL itself, such as

“Application software” that uses PostgreSQL as the preferred database
“Tooling software” that serves PostgreSQL software development and management
“Database software” that derives, wraps, forks, modifies, or extends PostgreSQL

And Pigsty just have a series of Docker Compose templates for these software, application and databases:

Name	Website	Type	State	Port	Domain	Description
Supabase	Supabase	DB	GA	8000	supa.pigsty	OSS Firebase Alternative, Backend as Platform
PolarDB	PolarDB	DB	GA	5532		OSS RAC for PostgreSQL
FerretDB	FerretDB	DB	GA	27017		OSS Mongo Alternative base on PostgreSQL
MinIO	MinIO	DB	GA	9000	sss.pigsty	OSS AWS S3 Alternative, Simple Storage Service
EdgeDB	EdgeDB	DB	Planned			OSS Graph Database base on PostgreSQL
NocoDB	NocoDB	APP	GA	8080	noco.pigsty	OSS Airtable Alternative over PostgreSQL
Odoo	Odoo	APP	GA	8069	odoo.pigsty	OSS ERP Software base on PostgreSQL
Jupyter	Jupyter	APP	GA	8888	lab.pigsty	OSS AI Python Notebook & Data Analysis IDE
Gitea	Gitea	APP	GA	8889	git.pigsty	OSS DevOps Git Service
Wiki	Wiki.js	APP	GA	9002	wiki.pigsty	OSS Wiki Software
GitLab	GitLab	APP	Planned			OSS GitHub Alternative, Code Management Platform
Mastodon	Mastodon	APP	Planned			OSS Decentralized Social Network
Keycloak	Keycloak	APP	Planned			OSS Identity & Access Management Component
Harbour	Harbour	APP	Planned			OSS Docker/K8S Image Repository
Confluence	Confluence	APP	Planned			Enterprise Knowledge Management System
Jira	Jira	APP	Planned			Enterprise Project Management Tools
Zabbix	Zabbix 7	APP	Planned			OSS Monitoring Platform for Enterprise
Grafana	Grafana	APP	Planned			Dashboard, Data Visualization & Monitoring Platform
Metabase	Metabase	APP	GA	9004	mtbs.pigsty	Fast analysis of data from multiple data sources
ByteBase	ByteBase	APP	GA	8887	ddl.pigsty	Database Migration Tool for PostgreSQL
Kong	Kong	TOOL	GA	8000	api.pigsty	OSS API Gateway based on Nginx/OpenResty
PostgREST	PostgREST	TOOL	GA	8884	api.pigsty	Generate RESTAPI from PostgreSQL Schemas
pgAdmin4	pgAdmin4	TOOL	GA	8885	adm.pigsty	PostgreSQL GUI Admin Tools
pgWeb	pgWeb	TOOL	GA	8886	cli.pigsty	PostgreSQL Web GUI Client
SchemaSpy	SchemaSpy	TOOL	Planned			Dump & Visualize PostgreSQL Schema
pgBadger	pgBadger	TOOL	Planned			PostgreSQL Log Analysis
pg_exporter	pg_exporter	TOOL	GA	9630		Expose PostgreSQL & Pgbouncer Metrics for Prometheus

How to prepare Docker?

To run docker compose templates, you need to install the DOCKER module on the node, If you don’t have the Internet access or having firewall issues, you may need to configure a DockerHub proxy, check the tutorial.