122 lines
No EOL
8.2 KiB
Markdown
122 lines
No EOL
8.2 KiB
Markdown
# Pilgrim: An SQL Migration Tool Worth The Pilgrimage
|
|
|
|
## Usage
|
|
|
|
```bash
|
|
$ pilgrim --url 'mariadb://localhost:3306/database' --up
|
|
$ pilgrim --url 'driver://user:pass@host:port/database?arg1=val1&arg2=val2' --dir ./migrations --up
|
|
$ pilgrim --url 'postgres://localhost:5432/database' --down '2024-09-27/1_CreateTableABC.sql'
|
|
$ pilgrim --driver 'driver' --username 'user' --password 'pass' --host 'host' --port 'port' --segments 'database' --args 'arg1=val1&arg2=val2'
|
|
$ pilgrim --validate_migration_order --validate_latest
|
|
```
|
|
|
|
### Configuration
|
|
|
|
The order of precedence for configured values is as follows:
|
|
|
|
1. Values provided in explicit cli arguments ( `--username`, `--password`, `--host`, `--port`, etc. )
|
|
2. Values provided in cli `--url`
|
|
3. Values provided in `PILGRIM_*` environment variables
|
|
- Values in `PILGRIM_URL` are of lowest precedence and are overridden by their
|
|
explicit `PILGRIM_*` equivalents
|
|
|
|
By default, pilgrim will search for a `.pilgrim` file in the directory it has
|
|
been run in and will use the values within to replace values that have not been
|
|
provided in a higher precedence form.
|
|
|
|
#### CLI Arguments
|
|
|
|
| Argument | Meaning |
|
|
|---------------------------------|---------------------------------------------------------------------------------------------------------------------------------|
|
|
| `--url` | The database url |
|
|
| `--driver` | The database driver ( must be one of `mysql`, `mssql`, `postgres`, `mariadb`, `oracle`, or `sqlite` ) |
|
|
| `--username` | The username to use |
|
|
| `--password` | The password to use |
|
|
| `--host` | The host/ip of the db server |
|
|
| `--port` | The port of the db server |
|
|
| `--segments` | The segments ( typically the db name ) |
|
|
| `--arguments` | The arguments for the db server |
|
|
| `--dir` | The directory of the db migrations |
|
|
| `--down` | The migration to downgrade to ( conflicts with `--up` ). Defaults to `false` |
|
|
| `--up` | The migration to upgrade to ( conflicts with `--down` ). Defaults to `false` |
|
|
| `--script` | The migration script to upgrade/downgrade to. Can also be `latest`. ( Defaults to `latest` ) |
|
|
| `--schema` | The migration table schema. Must exist prior to running migrations. Default value depends on database driver ( default schema ) |
|
|
| `--table` | The migration table name. Default value is "pilgrim_migrations" |
|
|
| `--disable_checksum_validation` | Disable checksum validation when running database migration |
|
|
| `--strict_ordering` | Enforce strict ordering, pilgrim will run a validation of migration execution order before running the migrations |
|
|
| `--validate_migration_order` | (Special) Runs a validation of the migration directory to ensure there is no uncertainty in migration order of execution. |
|
|
| `--validate_latest` | (Special) Validate that the provided database contains all migrations in the migration directory |
|
|
| `--validate_checksums` | (Special) Validate that all current checksums of migration files match the ones stored in the migration table |
|
|
|
|
##### Conflicts between `--down` and `--up`
|
|
|
|
If both `--down` and `--up` are provided, `pilgrim` will return an error and
|
|
will not migrate the database. The `--script` name provided must adhere to the
|
|
[migration naming convention and contain the folder it is located in](#migration-naming-convention-folder-structure).
|
|
|
|
#### Environment Variables
|
|
|
|
| Variable | Meaning | Example |
|
|
|-------------------|---------------------------------------------------------------------|-----------------------------------------------------------------------|
|
|
| PILGRIM_URL | The database url | PILGRIM_URL=driver://user:pass@host:port/database?arg1=val1&arg2=val2 |
|
|
| PILGRIM_DRIVER | The driver for the database | PILGRIM_DRIVER=postgres |
|
|
| PILGRIM_USERNAME | The username to use | PILGRIM_USERNAME=user |
|
|
| PILGRIM_PASSWORD | The password to use | PILGRIM_PASSWORD=pass |
|
|
| PILGRIM_HOST | The host/ip of the db server | PILGRIM_HOST=localhost |
|
|
| PILGRIM_PORT | The port of the db server | PILGRIM_PORT=9999 |
|
|
| PILGRIM_SEGMENTS | The segments ( typically the db name ) | PILGRIM_SEGMENTS=database; PILGRIM_SEGMENTS=db/seg1 |
|
|
| PILGRIM_ARGUMENTS | The arguments for the db server | PILGRIM_ARGUMENTS="arg1=val1&arg2=val2" |
|
|
| PILGRIM_DIRECTORY | The directory of the db migrations | PILGRIM_DIRECTORY=./migrations |
|
|
| PILGRIM_SCHEMA | The migration table schema. Must exist prior to running migrations. | PILGRIM_SCHEMA="public" |
|
|
| PILGRIM_TABLE | The migration table. | PILGRIM_TABLE="pilgrim_migrations" |
|
|
|
|
#### Migration Naming Convention & Folder Structure
|
|
|
|
`pilgrim` requires that migration files be named uniquely, and can be ordered.
|
|
To ensure this, migrations must be sorted by date, and then further within each
|
|
date-named directory, the contained migration files must begin with a number.
|
|
The numbers need not be unique ( you can have multiple scripts starting with
|
|
`1_`, for example ), but the name of the migration does.
|
|
|
|
Example migration directory:
|
|
|
|
```
|
|
migrations
|
|
├─2024-09-28
|
|
│ ├─1_CreateTableA.sql
|
|
│ ├─2_CreateViewA.sql
|
|
│ └─3_DropConstraintA.sql
|
|
├─2024-10-01
|
|
│ ├─1_CreateTableB.sql
|
|
│ └─1_CreateTableC.sql
|
|
└─2024-10-12
|
|
└─...
|
|
```
|
|
|
|
#### Migration Definition
|
|
|
|
Creating a pilgrim migration involves creating a regular SQL file, with some
|
|
mandatory comments contained within:
|
|
|
|
```sql
|
|
-- PILGRIM::UP
|
|
|
|
CREATE TABLE ...
|
|
|
|
-- PILGRIM::DOWN
|
|
|
|
DROP TABLE ...
|
|
```
|
|
|
|
Everything between `-- PILGRIM::UP` and `-- PILGRIM::DOWN` is run during the
|
|
`--up` phase of migration.
|
|
|
|
Everything after `--PILGRIM::DOWN` to the end of the file is run during the
|
|
`--down` phase of migration.
|
|
|
|
If a file is missing these comments, the migration will not be run, and the file
|
|
will be ignored.
|
|
|
|
## Development
|
|
|
|
See [Development Documentation](./docs/README.md) |