# Pilgrim: An SQL Migration Tool Worth The Pilgrimage
## Usage
```bash
$ pilgrim --url 'mariadb://localhost:3306/database' --up
$ pilgrim --url 'driver://user:pass@host:port/database?arg1=val1&arg2=val2' --dir ./migrations --up
$ pilgrim --url 'postgres://localhost:5432/database' --down '2024-09-27/1_CreateTableABC.sql'
$ pilgrim --driver 'driver' --username 'user' --password 'pass' --host 'host' --port 'port' --segments 'database' --args 'arg1=val1&arg2=val2'
$ pilgrim --validate_migration_order --validate_latest
```
### Configuration
The order of precedence for configured values is as follows:
1. Values provided in explicit cli arguments ( `--username`, `--password`, `--host`, `--port`, etc. )
2. Values provided in cli `--url`
3. Values provided in `PILGRIM_*` environment variables
- Values in `PILGRIM_URL` are of lowest precedence and are overridden by their
explicit `PILGRIM_*` equivalents
By default, pilgrim will search for a `.pilgrim` file in the directory it has
been run in and will use the values within to replace values that have not been
provided in a higher precedence form.
#### CLI Arguments
| Argument | Meaning |
|---------------------------------|---------------------------------------------------------------------------------------------------------------------------------|
| `--url` | The database url |
| `--driver` | The database driver ( must be one of `mysql`, `mssql`, `postgres`, `mariadb`, `oracle`, or `sqlite` ) |
| `--username` | The username to use |
| `--password` | The password to use |
| `--host` | The host/ip of the db server |
| `--port` | The port of the db server |
| `--segments` | The segments ( typically the db name ) |
| `--arguments` | The arguments for the db server |
| `--dir` | The directory of the db migrations |
| `--down` | The migration to downgrade to ( conflicts with `--up` ). Defaults to `false` |
| `--up` | The migration to upgrade to ( conflicts with `--down` ). Defaults to `false` |
| `--script` | The migration script to upgrade/downgrade to. Can also be `latest`. ( Defaults to `latest` ) |
| `--schema` | The migration table schema. Must exist prior to running migrations. Default value depends on database driver ( default schema ) |
| `--table` | The migration table name. Default value is "pilgrim_migrations" |
| `--disable_checksum_validation` | Disable checksum validation when running database migration |
| `--strict_ordering` | Enforce strict ordering, pilgrim will run a validation of migration execution order before running the migrations |
| `--validate_migration_order` | (Special) Runs a validation of the migration directory to ensure there is no uncertainty in migration order of execution. |
| `--validate_latest` | (Special) Validate that the provided database contains all migrations in the migration directory |
| `--validate_checksums` | (Special) Validate that all current checksums of migration files match the ones stored in the migration table |
##### Conflicts between `--down` and `--up`
If both `--down` and `--up` are provided, `pilgrim` will return an error and
will not migrate the database. The `--script` name provided must adhere to the
[migration naming convention and contain the folder it is located in](#migration-naming-convention-folder-structure).
#### Environment Variables
| Variable | Meaning | Example |
|-------------------|---------------------------------------------------------------------|-----------------------------------------------------------------------|
| PILGRIM_URL | The database url | PILGRIM_URL=driver://user:pass@host:port/database?arg1=val1&arg2=val2 |
| PILGRIM_DRIVER | The driver for the database | PILGRIM_DRIVER=postgres |
| PILGRIM_USERNAME | The username to use | PILGRIM_USERNAME=user |
| PILGRIM_PASSWORD | The password to use | PILGRIM_PASSWORD=pass |
| PILGRIM_HOST | The host/ip of the db server | PILGRIM_HOST=localhost |
| PILGRIM_PORT | The port of the db server | PILGRIM_PORT=9999 |
| PILGRIM_SEGMENTS | The segments ( typically the db name ) | PILGRIM_SEGMENTS=database; PILGRIM_SEGMENTS=db/seg1 |
| PILGRIM_ARGUMENTS | The arguments for the db server | PILGRIM_ARGUMENTS="arg1=val1&arg2=val2" |
| PILGRIM_DIRECTORY | The directory of the db migrations | PILGRIM_DIRECTORY=./migrations |
| PILGRIM_SCHEMA | The migration table schema. Must exist prior to running migrations. | PILGRIM_SCHEMA="public" |
| PILGRIM_TABLE | The migration table. | PILGRIM_TABLE="pilgrim_migrations" |
#### Migration Naming Convention & Folder Structure
`pilgrim` requires that migration files be named uniquely, and can be ordered.
To ensure this, migrations must be sorted by date, and then further within each
date-named directory, the contained migration files must begin with a number.
The numbers need not be unique ( you can have multiple scripts starting with
`1_`, for example ), but the name of the migration does.
Example migration directory:
```
migrations
├─2024-09-28
│ ├─1_CreateTableA.sql
│ ├─2_CreateViewA.sql
│ └─3_DropConstraintA.sql
├─2024-10-01
│ ├─1_CreateTableB.sql
│ └─1_CreateTableC.sql
└─2024-10-12
└─...
```
#### Migration Definition
Creating a pilgrim migration involves creating a regular SQL file, with some
mandatory comments contained within:
```sql
-- PILGRIM::UP
CREATE TABLE ...
-- PILGRIM::DOWN
DROP TABLE ...
```
Everything between `-- PILGRIM::UP` and `-- PILGRIM::DOWN` is run during the
`--up` phase of migration.
Everything after `--PILGRIM::DOWN` to the end of the file is run during the
`--down` phase of migration.
If a file is missing these comments, the migration will not be run, and the file
will be ignored.
## Development
See [Development Documentation](./docs/README.md)