584 lines
12 KiB
Markdown
584 lines
12 KiB
Markdown
# 数据库迁移工具 - 完整指南
|
||
|
||
## 📌 核心特点
|
||
|
||
- ✅ **独立工具，零耦合** - 与应用代码完全分离
|
||
- ✅ **生产就绪** - 编译成二进制，无需Go环境
|
||
- ✅ **灵活配置** - 支持命令行参数、环境变量、配置文件
|
||
- ✅ **Docker友好** - 挂载配置，修改无需重启容器
|
||
|
||
---
|
||
|
||
## 🚀 快速开始（3步）
|
||
|
||
> **黑盒模式**：迁移工具内部调用 `migration.RunMigrationsFromConfig()` 方法，自动处理配置加载、数据库连接、迁移执行等所有细节。你只需要提供配置文件和SQL文件即可。
|
||
|
||
### 1. 复制迁移工具模板
|
||
|
||
```bash
|
||
mkdir -p cmd/migrate
|
||
cp /path/to/go-common/templates/migrate/main.go cmd/migrate/
|
||
```
|
||
|
||
### 2. 创建迁移文件
|
||
|
||
```bash
|
||
mkdir -p migrations
|
||
```
|
||
|
||
创建 `migrations/20240101000001_create_users.sql`：
|
||
|
||
```sql
|
||
CREATE TABLE users (
|
||
id BIGINT PRIMARY KEY AUTO_INCREMENT,
|
||
username VARCHAR(255) NOT NULL,
|
||
email VARCHAR(255) NOT NULL,
|
||
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
|
||
);
|
||
```
|
||
|
||
### 3. 编译和使用
|
||
|
||
```bash
|
||
# 编译（生产环境推荐）
|
||
go build -o bin/migrate cmd/migrate/main.go
|
||
|
||
# 使用
|
||
./bin/migrate up # 使用默认配置
|
||
./bin/migrate up -config /path/to/config.json # 指定配置
|
||
./bin/migrate status # 查看状态
|
||
./bin/migrate -help # 查看帮助
|
||
```
|
||
|
||
---
|
||
|
||
## 💻 本地使用
|
||
|
||
### 开发环境
|
||
|
||
```bash
|
||
# 直接运行（需要Go环境）
|
||
go run cmd/migrate/main.go up
|
||
go run cmd/migrate/main.go up -config dev.json
|
||
|
||
# 编译后运行（推荐）
|
||
go build -o bin/migrate cmd/migrate/main.go
|
||
./bin/migrate up
|
||
./bin/migrate up -config config.prod.json
|
||
./bin/migrate up -c prod.json -d db/migrations
|
||
```
|
||
|
||
### 命令说明
|
||
|
||
```bash
|
||
./bin/migrate -help
|
||
|
||
# 输出：
|
||
# 用法: migrate [命令] [选项]
|
||
#
|
||
# 命令:
|
||
# up 执行所有待执行的迁移（默认）
|
||
# down 回滚最后一个迁移
|
||
# status 查看迁移状态
|
||
#
|
||
# 选项:
|
||
# -config, -c 配置文件路径（默认: config.json）
|
||
# -dir, -d 迁移文件目录（默认: migrations）
|
||
# -help, -h 显示帮助信息
|
||
```
|
||
|
||
### 配置方式
|
||
|
||
#### 方式1：配置文件（推荐开发环境）
|
||
|
||
`config.json`:
|
||
```json
|
||
{
|
||
"database": {
|
||
"type": "mysql",
|
||
"host": "localhost",
|
||
"port": 3306,
|
||
"user": "root",
|
||
"password": "password",
|
||
"database": "mydb"
|
||
}
|
||
}
|
||
```
|
||
|
||
#### 方式2：环境变量（推荐生产环境）
|
||
|
||
```bash
|
||
# 使用 DATABASE_URL（最简单）
|
||
export DATABASE_URL="mysql://root:password@localhost:3306/mydb"
|
||
./bin/migrate up
|
||
|
||
# 覆盖配置路径
|
||
export CONFIG_FILE="/etc/app/config.json"
|
||
export MIGRATIONS_DIR="/opt/app/migrations"
|
||
./bin/migrate up
|
||
```
|
||
|
||
#### 配置优先级（从高到低）
|
||
|
||
1. 命令行参数 `-config` 和 `-dir`（最高）
|
||
2. 环境变量 `CONFIG_FILE` 和 `MIGRATIONS_DIR`
|
||
3. 环境变量 `DATABASE_URL`
|
||
4. 默认值 `config.json` 和 `migrations`
|
||
|
||
---
|
||
|
||
## 🐳 Docker 使用
|
||
|
||
### 方式1：挂载配置文件（推荐）⭐
|
||
|
||
**优势**：修改配置无需重启容器！
|
||
|
||
#### docker-compose.yml
|
||
|
||
```yaml
|
||
version: '3.8'
|
||
|
||
services:
|
||
app:
|
||
build: .
|
||
ports:
|
||
- "8080:8080"
|
||
volumes:
|
||
# 挂载配置文件（推荐：修改配置无需重启容器）
|
||
- ./config.json:/app/config.json:ro
|
||
# 启动时先执行迁移，再启动应用
|
||
command: sh -c "./migrate up && ./server"
|
||
```
|
||
|
||
#### 使用方式
|
||
|
||
```bash
|
||
# 1. 启动服务
|
||
docker-compose up -d
|
||
|
||
# 2. 修改配置文件
|
||
vim config.json
|
||
|
||
# 3. 手动执行迁移（无需重启容器！）
|
||
docker-compose exec app ./migrate up
|
||
|
||
# 4. 查看状态
|
||
docker-compose exec app ./migrate status
|
||
```
|
||
|
||
### 方式2：指定配置文件路径
|
||
|
||
适用于多环境部署：
|
||
|
||
```yaml
|
||
services:
|
||
app:
|
||
build: .
|
||
volumes:
|
||
# 挂载不同环境的配置文件
|
||
- ./config.prod.json:/app/config.json:ro
|
||
command: sh -c "./migrate up -config /app/config.json && ./server"
|
||
```
|
||
|
||
```bash
|
||
# 手动切换环境（修改挂载的配置文件后）
|
||
docker-compose exec app ./migrate up
|
||
```
|
||
|
||
### 方式3：使用环境变量
|
||
|
||
无需配置文件（适用于简单场景）：
|
||
|
||
```yaml
|
||
services:
|
||
app:
|
||
build: .
|
||
environment:
|
||
DATABASE_URL: mysql://root:password@db:3306/mydb
|
||
command: sh -c "./migrate up && ./server"
|
||
|
||
# 注意：DATABASE_URL 的值应该指向你的数据库服务
|
||
# 例如：mysql://user:pass@your-db-host:3306/dbname
|
||
```
|
||
|
||
### Dockerfile
|
||
|
||
```dockerfile
|
||
FROM golang:1.21 as builder
|
||
WORKDIR /app
|
||
COPY go.mod go.sum ./
|
||
RUN go mod download
|
||
COPY . .
|
||
|
||
# 编译应用和迁移工具
|
||
RUN go build -o bin/server cmd/server/main.go
|
||
RUN go build -o bin/migrate cmd/migrate/main.go
|
||
|
||
FROM debian:bookworm-slim
|
||
WORKDIR /app
|
||
RUN apt-get update && apt-get install -y ca-certificates && rm -rf /var/lib/apt/lists/*
|
||
|
||
# 复制二进制文件和迁移文件
|
||
COPY --from=builder /app/bin/migrate .
|
||
COPY --from=builder /app/bin/server .
|
||
COPY --from=builder /app/migrations ./migrations
|
||
|
||
EXPOSE 8080
|
||
|
||
# 启动：先迁移，再启动应用
|
||
CMD ["sh", "-c", "./migrate up && ./server"]
|
||
|
||
# 注意：配置文件通过 volumes 挂载，不打包进镜像
|
||
```
|
||
|
||
---
|
||
|
||
## ☸️ Kubernetes 部署
|
||
|
||
### 使用 Job 执行迁移
|
||
|
||
```yaml
|
||
# k8s-job-migrate.yaml
|
||
apiVersion: batch/v1
|
||
kind: Job
|
||
metadata:
|
||
name: db-migrate
|
||
spec:
|
||
template:
|
||
spec:
|
||
containers:
|
||
- name: migrate
|
||
image: myapp:latest
|
||
command: ["./migrate", "up", "-config", "/etc/config/database.json"]
|
||
volumeMounts:
|
||
- name: config
|
||
mountPath: /etc/config
|
||
readOnly: true
|
||
volumes:
|
||
- name: config
|
||
configMap:
|
||
name: app-config
|
||
restartPolicy: OnFailure
|
||
```
|
||
|
||
### 部署流程
|
||
|
||
```bash
|
||
# 1. 创建 ConfigMap
|
||
kubectl create configmap app-config --from-file=config.json
|
||
|
||
# 2. 执行迁移
|
||
kubectl apply -f k8s-job-migrate.yaml
|
||
kubectl wait --for=condition=complete job/db-migrate
|
||
|
||
# 3. 部署应用
|
||
kubectl apply -f k8s-deployment.yaml
|
||
```
|
||
|
||
---
|
||
|
||
## 🔧 CI/CD 集成
|
||
|
||
### GitLab CI
|
||
|
||
```yaml
|
||
# .gitlab-ci.yml
|
||
stages:
|
||
- build
|
||
- migrate
|
||
- deploy
|
||
|
||
build:
|
||
stage: build
|
||
script:
|
||
- go build -o bin/migrate cmd/migrate/main.go
|
||
- go build -o bin/server cmd/server/main.go
|
||
artifacts:
|
||
paths:
|
||
- bin/
|
||
|
||
migrate:
|
||
stage: migrate
|
||
script:
|
||
- ./bin/migrate up -config config.prod.json
|
||
environment:
|
||
name: production
|
||
|
||
deploy:
|
||
stage: deploy
|
||
script:
|
||
- ./bin/server
|
||
```
|
||
|
||
### GitHub Actions
|
||
|
||
```yaml
|
||
# .github/workflows/deploy.yml
|
||
name: Deploy
|
||
|
||
on:
|
||
push:
|
||
branches: [main]
|
||
|
||
jobs:
|
||
deploy:
|
||
runs-on: ubuntu-latest
|
||
steps:
|
||
- uses: actions/checkout@v3
|
||
|
||
- name: Set up Go
|
||
uses: actions/setup-go@v4
|
||
with:
|
||
go-version: '1.21'
|
||
|
||
- name: Build
|
||
run: |
|
||
go build -o bin/migrate cmd/migrate/main.go
|
||
go build -o bin/server cmd/server/main.go
|
||
|
||
- name: Run Migrations
|
||
env:
|
||
DATABASE_URL: ${{ secrets.DATABASE_URL }}
|
||
run: ./bin/migrate up
|
||
|
||
- name: Deploy
|
||
run: ./bin/server
|
||
```
|
||
|
||
---
|
||
|
||
## 📁 迁移文件管理
|
||
|
||
### 文件命名规则
|
||
|
||
```
|
||
migrations/
|
||
├── 20240101000001_create_users.sql # Up 迁移
|
||
├── 20240101000001_create_users.down.sql # Down 回滚（可选）
|
||
├── 20240102000001_add_posts.sql
|
||
└── 20240102000001_add_posts.down.sql
|
||
```
|
||
|
||
格式：`{时间戳}_{描述}.sql`
|
||
|
||
### 创建迁移文件
|
||
|
||
```bash
|
||
# 获取时间戳
|
||
date +%Y%m%d%H%M%S
|
||
# 输出：20240101120000
|
||
|
||
# 创建迁移文件
|
||
vim migrations/20240101120000_create_posts.sql
|
||
```
|
||
|
||
### 迁移文件示例
|
||
|
||
**Up 文件**：
|
||
```sql
|
||
-- migrations/20240101000001_create_users.sql
|
||
CREATE TABLE users (
|
||
id BIGINT PRIMARY KEY AUTO_INCREMENT,
|
||
username VARCHAR(255) NOT NULL UNIQUE,
|
||
email VARCHAR(255) NOT NULL UNIQUE,
|
||
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
|
||
);
|
||
|
||
CREATE INDEX idx_users_email ON users(email);
|
||
```
|
||
|
||
**Down 文件**（可选）：
|
||
```sql
|
||
-- migrations/20240101000001_create_users.down.sql
|
||
DROP INDEX idx_users_email ON users;
|
||
DROP TABLE IF EXISTS users;
|
||
```
|
||
|
||
### 兼容性建议
|
||
|
||
使用条件语句确保迁移可重复执行：
|
||
|
||
```sql
|
||
-- 创建表
|
||
CREATE TABLE IF NOT EXISTS users (...);
|
||
|
||
-- 添加列
|
||
ALTER TABLE posts ADD COLUMN IF NOT EXISTS author_id BIGINT;
|
||
|
||
-- 创建索引
|
||
CREATE INDEX IF NOT EXISTS idx_posts_author ON posts(author_id);
|
||
```
|
||
|
||
---
|
||
|
||
## 🔍 常见问题
|
||
|
||
### Q: 为什么不直接在应用代码中调用？
|
||
|
||
**A**: **耦合度太高！** 独立工具的优势：
|
||
- ✅ 应用和迁移完全解耦
|
||
- ✅ 可以独立部署和执行
|
||
- ✅ 更灵活的部署策略
|
||
- ✅ 符合单一职责原则
|
||
|
||
### Q: 生产环境没有Go怎么办？
|
||
|
||
**A**: **编译成二进制文件**！
|
||
|
||
```bash
|
||
# 本地或CI中编译
|
||
go build -o bin/migrate cmd/migrate/main.go
|
||
|
||
# 部署二进制文件（不需要Go环境）
|
||
scp bin/migrate server:/app/
|
||
ssh server "/app/migrate up"
|
||
```
|
||
|
||
### Q: Docker中修改配置需要重启吗？
|
||
|
||
**A**: **不需要！** 使用挂载方式：
|
||
|
||
```yaml
|
||
volumes:
|
||
- ./config.json:/app/config.json:ro
|
||
```
|
||
|
||
修改后直接执行：
|
||
```bash
|
||
docker-compose exec app ./migrate up
|
||
```
|
||
|
||
### Q: 如何指定不同的配置文件？
|
||
|
||
**A**: 使用命令行参数：
|
||
|
||
```bash
|
||
# 开发环境
|
||
./migrate up -config config.dev.json
|
||
|
||
# 测试环境
|
||
./migrate up -config config.test.json
|
||
|
||
# 生产环境
|
||
./migrate up -config /etc/app/config.prod.json
|
||
```
|
||
|
||
### Q: 多个实例同时启动会有问题吗？
|
||
|
||
**A**: 不会。数据库会保证只有一个实例能执行迁移（通过版本号主键）。
|
||
|
||
### Q: Docker连不上数据库？
|
||
|
||
**A**: 注意主机名：
|
||
- ❌ `localhost`（容器内无法访问宿主机）
|
||
- ✅ `db`（docker-compose 服务名）
|
||
- ✅ `host.docker.internal`（Mac/Windows 访问宿主机）
|
||
|
||
---
|
||
|
||
## 💡 最佳实践
|
||
|
||
### 1. 开发环境
|
||
|
||
- 使用 `go run` 快速迭代
|
||
- 使用配置文件管理不同环境
|
||
|
||
```bash
|
||
go run cmd/migrate/main.go up -config config.dev.json
|
||
```
|
||
|
||
### 2. 测试环境
|
||
|
||
- 编译后部署，模拟生产环境
|
||
- 使用独立的数据库
|
||
|
||
```bash
|
||
go build -o bin/migrate cmd/migrate/main.go
|
||
./bin/migrate up -config config.test.json
|
||
```
|
||
|
||
### 3. 生产环境
|
||
|
||
- 编译后部署，先执行迁移再启动应用
|
||
- 使用环境变量管理敏感信息
|
||
|
||
```bash
|
||
go build -o bin/migrate cmd/migrate/main.go
|
||
DATABASE_URL="mysql://..." ./bin/migrate up
|
||
./bin/server
|
||
```
|
||
|
||
### 4. Docker 部署
|
||
|
||
- 多阶段构建，只包含二进制文件
|
||
- 挂载配置文件，灵活修改
|
||
|
||
```dockerfile
|
||
FROM golang:1.21 as builder
|
||
RUN go build -o bin/migrate cmd/migrate/main.go
|
||
|
||
FROM debian:bookworm-slim
|
||
COPY --from=builder /app/bin/migrate .
|
||
CMD ["sh", "-c", "./migrate up && ./server"]
|
||
```
|
||
|
||
### 5. CI/CD
|
||
|
||
- 在构建阶段编译
|
||
- 部署前执行迁移
|
||
|
||
```yaml
|
||
- build: go build -o bin/migrate cmd/migrate/main.go
|
||
- migrate: ./bin/migrate up
|
||
- deploy: ./bin/server
|
||
```
|
||
|
||
---
|
||
|
||
## 📊 推荐的项目结构
|
||
|
||
```
|
||
your-project/
|
||
├── cmd/
|
||
│ ├── migrate/
|
||
│ │ └── main.go # 迁移工具（独立）
|
||
│ └── server/
|
||
│ └── main.go # 应用主程序
|
||
├── migrations/ # 迁移SQL文件
|
||
│ ├── 20240101000001_create_users.sql
|
||
│ └── 20240101000001_create_users.down.sql
|
||
├── config.json # 配置文件
|
||
├── Dockerfile
|
||
├── docker-compose.yml
|
||
├── Makefile # 常用命令
|
||
└── go.mod
|
||
```
|
||
|
||
---
|
||
|
||
## 📚 更多资源
|
||
|
||
- [模板文件](./templates/) - 可直接复制的模板
|
||
- [完整文档](./docs/migration.md) - 详细功能文档
|
||
- [配置文档](./docs/config.md) - 配置说明
|
||
|
||
---
|
||
|
||
## 🎯 总结
|
||
|
||
使用 GoCommon 的迁移工具，你可以：
|
||
|
||
1. ✅ 复制一个模板文件到 `cmd/migrate/main.go`
|
||
2. ✅ 创建 SQL 迁移文件到 `migrations/`
|
||
3. ✅ 编译：`go build -o bin/migrate cmd/migrate/main.go`
|
||
4. ✅ 使用：`./bin/migrate up`
|
||
|
||
**核心优势**：
|
||
- 独立工具，零耦合
|
||
- 生产就绪，无需Go环境
|
||
- 灵活配置，支持多环境
|
||
- Docker友好，修改配置无需重启
|
||
|
||
**开箱即用，灵活强大！** 🎉
|
||
|