调整工具类的方法，优化方法调用及增加迁移工具及其用法
This commit is contained in:
583
MIGRATION.md
Normal file
583
MIGRATION.md
Normal file
@@ -0,0 +1,583 @@
|
||||
# 数据库迁移工具 - 完整指南
|
||||
|
||||
## 📌 核心特点
|
||||
|
||||
- ✅ **独立工具，零耦合** - 与应用代码完全分离
|
||||
- ✅ **生产就绪** - 编译成二进制，无需Go环境
|
||||
- ✅ **灵活配置** - 支持命令行参数、环境变量、配置文件
|
||||
- ✅ **Docker友好** - 挂载配置，修改无需重启容器
|
||||
|
||||
---
|
||||
|
||||
## 🚀 快速开始（3步）
|
||||
|
||||
> **黑盒模式**：迁移工具内部调用 `migration.RunMigrationsFromConfig()` 方法，自动处理配置加载、数据库连接、迁移执行等所有细节。你只需要提供配置文件和SQL文件即可。
|
||||
|
||||
### 1. 复制迁移工具模板
|
||||
|
||||
```bash
|
||||
mkdir -p cmd/migrate
|
||||
cp /path/to/go-common/templates/migrate/main.go cmd/migrate/
|
||||
```
|
||||
|
||||
### 2. 创建迁移文件
|
||||
|
||||
```bash
|
||||
mkdir -p migrations
|
||||
```
|
||||
|
||||
创建 `migrations/20240101000001_create_users.sql`：
|
||||
|
||||
```sql
|
||||
CREATE TABLE users (
|
||||
id BIGINT PRIMARY KEY AUTO_INCREMENT,
|
||||
username VARCHAR(255) NOT NULL,
|
||||
email VARCHAR(255) NOT NULL,
|
||||
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
```
|
||||
|
||||
### 3. 编译和使用
|
||||
|
||||
```bash
|
||||
# 编译（生产环境推荐）
|
||||
go build -o bin/migrate cmd/migrate/main.go
|
||||
|
||||
# 使用
|
||||
./bin/migrate up # 使用默认配置
|
||||
./bin/migrate up -config /path/to/config.json # 指定配置
|
||||
./bin/migrate status # 查看状态
|
||||
./bin/migrate -help # 查看帮助
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 💻 本地使用
|
||||
|
||||
### 开发环境
|
||||
|
||||
```bash
|
||||
# 直接运行（需要Go环境）
|
||||
go run cmd/migrate/main.go up
|
||||
go run cmd/migrate/main.go up -config dev.json
|
||||
|
||||
# 编译后运行（推荐）
|
||||
go build -o bin/migrate cmd/migrate/main.go
|
||||
./bin/migrate up
|
||||
./bin/migrate up -config config.prod.json
|
||||
./bin/migrate up -c prod.json -d db/migrations
|
||||
```
|
||||
|
||||
### 命令说明
|
||||
|
||||
```bash
|
||||
./bin/migrate -help
|
||||
|
||||
# 输出：
|
||||
# 用法: migrate [命令] [选项]
|
||||
#
|
||||
# 命令:
|
||||
# up 执行所有待执行的迁移（默认）
|
||||
# down 回滚最后一个迁移
|
||||
# status 查看迁移状态
|
||||
#
|
||||
# 选项:
|
||||
# -config, -c 配置文件路径（默认: config.json）
|
||||
# -dir, -d 迁移文件目录（默认: migrations）
|
||||
# -help, -h 显示帮助信息
|
||||
```
|
||||
|
||||
### 配置方式
|
||||
|
||||
#### 方式1：配置文件（推荐开发环境）
|
||||
|
||||
`config.json`:
|
||||
```json
|
||||
{
|
||||
"database": {
|
||||
"type": "mysql",
|
||||
"host": "localhost",
|
||||
"port": 3306,
|
||||
"user": "root",
|
||||
"password": "password",
|
||||
"database": "mydb"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### 方式2：环境变量（推荐生产环境）
|
||||
|
||||
```bash
|
||||
# 使用 DATABASE_URL（最简单）
|
||||
export DATABASE_URL="mysql://root:password@localhost:3306/mydb"
|
||||
./bin/migrate up
|
||||
|
||||
# 覆盖配置路径
|
||||
export CONFIG_FILE="/etc/app/config.json"
|
||||
export MIGRATIONS_DIR="/opt/app/migrations"
|
||||
./bin/migrate up
|
||||
```
|
||||
|
||||
#### 配置优先级（从高到低）
|
||||
|
||||
1. 命令行参数 `-config` 和 `-dir`（最高）
|
||||
2. 环境变量 `CONFIG_FILE` 和 `MIGRATIONS_DIR`
|
||||
3. 环境变量 `DATABASE_URL`
|
||||
4. 默认值 `config.json` 和 `migrations`
|
||||
|
||||
---
|
||||
|
||||
## 🐳 Docker 使用
|
||||
|
||||
### 方式1：挂载配置文件（推荐）⭐
|
||||
|
||||
**优势**：修改配置无需重启容器！
|
||||
|
||||
#### docker-compose.yml
|
||||
|
||||
```yaml
|
||||
version: '3.8'
|
||||
|
||||
services:
|
||||
app:
|
||||
build: .
|
||||
ports:
|
||||
- "8080:8080"
|
||||
volumes:
|
||||
# 挂载配置文件（推荐：修改配置无需重启容器）
|
||||
- ./config.json:/app/config.json:ro
|
||||
# 启动时先执行迁移，再启动应用
|
||||
command: sh -c "./migrate up && ./server"
|
||||
```
|
||||
|
||||
#### 使用方式
|
||||
|
||||
```bash
|
||||
# 1. 启动服务
|
||||
docker-compose up -d
|
||||
|
||||
# 2. 修改配置文件
|
||||
vim config.json
|
||||
|
||||
# 3. 手动执行迁移（无需重启容器！）
|
||||
docker-compose exec app ./migrate up
|
||||
|
||||
# 4. 查看状态
|
||||
docker-compose exec app ./migrate status
|
||||
```
|
||||
|
||||
### 方式2：指定配置文件路径
|
||||
|
||||
适用于多环境部署：
|
||||
|
||||
```yaml
|
||||
services:
|
||||
app:
|
||||
build: .
|
||||
volumes:
|
||||
# 挂载不同环境的配置文件
|
||||
- ./config.prod.json:/app/config.json:ro
|
||||
command: sh -c "./migrate up -config /app/config.json && ./server"
|
||||
```
|
||||
|
||||
```bash
|
||||
# 手动切换环境（修改挂载的配置文件后）
|
||||
docker-compose exec app ./migrate up
|
||||
```
|
||||
|
||||
### 方式3：使用环境变量
|
||||
|
||||
无需配置文件（适用于简单场景）：
|
||||
|
||||
```yaml
|
||||
services:
|
||||
app:
|
||||
build: .
|
||||
environment:
|
||||
DATABASE_URL: mysql://root:password@db:3306/mydb
|
||||
command: sh -c "./migrate up && ./server"
|
||||
|
||||
# 注意：DATABASE_URL 的值应该指向你的数据库服务
|
||||
# 例如：mysql://user:pass@your-db-host:3306/dbname
|
||||
```
|
||||
|
||||
### Dockerfile
|
||||
|
||||
```dockerfile
|
||||
FROM golang:1.21 as builder
|
||||
WORKDIR /app
|
||||
COPY go.mod go.sum ./
|
||||
RUN go mod download
|
||||
COPY . .
|
||||
|
||||
# 编译应用和迁移工具
|
||||
RUN go build -o bin/server cmd/server/main.go
|
||||
RUN go build -o bin/migrate cmd/migrate/main.go
|
||||
|
||||
FROM debian:bookworm-slim
|
||||
WORKDIR /app
|
||||
RUN apt-get update && apt-get install -y ca-certificates && rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# 复制二进制文件和迁移文件
|
||||
COPY --from=builder /app/bin/migrate .
|
||||
COPY --from=builder /app/bin/server .
|
||||
COPY --from=builder /app/migrations ./migrations
|
||||
|
||||
EXPOSE 8080
|
||||
|
||||
# 启动：先迁移，再启动应用
|
||||
CMD ["sh", "-c", "./migrate up && ./server"]
|
||||
|
||||
# 注意：配置文件通过 volumes 挂载，不打包进镜像
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ☸️ Kubernetes 部署
|
||||
|
||||
### 使用 Job 执行迁移
|
||||
|
||||
```yaml
|
||||
# k8s-job-migrate.yaml
|
||||
apiVersion: batch/v1
|
||||
kind: Job
|
||||
metadata:
|
||||
name: db-migrate
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
containers:
|
||||
- name: migrate
|
||||
image: myapp:latest
|
||||
command: ["./migrate", "up", "-config", "/etc/config/database.json"]
|
||||
volumeMounts:
|
||||
- name: config
|
||||
mountPath: /etc/config
|
||||
readOnly: true
|
||||
volumes:
|
||||
- name: config
|
||||
configMap:
|
||||
name: app-config
|
||||
restartPolicy: OnFailure
|
||||
```
|
||||
|
||||
### 部署流程
|
||||
|
||||
```bash
|
||||
# 1. 创建 ConfigMap
|
||||
kubectl create configmap app-config --from-file=config.json
|
||||
|
||||
# 2. 执行迁移
|
||||
kubectl apply -f k8s-job-migrate.yaml
|
||||
kubectl wait --for=condition=complete job/db-migrate
|
||||
|
||||
# 3. 部署应用
|
||||
kubectl apply -f k8s-deployment.yaml
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔧 CI/CD 集成
|
||||
|
||||
### GitLab CI
|
||||
|
||||
```yaml
|
||||
# .gitlab-ci.yml
|
||||
stages:
|
||||
- build
|
||||
- migrate
|
||||
- deploy
|
||||
|
||||
build:
|
||||
stage: build
|
||||
script:
|
||||
- go build -o bin/migrate cmd/migrate/main.go
|
||||
- go build -o bin/server cmd/server/main.go
|
||||
artifacts:
|
||||
paths:
|
||||
- bin/
|
||||
|
||||
migrate:
|
||||
stage: migrate
|
||||
script:
|
||||
- ./bin/migrate up -config config.prod.json
|
||||
environment:
|
||||
name: production
|
||||
|
||||
deploy:
|
||||
stage: deploy
|
||||
script:
|
||||
- ./bin/server
|
||||
```
|
||||
|
||||
### GitHub Actions
|
||||
|
||||
```yaml
|
||||
# .github/workflows/deploy.yml
|
||||
name: Deploy
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
|
||||
jobs:
|
||||
deploy:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v3
|
||||
|
||||
- name: Set up Go
|
||||
uses: actions/setup-go@v4
|
||||
with:
|
||||
go-version: '1.21'
|
||||
|
||||
- name: Build
|
||||
run: |
|
||||
go build -o bin/migrate cmd/migrate/main.go
|
||||
go build -o bin/server cmd/server/main.go
|
||||
|
||||
- name: Run Migrations
|
||||
env:
|
||||
DATABASE_URL: ${{ secrets.DATABASE_URL }}
|
||||
run: ./bin/migrate up
|
||||
|
||||
- name: Deploy
|
||||
run: ./bin/server
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📁 迁移文件管理
|
||||
|
||||
### 文件命名规则
|
||||
|
||||
```
|
||||
migrations/
|
||||
├── 20240101000001_create_users.sql # Up 迁移
|
||||
├── 20240101000001_create_users.down.sql # Down 回滚（可选）
|
||||
├── 20240102000001_add_posts.sql
|
||||
└── 20240102000001_add_posts.down.sql
|
||||
```
|
||||
|
||||
格式：`{时间戳}_{描述}.sql`
|
||||
|
||||
### 创建迁移文件
|
||||
|
||||
```bash
|
||||
# 获取时间戳
|
||||
date +%Y%m%d%H%M%S
|
||||
# 输出：20240101120000
|
||||
|
||||
# 创建迁移文件
|
||||
vim migrations/20240101120000_create_posts.sql
|
||||
```
|
||||
|
||||
### 迁移文件示例
|
||||
|
||||
**Up 文件**：
|
||||
```sql
|
||||
-- migrations/20240101000001_create_users.sql
|
||||
CREATE TABLE users (
|
||||
id BIGINT PRIMARY KEY AUTO_INCREMENT,
|
||||
username VARCHAR(255) NOT NULL UNIQUE,
|
||||
email VARCHAR(255) NOT NULL UNIQUE,
|
||||
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
|
||||
CREATE INDEX idx_users_email ON users(email);
|
||||
```
|
||||
|
||||
**Down 文件**（可选）：
|
||||
```sql
|
||||
-- migrations/20240101000001_create_users.down.sql
|
||||
DROP INDEX idx_users_email ON users;
|
||||
DROP TABLE IF EXISTS users;
|
||||
```
|
||||
|
||||
### 兼容性建议
|
||||
|
||||
使用条件语句确保迁移可重复执行：
|
||||
|
||||
```sql
|
||||
-- 创建表
|
||||
CREATE TABLE IF NOT EXISTS users (...);
|
||||
|
||||
-- 添加列
|
||||
ALTER TABLE posts ADD COLUMN IF NOT EXISTS author_id BIGINT;
|
||||
|
||||
-- 创建索引
|
||||
CREATE INDEX IF NOT EXISTS idx_posts_author ON posts(author_id);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔍 常见问题
|
||||
|
||||
### Q: 为什么不直接在应用代码中调用？
|
||||
|
||||
**A**: **耦合度太高！** 独立工具的优势：
|
||||
- ✅ 应用和迁移完全解耦
|
||||
- ✅ 可以独立部署和执行
|
||||
- ✅ 更灵活的部署策略
|
||||
- ✅ 符合单一职责原则
|
||||
|
||||
### Q: 生产环境没有Go怎么办？
|
||||
|
||||
**A**: **编译成二进制文件**！
|
||||
|
||||
```bash
|
||||
# 本地或CI中编译
|
||||
go build -o bin/migrate cmd/migrate/main.go
|
||||
|
||||
# 部署二进制文件（不需要Go环境）
|
||||
scp bin/migrate server:/app/
|
||||
ssh server "/app/migrate up"
|
||||
```
|
||||
|
||||
### Q: Docker中修改配置需要重启吗？
|
||||
|
||||
**A**: **不需要！** 使用挂载方式：
|
||||
|
||||
```yaml
|
||||
volumes:
|
||||
- ./config.json:/app/config.json:ro
|
||||
```
|
||||
|
||||
修改后直接执行：
|
||||
```bash
|
||||
docker-compose exec app ./migrate up
|
||||
```
|
||||
|
||||
### Q: 如何指定不同的配置文件？
|
||||
|
||||
**A**: 使用命令行参数：
|
||||
|
||||
```bash
|
||||
# 开发环境
|
||||
./migrate up -config config.dev.json
|
||||
|
||||
# 测试环境
|
||||
./migrate up -config config.test.json
|
||||
|
||||
# 生产环境
|
||||
./migrate up -config /etc/app/config.prod.json
|
||||
```
|
||||
|
||||
### Q: 多个实例同时启动会有问题吗？
|
||||
|
||||
**A**: 不会。数据库会保证只有一个实例能执行迁移（通过版本号主键）。
|
||||
|
||||
### Q: Docker连不上数据库？
|
||||
|
||||
**A**: 注意主机名：
|
||||
- ❌ `localhost`（容器内无法访问宿主机）
|
||||
- ✅ `db`（docker-compose 服务名）
|
||||
- ✅ `host.docker.internal`（Mac/Windows 访问宿主机）
|
||||
|
||||
---
|
||||
|
||||
## 💡 最佳实践
|
||||
|
||||
### 1. 开发环境
|
||||
|
||||
- 使用 `go run` 快速迭代
|
||||
- 使用配置文件管理不同环境
|
||||
|
||||
```bash
|
||||
go run cmd/migrate/main.go up -config config.dev.json
|
||||
```
|
||||
|
||||
### 2. 测试环境
|
||||
|
||||
- 编译后部署，模拟生产环境
|
||||
- 使用独立的数据库
|
||||
|
||||
```bash
|
||||
go build -o bin/migrate cmd/migrate/main.go
|
||||
./bin/migrate up -config config.test.json
|
||||
```
|
||||
|
||||
### 3. 生产环境
|
||||
|
||||
- 编译后部署，先执行迁移再启动应用
|
||||
- 使用环境变量管理敏感信息
|
||||
|
||||
```bash
|
||||
go build -o bin/migrate cmd/migrate/main.go
|
||||
DATABASE_URL="mysql://..." ./bin/migrate up
|
||||
./bin/server
|
||||
```
|
||||
|
||||
### 4. Docker 部署
|
||||
|
||||
- 多阶段构建，只包含二进制文件
|
||||
- 挂载配置文件，灵活修改
|
||||
|
||||
```dockerfile
|
||||
FROM golang:1.21 as builder
|
||||
RUN go build -o bin/migrate cmd/migrate/main.go
|
||||
|
||||
FROM debian:bookworm-slim
|
||||
COPY --from=builder /app/bin/migrate .
|
||||
CMD ["sh", "-c", "./migrate up && ./server"]
|
||||
```
|
||||
|
||||
### 5. CI/CD
|
||||
|
||||
- 在构建阶段编译
|
||||
- 部署前执行迁移
|
||||
|
||||
```yaml
|
||||
- build: go build -o bin/migrate cmd/migrate/main.go
|
||||
- migrate: ./bin/migrate up
|
||||
- deploy: ./bin/server
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📊 推荐的项目结构
|
||||
|
||||
```
|
||||
your-project/
|
||||
├── cmd/
|
||||
│ ├── migrate/
|
||||
│ │ └── main.go # 迁移工具（独立）
|
||||
│ └── server/
|
||||
│ └── main.go # 应用主程序
|
||||
├── migrations/ # 迁移SQL文件
|
||||
│ ├── 20240101000001_create_users.sql
|
||||
│ └── 20240101000001_create_users.down.sql
|
||||
├── config.json # 配置文件
|
||||
├── Dockerfile
|
||||
├── docker-compose.yml
|
||||
├── Makefile # 常用命令
|
||||
└── go.mod
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📚 更多资源
|
||||
|
||||
- [模板文件](./templates/) - 可直接复制的模板
|
||||
- [完整文档](./docs/migration.md) - 详细功能文档
|
||||
- [配置文档](./docs/config.md) - 配置说明
|
||||
|
||||
---
|
||||
|
||||
## 🎯 总结
|
||||
|
||||
使用 GoCommon 的迁移工具，你可以：
|
||||
|
||||
1. ✅ 复制一个模板文件到 `cmd/migrate/main.go`
|
||||
2. ✅ 创建 SQL 迁移文件到 `migrations/`
|
||||
3. ✅ 编译：`go build -o bin/migrate cmd/migrate/main.go`
|
||||
4. ✅ 使用：`./bin/migrate up`
|
||||
|
||||
**核心优势**：
|
||||
- 独立工具，零耦合
|
||||
- 生产就绪，无需Go环境
|
||||
- 灵活配置，支持多环境
|
||||
- Docker友好，修改配置无需重启
|
||||
|
||||
**开箱即用，灵活强大！** 🎉
|
||||
|
||||
Reference in New Issue
Block a user