【go-mysql-elasticsearch记录, 先看第五条】

go-mysql-elasticsearch使用记录：

Windows11 环境下使用 go-mysql-elasticsearch 使得mysql数据导入es：`

---

1. 启动命令

.\go-mysql-elasticsearch.exe -config=D:\soft\go-mysql-elasticsearch\etc\river.toml
指定目录的go-mysql-elasticsearch.exe 加上指定位置的river.toml配置文件
否则会默认使用./ect/river.toml, 大概率是linux的默认, windows难搞

---

2. panic: http: invalid pattern

错误信息

[2022/05/23 14:50:47] [info] binlogsyncer.go:211 register slave for master server 127.0.0.1:3306
panic: http: invalid pattern

goroutine 10 [running]:
net/http.(*ServeMux).Handle(0x1836440, {0x0, 0x0}, {0x1238aa0?, 0xc000094360})
        D:/soft/Go/src/net/http/server.go:2472 +0x24c
net/http.Handle(...)
        D:/soft/Go/src/net/http/server.go:2521
github.com/siddontang/go-mysql-elasticsearch/river.InitStatus({0x0, 0x0}, {0x0, 0x0})
        D:/soft/go-mysql-elasticsearch/river/metrics.go:52 +0x50
created by github.com/siddontang/go-mysql-elasticsearch/river.NewRiver
        D:/soft/go-mysql-elasticsearch/river/river.go:79 +0x465

版本更新后的问题, 配置文件river.toml 添加

# Inner Http status address
stat_addr = "127.0.0.1:12800"
stat_path = "/metrics"

3. canal start sync binlog err

[2022/05/23 15:00:19] [error] canal.go:230 canal start sync binlog err: make insert ES request err table softestimx_prod.bp_project_retrieval has 19 columns, but row data [*****] len is 10, close sync
[2022/05/23 15:00:19] [error] river.go:297 start canal err make insert ES request err table softestimx_prod.bp_project_retrieval has 19 columns, but row data [*****] len is 10, close sync;

es中字段比mysql表中的数据少, es中只有10个, mysql中有19个
解决方法 : 在配置文件river.toml 的[[rule]]下添加

# 仅同步以下列, 注, 此为mysql中 的字段
filter = ["id", "name"]

4. 出现utf8mb3之类的报错

[2022/05/23 15:12:34] [error] canal.go:230 canal start sync binlog err: [parser:1115]Unknown character set: 'utf8mb3'
[2022/05/23 15:12:34] [error] river.go:297 start canal err [parser:1115]Unknown character set: 'utf8mb3'

mysql版本过高, 降到5.X
原文: https://github.com/pingcap/tidb-tools/issues/449

5. 适用版本

go-mysql-elasticsearch作者的备注
MySQL supported version < 8.0
ES supported version < 6.0

mysql版本小于8.0
es版本小于6.0

真真就老六