Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fastsql 解析 sql语句失败 #2828

Closed
liasica opened this issue Jun 20, 2020 · 9 comments
Closed

fastsql 解析 sql语句失败 #2828

liasica opened this issue Jun 20, 2020 · 9 comments
Assignees
@agapple
Labels
bug
Milestone
v1.1.5

Comments

@liasica
Copy link

liasica commented Jun 20, 2020 

2020-06-21 02:04:31.573 [MultiStageCoprocessor-other-zxcanal-0] WARN  c.a.otter.canal.parse.inbound.mysql.tsdb.MemoryTableMeta - parse faield : CREATE TABLE `articles` (
  `article_id` bigint NOT NULL AUTO_INCREMENT COMMENT '文章id',
  `type` tinyint(1) NOT NULL COMMENT '文章类型',
  `cate` tinyint(1) DEFAULT '0' COMMENT '分类',
  `carousel` tinyint(1) DEFAULT '0' COMMENT '轮播',
  `author_id` bigint NOT NULL COMMENT '小编',
  `teacher_id` int DEFAULT NULL COMMENT '课程讲师',
  `is_fee` tinyint(1) DEFAULT NULL COMMENT '付费文章',
  `title` varchar(120) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL COMMENT '文章标题',
  `name` varchar(40) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL COMMENT '大咖名字',
  `sub_title` varchar(120) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL COMMENT '副标题',
  `summary` varchar(120) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL COMMENT '文章简介',
  `tags` json DEFAULT NULL COMMENT '标签',
  `seo` json DEFAULT NULL COMMENT 'seo关键词',
  `published_at` datetime NOT NULL COMMENT '发布时间',
  `cover` varchar(120) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL COMMENT '封面图',
  `carousel_img` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL COMMENT '轮播图片',
  `act_time` datetime DEFAULT NULL COMMENT '活动时间',
  `address` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL COMMENT '活动地点',
  `body` longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL COMMENT '文章内容',
  `suppliers` json DEFAULT NULL COMMENT '关联服务商',
  `created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
  `updated_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '更新时间',
  PRIMARY KEY (`article_id`),
  KEY `articles_type` (`type`),
  KEY `articles_cate` (`cate`),
  KEY `articles_carousel` (`carousel`),
  KEY `articles_teacher_id` (`teacher_id`),
  KEY `articles_author_id` (`author_id`),
  KEY `articles_published_at` (`published_at`),
  KEY `articles_title` (`title`),
  KEY `articles_tags` ((cast(json_extract(`tags`,_utf8mb4'$[*]') as char(40) array)))
) ENGINE=InnoDB AUTO_INCREMENT=1054 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci COMMENT='文章'
com.alibaba.fastsql.sql.parser.ParserException: syntax error, expect ), actual IDENTIFIER pos 1970, line 32, column 79, token IDENTIFIER array
	at com.alibaba.fastsql.sql.parser.SQLExprParser.accept(SQLExprParser.java:3758) ~[fastsql-2.0.0_preview_973.jar:2.0.0_preview_973]
	at com.alibaba.fastsql.sql.parser.SQLExprParser.primary(SQLExprParser.java:796) ~[fastsql-2.0.0_preview_973.jar:2.0.0_preview_973]
	at com.alibaba.fastsql.sql.dialect.mysql.parser.MySqlExprParser.primary(MySqlExprParser.java:493) ~[fastsql-2.0.0_preview_973.jar:2.0.0_preview_973]
	at com.alibaba.fastsql.sql.parser.SQLExprParser.expr(SQLExprParser.java:97) ~[fastsql-2.0.0_preview_973.jar:2.0.0_preview_973]
	at com.alibaba.fastsql.sql.parser.SQLExprParser.primary(SQLExprParser.java:331) ~[fastsql-2.0.0_preview_973.jar:2.0.0_preview_973]
	at com.alibaba.fastsql.sql.dialect.mysql.parser.MySqlExprParser.primary(MySqlExprParser.java:493) ~[fastsql-2.0.0_preview_973.jar:2.0.0_preview_973]
	at com.alibaba.fastsql.sql.parser.SQLExprParser.expr(SQLExprParser.java:97) ~[fastsql-2.0.0_preview_973.jar:2.0.0_preview_973]
	at com.alibaba.fastsql.sql.parser.SQLExprParser.parseSelectOrderByItem(SQLExprParser.java:2213) ~[fastsql-2.0.0_preview_973.jar:2.0.0_preview_973]
	at com.alibaba.fastsql.sql.parser.SQLExprParser.parseIndexRest(SQLExprParser.java:5378) ~[fastsql-2.0.0_preview_973.jar:2.0.0_preview_973]
	at com.alibaba.fastsql.sql.parser.SQLExprParser.parseIndex(SQLExprParser.java:4360) ~[fastsql-2.0.0_preview_973.jar:2.0.0_preview_973]
	at com.alibaba.fastsql.sql.dialect.mysql.parser.MySqlCreateTableParser.parseConstraint(MySqlCreateTableParser.java:1425) ~[fastsql-2.0.0_preview_973.jar:2.0.0_preview_973]
	at com.alibaba.fastsql.sql.dialect.mysql.parser.MySqlCreateTableParser.parseCreateTable(MySqlCreateTableParser.java:313) ~[fastsql-2.0.0_preview_973.jar:2.0.0_preview_973]
	at com.alibaba.fastsql.sql.dialect.mysql.parser.MySqlStatementParser.parseCreate(MySqlStatementParser.java:291) ~[fastsql-2.0.0_preview_973.jar:2.0.0_preview_973]
	at com.alibaba.fastsql.sql.parser.SQLStatementParser.parseStatementList(SQLStatementParser.java:231) ~[fastsql-2.0.0_preview_973.jar:2.0.0_preview_973]
	at com.alibaba.fastsql.sql.SQLUtils.parseStatements(SQLUtils.java:536) ~[fastsql-2.0.0_preview_973.jar:2.0.0_preview_973]
	at com.alibaba.fastsql.sql.repository.SchemaRepository.console(SchemaRepository.java:439) ~[fastsql-2.0.0_preview_973.jar:2.0.0_preview_973]
	at com.alibaba.otter.canal.parse.inbound.mysql.tsdb.MemoryTableMeta.apply(MemoryTableMeta.java:83) ~[canal.parse-1.1.4.jar:na]
	at com.alibaba.otter.canal.parse.inbound.mysql.dbsync.TableMetaCache.parseTableMeta(TableMetaCache.java:104) [canal.parse-1.1.4.jar:na]
	at com.alibaba.otter.canal.parse.inbound.mysql.dbsync.TableMetaCache.getTableMetaByDB(TableMetaCache.java:90) [canal.parse-1.1.4.jar:na]
	at com.alibaba.otter.canal.parse.inbound.mysql.dbsync.TableMetaCache.access$000(TableMetaCache.java:32) [canal.parse-1.1.4.jar:na]
	at com.alibaba.otter.canal.parse.inbound.mysql.dbsync.TableMetaCache$1.load(TableMetaCache.java:58) [canal.parse-1.1.4.jar:na]
	at com.alibaba.otter.canal.parse.inbound.mysql.dbsync.TableMetaCache$1.load(TableMetaCache.java:53) [canal.parse-1.1.4.jar:na]
	at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3527) [guava-18.0.jar:na]
	at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2319) [guava-18.0.jar:na]
	at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2282) [guava-18.0.jar:na]
	at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2197) [guava-18.0.jar:na]
	at com.google.common.cache.LocalCache.get(LocalCache.java:3937) [guava-18.0.jar:na]
	at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3941) [guava-18.0.jar:na]
	at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4824) [guava-18.0.jar:na]
	at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4830) [guava-18.0.jar:na]
	at com.alibaba.otter.canal.parse.inbound.mysql.dbsync.TableMetaCache.getTableMeta(TableMetaCache.java:195) [canal.parse-1.1.4.jar:na]
	at com.alibaba.otter.canal.parse.inbound.mysql.dbsync.LogEventConvert.getTableMeta(LogEventConvert.java:950) [canal.parse-1.1.4.jar:na]
	at com.alibaba.otter.canal.parse.inbound.mysql.dbsync.LogEventConvert.parseRowsEventForTableMeta(LogEventConvert.java:479) [canal.parse-1.1.4.jar:na]
	at com.alibaba.otter.canal.parse.inbound.mysql.MysqlMultiStageCoprocessor$SimpleParserStage.onEvent(MysqlMultiStageCoprocessor.java:280) [canal.parse-1.1.4.jar:na]
	at com.alibaba.otter.canal.parse.inbound.mysql.MysqlMultiStageCoprocessor$SimpleParserStage.onEvent(MysqlMultiStageCoprocessor.java:246) [canal.parse-1.1.4.jar:na]
	at com.lmax.disruptor.BatchEventProcessor.processEvents(BatchEventProcessor.java:168) [disruptor-3.4.2.jar:na]
	at com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:125) [disruptor-3.4.2.jar:na]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_181]
	at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_181]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_181]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_181]
	at java.lang.Thread.run(Thread.java:748) [na:1.8.0_181]
@ruyin-zh
Copy link

ddl与dcl类型的语句解析是存在问题的,如果不需要可以过滤掉这种类型的语句,一般消费的都是dml语句

@roamerway
Copy link

难道canal不能同步ddl吗？这个解析错误是在哪个阶段出现的？送kafka前，还是adapter拉取kafka数据后？

@liasica
Copy link
Author

liasica commented Jun 29, 2020

难道canal不能同步ddl吗？这个解析错误是在哪个阶段出现的？送kafka前，还是adapter拉取kafka数据后？

我是用canal同步MySQL -> Elasticsearch的，这个解析错误是出现在MySQL表变动之后，canal读取到变动后。

@agapple
Copy link
Member

agapple commented Aug 20, 2020

KEY articles_tags ((cast(json_extract(tags,_utf8mb4'$[*]') as char(40) array)))

这个特殊部分未支持解析 @wenshao

@wenshao
Copy link
Member

wenshao commented Aug 20, 2020

这个mysql数据库是什么版本？

@liasica
Copy link
Author

liasica commented Aug 21, 2020

KEY articles_tags ((cast(json_extract(tags,_utf8mb4'$[*]') as char(40) array)))

这个特殊部分未支持解析 @wenshao

是的，看起来是不支持json字段。

@liasica
Copy link
Author

liasica commented Aug 21, 2020

这个mysql数据库是什么版本？

8.0.12

@YiuTerran
Copy link

我也遇到类似的问题了…希望早点支持

@agapple agapple added this to the v1.1.5 milestone Apr 17, 2021
@agapple agapple added the bug label Apr 17, 2021
@agapple agapple self-assigned this Apr 17, 2021
@agapple
Copy link
Member

agapple commented Apr 19, 2021

升级druid版本为1.2.6后已解决

@agapple agapple closed this as completed Apr 19, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@agapple agapple

Labels
bug
Projects
None yet
Milestone
v1.1.5
Development

No branches or pull requests
6 participants
@liasica @agapple @wenshao @YiuTerran @ruyin-zh @roamerway