Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG]alwaysFallback odps-jdbc加速回退参数不生效 #155

Open
hnssyggg opened this issue Aug 23, 2024 · 0 comments
Open

[BUG]alwaysFallback odps-jdbc加速回退参数不生效 #155

hnssyggg opened this issue Aug 23, 2024 · 0 comments
Assignees
@lyman
Labels
bug

Comments

@hnssyggg
Copy link

hnssyggg commented Aug 23, 2024 •
edited
Loading

alwaysFallback参数在odps-jdbc中不管设置为false还是true，都走了回退，和odps客户端现象不一样

jdk-11.0.11
com.aliyun.odps
odps-jdbc
3.2.29和3.6.0

1、设置 private static final String jdbcUrl = "jdbc:odps:" + endpoint +
"?project=" + projectName +
"&accessId=" + accessKeyId +
"&accessKey=" + accessKeySecret +
"&charset=UTF-8" +
"&interactiveMode=true" + // 启用 MCQA 功能
"&alwaysFallback=true" +
"&enableOdpsLogger=true" +
"&autoSelectLimit=1000000000"; // 配置大数据量查询限制
执行日志如下

8月 23, 2024 5:40:12 下午 com.aliyun.odps.jdbc.utils.OdpsLogger info
信息: [connection-1] ODPS JDBC driver, Version 3.6.0
8月 23, 2024 5:40:12 下午 com.aliyun.odps.jdbc.utils.OdpsLogger info
信息: [connection-1] endpoint=http://service.cn-beijing.maxcompute.aliyun.com/api, project=dp_pdm_risk_prod, schema=null
8月 23, 2024 5:40:12 下午 com.aliyun.odps.jdbc.utils.OdpsLogger info
信息: [connection-1] JVM timezone : Asia/Shanghai
8月 23, 2024 5:40:12 下午 com.aliyun.odps.jdbc.utils.OdpsLogger info
信息: [connection-1] charset=UTF-8, logview host=null
8月 23, 2024 5:40:13 下午 com.aliyun.odps.jdbc.utils.OdpsLogger info
信息: [connection-1] Project timezone: Asia/Shanghai
8月 23, 2024 5:40:13 下午 com.aliyun.odps.jdbc.utils.OdpsLogger info
信息: [connection-1] load project meta infos time cost=359
8月 23, 2024 5:40:19 下午 com.aliyun.odps.jdbc.utils.OdpsLogger info
信息: [connection-1] Attach success, instanceId:2024082309401998gcgpjzky1uv3, attach and get tunnel endpoint time cost=5731
8月 23, 2024 5:40:19 下午 com.aliyun.odps.jdbc.utils.OdpsLogger info
信息: [connection-1] Connect to odps project dp_pdm_risk_prod successfully
8月 23, 2024 5:40:19 下午 com.aliyun.odps.jdbc.utils.OdpsLogger info
信息: [connection-1] Enabled SQL task properties: {odps.idata.useragent=JDBC-Version:3.6.0 SDK-Version:0.48.7-public}
8月 23, 2024 5:40:19 下午 com.aliyun.odps.jdbc.utils.OdpsLogger info
信息: [connection-1] Run SQL: select 资信名称2（int_fae_des） as 资信名称2（int_fae_des）__0, sum(case when (datediff(current_date,调用时间（crt_at）))=0 then 计费量（chg_cnt） else 0 end) as T日调用__nul__0, sum(case when (datediff(current_date,调用时间（crt_at）))=1 then 计费量（chg_cnt） else 0 end) as T-1日调用__nul__1, sum(case when (datediff(current_date,调用时间（crt_at）))=2 then 计费量（chg_cnt） else 0 end) as T-2日调用__nul__2, sum(case when (datediff(current_date,调用时间（crt_at）))=3 then 计费量（chg_cnt） else 0 end) as T-3日调用__nul__3, sum(case when (datediff(current_date,调用时间（crt_at）))=8 then 计费量（chg_cnt） else 0 end) as T-8日调用__nul__4, case when (sum(case when (datediff(current_date,调用时间（crt_at）))=1 then 计费量（chg_cnt） else 0 end)) <> 0
then
cast((sum(case when (datediff(current_date,调用时间（crt_at）))=0 then 计费量（chg_cnt） else 0 end)) as double)/cast((sum(case when (datediff(current_date,调用时间（crt_at）))=1 then 计费量（chg_cnt） else 0 end)) as double)
else (sum(case when (datediff(current_date,调用时间（crt_at）))=0 then 计费量（chg_cnt） else 0 end))
end as T日环比__nul__5, case when (sum(case when (datediff(current_date,调用时间（crt_at）))=2 then 计费量（chg_cnt） else 0 end)) <> 0
then
cast((sum(case when (datediff(current_date,调用时间（crt_at）))=1 then 计费量（chg_cnt） else 0 end)) as double)/cast((sum(case when (datediff(current_date,调用时间（crt_at）))=2 then 计费量（chg_cnt） else 0 end)) as double)
else (sum(case when (datediff(current_date,调用时间（crt_at）))=1 then 计费量（chg_cnt） else 0 end))
end as T-1日环比__nul__6, case when (sum(case when (datediff(current_date,调用时间（crt_at）))=3 then 计费量（chg_cnt） else 0 end)) <> 0
then
cast((sum(case when (datediff(current_date,调用时间（crt_at）))=2 then 计费量（chg_cnt） else 0 end)) as double)/cast((sum(case when (datediff(current_date,调用时间（crt_at）))=3 then 计费量（chg_cnt） else 0 end)) as double)
else (sum(case when (datediff(current_date,调用时间（crt_at）))=2 then 计费量（chg_cnt） else 0 end))
end as T-2日环比__nul__7, case when (sum(case when (datediff(current_date,调用时间（crt_at）))=8 then 计费量（chg_cnt） else 0 end)) <> 0
then
cast((sum(case when (datediff(current_date,调用时间（crt_at）))=1 then 计费量（chg_cnt） else 0 end)) as double)/cast((sum(case when (datediff(current_date,调用时间（crt_at）))=8 then 计费量（chg_cnt） else 0 end)) as double)
else (sum(case when (datediff(current_date,调用时间（crt_at）))=1 then 计费量（chg_cnt） else 0 end))
end as T-1日同比__nul__8, max(调用时间（crt_at）) as 调用时间（crt_at）__nul__9 from (
select CAST(cdt_nam AS string) AS 资信编码（cdt_nam）
,CAST(cdt_des AS string) AS 资信名称（cdt_des）
,CAST(spl_nam AS string) AS 供应商编码（spl_nam）
,CAST(spl_des AS string) AS 供应商名称（spl_des）
,CAST(int_fae_cod AS string) AS 资信编码2（int_fae_cod）
,CAST(int_fae_des AS string) AS 资信名称2（int_fae_des）
,CAST(tkn_id AS string) AS token_id（tkn_id）
,CAST(typ AS string) AS token属性（typ）
,CAST(bel_dep AS string) AS token归属（bel_dep）
,CAST(sce AS string) AS 场景名称（sce）
,CAST(cnt AS DOUBLE) AS 查询量（cnt）
,CAST(suc_cnt AS DOUBLE) AS 成功量（suc_cnt）
,CAST(chg_cnt AS DOUBLE) AS 计费量（chg_cnt）
,CAST(crt_at AS string) AS 调用时间（crt_at）
,CAST(cdt_cos AS DOUBLE) AS 计算单价（cdt_cos）
,CAST(cdt_nam1 AS string) AS 分类一级（cdt_nam1）
,CAST(cdt_nam2 AS string) AS 分类二级（cdt_nam2）
,CAST(cdt_nam3 AS string) AS 分类三级（cdt_nam3）
,CAST(cdt_nam4 AS string) AS 分类四级（cdt_nam4）
,CAST(ds AS string) AS （ds）
from pdm_risk_f_credit_by_scene_code_di
where ds>='20220701'
) as table_real group by 资信名称2（int_fae_des） having case when (sum(case when (datediff(current_date,调用时间（crt_at）))=2 then 计费量（chg_cnt） else 0 end)) <> 0
then
cast((sum(case when (datediff(current_date,调用时间（crt_at）))=1 then 计费量（chg_cnt） else 0 end)) as double)/cast((sum(case when (datediff(current_date,调用时间（crt_at）))=2 then 计费量（chg_cnt） else 0 end)) as double)
else (sum(case when (datediff(current_date,调用时间（crt_at）))=1 then 计费量（chg_cnt） else 0 end))
end >= 1.3 order by 调用时间（crt_at）__nul__9 desc, T-1日环比__nul__6 desc limit 10000;, Begin time: 1724406019569
8月 23, 2024 5:40:20 下午 com.aliyun.odps.jdbc.utils.OdpsLogger info
信息: [connection-1] InstanceId: 2024082309401998gcgpjzky1uv3
8月 23, 2024 5:40:21 下午 com.aliyun.odps.jdbc.utils.OdpsLogger info
信息: [connection-1] LogView: 省略.......................................
8月 23, 2024 5:40:43 下午 com.aliyun.odps.jdbc.utils.OdpsLogger warn
警告: [connection-1] Session execution log:Running in interactive mode, RetryCount: 0, QueryId:0
Log view:
8月 23, 2024 5:40:43 下午 com.aliyun.odps.jdbc.utils.OdpsLogger warn
警告: [connection-1] Session execution log:
8月 23, 2024 5:40:43 下午 com.aliyun.odps.jdbc.utils.OdpsLogger warn
警告: [connection-1] Session execution log:Query failed:RequestId=20240823174021a0cfdc0b090c2d12, ErrorCode=TaskFailed, ErrorMessage=ODPS-1830001: Resource not enough - ODPS-0130151: Too many instance count(1046), exceeds threshold limit 500.ODPS-1800001: Session exception - ODPS-0130151: Too many instance count(1046), exceeds threshold limit 500.
8月 23, 2024 5:40:43 下午 com.aliyun.odps.jdbc.utils.OdpsLogger warn
警告: [connection-1] Session execution log:Will fallback to offline mode
8月 23, 2024 5:40:43 下午 com.aliyun.odps.jdbc.utils.OdpsLogger warn
警告: [connection-1] Session execution log:Running in offline mode, RetryCount: 1, QueryId:0
Log view:
8月 23, 2024 5:40:43 下午 com.aliyun.odps.jdbc.utils.OdpsLogger warn
警告: [connection-1] Session execution log:.省略.......................................
8月 23, 2024 5:40:43 下午 com.aliyun.odps.jdbc.utils.OdpsLogger info
信息: [connection-1] It took me 23759 ms to run sql, instanceId: 2024082309401998gcgpjzky1uv3
8月 23, 2024 5:40:44 下午 com.aliyun.odps.jdbc.utils.OdpsLogger info
信息: [connection-1] It took me 483 ms to fetch all records, count:4
8月 23, 2024 5:40:44 下午 com.aliyun.odps.jdbc.utils.OdpsLogger info
信息: [connection-1] the statement has been closed
8月 23, 2024 5:40:44 下午 com.aliyun.odps.jdbc.utils.OdpsLogger info
信息: [connection-1] connection closed

2、设置 private static final String jdbcUrl = "jdbc:odps:" + endpoint +
"?project=" + projectName +
"&accessId=" + accessKeyId +
"&accessKey=" + accessKeySecret +
"&charset=UTF-8" +
"&interactiveMode=true" + // 启用 MCQA 功能
"&alwaysFallback=false" +
"&enableOdpsLogger=true" +
"&autoSelectLimit=1000000000"; // 配置大数据量查询限制
执行日志如下

8月 23, 2024 5:42:08 下午 com.aliyun.odps.jdbc.utils.OdpsLogger info
信息: [connection-1] ODPS JDBC driver, Version 3.6.0
8月 23, 2024 5:42:08 下午 com.aliyun.odps.jdbc.utils.OdpsLogger info
信息: [connection-1] endpoint=http://service.cn-beijing.maxcompute.aliyun.com/api, project=dp_pdm_risk_prod, schema=null
8月 23, 2024 5:42:08 下午 com.aliyun.odps.jdbc.utils.OdpsLogger info
信息: [connection-1] JVM timezone : Asia/Shanghai
8月 23, 2024 5:42:08 下午 com.aliyun.odps.jdbc.utils.OdpsLogger info
信息: [connection-1] charset=UTF-8, logview host=null
8月 23, 2024 5:42:09 下午 com.aliyun.odps.jdbc.utils.OdpsLogger info
信息: [connection-1] Project timezone: Asia/Shanghai
8月 23, 2024 5:42:09 下午 com.aliyun.odps.jdbc.utils.OdpsLogger info
信息: [connection-1] load project meta infos time cost=512
8月 23, 2024 5:42:15 下午 com.aliyun.odps.jdbc.utils.OdpsLogger info
信息: [connection-1] Attach success, instanceId:20240823094214923gkry28r68q4, attach and get tunnel endpoint time cost=5712
8月 23, 2024 5:42:15 下午 com.aliyun.odps.jdbc.utils.OdpsLogger info
信息: [connection-1] Connect to odps project dp_pdm_risk_prod successfully
8月 23, 2024 5:42:15 下午 com.aliyun.odps.jdbc.utils.OdpsLogger info
信息: [connection-1] Enabled SQL task properties: {odps.idata.useragent=JDBC-Version:3.6.0 SDK-Version:0.48.7-public}
8月 23, 2024 5:42:15 下午 com.aliyun.odps.jdbc.utils.OdpsLogger info
信息: [connection-1] Run SQL: select 资信名称2（int_fae_des） as 资信名称2（int_fae_des）__0, sum(case when (datediff(current_date,调用时间（crt_at）))=0 then 计费量（chg_cnt） else 0 end) as T日调用__nul__0, sum(case when (datediff(current_date,调用时间（crt_at）))=1 then 计费量（chg_cnt） else 0 end) as T-1日调用__nul__1, sum(case when (datediff(current_date,调用时间（crt_at）))=2 then 计费量（chg_cnt） else 0 end) as T-2日调用__nul__2, sum(case when (datediff(current_date,调用时间（crt_at）))=3 then 计费量（chg_cnt） else 0 end) as T-3日调用__nul__3, sum(case when (datediff(current_date,调用时间（crt_at）))=8 then 计费量（chg_cnt） else 0 end) as T-8日调用__nul__4, case when (sum(case when (datediff(current_date,调用时间（crt_at）))=1 then 计费量（chg_cnt） else 0 end)) <> 0
then
cast((sum(case when (datediff(current_date,调用时间（crt_at）))=0 then 计费量（chg_cnt） else 0 end)) as double)/cast((sum(case when (datediff(current_date,调用时间（crt_at）))=1 then 计费量（chg_cnt） else 0 end)) as double)
else (sum(case when (datediff(current_date,调用时间（crt_at）))=0 then 计费量（chg_cnt） else 0 end))
end as T日环比__nul__5, case when (sum(case when (datediff(current_date,调用时间（crt_at）))=2 then 计费量（chg_cnt） else 0 end)) <> 0
then
cast((sum(case when (datediff(current_date,调用时间（crt_at）))=1 then 计费量（chg_cnt） else 0 end)) as double)/cast((sum(case when (datediff(current_date,调用时间（crt_at）))=2 then 计费量（chg_cnt） else 0 end)) as double)
else (sum(case when (datediff(current_date,调用时间（crt_at）))=1 then 计费量（chg_cnt） else 0 end))
end as T-1日环比__nul__6, case when (sum(case when (datediff(current_date,调用时间（crt_at）))=3 then 计费量（chg_cnt） else 0 end)) <> 0
then
cast((sum(case when (datediff(current_date,调用时间（crt_at）))=2 then 计费量（chg_cnt） else 0 end)) as double)/cast((sum(case when (datediff(current_date,调用时间（crt_at）))=3 then 计费量（chg_cnt） else 0 end)) as double)
else (sum(case when (datediff(current_date,调用时间（crt_at）))=2 then 计费量（chg_cnt） else 0 end))
end as T-2日环比__nul__7, case when (sum(case when (datediff(current_date,调用时间（crt_at）))=8 then 计费量（chg_cnt） else 0 end)) <> 0
then
cast((sum(case when (datediff(current_date,调用时间（crt_at）))=1 then 计费量（chg_cnt） else 0 end)) as double)/cast((sum(case when (datediff(current_date,调用时间（crt_at）))=8 then 计费量（chg_cnt） else 0 end)) as double)
else (sum(case when (datediff(current_date,调用时间（crt_at）))=1 then 计费量（chg_cnt） else 0 end))
end as T-1日同比__nul__8, max(调用时间（crt_at）) as 调用时间（crt_at）__nul__9 from (
select CAST(cdt_nam AS string) AS 资信编码（cdt_nam）
,CAST(cdt_des AS string) AS 资信名称（cdt_des）
,CAST(spl_nam AS string) AS 供应商编码（spl_nam）
,CAST(spl_des AS string) AS 供应商名称（spl_des）
,CAST(int_fae_cod AS string) AS 资信编码2（int_fae_cod）
,CAST(int_fae_des AS string) AS 资信名称2（int_fae_des）
,CAST(tkn_id AS string) AS token_id（tkn_id）
,CAST(typ AS string) AS token属性（typ）
,CAST(bel_dep AS string) AS token归属（bel_dep）
,CAST(sce AS string) AS 场景名称（sce）
,CAST(cnt AS DOUBLE) AS 查询量（cnt）
,CAST(suc_cnt AS DOUBLE) AS 成功量（suc_cnt）
,CAST(chg_cnt AS DOUBLE) AS 计费量（chg_cnt）
,CAST(crt_at AS string) AS 调用时间（crt_at）
,CAST(cdt_cos AS DOUBLE) AS 计算单价（cdt_cos）
,CAST(cdt_nam1 AS string) AS 分类一级（cdt_nam1）
,CAST(cdt_nam2 AS string) AS 分类二级（cdt_nam2）
,CAST(cdt_nam3 AS string) AS 分类三级（cdt_nam3）
,CAST(cdt_nam4 AS string) AS 分类四级（cdt_nam4）
,CAST(ds AS string) AS （ds）
from pdm_risk_f_credit_by_scene_code_di
where ds>='20220701'
) as table_real group by 资信名称2（int_fae_des） having case when (sum(case when (datediff(current_date,调用时间（crt_at）))=2 then 计费量（chg_cnt） else 0 end)) <> 0
then
cast((sum(case when (datediff(current_date,调用时间（crt_at）))=1 then 计费量（chg_cnt） else 0 end)) as double)/cast((sum(case when (datediff(current_date,调用时间（crt_at）))=2 then 计费量（chg_cnt） else 0 end)) as double)
else (sum(case when (datediff(current_date,调用时间（crt_at）))=1 then 计费量（chg_cnt） else 0 end))
end >= 1.3 order by 调用时间（crt_at）__nul__9 desc, T-1日环比__nul__6 desc limit 10000;, Begin time: 1724406135363
8月 23, 2024 5:42:16 下午 com.aliyun.odps.jdbc.utils.OdpsLogger info
信息: [connection-1] InstanceId: 20240823094214923gkry28r68q4
8月 23, 2024 5:42:17 下午 com.aliyun.odps.jdbc.utils.OdpsLogger info
信息: [connection-1] LogView: 省略.......................................
8月 23, 2024 5:42:34 下午 com.aliyun.odps.jdbc.utils.OdpsLogger warn
警告: [connection-1] Session execution log:Running in interactive mode, RetryCount: 0, QueryId:0
Log view:
8月 23, 2024 5:42:34 下午 com.aliyun.odps.jdbc.utils.OdpsLogger warn
警告: [connection-1] Session execution log:
8月 23, 2024 5:42:34 下午 com.aliyun.odps.jdbc.utils.OdpsLogger warn
警告: [connection-1] Session execution log:Query failed:RequestId=20240823174217d2f2dc0b097e0471, ErrorCode=TaskFailed, ErrorMessage=ODPS-1830001: Resource not enough - ODPS-0130151: Too many instance count(1046), exceeds threshold limit 500.ODPS-1800001: Session exception - ODPS-0130151: Too many instance count(1046), exceeds threshold limit 500.
8月 23, 2024 5:42:34 下午 com.aliyun.odps.jdbc.utils.OdpsLogger warn
警告: [connection-1] Session execution log:Will fallback to offline mode
8月 23, 2024 5:42:34 下午 com.aliyun.odps.jdbc.utils.OdpsLogger warn
警告: [connection-1] Session execution log:Running in offline mode, RetryCount: 1, QueryId:0
Log view:
8月 23, 2024 5:42:34 下午 com.aliyun.odps.jdbc.utils.OdpsLogger warn
警告: [connection-1] Session execution log:省略.......................................
8月 23, 2024 5:42:34 下午 com.aliyun.odps.jdbc.utils.OdpsLogger info
信息: [connection-1] It took me 19322 ms to run sql, instanceId: 20240823094214923gkry28r68q4
8月 23, 2024 5:42:35 下午 com.aliyun.odps.jdbc.utils.OdpsLogger info
信息: [connection-1] It took me 409 ms to fetch all records, count:4
8月 23, 2024 5:42:35 下午 com.aliyun.odps.jdbc.utils.OdpsLogger info
信息: [connection-1] the statement has been closed
8月 23, 2024 5:42:35 下午 com.aliyun.odps.jdbc.utils.OdpsLogger info
信息: [connection-1] connection closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@lyman lyman

Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@lyman @hnssyggg