Who is Using OpenMLDB（欢迎社区成员留言反馈）

[lumianph]

Welcome to OpenMLDB Community

OpenMLDB is not just an opensource project, but also an open community. We are looking forward to hearing from you! We truly appreciate every piece of code, issue, comment and even just an emoji from you. This is because we strongly believe that an opensource project cannot grow without the community's effort and encouragement. Therefore, everyone of you is invited to join our community, and let us know your feedback.

We Want to Hear From You

If you are experiencing issues, or expecting new features, or are just interested in OpenMLDB, please do not hesitate to let us know. You are most welcome to leave your feedback about use cases, experience, or any other comments.

Here are an example of your comment.

Organization: 4Paradigm
Location: Beijing, China
Contact: [email protected]
Business scenario: We are using OpenMLDB for our recommendation systems. OpenMLDB is used to perform online-offline consistent feature extraction for both offline model training and online inference serving.
Suggestions: (We are looking forward to any comments and feedbacks from you!)

We appreciate every contribution from our community members. Your feedback is invaluable and motivates the development team to make OpenMLDB better. Meanwhile, please stay tuned to OpenMLDB's updates and new features. Thanks for your time.

欢迎加入 OpenMLDB 开源社区

OpenMLDB 不仅仅是一个开源项目，更是一个开放的社区。我们非常期待听到来自社区成员的声音。不管是一行代码的贡献，或者是一个 issue，一个留言，甚至仅仅是一个鼓励的表情符，都是 OpenMLDB 社区成长道路上最为珍贵的资产。因为我们坚定的相信，一个开源项目的成长离不开来自社区的反馈和鼓励。因此，今天我们在这里诚挚的邀请每一个社区小伙伴加入这个社区用户列表，倾听来自社区你的声音。

我们期待你的声音

如果你在 OpenMLDB 的过程中碰到了任何问题，或者期待新的功能，或者仅仅是对于 OpenMLDB 感兴趣期望进一步了解，我们期望你在此列表下留言，提供任何的使用场景、经验、反馈等信息。

以下是一个例子

公司：第四范式
地点：中国北京
联系方式：[email protected]
业务场景: 我们使用 OpenMLDB 为推荐系统提供线上和线下一致性的特征工程数据
想法/反馈: (提供对于 OpenMLDB 任何的期望或者意见反馈)

我们非常感激来自每一个社区成员的反馈。你们的意见和建议激励 OpenMLDB 研发团队不断努力，让 OpenMLDB 成长成为一个更具价值的项目。同时，请关注 OpenMLDB 的版本更新，我们会尽力倾听社区的声音，满足来自你的需求。谢谢！

[jessie-zhao]

Organization: Intel
Location: Beijing, China
Contact: [email protected]
Thanks a lot for open source OpenMLDB to provide machine learning feature extraction library, so that developers and enterprise customers can accelerate the development of AI applications. At this year HPC China, 4Paradigm technical experts presented an excellent topic: "Optimization Practice of New Database OpenMLDB Based on Modern Storage Architecture". The audience and our customers were very interested in this topic. We are evaluating how OpenMLDB can help enterprises use AI technology for various applications. Moreover, the audience was especially interested in the Intel PMem optimized OpenMLDB for better TCO and fast recovery. We are looking forward to this PMem based optimization to be released to developers. Thanks a lot

[lumianph]

Thank you for the suggestion. The PMem optimization will be released to the open source community in the near future.

[leochencipher]

Organization: AdvanceAI
Location: Beijing, China
Contact: [email protected]
Thanks for this open source project. After heard of this project from one tech sharing session, we meet with the 4Paradigm technical experts in person to understand more about the process and the underling logic of OpenMLDB. We are currently doing the internal testing of OpenMLDB to perform feature generation for both online serving and offline training cases and how it can fit into our current feature store. Will definitely raise more questions and provide feedbacks in the meantime.

[lumianph]

You are most welcomed! Please do not hesitate to contact us for any issues or suggestions.

[ShuhaoZhangTony]

Organization: Singapore University of Technology and Design
Location: Singapore
Contact: [email protected]
Business scenario: I am an assistant professor from SUTD, focusing on the research topic of data stream processing. We learned this project from this year’s VLDB paper “Optimizing in-memory database engine for AI-powered on-line decision augmentation using persistent memory”. We found that feature engineering systems are very popular and practical in industry for real-world AI applications. However, the research community may have not been fully aware of its value until this paper was published. We think this project may introduce interesting research outcomes on system optimization for feature engineering in the future. At this moment, we are interested in how the real-time feature processing can fit into data stream systems. We are looking forward to a collaboration with this project.

[lumianph]

We are also very interested in this topic. Let's figure out the challenges and solutions together.

[csjgwang]

This is a great project! I'm an assistant professor at Purdue University working on database systems. I got to know the project from their VLDB'21 paper. This project is a great example of how to combine databases and machine learning that I'm interested in. We met with the 4Paradigm team to discuss possible collaborations and I think this project will have a high impact in the future.

[lumianph]

Thank you for the encouragement! I believe your expertise will help further enhance OpenMLDB for more scenarios.

[bingshenghe]

Name: Bingsheng He
Affiliation: National University of Singapore
Location: Singapore
Comments: Open-source computing systems have been very important for reproducibility and experimental impacts in the research community. I am very glad that 4Paradigm has contributed this project to the open-source community. Feature engineering and its related data management tools are essential real-world machine learning applications. Through research collaborations with 4paradigm, I am very glad to be a part of this journey in OpenMLDB. I wish more and more users and developers can join this community to make your contribution in the future.

[lumianph]

Thanks Prof He! We are looking forward to more collaboration with you.

[HongHuangNeu]

Organization: Akulaku
Location: China
Contact: [email protected]
Business scenario: Our most important business scenario is using AI techniques to support financial risk control. In this particular scenario, accuracy and traceability are crucial. The biggest challenge we are facing is to ensure online-offline consistency for both offline model training and online model serving. We are planning to use OpenMLDB to tackle this challenge.

[lumianph]

We are so glad to receive your feedback. please do not hesitate to contact us if you have any questions about OpenMLDB.

[AstorScriabin]

Organization: BasicAI
Location: Chengdu
Contact: [email protected]
Business scenario: we are using OpenMLBD for storing annotation results created by our annotation tool.
Comments: OpenMLBD provides online and offline solutions which covers different deployment requirements. The performance of aggregation and filter looks great. We can summarize and visualize our annotations in a quite efficient way. 4Paradigm people are also wiling to provide supports.
Looking forward to building great tool chain with OpenMLBD.

[lumianph]

Sounds great! Do let us know if you need any help :-)

[yingjunwu]

Name: Yingjun Wu
Affiliation: Singularity Data
Location: Santa Clara, CA
Comments: Really glad to see that 4Paradigm contributes the OpenMLDB project to the open-source community. With OpenMLDB, enterprise users can easily develop, deploy, and maintain their ML models. I hope more contributors could join 4Paradigm in developing this vibrant project, and wish OpenMLDB to solve more real-world problems in the ML domain.

[lumianph]

Thank you for your comments! We are looking forward to our collaboration.

[jdmengxiangtao]

公司：京东科技
地点：中国北京
联系方式：[email protected]
业务场景: 实时特征计算；反欺诈领域应用
想法/反馈:
1.openmldb的整体架构设计非常棒，核心的技术难点解决的非常有先进性
2.针对openmldb做了非标准的压测，整体性能非常好，满足预期
3.希望官方能出个基准版压测报告：吞吐量以及内存占用等关键性指标
4.数据去重的场景，增量计算是否有办法实现

[lumianph]

非常感谢你们的使用测试和反馈！标准的基准性能报告我们正在撰写中，应该会在下个月初和新的版本一起 release，感谢多多关注~

[lumianph]

第一版性能测试报告：https://openmldb.feishu.cn/wiki/wikcnZRB9VRkqgD1vDFu1F9AaTh

[wang990099]

公司：华为
地点：杭州
联系方式：[email protected]
业务场景: 实时特征计算及抽取；搜推业务
想法/反馈:

1. 实时特征计算和高效抽取openmldb有着天然的优势，并且兼容sql语法极大的降低学习成本
2. 对于用户反馈的问题能够及时的响应与支持，在我遇到的在国内的开源项目中热情和贡献是有目共睹的
3. 对于小数据量实时特征抽取一直表现的即为优秀，后期加入多cpu并行处理大数据实时特征抽取也有了很大的提升
4. 文档也相比于之前有了很大的提升，内容更加的详细了
5. 项目编译希望提供单模块，多模块，全量可选择的编译方式。

[lumianph]

非常感谢肯定和支持！对于你反应的目前编译上的一些问题，我们尽快进行优化，希望降低用户部署的使用成本。

[ZuoVJ]

公司：三七互娱
地点：广州
业务场景: 实时特征计算及抽取；离线模型训练特征抽取；
想法/反馈:

• 希望增加数据质量监控，如监控特征计算一致性
• 希望增加特征管理功能，如特征计算方式管理与共享，特征版本，特征组织等，便于快速复用和传递做特征的经验。
• 当前的计算方式更多是一个应用场景对应一个很大的sql去做，不同的应用场景可能会导致相同的特征被重复计算，能不能对于子sql做到抽象和复用呢？

[lumianph]

非常感谢反馈。相关需求已经收到，我们会在近期的roadmap上进行一些设计。再次感谢使用和反馈！

[hzhzh007]

公司：芒果tv
地点：长沙
联系方式：[email protected]
业务场景: 实时特征计算及抽取；搜推业务
想法/反馈:

借助openMLDB来提升生产效率，希望能在这些方面改进：
1.特征治理，大sql 的方式在一定程度上管理还是不是太方便，如果能集成功能，分治成多个单一特征，按需组合，估计会好很多。
2.计算与存储分离问题，很多时候在线特征的拼接是个cpu 密集型的操作，如果和存储耦合得较紧密，在扩容等方便容易出现点问题，如果能中间抽象增加个代理层或胖client，存储与计算分离，对于线上使用会更友好和高效。

[lumianph]

你好，非常感谢反馈！这两方面的内容我们都有在考虑中。其中第一个我们目前正在开发with语法的支持，可以认为是把一个大sql可以拆解成多个小sql的形态，再通过with组合起来；第二个存算分离的架构其实在我们开源前的版本有出现过，但是里面带来了一些额外的overhead，主要openmldb的“存”主要是指内存，这一块我们会再尝试做一些技术上的优化改造。

[lunamage]

公司：值得买
地点：北京
业务场景: 推荐、搜索
想法/反馈:

1. openmldb解决了从0到1特征生成中线上线下一致的问题，不需要两套逻辑，成本也会降低；
2. 目前还在进行功能测试，但是对于怎么将已有特征‘迁移’至openmldb，涉及到各种数据源对接、sql语义是否能满足复杂特征计算、以及上线性能是否能达到预期，还需进一步测试；
3. 对于特征比较多的场景，SQL太大了，是否可以进一步产品化，可以有个特征管理的页面去对接openmldb。

[lumianph]

感谢使用和反馈。确实当SQL比较复杂时，对于SQL的开发以及管理都会有点复杂，我们也在调研看是否有比较直观化的方式对于复杂SQL进行管理。

[aiwtb]

公司：UBiX
地点：中国北京
联系方式：[email protected]
业务场景: 广告CTR/CVR预估，离线特征抽取、实时特征计算
想法/反馈:

1. 希望可以进一步提升特征计算的性能
   （1）离线特征计算：多个窗口构造多个统计特征，存在数据倾斜时，即便开启相关优化项，耗时几个小时还是难以接受；
   （2）实时特征计算：多个窗口构造多个统计特征，开启预聚合优化，还是无法达到10ms以下的预期，如果可以的话，还是希望比10ms再低一些，多留给模型计算一些时间；
   （3）目前的测试情况：单窗口构造两个统计特征和一个组合特征，一天的离线特征10分钟内可以算完，99%的实时特征计算在5ms以下；
2. 希望可以进一步降低使用成本，让特征工程技术普惠化
   （1）在线表不变的前提下增减特征、调整预聚合粒度都需要重新部署SQL，涉及的一些操作会增加使用上的复杂度；
3. 潜在的需求
   （1）因为广告排序前需要预估多条广告创意的CTR/CVR，所以需要计算多条创意的实时特征，OpenMLDB服务端对于实时特征计算的批量请求是否可以做并行处理，以降低整体耗时（对于这个问题，另一种解决方案是客户端做并行请求）；
   （2）在开启预聚合的情况下目前还不支持基于聚合特征做进一步操作：如嵌套其他函数，列的加减乘除等，官方正在开发的UDAF应该可以解决这个问题；
4. 真心希望OpenMLDB可以发展得越来越好，赋能各行各业！

[lumianph]

你好，非常感谢对于OpenMLDB的深度使用和非常有价值的建议！离线性能确实如我们交流，目前由于基于Spark，有一些固有的问题，后面我们尝试对接一个我们内部的自研离线引擎；在线部分目前也还有一些改进的空间。关于重新部署的使用方面，确实现在由于一些底层的设计导致了这方面的复杂化，我们看下能否在用户层面做一些友好化。预聚合功能目前确实还是有很多限制，我们也在逐步完善。UDAF v0.7.0版本没有赶上发布，目前开发已经完成啦。#2825

[zhengxiang5965]

公司：世纪互联
地点：中国北京
联系方式：[email protected]
业务场景: 我们使用 OpenMLDB 为IDC的AI节能应用提供数据处理和算法支持
想法/反馈: 关于服务发布功能是否可以更多方式更灵活一些？

[lumianph]

感谢反馈！服务发布功能我们后面增加一些其他更加灵活的方式，多谢！

[magicheng0816]

公司：soul
地点：北京
业务场景: 推荐
想法/反馈:
觉得好的方面：
1，sql方式做特征抽取，对于算法同学而言会比较友好，开发一个新特征的成本低
2，对于实时序列化的特征，除了能保证强实时外，还提供了很多窗口式的处理操作，算法同学对这类特征的处理会比较方便
3，代码质量有一定的保证，在维护社区，可以及时提供一定的技术支持
期望提升：
1， 对于复杂的特征，sql方式对于同学的要求会比较高，计算复用也依赖人，需要机制上来保障
2，不是一套完整的特征平台解决方案，感觉当前还是聚集在原始数据 + 特征抽取上，而对于特征管理，特征评估，特征复用等能力不具备，完全迁移使用，成本不小，但对于业务的助力感觉有限
3，增加对离线在线原始数据的一致性保障，期待增加样本工程能力

[alexab612]

公司：唯品会
地点：
业务场景: 新业务的搜索推荐场景
想法/反馈:
好的方面:

1. 特征抽取服务很好用,降低了特征开发难度,解除了对于工程侧的部分依赖
2. 提高了特征迭代的速度.

期望提升:

1. 文档完善,对于踩过的坑,可以在文档里多写几句, 避免重复踩坑.
2. 特征监控,特征管理等其他特征平台有的功能
3. 提高autoFe工具的易用性.
4. java的jdk支持windows 平台,因为工程侧的同学在测试联调时使用的是Windows.