Message-Driven,Event-Driven & Streaming 解析

Message-Driven,Event-Driven & Streaming 这三个词一定是近期消息领域高频词,但由于概念过于新,很多同学其实是不太理解这里的异同。正好有个间隙做产品调研,索性把三个概念重新梳理下讲给大家。

三个概念简单翻译过来是这样的:

  • Message-Driven(消息驱动的通信)
  • Event- Driven(事件驱动的通信)
  • Streaming(流模式)

 

首先我们要理解这三个模式都是类似异步通信的模式,发送消息的服务不会等待消费消息服务响应任何数据,做服务解耦是三个模式共同的特性。其次,只要是服务通讯领域内,在选型时还要考虑如下特性:

  1. 排序:是否可以保证的特定的顺序交付
  2. 事务:生产者或消费者是否可以参与分布式事务
  3. 持久化:数据如何被持久化,以及是否可以重放数据
  4. 订阅过滤:是否拥有根据 Tag或其他字段做订阅过滤的能力
  5. At – least -once(最少交付一次),at-most-once(最多交付一次),exactly-once (精确交付)。

通用背景介绍完,我们来依次看看各个模型到底代表的是啥意思。

 

消息驱动 Message-Driven

在消息驱动通信中,一般链路就是消息生产者(Producer)向消息消费者(Consumer)发送消息。模型如下

消息驱动模式下通常会用到中间件,比较常见的中间组件有 RocketMQ,Kafka,AMQP 等。这些中间件的目的是缓存生产者投递的消息直到消费者准备接收这些消息,以此将两端系统解耦。

在消息驱动架构中,消息的格式是基于消费者的需求制定的。消息传递可以是一对一,多对多,一对多或多对一。

消息驱动通讯比较常见的一个例子是商品订单推送,上游组件负责生成订单,下游组件负责接收订单并处理。通过这样的通讯方式上游生成组件其实无需关心整个订单的生命周期,更专注于如何快速生成订单,使单个组件的性能得以提升。

消息驱动模式在服务之间提供了轻的耦合(这部分耦合指代 Producer/Consumer SDK),并可以对生产和消费服务根据诉求进行扩展。

 

事件驱动 Event- Driven

首先要申明一个观点:事件驱动其实是对的消息驱动方法的改进。它对消息体大小,消息格式做了较为严格的限制,这层基于消息的限制封装其实就称为事件(Event)。

在事件驱动模式中,生产者发布事件来表示系统变更,任何感兴趣且有权限接入的服务都可以订阅这些事件,并将这些事件作为触发器来启动某些逻辑/存储/任务。

 

事件驱动的模式可以是一对一,多对一,一对多或多对多。通常情况下一般是多个目标根据过滤条件执行不同的事件。

在事件驱动架构中,事件的格式是由生产者根据事件标准协议制定的。由于更规范限制和封装,事件的生产者完全不需要关心有哪些系统正在消费它生成的事件。

事件不是命令,事件不会告诉消费者如何处理信息,他们的作用只是告诉消费者此时此刻有个事件发生了。事件是一份不可变的数据,重要的数据,它与消息的数据价值相同。通常情况下当某个事件发生并执行时,往往伴随着另一个事件的产生。

事件驱动提供了服务间的最小耦合,并允许生产服务和消费服务根据需求进行扩展。事件驱动可以在不影响现有服务的情况下添加各类新增组件。

事件驱动也可以举一个非常贴切的例子,我们以“客户购买完一款商品”为一个事件,举证在事件场景的应用。

  • CRM (客户关系系统)系统接收到客户购买信息,可自行更新客户的购买记录
  • EMR(库存管理系统) 系统接收到客户购买信息,动态调整库存并及时补货
  • 快递服务接收到客户购买信息,自行打单并通知快递公司派送

这么看,事件驱动模式是不是可以应用并出现在任何地方!

 

流 Streaming 

流是一组有序的无界事件或数据,执行操作通常是固定的某个事件段(E.g. 00:00 – 12:00)或一个相对事件(E.g. 过去12小时)。

通常情况下单个事件往往就是使用事件本身,但是对于流可能的操作大概率是过滤,组合,拆分,映射等等。

流的操作可以是无状态也可以是有状态的:

  • 对于单个事件操作是无状态的,包括过滤和映射
  • 依赖消息在流的时间或位置(E.g.  offset,time)是有状态的。有状态操作中,流处理逻辑必须保留一些已被消费消息的内存。有状态包括对数据做 Batch Size,Batch Window 等。

流这里也可以举一个比较简单的例子,比如我们的物流系统在物品通过一个物流节点时会生成一个事件,但是要查到这个物品完整的流转状态事件,则必须是各个物流节点单个事件的聚合。那这个聚合事件就是流事件。

kafka 是最典型的流式中间件,在流式场景中,事件的位置信息至关重要。通常情况下位置信息(E.g. offset)是由消费者托管的。

 

事件规范标准

聊完 Event 和 Streaming 是什么,我们再来补充一点有关于他们的规范。

事件规范存在的目的是为了清晰事件生产者和消费者的关系,目前主要有两部分:AsyncAPI 和 CloudEvents

AsyncAPI:基于事件API提供了与之对应的 Open API 和 Swagger 等。

CloudEvents:侧重于处理事件的元数据,之前写过一篇文章,可以查阅 https://canmeng.net/c/1009

 

一些拙见,如果表述错误欢迎批评指正。

 



800 thoughts on “Message-Driven,Event-Driven & Streaming 解析”

  • Hello there, I found your web site via Google while searching for a related topic, your website came up, it looks good. I’ve bookmarked it in my google bookmarks.

  • One other thing is that an online business administration training is designed for students to be able to efficiently proceed to bachelor degree education. The Ninety credit certification meets the other bachelor college degree requirements and when you earn your own associate of arts in BA online, you will have access to the most up-to-date technologies with this field. Several reasons why students need to get their associate degree in business is because they’re interested in the field and want to obtain the general instruction necessary prior to jumping in to a bachelor education program. Thx for the tips you provide in your blog.

  • Heya i am for the first time here. I found this board and I find It really useful & it helped me out much. I’m hoping to present something again and aid others such as you aided me.

  • Thanks for your post here. One thing I would really like to say is most professional job areas consider the Bachelor’s Degree like thejust like the entry level standard for an online college diploma. Although Associate Diplomas are a great way to start, completing the Bachelors uncovers many doors to various occupations, there are numerous on-line Bachelor Diploma Programs available through institutions like The University of Phoenix, Intercontinental University Online and Kaplan. Another issue is that many brick and mortar institutions offer Online variants of their college diplomas but normally for a greatly higher fee than the companies that specialize in online college diploma plans.

  • I have come across that nowadays, more and more people are being attracted to digital cameras and the discipline of images. However, as a photographer, you have to first expend so much period deciding the exact model of dslr camera to buy as well as moving via store to store just so you can buy the most inexpensive camera of the brand you have decided to decide on. But it will not end right now there. You also have take into consideration whether you can purchase a digital camera extended warranty. Thanks alot : ) for the good ideas I accumulated from your blog.

  • One more thing. In my opinion that there are lots of travel insurance web pages of respected companies that allow you enter your vacation details and have you the quotes. You can also purchase an international travel cover policy on internet by using your own credit card. All that you should do is always to enter all travel specifics and you can start to see the plans side-by-side. Merely find the system that suits your budget and needs after which it use your bank credit card to buy the item. Travel insurance online is a good way to do investigation for a trustworthy company with regard to international holiday insurance. Thanks for revealing your ideas.

  • One thing is that often one of the most prevalent incentives for making use of your credit card is a cash-back as well as rebate present. Generally, you get 1-5 back in various buying. Depending on the card, you may get 1 returning on most acquisitions, and 5 back on expenditures made going to convenience stores, filling stations, grocery stores plus ‘member merchants’.

  • I additionally believe that mesothelioma cancer is a exceptional form of melanoma that is often found in all those previously familiar with asbestos. Cancerous cellular material form inside the mesothelium, which is a protective lining that covers the vast majority of body’s organs. These cells typically form in the lining of the lungs, abdominal area, or the sac which actually encircles the heart. Thanks for revealing your ideas.

  • Throughout this great scheme of things you’ll secure a B- just for effort and hard work. Where exactly you actually confused everybody was first in your specifics. As people say, details make or break the argument.. And that could not be more accurate in this article. Having said that, allow me say to you exactly what did do the job. The text is definitely pretty engaging and that is most likely why I am making the effort to comment. I do not really make it a regular habit of doing that. 2nd, although I can certainly notice a leaps in reason you come up with, I am not really confident of how you seem to connect your ideas which inturn help to make your conclusion. For now I will subscribe to your issue however trust in the near future you connect the facts better.

  • I do agree with all the ideas you’ve presented in your post. They are very convincing and will certainly work. Still, the posts are very short for starters. Could you please extend them a bit from next time? Thanks for the post.

  • Thanks for the thoughts you are discussing on this blog site. Another thing I would like to say is always that getting hold of duplicates of your credit rating in order to examine accuracy of each detail could be the first activity you have to carry out in credit restoration. You are looking to clear your credit profile from harmful details errors that screw up your credit score.

  • certainly like your web-site but you need to check the spelling on several of your posts. A number of them are rife with spelling issues and I in finding it very bothersome to tell the reality nevertheless I will surely come again again.

  • Aw, this was a really nice post. In concept I would like to put in writing like this additionally ? taking time and actual effort to make a very good article? but what can I say? I procrastinate alot and not at all seem to get something done.

  • I have acquired some new issues from your site about pc’s. Another thing I have always imagined is that laptop computers have become something that each home must have for several reasons. They offer convenient ways to organize homes, pay bills, shop, study, hear music and even watch television shows. An innovative technique to complete these types of tasks is to use a laptop computer. These desktops are mobile, small, powerful and lightweight.

  • I have been surfing online greater than 3 hours nowadays, but I by no means discovered any interesting article like yours. It is beautiful price enough for me. Personally, if all site owners and bloggers made excellent content material as you probably did, the internet will likely be a lot more helpful than ever before.

  • There are some fascinating time limits in this article but I don?t know if I see all of them heart to heart. There may be some validity however I will take hold opinion till I look into it further. Good article , thanks and we wish extra! Added to FeedBurner as well

  • Thanks for the useful information on credit repair on all of this web-site. The things i would offer as advice to people is to give up the mentality they can buy today and shell out later. As a society many of us tend to repeat this for many factors. This includes getaways, furniture, in addition to items we wish. However, you need to separate the wants out of the needs. While you’re working to fix your credit score make some trade-offs. For example you are able to shop online to economize or you can check out second hand shops instead of highly-priced department stores regarding clothing.

  • One thing I would like to discuss is that weightloss routine fast may be accomplished by the perfect diet and exercise. A person’s size not simply affects appearance, but also the entire quality of life. Self-esteem, depressive disorder, health risks, in addition to physical capabilities are impacted in weight gain. It is possible to do everything right but still gain. If this happens, a problem may be the primary cause. While excessive food but not enough workout are usually responsible, common medical ailments and popular prescriptions could greatly increase size. Kudos for your post in this article.

  • Hiya, I’m really glad I’ve found this information. Nowadays bloggers publish just about gossips and web and this is really irritating. A good site with interesting content, that is what I need. Thanks for keeping this web site, I will be visiting it. Do you do newsletters? Can not find it.

  • One important thing is that when you’re searching for a education loan you may find that you’ll need a cosigner. There are many cases where this is correct because you may find that you do not have a past credit ranking so the bank will require that you’ve got someone cosign the financing for you. Interesting post.

  • We are a group of volunteers and starting a new scheme in our community. Your site offered us with valuable information to work on. You have done a formidable job and our whole community will be grateful to you.

  • This is very interesting, You are a very skilled blogger. I’ve joined your rss feed and look forward to seeking more of your wonderful post. Also, I’ve shared your site in my social networks!

  • Oh my goodness! a tremendous article dude. Thank you However I’m experiencing concern with ur rss . Don?t know why Unable to subscribe to it. Is there anyone getting an identical rss drawback? Anyone who is aware of kindly respond. Thnkx

  • Its like you read my thoughts! You seem to know a lot approximately this, such as you wrote the ebook in it or something. I feel that you could do with some to pressure the message home a bit, however other than that, this is magnificent blog. A great read. I’ll definitely be back.

  • Thanks for the thoughts you have shared here. In addition, I believe there are a few factors which really keep your automobile insurance premium straight down. One is, to consider buying autos that are inside good list of car insurance companies. Cars that happen to be expensive tend to be at risk of being snatched. Aside from that insurance policies are also based on the value of your vehicle, so the more pricey it is, then higher the particular premium you only pay.

  • Thanks for the tips you have discussed here. Something important I would like to state is that computer memory needs generally rise along with other advances in the technological know-how. For instance, if new generations of processors are brought to the market, there is usually a corresponding increase in the size and style calls for of both computer memory and also hard drive space. This is because the program operated by simply these cpus will inevitably increase in power to make new engineering.

  • Hi, Neat post. There’s a problem with your website in internet explorer, would check this? IE still is the market leader and a huge portion of people will miss your excellent writing due to this problem.

  • Thanks for revealing your ideas. I’d also like to say that video games have been actually evolving. Modern technology and innovative developments have made it easier to create genuine and enjoyable games. These kinds of entertainment games were not as sensible when the real concept was first of all being tried. Just like other areas of technological innovation, video games too have had to evolve by many many years. This is testimony on the fast growth and development of video games.

  • It is in reality a nice and helpful piece of info. I?m satisfied that you shared this helpful information with us. Please keep us informed like this. Thank you for sharing.

  • Thanks for the suggestions shared on your own blog. One more thing I would like to mention is that fat loss is not information about going on a fad diet and trying to shed as much weight that you can in a couple of weeks. The most effective way to shed weight is by getting it slowly and obeying some basic suggestions which can enable you to make the most through your attempt to drop some weight. You may realize and already be following a few of these tips, although reinforcing knowledge never hurts.

  • I believe that avoiding processed foods may be the first step to be able to lose weight. They may taste excellent, but prepared foods have got very little nutritional value, making you consume more just to have enough electricity to get through the day. If you’re constantly having these foods, converting to grain and other complex carbohydrates will let you have more energy while having less. Great blog post.

  • Thank you for some other informative site. Where else may I am getting that kind of information written in such an ideal way? I’ve a mission that I am simply now running on, and I have been at the glance out for such info.

  • I discovered more new things on this weight reduction issue. A single issue is a good nutrition is especially vital any time dieting. A big reduction in junk food, sugary meals, fried foods, sugary foods, pork, and white colored flour products may be necessary. Holding wastes bloodsuckers, and wastes may prevent goals for shedding fat. While specified drugs for the short term solve the issue, the nasty side effects usually are not worth it, plus they never present more than a short-term solution. It can be a known incontrovertible fact that 95 of diet plans fail. Many thanks for sharing your thinking on this blog site.

  • Aw, this was a very nice post. In idea I want to put in writing like this moreover ? taking time and precise effort to make an excellent article? but what can I say? I procrastinate alot and in no way appear to get one thing done.

发表评论

邮箱地址不会被公开。 必填项已用*标注

23 + = 27