Message-Driven,Event-Driven & Streaming 解析

Message-Driven,Event-Driven & Streaming 这三个词一定是近期消息领域高频词,但由于概念过于新,很多同学其实是不太理解这里的异同。正好有个间隙做产品调研,索性把三个概念重新梳理下讲给大家。

三个概念简单翻译过来是这样的:

  • Message-Driven(消息驱动的通信)
  • Event- Driven(事件驱动的通信)
  • Streaming(流模式)

 

首先我们要理解这三个模式都是类似异步通信的模式,发送消息的服务不会等待消费消息服务响应任何数据,做服务解耦是三个模式共同的特性。其次,只要是服务通讯领域内,在选型时还要考虑如下特性:

  1. 排序:是否可以保证的特定的顺序交付
  2. 事务:生产者或消费者是否可以参与分布式事务
  3. 持久化:数据如何被持久化,以及是否可以重放数据
  4. 订阅过滤:是否拥有根据 Tag或其他字段做订阅过滤的能力
  5. At – least -once(最少交付一次),at-most-once(最多交付一次),exactly-once (精确交付)。

通用背景介绍完,我们来依次看看各个模型到底代表的是啥意思。

 

消息驱动 Message-Driven

在消息驱动通信中,一般链路就是消息生产者(Producer)向消息消费者(Consumer)发送消息。模型如下

消息驱动模式下通常会用到中间件,比较常见的中间组件有 RocketMQ,Kafka,AMQP 等。这些中间件的目的是缓存生产者投递的消息直到消费者准备接收这些消息,以此将两端系统解耦。

在消息驱动架构中,消息的格式是基于消费者的需求制定的。消息传递可以是一对一,多对多,一对多或多对一。

消息驱动通讯比较常见的一个例子是商品订单推送,上游组件负责生成订单,下游组件负责接收订单并处理。通过这样的通讯方式上游生成组件其实无需关心整个订单的生命周期,更专注于如何快速生成订单,使单个组件的性能得以提升。

消息驱动模式在服务之间提供了轻的耦合(这部分耦合指代 Producer/Consumer SDK),并可以对生产和消费服务根据诉求进行扩展。

 

事件驱动 Event- Driven

首先要申明一个观点:事件驱动其实是对的消息驱动方法的改进。它对消息体大小,消息格式做了较为严格的限制,这层基于消息的限制封装其实就称为事件(Event)。

在事件驱动模式中,生产者发布事件来表示系统变更,任何感兴趣且有权限接入的服务都可以订阅这些事件,并将这些事件作为触发器来启动某些逻辑/存储/任务。

 

事件驱动的模式可以是一对一,多对一,一对多或多对多。通常情况下一般是多个目标根据过滤条件执行不同的事件。

在事件驱动架构中,事件的格式是由生产者根据事件标准协议制定的。由于更规范限制和封装,事件的生产者完全不需要关心有哪些系统正在消费它生成的事件。

事件不是命令,事件不会告诉消费者如何处理信息,他们的作用只是告诉消费者此时此刻有个事件发生了。事件是一份不可变的数据,重要的数据,它与消息的数据价值相同。通常情况下当某个事件发生并执行时,往往伴随着另一个事件的产生。

事件驱动提供了服务间的最小耦合,并允许生产服务和消费服务根据需求进行扩展。事件驱动可以在不影响现有服务的情况下添加各类新增组件。

事件驱动也可以举一个非常贴切的例子,我们以“客户购买完一款商品”为一个事件,举证在事件场景的应用。

  • CRM (客户关系系统)系统接收到客户购买信息,可自行更新客户的购买记录
  • EMR(库存管理系统) 系统接收到客户购买信息,动态调整库存并及时补货
  • 快递服务接收到客户购买信息,自行打单并通知快递公司派送

这么看,事件驱动模式是不是可以应用并出现在任何地方!

 

流 Streaming 

流是一组有序的无界事件或数据,执行操作通常是固定的某个事件段(E.g. 00:00 – 12:00)或一个相对事件(E.g. 过去12小时)。

通常情况下单个事件往往就是使用事件本身,但是对于流可能的操作大概率是过滤,组合,拆分,映射等等。

流的操作可以是无状态也可以是有状态的:

  • 对于单个事件操作是无状态的,包括过滤和映射
  • 依赖消息在流的时间或位置(E.g.  offset,time)是有状态的。有状态操作中,流处理逻辑必须保留一些已被消费消息的内存。有状态包括对数据做 Batch Size,Batch Window 等。

流这里也可以举一个比较简单的例子,比如我们的物流系统在物品通过一个物流节点时会生成一个事件,但是要查到这个物品完整的流转状态事件,则必须是各个物流节点单个事件的聚合。那这个聚合事件就是流事件。

kafka 是最典型的流式中间件,在流式场景中,事件的位置信息至关重要。通常情况下位置信息(E.g. offset)是由消费者托管的。

 

事件规范标准

聊完 Event 和 Streaming 是什么,我们再来补充一点有关于他们的规范。

事件规范存在的目的是为了清晰事件生产者和消费者的关系,目前主要有两部分:AsyncAPI 和 CloudEvents

AsyncAPI:基于事件API提供了与之对应的 Open API 和 Swagger 等。

CloudEvents:侧重于处理事件的元数据,之前写过一篇文章,可以查阅 https://canmeng.net/c/1009

 

一些拙见,如果表述错误欢迎批评指正。

 



100 thoughts on “Message-Driven,Event-Driven & Streaming 解析”

  • This is very interesting, You’re a very skilled blogger. I’ve joined your feed and look forward to seeking more of your excellent post. Also, I’ve shared your website in my social networks!

  • Good post. I learn something totally new and challenging on sites I stumbleupon everyday. It’s always helpful to read articles from other writers and use something from their web sites.

  • Nice post. I used to be checking continuously this blog and I’m impressed! Extremely useful information particularly the ultimate phase 🙂 I deal with such information much. I used to be seeking this particular info for a very long time. Thank you and best of luck.

  • Hi, There’s no doubt that your web site could possibly be having web browser compatibility problems. When I look at your site in Safari, it looks fine however, when opening in IE, it’s got some overlapping issues. I just wanted to provide you with a quick heads up! Apart from that, excellent blog!

  • I blog often and I genuinely appreciate your information. Your article has truly peaked my interest. I will take a note of your website and keep checking for new details about once a week. I subscribed to your RSS feed too.

  • I have come across that these days, more and more people will be attracted to camcorders and the field of digital photography. However, like a photographer, you must first expend so much time deciding which model of photographic camera to buy in addition to moving from store to store just so you could buy the most economical camera of the trademark you have decided to pick. But it would not end generally there. You also have to contemplate whether you should purchase a digital dslr camera extended warranty. Thanks for the good ideas I gained from your blog site.

  • Interesting article. It’s very unfortunate that over the last decade, the travel industry has already been able to to handle terrorism, SARS, tsunamis, bird flu virus, swine flu, along with the first ever true global economic collapse. Through all this the industry has proven to be sturdy, resilient and also dynamic, getting new solutions to deal with misfortune. There are generally fresh troubles and chance to which the marketplace must once again adapt and reply.

  • Thanks for some other wonderful post. Where else could anyone get that type of info in such an ideal way of writing? I’ve a presentation next week, and I’m at the look for such information.

  • Youre so cool! I dont suppose Ive learn anything like this before. So good to find someone with some authentic thoughts on this subject. realy thanks for beginning this up. this web site is one thing that is needed on the net, somebody with just a little originality. helpful job for bringing something new to the web!

  • Its like you learn my mind! You appear to grasp a lot about this, like you wrote the ebook in it or something. I believe that you could do with some to pressure the message home a bit, but instead of that, that is fantastic blog. An excellent read. I’ll certainly be back.

  • Hi there, just became aware of your blog through Google, and found that it is really informative. I am going to watch out for brussels. I will be grateful if you continue this in future. Many people will be benefited from your writing. Cheers!

  • Great beat ! I wish to apprentice at the same time as you amend your web site, how can i subscribe for a weblog site? The account helped me a applicable deal. I have been tiny bit acquainted of this your broadcast provided brilliant clear idea

  • Today, considering the fast life style that everyone is having, credit cards get this amazing demand throughout the economy. Persons from every field are using credit card and people who are not using the card have made arrangements to apply for even one. Thanks for sharing your ideas on credit cards.

  • Definitely consider that that you stated. Your favorite reason seemed to be at the internet the simplest factor to take into account of. I say to you, I definitely get annoyed at the same time as other folks think about worries that they plainly don’t recognize about. You managed to hit the nail upon the top and outlined out the entire thing without having side effect , folks could take a signal. Will probably be again to get more. Thank you

  • Hi my friend! I want to say that this post is awesome, nice written and include almost all vital infos. I would like to look more posts like this .

  • Hi there, just became aware of your blog through Google, and located that it is truly informative. I am gonna watch out for brussels. I will appreciate in case you proceed this in future. A lot of folks will likely be benefited from your writing. Cheers!

  • I like what you guys are up also. Such smart work and reporting! Carry on the excellent works guys I have incorporated you guys to my blogroll. I think it will improve the value of my web site 🙂

  • I have learned quite a few important things through your post. I would also like to state that there might be situation in which you will have a loan and never need a co-signer such as a National Student Aid Loan. When you are getting that loan through a standard financier then you need to be able to have a cosigner ready to allow you to. The lenders will certainly base any decision on a few aspects but the greatest will be your credit rating. There are some loan merchants that will also look at your job history and make a decision based on that but in most cases it will be based on on your credit score.

  • In these days of austerity and also relative anxiety about taking on debt, lots of people balk about the idea of having a credit card in order to make acquisition of merchandise and also pay for a trip, preferring, instead just to rely on this tried and trusted approach to making settlement – cash. However, in case you have the cash there to make the purchase 100 , then, paradoxically, this is the best time for you to use the credit cards for several causes.

  • Thanks for the strategies presented. One thing I additionally believe is that credit cards presenting a 0 monthly interest often appeal to consumers together with zero interest rate, instant acceptance and easy internet balance transfers, nonetheless beware of the top factor that can void your current 0 easy road annual percentage rate and throw one out into the terrible house quick.

  • I?ve been exploring for a bit for any high quality articles or weblog posts in this kind of house . Exploring in Yahoo I finally stumbled upon this web site. Reading this info So i am happy to show that I have an incredibly just right uncanny feeling I found out exactly what I needed. I so much unquestionably will make sure to do not fail to remember this website and provides it a look on a continuing basis.

  • Thanks for discussing your ideas. I might also like to convey that video games have been actually evolving. Better technology and innovative developments have aided create sensible and interactive games. All these entertainment games were not actually sensible when the actual concept was first of all being attempted. Just like other designs of technological know-how, video games as well have had to progress by many generations. This itself is testimony for the fast progression of video games.

  • I do like the manner in which you have framed this problem and it does provide me some fodder for consideration. Nonetheless, through just what I have observed, I just hope when the feedback stack on that folks keep on point and not embark on a soap box of some other news du jour. Yet, thank you for this superb piece and although I can not necessarily concur with this in totality, I respect your point of view.

  • Great – I should certainly pronounce, impressed with your website. I had no trouble navigating through all the tabs and related info ended up being truly easy to do to access. I recently found what I hoped for before you know it at all. Reasonably unusual. Is likely to appreciate it for those who add forums or something, site theme . a tones way for your client to communicate. Excellent task.

  • 100,000 Backlinks only $10,Subject to data from ahrefs.com.After paid $10 by PayPal (My PayPal:helloboy1979@gmail.com),Tell me your URL, email and comment content.I will complete the task within ten days.But It may take up to a month for the data updated from ahrefs.com.

  • Great weblog right here! Additionally your website a lot up fast! What host are you the usage of? Can I am getting your associate link on your host? I desire my website loaded up as quickly as yours lol|

  • Hey are using WordPress for your blog platform? I’m new to the blog world but I’m trying to get started and create my own. Do you need any coding expertise to make your own blog? Any help would be really appreciated!|

发表评论

邮箱地址不会被公开。 必填项已用*标注

− 1 = 1