Message-Driven,Event-Driven & Streaming 解析

Message-Driven,Event-Driven & Streaming 这三个词一定是近期消息领域高频词,但由于概念过于新,很多同学其实是不太理解这里的异同。正好有个间隙做产品调研,索性把三个概念重新梳理下讲给大家。

三个概念简单翻译过来是这样的:

  • Message-Driven(消息驱动的通信)
  • Event- Driven(事件驱动的通信)
  • Streaming(流模式)

 

首先我们要理解这三个模式都是类似异步通信的模式,发送消息的服务不会等待消费消息服务响应任何数据,做服务解耦是三个模式共同的特性。其次,只要是服务通讯领域内,在选型时还要考虑如下特性:

  1. 排序:是否可以保证的特定的顺序交付
  2. 事务:生产者或消费者是否可以参与分布式事务
  3. 持久化:数据如何被持久化,以及是否可以重放数据
  4. 订阅过滤:是否拥有根据 Tag或其他字段做订阅过滤的能力
  5. At – least -once(最少交付一次),at-most-once(最多交付一次),exactly-once (精确交付)。

通用背景介绍完,我们来依次看看各个模型到底代表的是啥意思。

 

消息驱动 Message-Driven

在消息驱动通信中,一般链路就是消息生产者(Producer)向消息消费者(Consumer)发送消息。模型如下

消息驱动模式下通常会用到中间件,比较常见的中间组件有 RocketMQ,Kafka,AMQP 等。这些中间件的目的是缓存生产者投递的消息直到消费者准备接收这些消息,以此将两端系统解耦。

在消息驱动架构中,消息的格式是基于消费者的需求制定的。消息传递可以是一对一,多对多,一对多或多对一。

消息驱动通讯比较常见的一个例子是商品订单推送,上游组件负责生成订单,下游组件负责接收订单并处理。通过这样的通讯方式上游生成组件其实无需关心整个订单的生命周期,更专注于如何快速生成订单,使单个组件的性能得以提升。

消息驱动模式在服务之间提供了轻的耦合(这部分耦合指代 Producer/Consumer SDK),并可以对生产和消费服务根据诉求进行扩展。

 

事件驱动 Event- Driven

首先要申明一个观点:事件驱动其实是对的消息驱动方法的改进。它对消息体大小,消息格式做了较为严格的限制,这层基于消息的限制封装其实就称为事件(Event)。

在事件驱动模式中,生产者发布事件来表示系统变更,任何感兴趣且有权限接入的服务都可以订阅这些事件,并将这些事件作为触发器来启动某些逻辑/存储/任务。

 

事件驱动的模式可以是一对一,多对一,一对多或多对多。通常情况下一般是多个目标根据过滤条件执行不同的事件。

在事件驱动架构中,事件的格式是由生产者根据事件标准协议制定的。由于更规范限制和封装,事件的生产者完全不需要关心有哪些系统正在消费它生成的事件。

事件不是命令,事件不会告诉消费者如何处理信息,他们的作用只是告诉消费者此时此刻有个事件发生了。事件是一份不可变的数据,重要的数据,它与消息的数据价值相同。通常情况下当某个事件发生并执行时,往往伴随着另一个事件的产生。

事件驱动提供了服务间的最小耦合,并允许生产服务和消费服务根据需求进行扩展。事件驱动可以在不影响现有服务的情况下添加各类新增组件。

事件驱动也可以举一个非常贴切的例子,我们以“客户购买完一款商品”为一个事件,举证在事件场景的应用。

  • CRM (客户关系系统)系统接收到客户购买信息,可自行更新客户的购买记录
  • EMR(库存管理系统) 系统接收到客户购买信息,动态调整库存并及时补货
  • 快递服务接收到客户购买信息,自行打单并通知快递公司派送

这么看,事件驱动模式是不是可以应用并出现在任何地方!

 

流 Streaming 

流是一组有序的无界事件或数据,执行操作通常是固定的某个事件段(E.g. 00:00 – 12:00)或一个相对事件(E.g. 过去12小时)。

通常情况下单个事件往往就是使用事件本身,但是对于流可能的操作大概率是过滤,组合,拆分,映射等等。

流的操作可以是无状态也可以是有状态的:

  • 对于单个事件操作是无状态的,包括过滤和映射
  • 依赖消息在流的时间或位置(E.g.  offset,time)是有状态的。有状态操作中,流处理逻辑必须保留一些已被消费消息的内存。有状态包括对数据做 Batch Size,Batch Window 等。

流这里也可以举一个比较简单的例子,比如我们的物流系统在物品通过一个物流节点时会生成一个事件,但是要查到这个物品完整的流转状态事件,则必须是各个物流节点单个事件的聚合。那这个聚合事件就是流事件。

kafka 是最典型的流式中间件,在流式场景中,事件的位置信息至关重要。通常情况下位置信息(E.g. offset)是由消费者托管的。

 

事件规范标准

聊完 Event 和 Streaming 是什么,我们再来补充一点有关于他们的规范。

事件规范存在的目的是为了清晰事件生产者和消费者的关系,目前主要有两部分:AsyncAPI 和 CloudEvents

AsyncAPI:基于事件API提供了与之对应的 Open API 和 Swagger 等。

CloudEvents:侧重于处理事件的元数据,之前写过一篇文章,可以查阅 https://canmeng.net/c/1009

 

一些拙见,如果表述错误欢迎批评指正。

 



100 thoughts on “Message-Driven,Event-Driven & Streaming 解析”

  • Right here is the perfect web site for everyone who wishes to find out about this topic. You understand a whole lot its almost hard to argue with you (not that I really will need to…HaHa). You certainly put a fresh spin on a subject that’s been discussed for a long time. Great stuff, just wonderful.

  • Thanks for your post. Another element is that being photographer will involve not only difficulty in recording award-winning photographs but hardships in establishing the best photographic camera suited to your needs and most especially situations in maintaining the grade of your camera. This is certainly very real and obvious for those photography fans that are in capturing a nature’s eye-catching scenes — the mountains, the particular forests, the wild or seas. Going to these adventurous places surely requires a photographic camera that can surpass the wild’s severe area.

  • Thanks for your post. One other thing is that often individual states have their own laws of which affect house owners, which makes it very difficult for the the nation’s lawmakers to come up with a new set of rules concerning property foreclosure on property owners. The problem is that each state features own legislation which may have impact in an unwanted manner when it comes to foreclosure policies.

  • I do love the manner in which you have presented this matter and it really does present us a lot of fodder for thought. On the other hand, from what I have personally seen, I just trust when the opinions stack on that people today remain on issue and don’t embark on a soap box of some other news du jour. Yet, thank you for this excellent point and although I do not really concur with it in totality, I regard your standpoint.

  • Thanks for the distinct tips provided on this weblog. I have realized that many insurers offer clients generous savings if they favor to insure several cars together. A significant quantity of households currently have several automobiles these days, particularly people with old teenage kids still living at home, along with the savings upon policies can soon begin. So it pays to look for a great deal.

  • I like the helpful information you provide in your articles. I?ll bookmark your blog and take a look at once more right here regularly. I am quite certain I?ll be informed a lot of new stuff right right here! Good luck for the following!

  • What i do not understood is if truth be told how you’re now not actually much more neatly-appreciated than you might be right now. You’re very intelligent. You recognize therefore significantly when it comes to this subject, made me for my part consider it from a lot of varied angles. Its like women and men don’t seem to be fascinated except it?s one thing to do with Woman gaga! Your individual stuffs nice. At all times care for it up!

  • Many thanks for sharing these kind of wonderful articles. In addition, an excellent travel as well as medical insurance plan can often eliminate those issues that come with traveling abroad. Your medical emergency can soon become too expensive and that’s sure to quickly set a financial stress on the family finances. Having in place the excellent travel insurance package prior to leaving is worth the time and effort. Thanks

  • I’m amazed, I must say. Rarely do I encounter a blog that’s both equally educative and amusing, and without a doubt, you’ve hit the nail on the head. The problem is something which not enough folks are speaking intelligently about. Now i’m very happy that I came across this in my hunt for something regarding this.

  • When I originally commented I appear to have clicked on the -Notify me when new comments are added- checkbox and from now on every time a comment is added I recieve 4 emails with the same comment. Is there an easy method you are able to remove me from that service? Appreciate it.

  • Thanks for your article. One other thing is the fact that individual states in the United states of america have their unique laws that affect householders, which makes it very difficult for the the legislature to come up with a brand new set of recommendations concerning foreclosures on people. The problem is that each state has got own regulations which may interact in a damaging manner in regards to foreclosure plans.

  • I’m usually to blogging and i actually respect your content. The article has really peaks my interest. I’m going to bookmark your website and hold checking for brand new information.

  • I would also love to add that in case you do not currently have an insurance policy or you do not remain in any group insurance, you may well take advantage of seeking assistance from a health broker. Self-employed or people with medical conditions ordinarily seek the help of the health insurance broker. Thanks for your blog post.

  • After I originally commented I appear to have clicked the -Notify me when new comments are added- checkbox and from now on each time a comment is added I get four emails with the exact same comment. There has to be a means you are able to remove me from that service? Thank you.

  • An impressive share! I have just forwarded this onto a friend who has been doing a little homework on this. And he actually ordered me breakfast simply because I found it for him… lol. So let me reword this…. Thank YOU for the meal!! But yeah, thanx for spending some time to discuss this topic here on your website.

  • It is the best time to make some plans for the long run and it’s time to be happy. I’ve learn this put up and if I could I desire to counsel you some interesting things or tips. Perhaps you could write subsequent articles relating to this article. I want to learn even more issues approximately it!

  • I just could not depart your website prior to suggesting that I actually enjoyed the standard info a person provide for your visitors? Is gonna be back often in order to check up on new posts

  • Just about all of what you mention is astonishingly appropriate and it makes me ponder why I hadn’t looked at this in this light previously. This particular article truly did switch the light on for me personally as far as this specific issue goes. But there is just one factor I am not too cozy with and while I make an effort to reconcile that with the actual central theme of your issue, let me see just what all the rest of your readers have to say.Nicely done.

  • One thing I want to say is that often before getting more pc memory, take a look at the machine within which it will be installed. When the machine is usually running Windows XP, for instance, the memory ceiling is 3.25GB. Putting in greater than this would just constitute some sort of waste. Make certain that one’s motherboard can handle this upgrade amount, as well. Great blog post.

  • I just like the valuable info you provide for your articles. I will bookmark your blog and test once more right here regularly. I’m quite sure I will be told plenty of new stuff proper here! Good luck for the next!

  • Thanks for these guidelines. One thing I should also believe is that often credit cards presenting a 0 apr often bait consumers with zero interest, instant acceptance and easy on the web balance transfers, nonetheless beware of the top factor that is going to void that 0 easy road annual percentage rate and also throw anybody out into the terrible house rapid.

  • I have noticed that credit score improvement activity must be conducted with techniques. If not, it’s possible you’ll find yourself causing harm to your standing. In order to be successful in fixing your credit ranking you have to be careful that from this moment in time you pay your complete monthly dues promptly before their appointed date. Really it is significant for the reason that by definitely not accomplishing that, all other steps that you will take to improve your credit rank will not be successful. Thanks for revealing your strategies.

  • Can I simply say what a aid to seek out someone who really is aware of what theyre talking about on the internet. You positively know methods to deliver a problem to light and make it important. Extra folks have to learn this and perceive this side of the story. I cant consider youre not more in style since you undoubtedly have the gift.

  • This is the best weblog for anyone who wants to search out out about this topic. You notice a lot its nearly hard to argue with you (not that I really would want?HaHa). You definitely put a new spin on a subject thats been written about for years. Nice stuff, simply nice!

  • One thing I’d like to comment on is that weightloss program fast is possible by the right diet and exercise. Ones size not only affects the look, but also the complete quality of life. Self-esteem, despression symptoms, health risks, as well as physical ability are affected in fat gain. It is possible to do everything right and at the same time having a gain. If this happens, a condition may be the perpetrator. While an excessive amount of food rather than enough work out are usually accountable, common medical conditions and key prescriptions can greatly amplify size. Thanks a bunch for your post in this article.

  • I would like to thnkx for the efforts you’ve put in writing this web site. I am hoping the same high-grade blog post from you in the upcoming also. In fact your creative writing abilities has encouraged me to get my own web site now. Actually the blogging is spreading its wings quickly. Your write up is a good example of it.

  • Nice post. I be taught something more difficult on totally different blogs everyday. It would all the time be stimulating to learn content from different writers and apply a bit of one thing from their store. I?d favor to use some with the content material on my blog whether you don?t mind. Natually I?ll give you a link in your web blog. Thanks for sharing.

  • I would like to thnkx for the efforts you’ve put in writing this blog. I’m hoping the same high-grade blog post from you in the upcoming as well. In fact your creative writing skills has encouraged me to get my own web site now. Really the blogging is spreading its wings fast. Your write up is a good example of it.

  • Its like you read my mind! You appear to know a lot about this, like you wrote the book in it or something. I think that you could do with a few pics to drive the message home a bit, but instead of that, this is excellent blog. A fantastic read. I will certainly be back.

  • Thanks for your tips. One thing I have noticed is that often banks along with financial institutions really know the spending routines of consumers and also understand that most people max away their own credit cards around the holidays. They sensibly take advantage of this fact and start flooding your current inbox and snail-mail box together with hundreds of no interest APR card offers right after the holiday season finishes. Knowing that if you’re like 98 in the American general public, you’ll rush at the opportunity to consolidate personal credit card debt and move balances towards 0 apr interest rates credit cards.

  • Thanks for your concepts. One thing I’ve noticed is the fact that banks and financial institutions know the dimensions and spending patterns of consumers and as well understand that most of the people max out their real credit cards around the vacations. They smartly take advantage of this particular fact and then start flooding a person’s inbox along with snail-mail box by using hundreds of 0 APR credit card offers just after the holiday season closes. Knowing that for anyone who is like 98 of all American public, you’ll soar at the chance to consolidate personal credit card debt and shift balances to 0 annual percentage rates credit cards.

  • Thanks for your write-up. I would also like to say this that the first thing you will need to perform is verify if you really need credit repair. To do that you will have to get your hands on a copy of your credit file. That should never be difficult, since the government makes it necessary that you are allowed to get one absolutely free copy of your own credit report on a yearly basis. You just have to consult the right individuals. You can either check out the website owned by the Federal Trade Commission or perhaps contact one of the main credit agencies right away.

  • Great article. It is quite unfortunate that over the last ten years, the travel industry has had to take on terrorism, SARS, tsunamis, flu virus, swine flu, along with the first ever real global downturn. Through it the industry has really proven to be solid, resilient and also dynamic, obtaining new ways to deal with adversity. There are continually fresh difficulties and chance to which the sector must all over again adapt and react.

发表评论

邮箱地址不会被公开。 必填项已用*标注

1 × 1 =