How do event sourced systems guarrantee reliability in the event delivery

Since in event-sourcing the event store does not use transactions, how can we guarantee that, if our business logic crashes, after it publishes an event, that the event won't be published twice when restarting the service?

In the case that the message is published and delivered twice, how can it be de-duplicated?

asked Nov 21 '18 at 13:11

tkiwi

537

add a comment |

In the case that the message is published and delivered twice, how can it be de-duplicated?

asked Nov 21 '18 at 13:11

tkiwi

537

add a comment |

In the case that the message is published and delivered twice, how can it be de-duplicated?

asked Nov 21 '18 at 13:11

tkiwi

537

In the case that the message is published and delivered twice, how can it be de-duplicated?

messaging event-sourcing

asked Nov 21 '18 at 13:11

tkiwi

537

asked Nov 21 '18 at 13:11

tkiwi

537

asked Nov 21 '18 at 13:11

tkiwi

537

asked Nov 21 '18 at 13:11

tkiwi

537

asked Nov 21 '18 at 13:11

tkiwi

537

add a comment |

1 Answer
1

active

oldest

votes

I don't know that the event store does not use transactions. I've seen transactional writes to ensure that the expected event version is being written.

If you are expecting at-least-once delivery, which I assume you are, then you must handle deduplication. It is sometimes recommended to maintain an index of all processed messages as a de-duping approach, but it is not completely safe in that you just minimize the section of code in which a duplicate can be created, but don't remove the possibility altogether. If you process a message, but do not update the index then you will reprocess the same message again. You should instead make all of your actions idempotent. That is, performing the same action twice will produce the same resulting state. If you process the same message twice, it should only update state once.

answered Nov 21 '18 at 15:38

CPerson

3366

Thanks. I have an additional question. I understand how the business logic should be idempotent. One problem that arises is that the emitted events will need to have new id's. If the id's are randomly generated however, which is the easiest solution, the emitted events won't be the same. How is this problem solved? Should we for example generate the id's based on a hashing algorithm, in an idempotent way as well?

– tkiwi
Dec 3 '18 at 13:54

1

@tkiwi Because the business logic is idempotent, you actually don't generate new events. Say you issue two commands to change a customer's name to "Bob". The first time it gets processed, the name is changed and an event is raised, CustomerNameChanged("Bob"). The second time it is processed, the name is already what it should be and so you don't raise an event. The same applies if a single command is processed twice for some reason.

– CPerson
Dec 6 '18 at 15:53

What if I issuing a command to change a customer's name to Bob CustomerNameChanged("Bob"), triggers another event based on my business logic, e.g. there's a check whether Bob has friends, which in turn, emits NotifyFriendsForNameModification(...). Other example, e.g. there's a limit on daily name changes, so we can't allow any further changes for some time and we emit an event NameModificationDisabled("Bob"). Hoping that my examples are making sense, when NotifyFriendsForNameModification is emitted let's say the service crashes so we retry. Could an event be emitted twice like that?

– tkiwi
Dec 20 '18 at 12:00

I think I understand your question. CustomerNameChanged triggers a process or saga which then sends a command, NotifyFriendsForNameModification back to the Customer. This would raise FriendsNotifiedForNameModification event containing a list of friends who should be notified. A process or saga would take that list and generate a ModifyFriendName command for each friend. Since commands are Idempotent, each friend will acknowledge the change once and only once, raising a FriendNameModified event. Idempotent commands are key here. Does that answer the question?

– CPerson
Dec 20 '18 at 19:09

add a comment |

Your Answer

StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});

}
});

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53412832%2fhow-do-event-sourced-systems-guarrantee-reliability-in-the-event-delivery%23new-answer', 'question_page');
}
);

Post as a guest

Name

Required, but never shown

1 Answer
1

active

oldest

votes

1 Answer
1

active

oldest

votes

I don't know that the event store does not use transactions. I've seen transactional writes to ensure that the expected event version is being written.

answered Nov 21 '18 at 15:38

CPerson

3366

Thanks. I have an additional question. I understand how the business logic should be idempotent. One problem that arises is that the emitted events will need to have new id's. If the id's are randomly generated however, which is the easiest solution, the emitted events won't be the same. How is this problem solved? Should we for example generate the id's based on a hashing algorithm, in an idempotent way as well?

– tkiwi
Dec 3 '18 at 13:54

1

@tkiwi Because the business logic is idempotent, you actually don't generate new events. Say you issue two commands to change a customer's name to "Bob". The first time it gets processed, the name is changed and an event is raised, CustomerNameChanged("Bob"). The second time it is processed, the name is already what it should be and so you don't raise an event. The same applies if a single command is processed twice for some reason.

– CPerson
Dec 6 '18 at 15:53

What if I issuing a command to change a customer's name to Bob CustomerNameChanged("Bob"), triggers another event based on my business logic, e.g. there's a check whether Bob has friends, which in turn, emits NotifyFriendsForNameModification(...). Other example, e.g. there's a limit on daily name changes, so we can't allow any further changes for some time and we emit an event NameModificationDisabled("Bob"). Hoping that my examples are making sense, when NotifyFriendsForNameModification is emitted let's say the service crashes so we retry. Could an event be emitted twice like that?

– tkiwi
Dec 20 '18 at 12:00

I think I understand your question. CustomerNameChanged triggers a process or saga which then sends a command, NotifyFriendsForNameModification back to the Customer. This would raise FriendsNotifiedForNameModification event containing a list of friends who should be notified. A process or saga would take that list and generate a ModifyFriendName command for each friend. Since commands are Idempotent, each friend will acknowledge the change once and only once, raising a FriendNameModified event. Idempotent commands are key here. Does that answer the question?

– CPerson
Dec 20 '18 at 19:09

add a comment |

I don't know that the event store does not use transactions. I've seen transactional writes to ensure that the expected event version is being written.

answered Nov 21 '18 at 15:38

CPerson

3366

Thanks. I have an additional question. I understand how the business logic should be idempotent. One problem that arises is that the emitted events will need to have new id's. If the id's are randomly generated however, which is the easiest solution, the emitted events won't be the same. How is this problem solved? Should we for example generate the id's based on a hashing algorithm, in an idempotent way as well?

– tkiwi
Dec 3 '18 at 13:54

1

@tkiwi Because the business logic is idempotent, you actually don't generate new events. Say you issue two commands to change a customer's name to "Bob". The first time it gets processed, the name is changed and an event is raised, CustomerNameChanged("Bob"). The second time it is processed, the name is already what it should be and so you don't raise an event. The same applies if a single command is processed twice for some reason.

– CPerson
Dec 6 '18 at 15:53

What if I issuing a command to change a customer's name to Bob CustomerNameChanged("Bob"), triggers another event based on my business logic, e.g. there's a check whether Bob has friends, which in turn, emits NotifyFriendsForNameModification(...). Other example, e.g. there's a limit on daily name changes, so we can't allow any further changes for some time and we emit an event NameModificationDisabled("Bob"). Hoping that my examples are making sense, when NotifyFriendsForNameModification is emitted let's say the service crashes so we retry. Could an event be emitted twice like that?

– tkiwi
Dec 20 '18 at 12:00

I think I understand your question. CustomerNameChanged triggers a process or saga which then sends a command, NotifyFriendsForNameModification back to the Customer. This would raise FriendsNotifiedForNameModification event containing a list of friends who should be notified. A process or saga would take that list and generate a ModifyFriendName command for each friend. Since commands are Idempotent, each friend will acknowledge the change once and only once, raising a FriendNameModified event. Idempotent commands are key here. Does that answer the question?

– CPerson
Dec 20 '18 at 19:09

add a comment |

I don't know that the event store does not use transactions. I've seen transactional writes to ensure that the expected event version is being written.

answered Nov 21 '18 at 15:38

CPerson

3366

I don't know that the event store does not use transactions. I've seen transactional writes to ensure that the expected event version is being written.

answered Nov 21 '18 at 15:38

CPerson

3366

answered Nov 21 '18 at 15:38

CPerson

3366

answered Nov 21 '18 at 15:38

CPerson

3366

answered Nov 21 '18 at 15:38

CPerson

3366

Thanks. I have an additional question. I understand how the business logic should be idempotent. One problem that arises is that the emitted events will need to have new id's. If the id's are randomly generated however, which is the easiest solution, the emitted events won't be the same. How is this problem solved? Should we for example generate the id's based on a hashing algorithm, in an idempotent way as well?

– tkiwi
Dec 3 '18 at 13:54

1

@tkiwi Because the business logic is idempotent, you actually don't generate new events. Say you issue two commands to change a customer's name to "Bob". The first time it gets processed, the name is changed and an event is raised, CustomerNameChanged("Bob"). The second time it is processed, the name is already what it should be and so you don't raise an event. The same applies if a single command is processed twice for some reason.

– CPerson
Dec 6 '18 at 15:53

What if I issuing a command to change a customer's name to Bob CustomerNameChanged("Bob"), triggers another event based on my business logic, e.g. there's a check whether Bob has friends, which in turn, emits NotifyFriendsForNameModification(...). Other example, e.g. there's a limit on daily name changes, so we can't allow any further changes for some time and we emit an event NameModificationDisabled("Bob"). Hoping that my examples are making sense, when NotifyFriendsForNameModification is emitted let's say the service crashes so we retry. Could an event be emitted twice like that?

– tkiwi
Dec 20 '18 at 12:00

I think I understand your question. CustomerNameChanged triggers a process or saga which then sends a command, NotifyFriendsForNameModification back to the Customer. This would raise FriendsNotifiedForNameModification event containing a list of friends who should be notified. A process or saga would take that list and generate a ModifyFriendName command for each friend. Since commands are Idempotent, each friend will acknowledge the change once and only once, raising a FriendNameModified event. Idempotent commands are key here. Does that answer the question?

– CPerson
Dec 20 '18 at 19:09

add a comment |

Thanks. I have an additional question. I understand how the business logic should be idempotent. One problem that arises is that the emitted events will need to have new id's. If the id's are randomly generated however, which is the easiest solution, the emitted events won't be the same. How is this problem solved? Should we for example generate the id's based on a hashing algorithm, in an idempotent way as well?

– tkiwi
Dec 3 '18 at 13:54

1

@tkiwi Because the business logic is idempotent, you actually don't generate new events. Say you issue two commands to change a customer's name to "Bob". The first time it gets processed, the name is changed and an event is raised, CustomerNameChanged("Bob"). The second time it is processed, the name is already what it should be and so you don't raise an event. The same applies if a single command is processed twice for some reason.

– CPerson
Dec 6 '18 at 15:53

What if I issuing a command to change a customer's name to Bob CustomerNameChanged("Bob"), triggers another event based on my business logic, e.g. there's a check whether Bob has friends, which in turn, emits NotifyFriendsForNameModification(...). Other example, e.g. there's a limit on daily name changes, so we can't allow any further changes for some time and we emit an event NameModificationDisabled("Bob"). Hoping that my examples are making sense, when NotifyFriendsForNameModification is emitted let's say the service crashes so we retry. Could an event be emitted twice like that?

– tkiwi
Dec 20 '18 at 12:00

I think I understand your question. CustomerNameChanged triggers a process or saga which then sends a command, NotifyFriendsForNameModification back to the Customer. This would raise FriendsNotifiedForNameModification event containing a list of friends who should be notified. A process or saga would take that list and generate a ModifyFriendName command for each friend. Since commands are Idempotent, each friend will acknowledge the change once and only once, raising a FriendNameModified event. Idempotent commands are key here. Does that answer the question?

– CPerson
Dec 20 '18 at 19:09

Thanks. I have an additional question. I understand how the business logic should be idempotent. One problem that arises is that the emitted events will need to have new id's. If the id's are randomly generated however, which is the easiest solution, the emitted events won't be the same. How is this problem solved? Should we for example generate the id's based on a hashing algorithm, in an idempotent way as well?

– tkiwi
Dec 3 '18 at 13:54

@tkiwi Because the business logic is idempotent, you actually don't generate new events. Say you issue two commands to change a customer's name to "Bob". The first time it gets processed, the name is changed and an event is raised, CustomerNameChanged("Bob"). The second time it is processed, the name is already what it should be and so you don't raise an event. The same applies if a single command is processed twice for some reason.

– CPerson
Dec 6 '18 at 15:53

What if I issuing a command to change a customer's name to Bob CustomerNameChanged("Bob"), triggers another event based on my business logic, e.g. there's a check whether Bob has friends, which in turn, emits NotifyFriendsForNameModification(...). Other example, e.g. there's a limit on daily name changes, so we can't allow any further changes for some time and we emit an event NameModificationDisabled("Bob"). Hoping that my examples are making sense, when NotifyFriendsForNameModification is emitted let's say the service crashes so we retry. Could an event be emitted twice like that?

– tkiwi
Dec 20 '18 at 12:00

I think I understand your question. CustomerNameChanged triggers a process or saga which then sends a command, NotifyFriendsForNameModification back to the Customer. This would raise FriendsNotifiedForNameModification event containing a list of friends who should be notified. A process or saga would take that list and generate a ModifyFriendName command for each friend. Since commands are Idempotent, each friend will acknowledge the change once and only once, raising a FriendNameModified event. Idempotent commands are key here. Does that answer the question?

– CPerson
Dec 20 '18 at 19:09

add a comment |

draft saved

draft discarded

Thanks for contributing an answer to Stack Overflow!

Please be sure to answer the question. Provide details and share your research!

But avoid …

Asking for help, clarification, or responding to other answers.

Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Name

Required, but never shown

Name

Required, but never shown

This page is only for reference, If you need detailed information, please check here

Search This Blog

Ufyukyu