How to best use Azure Devops release pipelines with microservices?

I'm migrating a microservices ci/cd pipeline from a teamcity+octopus setup to Azure Devops.

We currently have:

Multiple repos for each service and web site.

A teamcity ci build which triggers an octopus deployment for each build.

An overnight scheduled set of integration tests run to test the whole platform and if they succeed then all the dev services are promoted to our nightly environment.

A manually triggered promotion of all services from nightly->prod.

I'm trying to do something similar in azure devops but that concept of promoting many services/components together seems difficult and a bit ugly using the gates and some custom search for open bugs to prevent the deployment.

Is anyone able to suggest what is best practice for what I'm trying to achieve? Should I have a single release pipeline that pulls in all artifacts from all the repos?

edited Jan 3 at 12:47

SteveWilkinson

1,08311518

asked Jan 2 at 13:21

Rob

14326

add a comment |

I'm migrating a microservices ci/cd pipeline from a teamcity+octopus setup to Azure Devops.

We currently have:

Multiple repos for each service and web site.

A teamcity ci build which triggers an octopus deployment for each build.

An overnight scheduled set of integration tests run to test the whole platform and if they succeed then all the dev services are promoted to our nightly environment.

A manually triggered promotion of all services from nightly->prod.

Is anyone able to suggest what is best practice for what I'm trying to achieve? Should I have a single release pipeline that pulls in all artifacts from all the repos?

edited Jan 3 at 12:47

SteveWilkinson

1,08311518

asked Jan 2 at 13:21

Rob

14326

add a comment |

I'm migrating a microservices ci/cd pipeline from a teamcity+octopus setup to Azure Devops.

We currently have:

Multiple repos for each service and web site.

A teamcity ci build which triggers an octopus deployment for each build.

An overnight scheduled set of integration tests run to test the whole platform and if they succeed then all the dev services are promoted to our nightly environment.

A manually triggered promotion of all services from nightly->prod.

Is anyone able to suggest what is best practice for what I'm trying to achieve? Should I have a single release pipeline that pulls in all artifacts from all the repos?

edited Jan 3 at 12:47

SteveWilkinson

1,08311518

asked Jan 2 at 13:21

Rob

14326

I'm migrating a microservices ci/cd pipeline from a teamcity+octopus setup to Azure Devops.

We currently have:

Multiple repos for each service and web site.

A teamcity ci build which triggers an octopus deployment for each build.

An overnight scheduled set of integration tests run to test the whole platform and if they succeed then all the dev services are promoted to our nightly environment.

A manually triggered promotion of all services from nightly->prod.

Is anyone able to suggest what is best practice for what I'm trying to achieve? Should I have a single release pipeline that pulls in all artifacts from all the repos?

azure-devops

edited Jan 3 at 12:47

SteveWilkinson

1,08311518

asked Jan 2 at 13:21

Rob

14326

edited Jan 3 at 12:47

SteveWilkinson

1,08311518

asked Jan 2 at 13:21

Rob

14326

edited Jan 3 at 12:47

SteveWilkinson

1,08311518

edited Jan 3 at 12:47

SteveWilkinson

1,08311518

edited Jan 3 at 12:47

SteveWilkinson

1,08311518

asked Jan 2 at 13:21

Rob

14326

asked Jan 2 at 13:21

Rob

14326

asked Jan 2 at 13:21

Rob

14326

add a comment |

1 Answer
1

active

oldest

votes

Detailed blog post explaining the why's here

tldr…

Having a repo per micro service/releaseable unit/web front end makes sense.

Each repo/micro service/releaseable unit/web front end has a CI build that compiles, runs extensive unit tests and creates your build artifacts. If any of those unit tests fail, the build fails.

You can have a release pipeline for each CI Build as these pipelines can be used to deploy just an individual micro service if you want.

For this scenario, you can then have a release pipeline that uses the build artifacts from each CI build. It will use the latest successful CI Build for all the repos. This release pipeline will have 2 (or more if you need them) stages. A nightly/testing stage and a production stage. The nightly/testing stage grabs all the latest successful artifacts and deploys all the services. At the end of the deployment, runs your integration tests. And if those passes, it goes to the next stage (prod) where you have a manual approver who will approve the release to production.

So here’s where things get interesting. If you wanted to follow the same type of workflow as Rob is currently using, you can have this all-encompassing release be triggered on a schedule. In his workflow, it looks like this trigger is scheduled once a night. That does limit your release to at most 1 per 24 hours (or as many times as you schedule your trigger and yes, you can trigger more manually too). This seems a bit restrictive. Now, my pipeline has a bottleneck based on my scheduled trigger. If you wanted to, you can have the release be triggered on changes to the build artifacts! So anytime there is a successful build for any of the CI Builds for each individual repo, this will trigger the release pipeline. Which will deploy to nightly/testing. Run integration tests, and if everything looks good, gets passed to the production stage.

answered Jan 4 at 0:33

Abel Wang

561

add a comment |

Your Answer

StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});

}
});

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54007148%2fhow-to-best-use-azure-devops-release-pipelines-with-microservices%23new-answer', 'question_page');
}
);

Post as a guest

Name

Required, but never shown

1 Answer
1

active

oldest

votes

1 Answer
1

active

oldest

votes

Detailed blog post explaining the why's here

tldr…

Having a repo per micro service/releaseable unit/web front end makes sense.

Each repo/micro service/releaseable unit/web front end has a CI build that compiles, runs extensive unit tests and creates your build artifacts. If any of those unit tests fail, the build fails.

You can have a release pipeline for each CI Build as these pipelines can be used to deploy just an individual micro service if you want.

For this scenario, you can then have a release pipeline that uses the build artifacts from each CI build. It will use the latest successful CI Build for all the repos. This release pipeline will have 2 (or more if you need them) stages. A nightly/testing stage and a production stage. The nightly/testing stage grabs all the latest successful artifacts and deploys all the services. At the end of the deployment, runs your integration tests. And if those passes, it goes to the next stage (prod) where you have a manual approver who will approve the release to production.

So here’s where things get interesting. If you wanted to follow the same type of workflow as Rob is currently using, you can have this all-encompassing release be triggered on a schedule. In his workflow, it looks like this trigger is scheduled once a night. That does limit your release to at most 1 per 24 hours (or as many times as you schedule your trigger and yes, you can trigger more manually too). This seems a bit restrictive. Now, my pipeline has a bottleneck based on my scheduled trigger. If you wanted to, you can have the release be triggered on changes to the build artifacts! So anytime there is a successful build for any of the CI Builds for each individual repo, this will trigger the release pipeline. Which will deploy to nightly/testing. Run integration tests, and if everything looks good, gets passed to the production stage.

answered Jan 4 at 0:33

Abel Wang

561

add a comment |

Detailed blog post explaining the why's here

tldr…

Having a repo per micro service/releaseable unit/web front end makes sense.

Each repo/micro service/releaseable unit/web front end has a CI build that compiles, runs extensive unit tests and creates your build artifacts. If any of those unit tests fail, the build fails.

You can have a release pipeline for each CI Build as these pipelines can be used to deploy just an individual micro service if you want.

For this scenario, you can then have a release pipeline that uses the build artifacts from each CI build. It will use the latest successful CI Build for all the repos. This release pipeline will have 2 (or more if you need them) stages. A nightly/testing stage and a production stage. The nightly/testing stage grabs all the latest successful artifacts and deploys all the services. At the end of the deployment, runs your integration tests. And if those passes, it goes to the next stage (prod) where you have a manual approver who will approve the release to production.

So here’s where things get interesting. If you wanted to follow the same type of workflow as Rob is currently using, you can have this all-encompassing release be triggered on a schedule. In his workflow, it looks like this trigger is scheduled once a night. That does limit your release to at most 1 per 24 hours (or as many times as you schedule your trigger and yes, you can trigger more manually too). This seems a bit restrictive. Now, my pipeline has a bottleneck based on my scheduled trigger. If you wanted to, you can have the release be triggered on changes to the build artifacts! So anytime there is a successful build for any of the CI Builds for each individual repo, this will trigger the release pipeline. Which will deploy to nightly/testing. Run integration tests, and if everything looks good, gets passed to the production stage.

answered Jan 4 at 0:33

Abel Wang

561

add a comment |

Detailed blog post explaining the why's here

tldr…

Having a repo per micro service/releaseable unit/web front end makes sense.

Each repo/micro service/releaseable unit/web front end has a CI build that compiles, runs extensive unit tests and creates your build artifacts. If any of those unit tests fail, the build fails.

You can have a release pipeline for each CI Build as these pipelines can be used to deploy just an individual micro service if you want.

For this scenario, you can then have a release pipeline that uses the build artifacts from each CI build. It will use the latest successful CI Build for all the repos. This release pipeline will have 2 (or more if you need them) stages. A nightly/testing stage and a production stage. The nightly/testing stage grabs all the latest successful artifacts and deploys all the services. At the end of the deployment, runs your integration tests. And if those passes, it goes to the next stage (prod) where you have a manual approver who will approve the release to production.

So here’s where things get interesting. If you wanted to follow the same type of workflow as Rob is currently using, you can have this all-encompassing release be triggered on a schedule. In his workflow, it looks like this trigger is scheduled once a night. That does limit your release to at most 1 per 24 hours (or as many times as you schedule your trigger and yes, you can trigger more manually too). This seems a bit restrictive. Now, my pipeline has a bottleneck based on my scheduled trigger. If you wanted to, you can have the release be triggered on changes to the build artifacts! So anytime there is a successful build for any of the CI Builds for each individual repo, this will trigger the release pipeline. Which will deploy to nightly/testing. Run integration tests, and if everything looks good, gets passed to the production stage.

answered Jan 4 at 0:33

Abel Wang

561

Detailed blog post explaining the why's here

tldr…

Having a repo per micro service/releaseable unit/web front end makes sense.

Each repo/micro service/releaseable unit/web front end has a CI build that compiles, runs extensive unit tests and creates your build artifacts. If any of those unit tests fail, the build fails.

You can have a release pipeline for each CI Build as these pipelines can be used to deploy just an individual micro service if you want.

For this scenario, you can then have a release pipeline that uses the build artifacts from each CI build. It will use the latest successful CI Build for all the repos. This release pipeline will have 2 (or more if you need them) stages. A nightly/testing stage and a production stage. The nightly/testing stage grabs all the latest successful artifacts and deploys all the services. At the end of the deployment, runs your integration tests. And if those passes, it goes to the next stage (prod) where you have a manual approver who will approve the release to production.

So here’s where things get interesting. If you wanted to follow the same type of workflow as Rob is currently using, you can have this all-encompassing release be triggered on a schedule. In his workflow, it looks like this trigger is scheduled once a night. That does limit your release to at most 1 per 24 hours (or as many times as you schedule your trigger and yes, you can trigger more manually too). This seems a bit restrictive. Now, my pipeline has a bottleneck based on my scheduled trigger. If you wanted to, you can have the release be triggered on changes to the build artifacts! So anytime there is a successful build for any of the CI Builds for each individual repo, this will trigger the release pipeline. Which will deploy to nightly/testing. Run integration tests, and if everything looks good, gets passed to the production stage.

answered Jan 4 at 0:33

Abel Wang

561

answered Jan 4 at 0:33

Abel Wang

561

answered Jan 4 at 0:33

Abel Wang

561

answered Jan 4 at 0:33

Abel Wang

561

add a comment |

draft saved

draft discarded

Thanks for contributing an answer to Stack Overflow!

Please be sure to answer the question. Provide details and share your research!

But avoid …

Asking for help, clarification, or responding to other answers.

Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Name

Required, but never shown

Name

Required, but never shown

This page is only for reference, If you need detailed information, please check here

Search This Blog

Ufyukyu