Share file between release stages in Azure DevOps (VSTS)
Big Picture: We are using Azure DevOps release process (so far we create steps in designer, not YAML pipelines). We release to 3 environments with 3 different databases. Part of the release is deploy database from DACPAC file. Since using SqlPackage.exe to publish directly to database is not very transparent (you don't see and review the actual SQL script), we wanted to do release in 2 stages:
- Create SQL script from DACPAC and review it
- After approval run app and db deploy from previously generated script.
Issue: How to share sql script file between stages and how to see it for approval. Stages can be triggered on different agent.
What I've tried:
- Publishing script as build artifact - this won't work cause to generate script I need to connect to database and connecting to it should not be part of build process, especially connecting to production database.
- Publish artifact to Azure Pipelines as release step - it's not allowed on release, only for builds
Publish - Publish artifact to file share - I'm not sure how this exactly work, documentation is not very well done. Moreover regular windows file sharing would be difficult to set in our infrastructure, I would rather avoid it.
Any other suggestions?
azure-devops azure-pipelines azure-pipelines-release-task asked Nov 14 '18 at 13:11
Mateusz MoskaMateusz Moska
41349
add a comment |
Big Picture: We are using Azure DevOps release process (so far we create steps in designer, not YAML pipelines). We release to 3 environments with 3 different databases. Part of the release is deploy database from DACPAC file. Since using SqlPackage.exe to publish directly to database is not very transparent (you don't see and review the actual SQL script), we wanted to do release in 2 stages:
- Create SQL script from DACPAC and review it
- After approval run app and db deploy from previously generated script.
Issue: How to share sql script file between stages and how to see it for approval. Stages can be triggered on different agent.
What I've tried:
- Publishing script as build artifact - this won't work cause to generate script I need to connect to database and connecting to it should not be part of build process, especially connecting to production database.
- Publish artifact to Azure Pipelines as release step - it's not allowed on release, only for builds
Publish - Publish artifact to file share - I'm not sure how this exactly work, documentation is not very well done. Moreover regular windows file sharing would be difficult to set in our infrastructure, I would rather avoid it.
Any other suggestions?
azure-devops azure-pipelines azure-pipelines-release-task asked Nov 14 '18 at 13:11
Mateusz MoskaMateusz Moska
41349
add a comment |
Big Picture: We are using Azure DevOps release process (so far we create steps in designer, not YAML pipelines). We release to 3 environments with 3 different databases. Part of the release is deploy database from DACPAC file. Since using SqlPackage.exe to publish directly to database is not very transparent (you don't see and review the actual SQL script), we wanted to do release in 2 stages:
- Create SQL script from DACPAC and review it
- After approval run app and db deploy from previously generated script.
Issue: How to share sql script file between stages and how to see it for approval. Stages can be triggered on different agent.
What I've tried:
- Publishing script as build artifact - this won't work cause to generate script I need to connect to database and connecting to it should not be part of build process, especially connecting to production database.
- Publish artifact to Azure Pipelines as release step - it's not allowed on release, only for builds
Publish - Publish artifact to file share - I'm not sure how this exactly work, documentation is not very well done. Moreover regular windows file sharing would be difficult to set in our infrastructure, I would rather avoid it.
Any other suggestions?
azure-devops azure-pipelines azure-pipelines-release-task asked Nov 14 '18 at 13:11
Mateusz MoskaMateusz Moska
41349
Big Picture: We are using Azure DevOps release process (so far we create steps in designer, not YAML pipelines). We release to 3 environments with 3 different databases. Part of the release is deploy database from DACPAC file. Since using SqlPackage.exe to publish directly to database is not very transparent (you don't see and review the actual SQL script), we wanted to do release in 2 stages:
- Create SQL script from DACPAC and review it
- After approval run app and db deploy from previously generated script.
Issue: How to share sql script file between stages and how to see it for approval. Stages can be triggered on different agent.
What I've tried:
- Publishing script as build artifact - this won't work cause to generate script I need to connect to database and connecting to it should not be part of build process, especially connecting to production database.
- Publish artifact to Azure Pipelines as release step - it's not allowed on release, only for builds
Publish - Publish artifact to file share - I'm not sure how this exactly work, documentation is not very well done. Moreover regular windows file sharing would be difficult to set in our infrastructure, I would rather avoid it.
Any other suggestions?
azure-devops azure-pipelines azure-pipelines-release-task azure-devops azure-pipelines azure-pipelines-release-task asked Nov 14 '18 at 13:11
Mateusz MoskaMateusz Moska
41349
asked Nov 14 '18 at 13:11
Mateusz MoskaMateusz Moska
41349
asked Nov 14 '18 at 13:11
Mateusz MoskaMateusz Moska
41349
asked Nov 14 '18 at 13:11
Mateusz MoskaMateusz Moska
41349
asked Nov 14 '18 at 13:11
Mateusz MoskaMateusz Moska
41349
41349
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
While you can't use pipeline artefacts, you could use Universal Packages in Package Management to publish arbitrary files for later retrieval. Just dumping the file contents to the pipeline logs is the easiest way to allow people to inspect it.
You could also create a file with placeholders as a build artefact and merge in the final settings from Pipeline variables in each stage, that way you can keep them as a build artefact. That's what I tend to do for any files of this nature. Sounds like this won't apply for your generated SQL file.
Alternatively, if the "seeing for approval" piece is important you could generate it and write it to the log, upload it to Universal Package Management. Then ask for approval at the end of the stage. In the next stage, you then download the script from Universal Package Management or you regenerate it using the exact same task configuration before execution.
answered Jan 27 at 16:12
jessehouwingjessehouwing
67.8k10163237
1
I'm told that future versions of the "Publish Pipeline Artefact" is likely going to enable the scenario we all want.
– jessehouwing
Jan 28 at 10:50
Thanks for the idea, we finally did workaround with sending script to SFTP server, where we can access and review it and then in second stage we downloading the script and running.But ideal would be to publish artefact
– Mateusz Moska
Jan 29 at 10:51
1
Yeah Universal Package Management would allow a similar thing. Do you check the sha of the script after retrieving it from SFTP? Otherwise, you may end up consuming a different script than generated. Security seems important in your scenario ;).
– jessehouwing
Jan 29 at 11:22
Again, thanks for the idea :) you're right we should do check SHA sum, at the moment we are not doing it.
– Mateusz Moska
Jan 29 at 11:41
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53301052%2fshare-file-between-release-stages-in-azure-devops-vsts%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
While you can't use pipeline artefacts, you could use Universal Packages in Package Management to publish arbitrary files for later retrieval. Just dumping the file contents to the pipeline logs is the easiest way to allow people to inspect it.
You could also create a file with placeholders as a build artefact and merge in the final settings from Pipeline variables in each stage, that way you can keep them as a build artefact. That's what I tend to do for any files of this nature. Sounds like this won't apply for your generated SQL file.
Alternatively, if the "seeing for approval" piece is important you could generate it and write it to the log, upload it to Universal Package Management. Then ask for approval at the end of the stage. In the next stage, you then download the script from Universal Package Management or you regenerate it using the exact same task configuration before execution.
answered Jan 27 at 16:12
jessehouwingjessehouwing
67.8k10163237
1
I'm told that future versions of the "Publish Pipeline Artefact" is likely going to enable the scenario we all want.
– jessehouwing
Jan 28 at 10:50
Thanks for the idea, we finally did workaround with sending script to SFTP server, where we can access and review it and then in second stage we downloading the script and running.But ideal would be to publish artefact
– Mateusz Moska
Jan 29 at 10:51
1
Yeah Universal Package Management would allow a similar thing. Do you check the sha of the script after retrieving it from SFTP? Otherwise, you may end up consuming a different script than generated. Security seems important in your scenario ;).
– jessehouwing
Jan 29 at 11:22
Again, thanks for the idea :) you're right we should do check SHA sum, at the moment we are not doing it.
– Mateusz Moska
Jan 29 at 11:41
add a comment |
While you can't use pipeline artefacts, you could use Universal Packages in Package Management to publish arbitrary files for later retrieval. Just dumping the file contents to the pipeline logs is the easiest way to allow people to inspect it.
You could also create a file with placeholders as a build artefact and merge in the final settings from Pipeline variables in each stage, that way you can keep them as a build artefact. That's what I tend to do for any files of this nature. Sounds like this won't apply for your generated SQL file.
Alternatively, if the "seeing for approval" piece is important you could generate it and write it to the log, upload it to Universal Package Management. Then ask for approval at the end of the stage. In the next stage, you then download the script from Universal Package Management or you regenerate it using the exact same task configuration before execution.
answered Jan 27 at 16:12
jessehouwingjessehouwing
67.8k10163237
1
I'm told that future versions of the "Publish Pipeline Artefact" is likely going to enable the scenario we all want.
– jessehouwing
Jan 28 at 10:50
Thanks for the idea, we finally did workaround with sending script to SFTP server, where we can access and review it and then in second stage we downloading the script and running.But ideal would be to publish artefact
– Mateusz Moska
Jan 29 at 10:51
1
Yeah Universal Package Management would allow a similar thing. Do you check the sha of the script after retrieving it from SFTP? Otherwise, you may end up consuming a different script than generated. Security seems important in your scenario ;).
– jessehouwing
Jan 29 at 11:22
Again, thanks for the idea :) you're right we should do check SHA sum, at the moment we are not doing it.
– Mateusz Moska
Jan 29 at 11:41
add a comment |
While you can't use pipeline artefacts, you could use Universal Packages in Package Management to publish arbitrary files for later retrieval. Just dumping the file contents to the pipeline logs is the easiest way to allow people to inspect it.
You could also create a file with placeholders as a build artefact and merge in the final settings from Pipeline variables in each stage, that way you can keep them as a build artefact. That's what I tend to do for any files of this nature. Sounds like this won't apply for your generated SQL file.
Alternatively, if the "seeing for approval" piece is important you could generate it and write it to the log, upload it to Universal Package Management. Then ask for approval at the end of the stage. In the next stage, you then download the script from Universal Package Management or you regenerate it using the exact same task configuration before execution.
answered Jan 27 at 16:12
jessehouwingjessehouwing
67.8k10163237
While you can't use pipeline artefacts, you could use Universal Packages in Package Management to publish arbitrary files for later retrieval. Just dumping the file contents to the pipeline logs is the easiest way to allow people to inspect it.
You could also create a file with placeholders as a build artefact and merge in the final settings from Pipeline variables in each stage, that way you can keep them as a build artefact. That's what I tend to do for any files of this nature. Sounds like this won't apply for your generated SQL file.
Alternatively, if the "seeing for approval" piece is important you could generate it and write it to the log, upload it to Universal Package Management. Then ask for approval at the end of the stage. In the next stage, you then download the script from Universal Package Management or you regenerate it using the exact same task configuration before execution.
answered Jan 27 at 16:12
jessehouwingjessehouwing
67.8k10163237
answered Jan 27 at 16:12
jessehouwingjessehouwing
67.8k10163237
answered Jan 27 at 16:12
jessehouwingjessehouwing
67.8k10163237
answered Jan 27 at 16:12
jessehouwingjessehouwing
67.8k10163237
67.8k10163237
1
I'm told that future versions of the "Publish Pipeline Artefact" is likely going to enable the scenario we all want.
– jessehouwing
Jan 28 at 10:50
Thanks for the idea, we finally did workaround with sending script to SFTP server, where we can access and review it and then in second stage we downloading the script and running.But ideal would be to publish artefact
– Mateusz Moska
Jan 29 at 10:51
1
Yeah Universal Package Management would allow a similar thing. Do you check the sha of the script after retrieving it from SFTP? Otherwise, you may end up consuming a different script than generated. Security seems important in your scenario ;).
– jessehouwing
Jan 29 at 11:22
Again, thanks for the idea :) you're right we should do check SHA sum, at the moment we are not doing it.
– Mateusz Moska
Jan 29 at 11:41
add a comment |
1
I'm told that future versions of the "Publish Pipeline Artefact" is likely going to enable the scenario we all want.
– jessehouwing
Jan 28 at 10:50
Thanks for the idea, we finally did workaround with sending script to SFTP server, where we can access and review it and then in second stage we downloading the script and running.But ideal would be to publish artefact
– Mateusz Moska
Jan 29 at 10:51
1
Yeah Universal Package Management would allow a similar thing. Do you check the sha of the script after retrieving it from SFTP? Otherwise, you may end up consuming a different script than generated. Security seems important in your scenario ;).
– jessehouwing
Jan 29 at 11:22
Again, thanks for the idea :) you're right we should do check SHA sum, at the moment we are not doing it.
– Mateusz Moska
Jan 29 at 11:41
1
1
I'm told that future versions of the "Publish Pipeline Artefact" is likely going to enable the scenario we all want.
– jessehouwing
Jan 28 at 10:50
I'm told that future versions of the "Publish Pipeline Artefact" is likely going to enable the scenario we all want.
– jessehouwing
Jan 28 at 10:50
Thanks for the idea, we finally did workaround with sending script to SFTP server, where we can access and review it and then in second stage we downloading the script and running.But ideal would be to publish artefact
– Mateusz Moska
Jan 29 at 10:51
Thanks for the idea, we finally did workaround with sending script to SFTP server, where we can access and review it and then in second stage we downloading the script and running.But ideal would be to publish artefact
– Mateusz Moska
Jan 29 at 10:51
1
1
Yeah Universal Package Management would allow a similar thing. Do you check the sha of the script after retrieving it from SFTP? Otherwise, you may end up consuming a different script than generated. Security seems important in your scenario ;).
– jessehouwing
Jan 29 at 11:22
Yeah Universal Package Management would allow a similar thing. Do you check the sha of the script after retrieving it from SFTP? Otherwise, you may end up consuming a different script than generated. Security seems important in your scenario ;).
– jessehouwing
Jan 29 at 11:22
Again, thanks for the idea :) you're right we should do check SHA sum, at the moment we are not doing it.
– Mateusz Moska
Jan 29 at 11:41
Again, thanks for the idea :) you're right we should do check SHA sum, at the moment we are not doing it.
– Mateusz Moska
Jan 29 at 11:41
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53301052%2fshare-file-between-release-stages-in-azure-devops-vsts%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
